Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)

Started by g3gg0, July 15, 2013, 10:58:23 PM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

tjaja

Have you tried to remove the MLV's already on the card larry77?
And buffer fill method 0-3 gives me better results, but I don't know if that is card dependant.


After two hours of testing these two resolutions and frame rate got stuck in my head.

3584x1194 15fps fills up with 112MB per seconde.
1920x1080 30fps fills up around 98MB per seconde.

As in data rate these two should be equal, but clearly a larger image written less often after each other is easier on the cards to write.
Going up with the resolution while going down with the frame rate show that behavior.
Does that have to do with random vs sequantial writing?

In Buffer method 0 where it both cards start immediatly with recording the larger images it even produce a steady line on the graph
5DFree, 550D, canon 50mm 1.4

larry 777

Removing MLV's no problem, I'll try buffer fill method 0-3 , but what about FPS overide
it wasn't a problem with raw.rec. ?

larry 777

I guess it has to do with sound, for the rest , the nightly build seems very stable
and a dream to shoot with, thanks for the genius programming .

chmee

@g3gg0
started with mlv-support in raw2cdng, nearly done. but:

* (5DIII, feb/dontknowversion) examplerecording.
* in [MLVI] is stated videoFrameCount=215, but just found 212 RAWI [VIDF]-Blocks.
* for understanding audiopart - its realized simply writing RIFF-Header according to [WAVI]-Data and concatenating [AUDF]-Chunks.. right?

regards chmee
[size=2]phreekz * blog * twitter[/size]

g3gg0

@all:
sorry for the delays, quite busy with job and family atm :)
maybe you can draw a conclusion - as far i understand some modes suffer from the last changes and some others are much more stable now.
is that correct?

@chmee:
RAWI blocks? i think you mean VIDFs. thhey should match of course.
which recording do you mean? want to check that.
i would not rely on the information in the header. if recording stops premaurely you have zero there.
this is a hint, just for comfortable previews.

you have to build RIFF header on your own - just the wav information has the same format.
then simply concatenate all audio frames (in order), right.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

chmee

thx. jepp. mean [VIDF]. will upload the mlv when i'm back home, 2 days. (finally, its working. next days i will look on stripped mlv's, because as in raw the parted frames making me some headache in terms of "beautiful code")

regards chmee
[size=2]phreekz * blog * twitter[/size]

noix222

Quote from: g3gg0 on March 04, 2014, 09:37:00 AM
@all:
sorry for the delays, quite busy with job and family atm :)
maybe you can draw a conclusion - as far i understand some modes suffer from the last changes and some others are much more stable now.
is that correct?


Testing new builds its evident that the speed is improving... the only strange thing to me still is that the speed recording in 1x is not as stable recording in 5x... as i said befor i am able to record full hd 30fps continuous in 5x mode, but i cant do in 1x mode... the buffer goes from around 103mb/s in 5x compared to max 98mb/s in 1x mode.

thanks guys

tjaja

What settings did you use with 1080p 30fps in 3xzoom Noix222?
5DFree, 550D, canon 50mm 1.4

noix222

Quote from: tjaja on March 04, 2014, 06:27:35 PM
What settings did you use with 1080p 30fps in 3xzoom Noix222?

i wrote my tests on page 61 with builds magiclantern-v2.3.NEXT.2014Feb26.5D3113 & magiclantern-v2.3.NEXT.2014Mar02.5D3113 - got better results with the second.

5d mark 3 WITH a CF Komputerbay 64gb 1000x and SD Sandisk Extreme 45mb/s.

MLV_REC settings: status recording Debug _ memory hack ON _ extra hacks ON
                             buffer fill method 4 _ cf-only buffers 3 _ card spanning ON _ reserve card space ON

marekk

Is there a possibility to refresh free space field after each shot ? Now if think we need to restart camera to refresh it.

budafilms

Quote from: noix222 on March 04, 2014, 06:19:00 PM

Testing new builds its evident that the speed is improving... the only strange thing to me still is that the speed recording in 1x is not as stable recording in 5x... as i said befor i am able to record full hd 30fps continuous in 5x mode, but i cant do in 1x mode... the buffer goes from around 103mb/s in 5x compared to max 98mb/s in 1x mode.

thanks guys

Very strange. In which resolution you tried? 1920x1080?

Ben Tubby

Hi,

I've been shooting some test footage using mlv_rec on a Canon Mk3 113 firmware. The build I am running is the 07/03/2014.

I've noticed a problem with the MLV files and I couldn't find anyone else in the forum that seemed to be experiencing similar issues.

When I change a camera setting during recording (ISO or aperture) it seems to create a problem when it comes to converting to DNG using all of the current MLV conversion tools on both mac and windows.

It kind of 'trips up' the converting program and causes it to stop making DNG's from the rest of the MLV at the point that the ISO or aperture adjustments were made.

Using MlRawViewer on mac I was able to skip past the point in the clip where the camera settings changed, but extracting a full usable take is not possible.

If I do not adjust any camera settings during recording I can shoot any length of clip my card allows and convert to DNG with no problems.

I have tested many times and it is definitely the change in camera settings that causes this. Does anyone else have this problem?

I hope this feedback helps the very impressive ongoing development of the MLV format.

Ben

andy kh

 is there any way to get rid of pink dots on 650D/700D? i know there is an application to remove pink dots but thats not the best solution so is there any other better solution? is magic lantern working on this? or this pink dots wil remain forever if so i m gonna sell my 650D and get another camera.
5D Mark III - 70D

chmee

inside mlv_dump exists chroma smoothing, IF you convert directly into dng. see first entry in this thread.

-- DNG output --
--dng               output frames into separate .dng files. set prefix with -o
--no-cs             no chroma smoothing
--cs2x2             2x2 chroma smoothing
--cs3x3             3x3 chroma smoothing
--cs5x5             5x5 chroma smoothing


btw. g3gg0 a1ex and 1%
did you try to recalculate the raw-values?


regards chmee
[size=2]phreekz * blog * twitter[/size]

escho

Recalculation looks for me as a great idea. And doing this outsite of chroma-smoothing would make the cold pixel fix not be needed anymore, I guess? Rest the answer of the speed. Is this recalculation fast enough for  to can work with it?

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

a1ex

It won't replace the cold pixel fix; it will simply make these pixels harder to identify.

Other than fixing compatibility issues with Premiere (any others?) I don't see the point. You'll introduce some quantization errors, you'll have a harder time if you want to denoise or subtract dark frames (because you will lose the Gaussian distribution of shadow noise), and that's pretty much it.

The Nikon guys not exactly happy with this behavior in their cameras (for example http://forum.orpington-astronomy.org.uk/index.php?topic=8917.0 )

chmee

@escho no it doesnt help with dead/coldpixels.. and yes, its fast enough. in loop theres only one multiplication and one subtraction per value.

but: most of the users won't anything other than direct working inside premiere?! it could be just a commandline-option.. by now raw2cdng is the only one is doin' that, and the guys are happy.. thats the point. linux and mac users are desperated.. all GUIs would draw on it.

and to be honest: quantization errors are the least problem.. (not to say academic)
[size=2]phreekz * blog * twitter[/size]

a1ex

Yes, an option makes sense. Maybe even an option for linear DNG output, already demosaiced (in this case I'd go for AMaZE, for which we already have the source code).

noix222

hello guys... just tested again same settings 1920x1080 30fps with a 5dmark3 using build magiclantern-v2.3.NEXT.2014Mar08.5D3113

Again at 1x mode CF buffer dropped a lot staying on a average of 78-75mb/s.. sometimes lower out of CF and a constant 17mb/s from SD. But at 3x mode i was able to continuous record 30fps on average buffer 102-103mb/s total, with around 85mb/s out of CF and 17mb/s out of SD. SD speeds its constant recording on both modes.

the workflow is pretty easy and i had no errors on my footage converting mlv files using MLVMystic http://secondsparkproductions.com/index.php/blog/8-mlv-mystic-mlv-raw-to-dng-batch-converter

i also tested and really liked MlRawViewer http://www.magiclantern.fm/forum/index.php?topic=9560.msg91165#msg91165 it previews the file and you can easily export a ProRes422 (HQ) just pressing a rec button.

i wish i could help on anything more than just testing but i have no idea how to deal with all this codes..... ! but thanks everyone involved.....

using raw_rec the difference on buffer is lower i get constant 99mb/s  1x mode and a max of 101mb/s on 3x mode.

escho

Quote from: chmee on March 07, 2014, 10:39:30 PM
@escho no it doesnt help with dead/coldpixels..

Yes, Alex und you, you are right. Looks like my thoughts was a bit crazy yesterday :)

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed


escho

Thank You for the link. This descripes a nice method, which can be used on some cameras, to map out bad pixels inside the camera. But this cannot work on ML-Raw-video. The newer versions of raw2dng have their own cold-pixel fix, enabled per default. And in the newer versions of mlv_dunp you can enable the cold-pixel-fix of raw2dng for MLV-clips using the commandline-option --fixcp.

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

chmee

i'm not sure if the remapping is in between - kind of "everytime-remap-array-before-the-real-sensor-output". tests will show..
[size=2]phreekz * blog * twitter[/size]

dubzeebass


hyalinejim

Quote from: Ben Tubby on March 07, 2014, 02:38:48 PM

I've been shooting some test footage using mlv_rec on a Canon Mk3 113 firmware. The build I am running is the 07/03/2014. I've noticed a problem with the MLV files and I couldn't find anyone else in the forum that seemed to be experiencing similar issues.

When I change a camera setting during recording (ISO or aperture) it seems to create a problem when it comes to converting to DNG using all of the current MLV conversion tools on both mac and windows. It kind of 'trips up' the converting program and causes it to stop making DNG's from the rest of the MLV at the point that the ISO or aperture adjustments were made.

Using MlRawViewer on mac I was able to skip past the point in the clip where the camera settings changed, but extracting a full usable take is not possible. If I do not adjust any camera settings during recording I can shoot any length of clip my card allows and convert to DNG with no problems. I have tested many times and it is definitely the change in camera settings that causes this. Does anyone else have this problem?


I can confirm this issue. I thought I was going crazy! If you change the shutter speed, ISO or WB during mlv recording then the resulting file will be corrupt. You can test for this by playing back in camera where playback freezes on the frame where the settings were changed, giving the message:

frame and block size mismatch: 0x375F00 0x23A9AB 0x376EB4
(for example)

None of the conversion applications can convert these files. Here's an error message from MLVBrowseSharp:



I did notice that this issue doesn't exist in the Feb 23rd build, but does appear from Feb 24th onwards. Hope that is helpful. Keep up the good work!