MLV Lite

[Walter Schulz]

Covered in first post.

[dfort]

sorry i got a noob question - other than the early bugs for now, are there other disadvantages to MLV Lite files vs full RAW files?

What do you mean by "full" RAW files?

Differences from "full" MLV:

1. Metadata is only for what the settings were when recording was started (e.g. you get expo metadata, but if you change exposure, you don't get that new information).
2. No audio
3. No card spanning (recording to both CF and SD card)

Differences from RAW v1.0 is that some of the testers are finding that the older format still has a slight edge in performance when you're really pushing the envelope at the expense of almost no metadata to work with in post production.

In practice the difference are more significant. MLV is a big improvement over v1.0 RAW in terms of the amount of metadata saved in the files that can be useful in post production and even more significant, being able to record synced audio. However, there are issues with "pink frames" and other problems that have come up on occasion so some users have reverted to the old RAW format. MLV Lite is designed to ween users off of the old RAW format by giving "The sweet taste of MLV with none of the extra calories" -- makes sense?

@dmilligan -- hope you approve of my editorializing.

@WalterSchulz -- Your response was much more concise but I wanted to add my own thoughts about MLV Lite. Seems like my tests have been favoring the old format.

[Licaon_Kter]

EOS M, auto UI, warmup 64Mb

mlv_rec = 1728x... records >1000frames
mlv_lite = frameskip after 2 secs

mlv_rec = 1280x... records continously
mlv_lite = frameskip after 30 secs

[Danne]

There has been some posts about corrupted frames with mlv lite in MLP thread which when looking further into it was a user issue because of a mix up filming with regular MLV not mlv lite. No corrupted frames with MLV lite yet that is. Good news.
Here,s the faulty description.
http://www.magiclantern.fm/forum/index.php?topic=13512.msg163300#msg163300

[Danne]

Played around with a 7D at work. Downloaded magiclantern-Nightly.2016Feb29.7D203.zip and installed.
Ran filming in 60 mv720 mode fps around 1500 x 540 in size(not exact numbers). Actually produced a lot of corrupt frames when using global draw and focus peaking. Global draw off yields no corrupted frames, everything works ok. Surprisingly I had the same corrupt image problems with old format RAW when doing the same tests so this test doesn,t say anything more than that they are both corrupted. Here is a shortened mlv_lite file if of any use.
https://drive.google.com/file/d/0B4tCJMlOYfirZTR5SE1qUW9BZ28/view?usp=sharing

Conclusion. Both RAW and mlv lite can produce corrupt frames.

Gonna try some regular fps 24-25 fps footage as well. Update coming up.

*Regular HD(1728x972) 25fps with global draw on, focus peaking works fine with no corrupted frames with mlv lite.

[Steve_FR]

Eos-M Nightly 2/29/2016

5x Zoom Raw, Frame size: 1728x736 aspect 2:35 @ 23.97 fps (fps override enabled)
MLV 2.0            - 175 to 176 frames (7 seconds)
MLV Lite            - 150 to 160 frames (listed as "zero frames")
ML Raw 1.0    - 223 to 229 frames (9 seconds)

[reddeercity]

Actually produced a lot of corrupt frames when using global draw and focus peaking. Global draw off yields no corrupted frames, everything works ok.

Yes that been noted before with the 7D & 5D2 back a few years ago now.
I used to get corrupt frames when I use GD enabled (5D2)  , it's just to much for system to handle  at least at max. resolution .
I guess that why on the 5d2 the GD automatically is disable when raw video starts in MLV2.0 only thou.

[Danne]

That is important information. Harder to test the problem.

[dfort]

@dmilligan

Just a quick question. Would it be possible to implement something like LZ4 or FastLZ before the data hits the memory card? It seems that the bottle neck is in the writing speed, especially for cameras using SD cards. Only a tiny bit of compression would be needed to exceed the performance of RAW 1.0.

I have a feeling you're going to say that the ARM processor isn't powerful enough but thought I'd ask anyway.

[dmilligan]

Even memcpy (straight memory copy using the cpu) is too slow... The only way raw video works is with specialized external memory controller that can copy huge chunks of memory around very quickly with no CPU overhead.

[johannsebastianbach]

Hey fellas Is there any good way to preview MLV-Lite in realtime, like it works for normal MLV's with MlRawViewer? I'm not very up to date - neither is my 2008'Macbook

[dmilligan]

Talking about on a computer? Does MlRawViewer not work with these files?

[johannsebastianbach]

MLV Preview module is working fine on the camera. My Mac is too slow to preview in after effects (not to mention resolve) but was always previewing in realtime in MlRawViewer. Unfortunately MlRawViewer shows only a black screen if I try a MLV Lite.

Good news is: I just had a weekend shooting 120GB of MLV Lite and not a single pink frame with 1920x490 @ 50fps Global Draw ON with cropmarks and freakin' FOCUS PEAK  I'm so astonished, it was nice to be able to focus pull easily in slow motion which was impossible with normal MLV without getting pink frames.

My workflow normally was: previewing the MLV's with MlRawViewer and pick the ones I want to open with MLVFS for further processing (MLFVS is the Reason I like MLV instead of RAW, for space saving workflow without having to export to cDNG first). I'm using 5DIII OSX 10.11.3 MlRawViewer 1.4.3.

[DeafEyeJedi]

I second that as well ... MLV Lite files do not work with MLRV but with AE or DR it all seems fine like normal. I suspect it has to do with the heading of the file. Not that big of a deal but it would be nice if there was a fix for this small bug.

[johannsebastianbach]

Little problem over here, one of the 174 mlv´s lite  I´ve shot is previewing fine in OSX Finder, but when I open it in CameraRAW to debay its black. Tried MLVFS and MLP. Any fix for that? Thank ya´ll in advance.

EDIT: 5dIII, OSX11, ML1.1.3, 50fps, MLV Lite, 1920x490

[Walter Schulz]

70D?

[johannsebastianbach]

70D?

Sorry, forgot: 5dIII, OSX11, ML1.1.3, 50fps, MLV Lite, 1920x490

[Danne]

Corrupt frame? Do you have the actual mlv file or a sample dng?

[johannsebastianbach]

Corrupt frame? Do you have the actual mlv file or a sample dng?

It is one mlv of 174 recorded (didn't change any settings in the recording series) and all of the cDNG's in this perticular mlv. Finder preview works fine.

Here to download, thanks (MLFVS same as MLP): https://dl.dropboxusercontent.com/u/24873307/M13-1536_000470.dng

[Danne]

Thanks. Seems corrupted. Preview seems fine though. Only one bad fram out of 174 mlv files shot in 50fps? That seems to me like mlv_lite is a trustworthy format.

[johannsebastianbach]

Thanks. Seems corrupted. Preview seems fine though. Only one bad fram out of 174 mlv files shot in 50fps? That seems to me like mlv_lite is a trustworthy format.

It's all frames in the whole MLV. But one MLV out of 174. So not a corrupted frame - since the preview works fine I guess there is a way somehow to fix it.

Otherwise though mlv lite is extremly stable. I didn't have one corrupt frame (only this MLV) even with using focus peak in 50fps! The one and only reason left for me not to use it is the realtime previewing option. It doesn't work in MlRawViewer and now I had to export proxies which tooke about 50 hours to render just to preview and be able to choose the actual footage i am going to use for the cut - you cannot decently preview on the set and know only after days if you fucked something up maybe

[Danne]

50 hours. A lot of footage. Did you run it through MLP?
By the way. You can run MlRawViewer by selecting a dng file inside the dng folder. That means you can run MlRawViewer straight through MLVFS with Mlv-lite footage without the need to spend 50 hours on proxies  .

[johannsebastianbach]

That is an truly awesome advice! Didn't know MlRawViewer can open up a cDNG sequence. Thank you Danne! 

It's not a lot of footage (120gb of mlv), but just a slower macbook - 2 GHz Intel Core 2 Duo, 8 GB 1067 MHz DDR3, NVIDIA GeForce 9400M 256 MB. It is still usable for all work though, even cutting with proxies (CameraRaw workflow).

EDIT: About the corrupt MLV. MlRawViewer shows it black too, it's really weird, that OSX Preview shows it correctly.

[Danne]

Not really. The preview pic is only a preview. The actual raw data could therefore be corrupt while previewe shows correctly.

[dmilligan]

There is no separate preview data in an MLV file. If a preview somehow looks correct then the original raw data is intact, b/c that preview had to have come from the real original raw image data in the MLV file. I highly doubt there is image data corruption, something else is wrong, probably incorrect black level => Finder uses a hard coded back level and ignores the value in the file, so if the value in the file is wrong, preview will still show correctly.

Also, most cDNG converters (MLVFS included) don't embed a separate preview image in the DNGs (b/c Adobe Premiere and others don't like it), so the Finder is likely showing the full image data.