Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)

Started by g3gg0, July 15, 2013, 10:58:23 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

DeafEyeJedi

For those who are curious or in need of files (Dual-ISO RAW, MLV, CR2, MLV/RAW, FRSP DNG/MLV) from certain models such as 70D, 5D3 & the M... Perhaps to help speed up the process on getting the binaries up to date.

Or just want to play with certain files for testing/practice purposes.

https://mega.co.nz/#F!DokxSIhC!q5iOd_rIQTRbCXA51Kyz5Q

Happy Updates!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

tidelander

Is it possible to specify output directory (not output file) for mlv_dump.exe?
I have gigabytes of mlv files (silent picture time lapse) that I want to export to DNG and then to Lightroom. Currently I first copy files from CF card to hard drive, then export to DNG and then to Lightroom.
It would be nice if I could point mlv_dump.exe to mlv file on CF card and specify output directory for DNG files on my hard drive.
However, I was not able to find such feature in documentation. Does it exist?
I tried something like this:

mlv_dump --dng outdir in.mlv

But it does not work and complains "Error: Failed to open file outdir"

g3gg0

mlv_dump --dng -o /path/to/dir/fileprefix_  in.mlv
it will automatically append the frame number
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Danne

I,m really curious about this. According to below user he claims white balance is correctly translated from windows version of mlv_dump(not sure which version)
user mothaibaphoto
Quote...mlv_dump produces dng sequence with correct WB from camera.
http://www.magiclantern.fm/forum/index.php?topic=13152.msg149992#msg149992

When I run mlv_dump for mac(latest 2 may upload) dng files still have hard coded as shot neutral values?
Could this windows claim be confirmed by somebody?
Thanks

*Update
Correction here regarding mlv_dump white balance
http://www.magiclantern.fm/forum/index.php?topic=13152.msg150044#msg150044

Tao

Hi, I've done a quick search but I didn't find any answer, I started shooting MLV with a 5D3 (I used raw on 70D or 5D2) : is it normal than I have vertical lines when shooting ? Can I remove them ? It's quite disturbing but I don't see the in MLrawviewer.

Also, is MLV good for something like 48fps@maxrespossible-videos ? or should I go back to RAW just for that ? (I understand RAW had better writing speeds, I have 2 Lexar 64go 1066x)

Thanks to you all
5D Mark III w/ Sigma Art 50mm f/1.4 + Sigma Art 24mm f/1.4 + Canon L 70-200 f/4.0

Markus

The vertical lines are known issue. Some programs have a remove stripes function to remove them. I have suspected this issue occurs when you film raw in non native in between iso's. I haven't had this problem since only using "true" iso's.
100, 200, 400, 800, 1600 and so on. I set the camera iso setting's in canon menu so I can only use these when shooting raw.
Haven't confirmed this theory yet though.
Note you can also get vertical lines if filming at very high iso's...

Greg

Averaging in MLV_dump in combination with FRSP is a great feature.

4 frames = 1EV more into the shadows.
16 frames = 2EV more into the shadows.
With this we do not need HDR (bracketing), we get smooth movement of water or clouds.

I would like to reduce banding/FPN (vertical stripes).
I wonder how to fix it, maybe averaging function should median optical black (OB) area?

dmilligan

Quote from: Greg on August 12, 2015, 05:08:36 PM
I would like to reduce banding/FPN (vertical stripes).
I wonder how to fix it, maybe averaging function should median optical black (OB) area?
Capture a 'dark frame' (put lens cap on). Then subtract that .mlv from your real .mlv (there's an option to do this in mlv_dump, I think it's -s).

Greg

mlv_dump -o o.mlv -s dark.MLV light.MLV

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'light.MLV'
   - Rewrite MLV
   - Subtract reference frame 'dark.MLV' from single images
   - Output into 'o.mlv'
File light.MLV opened
File light.M00 not existing.
[ERROR] File 'dark.MLV' ends in the middle of a block
[ERROR] Failed to load subtract frame (4)

:o

g3gg0

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Greg

The original file is 3.6GB
I should create a new single frame, or averaged this?

In mlv_dump error will be the same.

g3gg0

the new single frame.
can you run the mlv_dump run that produces the single frame with parameter '-v' and upload the log?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

g3gg0

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

g3gg0

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Greg

WOW vertical stripes gone, it works.

The dark frame subtract hot pixels (works well in the shadows).
In highlights create a cold pixels.

I think the ideal solution is to average/median vertical lines in the dark frame.

g3gg0

redownload

--avg-vertical      [DARKFRAME ONLY] average the resulting frame in vertical direction, so we will extract vertical banding
--avg-horizontal    [DARKFRAME ONLY] average the resulting frame in horizontal direction, so we will extract horizontal banding

and please test by adding one of them along with option -a :)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

mesebar2

Hello everybody, and thank you for your amazing works!!!

I was wondering if it was possible to re-create a MLV video from its DNG files.
(A MLV i shot is not working properly, but i have the whole DNG sequence and would love to be able to create a new MLV out of it.)

Thank you in advance :)

Walter Schulz

So you have DNG and want to go back to MLV to create DNG?

mesebar2

Yes, that is the idea :)
For having 1 single file and playing with Mlrawviewer. When and if needed i can create DNGs again.

Danne

I thought of this to. Could be a handy container for mlvfs. Could even open up other dng relate stuff from other cameras.

Markus

To be able to do this with compressed DNGs would be ideal. The compressed size but in mlv container = dream come true! ;-)

dmilligan

Quote from: Markus on December 20, 2015, 05:45:55 PM
The compressed size but in mlv container = dream come true! ;-)

The dream is already true see the OP:
Quote
mlv_dump
- used for debugging and converting .mlv files
- can dump .mlv to legacy .raw + .wav files
- can dump .mlv to .dng  + .wav
- can compress and decompress frames using LZMA
- convert bit depth (any depth in range from 1 to 16 bits)

compression:
you can get a data reduction of ~60% with 12 bit files.
downconverting to 8 bits gives you about 90% data reduction.
this feature is for archiving your footage.

LZMA is kinda slow and doesn't compress as well, but LJ92 is superfast:
https://bitbucket.org/hudson/magic-lantern/branch/mlv_rec_lj92

I don't really know why this hasn't been merged, but I have implemented it for MLVFS.
So you can use this to compress stuff:
https://bitbucket.org/dmilligan/magic-lantern/branch/ml_dng_lj92 or mac binary
and MLVFS will decompress it on the fly (in realtime, in if you are limited by HDD speed, using this compression can actually make MLVFS faster).

MLVFS will also decompress LZMA compressed MLV files, but it's excruciatingly slow. MLVFS does not support the frame diff option however (MLVFS needs to provide "random access" to frames).

jmanord

Awesome! I may be misunderstanding the intended workflow, but is the attached mac binary used on an MLV file straight from the camera to compress it using the aforementioned LJ92? I received an immediate notification that the compression is complete and the resulting file is zero bytes. I extracted mlv_dump and tried running it from the command line, but received [ERROR] Missing file header. I tried using the normal mlv_dump.osx, the one included in the OP, and it seemed to work fine. Am I using your version on an incorrect target file? Thank you!

Full output:

MacDesktops-iMac:Test MLV Folder macdesktop$ ./mlv_dump -c -v -o test.MLV M30-0950.MLV

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'M30-0950.MLV'
   - Verbose messages
   - Rewrite MLV
   - Compress frame data
   - Output into 'test.MLV'
File M30-0950.MLV opened
File M30-0950.M00 not existing.
File M30-0950.IDX opened (XREF)
XREF table contains 1484 entries
Processing...
[ERROR] Missing file header
Processed 0 video frames
[ERROR] Failed to rewrite header in .MLV file
Done

flostro

Quote from: dmilligan on January 11, 2016, 06:15:54 PM
The dream is already true see the OP:
LZMA is kinda slow and doesn't compress as well, but LJ92 is superfast


Does this mean it is possible to use the LJ92 compression for mlv files? With mlv dump?

g3gg0

dmilligan can you make a pull request that i can merge?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!