Magic Lantern Forum

Using Magic Lantern => Post-processing Workflow => Topic started by: MarS on September 15, 2016, 07:08:35 PM

Title: MLVToMovie: MLV file converter for Mac OS X
Post by: MarS on September 15, 2016, 07:08:35 PM
Let me introduce my own little software available today for the community...  :)
http://mlvtomov.eliotis.com

(http://mlvtomov.eliotis.com/files/mlvtomovie-screenchsot.png)

Specifications:
- Native Mac OS X application written in Objective-C.
- Queue management to process as many files as you want..
- MLV files can be added to the queue from different directories.
- Movie files are build in the same directory as their original MLV files.
- DNG extraction with MLV_Dump.
- RGB conversion with DCRaw (parallelized for better performances).
- Movie conversion with FFmpeg.
- Dark and flat calibration with automatic averaging of MLV files.
- Lossless export AVI (codec rawvideo; yuv420p) and MOV (codec h264; qp=1; yuv420p).
- Preview export AVI (codec mpeg4; qscale=1; yuv420p) and MOV (h264; qp=17; yuv420p).
- Sound export if available (PCM S16 LE for AVI file; AAC for MOV file).
- Prevent computer sleep when working.
NB: MOV quality is not perfectly lossless (qp parameter should be set to 0 and not 1) to be readable by Quicktime player.

Software requirements:
- Mac OS 10.9 or higher.
- 64 bit processor (Intel Core 2 Duo or higher).

My first goal was to produce AVI lossless videos to process astronomical images with Registax, AutoStackkert, or AviStack. RAW videos and 3x crop allow to produce very nice high resolution images with telescopes and Canon cameras...
(http://astronomy.eliotis.com/tests/jupiter-5dmk3-magic-latern.jpg)

(http://astronomy.eliotis.com/gallery/lune-25-08-16-cn-212-red10-ml3x-w1024-v3.jpg)
Full size: http://astronomy.eliotis.com/gallery/lune-26-08-16-cn-212-red10-ml3x-100perc-v3.jpg (http://astronomy.eliotis.com/gallery/lune-26-08-16-cn-212-red10-ml3x-100perc-v3.jpg)

I'm not a video master at all but if you like MLVToMovie and need some improvements (according to MLV_Dump, DCRaw and FFmpeg capabilities) let me know. I'll do my best to help the community.

Wish it helps,
Sebastien.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: reddeercity on September 15, 2016, 08:18:00 PM
Thanks @MarS for the MLVtoMovie app , Just downloaded  I'll give it a test drive to night . 
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: squig on September 16, 2016, 01:39:05 AM
Cool, thx.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: ddelreal on September 16, 2016, 01:59:18 AM
Very cool, thank you for this.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: reddeercity on September 16, 2016, 02:00:06 AM
Just doing some tests and i see you don't have dual ISO Video supported , just ran a 5d2 dual iso file though.
I notice after rendering a .mov file it's H264 @ 180Mb/s so 1 min @1856x928 23.976p = 1.44 GB which can not be played in
QT player 10 or QT player 7 freezes after about 4-5 seconds then player blows up .
I'm on Yosemite 10.10.5 MacPro 4.1 dual quad core(16 threads) 24GB Ram, OSX on SSD
Why not have ProRes as a option a high bite rate h264 is really not very useful IMO as lest for editing plus going from 14bit to 8bit.
At least with FFmpeg ProRes you are only compressing to 10Bit . My suggestion would be to give the user the option to use ProRes , H264 , maybe even Tiff and or DPX image sequence . Also LUT support would be nice so some of use could export a LOG ready Video file/image sequence  (prores , etc....) . Also Maybe a optional export folder other then the source e.g. a different drive , i didn't know where the file was being exported until it was done , if you have 500 GB of raw to convert you will full you Drive . If source file are on the os drive , besides that it with speed up conversion exporting to a different drive even a USB drive.

I'll give the AVI a try and report back.

Edit: tried the AVI Why 8bit 4.2.0 ? give very bad flicker with dual iso ( yes i understand there not dual iso support)



Complete name                       : C:\Users\i5Dell\Desktop\M13-2146.avi
Format                                   : AVI
Format/Info                              : Audio Video Interleave
Format profile                           : OpenDML
File size                                : 3.71 GiB
Duration                                 : 1mn 4s
Overall bit rate                         : 495 Mbps
Writing application                      : Lavf57.25.100

Video
ID                                       : 0
Format                                   : YUV
Codec ID                                 : I420
Codec ID/Info                            : 8 bit Y plane followed by 8 bit 2x2 subsampled U and V planes.
Duration                                 : 1mn 4s
Bit rate                                 : 496 Mbps
Width                                    : 1 856 pixels
Height                                   : 928 pixels
Display aspect ratio                     : 2.000
Frame rate                               : 23.976 fps
Compression mode                         : Lossless
Bits/(Pixel*Frame)                       : 12.002
Stream size                              : 3.70 GiB (100%)

Audio
ID                                       : 1
Format                                   : PCM
Format settings, Endianness              : Little
Format settings, Sign                    : Signed
Codec ID                                 : 1
Duration                                 : 1mn 4s
Bit rate mode                            : Constant
Bit rate                                 : 1 536 Kbps
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 KHz
Bit depth                                : 16 bits
Stream size                              : 11.8 MiB (0%)
Alignment                                : Aligned on interleaves
Interleave, duration                     : 21 ms (0.51 video frame)


had to read this info off my PC Laptop.

Nice program but the export compression choices are very limited , plus no color space management . So my suggestion are
A: better export choices e.g. (FFmeg ProRes 4444) = to 10bit RGBA , Thou Apple ProRes would be better as it would be 12Bit.
B: Color Mangement , LUT support (to me thou i think this would be a bit of work)

So at least A: would make this a very good choice to export a quick file to ed

:)
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Lars Steenhoff on September 17, 2016, 03:23:24 PM
Very nice too see this program, I hope you have time to add more features as the potential is really awesome to simplify the conversion from mlv to quicktime.

Nice to have are:

LUT
CINELOG
Debayering to 4k
Output folder
Drag new source clips in the queue window
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Lars Steenhoff on September 17, 2016, 08:49:11 PM
Andy600 should be able to give insights about LUTS
http://www.magiclantern.fm/forum/index.php?action=profile;u=2412 (http://www.magiclantern.fm/forum/index.php?action=profile;u=2412)
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: pholler on September 18, 2016, 12:39:10 PM
Hi Sebastien!

I really like your converter. It works great on my Mac.

I personally have two issues with it. I cannot change the temp-directory where all the conversions happen. My SSD is limited so i can only convert small .mlv-files. It would be cool so set the directory in the preferences.
It would be very cool to drag-and-drop .mlv-files from finder into the sources-list.

Greetings
Peter
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: MarS on September 18, 2016, 10:26:50 PM
Thank you guys. :)

reddeercity, yes my first need was to render lossless for astronomical process so It's big and QT have synchronization issues when displaying. Very bad flicker with DualIso? With lossles it should be at least beautiful flicker! ;) I joke of course. Dual Iso is not supported yet but I'll try in a next release.

According to your feedbacks, I've spent a little bit of time on release 1.1.0 today so here it is:
-Drag & drop support of MLV files and directories (recursive scan) into the sources-list.
-Drag & drop support of MLV files and directories (recursive scan) on app icon.
-Allow directories selection with open file panel too -> selected directories are automatically scanned recursively for MLV files.
-Temporary directory can be edited in preference panel.
This release is already available on my website.

Now I'll work on news features soon. Here is my road map:
- a preset editor for FFmpeg exports so we should be able to export other movie formats.
- more DCRaw options (color space, saturation, brightness, gamma curve, Linear mode, white balance, 8 or 16 bits export for RGB conversion).
- DualIso support.

Few questions (I'm not a video master at all as I said):
- What's LUT and CINELOG?
- Lars Steenhoff, what do you mean by "Debayering to 4k"? Is there a Canon camera able to produce 4K with Magic Lantern yet?
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Kharak on September 18, 2016, 10:35:38 PM
Cinelog is a product which MarS can not implement as that would break the copyright.

You don't wanna start another RAWmagic thing here.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Andy600 on September 19, 2016, 12:24:42 AM
Re: Cinelog - the colorspace is not really a copyright issue but DCRaw will be the limiting factor due to the available demosaicing algorithms and highlight recovery options i.e. it will introduce colored artifacts to highlights at the settings required to reproduce highlight detail equivalent to ACR.

Dcraw highlight recovery is more like a highlight contrast effect. It's not simple linear scaling that could be undone later. The rebuilt highlights are compressed and so don't maintain the relationship to linear light. I'm currently working on a DCRaw mod for another app that improves things (a bit) and I've already added several other gamuts but it's opened up a big can of worms relating to DCRaw's choice of D65 for it's XYZ white point and there are a few other odd things happening to the color- but that's another topic.

But basically, if you want to add Cinelog-C then I have no problem with it. It's Cineon log (in lut form it would be data in, legal out) with Alexa Wide gamut primaries.

out=(300*LOG(in+27/2473)+685*LOG(10))/(1023*LOG(10))

        x        y
R  0.6840  0.3130
G  0.2210  0.8480
B  0.0861 -0.1020

W 0.3127 0.3290 (D65)


Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Danne on September 19, 2016, 11:21:29 AM
dcraw linear default could be set to -H 2 -b 2(or 3) to match acr output better. It,s not perfect linear equivalent but close. If -H 2 setting could be refined in code somehow this could probably be tweaked further to match. 
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Lars Steenhoff on September 20, 2016, 01:46:37 AM
Yes the raw recording of the 5d mk3 is 1080 but what i always do in adobe camera raw is import at 200% so in effect sampling directly to

3840x2160   (4k)

this gives a better quality over upsampling to 4k later.

there are some good reasons for wanting to go directly to 4k.
but my main one is to get the highest quality out of the 1080 raw.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Lars Steenhoff on September 20, 2016, 01:57:36 AM
and LUT is a color lookup table.

https://www.macprovideo.com/hub/photoshop/photoshop-making-color-look-up-tables-luts (https://www.macprovideo.com/hub/photoshop/photoshop-making-color-look-up-tables-luts)

they can contain color corrections that can be aplied to the footage.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: khuong on September 20, 2016, 12:06:43 PM
Thanks for your contribution Sebastien,

But, when I change the temp folder in the preferences it erase everything in that folder.

I accidentally change the temp folder to my desktop and tried to convert an .mlv file however when i quit the application it erased everything in my Desktop :'(

Is there any way to get/restore it back ?

Thanks
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Lars Steenhoff on September 21, 2016, 08:02:35 PM
wow that sounds pretty serious, hope you data will be recovered
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: MarS on September 22, 2016, 04:25:15 AM
khuong, I'm so sorry. temp folder, as its name says, is a temporary folder. The application needs to delete it automatically because MLV file extraction uses a lot of disk space.

Temp files do not go into the user's trashcan. To restore data, you'll need your Time Machine backup or a recovery application like Mac Data Recovery.

I'll publish immediately a workaround to avoid user's manipulation error.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: MarS on September 22, 2016, 04:55:08 AM
Workaround published...
Quote
September 22, 2016 - Release 1.1.1:
Security update: adds a subdirectory MLVToMovieTmp when user want to use a custom temporary directory. It'll avoid bad surprises if user selects accidentally its Desktop directory or other personal directories. Cause temporary directory is automatically deleted, user could loose personal data. Now, only the MLVToMovieTmp directory will be deleted.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: khuong on September 27, 2016, 11:28:34 AM
It is unfortunate that I lost my data for good  :-\
Anyway, thanks MarS for the workaround.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: MarS on October 04, 2016, 02:03:43 PM
Version 1.2.0 available:
Quote
October 4, 2016: Release 1.2.0:
- Flat dark support added.
- Calibration diagram added for help.
- Ask user for flat, dark and flat dark overwrite after averaging frames to save disk space.
- Wrong frame number solved with splitted files (>4GB if exFAT not supported).

I've added the diagram (see "Vertical stripes revisited (5D Mark III)" topic) as help to understand image calibration concept...
(http://astronomy.eliotis.com/magic-lantern/image-calibration-help.jpg)

I'm now working on preset editor and HDR support.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Lars Steenhoff on October 04, 2016, 03:39:30 PM
really nice progress!  :)

looking forward to the presets
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: KavesCanon on November 01, 2016, 08:30:21 AM
Wow

This software is amazing, I'm a newb here and I have been battling to get a proper workflow going.

My problems are mainly that I have a late 2011 Macbook Davinci Resolve does not work. I have yet to get a proper workflow going and I think it solved my problem.

My only question is I get green spots on my probably over exposed places on film, how is that stopped?

Great work done so far!!!!

Thanks

K
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: alejandro rumolino on February 22, 2017, 01:16:26 PM
Sorry for my english, which is the best conversion = MOV or AVI
MLV of origin = 810 Mo passed to mov = 375 Mo
Passed to AVI = 975 Mo and converted via QT to mov = 35 Mo!
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: alejandro rumolino on April 15, 2017, 01:19:39 PM
- Magic Lantern raw, MLVtoMovie !!
Shot on the Canon 6D in RAW 1600x900 scale 1080p 16/9 - Lenses : Sigma 50mm Art, CANON EF 100 mm f/2.8 Macro USM, Canon 24mm f/4L - FCPX, LUT Utility : Osiris Canon C LOG + C LOG to Rec709
https://vimeo.com/212953671
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Eskandar95 on July 05, 2017, 12:09:11 AM
Hey guys,

I'm using this great converter but cannot convert my .M01...M099 files. Any tips?

Thanks for the info
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: johnnay on October 07, 2017, 12:13:40 AM
OMG, this is just what I need!! I'm a total noob in terms of photography/videography but like the technology and like being able to shoot continuously at higher resolution. However, I have been struggling with workflow - I tried a lot of the myriad tools for ML and ended up with switch and davinci resolve, which is WAAAAAy too complicated for me but I managed to make it spit out quicktime movies in the end, which is really all I need so I can string them together in imovie (I know, how amateur!). :D

Anyways, MLVtoMovie worked great, and is exactly what I have been looking for! Thank you so much! It worked flawlessly and quickly, but I do have a question:

When I was using the latest ML nightly (14 bit) everything was find, but when I used the 10/12 bit build, MLVtoMovie stopped working (switch/resolve still works on these files however). I'm assuming it's because I'm using the 10bit output (for higher continuous resolution on my T3I/600d)? Is it possible you can make it work for 10/12 bit output?

Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Danne on October 07, 2017, 09:33:35 AM
You could try exchange mlv_dump inside Resources to a later version. Perhaps a version from inside Switch.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: johnnay on October 09, 2017, 08:20:09 AM
Thanks, I tried that (via a simple copy/paste, is that what you meant?) - now after selecting the source folder, the app hangs with "reading data" showing for the MLV, and the 'Start' button is greyed out.
Title: Re: MLVToMovie: MLV file converter for Mac OS X
Post by: Danne on October 09, 2017, 09:27:19 AM
Try contact the author directly.