raw2mlv processing

[Danne]

Ilia3101 is building a raw2mlv conversion tool. Very cool. Now we are actually able to wrap CR2 files and dng files back to our beloved MLV format. Well, more raw formats are possible but for now I focused on CR2 and dng files.

This automator app has only one purpose. To utilize Ilia3101 raw2mlv conversion tool, code here:
https://github.com/ilia3101/LibMLV

Forum post:
https://www.magiclantern.fm/forum/index.php?topic=24386.0
https://github.com/ilia3101/LibMLV


raw2mlv.app(Mac only!)
https://bitbucket.org/Dannephoto/raw2mlv/src/master/
Instructions and source code

Download:
https://bitbucket.org/Dannephoto/raw2mlv/downloads/raw2mlv.dmg

regarding gatekeeper
To supress gatekeeper hold ctrl button down(macOS Sierrra, Mojave, Catalina) while opening the application the first time. You can also change permissions from within privacy/security settings.



Good for? You tell me but I use it as follows:
1 - Transcode dualiso CR2 files into MLV files accessible in Mlv App
2 - Wrap timelapse CR2 file sequences into one chained MLV file accessible in Mlv App
3 - Wrap dng files back into MLV format

TODO:
Adding more options
Adding more raw format options
Add folder support. (Drag n´ drop only atm)

[ilia3101]

Yay!

[DeafEyeJedi]

Oh yeah baby!!! I friggin' love the second option (02) for getting all CR2's into a single MLV file.

This is nuts especially for time-lapse stuff! 

[IDA_ML]

If an option could be added to MLVApp to open single CR2 files, including DualISO ones, I think, I would say Goodbye to ACR for ever.  I like the results from MLVApp much better and for me such an option would be revolutionary! 

Genious work, Ilia3101 and Danne!

[Kharak]

Waiting anxiously for a Windows version

[cmh]

raw2mlv.exe (Windows x64)

Usage
There's two ways to create an mlv from your raw files:
- drop the files on raw2mlv.exe and you'll find an output.mlv in the folder where your files are located (this method might be limited to a maximum of 433 files).
- copy the files in a directory containing raw2mlv, open a powershell windows, navigate to you folder (or open one from the current folder using shift + right clic) and use the command line.

 
Example:
.\raw2mlv.exe "E:\DCIM\100CANON\IMG_168*.CR2" -o "C:\Users\SomeDude\Desktop\test.mlv"
This command will open all CR2 files starting by IMG_168 from the folder E:\DCIM\100CANON\ and create an mlv file on SomeDude's desktop.
Open this mlv in MLVApp and each frames will be made of the CR2s to be manipulated and saved.

Here's a quick demo.

Download
https://github.com/cmhamiche/raw2mlv/raw/master/raw2mlv.zip

Note
This is not a port of Danne's app but a placeholder for ilia3101's Windows build.

[Danne]

Cool. Feel free to maintain a windows version in above post.

[ilia3101]

@cmh Thank yuou wow!!! I knew it would compile on Windows.

Really wishing I had Windows (rare occasion) right now, I wanna test this.

I hope to create a makefile that willl work everywhere.

[Walter Schulz]

You can. Install an Windows 10 evaluation version inside a virtual machine.
https://www.microsoft.com/en-us/evalcenter/evaluate-windows-10-enterprise

[ilia3101]

I will use that thankjs.

[cmh]

If I'm being honest I'm not a dev and I barely understand what I'm doing. I'm just familiar with some unix systems.
I've yet to understand how to link dynamic library to get rid of some more dlls.
Adapting compile.sh to add Windows should be a trivial task if you check the build instructions I posted.

[ilia3101]

The instructions are very nice and useful. Thank you.

I agree it would be nice to remove some of the DLLs,  I will see what I can do

[DeafEyeJedi]

If an option could be added to MLVApp to open single CR2 files, including DualISO ones, I think, I would say Goodbye to ACR for ever.

It's bound to happen. Bet Adobe is about to shit a brick, if not already and yet once this option arise it would indeed be revolutionary! 

[Danne]

Really cool progress on your windows post @cmh. Thanks for that.

I updated with possibility to alter bitdepth, already included in raw2mlv. First post updated:

[2blackbar]

LOVE WINDOWS VERSION ! Thank You guys a lot, i can now edit cr2 files in single mlv

[garry23]

To help those of us who are not up to speed on mlv and 'video stuff', I wonder if some kind person could capture the workflow to achieve the following use case to simulate an  LE image.

Capture a sequence of .cr2s, say, 20.

Bring these together into an mlv file using the new tool.

Process the mlv in some app to merge the individual files together.

A1ex suggested the above in a previous post, but truly I have no idea about the video tools that have been created for mlv.

Cheers

Garry

[Danne]

Usage:
There's two ways to create an mlv from your raw files
- open a powershell from raw2mlv's folder (shift + right clic) and use the command line (raw2mlv.exe --help to get the arguments).
- drop your files on raw2mlv.exe and you'll find an output.mlv in the folder where your files are located.

Example:
.\raw2mlv.exe "E:\DCIM\100CANON\IMG_168*.CR2" -o "C:\Users\SomeDude\Desktop\test.mlv"
This command will open all CR2 files starting by IMG_168 from the folder E:\DCIM\100CANON\ and create an mlv file on SomeDude's desktop.
Open this mlv in MLVApp and each frames will be made of the CR2s to be manipulated and saved.

Above doesn't work for you @garry23?
Sidenote: funny thing I asked for explaining vids of your lua script once but not really happening right ?

[cmh]

Here's a quick demo on Windows.

Don't mind the quality (uploading it took longer than creating it) but it should do the job.
Keep in mind that the raw2mlv I made available is just Ilia3101's I compiled for Windows and not a port of Danne's fancier OSX tool.

Edit: even tho I haven't tried yet, there's ways to change the framerate either in raw2mlv or MLVApp.
That said, other than stop motion animations, I'd love to see a timelapse smoothed with Resolve's optical flow.

[Kharak]

When I have more than 430 dng's in a folder, the executable fails.
windows gives me this message: "The filename or extension is too long".

I have a bunch of folders with DNG's that I want converted to MLV, one folder I have has 430 dng's and it works, but another with 495 wont convert and I get that message. Folders with less than 430 convert fine and ones with more dont.

[cmh]

I can't help much on that.
Have you tried moving those files to a shorter path like C:\DNG.
If if doesn't help (and probably won't because I suspect that a long ass concatenated string is created) you could also try to remove the 260 character path limit by editing the registry.
I haven't try myself but if you feel adventurous and made backups, here's the procedure:
- run regedit.exe as administrator
- locate [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
- change data value [LongPathsEnabled] (DWORD) to "1"
- close regedit and restart Windows

Could you report if you make it work somehow?

[ilia3101]

I seem to remember that windows needs to use special versions of the file reading/writing functions for files over 4GB, do you think file size could be the problem?

[Lars Steenhoff]

What about the lens information, can this also be transferred form the raw to the mlv?

[cmh]

I don't have the necessary knowledge to answer that question Ilia3101, I've never written any code and I've no idea how fwrite is implemented in Visual C++.
I'm talking out of my ass but while I see some fwrite in raw2mlv.c, I can't find any buffer meaning that while it's cached, I guess it's relying on the operating system flushing (or disk controller whatever). I think it's good practice to define a buffer and split things up.

That said you might need some more info from Kharak, maybe he's working on a fat32 card or something.
Meanwhile I'll create 200 DNG to test this out...

Edit: so I've created 210 CR2s, made a 6,5gb mlv that contains the 210 frames and it's working fine. I'll try with DNGs...
Working fine with DNG using a long path. I'll try 500 files...

[ilia3101]

@Lars steenhoff I will add lens info soon, just haven't done it yet. As well as quite a lot of things.

Don't worry cmh, we did this in MLV App before and I can make the change later, it's not big. Although as it seems to work for you maybe it is completely fine.

Maybe Karak is using FAT32.

Can you test if this runs on your PC: https://drive.google.com/open?id=1SzEU5xDILbE6xMxWMTBUQ3b-TWkWDjA4

I have tried to compile with microsoft msvc and it works for me. Only one DLL needed for this: libraw

[Danne]

I seem to remember that windows needs to use special versions of the file reading/writing functions for files over 4GB, do you think file size could be the problem?

On mac I think it could be around 5000-6000 files before terminal chokes. Easy workaround using find xarg command but let´s see if mac gets some broken workflow first before developing further.