Magic Lantern Forum

Using Magic Lantern => Raw Video => Raw Video Postprocessing => Topic started by: Danne on November 10, 2019, 03:45:44 PM

Title: raw2mlv processing
Post by: Danne on November 10, 2019, 03:45:44 PM
Ilia3101 is building a raw2mlv conversion tool. Very cool. Now we are actually able to wrap CR2 files and dng files back to our beloved MLV format. Well, more raw formats are possible but for now I focused on CR2 and dng files.

This automator app has only one purpose. To utilize Ilia3101 raw2mlv conversion tool, code here:
https://github.com/ilia3101/LibMLV

Forum post:
https://www.magiclantern.fm/forum/index.php?topic=24386.0
https://github.com/ilia3101/LibMLV


raw2mlv.app(Mac only!)
https://bitbucket.org/Dannephoto/raw2mlv/src/master/
Instructions and source code

Download:
https://bitbucket.org/Dannephoto/raw2mlv/downloads/raw2mlv.dmg

regarding gatekeeper
To supress gatekeeper hold ctrl button down(macOS Sierrra, Mojave, Catalina) while opening the application the first time. You can also change permissions from within privacy/security settings.

(https://i.postimg.cc/Gmf270hw/Ska-rmavbild-2019-11-10-kl-15-30-07-png-scaled.png)

Good for? You tell me but I use it as follows:
1 - Transcode dualiso CR2 files into MLV files accessible in Mlv App
2 - Wrap timelapse CR2 file sequences into one chained MLV file accessible in Mlv App
3 - Wrap dng files back into MLV format

TODO:
Adding more options
Adding more raw format options
Add folder support. (Drag n´ drop only atm)

Title: Re: raw2mlv processing
Post by: ilia3101 on November 10, 2019, 07:23:41 PM
Yay!
Title: Re: raw2mlv processing
Post by: DeafEyeJedi on November 11, 2019, 03:27:11 AM
Oh yeah baby!!! I friggin' love the second option (02) for getting all CR2's into a single MLV file.

This is nuts especially for time-lapse stuff!  8)
Title: Re: raw2mlv processing
Post by: IDA_ML on November 11, 2019, 10:23:51 AM
If an option could be added to MLVApp to open single CR2 files, including DualISO ones, I think, I would say Goodbye to ACR for ever.  I like the results from MLVApp much better and for me such an option would be revolutionary! 

Genious work, Ilia3101 and Danne!
Title: Re: raw2mlv processing
Post by: Kharak on November 11, 2019, 04:00:49 PM
Waiting anxiously for a Windows version :)
Title: Re: raw2mlv processing
Post by: cmh on November 11, 2019, 04:36:01 PM
raw2mlv.exe (Windows x64)

Usage
There's two ways to create an mlv from your raw files:
- drop the files on raw2mlv.exe and you'll find an output.mlv in the folder where your files are located (this method might be limited to a maximum of 433 files).
- copy the files in a directory containing raw2mlv, open a powershell windows, navigate to you folder (or open one from the current folder using shift + right clic) and use the command line.


Example:
.\raw2mlv.exe "E:\DCIM\100CANON\IMG_168*.CR2" -o "C:\Users\SomeDude\Desktop\test.mlv"
This command will open all CR2 files starting by IMG_168 from the folder E:\DCIM\100CANON\ and create an mlv file on SomeDude's desktop.
Open this mlv in MLVApp and each frames will be made of the CR2s to be manipulated and saved.

Here's a quick demo.


Download
https://github.com/cmhamiche/raw2mlv/raw/master/raw2mlv.zip

Note
This is not a port of Danne's app but a placeholder for ilia3101's Windows build.
Title: Re: raw2mlv processing
Post by: Danne on November 11, 2019, 04:50:13 PM
Cool. Feel free to maintain a windows version in above post.
Title: Re: raw2mlv processing
Post by: ilia3101 on November 11, 2019, 06:58:16 PM
@cmh Thank yuou wow!!! I knew it would compile on Windows.

Really wishing I had Windows (rare occasion) right now, I wanna test this.

I hope to create a makefile that willl work everywhere.
Title: Re: raw2mlv processing
Post by: Walter Schulz on November 11, 2019, 07:14:05 PM
You can. Install an Windows 10 evaluation version inside a virtual machine.
https://www.microsoft.com/en-us/evalcenter/evaluate-windows-10-enterprise
Title: Re: raw2mlv processing
Post by: ilia3101 on November 11, 2019, 07:26:55 PM
I will use that thankjs.
Title: Re: raw2mlv processing
Post by: cmh on November 11, 2019, 07:29:22 PM
If I'm being honest I'm not a dev and I barely understand what I'm doing. I'm just familiar with some unix systems.
I've yet to understand how to link dynamic library to get rid of some more dlls.
Adapting compile.sh to add Windows should be a trivial task if you check the build instructions I posted.
Title: Re: raw2mlv processing
Post by: ilia3101 on November 11, 2019, 07:34:48 PM
The instructions are very nice and useful. Thank you.

I agree it would be nice to remove some of the DLLs,  I will see what I can do
Title: Re: raw2mlv processing
Post by: DeafEyeJedi on November 11, 2019, 09:13:47 PM
Quote from: IDA_ML on November 11, 2019, 10:23:51 AM
If an option could be added to MLVApp to open single CR2 files, including DualISO ones, I think, I would say Goodbye to ACR for ever.

It's bound to happen. Bet Adobe is about to shit a brick, if not already and yet once this option arise it would indeed be revolutionary!  8)
Title: Re: raw2mlv processing
Post by: Danne on November 11, 2019, 10:35:19 PM
Really cool progress on your windows post @cmh. Thanks for that.

I updated with possibility to alter bitdepth, already included in raw2mlv. First post updated:
(https://i.postimg.cc/Vvnf8rHm/Screenshot-2019-11-11-at-22-22-26-png-scaled.png)
Title: Re: raw2mlv processing
Post by: 2blackbar on November 12, 2019, 12:57:33 AM
LOVE WINDOWS VERSION ! Thank You guys a lot, i can now edit cr2 files in single mlv
Title: Re: raw2mlv processing
Post by: garry23 on November 12, 2019, 07:09:59 AM
To help those of us who are not up to speed on mlv and 'video stuff', I wonder if some kind person could capture the workflow to achieve the following use case to simulate an  LE image.

Capture a sequence of .cr2s, say, 20.

Bring these together into an mlv file using the new tool.

Process the mlv in some app to merge the individual files together.

A1ex suggested the above in a previous post, but truly I have no idea about the video tools that have been created for mlv.

Cheers

Garry
Title: Re: raw2mlv processing
Post by: Danne on November 12, 2019, 07:17:50 AM
Quote from: cmh on November 11, 2019, 04:36:01 PM
Usage:
There's two ways to create an mlv from your raw files
- open a powershell from raw2mlv's folder (shift + right clic) and use the command line (raw2mlv.exe --help to get the arguments).
- drop your files on raw2mlv.exe and you'll find an output.mlv in the folder where your files are located.

Example:
.\raw2mlv.exe "E:\DCIM\100CANON\IMG_168*.CR2" -o "C:\Users\SomeDude\Desktop\test.mlv"
This command will open all CR2 files starting by IMG_168 from the folder E:\DCIM\100CANON\ and create an mlv file on SomeDude's desktop.
Open this mlv in MLVApp and each frames will be made of the CR2s to be manipulated and saved.

Above doesn't work for you @garry23?
Sidenote: funny thing I asked for explaining vids of your lua script once but not really happening right ;)?
Title: Re: raw2mlv processing
Post by: cmh on November 12, 2019, 10:28:17 AM
Here's a quick demo on Windows.



Don't mind the quality (uploading it took longer than creating it) but it should do the job.
Keep in mind that the raw2mlv I made available is just Ilia3101's I compiled for Windows and not a port of Danne's fancier OSX tool.

Edit: even tho I haven't tried yet, there's ways to change the framerate either in raw2mlv or MLVApp.
That said, other than stop motion animations, I'd love to see a timelapse smoothed with Resolve's optical flow.
Title: Re: raw2mlv processing
Post by: Kharak on November 12, 2019, 01:12:53 PM
When I have more than 430 dng's in a folder, the executable fails.
windows gives me this message: "The filename or extension is too long".

I have a bunch of folders with DNG's that I want converted to MLV, one folder I have has 430 dng's and it works, but another with 495 wont convert and I get that message. Folders with less than 430 convert fine and ones with more dont.
Title: Re: raw2mlv processing
Post by: cmh on November 12, 2019, 01:26:55 PM
I can't help much on that.
Have you tried moving those files to a shorter path like C:\DNG.
If if doesn't help (and probably won't because I suspect that a long ass concatenated string is created) you could also try to remove the 260 character path limit by editing the registry.
I haven't try myself but if you feel adventurous and made backups, here's the procedure:
- run regedit.exe as administrator
- locate [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
- change data value [LongPathsEnabled] (DWORD) to "1"
- close regedit and restart Windows

Could you report if you make it work somehow?
Title: Re: raw2mlv processing
Post by: ilia3101 on November 12, 2019, 01:56:57 PM
I seem to remember that windows needs to use special versions of the file reading/writing functions for files over 4GB, do you think file size could be the problem?
Title: Re: raw2mlv processing
Post by: Lars Steenhoff on November 12, 2019, 02:56:21 PM
What about the lens information, can this also be transferred form the raw to the mlv?
Title: Re: raw2mlv processing
Post by: cmh on November 12, 2019, 03:04:51 PM
I don't have the necessary knowledge to answer that question Ilia3101, I've never written any code and I've no idea how fwrite is implemented in Visual C++.
I'm talking out of my ass but while I see some fwrite in raw2mlv.c, I can't find any buffer meaning that while it's cached, I guess it's relying on the operating system flushing (or disk controller whatever). I think it's good practice to define a buffer and split things up.

That said you might need some more info from Kharak, maybe he's working on a fat32 card or something.
Meanwhile I'll create 200 DNG to test this out...

Edit: so I've created 210 CR2s, made a 6,5gb mlv that contains the 210 frames and it's working fine. I'll try with DNGs...
Working fine with DNG using a long path. I'll try 500 files...
Title: Re: raw2mlv processing
Post by: ilia3101 on November 12, 2019, 03:30:42 PM
@Lars steenhoff I will add lens info soon, just haven't done it yet. As well as quite a lot of things.

Don't worry cmh, we did this in MLV App before and I can make the change later, it's not big. Although as it seems to work for you maybe it is completely fine.

Maybe Karak is using FAT32.

Can you test if this runs on your PC: https://drive.google.com/open?id=1SzEU5xDILbE6xMxWMTBUQ3b-TWkWDjA4

I have tried to compile with microsoft msvc and it works for me. Only one DLL needed for this: libraw :P
Title: Re: raw2mlv processing
Post by: Danne on November 12, 2019, 03:33:38 PM
Quote from: Ilia3101 on November 12, 2019, 01:56:57 PM
I seem to remember that windows needs to use special versions of the file reading/writing functions for files over 4GB, do you think file size could be the problem?
On mac I think it could be around 5000-6000 files before terminal chokes. Easy workaround using find xarg command but let´s see if mac gets some broken workflow first before developing further.
Title: Re: raw2mlv processing
Post by: cmh on November 12, 2019, 03:40:13 PM
Nevermind, 500 files gives the same error (433 is the maximum on my system) if you drag and drop on raw2mlv.exe but it processes the 500 files with a command.
I'll update my previous post to mention that limitation.

@Kharak here's what you can do:
- Copy raw2mlv.exe and all the dll's in the folder countaining your DNGs.
- Open a powershell windows and navigate to your folder (or shift + right clic and open the powershell windows from the folder)
- Type .\raw2mlv.exe *.dng -o output.mlv
Title: Re: raw2mlv processing
Post by: cmh on November 12, 2019, 03:56:01 PM
Quote from: Ilia3101 on November 12, 2019, 03:30:42 PM
Can you test if this runs on your PC: https://drive.google.com/open?id=1SzEU5xDILbE6xMxWMTBUQ3b-TWkWDjA4

I have tried to compile with microsoft msvc and it works for me. Only one DLL needed for this: libraw :P

It's working perfectly fine. Great job.
I'm putting that link in my previous post if you don't mind and will update it as soon as you give another one.
Edit: Oh wait, there's a problem with asterisks.
.\raw2mlv.exe *.CR2 worked previously but gives this error now:
Opening file *.CR2
failed to open file
Can't read file *.CR2
Title: Re: raw2mlv processing
Post by: ilia3101 on November 12, 2019, 04:22:34 PM
Ah asterisks, must be to do with mingw, it probably parses the asterisk, turning it in to loads of files before calling main. Windows probably does not support wildcards as much as Unix terminals.

I will try to find a solution.

Also did you call it from powershell? I wonder if normal cmd would work any different.
Title: Re: raw2mlv processing
Post by: cmh on November 12, 2019, 04:30:40 PM
cmd has the same problem unfortunately:
C:\Users\Windows\Desktop\raw2mlv>raw2mlv.exe *.CR2
Opening file *.CR2
failed to open file
Can't read file *.CR2

Just to be clear *.CR2 isn't a file name, it's a wildcard and it was working in powershell prior to that build. I phrased it poorly, sorry. It works in this one which is compiled with mingw:
https://github.com/cmhamiche/raw2mlv/raw/master/raw2mlv_Win64.zip

I reinstated the previous link in my first post until it's fixed.
Title: Re: raw2mlv processing
Post by: ilia3101 on November 12, 2019, 04:52:42 PM
I will have to find a way to support wildcards manually. Windows is not making things easy ☹️

Edit: wildcard support is easy, just needed to add a flag to microsoft linker. New build: https://drive.google.com/file/d/1o6CLtWfm_v9WrxUsorCnkwX8Y21sS45p/view?usp=sharing
Title: Re: raw2mlv processing
Post by: Walter Schulz on November 12, 2019, 05:10:34 PM
IANAP:
https://www.magiclantern.fm/forum/index.php?topic=7139.msg180824#msg180824
According to dcraw's Windows porter it just needed to be compiled with Visual Studio for wildcard support.

Title: Re: raw2mlv processing
Post by: ilia3101 on November 12, 2019, 05:12:42 PM
Thanks. But I just fixed it. Only needed to add "/link setargv.obj" to the link command, no visual studio required.

Source: https://docs.microsoft.com/en-us/cpp/c-language/expanding-wildcard-arguments?view=vs-2019
Title: Re: raw2mlv processing
Post by: Walter Schulz on November 12, 2019, 05:17:24 PM
Great!

Offtopic: Can you give cr2hdr binaries the same medicine? Just asking ...
And dcraw's Windows porter never made it to 9.28 and seems to be offline ...
Title: Re: raw2mlv processing
Post by: cmh on November 12, 2019, 05:21:00 PM
Quote from: Ilia3101 on November 12, 2019, 04:52:42 PM
I will have to find a way to support wildcards manually. Windows is not making things easy ☹️

Edit: wildcard support is easy, just needed to add a flag to microsoft linker. New build: https://drive.google.com/open?id=1o6CLtWfm_v9WrxUsorCnkwX8Y21sS45p
That link asks for an authorization. I've sent a request.
nvm https://drive.google.com/file/d/1o6CLtWfm_v9WrxUsorCnkwX8Y21sS45p/view works

Edit: yeah wildcards are working now, I'll upload this on my github and update my first post.
Title: Re: raw2mlv processing
Post by: ilia3101 on November 12, 2019, 05:29:56 PM
Quote from: Walter Schulz on November 12, 2019, 05:17:24 PM
Great!

Offtopic: Can you give cr2hdr binaries the same medicine? Just asking ...
And dcraw's Windows porter never made it to 9.28 and seems to be offline ...

I will try. Where do I find cr2hdr again? Also where was the dcraw Windows porter, was it on this forum?

@cmh Sorry, try the link now.
Title: Re: raw2mlv processing
Post by: Walter Schulz on November 12, 2019, 06:16:33 PM
Quote from: Ilia3101 on November 12, 2019, 05:29:56 PM
I will try. Where do I find cr2hdr again?

Not sure ...
https://builds.magiclantern.fm/jenkins/view/Utilities/job/cr2hdr-20bit/
https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c

Quote from: Ilia3101 on November 12, 2019, 05:29:56 PM
Also where was the dcraw Windows porter, was it on this forum?

I found his name and link as first item in Dave Coffin's dcraw page FAQ section:
Quote
Frequently Asked Questions

I don't have a C compiler. Could you send me an executable?
    Sergio Namias has built some current Windows EXE files here. Dcraw has also been ported to Amiga, MorphOS, BeOS, OS/2, and RISC OS.
Link is dead. This is latest Wayback Maschine backup:
https://web.archive.org/web/20190425220511/http://www.centrostudiprogressofotografico.it/en/dcraw/
Title: Re: raw2mlv processing
Post by: garry23 on November 12, 2019, 06:39:02 PM
@Danne

Ouch!

QuoteSidenote: funny thing I asked for explaining vids of your lua script once but not really happening right ;)?

...and there I have been writing 'loads of stuff' on my blog and getting zero feedback  ;)

  :)
Title: Re: raw2mlv processing
Post by: Danne on November 12, 2019, 06:42:08 PM
Quote from: garry23 on November 12, 2019, 06:39:02 PM
@Danne

Ouch!

...and there I have been writing 'loads of stuff' on my blog and getting zero feedback  ;)

  :)
Sure, but where´s the friggin´mooovie man  :-*
Title: Re: raw2mlv processing
Post by: garry23 on November 12, 2019, 07:10:11 PM
Don't do video; 'just' stills   8) :)
Title: Re: raw2mlv processing
Post by: Kharak on November 13, 2019, 02:41:26 AM
Quote from: cmh on November 12, 2019, 03:40:13 PM
Nevermind, 500 files gives the same error (433 is the maximum on my system) if you drag and drop on raw2mlv.exe but it processes the 500 files with a command.
I'll update my previous post to mention that limitation.

@Kharak here's what you can do:
- Copy raw2mlv.exe and all the dll's in the folder countaining your DNGs.
- Open a powershell windows and navigate to your folder (or shift + right clic and open the powershell windows from the folder)
- Type .\raw2mlv.exe *.dng -o output.mlv

I was narrowing down the exact number before it cracked, guess you found a better way.

Thanks for the fix. Would i go about the same way to create a Windows shortcut command, as a right click menu option?
Title: Re: raw2mlv processing
Post by: IDA_ML on November 13, 2019, 08:12:15 AM
CMH,  Ilia1301,

Glad to see you made the miracle happen. Congratulations!

Unfortunately, it does not work for me on my Win7x64 laptop here.  When I drag and drop the CR2 files onto the raw2mlv.exe file, as shown in CMH's demo video, the program stops working with the message:

"The program can't start because VCRUNTIME140_1.dll is missing from your computer.  Try reinstalling the program to fix this problem."


I get the same message when I just klick on raw2mlv.exe to start it.

Can you help me, please?  Thanks.
Title: Re: raw2mlv processing
Post by: 2blackbar on November 13, 2019, 08:53:04 AM
Download first link with win64 version in this topic, you have all dlls there, newest links dont have them
Title: Re: raw2mlv processing
Post by: cmh on November 13, 2019, 09:09:20 AM
Quote from: IDA_ML on November 13, 2019, 08:12:15 AM
CMH,  Ilia1301,
Glad to see you made the miracle happen. Congratulations!

Thanks for the kind words but all the credit goes to Ilia1301, I just created a post.

Quote from: IDA_ML on November 13, 2019, 08:12:15 AM
Unfortunately, it does not work for me on my Win7x64 laptop here.  When I drag and drop the CR2 files onto the raw2mlv.exe file, as shown in CMH's demo video, the program stops working with the message:

"The program can't start because VCRUNTIME140_1.dll is missing from your computer.  Try reinstalling the program to fix this problem."


Can you install the Microsoft Visual C++ Redistributable and report if this works ?
If it does work, I'll put a disclaimer and a link in my first post otherwise I'll have to include this dll in the archive and I don't know if Ilia1301 is ok with that.
Quick download: https://aka.ms/vs/16/release/vc_redist.x64.exe

Full link: https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads
Visual Studio 2015, 2017 and 2019
x64: vc_redist.x64.exe

Quote from: 2blackbar on November 13, 2019, 08:53:04 AM
Download first link with win64 version in this topic, you have all dlls there, newest links dont have them
I removed this link, the executable was compiled with mingw which required those particular libraries but not the visual studio redistributable.
The newest is compiled with visual studio but requires less dlls.
Title: Re: raw2mlv processing
Post by: andy kh on November 13, 2019, 09:15:16 AM
wow works great on my Win10X64
Title: Re: raw2mlv processing
Post by: IDA_ML on November 13, 2019, 10:27:34 AM
CMH,

I installed vc_redist.x64.exe as suggested but no luck.  RAW2MLV.exe crashes upon an attempt to start.  It generates an output file with 0 bytes of size.

I also tried it on a Win10x64 machine.  It does not work either.
Title: Re: raw2mlv processing
Post by: ilia3101 on November 13, 2019, 11:04:30 AM
@cmh please include any DLLs needed! if it helps
Title: Re: raw2mlv processing
Post by: cmh on November 13, 2019, 11:25:38 AM
Quote from: IDA_ML on November 13, 2019, 10:27:34 AM
CMH,

I installed vc_redist.x64.exe as suggested but no luck.  RAW2MLV.exe crashes upon an attempt to start.  It generates an output file with 0 bytes of size.

I also tried it on a Win10x64 machine.  It does not work either.

Right, I've included vcruntime140_1.dll
Would you please try this one?
https://github.com/cmhamiche/raw2mlv/raw/master/raw2mlv.zip

Title: Re: raw2mlv processing
Post by: IDA_ML on November 13, 2019, 01:12:50 PM
CMH,

Still no luck!  It crashes in exactly the same way as described above.  It does not even spit out a message indicating which DLL is missing.
Title: Re: raw2mlv processing
Post by: cmh on November 13, 2019, 01:27:39 PM
Just to be sure we're on the same page, could you follow those two procedures and report back ?

1. Drop the CR2/DNG files on raw2mlv.exe.
You'll find an output.mlv in the folder where your CR2/DNG are located (this method might be limited to a maximum of 433 files).

2. Copy the CR2/DNG in a directory containing raw2mlv.exe (and the dlls).
Open a cmd/powershell window, navigate to you folder (or open one from the current folder using shift + right clic) and use the command line.

Example, this command will open IMG_1687.CR2 create an mlv file named test.mlv:
raw2mlv.exe "C:\Users\SomeDude\Desktop\IMG_1687.CR2" -o "C:\Users\SomeDude\Desktop\test.mlv"

Title: Re: raw2mlv processing
Post by: IDA_ML on November 13, 2019, 05:22:59 PM
CMH,

Thank you so much for your patience with me.  I am at work right now and tested the software package from your post #46 on my work computer (Q6600 CPU with 8 GB of RAM and Nvidia GTX 750 GPU).  And it works like a charm according to your procedure #1 above.  RAW2mlv.exe does not seem to like my home laptop (Dell Precision M6600 with i7 CPU, 32 GB of RAM and 3000M GPU).  Is there anything I could change in the BIOS maybe to get it to work?
Title: Re: raw2mlv processing
Post by: Walter Schulz on November 13, 2019, 05:34:31 PM
BIOS/UEFI should be unrelated!
My recommendation: Check virus/internet protection software first and make sure it is not blocking. Run exe from command line and check for log entries (for
virus/internet protection and Windows event logs, too).
If problem persists I would recommend to enter a Teamviewer session with Process Monitor (Sysinternals/Microsoft) logging raw2mlv activities.
Title: Re: raw2mlv processing
Post by: IDA_ML on November 13, 2019, 05:51:22 PM
Thanks Walter,

After I get back from work, I will check if Windows defender blocks the RAW2mlv.exe file.  I don't have any other virus protection software installed.
Title: Re: raw2mlv processing
Post by: cmh on November 13, 2019, 06:14:44 PM
Quote from: IDA_ML on November 13, 2019, 05:22:59 PM
CMH,

Thank you so much for your patience with me.  I am at work right now and tested the software package from your post #46 on my work computer (Q6600 CPU with 8 GB of RAM and Nvidia GTX 750 GPU).  And it works like a charm according to your procedure #1 above.  RAW2mlv.exe does not seem to like my home laptop (Dell Precision M6600 with i7 CPU, 32 GB of RAM and 3000M GPU).  Is there anything I could change in the BIOS maybe to get it to work?

You're welcome. Is there any chance that the Dell Precision M6600 is on a 32 bits version of Windows ?
Other than that, you might want to check your antivirus like Walter Schulz suggested.
Title: Re: raw2mlv processing
Post by: IDA_ML on November 13, 2019, 06:22:11 PM
Quote from: cmh on November 13, 2019, 06:14:44 PM
You're welcome. Is there any chance that the Dell Precision M6600 is on a 32 bits version of Windows ?

No, I run Win7x64 on the M6600 and am very pleased with the overall performance.  Never had any problems with it.
Title: Re: raw2mlv processing
Post by: cmh on November 13, 2019, 06:51:09 PM
Right, you might want to try this slightly older build on your M6600 then:
https://github.com/cmhamiche/raw2mlv/raw/master/raw2mlv_Win64.zip
This build won't be maintained tho.
Eventually, you'll have to fix those problems if new features comes out.
Title: Re: raw2mlv processing
Post by: IDA_ML on November 13, 2019, 10:48:51 PM
CMH,

I have now tried the following on my M6600:

1) Disconnected the computer from the internet and deactivated Windows defender

2) Tried the new version from your post #46 and the old version from your post #54.  I pasted the files from the archives in C:\Temp along with a file called IMG_7673.CR2.  I used the command line as you suggested.  RAW2mlv crashes  again but before this happens, I get the following messages on the screen:

Output name set to C:\Temp\test.mlv
Opening file C:\Temp\IMG_7673.CR2
0.67220, -0.06350, -0,06930
-0.42870, 1.24600, 0.20280
-0.09080, 0.21629, 0.56680

Then the program crashes generating an empty test.mlv file.




 
Title: Re: raw2mlv processing
Post by: ilia3101 on November 13, 2019, 11:15:08 PM
Looks like it's having problems writing the file 🤔

Can you upload on of your files? I just want to see if it could be caused by that.
Title: Re: raw2mlv processing
Post by: IDA_ML on November 14, 2019, 08:43:41 AM
Sure, Ilia3103, here you go:

https://we.tl/t-TvhpqWZhNH

It is a 5D3 file.
Title: Re: raw2mlv processing
Post by: Danne on November 14, 2019, 08:55:49 AM
The file is not working to transcode on mac.
Title: Re: raw2mlv processing
Post by: ilia3101 on November 14, 2019, 09:54:31 AM
Interesting, does not work for me. Segmentation fault 11.

Update: I have found that it incorrectly reads width of the raw file as 3960 and the height as 2640. The mark 3's resolution is 5760x3840

UPDATE 2: I see this is not raw, it's sraw, not bayer. No idea what kind of image sraw is. The MLV format does not support this. Or maybe it does, but has never been implemented.
Title: Re: raw2mlv processing
Post by: cmh on November 14, 2019, 10:28:49 AM
That's what I was about to say, libraw shows "only bayer-pattern raw files supported, sorry...."
Title: Re: raw2mlv processing
Post by: ilia3101 on November 14, 2019, 10:35:16 AM
Rawtherapee can open it though, I wonder how they do it. I thought they use libraw.
Title: Re: raw2mlv processing
Post by: cmh on November 14, 2019, 11:09:01 AM
They are using darktable's implementation of rawspeed apparently.
Here's some oldish info on how to compile libraw with rawspeed support:
https://github.com/LibRaw/LibRaw/blob/master/README.RawSpeed.txt
https://gitlab.com/ImageMagick/LibRaw/blob/master/README.RawSpeed.txt
https://github.com/FMeinicke/QtRaw
edit: You'll probably loose your sanity fiddling with mingw tho
https://github.com/darktable-org/rawspeed

edit: I was wrong, for Rawtherapee it's dcraw they maintain internally.
Title: Re: raw2mlv processing
Post by: IDA_ML on November 14, 2019, 12:03:22 PM
Ooops, forgot to mention that this file was shot in mRAW format.  ACR opens it just fine, so I did not think of that.  Sorry guys.
Title: Re: raw2mlv processing
Post by: ilia3101 on November 14, 2019, 12:58:45 PM
Yeah will have to add rawspeed, and discuss how to support it in MLV.
Title: Re: raw2mlv processing
Post by: cmh on November 14, 2019, 01:00:55 PM
I was wrong, for Rawtherapee it's dcraw they maintain internally.
https://github.com/Beep6581/RawTherapee/issues/3988

Here's RT dcraw:
https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/dcraw.cc

and here's where they add new cameras:
https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json
Title: Re: raw2mlv processing
Post by: ilia3101 on November 14, 2019, 01:49:42 PM
Does anyone know what kind of image sraw and mraw is? I'm quite certain it's not bayer. Is it RGB or YUV? (I think i've heard something like that)

@cmh Interesting. Looks like they have the correct colour matrix for the mark II in that JSON... so it's a mystery why rawtherapee gives such wrong colours ::)
Title: Re: raw2mlv processing
Post by: cmh on November 14, 2019, 02:58:32 PM
YCbCr according to this:
http://lclevy.free.fr/cr2/#sraw
edit: it is encoded in YCbCr 4:2:2 and needs interpolation for the Cb and Cr values.

Title: Re: raw2mlv processing
Post by: Danne on November 14, 2019, 05:36:05 PM
Interesting. Approachgin the possibilty to include sraw and also non raw files into MLV wrapper  8)
Title: Re: raw2mlv processing
Post by: cmh on November 14, 2019, 05:51:41 PM
I'm reading this right now:
https://github.com/lclevy/libcraw2

"Libcraw2 has two kind of interfaces:
an C API, mainly used by crawtool, to offer CR2 data handling in a Camera agnostic way
an Python extension, PyCraw2 a simpler API which goal is to offer all required functions to render RAW data into a 16bits RGB image, either from RAW (RGGB) or sraw/mraw (YCbCr) data."
Title: Re: raw2mlv processing
Post by: vstrglv on November 14, 2019, 07:07:24 PM
https://github.com/cmhamiche/raw2mlv/raw/master/raw2mlv_Win64.zip works very good on win10.64 . Thank you very much!
But if i try CR2 dual iso  from 5d3 to raw2mlv, then mlv to MLV App 1.9, Forced Dual ISO, Export as DNG there is only a pink frame. If an out format is tif there is no problem.What is wrong?
Title: Re: raw2mlv processing
Post by: Danne on November 14, 2019, 07:41:07 PM
Try open the dng in adobe camera raw. Set white balance and all is fine.
Title: Re: raw2mlv processing
Post by: cmh on November 14, 2019, 07:42:40 PM
Ilia3101 might answer that question I certainly cannot, but I think posting the cr2 file might be helpful (and I could try myself out of curiosity).
@Danne is it a known bug?
Title: Re: raw2mlv processing
Post by: Danne on November 14, 2019, 08:01:21 PM
Yes, preview is wacky but information seems ok. Asshotneutral tag might be off too. Easily reproduced. Transcode a cr2 into a mlv. Open it up in mlv app and export it to a dng. Now check the file...
Title: Re: raw2mlv processing
Post by: vstrglv on November 14, 2019, 08:10:35 PM
Quote from: Danne on November 14, 2019, 07:41:07 PM
Try open the dng in adobe camera raw. Set white balance and all is fine.
Thank you! Tint was max.-+150
Title: Re: raw2mlv processing
Post by: Danne on November 14, 2019, 08:17:55 PM
Yes, white balance is incorrectly added to the dng file. I checked a dng and this is what in the As Shot Neutral tag from my particular file:
As Shot Neutral                 : 0.0004656612873 1 0.0004656612873
Good thing is that it´s a nice, usable dng otherwise.
Title: Re: raw2mlv processing
Post by: ilia3101 on November 14, 2019, 08:44:33 PM
Quote from: cmh on November 14, 2019, 02:58:32 PM
YCbCr according to this:
http://lclevy.free.fr/cr2/#sraw
edit: it is encoded in YCbCr 4:2:2 and needs interpolation for the Cb and Cr values.



Thanks

Quote from: cmh on November 14, 2019, 05:51:41 PM
I'm reading this right now:
https://github.com/lclevy/libcraw2

"Libcraw2 has two kind of interfaces:
an C API, mainly used by crawtool, to offer CR2 data handling in a Camera agnostic way
an Python extension, PyCraw2 a simpler API which goal is to offer all required functions to render RAW data into a 16bits RGB image, either from RAW (RGGB) or sraw/mraw (YCbCr) data."

Looks liek a nice library. Still need to decide on an official way of including other types of image data like RGB in MLVs, only bayer images are properly supported.
Title: Re: raw2mlv processing
Post by: cmh on November 14, 2019, 09:15:18 PM
Libcraw2 doesn't seem maintained but the author is aware of magic lantern's forum. Bugging him on github or tweeter could be worth the shot.  Maybe Rawtherapy's dcraw instead of libraw is a better solution since it is an active project (it's not a lib tho, not sure).
Title: Re: raw2mlv processing
Post by: masc on November 14, 2019, 10:13:55 PM
I tried to integrate RAW2MLV to MLVApp. To use this, compile and install raw2mlv like this (path of MLVApp executable):
(https://user-images.githubusercontent.com/30245296/68895859-7c151a00-072a-11ea-92c5-cb7a8651b622.png)
If MLVApp finds this executable, it will show this action:
(https://user-images.githubusercontent.com/30245296/68895881-8afbcc80-072a-11ea-95ec-a49d7fd8026e.png)
Triggering brings:
(https://user-images.githubusercontent.com/30245296/68895897-93540780-072a-11ea-97f8-4a85999b68f3.png)
Select single picture or a sequence folder, then click "Transcode and Import", choose a destination folder. Then it will need some time - WAIT! - and the converted files show up in the session.

TESTED ON OSX ONLY! Linux might work too, on Windows I expect some additions. AND: only the "good" use case works - no error handling at all. So if you do s*** you will get s***. :P
Title: Re: raw2mlv processing
Post by: Levas on November 14, 2019, 10:19:39 PM
Found out about this raw2mlv today  :o
Haven't tried it...yet, but already:
Great, nice & thanks  :D 8)
Title: Re: raw2mlv processing
Post by: cmh on November 14, 2019, 10:26:47 PM
Darktable use rawspeed and is able to open sraw/mraw just fine btw.

@masc
Nice. People will use a gui, that's great.
Title: Re: raw2mlv processing
Post by: cmh on November 15, 2019, 12:01:54 AM
@masc Yeah, it's working great on Windows if I change line 1344 in platform/qt/MainWindow.cpp to

#ifdef Q_OS_WIN
    //raw2mlv available?
    if( !QFileInfo( QString( "%1/raw2mlv.exe" ).arg( QCoreApplication::applicationDirPath() ) ).exists() )
        ui->actionTranscodeAndImport->setVisible( false );
#endif
#ifdef Q_OS_LINUX
    //raw2mlv available?
    if( !QFileInfo( QString( "%1/raw2mlv" ).arg( QCoreApplication::applicationDirPath() ) ).exists() )
        ui->actionTranscodeAndImport->setVisible( false );
#endif


Easy peasy.
Title: Re: raw2mlv processing
Post by: IDA_ML on November 15, 2019, 01:44:57 AM
Just tested RAW2mlv.exe with a CR2 file from the 100D.  Works perfectly on my M6600.  I can't believe my eyes, how well the skin tone white ballance works on portrait stills, just with a single mouse click.

Obviously, sRAW and mRAW files are the trouble makers.  It would be nice if these two formats could also be processed with RAW2mlv - very convenient for wedding and event shooting where the huge full-size RAW files are not needed.  Workflow is much faster and saves a lot of disk space.

Masc,

I think, now is the time for adding two more features for stills processing to MLVApp:

1)  Export to JPEG with minimum compression, something like 11 in Photoshop.  An adjustable compression ratio would even be better.  Right now, we only have the PNG and DNG export formats for current frame stills export which requires JPEG conversion in an external software.

2)  Adding a crop tool with adjustable aspect ratio similar to the one in ACR.   

Is this doable?
Title: Re: raw2mlv processing
Post by: garry23 on November 15, 2019, 07:22:04 AM
...and don't forget the idea of being able to do statistics, eg mean, i.e. create an image from statistically merging the x images in an mlv, as can be done in Ps.

This, of will help in creating an mlv focused tool to simulate LEs in post.

If this could be done, it would be a fantastic feature.

;)
Title: Re: raw2mlv processing
Post by: masc on November 15, 2019, 08:15:42 AM
Quote from: cmh on November 15, 2019, 12:01:54 AM
@masc Yeah, it's working great on Windows if I change line 1344 in platform/qt/MainWindow.cpp to

#ifdef Q_OS_WIN
    //raw2mlv available?
    if( !QFileInfo( QString( "%1/raw2mlv.exe" ).arg( QCoreApplication::applicationDirPath() ) ).exists() )
        ui->actionTranscodeAndImport->setVisible( false );
#endif
#ifdef Q_OS_LINUX
    //raw2mlv available?
    if( !QFileInfo( QString( "%1/raw2mlv" ).arg( QCoreApplication::applicationDirPath() ) ).exists() )
        ui->actionTranscodeAndImport->setVisible( false );
#endif


Easy peasy.
That's not all... OSX doesn't work this way. And about Linux I am not sure.

Quote from: IDA_ML on November 15, 2019, 01:44:57 AM
I think, now is the time for adding two more features for stills processing to MLVApp:

1)  Export to JPEG with minimum compression, something like 11 in Photoshop.  An adjustable compression ratio would even be better.  Right now, we only have the PNG and DNG export formats for current frame stills export which requires JPEG conversion in an external software.

2)  Adding a crop tool with adjustable aspect ratio similar to the one in ACR.   

Is this doable?
1) I can offer the standard Qt jpg export for single frame export.
2) This is still very difficult. And I don't think it will take longer than 3 days until someone asks for free turning, perpective stretching, etc. ... and all this is a very huge amount of work.

@garry23: could you please explain a bit more? I don't understand what you are asking. What are LEs? Do you mean an averaged frame? This is possible already.
Title: Re: raw2mlv processing
Post by: IDA_ML on November 15, 2019, 09:39:47 AM
Quote from: masc on November 15, 2019, 08:15:42 AM
1) I can offer the standard Qt jpg export for single frame export.

Thanks a lot, Masc, for considering this.  Yes, please implement it.  It would be an important step towards single CR2 files processing in MLVApp.
Title: Re: raw2mlv processing
Post by: garry23 on November 15, 2019, 10:39:23 AM
@masc

Forgive me, I feel an interloper here, as I don't do any video stuff.

Currently, if I don't have my NDs with me or the wrong one, I can simulate a longer exposure (LE) by taking x images of y sec exposure, to cover the same motion blur time as one exposure at x times y seconds.

I then send all the images to Photoshop as layers and use the Ps statistics feature to blend them all together with mean or median.

I'm hopeful that what is going on with raw2mlv will give me a single tool that allows me to do the same in the mlv post processing app, eg and ideally

- as with Dual post processing, seamlessly select x images in LR and export to the mlv post processing app, via the raw2mlv tool
- post process the mlv file, ie merge with mean or median statistics
- return to LR

Or do the above via a standalone method, ie not coupled to LR.

I also hopeful the current developments will soon be available without compiling on winX  ;)

Bottom line: I'm really looking forward to using this approach.

Thanks for all those involved in pushing the various tools along.

Cheers

Garry
Title: Re: raw2mlv processing
Post by: cmh on November 15, 2019, 11:17:36 AM
Quote from: masc on November 15, 2019, 08:15:42 AM
That's not all... OSX doesn't work this way. And about Linux I am not sure.

That was to be expected, I know nothing about qt or c++. QString QFileInfo::baseName () const seems to be a good way to strip the extension but I wouldn't know where to put any of that. I know you'll have to test for input validation later anyway. Meanwhile, I'll just replace raw2mlv by raw2mlv.exe and reupload it real quick.

Title: Re: raw2mlv processing
Post by: histor on November 15, 2019, 11:31:44 AM
Quote from: garry23 on November 15, 2019, 10:39:23 AM
- post process the mlv file, ie merge with mean or median statistics

Already implemented in mlv_dump. Simple average only. Not yet included in the GUI.

-- Image manipulation --
-a                  average all frames in <inputfile> and output a single-frame  MLV from it
--avg-vertical      [DARKFRAME ONLY] average the resulting frame in vertical direction, so we will extract vertical banding
--avg-horizontal    [DARKFRAME ONLY] average the resulting frame in horizontal direction, so we will extract horizontal banding
-s mlv_file         subtract the reference frame in given file from every single frame during processing
-t mlv_file         use the reference frame in given file as flat field (gain correction)
Title: Re: raw2mlv processing
Post by: masc on November 15, 2019, 11:34:55 AM
@garry23:
You can do all this with this MLVApp revision I commited yesterday.
- transcode and import all your cr2 pics into one MLV
- export the MLV as averaged MLV
- open the averaged MLV (this one has one single frame)
- apply dualiso (or not)
- do your settings
- export e.g. PNG or DNG (for LR)
Title: Re: raw2mlv processing
Post by: garry23 on November 15, 2019, 11:42:10 AM
@masc

I owe you a beer  :)

I guess the only thing I now need is to wait for standalone, compiled Win version  ;)

Once again, thanks for all your work.

Cheers

Garry
Title: Re: raw2mlv processing
Post by: masc on November 15, 2019, 11:43:03 AM
Haha. Try the one from @cmh. Thanks.
Title: Re: raw2mlv processing
Post by: garry23 on November 15, 2019, 12:37:32 PM
@masc @cmh

As I say, I'll wait.

Just tried it, but running MLVapp.exe and I get the qt platform plugin error.

As I say, I'll wait until someones says it runs without 'fiddling'  ;)

Cheers

Garry
Title: Re: raw2mlv processing
Post by: cmh on November 15, 2019, 01:03:14 PM
Quote from: garry23 on November 15, 2019, 12:37:32 PM
As I say, I'll wait until someones says it runs without 'fiddling'  ;)
I respect that.

I'll still create a virtual machine real quick just to see if I can reproduce that error.

Edit: in the meantime, you can still use Ilia3101's build of raw2mlv and use it as shown in the video you've requested.

Edit: nvm the video isn't about long exposure.
Title: Re: raw2mlv processing
Post by: Quentin on November 15, 2019, 06:03:39 PM
edit: Here's a Windows version for those eager to test it.
https://github.com/cmhamiche/raw2mlv/raw/master/MLV-App_raw2mlv_Win64.zip

Broken link, 404
Title: Re: raw2mlv processing
Post by: cmh on November 15, 2019, 06:21:21 PM
You'll probably have to wait for an official release unfortunately.
I might post another link later (but it requires recompiling qt to get a static version, which takes some time).
Title: Re: raw2mlv processing
Post by: masc on November 15, 2019, 06:25:23 PM
No need for a static version. Just search and copy the missing dlls from your Qt directory.
Title: Re: raw2mlv processing
Post by: cmh on November 15, 2019, 06:40:09 PM
Yeah, but it won't solve garry23's error, I tried on a fresh vm (dropping qwindows.dll doesn't work).
"could not find or load the Qt platform plugin "windows" in"
If people have another qt app installed they might not get this error, I don't know for sure.
So there's few potential options that could solve this issue that I'm aware of:
- using an installer to add QT_QPA_PLATFORM_PLUGIN_PATH to their system environment (and include the content of qt's \plugins\platforms with the rest of the DLLs) and I'm not sure it would work
- maybe using visual studio and I'd rather not.
- building a static win64 version.
Title: Re: raw2mlv processing
Post by: masc on November 15, 2019, 07:15:01 PM
You need about 10 or more dlls from Qt dir. Get some inspiration from offical (non static) version. MLVApp won't compile with VisualStudio.
Title: Re: raw2mlv processing
Post by: cmh on November 15, 2019, 08:41:18 PM
Sure.
https://github.com/cmhamiche/mlvapp_win/raw/master/MLV.App.v1.9.Win64_raw2mlv.zip
edit: added missing imageformats and platforms folders plus dfort's pixel maps.
I hope it works fine. Meanwhile qt's still compiling.

edit: New build with single frame jpg export available.
I'll put the link in MLV App's thread (Sony raw will have to wait).
https://www.magiclantern.fm/forum/index.php?topic=20025.msg222468#msg222468
Title: Re: raw2mlv processing
Post by: ilia3101 on November 16, 2019, 01:12:50 PM
Quote from: cmh on November 15, 2019, 08:41:18 PM
(Sony raw will have to wait).

What does this mean
Title: Re: raw2mlv processing
Post by: masc on November 16, 2019, 01:22:51 PM
Quote from: Ilia3101 on November 16, 2019, 01:12:50 PM
What does this mean
I filtered file extensions in MLVApp, to not send e.g. mp3, exe, docx, ... to your lib. I had a look what libraw supports and took just the most important file extensions. See latest commit messages on repos, then you'll understand. Yesterday evening I got some Sony raw files and tried them out... after I added the extension to the filter. Looks like I did that after @cmh compiled...
Title: Re: raw2mlv processing
Post by: cmh on November 16, 2019, 05:02:03 PM
I've compiled a new build with the last commit:
https://www.magiclantern.fm/forum/index.php?topic=20025.msg222468#msg222468
Title: Re: raw2mlv processing
Post by: DeafEyeJedi on November 20, 2019, 07:11:00 AM
This is all fantastic progress. Loving the collaborations from everyone on here.  8)

Especially @garry23 for making sure we don't put photography on the back burner.
Title: Re: raw2mlv processing
Post by: 2blackbar on November 29, 2019, 06:00:03 AM
Would it be possible to make dng's from google pixel cameras working ? They shoot raw as well and i tried to convert their dng by importing in mlvapp , that didnt worked, shows gray image, i can edit them as regular dng files in other programs like rawtherapee but id love to open them in amazing MLVApp.
Heres link to the dng, i dont know how different it is from other dng's :
https://drive.google.com/open?id=1gUkQd9ssmG2dH-JlAtNnAj25weGy6YiI
Title: Re: raw2mlv processing
Post by: AF-OFF on December 08, 2019, 07:32:22 PM
Quote from: masc on November 14, 2019, 10:13:55 PM
I tried to integrate RAW2MLV to MLVApp. To use this, compile and install raw2mlv like this (path of MLVApp executable):
(https://user-images.githubusercontent.com/30245296/68895859-7c151a00-072a-11ea-92c5-cb7a8651b622.png)
If MLVApp finds this executable, it will show this action:
(https://user-images.githubusercontent.com/30245296/68895881-8afbcc80-072a-11ea-95ec-a49d7fd8026e.png)
Triggering brings:
(https://user-images.githubusercontent.com/30245296/68895897-93540780-072a-11ea-97f8-4a85999b68f3.png)
Select single picture or a sequence folder, then click "Transcode and Import", choose a destination folder. Then it will need some time - WAIT! - and the converted files show up in the session.

TESTED ON OSX ONLY! Linux might work too, on Windows I expect some additions. AND: only the "good" use case works - no error handling at all. So if you do s*** you will get s***. :P

This doesn't work for me on any of my macs. Any help would be appreciated.
I have MBP with Yosemite, MBP with Catalina and Macpro5.1 with High sierra.
What I dit, just extracted the executeable from the raw2mlv.app and copied it to the MLV10.1 package inside of Macos folder. Checking in Terminal, it shows the correct right like mlv or ffmpeg executables.

Fortunately, raw2mlv.app standalone works well

I also became curious, if it works with CHDK - generated cr2 files from my sx220 - sadly it doesn't
The files are recognised by the mac finder and raw2mlv generates a mlv video clip, but this is pure white only. So maybe this is no real cr2 file I assume. Anyone using CHDK also who can check this?

thanks
Title: Re: raw2mlv processing
Post by: masc on December 08, 2019, 07:45:06 PM
Quote from: AF-OFF on December 08, 2019, 07:32:22 PM
This doesn't work for me on any of my macs. Any help would be appreciated.
I have MBP with Yosemite, MBP with Catalina and Macpro5.1 with High sierra.
What I dit, just extracted the executeable from the raw2mlv.app and copied it to the MLV10.1 package inside of Macos folder. Checking in Terminal, it shows the correct right like mlv or ffmpeg executables.

Fortunately, raw2mlv.app standalone works well
Do you use MLVApp 1.10 official release? This function is not included in the release. You'll need to compile MLVApp master branch yourself, in order to use this. It was "too much beta" for beeing included to the release.

Quote from: AF-OFF on December 08, 2019, 07:32:22 PM
I also became curious, if it works with CHDK - generated cr2 files from my sx220 - sadly it doesn't
The files are recognised by the mac finder and raw2mlv generates a mlv video clip, but this is pure white only. So maybe this is no real cr2 file I assume. Anyone using CHDK also who can check this?
Got this with a few other files too... did not found out why. This is the beginning of this tool... still much work to do.
Title: Re: raw2mlv processing
Post by: AF-OFF on December 08, 2019, 07:50:44 PM
Ah thanks for clarification Masc - I did understand Your post which I was quoting as this has to be done manualy anyways. Yes using now the newest 1.10 release
Title: Re: raw2mlv processing
Post by: masc on December 08, 2019, 07:58:20 PM
Yes, correct, it has to be done manually in addition to the "self compiled version".
Title: Re: raw2mlv processing
Post by: AF-OFF on December 08, 2019, 08:04:28 PM
Ah ok
1. compile yourself MLVApp master branch
2. put the executable inside of MLVApp

Thanks again for clarification Masc
Title: Re: raw2mlv processing
Post by: 2blackbar on December 13, 2019, 04:30:02 AM
HEy guys i extracted camera matrices from dng files its quite easy with exiftool and dng file that you can easily obtain over internet.
https://exiftool.org/
I would like to compile this new version with 2 new camera matrices so they could work in MLVApp, pixel 2 XL and sigma fp, can someone help ?
https://drive.google.com/open?id=1LTOjL_AXI2MKwlQb7c_duNt5vV5fxVMs
Im installing ms visualstudio... or not, i dont have 8gb free atm, any other way to compile windows build without having to download over 7GB just to get small exe compiled ?
Title: Re: raw2mlv processing
Post by: ilia3101 on December 13, 2019, 07:44:17 PM
Thanks. You don't need all of Visual Studio, only the "Build Tools for Visual Studio" - I think this is the one: https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2019

I think it's still a few GB. I don't know how to solve windows for you.
Title: Re: raw2mlv processing
Post by: 2blackbar on December 16, 2019, 01:50:28 AM
with help of user cmh i compiled raw2mlv on windows, he explained to me what You have to do:
QuoteDownload and install msys2-x86_64-20190524.exe from http://www.msys2.org/ (default path).

Update the package database and core system packages:
$ pacman -Syu
It will show "terminate MSYS2 without returning to shell and check for updates again"
Close MSYS2, run it again from Start menu. Update the rest:
$ pacman -Su

You need a build environment and libraw:
$ pacman -S git mingw-w64-x86_64-toolchain mingw-w64-x86_64-gcc mingw-w64-x86_64-binutils mingw-w64-x86_64-cmake mingw-w64-x86_64-libraw

MSYS2 is now set, close it and open MSYS2 MinGW 64-bit from the start menu:

Clone LibMLV's repo from github:
$ git clone https://github.com/ilia3101/LibMLV.git

Now cd into raw2mlv's folder:
$ cd LibMLV/raw2mlv/

Clone Libraw's repo from github:
$ git clone https://github.com/LibRaw/LibRaw.git -b 0.19-stable

With a decent editor (like notepad++), edit C:\msys64\home\<username>\LibMLV\raw2mlv\makefile

CC=gcc
FLAGS=-c -O3

main: raw2mlv_c LibMLV
   $(CC) raw2mlv.o libraw_r-19.dll ../LibMLV/build/unix/libMLV.dll -o raw2mlv -lm -lstdc++

raw2mlv_c:
   $(CC) $(FLAGS) raw2mlv.c

LibMLV:
   $(MAKE) -C ../LibMLV/build/unix


Edit C:\msys64\home\<username>\LibMLV\LibMLV\build\unix\makefile

CC=gcc
FLAGS=-c -fPIC -O3

main: MLVWriter MLVReader MLVFrameUtils
   ar rcs libMLV.dll *.o
   $(CC) -shared MLVWriter.o MLVReader.o MLVFrameUtils.o -o libMLV.dll

MLVWriter:
   $(CC) ../../MLVWriter.c $(FLAGS)

MLVReader:
   $(CC) ../../MLVReader.c $(FLAGS)

MLVFrameUtils:
   $(CC) ../../MLVFrameUtils.c $(FLAGS)


Copy libraw_r-19.dll from C:\msys64\mingw64\bin in C:\msys64\home\<username>\LibMLV\raw2mlv

Compile
$ mingw32-make
So makefile in raw2mlv looks like this :
CC=gcc
FLAGS=-c -O3

ifeq ($(shell uname -s), Linux)
main: raw2mlv_c LibMLV
$(CC) raw2mlv.o libraw_r-19.dll ../LibMLV/build/unix/libMLV.dll -o raw2mlv -lm -lgomp -lstdc++
else
main: raw2mlv_c LibMLV
$(CC) raw2mlv.o libraw_r-19.dll ../LibMLV/build/unix/libMLV.dll -o raw2mlv -lm -lstdc++
endif

raw2mlv_c:
$(CC) $(FLAGS) raw2mlv.c

LibMLV:
$(MAKE) -C ../LibMLV/build/unix




makefile in unix folder looks like this :
CC=gcc
FLAGS=-c -fPIC -O3

main: MLVWriter MLVReader MLVFrameUtils
ar rcs libMLV.a *.o
$(CC) -shared *.o -o libMLV.so

MLVWriter:
$(CC) ../../MLVWriter.c $(FLAGS)

MLVReader:
$(CC) ../../MLVReader.c $(FLAGS)

MLVFrameUtils:
$(CC) ../../MLVFrameUtils.c $(FLAGS)

And it compiles fine.Then just copy exe where you have old one with dlls/overwrite
Im posting this for myself and other in the future who would like to tinker with this on windows
We have very helpful people on forums, in last couple of weeks i was able to compile my own versions of Magic lantern, MLVapp and now raw2mlv , thanks a lot guys ! half year ago i wouldnt even dare to try despite getting dirty with sourcecode before.Setting eveyrthing up is sometimes pain in the butt and i appreciate all help.
Title: Re: raw2mlv processing
Post by: ilia3101 on December 17, 2019, 10:19:03 PM
Very good! I will make an MSYS2 build folder. Would this work with minGW/cygwin too?
Title: Re: raw2mlv processing
Post by: 2blackbar on December 18, 2019, 10:36:45 PM
I have cygwin installed for ML compiling and yes i did it in very similar fashion with help of Daniel Fort tutorial on forum but there was an issue there with utilities updated on server and tutorial calling for too old version its fixed now.
Title: Re: raw2mlv processing
Post by: vstrglv on March 06, 2020, 12:10:39 PM
Link https://github.com/cmhamiche/raw2mlv/raw/master/raw2mlv.zip is dead.Reload please.
Title: Re: raw2mlv processing
Post by: masc on March 06, 2020, 12:13:19 PM
Find official versions at https://github.com/ilia3101/LibMLV/releases
Title: Re: raw2mlv processing
Post by: vstrglv on March 06, 2020, 12:46:10 PM
Thanks a lot! It works! It neads vcruntime140_1.dll. I am on Win10. Max number of *.dng files is between 600 and 700.
Title: Re: raw2mlv processing
Post by: ilia3101 on April 19, 2020, 10:26:30 PM
Added binning feature to raw2mlv - now you can simulate how your raw photo would look if it was shot as a MLV video.

Simply add "--binning 3" to your command (or 5 if you want 5x5 binning)

Some example MLVs created from 5Ds stills: https://drive.google.com/file/d/19Np6y-0iHeihtM875aHNdCSwEALff6DW/view?usp=sharing

Wouldn't it be nice if the 5Ds could shoot MLV ;) ;) ;)

(https://i.postimg.cc/NFMKvgLn/video5-frame-1.jpg) (https://i.postimg.cc/sjP7Sxz3/video5-frame-1.jpg)
Title: Re: raw2mlv processing
Post by: 2blackbar on November 24, 2020, 01:23:27 AM
OK i see some stuff has changed, does anyone have compiled windows version of raw2mlv with most recent changes ? Im planning to get lumix tz80 for quite cheap but its RAW matrices in MLVApp are only with pink and cyan channel so whole image is not useable,  yellow chanel is missing, so i wanted to see if its fixed in recent raw2mlv.
RAW Samples from tz80 are here on bottom :
https://www.photographyblog.com/reviews/panasonic_lumix_dmc_tz80_review/sample_images

Im having trouble to compile raw2mlv on windows.
Insteresting thing in this camera raw image in MLVApp is that it shows a bit more field of view ( not really that usable tho)than lightroom for example, it shows camera lens border...
(https://funkyimg.com/i/38SbH.jpg)
Yeah i could edit these in lightroom but lightroom dont stand a chance against reinhard tonemapping from MLVApp.
Title: Re: raw2mlv processing
Post by: Walter Schulz on March 16, 2022, 08:37:35 PM
@Danne: Can you take a look into that?
https://www.reddit.com/r/MagicLantern/comments/tfmx8x/should_i_be_worried_or_is_this_just_a_false_alarm/
Title: Re: raw2mlv processing
Post by: Danne on March 17, 2022, 05:52:58 AM
Says raw2mlv? It's ilia3101 program. Can't really see what's going on. What sources are being used? Maybe I am missing info?
Title: Re: raw2mlv processing
Post by: masc on March 17, 2022, 09:01:02 AM
These are the sources for the version included in MLVApp (or maybe slightly older, but not "much"... okay... maybe also the tagged release... but in theory it all works the same).
https://github.com/ilia3101/LibMLV/tree/b2e58cba5bb478af0775b18b5e1601014a6bb514
It uses LibRAW, LibMLV. All included in subfolder application raw2mlv. Can't see any virus... but maybe someone else...
Title: Re: raw2mlv processing
Post by: TonyLeps on October 24, 2022, 07:14:16 PM
Is there a way to embed extracted WAV files into the MLV?

Thank you so much for this tool. It's been very helpful to better process cdng files I exported before MLVApp.
Title: Re: raw2mlv processing
Post by: essbe on March 14, 2023, 11:28:37 PM
I am only using Linux, and I really like the workflow with doing all grading in the mlvapp before exporting it to a better format for editing, which makes the whole editing experience smoother. For various reasons I have started to look at alternatives to the 5d mk3 and EOS M. I tried converting some photo dng samples I downloaded from the Sigma FP, but when loading into MLVApp there were black bars, pink color etc, so I guess the FP is not supported, or do the cinema dng files work? I have also been looking for cdng-files for the original BMPCC online, with no luck, to see if that camera would work in MLVApp after converting cdng to mlv. Does anyone know if it works?