Author Topic: MCMLVConverter: multithreaded MLV to DNG conversion (Windows)  (Read 1082 times)

Filmschmiede

  • New to the forum
  • *
  • Posts: 2
  • Part time filmmaker, part time developer
Hello community!

Since MLV to CDNG conversion is rather slow with compressed raw files and I couldn't find a converter that does it quicker than mlv_dump, I wrote my own. It still uses mlv_dump but executes several instances simultaneously. It also displays more verbose progress information and conversion speeds of the respective threads. I only have a Windows 64bit version.

You can download it here: MCMLVConverter.zip
There is no installer and it comes with its own little DLL-hell but I hope that's okay. I've tested it on Windows 7 and 10.

If your SSDs are quick enough you can have as many threads as you have logical CPU cores and still have a performance increase. I do not recommend using multithreaded conversion on HDDs, however, because of fragmentation.

The software's feature set is very basic. It provides every mlv_dump switch and allows to collect the results in a single directory but that's it. It only converts from MLV to CDNG and (apart from what mlv_dump provides) can do neither processing nor preview. It's made to fit my personal workflow needs closely but maybe someone else's, too.

You need your own mlv_dump.exe for it to work. Should you be experimenting a lot it may come in handy that you can easily use different mlv_dump executables. The software saves its settings so that you don't have to enter everything over and over again.

Here is a screenshot of it during operation, being 5x as fast as single-threaded, running with eight threads on four physical cores with hyperthreading. You can see the CPU usage in the task manager.




Maybe it's useful to someone else. I am happy to hear your complaints and suggestions concerning usability. But I currently do not intend to extend its feature set toward anything but conversion.
You may, however, download the source code and do whatever you like with it: MCMLVConverter_source.zip
But don't expect good architecture or even comments ::). It's been a quick hack after all.

Cheers...

   ...Mike

IDA_ML

  • Hero Member
  • *****
  • Posts: 603
Re: MCMLVConverter: multithreaded MLV to DNG conversion (Windows)
« Reply #1 on: July 15, 2018, 05:38:48 PM »
Hello Mike,

It's good to see that there are new community members like you that are willing to help and contribute.  So, welcome to Magic Lantern!

Did you try MLVFS?  It creates the virtual folders with cDNG files almost instantaneously and to save them permanently on your HDD or SSD is just a drag and drop away in Explorer.  How long will it take depends only and entirely on the write speed of your storage media.

If you have some free RAM on your computer and do not have too much footage to convert, you could build a RAM Disk and place your MLVFS folder there.  Once you copy your MLV files to that folder and click on your MLVFS icon, creation of the virtual folders with cDNG files is lightning fast. 

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5210
Re: MCMLVConverter: multithreaded MLV to DNG conversion (Windows)
« Reply #2 on: July 15, 2018, 06:11:34 PM »
Actually not a bad idea to do multiprocessing on ssd drives. I do the same on mac app Switch.

Filmschmiede

  • New to the forum
  • *
  • Posts: 2
  • Part time filmmaker, part time developer
Re: MCMLVConverter: multithreaded MLV to DNG conversion (Windows)
« Reply #3 on: July 16, 2018, 11:12:14 AM »
Thank you for the welcome!

I have at least looked at MLVFS. However, I am a bit weird. While I do have the occasional paid project, filmmaking is by far not my main income so I can afford some inefficiency and instead focus on long-term data availability, especially for my own works. Of course the main issue is vendors of editing software going bankrupt (or switching to subscriptions), making your project files useless. The other issue is the media files themselves. I'd like to have them in open, well documented and widely supported file formats while at the same time keeping most (if not all) of the original image quality. So if push came to shove I could always write my own converter. (c)DNG is my video format of choice. I have misjudged the price drop of storage space, though. But that's a different story. I'm also aware that as soon as multilayered sensors are available, debayering will become a dying art and support fill fade out ten or twenty years after that, but what can you do...
I have old projects from the 90s, already "transcoded" once from VHS to MPEG and I assume I'll have to transcode them at least once again in the next 20 years. No one really cares about MPEG any longer, even today. In a couple of years I'll be regrading this for HDR and P3 (or whatever the camera was able to capture and my future mastering display is capable of). Not a problem with the original raw files (the realtime sequences are Canon MOV, because it was before ML-RAW which meant that I wasn't using ML at the time, but I think you get where I'm going with this).

I also try to keep the editing system environment as simple as it can get. Currently I need to have Resolve Studio installed and nothing else. MLVFS and its dependency on other components is a bit too much for me. But don't get me wrong. Had I to live off of film production I would use MLVFS because of its efficiency. It seems like the perfect solution for those workflows.

Cheers...

   ...Mike