Magic Lantern Forum

Using Magic Lantern => Raw Video => Raw Video Postprocessing => Topic started by: kunchenguid on September 18, 2014, 09:14:05 AM

Title: RawFlow - open source batch process tool for MLV on Windows
Post by: kunchenguid on September 18, 2014, 09:14:05 AM
In my last trip to Utah I shot about 300 raw video clips with my 5D and when I got back home, I found there's no easy way to batch process so many MLV files.

So I spent a few hours to build this tool and I named it "RawFlow". The purpose is to free us from manually running the basic processing workflow for a large amount of MLV files.
And I'm making it open source so that people can contribute and make it more powerful.

To download, please visit:
https://github.com/kunchenguid/rawflow/releases (https://github.com/kunchenguid/rawflow/releases)

The usage is pretty simple - just drag and drop your MLV files onto RawFlow.exe and it will just start working. A sub directory will be created for each MLV file in the same parent folder (this can be configured), DNG and WAV files will be extracted and put into the sub directory, and a small proxy video file would be generated there also.

Running the tool requires Microsoft .Net Framework v4.5. Download from Microsoft: http://www.microsoft.com/en-us/download/details.aspx?id=42642 (http://www.microsoft.com/en-us/download/details.aspx?id=42642)

To contribute or get access to the source code, please visit:
https://github.com/kunchenguid/rawflow (https://github.com/kunchenguid/rawflow)

Hope this is helpful.
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: Kharak on September 18, 2014, 03:02:32 PM
Gonna give it a try later today

Need a good tool to make super light proxies

Rawanizer had this. But seemed to only work on occasions and its only raw
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: thomas132 on December 24, 2014, 04:33:36 PM
Massive thanks for this!
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: eric_marciniak on April 03, 2015, 05:01:12 AM
This is awesome! Is there any way to make it so it doesn't create proxy videos? Even on my overclocked 3930k it takes a long time to create the Tiff's and make the proxy video.
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: CFipp53 on April 28, 2015, 09:34:07 PM
Quote from: eric_marciniak on April 03, 2015, 05:01:12 AM
This is awesome! Is there any way to make it so it doesn't create proxy videos? Even on my overclocked 3930k it takes a long time to create the Tiff's and make the proxy video.

I took a quick look at the source code and commented out the parts for the tiffs/proxy video

Until I get a chance to do a better job of customizing this open source program, this download will allow you to run RawFlow without generating proxy videos, and saving a lot of time if you are only interested in the DNGs.

https://drive.google.com/file/d/0B9srM2vAy8P8QWs1Q0YyTlRweU0/view?usp=sharing (https://drive.google.com/file/d/0B9srM2vAy8P8QWs1Q0YyTlRweU0/view?usp=sharing)

Thanks for sharing the original and making it open source. kunchenguid, you the man
CF
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: eric_marciniak on May 14, 2015, 02:38:42 AM
Thanks a ton CFipp53!
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: 5Djp on June 03, 2015, 04:38:37 AM
wow.. i was just looking for Tony Beccars plugin, but downloaded this.. and it looks like it's extracting fine!  Many thanks!
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: 5Djp on June 03, 2015, 04:40:23 AM
And to CFipp53!
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: anoopgopinath on October 04, 2015, 05:22:36 AM
hi iam anoop

recently i shoot some footage for my first short movie, i try to process mlv raw footages using rawflow and footage was corrupted. when try to open footage it plays first frame but remains seems black

please tell me how to recover corrupted footage
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: Dionisgr on November 22, 2015, 12:40:46 PM
Very handy program!Thank you very much! But you could add it remove pink dots for Canon 650d.
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: kunchenguid on December 26, 2015, 04:41:37 AM
Quote from: eric_marciniak on April 03, 2015, 05:01:12 AM
This is awesome! Is there any way to make it so it doesn't create proxy videos? Even on my overclocked 3930k it takes a long time to create the Tiff's and make the proxy video.

Thanks for the feedbacks!! I have heard a lot of requests about this so I created a new release that disables proxy video generation by default and added a configuration setting for people who want it.

https://github.com/kunchenguid/rawflow/releases/tag/v0.2-beta (https://github.com/kunchenguid/rawflow/releases/tag/v0.2-beta)
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: kunchenguid on December 26, 2015, 04:45:25 AM
Quote from: CFipp53 on April 28, 2015, 09:34:07 PM
I took a quick look at the source code and commented out the parts for the tiffs/proxy video

Until I get a chance to do a better job of customizing this open source program, this download will allow you to run RawFlow without generating proxy videos, and saving a lot of time if you are only interested in the DNGs.

https://drive.google.com/file/d/0B9srM2vAy8P8QWs1Q0YyTlRweU0/view?usp=sharing (https://drive.google.com/file/d/0B9srM2vAy8P8QWs1Q0YyTlRweU0/view?usp=sharing)

Thanks for sharing the original and making it open source. kunchenguid, you the man
CF

Thanks a lot CF for making the workaround. I just created a new release that made this configurable. Feel free to send me pull requests on github if you want any changes incorporated.
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: kunchenguid on December 26, 2015, 04:54:35 AM
Quote from: Dionisgr on November 22, 2015, 12:40:46 PM
Very handy program!Thank you very much! But you could add it remove pink dots for Canon 650d.

Thanks man. I'm not very familiar with the pink dots issue but will look into it when I got a chance.
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: dfort on December 26, 2015, 08:43:27 PM
It looks like RawFlow is using dcraw so it should be easy to implement the same "badpixel" method like in Danne's MLP workflow (http://magiclantern.fm/forum/index.php?topic=13512). I have mapped the focus pixels on 1280x720 cropped video for EOSM. That should also work for @Dionisgr on the 650D.

By the way, I started a topic about focus pixels that you might find interesting.

http://www.magiclantern.fm/forum/index.php?topic=16054.0
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: phil7 on December 30, 2015, 10:13:50 PM
HI

I tried to install the program on my WIN 10 system PC but it crashes everytime I try to run it. It states it cannot find the log4net file but it is in the folder where the application was extracted. Help.
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: Quentin on April 27, 2017, 06:26:38 AM
Please make frame names to contain original name of MLV, not "frame"
It makes more sense if you shoot 30 videos and you post process them and you need to find the original name or think of a new one
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: TobWag on April 27, 2017, 07:17:16 AM
Quote from: kunchenguid
To download, please visit:
https://github.com/kunchenguid/rawflow/releases

Anybody knows another alternative? It looks like this SW is not developed any further. The last release is from December 2015.
Title: Re: RawFlow - open source batch process tool for MLV on Windows
Post by: reddeercity on November 14, 2017, 05:19:15 AM
Quote from: TobWag on April 27, 2017, 07:17:16 AM
.... not developed any further. The last release is from December 2015.
I downloaded the source code , opens up in Microsoft Visual Studio Community 2015 . Looks like a good project to update -- it looks like simple c# code at least that what Visual Studio is telling me . It would be a good project for myself or anyone here to get there feet wet in coding or at least start to under programing that what makes Visual Studio Community so good , it give hints and explanation's on what the syntax is doing etc. ..... It's helped me a lot  :D