Magic Lantern Forum

Using Magic Lantern => Post-processing Workflow => HDR and Dual ISO Postprocessing => Topic started by: l_d_allan on December 08, 2013, 06:00:25 AM

Title: Work-around for accomplishing semi-multi-threading for cr2hdr.exe
Post by: l_d_allan on December 08, 2013, 06:00:25 AM
My experience is that cr2hdr.exe is rather slow. YMMV.

I've got an i7 with 4x2 cores, including hyper-threading. Generally, I can watch TaskManager monitor CPU usage at 12 to 13% ... which I interpret as the equivalent of one core of the eight. Bummer.

It would be great if cr2hdr.exe was multi-threaded, but it doesn't appear to be. And that's non-trivial to implement.

Here's a work-around for a scenario when you've got a LOT of Dual-ISO files to prep. For example,

That gets the TaskManager CPU monitor showing much greater than 12%, up to 100%, and the files are prepped in much less "wall time".  The /belownormal argument for "start" let the computer remain responsive, rather than sluggish while the CPU grinds away.

(obsolete) You should run Adobe's DngConverter.exe afterwards on the .dng files to accomplish compression ... otherwise the .dng's will be uncompressed and large.

Flawed? Room for improvement?

Edits:

S:
cd \_WorkInProgress\Temp
for %%n in (0 1 2 3 4 5 6 7 8 9) do start /belownormal for %%c in (YYMM*_?????%%n_*.cr2) do cr2hdr --compress %%c
pause

Title: Re: Work-around for accomplishing semi-multi-threading for cr2hdr.exe
Post by: Audionut on December 08, 2013, 06:30:47 AM
There is a script here (http://www.magiclantern.fm/forum/index.php?topic=8520.0) for running multiple instances of cr2hdr.
Title: Re: Work-around for accomplishing semi-multi-threading for cr2hdr.exe
Post by: l_d_allan on December 13, 2013, 05:43:05 AM
Thanks for bringing that to my attention. I'll take a look at it. Sorry ... didn't realize I was "re-inventing the wheel" ... my bad.
Title: Re: Work-around for accomplishing semi-multi-threading for cr2hdr.exe
Post by: l_d_allan on December 20, 2013, 05:41:25 PM
Quote from: Audionut on December 08, 2013, 06:30:47 AM
There is a script here (http://www.magiclantern.fm/forum/index.php?topic=8520.0) for running multiple instances of cr2hdr.

Again, thanks for the info.

Not meaning to be argumentative, but ...
It does seem a bit simpler, with less setup, to use the "embedded for loops" approach from the "vanilla" Windows command line:

> for %n in (0 1 2 3 4 5 6 7 8 9) do start /belownormal for %c in (*%n_DI_*.cr2) do cr2hdr %c

or not? Could be "pride of authorship" on my part?

I suppose a "vanilla" Windows installation can run .vbs scripts without installing anything? I'm mostly ignorant about .vbs.

BTW: I've thought about redirecting the output to >> CrdHr2Log.txt so I could more easily check that no errors showed up with "Find". Haven't tried that yet, however.  Using Adobe's Bridge, a quick check is that there should be the same number of .cr2's as .dng's in the subdirectory with cr2hdr.exe
Title: Re: Work-around for accomplishing semi-multi-threading for cr2hdr.exe
Post by: engardeknave on December 20, 2013, 08:04:09 PM
That runs cr2hdr on a couple explicitly defined files. If you had tried my script, you see that it runs on a folder containing an unknown number of dual iso and regular photos, converts them to dng, compresses them with adobe dng converter, and sorts them into various subfolders. It also maintains the number of simultaneously running processes at predefined level, and provides a basic framework into which additional command line processes can easily be added. I use it to run Enfuse and Exiftool.

There's also a VB.net application that does cr2hdr processing, has nice a GUI, and almost no configuration necessary. It doesn't seem like there's any reason to find new ways of doing this, but you're certainly welcome to do so and let us know if you come up with something better.
Title: Re: Work-around for accomplishing semi-multi-threading for cr2hdr.exe
Post by: l_d_allan on December 20, 2013, 09:40:06 PM
Quote from: engardeknave on December 20, 2013, 08:04:09 PM
That runs cr2hdr on a couple explicitly defined files.

Not really. It runs on all .cr2 files within the range of ...
131215_189500_*.cr2 thru 131219_190999_*.cr2 ... hundreds of files in this case ... could be thousands. Six threads.

But I would be the first to acknowledge that a .vbs approach has more flexibility and power. And thanks for putting it together. I just haven't had time to get up to speed on it.
Title: Re: Tweaked .. Work-around for accomplishing semi-multi-threading for cr2hdr.exe
Post by: l_d_allan on February 17, 2014, 11:29:46 AM
I've been using the .vbs approach (thanks engardeknave), but also tweaked the .bat approach with the --compress argument to cr2hdr.exe, and also tweaked/simplified the workflow:

S:
cd \_WorkInProgress\Temp
for %%n in (0 1 2 3 4 5 6 7 8 9) do start /belownormal for %%c in (YYMM*_?????%%n_*.cr2) do cr2hdr --compress %%c
pause