Work-around for accomplishing semi-multi-threading for cr2hdr.exe

Started by l_d_allan, December 08, 2013, 06:00:25 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

l_d_allan

My experience is that cr2hdr.exe is rather slow. YMMV.

I've got an i7 with 4x2 cores, including hyper-threading. Generally, I can watch TaskManager monitor CPU usage at 12 to 13% ... which I interpret as the equivalent of one core of the eight. Bummer.

It would be great if cr2hdr.exe was multi-threaded, but it doesn't appear to be. And that's non-trivial to implement.

Here's a work-around for a scenario when you've got a LOT of Dual-ISO files to prep. For example,

  • I came home from a Christmas concert with 600+ images having high contrast, and a LOT of them are Dual-ISO.
  • I have a numbering convention with YYMMDD + 6 digit sequence number
  • Range of 131207r_184250_DI_Description to 131207r_184850_DI_Description (Edit: _DI_ obsolete)
  • for %n in (0 1 2 3 4 5 6 7 8 9) do start /belownormal for %c in (*%n_DI_* .cr2) do cr2hdr --compress %c
  • within .bat file: for %%n in (0 1 2 3 4 5 6 7 8 9) do start /belownormal for %%c in (*%%n_DI_*.cr2) do cr2hdr --compress %%c

That gets the TaskManager CPU monitor showing much greater than 12%, up to 100%, and the files are prepped in much less "wall time".  The /belownormal argument for "start" let the computer remain responsive, rather than sluggish while the CPU grinds away.

(obsolete) You should run Adobe's DngConverter.exe afterwards on the .dng files to accomplish compression ... otherwise the .dng's will be uncompressed and large.

Flawed? Room for improvement?

Edits:

  • A .vbs solution exists that works well, and is significantly advanced compared to this .bat approach
  • See post #6 in this thread for several tweaks/simplifications
  • At the time of the OP, I wasn't aware of the --compress argument to cr2hdr  ... my bad

S:
cd \_WorkInProgress\Temp
for %%n in (0 1 2 3 4 5 6 7 8 9) do start /belownormal for %%c in (YYMM*_?????%%n_*.cr2) do cr2hdr --compress %%c
pause


Audionut


l_d_allan

Thanks for bringing that to my attention. I'll take a look at it. Sorry ... didn't realize I was "re-inventing the wheel" ... my bad.

l_d_allan

Quote from: Audionut on December 08, 2013, 06:30:47 AM
There is a script here for running multiple instances of cr2hdr.

Again, thanks for the info.

Not meaning to be argumentative, but ...
It does seem a bit simpler, with less setup, to use the "embedded for loops" approach from the "vanilla" Windows command line:

> for %n in (0 1 2 3 4 5 6 7 8 9) do start /belownormal for %c in (*%n_DI_*.cr2) do cr2hdr %c

or not? Could be "pride of authorship" on my part?

I suppose a "vanilla" Windows installation can run .vbs scripts without installing anything? I'm mostly ignorant about .vbs.

BTW: I've thought about redirecting the output to >> CrdHr2Log.txt so I could more easily check that no errors showed up with "Find". Haven't tried that yet, however.  Using Adobe's Bridge, a quick check is that there should be the same number of .cr2's as .dng's in the subdirectory with cr2hdr.exe

engardeknave

That runs cr2hdr on a couple explicitly defined files. If you had tried my script, you see that it runs on a folder containing an unknown number of dual iso and regular photos, converts them to dng, compresses them with adobe dng converter, and sorts them into various subfolders. It also maintains the number of simultaneously running processes at predefined level, and provides a basic framework into which additional command line processes can easily be added. I use it to run Enfuse and Exiftool.

There's also a VB.net application that does cr2hdr processing, has nice a GUI, and almost no configuration necessary. It doesn't seem like there's any reason to find new ways of doing this, but you're certainly welcome to do so and let us know if you come up with something better.

l_d_allan

Quote from: engardeknave on December 20, 2013, 08:04:09 PM
That runs cr2hdr on a couple explicitly defined files.

Not really. It runs on all .cr2 files within the range of ...
131215_189500_*.cr2 thru 131219_190999_*.cr2 ... hundreds of files in this case ... could be thousands. Six threads.

But I would be the first to acknowledge that a .vbs approach has more flexibility and power. And thanks for putting it together. I just haven't had time to get up to speed on it.

l_d_allan

I've been using the .vbs approach (thanks engardeknave), but also tweaked the .bat approach with the --compress argument to cr2hdr.exe, and also tweaked/simplified the workflow:

S:
cd \_WorkInProgress\Temp
for %%n in (0 1 2 3 4 5 6 7 8 9) do start /belownormal for %%c in (YYMM*_?????%%n_*.cr2) do cr2hdr --compress %%c
pause



  • Put all the .cr2's in the .\Temp sub-directory
  • Don't worry about separating by which .cr'2 are Dual-ISO and which aren't (change to OP)
  • Run the above .bat file to have 10 processes going in BelowNormal priority, so foreground window stays responsive
  • Move the compressed .dng's to the preferred sub-directory. These were shot with Dual-ISO enabled.
  • Set DngConverter.exe to "Skip source image if destination image already exists"
  • Run DngConverter.exe on all the .cr2's in .\Temp with the preferred destination. This will convert and compress the .cr2's that were not shot with Dual-ISO enabled.
  • FWIW: I archive the .cr2's until the hard drive starts getting full, then delete the older ones.
  • As noted in OP, I use a file naming convention of YYMMDD_######_Description. (change to OP ... no longer have _DI_ "hint")