Dual ISO - massive dynamic range improvement (dual_iso.mo)

[papkee]

I didn't try both of them, but I don't think it can work in this way. Try with the plain raw2dng.

Actually, it DOES work, but only by using the command line. The problem appears to be with dragging and dropping files onto cr2hdr.exe. Whenever I try with DNGs, either from raw2dng or your special one for the 650d it just quickly closes. Any thoughts? I don't want to have to command line convert every one of my 433 frame individually.

[a1ex]

What happens if you drag a small number of files? There may be a limit on command-line length.

[engardeknave]

I don't want to have to command line convert every one of my 433 frame individually.

http://www.magiclantern.fm/forum/index.php?topic=8520.0

[Bassaidai]

I don't want to have to command line convert every one of my 433 frame individually.

You can also drag and drop the CR2 files by packs of tens or twentys. This will fire up multiple instances of the converter and hence drastically speed up the overall conversion progress (on multicore PCs at least...)

[a1ex]

I'm experimenting with two fast versions of cr2hdr:

cr2hdr-fast-halfres.exe: roughly 5x as fast, but it skips all the expensive postprocessing steps used to recover details in highlights and shadows. You will get lots of aliasing with it.

cr2hdr-fast-fullres.exe: roughly 3x as fast; this one has full-resolution detail in midtones and shadows, but also lots of noise. It doesn't seem to recover more than 11 stops of DR.

Neither of these two versions are meant for the final output, but just for getting some quick previews. May be helpful if you shoot say 2000 pictures from which you want to select 50 good ones. They may also be useful for troubleshooting (because the first one minimizes noise and the second one minimizes aliasing).

Both of them are bad in highlights (of course, when compared with edge-directed interpolation).

If a picture comes out clean with the fast-fullres version, it probably doesn't need dual ISO.

[papkee]

What happens if you drag a small number of files? There may be a limit on command-line length.

Even dragging one .dng doesn't work. The program flashes for a second before closing, and the file is unchanged. Could it be something with the way windows 8 interacts with command line programs? I read somewhere that sometimes Windows puts "" around the file path.

[Legushka]

Even dragging one .dng doesn't work. The program flashes for a second before closing, and the file is unchanged. Could it be something with the way windows 8 interacts with command line programs? I read somewhere that sometimes Windows puts "" around the file path.

for windows: install total comander, open folder witch  cr2hdr, press ctrl+enter on cr2hd.exe, press crrl+enter on Raw image, press enter.

[angarsk8]

Hello everyone. I've been following this particular forum thread quite a long time now but I have never read anything related to the usage of ML on a Canon 40d, which is the camera I currently use. I had a canon 600d but it was stolen , so now I only have my 4 years old 40d which is a great camera for stills, has a nice build quiality, fair resolution, fast, etc... I would be pleased to hear that is possible to install ML on a 40d, I would love to use the Dual Iso feature specially because of the poor dynamic range of the 40d that has the same 11 stops of DR of the 600d. thank you very much.

Andrés Garciá Hurtado.

[Walter Schulz]

Please select the proper forum next time ...
And use the search function ...
http://www.magiclantern.fm/forum/index.php?topic=1452.msg78621#msg78621
And read F.A.Q.

[Marsu42]

If a picture comes out clean with the fast-fullres version, it probably doesn't need dual ISO.

A human readable clear text message (maybe in %) from cr2hdr would be nice to let the user know how much a shot profited from being dual_iso... this could also be used as an error code to automatically trash dual_iso shots that were also taken in normal mode. Just a suggestion and if it's very easy to add of course.

[a1ex]

Good idea, will think about it. Not sure how to quantify this though, since "clean" is subjective (so this is a test where you can judge with your own eyes).

[Marsu42]

Good idea, will think about it. Not sure how to quantify this though, since "clean" is subjective (so this is a test where you can judge with your own eyes).

Indeed, if I'd have a way to quantify this I'd have written it :-) ... when in doubt use your judgement, you have looked at more dual_iso shots than anybody else. One certain point for the dual_iso shot though is if there are not only cleaner shadows above a certain ev threshold, but blown highlights in one exposure that can be reconstructed with the other (i.e. not both slices are underexposed).

[a1ex]

It already displays the percentage of reconstructed highlights and deep shadows (these are the areas where it applies the advanced processing). I can't say yet clearly what values are good and what values are bad though.

If the highlight percentage is near zero, the shot was probably underexposed (so consider it shooting at high ISO only). If it's close to 100%, consider shooting it at the low ISO only.

If you enable the intermediate DNGs, you can see the two exposures individually (bright.dng and dark.dng), so you can see exactly where you had recovered highlights and where you had noisy shadows. Maybe I should add a command-line flag for that.

[Marsu42]

If you enable the intermediate DNGs, you can see the two exposures individually (bright.dng and dark.dng), so you can see exactly where you had recovered highlights and where you had noisy shadows. Maybe I should add a command-line flag for that.

I'm all in favor for some command line options rather than patching & recompiling the source, getting both exposures apart to see what was what would certainly help to understand what dual_iso can do and what not, i.e. how to expose correctly in the first place.

[engardeknave]

I think command line flags would make customization and testing a lot easier. Perhaps you could add a destination directory option too.

[Legushka]

"Cr2hdr.exe /?" or "Cr2hdr.exe / help" option does not work.
want to know all the parameters ( Cr2hdr.exe /... ) with which you can run cr2hdr.

[a1ex]

Here you go:

cr2hdr-cmd.exe

Code Select Expand


Command-line usage: cr2hdr [OPTIONS] [FILES]

Shortcuts:
--fast          : disable most postprocessing steps (fast, but low quality)
                  (--mean23, --no-cs, --no-fullres, --no-alias-map, --no-stripe-fix)

Interpolation methods:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--mean23        : average the nearest 2 or 3 pixels of the same color from the Bayer grid (faster)

Chroma smoothing:
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--cs3x3         : apply 3x3 chroma smoothing in noisy and aliased areas
--cs5x5         : apply 5x5 chroma smoothing in noisy and aliased areas
--no-cs         : disable chroma smoothing

Bad pixel handling:
--really-bad-pix: aggressive bad pixel fix, at the expense of detail and aliasing
--no-bad-pix    : disable bad pixel fixing (try it if you shoot stars)
--black-bad-pix : mark all bad pixels as black (for troubleshooting)

Other postprocessing steps:
--no-fullres    : disable full-resolution blending
--no-alias-map  : disable alias map, used to fix aliasing in deep shadows
--no-stripe-fix : disable horizontal stripe fix

Troubleshooting options:
--debug-blend   : save intermediate images used for blending:
    dark.dng        the low-ISO exposure, interpolated
    bright.dng      the high-ISO exposure, interpolated and darkened
    halfres.dng     half-resolution blending (low noise, high aliasing)
    fullres.dng     full-resolution blending (minimal aliasing, high noise)
    *_smooth.dng    images after chroma smoothing
--debug-black   : save intermediate images used for black level subtraction
--debug-amaze   : save AMaZE input and output
--debug-edge    : save debug info from edge-directed interpolation
--debug-alias   : save debug info about the alias map
--iso-curve     : plot the curve fitting results for ISO and black offset (requires octave)
--mix-curve     : plot the curve used for half-res blending (requires octave)
--fullres-curve : plot the curve used for full-res blending (requires octave)


I didn't play much with it, so there may be surprises. Most of these things were already available as compile-time options (except for bad pixel handling, where I've fine-tuned the algorithms and I've added a new option).

[Marsu42]

Here you go

Thanks! ... but what would "octave" be :-o ? Also, I guess a higher cs setting is there for reducing color bleeding on edges, or is there anything else it's good for?

[chris_overseas]

Thanks! ... but what would "octave" be :-o ? Also, I guess a higher cs setting is there for reducing color bleeding on edges, or is there anything else it's good for?

Octave is presumably this: http://www.gnu.org/software/octave/

[Marsu42]

Octave is presumably this: http://www.gnu.org/software/octave/

Oh my, thanks - I'm embarrassed to say this even shows up as the #1 google result, I just didn't expect it to be something like this :-o

[rufustfirefly]

I'm still periodically building win32 and linux 32bit binaries for cr2hdr, available here:

https://bitbucket.org/rufustfirefly/magic-lantern/downloads

in case anyone is interested. I try to build a new version whenever I see a Mercurial check-in related to the cr2hdr code, usually once a day, at most.

[Legushka]

make a separate thread for cr2hdr in bitbucket.org
that it was possible to see the changes by version-)
Thank you very much for the work done.

[a1ex]

https://bitbucket.org/hudson/magic-lantern/commits/all?search=cr2hdr

[a1ex]

I'm experimenting with some noise reduction, with an algorithm along these lines:

http://www.computersdontsee.net/index.php/post/2013/02/09/Introducing-NL-means
http://bengal.missouri.edu/~kes25c/nl2.pdf
http://scholar.harvard.edu/files/stanleychan/files/chan_zickler_lu_2013_0.pdf
http://www.cs.tut.fi/~foi/3D-DFT/ - watch the animation




According to the logs, this squeezes roughly one more stop of dynamic range, and it seems to whiten the noise too.

Download: cr2hdr-denoice.exe

The denoising step is not enabled by default; you need to specify --denoice in the command line.

Now the big question: how does this compare with state-of-art denoising software? (Lightroom, Neat Image, Noise Ninja, whatever you use). I don't have license for any of these, so I don't know if I'm reinventing the wheel or not.

For comparison, I suggest processing the CR2 with default options, then with --denoice, then postprocessing both DNGs in your favorite editor and denoise them so they have similar noise levels, and see which one holds more detail. Your quest is to find out whether my denoising step helps or hurts, and how much.

Original CR2 files for these examples: 7O4C8804.CR2, 6X8A0381.CR2 and IMG_5762.CR2 (credits: AdamTheBoy, daancalo2013 and Danne). To get the DNGs from the top row, process them with default settingss, and to get the DNGs from the bottom row, process them with --denoice.

Please note that each test run will create a different DNG (the denoising algorithm uses monte carlo sampling). It may be interesting to see what happens if you average the output from a few test runs (didn't try). Also it may be worth trying --denoice --cs5x5.

[Legushka]

jpeg (-x) + jpeg (0) + jpeg (+ x) -> hdr, where x-expo
cr2 (iso100/1600) -> dng

whether it is possible to do so in cr2hdr: cr2 (iso100/1600) -> jpeg (-x) +{ jpeg (0) +} jpeg (+ x)?