Dual ISO - massive dynamic range improvement (dual_iso.mo)

[feureau]

Oooh! Thanks!

I'll play with the DNG input.

Btw, I played with the cr2hdr-hot and cr2hdr-denoice to batch convert about 100 files, and it keeps crashing after converting about 40 files give or take a few.

[a1ex]

Same with the one from the first post?

[escho]

I'll take a closer look at the denoising algorithm somewhen; noticed that darktable uses the same algorithm, so I'll have a look at how they have fine-tuned it. Nice read: http://www.darktable.org/2012/12/profiling-sensor-and-photon-noise/

The profiled denoising in darktable is a good thing. The fine-tuning is done with user-measured curves, I think. But this kind of denoising does not fit to all kind of pictures. For my moon pics for example I cannot use it.Too much details go and too much artifacts come in. But for most of the other pics, it´s a great thing as I said before.

Edgar

[a1ex]

Can you upload one so I can use it as a test shot?

[feureau]

Same with the one from the first post?

Using the one I got from this one:

Here's a version that should be pretty good for everyday use.

cr2hdr-hot.exe

What's new:
- better hot pixel handling (solves the shot from Marsu42 with --really-bad-pix, also fireworks are no longer mistaken for bad pixels)
- fixed a bug that caused massive shadow aliasing in the version with many command-line switches
- minor fixes

What's not:
- I've removed the denoising option for now, because DxO puts it to shame. I'll add it back when I'll get comparable results in terms of recovered detail, without the plastic look.

Have a quick play with it; if there are no major bugs (like stopping after 30 pictures, crashing, or regressions in image quality) I'll update the first post tonight. I've tested it on 700 pics in a single batch with --fast, 100 pics with default settings, and 2 pictures in valgrind, so I believe it should be pretty solid now.

Update: after running my usual tests, I've noticed the new hot pixel algorithm (in aggressive mode) has little or no effect on resolution charts. Another milestone completed!

Also, I tried putting in the DNG to cr2hdr but it won't spit out any result. I was just wondering what it'd look like if I use DxO for prime denoising then exporting as DNG and putting that DNG into cr2hdr.

[a1ex]

Oh well... it must be 4 a.m. where you live... (I've asked something, you answered something else).

For DNG, I convert CR2 files with Adobe DNG Converter, and cr2hdr still accepts them.

[feureau]

Oh well... it must be 4 a.m. where you live... (I've asked something, you answered something else).

For DNG, I convert CR2 files with Adobe DNG Converter, and cr2hdr still accepts them.

*checks the clock*

Dammit. I should get some sleep now. Spot on on the time zone. Let me brain up when I get up this afternoon.

Let me try Adobe-ing that dxo DNGs.

Thanks.

[Walter Schulz]

SCNR!

Let me try Adobe-ing that dxo DNGs.

Made my day!

Ciao
Walter

[a1ex]

Note that:
1) dxo dng's are linear (demosaiced); cr2hdr only works on bayer data
2) denoising must be applied after "deinterlacing", not before (try it with pen and paper on a 3x3 kernel if you don't believe)

[feureau]

Same with the one from the first post?

Oh well... it must be 4 a.m. where you live... (I've asked something, you answered something else).

For DNG, I convert CR2 files with Adobe DNG Converter, and cr2hdr still accepts them.

Okay, I brained up. Not sure which first post you're referring to though, but I'm not running the pre-built from first post. I'm using the ones you posted: the cr2hdr-denoice.exe, and cr2hdr-hot.exe Drag and dropping about 166 cr2 files on it. It will run like normal but after about 20-40 files, it will just stop. On the cr2hdr-denoice, it will look like it's doing something, but it won't progress, and if I open the task manager, the CPU is at 0. On the cr2hdr-hot, it will just crash and give out that window where windows ask if you want to wait or just close the program.

Before this, I've been running the cr2hdr-exp and it runs with no problem at all. (all this on win 7 x64)

[a1ex]

I've asked you to try it and tell me if it has the same problem. Also check post #1620.

[l_d_allan]

the cr2hdr-hot.exe work pretty good but if you drag more than 40 pics in one single click it will crash somewhere around picture 40;

Sorry to be slow participating in this thread.

Win-7-64

I've had intermittent cr2hdr.exe crashes when dragging lots of .cr2's onto cr2hdr.exe. It wasn't repeatable enough to report .. (sorry?). I didn't notice whether it had anything to do with 40 or so files. There was info from APPCRASH I could watch for, and send screen-shots of nitty-gritty details, if that would help.

I haven't seen this happen when I launch from the command line, such as:
for %c in (*.cr2) do cr2hdr %c
or
for %n in (1895 1896 1897 1898 1899 1900) do start for %c in (*%n*.cr2) do cr2hdr %c

My speculation is that using a command line, there might be more information showing up that didn't disappear when the APPCRASH dialog window was dismissed. Or not?

[a1ex]

Make sure you use the latest binary from the first post. The ones posted in the thread are just experiments to get some quick feedback, but the one from the first post is more or less stable (and it's also the latest one). The only thing missing is denoice.

[escho]

Can you upload one so I can use it as a test shot?

Is a 16bit png ok or do you need a cr2 (don´t know, where to upload cr2)?

Edgar

[a1ex]

16-bit pgm is OK only if you got it by running dcraw -4 -E. You can upload on Dropbox or whatever other file sharing service.

[escho]

Find a solution for CR2: IMG_6550.CR2

That`s one on my not so good moon pics, captured with ISO1600 (don´t know why  ) With darktable and the profiled denoising too much details get lost and the picture looks not natural.

Edgar

[a1ex]

Got it, there's very little noise on the moon (only on the dark areas). Do you also have the JPG as a reference image of what not to do?

[escho]

Here you are:

profiled denoising: img_0001_01.jpg not natural
bilinear filter: img_0001.jpg some little noise, but more natural

Both versions are made with darktable filters "dead pixel" and "chromatic abberation". I made the black a little more black to make the space free of noise.

Edgar

[feureau]

I've asked you to try it and tell me if it has the same problem. Also check post #1620.

Welp. I downloaded the one from the first post yesterday, and I've been cr2hdr-ing hundreds of photos from two different cameras, and so far, no crashes at all. Thanks for this.

Didn't try the DNG compression tag thingy though. How do I use that without having to command line each files?

[a1ex]

On Linux I use cr2hdr --compress *.CR2

It doesn't work on Windows on plain command prompt (it might work on cygwin or msys shell, didn't try).

[mgrant]

I read through quite a bit of this thread which is 67 pages as of my writing.  Is the version of cr2hdr linked from the first post the most current/recommended version to use?  Please could you (a1ex) update the first post if so or if not or add links to the other useful versions?  Thanks.

Perhaps a stupid question... Will it be possible at some point to do this post processing in the camera?  Or is that just not technically feasible?

I use lightroom.  Is there an easy way to incorporate this cr2hdr step into a workflow without having to go to the command line?

Thanks!

[l_d_allan]

Is the version of cr2hdr linked from the first post the most current/recommended version to use?

Here's the link I use:
https://bitbucket.org/a_d_/magic-lantern/downloads

There is a date-stamp to indicate whether the cr2hdr file for Mac or Windows has been refreshed since the last time you downloaded it.

Note that .zip file won't necessarily have the other needed files: dcraw.exe, exiftool.exe, readme.txt

The other randomly named files may be for modules. Or not?

[mgrant]

I'm having trouble where the processed images is actually darker.  These were taken at 100/3200:

interlaced image:


cr2hdr processed image:


C:\Users\Michael\Pictures\2013\2013-12-27>cr2hdr DUAL3071.CR2
cr2hdr: a post processing tool for Dual ISO images

Last update: 2f4e2df on 2013-12-02 17:45:36 UTC by alex:
cr2hdr: fix DNG compression under Windows

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default
)

Input file      : DUAL3071.CR2
Camera          : Canon EOS 7D
Full size       : 5360 x 3516
Active area     : 5202 x 3465
White level     : 12500
Black borders   : 158 left, 51 top
Black level     : 1955
ISO pattern     : dBBd GBRG
Noise levels    : 5.47 33.98 39.40 5.28 (14-bit)
ISO difference  : 5.09 EV (3402)
Black delta     : 9.28
Dynamic range   : 10.96 (+) 8.06 => 13.15 EV (in theory)
Looking for hot/cold pixels...
Hot pixels      : 7
Cold pixels     : 27592
AMaZE interpolation ...
Amaze took 5.49 s
Edge-directed interpolation...
Semi-overexposed: 6.46%
Deep shadows    : 90.22%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 3.0 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 3.72 (16-bit), ideally 3.68
Dynamic range   : 13.47 EV (cooked)
Black adjust    : 0
Output file     : DUAL3071.DNG
    1 image files updated

The raw file is here: https://dl.dropboxusercontent.com/u/1700402/DUAL3071.CR2

What are you all doing to get these amazing images?  I can brighten up the shadow in lightroom but it gets really grainy.  I feel I am doing something wrong.

[tron]

First, I increase exposure while paying attention not to clip highlights. Then I increase shadows. But 7D is noisier than the 5D2 I experiment with...

[Luiz Roberto dos Santos]

taken at 100/3200

This is too extreme settings. The actual maximum ISO your camera is 1600, do not pass it, otherwise will have images with a lot of noise. I would recommend you to use a maximum of 100/800. Above that, it is not worth using dual_iso.

What are you all doing to get these amazing images?  I can brighten up the shadow in lightroom but it gets really grainy.  I feel I am doing something wrong.

This darkness in the image is caused by the linearization of gamma. Try increasing the exposure and then down the highlights... will leave with much noise, but will give to correct. I recommend you also use the profile VisionColor, the VisionLog, can recover enough without giving too much noise.
To remove the noise, an alternative is the PRIME noise reduction, of DxO (is really awesome, try). To sharpen, not use unsharp, use deconvolution.