Dual ISO - massive dynamic range improvement (dual_iso.mo)

[Marsu42]

It looks at the optical black area (see dcraw -4 -E) and computes the noise levels from there, assuming that's the read noise. [...] Note the stdev is heavily influenced by hot pixels. Maybe I should replace it with some robust statistic.

So basically, this means the dr given by cr2hdr doesn't really reflect the actual dr of the image. No big deal as the picture itself is important, but it's a puzzler b/c the read noise spec from sensorgen of the 7d @ iso 100 is way lower than the 60d, but in cr2hdr it's the other way round. The 7d is said to have more "pattern" noise, maybe that is an influence on the stdev, too, like hot pixels?

Btw: What's up with the 20bit version, I'm using that one for ages, but it isn't merged. Was the reason that it's a lot slower but doesn't show reasonable iq gain from the 16bit version, or are there any other problems left with 20bit?

[a1ex]

Yes, pattern noise increases the stdev as well. Do you have some sample images to show the difference?

The 20-bit version is merged.

Are you asking about those few changes from here? These are not merged because they didn't pass my test suite.

[Marsu42]

Yes, pattern noise increases the stdev as well. Do you have some sample images to show the difference?

Well, I can send you two random 100/800 shots from the 60d and 7d1 that show the different "theoretical dr" line. But as my 60d is currently broken (I'll have someone look at the fuses, but that'll take time) I can only give you real "requested" test images from the 7d1.

What would be interesting to know which one of all the crop sensors before the 80d is how good for dual_iso. Sure, maybe the difference is minor and maybe 0,5ev, but if someone like me really wants Canon crop with enhanced dr and doesn't really care which old model to buy, it might matter.

Again, only looking at the sensorgen/dxo data the 7d1 should do best... even though it has a bad rep b/c of the pattern noise (which is on 5d2, too). I haven't tested how much the pattern noise affects dual_iso and the amaze algorithm, as it only appears visibly after pushing the shadows a *lot* which shouldn't be necessary using dual_iso.

The 20-bit version is merged.

Wupps, I missed that :-o

Are you asking about those few changes from here? These are not merged because they didn't pass my test suite.

I didn't try these either, but I'm keeping all my original non-cr2hdr'ed shots around in case some improvements are made :-)

[chris88]

Good evening,
I did not read 115 pages, but today I played with the Dual-ISO feature and got an interesting banding issue when using the new 20bit conversion.

Here the result with the classic cr2hdr:


Here the result when converting with the cr2hdr-20bit :



My question now is: Why do I get such a bad result with the 20bit converter?

Best regards,
Chris

[Danne]

20bit is merged into a unified cr2hdr. Could you tell us what build you are using and what program used to convert? Upload the original footage?

[chris88]

Sure,
I used the version of cr2hdr from here:

http://www.magiclantern.fm/forum/index.php?topic=11056.0

So the setup was:
Lightroom Dual ISO Converter 2.1
Lightroom Dual ISO Converter (cr2hdr) plugin (v3.0 BETA3).
Adobe Lightroom CC Version 2015.7

In the plugin I have chosen (via the export dialog):
1) For the first picture "cr2hdr (classic)"
2) For the second picture "cr2hdr-20bit (Experimental)"

The original footage can be found here:
http://www.file-upload.net/download-12038109/IMG_5983.CR2.html

I had a lot of pictures, but as the other conversions were all OK, I saw the banding issue with the 20bit version in this picture. So I thought this report might be helpful in development.

[Walter Schulz]

You are using an outdated plug-in. Newest one does not have a 20-bit option (because 20bit version is the only one supported after merging).
Most likely you are using outdated cr2hdr binaries, too.
Visit https://bitbucket.org/kichetof/lr_cr2hdr/downloads and download cr2hdr.3.0BETA3.3.lrplugin.zip

[chris88]

Oh, thank you for this information,
I only found the thread mentioned, but did not know that there is a newer version of the plugin available. With the supplied cr2hdr (in the bin directory) [Version 1779727 from 2015-09-30] all is working well. But I do not know if it is now 20bit or not, because there is no option anymore to choose from.

By the way: I also cannot compress the DNG file. Is this normal that I can not choose this from the options? (grayed out)

[Walter Schulz]

As said before: There is no other version. 20bit version has been merged/unified and is the only one around.
And if you are using LR you don't have to compress the file. LR will do it after changing metadata (we do not know why LR does it and if is unintended or not). It's lossless compression, however. No lossy compression with LR available.

[chris88]

Perfect. Thank you so much for your help. I think hereby all my questions are answered. Maybe there is a possibility to link to the newest version, because searching for DualIso will lead you to the thread where the old version is linked.

So, thanks again.
Dankeschön Walter :-)

[Danne]

If on mac there,s a program I started building here if you want to try it out feel free to do so.
https://bitbucket.org/Dannephoto/cr2hdr

Forum thread
http://www.magiclantern.fm/forum/index.php?topic=15108.msg146822#msg146822

Regarding compressing options they require you install adobe dng converter.

[chris88]

Thanks Danne for your suggestion. Unfortunately I do not have a MA. But maybe this will be useful one day...

[mosaster]

Is it available for Canon 5D classic?
As the first post reads "most cameras" that does not mean all Canon cameras, right? What about the old 5d?

[keepersdungeon]

Is it available for Canon 5D classic?
As the first post reads "most cameras" that does not mean all Canon cameras, right? What about the old 5d?

It's no longer maintained but there's a working build available here check it out
http://builds.magiclantern.fm

Ps: Not sure if the dualiso module works on it though.

[Johannes]

Hey Guys!

is there a special way to convert the CR2s?

I'm not sure if somebody mentioned to use the cr2hdrConverter + some extra software to get better colors etc..

[Danne]

Reply #2860 shouldn,t be too far away?

[A Dun]

When I export photos with cr2hdr, the file becomes 44M.What information is added？Interpolated pixels?

[A Dun]

If you use dual ISO without cr2hdr decoding can get 16bit uncompressed photos,why is the photos using dual ISO taken as the same size as the photos taken without dual ISO?Does it say that the increase in bit depth does not cause an increase in the size of the photo?Or increase the value of bit depth equal to the loss of pixels?

[Walter Schulz]

Dual-ISO is 14 bit digital output.
Pixel interpolation is part of de-mosaicing process and not applied to RAW/DNG.

Canon's CR2 files do have losless compression. Therefore you will have smaller file sizes capturing uniform scenes (fog) compared to scenes where compression fails (sand).
ML Dual-ISO DNG does not do compression. Therefore you will get exactly the same file sizes because every single sensor pixel data value is written to file.

Devs are experimenting with losless DNG compression, though. 

[A Dun]

Thanks!

Now that Dual-ISO is14bit digital output,why use cr2hdr export images, the image becomes 16bit?Is this 16bit interpolated by cr2hdr?

[Walter Schulz]

Windows, cr2hdr.exe and wildcards ...
Maybe a way to enable wildcards running GCC (IANAP):  https://gcc.gnu.org/ml/gcc-patches/2017-01/msg01299.html

A1ex, is this a solution or is it necessary to compile sources using Visual Studio?

[a1ex]

Looks good to me. No idea whether Visual Studio libraries offer a similar workaround.

[emayd]

What I am doing wrong?

CR2 before processing:

Crop


After processing in cr2hdr


Log:

cr2hdr: a post processing tool for Dual ISO images

Last update: ab1e90c on 2015-11-24 09:50:19 UTC by a1ex:
cr2hdr: moved safeguard from median_int_wirth to kth_smallest_i...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (default)

Input file      : DUAL0022.CR2
Camera          : Canon EOS 5D Mark III
Full size       : 5920 x 3950
Active area     : 5796 x 3870
Black borders   : 124 left, 80 top
Black level     : 2046
ISO pattern     : dBBd RGGB
White levels    : 15154 13824
Noise levels    : 7.87 35.56 34.86 7.83 (14-bit)
ISO difference  : 5.92 EV (6038)
Black delta     : 1.76
Black adjust    : -6
Dynamic range   : 10.71 (+) 8.37 => 14.29 EV (in theory)
Looking for hot/cold pixels...
Hot pixels      : 81
Cold pixels     : 10476
AMaZE interpolation ...
Amaze took 5.14 s
Edge-directed interpolation...
Semi-overexposed: 3.07%
Deep shadows    : 96.18%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 3.0 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 30.61 (20-bit), ideally 30.60
Dynamic range   : 14.74 EV (cooked)
Black adjust    : 2
AsShotNeutral   : 0.57 1 0.55, 4308K/g=1.09 (gray max)
Output file     : DUAL0022.DNG (already exists, overwriting)
DUAL0022.DNG    : copying EXIF from DUAL0022.CR2

[dmilligan]

Doesn't look like you are doing anything wrong.

[emayd]

Doesn't look like you are doing anything wrong.

Here is a link to CR2 file - http://download.maydanik.de/DUAL0022.CR2

if everything right, why the final result too dark?

seems something wrong with bright extraction. Here is snapshot of bright.dng after processing in debug mode.