Dual ISO - massive dynamic range improvement (dual_iso.mo)

[1%]

I'm using auto ettr always on so it picks when dual ISO happens. Maybe if I combined with the auto expo module it wouldn't do this but then the 2 tend to fight.

[1%]

wat!



Pre-built from 1st post? Latest commits touch this?

[a1ex]

Fixed, long live Microsoft.

[DeafEyeJedi]

Small update to cr2hdr:
- fixed cold pixel handling
- experimental DNG compression (you need to install Adobe DNG Converter; should be autodetected)

Code: [Select]

DNG compression (requires Adobe DNG Converter):
--compress      : Lossless DNG compression
--compress-lossy: Lossy DNG compression (be careful, may destroy shadow detail)

Download it from first post. It works pretty good for me (currently running it on a few thousands of pictures), but I'm still waiting for a detailed bug report for post #1602 (I could not reproduce it).

Also, auto ETTR should do slightly better choices when linked to dual ISO:
- dual ISO is only enabled by ETTR if the DR improvement is at least 1 stop (so it should no longer choose 100/200, for example)
- if the calculations result in something like 200/800, it will switch to 100/800 (better highlight protection at very small cost)

Hello A1ex -- hope you enjoyed your Thanksgiving weekend. I wanted to follow up regarding your recent update to your cr2hdr and I clicked on the Mac version link that is on the first post (according to your statement) and apparently the link is dead as of now.

Perhaps is the reason because of the recent bug regarding the black area losing resolution by half.

Should I stick with beta version .13?

Thanks!

[a1ex]

I don't maintain the Mac version, only the source code and the windows exe. But you can run it under Wine.

[swinxx]

An updated mac version would be really great. Thx

[1%]

Heh, for some reason it doesn't work in batch with barracuda + compress. works by itself tho

[diogo.fagundes]

I'm having some trouble here with the cr2hdr.exe on Mac ...I have recorded a video with the hdr function and in Raw mode, then I take the raw file and use RAWMagic to transform it in .dng folders. When i drag the dng folder to the cr2hdr.exe the program starts to erase the dng files... and it appears a message saying "Doesn't look like interlaced ISO" (but in the preview of the computer i can see that the files are with diferent ISOs)  ... and when I rename the .dng to .CR2 and than drag to the cr2hdr.exe it appears an .txt file saying the same thing... what I'm doing wrong?

[dmilligan]

...I have recorded a video with the hdr function and in Raw mode

Sounds like maybe you are confusing the HDR mode with the dual ISO mode, they are different things.

[diogo.fagundes]

yea.. for sure... only now i understood what is the dual iso.. but, so where is the dual iso submenu? ... i'm not finding it, and i have already turned the dual_iso mode on, and restarted the camera...

Edited: ok, I already found it.... it is on the topic expo of the ml menu... then dual iso...

[Marsu42]

My current main issue with the otherwise great dual_iso shots is the white balance, or to be exact the lack of.

It's not so much that cr2hdr doesn't copy the Canon (a)wb from the cr2 to the dng, but that after manually setting the *same* wb on the same shot, one normal, one dual_iso, the result isn't the same which esp. seems to result in a tint (cyan/magenta) cast.

I haven't done enough comparison shots of the same scene and will buy a color calibrator for my monitor to exclude this problem source - but does anyone experience the same problem? I guess I have to shoot a grey/color card more often with dual_iso for calibration...

[ItsMeLenny]

My current main issue with the otherwise great dual_iso shots is the white balance, or to be exact the lack of.

It's not so much that cr2hdr doesn't copy the Canon (a)wb from the cr2 to the dng, but that after manually setting the *same* wb on the same shot, one normal, one dual_iso, the result isn't the same which esp. seems to result in a tint (cyan/magenta) cast.

I haven't done enough comparison shots of the same scene and will buy a color calibrator for my monitor to exclude this problem source - but does anyone experience the same problem? I guess I have to shoot a grey/color card more often with dual_iso for calibration...

The trick would be finding out which ISO is best to white balance off for the dual ISO.
The low iso, the high iso, or an iso directly between?

[a1ex]

I think it's an exif issue. Can you try if this helps?

- convert the CR2 to DNG with Adobe DNG (lossless)
- process the DNG with cr2hdr

I always set WB manually, so I can say I don't have this problem

If it's daylight, it's easy (something around 5500 in sunlight, 6500 in shadow). A little magenta boost when shooting grass with older lenses. Indoors, I found balancing on something white a bit too harsh (I prefer to go a little warmer from there). Sometimes I get much better results if I do the white balance directly on skintones.

I have a small modification to ufraw that lets me do spot WB on skintones; if anyone is interested I can share it.

[budafilms]

Share it!

[a1ex]

Alright, see here: http://www.magiclantern.fm/forum/index.php?topic=9469

[Marsu42]

The trick would be finding out which ISO is best to white balance off for the dual ISO.
The low iso, the high iso, or an iso directly between?

Yes, you're correct, that might be it - I always copied the wb setting from the lower iso setting (usually 100) and haven't tried how much it changes when going to the upper (800 or 1600) value.

I hope that it's not that the wb of the dual_iso shots is completely screwed up because cr2hdr doesn't consistently merge the shots, resulting in different wb areas and requiring local correction in post?

[a1ex]

cr2hdr only does linear processing, so it shouldn't alter colors; it should look exactly like a ISO 100 image, except for noise.

Does WB stay there if you take a regular CR2 and convert it to DNG with Adobe converter?

[Marsu42]

Does WB stay there if you take a regular CR2 and convert it to DNG with Adobe converter?

No, I didn't try yet, will do and report back in the evening :-) ... that's supposed to copy the camera (a)wb correctly?

I think it's an exif issue. Can you try if this helps?
- convert the CR2 to DNG with Adobe DNG (lossless)
- process the DNG with cr2hdr

Not to be misunderstood: My problem also occurs after *manually* setting the wb, cannot seem to find the correct setting as there seems to be a color cost somewhere at any wb value

I really hope I'm not making this up and causing trouble... the problems I think I'm experiencing are slight mind you, but there seems to be a magenta/cyan cast in pastel autumn light beige/grey/green autumn colors. I really have to get that monitor calibration and take more dual_iso and normal shots to isolate the problem, but it's such a hassle to quickly switch because with the current implementation I have to set another ec and fec to get from dual_iso to normal and back.

[a1ex]

If you do some tests, I think it's best to stay in M. Fix some exposure settings, set ISO 100 from Canon menu, set some recovery ISO from ML (800, 1600, whatever), enable "alternate frames only" and fire away. All shots should look identical in colors and exposure if you use the same parameters for developing.

[Marsu42]

Here's a broken dual_iso shot, the current cr2hdr produces a yellow overlay on the left side:

https://bitbucket.org/Marsu42/ml-mod60d/downloads/DUAL6363.CR2

I hope it's a cr2hdr problem and not dual_iso not working on the 6d properly or even my hardware :-o though I don't think the latter is the reason, this is the first time I experience this problem. Alex not currently committing code doesn't mean it's impossible to *code* an updated cr2hdr version without *committing* the code, correct :-> ?

[a1ex]

This is caused by blooming (strong overexposure near the edge => the bright sun overflowed onto the optical black area). You can see the effect with dcraw -E DUAL6363.CR2.

Workaround for this shot: comment out the processing of the top black bar (around line 729).

[Marsu42]

This is caused by blooming (strong overexposure near the edge => the bright sun overflowed onto the optical black area). You can see the effect with dcraw -E DUAL6363.CR2.

Ok, thanks for the diagnosis, I would never have though of this... if this is what can happen and since dual_iso is bound to be used in high contrast (i.e. sunrise/sunset) shots - would this justify adding an command line switch to fix these shots?

[a1ex]

On the left bar I simply went for ignoring some border pixels. But you are right, disabling black correction (either completely or just on one of the bars) can be helpful for troubleshooting.

[feureau]

Here's a version that should be pretty good for everyday use.

cr2hdr-hot.exe

What's new:
- better hot pixel handling (solves the shot from Marsu42 with --really-bad-pix, also fireworks are no longer mistaken for bad pixels)
- fixed a bug that caused massive shadow aliasing in the version with many command-line switches
- minor fixes

What's not:
- I've removed the denoising option for now, because DxO puts it to shame. I'll add it back when I'll get comparable results in terms of recovered detail, without the plastic look.

Have a quick play with it; if there are no major bugs (like stopping after 30 pictures, crashing, or regressions in image quality) I'll update the first post tonight. I've tested it on 700 pics in a single batch with --fast, 100 pics with default settings, and 2 pictures in valgrind, so I believe it should be pretty solid now.

Update: after running my usual tests, I've noticed the new hot pixel algorithm (in aggressive mode) has little or no effect on resolution charts. Another milestone completed!

Since DxO won't read DNG, is it possible to either

1. Put the denoising option back

2. Have a version of CR2HDR that can read DNG for its inputs so that, using DxO, we can denoise the CR2 first, and export it as DNG for HDR conversion?

[a1ex]

I'll take a closer look at the denoising algorithm somewhen; noticed that darktable uses the same algorithm, so I'll have a look at how they have fine-tuned it. Nice read: http://www.darktable.org/2012/12/profiling-sensor-and-photon-noise/

The current version already reads DNG files, but I'm not sure what you want to do with this.