Dual ISO - massive dynamic range improvement (dual_iso.mo)

Started by a1ex, July 16, 2013, 06:33:50 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

IliasG

I  am on win-vista32 bit, is your result on linux ?. did you checked the dng that I uploaded ?.

a1ex

Time for some pixel peeping - how dual ISO postprocessing (cr2hdr) evolved from the first public release until today?



More samples here: https://www.dropbox.com/sh/sta87fj2pghv6o4/vPXNvbhZdJ

(when reviewing them, I realized the 20-bit improvement is a lot smaller than I thought)


a1ex

If your evaluation criteria is only the aliasing in that particular test chart, yes.

ted ramasola

I looked at the group of 8 and pretty much eliminated the left group of 4. Among the right group of 4 i looked at aliasing and how the smoothing was already at the point of removing parts of detail. And also which handled color noise best. And also "color smear", for lack of a better term. Greenish noise is "slightly" more obvious in latest and previous compared to amze-edge.
The left group of 4 had FPN clearly.
5DmkII  / 7D
www.ramasolaproductions.com
Texas

Marsu42

Quote from: a1ex on March 26, 2014, 09:53:10 PM
Time for some pixel peeping - how dual ISO postprocessing (cr2hdr) evolved from the first public release until today?

On a subjective scale and w/o comparing pixels, I'd say the latest cr2hdr versions can deal even with my most oddly exposed shots and show amazing ability to raise shadow detail out of nowhere - well done!

The wb green/magenta problem persists atm as the latest exif copy attempt doesn't really work for me, but I'm used to find some reasonable wb settings by now manually and use the acr wb brush to remove tint casts. My one possible worry is that the 6d is somehow a bit different than 5d3 as I seem to be much more concerned about this, but then again maybe I'm just shooting different scenes.

Quote from: a1ex on March 26, 2014, 09:53:10 PM(when reviewing them, I realized the 20-bit improvement is a lot smaller than I thought)

Never mind, 20bit sounds cool, and even if it only helps a bit what is all that multi-core multi-gb ram stuff inside my laptop for :-p ... I don't even play games anymore!

Luiz Roberto dos Santos

Quote from: a1ex on March 26, 2014, 07:37:31 AM
What could be more efficient than a batch process that you start, go out and have a beer, come back in a few hours and find everything almost done?

I will test it  ;D

Quote from: a1ex on March 26, 2014, 07:37:31 AM
Lossless DNG: interesting idea (some sort of a raw pre-cooker). This could include many other things like FPN correction, deflicker (via the soft-film curve), highlight recovery, bad pixel fix, raw operations like average/median/max and so on.

Yeah, I don't thought on first moment for this way, but this is a really good idea.

Quote from: a1ex on March 26, 2014, 07:37:31 AM
BTW, you can hack the cr2hdr source to accept regular cr2's (just trick it into believing it's a dual ISO file, let it accept ISO 100/100, enter some valid line configuration, like 0,1,1,0, and optionally skip all the processing and jump to output).

I will do this, I find it very useful.

Quote from: a1ex on March 26, 2014, 09:53:10 PM
Time for some pixel peeping - how dual ISO postprocessing (cr2hdr) evolved from the first public release until today?

Is not possible do the SSIM (structural similarity) analysis? As "original source", it would be possible to use supersampling/super-resolution, with various pictures, as in the case PhotoAcute.
That way we do not fall into the subjectivity...

Marsu42

Quote from: Luiz Roberto dos Santos on March 26, 2014, 11:14:16 PM
Yeah, I don't thought on first moment for this way, but this is a really good idea.

Btw I'd like to +1 this - dealing with high dr shots is a pita in current acr, the highlight recovery isn't enough so you need to apply a custom tone curve which is a big hassle.

For some shots, I'm currently experimenting with --soft-film even though the "baked in" wb doesn't sound too appealing. But if it really has no (other) drawbacks, it might be a good solution for high dr non dual_iso shots, too, at least until Adobe can handle these shots as DxO can.

a1ex

Quote from: Luiz Roberto dos Santos on March 26, 2014, 11:14:16 PM
Is not possible do the SSIM (structural similarity) analysis? As "original source", it would be possible to use supersampling/super-resolution, with various pictures, as in the case PhotoAcute.
That way we do not fall into the subjectivity...

You can generate ground-truth images from two bracketed shots (lookup fake_dual_iso earlier in this thread). Maybe combine them with CeroNoice to get a noise-free estimation.

Marsu42

Regression: After running 20bit over my collection 2/3 of the shots are broken with Lightroom. This is NOT visible when viewing the dng with some other viewer like XnView, but in acr it's about +1ev brighter than legacy 16bit and has a somehow screwed tone curve. I'm sending alex a problem sample, but anyone beware.

Audionut

I've noticed LR having a problem also.  ACR reads them fine.  Since it's intermittent, I haven't had a chance to do sufficient testing.

Marsu42

Quote from: Audionut on March 28, 2014, 04:12:44 PM
I've noticed LR having a problem also.  ACR reads them fine.  Since it's intermittent, I haven't had a chance to do sufficient testing.

I'm really puzzled atm - but it also might be that the 16bit version was broken all along, which means I have to re-process all of my 1500 dual_iso shots. Yippee.

Audionut

I haven't noticed any problems with compatibility, with the 16bit build.  Only the latest 20bit source.

Marsu42

Quote from: Audionut on March 28, 2014, 04:34:01 PM
I haven't noticed any problems with compatibility, with the 16bit build.  Only the latest 20bit source.

I usually use 100/800 and underexpose for highlights, and I always had to add heavy +ec in Lightroom to get decent results (and of course adjust the broken wb). It might be a 6d specific problem... doh. The results are also ok after re-adjusting lr for the latest 20bit, but a couple of days work what you get for using bleeding edge software on an unsupported model :-p

Audionut

Honestly, I think the biggest problem, is that you may simply be underexposing to much.

dual_iso does not fix shot noise.  If you need clean shadows, it's not simply a case of increasing ISO to remove electronic noise, you also have to expose sufficiently also.  Exposure is controlled solely with lens diameter, shutter, and aperture.

When you attempt to capture highlights, that you would otherwise simply blow to white, without dual_iso, you're more likely to lower exposure to capture these highlights.  Here, you are pushing the midtones/shadows further into shot noise dominance.  Note:  Generally, this situation would be pushing the exposure into read noise dominance, but since dual_iso lowers the read noise significantly, shot noise becomes more relevant.

If you're underexposing these highlights, that you would otherwise blow to white, then the situation becomes worse.  You've underexposed your midtones with dual_iso (to capture the highlights), and then you are underexposing even further.

You almost always have to increase exposure in LR, because you are underexposing your midtones with dual_iso.
Lets me try and show an example.  Let's say that this is our scene lighting.

+------+------+

Lets say that the above represents 14 EV of dynamic range.  The middle + represents the midtones, and these midtones are 7 EV below the highlights.  Remember, normal ISO is rated for around 11 EV of DR, and this is at the noise floor of the camera.  The DR capability of the camera, where noise levels are deemed to be significant, is actually closer to 9 EV of DR.
So in normal circumstances, you might expose as such.

+------+---+

In other words, you sacrifice 2-3 stops of highlight detail, to expose your midtones correctly.  In raw ADU's, the midtones are around 4 EV from sensor saturation (overexposure). 

So where we use dual_iso to capture these highlights (that would otherwise be overexposed), the midtone point has been shifted 2-3 stops lower (instead of being 4 EV below sensor saturation, it is now 6-7 EV below sensor saturation).  Hence, in PP, you need to increase exposure 2-3 stops to compensate.  This isn't a deficiency with dual_iso, it is an expected outcome.  a1ex's soft-film highlight compression can be used to automatically compress these highlights, so that the midtone point of your exposure (the rendered image), is more closely aligned with the midtone point of a standard Canon image.

Here, you might not need to increase the exposure in PP, because soft-film is applying an exposure correction automatically.  I will agree though, that in it's current state, soft-film is difficult to use, because it requires manual intervention regarding the burned WB values.  a1ex has mentioned that he is interested in using WB processing, to automatically determine the "correct" value for soft-film use.

Remember, raw based processing, is all about the raw processor determining what the 0's and 1's are supposed to represent.  What you have to do in PP is irrelevant.  The determining factor, is ensuring that your exposure is correct, during capture.

Marsu42

Quote from: Audionut on March 28, 2014, 05:21:23 PM
Honestly, I think the biggest problem, is that you may simply be underexposing to much.

We had this discussion, I often just want absolute highlight safety for wildlife shots and thus the max. dynamic range, my goal is not to have optimal shadow noise but to prevent clipping in changing light. No matter that, dual_iso and cr2hdr should just work, no matter how you expose.

Audionut

Quote from: Marsu42 on March 28, 2014, 05:48:11 PM
We had this discussion, I often just want absolute highlight safety for wildlife shots and this the max. dynamic range, my goal is not to have optimal shadow noise but to prevent clipping in changing light.

That's fine.  You are free to make whatever exposure decisions you want.  ;)


Quote from: Marsu42 on March 28, 2014, 05:48:11 PM
No matter that, dual_iso and cr2hdr should just work, no matter how you expose.

It does.

Marsu42

Quote from: Audionut on March 28, 2014, 05:51:59 PM
That's fine.  You are free to make whatever exposure decisions you want.  ;)

That's good to hear, and of course I'm always open to suggestions from our local wildlife experts :-p

Quote from: Audionut on March 28, 2014, 05:51:59 PM
It does.

Indeed. On 6d, since yesterday.

Audionut

Quote from: Marsu42 on March 28, 2014, 06:13:32 PM
Indeed. On 6d, since yesterday.

That's cool.  I hope you enjoy dual_iso.  What a shame it only started working for you, yesterday.

arturochu

any updates to cr2hdr for video? (no flicker mode)
Chu

Marsu42

Quote from: Audionut on March 28, 2014, 06:35:17 PM
That's cool.  I hope you enjoy dual_iso.  What a shame it only started working for you, yesterday.

Well, not quite as the awb for underexposed shots is still broken (and even for normal exposures it's other than Canon) - but I think it's getting there. At least I now know why I with the 6d and what I use it for was so worried about some dual_iso oddities other than the big 5d3 folk.

NedB

Girls, girls, you're both pretty. Why don't the two of you just grow up and shut up if you can't be polite in a public forum. Just PM each other and get it over with.

My two cents. Cheers!
550D - Kit Lens | EF 50mm f/1.8 | Zacuto Z-Finder Pro 2.5x | SanDisk ExtremePro 95mb/s | Tascam DR-100MkII

Marsu42

Quote from: NedB on March 28, 2014, 08:35:20 PM
Girls, girls, you're both pretty. Why don't the two of you just grow up and shut up if you can't be polite in a public forum. Just PM each other and get it over with.

Thanks for your input Ned, I'll keep your post in mind as an inspiration as you could also have pm'ed your critique this to us - but now I know who's really grown up around here :->

Audionut

Ok, that's enough off topic fun and games, from all of us.

I'm supposed to be setting an example  ::)

a1ex

Minor update to the 20-bit cr2hdr to fix a regression in bright/dark line detection (thanks g3gg0 for a sample). See my previous posts for download link.