Dual ISO - massive dynamic range improvement (dual_iso.mo)

[djronbxs]

what do we need to download for the fix now from your first post ? cr2hdr or the dual iso module ?

[a1ex]

@alex: Can you please check:

https://www.dropbox.com/s/cvdz6nz2t0rvnxy/DUAL0650.CR2

I converted it with the Oct-10 version (latest I guess) and at 100% (and more) patterns are visible.

I must be blind, I can't see any patterns here (except for some noise in shadows, which is normal). Not even with crazy curves with extreme contrast.

[tron]

I must be blind, I can't see any patterns here (except for some noise in shadows, which is normal). Not even with crazy curves with extreme contrast.

Alex I used Oct 10th cr2hdr.exe when I posted this. When I used Oct 11th version it seemed OK to me too!!!!
In fact I thought that maybe that's why you changed it!
What I still see is zig zag lines in a specific part where there is a round object. But if I recall correctly you had mentioned this yourself a few months ago as a characteristic of the procedure.

Sorry for not updating my observations with Oct 11th cr2hdr.exe

[Danne]

Nice to see more improvements upon dual iso and movie mode.
A question. @tron or anybody. Eager to try the changed prefix for dual iso which I understand is unreliable. Still, selecting this in dual iso the filenames are still without the new prefix I see some of you get? I use a 5d mark 3. Am I missing something? I,m in single shooting mode, Taking one picture, waiting. Still no changes in prefix?

[swinxx]

Hi alex,

here is a Dual Iso Shot where i shot a face of a person and a bright sky. when i use the latest cr2hdr 1.5 app for mac the dngs look strange.. (especially the hair of the guy - like there is less resolution)

here are converted dual iso dngs:
https://www.dropbox.com/sh/ukdu06xedivedog/FyiCLZcHkm
 
and here are the unconverted dngs out of rawmagic 1.0
https://www.dropbox.com/sh/guqv68nxfn8uvc1/UKS4lnnjA9

thx. sw

[Audionut]

Nice to see more improvements upon dual iso and movie mode. A question. @tron or anybody. Eager to try the changed prefix for dual iso which I understand is unreliable. Still, selecting this in dual iso the filenames are still without the new prefix I see some of you get? I use a 5d mark 3. Am I missing something? I,m in single shooting mode, Taking one picture, waiting. Still no changes in prefix?

Do you have a custom prefix set via Canon menus?

[Danne]

Nice @Audionut. Thanks. Had to change under menu "filename" to "preset code".
Tried to shoot in burst mode and it didn,t miss naming to "DUAL" once. Maybe isn,t that unreliable after all? PLease do not take this feature away @Alex. Gonna save me tooons of time
By the way. Had no idea I could change naming prefix in the camera, sweet.
Thanks a lot for yet another feature Alex

[Audionut]

Tried to shoot in burst mode and it didn,t miss naming to "DUAL" once.

Set dual ISO to alternate frames only and then rapid fire
Depending on the scene and the SNR limits in AETTR with linked dual ISO, it could miss the prefix there also when trigger happy.

[Danne]

hehe, I see. Luckily I,m not that trigger happy atm but will try to see what happens. Still very useful

[Audionut]

I wouldn't use dual ISO on large shoots without the prefix.  PP is already a time consuming job.

So for 2000 photos it would take 1 hour 8 minutes

On top of the process time for the actual dual ISO shots.  So if I have 100 dual ISO shots, I've doubled that part of the processing time for no other reason then there is no other way to distinguish which shots need to be processed. 

Then there's the file management.  Prefixed shots get dumped into a separate folder and the processed DNG's copied into the shot folder.  This way I can keep the original CR2's separate for later processing (updated cr2hdr or whatever), without having to process all 2000 photos again, and, I don't have interlaced CR2's polluting the image viewer.

[a1ex]

here is a Dual Iso Shot where i shot a face of a person and a bright sky. when i use the latest cr2hdr 1.5 app for mac the dngs look strange.. (especially the hair of the guy - like there is less resolution)

Is this a regression? (was it rendered better with some older version?)

I'm not sure where to look, it seems OK to me.

[a1ex]

Alex I used Oct 10th cr2hdr.exe when I posted this. When I used Oct 11th version it seemed OK to me too!!!!
In fact I thought that maybe that's why you changed it!

I didn't expect any change in banding patterns (since the last changes were mostly for deep shadows). It may also be because I run all my test shots through ufraw, without any sharpening or denoising.

I have an idea for horizontal banding correction, so I'm looking for some test shots where this is clearly visible.

[swinxx]

hi alex.

i don´t know if that shot would be better with older versions.
perhaps it is too much difference between the brightness and the darker areas in that special shot.?

thx. sw

[akumiszcza]

On top of the process time for the actual dual ISO shots.  So if I have 100 dual ISO shots, I've doubled that part of the processing time for no other reason then there is no other way to distinguish which shots need to be processed. 

If the prefix works for you, no reason then. But I get about 1 in 10 shots wrongly prefixed, both ways.

Then there's the file management.  Prefixed shots get dumped into a separate folder and the processed DNG's copied into the shot folder.  This way I can keep the original CR2's separate for later processing (updated cr2hdr or whatever), without having to process all 2000 photos again, and, I don't have interlaced CR2's polluting the image viewer.

That's what dualiso.vbs script does – it renames all files according to exif date and time, moves real Dual-ISO cr2s to separate folder than dngs and even makes a "_dualiso" suffix to Dual-ISO .cr2 and .dng to make them easier to distinguish. Anyway, I've already almost processed a Dual-ISO photo with some batch processing because I didn't notice it was Dual-ISO on quick preview (and it had no prefix). The result would have been strange…

[Audionut]

If the prefix works for you, no reason then. But I get about 1 in 10 shots wrongly prefixed, both ways.

So disable it!

[akumiszcza]

So disable it!

Yeah. That's the best remedy. Thanks…


BTW. Would it be possible to have a message during a shot review taken after SET mode in ETTR? 50D does not have ettr confirmation beeps and I don't know if ettr worked during LV after pressing SET, making the mode less useful than other. It would be great to have a message saying if ettr failed or not, especially since set mode is the fastest 

[tron]

Can the dcraw.exe fall back to sep 20 or before version? It does not work...
On the other hand Sept 20  version - or earlier, I never checked dcraw.exe creation date -anyway - works FINE (Even with newest cr2hdr.exe)

[a1ex]

See http://www.magiclantern.fm/forum/index.php?topic=7139.msg81571#msg81571

[tron]

@alex: Thanks. I got dcraw.zip

[Audionut]

BTW. Would it be possible to have a message during a shot review taken after SET mode in ETTR? 50D does not have ettr confirmation beeps and I don't know if ettr worked during LV after pressing SET, making the mode less useful than other. It would be great to have a message saying if ettr failed or not, especially since set mode is the fastest

Can't you judge the results from the histogram?

[Marsu42]

It's possible; when ETTR switches from dual to non-dual ISO, the change may not be applied right away; so if you catch the wrong timing, you are out of luck. I've added a warning for now. Having a file list (like with intervalometer) may be easier to sync.

An idea concerning the prefix problem: Would switching from "_MG_" Adobe RGB to "IMG_" sRGB work? Since dual_iso is about raw, it doesn't matter what color space the sidecar jpeg is in, but using a Canon rename function might fix the sync problem if switching the color space prop(?) has no lag.

[a1ex]

Small update in cr2hdr regarding horizontal banding. Basically, after matching the brightness (ISO + black difference), there's still some room for improvement, so this update does some fine-tuning for every dark line (so it matches the brightness of its neighbours).

The result is pretty subtle. I didn't find any example with singificant banding in final output, but it's clearly visible in the intermediate results. Here's a full resolution blending (the one that minimizes aliasing, but has pretty high noise):

Before:


After:


So, the algorithm simply has cleaner data to work with, and it's less likely to pick up false detail. Also, the noise in full-resolution areas is less correlated (whiter). With this, I was able to push the algorithm even more towards full-res shadow detail without increasing the artifacts (so the forest shot is still rendered very well).

Example:

Before:


After:



For prefix, I think switching the color space has the same sync issue. Didn't try, but I don't see why it will be different.

There's no beep for ETTR with SET on any other camera; I look at raw zebras, histogram and ETTR hint.

[1%]

I just turn the beep off globally... too much beeping in general.

[Danne]

Very nice improvement on the horizontal banding!

[Marsu42]

Very nice improvement on the horizontal banding!

I'm re-processing all my dual_iso cr2 now, these updates really keep my computer busy :-p ... fortunately with Lightroom you just need to replace the old dng & update the preview, all the metadata is in the LR database. Also afterwards I'm converting the ML dng to ACR dng which about halves the file size.