Dual ISO - massive dynamic range improvement (dual_iso.mo)

[davidcl0nel]

DualIso works well on 5D3, thank you for this feature! With the first tests the detail loss of structures was a little bit to much, but with newer converters it is much better. But the shadow recovery works very very good.

But I have a little "other" request. I install the ML-version from the dropbox onto the CF-Card (not SD), and it's working good. Yesterday I want to format the CF, because of 300+ pics, and the format menu cleans the whole card, with ML. Grrr, I had to copy it again with the card reader.

With the ML-version of my 60D I know the "feature", which reads the whole ML-directory (and autoexec) into the RAM, formats the card, and rewrite ML back to the card. Is this feature broken or is it deactivated on purpose?

[a1ex]

That part needs to be reverse engineered almost from scratch because of the double card slot. Same for the installer.

[Danne]

A little testing with Alex new aliasing changes in the experimental cr2hdr_.exe and the most recent one from the first post.
A little more jagged edges in the aliasing parts in some motifs but it also completely removes aliasing from other motifs in the experimental. Hard to tell what is best, I,m leaning towards the experimental, especially from the pic of the old woman and of course from the store. Here,s some pics. I strived to get the motif with both isos.
The originals can be downloaded from this link (119mb 4 pictures)
https://docs.google.com/file/d/0B4tCJMlOYfirRlFjZ3ljVEtDc0U/edit?usp=sharing



experimental


recent




experimental


recent


experimental


recent




experimental


recent


Magentaissue in some pics from the experimantal build, not seen in the converter from the first post.
I also only manage to convert 35 pictures, cr2 with the converter from the first post and the it quits. This don,t happen with the experimental build.


If any of these are of interest as originals please tell me which and I,ll upload those.

*post updated with link to the originals(see above)

[a1ex]

Yes, upload them. These are very good for my test shot collection.

[Danne]

@Alex
Post updated with originals
https://docs.google.com/file/d/0B4tCJMlOYfirRlFjZ3ljVEtDc0U/edit?usp=sharing
Thanks!
//D

[NateVolk]

@Danne - What causes the magenta issue in the highlights?  I have a few pics I can't get that to go away on...

[a1ex]

Download cr2hdr_exp.exe again and upload the CR2's if you still have trouble. It was hot pixels in overexposed areas tricking the white level autodetection code.

[NateVolk]

Thanks A1ex,  That fixed the magenta highlight issues, but still major ailiasing.  I probably exposed way wrong and didn't keep things covered by both iso's.

One thought I had that might make things easier shooting is to be able to reverse the iso's.  Meaning shoot at iso 1600 and use iso 100 as recovery for the highlights.  I did a real estate shoot where if I exposed to not clip the highlights at iso 100 and set the dual iso to 1600 to save the mid tones, the live view was basically black and I couldn't frame my shots.  If I switched to 1600, then I could see the scene more properly exposed.  I would then switch back to 100 to shoot...

Maybe your script is already that smart, I haven't tested it yet.  I also have found that for video to come out correct, I have to reverse the settings as well...

Thanks for all your accomplishments, it really is mind boggling!

[Audionut]

@NateVolk

You can shoot with the higher ISO and then select the highlight recovery ISO in the dual-ISO menu.  You can use the +-EV settings also to either consistently use +whatever EV for recovery or consistently -whatever EV.

[AdamTheBoy]

I'm having an issue I would like help with please, I don't think I've seen this before on the forum and I couldn't find it with a search.

A number of my pictures will begin processing only to say "file doesn't look like dual ISO"

The images are definitely dual ISO so I'm not really sure where to start troubleshooting, any suggestions?  Thanks very much.

[a1ex]

For the 1000th time: if you are having issues and you don't upload some CR2 files, there's nothing I can do to help you.

[AdamTheBoy]

I'm sorry about that, that should have been obvious to me.  Here's the .cr2 file. https://www.dropbox.com/s/s2tew00wftvlef4/7O4C8804.CR2

[a1ex]

Uploaded a fix (download cr2hdr_exp.exe again). Can you check with your other pics?

[Audionut]

a1ex can you add something to the filename to differentiate dual-ISO's? 

I shot 1000 photos the other day with dual-ISO on alternate frames thinking that would be a great idea.  1 back-up with each dual-ISO shot.  It worked great, and I love the results of +2EV shadow recovery, but processing them was a pita, simply because they take up space otherwise, if you don't delete the original CR2's.

Canon uses different filenames for AdobeRGB and sRGB color spaces.  Perhaps you could hook into that?

Once everything gets closer to RC, I would like to suggest the option of deleting the original CR2's after being processed by cr2hdr.

[a1ex]

My workflow is to drag all the files on cr2hdr: it will autodetect which shots were dual and which not. Sure, it takes a bit of time. My autoexposure script uses DNG by default, so it's quite straightforward: I just run both tools, leave it crunching numbers for a few hours, and I get nearly-finished pics.

For changing file names, I can add an option to shoot dual ISOs in adobe RGB, and normal pics in sRGB, for example. Or, I can change the first 4 letters (suggestion?)

I would not suggest deleting the CR2 files. The algorithm is improving, so if you reprocess the same thing a month later, you already get better results.

[Audionut]

I was selecting everything over 26000kb to drop on the exe.  Still takes time to process 000's of photos though. 

The first 4 letters are a good idea.  DUAL0001.CR2 

[a1ex]

Small problem with this approach: they are no longer sorted (you need to rename them back in post so you can watch them in the same order). Plus, syncing the file names with the alternate option (semi-burst) is tricky (it's too easy to mis-label the pictures, because capturing and saving are done at different times).

[Audionut]

Hmmmm.  I would be happy if their was a cmd option to move processed files.  Including non dual-ISO files put through the exe.
I could dump the CF onto an SSD and have all the processed files moved to storage.  Not sure of the usefulness of this for others though.

The seperation of processed and original dual-ISO's is what I'm chasing.  I can't think of any easier ways.

edit:  And thanks.  Just noticed the commit.

[robinlee]

Hi Alex, can you post the download link for this on your first page please?

Thanks
Robin

Uploaded a fix (download cr2hdr_exp.exe again). Can you check with your other pics?

[a1ex]

Small update for cr2hdr_exp.exe: better black level handling. This is only noticeable in extremely underexposed shots (e.g. when you need to push the shadows at +10 EV).

Before and after (CR2 from AdamTheBoy):


For grading: I've developed the DNG at +4 EV ... +12 EV in 1 EV increments and ran the shots through enfuse. The image was quite underexposed, next time try using ETTR

Just for fun: here's how it would have looked at ISO 100. Good luck denoising that.


For higher-res crops, download the CR2 and grade it yourself. This update was about color cast in shadows, not about fine detail.

I'll update it on the first page and in raw2dng when I'll solve the remaining issues in highlights.

@IliasG: you around? can you check this and see if you get better results?

[IliasG]

Small update for cr2hdr_exp.exe: better black level handling. This is only noticeable in extremely underexposed shots (e.g. when you need to push the shadows at +10 EV).

Before and after (CR2 from AdamTheBoy):
http://acoutts.com/a1ex/7O4C8804_before.jpg http://acoutts.com/a1ex/7O4C8804_after.jpg

For grading: I've developed the DNG at +4 EV ... +12 EV in 1 EV increments and ran the shots through enfuse. The image was quite underexposed, next time try using ETTR

Just for fun: here's how it would have looked at ISO 100. Good luck denoising that.
http://acoutts.com/a1ex/7O4C8804_dark.jpg

For higher-res crops, download the CR2 and grade it yourself. This update was about color cast in shadows, not about fine detail.

I'll update it on the first page and in raw2dng when I'll solve the remaining issues in highlights.

@IliasG: you around? can you check this and see if you get better results?

Yep, around and just checked. Much better about color cast at darks, but at a cost. The noise is harsh and it's not points (easy to denoise) but like candles. And I think that there is too much black clipping.

As I see here the Black Level in the CR2 file is around 2032-2034 for the low ISO lines and 2062-2066 for the hi ISO lines. How do you normalize for this ?. (In the meantime I have to read again your pdf I suppose ..).
This shows clearly that Canon tries to normalize at the desired average of 2048 as BL but how exactly ??. Is it just a data sliding up or down or a multiplication ?.

It happens this shot is with f/1.8 so there should be data scaling (due to wide aperture) and the White Level must be around 16000 ... the the DNG's exif says 49600 !! ?? even lower than 4*13000 ...

In the case of white level I still vote for a Lookup table.

[ted ramasola]

@a1ex,

Took this dual ISO shot while i was testing burst mode. I know a res chart is not the best for testing dynamic range which is the benefit of using dual ISO but I figured the behavior of aliasing and moire can be better observed and tweaked in a repeatable test environment.

Just hoping it helps.

From dual iso shot 200 cam 1600 recovery. o=sharpening in acr



From normal cr2 same settings at 200 iso

[ted ramasola]

If dual iso can be ported to raw video in the 7D, it sure would be much better in handling moire and aliasing.

Here's a chart from burst mode compared to dual iso in my previous post.

[a1ex]

@Ilias: I split the raw file in 4 sub-images (group by y mod 4) and bring the black level to 2048 for each one (scaling to keep the old white level and taking care of drift). Then I run the interpolation routines. At the end, I average the black level again (to compensate for what the interpolation might do to noise), and it still finds a few units (for this image, 3 units).

The output DNGs are 16 bit. Not sure if the extra 2 bits have anything useful in them or just rounding noise (but at least they open pretty fast in ufraw compared to CR2's).

@Ted: the result with resolution chart will vary greatly with exposure (so, for best results, you should be careful when exposing). Yes, the VAF will help a lot for video (and I'm sure it will work on 7D too).

In the mean time... do you have a VAF for 7D? if so, can you shoot the test chart in photo mode with dual ISO at some extreme setting? (where aliasing appears normally)

[Canon eos m]

I'll update it on the first page and in raw2dng when I'll solve the remaining issues in highlights.

Hi Alex is the raw2dng now updated on the first page?

Wondering if it would be possible to do audio with dual ISO active during video taking on my 5D Mark 3.