Dual ISO - massive dynamic range improvement (dual_iso.mo)

Started by a1ex, July 16, 2013, 06:33:50 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

mikeortner

Can someone tell me what i am doing wrong, when the cr2hdr.app(OSX) is not processing the dng files of my raw video?
"Hmm... There are no more Dual ISO .CR2 left in this folder."

do i have to rename the dng to cr2?

Audionut

Quote from: a1ex on September 14, 2013, 01:14:23 PM
What about a setting like this in ETTR?

Link to Dual ISO
- OFF (old way)
- SNR priority (as it's now)
- Highlight priority (similar to SNR, but if dual ISO reaches 100/max, ignore the SNR and capture all the highlights).

They look good.

I like the highlight priority option.  I find it tends to use 100/max twice at the end and then gives up.  This would have to help convergence (I shot through the viewfinder), as instead of giving up after 4 shots, based on my testing it would often give up after 2 (save the highlights and be done with it).

Quote from: a1ex on September 14, 2013, 01:14:23 PM
The biggest problem when metering outside LV is that current heuristics (simple and fast) sometimes get tricked because of black level difference. With a black difference of around 30 (typical for ISO 100/6400), the metering error can be up to 4 stops (!).

In LiveView, ETTR is metering on only one of the two exposures (no black difference), and estimates the other exposure from DxO data. That's why it works a lot better than metering outside LV.

Can you make it sample just the darker exposure (outside of LV)?  Would reading the current setting of dual ISO help to determine the black level difference?

RenatoPhoto

Quote from: mikeortner on September 14, 2013, 04:39:38 PM
Can someone tell me what i am doing wrong, when the cr2hdr.app(OSX) is not processing the dng files of my raw video?
"Hmm... There are no more Dual ISO .CR2 left in this folder."

do i have to rename the dng to cr2?

First look at the cr2 files at 100% or 200% magnification and you should see the interlaced dual iso.  If you do not see the horizontal lines then you do not have dual iso image.

I cannot help you with apple stuff..
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

a1ex

QuoteCan you make it sample just the darker exposure (outside of LV)?

If the bright exposure has black level at say 2033 and the dark one has black level at 2063, in deep shadows, the current code will sample the bright exposure thinking it's actually the dark one (since it simply returns min or max from two neighbor pixels).

Ideally, bright exposure = dark exposure * gain (that is, bright EV = dark EV + ISO spacing in EV).

Say the midtones are at 3 EV above black in the dark exposure (that is, 2063+8). In the bright exposure, they will be at 2033+8*64 for ISO 6400. So, midtones will be metered at 4.5/8.95 instead of 3/9 (so, at 3 EV I will have a 1.5-stop error in metering).

To correct this, I could either:
- figure out which lines are bright and which lines are dark before sampling anything (requires rewriting a lot of things)
- figure out how big the error is (because I can find the black difference easily) and correct the metered values

Edit: actually the exposure inversion is not a very big problem (it only appears in really deep shadows); the problem is the magnitude of the black level difference (which alters metering results).

khurra

does cr2hdr work on 650d RAW files?

i converted the 650d dual iso RAW file to DNG using raw2dng and then changed file names to .CR2 and then tried opening with the cr2hdr for mac, but got this error:

Input file : 000000.CR2
sh: exiftool: command not found sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : 000000.txt


Audionut

Quote from: RenatoPhoto on September 14, 2013, 02:41:31 AM
We know that DUAL ISO should be only used in scenes where the dynamic range is greater then what the camera can capture.

Not any longer, not with SNR limits.  It can also be used to tone map in camera.  Say I have a scene of 9EV of dynamic range, but in this scene the skin tones are at -5EV or lower, I can enable the required SNR limit, and boost the skin tones to -3EV (much less noise in critical areas).

I haven't increased DR, because the DR was scene limited.  I did lift the skin tones though.

With adjustments to both midtone/shadow detail, on scenes with large dynamic range, I can also do my best in camera to lift skin tones while leaving the shadows to fall where they may (minimize EV spacing).

nanomad

Quote from: khurra on September 14, 2013, 05:51:21 PM
does cr2hdr work on 650d RAW files?

i converted the 650d dual iso RAW file to DNG using raw2dng and then changed file names to .CR2 and then tried opening with the cr2hdr for mac, but got this error:

Input file : 000000.CR2
sh: exiftool: command not found sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : 000000.txt
They work but you are missing the dcraw executable
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

khurra

Quote from: nanomad on September 14, 2013, 08:17:01 PM
They work but you are missing the dcraw executable

so i need to follow these instructions:  http://bellavist.com/blog/?p=466

on my computer running OSX lion and then try running the mac gui verison of cr2hdr again?

thanks for your help nanomad!!

a1ex

ETTR seems to work a bit better now with dual ISO. There's a little quirk that you may want to be aware of:

- without dual ISO, it will try hard to meet or exceed the SNR limits (clipping as many highlights as needed)
- with dual ISO, SNR limits are respected until dual ISO gets maxed out (100 / max auto ISO). After that, it will not clip any more highlights than allowed by "highlight ignore" (but it will also get a lower SNR).

I recommend triggering it via SET (to meter in LiveView) whenever the lighting changes. This mode converged very quickly in my tests, even in the most difficult lighting (I've tried a light bulb in a very dark room).

RenatoPhoto

Quote from: a1ex on September 14, 2013, 11:41:30 PM
ETTR seems to work a bit better now with dual ISO. There's a little quirk that you may want to be aware of:

When i try to do hg update I get the following error and ettr.c does not update.  Any  ideas what I can do?

hg update
modules/bolt_rec/README.rst: untracked file differs
abort: untracked files in working directory differ from files in requested revision
magiclantern@magiclantern-VirtualBox:~/magic-lantern$


http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X


RenatoPhoto

Quote from: a1ex on September 15, 2013, 12:08:59 AM
Try deleting it.

Thank you very much.  I deleted README.rst
and hg update...
Now I am testing.  It feels wonderful!
Thank you again your are a magician!
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

RenatoPhoto

I had a quick look at the images and everything has fallen in its place.
Nice images, good capture of dynamic range, proper automatic decisions on when to use DUAL ISO, very responsive to SNR settings.  The usability factor seem very good.
It seems PERFECT!  (I will probably find something else to complain about... :o)

If I could, I would nominate you for a Nobel Prize for ingenuity.  ;D ;D ;D

P.S I was forgetting to mention that now the camera can capture what I see and beyond!!!!

P.S. 2 Maybe they should add this features to imaging space telescopes and I bet we could see little green men on the sun!!
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

mikeortner

Quote from: RenatoPhoto on September 14, 2013, 05:31:59 PM
First look at the cr2 files at 100% or 200% magnification and you should see the interlaced dual iso.  If you do not see the horizontal lines then you do not have dual iso image.

I cannot help you with apple stuff..

Yes the lines are there, i already checked that before ... maybe i should try the PC version

Danne

The aettr together with dual iso works really well now :). THanks for that. Trying to understand when to use what settings in snr for shadow and midtone. 8/4 is the most extreme and 1/1 the least extreme sort of speak. What would be the best way to find rules for when to use what? I read comments about skintones etc. Anybody tried any settings? Is there a way to visually understand the optimal snr settings maybe with raw zebras, histogram?
Thanks again
//D

a1ex

Audionut did some tests here and I chose the defaults according to them: http://www.magiclantern.fm/forum/index.php?topic=5200.msg70150#msg70150

Also see this: http://photography-on-the.net/forum/showthread.php?t=1326078&page=2&highlight=snr

I'm thinking to draw something to show where midtones and shadows are metered exactly. If you remember the old bulb ramping with percentiles, it would be something like that (since the math is the same).

Canon eos m

Quote from: a1ex on September 15, 2013, 10:17:35 AM
Audionut did some tests here and I chose the defaults according to them: http://www.magiclantern.fm/forum/index.php?topic=5200.msg70150#msg70150

Also see this: http://photography-on-the.net/forum/showthread.php?t=1326078&page=2&highlight=snr

I'm thinking to draw something to show where midtones and shadows are metered exactly. If you remember the old bulb ramping with percentiles, it would be something like that (since the math is the same).

Alex, I noticed something today. With Dual_ISO & AETTR on my 5DM3 on the numerical EV showing at the bottom right end of the screen under my light situations shows +0.3 EV when ML ISO is at 100 and Canon ISO at 3200; and +4.0 EV when ML ISO is at 3200 and Canon ISO at 100. The same for my EOS M.

Could there be a valid explanation for the cameras behaving this way?

Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)

a1ex

Canon metering always meters the image that you are previewing.

Canon eos m

Quote from: a1ex on September 15, 2013, 10:39:33 AM
Canon metering always meters the image that you are previewing.
Okay. That makes sense. Thanks as always. 
Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)

akumiszcza

I'm testing new build with auto ettr & dual iso link on my 50D. The idea is great, photos look nice. Some issues:
- Quite often auto ettr starts (LV turns on for a split second and turns off), but It does not do anything. It does not say "Expo out of limits" or "Whoops". Thinking it's on I shoot and get much underexposed photo. When I turn LV on and press SET, Auto ETTR works fine. Would it be possible to write "auto ettr ok" on image review after the shot if ettr worked and finished with no errors?
- auto snap and always on does not work on 50D still. It works with 1%'s tragic lantern builds – tl50d compiled by Andy600, but not nightly builds.

It looks like exposure limits for auto ettr is greater than before, too.

a1ex

Status messages solved. Also found some more bugs regarding convergence.

Exposure limits are Canon limits, nothing should change here. I didn't check them very well though.

For auto snap, one has to compare 1% repo with ML one, extract the relevant pieces of code (most likely ettr.c, dual_iso.c and the raw backend) and create a pull request. I prefer this to be done by a 50D user.

Marsu42

@alex (sorry if this request has been issued before, you know, long threads and all...):

Could you please change the "dr gained" field into a setting so that the user can set a "min. dr gain" value? On high iso, doing dual iso doesn't make much of a difference dr-wise and isn't worth the postprocessing hassle, so this would only enable dual_iso on lower base iso.

Danne

THanks for linking ALex. Seem 6ev is a good compromise getting about enough grain.

Audionut

I haven't done any tests yet, but judging from that last test, I would use 7EV or 8EV for the midtones with dual ISO, depending on your preference of EV separation.

6 EV could lift the midtones well while still retaining good highlight detail, back before it had the help of dual ISO.  7EV should now give you good midtone detail with little EV separation (100/400).  8EV would give you excellent midtone detail (metered correctly), with only 100/800.

Of course the recovery setting is dependent on scene luminance.


Danne