Green tint in dual ISOs and cernoice output

[Marsu42]

Marsu42's files (6D)

Doh, looking at the 6d/5d3 differences I now understand why the tint issue is such a huge problem for me while alex and other 5d3 folk don't seem that concerned.

As I wrote above, I'm completely puzzled what's going wrong with dual_iso tint and how it is computed from the cr2 tags so I can only hope someone else is smart enough to figure it out :-o

Edit: I do really hope this is just a metatag issue and there's nothing wrong with dual_iso itself on the 6d that cannot be fixed by a cr2hdr update

[ayshih]

PR updated with new logic.  In brief, "WB RGGB Levels Measured" will be used if there is an indication that it preserves the dual-ISO behavior (50D), otherwise "Raw Measured RGGB" is used (everything else so far).  Which tag is used is printed in the cr2hdr output (right near the end), as well as a warning message about a possibly out-of-date exiftool if it can't find these tags.  So, here's where we stand:

Marsu42's files (6D):

Code Select Expand


IMG_8666.CR2  5900  +8   (normal)
DUAL8668.CR2  4150  +25  (Raw Measured RGGB)
DUAL8669.CR2  4400  +27  (Raw Measured RGGB)
--
default       5600  +20


Audionut's files (5D3):

Code Select Expand


_46A9808.CR2  4350  +48  (normal)
_46A9809.CR2  4400  +28  (Raw Measured RGGB)
--
default       5500  +19


My files (50D):

Code Select Expand


IMG_0689.CR2  5450  +13  (ISO 100)
DUAL0688.CR2  5700  -4   (ISO 100/800, exposed for 800)
--
IMG_0709.CR2  4800  +27  (ISO 400)
DUAL0708.CR2  4750  +27  (ISO 400/100, exposed for 400)
--
IMG_0707.CR2  4400  +21  (ISO 400)
DUAL0706.CR2  3900  +19  (ISO 400/100, exposed for 400)
--
IMG_0715.CR2  4850  +30  (ISO 400)
IMG_0717.CR2  4800  +28  (ISO 100, exposed for 400)
DUAL0714.CR2  4650  +26  (ISO 400/100, exposed for 400)
DUAL0716.CR2  4500  +26  (ISO 100/400, exposed for 400)
--
IMG_0718.CR2  4300  +23  (ISO 100)
DUAL0719.CR2  4000  +3   (ISO 200/100, exposed for 100)
DUAL0720.CR2  3500  +10  (ISO 400/100, exposed for 100)
DUAL0721.CR2  3400  -24  (ISO 800/100, exposed for 100)
DUAL0722.CR2  3900  ***  (ISO 1600/100, exposed for 100)
IMG_0723.CR2  4250  +24  (ISO 100)
--
default       4850  +46


Just with these shots, this fix seems to work decently on everything but the 6D, but there needs to be more testing across a variety of lighting conditions.

[Marsu42]

Just with these shots, this fix seems to work decently on everything but the 6D

Doh (yes, I know I already said that) - let me know if you need any additional testing on the 6d, or maybe alex/1% might share some insight on if there's a problem with the metatags or dual_iso itself on the 6d...

[ayshih]

More test images for the 6D would definitely be useful, with different scenes and different lighting.  Perhaps there's something specific to that scene that is throwing off the 6D, and the other cameras would have trouble too.

I've added the results from more 50D shots to my earlier post.  As expected, this fix tends to struggle when one of the ISOs is significantly overexposed (which, of course, is typically the case when dual ISO is used for shadow recovery).

[Marsu42]

More test images for the 6D would definitely be useful, with different scenes and different lighting. 

Next batch, colorchecker with artificial light, normal cr2 and dual_iso (100/400, 100/800, 100/1600): https://bitbucket.org/Marsu42/ml-aiso/downloads/dual_iso_test_1_a.zip

I ran the 20bit cr2hdr with your latest commit on them, but and the acr "as shot" from the normal cr2 (3500/39) is still completely different from the "as shot" computed by the new cr2hdr (2250/24, 2500/18, 2400/16). Btw the acr "auto" values are about the same for normal cr2 and dual_iso.

It's just that in this case the dual_iso shots happen to look subjectively better because the normal cr2 shows the color cast of the energy saver bulb... but the point is that the dual_iso "as shot" computed wb still hasn't got anything to do with the normal cr2 "as shot" canon awb - and that's what it is about, isn't it?

Of course all these don't look *exactly* as they should in comparison to getting the correct temperature/tint with the lightroom wb dropper (2650/29), but this isn't to be expected anyway.

Last not least, in this cr2hdr there's a regression - the last scanline on the very bottom is broken and shows artifacts.

[ayshih]

I ran the 20bit cr2hdr with your latest commit on them, but and the acr "as shot" from the normal cr2 (3500/39) is still completely different from the "as shot" computed by the new cr2hdr (2250/24, 2500/18, 2400/16). Btw the acr "auto" values are about the same for normal cr2 and dual_iso.

(You have a small typo; the first one is 2550/24.)  I'm encouraged by how consistent the white balances are for the three dual ISO shots.  I'm not surprised that it's so different from the normal shot; test shots should probably stick to lighting temperatures above ~3500 K.  I'm not sure what ACR actually does when you select "Auto"; if it actually reads the "WB...Auto" tag, it applies additional logic.

It's just that in this case the dual_iso shots happen to look subjectively better because the normal cr2 shows the color cast of the energy saver bulb... but the point is that the dual_iso "as shot" computed wb still hasn't got anything to do with the normal cr2 "as shot" canon awb - and that's what it is about, isn't it?

Canon seems to refuse to accept a temperature less than ~3500 K for auto white balance, but reverse engineering their exact logic (i.e., step (5) in my list above) would be non-trivial.  This difficulty would be present whether the fix is based on EXIF information or the raw data.  So, at least for now, it's beyond the scope of this fix to get matching at such temperatures.  But, on the plus side, this mismatch actually gives you a better white balance.

Last not least, in this cr2hdr there's a regression - the last scanline on the very bottom is broken and shows artifacts.

Hmm, I haven't looked at the 20-bit branch at all, but I've barely touched cr2hdr.c, so it should be "easy" to track down the regression.

[Audionut]

Last not least, in this cr2hdr there's a regression - the last scanline on the very bottom is broken and shows artifacts.

Yes, this is an error in the latest code form a1ex.

Reported by IliasG here.

I'm not sure what ACR actually does when you select "Auto"; if it actually reads the "WB...Auto" tag, it applies additional logic.

Based on observation, I would say that Adobe applies it's own logic.

[Marsu42]

I'm encouraged by how consistent the white balances are for the three dual ISO shots.  I'm not surprised that it's so different from the normal shot; test shots should probably stick to lighting temperatures above ~3500 K.

Ok, so here's another proper batch sample in daylight with awb from both 6d and 60d. I shot iso 100/200/400/800/1600 once as a normal cr2 with proper exposure, then once dual_iso with iso 100 exposure (i.e. high dual_iso overexposed), then once with "under"exposure even for dual_iso for highlight safety. I didn't have time to look at the wb though, have to work trough the week...

FIXED ARCHIVE BELOW!

Hmm, I haven't looked at the 20-bit branch at all, but I've barely touched cr2hdr.c, so it should be "easy" to track down the regression.

I posted a bug description in the dual_iso thread.

[ayshih]

Ok, so here's another proper batch sample in daylight with awb from both 6d and 60d.

For the dual ISO shots, I've labeled it such that the first ISO is the properly exposed ISO, so if the second ISO is greater, then there is overexposure going on.

Marsu42's 6D shots (uses Raw Measured RGGB):

Code Select Expand


100       5950  +10
200       5900  +11
400       5900  +10
800       5900  +9
1600      5900  +8
100/200   4650  0
100/400   4650  -1
100/800   4650  -1
100/1600  4700  +1
200/100   4600  -2
400/100   7800  -78
800/100   7800  -78
1600/100  7800  -78
--
default   5600  +20

Although the fix doesn't work well for the 6D, at least the results are consistent, even with overexposure.  There does appear to be an additional problem with underexposure.

Marsu42's 60D shots (uses WB RGGB Levels Measured, like 50D):

Code Select Expand


(removed due to mini_iso changing the black level)

The fix performs spectacularly bad!  I haven't dived into the actual raw data, but I noticed that the nominal black level appears to change with ISO: 64 at 100, 96 at 200, 128 at 400, 192 at 800, and 320 at 1600.  Are you running mini_iso or otherwise tweaking your black level?  Or does the stock 60D do this?  I wonder whether the changing black level is what is making the EXIF tags completely unreliable for this fix.

Also, annoyingly, "Raw Measured RGGB" is always "0 0 0 0" for these 60D files, so it's harder to guess at what's going on inside the camera.

[Audionut]

mini_iso has an option to do that to the BL.  It maintains the lowest BL, while still being large enough to capture the stdev.

[Marsu42]

Although the fix doesn't work well for the 6D, at least the results are consistent, even with overexposure. There does appear to be an additional problem with underexposure

Hmmyes, I guess that's why your fix wasn't working for me as I usually underexpose dual_iso for complete highlight safety :-o

The fix performs spectacularly bad!

Oh no, sorry for wasting your time - I indeed had mini_mo also copied to my 60d and then forgot about it... but thanks again for looking at the tint problem as this need a lot of trial & error work :-o

Link to fixed archive : https://bitbucket.org/Marsu42/ml-aiso/downloads/dual_iso_samples_6d_60d_FIXED.zip

[Audionut]

But underexposing 100/800 should be about the same as proper exposure for 800/100, shouldn't it?

Depends how you are metering. 

If you're metering 100/800 for ISO 100, and underexposing, you have ISO 800 picking up the slack.  You're underexposed at ISO 100 (highlight safety), and half of the resolution is being scanned at ISO 800 (reduce noise in shadows).   Note:  You remove the electronic noise with ISO 800 (dual_iso), but dual_iso does nothing for shot noise.

If you're metering 800/100 for ISO 800, you should be applying a shadow priority.  In this case, you may not know if the ISO 100 scan will capture all of the highlights.

[Marsu42]

If you're metering 800/100 for ISO 800, you should be applying a shadow priority.  In this case, you may not know if the ISO 100 scan will capture all of the highlights.

Indeed, that's why I don't see any reason for high/low to be there anyway - as argued multiple times, you can always try to recover details from shadows with cr2hdr wizardry, but if half of the highlights are clipped that's that and this is visible on my standard furry shots.

[Audionut]

Indeed, that's why I don't see any reason for high/low to be there anyway

I don't use it.  But where shadow priority is significantly greater then highlight priority, I can see how it would be useful.

I like to look at the zebras for exposure.  They don't lie.
I don't fully understand the maths (little reminder I am maths stupid), but changing line 582 of zebra.c as so,

int underexposed = ev_to_raw(- (raw_info.dynamic_range - 100) / 115.0);

Changes the underexposed zebras to around -9 EV.  This is much more aligned with an acceptable noise limit in the shadows @ ISO 100, then the actual noise floor of the camera (SNR=1:1 = reported DR).

you can always try to recover details from shadows with cr2hdr wizardry, but if half of the highlights are clipped that's that and this is visible on my standard furry shots.

Here, you're in extreme highlight priority, including full resolution.  Depending on how bright that detail (that needs to retain full resolution) is, you may still be able to get away with 100/200 or 100/400.
I would use zebras in this situation also.  Look for the half zebras.

Needing 9 gazillion stops of DR means you have to use large ISO spacings (100/1600 for example).  But the biggest jumps in noise reduction in the shadows, actually come with the next highest ISO.  ISO 100/200 for example.

If you can use ISO 100/200, or ISO 100/400, and the zebras are showing the (wanted) highlights retaining full resolution, then why not?

[ayshih]

Oh no, sorry for wasting your time - I indeed had mini_mo also copied to my 60d and then forgot about it... but thanks again for looking at the tint problem as this need a lot of trial & error work :-o

Okay, so with the latest batch of shots, all the black levels are ~2048 as expected, and all the "Raw Measured RGGB" values are nonzero.

Marsu42's 60D shots (uses WB RGGB Levels Measured, like 50D):

Code Select Expand


100       5100  +3
200       5150  -1
400       5150  -1
800       5100  -1
1600      5100  -1
100/200   4150  -11
100/400   4650  -16
100/800   4800  -28
100/1600  3150  -99
200/100   4200  -9
400/100   4700  -1
800/100   4150  -23
1600/100  2150  -94
--
default   5000  +11


The EXIF-based fix appears to give plausible but incorrect values up to a 2-stop difference in the ISOs, starts to diverge significantly in tint at a 3-stop difference, and then is completely off at a 4-stop difference.

Based on the results from these three cameras, I think this may be what we can typically expect from the EXIF-based approach: sometimes decent results up to a 2-stop difference in the ISOs, with effectiveness varying with scene and camera model.

I guess it's time to investigate the raw-data approach.

[a1ex]

I guess it's time to investigate the raw-data approach.

Playing with a custom WB algorithm: https://www.dropbox.com/sh/uz75fw290v4n7v7/ASbJRAUXk-
- auto: this is ufraw's auto white balance (most likely a simlple averaging)
- graymax: this is my algorithm, which attempts to maximize the number of gray pixels; this is done by looking for a peak on a vectorscope in EV space, blurred and weighted towards daylight. The idea is roughly similar to this one: http://www.stanford.edu/~sujason/ColorBalancing/robustawb.html

Some of the raw files can be found here: https://www.dropbox.com/sh/sta87fj2pghv6o4/vPXNvbhZdJ
Most of them were taken from the dual iso thread.

Also worth reading: http://www.acad.bg/rismim/itc/sub/archiv/Paper3_1_2012.pdf

If this algorithm gets the WB pretty much right, this will be very useful for AMaZE (reducing highlight alias even more, since AMaZE relies heavily on correct WB applied before demosaicing) and also for the highlight compression (--soft-film). If this algorithm results in significantly fewer pictures that require manual WB tweaking in post, it should be a nice timesaver.

I don't know how LR does its auto white balance, but I guess Marsu would have already used that one if it was any good

[Marsu42]

I don't know how LR does its auto white balance, but I guess Marsu would have already used that one if it was any good

Actually ACR "auto" wb isn't that bad! For my taste it just usually lands a bit too much in the cyan-cold region (yes, my monitor is calibrated) so I seldom use it - but it's fine as a base for dual_iso shots with its currently broken wb.

If cr2hdr could emulate this ACR awb it would be perfect for most people, I guess this is  about what can be done with a simple algorithm w/o trying to be as "intelligent" as Canon is and w/o reverse engineering what Canon does in camera.

Let me know if you need any help trying something out on ACR/LR, but I'm currently too busy to contribute anything else than a quick test and feedback :-o

[a1ex]

I've just commited the code; maybe you can check if it works in ACR and maybe some comparison shots with ACR AWB would be nice. Indeed, this one is a fairly simple algorithm without any reverse engineering.

[Audionut]

_46A0416.jpg / Adobe Auto WB  (4350k / +30 magenta).  I would class this as pretty accurate, and by accurate I mean, the light source (afternoon sun) appears white.  My personal preference is around 600k warmer.

I think Marsu42 uploaded a zip earlier with some color checkers, shot in varying light conditions.  I know you have shown that a grey card is not 100% accurate, but it removes any personal preference (bias), from what should be an accurate (white = white, grey = grey) algorithm. JMO.

The x-rite color checker, also contains patches, that are progressively warmer and cooler (not sure if Marsu42 captured those).  This may be useful for producing a warmer preset. 

[Marsu42]

The x-rite color checker, also contains patches, that are progressively warmer and cooler (not sure if Marsu42 captured those).  This may be useful for producing a warmer preset. 

Yes, I have captured the full x-rite - the cr2 files normal and dual_iso are in https://bitbucket.org/Marsu42/ml-aiso/downloads/dual_iso_samples_6d_60d_FIXED.zip

[Audionut]

as_shot = --wb=graymax
auto = ACR auto WB
measured = measured reading from grey patch with WB tool.

2000k and 50000k are the limits of Adobe.
+/-150 tint is the limit of Adobe.

Code Select Expand

        as_shot     auto        measured
indoor  2950 -11    2850 +0     3000 -9
indoor1 4400 +15    3300 +0     4900 +1
cloudy  5700 +2     6200 +0     5450 +2
sun     4700 +31    5200 +26    5200 +10
fluro   3550 +51    3600 +30    3500 +48


Flash.

Code Select Expand

        as_shot     auto        measured
direct  5150 +6     4600 +0     6600 +3
diffuse 4600 +4     3750 +0     6450 -7
CTB     4700 +4     7500 +0     50000 +22
CTO     2750 +5     2850 +0     2900 +1
+green  5050 +79    4400 +30    5800 +85
SFX1    3000 -40    7500 +0     50000 -150
SFX2    2550 -1     2850 +4     2000 +13
SFX3    2950 -53    2850 +0     2000 -150
SFX4    4200 -8     3250 +0     5100 -20
SFX5    5750 +28    4550 +9     7350 +26


https://www.dropbox.com/sh/kqqswu894uxu5r4/WVVperRfh-

[a1ex]

https://www.dropbox.com/sh/lrutvdhiqmwdwgb/DsPUrbs7E1

The SFX series were quite evil

[a1ex]

Updated the vectorscopes with some funky graphs (Kelvin<->multiplier mapping ported from ufraw).

https://bitbucket.org/hudson/magic-lantern/commits/d53afebd9579
https://www.dropbox.com/sh/uz75fw290v4n7v7/ASbJRAUXk-
https://www.dropbox.com/sh/lrutvdhiqmwdwgb/DsPUrbs7E1

As you know, the raw vectorscope I'm using for finding the white balance is a XY histogram of log2(red) - log2(green) vs log2(blue) - log2(green), where I'm looking for a peak (that is, maximize the number of gray pixels). The overlaid lines represent the location of Kelvin/green values converted to raw multipliers (for AsShotNeutral).

If the Kelvin<->multiplier routines also match Adobe values, these routines will be very useful for converting Canon white balance from kelvin to AsShotNeutral multipliers. This was one of the promises of MLV: correct white balance metadata.

Now, I don't fully understand all this math and I'm not sure if these graphs make any sense. But at least the printed kelvin/green values seem to match ufraw's kelvin/green when you select Camera WB.

Notice the curves are camera-specific; they depend on the color matrix. Try ignoring the color matrix in the Kelvin routine (that is, turn these if (0)'s from kelvin.c into if(1)) and see how the graphs change.

I would also like to know the relationship between ufraw's green balance (which is simply a multiplier applied to the green channel), Adobe's tint and Canon's green-magenta WBShift.

[Marsu42]

Adobe's tint and Canon's green-magenta WBShift.

Yes, that would be quite interesting to know indeed, esp. as the "tint" matter is often left out of the equation and wb is only thought of as "color temperature" even with photogs who are supposed to know stuff.

Concerning your grey wb: Audionut seems to have more time so I won't duplicate his recent chart tests, but I tried a few dual_iso shots and it looks fine so far and makes soft film cure more usable - heads up!

[a1ex]

Yes, that would be quite interesting to know indeed, esp. as the "tint" matter is often left out of the equation and wb is only thought of as "color temperature" even with photogs who are supposed to know stuff.

Indeed. When shooting in daylight you may get away with a neutral tint, but once you move into artificial lights, adjusting tint becomes essential.

With some lenses, even the dominant color of the background leaks onto the main subject. So if you have an animal in the middle of some green grass (you know this one better), you may have to adjust the tint towards magenta.

Also, from the graphs, you can notice the kelvin/green adjustment seems to be ill-defined, that is, you just can't cover the entire mutliplier space (with green normalized to 1) just with these two parameters. This doesn't matter on most practical situations, but only on extreme cases, like Audionut's color filters, or when shooting infrared. I have included an infrared picture in the test series, and its vectorscope looks like this: