Green tint in dual ISOs and cernoice output

Started by engardeknave, January 31, 2014, 11:00:49 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

ayshih

What'd I'd like is for alternating frames of regular shots and dual ISO shots to have the same white balance, even if the in-camera white balance is inaccurate.  When using tools like the Lightroom plugin, using dual ISO then becomes very low "friction" for the casual user.

Quote from: a1ex on March 18, 2014, 06:36:39 PM
Did you check if converting CR2's to Adobe DNG before running cr2hdr helps? Did you try to copy the exif info from an Adobe DNG file to cr2hdr output?

If I convert the dual ISO CR2 directly to a DNG using ACR, it's clear that Adobe just divides the "WB RGGB Levels As Shot" values (i.e., the algorithm suggested above).  The correct "As Shot Neutral" values should be close to "0.653382 1 0.510137".  Thus, Adobe is not being "smart" in any way, and we need to fix the "As Shot" values.

As Shot Neutral                 : 0.344781 1 0.747991
...
WB RGGB Levels As Shot          : 2970 1024 1024 1369
Color Temp As Shot              : 7448
WB RGGB Levels Auto             : 2970 1024 1024 1369
Color Temp Auto                 : 7448
WB RGGB Levels Measured         : 5160 4224 582 1457
Color Temp Measured             : 10719


As additional information, the "Auto" white balance information for regular shots taken at ISO 100 and ISO 800 is nearly the same (and correct), so the bad "Auto" values in the dual ISO shot is not just a straightforward mixing of the two.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

ayshih

Ah, slight correction.  Adobe may not be looking at the "WB RGGB Levels As Shot" values directly.  The CR2 has tags "Red Balance" and "Blue Balance", clearly calculated from the "WB...As Shot" values, that can be inverted to obtain the "As Shot Neutral" values.  That is, if we figure out how to fix the "WB...As Shot" values, we also need to recalculate these "*Balance" values to make sure that the white balance information is consistent in the CR2.

From a diff, here's the list of tags that may be useful for the fix, if there's any hope of not looking at the image data itself.

Regular CR2 (ISO 100)

Measured RGGB                   : 858 1024 1024 389
WB RGGB Levels As Shot          : 2009 1117 1117 2147
Color Temp As Shot              : 3611
WB RGGB Levels Auto             : 2009 1117 1117 2147
Color Temp Auto                 : 3611
WB RGGB Levels Measured         : 1376 1115 1117 3177
Color Temp Measured             : 2400
Raw Measured RGGB               : 151580 192577 185025 69494
WB RGGB Levels                  : 2009 1117 1117 2147
Blue Balance                    : 1.922113
Red Balance                     : 1.798568


Dual ISO CR2 (ISO 100/800)

Measured RGGB                   : 185 1024 1024 675
WB RGGB Levels As Shot          : 2970 1024 1024 1369
Color Temp As Shot              : 7448
WB RGGB Levels Auto             : 2970 1024 1024 1369
Color Temp Auto                 : 7448
WB RGGB Levels Measured         : 5160 4224 582 1457
Color Temp Measured             : 10719
Raw Measured RGGB               : 99230 126060 976974 359041
WB RGGB Levels                  : 2970 1024 1024 1369
Blue Balance                    : 1.336914
Red Balance                     : 2.900391



Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

ayshih

I should point out the hard-coded white balance on dual ISO shots is not a green tint on the 50D, but rather the shots are awash in magenta.  In ACR, the temperature is 4850, and the tint is +46.  Even if we don't make any headway on reconstructing or recalculating a proper white balance, the hard-coded defaults in cr2hdr ought to be model-specific.

Edit: corrected the temperature/tint numbers
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Audionut

Quote from: a1ex on March 19, 2014, 12:33:34 PM
There's another benefit of this WB correction: when using the new --soft-film setting, you will already have a good guess for WB (either from exif or from some sort of auto), which may be close enough to make the --wb setting fully optional.

Good point.

Do you think there are digic registers involved?  If so, can you lead me down a path that doesn't involve hand checking each register.

a1ex

I'm not sure how Canon's Auto WB algorithm works, but... can you check if it depends on the picture style sharpness setting?

(just to confirm a wild guess)

Audionut


a1ex

Alright, so it's likely to be computed from raw data directly. (Audionut, you did the test with a dual ISO file, right?)

I don't know what exactly it's doing, but if it's doing an average or median of some pixels, and we know which these pixels are, it might be possible to find out in which direction it biases and undo that bias.

It may be as simple as their algorithm (WBINTEG strings in Canon firmware) is averaging different ratios of dark/bright pixels for the 3 color channels; if that's true, all we need to know to perform the correction would be these ratios.

So, it may be helpful if somebody could take pairs of images, like ayshih did, of the same scene at regular ISO vs dual ISO (for example, ISO 400 vs 100/1600, or whatever you think it's useful). Try a few more scenes in different lighting (to cover a large part of the white balance range). Then write down a table like this:


RGGB regular           RGGB dual ISO (as shot, for both columns)
1,2,2,3                4,5,5,6
7,8,8,9                10,11,11,12
...


Once we have such a table, we can try to look for patterns.

Audionut

I tested both Canon and dual_iso.
I tested just sharpness settings 1 and 7 previously.  Can someone confirm with other sharpness values?

I'm seeing the same AsShot numbers for dual_iso (with Canon Auto WB), regardless of the (color) temperature of the scene.

a1ex

Sharpness: 1 and 7 are enough, no more tests needed.

As Shot test (5D3, Auto WB):


WB RGGB Levels As Shot          : 1935 1024 1024 1664
Color Temp As Shot              : 4902
WB RGGB Levels As Shot          : 1678 1024 1024 2897
Color Temp As Shot              : 3200

WB RGGB Levels As Shot          : 1473 1024 1024 2405
Color Temp As Shot              : 3200
WB RGGB Levels As Shot          : 1446 1024 1024 2351
Color Temp As Shot              : 3200

WB RGGB Levels As Shot          : 2217 1024 1024 1978
Color Temp As Shot              : 4710
WB RGGB Levels As Shot          : 1691 1024 1024 2842
Color Temp As Shot              : 3242


Shots 1,3,5: ISO 400.
Shots 2,4,6: ISO 100/1600.
1,2: outside, 3,4: tungsten light, 5,6: computer monitor.

These were handheld, so don't rely on their accuracy.

ayshih

Some exploratory math using my dual ISO (100/800) shot!

Raw Measured RGGB

99230 126060 976974 359041

My guess is that these are straight sums of a subset of the pixels in the raw data.  In fact, given how the second G value is nearly 8 times the first G value (the ISO ratio!), it looks like the sum is not mixing the two ISOs together (this may be different on cameras other than the 50D).  Thus, the first two values (RG) are for ISO 100, and the second two values (GB) are for ISO 800.

Measured RGGB
Now to see if I can reproduce the values reported in "Measured RGGB" ("185 1024 1024 675").  Averaging the two G values in "Raw Measured RGGB",

99230 551517 551517 359041

and then renormalizing so the two G values are 1024

184 1024 1024 667

It's very close!

WB RGGB Levels Measured
Of course, the above isn't particularly useful for our objective here.  Can we reproduce "WB RGGB Levels Measured" ("5160 4224 582 1457")?  If I divide the "Raw Measured RGGB" into 5.3e8 (a magic number), the result is

5341 4204 543 1476

The match is good, but not great.  There may be some hidden, channel-dependent corrections that need to be applied, which would complicate any approach that starts with the raw data.

WB RGGB Levels Measured (corrected)
What can we do if we start with just "WB RGGB Levels Measured" ("5160 4224 582 1457")?  We can renormalize R and B values such that their respective G values are 1116 (you'll see why)

1363 1116 1116 2794

This should be compared against the "WB RGGB Levels Measured" values from the regular shot ("1376 1115 1117 3177").  It's rather close, with the discrepancy in the GB value pair almost certainly due to the fact that the dual ISO shot has saturation (the GB pair is for ISO 800, remember).

WB RGGB Levels Auto
I have no guesses on how to reproduce "WB RGGB Levels Auto" ("2970 1024 1024 1369").  The same procedure would probably take the corrected "WB RGGB Levels Measured" to the corrected "WB RGGB Levels Auto" (i.e., something close to "2009 1117 1117 2147").  From Google searching, it looks like, on some cameras, the values for "WB RGGB Levels Measured" and "WB RGGB Levels Auto" can be the same, but that is definitely not the case on the 50D.  There looks to be an additional transformation needed.  Can you guys compare these tags for your shots?
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Audionut

In autoWB mode, I'm seeing the same values in,

WB RGGB Levels AsShot
WB RGGB Levels Auto
WB RGGB Levels Measured

5D3.

Marsu42

Quote from: Audionut on March 19, 2014, 04:55:12 PM
I'm seeing the same AsShot numbers for dual_iso (with Canon Auto WB), regardless of the (color) temperature of the scene.

Yup, that's the problem, what really hurts is that the tint value (magenta/cyan) isn't measured as the organge/blue wb is much more subjective and doesn't matter that much.

Audionut

I'm pretty sure that all color settings are derived from the RGB values.  And it's the funky RGB values that are throwing off the G/M calculations.

ayshih

Quote from: Audionut on March 20, 2014, 06:55:21 AM
In autoWB mode, I'm seeing the same values in,

WB RGGB Levels AsShot
WB RGGB Levels Auto
WB RGGB Levels Measured

5D3.

Hmm, even on dual ISO shots?  Can you post the tags like the following from one of your dual ISO shots?

50D Dual ISO CR2 (ISO 100/800)

Measured RGGB                   : 185 1024 1024 675
WB RGGB Levels As Shot          : 2970 1024 1024 1369
Color Temp As Shot              : 7448
WB RGGB Levels Auto             : 2970 1024 1024 1369
Color Temp Auto                 : 7448
WB RGGB Levels Measured         : 5160 4224 582 1457
Color Temp Measured             : 10719
Raw Measured RGGB               : 99230 126060 976974 359041
WB RGGB Levels                  : 2970 1024 1024 1369
Blue Balance                    : 1.336914
Red Balance                     : 2.900391


Is the ISO pattern on the 5D3 GBRG or RGGB?  The 50D is GBRG.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Audionut

Yes, both types.

Here, dual_iso
MeasuredRGGB                    : 505 1024 1024 523
ColorDataVersion                : 10 (1DX/5DmkIII/6D/70D/100D/650D/700D/M)
WB_RGGBLevelsAsShot             : 1893 1024 1024 2260
ColorTempAsShot                 : 4044
WB_RGGBLevelsAuto               : 1893 1024 1024 2260
ColorTempAuto                   : 4044
WB_RGGBLevelsMeasured           : 1893 1024 1024 2260
ColorTempMeasured               : 4044
RawMeasuredRGGB                 : 36328 68077 67802 34351


No WB RGGB Levels, or Blue/Red Balance.  But there is these

WhiteBalanceRed                 : 0
WhiteBalanceBlue                : 0

ayshih

After a bunch of investigation, here is my current state of understanding (long post!).

How Canon determines white balance
(1) Choose a subset of the sensor pixels.  More on this later.

(2) Add up the (black-level-subtracted) ADC values from this pixel set.  These values are recorded as "Raw Measured RGGB", e.g.:

Raw Measured RGGB               : 99230 126060 976974 359041

In this 50D dual ISO shot, the RG pair appears to only sample the primary ISO (100) while the GB pair appears to only sample the recovery ISO (800), which strongly indicates that there is line skipping going on in step (1).  If the ADC values are not being scaled down in this sum, then it could mean that the white balance is being computed from as few as ~100 pixels.

(3) Divide the previous values into large number that is proportional to the number of pixels in that channel (essentially, this is the inverse of the average ADC value).  These values are recorded as "WB RGGB Levels Measured", e.g.:

WB RGGB Levels Measured         : 5160 4224 582 1457

Because the dividend appears to be channel-dependent, I suspect that Canon masks out a subset of the pixels in each channel (possibly those that fall below a certain threshold) to keep them from biasing the white balance.

(4) Transform the "WB RGGB Levels Measured" values into a temperature ("Color Temp Measured") and a tint (not recorded in the CR2).

Color Temp Measured             : 10719


(5) If the temperature is outside of some "acceptable" range, choose a better temperature through some algorithm.  This value is recorded as "Color Temp Auto", e.g.:

Color Temp Auto                 : 7448

That is, Canon decided that 10719 K was way too hot, and set a new temperature of 7448 K.  As an alternate example, from a regular shot in tungsten lighting:

Color Temp Measured             : 2400
Color Temp Auto                 : 3611

Canon decided that the "Measured" temperature was too cold, and increased the temperature.  I suspect this is why I typically can't rely on AWB for tungsten lighting.

(6) Using the (corrected) temperature and the previously determined tint, which has not been changed, invert the transforms in step (5).  These values are stored in "WB RGGB Levels Auto", e.g.:

WB RGGB Levels Auto             : 2970 1024 1024 1369

Even when the temperature is not changed, the "WB...Measured" and "WB...Auto" values can be slightly different, which suggests that this transform-inverse-transform step always takes place.

(7) Assuming the camera is set to AWB, copy these values to "WB RGGB Levels As Shot"

WB RGGB Levels As Shot          : 2970 1024 1024 1369
Color Temp As Shot              : 7448

and, on the 50D at least, compute "Red Balance" and "Blue Balance"

WB RGGB Levels                  : 2970 1024 1024 1369
Blue Balance                    : 1.336914
Red Balance                     : 2.900391


White balance approaches for dual ISO shots
The convenient approach is to use the EXIF information to construct a better white balance.

  • For cameras like the 5D3, we may be fortunate that the RGGB values are always determined from one ISO (as opposed to mixing them together), and so you'll just have to set ISOs accordingly.  For example, 100/800 might have correct white balance, while 800/100 does not.  If the ISOs are mixed, then all balances will be thrown off by saturation, with no obvious way to correct them.
  • For cameras like the 50D, it looks to be "easy", because you can just renormalize "WB RGGB Levels Measured" (see my earlier post).  One color balance (i.e., red or blue) will be correct, and the other one may be somewhat off due to saturation.  Note that using these corrected "WB RGGB Levels Measured" values for "WB RGGB Levels As Shot" values will circumvent the sanity check that Canon applies to the temperature.
In either case, the "WB RGGB Levels As Shot" values need to be converted in cr2hdr and fed into dng_set_wbgain.

The alternate approach is to estimate the white balance in cr2hdr, since we're looking at all of the raw data anyway.  We would just need to mimic steps (1)–(3) and (7), and then feed the values into dng_set_wbgain.  It'd be very difficult to replicate steps (4)–(6) because we don't know what Canon's transforms and algorithm are.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Marsu42

Respect, I'd never have figured that out! I admit I still don't understand the difference between shooing 100/800 and 800/100 on 5d3 (and supposedly 6d), but I hope your calculations will go into cr2hdr asap as this is the last real issue with dual_iso.

Quote from: ayshih on March 22, 2014, 09:43:01 PM
The alternate approach is to estimate the white balance in cr2hdr, since we're looking at all of the raw data anyway.

Very interesting idea as you seem to think this might fix Canon's faulty awb "intelligence", but I'd vote to at least add a default option to make the dual_iso temperature/tint the same as normal shots for easy mixing both types in post.

chmee

@Audionut
its the nature of "looking good", its subjective..

mathematical accurate wb isnt always the best choice, neither from the body nor the raw-converter. indoor pictures often have two whitebalanced points :) so. i wouldnt care about fiddling with manual adjusting wb-values in dng/cr2-tags. we talked about it longer time ago for raw_rec. its ok, if its near the good wb-point. my 2 cent.

http://www.magiclantern.fm/forum/index.php?topic=6779.0
[size=2]phreekz * blog * twitter[/size]

ayshih

Okay, I just put in a PR for the EXIF-based solution: https://bitbucket.org/hudson/magic-lantern/pull-request/453/white-balance-fix-for-dual-iso/diff

On my 50D, it gets in the right ballpark, but if one ISO is significantly overexposed, the white balance can get biased.  Still, it's definitely a lot better than the hard-coded values!

Folks should test this on different models!
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Audionut

Excellent work.

Quote from: ayshih on March 22, 2014, 09:43:01 PM
(4) Transform the "WB RGGB Levels Measured" values into a temperature ("Color Temp Measured") and a tint (not recorded in the CR2).

If it's not recorded, then it must be determined by the raw convertor.  And idea might be to test various convertors and see if they produce the same tint value.  If they all produce the same value, either, there is documentation describing the process of calculating tint, or the value is indeed recorded in the exif.

Will be a few hours before I can test your pull request on 5D3.


Quote from: chmee on March 22, 2014, 11:58:01 PM
mathematical accurate wb isnt always the best choice,

From a subjective standpoint, almost never :) 
I do like to see an accurate WB from a color neutral target though.  It provides a baseline.  For some reason or another, I always tend to go to yellow with skin tones.

Marsu42

Quote from: ayshih on March 23, 2014, 01:21:20 AM
Folks should test this on different models!

The good news: on 6D with 100/800, the color temperature is near to the non-dual_iso wb.

The bad news: the tint is still the same and completely elsewhere with ACR, in this case magenta ... here are the samples, no artistic value but just out of my window :-)

original cr2: https://bitbucket.org/Marsu42/ml-aiso/downloads/IMG_8666.CR2
dual_iso cr2 same exposure: https://bitbucket.org/Marsu42/ml-aiso/downloads/DUAL8668.CR2
dual_iso cr2 -2ev: https://bitbucket.org/Marsu42/ml-aiso/downloads/DUAL8669.CR2


ayshih

Quote from: Audionut on March 23, 2014, 03:56:12 AM
If it's not recorded, then it must be determined by the raw convertor.  And idea might be to test various convertors and see if they produce the same tint value.  If they all produce the same value, either, there is documentation describing the process of calculating tint, or the value is indeed recorded in the exif.
I suspect each converter has its own camera-specific profile to interpret its own temperature and tint from the white balance numbers.  It's clear that the camera internal transformation differs from the ACR profile because ACR reports a different temperature than what is recorded in "Color Temp As Shot".

Quote from: Marsu42 on March 23, 2014, 09:13:33 AM
The good news: on 6D with 100/800, the color temperature is near to the non-dual_iso wb.

The bad news: the tint is still the same and completely elsewhere with ACR, in this case magenta ... here are the samples, no artistic value but just out of my window :-)
Thanks for testing, but it looks like my fix isn't doing anything at all with 6D files!  You're still getting stuck with the default white balance (which translates to 5600/+20 in ACR).  That's because your CR2 files are missing nearly all of the EXIF tags I've been studying, with the only one left being "Measured RGGB".  I can extend the fix to cover this situation, but I just want to confirm that you aren't somehow stripping those tags.  This exiftool reference table suggests that those tags should be there.

Edit: the problem turned out to be an outdated exiftool
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

ayshih

Ugh, I've updated the PR, but you're not going to like the results.  According to Lightroom, the temperature/tints for your files are:
IMG_8666.CR2  5900  +8
DUAL8668.DNG  4400  +42
DUAL8669.DNG  5000  +48

The hard-coded values (5600/+20) just happen to be close to the real white balance, whereas trying to get the white balance from "Measured RGGB" is doing fairly bad here.  For the 50D, my earlier calculations showed that "Measured RGGB" was much worse to use than "WB RGGB Levels Measured" because the G channels (at different ISOs) would get averaged together, while the R and B channels were at different ISOs.  It doesn't look like that's the case on the 6D, but it's hard to tell without the other tags!

For comparison of the effectiveness of the fix, when I can use "WB RGGB Levels Measured", here are some numbers from my 50D:
IMG_0689.CR2  5450  +13
DUAL0688.DNG  5700  -4

A little bit off, but certainly better than the hard-coded values (4850/+46).

I'd like to see some results from the 5D2/5D3 folks.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Audionut

Your update gets close to the dual_iso CR2.
However, the WB of the dual_iso CR2 is different to normal ISO.

Also, auto WB in LR produces the same WB/tint values for normal ISO, and dual_iso.CR2, but is always different for the dual_iso.DNG.

5D3
Normal ISO
https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_46A9808.CR2

dual_iso
https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_46A9809.CR2

ayshih

Quote from: ayshih on March 23, 2014, 04:01:29 PM
That's because your CR2 files are missing nearly all of the EXIF tags I've been studying, with the only one left being "Measured RGGB".  I can extend the fix to cover this situation, but I just want to confirm that you aren't somehow stripping those tags.  This exiftool reference table suggests that those tags should be there.

Ah!  My exiftool was fairly old, and apparently couldn't read the white balance information on newer cameras (6D, 5D3, ...).  With an updated exiftool, here's the comparison of using three different tags to get the white balance information.

Marsu42's files (6D):

IMG_8666.CR2  5900  +8   (normal)
DUAL8668.CR2  4400  +42  (Measured RGGB)
              4700  +29  (WB RGGB Levels Measured)
              4150  +25  (Raw Measured RGGB)
DUAL8669.CR2  5000  +48  (Measured RGGB)
              4700  +29  (WB RGGB Levels Measured)
              4400  +27  (Raw Measured RGGB)


Audionut's files (5D3):

_46A9808.CR2  4350  +48  (normal)
_46A9809.CR2  4800  +45  (Measured RGGB)
              3400  +42  (WB RGGB Levels Measured)
              4400  +28  (Raw Measured RGGB)


My files (50D):

IMG_0689.CR2  5450  +13  (normal)
DUAL0688.CR2  ----  --   (Measured RGGB, not usable)
              5700  -4   (WB RGGB Levels Measured)
              4250  +15  (Raw Measured RGGB)
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x