Green tint in dual ISOs and cernoice output

[engardeknave]

It looks like the tag that ACR is reading for tint is "AsShotNeutral". (After you change something in ACR, it writes a "tint" tag, but this is apparently the value ACR uses to generate that tag.)

Code: [Select]

exiftool *.dng -asshotneutral

exiftool img_0000.dng -asshotneutral="0.473635 1.05 0.624"
In all the Dual ISO .dngs I looked at so far, the values are always 0.473635, 1, 0.624. As far as correcting the tint, I haven't tried many files yet but I did try a few different numbers for the middle value, which changed the tint in ACR as follows:

1.100 = +27
1.050 = +10
1.010 = -4
1.000 = -8
0.999 = -8
0.990 = -11
0.900 = -46

Then I tried a CeroNoice/dng_validate .tif file. These show up completely green in ACR. To get a ceronoise .tif that isn't green in ACR, do the following (probably will have to change the values for AsShotNeutral):

Code: [Select]

ceronoice img_0001.cr2 img_0002.cr2 img_0003.cr2
dng_validate.exe -3 out out.dng
exiftool out.tif -asshotneutral="0.473635 1 0.624"

[a1ex]

Nice find. Did you try only raw files from 5D2 or also from other cameras?

[Marsu42]

It looks like the tag that ACR is reading for tint is "AsShotNeutral". (After you change something in ACR, it writes a "tint" tag, but this is apparently the value ACR uses to generate that tag.)

Strange, as acr defaults to "adobe standard" for camera calibrarion - well, but as long as it works :-p

It's great you're working on this, dual_iso is great but the tint problem is annoying and needs a lot of time of manual fine-tuning to get an acceptable result, and even then it's just guesswork.

The question is: how do you tell acr what tint (and wb) is correct? Currently when importing a dual_iso dng in LR, the "as shot" option defaults to a fixed nonsense value, and I'm not sure what the "auto" option does even if seems to produce ok-ish results.

[engardeknave]

Did you try only raw files from 5D2 or also from other cameras?

I'm working on .dngs produced by cr2hdr from the 5D2.

Strange, as acr defaults to "adobe standard" for camera calibrarion

Actually, it looks like mine is defaulting to "As Shot".

The question is: how do you tell acr what tint (and wb) is correct?

I have no idea if it's always off by the same amount or not. As I convert more files, I'll learn more. I don't place much confidence in what the camera thinks is the correct value though, even shooting normally.

[Marsu42]

Actually, it looks like mine is defaulting to "As Shot".

I was talking about the camera calibration (at the very bottom of lr dev module) as this is where the "AsShotNeutral" tag belongs - or is this really for the wb (at the very top)... there's no "neutral" option in there? My wb also defaults to "as shot".

[engardeknave]

Oh, I just misread that. I don't use Lightroom.

Did some investigating and the camera profiles "Adobe Standard", "Camera Standard", and "Camera Neutral" all had the same white balance and tint values, with "Embedded" being slightly different. Then I did a exiftool -asshotneutral="0.473635 1.05 0.624". For all those camera profiles, the white balance presets "As Shot" and "Auto" changed while all other remained exactly the same. So ACR appears to calculate the values of "As Shot" and "Auto" using AsShotNeutral regardless of camera profile.


Adobe Standard/Camera Standard/Camera Neutral
As Shot   4800 -8
Auto      5850 +5
Daylight  5500 +10
Cloudy    6500 +10
Shade     7500 +10
Tungsten  2850  0
Flouresc  3800 +21
Flash     5500  0
Custom    5001 +10

Embedded
As Shot   4750 -9
Auto      5800 +4
Other values unchanged


After -asshotneutral="0.473635 1.05 0.624"

Adobe Standard/Camera Standard/Camera Neutral
As Shot   5000 +10
Auto      5850 +5
Other values unchanged

Embedded
As Shot   4950 +9
Auto      5800 +4
Other values unchanged

[engardeknave]

God damn it, I just realized now that I'm getting 4800/-8 for every single Dual ISO.

[Marsu42]

God damn it, I just realized now that I'm getting 4800/-8 for every single Dual ISO.

Oh my, we're making real progress here, I'm always getting 5600/+20 for "AsShot" with the 6d :-p ... "Auto" differs and usually looks ok-ish as I wrote above, but I didn't do any test shots yet to see how dual_iso acr-autowb matches non-dual_iso Canon-autowb or a calibration with x-rite passport. So much still to be done here, doh.

[engardeknave]

I just tried a bunch of tags in some .cr2 files and it seems like none of the obvious suspects except for one has any effect in ACR. That tag would be WB_RGGBLevelsAsShot. This one does vary in each file, and it is indeed copied to the .dng output of cr2hdr. But it doesn't seem to have an effect on .dng files in ACR. AsShotNeutral does not exist in the .cr2. So it would seem we have to translate WB_RGGBLevelsAsShot values into AsShotNeutral values.

In my cr2s the values for WB_RGGBLevelsAsShot are something like: 2564 1168 1168 2445, with the two middle values never varying (over the 8 or so files I looked at). In my dngs, I always get the same values for AsShotNeutral: 0.4736350037 1 0.624.

[Marsu42]

I just tried a bunch of tags in some .cr2 files and it seems like none of the obvious suspects except for one has any effect in ACR. That tag would be WB_RGGBLevelsAsShot. This one does vary in each file, and it is indeed copied to the .dng output of cr2hdr. But it doesn't seem to have an effect on .dng files in ACR. AsShotNeutral does not exist in the .cr2. So it would seem we have to translate WB_RGGBLevelsAsShot values into AsShotNeutral values.

Yes, that does sound reasonable - I also suspect acr somehow computes the "as shot" wb itsself as I also couldn't find any cr2 tag that does the trick. I then gave up, great you're more persistent!
 

In my cr2s the values for WB_RGGBLevelsAsShot are something like: 2564 1168 1168 2445, with the two middle values never varying (over the 8 or so files I looked at). In my dngs, I always get the same values for AsShotNeutral: 0.4736350037 1 0.624.

The obvious questions are:

* What's the algorithm to get a correct AsShotNeutral from WB_RGGBLevelsAsShot (*if* that's the only basis of the computation)?

* Obviously, acr for some reason fails to do this with dual_iso shots and we're getting a fixed/wrong AsShotNeutral, but it's working on non_dual iso shots - why's that?

I admit I'm completely clueless on how to go on from here, I'm certainly not geek enough to figure out how acr computes the value and apply this knowledge to cr2hdr ... which is unfortunate, since the wb/tint problem is serious enough to consider the dual_iso->acr workflow as buggy. One possibility would be to ask a question in some Adobe forum where the acr devs hang out.

Edit: One possible test would be to run acr not on the interlaced dual_iso cr2, but on a half-res dng or modified cr2 with only one iso - maybe then acr would compute the correct AsShotNeutral, and we could simply copy this tag in the real cr2->dng workflow?

[engardeknave]

I also suspect acr somehow computes the "as shot" wb itsself

WB_RGGBLevelsAsShot did affect both "auto" and "as shot".

Obviously, acr for some reason fails to do this with dual_iso shots and we're getting a fixed/wrong AsShotNeutral, but it's working on non_dual iso shots - why's that?

It probably just treats dngs differently.

I admit I'm completely clueless on how to go on from here, I'm certainly not geek enough to figure out how acr computes the value and apply this knowledge to cr2hdr

We change the values of AsShotNeutral in a .dng and watch how it affects temperature and tint in ACR. We do the same thing with WB_RGGBLevelsAsShot in a .cr2. It needs something like +18 tint, which will be easy to figure out without even fundamentally understanding how it works. For now I've been doing +.05 on the middle value of AsShotNeutral. See "After -asshotneutral" in #5 above.

One possible test would be to run acr not on the interlaced dual_iso cr2, but on a half-res dng or modified cr2 with only one iso - maybe then acr would compute the correct AsShotNeutral, and we could simply copy this tag in the real cr2->dng workflow?

The value is already there in the .cr2, and it does get copied to the .dng. Adobe just doesn't use it when it opens a .dng file.

[Marsu42]

0
We change the values of AsShotNeutral in a .dng and watch how it affects temperature and tint in ACR. We do the same thing with WB_RGGBLevelsAsShot in a .cr2. It needs something like +18 tint, which will be easy to figure out without even fundamentally understanding how it works. For now I've been doing +.05 on the middle value of AsShotNeutral. See "After -asshotneutral" in #5 above.

Thanks again for taking this trial & error procedure upon you, I currently don't have the time like in the olden days for something like this :-o

[engardeknave]

My investigation dead-ended when I googled "RGGB" and found a thread on this weird, obscure forum. The problem has either been solved already, or smart people who know math need to take a look at it.

[Marsu42]

My investigation dead-ended when I googled "RGGB" and found a thread on this weird, obscure forum. The problem has either been solved already, or smart people who know math need to take a look at it.

Sorry to hear this, but it isn't unexpected - as I don't understand one word of the linked thread, I'd throw my other suggestion in the ring again: Run only one part of the dual_iso cr2 through acr & copy the resulting wb/tint (which is hopefully correct) to the actual full cr2hdr output...

[engardeknave]

I'm currently using exiftool to bump AsShotNeutral up by a static amount, just so that's it's not a catastrophe if I forget to adjust it. This is fine for my purposes.

[ayshih]

In my cr2s the values for WB_RGGBLevelsAsShot are something like: 2564 1168 1168 2445, with the two middle values never varying (over the 8 or so files I looked at). In my dngs, I always get the same values for AsShotNeutral: 0.4736350037 1 0.624.

Those AsShotNeutral values are the default values in chdk-dng.c, so that's why DNGs all have that fixed white balance (with the conversion to temperatures and tint varying with camera and calibration).  With DNGs, ACR seems to only look at the AsShotNeutral values, so it always sees a fixed white balance for dual_iso DNGs, even if WB_RGGBLevelsAsShot is saying something different.

* What's the algorithm to get a correct AsShotNeutral from WB_RGGBLevelsAsShot (*if* that's the only basis of the computation)?

According to the thread linked above, the proper conversion looks to be straightforward.  The four values for WB_RGGBLevelsAsShot {R, G, G, B} convert to the three values for AsShotNeutral as {G/R, G/G, G/B}.  For example, {2564 1168 1168 2445} converts to {0.4555 1 0.4777}.  I don't know exactly how to handle the two G values if they are different, but if they are more than 1 unit different, something is probably wrong with your camera sensor.

As of November, chdk-dng.c has the function dng_set_wbgain for specifying values different from the default AsShotNeutral values be written to DNGs, so the above conversion should be "easy" to implement.

[a1ex]

Alright, so the algorithm for fixing it should be as easy as reading WB_RGGBLevelsAsShot with exiftool and plugging these numbers in dng_set_wbgain?

(I still don't understand what exactly is WB_RGGBLevelsAsShot: is it the WB setting at the moment of taking the picture, as you have set in Canon menu?)

[ayshih]

I took a regular photo followed by a dual ISO (100/800) photo of the same scene.

Regular CR2, with the camera set to "Tungsten"
The "WB RGGB Levels" values and the "WB RGGB Levels As Shot" values are the same.  Curiously, the "As Shot" values are slightly different from the apparently hard-coded "Tungsten" values.  If I set the camera to "Auto", the "As Shot" values match the "Auto values".

Code: [Select]

WB RGGB Levels As Shot          : 1656 1082 1082 2121
Color Temp As Shot              : 3210
WB RGGB Levels Auto             : 2012 1117 1117 2155
Color Temp Auto                 : 3604
WB RGGB Levels Measured         : 1379 1115 1117 3185
Color Temp Measured             : 2400
WB RGGB Levels Daylight         : 2127 1024 1024 1434
Color Temp Daylight             : 5200
WB RGGB Levels Shade            : 2422 1024 1024 1228
Color Temp Shade                : 7000
WB RGGB Levels Cloudy           : 2275 1024 1024 1317
Color Temp Cloudy               : 6000
WB RGGB Levels Tungsten         : 1657 1083 1083 2125
Color Temp Tungsten             : 3200
WB RGGB Levels Fluorescent      : 1938 1040 1040 1931
Color Temp Fluorescent          : 3793
WB RGGB Levels Kelvin           : 2378 1024 1024 1254
Color Temp Kelvin               : 6730
WB RGGB Levels Flash            : 2310 1024 1024 1301
Color Temp Flash                : 6250
Raw Measured RGGB               : 57436 67183 73377 26751
...
WB RGGB Levels                  : 1656 1082 1082 2121

Regular CR2 converted to DNG using ACR
The "As Shot Neutral" values that are added appear to follow the suggested conversion: 1082/1656 = 0.653382 and 1082/2121 = 0.510137.

Code: [Select]

As Shot Neutral                 : 0.653382 1 0.510137

Dual ISO CR2
Note that the "Auto" values are clearly being thrown off by the ISO switching!  The "As Shot" values match these bad "Auto" values when the camera is set to "Auto" white balance.

Code: [Select]

WB RGGB Levels As Shot          : 1656 1082 1082 2121
Color Temp As Shot              : 3210
WB RGGB Levels Auto             : 2970 1024 1024 1369
Color Temp Auto                 : 7448
WB RGGB Levels Measured         : 5212 4258 582 1465
Color Temp Measured             : 10675
WB RGGB Levels Daylight         : 2127 1024 1024 1434
Color Temp Daylight             : 5200
WB RGGB Levels Shade            : 2422 1024 1024 1228
Color Temp Shade                : 7000
WB RGGB Levels Cloudy           : 2275 1024 1024 1317
Color Temp Cloudy               : 6000
WB RGGB Levels Tungsten         : 1657 1083 1083 2125
Color Temp Tungsten             : 3200
WB RGGB Levels Fluorescent      : 1938 1040 1040 1931
Color Temp Fluorescent          : 3793
WB RGGB Levels Kelvin           : 2378 1024 1024 1254
Color Temp Kelvin               : 6730
WB RGGB Levels Flash            : 2310 1024 1024 1301
Color Temp Flash                : 6250
Raw Measured RGGB               : 79883 98825 796744 289355
...
WB RGGB Levels                  : 1656 1082 1082 2121

Dual ISO DNG
The "As Shot Neutral" values in the dual ISO DNG are just the defaults ("0.473635 1 0.624"), and the white balance is way off.  If I patch it with "0.653382 1 0.510137", the white balance becomes as expected.

Alright, so the algorithm for fixing it should be as easy as reading WB_RGGBLevelsAsShot with exiftool and plugging these numbers in dng_set_wbgain?

So, it looks like this algorithm will work as long as the white balance is not set to "Auto" on the camera.  The "Auto" values are being affected by the ISO switching, and I don't know how to fix it.

[Audionut]

The "As Shot" and "WB RGGB Levels" are the same in both.

Wouldn't it make sense to make the WB from them?

[ayshih]

If you set the camera to "Auto" white balance, you get:

Code: [Select]

WB RGGB Levels As Shot          : 2970 1024 1024 1369
Color Temp As Shot              : 7448
WB RGGB Levels Auto             : 2970 1024 1024 1369
Color Temp Auto                 : 7448
...
WB RGGB Levels                  : 2970 1024 1024 1369

They should be close to "2012 1117 1117 2155".  In other words, the "As Shot" values are bad because the "Auto" values are bad.  There's probably some way to correct the values, possibly by using the difference between the ISOs and/or using the "Measured" values (I have no clue what those are).

[a1ex]

It sounds like when WB is set to auto, cr2hdr should compute WB on its own somehow (after conversion).

Some tests which may give useful clues:

Did you check if converting CR2's to Adobe DNG before running cr2hdr helps? Did you try to copy the exif info from an Adobe DNG file to cr2hdr output?

[Audionut]

It sounds like when WB is set to auto, cr2hdr should compute WB on its own somehow (after conversion).

Waste of time IMO.  There are a million programs that can calculate WB. 
You can create an import preset in LR, that allows you to automatically import images, with LR's own automagic WB, for instance.  Or sunny, cloudy, user defined WB, etc, etc.

Some tests which may give useful clues:

I'll test this later tonight.

[chris_overseas]

Waste of time IMO.  There are a million programs that can calculate WB. 
You can create an import preset in LR, that allows you to automatically import images, with LR's own automagic WB, for instance.  Or sunny, cloudy, user defined WB, etc, etc.

A preset doesn't really help much if I import a large number of photos that are a mixture of dual and non-dual ISO images, some of which were taken with auto WB and some of which were not (that's a common situation for me). It would certainly be more convenient if the auto WB value was correct in at least the DNG output by cr2hdr, if not the dual ISO CR2 itself. Or am I missing a trick here?

[Audionut]

If the correct AsShot values could be interpreted, and plugged into the dual_iso output, that would be one thing.

But to develop code that attempts a WB setting from the image data, this is the same thing that PP tools do.

Also note, there is no correct auto WB.  Either in camera, or otherwise.  It's a best guess based on the image data.

[a1ex]

There's another benefit of this WB correction: when using the new --soft-film setting, you will already have a good guess for WB (either from exif or from some sort of auto), which may be close enough to make the --wb setting fully optional.

Right now, if you want to use --soft-film and --wb, you need 2 passes of cr2hdr: first pass to get a good approximation of WB (can be with --fast, but still, it's an extra step), then adjust WB in ufraw and read the multipliers, then run a second pass for the final render.