[rec_raw] Whitebalance AsShotNeutral in Raw-Header?

[chmee]

the question is mainly in the subject. Are we able to read out the colortemperature or the XY-formed Data? I've seen there is some "reserved" room for some more data..

regards chmee

[a1ex]

It can be read, but there are a bunch of formats, and I don't know how to convert between them.

CHDK DNG code uses RGB multipliers. I don't know how to get them for white balances other than custom.

[chmee]

AsShotNeutral in DNG-Tag-Header is a 3-rational-value-entry - f.e. 0.4145 1 0.6259 (Colortemp ~5500). Playing with 10bit log i see, i cant remove false colors with slightly moving blacklevel, but with changing the whitebalance a.k.a AsShotNeutral-values.. by now i'm not into it as well, but to get some values measured with the first frame would be a good start to reengineer - if the body give some values instead of calculating them from the blacklevel?! if i have read that right, the blacklevel can be a 3-value entry as well, for every component..

(i have to look into the chdk-dng.code)
(btw resolve handles 10bit|log quite well and automatically adjusts ev and whitebalance)

regards chmee

[marekk]

We can get this info from CR2 exif. I think values for Daylight, Shade, Cloudy are fixed.. The question is how to get actual values of "WB RGGB Levels As Shot" and "Color Temp As Shot" directly from the body an convert them to DNG AsShotNeutral format..
 

Code: [Select]

Color Data Version              : 9 (60D/1100D)
WB RGGB Levels As Shot          : 1967 1024 1024 1796
Color Temp As Shot              : 4518
WB RGGB Levels Auto             : 1967 1024 1024 1796
Color Temp Auto                 : 4518
WB RGGB Levels Measured         : 1857 1023 1024 1825
Color Temp Measured             : 4286
WB RGGB Levels Daylight         : 2162 1024 1024 1641
Color Temp Daylight             : 5200
WB RGGB Levels Shade            : 2503 1024 1024 1391
Color Temp Shade                : 7000
WB RGGB Levels Cloudy           : 2330 1024 1024 1507
Color Temp Cloudy               : 6000
WB RGGB Levels Tungsten         : 1531 1024 1024 2497
Color Temp Tungsten             : 3200
WB RGGB Levels Fluorescent      : 1964 1024 1024 2325
Color Temp Fluorescent          : 3807
WB RGGB Levels Kelvin           : 2208 1024 1024 1603
Color Temp Kelvin               : 5380
WB RGGB Levels Flash            : 2405 1024 1024 1494
Color Temp Flash                : 6236

[chmee]

We have a quadrupel of values f.e. daylight:

2162 1024 1024 1641 (RGGB) i think we can merge G1 and G2 into one value. -> so changing into a tripel 2162 1024 1641. if i look into quoted values, we set one value (G) as absolute 1, we set R and B into relation to G.. should work..

(1024/2162) (1024/1024) (1024/1641) -> 0.473635 1 0.624 (5200°K)
(its not far away from the values i wrote above )

But i'm asking for the values AsShot measured right before the recording (or with the first frame). i think, it could be worth, saving them into the raw_rec_header..

(edit) In the chdk-source theres (line110) the generation of the array (3*rational = 6*int) and in save_dng fields [2] and [4] are statically changed (line567/568)

Code: [Select]

static int cam_AsShotNeutral[]          = {1000,1000,1000,1000,1000,1000};
//..
cam_AsShotNeutral[2] = 2477; /* 5D3 6000K */
cam_AsShotNeutral[4] = 1462;
cam_AsShotNeutral[] = {1000,1000,2477,1000,1462,1000};
GRB? instead of RGB?

2477 1024 1462 -> (5D3@6000°K) -> 0.41340 1 0.70041
data from marekks list (60D/1100D) -> 2330 1024 1507 -> 0.43948 1 0.67949

[marekk]

I converted CR2 to DNG using Adobe DNG Converter and it looks like your calculations are good.

CR2:

Code: [Select]

WB RGGB Levels As Shot          : 2341 1024 1024 1608
Color Temp As Shot              : 5615
DNG:

Code: [Select]

As Shot Neutral                 : 0.43742 1 0.636816
-----

Code: [Select]

Color Matrix 1                  : 0.7428 -0.1897 -0.0491 -0.3505 1.0963 0.2929 -0.0337 0.1242 0.6413
Color Matrix 2                  : 0.6719 -0.0994 -0.0925 -0.4408 1.2426 0.2211 -0.0887 0.2129 0.6051
I think that values set for WB RGGB and As Shot Neutral are related to ColorMatrix.
DCRAW has info only about Color Matrix 2 for 60D and I set values for ColorMatrix in raw.c based on dcraw setting. It looks like 60D sets different ColorMatrix than dcraw....

Chdk code for dng reads actual values for WB and saves it as follows:

Code: [Select]

    get_property_case(camera_info.props.wb_adj, &wb, sizeof(wb)); 
    cam_AsShotNeutral[1]=wb[1];
    cam_AsShotNeutral[3]=wb[0];
    cam_AsShotNeutral[5]=wb[2];
If found this info on Adobe forum:

When you have the White Balance set to As Shot, the white reference is usually the "as-shot neutral" (as explained in the DNG spec, "AsShotNeutral specifies the selected white balance at time of capture, encoded as the coordinates of a perfectly neutral color in linear reference space values."). The translation from this so-called AsShotNeutral to temperature & tint values depends on the ColorMatrix tags, which in turn depends on the profile being used.
 
So when you change the profile, you potentially change the color matrix (or matrices) being used, which in turn potentially changes the translation to temp/tint values.
 
If you specify a custom temp/tint for the WB, then you are setting the white reference based on temp/tint values (i.e., effectively xy coordinates), as opposed to the AsShotNeutral.

[chmee]

nice finding. thx.

[CHDK] ah. thats why i ve been unsure about the value-order..

regards chmee

[marekk]

I think we should to set also a:
Calibration Illuminant 1        : Standard Light A
Calibration Illuminant 2        : D65

This two parameters are included for example in Iconoskop DNG files (different values).

Standard Light a is 17, D65 is 21.

[chmee]

i think i've set this two illuminants for the color matrices in my cdng's.. but if we have to recalculate the color matrices  holy boly.. i'm not ready yet, but if its only numbers, we can handle that

[marekk]

I think "As Shot Neutral" in chdk-dng.c is set incorrectly, GRB as you wrote.

If we would like to preserve 6000K it should be:

static int cam_AsShotNeutral[]          = {100000,100000,100000,100000,100000,100000};
cam_AsShotNeutral[0] = 41340;
cam_AsShotNeutral[4] = 70041;

I tested it with daylight settings for 60D's DNG and there is a huge difference.
Default settings:
https://www.dropbox.com/s/u5cdmfkhgp0idgr/def.jpg

Modified:
https://www.dropbox.com/s/ay3qdivya3h1ajc/wb.jpg

i think i've set this two illuminants for the color matrices in my cdng's.. but if we have to recalculate the color matrices  holy boly.. i'm not ready yet, but if its only numbers, we can handle that

[chmee]

brilliant. thats a huge step forward. but we have to wake up 1%, a1ex or g3gg0 for implementing the measured data

regards chmee

[IliasG]

i think i've set this two illuminants for the color matrices in my cdng's.. but if we have to recalculate the color matrices  holy boly.. i'm not ready yet, but if its only numbers, we can handle that

No need to recalculate, you can find them ready in exif data of DNG converted CR2s or in Adobe's standard dcp profiles for each model.
In fact Dcraw matrices is a copy of Adobe's D65 matrices.

[IliasG]

brilliant. thats a huge step forward. but we have to wake up 1%, a1ex or g3gg0 for implementing the measured data

regards chmee

I didn't read the DNG specs but as I understand it the "As shot neutral" tag contains the as shot multipliers and not the measured ones. If the camera WB setting is at auto those two are the same, but if we have manually set WB there can be a great difference. I would prefer both but if the available space is only enough for one I vote for the measured WB.

[a1ex]

These numbers are camera-specific, right?

For getting the current WB setting, please see reply #1.

[chmee]

@a1ex is it possible to simulate one photo-shot (without a pictureoutput) right before the movie-recording begins? saving the exifs from this shot as separated file with the same name as the raw-file?

Example:
M15-1212.raw
M15-1212.exif

or much simpler
M15-1212.raw
M15-1212.CR2

(merging of CR2-Metadata then in the converter)


regards chmee

[marekk]

I shot some raw movies in Prague last week and now I'm trying to process tens of shots and it's really difficult to manually set white balance for each movie...

[chmee]

I#m working on that..

[g3gg0]

its implemented in the new .mlv version of raw_rec

[chmee]

g3gg0 point us to the thread or some infos on that!
lol. found. (whitebalance. is it or will it be implemented?)
http://www.magiclantern.fm/forum/index.php?topic=7122.0

regards chmee

[a1ex]

Only the easy part is implemented (saving the internal WB info from Canon). The hard part (how to put that info in a DNG) is not

[chmee]

im ready to write a mlv2cdng converter

[chmee]

so, i did a trial in raw2cdng v1.3.0 with raw-files. all experience will be taken into the mlv-converter..
http://www.magiclantern.fm/forum/index.php?topic=5618.250

regards chmee

[Jip-Hop]

There's this feature request for Auto White Balance in MLV App: https://github.com/ilia3101/MLV-App/issues/168. But it would be great if the MLV files already came with the measured white balance when the camera is set to Auto White Balance. So I figured the in-camera measured white balance must be in the exif data. I thought perhaps I can make my EOS M take a picture before starting RAW video recording. So I can have access to the measured white balance, and apply it from the photo to the video. Which is exactly what chmee suggested.

@a1ex is it possible to simulate one photo-shot (without a pictureoutput) right before the movie-recording begins? saving the exifs from this shot as separated file with the same name as the raw-file?

Example:
M15-1212.raw
M15-1212.exif

or much simpler
M15-1212.raw
M15-1212.CR2

(merging of CR2-Metadata then in the converter)


regards chmee

Seems like you guys almost cracked this puzzle in 2013! Where did it go from here? Would love to know if this turned out to be impossible or still worth looking into.