CMOS/ADTG/Digic register investigation on ISO

[a1ex]

So, what's all this stuff about "sensor update"?
Just a small improvement in dynamic range in photo mode (around 0.3...0.5 0.8 stops). We were able to fine-tune the amplifier gains in order to squeeze a little more highlight detail.





Graphs for the other cameras can be found here: http://www.magiclantern.fm/forum/index.php?topic=10111.msg97780#msg97780

[February 23] 5D3 reaches nearly 0.8 stops of improvement: 11.77 EV at ISO 115, and also a new ISO 66.
[January 17] 5D2 reaches 11.92 EV of dynamic range at ISO 81.

Only 0.5 EV? That's way too small!
Yes. You may take a look at Dual ISO (which will get an additional 0.3...0.5 stop boost with this "sensor update"; also resolution issues were largely solved), or you may consider switching to Nikon.

Wait a minute, that means less noise, right?
Well, it means you get a little more detail in highlights. This doesn't mean less noise per se (the new ISOs will be just as noisy in shadows as the old ones), but it will let you shift the exposure to the right by 1/3 ... 2/3 EV and collect more photons. This will result in lower noise.

For example, on 5D Mark III I could lower the ISO by 0.37 stops from 100, resulting a new ISO 77. After some more tweaking, I've got 0.6 stops below ISO 100 => ISO 66.

Sample images?
No relevant samples yet, sorry.

How exactly are you getting more highlight detail compared to Canon firmware?
The signal from the sensor seems to be amplified in 2 stages: a CMOS amplifier (which operates in full stops - powers of 2 - and we have tweaked it when implementing Dual ISO) and an ADTG amplifier which can be configured in finer increments. After these two stages, the signal is digitized (with an ADC), probably tweaked digitally, and saved to CR2. We have noticed the ADTG amplifier tends to run a little "hot" (that means, it gets saturated a little too early - nothing to do with temperature).

To get the extra highlight detail, one has to reduce the gain for the ADTG amplifier until the ADC will no longer be saturated. At this point, the white level (maximum recorded level in the raw file) will begin to decrease and no more detail will be recovered (since now the CMOS itself or the CMOS amplifiers will get saturated instead).

To play with these gains, scroll down to the research tools section.

Does this mean Canon did not fully optimize their sensor for low noise?
I'd say they simply left a safety margin in their code to make sure the ADC is always saturated (that is, to make sure white is always recorded as white).

How are intermediate ISOs implemented in Canon cameras?
I will try to answer this question in a detailed paper (including how exactly I've reached the conclusion).

For now, you may take a look at these graphs: https://www.dropbox.com/sh/onppbwy44fqomxa/P75rs6pgTW



What's with that message about f1.2 lenses?
According to DxO, Canon applies a digital ISO boost at wide apertures (about 0.5 stops on APS-C cameras, and 0.2 stops on FF from my findings) to compensate for light loss. Since the digital gain is burned into the CR2 files, at wide apertures you will lose a small amount of highlight details (under 0.1 stops on FF, did not check on APS-C). Also you may get some extra noise from round-off errors, and if your raw editor does not handle white level properly, you may lose even more highlights (details here).

We are going to fix this by turning off the digital gain.

Any side effects?
None yet.

What about Dual ISO?
You will be able to use both tweaks (it will just work).

What about video?
I've got only 0.1 stops of improvement on 5D3, but didn't try too hard.

What's the current state?
Research. We are trying to optimize the parameters that influence ISO, understand their effects (did we really gain 0.5 stops of DR or are we just daydreaming?) and port the results on other cameras. Scroll down for some research tools.

Will it work on my camera?
- If you have a 550D or newer camera, it will most likely work.
- If you have a 7D, no idea yet.
- If you have a 5D Mark II, 50D or 500D, don't get too excited. I've barely got 0.15 stops of improvement on 5D2. Mystery solved!
- Confirmed working on 5D3, 550D, 600D, 650D, 700D, 60D, 6D, 5D2, 50D and 500D.

How can I help?
- Play with the research tools below and report your findings.
- If you have access to laboratory equipment and you can measure the real ISO, you can help us check the validity of our ISO theory.

Where's the download link?!??!?!!??!!!!!!!!!!!!!!!!!!?!?!?!?!?!
Take it easy, the current state is research. As in, "If we knew what it was we were doing, it would not be called research, would it?"

But if you have some basic coding/math skills - enough so you can follow the entire discussion without getting dizzy - I have some nifty research tools for you:

raw_diag.mo (source)- cross-platform, anyone can run it

This tool does technical analysis on the raw image data (black/white levels, noise, dynamic range, SNR curve):


Alternative: RawDigger (proprietary, nonfree).

iso_regs.mo - 5D3 only, requires either the crop_rec_4k build, or a custom ML build from the iso-research branch:

A research tool (or hacker's tool if you prefer) that lets you change most ISO-related parameters on 5D Mark III only and study their effect. Details here.


adtg_gui.mo - cross-platform, requires CONFIG_GDB=y a custom ML build from the iso-research branch (on 5D3, the crop_rec_4k build works fine as well):

The good old ADTG/CMOS tool updated to also intercept DIGIC (ENGIO) registers.



For sources, please check the branch iso-research on the main repository.

When it will be released?
When it's ready. I also want to summarize the findings in a small paper (like the Dual ISO PDF), so I need a little time.

Any recommended reading?
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html
http://www.clarkvision.com/articles/evaluation-canon-5diii/index.html (main site: http://www.clarkvision.com/articles/ )
http://www.dxomark.com/About/In-depth-measurements/Measurements/ISO-sensitivity





Original message:

Now some fun stuff:

Compile ML with CONFIG_GDB=y and load the adtg_gui module. Override the column gain registers (0x8882-0x8888) to roughly half the original value.

My results (with 978bf96 and these changes; shutter 1/30):
Unaltered ISOs:

Code: [Select]

ISO    ADTG default   white level    noise stdev    DR
100    0x41b          15282          6.62           10.96
160    0x435          13306          5.53           10.99
200    0x54c          15282          6.83           10.92
250    0x435          15282          8.55           10.60
1600   0x454          15282          11.87          10.12

ISOs altered with ADTG gain overriden to 0x250:

Code: [Select]

ISO    ADTG overriden white level    noise stdev    DR
100    0x250          11911          3.68           11.38
160    0x250          9935           3.25           11.24
200    0x250          11924          3.88           11.31
250    0x250          11917          3.84           11.33
1600   0x250          11915          6.46           10.58

So... we can get almost half-stop of DR just by tweaking some registers?! Looking forward to see the RawDigger results.

[Audionut]

Keen to try the results when I get home.  I would be happy with an indicator to notify for true overexposure vs guessed.  That's probably a bit ugly though.

[a1ex]

Well, I think I found a possible bug. If the image is not overexposed, and max stops at say 13000, and there aren't 10 pixels of the same value to confirm that maximum, my algorithm will assume a white level of 12000 (or whatever the initial guess was).

If the heuristic will take the unconfirmed max if there's no confirmed one, it might be a little better. Also, the safety margin probably can be made smaller (100? need to experiment).

The main problem is that I don't know how to tell for sure whether the image is clipped or not (only through heuristics).

[Audionut]

Can you discard for hot pixels and average the next bunch?  Maybe some weighting towards the max values.

Or discard as hot pixels if their value is above 16000. 

[a1ex]

Yes, something like averaging between 5th max and 50th max might work pretty well. Or maybe just the 5th max minus a small amount.

If you play with the algorithm, be sure to test it at 30-second exposure. This was the main reason for a big safety margin (if you over-estimate the white level by just 1 unit, auto ETTR will no longer work properly).

[Audionut]

Quick test before bed.

Default

Code: [Select]

ISO Saturation S.Dev SNR DR
100 16383 7.14 2294          11.21
1/4th shutter.  ML reported the same saturation level, or there abouts.  To tired to be testing properly.

ADTG tweak

Code: [Select]

ISO Saturation S.Dev SNR DR
100 13045 4.04 3329          11.73

I also noticed that if you only adjust 1 bank of registers, the saturation level differs (lower) for the G2 channel.  My first thought, highlight recovery.  Why blow both green channels and rely on post to pull detail from RB, when you can keep actual detail in one of the G channels.

edit:  Of course, that would only work if the ADC clips before the pixel wells saturate.

11.73 stops before dualISO.  You'll be spanking Nikon soon!

[a1ex]

I think there are 8 columns where you can tweak the gains.

I also think ADC does clip off a little earlier before the pixel wells saturate. With this tweak, notice that DR at 160/200/250 is now almost identical, with slight advantage towards 250 (differences being probably from quantization error in the noise).

[Audionut]

I think there are 8 columns where you can tweak the gains.

Yeah, they're in 2 batches of 4.

I also think ADC does clip off a little earlier before the pixel wells saturate.

Well, it actually looks to be around 0.5EV.  0.5EV increased exposure before saturation (tweaked vs normal). Needs further testing, but shows that not only do we gain 0.5EV extended range in the shadows, we also gain 0.5EV increased SNR through the entire exposure.  It's going to need your magic on the white level, there's pink highlights in saturated tweaked ISOs.  I should probably look at raw2dng, but I'm much to sleep deprived for that atm!


Looks like you nailed that register too.  Going lower continues to drop the noise, but the white level drops at a rate where DR remains equal (or there abouts).  Not to mention that you get to a point where even with full sensor saturation, there isn't enough gain to deliver any signal.  I'd be interested to find out what the measured ISO is with 0x250.  DxO suggests measured ISO is 80 for camera ISO 100, which suggests Canon choose the register value as a compromise between noise and mapping the maximum value to 14bits.

[Audionut]

With this tweak, notice that DR at 160/200/250 is now almost identical, with slight advantage towards 250 (differences being probably from quantization error in the noise).

I need to brush up on quantization.

As expected, intermediate ISOs like 160 or 250 do not cause any changes in ADTG/CMOS con-figuration. These ISOs are obtained by applying some digital gain to the raw data acquired at the nearest full-stop ISO, and this gain is configured from the DIGIC register 0xC0F08030 (SHAD_GAIN).

A different register again?  Digital and (0x8882-0x8888) being analog?

Makes you wonder what they were thinking with ISO 50.  They could have made that below 100 ISO useful ffs.  dmilligan seems to think decisions like this is par for the course.   

[Greg]

For 500D:

Code: [Select]

ADTG3[0] = 0x149 (default) -> 0x0 (trick)
ADTG3[3] = 0x14a (default) -> 0x1 (trick)


[a1ex]

A different register again?  Digital and (0x8882-0x8888) being analog?

Yes. I'm just not sure how to override it in photo mode on 5D3. On 5D2 it works (add raw_blinkies.o in the Makefile) but there's little or no change to DR numbers. It could be used to remap the values back to 14 bits, but we need to check the histogram for gaps.

So, the ADTG gain may be yet another amplifier stage?

In LiveView (raw video + silent pics in movie mode), the behavior is different. ISO 160/200/250 result in no difference at all in the raw data (so the raw histogram doesn't change at all). Instead, the JPEG is developed with different gains, and these gains are adjusted from 0xC0F08030. I read this register in photo mode to figure out how to adjust the white point for correct clipping warnings, and this register is also used for gradual exposure, for example.

[Audionut]

Looks like the ADC is clipping 0.7EV before sensor saturation...................

edit:  deleted graphs and conclusion since it was way off the mark.

[Greg]

F1.4 T1/4s ISO 100 (-3EV in ACR) :


F1.4 T1/4s ISO 200 HTP + ADTG trick (-3EV in ACR) :

[Audionut]

Signal on the left of the image is throwing the ML calculations out of whack.

Images such as this.

https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/_46A5167.CR2

ML reports,

Mean: 2048
Dev:   8.30
Max:   15463

RawDigger shows the Std.Dev as 6.52, same Mean and Max values.

If I measure right to the image edge in RawDigger, I still see a low S.D value.  If I measure from the very top left of the optical black, RawDigger reports S.D of 7.34.

If I make sure the left side of the image contains near black, ML doesn't have the issue.

edit:  I'm being lazy, I haven't even looked at your code.  I'll head off and do that now, and test some other means if I understand how it's being coded.

Code: [Select]

    if (raw_info.active_area.x1 > 10) /* use the left black bar for black calibration */
    {
        autodetect_black_level_calc(
            16, raw_info.active_area.x1 - 16,
            raw_info.active_area.y1 + 20, raw_info.active_area.y2 - 20,
            3, 16,
            &mean1, &stdev1
        );
        autodetect_black_level_calc(
            16, raw_info.active_area.x1 - 16,
            raw_info.active_area.y1 + 22, raw_info.active_area.y2 - 20,
            3, 16,
            &mean2, &stdev2
        );
    }
I don't understand what values are shifting the scanning area, and what values are setting the scanning size.  I'd like it to scan almost the entire optical black area leaving a few pixels off left and right, and more pixels top and bottom.

[SpcCb]

@a1es: Sorry for late reply; as dmilligan said, RON is Read On Noise.

I pointed you this because in conventional photography (I mean short exposure at 'temperate' temperature), when we speak about SNR, N (noise) is quasi resumed to RON. So where we get the smaller RON we get the best SNR. Of course I suppose S (signal) is the same to compare.

About DR, I'm surprised what you use an offset at 500 (?). Unit is not ADU?
Typical offset is 2048 ADU for 5D3, 1024 ADU for 5D2 and 128 ADU for 5D1.
I presume you know it, but in case; these values are to ensure sigma from RON + FPN will not produce clipped pixels because of low values. Of course, most the offset is high, most you can expect high RON + FPN levels, and less DR.

I find your studies very interesting because it's sure all of that could be optimised, it should be possible to get a DR gain (not 5EV of course) and a better RON, FPN, FWC and detectability.

[Audionut]

I think there are 8 columns where you can tweak the gains.

ADTG2 adjusts blue and green channel 1
ADTG4 adjusts red and green channel 2

Would have been nice to dial the green sensitivity back, bring it more into line with the sensitivity of the RB channels.I'm playing with adjustments to each column, but it doesn't look like it will bring any benefit to the table.

I pointed you this because in conventional photography (I mean short exposure at 'temperate' temperature), when we speak about SNR, N (noise) is quasi resumed to RON. So where we get the smaller RON we get the best SNR. Of course I suppose S (signal) is the same to compare.

Increased SNR from reduced noise assumes the signal stayed the same.  This ADTG tweak is reducing the noise more then the signal for instance.

About DR, I'm surprised what you use an offset at 500 (?). Unit is not ADU?

That was for the white level that ML uses.  As a1ex mentioned, if ML uses a white level value above the actual peak white level in the image, things get broken.

[Audionut]

edit:  fixed graphs.   All test images taken with 1/13s shutter and within 2 minutes, so temperature influence should be minimal.





For dynamic range I averaged the white level of each ISO set.  With normal ISO 3200 and Canon pulled ISO 3200, the white level was a little higher then normal so these were not averaged and were calculated separately.  I'll paste the data in numbers a little later.

Looks like the ADC is clipping 0.7EV before sensor saturation.

I was a little overzealous.  0.3EV is closer to the truth.

[a1ex]

Code: [Select]

        autodetect_black_level_calc(
            16, raw_info.active_area.x1 - 16,
            raw_info.active_area.y1 + 20, raw_info.active_area.y2 - 20,
            3, 16,
            &mean1, &stdev1
        );

I don't understand what values are shifting the scanning area, and what values are setting the scanning size.  I'd like it to scan almost the entire optical black area leaving a few pixels off left and right, and more pixels top and bottom.

Look at the definition of autodetect_black_level_calc:

Code: [Select]

static void autodetect_black_level_calc(int x1, int x2, int y1, int y2, int dx, int dy, int* out_mean, int* out_stdev)

and read it as "scan from x1 to x2, from y1 to y2, with downsampling factors dx and dy".

So, it has a safety margin on 16 pixels horizontally, on both sides.

However, the optical black size according to dcraw is 124, in ML it's assummed to be 126+20. This is where the bug comes from.

I changed skip_left and skip_right to 120+16 and it seems fine now (stdev the same with both dark frame and fully blown-out image, and no black bars on zebras). Need to run the tests again.

(just as with white level, the skip offsets should be overestimated a tiny bit, but not too much)

[a1ex]

Some more findings:

1) 0x8880 is black level. Changing it doesn't seem to bring any improvement (DR decreases a little, not sure why).

2) The sweet spot seems to be where the white level is still the same as with the unmodified ISOs. Here it's around 0x320 => the real ISO under 100 might be ISO 80.

3) As you know from the dual ISO PDF, you can override the ISO gains from CMOS[0]. Now check the dynamic range of ISO 100 overriden to 1600, and ISO 1600 overriden to 100 (with tweak disabled). Here, the DR is roughy 0.1 EV higher whenever you select 100 in Canon menu.

This means dual ISO should work better as 100/1600 than as 1600/100. Anyone confirmed this experimentally? (though it's quite unlikely to notice a difference of 0.1 stops)

At ISO 1600, the difference vanishes when enabling the tweak. At ISO 100, the difference is still present, and it gets up to 1 stop when you select 25600 in Canon menu and 100 as override. So, there are some other settings used for these higher ISOs (and finding them may result in some more parameters that can be tweaked).

[SpcCb]

(...)
3) As you know from the dual ISO PDF, you can override the ISO gains from CMOS[0]. Now check the dynamic range of ISO 100 overriden to 1600, and ISO 1600 overriden to 100 (with tweak disabled). Here, the DR is roughy 0.1 EV higher whenever you select 100 in Canon menu.

This means dual ISO should work better as 100/1600 than as 1600/100. Anyone confirmed this experimentally? (though it's quite unlikely to notice a difference of 0.1 stops)

At ISO 1600, the difference vanishes when enabling the tweak. At ISO 100, the difference is still present, and it gets up to 1 stop when you select 25600 in Canon menu and 100 as override. So, there are some other settings used for these higher ISOs (and finding them may result in some more parameters that can be tweaked).

Some month ago I noticed some strange things about DR, black & white levels; it was when using high ISO recovered by low ISO in Dual ISO. But it is with a 5D2, not a 5D3.

[a1ex]

White level in LiveView is set to 15000 as a one-size-fits-all code. If the true white level is higher than that, it can't be wrong by more than 0.14 stops. If the true white is 15600, the error is roughly 0.07 stops.

You can always squeeze this extra DR with exiftool.

[Audionut]

1) 0x8880 is black level. Changing it doesn't seem to bring any improvement (DR decreases a little, not sure why).

Raising it looks to improve things here.  Raising it with Canon ISO seems to clip the RGBG channels earlier, but you can offset it with (0x8882-0x8888) 


All values below RGBG. 
Normal ISO
(0x8882-0x8888)
(0x8882-0x8888)+(0x8880)


Optical black average/SD

Code: [Select]

2048.0/7.27  2047.8/7.22  2048.0/7.10  2048.17.30
2047.9/5.63  2047.9/5.54  2048.1/5.49  2048.05.62
3932.5/5.59  3932.3/5.52  3923.8/20.4  3932.5/5.55    //Blue channel borked in OB
Here, I measured the levels/SD in the RGB patches in a colorchecker passport.  R value from R patch and so on.

Code: [Select]

2297/28.9  4374/45.0  1741/25.7  4344/44.6
1778/22.8  3339/33.7  1330/20.1  3354/34.8
1894/23.3  3454/31.0  1424/19.9  3469/32.0

Here I measured the RGBG levels/SD from the yellow colorchecker patch. (Brighter patch)

Code: [Select]

7962/50.1  10992/70.6  3724/24.5  10943/71.9
6633/39.3  8922/55.3   3336/19.0  8946/55.0
8624/39.1  10905/49.8  5313/18.8  10931/50.7
With the blue channel giving strange reading in the OB, I measured the black colorchecker patch.

Code: [Select]

2413/12.7  2683/20.8  2361/11.9  2885/20.0
2309/9.89  2610/15.9  2286/9.0   2548/15.8
4109/10.6  4541/14.5  4260/9.45  4585/15.4
Biggest difference looks to be in the green channel.  Wonder if there are other registers that perform a similar action?
This register is adjusting OB level, so would appear to be digital. 

[a1ex]

From what I've read in the ADTG datasheet (look in the dual ISO PDF at page 15), it's done by a feedback loop (there is a DAC somewhere inside). So I think black level is analog.

So... raising the black level could reduce the noise in midtones?

[Audionut]

It looks to increase the SNR through the entire exposure.  I guess we can exploit it by reducing the gain (highlight biased?) with (0x8882-0x8888) and preventing saturation.

These maybe of interest:

http://documents.stsci.edu/hst/acs/documents/handbooks/DataHandbookv6/acs_Ch42.html
http://documents.stsci.edu/hst/acs/documents/handbooks/DataHandbookv6/acs_Ch43.html
http://www.stsci.edu/hst/wfc3/documents/handbooks/currentDHB/wfc3_Ch52.html

Rodger Clark has Gain e/DN values.  Here for the 5D3: http://www.clarkvision.com/articles/evaluation-canon-5diii/


Also, the standard (0x8882-0x8888) values seem to vary slightly on startup.  Haven't nailed down what causes the changes. 

When adjusting, I've been keeping the slight offsets in each column rather then setting all values the same.

[Audionut]

I've been trying to figure out how increasing the black level via that register is keeping a similar/smaller Standard Deviation.

It must be clamping the black value up to 2048, and hence by clamping it up further (stronger feedback), it reduces noise further.

Negative feedback loops
    When the fed-back output signal is out of phase with the input signal. This occurs when the fed-back signal is anywhere from 90° to 270° with respect to the input signal. Negative feedback is generally used to correct output errors or to lower device output gain to a pre-determined level. In feedback amplifiers, this correction is generally for waveform distortion reduction or to establish a specified gain level. A general expression for the gain of a negative feedback amplifier is the asymptotic gain model.

Positive feedback loops
    When the fed-back signal is in phase with the input signal. Under certain gain conditions, positive feedback reinforces the input signal to the point where the output of the device oscillates between its maximum and minimum possible states. Positive feedback may also introduce hysteresis into a circuit. This can cause the circuit to ignore small signals and respond only to large ones. It is sometimes used to eliminate noise from a digital signal. Under some circumstances, positive feedback may cause a device to latch, i.e., to reach a condition in which the output is locked to its maximum or minimum state.