CMOS/ADTG/Digic register investigation on ISO

[Audionut]

Yes.

The raw composites of this image.

[Greg]

500D with ADTG on live view also has a constant vertical banding.
Maybe 500D need new fabric calibration 

[Audionut]

Some further analysis following the bayer pattern.

Code: [Select]

ab cd ef gh
0 RG RG RG RG
1 gb gb gb gb
2 rg rg rg rg
3 GB GB GB GB
4 RG RG RG RG
5 gb gb gb gb
6 rg rg rg rg
7 GB GB GB GB
8 RG RG RG RG
Column a - row 0 starts at pixel position 122 from left, and 80 from top.
Optical black finishes at 121 pixels from left, and 79 pixels from top.  However, pixel row 79 contains some amount of signal.

Column a is ADTG4[8888]
Column b is ADTG2[8884]
Column c is ADTG4[8884]
Column d is ADTG2[8888]
Column e is ADTG4[8886]
Column f is ADTG2[8882]
Column g is ADTG4[8882]
Column h is ADTG2[8886]

8882 looks to only effect a single column.  8884/8886/8888 look to have a predominate effect on the listed column, and a smaller effect on all other columns.

Further details to follow.

ADTG2/4[8,9,A,B] is also more complex.

[SpcCb]

Maybe it can help: OB zone contains multiple Reference Areas.

Here is crops, top-left, at zoom 4x with color marks.
White mark corresponding to the standard output image.
Orange responding like the standard output image but does not get light.

5D2:


5D3:


Note: It needs to be confirmed, however OBRA of the 5D2 should be similar on the 50D and 550D (maybe other from the same generation). And OBRA of the 5D3 should be similar on the 6D.

[a1ex]

These zones are definitely helpful (I need this data for cr2hdr too). Thinking to add a display mode in raw_diag to show exactly this cropped view, and some statistics for each zone.

Anyone can help me counting the pixels of each zone and write down their sizes?

888x: didn't check offline, but if you try to bump each one in LiveView, you get a brighter vertical line in the Magic Zoom window (and all of them behave in the same way). I believe the order is ADTG2[8882], ADTG4[8882], ADTG2[8884], ADTG4[8884], ADTG2[8886], ADTG4[8886], ADTG2[8888], ADTG4[8888].

At least, if you bump the first 2, you get a thicker vertical, and if you bump the first 4, you get 50% dark and 50% bright stripes.

When checking these gains, it can be useful to turn off demosaicing. For offline analysis, I prefer dcraw -4 -E folllowed by octave.

[Audionut]

Anyone can help me counting the pixels of each zone and write down their sizes?

I'm not entirely sure what you mean by "zone".

Would it be best to sample the first 8 columns of the image?

Some other useful things, WL for each color channel.
Also, histogram of the top stop, or top two stops of exposure.

[g3gg0]

according to analog devices' manuals for CCD ADTGs, they have a more or less fixed use case.
the CCD readout happens in a serial way - you scan pixel for pixel in every line from top to bottom.

top optical black lines:
usually more than 10 lines to calibrate black level before sensor readout happens.
result is used for optical black clamping (= moving analog "zero" to a specific voltage)

left optical black pixels:
as they come right before the pixels in the current line, they are used to update current line black level for this line.

so now a question raises...
if we use the output of the AD converter which is in those OB areas the output of a closed loop filter,
can we be sure to get something useful from it?
i say "no, not without paying attention and knowing what we do"

why?
check the datasheet of e.-g. AD9992 from ADI, which is the manufacturer canon uses for their customized ADCs/ADTGs

there you see that the analog value from the CCD goes into a correlated double sampling path which will measure
sensel's "empty" and "exposed" values and gives out the delta for further processing.
this value then is added to a variable voltage to push "black" a bit higher that 0V. (=black clamp)
thats the black level we know from raw, just in analog domain.
this variable voltage is estimated by averaging OB areas and adjusting it to keep black level at some target value.

in OB digital data we get not just the charge of the sensels, but also the correction value that updates.
as closed loop filters usually oscillate and converge to the target value slowly, the first few DIGITAL OB lines would be trash and only reflect the filter's learning phase.
same for the first few OB pixels in every line.

conclusion:
throw away the first n lines/pixels of OB areas.
how much is n? i dont know, it depends on the loop parameters in the chip.
ADI recommends 20 OB pixels and 10 OB lines so i would say:
 -> skip top 10 lines and the leftmost 40 pixels

[g3gg0]

additional note:
check the topmost lines in raw data of above images. it shows the filter learning its black clamp
randomly oscillating line brightness, so clamp is updated per-line.
seems to match my theories.

can someone average all pixels in the OB lines? (all pixels, no matter which color)
these values plotted should show a converging oscillation.

(i have to watch cartoons right atm....)

[a1ex]

can someone average all pixels in the OB lines? (all pixels, no matter which color)
these values plotted should show a converging oscillation.

Dark frames at ISO 100 and 100/1600, 1/30, without any other tweaks applied (I had them saved from the first experiments on dual ISO).



Octave script:

Code: [Select]

function top_ob(im, name, outfile)
top = im(1:80,:);
top_ex = im(1:120,:);
figure
plot(mean(top_ex'), 'r'), hold on
plot(mean(top'))
legend('Active area', 'Top OB', 'location', 'southeast')
title(name)
xlabel('Line number')
ylabel('Raw average (per-line)')
set(gca, 'position', [0.18 0.18 0.7 0.7])
grid on
print("-dpng", "-r70", outfile)
end

octave:31> top_ob(im100_1600, "ISO 100/1600", "top_ob_100_1600.png")
octave:32> top_ob(im100, "ISO 100", "top_ob_100.png")

To load the raw data, you need to run "dcraw -4 -E foobar.cr2" first, then load the pgm in octave with imread.

Also note that with dual ISO, in the OB area, the higher ISO always ends up with the lower OB values (more graphs in the dual ISO PDF). I used this trick to autodetect which lines are dark and which are bright in the dual ISO preview experiment (not in cr2hdr); it worked very well, except for 100/200 where the difference was too weak and the algorithm got confused by noise. This behavior should give a hint about how the feedback loop works.

I'm not entirely sure what you mean by "zone".

Those rectangles highlighted by SpcCb. Inside one zone, the noise pattern is the same, but each zone has its own noise pattern (see g3gg0's theory about different clamping modes).

(i have to watch cartoons right atm....)

Good idea

[a1ex]

Some interesting autocorrelations regarding vertical FPN (5D3, ISO 100 pulled from 200):

Code: [Select]

function plot_xcov(x)
x = xcov(x, 49, 'coeff');
stem(x)
end

# average each column
octave:2> b = mean(im);

# look for autocorrelations
octave:11> plot_xcov(b), print -dpng -r70 0.png
octave:12> plot_xcov(b(1:8:end)), print -dpng -r70 1.png
octave:13> plot_xcov(b(2:8:end)), print -dpng -r70 2.png
octave:14> plot_xcov(b(3:8:end)), print -dpng -r70 3.png
octave:15> plot_xcov(b(4:8:end)), print -dpng -r70 4.png
octave:16> plot_xcov(b(5:8:end)), print -dpng -r70 5.png
octave:17> plot_xcov(b(6:8:end)), print -dpng -r70 6.png
octave:18> plot_xcov(b(7:8:end)), print -dpng -r70 7.png
octave:19> plot_xcov(b(8:8:end)), print -dpng -r70 8.png

# standard deviation of the vertical FPN
octave:22> std(b)
ans =  0.73383
octave:23> std(b(1:8:end))
ans =  0.75365
octave:24> std(b(2:8:end))
ans =  0.75451
octave:25> std(b(3:8:end))
ans =  0.63608
octave:26> std(b(4:8:end))
ans =  0.65589
octave:27> std(b(5:8:end))
ans =  0.73094
octave:28> std(b(6:8:end))
ans =  0.79068
octave:29> std(b(7:8:end))
ans =  0.75369
octave:30> std(b(8:8:end))
ans =  0.69111

# standard deviation of the entire dark frame
octave:35> std(im(:))
ans =  3.1371



First graph: notice the autocorrelations at lags multiple of 16.
Subsequent graphs: notice the autocorrelations in columns 0, 4, 5 and 6 (modulo 8 ).
Also notice the lower stdev in columns 2, 3 and 7 (modulo 8 ). Not sure if relevant.

Why modulo 8? because there seem to be 8 parallel processing channels (there are 8 gains in the 888x set) and each of these gains affects one column in a group of 8.

You draw the conclusions.

[Audionut]

My first guess would be that 8882 are masters of some description, but that doesn't explain columns 5 and 6.

Can you compare with Canon default?  Or with one of mine

Canon pulled ISO 200
ADTG gain -16
[fe] - 3
[8,9,A,B] - 2

Mine also contains the faint banding type noise.  Does iso_regs scale [8,9,A,B], keeping the deltas between them?  If not, then I guess that is the reason why I got the banding noise.  Else, a value of 2 is obviously to low for my 5D3.

[Greg]

This register seems to associated with constant banding.
500D C0f0[4908]

[ayshih]

left optical black pixels:
as they come right before the pixels in the current line, they are used to update current line black level for this line.
...
in OB digital data we get not just the charge of the sensels, but also the correction value that updates.
as closed loop filters usually oscillate and converge to the target value slowly, the first few DIGITAL OB lines would be trash and only reflect the filter's learning phase.
same for the first few OB pixels in every line.

Here are some plots that I posted much earlier in this thread here, with accompanying explanation here, that show the effectiveness of the per-line black-level clamp for a normal exposure and a massive overexposure in the 50D.  Each curve on the plot is the histogram of a pair of columns in the left optical-black region, with the top curve adjacent to the actual image region.



You can see both oscillation and convergence, even for the normally exposed image.

[SpcCb]

These zones are definitely helpful (I need this data for cr2hdr too). Thinking to add a display mode in raw_diag to show exactly this cropped view, and some statistics for each zone.

Anyone can help me counting the pixels of each zone and write down their sizes?

(...)

For 5D2:

Code: [Select]

blue: 0;4 5792;26
green: 0;31 158;33
yellow: 158;31 5792;32
red: 0;34 158;37
cyan: 0;37 78;3804
orange: 78;37 158;3804
For 5D3:

Code: [Select]

blue: 0;5 5918;31
green: 0;32 122;41
yellow: 122;32 5918;41
red: 0;42 122;47
cyan: 0;47 122;56
orange: 0;56 122;3950
Edit: Add 5D3, but for other cameras I don't have RAW files.
Note: Coordinates origin is top-left(0;0).

[a1ex]

Does iso_regs scale [8,9,A,B], keeping the deltas between them?

Yes. It considers the first one as reference (that one will get the value from menu) and all the others as delta (so, when you set it to 0 or 1 or maybe even 2 or 3, it may underflow and cause strong banding). I preferred this underflow instead of clamping to 0 because in this way you see when you went too far.

However, ADTG gains 888x are not scaled in iso_regs (these are simply set all of them to the same value).

Can you compare with Canon default?

With my ISO 100 dark frame from early dual ISO experiments:



Didn't expect this change; need to check a few more dark frames to be sure.

Or with one of mine

The autocorrelation analysis was performed on a dark frame, not on a regular image.

[Audionut]

Yes. It considers the first one as reference (that one will get the value from menu) and all the others as delta (so, when you set it to 0 or 1 or maybe even 2 or 3, it may underflow and cause strong banding). I preferred this underflow instead of clamping to 0 because in this way you see when you went too far.

However, ADTG gains 888x are not scaled in iso_regs (these are simply set all of them to the same value).

In this case, I should be safe with a value of 2, since the minimum value for the other registers, is within 2 of the first one.
Also, I was scaling 888x with mini-iso.

I will search for the cause of the problem.

[Audionut]

Didn't expect this change; need to check a few more dark frames to be sure.

Canon ISO 100

Canon pulled ISO 200
ADTG gain -16
[fe]  3
[8,9,A,B]  4 (to be safe)

The autocorrelation analysis was performed on a dark frame, not on a regular image.

Sorry, I thought OB would be sufficient.

[ayshih]

Anyone can help me counting the pixels of each zone and write down their sizes?

For 5D2:

Code: [Select]

blue: 0;4 5792;26
green: 0;31 158;33
yellow: 158;31 5792;32
red: 0;34 158;37
cyan: 0;37 78;3804
orange: 78;37 158;3804
For 5D3:

Code: [Select]

blue: 0;5 5918;31
green: 0;32 122;41
yellow: 122;32 5918;41
red: 0;42 122;47
cyan: 0;47 122;56
orange: 0;56 122;3950
Edit: Add 5D3, but for other cameras I don't have RAW files.
Note: Coordinates origin is top-left(0;0).

Don't forget that there can be indexing differences between the CR2 files and the buffer that ML uses (that can be dumped with RAW_DEBUG_DUMP).  For example, ML intentionally skips the buffer ahead one line for both the 5D2 and the 50D to put the red pixel on even-parity lines, but there could be other discrepancies too.

[a1ex]

The DNG buffer can be also dumped with raw_diag (no need to edit the source code; it's in the menu).

I'd say it's a good idea to align the internal DNG buffer to the output CR2 (+/- 1 line/column for forcing the active area starting with a red pixel); should make it easier if one wants to repeat some OB analysis with PC-based tools and compare it to raw_diag.

[SpcCb]

Don't forget that there can be indexing differences between the CR2 files and the buffer that ML uses (that can be dumped with RAW_DEBUG_DUMP).  For example, ML intentionally skips the buffer ahead one line for both the 5D2 and the 50D to put the red pixel on even-parity lines, but there could be other discrepancies too.

Of course area coordinates I provided are from full original Canon CR2. ML features like dual_iso or mini_iso was not used.

[a1ex]

Updated raw_diag to show OB areas on all 4 sides, like this (no zones defined yet):



Interesting thing: the in-camera raw buffer is 16 pixels larger than the CR2 one (as reported by dcraw, exiftool and Adobe DNG), so there seems to be a small part of OB which is not saved (or at least I couldn't find it in the CR2). Note the right margin has 18 pixels, and to get a tight fit I had to use skip_left = 122 (dcraw uses 124), so my best guess is that 2 pixels from the right OB will get shifted back to the left side before saving the CR2, and the remaining 16 are discarded.

Now I'd like to ask you to help with something really easy:
1) grab raw_diag.mo from first post, enable "Optical Black zones", take a picture and post the screenshot (you'll find it under raw_diag directory on your card).
2) if you have some coding skills (or, to be more precise, text editing and command-line skills), fine-tune the photo offsets for your camera in raw.c, like I did right now for 5D3; the active area in raw_diag should look (more or less) like in my screenshot.

Exact matching with CR2 is hard because a struct raw_pixblock should be aligned in memory at 16 bits, which means the raw buffer can be shifted horizontally only in multiples of 16 pixels. So, don't strive for that (approximate match is OK).

[SpcCb]

(...)

Now I'd like to ask you to help with something really easy:
1) grab raw_diag.mo from first post, enable "Optical Black zones", take a picture and post the screenshot (you'll find it under raw_diag directory on your card).
2) if you have some coding skills (or, to be more precise, text editing and command-line skills), fine-tune the photo offsets for your camera in raw.c, like I did right now for 5D3; the active area in raw_diag should look (more or less) like in my screenshot.

(...)

Here is for 5D2:



Note: I had a problem with the last raw_diag.mo, get a error with previous ML build (2014-02-04_ad):

Code: [Select]

Linking..
tcc: error: undefined symbol 'camera_model_short'
[E] failed to link modulesBut it works with the last nighty (2014-03-03).
Beside mini_iso (2014-01-31) no longer works with these last builds, I get the same error.
Looks like it's since 'camera_model_short' became private (?).

[ayshih]

Here's the optical-black screenshot for the 50D:


I've double-checked the skip offsets, and skip_top and skip_left are as small as they can be before a gap appears in the raw zebras.

[a1ex]

@ayshih: it's fine; there may be roundoff errors from raw_get_gray_pixel or the code that uses it. You probably tried to decrease skip_top by 2 and got a top bar in zebras => just leave it like this.

@SpcCb: that colorful palette is likely a bug in the screenshot code (another reason to rewrite it) or in the file copying code. Reproduced it.

Regarding linking errors:

camera_model_short became private because most modules were checking only the camera model name, but not the firmware version. This could cause trouble if somebody loads a module expecting a different firmware version (the upgrades in progress are 5D3 113->123, 7D 203->205 and 700D 111->113). So, to force all modules do proper checking, I made the old way obsolete.

mini_iso is undergoing major rewrite (one of the reasons it's not yet published, besides the research part being in progress and new findings being made). You can apply all these tweaks via ADTG GUI (but with more clicks) and from iso_regs (but that one is 5D3-only) and I want the new module to just work rather than having to fiddle with it (it will be no longer a research tool). I'll refactor 70% of it as a reusable library for patching things around in Canon code/data areas, and publish this one first.

Once I'll publish mini_iso, I expect people trying to port it on the other cameras. I want this to happen after the theory is fully confirmed, documented and working at least on two cameras from different generations. Reasons:
- to make sure the implementation is indeed portable (unlike iso_regs which is hardcoded for 5D3)
- to have some instructions for porting (there are much more addresses and constants to be found, compared to say dual_iso, and some of them can't be found by pattern matching without understanding how things work)
- I don't want to throw away the porting work every time the theory gets changed.

The current research tools from first post should work on current nightly (binaries are up to date).

[Audionut]

and I want the new module to just work rather than having to fiddle with it (it will be no longer a research tool).

Depending on the complexity with different camera models, a simple on/off, sounds like a good idea.