CMOS/ADTG/Digic register investigation on ISO

[ayshih]

Are the whites recorded a solid white (16383)? I'd like to see the CR2 files.

The green channels are solid white at 16383; the red channels are at 16065, and the blue channels are at 16001. ...  What needs more investigation is the "rise" before each channel reaches its max with the tweaked settings.

I take that back.  It looks like the overexposed regions are not solid white, and the pixel-to-pixel differences in saturation is what is producing that "rise" in the histogram.  Here are plots of where the pixels have values shy of the overall white level (in one of the green channels), for the default settings (13200..13400) and the tweaked settings (16100..16350).  I'd expect to see just the border of the overexposed region.


There does not appear to be any noticeable banding in the per-column black level when looking at the top black bar.

[a1ex]

I choose the sweet spot as the minimum gain where a fairly large solid area is really solid white and stays at the original white level (except for isolated bad pixels). I don't go below this level (basically a binary search that looks at the output of this function and chooses the lowest gain that still results in the same white level as Canon):

Code: [Select]

/* calibration */
static int raw_center_white()
{
    int xc = (raw_info.active_area.x1 + raw_info.active_area.x2) / 2;
    int yc = (raw_info.active_area.y1 + raw_info.active_area.y2) / 2;
    int w = lv ? 100 : 500;
    int h = lv ? 100 : 500;
    int min = INT_MAX;
    for (int x = xc-w; x < xc+w; x++)
    {
        for (int y = yc-h; y < yc+h; y++)
        {
            /* a little bad pixel filter */
            int p1 = raw_get_pixel(x, y);
            int p2 = raw_get_pixel(x+1, y);
            int p3 = raw_get_pixel(x+1, y+1);
            int p4 = raw_get_pixel(x+1, y+1);
            int p = MAX(MAX(p1,p2), MAX(p3,p4));
            if (p > raw_info.black_level)
            {
                min = MIN(min, p);
            }
        }
    }
    bmp_draw_rect(COLOR_BLACK, RAW2BM_X(xc-w), RAW2BM_Y(yc-h), RAW2BM_DX(2*w), RAW2BM_DX(2*h));
    return min;
}


[hjfilmspeed]

This is amazing!

[a1ex]

Some new graphs:

5D3 and 5D2: I've measured them myself with latest raw_diag. Camera pointed to light bulb, left edge dark to avoid surprises, manual lens to exclude aperture trickery, shutter 1/30. I have simply read the DR number, with and without ADTG trick tuned to sweet-spot (didn't write down noise & other numbers).

Notice that ADTG trick reduces the actual ISO (by definition, it captures more highlights), so it makes sense to plot these new ISOs at the correct X position. This means there's no 0.5 stops improvement, but only 0.35 (what you get at ISO 100).

For 5D3, I've included 160 ISO multiples too. Notice these are not really ISO 160 according to clipping point, so I've estimated the real ones with iso_regs.mo (you can check the formula in the source code).



For the others, I took the DR numbers from DxO (screen) and the calibration values from early testers. Take them with a grain of salt (I did not measure anything here).







The script I used for these graphs, including test data (run it with Octave, might work in Matlab too): dr.m. You are welcome to grab raw_diag and confirm (or infirm) the measurements.

Note the DR in these graphs is per-pixel (so they can't be compared directly if the cameras have different resolutions). If anyone would like me to normalize them (at say 8 megapixels or whatever), let me know.

I don't have calibration data for 600D (need a volunteer). Also I am 99% sure it will work on 1100D and EOS M (need volunteers who can read hexdumps). For 50D and 500D, I will have to take a closer look. 7D might require dual DIGIC trickery (didn't look at it yet).

[Marsu42]

Notice that ADTG trick reduces the actual ISO (by definition, it captures more highlights), so it makes sense to plot these new ISOs at the correct X position. This means there's no 0.5 stops improvement, but only 0.35 (what you get at ISO 100).

Thanks for the clarification, actually I was already wondering about that, but felt too much intellectually challenged looking at all the data to ask :-o ... do you have a table with numbers from the grap with the actual dynamic range gained for each full stop iso?

[a1ex]

Yes, there's a link to the script right below the graphs.

[Marsu42]

Yes, there's a link to the script right below the graphs.

Thanks, I didn't realize I can just open this with notepad and copy/past the data.

[hjfilmspeed]

So, as far as iso is concerned, why did I buy a 5d3 over 5d2?

[Marsu42]

So, as far as iso is concerned, why did I buy a 5d3 over 5d2?

The 5d3 (and more so: 6d) have less banding than 5d2, and this is what kills your dynamic range after lifting shadows much faster than just random noise. But of course as all reviews stated, the 5d3 does just about everything better than the 5d2 - but the sensor generation is the same. If you want high dr high res, get a d800.

[hjfilmspeed]

I know I know but its funny that the charts are showing better DR results with the 5d2. Believe me aside from native DR, I love my 5d3 + ML. I was about to sell and get a D800 but then ML saved me! 5D3 +ML is amazing. With these new ISOs, Im soooo excited.

So as this module evolves, are we going to need to input these settings and fine tune it for our specific cam or will it be more basic like "awesome ML ISO ON or OFF" ha ha either way Im happy!

[l_d_allan]

disregard

[l_d_allan]

That's why I'm very excited that now there's every possibility that ML might conjure up 1/2ev of dynamic range out of thin air ... makes you really, really wonder if there's a hidden catch and why Canon doesn't include this, maybe they don't want to confuse users with too much iso wizardry?

This is great stuff ... 0.3 to 0.5 DR improvement is nothing to sneeze at. This thread reminds me of the classic book from the early 80's, "Soul of a New Machine", except on Internet time at 3x to 10x to 100x faster pace.

Much of this thread is over my head, but I do have a concern that parameters are being optimally tweaked to specific cameras ... A1ex's 5d3 and AudioNut's 6d. Especially with analog, what works on AudioNut's 6d might be sub-optimal compared to the default on my 6d.

Or will the end result at release be tunable to a specific camera, like Auto-Dot-Tune? It might be like AudioNut using Auto-Dot-Tune with his 6d+50mm f1.8 and indicating the appropriate value was -7. Well, yes, on his camera, but unlikely to be optimal on my camera. And maybe even worse than the default of zero.

Or does that not apply?

FWIW: I think in an earlier post on this thread, someone mentioned that an incorrect value for some ATDG register (don't recall the specific) might mess up A-ETTR. Am I the only one concerned about such an issue?

[l_d_allan]

At first glance Dual-ISO looks softer but there is a focus difference between the shots.  This can be seen by examining the "MA" letters near the top left.

I am also impressed by the image of the outdoor porch light. However, the left tube seems to be missing some of the yellow crud at the bottom. Or is that accounted for by the focus difference? The right side yellow crud seems mostly intact.

And would there be a difference between
* ATDG trick + Dual-ISO and
* "just" Dual-ISO?
That would seem to be a fourth combination.

Or do I have a flawed understanding of what is going on?

[a1ex]

So as this module evolves, are we going to need to input these settings and fine tune it for our specific cam or will it be more basic like "awesome ML ISO ON or OF" ha ha either way Im happy!

Right now you enter the low-level settings from iso_regs or adtg_gui, but I'm working on a user-friendly frontend.

Or will the end result at release be tunable to a specific camera, like Auto-Dot-Tune? It might be like AudioNut using Auto-Dot-Tune with his 6d+50mm f1.8 and indicating the appropriate value was -7. Well, yes, on his camera, but unlikely to be optimal on my camera.

It seems there's a small variation between cameras of the same model (I believe it's within +/- 0.1 stops). I'll include a calibration routines (so default settings would be fairly good - just turn it on - but you'll be able to find the sweet spot for your camera if you wish, or dial a custom ISO).

Also, if you shoot JPEG, you may be able to dial the ISO even lower, because with most picture styles, the white point seems to be a little lower than white level in raw.

[ayshih]

5D2 mistery solved!!!

Keyword: SendDataToDfe (it's yet another catgory of registers). Updated ADTG GUI.

For the 50D:

Code: [Select]

        SEND_DATA_TO_DFE_FUNC = 0xffa71598;

I confirmed that each of the four 1d0x registers changes the gain of a fourth of the columns, but I haven't yet done anything more (e.g., generating the analogue of your 5D2 graph).

[a1ex]

Nice. On 50D, from your older results, I believe you can recover all the highlights only via Saturate Offset (and you might even benefit from *increasing* the DFE gain, though it's probably not noticeable).

However, I remember reading 50D has its ISO 100 somehow pulled from 200 (not sure exactly how), so things may be different at ISO 200 and above.

Can you tell me some default values for these DFE registers and send me a RAM dump via PM? (Debug -> Dump ROM and RAM -> RAM4.bin). You may also look them up yourself in the hexdump and only show me the relevant section (e.g. if register 1d02 is 0x456 and you search for 0x1d020456, you will find something that looks like an array of values for these registers; it should be fairly obvious where this array starts and stops).

[Audionut]

However, I remember reading 50D has its ISO 100 somehow pulled from 200 (not sure exactly how), so things may be different at ISO 200 and above.

http://www.magiclantern.fm/forum/index.php?topic=7102.0

[a1ex]

@ayshih, can you write down a table with register values at all the ISOs? (photo mode, of course)

I'm interested especially in CMOS[0] (CMOS gain), C0F08030 (digital gain aka SHAD_GAIN), 0xC0F08034 (BW offset aka SHAD_PRESETUP), C0F0819C (Saturate Offset), and the DFE/ADTG gains. If you notice other relevant registers that are changed with ISO, you may include these too.

(actually I'm interested in a table with these values for all other cameras; simply write down the numbers from ADTG GUI)

[ayshih]

Nice. On 50D, from your older results, I believe you can recover all the highlights only via Saturate Offset (and you might even benefit from *increasing* the DFE gain, though it's probably not noticeable).

However, I remember reading 50D has its ISO 100 somehow pulled from 200 (not sure exactly how), so things may be different at ISO 200 and above.

The low white level of 13432 is the case at only ISO 100.  At higher ISOs, the white level is 15760, so just like with the 5D2, decreasing the DFE gain could be useful.  (This does suggest that ISO 100 is handled differently in some way...)

Can you tell me some default values for these DFE registers and send me a RAM dump via PM? (Debug -> Dump ROM and RAM -> RAM4.bin). You may also look them up yourself in the hexdump and only show me the relevant section (e.g. if register 1d02 is 0x456 and you search for 0x1d020456, you will find something that looks like an array of values for these registers; it should be fairly obvious where this array starts and stops).

As an example, the default values for ISO 100 are:
1d02 = 0x419
1d04 = 0x41a
1d06 = 0x41e
1d08 = 0x41c

I'll send you the array by PM.

@ayshih, can you write down a table with register values at all the ISOs? (photo mode, of course)

Okay, I've already have some partial tables, but I might as well put together a comprehensive table tomorrow.

[g3gg0]

hmm it looks like the DFE is the fpga in 5D2..

[PressureFM]

hmm it looks like the DFE is the fpga in 5D2..

What would that imply?

Sorry if it comes off as being ignorant 

[dmilligan]

trying to get adtg_gui to run on the 1100D, but I'm getting memory errors with CONFIG_GDB=y

Code: [Select]

underflow/overflow[31/0] malloc(0) at rbf_font.c:32, task ml_init

Allocated RAW   694kB, peak 716kB
malloc          157kB, 187kB used
AllocateMemory  946kB, 320 B used
shoot_malloc    30MB, 506kB used
stack_space     505kB

any ideas?

[a1ex]

The binary got bigger than 512K (MemSiz in the compile log). You can disable some debug stuff from features.h.

(there is a check for this, but only on cache-hacked boot, which I'm not quite convinced it's the way to go)

[1%]

Dunno what your total free is but you can try boosting the bin size to 640K or getting more free memory by putting the bvram mirror in another memory area. esp. just to get GDB working.

[dmilligan]

Okay I got GDB working, but adtg_gui fails to load, 'Module Init Failed' is all I can tell you, the console text is so small it's unreadable. I already tried the free_space / 4 thing you mentioned. Free memory is 272K + 894K