CMOS/ADTG/Digic register investigation on ISO

[timbytheriver]

@names_are_hard Thank you for your comprehensive answer. I will have to reflect awhile on the information to understand it fully.

@Audionut Thanks! I will explore the 'SET' button option. I am conscious that there are thousands of registers as yet unknown! So maybe the values I am trying to find are amongst those.

**UPDATE**
Used adtg_gui to spy registers: > 'Changed since now' and 'ISO related'
If anyone is looking for these values they report as follows:

Code: [Select]

ADTG24[7] = 0x0
ADTG24[8] = 0x5a
ADTG24[9] = 0x5a
ADTG24[a] = 0x5a
ADTG24[b] = 0x5a
ADTG24[1b] = 0x320

They've no doubt been discussed before – but possibly way back in the thread.

[Audionut]

In adtg_gui.c there are no hardcoded values for registers.  adtg_gui.c is looking up the values (as currently set by Canon) and displaying them to the user.  From what I can gather, however adtg_gui does that is your answer.

iso_regs is an old attempt to bring register tweaking to a more user friendly level.  It's 5D3 only and contained hard coded values because it was useful for the specific task.  Those values were determined from looking at what adtg_gui displayed.

They've no doubt been discussed before – but possibly way back in the thread.

You haven't read the entire thread.....shame. 

Might find something useful here too: https://www.magiclantern.fm/forum/index.php?topic=6751.0

[timbytheriver]

Thanks! More reading! lol

For the record: I have indeed read this entire thread through – twice. It just takes a long time to filter through me!

[timbytheriver]

Looking at your code, I think you have an integer overflow bug when you're setting registers using 0x400 + reg_x pattern.  Because the addresses you're assigning to are later considered to be 2 bytes wide, and menu input + the constant could make this higher than uint16_t max.  This could lead to menu choices giving the user a radically different result than they expect.  You should probably clamp the value to between 0 and uint16_t max.  Also since your variable is signed, underflow will lead to sign extension and really mess your values up.

@names_are_hard So far, with the code as-is I'm seeing the kinds of changes I'd expect to the regs targeted. With the integer overflow bug, are we talking values that the user thinks they're inputing at 10 are actually [say] 100? Is that the sort of discrepancy you outline?

[names_are_hard]

Tim - read around "integer overflow" if you want to understand the problem, it's too off topic to go into here.  The short version is if you do user_value + 0x401 and the user puts in 0xfc00, the result is not 0x10001 as you'd expect, but 0x1.  I guess there's some risk when assigning unexpected values to control registers.

[timbytheriver]

Audionut on March 02, 2014, 03:20:11 PM
Does iso_regs scale [8,9,A,B], keeping the deltas between them?

a1ex
Yes. It considers the first one as reference (that one will get the value from menu) and all the others as delta (so, when you set it to 0 or 1 or maybe even 2 or 3, it may underflow and cause strong banding). I preferred this underflow instead of clamping to 0 because in this way you see when you went too far. However, ADTG gains 888x are not scaled in iso_regs (these are simply set all of them to the same value).

I'm reviewing whether the decimal adjustments in my Q iso-regs menus are using the correct math. Line 1205 > https://bitbucket.org/rivertim/magic-lantern-danneclone/src/ddebb19831d7ce7472bb80db9926bc69b3bc2d6d/modules/crop_rec/crop_rec.c

Is the "deltas between them" described by @a1ex and @Audionut in this context meaning to keep the integer relationship constant? e.g.

Say for: ADTG2 8,9,a,b values = 0x60,0x61,0x62,0x60 meaning x, +1, +1, -2 Do the relationships scale as they change?

Sorry for math duh-ness! 

**EDIT**

a1ex
However, ADTG gains 888x are not scaled in iso_regs (these are simply set all of them to the same value).

I notice in adtg_gui my values for these (888x) registers are different:

Code: [Select]

ADTG2[8882] = 0x436
ADTG2[8884] = 0x438
ADTG2[8886] = 0x439
ADTG2[8888] = 0x43a

ADTG4[8882] = 0x437
ADTG4[8884] = 0x437
ADTG4[8886] = 0x437
ADTG4[8888] = 0x43a


Should they in fact all be set to the same value?

Thanks!

[a1ex]

Short answer: everything from this thread needs to be taken with a grain of salt (as none of us is expert in sensor calibration), so it's probably best to verify each hypothesis from scratch.

The 5D3 reads out 8 columns at a time (that is, 8 analog circuits operating in parallel), and the 888x gains (4x2 = 8 values) are for each group of columns. So, one gain affects columns 1, 9, 17 etc, another gain affects columns 2, 10, 18 etc, and so on. The 8,9,a,b are IIRC similar. These values should be tweaked so all these 8 groups of columns have consistent output; otherwise, you get vertical stripes / banding. Other cameras may read out a different number of columns at a time (usually 4 or 2), so you'll see fewer registers there.

The 888x registers are linear gains (i.e. if you set them to 1000 vs 2000, that's a difference of 1 EV), so it's probably best to scale the factory values (camera-specific) by the same constant. The 8/9/A/B gains are adjusted in fractions of EV (about 0.0059 EV/step?), so it's probably best to keep the deltas constant.

Now, since vertical stripes also appear at default configurations at least on 5D3, it's probably best to re-calibrate all these gains from scratch somehow. Related work.

[timbytheriver]

@a1ex Many thanks for clarification and links.

These values should be tweaked so all these 8 groups of columns have consistent output; otherwise, you get vertical stripes / banding…

I've found this also true of the ADTG2/4 0xFE register. Uneven values between the banks give a similar result.

[timbytheriver]

I'm trying to target Digital Gain regs with an adjustment. I find this in iso_regs:

Code: [Select]

if (reg == 0xc0f37ae4 || reg == 0xc0f37af0 || reg == 0xc0f37afc || reg == 0xc0f37b08)
        {
            default_digital_gain = val;
            if (digital_gain) *copy_ptr = digital_gain;
        }

I've tried:

Code: [Select]

adtg_new[40] = (struct adtg_new) {6, 0xc0f37ae4, 0x200 + gain_dgain};
adtg_new[41] = (struct adtg_new) {6, 0xc0f37af0, 0x200 + gain_dgain};
adtg_new[42] = (struct adtg_new) {6, 0xc0f37afc, 0x200 + gain_dgain};
adtg_new[43] = (struct adtg_new) {6, 0xc0f37b08, 0x200 + gain_dgain};

or

Code: [Select]

adtg_new[44] = (struct adtg_new) {6, 0x8030, 0x200 + gain_dgain};
Do any of these reg addresses look correct?

Cheers!

[Levas]

Was messing with ADTG register override module and found some interesting registers for video mode (Not sure if these were mentioned before  ).
2 registers bump the gain, without dynamic range penalty!
Edit: False alarm, the registers seemed to drop frames per second, from 25 to about 8 fps, so while ML was reporting 25fps based on timer registers, the recordings were in 8fps, so more light hitting the sensor...
combined they give about 1.66 stop of extra light, without sacrificing dynamic range, works on all iso's.
Not sure what the trickery is done in the background, but rolling shutter becomes huge with these 2 registers override.



Now comes the weirdness, the registers only become visible in ADTG register override module, after a photo or video is taken.
To further investigate, I made a crop_rec preset with these two registers overriden, but they don't get activated the first time.
Crop_rec preset is loaded, nothing happens, record one video and after that the settings kick in, why doesn't it work the first time without having to record a video first ?

Also tried to take photos with these settings, but photos are messed up, purple or green color, sometimes even messed up frame, timing issues or something.

I don't know how values look on other cams, but I lowered the most left digit of these values until live view becomes brighter.



Side by side comparison, same ISO, same Shuttertime and same Diaphragm Aperture  . Left with ADTG2 registers 23 and 24 overrides, on the right, basic values (with 1.66 exposure correction in post).

Here are the two dng frames on google drive:
https://drive.google.com/open?id=1mwtcX7FVyrJTth-Xjqpr0-cdIJxJj9XE
https://drive.google.com/open?id=11MEW5mOHFBXrBlrssz7uTj0FJx5-Xzj6

[a1ex]

2 registers bump the gain, without dynamic range penalty!
combined they give about 1.66 stop of extra light, without sacrificing dynamic range, works on all iso's.
Not sure what the trickery is done in the background, but rolling shutter becomes huge with these 2 registers override.

Confirmed on 5D3 - really interesting. It appears to reduce the frame rate as well.

ADTG[23]: default 0xF00, setting to 0x200 or some other magic values will cut the frame rate in half*, doubling the exposure time.
ADTG[24]: default 0x2d1e, setting to 0x1d1e will cut the frame rate in half*, doubling the exposure time.
* changing both registers together will result in 1/3 of the original frame rate (e.g. 24 -> 8 fps), and 3x exposure time.

This could explain the results, as the dynamic range does not change significantly with shutter speed.

In addition, changing any of those 2 registers in photo mode appears to have no effect (possibly because of the mechanical shutter), but changing both of them together results in garbled image (possibly timing issues because of slower readout). With FRSP, changing any or both of those 2 registers results in garbled image (?!)

Definitely worth a closer look, in particular at higher ISOs sorry, false alarm

[Danne]

Tried some on eosm but couldn't get these results.
Reducing framerate, hmm...

[Levas]

changing both registers together will result in 1/3 of the original frame rate (e.g. 24 -> 8 fps), and 3x exposure time.

You're right, only checked individual frames yesterday, didn't process them to a video file.
Now I did test some more and processed to a video file, and it's indeed about 3 times slower fps...
That explains why the exposure is so much better...damn was hoping to have found some magic register which could gave me ultimate low light performance on iso6400 

But it's interesting to know that there are other registers then A-timer and B-timer that influence fps...now if it only could speed up instead of slow down fps...

[names_are_hard]

ADTG[24]: default 0x2d1e, setting to 0x1d1e will cut the frame rate in half*, doubling the exposure time.

The obvious first test for increasing frame rate would be ADTG[24] = 0x4d1e.  But don't blame me if it makes your cam explode   0x3d1e would be my second choice.

[Levas]

Obviously tried, but didn't work, couldn't increas fps speed with these two registers