Author Topic: CMOS/ADTG/Digic register investigation on ISO  (Read 611127 times)

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1125 on: October 30, 2019, 02:20:24 PM »
Ok. Went into crop_rec submenu > Advanced > reg_gain = 250 (12bit) Is that the value format it's expecting?

Recorded results with this give correct crop_rec resolution 3072x1536 but is definitely 14bit lossless.

Should I leave atdg_gui OFF now and just be trying in this submenu? Bit lost now!
5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1126 on: October 30, 2019, 02:25:41 PM »
It will be 14bitlossless container but it's really 12bit. Open up a recorded file in mlv app.
You can work in adtg gui and develop some magic in there and then when ready publish your stuff here. But then just work outside crop_rec.

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1127 on: October 30, 2019, 02:33:34 PM »
MLVApp > info says file is '14bit Lossless'

I changed reg_gain to 250 Is this correct?

Once this setting is in reg_gain, do I then enable adtg_gui and start altering regs also in there? Is this correct?

I haven't yet grasped how/whether I'm supposed to run these two modules together.
5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1128 on: October 30, 2019, 02:48:26 PM »
Check white level. It should say 5000 or 6000 in mlv app if 12bit.
If you change reg_gain you can increase and get closer to 14bit or reduce to go towards 10bit. Don't put in 250. Default when 12bit selected is 250. You simply increase or decrease this number.

You can't use adtg and crop rec together. Thought we established this already.

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1129 on: October 30, 2019, 03:04:17 PM »
As an arbitrary start I've set reg_gain to 5

Disabled adtg_gui

Shot a test 200ISO, UHD 3072x1536

File in MLVApp is showing info > 14bit lossless,
White level = 16200

Either I haven't grasped something fundamental (likely) – or something's amiss.

My ultimate goal is to be able to adjust the ISO regs when in extended resolutions (UHD, 3K, Anam.)

Any pointers towards that goal please?  :)
5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1130 on: October 30, 2019, 03:06:27 PM »
And bitdepth in crop mode sub menu? Was it set to 12bit?
If reg_gain is set to 5 it will hardly be noticed. Try moving it in 20 or 30 increments. As I said. In this case you only add or reduce the 12bit analog gain already set to 250.

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1131 on: October 30, 2019, 03:20:37 PM »
Ah.  ;) Missing bit of puzzle: Once 12bit is selected in crop mode sub menu (I thought setting it in reg_gain was enough).

File in MLVApp is now 12bit lossless, 3072x1536, white level 6000

Do I now play about with other registers in the crop mode sub menu to continue iso experiments? (CMOS, et al) Any I should target / avoid touching?

BTW The reg_gain setting seems not to hold after a restart. Is this expected?

Thanks for your help!  :)
5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1132 on: October 30, 2019, 03:30:24 PM »
Not kept after restart.
Good luck!

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1133 on: October 30, 2019, 04:53:30 PM »
@Danne

Is reg_gain adjustment the same as adjusting ADTG2 in atdg_gui.mo ? Just alters bit depth?

Where is the missing CMOS 0 in your crop_rec submenu?

Getting some results now…  :)
5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1134 on: October 30, 2019, 07:48:15 PM »
You really need to read my posts again. Cmos0 is iso...

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1135 on: October 31, 2019, 10:39:28 AM »
You really need to read my posts again. Cmos0 is iso...

Sorry, but you didn't seem definitive about CMOS 0 here:

Not sure about cmos0, it´s iso right? …

I have indeed read almost every post in this long thread – some many times! it's just that the information takes longer for some to digest more than others. :)

@Danne
Where is the missing CMOS 0 in your crop_rec submenu? …

Again: Why are the:

CMOS 0
ADTG 0xFE
ADTG 8/9/A/B

registers not accessible to adjust in your crop_rec submenu > advanced? These registers seem to make important changes to iso behaviour!

Not a criticism – I just genuinely want to understand this puzzle and keep the investigation alive :)

Many thanks.

5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1136 on: October 31, 2019, 11:46:01 AM »
Well, I have added a lot of regs to be able to modify and test directly in crop_rec. But not all. Only those I found useful and had tie to add. That´s why I asked for specific registers to put in there.
Code: [Select]
ADTG 0xFE
ADTG 8/9/A/B
Above are renamed regs. I have no idea what they are. But in adtg_gui you should be able to find them. There´s a category that grouped gain registers. So instead of adding all kinds of registers it would be better for you to just use adtg_gui, build some badass combination in adtg_gui, write your regs down and put them here in a post. Then I can take a look and point to where they can put into crop_rec.c.
Alternatively you can check into crop_rec.c and track down how I have added regs and then put in the regs you want yourself in there. It´s up to you.

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1137 on: October 31, 2019, 12:21:07 PM »
Very useful info. Lots of homework.  :D

Many thanks Danne.

5D3 1.1.3
5D2 2.1.2

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1138 on: November 01, 2019, 11:15:29 AM »
Did some initial experiments yesterday concentrating on modifying CMOS (0) in adtg_gui (I know! You've all been here before…) Anyway,…

5D3_1.1.3
ISO 200
1080p
14Bit lossless
Danne's crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Oct15.5D3113 build

CMOS (0) Default value is 0x113 (?) (275?)

Making small increases until 0x120 produces tiny upward brighter steps until 0x121 where any further brightness is coupled with 'scanlines' across the image (similar to dual-iso lines) and is unusable. Shot quite a few tests which don't show any improvements over default.

Tried Danne's earlier test settings at ISO 200 of:

CMOS(0) 0x333

ADTG2 (8882] 0x13b 
ADTG2 (8884] 0x13b
ADTG2 (8886] 0x13b
ADTG2 (8888] 0x13b

ADTG4 (8882) 0x13b
ADTG4 (8884) 0x13b
ADTG4 (8886) 0x13b
ADTG4 (8888) 0x13b


Images now show (by eye!) the improvement in less noisy shadows, even when compared with the untweaked ISO 400 – and the highlights don't appear to take a heavy hit.

PNG exports from MLVApp:

https://drive.google.com/open?id=1xfJXOPUoql6LwvL7ro7dZLySD8Eju8P_
https://drive.google.com/open?id=14k-ZN-WxelVEcSGZ9-NOvm4jXYhVZjIo
https://drive.google.com/open?id=16mKD39IGkJMPTKgM-w_btCM_41ryK58q


@Danne If we could access just these registers (CMOS(0), ADTG2 / 4) for starters via crop_rec submenu it would be great to test on crop resolutions. I doubt very much that I can find any 'badass combination in adtg_gui' that's better than you already have – but I'll have a go!


Code: [Select]
ADTG 0xFE
ADTG 8/9/A/B
Above are renamed regs. I have no idea what they are. But in adtg_gui you should be able to find them.

@a1ex Do you know how these iso_regs values: ADTG Preamp, ADTG 0XFE, correspond to the same ones in atdg_gui so that they can be accessed by Danne's crop_rec version?

5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1139 on: November 01, 2019, 03:01:40 PM »
If I understand this correctly you are standing on iso 100 and increase cmos 0 until it almost reaches iso 200? Then you pull back gain so it looks like you are back into iso 100? Or maybe you are going from iso 200 to iso 400 and then pull analog gan back into iso 200?
Anyway. A1ex can correct me if I´m wrong here. Both cmos 0 and analog gain registers are not going to achieve anything but very small refinements in dynamic range and when modified. By lowering analog ain regs you are loosing dr and by pushing cmos 0 you are only jumping to the next iso. So that leaves to look for what regs to modify to get any possible refinements? What if you exclude both cmos 0 and analog gain regs altogether and focus on the other registers? For instance, start on iso 200 and try to capture more dr with the other regs? What regs idk...

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1140 on: November 01, 2019, 03:08:31 PM »
I was always starting on Canon base ISO 200 and tweaking the cmos 0 from there. Reproducing your own experiments was a starting point for me so that I know I'm in the ballpark! Hence the cmos , adtg2 / 4 tweaks.

I will have another look and see what other regs might improve DR. Maybe @A1lex has some pointers?

Thanks!
5D3 1.1.3
5D2 2.1.2

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1141 on: November 06, 2019, 02:10:07 PM »
I have enabled raw_diag.mo and tried to get a profile using > Dummy bracket

The grid appears, but overlaid with a message 'You may need to solder some RAM chips'. No SNR curve is produced.

Sounds bad-news :( Can this be fixed?



**EDIT**
Thanks! @a1ex informed me that this is an 'out of RAM' message. I temporarily disabled mlv_lite and mlv_snd modules while using the raw_diag tools and it now works as intended!  :D
**/EDIT**
5D3 1.1.3
5D2 2.1.2

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1142 on: November 28, 2019, 04:42:23 PM »
@Danne Just doing some reading about lua scripting - I hadn't really looked at this before. Is it possible to call up [iso] register changes using lua? Or even the 'Don't click me' button… ? Even I could maybe manage a lua script …  ;)

5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1143 on: November 28, 2019, 04:47:45 PM »
Sorry, havn´t got around to help you with your regs in crop_rec yet.
Well, you can enable anything in ml menus with lua I think but there´s still issues running crop_rec and adtg_gui at the same time.
I think you´re better off now looking into the crop_rec.c code and try to understand where to put in your newly found reg settings.

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1144 on: November 28, 2019, 05:06:52 PM »
@Danne Ah, ok. Yes I've looked into your 'clean-iso' crop_rec.c and stared very hard at it  ??? – I'm looking at these lines:

Code: [Select]
adtg_new[13] = (struct adtg_new) {6, 0x8882, 250};
adtg_new[14] = (struct adtg_new) {6, 0x8884, 250};
adtg_new[15] = (struct adtg_new) {6, 0x8886, 250};
adtg_new[16] = (struct adtg_new) {6, 0x8888, 250};

adtg_new[17] = (struct adtg_new) {6, 0x8882, 250};
adtg_new[18] = (struct adtg_new) {6, 0x8884, 250};
adtg_new[19] = (struct adtg_new) {6, 0x8886, 250};
adtg_new[20] = (struct adtg_new) {6, 0x8888, 250};


and wondering how to translate ADTG2/4 8, 9, a, b, and ADTG4 (fe) into these lines. I especially don't understand the {6, 0x8882, 250}; lines. If I could crack these lines maybe it would mean I could relieve you of your help! :)



5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1145 on: November 28, 2019, 06:36:38 PM »
0x8882 is one of the analog iso regs. 250 its value. So I guess 0x2 and add the value you want after.

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1146 on: November 28, 2019, 07:06:41 PM »
Maybe I've misunderstood  ::)

I thought the reg identifier is say:

Code: [Select]
ADTG2 [8]
or
Code: [Select]
ADTG4 [fe]

and the value is say, 0x512 or whatever…

then what is the…

Code: [Select]
adtg_new [13]

and the 6 value?

bit?

Confused now… ???



5D3 1.1.3
5D2 2.1.2

timbytheriver

  • Senior
  • ****
  • Posts: 440
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1147 on: November 28, 2019, 07:25:24 PM »
So if I want to change:

ADTG2 reg 8 to value 0x2 (decimal 2)

I do:

Code: [Select]
adtg_new[?] = (struct adtg_new) {?, 8, 2};
adtg_new[?] = (struct adtg_new) {?, 9, 2};
adtg_new[?] = (struct adtg_new) {?, a, 2};
adtg_new[?] = (struct adtg_new) {?, b, 2};
?
5D3 1.1.3
5D2 2.1.2

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6624
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1148 on: November 28, 2019, 08:37:04 PM »
Sorry, hopefully someone else can help you here. You should be able to test on your own. Check for analog gain regs in adtg gui and you'll see where to put in stuff. No guarantees your reg works though.
Sorry but need my focus elsewhere now. Too much work...

names_are_hard

  • Contributor
  • Member
  • *****
  • Posts: 206
  • 200D idiot
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #1149 on: November 29, 2019, 04:14:27 AM »
Tim - could you explain a bit more what it is you don't understand?  It's reading like maybe you don't recognise what the struct cast is doing?  Either that, or from my skim reading of Danne's code, the parts I list below?  I'll give my attempt to explain, my apologies if it's too low-level, or too high-level!

In answer to your specific question, which I will change to remove ambiguity of 2 vs 2:
Quote
[how to set] ADTG2 reg 8 to value 0x3 (decimal 3)

I believe you would create a new element in the adtg_new array, then set it like this:
adtg_new[22] = (struct adtg_new) {2, 8, 3};
However, literal 8 looks very suspicious to me given the other regs being in 0x8800 range.

IF YOU USE EXACTLY THAT CODE IT IS WRONG, MAY CRASH AND MAY BRICK YOUR CAM.  Now you have to read my long explanation below to understand why (need to extend adtg_new array).

Please bear in mind I have never looked at this code before and I haven't read all this thread.  I have good experience with C but none with this code, and I understand hardware register manipulation can be very risky.  You are taking the risk, not me, and I recommend you not trust my explanation.

I am working from a specific version of crop_rec.c, and it might be a stupid version!  It was simply the first version that a search found for me:
https://bitbucket.org/Dannephoto/magic-lantern/raw/49f23e49f96129f0777d10e65fdf82942ef48522/modules/crop_rec/crop_rec.c

All that said, it looks to me that dst is how you select which ADTGX is used.  dst is passed into the function and the lower 4 bits are masked in (int dst = cs & 0xF;).

It's important to note that I think adtg_new is an array of changes that we wish to make to registers (if I've got that part wrong my following reasoning will be quite wrong).  Here's the array declaration:
Quote
    /* expand this as required */
    struct adtg_new adtg_new[22] = {{0}};

Here's how a single element of the array is declared:
Quote
    struct adtg_new
    {
        int dst;
        int reg;
        int val;
    };

So, there are max 22 registers the code expects to change.  Here's where an example element of the array is initialised:
Quote
adtg_new[13] = (struct adtg_new) {6, 0x8882, 250};
This sets the 14th element (or 13th depending on how you like to think about 0 indexing...) to have dst == 6, reg == 0x8882, val == 250.  I will later call this "e", and "e.dst", "e.reg" etc.

Here's the code that I think is responsible for making changes to registers:
Quote
            if ((reg == adtg_new[i].reg) && (dst & adtg_new[i].dst))
            {
                int new_value = adtg_new[i].val;
                dbg_printf("ADTG%x[%x] = %x\n", dst, reg, new_value);
                *(uint16_t*)copy_ptr = new_value;
This only changes a reg if some conditions are met.  It's inside a loop that covers all 22 elements of the array.  We read from data_buf, which I haven't tried to understand but I assume holds some list of potential registers to change.  We check every reg struct in adtg_new against every element of data_buf.  If the buffer holds a reg value that matches our e.reg, we then check if passed in dst has at least the bits set that e.dst does.  Now, look at the dbg_printf!  If the passed in dst is 6, with our e, it would print:
ADTG6[8882] = 250

So we know that the passed in dst says which ADTGX to use, and the first field of the struct selects if your change should be applied.  If you want your change to apply to all, use 0xf, due to the logical "&" used earlier (you probably shouldn't do this!  I don't know what the ADTG registers are but applying to all is likely dangerous and wrong).  e.dst == 0x6 will cause the change to be applied if the passed in dst is 4, 2, or 6; i.e., if the passed in dst is ADTG2, ADTG4 or ADTG6.