Author Topic: CMOS/ADTG/Digic register investigation on ISO  (Read 449244 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10439
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #925 on: August 02, 2017, 08:06:16 AM »
Are you sure pattern noise has been introduced?  All of the other noise sources help to hide pattern noise.

Pattern noise removal really needs an average of several dark frames to be effective.

Maybe my pixel peeping skills aren't the best, but all files (input and output) and commands used are on the Jenkins page linked in that thread. The dark frame is averaged from 31 frames.

keepersdungeon

  • Member
  • ***
  • Posts: 135
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #926 on: August 20, 2017, 12:09:25 PM »
Thanks Greg,

I can get 3xzoom/crop mode on 6d by overriding the following registers:
                      1080p     10xzoom     3xzoom(1:1 crop)

ADTG2-805f      2b0        13c             No change needed
ADTG2-8061     2b0        13c             No change needed

CMOS[6]          400         d0             d0       (=Changing the "d" moves horizontal position on the sensor)
CMOS[7]             0         208          206       (Lowering the 8 to 6 moves the white bar outside 16:9 view)
ADTG2-800c         2           0              0         (vertical lines to skip)
ADTG2-8000        6           5             5       (Don't know what it does, if it is 6 you get some vertical interlaced looking footage...)


With these settings I get 3xzoom/crop mode, correct preview in live view on the 6d
Raw recording works, gives correct framing.
I can even record canon "native" h.264 with it, and it works too!

Just saw this! didn't even know it was possible on 6D
Anws was trying it,  and when I hit the crop 1:1 in the raw settings I can see 1826*1092 x3.0 but the recording is identical as the footage without hitting zoom
I missed something again didn't I?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10439
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #927 on: September 25, 2017, 02:57:06 PM »
Just posted new builds of the iso-research modules (adtg_gui, iso-regs and raw_diag), to be loaded on top of the current crop_rec_4k builds (as the older crop_rec is no longer on the download page).

http://builds.magiclantern.fm/modules.html#iso-research

If you decide to experiment with them, I'm looking forward to seeing your results.

kichetof

  • Senior
  • ****
  • Posts: 460
  • Take a beer and enjoy it!
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #928 on: September 26, 2017, 06:41:18 PM »
@a1ex I would try to play with it, but I've an error with adtg module:

Code: [Select]
ML ASSERT:
new_entry->name
at ../../src/menu.c:1241 (menu_add_internal), task module_task
lv:0 mode:3

module_task stack: 1db7b8 [1dba50-1d7a50]
0xUNKNOWN  @ b68c8:1db878
0x00072678 @ a0f318:1db860
0x0006EA48 @ 726a4:1db848
0x0006EA48 @ 6ee30:1db818
0x00069FF0 @ 6ec90:1db7e8
0x00069868 @ 6a05c:1db7b8

Magic Lantern version : crop_rec_4k.2017Sep26.5D3113
Mercurial changeset   : 382dafb8a0bd (crop_rec_4k) tip
Built on 2017-09-25 23:32:14 UTC by [email protected]
Free Memory  : 134K + 3143K

It seems that there are a lot of changes in menu.c between crop_rec_4k and iso-research, I'll try to find how to solve it
https://bitbucket.org/hudson/magic-lantern/branches/compare/crop_rec_4k%0Diso-research#chg-src/menu.c

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10439
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #929 on: September 26, 2017, 07:03:36 PM »
Right - it needs a valid name, such as .name = "(empty)". Did that to avoid a bunch of null pointer issues, but only tried iso_regs and raw_diag from the latest build.

Hopefully fixed, but unable to test right now.

kichetof

  • Senior
  • ****
  • Posts: 460
  • Take a beer and enjoy it!
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #930 on: September 26, 2017, 08:59:00 PM »
Hopefully fixed, but unable to test right now.

You're the king!

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10439
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #931 on: October 06, 2017, 05:31:49 PM »
Here's a quick example on how adtg_gui can be used to find interesting stuff.



In the crop_rec_4k branch I've used raw type 0x12 (DIGIC 5) in order implement 8..12-bit lossless compression. This works by keeping the 14-bit container (as I don't know how to reconfigure the encoder for other bit depths) and darkening the image (applying a digital gain aka SHAD_GAIN) so it ends up with fewer levels (as many as a real 8..12-bit image).

This trick appears to work - it improves the compression ratio.

Problem: LiveView also gets darkened (as a side effect of bit depth reduction).

By default, ML uses raw type CCD (0x10 on D5, 0x5 on D4), as this one gives consistent white levels. This raw type is not affected by ISO changes, therefore it doesn't react to digital gain (so there's no easy way to darken it digitally). I could darken it from analog controls, but didn't want to touch them for bit depth reduction.

Anyway - I'm currently trying to find a way to brighten the image of raw type 0x12 (DEFCORRE, apparently after SHAD in the pipeline, but before other image processing steps, such as the picture style), but *without* brightening the raw buffer contents. That means, I'm looking for some sort of digital gain later in the pipeline (after DEFCORRE).

Incidentally, I've noticed our low-light exposure simulation feature (Expo Override + ExpSim enabled), after a certain threshold (a few seconds of exposure at some high ISO), it brightens the LiveView image, but the raw histogram is no longer pushed to the right. In these (extreme) cases, on DIGIC 5, ML uses something like this:
Code: [Select]
call("lvae_setdispgain", some_gain); /* 1024 = 1.0 */

Problem: on 5D3, this call only works in photo mode...

On DIGIC 4, we've identified this register by brute force:
Code: [Select]
#define ISO_PUSH_REGISTER 0xc0f0e0f8

But that doesn't work on 5D3.

Looking in lvae_setdispgain (FF72C0B0 in 5D3 1.2.3), it stores the gain at 0x68C8C, prints a message and sets some flags. The actual configuration is likely done by some other code that reads this value. Unfortunately, that code was not identified by IDA's autoanalysis.

How to find it? This is likely some register used to configure LiveView, like those handled by adtg_gui. So, I've tried to find it using this tool.

How?

1) entered photo mode LiveView (where our function works), enabled ADTG Registers, DIGIC registers and selected "Modified from now on"
2) back to LiveView, then back to ML menu -> a bunch of registers changed. Selected "Modified from now on" again until things settled.
3) called the analyzed function (had it on "don't click me" with random gains) and noticed how the LiveView brightness changed
4) went back to adtg_gui menu and it printed these modified registers (5D3 1.1.3):
Code: [Select]
C0F42730: 0x1010202 (was 0x1)       - from AeWb:ff33ff94:6b84fc
C0F4273C: 0x46f076c (was 0x5640483) - from AeWb:ff33ff94:6b8514
C0F42740: 0x6900690 (was 0x3fd03fd) - from AeWb:ff33ff94:6b851c
C0F42744: 0x4040404 (was 0x0)       - from AeWb:ff33ff94:6b8524

5) Locked the above registers and confirmed the brightness could no longer be changed with lvae_setdispgain. That's a good sign - it means the register I'm looking for is one of these (or maybe all of them).

6) Changed the above registers to their old value (the "was" one). Result: LiveView back to original brightness (before calling lvae_setdispgain). Another good sign.

7) Now, to find out what they do - the "trial and error" part:

C0F42730 looks like a series of 4 8-bit values. Each of them appears to be a boost of the 4 Bayer channels. Valid values: 0...3 (4 is the same as 0). Making them equal will make the image brighter; otherwise, the colors will be changed.

C0F42744 is similar, but valid values go from 0 to 7. It also gives some Bayer artifacts.

The remaining two are 16-bit values (two values packed in one register). Probably some sort of WB gains.

C0F4273C: the lower half looks like red gain, the higher half looks like blue gain (range 0 - 7FF)
C0F42740: these look like green gains; if the two are different, some interesting artifacts appear - just like with C0F42744.

8.) With the above registers 4 locked, white balance controls (from Canon menu) are no longer working. This makes sense - these are all changed from the AeWb task.

Overriding just C0F4273C/40 does not completely lock WB controls - they still work in some rough increments.

9) Switched to Movie mode and tried to adjust the above registers. It works!

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3044
  • 5D3 / M1 / 7D / 70D / SL1
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #932 on: October 06, 2017, 06:08:29 PM »
Once again excellent progress with this troubleshoot of yours, @a1ex!  8)
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

deix

  • Just arrived
  • *
  • Posts: 1
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #933 on: October 29, 2017, 02:27:34 PM »
Hi there,

will the "raw_diag.mo" be loadable also for the main branch, or is it only available for experimental crop_rec_4k build - I get the following version error while using the main build "magiclantern-Nightly.2017Oct04.60D111.zip"

"Wrong version(v7.0, expected v6.0)
http://ibb.co/fhzJrR

Best regards
Michal Powalko

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10439
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #934 on: October 29, 2017, 10:27:39 PM »
Will be after merging lua_fix; meanwhile, you can use that experimental build to load it.

The other two modules also require the patch manager (not included in lua_fix, so they won't load on that build).