Author Topic: CMOS/ADTG/Digic register investigation on ISO  (Read 451037 times)

Danne

  • Hero Member
  • *****
  • Posts: 3480
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #900 on: June 29, 2016, 10:45:23 PM »
All color noise gone? Cannot believe my eyes.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3045
  • 5D3 / M1 / 7D / 70D / SL1
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #901 on: June 30, 2016, 07:27:09 AM »
Here you go @michael84...

Canon ISO


ML ISO
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

michael84

  • New to the forum
  • *
  • Posts: 8
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #902 on: July 04, 2016, 01:19:02 PM »
ML ISOs tend to reduce what i recall i called "amplifier glow" (magenta-like glow in the shadows, visible when pushing high ISOs and/or in long exposures). Especially the preamp seems to be the major contributor here. I tried various settings starting from Canon ISO 200, from ADTG only (reduced until WL drop), to 0xFE gain 0 and preamp standard, to 0xFE 0 and pregain 1. Only lowering the pregain reduces the glow significantly.

Only ADTG: https://www.dropbox.com/s/qb09hezgo8crxml/5D3_ML%20ISO%20149_only%20ADTG%20crop.jpg?dl=0

Only 0xFE: https://www.dropbox.com/s/o2a6ebavivq6o79/5D3_ML%20ISO%20137_only%200xFE%20crop.jpg?dl=0

0xFE=0 and pregain 1: https://www.dropbox.com/s/jgls31kr5vfls4f/5D3_ML%20ISO%20110_0xFE%280%29_pregain%281%29_crop.jpg?dl=0


At high ISOs, ML ISO settings tend to clean up the colors a bit, and move the a bit toward green (presumably because the amp glow is reduced):

ISO 3200, Canon: https://www.dropbox.com/s/1ookn81oekq9pvb/5D3_C_ISO_3200%20Crop%202.jpg?dl=0

ISO 3200, ML (pulled from 6400): https://www.dropbox.com/s/afukgac8y90br1s/5D3_ML%20ISO%203200%28from6400%29%20Crop%202.jpg?dl=0


Now lets push the exposure a little bit:

Canon: https://www.dropbox.com/s/pdymz714lpov89u/5D3_C_ISO_3200%20Crop%202%20%28%2B1%20EV%29.jpg?dl=0

ML: https://www.dropbox.com/s/yiq2qdeb4exkq95/5D3_ML_ISO_3200%20Crop%202%20%28%2B1%20EV%29.jpg?dl=0


BTW, is there any way to convert a DNG to ML RAW video or MLV? So that i could try the vertical stripes fix from MLV Mystic to reduce the banding noise?

l_d_allan

  • Member
  • ***
  • Posts: 247
Not meaning to be impatient, but did the research on this get to the point of making it into a "Nightly Build"?

The info on the introductory OP from Jan, 2014 seems to indicate it is still experimental if someone is willing and able to do a compile from source. Or not?

markanini

  • New to the forum
  • *
  • Posts: 35
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #904 on: September 12, 2016, 01:39:44 AM »
Do devs need help with testing on 600D at this time?
Gear: Canon 600D + EF-S 18-55mm IS II, Yongnuo YN560IV.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10457
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #905 on: September 12, 2016, 09:47:24 AM »
Yes, any test you find on this thread, that was not done on 600D.

Same for any other camera ;)

markanini

  • New to the forum
  • *
  • Posts: 35
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #906 on: September 12, 2016, 05:39:41 PM »
Yes, any test you find on this thread, that was not done on 600D.

Same for any other camera ;)

I'll start reading through the thread when I'm less busy. Though I don't mind  if I can be pointed to specific test procedures in the mean time.
Gear: Canon 600D + EF-S 18-55mm IS II, Yongnuo YN560IV.

DerMax

  • New to the forum
  • *
  • Posts: 10
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #907 on: September 28, 2016, 02:17:27 PM »
Lets say I'd like to play with iso_regs module..
where can I find it? :)

Walter Schulz

  • Hero Member
  • *****
  • Posts: 5763
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #908 on: September 28, 2016, 02:44:19 PM »
You may want to read first post of this very thead.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10457
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #909 on: September 28, 2016, 03:02:36 PM »
Updated first post - on 5D3, you may now use the crop_rec build to run these tools (they no longer require GDB).

FYI, the latest adtg_gui contains an experimental 3K crop mode for 5D3 as well.

DerMax

  • New to the forum
  • *
  • Posts: 10
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #910 on: September 28, 2016, 05:24:51 PM »
Thank You! ;)

jasonrwoods

  • Just arrived
  • *
  • Posts: 2
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #911 on: January 25, 2017, 06:32:29 PM »
Been lurking for a while, and have been running ML on my 6D for over a year and love it.  I finally am able to compile on my Mac after a week of trial and much error (I know nothing about coding).  I'm trying to run the ADTG module and edit the registers.  (yes, I've enabled CONFIG_GDB)

My problem is that when I try to run the adtg_gui module, I get these errors:

Quote
tcc: error: undefined symbol 'strstr'
tcc: error: undefined symbol 'patch_hook_function'
[E]failed to link modules

Can anyone help a newbie out?


a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10457
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #912 on: January 25, 2017, 07:11:45 PM »
Try compiling ML core as well from the iso-research branch, not just the module.

jasonrwoods

  • Just arrived
  • *
  • Posts: 2
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #913 on: January 25, 2017, 11:47:02 PM »
Try compiling ML core as well from the iso-research branch, not just the module.

Ah, thank you!!!

festr

  • New to the forum
  • *
  • Posts: 30
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #914 on: May 01, 2017, 06:10:46 PM »
adtg_gui.mo - is there new version which is compatible with current experimental branches crop_rec? the currnet link on 1st page does not work with it

bpv5P

  • Member
  • ***
  • Posts: 167
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #915 on: July 14, 2017, 02:36:14 AM »
No progress? Need any testers for 600D? I could compile and help with some data. I would really like to see this on nightly builds...

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10457
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #916 on: July 14, 2017, 08:06:43 AM »
Welcome - feel free to experiment and share your findings.

FYI - the raw_diag module is on the downloads page for quite some time.

bpv5P

  • Member
  • ***
  • Posts: 167
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #917 on: July 18, 2017, 04:33:04 PM »
Welcome - feel free to experiment and share your findings.

FYI - the raw_diag module is on the downloads page for quite some time.


Cool, thanks a1ex. I'll do some test and send some data...

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 1029
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #918 on: July 30, 2017, 10:47:13 AM »
Probably got a freaky request but care to share a working adtg_gui.mo for 7D or is it impossible?
70D.112 & 100D.101

Kharak

  • Hero Member
  • *****
  • Posts: 708
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #919 on: July 30, 2017, 10:28:17 PM »
@a1ex

Do you think we will see ISO 66 in any of the Nightlies or Experimentals? Or will it always be a "thing" from adtg_gui.mo? Also someone posted examples with the negative ML ISO and how there was almost no color noise? I don't remember where I saw those examples.
once you go raw you never go back

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10457
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #920 on: July 31, 2017, 12:20:13 AM »
It's on the back burner, but cannot promise any time frame.

As usual, the priority will get higher once there will be interest in experimenting with this stuff, rather than just asking for it to be done.

BTW, updated the iso-research branch for latest codebase and solved this error, but it's completely untested (other than checking whether it compiles). Sorry - currently working on something else.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3045
  • 5D3 / M1 / 7D / 70D / SL1
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #921 on: July 31, 2017, 01:41:20 AM »
Probably got a freaky request but care to share a working adtg_gui.mo for 7D or is it impossible?

+1
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3444
  • Blunt and to the point
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #922 on: August 01, 2017, 12:47:26 AM »
Probably got a freaky request but care to share a working adtg_gui.mo for 7D or is it impossible?

http://www.magiclantern.fm/forum/index.php?topic=10111.msg144472#msg144472

As usual, the priority will get higher once there will be interest in experimenting with this stuff, rather than just asking for it to be done.

Do you have some specific things that need doing for this branch (things I am capable of)?

BTW, updated the iso-research branch for latest codebase and solved this error, but it's completely untested (other than checking whether it compiles). Sorry - currently working on something else.

My 6D is still in one piece.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10457
  • 5D Mark Free
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #923 on: August 01, 2017, 01:45:59 AM »
Do you have some specific things that need doing for this branch (things I am capable of)?

Restoring the missing screenshots would be a good starting point. It will probably take a while to re-read the thread and refresh my memory on all this stuff.

Some issues with raw_diag mentioned here, didn't look into them yet: http://www.magiclantern.fm/forum/index.php?topic=16516.msg186527#msg186527

Try to reproduce the above findings on different camera models. Some may need different registers.

Simply play with adtg_gui and document register behavior from your camera.

Find out how low you can push the ISO without getting artifacts in highlights. Different cameras (of the same model) might have different results.

Figure out the registers required for the ISO tweaks in LiveView (some are the same, others are not).

Script various test scenarios in Lua (e.g. cycle through all ISOs, take test pictures, change settings, create tables with results).

Evaluate the repeatability of our measurements under various test scenes.

Propose a way to measure the sensor response curve without expensive lab equipment.

Try the pattern noise reduction algorithm from MLVFS or mlv_dump on steroids on the images taken with ISO tweaks (IIRC 5D3 has problems with this, 6D less so). Try to fine-tune the parameters.

Figure out how to reproduce the ETTR + Dual ISO crash on the patchmgr branch (doesn't happen on unified; happens with low probability on patchmgr and any other branches that use this backend).

Figure out why this dark frame introduced additional pattern noise, instead of removing it.

Just some random ideas. Don't limit yourself to these :D

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3444
  • Blunt and to the point
Re: CMOS/ADTG/Digic register investigation on ISO
« Reply #924 on: August 02, 2017, 07:08:04 AM »
Figure out why this dark frame introduced additional pattern noise, instead of removing it.

Are you sure pattern noise has been introduced?  All of the other noise sources help to hide pattern noise.

Pattern noise removal really needs an average of several dark frames to be effective.