CMOS/ADTG/Digic register investigation on ISO

Started by a1ex, January 10, 2014, 12:11:01 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Danne

All color noise gone? Cannot believe my eyes.

DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

michael84

ML ISOs tend to reduce what i recall i called "amplifier glow" (magenta-like glow in the shadows, visible when pushing high ISOs and/or in long exposures). Especially the preamp seems to be the major contributor here. I tried various settings starting from Canon ISO 200, from ADTG only (reduced until WL drop), to 0xFE gain 0 and preamp standard, to 0xFE 0 and pregain 1. Only lowering the pregain reduces the glow significantly.

Only ADTG: https://www.dropbox.com/s/qb09hezgo8crxml/5D3_ML%20ISO%20149_only%20ADTG%20crop.jpg?dl=0

Only 0xFE: https://www.dropbox.com/s/o2a6ebavivq6o79/5D3_ML%20ISO%20137_only%200xFE%20crop.jpg?dl=0

0xFE=0 and pregain 1: https://www.dropbox.com/s/jgls31kr5vfls4f/5D3_ML%20ISO%20110_0xFE%280%29_pregain%281%29_crop.jpg?dl=0


At high ISOs, ML ISO settings tend to clean up the colors a bit, and move the a bit toward green (presumably because the amp glow is reduced):

ISO 3200, Canon: https://www.dropbox.com/s/1ookn81oekq9pvb/5D3_C_ISO_3200%20Crop%202.jpg?dl=0

ISO 3200, ML (pulled from 6400): https://www.dropbox.com/s/afukgac8y90br1s/5D3_ML%20ISO%203200%28from6400%29%20Crop%202.jpg?dl=0


Now lets push the exposure a little bit:

Canon: https://www.dropbox.com/s/pdymz714lpov89u/5D3_C_ISO_3200%20Crop%202%20%28%2B1%20EV%29.jpg?dl=0

ML: https://www.dropbox.com/s/yiq2qdeb4exkq95/5D3_ML_ISO_3200%20Crop%202%20%28%2B1%20EV%29.jpg?dl=0


BTW, is there any way to convert a DNG to ML RAW video or MLV? So that i could try the vertical stripes fix from MLV Mystic to reduce the banding noise?

l_d_allan

Not meaning to be impatient, but did the research on this get to the point of making it into a "Nightly Build"?

The info on the introductory OP from Jan, 2014 seems to indicate it is still experimental if someone is willing and able to do a compile from source. Or not?

markanini

Do devs need help with testing on 600D at this time?
Gear: Canon 600D & Magic Lantern Nightly.

a1ex

Yes, any test you find on this thread, that was not done on 600D.

Same for any other camera ;)

markanini

Quote from: a1ex on September 12, 2016, 09:47:24 AM
Yes, any test you find on this thread, that was not done on 600D.

Same for any other camera ;)

I'll start reading through the thread when I'm less busy. Though I don't mind  if I can be pointed to specific test procedures in the mean time.
Gear: Canon 600D & Magic Lantern Nightly.

DerMax

Lets say I'd like to play with iso_regs module..
where can I find it? :)

Walter Schulz

You may want to read first post of this very thead.

a1ex

Updated first post - on 5D3, you may now use the crop_rec build to run these tools (they no longer require GDB).

FYI, the latest adtg_gui contains an experimental 3K crop mode for 5D3 as well.


jasonrwoods

Been lurking for a while, and have been running ML on my 6D for over a year and love it.  I finally am able to compile on my Mac after a week of trial and much error (I know nothing about coding).  I'm trying to run the ADTG module and edit the registers.  (yes, I've enabled CONFIG_GDB)

My problem is that when I try to run the adtg_gui module, I get these errors:

Quotetcc: error: undefined symbol 'strstr'
tcc: error: undefined symbol 'patch_hook_function'
[E]failed to link modules

Can anyone help a newbie out?


a1ex

Try compiling ML core as well from the iso-research branch, not just the module.

jasonrwoods

Quote from: a1ex on January 25, 2017, 07:11:45 PM
Try compiling ML core as well from the iso-research branch, not just the module.

Ah, thank you!!!

festr

adtg_gui.mo - is there new version which is compatible with current experimental branches crop_rec? the currnet link on 1st page does not work with it

bpv5P

No progress? Need any testers for 600D? I could compile and help with some data. I would really like to see this on nightly builds...

a1ex

Welcome - feel free to experiment and share your findings.

FYI - the raw_diag module is on the downloads page for quite some time.

bpv5P

Quote from: a1ex on July 14, 2017, 08:06:43 AM
Welcome - feel free to experiment and share your findings.

FYI - the raw_diag module is on the downloads page for quite some time.


Cool, thanks a1ex. I'll do some test and send some data...

nikfreak

Probably got a freaky request but care to share a working adtg_gui.mo for 7D or is it impossible?
[size=8pt]70D.112 & 100D.101[/size]

Kharak

@a1ex

Do you think we will see ISO 66 in any of the Nightlies or Experimentals? Or will it always be a "thing" from adtg_gui.mo? Also someone posted examples with the negative ML ISO and how there was almost no color noise? I don't remember where I saw those examples.
once you go raw you never go back

a1ex

It's on the back burner, but cannot promise any time frame.

As usual, the priority will get higher once there will be interest in experimenting with this stuff, rather than just asking for it to be done.

BTW, updated the iso-research branch for latest codebase and solved this error, but it's completely untested (other than checking whether it compiles). Sorry - currently working on something else.

DeafEyeJedi

Quote from: nikfreak on July 30, 2017, 10:47:13 AM
Probably got a freaky request but care to share a working adtg_gui.mo for 7D or is it impossible?

+1
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Audionut

Quote from: nikfreak on July 30, 2017, 10:47:13 AM
Probably got a freaky request but care to share a working adtg_gui.mo for 7D or is it impossible?

http://www.magiclantern.fm/forum/index.php?topic=10111.msg144472#msg144472

Quote from: a1ex on July 31, 2017, 12:20:13 AM
As usual, the priority will get higher once there will be interest in experimenting with this stuff, rather than just asking for it to be done.

Do you have some specific things that need doing for this branch (things I am capable of)?

Quote from: a1ex on July 31, 2017, 12:20:13 AM
BTW, updated the iso-research branch for latest codebase and solved this error, but it's completely untested (other than checking whether it compiles). Sorry - currently working on something else.

My 6D is still in one piece.

a1ex

Quote from: Audionut on August 01, 2017, 12:47:26 AM
Do you have some specific things that need doing for this branch (things I am capable of)?

Restoring the missing screenshots would be a good starting point. It will probably take a while to re-read the thread and refresh my memory on all this stuff.

Some issues with raw_diag mentioned here, didn't look into them yet: http://www.magiclantern.fm/forum/index.php?topic=16516.msg186527#msg186527

Try to reproduce the above findings on different camera models. Some may need different registers.

Simply play with adtg_gui and document register behavior from your camera.

Find out how low you can push the ISO without getting artifacts in highlights. Different cameras (of the same model) might have different results.

Figure out the registers required for the ISO tweaks in LiveView (some are the same, others are not).

Script various test scenarios in Lua (e.g. cycle through all ISOs, take test pictures, change settings, create tables with results).

Evaluate the repeatability of our measurements under various test scenes.

Propose a way to measure the sensor response curve without expensive lab equipment.

Try the pattern noise reduction algorithm from MLVFS or mlv_dump on steroids on the images taken with ISO tweaks (IIRC 5D3 has problems with this, 6D less so). Try to fine-tune the parameters.

Figure out how to reproduce the ETTR + Dual ISO crash on the patchmgr branch (doesn't happen on unified; happens with low probability on patchmgr and any other branches that use this backend).

Figure out why this dark frame introduced additional pattern noise, instead of removing it.

Just some random ideas. Don't limit yourself to these :D

Audionut

Quote from: a1ex on August 01, 2017, 01:45:59 AM
Figure out why this dark frame introduced additional pattern noise, instead of removing it.

Are you sure pattern noise has been introduced?  All of the other noise sources help to hide pattern noise.

Pattern noise removal really needs an average of several dark frames to be effective.