I just read the news about the increased DR.
Thats incredible and everyone working on that has my deepest appreciation!
But as a newby i dont really know how to implement it in current status of tragic lantern fpr my 6D.
Could anyone give me a hint?
Greetings
find (http://en.wikipedia.org/wiki/Reverse_engineering) the ADTG registers (http://www.magiclantern.fm/forum/index.php?topic=6751.0) that control black level and gain (http://www.magiclantern.fm/forum/index.php?topic=9867.msg95847#msg95847), then optimize their values (http://www.magiclantern.fm/forum/index.php?topic=9867.msg96106#msg96106)
Its not finished for 5DIII yet even. Wait till its out and testable there at least.
Quote from: 1% on January 17, 2014, 09:51:44 PM
Its not finished for 5DIII yet even. Wait till its out and testable there at least.
I wrote alex the correct addresses, and mini_iso.mo works ... the problem is that the 6d raw backend seems to be broken (tried my own compiles and the TL Jan nightly, even reset the settings to default): calibration always stops at the first -1ev step while it *should* continue iterating until it has found the sweet spot. The raw_diag module also fails with "raw err", I dunno.
1%, can you try it on your 6d and create a mini_iso.cfg file? I'm currently out of ideas. Here's the link to the dev module binaries:
edit by Audionut: Binaries removed.
Yea, that hasn't been released on my end so I haven't been able to try it or see what/why is broken.
Tried the rawdiag module and only get the white level and blank graph but no error.
The dark frame works but I can't take a screenshot in the QR screen.
Quote from: 1% on January 18, 2014, 12:13:21 AM
Yea, that hasn't been released on my end so I haven't been able to try it or see what/why is broken.
Did you try if the mini_iso.mo results in anything else than -1ev calibration on your 6d builds? Remembering what you wrote before, the raw stuff miraculously seems to work better on your compiles than mine, I have no idea why. I've pm'ed you the link to the module again in case you've missed the window of opportunity.
Quote from: 1% on January 18, 2014, 12:13:21 AM
Yea, that hasn't been released on my end so I haven't been able to try it or see what/why is broken.
Tried the rawdiag module and only get the white level and blank graph but no error.
The dark frame works but I can't take a screenshot in the QR screen.
It looks like on the 500D (I click "screenshot - 10s" and took a picture)?
Optical black:
(http://s8.postimg.org/6jyxx17yd/VRAM3.png)
Dark frame :
(http://s30.postimg.org/5uwt9850h/VRAM0.png)
SNR curve:
(http://s27.postimg.org/47kl3qxj7/VRAM1.png)
That's how mine looks... when I hit erase I get "Delete?". The calibration ran, says gain -2.48ev at iso 800. It changes when I change ISO.
You can calibrate each ISO step. Make sure the scene is VERY overexposed before running calibration.
The default values are what a1ex gets on his 5D3. Mine are,
-40, -44, -46, -45, -44, -49, -55, -58
Ok, ran it on random stuff so its probably not calibrated right. The first actual ISO was 42. I'll do it properly and see what happens.
Quote from: 1% on January 18, 2014, 01:00:38 AM
Ok, ran it on random stuff so its probably not calibrated right. The first actual ISO was 42. I'll do it properly and see what happens.
That's great, at least you're not getting iso 50 (-1ev) all the time like I did with my builds, just be sure to blow *all* channels sky high. Your task now :-) is to calibrate each full stop iso, and then send the module cfg file to alex (or post it here as code), it'll then get hardcoded into the module as default.
Quote from: 1% on January 18, 2014, 01:00:38 AM
The first actual ISO was 42.
a1ex had -37. Not sure why it would be different.
Quote from: Marsu42 on January 18, 2014, 01:03:27 AM
just be sure to blow *all* channels sky high.
Yes, this is very important else the results will be borked.
Yep, its evening here so I was able to do ISO 800 and 3200 but not 100.
It doesn't matter if its blurry, or the shutter speed. I've calibrated mine fine with a 4 second shutter ;-)
delta.gain = -100
calibrated.gain.100 = -100
calibrated.gain.800 = -124
calibrated.gain.3200 = -52
This is what I got from the 4" exposure at ISO 100. I need to do a proper one but its not crashing at the least.
I would try it manually and see if you can make the white level drop. I deleted my local copy of the code and a1ex has removed it from his dropbox, so I can't look to see if it's something simple. Doubtful if I would be able to though, marsu42 is better at coding then I.
A "proper" calibration is one where all the channels are saturated. Doesn't matter what's in the scene, as long as it's blown! I point mine at the TV handheld.
Ok, I'll try a proper one but right now busy ATM.
I love reading your stuff, although i dont understand a word:)
Always love to see all your afforts! Thanks a lot guys!
This is what it looks like. 1" exposure at a light. The higher ISOs were more completely blown out:
delta.gain = -121
calibrated.gain.100 = -36
calibrated.gain.200 = -33
calibrated.gain.400 = -39
calibrated.gain.800 = -41
calibrated.gain.1600 = -43
calibrated.gain.3200 = -52
calibrated.gain.6400 = -3
calibrated.gain.12800 = -121
Still looks a bit funky. I wouldn't expect ISO 200 to have a lower negative gain value (although I don't have a large sample base to compare from), and ISOs 6400-12800 look completely borked.
Quote from: Greg on January 18, 2014, 12:52:20 AM
Optical black:
(http://s8.postimg.org/6jyxx17yd/VRAM3.png)
If you guys are getting borked results from the optical black, I'd say that's where your problem would be.
The source for raw_diag is here (http://www.magiclantern.fm/forum/index.php?topic=9867.msg96513#msg96513).
Quote from: Audionut on January 18, 2014, 03:42:36 AM
Still looks a bit funky. I wouldn't expect ISO 200 to have a lower negative gain value (although I don't have a large sample base to compare from), and ISOs 6400-12800 look completely borked.
I agree, but at the same time the 6d sensor is the newest generation so Canon might have done some tweaks giving unexpected results with the current test module. Unfortunately, given the fact that the raw part of TL is rather dodgy, it's very hard to say what the problem is and alex cannot help here :-\ ... 1%: are these results at least reproduceable?
One has to diagnose raw_update_params() from raw.c and see why it's returning 0. That's the cause of "Raw error" from raw_diag.
You can define RAW_DEBUG in raw.c, which will print a bunch of debug messages. These may give some clues.
This problem affects 6D and 600D so far.
Quote from: Audionut on January 18, 2014, 03:57:32 AM
If you guys are getting borked results from the optical black, I'd say that's where your problem would be.
The source for raw_diag is here (http://www.magiclantern.fm/forum/index.php?topic=9867.msg96513#msg96513).
Got the raw_diag.mo and loaded it on my 6d.
But how do you use it, found the option RAW Diagnostics in the debug menu, and put it "on". also auto screenshot is "on".
Now I take a picture and nothing happens(besides taking the picture) no extra info on the screen and no trace of a screenshot on the memory card ?
Quote from: Levas on January 18, 2014, 05:11:32 PM
But how do you use it
the answer to your question is the post right above yours:
Quote from: a1ex on January 18, 2014, 10:09:51 AM
One has to diagnose raw_update_params() from raw.c and see why it's returning 0. That's the cause of "Raw error" from raw_diag.
You can define RAW_DEBUG in raw.c, which will print a bunch of debug messages. These may give some clues.
This problem affects 6D and 600D so far.
let me translate that for you: it doesn't work on the 6D, somebody needs to fix the backend *cough* 1%
I usually don't get raw_error, just missing black level data. But the histogram shows ettr and white level is detected. I'll turn on the debug and see since zebras still work, etc.
Quote from: dmilligan on January 18, 2014, 05:33:50 PM
the answer to your question is the post right above yours:
let me translate that for you: it doesn't work on the 6D, somebody needs to fix the backend *cough* 1%
Thanks for the translation, those where 3 long sentences in that post to just say it doesn't work on a 6d and 600d ;)
Quote
delta.gain = -97
calibrated.gain.100 = -24
calibrated.gain.200 = -33
calibrated.gain.400 = -39
calibrated.gain.800 = -41
calibrated.gain.1600 = -43
calibrated.gain.3200 = -51
calibrated.gain.6400 = -52
calibrated.gain.12800 = -123
If I keep the reference white 15268 I get these values. ISO 128K is fake according to dual ISO adtg so maybe that is what is throwing it off. Problem is that its not consistent. White level in raw digger says 15070. In my tests I'm missing over saturation on the green channel so maybe that is what some of the problem is too.
If you remember in 6D (raw.c) we did this
Quote
#ifdef CONFIG_6D
#undef WHITE_LEVEL
#define WHITE_LEVEL 13000
#endif
So either its wrong and white level is actually 15k or something is up elsewhere. The "raw error" happens when the state object misses and then you get no overlays or raw info, once in a whiiiiiile. I'll have to test some more to see why the 2 graphs don't work in raw_debug.
Also checked with raw_debug
W/L 15330
W/L 8998, with mini ISO
White level is autodetected, it doesn't care about the definitons in raw.c. Bad pixels (if they are really severe) may be tricky; I have some sort of filter.
If the image is not really overexposed, it may get tricked and estimate a value higher than normal. So, I would check again ISO 100 (maybe with a higher exposure time). Also check with a manual lens, because Canon plays some tricks at large apertures like f1.4.
I got raw error on 5D3 when pressing half-shutter very quickly after taking a picture (that is, when it tried to run when the image was no longer there).
When you check the white level, make sure you set the gain at the calibrated value. That one should be the sweet spot.
Ok, so that define doesn't matter.
Also I think I'm off by 10px I think on the overlays. I see a tint tiny line of noise and that will throw things off?
height = 3708-12;
skip_top = 38; // DCRAW
= 3658
Raw digger says height = 3648.
I only used a F4-5.6 so maybe I should throw it on a dumb extension tube and point outside instead of a light.
Tried to get the shot like this:
http://imgur.com/Poax9nz
As long as the little rectangle covers only the bright part of the light bulb, it's fine.
For skip offset, the best way is to save the DNG (define RAW_DEBUG_DUMP in raw.c), develop it with dcraw -E, and count the border pixels from this image. Sometimes the buffer address does not match what gets saved in the CR2 (it may be off by a few lines/columns).
To check skip_left, take a overexposed picture (with the overexposed part touching the left side) and a dark frame. Compare the stdev reported by raw_diag (should be nearly identical, say +/- 0.1). If the skip value is too big, stdev will be higher in the bright image. If it's too small, raw zebra will have a dark line near the edge.
Left appears good, no lines, everything is aligned. For top tried 48 and then 58, it got rid of the line and now all graphs work!!!
So now I should try a better test of the correction. Looking through the shots, some of the ISO 100 did have pink highlights.
And for 500D/600D/etc not getting the graphs means the offsets could be wrong by a tiny bit.
Black mean iso 800, 2048, stdev 7.46 DR 10.792... now how to get these to auto screenshot.
I took a white frame and then a black frame. STdev was 7.33 and 7.34 so left should be right.
Quote
delta.gain = -33
calibrated.gain.100 = -37
calibrated.gain.200 = -34
calibrated.gain.400 = -39
calibrated.gain.800 = -41
calibrated.gain.1600 = -43
calibrated.gain.3200 = -52
calibrated.gain.6400 = -56
calibrated.gain.12800 = -156
New chart on an even more over exposed shot. This time it was pretty consistent, the ISO 200 is odd, maybe another run? Would iso 256k produce -256?
Download the stuff again, I've added the screenshot trick today.
500D is OK now, was a typo in the offsets in raw_diag. 600D just keeps saying raw error.
600D is broken because:
https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/commits/fe768c4e30178ab7625346d8530e6b135c92a11d
Heh at least 1st 2 values (100, 200) are consistent now, shooting again.
Transplanted, can you try the calibration for it?
I tried EOSM and the diag module works (with graphs) but mini doesn't load. For 600D I have to download the repo and build it then merge up. Its not on this laptop :(
I'll get on it.
Yeah, mini doesn't load because i don't have the addresses yet.
The big trouble with EOS M: is there any way to disable the LiveView completely?
If yes, it's easy. Fire up ADTG GUI, with LV disabled (if such thing exists), and find the default value for register 8882 (it's in the known register list) at ISO 100.
The thing is that ADTG GUI will also intercept the value in LV (which is not the one i'm looking for).
Then, look it up in a RAM dump (if ADTG value is say 0x415, lookup 0x88820415 (little endian) and write up the address. This is an array with register values at each ISO (100, 125, 160, 200 and so on). Write down the address and default value for 8882nnnn at ISO 100, 125, 160 and 12800. That's it.
The spacing between two ISOs is likely 24 bytes (so far, the only exception was 5D3 with 48 bytes).
For reference, on 6D it's at 0x417443F4 and on 600D it's at 0x403D384c.
Nope, no way to turn off LV completely but the photo regs only seem to crop up in the log when you take a pic. May not be true for the gain but was for dual ISO.
Well good news is 600D is working.
Quote from: dmilligan on January 18, 2014, 05:33:50 PM
let me translate that for you: it doesn't work on the 6D, somebody needs to fix the backend *cough* 1%
The trick itself works, the calibration doesn't which of curse makes it worthless unless you're very good at guessing...
Quote from: 1% on January 18, 2014, 11:41:04 PM
Well good news is 600D is working.
So the 600d fix doesn't help for 6d :-o ? And is debugging raw.c as alex requested still current - as far as I can read it 1% is no at it and we other 6d users better wait a little as duplicating debugging is a waste of time?
Quote from: Marsu42 on January 19, 2014, 03:17:38 PM
and we other 6d users better wait a little as duplicating debugging is a waste of time?
You could debug and describe your findings. You might miss something in the process that another might see, and
vice-versa.
Quote from: Audionut on January 19, 2014, 03:28:54 PM
You could debug and describe your findings. You might miss something in the process that another might see, and vice-versa.
Sure, but given the very limited time I currently have to work on and for ML I'd rather do something no one else currently is doing, if any I'd rather do some more tweaks to my auto_iso module - that's why I'd like to know if 1% (or someone else) has taken it upon him/her to get mini_iso to the 6d.
I put up another 6D build yesterday when I got rid of the extra pixels at the top. So the calibration works now as far as I can tell. 600D was missing a state object so it should be working in main.
Quote from: 1% on January 19, 2014, 07:17:04 PM
I put up another 6D build yesterday when I got rid of the extra pixels at the top. So the calibration works now as far as I can tell. 600D was missing a state object so it should be working in main.
Do you mean that we can use the raw_diag module with 6d now ?
Cause I tried, but I probably don't know how to use it.
Find the option in the debug menu to put RAW Diagnostics on, analysis: Optical black and auto screenshot on.
I'll take a photo and then what ?
Do you have picture quality set to RAW and image review enabled in Canon menu?
Then what? You get a graph.
Quote from: a1ex on January 19, 2014, 09:21:16 PM
Do you have picture quality set to RAW and image review enabled in Canon menu?
Hmm. I've got image review disabled (never use that function...), turn it on...
Aha, there's the graph and numbers... I never would have guessed it would rely on image review, thanks.
Quote from: Levas on January 19, 2014, 10:13:39 PM
I never would have guessed it would rely on image review, thanks.
A lot of the recent ML functions seem to rely on image review as this is the situation when ML is able to analyze the picture and tell that a picture has been taken at all ... so when in doubt try with it on.
enabled = 1
calibrated.gain.100 = -37
calibrated.gain.200 = -34
calibrated.gain.400 = -39
calibrated.gain.800 = -41
calibrated.gain.1600 = -43
calibrated.gain.3200 = -52
calibrated.gain.6400 = -56
calibrated.gain.12800 = -156
This is how it did through a dumb extender tube. It does seem to be lowering the stdev and increasing DR if you take 1 pic with mini ISO and one without.
This one is good, all checksums passed.
Quote from: a1ex on January 20, 2014, 08:41:50 PM
This one is good, all checksums passed.
Ok, so the 6d gains a little less than 5d3 which seems reasonable since the 6d has more "default" dr ... but -156@12800?!
This kinda proves further the iso 128k is fake. Its Iso 6400 - another EV of gain.
Quote from: 1% on January 20, 2014, 09:03:37 PM
This kinda proves further the iso 128k is fake. Its Iso 6400 - another EV of gain.
Right, after thinking about it again that would be it - Canon really simply clips 1ev by digital gain. Strange it seems to be different on 5d3, isn't it - Canon graced the big model with a "real" 12800 while the 6d is maxed out @6400?
Yeah, it's pushed via ADTG gain. Override it in ADTG GUI to around 5x the original value and compare the noise stdev values from raw_diag; I got no real improvement in noise, so this gain seems to be very similar to a digital multiplication (without the roundoff errors).
Quote from: a1ex on January 20, 2014, 09:18:27 PM
I got no real improvement in noise, so this gain seems to be very similar to a digital multiplication (without the roundoff errors).
That's on 5d3 or 6d? Meaning both cameras don't have a real 12800 mode, or only the 6d is lacking it?
5D3 and 60D. But I guess ADTG gain behaves in the same way everywhere.
Quote from: a1ex on January 20, 2014, 09:36:15 PM
5D3 and 60D. But I guess ADTG gain behaves in the same way everywhere.
Weeeelll, but why is the 5d3 12800 calibration smaller than 6400-1ev like it is on 6d? Sorry to be inquisitive, but this still troubles me.
Because the rest is done with CMOS gain (an earlier amplifier stage).
ISO 6400 on 6D is cleaner than 5D3's 12800 (and with a lot more DR too), so there's no reason to get troubled.
http://www.clarkvision.com/articles/evaluation-canon-6d/index.html vs http://www.clarkvision.com/articles/evaluation-canon-5diii/index.html
Also, this DR trick is not going to make the 5D3 as good as unmodified 6D, because it can't reduce shadow noise. Both cameras will get a little more highlight details, but shadow noise will stay the same.
(Of course, you'll be able to ETTR a little more, and this will reduce the noise. But if you don't ETTR, you won't get any less noise.)
Quote from: a1ex on January 20, 2014, 09:58:21 PM
(Of course, you'll be able to ETTR a little more, and this will reduce the noise. But if you don't ETTR, you won't get any less noise.)
+0.3EC will fix it!
In
my ML vs Canon ISO tests, I always +0.3EC. If Canon clips the highlight data and ML doesn't, that's Canons problem! ML fixes it by allowing you to
expose 0.3EV higher, and gain all the benefits that come with it.
I realise the scientific standpoint of understanding the effects of the gain alone. But as a photographer, ML ISO allows me to expose 0.3EV higher. I think it's important to point out that you do gain an increase in SNR through the entire exposure, you just need to learn to expose for ML, ie: +0.3EC.
Yes, this counts as ETTR'ing a little more.
I'm confused about the 12800 iso if it's real or not on the 6d.
We're talking about the canon 6d right ? (cause I'm also seeing people talking about 60d).
I always assumed that only the high (H1 and H2) and low (L1) where fakes(fake meaning digital corrected in camera, after the data leaves the cmos).
Does this mean that when I need iso 12800 or 25600 I get just as good results in using iso 6400 and do a exposure correction in a raw editor :o?
According to the data from Roger Clark, yes.
My current theory says these ISOs are pushed analogically, not digitally. Audionut is challenging it, so we don't have a final word on this. Either way, if it's pushed analogically without any noise improvement, it's just as useless as a digital push.
You need to confirm the theory experimentally and see for yourself. Don't just believe us blindly ;)
Quote from: a1ex on January 21, 2014, 09:46:09 AM
According to the data from Roger Clark, yes.
My current theory says these ISOs are pushed analogically, not digitally. Audionut is challenging it, so we don't have a final word on this. Either way, if it's pushed analogically without any noise improvement, it's just as useless as a digital push.
You need to confirm the theory experimentally and see for yourself. Don't just believe us blindly ;)
Did a not so scientific test.
Took 3 pictures in a row(didn't change focus), shutter time and f stop 3 times the same (1/4000th and f8.0), only changed the iso from 25600 down to 6400.
Openend the raw's in canon's digital photo professional (which seems more logic to me for this test then using lightroom) and bumped the exposure for the iso 6400 and 12800.
Then opened the raw's side to side to compare at 100% (6400 VS 25600) the bumped 6400 looks more harsh/crude to me. The picture loses fine detail when bumping up the iso.
Did the same again with iso 1600 VS 6400, again the bumped 1600 looks more harsh/crude, and had less fine detail.
So for now, I'm using 25600 when I need it...
Hmm should do this test again with 25600 compared to 102400...
Post the raw files.
How do I post them, Is it possible to upload them somewhere on the forum or do I have to put them somewhere on the internet and post some links ?
Did the test with 102400 VS 25600 bumped 2 stops.
Detail level looks the same to me (although it is difficult to compare if your'e looking at 100% level of pictures taken at ISO 102400 and one at 25600 bumped 2 stops...)
BUT the 102400 has definitely less chroma noise, is it possible for canon to do some chrome noise reduction BEFORE the raw files are saved ?
How raw is raw ?
Quote from: Levas on January 21, 2014, 11:38:21 AMis it possible for canon to do some chrome noise reduction BEFORE the raw files are saved
I'm rather positive they do, you see that when comparing 5d3 & 6d samples - at most iso settings, the 6d files magically have less chroma noise than 5d3, but about the same luma noise. Some test also indicate that the 6d files are a bit less sharp than 5d3 w/o nr, so 1+1 = forced noise reduction in raw.
No raw file is REALLY raw.
Quote from: a1ex on January 21, 2014, 11:09:50 AM
Post the raw files.
https://drive.google.com/folderview?id=0B1BxGc3dfMDaNkl2SUVzdlNHUUE&usp=sharing
Blind test (though it's really easy to cheat). Which is which?
(http://a1ex.magiclantern.fm/bleeding-edge/hi-iso/a.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/hi-iso/A-full.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/hi-iso/b.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/hi-iso/B-full.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/hi-iso/c.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/hi-iso/C-full.jpg)
ufraw-batch --exposure=4 6400.CR2
ufraw-batch --exposure=3 12800.CR2
ufraw-batch --exposure=2 25600.CR2
My .ufrawrc (http://a1ex.magiclantern.fm/bleeding-edge/hi-iso/.ufrawrc) (so you can check the other settings and reproduce the results).
In real shooting, also consider how many highlights you have to throw away.
Quote from: a1ex on January 21, 2014, 09:46:09 AM
Audionut is challenging it, so we don't have a final word on this.
I am? I corrected myself in the ADTG thread. On the 5D3, ISO 3200 is probably the last
useful ISO. I've posted (http://www.magiclantern.fm/forum/index.php?topic=9867.msg97110#msg97110) a few examples (http://www.magiclantern.fm/forum/index.php?topic=9867.msg96607#msg96607) now that show this.
If you mean that last data sample of mine on saturate offset, I'm not confident either way, just that the data was showing strange results.
---------------------------------------------------------------------
If that last 0.1-0.2 EV of shadow detail is more important then 1EV of highlight detail, you
may benefit from ISO 6400 vs ISO 3200. And while the data suggests that ML ISO 6400 can give 9.1-9.2 EV of dynamic range, it does not consider the other sources of noise inherent in digital photography. Bear in mind that the dynamic range data provided by most sources (including ML displays), is basing this figure on the difference between the maximum saturation level (white level) of the signal being captured and the noise floor (https://en.wikipedia.org/wiki/Noise_floor). It doesn't care how noisy the signal is between these 2 points. This is where SNR curves are useful, as they display the SNR (image quality) through the entire exposure. A general assumption for
usable dynamic range, where the shadows aren't concealed by an excessive amount of noise, is to subtract 2EV from rated values (and this is subject to processing techniques and viewer tolerance on image noise). If we apply this to the data we have for ISO 6400 on the 5D3, we are now left with around 7EV of useful data. Suddenly that 1EV of highlight data that is getting thrown out when boosting ISO another stop (ISO 3200 vs ISO 6400 in this example), might actually be useful. We've got 8EV of usable dynamic range with ISO 3200 vs 7.1EV of useful dynamic range with ISO 6400, and we haven't even considered the noisiest source in the signal yet, the shot noise.
At these low light levels (where ISO 3200/6400 is exposed correctly), the dominant noise source is the light itself (shot noise). This noise in our images is controlled entirely on the number of photons (the amount of light) hitting the sensor. As I like to point out at every opportunity,
exposure (light hitting the sensor) is only controlled by lens diameter, shutter and aperture. ISO
does not control the light hitting the sensor.
Bumping ISO does not effect the shot noise of the image, period.
Analog ISO (voltage gain) is useful for boosting the the signal from the sensor. The voltage gain is applied
at the sensor (http://www.imagesensors.org/Past%20Workshops/2003%20Workshop/2003%20Papers/05%20Sakakibara%20et%20al.pdf). So all those little bits of electronics further down the chain (ADC, DAC, etc), that have
their own inherent Signal to Noise ratio, receive a full signal (from the sensor), and retain their full SNR capability.
Analog ISO actually reduces the noise from the camera electronics by boosting the signal
through all of the remaining electronics and increasing the sensitivity of the amplifier (http://www.clarkvision.com/articles/iso/index.html)*. Random noise and Fixed Pattern Noise (banding noise), are reduced with increasing ISO (http://www.clarkvision.com/articles/evaluation-canon-5diii/). The apparent noise increase with increasing ISO is that pesky shot noise that ISO
does not control.
edit: Readout noise is also increased with gain, but is also dominated by shot noise at low exposures.
The reason why we gain any improvement at all right near the noise floor for increasing ISO,
where our images are shot noise limited (http://www.microscopyu.com/tutorials/java/digitalimaging/signaltonoise/), is because the camera electronics are still dominating the total noise near the noise floor.
*Note: Rodger Clark specifically states that ISO does not increase the sensitivity. He is referring to the sensitivity of the sensor.
-------------------------------------------------------
If you want to conduct your own testing, I would suggest that you expose your minimum test ISO at maximum saturation. This ensures that the base line exposure contains the lowest amount of shot noise. In this way, your testing procedure contains controlled shot noise for the tests. *see notes below
Remember, if you have to reduce exposure when increasing ISO (because you're tossing away 1EV of highlight data with every stop), you increase the shot noise with the exposure adjustment, and ISO does not reduce it.
Throw in tonal range with exposed bit depth (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html), and you're on your own!
*Note: You should not change exposure settings with increasing ISO. We know the highlights will get clipped by 1EV, it's the shadow detail that important in this test.
Quote from: a1ex on January 21, 2014, 03:30:41 PM
Blind test (though it's really easy to cheat). Which is which?
ooh, challenging.
After ten minutes of pixel peeping to the three 100% crops...
The first one is the most corrected with exposure (it looks more greenish then the other ones) so iso 6400
The middle one has the most evenly blue and red chroma in it, I would say untouched exposure, iso 25600
The last one is a little more greenish and has less red chroma, iso 12800.
BUT I wouldn't even bet 10 dollar on it ;D
These differences are hardly noticeable at all...
Left 12800, middle 6400, right 25600. You can check it by matching the alignment between images with the raw filenames.
I've posted it on Twitter too, and one guy got it right: https://twitter.com/autoexec_bin/status/425639803510484993
Now draw your own conclusions. If both 6400 and 25600 are nearly the same in shadows, and one of them has a 2-stop advantage in highlights, which one would you choose?
I'm convinced.
Foolish me, buying a 6d thinking that I could take pictures at a "real" iso of 25600 and make incredible time lapses of stars in the sky.
Guess we won't go higher then 6400 anymore...
And probably the same for raw video, better to go 6400 at max ?
The twitter reactions are fun, you didn't tell them you pushed the 6400 iso 2 stops and 12800 iso 1 stop, they have no clue why iso 6400 looks that terrible ;D
Quote from: Levas on January 21, 2014, 05:48:20 PM
and make incredible time lapses of stars in the sky.
Well, just look at what dmilligan is doing with a 1100D: www.magiclantern.fm/forum/index.php?topic=9860
Quote from: a1ex on January 21, 2014, 05:51:57 PM
Well, just look at what dmilligan is doing with a 1100D: www.magiclantern.fm/forum/index.php?topic=9860
Says in his description that he uses a 60Da, but nonetheless, I know what the small body's are capable off.
Had a 1000d for years. Made many time lapses with it, wasn't afraid to use his highest iso (an incredible 1600 8)).
Cropping the images 3840 by 2160 pixels and then downscale to 1920 by 1080, little frame blending with the resulting videoclips and most of the noise was gone.
But couldn't live with the fact that the 1000d did not have magic lantern support... ::)
And of course I always dreamed off Full frame and there was the canon 6d!
Silly enough I've shot more raw video with it, then taking pictures ::)
But if the iso's above 6400 are indeed fake (which seems to be the case), I'd rather liked it if canon called them H1, H2, H3 and H4. Just so we know there's some image trickery going on.
Right, only the last photo was with 1100D. Details here: http://www.magiclantern.fm/forum/index.php?topic=9923.msg95567#msg95567
Quote from: 1% on January 20, 2014, 08:40:18 PM
enabled = 1
calibrated.gain.100 = -37
calibrated.gain.200 = -34
calibrated.gain.400 = -39
calibrated.gain.800 = -41
calibrated.gain.1600 = -43
calibrated.gain.3200 = -52
calibrated.gain.6400 = -56
calibrated.gain.12800 = -156
This is how it did through a dumb extender tube. It does seem to be lowering the stdev and increasing DR if you take 1 pic with mini ISO and one without.
Quote from the link a1ex posted:
http://www.clarkvision.com/articles/evaluation-canon-6d/index.html (http://www.clarkvision.com/articles/evaluation-canon-6d/index.html)
" The constant dark level with long exposure time indicates the camera has on-sensor dark current suppression. This, however, does not suppress noise from dark current. But it results in a uniformly dark level that needs no post processing correction. No long exposure dark frames are needed when making long exposures if recording raw. "
Can this also mess up the numbers found by 1% for the 6d ?
Quote from: Audionut on January 21, 2014, 04:07:04 PM
If that last 0.1-0.2 EV of shadow detail is more important then 1EV of highlight detail, you may benefit from ISO 6400 vs ISO 3200. ...
Thanks, Audionut, for that detailed explanation. I was getting a bit confused by the discussion of noise and the claim of "fake" ISOs, and your explanation (as well as one of your links (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html)) helped greatly to straighten me out. In essence, going to higher ISOs reduces the read noise as measured in equivalent electrons, but once the read noise is much smaller than the photon shot noise, the SNR will improve only slightly at best. Thus, ISOs >~ 6400 can appear to be "fake" even if they are real analog gain, simply because you're already dominated throughout the image by photon shot noise.
Quote from: ayshih on January 21, 2014, 10:50:25 PM
In essence, going to higher ISOs reduces the read noise as measured in equivalent electrons, but once the read noise is much smaller than the photon shot noise, the SNR will improve only slightly at best. Thus, ISOs >~ 6400 can appear to be "fake" even if they are real analog gain, simply because you're already dominated throughout the image by photon shot noise.
That's correct. I took some sample images before bed last night, but I missed a couple of wanted sample points, stay tuned!
Here's a sneak peak of what to expect though. ISO 800 vs ISO 12800 (5D3)
(https://dl.dropboxusercontent.com/u/34113196/ISO/Noise/Sample1.jpg) (https://dl.dropboxusercontent.com/u/34113196/ISO/Noise/Sample2.jpg)
And ISO 800 vs ISO 12800 where the samples were brightness corrected in camera (shot noise effected).
(https://dl.dropboxusercontent.com/u/34113196/ISO/Noise/Sample3.jpg) (https://dl.dropboxusercontent.com/u/34113196/ISO/Noise/Sample4.jpg)
read the article mentioned by Audionut, learned some new things there.
From what I read in the article, the only way to know/see if an iso is "real" or digitally pushed is to look at the highlights.
If iso 25600 is derived from iso 6400 pushed by 2 stops, it should loose 2 stops of highlight data.
Shadows and midtones show no visible difference.
Wrong way around. The highlights will
always move +/- 1EV for a 1 stop ISO adjustment. The secret is in the shadows.
We know we lose 1EV of highlight data between ISO 3200 and ISO 6400, but what do we gain in the shadows!
Quote from: Levas on January 21, 2014, 09:50:14 PM
But it results in a uniformly dark level that needs no post processing correction. No long exposure dark frames are needed when making long exposures if recording raw. "
Can this also mess up the numbers found by 1% for the 6d ?
No. ISO calibration is done on the white level.
Quote from: ayshih on January 21, 2014, 10:50:25 PM
your explanation (as well as one of your links (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html)) helped greatly
The whole article is very well written and informative: http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html
FYI, Emil Martinec (http://hamilton.uchicago.edu/~ejm/) (the author of this article) is the author of the AMaZE demosaic algorithm (https://code.google.com/p/rawtherapee/source/browse/rtengine/amaze_demosaic_RT.cc) (one of the best ones) and he also pointed out the theory behind Dual ISO 5 years ago: http://www.luminous-landscape.com/forum/index.php?topic=26838
another approach, raw_diag used on iso 12800, 25600, 51200 and 102400
The gaps in the data in 51200 and even wider gaps in the 102400 say's it all.
No extra gaps in the data going from 12800 to 25600.
(https://lh6.googleusercontent.com/RBsQxHL70_08qX5x_8KWacCxje1k3JshWi81Pg_rlnK1AHLhsPEQEAXfLYSlatuVHA=w1808-h835)
(https://lh5.googleusercontent.com/aOzt8DBRz6i2g_myyREcrG2hJSYA6KyBpozSQ5rmcmA1DbJnEMAVLAwsa6xUfcsHAA=w1808-h835)
(https://lh6.googleusercontent.com/W6u-cYfJ9OqqGPA8xPw_5vcyENvu0Rr6d-i9yClZVwlviUqg6KjAQd61RvGek636bQ=w1808-h835)
(https://lh5.googleusercontent.com/LhlsQG7S2G3UJr7isiiFBRaGq34CpNmk8-LCfvYGlg4DTx_dL-AKH7Ew4FGc48GL5w=w1808-h835)
Hmm, these links don't work the way I want it.
4 graphs can be found here:
https://drive.google.com/folderview?id=0B1BxGc3dfMDaZFp6bDdmc2ZLSk0&usp=sharing
Quote from: dmilligan on January 18, 2014, 05:33:50 PM
let me translate that for you: it doesn't work on the 6D
It does by now, but help is required - see here: http://www.magiclantern.fm/forum/index.php?topic=10292