5D MK3 Raw sensor ISO noise samples

[squig]

Here's how the fixed pattern noise looks +5 stops no noise reduction applied.
Downloadable DNGs for your viewing pleasure https://drive.google.com/folderview?id=0BzJ3L6nv6Fn0TmF0N0xLSFBaalk&usp=sharing

Cropped


Scaled

[Audionut]

Any chance of uploading the full res files somewhere.  Minimal compressed JPG's should be fine if you take care to make sure there is no noise reduction in your workflow.

edit:  Ta!

It's hard to see the fixed pattern noise under all the read noise, although you can see the read noise becoming dominant in higher ISO's (as expected).

The crops don't really show the pattern noise at all (cropped to small), it's all read noise.

[IliasG]

Here's how the fixed pattern noise looks +5 stops no noise reduction applied.
Downloadable DNGs for your viewing pleasure https://drive.google.com/folderview?id=0BzJ3L6nv6Fn0TmF0N0xLSFBaalk&usp=sharing

Cropped
http://i.imgur.com/OSJ6Wip.jpg

Scaled
http://i.imgur.com/22e4cHf.jpg

Squig, many thanks for the DNG black frames.

Most samples have a Black Level of 2047-2048 but some of the high ISO have different BL for any channel (ISO 3200: R,G1=2046, B,G2=2048).
ML code adds half the stdev at the measured BL giving difference of +3 up to +14 (ISO 6400 measured BL 2047, exif tag = 2061 .. this looks too much .. )
 
Looks like the video-raw samples follow the normal (photo-raw) Canon pattern ..

ISO 100, 200, 400 ... derived mostly* by analog amplification
Read noise is ISO100, 200 - 6400, ... 7.55, 7.9,  8.1, 8.65, 10.0, 12.6, 18.5

*Looks like there is a part of digital amp (by 1.07 ?) as there are some gaps and some "half gaps" in the histogram. I believe that there is a histogram stretching at the "video raws" so that it covers all values up to 16382 instead of the 15283 limit for the "Photo raws"

ISO 125, 250, 500 ... derived from ISO 100, 200, 400 ... by digital amplification by 1.25
Read noise is ISO125, 250 - 4000, ... 9.9, 10.2, 10.5, 11.2, 12.9, 16.4,

ISO 160, 320, 640 ... derived from ISO 200, 400, 800 ... by digital amplification by 0.80
Read noise is ISO160, 320 - 5000, ... 6.1,   6.3,   6.7,   7.7,   9.7, 14.2

I had the hope that those video-raw pixels come of binning 8 photo-raw pixels and this would decrease the noise floor by 2 stops ... but NO .. the noise floor is almost the same (a bit worse) as with the photo-raw !!. So DR is not improved !!.
Reasons for not having improved noise floor can be either pixel skipping or the system (sensor-signal transfer-DAC) working at higher speed resulting in more noise. .. or both.
Anyway, without detailed measures over all the luminance scale, we cannot have a correct conclusion.

The impression (at another thread) that ISOs 160, 320 .. are noisier is faulty.
If the samples was derived by converting with Dcraw the reason they look as noisier is the (by default) histogram stretching giving in fact a seamless +1-2 stop EV.

This together with a wrongly lower than the correct White Point that ML sets .. The exif White Level tag for ISOs 160, 320 .. is 11566 while it should be 16382*0.8=13105 if no histogram stretching happens for those ISOs and 16382 if histogram stretching exists.

It is obvious that to fully characterize the video-raws we need video-raw samples with a burned (white clipped) area.

Please anyone who can provide video-raw samples (made by the last build) at all ISO settings, which include burned areas (and not only),  .. do upload

[a1ex]

ML code adds half the stdev at the measured BL giving difference of +3 up to +14 (ISO 6400 measured BL 2047, exif tag = 2061 .. this looks too much .. )

These samples look a bit magenta to me (or maybe I'm color blind?). If it would have been too much, these things would be green.

I'd say it's a bit too little, or very close to just right.

[nanomad]

I see a magenta cast too

[IliasG]

These samples look a bit magenta to me (or maybe I'm color blind?). If it would have been too much, these things would be green.

I'd say it's a bit too little, or very close to just right.

You are not color blind for sure.

But we cannot judge color casts from the black frames. Most times the dominant color there is red-magenta because (in the total absence of light) it depends on the WB multipliers and R & B are always bigger than Green. Try to change WB to "tungsten" and the dominant color is blue.

BTW the prefixed WB multipliers by ML are now G=1, R=2.477, B=1.462 which gives a very unnatural WB as if it is (in Lightroom4.4) 6200K/+66 tint (strong shift to magenta ) which makes the "as shot" conversions .. you know what non color blinds see ...

As a reference Canon gives (in exif. maker data) the Daylight multipliers as G=1.0, R=1.980, B=1.5977

The conversation about BL and magenta cast started because early ML builds calculated BL very low, less than 2040. When the BL was correct there was no objection.
Now (after some improvements of the code) looks like ML gives correct BL by calculating the average of optically black pixels and then adds this stdev/2 to increase the correct result at a wrong level.

A useful (and easy) improvement for the BL is the per channel Black Levels ..

The thing about higher than the correct Black Level is not only color casts and difficulty to build a faithful WB and color profile. It is about clipping data also.

I 'll have to experiment a bit changing exif data to provide samples supporting my position .. wait until tomorrow.

[squig]

I shot that test with the 21 May build so it's post magenta cast fix, the MK3 is known for its magenta cast in raw stills. As you can see from the DNGs I uploaded I haven't messed with the setting in ACR except to 0 the tint and set a daylight white balance. I think 640 will look noisier than 800 in a real world test, I'll get around to one soon. When the weather clears up I'll get on to the white clip test.

[IliasG]

I shot that test with the 21 May build so it's post magenta cast fix, the MK3 is known for its magenta cast in raw stills. As you can see from the DNGs I uploaded I haven't messed with the setting in ACR except to 0 the tint and set a daylight white balance. I think 640 will look noisier than 800 in a real world test, I'll get around to one soon. When the weather clears up I'll get on to the white clip test.

Ohh .. I missed to scroll down the exif so I didn't see your settings.
Good that you used a better than the "as shot" WB but in the case of black frames we need around 4000-4500K to balance R vs B channel multipliers and take a more representative visual effect of the noise.

Regarding ISO 640 vs 800.
I just discovered that at ISO 640 the exif.White Level is 8918 !!. This looks like a bug, the WL of ISO 640, ... cannot be less than 0.80*ISO800_WL. and in order to look 1/3 ev darker it should have the same WL as the ISO 800 ...

The exif Black Level is 2050 and the measured avg. = 4047.15 = 2.85 avg. clipped levels.

At ISO 800 exif.WL = 15000 exif.BL = 2051 measured avg = 2046.8 = 4.2 avg clipped levels.

So in the end the ISO 640 black frame takes a seamless +2/3 EV compensation due to the wrong very low WL and 1.35 avg. levels less clipping at the blacks .. No wonder that it looks noisier.

I am waiting for the white clip test to clarify the actual White Levels.
Even better would be to additionally have 1-2 raw shots (both photo and video) under direct sunlight of a x-rite colorchecker passport,  in order to build a dcp color profile ..
Just for the case that the video-raw needs a different profile than the photo-raw... and I think this different profiling is possibly needed due to the scaling of the video-raw values.

[squig]

I don't have the colorchecker but I've asked Samuel if he can help out.

[Samuel H]

I've shot a DataColor SpyderChecker with a 5D2+RAW:
http://www.dvxuser.com/V6/showthread.php?307967-Canon-RAW-video-color-accuracy

I think the colors are so good out of the gate that it makes no sense trying to build a better color profile. I'd be happy just working out which are the best ISO values to use.

Still, if you want a DNG from that test, send me a PM with an email address.

[IliasG]

@Samuel,

Thanks for the offer.

I am interested in shots with 5DIII of the X-rite passport color checker because I only have the "Color Checker Passport" profiler available.

Do Datacolor's Spyder checker and X-rite passport have the exactly same color patches ??. To make dcp profile with Datacolor's  I need their dedicated profiler .. No ?.

[Samuel H]

No idea. Probably not.

[Samuel H]

These are the Spydercheckr colors:
http://spyder.datacolor.com/wp-content/uploads/datacolorproductliterature/SpyderCheckr%20Color%20Data.pdf

[Audionut]

Just looking at the skin colors, they are different.

Here's the x-rite.
http://www.babelcolor.com/download/RGB%20Coordinates%20of%20the%20Macbeth%20ColorChecker.pdf

[IliasG]

These are the Spydercheckr colors:
http://spyder.datacolor.com/wp-content/uploads/datacolorproductliterature/SpyderCheckr%20Color%20Data.pdf

It is very different from Macbeth cc24. So no way to build dcp profiles with free software.

But I think we can make icc profiles with Agryllcms (if needed).

[Gwydien]

are these photo's?  Cus when i did this test with video it was undenyably clearer on 320,640,1250,2500, and 5000

[Audionut]

are these photo's?  Cus when i did this test with video it was undenyably clearer on 320,640,1250,2500, and 5000

They are RAW images.

H.264 video will be cleaner with ISO's 160,320 et all because the camera in doing in-house ETTR and exposure reduction.

[IliasG]

are these photo's?  Cus when i did this test with video it was undenyably clearer on 320,640,1250,2500, and 5000

Those ISOs 160 - 320 ... were darker than the rest isn't it ?.

If we change the Black Level and White Level tags at the DNG's exif with the correct values they will look cleaner also ...

[uninexus]

Is there a decisive conclusion as to some ISO settings being less noisy than others ? If there isn't - can I help somehow to make that determination ? If there is - is it muliples of 100 ? Is everyone in agreement ?

[a1ex]

Yes. ISO 160, 200 and 250 are *identical* in raw.

@squig: please update the test with today's build; older ML had some difference in white levels at intermediate ISOs, because I've assumed it behaves like in photo mode (but it doesn't).

[IliasG]

Yes. ISO 160, 200 and 250 are *identical* in raw.

@squig: please update the test with today's build; older ML had some difference in white levels at intermediate ISOs, because I've assumed it behaves like in photo mode (but it doesn't).

Hi a1ex,

I suppose by *identical* you mean that they have the same Black & White clipping points which is real.

Supposing that the data which ML currently use didn't change from the above (old) samples (White Level is not a data change ) ..

If "integer" and intermediate ISOs were identical then the noise at the Black Frames would be identical also .. but it is not.

The content at these Black Frames is different as if they are scaled digitally ..




It's obvious that ISO160 is a digitally compressed ISO200 and ISO125 is a digitally expanded ISO100.

Now the question is, what is happening with the highlights ??. Are the highlights for ISO160 (which extend to the same clipping point as ISO200) real or fake (data manipulation) ??.

We have to investigate a bit with "White Frames" .. searching for non linearities. If there is a non linear data manipulation then, after WB, the clipped highlights should have a color cast and we'll have to take care of it by setting a lower White point for this case.

If all range is linear then it means that ISO 200 is clipped with no reason ..

[a1ex]

If I keep the same black and white levels, the raw histogram doesn't change. Also, if I change the digital ISO in image fine-tuning menu, the image doesn't change either.

If I change the white level in exif, the image is developed differently in ufraw (higher white level = darker image).

I don't know what RawDigger is doing to show these graphs, because I can't look inside. I can try to repeat the tests myself - were they taken with lens cap on?

In YUV (JPEG), the highlights at ISO 160 are a bit fake. ML is faking the histogram in a similar way in photo mode LV, so the histogram matches the CR2 one (this is needed for auto ETTR and highlight warnings in photo mode LiveView).

[a1ex]

OK, here's my test:



red = iso160, green = iso200, blue = iso250

5d3, movie mode, 1/90, lens cap on, dng silent pics
black/white: 2047 15000

Code: [Select]

import os, sys, subprocess, shlex
from pylab import *

def get_raw_data(file):
    cmd1 = "dcraw -c -D -4 -o 0 -h -T '%s'" % file
    cmd2 = "convert - -type Grayscale -format %c histogram:info:-"

    p1 = subprocess.Popen(shlex.split(cmd1), stdout=subprocess.PIPE)
    p2 = subprocess.Popen(shlex.split(cmd2), stdin=p1.stdout, stdout=subprocess.PIPE)
    lines = p2.communicate()[0].split("\n")
    X = []
    for l in lines[1:]:
        p1 = l.find("(")
        if p1 > 0:
            p2 = l.find(",", p1)
            level = int(l[p1+1:p2])
            count = int(l[:p1-2])
            X += [level]*count
    return array(X)

a = get_raw_data("160.dng")
b = get_raw_data("200.dng")
c = get_raw_data("250.dng")

hist(a, 500, color='r')
hist(b+0.2, 500, color='g')
hist(c-0.2, 500, color='b')
show()


[IliasG]

@ a1ex,

I used squig's (outdayted) samples for my histograms. Can you share your samples ?.

In Dcraw command line you use -h which (at least in my system) gives demosaiced data .. not a big problem as there is minimal interpolation. I really don't understand how you take gappless histogram. Rawanalyze show "gappfull" histo also.

Can you export histograms per channel ?. If not try http://www.libraw.org/download 4channels.exe

[Audionut]

Here are 6 samples.  ISO's 160, 200 and 250.  Both with lens cap on, and of a white image as close as I could get to saturation @ ISO 200.

https://dl.dropboxusercontent.com/u/34113196/ML/D46A3602.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3603.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3604.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3605.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3606.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3607.CR2

I couldn't really understand the results shown by RawDigger and a1ex's observation that these ISO's are the identical raw data.
So I got my test bunny, and again shot @ ISO 200 as close to saturation as possible.  Then with the same shutter/aperture, I shot again @ ISO's 160 and 250.

I don't really use dcraw so my process might be flawed, but to me, it's clear that ISO 250 has the raw data pushed into saturation.  Based on that observation, ISO 160 simply has all of it's data pushed further south into the noise floor.

So the questions for me, am I simply not using the correct process to recover the detail, or has it indeed been pushed into saturation for the digital pushed ISO's, never to be recovered*?  If I am using the correct process, when in the pipeline is the digital manipulation occurring?  Have the digitally pulled ISO's been pulled into the noise floor of the camera?  Or is it done after the effects of noise etc etc in the circuitry of the camera?


The bunny test images.
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3615.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3616.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3617.CR2

*While I can understand that it is (was), the same initial raw data, if it's being pushed into saturation with a resulting loss of detail, it's open to interpretation on whether it's still the same raw data from a usability stand point.