Magic Lantern Forum

Using Magic Lantern => Raw Video => Topic started by: squig on May 23, 2013, 05:11:53 AM

Title: 5D MK3 Raw sensor ISO noise samples
Post by: squig on May 23, 2013, 05:11:53 AM
Here's how the fixed pattern noise looks +5 stops no noise reduction applied.
Downloadable DNGs for your viewing pleasure https://drive.google.com/folderview?id=0BzJ3L6nv6Fn0TmF0N0xLSFBaalk&usp=sharing

Cropped
(http://i.imgur.com/OSJ6Wip.jpg)

Scaled
(http://i.imgur.com/22e4cHf.jpg)
Title: Re: 5D MK3 Raw fixed pattern noise samples
Post by: Audionut on May 23, 2013, 05:27:02 AM
Any chance of uploading the full res files somewhere.  Minimal compressed JPG's should be fine if you take care to make sure there is no noise reduction in your workflow.

edit:  Ta!

It's hard to see the fixed pattern noise under all the read noise, although you can see the read noise becoming dominant in higher ISO's (as expected).

The crops don't really show the pattern noise at all (cropped to small), it's all read noise.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on May 23, 2013, 06:23:31 PM
Quote from: squig on May 23, 2013, 05:11:53 AM
Here's how the fixed pattern noise looks +5 stops no noise reduction applied.
Downloadable DNGs for your viewing pleasure https://drive.google.com/folderview?id=0BzJ3L6nv6Fn0TmF0N0xLSFBaalk&usp=sharing

Cropped
http://i.imgur.com/OSJ6Wip.jpg

Scaled
http://i.imgur.com/22e4cHf.jpg

Squig, many thanks for the DNG black frames.

Most samples have a Black Level of 2047-2048 but some of the high ISO have different BL for any channel (ISO 3200: R,G1=2046, B,G2=2048).
ML code adds half the stdev at the measured BL giving difference of +3 up to +14 (ISO 6400 measured BL 2047, exif tag = 2061 .. this looks too much .. )

Looks like the video-raw samples follow the normal (photo-raw) Canon pattern ..

ISO 100, 200, 400 ... derived mostly* by analog amplification
Read noise is ISO100, 200 - 6400, ... 7.55, 7.9,  8.1, 8.65, 10.0, 12.6, 18.5

*Looks like there is a part of digital amp (by 1.07 ?) as there are some gaps and some "half gaps" in the histogram. I believe that there is a histogram stretching at the "video raws" so that it covers all values up to 16382 instead of the 15283 limit for the "Photo raws"

ISO 125, 250, 500 ... derived from ISO 100, 200, 400 ... by digital amplification by 1.25
Read noise is ISO125, 250 - 4000, ... 9.9, 10.2, 10.5, 11.2, 12.9, 16.4,

ISO 160, 320, 640 ... derived from ISO 200, 400, 800 ... by digital amplification by 0.80
Read noise is ISO160, 320 - 5000, ... 6.1,   6.3,   6.7,   7.7,   9.7, 14.2

I had the hope that those video-raw pixels come of binning 8 photo-raw pixels and this would decrease the noise floor by 2 stops ... but NO .. the noise floor is almost the same (a bit worse) as with the photo-raw !!. So DR is not improved !!.
Reasons for not having improved noise floor can be either pixel skipping or the system (sensor-signal transfer-DAC) working at higher speed resulting in more noise. .. or both.
Anyway, without detailed measures over all the luminance scale, we cannot have a correct conclusion.

The impression (at another thread) that ISOs 160, 320 .. are noisier is faulty.
If the samples was derived by converting with Dcraw the reason they look as noisier is the (by default) histogram stretching giving in fact a seamless +1-2 stop EV.

This together with a wrongly lower than the correct White Point that ML sets .. The exif White Level tag for ISOs 160, 320 .. is 11566 while it should be 16382*0.8=13105 if no histogram stretching happens for those ISOs and 16382 if histogram stretching exists.

It is obvious that to fully characterize the video-raws we need video-raw samples with a burned (white clipped) area.

Please anyone who can provide video-raw samples (made by the last build) at all ISO settings, which include burned areas (and not only),  .. do upload
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: a1ex on May 23, 2013, 06:29:56 PM
QuoteML code adds half the stdev at the measured BL giving difference of +3 up to +14 (ISO 6400 measured BL 2047, exif tag = 2061 .. this looks too much .. )

These samples look a bit magenta to me (or maybe I'm color blind?). If it would have been too much, these things would be green.

I'd say it's a bit too little, or very close to just right.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: nanomad on May 23, 2013, 10:18:54 PM
I see a magenta cast too
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on May 24, 2013, 01:26:31 AM
Quote from: a1ex on May 23, 2013, 06:29:56 PM
These samples look a bit magenta to me (or maybe I'm color blind?). If it would have been too much, these things would be green.

I'd say it's a bit too little, or very close to just right.

You are not color blind for sure.

But we cannot judge color casts from the black frames. Most times the dominant color there is red-magenta because (in the total absence of light) it depends on the WB multipliers and R & B are always bigger than Green. Try to change WB to "tungsten" and the dominant color is blue.

BTW the prefixed WB multipliers by ML are now G=1, R=2.477, B=1.462 which gives a very unnatural WB as if it is (in Lightroom4.4) 6200K/+66 tint (strong shift to magenta ) which makes the "as shot" conversions .. you know what non color blinds see ...

As a reference Canon gives (in exif. maker data) the Daylight multipliers as G=1.0, R=1.980, B=1.5977

The conversation about BL and magenta cast started because early ML builds calculated BL very low, less than 2040. When the BL was correct there was no objection.
Now (after some improvements of the code) looks like ML gives correct BL by calculating the average of optically black pixels and then adds this stdev/2 to increase the correct result at a wrong level.

A useful (and easy) improvement for the BL is the per channel Black Levels ..

The thing about higher than the correct Black Level is not only color casts and difficulty to build a faithful WB and color profile. It is about clipping data also.

I 'll have to experiment a bit changing exif data to provide samples supporting my position .. wait until tomorrow.

Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: squig on May 24, 2013, 02:00:10 AM
I shot that test with the 21 May build so it's post magenta cast fix, the MK3 is known for its magenta cast in raw stills. As you can see from the DNGs I uploaded I haven't messed with the setting in ACR except to 0 the tint and set a daylight white balance. I think 640 will look noisier than 800 in a real world test, I'll get around to one soon. When the weather clears up I'll get on to the white clip test.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on May 24, 2013, 06:55:00 AM
Quote from: squig on May 24, 2013, 02:00:10 AM
I shot that test with the 21 May build so it's post magenta cast fix, the MK3 is known for its magenta cast in raw stills. As you can see from the DNGs I uploaded I haven't messed with the setting in ACR except to 0 the tint and set a daylight white balance. I think 640 will look noisier than 800 in a real world test, I'll get around to one soon. When the weather clears up I'll get on to the white clip test.

Ohh .. I missed to scroll down the exif so I didn't see your settings.
Good that you used a better than the "as shot" WB but in the case of black frames we need around 4000-4500K to balance R vs B channel multipliers and take a more representative visual effect of the noise.

Regarding ISO 640 vs 800.
I just discovered that at ISO 640 the exif.White Level is 8918 !!. This looks like a bug, the WL of ISO 640, ... cannot be less than 0.80*ISO800_WL. and in order to look 1/3 ev darker it should have the same WL as the ISO 800 ...

The exif Black Level is 2050 and the measured avg. = 4047.15 = 2.85 avg. clipped levels.

At ISO 800 exif.WL = 15000 exif.BL = 2051 measured avg = 2046.8 = 4.2 avg clipped levels.

So in the end the ISO 640 black frame takes a seamless +2/3 EV compensation due to the wrong very low WL and 1.35 avg. levels less clipping at the blacks .. No wonder that it looks noisier.

I am waiting for the white clip test to clarify the actual White Levels.
Even better would be to additionally have 1-2 raw shots (both photo and video) under direct sunlight of a x-rite colorchecker passport,  in order to build a dcp color profile ..
Just for the case that the video-raw needs a different profile than the photo-raw... and I think this different profiling is possibly needed due to the scaling of the video-raw values.

Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: squig on May 24, 2013, 07:50:07 AM
I don't have the colorchecker but I've asked Samuel if he can help out.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Samuel H on May 24, 2013, 09:21:17 AM
I've shot a DataColor SpyderChecker with a 5D2+RAW:
http://www.dvxuser.com/V6/showthread.php?307967-Canon-RAW-video-color-accuracy

I think the colors are so good out of the gate that it makes no sense trying to build a better color profile. I'd be happy just working out which are the best ISO values to use.

Still, if you want a DNG from that test, send me a PM with an email address.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on May 24, 2013, 05:44:14 PM
@Samuel,

Thanks for the offer.

I am interested in shots with 5DIII of the X-rite passport color checker because I only have the "Color Checker Passport" profiler available.

Do Datacolor's Spyder checker and X-rite passport have the exactly same color patches ??. To make dcp profile with Datacolor's  I need their dedicated profiler .. No ?.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Samuel H on May 24, 2013, 09:17:32 PM
No idea. Probably not.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Samuel H on May 25, 2013, 10:42:33 AM
These are the Spydercheckr colors:
http://spyder.datacolor.com/wp-content/uploads/datacolorproductliterature/SpyderCheckr%20Color%20Data.pdf
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Audionut on May 25, 2013, 10:49:21 AM
Just looking at the skin colors, they are different.

Here's the x-rite.
http://www.babelcolor.com/download/RGB%20Coordinates%20of%20the%20Macbeth%20ColorChecker.pdf
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on May 26, 2013, 12:20:37 AM
Quote from: Samuel H on May 25, 2013, 10:42:33 AM
These are the Spydercheckr colors:
http://spyder.datacolor.com/wp-content/uploads/datacolorproductliterature/SpyderCheckr%20Color%20Data.pdf

It is very different from Macbeth cc24. So no way to build dcp profiles with free software.

But I think we can make icc profiles with Agryllcms (if needed).
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Gwydien on May 26, 2013, 02:07:25 AM
are these photo's?  Cus when i did this test with video it was undenyably clearer on 320,640,1250,2500, and 5000
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Audionut on May 26, 2013, 02:10:31 AM
Quote from: Gwydien on May 26, 2013, 02:07:25 AM
are these photo's?  Cus when i did this test with video it was undenyably clearer on 320,640,1250,2500, and 5000

They are RAW images.

H.264 video will be cleaner with ISO's 160,320 et all because the camera in doing in-house ETTR and exposure reduction.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on May 26, 2013, 05:06:41 AM
Quote from: Gwydien on May 26, 2013, 02:07:25 AM
are these photo's?  Cus when i did this test with video it was undenyably clearer on 320,640,1250,2500, and 5000

Those ISOs 160 - 320 ... were darker than the rest isn't it ?.

If we change the Black Level and White Level tags at the DNG's exif with the correct values they will look cleaner also ...
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: uninexus on June 05, 2013, 07:56:05 AM
Is there a decisive conclusion as to some ISO settings being less noisy than others ? If there isn't - can I help somehow to make that determination ? If there is - is it muliples of 100 ? Is everyone in agreement ?
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: a1ex on June 05, 2013, 10:07:22 AM
Yes. ISO 160, 200 and 250 are *identical* in raw.

@squig: please update the test with today's build; older ML had some difference in white levels at intermediate ISOs, because I've assumed it behaves like in photo mode (but it doesn't).
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on June 11, 2013, 09:51:43 PM
Quote from: a1ex on June 05, 2013, 10:07:22 AM
Yes. ISO 160, 200 and 250 are *identical* in raw.

@squig: please update the test with today's build; older ML had some difference in white levels at intermediate ISOs, because I've assumed it behaves like in photo mode (but it doesn't).

Hi a1ex,

I suppose by *identical* you mean that they have the same Black & White clipping points which is real.

Supposing that the data which ML currently use didn't change from the above (old) samples (White Level is not a data change ) ..

If "integer" and intermediate ISOs were identical then the noise at the Black Frames would be identical also .. but it is not.

The content at these Black Frames is different as if they are scaled digitally ..

(http://i.imgur.com/q1vAo9x.png) (http://i.imgur.com/EGIsJQs.png)
(http://i.imgur.com/xNKabP0.png) (http://i.imgur.com/15ufXdP.png)

It's obvious that ISO160 is a digitally compressed ISO200 and ISO125 is a digitally expanded ISO100.

Now the question is, what is happening with the highlights ??. Are the highlights for ISO160 (which extend to the same clipping point as ISO200) real or fake (data manipulation) ??.

We have to investigate a bit with "White Frames" .. searching for non linearities. If there is a non linear data manipulation then, after WB, the clipped highlights should have a color cast and we'll have to take care of it by setting a lower White point for this case.

If all range is linear then it means that ISO 200 is clipped with no reason ..


Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: a1ex on June 11, 2013, 10:16:31 PM
If I keep the same black and white levels, the raw histogram doesn't change. Also, if I change the digital ISO in image fine-tuning menu, the image doesn't change either.

If I change the white level in exif, the image is developed differently in ufraw (higher white level = darker image).

I don't know what RawDigger is doing to show these graphs, because I can't look inside. I can try to repeat the tests myself - were they taken with lens cap on?

In YUV (JPEG), the highlights at ISO 160 are a bit fake. ML is faking the histogram in a similar way in photo mode LV, so the histogram matches the CR2 one (this is needed for auto ETTR and highlight warnings in photo mode LiveView).
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: a1ex on June 11, 2013, 10:48:26 PM
OK, here's my test:

(http://a1ex.magiclantern.fm/bleeding-edge/raw/iso.png)

red = iso160, green = iso200, blue = iso250

5d3, movie mode, 1/90, lens cap on, dng silent pics
black/white: 2047 15000


import os, sys, subprocess, shlex
from pylab import *

def get_raw_data(file):
    cmd1 = "dcraw -c -D -4 -o 0 -h -T '%s'" % file
    cmd2 = "convert - -type Grayscale -format %c histogram:info:-"

    p1 = subprocess.Popen(shlex.split(cmd1), stdout=subprocess.PIPE)
    p2 = subprocess.Popen(shlex.split(cmd2), stdin=p1.stdout, stdout=subprocess.PIPE)
    lines = p2.communicate()[0].split("\n")
    X = []
    for l in lines[1:]:
        p1 = l.find("(")
        if p1 > 0:
            p2 = l.find(",", p1)
            level = int(l[p1+1:p2])
            count = int(l[:p1-2])
            X += [level]*count
    return array(X)

a = get_raw_data("160.dng")
b = get_raw_data("200.dng")
c = get_raw_data("250.dng")

hist(a, 500, color='r')
hist(b+0.2, 500, color='g')
hist(c-0.2, 500, color='b')
show()

Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on June 12, 2013, 02:49:04 AM
@ a1ex,

I used squig's (outdayted) samples for my histograms. Can you share your samples ?.

In Dcraw command line you use -h which (at least in my system) gives demosaiced data .. not a big problem as there is minimal interpolation. I really don't understand how you take gappless histogram. Rawanalyze show "gappfull" histo also.

Can you export histograms per channel ?. If not try http://www.libraw.org/download 4channels.exe
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Audionut on June 12, 2013, 04:25:21 AM
Here are 6 samples.  ISO's 160, 200 and 250.  Both with lens cap on, and of a white image as close as I could get to saturation @ ISO 200.

https://dl.dropboxusercontent.com/u/34113196/ML/D46A3602.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3603.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3604.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3605.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3606.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3607.CR2

I couldn't really understand the results shown by RawDigger and a1ex's observation that these ISO's are the identical raw data.
So I got my test bunny, and again shot @ ISO 200 as close to saturation as possible.  Then with the same shutter/aperture, I shot again @ ISO's 160 and 250.

I don't really use dcraw so my process might be flawed, but to me, it's clear that ISO 250 has the raw data pushed into saturation.  Based on that observation, ISO 160 simply has all of it's data pushed further south into the noise floor.

So the questions for me, am I simply not using the correct process to recover the detail, or has it indeed been pushed into saturation for the digital pushed ISO's, never to be recovered*?  If I am using the correct process, when in the pipeline is the digital manipulation occurring?  Have the digitally pulled ISO's been pulled into the noise floor of the camera?  Or is it done after the effects of noise etc etc in the circuitry of the camera?


The bunny test images.
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3615.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3616.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/D46A3617.CR2

*While I can understand that it is (was), the same initial raw data, if it's being pushed into saturation with a resulting loss of detail, it's open to interpretation on whether it's still the same raw data from a usability stand point.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: a1ex on June 12, 2013, 09:15:24 AM
Wait a minute, we were talking about video, not photo. They do not behave in the same way.

My samples: 160 (http://a1ex.magiclantern.fm/bleeding-edge/raw/noice160.dng)  200 (http://a1ex.magiclantern.fm/bleeding-edge/raw/noice200.dng)  250 (http://a1ex.magiclantern.fm/bleeding-edge/raw/noice250.dng)

dcraw -h does not do demosaicing, it just groups a RGGB cell in a RGB pixel.

The difference in squig's files comes from different white levels, because I thought raw in movie mode behaves like in photo mode when you change the ISO.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Audionut on June 12, 2013, 09:32:28 AM
Quote from: a1ex on June 12, 2013, 09:15:24 AM
Wait a minute, we were talking about video, not photo. They do not behave in the same way.

That explains that.

Must remember that camera does video too!  :o
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on June 12, 2013, 10:00:24 AM
Quote from: Audionut on June 12, 2013, 04:25:21 AM
Here are 6 samples.  ISO's 160, 200 and 250.  Both with lens cap on, and of a white image as close as I could get to saturation @ ISO 200.

.....

I couldn't really understand the results shown by RawDigger and a1ex's observation that these ISO's are the identical raw data.
So I got my test bunny, and again shot @ ISO 200 as close to saturation as possible.  Then with the same shutter/aperture, I shot again @ ISO's 160 and 250.

I don't really use dcraw so my process might be flawed, but to me, it's clear that ISO 250 has the raw data pushed into saturation.  Based on that observation, ISO 160 simply has all of it's data pushed further south into the noise floor.

So the questions for me, am I simply not using the correct process to recover the detail, or has it indeed been pushed into saturation for the digital pushed ISO's, never to be recovered*?  If I am using the correct process, when in the pipeline is the digital manipulation occurring?  Have the digitally pulled ISO's been pulled into the noise floor of the camera?  Or is it done after the effects of noise etc etc in the circuitry of the camera?
...

*While I can understand that it is (was), the same initial raw data, if it's being pushed into saturation with a resulting loss of detail, it's open to interpretation on whether it's still the same raw data from a usability stand point.

Hello Audionut,

As Alex said we need video-raw files not photo raw. Although your samples will be useful to compare histograms between the two modes.

For a start we need video-raws with burned areas (totally saturated) for every ISO to declare the White clipping point. Then we can continue investigating what happens near saturation.

We will have answers for your questions when we will have proper samples. For the moment just keep in mind that digital push-pull takes place "after the effects of noise etc etc in the circuitry of the camera". By pushing by 1.25 (for ISOs 125-250 ..) it depends on where is the "integer" ISOs real White point, if it's at 0.80 of the max 14bit level then nothing changes ..
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on June 12, 2013, 10:16:30 AM
Quote from: a1ex on June 12, 2013, 09:15:24 AM
Wait a minute, we were talking about video, not photo. They do not behave in the same way.

My samples: 160 (http://a1ex.magiclantern.fm/bleeding-edge/raw/noice160.dng)  200 (http://a1ex.magiclantern.fm/bleeding-edge/raw/noice200.dng)  250 (http://a1ex.magiclantern.fm/bleeding-edge/raw/noice250.dng)

dcraw -h does not do demosaicing, it just groups a RGGB cell in a RGB pixel.

The difference in squig's files comes from different white levels, because I thought raw in movie mode behaves like in photo mode when you change the ISO.

Hello Alex, thanks for the samples.

As I wrote for Dcraw's -h the interpolation is minimal, just combines the two green channels of a 2X2 square into one. So you loose a little info regarding channel imbalance ..

BTW I think it would be useful for raw2dng and the pattern correction it does if it have available the per channel Black level. Just run 4 loops with N/4 samples instead of 1 with N samples. I think Adobe's converters make use of those four independent values and others will follow .. It can make a difference at high ISOs.

I can change the metadata in squig's samples so this wrong WL was not a problem.

Did you checked Libraw's 4channel code ?.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: a1ex on June 12, 2013, 10:24:50 AM
Running through the raw samples for vertical banding, collected from the forum, I've noticed the white level is 16382 or 16383. The 16383 one may be because some samples were already corrected for banding, and I've reversed the raw data.

If this is true in LiveView for all ISOs, that's interesting. Will check on the other cameras.
edit: on 5D2 it varies between 15850 and 15882.

I didn't check the 4channel, because the conclusion is clear for me: digital ISO has no effect on LiveView raw data.

I can compute 4 black levels, no big deal. Though it's unlikely to make a difference, can you find some examples?
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: squig on June 12, 2013, 11:29:02 AM
Quote from: a1ex on June 12, 2013, 10:24:50 AM
I didn't check the 4channel, because the conclusion is clear for me: digital ISO has no effect on LiveView raw data.
(http://i.imgur.com/wjdCtd5.jpg)
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: a1ex on June 12, 2013, 11:36:24 AM
See the post from Ilias. You need to do the ISO test again btw; the ML version you have used had wrong metadata for white levels.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on June 12, 2013, 12:26:04 PM
Quote from: a1ex on June 12, 2013, 10:24:50 AM
Running through the raw samples for vertical banding, collected from the forum, I've noticed the white level is 16382 or 16383. The 16383 one may be because some samples were already corrected for banding, and I've reversed the raw data.

If this is true in LiveView for all ISOs, that's interesting. Will check on the other cameras.
edit: on 5D2 it varies between 15850 and 15882.

I didn't check the 4channel, because the conclusion is clear for me: digital ISO has no effect on LiveView raw data.

I can compute 4 black levels, no big deal. Though it's unlikely to make a difference, can you find some examples?

You are correct, whith the up to date samples there is no difference between ISO 160, 200, 250. May be because of a different LV mode vs squig's (now ancient) samples.

Does White Level goes up to 14bit limit with all ISOs ?. If so 15000 is over conservative, you can set it at 16000 leaving 382 levels clipped just in case a setting makes something different.
Hi ISO NR and long exposures comes in my mind which (in photo mode) results in a different clipping .. instead of a single level it becomes a bell distribution and to prevent magenta casts we have to clip at the lower value of this distribution.
Same as Gulliermo's article .. http://translate.google.com/translate?u=http%3A%2F%2Fwww.guillermoluijk.com%2Ftutorial%2Fsatlevel%2Findex.htm&langpair=es|en&hl=EN&ie=UTF-8

But first we have to check the behavior near saturation (magenta casts).   

I also think that we have somehow corrected data. The sad thinks is that this correction can be erroneous .. sometimes in black frames we see a black band with no noise and I think this is exactly erroneous over-correction.

I could find samples with heavy BL imbalance at high ISOs but only for photo mode which is not useful as we see that LV data are different.

PS. When calculating BL sometimes there are ouliers which disturb the result .. if it's possible to remove them the result would be more accurate.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: a1ex on June 12, 2013, 12:35:46 PM
Yeah, I'm trying to warn people not to record in photo mode; I've disabled the raw_rec operation in photo mode, and other guys are enabling it back...

Sigh...

For outliers, median rejects them very well, but it's harder to compute than mean. But yeah, a hot pixel in that area may affect the results.

Do you have any samples with black band and no noise?
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: IliasG on June 13, 2013, 11:06:59 AM
Quote from: a1ex on June 12, 2013, 12:35:46 PM
..
For outliers, median rejects them very well, but it's harder to compute than mean. But yeah, a hot pixel in that area may affect the results.

Do you have any samples with black band and no noise?

Whats the current state with hot/dead pixels in video raws ?. are they corrected in camera ?.

I have some samples from Canon40D photo-raw black frames with black band  .. I can post when I 'll be back home (weekend) but I don't think they are useful.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: a1ex on June 13, 2013, 11:39:00 AM
Some cameras set them to 0 (e.g. 60D), others correct them and output clean raw data (5D3). I don't know if there are other situations.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Danne on June 17, 2013, 02:30:13 PM
Quote from: a1ex on June 13, 2013, 11:39:00 AM
Some cameras set them to 0 (e.g. 60D), others correct them and output clean raw data (5D3). I don't know if there are other situations.

Are you sure that 5d mark 3 "cleans" the rawfile in camera? When in 5x mode I sure get a few dead/hotpixels. I can even see the in playback mode of the rawfile. See posted example. Of course this gets visible in high iso and underexposed shots. Acr is removing them but when I go straight to premiere with the ginger plugin they show up.

(http://s23.postimg.org/im297itqj/beaver_1.jpg)
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Audionut on June 27, 2013, 03:59:29 AM
Some updated samples.  Original DNG's (https://www.dropbox.com/sh/ty683rlgukj2j54/k2tz0rKOdm)

As expected, no difference (slight differences from varying shot to shot read noise) from digital ISO's as compared to their base ISO's.
Massive jump in read noise @ ISO 800.  Read noise then diminishes through ISO 1600 and ISO 3200 where pattern noise starts to dominate.  All noise then becomes amplified through ISO's 6400, 12800 and 25600.

Some resized samples at base ISO's.  +5EV - Temp 3000 - Tint -150

100
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/100.jpg)

200
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/200.jpg)

400
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/400.jpg)

800
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/800.jpg)

1600
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/1600.jpg)

3200
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/3200.jpg)

6400
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/6400.jpg)

12800
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/12800.jpg)

25600
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/25600.jpg)


A sample @ ISO 800 Photo Mode
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/ISO%20800%20Photo%20Mode.jpg)

The pattern noise in video mode is also destructive as it's not fixed.  It can't be averaged out.
Here is the ISO 800 shot in photo mode with 16 frames averaged to remove the fixed pattern noise.
(https://dl.dropboxusercontent.com/u/34113196/ML/Noise%20samples/Averaged_dark_frames.jpg)
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: eyeland on June 29, 2013, 04:58:36 PM
So the conclusion is that the digitally pulled/pushed ISO values have no effect on noise in RAW?
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: Audionut on June 30, 2013, 02:44:08 AM
Quote from: Audionut on June 27, 2013, 03:59:29 AM
As expected, no difference (slight differences from varying shot to shot read noise) from digital ISO's as compared to their base ISO's.
Title: Re: 5D MK3 Raw sensor ISO noise samples
Post by: heniloux on July 07, 2013, 08:19:46 PM
Can one conclude that according to AUDIONUTS tests that the BASE STOP of the RAW function would be iso 100? i.e the rating with the least amount of noise?