Trade resolution for sensitivity / less noise (pixel binning)

[swissbear]

In theory, it is possible to trade resolution for either increased sensor sensitivity or less noise.

The idea is quite simple: reduce 4 or 9 pixels to one.

For increased sensitivity, you add up the R/G/B values from each pixel. This increases the sensitivity by 4 or even 9 stops. Problem: as only 1/4 or 1/9 of the sensor's DR is used, 2 or 3-4 bits of information are lost.

For less noise, you take the average of the R/G/B values from each pixel. I have not found any downsides beside the loss of resolution.

Of course, both manipulations can be done in post, but i don't know any software that does it.
BUT, especially the increased sensitivity would be quite nice in-camera for review purposes.

Is it even possible to tinker around with CR2's?
Increasing "ISO" by 9 stops is quite a bang - is the sensor linear enough on the dark side or is this just a nice theory?

[Audionut]

Pixel binning.

So while you can do pixel binning in post if you want, it doesn’t particularly gain much. So is it still worth doing? Yes, when done during capture.

The key issue is overcoming the noise floor, and once the noise is present, no amount of post work with remove it. Read noise is one of the problems, which is the noise generated when the pixels are read from the sensor. If pixel binning is done on sensor, i.e the sets of pixels are combined before being read, then the read noise will be greatly reduced (one portion of noise per final pixel instead of 4/9/etc). This means that additive binning really can provide some benefit in this case by reducing that source of noise. This is performed in some point and shoot cameras where high ISO images come out with reduced resolution, but also on high end cameras such as the Phase One medium format backs with the Sensor+ setting.

[a1ex]

For increased sensitivity, you add up the R/G/B values from each pixel. This increases the sensitivity by 4 or even 9 stops.

How exactly did you reach these numbers? Averaging 4 pixels will give you 1 stop and 16 pixels will give you 2 stops.

If pixel binning is done on sensor, i.e the sets of pixels are combined before being read, then the read noise will be greatly reduced

Not on Canons, because most of the noise is introduced after the data leaves the sensor.

And 5D3 already does this in LiveView according to this:

Hint: http://www.chipworks.com/en/technical-competitive-analysis/resources/blog/full-frame-dslr-cameras-canon-stays-the-course/

"Canon’s X030334 uses an additional transistor (T5) that can be used to  sum the FD nodes of three rows of even (or odd) pixels of the same  color."

So, there's no benefit if you would do this in camera (assumming you can figure out how to do this).

[swissbear]

How exactly did you reach these numbers? Averaging 4 pixels will give you 1 stop and 16 pixels will give you 2 stops.

I'm by far no expert on this topic. Next try: if the light receiving area is doubled the sensitivity is also doubled because when summing up two pixels, to reach 100% you need 2*50%. Going on with this i'd get for summing up 2x2 pixel a sensitivity increase of 2 stops, for summing up 3x3 a bit more than 3 stops.

There really is no much benefit for doing the averaging thing in camera, but you need to change some very few lines of code from the summing up procedure.

On the other hand, if the summing up is well implemented, then the camera meters for the "wrong" ISO and you get a quite regular CR2 file which fits in the workflow.

Does anyone know a piece of software that can do these calculations?
Or is there a documentation fo the CR2 file format so i can try to do it myself?

[chmee]

if the light receiving area is doubled the sensitivity is also doubled

hmm. in one dimension? example: you're getting a better snr in horizontal, and no benefit in vertical, sums up in 0,5 snr-plus looking on whole picture. why you dont use photoshop (or something, that has a good rescaler) - you're getting your goals. mostly all photoguys're doin this all the day. IF you had a full 5DIII-RAW (5760*3240) as output, you'd do this, but you dont have with ml-raw/mlv.

..know a piece of software that can do these calculations?..

binning is a good trick, but it does not give the best output. binning changes moiré and aa-behaviour as well, compared to algorithms like bilinear/bicubic its worse.

regards chmee

[a1ex]

for summing up 2x2 pixel a sensitivity increase of 2 stops, for summing up 3x3 a bit more than 3 stops.

http://www.statlect.com/normal_distribution_linear_combinations.htm

If you average 4 pixels from a flat area, corrupted by Gaussian noise, per-pixel noise will be reduced by log2(sqrt(4)) = 1 EV.
From 16 pixels, log2(sqrt(16)) = 2 EV.
From 9 pixels, you get 1.58 EV.
For 3 EV, you need to average only 64 pixels.

If you do the averaging before the analog amplifiers, you will average out only the photon shot noise.
If you do this in post, you will average out the electronics noise too.

you need to change some very few lines of code from the summing up procedure.

Looking forward to see your patch.

[swissbear]

the averaging for better noise results is just a by-product of my main idea which i now learned is called "binning". I am talking mainly about the procedure where the values of several pixels are summed up (without dividing it afterwards by the number of pixels).
As the CCD sensor is not anymore used, only software based binning is an option. Whether it is possible to bin as a first step when the sensor data reaches the image processor (aka DIGIC) or only after the CR2 file is output, i have no clue. The first would be nice, as all other software based image processing stuff can still be applied.

Looking forward to see your patch.

When binning is written, its a very small change to also have an averaging procedure.

The link provided by audionut is informative and tells me at least in high ISO's, its not really an option. But what if you push some 400 ISO (on 600D/50D, a rather clean image) by 2 or 3 stops, that means by adding up the pixel values from a 2x2 or 3x3 block?

If i find some piece of software that can sum up pixels,  i will do some tests that show whether this idea should be pursued further.

[a1ex]

Dividing by a constant will not change the noise or the sensitivity in this case (it will only trim a few bits which are just noise anyway).

ISO 1600 will be cleaner than 400 pushed. See the noise values in electrons, http://www.clarkvision.com/articles/evaluation-canon-5diii/index.html

Please read this article, then follow this thread, and come back after you will understand the math behind these things.

[swissbear]

So, i can conclude that a to 25% downscaled 1600 ISO image has less noise than a "binned" (taking the sum of every 2x2 pixel block) 400 ISO image which is 2 stops underexposed.

In this case my idea is obsolte and this thread can be closed.

Thank you for the education!