## Pixel binning patterns in LiveView

Started by a1ex, January 21, 2016, 08:52:22 AM

0 Members and 1 Guest are viewing this topic.

#### bouncyball

Quote from: Levas on January 13, 2019, 11:36:37 AM
If somehow the raw file can be reconstructed to a new raw file according to the binning pattern, then you have a good starting point:
I really dig the unbining idea. I need to understand more the bining theory and need some visual examples.

#### Levas

As far as I know, the Canon's do horizontal binning the following way:
You have a standard Bayer pattern.
And on a horizontal line, each chunk of 6 pixels is readout and all 3 red pixels are averaged to 1 value and all green pixels are averaged to one value etc. etc.

To unbin this binned image I was thinking to do something like this:
First you're gonna stretch the raw file by a factor 3 in horizontal direction, so copy each red/green pair 3 times, and in the next line, copy each green/blue pair three times.
So for example in a red/green line, you end up with each 6 pixels having 3 times the same green value and 3 times the same red value.
In each 6 pixel block, you can keep the values of the pixels in the middle, So pixel 3 and 4 in a chunk of 6 are already good.
The green pixel on the left side of a chunk of 6 has to averaged with the green neighbour pixel in the chunk of 6 pixels that are on the left side.
The green pixel at the right end of a chunk of 6 pixels, has to be averaged with a neighbouring green pixel on the right side of the chunk of 6 pixels.
Looks like this:

Not sure if this is clear enough ?
Also not sure if this is gonna give better results
But if anyone can do this, I'd love to find out how the end results look.

#### bouncyball

I think it is a bit wrong. Look at the 1st post of this thread. a1ex showed the 5d3 and all other (lower) camera bining methods. The averaging is done not exactly by 6 raw line pixels but a next pixel for every color (R,G+1 for first line and G,B+1 for second line). 5D3 is a complete another story with 3 by 3 binning.

I thought about what you've showed above and the algo you proposed, and according to the initial and the end values and their differences this is very inaccurate approach. Anyway after averaging the information is lost forever and it is impossible to recover it except only very basic cases.

Well... I'm gonna think about it more, but... maybe a1ex could suggest, does it worth a try or not.

regards
bb

#### Levas

You're right, it's not done in chunks of 6 pixels, there is a shift for the other color channel.

The suggested algo does indeed not approach the original raw data, but that's never possible, like you said, the data is lost forever.

I expect that such an algorythm or any variation on it, can give better results in the pixelated aliasing that happens on round curves or straight lines.

And indeed curious if Alex has any ideas, did a great job with dual-iso.

#### reddeercity

Found a different pinning mode on the 5d2 in 3x crop_mode (5x zoom)
`ADTG12[100c]0x01`
This give a vertical squeeze similar to 3x5 in 1:1FHD(720p mode on other cams)

Tried to correct the W/B & colors , bad black levels plus other issue -- all blue channel it seems
In 3x crop_mode 5632x514

M11-1730_frame_1_5632x514.png

Un-squeezed @ 167%

M11-1730_frame_1_5632x856_167precent_vertical.png

the original dng's -- 5632x514

M11-1730_frame_1_5632x514.dng

Exported DNG from MLV App with 167% vertical stretch

M11-1730_frame_1_5632x857_167precent_vertical.dng

#### Levas

Looks indeed like there is missing a color channel.
I think you did a lineskipping factor of 1 in 5x zoom mode.
That would cause the missing color channel, you're only reading the Blue/Green pixel lines.
If that's the case, the unsqueezing factor should be 2 instead 1.67

#### reddeercity

Yea , your right -- I just check it that even better , now if I can get all the channel working .

5632x1028 from 5632x514 with 200% vertical stretch

M11-1730_frame_1_5632x1028_200precent_vertical.png

#### Teamsleepkid

can we use it for black and white if we don't get the other color channel? just sayin..
EOS M

#### ilia3101

Quote from: reddeercity on January 16, 2019, 03:26:31 AM
Yea , your right -- I just check it that even better , now if I can get all the channel working .

5632x1028 from 5632x514 with 200% vertical stretch

M11-1730_frame_1_5632x1028_200precent_vertical.png

No way to get 3 channels with 2x stretch I think, unless you make the camera do a really complex binning pattern.

#### Levas

Ilia is right, you will never get 3 channels by using a lineskipping value of 1.
You always end up with 2 color channels, you are reading only the green/red pixel lines, or only the green/blue pixel lines, because of the lineskipping value of 1.

This post from Alex is still interesting, I'm not sure if it is possible.
But it would be cool to explore if their is some sort of way to create decent images out of 2 color channels instead of 3.
Quote from: a1ex on December 02, 2017, 11:59:48 AM

Food for thought: can we recover blue from red?

#### a1ex

An interesting pixel binning mode found by mistake by Danne & Masc on the EOS M.

Quote from: masc on February 17, 2019, 06:20:57 PM

Test images:
Quote from: masc on February 17, 2019, 10:01:05 PM
Okay. Done. Here you are: https://www.dropbox.com/s/70ig9z535kixmk0/binningtest3.zip?dl=0

Quote from: a1ex on February 18, 2019, 08:50:45 PM
Pretty interesting binning pattern. It is, indeed, 3x3 readout with line/column skipping, i.e. using only one pixel out of 9. However, it has an additional property - every two columns are adjacent. This results in... even more moirĂ©!

Details: extreme-moire.html

Also confirmed the following binning pattern for the regular 1080p 3x3 mode on EOS M, likely used in most other Canons, with the same test code, on these test images. No write-up for this one yet.

Quote from: a1ex on January 21, 2016, 08:52:22 AM

#### Levas

Ok found something
Danne told me how he got the ugly moire, (3x3 with horizontal pixel skipping and vertical line skipping).
And I can get it on the 6d too.
On 6d it has to do with 1x zoom and 5x zoom mode.
If you are in 5x zoom mode, no horizontal pixel binning is done.
In 5x zoom, when you alter 2 CMOS registers and the 800c lineskipping register and the 8000 register, to transform 5x zoom to normal full sensor view...you get the ugly moire.

So got me thinking, what crop_presets I made for 5x zoom mode which uses full sensor view...Oh no, the 1x3 readout mode
Remember this post:
https://www.magiclantern.fm/forum/index.php?topic=16516.msg210005#msg210005

I kept thinking about it, why does unstretched horizontal footage looks that bad, it's made of pixelbinned data where is al that moire coming from...
Well now we know, the example you see there is made in 5x zoom mode, so no horizontal pixelbinning was done there
Did some testing with a new 1x3 mode in 1 x zoom, and stretching that 3 times horizontal looks way better
Getting late now, but maybe I can do some outdoor tests tomorrow, with some nice trees with breaches and stuff.

#### theBilalFakhouri

That's great @Levas, Maybe there is a register to control Binning Pixels? One of the ADTG registers? or maybe one of the CMOS registers that we don't know what's doing like CMOS 8? of course this only a guess, it will be cool if you or Danne start overriding and finding which registers control the Binning process (mv1080 vs x5 registers values), I don't have my cam to do that

#### Levas

Better late then never, here is some 1x3 footage from the 6d, stretched 3x horizontal to get 5K.
It's not bad, but once you compare it to a photo taken from the same scene, you see that this movie example is far from 5K
It's still a lot better then what I got before, now I'm using a setting that actually bins horizontally.
Here's the folder with the 5K movie clip and 3 JPG, one photo, one frame from the movie clip(1x3 stretched 3x horizontal) and one frame from a 3x3 movie blown up 300%.

#### Danne

Horizontal? Care yo share source?

#### Levas

Sure, uploaded the source in the same folder:

First I had better results with stretching vertical, but that was because I was doing 1x3 without pixelbinning...didn't know that
So with horizontal stretch I got real bad results, lots of moire.
Now I'm doing 1x3 with pixelbinning and the results are better.

#### Levas

Quote from: theBilalFakhouri on February 19, 2019, 11:23:01 PM
That's great @Levas, Maybe there is a register to control Binning Pixels? One of the ADTG registers? or maybe one of the CMOS registers that we don't know what's doing like CMOS 8? of course this only a guess, it will be cool if you or Danne start overriding and finding which registers control the Binning process (mv1080 vs x5 registers values), I don't have my cam to do that

I guess that there is a ADTG register that controls the pixelbinning, probably one of these, haven't done any testing, but these look interesting as differences after pressing the magnifying button to go from 1 x zoom to 5 x zoom mode

#### Danne

Cool. Will check out during next week.

#### Levas

I think I found the registers for binning.

When I'm in 5x zoom mode, and I lock the values of ADTG2 [8183] and ADTG2 [8184] and go back to 1x zoom. I get really bad moire.

Value for 8183 and 8184 in 5x zoom are '0'
Normally in 1x zoom they are:
8183 = 0x35 (53 in decimals)
8184 = 0x3b (59 in decimals)

Not sure what those values mean in 1x zoom...but I'm curious what standard values for 8183 and 8184 on the 5d3 are, which does vertical binning.
Still have a little hope for vertical binning as on the 5d3 is just a register setting

#### Levas

Comparison, at 200%.
Above, normal horizontal pixelbinning.
Below, no horizontal pixelbinning (Register 8183 and 8184 set to value 0).

#### Danne

Wow. Good catch dude. think you nailed it:
Tested on my eosm(mcm rewired mode):
`                adtg_new[18] = (struct adtg_new) {6, 0x8183, 0x21};                adtg_new[19] = (struct adtg_new) {6, 0x8184, 0x7b};`

Zeroed:
`                adtg_new[18] = (struct adtg_new) {6, 0x8183, 0x0};                adtg_new[19] = (struct adtg_new) {6, 0x8184, 0x0};`

Keep looking maybe we can even eliminate focus pixels?

#### theBilalFakhouri

That's Great! Something is happening here, I hope we are not far away from vertical pixel binning but maybe also it is not that simple , we will see

#### a1ex

Very nice find!

Quote from: Levas on February 24, 2019, 01:18:19 PM
... I'm curious what standard values for 8183 and 8184 on the 5d3 are

`# same in 25/30/50/60p; same as EOS M700D/1080p24.log:57631: 1.216.977         Evf:000179b0:91:01: [REG] ADTG:[0x81830021]700D/1080p24.log:57638: 1.217.004         Evf:000179b0:91:01: [REG] ADTG:[0x8184007b]# same in 1080p crop modes700D/x5.log:1364: 4.437.292         Evf:000179b0:91:01: [REG] ADTG:[0x81830000]700D/x5.log:1371: 4.437.319         Evf:000179b0:91:01: [REG] ADTG:[0x81840000]# same in 25/30/50/60p; not set in x55D3/1080p24.log:17153:  1.178535         Evf:00011754:91:01: [REG] ADTG:[0x81830038]5D3/1080p24.log:17167:  1.178565         Evf:00011754:91:01: [REG] ADTG:[0x8184002d]# same in 25/30/50/60p and 640x480 crop 50/60p; not set in x5; same as 600D60D/1080p25.LOG:14717:  1.987841         Evf:ff2c98b4:91:01: [REG] ADTG:[0x81830025]60D/1080p25.LOG:14730:  1.987874         Evf:ff2c98b4:91:01: [REG] ADTG:[0x81840021]# large value?1100D/720p25-full.LOG:1077: 0.479142         Evf:ff2dbc54:91:01: [REG] ADTG:[0x818301c7]1100D/720p25-full.LOG:1078: 0.479177         Evf:ff2dbc54:91:01: [REG] ADTG:[0x818401c0]# same in 25/30p; not set in x55D2/1080p24.log:27234:  0.533919  LiveViewMg:ffa35de8:91:01: [REG] ADTG:[0x1183003c]5D2/1080p24.log:27261:  0.534003  LiveViewMg:ffa35de8:91:01: [REG] ADTG:[0x1184003a]# close to 1100D?500D-dm-0009.log:14833:5C89C> LiveViewMg:ff22f9b4:91:01: [REG] ADTG:[0x118301c6]500D-dm-0009.log:14836:5C933> LiveViewMg:ff22f9b4:91:01: [REG] ADTG:[0x118401c0]`

#### DeafEyeJedi

Thumbs up @Levas!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

#### Levas

Thanks for the numbers Alex.