Pixel binning patterns in LiveView

[Levas]

For testing in adtg you can override register '800c', which also can be seen in adtg if 'known registers' is selected.
That's the lineskipping value, default value = 2 (skipping 2 lines) , use value 0 (No lineskipping) for 3x1 mode.
 
This way you will have 3x1 mode, although you can't go beyond normal vertical resolution so it probably stops around 1200 vertical resolution (getting more resolution in adtg needs a lot more registers to change).
So this will be the easiest way for testing things in adtg and see what other registers do to the liveview, since you only have to change register 800c to get 3x1 mode.

You can also see for yourself what a lineskipping value of 1 does (3x2 mode), you will loose one of the bayer colors, not sure which one, red or blue, but you can play with the results and see if it is usable for monochrome.

[theBilalFakhouri]

Actually you are talking about 1x3 Binning "800c" for Vertical but 3x1 Binning there is a register for Horizontal maybe a CMOS register?

[Levas]

Little confusing this talk about 3x1, 1x3 
But you're right, you're working in 5x zoom mode, so lineskipping is already at value 0 and now you have to enable horizontal pixelbinning.
On 6d that is done by CMOS 6 register.
You can use ADTG and see what value CMOS 6 has in full view mode and 5x zoom mode.
Now you can lock the value of CMOS 6 in full view mode in ADTG module and go to 5x zoom mode, this should give 3x1, right 

EDIT, found your spreadsheet with crop_rec registers, you probably need to change CMOS 5 on your camera.

[a1ex]

Let´s start 2019 with some eosm love. The setting is called:
4K squeeze 24fps 
It´s working continuously with 10 bit and build files when set to 2.35:1 through crop_rec submenu that gives files sized to 4000x1704. The files are actually recorded into a squeezed mode(4000x696 or 4000x 568 when 2.35:1 is selected). Thanks to mlv metadata the upscaling is done with the default scale tag so it´s merely a question of opening up the file in Mlv App or acr and the unsqueezing will be done automatically.

To use it enable 'framing' to get a preview in liveview. It is working in regular video mode(NOT x3 zoom mode). Set bitrate to 10bit and ratio to 2.35:1 for continuous recording.
Please report back how it works and feel free to share content recorded.
By the way. Do not miss out on this progress!
https://www.magiclantern.fm/forum/index.php?topic=17695.msg209985#msg209985

Build for download in following post:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg208959#msg208959


MLV test file:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/short_M01-0722.MLV


TODO:
Focus pixel map(dfort is looking into it).
This mode actually also works in mv720p which yield 5x1 squeeze. Will check if it´s actually useful in real life or not.

One minor nitpick: if you zoom in at 1:1 and look at diagonal edges, you will see some jagged lines (as if the columns were somehow reordered).

These artifacts are exactly what I'm talking about in these comments:

Code: [Select]

            /* 3x1 binning (bin every 3 lines, read every column) */
            /* doesn't work well, figure out why */
            case CROP_PRESET_3x1:
                /* ADTG2/4[0x800C] = 2: vertical binning factor = 3 */
                /* ADTG2[0x8806] = 0x6088 on 5D3 (artifacts worse without it) */
                adtg_new[2] = (struct adtg_new) {6, 0x800C, 2};
                if (is_5D3) {
                    /* this register is model-specific */
                    adtg_new[3] = (struct adtg_new) {2, 0x8806, 0x6088};
                }
                break;

I don't remember whether the artifact is present in the 3x1 mode from the bleeding edge 700D crop_rec; maybe Bilal can double-check, if he's still around. The issue is present on 5D3 too, but I wasn't able to fix it there.

Anyway, my suggestion for a 4K stretched mode would be to rotate it at 90 degrees (i.e. take Bilal's 1736x2214 in 1x3 and attempt to squeeze it even more, to get larger heights, up to 3478).

Why? That way, you will not skip any physical pixels on the sensor; all Canon cameras I've looked at appear to be able bin pixels horizontally, every 3 columns. Since this camera is not able to bin in the vertical direction, you'll just read out every line.

With the bleeding edge 700D crop_rec, you should get 1440x3478 at 19.2 FPS:

Code: [Select]

[crop_rec] source: 1736x1160 @ 23.976 fps
[crop_rec] target: 1440x3478 @ 19.194 fps
[crop_rec] ylimit: 1398,3478
[crop_rec] timerA: 528 -> 446
[crop_rec] timerB: 2527 -> 3738
[crop_rec] delta : X-296 Y+2318 [z=1]

From there, you might be able to reduce the horizontal resolution even further; I just don't remember where the hardcoded 1440 limit comes from, and have no way to re-test it atm. For 24 FPS at full height, you need to go down to about 1080 pixels horizontally, but I'm not sure whether the hardware will accept it:

Code: [Select]

[crop_rec] source: 1736x1160 @ 23.976 fps
[crop_rec] target: 1080x3478 @ 23.976 fps
[crop_rec] ylimit: 1864,3488
[crop_rec] timerA: 528 -> 356
[crop_rec] timerB: 2527 -> 3749
[crop_rec] delta : X-656 Y+2318 [z=1]



Expected result: much lower aliasing.

If that works, you can try to go even further; for squeezed 16:9 you would require only 3478 * 9 / 16 / 3 = 652 pixels horizontally. If the hardware accepts it, that should give some half-decent rolling shutter, too.

Happy New Year!

[Danne]

Thank a1ex. Will dig more into this later on. 1x3 is very nice but it comes with the price of much shorter recording times.
Anyway. I started out with squeezing it 5x and yes, would give x5 the height. Also gonna look further into this.
Happy new year!

Edit: squeeze 1x3 mode more. Yes that would be much better probably. On 5D3 this probably would be a game changer...

[a1ex]

1x3 is very nice but it comes with the price of much shorter recording times.

That's true, but keep in mind Bilal is correcting the image by shrinking it vertically (e.g. 1736x2214 -> 1736x738), so his result is going to be  better than a plain 1:1 readout at 1736x738 (it would be an oversampled image). But you could also keep the vertical resolution enlarge the image horizontally to get *interpolated* 5208x2214 (of course, with much lower image quality, compared to a 5208x2214 1:1 readout) or maybe something in-between (likely a better compromise).

My suggestion is to apply this procedure on a 1440x3478 1x3 readout, or 1080x3478 if the hardware accepts it, then crop it to e.g. 652x3478 in mlv_lite and correct it to 1956x3478 in post. That way, you'd get a vertical 3.5K 16:9 (interpolated) requiring about 50-60 MB/s at 24p (i.e. likely continuous with the UHS hack), without skipping any physical pixels on the sensor.

Then, it might be interesting to try some smarter interpolation algorithm (such as the one in cr2hdr, or something tweaked specifically for this kind of readout). I'm sure it won't match a plain 1:1 1956x3478 readout (which is doable at 16.462 FPS), but it might be worth trying.

Now, whether the interpolated 1956x3478 is any better than 1080x1920, that remains to be seen (and I'd say it largely depends on the upscaling algorithm).

Of course, if you only need 2.35:1 or wider at 24p, without rolling shutter constraints, then just stick with Bilal's settings. The 1736x2214 1x3 readout will give a 2.35:1 output between 1736x738 (oversampled) and 5208x2214 (upsampled); the sweet spot is probably somewhere in-between, depending on the upscaling algorithm.

[Danne]

Great description/explanation. Will be an interesting journey to see where this leads.

[Levas]

Already tried the method Alex describes long time ago.
You would think that horizontal stretching gives the best results, since horizontal, pixels are binned. Sounds a bit like anamorphic shooting with a 3x squeeze, right...
But unfortunately the results are really bad, not sure why, probably has to due with how the binning is done.
I did get the best results with lineskipping and stretching the image 3 times.
Still wasn't really happy with the end result..both rescaling with 'cubic' and 'zinc(lanczos3)' give poor results in my opinion, but as Alex said, maybe there is a more proper way to rescale this kind of stuff.

Here's a quick side to side comparison viewed at 100% pixel level, the left is 1624x2070 stretched to -> 4872x2070 and the right is 5120x610 stretched to -> 5120x1830


The 5120x610 stretched to 5120x1830 doesn't look that bad, but the stretching is really visible in stuff that has a certain angle to it, probably everything with straight line at 45 degrees look horrible 

[Levas]

One minor nitpick: if you zoom in at 1:1 and look at diagonal edges, you will see some jagged lines (as if the columns were somehow reordered).

I believe those jagged lines are happening when you’re recording all horizontal pixels in non-zoom mode.
If you’re using the same settings in 5x zoom mode, those jagged lines are gone.

[Danne]

Not by my computer levas but wouldn't x5 mode simply put this mode into 1×1? No squeeze that is?

[Levas]

Not if you’re overriding register 800C (lineskipping)with value 2.

[a1ex]

Here's a quick side to side comparison viewed at 100% pixel level, the left is 1624x2070 stretched to -> 4872x2070 and the right is 5120x610 stretched to -> 5120x1830

If you happen to still have the original files, mind sharing them? A still CR2 from a similar scene could be interesting for simulating this kind of binning, too.

Edit: here's a quick simulation with a resolution chart:

binning-modes.html (18.5 MB)

Edit: received the original files, thanks, will check them later.

[Danne]

hm, 1x3 mode seems to be the better bet. Thanks for sharing visuals. Very revealing.
Just started with squeeze modes so gonna take some time to fully get what and how to do this the best way. @Levas. I can not get these squeeze modes to work with x3 mode or setting 8000 to 5 so feel free to share whatever you get working with less jagged edges.

Published a new build which now includes both 4k 3x1 and 4k 5x1 24 fps. Both settings will work straight out of the box in Mlv App thanks to masc swift follow up which now includes 5.0x height increase. DOne automatically thanks to mlv metadata fully working with crop_rec.c code.
So:
    "4K 3x1 24fps",
    "4K 5x1 24fps",
Runs in regular non crop mode.

As usual downloads here:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg208959#msg208959

[DeafEyeJedi]

Fabulous stuff. Thanks for sharing the resolution charts re: binning-modes @a1ex. That was a fun read through!

Agreed 1x3 modes seems to be the sweet spot @Danne!

[theBilalFakhouri]

Nice stuff happening on the Binning mods! Here is some samples showing all different Binning mods in 700D the Primary photo is 1x1 Full-Resolution LiveView 5208x3478 1x1 and the other are scaled, stretched to to original size of Full-Resolution LiveView then I cropped the image on specific area to show the details difference between them:

1x1 Readout 5208x3478


1x3 Readout 1736x3478 Upscaled to 5208x3478


3x3 Readout 1736x1160

 
3x1 Readout 5208x1160 Upscaled to 5208x3478


5x3 Readout 1736x696  Upscaled to 1736x1160


So 1x3 Binning is the sweet spot between 1x1 and 3x3 readout and 3x1 is the worst here unlike Levas results on 6D? And also 1x3 Binning mod will give more details if we upscaled it otherwise stretching it down (the vertical) will not give that details only will clear the aliasing and moire issues. Of course 5x3 Binning is out of planet Earth.

Original DNGs: Download
Note: Full-Resolution LiveView gives brighter exposure compared to all other mods not sure why (in bleeding edge crop_rec).

I am assuming if we get smart algorithm to re-scale 1x3 readout to the original size, will we get same details as 1x1? or maybe more close to it?

Sorry for off topic, but you are talking here

[Levas]

Danne started a discussion with this post about a 4K preset (squeezed) for the EOS-M

https://www.magiclantern.fm/forum/index.php?topic=9741.msg209998#msg209998

[Levas]

@theBilalFakhouri
I see jagged lines in your 3x1 readout.
I can record in 3x1 with and without these jagged lines.

I use a value of 6 for the 8000 register and a value of 2 for 800C register:
8000=6
800C=2
And I use the 5x zoom mode with the above settings.
 

[Danne]

Levas. Would you mind sharing two movie files. One with squeezed registers in non cropped mode, the other one with same settings but in x5 mode. I try exactly what you describe but output is not the same when in x5 mode.

[Levas]

Here are two example DNG's.
Both are recorded with the same crop_preset, the one with the jagged lines is in non zoom and the normal one is in 5x zoom.
https://drive.google.com/drive/folders/11P6HUC3xNHGNSGLFnV42CctoQjKn9WqY?usp=sharing

Screenshot comparing both DNG's:

[Danne]

That is very strange and also the solution to the issue with jagged lines. Could you share the full set of registers you used here? It should be doable for the eosm as well.
Also would be nice with the actual MLV files.

[Levas]

Ok, now the fun part, I changed the 8000 register in the crop_preset I just used from value 6 to 5.
And after re-adjusting CMOS 7, I have no jagged lines in either non zoom and 5x zoom mode 

So conclusion for now I guess:
The jagged lines appear only in non zoom mode when register 8000 has value 6 (but the jagged lines don't show up in 5x zoom)
When register 8000 has value 5, there aren't jagged lines in any mode, non zoom and 5x zoom.

For crop_presets on the 6d I use:
CMOS 6 for controlling horizontal binning on or off.  (On the 700d and probably most other cameras this is done with CMOS 5)
CMOS 7 for adjusting which vertical part of the sensor is read out and needs finetuning depending on resolution setting, otherwise you get the purple hazy preview/recordings

Register 800c for controlling vertical lineskipping, 0 for no line skipping, 2 for reading every third line.
Register 8000 for controlling jagged lines I guess  (trial and error between value 5 and 6)

All other registers like shutter blanking are done automatiically by a piece of code from Alex which all others are using too and values for resolution and  A and B timers are done as usual.

[Danne]

I still would prefer the full register code, number and all. Could you upload your version of crop_rec.c that you knw will produce non jagged edges?
Don´t you have the exact registry changes for the dng files you uploaded somewhere?


I use following registers:

Code: [Select]

case CROP_PRESET_4K_3x1_EOSM:
                cmos_new[5] = 0x200;            /* vertical (first|last) */
                cmos_new[7] = 0xf20;
                break;

Code: [Select]

     case CROP_PRESET_4K_3x1_EOSM:
adtg_new[0] = (struct adtg_new) {6, 0x800C, 2};
                adtg_new[2] = (struct adtg_new) {6, 0x8000, 5}; /* added to test Levas non jagged edge version out */
break;

Code: [Select]

    switch (reg)
    {
        case 0xC0F06804: return 0x4a6040a;
        case 0xC0F06824: return 0x4ca;
        case 0xC0F06828: return 0x4ca;
        case 0xC0F0682C: return 0x4ca;
        case 0xC0F06830: return 0x4ca;     
        case 0xC0F06010: return 0x45f;
        case 0xC0F06008: return 0x45f050f;
        case 0xC0F0600C: return 0x45f045f;
        case 0xC0F06014: return 0x405;
        case 0xC0F0713c: return 0x320;
case 0xC0F07150: return 0x300;

    }

    return 0;

[Levas]

Uploading MLV's (can take a few minutes), one with jagged lines and one without jagged lines, both recorded with the same crop_preset.
It's done with crop_preset '5K' in the crop_rec.c file.

All in the same folder as previous DNG's:
https://drive.google.com/drive/folders/11P6HUC3xNHGNSGLFnV42CctoQjKn9WqY?usp=sharing

[Levas]

It's from a very old template.

Also changed some stuff to fix shutter times for the right video mode the crop preset is made for:

Line 182:

Code: [Select]

/* helper to allow indexing various properties of Canon's video modes */
static inline int get_video_mode_index()
{
    return

    (crop_preset == CROP_PRESET_HD100) ? 3:
    (crop_preset == CROP_PRESET_HD75) ? 3:
    (crop_preset == CROP_PRESET_HD50) ? 3:
    (crop_preset == CROP_PRESET_2K) ? 1:
    (crop_preset == CROP_PRESET_ULTRA_HD) ? 1:
    (crop_preset == CROP_PRESET_4K) ? 1:
    (crop_preset == CROP_PRESET_5K) ? 1:
    (crop_preset == CROP_PRESET_FullRes) ? 1:
    (crop_preset == CROP_PRESET_1x3) ? 1:
    (crop_preset == CROP_PRESET_HD2) ? 3:
    (crop_preset == CROP_PRESET_3x3_1X) ? 1:
    4;
}

corresponds to this table at line 545:

Code: [Select]

/* from SENSOR_TIMING_TABLE (fps-engio.c) */
/* hardcoded for 6d */
const int default_timerA[] = { 0x27f, 0x2d9, 0x21f, 0x27F, 0x27f };
const int default_timerB[] = { 0x63f, 0x493, 0x61b, 0x31f, 0x334 };
const int default_fps_1k[] = { 25000, 29921, 29978, 50000, 59964 };

As you can see in the upper piece of code, I set preset '5K' to value 1, so now the rest of Alex's code takes the A and B timer values from column 1 in the table shown in the second piece of code.
Counting starts with column 0, so column 1 means A timer = 0x2d9, B timer = 0x493 and FPSx1000 = 29921 (Default values for Canon 6d in 5x zoom mode)

[Danne]

Thanks Levas. Let´s compare: In short from your crop_rec.c:

Code: [Select]

            case CROP_PRESET_5K:
                cmos_new[6] = 0x0;
                cmos_new[7] = 0xc+ delta_head4;
                break;

Code: [Select]

            case CROP_PRESET_5K:
            adtg_new[11] = (struct adtg_new) {6, 0x8000, 6};
            adtg_new[12] = (struct adtg_new) {6, 0x800C, 2};
        break;

Code: [Select]

static inline uint32_t reg_override_5K(uint32_t reg, uint32_t old_val)
{
    switch (reg)
    {
        /* raw resolution (end line/column) */
        case 0xC0F06804: return 0x2800520; // 5472x3604 5x zoom;

        case 0xC0F06824: return 0x535;
        case 0xC0F06828: return 0x535;
        case 0xC0F0682C: return 0x535;
        case 0xC0F06830: return 0x535;
       
        case 0xC0F06010: return 0x537;
        case 0xC0F06008: return 0x5370537;
        case 0xC0F0600C: return 0x5370537;

        case 0xC0F06014: return 0x2fe; //8fb;

        case 0xC0F0713c: return 0x280;
        case 0xC0F07150: return 0x27f;
    }

    return 0;
}

When register 8000 has value 5, there aren't jagged lines in any mode, non zoom and 5x zoom.

If understanding correctly you change registry 8000 to 5 so this changes:

Code: [Select]

            case CROP_PRESET_5K:
            adtg_new[11] = (struct adtg_new) {6, 0x8000, 5};
            adtg_new[12] = (struct adtg_new) {6, 0x800C, 2};
        break;
This will totally mess up my bag over here and put liveview into som messy x3zoom. But with you the magic non jagged edges thingy appears? This works in both non crop mode and in x5 mode giving the squeezed 3x1 footage?


EDIT:
Your Mlv files opened up into Mlv App and height set to 3.0x verifies your findings. Totally amazing. It´s 4k the right way with a very reduced overhead.