12-bit (and 10-bit) RAW video development discussion

[Levas]

Messing around with compiling magic lantern for 6d
and was wondering about the bit modes:

Probably already tried by other users, but did somebody test what happens when you use values of 0x020 or 0x040 ?

+    /* for PACK16_MODE, DSUNPACK_MODE, ADUNPACK_MODE (mask 0x131) */
Add a comment to this line
+    const uint32_t MODE_16BIT       = 0x130;
+    const uint32_t MODE_14BIT       = 0x030;
+    const uint32_t MODE_12BIT       = 0x010;
+    const uint32_t MODE_10BIT       = 0x000;
+    const uint32_t bit_modes[] = { MODE_14BIT, MODE_12BIT, MODE_10BIT };

[dfort]

Code: [Select]

#undef RAW_DEBUG_TYPE   /* this lets you select the raw type (for PREFERRED_RAW_TYPE) from menu */

just change #undef to #define

Ok--so it wasn't that simple but eventually it compiled.

Look carefully at the code

Code: [Select]

static struct menu_entry debug_menus[] = {
    {
        .name = "LV raw type",
        .priv = &lv_raw_type,
        .max = 64,
        .help = "Choose what type of raw stream we should use in LiveView.",
        .help2 = "See lv_af_raw, lv_rshd_raw, lv_set_raw, KindOfCraw...",
    },
};

There it is!



So here's what I'm doing to get there--obviously still a work in progress:

https://bitbucket.org/daniel_fort/magic-lantern/branch/10bit_12bit_experiments#commits

[Danne]

Thanks for this work being done and especially now on the eos M. I did some testing with the raw stream from debug menu. Ran a recording on stream 64-45 but all of them exhibits the same focus pixel pattern. Are we certain streams are actually changed? How many streams are there anyway?
Also want to highlight the crop rec 3x3 720p mode which will yield superior H.264 HDR footage on the eos M and of course even better with the 5D mark III. Here is my workaround to output correct aspect ratio through ffmpeg.
http://www.magiclantern.fm/forum/index.php?topic=15108.msg175407#msg175407

[a1ex]

I might have found a way to use 16px (or even 8px) increments horizontally without crashing Digic 5 cameras. The trick was to make sure W * H (yes, the product) is multiple of 16 bytes. The previous assumption was that W must be multiple of 8 bytes, and since H was always an even value, this appeared to work. Then, I've noticed 1904x1072 works fine, while 1872x1054 crashes. In both cases, (W*14/8) mod 8 is 4, so according to my previous hypothesis, they both should crash. Also, 1872x1072 works, 1872x1052 works, 1870x1052 fails, 1870x1056 works. See the pattern?

Still wondering why it works that way.

Committed on the 12-bit branch. This should make pixel peepers happy

I did some testing with the raw stream from debug menu. [...] Are we certain streams are actually changed?

If you add a variable (with a menu) that is not used anywhere else, guess what will happen

Check commit c3488df.

[Surin Dmitriy]

Is there any build with crop + 10 12 bit + fine resolution? I find crop + 10 12 bit, also find 10 12 bit + fine resolution...

[ilia3101]

Wow! happy to see the higher resolution back on the 5D2, it works at 1872 pixels, but it has a bug that allows it to go up to 1879, and once it has gone up there, it goes down to odd numbers like 1863, and the weird numbers cause recording to stop after 3 seconds and then all kinds of 'corruption' messages appear. But 1872 works!!!!! only tried 14 bit so far

[a1ex]

Good catch, pushed a fix. Try it at all bit depths

[reddeercity]

I might have found a way to use 16px (or even 8px) increments horizontally without crashing Digic 5 cameras.  Also, 1872x1072 works, 1872x1052 works, 1870x1052 fails, 1870x1056 works. See the pattern?
This should make pixel peepers happy

Thanks A1ex  , Cool The Mighty 5D2 Gets it's wings back 
Funny    that's one of the mean reasons I started to get a compiling machine (Ubuntu) up and running , made my day !

[reddeercity]

compiled new update from a1ex on 5d2 , yes I'm back to 1872x1054 , 10bit thou was all corrupted (colored lines )12bit was almost perfect
every other frame had the top 100 lines or so shifted over a bit . This was a quick test used Auto preview for liveview that was good no lockup or freeze there .
Sample Cdng's 12bit 1872x1054  Sample all corrupted Cdng's 10bit 1872x1054 from my dropbox

[Danne]

If you add a variable (with a menu) that is not used anywhere else, guess what will happen 

Not much I guess 
And I need to learn C...

[ilia3101]

5D2 update: whatever changes happened overnight, it has turned 1872 in to 1880, but now there are constant messages all over the screen saying things I don't understand, but it does work, 1880x1056 works, so does 1880x1248, but not 1880x1250, which seems to fit the pattern.

[teatotalTED]

1250 is not divisible by 16? See A1ex post up a bit.^ I dunno thou.

[NedB]

Guys - sorry for (stupid?) question (not a beginner, been using ML since 2012). Downloaded and installed dfort's 10-bit/12-bit build from Nov. 19 for my 550D. But, try as I might, I cannot find where to set the bit depth to 10-bit or 12-bit for testing purposes. Anybody wanna cut me some slack and point it out explicitly for me? Does it require certain modules to be loaded or other settings to be set correctly in order to appear? Thanks, gentlemen!

NedB


Edit: Thank you very much for insanely quick reply, Levas!

[Levas]

10 and 12 bit options are only available in the raw_rec module and not available in the probably more used MLV_rec module.
So enable raw_rec module and unleash the power 

[dfort]

First of all -- thanks to dmilligan for having the patience to work with me on this.

NO MORE FOCUS PIXELS!



Here's what does the trick--only working on the EOSM at this time:

Code: [Select]

#ifdef CONFIG_EOSM
#define PREFERRED_RAW_TYPE 18
#endif

This is a before shot (using PREFERRED_RAW_TYPE 34)



and here is what it looks like with PREFERRED_RAW_TYPE 18



I'm still running some tests and haven't gotten it working with the crop_rec module yet but I wanted to share the news.

[Danne]

Holy shit.

[Lars Steenhoff]

Great work  guys! 

[DeafEyeJedi]

You guys are the real Focus Pocus!

[g3gg0]

updated mlv_rec and mlv_play for bit depth selection.
mlv_play requires raw_twk if you want 10/12 bit playback.

on my 5D3 setup recording still is a bit buggy, every other frame is corrupted.
anyone can confirm?

[ilia3101]

5D2: I can confirm every other frame is corrupted in 10 and 12 bit mlv_rec, top half of the image in 10 bit and a small bar of noise at the top for 12 bit. mlv_play + raw_twk does work with 10 and 12 bit, but everything is very green and crushed, probably black level.

[dfort]

...every other frame is corrupted...

I don't have a 5D3 to run the test I just did with the EOSM but I saw lots of PREFERRED_RAW_TYPE settings that had the alternating frame corruption issue you are describing. I went through all of the 64 possible settings until I found the one that doesn't show focus pixels. Maybe it is time to recheck the PREFERRED_RAW_TYPE setting on other cameras too?

Checked out your latest changes to mlv_dump but I'm getting a "Segmentation fault: 11" when turning on "--cs2x2". It looks like this isn't an issue specific to the 10bit_12bit branch.

http://www.magiclantern.fm/forum/index.php?topic=7122.msg167507#msg167507

[reddeercity]

@dfort great stuff the only thing missing is MV1080 , I think I'll keep my eye on a good used EOSM 
@g3gg0 same as @Ilia3101 on my 5D2 even tried different Liveview previews setting even the ones that freeze up Liveview
Also tried  29.97p fps in canon menu 1856x1044 raw mlv. Still the same as 24p

Corrupted Frame 10bit 1856x1044 23.976p MLV
 
M25-2137_000545 by RedDeerCityTV, on Flickr

Good Frame 10bit 1856x1044 23.976p MLV

M25-2137_000546 by RedDeerCityTV, on Flickr

Corrupted Frame 12bit 1856x1044 23.976p MLV

M25-2139_000000 by RedDeerCityTV, on Flickr

Good Frame 12bit 1856x1044 23.976p MLV

M25-2139_000001 by RedDeerCityTV, on Flickr

[dfort]

@dfort great stuff the only thing missing is MV1080

The really great stuff for the EOSM is happening with @rbrune here:

http://www.magiclantern.fm/forum/index.php?topic=16608.msg175360#msg175360

A camera you can pick up for $150-$200 USD that can shoot 10-bit raw video--and you can adapt just about any lens from a C-mount to a PL-mount?

[Levas]

Did a quick test with the updated mlv_rec from g3gg0 on the 6d and got ZERO corrupted frames with 10bit.
Recorded 3 MLV files in 10bit and all frames are fine, even the first ones.

I used mlv_dump.osx (build from may 2015) and all frames look fine in Rawtherapee.

[Teamsleepkid]

no focus pixels!