Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 627784 times)

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1685
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #500 on: November 24, 2016, 04:21:03 PM »
Messing around with compiling magic lantern for 6d
and was wondering about the bit modes:

Probably already tried by other users, but did somebody test what happens when you use values of 0x020 or 0x040 ?

+    /* for PACK16_MODE, DSUNPACK_MODE, ADUNPACK_MODE (mask 0x131) */
Add a comment to this line
+    const uint32_t MODE_16BIT       = 0x130;
+    const uint32_t MODE_14BIT       = 0x030;
+    const uint32_t MODE_12BIT       = 0x010;
+    const uint32_t MODE_10BIT       = 0x000;
+    const uint32_t bit_modes[] = { MODE_14BIT, MODE_12BIT, MODE_10BIT };

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #501 on: November 24, 2016, 05:25:44 PM »
Code: [Select]
#undef RAW_DEBUG_TYPE   /* this lets you select the raw type (for PREFERRED_RAW_TYPE) from menu */
Quote
just change #undef to #define

Ok--so it wasn't that simple but eventually it compiled.

Look carefully at the code

Code: [Select]
static struct menu_entry debug_menus[] = {
    {
        .name = "LV raw type",
        .priv = &lv_raw_type,
        .max = 64,
        .help = "Choose what type of raw stream we should use in LiveView.",
        .help2 = "See lv_af_raw, lv_rshd_raw, lv_set_raw, KindOfCraw...",
    },
};

There it is!



So here's what I'm doing to get there--obviously still a work in progress:

https://bitbucket.org/daniel_fort/magic-lantern/branch/10bit_12bit_experiments#commits
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #502 on: November 24, 2016, 08:22:59 PM »
Thanks for this work being done and especially now on the eos M. I did some testing with the raw stream from debug menu. Ran a recording on stream 64-45 but all of them exhibits the same focus pixel pattern. Are we certain streams are actually changed? How many streams are there anyway?
Also want to highlight the crop rec 3x3 720p mode which will yield superior H.264 HDR footage on the eos M and of course even better with the 5D mark III. Here is my workaround to output correct aspect ratio through ffmpeg.
http://www.magiclantern.fm/forum/index.php?topic=15108.msg175407#msg175407

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #503 on: November 24, 2016, 09:14:45 PM »
I might have found a way to use 16px (or even 8px) increments horizontally without crashing Digic 5 cameras. The trick was to make sure W * H (yes, the product) is multiple of 16 bytes. The previous assumption was that W must be multiple of 8 bytes, and since H was always an even value, this appeared to work. Then, I've noticed 1904x1072 works fine, while 1872x1054 crashes. In both cases, (W*14/8) mod 8 is 4, so according to my previous hypothesis, they both should crash. Also, 1872x1072 works, 1872x1052 works, 1870x1052 fails, 1870x1056 works. See the pattern?

Still wondering why it works that way.

Committed on the 12-bit branch. This should make pixel peepers happy ;)

I did some testing with the raw stream from debug menu. [...] Are we certain streams are actually changed?

If you add a variable (with a menu) that is not used anywhere else, guess what will happen ;)

Check commit c3488df.

Surin Dmitriy

  • New to the forum
  • *
  • Posts: 45
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #504 on: November 25, 2016, 12:47:18 AM »
Is there any build with crop + 10 12 bit + fine resolution? I find crop + 10 12 bit, also find 10 12 bit + fine resolution...
5D3

ilia3101

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 940
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #505 on: November 25, 2016, 01:08:44 AM »
Wow! happy to see the higher resolution back on the 5D2, it works at 1872 pixels, but it has a bug that allows it to go up to 1879, and once it has gone up there, it goes down to odd numbers like 1863, and the weird numbers cause recording to stop after 3 seconds and then all kinds of 'corruption' messages appear. But 1872 works!!!!! only tried 14 bit so far :D

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #506 on: November 25, 2016, 01:21:26 AM »
Good catch, pushed a fix. Try it at all bit depths ;)

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2283
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #507 on: November 25, 2016, 02:28:45 AM »
I might have found a way to use 16px (or even 8px) increments horizontally without crashing Digic 5 cameras.  Also, 1872x1072 works, 1872x1052 works, 1870x1052 fails, 1870x1056 works. See the pattern?
This should make pixel peepers happy ;)
Thanks A1ex  , Cool The Mighty 5D2 Gets it's wings back  :D
Funny  :P  that's one of the mean reasons I started to get a compiling machine (Ubuntu) up and running , made my day !

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2283
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #508 on: November 25, 2016, 09:09:52 AM »
compiled new update from a1ex on 5d2 , yes I'm back to 1872x1054 , 10bit thou was all corrupted (colored lines )12bit was almost perfect
every other frame had the top 100 lines or so shifted over a bit . This was a quick test used Auto preview for liveview that was good no lockup or freeze there .
Sample Cdng's 12bit 1872x1054  Sample all corrupted Cdng's 10bit 1872x1054 from my dropbox

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #509 on: November 25, 2016, 09:44:23 AM »
Quote
If you add a variable (with a menu) that is not used anywhere else, guess what will happen  :P
Not much I guess  :P
And I need to learn C...

ilia3101

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 940
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #510 on: November 25, 2016, 01:24:39 PM »
5D2 update: whatever changes happened overnight, it has turned 1872 in to 1880, but now there are constant messages all over the screen saying things I don't understand, but it does work, 1880x1056 works, so does 1880x1248, but not 1880x1250, which seems to fit the pattern.

teatotalTED

  • New to the forum
  • *
  • Posts: 35
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #511 on: November 25, 2016, 01:39:18 PM »
1250 is not divisible by 16? See A1ex post up a bit.^ I dunno thou.

NedB

  • Freshman
  • **
  • Posts: 85
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #512 on: November 25, 2016, 09:16:49 PM »
Guys - sorry for (stupid?) question (not a beginner, been using ML since 2012). Downloaded and installed dfort's 10-bit/12-bit build from Nov. 19 for my 550D. But, try as I might, I cannot find where to set the bit depth to 10-bit or 12-bit for testing purposes. Anybody wanna cut me some slack and point it out explicitly for me? Does it require certain modules to be loaded or other settings to be set correctly in order to appear? Thanks, gentlemen!

NedB


Edit: Thank you very much for insanely quick reply, Levas!
550D - Kit Lens | EF 50mm f/1.8 | Zacuto Z-Finder Pro 2.5x | SanDisk ExtremePro 95mb/s | Tascam DR-100MkII

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1685
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #513 on: November 25, 2016, 09:20:53 PM »
10 and 12 bit options are only available in the raw_rec module and not available in the probably more used MLV_rec module.
So enable raw_rec module and unleash the power  ;D

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #514 on: November 25, 2016, 11:05:15 PM »
First of all -- thanks to dmilligan for having the patience to work with me on this.

NO MORE FOCUS PIXELS!



Here's what does the trick--only working on the EOSM at this time:

Code: [Select]
#ifdef CONFIG_EOSM
#define PREFERRED_RAW_TYPE 18
#endif

This is a before shot (using PREFERRED_RAW_TYPE 34)



and here is what it looks like with PREFERRED_RAW_TYPE 18



I'm still running some tests and haven't gotten it working with the crop_rec module yet but I wanted to share the news.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #515 on: November 25, 2016, 11:33:31 PM »
Holy shit.

Lars Steenhoff

  • Senior
  • ****
  • Posts: 473
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #516 on: November 26, 2016, 12:15:55 AM »
Great work  guys!  :)

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #517 on: November 26, 2016, 12:23:24 AM »
You guys are the real Focus Pocus!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3166
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #518 on: November 26, 2016, 02:43:36 AM »
updated mlv_rec and mlv_play for bit depth selection.
mlv_play requires raw_twk if you want 10/12 bit playback.

on my 5D3 setup recording still is a bit buggy, every other frame is corrupted.
anyone can confirm?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

ilia3101

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 940
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #519 on: November 26, 2016, 03:42:55 AM »
5D2: I can confirm every other frame is corrupted in 10 and 12 bit mlv_rec, top half of the image in 10 bit and a small bar of noise at the top for 12 bit. mlv_play + raw_twk does work with 10 and 12 bit, but everything is very green and crushed, probably black level.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #520 on: November 26, 2016, 03:49:36 AM »
...every other frame is corrupted...

I don't have a 5D3 to run the test I just did with the EOSM but I saw lots of PREFERRED_RAW_TYPE settings that had the alternating frame corruption issue you are describing. I went through all of the 64 possible settings until I found the one that doesn't show focus pixels. Maybe it is time to recheck the PREFERRED_RAW_TYPE setting on other cameras too?

Checked out your latest changes to mlv_dump but I'm getting a "Segmentation fault: 11" when turning on "--cs2x2". It looks like this isn't an issue specific to the 10bit_12bit branch.

http://www.magiclantern.fm/forum/index.php?topic=7122.msg167507#msg167507
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2283
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #521 on: November 26, 2016, 05:54:40 AM »
@dfort great stuff the only thing missing is MV1080 , I think I'll keep my eye on a good used EOSM  :)
@g3gg0 same as @Ilia3101 on my 5D2 even tried different Liveview previews setting even the ones that freeze up Liveview
Also tried  29.97p fps in canon menu 1856x1044 raw mlv. Still the same as 24p

Corrupted Frame 10bit 1856x1044 23.976p MLV
 
M25-2137_000545
by RedDeerCityTV, on Flickr

Good Frame 10bit 1856x1044 23.976p MLV

M25-2137_000546
by RedDeerCityTV, on Flickr

Corrupted Frame 12bit 1856x1044 23.976p MLV

M25-2139_000000
by RedDeerCityTV, on Flickr

Good Frame 12bit 1856x1044 23.976p MLV

M25-2139_000001
by RedDeerCityTV, on Flickr

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #522 on: November 26, 2016, 07:43:54 AM »
@dfort great stuff the only thing missing is MV1080

The really great stuff for the EOSM is happening with @rbrune here:

http://www.magiclantern.fm/forum/index.php?topic=16608.msg175360#msg175360

A camera you can pick up for $150-$200 USD that can shoot 10-bit raw video--and you can adapt just about any lens from a C-mount to a PL-mount?
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1685
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #523 on: November 26, 2016, 08:51:53 AM »
Did a quick test with the updated mlv_rec from g3gg0 on the 6d and got ZERO corrupted frames with 10bit.
Recorded 3 MLV files in 10bit and all frames are fine, even the first ones.

I used mlv_dump.osx (build from may 2015) and all frames look fine in Rawtherapee.




Teamsleepkid

  • Member
  • ***
  • Posts: 244
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #524 on: November 26, 2016, 08:53:34 AM »
no focus pixels!
EOS M