Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 627786 times)

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1685
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #525 on: November 26, 2016, 08:53:41 AM »
Forget to mention.
The canon liveview freezes during recording.
The ML grayscale doesn't freeze during recording, but doesn't look like it can handle 10bit right now.

Is this normal, does canon live view freezes with the use of 10bit on all camera models ?

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #526 on: November 26, 2016, 08:56:01 AM »
Gonna shoot some files on my 5D mark III soon.
Quote
Checked out your latest changes to mlv_dump but I'm getting a "Segmentation fault: 11" when turning on "--cs2x2". It looks like this isn't an issue specific to the 10bit_12bit branch.
This issue is solved in unified so it,s in the 10-bit/12-bit code?

http://www.magiclantern.fm/forum/index.php?topic=7122.msg174592#msg174592

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #527 on: November 26, 2016, 09:30:10 AM »
...I used mlv_dump.osx (build from may 2015)...

Really?

...and all frames look fine in Rawtherapee.

Care to share screenshots?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #528 on: November 26, 2016, 09:34:58 AM »
This issue is solved in unified so it,s in the 10-bit/12-bit code?

Yes--so it looks like somehow this commit didn't make it into the 10bit_12bit branch. Just applied it and --cs2x2 works fine.

This is needed to get rid of the hot pixels that sometimes appear in high contrast boundaries. Not sure if those are only in cameras that have (had?) the focus pixel issue.

5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #529 on: November 26, 2016, 10:17:32 AM »
g3gg0, you are king. And major thanks to Greg for work on this via DARK code? http://www.magiclantern.fm/forum/index.php?topic=13408.msg172108#msg172108. Who did I forget?
Just added 10/12-bit branch into unified and added the raw_twk.mo from here http://www.magiclantern.fm/forum/index.php?topic=13163.msg174991#msg174991
and playback is more or less realtime with all bits 10,12,14. it even works on the eos M!, just tested briefly.

Quote
on my 5D3 setup recording still is a bit buggy, every other frame is corrupted.
anyone can confirm?
I get corrupted files(white level off I think) on my 5D mark III every now and then in both 10 and 12 bit. Also from time to time a text with "raw detect error"(I think it says) on the left. Canon eos M is the more stable participant right now it seems(thanks dmilligan, a1ex, dfort and others)
Shortened examples of corrupted files
test_1
https://drive.google.com/file/d/0B4tCJMlOYfircGtCRlNSWFFGU2M/view?usp=sharing
test_2
https://drive.google.com/file/d/0B4tCJMlOYfirckxRMmJqMm1EaFE/view?usp=sharing

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3166
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #530 on: November 26, 2016, 12:30:22 PM »
@Danne:
happy to hear that :)

yeah the black/white level thing is maybe still an issue. tried to fix it by scaling white level properly when upconverting to 16 bit in mlv_play.
but maybe there is still smth twisted.

does mlv_lite perform better with recording, i.e. no corrupted frames?
didnt test t.b.h.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #531 on: November 26, 2016, 12:44:04 PM »
Only tested with mlv lite. Is 10 bit working with mlv_rec?? Not home to test atm.
One quirk with eos m is that play, pause, delete buttons are suppressed when running playback through raw_twk.mo. Buttons works great with 5D mark III.

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3166
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #532 on: November 26, 2016, 01:25:24 PM »
i updated it also, so yes.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #533 on: November 26, 2016, 03:26:32 PM »
Will check later. Most greatful.
If anybody wants to try here are builds for 5D mark III and eos m with latest stuff. Raw_twk.mo included.(removed for now, causing corrupted black level)

Briefly tested. If anything breaks you get to keep all pieces.

*Black level issues fixed by g3gg0. New builds up
https://bitbucket.org/Dannephoto/cr2hdr/downloads

baccinoluigi

  • New to the forum
  • *
  • Posts: 42
  • 5DMK3/ML113
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #534 on: November 26, 2016, 04:48:00 PM »
Quote
Will check later. Most greatful.
If anybody wants to try here are builds for 5D mark III and eos m with latest stuff. Raw_twk.mo included.
Briefly tested. If anything breaks you get to keep all pieces.
https://bitbucket.org/Dannephoto/cr2hdr/downloads

Hi i tested the last build on 5d Mark III sometimes it work correctly
but often the video is completely black
i send you a dng : https://www.dropbox.com/s/odn6xxrubhup6hz/M28-1717_000000.dng?dl=0

 
 

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #535 on: November 26, 2016, 05:12:15 PM »
@baccinoluigi
Any difference in stability between mlv_rec and mlv_lite?

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1685
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #536 on: November 26, 2016, 05:15:16 PM »
Had some time to do some more mlv_rec testing on 6d.

Tested 10bit, 12bit and 14bit.
10bit works, for all frames and both old mlv_dump and newest mlv_dump works.
12bit works, only old mlv_dump gives purple vertical lines, newest mlv_dump detects false cold pixels and gives frames with too much specles because of the false cold pixel detection.
14bit, still works ;D

I'm using rawtherapee to view the files.

Uploading some dng files right now (few minutes uploading remaining)
http://drive.google.com/open?id=0B1BxGc3dfMDabVFDeG11QjBmWlk


baccinoluigi

  • New to the forum
  • *
  • Posts: 42
  • 5DMK3/ML113
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #537 on: November 26, 2016, 05:23:15 PM »
@Danne
 
Tested again but mlv_rec and mlv_lite have the same issue

14 bit is more stable

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1685
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #538 on: November 26, 2016, 07:30:17 PM »
Still wondering, how is liveview during recording on other cams ?
On the 6d, canon liveview freezes during 10bit recording, Grayscale preview is messed up during 10bit recording.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #539 on: November 26, 2016, 09:16:50 PM »
@baccinoluigi:

Workaround:
Code: [Select]
exiftool *.dng -BlackLevel=128

Is the issue specific to a particular build, or all of them are buggy?

@Levas:

Your question was already answered by dmilligan earlier in this thread.

robertgl

  • New to the forum
  • *
  • Posts: 40
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #540 on: November 26, 2016, 09:23:06 PM »
Hello 50D user here
1. Magenta frames. It shows up on 10,12 and 14 when using the available modified fw. I haven't tried to change the black levels with mlv_dump, assuming that's the way to do it

2. The first frame keeps showing up in the even frames. So it's like 0,1,0,3,0,5,0,7,0,9,

3. at 1856x*** I get full frames interspersed with that continuing first frame showing up in even frames, while if I go to 1920x**** I get torn frames  with most of the frame being the first frame repeating, while a small sliver is the properly sequenced frame

4. Grayscale preview seems to run fine when recording

5. fingers crossed that the bugs can be ironed out because recordings look amazing and have a smaller files size  8)

Am exporting with : mlv_dump --dng --no-fixcp --no-stripes
debayering in rawtherapee

eNnvi

  • New to the forum
  • *
  • Posts: 48
  • 700D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #541 on: November 26, 2016, 09:29:56 PM »
testing on 700D, builded from official hudson repository, raw_video_10bit_12bit branch about 10 minutes ago.

Found a "BUG", setting full HD res, no crop mode, going over the limit it uses max res possible on camera (1736x976) not divisible by 16 so crash happens after a few secs recording, this in both mlv_rec and raw_rec modules.

here logs after that crash
https://drive.google.com/open?id=0BxMUvr5E8ANaMDZUbkJnTHFya28  LOG000.log
https://drive.google.com/open?id=0BxMUvr5E8ANaci12Y2xUSU1MREE CRASH00.log


baccinoluigi

  • New to the forum
  • *
  • Posts: 42
  • 5DMK3/ML113
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #542 on: November 26, 2016, 09:44:04 PM »
@a1ex
Quote
Is the issue specific to a particular build, or all of them are buggy?

The issue it's just in the build downloaded here : https://bitbucket.org/Dannephoto/cr2hdr/downloads
whith raw_twk.mo for 5dMkIII

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #543 on: November 26, 2016, 10:04:22 PM »
Difference between raw_rec (MLV Lite) and mlv_rec on EOSM.

raw_rec is fine at all bit depths.

mlv_rec is fine at 14 bit, shows some corruption on every other frame at 12 bit and the corruption is worse at 10 bit.

12bit:


10bit:


No focus pixels though.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #544 on: November 26, 2016, 10:30:18 PM »
Tested some more around raw_twk on my eos m. Now I,m not sure if raw_twk is to be implemented on all cams. Anyway. Since it works really fast I think it,s a badass feature.
When recording mlv_rec and sound on an error message is shown Threads failed to start. When raw_twk.mo is off recording works. Is sound even working on eos m to begin with? I am not too familiar with this cam.
Mlv_play module has no start, stop, pause buttons when run through raw_twk. Can,t scroll around to the next video etc.
Like dfort points out mlv lite is producing good files while mlv_rec inherits corruption.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7141
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #545 on: November 26, 2016, 10:54:59 PM »
It seems raw_twk is causing mlv_lite(raw_rec) to produce the files with bad black levels on the 5D mark III. It happens once you start mlv_play and after this the next file usually turns out bad. When raw_twk is off everything seems fine when running mlv_lite. I will change my shared builds to builds without the raw_twk.mo for now.

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1685
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #546 on: November 26, 2016, 10:58:48 PM »
@alex ah I missed that one from Dmilligan
@Dmilligan

Can you help me here, I can see the following code in edmac_memcpy.c (see italic below)
I wanna fix live view during recording for 6d, how do I find a free edmac channel with free edmac channels ?


/** this method bypasses Canon's lv_save_raw and slurps the raw data directly from connection #0 */
#ifdef CONFIG_EDMAC_RAW_SLURP

/* for other cameras, find a free channel with find_free_edmac_channels  */
#ifdef CONFIG_5D3
uint32_t raw_write_chan = 4;
#endif

#ifdef CONFIG_60D
uint32_t raw_write_chan = 1;
#endif

#ifdef CONFIG_600D
// write-index 1, 4, 6, 8, 10, 11, 13
uint32_t raw_write_chan = 4;
#endif

#ifdef CONFIG_EOSM
uint32_t raw_write_chan = 0x12;
#endif

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1685
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #547 on: November 26, 2016, 11:03:51 PM »
I can see with show edmac in the Debug menu and scroll trough the channels, that channel 7, 8 and 13 are empty...
Is it that easy ?

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1685
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #548 on: November 26, 2016, 11:16:41 PM »
Well...I can confirm that adding this line


#ifdef CONFIG_6D
uint32_t raw_write_chan = 7;
#endif



And compiling mlv_rec doesn't work... :-\

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3166
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #549 on: November 26, 2016, 11:17:58 PM »
@Danne:
black level issue should be fixed now
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!