12-bit (and 10-bit) RAW video development discussion

[Levas]

Forget to mention.
The canon liveview freezes during recording.
The ML grayscale doesn't freeze during recording, but doesn't look like it can handle 10bit right now.

Is this normal, does canon live view freezes with the use of 10bit on all camera models ?

[Danne]

Gonna shoot some files on my 5D mark III soon.

Checked out your latest changes to mlv_dump but I'm getting a "Segmentation fault: 11" when turning on "--cs2x2". It looks like this isn't an issue specific to the 10bit_12bit branch.

This issue is solved in unified so it,s in the 10-bit/12-bit code?

http://www.magiclantern.fm/forum/index.php?topic=7122.msg174592#msg174592

[DeafEyeJedi]

...I used mlv_dump.osx (build from may 2015)...

Really?

...and all frames look fine in Rawtherapee.

Care to share screenshots?

[dfort]

This issue is solved in unified so it,s in the 10-bit/12-bit code?

Yes--so it looks like somehow this commit didn't make it into the 10bit_12bit branch. Just applied it and --cs2x2 works fine.

This is needed to get rid of the hot pixels that sometimes appear in high contrast boundaries. Not sure if those are only in cameras that have (had?) the focus pixel issue.

[Danne]

g3gg0, you are king. And major thanks to Greg for work on this via DARK code? http://www.magiclantern.fm/forum/index.php?topic=13408.msg172108#msg172108. Who did I forget?
Just added 10/12-bit branch into unified and added the raw_twk.mo from here http://www.magiclantern.fm/forum/index.php?topic=13163.msg174991#msg174991
and playback is more or less realtime with all bits 10,12,14. it even works on the eos M!, just tested briefly.

on my 5D3 setup recording still is a bit buggy, every other frame is corrupted.
anyone can confirm?

I get corrupted files(white level off I think) on my 5D mark III every now and then in both 10 and 12 bit. Also from time to time a text with "raw detect error"(I think it says) on the left. Canon eos M is the more stable participant right now it seems(thanks dmilligan, a1ex, dfort and others)
Shortened examples of corrupted files
test_1
https://drive.google.com/file/d/0B4tCJMlOYfircGtCRlNSWFFGU2M/view?usp=sharing
test_2
https://drive.google.com/file/d/0B4tCJMlOYfirckxRMmJqMm1EaFE/view?usp=sharing

[g3gg0]

@Danne:
happy to hear that

yeah the black/white level thing is maybe still an issue. tried to fix it by scaling white level properly when upconverting to 16 bit in mlv_play.
but maybe there is still smth twisted.

does mlv_lite perform better with recording, i.e. no corrupted frames?
didnt test t.b.h.

[Danne]

Only tested with mlv lite. Is 10 bit working with mlv_rec?? Not home to test atm.
One quirk with eos m is that play, pause, delete buttons are suppressed when running playback through raw_twk.mo. Buttons works great with 5D mark III.

[g3gg0]

i updated it also, so yes.

[Danne]

Will check later. Most greatful.
If anybody wants to try here are builds for 5D mark III and eos m with latest stuff. Raw_twk.mo included.(removed for now, causing corrupted black level)
Briefly tested. If anything breaks you get to keep all pieces.

*Black level issues fixed by g3gg0. New builds up
https://bitbucket.org/Dannephoto/cr2hdr/downloads

[baccinoluigi]

Will check later. Most greatful.
If anybody wants to try here are builds for 5D mark III and eos m with latest stuff. Raw_twk.mo included.
Briefly tested. If anything breaks you get to keep all pieces.
https://bitbucket.org/Dannephoto/cr2hdr/downloads

Hi i tested the last build on 5d Mark III sometimes it work correctly
but often the video is completely black
i send you a dng : https://www.dropbox.com/s/odn6xxrubhup6hz/M28-1717_000000.dng?dl=0

 
 

[Danne]

@baccinoluigi
Any difference in stability between mlv_rec and mlv_lite?

[Levas]

Had some time to do some more mlv_rec testing on 6d.

Tested 10bit, 12bit and 14bit.
10bit works, for all frames and both old mlv_dump and newest mlv_dump works.
12bit works, only old mlv_dump gives purple vertical lines, newest mlv_dump detects false cold pixels and gives frames with too much specles because of the false cold pixel detection.
14bit, still works

I'm using rawtherapee to view the files.

Uploading some dng files right now (few minutes uploading remaining)
http://drive.google.com/open?id=0B1BxGc3dfMDabVFDeG11QjBmWlk

[baccinoluigi]

@Danne
 
Tested again but mlv_rec and mlv_lite have the same issue

14 bit is more stable

[Levas]

Still wondering, how is liveview during recording on other cams ?
On the 6d, canon liveview freezes during 10bit recording, Grayscale preview is messed up during 10bit recording.

[a1ex]

@baccinoluigi:

Workaround:

Code: [Select]

exiftool *.dng -BlackLevel=128

Is the issue specific to a particular build, or all of them are buggy?

@Levas:

Your question was already answered by dmilligan earlier in this thread.

[robertgl]

Hello 50D user here
1. Magenta frames. It shows up on 10,12 and 14 when using the available modified fw. I haven't tried to change the black levels with mlv_dump, assuming that's the way to do it

2. The first frame keeps showing up in the even frames. So it's like 0,1,0,3,0,5,0,7,0,9,

3. at 1856x*** I get full frames interspersed with that continuing first frame showing up in even frames, while if I go to 1920x**** I get torn frames  with most of the frame being the first frame repeating, while a small sliver is the properly sequenced frame

4. Grayscale preview seems to run fine when recording

5. fingers crossed that the bugs can be ironed out because recordings look amazing and have a smaller files size 

Am exporting with : mlv_dump --dng --no-fixcp --no-stripes
debayering in rawtherapee

[eNnvi]

testing on 700D, builded from official hudson repository, raw_video_10bit_12bit branch about 10 minutes ago.

Found a "BUG", setting full HD res, no crop mode, going over the limit it uses max res possible on camera (1736x976) not divisible by 16 so crash happens after a few secs recording, this in both mlv_rec and raw_rec modules.

here logs after that crash
https://drive.google.com/open?id=0BxMUvr5E8ANaMDZUbkJnTHFya28  LOG000.log
https://drive.google.com/open?id=0BxMUvr5E8ANaci12Y2xUSU1MREE CRASH00.log

[baccinoluigi]

@a1ex

Is the issue specific to a particular build, or all of them are buggy?

The issue it's just in the build downloaded here : https://bitbucket.org/Dannephoto/cr2hdr/downloads
whith raw_twk.mo for 5dMkIII

[dfort]

Difference between raw_rec (MLV Lite) and mlv_rec on EOSM.

raw_rec is fine at all bit depths.

mlv_rec is fine at 14 bit, shows some corruption on every other frame at 12 bit and the corruption is worse at 10 bit.

12bit:


10bit:


No focus pixels though.

[Danne]

Tested some more around raw_twk on my eos m. Now I,m not sure if raw_twk is to be implemented on all cams. Anyway. Since it works really fast I think it,s a badass feature.
When recording mlv_rec and sound on an error message is shown Threads failed to start. When raw_twk.mo is off recording works. Is sound even working on eos m to begin with? I am not too familiar with this cam.
Mlv_play module has no start, stop, pause buttons when run through raw_twk. Can,t scroll around to the next video etc.
Like dfort points out mlv lite is producing good files while mlv_rec inherits corruption.

[Danne]

It seems raw_twk is causing mlv_lite(raw_rec) to produce the files with bad black levels on the 5D mark III. It happens once you start mlv_play and after this the next file usually turns out bad. When raw_twk is off everything seems fine when running mlv_lite. I will change my shared builds to builds without the raw_twk.mo for now.

[Levas]

@alex ah I missed that one from Dmilligan
@Dmilligan

Can you help me here, I can see the following code in edmac_memcpy.c (see italic below)
I wanna fix live view during recording for 6d, how do I find a free edmac channel with free edmac channels ?


/** this method bypasses Canon's lv_save_raw and slurps the raw data directly from connection #0 */
#ifdef CONFIG_EDMAC_RAW_SLURP

/* for other cameras, find a free channel with find_free_edmac_channels  */
#ifdef CONFIG_5D3
uint32_t raw_write_chan = 4;
#endif

#ifdef CONFIG_60D
uint32_t raw_write_chan = 1;
#endif

#ifdef CONFIG_600D
// write-index 1, 4, 6, 8, 10, 11, 13
uint32_t raw_write_chan = 4;
#endif

#ifdef CONFIG_EOSM
uint32_t raw_write_chan = 0x12;
#endif

[Levas]

I can see with show edmac in the Debug menu and scroll trough the channels, that channel 7, 8 and 13 are empty...
Is it that easy ?

[Levas]

Well...I can confirm that adding this line


#ifdef CONFIG_6D
uint32_t raw_write_chan = 7;
#endif



And compiling mlv_rec doesn't work...

[g3gg0]

@Danne:
black level issue should be fixed now