12-bit (and 10-bit) RAW video development discussion

[DeafEyeJedi]

Alright guys here we go with Round Two of me going nuts! 

This time it includes the crop_rec.mo together with 10/12bit options along with a raw_twk.mo that has been inserted as well for each build.

https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads

Special credit goes out to @Danne and @dfort for pushing this forward (Thanks for putting up with my shit this AM) as they were extremely helpful on making this merge possible.

Also I'm still in the process of trying (still a noob myself) to merge this together with 100D/70D from @nikfreak's branch. So will need more time on this meanwhile the previous build from yesterday is still online for now until then. Enjoy!

[andy kh]

Also I'm still in the process of trying (still a noob myself) to merge this together with 100D/70D from @nikfreak's branch.

Sounds great@deafeyejedi

[hammermina]

didnt work on 550d
got only color pixel mashup,but the recording times are amazing...

[Walter Schulz]

Post-processing tool used?

1. Try ExFAT file system. Format card using cardreader, copy ML files to card and redo installation.
2. Try lower resolutions.
Report back with results.

[Markus]

Crashlog from yesterday when I tried to get 10-12bit working on 5d3 123:

ASSERT: hLvJob->hJpegMemSuite
at ./Epp/Vram/VramStage.c:891, task Epp
lv:1 mode:3


Magic Lantern version : Nightly.2016Nov30.5D3123
Mercurial changeset   : 083982718d31+fd61d21d1429+ (5D3-113_123_10bit_12bit)
Built on 2016-11-30 08:02:13 UTC by rosiefort@RosieFoComputer.
Free Memory  : 165K + 3736K

Not sure if this is helpful?
I'll try out 113.

[DeafEyeJedi]

7D users: Please try this build and let me know if it's any better (or worse) than yesterday's build?

https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/crop_rec10bit_12bit_twk.2016Dec01.7D203.zip

Happy December and Thanks to @nikfreak for the tip.

*edit*

Thanks guys for tests and my apologies for this bad build. Will get to the bottom of this after some cleaning up.

[hammermina]

ok now it worked but picture glitches....
i can only preview in 550d on desktop there ist still pixelmashup
i use raw2cdng 1.75
what do you use?

[Surin Dmitriy]

Alright guys here we go with Round Two of me going nuts! 

This time it includes the crop_rec.mo together with 10/12bit options along with a raw_twk.mo that has been inserted as well for each build.

https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads

Special credit goes out to @Danne and @dfort for pushing this forward (Thanks for putting up with my shit this AM) as they were extremely helpful on making this merge possible.

Also I'm still in the process of trying (still a noob myself) to merge this together with 100D/70D from @nikfreak's branch. So will need more time on this meanwhile the previous build from yesterday is still online for now until then. Enjoy!

All fine, but x5 zoom crash the camera first time whith in the raw twk activated. If deactivate reboot and activate raw twk again works ok/.

[Danne]

http://www.magiclantern.fm/forum/index.php?topic=5601.msg175786#msg175786
The build for 7D above is broken. Don,t use it.

[Markus]

5D3 113 Results:

Works
12bit & 10bit RAW Full HD
3xcrop with good preview works. Even with 10bit.
5XZoom without rawtwk loaded works
recorded 2.5k successfully but preview bugged out when the recording starts.

Things that dont:
12bit MLV screen tearing in the top part of the frame as MitchLally noticed.
10bit MLV gives colorful corruption on every other frame top third part
5xZoom with raw twk results in crash as Surin Dmitriy noted

[Danne]

@Markus
Any difference in corruption for between raw_rec and mlv_rec?

[MitchLally]

Confirmed 5X Zoom causes freeze up, pop battery to rectify. Turned 5x off in Liveview Zoom Tweaks to continue testing.

Tested in mlv_rec only as most of my work requires sound. 5D3 113, tested all 3 bit depths. Global Draw on, mlv_play, raw_twk loaded. 1920x1080 24p. Used deafeyejedi's latest build. Used MLVFS (vert stripe fix on) and Davinci Resolve 12.5 to decode. Exported at h264.

Here is what I have:

694e0a161e
(definitely worth hitting that download button to see the original file)

As you can see, 14bit is normal, 12bit looks normal, 10bit every second(ish) frame is corrupted in top third. Plays back in mlv_play this way too.

Upon further inspection – testing 12bit whilst moving the camera I encountered this (keep an eye on the top portion of the frame):

9a8cd67e0b

Also it seems that it isn't decoding all the frames...  Same number of frames in the folder as 10/14 bit... Could this be a resolve issue? Or MLVFS?

Awesome progress so far, thank you to everyone working on this. I am suuuper keen to see at least 12bit working in mlv_rec! That would be a dream come true.

Here's some size comparisons:

https://s15.postimg.org/71b8gur6j/Screen_Shot_2016_12_01_at_10_23_34_pm.jpg

[Danne]

Raw_rec works without glitches 10bit/12bit?
Is raw_twk causing the 5x zoom to freeze?

[teatotalTED]

On 550D rawrec 10bit produces multicolor corruption to top 1/3rd of every other frame. mlvrec 10bit produces 16pixel offset to right every other frame but image intact. Playback in camera totally corrupt.

[MitchLally]

Yes it seems to be caused by raw_twk. Can zoom 5X fine without this module loaded.

Raw_rec works without glitches 10bit/12bit.

Here's raw_rec at 10 and 12 respectively:

e7db2974ed

Exactly same workflow as mentioned in last post.

[Markus]

Raw_rec works without glitches 10bit/12bit?
Is raw_twk causing the 5x zoom to freeze?

@Danne
Yes rawrec 10-12bit works as far as I can see.
Yes raw_twk causes freeze.

[a1ex]

Doing some more testing with 5D2 on 10-12bit raw with updated resolutions. [...] Anything with 1880 in all 3 bit ranges are totally corrupted and crashes the recording session , ( early stop etc..) but not the camera .

1880 works just fine here on 5D2 (just some minor UI quirks when fine-tuning the resolution). Playback tested on classic mlv_play (14-bit only) and checked the first DNG with mlv_dump (all bit depths).

[LEVISDAVIS]

Canon 50D: 11/29 & 11/30 Firmware: Getting the same 10-bit and 12-bit results as the 550D (@ teatotalTED), it's the bottom portion of the screen that repeats. FYI: The image splits in the middle of the frame. However, one time it split at the top 2/3rds. The split image repeats every other frame.

Meanwhile: 14-bit is as solid as ever. (** First time using MLVFS and the image is really something worth broadcasting! **)

[Oswald]

I tried the Deafeyejedi:s 7d build and it worked worse than previous one:
1) when trying to film with raw_lite, it just says data corruption at slot 21, 22, 23 or something like that.
2) crop mode freezes whole live view, but not the camera
3) i think that dng:s are more corrupt than before.

Also i tested the your 100d build and it has same problem than before, so, when recording start, it freezes the live view.

[reddeercity]

1880 works just fine here on 5D2 (just some minor UI quirks when fine-tuning the resolution). Playback tested on classic mlv_play (14-bit only) and checked the first DNG with mlv_dump (all bit depths).

Ok , I'll recompile the build and try again maybe I messed up some where or missed something .

[Steve_FR]

Tested Eos M running Crop_rec nov30th build from DeafEyeJedi.

Recording Raw 10 bit @ 1728x692 (2.50:1) using movie crop mode works smoothly, with live view display working great during raw REC.
Playback of the last 10bit raw clip using mlv_play and raw_twk enabled works great, but there are no controls to start, stop, track back or track forward. I can interrupt the raw playback with a half-shutter press, and return to normal liveview. I can play the clip as many times as I want if I keep interrupting it before it finishes, but if I allow the clip to play until the last frame, the camera appears to hang on that frame. A half shutter returns to liveview. Afterward, raw playback soft hangs on a black screen (half shutter returns to liveView). The only way for me to see the raw playback again is to power cycle the camera. The camera is not crashed or frozen, though.

[eNnvi]

Ok share my report.

Tested on 700D, after selecting 10bit it doesn't allow to select 12 bit, it shows in the dropdown list but non way to select it.

Anyway i think we MUST find a unique way for testing as i can see that many corrupted frames take place only when converting from .mlv to the dng sequence.

So please when talking about corrupted frames also say what decodifier you're using and (eventually) the options with it.



Here is my very short test:
Camera: 700D
Decodified with: MLVFS (running as service)
OS: Windows
BPP: 10 bit
Resolution: 1736x784
Memory card: Scheda SanDisk Extreme 32 GB SDHC UHS-I - 45 MB/S
raw_twk: no
Corrupted frames: none
Build: 30th november from DeafEyeJedi
Notes:
got liveview working, quite slow to be honest but fast enough for now


I'll do further testing on saturday morning (yeah sunlight shots )

I'd like to know what's the repo we should consider if we want to help (so check last versions and so on), is the hudson's official bitbucket repo or should i browse many other repos?


If you need help (also programming, got some skill in C) just ask

[dfort]

Tested on 700D, after selecting 10bit it doesn't allow to select 12 bit, it shows in the dropdown list but non way to select it.

I made new test builds and selecting 12 bit is working fine on the 700D. Haven't done extensive tests yet but it seems that mlv_rec has the corrupt frames issue while raw_rec (MLV Lite) is ok.

Lots of exciting development going on. A few of us have been merging 10bit_12bit with crop_rec and raw_twk and posting test builds. Here's my latest--all platforms including 5D3.123 but excluding 1100D, 70D and 100D. I'll keep it up for a few days or when there are new commits to the development branches.

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-10bit_12bit_raw_twk_crop_rec.2016Dec01.zip

Note that crop_rec only works with 5D3 and EOSM and some combinations of modules aren't working. Lots of testing opportunities but please make sure you report where you got the build, date, if you are using mlv_rec or raw_rec and if you find an issue report it in the appropriate forum topic.

[Rewind]

selecting 12 bit is working fine on the 700D

So LV display during recording works on 700D too? No freezing?

[dfort]

So LV display during recording works on 700D too? No freezing?

No--during recording LiveView still freezes. This was also the case with the EOSM but it was resolved so there's still hope for the other platforms. I believe that it was the 
EOSM: enable CONFIG_EDMAC_RAW_SLURP commit by dmilligan that did the trick.