12-bit (and 10-bit) RAW video development discussion

[benoit]

Hi,

I'm new to this forum, I 've read all this thread about 10/12 bit raw and many other threads on raw, so I apologise if my question is too simple.
First above all, I want to thanks the magic lantern team    ! and reddeercity for his 5D2 support with tutorials all along so many years ...
Now my question : with my 5D2 and Dec18.5D2212 software, I can't get "crop" or "not crop" without Liveview freezing.
So reddeercity how can you film 8 min ? with hdmi output ? with "magic" settings ?

Let me know, please.

David

[honza123]

Testing of 10bit-raw video /EOS 600D/ at very low light conditions (ISO 1600-6400)
www.youtube.com/watch?v=FVdyuj1OHJg

[dariSSight]

@reddeercity which build did you use for flawless 10bit 8min recording?

[ch_d]

@reddeercity - yes, link please.

[ShootMeAlready]

@paulharwood

From my experience with ML raw, hot pixels increase with higher ISO and record time.
Try ISO 100 or 400 max, and limit the record time per clip.  How long did you record with T3i before hot pixel appeared?

[PaulHarwood856]

Hey ShootMeAlready,

     This was a test in case of low light conditions. So I had the ISO at 3200. I'll make sure to do the same test at ISO 100 or 400 to see if there are no hot pixels. Thanks a bunch!

- Paul Harwood

[PaulHarwood856]

Ok, here's the debug build:

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-10bit_12bit_RAW_DEBUG_TYPE.2016Dec29.600D102.zip

It might work right away. The issue is that several cameras need to have the PREFERRED_RAW_TYPE defined, the 600D might be one of them. I started with a value of 0x10 which might do the trick but you can try other values. Go to the Debug menu, select the first item and you'll be able to change it.



Here's a more detailed explanation of what is going on:

https://bitbucket.org/hudson/magic-lantern/pull-requests/780/increase-raw_debug_type-limit/diff

Hey dfort,

     So far I've tested 0x10 in the debug build, and got hot pixels, but no highlight issues with the fan in the shot. I'll keep testing, but just wanted to give you an update.

- Paul Harwood

[a1ex]

From this analysis, PREFERRED_RAW_TYPE 0x10 appears to be the best choice for all DIGIC 5 and 6 cameras, and likely 0x5 for all DIGIC 4.

Can you confirm the above value? (600D, and also other cameras where PREFERRED_RAW_TYPE is not yet defined)

You should get:
- no bad pixels
- no changes from digital ISO (e.g. ISO 800 should be identical to 640 on the raw histogram)

[PaulHarwood856]

Hey Alex,

     Here is a link a dng from the 0x10 Debug. I saw the hot pixels in MLV Producer, but in this dng generated from cr2hdr.app I am only seeing a little artifacting in the candle and pink highlights in the fan. Also, is getting Err 70 normal when trying to playback raw files in a debug build?

- Paul Harwood

[a1ex]

0x10 is not relevant for 600D.

[dfort]

Oops -- @PaulHarwood856 could you go into the Debug menu, change it to 0x5 and run another test please?

[ilia3101]

As it is now well known, the 5D2 and most other cameras don't work in 10/12 bit, and definitely won't until CONFIG_EDMAC_RAW_SLURP is enabled, so I have decided I would like to help fix it on the 5D2. All I can do right now is compile Magic Lantern(with my own idiotic changes), and I've successfully ran the ROM disassembly script, so I don't really have much of a clue how to do anything, but I do have too much free time(sometimes), so is this something I could do with limited ability but lots of free time? Most complex thing I've ever done with C is getting two Arduinos transmitting numbers at each other at slightly over 1 Mbit/hour(yeah that slow ) with analog 433mhz antennas, so I don't know if my coding skills are enough.

Basically all I ask is, might I have enough ability to get it done or help out? and if so, where to start?

[reddeercity]

@ch_d  & dariSSight sorry for the delay , so the build I'm using is from dec.01/2016.
But I don't think that really matters , as mlv 2.0+audio code has not changed other then
implementing bit reduction code , so every thing else should be the same , where as with raw_rec (mlv Lite)
code has gone under I lot of work to get the limited amount of cam to work at this point.
So I'm betting that any of the MLV2.0 builds with bit reduction should work find ,
just in case there don't work for you guy's here the one I used  10bit_12bit_raw_2016Dec01.5D2212.
This build came from dfort so I didn't even compile the build myself.

Now on to something different , I said I would post the results of my 5D2 10bit 2.1K green screen 8:00 min shot .
This was just a test , but work so well that I put more work in to it and ended up as a short talk/explanation of 10,12bit raw video.
I was very impressed was the results , even thou I had a few issue with setting the shot up, focusing(as I was by myself , No auto focus in 3x crop mode)
The setting I used was mlv+audio 10bit 2144x1076 23.976p from 30p (not sure if 24 from 30 make any differences)
Auto preview for Liveview , audio set to 16bit 48khz (I ran the audio from my Zoom N4h in to the ext. mic input on cam)
ISO 400 , EF 24-70 f2.8L set to 35mm (Crop mode -- 2.60 crop x 35mm= 91mm) and had my zactuo evf connected (I needed something to reference to even thou it only shows
the center crop of the 2.1K image & B/W , it let me know when the recording stops but never did 
Process the 10bit mlv+audio on my PC with acr a.e. (on the my Mac the 10bit frames don't show up just all black from mlvfs, do show image in Resolve thou)
as I wanted to sharping up the image ( little soft on the focus) started with the adobe neutral profile and just pulled down the contrast & desaturated (flatted out)
export as 16bit tiff , imported in to apple compressor made my prores4444xq , graded in FCPX . I could have spent more time grading but I was looking for a quick turn around.
Any ways I hope everyone Enjoys it.
Happy New Years to everyone here on the forum 

[budafilms]

@reddeercity

Very good video and explanation about the last news!

[dfort]

A few of us have been poking around the lv_raw_dump code for various cameras and it looks like the 1100D has very similar code to the 600D. I was able to get a grasp how to find the DEFAULT_RAW_BUFFER for two other similar cameras, the EOSM and 700D but the code for these cameras are vastly different--maybe because the 1100D/600D (T3/T3i) are Digic 4 while the EOSM/700D (T5i) are Digic 5 cameras?

It looks like the only cameras that are not working with 10/12 bit are all Digic 4 or dual Digic 4 cameras. The 60D and 600D are also Digic 4 and they are working so perhaps it will be possible to get all of the ML supported cameras working if we can understand how to follow these instructions?

Code: [Select]

/* hardcode Canon's raw buffer directly */
/* you can find it from lv_raw_dump, arg1 passed to dump_file:
 *
 * raw_buffer = get_raw_buffer()
 * sprintf_maybe(filename, '%08lx.mm1', raw_buffer)
 * ...
 * dump_file(filename, raw_buffer, 7*something...)
 */

[EDIT] Thanks to @dmilligan and @eNnvi for taking the time to explain the EOSM code to me.

[budafilms]

5D mark III 1.1.3 works like a charm.

- Only in zoom 5x can't visualize. But if you press Info, can visualize at 10x zoom.
- I tried 10bits with zoom mode, 3,5K 4:1/5/1 2,5K, etc... aspect ratio and everything works perfect, continuous recording, except over 105 Mb/s or more.

52 Degree Celsius after 30 minutes playing and changing. I don't know if is too much. (A tea it's around 70?  )

[dfort]

I'm going to be cleaning up the mess in my bitbucket download area  and posting new test builds but first a download count and preview of coming attractions:

Checkmarks are for cameras that are working with CONFIG_EDMAC_RAW_SLURP:

5D3.123	-	118	√	* made a few attempts at merging crop_rec but was unsuccessful
5D3.113	-	56	√	* attempted merging crop_rec with limited success and had 129 downloads
6D.116	-	58	√
5D2.212	-	31
7D.203	-	22
EOSM.202	-	19	√
600D.102	-	17	√
70D.112	-	17	√	* might have build issues? (Dec 23)
100D.101	-	16	√	* might have build issues? (Dec 23)
650D.104	-	15	√
1100D.105	-	15	√	* allocate_raw_buffer branch has CONFIG_EDMAC_RAW_SLURP working but not 10bit/12bit (Dec 19)
60D.111	-	14	√
700D.114	-	13	√
550D.109	-	11
500D.111	-	8
50D.109	-	6

The next round of test builds will not include the 70D and 100D because there might have been build issues with them and since they are not merged into the main repository I'll need the help of nikfreak with them. Also, the 1100D won't go up until we can track down the DEFAULT_RAW_BUFFER for that camera. The 600D, 650D and 700D test builds will define the PREFERRED_RAW_TYPE to see if that helps those cameras.

Give it a few hours for the new test builds to show up here:

https://bitbucket.org/daniel_fort/magic-lantern/downloads

[andy kh]

Oh very unfortunate for being a  70D user

[dfort]

Oh very unfortunate for being a  70D user

70D and 100D users should be giving their feedback to the developers so these platforms will be accepted into the unified branch. Once they are in the main branch adding them to the raw_video_10bit_12bit branch is much easier.

https://bitbucket.org/hudson/magic-lantern/pull-requests/760/adding-support-for-the-70d-112/diff
https://bitbucket.org/hudson/magic-lantern/pull-requests/757/adding-support-for-the-eos-100d-sl1/diff

This also applies to 5D3.123 users:

https://bitbucket.org/hudson/magic-lantern/pull-requests/611/support-for-5d3-firmwares-123-and-113-in/diff

[PaulHarwood856]

Hey dfort,

     I'll make sure to test 0x5 on the debug build. Thanks for the heads up.

- Paul Harwood

[PaulHarwood856]

Hey reddeercity,

     Thanks so much for making this video! This explains everything really well, much appreciated!

- Paul Harwood

[Asiyuk]

Hi guys,

I have a problem with 3 of 100 video that I shoot just few days ago. Other 97 video files are  really good. I shoot 10bit mlv on my 5d3_1.1.3 with 24dec Dfort's build. The problem is that all dng sequence is extremly dark, contrast and all green. I attach some screenshots from davinci resolve and 2 original dng. I tried  also adobe after effects and lightroom, but have the same problem. I used mlvfs and mlv_dump to get dng, but i see this problem.





here is dng: https://we.tl/1Dp6BTOiB8

Does enyone have some advice for me?

English is my third language, but I hope, you understand

[reddeercity]

@PaulHarwood856 & budafilms thanks for the positive feed back 

[dfort]

Test builds are up.

https://bitbucket.org/daniel_fort/magic-lantern/downloads

@reddeercity - Thanks for the video. So it was shot in 10bit on a 5D2 in crop mode? If you think it works good now think how good it will be once that camera is working with CONFIG_EDMAC_RAW_SLURP.

[SpcCb]

Thanks reddeercity, nice presentation

So it's working only in crop mode on the 5D2? Or you are using this mode because of aliasing in scaled mode?
Ok, I saw in the other thread you said there'r some problems in full mode.