12-bit (and 10-bit) RAW video development discussion

[dfort]

dfort probably didn't enable edmac_raw_slurp before compiling for 100d. Just a guess as I didn't try myself.

@nikfreak - Could you make a 10bit_12bit branch in your repository that includes the 70D and 100D? I don't have these cameras available for testing so I can't be sure that I put all the pieces in the right place. Just a thought--maybe do your 70D and 100D pull requests on the raw_video_10bit_12bit branch? Seems that this is where all the action is happening and since this is a development branch maybe the developers with the merge key are more receptive to adding new platforms here as opposed to the "stable" unified branch?

@dariSSight - The 5D2.212 won't be much better with 10bit/12bit until CONFIG_EDMAC_RAW_SLURP is working on that camera. Seems to be a matter of finding a free edmac channel and sniffing out where the DEFAULT_RAW_BUFFER is located -- it is different on every camera.

[dfort]

Looks like there's lots of requests for crop_rec with the 5D3.113. Since that platform seems to be well tested with 10bit/12bit I put up a test build that includes crop_rec:

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-raw_video_10bit_12bit_crop_rec.2016Dec24.5D3113.zip

The 5D3.123 test build also includes the crop_rec module.

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-raw_video_10bit_12bit.2016Dec20.5D3123.zip

[ewinemiller]

The 5D3.123 test build also includes the crop_rec module.

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-raw_video_10bit_12bit.2016Dec20.5D3123.zip

Hey dfort.

I appreciate the builds, but if I try your 5D3 1.23 build from the 20th, it won't load the modules. I get failed to link modules error.

Regards.

[Camerite]

The 5D3 123 build seems to load all modules but crop rec. When activated the camera gives errors on all modules.

[kamranjon]

More than happy to help provide testing on the 7D.

[dfort]

I appreciate the builds, but if I try your 5D3 1.23 build from the 20th, it won't load the modules. I get failed to link modules error.

Strange, check reply #862 from @RenatoPhoto

More than happy to help provide testing on the 7D.

There is a test build posted but like the 5D2 it needs the CONFIG_EDMAC_RAW_SLURP treatment for it to work properly.

[vstrglv]

Looks like there's lots of requests for crop_rec with the 5D3.113. Since that platform seems to be well tested with 10bit/12bit I put up a test build that includes crop_rec:
https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-raw_video_10bit_12bit_crop_rec.2016Dec24.5D3113.zip

Thank you very much! Quick test - frame issue:
10bit,12bit, crop1:1 and not crop mode - preview framing during recording  is not correct. All screen is filled with image (there are no horizontal black stripes).
Sometimes this issue takes place in 14bit mode too. I could not find a regularity.   

[andy kh]

tried dfort's test build for 70D but no luck

[ewinemiller]

Strange, check reply #862 from @RenatoPhoto

It's crop_rec.mo. If I enable it, no modules load. I don't think he tested that. Looks like he was using the classic crop stuff.

[RenatoPhoto]

Guys!  I did not use crop.rec!!!
@dfort

I used the standard crop mode using the magnify button..  So it works on crop mode.
crop.rec is and advanced version of crop mode that centers the images and has other nice features.
It is very good but is not incorporated into unified.. it is a separate branch.

[vstrglv]

Thank you very much! Quick test - frame issue:
10bit,12bit, crop1:1 and not crop mode - preview framing during recording  is not correct. All screen is filled with image (there are no horizontal black stripes).
Sometimes this issue takes place in 14bit mode too.

This issue takes place only for mvl_rec mode. If preview options is set to "ML Grayscale"  this issue disappears and  mode is colour during  recording.

[RenatoPhoto]

@dfort
Also forgot to mention that at the start/stop of recording 5D3-123 with mlv_rec moldule I get this message:
Hack error at 0:
excpectede3a00032, got e1a00000

[dfort]

@g3gg0 updated mlv_dump and put up binaries for Windows, Mac and Linux so I removed the mlv_dump builds I was hosting. These new builds work with 10bit/12bit files.
http://www.magiclantern.fm/modules/modules/mlv_dump.zip/mlv_dump.zip

Some of you are posting links to the test builds which is fine but just a warning that I've been removing old builds to avoid confusion. That means that you'll need to save previous versions in case something breaks or you want to make a regression test.

There have been some reports that things aren't working properly on a few cameras that should be working. It might be the way I'm compiling. In most cases I'm just compiling the raw_video_10bit_12bit branch but in others I merged crop_rec and other branches. Maybe I should be merging unified too since eventually the goal is to get this into the nightly builds?

Here are the latest download counts of the test builds. Checkmarks are for cameras that are working with CONFIG_EDMAC_RAW_SLURP:

5D3.123	-	76	√	* with crop_rec (Dec 20)
5D3.113	-	55	√	* without crop_rec (Dec 18)
5D3.113	-	51	√	* with crop_rec (Dec 24)
6D.116	-	24	√	* new build works with 10bit/12bit (Dec 23)
5D2.212	-	21
7D.203	-	15
EOSM.202	-	14	√
100D.101	-	14	√	* might have build issues? (Dec 23)
600D.102	-	13	√
70D.112	-	13	√	* might have build issues? (Dec 23)
700D.114	-	12	√
60D.111	-	10	√
650D.104	-	10	√
1100D.105	-	9	√	* allocate_raw_buffer branch has CONFIG_EDMAC_RAW_SLURP working but not 10bit/12bit (Dec 19)
550D.109	-	8
50D.109	-	6
500D.111	-	3

Although this might not be an accurate list of cameras sorted by popularity, it does show quite a bit of interest in 10bit/12bit raw video from users of the higher end and full frame cameras. I was surprised by the amount of interest in the 5D3.123--looks like users want raw video and clean HDMI output.

There a few older models left to solve. Check out reply #754 for some hints:

From what I've tried so far on those old models (without much success yet), we have at least two problems:
- the raw buffer is not allocated all the time (as it's done on the new models) -> easy to fix, see the 1100D raw branch
- the vsync hook currently used doesn't appear to be the best for getting a raw image
- the exact behavior of EDMAC in case of input/output dimension mismatch is not very well known

Solving those models may require either some serious low-level research, or a great dose of luck. I'm just scratching the surface here.

[Lars Steenhoff]

for monitoring with an external monitor clean hdmi is quite nice.

and as a bonus the zoom mode wich shows in black and white with the 5x crop mode is showing a preview on the external monitor in color. ( last time i checked wich is a while ago)

[Quentin]

How fast in terms of FPS/Frame size is 12/10 bit RAW recording ?
I tried to follow the thread but its since 2013 and was inmature.
Any clues ?

[Rewind]

You can use the usual formula to check if it fits in your camera controller's speed limit:
W * H * BD * FPS / 8 /1024 / 1024
(W &  H — frame width and height in pixels, BD — bitdepth, which now can be 10 or 12 instead of 14, FPS is frames per second, and converting to megabytes per second). Result is required speed in MB/s

For example, for 650D we have a speed limit in 40MB/s.
We wish to film 10 bit raw video 1728x736 at 25fps.
1728*736*10*25/8/1024/1024 = 37.9 MB/s which fits fine in controller's limit. So in practice we can shoot continuously with these parameters.

[Quentin]

Thank you very much 
Its reasonable. Cant wait to try it out.
ML is so cool

[ShootMeAlready]

On my 70D MLV files tried the latest mlv_dump, all were failed.  This agrees with mystic not being able to convert any of the same MLV.

G:\DCIM\100CANON>mlv_dump --dng M23-2332.MLV

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'M23-2332.MLV'
   - Convert to DNG frames
   - Output into 'M23-2332_'
File M23-2332.MLV opened
File M23-2332.M00 not existing.
Processing...
[ERROR] Invalid block size at position 0x00000400
Processed 0 video frames
Done

my 600D MLV works fine with same program.

[dfort]

@nikfreak -- Need your help with the 70D. Could you create a 10bit/12bit branch in your repository with the appropriate merges? I got it to compile but I must have missed something. Don't have a 70D here to check things out. It would be great if you could do the same for the 100D.

Thanks!

[maxotics]

I'm happy to test the 12/10 bit on my 7D.  But would need instructions.  I can't make complete sense of it reading threads.  Again, only if you need someone to test.  I can wait

[ShootMeAlready]

600D, 1280 @2.35, 24 fps, 10 bit. 
I am getting pink frames (at top 25%) every other frame.
What's this about?
By bit depth & rate calculation it should be continuous.
My card is Sandisk Ex. Pro, 95 Mb/s so its not the card speed.


Reformat card, loaded latest 600D, all is good. Was able to get [email protected]/24fps  for about 2 seconds.

[whyfy]

Canon 60D 10Bit RAW video, Overexposure testing https://youtu.be/BlmajDezBB4

[dfort]

I'm happy to test the 12/10 bit on my 7D.

This isn't working on the 7D yet. A few people are looking into it. The hard part seems to be finding the memory location for DEFAULT_RAW_BUFFER.

[Teamsleepkid]

maxotics check out 10 bit raw on your eosm. I saw your vids before on youtube and vimeo. and its one of the main reasons i even got an eosm. theres not much info out there..you had some of the only raw vids available to watch. 10 bit raw on eosm is really great. give it a try.

[Teamsleepkid]

@quentin check this out http://rbrune.github.io/mlraw/ noticed it has settings for 10 bit and 12 bit now:)