Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 479178 times)

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3733
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #875 on: December 24, 2016, 08:09:55 PM »
dfort probably didn't enable edmac_raw_slurp before compiling for 100d. Just a guess as I didn't try myself.

@nikfreak - Could you make a 10bit_12bit branch in your repository that includes the 70D and 100D? I don't have these cameras available for testing so I can't be sure that I put all the pieces in the right place. Just a thought--maybe do your 70D and 100D pull requests on the raw_video_10bit_12bit branch? Seems that this is where all the action is happening and since this is a development branch maybe the developers with the merge key are more receptive to adding new platforms here as opposed to the "stable" unified branch?

@dariSSight - The 5D2.212 won't be much better with 10bit/12bit until CONFIG_EDMAC_RAW_SLURP is working on that camera. Seems to be a matter of finding a free edmac channel and sniffing out where the DEFAULT_RAW_BUFFER is located -- it is different on every camera.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3733
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #876 on: December 24, 2016, 08:49:20 PM »
Looks like there's lots of requests for crop_rec with the 5D3.113. Since that platform seems to be well tested with 10bit/12bit I put up a test build that includes crop_rec:

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-raw_video_10bit_12bit_crop_rec.2016Dec24.5D3113.zip

The 5D3.123 test build also includes the crop_rec module.

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-raw_video_10bit_12bit.2016Dec20.5D3123.zip
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

ewinemiller

  • New to the forum
  • *
  • Posts: 35
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #877 on: December 24, 2016, 09:56:13 PM »
The 5D3.123 test build also includes the crop_rec module.

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-raw_video_10bit_12bit.2016Dec20.5D3123.zip

Hey dfort.

I appreciate the builds, but if I try your 5D3 1.23 build from the 20th, it won't load the modules. I get failed to link modules error.

Regards.

Camerite

  • New to the forum
  • *
  • Posts: 6
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #878 on: December 24, 2016, 10:30:47 PM »
The 5D3 123 build seems to load all modules but crop rec. When activated the camera gives errors on all modules.

kamranjon

  • New to the forum
  • *
  • Posts: 37
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #879 on: December 24, 2016, 10:59:41 PM »
More than happy to help provide testing on the 7D.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3733
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #880 on: December 24, 2016, 11:27:20 PM »
I appreciate the builds, but if I try your 5D3 1.23 build from the 20th, it won't load the modules. I get failed to link modules error.

Strange, check reply #862 from @RenatoPhoto

More than happy to help provide testing on the 7D.

There is a test build posted but like the 5D2 it needs the CONFIG_EDMAC_RAW_SLURP treatment for it to work properly.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

vstrglv

  • Member
  • ***
  • Posts: 183
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #881 on: December 25, 2016, 08:24:33 AM »
Looks like there's lots of requests for crop_rec with the 5D3.113. Since that platform seems to be well tested with 10bit/12bit I put up a test build that includes crop_rec:
https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-raw_video_10bit_12bit_crop_rec.2016Dec24.5D3113.zip
Thank you very much! Quick test - frame issue:
10bit,12bit, crop1:1 and not crop mode - preview framing during recording  is not correct. All screen is filled with image (there are no horizontal black stripes).
Sometimes this issue takes place in 14bit mode too. I could not find a regularity.   
Canon 5D3,1.1.3 CF-SanDisk Extreme PRO,160MB/s, 256GB, SD-SanDisk Extreme, 90MB/s, 64GB.

andy kh

  • Hero Member
  • *****
  • Posts: 513
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #882 on: December 25, 2016, 10:57:01 AM »
tried dfort's test build for 70D but no luck
5D Mark III - 70D

ewinemiller

  • New to the forum
  • *
  • Posts: 35
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #883 on: December 25, 2016, 03:54:16 PM »
Strange, check reply #862 from @RenatoPhoto

It's crop_rec.mo. If I enable it, no modules load. I don't think he tested that. Looks like he was using the classic crop stuff.

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #884 on: December 25, 2016, 04:43:57 PM »
Guys!  I did not use crop.rec!!!
@dfort

I used the standard crop mode using the magnify button..  So it works on crop mode.
crop.rec is and advanced version of crop mode that centers the images and has other nice features.
It is very good but is not incorporated into unified.. it is a separate branch.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

vstrglv

  • Member
  • ***
  • Posts: 183
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #885 on: December 25, 2016, 04:54:24 PM »
Thank you very much! Quick test - frame issue:
10bit,12bit, crop1:1 and not crop mode - preview framing during recording  is not correct. All screen is filled with image (there are no horizontal black stripes).
Sometimes this issue takes place in 14bit mode too.
This issue takes place only for mvl_rec mode. If preview options is set to "ML Grayscale"  this issue disappears and  mode is colour during  recording.
Canon 5D3,1.1.3 CF-SanDisk Extreme PRO,160MB/s, 256GB, SD-SanDisk Extreme, 90MB/s, 64GB.

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #886 on: December 25, 2016, 05:55:15 PM »
@dfort
Also forgot to mention that at the start/stop of recording 5D3-123 with mlv_rec moldule I get this message:
Hack error at 0:
excpectede3a00032, got e1a00000
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3733
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #887 on: December 26, 2016, 09:10:21 PM »
@g3gg0 updated mlv_dump and put up binaries for Windows, Mac and Linux so I removed the mlv_dump builds I was hosting. These new builds work with 10bit/12bit files.
http://www.magiclantern.fm/modules/modules/mlv_dump.zip/mlv_dump.zip

Some of you are posting links to the test builds which is fine but just a warning that I've been removing old builds to avoid confusion. That means that you'll need to save previous versions in case something breaks or you want to make a regression test.

There have been some reports that things aren't working properly on a few cameras that should be working. It might be the way I'm compiling. In most cases I'm just compiling the raw_video_10bit_12bit branch but in others I merged crop_rec and other branches. Maybe I should be merging unified too since eventually the goal is to get this into the nightly builds?

Here are the latest download counts of the test builds. Checkmarks are for cameras that are working with CONFIG_EDMAC_RAW_SLURP:

5D3.123-76* with crop_rec (Dec 20)
5D3.113-55* without crop_rec (Dec 18)
5D3.113-51* with crop_rec (Dec 24)
6D.116-24* new build works with 10bit/12bit (Dec 23)
5D2.212-21
7D.203-15
EOSM.202-14
100D.101-14* might have build issues? (Dec 23)
600D.102-13
70D.112-13* might have build issues? (Dec 23)
700D.114-12
60D.111-10
650D.104-10
1100D.105-9* allocate_raw_buffer branch has CONFIG_EDMAC_RAW_SLURP working but not 10bit/12bit (Dec 19)
550D.109-8
50D.109-6
500D.111-3

Although this might not be an accurate list of cameras sorted by popularity, it does show quite a bit of interest in 10bit/12bit raw video from users of the higher end and full frame cameras. I was surprised by the amount of interest in the 5D3.123--looks like users want raw video and clean HDMI output.

There a few older models left to solve. Check out reply #754 for some hints:

From what I've tried so far on those old models (without much success yet), we have at least two problems:
- the raw buffer is not allocated all the time (as it's done on the new models) -> easy to fix, see the 1100D raw branch
- the vsync hook currently used doesn't appear to be the best for getting a raw image
- the exact behavior of EDMAC in case of input/output dimension mismatch is not very well known

Solving those models may require either some serious low-level research, or a great dose of luck. I'm just scratching the surface here.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Lars Steenhoff

  • Senior
  • ****
  • Posts: 453
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #888 on: December 27, 2016, 12:53:14 AM »
for monitoring with an external monitor clean hdmi is quite nice.

and as a bonus the zoom mode wich shows in black and white with the 5x crop mode is showing a preview on the external monitor in color. ( last time i checked wich is a while ago)

Quentin

  • Member
  • ***
  • Posts: 120
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #889 on: December 27, 2016, 08:23:16 AM »
How fast in terms of FPS/Frame size is 12/10 bit RAW recording ?
I tried to follow the thread but its since 2013 and was inmature.
Any clues ?

Rewind

  • Senior
  • ****
  • Posts: 356
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #890 on: December 27, 2016, 08:39:01 AM »
You can use the usual formula to check if it fits in your camera controller's speed limit:
W * H * BD * FPS / 8 /1024 / 1024
(W &  H — frame width and height in pixels, BD — bitdepth, which now can be 10 or 12 instead of 14, FPS is frames per second, and converting to megabytes per second). Result is required speed in MB/s

For example, for 650D we have a speed limit in 40MB/s.
We wish to film 10 bit raw video 1728x736 at 25fps.
1728*736*10*25/8/1024/1024 = 37.9 MB/s which fits fine in controller's limit. So in practice we can shoot continuously with these parameters.

Quentin

  • Member
  • ***
  • Posts: 120
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #891 on: December 27, 2016, 08:42:59 AM »
Thank you very much  :)
Its reasonable. Cant wait to try it out.
ML is so cool

ShootMeAlready

  • Senior
  • ****
  • Posts: 262
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #892 on: December 28, 2016, 04:12:34 PM »
On my 70D MLV files tried the latest mlv_dump, all were failed.  This agrees with mystic not being able to convert any of the same MLV.

G:\DCIM\100CANON>mlv_dump --dng M23-2332.MLV

 MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'M23-2332.MLV'
   - Convert to DNG frames
   - Output into 'M23-2332_'
File M23-2332.MLV opened
File M23-2332.M00 not existing.
Processing...
[ERROR] Invalid block size at position 0x00000400
Processed 0 video frames
Done

my 600D MLV works fine with same program.
T3i+ML & 70D.112+ML, Tokina 11-16 2.8, Sigma 18-35 1.8, 50-150 II 2.8, 50 1.4, Canon 28 1.8, 35 2, 85 1.8 "Shoot Wide and Prosper"

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3733
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #893 on: December 28, 2016, 04:36:28 PM »
@nikfreak -- Need your help with the 70D. Could you create a 10bit/12bit branch in your repository with the appropriate merges? I got it to compile but I must have missed something. Don't have a 70D here to check things out. It would be great if you could do the same for the 100D.

Thanks!
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

maxotics

  • Hero Member
  • *****
  • Posts: 531
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #894 on: December 28, 2016, 04:54:39 PM »
I'm happy to test the 12/10 bit on my 7D.  But would need instructions.  I can't make complete sense of it reading threads.  Again, only if you need someone to test.  I can wait ;)

ShootMeAlready

  • Senior
  • ****
  • Posts: 262
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #895 on: December 28, 2016, 05:53:00 PM »
600D, 1280 @2.35, 24 fps, 10 bit. 
I am getting pink frames (at top 25%) every other frame.
What's this about?
By bit depth & rate calculation it should be continuous.
My card is Sandisk Ex. Pro, 95 Mb/s so its not the card speed.


Reformat card, loaded latest 600D, all is good. Was able to get 1920@2.35/24fps  for about 2 seconds.
T3i+ML & 70D.112+ML, Tokina 11-16 2.8, Sigma 18-35 1.8, 50-150 II 2.8, 50 1.4, Canon 28 1.8, 35 2, 85 1.8 "Shoot Wide and Prosper"

whyfy

  • New to the forum
  • *
  • Posts: 34
  • 60D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #896 on: December 28, 2016, 06:49:16 PM »
Canon 60D 10Bit RAW video, Overexposure testing

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3733
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #897 on: December 29, 2016, 08:53:51 AM »
I'm happy to test the 12/10 bit on my 7D.

This isn't working on the 7D yet. A few people are looking into it. The hard part seems to be finding the memory location for DEFAULT_RAW_BUFFER.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Teamsleepkid

  • Member
  • ***
  • Posts: 229
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #898 on: December 29, 2016, 09:44:23 AM »
maxotics check out 10 bit raw on your eosm. I saw your vids before on youtube and vimeo. and its one of the main reasons i even got an eosm. theres not much info out there..you had some of the only raw vids available to watch. 10 bit raw on eosm is really great. give it a try.
EOS M

Teamsleepkid

  • Member
  • ***
  • Posts: 229
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #899 on: December 29, 2016, 09:51:02 AM »
@quentin check this out http://rbrune.github.io/mlraw/ noticed it has settings for 10 bit and 12 bit now:)
EOS M