Author Topic: uncompressed 14-bit RAW video recording  (Read 749433 times)

dfsgfeiuhd

  • New to the forum
  • *
  • Posts: 19
  • 600d
Re: uncompressed 14-bit RAW video recording
« Reply #1125 on: May 17, 2013, 10:09:30 AM »
On the 600d there is no Menu to set the resolution for me, yet people reported that was working before. I am a bit confused. Also, the buffer always fills first before writing the files to card, no co-recording while buffering.

I use the latest nightly build and the latest BeepsTwice.zip.

Is there more files required? I tried two different cards and the feature is not hidden in ML Menu either. Might be a bug.

DjJuvan

  • New to the forum
  • *
  • Posts: 47
Re: uncompressed 14-bit RAW video recording
« Reply #1126 on: May 17, 2013, 11:28:10 AM »
Developers... this is maybe just my wild idea, but what about using SD card as backup temporary buffer for writing RAW video on CF cards? It could lower the chances of dropping frames because of too slow writing on CF cards and filling the in-camera buffer to quick. It could be than possible to record 1080p videos with slower CF cards too.

Just a thought?

N/A

  • Hero Member
  • *****
  • Posts: 576
  • Dreaming in 14 bit
Re: uncompressed 14-bit RAW video recording
« Reply #1127 on: May 17, 2013, 11:30:37 AM »
On the 600d there is no Menu to set the resolution for me, yet people reported that was working before. I am a bit confused. Also, the buffer always fills first before writing the files to card, no co-recording while buffering.

I use the latest nightly build and the latest BeepsTwice.zip.

Is there more files required? I tried two different cards and the feature is not hidden in ML Menu either. Might be a bug.

Did you put raw_rec module into the modules folder of the sd card? Works great on my 600D.
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

swinxx

  • Hero Member
  • *****
  • Posts: 602
Re: uncompressed 14-bit RAW video recording
« Reply #1128 on: May 17, 2013, 11:31:26 AM »
@DjJuvan

check Reply #1117, nearly identical thoughts..

greets

jpgentry

  • New to the forum
  • *
  • Posts: 8
Re: uncompressed 14-bit RAW video recording
« Reply #1129 on: May 17, 2013, 11:35:23 AM »
The bigger incentive for this would be pushing past 1080P to 2 or 2.5K resolution...

Developers... this is maybe just my wild idea, but what about using SD card as backup temporary buffer for writing RAW video on CF cards? It could lower the chances of dropping frames because of too slow writing on CF cards and filling the in-camera buffer to quick. It could be than possible to record 1080p videos with slower CF cards too.

Just a thought?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12222
  • Maintenance mode
Re: uncompressed 14-bit RAW video recording
« Reply #1130 on: May 17, 2013, 11:43:27 AM »
Benchmarks show a speed increase of about 20MB/s. The workflow and coding will be a bit more complex though.

For now, you have a CF+SD benchmark in Debug menu.

Grunf

  • New to the forum
  • *
  • Posts: 43
Re: uncompressed 14-bit RAW video recording
« Reply #1131 on: May 17, 2013, 11:50:20 AM »
Quick question (please ignore if I'm off kilter here):

Is it feasible to use CPU to shift away 4 bits and generate 8-bit 47MB/s 1080p24 raw file?
It would tax the CPU but it will lower the write speed to "manageable" 47MB/sec for those who don't need full 14-bit of colour?

CPU-assisted memcopy() should be good for approx 73MB/sec, so maybe it would suffice to copy the buffer to memory, do the quick "compression" by shifting away least significant 4 bits in CPU, and then using EDMAC to dump it to CF-card? (if this ARM-implementation has vector OP's that can do shift ops on wide-registers then it might not get so hot :) )

It would allow 1080p24 4:4:4 8-bit "raw" as many of us sought after for long time and still fit into reasonably quick CF-card bandwith :)



savale

  • Freshman
  • **
  • Posts: 69
Re: uncompressed 14-bit RAW video recording
« Reply #1132 on: May 17, 2013, 12:07:52 PM »
As developer (but not yet involved in the raw_rec recording dev) I see some options to improve the raw recording or add an option for slower cards:

1) if there is enough RAM / Cpu cycles left: I think the best way would be to use lossless compression to do the job. I can imagine this will move the problem to the processor, but there is a chance...

2) add an option to convert to 8bit raw? This will most probably introduces some nasty banding effects, but I am not sure if that is noticeable in real situations? This is quite easy to test and code I guess.

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3143
Re: uncompressed 14-bit RAW video recording
« Reply #1133 on: May 17, 2013, 12:11:56 PM »
As developer (but not yet involved in the raw_rec recording dev) I see some options to improve the raw recording or add an option for slower cards:

1) if there is enough RAM / Cpu cycles left: I think the best way would be to use lossless compression to do the job. I can imagine this will move the problem to the processor, but there is a chance...

2) add an option to convert to 8bit raw? This will most probably introduces some nasty banding effects, but I am not sure if that is noticeable in real situations? This is quite easy to test and code I guess.

both impossible due to CPU being way too slow
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

bumkicho

  • Member
  • ***
  • Posts: 102
Re: uncompressed 14-bit RAW video recording
« Reply #1134 on: May 17, 2013, 12:13:49 PM »
As developer (but not yet involved in the raw_rec recording dev) I see some options to improve the raw recording or add an option for slower cards:

1) if there is enough RAM / Cpu cycles left: I think the best way would be to use lossless compression to do the job. I can imagine this will move the problem to the processor, but there is a chance...

2) add an option to convert to 8bit raw? This will most probably introduces some nasty banding effects, but I am not sure if that is noticeable in real situations? This is quite easy to test and code I guess.

I can't see why I would want either one of these options for myself. Just buy faster cards.

ajay

  • New to the forum
  • *
  • Posts: 18
Re: uncompressed 14-bit RAW video recording
« Reply #1135 on: May 17, 2013, 12:33:04 PM »
Another RAW video, but from a perspective as a bird photographer/videoagrapher. I was quite anxious to see how the 5DM3's RAW output would look using birds as a model. Here's a pair of Eastern Bluebirds preening. I filmed in 1920x840 perspective due to bandwidth limitations when using 5x zoom using Trancend's 1000x 32GB card. I have gotten it to work reliably now at 1920 x 1080 but only at 1x not 5x. 5x drops frames @ 1080p.

Also...I have no idea if I am correctly using the 5x mode. I set zoom to 5x, then record through ML's menu. Is this correct? I found that in 5x mode, what I see on the LCD screen is not what is being recorded. The subject that I want centered is usually off to the right or left of the zoomed display. (That's why in 5x mode in this video, the birds are chopped off frame. This test was really to see how the resolution looked at 5x.)

Anyhow...I am so elated to finally get excellent quality video from the 5DM3. I can't thank you guys enough for doing this. I will make a donation today!

Here's the clip...I did color correction and a bit of sharpening to the DNG's.


Thank you ML team!

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12222
  • Maintenance mode
Re: uncompressed 14-bit RAW video recording
« Reply #1136 on: May 17, 2013, 12:39:10 PM »
I've added live preview in 5x mode, so you can see exactly what you are recording. Just wait for next build.

ajay

  • New to the forum
  • *
  • Posts: 18
Re: uncompressed 14-bit RAW video recording
« Reply #1137 on: May 17, 2013, 12:53:11 PM »
I've added live preview in 5x mode, so you can see exactly what you are recording. Just wait for next build.

Thanks Alex. I don't know if you realize this, but you guys are making A LOT of frustrated Canon users very happy.  :)

savale

  • Freshman
  • **
  • Posts: 69
Re: uncompressed 14-bit RAW video recording
« Reply #1138 on: May 17, 2013, 12:56:08 PM »
I've added live preview in 5x mode, so you can see exactly what you are recording. Just wait for next build.

Thanks! I suggested probably impossible raw_rec features: (8bit raw or lossless compression). But do you see any other improvement possible for the 5d2 users (because the compact flash card interface seems to be maxed out...)

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3143
Re: uncompressed 14-bit RAW video recording
« Reply #1139 on: May 17, 2013, 01:01:48 PM »
Thanks! I suggested probably impossible raw_rec features: (8bit raw or lossless compression). But do you see any other improvement possible for the 5d2 users (because the compact flash card interface seems to be maxed out...)

answered here:

both impossible due to CPU being way too slow

and we had inspected 14->12 bit reduction hardware, but its too complex yet.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Mr. Propaganda Guy

  • New to the forum
  • *
  • Posts: 8
Re: uncompressed 14-bit RAW video recording
« Reply #1140 on: May 17, 2013, 01:12:09 PM »
If you guys get this running on the 7D, "Hope you do" you think it'll be possible to use the 2nd processor to add compression to the RAW? If so, what compression would be possible, Mjpeg, ProRes, DNxHD?

tihon

  • Freshman
  • **
  • Posts: 74
Re: uncompressed 14-bit RAW video recording
« Reply #1141 on: May 17, 2013, 01:13:26 PM »
Do you mean: recording with 12bit dng raw?

Gonna be really cool! 14 bit is too much as  i think and make the file size to big to rec
Cinema, cinema, cinema

mnememedia

  • New to the forum
  • *
  • Posts: 2
Re: uncompressed 14-bit RAW video recording
« Reply #1142 on: May 17, 2013, 01:16:36 PM »
Hi all,

First, many thanks to the team and all the work they've done so far. Keep it up! The quality afforded by RAW capture is simply stunning. Let me know what you think of my quick sunset shoot I did this evening. As always, be sure to download the original to see the full detail.



Mr. Propaganda Guy

  • New to the forum
  • *
  • Posts: 8
Re: uncompressed 14-bit RAW video recording
« Reply #1143 on: May 17, 2013, 01:22:42 PM »
What would be the file size of 1 frame at 1920x1080 12bit RAW?

Brian@202020

  • New to the forum
  • *
  • Posts: 12
Re: uncompressed 14-bit RAW video recording
« Reply #1144 on: May 17, 2013, 01:26:34 PM »
Any chance any of the SD card based Canon DSLR's internal SD hardware can harness the fast data rates of the new Panasonic MicroP2 SD cards? At 2Gb/s they are currently the only UHS-II cards I know of.

http://www.videomaker.com/videonews/2013/04/nab-2013-panasonic-microp2-resizes-p2-to-sd

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5487
Re: uncompressed 14-bit RAW video recording
« Reply #1145 on: May 17, 2013, 01:28:29 PM »
An easy to follow mac-tutorial focusing on fast conversion to AppleProRes422HQ and a bunch of other stuff like creating xmp-templates, setting upp after effects, premiere pro.



Thanks!
//D

tihon

  • Freshman
  • **
  • Posts: 74
Re: uncompressed 14-bit RAW video recording
« Reply #1146 on: May 17, 2013, 01:29:43 PM »
What would be the file size of 1 frame at 1920x1080 12bit RAW?

Approximate RAW file
size on a Nikon D7000   12 bit   14 bit
compressed   12.6 MB   15.7 MB
uncompressed   14.9 MB   18.8 MB

sooo. we can see the difference in %. The filesize is 20% less in 12 bit then 14 bit
Cinema, cinema, cinema

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3143
Re: uncompressed 14-bit RAW video recording
« Reply #1147 on: May 17, 2013, 01:29:55 PM »
If you guys get this running on the 7D, "Hope you do" you think it'll be possible to use the 2nd processor to add compression to the RAW? If so, what compression would be possible, Mjpeg, ProRes, DNxHD?
no, none
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3143
Re: uncompressed 14-bit RAW video recording
« Reply #1148 on: May 17, 2013, 01:30:48 PM »
Do you mean: recording with 12bit dng raw?

Gonna be really cool! 14 bit is too much as  i think and make the file size to big to rec

as said, its too complex yet. maybe in 1-2 years
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Grunf

  • New to the forum
  • *
  • Posts: 43
Re: uncompressed 14-bit RAW video recording
« Reply #1149 on: May 17, 2013, 01:34:59 PM »
OK, if using CPU is out of the question then only option is to try to offload the data to CF interface as quick as possible.

As far as I understand, EDMAC has >600MB/sec throughput so as long as CF can swallow it, there is nothing stopping you to record higher resolutions and/or framerates.

Theoretically there is nothing preventing someone from using for example , Ti TMS320VC54x DSP as "faux CF-card" and letting it shuffle data further to SSD. :D

Current CF UDMA7 raw-interface speed is 167MB/sec so that would suffice for 1080p50 raw :D