Author Topic: uncompressed 14-bit RAW video recording  (Read 763849 times)

mcnys

  • New to the forum
  • *
  • Posts: 14
Re: uncompressed 14-bit RAW video recording
« Reply #1150 on: May 17, 2013, 01:39:38 PM »
ok this will sound amazingly stupid...but still...after reading this:

"With a little guesswork, we were lucky to notice that on the inside, the little EOS-M is very similar to the 5D Mark III. So, with a few tricks we were able to dump the firmware and print the familiar "Hello World" message – the proof that Magic Lantern will work on the EOS-M."

so i'm thinking, what kind of sd card controller does eos-m have? maybe its faster than 600-650d? if yes then maybe its the ultimate cheap raw video camera?

Yomommassis

  • New to the forum
  • *
  • Posts: 42
Re: uncompressed 14-bit RAW video recording
« Reply #1151 on: May 17, 2013, 01:43:03 PM »
I'm really surprised that the 5D mark 2 can handle any of this considering video was added in a firmware update and people have said the cpu was too slow to handle 60 FPS...yet here it is recording 24fps RAW video

Damn.

If the 5D mark 2 can handle this I have no doubt in my mind we will see the whole canon dslr line up shooting raw in the future
EOS 5D MkIII (ML alpha 3)| EOS 5D MkII (ML v2.3) | EOS 7D (ML alpha 2) | EOS T2i (ML v2.3)
f/2.8 70-200MM L USM | f/1.4 50mm USM | f/1.4 35mm L USM | T/1.5 24mm
Zoom H4n | Rode NTG2

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1857
  • Have you tried turning it off and on again?
Re: uncompressed 14-bit RAW video recording
« Reply #1152 on: May 17, 2013, 01:57:59 PM »
@mcnys - I think the EOS-M has a UHS-I slot but a lot also depends on the buffer size. UHS-I can write faster than SD but if the buffer isn't up to it there won't be much, if any, benefit over SD
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

AnotherDave

  • Senior
  • ****
  • Posts: 348
Re: uncompressed 14-bit RAW video recording
« Reply #1153 on: May 17, 2013, 01:59:14 PM »
OK, if using CPU is out of the question then only option is to try to offload the data to CF interface as quick as possible.

As far as I understand, EDMAC has >600MB/sec throughput so as long as CF can swallow it, there is nothing stopping you to record higher resolutions and/or framerates.

Theoretically there is nothing preventing someone from using for example , Ti TMS320VC54x DSP as "faux CF-card" and letting it shuffle data further to SSD. :D

Current CF UDMA7 raw-interface speed is 167MB/sec so that would suffice for 1080p50 raw :D

I think it is entirely possible to send the data that is offloading to the CF to a higher speed SSD via some sort of CF extender cable to SSD converter.  Fortunately, CF extender cables do exist... but currently there is no SSD to CF converter.

It has to be possible.  If one of you knows how to do it, go for it!  You'll be a millionaire thousandaire overnight!

This would solve all throughput problems and storage issues recording RAW from a DSLR if it worked, wouldn't it?

t2scorp

  • New to the forum
  • *
  • Posts: 25
Re: uncompressed 14-bit RAW video recording
« Reply #1154 on: May 17, 2013, 02:13:17 PM »
I'm really surprised that the 5D mark 2 can handle any of this considering video was added in a firmware update and people have said the cpu was too slow to handle 60 FPS...yet here it is recording 24fps RAW video

Damn.

If the 5D mark 2 can handle this I have no doubt in my mind we will see the whole canon dslr line up shooting raw in the future

The slow CPU is not involved. The RAW output is just saving the files to an other storage...

Alia5

  • Freshman
  • **
  • Posts: 99
  • 600D
Re: uncompressed 14-bit RAW video recording
« Reply #1155 on: May 17, 2013, 02:17:38 PM »
On my 600D i got zero magenta frames in videomode (720p) with 1120x400 crop (new version... HDR thingy)
with the linescipping factor of 1.66x that almost gives me a nice 16:9 ratio when using pretty much of the sensor

works like charm!
1% did an excellent job in porting!

excpect some testfootage!

cinema5D

  • New to the forum
  • *
  • Posts: 14
Re: uncompressed 14-bit RAW video recording
« Reply #1156 on: May 17, 2013, 02:21:19 PM »
The 25p version of the RAW module works flawlessy with our 128GB (!) Kompterbay cards in 1080p! ARTICLE LINK.

We just recorded a 4,23GB clip without any frame drop. The compile we used came from this post earlier in this thread. If only we could convert the large RAW files to dng's with that Mac app...  :-*

We also updated our step-by-step guide. Looking forward to more amazingness coming from this development here. Thanks to everyone involved.

N/A

  • Hero Member
  • *****
  • Posts: 576
  • Dreaming in 14 bit
Re: uncompressed 14-bit RAW video recording
« Reply #1157 on: May 17, 2013, 02:22:27 PM »
Here's some high ISO (2800) raw footage from 600D with the latest build.


1280x400 in 640x480 mode. Works like a charm now, no pink or skipped frames, only those weird blue/red/yellow pixelated areas at high ISO.
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: uncompressed 14-bit RAW video recording
« Reply #1158 on: May 17, 2013, 02:35:20 PM »
Thanks to the ML crew and  lourenco for the new build. I'm having a few issues but I gotta suss whether it's the camera or something in post before I make a report.


Grunf

  • New to the forum
  • *
  • Posts: 43
Re: uncompressed 14-bit RAW video recording
« Reply #1159 on: May 17, 2013, 02:38:40 PM »
I think it is entirely possible to send the data that is offloading to the CF to a higher speed SSD via some sort of CF extender cable to SSD converter.  Fortunately, CF extender cables do exist... but currently there is no SSD to CF converter.

It has to be possible.  If one of you knows how to do it, go for it!  You'll be a millionaire thousandaire overnight!

This would solve all throughput problems and storage issues recording RAW from a DSLR if it worked, wouldn't it?

Hmm... I don't think "cable" will suffice as there will me buffering involved in-between.

Basically, we are looking for DSP (or even better, SOC) with both SATA and CF UDMA-7 interfaces. But CF interface needs to be "inverted". Thus, such apparatus needs to emulate CF and send buffered data to SATA-interface.

there are lot's of SOC's providing interface *to* CF, but I wonder if there are any that can be reversed. I guess we have a little hardware project on our hands, no? ;)

Side 8 describes CF interface specs:

http://www.ti.com/lit/an/spra803/spra803.pdf

As far as I understand, we are talking 16-bit wide I/O bus.

Grunf

  • New to the forum
  • *
  • Posts: 43
Re: uncompressed 14-bit RAW video recording
« Reply #1160 on: May 17, 2013, 02:50:46 PM »
Hmm...thinking further, CF interface is noting else but old-style 16-bit wide "IDE" harddrive interface. So what we actually need is IDE (with UDMA-7 support) to SATA interface card.

support for SATA rev 1. would be enough as it supports up to 150MB/sec and UDMA-7 tops out at 157MB/sec.


nanomad

  • Administrator
  • Hero Member
  • *****
  • Posts: 2918
  • All your websites are belong to us
Re: uncompressed 14-bit RAW video recording
« Reply #1161 on: May 17, 2013, 02:52:24 PM »
You need an IDE <-> SATA converter that's actually capable of using UDMA7
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

mcnys

  • New to the forum
  • *
  • Posts: 14
Re: uncompressed 14-bit RAW video recording
« Reply #1162 on: May 17, 2013, 02:54:49 PM »
@mcnys - I think the EOS-M has a UHS-I slot but a lot also depends on the buffer size. UHS-I can write faster than SD but if the buffer isn't up to it there won't be much, if any, benefit over SD

lol, so basically if canon raised manufacturing costs by few dollars more, and made buffer size at least twice the size it would have been, amazing little raw camera...oh well too bad  :/

Grunf

  • New to the forum
  • *
  • Posts: 43
Re: uncompressed 14-bit RAW video recording
« Reply #1163 on: May 17, 2013, 03:00:17 PM »
OK guys, I think this can be done!

What we need is Bi-Directional converter for serial and parallel ATA that supports UDMA-7 on PATA side. SATA side isn't important as even the lowest speed (rev.1 ) is 150MB/s which is almost as good as UDMA-7 157MB/s.

I found one here for 38 dollars:

http://www.cooldrives.com/saidecomisat1.html

Quick check of specs:
Serial ATA 1.0 Specification compliant
Automatic Serial ATA 1.5 Gbps speed negotiation
ATA / ATAPI PIO mode 0 to 4
ATA / ATAPI Ultra DMA of transfer rate 16.7, 25, 33, 48, 66, 100, 133, and 150MB/s.

It supports UDMA-7 on PATA-side!!!

Theoretically, only thing needed to make 5D III record to SATA drive would be female "faux CF"-to-PATA passive adapter. I don't know if such exist but I could make one by disassembling old CF card and soldering new lines. :D


Should we fork this into separate project in order not to fill up this thread?
Mods?

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3154
Re: uncompressed 14-bit RAW video recording
« Reply #1164 on: May 17, 2013, 03:01:00 PM »
Should we fork this into separate project in order not to fill up this thread?

good idea, just create a thread about that ;)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

peter_f

  • New to the forum
  • *
  • Posts: 2
Re: uncompressed 14-bit RAW video recording
« Reply #1165 on: May 17, 2013, 03:09:22 PM »
Just an update regarding the Komputerbay 128GB speed: I can record 1920x1152 w/o skipping frames at 88.5 MB/s. I did turn global draw and sound recording off though.

AnotherDave

  • Senior
  • ****
  • Posts: 348
Re: uncompressed 14-bit RAW video recording
« Reply #1166 on: May 17, 2013, 03:14:03 PM »
There are CF extenders that were designed to move CF ports to different places that might be of help!

Google - CF extender.

JohnBarlow

  • Freshman
  • **
  • Posts: 65
Re: uncompressed 14-bit RAW video recording
« Reply #1167 on: May 17, 2013, 03:55:17 PM »
To the devs  --->

Is it possible to bit shift or similar bit trix the raw stream to get to 12 bit instead of 14?

I am thinking this could be good for 550d/600d shooters with card controller limitations, giving an effective 4x speed increase

KMA_WWC

  • Freshman
  • **
  • Posts: 70
Re: uncompressed 14-bit RAW video recording
« Reply #1168 on: May 17, 2013, 04:01:34 PM »
Guys PLEASE STOP asking the same questions here. You are filling the thread with the same questions that were asked before!!! Please read the thread and you'll find the answers to all your questions. I understand its 47 pages, but some of the question been answered a page before, so at least check one page in advance or make a new thread!!! Thank you!
Canon 5D Mark III

bumkicho

  • Member
  • ***
  • Posts: 102
Re: uncompressed 14-bit RAW video recording
« Reply #1169 on: May 17, 2013, 04:03:34 PM »
To the devs  --->

Is it possible to bit shift or similar bit trix the raw stream to get to 12 bit instead of 14?

I am thinking this could be good for 550d/600d shooters with card controller limitations, giving an effective 4x speed increase

If you just read posts in a previous page...

JohnBarlow

  • Freshman
  • **
  • Posts: 65
Re: uncompressed 14-bit RAW video recording
« Reply #1170 on: May 17, 2013, 04:09:43 PM »
Ok my bad, must have missed it

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: uncompressed 14-bit RAW video recording
« Reply #1171 on: May 17, 2013, 04:27:29 PM »
EOS -M memory is too low.. only 50MB total. It will work at some point and give you some bursts. Right now the silent pic shooting is close enough.

AnotherDave

  • Senior
  • ****
  • Posts: 348
Re: uncompressed 14-bit RAW video recording
« Reply #1172 on: May 17, 2013, 04:32:38 PM »
I read yesterday that figuring out how to record past the 4gb file size limit "should be possible" but other things are a priority at the moment - like assigning a start/stop button for raw recording.

So.... (to the developers) which button were you thinking?  I'd like it to be the shutter release!  :-)  Can we vote?

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5938
Re: uncompressed 14-bit RAW video recording
« Reply #1173 on: May 17, 2013, 04:48:40 PM »
I read yesterday that figuring out how to record past the 4gb file size limit "should be possible" but other things are a priority at the moment - like assigning a start/stop button for raw recording.

So.... (to the developers) which button were you thinking?  I'd like it to be the shutter release!  :-)  Can we vote?

Actually, the latest build has a start/stop on the rec button. on my mark 3 that is

savale

  • Freshman
  • **
  • Posts: 69
Re: uncompressed 14-bit RAW video recording
« Reply #1174 on: May 17, 2013, 05:20:15 PM »
I looked into the code and do see some optimization (for cpu speed) possible, but it will cost a little mem. It might make room for possible new features but I don't know yet. Maybe this weekend I will look into how to setup a build environment so I can start developing. Is there a guide available how to setup a build env? Thnx