uncompressed 14-bit RAW video recording

Started by g3gg0, April 27, 2013, 12:07:12 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

ashtrai

Quote from: mattmvpmedia on May 15, 2013, 09:13:01 PM
I'm using a SanDisk Extreme Pro 95MB/s 16GB card on my 600D (T3i). When shooting 960x540 raw I get a few dropped frames but not bad. I shot a comparison test of a scene with 1. Normal Canon 1080p h.264 quicktime movie settings and then 2. Magic Lantern cropped sensor 960x540 raw. I used raw2dng on Mac and then After Effects to process the dng files for the 960x540 and exported a ProResHQ movie.

There is currently too much tearing and blocking of magenta blocks for the raw footage to be usable, but the detail in the frames that don't have the tearing is an incredible improvement over the h.264 quicktime movie. Even when I blow up the 960x540 ProRes to 1920x1080 and compare it with the standard Canon 1920x1080 h.264 file the footage shot 960x540 raw blows the h.264 away in terms of detail and dynamic range. Love it.

I'm definitely looking to get a 5D mark iii now, but if the tearing and blocking can be fixed on the 600D it is a huge improvement to what it currently shoots in h.264, even when shooting 960x540 raw. Fantastic job by the ML team!

Can you post the ProRes sample on mega?

payne

Quote from: mattmvpmedia on May 15, 2013, 09:13:01 PM
I'm using a SanDisk Extreme Pro 95MB/s 16GB card on my 600D (T3i). When shooting 960x540 raw I get a few dropped frames but not bad. I shot a comparison test of a scene with 1. Normal Canon 1080p h.264 quicktime movie settings and then 2. Magic Lantern cropped sensor 960x540 raw. I used raw2dng on Mac and then After Effects to process the dng files for the 960x540 and exported a ProResHQ movie.

There is currently too much tearing and blocking of magenta blocks for the raw footage to be usable, but the detail in the frames that don't have the tearing is an incredible improvement over the h.264 quicktime movie. Even when I blow up the 960x540 ProRes to 1920x1080 and compare it with the standard Canon 1920x1080 h.264 file the footage shot 960x540 raw blows the h.264 away in terms of detail and dynamic range. Love it.

I'm definitely looking to get a 5D mark iii now, but if the tearing and blocking can be fixed on the 600D it is a huge improvement to what it currently shoots in h.264, even when shooting 960x540 raw. Fantastic job by the ML team!

Do we know if this build will run on the 60D as well or are the 600D and 60D (although very very similar) not similar enough on the inside?

sicetime

is there a place that has compiled all different cards against the resolution they are capable of reaching (with out dropped frames)?

tonybeccar

Has anyone tried with the Toshiba 1066x Excercia Pro compact flash cards?? They're supposed to be 30 MB/s faster! They say they're prepared to record 4k video...

mattmvpmedia

Here's the standard Canon h.264 1080 24p file:


Here's the 960x540 raw file (shot with ML raw, converted to DNG files, processed with Adobe AE, exported to ProResHQ).


mattmvpmedia

Sorry about that, forgot to remove the "s" from https -

Here's the standard Canon h.264 1080 24p file:


Here's the 960x540 raw file (shot with ML raw, converted to DNG files, processed with Adobe AE, exported to ProResHQ).

Vegandelight

Quote from: tonybeccar on May 15, 2013, 09:47:53 PM
Has anyone tried with the Toshiba 1066x Excercia Pro compact flash cards?? They're supposed to be 30 MB/s faster! They say they're prepared to record 4k video...

I don't think you'll be able to break 100mb/s with that card either, it's hardware related limit at the moment

Habitat

Sell your 600D's and save up for Mark III's - the whole thing is jarring lol

1%

Try some different resolutions and modes.. You seem to be getting more dropped frames than I am. But this middle is gone dealie is happening on 5d2 as well sometimes. Simply it has to mature a bit more...

ajay

Can someone please set me straight....I have been trying to get 1:1 cropping while recording RAW and I have scoured everywhere trying to figure out how to do this so I don't have to bug people, but I cannot for the life of me figure out how to record RAW in anything other than full-frame. What do I need to do to crop/pan while recording?

Sorry to be a bother...

AJ

Dash

4k. But not in RAW mode.

Quote from: tonybeccar on May 15, 2013, 09:47:53 PM
Has anyone tried with the Toshiba 1066x Excercia Pro compact flash cards?? They're supposed to be 30 MB/s faster! They say they're prepared to record 4k video...

marcb

Rob Galbraith has posted a 57.5MB/s write speed for the Canon 7D with FW 1.2.5 and the Lexar Professional 1000X 32GB CF. I'm curious to see what kind of RAW video capabilities the 7D could offer with Magic Lantern. Has anyone achieved higher write speeds on the 7D? http://www.robgalbraith.com/bins/camera_wb_multi_page.asp?cid=6007-10294

1%

7d probably == 5d2


But g3ggo said the raw image is on the other proc so harder to get at.. which is why its not done.

jpgentry

Hi Guys.  A few observations if you can confirm them:

It seems that on all cameras/card combinations frame skipping is an issue including the 5d3.  I have the 5d3 with 128GB 1000x KomputerBay CF card.  I start skipping frames about 10 seconds in at 1920x1080.  I also skip frames at: 1920x960, 1920x840.  I do not skip frames at 1920x720.  I'm still wondering how others are reporting that they are recording 1920x1080 video on the 5d3.  I'm thinking the announcements around the internet are premature about 1080p RAW on the 5d3 being a reality.

The encoding of RAW into a finalized usable clip is very slow and tedious.  While the workflow may improve, I don't see the encoding time improving drastically when working with large amounts of footage.

Is the development process showing signs of cracking these issues?  Are we even sure that we're not bumping into limitations of the camera controller on the 5d3 causing frame skipping?  I guessuntil someone gets the 1066 card we will not know...

I've been holding back on the c100 until I hear where this is going, but this does not yet seem quite the holy grail that Cinema5d and nofilmschool are putting out on the street... 

Danne

Quote from: jpgentry on May 15, 2013, 10:42:40 PM
Hi Guys.  A few observations if you can confirm them:

It seems that on all cameras/card combinations frame skipping is an issue including the 5d3.  I have the 5d3 with 128GB 1000x KomputerBay CF card.  I start skipping frames about 10 seconds in at 1920x1080.  I also skip frames at: 1920x960, 1920x840.  I do not skip frames at 1920x720.  I'm still wondering how others are reporting that they are recording 1920x1080 video on the 5d3.  I'm thinking the announcements around the internet are premature about 1080p RAW on the 5d3 being a reality.

The encoding of RAW into a finalized usable clip is very slow and tedious.  While the workflow may improve, I don't see the encoding time improving drastically when working with large amounts of footage.

Is the development process showing signs of cracking these issues?  Are we even sure that we're not bumping into limitations of the camera controller on the 5d3 causing frame skipping?  I guessuntil someone gets the 1066 card we will not know...

I've been holding back on the c100 until I hear where this is going, but this does not yet seem quite the holy grail that Cinema5d and nofilmschool are putting out on the street...

Have you tried with an older firmware?

aaphotog

Quote from: jpgentry on May 15, 2013, 10:42:40 PM
Hi Guys.  A few observations if you can confirm them:

It seems that on all cameras/card combinations frame skipping is an issue including the 5d3.  I have the 5d3 with 128GB 1000x KomputerBay CF card.  I start skipping frames about 10 seconds in at 1920x1080.  I also skip frames at: 1920x960, 1920x840.  I do not skip frames at 1920x720.  I'm still wondering how others are reporting that they are recording 1920x1080 video on the 5d3.  I'm thinking the announcements around the internet are premature about 1080p RAW on the 5d3 being a reality.

The encoding of RAW into a finalized usable clip is very slow and tedious.  While the workflow may improve, I don't see the encoding time improving drastically when working with large amounts of footage.

Is the development process showing signs of cracking these issues?  Are we even sure that we're not bumping into limitations of the camera controller on the 5d3 causing frame skipping?  I guessuntil someone gets the 1066 card we will not know...

I've been holding back on the c100 until I hear where this is going, but this does not yet seem quite the holy grail that Cinema5d and nofilmschool are putting out on the street...
Your card is too slow. Thats a cheaper card which really doesnt have the write speed that the Lexar(I believe) has

pravdomil

Today I have been playing with raw_rec and there is result.

I can send you module file, if you wish (no warranty). You don't want to see source.

But I cannot compute seconds dividing frames by fps. Time goes faster. Any ideas?
Nice work!

Andy600

@mattmvpmedia - Thanks for posting the comparisons. To my eyes there is hardly anything in it. I think anyone criticizing the hardware limitations of these lower-end cameras should remember where we were a week ago. We (600d users) now have massively increased post production flexibility and virtually no loss in quality. I'll take that anyday! :)
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

payne

Quote from: Danne on May 15, 2013, 10:52:12 PM
Have you tried with an older firmware?

I second that question. I had heard that the older firmware Andrew is using allows 1920x1280.

Vegandelight

I find it really hard to know what im actually shooting at 5x and 10x. The frame isnt correct i guess cause im not getting what i think i am ;)

Nspa32

Can someone please let me know if we can get this working on the 600D would it also work on the 550D? I mean, why wouldn't it? Aren't the internals identical?

mixmastermike

Quote from: aaphotog on May 15, 2013, 11:03:25 PM
Your card is too slow. Thats a cheaper card which really doesnt have the write speed that the Lexar(I believe) has

Yeah i think its becoming evident that the "cheapo" Komputerbay cards are not giving write speeds the same as Lexar.

I believe in trying to get the best deal out there but your card is where you store your work, without it you have nothing. I don't want to even have the worry of a card going down thats why Im willing to pay a little more fore the Lexar.

Danne

Quote from: mixmastermike on May 15, 2013, 11:56:14 PM
Yeah i think its becoming evident that the "cheapo" Komputerbay cards are not giving write speeds the same as Lexar.

I believe in trying to get the best deal out there but your card is where you store your work, without it you have nothing. I don't want to even have the worry of a card going down thats why Im willing to pay a little more fore the Lexar.

I,m not so sure about that. Unless some more tests we can,t exclude that it could be firmware related.

Vegandelight

Quote from: Nspa32 on May 15, 2013, 11:51:51 PM
Can someone please let me know if we can get this working on the 600D would it also work on the 550D? I mean, why wouldn't it? Aren't the internals identical?

It's been said it probably wont work. Read a few pages back.

platu

Quote from: jpgentry on May 15, 2013, 10:42:40 PM
Hi Guys.  A few observations if you can confirm them:

It seems that on all cameras/card combinations frame skipping is an issue including the 5d3.  I have the 5d3 with 128GB 1000x KomputerBay CF card.  I start skipping frames about 10 seconds in at 1920x1080.  I also skip frames at: 1920x960, 1920x840.  I do not skip frames at 1920x720.  I'm still wondering how others are reporting that they are recording 1920x1080 video on the 5d3.  I'm thinking the announcements around the internet are premature about 1080p RAW on the 5d3 being a reality.

The encoding of RAW into a finalized usable clip is very slow and tedious.  While the workflow may improve, I don't see the encoding time improving drastically when working with large amounts of footage.

Is the development process showing signs of cracking these issues?  Are we even sure that we're not bumping into limitations of the camera controller on the 5d3 causing frame skipping?  I guessuntil someone gets the 1066 card we will not know...

I've been holding back on the c100 until I hear where this is going, but this does not yet seem quite the holy grail that Cinema5d and nofilmschool are putting out on the street...


I just ordered and received the same CF card as you... the KomputerBay 128gb 1000x CF card and have been running tests with each of the various Raw builds over the last 3 days.  The latest build for 5D3 someone just posted from today seems to be the best in terms of speed... see http://www.magiclantern.fm/forum/index.php?topic=5413.25 (Reply #30).  This includes some small memory tweaks that definitely helped me get a little bit longer runtimes as well as moved me up to the next resolution without dropped frames.

Here has been my experience with the KomputerBay 128 GB 1000x CF card...

For my tests, I tried Global draw turned ON and OFF and there was no significant difference except maybe a second or two extra of recording time but the extra time was not consistent.  So I decided to leave Global Draw = ON (with just peaking and crop marks enabled).


My tests using the ML benchmark utility shows read speeds that range anywhere from 52MB/s to 72MB/s.  Using CrystalDiskMark and ATTO Disk Benchmark, I get similar write speeds so my particular card seems to have a write speed that varies a bit.  I would say the average write speed that I get most of the time is about 65MB/s which is completely in line with the ML benchmarks and the listed speed needed by each resolution listed under the RAW video section of ML.  So I would trust the ML benchmarks and recommended write speeds for each resolution.  I have also confirmed that at least one other person is getting similar write speed using this card.

That said, the buffer/memory tweak Alex made today did have a significant impact when using my card. 

Here are my tests Before today's build...
1920 x 720 or less resolution (no skipped frames for complete 4gb file)
1920 x 840 (no skipped frames for complete 4 gb file)... before today's build, this was the best I could do.
1920 x 900 (no skipped frames for complete 4 gb file)... tweaks added to today's build allowed me to use resolution for first time
1920 x 960 (maybe 10 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (maybe 1-2 seconds before frame skipping begins... unusabe beyond that)

After using today's build...
1920 x 900 (no skipped frames for complete 4 gb file) before today's build, I could not use this resolution
1920 x 960 (went from 10 seconds to 20 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (went from 1-2 seconds to 5 seconds before frame skipping begins... unusabe beyond that)

So the tweaks made by Alex today have definitely helped so I encourage further memory/buffer optimizations... they are making a difference.

I don't know how EOSHD, Nuemann Films, and Cinema 5D are getting 1080P and greater without dropped frames. They say they are using the same card.  Maybe some of these cards are getting faster write times.  Or maybe they are just getting that resolution for a limited time but a bit longer than me before frame skipping appears.  If any of them can chime in here to clarify, it would helpful for those considering the Komputerbay cards.  This card is by far the most realistic in terms of pricing for the vast majority of people who want to take advantage of this RAW update and plan on using it beyond test videos and personal work.  None of the other cards comes close in terms of price/gb.  The other solutions by Lexar and Toshiba (soon) are more like $650 for 128gb vs $178 or KomputerBay. I'm sure this will eventually change, but that could take a year or longer before cheaper alternatives are available.  But there is no getting around the fact that the less than ideal write speed of these cards may prevent continuous 1080p for some (depending on their particular card?).  But continued memory optimization and other techniques by the ML team may be able to extend 1080p recording from 5 seconds to 30 seconds or possibly more when using this card.  If that can be attained, the vast majority of folks will be able use this resolution in many shooting scenarios, short of documentary or event work.  I do realize that smaller resolutions can be scaled up in post nicely, but I tend to avoid that as do many others I'm sure.

Lastly, on the topic of workflow mentioned above... it's completely worth the time as far as I'm concerned.  The difference is night and day.  There are a lot of test videos coming out now, some of which don't fully capture the quality improvement gained here.  EOSHD, Nuemann Films, and Cinema 5D did great job of showing what's possible.  Just wait until some shorts and features by other skillful DOPs start to appear online.  While I don't love the added work required in post, I find it impossible to go back to H.264 after getting used to the look of my footage now...it's improved that much.

Thank you Alex and team for this... also to g3gg0 who apparently had a huge role in making this particular breakthrough happen.