RAW resolutions on 5D II

[airthug]

I'm sorry to start *another* thread on what has been covered all over this board (and yes, I've been searching and reading like crazy, for days!)

AS OF TODAY- MARCH 26 2016- What's the closest to 16:9 1080 HD resolution that can be continuously recorded in RAW?

I've seen reference to different (better?) results using older ML builds?  Not sure I can wrap my head around why the latest isn't the most capable?

AND!  The updated Google Doc of camera capabilities- https://docs.google.com/spreadsheets/d/16cgnRivbUv7nA9PUlCLmLdir3gXdIN3pqzCNAAybepc/edit#gid=5 says the 5D II (yes, 5D 2!) can record "1856x1004, continuous"

I can't get this to work, which seems obvious because isn't the data rate for that res around 77 Mb/sec?... which is faster than the Mk II can write to a card, regardless of card speed.

Also, if the write-speed is the limitation, I'm seeing a lot of suggestions to turn off the globe draw overlays during recording... but if that improves performance, wouldn't that imply the PROCESSOR is the bottleneck?  Or, in other words, if the hardware for writing the the card is limited at 75ish Mb/sec ish? then why would it write faster or slower with/without globe draw?

If you guys are busy, don't sweat the more complex questions/explanations... but any help getting as close as possible to 16:9/1080 24fps RAW out of my Mark II would be much, much, MUCH appreciated.  It's eaten up a lot of time reading all the various threads here, and then trying to make sense of all the different info I'm seeing... some from 2013, some from 2015, some more recent...

Please help a 2016 Mk II RAW-wannabe out??

-Ryan

[reddeercity]

You need to read the raw video thread more , this has been ask & answered  many times .
http://www.magiclantern.fm/forum/index.php?topic=16650.msg162823#msg162823
minimum CF card for 5d2 1000x or faster , I use Lexar 1000x & 1066x .
and yes to need to disable GD , this should happen by default for 5d2 .

[airthug]

Thanks for the reply.  That link was exactly what I was looking for!  I admit I intentionally skipped over the MLV-Lite thread... wasn't really interested in a "lite" version of MLV... but I see now, 1856x1004 continuous on 5D Mk II is ONLY with MLV-Lite or the old .raw format... which explains why I wasn't able to do it.  And now I see the Light 

For anyone else that comes across this thread... RE:MLV-lite

Differences from "full" MLV:
    1. Metadata is only for what the settings were when recording was started (e.g. you get expo metadata, but if you change exposure, you don't get that new information).
    2. No audio
    3. No card spanning (recording to both CF and SD card)


Differences from RAW v1.0 is that some of the testers are finding that the older format still has a slight edge in performance when you're really pushing the envelope at the expense of almost no metadata to work with in post production.

In practice the difference are more significant. MLV is a big improvement over v1.0 RAW in terms of the amount of metadata saved in the files that can be useful in post production and even more significant, being able to record synced audio. However, there are issues with "pink frames" and other problems that have come up on occasion so some users have reverted to the old RAW format. MLV Lite is designed to ween users off of the old RAW format by giving "The sweet taste of MLV with none of the extra calories"

I'll give that a crack and see how it goes.

By the way- I freaking love Magic Lantern, so THANK YOU to everyone that has worked on this over the years.  Truly an amazing feat, creating such a rich feature set and adding capabilities to existing hardware.  RAW video vs H.264 is absurdly better (obviously).  Anyway- I just wanted to share my great appreciated for ML and all the time and work people have put in to it... thank you!!!!!!

[airthug]

reddeercity,

After some trying it out, I'm still getting stops/dropped frames.  I'm using a Sandisk Extreme card, 120 Mb/sec... so pretty sure it's not the card. Feb 2016 build.  5D Mk II

In the MLV-Lite thread, you said:

I did a more comprehensive test with MLV Lite module on my 5D2 with the updated raw_rec.mo
I use the latest Nightly Build plus I compared mlv lite to full mlv ( use 2 different nighty for full MLV , feb15/2014 &Feb13/2016) and used the Old Raw format for Oct24/2013.
All tests done on CF Lexar 32GB 1066x
MLV Lite:
1856x1004 @ 23.976 with GD (global draw)  enabled  - 1134 Frames
1856x1004 @ 23.976 with GD disabled - continuous until full (74.5MB/s write speed)
1856x1004 @ 23.976 with GD disabled + HDMI enabled (Ninja HDMI Hard Drive recorder connected) - 3215 Frames
1856x1044 @ 23.976 With GD enabled - 1089 Frames
1856x1044 @ 23.976 With GD disabled - 1662 Frames

Full MLV (Feb13/2016 Nightly build):
1856x1004 @ 23.976 + audio With GD enabled fill rate "0" - 1317 Frames
1856x928   @ 23.976 + audio With GD enabled fill rate "0" + HDMI enabled (Ninja HDMI Hard Drive recorder connected) - 2776 Frames
1856x928   @ 23.976 + audio With GD enabled fill rate "0" - continuous until full

Full MLV (Feb15/2014 Nightly build) : This is build I use 99% of the time for work ( Yea multiple of 16) Work better on 5D mark 2
1856x928 @ 23.976 fill rate "0"  + audio  -  continuous until full
1856x928 @ 23.976 fill rate "0"  + audio (GD disables automatically on recording raw video) + HDMI enabled (Ninja HDMI Hard Drive recorder connected) -  continuous until full
1856x1004 @ 23.976 fill rate "0" + audio  -  1788 Frames
1856x1044 @ 23.976 fill rate"0" + audio  -  748 Frames
1872x936  @ 23.976 fill rate "0" + audio -    continuous until full
1872x936  @ 23.976 + audio fill rate "0" + HDMI enabled (Ninja HDMI Hard Drive recorder connected) - continuous until full

Raw Format .raw (Old Raw) **Note** GD is automatically disabled while Raw video records
1856x1004 @23.976 + HDMI enabled (Ninja HDMI Hard Drive recorder connected) (74.6MB/s Write speed ) - continuous until full
1856x1044 @ 23.976  (77.5MB/s Write speed) -  continuous until full
1872x1012 @ 23.976 (76 MB/s write Speed)   -  continuous until full
1872x1054 @ 23.976 (78MB/s write speed)   -  2492 Frames
1872x1248 @ 23.976 (94MB/s write speed)  -  265 Frames
1880x1016 @ 23.976 ( 76.6MB/s write speed) -  continuous until full
1880x1058 @ 23.976 (78.4 MB/s write speed) - 2300 Frames
1880x1250 @ 23.976 ( 95MB/s write speed)   -  246 Frames

Google Doc for max continuous 16:9 for 5D II says 1856x1004... which I just realized isn't even 16:9?!  So that must be a mistake I guess...

Using the old RAW (1.0), interestingly enough, Globe Draw does NOT automatically turn off!  I can record (for about 1200 frames) at 1856x1044 WITH the globe draw zebras, waveform, etc.  When I turn GD off, I can record just a bit over 3700 frames, before recording stops at 4 Gb.  Maybe that's considered continuous for RAW 1.0?

Using new (full) MLV RAW... I'm actually getting continuous at 1760x990 (staying with 16:9)... WITH MLV sound turned on, too!  Pretty stoked with that, hard to imagine the 5DII doing much better than that, right?

You do say you are using ML Nightly Build from Feb 15/2014.  Given what I'm getting with the current build, would you still recommend trying that older one out... do you think it could get me any better than I'm already getting?

I really appreciate the help!  The knowledge base among users here is incredible... which is good, because some of this is a bit complex (IMO anyway), and searching and navigating through pages upon pages of posts... some quite old... is kind of a nightmare.  So again, thanks for responding and helping me out!  There are probably some other 5D II users out there this helps, too...

[reddeercity]

reddeercity,
After some trying it out, I'm still getting stops/dropped frames. 
I'm using a Sandisk Extreme card, 120 Mb/sec... so pretty sure it's not the card. Feb 2016 build.  5D Mk II

https://www.sandisk.com/home/memory-cards/compact-flash/extreme-compactflash

2. 16GB: up to 120MB/s read speed; up to 60MB/s write speed
32GB-128GB: up to 120MB/s read speed; up to 85MB/s write speed

Minimum CF card for 5d2 1000x or faster , I use Lexar 1000x & 1066x .

http://www.lexar.com/products/lexar-professional-1066x-compactflash-card

64GB—up to 160MB/s read, 155MB/s write

http://www.lexar.com/support/products/5163

The Lexar® Professional 1000x CompactFlash card leverages UDMA 7 technology to provide industry-leading performance with 1000x (150MB/s) minimum guaranteed sustained transfer speed.

Do a CF card benchmark in Camera ,
http://www.magiclantern.fm/forum/index.php?topic=12630.msg161468#msg161468
http://www.magiclantern.fm/forum/index.php?topic=14927.msg144901#msg144901

My 155MB/s write speed CF card can only maintain 72MB/s Raw Video (1872x936 or 1856x928 @ 23.976p Plus Audio W/HDMI enable) until full
Or 2048x872 + Audio @ 23.976p in 3xCrop Mode which Takes about  75-77 MB/s

[airthug]

Well shit 

I might not totally be hosed, though... because I misspoke myself, it's a Sandisk Extreme PRO / "160 Mb/s"
https://www.sandisk.com/home/memory-cards/compact-flash/extremepro-compactflash

SanDisk Extreme PRO CompactFlash Memory Card delivers a minimum sustained write speed of 65MB/s with an industry-leading shot speed of up to 150MB/s for 16GB - 128GB cards and up to 140MB/s for the 256GB card. And, with transfer speeds of up to 160MB/s, it makes it easy to move even large files for storage and editing. This memory card is also UDMA 7 enabled.

Here are two benchmarks- First with old .RAW rec turned on, and then with MLV Raw + MLV Sound turned on (and raw rec turned off, obvi)
http://www.flickr.com/photos/airthug/26057638346/in/dateposted-public/
Raw module benchmark by airthug, on Flickr
Mlv module benchmark by airthug, on Flickr

[Walter Schulz]

You have to run both tests in the same mode.
If you compare PLAY-PH LCD (using RAW_REC.mo) and MV-1080 LCD (using MLV_REC.mo) you are not comparing MLV_REC and RAW_REC but the modes itself.
PLAY will always show the higher number because in this mode almost all ML tasks are surpressed. It's telling you what the card is able to do without most of ML's overhead.
MV-1080 number is closer to what you can expect doing video recording. Benchmark runs with other tasks active.

[reddeercity]

@airthug the second image you posted is more realistic , that closer to the norm .
You will not get 16x9 or 1.85:1 at full resolution without frame drop on the New MLV full
MLV Lite or new builds with the old .raw format period.

If you need continuous recording @ those A.R. then choose smaller frame size or use 2:1
Which I do,  or use the very old builds of .raw format. (Not Audio or metadata in .raw)
You could try 3xCrop mode , you should get 1.85:1 A.R. as there a little more write speed there.

If you still have drop frame , either the card is bad or you have camera setting wrong or not optimized for max write speed.

My 5D2 Camera setting 
1872x938 23.976p + audio
Global Draw Off when recording
Preview Mode - Hack preview
Extra Hacks - On
Frame Over Ride = On (Exact 23.976)
Buffer fill rate - 0 or 1
In the Canon menu use Large Raw in Photo Mode &  1080 24p in Video mode

**Note** I use a 18 month old built that let me record at 1872 , New builds are limited to 1856.

[airthug]

Thank you very much, I think I have a decent handle on what I'm getting vs could/can't get...

Based on your settings above, I guess I'm getting all I can expect from my 5D II:
-Using the old RAW 1.0 I can record to exactly 4Gb at 1856x1044, GD off, but with preview on (4Gb = just a bit over 3700 frames).  Am I correct in my observation that RAW 1.0 stops at 4Gb rather than splitting into a new file?  or is it just that the camera is at it's limit and can't handle wrapping one file and starting another?

Using (full) MLV RAW... I'm actually getting continuous at 1760x990 (staying with 16:9)... GD off, but preview AND MLV SOUND on.  I'm pretty stoked with that, as it's a very usable res to scale up to 1080.

What got me confused and starting this thread was the RAW capabilities Google Doc says max continuous 16:9 for 5D II is 1856x1004- which isn't 16:9?  Maybe 1004 is a typo, and it should be 1044 like I'm successfully getting (till 4Gb) using RAW 1.0.... which I didn't think to try when I read the doc.  I thought other people were getting 1856x10** MLV RAW, maybe even with MLV Sound... which struck me as impossible, but if someone figured it out I wanted to tap my 5DII's full potential  (isn't that what Magic Lantern is all about? ;-)

Anyway- does RAW 1.0 always stop at 4Gb?

THANK YOU FOR THE HELP!!!!!!!!

[DeafEyeJedi]

Re: 4Gb ... Maybe you forgot to exFAT your cards? My wrong. Thanks @reddeercity ... Check below!

[reddeercity]

Re: 4Gb ... Maybe you forgot to exFAT your cards?

5D2 is only Fat32.

@airthug Yes spanning files works past 4GB you have just pick a very large frames that can not be recorded continuously .
If you what to record with Old Raw @ 16x9 & if you read my post it says I used Oct.24th, 2013 build not the latest nightly.

used the Old Raw format for Oct24/2013

1856x1044 @ 23.976  (77.5MB/s Write speed) -  continuous until full
1880x1016 @ 23.976 ( 76.6MB/s write speed) -  continuous until full

By full I mean no drop frames on a 64GB card.
FYI: Old Raw that I use from time to time but is becoming obsolete

[DeafEyeJedi]

@reddeercity -- excuse me as I was typing this my gut feeling was telling me that you were gonna correct me from wrong. Thanks for the friendly reminder once again and it's been quite awhile since I owned this even tho I could have dreamed of shooting it in exFAT.

[airthug]

Thank you, thank you, and THANK YOU!!!
I REALLY appreciate your help (and patience).

Cheers!

[reddeercity]

@ DeafEyeJedi no worries !
All is good