Messages

Quick test with this build for the 29th -- I'm seeing write speeds of 68-69MB/sec on the KomputerBay 64GB 1000x.  Very nice indeed.  And optional AF-button use for recording.  Fantastic!  Thanks for this.

The Live view image is still active i.e. not frozen/hack3d during record.  Has that feature been activated yet on the 5D2?

+1 on *not* using the LV button for recording.  Maybe I'm missing something, but this makes it impossible to turn off live view to save power when raw recording is enabled.  i.e. Hitting live view always triggers recording.  The Pravdomil build solves that problem elegantly with the joystick.

Between the joystick and the AF button, I'd prefer the AF button.  I have accidentally triggered raw recordings when simply using the joystick to check focus in 5x/10x modes.  The AF button would be a much more solid press with less room for error.

Yep, 5D2 -- sounds like you've figured it out, but yes, when in Live View just cycle through the "info" button until all the Canon graphics go away.  In my case this makes the difference between between short clips and unlimited filming at 1720x960.

Fyi I've been using RawTherapee, which is open source / free.  Similar controls/layout to ACR, and has produced nice results so far, and can also be batched via command line for the technically inclined.  "Auto Levels" is a setting right in the first pane, and it's a simple click to disable this feature to avoid frame to frame exposure changes.

On another note, now that I've had a full day to do some actual shooting with this module, I'm seeing serious frame skipping.  This is with the build from the 24th.  I haven't stepped through frame by frame, but it almost looks like one frame every second, at regular intervals.  I shot a batch of stuff yesterday which is totally unusable because of this.  Is anyone else noticing this?

I have frame skipping set to "off", so recording should simply stop if the card can't keep up.  However I'm still seeing solid write speeds in the 65-66MB/sec range, so that shouldn't be happening, and recording is definitely not stopping prematurely.  In fact, the buffer is pegged at one star most of the time at 1720x960, which is great.

One thing I'm noticing is that Live View does not disable during recording.  When not recording, the live view framing rect is an overlay but does not matte out the external portion of the screen.  When I hit record, the external portion is then matted out, but Live View continues to update inside the rect.  Is this the intended behavior?


Edit:  Never mind!  ffmpeg issue.  Frame rate must be specified before the other parameters, otherwise it assumes 25p and drops frames to fit whatever rate you specify downstream.

Carry on...

Super, thank you!

Terminal command is super simple: "cat raw1.RAW raw1.R00 > raw1-combined.RAW" (for example). raw2dng understands and decodes the output fine for me on OSX 10.8.3.

Huh.  You know what, I'm still using an older, command-line build of raw2dng and I bet that's the problem with the larger files.  I need command line because I have it running as part of a one-click script that batches a whole card to .MOV via RawTherapee, while offloading all the files to second drive.

Has anyone compiled a more recent version for OS X that handles 4GB+ files?

Thanks!

Pravdomil, this is fantastic work.  Fastest build yet for the 5D2, by a significant margin.  And file spanning to boot.

I just did 2000 frames at 1720x960, write speeds of 64-65MB/sec on a Lexar 32GB 1000x.  That's beyond the 4GB limit.  As of yesterday I was seeing 59-60MB/sec.  Again, bravo.

Couple questions:

1.  Does this incorporate the speed enhancements from today's (the 24th) build?  Because as of a few minutes ago, that was also the fastest build yet.  63-64MB/sec on my card.  If you haven't incorporated this yet, are we likely to see even further speed improvements when you do?

2.  I'm on Mac, so I can't use totalcmd.  I just tried cat (cat raw1 raw2 > raw3) and sed (sed '' raw1 raw2 > raw3), but this didn't work.  Raw2dng said "this ain't no lv_rec".  These two methods will typically both work on binary files.  How do the spanned clips need to be joined?

Regardless it is clear that whatever optimizations are present here should certainly be merged into the standard tree.  This has the potential to make previously marginal resolutions reliably usable, and that's big.

Thanks!


Edit:  65-66MB/sec on my Komputerbay 1000x.  Hot.

Man you're still outgunning me on these write speeds -- impressive.

But I just made head-smacking discovery.  Disabling all stock Canon screen overlays via the "info" button gives me a boost from 59-60MB/sec to a solid 63-64MB/sec with the Komputerbay 64GB.  I can now hit the 4GB limit on today's build at 1720x968.  Amazing.  People, absolutely disable both ML global draw *and* all Canon overlays for max write speed!

Note that my Lexar 32Gb 1000x still doesn't max out at 1720x968.  It's always a touch slower than the Komputerbay.

Regardless I have to wonder what other simple fixes I may be overlooking that may account for the write speed differences I'm seeing from people with the same exact card.

Ok silly question -- how do I keep the LCD from shutting off while filming longer clips?  Not that I really *can* film long clips just yet, but you know, I'm an optimist.

Edit:  Never mind, *both* Canon and ML power saving options need to be switched off.  Carry on...

Hey, cool, thanks for the upload!  I didn't even realize you could save settings.  I see it in the menu now.  Anyway, no change in my performance.  I can only assume there is great variability in the cards themselves and that is what accounts for the different results people are seeing.  Who knows!  I think there is more to come in terms of performance enhancement though.  Thanks again to the devs...

1300 frames is really good!  I'm also on the Komputerbay 64GB 1000x.  It would be perfect if I could do 1300 frames, but the longest clip I have seen at 1720x960 was around 850 frames, 61MB/sec.  A "bad" shot will be more like 500 frames.  What write speed are you seeing at 1720x960?

Re: 1066x cards, you would have to get one and test it.  Even though we are operating well below the maximum rate of these cards on the 5D2, there are still differences between them.  My Lexar 32GB is also 1000x but the Komputerbay is 1-2 MB/sec faster.  This is enough for a few hundred frames at 1720x960.  It is possible that a few MB/sec could make the difference between limited and unlimited recording for a particular resolution.

Alternatively, when you need a more standard 16:9 aspect ratio, you can try 1720x 960.  This looks far better than the stock H264 and only requires minimal scaling to get to 1080.  I'm seeing anywhere from 500 to 800 frames at this size, 61MB/sec.  It sounds like you might do a little better.  Let us know how many frames you get if you test it.

If you can record indefinitely at 1880 x 840 on the Mark II you're doing about as well as can be done.  My Komputerbay 64GB just arrived today and it's definitely faster than my Lexar 32GB 1000x.  But I'm not getting unlimited at 840.  So far I'm seeing max write speeds of about 61/sec, and that's on a good take.  This translates to about 1100-1300 frames at 840.  On the Lexar, sustained 60/sec was the best case, and most often it was high 50s.  Frequently I would not break 1000 frames at 840, so the Komputerbay is an improvement.  This is all with global draw and audio off.

5D2 with both Lexar and Komputerbay 1000x cards and happy to test.  However my gcc-fu is a little rusty and I'd be better off trusting you to upload a functioning module.  If you can do that, count me (and probably a bunch of others) in.

Yeah 64GB 1000x Komputerbay bought from:

http://www.amazon.co.uk/gp/browse.html?ie=UTF8&marketplaceID=A1F83G8C2ARO7P&me=A15KH8GEK4WQL1

I was thinking of getting a 32GB Lexar, but if this is faster as you say then I'll just grab more of these.

As I understand it from elsewhere on the forum, Komputerbay is just a supplier to Lexar and the cards are otherwise identical.  So I don't know what accounts for the difference in write speeds.  But if you're already getting A-list write speeds, then if it ain't broke, don't fix it!

20 May build, 5d2, 1720x960, Lens cap on, GD off, sound off, hacked off:

1st clip: Buffer had just reached 4 stars, then I reached the 4GB file limit, so not perpetual recording at that resolution for the 64GB Komputerbay in a 5d2.

2nd clip: 1242 frames then dropped one, buffer filled quicker this time.

3rd clip: Buffer coasting at 3 stars at limit.

4th clip: 5 stars flashing at limit.

5th clip: 3 stars flashing at limit.

67MB/s was the peak speed, but averaged 65.8MB/s to 66.2MB/s.

Will do more tests with hacked and complex images

Wow, impressive!  That's with the 64GB Komputerbay 1000x?  Not sure why I'm not getting these speeds with the Lexar 1000x, but it may be time to pick up a Komputerbay.

1880x840 is great but if you need to fill a standard HD frame, try 1720x960.  When blown up to 1920x1080 it still looks worlds better than H264.  I'm getting about 450-600 frames at this resolution on the 5D2, at a write speed of about 59MB/sec on a Lexar 1000x 32GB.  An improvement to 64-65MB/sec would probably enable unlimited recording at this resolution.  I'm optimistic that Hack3d mode or other write speed improvements will get us there.  Is anyone seeing reliable 65MB/sec speeds on any Komputerbay cards?

Given the relatively brief recording times we're getting here on the 5d2 for anything approaching 1080p, it would be great if we could enable an optional 10-second pre-record delay in the menus.  Just like the 10-second delay mode for stills.  For those of us doing solo shoots, narrative, video blogging, or anything else where the shooter also needs to get into the shot, this would allow enough time to do so.  Ideally, this would trigger the audio sync "beep" when recording actually starts, as a kind of 'on air' notification.

In the earlier builds, the "allocating" phase that appeared right after starting record actually functioned that way.  I'm getting more frames now with the later builds, but I'm wasting them running into the shot and getting set up.

To be clear, this would be an optional item, in the raw menu, settable for when it's useful.

Thoughts?

Set 2048 as start sector.

Ok, gave that a shot.  That seemed to drop the write speeds a few MB/s.  I was around 59-60 MB/s before the 2048 setting, now I'm at 57-58 MB/s.  Reformatting the card in the camera restored the original partition settings and the write speeds are now about 59-60 again.

So, couple questions:

One, how exactly is the correct start sector on a FAT32 volume calculated, and are we sure 2048 is correct?

Two, is there any reason CF card performance would degrade over time from recording data at rates like these?

Thanks!

Hi guys.

Interested in trying partition alignment on my Lexar 1000x 32GB.  If this results in repeatable speed boost across the board this may be worthy of a thread in its own right with clear instructions for Win/Mac.  For the mean time...

I'm on Mac OS X Mountain Lion.  Here's what gdisk reports about the Lexar:

--

Disk /dev/disk2: 62521200 sectors, 29.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 401BD3D8-8D0B-494B-A00D-87E8368CE132
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 62521166
Partitions will be aligned on 1-sector boundaries
Total free space is 1004 sectors (502.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              63        62520191   29.8 GiB    0700  Microsoft basic data

--

I believe I can use iPartition to adjust the start sector of the card.  Ideally what sector should it be?  Or is the card currently aligned?

Thanks!

I just tested the May 17th build and got 518 frames @ 960. So, a bit of a step down, but not too bad.

That's pretty solid.  Were you able to unwrap the frames?  518 usable frames beats 0. 

That is the correct build. Somewhere between 62-67MB/s

Ok tested the "evening 16th" build.  I'm definitely not seeing your write speeds -- still in the high 50MB/s, occasionally 60MB/s.  But I can confirm the files can't be unpacked with raw2dng.  So... back to the 17th build.

Are you still getting 500+ frames @ 960 on the builds where you can unwrap the files?  Which builds?  If so, I wonder where the speed is coming from.

I just tested again with the May 16th evening build. Got 608 frames at 960, and 262 at 1080, with Komputerbay 32gb 1000x.

Ok that's impressive.  If I could get 608 frames @ 960, I'd be golden.  Is that the "e1d271904e4c" build from post #83 on this thread?  What kind of write speeds are you seeing on the LCD during capture?

@Hazer:
It looks like the DNGs from raw2dng do not have previews. Adobe DNG Converter adds previews to the filez.

Actually, I have that feature turned off, along with all the other options.  i.e. I have JPEG previews, embedded fast load data, lossy compression, and embedded original all turned off.  I have "preserve pixel count" turned on.  Hit convert.

Again, not sure what changes if any are happening to the original image data, but the resulting image *looks* the same and appears to respond the same to highlight recovery and so forth.  Further investigation is probably necessary.