Messages

If you have a super fast card I recommend setting CBR to 20x and Flush to 1.  Then enable spotmeter and select the 0-255 option.  Then set ISO to 128,000 and use your shutter or ND filters to underexpose to "25" on the 0-255 spotmeter.  This always shoots my bitrate up to it's max but my cards, which are "200x" 170Mbps (21MB/s)*, peak at 200 Mbps for 1 second then the buffer can't keep up** so the recording stops :/

*ML Benchmarked my card at 21.5MB/s
**my Flush rate is set to 2 (my camera throws err70 if flush=1)

What is the significance of setting the flush rate so much lower? Is it useful so that when we record at a higher bitrate we won't overflow the buffer? If that's all, I would keep the flush rate at 2-4. I didn't have a single issue in my dozen or so test recordings that were all 150-200 Mbps. Setting it to a lower flush rate requires a faster card. Not in terms of bandwidth, but latency of access. Use the buffer as a buffer and it'll behave properly.

I am using SanDisk's Extreme 16 GB cards. You can get them on sale at ~$50, normally at $60. For the price vs performance, it's hard to beat this card. Have a look here:
http://www.robgalbraith.com/bins/camera_wb_multi_page.asp?cid=6007-10294
For comparing all of the cards. This has been the most reliable site for card performace. Caveat: Rob has retired the website. This happened before rumors of the 2.0 firmware that Canon released which drastically improved memory handling and thus performance. However, it's a solid baseline for at least another 6 months.

You shouldn't have to use CBR only, set qscale to max (do you guys have that yet?) and then camera records at fixed quality. All frames will be QP10.

There is no such thing as real CBR on canon anyways. Its all VBR just with constraints via quality/qp H264 parameters. You're not directly controlling the rate, you're just telling the camera "even if frame size is *friggin huge* pick a high QP for the next frame.

Fixed quality would be fixed quantizer, which would be VBR, no? Due to general ignorance (which it looks like you guys do a really good job at helping alleviate) most think that CBR > VBR in terms of quality. However, VBR > CBR because it's much closer to assuming fixed quality for each frame. THis really is a better mode to record in.

Thanks 1%
Does that mean a static scene may just ignore "CBR" settings and base it off the first frame's complexity?
Well I don't have a problem maxing out the bitrate indoors, by underexposing with ISO set to 128000, which get my card's full 160Mbps but it's too noisy so I'm trying to get that max bitrate with proper exposure.

Go find some grass. First attempts at all I-Frame encoding gave me ~230 Mbps. I haven't gotten around to 'CBR' testing yet.

@1%:
> You guys haven't pushed luma/chroma quality up yet
you mean QP(Y;U;V)? that is according to analysis tools exactly the same as QP (10).

one correction: you said, camera flushes after each GOP - you meant after each two GOPs, as this is one second

at least via x.264, you could control the quantizer based off of scalar values. Do you guys have controls equivalent to deadzone_inter, deadzone_intra, and chroma_qp_offset?

to explain myself the way how the encoding works and what i am tuning, i made some screenshots.
...
thats the reason why I-frames eat so much bandwith without huge improvements.

It took me a long while to understand this process back in the day when I first started working with video encoding. This explanation you just gave is perhapse the best I've ever seen online. It'll be a valuable resource for those just starting out trying to figure out what mode to use.

As a followup to restate what others have said, the chief reason why all-I is required is for special effects work. Most pieces of software that do this need to cut into a video at an I-Frame and not havfing to wait 3, 4, 5, 10 frames later works much more easily without losing quality

*crosses finger in hopes the 7D has enough processing power and bandwidth for 48fps*

The camera produces 40-45 Mbps video by default. Myself and others have been able to produce 150-200 Mbps video without much cause for alarm. Doubling frame rate and keeping QScale down a touch would result in the same bitrate and that is what matters to the camera. Once he understands how to configure the timers, then I don't see why it would be an issue.

Thanks Digital! You're always getting me to double check my work and learn more lol

You are right the Ariy disk does begin to overlap pixels around f/6.3 but it isn't really noticeable until f/8 so most people consider that diffraction limited.

Unfortunately I think with "line skipping" to down-res the image to 2.1MPs we do not benefit from an increased diffraction limit like we did with Photo mode's pixel binding method which physically makes each pixel sensor larger rather then in the line-skipping method we still have the same sized (4.3micron in this case) pixel sensors :/

You're welcome. Most people refer to f/8 since it's the closest full stop measurement near the diffraction limit . I don't currently have the facilities to test where we start loosing detail between the 18 MP sensor and the resulting 1080p video. However, this is what I've looked at so far...


Ok, so I've done some testing myself. The only consistent scene I've been able to record (partially overcast day) is a patch of grass. Detail here is similar to leaves, but with a lot less movement. Using the all I-frame firmware, I acheived a max average bitrate of ~281 Mbps using CBR @ 20x and the flush rate at 4 frames. I had ML stop recording at 60 seconds and the buffer started out at 28% and then leveled off at 22-23%.

All recording was done with a VAF-7D installed as well and sharpness set to 1. I have screens showing a significant increase in perceptable detail, but I don't have a quick enough of a connection to upload them at this time as they are ~5 MB png files. The artifacting that I previously saw in my test scene with leaves showed scaling artifacts when in-camera sharpening was set to 0. With the VAF-7D, I saw none of these artifacts, even on the grass blades.

I noticed it was a bit hard to get the bitrate above ~230 Mbps for the scene. I had only 1 issue with my CF card not writing out all of the frames and throwing an Error 70 message, but I only had ~70 MB left on the card so I'm assuming it was due to fragmented freespace. When I did this scene for testing the speed of the card w/ the bitrates, I adjusted the 12 points (not worth the words to explain why) on a logarithmic scale (e^(x*ln(20)/11)) from 1 to 20. The file sizes were as follows:
1x   333 MB
1.3x   408 MB
1.7x   531 MB
2.3x   744 MB
3x   975 MB
3.9x   1.2 GB
5.1x   1.62 GB
6.7x   1.73 GB
8.8x   1.63 GB
11.6x   1.66 GB
15.2x   1.72 GB
20x   1.62 GB (ended ~2 seconds early)

These were I-Frame btw... Based on these numbers and the ones others have produced with similar tests, it appears that the sweet spot of recording is 4x-7x. I'll see what I can do for some controlled tests in this area because it appears that the encoder is currently hitting a wall. No reason to give users options to go beyond 10x atm. When/if we establish a best-case upper limit, we could just set it there so we don't have people trying to figure out if they should push their hardware harder?

It might help to know that once you hit an aperture of f/8 the pixels in the 7D start to diffract which will blur the image a bit more and more the further you go passed that threshold.

It's ~f/6.9 for the 7D due to the 18 MP sensor. However, with 1080p video, you're then talking about effectively a ~2.1 MP sensor so the airy disk can technically be larger. On top of that, since the video is created with line skipping, anything over f/7.1 may still in hibit resolution. One more thing I want to look into aside from the pixel-peeping effects of the VAF-7D.

A more finite concern is the 7D's actual resolving power. I can scale a 1080p frame down to 720p, sharpen, and scale it back up to 1080p and scale it back up to 1080p and it's very hard to tell the difference between the two. Granted the camera can resolve pixel level detail at 18 MP and a 1080p frames is ~2.1 MP. Canon does some detail blurring when sharpenss is turned on and this removes quite a bit of aliasing, but not enough to remove moire. I now own a VAF-7D from Mosaic Engineering. When I get home from my travels/vacation/holiday, I'm going to do some pixel peeping to see to what degree it provides anti-aliasing. Prior to its use, I noticed that there is are quite a bit of scaling artifacts when sharpening is turned off. However, those were mitigated when sharpening was on 1. The VAF-7D might allow greating resolving power when sharpening turned off since we then don't have to deal with a smoothed then sharpened image with sharpening turned on. Plus it removes 80% of moire so I'm good with that

As for practicality, ignoring the 4 GB limit of FAT 32 is a more admirable goal. If there is anyway we can alter the paradigm of the file writing to make it behaving like a stream of data being written w/o prior knowledge of the begining of the file, then the 4 GB limit becomes moot. Granted, this might break playback of the files on the camera, but it would solve the scene time issue when recording at a higher bitrate.

But anyhow, I an anxiously awaiting alpha 2 and cannot wait to get home so I can help with bug testing .

Anyway I tried crunching the numbers myself and this what I got:
Frame size: 1920x1080 resolution = 2,073,600 pixels

8bit Y' Channel: 1920x1080 resolution = 2,073,600 pixels * 8bits per pixel = 16,588,800 bits per frame
8bit Cb Channel: 960x540 resolution = 518,400 pixels * 8bits per pixel = 4,147,200 bits per frame
8bit Cr Channel: 960x540 resolution = 518,400 pixels * 8bits per pixel = 4,147,200 bits per frame

Perfect!

Total "YUV" Channels: (Y'+Cb+Cr=) 3840x2160 resolution = (Y'+Cb+Cr=) 8,294,400 pixels * (Y'+Cb+Cr=) 24bits per pixel = 33,177,600 bits per frame

First, 3840x2160 * 24 bpp = 8,294,400 * 24 = 19,9065,600 bits per frame based on that math.
You're close, but you forgot your order of operations, PEMDAS: Parenthases, Exponents, Multiply, Divide, Add, Substract; so a few things are off here.

When you summed the frame's components (yes, it's YCbCr, similar to YUV but I know YUV was close enough to jog someone elses memory for which it was, thanks You added up the dimensions 1920+960+960=3840 and then you multipled them together. You have to multiply them out first and then sum them.

This means Y'+Cb+Cr = 2,073,600 + 518,400 + 518,400 = 3,110,400 pixels of data we're looking at per frame, not 8,294,400 pixels.

Your initial math up on top is perfect so I don't know where you psyched yourself out and deviated from what you had. You summed your 8 bits per pixel before you multiplied them. Thus you computed a video frame that had a 24-bit depth. Damn, I *wish* the 7D could do that kind of dynamic range . A logical overview could explain the reasoning of your number based on doing file size calculations for the rendered frame in RGB, however. Each channel has an 8-bit color depth so you want to multiply each channel by 8 (2^8 = 256 which we love/hate for dynamic range). So...

Since:
2,073,600 * 8 + 518,400 * 8 + 518,400 * 8 = 8 * (2,073,600 + 518,400 + 518,400) = 8-bit depth * 3,110,400 pixels in a frame = 24,883,200 bits per frame

You'll note that this is the same as 16,588,800 + 4,147,200 + 4,147,200 (your numbers!) = 24,883,200 bits per frame

This next part then changes to:
Frame rate: 23.976 frames per second * 24,883,200 bits per frame = 596,599,603.2 bits per second
for uncompressed 1080p @ 4:2:0. For the rest of it, I'll round up to 24 fps to keep it a little simpler. This pushes the bps to 597,196,800 or ~597.2 Mbps

I've seen those same compression ratios. Adjusted numbers for the frame size put
H.264 I-Frame ~= 3,554,742.857 bits
H.264 P-Frame = 1,244,160 bits
H.264 B-Frame = 497,664 bits

1 sec of H.264 video with 2 I-Frames and 22 P-Frames is then ~7109485.714 bits + 27,371,520 bits = 34,481,005.714 bits.
If we had all I-Frame compression, that would then be ~85.3 Mbps. For a GOP of 3, which we know is doable, we'd get ~48.34 Mbps

From the content that I've recorded from my 7D already , I've seen data rates from 41-47 Mbps. We then know that Canon is overprovisioning a bit as it is. If I scale up the ~34.48 Mbps up to an assumed target average of 45 Mbps, then all I-Frame compression gives us a target of ~111.3 Mbps and GOP=3 data rate of ~63.09 Mpbs. Considering the ~597.2 Mbps for an uncompressed frame, that would give us a ~9.47:1 compression ratio compared to ~13.27:1 stock compression ratio.

B-Frames are out of the question since the encoder is hardware, not software. As the DIGIC chips haven't been fully cracked, then the hardware encoders are probably a bit futher off, though maybe this circuitry is in the DIGIC chips? My initial guestimate of 100 Mbps video being sufficient for the capabilities of the camera still match with the assumed ~63 Mbps of a GOP=3 scene. The best way to get better quality is seeing if the encoder can do 4:2:2 sampling and give us color detail back. This would inflate the frame size by 25%, thus our data rate as well. Our ~63 Mbps would go to ~78-79 Mbps, still under the 100 Mbps.

Found 2 bugs, but I'm not sure how important they are atm. I need to do some testing to determine which file writing issue is the one complaining and the other is one that requires a little digging to fix, I assume.

In brevity, poping a card into a 7D from a different Canon presents with a "dir error" but can still record video just fine. I noticed that this is prevalent to having Sensor Cleaning turned on, but w/o it seems to fix itself sometimes. I have to do more digging and testing when I'm back from vacation. Thought I lost 6 videos and about a dozen shots as a result of this. However, I had data recovery with me and then found out it's just being quirky.

Next up *should* be a simple fix. It is assumed that all movies start with "MVI", when in fact this can now be changed in v2 of the 7D's firmware. It's equivalent to ano annoyance so no rush to fix it.


I wanted to respond to the bit rate topic in general. First, as a forward I'm not a developer but I'll borrow the term "observant user" from another on here, I've had several years mucking around with x264 for encoding my own video for archival of TV shows on my server. I've followed the developement heavily a few years back and learned a few things from their forums. Generally speaking, lossy encoded video can easily be encoded at a near perceptibly-lossless quality at a data rate of 1 bit per pixel. H.264 uses a YUV image formate if memory serves. It's not encoded as RGB. The type of video format (wrong term, but I think you guys get the gist of it) is 4:2:0. This means that the color information is saved at 1/4th resolution of the luminosity information. An RGB image, like a JPEG or PNG or BMP is equivalent to 4:4:4, aside from any other forms of data compression applied to the image. To adjust the math from another user, this means that the total color in one of the 7D's video frames is 1/2 the amount of data as luminosity (1/4 res. color + 1/4 res. color = 1/2 res. luminosity). The numbers look a bit like this

1920*1080 x1 luminosity channel
960*540 x2 color channels

2073600 luminosity pixels
518400 x2 color pixels

Generally speaking our images are being recorded with 8 bit color depth across the 2 color channels and luminosity channel, and all at 24 fps (23.976, but for the sake of argument, let's keep the numbers simple right now )

8 bit depth, 2 color channels, 1 luminosity channel
24 fps

This gives us a total of 597,196,800 bits/sec required for uncompressed frames at 4:2:0. This number is half the previous one mentioned because it was assumed that we had RGB channels at full resolution when the image format for video is a little different.

Now remember me mentioning the 1 bit per pixel (bpp for short) for lossy encoded video? Take out number above, convert the 8 bits to 1 bit by dividing by 8, and that gives us a theoretical maximum bit rate of 74,649,600 bps or ~74.7 Mbps for optimal imformation capture. Granted, there are inefficiencies so going up to 100 Mbps should be okay, but 200-250 Mbps might be a touch excessive .


Now, the 4 GB limit bugs me. I know why it's there and I understand how it is a limitation when looking at the *whole* file. But when it comes to the 7D writing the data down to the file, is there any way we can hijack this process a little, because it's a bit retarded. When I recorded TV, the data stream can just keep writing data past the 4 GB limit because it's not caring about the file size. I didn't have any problems reading a 7 GB file on a Win XP or Mac OS computer. If it possible to just have the camera just write the data out, then the 4GB limit for writing a stream of data is irrelevant.

totally awesome

Read his response, this is not the case. We will still have to load the firmware from the on camera update screen

it was doing that before, we use FIR for that alpha as
a) alphas are not for productional use, just for testing features
b) it is safer not to modify flash before we are not sure that it is safe
c) hey this device is worth 2k$, i wont recommend you to patch the flags yet

this shutdown problem isnt linked to startup method. it also happens in FIR mode.

a) Yeah, but one can have some serious fun
b) I've help with embedded platforms before, I understand what's going on
c) Yeah yeah yeah :p. Bricking this beautifully expensive tech is out of the question

You have no idea how much I (and obviously many others) appreciate the work you're doing. Thank you for your dedication to this aging camera. I was fearful I'd have to drop ~500 on a T2i/T3i to get this toolset for recording video on a trip I'll be taking. I'm just antsy for more usable features. It's like Christmas for 7D owners yet we get to impatiently watching the toys being made in front of us .

just to keep you informed:
intervalometer, bracketing (HDR, apterture) and bulb ramping seems to work fine.
i am just having shutdown issues which means, powering off the camera will turn the screen black, but it will still consume power.
this happens only after one of these functions was used.

removing battery and re-insert will of course work
as this looks like a generic problem of shutting down (how tasks should behave) we are investigating this atm.

Since you mention shutdown, is it safe to assume it cleanly boots from the card now?

Yes, it's installed into RAM right now without autoboot

right, and this is known as a general ML issue.
there might be a fix for this somwhen, when we know how to deal with that.
(its about digic registers)

Ahhh, that makes more sense for focus peaking and zebras etc. The algorithms are working off of the video signal not the displayed resolution in Live View. That explains why I noticed focus peaking picking up the moire when in 720p mode as well...

A temp fix would be based off of the displayed markers (2x2 colored squares by the looks of it), and could just be limited by the crop marks like a mask. Though I can understand how that approach hasn't been taken as it's a tad bit lazy and wouldn't solve the actual problem of limiting the algorithms to just the pixels inside the cropped area. I'm happy to see that the whole ML team has the desire for a quality firmware.

I'm not sure if this in intended behavior or not, or even specific to the alpha or not, but I'll post it anyhow.

With Zebras turned on with underexposure clipping, along with crop marks that black out a portion of the video, the underexposure zebra lights up the cropped out, black out area instead of just portions inside the video frame. Granted teh crop marks are in the video frame, but this is improper behavior.

@Digital Corpus:
the level bar drawing is not 100% accurate at ~45° if you wanted to report that.
also its drawing over grid.

thanks for feedback, guys

45˚ is a tough one with line drawing since you have to pick a side to drawn from etc. The specific bug is that the grid is not redrawn before the leveling line so it erases the grid wherever there was an overlap. Minor detail.

I have to get used to the UI design before I can report more. So far so good.

First time I've played with ML. I've already noticed a bug with the display and leveling. When I have time (about to leave for work) I'll provide details. However, I do want to say that benchmarking a Sandisk Extreme 16GB card (60MB/sec rated) produced this:

(sorry, had to remove the white and make transparent)


However, this seems to be very solid and I hope it can be of some use to me on my vacation next week. Thank you very much for your work on this.