Messages

Thanks for the video.

Could you record from the HDMI out while zoomed in?

Yes. But, it doesn't look great so far. I'm going to try to get a full "resolution chart test of HDMI out" video up in the next week or so. I've done some playing around to try to see what settings and tests make sense. It sounds like you'd like to see one with zoomed in (I assume using the magnifying glass to go to x5 and x10?).

But the pillarbox remains.

Sorry for my confusion and for not being clear. My original post meant that sometimes the letterboxes were there (hence the 1620x912) and sometimes they were not (hence the 1620x1080), but were somewhat inconsistent based on the settings I had. Usually, a press of the ML menu button would correct it.

OMG you can barely tell! That is stunning, no wonder the camera makes such soft images. Would be interesting to see some motion tests, to compare deinterlacing effects.

Looks like great news for long form recording though. Thanks for taking the time.

You're welcome. The reason that I shot the trees was to get fine detail and independent motion of the detail. In my experience, the combination of these two is what taxes the codec and results in a lot of compression artifacts at lower bitrates. Using higher bitrates cleans images such as this up quite nicely. Unfortunately, the zoom and interlacing challenges currently make this difficult to see. I agree, though. More tests should be interesting.

There's always a box around the video. Maybe your TV had auto zoom turned on.

You can remove the box (well, the top and bottom): In the ML menu, Display->Clear overlays

Played around with recording from the clean HDMI out feature this weekend. Seemed to work quite well. ML was stable, though there were a few times that the image on the HDMI out was inconsistent: sometimes it showed the full 1620x1080 and sometimes only had the framed 1620x912 box even though my settings were to show the full image. I'll try to see if I can replicate it and, perhaps even record it.

Here's a thread I started in the "Post your videos" area with a link to a short video I made of the image from HDMI out and comparison with internal recording: http://www.magiclantern.fm/forum/index.php?topic=3489.0

Thanks a1ex! This is working great!

This is a 70 second video comparing the results of using the 5D Mark III's internal codec (ALL-I) with the video from an external recorder via ML's clean HDMI out. The external recorder is the HyperDeck Shuttle and ML is a1ex's Alpha 2 version for the 5D Mark III

As is discussed in the notes on the Vimeo, the current challenges in getting good results this configuration are: 1) the video via HDMI out is 1620x1080, requiring a zoom to get the same 16x9 image as what the 5D records; 2) The HDMI output is at 1080i, requiring deinterlacing; and 3) the output of the HDMI is 8-bit and might be 4:2:2, but could be 4:2:0.

1% - I just wanted to put a note of thanks (again) here. I sincerely appreciate your continued work on this.

I have had limited time in the last couple of weeks to do more testing/experimenting, but hope to do more soon.

Also - sorry if I'm missing something - people seem to be comparing the new encoding to the original FW at CBR 1.0x, but would it not be much more appropriate to be comparing it to CBR 3.0x / 2.0x or whatever the card can manage?  (We already know that CBR 2.0x is better than CBR 1.0x.)  Can people see a difference between the new encoding and "normal" CBR 3.0x?

Leon - one problem with trying to compare CBR 3.0x with the new approach is that CBR 3.0x will fail/stop as soon as the picture becomes suitably complex - which is exactly when the high bitrate will have the most benefit. On a static (no motion), low light frame, I am confident that the new approach is at least as good as CBR 3.0x. I was never able to consistently record CBR 3.0x on a complex daylight shot. The highest reliable rate (with no sound) was around 2.3x or 2.4x for me.

So, for me, not only is this new bitrate approach getting superior picture quality - a subjective comparison to what I could get with 2.3x CBR, and objective compared to Canon FW - but, it is superior in that it is far more stable, allowing for its practical use. The video of the horse stables (link posted in a post of mine above) simply would not have been possible using CBR 3.0x and possibly not even stable at 2.3x. I shot that video without the camera/buffer ever failing in 3 hours of shooting.

Finally, I'll add that the highest average bitrate that I recall seeing with CBR 3.0x or 2.3x was around 95 Mbps. The video that I am shooting with 1%'s enhancement is averaging 120 Mbps consistently in shots that are a little complex. Indoors, I don't think it gets that high, unless you are at a very high ISO. My point is, the quality settings are driving the bitrate that high. They couldn't get that high with CBR, because it would fail.

Andy600 - I don't see the problem that you indicated, so I'm guessing that it is the suspected card problem.

Here are my results (all at GOP=3):
The buffer save function has worked flawlessly in real conditions. I was able to overflow the buffer by using TV static and moving the camera to a blank wall and then quickly to the static at 60% warning. I could not overflow it at 50% warning. In short, the save function seems much more robust than before.

Here are two comparison shots.
Full: https://picasaweb.google.com/lh/photo/8AgT16svWHKXAOzcNPd8BxfNx8dP82C1K8bmBL14X3k?feat=directlink
Zoomed 300%: https://picasaweb.google.com/lh/photo/I38AfErPsBMBzwdyzCZJbBfNx8dP82C1K8bmBL14X3k?feat=directlink

On these images from left to right:
Left is when the bitrate (and QS) is high (wind wasn't blowing much). BR avg=160 Mbps; peak=250 Mbps Slice=87
Middle is when the bitrate was dropped (but after wind has calmed a bit). BR avg=90 Mbps; peak=129 Mbps Slice=127
Right is Canon FW: BR avg=40 Mbps; peak=155 Mbps
Note that Left and Middle are from the same clip, just at different points based on the bitrate.

To me, the Canon FW is clearly inferior at full resolution (I think it was reporting Q-01 or Q+00). I can barely make out the difference between the other two. The difference between the middle and left is apparent in the 300% zoom of the footage, but the middle is still superior to the Canon FW.

In playing the video, I still cannot tell by watching when the buffer save function kicks in or lets up.

An oddity: Slice nearly always goes to 126 or 127. I had it go to -127 a few times. Also, when changing slice, I only had it stop at something other than 126 or 127 on the way down. I wonder If it is always going to 127 anyway, then why not just jump it there in the first place? Also, could there be a user setting for the min slice (or max QS)? This might allow the user to set it to a level that is less likely to get to the overflow (and perhaps might make it more workable for slower cards?). Given the frame captures above, I'd wonder if I could tell the difference between slice=120 and slice=87? And, if slice=120 saved me from getting to 127, all the better? I'd bet that 190 Mbps I frames would be nearly indistinguishable from 250 Mbps, but I could be wrong. Edit: I take that back, I want 250 Mbps when I can get it .

1% - Still working with this latest build and tried GOP=3. It looks to be absolutely awesome. The buffer save is clearly working more effectively now. I gave it quite a workout for about 15 minutes. I'm trying to look at the footage now to see if I can see the difference. My first pass through, I couldn't see when the save function kicks in - so this is great! No more "breathing". I tried to overflow the buffer many times and haven't been able to yet. The recording didn't stop on me in my few tries.

Also, I assume you're seeing this: peak bitrate at 250 Mbps on the i-frames! Craziness!

Thanks again, 1%. And thank you for saying, "user proof" rather than "idiot proof"! The latter might have been more accurate when referring to me, I think. 

We're supposed to have some wind here today, so perhaps I'll be able to test out with some blowing leaves again, since I have experienced how this particular feature has behaved in this circumstance.

I haven't noticed any in the raw footage. Do you ask because you see it in the edited footage or shots? If so, perhaps I'm just not seeing it.

I haven't shot anything challenging with this new build yet. But, I wanted to give folks a heads up in case they try this beta (any version, probably). I had left the audio on accidentally when engaging CBR on this. I got an Err70. Recovered fine, but Err70 several times without writing a movie file. ML log was written. So, be sure you have disabled audio in the Canon menu before testing this! Otherwise, this version is working - again, haven't shot anything that would tax the bitrate yet.

Also, I edited the video that I shot last week. It isn't "wow look at this picture quality" material. It was me handholding my 600D with my Zeiss 50mm 1.4 and chasing my daughter's classmates around a horse stable, trying to get exposure on the fly (weighing overexposure against seeing into shadows - literally). Also, the color grade is not one that necessarily wows (I was going for a look). But, I shot it all with 1%'s bitrate enhancement and I will say that, unlike working with normal footage, never did the grading or sharpness get to a point that I thought constituted a breakdown. I was able to push sharpening far further than normal. https://vimeo.com/49988361

I'll continue testing and fill out the form when I feel like I have tried enough.

Here are two things I found today:
Setup on this test: ML on empty SD card. 4 GB CF card used for recording images/movies.

Minor display bug. The GB remaining (top right corner) does not refresh properly. It doesn't appear to erase the previous value before writing the new value.  When you go to the ML menu and come back, the screen refresh cleans this up so that the GB is not messed up (i.e., it displays properly). See screen capture here: https://picasaweb.google.com/lh/photo/gtRb7lAR1hphpAmeVO8Eus6bkbuOKDnQM31wDLF3lbE?feat=directlink

Possible error-inducing feature. This happened to the recording that I was doing while trying to capture the image referenced above. It was tricky, as I had to time it so that the GB remaining would change after I started the screen capture timer, but before the timer elapsed. It took several tries. I believe this is relevant, as you'll see. I let the CF card fill up by continuing to record. When the card filled up on this round, I got Canon error "Err 04 / Cannot save images because Card1 is full. / Replace Card1." To be clear, this did not require a battery pull. The .DAT file remained on the card, but the associated .mov file was blank. The ML debugging log (reported on screen to be writing) was also empty - it was there, just 0 KB. I tried to replicate with both no ML and ML and could not... until I tried the screen capture during recording.

My hypothesis is that the Canon firmware notes space remaining at the beginning of recording and subtracts off the recording size from this, stopping the recording in time to convert the .DAT file to .mov. The screen captures (I did three or four) caused the card to be unexpectedly full before the Canon FW could stop the recording and convert as normal. My own view is that I'm not sure that it is worth trying to fix this with anything other than a warning not to do a screen capture if you plan on filling up the card (perhaps this warning already exists and I just haven't seen it!).

Thanks, 1%! I'll give it a shot later today, I hope, and report back.

I was using elapsed time. It seemed accurate.

Global draw doesn't affect BR and stoppages much. I think the problem is you're over 4gb (hence the red) and the BR measuring and buffer functions are getting inaccurate or the card is slowing down.

Nope, not over 4gb. No clip was longer than 1.5 GB.

More photos of test shots of leaves blowing on trees - exciting stuff, I know  .  I shot these today during some wind. To me, the difference between the Canon FW and either GOP=1 and GOP=3 is visible without pixel peeping on the 300% zooms. My wife noticed on the raw video. I didn't bother posting another vid, as I think the captures tell the story.

https://picasaweb.google.com/lh/photo/jEy4yTlS6DyEXEu46gA9DBfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/Q-Oio1E4HegePZwbTqVt2xfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/vDQ2AsTA_cw9hamymrO-xBfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/uDL8Pka3l0N2o1gWm9hdWxfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/qvH_zxX9KWx1AxLp147P4BfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/mzN3khs9lDsxkgt5zt-VeRfNx8dP82C1K8bmBL14X3k?feat=directlink

Without 3x crop mode:
GOP=3 BR: Avg=139 Mbps, Peak=207 Mbps
GOP=1 BR: Avg=128 Mbps, Peak=129 Mbps
Canon FW BR: Avg=45 Mbps, Peak=138 Mbps

With 3x crop mode:
GOP=3 BR: Avg=121 Mbps, Peak=168 Mbps
GOP=1 BR: Avg=115Mbps, Peak=128Mbps
Canon FW BR: Avg=46Mbps, Peak=100 Mbps

1% a couple of things happened while shooting these tests that I've not experienced before. First, the recording would stop with a "camera stopped recording" message without a buffer overflow. I was watching the debug info (Slice was around 122 in many cases) and the buffer % was in the 20% to 30% range. Nothing obvious shows up on the bitrate (it was in the 120's when many of the clips stopped). Also, this happened 30+ seconds into the footage. In other words, the buffer save function seemed to be effective and not the cause of most of my stopped recordings.

Second, (this might be related). The recording time turned red. What does this mean? In some cases, it turned red just before the recording stopped. In other cases, the recording would continue with the time in red for 30 seconds or more. Is this a sensor or processor temp warning? If so, we have another element to consider. This was the first time I had let the recordings go for a long time. If it is a digic temp warning, then perhaps the polling rate could be affecting it. Also, I didn't seem to have the stopped recording issue the one or two times that I had rebooted and then failed to turn debug on. To the extent that the debug adds overhead, perhaps turning off GlobalDraw might help?

Some real world experience with this: My daughter's class just took a field trip to horse stables this morning. In 3 hours, I shot about 45 minutes or more of video (clips about 20 seconds each). Used GOP=3.  It was a sunny, windy day. No test shot... just shooting as I normally would. No overflow once. The buffer save function only had to kick in a few times and in the real footage, I can't tell. It looks like my avg bitrate was around 80-90 Mbps, with some peaks in the 130 range. I'll probably edit the footage this weekend and post the video. I wouldn't have been able to do this with the old CBR. Worked great!

BTW JasonATL what settings were you using for the GOP test? Did you just increase the CBR and change GOP settings? Also did you alter DBlock A/B settings?

On the waterfall video, I don't know (I wasn't keeping very good notes  -- I'm sure about the GOP, though, since I can infer it in Bitviewer). I have tried different settings for PicPC, DBlock A/B (with notes). There appear to be differences in what I can only describe as "grain" - and I could only see it at 300% zoom, and even then, it wasn't strong. But, I could just be seeing things. Even so, while there appeared to be differences, I would hesitate to call one "better" than the other. At the same GOP length and bitrate, there isn't a very noticeable difference to me in these settings. In the end, I couldn't determine a preference for one setting over another. Is there a setting for these that you prefer? If so, what should I look for?

I agree that GOP=3 is about the sweet spot. GOP=4 works fine and GOP=2 works fine. But, again, I see very little difference among these in the tests I've done.

Andy600 - I don't think that a 550D has a very powerful OLPF. Otherwise, we wouldn't have the moire problem that we do. Still, it could have one. Perhaps when I finally get my own 5D3 (the one I use from time to time is my wife's), I'll have the guts to take my 550D apart. (because at that point, it will be fourth instead of third). I don't believe (I use this word purposefully) that removing the filter improves true resolution on the 5D3. In the end, this shouldn't be a matter of "faith" (e.g., I shouldn't believe or not believe it). But, right now, people are using faith rather than evidence, because there has been no evidence. I have not seen a side-by-side controlled experiment nor have I seen an actual resolution chart shot with video on a camera that removed the OLPF on the 5D3. I recall that James Miller shot the same scene (one with and one without the OLPF), but on different days in different light. Does perceived resolution increase? Perhaps. True resolution? I doubt it. But, I'm willing to believe it if I see the evidence. As I said, it shouldn't be about faith.

People believed the Nikon D800 had much higher resolution than the 5D3 (we're talking video here). In my tests of shooting a resolution chart, the D800 comes in at about the same resolution as the 5D3. If any, it MIGHT have about 20 more lines. But, if the 5D3 is about 800 lines, it isn't as if the D800 is Full 1080p (and the moire you get with the D800!). My point is that perceived sharpness and actual resolution are two different things. This is one reason that I'm happy about having more latitude in sharpening the 600D (and the 5D3) footage with improved (or less?) compression.

Marvin - Thanks. I don't know enough about H264 to answer your question precisely. 1% has been able to change the encoder's parameters. What this could mean is that the encoder is not compliant with the profile specs or that the change in the encoder preferences abandons the profile (is that even possible?). If you are knowledgeable in this area, perhaps you could shed some light on it for us and help us better understand what parameters we can control and how they will help give us the best output here. I was intrigued by your testing with the 5D3 and look forward to being able to get a better bitrate from my 5D3.

ilguercio -I suspect that, if this was your own footage and you were editing and color grading, you would notice the difference. What I have really noticed is that I can sharpen the footage and push the colors much more than before without it "breaking down." What I mean by this is that with sharpening and color grading, the compression artifacts show up quickly. With such a high bitrate resulting in such a good image to start with, the image is more robust to changes. It also looks far less "digital" to me with a much more natural look. This subjective aspect of the picture sometimes isn't easy to pinpoint as to the exact part of the picture to blame. Finally, what I will try to demonstrate in my next video will be a more detailed image. In this case, what looks like "mush" with Canon FW turns into a very nice picture.

I understand your point about resolution. I would like to see more resolution, too. But, the key point here is that the Canon FW is not giving us the best of what (little) resolution we have due to the bitrate and encoder parameters. 1% has at least made huge progress in fixing this. I am excited about the having this on the 5D3 (and other things in development, as well). But, I am quite impressed with what I'm getting out of my 600D now.

I finally edited together 20 seconds or so of a waterfall. Shot with Canon FW, GOP=1 (ALL-I), and GOP=3. The links to screen captures are below, too. This difference in quality is huge, in my opinion. I did not test the ML 2.3 CBR 2.0x or something like that on this particular shot. I'm fairly confident from experience that CBR 3.0 would have caused a buffer overflow, but maybe not.


Of course, download the original file to see this without Vimeo's compression.

GOP=3: Avg 149 Mbps, Peak 170 Mbps
GOP=1 (ALL-I): Nearly constant 169 Mbps
Canon FW: 40 Mbps

Still 1: https://picasaweb.google.com/102645356447404039003/September192012#5789851335385308482
Still 2 (300% zoom): https://picasaweb.google.com/102645356447404039003/September192012#5789851289298012066

I think that ALL-I probably looks the best overall here, but GOP=3 is very close. The increased detail you can see in the rocks, leaf, and shadows in the waterfall is visible at 100% and obvious at the 300% zoom, in my opinion.

I also did GOP=2, 4, and 6 (not shown in the video) for this same subject and don't really see a large difference among them.

Thanks, A1ex. I appreciate everyone's work on this.

Just trying to offer help in any way possible. Let me know if there is anything that I can test/try that might be helpful.

Miyake - I think that's a great idea! I would be willing to do what it sounds like Marvin has done for the 5D3 in testing sets of parameters for the 600D (and if anyone cares to take this to the 550D, I'll test it, too). What would be really nice is to have a menu that allows me to choose a profile to load and use (sort of like the overlays). At least for this testing phase, it could be more efficient.

Marvin and A1ex -
1. Perhaps I misunderstand, but can I change those parameters on an 5D3 without ML by using the .ini files? How does the 5D3 load them? Do I just put them on the root of the SD or CF card and it automatically loads them?
2. Before I try this, is Err70 an error that needs a battery pull? Or, does it pose a greater risk to the camera?
3. If answers to 1. and 2. are positive, then is there anything I can do along these lines or is all understood by you already?
4. Is the 5D3 the only camera that takes such an .ini file?

I'm using instant but its not super accurate either (occasionally pops up to like 200-400, not real). I'm having stopping issues at high ISO like 2500. Then again 2500 looks like crap in day time. Need some math help. I'll push what I just did when I test it some more.

1% - Just curious: How do you know that 200-400 is not real? The reason that I ask is that VLC was reporting input bitrates exactly that high in some of my early test clips (and I just haven't checked in my later clips). Bitrate viewer does not show them that high, but my version of bitrate viewer only shows average bitrate in one second intervals (is there a way to get it to report bitrate at higher frequency?).