Bit rate investigation

[Andy600]

Hi 1%,

Thanks as ever for your work!

I'm a bit confused by the additional slice menu. Do we need all these extra settings to be user adjustable? I like the idea of the min/max BR setting. For me its a case of the simpler the better.

I would be happy with locking DBlock at 0 and just having GOP and BR min/max settings being user adjustable. I confess I still don't fully understand slices etc

A

[1%]

Below Min BR is where it starts to raise quality by 1. Max is about where it will start dropping by 1. Both of these run every second to try to smooth things out before it gets to the warning level.

If the buffer warning goes off it drops quality by 1 at "drop by 1 bit rate" and by 3 at the drop by 3 rate.

Point of adjusting it is so that you can customize it to your card. I can now use it on my slow card too.
I actually wanted to let people choose min/max slice but only implemented that in the config file.

Slices are simple, its just like qscale. Lower values mean higher quality, higher values mean lower quality. You don't really have to mess with the slice setting besides 0 and non 0.

I can try for less settings but then people with slower/faster cards are out of luck. The way it was going before it would raise to 87 immediately and then the buffer would fill again and it was an up/down yoyo on really difficult scenes.

Really I should make a guide at some point once its all good and tested. The big question though is if quality has dropped since the functions are less aggressive.

[3pointedit]

If allowing many parameters is used to suit individual cards, could the camera benchmark a card and set quality to best capacity?

[1%]

Things change between all-i and gop3, also I'm not confident enough to just run a test and call it a day. Cards degrade over time until reformatted. I haven't done mine until recently and it was barely doing 80mbps from all of the testing. Its just 4 settings in a menu. Before we can do something so finite I'll have to do more testing and so will others. I don't know if what I did is better on video, just better at stopping overruns.

Also noticed: br indicator for all I seems off by a bunch.

[Rush]

I am sorry, if I want to turn off bitrate autocontrol in latest builds - should I just set Slice to 0?
I was playing with settings and faced problem with low bitrate blockiness in the middle of recorded video, so I want to switch to old method of bitrate control.

[1%]

Yes that would work. I'm working on fixing that actually. I don't want to use that -128. I think its good for stopping overages but causes dreaded blockiness in the middle.

* this is also the reason why I  need to put that min/max slice

https://bitbucket.org/OtherOnePercent/tragic-lantern/downloads/autoexec.bin.SliceMin.6

This should have indicators under control and no longer go to "-128". See if its still blocky.

[Rush]

Thanks for reply, 1%.
I changed Slice to 0, but when I start recording it still tries to avoid buffer fill and lower bitrate to blocky image. And Slice becomes non zero value.
Is there any other way to disable bitrate autocontrol?
I would like to have old manually controllable bitrate with buffer overflows, to be sure that my shots will not suffer bitrate drops.

[1%]

It will change to non 0 if you turn debug on.

If you leave debug off CBR prediction works.

So set to 0, reboot, leave debug off.
I named the debug variable the same as the variable that sets it. Have to fix that and turn off showing I/P, etc when in slice mode too.

What slice does it get really blocky at? I'm hoping to fix the blockiness so need input. Maybe add constant slice like qscale mode too.


If you hate buffer save, you can now record at fixed quality buffers be damned. Also fixed that debug setting slice in cbr issue. Still working on getting wav to be skip free.

****Canon CBR fixed for real. It was still going back to slices after a little while. I couldn't tell until I put the indicator.
https://bitbucket.org/OtherOnePercent/tragic-lantern/downloads/autoexec.bin-10-11-12

[JasonATL]

1% - I just wanted to put a note of thanks (again) here. I sincerely appreciate your continued work on this.

I have had limited time in the last couple of weeks to do more testing/experimenting, but hope to do more soon.

[aace]

I'm very interested in this. I've been using the nightly builds. To use this am I replacing the autoexe.bin file with these or should I just wait for it to be implemented into a main release? I'd like to try this but not at the expense of my camera.

[1%]

Haven't broken my camera yet.

[AriLG]

I'm very interested in this. I've been using the nightly builds. To use this am I replacing the autoexe.bin file with these or should I just wait for it to be implemented into a main release? I'd like to try this but not at the expense of my camera.

It's only for 600D, IIUC (or not ?...

[1%]

I've released a couple of non-600d builds but the good stuff is for the camera I have + have firmware to.

[reddono]

I've released a couple of non-600d builds but the good stuff is for the camera I have + have firmware to.

Sorry, does this mean that the magic lantern team doenst believe in your experiment? I dont see the reason for not include your work in the night build...

[1%]

I've never made a pull request.  Also use a lot of cache hacks. Few other cameras break 4gb limit. I also use movie remap. These things probably keep it from being integrated. I just pull ML changes to my code tree so its not like you're missing anything official. Also playback issues in camera since videos violate baseline spec on gop, bitrate and picqpy/pc.

[nanomad]

Cache hacks are the primary reason. They may cause unwanted side effects :/
That being said, the only cameras this hack will work decently are the 1100D, the 600D and (possibily) the new Digic V. The other lack the ability to overcome 4Gigs of footage

[1%]

If MJPEG gets working untethered (right now makes 0 size jpegs that don't show in ram) at movie YUV sizes H.264 will be obsolete anyway. Camera has AVI player in fw and it runs all the time. Then if separate wav stops stretching/skipping we have a full replacement for canon's consumerish/gutted implementation. Also dunno how fast it updates, LV fps > recording fps it seems. On eos utiity its a little jerky but that's because its going over USB. Copying the pics might give 60FPS in "1080"

[CaptainHook]

Hey guys, is there any other place MJPEG is discussed on this forum, or is it just in this thread? I've read quite a bit of the thread but not all of it..
Is it discussed anywhere why Alex isn't that interested in higher bitrates etc (deeper in this thread maybe)? Is it because MJPEG is a better option?
We have 2 5Dmk3's (and a mk2) and i'm really interested to know what might be possible to improve the video quality. Alpha 2 is really great so far though.
Thanks for all the hard work!

[Rush]

CaptainHook, Topic: (M)JPEG encoder

[CaptainHook]

Thank you!

[skydragon]

Really I should make a guide at some point once its all good and tested.

1% - Is there any way you could write up a quick user guide in this thread, or at least a basic settings template to use that gave maximum video quality, along with a few comments as to what does what.

I downloaded your dev work, but couldn't make out what to do in the UI, so stopped ;-)  I'd really like to try the All-I mode, or whatever you recommend as best from your tests.

Reason i ask is I'd really like to try and improve the quality of video on my 600D. I've tried doing some tests with ML 2.3 and I find it very difficult to see any difference between CBR x1.0 and CBR x 2.8. I've tried putting screen grabs side by side and doing zoom in's but there appears to be little or no difference between the two bitrates in use (which is not what I was expecting).

[1%]

Here is an explanation of the settings:


From the top:

Canon CBR: All of the numbers that make up Canon Predictive CBR. Set all to 1 x value or alter as you see fit. See what QPs show up in the files.

Slice CBR - Bit rate based on frame quality.

Jp Slice - The "Quality" of each frame
Lock Slice - Disabled/Enabled - Set slice to "JP slice" value and don't change it.


Taper rates - Enabled/Disabled. When bit rate is lower than min BR raise slice by 1. When BR is above Max BR drop slice by one. Hence, taper the rate to where you want.

Drop By 1 - When buffer starts overflowing.. as set by BuffWarnLevel drop slice by 1 at this rate until it stops.
Drop by 3 - When buffer starts overflowing drop by 3 above this bit rate, its an emergency!

Old Bit rate menu:

CBR - Use slice CBR or canon "cbr". VBR, use the qscale value

DblockA/B - Set deblocking filter. -6,-6 is off, you can look this up its just like H264

PicPC- Raise chroma offset. Negative values are better. Caveat: at 0 you raise luma offset which is technically  up a level from stock. You can't raise both or camera freezes so its  made this way.

Gop- "flush rate", group of pictures... 1 is all I, 3 is a good mix. 4 is being used on 7d. Raising it higher creates more predictive frames which is better on compression but harder on your buffer.

Debug- Show values of slice or canon CBR on the screen.
Averaging - Scale average values in canon CBR - they go higher.

You can use Canon CBR, Slice CBR or Qscale to control rates. The settings for each mode work in that mode. So changing X values in slice or qscale will do nothing.


What I'm using: MinBR :80 Max:125
Drop By 1, 85, drop by 3 100, tapering enabled. I turn off deblocking filter and leave picqPC at 0.
I mostly shoot gop3. Gop 1 is nice but rates are very high and writing is pretty much continuous.


MaX quality: Slice Lock: Enabled, Slice 87, picPC :0

This will shoot a video of 100% QP10 frames if your card can take it.

You won't really notice if you're not doing post and delivering to youtube. If you're mixing with "pro" footage, grading, and delivering to Bluray or theater you will.

Some ISOs produce X bit rate at max quality, you can't artificially squeeze more data out. You're not getting 200mbps at ISO 80 unless the scene is very complex.

Buffer warning level is very important. I set it at 50%. In gop 3 or 1 if buffer climbs above like 30% camera is starting to get behind in writing out the frames.

Hope this makes some sense.

[skydragon]

I've just done a (non-scientific) test.

I arranged some objects, lit them with a LED panel and using a 600D on a fluid head tripod, with a sandisk 95MB/sec SD card and using ISO160, 1/50th, f6.4 i shot some video. I used a neutral picture profile with in-camera sharpening turned down to minimum.

Firstly here is a screenshot from the scene i shot,  (apologies I just gathered some objects together quickly to use)



Firstly I shot a video panning slowly from right to left (about a six second pan) using ML 2.3 and CBR x1.8. The resulting video was an average bitrate of 61678kbps

Then using 1%'s version of ML, I shot another video using the exact same conditions and panning at a very similar speed, but at All-I and an average bitrate of 103320 kbps.

On playback afterwards, on a 23" PC monitor (frustratingly) I could not see any visual difference between the two clips.

I then did a screen capture of a mid point frame from each video and in photoshop zoomed to 300% and compared a similar area from each video, to see if closer scrutiny would reveal a difference.

The ML2.3 CBR x1.8 is on the left, the 1% ML is on the right. I can't really see much or any appreciable difference.



I'm really puzzled why cranking the bitrate up and making the capture All-I appears to make no perceivable viewing difference to the video quality...

NB; I'm not saying that in other lighting conditions or scenes there might not be benefits in having a higher bitrate or All-I, but I'm struggling to see how to improve 600D video for 'general' usage.

Is it perhaps that the (less than 1080HD) resolution used internally within the 600D before up-scaling to 1080 is the quality bottleneck....or is it that the 600D's H264 encoder is at or near it's quality limit and not much more can be squeezed out of it.

Any ideas?

Thanks,

[1%]

You encode X at the best quality, you still have X. Compare with silent pic of the YUV buffer (big one, not the small 1024 one). Upscaling is one factor, limited real resolution is another. We're only messing with the encoder, not input.

Canon CBR dropping quality on scenes to keep it around 40-60mbps is where it gets worse, this kinda prevents that. Try QP of 127 (or higher) locked and you'll see a difference. With larger sized frames and weaker compression the grading gets a little bit better or at least easier. The only other things to do to improve quality are 1:1 sizing to kill the upscaling or 4:2:2 color. Encoder is as good as it gets unless we make it record at like QP0 or something... even then its still going to look like the YUV frame.

Also, the point of compression is to use visual tricks to make the difference not noticeable. Like a 320kb/s MP3 vs a 128kbps one can you really tell THAT much of a difference? But in a normal workflow you'll re-compress again and its better to start with more data than less, even if you end up throwing some of it away.

[skydragon]

Thanks 1%. Understood that a higher bitrate may mean easier post.

Does this all mean that irrespective of encoding tweaks, the internal resolution of the 600D will mean that this is as good as video quality will get?