Bit rate investigation

Started by Audionut, July 19, 2012, 04:54:03 AM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

Rush

Just got SanDisk 95MB/s and tried to get maximum quality on 600D with the help of GOP-control.

What I get:

GOP1 - is the way to go for high bitrates, but this compression is not very effective - and it don't use buffer at all. 600D is limited to stable 160 mbps with it.
GOP12 - effective but it can't go higher than 100 mbps - because of high buffer use. Not stable.
GOP3 - is the platinum! It still limited to stable 160 mbps just like GOP1 (I think that it is ~170mbps SD interface limit of 600D), but is more effective than GOP1.
For example - it can go to 160mbps and compress with QScale -16 quality, while GOP1 can compress only QScale -12 quality with same bitrate. It uses buffer, but very stable and limited not to buffer, but to ~170mbps of SD interface.
I'd tried GOP6 - but I see that it hits buffer limit, not SD interface limit (GOP4 is as good as GOP3, but it is limited to buffer, not interface. There is thin edge between GOP3 and GOP4 - between interface and buffer limits)

And the most impressive for me was stable audio recording with 160 mbps video! GOP3 ftw! 8)

Two words about compression effectiveness:
Seems like GOP3 is only 10-15% less effective than stock GOP12 (handles same QScale with only 10-15% higher bitrate)
But GOP3 160mbps bitrate limit is ~50% higher than no-audio StockGOP12 (100mbps) and ~150% (!!!) higher than audio StockGOP12 (60mbps)!
So stable 160mbps of GOP3 is like 140 mbps of stock GOP12 +audio, which don't cause recording stops!

Thank you, developers! (is 1% - the main developer of GOP control? thank you!)
I have a question - how can I set my bitrate limit, so it will switch to lower -QScale if hits 165mbps limit?
CBR 3.0x drops QScale when bitrate is higher than 155mbps, not 165 mbps - but still is pretty nice working (I am so happy that it is stable, with audio, 155mbps!).
It seems that with CBR 3.0x 155mbps limit optimal will be GOP4 - slightly more effective and hits buffer limit only with 170mbps, so CBR 3.0x will do the work not to hit it. But I am not sure about GOP4 buffer fill stability, GOP3 will be more safe to use.

And another question - when GOP-control will be available in ML nightly builds? It is very useful and looks stable.

p.s. separate WAV recording don't work for me - it just fills buffer. But I don't need it now - happy with GOP3 audio recording.
Greetings from Russia!

1%

You have to have canon audio disabled for separate wav, just override on.

I have to rewrite the whole thing for it to work with the nightlies, the menu system changed.

Plan to add back the good stuff and leave out some extra stuff like all the canon cbr tinkering. I use gop3 too, I think its most effective because you still make some P frames. 600D video is eating digiV. :)


g3ggo also has separate flush and gop controls but I don't have the addresses for 600D, would be nice to try it and see what it does.

Quotehow can I set my bitrate limit, so it will switch to lower

If you use slice control it will drop when the buffer goes over the buffer warning level. Drops by 1 at x rate and then by 3 at x rate for when its getting really overloaded. The leveling eases it off slowly if over a certain rate. There are more detailed infos in this thread some posts back.

If you're just recording fixed qscale value its not set up to move either way... that is for fixed qp recording.

Right now mainly working on 6D since it still needs help and the only really missing from here are some bug fixes and image effects like peripheral correction, etc.







Rush

Quote from: 1% on March 17, 2013, 09:06:37 PM
You have to have canon audio disabled for separate wav, just override on.
Yes, I tested it with disabled Canon audio. It just fills buffer...

QuoteIf you use slice control it will drop when the buffer goes over the buffer warning level. Drops by 1 at x rate and then by 3 at x rate for when its getting really overloaded.
I tried this bitrate control with my Class10 card in october, but it worked really bad for me - it always dropped bitrate to <40 mbps so the blocks appears (but my card was able to record 80-90 mbps stable)... I did hate how it worked... Do you use it yourself? Is it works good for you?

I think that  GOP-settings should be tweaked according to SD card speed.
On slow cards it better to use longer GOPs (6-9). On fast cards - GOP3 or GOP4 will be the top quality.

To determine which GOP is best for your card, you should:
step 1) determine speed of card - turn off audio, switch to GOP1 mode + QScale-16, slowly increase complexity of scene and watch for bitrate and buffer values. (increase ISO to increase complexity) remember which value of bitrate causing red buffer load - it is max bitrate for your card.
step 2) determine which GOP is the best - turn off audio, switch to GOP12 + QScale-16, slowly increase complexity of scene and watch for bitrate. If video stops before max bitrate from step 1), then lower GOP value and repeat step 2). If video stops when bitrate hits max bitrate (from step 1) - then it is your card's optimal GOP value.
step 3) now set to you optimal GOP value and switch to CBR mode. Increase CBR value just before buffer starts to became red.
step 4) now repeat steps 2-3, but with audio turned on to get optimal GOP with audio.

For SanDisk 95mbps which can handle >160mbps best settings:
CBR 3.0x, GOP3 or GOP4. This will work with and without audio.

For my Kingston 100x Class 10 card, which can handle 110 mbps max:
w/o audio: CBR 2.2x, GOP8. Decreasing GOP will lead to lack of card speed, increasing GOP will lead to buffer limit.
with audio: it hard to determine... short GOPs surprisingly unusable cause of buffer overflows. long GOPs is better but don't hit 110 mbps cause of buffer... seems that on slow cards it hard to keep both high bitrate video and audio...
Greetings from Russia!

1%

I use it for myself, don't have problems with the audio filling the buffer. It doesn't use that buffer. But I can't record audio on with different gops. After a little bit I get an error 70 because the audio funcitons weren't patched. My only issue with separate wav was syncing it and skips before it used dma memcopy.

If fixed qscale works for you, use it. Camera sets quality by slice, the other 2 functions are just abstractions. 5d3/6d just use qp directly now.

Also crop mode rates > regular rates.






Rush

Quote from: 1% on March 17, 2013, 11:23:20 PMIf fixed qscale works for you, use it. Camera sets quality by slice, the other 2 functions are just abstractions. 5d3/6d just use qp directly now.

Also crop mode rates > regular rates.
Ah, I see.. That is because higher ISO noise in crop mode. ISO noise have great impact on required bitrate to maintain all moving tiny noise dots.

Can I use fixed slice? To avoid autoadjusting of bitrate.
I heard that slice quality can be higher than QScale-16. Is it true?
Greetings from Russia!

1%

Yep, you can use fixed slice. Just set the lock to on and turn off leveling. Crop mode buffer isn't just noise, its bigger too.

xaled

Hi,

Is it "normal", that when you play recorded videos with changed GOP and QScale parameters in the camera (600d) you get all kind of strange color splashing over the screen?

If I play the same file on the pc, everything is fine.   

nanomad

EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

xaled

Ok, thanks, that's actially kind of obvious afterwards.
For a second I thought my cam was somehow broken.

Rush

I found bug in high bitrate handling.
It works like a charm with enabled sound up to 160 mbps stable. But works only in 25p mode! If I switch to 24p - buffer just fills in a few seconds. 24p works with enabled sound only with stock bitrate. Once I disable sound - high bitrates works nice.
But high bitrates in 25p mode works great both with and without sound! It is strange for me.
1%, what are your thoughts about it? (can I ask you to make new .bin with latest changes?)
Greetings from Russia!

1%

I want to rewrite the whole thing and try to use engio writes to set the slice instead of just changing it in memory... its still WIP since I've been working on 6D.... BUT sound works in 25p? That is good, need to find out what is different and make sound work in all modes. Audio functions aren't patched in theory it shouldn't be working anywhere so this is actually a good find.

Wow... holy shit... it really does. 25P is the only one.

N/A

Whoa. Tried it on my T3i, works perfectly with sound on, averaged 161 mbps. Why do you think this is? Surely 25p would require even more data than 24p??
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

1%

Hopefully I can find out and apply it to all modes... and hopefully audio stays in sync. 25P must not check it so we have to set up sound like it does all the time.

N/A

Not an expert on bitrate specifics, but this is what VLC was telling me, This is 25p, audio on, slice 87, GOP1. Holy shit.

7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

1%

I have to take a log while its doing it and compare to a log of when it mutes the audio.

I've started on the second version. Will be a little bit before everything is ported (and figured out). Audio on in all modes and digic set bit rate parameters might be the trick to get perfect videos (fixed fps override on digic V). Going to try to roll picoC into this one so we have scripting.. right now don't see too many scripts available. Hopefully can have everything both ways, cCBR, VBR and digic fixed + dynamic quality.

Getting the easy features out of the way first... hopefully audio on can work for all cameras that only do canon cbr. Have you checked if the audio stays in sync/mutes or skips? I guess DigicIV isn't tapped out yet.

*PS. This proves audio isn't being stopped by the buffer but instead canon bullshit.

Yoshiyuki Blade

Though this may be known already, I've found that FPS override with audio recorded separately allows for high bitrates. I don't know how high, but its much higher than default; I can maintain a constant Q-16 while recording indoors without a hitch, but the bitrate indoors rarely exceeds 100 Mbps, which is below what my card can handle.

Of course, the issue is with the sound sync. I still have no idea how to systematically sync the AV back together. I think it's mainly a problem with the true framerate that FPS override records at, which doesn't seem to be the FPS that it says it records at. Once I can figure that out, it should be cake.

ilguercio

But still, is it worth bumping up the CBR or the VBR value? For most users default is ok, some of us just want the ability to control P and I frames.
Canon EOS 6D, 60D, 50D.
Sigma 70-200 EX OS HSM, Sigma 70-200 Apo EX HSM, Samyang 14 2.8, Samyang 35 1.4, Samyang 85 1.4.
Proud supporter of Magic Lantern.

1%

I dunno... i'm not really going to mess with those modes, just leave them as options. I want to set slice per frame via digic + other parameters like deblocking, etc. The old functions changed everything in memory. Maybe it will be faster to skip that step or maybe it will let you enter more/higher values.

Would be interesting to see what happens with different gop/flush rate, 7D and video hacks have options for that but I don't have the locations for 600D.

Maybe try again to fix vsync with coutt's evf state thing.

At the least you get new menu + movie mode remap :)

Rush

Quote from: ilguercio on March 28, 2013, 01:24:29 AM
But still, is it worth bumping up the CBR or the VBR value? For most users default is ok, some of us just want the ability to control P and I frames.
I'd posted camparison earlier. High bitrate is better - much less compression artefacts with high ISOs.
Greetings from Russia!

mike.charal

Thanks to 1% and all the devs for your hard work and success. It looks like I'm quite late to join, but I read all 20 pages if that counts haha.

Forgive me if this is not the venue to pose this. I've been experimenting with the slice settings all night and I've found my card can sustain about 90 mbps for long periods of time. If I know what my scene complexity is and that it will not change significantly, I've had great success with locking the slice and disabling taper rates. I get a sustained 90 mbps, which is great for high ISO.

When I let the slices change depending on the buffer, I can't seem to keep them under control regardless of how I set everything up. It yoyos back and forth when the buffer fills and it's very obvious in the video that the quality drops out. In bitrate viewer, this particular video sits around 100 mbps but dips down to 20 mbps many times even though I've set the max bitrate not to exceed 85 and the min to 45.

Am I missing something obvious here? Maybe bitrate is dependent on something else entirely? As I understand it, the CBR multiplier doesn't really affect anything in slice mode. Would a faster card solve the issue of yoyoing? It's just a sandisk ultra 30 MB/s SDXC UHS-I 64GB. I just thought I'd be able to sustain a higher bitrate with that card than with the stock firmware encoding (like I said 80-90, not the 160 people are getting on here with extreme cards).

Thanks for any help! (Also, I'm set to GOP 3, deblock a/b=-1, PicQPC=-4 if that matters. GOP 3 seemed like a good balance for me too.)

Rush

mike.charal, do you try it with sound off? It is a problem to raise bitrate if sound is recording and it is not 25p mode.
Greetings from Russia!

1%

Set buffer warning level. That is what tells it to drop. I set mine to like 50%/60%

N/A

The PAL 25p setting with sound on works perfectly with slice 87 GOP 1. BUT  then under tungsten I get that weird rolling shadow look, coming from 25p and 60 hz not playing nice together I'm guessing? Need more tests but it happened every time I was using tungsten lighting.
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

Rush

Quote from: N/A on April 06, 2013, 09:39:01 PM
Yes, sure rolling shadows in NTSC country... But current implemetation of bitrate control have troubles when it is not 25p mode.
Try 24p or 30p with disabled sound...
Greetings from Russia!

N/A

Yeah I have been so far, but cam audio is a god send for lining up the video to recorded dialogue from the recorder. Ah well, can always have the actors clap I suppose.  ;)
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?