Bit rate investigation

Started by Audionut, July 19, 2012, 04:54:03 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

driftwood

Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

DTSET123


Marvin

Film is truth 24 frames per second.

driftwood

Good work Marvin:

Can you experiment with the two patches ratecontrol= in ciombination with maxbitrate / minbitrate ?

Change ratecontrol value to =1 then =0 together with entering values into min/max bitrate.

I'm still looking for a value for cpb ncoded picture buffer/dpb. Maybe Maxbitrate is like GH2's Top settings (declared bitrate) and Minbitrate could be a reinterpret of 'Bottom' settings which we worked out to be the CPB size.

We need higher coded picture buffer settings for adjusting the bitrate higher - ie they work in unison.

Failing this, somewhere there must be something in the code that declares a cpb. We need to find it.
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

Marvin

Quote from: driftwood on November 24, 2012, 01:31:20 PM
Good work Marvin:

Can you experiment with the two patches ratecontrol= in ciombination with maxbitrate / minbitrate ?

Change ratecontrol value to =1 then =0 together with entering values into min/max bitrate.

I'm still looking for a value for cpb ncoded picture buffer/dpb. Maybe Maxbitrate is like GH2's Top settings (declared bitrate) and Minbitrate could be a reinterpret of 'Bottom' settings which we worked out to be the CPB size.

We need higher coded picture buffer settings for adjusting the bitrate higher - ie they work in unison.

Failing this, somewhere there must be something in the code that declares a cpb. We need to find it.

both resulted in ERR70 when i tried
Film is truth 24 frames per second.

driftwood

Together with testing out the new GH3 (love it!) I'm analysing 5dmkiii mov files to rip out SPS, etc... report back here soon.
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

1%

Found a side effect of gop changing.

Canon has a sync function when recording with audio enabled. If gop is changed sync makes an overrun error and then error 70s the camera. So if your recordings w/audio are erroring, turn gop back to default of 12 or 15 or 30 depending on fps. Don't know if can be patched out yet. Maybe also related to separate wav sync issues? Does this apply to 7d too?

driftwood

I havent forgotten this thread. I hope to have a MKIII in my hands v soon to test.
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

wenen

Quote from: 1% on December 09, 2012, 03:38:09 AM
Found a side effect of gop changing.

Canon has a sync function when recording with audio enabled. If gop is changed sync makes an overrun error and then error 70s the camera. So if your recordings w/audio are erroring, turn gop back to default of 12 or 15 or 30 depending on fps. Don't know if can be patched out yet. Maybe also related to separate wav sync issues? Does this apply to 7d too?

So what settings do you recommend for the 600D now?

DTSET123

@driftwood
Saw your short GH3 vs GH2 movie- funniest thing i have ever seen! Actors did a good job, well done!

1%

[quote[So what settings do you recommend for the 600D now?[/quote[

If you're shooting built in audio. You have to go back to stock gop. If you're shooting just video with audio off you can use whatever you want like gop 3, etc.

driftwood

Ok Here's a look at the h264 encoder found inside the MKIII in-depth. Thanks to hjfilmspeed for supplying the test files.

First and foremost the encoder uses only 4x4 transform (along with the standard 16x16) and employs High Profile Level 5.1 for All-I and High Profile Level 4.1 for IPB. Everything is encoded using CAVLC (All-I and IPB) and NO scaling matrices are found to be used if my analysis using Elecard is correct.

Contrary to the Canon 5DMKiii encoder, the new Panasonic GH3 encoder uses Adaptive 8x8 and 4x4 transform and is thus 12% more efficient. It's All-I rec modes chooses High Profile lvl 5.0 CAVLC to encode - IPB employs CABAC High Profile level 4.1. All the GH3 modes use the same scaling matrix algorithms. So you can see there are marked differences between the GH3 and the 5DMK3.

I will attach graphs and sequence parameter sets for each of the Canon 5DMKIII modes shortly.

The dropbox link to the folder/files is;-

https://www.dropbox.com/sh/zqayfoniqo1s6dz/GO9zr30AMb

if you can't see the dropbox files for some reason (though Im sure sharing the link should work) email me at [email protected]
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

wenen

Quote from: 1% on December 13, 2012, 04:10:44 PM
[quote[So what settings do you recommend for the 600D now?[/quote[

If you're shooting built in audio. You have to go back to stock gop. If you're shooting just video with audio off you can use whatever you want like gop 3, etc.

Btw one more question: Do you have any new setting for the 600D to max out the quality? Have you done any comparison to the GH2?

hjfilmspeed

Ha Ha Glad to be of assistance drifftwood (tho i feel i hardly did anything compare to what all you guys are doing). I know nothing about how all this works but i do know a little bit about video and cinema. if theres anything else i can do to push forward the quality of video from the 5d3 or the all-i codec in general let me know!

1%

QuoteDo you have any new setting for the 600D to max out the quality

I have no GH2 (so can't compare) but Q87 in crop mode is camera max. Can't really do anything else to improve quality, just need a fast card to handle what is there. The only other improvements won't be H.264. As shown above, canon cameras are missing most encoder tweaks/settings even on 5dIII/6D/M.  With UHS-1 on 6D and similar slice QPD functions we'll see what DigicV can do.

wenen

Quote from: 1% on December 20, 2012, 12:58:51 AM
I have no GH2 (so can't compare) but Q87 in crop mode is camera max. Can't really do anything else to improve quality, just need a fast card to handle what is there. The only other improvements won't be H.264. As shown above, canon cameras are missing most encoder tweaks/settings even on 5dIII/6D/M.  With UHS-1 on 6D and similar slice QPD functions we'll see what DigicV can do.

Ok I have my camera at Q87 and gop1 now. Shuld I use CBR 3.0x or VRB -16 settings?

1%

CBR and VBR and quality by slice are all mutually exclusive. CBR3x will just increase frame prediction sizes and hopefully pick a better quality. You want lock slice and slice set to 87 in "CBR" mode, the x won't matter (just keep it over 1x), it is not used. Old CBR only gets used when slice is 0. If you see Q scale indicators on the screen then its in CBR or VBR. if you see slice then you're in slice mode.
Q87 is above qscale -16 (this is like slice 112)

There is a guide a few posts back on what the settings do.

wenen


1%

Yea its not super dramatic. Main difference is when grading you have more robust footage and you get to capture fine details. Also gop1 has different motion blur. Sometimes you max out your scene. YUV buffer isn't modified so its still compressing the same image... just less. Also a couple of P frames sometimes helps like gop3. Tests were done months ago on fine details. You're scene is slightly under so a lot of stuff is going to get crushed and hidden already.

Waiting to try this on the 6D where there is bandwidth to spare and UHS-1. Hoping to crack 250Mbps but we'll see. You also get the other features like named bracketing/remap/etc.

Yoshiyuki Blade

First post on the "new" forums, but I've been following ML for a while and donated a little a while back. Thanks for all your hard work, everyone! I've only recently found this thread and it seems very exciting. I'm curious about where you guys are headed with this now.

The most recent posts seem to focus more on the h.264 side of things, but a couple months ago, there was crazy talk about implementing an mjpeg encoder directly with the raw 422 data, which was wonderful to imagine. It had me thinking about how much goes into the pipeline of image processing at the moment you press that record button. Where do you suppose would be the best place to min/max for video quality?

With my 5D2, I found that the silent pics written while recording (1872x1080) don't really represent the actual resolution being recorded. I've done subjective tests with various resolutions listed in the 422toimage converter xml files and it's probably 1720x974. If bypassing the upscale process to 1080p is possible, that alone would probably help a lot in increasing the overall quality of the video. Less bits is required for the same quality and less resolution to encode frees up more cpu performance. And boy does audio take up a ton of overhead! With audio on, the video bitrate can only go up to the 60 megabits before crashing, but can go up to the 120s with it off. I can record at Q-16 constantly in fairly busy scenes that way. What's the deal with that lol.

I also noticed that the difference between the (converted) raw 422 image and when I downsample it to 420 is very negligible. However, I haven't tested it on bright vivid reds yet, where perceiving the difference in chroma resolution is most apparent. I wonder if trading performance for chroma resolution is worth the extra cost. Anyway, thanks again guys. I'm not literate in programming or understanding the low level stuff that goes on, but I do like to get into the nitty gritty technical details of video as much as possible.

1%

Canon does use adaptive 8x8

H264.ini: Transform8x8Flag = 2

Also scaling matrices can be chosen.

ScalingMatrices = 0
pScalingMatrixAddr[0] = 0
pScalingMatrixAddr[1] = 0
pScalingMatrixAddr[2] = 0
pScalingMatrixAddr[3] = 0
pScalingMatrixAddr[4] = 0
pScalingMatrixAddr[5] = 0

You just looked at default encoded videos. This needs more investigation.

feureau

Quote from: 1% on December 09, 2012, 03:38:09 AM
Found a side effect of gop changing.

Canon has a sync function when recording with audio enabled. If gop is changed sync makes an overrun error and then error 70s the camera. So if your recordings w/audio are erroring, turn gop back to default of 12 or 15 or 30 depending on fps. Don't know if can be patched out yet. Maybe also related to separate wav sync issues? Does this apply to 7d too?

I've been playing with flush and GOP on the 7D. I know this much:

When GOP is set to 1, and audio recording turned on, you have to set the flush to match the video framerate. For instance, if you shoot at 24p, the flush has to be set at 24, or it will stop recording after a few moments.

On the ML alpha 2 for 7D, the flush can be set to maximum of 50. If I use 720p and shoot at 50p, with GOP at 1, with audio recording on, it will record up to the full 4 gig.

But when the 720p is at 60, since flush can't be set at 60, with GOP at 1, the recording will fail after a few seconds.

The GOP number doesn't matter much in regards to erroring, but the flush is key.

1%

On 600D its one setting. Gop is actualy fps/2 per default even on 6D.

g3gg0

Quote from: 1% on December 09, 2012, 03:38:09 AM
Found a side effect of gop changing.

Canon has a sync function when recording with audio enabled. If gop is changed sync makes an overrun error and then error 70s the camera. So if your recordings w/audio are erroring, turn gop back to default of 12 or 15 or 30 depending on fps. Don't know if can be patched out yet. Maybe also related to separate wav sync issues? Does this apply to 7d too?

didnt notice this post earlier.
yeah thats the reason why 7D alpha 2 disables audio when video hacks are active.
didnt patch the MOVW_AddAudio routine yet.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

1%

If you figure it out, let me know. I can probably patch the assert but even if I do it might just mute/stop audio anyway.