Author Topic: Bit rate investigation  (Read 105461 times)

driftwood

  • Freshman
  • **
  • Posts: 70
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #425 on: November 23, 2012, 12:25:36 PM »
ok thanks
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

DTSET123

  • Member
  • ***
  • Posts: 187
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #426 on: November 24, 2012, 09:57:08 AM »
Driftwood- I'm so glad see you join the game  :D

Marvin

  • New to the forum
  • *
  • Posts: 31
  • Personality is destiny
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #427 on: November 24, 2012, 11:59:35 AM »
@driftwood

please take a look at this post:
http://www.magiclantern.fm/forum/index.php?topic=1565.msg11808#msg11808

some tests have already been done, might save you some time :)
Film is truth 24 frames per second.

driftwood

  • Freshman
  • **
  • Posts: 70
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #428 on: November 24, 2012, 01:31:20 PM »
Good work Marvin:

Can you experiment with the two patches ratecontrol= in ciombination with maxbitrate / minbitrate ?

Change ratecontrol value to =1 then =0 together with entering values into min/max bitrate.

I'm still looking for a value for cpb ncoded picture buffer/dpb. Maybe Maxbitrate is like GH2's Top settings (declared bitrate) and Minbitrate could be a reinterpret of 'Bottom' settings which we worked out to be the CPB size.

We need higher coded picture buffer settings for adjusting the bitrate higher - ie they work in unison.

Failing this, somewhere there must be something in the code that declares a cpb. We need to find it.
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

Marvin

  • New to the forum
  • *
  • Posts: 31
  • Personality is destiny
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #429 on: November 26, 2012, 05:32:25 PM »
Good work Marvin:

Can you experiment with the two patches ratecontrol= in ciombination with maxbitrate / minbitrate ?

Change ratecontrol value to =1 then =0 together with entering values into min/max bitrate.

I'm still looking for a value for cpb ncoded picture buffer/dpb. Maybe Maxbitrate is like GH2's Top settings (declared bitrate) and Minbitrate could be a reinterpret of 'Bottom' settings which we worked out to be the CPB size.

We need higher coded picture buffer settings for adjusting the bitrate higher - ie they work in unison.

Failing this, somewhere there must be something in the code that declares a cpb. We need to find it.

both resulted in ERR70 when i tried
Film is truth 24 frames per second.

driftwood

  • Freshman
  • **
  • Posts: 70
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #430 on: November 27, 2012, 04:50:54 AM »
Together with testing out the new GH3 (love it!) I'm analysing 5dmkiii mov files to rip out SPS, etc... report back here soon.
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #431 on: December 09, 2012, 03:38:09 AM »
Found a side effect of gop changing.

Canon has a sync function when recording with audio enabled. If gop is changed sync makes an overrun error and then error 70s the camera. So if your recordings w/audio are erroring, turn gop back to default of 12 or 15 or 30 depending on fps. Don't know if can be patched out yet. Maybe also related to separate wav sync issues? Does this apply to 7d too?

driftwood

  • Freshman
  • **
  • Posts: 70
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #432 on: December 13, 2012, 03:38:02 AM »
I havent forgotten this thread. I hope to have a MKIII in my hands v soon to test.
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

wenen

  • New to the forum
  • *
  • Posts: 22
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #433 on: December 13, 2012, 11:37:01 AM »
Found a side effect of gop changing.

Canon has a sync function when recording with audio enabled. If gop is changed sync makes an overrun error and then error 70s the camera. So if your recordings w/audio are erroring, turn gop back to default of 12 or 15 or 30 depending on fps. Don't know if can be patched out yet. Maybe also related to separate wav sync issues? Does this apply to 7d too?

So what settings do you recommend for the 600D now?

DTSET123

  • Member
  • ***
  • Posts: 187
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #434 on: December 13, 2012, 01:43:29 PM »
@driftwood
Saw your short GH3 vs GH2 movie- funniest thing i have ever seen! Actors did a good job, well done!

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #435 on: December 13, 2012, 04:10:44 PM »
[quote[So what settings do you recommend for the 600D now?[/quote[

If you're shooting built in audio. You have to go back to stock gop. If you're shooting just video with audio off you can use whatever you want like gop 3, etc.

driftwood

  • Freshman
  • **
  • Posts: 70
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #436 on: December 18, 2012, 03:03:45 AM »
Ok Here's a look at the h264 encoder found inside the MKIII in-depth. Thanks to hjfilmspeed for supplying the test files.

First and foremost the encoder uses only 4x4 transform (along with the standard 16x16) and employs High Profile Level 5.1 for All-I and High Profile Level 4.1 for IPB. Everything is encoded using CAVLC (All-I and IPB) and NO scaling matrices are found to be used if my analysis using Elecard is correct.

Contrary to the Canon 5DMKiii encoder, the new Panasonic GH3 encoder uses Adaptive 8x8 and 4x4 transform and is thus 12% more efficient. It's All-I rec modes chooses High Profile lvl 5.0 CAVLC to encode - IPB employs CABAC High Profile level 4.1. All the GH3 modes use the same scaling matrix algorithms. So you can see there are marked differences between the GH3 and the 5DMK3.

I will attach graphs and sequence parameter sets for each of the Canon 5DMKIII modes shortly.

The dropbox link to the folder/files is;-

https://www.dropbox.com/sh/zqayfoniqo1s6dz/GO9zr30AMb

if you can't see the dropbox files for some reason (though Im sure sharing the link should work) email me at driftwood.brighton@gmail.com
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

wenen

  • New to the forum
  • *
  • Posts: 22
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #437 on: December 19, 2012, 01:05:36 PM »
[quote[So what settings do you recommend for the 600D now?[/quote[

If you're shooting built in audio. You have to go back to stock gop. If you're shooting just video with audio off you can use whatever you want like gop 3, etc.

Btw one more question: Do you have any new setting for the 600D to max out the quality? Have you done any comparison to the GH2?

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #438 on: December 19, 2012, 09:21:04 PM »
Ha Ha Glad to be of assistance drifftwood (tho i feel i hardly did anything compare to what all you guys are doing). I know nothing about how all this works but i do know a little bit about video and cinema. if theres anything else i can do to push forward the quality of video from the 5d3 or the all-i codec in general let me know!

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #439 on: December 20, 2012, 12:58:51 AM »
Quote
Do you have any new setting for the 600D to max out the quality

I have no GH2 (so can't compare) but Q87 in crop mode is camera max. Can't really do anything else to improve quality, just need a fast card to handle what is there. The only other improvements won't be H.264. As shown above, canon cameras are missing most encoder tweaks/settings even on 5dIII/6D/M.  With UHS-1 on 6D and similar slice QPD functions we'll see what DigicV can do.

wenen

  • New to the forum
  • *
  • Posts: 22
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #440 on: December 20, 2012, 12:35:39 PM »
I have no GH2 (so can't compare) but Q87 in crop mode is camera max. Can't really do anything else to improve quality, just need a fast card to handle what is there. The only other improvements won't be H.264. As shown above, canon cameras are missing most encoder tweaks/settings even on 5dIII/6D/M.  With UHS-1 on 6D and similar slice QPD functions we'll see what DigicV can do.

Ok I have my camera at Q87 and gop1 now. Shuld I use CBR 3.0x or VRB -16 settings?

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #441 on: December 20, 2012, 09:13:45 PM »
CBR and VBR and quality by slice are all mutually exclusive. CBR3x will just increase frame prediction sizes and hopefully pick a better quality. You want lock slice and slice set to 87 in "CBR" mode, the x won't matter (just keep it over 1x), it is not used. Old CBR only gets used when slice is 0. If you see Q scale indicators on the screen then its in CBR or VBR. if you see slice then you're in slice mode.
Q87 is above qscale -16 (this is like slice 112)

There is a guide a few posts back on what the settings do.

wenen

  • New to the forum
  • *
  • Posts: 22
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #442 on: December 21, 2012, 02:10:10 PM »
Watching my difference test.

200% zoom



1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #443 on: December 22, 2012, 05:49:00 PM »
Yea its not super dramatic. Main difference is when grading you have more robust footage and you get to capture fine details. Also gop1 has different motion blur. Sometimes you max out your scene. YUV buffer isn't modified so its still compressing the same image... just less. Also a couple of P frames sometimes helps like gop3. Tests were done months ago on fine details. You're scene is slightly under so a lot of stuff is going to get crushed and hidden already.

Waiting to try this on the 6D where there is bandwidth to spare and UHS-1. Hoping to crack 250Mbps but we'll see. You also get the other features like named bracketing/remap/etc.

Yoshiyuki Blade

  • Member
  • ***
  • Posts: 103
  • 5D2
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #444 on: January 04, 2013, 03:05:36 AM »
First post on the "new" forums, but I've been following ML for a while and donated a little a while back. Thanks for all your hard work, everyone! I've only recently found this thread and it seems very exciting. I'm curious about where you guys are headed with this now.

The most recent posts seem to focus more on the h.264 side of things, but a couple months ago, there was crazy talk about implementing an mjpeg encoder directly with the raw 422 data, which was wonderful to imagine. It had me thinking about how much goes into the pipeline of image processing at the moment you press that record button. Where do you suppose would be the best place to min/max for video quality?

With my 5D2, I found that the silent pics written while recording (1872x1080) don't really represent the actual resolution being recorded. I've done subjective tests with various resolutions listed in the 422toimage converter xml files and it's probably 1720x974. If bypassing the upscale process to 1080p is possible, that alone would probably help a lot in increasing the overall quality of the video. Less bits is required for the same quality and less resolution to encode frees up more cpu performance. And boy does audio take up a ton of overhead! With audio on, the video bitrate can only go up to the 60 megabits before crashing, but can go up to the 120s with it off. I can record at Q-16 constantly in fairly busy scenes that way. What's the deal with that lol.

I also noticed that the difference between the (converted) raw 422 image and when I downsample it to 420 is very negligible. However, I haven't tested it on bright vivid reds yet, where perceiving the difference in chroma resolution is most apparent. I wonder if trading performance for chroma resolution is worth the extra cost. Anyway, thanks again guys. I'm not literate in programming or understanding the low level stuff that goes on, but I do like to get into the nitty gritty technical details of video as much as possible.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #445 on: January 06, 2013, 07:46:32 PM »
Canon does use adaptive 8x8

H264.ini: Transform8x8Flag = 2

Also scaling matrices can be chosen.

ScalingMatrices = 0
pScalingMatrixAddr[0] = 0
pScalingMatrixAddr[1] = 0
pScalingMatrixAddr[2] = 0
pScalingMatrixAddr[3] = 0
pScalingMatrixAddr[4] = 0
pScalingMatrixAddr[5] = 0

You just looked at default encoded videos. This needs more investigation.

feureau

  • Hero Member
  • *****
  • Posts: 604
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #446 on: January 18, 2013, 04:50:35 PM »
Found a side effect of gop changing.

Canon has a sync function when recording with audio enabled. If gop is changed sync makes an overrun error and then error 70s the camera. So if your recordings w/audio are erroring, turn gop back to default of 12 or 15 or 30 depending on fps. Don't know if can be patched out yet. Maybe also related to separate wav sync issues? Does this apply to 7d too?

I've been playing with flush and GOP on the 7D. I know this much:

When GOP is set to 1, and audio recording turned on, you have to set the flush to match the video framerate. For instance, if you shoot at 24p, the flush has to be set at 24, or it will stop recording after a few moments.

On the ML alpha 2 for 7D, the flush can be set to maximum of 50. If I use 720p and shoot at 50p, with GOP at 1, with audio recording on, it will record up to the full 4 gig.

But when the 720p is at 60, since flush can't be set at 60, with GOP at 1, the recording will fail after a few seconds.

The GOP number doesn't matter much in regards to erroring, but the flush is key.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #447 on: January 18, 2013, 05:38:05 PM »
On 600D its one setting. Gop is actualy fps/2 per default even on 6D.

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3183
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #448 on: January 18, 2013, 07:07:42 PM »
Found a side effect of gop changing.

Canon has a sync function when recording with audio enabled. If gop is changed sync makes an overrun error and then error 70s the camera. So if your recordings w/audio are erroring, turn gop back to default of 12 or 15 or 30 depending on fps. Don't know if can be patched out yet. Maybe also related to separate wav sync issues? Does this apply to 7d too?

didnt notice this post earlier.
yeah thats the reason why 7D alpha 2 disables audio when video hacks are active.
didnt patch the MOVW_AddAudio routine yet.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #449 on: January 19, 2013, 01:18:08 AM »
If you figure it out, let me know. I can probably patch the assert but even if I do it might just mute/stop audio anyway.