Author Topic: Bit rate investigation  (Read 74328 times)

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #400 on: November 07, 2012, 09:43:52 PM »
For now this is where its at. Better than everything except 5DIII and dedicated video cameras.

Also, one thing I left out: Set slice to 0 to use canon CBR.

skydragon

  • New to the forum
  • *
  • Posts: 22
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #401 on: November 09, 2012, 09:57:33 PM »
Quote
Better than everything except 5DIII and dedicated video cameras

Just talking about pure 1080P HD Video quality (not build quality or camera ergonomics or features) don't you think a hacked Panasonic GH2 can produce 'better' HD video that an EOS currently can ?

I know the 'soft' video look a Canon EOS camera produces can be very nice for some uses, but at the end of the day although the EOS file output is 1080p, you could argue it isn't actually a 1080 HD video at all, due to the internal camera resolutions and upscaling etc.

I love my 600D but really wish there was a way of getting better sharper, higher resolution video out of it...
Canon 600D with Battery Grip, ML 2.3, Sigma 17-50mm 2.8, Swivi viewfinder loupe, RJ Follow focus. Loupe, FF and Camera/Battery Grip mounted on a 4mm thick alloy plate.

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #402 on: November 13, 2012, 09:54:16 PM »
Just talking about pure 1080P HD Video quality (not build quality or camera ergonomics or features) don't you think a hacked Panasonic GH2 can produce 'better' HD video that an EOS currently can ?

I know the 'soft' video look a Canon EOS camera produces can be very nice for some uses, but at the end of the day although the EOS file output is 1080p, you could argue it isn't actually a 1080 HD video at all, due to the internal camera resolutions and upscaling etc.

I love my 600D but really wish there was a way of getting better sharper, higher resolution video out of it...

This topic would be good to read up on:  http://www.magiclantern.fm/forum/index.php?topic=3492.0
and this one too: http://www.magiclantern.fm/forum/index.php?topic=3489.0
At this point i totally agree. I just want want amazing detail from the canon dslr video. I dont think its gunna happen though for under $7k from canon.

3pointedit

  • Senior
  • ****
  • Posts: 263
  • 550D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #403 on: November 14, 2012, 10:08:30 AM »
Ha, but it can happen from a sub $1k sony handycam...
550D on ML-roids

skydragon

  • New to the forum
  • *
  • Posts: 22
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #404 on: November 15, 2012, 05:35:40 PM »
I'm sad to say I've now defected over to a Panasonic GH2 ;-)

The build quality is poor compared to a Canon, but the 1080p 24fps AVCHD video, hacked to >60Mbps is simply amazing quality. Comparing it side-by-side with a friends 5d MkII or my 600D it's a massive improvement in resolution, not just a subtle change.

...I'll come back to Canon as soon as they make a DSLR with proper 1080p video rather than the current 'soft' upscaled offering.

I really do miss the great features that ML provides, it really does make using a DSLR so much easier and it's one of the downsides of using a GH2 rather than an EOS.

Hope to return back to Canon, in a few canon model generations time, if and when they get their act together ;-)

in the meantime thanks and best wishes to the guys at ML.
Canon 600D with Battery Grip, ML 2.3, Sigma 17-50mm 2.8, Swivi viewfinder loupe, RJ Follow focus. Loupe, FF and Camera/Battery Grip mounted on a 4mm thick alloy plate.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #405 on: November 18, 2012, 05:59:49 PM »
Should compare slice 87 600D vs GH2, would be interesting to see. I thought GH2 uprezzed too but maybe that was only GH1.

KahL

  • Freshman
  • **
  • Posts: 64
  • kahleem poole-tejada.CINEMATOGRAPHER
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #406 on: November 21, 2012, 06:27:20 AM »
Crap, I am SO lost, guys. How do I setup and install this version?

driftwood

  • Freshman
  • **
  • Posts: 70
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #407 on: November 21, 2012, 04:53:07 PM »
Seems to be some good work going on here. Where are we at in terms of encoder patching? I could be of some help - haven't ML'd my 5DmkII , 7D or 60D yet but you never know...

The 5DMKII, 7D 60D are h264 Profile IDC baseline level 5 which isn't a brilliant starting point... but according to the spec allows a maximum video bitrate of 135,000. Max decoded picture buffer of 110,400 at 1920x1080. (Having said that, if you look at the h264 stream / PPS there could the usual baseline 66 constraints on these three cameras)

Have you found a way of raising the decoder picture buffer yet (dpb/cpb)? Youve got to stop the buffer underuns/overruns and tests need to be done on buffer analysis / stream analaysis software (like elecard or CodecVisa, etc...)

Have any Quantisation scaling matrices been found ? Are they being employed?

With the 5DMKIII being level 5.1 High profile (thats slightly higher than the new Pany GH3 you should be able to do something. You've got adaptive 8x8/4x4 transform, scaling tables, cabac or cavlc, in-loop deb locking etc...

Here's the stock bitrates of the GH3 in ALL-I 72Mbps .mov mode.

Bit Rate = 71680000 (69999 = bit_rate_value_minus1/SchedSelIdx/*0*...)
CPB Size (the coded picture buffer) = 64512000 (62999 = cpb_size_value_minus1(SchedSelIdx/*0*...)

Intitial QP=20/ QP=20...

Actually, someone email me a very small All-I file / dropbox me. driftwood.brighton@gmail.com

Hmmm... need to get hold of a MKIII.



Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #408 on: November 21, 2012, 07:13:52 PM »
Wow is this the Drifwood that made the gh2 hacks hard round the world? (Not sarcastic I'm genuinely excited)
I'd be more then happy to send u a clip when I get a chance. I'm desperate for better sharper video from the mark III at this point!

CaptainHook

  • Freshman
  • **
  • Posts: 52
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #409 on: November 21, 2012, 07:18:48 PM »
It would be pretty epic if Driftwood started to help out with ML !!

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #410 on: November 21, 2012, 07:32:08 PM »
Driftwood from this video?
:)

For 5D Mark III, this post summarizes our findings pretty well.

Default encoder settings:
H264-alli.ini
H264-ipb.ini

(not sure what all these things mean, some can be changed, others give err70)

Shizuka

  • New to the forum
  • *
  • Posts: 36
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #411 on: November 21, 2012, 07:34:33 PM »
Seems to be some good work going on here. Where are we at in terms of encoder patching? I could be of some help - haven't ML'd my 5DmkII , 7D or 60D yet but you never know...

The 5DMKII, 7D 60D are h264 Profile IDC baseline level 5 which isn't a brilliant starting point... but according to the spec allows a maximum video bitrate of 135,000. Max decoded picture buffer of 110,400 at 1920x1080. (Having said that, if you look at the h264 stream / PPS there could the usual baseline 66 constraints on these three cameras)

Have you found a way of raising the decoder picture buffer yet (dpb/cpb)? Youve got to stop the buffer underuns/overruns and tests need to be done on buffer analysis / stream analaysis software (like elecard or CodecVisa, etc...)

Have any Quantisation scaling matrices been found ? Are they being employed?

With the 5DMKIII being level 5.1 High profile (thats slightly higher than the new Pany GH3 you should be able to do something. You've got adaptive 8x8/4x4 transform, scaling tables, cabac or cavlc, in-loop deb locking etc...

Here's the stock bitrates of the GH3 in ALL-I 72Mbps .mov mode.

Bit Rate = 71680000 (69999 = bit_rate_value_minus1/SchedSelIdx/*0*...)
CPB Size (the coded picture buffer) = 64512000 (62999 = cpb_size_value_minus1(SchedSelIdx/*0*...)

Intitial QP=20/ QP=20...

Actually, someone email me a very small All-I file / dropbox me. driftwood.brighton@gmail.com

Hmmm... need to get hold of a MKIII.

For the non-5D3 cameras, there's nothing that really can be done:
QSM isn't available in baseline profile, and maxDPB is just a specification specifying how many reference frames can be used for video. For canon's encoder, it only uses the previous frame to be predicted from, ie. 1 reference frame.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #412 on: November 21, 2012, 07:44:10 PM »
I don't know how far they actually implement the profiles. Seems like they just add features the profile should have and call it a day. The bit rates and some other stuff  violate baseline.

Kabuto1138

  • Freshman
  • **
  • Posts: 87
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #413 on: November 21, 2012, 10:20:52 PM »
So, is high bitrates for the 5d2 etc. officially not happening?


hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #414 on: November 22, 2012, 12:51:18 AM »
Driftwood and ML! shhhoooooooo weeeee this is exciting!

Is it possible to have bit rate options on the 5d3 alpha versions that run off the card? i like the idea of running my off the card. seems allot safer.

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3654
  • Blunt and to the point
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #415 on: November 22, 2012, 06:12:41 AM »
Default encoder settings:
H264-alli.ini
H264-ipb.ini

(not sure what all these things mean, some can be changed, others give err70)

"Transform8x8Flag = 2"  8x8 DCT increases quality and implies High Profile.  God only knows what that flag changes though.  Would need to adjust the setting and analyze the encoded video.

"EntropyCodingMode = 0"  CABAC.  http://en.wikipedia.org/wiki/Entropy_encoding

"IntraPicInitQP = 20 - InterPicInitQP = 20"  Initial encoded QP?  Lower values would increase quality but increase stress on the buffer.

"QpOffsetForB = 0"  QP offset for B-frames?  http://mewiki.project357.com/wiki/X264_Settings#pbratio

"MinQpI = 10 - MinQpP = 10 - MinQpB = 10 - MaxQpI = 51 - MaxQpP = 51 - MaxQpB = 51"  Min and Max Quantizer to use for encoding.  MinQP should be reduced to 0, but might put to much stress on the buffer.  MaxQP should be left alone.  MinQP could be reduced to 0 if used in conjunction with,
"MinBitrate = 0 - MaxBitrate = 0"

"RateControlEnable = 2"  Different rate control methods?  Again, would need to adjust and analyze the encoded video.

driftwood

  • Freshman
  • **
  • Posts: 70
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #416 on: November 22, 2012, 02:40:09 PM »
Yes, its Driftwood from GH2 settings, anyone gotta problem with that!?

Yeah, judging by baseline 66 work I did on the GX1 camera there's not much one can do with any of the canons prior to the 5DmkIII - sure you can get higher bitrates and change GOP but most baseline profiles are constrained. See constrain flags in the h264 SPS set.

So the 5Dmk3 has distinct possibilities: Gonna have to sell the other canons to buy one ;-)


Level 5 mode:

I would perhaps leave the initial QPs alone - anything lower than 16 causes problems. There is also minimum QP for each P_slice type at 10 - the GH3 has something similar I believe. Subsequently, in these recording modes a scaling matrix is not being used/implemented. As a whole Scalers work best at coding the luma frequencies better - ring artefacts etc... but Canon's encoder interp. probably deems them unneccesary. Are the recordings when analysed in SPS/PPS using QP_delta offsets like =0 (no change in QP from previous prediction) or other values (e.g.-4, -6, -8, -10) that indicate a change of QP?

QpOffsetForB = 0 For B frames no QP offset variation is being employed?


Transform8x8Flag = 2 probably means adaptive 8x8/4x4 transform on residuals - ie most of the picture is coded with 8x8 and some of the luma residuals are receiving 4x4. My 'hunch' is if it were =0 all luma residual samples are receiving 4x4 block transform, or =1 are luma samples are in residual 8x8 block transform. I would leave that alone if this was confirmed as the case. A quick check with stream analysis will confirm transform being used.

EntropyCodingMode = 0
You need to establish the order of the setting e.g. does = 0 mean cavlc, 1= cabac, 3= Exp Golomb vlc - one or the other. (therefore probably 0, 1 or 2 are the available settings)

RateControlEnable = 2 - needs experimenting (first with =0, then =1) Could be some sort of adaptive rate control schemes. Could be any of the following:-

=0 - Original quadratic rate control scheme based on JVT spec
=1 - Extension of quadratic scheme for all Intra and IBsBsBs… coding.
=2 - Basic extension of quadratic scheme to better support hierarchical codingstructures
=3 - Extension of quadratic scheme with slice type separation

-------
Level: 4.1 modes:

IntraPicInitQP = 28
InterPicInitQP = 28

I see no reason why these cant be improved (changed to =20 as a starting point )  with tests on added bitrate to correct rate control.

I really need the developer to find a patch in their firmware/ROM/RAM loadables for coded picture buffer (surely there's something in there?) as just adding bitrate requires more buffer bitrate (whats the ceiling?) In the GH2 ptools we have it as a 'bottom' setting, 'Top' setting being bitrate. Also anything regarding Frame buffers (and sizes), and frame size in bits?

Can someone load up a 5dmk3 PPS produced from elecard or any other h264 stream analysis software?
And can someone please email me a few small sized original .mov recordings from the mk3?

more to come...
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #417 on: November 22, 2012, 06:01:28 PM »
ill send ya some stuff. just shot a bunch.

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #418 on: November 22, 2012, 06:30:31 PM »
ok driftwood just sent you a bunch of clips but i have no idea if they went through. they are probably to big for email.

driftwood

  • Freshman
  • **
  • Posts: 70
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #419 on: November 22, 2012, 08:01:27 PM »
You can always upload to a file share site or use dropbox to share sample files.
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #420 on: November 22, 2012, 10:00:28 PM »
You can always upload to a file share site or use dropbox to share sample files.

Did you get em? sent them here.

driftwood

  • Freshman
  • **
  • Posts: 70
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #421 on: November 22, 2012, 10:26:16 PM »
Not yet  - do you have my correct email? Link me. Itd be easier.
Canon 60D, Canon 5DMK3, Lexar 1000x 128GB CF, Panasonic (shhhh!) GH2s & GH3s. :-)

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #422 on: November 23, 2012, 12:36:48 AM »
Quote
there's not much one can do with any of the canons prior to the 5DmkIII

Its not really so much baseline as my build already violates it according to avi-naptic. Its that all you can set in the older canons is gop, slice quality, de-blocking filter and qpy and qpc offset. There is a set of parameters sent directly to the "JPCORE" encoder but I haven't been able to figure them out, its just a bunch of numbers put into registers.

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #423 on: November 23, 2012, 03:56:24 AM »
Not yet  - do you have my correct email? Link me. Itd be easier.

driftwood.brighton@gmail.com

Is this correct?

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #424 on: November 23, 2012, 04:04:18 AM »
just sent you a dropbox with 1080 30p all-i and ipb 1080 24p all-i and ipb 720 60p all-i and ipb Mr Driftwood. should be 6 clips when there done syncing