Author Topic: Bit rate investigation  (Read 74327 times)

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #300 on: September 19, 2012, 04:28:53 PM »
Thanks for the test JasonATL. Downloading the original for a better look. Personally I've found GOP3 seems to be the general sweetspot for bitrate vs quality.

EDIT: just looked at the original. The zoomed part of the video really shows the difference between standard Canon CBR and increased BR. Nice job! It will certainly help with chromakey work and when upscaling 720p footage.

Just a thought about resolution. Some guys removed the OLPF from their 5d MkIII's and the resolution was much better though obviously more susceptible to moire and aliasing. Has anyone tried this with a lower spec camera? secondhand 550d's are going very cheap these days so might be worth risking?
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

Marvin

  • New to the forum
  • *
  • Posts: 31
  • Personality is destiny
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #301 on: September 19, 2012, 04:46:31 PM »
For H264 profiles and levels, here are some information for reference:

http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Profiles

http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

With H264 parameters, here is a table that correspond each profile and its value:

Profile_IDC:

66   Baseline
77   Main
88   Extended

(FRExt profies):

100   High
110   High 10bit
122  High 4:2:2
244   High 4:4:4
44    CAVLC 4:4:4 Intra
118  Multiview High Profile
128  Stereo High Profile

IntraProfile = 0/1, Activate Intra Profile for FRExt (0: false, 1: true, e.g. Profile_IDC=110, IntraProfile=1  =>  High 10 Intra Profile)

Levels are very simple, Level_IDC = 51 means 5.1, 41 means 4.1, just take out the dot :-)
Film is truth 24 frames per second.

Marvin

  • New to the forum
  • *
  • Posts: 31
  • Personality is destiny
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #302 on: September 19, 2012, 04:51:42 PM »
Just a thought about resolution. Some guys removed the OLPF from their 5d MkIII's and the resolution was much better though obviously more susceptible to moire and aliasing. Has anyone tried this with a lower spec camera? secondhand 550d's are going very cheap these days so might be worth risking?

Removing OLPF won't help much about resolution, it was just an urban legend, those people who removed it have already put it back because it brings significant downsides to the image, such as serious IR pollution, focus shift and dust.
Film is truth 24 frames per second.

Rush

  • Member
  • ***
  • Posts: 159
  • 5D3, EOS M
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #303 on: September 19, 2012, 04:56:02 PM »
Just a thought about resolution. Some guys removed the OLPF from their 5d MkIII's and the resolution was much better though obviously more susceptible to moire and aliasing. Has anyone tried this with a lower spec camera? secondhand 550d's are going very cheap these days so might be worth risking?
As I know, later tests showed that there were no improvement in detail with removed OLPF. And the image has red tint (no IR filter) which breaks correct colors.
Greetings from Russia!

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #304 on: September 19, 2012, 05:02:56 PM »
Thanks guys. re: OLPF - I remember thinking it was a crazy idea at the time but didn't check back to see if they had stuck with it.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #305 on: September 19, 2012, 05:23:33 PM »
Profile is not that simple. The numbers don't match. Don't know exactly where to set.

Code: [Select]
if arg2 == 0 /*EQ*/:
                *0xC0E1000C = 41104 < Normal Jpeg?
                DryosDebugMsg(0x15, 0x2, '[JPCORE] JP62_OPMR3 %#lx', 0xa090) => ret_DryosDebugMsg_FF1E35E4
            if arg2 != 0 /*NE*/:
                *0xC0E1000C = 41091 < H264
                DryosDebugMsg(0x15, 0x2, '[JPCORE] H264_SPS_PPS JP62_OPMR3 %#lx', 0xa083) => ret_DryosDebugMsg_FF1E35E4


What are some lower FRext profiles.

I find 88 and 12 being thrown around in there.

jgharding

  • New to the forum
  • *
  • Posts: 43
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #306 on: September 19, 2012, 05:29:07 PM »
Something I've not quite grasped: what stage are picture profiles applied at? I'm looking for a way to get optimum bitrate for 4:0:0, that is to say, black and white. Would an All-|I hack and monochrome picture profile be the best way?
Zeiss primes, 600D, a lot of shadow. http://www.jgharding.com

JasonATL

  • Freshman
  • **
  • Posts: 54
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #307 on: September 19, 2012, 05:30:04 PM »
Andy600 - I don't think that a 550D has a very powerful OLPF. Otherwise, we wouldn't have the moire problem that we do. Still, it could have one. Perhaps when I finally get my own 5D3 (the one I use from time to time is my wife's), I'll have the guts to take my 550D apart. (because at that point, it will be fourth instead of third). I don't believe (I use this word purposefully) that removing the filter improves true resolution on the 5D3. In the end, this shouldn't be a matter of "faith" (e.g., I shouldn't believe or not believe it). But, right now, people are using faith rather than evidence, because there has been no evidence. I have not seen a side-by-side controlled experiment nor have I seen an actual resolution chart shot with video on a camera that removed the OLPF on the 5D3. I recall that James Miller shot the same scene (one with and one without the OLPF), but on different days in different light. Does perceived resolution increase? Perhaps. True resolution? I doubt it. But, I'm willing to believe it if I see the evidence. As I said, it shouldn't be about faith.
 
People believed the Nikon D800 had much higher resolution than the 5D3 (we're talking video here). In my tests of shooting a resolution chart, the D800 comes in at about the same resolution as the 5D3. If any, it MIGHT have about 20 more lines. But, if the 5D3 is about 800 lines, it isn't as if the D800 is Full 1080p (and the moire you get with the D800!). My point is that perceived sharpness and actual resolution are two different things. This is one reason that I'm happy about having more latitude in sharpening the 600D (and the 5D3) footage with improved (or less?) compression.

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #308 on: September 19, 2012, 06:15:09 PM »
Resolution was a wrong choice of words from me. I think there is a definite increase in sharpness without the OLPF in James Millers' test shots but the IR and dust problems caused by removing it seem a step to far.

BTW JasonATL what settings were you using for the GOP test? Did you just increase the CBR and change GOP settings? Also did you alter DBlock A/B settings?
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

JasonATL

  • Freshman
  • **
  • Posts: 54
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #309 on: September 19, 2012, 06:47:27 PM »
BTW JasonATL what settings were you using for the GOP test? Did you just increase the CBR and change GOP settings? Also did you alter DBlock A/B settings?

On the waterfall video, I don't know (I wasn't keeping very good notes  :-[ -- I'm sure about the GOP, though, since I can infer it in Bitviewer). I have tried different settings for PicPC, DBlock A/B (with notes). There appear to be differences in what I can only describe as "grain" - and I could only see it at 300% zoom, and even then, it wasn't strong. But, I could just be seeing things. Even so, while there appeared to be differences, I would hesitate to call one "better" than the other. At the same GOP length and bitrate, there isn't a very noticeable difference to me in these settings. In the end, I couldn't determine a preference for one setting over another. Is there a setting for these that you prefer? If so, what should I look for?

I agree that GOP=3 is about the sweet spot. GOP=4 works fine and GOP=2 works fine. But, again, I see very little difference among these in the tests I've done.

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #310 on: September 19, 2012, 07:22:50 PM »
On the waterfall video, I don't know (I wasn't keeping very good notes  :-[ -- I'm sure about the GOP, though, since I can infer it in Bitviewer). I have tried different settings for PicPC, DBlock A/B (with notes). There appear to be differences in what I can only describe as "grain" - and I could only see it at 300% zoom, and even then, it wasn't strong. But, I could just be seeing things. Even so, while there appeared to be differences, I would hesitate to call one "better" than the other. At the same GOP length and bitrate, there isn't a very noticeable difference to me in these settings. In the end, I couldn't determine a preference for one setting over another. Is there a setting for these that you prefer? If so, what should I look for?

I agree that GOP=3 is about the sweet spot. GOP=4 works fine and GOP=2 works fine. But, again, I see very little difference among these in the tests I've done.

I don't really have preferred settings as such but adjusting the DBlock A & B to -1 gives a uniform fine grain look. Setting -2/-1 makes the pixels look elongated. Setting both A&B to 0 seems cleanest.

I've tried setting I factor to 2x when trying out ALL-I but it doesn't appear to do much to the actual image. Also tried setting P Factor to a higher setting than I Factor when shooting GOP 3 etc (my thinking is to bring the slice numbers closer together so the bitrate remains fairly constant?). Not sure this does much either. I'm not sure what increasing GOP0, GOP1, GOP2, GOP3, GOP4 Factor actually does? 1%?

I think we're splitting hairs TBH. Increasing bit rate and setting GOP to 3 or 1 shows a marked improvement over Canon FW defaults which I think is a good enough reason to use it and like you stated you can push the image a little further in post.

I've read a lot about ALL-I and GOP on the GH2 forums and the general consensus is that regardless of image quality gains or losses, shooting ALL-I gives a more filmic look to motion but there isn't much in it if you keep GOP under 6.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #311 on: September 19, 2012, 07:35:18 PM »
If you lock sliceqpd CBR/VBR doesn't do a thing. The most they did to begin with was setting slice based on predictions or fixed value. So CBR was just raising the sliceqpd when you set higher numbers for the prediction and thus you got higher bit rates. Notice how it tries to set Q+24 when sliceqpd is a low number. In all I it seems to be broken and not do anything. Probably because prediction is done from P frames.

You can still change gop and use old CBR/VBR just set slice to 0 and reboot.

Upping picqpy boosts luma quality by a good %. PicQPC is for chroma quality and while you can change it its calculated off picqpy so the best PicQPy has the best PC too. All we could go was 1 up from default, others froze camera. It shows when you play back the movies and the camera can't (luma goes crazy), that's not from gop.

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #312 on: September 19, 2012, 07:44:40 PM »
Ah, that kind of makes sense now. So it's probably better to not change factor settings and just set BR and GOP?

I've been using PicPC set to 0. Didn't go any higher. I actually didn't try to  ::)
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #313 on: September 19, 2012, 07:55:09 PM »
If you turn picpc to 0 then py will be 20 vs default of 26. From H.264 spec this was a quality increase.

Best set by slice but then you're at risk of breathing and buffer underruns. Depends on what you're shooting and stability needs. When BR is reported correctly and the quality dropping is spot on it should be better Q than CBR hands down.

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #314 on: September 19, 2012, 07:57:47 PM »
If you turn picpc to 0 then py will be 20 vs default of 26. From H.264 spec this was a quality increase.

Best set by slice but then you're at risk of breathing and buffer underruns. Depends on what you're shooting and stability needs. When BR is reported correctly and the quality dropping is spot on it should be better Q than CBR hands down.

Got it :)
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

3pointedit

  • Senior
  • ****
  • Posts: 263
  • 550D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #315 on: September 20, 2012, 02:34:12 AM »
I see that Marvin referred to encoding parameters for anamorphic and scale, earlier. I wondered if higher frame rates (above 60fps) could be achieved at lower resolution (SD 640x480) but in 16x9 aspect, that is an anamorphic version of SD?
550D on ML-roids

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #316 on: September 20, 2012, 05:05:03 AM »
What is the highest LV fps you get? That is probably best we're going to do. If anything is found it will for sure be implemented.

JasonATL

  • Freshman
  • **
  • Posts: 54
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #317 on: September 20, 2012, 08:04:58 PM »
Some real world experience with this: My daughter's class just took a field trip to horse stables this morning. In 3 hours, I shot about 45 minutes or more of video (clips about 20 seconds each). Used GOP=3.  It was a sunny, windy day. No test shot... just shooting as I normally would. No overflow once. The buffer save function only had to kick in a few times and in the real footage, I can't tell. It looks like my avg bitrate was around 80-90 Mbps, with some peaks in the 130 range. I'll probably edit the footage this weekend and post the video. I wouldn't have been able to do this with the old CBR. Worked great!

JasonATL

  • Freshman
  • **
  • Posts: 54
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #318 on: September 22, 2012, 12:15:08 AM »
More photos of test shots of leaves blowing on trees - exciting stuff, I know  ;).  I shot these today during some wind. To me, the difference between the Canon FW and either GOP=1 and GOP=3 is visible without pixel peeping on the 300% zooms. My wife noticed on the raw video. I didn't bother posting another vid, as I think the captures tell the story.

https://picasaweb.google.com/lh/photo/jEy4yTlS6DyEXEu46gA9DBfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/Q-Oio1E4HegePZwbTqVt2xfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/vDQ2AsTA_cw9hamymrO-xBfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/uDL8Pka3l0N2o1gWm9hdWxfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/qvH_zxX9KWx1AxLp147P4BfNx8dP82C1K8bmBL14X3k?feat=directlink
https://picasaweb.google.com/lh/photo/mzN3khs9lDsxkgt5zt-VeRfNx8dP82C1K8bmBL14X3k?feat=directlink

Without 3x crop mode:
GOP=3 BR: Avg=139 Mbps, Peak=207 Mbps
GOP=1 BR: Avg=128 Mbps, Peak=129 Mbps
Canon FW BR: Avg=45 Mbps, Peak=138 Mbps

With 3x crop mode:
GOP=3 BR: Avg=121 Mbps, Peak=168 Mbps
GOP=1 BR: Avg=115Mbps, Peak=128Mbps
Canon FW BR: Avg=46Mbps, Peak=100 Mbps

1% a couple of things happened while shooting these tests that I've not experienced before. First, the recording would stop with a "camera stopped recording" message without a buffer overflow. I was watching the debug info (Slice was around 122 in many cases) and the buffer % was in the 20% to 30% range. Nothing obvious shows up on the bitrate (it was in the 120's when many of the clips stopped). Also, this happened 30+ seconds into the footage. In other words, the buffer save function seemed to be effective and not the cause of most of my stopped recordings.

Second, (this might be related). The recording time turned red. What does this mean? In some cases, it turned red just before the recording stopped. In other cases, the recording would continue with the time in red for 30 seconds or more. Is this a sensor or processor temp warning? If so, we have another element to consider. This was the first time I had let the recordings go for a long time. If it is a digic temp warning, then perhaps the polling rate could be affecting it. Also, I didn't seem to have the stopped recording issue the one or two times that I had rebooted and then failed to turn debug on. To the extent that the debug adds overhead, perhaps turning off GlobalDraw might help?

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #319 on: September 22, 2012, 02:07:05 AM »
Global draw doesn't affect BR and stoppages much. I think the problem is you're over 4gb (hence the red) and the BR measuring and buffer functions are getting inaccurate or the card is slowing down. This has to be fixed along with making BR more accurate. The math is set up for 1s and we're doing 1/10ths of a second. I've experienced it as well and will have to test more when work winds down a bit. You have to fix the over 4gb clips too with g3gg0's movie fixer before you edit them.

*Try to do 1gb write test and see where it ends up, mine drops a bit from 20 so probably a complexity increase will affect things more at that point until card recovers.

FilmMan

  • New to the forum
  • *
  • Posts: 12
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #320 on: September 22, 2012, 03:09:08 AM »
Jason,

There is a noticeable difference.  I downloaded Trees 3x3 and Trees 3x1, both ungraded versions.  Quick color grade and added sharpening.  The Canon FW degraded more(especially noticeable in the leaves) whereas GOP1 and GOP3 held significantly better (virtually intact).  Thanks for posting!  Cheers.

FilmMan

  • New to the forum
  • *
  • Posts: 12
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #321 on: September 22, 2012, 03:12:21 AM »
PS.  I felt GOP 3 had the edge over GOP 1 too. 

JasonATL

  • Freshman
  • **
  • Posts: 54
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #322 on: September 22, 2012, 03:53:01 AM »
Global draw doesn't affect BR and stoppages much. I think the problem is you're over 4gb (hence the red) and the BR measuring and buffer functions are getting inaccurate or the card is slowing down.

Nope, not over 4gb. No clip was longer than 1.5 GB.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #323 on: September 22, 2012, 05:36:27 AM »
Must be another bug with the time function. Wonder if elapsed time is accurate at all. Is that what you're using or time till 4gb or the other one? I'll have to see why it turns red exactly. For sure related to how fast its counting now and any possible inaccuracy from msleep.

JasonATL

  • Freshman
  • **
  • Posts: 54
Re: Bit rate investigation - altering ratio between P and I frame sizes
« Reply #324 on: September 22, 2012, 02:43:39 PM »
I was using elapsed time. It seemed accurate.