Messages

Stream cut ~90mbps Gop3 Video

That's looking really good.  GOP size correct and you have a nice low IP ratio.

And a quick look on the laptop the video looks great.  No blocking with the fast motion and details appear to be fine.

Unless you need 50/60p, just record at 1920x1080 with whatever cropmarks you need and then crop/resize in post.

240 is unworkable.. buffer fills at very low BR and recording stops. Qscale says +24? and BR is in the 1/2s (all options at .1x) before I can get it to not stop instantly.

A large GOP size should reduce bitrate, so there must be something else funky going on.

Are you positive that the changes you are making are having the correct effects to GOP?  Record a short sample and load that into bitrate viewer, switch to frame based mode and count the number of frames in between the I frames (large bitrate spikes).
Or send the files to g3gg0 who can check with the stream analyzer.

Once you are sure you are getting the correct changes to GOP, we will need to test if the Canon code has any scene detection. 

I'll try 240 and see what happens. Did you notice quality fall in the sample pics? I don't think its quite as bad as low BR I/P. the frames are about the same size (~5mb) . Bit rate for I only goes up to 140 without buffer filling.

Really need a scene shot form the same position (tripod) with the same lighting conditions.
Our eyes tend to bias to the bright side.

I need a consistant way to test what this is doing to quality, any ideas. I don't have curtains, test chart is hard to tell a diff.

Use anything with lots of details/texture..  Also need to test motion.  I'd suggest a static scene and walking through the frame.

So what does increasing gop length do? Should be theretically better, right?

GOP is a Group Of Pictures.  Since the Canon codec only has I & P frames and the default GOP IIRC being 12, a GOP would look like this.

IPPPPPPPPPPP

And this just repeats, IPPPPPPPPPPPIPPPPPPPPPPPIPPPPPPPPPPP

I frames are larger then P frames.  As I mentioned earlier in this thread, they can be up to 4x larger.  I frames need to be larger as they are the reference frames for P frames.  P frames are not a complete picture, they only contain the detail that has changed since the last I frame.

Baring any scene detection that might be in the Canon code, if we increase the GOP size to a more general 10 x framerate (24fpsx10) 240, we can have 1 large I frame followed by 239 P frames.

Where as, with the default Canon GOP size, for the same 240 frames, we would have a total of 20 (large) I frames.

Negative numbers result in less in-loop deblocking and of course positive numbers result in more in-loop deblocking.

H264 encoder (x264) is smart enough that it reduces deblocking for higher bitrates, and with high enough bitrate will disable deblocking altogether.  I seriously doubt that's happening in the Canon encoder though.

Judging by the OP results in the bitrate thread, it looks like we could easily reduce deblocking strength at higher bitrates.  Try and squeeze as much detail out as possible.

Hi 1%,

Sorry for going awol, life got busy.  Any chance you can make a unified build?

Can now set Gop lenght and its verifiable + written. Bad news is when buffer overruns you get error 70. Bit rate seems lower, quality untested yet. Def way less P frames.

Increasing GOP length should result in more P frames for every I frame.

With more investigation....

NSTUB(0xFF1C94E0, mvrSetDeblockingFilter) < Good or bad? And how do you set it.

NSTUB(0xFF1C9AB4, AJ_MvrSet_HD_VGA_Gop_size) has no arguments? Sets a bunch of the other parameters.

Being able to adjust the deblocking filter would be handy.

Cheers 1%, I'll find some time to test the new build tomorrow.

With AF microadjustment, since focus step can be controlled, wouldn't it be possible to code, shutter pressed adjust focus step by user defined amount, capture frame?

Don't expect the code to be fast enough to caption motion, but it would work fine for portraits.

Bug report.

I ratio 1.0 to 1.6 scale up bitrate as would be expected.

I ratio from 1.7 through to 2.9 result is bitrate of 2mbps.  <--Something wrong here.

I ratio 3.0 results in very large bitrate increase.

Assuming this is currently backwards, so this is effecting the P frames.

Also... any affect on quality? Seems like not scaling everything the same leads to higher bit rates.

I haven't looked at quality yet.  I was getting to much (bitrate & IP ratio) variation in the tests I was doing.  I would like to be able to consistently get P frames a certain percentage smaller then I frames.

Then I can concentrate on getting the same bitrate at various IP ratios to be able to judge quality effects.

I am finding that 0.1 P 3.0 I as it stands, does allow a higher (average) bitrate to be captured.

P frame/ I frame could also be backwards.

That makes sense.  I was getting very confused with the options as they currently work. 

It doesn't do anything in qscale mode. That ignores the scale factor and only sets qscale. Try CBR of over 1.

Sorry.  In the last 3 screenshots.  Qscale -16 was used in the top one.  CBR mode was used in the last 2.  I don't recall what CBR it was, I was just adjusting each one until I was getting the maximum bitrate before buffer full.  I think I got to CBR 1.6x.

I think this sets p frame and i frame bitrate size. It does use the canon function... that is all we have in bitrate.c. It takes the average of the values and scales it. So for CBR X adjust bit rate factpr of P and I to X. When I encoded at 2.0x and set 1.3 I to 1.0 P, I frame was ~1.4 x the size of the P frame next to it.

Ok, that makes a little more sense for me.  Except that in all my tests setting 0.1 P and 3.0 I was the biggest difference I could make.  And actually reduced the size of I frames compared to P frames.
edit:  And if I'm understanding you correctly, the exact opposite should have occurred.

I think the biggest problem with the current method is that it's scaling the frames to some internal Canon code.  So the results will always vary (almost unpredictable).

There needs to be a way of just scaling the P frames to the I frames.  So for instance, whatever internal Canon code decides the size (bitrate) of an I frame, lets scale our P frames to that.

P frames always 10% smaller then I frames. Or 20% or 30% etc etc.

I'll try to scale individual numbers vs average next and see if that changes anything. I don't think anyone else has a fast card to test so any test is good.

Look forward to testing your changes.

What I have noticed so far.

IP ratio should really be 1 setting.   ie:  Setting say 1.5 should mean that I frames are 1.5 times larger then P frames.

As for my tests below, I'm still not sure if I should be posting them as they are very confusing. 

All these tests done at CBR 0.5x



Adjusting P setting down, reduces bitrate substantially for the same CBR setting.  Increasing the IP ratio.
Note:  In previous tests, reducing the bitrate in Qscale mode also increased IP ratio.  So it appears that there in no actual adjustment of IP ratio happening here.  Just bitrate changes.


Adjusting P setting up, slightly increases bitrate.  Same IP ratio.


Adjusting I setting either up or down, increases bitrate slightly with very minor changes to IP ratio.

Adjusting P setting to 0.1x and I setting to 3.0x increases bitrate substantially while reducing IP ratio.



Unfortunately, this is all scene dependent.  Here is another test at various settings until buffer full recording stop.
Qscale 16


P 3.0x - I 0.1x


P 0.1x - I 3.0x


Here, the only changes worth mentioning are that the last test was able to record for longer before buffer full.


Conclusion.  Canon has some funky code going on to adjust IP ratio.  This seems to be based on scene complexity.  Probably to try and help it maintain a constant bitrate.

This patch does not appear to be changing IP ratio.  It appears to me as if it's affecting the bitrate distribution.  For instance, adjusting P ratio down, isn't actually adjusting the ratio of P frames in anyway, simply saying, for this CBR setting, encode at a lower bitrate.  And it's Canons internal code that is making the changes to IP raito.

Cheers 1%.

I'll run some tests and report back.

Looks like you're making some good progress there.

I'm probably getting to far ahead here, but are there plans of being able to make this a user adjustable parameter, or will it be a fixed parameter based on testing.

x264 sets it's IP ratio @ 1.4.  So it's probably a good starting point 

I see the difference in the leaves, I think they just assume that difference not big enough.

Well it's a fairly significant difference for mine.  The high bitrate encode does a reasonable job of maintaining detail, the default settings just blur.

Seems encoder quality is so "poor" because canon wastes bandwidth. 

It's just a poor encoder, period.  And to be honest, at best, the most that GOP and/or IP ratio changes would make (to quality) are similar to what I posted in the first page of this thread.  And if people are having a hard time noticing those differences, I'm fighting a losing battle to get GOP/IP ratio adjustment implemented.

How do the ratios look by default... still this bad? What about scenes with motion?

Default was very similar to the low bitrate encode.  I frames around 4 times larger.  I haven't tested any motion scenes.

I'll find some time in the next few days to run tests on sequences with motion.  I'll also do some bitrate tests involving people.  Maybe if people see the lack of fine details in peoples faces with the default bitrates compared to high bitrate encodes, it might have more of an effect then a leaf.

Why is GOP size so important.  Look at the last example I posted above.  MVI_8661.  There are approximately 31 I frames in that sequence.  I can tell you that the scene was almost static, a little zoom or 2.  With a massive GOP size, lets say 500 frames (ignoring seeking ability, editing ability, spec requirements etc), a good encoder would have only needed say 3 or 4 I frames.

So instead of having 31 frames @ 48Mbps, we could have gotten away with only 4 frames @ 48Mbps, which would have lowered the overall bitrate of that sequence.  And given I was obviously shooting for a very low bitrate, I could have either had the same quality with a lower final bitrate (filesize), or a higher quality encode at the same bitrate.

Alex.  Regardless of any quality improvements there may or may not be with adjustable GOP and/or IP ratio sizes, there is a small demand for this functionality.  I've already donated once, and I would like to offer a $20 bounty.  After 2.3 goes final of course.

Read? Write? Remember camera isn't using the new UHC-1 bus speed so real write is probably ~15-20MB/s...

Sorry, that was read speed.  I am unsure how to completely bypass windows caching to determine true write speeds.  But considering the card reaches advertised read speeds, and the advertised write speed is 90MB/s, even if it was only getting 1/2 advertised write speeds, that's still way in excess of what the camera (550D atleast) is capable of writing.

Need to run custom code to find out - but you can see it with bitrate viewer.

Ouch, I frames should not need to be anywhere near 2x the size of P frames.

It appears as if it's a variable IP ratio also.  edit:  On second thoughts, it's probably not a variable IP ratio, simply that 235Mbps is the maximum frame size the camera is capable of.



Even more hurt.  I frames 4x larger then P frames 
That makes sense, I'm shooting for low bitrate and Canon is doing it's best to waste bits 

A reduction in I frame size would allow an increase in P frame size, for the same GOP size.

edit:  I frames being reference frames for all the P frames in that GOP, they do need to be larger (better quality).  4x larger, no!


Does anyone see the difference in the screenshots I posted on the first page of this thread?

GOP length is fps/2 - buf[3] in PROP_VIDEO_MODE. You can't just change it - it will record a few frames and stop. But it will record them with new GOP length, which is a good sign.

A larger GOP length would certainly help.  10x fps is considered a fairly standard encoding setting.  Of course, this isn't taking into account any format (blu-ray etc) requirements.

Reference frame sizes for P and I are scaled when you change the "X" in bitrate dialog, by a constant factor with current implementation. It is possible to alter their ratio (for example, you can get equal size for I and P frames).

May I ask what the current ratio is? 

If it's not possible to change the GOP length, AFAIK, altering the IP ratio would help.

I don't think we really need a "fix"... I think people need faster cards.

If you can find me something faster then a Sandisk Extreme Pro, I'd be all for it.  But at the end of the day, it wouldn't matter as the bitrates are limited by the cameras.

I can get transfers in excess of 80MB/s on a PC.  This is way in excess of the 100Mbps that the camera tops out at.

So, unless there are visible benefits of this (which I didn't see yet), it will stay as wontfix.

Do you not see the visual difference in the screenshots I posted above?

IMO, a true CBR mode wouldn't increase quality per se, but it would allow users to record at the maximum bitrate of their DSLR's/Memory cards without fear of recording being stopped as currently happens with the current modes.

Can 550D do steady 99mbps?

550D - Sandisk Extreme Pro 95MB/s

Firmware default.


About the best I can get with in camera audio recording.


Max bitrate I can get with off camera audio


I don't have a test with ML CBR.  But as has been mentioned already, it is not true CBR.
I'm fairly sure I've seen Alex mention that this has to do with the way the Qscale works.  It's to slow to update iirc.

edit:  As can be seen in the last screenshot, these are all 1080p/30fps.  Could probably squeeze are few more Mbps out of it at 24fps, but I hate motion judder.

Firmware Default
http://dl.dropbox.com/u/34113196/ML/Firmware%20Default.png

Q15
http://dl.dropbox.com/u/34113196/ML/Q15.png

1 has detail in the leaves.  The other has blurry green things with no detail.

Cropped 300% resize to exaggerate.




edit:  Considering it's a poor encoder to start with and uses 40mbps @ default for the quality it produces (lack of fine details), is it really any wonder that doubling the bitrate increases quality?

The Q15 sample above was the max of my card @ 99mbps.  And although it's much better then deafult, IMO, it would take around 200mbps before it produced really good results.
Such a shame that the codec can't be replaced.