Bit rate investigation

[a1ex]

Hm, the first one looks pretty defocused.

Now, depending on scene complexity, with CBR 1x, QScale may be -16, may be -8, may be 0... may be +16 in extreme cases...

So, for timelapse recording it's probably a great idea to set it at -16 without worrying.

[1%]

Any quick way to lock the ratio to 1.5x I to P? To see what happens... Or set I+P equal?

[a1ex]

Yes, in bitrate.c, rather than scaling all those numbers by a constant factor, compute the average first and scale that one.

[1%]

MOV_OPT_NUM_PARAMS; = I frame?
MOV_GOP_OPT_NUM_PARAMS = p frame? or gop size?

[a1ex]

OPT_NUM_PARAMS: those are 2, one is P, the other is I.

The others... I don't know.

[1%]

Code: [Select]

   
int avg=0;

        for (j = 0; j < MOV_OPT_NUM_PARAMS; j++)
        {   int* opt0 = (int*) &(mvr_config_copy.fullhd_30fps_opt_size_I) + i * MOV_OPT_STEP + j;
// int* opt = (int*) &(mvr_config.fullhd_30fps_opt_size_I) + i * MOV_OPT_STEP + j;
            if (*opt0 < 10000) { bmp_printf(FONT_LARGE, 0, 50, "opt_set: err %d %d %d ", i, j, *opt0); return; }

avg += (*opt0) / (j+1) ;
          // (*opt) = avg * num / den;
//          (*opt) = (*opt0) * num / den;

}

int* opt = (int*) &(mvr_config.fullhd_30fps_opt_size_I) + i * MOV_OPT_STEP;
            (*opt) = avg * num / den;



/Bad At C
This will compute the average?

I tried this... not sure if it worked. Overall bit rate is lower but still hits 90s.

[1%]

Code: [Select]

void opt_set(int num, int den)
{
    int i, j;
    int avgI=0;
    int avgP=0;
for ( int qq =0; qq < 9; qq++)
{ int* opt1 = (int*) &(mvr_config_copy.fullhd_30fps_opt_size_I) + qq * MOV_OPT_STEP + 0;
int* opt2 = (int*) &(mvr_config_copy.fullhd_30fps_opt_size_I) + qq * MOV_OPT_STEP + 1;
avgI += (*opt1) / (qq+1);
avgP += (*opt2) / (qq+1);
}




    for (i = 0; i < MOV_RES_AND_FPS_COMBINATIONS; i++) // 7 combinations of resolution / fps
    {
#ifdef CONFIG_500D
#define fullhd_30fps_opt_size_I fullhd_20fps_opt_size_I
#define fullhd_30fps_gop_opt_0 fullhd_20fps_gop_opt_0
#endif

#ifdef CONFIG_5D2
#define fullhd_30fps_opt_size_I v1920_30fps_opt_size_I
#endif

int* opt = (int*) &(mvr_config.fullhd_30fps_opt_size_I) + i * MOV_OPT_STEP + 0;
                  (*opt) = avgI * num / den;
       
opt = (int*) &(mvr_config.fullhd_30fps_opt_size_I) + i * MOV_OPT_STEP + 1;
  (*opt) = avgP * num / den;



+ Delete the first j loop.

Bit rate gets very serious... instant BR catches up to average very quickly. Scaling factors also more noticeable.

Did I do this right?
I can't scale B/R viewer like you guys have it.



Still spiked on Lower ISO + No motion

[1%]

.

[Audionut]

Looks like you're making some good progress there.

I'm probably getting to far ahead here, but are there plans of being able to make this a user adjustable parameter, or will it be a fixed parameter based on testing.

x264 sets it's IP ratio @ 1.4.  So it's probably a good starting point 

[1%]

Don't know. I'm just testing. I'm not sure which is I/P yet or if its right. I can scale them differently.. I think I almost have to as I can stop recording much easier and I'm seeing much higher bit rates, esp in crop.

*I am going to try to make adjustable independent I/P scaling and maybe a unified bin for testing it.  Then we can see if its doing anything useful.
As is here: https://bitbucket.org/OtherOnePercent/tragic-lantern/changeset/1d40d7f8736d


Have to see if 550D and others have different numbers of movie modes. 600D has 9 so the averaging loop has 9.

[1%]

http://www.qfpost.com/file/d?g=VoeCZ1i1x

Unified you can test.

500D and 5d2 may be incomplete as their scaling is different.

Also, find if I/P is right.

[Audionut]

Cheers 1%.

I'll run some tests and report back.

[Audionut]

What I have noticed so far.

IP ratio should really be 1 setting.   ie:  Setting say 1.5 should mean that I frames are 1.5 times larger then P frames.

As for my tests below, I'm still not sure if I should be posting them as they are very confusing. 

All these tests done at CBR 0.5x



Adjusting P setting down, reduces bitrate substantially for the same CBR setting.  Increasing the IP ratio.
Note:  In previous tests, reducing the bitrate in Qscale mode also increased IP ratio.  So it appears that there in no actual adjustment of IP ratio happening here.  Just bitrate changes.


Adjusting P setting up, slightly increases bitrate.  Same IP ratio.


Adjusting I setting either up or down, increases bitrate slightly with very minor changes to IP ratio.

Adjusting P setting to 0.1x and I setting to 3.0x increases bitrate substantially while reducing IP ratio.



Unfortunately, this is all scene dependent.  Here is another test at various settings until buffer full recording stop.
Qscale 16


P 3.0x - I 0.1x


P 0.1x - I 3.0x


Here, the only changes worth mentioning are that the last test was able to record for longer before buffer full.


Conclusion.  Canon has some funky code going on to adjust IP ratio.  This seems to be based on scene complexity.  Probably to try and help it maintain a constant bitrate.

This patch does not appear to be changing IP ratio.  It appears to me as if it's affecting the bitrate distribution.  For instance, adjusting P ratio down, isn't actually adjusting the ratio of P frames in anyway, simply saying, for this CBR setting, encode at a lower bitrate.  And it's Canons internal code that is making the changes to IP raito.

[1%]

It doesn't do anything in qscale mode. That ignores the scale factor and only sets qscale. Try CBR of over 1.

I think this sets p frame and i frame bitrate size. It does use the canon function... that is all we have in bitrate.c. It takes the average of the values and scales it. So for CBR X adjust bit rate factpr of P and I to X. When I encoded at 2.0x and set 1.3 I to 1.0 P, I frame was ~1.4 x the size of the P frame next to it.

I'll try to scale individual numbers vs average next and see if that changes anything. I don't think anyone else has a fast card to test so any test is good.

[Audionut]

It doesn't do anything in qscale mode. That ignores the scale factor and only sets qscale. Try CBR of over 1.

Sorry.  In the last 3 screenshots.  Qscale -16 was used in the top one.  CBR mode was used in the last 2.  I don't recall what CBR it was, I was just adjusting each one until I was getting the maximum bitrate before buffer full.  I think I got to CBR 1.6x.

I think this sets p frame and i frame bitrate size. It does use the canon function... that is all we have in bitrate.c. It takes the average of the values and scales it. So for CBR X adjust bit rate factpr of P and I to X. When I encoded at 2.0x and set 1.3 I to 1.0 P, I frame was ~1.4 x the size of the P frame next to it.

Ok, that makes a little more sense for me.  Except that in all my tests setting 0.1 P and 3.0 I was the biggest difference I could make.  And actually reduced the size of I frames compared to P frames.
edit:  And if I'm understanding you correctly, the exact opposite should have occurred.

I think the biggest problem with the current method is that it's scaling the frames to some internal Canon code.  So the results will always vary (almost unpredictable).

There needs to be a way of just scaling the P frames to the I frames.  So for instance, whatever internal Canon code decides the size (bitrate) of an I frame, lets scale our P frames to that.

P frames always 10% smaller then I frames. Or 20% or 30% etc etc.

I'll try to scale individual numbers vs average next and see if that changes anything. I don't think anyone else has a fast card to test so any test is good.

Look forward to testing your changes.

[1%]

P frame/ I frame could also be backwards. I can also make it scale in qscale, right now for that it uses a factor of 1 on everything.

After that probably have to look in the firmware. Also... any affect on quality? Seems like not scaling everything the same leads to higher bit rates.

[Audionut]

Also... any affect on quality? Seems like not scaling everything the same leads to higher bit rates.

I haven't looked at quality yet.  I was getting to much (bitrate & IP ratio) variation in the tests I was doing.  I would like to be able to consistently get P frames a certain percentage smaller then I frames.

Then I can concentrate on getting the same bitrate at various IP ratios to be able to judge quality effects.

I am finding that 0.1 P 3.0 I as it stands, does allow a higher (average) bitrate to be captured.

P frame/ I frame could also be backwards.

That makes sense.  I was getting very confused with the options as they currently work. 

[Audionut]

Bug report.

I ratio 1.0 to 1.6 scale up bitrate as would be expected.

I ratio from 1.7 through to 2.9 result is bitrate of 2mbps.  <--Something wrong here.

I ratio 3.0 results in very large bitrate increase.

Assuming this is currently backwards, so this is effecting the P frames.

[a1ex]

Small hint: you can try to expose the raw coefficients (2+5 if I remember well) on the user interface, or in config file. This way, the testers can try any values and probably come up with more meaningful results.

After we understand the exact meaning of each coefficient, we can think at some user-friendly controls (like i/p ratio or something like this).

[ilguercio]

I'm sorry if i can't follow you well with the maths but, can you tell me briefly what's going on?
If you need some testing i can do it on my 50D with a pretty good card (8GB 400x) .

[1%]

Try it and see if it works for you. I guess next step is toggling averaging and dumping the coefs to screen. Then once we know what numbers go in there we can load all from the config file. Will be more of a pain to toggle unless you overwrite the config file with PTP or something. I'll switch I/P too.

[Chungdha]

I ran new 2.3 on 550D with Sandisk extreme PRO 95mb/s and tried out CBR 3x with audio off and it ran quite long till I ran into a too complex scene. But Audio on could only do CBR 1.6x I wonder why the sound recording is taking basically like half the buffer while its should be processing less information than the video part.

[1%]

Wonder no more: audio takes up ~40mbps

[1%]

New bin:

* Switch I/P
* Turn on/off I & P averaging
* Use I/P in qscale mode

I tried setting the scale over 3.0x  but that actually produced lower bit rates. I'll have to set up a debug mode so values can be seen on the screen next.

http://www.qfpost.com/file/d?g=SQBAcMzad


With more investigation....

NSTUB(0xFF1C94E0, mvrSetDeblockingFilter) < Good or bad? And how do you set it.

NSTUB(0xFF1C9AB4, AJ_MvrSet_HD_VGA_Gop_size) has no arguments? Sets a bunch of the other parameters.

[Audionut]

With more investigation....

NSTUB(0xFF1C94E0, mvrSetDeblockingFilter) < Good or bad? And how do you set it.

NSTUB(0xFF1C9AB4, AJ_MvrSet_HD_VGA_Gop_size) has no arguments? Sets a bunch of the other parameters.

Being able to adjust the deblocking filter would be handy.

Cheers 1%, I'll find some time to test the new build tomorrow.