Bit rate investigation

[Audionut]

Firmware Default
http://dl.dropbox.com/u/34113196/ML/Firmware%20Default.png

Q15
http://dl.dropbox.com/u/34113196/ML/Q15.png

1 has detail in the leaves.  The other has blurry green things with no detail.

Cropped 300% resize to exaggerate.




edit:  Considering it's a poor encoder to start with and uses 40mbps @ default for the quality it produces (lack of fine details), is it really any wonder that doubling the bitrate increases quality?

The Q15 sample above was the max of my card @ 99mbps.  And although it's much better then deafult, IMO, it would take around 200mbps before it produced really good results.
Such a shame that the codec can't be replaced.

[1%]

There may be more parameters to the encoder, just have to look. Its clear for me higher bitrate = more data and lower bitrate = less data = lower quality. I mean why else can you put .1 bitrate and record for 1/2 an hour? Higher bitrate not having better quality would mean that the settings are being rejected but they're clearly not.

The problem is the bit rate setting itself. I set to 1.1 and I get bit rates of 45-60... the higher settings just peak the bit rate really high and thats why recording stops. Above 65mbps with card not fresh it stops recording. I've been watching the instant rates and the higher x numbers still seem to give the same base bit rate.

 If they only didn't peak it so fast and started at a higher value it would be more stable while keeping quality. As it stands the settings just crank it till you overrun the buffer then try to lower it but never fast enough.  I.e it goes from 59 to 79 or 89 and then the buffer runs out depending on card speed and scene complexity. 

[1%]

Ok, I bought a new card. Patriot EP SDXC 64g. It is tested for at least 15MB/s continuous write. Last night I thought 99mbps is tops on this camera... it is not. I saw 107mbps peak. The reason you're all having trouble is that your cards are not fast enough. Class 10 is 10MB/s continuous write and 100mbps is 12.5MB/s. 

With the new card I can shoot 99mbps all day, complex scene in 2.x factor, 1250 ISO. At 3.0 the BR starts going above the 100s and eating the buffer. I also found out audio is ~40Mbps. I can shoot at 1.4x and get ~61Mbps instant with 1 bar in the buffer. Higher doesn't work. i almost wish I had bought a faster pro card but I'm not paying $180 for a few extra Mb. So while this feature is good when SDXC continuous speeds go up so will bit rate. I don't think there is a limit as 600D supports UHC-1 but maybe not the faster mhz bus speed (anyone try?). That 200mbps? In a year or with expensive cards it might happen.

*Update... I saw instant bit rate of 210.. do you think this is an error?

[ilguercio]

I got huge bitrates for a few seconds with my 50D, like 188 mbps, but the that was probably too much for the camera.
Still, if there is no such appreciable gain in quality i see no point in using such bitrates since they're slower to process and tae up a lot of space.

[weldroid]

This guy here?
http://www.clickok.se/PartDetail.aspx?q=p:4664058

15 MB/s is what... 120 mb/s, so it should be able to handle more than 100 mb/s.

[1%]

15MB/s is about there yes... I think 120mbps stable is a bit hopeful. That card tested 18MB/s on a 1gb file and 15 on a 4gb file. So far 99 is stable for me... above 100 is where it starts to get flaky. Gotta admit 99mbps at high ISO is pretty good. I only paid $US60... much better than US $179

Still, if there is no such appreciable gain in quality i see no point in using such bitrates since they're slower to process and tae up a lot of space.

According to? The gain comes out in post [but we'll see ]. Not much "visible" difference between 4:2:2 and 4:2:0 for most applications but then 4:2:2 camera costs $$$$ and 4:2:0 camera is cheap.

Also we don't know whats up with extending 4gb limit or other encoder parameters on 600D. Can 550D do steady 99mbps? Or 60D? I guess 50D didn't do too bad but I think movie mode on that is completely added by ML? People were saying 1.4x is last stable factor on 60D and I think alex was even going to take higher bit rates out; but on 600D 1.4x works with sound and before I could barely do 1.1 or 1.2 at higher complexity + iso.

I think this card was only available since 5/12.... SD was simply not capable of these speeds before without REALLY expensive cards nobody would buy.

[Audionut]

Can 550D do steady 99mbps?

550D - Sandisk Extreme Pro 95MB/s

Firmware default.


About the best I can get with in camera audio recording.


Max bitrate I can get with off camera audio


I don't have a test with ML CBR.  But as has been mentioned already, it is not true CBR.
I'm fairly sure I've seen Alex mention that this has to do with the way the Qscale works.  It's to slow to update iirc.

edit:  As can be seen in the last screenshot, these are all 1080p/30fps.  Could probably squeeze are few more Mbps out of it at 24fps, but I hate motion judder.

[a1ex]

True CBR in H.264 may need multiple passes for each frame. As in: compress at some fixed quality factor (QScale). Is frame size too big? Reduce quality and recompress. Too small? Increase quality and recompress. Rinse, repeat.

OK for encoding on PC, but not OK in real time. And the algorithm is hardcoded in Canon firmware - it basically uses frame size info from frame X to decide quality factor on frame X+1. Sure, there are P frames, I frames, GOP and other similar stuff that make changing the algorithm very difficult.

So, unless there are visible benefits of this (which I didn't see yet), it will stay as wontfix.

[Audionut]

So, unless there are visible benefits of this (which I didn't see yet), it will stay as wontfix.

Do you not see the visual difference in the screenshots I posted above?

IMO, a true CBR mode wouldn't increase quality per se, but it would allow users to record at the maximum bitrate of their DSLR's/Memory cards without fear of recording being stopped as currently happens with the current modes.

[1%]

I don't think we really need a "fix"... I think people need faster cards. Camera write > card write at this point. Were you able to see any other parameters for the encoder? P frames, I frames and GOP I think can alter quality too.

https://docs.google.com/viewer?a=v&q=cache:irMNf5l2YKIJ:www.acti.com/download_file/Product/support/DesignSpec_Note_GOP_20091120.pdf+&hl=en&gl=us&pid=bl&srcid=ADGEESjGSHsSrBtVUnnS1hD4VabEF8doangw_kfGHKdtFzskQPYa18On-7UgzKquwOiCOI939UWnuSnLKkthDpDArZMRhbmcNIRsz0RmPkfMfvfNAo2QOCtcT97oSM8ChjvHYdUHEP3i&sig=AHIEtbQ5C55Uiw30BabakL0sKOumDxhbag

[a1ex]

GOP length is fps/2 - buf[3] in PROP_VIDEO_MODE. You can't just change it - it will record a few frames and stop. But it will record them with new GOP length, which is a good sign.

Reference frame sizes for P and I are scaled when you change the "X" in bitrate dialog, by a constant factor with current implementation. It is possible to alter their ratio (for example, you can get equal size for I and P frames).

[Audionut]

I don't think we really need a "fix"... I think people need faster cards.

If you can find me something faster then a Sandisk Extreme Pro, I'd be all for it.  But at the end of the day, it wouldn't matter as the bitrates are limited by the cameras.

I can get transfers in excess of 80MB/s on a PC.  This is way in excess of the 100Mbps that the camera tops out at.

[Audionut]

GOP length is fps/2 - buf[3] in PROP_VIDEO_MODE. You can't just change it - it will record a few frames and stop. But it will record them with new GOP length, which is a good sign.

A larger GOP length would certainly help.  10x fps is considered a fairly standard encoding setting.  Of course, this isn't taking into account any format (blu-ray etc) requirements.

Reference frame sizes for P and I are scaled when you change the "X" in bitrate dialog, by a constant factor with current implementation. It is possible to alter their ratio (for example, you can get equal size for I and P frames).

May I ask what the current ratio is? 

If it's not possible to change the GOP length, AFAIK, altering the IP ratio would help.

[a1ex]

Need to run custom code to find out - but you can see it with bitrate viewer.

[1%]

Hehehe, were there is a will there is a way.

I can get transfers in excess of 80MB/s on a PC

Read? Write? Remember camera isn't using the new UHC-1 bus speed so real write is probably ~15-20MB/s... they have not made a faster card yet Maybe when UHS speed is higher the reggo write will go up like it did. Or we can tweak something to enable this? Far shot...

Need to run custom code to find out

The pile of things to try for ML 2.4 is growing but I guess that is a good thing

[Leon]

It'a not just about improving quality.  A big problem is how the max CBR setting that a card can cope with changes unpredictably.  For example, a class 10 Transcend card may manage 1.3x right after formatting, but less than 1.0x after it as been used a bit or some files deleted.  It drives me nuts.  This is quite annoying, and it would be great to have a feature that would automatically reduce the bitrate or CBR setting if required during recording.  I think unexpected stopping / restarting of video is certainly one of the biggest problems with recording video on the 550D.  It happens too often, even at the firmware default.

[1%]

What is:

Code: [Select]


static void load_h264_ini()
{
    gui_stop_menu();
    call("IVAParamMode", CARD_DRIVE "ML/H264.ini");
    NotifyBox(2000, "%s", 0x4da10);
}

And this sets gop and frame sizes ratios in an array?

Code: [Select]


void opt_set(int num, int den)
{
    int i, j;
   

    for (i = 0; i < MOV_RES_AND_FPS_COMBINATIONS; i++) // 7 combinations of resolution / fps
    {
#ifdef CONFIG_500D
#define fullhd_30fps_opt_size_I fullhd_20fps_opt_size_I
#define fullhd_30fps_gop_opt_0 fullhd_20fps_gop_opt_0
#endif

#ifdef CONFIG_5D2
#define fullhd_30fps_opt_size_I v1920_30fps_opt_size_I
#endif
        for (j = 0; j < MOV_OPT_NUM_PARAMS; j++)
        {
            int* opt0 = (int*) &(mvr_config_copy.fullhd_30fps_opt_size_I) + i * MOV_OPT_STEP + j;
            int* opt = (int*) &(mvr_config.fullhd_30fps_opt_size_I) + i * MOV_OPT_STEP + j;
            if (*opt0 < 10000) { bmp_printf(FONT_LARGE, 0, 50, "opt_set: err %d %d %d ", i, j, *opt0); return; }
            (*opt) = (*opt0) * num / den;
        }
        for (j = 0; j < MOV_GOP_OPT_NUM_PARAMS; j++)
        {
            int* opt0 = (int*) &(mvr_config_copy.fullhd_30fps_gop_opt_0) + i * MOV_GOP_OPT_STEP + j;
            int* opt = (int*) &(mvr_config.fullhd_30fps_gop_opt_0) + i * MOV_GOP_OPT_STEP + j;
            if (*opt0 < 10000) { bmp_printf(FONT_LARGE, 0, 50, "gop_set: err %d %d %d ", i, j, *opt0); return; }
            (*opt) = (*opt0) * num / den;
        }
    }
[/quote]


[nanomad]

Some kind of test probably

[Audionut]

Read? Write? Remember camera isn't using the new UHC-1 bus speed so real write is probably ~15-20MB/s...

Sorry, that was read speed.  I am unsure how to completely bypass windows caching to determine true write speeds.  But considering the card reaches advertised read speeds, and the advertised write speed is 90MB/s, even if it was only getting 1/2 advertised write speeds, that's still way in excess of what the camera (550D atleast) is capable of writing.

Need to run custom code to find out - but you can see it with bitrate viewer.

Ouch, I frames should not need to be anywhere near 2x the size of P frames.

It appears as if it's a variable IP ratio also.  edit:  On second thoughts, it's probably not a variable IP ratio, simply that 235Mbps is the maximum frame size the camera is capable of.



Even more hurt.  I frames 4x larger then P frames 
That makes sense, I'm shooting for low bitrate and Canon is doing it's best to waste bits 

A reduction in I frame size would allow an increase in P frame size, for the same GOP size.

edit:  I frames being reference frames for all the P frames in that GOP, they do need to be larger (better quality).  4x larger, no!


Does anyone see the difference in the screenshots I posted on the first page of this thread?

[Audionut]

Why is GOP size so important.  Look at the last example I posted above.  MVI_8661.  There are approximately 31 I frames in that sequence.  I can tell you that the scene was almost static, a little zoom or 2.  With a massive GOP size, lets say 500 frames (ignoring seeking ability, editing ability, spec requirements etc), a good encoder would have only needed say 3 or 4 I frames.

So instead of having 31 frames @ 48Mbps, we could have gotten away with only 4 frames @ 48Mbps, which would have lowered the overall bitrate of that sequence.  And given I was obviously shooting for a very low bitrate, I could have either had the same quality with a lower final bitrate (filesize), or a higher quality encode at the same bitrate.

Alex.  Regardless of any quality improvements there may or may not be with adjustable GOP and/or IP ratio sizes, there is a small demand for this functionality.  I've already donated once, and I would like to offer a $20 bounty.  After 2.3 goes final of course.

[1%]

So 235MBps is the theoretical limit... and camera write would be 30MB/s... no card exists that can do this without UHS mode.. yet. Makes sense what we are getting... about 1/2 of top bit rate at 1/2 write speed.


I see the difference in the leaves, I think they just assume that difference not big enough. Seems encoder quality is so "poor" because canon wastes bandwidth.  How do the ratios look by default... still this bad? What about scenes with motion?

[Audionut]

I see the difference in the leaves, I think they just assume that difference not big enough.

Well it's a fairly significant difference for mine.  The high bitrate encode does a reasonable job of maintaining detail, the default settings just blur.

Seems encoder quality is so "poor" because canon wastes bandwidth. 

It's just a poor encoder, period.  And to be honest, at best, the most that GOP and/or IP ratio changes would make (to quality) are similar to what I posted in the first page of this thread.  And if people are having a hard time noticing those differences, I'm fighting a losing battle to get GOP/IP ratio adjustment implemented.

How do the ratios look by default... still this bad? What about scenes with motion?

Default was very similar to the low bitrate encode.  I frames around 4 times larger.  I haven't tested any motion scenes.

I'll find some time in the next few days to run tests on sequences with motion.  I'll also do some bitrate tests involving people.  Maybe if people see the lack of fine details in peoples faces with the default bitrates compared to high bitrate encodes, it might have more of an effect then a leaf.

[1%]

It was enough difference for me to go buy a faster card.

[nanomad]

It was enough difference for me to go buy a faster card.

Agreed, the difference is definitely there

[weldroid]

Actually, the difference is very easy to visualize: low FPS-es with FPS override let us choose virtually any VBR levels up to (down to) -16 irrespective of SD card capabilities. So just shoot something with lots of details with -8 and -16 and add some sharpening in post...

https://dl.dropbox.com/u/4848144/0.png

vs

https://dl.dropbox.com/u/4848144/-16.png