Bit rate investigation

[1%]

This is how profile is set:

*0xC0E1000C = 41091

Other mode is for something else...

What is what?

Code Select Expand


   if arg2 == 0 /*EQ*/:
                *0xC0E1000C = 41104
                DryosDebugMsg(0x15, 0x2, '[JPCORE] JP62_OPMR3 %#lx', 0xa090) => ret_DryosDebugMsg_FF1E35E4
            if arg2 != 0 /*NE*/:
                *0xC0E1000C = 41091
                DryosDebugMsg(0x15, 0x2, '[JPCORE] H264_SPS_PPS JP62_OPMR3 %#lx', 0xa083) => ret_DryosDebugMsg_FF1E35E4



Value of 0xA083 is 786366719 or 0x2EDF00FF
0xA090 is 0.

[jgharding]

A bit of film maker feedback here

I've been using CBR2.8 mode for promo shoots, it's great, at high ISO settings in real-world situation I can remove much more noise in post, as said noise not so compressed mushed into the image. Thanks, it's amazing to use so far.

With regard to GOP, I and most of those I know would always choose an I-frame option if available. There does seem to be a subtle aesthetic difference between long GOP codecs and those that compress individual frames. Also from the point of view of buffer over-runs, it sounds quite practical to simply set GOP to one (I-frame) a d then pump the bitrate to around the peak speed of the camera writer. Card space is no longer the issue really, it's just getting the best quality we can, and it seems that's on the way now!

If you can make a unified for this I'll poke around a bit, though I'm no coder I have quite an intuition for such things

[jgharding]

Also, I'm interested to see if other Profile levels can be used. I see that Canons are using the Baseline profile in H264, I wonder if the two levels above this, Main and High, are available to the EOS hardware encoder?

If so, the video quality for a given bit rate should increase remarkably, as should the power required to decode the video in real time. While the latter is negligible in production terms, the former could be quite wonderful, if indeed alternative profiles are both accessible and effective.

[Leon]

@1% : Why are you changing settings for slices?  Slices, AFAIK, are only used to help decoding on multi-thread decoders, which is qhy BluRay asks for 4 slices...  Increasing the number of slices decreases the quality, which is why the default is 0 in x264...

See http://forum.doom9.org/showthread.php?t=148826 (regarding BluRay compatibility):

With "Slices" they mean that each frame should consist of at least 4 slices (parts), that can be decoded/processed independently.

If a stream consists of several slices, this allows (but doesn't enforce) slice-level multi-threading.

So if an encoder relies on slice-level multi-threading, but the stream doesn't consist of several slices, that encoder will fail to decode the stream (at an acceptable speed).

I guess they included the "4 slices" requirement in the BD specs, because they wanted to make sure that even such decoders will be able to decode the BD streams.

Anyway, all the state-of-the-art H.264 decoders (ffmpeg-MT, CoreAVC, DivX H.264, etc.) use frame-level multi-treading and hence work with single-slice streams just fine.

It appears that also the hardware decoders used in "stand-alone" BD players are capable of decoding Level 4.1 H.264 streams without multiple slices.

Also it was found that slices reduce compressibility. In case of x264 the loss was around -0.1 PSNR for 4 slices (link). And: The more slices/threads, the bigger the loss!

Frame-based multi-threading not only gives more speed-up, also the loss in quality/compressibility is very small - even at high number of threads.


Or is it something else you're calling a "slice"?

[Leon]

Has anyone worked out why including audio, which is only 1.5 Mbps, and is completely uncompressed, has such a massive effect on the maximum video bitrate?  It seems really strange.

[jgharding]

Perhaps it takes some processor grunt to thread the two together in the MOV, so disabling it altogether frees up a lot of cycles for video encoding. just a guess though

[1%]

Increasing the number of slices decreases the quality, which is why the default is 0 in x264...

Not the same thing. Jpcore slice qp is basically the quality of the input frame. All VBR/CBR does is alter this parameter... ie at Qscale +16 sliceqpd is high number  (low quality, low BR). At -16 its 112 which is the highest official quality. Camera goes down to 86/87 in theory which is like qscale -25 or something.

In the file itself slice qp delta per frame stays at -10 after 112... but BR increases from 112-87 so in theory you're getting higher than q-16.

I have a theory too that encoder just alters sliceqpd and line skip (not sure how) for a given frame. That's why at high quality with lots of complexity it kills the buffer and dies... that input frame gets bigger and bigger and it gets harder to encode it. In crop mode BR gets much higher.

All CBR does is compute slice quality on the fly from those I/P/Gopt sizes. When you lock the slice, qscale and all of those parameters don't do anything. The other Jpcore registers aren't touched by many functions and there is only one path which shows JPcore OPMR3 as SPS/PPS. Another register is frame size and another is from deblocking filter.  Not much left over.

Also I have an idea. 600D released in 2011? TMP19A43CDXBG released when, and is it the controller for SD Card? What is the highest bus speed it supports? Toshiba is one of the developers of high speed SD technology and made some of the first UHS-1 cards ... in 2009/10. Quite possible it could support higher speeds just canon didn't implement them, there were all of 4 cards on the market. What is the new chip used in the T4i? Does 5dMk3 support UHS-1, I think it has dual slot?

We have functions like:
NSTUB(0xFF3BACE8, SDCOM_ChangeBusClock)
but no list of available clocks.

Camera first does read speed test and sets parameters in 0xCD7C (aAJ_0x14320_SDIOTSK_QuickReadData. I think there are 2 options based on how the read speed goes but all of that can now be modified. The bandwith of the camera itself can go much higher than now and if SD controller supports another mode it will probably happily write as fast as the SD card can go, 1/2 the bottleneck is there.

Anyone with 5d3 look? Or know what the Toshiba chip supports or how SD driver section of fw works? Already canon had planned for 64bit file addressing, not used till T4i. If we can do something here maybe we can record at slice 87 all day.

Thoughts?

[Rush]

Does 5dMk3 support UHS-1, I think it has dual slot?

Sadly, 5DmkIII' controller don't support UHS-I rates (that's why it is reccommended to use much faster CF). 650D is the first Canon DSLR with UHS-I support.

[1%]

So we have to look at t4i firmware/HW when available. Right now hopefully we are using 48mhz bus instead of the 24mhz bus. All I see is Mode 3 in the logs but I'm not sure if that means 48mhz or just high speed mode. I only have class 10 cards.


Also, while qscale can only be changed by 1, I wonder how much slice can be changed on the fly.  This could lead to a better CBR function that drops quality when bit rate goes past certain point. Another thing to consider.

[a1ex]

Is the .422 data (silent pics) considered the state of the feed for the h264 encoder?

Yes, this is the input for the H264 encoder. You can alter some data in those buffers and it will show up in the recorded video.

[deleted.account]

Cheers for the reply. :-)

[1%]

Size of Simple 5x5 .422 is ~3mb. If we get it down to 1.0 mb or less we could record it fairly well. Maybe could be put through jpeg path but I don't know if any of them keep the 4:2:2. At 3mb would need like 480 megabits. My card does about 19MB write.

Even written as 1 frame of jpeg, could be made as image sequence in AE and then wav sound added. If there really is intermediate jpeg step already it would be even easier, just stop it from being passed to the H264E/Movie writer. I bet overhead goes down too.

I still think some buffer overruns are because compressor stop compressing. Buffer start filling from 1%-5% to suddenly 40, 60, 75, stop. Like new frames stop being written out. How can it do that at <90mbps when it records steadily at 120-130+ for minutes at a time with quality constant.

[Leon]

Size of Simple 5x5 .422 is ~3mb

What is a "Simple 5x5"?

Has anyone seen any way of preventing the frames being upscaled from 1728x972 to 1080p.  I reckon that's not helping either.  It should also help with using micro-moire removal filters in post, etc.

[1%]

Simple 5x5 is a simple silent pic 5x5. Slice quality may only come into play when the encoder starts.


Also, quite hilarious:

http://www.learn.usa.canon.com/resources/articles/2012/ipp_ipb_all_i_compare.shtml


So 600D is 5dMK2.5? I guess this means all I is the best for quality (horrible for file size), especially with a fixed qp slice. Also explains why buffer is written immediately, there is no additional pass or prediction being done.

[unity2k]

Don't miss the second page, that's where you'll find the punchline!

To you guys who are trying to increase our bit rates, while I cannot make a contribution besides the financial, there are those of us out here wishing you all the best of luck in finding the Magic Bullet for Magic Lantern that might allow us to film with higher quality images as a result of your hard work. Thanks for your contributions.

[1%]

Going to find/fix that 30 minute limit sooner or later. Then its 5d2.6

Also here is the .422 format from a1ex before it gets lost in the other thread:

http://www.fourcc.org/yuv.php#UYVY

[Leon]

@1% : I think the "5x5" part only applies to Hi-Res.  If you select "Simple" it seems to completely ignore whether it is 5x5, 2x1, etc.  I just had a wee play with the options, and an 18MP Hi-Res 5x5 comes out at 34MB, so it does seem to be uncompressed with 4:2:2 subsampling – I got something muddled up before and thought the full-res pics were much smaller.

Can burst not be done at higher quality and faster?  With a UHS-1 Class 10 card, I only get 1.6 fps at only 1056x720 resolution.  If the H.264 encoder is getting 1080p (or are they 972p at this point?) .422, can these not be written directly to the card?  What's limiting?

So if the .422 files aren't compressed, and they are the "input" to the H.264 encoder, what is this jpcore slice qp setting that controls "input" quality, and why does it not seem to exist for x264?

[1%]

What's limiting?

The zooming.

I'm not sure if h.264 is fed from the LV buffer which is only 720x480? or from full size yuv buffer. You'd have ML graphics on the video, would you not? The simple pics come out worse quality than the encoded frames so something must be missing. If I'm not mistaken, the high-res ones clear the overlays too.

I think there are several steps to compression as there are multiple passes and mention of jpeg encoding being finished. So it could go YUV -> jpeg path -> h264E. You're not going to alter the quality of the raw YUV, just the compressor input.

The YUV has to be compressed with SOMETHING before we put it to the card. The simple frames are 3 MB. Card writes at ~20MB... raw would give you 6FPS only. The buffer is 1GB big and if not much is being used in ALL-I, the frames must be much smaller. I don't think the camera could take raw YUV, compress it with H264 and run several passes (for P frames) then write it to the card at over 30fps without some intermediate step.

x264 has slice QP parameter but it will not go below -10 on videos from the camera. But bit rate does rise if lower jpcore slice like 90/95 is used vs 100/110 and recording stops sooner filming the same thing. Q-scale -16 is only 112

[Andy600]

1% - random question - do you think frame size can be increased at a lower bit rate?

[Leon]

In burst mode, there isn't any zooming though, is there?  It's only managing to write 2MB per second in silent frames to my camera when I use burst mode.

Seems so strange that it would waste processing encoding to jpeg and then decoding it to send it to the H264 encoder, unless it is somehow forced by the way their hardware is wired up and communicates.  If the jpcore part is already a bit compressed, maybe that can be written directly to the card bypassing x264.  We only need a 7:1 compression overall (or 4:1 compared to 4:2:2) to get 1080p down to being 20MB/s.

[1%]

do you think frame size can be increased at a lower bit rate

I'm hoping so. Especially if we can capture the input and skip h.264 all together. There is already resizing for digital zoom and 720p/640, etc. The camera sees the full frame either way.

forced by the way their hardware is wired up and communicates

Probably this. Depends if its easier to compress a jpeg to h.264 realtime or raw YUV. It already has to encode the full frame (w/ line skip) so it would be grabbing 30MB of YUV per frame and compressing it x 30FPS + resizing + passes for P frames. That is 900MB/s. What about 60fps... 720P not that much smaller. Buffer would be full all the time. We can already see they couldn't just dump .422 to card because of speed limitations.

Someone with more knowledge of assembly+ HW encoding can correct me but this is what I've gleaned so far.

[1%]

Found new thing about CBR mode. If you're using ALL-I CBR will not adjust qscale/slice quality. This means that next qscale value is calculated from P frames not I frames. Can anyone confirm? Set slice 0, reboot, turn on debugging and see if slice quality changes in all I. Even in gop2 qscale changes, gop 1 is a default of 120, I think thats -8.


*Edit:

More good news... slice QP can be adjusted by as much as you want. For now I set it to drop by 4 when buffer gets to warning level and no errors + no buffer overruns  but quality drops down quickly so I need to find a good mathematical solution either by buffer or bitrate.

Something quick:

https://bitbucket.org/OtherOnePercent/tragic-lantern/downloads/autoexec.bin.quickcbr

If buffer warning drop slice q by 2, if BR < 75mbps raise by 1. Its working at ISO5k in crop mode but its too dark right now for me to test + fine tune. Mainly doing this for ALL-I ... I'll try in longer gops next.


In daylight it still works, ramp down is good, ramp up is a little too fast I think. 1 second polling for BR and buffer is a little too slow, would like .5 seconds. Bit rate is a little bit jaggy (follows card write) but better quantizer used overall than CBR function or VBR function and mostly stops overruns unless they happen really suddenly or scene is already too big to encode. I set buffer warning to like 50/60% and had success at ISO 80-2500.

Low points of BR are usually  above default rate of 40. Footage is overall 1GB per minute. A 7minute continues shot was ~7GB. Average BR is over 120, peaks 160-200. The drop downs aren't really noticeable in watching, I have to see if they show up in post but I doubt they will as BR at worst quantizer is still usually very high.

[1%]

V2, quicker polling (1/2 sec). Instant BR is only 1/2 of actual but now average stays around 100 in gop3 at 50% buffer warning.

https://bitbucket.org/OtherOnePercent/tragic-lantern/downloads/autoexec.bin.600D.Exfat.QuickPollingAutoSlice

[Andy600]

V2, quicker polling (1/2 sec). Instant BR is only 1/2 of actual but now average stays around 100 in gop3 at 50% buffer warning.

https://bitbucket.org/OtherOnePercent/tragic-lantern/downloads/autoexec.bin.600D.Exfat.QuickPollingAutoSlice

This is looking promising 1%

I shot some trees, grass and parked cars for a few mins using ALL-I. Bit rate was around 75-130mbps (q was mainly around 10 but hit 22 according to AVInaptic) with buffer warning set on 50% (that came on a few times but kept recording fine).

I can't quite put my finger on it but I think there is an improvement in over all detail. There is a very slight increase in noise at low ISO's using ALL-I but it's very fine grain and actually looks quite nice to my eyes. I'm testing it with a new LOG PS that I'm developing and it's looking great.

I need to shoot some fast moving stuff but it's getting dark here. Might try some night shots.

[1%]

There is a very slight increase in noise at low ISO's

Probably from deblocking filter being off?

All I pulls higher BR and writes constantly. Slice pull back only happens on buffer warning. With faster polling it ramps up right away as soon as BR falls. If you look at peaks there are some drops but I can't tell for the few frames it does it for.

Gop 3 compresses some frames so BR is more jerky (like I frame is 120 and P frame compressed to 85) but buffer overrun less likely. Q22 I think is when quality dropped but some Q22 better than Q16 or Q17 all the way in CBR.


I got streameye stuidio now so that helps to see what is going on. Seems like this is the best we can get from default encoder. Wish kunie and jason_atl would test because they have some really clear primes.

Moving car footage and buffer stopping will tell too.