Bit rate investigation

Started by Audionut, July 19, 2012, 04:54:03 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

AriLG

Quote from: nanomad on September 14, 2012, 04:09:42 PM
It Will probably work on any camera but you will probably be stuck with a low video duration
How low ? I have seen the number 4 minutes somewhere...
T3i (main), T2i
------------------
It's not about accuracy,  it's about Aesthetics

nanomad

ALL-I should be approximately 1Gb per minute of video and you're limited by the 4GiB file limit
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

AriLG

Quote from: nanomad on September 14, 2012, 07:39:39 PM
ALL-I should be approximately 1Gb per minute of video and you're limited by the 4GiB file limit
OK...wow

We are starting to reach RAW numbers...  :)

So, another thing that makes the 600d worth buying ?...
T3i (main), T2i
------------------
It's not about accuracy,  it's about Aesthetics

FilmMan

Quote from: Andy600 on September 14, 2012, 12:58:49 PM
Shot some of this using GOP3 and ALL-I :) https://vimeo.com/groups/superneutral/videos/49436519 (Vimeo compression sucks)

Andy600,

Awesome.   By the way, it takes guts to fly like that.  Looks like alot of fun.

1%

600D in crop is also the closest to HD size. So when 4:2:2 comes it will be up there with 5dIII if it can write/process fast enough.

Other cameras have much smaller LV buffers and upress much more. So from current info 2k would be 2k from silent pic size. Right now 1080P is uprezzed from silent pic size too. Big disappointment, huh?

FilmMan

1%,

1.  How stable is your bitrate program?  I inboxed Alex and if the program is stable he'd port them.  Myself and others, who have the 5D2, would love to be early adopters and test your bitrate program.  To be fair, donations would come too from myself, and others indicated they'd donate too.   

2.  Alex mentioned there needs to be alot of work for the mjpeg.  I'm hopeful but if it is uprezzed would the quality suffer?  Interesting stuff here with the 40d,  but only 1024x680. http://www.magiclantern.fm/forum/index.php?topic=1452.50

Everyone is hoping for your successes.  Lot's of new cameras coming on the scene, but these Canon cameras new and old, seem to have the horsepower but unleashing that power is the trick.  You've accomplished so much in such short order with your bitrate.  Cheers.

1%

Its stable enough. Only problem is the buffer saving function will drop quality really really quick.

I think MJPEG is worth it. You're already uprezzing the same thing, just losing extra color. A1ex hasn't released any code so we can't work on it without starting from scratch. Perhaps more donations for this are in order. There are a few devs at least willing to give it a go, just need a starting point. Even if you have to convert the movies before opening them its still a win. Audio is pretty much done and I don't think we can push the encoder much further.


FilmMan

Quote from: 1% on September 15, 2012, 07:50:23 PM
Its stable enough. Only problem is the buffer saving function will drop quality really really quick.


Are you going to forward to Alex so he can port? 
(I'll donate just to try your bitrate program.  Also, I'll post on that other camera forum and ask people to donate so they can try.)

Quote from: 1% on September 15, 2012, 07:50:23 PM

I think MJPEG is worth it. You're already uprezzing the same thing, just losing extra color. A1ex hasn't released any code so we can't work on it without starting from scratch. Perhaps more donations for this are in order. There are a few devs at least willing to give it a go, just need a starting point. Even if you have to convert the movies before opening them its still a win. Audio is pretty much done and I don't think we can push the encoder much further.


I'm trying to understand here, so excuse my ignorance.  Do the cameras downsize to a specific video size then upsize thru the encorder and leave out color info?  So as Chucho stated, 5d2 is  1120x752. http://www.magiclantern.fm/forum/index.php?topic=2649.msg11595#new
For example, does the 5D2  insides downsample to that level for video (for 5D2: 1120 x 752) then uprezz to 1080p changing the 422 color space to 4:2:0 in the process?  Or am I totally out to lunch here?

Are you going to ask Alex for the code?  You were amazing on the speed with what you accomplished with the bitrate. 
Cheers


JasonATL

1% - I have been shooting a lot of test footage, varying GOP and other parameters. I have a few observations and questions (more observations are sure to come).

I tried setting slice to 86. When recording during this, it resulted in a camera error that was a little scary. It said to pull the battery and then it tried to save the file (never did). It recovered fine, but didn't save the movie. I won't be trying slice=86 again.

As we have discussed in this thread, the buffer saving function drops the quality and this is visible to me in the footage. For a scene in which the buffer fills, then is saved, then fills again, etc., I describe this as a "breathing" or "pulsing" look, in which the footage has a regular pulse revealed when the save function kicks in. The frames during which the slice is lowered don't look so bad by themselves. The problem is that they look different and hence, there is this "pulsing" look to the footage that has regular changes.

It seems that there might be an advantage to the enhancing the buffer save function. As I understand it, it increases slice by 2 when buffer is above the warning and then drops it by 1 when bitrate < 75. The polling must be really quick on this, as the changes are quite quick. Perhaps the increment by which the slice is changed could be a function of distance of the buffer from its warning level? What I have in mind is something along the lines of:
slice = max(87, 87+x*[buffer - warning]), or
slice = max(87,87+x*sign(buffer-warning)*(buffer-warning)^2)
Perhaps x could be set so that the maximum slice value is achieved at 10% above the buffer warning? Is something like this possible? Do you think it might help?

An offset to warning could also be used to preventatively start raising slice as it nears the warning level. However, the user could do this themselves by setting a lower warning level, or if it was there might set buffer warning higher.

I know I say this in every post, but this really is great! Thanks again for your work on this. I am definitely getting the highest quality footage that I have ever seen with my 600D with this enhancement.

1%

Firefox ate my reply but the cliff notes are:


Mjpeg the way to go. Its a1ex's code so you have to convince him. He already sees all of this but has no 600D and isn't a big bit rate fan. You pretty much have it right tho. LV buffer is what it sees through the line skps and then it uprezzes that (process is called something in imaging, not new). Overhead will be saved from the uprezzing (probably why 720P has 60FPS). Only danger is that we put in a lot of work and its impossible hw wise, etc. Donate for that rather than polishing canon's turd of an H264 implementation.

Right now everything is 5DIII but there are a few devs willing to work on it, just we have no starting point besides the LV buffer address. A1ex some sort of implementation already and its for 5dII so probably more portable. I think he wanted to make something really good before releasing but other things took over. I think its really good already even if you can't play the videos back in camera or have to convert them before viewing, etc. All I saw is demo vids.

Other cameras are a lot of effort to port and I don't have the firmwares which take 2 days to decompile anyway. The other cameras may not have the same functions (50d gop didn't change) plus crippled by the 4gb limit which gives you 4 minutes (550D out already)... a little too short. Also don't have the cameras themselves on hand so I'd be coding blind and a simple mistake could be a headache. MJPEG I think is less camera specific.

---
I do see what you're saying with the buffer leveling function and its pretty crappy. I think miyake was going to look at this stuff and his better C skills probably help.

Quote

if (MVR_BUFFER_USAGE > (int)buffer_warning_level) // Drop Slice if buffer
     {   if ( (bitrate_mode == 1) && (qscale_slice >4) )
       qscale_slice -= 3;
   }
   if ( MVR_BUFFER_USAGE < ((int)buffer_warning_level-25) ) //Raise if BR getting low.
   { if ( (bitrate_mode == 1) && (qscale_slice < 45) && (measured_bitrate<75) && (measured_bitrate!=0) )
      qscale_slice++;
   }


That's basically all of it... if buffer gets a warning drop slice quickly until the warning goes away, if buffer is 25% under utilised raise quality. So quality drops too quickly with card write and then suddenly jumps up to max where it probably should be more conservative. Kind of a quality yo-yo. Breathing was a good analogy. This is just what I came up with in a short amount of time. There is also a whole nother scale for slice 0-79. Some are lower qualities but I think some are not. Its weird, it mirrors and wraps around in the same range (something-7F).  Could be an avenue for better control. H264 itself still doesn't want to write a quantizer better than 10 even if the input frame is bigger/better.  I mentioned 86 in the commit but it never made it here and got buried. It will just freeze your camera but.... why.. and also everything "works" like LV and camera just says busy if you try to stop recording.



miyake

QuoteI think miyake was going to look at this stuff and his better C skills probably help.

I want to help you. But I'm now learning a lot of your bitrate work ....
   @@@@  / ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄
   (゜д゜@ < catching up with you!
   ┳⊂ )   \____________
  [[[[|凵ノ⊃
   ◎U□◎ =3 キコキコキコ

JasonATL

1% - Thank you for the thoughtful reply.

Here's another idea of a possibly gentler decrease in qslice when buffer fills (in the same spirit as the suggested algorithm above in that it sets the slope of the qslice to depend on the severity of the buffer situation). You have probably thought of such an approach already, but I just want to encourage you not to give up!

Quoteif (MVR_BUFFER_USAGE > (int)buffer_warning_level) // Drop Slice if buffer
     {   if ( (bitrate_mode == 1) && (qscale_slice >4) )
       qscale_slice -= (int)((MVR_BUFFER_USAGE - (int)buffer_warning_level)/2 + 1); // slope depends on buffer vs. warning
   }
   if ( MVR_BUFFER_USAGE < ((int)buffer_warning_level-25) ) //Raise if BR getting low.
   { if ( (bitrate_mode == 1) && (qscale_slice < 45) && (measured_bitrate<75) && (measured_bitrate!=0) )
      qscale_slice++;
   }

The "/2"  can be /3 or /4 to make it even gentler. Given the results with -= 3 that you had before, I thought that /3 might work well, since, as written, if buffer warning level is 70% with /3 in the code, you would get -= 3 at 76% buffer, changing to -= 4 at 79% buffer. Again, user can set warning level to 60% if 70% doesn't work.

nanomad

If someone can collect buffer % vs time data in different scenarios we can probably come up with a prettier function   ::)
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

FilmMan

@1%,
Thanks for the reply.   

Here's some thoughts.

People are excited about the new GH3 having 72mbp/s All-I codec.  People wanted the quality for video. 

Now you have attained in the neighborhood of 100 mbp/s with stability (although the buffer quality drop exists, I think you and the team will figure it out in the next while).   Also, there is the 4GB limit for some cameras which gives about 4 minutes of record time.  I think for most people this won't be an issue.  I'd even be okay with 1 minute of record time. 

Once the bitrate is to your satisfaction, I can then present to others and mention to donate as this bitrate exists and the ML team wants to explore the possibility of mpeg 422?  Or I can keep talking about the current progress about the bitrate and the future mpeg possibility and encourage people to donate now?  This is very exciting stuff. 

The new Sony and Nikon have come with hdmi out in their new models.    However, an upped bitrate which you developed, may
add enough to the video quality to save people from buying new cameras?  Only time will tell.

To everyone working on this, hats off to you.  All of your efforts are very appreciated.

1%

QuoteIf someone can collect buffer % vs time data in different scenarios we can probably come up with a prettier function

This exactly. I've tried -2, -1, -4 and lower buffer percentages for the rise. Nothing has worked as well. /2 might work or speeding up the polling beyond 1/2 second. Instant BR can be multiplied back for accuracy. Of course this will vary with complexity/iso, etc. We can try static to test and the way it is now it can almost pull it off, much better than just qscale number and close to CBR. Also it might not need to drop quality down so far. Need to find a number where it can recover from an overfilling buffer but isn't pixelated so badly. Some more gradual leveling might be better.
Also might help to add the other #s and see what kind of qp they produce in the file.

Can try changing the gop on 550D (4mins better than 0 minutes), not sure what happened for a1ex on 60D (if it stopped writing or just gave the err70 that needs to be jumped). The other stuff is a bit harder and definitely needs fw to be looked at.

Quotepossibility of mpeg 422

Donations specifically for this purpose might encourage A1ex to release some info and that people are interested. i'm just in it for the video.

1%

Increased polling to 100ms. BR indicators are off, you're not getting 200-300 but you are getting a more stable BR.

I haven't pushed it because I'm not sure if its better or worse. Quality shifts aren't as extreme but also doesn't ramp up as quick. Haven't checked QPs in the files yet.

http://pastebin.com/QyCaEjek

https://bitbucket.org/OtherOnePercent/tragic-lantern/downloads/autoexec.bin.lessaggressive-faster


JasonATL

Quote from: nanomad on September 16, 2012, 03:59:32 PM
If someone can collect buffer % vs time data in different scenarios we can probably come up with a prettier function   ::)

What did you have in mind? Here's a graph of time (x-axis), bitrate, and buffer.

https://picasaweb.google.com/lh/photo/VNwz0VaV1ZhKUV9YQEfYgBfNx8dP82C1K8bmBL14X3k?feat=directlink
(how do attach pics here?)

I shot this using TV static. However, I aimed the camera above the TV and slowly panned down to increase the amount of static in the shot. I then panned back up to decrease. I recorded my screen using a second camera and got bitrate data from Bitrate Viewer. You can see the buffer save function kick in.  Any suggested modifications of this experiment in order to collect more relevant data?

My settings are:
CBR 3.0x
DBlockA = -6
DBlockB = -6
PicPC = -7
GOP = 3
BuffWarnLevel = 60%
Averaging = OFF

1% - Let me know what other type of test or data I can help with.

JasonATL

A couple of more runs with this type of test. As I said above, I used TV static to have a reliable source to cause a high bitrate. This time, I panned slowly to increase amount of TV static in the shot and paused for at least 8 seconds before increasing panning again to increase the static. Then, I looked at the bitrate graph of the resulting file.

It appears that (at least for 8 second intervals), a bit rate somewhere around 160-170 Mbps can be sustained by the buffer and my cards at GOP=3. I used a Lexar 600x SD and Sandisk Extreme (45 Mbps). Results were the same for both. On the Lexar, the buffer was at 25% during the 10 or so seconds of 160 Mbps bitrate. When bitrate went to 175 Mbps, the buffer increased steadily until it hit the warn level after about 7 seconds. This 7 seconds corresponds to my experience shooting in less controlled situations (outdoors). Using the Sandisk card, the buffer was at 30% during 8 seconds of 170 Mbps bitrate. The buffer warning and save function kicked in after 3 seconds when bitrate increased to 184 Mbps.

Graphs here:
Lexar: https://picasaweb.google.com/102645356447404039003/ML_GOP_BR?authkey=Gv1sRgCIuEydWZ1ZXPsAE#5788868294372408914
Sandisk: https://picasaweb.google.com/102645356447404039003/ML_GOP_BR?authkey=Gv1sRgCIuEydWZ1ZXPsAE#5788868296496242962

Do we know the size of the 600D available buffer? My guess would be around 200MB. If this is the case, then the save function could be written to limit the bitrate directly rather than wait on the buffer warning?

Bottom line is that, for me, it appears at the limit is around a sustained rate of 170 Mbps. It would be nice to allow short bursts a little higher, but perhaps brute force is necessary here to assure stability?

1%

Problem is with the 100ms polling, BR indicators are off by a little. Average bit rate also seems to fail after 4gb but we're using instant to adjust anyway. Need to fix it. CBR shouldn't make a difference as its neutered from changing quality. I set it to 2x and forgot it.  Check BR in the file.

I've stopped it from adjusting down to 21, Now it should stop at 19 DRF/qscale. Maybe less pixelation but maybe not fast enough to stop an overflow, have to test.  We're polling in 1/10s of a second now.

https://bitbucket.org/OtherOnePercent/tragic-lantern/downloads/autoexec.bin

Not sure how to graph it exactly. Wish I could keep it it under 150 real MBps as thats where my card write is consistent. Complex scenes also slow the encoder and lead to the buffer filling. Sometimes even the lowest quality can't stop it. But much better than old CBR set to 3x. I'm testing from 80-1250 ISO... this will affect it much, especially w/ static.

*seems better.

Had only 1 stop in crop mode at ISO 640 on a moving tree that would have eaten buffer before. Going to check BR/quality.

JasonATL

Quote from: 1% on September 16, 2012, 10:31:48 PM
Not sure how to graph it exactly. Wish I could keep it it under 150 real MBps as thats where my card write is consistent. Complex scenes also slow the encoder and lead to the buffer filling. Sometimes even the lowest quality can't stop it. But much better than old CBR set to 3x. I'm testing from 80-1250 ISO... this will affect it much, especially w/ static.

1% - Another great improvement and the polling increase should help, too. Again, thank you for continuing to move this forward.

I think it isn't so much the complexity of the scene, but the combination of the complexity, the buffer state (how full it is), and the QS. I've done more testing. I think we're learning a lot now that can help.

In my latest test (and I've done the same experiment a few times to suggest that this is a representative result), again with the TV static and panning down to control how much, I got the following results, ending in a buffer overflow.
Bitrate viewer graph: https://picasaweb.google.com/lh/photo/bjPc-wCz4ooixLkAn3DbOBfNx8dP82C1K8bmBL14X3k?feat=directlink
You can see the video of the ML screen and debug here:


The complexity of the full screen of static that occurs at 0:17 in the recording (0:21 in the video of the screen) is not associated with an overflow because QS is already at a low number, so the bitrate only gets to about 150 there. A similar thing happens at 0:28. But, at the end, this scene (same complexity) comes at a time when QS is high, so bitrate skyrockets. In this case, QS does not start decreasing until it is too late, because at the high bitrate, the buffer is filling too fast.

The solution? The amount by which you lower qscale_slice should depend on both the buffer state and the bitrate. Why? Because the buffer will fill faster at the higher bitrate and, therefore, more drastic measures are necessary. I'm not even sure this will do it. However, it gives it more of a chance. What could also help is to use the bitrate and current buffer state to PREDICT the buffer state and therefore lower qscale_slice accordingly. It seems like this approach could reduce the chances even further of a buffer overflow.

And yes, I'm going back on my original suggestion on a gentler slope. I apologize. It was my hope that it would soften that appearance of the grain. But, I think the key is to make it only as steep or as gentle as necessary.

1%

Seems like that is what canon tried to do with CBR. Maybe can poll what it thinks qscale should be in non all-I modes. Also might help to lower quality faster when BR is already high and less/slower when BR is low. I'll have to figure out why its inaccurate first. Also maybe stop raising if BR is already at X amount regardless of room in the buffer.

JasonATL

Quote from: 1% on September 17, 2012, 01:01:50 AM
Also maybe stop raising if BR is already at X amount regardless of room in the buffer.

I see what you are saying. If BR is, say, 120 Mbps, then the scene is obviously complex at the current QS, so why raise the QS? Makes sense to me. Again, it could even be better to combine it with the buffer level. Either or both would help with the "breathing" issue, I think. I would just want that level to be set suitably high.

Also, in regards to earlier statements regarding the utility of this whole exercise.
1. I don't think that the 4GB or 4 min limit will be of concern to most who would use this. My shots are rarely more than a couple of minutes long.
2. I think that, if this feature can be further refined (and it is darn near good enough for me already), that most will find this one of the best features of ML. People want clean HDMI out to be able to get 4:2:2 or increase bitrate. This satisfies the latter. And, seeing the results that I'm seeing, would really call into question the need for 4:2:2 for most people, I suspect.

1%

I do a bit of chroma key stuff and seeing the color on the demo vid was great. Also you gain some zoom out of it like in crop mode but lose the in camera uprezing. Also would kill 30 minute limit which nobody has been able to so far.

I tried some quantizers in the lower end and a lot of them freeze the camera. The ones that made videos had really low DRF like 52 or 48. So if we really wanted to put the brakes on video rate.... I should try 85-80 again and make sure there is no higher q in there as some of them froze at X5, X3, etc.

Probably the time try to port to other cameras will be when 600D is maxed out like with audio and most stuff is already discovered and bugs worked out.

JasonATL

Understood. I would love to have 4:2:2 at this quality. Just saying that having both would be great!

One more point to consider. Is there a way to start a recording at a lower QS and ramp up? Your point earlier to ramp up QS only if below a specific BR made me think that this would be a "safer" way to start a recording. Without it, if there is a very complex scene when someone hits record, then the buffer will fill before the save function has a chance to kick in. This happened to me yesterday with your previous build. By starting QS below its max and quickly ramping up if BR is sufficiently low would only take a few frames at lower quality in return for possibly a more stable function.