h264.ini - how it really works?

[Sijimo]

I recall when i had 5D2 it was super easy to change bitrate settings inside ML menu, right now i can't figure out how to do it on 5D3.
After some research i have downloaded some ini's, tried to put them in ML folder on my CF but still can't figure out how it works.
I have 1000x computerbay 128gb for RAW video and now i also want to record H264 MOV at 2x or 3x bitrate.

Is it possible somehow with the current nightly build?

[eyeland]

Bump

[Audionut]

So I figured out some stuff about H.264 INI. bitrate.c 5d3/6D will have to be different. Had a start from what was discussed in the I/P thread.

The file:

Code: [Select]


Transform8x8Flag = 2
Profile = 100
Level = 51
BitRate = 110000000
EntropyCodingMode = 1
IntraPicInitQP = 5
InterPicInitQP = 5
QpOffsetForB = 0
MinQpI = 1
MinQpP = 1
MinQpB = 1
MaxQpI = 13
MaxQpP = 13
MaxQpB = 13
MinBitrate = 60000000
MaxBitrate = 110000000
SarWidth = 0
SarHeight = 0
AspectRatioIdc = 1
VideoFmtAndVspFlag = 81
VideoFullRangeFlag = 1
TimingInfoPresentFlag = 0
RateControlEnable = 1
ScalingMatrices = 0
pScalingMatrixAddr[0] = 0
pScalingMatrixAddr[1] = 0
pScalingMatrixAddr[2] = 0
pScalingMatrixAddr[3] = 0
pScalingMatrixAddr[4] = 0
pScalingMatrixAddr[5] = 0


Transform8x8Flag = 2 Transform flag. Already set to enabled.
Profile = 100 Cannot be changed. Camera has no other profiles loaded.
Level = 51 Can change this lower. 50 is all I've tried so far. But why?
BitRate = 110000000 Target bit rate for CBR/Rate Control. Adhered to for CBR more. Loosly in RC. Max is ~200-300
EntropyCodingMode = 1 1 is CABAC 0 is CAVLC
IntraPicInitQP = 5 Initial QP for inter and intra frames. Limit of 7 for "rate control" mode.
InterPicInitQP = 5 But a limit of 1 for CBR/VBR... haven't tried 0. MUCH better than any digic IV.
QpOffsetForB = 0 Have not tried but probably QP offset for B frames. I guess can be lower or higher than I.
MinQpI = 1 Min QPs for all modes. Can be 7 in RC, can be at least 1 in the other modes. Not strictly adhered
MinQpP = 1 to for VBR and CBR it seems. There can only be 13 QPs in any mode. 7-20 or 1-13 are a start.
MinQpB = 1
MaxQpI = 13  Max QP for modes that switch QP like VBR and RC. RC will only do 6-20 I believe. VBR will do
MaxQpP = 13 1-13 which seems best.
MaxQpB = 13
MinBitrate = 60000000 Min and Max for the VBR function. Don't do anything in other modes but I think max
MaxBitrate = 110000000 has to match target bit rate or you will get a crash.

RateControlEnable = 1

0= CBR - Stays at initQP and tries to get to the target bit rate. Should change QPs but really doesn't in any consistent way. Can use all QPs. 0 should be tried. Drops video when buffer or complexity is too high. Will record with audio and won't stop until sync is lost, which is real reason why audio stops w/ high bit rate, not buffer.

1 = VBR - Uses Min/Max and I think target. Also seems to switch QPs to keep rate in check. Initial QP can't be 1 I think or it won't switch. I set 5 and it hovered around 2, using 1-13 range. If you have settings too high, camera may freeze. I like this mode the best so far.

2 = Standard buffer saving function like tragic lantern but slightly faster and more stoppy. This is what you get by default but we have no good access to the MVR config. I think its on another proc. Will keep BR pretty low and can only use QPs 7-20. I should try with 1-13 again. It kicks in at the 1st buffer square now but still has a tendency to stop. Should have emphasized buffer and not frame size/prediction, tsk tsk.

ScalingMatrices = 0 Need to test this as I think it means you can put custom scaling matrixes to the encoder. The other fields are for address so I guess you load them into memory and then it uses them. Finding how this works would give a big boost to quality according to what I've read.


Other stuff:

Loading the INI can produce a lot of Err70s. Some of these should be patched I think. Some are from settings being incorrect. Sometimes it does it just because it feels like it. A good idea is to wait till the config finishes loading and the text is off the screen.

Another option will be to set the params 1 at a time from memory and not a file. This would make things a bunch easier. The only other solution is to auto-load configs or edit the files with ML and then make the camera reload them.
Having to load the config every start and hope it doesn't error 70 is a pain.

On 6D params are uploaded:
NSTUB(0xFF1F130C, str:ChangeIVAParam_ls_can_open._Transform8x8Flag_P)

Maybe there are others hidden somewhere or we can gain control of the video encoder. The future looks promising. The present is at least 100MBps with LOW qps and audio working.

RateControl will use 1-13, the crash happens if you let it use more than 13 qps.

Code: [Select]


average DRF                 7.200181
standard deviation          5.679766
max DRF                     13
                           
DRF=1                       219 ( 19.837 %) ####
DRF=2                       337 ( 30.525 %) ######
DRF=3                         0 (  0.000 %)
DRF=4                         1 (  0.091 %)
DRF=5                         2 (  0.181 %)
DRF=6                         2 (  0.181 %)
DRF=7                         2 (  0.181 %)
DRF=8                         1 (  0.091 %)
DRF=9                         1 (  0.091 %)
DRF=10                        1 (  0.091 %)
DRF=11                        2 (  0.181 %)
DRF=12                        1 (  0.091 %)
DRF=13                      535 ( 48.460 %) ##########
DRF>13                        0 (  0.000 %)
                           
I-slices average DRF        7.200181
I-slices std. deviation     5.679766
I-slices max DRF            13


[tjaja]

Code: [Select]

Transform8x8Flag = 2
Profile = 100
Level = 51
BitRate = 440000000
EntropyCodingMode = 1
IntraPicInitQP = -1
InterPicInitQP = -1
QpOffsetForB = 0
MinQpI = 1
MinQpP = 1
MinQpB = 1
MaxQpI = 7
MaxQpP = 7
MaxQpB = 7
MinBitrate = 90000000
MaxBitrate = 440000000
SarWidth = 0
SarHeight = 0
AspectRatioIdc = 1
VideoFmtAndVspFlag = 81
VideoFullRangeFlag = 1
TimingInfoPresentFlag = 0
RateControlEnable = 2
ScalingMatrices = 0
pScalingMatrixAddr[0] = 0
pScalingMatrixAddr[1] = 0
pScalingMatrixAddr[2] = 0
pScalingMatrixAddr[3] = 0
pScalingMatrixAddr[4] = 0
pScalingMatrixAddr[5] = 0

Gave me


It plays choppy on the pc. Maybe because the datarate is too high, then I'm not complaining.

[1%]

its not too bad.. know that you can set it so high that the encoder can't keep up and drops or mangles frames. I could barely play back some 400Mb/s clips and above some point it causes aberrations... like near the area you're at. Probably 150-160 was where it was semi-balanced

Also, ALL-I is nice compared to 24Mb/s IBP but not so nice compared to 100Mb/s IBP. There is a ratio to this, something like 2:1. So you need 100Mb/s all I to be like 50Mb/s IBP... its on H264 sites.

[Audionut]

Code: [Select]

Transform8x8Flag = 2
Profile = 100
Level = 51
BitRate = 840000000
EntropyCodingMode = 1
IntraPicInitQP = -1
InterPicInitQP = -1
QpOffsetForB = 0
MinQpI = 1
MinQpP = 1
MinQpB = 1
MaxQpI = 13
MaxQpP = 13
MaxQpB = 13
MinBitrate = 200000000
MaxBitrate = 840000000
SarWidth = 0
SarHeight = 0
AspectRatioIdc = 1
VideoFmtAndVspFlag = 81
VideoFullRangeFlag = 1
TimingInfoPresentFlag = 0
RateControlEnable = 2
ScalingMatrices = 0
pScalingMatrixAddr[0] = 0
pScalingMatrixAddr[1] = 0
pScalingMatrixAddr[2] = 0
pScalingMatrixAddr[3] = 0
pScalingMatrixAddr[4] = 0
pScalingMatrixAddr[5] = 0

Code: [Select]

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.1
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 2 frames
Format settings, GOP                     : M=3, N=12
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 22s 523ms
Bit rate                                 : 135 Mbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 2.720
Stream size                              : 363 MiB (98%)
Language                                 : English
Encoded date                             : UTC 2014-03-05 12:40:23
Tagged date                              : UTC 2014-03-05 12:40:23
Color primaries                          : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics                 : BT.709-5, BT.1361
Matrix coefficients                      : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177


Setting the bitrate/maxbitrate above 100000000 causes crashes or other problems.
MaxQp controls the bitrate in RC 2.  I tried setting this to 10, it worked (175 Mbps), but the buffer was filling up.  When the buffer fills, you also gets crashes or other problems.

The same config with RC 1 only delivers around 60 Mbps.  Adjusting MaxQp didn't seem to make any difference.  This also crashes with bitrate/maxbitrate above 100000000.  Can't seem to get the bitrate any higher in RC 1 mode.

Didn't bother to check All I mode.

Lexar 1000x 16GB

[kgv5]

Did you guys see any improvement in image quality?

[tjaja]

Until now I haven't.

[tjaja]

Original IPB produces:

www.alisultan.nl/magicbug/original.png

New IPB with this code

Code: [Select]

Transform8x8Flag = 2
Profile = 100
Level = 51
BitRate = 700000000
EntropyCodingMode = 1
IntraPicInitQP = -1
InterPicInitQP = -1
QpOffsetForB = 0
MinQpI = 1
MinQpP = 1
MinQpB = 1
MaxQpI = 7
MaxQpP = 7
MaxQpB = 7
MinBitrate = 450000000
MaxBitrate = 700000000
SarWidth = 0
SarHeight = 0
AspectRatioIdc = 1
VideoFmtAndVspFlag = 81
VideoFullRangeFlag = 1
TimingInfoPresentFlag = 0
RateControlEnable = 2
ScalingMatrices = 0
pScalingMatrixAddr[0] = 0
pScalingMatrixAddr[1] = 0
pScalingMatrixAddr[2] = 0
pScalingMatrixAddr[3] = 0
pScalingMatrixAddr[4] = 0
pScalingMatrixAddr[5] = 0

produces

www.alisultan.nl/magicbug/newbitrate.png

The stream of the original:

Code: [Select]

Video
ID                                       : 1
Formaat                                  : AVC
Formaat/Info                             : Advanced Video Codec
Formaatprofiel                           : High@L4.1
Formaatinstellingen, CABAC               : Nee
Formaatinstellingen, ReFrames            : 2 frames
Format_Settings_GOP                      : M=3, N=15
Codec-ID                                 : avc1
Codec-ID/Info                            : Advanced Video Coding
Duur                                     : 10s 210ms
Bitrate                                  : 30,7 Mbps
Breedte                                  : 1 920 pixels
Hoogte                                   : 1 080 pixels
Beeldverhouding                          : 16:9
Frameratemodus                           : Constant
Framerate                                : 29,970 fps
ColorSpace                               : YUV
ChromaSubsampling                        : 4:2:0
BitDepth/String                          : 8 bits
Scansoort                                : Progressief
Bits/(Pixel*Frame)                       : 0.494
Stroomgrootte                            : 37,4 MiB (100%)
Taal                                     : Engels
Datum codering                           : UTC 2014-03-05 16:56:55
Tagdatum                                 : UTC 2014-03-05 16:56:55
colour_primaries                         : BT.709
transfer_characteristics                 : BT.709
matrix_coefficients                      : BT.709
The new bitrate

Code: [Select]

Video
ID                                       : 1
Formaat                                  : AVC
Formaat/Info                             : Advanced Video Codec
Formaatprofiel                           : High@L5.1
Formaatinstellingen, CABAC               : Ja
Formaatinstellingen, ReFrames            : 2 frames
Format_Settings_GOP                      : M=3, N=15
Codec-ID                                 : avc1
Codec-ID/Info                            : Advanced Video Coding
Duur                                     : 10s 310ms
Bitrate                                  : 134 Mbps
Breedte                                  : 1 920 pixels
Hoogte                                   : 1 080 pixels
Beeldverhouding                          : 16:9
Frameratemodus                           : Constant
Framerate                                : 29,970 fps
ColorSpace                               : YUV
ChromaSubsampling                        : 4:2:0
BitDepth/String                          : 8 bits
Scansoort                                : Progressief
Bits/(Pixel*Frame)                       : 2.159
Stroomgrootte                            : 165 MiB (100%)
Taal                                     : Engels
Datum codering                           : UTC 2014-03-05 16:57:28
Tagdatum                                 : UTC 2014-03-05 16:57:28
colour_primaries                         : BT.709
transfer_characteristics                 : BT.709
matrix_coefficients                      : BT.709
EDIT
High detail scenes crashes immediately. Trying to find a balanced setting.

[Luiz Roberto dos Santos]

Level 5.2 is not supported? And, it's config. is ALL-I [GOP 1]? Maybe this instability is because you set very high maxbitrate... try to config. less, and analyse it with appropriate tools (push to the limit curves, for example, this will show how different macroblocks stay).

[1%]

Yea, tried 5.2 before and no dice

It will crash/hang/stop on a complex scene mainly in VBR/CBR modes, the rate control usually pulls through. Also watch for complex scenes causing bottom frame garble if you set the QP too high and it starts throwing away data.

I think VBR is the hardest encoding mode to get working across scenes. Plus interesting to compare profile 4.x and 5.1.

[tjaja]

Any recommendation for ini settings to try out? I don't have a 6D but a 5DIII.

[ricordu]

Same question here !
Would be great to get explanation on how to have those option available again...

ricordu


Threads merged //Audionut

[anDyIII]

1%, could your rc/vbr/cbr.ini work on ML 1.2.3?
Are there any news about bitrate on 1.2.3?
Thanks

[tjaja]

Only way is to try it yourself.

[Audionut]

I've merged some relevant posts into this thread.

It plays choppy on the pc. Maybe because the datarate is too high, then I'm not complaining.

its not too bad.. know that you can set it so high that the encoder can't keep up and drops or mangles frames.

Looks to be if you set the MaxQp setting to low.

I've used MaxQp 7 on some footage with no fine detail, in others words, a fairly low bitrate.  But I still noticed some frame drops in the footage.

This is using RateControlEnable 2 (IPB).  And I'm not sure yet how the bitrate settings effect the results.  MaxQp 13 looks very safe.

IIRC, changing QpOffsetForB caused instant crash.

Did you guys see any improvement in image quality?

Until now I haven't.

Same here.  I think the higher bitrates may preserve edge detail a little better, buy I have bias.  I really need to do a proper comparison and create a poll.

On the old digic4 cameras, I did notice a difference.

Firmware Default
http://dl.dropbox.com/u/34113196/ML/Firmware%20Default.png

Q15
http://dl.dropbox.com/u/34113196/ML/Q15.png

1 has detail in the leaves.  The other has blurry green things with no detail.

Cropped 300% resize to exaggerate.

[far.in.out]

How do I get to play with this on 50D?

[tjaja]

 1% or a1ex, could it be I semi bricked My camera and that i should register a default ini Again to repair it?

Situation: every time i record a video with h264 the buffer overruns: no matter what card.
It didnt happen before and If i scan through the forum I can conclude some functions can be registered and stays that way if you bypass certain checks.

If so: does anyone know the standard setting for 5d3 so i can load it?

If i bricked it, c'est la vie, i knew it was experimental.

[a1ex]

Does it help if you clear Canon settings, or if you start without a ML card?

I didn't play much with these ini files, but I thought all of them are temporary.

The default ini files can be found on bitbucket: https://bitbucket.org/hudson/magic-lantern/src/tip/platform/5D3.113

(the defaults were extracted from the 1.1.3 firmware; I didn't check if anything changed on 1.2.3)

[tjaja]

Sorry for the late response: Will update after the weekend when didi and I are together

[pacus]

hey ist there a good working h264.ini out there, that delivers a better image quality?

I understand the varibales but, maybe there is a testet one out there that works fine...

Thanks for your help.

[tjaja]

@ A1ex

Tried without magiclantern on the sd: problem gone. Can't recreate the issue.

[MGerard]

I found some usable settings for the h264.ini that give a small but noticeable differences in the clarity of the picture (on the 5D3), meaning less blocking.

Unfortunately, as reported before, there is a great chance for error 70, especially when pressing record again after the first recording. It seems to me that changing the image itself, altering the angle etc. gives a small chance that the error 70 will not happen. Maybe it has to do with some buffer not being cleared before the recording starts or something like that - reloading the .ini also gives a greater chance that the error will not happen, but that is pure speculation on one side as it happens or not... It would be cool if some devs may take a look at this problem.

If anyone wants to see the results I got, please PM me.

Thanks in advance!

[dubzeebass]

Why not just p post them

[Audionut]

I found that if I gave the camera some time before making changes, or after making changes (loading the ini for instance), it seemed to behave well here.

And yes, you should just post them