5d3/6D IVA Params Hacking. The new BR control on FF digic V.

[1%]

So I figured out some stuff about H.264 INI. bitrate.c 5d3/6D will have to be different. Had a start from what was discussed in the I/P thread.

The file:

Code: [Select]


Transform8x8Flag = 2
Profile = 100
Level = 51
BitRate = 110000000
EntropyCodingMode = 1
IntraPicInitQP = 5
InterPicInitQP = 5
QpOffsetForB = 0
MinQpI = 1
MinQpP = 1
MinQpB = 1
MaxQpI = 13
MaxQpP = 13
MaxQpB = 13
MinBitrate = 60000000
MaxBitrate = 110000000
SarWidth = 0
SarHeight = 0
AspectRatioIdc = 1
VideoFmtAndVspFlag = 81
VideoFullRangeFlag = 1
TimingInfoPresentFlag = 0
RateControlEnable = 1
ScalingMatrices = 0
pScalingMatrixAddr[0] = 0
pScalingMatrixAddr[1] = 0
pScalingMatrixAddr[2] = 0
pScalingMatrixAddr[3] = 0
pScalingMatrixAddr[4] = 0
pScalingMatrixAddr[5] = 0


Transform8x8Flag = 2 Transform flag. Already set to enabled.
Profile = 100 Cannot be changed. Camera has no other profiles loaded.
Level = 51 Can change this lower. 50 is all I've tried so far. But why?
BitRate = 110000000 Target bit rate for CBR/Rate Control. Adhered to for CBR more. Loosly in RC. Max is ~200-300
EntropyCodingMode = 1 1 is CABAC 0 is CAVLC
IntraPicInitQP = 5 Initial QP for inter and intra frames. Limit of 7 for "rate control" mode.
InterPicInitQP = 5 But a limit of 1 for CBR/VBR... haven't tried 0. MUCH better than any digic IV.
QpOffsetForB = 0 Have not tried but probably QP offset for B frames. I guess can be lower or higher than I.
MinQpI = 1 Min QPs for all modes. Can be 7 in RC, can be at least 1 in the other modes. Not strictly adhered
MinQpP = 1 to for VBR and CBR it seems. There can only be 13 QPs in any mode. 7-20 or 1-13 are a start.
MinQpB = 1
MaxQpI = 13  Max QP for modes that switch QP like VBR and RC. RC will only do 6-20 I believe. VBR will do
MaxQpP = 13 1-13 which seems best.
MaxQpB = 13
MinBitrate = 60000000 Min and Max for the VBR function. Don't do anything in other modes but I think max
MaxBitrate = 110000000 has to match target bit rate or you will get a crash.

RateControlEnable = 1

0= CBR - Stays at initQP and tries to get to the target bit rate. Should change QPs but really doesn't in any consistent way. Can use all QPs. 0 should be tried. Drops video when buffer or complexity is too high. Will record with audio and won't stop until sync is lost, which is real reason why audio stops w/ high bit rate, not buffer.

1 = VBR - Uses Min/Max and I think target. Also seems to switch QPs to keep rate in check. Initial QP can't be 1 I think or it won't switch. I set 5 and it hovered around 2, using 1-13 range. If you have settings too high, camera may freeze. I like this mode the best so far.

2 = Standard buffer saving function like tragic lantern but slightly faster and more stoppy. This is what you get by default but we have no good access to the MVR config. I think its on another proc. Will keep BR pretty low and can only use QPs 7-20. I should try with 1-13 again. It kicks in at the 1st buffer square now but still has a tendency to stop. Should have emphasized buffer and not frame size/prediction, tsk tsk.

ScalingMatrices = 0 Need to test this as I think it means you can put custom scaling matrixes to the encoder. The other fields are for address so I guess you load them into memory and then it uses them. Finding how this works would give a big boost to quality according to what I've read.


Other stuff:

Loading the INI can produce a lot of Err70s. Some of these should be patched I think. Some are from settings being incorrect. Sometimes it does it just because it feels like it. A good idea is to wait till the config finishes loading and the text is off the screen.

Another option will be to set the params 1 at a time from memory and not a file. This would make things a bunch easier. The only other solution is to auto-load configs or edit the files with ML and then make the camera reload them.
Having to load the config every start and hope it doesn't error 70 is a pain.

On 6D params are uploaded:
NSTUB(0xFF1F130C, str:ChangeIVAParam_ls_can_open._Transform8x8Flag_P)

Maybe there are others hidden somewhere or we can gain control of the video encoder. The future looks promising. The present is at least 100MBps with LOW qps and audio working.

RateControl will use 1-13, the crash happens if you let it use more than 13 qps.

Code: [Select]


average DRF                 7.200181
standard deviation          5.679766
max DRF                     13
                           
DRF=1                       219 ( 19.837 %) ####
DRF=2                       337 ( 30.525 %) ######
DRF=3                         0 (  0.000 %)
DRF=4                         1 (  0.091 %)
DRF=5                         2 (  0.181 %)
DRF=6                         2 (  0.181 %)
DRF=7                         2 (  0.181 %)
DRF=8                         1 (  0.091 %)
DRF=9                         1 (  0.091 %)
DRF=10                        1 (  0.091 %)
DRF=11                        2 (  0.181 %)
DRF=12                        1 (  0.091 %)
DRF=13                      535 ( 48.460 %) ##########
DRF>13                        0 (  0.000 %)
                           
I-slices average DRF        7.200181
I-slices std. deviation     5.679766
I-slices max DRF            13


[Marvin]

Great work!

Level might be an arbitary value, just a flag or something, it's implemented as 4.1 for IPB, 5.1 for ALL-I

Did you shoot any samples? what's the max average bitrate you got after the hack? under what ISO?

[1%]

It produces similar bit rates and results to 600D. IPB makes higher rates for some reason before stopping. I can set it to target whatever rate I write to the file. It does crash/stop a bit more than the custom written version for 600D. Rate control halts with CRF 13 so I had videos producing almost 1/2 frames at CRF 1 and 2 and then 1/2 at like CRF 13. Averages out to better than stock. Video looks just like 600D video but I didn't pixel peep.

Its less dependent on ISO than digicIV. I made an 80Mbps 640x480 video with CBR. I can't open any files with bit rate viewer so haven't seen the graph.

There are a few test functions in the firmware to set some of these. Also need to see if my IPB videos are 5.1 or 4.1, I never new profile was supposed to be lower.

*, they are 5.1

[Marvin]

It produces similar bit rates and results to 600D. IPB makes higher rates for some reason before stopping. I can set it to target whatever rate I write to the file. It does crash/stop a bit more than the custom written version for 600D. Rate control halts with CRF 13 so I had videos producing almost 1/2 frames at CRF 1 and 2 and then 1/2 at like CRF 13. Averages out to better than stock. Video looks just like 600D video but I didn't pixel peep.

Its less dependent on ISO than digicIV. I made an 80Mbps 640x480 video with CBR. I can't open any files with bit rate viewer so haven't seen the graph.

There are a few test functions in the firmware to set some of these. Also need to see if my IPB videos are 5.1 or 4.1, I never new profile was supposed to be lower.

*, they are 5.1

You got IPB file with level 5.1? that's pretty impressive, the max bitrate is 300Mbps with 5.1, and only 62Mbps with 4.1 (according to spec)

In my tests I was able to hit 62Mbps with IPB mode. 150Mbps in ALL-I mode.

Maybe high bitrate IPB is better than standard bitrate ALL-I.

[1%]

It definitely seems less prone to stop even though gop is standard. I don't know All I at QP 1 or IPB at QP1 is better. 5d3 should be able to match this so if someone loads up a fast CF card and sets BR to 300 we can see the upper limit.

[1%]

I've turned on the 4x4 matrixes just by toggling the parameter in the config file:

This is 1. Movies don't play back.
Matrix coefficients                      : BT.709-2, SMPTE 274M, SMPTE 296M

Code: [Select]


 Custom intra4X4 luma     
                              16  21  26  31
                              21  26  31  36
                              26  31  36  31
                              31  36 196   4

setting of 2:

Code: [Select]


Custom intra4X4 luma     
                              12  26  40  56
                              26  40  56 213
                              40  56  48  21
                              56  48  21 213

Setting of 3 did not work. Which one is better? 1 or 2?

2 Also won't play back. hmmm data looks like its there.

the sps/pps and nal uints are all screwed up again.


Setting both initQPs to -1 lets them be auto picked and will result in RC respecting your min/max and VBR doing actual VBR. It picked 1-26. For CBR not recommended as its a fixed QP mode. Initial is pretty much what it encodes at for the entire duration.

[Audionut]

It produces similar bit rates and results to 600D. IPB makes higher rates for some reason before stopping.

It should also produce better overall quality.  Properly encoded B-frames can decrease bitrate by 20% or so.  In other words, using B-frames at the same quality should reduce file size by 20%, or, using B-frames should increase quality by 20% at the same file size.  I highly doubt the Canon implementation nets 20%, but 10% shouldn't be unreasonable.

You got IPB file with level 5.1? that's pretty impressive, the max bitrate is 300Mbps with 5.1, and only 62Mbps with 4.1 (according to spec)

With High Profile, yes.  Has it been confirmed that 5d3/6d uses high profile?  (edit:  nevermind, I am catching up on the other thread) The digicIV cameras used baseline profile which reduces the max bitrates to 240Mbps and 50Mbps respectively.  Bear in mind that the level flag is just that, a flag.  You could flag a 8000x8000 resolution video @ 9 gazillion Mbps with 300 reference frames as level 1 if you wanted to.  Afaik, a lot of decoders these days ignore the flag and run a quick analyses themselves to determine playback.

Code: [Select]


 Custom intra4X4 luma     
                              16  21  26  31
                              21  26  31  36
                              26  31  36  31
                              31  36 196   4

setting of 2:

Code: [Select]


Custom intra4X4 luma     
                              12  26  40  56
                              26  40  56 213
                              40  56  48  21
                              56  48  21 213

Setting of 3 did not work. Which one is better? 1 or 2?

Matrices and how they effect encoding is a fairly detailed topic, however, some very basic theories.  The numbers at the top left represent large flat areas, and the numbers at the bottom right represent very fine details.  Larger numbers mean more detail removal (in a nutshell).

Also, left to right and top to bottom represent vertical and horizontal frequencies.  So in those 2 matrices above, the 1st one is applying some extra compression (detail removal) to the horizontal frequencies, and the second is applying more compression to the vertical frequencies.  This could be very useful due to the way the encoder scales the information from the sensor but would require a decent amount of testing to ascertain best case.

See:  http://en.wikipedia.org/wiki/Quantization_%28image_processing%29

JVT Matrix, which iirc is an official H.264 matrix.

Code: [Select]

#jvt

INTRA4X4_LUMA =
 6,13,20,28,
13,20,28,32,
20,28,32,37,
28,32,37,42

INTRA4X4_CHROMAU =
 6,13,20,28,
13,20,28,32,
20,28,32,37,
28,32,37,42

INTRA4X4_CHROMAV =
 6,13,20,28,
13,20,28,32,
20,28,32,37,
28,32,37,42

INTER4X4_LUMA =
10,14,20,24,
14,20,24,27,
20,24,27,30,
24,27,30,34

INTER4X4_CHROMAU =
10,14,20,24,
14,20,24,27,
20,24,27,30,
24,27,30,34

INTER4X4_CHROMAV =
10,14,20,24,
14,20,24,27,
20,24,27,30,
24,27,30,34

INTRA8X8_LUMA =
 6,10,13,16,18,23,25,27,
10,11,16,18,23,25,27,29,
13,16,18,23,25,27,29,31,
16,18,23,25,27,29,31,33,
18,23,25,27,29,31,33,36,
23,25,27,29,31,33,36,38,
25,27,29,31,33,36,38,40,
27,29,31,33,36,38,40,42

INTER8X8_LUMA =
 9,13,15,17,19,21,22,24,
13,13,17,19,21,22,24,25,
15,17,19,21,22,24,25,27,
17,19,21,22,24,25,27,28,
19,21,22,24,25,27,28,30,
21,22,24,25,27,28,30,32,
22,24,25,27,28,30,32,33,
24,25,27,28,30,32,33,35
Here is a overview of the frequencies for an 8x8 matrix.



I hope to have a 5d3 in around 2 months.  I hope that I can be much more useful in testing then.

[1%]

It should be using the 5.1 in IPB. When its not set the BR is limited to 60MBps. The matrix files don't play back, unfortunately, don't know why.

[g3gg0]

The matrix files don't play back, unfortunately, don't know why.

can you upload (a small) one to http://upload.g3gg0.de ?

[1%]

They are all on the 6D thread.


I can make some more when I get back from work:

http://www.qfpost.com/file/d?g=3TjJxDJYB

[Audionut]

I can make some more when I get back from work:

http://www.qfpost.com/file/d?g=3TjJxDJYB

Here is what mkvtoolnix reported when trying to mux the mov to mkv.

Warning: 'F:\Downloads\MVI_6990.MOV' track 0: The AVC video track is missing the 'CTTS' atom for frame timecode offsets. However, AVC/h.264 allows frames to have more than the traditional one (for P frames) or two (for B frames) references to other frames. The timecodes for such frames will be out-of-order, and the 'CTTS' atom is needed for getting the timecodes right. As it is missing the timecodes for this track might be wrong. You should watch the resulting file and make sure that it looks like you expected it to.

[1%]

Might be worth trying with timecode turned off. If we find how the matrices work we can set custom 4x4's which enhance detail. I'll dig through some more of the TI docs because canon pretty much cribbed TI's proc and removed some modes.

[1%]

According to H264/14496-10 doc

Transform8x8flag is

0 - Disabled. Only 4x4 transforms are used (default).
1 - Allows the additional use of 8x8 transform. Results in optimal RD performance
since it considers all possible modes
2 - Consider only 8x8 transform modes (i.e. disables 4x4 transform)


scaling matrix and all the addresses mean:

0 Not Present – Disabled (Default)
1 Present only in SPS
2 Present only in PPS
3 Present in both SPS and PPS

they are called ScalingListPresentFlag0
first 5 are for 4x4,  6+7 are for 8x8 which is not present in config

Setting transform to 1 and scaling matrixes to 3 produces video with these characteristics and its playable. all other combos so far seem not to work. 4x4 custom flag is not present however

Color primaries                          : BT.709-2, SMPTE 274M, SMPTE 296M
Transfer characteristics                 : BT.709-2, SMPTE 170M, SMPTE 274M, SMPTE 293M, SMPTE 296M
Matrix coefficients                      : BT.709-2, SMPTE 274M, SMPTE 296M

vs default:

Color primaries                          : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics                 : BT.709-5, BT.1361
Matrix coefficients                      : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177

*This breaks 60P

[Marvin]

What do you mean by, "This breaks 60P?"

[1%]

720P60 records green instead of video with these settings and the height is wrong.

Here are some encoder docs... we have pretty much the same thing in 5d3/6d

Encoder Parameters
http://www.ti.com/general/docs/lit/getliterature.tsp?literatureNumber=spraba9&fileType=pdf

canon implemented some differently...

data sheet:
http://www.ti.com/general/docs/lit/getliterature.tsp?literatureNumber=sprugx8a&fileType=pdf



FAQ: http://processors.wiki.ti.com/index.php/DaVinciHD_Codecs_FAQ

    How does the target bit rate relate to the achieved output bit rate?

The transrater produces a bit-stream that has an average bit rate which is within +/- 5% of the target bit rate

    What is the maximum resolution supported by the TI encoder?

Currently, TI has 2 separate encoders on DM6467. The 720p encoder can support resolutions from QCIF up to 720p. It can also support 1920 x 544 resolution as a special case. The 1080p encode can support resolutions from 640 x 448 up to 1920 x 1088. The restriction on 1080p30 encoder is that the width must be a multiple of 32, and height must be a multiple of 32.Also if the input resolution is 1920x1080, the last 8 lines of the frame need to padded and provided by the application to the encoder

    Can the Hdvicps do encode/decode for 422 chroma format?

No, the Hdvicps do not support 422 chroma format.

    How to do the 422 --> 420 chroma conversion (for encoder) and 422 --> 420 chroma conversion (for display) on DM6467?

The VDCE engine on the DM6467 can be used for the chroma conversions. Alternately, you could even use the DSP to do the chroma conversion.

    Can the VDCE do up-scaling of YUV?

No, the VDCE can be used only for down-scaling the YUV. It does not support up-scaling.

So 422 and 2k or anything is out with the hardware encoder. Scaling is NOT so if YUV buffer is scaled it could be read and encoded via something else.. but dma/CPU are the only options.

[Marvin]

Nice find, well done

does this mean, 1080/50/60p is possible in the hardware?

[1%]

Means its not possible, nor is 4:2:2

[Marvin]

ok, well I guess 1D C uses a different h264 encoder then.

[1%]

Makes me wonder what it uses.  They did mod the design so its not 1:1. But the FPS in 1080 and 720 and behavior + parameters match.

Also camera has no such limitation itself, you could create a 4K 4:22 buffer I bet. Probably easy to switch out encoder part... 650D has no ivaparam but has digic V. well... easy if you're the factory. Would explain the almost identical performance of 600d with digic IV.

[Marvin]

ok then, maybe we'll just leave the mystery as it is?

which directions are we heading towards now?　focusing on the scaling and bitrate parameters?

[nanomad]

I did a quick search and couldn't find an HW realtime 4K encoder. I'd really like to see the internal of a 1Dc

[1%]

which directions are we heading towards now?

Best thing would be to make the YUV buffer 1:1` and encode that. or figure out the LV jpegs (422). The scaling is what really degrades the image.

You won't get much from anything except 1d firmware decompile. the encoder probably inside the digic package... and I thouht 1dc 2k/4k is jpeg, not 264

[nanomad]

Ah, that explains it then

[Marvin]

Best thing would be to make the YUV buffer 1:1` and encode that. or figure out the LV jpegs (422). The scaling is what really degrades the image.

You won't get much from anything except 1d firmware decompile. the encoder probably inside the digic package... and I thouht 1dc 2k/4k is jpeg, not 264

Only 4K is M-JPEG, all other recording formats are H264.

[1%]

Hmm... is it 4:2:2 though? they co-designed with ti  so they could have solved some of the regular encoder's problems. They renamed the parameters slightly already. Have to look inside to see.