Author Topic: 5d3/6D IVA Params Hacking. The new BR control on FF digic V. (Read 23297 times)

Audionut · « **Reply #50 on:** April 23, 2013, 02:15:15 AM »

IPB always level 4.1
All-I always level 5.1

No CABAC

Quote from: 1% on April 23, 2013, 01:58:16 AM

Also is it showing the correct drive on the screen when it loads a config? Not sure what ML takes as CARD_DRIVE

If I create a ML folder on the CF card to place the ini files. It complains about having ML on both cards and won't load.

edit: It doesn't show a drive. Just "ML/whatever.ini"

1% · « **Reply #51 on:** April 23, 2013, 02:29:41 AM »

Its weird configs aren't working on 5d3, if it says config loaded in the menu the location must be getting set to 1.

You'd have to check logs and see if ivaparam mode messages are in them. The text on the screen is just a return so whether it loaded or not it will say what you tried to load.

If 5d3 can record QP1 to the CF card without jerking or defective frames then that is theoretically the best quality. The only other parameters that seem to matter are CABAC vs CAVLC and transform 8x8 flag. The profile is only loosely enforced and doubly so without any rate control mechanism active. All frames will be QP1... there is another thing we can do but I have to reverse it... set SPS/PPS directly. But its complicated since its 1 function and has to set P/I/B and "0" "1" frames. It should work if the QP/Target locations worked tho.

Also gop can be altered with video parameters but I don't know if the lenght on 5d3 is the same as 6D.

Audionut · « **Reply #52 on:** April 23, 2013, 02:55:35 AM »

Quote from: 1% on April 23, 2013, 02:29:41 AM

You'd have to check logs and see if ivaparam mode messages are in them.

I'm not sure how to do this.

1% · « **Reply #53 on:** April 23, 2013, 03:23:08 AM »

PROP_VIDEO_MODE is 0x80000039 .. you can look with property browser.. need the lenght to try changing gop.

http://www.qfpost.com/file/d?g=vv13nWgb7

Has debug mesage logging on don't click me.

Hit it once to start log and again to save it. Saves to root of the card like 1MB.

Audionut · « **Reply #54 on:** April 23, 2013, 03:38:16 AM »

https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/DM.LOG

I searched the log for 'iva' and nothing was found.

I was in override mode. Switched that off, selected VBR mode and loaded config. Recorded small sample. Switched to CBR mode and loaded config and recorded small sample.

1% · « **Reply #55 on:** April 23, 2013, 04:25:18 AM »

Thing with debug log is that canon dumps a ton of messages and its easy to overwrite stuff. Don't even record a movie, just start and hit load config manually like 10 times.

This is what it looks like:

encdrwrapSetIVAParamMode:B:/ML/rc.ini

If this isn't showing maybe have to call it by address on 5d3?

Audionut · « **Reply #56 on:** April 23, 2013, 04:48:15 AM »

encdrwrapSetIVAParamMode:ML/cbr.ini

Missing the B:/ which I assume is the card drive.

https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/DM1.LOG

Audionut · « **Reply #57 on:** April 23, 2013, 05:13:58 AM »

Quote from: 1% on April 23, 2013, 01:58:16 AM

It should work Like:

Autoload ON: Set initqp or BR and changes don't take effect, read from config.
Override: Loads Config + Overrides InitQP + Target Rate if set
Autoload OFF: pref from a cold boot- change INITQP and Target, should apply when you leave menu

It may show an initQP/BR set with autoload or when it hasn't updated yet. Then should write OFF over it.

Can't comment on the reading from config, but the only way to adjust Initqp is with Override. In any other mode, Bitrate and InitQP are greyed out and can't be adjusted.

1% · « **Reply #58 on:** April 23, 2013, 06:07:55 AM »

Set config to disable or set autoload to off & reboot. You have to reload the menu to refresh it.

a1ex · « **Reply #59 on:** April 23, 2013, 09:06:19 AM »

On 5D3, you need to say A:/ or B:/. The FIO calls have a wrapper that autodetects the drive, but here you need to write it explicitely.

Audionut · « **Reply #60 on:** April 23, 2013, 09:41:43 AM »

Might be worth having a user option to select. Or detect which drive ML is installed on and run from there.

a1ex · « **Reply #61 on:** April 23, 2013, 09:49:57 AM »

It already does that, but for regular file I/O calls, not for hacks.

1% · « **Reply #62 on:** April 23, 2013, 05:57:31 PM »

http://www.qfpost.com/file/d?g=mF0qzViJY

Ok, now loads configs from B:/. It was turning on IVA mode since you couldn't change QP/BR but drive was not being selected. Thanks!

Audionut · « **Reply #63 on:** April 24, 2013, 12:24:15 AM »

Yep, that did it.

I'll do some testing today and report back.

edit: The ability to edit ini files in camera would be cool. Am I asking to much?

Audionut · « **Reply #64 on:** April 24, 2013, 01:42:24 AM »

I made 1 recording using this ini.
Just realized as I was copy pasting that bitrate and maxbitrate weren't the same. Maybe 1 to many 0's in max bitrate ;-)

Code: [Select]

Transform8x8Flag = 2
Profile = 100
Level = 51
BitRate = 800000000
EntropyCodingMode = 1
IntraPicInitQP = -1
InterPicInitQP = -1
QpOffsetForB = 0
MinQpI = 1
MinQpP = 1
MinQpB = 1
MaxQpI = 26
MaxQpP = 26
MaxQpB = 26
MinBitrate = 100000000
MaxBitrate = 8000000000
SarWidth = 0
SarHeight = 0
AspectRatioIdc = 1
VideoFmtAndVspFlag = 81
VideoFullRangeFlag = 1
TimingInfoPresentFlag = 0
RateControlEnable = 1

That worked for the first recording, then adjusted shutter timing to get 23.976 shutter.

I got 2 crashlogs. This one and another the same except for task MovieRecorder
lv:1

Code: [Select]

ASSERT: 0
at ./MovieRecorder/EncDrvWrapper.c:1865, task MovieRecorder
lv:0 mode:3


Magic Lantern version : v2.3.NEXT.2013Apr23.5D3113
Mercurial changeset   : badff1b15d04 (unified) tip
Built on 2013-04-23 15:51:36 by user@D610.
Free Memory  : 253K + 3838K

And https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/LOG000.LOG

With bitrate from the above ini changed to 8000000000 to match maxbitrate, first time hit record camera, crash, battery pull. Put battery in and try again, err 70. Guess it doesn't like stupid bitrate.

When I muxed a file into mkv, mkvtoolnix reported,

Code: [Select]

Warning: 'O:\DCIM\100EOS5D\D46A6713.MOV': This file contains at least one frame with a negative timecode. All timecodes will be adjusted by 00:00:00.125 so that none is negative anymore.

Bitrate viewer still complained trying to open the file and I have no access to streameye :-(

Using vbr config and switching to override mode without loading the config again, crash.

https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/LOG001.LOG

Using vbr config, switch to override, load config and all works good. Restart camera still in override mode and crash on record, remove battery and try again err 70. Same crash logs as before, not sure if the log000 are the same.

https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/LOG002.LOG

Tried changing the GOP, mediainfo reports standard GOP, but I don't really trust it, and I can't think of any other way I have to check.
Here is a small file with GOP 3 to check.
https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/D46A6721.MOV

Using the below ini, push record, LV and ML overlays all keep working fine, you can't stop recording, nothing it written to card and ML says error writing, restart, and restarts.

Code: [Select]

Transform8x8Flag = 2
Profile = 100
Level = 51
BitRate = 500000000
EntropyCodingMode = 1
IntraPicInitQP = -1
InterPicInitQP = -1
QpOffsetForB = 2
MinQpI = 1
MinQpP = 1
MinQpB = 1
MaxQpI = 15
MaxQpP = 15
MaxQpB = 15
MinBitrate = 50000000
MaxBitrate = 200000000
SarWidth = 0
SarHeight = 0
AspectRatioIdc = 1
VideoFmtAndVspFlag = 81
VideoFullRangeFlag = 1
TimingInfoPresentFlag = 0
RateControlEnable = 1

At some stage in the future, I wonder if it's possible to put some sanity checks in.

1% · « **Reply #65 on:** April 24, 2013, 02:55:30 AM »

Quote

Am I asking to much?

I'm a bit of a shitty coder but there may be a way to change the parameters on the fly. An in camera text editor would take care of it too. There is also probably a better way to handle all of this.

Give it a second to load up at startup, don't hit record right away. Go in and out of the menu and check that loaded says yes. I may have to patch a few more errors for 5D3 too, thought I got all the asserts. Once you have decent settings its usually good to go on startup at least from my experience with 6D.

Gop is probably going to have to be done from video parameters property. It didn't work on 6D till I did it that way.

Min/Max only apply to VBR mode. The other modes ignore it. We can add some sanity checks when we figure out what's good. I think max 500Mb/s in IPB and 300 in all I. Its just a target rate anyway, if it can't generate it at QP1 it can't go any higher.

If you override a dynamic config it won't flip QPs, just record fixed QP.

Any stutter or green frames, etc? I had that problem at QP1 with flush rate but it was generating 400Mb/s

Quote

[LVKAO] Unfortunately Bad Face

This is amusing... maybe related to why the not writing error happens. I have to investigate it more rather than just resetting after 10s.

Audionut · « **Reply #66 on:** April 24, 2013, 03:13:52 AM »

Quote from: 1% on April 24, 2013, 02:55:30 AM

Any stutter or green frames, etc? I had that problem at QP1 with flush rate but it was generating 400Mb/s

Haven't noticed any stuttering. Green frames only once with as you say QP1 and flush rate set to stop bitrate pumping. Was only the first second or 2 iirc.

1% · « **Reply #67 on:** April 24, 2013, 03:49:04 AM »

Makes me wish I had cf

Maybe those errors are write errors if 5d3 isn't experiencing them. I though the encoder wasn't keeping up but I think the encoder just can't write out fast enough.

That assert is:
FF1EA0F4 arString = %s , pString = %s

It looks like a file reading error.

I can patch it but it could be from how you edit the files. Some text editors screw things up. I used notepad++ on windows and pluma on linux. I had problems with notepad for some reason.

Audionut · « **Reply #68 on:** April 24, 2013, 06:33:01 AM »

I found the frame skips. When the buffer doesn't flush fast enough. It must be related to CABAC (processor load) also.
I can push close to 200MB/s with CAVLC, but with CABAC I get to around 150MB/s and it shits itself. Tried playing with a heap of flush rate options but doesn't appear to be any difference, which is what makes me think it's processor related.

Just to clarify, if I push to hard in CAVLC it either stops recording or errors out. But I can adjust the flush rate to get that high without frame loss (skipping).
I can push CABAC high enough to stop or error also, but only around 150MB/s before it cracks the shits and starts throwing frames away.

Audionut · « **Reply #69 on:** April 25, 2013, 02:38:33 PM »

Can I get some pointers how I can merge your changes with the source to build my own?

1% · « **Reply #70 on:** April 25, 2013, 03:44:48 PM »

Swap bitrate 5d3.c for bitrate-6d in the makefile

https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/raw/7caa1b2946342c74ddd9abc0c2132776108045b2/src/bitrate-6d.c

Audionut · « **Reply #71 on:** April 25, 2013, 04:52:02 PM »

Cheers, that did the trick.

1% · « **Reply #72 on:** April 29, 2013, 04:28:48 PM »

Just in case anyone needs this: 5d3 w/ br and TCC. I dunno if LV rec should have been recompiled: I guess test it.

http://www.qfpost.com/file/d?g=4uFsXjjgm

Audionut · « **Reply #73 on:** April 29, 2013, 04:41:33 PM »

What's TCC?

1% · « **Reply #74 on:** April 29, 2013, 05:26:00 PM »

Its like picoC but faster. Lets you run the LV recorder.

News:

Author Topic: 5d3/6D IVA Params Hacking. The new BR control on FF digic V. (Read 23297 times)