Directly setting deblocking filter: Please help my math!

[1%]

I've tried setting deblocking filter through the canon functions but its a crapshoot. Sometimes it changes, sometimes it says invalid value. Sometimes it says nothing at all. Logging Jpcore is difficult because the log is flooded with engine writes (good for figuring out encoder, I guess?).

Values for the filter are -6 to 6.

The 2 functions do this:
NSTUB(0xFF1E3228, str:[JPCORE]_DBFALPHA_DBFBETA)

aAJ_JpcoreSliceqpdD_Qscale_0x00_to_0x20.off_0x1 /*0x6CE1*/ = BYTE((arg0 & 0xF)) -Alpha
aAJ_JpcoreSliceqpdD_Qscale_0x00_to_0x20.off_0x2 /*0x6CE2*/ = BYTE((arg1 & 0xF)) -Beta

and

NSTUB(0xFF1C94E0, mvrSetDeblockingFilter)
str:[JPCORE]_DBFALPHA_DBFBETA(BYTE(*(arg0)), BYTE(arg0->off_0x4)) => ret_str:[JPCORE]_DBFALPHA_DBFBETA_FF1C9514 - Setting B from A + something?




I've used mvrSetDeblockingFilter and got these results. Some B values aren't valid and the function sets something else like B to 1 or 7, etc.

How do I compute this from the signed integers so I can skip calling functions?

Code Select Expand


Deblocking Filter:

A=-6   1800752394  6B554D0A
B=-6   175118849   A701A01

A=-5   1800752395 6B554D0B
B=-6   1918960965 72610545

a=-4   1880752396 701A010C
b=-6   208673281  C701A01

a=0 1880752384 701A0100
b=0 7346689    701A01

a=6 1880752390 701A0106
b=6 108009985  6701A01

a=3 1880752387  701A0103
b=3 57678337    3701A01

-3 1703949    1A000D
-3 218110464  D001A00


[a1ex]

You can try something like this:

Code Select Expand


*(int8_t*)0x6CE1 = alpha;
*(int8_t*)0x6CE2 = beta;


mvrSetDeblockingFilter receives a structure (or array) with two integers (int32) - first is alpha, second is beta.

[1%]

I did that to read them, but what do I throw in there? I made them ints, I should try them as the proper  size.

[a1ex]

Since we are using little endian, "int" would work just as well as "int8_t" in this case.

[1%]

so back to turning 6 into 0x701A0106/0x6701A01 and the like

[1%]

Ok, I did this:

int8_t* xalpha = (int8_t*) (0x6ce1);
int8_t* xbeta = (int8_t*) (0x6ce2);


bmp_printf(FONT_MED, 10,20, "A: %d  B: %d dblock %X", *xalpha, *xbeta, xalpha);


and changed to:

void mvrSetDeblockingFilter(int32_t *, int32_t *);  //-6 to 6


Now when I read from the JPcore array, I get values like 6, 5, 4, 3, 2, 1, 0, 15, 14... 10

So does this mean I'm on the right track and it was a size issue?

I still get some warnings but it seems to be working ok.
i.e
note: expected 'int32_t *' but argument is of type 'int16_t *

[a1ex]

When you call mvrSetDeblockingFilter, you need to do it like this:

Code Select Expand


int arg[2];
arg[0] = alpha;
arg[1] = beta;
mvrSetDeblockingFilter(arg);


Or, equivalent, but without calling mvrSetDeblockingFilter:

Code Select Expand


*(int8_t*)0x6CE1 = alpha;
*(int8_t*)0x6CE2 = beta;


You can also try to check if you set things properly with this:

Code Select Expand


bmp_hexdump(FONT_SMALL, 0, 300, 0x6CE0, 32*10);


[1%]

Thanks!

I have to change

void mvrSetDeblockingFilter(int32_t *); to pass the array? Do I have to put the bracket [] somewhere?

Well without the bracket, what I read out of 6CE0 changes along with what I'm setting. The debug values match what I passed in, I think negative numbers combine with 0xF. The bin dump shows both halves of 6ceo changing and showing numbers much like what I was getting at first.





Dump output is probably a good chunk of jpcore array.

[a1ex]

Yes, arrays in C are just like pointers.

This may help: http://pw1.netcom.com/~tjensen/ptr/pointers.htm

[1%]

Yes, and I need help on figuring out sizes too, esp from the assembly. The more I do the better I get

mvrSetDeblockingFilter (alpha = 6, beta = 6)

mvrSetDeblockingFilter (alpha = -6, beta = -6)

Its working correctly now this way. I can set it directly like the stuff in MVR config but just have to add 15 to the negative numbers.

Is deblocking stronger at -6 (like qscale) or at 6... and need to see what this will do to the footage. Being able to access the rest of Jpcore configs will help too although the encoder seems to use mostly engio writes. Makes HUGE log similar to what you did for "how the camera takes a picture".


ie...
[ENG] [ENGIO](Addr:0xc0f1466c, Data:0x 3201920) EngDrvOut -< maybe setting size?

[a1ex]

Do you see any difference with different deblocking settings?

0xc0f14xxx is related to display device - http://magiclantern.wikia.com/wiki/Register_Map/Brute_Force/C0F14xxx#C0F140xx

[1%]

I haven't done enough testing to find out how much the difference is.

http://forum.doom9.org/showthread.php?t=109747

Nothing Shown.
C0F1466C [ ] [N]

[1%]

Ok, some usefulness:

For H264 encoders so should mean the same thing for us. Alpha:beta
Low- 0:3
Medium- 1:-1
High- 0:-3

-2:-1 - Low, Preserve Details
1:2 - Remove blocks at expense of details

-6:-6 - Pretty much off.

[Audionut]

Negative numbers result in less in-loop deblocking and of course positive numbers result in more in-loop deblocking.

H264 encoder (x264) is smart enough that it reduces deblocking for higher bitrates, and with high enough bitrate will disable deblocking altogether.  I seriously doubt that's happening in the Canon encoder though.

Judging by the OP results in the bitrate thread, it looks like we could easily reduce deblocking strength at higher bitrates.  Try and squeeze as much detail out as possible.

[1%]

Also, to put this in other cameras, mvrSetDeblockingFilter needs to be found and placed in stubs.s for every camera that needs it.