ImagePlayDriver (JPEG encoding/decoding and image buffer manipulation)

[Maqs]

Hey,

here's some stuff I found. Doesn't work in LV though, as Canon locks the resources it needs (see ResLock stuff thread).

The following is by no means complete:

Code: [Select]

// start image player (driver only)
// there's also StartImagePlayer(...)
StartImpDrvOnly();
// start image play driver, returns handle
void *img_play_driver_handle = StartImagePlayDriver();

// do something with it
void *handle = Execute.....(img_play_driver_handle, params, cb, cb_priv);
wait for Execute... to finish, i.e. using a semaphore
// delete handle
DeleteImagePlayDriverHandle(handle);

// stop image play driver by handle
StopImagePlayDriver(img_play_driver_handle);
// stop image player
StopImagePlayer();

There's quite a list of things you can do:

• void *ExecuteJpegAnalysis(void *img_play_driver_handle, struct jpeg_decode_params, void (jpeg_decode_cb)(void *cb_priv, int err), void *cb_priv);
  Analyze (as in decode, but return meta data only) a JPEG
• void *ExecuteJpegDecoding(void *img_play_driver_handle, struct jpeg_decode_params, void (jpeg_decode_cb)(void *cb_priv, int err), void *cb_priv);
  Decode a JPEG
• void *ExecuteYuvResizing(void *img_play_driver_handle, struct yuv_resize_params, void (yuv_resize_cb)(void *cb_priv, int err), void *cb_priv);
  Resize YUV buffer
• void *ExecuteYuvRotating(void *img_play_driver_handle, struct yuv_rorate_params, void (yuv_rorate_cb)(void *cb_priv, int err), void *cb_priv);
  Rotate YUV buffer
• void *ExecuteLowPassFilter(void *img_play_driver_handle, struct lpf_params, void (lpf_cb)(void *cb_priv, int err), void *cb_priv);
  Some low pass filter?
• void *ExecuteRectangleCopy(void *img_play_driver_handle, struct rect_copy_params, void (rect_copy_cb)(void *cb_priv, int err), void *cb_priv);
  Rect to rect copying
• void *ExecuteRectangleColoring(void *img_play_driver_handlestruct rect_color_params, void (rect_color_cb)(void *cb_priv, int err), void *cb_priv);
  Guess what ;-)
• void *ExecuteJpegEncoding(void *img_play_driver_handle, struct jpeg_encode_params, void (jpeg_encode_cb)(void *cb_priv, int err), void *cb_priv);
  Encode jpeg

All the functions above result in a call to the specified callback with the specified value as the first and an error code (0 meaning success) as the second parameter. They immediately return with the return value being an image play driver handle that needs to be deleted _after the operation finishes_ using

Code: [Select]

DeleteImagePlayDriverHandle(void *handle);
The uninit functions can also be called from the callback, but Canon code usually uses a semaphore to wait after calling the Execute* function (GiveSemaphore in the callback, TakeSemaphore after the function call).

I only checked ExecuteJpegDecoding/ExecuteJpegAnalysis and ExecuteJpegEncoding so far. The parameters should be something like that:

Code: [Select]

struct jpeg_decode_params
{
  struct memSuite *in_suite; // needed!
  unsigned int in_size; // needed!
  int unknown_1;
  int unknown_2;
  int unknown_3;
  int out_addr; // address of output
  int out_size; // size of buffer for output?
  int x;
  int y;
  int width;
  int height;
  int Pfil;
  int PreRes;
  int xxa;
  int xxb;
  int xya;
  int xyb;
};

struct jpeg_encode_params
{
  int yuv_maybe; // always 1?
  void *baseAdd; // address in memory
  unsigned int baseW;
  int x;
  int y;
  int width;
  int height;
  int Pfil; // 1
  int PreRes; // 1
  int xxa; // 8
  int xya; // 8
  int jpgAdd; // allocated buffer for output
  unsigned int maxSize; // size in output buffer
  int unknown; // 2
  int Qscal; // 2
  unsigned int *size; // used to return real size, must not be NULL
};
Feel free to try around and fill in the gaps. 

[reddeercity]

I what to bump this out of the moth balls  , this looks like this could help me with my D4 development in UHD and Lossless TWOin TWOout
and of course 4k Mjpeg .

[a1ex]

Some of these - ExecuteRectangleCopy and ExecuteRectangleColoring - are already emulated well in QEMU. The others are perfect for understanding how the image processor works. They are very similar to ProcessTwoInTwoOutLosslessPath, too.

Wanted: io_trace_full logs while running these (and all other "easy" image processing routines, like FA_MaxSelectTestImage, EekoAddRawPath & friends).

Caveat: io_trace logging is very likely to crash on DIGIC 4 (figure out why).

[reddeercity]

Thanks a1ex ,
I stumbled upon  this thread thought the old magic lantern ml-devel groups.google 
Lots of great old D4 reverse engineering information , found some very useful info specially for the 5d2 .

I give i/o trace a try again , I did try a month ago but messed up something so I moved on .
Looks like there's no getting away from it (QEMU) -- It coming apparent that if I want my D4 development
to progress I'll need to get it up and running once and for all .

What I really was looking for was the jpeg encoder for mjpeg , and discovered that d4 has some retains for mjpeg

On 5D2, th syntax is: call("lv_save_jpeg", 1).
yep, lv_save_jpeg is the function that enabled the flag i told.
its the same as setting the memory variable via ptp.

lvcdevResourceGet is the key to MJPEG. The problem is that it's not called on new cameras, but only on old ones and only in low-res liveview modes. On 5D3, attempt to call it via TryPostEvent results in ERR70.

NSTUB(0xFF111928, str:lvcdevResourceGet_LVC_OUTPUT_DEVICE_T) looks like its supposed to output jpeg or .422 depending on which paths are picked.

Code: [Select]

172:  7211.112 [JOB] ERROR GetImageBuffer failed
    173:  7211.223 [LV] ##LV_CaptureTestImage 00000007
    174:  7211.510 [LV] ##ChangeProperty PROP_LV_DISPSIZE = 1 (1)
    175:  7212.062 ERROR [GMT] GmtEvProc_lv_old_start - State:6
    176:  7337.684 [CAPE] ERROR :VRAM(0x445c380)WB(0x40c1260)
    177:  7337.783 [CAPE] ERROR :JUKAI WR(0xf528580)RD(0x0)
    178:  7337.851 [CAPE] ERROR :JPEG WR(0x0)RD(0xc234700)
    179:  7337.918 [CAPE] ERROR :FEN YWR(0x0)WR(0xf6584d0)RD(0x572dc0)
    180:  7337.963 [CAP]:ISR >>>>>>>>> HEAD ERROR >>> FIFO ERROR
    181:  7338.013 [LV] ERROR [EVF] NotifyHeadErrorCBR
    182:  7338.110 [LV] ERROR [PATH] VRAM[W](0xc267a20)   YUV[W](0x40d4b80)
    183:  7338.184 [LV] ERROR [PATH] FENC[W](0xf575c00) QUARK[W](0x0)
    184:  7338.252 [LV] ERROR [PATH] FLIC[W](0x6000978)  TA10[R](0x572dc0)
    185:  7338.308 [LV] ERROR [PATH] JPEG[R](0x729a60)  JPEG[W](0x0)
    186:  7353.176 [LV] ERROR [PATH] Path Incomplete[100]
    187:  7353.364 [LV] ERROR [PATH] (Rec_HD)CRAW[W](0x44332200->0x46e9460)
    188:  7353.933 [LV] ERROR [PATH] (Rec_HD)CRAW[W](0x44332200->0x46e9460)
    189:  7387.811 [MR] mvrFixQScale FIX_QSCALE(0:VBR, 1:FIX) = 0
    190:  7388.091 [MR] mvrSetDeblockingFilter (alpha = -6, beta = -6)
    191:  7388.181 [MR] mvrSetQscaleYC (Y:-12, C:0)
    192:  7390.411 [MR] mvrChangeAckCBR : Video - Mode=0, Type=0, Rate=24, GOP=3
    193:  7713.703 [CAPE] ERROR :VRAM(0x4476fa0)WB(0x40dc620)
    194:  7713.816 [CAPE] ERROR :JUKAI WR(0xf52a380)RD(0x0)
    195:  7713.877 [CAPE] ERROR :JPEG WR(0x0)RD(0xc234700)
    196:  7713.942 [CAPE] ERROR :FEN YWR(0x0)WR(0xf658580)RD(0x572dc0)
    197:  7713.985 [CAP]:ISR >>>>>>>>> HEAD ERROR >>> FIFO ERROR
    198:  7714.036 [LV] ERROR [EVF] NotifyHeadErrorCBR
    199:  7714.125 [LV] ERROR [PATH] VRAM[W](0xf156ce0)   YUV[W](0x40f0cc0)
    200:  7714.186 [LV] ERROR [PATH] FENC[W](0xf575c00) QUARK[W](0x0)
    201:  7714.256 [LV] ERROR [PATH] FLIC[W](0x6000978)  TA10[R](0x572dc0)
    202:  7714.317 [LV] ERROR [PATH] JPEG[R](0x729a60)  JPEG[W](0x0)
    203:  7728.538 [LV] ERROR [PATH] Path Incomplete[100]
    204:  7728.722 [LV] ERROR [PATH] (Rec_HD)CRAW[W](0x44332200->0x46e9460)

Code: [Select]

lock = TwoInTwoOutJpegPath_CreateResLockEntry(ttj_struct)
TwoInTwoOutJpegPath_work_maybe(lock, ttj_struct, &output_size, job_id_maybe)
TwoInTwoOutJpegPath_DeleteResLockEntry(lock)
chunk = GetFirstChunkFromSuite_maybe(ttj_struct.off_0x88)
jpeg_address = GetMemoryAddressOfMemoryChunk(chunk)


Code: [Select]

rmtFAPCLVStart
SyncSendPipeEvent [0][0][16]
Dispatch : Cur = 0, Event = 16, Param = 0x0
SetOutPutDevice:2,1,2
Change OutPutDevice to PCTFT = 3
pLvCtrlCBR OutPutDevice=(0x3)
GUI_OTHEREVENT:24
WriteDMACInterrupt(2)(0x2)(0)
[ENGIO](Addr:0xc0f04204, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f04200, Data:0x       4) EngDrvOut
[PATH] WriteEDmacYuvCompleteCBR
IDLEHandler OTHER_REMOTE_LV_CTRL new, old(0x3)(0x1)
GUI_SetLvOutputDeviceState (3)
PropCBR camctrl:0x80050011,0x3
ptpPropChangeEvCBR[80050011][4][3]
SendPipeEvent [0][0][9]
changeCBR PropID(0x80050011)Parameter(3)Size(4)
[ENGIO](Addr:0xc0f07134, Data:0x       4)[0]
[EVF] evfSetParamInterrupt
[LVDS] CheckFrame LoopNum1 0
[LVDS] CheckFrame LoopNum2 6 TempFrameCheck 0x3F
[GCTR] LVGAIN_CTRL_GetHShadingData : [49][49][0]
WriteDMACInterrupt(0)(0x2)(0)
[ENGIO](Addr405796b4][0][0]a:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f04000, Data:0x       4) EngDrvOut
[PATH] WriteEDmacWbIntegCompleteCBR

[IRCTR] GetVShadingData(Iso=49)
[IRCTR] GetIRShadingData(po0=80[125](80) Av=2a)
[DCTR] LVDEFECT_CTRL_GetDefmData(H=0)(v=0)
[IRCTR] GetIRShadingData(po0=80[125](80) Av=2a)
WriteDMAC0x4f634c80 - 0x4f634c80] O](Addr:0xc0f04304, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f04300, Data:0x       4) EngDrvOut
[PATH] WriteEDmacVramCompleteCBR

ReadDMACInterrupt(9)(0x2)(0)
[ENGIO](Addr:0xc0f04904, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f04900, Data:0x       4) EngDrvOut
[PATH] ReadEDmacHivshd_V_CompleteCBR
[ENGIO](Addr:0xc0f07148, Data:0x       4)[0]
[EVF] evfReadOutDoneInterrupt
[LVDS] CheckFrame LoopNum1 0
[LVDS] CheckFrame LoopNum2 6 TempFrameCheck 0x3F
[BUFF] RELEASEWRITING[VramArray] (State[3] Addr[0x40d07800][0])
[BUFF] RELEASEWRITING[YuvArray] (State[3] Addr[0x44000080][0])
[BUFF] WRITING[VramArray] (State[1] Addr[0x4f11d800][2])
[BUFF] WRITING[YuvArray] (State[1] Addr[0x44000080][0])
[VRAM] FrameParam[0x90d0e4][0x4f11d800][0x44000080][0x0]
[PATH] Af_StopPath
[ENGIO](Addr:0xc0f100a0, Data:0x       4) EngDrvOut
[ENGIO](Addr:0xc0f100d0, Data:0x       4) EngDrvOut
[ENGIO](Addr:0xc0f10040, Data:0x80000000) EngDrvOut
[PATH] Af_StopPath Complete
[PATH] ObInteg_StopPath[0x0]
[PATH] Hist_StopPath[151de]
[ENGIO](Addr:0xc0f150c0, Data:0x80000000) EngDrvOut
[ENGIO](Addr:0xc0f15080, Data:0x80000000) EngDrvOut
[ENGIO](Addr:0xc0f16018, Data:0x      11) EngDrvOut
[ENGIO](Addr:0xc0f16010, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f16014, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f16008, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f1600c, Data:0x       0) EngDrvOut
[PATH] Hist_StopPath Complete
[PATH] WbInteg_StopPath
[ENGIO](Addr:0xc0f0b0c4, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f0b0c4, Data:0x       1) EngDrvOut
[PATH] WbInteg_StopPath Complete[f529c00]
[ENGIO](Addr:0xc0f09000, Data:0x80000000)[0]
[ENGIO](Addr:0xc0f0b000, Data:0x       1)[1]
[ENGIO](Addr:0xc0f0b004, Data:0x       0)[2]
[ENGIO](Addr:0xc0f11400, Data:0x       1)[0]
[ENGIO](Addr:0xc0f11404, Data:0x       0)[1]
[ENGIO](Addr:0xc0f1100c, Data:0x       0)[2]
[ENGIO](Addr:0xc0f11340, Data:0x       1)[3]
[ENGIO](Addr:0xc0f11380, Data:0x       1)[4]
[ENGIO](Addr:0xc0f2a000, Data:0x80000000)[0]
[ENGIO](Addr:0xc0f11240, Data:0x       1)[0]
[ENGIO](Addr:0xc0f11244, Data:0x       0)[1]
[ENGIO](Addr:0xc0f2c004, Data:0x80000000)[2]
[ENGIO](Addr:0xc0f2c008, Data:0x       0)[3]
[ENGIO](Addr:0xc0f2c00c, Data:0x       0)[4]
[PATH] Rec_HD_StopVramPath[db0400][0]
[ENGIO](Addr:0xc0f0d000, Data:0x80000000)[0]
[ENGIO](Addr:0xc0f11004, Data:0x       0)[1]
[ENGIO](Addr:0xc0f111c0, Data:0x       1)[2]
[ENGIO](Addr:0xc0f112c0, Data:0x       1)[3]
[ENGIO](Addr:0xc0f11300, Data:0x       1)[4]
[ENGIO](Addr:0xc0f11580, Data:0x       1)[5]
[ENGIO](Addr:0xc0f11584, Data:0x       0)[6]
[ENGIO](Addr:0xc0f11440, Data:0x       1)[7]
[ENGIO](Addr:0xc0f11444, Data:0x       0)[8]
[ENGIO](Addr:0xc0f11044, Data:0x       0)[9]
[ENGIO](Addr:0xc0f1a000, Data:0x       1) EngDrvOut
[ENGIO](Addr:0xc0f1a004, Data:0x       0) EngDrvOut
[PATH] Rec_HD_StopPreproPath[c628000][0]
[ENGIO](Addr:0xc0f08130, Data:0x80000000)[0]
[ENGIO](Addr:0xc0f083b0, Data:0x       0)[1]
[ENGIO](Addr:0xc0f08180, Data:0x80000000)[2]
[ENGIO](Addr:0xc0f08020, Data:0x80000000)[3]
[ENGIO](Addr:0xc0f08000, Data:0x80000000)[4]
[ENGIO](Addr:0xc0f08540, Data:0x80000000)[5]
[ENGIO](Addr:0xc0f080a0, Data:0x80000000)[6]
[ENGIO](Addr:0xc0f08090, Data:0x80000000)[7]
[ENGIO](Addr:0xc0f04934, Data:0x       3) EngDrvOut
ReadDMACInterrupt(9)(0x10)(1)
[ENGIO](Addr:0xc0f04904, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f04934, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f04900, Data:0x       4) EngDrvOut
DummyCBR
[ENGIO](Addr:0xc0f04834, Data:0x       3) EngDrvOut
ReadDMACInterrupt(8)(0x10)(1)
[ENGIO](Addr:0xc0f04804, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f04834, Data:0x       0) EngDrvOut
[ENGIO](Addr:0xc0f04800, Data:0x       4) EngDrvOut
[PATH] ReadEDmacDefCorreAbortCBR
[VRAM]vram_SetSenserDriveParameter(73, 1271, 0 1)
[VRAM]vram_SetSenserDriveParameter(0, 0, 0 0)
SDRV_SetDeviceParameter[326]


Code: [Select]

Some fields from that big structure:

0x00: always points to 0xa0000, passed to CreateResLockEntry
0x04: always 0x1f, passed to CreateResLockEntry
0x08: engio list, enables c0f13000 and configures a few others
0x0c: 0
0x10: engio list, disables c0f13000
0x14: empty engio list?!
0x18: engio list, starts at c0f03218
0x1c: 0
0x20: 0
0x24: 0
0x28: 0
0x2c: 0
0x30: 0
0x34: 0
0x38: ffff
0x3a: image width (16 bit)
0x3c: image height (16 bit)
0x3e: some sort of counter, increases when changing video modes
0x40: 0
0x44: resource lock gets saved there
0x48: a
0x4c: 5
0x50: -1
0x54: 3
0x58: input image buffer address
0x5c: 0

0x80: assigned with 0x20000001
0x88: memory suite where the jpeg data will be written (it's pre-allocated)
fields pointing to ROM addresses seem to be engio structures, they change with video modes
These are a few very interesting things I discovered , in the "MJpeg thread & the ML Google Group
I'm liking the Resource stuff , not sure if it's 5d2 but I'm sure it d4 I think