Author Topic: Canon 40D  (Read 305932 times)

domasa

  • Freshman
  • **
  • Posts: 95
Re: Canon 40D
« Reply #450 on: October 31, 2019, 08:33:10 PM »
Now is visible. Incredibly  8)

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #451 on: November 07, 2019, 04:58:39 PM »
Still working, current status ... 

Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #452 on: November 12, 2019, 12:05:00 PM »
I'm stuck at the moment  :(. My issue in inside LiveView with the HD buffers.

When the camera is started in liveView the HD buffers are correct, and my histogram is correct and it shows the histogram of LiveView.
When I capture a image in LiveView, the HD buffer seems to be locked to the image taken and thus the histogram is locked to the image taken.
So when I return to Live View, the HD buffer is locked to the captured image and does no longer show the histogram of LiveView. When I take
another image the HD buffer gets updated to this new image, and the histogram is changed. The buffer returned to me is not HD buffer with
LiveView content but the image buffer of the last captured image.

I'm going to make screenshot work for 40D (yuv mode) so I can dump correct images and then I guess I need to run the 40D FW in QEMU
with breakpoints in the firmware. There is a function "lvGetResources" that prints out the correct LiveView YUV buffer address in the log, but
ghidra fails to decode which function(s) call lvGetResources. I would like to get that call tree.
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

srsa

  • New to the forum
  • *
  • Posts: 22
Re: Canon 40D
« Reply #453 on: November 12, 2019, 07:56:05 PM »
I would like to get that call tree.
In case qemu can't get that far, you could also try using cache hacks (src/cache_hacks.h) on the camera.

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #454 on: November 12, 2019, 10:10:45 PM »
YES thanks, good idea, I just need to intercept the call using cache hacks and read the LR register, then I can  find the return address of the caller
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #455 on: November 20, 2019, 10:53:24 PM »
I'm having trouble, I hoped I could get a version before xmas, but it looks badly. It's a simple requirement I thought was allready done, but its not : The position of the HD vram buffers are not static.

I havent found the caller of lvGetResources yet, QEMU/gdb won't break, and cache hacks won't do the trick in the real camera either, I'v had troubles in the past with cache hacks on 40D, like I can't use cache hacks on the canon debugmsg function, this does however work in QEMU. My last try is going to use cache hack in QEMU (next week) - This should work.


Scenarios tested:

1. Live Mode

-> Start camera
-> goto LiveMode
-> save all HD buffer 1,2,3.
Result : All three images is Live HD images  :)

2. Live Mode + image capture
-> Start camera
-> goto LiveMode
-> capture image (take photo)
-> re-enter LiveMode
-> save all HD buffer 1,2,3.
Result : All three images is the captured photo - NOT live mode images  :(

3. Photo Mode
-> Start camera
-> Capture image (take photo)
-> Goto Live Mode
-> save all HD buffer 1,2,3.
Result : All three images is noise  :(

.. back to basics ..

Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

srsa

  • New to the forum
  • *
  • Posts: 22
Re: Canon 40D
« Reply #456 on: November 22, 2019, 07:31:48 PM »
I'v had troubles in the past with cache hacks on 40D
I've had similar experience on PowerShots.
I found that the firmware's (and CHDK's) cache manipulation functions counter-act cache hacks. The locked part of cache remained locked, but the content vanished from time to time.
The real solution (IMHO) would be a rewrite of all offending cache manipulation functions, so that they don't touch the locked part. A less reliable way would be re-adding the hacks at the end of each such function.
I chose an even less reliable solution: I re-added the hacks every 10ms, using a task that cycles once every 10ms. It was good enough for doing research.

As far as I know, some ML ports and modules rely on cache hacks - I have no idea how they get away without experiencing similar issues.

One more tip: If I see it correctly, lvGetResource only has data references (no b or bl targets it directly). You could get away just by patching those references using the data cache.

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #457 on: November 27, 2019, 09:26:17 AM »
Srsa .. You are right about cache hacks. The QEMU "Digic" processor works better in some cases, but on LvGetResources I ran out of luck. Ghidra shows no direct reference to lvGetResource, but alot of function is'nt called directly but indirectly just like lvGetResource, but in most cases I am able to find a pointer to the function, being passed as a argument to another function, but again with lvGetResource I ran out of luck. Ghidra is really good, but in all cases is does'nt understand functions that does'nt start with a stmdb, like if there a mov before the stmdb ghidra get confused, and having to manually make these function is waste of time, and I have a feeling that the pointer to the function or direct call is hidden somewhere.

I wanted was to find the memory (struct) that points to the LV buffers, as this is the very last task before making a beta build.

For now in working in a different direction, I'm have dump'ed the 256 MB ram and have disected this into images as green(rgb),grey,luma(YUV422),chroma(YUV422) in 8 different sizes (512,720,864,972,1024,1944,3888), which resulted im over 6000 images I need to review, which was converted into a movie using ffmpeg, which I need to watch :). If the LiveView buffers are there, I should beable to find them.

if this failed I might try your to 10 ms re-adding hack cache method.

Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

srsa

  • New to the forum
  • *
  • Posts: 22
Re: Canon 40D
« Reply #458 on: November 27, 2019, 06:57:55 PM »
Ghidra shows no direct reference to lvGetResource, but alot of function is'nt called directly but indirectly just like lvGetResource, but in most cases I am able to find a pointer to the function, being passed as a argument to another function, but again with lvGetResource I ran out of luck.
Assuming lvGetResource is the function that references the "[LV] lvGetResource 0x%lX (0x%lX)" string, there are 4 data references pointing to it (= the function's address appears 4 times). These references are a few kBytes "above" the function. It looks like Ghidra doesn't automatically notice references like that. I found them by placing the cursor to the function's first instruction and choosing "Search" -> "For direct references" in the CodeBrowser's menu. I then had to turn those bytes into pointers (P button). The references all appear to be part of a huge struct, and that struct is referenced from a function. I'm not familiar with this style of code as PowerShots implement state machines differently.

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #459 on: November 27, 2019, 11:51:13 PM »
Assuming lvGetResource is the function that references the "[LV] lvGetResource 0x%lX (0x%lX)" string, there are 4 data references pointing to it (= the function's address appears 4 times). These references are a few kBytes "above" the function. It looks like Ghidra doesn't automatically notice references like that. I found them by placing the cursor to the function's first instruction and choosing "Search" -> "For direct references" in the CodeBrowser's menu. I then had to turn those bytes into pointers (P button). The references all appear to be part of a huge struct, and that struct is referenced from a function. I'm not familiar with this style of code as PowerShots implement state machines differently.

Oh yes, I see it clearly now. Huge struct, that will take some time to dig into that. I never learned the "Search" -> "For direct references" until now, thank you for pointing that one out.
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #460 on: November 29, 2019, 07:18:22 AM »
I have located LiveView, but they're not at a fixed address.

I dumped the ram and converted into different size and formats 512,720,864,972,1024,1080,1944,3888 versus grey,green (rgb),luma(Y422),chroma(Y422),chroma(YUV411) and finally converted them into a movie with ffmpeg, that way I analyzed around 10000 images. Quite effective. This method took some time but it's really nice. I found multiple images in different sizes all over the place, and I found what I was looking for the LiveView buffer. The images I found (smaller size) was:

1. HD buffers
Numbers    : 32
Size          : 1024 x 680
Address    : Dynamic (changes when a image is taken)
Encoding   : YUV422

2. LiveView buffers
Numbers    : 2
Size       : 720 x 214
Address    : Dynamic (changes when a image is taken)
Encoding   : YUV411

3. Miniature (NOT LV image)
Numbers    : 1
Size       : 180 x 320
Address    : Dynamic (changes when a image is taken)
Encoding   : YUV422

Next step is to locate the memory that references these images - or finding the pointers passed to lvGetResources .. seems to be same issue  ???.
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #461 on: December 09, 2019, 09:09:58 PM »
Finally I made some progress again :). I had to ditch the previous attempt and re-read the disassembled code in order to hunt the correct HD images, this was a pretty good idea. A function called within lvGetResources actually stored the main input parameter (first argument to lvGetResources) at 0x1c804. This pointer is static allocated and points to structure in memory (possibly allocated), at offset 0x94 in this struct is a pointer to one of the current YUV HD images. The image pointer is always correct and is updated after each shot. Now I can shoot images in livemode with correct histogram and vectorscope.

Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #462 on: December 13, 2019, 12:57:05 PM »
I'm working on the bootloader, using the recovery branch.

My main issue is that the old bootloader does not work everytime as expected - it too much a hack, so I need to remake it the right way.

It seems that Call("") on 40D supports : 

"EnableFirmware"
"DisableFirmware"
"EnableBootDisk"
"DisableBootDisk"
"EnableMainFirm"
"DisableMainFirm"

Any one tryed these out on other cameras ? or have clues to which one to use inorder to make autoexec.bin being loaded ?
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #463 on: January 11, 2020, 10:59:25 AM »
Started working again, it's rainly constantly here in denmark, so no time for astrophotography.

Digged into FPS override mode and finally got it working (low light mode only). It turns out there is no shadow memory area on Canon
40D, so a lot of peripheral registers can't be read. The solution was to create virtual (fake) registers for fps register A and B which is
used in fps override. 

Can't do above 30 fps, only below.

Live Mode buffer images : Upper image is standard live mode in "M" mode (30 fps) with the settings that gave me the brightess image. Lower image is 1 fps with
exposure lock (ISO 3200 "H") and approx 1 second imtegration.  Hot pixels are present :)



Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

Dezmond2

  • Just arrived
  • *
  • Posts: 1
Re: Canon 40D
« Reply #464 on: January 14, 2020, 03:02:40 PM »
Where download current version for install and testing ?

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #465 on: January 20, 2020, 11:41:21 AM »
Hi Dezmond2

I'm working on the bootloader, that the last thing needed.

I have a basic working bootloader for 40D (abit similar to 450D and 5D classic bootloaders), but it's not perfect, and I wish to have a bootloader that cannot be activated on the wrong cameras or wrong 40D versions, as this might brick the camera. I have no idea when it done, but my hope is within the next 3-4 weeks.

I will post the complete packages here in this thread.
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • New to the forum
  • *****
  • Posts: 38
Re: Canon 40D
« Reply #466 on: Yesterday at 03:37:27 PM »
Notes on jpeg recording

The jpeg in live mode 1024x680 (no zoom) has maximum size 330.8k. The header is hardcoded (and so are the jpeg-edmac channels), but the actual file can vary in length, and is typically smaller. I can't not find the true lenght of the jpeg unless I search the entire 330.8k block for the jpeg end container, easy solution is just to save the entire 330.8k block.  Inorder to have a efficient recording system, these images must be copied using edmac into larger buffers, which can be saved. Larger buffers increase the write speed to the CF card, small buffers lower the bandwidth. The FIO file system used dma.  The big issue there is not the size - the jpeg is small, but it the fact that there are no "free" edmac avilable on the DIGIC III, (free = not used by firmware). All edmac are used (no wonder that the DIGIC4 have many more of these). The firmware even seems to be using some edmac for multiple operations. So stealing a edmac is gonna be difficult- but I guess not impossible. The jpeg-edmac channels used for encoding and tranfering the jpeg image is used in approx 5ms per frame, so in the remaining time they seems to be unused. At 24 fps I have around 36 ms for tranfering the image to another location, before the jpeg-edmac is used again, this should be possible.

Minimum CF bandwidth needed at 24 fps = 24 * 330.8k  ~ 8 mb/s. 

These bandwidth requirements are also below the capabilities of the CF system, the bandwidth limit is around 14 mb/s and overclocked around 20 mb/s, so this should also be possible, and higher fps should also be possible. I could also "just" try to hijack a simple function which is called just after jpeg tranfering completes, then save the frame directly, but this requires that the CF system is capable of 8 mb/s in small blocks and that the LV system can cope with the additional code overhead. Also, even more interessing, the Live Mode jpeg is 1024x680, but the raw image is larger 1334x872, and in zoom x5 mode, the raw size is close to HD = 1872x1066 pixels.
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d