Author Topic: Canon 80D  (Read 121044 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10704
  • 5D Mark Free
Re: Canon 80D
« Reply #225 on: January 03, 2018, 01:22:37 PM »
Finally :D

Still, the emulation didn't advance much; only got rid of a warning about some missing property. Time to get some more logs from real hardware and move beyond bootloader stage.

Updated the digic6-dumper branch (including these cache changes) and I'm now able to get a ROM dump from main firmware, in QEMU. Will let sombree tweak it further, should it require any more changes.

If that works, the next steps would be:
- getting a full diagnostic log, including MPU messages (mostly for emulation, but also helpful with porting)
- finding the remaining stubs (the "easy but boring" part)
- identifying the display buffer, printing hello world, opening ML menu...

goldenchild9to5

  • Hero Member
  • *****
  • Posts: 506
Re: Canon 80D
« Reply #226 on: January 03, 2018, 02:22:39 PM »
Happy New Year everyone.. @a1ex @sombree awesome job with the 80D. 

sombree

  • New to the forum
  • *
  • Posts: 34
Re: Canon 80D
« Reply #227 on: January 03, 2018, 02:49:31 PM »
I have good news - with latest changes we're able to run code on camera  :D
Full LOG000.LOG from the camera.
Card led blinking: click  :D

Danne

  • Hero Member
  • *****
  • Posts: 3612
Re: Canon 80D
« Reply #228 on: January 03, 2018, 03:17:29 PM »
Wow, great to hear about this.

goldenchild9to5

  • Hero Member
  • *****
  • Posts: 506
Re: Canon 80D
« Reply #229 on: January 03, 2018, 03:37:41 PM »
@sombree Awesome..  :)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10704
  • 5D Mark Free
Re: Canon 80D
« Reply #230 on: January 03, 2018, 07:21:37 PM »
Committed some logging experiments - please try the intermediate changesets (not just the last one) and report their outcome.

sombree

  • New to the forum
  • *
  • Posts: 34
Re: Canon 80D
« Reply #231 on: January 03, 2018, 08:04:03 PM »
Link to logs.
LOG000.LOG - latest commit is d54669e
LOG001.LOG - latest commit is f264082
LOG002.LOG - latest commit is 122cb55
LOG003.LOG - latest commit is f492292

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10704
  • 5D Mark Free
Re: Canon 80D
« Reply #232 on: January 03, 2018, 08:29:59 PM »
Excellent - all of them worked! Committed some more.

sombree

  • New to the forum
  • *
  • Posts: 34
Re: Canon 80D
« Reply #233 on: January 03, 2018, 08:49:18 PM »
Link to logs.
LOG004.LOG - latest commit is 85d4880
LOG005.LOG - latest commit is e439c1d
LOG006.LOG - latest commit is 07f0319

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10704
  • 5D Mark Free
Re: Canon 80D
« Reply #234 on: January 03, 2018, 09:03:05 PM »
There are two memory regions commented out in 85d4880, since they might give trouble; can you try these as well?

If they don't work (crash or whatever), try smaller sizes.

sombree

  • New to the forum
  • *
  • Posts: 34
Re: Canon 80D
« Reply #235 on: January 03, 2018, 09:15:35 PM »
Link to LOG007 + one RAM region. EE00 doesn't crash the camera, but no file is created. I've tried something like dump_file("EE00.BIN", 0xEE000000, 0x01000000) but still no luck.
LOG007 - latest commit is f679e3d.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10704
  • 5D Mark Free
Re: Canon 80D
« Reply #236 on: January 03, 2018, 09:40:18 PM »
f679e3d didn't work quite well, can you disable sensor cleaning and - if possible - the main display (on other cameras it's done with the INFO button) and run it again?

The logs are trimmed if too many messages are present.

I should probably enable the full dm-spy-experiments code, which doesn't have this limitation.

sombree

  • New to the forum
  • *
  • Posts: 34
Re: Canon 80D
« Reply #237 on: January 03, 2018, 09:51:26 PM »
It's still cut - link. I've also tried with touch disabled but it's still not enough to capture whole log.

michael1998

  • Just arrived
  • *
  • Posts: 1
Re: Canon 80D
« Reply #238 on: January 04, 2018, 03:17:07 AM »
so I basically read all the ten pages, and I'm not that familiar with coding and programming so I was not able to understand that much, but it looks like you guy made some progress so my question is do you guys think that it is possible to have it for the canon 80d ?? I'm not rushing into things or anything  you guys are doing your best and i appreciate that  but I'm just curious to know at what stage is it now does it still need a lot of work yet or is it at it final stage and I have the  canon 80d if there any way I can help with just tell me we can run some tests on my camera

core51

  • Just arrived
  • *
  • Posts: 1
Re: Canon 80D
« Reply #239 on: January 04, 2018, 09:18:25 AM »
How can I help am feeling like dumb. I want to help :) but while looking at your posts and comment I don't know what you guys are talking about. Am not a tech guy but currently own 80d and want to help.

ConanTheBarber

  • Just arrived
  • *
  • Posts: 2
Re: Canon 80D
« Reply #240 on: January 11, 2018, 04:10:25 AM »
selfishly, all I need is Focus Peaking for video!
 ;D

I would also REALLY enjoy Aperture Priority and other Priority filming modes. Anyone think this mighthinks possibility?

kallitokaco

  • Just arrived
  • *
  • Posts: 1
Re: Canon 80D
« Reply #241 on: January 14, 2018, 12:55:03 PM »
Hi,

professional c/c++ programmer here with an 80D and lack of time and fundamental understanding of ML. First of all, nice work guys.

I started to look at the MPU log.c and I'm curios why the buffer in mpu_recv_log() is updated by mpu_recv() after the buffer is already decoded? Shouldn't it be updated in advance to log the current msg?
And is there a reason why the argument "size" of mpu_recv_log() is not used, but the [-1]-Hack to get the size, although at [ 0 ] is always the encoded size, or is this not always the case?

Greetings

teq

  • Just arrived
  • *
  • Posts: 1
Re: Canon 80D
« Reply #242 on: January 15, 2018, 08:59:07 PM »
Is there any solution to avoid the 30 minute video limit?  As it stands now, the camera stops recording after 30 minutes.  I need to be able to record forever but I can't find any mention of this feature in the feature notes.

Walter Schulz

  • Hero Member
  • *****
  • Posts: 5874
Re: Canon 80D
« Reply #243 on: January 15, 2018, 09:27:24 PM »
Ask Canon to change firmware. Tariff penalty for cams recording > 29:59 was ditched mid 2016.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10704
  • 5D Mark Free
Re: Canon 80D
« Reply #244 on: January 15, 2018, 09:50:00 PM »
why the buffer in mpu_recv_log() is updated by mpu_recv() after the buffer is already decoded?

Shouldn't it be updated in advance to log the current msg?

In Canon code, this function is called indirectly (a pointer to it - mpu_recv_cbr - is stored to RAM) whenever a message is received from the MPU, so we have replaced that pointer with one to our own function (mpu_recv_log). Our function prints the message in Canon's log file, then calls Canon's function to process the message. So, it keeps the original functionality (whatever Canon code is supposed to do with each message from the MPU) and it logs each call. The protocol is documented here.

Quote
And is there a reason why the argument "size" of mpu_recv_log() is not used, but the [-1]-Hack to get the size, although at [ 0 ] is always the encoded size, or is this not always the case?

From the above link, mpu_recv gets a pointer to payload_size (second char in the message), so it happens that buf[-1] is still available for reading. It's not really necessary to capture it though; payload_size is enough to interpret the messages. The logging code was written before we understood the message format (it was adapted from here), and it was done that way to match the format from mpu_send (which receives a "full" message, with two size arguments).

But you are right, message_size can be dropped without any loss of information (now that we know the the first two parameters are always redundant). It would require updating mpu.c (to add logic for message_size, rather than just replaying it), extract_init_spells.py and any other scripts that parse this kind of logs, though. And, of course, a script for updating any old logs to the new format.

BTW - nice to see others reading the low-level code and trying to understand what's going on.



Forgot to mention - since the code from the digic6-dumper branch is now able to dump the RAM while running Canon firmware, here's an easy coding task: find the image buffer addresses (Canon menus, overlays, LiveView image) in the RAM dump.

I didn't look into it yet, so can't give many hints right now, but I'm pretty sure the image buffers should be somewhere in RAM, and CHDK folks already documented the image format.

OlRivrRat

  • Senior
  • ****
  • Posts: 268
Re: Canon 80D
« Reply #245 on: January 16, 2018, 06:36:41 AM »
"find the image buffer addresses (Canon menus, overlays, LiveView image) in the RAM dump"

           What would what You are asking to be looked for Look Like ?
ORR~DeanB  ~~  80D-ML  &  SL1+ML  &  5D2+ML  &  5DC+ML  &  70D+ML.111B(DEAD)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10704
  • 5D Mark Free
Re: Canon 80D
« Reply #246 on: January 16, 2018, 09:07:00 AM »
Like... what you are seeing on the camera screen.

https://stackoverflow.com/questions/37867570/finding-images-in-ram-dump
https://w00tsec.blogspot.com/2015/02/extracting-raw-pictures-from-memory.html

The google-fu is also part of the exercise; it was meant to be a time saver for me, not I solve this and then teach you what I did. I already do that for the more difficult topics.

ConanTheBarber

  • Just arrived
  • *
  • Posts: 2
Re: Canon 80D
« Reply #247 on: January 17, 2018, 12:14:21 AM »
You guys working on these things. You have my utmost respect. Your work is exceptional! 

I am def tech savy but am not much of a coder. I do own an 80D and would like to help as well.

Is there any way to set this thread as a notification, whenever there is a new message?

OlRivrRat

  • Senior
  • ****
  • Posts: 268
Re: Canon 80D
« Reply #248 on: January 17, 2018, 08:00:19 AM »
      @Alex

What the heck is "The google-fu"

"it was meant to be a time saver for me,"  > Assumed that & was wanting to see if I could help ~

"not I solve this and then teach you what I did."  > Assume that Neither of Us have the time left for that to happen &

Certainly Didn't think I was asking for that ~

Maybe this is 1 of those "If You have to ask then You probably don't have the necessary resources to participate" situations ~

Sorry to be a Bother ~
ORR~DeanB  ~~  80D-ML  &  SL1+ML  &  5D2+ML  &  5DC+ML  &  70D+ML.111B(DEAD)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10704
  • 5D Mark Free
Re: Canon 80D
« Reply #249 on: January 17, 2018, 10:42:59 AM »
Google-fu is the most important skill you need to port ML on any camera.

We are reverse engineering an undocumented system (a newer hardware platform, different from what we already know). Translation: we don't know much about it; we are trying to understand how it works. If I had known how the image buffers look like in the RAM dump, I would already knew where they are. The point is to find something, in the RAM dump, that resembles (to some extent) what you are seeing on the screen (Canon menus). This task is well documented on other websites (for other devices) and can be done with existing GUI software (it doesn't require coding skills).

What I knew about this task was already mentioned:
Quote
I didn't look into it yet, so can't give many hints right now, but I'm pretty sure the image buffers should be somewhere in RAM, and CHDK folks already documented the image format.

To tell more details, I'd have to solve the task myself.

Already wasted too much time on this; will come back when there will be some meaningful progress. Current state allows anyone to print Hello World (and maybe even launch ML menu) within a couple of evenings, by following existing guides.