ML on EOS-M2

[a1ex]

However, the MD5 values from the "original" ROM1 and the new QEMU dump don't match. Is this expected?

"boot=1" works by patching the ROM. If you do the same change in a hex editor, you'll get the second MD5.

[dfort]

Wow--big progress! Did you already try it on your M2?

Not yet. Still in QEMU. Trying to run Hello World but my MacBook doesn't look like an EOSM2:



I found everything in stubs.S but then I started going through consts.h and yikes! Looks like finding stubs was the easy part. I did read the part about firmware signature failing in QEMU so I made a new branch, merged QEMU and EOSM2 but found out that if there is no value for "#define SIG_EOSM2_103" then I get a blank QEMU screen. Is this a Catch-22?

Oh well, at least I've got a good stubs.S file -- uh, I think.

[a1ex]

Time to learn some debugging tricks

Look at recent QEMU updates - one of them is compiling ML with debug information. So far I have only used it for translating code addresses from ML code into source code lines (addr2line). Let's see if we can enable source code debugging in GDB, so we can run the code step by step, as we would with a PC program.

Notes:
- This is the first time I'm trying this trick. I have used source code debugging in IDA + QEMU before, but that was based on decompiled code (and the free version can't do debugging if I'm not mistaken). Here I want to run ML step by step, on the original source code.
- I want the QEMU updates in the mainline, as there are some APIs for debugging ML as well (there's a PR open), which means you can merge "qemu" in your EOSM2 branch. Or, simply experiment there, and once the debugging session is over, move the changes to the "clean" branch (this is what I usually do if I need the dm-spy-experiments branch).

Enough chit-chat, let's try it:

Code: [Select]

# from ML directory
hg update qemu -C
cd platform/EOSM2.103
make clean; make
# mount the SD image
make install

Next, we have to set up a breakpoint in our code (autoexec.bin); we can do that from debugmsg.gdb.

Without debugging symbols, we would do it like this and debug the assembly code (which is not very easy):

Code: [Select]

# breakpoint on autoexec.bin
b *0x800000

To use debugging symbols in gdb, we'll have to use symbol-file (to load the symbols from an elf file). The .o files that appear when compiling ML are elf files. Also, the files without extension, "autoexec" and "magiclantern", are elf files. The first one refers to reboot.c (look it up in Makefile.src) and gets loaded where Canon's bootloader loads autoexec.bin (at AUTOEXEC_BASE = 0x800000), while the second refers to the regular ML code (starting in boot-hack.c), that gets loaded into its final location (RESTARTSTART).

Why all this trouble? Because our binary can't stay at 0x800000. How comes? Well, autoexec.bin (which is a file loaded in Canon code, probably for debugging or development purposes) wasn't meant (read: designed by Canon) to be a program to run alongside Canon's main firmware; therefore DryOS is free to use the addresses near 0x800000 (where we are loaded). So, we have to move somewhere else. More on that later.

Back to our debugging session: the issue is in reboot.c (that's where this message is printed from), so we are going to run "autoexec.bin" as a plain binary (machine code), and use "autoexec" (elf) for the debugging symbols. Add this to debugmsg.gdb:

Code: [Select]

# breakpoint on autoexec.bin (cstart)
symbol-file ../magic-lantern/platform/EOSM2.103/autoexec
b cstart

You can keep both breakpoints, as there are some other things that execute before cstart (such as checking the integrity of the binary file, to avoid executing random code when loading from a corrupted filesystem). Without further ado, let's dive in:

Code: [Select]

# from QEMU directory (assuming default paths and bash)
. ./export_ml_syms.sh EOSM2.103
./run_canon_fw.sh EOSM2,firmware="boot=1" -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb

We'll hit the first breakpoint first:

Code: [Select]

Now jump to AUTOEXEC.BIN!!
...
Breakpoint 3, 0x00800000 in _start ()
=> 0x00800000 <_start+0>: 0c 40 8f e2 add r4, pc, #12

Type "continue" ("c") to reach cstart:

Code: [Select]

Breakpoint 4, cstart () at ../../src/reboot.c:215
215     int s = compute_signature((int*)SIG_START, SIG_LEN);

Type "layout src". Now you are debugging on the original source code.

Tip: you can set breakpoints like this:

Code: [Select]

b reboot.c:216
Refer to GDB manual for the stepping commands, or use a GDB GUI.

Let's try nemiver:

Code: [Select]

. ./export_ml_syms.sh EOSM2.103
./run_canon_fw.sh EOSM2,firmware="boot=1" -s -S & nemiver --gdb-binary=$(which arm-none-eabi-gdb) --remote="localhost:1234" ../magic-lantern/platform/EOSM2.103/autoexec



For some reason, I'm unable to step. Too bad, as it seems to be working for these guys. Maybe someone else can figure it out?

Let's try gdbgui:

Code: [Select]

. ./export_ml_syms.sh EOSM2.103
./run_canon_fw.sh EOSM2,firmware="boot=1" -s -S & gdbgui -g "arm-none-eabi-gdb"

Make sure you view its GUI with Google Chrome and enter this at the gdb prompt:

Code: [Select]

source EOSM2/debugmsg.gdb



This one appears to work a little better, you can run the program step by step using the GUI, but still gives a bunch of errors. Go figure.

cgdb is a terminal-based GUI, but doesn't like QEMU printing debug messages on the same window. Fortunately, nkls provided a script to split the terminal window in two - splitgdb.sh. Replace its contents with something like this:

Code: [Select]

#!/usr/bin/env bash

. ./export_ml_syms.sh $1.$2
tmux new-session -d "./run_canon_fw.sh $1,firmware='boot=1' -s -S"
tmux split-window -h "cgdb -d arm-none-eabi-gdb -x $1/debugmsg.gdb"
tmux attach-session -d


Code: [Select]

./splitgdb.sh EOSM2 103




The user interface seems to be very rough though, and it doesn't like our colored debug messages either...

Let's try the "classic" ddd:

Code: [Select]

. ./export_ml_syms.sh EOSM2.103
./run_canon_fw.sh EOSM2,firmware="boot=1" -s -S & ddd --debugger "arm-none-eabi-gdb -x EOSM2/debugmsg.gdb"

This works. The interface is not pretty, but at least it works.

(I've ran EOSM ML on EOSM2 ROM, in case you are wondering)



Other suggestions?

[dfort]

Code: [Select]

# from ML directory
hg update qemu -C
cd platform/EOSM2.103


Oh I wish. For now I'm having to merge EOSM2.103_wip, qemu-EOSM2-wip_1, qemu-build-tweaks-2 and of course qemu into a qemu-wip branch in order to get all of the pieces working with EOSM2 on a Mac. I'm keeping these separate branches to keep a couple of my pull requests in order. The build tweaks for the Mac are working nicely as is the EOSM2 QEMU setup. It will be a happy day for me when these get merged into the qemu branch. 

Code: [Select]

Now jump to AUTOEXEC.BIN!!
...
Breakpoint 3, 0x00800000 in _start ()
=> 0x00800000 <_start+0>: 0c 40 8f e2 add r4, pc, #12

Type "continue" ("c") to reach cstart:

Code: [Select]

Breakpoint 4, cstart () at ../../src/reboot.c:215
215     int s = compute_signature((int*)SIG_START, SIG_LEN);


I'm seeing something slightly different probably because I'm working on a different autoexec.bin.

Code: [Select]

Breakpoint 1 at 0x4398
Breakpoint 2 at 0x7360
Breakpoint 3 at 0xff0c5144
Breakpoint 4 at 0x800000
Breakpoint 5 at 0x8646ec: file ../../src/fw-signature.h, line 47.
(gdb) c
Continuing.
...
Now jump to AUTOEXEC.BIN!!
0010E5F4: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010E5F4: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D

Breakpoint 4, 0x00800000 in _start ()
(gdb) c
Continuing.

Breakpoint 5, cstart () at ../../src/reboot.c:215
215     int s = compute_signature((int*)SIG_START, SIG_LEN);
(gdb)


Type "layout src". Now you are debugging on the original source code.

Nice! Got the breakpoints tip working too.

Let's try nemiver:

Spent too many hours trying to get nemiver working on Mac (building from the source code) but never succeeded.

Moving on to gdbgui. It this one is coded in python and is easy to install on Mac:

Code: [Select]

pip install gdbgui --upgrade
Yeah, easy but it took me a while because the instructions on the project page didn't work for me. Maybe because I'm using the Homebrew python that doesn't need root access to install packages? So if you're on a Mac and followed my ML tutorial, don't do this:

Code: [Select]

sudo pip install gdbgui --upgrade --user
Now we're on the same page--sort of.



It is getting late so I'll skip to ddd:

Code: [Select]

brew tap homebrew/core
brew install ddd
And...

Code: [Select]

Setting BOOTDISK flag to FFFFFFFF
Error: Unresolved inheritance operation

Xt error (Unresolved inheritance operation).

Oops!  You have found a bug in DDD.

If you can reproduce this bug, please send a bug report
to <ddd@gnu.org>, giving a subject like

    DDD 3.3.12 (i386-apple-darwin16.3.0) gets Xt error

To enable us to fix the bug, you should include the following information:
* What you were doing to get this message.  Report all the facts.
* The contents of the `~/.ddd/log' file as generated by this session.
Please read also the section "Reporting Bugs" in the DDD manual.

We thank you for your support.

ddd: Cannot save options
[1]   Done                    ./run_canon_fw.sh EOSM2,firmware="boot=1" -s -S

Ugh!

I'll try cgdb tomorrow.

[dfort]

Ok--trying out cgdb. The splitgdb.sh script uses tmux which I already had on my system, don't when I installed it but it is also available from brew as is cgdb:
 

Code: [Select]

brew install cgdb
Wow, this is one is impressive when run in full screen mode.



A couple of questions:

• Why isn't this version of splitgdb.sh by nkls in the qemu branch? I tried the current script and it skipped most of the breaks we put in debugmsg.gdb, missed the jump to autoexec.bin and launched the Canon GUI.
• Is the firmware signature it displayed to be trusted?

Code: [Select]

│Breakpoint 2 at 0x7360     
│Breakpoint 3 at 0xff0c5144 
│Breakpoint 4 at 0x800000   
│Breakpoint 5 at 0x8646ec: file ../../src/fw-signature.h, line 47. 
│(gdb) c                     
│Continuing.                 
│
│Breakpoint 4, 0x00800000 in _start ()                   
│(gdb) c                     
│Continuing.                 
│
│Breakpoint 5, cstart () at ../../src/reboot.c:215       
│(gdb) print s               
│$1 = 0xceeeeeec             
│(gdb) print _signature     
│$2 = 0x2d7c6dcf             
│(gdb)   

[EDIT] Never mind, it is showing the EOSM_202 signature I'm using as a temp. Still find myself in a Catch 22 and can't get "hello world" working much less find the firmware signature. That has to be done in camera, right?

Code: [Select]

│Breakpoint 5, cstart () at ../../src/reboot.c:211         
│(gdb) print s               
│No symbol "s" in current context.                         
│(gdb) print _signature       
│No symbol "_signature" in current context.                 
│(gdb)
Continuing from there:

Code: [Select]

DRYOS PANIC: Module Code = 1, Panic Code = 2
Catch 22 -- Need to run "hello world" to get the firmware signature but QEMU doesn't run "hello world" because it can't get the firmware signature. If we could get the firmware signature maybe "hello world" will run on QEMU but then, what's the point?

Is it too early to turn on the camera bootflag? (Not that I have a clue how to do that.)

[a1ex]

Why isn't this version of splitgdb.sh by nkls in the qemu branch?

The version from nkls is there. What I wrote above was something I came up with at the time of writing; I didn't use this approach before.

Would be nice to improve it to accept any number of command-line options (and pass all of them to QEMU).

Is the firmware signature it displayed to be trusted?

Yes, but only after compute_signature() returns. Not before.

The value from my screenshots should be valid, although I have not set up an EOSM2 platform directory yet (so it's untested).

Oops!  You have found a bug in DDD.

That's too bad - a very powerful piece of software apparently unmaintained for about 8 years...

DRYOS PANIC: Module Code = 1, Panic Code = 2

That's a good sign - this message can only appear from the main firmware, so we are no longer in bootloader context. Still, probably something went wrong when patching the startup process.

Time to debug what happens between loading autoexec.bin and this message. Some useful tools:

Code: [Select]

-d calls or callstack
-d ramw (RAM writes)
-d autoexec (to avoid verbose messages from the bootloader)

For example, -d autoexec,ramw,calls,io,v should give some very good insights into how the startup code is patched, and why it doesn't work. The log will be huge though, and it will miss the call to copy_and_restart from boot-hack.c (because it's not actually a call, but a direct jump), but will catch the next call (from copy_and_restart to reloc_entry), which looks like this (500D, since that's the binary I happened to have on the virtual card):

Code: [Select]

       (big bunch of zeroing out some RAM - zero_bss)
       (big bunch of copying from ROM to RAM - blob_memcpy)
       [ram] at 0x0004D4D4:000B8818 [0x000BB300] <- 0xE12FFF1E: was 0xEBFFF769;
       [ram] at 0x0004D4DC:000B8818 [0x000B9154] <- 0xCB400   : was 0x4C5C4;
       [ram] at 0x0004D4E4:000B8818 [0x000B8704] <- 0x7E400   : was 0x0;
       [ram] at 0x0004D4FC:000B8818 [0x000B90BC] <- 0xEBCCCAFD: was 0xEB0F6D03;
       [ram] at 0x0004D514:000B8818 [0x000B9144] <- 0xEBBD78E5: was 0xEB001AEB;
       [ram] at 0x0004D51C:000B8818 [0x000B9160] <- 0x4D010   : was 0xFF011DBC;
       call 0xB8824(0, c00003e0, 400, b8824)                                     at [4d560:86ba30] (copy_and_restart)
        [FlashIF] at 0x000B882C:0004D564 [0xC0000010] <- 0xD9C5D9C5: 'Write enable' enabled
        [*unk*] at 0x000B8838:0004D564 [0xC020010C] <- 0x1       : ???
        ...
        (more startup activity follows)

Right before this call, you'll see the patching process (those "hijack" macros - where they go). I can use this log to check if that went well.

Tip: in the qemu branch, in boot-hack.c (but not in minimal.c), there are some startup messages (qprint*). These are useful - to see them on the QEMU console, compile with CONFIG_QEMU=y from the regular target (not minimal). Binaries compiled in this way will not run on the camera. More about this on the 1300D thread.

[dfort]

The version from nkls is there. What I wrote above was something I came up with at the time of writing; I didn't use this approach before.

Would be nice to improve it to accept any number of command-line options (and pass all of them to QEMU).

Yes, but your latest version of splitgdb.sh seems to be working more better. 

...I have not set up an EOSM2 platform directory yet (so it's untested).

I have. It obviously isn't finished but it compiles.

https://bitbucket.org/daniel_fort/magic-lantern/branch/EOSM2.103_wip

[dfort]

I'm still inching my way through consts.h and hit another speed bump. Got to YUV422_LV_BUFFER_1,2,3 and looked up the link to the wiki that's in the source code:

// http://magiclantern.wikia.com/wiki/VRAM_ADDR_from_code
// stateobj_disp[1]

Not sure what stateobj_disp[1] referrers to but determined that the wiki page wants me to run this code on the EOSM2.103:

Code: [Select]

int i;
unsigned int *bmp_ram_addr = bmp_vram_info;
for (i=0; i<2; i++)
  DebugMsg( DM_MAGIC, 3, "bmp_vram[]=0x%08x, 0x%08x, 0x%08x",
    bmp_ram_addr[3*i+0],  bmp_ram_addr[3*i+1], bmp_ram_addr[3*i+2] );
unsigned int *vram_info_addr = vram_info;
for (i=0; i<3; i++)
  DebugMsg( DM_MAGIC, 3, "vram_info[]=0x%08x, w=%4d, h=%4d, p=%4d, n=%4d",
      vram_info_addr[5*i+0],  vram_info_addr[5*i+1],
      vram_info_addr[5*i+2], vram_info_addr[5*i+3], vram_info_addr[5*i+4] );
unsigned int *stateobj_disp = 0x90494+0x128; // see ff137acc SetBitmapVramAddress
DebugMsg( DM_MAGIC, 3, "stateobj_disp+0xb0[]=0x%08x,0x%08x,0x%08x,",
  stateobj_disp[0], stateobj_disp[1], stateobj_disp[2]);

I was able to figure out the changes to the code but not sure where to run it--probably in "Don't click me?" (run_test in src/debug.c) However, that seems to be something that needs to be run in the camera, or can it be done in QEMU? Also not having much luck getting the firmware signature in QEMU, seems like I'm at the point of being able to run "Hello world" on the camera but can't seem to be able to get it working in QEMU. (I also tend to use "seem" way too much.)

The unexpected  benefactor of this port is the original EOSM as I keep finding blips in the code for that platform.

[a1ex]

60D in QEMU (from don't click me):

Code: [Select]

. ./export_ml_syms.sh 60D.111 # needed for DebugMsg
./run_canon_fw.sh 60D,firmware="boot=1" -d debugmsg
...
[    run_test:1fe0bc9c ] (32:03) bmp_vram[]=0xc0f140d0, 0x00000000, 0x03f87100
[    run_test:1fe0bcc0 ] (32:03) bmp_vram[]=0xc0f140d4, 0x00000000, 0x03f87100
[    run_test:1fe0bce8 ] (32:03) stateobj_disp+0xbc[]=0x43f80008,0x5c307800,0x40a8e470,

From real camera (the easiest way was to use printf):

Code: [Select]

# 60D 1.1.1, 0x2458+0xBC
bmp_vram[]=0xc0f140d0, 0x00000000, 0x03f87100
bmp_vram[]=0xc0f140d4, 0x00000000, 0x03f87100
stateobj_disp+0xbc[]=0x43f80008,0x41b07800,0x48a902c4

In both cases, stateobj_disp[1] shows a valid address for the image buffer. It's not always the same; on this model there are 3 possible addresses; they are not fixed (as we thought in early days) but allocated dynamically. In many cases (but not always), this dynamic allocation gives the same results every time, so hardcoding the image buffer addresses usually works. This area really needs some cleanup (wip here).

However, D5 models don't seem to work that well in QEMU:

Code: [Select]

# 5D3 1.1.3, 0x246A4+0x118, regular photo mode
[    run_test:000753ec ] (32:03) bmp_vram[]=0xc0f140d0, 0x00000000, 0x00dc3100
[    run_test:00075410 ] (32:03) bmp_vram[]=0xc0f140d4, 0x00000000, 0x00dc3100
[    run_test:00075438 ] (32:03) stateobj_disp+0x118[]=0x40dbc008,0x00000000,0x40881520,

# 5D3 1.1.3, 0x246A4+0x118, playback mode (not exactly working well in QEMU)
[    run_test:000753ec ] (32:03) bmp_vram[]=0xc0f140d0, 0x00000000, 0x00d43100
[    run_test:00075410 ] (32:03) bmp_vram[]=0xc0f140d4, 0x00000000, 0x00d43100
[    run_test:00075438 ] (32:03) stateobj_disp+0x118[]=0x00000000,0x40881414,0x00000000,

# 700D 1.1.4, 0x23C20+0x118 (in QEMU it starts in movie mode and asks for a lens)
[     ml_init:0007f1d0 ] (32:03) bmp_vram[]=0xc0f140d0, 0x00000000, 0x01307100
[     ml_init:0007f1f4 ] (32:03) bmp_vram[]=0xc0f140d4, 0x00000000, 0x01307100
[     ml_init:0007f21c ] (32:03) stateobj_disp+0x118[]=0x41300008,0x00000000,0x4092e400,

# 700D 1.1.4, 0x23C20+0x118, playback mode
[     ml_init:0007f1ec ] (32:03) bmp_vram[]=0xc0f140d0, 0x00000000, 0x01387100
[     ml_init:0007f210 ] (32:03) bmp_vram[]=0xc0f140d4, 0x00000000, 0x01387100
[     ml_init:0007f238 ] (32:03) stateobj_disp+0x118[]=0x41380008,0x00000000,0x4092e3a8

# EOSM 2.0.2, 0x3E650+0x114 (no GUI in QEMU)
[     ml_init:0009e70c ] (32:03) bmp_vram[]=0xc0f140d0, 0x00000001, 0x01387100
[     ml_init:0009e730 ] (32:03) bmp_vram[]=0xc0f140d4, 0x00000001, 0x01387100
[     ml_init:0009e758 ] (32:03) stateobj_disp+0x114[]=0x41380008,0x00000000,0x00000000,

In 5D3 1.1.3 playback, there is some nonzero value, but I don't recognize it as a display buffer address (it might be correct), while on 700D and EOSM I've only got 0 in QEMU. Probably that's the current state of the emulation.

In any case, this test does not reveal any new addresses, so you don't really need to run it; matching the stub with other models should be enough.

[dfort]

In any case, this test does not reveal any new addresses, so you don't really need to run it; matching the stub with other models should be enough.

This is proving to be a devilishly difficult stub to ferret out of the disassembly. I found only one match in one of my EOSM.202 disassemblies:

Code: [Select]

"[LVEVF] evVdInterrupt LastRamClear Start!![4f1d7800]":
That matches the first of the three buffers I'm searching for:

Code: [Select]

#define YUV422_LV_BUFFER_1 0x4F1D7800
#define YUV422_LV_BUFFER_2 0x4F5E7800
#define YUV422_LV_BUFFER_3 0x4F9F7800

Though it is sort of a dead end because all the other models, including the EOSM2 shows this:

Code: [Select]

"evVdInterrupt LastRamClear Start!![%lx]":
There's a lot of "0x40000000" all over the disassemblies and maybe that's a clue?

I'm probably on the wrong path but I searched through the forum and wiki and came up empty handed. Also don't know if I should be pressing forward finding stubs or if I should try to get "Hello world" working in QEMU in order to find the firmware signature.

In any case, I'll keep searching.

[a1ex]

Couldn't find it from LastRamClear Start, but this does apply to both M and M2.

[dfort]

I can see the hint near ImageEffect that shows the values for YUV422_LV_BUFFER_DISPLAY_ADDR but I'm still clueless where to find YUV422_LV_BUFFER_1,2,3.

I'll keep going and get back to this later, maybe it will become obvious the second time around.

[a1ex]

Ah, right. I've found one in QEMU - 0x4f3d7800. You could use that value for all of them, as a workaround.

I've found it by pressing P to go to play mode, then printing the contents of 0x90494+0x12C in gdb. I was lucky, as this didn't work on the other D5 models I've tried before (double-checked today, still no luck there).

I've deleted the YUV422_(LV|HD)* constants locally, but there are some bits of code that had to be updated (on the yuv_buffers branch mentioned before, merged with new-lv-buffer-detection - which attempts to autodetect these addresses at runtime, rather than using hardcoded values). Unfortunately, the current state is "compiles, but doesn't work" (after more than half a day of fiddling with it), and I doubt I'll be able to fix it today.

[dfort]

Progressing slowly.

Some of the stubs and constants have hints that include which two numbers to add together to get the value we're looking for. This helps when stub hunting by pattern matching.

For example, in consts.h I think I may have found these for the EOSM2.103:

Code: [Select]

#define CURRENT_GUI_MODE (*(int*)0x928BC) // in SetGUIRequestMode
#define ISO_ADJUSTMENT_ACTIVE ((*(int*)0x96388) == 0xF) // dec ptpNotifyOlcInfoChanged and look for: if arg1 == 1: MEM(0x79B8) = *(arg2)

But it is easier to check against the disassembly if we write them out like this:

Code: [Select]

#define CURRENT_GUI_MODE (*(int*)(0x92860+0x5C)) // in SetGUIRequestMode
#define ISO_ADJUSTMENT_ACTIVE ((*(int*)(0x96338+0x50)) == 0xF) // dec ptpNotifyOlcInfoChanged and look for: if arg1 == 1: MEM(0x79B8) = *(arg2)

I'm not sure if ISO_ADJUSTMENT_ACTIVE is valid on the EOSM2 because it is missing on the EOSM. From what I can figure out the EOSM should be this:

Code: [Select]

#define ISO_ADJUSTMENT_ACTIVE ((*(int*)(0x4BE28+0x44)) == 0xF) // dec ptpNotifyOlcInfoChanged and look for: if arg1 == 1: MEM(0x79B8) = *(arg2)

On the 100D it is commented out:

Code: [Select]

// #define ISO_ADJUSTMENT_ACTIVE ((*(int*)(0x6B930)) == 0xF)       // dec ptpNotifyOlcInfoChanged and look for: if arg1 == 1: MEM(0x79B8) = *(arg2)
but maybe the problem is because it is the wrong value? Shouldn't it be 0x6B860 instead of 0x6B930? Easier to check the disassembly if it is written like this:

Code: [Select]

#define ISO_ADJUSTMENT_ACTIVE ((*(int*)(0x6B810+0x50)) == 0xF)       // dec ptpNotifyOlcInfoChanged and look for: if arg1 == 1: MEM(0x79B8) = *(arg2)
I'm not sure how to "look for: if arg1 == 1: MEM(0x79B8) = *(arg2)" or if I'm even on the right track here but this does fit into a pattern using the values from the 5D3.113 and 700D.114.

[nikfreak]

For firmware 1.01 on 100D it's like you found out:

Code: [Select]

#define ISO_ADJUSTMENT_ACTIVE ((*(int*)(0x6B810+0x50)) == 0xF)


I am unsure myself why it's commented out but the "wrong" value was correct one of the older A/B/C revision on 1.00 firmware. I was mainly doing copy&paste for the revisions as well while upgrading support for fw 1.01. Anyways thx for pointing it out as it has tbd for 70D too (942C0+50).

// dec ptpNotifyOlcInfoChanged and look for: if arg1 == 1: MEM(0x79B8) = *(arg2)

that's also copy&paste from one of the older supported cams.

[dfort]

I don't want to come off like a little kid asking "why" all the time but as I'm working my way through consts.h some things pique my curiosity.

MOV_RES_AND_FPS_COMBINATIONS

Found this constant used in bitrate.c:

Code: [Select]

    for (i = 0; i < MOV_RES_AND_FPS_COMBINATIONS; i++) // 7 combinations of resolution / fps

Makes sense because these are all the possible settings I could come up with:

1-	1920	24	NTSC/PAL
2-	1920	25	PAL
3-	1920	30	NTSC
4-	1280	50	PAL
5-	1280	60	NTSC
5-	640	25	PAL
7-	640	30	NTSC

However, this is what is in consts.h:

5D3.113

Code: [Select]

#define MOV_RES_AND_FPS_COMBINATIONS 5 // 3 fullhd, 2 hd, not changing the two VGA modes; worth trying with 9

100D.101

Code: [Select]

#define MOV_RES_AND_FPS_COMBINATIONS 4 // 2FHD, 1HD, 1VGA

700D.114, EOSM.202

Code: [Select]

#define MOV_RES_AND_FPS_COMBINATIONS 9
Ok--so on the 5D3 we aren't messing around with the VGA modes but the 100D should have the same combinations as the other cameras and how do you come up with 9 combinations? 1920/24 is available in both NTSC and PAL so that might give you 8 but where's the final combination?

HALFSHUTTER_PRESSED

The hint is to look for string "[MC] permit LV instant", it's the struct referenced in this function. I can find "[MC] permit LV instant" in the disassemblies of all the cameras except the EOSM2. There is a function named "LvPermit(DelivLvLock)" that might be what replaced it but I can't be sure. How to figure this out? Is it possible to do a half-shutter press in QEMU?

[EDIT] Off topic but on the 700D:

Code: [Select]

#define AE_STATE (*(int8_t*)(0x366B8 + 0x1C))
#define AE_VALUE (*(int8_t*)(0x366B8 + 0x1D))
Shouldn't it be?

Code: [Select]

#define AE_STATE (*(int8_t*)(0x366B4 + 0x1C))
#define AE_VALUE (*(int8_t*)(0x366B4 + 0x1D))
I've got a 700D but not sure how to check this.

[nikfreak]

100D is crippled by Canon. It really only has 4 movie modes to select in Canon menu and this camera even profits from the bitrate modifications as it has no ALL-I option by default  700D should have same amount of movie modes as 650D.

Take this for 700D.114:

Code: [Select]

#define AE_STATE (*(int8_t*)(0x367B4 + 0x1C))
#define AE_VALUE (*(int8_t*)(0x367B4 + 0x1D))
...


#define HALFSHUTTER_PRESSED (*(int*)0x24884) is ok [0x2486C+0x18].

[a1ex]

MVR: see platform/include/mvr.h - some of them are commented. Basically, look up these functions in the firmware and map (or verify) the offsets in the mvr_config structure.

On recent models, there are some gaps (2 positions) between 720p and VGA modes. Not sure what's up with them and whether they are used or not (I'll find out in QEMU, if I'll ever manage to emulate LiveView and video recording - light years from current state). I tend to say we should update all of them to 5, to be on the safe side (so bitrate adjustment won't overwrite memory locations we are unsure about), even if that will leave the VGA modes uncovered.

That 4 on 100D is probably a mistake (thanks for pointing out, as I didn't notice it when looking at the PR). I bet bitrate control doesn't work in 50p mode, but works on 60 (or maybe the other way).

[MC] permit LV instant - all D4 and D5 models have it.

Half-shutter support in QEMU is very incomplete.

The AE_VALUEs are for the autoexpo module. Not sure how to use it, couldn't figure it out...

[JohanJ]

100D is crippled by Canon. It really only has 4 movie modes to select in Canon menu

In fact all 7 modes are available also from canon menu, just that only the ones fitting to the chosen video mode (PAL/NTSC) are shown.

[EDIT] On 60D there are 2 additional VGA modes Crop 640x480 NTSC and Crop 640x480 PAL. So here you get in  total 9. Just an idea (even though it is another cam generation)

[dfort]

On 60D there are 2 additional VGA modes Crop 640x480 NTSC and Crop 640x480 PAL. So here you get in  total 9. Just an idea (even though it is another cam generation)

Right, and with FEATURE_CROP_MODE_HACK it adds crop modes to 1920/24 and 25 so that does get 9 possible MOV_RES_AND_FPS_COMBINATIONS. However, it looks like maybe it should be changed to 5 for the reasons that a1ex noted. I'll leave it at 9 on the EOSM2 for now and follow along with any changes to the other cameras.

...
Take this for 700D.114:
...
#define HALFSHUTTER_PRESSED (*(int*)0x24884) is ok [0x2486C+0x18].

That's a clue that got me closer.

5D3	[0x251C4+0x10]
100D	[0x6674C+0x18]
700D	[0x2486C+0x18]
EOSM	[0x3F204+0x20]
EOSM2	[0x910A8+0x??]

[MC] permit LV instant - all D4 and D5 models have it.

Pretty sure that on the EOSM2 disassembly it shows up as - judge_permit_lv.

Take this for 700D.114:

Code: [Select]

#define AE_STATE (*(int8_t*)(0x367B4 + 0x1C))
#define AE_VALUE (*(int8_t*)(0x367B4 + 0x1D))
...



That's right--my bad.

The AE_VALUEs are for the autoexpo module. Not sure how to use it, couldn't figure it out...

Hokey Smoke!

autoexpo.mo on 700D


Don't want to get into this right now but maybe this is something that should be looked into:
Issue #2643 Cannot get module autoexpo to work on EOS 700d

[EDIT] Issue with autoexpo.mo on 700D mentioned here:

Hello,

I have download this module for the 700D and it doesn't work.

Then I will instal cygwin and check out the code source and after read this post, see by debug that the AE_STATE is never set (always 0).
So I change the consts.h file for the 700D platform in replacement of the constants AE_STATE ans VALUE as written somewhere in a topic page after having check in hexa code. It will be then the following :

#define EXPO_COMP (*(int16_t*)(0x367B4+0x1C))
#define AE_VALUE (EXPO_COMP-1) * 8 / 2048
#define AE_STATE (*(int8_t*)(0x367B4+0x1C))

It works (I check by printing the values of ae, iso, and so on).

I can check in if you want.

Just a question, what it is the use of AE_STATE ?

Xav

[Licaon_Kter]

But you can't use autoexpo.mo anyway on M/M2 because you don't have (one) more hardware buttons, right?

[dfort]

Which buttons are missing?

autoexpo.mo appears to be working on the EOSM though I have no idea how to work it.

There are lots of reports of it not working on the 700D and it looks like there is a solution to that.

[a1ex]

Pretty sure that on the EOSM2 disassembly it shows up as - judge_permit_lv.

Right - it does the same thing.

Pressing half-shutter in QEMU gives plenty of info regarding this event (enough to identify this address, but not enough to change things on the user interface or pass the API tests):

Code: [Select]

Key event: 2a -> 0e0e002d
[MPU] Sending : 06 05 06 26 01 00
[    PowerMgr:ff1470d4 ] (00:01) <0 0 0 0 0 0 7> [PM] DisablePowerSave (Counter = 2)
[    PowerMgr:ff147144 ] (00:01) <0 0 0 0 0 0 7> [PM] EnablePowerSave (Counter = 1)
[    MainCtrl:ff0d01c0 ] (9c:01) <0 0 0 0 0 0 7> ID:26(30)
[    MainCtrl:ff0cd8ac ] (89:03) <0 0 0 0 0 0 7> bindReceiveSwitch (38, 1)
[    MainCtrl:ff0cdd88 ] (89:03) <0 0 0 0 0 0 7> COM_SW_SOMETHING
[    MainCtrl:ff0d8ba0 ] (85:03) <0 0 0 0 0 0 7> GUI_Control:84 0x0
[ GuiMainTask:ff0d8f54 ] (84:01) <0 0 0 0 0 0 7> GUI_CONTROL:84
[MPU] Sending : 06 04 05 00 00 00
[ GuiMainTask:ff1470d4 ] (00:01) <0 0 0 0 0 0 7> [PM] DisablePowerSave (Counter = 2)
[ GuiMainTask:ff147144 ] (00:01) <0 0 0 0 0 0 7> [PM] EnablePowerSave (Counter = 1)
[ GuiMainTask:ff1c0728 ] (84:03) <0 0 0 0 0 0 7> GUICMD_PRESS_BUTTON_SOMETHING(0x0)
[    EventMgr:ff224a04 ] (8d:01) <0 0 0 0 0 0 7> emDeliverMulticastEvent : SW1ON
[    MainCtrl:ff0d01c0 ] (9c:01) <0 0 0 0 0 0 7> ID:0(31)
[    MainCtrl:ff154660 ] (9c:03) <0 0 0 0 0 0 7> MeteringStart
[    MainCtrl:ff3da3b0 ] (c0:03) <1 0 0 0 0 0 7> MainCtrl GuiCancelMoviePlay(4151)
[    MainCtrl:ff3e3814 ] (c0:02) <1 0 0 0 0 0 7> [G_P]PC:0xff3e3804 LR:0xff154718 SP:0x197f38
[    MainCtrl:ff0dffe4 ] (2c:03) <1 0 0 0 0 0 7> MVP_CancelMoviePlay
[    MainCtrl:ff0d8ba0 ] (85:03) <1 0 0 0 0 0 7> GUI_Control:80 0x0
[    MainCtrl:ff1549a4 ] (89:03) <1 0 0 0 0 0 7> EVENTID_METERING_START
[      RscMgr:ff0e87dc ] (80:02) <1 0 0 0 0 0 7> srmEventDispatch: Current = 0, dwEventID = 12, dwParam = -442503148 S
[      RscMgr:ff0e89e4 ] (80:02) <1 0 0 0 0 0 7> srmEventDispatch: Current = 0, dwEventID = 12, dwParam = -442503148 E
[ GuiMainTask:ff1c1880 ] (84:01) <1 0 0 0 0 0 7> gui control end
[ GuiMainTask:ff1c18a0 ] (84:01) <1 0 0 0 0 0 7> 0msec = 47810 - 47810
[ GuiMainTask:ff1c18bc ] (84:01) <1 0 0 0 0 0 7> 3584msec = 776448 - 780032
[ GuiMainTask:ff0d8f54 ] (84:01) <1 0 0 0 0 0 7> GUI_CONTROL:80
[ GuiMainTask:ff1c1880 ] (84:01) <1 0 0 0 0 0 7> gui control end
[ GuiMainTask:ff1c18a0 ] (84:01) <1 0 0 0 0 0 7> 0msec = 47810 - 47810
[ GuiMainTask:ff1c18bc ] (84:01) <1 0 0 0 0 0 7> 0msec = 780544 - 780544
[     CtrlSrv:ff3ab264 ] (83:03) <1 0 0 0 0 0 7> IDLEHandler PRESS_SW1_BUTTON
[    Fstorage:ff1c8fac ] (9e:03) <1 0 0 0 0 0 7> fssSW1On

[         CEC:ff347750 ] (c2:01) <1 0 0 0 0 0 7> cecCompleteStart Start
[         CEC:ff357118 ] (00:26) <1 0 0 0 0 0 7> [I2C] localI2C_Read : 378 (Task : CEC)
   980: 47851.520 WARN [I2C] localI2C_Read : 378 (Task : CEC)
[         CEC:ff347750 ] (c2:01) <1 0 0 0 0 0 7> cecCompleteStart Start
[         CEC:ff357118 ] (00:26) <1 0 0 0 0 0 7> [I2C] localI2C_Read : 378 (Task : CEC)
   981: 47868.416 WARN [I2C] localI2C_Read : 378 (Task : CEC)
[         CEC:ff347750 ] (c2:01) <1 0 0 0 0 0 7> cecCompleteStart Start
[         CEC:ff357118 ] (00:26) <1 0 0 0 0 0 7> [I2C] localI2C_Read : 378 (Task : CEC)
   982: 47888.640 WARN [I2C] localI2C_Read : 378 (Task : CEC)

Key event: aa -> 0e0e002e
[MPU] Sending : 06 04 05 0b 00 00
[    PowerMgr:ff1470d4 ] (00:01) <1 0 0 0 0 0 7> [PM] DisablePowerSave (Counter = 2)
[    PowerMgr:ff147144 ] (00:01) <1 0 0 0 0 0 7> [PM] EnablePowerSave (Counter = 1)
[    EventMgr:ff224b7c ] (8d:01) <1 0 0 0 0 0 7> emDeliverMulticastEvent : SW1OFF
[    MainCtrl:ff0d01c0 ] (9c:01) <1 0 0 0 0 0 7> ID:B(32)
[    MainCtrl:ff154748 ] (9c:03) <1 0 0 0 0 0 7> MeteringTimerStart
[    MainCtrl:ff0d8ba0 ] (85:03) <0 0 0 0 0 0 7> GUI_Control:81 0x0
[    MainCtrl:ff1549a4 ] (89:03) <0 0 0 0 0 0 7> EVENTID_METERING_TIMER_START
[      RscMgr:ff0e87dc ] (80:02) <0 0 0 0 0 0 7> srmEventDispatch: Current = 0, dwEventID = 13, dwParam = -442503148 S
[      RscMgr:ff0e89e4 ] (80:02) <0 0 0 0 0 0 7> srmEventDispatch: Current = 0, dwEventID = 13, dwParam = -442503148 E
[ GuiMainTask:ff0d8f54 ] (84:01) <0 0 0 0 0 0 7> GUI_CONTROL:81
[ GuiMainTask:ff1c1880 ] (84:01) <0 0 0 0 0 0 7> gui control end
[ GuiMainTask:ff1c18a0 ] (84:01) <0 0 0 0 0 0 7> 0msec = 47880 - 47880
[ GuiMainTask:ff1c18bc ] (84:01) <0 0 0 0 0 0 7> 0msec = 844032 - 844032
[     CtrlSrv:ff3ab2f4 ] (83:03) <0 0 0 0 0 0 7> IDLEHandler UNPRESS_SW1_BUTTON
[         CEC:ff347750 ] (c2:01) <0 0 0 0 0 0 7> cecCompleteStart Start
[         CEC:ff357118 ] (00:26) <0 0 0 0 0 0 7> [I2C] localI2C_Read : 378 (Task : CEC)

The additional values are the ones found in that function:

Code: [Select]

    APPEND("(%02x:%02x) <%x %x %x %x %x %x %x> ", r0, r1, MEM(0x910C8), MEM(0x910CC), MEM(0x910D4), MEM(0x910F8), MEM(0x910FC), MEM(0x91100), MEM(0x91178));


Code: [Select]

static uint32_t MEM(uint32_t addr)
{
    uint32_t buf;
    cpu_physical_memory_read(addr, &buf, sizeof(buf));
    return buf;
}

Therefore, the first one is what we are looking for (the one compared to 1 in the code).

This gives another hint to find half-shutter state: MeteringStart and MeteringTimerStart. I knew it was related to EVENTID_METERING_START / EVENTID_METERING_TIMER_START, but I didn't know where exactly it was set - this one was found with the mem_spy tool (now a module), that monitors memory addresses to find "interesting" changes.

To quote g3gg0:

... would have saved a lot of work if we had that earlier

P.S. Here's an easy coding task - print the name of the buttons in the logs.

[Licaon_Kter]

Which buttons are missing?

autoexpo.mo appears to be working on the EOSM though I have no idea how to work it.

Looks ok now though, you can change lower limits with left-right and upper values with rotating wheel, strange that (IIRC) this was not possible in the past, albeit the sourcode was not modified since forever. 

[dfort]

Oh where oh where has my little fonts gone?

Been here before and thought I had my head wrapped around the solution but probably not so let's start from scratch.

There is a block of font addresses in consts.h. Let's look at what you would think would be the closest cousin to the EOSM2 -- the EOSM:

Code: [Select]

// http://magiclantern.wikia.com/wiki/Fonts
#define BFNT_CHAR_CODES    0xffcb9c04
#define BFNT_BITMAP_OFFSET 0xffcbcb88
#define BFNT_BITMAP_DATA   0xffcbfb0c

The base address of DIGIC 5 cameras like the EOSM2 (and EOSM, 700D, 100D, etc.) is 0xFF0C0000 and addresses greater than the base address should show up in the ROM1.BIN disassembly. In fact it is quite easy to find where the fonts are in the disassembly because of strings like "HCanonGothic" and "CanonMonospace".

However, here is where the EOSM2 differs from its closest cousins. The fonts are located in ROM0.BIN like the 5D3.

If you follow the wikia link it will eventually lead you to a python script named find_fnt.py. A copy of that script is also located in magic-lantern/contrib/indy/. What it can do is display the font information buried in ROM0.BIN. The script was written back in the day when DIGIC 4 cameras were the norm so it assumes the base address is 0xFF010000 but it can accept a different address via a command line argument. Let's try it out on the EOSM2.

python find_fnt.py ROM0.BIN ff0c0000

Code: [Select]

Find bitmap fonts in Canon DSLR firmwares
Arm.Indy. based on work by Pel, Trammel Hudson and A1ex

0xff1e4838: FNT
0xff1e483c: (+0x04) 0xffd8
0xff1e483e: (+0x06) font_width = 40
0xff1e4840: (+0x08) charmap_offset = 0x24
0xff1e4844: (+0x0c) charmap_size = 0x31b4
0xff1e4848: (+0x10) bitmap_size = 0x8ab3e
0xff1e484c: (+0x14) font name = 'HCanonGothic///'
0xff1e485c: (+0x24) char_codes[]. 3181 chars
0xff1e7a10: (+0x31d8) offsets[]. Last offset value = 0x8ab08
0xff1eabc4: (+0x638c) bitmaps[]
  0xff2756cc: (+0x90e94) last bitmap
  +0x00: bitmap width = 28
  +0x02: bitmap height = 28
  +0x04: char width = 36
  +0x06: X offset = 4
  +0x08: Y offset = 16
    bitmap size = 0x70

0xff275704: FNT
0xff275708: (+0x04) 0xffd8
0xff27570a: (+0x06) font_width = 40
0xff27570c: (+0x08) charmap_offset = 0x24
0xff275710: (+0x0c) charmap_size = 0x188
0xff275714: (+0x10) bitmap_size = 0x31c4
0xff275718: (+0x14) font name = 'CanonMonospace'
0xff275728: (+0x24) char_codes[]. 98 chars
0xff2758b0: (+0x1ac) offsets[]. Last offset value = 0x3142
0xff275a38: (+0x334) bitmaps[]
  0xff278b7a: (+0x3476) last bitmap
  +0x00: bitmap width = 22
  +0x02: bitmap height = 22
  +0x04: char width = 22
  +0x06: X offset = 0
  +0x08: Y offset = 0
    bitmap size = 0x42

Now the EOSM2 is showing that it is also different from the 5D3 because the 5D3 shows each block twice at different addresses which allows you to easily figure out the ROM0 size.

Last time I searched for fonts was on the 5D3.134 firmware update and I didn't have QEMU working but now that I do I can check out the ROM layout:

EOSM2

Code: [Select]

F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror

So that means instead of using an address that starts at 0xFF we need to use 0xF[0-7] for ROM0 addresses. The 5D3 uses 0xF7 so let's use that for the EOSM2:

EOSM2

Code: [Select]

// http://magiclantern.wikia.com/wiki/Fonts
#define BFNT_CHAR_CODES    0xf71e485c
#define BFNT_BITMAP_OFFSET 0xf71e7a10
#define BFNT_BITMAP_DATA   0xf71eabc4
Oops--there is something wrong here. How do I know? Because on the 5D3 find_fnt.py shows this:

python find_fnt.py ROM0.BIN ff0c0000

Code: [Select]

0xff423a50: (+0x24) char_codes[]. 3165 chars
And the addressed used in consts.h is:

Code: [Select]

#define BFNT_CHAR_CODES    0xF7363A50
Revisiting the last time I ventured into the font rabbit hole:

Therefore, 0xF[0-7][3B]63A50 should all be valid choices for BFNT_CHAR_CODES.

[3B]?

So confused -- why doesn't find_fnt.py just spit out the right numbers?