ML on EOS-M2

Started by Palpatine, September 22, 2015, 02:48:23 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

jonkjon

Is this something that needs to be built from scratch or is there something to build from? I have some c experience (mostly windoze) and i'd be willing to take a look. I have perused the getting started stuff but it pretty much just shows how to setup a compiler etc. I haven't looked at the source code at all.

--Jon

godashram

the m2 was dumped (a1ex has the dump I made)

from there, i guess it would be a port of the existing build, beyond that, no clue.
Canon T5I, EOS M2

Walter Schulz

What you are looking for is a maintainer. Someone knowing C, assembler programming embedded devices (preferable ARM architecture) and willing and able to spend several hundred hours of his/her time in porting ML to M2. And don't expect any of the devs asking for that additional workload. They will support this maintainer-to-be, of course.

bender

any updates for the M2?  :)

brocolimayo

same question :) my programming skills are almost zero, but I'd love to help, I own a m2 too, and would be wonderful to have ml on it :)

godashram

nope, as far as I know, there isn't anyone willing to work on it.
Canon T5I, EOS M2

Skylin3

Bump. Is there any progress with m2? I can help with testing ML, I have m2.

dfort

Got one.



Not sure if I'm ready for this but I was going to quietly work on Magic Lantern for the EOSM2. Seems like this one should be one of the "easier" ports.

Then I thought, that's no way to work on an open source project, let's get this out there in the open. My first stumbling block is setting the boot flag on the camera. How to do this without bricking the camera? Tried searching for the answer and it looks like there is already a working .FIR file for this camera -- so how can I get my hands on it?

a1ex

Quote from: dfort on May 24, 2017, 05:50:16 AM
Seems like this one should be one of the "easier" ports.

Definitely, as it's plain DIGIC 5 (unlike 1300D, which looks like a mix between D4, 5 and 6).

The boot flag is "traditionally" enabled after getting the Hello World working. There are two flavors: the minimal target (which requires a tiny set of stubs and only compiles a few small files) and the regular one (CONFIG_HELLO_WORLD in the full source code, which also requires file I/O for loading fonts, and a bunch of other initializations). Once the second version is working, CONFIG_DUMPER_BOOTFLAG will be straightforward to compile and run. The "user-friendly" installer comes a bit later, as it requires some more stubs.

The boot flag can be also enabled earlier, from the bootloader context; we did that on the DIGIC 2 and 3 models, blindly (without having the display available for debugging or user feedback). Now that we have display access from bootloader, that's a valid option as well (though I wasn't comfortable doing this step on 7D Mark II).

Until enabling the boot flag, you'll need my assistance to run a binary on the camera. You can run them in QEMU though, by setting "boot=1" though.

Will explain how to get the Hello World on models that don't boot to Canon GUI on the 1300D thread, as it was covered on IRC a few days ago.

a1ex

I'll start with a walkthrough using latest QEMU and IDA. First step is to assume it's very similar to EOSM, so after installing QEMU, head over to qemu/qemu-2.5.0/hw/eos/model_list.c and copy the basic entries from the section for EOSM:


    {
        .name                   = "EOSM2",
        .digic_version          = 5,
     }


The others are likely model-specific, so let's not assume too much from the beginning. Let's try:


./run_canon_fw.sh EOSM2,firmware="boot=0"
...
FFFF3F50: MCR p15,0,Rd,cr6,cr0,0:  946_PRBS0 <- 0x3F       (00000000 - FFFFFFFF, 0x100000000)
FFFF3F58: MCR p15,0,Rd,cr6,cr1,0:  946_PRBS1 <- 0x3D       (00000000 - 7FFFFFFF, 0x80000000)
FFFF3F60: MCR p15,0,Rd,cr6,cr2,0:  946_PRBS2 <- 0xE0000039 (E0000000 - FFFFFFFF, 0x20000000)
FFFF3F68: MCR p15,0,Rd,cr6,cr3,0:  946_PRBS3 <- 0xC0000039 (C0000000 - DFFFFFFF, 0x20000000)
FFFF3F70: MCR p15,0,Rd,cr6,cr4,0:  946_PRBS4 <- 0xFF00002F (FF000000 - FFFFFFFF, 0x1000000)
FFFF3F78: MCR p15,0,Rd,cr6,cr5,0:  946_PRBS5 <- 0x37       (00000000 - 0FFFFFFF, 0x10000000)
FFFF3F80: MCR p15,0,Rd,cr6,cr6,0:  946_PRBS6 <- 0xF700002F (F7000000 - F7FFFFFF, 0x1000000)
...
K355 READY
128K Sector FROM From BL 0xffff
[SF] InstallSerialFlash 6 0xc022c0d4 0x0 0x1000000 1
...
K355 ICU Firmware Version 1.0.2 ( 6.0.5 )


OK, so we've got a rough idea about its memory map (same as other DIGIC 4 and 5 models). We also noticed it has a serial flash of size 0x1000000. This is similar to 100D (lookup serial_flash_size in model_list.c), so it may be a good idea to reuse the serial flash data from this model:

        .serial_flash_size      = 0x1000000,


Unfortunately, this step doesn't change anything in the emulation. Let's look at the next messages:

...
    10:    46.336 [PROPAD] ERROR Not Exist Valid ComboPackages!! 0x10000
...
    14:    56.576 [STARTUP] InitializeIntercom
...
    42:   722.688 ERROR [DL] ########## DL ERROR!!!!! ###########


- Many errors regarding missing properties. These are either stored in ROM, or in the serial flash, or they come from the MPU. The ones from ROM should not pose any problems, since we have the complete ROM. For the others, we did not see any sign of activity, so we have to dig in to find out whether anything is different (usually these communication channels may use different registers, depending on model, but overall the communication protocol is the same).

- DL ERROR: if you look in EOSM/debugmsg.gdb, you'll find a way to get past this message (by skipping the initialization of "DL", whatever that is).

- The debug messages are not very informative. To get more insights, find out a few essential functions (such as task_create and DebugMsg) and place them in EOSM2/debugmsg.gdb (look at other models for how the declarations should look like).

OK, so at this point we have to disassemble the firmware. If you use IDA, the initial steps would be:
- load ROM1.BIN
- select ARM processor, architecture ARMv5TEJ (we have ARMv5TE on all models from DIGIC 2 to DIGIC 5)
- uncheck 'Delete instructions with no xrefs' (we have plenty of those)
- uncheck 'Perform no-return analysis' (IDA usually gets it wrong)
- RAM section: 0, size 0x100000 (we can always add more later)
- ROM start address: 0xFF000000
- ROM size: 0xFFFFFC (off-by-one bug in IDA; I have an older version, so YMMV)
- File load address / size: same as ROM.

Now that our binary is loaded, go to 0xFFFF0000 and press "c" to mark this as code. This is the bootloader start address (aka HIVECS in ARM documentation). Next step is to find where the main firmware starts (where the code path leaves the bootloader). It's easy to assume similarity with other models, but let's try to find it from scratch (just to show off the latest QEMU tool):


./run_canon_fw.sh EOSM2,firmware="boot=0" -d calls
...
call 0xFFFF0FCC(0, ffffdfff, 2000, ffff0b20)                                     at [ffff09c8:0]
call 0x100000(0, 11836c, 11836c, 100000)                                        at [ffff1010:ffff09cc]
...


OK, so the bootloader runs from RAM.

Where does the bootloader end? When it stops executing from its address range (near 0xFFFF0000 or 0x100000):


  return 0 to 0x10013C                                                           at [1022f4:ffff1014]
return 1 to 0xFFFF1014                                                          at [1000bc:ffff09cc]
return 0 to 0xFFFF09CC                                                           at [ffff101c:0]
Warning: R10 not restored (0xa -> 0x1)                                           at [ffff101c:ffff09cc]
PC jump? 0xF80C0000 lr=ffff09cc                                                  at [ffff0a04:ffff09cc]
0xffff0a04:  e1a0f000      mov pc, r0
PC jump? 0xFF0C000C lr=ffff09cc                                                  at [f80c0000:ffff09cc]
0xf80c0000:  e59ff0c4      ldr pc, [pc, #196] ; 0xfffffffff80c00cc


Ta-da! Same as other DIGIC 5 models: ROM1 startup code is at 0xF80C0000, but code from main firmware expects to run from a mirror copy (details). So, our ROMBASEADDR is 0xFF0C0000.

Let's go there in IDA and mark this section as code. At this point, IDA already recognized a few functions, but let's get some more from the QEMU execution trace:


./run_canon_fw.sh EOSM2,firmware="boot=0" -d idc
...
EOSM2.idc saved.


Now load this IDC script into IDA, or convert it for your favorite disassembler, and start looking at the functions called during the execution in QEMU:

PC jump? 0xFF0C000C lr=ffff09cc                                                  at [f80c0000:ffff09cc]
...
call 0xFF0C1BD4(1000, 698, eeeeeeee, 1000)                                       at [ff0c0dbc:0]
call 0x866B4(f88, 74, eeeeeeee, 1000)                                           at [ff0c1be4:ff0c0dc0]
return ffc to 0xFF0C1BE8                                                        at [866f4:ff0c0dc0]
call 0x3168(f88, ff0c57a4, 0, 0)                                                at [ff0c1c80:ff0c0dc0]
...


0xFF0C1BD4 is cstart, 0x866B4 is bzero32, 0x3168 must be create_init_task and 0xFF0C57A4 must be init_task. Some of these functions are called from RAM, so we'll have to identify where its contents come from (what is copied where). TODO.

Next step, for the emulation, would be to find the MPU registers (near InitializeIntercom), mpu_send/mpu_recv stubs may also help, and we also need to check the serial flash communication.

For executing user code on the camera, next step would be to reserve memory for our application. This is usually done by shrinking the "malloc" buffer. Take a look at boot-hack.c from the "qemu" branch (as it's a bit more verbose and better explained). These debug messages can be seen on the QEMU-boot-check page.

To be continued.

dfort

Wow.

Let's take half a step back before embarking on this journey. When I posted my intent to attempt this port I skipped one important step before asking for a firmware dumper--check to see if there is already one available. The list of available ROM dumpers are listed at the bottom of the Magic Lantern Nightly Builds page. The one for the EOSM2 point to a post on this topic with the links to download the dumpers.

It appears that the EOSM2 ROM was first dumped about 1.5 years ago. I'm assuming that it was the original firmware version. The camera I got has 1.0.2 installed and 1.0.3 is available. My question is, should I try to make a dump of 1.0.2 before updating the firmware or does it matter? Maybe the dumpers won't work with 1.0.3? Maybe we should archive a 1.0.2 dump before starting? The 1.0.3 firmware update is a minor fix that Canon released for most of their cameras to deal with this issue:
QuoteCorrects a phenomenon in which when using the camera with the EF-S 18-135mm f/3.5-5.6 IS USM or EF 70-300mm f/4-5.6 IS II USM lens, even if lens aberration correction is set to "Enable", correction will not be applied.

[EDIT]

hello-M2.fir running on EOSM.102


However, dumperM2.fir doesn't dump the ROM. I tried various cards including an old 2GB card that has worked on all the other ROM dumps I made.

a1ex

The dumper linked above runs on the main firmware (in contrast with the portable ROM dumper, which runs from bootloader), so it has no issues with card sizes, but it's limited to DIGIC 4 and 5 models. Does it help if you follow the instructions in the linked post? (it's different from the portable dumper you are used to)

You can start from the latest firmware.

dfort

Quote from: a1ex on May 24, 2017, 09:23:14 PM
Does it help if you follow the instructions in the linked post?

Doh!

The single file dump is named "(NULL)" on the sd card and it is 33.6 MB. I also updated the firmware to 1.0.3 though I also dumped 1.0.2 just to have it.

Did the change to model_list.c and tried running qemu but it stopped on:
eos_load_image: file not found './EOSM2/ROM0.BIN'
./run_canon_fw.sh: line 35: 16058 Abort trap: 6           $QEMU_PATH/arm-softmmu/qemu-system-arm -drive if=sd,format=raw,file=sd.img -drive if=ide,format=raw,file=cf.img -M $*


So I renamed (NULL) to ROM0.BIN then it stopped on "file not found './EOSM2/ROM1.BIN'" but since the dumper only creates the one file I'm afraid I hit another wall.

Quote from: a1ex on October 04, 2014, 08:30:21 AMIt dumps 32 MB from 0xF7000000, which covers both ROM0 and ROM1 - that's what you need to run ML in QEMU. For analyzing the code, you can load ROM1 (the second half of that file) at 0xFF000000.

I'm searching how to run ML in QEMU with a single file dump but so far I can't find an answer.


a1ex

Split it in two halves: first ROM0, second ROM1.

Back then, it was much easier to patch Canon code to dump everything in one file, so it covers both ROMs. For this reason, older QEMU versions used to require the ROMs to be joined like this (but it was changed to allow using the backup ROM copies already saved by ML on the cards under ML/LOGS, and to allow more unusual ROM configurations).

dfort

Making some progress. Mac uses the BSD version of split which doesn't have the -n option but I was able to split the dump in two like this:
split -b 16800000 "(NULL)" ROM

That gave me two files, ROMaa and ROMab which I renamed ROM0.BIN and ROM1.BIN and placed in ~/qemu/EOSM2 then ran:./run_canon_fw.sh EOSM2,firmware="boot=0"
./run_canon_fw.sh: line 10: losetup: command not found
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]
./run_canon_fw.sh: line 10: losetup: command not found
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]
CHK version_gen.h
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - DFFFFFFF: eos.iomem
[EOS] loading './EOSM2/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './EOSM2/ROM1.BIN' to 0xF8000000-0xF8FFA6FF
[MPU] FIXME: no MPU spells for EOSM2.
[MPU] FIXME: no MPU button codes for EOSM2.
Setting BOOTDISK flag to 0


It looks like this:



QEMU didn't crash but it also isn't displaying the output from the minimal qemu autoexec.bin.

dfort

Took some more baby steps. First a fall.
Quote from: a1ex on May 24, 2017, 12:17:11 PM
OK, so we've got a rough idea about its memory map (same as other DIGIC 4 models). We also noticed it has a serial flash of size 0x1000000. This is similar to 100D (lookup serial_flash_size in model_list.c), so it may be a good idea to reuse the serial flash data from this model:

        .serial_flash_size      = 0x1000000,


All that did was to consistently give me this error when running QEMU:

[EOS] loading './EOSM2/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './EOSM2/ROM1.BIN' to 0xF8000000-0xF8FFA6FF
Could not open ./EOSM2/SFDATA.BIN


I don't have IDA but I do have ARM-console and the good old reliable disassemble.pl from the finding stubs tutorial. So I took another baby step and tried to disassembly the ROM1.BIN that was split off the single file dumperM2.fir dump. I ran the usual "perl disassemble.pl 0xFF000000 ROM1.BIN" but something looked wrong.

ff0ba700: e59ff0c4 ldr pc, [pc, #196] ; ff0ba7cc: (ff0c000c)
"gaonisoy":


All indications are that it should be more like the EOSM and 700D and start on 0xFF0C0000 so I did a second disassemble, "perl disassemble.pl 0xFF005900 ROM1.BIN"

ff0c0000: e59ff0c4 ldr pc, [pc, #196] ; ff0c00cc: (ff0c000c)
"gaonisoy":


Now it "looks" right. Well, "looks" is mostly what I'm going on right now. I don't really know what I'm doing. That weird offset might be a result of splitting the dump in two? The EOSM2 "looks" like a combination of the EOSM, 100D with wifi features added on. This means that I should be able to find some stubs using the pattern matching method I used for the minor firmware updates. Of course porting a new camera is certainly not a minor undertaking!

I want to continue getting QEMU working properly on the Mac and figure things out the "right" way but now that I've got a disassembly at least I'm on familiar territory so do you think I'm at the stage where I can start hunting down some stubs?

a1ex

You need two *equal* halves, without any approximation.

(edit: updated QEMU to give better warnings in this case)

SFDUMP.BIN can probably be used from 700D or EOSM as well, but these cameras have a smaller size. Padding it with zeros may or may not work. How to get it? It's covered in QEMU's install script.

dfort

Quote from: a1ex on May 25, 2017, 08:40:40 AM
You need two *equal* halves, without any approximation.

(edit: updated QEMU to give better warnings in this case)

Yeah, that seems to be a problem. "split -b 16800000 "(NULL)" ROM" produced two files that aren't exactly equal. I'll keep up with the QEMU branch commits on this project.

-rw-r--r--   1 rosiefort  staff  16800000 May 24 15:15 ROM0.BIN
-rw-r--r--   1 rosiefort  staff  16754432 May 24 15:15 ROM1.BIN


[EDIT] Looks like the lesson learned here is not to trust the finder because it rounds KB to MB.



I might be able to get SFDUMP.BIN from a 100D.

dfort

The equal split did the trick. Now I'm seeing the messages a1ex posted back on Reply #34. Also made a new disassembly and it looks good so I'll start hunting for stubs.

Quote from: a1ex on May 24, 2017, 12:17:11 PM
To be continued.

Looking forward to it!

a1ex

Alright, so let's start from where we've left off.

Quote from: a1ex on May 24, 2017, 12:17:11 PM
0xFF0C1BD4 is cstart, 0x866B4 is bzero32, 0x3168 must be create_init_task and 0xFF0C57A4 must be init_task. Some of these functions are called from RAM, so we'll have to identify where its contents come from (what is copied where). TODO.

Since the ROMs are placed at some high address (after 0xF0000000), and we've noticed some functions called outside this range, a good guess would be that some of the functions may be copied from ROM to RAM during startup. Let's examine the memory access patterns. We are interested in some code that reads from ROM and writes to RAM. If you have no trouble following the assembly code for the startup sequence, you'll spot this piece of code right away. Otherwise, let's ask for some help from QEMU:

Running QEMU with -d help gives, among many other options:

ram        EOS: log all RAM reads and writes
rom        EOS: log all ROM reads and writes
ramr       EOS: log all RAM reads
romr       EOS: log all ROM reads
ramw       EOS: log all RAM writes
romw       EOS: log all ROM writes


We could try romr,ramw (or rom,ram if you don't mind a little more verbosity):

./run_canon_fw.sh EOSM2,firmware="boot=0" -d romr,ramw


The log is huge, but if you scroll around, there are some large blocks that look similar. Some of them are clearly a copying operation (read some value from ROM, write it to RAM). One of these covers the functions we are interested in:

[rom1]     at 0xFF0C000C:FFFF09CC [0xFFD1F02C] -> 0xE92D4010
[ram]      at 0xFF0C000C:FFFF09CC [0x00001900] <- 0xE92D4010: was 0x0;
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFD1F030] -> 0xE1A04001
[ram]      at 0xFF0C00A4:FFFF09CC [0x00001904] <- 0xE1A04001: was 0x0;
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFD1F034] -> 0xE59F1040
...
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFDD60C8] -> 0xFFD1EFFC
[ram]      at 0xFF0C00A4:FFFF09CC [0x000B899C] <- 0xFFD1EFFC: was 0x0;


In other words, a memory block from 0xFFD1F02C to 0xFFDD60C8 (actually 0xFFDD60CB, since we are looking at 32-bit operations) is copied to 0x1900 - 0xB899F. The size of the copied block is 0xb70a0 bytes. To extract this block, grab the terminal and run this under a Bash prompt:


dd if=ROM1.BIN of=1900.BIN bs=1 skip=$((0xD1F02C)) count=$((0xB70A0))


(please note the above numbers are only valid for firmware 1.0.2)

Now you can disassemble this file starting from 0x1900. In IDA, load this file as Additional binary file. If you use ARM-console, this should auto-detect the above copying operation (so you can simply start browsing the functions copied into RAM).

There are other similar blocks copied into RAM (newer DIGIC 6 and 7 models have plenty of those), and finding them manually can become tedious, so it's a good candidate for automation.

One of these blocks is the interrupt handler routine. On ARM, hardware interrupts are executed from address 0x18 (see e.g. ARM ARM - A2.6 - Exceptions or these slides). That means, we should expect to find some code that writes something to address 0x18 (that can be anywhere in the boot process before enabling interrupts), and on EOS firmwares, at this address you'll find a jump to 0x4B0 on many models, or something close on others. Let's examine it in GDB:


./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S & arm-none-eabi-gdb -x debug-logging.gdb

(note: we will need to create a GDB file for EOS M2, named EOSM2/debugmsg.gdb to match other models, but since we don't have one yet, we can just use the generic version - which we are going to include in the platform-specific file later)

Let it run for a while (continue), then hit CTRL-C and examine the exception vector (located at address 0):


(gdb) disas 0,0x40
Dump of assembler code from 0x0 to 0x40:
   0x00000000: nop ; (mov r0, r0)
   0x00000004: ldr pc, [pc, #20] ; 0x20
   0x00000008: ldr pc, [pc, #20] ; 0x24
   0x0000000c: ldr pc, [pc, #20] ; 0x28
   0x00000010: ldr pc, [pc, #20] ; 0x2c
   0x00000014: nop ; (mov r0, r0)
   0x00000018: ldr pc, [pc, #16] ; 0x30
   0x0000001c: ldr pc, [pc, #16] ; 0x34
   0x00000020: ; <UNDEFINED> instruction: 0xff0c0fec
   0x00000024: ; <UNDEFINED> instruction: 0xff0c105c
   0x00000028: ; <UNDEFINED> instruction: 0xff0c1000
   0x0000002c: ; <UNDEFINED> instruction: 0xff0c1018
   0x00000030: ; <UNDEFINED> instruction: 0x000004b0
   0x00000034: ; <UNDEFINED> instruction: 0xff0c1060
   0x00000038: ; <UNDEFINED> instruction: 0xffff0b08
   0x0000003c: ; <UNDEFINED> instruction: 0xffff0b20


That means, once a hardware interrupt happens (in other words, some device wants to tell the main CPU that it needs attention), the program counter will jump to 0x18, and there it will find a LDR PC, =0x4B0. That's where Canon's interrupt handler is located. Let's find its contents:

- Method 1: scan the memory access log for a copy operation that covers 0x4B0 and above. Easy to find.

- Method 2: disassemble it directly from GDB. How much? 0x1900 - 0x4B0 would be the upper limit (since we already know what's at 0x1900). The interrupt handler is much smaller than that - 512 bytes are more than enough:

disas 0x4B0,0x6B0


Method 3: dump the memory from gdb:

(gdb) dump memory 4B0.BIN 0x4B0 0x6B0


Method 4: dump the memory from QEMU monitor:

(qemu) memsave 0x4B0 0x200 4B0.BIN


For the debugmsg.gdb file, we'll need some basic stubs: DebugMsg and task_create (easy to find - one is used for most debug messages, the other can be easily identified from things that look like task names) and two addresses: CURRENT_TASK and CURRENT_ISR (interrupt service routine). We can find the latter in the interrupt handler - quote from debugmsg.gdb:
Quote
# CURRENT_ISR:
#   From interrupt handler (PC=0x18), find an expression that evaluates to
#   the current interrupt ID if any is running, or 0 if a normal task is running.
#   - on DIGIC 4/5, interrupt ID is MEM(0xC0201004) >> 2
#   - on DIGIC 6,   interrupt ID is MEM(0xD4011000)

If you can find that by examining disassembled code, great. If not, QEMU to the rescue:

./run_canon_fw.sh EOSM2,firmware="boot=0" -d io,int,nochain -singlestep
...
Taking exception 5 [IRQ]
[INT]      at 0x00000510:0000050C [0xC0201004] -> 0x28      : Requested int reason 28 (INT 0Ah)
[INT]      at 0x00000594:00000568 [0xC0201010] <- 0xA       : Enabled interrupt 0Ah


(note: -d nochain -singlestep gives more precise locations in I/O logs - otherwise, QEMU groups the ARM instructions in blocks for faster execution, so the reported location will be approximate)

OK, so the register we are looking for is read at 0x510. Let's disassemble with GDB:

(gdb) disas 0x4B0,0x6B0
...
   0x0000050c: ldr r0, [pc, #368] ; 0x684
   0x00000510: ldr r4, [r0]
   0x00000514: str r4, [pc, #304] ; 0x64c
...
(gdb) x 0x684
0x684: 0xc0201004
(gdb) x 0x64c
0x64c: 0x00000028
(gdb) print 0x28>>2
$1 = 0xa


The register we are looking for is loaded in R0, then the interrupt handler reads from this register (result in R4) and stores its value at 0x64C. If you check the memory contents at this address, you'll find 0x28. The interrupt ID would be 0x28 >> 2 = 0x0A, which is DryOS timer interrupt (which fires every 10ms and is used for the task switch - see e.g. FreeRTOS Tick for some background info).

Note: ARM has a generic hardware interrupt (PC jumping at 0x18), but in an embedded system we usually have more devices that can trigger an interrupt. To tell which device needs attention, it's helpful to identify these devices somehow - and an interrupt ID is used for that. On EOS, there is a custom interrupt controller, and on DIGIC 4 and 5 models, the register used for identifying the interrupt ID is 0xC0201004. The interrupt ID is hardwired to the device that triggers it (for example, SIO3 and MREQ, which are used for MPU communication, always use interrupts 0x36 and 0x50). This mapping can change across models, but usually it's consistent (at least within the same generation of models). Probably g3gg0 can explain these concepts a bit better, as he identified all this stuff a few years ago, when I was mostly clueless about how these things work.

OK, so the memory contents at 0x64C can tell us about the interrupt currently handled. There's a problem - this address is not cleared when the interrupt handler is done, so, just by looking at it, we can't tell whether the code is servicing an interrupt or running a regular task. We'll need to look at some other address - and in many cases the previous address is a good candidate:

   0x000004d4: ldr r1, [pc, #364] ; 0x648
   0x000004d8: cmp r1, #0
   0x000004dc: add r1, r1, #1
   0x000004e0: str r1, [pc, #352] ; 0x648
   ...
   0x00000598: ldr r1, [pc, #168] ; 0x648
   0x0000059c: sub r1, r1, #1
   0x000005a0: str r1, [pc, #160] ; 0x648


This 0x648 looks like a counter that tells how many nested interrupts we are handling (yes, they can be nested - unfortunately, as this makes it a lot harder to understand, debug, emulate and so on). Let's confirm its functionality with RAM tracing:

./run_canon_fw.sh EOSM2,firmware="boot=0" -d io,int,ram,nochain -singlestep
...
[tcm_code] at 0x000004D4:19980218 [0x00000648] -> 0x0       
[tcm_code] at 0x000004E0:19980218 [0x00000648] <- 0x1       : was 0x0;
...
[tcm_code] at 0x0000050C:0000050C [0x00000684] -> 0xC0201004
[INT]      at 0x00000510:0000050C [0xC0201004] -> 0x28      : Requested int reason 28 (INT 0Ah)
[tcm_code] at 0x00000514:0000050C [0x0000064C] <- 0x28      : was 0x0;
...
[INT]      at 0x00000594:00000568 [0xC0201010] <- 0xA       : Enabled interrupt 0Ah
[tcm_code] at 0x00000598:00000568 [0x00000648] -> 0x1       
[tcm_code] at 0x000005A0:00000568 [0x00000648] <- 0x0       : was 0x1;


Looks right!

Note: "Enabled interrupt 0Ah" means the interrupt routine finished handling it, so the same interrupt can be triggered again from now on (you can't have the same interrupt nested with itself). The same register configuration happens when an interrupt is enabled for the first time, and that's why the message reads "enabled interrupt". Might be a bit confusing at first sight.

Now that we know what 0x648 does, let's use it as boolean (0 = regular task, nonzero = handling some interrupt):

macro define CURRENT_ISR  (*(int*)0x648 ? (*(int*)0x64C) >> 2 : 0)


Next is CURRENT_TASK. One way to find that is by pattern matching in task_create and its subroutines.

I'll try an alternate method, based on one of the recent addition to QEMU: the ability to track function calls and returns. From the ARM calling convention (AAPCS, or the summary from wikipedia), you'll notice that registers R4-R11 are supposed to be unchanged after a function call. When tracking function calls, the logging code checks this assumption as well, and will print lots of warnings if a task switch happens without the logger knowing about it. Let's use these warnings to narrow down the location where DryOS switches between tasks:


./run_canon_fw.sh EOSM2,firmware="boot=0" -d callstack,ramw


The log is huge, but we are going to look for some warnings about registers (run with only -d callstack to see them without the clutter from RAM writes). These warnings are likely caused by some memory writes performed just before the warnings. We are looking for some invariant address (one that doesn't change with the warnings):

./run_canon_fw.sh EOSM2,firmware="boot=0" -d callstack,ramw |& grep -B 20 "not restored"
...
[ram]      at 0x000019D8:0000E24C [0x0018F8FC] <- 0xE24C    : was 0xE24C;
[ram]      at 0x00001A04:0000E24C [0x0008FBCC] <- 0x17EED4  : was 0x17EF28;
[ram]      at 0xFF0C1064:0000E24C [0x0018F8FC] <- 0xE24C    : was 0xE24C;
...
[ram]      at 0xFF0C1074:0000E24C [0x0018F8C0] <- 0x60000093: was 0x19980218;
[ram]      at 0xFF0C1078:0000E24C [0x0017EF78] <- 0x18F8C0  : was 0x18F8D0;
Warning: R4 not restored (0x17ef28 -> 0x17eed4)                                  at [ff0c1088:e24c]
...
[ram]      at 0x000019D8:0000E24C [0x0018E9A4] <- 0xE24C    : was 0xE24C;
[ram]      at 0x00001A04:0000E24C [0x0008FBCC] <- 0x17EF28  : was 0x17EED4;
[ram]      at 0xFF0C1064:0000E24C [0x0018E9A4] <- 0xE24C    : was 0xE24C;
...
[ram]      at 0xFF0C1074:0000E24C [0x0018E968] <- 0x60000093: was 0x18E9FC;
[ram]      at 0xFF0C1078:0000E24C [0x0017EF24] <- 0x18E968  : was 0x18E970;
Warning: R4 not restored (0x17eed4 -> 0x17ef28)                                  at [ff0c1088:e24c]


Most of these addresses are different for each warning message, except one. Let's try it. Find task_create and DebugMsg (easy) and add everything to EOSM2/debugmsg.gdb (look at other cameras for a template):

# ./run_canon_fw.sh EOSM2 -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb

source -v debug-logging.gdb

macro define CURRENT_TASK 0x8FBCC
macro define CURRENT_ISR  (*(int*)0x648 ? (*(int*)0x64C) >> 2 : 0)

b *0x4398
DebugMsg_log

b *0x7360
task_create_log



./run_canon_fw.sh EOSM2 -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb
...
[      init:ff352264 ] task_create(PowerMgr, prio=20, stack=400, entry=ff352088, arg=0)
[      init:ff1470d4 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[      init:0003671c ] task_create(DbgMgr, prio=1f, stack=0, entry=36628, arg=46e584)
[      init:ff0c3334 ] (8b:16)
K355 ICU Firmware Version 1.0.2 ( 6.0.5 )
[      init:ff0c3348 ] (8b:05)
ICU Release DateTime 2013.12.02 09:28:54
[      init:ff0f5ea0 ] (00:03) [SEQ] CreateSequencer (Startup, Num = 6)
[      init:ff0f5f28 ] task_create(Startup, prio=19, stack=2800, entry=ff0f5d6c, arg=46e880)
[      init:ff0f60f4 ] (00:02) [SEQ] NotifyComplete (Startup, Flag = 0x10000)
[      init:ff0f6158 ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x10000, Flag = 0x10000)
[      init:ff0c33d4 ] task_create(TaskMain, prio=1d, stack=0, entry=ff0c28d8, arg=0)
[   Startup:ff0f5db4 ] (00:05) [SEQ] seqEventDispatch (Startup, 0)
[   Startup:ff0c373c ] (8b:05) startupEntry
[   Startup:ff0c375c ] task_create(Startup2, prio=11, stack=400, entry=ff0c3604, arg=0)
[  Startup2:ff1310c4 ] (02:16) PROPAD_CreateFROMPropertyHandle DRAMAddr 0x416d5b00
[  Startup2:ff14a204 ] (00:01) [SF] IsAddressSerialFlash 0x2d0000
...


We now have task information and debug messages!

To make this information available in QEMU (not just in GDB), add this to model_list.c under the EOSM2 section:

        .current_task_addr      = 0x8FBCC,


That's it for today.

a1ex

Quote from: a1ex on May 26, 2017, 11:09:39 PM
There are other similar blocks copied into RAM (newer DIGIC 6 and 7 models have plenty of those), and finding them manually can become tedious, so it's a good candidate for automation.


./run_canon_fw.sh EOSM2,firmware="boot=0" -d romcpy |& grep ROMCPY
[ROMCPY] 0xFFFF0000 -> 0x0        size 0x40       at 0xFFFF0980
[ROMCPY] 0xFFFE0000 -> 0x100000   size 0xFF2C     at 0xFFFF0FCC
[ROMCPY] 0xFFD1F02C -> 0x1900     size 0xB70A0    at 0xFF0C000C
[ROMCPY] 0xFF0C0E04 -> 0x4B0      size 0x1E8      at 0xFF0C0D70
[ROMCPY] 0xFFA12904 -> 0x4E0E98   size 0xC7C      at 0x8645C   


This should work on all models from DIGIC 2 to DIGIC 7 (both EOS and PowerShot).

Quote from: a1ex on May 26, 2017, 11:09:39 PM
We now have task information and debug messages!

... and also an option to show all context switches:

./run_canon_fw.sh EOSM2,firmware="boot=0" -d tasks
...
Task switch to idle:ff0c108c                                                     at [idle:1db8:ff0c1c84]
Task switch to init:ff0c1064                                                     at [init:1a18:c73c]
K355 READY
128K Sector FROM From BL 0xffff
...
SerialFlash Initialize
Task switch to Startup:ff0c108c                                                  at [Startup:1db8:c9e0]
Task switch to Startup2:ff0c1064                                                 at [Startup2:1a18:228c]
...
Task switch to DbgMgr:ff0c1064                                                   at [DbgMgr:1a18:e24c]
     4:    54.784 [PROPAD] PROPAD_CreateFROMPropertyHandle DRAMAddr 0x416d5b00
Task switch to Startup:ff0c1064                                                  at [Startup:1a18:2e40]
[RTC] !! RTC_TIME_CORRECT_CHANGE!  0x0 ---> 0x9a
[RTC] !! RTC UNDER MIN_YEAR !!
Task switch to PropMgr:ff0c1064                                                  at [PropMgr:1a18:da2c]
...


8)

DeafEyeJedi

All of this is smoking hot guys and have been on the outlook for a used M2 body just because I would like to join the foam party!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

a1ex

The nice part is that all other models can follow (more or less) the same procedure. Credits go to dfort for getting the tutorials ball rolling - that's the reason I've picked this model for a walkthrough.

In other words, owners of other models will no longer have excuses for not knowing how to port ML to their camera :D

Anyway, let's continue figuring out the emulation.

Quote from: a1ex on May 24, 2017, 12:17:11 PM

        .serial_flash_size      = 0x1000000,


Unfortunately, this step doesn't change anything in the emulation.

Oops, I was wrong here. To quote Ange Albertini - https://twitter.com/angealbertini/status/773650987839926272
QuoteReverse engineering tip: it's perfectly fine to
- have no idea what to do next
- have made wrong assumptions
- take 'too long'

Here's the detail I've missed the first time:

SerialFlash Initialize
[EEPROM-DMA]! [0x2D0000] -> [0x416D5B00] (0x1DC400 bytes)


Also, error messages like this will disappear after declaring .serial_flash_size:

     6:    16.640 [PROPAD] ERROR Not Exist Valid ComboPackages!! 0x2D0000


That's good - it probably means we have nothing else to fix regarding serial flash. Running with -d sflash shows plenty of serial flash activity (but I swear I did ran this command back then and noticed none of it...)

Let's move on to the next item - MPU messages. You can find the registers used for these in InitializeIntercom (surrounded in the disassembly by "InitializeIntercom" ... "InitializeIntercom End(%#x)"). The function call between these two strings is:

ROM:FF0C3A9C                 LDR     R2, =0xC020302C
ROM:FF0C3AA0                 LDR     R0, =0xC022006C
ROM:FF0C3AA4                 MOV     R1, #0x50 ; 'P'
ROM:FF0C3AA8                 BL      sub_3AD0


Looking up these registers, you'll find the former on the section with DIGIC 5 defaults (so nothing to change about it), and the latter on the EOSM, 100D and some other models. Let's copy it:

        .mpu_request_register   = 0xC022006C,
        .mpu_status_register    = 0xC022006C,


Side note: we were very lucky, as EOS M2 is very similar to 100D and other models from the same generation. 1300D was a lot more different (used registers different from all other models, different registers for request and status - unlike all other models - and also different values for driving these GPIOs). Refer to this commit for the gory details - generally you'll find this kind of changes with cameras from different generations. I expect this kind of changes on DIGIC 6 models - didn't look into this part yet because 1) mixed ARM/Thumb code is a lot harder to reverse engineer (at least with the tools I know) and 2) I don't have any MPU communication logs from a DIGIC 6 models (it might work with replaying one from a recent DIGIC 5, no idea).

Still, after enabling the MPU registers, there's very little MPU activity:

[MPU] Received: 06 04 02 00 00 00  (unknown spell)
[MPU] Received: 0a 08 03 57 10 2f 00 01 00 00  (unknown spell)
[MPU] Received: 08 07 03 6a 01 08 00 00  (unknown spell)


And these warnings:

[MPU] FIXME: no MPU spells for EOSM2.
[MPU] FIXME: no MPU button codes for EOSM2.


Let's try our luck by assuming the MPU communication is similar to 100D. That model already boots Canon GUI, so it looks like a perfect match. Lookup this message in the source code to see what file you should edit next:

grep -nr "no MPU spells" qemu-2.5.0/
qemu-2.5.0/hw/eos/mpu.c:742:        MPU_EPRINTF("FIXME: no MPU spells for %s.\n", s->model->name);


So we have to edit mpu.c (near line 742). Open it and notice this similarity trick worked on a few other models (such as 1200D or 1100D). Add an entry for EOS M2, using the 100D as model (look for MPU_SPELL_SET_OTHER_CAM).

Looks like it's working! There's a lot of MPU activity now.

Still, the screen is filled with DL messages. Let's look it up in other models - it's already patched on EOS M:

EOSM/debugmsg.gdb:17:# patch DL to avoid DL ERROR messages
EOSM/debugmsg.gdb-18-set *(int*)0xFF1BE4AC = 0xe3a00015


Looks like it might be exactly what we need.

Wait a minute - what's up with these black magic values? The first one is a ROM address (obvious if you managed to follow the walkthrough until here), but what about the second one?

The code near that address, on EOS M, looks like this:

ROM:FF1BE4AC                 BL      sub_FF1BDFD4
ROM:FF1BE4B0                 CMP     R0, #0x15


In other words, it calls a function and checks whether it returns 0x15. Therefore, 0xe3a00015 must be some ARM instruction. You could throw it at a disassembler (tip: there's even an online disassembler on Twitter :) ) or you could find a similar pattern in arm-mcr.h. Whatever method you choose, you'll find out it's MOV R0, #0x15.

Use your pattern matching skills to find the same piece of code on EOS M2 (nearby string: "IsPossibleActiveSweep OVERRUN") and add the patch to EOSM2/debugmsg.gdb.

Run the EOSM2 firmware under GDB and notice it now gets stuck at localI2C_Write. Look it up in the GDB scripts and notice the same thing was (again) patched on EOSM.


EOSM/debugmsg.gdb:20:# patch localI2C_Write to always return 1 (success)
EOSM/debugmsg.gdb-21-set *(int*)0xFF3460C4 = 0xe3a00001
EOSM/debugmsg.gdb-22-set *(int*)0xFF3460C8 = 0xe12fff1e


You may ask: why there are no comments for these black magic values?
1) the ROM addresses have a comment (the name of the function they are referring to).
2) these ARM instructions are obvious to me (how else you could write "return 1" in ARM assembler?)
3) the comment does describe what the patch is doing, what function is patched, what's the meaning of the returned value... what else should I include?

OK, find this one for EOSM2 (easy) and run it again under gdb.

Emulation goes a bit further, and you can spot this message on the last few lines:

    61:   203.008 [STARTUP] ERROR SerialFlash Version!! Firm : 6.0.3 SF : 4.2.1


Look up the string in the disassembly, and also the code that references it:

ROM:FF0C4270                 MOV     R2, #5
ROM:FF0C4274                 ADD     R1, SP, #0x30
ROM:FF0C4278                 ADR     R0, a6_0_3      ; "6.0.3"
ROM:FF0C427C                 BL      sub_FF0C8F40
ROM:FF0C4280                 CMP     R0, #0
ROM:FF0C4284                 BEQ     loc_FF0C42A4

ROM:FF0C4290                 ADR     R3, a6_0_3      ; "6.0.3"
ROM:FF0C4294                 ADR     R2, aErrorSerialfla ; "ERROR SerialFlash Version!! Firm : %s S"...
ROM:FF0C42A0                 BL      DebugMsg
ROM:FF0C42A4
ROM:FF0C42A4 loc_FF0C42A4               


So, to avoid this message, we need sub_FF0C8F40 to return 0. Looking at its arguments, it compares "6.0.3" to something else (something from the stack, likely a local variable), and there's a third argument set to 5. Looking at its code, it appears to do something similar to strncpy. Let's try to bypass this function call and let the caller code behave as if sub_FF0C8F40 would return 0:

# skip SerialFlash version check
set *(int*)0xFF0C427C = 0xe3a00000


We have just replaced the BL sub_FF0C8F40 call with MOV R0, #0.

The emulation goes a lot further now, it initializes RscMgr, FileMgr, PropMgr, starts a bunch of tasks and... gets stuck in an infinite loop at:

[   Startup:ff0c5138 ] (8b:03) Wait LeoLens Complete


Look up the string in the disassembly, and notice the decision for exiting this loop is made here:

ROM:FF0C512C loc_FF0C512C                            ; CODE XREF: sub_FF0C4E80+2CCj
ROM:FF0C512C                 ADR     R2, aWaitLeolensCom ; "Wait LeoLens Complete"
...
ROM:FF0C5144                 LDR     R0, [R4,#0x28]
ROM:FF0C5148                 CMP     R0, #0
ROM:FF0C514C                 BEQ     loc_FF0C512C
...


So, we need R0 to be nonzero in order to avoid the branch that prints "Wait LeoLens Complete". The memory address where R0 is loaded from is R4 + 0x28. What the contents of R4 is supposed to be, is not clear from the disassembly. Hit CTRL-C in gdb and set a breakpoint at FF0C5148 to find out:

(gdb) b *0xFF0C5148
Breakpoint 3 at 0xff0c5148
(gdb) c
Continuing.


Shortly, gdb prompts us again, this time with the program stopped exactly where the loop condition is checked. Examine the registers:

Breakpoint 3, 0xff0c5148 in ?? ()
(gdb) info registers
r0             0x0 0x0
r4             0x8fad4 0x8fad4


So, the memory address the program expects to become nonzero is 0x8fad4+0x28 = 0x8fafc. Look up what other code uses this address (references to either 0x8fad4 - the main data structure - or 0x8fafc - the address of the structure field expected to be updated) and you'll find out this tiny function:

ROM:FF0C35F4                 LDR     R1, =dword_8FAD4
ROM:FF0C35F8                 MOV     R0, #1
ROM:FF0C35FC                 STR     R0, [R1,#(dword_8FAFC - 0x8FAD4)]
ROM:FF0C3600                 BX      LR


OK, so the value at this memory address is expected to become 1 at some point.

Who calls this function? Look it up (references to FF0C35F4) and you'll find this snippet:

ROM:FF0C3B30                 LDR     R0, =loc_FF0C35F4
ROM:FF0C3B34                 MOV     R1, #0
ROM:FF0C3B38                 BL      sub_FF0F7B18
...
ROM:FF0C3B44                 ADRNE   R2, aInitializeprop ; "InitializePropertySatellite (%#x)"


It must be a tiny function passed as pointer to another function (which is why it wasn't recognized as function by IDA - it's not called directly). When it's supposed to be called? No idea, probably from a missing property that's not present in the serial flash dump we took from 100D. Just a guess.

The property code is usually quite complex - in this case it's a lot easier to just patch the affected loop and hope it will work. If the value would have been clearly retrieved from some MMIO register, or set by some interrupt handler, it would probably have been better to emulate that register or that interrupt (because, on the way to that function expected to be called, the code may do a bunch of other initializations, side effects and whatnot).

In this case, let's just patch the value in gdb and see how it goes:

(gdb) set *(int*)($r4 + 0x28) = 1
(gdb) c
Continuing.
[   Startup:ff0c5138 ] (8b:03) Wait LeoLens Complete

Breakpoint 3, 0xff0c5148 in ?? ()
(gdb) c
Continuing.
[   Startup:ff0e29a8 ] (30:03) MOVW_Initialize
[   Startup:0003671c ] task_create(MovWriter, prio=15, stack=2000, entry=36628, arg=67262c)
[   Startup:ff0e6f34 ] (2f:03) MVR_Initialize
[   Startup:0003671c ] task_create(MovieRecorder, prio=11, stack=2000, entry=36628, arg=672d4c)


Looks like the emulation went past that point. Let's add our trick to the gdb script:

# break infinite loop at Wait LeoLens Complete
b *0xFF0C5148
commands
  printf "Patching LeoLens (infinite loop)\n"
  set *(int*)($r4 + 0x28) = 1
  c
end


The emulation goes further, mounts the SD card (CSMgrTask), starts even more tasks, and appears to be stuck at:

   167:   851.456 [TCH]Touch IC Ver : 0x0000


At this point, seeing there's no progress after 10 seconds or so, you'll be tempted to hit CTRL-C. Or you can forget the emulation window open (e.g. for posting gibberish like this on the forum), and when you hit ALT+TAB you notice this:

  1383: 57907.968 WARN [I2C] localI2C_Read : 378 (Task : CEC)
  1384: 57927.680 WARN [I2C] localI2C_Read : 378 (Task : CEC)
  1385: 57947.392 WARN [I2C] localI2C_Read : 378 (Task : CEC)
  1386: 57966.592 WARN [I2C] localI2C_Read : 378 (Task : CEC)




Wait, what?

Trying to navigate this menu gives no reaction. We forgot to address this warning (at the beginning of the emulation log):

[MPU] FIXME: no MPU button codes for EOSM2.


Use the button codes from 100D and - to quote kichetof - take a beer and enjoy it!



Tip: comment out DebugMsg calls from the GDB script for a much faster boot.

Exercises:
- (easy) patch the loop that times out (probably when initializing the touch IC) and the localI2C_Read loop;
- (easy) print Hello World with the minimal ML target;
- (moderate) print Hello World with the regular ML source code;
- (moderate) implement touch screen emulation;
- (moderate) bring ML menu in QEMU;
- (harder) full ML port, with most features working;
- (extreme) LiveView emulation.

yazcui

M2 owner here. All this jargon is way over my head, but I'm thrilled to learn that ML may be available for the M2 soon. Thanks for your efforts!

dfort

I'm trying to follow using the tools I've got available to me. QEMU on a Mac (seems to be working), ARM-console on Mac (sort of half-way works), good ol' disassemble.pl and a fine arts degree (?).

I'm on firmware version 1.0.3 so my numbers don't quite match up to the 1.0.2 dump a1ex is using.

Quote from: a1ex on May 26, 2017, 11:09:39 PM
...One of these covers the functions we are interested in:

[rom1]     at 0xFF0C000C:FFFF09CC [0xFFD1F02C] -> 0xE92D4010
[ram]      at 0xFF0C000C:FFFF09CC [0x00001900] <- 0xE92D4010: was 0x0;
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFD1F030] -> 0xE1A04001
[ram]      at 0xFF0C00A4:FFFF09CC [0x00001904] <- 0xE1A04001: was 0x0;
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFD1F034] -> 0xE59F1040
...
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFDD60C8] -> 0xFFD1EFFC
[ram]      at 0xFF0C00A4:FFFF09CC [0x000B899C] <- 0xFFD1EFFC: was 0x0;


In other words, a memory block from 0xFFD1F02C to 0xFFDD60C8 (actually 0xFFDD60CB, since we are looking at 32-bit operations) is copied to 0x1900 - 0xB899F. The size of the copied block is 0xb70a0 bytes. To extract this block, grab the terminal and run this under a Bash prompt:


dd if=ROM1.BIN of=1900.BIN bs=1 skip=$((0xD1F02C)) count=$((0xB70A0))


(please note the above numbers are only valid for firmware 1.0.2)

Now you can disassemble this file starting from 0x1900. In IDA, load this file as Additional binary file. If you use ARM-console, this should auto-detect the above copying operation (so you can simply start browsing the functions copied into RAM).

This is the same bit in 1.0.3:

[rom1]     at 0xFF0C000C:FFFF09CC [0xFFD1F0E4] -> 0xE92D4010
[ram]      at 0xFF0C000C:FFFF09CC [0x00001900] <- 0xE92D4010: was 0x0;
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFD1F0E8] -> 0xE1A04001
[ram]      at 0xFF0C00A4:FFFF09CC [0x00001904] <- 0xE1A04001: was 0x0;
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFD1F0EC] -> 0xE59F1040
...
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFDD6180] -> 0xFFD1F0B4
[ram]      at 0xFF0C00A4:FFFF09CC [0x000B899C] <- 0xFFD1F0B4: was 0x0;


I see a memory block from 0xFFD1F0E4 to 0xFFDD6180. I'm not sure how you came up with 0xFFDD60CB from 0xFFDD60C8 to make it conform to a 32-bit operation. I tried looking it up but don't quite get it.

Moving on, 0xFFDD6180 - 0xFFD1F0E4 = 0xB709C. It starts at 0x1900 and we need to calculate the end (0x1900 + 0xB709C = 0xB899C) so it is copied to 0x1900 - 0xB899C. The size of the copied block is 0xB709F but it needs to be rounded up to be evenly divisible by 32 (0x20) right? So if I followed the lesson plan properly it is also 0xB70A0 just like the 1.0.2 firmware.

dd if=ROM1.BIN of=EOSM2.103.0x1900.BIN bs=1 skip=$((0xD1F0E4)) count=$((0xB70A0))

That's the name I used for ARM-console to auto-detect. I also did a "perl disassemble.pl 0x1900 EOSM2.103.0x1900.BIN" in order to get a "second opinion" and both methods seem to be in agreement.

[EDIT] Set up a branch to start the port. First step was just to create an EOSM.202 build but call it EOSM2.103.