Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - a1ex

Pages: [1] 2 3 ... 332
1
Camera-specific discussion / Re: ML on EOS-M2
« on: Yesterday at 11:47:06 AM »
The nice part is that all other models can follow (more or less) the same procedure. Credits go to dfort for getting the tutorials ball rolling - that's the reason I've picked this model for a walkthrough.

In other words, owners of other models will no longer have excuses for not knowing how to port ML to their camera :D

Anyway, let's continue figuring out the emulation.

Code: [Select]
        .serial_flash_size      = 0x1000000,

Unfortunately, this step doesn't change anything in the emulation.

Oops, I was wrong here. To quote Ange Albertini - https://twitter.com/angealbertini/status/773650987839926272
Quote
Reverse engineering tip: it's perfectly fine to
- have no idea what to do next
- have made wrong assumptions
- take 'too long'

Here's the detail I've missed the first time:
Code: [Select]
SerialFlash Initialize
[EEPROM-DMA]! [0x2D0000] -> [0x416D5B00] (0x1DC400 bytes)

Also, error messages like this will disappear after declaring .serial_flash_size:
Code: [Select]
     6:    16.640 [PROPAD] ERROR Not Exist Valid ComboPackages!! 0x2D0000

That's good - it probably means we have nothing else to fix regarding serial flash. Running with -d sflash shows plenty of serial flash activity (but I swear I did ran this command back then and noticed none of it...)

Let's move on to the next item - MPU messages. You can find the registers used for these in InitializeIntercom (surrounded in the disassembly by "InitializeIntercom" ... "InitializeIntercom End(%#x)"). The function call between these two strings is:
Code: [Select]
ROM:FF0C3A9C                 LDR     R2, =0xC020302C
ROM:FF0C3AA0                 LDR     R0, =0xC022006C
ROM:FF0C3AA4                 MOV     R1, #0x50 ; 'P'
ROM:FF0C3AA8                 BL      sub_3AD0

Looking up these registers, you'll find the former on the section with DIGIC 5 defaults (so nothing to change about it), and the latter on the EOSM, 100D and some other models. Let's copy it:
Code: [Select]
        .mpu_request_register   = 0xC022006C,
        .mpu_status_register    = 0xC022006C,

Side note: we were very lucky, as EOS M2 is very similar to 100D and other models from the same generation. 1300D was a lot more different (used registers different from all other models, different registers for request and status - unlike all other models - and also different values for driving these GPIOs). Refer to this commit for the gory details - generally you'll find this kind of changes with cameras from different generations. I expect this kind of changes on DIGIC 6 models - didn't look into this part yet because 1) mixed ARM/Thumb code is a lot harder to reverse engineer (at least with the tools I know) and 2) I don't have any MPU communication logs from a DIGIC 6 models (it might work with replaying one from a recent DIGIC 5, no idea).

Still, after enabling the MPU registers, there's very little MPU activity:
Code: [Select]
[MPU] Received: 06 04 02 00 00 00  (unknown spell)
[MPU] Received: 0a 08 03 57 10 2f 00 01 00 00  (unknown spell)
[MPU] Received: 08 07 03 6a 01 08 00 00  (unknown spell)

And these warnings:
Code: [Select]
[MPU] FIXME: no MPU spells for EOSM2.
[MPU] FIXME: no MPU button codes for EOSM2.

Let's try our luck by assuming the MPU communication is similar to 100D. That model already boots Canon GUI, so it looks like a perfect match. Lookup this message in the source code to see what file you should edit next:
Code: [Select]
grep -nr "no MPU spells" qemu-2.5.0/
qemu-2.5.0/hw/eos/mpu.c:742:        MPU_EPRINTF("FIXME: no MPU spells for %s.\n", s->model->name);

So we have to edit mpu.c (near line 742). Open it and notice this similarity trick worked on a few other models (such as 1200D or 1100D). Add an entry for EOS M2, using the 100D as model (look for MPU_SPELL_SET_OTHER_CAM).

Looks like it's working! There's a lot of MPU activity now.

Still, the screen is filled with DL messages. Let's look it up in other models - it's already patched on EOS M:
Code: [Select]
EOSM/debugmsg.gdb:17:# patch DL to avoid DL ERROR messages
EOSM/debugmsg.gdb-18-set *(int*)0xFF1BE4AC = 0xe3a00015

Looks like it might be exactly what we need.

Wait a minute - what's up with these black magic values? The first one is a ROM address (obvious if you managed to follow the walkthrough until here), but what about the second one?

The code near that address, on EOS M, looks like this:
Code: [Select]
ROM:FF1BE4AC                 BL      sub_FF1BDFD4
ROM:FF1BE4B0                 CMP     R0, #0x15

In other words, it calls a function and checks whether it returns 0x15. Therefore, 0xe3a00015 must be some ARM instruction. You could throw it at a disassembler (tip: there's even an online disassembler on Twitter :) ) or you could find a similar pattern in arm-mcr.h. Whatever method you choose, you'll find out it's MOV R0, #0x15.

Use your pattern matching skills to find the same piece of code on EOS M2 (nearby string: "IsPossibleActiveSweep OVERRUN") and add the patch to EOSM2/debugmsg.gdb.

Run the EOSM2 firmware under GDB and notice it now gets stuck at localI2C_Write. Look it up in the GDB scripts and notice the same thing was (again) patched on EOSM.

Code: [Select]
EOSM/debugmsg.gdb:20:# patch localI2C_Write to always return 1 (success)
EOSM/debugmsg.gdb-21-set *(int*)0xFF3460C4 = 0xe3a00001
EOSM/debugmsg.gdb-22-set *(int*)0xFF3460C8 = 0xe12fff1e

You may ask: why there are no comments for these black magic values?
1) the ROM addresses have a comment (the name of the function they are referring to).
2) these ARM instructions are obvious to me (how else you could write "return 1" in ARM assembler?)
3) the comment does describe what the patch is doing, what function is patched, what's the meaning of the returned value... what else should I include?

OK, find this one for EOSM2 (easy) and run it again under gdb.

Emulation goes a bit further, and you can spot this message on the last few lines:
Code: [Select]
    61:   203.008 [STARTUP] ERROR SerialFlash Version!! Firm : 6.0.3 SF : 4.2.1

Look up the string in the disassembly, and also the code that references it:
Code: [Select]
ROM:FF0C4270                 MOV     R2, #5
ROM:FF0C4274                 ADD     R1, SP, #0x30
ROM:FF0C4278                 ADR     R0, a6_0_3      ; "6.0.3"
ROM:FF0C427C                 BL      sub_FF0C8F40
ROM:FF0C4280                 CMP     R0, #0
ROM:FF0C4284                 BEQ     loc_FF0C42A4

ROM:FF0C4290                 ADR     R3, a6_0_3      ; "6.0.3"
ROM:FF0C4294                 ADR     R2, aErrorSerialfla ; "ERROR SerialFlash Version!! Firm : %s S"...
ROM:FF0C42A0                 BL      DebugMsg
ROM:FF0C42A4
ROM:FF0C42A4 loc_FF0C42A4               

So, to avoid this message, we need sub_FF0C8F40 to return 0. Looking at its arguments, it compares "6.0.3" to something else (something from the stack, likely a local variable), and there's a third argument set to 5. Looking at its code, it appears to do something similar to strncpy. Let's try to bypass this function call and let the caller code behave as if sub_FF0C8F40 would return 0:
Code: [Select]
# skip SerialFlash version check
set *(int*)0xFF0C427C = 0xe3a00000

We have just replaced the BL sub_FF0C8F40 call with MOV R0, #0.

The emulation goes a lot further now, it initializes RscMgr, FileMgr, PropMgr, starts a bunch of tasks and... gets stuck in an infinite loop at:
Code: [Select]
[   Startup:ff0c5138 ] (8b:03) Wait LeoLens Complete

Look up the string in the disassembly, and notice the decision for exiting this loop is made here:
Code: [Select]
ROM:FF0C512C loc_FF0C512C                            ; CODE XREF: sub_FF0C4E80+2CCj
ROM:FF0C512C                 ADR     R2, aWaitLeolensCom ; "Wait LeoLens Complete"
...
ROM:FF0C5144                 LDR     R0, [R4,#0x28]
ROM:FF0C5148                 CMP     R0, #0
ROM:FF0C514C                 BEQ     loc_FF0C512C
...

So, we need R0 to be nonzero in order to avoid the branch that prints "Wait LeoLens Complete". The memory address where R0 is loaded from is R4 + 0x28. What the contents of R4 is supposed to be, is not clear from the disassembly. Hit CTRL-C in gdb and set a breakpoint at FF0C5148 to find out:
Code: [Select]
(gdb) b *0xFF0C5148
Breakpoint 3 at 0xff0c5148
(gdb) c
Continuing.

Shortly, gdb prompts us again, this time with the program stopped exactly where the loop condition is checked. Examine the registers:
Code: [Select]
Breakpoint 3, 0xff0c5148 in ?? ()
(gdb) info registers
r0             0x0 0x0
r4             0x8fad4 0x8fad4

So, the memory address the program expects to become nonzero is 0x8fad4+0x28 = 0x8fafc. Look up what other code uses this address (references to either 0x8fad4 - the main data structure - or 0x8fafc - the address of the structure field expected to be updated) and you'll find out this tiny function:
Code: [Select]
ROM:FF0C35F4                 LDR     R1, =dword_8FAD4
ROM:FF0C35F8                 MOV     R0, #1
ROM:FF0C35FC                 STR     R0, [R1,#(dword_8FAFC - 0x8FAD4)]
ROM:FF0C3600                 BX      LR

OK, so the value at this memory address is expected to become 1 at some point.

Who calls this function? Look it up (references to FF0C35F4) and you'll find this snippet:
Code: [Select]
ROM:FF0C3B30                 LDR     R0, =loc_FF0C35F4
ROM:FF0C3B34                 MOV     R1, #0
ROM:FF0C3B38                 BL      sub_FF0F7B18
...
ROM:FF0C3B44                 ADRNE   R2, aInitializeprop ; "InitializePropertySatellite (%#x)"

It must be a tiny function passed as pointer to another function (which is why it wasn't recognized as function by IDA - it's not called directly). When it's supposed to be called? No idea, probably from a missing property that's not present in the serial flash dump we took from 100D. Just a guess.

The property code is usually quite complex - in this case it's a lot easier to just patch the affected loop and hope it will work. If the value would have been clearly retrieved from some MMIO register, or set by some interrupt handler, it would probably have been better to emulate that register or that interrupt (because, on the way to that function expected to be called, the code may do a bunch of other initializations, side effects and whatnot).

In this case, let's just patch the value in gdb and see how it goes:
Code: [Select]
(gdb) set *(int*)($r4 + 0x28) = 1
(gdb) c
Continuing.
[   Startup:ff0c5138 ] (8b:03) Wait LeoLens Complete

Breakpoint 3, 0xff0c5148 in ?? ()
(gdb) c
Continuing.
[   Startup:ff0e29a8 ] (30:03) MOVW_Initialize
[   Startup:0003671c ] task_create(MovWriter, prio=15, stack=2000, entry=36628, arg=67262c)
[   Startup:ff0e6f34 ] (2f:03) MVR_Initialize
[   Startup:0003671c ] task_create(MovieRecorder, prio=11, stack=2000, entry=36628, arg=672d4c)

Looks like the emulation went past that point. Let's add our trick to the gdb script:
Code: [Select]
# break infinite loop at Wait LeoLens Complete
b *0xFF0C5148
commands
  printf "Patching LeoLens (infinite loop)\n"
  set *(int*)($r4 + 0x28) = 1
  c
end

The emulation goes further, mounts the SD card (CSMgrTask), starts even more tasks, and appears to be stuck at:
Code: [Select]
   167:   851.456 [TCH]Touch IC Ver : 0x0000

At this point, seeing there's no progress after 10 seconds or so, you'll be tempted to hit CTRL-C. Or you can forget the emulation window open (e.g. for posting gibberish like this on the forum), and when you hit ALT+TAB you notice this:
Code: [Select]
  1383: 57907.968 WARN [I2C] localI2C_Read : 378 (Task : CEC)
  1384: 57927.680 WARN [I2C] localI2C_Read : 378 (Task : CEC)
  1385: 57947.392 WARN [I2C] localI2C_Read : 378 (Task : CEC)
  1386: 57966.592 WARN [I2C] localI2C_Read : 378 (Task : CEC)



Wait, what?

Trying to navigate this menu gives no reaction. We forgot to address this warning (at the beginning of the emulation log):
Code: [Select]
[MPU] FIXME: no MPU button codes for EOSM2.

Use the button codes from 100D and - to quote kichetof - take a beer and enjoy it!



Tip: comment out DebugMsg calls from the GDB script for a much faster boot.

Exercises:
- (easy) patch the loop that times out (probably when initializing the touch IC) and the localI2C_Read loop;
- (easy) print Hello World with the minimal ML target;
- (moderate) print Hello World with the regular ML source code;
- (moderate) implement touch screen emulation;
- (moderate) bring ML menu in QEMU;
- (harder) full ML port, with most features working;
- (extreme) LiveView emulation.

2
The results of readelf ... grep commands were the same on both Mac and Cygwin?

As a workaround, you can manually delete these symbols from "localsyms", then run "make" again. The localsyms file will not be re-created (unless you also edit some TCC sources), but libtccx.o (which is the one used by ML) will be re-created using your modified localsyms.

3
General Help Q&A / Re: Detect shutter action in Lua
« on: May 28, 2017, 10:42:19 PM »
An image triggered from Lua or taken by the user?

I have the first case implemented locally (with an API test that looks whether the CR2 or JPG was saved). Second case is available in ML core, just has to be exposed to Lua somehow.

4
Okay, tcc_new and the other missing symbols should *not* be in localsyms, but for some reason, they are. You could try running the localsyms rule from Makefile manually (maybe step by step) to see where it fails. Outside Makefile, it would look somewhat like this:
Code: [Select]
readelf libtcctmp.o -Ws | awk "{print \$8}" | sort | uniq | grep -v tcc_new

The output should contain all other symbols, except tcc_new.

5
Some recent additions (some of them were covered in other threads):

* Cleaned up the self-test log to include installation instructions (the self-test begins with reinstalling QEMU from scratch, so you can also use it to check the expected output of the commands etc).

* Call stack reconstruction (inspired from Panda's callstack_instr, but heavily customized for EOS firmware). This makes the call stack available to other analysis tools (e.g. when memcheck wants to print an error). Enable with -d callstack . Works on DIGIC 4 to 6 and Eeko; almost there on DIGIC 2 and 3; for best results, .current_task_addr should also be defined (because each task has its own stack).

* Call/return trace. Every function call and return (that could be identified automatically) is logged on the console, including arguments and return values (enable with -d calls ). Works best on DIGIC 4 to 6 and Eeko, almost there on DIGIC 2 and 3. Interrupts are also handled (though it still has some trouble with corner cases). Example for Eeko.

Tip: you can load these indented logs in your text editor, configure it for Python source code, and you've got collapse buttons :)

* Task switches. Make sure you have .current_task_addr defined in model_list.c and enable with -d tasks . Works on DIGIC 2 to 7 and Eeko. Example for EOS M2.

* Memory blocks copied from ROM to RAM can be identified automatically (-d romcpy). They are listed on the self-test log, and they should be a big time-saver when setting up the disassembler (usually you'll need to know what are the blocks worth disassembling, besides the ROM). Works on DIGIC 2 to 7. Example for EOS M5 (can be cross-checked with values found manually):
Code: [Select]
[ROMCPY] 0xE001AF2C -> 0xDF020000 size 0x3C0      at 0xE0005AA8
[ROMCPY] 0xE001B2E4 -> 0x4000     size 0xF1C      at 0xE000492C
[ROMCPY] 0xE115CF88 -> 0x8000     size 0x6054C    at 0xE002003C
[ROMCPY] 0xE11BD4D4 -> 0x1900000  size 0x1444     at 0xE0020060
[ROMCPY] 0xE11BE918 -> 0xDFFC4900 size 0x152A0    at 0xE0020084

* Symbol names from .elf files. QEMU has this feature built in, but had to customize it a bit (for example, it didn't recognize our stubs as function calls, because our build system doesn't mark them as such). Usage (requires bash):
Code: [Select]
. ./export_ml_syms.sh 550D.109
./run_canon_fw.sh 550D.109,firmware="boot=1" ...
...
Task switch to init:ff010470                                                     at [init:ff0164c4:ff076f18]
call 0x8B010 my_init_task(0, 8b010, 19980218, 19980218)                          at [init:ff0771dc:0]
 call 0xFF018D1C init_task(0, 8b010, 19980218, 19980218)                         at [init:8b014:ff0771e0] (my_init_task)
  call 0xFF0108FC(0, 8b010, 19980218, 19980218)                                  at [init:ff018d20:8b018]

In other words, if you want the log to use the function names you have annotated in the disassembly, just add them to stubs.S and compile ML; the above script will load them. If you don't want to run autoexec.bin, you can probably just load stubs.o (which is an elf file), or you can probably create a stubs file just for analysis (one that will not be included in ML, because it will usually have a lot more names than you actually want to call).

There's a lot of room for improvement on this one (for example, right now function names are only listed in the call/return trace). Adding this info in more places is generally an easy coding task - find what parts of the log you would like to have function names rather than just raw addresses, and change the output strings accordingly). Automating symbol name export from your favorite disassembler should also be easy (and in many cases, the code is available online from other open source tools; just needs to be integrated here).

* Memcheck warnings closer to valgrind's (e.g. show where the affected memory block was allocated or freed).

* Basic ABI checking (e.g. R4-R11 and SP should be restored after function call). This is done while creating the call stack (-d callstack).

* The auto-generated IDC file should be much more accurate (because we finally have a sane call/return trace).

A walkthrough for some of these features can be found in the EOS M2 topic, where, with dfort's help, I'd like to show in detail how to port Magic Lantern on a new camera from scratch.

Tip: to find all these analysis options, run QEMU with -d help. The ones interesting for us are:
Code: [Select]
int        show interrupts/exceptions in short format
exec       show trace before each executed TB (lots of logs)
nochain    do not chain compiled TBs so that "exec" and "cpu" show complete traces
io         EOS: log low-level I/O activity
mpu        EOS: log low-level MPU activity
sflash     EOS: log low-level serial flash activity
sdcf       EOS: log low-level SD/CF activity
uart       EOS: log low-level UART activity
ram        EOS: log all RAM reads and writes
rom        EOS: log all ROM reads and writes
ramr       EOS: log all RAM reads
romr       EOS: log all ROM reads
ramw       EOS: log all RAM writes
romw       EOS: log all ROM writes
ram_dbg    EOS: self-test for the RAM logging routines
callstack  EOS: reconstruct call stack (implies nochain,singlestep)
calls      EOS: log function calls (implies callstack,ramr,nochain,singlestep)
idc        EOS: export called functions to IDA (implies callstack,nochain,singlestep)
tasks      EOS: log task switches (implies callstack,nochain,singlestep)
romcpy     EOS: find memory blocks copied from ROM to RAM
memchk     EOS: check memory usage (malloc/free, uninitialized values)
v         
verbose    EOS: very detailed debug messages

Happy reversing!

6
Camera-specific discussion / Re: ML on EOS-M2
« on: May 28, 2017, 03:13:05 PM »
There are other similar blocks copied into RAM (newer DIGIC 6 and 7 models have plenty of those), and finding them manually can become tedious, so it's a good candidate for automation.

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d romcpy |& grep ROMCPY
[ROMCPY] 0xFFFF0000 -> 0x0        size 0x40       at 0xFFFF0980
[ROMCPY] 0xFFFE0000 -> 0x100000   size 0xFF2C     at 0xFFFF0FCC
[ROMCPY] 0xFFD1F02C -> 0x1900     size 0xB70A0    at 0xFF0C000C
[ROMCPY] 0xFF0C0E04 -> 0x4B0      size 0x1E8      at 0xFF0C0D70
[ROMCPY] 0xFFA12904 -> 0x4E0E98   size 0xC7C      at 0x8645C   

This should work on all models from DIGIC 2 to DIGIC 7 (both EOS and PowerShot).

We now have task information and debug messages!

... and also an option to show all context switches:
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d tasks
...
Task switch to idle:ff0c108c                                                     at [idle:1db8:ff0c1c84]
Task switch to init:ff0c1064                                                     at [init:1a18:c73c]
K355 READY
128K Sector FROM From BL 0xffff
...
SerialFlash Initialize
Task switch to Startup:ff0c108c                                                  at [Startup:1db8:c9e0]
Task switch to Startup2:ff0c1064                                                 at [Startup2:1a18:228c]
...
Task switch to DbgMgr:ff0c1064                                                   at [DbgMgr:1a18:e24c]
     4:    54.784 [PROPAD] PROPAD_CreateFROMPropertyHandle DRAMAddr 0x416d5b00
Task switch to Startup:ff0c1064                                                  at [Startup:1a18:2e40]
[RTC] !! RTC_TIME_CORRECT_CHANGE!  0x0 ---> 0x9a
[RTC] !! RTC UNDER MIN_YEAR !!
Task switch to PropMgr:ff0c1064                                                  at [PropMgr:1a18:da2c]
...

8)

7
Hm, no error here...

Code: [Select]
cat localsyms
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-readelf libtcctmp.o -Ws
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-readelf libtccx.o -Ws

8
Try:
Code: [Select]
cd tcc   # from magic-lantern root dir
make clean
make

9
Camera-specific discussion / Re: Canon 100D / SL1 (Beta-4a)
« on: May 27, 2017, 05:38:34 PM »
It's probably related to ML's exposure simulation (which configures the LiveView at much higher brightness than Canon firmware, and the difference is visible at very long exposure times with very dark scenes - including BULB mode).

Can probably be fixed by decreasing MAX_ISO_BV. Maybe also worth including it in selftest.mo somehow.

10
Step 1: expose the audio trigger functionality to Lua.
Step 2: write a script that allows all these options.

Currently, you can enable autofocus from the shoot preferences menu, but without delays and so on. Autofocus functionality is already available in Lua (on the experimental build).

11
Feature Requests / Re: draw enhancements
« on: May 27, 2017, 10:15:10 AM »
The CHDK circle approximation is a mystery to me as well (and there were comments saying these circles are not even very round).

CHDK also has a draw_ellipse function based on Bresenham algorithm - this one should be more accurate, and probably easier to start from in order to implement an arc function. Still, it only computes one quadrant and draws all the others, which makes it hard to turn it into a generic draw_arc without understanding the math behind it.

Of course, you can use plain sin/cos, but I bet a naive implementation will likely give ugly results (overlapped and/or missing pixels, unless you approximate it with lines, and even in this case it's tricky to get the pixel mapping right). And it will be a lot slower.

I believe it's best to get this function from some embedded GUI library, rather than reinventing the wheel. At a quick search, I've found uGUI, which has DrawArc, but it's only limited to 90-degree arcs (any of the 4 quadrants).

12
Full log?

Best guess: I'm using a trick to strip most TCC symbols - everything but the API we use - in tcc/Makefile near localsyms. This file should contain all TCC's symbols except those listed in the error message. Some of the tools used to create this file might not be available on Windows.

13
Camera-specific discussion / Re: ML on EOS-M2
« on: May 26, 2017, 11:09:39 PM »
Alright, so let's start from where we've left off.

0xFF0C1BD4 is cstart, 0x866B4 is bzero32, 0x3168 must be create_init_task and 0xFF0C57A4 must be init_task. Some of these functions are called from RAM, so we'll have to identify where its contents come from (what is copied where). TODO.

Since the ROMs are placed at some high address (after 0xF0000000), and we've noticed some functions called outside this range, a good guess would be that some of the functions may be copied from ROM to RAM during startup. Let's examine the memory access patterns. We are interested in some code that reads from ROM and writes to RAM. If you have no trouble following the assembly code for the startup sequence, you'll spot this piece of code right away. Otherwise, let's ask for some help from QEMU:

Running QEMU with -d help gives, among many other options:
Code: [Select]
ram        EOS: log all RAM reads and writes
rom        EOS: log all ROM reads and writes
ramr       EOS: log all RAM reads
romr       EOS: log all ROM reads
ramw       EOS: log all RAM writes
romw       EOS: log all ROM writes

We could try romr,ramw (or rom,ram if you don't mind a little more verbosity):
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d romr,ramw

The log is huge, but if you scroll around, there are some large blocks that look similar. Some of them are clearly a copying operation (read some value from ROM, write it to RAM). One of these covers the functions we are interested in:
Code: [Select]
[rom1]     at 0xFF0C000C:FFFF09CC [0xFFD1F02C] -> 0xE92D4010
[ram]      at 0xFF0C000C:FFFF09CC [0x00001900] <- 0xE92D4010: was 0x0;
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFD1F030] -> 0xE1A04001
[ram]      at 0xFF0C00A4:FFFF09CC [0x00001904] <- 0xE1A04001: was 0x0;
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFD1F034] -> 0xE59F1040
...
[rom1]     at 0xFF0C00A4:FFFF09CC [0xFFDD60C8] -> 0xFFD1EFFC
[ram]      at 0xFF0C00A4:FFFF09CC [0x000B899C] <- 0xFFD1EFFC: was 0x0;

In other words, a memory block from 0xFFD1F02C to 0xFFDD60C8 (actually 0xFFDD60CB, since we are looking at 32-bit operations) is copied to 0x1900 - 0xB899F. The size of the copied block is 0xb70a0 bytes. To extract this block, grab the terminal and run this under a Bash prompt:

Code: [Select]
dd if=ROM1.BIN of=1900.BIN bs=1 skip=$((0xD1F02C)) count=$((0xB70A0))

(please note the above numbers are only valid for firmware 1.0.2)

Now you can disassemble this file starting from 0x1900. In IDA, load this file as Additional binary file. If you use ARM-console, this should auto-detect the above copying operation (so you can simply start browsing the functions copied into RAM).

There are other similar blocks copied into RAM (newer DIGIC 6 and 7 models have plenty of those), and finding them manually can become tedious, so it's a good candidate for automation.

One of these blocks is the interrupt handler routine. On ARM, hardware interrupts are executed from address 0x18 (see e.g. ARM ARM - A2.6 - Exceptions or these slides). That means, we should expect to find some code that writes something to address 0x18 (that can be anywhere in the boot process before enabling interrupts), and on EOS firmwares, at this address you'll find a jump to 0x4B0 on many models, or something close on others. Let's examine it in GDB:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S & arm-none-eabi-gdb -x debug-logging.gdb
(note: we will need to create a GDB file for EOS M2, named EOSM2/debugmsg.gdb to match other models, but since we don't have one yet, we can just use the generic version - which we are going to include in the platform-specific file later)

Let it run for a while (continue), then hit CTRL-C and examine the exception vector (located at address 0):

Code: [Select]
(gdb) disas 0,0x40
Dump of assembler code from 0x0 to 0x40:
   0x00000000: nop ; (mov r0, r0)
   0x00000004: ldr pc, [pc, #20] ; 0x20
   0x00000008: ldr pc, [pc, #20] ; 0x24
   0x0000000c: ldr pc, [pc, #20] ; 0x28
   0x00000010: ldr pc, [pc, #20] ; 0x2c
   0x00000014: nop ; (mov r0, r0)
   0x00000018: ldr pc, [pc, #16] ; 0x30
   0x0000001c: ldr pc, [pc, #16] ; 0x34
   0x00000020: ; <UNDEFINED> instruction: 0xff0c0fec
   0x00000024: ; <UNDEFINED> instruction: 0xff0c105c
   0x00000028: ; <UNDEFINED> instruction: 0xff0c1000
   0x0000002c: ; <UNDEFINED> instruction: 0xff0c1018
   0x00000030: ; <UNDEFINED> instruction: 0x000004b0
   0x00000034: ; <UNDEFINED> instruction: 0xff0c1060
   0x00000038: ; <UNDEFINED> instruction: 0xffff0b08
   0x0000003c: ; <UNDEFINED> instruction: 0xffff0b20

That means, once a hardware interrupt happens (in other words, some device wants to tell the main CPU that it needs attention), the program counter will jump to 0x18, and there it will find a LDR PC, =0x4B0. That's where Canon's interrupt handler is located. Let's find its contents:

- Method 1: scan the memory access log for a copy operation that covers 0x4B0 and above. Easy to find.

- Method 2: disassemble it directly from GDB. How much? 0x1900 - 0x4B0 would be the upper limit (since we already know what's at 0x1900). The interrupt handler is much smaller than that - 512 bytes are more than enough:
Code: [Select]
disas 0x4B0,0x6B0

Method 3: dump the memory from gdb:
Code: [Select]
(gdb) dump memory 4B0.BIN 0x4B0 0x6B0

Method 4: dump the memory from QEMU monitor:
Code: [Select]
(qemu) memsave 0x4B0 0x200 4B0.BIN

For the debugmsg.gdb file, we'll need some basic stubs: DebugMsg and task_create (easy to find - one is used for most debug messages, the other can be easily identified from things that look like task names) and two addresses: CURRENT_TASK and CURRENT_ISR (interrupt service routine). We can find the latter in the interrupt handler - quote from debugmsg.gdb:
Quote
# CURRENT_ISR:
#   From interrupt handler (PC=0x18), find an expression that evaluates to
#   the current interrupt ID if any is running, or 0 if a normal task is running.
#   - on DIGIC 4/5, interrupt ID is MEM(0xC0201004) >> 2
#   - on DIGIC 6,   interrupt ID is MEM(0xD4011000)

If you can find that by examining disassembled code, great. If not, QEMU to the rescue:
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d io,int,nochain -singlestep
...
Taking exception 5 [IRQ]
[INT]      at 0x00000510:0000050C [0xC0201004] -> 0x28      : Requested int reason 28 (INT 0Ah)
[INT]      at 0x00000594:00000568 [0xC0201010] <- 0xA       : Enabled interrupt 0Ah

(note: -d nochain -singlestep gives more precise locations in I/O logs - otherwise, QEMU groups the ARM instructions in blocks for faster execution, so the reported location will be approximate)

OK, so the register we are looking for is read at 0x510. Let's disassemble with GDB:
Code: [Select]
(gdb) disas 0x4B0,0x6B0
...
   0x0000050c: ldr r0, [pc, #368] ; 0x684
   0x00000510: ldr r4, [r0]
   0x00000514: str r4, [pc, #304] ; 0x64c
...
(gdb) x 0x684
0x684: 0xc0201004
(gdb) x 0x64c
0x64c: 0x00000028
(gdb) print 0x28>>2
$1 = 0xa

The register we are looking for is loaded in R0, then the interrupt handler reads from this register (result in R4) and stores its value at 0x64C. If you check the memory contents at this address, you'll find 0x28. The interrupt ID would be 0x28 >> 2 = 0x0A, which is DryOS timer interrupt (which fires every 10ms and is used for the task switch - see e.g. FreeRTOS Tick for some background info).

Note: ARM has a generic hardware interrupt (PC jumping at 0x18), but in an embedded system we usually have more devices that can trigger an interrupt. To distinguish between them, it's helpful to identify them somehow - on EOS, there is a custom interrupt controller, and on DIGIC 4 and 5 models, the register used for identifying the interrupt ID is 0xC0201004. The interrupt ID is hardwired to the device that triggers it (for example, SIO3 and MREQ, which are used for MPU communication, always use interrupts 0x36 and 0x50). This mapping can change across models, but usually it's consistent (at least within the same generation of models). Probably g3gg0 can explain these concepts a bit better, as he identified all this stuff a few years ago, when I was mostly clueless about how these things work.

OK, so the memory contents at 0x64C can tell us about the interrupt currently handled. There's a problem - this address is not cleared when the interrupt handler is done, so, just by looking at it, we can't tell whether the code is servicing an interrupt or running a regular task. We'll need to look at some other address - and in many cases the previous address is a good candidate:
Code: [Select]
   0x000004d4: ldr r1, [pc, #364] ; 0x648
   0x000004d8: cmp r1, #0
   0x000004dc: add r1, r1, #1
   0x000004e0: str r1, [pc, #352] ; 0x648
   ...
   0x00000598: ldr r1, [pc, #168] ; 0x648
   0x0000059c: sub r1, r1, #1
   0x000005a0: str r1, [pc, #160] ; 0x648

This 0x648 looks like a counter that tells how many nested interrupts we are handling (yes, they can be nested - unfortunately, as this makes it a lot harder to understand, debug, emulate and so on). Let's confirm its functionality with RAM tracing:
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d io,int,ram,nochain -singlestep
...
[tcm_code] at 0x000004D4:19980218 [0x00000648] -> 0x0       
[tcm_code] at 0x000004E0:19980218 [0x00000648] <- 0x1       : was 0x0;
...
[tcm_code] at 0x0000050C:0000050C [0x00000684] -> 0xC0201004
[INT]      at 0x00000510:0000050C [0xC0201004] -> 0x28      : Requested int reason 28 (INT 0Ah)
[tcm_code] at 0x00000514:0000050C [0x0000064C] <- 0x28      : was 0x0;
...
[INT]      at 0x00000594:00000568 [0xC0201010] <- 0xA       : Enabled interrupt 0Ah
[tcm_code] at 0x00000598:00000568 [0x00000648] -> 0x1       
[tcm_code] at 0x000005A0:00000568 [0x00000648] <- 0x0       : was 0x1;

Looks right!

Note: "Enabled interrupt 0Ah" means the interrupt routine finished handling it, so the same interrupt can be triggered again from now on (you can't have the same interrupt nested with itself). The same register configuration happens when an interrupt is enabled for the first time, and that's why the message reads "enabled interrupt". Might be a bit confusing at first sight.

Now that we know what 0x648 does, let's use it as boolean (0 = regular task, nonzero = handling some interrupt):
Code: [Select]
macro define CURRENT_ISR  (*(int*)0x648 ? (*(int*)0x64C) >> 2 : 0)

Next is CURRENT_TASK. One way to find that is by pattern matching in task_create and its subroutines.

I'll try an alternate method, based on one of the recent addition to QEMU: the ability to track function calls and returns. From the ARM calling convention (AAPCS, or the summary from wikipedia), you'll notice that registers R4-R11 are supposed to be unchanged after a function call. When tracking function calls, the logging code checks this assumption as well, and will print lots of warnings if a task switch happens without the logger knowing about it. Let's use these warnings to narrow down the location where DryOS switches between tasks:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d callstack,ramw

The log is huge, but we are going to look for some warnings about registers (run with only -d callstack to see them without the clutter from RAM writes). These warnings are likely caused by some memory writes performed just before the warnings. We are looking for some invariant address (one that doesn't change with the warnings):
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d callstack,ramw |& grep -B 20 "not restored"
...
[ram]      at 0x000019D8:0000E24C [0x0018F8FC] <- 0xE24C    : was 0xE24C;
[ram]      at 0x00001A04:0000E24C [0x0008FBCC] <- 0x17EED4  : was 0x17EF28;
[ram]      at 0xFF0C1064:0000E24C [0x0018F8FC] <- 0xE24C    : was 0xE24C;
...
[ram]      at 0xFF0C1074:0000E24C [0x0018F8C0] <- 0x60000093: was 0x19980218;
[ram]      at 0xFF0C1078:0000E24C [0x0017EF78] <- 0x18F8C0  : was 0x18F8D0;
Warning: R4 not restored (0x17ef28 -> 0x17eed4)                                  at [ff0c1088:e24c]
...
[ram]      at 0x000019D8:0000E24C [0x0018E9A4] <- 0xE24C    : was 0xE24C;
[ram]      at 0x00001A04:0000E24C [0x0008FBCC] <- 0x17EF28  : was 0x17EED4;
[ram]      at 0xFF0C1064:0000E24C [0x0018E9A4] <- 0xE24C    : was 0xE24C;
...
[ram]      at 0xFF0C1074:0000E24C [0x0018E968] <- 0x60000093: was 0x18E9FC;
[ram]      at 0xFF0C1078:0000E24C [0x0017EF24] <- 0x18E968  : was 0x18E970;
Warning: R4 not restored (0x17eed4 -> 0x17ef28)                                  at [ff0c1088:e24c]

Most of these addresses are different for each warning message, except one. Let's try it. Find task_create and DebugMsg (easy) and add everything to EOSM2/debugmsg.gdb (look at other cameras for a template):
Code: [Select]
# ./run_canon_fw.sh EOSM2 -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb

source -v debug-logging.gdb

macro define CURRENT_TASK 0x8FBCC
macro define CURRENT_ISR  (*(int*)0x648 ? (*(int*)0x64C) >> 2 : 0)

b *0x4398
DebugMsg_log

b *0x7360
task_create_log

Code: [Select]
./run_canon_fw.sh EOSM2 -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb
...
[      init:ff352264 ] task_create(PowerMgr, prio=20, stack=400, entry=ff352088, arg=0)
[      init:ff1470d4 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[      init:0003671c ] task_create(DbgMgr, prio=1f, stack=0, entry=36628, arg=46e584)
[      init:ff0c3334 ] (8b:16)
K355 ICU Firmware Version 1.0.2 ( 6.0.5 )
[      init:ff0c3348 ] (8b:05)
ICU Release DateTime 2013.12.02 09:28:54
[      init:ff0f5ea0 ] (00:03) [SEQ] CreateSequencer (Startup, Num = 6)
[      init:ff0f5f28 ] task_create(Startup, prio=19, stack=2800, entry=ff0f5d6c, arg=46e880)
[      init:ff0f60f4 ] (00:02) [SEQ] NotifyComplete (Startup, Flag = 0x10000)
[      init:ff0f6158 ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x10000, Flag = 0x10000)
[      init:ff0c33d4 ] task_create(TaskMain, prio=1d, stack=0, entry=ff0c28d8, arg=0)
[   Startup:ff0f5db4 ] (00:05) [SEQ] seqEventDispatch (Startup, 0)
[   Startup:ff0c373c ] (8b:05) startupEntry
[   Startup:ff0c375c ] task_create(Startup2, prio=11, stack=400, entry=ff0c3604, arg=0)
[  Startup2:ff1310c4 ] (02:16) PROPAD_CreateFROMPropertyHandle DRAMAddr 0x416d5b00
[  Startup2:ff14a204 ] (00:01) [SF] IsAddressSerialFlash 0x2d0000
...

We now have task information and debug messages!

To make this information available in QEMU (not just in GDB), add this to model_list.c under the EOSM2 section:
Code: [Select]
        .current_task_addr      = 0x8FBCC,

That's it for today.

14
Camera-specific discussion / Re: ML on EOS-M2
« on: May 25, 2017, 08:40:40 AM »
You need two *equal* halves, without any approximation.

(edit: updated QEMU to give better warnings in this case)

SFDUMP.BIN can probably be used from 700D or EOSM as well, but these cameras have a smaller size. Padding it with zeros may or may not work. How to get it? It's covered in QEMU's install script.

15
Camera-specific discussion / Re: ML on EOS-M2
« on: May 24, 2017, 11:15:20 PM »
Split it in two halves: first ROM0, second ROM1.

Back then, it was much easier to patch Canon code to dump everything in one file, so it covers both ROMs. For this reason,
older QEMU versions used to require the ROMs to be joined like this (but it was changed to allow using the backup ROM copies already saved by ML on the cards under ML/LOGS, and to allow more unusual ROM configurations).

16
Camera-specific discussion / Re: ML on EOS-M2
« on: May 24, 2017, 09:23:14 PM »
The dumper linked above runs on the main firmware (in contrast with the portable ROM dumper, which runs from bootloader), so it has no issues with card sizes, but it's limited to DIGIC 4 and 5 models. Does it help if you follow the instructions in the linked post? (it's different from the portable dumper you are used to)

You can start from the latest firmware.

17
Camera-specific discussion / Re: ML on EOS-M2
« on: May 24, 2017, 12:17:11 PM »
I'll start with a walkthrough using latest QEMU and IDA. First step is to assume it's very similar to EOSM, so after installing QEMU, head over to qemu/qemu-2.5.0/hw/eos/model_list.c and copy the basic entries from the section for EOSM:

Code: [Select]
    {
        .name                   = "EOSM2",
        .digic_version          = 5,
     }

The others are likely model-specific, so let's not assume too much from the beginning. Let's try:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0"
...
FFFF3F50: MCR p15,0,Rd,cr6,cr0,0:  946_PRBS0 <- 0x3F       (00000000 - FFFFFFFF, 0x100000000)
FFFF3F58: MCR p15,0,Rd,cr6,cr1,0:  946_PRBS1 <- 0x3D       (00000000 - 7FFFFFFF, 0x80000000)
FFFF3F60: MCR p15,0,Rd,cr6,cr2,0:  946_PRBS2 <- 0xE0000039 (E0000000 - FFFFFFFF, 0x20000000)
FFFF3F68: MCR p15,0,Rd,cr6,cr3,0:  946_PRBS3 <- 0xC0000039 (C0000000 - DFFFFFFF, 0x20000000)
FFFF3F70: MCR p15,0,Rd,cr6,cr4,0:  946_PRBS4 <- 0xFF00002F (FF000000 - FFFFFFFF, 0x1000000)
FFFF3F78: MCR p15,0,Rd,cr6,cr5,0:  946_PRBS5 <- 0x37       (00000000 - 0FFFFFFF, 0x10000000)
FFFF3F80: MCR p15,0,Rd,cr6,cr6,0:  946_PRBS6 <- 0xF700002F (F7000000 - F7FFFFFF, 0x1000000)
...
K355 READY
128K Sector FROM From BL 0xffff
[SF] InstallSerialFlash 6 0xc022c0d4 0x0 0x1000000 1
...
K355 ICU Firmware Version 1.0.2 ( 6.0.5 )

OK, so we've got a rough idea about its memory map (same as other DIGIC 4 models). We also noticed it has a serial flash of size 0x1000000. This is similar to 100D (lookup serial_flash_size in model_list.c), so it may be a good idea to reuse the serial flash data from this model:
Code: [Select]
        .serial_flash_size      = 0x1000000,

Unfortunately, this step doesn't change anything in the emulation. Let's look at the next messages:
Code: [Select]
...
    10:    46.336 [PROPAD] ERROR Not Exist Valid ComboPackages!! 0x10000
...
    14:    56.576 [STARTUP] InitializeIntercom
...
    42:   722.688 ERROR [DL] ########## DL ERROR!!!!! ###########

- Many errors regarding missing properties. These are either stored in ROM, or in the serial flash, or they come from the MPU. The ones from ROM should not pose any problems, since we have the complete ROM. For the others, we did not see any sign of activity, so we have to dig in to find out whether anything is different (usually these communication channels may use different registers, depending on model, but overall the communication protocol is the same).

- DL ERROR: if you look in EOSM/debugmsg.gdb, you'll find a way to get past this message (by skipping the initialization of "DL", whatever that is).

- The debug messages are not very informative. To get more insights, find out a few essential functions (such as task_create and DebugMsg) and place them in EOSM2/debugmsg.gdb (look at other models for how the declarations should look like).

OK, so at this point we have to disassemble the firmware. If you use IDA, the initial steps would be:
- load ROM1.BIN
- select ARM processor, architecture ARMv5TEJ (we have ARMv5TE on all models from DIGIC 2 to DIGIC 5)
- uncheck 'Delete instructions with no xrefs' (we have plenty of those)
- uncheck 'Perform no-return analysis' (IDA usually gets it wrong)
- RAM section: 0, size 0x100000 (we can always add more later)
- ROM start address: 0xFF000000
- ROM size: 0xFFFFFC (off-by-one bug in IDA; I have an older version, so YMMV)
- File load address / size: same as ROM.

Now that our binary is loaded, go to 0xFFFF0000 and press "c" to mark this as code. This is the bootloader start address (aka HIVECS in ARM documentation). Next step is to find where the main firmware starts (where the code path leaves the bootloader). It's easy to assume similarity with other models, but let's try to find it from scratch (just to show off the latest QEMU tool):

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d calls
...
call 0xFFFF0FCC(0, ffffdfff, 2000, ffff0b20)                                     at [ffff09c8:0]
 call 0x100000(0, 11836c, 11836c, 100000)                                        at [ffff1010:ffff09cc]
...

OK, so the bootloader runs from RAM.

Where does the bootloader end? When it stops executing from its address range (near 0xFFFF0000 or 0x100000):

Code: [Select]
  return 0 to 0x10013C                                                           at [1022f4:ffff1014]
 return 1 to 0xFFFF1014                                                          at [1000bc:ffff09cc]
return 0 to 0xFFFF09CC                                                           at [ffff101c:0]
Warning: R10 not restored (0xa -> 0x1)                                           at [ffff101c:ffff09cc]
PC jump? 0xF80C0000 lr=ffff09cc                                                  at [ffff0a04:ffff09cc]
0xffff0a04:  e1a0f000      mov pc, r0
PC jump? 0xFF0C000C lr=ffff09cc                                                  at [f80c0000:ffff09cc]
0xf80c0000:  e59ff0c4      ldr pc, [pc, #196] ; 0xfffffffff80c00cc

Ta-da! Same as other DIGIC 5 models: ROM1 startup code is at 0xF80C0000, but code from main firmware expects to run from a mirror copy (details). So, our ROMBASEADDR is 0xFF0C0000.

Let's go there in IDA and mark this section as code. At this point, IDA already recognized a few functions, but let's get some more from the QEMU execution trace:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d idc
...
EOSM2.idc saved.

Now load this IDC script into IDA, or convert it for your favorite disassembler, and start looking at the functions called during the execution in QEMU:
Code: [Select]
PC jump? 0xFF0C000C lr=ffff09cc                                                  at [f80c0000:ffff09cc]
...
call 0xFF0C1BD4(1000, 698, eeeeeeee, 1000)                                       at [ff0c0dbc:0]
 call 0x866B4(f88, 74, eeeeeeee, 1000)                                           at [ff0c1be4:ff0c0dc0]
 return ffc to 0xFF0C1BE8                                                        at [866f4:ff0c0dc0]
 call 0x3168(f88, ff0c57a4, 0, 0)                                                at [ff0c1c80:ff0c0dc0]
...

0xFF0C1BD4 is cstart, 0x866B4 is bzero32, 0x3168 must be create_init_task and 0xFF0C57A4 must be init_task. Some of these functions are called from RAM, so we'll have to identify where its contents come from (what is copied where). TODO.

Next step, for the emulation, would be to find the MPU registers (near InitializeIntercom), mpu_send/mpu_recv stubs may also help, and we also need to check the serial flash communication.

For executing user code on the camera, next step would be to reserve memory for our application. This is usually done by shrinking the "malloc" buffer. Take a look at boot-hack.c from the "qemu" branch (as it's a bit more verbose and better explained). These debug messages can be seen on the QEMU-boot-check page.

To be continued.

18
General Development Discussion / Re: Canon EOS 1300D / Rebel T6
« on: May 24, 2017, 10:31:51 AM »
A summary of the recent IRC discussions.

Quote
Not finding an easy way to spit out debug messages when debugging it running under qemu.

To print debug info from ML to the QEMU console, there is qprintf (in qemu-utils.c). I'd like to turn this into a "standard" debugging API, making it available anywhere in the source code (so it won't get compiled in the regular binary, but activated with CONFIG_QEMU=y). Halfway done on the "qemu" branch.

To control QEMU's verbosity, try running with "-d help" (there are many options). Note: most of these are on QEMU 2.5.0 (qemu branch in our tree). They are not ported to 2.9.0, where not all the basics are working properly yet.

Question:  Where is it we are trying to locate to ?

RELOCADDR is in RAM (our modified copy of Canon's startup code). On 1300D, main firmware starts at FE0C0000, not FE010000, so we'll relocate the startup code from there, until being able to replace their init_task with our version. Once there, we can launch our own task(s) alongside Canon's.

Quote
I can see the HIJACK_TASK_ADDR is around here on the 60D (0x1a20 to be exact), so I'm wondering if this is a table of tasks / interrupt vectors or something ?

HIJACK_TASK_ADDR is probably the same as CURRENT_TASK in GDB and current_task in stubs.S (pointer to the current task structure - see tasks.h).

In 1300D/debugmsg.gdb:
Code: [Select]
macro define CURRENT_TASK 0x31170
macro define CURRENT_ISR  (*(int*)0x31174 ? (*(int*)0x640) >> 2 : 0)

Also, to see tasks starting:
Code: [Select]
# this is valid on all firmware versions
b *0x38FC
task_create_log

# this one is for firmware 1.0.1
b *0xFE11D6B4
DebugMsg_log

This debugmsg.gdb is committed on the "qemu" branch, but I've only tested on firmware 1.0.1 (only noticed there's a newer firmware available after committing).

Quote
hello world

Code: [Select]
23:22 < KennetRunner> or is it at a stage where I run hello world on my camera ?
23:24 < alexML> well, you can override the image buffer address before Canon GUI initializes the display (so ML code has
                something to draw on)
23:24 < alexML> ML gets the buffer from bmp_vram_info[1].vram2 (where bmp_vram_info is a stub)
23:25 < alexML> rather than waiting for this to get valid (nonzero), just set it to something outside the normal RAM range
                (e.g. 0x50000000 should be fine) and set QEMU to display from the same address
23:26 < alexML> this should give a hello world from the minimal platform, before getting the GUI working on the vanilla
                firmware
23:26 < KennetRunner> I'll have a bash at that then...
23:27 < alexML> in qemu, look at s->disp.bmp_vram

I've also set up a job on the build server for QEMU 1300D tests, where you can find some startup logs with various levels of detail (firmware 1.0.1 for now):

http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-1300D/

19
Code: [Select]
SD: CMD12 in a wrong state
[SDIO] Error

That's fine, I get those too. I'm not sure if this indicates an emulation bug / incomplete model, or it's just how the (simplified) SD driver used in bootloader is supposed to behave (note: CMD12 is STOP_TRANSMISSION).

In the main firmware, you'll get a similar error about CMD1; this one is OK, as it appears to be the way Canon code probes for MMC cards. Regular SD cards are probably not supposed to reply to CMD1 outside the SPI transfer mode (at least that's my understanding), so the SD emulation backend prints some messages. The full conversation can be watched with -d sdcf (or -d sdcf,io for more details) and cross-checked with the SD specification.

If the reader is familiar with SD protocol, I'd welcome any insights (in particular, for the UHS-I initialization sequence).


Note that I'm running macosx 10.12.4 and there doesn't seem to be a readily available losetup. Is this necessary?

To avoid corrupting the data on the SD image, it's best to prevent starting the emulation if the image is mounted by the user (as there will be two processes wanting to write on the same card image, without knowing about each other). This is done on Linux by checking /proc/mounts (losetup is used with "loopback" devices - that is, when mounting an image as a filesystem). Don't know how this works on Mac, and don't know whether my method is portable across other Linux distributions either.



The other issues appear to be (more or less) just annoyances (as they don't print what the user expects to see) and probably easy to fix.

20
Camera-specific discussion / Re: ML on EOS-M2
« on: May 24, 2017, 09:43:20 AM »
Seems like this one should be one of the "easier" ports.

Definitely, as it's plain DIGIC 5 (unlike 1300D, which looks like a mix between D4, 5 and 6).

The boot flag is "traditionally" enabled after getting the Hello World working. There are two flavors: the minimal target (which requires a tiny set of stubs and only compiles a few small files) and the regular one (CONFIG_HELLO_WORLD in the full source code, which also requires file I/O for loading fonts, and a bunch of other initializations). Once the second version is working, CONFIG_DUMPER_BOOTFLAG will be straightforward to compile and run. The "user-friendly" installer comes a bit later, as it requires some more stubs.

The boot flag can be also enabled earlier, from the bootloader context; we did that on the DIGIC 2 and 3 models, blindly (without having the display available for debugging or user feedback). Now that we have display access from bootloader, that's a valid option as well (though I wasn't comfortable doing this step on 7D Mark II).

Until enabling the boot flag, you'll need my assistance to run a binary on the camera. You can run them in QEMU though, by setting "boot=1" though.

Will explain how to get the Hello World on models that don't boot to Canon GUI on the 1300D thread, as it was covered on IRC a few days ago.

21
General Help Q&A / Re: Lua vs ML data
« on: May 23, 2017, 08:02:22 PM »
Any test case we can use for cross-checking the math?

Code: [Select]
    else if(!strcmp(key,"value")) lua_pushnumber(L, lens_info.aperture / 10.0);
    else if(!strcmp(key, "focus_distance")) lua_pushinteger(L, lens_info.focus_dist * 10);
    else if(!strcmp(key, "focal_length")) lua_pushinteger(L, lens_info.focal_len);

22
Nice. This is the cocoa interface, right? Do you have menus that allow you to switch between serial console, VGA and so on?

BTW, after this commit it should install cleanly on clang and/or Mac.


23
Scripting Corner / Re: Lua Scripting (lua.mo)
« on: May 21, 2017, 01:53:08 AM »
Suspecting some stack corruption from lua_getfield (which appears to require a lua_pop, but that doesn't happen everywhere, see e.g. lua_dryos.c), I've started to dig inside the Lua C API (which I still don't really understand) and got some questions:

1) Is there some easy way to check the stack usage of our C functions? (for example, to make sure each lua_getfield call is paired with a lua_pop, but not only)

I can imagine some API test that calls every such function 1000 times or so to check whether it will overflow the stack, but that doesn't look very nice.

The LUA_FIELD_* macros appear to address this purpose, but they are not used everywhere; as a first step, I'd refactor all the calls to lua_getfield to use similar macros. But I suspect there are more instances of similar behavior, not obvious from the function names.

2) I've also stumbled upon the LUA_PARAM* macros. The Lua manual recommends luaL_checkinteger/number/string/whatever for the same purpose. Any reason we use those long macros, instead of the standard functions?

3) Most of the C tables allow setting arbitrary fields besides the predefined ones (but not all - font doesn't). What is the rationale behind this? Does any of the existing scripts make use of this feature?

I'm tempted to make them read-only, so it would catch typos when accessing these fields, rather than creating new fields that have no effect. Example: you could write in a script: lv.emabled = true (typo) and with the current behavior, you'll probably spend some time figuring out why this doesn't enter LiveView.

4) Most of the fields (from various objects) appear in 3 places (and it's easy for them to become inconsistent, as it doesn't give a compiler error, and I couldn't come up with a way to check them in api_test.lua either). For example, in lua_battery, all the fields (level, id, performance etc) appear in luaCB_battery_index, in luaCB_battery_newindex (on the long line with strcmp) and on lua_battery_fields. Any suggestions for reducing these repetitions?

24
Camera-specific discussion / Re: Canon 1200D
« on: May 20, 2017, 08:47:44 PM »
Does it crash in the default configuration? (after resetting ML to defaults)

BTW, this kind of crash might depend on the files you are transferring from the card (could be their number, size, or even the contents of some particular file, or a combination of factors). Therefore, to reproduce it, you might have to copy the images back on the card. Is this true? (in other words, can you reproduce the issue regardless of how many files you have on the card?)

Pages: [1] 2 3 ... 332