Author Topic: Tutorial: finding stubs  (Read 15340 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10180
  • 5D Mark Free
Tutorial: finding stubs
« on: June 03, 2014, 02:35:25 PM »
Quick tutorial for finding stubs using the GPL tools from CHDK (very easy to get started under Linux).

Background

These stubs are functions from Canon firmware, that can be called from ML code. To use them, one needs to know:
- the function address in Canon code (that's the stub you need to find). The exact address is unique for each camera.
- the function declaration (what parameters it accepts?) - declared in some header file, and usually common for all cameras.
- what does it do (if it's not obvious from its name and usage context, a comment is welcome)

0) Preparation
- make sure you can compile ML yourself
- take a look at http://magiclantern.wikia.com/wiki/ASM_introduction

1) Downloading the tools
- grab disassemble.pl
- make the following changes (your paths may be different; these are the paths for the default ML toolchain):

Code: [Select]
@@ -25,14 +25,14 @@
 
 # adjust these for your needs (note final slash):
 #$path = "$ENV{'HOME'}/gcc-4.1-arm/bin/";
-$path = "";
+$path = "$ENV{'HOME'}/gcc-arm-none-eabi-4_8-2013q4/bin/";
 
 # note on "strings": default is a minimum length of 4 chars.
 # So if u are hunting for e.g. "FI2" add -n3
 # However, it gives a lot of false positive.
 $strdump = "strings -t x";
-$objdump = "${path}arm-elf-objdump";
-$objcopy = "${path}arm-elf-objcopy";
+$objdump = "${path}arm-none-eabi-objdump";
+$objcopy = "${path}arm-none-eabi-objcopy";
 
 if (@ARGV != 2) {
  die("Usage: $0 0x<offset> <dump.bin>");

2) Disassembling

You can find a ROM dump from your camera under ML/LOGS, on your card. Usually, the one containing code is ROM1 and loads at 0xFF000000. So, run the script with:

Code: [Select]
perl disassemble.pl 0xFF000000 ROM1.BIN

This script saves two interesting files: ROM1.BIN.strings and ROM1.BIN.dis. For finding stubs, you are interested in the disassembled code, ROM1.BIN.dis. Open that in a text editor.

3) Identifying stubs

Most ML stubs can be found from strings. Most of them are functions, so they start with a PUSH instruction (or STMFD in IDA). The PUSH usually indicates a start of a function, but not always; there may be one or two instructions before it. If in doubt, look for other code calling your function (code references to it, usually BL instructions).

Some common patterns:


- a function referencing the string inside its body (for example, GUI_Control):

Code: [Select]
GUI_Control:
   push ...
   ...
   DebugMsg(..., ..., "GUI_Control: ...");
   ....
   pop ...

which, in pure ASM, looks like this:
Code: [Select]
loc_ff0ded1c: ; 21 refs
ff0ded1c: e92d40f8 push {r3, r4, r5, r6, r7, lr}  ; ff0ded1c is the address you are looking for; notice there are a few references to it (other Canon code calling it)
...
ff0ded34: e28f2e16 add r2, pc, #352 ; ff0dee9c: (5f495547)  *"GUI_Control:%d 0x%x"
ff0ded38: e3a01003 mov r1, #3
ff0ded3c: e3a00085 mov r0, #133 ; 0x85
ff0ded40: eb3c9b92 bl loc_5b90 ; this must be DebugMsg (aka DryosDebugMsg in stubs.S)



- another function calling the function you are looking for, and printing an error message if it fails. For example, if you look for GetSizeOfMaxRegion:

Code: [Select]
AllocateMemory:
   ...
   BL GetSizeOfMaxRegion
   if (returned_value != 0)
   {
       DebugMsg(..., ..., "[MEM] ERROR GetSizeOfMaxRegion [%#x]", returned_value);
   }

which, in pure ASM, looks like this:

Code: [Select]
ff9fa1c0: ebffff39 bl loc_ff9f9eac ; this is GetSizeOfMaxRegion you are looking for
ff9fa1c4: e3500000 cmp r0, #0
ff9fa1c8: 11a03000 movne r3, r0
ff9fa1cc: 128f20d0 addne r2, pc, #208 ; ff9fa2a4: (4d454d5b)  *"[MEM] ERROR GetSizeOfMaxRegion [%#x]"
ff9fa1d0: 13a01016 movne r1, #22
ff9fa1d4: 13a00000 movne r0, #0
ff9fa1d8: 1bfff05c blne loc_ff9f6350 ; this must be DebugMsg (aka DryosDebugMsg in stubs.S)

Repeat the process for other stubs, until you get comfortable with the workflow.

Other stubs may not be easy to find for strings; in this case, you may look for the context (other functions calling it, which are identifiable from strings). Comparing with a dump from another camera, which has these stubs, is helpful.

Other stubs are not functions, but data structures. You will need to find some functions using those structures first.

A very useful hint: the differences between related stubs may be constant, or at least helpful to get you within the ballpark. For example, the offset between FIO_SeekFile and FIO_SeekSkipFile is usually 0xD0. Check the stub files for more cameras, and look for such similarities. It's not a hard rule though, but it helps in many cases.

A note about DIGIC V cameras: they copy a section from the ROM, starting from assert_0 (string "Assert: File %s,  Expression %s,  Line %d"), to 0x1900. So, there will be a difference between the address where your function may run, and the address where it's placed in the ROM. This difference is called RAM_OFFSET (it's declared in stubs.S). So, if say AllocateMemory is at 0xFF9FA160 in ROM (5D3.123), its RAM copy will be at 0xFF9FA160 - RAM_OFFSET = 0xFF9FA160 - 0xFF9F07C0 = 0x99a0. So, you will find many BL calls to 0x99a0, for example:
Code: [Select]
ff0e7644: eb3c88d5 bl loc_99a0
...
ff0e7658: e59f20e8 ldr r2, [pc, #232] ; ff0e7748: (ff0e5904)  **"%s : AllocateMemory(READ_ONE_PARAM)"

Function calls are usually relative to the program counter. Therefore:
- if the code calls a RAM function from another RAM function, and you look at the ROM copy of the caller, you will see BL rom_address
- if the code calls a RAM function from a ROM function, and you look at the caller code (the ROM function), you will see BL ram_address
- if the code calls a ROM function from a RAM one, and you look at the ROM copy of the caller, you are screwed :) (I'll let you do the math for this one)

4) Testing out

Once you have guessed a stub, put it in ML's stubs file (platform/camera.version/stubs.S), look it up in ML code to see where it's used (so you know what exactly to test), compile and test it out. There is also a stub testing routine under Debug menu, and running it is always a good idea. If the stub test routine does not contain yet a test for your stub, consider adding one.

Double-check that your stub is indeed at the beginning of the function. Jumping in the middle of a function may have undesired consequences ;)

Also double-check that you are not calling EraseSectorOfRom or other stuff like that.

Other tools:

You may prefer to use Pelican's ARMu (freeware, but not free software) or IDA (commercial, with demo version available). You can find some hints for these programs here and here, but tutorials are welcome.

I still use ARM-console for browsing around and decompling, but it may be a little difficult to install. If you don't need a decompiler, browsing the ASM from disassemble.pl or ARMu should be enough.

vertigopix

  • Senior
  • ****
  • Posts: 281
  • EOS M - 60D - 6D
Re: Tutorial: finding stubs
« Reply #1 on: June 03, 2014, 03:44:55 PM »
For the pre-built VM the correct path is :

Code: [Select]
$path = "$ENV{'HOME'}/magiclantern/arm-toolchain/bin/";

vertigopix

  • Senior
  • ****
  • Posts: 281
  • EOS M - 60D - 6D
Re: Tutorial: finding stubs
« Reply #2 on: June 03, 2014, 06:11:14 PM »
If anybody can PM me the ROM1.BIN from an EOSM.202 it would be great !   ;)

vertigopix

  • Senior
  • ****
  • Posts: 281
  • EOS M - 60D - 6D
Re: Tutorial: finding stubs
« Reply #3 on: June 04, 2014, 10:02:52 AM »
If anybody can PM me the ROM1.BIN from an EOSM.202 it would be great !   ;)

Thank you tweak  :)

EVVK

  • New to the forum
  • *
  • Posts: 24
Re: Tutorial: finding stubs
« Reply #4 on: June 06, 2014, 12:45:42 AM »
Great guide! Thanks. Maybe it should be wikified as well? :)

Is there anyway to extract ROM1.BIN from vendors firmware update (FIR) file?
data_body from dissect_fw3 seems different/wrong.

I'd really want to compare the .dis file for other models to get some confidence if I've got this right.

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 980
Re: Tutorial: finding stubs
« Reply #5 on: June 20, 2014, 09:36:01 PM »

Is there anyway to extract ROM1.BIN from vendors firmware update (FIR) file?

I have the same question.
70D.112 & 100D.101

Andy600

  • Hero Member
  • *****
  • Posts: 1802
  • Have you tried turning it off and on again?
Re: Tutorial: finding stubs
« Reply #6 on: July 11, 2014, 07:42:45 PM »
Just had look for the SRM stubs for the 50D and the ones needed show:

Code: [Select]
        Line 177576: ff0a1db8: 128f2e1b addne r2, pc, #432 ; ff0a1f70: (424f4a5b)  *"[JOB ERROR] SRM_AllocateMemoryResourceFor1stJob failed [%#x]"
  Line 177691: "[JOB ERROR] SRM_AllocateMemoryResourceFor1stJob failed [%#x]":
  Line 177955: ff0a2338: 124f2e3d subne r2, pc, #976 ; ff0a1f70: (424f4a5b)  *"[JOB ERROR] SRM_AllocateMemoryResourceFor1stJob failed [%#x]"
Line 2284867: ff8a1db8: 128f2e1b addne r2, pc, #432 ; ff8a1f70: (424f4a5b)  *"[JOB ERROR] SRM_AllocateMemoryResourceFor1stJob failed [%#x]"
  Line 2284982: "[JOB ERROR] SRM_AllocateMemoryResourceFor1stJob failed [%#x]":
Line 2285247: ff8a2338: 124f2e3d subne r2, pc, #976 ; ff8a1f70: (424f4a5b)  *"[JOB ERROR] SRM_AllocateMemoryResourceFor1stJob failed [%#x]"

nothing comes up for
Code: [Select]
SRM_FreeMemoryResourceFor1stJob  ???


One curious thing I saw was regarding 720p H.264 as the 50D can't currently do it. Not sure if this is the key to it?

Code: [Select]
        Line 609442: ff227678: e28f2f55 add r2, pc, #340 ; ff2277d4: (34363248)  *"H264E InitializeH264EncodeFor720p"
Line 609531: "H264E InitializeH264EncodeFor720p":
Line 2717530: ffa27678: e28f2f55 add r2, pc, #340 ; ffa277d4: (34363248)  *"H264E InitializeH264EncodeFor720p"
Line 2717619: "H264E InitializeH264EncodeFor720p":
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3203
  • 60Da / 1100D / EOSM
Re: Tutorial: finding stubs
« Reply #7 on: July 11, 2014, 11:24:56 PM »
nothing comes up for
Code: [Select]
SRM_FreeMemoryResourceFor1stJob  ???
look for DeleteSkeltonJob, SRM_FreeMemoryResourceFor1stJob is called from that function

zloe

  • New to the forum
  • *
  • Posts: 17
Re: Tutorial: finding stubs
« Reply #8 on: January 10, 2015, 11:11:54 AM »
Is there anyway to extract ROM1.BIN from vendors firmware update (FIR) file?
I have the same question.

Same for me.
I'd like to look into the new firmware of the 6D (1.1.6), but I don't want to install it yet on my camera.
EOS 6D, EOS 650D both with ML, EOS 80D (no ML, ... yet)

FelixJongleur42

  • New to the forum
  • *
  • Posts: 4
Re: Tutorial: finding stubs
« Reply #9 on: December 29, 2016, 05:29:07 PM »
Thank you for this tutorial.
I would be interested in supporting the 60D 1.1.2 - also to understand what it takes to support a new firmware version.
Apart from the build process (which I could probably figure out) - would it be also necessary to (re-)find all stubs in order to rewrite stubs.S ? And if so, how would we retrieve the ROM1.BIN if ML does not support it yet?
If my assumptions are correct could you edit the statement
Quote
The exact address is unique for each camera.
to
Quote
The exact address is unique for each camera model and firmware version.
in your initial post in order to clarify this.

Thank you!

kennetrunner

  • New to the forum
  • *
  • Posts: 16
Re: Tutorial: finding stubs
« Reply #10 on: May 11, 2017, 01:52:44 PM »
Is there a set of stubs that must be found to allow ML to work on a model (just starting stub hunting for 1300D)

Thx .. ken
1300D.110

kennetrunner

  • New to the forum
  • *
  • Posts: 16
Re: Tutorial: finding stubs
« Reply #11 on: May 14, 2017, 06:44:10 PM »
So just checking my logic (based on @a1ex post above...)

In the 1300D disassembly (ROM1 to 0xFE000000) I'm seeing:
Code: [Select]
loc_fe0cc88c: ; 22 refs
fe0cc88c: e92d40f8 push {r3, r4, r5, r6, r7, lr}
fe0cc890: e1a05000 mov r5, r0
fe0cc894: e1a06001 mov r6, r1
fe0cc898: e1a04002 mov r4, r2
fe0cc89c: e1a03000 mov r3, r0
fe0cc8a0: e58d1000 str r1, [sp]
fe0cc8a4: e28f2e16 add r2, pc, #352 ; fe0cca0c: (5f495547)  *"GUI_Control:%d 0x%x"
fe0cc8a8: e3a01003 mov r1, #3
fe0cc8ac: e3a00085 mov r0, #133 ; 0x85
fe0cc8b0: eb014ab7 bl loc_fe11f394
fe0cc8b4: e1a03004 mov r3, r4
fe0cc8b8: e1a02006 mov r2, r6
fe0cc8bc: e1a01005 mov r1, r5
fe0cc8c0: e28dd004 add sp, sp, #4
fe0cc8c4: e8bd40f0 pop {r4, r5, r6, r7, lr}
fe0cc8c8: e3a00000 mov r0, #0
fe0cc8cc: eafffed6 b loc_fe0cc42c
loc_fe0cc8d0:

So in the stubs.S file I would have
Code: [Select]
/** GUI **/
NSTUB(0xFE0CC88C,   GUI_Control)

Seem correct ?


Also, I'm guessing for the stubs in the 0xXXXXX (RAM ??) range I have to somehow dump the memory contents from qemu after it has run and find what part of the ROM got copied there ?

Sorry for the deluge of questions... :-)

1300D.110

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10180
  • 5D Mark Free
Re: Tutorial: finding stubs
« Reply #12 on: May 14, 2017, 07:48:08 PM »
Seems correct (for both questions).

For more details about the RAM functions, try the "copied" keyword in the search box, in the 1300D, 7D2 or EOS M5 topics. Or, "copied to RAM" when searching the entire forum.

You can also get the copied sections from the main ROM with dd. For EOS M5 I've used these commands:

Code: [Select]
dd if=ROM1.BIN bs=1 skip=$((0x001AF2C)) count=$((0x3B8)) of=0xDF020000.bin
dd if=ROM1.BIN bs=1 skip=$((0x001B2E4)) count=$((0xF1C)) of=0x4000.bin
dd if=ROM1.BIN bs=1 skip=$((0x115CF88)) count=$((0x6054C)) of=0x8000.bin
dd if=ROM1.BIN bs=1 skip=$((0x11BD4D4)) count=$((0x1444)) of=0x1900000.bin
dd if=ROM1.BIN bs=1 skip=$((0x11BE918)) count=$((0x152A0)) of=0xDFFC4900.bin

If you use IDA, load each RAM blob as "additional binary file".

dfort

  • Hero Member
  • *****
  • Posts: 1799
Re: Tutorial: finding stubs
« Reply #13 on: May 14, 2017, 08:23:26 PM »
Usually, the one containing code is ROM1 and loads at 0xFF000000.

In the 1300D disassembly (ROM1 to 0xFE000000)

Just curious how you figured that out.
EOSM.202 EOSM.203 EOSM2.103 700D.115 5D3.*

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10180
  • 5D Mark Free
Re: Tutorial: finding stubs
« Reply #14 on: May 14, 2017, 09:09:45 PM »
The ROM address?

In the general case, you need to find a value for which the cross-references in the code start to make sense. Many things use relative addressing, others use absolute addressing (e.g. for calling subroutines or for referencing strings). You need to find some clues, and those things that use absolute addressing are a good choice.

On EOS firmware, from the startup code (near the "gaonisoy" signature), the first instructions are, on many models, position-independent (so you can start with any initial guess). On 550D, with disassemble.pl, starting with the (wrong) address 0 from a ROM dump saved from 0xff010000:

Code: [Select]
       0: e59ff0bc ldr pc, [pc, #188] ; 0xc4
      c4: ff01000c undefined instruction 0xff01000c

If you load the binary at any other address (for example, 0xAA000000), the result is the same:
Code: [Select]
aa000000: e59ff0bc ldr pc, [pc, #188] ; 0xaa0000c4
aa0000c4: ff01000c undefined instruction 0xff01000c

So, the start address is hardcoded in this way on many models. The code is position-independent (no matter where you run it from, the value 0xff01000c is loaded in the PC register). After that, execution continues from 0xff01000c, so this address is a good clue about where the code will be executing from.

1300D is different though - it uses a relative jump (which is also position independent, but has no absolute address hardcoded):
Code: [Select]
FE0C0000                 B       sub_FE0C000C                           ; if ROM1 is loaded at the correct address
AA0C0000                 B       sub_AA0C000C                           ; if ROM1 is loaded at some other address (e.g. 0xAA000000)

Just from this startup code, we don't know where Canon code expects the main firmware to be loaded. However, this info can be found in the bootloader (for example, after deciding not to load autoexec.bin):

Code: [Select]
FE011134                 LDR     R7, =sub_FE0C0000
...
FE0111AC                 MOV     R1, R7
...
FE01B210                 MOV     PC, R1

In this case, the PC was loaded with 0xFE0C0000, no matter where the bootloader code was loaded for the initial analysis (I could have loaded it at 0 and the result would have been the same). On 1300D, I believe the bootloader is executing from 0xFFFF0000 (at least that's how I've got it running in QEMU, but dumping from this address gives... nothing, i.e. 0xFF everywhere; but the reset vectors at FE010000 do contain absolute addresses in the FFFFxxxx range, similar to older bootloaders).


Another way: from similarity with other models (so far, most of them*) use the same ROM layout, which is displayed in QEMU as well). Once you know the ROM1 size, subtract it from 0x100000000.

*) except EOS M5 and probably all D7 models.