Firmware Dumping and Loading into IDA

[AgentJJ]

I've been up and down this forum. 
Where can I find info on dumping the canon firmware, decrypting and loading into IDA (or similiar)?
This is particularly for the recent 650D firmware release.  I've some experience with C/C++, uControllers and the like and would be very interested in joining development to move things forward.
Thanks

[g3gg0]

when booting the latest versions, you should find ROM dumps in your ML/LOG directory.
you can load ROM1 to 0xF8000000

[AgentJJ]

when booting the latest versions, you should find ROM dumps in your ML/LOG directory.
you can load ROM1 to 0xF8000000

Perfect, thanks!

[cthornhill]

g3ggo - is it universal that we can always load ROM1 to 0xF8000000? I saw this post, but no other clear (that I saw) note on the correct location of ROM1. I saw a reference that indicated ROM0 may be fixed in size on all cameras, so this makes sense,  but I did not find a hadcy table or note about both rome.

Just want to  be sure - any advice welcome as I get started going through the ROMs.

[nanomad]

The best option is to check the Makefile / stubs of each platform as it defines the correct ROM start address.
Note that some (all?) digicV models also relocate some functions into RAM at boot (check from RAM_OFFSET stub)

[g3gg0]

some words i wrote earlier:

flash (ROM1) on normal cameras starts at 0xF8000000 and is either 0x00800000, 0x01000000 ox 0x02000000 large.
after that size it repeats over and over. canon uses this intentionally for bootloader reset vector that must be found on 0xFFFF0000.
at 0xF0000000 is ROM0 which is rarely used. (flash ic usually not populated, just on 5d2 iirc)
if not populated, reading there will give some random noise or fading bits.

the REAL_IC i explained is ROM1 at 0xF8000000.
same applies to any ROM0 (like on 5D2/3) - it starts at 0xF0000000.

why i am so sure?
a) flashes are "selected" by "chip enable" lines. those are electronic logics.
since computers are built, those chip enable lines are driven by N/AND gates that compare the uppermost address bits.
guess why RAM is at 0x0xxxxxxx, uncached RAM at 0x4xxxxxxx, FPGAs at 0x8xxxxxxx, IOs at 0xCxxxxxxx and flash at 0xFxxxxxxx...
they use the upper 4 bits as selector for the device. RAM, FPGA, IO, flash etc.

b) there are *no* registers that control memory mapping on our CPUs. those you talk of are just cachable and execution bits.
nothing more. i implemented them in my emulator. i read ARM_ARM.PDF quite often.
i know that ARM *can* have a MMU, but just read the system identification registers via MCR...
we have *no* model with a MMU.

about memory maps.
well the ROM0 physically starts at 0xF0000000, but canon seems to *use* the mirror image at 0xF0800000.
just check the ROM size - its 0x00800000, so canon uses the first mirror image.
there is no real reason. they could compile the software with 0xF7000000 too and it will work the same.

those mirror images are explicitely used by canon for reason on ROM1.
as said, ARM requires reset vector at address 0xFFFF0000.
but for the main firmware they could choose any address.

but:
i dont say that it will always be like this. there might be models that have 4 flashes (i dont think that will ever happen)
or models that have only one flash and that starts at 0xF0000000 and repeats until 0xFFFFFFFF. (no subselection)
i just say that there is no kind of random flash mapping somewhere in memory.

also:
where the code jumps to, so which mirror image mirror of the available ones canon chose, depends on camera model.
as said, we have to check where the code jumps to  by looking at absolute addresses in flash.
i prefer to follow the whole code execution flow and to know which code jumps where.
if that is 0xF80C0000, 0xFF0C0000 or 0xFC0C0000 is something *not* important for dumping.
its just important when you load the file in IDA - and that is done *after* dumping the flashes.

conclusion:
so when dumping memories, dump ROM0 at 0xF0000000 - 0xF1000000 and ROM1 at 0xF8000000-0xFA000000.
then check when the memory content repeats.
on 5D3 you will see 0xF0000000 and 0xF0080000 will contain the same data -> then you have your flash size for ROM0.
and 0xF8000000 and 0xF9000000 contain the same data -> now you have your flash size for ROM1

there is *no* reason why to dump 256 megabytes from a location where the least models have any flash.
and if there is any flash, its a cheap 8MiB one.

[AgentJJ]

All of these chips on are cameras are just 32-bit addressable space, correct?  I'm assuming they don't have any kind of LPAE or 36/40-bit address extensions like some of the new ARM cores, right?

[marekk]

I'm trying to port 1% bitrate code to 60D. I've got ROM1 dump (0xf80000000). ROMBASEADDR  defined in makefile = 0xFF010000. I'm using disassemble.pl from chdk website. I've found all addresses in 600d code using disassemble.pl 0xff010000 600d.bin ... How should I find correct addresses in disassembled 60D rom ? I've found correct code but I'm not sure how should I start disassemble.pl..

[cthornhill]

Thanks to all involved. this is very helpful...maybe in time it can all become a 'formal' wiki page. Sorry if I missed other posts, and I appreciate the refresher. Getting up to speed is a little daunting, but I will get there.

I second marekk's note about being sure how to get the addresses...any 'spoon feeding' is appreciated by those of us getting started. Platform differences make it just that much more confusing too...(mac/linux/windows)...but I will get my mac in shape eventually...:-)

Cecil

[friend]

Hi,

when booting the latest versions, you should find ROM dumps in your ML/LOG directory.
you can load ROM1 to 0xF8000000

Hm, does this happen every time I am reflashing ML firmware? Looks like it doesn't

So, how to force it then without building custom ML with debug? Remove ML, disable bootflag, make card not bootable, revert to native canon firmware and start from scratch?

Thanks for your help.

P.S. Camera is 550D

[pravdomil]

[Pelican]

If somebody finds IDA too expensive or too difficult to use there is a simple tool for disassembling the ARM code, find stubs, compare different cameras' fw, etc.
It's ARMu written by me.
http://armu.pel.hu

[pravdomil]

and does anybody use some ARM decompiler? I'm not good in ASM.

[a1ex]

and does anybody use some ARM decompiler? I'm not good in ASM.

Noticed the question a bit late.

I use mostly this one: http://magiclantern.wikia.com/wiki/Decompiling
and also the HexRays decompiler, when the above one fails.

[a1ex]

Updated settings for current dumps:

- load ROM1.BIN on [DIGIC 2...6], or ROM0.BIN on [DIGIC 7]
- [DIGIC 2...5] select ARM processor, architecture ARMv5TEJ (we have ARMv5TE on all models from DIGIC 2 to DIGIC 5)
- [DIGIC 6...7] select ARM processsor, architecture ARMv7 A&R (most likely Cortex R4 on D6 and A9 on D7)
- uncheck 'Delete instructions with no xrefs' (we have plenty of those)
- uncheck 'Perform no-return analysis' (IDA usually gets it wrong)
- RAM section: 0, size 0x100000 (we can always add more later)
- ROM1 start address: 0xFF800000 for 8MB dumps, 0xFF000000 for 16MB dumps, 0xFE000000 for 32MB dumps, ROMBASEADDR for old non-fullsize dumps
- ROM0 start address: 0xE0000000 on [DIGIC 7], 0xF0000000 on [DIGIC 2..5] (data only)
- ROM1 size: 0x7FFFFC or 0xFFFFFC or 0x1FFFFFC (off-by-one bug in IDA; I have an older version, so YMMV)
- ROM0 size: actual file size
- File load address / size: same as ROM.
- [DIGIC 2...5] go to FFFF0000 and press C (that's the bootloader)
- [DIGIC 2...5] go to ROMBASEADDR (usually FF810000 or FF010000 or FF0C0000) and press C (that's the main firmware)
- [DIGIC 6] [optional, may slow down IDA]: Load Additional Binary, ROM1.BIN, loading segment 0, offset FC000000, num. bytes 0x1FFFFFC (only if your IDA has the off-by-one bug)
- [DIGIC 6] go to FC000000 or FE000000 to find the start address (32-bit); go there and press C (that's the bootloader)
- [DIGIC 6] go to ROMBASEADDR (usually FE0A0000) and press C (that's the main firmware)
- [DIGIC 7] go to E0000000 and press C (that's the bootloader)
- [DIGIC 7] go to E0040000, set the Thumb bit (ALT+G, T=1) and press C (that's the main firmware)
- [DIGIC 7] first instruction configures the reset vector (VBAR); interrupts are executed in Thumb mode


After loading the ROM, you will most likely need to load additional blobs, copied by Canon firmware from ROM into RAM and executed from there. You can find these by running the ROM in QEMU with "-d romcpy" (or just look them up in the QEMU test suite log).

Code: [Select]

./run_canon_fw.sh 60D,firmware="boot=0" -d romcpy

[ROMCPY] 0xFFFF0000 -> 0x0        size 0x40       at 0xFFFF05FC
[ROMCPY] 0xFFFF2A3C -> 0x100000   size 0xD1A8     at 0xFFFF12EC
[ROMCPY] 0xFF66A010 -> 0x1900     size 0x34B24    at 0xFF01000C
[ROMCPY] 0xFF0101E4 -> 0x4B0      size 0x214      at 0xFF010150

dd if=ROM1.BIN of=60D.0x4B0.bin bs=1 skip=$((0x101E4)) count=$((0x214))

Then, run the firmware in QEMU to get an initial IDC script with all the functions called during emulation:

Code: [Select]

./run_canon_fw.sh 60D,firmware="boot=0" -d idc
...
60D.idc saved.

Getting around:
- find out where the bootloader code jumps to main firmware
- follow the startup code in main firmware until you find "cstart" (it calls bzero32, then passes init_task as parameter to create_init_task)
- look up something that resembles a debug message, passed as third argument (in R2) to a function called thousands of times (that's DebugMsg)
- look up something that resembles a task name (e.g. GuiMainTask, CSMgrTask) and find task_create
- look up AllocateMemory (usually prints NG AllocateMemory and calls GetMemoryInformation on failure)
- look up the "Illegal inter" string and find out who calls it (you'll find the interrupt handler; there's one for each DryOS core in the firmware)
- look up some more functions with the Finding Stubs tutorial.

You should now have the basics to start browsing around. Have fun!

[a1ex]

Quick tip if you are using QEMU and IDA (or if you can import IDC scripts):

- identify task_create, register_func, register_interrupt and CreateStateObject
- add logging stubs for these into CAM/debugmsg.gdb
- run the firmware under QEMU+GDB as usual, without loading ML:

Code: [Select]

./run_canon_fw.sh EOSM2,firmware="boot=0" -d debugmsg -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb
- you get an IDC file with lots of named functions (saved directly from the GDB script)
- for example, on EOSM2, you get over 1000 named functions (not including state objects) and nearly 5800 functions total (including state object functions, named after the state machine + position in the state matrix)
- Thumb functions are handled, too (tested on 80D, EOSM5 and 5D3 Eeko secondary core)

(commit)

[calle2010]

Perhaps it is obvious but not mentioned here yet: Running qemu with "-d romcpy" creates a file "romcpy.sh" which is helpful to create the additional blobs.

[sombree]

a1ex, is this correct way to load additional binary blob? For example, I want to load "0xFF011868 -> 0x4000     size 0x15F14    at 0xFE0A00DA":

[IreuN]

Here's what I'm doing to load 70D 1.1.2 firmware ( Digic 5 )

1. Load ROM1.bin
2. Select ARM
3. Procesor options -> Edit ARM Architecture -> ARMv5TEJ -> OK -> OK https://i.imgur.com/zXWXlaM.png
4. Kernel Options 1 -> Uncheck ''Delete instructions with no xrefs' and 'Perform no-return analysis' -> OK https://i.imgur.com/IEPqtAg.png
5. Press OK to load the .bin
6. Check 'create RAM section'
7. Here are the values that I've put into 'Disassembly memory organisation' https://i.imgur.com/oB2yW0G.png
8. Jump to FFFF0000, press C
9. JUMP to 'i don't remember' look at A1ex post, and press C

I believe that works good, hope that will help somebody.

I'm having a problem with running './run_canon_fw.sh 70D,firmware="boot=0" -d romcpy' command. Im getting the fallowing error:

Code: [Select]

(process:13481): GLib-WARNING **: 22:40:59.262: ../../../../glib/gmem.c:489: custom memory allocation vtable not supported
I've changed the branch to 70D_merge_fw112, repeated all the steps, no luck. I haven't run 'install.sh' script from 'contrib/qemu' though. Was that necessary?

[names_are_hard]

My quick searching suggests this is a qemu + glib interaction and version dependent.  What versions of qemu and glib are you using for the build?

[IreuN]

All right, the problem was what i expected. After running 'install.sh' again from the 70D branch it was resolved. Now I'm getting 'SFDATA.BIN missing' error. So I need https://bitbucket.org/hudson/magic-lantern/src/unified/modules/sf_dump/. A couple of days and I think I will be able to the qemu successfully. 

[names_are_hard]

Cool, glad you passed a roadblock!  Please say what the solution was, so that people searching for your problem find an answer