Portable ROM dumper

[a1ex]

Looks like the 7D is working! Updated first post as well.

FIR files:

DIGIC 4+:  1300D  2000D  4000D
DIGIC 6:  5D4  750D  760D  80D
DIGIC 7:  200D  6D2  77D  800D
DIGIC 8:  EOSR  M50  SX70  SX740
Master/Slave:  5DS  5DSR  7D2 7D
Oldies:  1000D  30D  400D  40D  450D  5D

- built from 2a15b7d c019793 with CONFIG_BOOT_FULLFAT=y CONFIG_BOOT_DUMPER=y CONFIG_BOOT_SROM_DUMPER=y
- green = confirmed working (either the last version, or a slightly older one)
- blue = not tested, but likely to work (based on other similar models, or on previous tests)
- purple = not tested, there may be surprises, but fixable (based on previous tests)
- orange = not tested, but unlikely to work (based on previous failures)

[Walter Schulz]

7D.206 with SanDisk  Extreme 60 MB/s, 32 GB

Contents of ROM0.MD5 and ROM1.MD5:

Code: [Select]

994fce5ee9ea3bb6df3ba0eaddf3e46f  ROM0.BIN
0de4cc03919f939c4ec691eb5fcfd744  ROM1.BIN

MD5 check on ROM0.BIN and ROM1.BIN running by PC:

Code: [Select]

50838bbf29aec4c6e62bee320d5c9138 J:\ROM0.BIN
0de4cc03919f939c4ec691eb5fcfd744 J:\ROM1.BIN
MD5 check for file ROM0.BIN differs.

BTW: No timestamps for ROMx.BIN and ROMx.MD5.

[dfort]

Confirming Walter's findings. Both the AUTOEXEC.BIN and DUMP_7D.FIR are working. Wish I had this when I dumped the 2.0.6 firmware.



Throwback to Dec. 31, 1969 on the date stamp.

By the way, I tried compiling with the options in my Makefile.user file -- didn't work. For the record, putting the compile options in the command line worked fine.

Speaking of dual processor cameras -- this is the firmware for the slave processor, right? Is there a way to dump the firmware for the master processor or am I not understanding how this works?

[critix]

On 1300D not working. Not dumping... I try with 8G card... I will try tomorow with another.

[Walter Schulz]

You can try to create a < 2 GB partition.

- the portable ROM dumper (you must format the card to a very small size, or dd this 256MB image - howto)

[a1ex]

By the way, I tried compiling with the options in my Makefile.user file -- didn't work. For the record, putting the compile options in the command line worked fine.

In Makefile.user, these options have to be one per line.

Speaking of dual processor cameras -- this is the firmware for the slave processor, right? Is there a way to dump the firmware for the master processor or am I not understanding how this works?

Dual processor (I'm talking specifically about master/slave configurations, not the kind of dual processor encountered in 5D4 or DIGIC 7):
- this ROM dumper only dumps the "primary" firmware (slave on 7D, master on 7D2/5DS/5DSR)
- secondary core is loaded with a dummy firmware, i.e. a while(1)
- dumping the secondary core requires understanding the communication APIs from Canon firmware (see e.g. how Dual ISO is implemented, but that method can be used only from main firmware)
- for 7D2/5DS/5DSR, see g3gg0's experiments on 5DS

I can look into that if you think it helps, but so far I've considered it low priority, given how many other unfinished things I already have with single-core models.

On 1300D not working. Not dumping... I try with 8G card... I will try tomorow with another.

Cross-checking in QEMU with the old dumper, but couldn't see a real reason why it won't work (except maybe for the caching stuff). If it still doesn't work, you may use these to narrow down:

- 1300D_D1.FIR (old method, requires a very small card, caches disabled, I/O trace very similar to the old one, with minor exceptions: display buffer address and an additional flush before disabling the caches)
- 1300D_D2.FIR (new method, no card size restrictions, caches disabled, I/O trace very similar until it starts to dump, i.e. as expected)
- DMP1300D.FIR from above is similar to 1300D_D2.FIR, but with caches enabled.

You can try to create a < 2 GB partition.

- the portable ROM dumper (you must format the card to a very small size, or dd this 256MB image - howto)

The filesystem size restrictions only apply to older dumpers (500D is the only exception I know; confirmed in QEMU that no other camera has this issue).

The new FIRs should work on 64GB cards or larger, too, as long as they are formatted as FAT32. Just checked on:
- a physical 64GB (58.1 GiB) SD with physical 5D3 (with autoexec.bin)
- a virtual 64GiB SD (formatted in a virtual 5D2 with a virtual SD to CF adapter) with emulated 1300D.
- a virtual 256GiB SD (formatted in a virtual 5D3 with card_fmt) with emulated 1300D and 450D [oh yeah, I've got a way to test card_fmt!]

[dfort]

In Makefile.user, these options have to be one per line.

My Makefile.user that didn't work (Mac):

Code: [Select]

#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)

# CONFIG_QEMU = y
# LOG_INTERRUPTS = y
# CONSOLE_DEBUG = y
# CONFIG_DEBUGMSG = y
# CONFIG_DEBUG_INTERCEPT_STARTUP = y
# CONFIG_DEBUG_INTERCEPT = y
# CONFIG_GDB      = y
# CONFIG_GDBSTUB  = y
# CONFIG_MMIO_TRACE=y

# Recovery branch options:
CONFIG_BOOT_FULLFAT=y
CONFIG_BOOT_DUMPER=y
CONFIG_BOOT_SROM_DUMPER=y
Am I missing any juicy options that could be turned on in Makefile.user?

- this ROM dumper only dumps the "primary" firmware ...

I can look into that if you think it helps, but so far I've considered it low priority...

Well, the MPU messages are in there, right?

Confirmed - MPU messages are on the Master processor. Actually, g3gg0 tried to log them back in 2012 (!)

There's probably some other stuff in there too. I can't get the 7D to do what reddeercity is doing with the 5D2 because some of the code seems to be running on the Master processor. In fact the raw_video_10bit_12bit_LVState branch won't compile on the 7D and it seems to be an issue with something in the 7D_MASTER code.

However, even with a Master processor dump the chances of me doing anything useful with it are rather slim.

@IDA_ML - Are you following this?

[critix]

Cross-checking in QEMU with the old dumper, but couldn't see a real reason why it won't work (except maybe for the caching stuff). If it still doesn't work, you may use these to narrow down:

- 1300D_D1.FIR (old method, requires a very small card, caches disabled, I/O trace very similar to the old one, with minor exceptions: display buffer address and an additional flush before disabling the caches)
- 1300D_D2.FIR (new method, no card size restrictions, caches disabled, I/O trace very similar until it starts to dump, i.e. as expected)
- DMP1300D.FIR from above is similar to 1300D_D2.FIR, but with caches enabled.

Unfortunately, I tried with 1G cards, but the same result.
I also tested with 1300D_D1.FIR and 1300D_D2.FIR
It stops at

Code: [Select]

- Dumping ROM0...

[a1ex]

Let's try the following:
- format the 1G card in the camera
- place the old dumper on the card
- run it (I expect it to work)
- format the card again in the camera (important; if old ROM files are still on the card, D1.FIR will just lock up like in your screenshot)
- copy 1300D_D1.FIR (which is pretty much identical to the old one, I don't see why it won't work)
- once that works, try 1300D_D2.FIR

Meanwhile I'm preparing some verbose FIRs to see exactly where it locks up.

Might have found the issue, hold a second. (nope, that won't explain the crash)

[critix]

I followed the steps you said. I see it now goes with 1300D_D1.FIR.
We wait to finish and try with 1300D_D2.FIR.

Code: [Select]

  Magic Lantern Rescue
 ----------------------------
 - Model ID: 0x404 1300D
 - Camera model: Canon EOS 1300D / KISS X80
 - Firmware version: 1.1.0 / 4.4.6 37(0b)
 - IMG naming: 100CANON/IMG_6797.JPG
 - Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
 - ROMBASEADDR: 0xFE0C0000
 - Open for write 1061E0 E92D47F0
 - 101F64 Card init => 2
 - Dumping ROM0...
 - MD5: c38d7deeecee5432c254ba563cc503b2
 - Dumping ROM1...
 - MD5: fb70c66a568d05504bdc1fa076d4271f
 - No serial flash.
 - Saving RESCUE.LOG ...
OK... 1300D_D2.FIR is not working...
I'm still waiting to end

Code: [Select]

Dumping ROM0 ...Maybe in the end it will go ... I have a little patience ...

[a1ex]

Looks good, so at least the card initialization (common to both methods) is working.

Under the same conditions, 1300D_D2.FIR locks up? Here's a more verbose version that otherwise does the same thing:

1300D_D3.FIR

[critix]

OK ... so 1300D_D2.FIR does not work ...
I ran 1300D_D3.FIR and stopped at the line:

Code: [Select]

WR 000000FA  1 42005FA0
WR 00000480 80 F0000000I hope I could see the writing ...

[a1ex]

Alright, file I/O DMA (SDDMA) locked up while trying to read from ROM. Will fix later.

[a1ex]

Let's try the 1300D again:

1300D_D4.FIR

This time, I'm copying the ROM contents to RAM before saving to card.

[critix]

Yeah ... now it's OK without problems ...

Code: [Select]

  Magic Lantern Rescue
 ----------------------------
 - Model ID: 0x404 1300D
 - Camera model: Canon EOS 1300D / KISS X80
 - Firmware version: 1.1.0 / 4.4.6 37(0b)
 - IMG naming: 100CANON/IMG_6797.JPG
 - Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
 - ROMBASEADDR: 0xFE0C0000
 - card_bootflags 1069cc
 - boot_read/write_sector 1071c0 1072b8
 - 101F64 Card init => 2
 - Dumping ROM0... 100%
 - MD5: 66354cabd287d45faae4c6158ba09606
 - Dumping ROM1... 100%
 - MD5: f534bbc469bd73f4e1bded438a2613d8
 - No serial flash.
 - Saving RESCUE.LOG ...

[a1ex]

Finally

Updated all FIR files with the latest codebase.

Just curious - is the dump directly usable with QEMU, or does it still require the dd trick as described here?

[critix]

I'll test in two hours
Unfortunately, it does not go without

Code: [Select]

dd if = ROM1.BIN of = BOOT.BIN bs = 64k skip = 1 count = 1
dd if = BOOT.BIN of = ROM1.BIN bs = 64k seek = 511

[Walter Schulz]

7D.206 here again with updated dumper:

Code: [Select]

Magic Lantern Rescue
 ----------------------------
 - Model ID: 0x250 7D
 - Camera model: Canon EOS 7D
 - Firmware version: ??? / ???
 - IMG naming: 100EOS7D/IMG_0000.JPG
 - User PS: ??? ??? ???
 - Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
 - ROMBASEADDR: 0xFF010000
 - card_bootflags 109a18
 - boot_read/write_sector 109d54 109d64
 - Patching 104294 from e3500001 to e3500000
 - 104254 Card low-level init => F4240
 - 1026EC Card init => 0
 - Patching 1026FC from e3510001 to e3510000
 - 1026EC Card init #2 => 1
 - Dumping ROM0... 100%
 - MD5: 8206fa3fda73c2ead57297bdea24f9fd
 - Dumping ROM1... 100%
 - MD5: 0de4cc03919f939c4ec691eb5fcfd744
 - No serial flash.
 - Saving RESCUE.LOG ...
ROM0.BIN checksum still not matching results computed by PC:

Code: [Select]

15df32dc1fccf481a812ae0fa19ebfe9 J:\ROM0.BIN
0de4cc03919f939c4ec691eb5fcfd744 J:\ROM1.BIN
Compared both files with those generated by dfort's ML build:
MD5 match for ROM1.BIN but not ROM0.BIN

[critix]

Yes you are right. Checksum is not the same for ROM0.BIN, but it is the same as ROM1.BIN:
For ROM0.BIN:

Code: [Select]

cat ROM0.MD5
66354cabd287d45faae4c6158ba09606  ROM0.BIN
md5sum ROM0.BIN
387d96a501c80ee5a1291e6a4bbbb636  ROM0.BINFor ROM1.BIN:

Code: [Select]

cat ROM1.MD5
f534bbc469bd73f4e1bded438a2613d8  ROM1.BIN
md5sum ROM1.BIN
f534bbc469bd73f4e1bded438a2613d8  ROM1.BIN

[polkah]

Hey, so i don't know if it can be of any use to you guys, but i tested it on my 80d, here what i got :
in the "rescue" file it says
  Magic Lantern Rescue
 ----------------------------
 - Model ID: 0x350 80D
 - Camera model: Canon EOS 80D
 - Firmware version: 1.0.1 / 6.2.2 9C(84)
 - IMG naming: 100CANON/IMG_5727.JPG
 - User PS: CineStyle Marvels Advanced 3.4
 - Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
 - ROMBASEADDR: 0xFE0A0000
 - card_bootflags 109a00
 - boot_read/write_sector 109e90 109f58
 - 101DA8 Card init => 2
 - Dumping ROM1... 100%
 - MD5: 67b48c0a6b19664f261dc502afaabf38
 - 105774: \n**** SROM(SIO%d) Menu ****\n
 - 105724: tag c0820200
 - sf_init 105710
 - 104f28: Read Address[0x%06x-0x%06x]:0x
 - 104578: tag d20b0000
 - sf_command_sio 10456C
 - Reading serial flash... 100%
 - Writing SFDATA.BIN... 100%
 - MD5: 99821e45b63d737ccd055bd8a6ed1367
 - Saving RESCUE.LOG ...

And, it seems like my camera still works, so... hooray.
I literally have no clue about what any of this mean, but if it can be any kind of help... yay
If you'd like more infos, just let me know, if this is totally useless and a complete waste of everyone's time... let me know as well

[a1ex]

It's portable code, i.e. same binary code attempting to run on all EOS models. It's a bit more verbose than required; it prints the address of functions it's going to call in Canon code (which were identified usually from strings).

For 80D, a ROM dumper was already available, so the new one doesn't bring much additional value (maybe just fewer restrictions, as the old one required a very small card). Still, it's good to know it's working on this camera, so... thanks for testing.

[Walter Schulz]

@polkah: Can you run an additional MD5 check on ROM1.BIN and SFDATA.BIN?

Windows CLI:

Code: [Select]

powershell get-filehash *.BIN -A MD5 | format-list

[polkah]

@polkah: Can you run an additional MD5 check on ROM1.BIN and SFDATA.BIN?

Windows CLI:

Code: [Select]

powershell get-filehash *.BIN -A MD5 | format-list

how'd you do that ?

[Walter Schulz]

Windows:
Hit Windows button
Type "cmd" (without quotation marks) and press Enter button
You will see a command prompt window, white characters on black ground.
Type (or copy)

Code: [Select]

powershell "get-filehash x:\*.bin -A MD5 | fl"replace "x" with your card's drive letter and press Enter button
You can copy results after marking them (Mouse) and pressing Enter

[morgan20]

Can confirm 6D2 working. But the ROM0 hash differs from the previous dump I got with the original 6D2 dumper. The hashes in *.MD5 files are same as the ROMs' hashes.