Portable ROM dumper

Started by a1ex, January 25, 2016, 09:29:53 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

g3gg0

wondering why it failed in the first place.
maybe some counter overflowing? hmmm
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

dfort

Got my hands on a 500D and thought I'd try dumping the 1.1.2 firmware but ran into this:



Same issue with 1.1.1. I have used the portable dumper a few times and have never seen this before.

a1ex

Old-style model; covered in first post.

The good old blind dumper appears to work in QEMU (should work on all D4 and D5 models with bootflag enabled, except 7D). Make sure you have a valid image on the card, then go to PLAY mode. Split the dump in two, like you did on M2.

dfort

Well that does look similar to what we were doing a year ago. Tried the "blind dumper" and it gave me a file named "As" that was apparently the ROM1.BIN and didn't need to be split. Disassembled it and it looks good.

Thanks again!

a1ex

Quote from: a1ex on July 29, 2016, 12:06:45 PM
The issue: Canon's bootloader routines for file I/O copy the data to some cacheable (!) memory; when that buffer reaches 0x4000 bytes, it's written to card using DMA.

Canon finally fixed this in DIGIC 7 8)

The bug is, however, present in DIGIC 6 and earlier.

Updated autoexec.bin (first post) with:
- DIGIC 6 support, including serial flash dump (thanks t3r4n)
- DIGIC 7 support, when the time will come
- same portable binary loads on DIGIC 2, 3, 4, 5, 6, 7 AND 8!

ROM dumpers ready for 200D, 77D, 6D2 and 800D; will post the FIR versions in the DIGIC 7 thread.

These dumpers still require a very small card, but just formatting with a smaller filesystem will do the trick. The easiest way is (still) to write the QEMU SD image onto the card (howto).

The issue can be reproduced in QEMU on a large SD image (or by running from a physical card), so it's clearly not a caching issue. It can be reproduced from the FROMUTILITY menu, without loading AUTOXEC.BIN (proof of concept and details available on request). I believe are two different issues in Canon code: writing from cacheable memory (fixed on D7) and large card support (still present on D7).

rafaelbf

Quote from: dfort on April 10, 2018, 08:17:19 PM
Got my hands on a 500D and thought I'd try dumping the 1.1.2 firmware but ran into this:

Same issue with 1.1.1. I have used the portable dumper a few times and have never seen this before.

Hi dfort,

sorry for the late reply... I've tried Portable Dumper on my 500D, same screen on booth firmware, even with low capacity card.

dfort

@rafaelbf -- We need to use the blind dumper on the 500D.

Quote from: a1ex on January 25, 2016, 09:29:53 AM
Supported cameras:
- most DIGIC 4 (exceptions: 500D, 50D, 5D2, 7D)

Quote from: a1ex on April 10, 2018, 08:26:27 PM
The good old blind dumper appears to work in QEMU (should work on all D4 and D5 models with bootflag enabled, except 7D). Make sure you have a valid image on the card, then go to PLAY mode. Split the dump in two...

At first I had some issues splitting the dump in two but eventually figured it out.

DrEVILish

I have just ordered a 5Ds, which has D6+ like the 7D2, let me know if I can be of assistance, with testing and dumping.

JagoUK

7Dmk2 (Dual digic 6) dumped


Had to tell it to dump to CF but it actually dumped to the SD.

a1ex

Updated with serial flash support for DIGIC 5 models (first post). Tested only in QEMU; please try and report back.

To check:
- make sure the MD5 sums are correct for all files (including SFDATA.BIN if present)
- make sure SFDATA.BIN looks like valid data (i.e. not full of zeros or full of FF or otherwise containing garbage - this condition is not covered by the checksums)

QEMU test results:

Testing portable ROM dumper...
     5D: skipping
    5D2: skipping
    5D3: SD: ROM0.BIN: OK ROM1.BIN: OK
    5D4: SD: ROM1.BIN: OK SFDATA.BIN: OK
     6D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    6D2: SD: ROM0.BIN: OK ROM1.BIN: OK
     7D: CF: ROM0.BIN: OK ROM1.BIN: OK
   7D2M: ROMs not saved
    40D: skipping
    50D: skipping
    60D: SD: ROM0.BIN: OK ROM1.BIN: OK
    70D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    77D: SD: ROM0.BIN: OK ROM1.BIN: OK
    80D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   400D: skipping
   450D: skipping
   500D: skipping
   550D: SD: ROM0.BIN: OK ROM1.BIN: OK
   600D: SD: ROM0.BIN: OK ROM1.BIN: OK
   650D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   700D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   750D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   760D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   800D: SD: ROM0.BIN: OK ROM1.BIN: OK
   100D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   200D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1000D: skipping
  1100D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1200D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1300D: SD: ROM0.BIN: OK ROM1.BIN: OK
   EOSM: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
  EOSM2: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK


7D2 has the serial flash on the other CPU...

dfort

Quote from: a1ex on October 06, 2018, 06:18:53 PM
please try and report back.

Better late than never!

Worked perfectly on the 700D and EOSM. All MD5 sums checked out. SFDATA.BIN looks valid. First time I tried it on a 32GB card and it didn't work but it was perfect with a 1GB card.

The 7D didn't work no matter what size card I tried -- all the way down to an old 64MB CF card. It kept getting stuck on "Dumping ROM0..."



This note about the 7D probably still applies.

The reason I looked up this topic was to ask a question about the 7D. Would it be possible to dump the master? When I worked on the 2.0.6 firmware update I used the display based dumper and I'm fine with that but would like to know if it is possible and if so I need some guidance on what addresses to dump. Then to figure out how to disassemble it.

dfort

Quote from: a1ex on October 06, 2018, 06:18:53 PM
...Tested only in QEMU; please try and report back....

  1300D: SD: ROM0.BIN: OK ROM1.BIN: OK


Would asking for a 1300D ML-SETUP.FIR in order to run this test be an unreasonable request?

critix

Hi.
I have tried to find the issue of displaying Model Camera, Firmware version and IMG naming for models like 1300D.
I extracted the following files into a directory and compiled with for offline running:
compiler.h
prop_diag.c
prop_diag.h
property.h
propvalues.h

gcc prop_diag.c -o prop_diag

Then I ran:
./prop_diag 1300D_ROM1.BIN

The prop_diag.c file returns camera information, specifically: Camera Model, Firmware version and IMG naming. But that file can also run offline, in the sense that you give it a ROM file from which it tries to find the information above. If you run it through autoexec, then he tries to find the camera software information. If you run it offline, then he reads the given file as a parameter and tries to find that information.

To not compile portable.000 and run qemu, I chose to run it offline.
Now that I can run offline, I can make changes to the software and try to see why that information is not available.

The problem I've encountered is on the function:
check_terminator (0, last, 0).
There is no information for Digic4 +.
Maybe this feature needs to be changed for these device models?

With autoexec, for Digic 7 and Digic 6, guess_prop is called differently, with other values than the rest.
void prop_diag()
{
    if (is_digic7())
    {
        /* other models may lock up while reading this, so test first */
        guess_prop((void*)0xE0000000, 0x2000000, 1, 0);
    }
    else if (is_digic6())
    {
        guess_prop((void*)0xFC000000, 0x2000000, 1, 0);
    }
    else
    {
        guess_prop((void*)0xF0000000, 0x1000000, 1, 0);
        guess_prop((void*)0xF8000000, 0x1000000, 1, 0);
    }
    print_camera_info();
}

Neither Digic8 is found.
Canon 1300D, 500D, EOS M, EOS M2

dfort

@critix helped get this running on my Mac. Needed to make a few changes.

Call malloc.h this way:

prop_diag.c
#include <stdio.h>
#include <malloc/malloc.h>


The compiler will error out because it can't find features.h, just comment it out:

compiler.h
//#include <features.h>
#include <stdint.h>
#include <limits.h>
#include <sys/types.h>


Here's what it does on my EOSM ROM1.BIN

./prop_diag ROM1.BIN
Loading ROM1.BIN...
Scanning from 0x1056cb000 to 0x1066cb000...
Trying offset 0x60000, status=0xffff, size=0x3b24...
Prop    c000004      4 3
Prop    c000002  15084 3d
Trying offset 0x80000, status=0x0, size=0xf84...
Skipping inactive block 0x80000, status=0x0, size=0xf84...
Trying offset 0x81000, status=0x0, size=0xf84...
Skipping inactive block 0x81000, status=0x0, size=0xf84...
Trying offset 0x82000, status=0x0, size=0xf84...
Skipping inactive block 0x82000, status=0x0, size=0xf84...
Trying offset 0x83000, status=0x0, size=0xf84...
Skipping inactive block 0x83000, status=0x0, size=0xf84...
Trying offset 0x84000, status=0x0, size=0xf84...
Skipping inactive block 0x84000, status=0x0, size=0xf84...
Trying offset 0x85000, status=0x0, size=0xf84...
Skipping inactive block 0x85000, status=0x0, size=0xf84...
Trying offset 0x86000, status=0x0, size=0xf84...
Skipping inactive block 0x86000, status=0x0, size=0xf84...
Trying offset 0x87000, status=0x0, size=0xf84...
Skipping inactive block 0x87000, status=0x0, size=0xf84...
Trying offset 0x88000, status=0x0, size=0xf84...
Skipping inactive block 0x88000, status=0x0, size=0xf84...
Trying offset 0x89000, status=0x0, size=0xf84...
Skipping inactive block 0x89000, status=0x0, size=0xf84...
Trying offset 0x8a000, status=0x0, size=0xf84...
Skipping inactive block 0x8a000, status=0x0, size=0xf84...
Trying offset 0x8b000, status=0x0, size=0xf84...
Skipping inactive block 0x8b000, status=0x0, size=0xf84...
Trying offset 0x8c000, status=0x0, size=0xf84...
Skipping inactive block 0x8c000, status=0x0, size=0xf84...
Trying offset 0x8d000, status=0x0, size=0xf84...
Skipping inactive block 0x8d000, status=0x0, size=0xf84...
Trying offset 0x8e000, status=0x0, size=0xf84...
Skipping inactive block 0x8e000, status=0x0, size=0xf84...
Trying offset 0x8f000, status=0x0, size=0xf84...
Skipping inactive block 0x8f000, status=0x0, size=0xf84...
Trying offset 0x90000, status=0x0, size=0xf84...
Skipping inactive block 0x90000, status=0x0, size=0xf84...
Trying offset 0x91000, status=0x0, size=0xf84...
Skipping inactive block 0x91000, status=0x0, size=0xf84...
Trying offset 0x92000, status=0x0, size=0xf84...
Skipping inactive block 0x92000, status=0x0, size=0xf84...
Trying offset 0x93000, status=0x0, size=0xf84...
Skipping inactive block 0x93000, status=0x0, size=0xf84...
Trying offset 0x94000, status=0x0, size=0xf84...
Skipping inactive block 0x94000, status=0x0, size=0xf84...
Trying offset 0x95000, status=0x0, size=0xf84...
Skipping inactive block 0x95000, status=0x0, size=0xf84...
Trying offset 0x96000, status=0x0, size=0xf84...
Skipping inactive block 0x96000, status=0x0, size=0xf84...
Trying offset 0x97000, status=0x0, size=0xf84...
Skipping inactive block 0x97000, status=0x0, size=0xf84...
Trying offset 0x98000, status=0x0, size=0xf84...
Skipping inactive block 0x98000, status=0x0, size=0xf84...
Trying offset 0x99000, status=0x0, size=0xf84...
Skipping inactive block 0x99000, status=0x0, size=0xf84...
Trying offset 0x9a000, status=0x0, size=0xf84...
Skipping inactive block 0x9a000, status=0x0, size=0xf84...
Trying offset 0x9b000, status=0x0, size=0xf84...
Skipping inactive block 0x9b000, status=0x0, size=0xf84...
Trying offset 0x9c000, status=0x0, size=0xf84...
Skipping inactive block 0x9c000, status=0x0, size=0xf84...
Trying offset 0x9d000, status=0x0, size=0xf84...
Skipping inactive block 0x9d000, status=0x0, size=0xf84...
Trying offset 0x9e000, status=0x0, size=0xf84...
Skipping inactive block 0x9e000, status=0x0, size=0xf84...
Trying offset 0x9f000, status=0x0, size=0xf84...
Skipping inactive block 0x9f000, status=0x0, size=0xf84...
Trying offset 0xa0000, status=0x0, size=0xf84...
Skipping inactive block 0xa0000, status=0x0, size=0xf84...
Trying offset 0xa1000, status=0x0, size=0xf84...
Skipping inactive block 0xa1000, status=0x0, size=0xf84...
Trying offset 0xa2000, status=0x0, size=0xf84...
Skipping inactive block 0xa2000, status=0x0, size=0xf84...
Trying offset 0xa3000, status=0x0, size=0xf84...
Skipping inactive block 0xa3000, status=0x0, size=0xf84...
Trying offset 0xa4000, status=0x0, size=0xf84...
Skipping inactive block 0xa4000, status=0x0, size=0xf84...
Trying offset 0xa5000, status=0x0, size=0xf84...
Skipping inactive block 0xa5000, status=0x0, size=0xf84...
Trying offset 0xa6000, status=0x0, size=0xf84...
Skipping inactive block 0xa6000, status=0x0, size=0xf84...
Trying offset 0xa7000, status=0x0, size=0xf84...
Skipping inactive block 0xa7000, status=0x0, size=0xf84...
Trying offset 0xa8000, status=0x0, size=0xf84...
Skipping inactive block 0xa8000, status=0x0, size=0xf84...
Trying offset 0xa9000, status=0x0, size=0xf84...
Skipping inactive block 0xa9000, status=0x0, size=0xf84...
Trying offset 0xaa000, status=0xffff, size=0xf84...
Prop    2000000      4 ??
Prop    2000001     36 2.0.2
Prop    2000005     36 9.9.8 B8(3a)
Prop    2000002      4
Prop    2000003      4 ????
Prop    2000004     32 Daniel A. Fort
Prop    2000006     16   
Prop    2000007     36 0.0.0
Prop    2010000      4 ?
Prop    2010001      4 d
Prop    2010002      4 d
Prop    2010003      4
Prop    2010004      4
Prop    2010005      4
Prop    2010009      4
Prop    201000a      4 P?
Prop    201000b      4
Prop    2010006      4
Prop    2010007      4
Prop    2010008      4
Prop    201000d      4
Prop    201000e      4
Prop    201000f      4
Prop    2010010      4 d
Prop    2010011      4 e
Prop    2010012      4 d
Prop    2010013      4
Prop    2010014      4
Prop    2020000     44
Prop    2020008     44
Prop    2020009     44
Prop    202000a     44
Prop    202000b     44
Prop    2020005      4
Prop    2020006      4
Prop    2020001      4
Prop    2020002      4
Prop    2020003      4
Prop    2020004      4
Prop    202000c      4
Prop    202000d     40
Prop    202000e     40
Prop    202000f     40
Prop    2020010     40
Prop    2020011     40
Prop    2030000     28
Prop    2030001     12
Prop    2030002     36
Prop    2030004      4
Prop    2030005      4
Prop    2030003      4
Prop    2030006     12
Prop    2040000      4
Prop    2040001      4
Prop    2040002      4
Prop    2040003      4
Prop    2040004      4
Prop    2040005      4
Prop    2040007      4
Prop    2040008      4
Prop    2040009      4
Prop    204000a      4
Prop    204000b      4
Prop    204000c      4
Prop    204000d      4
Prop    204000f     76
Prop    2050000    184
Prop    2050001      4 ?!
Prop    2050002      4
Prop    2050004      8 IMG_
Prop    2050005      4
Prop    2050003      4
Prop    2050006      4
Prop    2050007      4
Prop    2050008      4
Prop    2050009      8 IMG
Prop    205000a      4
Prop    205000b      8 |

Prop    205000d      4
Prop    205000e      4
Prop    205000f      4
Prop    2050013      4
Prop    2050014      4
Prop    2050016      4
Prop    2050017      4
Prop    2050018     48
Prop    2050019      8
Prop    205001a      4
Prop    205001c      4
Prop    205001f      4
Prop    2050020      4
Prop    2050011      4
Prop    2050021      4
Prop    2050023      4
Prop    2050024      4
Prop    2050025      4
Prop    2050022      4
Prop    205001b      4
Prop    205001d     40
Prop    2050026      4
Prop    2050027      4
Prop    2050028      4
Prop    2050029      4
Prop    205001e      4
Prop    205002a      4
Prop    205002b      4
Prop    205002d      4
Prop    205002e      8 |

Prop    205002f      4
Prop    2060000      4
Prop    2060001     24
Prop    2060002     24
Prop    2060003     24
Prop    2060004     24
Prop    2060005     24
Prop    2060006     24
Prop    2060007     24
Prop    2060008     24
Prop    2060009     24
Prop    206000a      4 ?
Prop    206000b      4 ?
Prop    206000c      4 ?
Prop    206000d     24
Prop    206000e     24
Prop    206000f     24
Prop    2060010     24
Prop    2060011     24
Prop    2060012     24
Prop    2060013     24
Prop    2060014     24
Prop    2060015     24
Prop    2060016      4 ?
Prop    2060017      4 ?
Prop    2060018      4 ?
Prop    2060019     24
Prop    206001a     24
Prop    206001b     24
Prop    206001c     24
Prop    206001e     24
Prop    2060021    300
Prop    2060020     24
Prop    206001f     24
Prop    2070000      4
Prop    2070001      4
Prop    2070002      8 ?23\
Prop    2070004     16 /
Prop    2070005      4
Prop    2070006      4
Prop    2070003      4
Prop    2070007      4 ????
Prop    2070008      4 ????
Prop    2070009      4
Prop    207000a      4
Prop    2070012      4
Prop    2070013     44
Prop    2070014      4
Prop    2070015      4
Prop    2070016     48
Prop    2070017      4
Prop    2070018      4
Prop    2080000      4
Prop    2080001      4
Prop    2090000      4
Prop    2090001      4
Prop    2090002      4
Prop    2090003      4
Trying offset 0xb0000, status=0x0, size=0xf84...
Skipping inactive block 0xb0000, status=0x0, size=0xf84...
Trying offset 0xb1000, status=0x0, size=0xf84...
Skipping inactive block 0xb1000, status=0x0, size=0xf84...
Trying offset 0xb2000, status=0x0, size=0xf84...
Skipping inactive block 0xb2000, status=0x0, size=0xf84...
Trying offset 0xb3000, status=0x0, size=0xf84...
Skipping inactive block 0xb3000, status=0x0, size=0xf84...
Trying offset 0xb4000, status=0x0, size=0xf84...
Skipping inactive block 0xb4000, status=0x0, size=0xf84...
Trying offset 0xb5000, status=0x0, size=0xf84...
Skipping inactive block 0xb5000, status=0x0, size=0xf84...
Trying offset 0xb6000, status=0x0, size=0xf84...
Skipping inactive block 0xb6000, status=0x0, size=0xf84...
Trying offset 0xb7000, status=0x0, size=0xf84...
Skipping inactive block 0xb7000, status=0x0, size=0xf84...
Trying offset 0xb8000, status=0x0, size=0xf84...
Skipping inactive block 0xb8000, status=0x0, size=0xf84...
Trying offset 0xb9000, status=0x0, size=0xf84...
Skipping inactive block 0xb9000, status=0x0, size=0xf84...
Trying offset 0xba000, status=0x0, size=0xf84...
Skipping inactive block 0xba000, status=0x0, size=0xf84...
Trying offset 0xbb000, status=0x0, size=0xf84...
Skipping inactive block 0xbb000, status=0x0, size=0xf84...
Trying offset 0xbc000, status=0x0, size=0xf84...
Skipping inactive block 0xbc000, status=0x0, size=0xf84...
Trying offset 0xbd000, status=0x0, size=0xf84...
Skipping inactive block 0xbd000, status=0x0, size=0xf84...
Trying offset 0xbe000, status=0x0, size=0xf84...
Skipping inactive block 0xbe000, status=0x0, size=0xf84...
Trying offset 0xbf000, status=0x0, size=0xf84...
Skipping inactive block 0xbf000, status=0x0, size=0xf84...
Trying offset 0xf00000, status=0x0, size=0x1132c...
Skipping inactive block 0xf00000, status=0x0, size=0x1132c...
Trying offset 0xf20000, status=0xffff, size=0x1132c...
Prop    5010000     28
Prop    5010001    544
Prop    5010002   2204
Prop    5010003   1264
Prop    5010004     64
Prop    5010005     44
Prop    5010006    784
Prop    5010007     28
Prop    5010008     24
Prop    5010009     84
Prop    501000a    504
Prop    501000b    224 ?
Prop    501000c      4
Prop    501000d    288 $
Prop    501000e    448
Prop    4000000     40
Prop    4000001     40
Prop    4000002     40
Prop    4000003     40
Prop    4000004     40
Prop    4010000      4
Prop    4010001  16704 @A?
Prop    4010002      4
Prop    4010003  16704 @A?
Prop    4010004      4
Prop    4010005  16704 @A?
Prop    e000001   2048
Prop    e000002   2048 ????
Prop    e000003   1024 ????
Prop    e000004   6252 ????
Prop    e020001     56
Prop    e020002    484
Prop    e030000   1024
Prop    e040000      4
Prop    e060000      8 0000
Prop    e060001      4
Prop    e060002      8 0000
Prop    e060003      8 0000
Prop    2090000      4
Prop    e070000     64 Daniel A. Fort
Prop    e070001     64 Daniel A. Fort
Trying offset 0xf60000, status=0xffff, size=0x1aedc...
Prop    b000000 110256 <
- Camera model: ???
- Firmware version: 2.0.2 / 9.9.8 B8(3a)
- IMG naming: 100?????/IMG_5912.JPG
- User PS: CineStyle logNeutral EOSHD C-LOG


Quote from: critix on January 10, 2019, 06:15:49 PM
There is no information for Digic4 +.
Maybe this feature needs to be changed for these device models?

Right, here's what happens on the 1300D.

./prop_diag ROM1.BIN
Loading ROM1.BIN...
Scanning from 0x10a954000 to 0x10c954000...
Trying offset 0x12aeb00, status=0x7000000, size=0xe0...
- Camera model: ???
- Firmware version: ??? / ???
- IMG naming: 100?????/????0000.JPG
- User PS: ??? ??? ???


[EDIT] Noticed that this is missing (from the 1300D branch too) though it isn't a solution to the Digic 4+ issue.

src/propvalues.h
#define MODEL_EOS_1300D  0x80000404

a1ex

The property data structures were changed a bit.

EOS M (same as most other models):

xxd -e -s 0x99000 -l 0x1000 -e EOSM/ROM1.BIN   # note: my offsets differ

00099000: 0000ffff 00000f84 02000000 00000f78  ............x...
00099010: 00000000 000000f4 02000000 0000000c  ................
00099020: 0000ffff 02000001 0000002c 2e302e32  ........,...2.0.
00099030: 38420032 29613328 006f6600 0000000c  2.B8(3a).fo.....
...
00099f60: 02090002 0000000c 00000000 02090003  ................
00099f70: 0000000c 00000000 0000ffff 00ff0000  ................
00099f80: 0f000000 ffffffff ffffffff ffffffff  ................


1300D:

xxd -e -s 0xC29000 -l 0x1000 -e 1300D/ROM1.BIN

00c29000: 00000002 00000000 00000000 0000ffff  ................
00c29010: 00000f6c 02000000 00000f54 00000000  l.......T.......
00c29020: 000000bc 02000000 0000000c 0000ffff  ................
00c29030: 02000001 00000018 2e312e31 37330030  ........1.1.0.37
...
00c29f60: 0000ffff 00ff0000 0f000000 ffdfd0f4  ................
00c29f70: ffffffff ffffffff ffffffff ffffffff  ................


M50 (notice the "active" flag differs, too; offset 0 is a false friend):

xxd -e -s 0x19dc000 -l 0x4000 -e M50/ROM0.BIN

019dc000: 0000ffff ffffffff ffffffff ffffffff  ................
019dc010: ffffffff ffffffff ffffffff ffffffff  ................
019dc020: 00ffffff ffffffff ffffffff ffffffff  ................
019dc030: 00003e28 02000000 00003df0 00000000  (>.......=......
019dc040: 00000100 02000000 0000000c 0000ffff  ................
019dc050: 02000001 0000002c 2e302e31 34330031  ....,...1.0.1.34
...
019dfe10: 00000000 00000000 00000000 0000ffff  ................
019dfe20: 00ff0000 1f000000 ffffffff ffffffff  ................

critix

I tested and not work with :
xxd -e -s 0xC29000 -l 0x1000 -e 1300D/ROM1.BIN
It work with
xxd -e -s 0xc20000 -l 0x1000 -e 1300D/ROM1.BIN
00c20000: 00000002 00000000 00000000 0000ffff  ................
00c20010: 00000f6c 02000000 00000f54 00000000  l.......T.......
00c20020: 000000bc 02000000 0000000c 0000ffff  ................
00c20030: 02000001 00000018 2e312e31 37330030  ........1.1.0.37
00c20040: 29623028 00008700 02000005 00000018  (0b)............
00c20050: 2e342e34 37332036 29623028 00008700  4.4.6 37(0b)....
00c20060: 02000002 0000000c 00000000 02000003  ................
....

Canon 1300D, 500D, EOS M, EOS M2

dfort

Strange -- both addresses work over here:

xxd -e -s 0xc29000 -l 0x1000 -e 1300D/ROM1.BIN
00c29000: 00000002 00000000 00000000 0000ffff  ................
00c29010: 00000f6c 02000000 00000f54 00000000  l.......T.......
00c29020: 000000bc 02000000 0000000c 0000ffff  ................
00c29030: 02000001 00000018 2e312e31 37330030  ........1.1.0.37
00c29040: 29623028 00008700 02000005 00000018  (0b)............
00c29050: 2e342e34 37332036 29623028 00008700  4.4.6 37(0b)....
00c29060: 02000002 0000000c 00000000 02000003  ................



xxd -e -s 0xc20000 -l 0x1000 -e 1300D/ROM1.BIN
00c20000: 00000002 00000000 00000000 00000000  ................
00c20010: 00000f6c 02000000 00000f54 00000000  l.......T.......
00c20020: 000000bc 02000000 0000000c 0000ffff  ................
00c20030: 02000001 00000018 2e312e31 37330030  ........1.1.0.37
00c20040: 29623028 00008700 02000005 00000018  (0b)............
00c20050: 2e342e34 37332036 29623028 00008700  4.4.6 37(0b)....
00c20060: 02000002 0000000c 00000000 02000003  ................


a1ex

Yes, these offsets are changing at every camera startup, I think. There are a couple of property blocks with similar content, but only one of them is active (the one with the blue field 0000FFFF on most models; exception: M50, where the active block is marked with FFFFFFFF). Some of these property blocks are used by Canon for saving their settings (yes, they are reflashing the ROM at every shutdown), others appear to be fixed or possibly updated only on demand (e.g. calibration data or image capture configuration).

Fixes pushed, sorry for spoiling the fun...

dfort


a1ex

As I was cleaning up the M50/SX70 dumper source in order to publish it, I've noticed the same method could be used for all models, starting with the good old 5D. I ended up spending the entire weekend reworking this.

Previously, g3gg0 noticed the unreliability of Canon routines and tried to integrate a full-fledged FAT library (FullFAT), with some low-level SD driver written from scratch. For some reason, that low-level SD driver didn't work on my cameras, and since I wasn't able to understand its internals, I kept trying to debug the high-level Canon routines. Which worked, to some extent, on most of the models where they were present. On the M50/SX70, these routines were gone, so I had to find some other method.

Revisiting g3gg0's approach, I've noticed adapting it for M50, and then for all other EOS models, was just a matter of replacing his low-level routines (which are likely model-specific) with Canon's sector I/O routines (which are present in all EOS firmwares I've looked at, from DIGIC 2 to DIGIC 8 ). I did just that and tested the new dumper in QEMU:

Testing portable ROM dumper...
     5D: CF: ROM0.BIN: OK ROM1.BIN: OK
    5D2: CF: ROM0.BIN: OK ROM1.BIN: OK
    5D3: SD: ROM0.BIN: OK ROM1.BIN: OK
    5D4: SD: ROM1.BIN: OK SFDATA.BIN: OK
     6D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    6D2: SD: ROM0.BIN: OK ROM1.BIN: OK
     7D: CF: ROM0.BIN: OK ROM1.BIN: OK
   7D2M: SD: ROM1.BIN: OK
    40D: CF: ROM0.BIN: OK ROM1.BIN: OK
    50D: CF: ROM0.BIN: OK ROM1.BIN: OK
    60D: SD: ROM0.BIN: OK ROM1.BIN: OK
    70D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    77D: SD: ROM0.BIN: OK ROM1.BIN: OK
    80D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   400D: CF: ROM0.BIN: OK ROM1.BIN: OK
   450D: SD: ROM0.BIN: OK ROM1.BIN: OK
   500D: SD: ROM0.BIN: OK ROM1.BIN: OK
   550D: SD: ROM0.BIN: OK ROM1.BIN: OK
   600D: SD: ROM0.BIN: OK ROM1.BIN: OK
   650D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   700D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   750D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   760D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   800D: SD: ROM0.BIN: OK ROM1.BIN: OK
   100D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   200D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1000D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1100D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1200D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1300D: SD: ROM0.BIN: OK ROM1.BIN: OK
   EOSM: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
  EOSM2: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    M50: SD: ROM0.BIN: OK ROM1.BIN: OK
   SX70: SD: ROM0.BIN: OK ROM1.BIN: OK


Would this work on real hardware? Let's find out!

AUTOEXEC.BIN

(source: feb62ea fd2d938 2a15b7d with CONFIG_BOOT_FULLFAT=y CONFIG_BOOT_DUMPER=y CONFIG_BOOT_SROM_DUMPER=y)

I've tested the following:
- 5D2: 32GB CF
- 5D3: 32GB and 256MB SD
- 500D: 32GB (didn't work) and 256MB SD

An older version of this code was confirmed to work on M50 and SX70.

In QEMU, I've checked all EOS models from the test suite on the 256MB image (FAT16) and on a 8GB (FAT32) image. The only issue I could find was with 500D on the large image; figure out why.

I didn't address caching issues yet (I just left them disabled), so the new dumper is not going to be very fast (at least not yet).

EXFAT is not supported.




Actually, the story begins somewhere around this, when I was trying to fix the emulation of various CPU info registers (i.e. sync the emulation with the logs posted by various users). As most of the CPU info logs were actually screenshots, I thought "OK, I'll get a plain-text log from the 5D3 real quick". I took the camera out of the bag, compiled the portable codebase, enabled the CPUINFO functionality, made some small changes to save all that info to a log file, tested it in QEMU as usual, then ran it on the camera it without thinking too much. It seemed to work, but afterwards... the card went unreadable (and the camera did not boot any more from it either). The worst part - this was the moment I realized that card contained all my holiday photos! (as they were not downloaded yet).

What happened? The QEMU image was a small 256MB one (which Canon's bootloader I/O routines have no trouble with), so my "offline" test worked fine. In the camera I had a 32GB card formatted as FAT32. Turns out, these I/O routines do not like large filesystems. At all.

After imaging the raw card contents, I ran the same code on a good filesystem, to see exactly what kind of damage it did. Nearly the entire partition table was zeroed out! Both copies of the FAT! Luckily, the data area was largely unaffected. Needless to say - I've spent the next few days grepping the filesystem and writing custom file recovery scripts, as testdisk/photorec didn't work. In the end, I was able to recover nearly everything.

Yes, all of this damage was done "just" by trying to save a small log file.

This was my motivation for getting rid of these Canon I/O routines once for all.

dfort

EOSM - MD5 on ROM0 didn't match. Tried multiple times with cards from 1 to 32GB. However, dump seems fine and it works in QEMU. ROM0 md5 issue not reproducible in QEMU.
  Magic Lantern Rescue
----------------------------
- Model ID: 0x331 M
- Camera model: ???
- Firmware version: 2.0.2 / 9.9.8 B8(3a)
- IMG naming: 100?????/IMG_5914.JPG
- User PS: CineStyle logNeutral EOSHD C-LOG
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF0C0000
- card_bootflags 10a63c
- boot_read/write_sector 10aec8 10afbc
- 102798 Card init => 2
- Dumping ROM0... 100%
- MD5: 8f2ab35008ead4f09ce9a2a5f9ce42f2
- Dumping ROM1... 100%
- MD5: 351d4472390b9074162692e7f039cc88
- 0: \n**** SROM(SIO%d) Menu ****\n
- 107c64: \n**** SROM Menu ****\n
- 107B4C: tag c022c000
- sf_init 107B48
- 10717c: Read Address[0x%06x-0x%06x]:0x
- 106A94: tag c0820000
- sf_command_sio 106A44
- Reading serial flash... 100%
- Writing SFDATA.BIN... 100%
- MD5: 395f84348339a032ce14b67998c74af8
- DONE!


EOSM2 - Seemed to work perfectly (I'm still having Mac QEMU issues with all dumps from this camera)
  Magic Lantern Rescue
----------------------------
- Model ID: 0x355 M2
- Camera model: ???
- Firmware version: 1.0.3 / 6.0.6 7A(2b)
- IMG naming: 100?????/IMG_1436.JPG
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF0C0000
- card_bootflags 10a6e4
- boot_read/write_sector 10af38 10b02c
- 102264 Card init => 2
- Dumping ROM0... 100%
- MD5: bae9e17718452b4c5cd904a0004615d2
- Dumping ROM1... 100%
- MD5: 7183d78b02c51297fb465b89efee71f1
- 107860: \n**** SROM(SIO%d) Menu ****\n
- 107734: tag c0400000
- sf_init 107734
- 106f44: Read Address[0x%06x-0x%06x]:0x
- 1066AC: tag c0820000
- sf_command_sio 10666C
- Reading serial flash... 100%
- Writing SFDATA.BIN... 100%
- MD5: f193d5bac8f48049948f62d2fdfdb482
- DONE!


700D - perfect
  Magic Lantern Rescue
----------------------------
- Model ID: 0x326 700D
- Camera model: ???
- Firmware version: 1.1.5 / 3.0.2 21(01)
- IMG naming: 100?????/IMG_4984.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF0C0000
- card_bootflags 10ab54
- boot_read/write_sector 10b3e0 10b4d4
- 10266C Card init => 2
- Dumping ROM0... 100%
- MD5: d5929b7a5ad99f511fc83d7d0b48b85f
- Dumping ROM1... 100%
- MD5: 6a5b5cdf62a73870f72b06e467219ed1
- 0: \n**** SROM(SIO%d) Menu ****\n
- 107e48: \n**** SROM Menu ****\n
- 107D30: tag c022c000
- sf_init 107D2C
- 107360: Read Address[0x%06x-0x%06x]:0x
- 106C78: tag c0820000
- sf_command_sio 106C28
- Reading serial flash... 100%
- Writing SFDATA.BIN... 100%
- MD5: d83d31e10ea740d670c1636a3233ff4c
- DONE!


7D (didn't work) screen blanks out after ROM0 md5. Tried with various sized cards from 32MB to 4GB. Saves ROM0.BIN but md5 checksum doesn't match. Also saves a partial ROM1.BIN.

Ottoga

@Alex / @Dfort,

Can I please have a reminder on the testing process and I will test it on my 7D-1. and will advise the results.

EOS 7D.203, EFS 55-250mm, EF 75-300 III, Tamron 16-300 DiII VC PZD Macro, SpeedLite 580EX II.

critix

Alex, can you make the FIR file for those who do not have bootflag enabled? I want to test the 1300D.
Thank you
Canon 1300D, 500D, EOS M, EOS M2

a1ex

ROM0 MD5 not matching: expected, short answer in first post, will look up the long answer(s) later.

7D: that's actually big progress; IIRC it didn't even start. Now it was able to create a file, if I understand well.

Maybe it's some supervisor (MPU? second core?) that's turning off the camera after some inactive time. The 5D3 does that after several seconds if you run the dumper with battery door open. Guess: maybe the dumper just needs to be faster?

I've spent another couple of hours to track down caching issues. Updated autoexec.bin from previous post - is it still working? I've only tested on 5D3. Expecting it to be much faster.

Sorry, have to run now, see you tonight.

dfort

Busy day but I finally got a chance to give the new dumper on the 7D.

  Magic Lantern Rescue
----------------------------
- Model ID: 0x250 7D
- Camera model: Canon EOS 7D
- Firmware version: ??? / ???
- IMG naming: 100EOS7D/IMG_0000.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF010000
- card_bootflags 109a18
- boot_read/write_sector 109d54 109d64
- Patching 104294 from e3500001 to e3500000
- 104254 Card low-level init => F4240
- 1026EC Card init => 0
- Patching 1026FC from e3510001 to e3510000
- 1026EC Card init #2 => 1
- Dumping ROM0... 100%
- MD5: 55edc9e76de2ba7bae387f77d9a9c7cc
- Dumping ROM1... 100%
- MD5: ef25a835383698be6888d4395088a8e2
- No serial flash.
- DONE!


Screen goes dark right away so I wasn't sure when it was finished. First time I didn't wait long enough but second time I just let it run for a couple minutes. Used a 512MB card. ROM0 md5 mismatch as expected but the dump looks good.

[EDIT] Checked 700D, EOSM and EOSM2 -- only the EOSM is showing a different md5 checksum on ROM0, the other cameras had perfect checksums.