Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: a1ex on January 25, 2016, 09:29:53 AM

Title: Portable ROM dumper
Post by: a1ex on January 25, 2016, 09:29:53 AM
This is a small program that saves the contents of your camera's ROM contents on the card.
It won't modify your camera - at least, not intentionally :)

This is *NO* Magic Lantern build

Latest download: autoexec.bin (http://a1ex.magiclantern.fm/debug/portable-rom-dumper/autoexec.bin) (2020Aug17, updated for EOS R/RP/90D/250D - only after enabling bootflag via UART)

DIGIC 4+:  1300D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMP1300D.FIR)  2000D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMP2000D.FIR)  4000D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMP4000D.FIR)
DIGIC 6:  5D4 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_5D4.FIR)  750D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP750D.FIR)  760D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP760D.FIR)  80D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_80D.FIR)
DIGIC 7:  200D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP200D.FIR)  6D2 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_6D2.FIR)  77D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_77D.FIR)  800D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP800D.FIR)
DIGIC 8:  EOSR EOSRP 250D 90D M6 II G7X III M50 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_M50.FIR)  SX70 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMPSX70.FIR)  SX740 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMPSX740.FIR)
Master/Slave:  5DS (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_5DS.FIR)  5DSR (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP5DSR.FIR)  7D2 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_7D2.FIR) 7D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP__7D.FIR)
Oldies:  1000D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMP1000D.FIR)  30D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_30D.FIR)  400D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP400D.FIR)  40D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_40D.FIR)  450D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP450D.FIR)  5D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP__5D.FIR)

- built from c019793 (https://bitbucket.org/hudson/magic-lantern/branch/recovery) with CONFIG_BOOT_FULLFAT=y CONFIG_BOOT_DUMPER=y CONFIG_BOOT_SROM_DUMPER=y
- green = confirmed working (either the last version, or a slightly older one)
- blue = not tested, but likely to work (based on other similar models, or on previous tests)
- purple = not tested, there may be surprises, but fixable (based on previous tests)
- orange = not tested, but unlikely to work (based on previous failures)
- red = not working, no idea how to fix

Supported cameras:
- tested in QEMU: 5D, 5D2, 5D3, 5D4, 6D, 6D2, 7D, 7D2, 40D, 50D, 60D, 70D, 77D, 80D, 400D, 450D, 500D, 550D, 600D, 650D, 700D, 750D, 760D, 800D, 100D, 200D, 1000D, 1100D, 1200D, 1300D, EOSM, EOSM2, M50, SX70.
- other models from the same generation may work, too (see the FIR list for models not yet running ML).

Not supported:
- cameras running PowerShot firmware (including EOS M3, M5, M6, M10, M100)

Requirements:
- a memory card formatted as FAT12/16/32 (exFAT won't work)
- for autoexec.bin: boot flag enabled in the camera (e.g. ML already installed) + bootable card (EOSCard, MacBoot, make_bootable.sh, QEMU image)
- FIR versions do not require any boot flags (just place on the card and run Firmware Update)
- check MD5 checksums after dumping (important!)

Source code:
- recovery (https://bitbucket.org/hudson/magic-lantern/branch/recovery) branch
- compiled from platform/portable.000 with CONFIG_BOOT_FULLFAT=y, CONFIG_BOOT_DUMPER=y and CONFIG_BOOT_SROM_DUMPER=y




Old limitations (for 2018 and earlier dumpers only):

Quote

Requirements:
- a very small SD card or filesystem (important!)
- no important files on the card (these routines are buggy and may destroy the filesystem!!!)
- boot flag enabled in the camera (e.g. ML already installed) + bootable card (EOSCard, MacBoot, make_bootable.sh, QEMU image)
- alternative: FIR version (https://builds.magiclantern.fm/#rom-dumpers) does not require any boot flags
- check MD5 checksums after dumping (important!)

Formatting a larger card at a much lower capacity (e.g. 256MB) does the trick. For example, you can write the SD image that comes with QEMU (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/sd.img.xz) to your SD or CF card (follow this guide (https://thepihut.com/blogs/raspberry-pi-tutorials/17789160-backing-up-and-restoring-your-raspberry-pis-sd-card)). This image contains the portable display test (http://www.magiclantern.fm/forum/index.php?topic=14732) and is bootable (so, you can test it in the camera straight away).





Original post:

Lately I've got a few softbricked cameras to diagnose, and struggled a bit with the ROM dumper from bootloader: it wasn't quite reliable. A while ago, g3gg0 reimplemented it with low-level routines (which worked on his camera, but not on mine). Today I looked again at the old approach, and it looks like the file I/O routines from bootloader had to be called from RAM, not from ROM.

So, I've updated the code and need some testing. I've emulated this in QEMU, but the results may be different on real hardware.

What you have to do:

- download autoexec.bin (http://a1ex.magiclantern.fm/debug/portable-rom-dumper/autoexec.bin)
- place it on a card without any important data on it (it might corrupt the filesystem if anything goes wrong)
- the display looks roughly like this: http://www.magiclantern.fm/forum/index.php?topic=14732
- after it's finished, look on the card, and you will find 4 files: ROM[01].BIN and ROM[01].MD5.
- you don't have to upload them, just check the MD5 checksum:
  - Windows: you may use http://www.winmd5.com/ (http://winmd5)
  - Mac, Linux: md5sum -c *.MD5
- repeat the test on the same card (let it overwrite the files), then on a card with different size (and maybe different filesystem).

Some cameras have only ROM1 connected, so dumping ROM0 will give just random noise. In this case, the ROM0 checksum may not match, but that's OK.

The ROM dumper should recognize all ML-enabled cameras, except for 5D2, 50D and 500D. These old models do not appear to have file writing routines in the bootloader (or, at least I could not find them). The QEMU simulation works even on exotic models like 1200D or EOS M2.

So, you don't have to upload any files or screenshots. Simply verify the MD5 checksums on your PC (if in doubt, paste the md5sum output).

That's it, thanks for testing.
Title: Re: Portable ROM dumper
Post by: nikfreak on January 25, 2016, 06:46:35 PM
Both 100D.100B and 70D.111A get stuck at "Dumping ROM0...". No LED activity neither any sign of placing / writing files to sdcard on both cams.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on January 25, 2016, 07:04:01 PM
650D
Magic Lantern Rescue
----------------------------
- Model ID: 0x301 650D
- Boot flags: FIR=0 BOOT=-1 Ram=-1 UPD=-1
- ROMBASEADDR: oxFF0C0000
- Dumping ROM0...


... and doing nothing else. No LED activity, no file.

7D: Pretty much the same. Different strings for Model ID and ROMBASEADDR of course.
Title: Re: Portable ROM dumper
Post by: mk11174 on January 26, 2016, 09:17:59 AM
700D works and gives Files, But ROM1.MD5 checksum changes each time, ROM0.MD5 stays the same.

I used winmd5free, I ran 1 test, got the checksum from ROM0.MD5 and copy and pasted it in the Original text box, then ran test again and loaded new ROM0.MD5 and checksum was same as first.

Did same steps for ROM1.MD5, but checksum was differ.

Did I even do it correctly or were you wanting something else?
Title: Re: Portable ROM dumper
Post by: a1ex on January 26, 2016, 01:00:04 PM
On 700D, ROM0 does not appear to be connected, so you are interested only in ROM1.

So, in the Browse dialog, you select ROM1.BIN, and check if the number in the "Current file MD5 checksum value" box is the same as the one from ROM1.MD5. However, I already know, from the user with the bricked 700D (http://www.magiclantern.fm/forum/index.php?topic=16487), that it doesn't work well on this camera :(

Any 600D user can try it?
Title: Re: Portable ROM dumper
Post by: mk11174 on January 26, 2016, 04:30:09 PM
Yeah, no same  :(

Only can try with 1 card though, don't have any extra cards at this time.  :-[
Title: Re: Portable ROM dumper
Post by: mk11174 on January 26, 2016, 08:47:15 PM
600D

Model ID: 0x286 600D
Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
ROMBASEADDR: 0XFF010000
Dumping ROM0...

nothing on card no led
Title: Re: Portable ROM dumper
Post by: a1ex on March 04, 2016, 07:02:09 PM
Made some updates. Can you try again on 600D, 650D and 700D?

I don't expect it to work on 7D, but who knows.

edit: it's much slower this time (5-10 minutes per file), and there's no LED activity
Title: Re: Portable ROM dumper
Post by: Walter Schulz on March 05, 2016, 11:24:27 AM
7d hangs after line "- Dumping ROM 0. . .".

650D does simply fly through "-Dumping ROM 0. . .". Much too fast, about 20 seconds between ROM and MD5 line.
Hangs after ROM 1 message. MD5 file seems to be corrupted. Windows wants to check the card, other files seems to be out of order, too.

Boys, take warning about card corruption seriously!
Title: Re: Portable ROM dumper
Post by: DeafEyeJedi on July 26, 2016, 12:03:49 PM
Quote from: Walter Schulz on March 05, 2016, 11:24:27 AM
7d hangs after line "- Dumping ROM 0. . .".

I can also confirm this as well for the 7D.
Title: Re: Portable ROM dumper
Post by: a1ex on July 29, 2016, 12:06:45 PM
I managed to get correct ROM dumps on 5D3 on an 8GB card that was previously failing (filesystem corruption). It's still not perfect, the first file saved ends up corrupted, but seems to be a small step forward.

The issue: Canon's bootloader routines for file I/O copy the data to some cacheable (!) memory; when that buffer reaches 0x4000 bytes, it's written to card using DMA. However, patching Canon code to use an uncacheable memory pointer didn't help.

Before, I was disabling data and instruction caches, and I could get correct dumps on an old 256MB (yes, MB) card.

Today I've tried to disable the write buffer (SCTLR bit 3), but that didn't seem to help. What helped was disabling the cache and write buffer in the memory regions that are set up at camera startup (see http://magiclantern.wikia.com/wiki/Memory_map ):

static inline void disable_all_caches()
{
    asm(
        "mrc p15, 0, r0, c1, c0, 0\n"   /* read SCTLR */
        "bic r0, #0x4\n"                /* disable data and unified caches */
        "bic r0, #0x8\n"                /* disable write buffer */
        "bic r0, #0x1000\n"             /* disable instruction cache */
        "mcr p15, 0, r0, c1, c0, 0\n"   /* write back SCTLR */
        "mov r0, #0\n"
        "mcr p15, 0, r0, c2, c0, 0\n"   /* disable data cache on all memory regions */
        "mcr p15, 0, r0, c2, c0, 1\n"   /* disable instruction cache on all memory regions */
        "mcr p15, 0, r0, c3, c0, 0\n"   /* disable write buffer on all memory regions */
        : : : "r0"
    );
}


Still, the first file is always saved with incorrect checksum on the 8GB card. To work around this, I dump the two ROMs twice => 4 files. First has incorrect checksum, next 3 are fine, and the behavior is very consistent.

All 4 files end up correct on the 256MB card.

I've updated the autoexec.bin from the first post, and I'd like you to try it on all ML-enabled cameras, except 7D.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on July 29, 2016, 08:12:59 PM
(http://picload.org/image/rrocplrl/img_20160729_195800.png)

Used a 1 GB card formatted with cam.
EOScard with "classic" ML.
Replaced autoexec.bin.
Inserted into cam and - after a while - done.

Anything else to test on 650D? Other card sizes, file systems?
Title: Re: Portable ROM dumper
Post by: a1ex on July 29, 2016, 09:07:54 PM
Yes, on the card, from a Linux or Mac shell, run this command: md5sum -c *.MD5

For Windows, cygwin might have the equivalent command, otherwise you may have to use a GUI.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on July 29, 2016, 09:34:08 PM
Or commandline (Powershell 4.0)

PS H:\> get-filehash ROM*.BIN -Algorithm MD5

Algorithm       Hash
---------       ----
MD5             D5929B7A5AD99F511FC83D7D0B48B85F
MD5             A1C1204A28D2453F11374C8690CB0C35
MD5             D5929B7A5AD99F511FC83D7D0B48B85F
MD5             A1C1204A28D2453F11374C8690CB0C35


EDIT: BTW, is it intended to use lowercase for first bin and uppercase for the rest?

rom0a.bin
ROM1A.BIN
ROM0B.BIN
ROM1B.BIN
Title: Re: Portable ROM dumper
Post by: a1ex on July 29, 2016, 10:02:39 PM
Looks like it finally worked, right?

The case difference is not intended. Maybe it has to do with the corruption I'm experiencing for the first file?
Title: Re: Portable ROM dumper
Post by: nikfreak on December 06, 2016, 10:49:05 PM
canon released a bunch of firmware updates recently. Any plans on updating the portable dumper so it works for all cameras?
Title: Re: Portable ROM dumper
Post by: a1ex on December 06, 2016, 10:55:13 PM
Quote from: a1ex on November 01, 2016, 08:38:20 PM

[...]

Testing portable ROM dumper...
   5D: skipping
  5D2: skipping
  5D3: ROM0.BIN: OK ROM1.BIN: OK
  5D4: ROM1.BIN: OK
   6D: ROM0.BIN: OK ROM1.BIN: OK
   7D: ROMs not saved
7D2M: ROM1.BIN: OK
  40D: skipping
  50D: skipping
  60D: ROM0.BIN: OK ROM1.BIN: OK
  70D: ROM0.BIN: OK ROM1.BIN: OK
  80D: ROM1.BIN: OK
400D: skipping
450D: skipping
500D: skipping
550D: ROM0.BIN: OK ROM1.BIN: OK
600D: ROM0.BIN: OK ROM1.BIN: OK
650D: ROM0.BIN: OK ROM1.BIN: OK
700D: ROM0.BIN: OK ROM1.BIN: OK
750D: ROM1.BIN: OK
760D: ROM1.BIN: OK
100D: ROM0.BIN: OK ROM1.BIN: OK
1000D: skipping
1100D: ROM0.BIN: OK ROM1.BIN: OK
1200D: ROM0.BIN: OK ROM1.BIN: OK
EOSM: ROM0.BIN: OK ROM1.BIN: OK

[...]

Title: Re: Portable ROM dumper
Post by: nikfreak on December 08, 2016, 12:03:05 PM
Can you post a dumper for 7D1 please?
Title: Re: Portable ROM dumper
Post by: DeafEyeJedi on December 08, 2016, 06:53:23 PM
Quote from: nikfreak on December 08, 2016, 12:03:05 PM
Can you post a dumper for 7D1 please?

+1 as this should open some doors for newer firmwares that were released by Canon recently.
Title: Re: Portable ROM dumper
Post by: nikfreak on January 02, 2017, 02:47:53 PM
Quote from: nikfreak on December 08, 2016, 12:03:05 PM
Can you post a dumper for 7D1 please?

or maybe update the portable binary to save the 7D ROM?
Title: Re: Portable ROM dumper
Post by: a1ex on January 02, 2017, 03:28:50 PM
I don't have one ready; the portable codebase probably requires some sort of IPC communication.

Understanding it could be useful for other things as well (QEMU, 7D2, running code on master without FIR).

On 7D, I don't even know how to blink the LED from bootloader context (source) (http://www.magiclantern.fm/forum/index.php?topic=17610), but can be found out from the dm-spy branch (binary search to find out where the LED starts to blink with the usual register commands, to see what Canon code does).

If you have enough patience, g3gg0 committed some QR code experiments on the portable codebase. Might be useful.
Title: Re: Portable ROM dumper
Post by: dfort on February 02, 2017, 12:46:46 AM
Used the portable dumper to get a 700D.115 ROM1.BIN. Would this be the first step to porting a firmware update?

Notes: An exfat formatted card didn't work and neither did an old 8GB card but a fairly new 32GB SanDisk Extreme PRO worked--the second time I tried it. The first time it only saved the ROM0B.MD5 file. Interesting that the card was renamed to EOS_DEVELOP.
Title: Re: Portable ROM dumper
Post by: dfort on October 20, 2017, 10:23:02 PM
Got my hands on a 7D and the first thing I tried was to get a dump of the latest 2.0.6 firmware just to see if some of the stubs that I'm not finding porting 2.0.3 to the crop_rec_4k branch. Yeah, that's a long shot. Of course I hit the wall with the Portable ROM dumper. I also tried the DUMP1300.FIR and dumperM2.fir, another long shot that didn't work. Is there any way to dump the 7D ROM? There must be but maybe the original porting notes are long forgotten? I couldn't dig up anything in previous forum posts.
Title: Re: Portable ROM dumper
Post by: dfort on October 21, 2017, 07:16:38 AM
Just an interesting observation. On the 7D the portable ROM dumper works fine in QEMU but not on the camera!

(https://farm5.staticflickr.com/4447/37565757710_d5786ecbc4.jpg) (https://flic.kr/p/ZeyjZL)

Of course to run QEMU you need a ROM dump. That's a Catch 22.
Title: Re: Portable ROM dumper
Post by: nikfreak on October 21, 2017, 10:59:02 AM
check also my requests above.
You'll be amazed that adtg_gui makes problems, too.
For the moment I'd rather focus on getting dual_iso as well as lossless compression to work (instead of adding support for 2.0.6)
Title: Re: Portable ROM dumper
Post by: dfort on October 21, 2017, 07:29:30 PM
I'm just trying to get familiar with the 7D and going through every stub and constant is one way to do that. Updating the firmware at this point isn't a priority, getting it ported to crop_rec_4k is, though it won't be as easy as the 6D and 650D. Of course if you're working on it the odds are good.

Back on topic, I did a little more digging into the recovery branch and found a way to compile a portable ROM dumper.

# recovery (portable display test, ROM dumper, CPU info...)
hg update recovery -C
cd ../../platform/portable.000
make install ML_MODULES_DYNAMIC= CONFIG_BOOT_DUMPER=y


With the latest commit on the recovery branch [1cfee5fb3c40 (https://bitbucket.org/hudson/magic-lantern/commits/1cfee5fb3c40917f546d6e581434833ddcfcf196)] all I get is noise on the screen, this happens in both QEMU and on the camera.

(https://farm5.staticflickr.com/4494/23982597168_39a5363475.jpg) (https://flic.kr/p/Cxg5AC)

This first appeared in commit 987a55497b04 (https://bitbucket.org/hudson/magic-lantern/commits/987a55497b04f9bd6f722e8845a0196677f913d9), disp_dump: added experimental display based dumper.

So stepping back one commit I can make a portable ROM dumper that works in QEMU but not on the camera. It is a little different than the pre-compiled dumper posted by a1ex on the first post of this topic.

(https://farm5.staticflickr.com/4488/37803264672_8c8e720540.jpg) (https://flic.kr/p/ZAxBxo)

Regressing all the way back to the big bang of the recovery branch I still couldn't get a ROM dumper that works in camera on the 7D.

Another issue I've been having is not being able to get to the Canon menu, much less the ML menu, in QEMU but that's a different topic.
Title: Re: Portable ROM dumper
Post by: a1ex on October 21, 2017, 09:51:58 PM
Well, 7D is a different beast. ML was ported to this camera a long time ago - check the old wiki (http://magiclantern.wikia.com/wiki/7D_support) for notes.

The recent dumper experiments are for 5DS (and they are also be useful for 7D2 (http://www.magiclantern.fm/forum/index.php?topic=13746.msg190105#msg190105)). Both are dual digic models, with the same quirk: the portable ROM dumper works in QEMU, but not on the camera. You need a second camera to decode the blinks.

Emulation would require starting two instances of QEMU and modeling the communication between them somehow. I remember reading about a QEMU fork with multiprocessor support (maybe the Xilinx version, or something based on that). Low priority from my side, for now.
Title: Re: Portable ROM dumper
Post by: dfort on October 21, 2017, 10:50:39 PM
I found old Google Group discussions like this one (https://groups.google.com/forum/#!topic/ml-devel/Dh2jGGyh280) and this (https://groups.google.com/forum/#!topic/ml-devel/ljZ4Ko8lu30) that pre-dates the old wiki.

Then there's this CHDK universal dumper (http://chdk.wikia.com/wiki/Canon_Basic/Scripts/Dumper) that looks interesting. It runs on "CanonBasic (http://chdk.wikia.com/wiki/Canon_Basic)" scripting language and is listed as working on the 7D as well as most other EOS cameras. I tried following the tutorials but no luck.

Oh well, though I discovered something new but what's old is new to me.
Title: Re: Portable ROM dumper
Post by: a1ex on October 21, 2017, 11:00:03 PM
EOS firmware doesn't have Canon Basic, though DIGIC 5 models apparently have a similar scripting language (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190986#msg190986). It's not present on DIGIC 6, unlikely to be in D7, so for me there's very little reason to look into it.

Canon Basic dumper is only for PowerShot firmware (that includes EOS M3 and newer models).
Title: Re: Portable ROM dumper
Post by: g3gg0 on October 22, 2017, 01:33:36 AM
there is an advanced display based dumper available, allowing reasonable transfer speeds.

here a vid: https://www.youtube.com/watch?time_continue=117&v=9mNhph9BfNA
the current implementation is quite a bit faster and doesnt take so long during dot calibration iirc.

how it works:
"master" camera booted in dumper, displays some patterns.
"slave" camera, in video mode, scans the screen and waits for the dots to appear.
after they went through setup, ROM content is shown on screen and slave camera writes a plain ROM image to its card structured dump file, that must be "assembled" on PC later.

how to use:
- get the recovery branch (https://bitbucket.org/hudson/magic-lantern/branch/recovery) and build the dumper (make sure disp_dump() is being called) for your camera to dump (lets call it master)
- save the "rom_dump" module sources
- switch to unified and compile the "rom_dump" module there
- put it into your fully supported camera (lets call it slave)
- master: boot rescue system
- slave: aim your lens towards the master camera
- slave: fine tune the focus to get a good separation between dots (check corners too!)

hint: if the "all dot" display time on master is too short for proper lens setup, reboot it and do one more phase.
generally its a good idea to simply let the master run its dumping process and checking on slave that all positions (4 corners and center)
give a proper white dot and close to black lines inbetween

- as soon the "all dot" display has fully faded on master and you have a black screen, start the module on slave (Debug -> Dump memory)
- slave: waits for the first 16 "setup dots" to appear one after another
- slave: waits for corner dots to calculate dimensions
- slave: waits for all dots

then it should work.
later when you are done with dumping, use the unpack_block (gcc -o unpack_block -c unpack_block.c) utility to re-assemble a proper ROM file.

hints:
- i worked with a zoom lens which worked fine
- disable IS (!!) as it will slightly move all the time
- use a power supply, it will take some time ;)  (dont remember, maybe 4 hours?)
- when you ran out of battery, modify the dump area in the dumper and start where it ended before
- when you have multiple partial rom dumps (DUMP.BIN) due to some troubles, just concatenate all files using e.g. cat command and pass it to unpack_block
- unpack_block will only pick blocks that weren't damaged and have a valid CRC
- if a fly passes by, then you might miss a block or two ;)
- if you constantly have trouble due to lens quality or a scratch in display, head to disp_dump.c (https://bitbucket.org/hudson/magic-lantern/commits/987a55497b04f9bd6f722e8845a0196677f913d9?at=recovery#Lsrc/disp_dump.cT16) and play with the values (e.g. increase BIT_WIDTH/BIT_SPACING and decrease BIT_COUNT_* so all will fit on screen)
- when modifying parameters in the dumper, the module will autodetect these changes (transmitted via setup dots)
Title: Re: Portable ROM dumper
Post by: dfort on October 22, 2017, 02:50:22 AM
Wow, that is so cool. I've got to try it.

So the 7D "master" is set up and working properly because it is displaying the patterns. I thought something was wrong, then again I didn't wait 4 hours for it to finish running "The Matrix" like code.

I take it that the "slave" can be another ML enabled camera and doesn't have to be the same model, all that's need is for it to run the "rom_dump" module. That rom_dump module is in the recovery branch but you're saying it needs to be compiled in the unified branch--Hum, ok. Is the EOSM a fully supported camera? I've got power supplies for both the 7D and EOSM.

Need to do this when I've got plenty of time and when there are no flies in the room.  ;D
Title: Re: Portable ROM dumper
Post by: g3gg0 on October 22, 2017, 10:51:32 AM
Quote from: dfort on October 22, 2017, 02:50:22 AM
So the 7D "master" is set up and working properly because it is displaying the patterns. I thought something was wrong, then again I didn't wait 4 hours for it to finish running "The Matrix" like code.
wondering what you want with the 7D - do you mean 7D2?

Quote from: dfort on October 22, 2017, 02:50:22 AM
I take it that the "slave" can be another ML enabled camera and doesn't have to be the same model, all that's need is for it to run the "rom_dump" module. That rom_dump module is in the recovery branch but you're saying it needs to be compiled in the unified branch--Hum, ok. Is the EOSM a fully supported camera? I've got power supplies for both the 7D and EOSM.

exactly, any camera where the raw backend works, should be fine.
i used the raw backend because of the resolution you get there. HD bufs could also work, but i felt like raw is the more supported choice ;)

just try it and report :)

when starting the module, it needs a possibly pitch black image. at least everything should have the same dark color.
it then does a full-screen averaging to get the base (off) color. then it waits for some dots to appear. 16 dots to be exact.
the significant jump in brightness is considered as "on" level.

after all 16 dots were located, they are used to communicate the dimensions of the matrix.
and then the matrix size is determined. etc.
Title: Re: Portable ROM dumper
Post by: 12georgiadis on October 22, 2017, 01:32:54 PM
Quote from: g3gg0 on October 22, 2017, 10:51:32 AM
wondering what you want with the 7D - do you mean 7D2?
G3gg0, I think it's about the 7DmkI which can become beast with a great crop_rec_4k port. It has the potential to handle all the features seen with the 5dmkIII.

Title: Re: Portable ROM dumper
Post by: g3gg0 on October 22, 2017, 06:05:58 PM
hmh why then run the rom dumper - you can simply run ML ?
Title: Re: Portable ROM dumper
Post by: dfort on October 22, 2017, 06:36:34 PM
I'd like to take a look at the 2.0.6 firmware for the 7D. Maybe there's something in there that is missing from 2.0.3? Doing a firmware update on this camera seems to be at the bottom of everyone's list so I'm not really pushing that though I do tend to be a bottom feeder when it comes to things to do in ML.

Besides, if running that master/slave setup is the way to get a dump out of some of the new cameras it wouldn't hurt getting familiar with the process.
Title: Re: Portable ROM dumper
Post by: g3gg0 on October 22, 2017, 06:54:33 PM
okay then it is experimenting with stuff, got it :)
Title: Re: Portable ROM dumper
Post by: dfort on October 23, 2017, 06:28:22 AM
Having issues getting this to work. First of all, here's my setup:

(https://farm5.staticflickr.com/4458/24019142318_ba236c85d5.jpg) (https://flic.kr/p/CAuocq)

The 7D is the "master" and the EOSM is the "slave." The image looks nice and sharp--using a 55mm f/2.8 Nikon Micro Nikkor. Never mind the low battery warning, that always happens with this power adapter using the wimpy 120 volts AC here in the U.S.

(https://farm5.staticflickr.com/4501/37823285006_ca73562db4.jpg) (https://flic.kr/p/ZCjdTy)

I tried several times but the display always shows "C0" and there is no card write activity. Screenshot didn't work with rom_dump running so I had to do a phone photo.

(https://farm5.staticflickr.com/4448/37872115941_31db62fffb.jpg) (https://flic.kr/p/ZGCuCe)

Looking at the code it should be giving feedback on the screen like "waiting for SEQAAAA", "waiting for idle", "waiting for XRES", etc. If it is having problems it should be printing out something like "Data calib dot 2 detection failed" but I'm getting nothing like that, just "C0".

Quote from: g3gg0 on October 22, 2017, 01:33:36 AM
- if you constantly have trouble due to lens quality or a scratch in display, head to disp_dump.c (https://bitbucket.org/hudson/magic-lantern/commits/987a55497b04f9bd6f722e8845a0196677f913d9?at=recovery#Lsrc/disp_dump.cT16) and play with the values (e.g. increase BIT_WIDTH/BIT_SPACING and decrease BIT_COUNT_* so all will fit on screen)
- when modifying parameters in the dumper, the module will autodetect these changes (transmitted via setup dots)

I doubt theres any problem with the lens and there's no scratch in the display. Everything looks nice and sharp and all the dots seem to be showing up on the screen. I guess it wouldn't hurt playing around with the values but thought I'd report my first failed attempt.

Any tips like what ISO setting and shutter angle worked best? Maybe it needs FPS override? The EOSM always defaults to mv720 at 29.97 so maybe that's the problem?
Title: Re: Portable ROM dumper
Post by: Kharak on October 23, 2017, 09:41:14 AM
Sorry, but is that the Matrix ? :)
Title: Re: Portable ROM dumper
Post by: dfort on October 23, 2017, 01:07:01 PM
The Matrix is a 1999 science fiction action film. Google "Matrix code" and you'll see a resemblance between the graphics used in the movie and the patterns on the LiveView screen of the "master" camera when using the display based ROM dumper.

Back on topic -- switched the "slave" camera to a 5D3.113 and it started working right away. One camera is on AC, the other on battery. I'm starting by doing a dump of the 7D.203 to see if it matches the regular ML dump before venturing into the still unknown 2.0.5 and 2.0.6 firmware. I've got a rather busy week so it might take me a while before I have anything to report.

[EDIT] Other than to report that a fully charged battery won't make it all the way through the process so it will either have to be done in sections following the instructions posted by @g3gg0 or both cameras need to be on external power supplies. The 4MB file processed with unpack_block and disassembled with disassemble.pl (http://www.magiclantern.fm/forum/index.php?topic=12177.0) matches perfectly with the ML produced ROM dump.
Title: Re: Portable ROM dumper
Post by: g3gg0 on October 23, 2017, 04:47:22 PM
okay thanks for your feedback!

so on the EOS M the module fails to find the C0-dot in the raw area. maybe i am missing some code to init raw buffer access.
never checked how to do that on EOS M.

good to hear that it finally worked :)
Title: Re: Portable ROM dumper
Post by: nikfreak on October 23, 2017, 05:38:42 PM
dfort, send me the 2.0.6 dump as private link by PM plz once you have it. Thx in advance
Title: Re: Portable ROM dumper
Post by: dfort on October 23, 2017, 06:25:19 PM
Sure will @nikfreak it will probably take me a few days until I've got another large block of time to run this. Since it takes so long and the test with the 7D.203 went so well I'm going straight for the 7D.206.

@g3gg0 - It would be nice if it worked on the EOSM, it is working perfectly on the 5D3.113.

One thing I noticed is that if I compile for the EOSM in the recovery branch that camera also calls disp_dump() and starts displaying the dots on the screen instead of dumping the ROM. No problem if I back up to a changeset before you added the experimental display based dumper.
Title: Re: Portable ROM dumper
Post by: g3gg0 on October 23, 2017, 07:32:12 PM
btw, the reason why i made this weird dumper...

i wanted to dump the 5Ds and i didnt want to use a GPIO to toggle a LED that is sampled with a LDR, then fed into a sound card, then saved as audio, then reconstruct the data.
with a data rate that is smaller than snail mail ;)

the QR code experiment tend to have failed because binary transfers are a pain in the ass and the decoders were even more pain in the ass.
(thats why i will close this branch soon)

and if we have such a nice display and soooooo many light sensitive sensor elements in a camera (the image sensor...), why then not use them along with a module that decodes the data?


possible ToDos:
- not only OOK (on off keying), but also use color and brightness (0%, 50%, 100% for each RGB) gives 9 possible states and thus ~3 bits of information per pixel
- compress the binary data, which would imply we are no longer block aligned
Title: Re: Portable ROM dumper
Post by: sombree on October 25, 2017, 05:01:36 PM
Should it be this fast?
https://drive.google.com/open?id=0B1pPwPvDPAHRQWR2RXAxYWlfZjQ
Title: Re: Portable ROM dumper
Post by: g3gg0 on October 25, 2017, 08:46:27 PM
woohooo, thats quite fast :D
no, it should be slower by a factor of 10 or so...

which camera model? 80D?
Title: Re: Portable ROM dumper
Post by: sombree on October 25, 2017, 09:11:49 PM
Yes, it's 80D with autoexec.bin built from recovery branch with some modifications (https://hastebin.com/ugavohazir.diff) :)
Title: Re: Portable ROM dumper
Post by: dfort on October 26, 2017, 12:03:04 AM
Wish the dump I'm attempting would go that fast. Here's my new setup:

(https://farm5.staticflickr.com/4512/37902523892_b7dce0ec5e.jpg) (https://flic.kr/p/ZKjkRf)

After the master (7D) displays all dots twice and goes to a black screen I start the slave (5D3) and it goes through the setup. I tried getting a shot of some of that exciting action:

(https://farm5.staticflickr.com/4505/37879972566_e663aeedab.jpg) (https://flic.kr/p/ZHjL8h)

Then it starts--Yay!

(https://farm5.staticflickr.com/4472/37879974016_32bbcf15f5.jpg) (https://flic.kr/p/ZHjLyh)

Though at some point is usually starts failing:

(https://farm5.staticflickr.com/4493/37224095844_81ae12abf6.jpg) (https://flic.kr/p/YHndPW)

This goes on for hours and eventually the 5D3 seems to be telling me to get a life because it shuts down--always when I'm not watching.

Tried to see what I could get out of a failed dump:

./unpack_block DUMP.BIN 1.BIN
First header received, block size is 738 byte
Base address is 0xFC000000
Missing block of 0x1CD4 bytes at 0xFC0002E2-0xFC001FB5, assuming 0xFF
Missing block of 0x1CD4 bytes at 0xFC002298-0xFC003F6B, assuming 0xFF
Missing block of 0xBB62 bytes at 0xFC00424E-0xFC00FDAF, assuming 0xFF
0xFC152E92: Data CRC error, trying to correct
  Bit #296 error, fixed
0xFC1545A2: Data CRC error, trying to correct
  Bit #296 error, fixed
...
0xFC17524E: Data CRC error, trying to correct
  [ERROR] correction failed
  => Unpack failed for 0x2E2 bytes at 0xFC17524E-0xFC17552F, assuming 0xA5
Dumped 0xFC000000-0xFC175530


Instead of figuring out where to restart, how much to dump, etc. I thought why not dump smaller chunks so I looked through the code and it looks like this is where the change should be made:

reboot.c
    print_line(COLOR_CYAN, 3, "  Magic Lantern Rescue\n");
    print_line(COLOR_CYAN, 3, " ----------------------------\n");
   
    print_model();
    prop_diag();
    print_bootflags();
    find_gaonisoy();
   
    disp_dump(0xFC000000, 0x02000000);

    #if defined(CONFIG_BOOT_DUMPER)
        /* pick one method for dumping the ROM */
        #if defined(CONFIG_FULLFAT)
            dump_rom_with_fullfat();
        #else
            dump_rom_with_canon_routines();
        #endif
    #endif


Specifically, disp_dump(0xF8000000, 0x01000000) should be what we need for a ROM1.BIN from the 7D, right? Hum--if the model is recognized maybe the dump addresses could be adjusted? So far I believe that the only currently supported camera that the portable dumper cannot be used on is the 7D. Also, it would be nice if the display dumper would be called only if the portable dumper fails.

Those options look intriguing so I looked up where they are being defined and found this in the recovery branch Makefile.user.default:

# Work in progress - load ML as position-independent code (PIC)
CONFIG_PIC          = n

# recovery: use the fullfat library for low-level card access
CONFIG_FULLFAT      = n

# recovery: print CHDK cpuinfo
CONFIG_CPUINFO      = n

# recovery: dump ROM from bootloader (either FULLFAT or Canon routines)
CONFIG_BOOT_DUMPER  = n

# recovery: enable boot flag from bootloader
CONFIG_BOOT_BOOTFLAG = n


Should I add any of this to my Makefile.user?
Title: Re: Portable ROM dumper
Post by: g3gg0 on October 26, 2017, 08:58:20 PM
well my personal favorite is:
leave it as a swiss army knife - you can never make it fit for every purpose.
people who are into dumping cameras, will know how and what to change.
(dont know if alex agrees on this)

but it is absolutely a good idea to make it "detecting" if the portable dumping failed and then show the dot-dumper.
Title: Re: Portable ROM dumper
Post by: a1ex on October 26, 2017, 09:02:18 PM
There is a catch - usually, when the portable dumper fails, it freezes. To implement a fallback mechanism, one would probably have to set up a hardware timer and an interrupt - not worth the hassle IMO.

I'd just add a config option for the new dumper (already done locally, just not yet tested).
Title: Re: Portable ROM dumper
Post by: dfort on October 27, 2017, 01:01:04 AM
I was thinking of all sorts of options but a simple one that builds either a portable dumper or a display dumper should be good enough.

Success!

Got a good looking 7D.206 ROM1.BIN using the display dumper. No luck getting a ROM0.BIN but that shouldn't be a show stopper.

Quote from: dfort on October 26, 2017, 12:03:04 AM
...disp_dump(0xF8000000, 0x01000000) should be what we need for a ROM1.BIN from the 7D, right?

I was right about that but it kept failing at different points. Sure, unpack_block should fix it but after 24 hours it was still working on the dump so I tried doing it in smaller chunks.

disp_dump(0xF8000000, 0x00400000);
disp_dump(0xF8400000, 0x00400000);
disp_dump(0xF8800000, 0x00400000);
disp_dump(0xF8C00000, 0x00400000);


This gave me four DUMP.BIN files that could be concatenated together:

cat 1-0xF8000000/DUMP.BIN 2-0xF8400000/DUMP.BIN 3-0xF8800000/DUMP.BIN 4-0xF8C00000/DUMP.BIN > ROM_concatenated.BIN


Then run through unpack_block:

./unpack_block ROM_concatenated.BIN ROM1.BIN
First header received, block size is 738 byte
Base address is 0xF8000000
Missing block of 0x1CD4 bytes at 0xF80002E2-0xF8001FB5, assuming 0xFF
Missing block of 0x1CD4 bytes at 0xF8002298-0xF8003F6B, assuming 0xFF
Missing block of 0xBB62 bytes at 0xF800424E-0xF800FDAF, assuming 0xFF
Missing block of 0xFA bytes at 0xF83FFF06-0xF83FFFFF, assuming 0xFF
Missing block of 0x3323D4 bytes at 0xF85DDA56-0xF890FE29, assuming 0xFF
Missing block of 0xBAD7A bytes at 0xF8945098-0xF89FFE11, assuming 0xFF
Missing block of 0xFA bytes at 0xF8BFFF06-0xF8BFFFFF, assuming 0xFF
Missing block of 0x8FC7C bytes at 0xF8C301B6-0xF8CBFE31, assuming 0xFF
Missing block of 0xD7214 bytes at 0xF8EE8D8C-0xF8FBFF9F, assuming 0xFF
Dumped 0xF8000000-0xF8FFF660


Yay! A good looking ROM1.BIN file that is the right size.

(https://farm5.staticflickr.com/4458/37952674741_35424aa64f.jpg) (https://flic.kr/p/ZPKnX4)

I tried that same trick to get a ROM0.BIN but everything I tried (F0000000 - F7000000) ended up like this:

(https://farm5.staticflickr.com/4448/37952673741_7d26353cc8.jpg) (https://flic.kr/p/ZPKnDP)

Go figure.

In any case I'm happy with what I've got out of this exercise.
Title: Re: Portable ROM dumper
Post by: g3gg0 on October 27, 2017, 05:58:36 PM
wondering why it failed in the first place.
maybe some counter overflowing? hmmm
Title: Re: Portable ROM dumper
Post by: dfort on April 10, 2018, 08:17:19 PM
Got my hands on a 500D and thought I'd try dumping the 1.1.2 firmware but ran into this:

(https://farm1.staticflickr.com/888/27499298658_66e75244ac.jpg) (https://flic.kr/p/HU26qW)

Same issue with 1.1.1. I have used the portable dumper a few times and have never seen this before.
Title: Re: Portable ROM dumper
Post by: a1ex on April 10, 2018, 08:26:27 PM
Old-style model; covered in first post.

The good old blind dumper (https://a1ex.magiclantern.fm/bleeding-edge/blind-dumper/autoexec.bin) appears to work in QEMU (should work on all D4 and D5 models with bootflag enabled, except 7D). Make sure you have a valid image on the card, then go to PLAY mode. Split the dump in two, like you did on M2.
Title: Re: Portable ROM dumper
Post by: dfort on April 10, 2018, 09:18:10 PM
Well that does look similar to what we were doing a year ago (https://www.magiclantern.fm/forum/index.php?topic=15895.msg185121#msg185121). Tried the "blind dumper" and it gave me a file named "As" that was apparently the ROM1.BIN and didn't need to be split. Disassembled it and it looks good.

Thanks again!
Title: Re: Portable ROM dumper
Post by: a1ex on April 24, 2018, 09:54:20 AM
Quote from: a1ex on July 29, 2016, 12:06:45 PM
The issue: Canon's bootloader routines for file I/O copy the data to some cacheable (!) memory; when that buffer reaches 0x4000 bytes, it's written to card using DMA.

Canon finally fixed this in DIGIC 7 8)

The bug is, however, present in DIGIC 6 and earlier.

Updated autoexec.bin (first post) with:
- DIGIC 6 support, including serial flash dump (thanks t3r4n)
- DIGIC 7 support, when the time will come
- same portable binary loads on DIGIC 2, 3, 4, 5, 6, 7 AND 8!

ROM dumpers ready for 200D, 77D, 6D2 and 800D; will post the FIR versions in the DIGIC 7 thread (https://www.magiclantern.fm/forum/index.php?topic=19737).

These dumpers still require a very small card, but just formatting with a smaller filesystem will do the trick. The easiest way is (still) to write the QEMU SD image (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/sd.img.xz) onto the card (howto (https://thepihut.com/blogs/raspberry-pi-tutorials/17789160-backing-up-and-restoring-your-raspberry-pis-sd-card)).

The issue can be reproduced in QEMU on a large SD image (or by running from a physical card (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst#rst-header-running-from-the-physical-sd-cf-card)), so it's clearly not a caching issue. It can be reproduced from the FROMUTILITY menu, without loading AUTOXEC.BIN (proof of concept and details available on request). I believe are two different issues in Canon code: writing from cacheable memory (fixed on D7) and large card support (still present on D7).
Title: Re: Portable ROM dumper
Post by: rafaelbf on June 01, 2018, 04:50:44 AM
Quote from: dfort on April 10, 2018, 08:17:19 PM
Got my hands on a 500D and thought I'd try dumping the 1.1.2 firmware but ran into this:

Same issue with 1.1.1. I have used the portable dumper a few times and have never seen this before.

Hi dfort,

sorry for the late reply... I've tried Portable Dumper on my 500D, same screen on booth firmware, even with low capacity card.
Title: Re: Portable ROM dumper
Post by: dfort on June 01, 2018, 10:29:20 PM
@rafaelbf -- We need to use the blind dumper on the 500D.

Quote from: a1ex on January 25, 2016, 09:29:53 AM
Supported cameras:
- most DIGIC 4 (exceptions: 500D, 50D, 5D2, 7D)

Quote from: a1ex on April 10, 2018, 08:26:27 PM
The good old blind dumper (https://a1ex.magiclantern.fm/bleeding-edge/blind-dumper/autoexec.bin) appears to work in QEMU (should work on all D4 and D5 models with bootflag enabled, except 7D). Make sure you have a valid image on the card, then go to PLAY mode. Split the dump in two...

At first I had some issues splitting the dump in two but eventually figured it out (https://www.magiclantern.fm/forum/index.php?topic=15895.msg185214#msg185214).
Title: Re: Portable ROM dumper
Post by: DrEVILish on June 15, 2018, 11:54:04 PM
I have just ordered a 5Ds, which has D6+ like the 7D2, let me know if I can be of assistance, with testing and dumping.
Title: Re: Portable ROM dumper
Post by: JagoUK on August 29, 2018, 04:15:14 AM
7Dmk2 (Dual digic 6) dumped
(https://image.ibb.co/d40bsp/dumprom.png)

Had to tell it to dump to CF but it actually dumped to the SD.
Title: Re: Portable ROM dumper
Post by: a1ex on October 06, 2018, 06:18:53 PM
Updated with serial flash support for DIGIC 5 models (first post). Tested only in QEMU; please try and report back.

To check:
- make sure the MD5 sums are correct for all files (including SFDATA.BIN if present)
- make sure SFDATA.BIN looks like valid data (i.e. not full of zeros or full of FF or otherwise containing garbage - this condition is not covered by the checksums)

QEMU test results:

Testing portable ROM dumper...
     5D: skipping
    5D2: skipping
    5D3: SD: ROM0.BIN: OK ROM1.BIN: OK
    5D4: SD: ROM1.BIN: OK SFDATA.BIN: OK
     6D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    6D2: SD: ROM0.BIN: OK ROM1.BIN: OK
     7D: CF: ROM0.BIN: OK ROM1.BIN: OK
   7D2M: ROMs not saved
    40D: skipping
    50D: skipping
    60D: SD: ROM0.BIN: OK ROM1.BIN: OK
    70D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    77D: SD: ROM0.BIN: OK ROM1.BIN: OK
    80D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   400D: skipping
   450D: skipping
   500D: skipping
   550D: SD: ROM0.BIN: OK ROM1.BIN: OK
   600D: SD: ROM0.BIN: OK ROM1.BIN: OK
   650D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   700D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   750D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   760D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   800D: SD: ROM0.BIN: OK ROM1.BIN: OK
   100D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   200D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1000D: skipping
  1100D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1200D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1300D: SD: ROM0.BIN: OK ROM1.BIN: OK
   EOSM: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
  EOSM2: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK


7D2 has the serial flash on the other CPU...
Title: Re: Portable ROM dumper
Post by: dfort on December 18, 2018, 10:22:01 PM
Quote from: a1ex on October 06, 2018, 06:18:53 PM
please try and report back.

Better late than never!

Worked perfectly on the 700D and EOSM. All MD5 sums checked out. SFDATA.BIN looks valid. First time I tried it on a 32GB card and it didn't work but it was perfect with a 1GB card.

The 7D didn't work no matter what size card I tried -- all the way down to an old 64MB CF card. It kept getting stuck on "Dumping ROM0..."

(https://farm5.staticflickr.com/4864/45459980205_5885988812.jpg) (https://flic.kr/p/2cg9hja)

This note (https://www.magiclantern.fm/forum/index.php?topic=16534.msg192060#msg192060) about the 7D probably still applies.

The reason I looked up this topic was to ask a question about the 7D. Would it be possible to dump the master? When I worked on the 2.0.6 firmware update I used the display based dumper (https://www.magiclantern.fm/forum/index.php?topic=16534.msg192336#msg192336) and I'm fine with that but would like to know if it is possible and if so I need some guidance on what addresses to dump. Then to figure out how to disassemble it.
Title: Re: Portable ROM dumper
Post by: dfort on December 20, 2018, 05:57:53 PM
Quote from: a1ex on October 06, 2018, 06:18:53 PM
...Tested only in QEMU; please try and report back....

  1300D: SD: ROM0.BIN: OK ROM1.BIN: OK


Would asking for a 1300D ML-SETUP.FIR in order to run this test be an unreasonable request?
Title: Re: Portable ROM dumper
Post by: critix on January 10, 2019, 06:15:49 PM
Hi.
I have tried to find the issue of displaying Model Camera, Firmware version and IMG naming for models like 1300D.
I extracted the following files into a directory and compiled with for offline running:
compiler.h
prop_diag.c
prop_diag.h
property.h
propvalues.h

gcc prop_diag.c -o prop_diag

Then I ran:
./prop_diag 1300D_ROM1.BIN

The prop_diag.c file returns camera information, specifically: Camera Model, Firmware version and IMG naming. But that file can also run offline, in the sense that you give it a ROM file from which it tries to find the information above. If you run it through autoexec, then he tries to find the camera software information. If you run it offline, then he reads the given file as a parameter and tries to find that information.

To not compile portable.000 and run qemu, I chose to run it offline.
Now that I can run offline, I can make changes to the software and try to see why that information is not available.

The problem I've encountered is on the function:
check_terminator (0, last, 0).
There is no information for Digic4 +.
Maybe this feature needs to be changed for these device models?

With autoexec, for Digic 7 and Digic 6, guess_prop is called differently, with other values than the rest.
void prop_diag()
{
    if (is_digic7())
    {
        /* other models may lock up while reading this, so test first */
        guess_prop((void*)0xE0000000, 0x2000000, 1, 0);
    }
    else if (is_digic6())
    {
        guess_prop((void*)0xFC000000, 0x2000000, 1, 0);
    }
    else
    {
        guess_prop((void*)0xF0000000, 0x1000000, 1, 0);
        guess_prop((void*)0xF8000000, 0x1000000, 1, 0);
    }
    print_camera_info();
}

Neither Digic8 is found.
Title: Re: Portable ROM dumper
Post by: dfort on January 10, 2019, 08:09:39 PM
@critix helped get this running on my Mac. Needed to make a few changes.

Call malloc.h this way:

prop_diag.c
#include <stdio.h>
#include <malloc/malloc.h>


The compiler will error out because it can't find features.h, just comment it out:

compiler.h
//#include <features.h>
#include <stdint.h>
#include <limits.h>
#include <sys/types.h>


Here's what it does on my EOSM ROM1.BIN

./prop_diag ROM1.BIN
Loading ROM1.BIN...
Scanning from 0x1056cb000 to 0x1066cb000...
Trying offset 0x60000, status=0xffff, size=0x3b24...
Prop    c000004      4 3
Prop    c000002  15084 3d
Trying offset 0x80000, status=0x0, size=0xf84...
Skipping inactive block 0x80000, status=0x0, size=0xf84...
Trying offset 0x81000, status=0x0, size=0xf84...
Skipping inactive block 0x81000, status=0x0, size=0xf84...
Trying offset 0x82000, status=0x0, size=0xf84...
Skipping inactive block 0x82000, status=0x0, size=0xf84...
Trying offset 0x83000, status=0x0, size=0xf84...
Skipping inactive block 0x83000, status=0x0, size=0xf84...
Trying offset 0x84000, status=0x0, size=0xf84...
Skipping inactive block 0x84000, status=0x0, size=0xf84...
Trying offset 0x85000, status=0x0, size=0xf84...
Skipping inactive block 0x85000, status=0x0, size=0xf84...
Trying offset 0x86000, status=0x0, size=0xf84...
Skipping inactive block 0x86000, status=0x0, size=0xf84...
Trying offset 0x87000, status=0x0, size=0xf84...
Skipping inactive block 0x87000, status=0x0, size=0xf84...
Trying offset 0x88000, status=0x0, size=0xf84...
Skipping inactive block 0x88000, status=0x0, size=0xf84...
Trying offset 0x89000, status=0x0, size=0xf84...
Skipping inactive block 0x89000, status=0x0, size=0xf84...
Trying offset 0x8a000, status=0x0, size=0xf84...
Skipping inactive block 0x8a000, status=0x0, size=0xf84...
Trying offset 0x8b000, status=0x0, size=0xf84...
Skipping inactive block 0x8b000, status=0x0, size=0xf84...
Trying offset 0x8c000, status=0x0, size=0xf84...
Skipping inactive block 0x8c000, status=0x0, size=0xf84...
Trying offset 0x8d000, status=0x0, size=0xf84...
Skipping inactive block 0x8d000, status=0x0, size=0xf84...
Trying offset 0x8e000, status=0x0, size=0xf84...
Skipping inactive block 0x8e000, status=0x0, size=0xf84...
Trying offset 0x8f000, status=0x0, size=0xf84...
Skipping inactive block 0x8f000, status=0x0, size=0xf84...
Trying offset 0x90000, status=0x0, size=0xf84...
Skipping inactive block 0x90000, status=0x0, size=0xf84...
Trying offset 0x91000, status=0x0, size=0xf84...
Skipping inactive block 0x91000, status=0x0, size=0xf84...
Trying offset 0x92000, status=0x0, size=0xf84...
Skipping inactive block 0x92000, status=0x0, size=0xf84...
Trying offset 0x93000, status=0x0, size=0xf84...
Skipping inactive block 0x93000, status=0x0, size=0xf84...
Trying offset 0x94000, status=0x0, size=0xf84...
Skipping inactive block 0x94000, status=0x0, size=0xf84...
Trying offset 0x95000, status=0x0, size=0xf84...
Skipping inactive block 0x95000, status=0x0, size=0xf84...
Trying offset 0x96000, status=0x0, size=0xf84...
Skipping inactive block 0x96000, status=0x0, size=0xf84...
Trying offset 0x97000, status=0x0, size=0xf84...
Skipping inactive block 0x97000, status=0x0, size=0xf84...
Trying offset 0x98000, status=0x0, size=0xf84...
Skipping inactive block 0x98000, status=0x0, size=0xf84...
Trying offset 0x99000, status=0x0, size=0xf84...
Skipping inactive block 0x99000, status=0x0, size=0xf84...
Trying offset 0x9a000, status=0x0, size=0xf84...
Skipping inactive block 0x9a000, status=0x0, size=0xf84...
Trying offset 0x9b000, status=0x0, size=0xf84...
Skipping inactive block 0x9b000, status=0x0, size=0xf84...
Trying offset 0x9c000, status=0x0, size=0xf84...
Skipping inactive block 0x9c000, status=0x0, size=0xf84...
Trying offset 0x9d000, status=0x0, size=0xf84...
Skipping inactive block 0x9d000, status=0x0, size=0xf84...
Trying offset 0x9e000, status=0x0, size=0xf84...
Skipping inactive block 0x9e000, status=0x0, size=0xf84...
Trying offset 0x9f000, status=0x0, size=0xf84...
Skipping inactive block 0x9f000, status=0x0, size=0xf84...
Trying offset 0xa0000, status=0x0, size=0xf84...
Skipping inactive block 0xa0000, status=0x0, size=0xf84...
Trying offset 0xa1000, status=0x0, size=0xf84...
Skipping inactive block 0xa1000, status=0x0, size=0xf84...
Trying offset 0xa2000, status=0x0, size=0xf84...
Skipping inactive block 0xa2000, status=0x0, size=0xf84...
Trying offset 0xa3000, status=0x0, size=0xf84...
Skipping inactive block 0xa3000, status=0x0, size=0xf84...
Trying offset 0xa4000, status=0x0, size=0xf84...
Skipping inactive block 0xa4000, status=0x0, size=0xf84...
Trying offset 0xa5000, status=0x0, size=0xf84...
Skipping inactive block 0xa5000, status=0x0, size=0xf84...
Trying offset 0xa6000, status=0x0, size=0xf84...
Skipping inactive block 0xa6000, status=0x0, size=0xf84...
Trying offset 0xa7000, status=0x0, size=0xf84...
Skipping inactive block 0xa7000, status=0x0, size=0xf84...
Trying offset 0xa8000, status=0x0, size=0xf84...
Skipping inactive block 0xa8000, status=0x0, size=0xf84...
Trying offset 0xa9000, status=0x0, size=0xf84...
Skipping inactive block 0xa9000, status=0x0, size=0xf84...
Trying offset 0xaa000, status=0xffff, size=0xf84...
Prop    2000000      4 ??
Prop    2000001     36 2.0.2
Prop    2000005     36 9.9.8 B8(3a)
Prop    2000002      4
Prop    2000003      4 ????
Prop    2000004     32 Daniel A. Fort
Prop    2000006     16   
Prop    2000007     36 0.0.0
Prop    2010000      4 ?
Prop    2010001      4 d
Prop    2010002      4 d
Prop    2010003      4
Prop    2010004      4
Prop    2010005      4
Prop    2010009      4
Prop    201000a      4 P?
Prop    201000b      4
Prop    2010006      4
Prop    2010007      4
Prop    2010008      4
Prop    201000d      4
Prop    201000e      4
Prop    201000f      4
Prop    2010010      4 d
Prop    2010011      4 e
Prop    2010012      4 d
Prop    2010013      4
Prop    2010014      4
Prop    2020000     44
Prop    2020008     44
Prop    2020009     44
Prop    202000a     44
Prop    202000b     44
Prop    2020005      4
Prop    2020006      4
Prop    2020001      4
Prop    2020002      4
Prop    2020003      4
Prop    2020004      4
Prop    202000c      4
Prop    202000d     40
Prop    202000e     40
Prop    202000f     40
Prop    2020010     40
Prop    2020011     40
Prop    2030000     28
Prop    2030001     12
Prop    2030002     36
Prop    2030004      4
Prop    2030005      4
Prop    2030003      4
Prop    2030006     12
Prop    2040000      4
Prop    2040001      4
Prop    2040002      4
Prop    2040003      4
Prop    2040004      4
Prop    2040005      4
Prop    2040007      4
Prop    2040008      4
Prop    2040009      4
Prop    204000a      4
Prop    204000b      4
Prop    204000c      4
Prop    204000d      4
Prop    204000f     76
Prop    2050000    184
Prop    2050001      4 ?!
Prop    2050002      4
Prop    2050004      8 IMG_
Prop    2050005      4
Prop    2050003      4
Prop    2050006      4
Prop    2050007      4
Prop    2050008      4
Prop    2050009      8 IMG
Prop    205000a      4
Prop    205000b      8 |

Prop    205000d      4
Prop    205000e      4
Prop    205000f      4
Prop    2050013      4
Prop    2050014      4
Prop    2050016      4
Prop    2050017      4
Prop    2050018     48
Prop    2050019      8
Prop    205001a      4
Prop    205001c      4
Prop    205001f      4
Prop    2050020      4
Prop    2050011      4
Prop    2050021      4
Prop    2050023      4
Prop    2050024      4
Prop    2050025      4
Prop    2050022      4
Prop    205001b      4
Prop    205001d     40
Prop    2050026      4
Prop    2050027      4
Prop    2050028      4
Prop    2050029      4
Prop    205001e      4
Prop    205002a      4
Prop    205002b      4
Prop    205002d      4
Prop    205002e      8 |

Prop    205002f      4
Prop    2060000      4
Prop    2060001     24
Prop    2060002     24
Prop    2060003     24
Prop    2060004     24
Prop    2060005     24
Prop    2060006     24
Prop    2060007     24
Prop    2060008     24
Prop    2060009     24
Prop    206000a      4 ?
Prop    206000b      4 ?
Prop    206000c      4 ?
Prop    206000d     24
Prop    206000e     24
Prop    206000f     24
Prop    2060010     24
Prop    2060011     24
Prop    2060012     24
Prop    2060013     24
Prop    2060014     24
Prop    2060015     24
Prop    2060016      4 ?
Prop    2060017      4 ?
Prop    2060018      4 ?
Prop    2060019     24
Prop    206001a     24
Prop    206001b     24
Prop    206001c     24
Prop    206001e     24
Prop    2060021    300
Prop    2060020     24
Prop    206001f     24
Prop    2070000      4
Prop    2070001      4
Prop    2070002      8 ?23\
Prop    2070004     16 /
Prop    2070005      4
Prop    2070006      4
Prop    2070003      4
Prop    2070007      4 ????
Prop    2070008      4 ????
Prop    2070009      4
Prop    207000a      4
Prop    2070012      4
Prop    2070013     44
Prop    2070014      4
Prop    2070015      4
Prop    2070016     48
Prop    2070017      4
Prop    2070018      4
Prop    2080000      4
Prop    2080001      4
Prop    2090000      4
Prop    2090001      4
Prop    2090002      4
Prop    2090003      4
Trying offset 0xb0000, status=0x0, size=0xf84...
Skipping inactive block 0xb0000, status=0x0, size=0xf84...
Trying offset 0xb1000, status=0x0, size=0xf84...
Skipping inactive block 0xb1000, status=0x0, size=0xf84...
Trying offset 0xb2000, status=0x0, size=0xf84...
Skipping inactive block 0xb2000, status=0x0, size=0xf84...
Trying offset 0xb3000, status=0x0, size=0xf84...
Skipping inactive block 0xb3000, status=0x0, size=0xf84...
Trying offset 0xb4000, status=0x0, size=0xf84...
Skipping inactive block 0xb4000, status=0x0, size=0xf84...
Trying offset 0xb5000, status=0x0, size=0xf84...
Skipping inactive block 0xb5000, status=0x0, size=0xf84...
Trying offset 0xb6000, status=0x0, size=0xf84...
Skipping inactive block 0xb6000, status=0x0, size=0xf84...
Trying offset 0xb7000, status=0x0, size=0xf84...
Skipping inactive block 0xb7000, status=0x0, size=0xf84...
Trying offset 0xb8000, status=0x0, size=0xf84...
Skipping inactive block 0xb8000, status=0x0, size=0xf84...
Trying offset 0xb9000, status=0x0, size=0xf84...
Skipping inactive block 0xb9000, status=0x0, size=0xf84...
Trying offset 0xba000, status=0x0, size=0xf84...
Skipping inactive block 0xba000, status=0x0, size=0xf84...
Trying offset 0xbb000, status=0x0, size=0xf84...
Skipping inactive block 0xbb000, status=0x0, size=0xf84...
Trying offset 0xbc000, status=0x0, size=0xf84...
Skipping inactive block 0xbc000, status=0x0, size=0xf84...
Trying offset 0xbd000, status=0x0, size=0xf84...
Skipping inactive block 0xbd000, status=0x0, size=0xf84...
Trying offset 0xbe000, status=0x0, size=0xf84...
Skipping inactive block 0xbe000, status=0x0, size=0xf84...
Trying offset 0xbf000, status=0x0, size=0xf84...
Skipping inactive block 0xbf000, status=0x0, size=0xf84...
Trying offset 0xf00000, status=0x0, size=0x1132c...
Skipping inactive block 0xf00000, status=0x0, size=0x1132c...
Trying offset 0xf20000, status=0xffff, size=0x1132c...
Prop    5010000     28
Prop    5010001    544
Prop    5010002   2204
Prop    5010003   1264
Prop    5010004     64
Prop    5010005     44
Prop    5010006    784
Prop    5010007     28
Prop    5010008     24
Prop    5010009     84
Prop    501000a    504
Prop    501000b    224 ?
Prop    501000c      4
Prop    501000d    288 $
Prop    501000e    448
Prop    4000000     40
Prop    4000001     40
Prop    4000002     40
Prop    4000003     40
Prop    4000004     40
Prop    4010000      4
Prop    4010001  16704 @A?
Prop    4010002      4
Prop    4010003  16704 @A?
Prop    4010004      4
Prop    4010005  16704 @A?
Prop    e000001   2048
Prop    e000002   2048 ????
Prop    e000003   1024 ????
Prop    e000004   6252 ????
Prop    e020001     56
Prop    e020002    484
Prop    e030000   1024
Prop    e040000      4
Prop    e060000      8 0000
Prop    e060001      4
Prop    e060002      8 0000
Prop    e060003      8 0000
Prop    2090000      4
Prop    e070000     64 Daniel A. Fort
Prop    e070001     64 Daniel A. Fort
Trying offset 0xf60000, status=0xffff, size=0x1aedc...
Prop    b000000 110256 <
- Camera model: ???
- Firmware version: 2.0.2 / 9.9.8 B8(3a)
- IMG naming: 100?????/IMG_5912.JPG
- User PS: CineStyle logNeutral EOSHD C-LOG


Quote from: critix on January 10, 2019, 06:15:49 PM
There is no information for Digic4 +.
Maybe this feature needs to be changed for these device models?

Right, here's what happens on the 1300D.

./prop_diag ROM1.BIN
Loading ROM1.BIN...
Scanning from 0x10a954000 to 0x10c954000...
Trying offset 0x12aeb00, status=0x7000000, size=0xe0...
- Camera model: ???
- Firmware version: ??? / ???
- IMG naming: 100?????/????0000.JPG
- User PS: ??? ??? ???


[EDIT] Noticed that this is missing (from the 1300D branch too) though it isn't a solution to the Digic 4+ issue.

src/propvalues.h
#define MODEL_EOS_1300D  0x80000404
Title: Re: Portable ROM dumper
Post by: a1ex on January 11, 2019, 08:36:27 AM
The property data structures were changed a bit.

EOS M (same as most other models):

xxd -e -s 0x99000 -l 0x1000 -e EOSM/ROM1.BIN   # note: my offsets differ

00099000: 0000ffff 00000f84 02000000 00000f78  ............x...
00099010: 00000000 000000f4 02000000 0000000c  ................
00099020: 0000ffff 02000001 0000002c 2e302e32  ........,...2.0.
00099030: 38420032 29613328 006f6600 0000000c  2.B8(3a).fo.....
...
00099f60: 02090002 0000000c 00000000 02090003  ................
00099f70: 0000000c 00000000 0000ffff 00ff0000  ................
00099f80: 0f000000 ffffffff ffffffff ffffffff  ................


1300D:

xxd -e -s 0xC29000 -l 0x1000 -e 1300D/ROM1.BIN

00c29000: 00000002 00000000 00000000 0000ffff  ................
00c29010: 00000f6c 02000000 00000f54 00000000  l.......T.......
00c29020: 000000bc 02000000 0000000c 0000ffff  ................
00c29030: 02000001 00000018 2e312e31 37330030  ........1.1.0.37
...
00c29f60: 0000ffff 00ff0000 0f000000 ffdfd0f4  ................
00c29f70: ffffffff ffffffff ffffffff ffffffff  ................


M50 (notice the "active" flag differs, too; offset 0 is a false friend):

xxd -e -s 0x19dc000 -l 0x4000 -e M50/ROM0.BIN

019dc000: 0000ffff ffffffff ffffffff ffffffff  ................
019dc010: ffffffff ffffffff ffffffff ffffffff  ................
019dc020: 00ffffff ffffffff ffffffff ffffffff  ................
019dc030: 00003e28 02000000 00003df0 00000000  (>.......=......
019dc040: 00000100 02000000 0000000c 0000ffff  ................
019dc050: 02000001 0000002c 2e302e31 34330031  ....,...1.0.1.34
...
019dfe10: 00000000 00000000 00000000 0000ffff  ................
019dfe20: 00ff0000 1f000000 ffffffff ffffffff  ................
Title: Re: Portable ROM dumper
Post by: critix on January 11, 2019, 10:56:56 AM
I tested and not work with :
xxd -e -s 0xC29000 -l 0x1000 -e 1300D/ROM1.BIN
It work with
xxd -e -s 0xc20000 -l 0x1000 -e 1300D/ROM1.BIN
00c20000: 00000002 00000000 00000000 0000ffff  ................
00c20010: 00000f6c 02000000 00000f54 00000000  l.......T.......
00c20020: 000000bc 02000000 0000000c 0000ffff  ................
00c20030: 02000001 00000018 2e312e31 37330030  ........1.1.0.37
00c20040: 29623028 00008700 02000005 00000018  (0b)............
00c20050: 2e342e34 37332036 29623028 00008700  4.4.6 37(0b)....
00c20060: 02000002 0000000c 00000000 02000003  ................
....

Title: Re: Portable ROM dumper
Post by: dfort on January 11, 2019, 05:58:34 PM
Strange -- both addresses work over here:

xxd -e -s 0xc29000 -l 0x1000 -e 1300D/ROM1.BIN
00c29000: 00000002 00000000 00000000 0000ffff  ................
00c29010: 00000f6c 02000000 00000f54 00000000  l.......T.......
00c29020: 000000bc 02000000 0000000c 0000ffff  ................
00c29030: 02000001 00000018 2e312e31 37330030  ........1.1.0.37
00c29040: 29623028 00008700 02000005 00000018  (0b)............
00c29050: 2e342e34 37332036 29623028 00008700  4.4.6 37(0b)....
00c29060: 02000002 0000000c 00000000 02000003  ................



xxd -e -s 0xc20000 -l 0x1000 -e 1300D/ROM1.BIN
00c20000: 00000002 00000000 00000000 00000000  ................
00c20010: 00000f6c 02000000 00000f54 00000000  l.......T.......
00c20020: 000000bc 02000000 0000000c 0000ffff  ................
00c20030: 02000001 00000018 2e312e31 37330030  ........1.1.0.37
00c20040: 29623028 00008700 02000005 00000018  (0b)............
00c20050: 2e342e34 37332036 29623028 00008700  4.4.6 37(0b)....
00c20060: 02000002 0000000c 00000000 02000003  ................

Title: Re: Portable ROM dumper
Post by: a1ex on January 11, 2019, 08:19:51 PM
Yes, these offsets are changing at every camera startup, I think. There are a couple of property blocks with similar content, but only one of them is active (the one with the blue field 0000FFFF on most models; exception: M50, where the active block is marked with FFFFFFFF). Some of these property blocks are used by Canon for saving their settings (yes, they are reflashing the ROM at every shutdown (https://www.magiclantern.fm/forum/index.php?topic=19369)), others appear to be fixed or possibly updated only on demand (e.g. calibration data or image capture configuration).

Fixes pushed, sorry for spoiling the fun...
Title: Re: Portable ROM dumper
Post by: dfort on January 12, 2019, 12:27:55 AM
Happy days are here again!

(https://farm5.staticflickr.com/4913/32831843838_1e9c4b71e1.jpg) (https://flic.kr/p/S2eMk5)

(https://farm8.staticflickr.com/7830/39741788943_1caa4da585.jpg) (https://flic.kr/p/23xR4gD)
Title: Re: Portable ROM dumper
Post by: a1ex on January 14, 2019, 12:38:47 AM
As I was cleaning up the M50/SX70 dumper (https://www.magiclantern.fm/forum/index.php?topic=23296.msg210311#msg210311) source in order to publish it, I've noticed the same method could be used for all models, starting with the good old 5D. I ended up spending the entire weekend reworking this.

Previously, g3gg0 noticed the unreliability of Canon routines and tried to integrate a full-fledged FAT library (FullFAT (http://code.google.com/p/fullfat/)), with some low-level SD driver written from scratch. For some reason, that low-level SD driver didn't work on my cameras, and since I wasn't able to understand its internals, I kept trying to debug the high-level Canon routines. Which worked, to some extent, on most of the models where they were present. On the M50/SX70, these routines were gone, so I had to find some other method.

Revisiting g3gg0's approach, I've noticed adapting it for M50, and then for all other EOS models, was just a matter of replacing his low-level routines (which are likely model-specific) with Canon's sector I/O routines (which are present in all EOS firmwares I've looked at, from DIGIC 2 to DIGIC 8 ). I did just that and tested the new dumper in QEMU:

Testing portable ROM dumper...
     5D: CF: ROM0.BIN: OK ROM1.BIN: OK
    5D2: CF: ROM0.BIN: OK ROM1.BIN: OK
    5D3: SD: ROM0.BIN: OK ROM1.BIN: OK
    5D4: SD: ROM1.BIN: OK SFDATA.BIN: OK
     6D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    6D2: SD: ROM0.BIN: OK ROM1.BIN: OK
     7D: CF: ROM0.BIN: OK ROM1.BIN: OK
   7D2M: SD: ROM1.BIN: OK
    40D: CF: ROM0.BIN: OK ROM1.BIN: OK
    50D: CF: ROM0.BIN: OK ROM1.BIN: OK
    60D: SD: ROM0.BIN: OK ROM1.BIN: OK
    70D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    77D: SD: ROM0.BIN: OK ROM1.BIN: OK
    80D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   400D: CF: ROM0.BIN: OK ROM1.BIN: OK
   450D: SD: ROM0.BIN: OK ROM1.BIN: OK
   500D: SD: ROM0.BIN: OK ROM1.BIN: OK
   550D: SD: ROM0.BIN: OK ROM1.BIN: OK
   600D: SD: ROM0.BIN: OK ROM1.BIN: OK
   650D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   700D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   750D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   760D: SD: ROM1.BIN: OK SFDATA.BIN: OK
   800D: SD: ROM0.BIN: OK ROM1.BIN: OK
   100D: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
   200D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1000D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1100D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1200D: SD: ROM0.BIN: OK ROM1.BIN: OK
  1300D: SD: ROM0.BIN: OK ROM1.BIN: OK
   EOSM: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
  EOSM2: SD: ROM0.BIN: OK ROM1.BIN: OK SFDATA.BIN: OK
    M50: SD: ROM0.BIN: OK ROM1.BIN: OK
   SX70: SD: ROM0.BIN: OK ROM1.BIN: OK


Would this work on real hardware? Let's find out!

AUTOEXEC.BIN (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/AUTOEXEC.BIN)

(source: feb62ea fd2d938 2a15b7d (https://bitbucket.org/hudson/magic-lantern/branch/recovery) with CONFIG_BOOT_FULLFAT=y CONFIG_BOOT_DUMPER=y CONFIG_BOOT_SROM_DUMPER=y)

I've tested the following:
- 5D2: 32GB CF
- 5D3: 32GB and 256MB SD
- 500D: 32GB (didn't work) and 256MB SD

An older version of this code was confirmed to work on M50 and SX70.

In QEMU, I've checked all EOS models from the test suite on the 256MB image (FAT16) and on a 8GB (FAT32) image. The only issue I could find was with 500D on the large image; figure out why.

I didn't address caching issues yet (I just left them disabled), so the new dumper is not going to be very fast (at least not yet).

EXFAT is not supported.




Actually, the story begins somewhere around this (https://www.magiclantern.fm/forum/index.php?topic=2864.msg209934#msg209934), when I was trying to fix the emulation of various CPU info (https://www.magiclantern.fm/forum/index.php?topic=17714.) registers (i.e. sync the emulation with the logs posted by various users). As most of the CPU info logs were actually screenshots, I thought "OK, I'll get a plain-text log from the 5D3 real quick". I took the camera out of the bag, compiled the portable codebase, enabled the CPUINFO functionality, made some small changes to save all that info to a log file, tested it in QEMU as usual, then ran it on the camera it without thinking too much. It seemed to work, but afterwards... the card went unreadable (and the camera did not boot any more from it either). The worst part - this was the moment I realized that card contained all my holiday photos! (as they were not downloaded yet).

What happened? The QEMU image was a small 256MB one (which Canon's bootloader I/O routines have no trouble with), so my "offline" test worked fine. In the camera I had a 32GB card formatted as FAT32. Turns out, these I/O routines do not like large filesystems. At all.

After imaging the raw card contents, I ran the same code on a good filesystem, to see exactly what kind of damage it did. Nearly the entire partition table was zeroed out! Both copies of the FAT! Luckily, the data area was largely unaffected. Needless to say - I've spent the next few days grepping the filesystem and writing custom file recovery scripts, as testdisk/photorec didn't work. In the end, I was able to recover nearly everything.

Yes, all of this damage was done "just" by trying to save a small log file.

This was my motivation for getting rid of these Canon I/O routines once for all.
Title: Re: Portable ROM dumper
Post by: dfort on January 14, 2019, 06:32:44 AM
EOSM - MD5 on ROM0 didn't match. Tried multiple times with cards from 1 to 32GB. However, dump seems fine and it works in QEMU. ROM0 md5 issue not reproducible in QEMU.
  Magic Lantern Rescue
----------------------------
- Model ID: 0x331 M
- Camera model: ???
- Firmware version: 2.0.2 / 9.9.8 B8(3a)
- IMG naming: 100?????/IMG_5914.JPG
- User PS: CineStyle logNeutral EOSHD C-LOG
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF0C0000
- card_bootflags 10a63c
- boot_read/write_sector 10aec8 10afbc
- 102798 Card init => 2
- Dumping ROM0... 100%
- MD5: 8f2ab35008ead4f09ce9a2a5f9ce42f2
- Dumping ROM1... 100%
- MD5: 351d4472390b9074162692e7f039cc88
- 0: \n**** SROM(SIO%d) Menu ****\n
- 107c64: \n**** SROM Menu ****\n
- 107B4C: tag c022c000
- sf_init 107B48
- 10717c: Read Address[0x%06x-0x%06x]:0x
- 106A94: tag c0820000
- sf_command_sio 106A44
- Reading serial flash... 100%
- Writing SFDATA.BIN... 100%
- MD5: 395f84348339a032ce14b67998c74af8
- DONE!


EOSM2 - Seemed to work perfectly (I'm still having Mac QEMU issues with all dumps from this camera)
  Magic Lantern Rescue
----------------------------
- Model ID: 0x355 M2
- Camera model: ???
- Firmware version: 1.0.3 / 6.0.6 7A(2b)
- IMG naming: 100?????/IMG_1436.JPG
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF0C0000
- card_bootflags 10a6e4
- boot_read/write_sector 10af38 10b02c
- 102264 Card init => 2
- Dumping ROM0... 100%
- MD5: bae9e17718452b4c5cd904a0004615d2
- Dumping ROM1... 100%
- MD5: 7183d78b02c51297fb465b89efee71f1
- 107860: \n**** SROM(SIO%d) Menu ****\n
- 107734: tag c0400000
- sf_init 107734
- 106f44: Read Address[0x%06x-0x%06x]:0x
- 1066AC: tag c0820000
- sf_command_sio 10666C
- Reading serial flash... 100%
- Writing SFDATA.BIN... 100%
- MD5: f193d5bac8f48049948f62d2fdfdb482
- DONE!


700D - perfect
  Magic Lantern Rescue
----------------------------
- Model ID: 0x326 700D
- Camera model: ???
- Firmware version: 1.1.5 / 3.0.2 21(01)
- IMG naming: 100?????/IMG_4984.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF0C0000
- card_bootflags 10ab54
- boot_read/write_sector 10b3e0 10b4d4
- 10266C Card init => 2
- Dumping ROM0... 100%
- MD5: d5929b7a5ad99f511fc83d7d0b48b85f
- Dumping ROM1... 100%
- MD5: 6a5b5cdf62a73870f72b06e467219ed1
- 0: \n**** SROM(SIO%d) Menu ****\n
- 107e48: \n**** SROM Menu ****\n
- 107D30: tag c022c000
- sf_init 107D2C
- 107360: Read Address[0x%06x-0x%06x]:0x
- 106C78: tag c0820000
- sf_command_sio 106C28
- Reading serial flash... 100%
- Writing SFDATA.BIN... 100%
- MD5: d83d31e10ea740d670c1636a3233ff4c
- DONE!


7D (didn't work) screen blanks out after ROM0 md5. Tried with various sized cards from 32MB to 4GB. Saves ROM0.BIN but md5 checksum doesn't match. Also saves a partial ROM1.BIN.
Title: Re: Portable ROM dumper
Post by: Ottoga on January 14, 2019, 10:23:06 AM
@Alex / @Dfort,

Can I please have a reminder on the testing process and I will test it on my 7D-1. and will advise the results.

Title: Re: Portable ROM dumper
Post by: critix on January 14, 2019, 10:51:16 AM
Alex, can you make the FIR file for those who do not have bootflag enabled? I want to test the 1300D.
Thank you
Title: Re: Portable ROM dumper
Post by: a1ex on January 14, 2019, 02:38:29 PM
ROM0 MD5 not matching: expected, short answer in first post, will look up the long answer(s) later.

7D: that's actually big progress; IIRC it didn't even start. Now it was able to create a file, if I understand well.

Maybe it's some supervisor (MPU? second core?) that's turning off the camera after some inactive time. The 5D3 does that after several seconds if you run the dumper with battery door open. Guess: maybe the dumper just needs to be faster?

I've spent another couple of hours to track down caching issues. Updated autoexec.bin from previous post - is it still working? I've only tested on 5D3. Expecting it to be much faster.

Sorry, have to run now, see you tonight.
Title: Re: Portable ROM dumper
Post by: dfort on January 15, 2019, 03:52:31 AM
Busy day but I finally got a chance to give the new dumper on the 7D.

  Magic Lantern Rescue
----------------------------
- Model ID: 0x250 7D
- Camera model: Canon EOS 7D
- Firmware version: ??? / ???
- IMG naming: 100EOS7D/IMG_0000.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF010000
- card_bootflags 109a18
- boot_read/write_sector 109d54 109d64
- Patching 104294 from e3500001 to e3500000
- 104254 Card low-level init => F4240
- 1026EC Card init => 0
- Patching 1026FC from e3510001 to e3510000
- 1026EC Card init #2 => 1
- Dumping ROM0... 100%
- MD5: 55edc9e76de2ba7bae387f77d9a9c7cc
- Dumping ROM1... 100%
- MD5: ef25a835383698be6888d4395088a8e2
- No serial flash.
- DONE!


Screen goes dark right away so I wasn't sure when it was finished. First time I didn't wait long enough but second time I just let it run for a couple minutes. Used a 512MB card. ROM0 md5 mismatch as expected but the dump looks good.

[EDIT] Checked 700D, EOSM and EOSM2 -- only the EOSM is showing a different md5 checksum on ROM0, the other cameras had perfect checksums.
Title: Re: Portable ROM dumper
Post by: a1ex on January 16, 2019, 09:06:18 AM
Looks like the 7D is working (https://bitbucket.org/hudson/magic-lantern/commits/6f48e3328b3854067734476fc1eca7451ee6348b)! Updated first post as well.

FIR files:

DIGIC 4+:  1300D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMP1300D.FIR)  2000D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMP2000D.FIR)  4000D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMP4000D.FIR)
DIGIC 6:  5D4 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_5D4.FIR)  750D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP750D.FIR)  760D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP760D.FIR)  80D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_80D.FIR)
DIGIC 7:  200D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP200D.FIR)  6D2 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_6D2.FIR)  77D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_77D.FIR)  800D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP800D.FIR)
DIGIC 8:  EOSR (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMPEOSR.FIR)  M50 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_M50.FIR)  SX70 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMPSX70.FIR)  SX740 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMPSX740.FIR)
Master/Slave:  5DS (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_5DS.FIR)  5DSR (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP5DSR.FIR)  7D2 (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_7D2.FIR) 7D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP__7D.FIR)
Oldies:  1000D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DMP1000D.FIR)  30D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_30D.FIR)  400D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP400D.FIR)  40D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP_40D.FIR)  450D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP450D.FIR)  5D (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/DUMP__5D.FIR)

- built from 2a15b7d c019793 (https://bitbucket.org/hudson/magic-lantern/branch/recovery) with CONFIG_BOOT_FULLFAT=y CONFIG_BOOT_DUMPER=y CONFIG_BOOT_SROM_DUMPER=y
- green = confirmed working (either the last version, or a slightly older one)
- blue = not tested, but likely to work (based on other similar models, or on previous tests)
- purple = not tested, there may be surprises, but fixable (based on previous tests)
- orange = not tested, but unlikely to work (based on previous failures)
Title: Re: Portable ROM dumper
Post by: Walter Schulz on January 16, 2019, 07:56:57 PM
7D.206 with SanDisk  Extreme 60 MB/s, 32 GB

Contents of ROM0.MD5 and ROM1.MD5:

994fce5ee9ea3bb6df3ba0eaddf3e46f  ROM0.BIN
0de4cc03919f939c4ec691eb5fcfd744  ROM1.BIN


MD5 check on ROM0.BIN and ROM1.BIN running by PC:
50838bbf29aec4c6e62bee320d5c9138 J:\ROM0.BIN
0de4cc03919f939c4ec691eb5fcfd744 J:\ROM1.BIN


MD5 check for file ROM0.BIN differs.

BTW: No timestamps for ROMx.BIN and ROMx.MD5.

Title: Re: Portable ROM dumper
Post by: dfort on January 16, 2019, 08:27:02 PM
Confirming Walter's findings. Both the AUTOEXEC.BIN and DUMP_7D.FIR are working. Wish I had this when I dumped the 2.0.6 firmware (https://www.magiclantern.fm/forum/index.php?topic=16534.msg192336#msg192336).

(https://farm5.staticflickr.com/4883/31825859507_15258c8caf.jpg) (https://flic.kr/p/QukReD)

Throwback to Dec. 31, 1969 on the date stamp.

By the way, I tried compiling with the options in my Makefile.user file -- didn't work. For the record, putting the compile options in the command line worked fine.

Speaking of dual processor cameras -- this is the firmware for the slave processor, right? Is there a way to dump the firmware for the master processor or am I not understanding how this works?
Title: Re: Portable ROM dumper
Post by: critix on January 16, 2019, 08:43:45 PM
On 1300D not working. Not dumping... I try with 8G card... I will try tomorow with another.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on January 16, 2019, 08:48:58 PM
You can try to create a < 2 GB partition.

Quote from: a1ex on January 01, 2019, 12:00:03 AM

- the portable ROM dumper (http://www.magiclantern.fm/forum/index.php?topic=16534.0) (you must format the card to a very small size, or dd this 256MB image (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/sd.img.xz) - howto (https://thepihut.com/blogs/raspberry-pi-tutorials/17789160-backing-up-and-restoring-your-raspberry-pis-sd-card))
Title: Re: Portable ROM dumper
Post by: a1ex on January 16, 2019, 10:06:56 PM
Quote from: dfort on January 16, 2019, 08:27:02 PM
By the way, I tried compiling with the options in my Makefile.user file -- didn't work. For the record, putting the compile options in the command line worked fine.

In Makefile.user, these options have to be one per line.

Quote from: dfort on January 16, 2019, 08:27:02 PM
Speaking of dual processor cameras -- this is the firmware for the slave processor, right? Is there a way to dump the firmware for the master processor or am I not understanding how this works?

Dual processor (I'm talking specifically about master/slave configurations, not the kind of dual processor encountered in 5D4 or DIGIC 7):
- this ROM dumper only dumps the "primary" firmware (slave on 7D, master on 7D2/5DS/5DSR)
- secondary core is loaded with a dummy firmware, i.e. a while(1)
- dumping the secondary core requires understanding the communication APIs from Canon firmware (see e.g. how Dual ISO is implemented, but that method can be used only from main firmware)
- for 7D2/5DS/5DSR, see g3gg0's experiments on 5DS (https://www.magiclantern.fm/forum/index.php?topic=22267.0)

I can look into that if you think it helps, but so far I've considered it low priority, given how many other unfinished things I already have with single-core models.

Quote from: critix on January 16, 2019, 08:43:45 PM
On 1300D not working. Not dumping... I try with 8G card... I will try tomorow with another.

Cross-checking in QEMU with the old dumper (https://www.magiclantern.fm/forum/index.php?topic=17969.msg172875#msg172875), but couldn't see a real reason why it won't work (except maybe for the caching stuff). If it still doesn't work, you may use these to narrow down:

- 1300D_D1.FIR (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/1300D_D1.FIR%5B/url) (old method, requires a very small card, caches disabled, I/O trace very similar to the old one, with minor exceptions: display buffer address and an additional flush before disabling the caches)
- 1300D_D2.FIR (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/1300D_D2.FIR%5B/url) (new method, no card size restrictions, caches disabled, I/O trace very similar until it starts to dump, i.e. as expected)
- DMP1300D.FIR from above is similar to 1300D_D2.FIR, but with caches enabled.

Quote from: Walter Schulz on January 16, 2019, 08:48:58 PM
You can try to create a < 2 GB partition.
Quote from: a1ex on January 01, 2019, 12:00:03 AM
- the portable ROM dumper (you must format the card to a very small size, or dd this 256MB image - howto)

The filesystem size restrictions only apply to older dumpers (500D is the only exception I know; confirmed in QEMU that no other camera has this issue).

The new FIRs should work on 64GB cards or larger, too, as long as they are formatted as FAT32. Just checked on:
- a physical 64GB (58.1 GiB) SD with physical 5D3 (with autoexec.bin)
- a virtual 64GiB SD (formatted in a virtual 5D2 with a virtual SD to CF adapter) with emulated 1300D.
- a virtual 256GiB SD (formatted in a virtual 5D3 with card_fmt (https://www.magiclantern.fm/forum/index.php?topic=13983)) with emulated 1300D and 450D [oh yeah, I've got a way to test card_fmt!]
Title: Re: Portable ROM dumper
Post by: dfort on January 17, 2019, 07:09:13 AM
Quote from: a1ex on January 16, 2019, 10:06:56 PM
In Makefile.user, these options have to be one per line.

My Makefile.user that didn't work (Mac):

#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)

# CONFIG_QEMU = y
# LOG_INTERRUPTS = y
# CONSOLE_DEBUG = y
# CONFIG_DEBUGMSG = y
# CONFIG_DEBUG_INTERCEPT_STARTUP = y
# CONFIG_DEBUG_INTERCEPT = y
# CONFIG_GDB      = y
# CONFIG_GDBSTUB  = y
# CONFIG_MMIO_TRACE=y

# Recovery branch options:
CONFIG_BOOT_FULLFAT=y
CONFIG_BOOT_DUMPER=y
CONFIG_BOOT_SROM_DUMPER=y


Am I missing any juicy options that could be turned on in Makefile.user?

Quote from: a1ex on January 16, 2019, 10:06:56 PM
- this ROM dumper only dumps the "primary" firmware ...

I can look into that if you think it helps, but so far I've considered it low priority...

Well, the MPU messages are in there, right?

Quote from: a1ex on November 24, 2017, 08:14:55 AM
Confirmed - MPU messages are on the Master processor. Actually, g3gg0 tried to log them (https://bitbucket.org/hudson/magic-lantern/commits/f856de937243) back in 2012 (!)

There's probably some other stuff in there too. I can't get the 7D to do what reddeercity is doing with the 5D2 because some of the code seems to be running on the Master processor. In fact the raw_video_10bit_12bit_LVState branch won't compile on the 7D and it seems to be an issue with something in the 7D_MASTER code.

However, even with a Master processor dump the chances of me doing anything useful with it are rather slim.

@IDA_ML - Are you following this?
Title: Re: Portable ROM dumper
Post by: critix on January 17, 2019, 08:18:24 AM
Quote from: a1ex on January 16, 2019, 10:06:56 PM

Cross-checking in QEMU with the old dumper (https://www.magiclantern.fm/forum/index.php?topic=17969.msg172875#msg172875), but couldn't see a real reason why it won't work (except maybe for the caching stuff). If it still doesn't work, you may use these to narrow down:

- 1300D_D1.FIR (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/1300D_D1.FIR%5B/url) (old method, requires a very small card, caches disabled, I/O trace very similar to the old one, with minor exceptions: display buffer address and an additional flush before disabling the caches)
- 1300D_D2.FIR (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/1300D_D2.FIR%5B/url) (new method, no card size restrictions, caches disabled, I/O trace very similar until it starts to dump, i.e. as expected)
- DMP1300D.FIR from above is similar to 1300D_D2.FIR, but with caches enabled.
Unfortunately, I tried with 1G cards, but the same result.
I also tested with 1300D_D1.FIR and 1300D_D2.FIR
It stops at

- Dumping ROM0...

(https://i.ibb.co/3Rq80c6/IMG-20190117-090232.jpg) (https://ibb.co/3Rq80c6)

(https://i.ibb.co/pKDx4P4/IMG-20190117-085256.jpg) (https://ibb.co/pKDx4P4)

Title: Re: Portable ROM dumper
Post by: a1ex on January 17, 2019, 08:31:23 AM
Let's try the following:
- format the 1G card in the camera
- place the old dumper on the card
- run it (I expect it to work)
- format the card again in the camera (important; if old ROM files are still on the card, D1.FIR will just lock up like in your screenshot)
- copy 1300D_D1.FIR (which is pretty much identical to the old one, I don't see why it won't work)
- once that works, try 1300D_D2.FIR

Meanwhile I'm preparing some verbose FIRs to see exactly where it locks up.

Might have found the issue, hold a second. (nope, that won't explain the crash)
Title: Re: Portable ROM dumper
Post by: critix on January 17, 2019, 08:52:20 AM
I followed the steps you said. I see it now goes with 1300D_D1.FIR.
We wait to finish and try with 1300D_D2.FIR.
  Magic Lantern Rescue
----------------------------
- Model ID: 0x404 1300D
- Camera model: Canon EOS 1300D / KISS X80
- Firmware version: 1.1.0 / 4.4.6 37(0b)
- IMG naming: 100CANON/IMG_6797.JPG
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFE0C0000
- Open for write 1061E0 E92D47F0
- 101F64 Card init => 2
- Dumping ROM0...
- MD5: c38d7deeecee5432c254ba563cc503b2
- Dumping ROM1...
- MD5: fb70c66a568d05504bdc1fa076d4271f
- No serial flash.
- Saving RESCUE.LOG ...


OK... 1300D_D2.FIR is not working...
I'm still waiting to end Dumping ROM0 ...
Maybe in the end it will go ... I have a little patience ...
Title: Re: Portable ROM dumper
Post by: a1ex on January 17, 2019, 09:15:06 AM
Looks good, so at least the card initialization (common to both methods) is working.

Under the same conditions, 1300D_D2.FIR locks up? Here's a more verbose version that otherwise does the same thing:

1300D_D3.FIR (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/1300D_D3.FIR)
Title: Re: Portable ROM dumper
Post by: critix on January 17, 2019, 09:23:22 AM
OK ... so 1300D_D2.FIR does not work ...
I ran 1300D_D3.FIR and stopped at the line:
WR 000000FA  1 42005FA0
WR 00000480 80 F0000000

I hope I could see the writing ...
Title: Re: Portable ROM dumper
Post by: a1ex on January 17, 2019, 09:57:08 AM
Alright, file I/O DMA (SDDMA) locked up while trying to read from ROM. Will fix later.
Title: Re: Portable ROM dumper
Post by: a1ex on January 19, 2019, 07:36:31 AM
Let's try the 1300D again:

1300D_D4.FIR (https://a1ex.magiclantern.fm/debug/portable-rom-dumper/new/1300D_D4.FIR)

This time, I'm copying the ROM contents to RAM before saving to card.
Title: Re: Portable ROM dumper
Post by: critix on January 19, 2019, 07:54:00 AM
Yeah ... now it's OK without problems ...
  Magic Lantern Rescue
----------------------------
- Model ID: 0x404 1300D
- Camera model: Canon EOS 1300D / KISS X80
- Firmware version: 1.1.0 / 4.4.6 37(0b)
- IMG naming: 100CANON/IMG_6797.JPG
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFE0C0000
- card_bootflags 1069cc
- boot_read/write_sector 1071c0 1072b8
- 101F64 Card init => 2
- Dumping ROM0... 100%
- MD5: 66354cabd287d45faae4c6158ba09606
- Dumping ROM1... 100%
- MD5: f534bbc469bd73f4e1bded438a2613d8
- No serial flash.
- Saving RESCUE.LOG ...
Title: Re: Portable ROM dumper
Post by: a1ex on January 19, 2019, 08:13:35 AM
Finally :D

Updated all FIR files with the latest codebase.

Just curious - is the dump directly usable with QEMU, or does it still require the dd trick as described here (https://www.magiclantern.fm/forum/index.php?topic=17969.msg172893#msg172893)?
Title: Re: Portable ROM dumper
Post by: critix on January 19, 2019, 08:16:24 AM
I'll test in two hours
Unfortunately, it does not go without
dd if = ROM1.BIN of = BOOT.BIN bs = 64k skip = 1 count = 1
dd if = BOOT.BIN of = ROM1.BIN bs = 64k seek = 511
Title: Re: Portable ROM dumper
Post by: Walter Schulz on January 19, 2019, 10:08:03 AM
7D.206 here again with updated dumper:
Magic Lantern Rescue
----------------------------
- Model ID: 0x250 7D
- Camera model: Canon EOS 7D
- Firmware version: ??? / ???
- IMG naming: 100EOS7D/IMG_0000.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF010000
- card_bootflags 109a18
- boot_read/write_sector 109d54 109d64
- Patching 104294 from e3500001 to e3500000
- 104254 Card low-level init => F4240
- 1026EC Card init => 0
- Patching 1026FC from e3510001 to e3510000
- 1026EC Card init #2 => 1
- Dumping ROM0... 100%
- MD5: 8206fa3fda73c2ead57297bdea24f9fd
- Dumping ROM1... 100%
- MD5: 0de4cc03919f939c4ec691eb5fcfd744
- No serial flash.
- Saving RESCUE.LOG ...


ROM0.BIN checksum still not matching results computed by PC:
15df32dc1fccf481a812ae0fa19ebfe9 J:\ROM0.BIN
0de4cc03919f939c4ec691eb5fcfd744 J:\ROM1.BIN


Compared both files with those generated by dfort's ML build:
MD5 match for ROM1.BIN but not ROM0.BIN
Title: Re: Portable ROM dumper
Post by: critix on January 19, 2019, 11:29:59 AM
Yes you are right. Checksum is not the same for ROM0.BIN, but it is the same as ROM1.BIN:
For ROM0.BIN:
cat ROM0.MD5
66354cabd287d45faae4c6158ba09606  ROM0.BIN
md5sum ROM0.BIN
387d96a501c80ee5a1291e6a4bbbb636  ROM0.BIN

For ROM1.BIN:
cat ROM1.MD5
f534bbc469bd73f4e1bded438a2613d8  ROM1.BIN
md5sum ROM1.BIN
f534bbc469bd73f4e1bded438a2613d8  ROM1.BIN
Title: Re: Portable ROM dumper
Post by: polkah on January 24, 2019, 02:10:29 AM
Hey, so i don't know if it can be of any use to you guys, but i tested it on my 80d, here what i got :
in the "rescue" file it says
  Magic Lantern Rescue
----------------------------
- Model ID: 0x350 80D
- Camera model: Canon EOS 80D
- Firmware version: 1.0.1 / 6.2.2 9C(84)
- IMG naming: 100CANON/IMG_5727.JPG
- User PS: CineStyle Marvels Advanced 3.4
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFE0A0000
- card_bootflags 109a00
- boot_read/write_sector 109e90 109f58
- 101DA8 Card init => 2
- Dumping ROM1... 100%
- MD5: 67b48c0a6b19664f261dc502afaabf38
- 105774: \n**** SROM(SIO%d) Menu ****\n
- 105724: tag c0820200
- sf_init 105710
- 104f28: Read Address[0x%06x-0x%06x]:0x
- 104578: tag d20b0000
- sf_command_sio 10456C
- Reading serial flash... 100%
- Writing SFDATA.BIN... 100%
- MD5: 99821e45b63d737ccd055bd8a6ed1367
- Saving RESCUE.LOG ...

And, it seems like my camera still works, so... hooray.
I literally have no clue about what any of this mean, but if it can be any kind of help... yay
If you'd like more infos, just let me know, if this is totally useless and a complete waste of everyone's time... let me know as well
Title: Re: Portable ROM dumper
Post by: a1ex on January 24, 2019, 10:40:16 AM
It's portable code, i.e. same binary code attempting to run on all EOS models. It's a bit more verbose than required; it prints the address of functions it's going to call in Canon code (which were identified usually from strings).

For 80D, a ROM dumper was already available, so the new one doesn't bring much additional value (maybe just fewer restrictions, as the old one required a very small card). Still, it's good to know it's working on this camera, so... thanks for testing.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on January 24, 2019, 11:09:19 AM
@polkah: Can you run an additional MD5 check on ROM1.BIN and SFDATA.BIN?

Windows CLI:
powershell get-filehash *.BIN -A MD5 | format-list

Title: Re: Portable ROM dumper
Post by: polkah on January 25, 2019, 04:10:47 PM
Quote from: Walter Schulz on January 24, 2019, 11:09:19 AM
@polkah: Can you run an additional MD5 check on ROM1.BIN and SFDATA.BIN?

Windows CLI:
powershell get-filehash *.BIN -A MD5 | format-list
how'd you do that ?
Title: Re: Portable ROM dumper
Post by: Walter Schulz on January 25, 2019, 04:41:19 PM
Windows:
Hit Windows button
Type "cmd" (without quotation marks) and press Enter button
You will see a command prompt window, white characters on black ground.
Type (or copy)
powershell "get-filehash x:\*.bin -A MD5 | fl"
replace "x" with your card's drive letter and press Enter button
You can copy results after marking them (Mouse) and pressing Enter
Title: Re: Portable ROM dumper
Post by: morgan20 on January 27, 2019, 06:13:47 AM
Can confirm 6D2 working. But the ROM0 hash differs from the previous dump I got with the original 6D2 dumper. The hashes in *.MD5 files are same as the ROMs' hashes.



(https://i.ibb.co/86ZDcwW/IMG-20190127-102055.jpg) (https://ibb.co/86ZDcwW)
Title: Re: Portable ROM dumper
Post by: polkah on January 28, 2019, 01:27:23 PM
Here the result, hope it'll help, don't hesitate if you need anything more:
Algorithm : MD5
Hash      : 67B48C0A6B19664F261DC502AFAABF38
Path      : G:\ROM1.BIN

Algorithm : MD5
Hash      : 99821E45B63D737CCD055BD8A6ED1367
Path      : G:\SFDATA.BIN
Title: Re: Portable ROM dumper
Post by: Walter Schulz on January 28, 2019, 01:53:19 PM
Thanks!
Better results than critix's and mine: Checksum computed in cam match with those from PC.
Title: Re: Portable ROM dumper
Post by: timoxd7 on January 30, 2019, 01:38:19 AM
I tested it on a 750D, worked and right MD5:

  Magic Lantern Rescue
----------------------------
- Model ID: 0x393 750D
- Camera model: Canon EOS K393
- Firmware version: 1.0.0 / 8.5.1 B4(52)
- IMG naming: 100CANON/IMG_9903.JPG
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFE0A0000
- card_bootflags 10b4ac
- boot_read/write_sector 10b904 10b9cc
- 101CA4 Card init => 2
- Dumping ROM1... 100%
- MD5: b54721cec6d5ba1ca1c248765f73739d
- 105fb0: \n**** SROM(SIO%d) Menu ****\n
- 105F60: tag c0820200
- sf_init 105F4C
- 105764: Read Address[0x%06x-0x%06x]:0x
- 104DB4: tag d20b0000
- sf_command_sio 104DA8
- Reading serial flash... 100%
- Writing SFDATA.BIN... 100%
- MD5: f393c9b3d25485c4b5016ca20e39dedf
- Saving RESCUE.LOG ...
Title: Re: Portable ROM dumper
Post by: a1ex on February 05, 2019, 06:22:27 PM
Some minor updates:
- for 5DS/R (old dumper didn't work, new one tested in QEMU, not yet confirmed on real hardware)
- for the old 5D (prop_diag working)
- for models with narrow screens (fewer strings overflowing)

I'd like a test on 400D and 30D, if anyone happens to have one. Low priority, just for fun.

Edit Feb.10: confirmed on 5DS R.
Title: Re: Portable ROM dumper
Post by: eduperez on February 18, 2019, 10:16:59 PM
Results on a 400D:

  Magic Lantern Rescue
----------------------------
- Model ID: 0x236 400D
- Camera model: ???
- Firmware version: ???
- IMG naming: 100?????/????0000.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF810000
- card_bootflags 101c0c
- boot_read/write_sector 10735c 107374
- 1023a0: cf_dir (cfata_init error)\n
- 1020d8: cf_read_dma (cfata_init error)\n
- 107260 Card init => 0
- Dumping ROM0... 100%
- MD5: 2c7ab85a893283e98c931e9511add182
- Dumping ROM1... 100%
- MD5: 51d4dc45a6cf2cf1ea077ac13c404786
- No serial flash.


MD5 verified on computer.
Title: Re: Portable ROM dumper
Post by: codezion on February 19, 2019, 08:19:04 PM
Quote from: a1ex on February 05, 2019, 06:22:27 PM

I'd like a test on 400D and 30D, if anyone happens to have one. Low priority, just for fun.


I have a 30D if you want me to try anything. I must admit though that I am an absolute beginner in this space and will need some handholding to get me going.
Title: Re: Portable ROM dumper
Post by: eduperez on February 19, 2019, 11:31:35 PM
Quote from: codezion on February 19, 2019, 08:19:04 PM
I have a 30D if you want me to try anything. I must admit though that I am an absolute beginner in this space and will need some handholding to get me going.

Just download the file for your camera from the first post in this thread to a memory card, then place the card in the camera, and follow the firmware update procedure (it should be explained in the camera's manual, if it is not obvious by following the menus). Read the instructions on the screen, wait until it tells you to take the battery out, then share the new files that got created in the memory card.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on February 24, 2019, 12:56:48 PM
Run build from 17. Feb. on 7D (classic) two times from a bootable card.

=== Run 1 ===
  Magic Lantern Rescue
----------------------------
- Model ID: 0x250 7D
- Camera model: Canon EOS 7D
- Firmware version: ???
- IMG naming: 100EOS7D/IMG_0000.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF010000
- card_bootflags 109a18
- boot_read/write_sector 109d54 109d64
- Patching 104294 from e3500001 to e3500000
- 104254 Card low-level init => F4240
- 1026EC Card init => 0
- Patching 1026FC from e3510001 to e3510000
- 1026EC Card init #2 => 1
- Dumping ROM0... 100%
- MD5: a4c2c9e93c8a65ae8b9675e66a63b7ec
- Dumping ROM1... 100%
- MD5: 0f38a9a5f0aaf973a540ddc7f17cfe77
- No serial flash.
- Saving RESCUE.LOG ...


Text in MD5 files:
a4c2c9e93c8a65ae8b9675e66a63b7ec  ROM0.BIN
0f38a9a5f0aaf973a540ddc7f17cfe77  ROM1.BIN

Manual checksum for both BINs:
Hash      : 6D051D73A55B8C0733D7B01CF6E2DA16
Hash      : 0F38A9A5F0AAF973A540DDC7F17CFE77
=== END ===

=== Run 2 ===
  Magic Lantern Rescue
----------------------------
- Model ID: 0x250 7D
- Camera model: Canon EOS 7D
- Firmware version: ???
- IMG naming: 100EOS7D/IMG_0000.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF010000
- card_bootflags 109a18
- boot_read/write_sector 109d54 109d64
- Patching 104294 from e3500001 to e3500000
- 104254 Card low-level init => F4240
- 1026EC Card init => 0
- Patching 1026FC from e3510001 to e3510000
- 1026EC Card init #2 => 1
- Dumping ROM0... 100%
- MD5: 516c13deff73ba670a44e2ed6d6a84ee
- Dumping ROM1... 100%
- MD5: 0f38a9a5f0aaf973a540ddc7f17cfe77
- No serial flash.
- Saving RESCUE.LOG ...


Text in MD5 files:
516c13deff73ba670a44e2ed6d6a84ee  ROM0.BIN
0f38a9a5f0aaf973a540ddc7f17cfe77  ROM1.BIN

Manual checksum for both bins:
Hash      : 6D051D73A55B8C0733D7B01CF6E2DA16
Hash      : 0F38A9A5F0AAF973A540DDC7F17CFE77
=== END ===


Observation: Manual checksum consistent. Checksum computed in cam for ROM0.BIN is inconsistent.
Title: Re: Portable ROM dumper
Post by: a1ex on February 24, 2019, 04:42:44 PM
Quote from: g3gg0 on July 12, 2013, 11:28:06 PM
at 0xF0000000 is ROM0 which is rarely used. (flash ic usually not populated, just on 5d2 iirc)
if not populated, reading there will give some random noise or fading bits.

Quote from: a1ex on January 25, 2016, 09:29:53 AM
Some cameras have only ROM1 connected, so dumping ROM0 will give just random noise. In this case, the ROM0 checksum may not match, but that's OK.

On the "slave" side of the 7D, where this dumper runs, only ROM1 is connected. Reading from ROM0 gives only electrical noise.

The dumper doesn't know, or attempt to find out, which cameras use ROM0 and which ones don't. It just dumps both.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on February 24, 2019, 04:49:29 PM
Thanks, understood. Puzzling (me): Both runs gave exactly the same data for ROM0.BIN. That kind of noise sounds deterministic. Haven't looked into it, though (not my strong side, debugging binaries).
EDIT: Fine string of repeating non-sense it is.
ROM0.BIN: 16.384 KB -> 7-zip -> 3 KB
ROM1.BIN: 16.384 KB -> 7-zip -> 3.027 KB
Title: Re: Portable ROM dumper
Post by: a1ex on February 24, 2019, 05:00:53 PM
Yes, it's not exactly Gaussian noise, but rather something with very low entropy. And yes, in some cases it appears to be deterministic, or it may flip only a small number of bits.

In any case, it's not used by the firmware, so it's not a big deal if the checksum doesn't match.
Title: Re: Portable ROM dumper
Post by: scrax on February 26, 2019, 09:05:08 AM
On 600D it works and gives correct checksum only for ROM1.BIN
Title: Re: Portable ROM dumper
Post by: calle2010 on March 11, 2019, 09:52:33 PM
I can confirm that the latest 77D.FIR works. Checksums displayed (and in RESCUE.LOG) of ROM0.BIN and ROM1.BIN match the values calculated on the saved files. ROM1.BIN dumping and checksum calculation is very slow.

QuoteMagic Lantern Rescue
----------------------------
- Model ID: 0x408 77D
- Camera model: Canon EOS 77D / 9000D
- Firmware version: 1.0.2 / 7.3.6 6E(44)
- IMG naming: 100CANON/IMG_2067.JPG
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xE0040000
- boot_read/write_sector 106f85 107081
- 10190B Card init => 2
- Dumping ROM0... 100%
- MD5: a12fc3b5b380e81352f8e5d4ae5c3983
- Dumping ROM1... 100%
- MD5: ee61883e763361f9f8374960a219088b
- No serial flash.
- Saving RESCUE.LOG ...
Title: Re: Portable ROM dumper
Post by: Md Rajib on April 10, 2019, 11:47:19 PM
i have try 800D. but not working and showing massage "No Serial Flash". Please Help me. how to install magic lantern on EOS 800D
Title: Re: Portable ROM dumper
Post by: scrax on April 19, 2019, 09:20:20 AM
Quote from: a1ex on January 25, 2016, 09:29:53 AM
Latest download: autoexec.bin (http://a1ex.magiclantern.fm/debug/portable-rom-dumper/autoexec.bin) (2019Feb17, c019793)

- red = not working, no idea how to fix


Is this still true? Because in the EOS R thead seem solved, right?
Maybe this build don't have the fixes for the R ?

And maybe will be usefull to have also the .FIR build in first post?
Title: Re: Portable ROM dumper
Post by: kitor on April 19, 2019, 09:32:22 AM
If you need dumper for R, I can PM you one (but as autoexec.bin, not .fir), yesterday dumped 1.2.0 firmware so it works  ;).
Still won't work without bootflag enabled via UART.

From my knowledge, FIR encryption on R / RP is still a mystery.

(https://i.imgur.com/ZSzGtrI.jpg)
Title: Re: Portable ROM dumper
Post by: scrax on April 19, 2019, 06:45:26 PM
Ohhh right.. what a stupid question...
for the fir is needed the encription key, for the .bin bootflag -> UART
Title: Re: Portable ROM dumper
Post by: jcompton on May 06, 2019, 05:11:03 PM
Works on my 1300D in what sounds like expected fashion:

ROM1 MD5 matches
ROM0 MD5 doesn't. (my ROM0 as saved on card is just a huge stream of 0x00000100)
Title: Re: Portable ROM dumper
Post by: dfort on May 07, 2019, 04:26:02 PM
Just for the record, DUMP_M50.FIR on the first post doesn't save a valid ROM1.BIN dump. The only way I found to get a valid ROM1.BIN dump on the M50 is using the April 1 "fishy" build (https://a1ex.magiclantern.fm/bleeding-edge/M50/magiclantern-fishy.2019Apr01.M50101.zip) -- firmware 1.0.1 only.

There must be a way to get a valid ROM1.BIN because @leathc was able to get one back in January -- but only for 1.0.1.
Title: Re: Portable ROM dumper
Post by: chapan on May 21, 2019, 04:25:14 PM
Any chances for ROM dumper for 1500D?
Title: Re: Portable ROM dumper
Post by: a1ex on May 21, 2019, 10:58:24 PM
The canonical name of 1500D - according to Wikipedia - appears to be EOS 2000D (https://en.wikipedia.org/wiki/Canon_EOS_2000D). The dumper was already confirmed to work (https://www.magiclantern.fm/forum/index.php?topic=23606) on this camera.
Title: Re: Portable ROM dumper
Post by: chapan on May 22, 2019, 05:15:20 PM
I am new to this; does the file need to have a particular name or will the camera try to load any file with a ".fir" extension?
Title: Re: Portable ROM dumper
Post by: a1ex on May 22, 2019, 05:32:55 PM
It will try to load any file with FIR extension and correct model ID. Some models will require a 8.3 filename (with exactly 8 characters in the name). If the file is still not recognized, format the card from the camera (not from PC) and try again.
Title: Re: Portable ROM dumper
Post by: cifra78 on July 17, 2019, 04:37:42 PM
Canon 6D MKII:

Magic Lantern Rescue
----------------------------
- Model ID: 0x406 6D2
- Camera model: Canon EOS K406 / 6D Mark II
- Firmware version: 1.0.4 / 6.4.5 71(3e)
- IMG naming: 100CANON/IMG_1206.JPG
- User PS: CineStyle  C_LOG_htp
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xE0040000
- boot_read/write_sector 106f59 107055
- 1018F7 Card init => 2
- Dumping ROM0... 100%
- MD5: 4099deb7e6ce5124ff717b15cce80981
- Dumping ROM1... 100%
- MD5: 65e94999c18453b440c10f4a29d11a92
- No serial flash.
- Saving RESCUE.LOG ...
Title: Re: Portable ROM dumper
Post by: acasta on July 22, 2019, 11:00:19 PM
Hi,

I recently got interested in ML, but unfortunately both my Canon cameras are not supported yet.
One is the old 40D so I'd like to try and help make the port proceed a bit, hopefully.
A starting point could be what described here:
https://www.magiclantern.fm/forum/index.php?topic=1452.msg195051#msg195051 (https://www.magiclantern.fm/forum/index.php?topic=1452.msg195051#msg195051)
However, I'm stuck with the preliminary step of rom dump.
I tried DUMP_40D.FIR with 4 different CF cards, also old ones with 256 MB capacity, but it does not seem to work: the MD5 for ROM1.BIN is different each time (even if the check with PC always succeeds).
ROM0.MD5 is always the same though...

Here is a sample of my logs:
  Magic Lantern Rescue
----------------------------
- Model ID: 0x190 40D
- Camera model: Canon EOS 40D
- Firmware version: 1.1.1 / 4.0.1 6C(3e)
- IMG naming: 100CANON/IMG_2435.JPG
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF810000
- card_bootflags 101f34
- boot_read/write_sector 108350 108354
- Patching 10281C from e3510001 to e3510000
- 1027DC Card low-level init => F4240
- 101E18 Card init => 0
- Patching 101E28 from e3510001 to e3510000
- 101E18 Card init #2 => 1
- Dumping ROM0... 100%
- MD5: 2c7ab85a893283e98c931e9511add182
- Dumping ROM1... 100%
- MD5: 68e2c7549d97b6394f10607b6718606f
- No serial flash.
- Saving RESCUE.LOG ...


Any idea about what's wrong? Has DUMP_40D.FIR ever been tested in a camera, or only in QEMU?
I could try to do the dump in another way, but I understand I'd need ML to have a bootable camera... Could someone please point me at alternative ways to do that?
Title: Re: Portable ROM dumper
Post by: a1ex on July 27, 2019, 12:04:55 PM
Quote from: acasta on July 22, 2019, 11:00:19 PM
I tried DUMP_40D.FIR with 4 different CF cards, also old ones with 256 MB capacity, but it does not seem to work: the MD5 for ROM1.BIN is different each time (even if the check with PC always succeeds).
ROM0.MD5 is always the same though...

Already answered this one in the 40D thread (noticed this message afterwards).

Quote from: a1ex on July 25, 2019, 05:19:54 PM
That's probably alright - Canon firmware reflashes the ROM at every shutdown, to save their settings. If you compare the two ROMs, you will see differences only in the settings area (not in the executable code).

To get the same MD5 every time, you need to avoid starting the main Canon firmware between the two attempts (i.e. just run the dumper twice, possibly on different cards, without booting the camera normally in-between).
Title: Re: Portable ROM dumper
Post by: chapan on August 07, 2019, 05:51:12 PM
I tried running DMP2000D.FIR on my Canon EOS Rebel T7 and RESCUE.LOG shows this:

Magic Lantern Rescue
----------------------------
- Model ID: 0x432 2000D
- Camera model: Canon EOS Rebel T7 / K432
- Firmware version: 1.0.0 / 2.3.2 13(03)
- IMG naming: 100CANON/IMG_0786.JPG
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFE0C0000
- card_bootflags 1069ec
- boot_read/write_sector 1071e0 1072d8
- 101F70 Card init => 2
- Dumping ROM0... 100%
- MD5: 66354cabd287d45faae4c6158ba09606
- Dumping ROM1... 100%
- MD5: 65a90329df0b77b083a27a1f5583810f
- No serial flash.
- Saving RESCUE.LOG ...


But when I try to check the MD5 I get this:

root@craig-ubuntu:~# md5sum -c ROM0.BIN
md5sum: ROM0.BIN: no properly formatted MD5 checksum lines found
root@craig-ubuntu:~# md5sum -c ROM1.BIN
md5sum: ROM1.BIN: no properly formatted MD5 checksum lines found


I tried recreating the ROM files several times but the results are always the same.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on August 07, 2019, 07:47:12 PM
md5sum ROM?.BIN -c ROM?.MD5
Title: Re: Portable ROM dumper
Post by: chapan on August 14, 2019, 11:23:24 PM
This is what I see for Canon EOS Rebel T7.

Magic Lantern Rescue
----------------------------
- Model ID: 0x432 2000D
- Camera model: Canon EOS Rebel T7 / K432
- Firmware version: 1.0.0 / 2.3.2 13(03)
- IMG naming: 100CANON/IMG_0786.JPG
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFE0C0000
- card_bootflags 1069ec
- boot_read/write_sector 1071e0 1072d8
- 101F70 Card init => 2
- Dumping ROM0... 100%
- MD5: 66354cabd287d45faae4c6158ba09606
- Dumping ROM1... 100%
- MD5: 65a90329df0b77b083a27a1f5583810f
- No serial flash.
- Saving RESCUE.LOG ...


root@craig-ubuntu:~# ls -l ROM*
-rw-r--r-- 1 root root 33554432 Dec 31  1979 ROM0.BIN
-rw-r--r-- 1 root root       43 Dec 31  1979 ROM0.MD5
-rw-r--r-- 1 root root 33554432 Dec 31  1979 ROM1.BIN
-rw-r--r-- 1 root root       43 Dec 31  1979 ROM1.MD5

root@craig-ubuntu:~# md5sum -c ROM0.MD5
ROM0.BIN: FAILED
md5sum: WARNING: 1 computed checksum did NOT match
root@craig-ubuntu:~# md5sum -c ROM1.MD5
ROM1.BIN: OK

md5sum: ROM1.BIN: no properly formatted MD5 checksum lines found
ROM1.BIN: OK

Does that mean ROM1.BIN is the good firmware?




r
Title: Re: Portable ROM dumper
Post by: chapan on August 20, 2019, 04:50:28 PM
Dumping EOS Rebel T7 gives this:

- Model ID: 0x432 2000D
- Camera model: Canon EOS Rebel T7 / K432
- Firmware version: 1.0.0 / 2.3.2 13(03)
- IMG naming: 100CANON/IMG_0786.JPG
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFE0C0000
- card_bootflags 1069ec
- boot_read/write_sector 1071e0 1072d8
- 101F70 Card init => 2
- Dumping ROM0... 100%
- MD5: 66354cabd287d45faae4c6158ba09606
- Dumping ROM1... 100%
- MD5: 65a90329df0b77b083a27a1f5583810f
- No serial flash.


-rw-r--r-- 1 root root 33554432 Aug 15 15:05 ROM0.BIN
-rw-r--r-- 1 root root 33554432 Aug 15 15:05 ROM1.BIN


The MD5 checksum for ROM1.BIN is good. If I run disassemble.pl I get this:

root@craig-ubuntu:/usr/local/qemu-eos/1500D# perl disassemble.pl 0xFE0C0000 ROM1.BIN
offset + filesize - 1 &gt; 0xffffffff. We can&apos;t wrap around!

game over at disassemble.pl line 50.


Does that mean the ROM1.BIN file is too big? Is the ROMBASEADDR of 0xFE0C0000 from the RESCUE.LOG the correct address to use?




Title: Re: Portable ROM dumper
Post by: names_are_hard on August 20, 2019, 10:28:10 PM
Try this:
perl disassemble.pl 0xFE000000 ROM1.BIN

Magiclantern is a bit inconsistent about what "base address" means.  In some places it uses it to mean "entry point address", which is confusing.  0xFE000000 is the base address, ie, the address at which the first byte in the ROM is loaded into memory.  0xFE0C0000 is the entry point address, the address at which execution of the code starts.
Title: Re: Portable ROM dumper
Post by: chapan on September 03, 2019, 01:46:10 AM
That solved the disassemble problem for 1500D.  :)

Given what you said in your post, would these be correct parameters to use in hw/eos/model_list.c?

           .firmware_start         = 0xFE0C0000,
           .rom1_addr              = 0xFE000000,

And how would I know what to use for ram_size?
Title: Re: Portable ROM dumper
Post by: names_are_hard on September 03, 2019, 03:11:09 AM
I'd expect that to be right for firmware_start and rom1_addr.  I don't know the ram size for 1500D.  model_list.c has 256MB for 1300D, so I'd guess 1500D is either the same, or maybe 512MB since it's a later camera.

If you want, try:
.ram_size               = 0x10000000

and see if it makes your camera explode.  That's 256MB.  Try 0x20000000 for 512MB.  I don't know what the risk is if you get the ram size wrong.  Maybe the correct size is listed in some other thread.
Title: Re: Portable ROM dumper
Post by: a1ex on September 03, 2019, 07:09:59 AM
If you declare - in QEMU - a RAM size smaller than physical size, emulation will not run. The firmware will attempt to address memory outside the declared size (unmapped). In particular, the RscMgr task is going to initialize its data structures, covering pretty much the entire RAM.

If you declare a RAM size larger than physical size, nothing obvious will happen. There will be some memory that's never going to be addressed. The emulation will run just as well as if you would declare the correct size.

RAM size is not currently used in ML.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on December 14, 2019, 06:17:58 AM
Got a new cam: 250D/Rebel SL3/Kiss X10/200D Mark II (seriously, Canon?) with firmware version 1.0.1.
Is there a ROM dumper for this DIGIC 8 cam?
Title: Re: Portable ROM dumper
Post by: a1ex on December 14, 2019, 07:30:14 AM
This model, alongside G7X III (https://www.magiclantern.fm/forum/index.php?topic=23296.msg221692#msg221692) and 90D (maybe also M6 II), uses the same encryption as EOS R/RP, so the "state-of-art" way to dump the firmware is via UART (for now).

We actually made some progress figuring it out (thanks Indy), and it's likely that solving one of these will work for all others.
Title: Re: Portable ROM dumper
Post by: DeafEyeJedi on December 19, 2019, 06:45:05 AM
Quote from: Walter Schulz on December 14, 2019, 06:17:58 AM
...(seriously, Canon?)...

Seriously that's just flabergasting to hear... Does it feel more like 200D Mark II than 250D?
Title: EOS 800D- Portable ROM dumper
Post by: tanvir5971 on March 03, 2020, 07:45:11 PM
After formated the 64gb sdxc card as FAT32, i put the "DUMP800D.FIR" in the card. As per instructions i try to dum the ROM but error occured. i dont know how to insert picture here so link of the Screenshot is given below.

https://photos.app.goo.gl/NMa5juPu3ufbEWVQ7
Title: Re: Portable ROM dumper
Post by: Walter Schulz on March 04, 2020, 07:10:54 PM
Try using a smaller card or repartition to a much, much smaller size. Do not expect to make ExFAT work.
Title: Re: Portable ROM dumper
Post by: Pangu on March 29, 2020, 04:00:33 AM
Quote from: names_are_hard on September 03, 2019, 03:11:09 AM
I'd expect that to be right for firmware_start and rom1_addr.  I don't know the ram size for 1500D.  model_list.c has 256MB for 1300D, so I'd guess 1500D is either the same, or maybe 512MB since it's a later camera.

If you want, try:
.ram_size               = 0x10000000

and see if it makes your camera explode.  That's 256MB.  Try 0x20000000 for 512MB.  I don't know what the risk is if you get the ram size wrong.  Maybe the correct size is listed in some other thread.
Did this actually worked? I am looking forward for a fix. My dump gives exact same results. Thank you.
Title: Re: Portable ROM dumper
Post by: ilia3101 on April 17, 2020, 02:14:22 PM
(https://i.postimg.cc/k596F9FX/IMG-20200417-131122.jpg)

  Magic Lantern Rescue
----------------------------
- Model ID: 0x382 5DS
- Camera model: Canon EOS K331
- Firmware version: 1.1.1 / 8.1.2 94(37)
- IMG naming: 100?????/5DS_7425.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFE0A0000
- card_bootflags 10c074
- boot_read/write_sector 10c698 10c784
- 102A08 Card init => 1
- Dumping ROM1... 100%
- MD5: 1c4033b9cf1a088f29280ba4284216a6
- No serial flash.
- Saving RESCUE.LOG ...


1c4033b9cf1a088f29280ba4284216a6  ROM1.BIN

Confirmed MD5:
MD5 (/Volumes/EOS_DIGITAL/ROM1.BIN) = 1c4033b9cf1a088f29280ba4284216a6
Title: Re: Portable ROM dumper
Post by: monmond on August 25, 2020, 08:12:31 AM
Hi, I'm newb here.
btw, I have 6D2 and this is the text on dumper screen

QuoteMagic Lantern Rescue
----------------------------
- Model ID: 0x406 6D2
- Camera model: Canon EOS K406 / 6D Mark II
- Firmware version: 1.0.4 / 6.4.5 71(3e)
- IMG naming: 100CANON/IMG_6018.JPG
- User PS: EOSHD C-LOG EOSHD VIVID SKINTONE EOSHD MONOCHROME
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xE0040000
- boot_read/write_sector 106f59 107055
- 1018F7 Card init => 2
- Dumping ROM0... 100%
- MD5: 36bfe44f2a0a9c028a4d1ce5e3f5ae16
- Dumping ROM1... 100%
- MD5: ef9966e6391c45cd235841132bd75480
- No serial flash.
- Saving RESCUE.LOG ...

via 4gb sd card.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on September 05, 2020, 06:33:57 AM
250D not working yet.
Enabled bootflag via srsa's script and used latest autoexec.bin on a 128 GB ExFAT and 8 GByte FAT32 (Eye-Fi) card. Same result.

(https://abload.de/img/img_98815ck8a.png)

Title: Re: Portable ROM dumper
Post by: yourboylloyd on September 05, 2020, 06:38:48 AM
Can also confirm that it doesn't work on EOS R6.

Camera COMPLETELY locks up. No lights, no button presses change, nothing. Camera is completely unusable. Resets on battery pull. So Bootflag confirmed at least :)
Title: Re: Portable ROM dumper
Post by: kitor on September 05, 2020, 10:38:46 AM
Something tells me that display access may be completely different.

@a1ex, can you prepare one that logs only to file? Similar to what we initially attempted on R.

QuoteCamera COMPLETELY locks up. No lights, no button presses change, nothing. Camera is completely unusable. Resets on battery pull. So Bootflag confirmed at least :)

No reaction on buttons, etc is normal. Question is if you wait a few minutes in this state, is card access led activating / any new files made on SD card?
I doubt, if memory locations for display buffers are different, but there's a chance it's doing things silently ;)

@Walter - this screenshot looks like it's there, same location but display buffers works differently (?) Or at least display resolution is detected wrong, as every next line seems to start at wrong position.
Title: Re: Portable ROM dumper
Post by: a1ex on September 05, 2020, 11:09:40 AM
The display issue is just a matter of fixing the resolution (in particular, the horizontal one). It's not auto-detected, but hardcoded, based on model ID.

The red thing is an error message - probably the dumper couldn't find suitable stubs for writing to card.

However, Canon Basic dumper appears to provide a complete ROM dump for now, so the portable ROM dumper is no longer essential. Just nice to have, as a proof of concept (portable code running on all cameras). I should be able to troubleshoot it in QEMU using the Canon Basic dumps - these include the bootloader, from what I've seen.
Title: Re: Portable ROM dumper
Post by: Ant123 on September 05, 2020, 11:32:38 AM
Quote from: a1ex on September 05, 2020, 11:09:40 AM
I should be able to troubleshoot it in QEMU

Please don't forget to commit the QEMU branch to the repository. :)
The "Source code" link from the main page at this moment leads to the removed repisitory on bitbucket.org
Title: Re: Portable ROM dumper
Post by: kitor on September 05, 2020, 01:38:41 PM
Quote from: a1ex on September 05, 2020, 11:09:40 AM
However, Canon Basic dumper appears to provide a complete ROM dump for now, so the portable ROM dumper is no longer essential. Just nice to have, as a proof of concept (portable code running on all cameras).

Let me disagree with that, it's still a nice tool for broken camera diagnosis :) (as long as bootflag is enabled, but now it seems that we have a way to enable it on all current models)

Not to mention all the coverage that will hit the news if you'll post photo of R5 running ML Rescue ;)
Title: Re: Portable ROM dumper
Post by: coon on September 05, 2020, 03:21:08 PM
Quote from: Walter Schulz on September 05, 2020, 06:33:57 AM
250D not working yet.
Enabled bootflag via srsa's script and used latest autoexec.bin on a 128 GB ExFAT and 8 GByte FAT32 (Eye-Fi) card. Same result.

(https://abload.de/img/img_98815ck8a.png)

I had the same issue with the RP. See R / RP thread: https://www.magiclantern.fm/forum/index.php?topic=22770.msg230480#msg230480

Changing the resolution values for disp_xres and disp_yres in src/disp_direct.c to the proper value for the RP did fix the issue. However, I cannot find the difinitions for the 250D there!?
Title: Re: Portable ROM dumper
Post by: Walter Schulz on September 05, 2020, 05:40:26 PM
Quote from: kitor on September 05, 2020, 01:38:41 PM
Let me disagree with that, it's still a nice tool for broken camera diagnosis :)

And I predict we will see that more often in the future. At the moment all those not-so-versed-in-coding-people get stuck when they try to make ML code run on unsupported firmware or trying to install ML on unsupported cams. We had a lot of "discussions" how to make it run anyway by gifted fellows (renaming the file for example ... hey, it will work: Checksum is identical! Hooray!).
Canon Basic Script however is plain text and word will get around. All safeguards implemented (if any) checking for proper firmware or camera type can be disabled with just a few characters (' -> You are a comment now). And here we go poking the hell out of a 1D X Mark III by using a script for M6 II!

I suggest a1ex to switch to billing per starting hour for those <censored> if there cams have to be reanimated.

It will happen. It will happen soon. It will happen quite often.
Title: Re: Portable ROM dumper
Post by: Ant123 on September 05, 2020, 06:00:34 PM
Walter Schulz
If this bothers you, why don't you force new users to read the FAQ before they can post on the forum. It would also be possible to hide links and attachments from unregistered visitors.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on September 05, 2020, 06:15:14 PM
I'm not a mod, I'm not an administrator. Therefore I have no permissions to change anything on this server. And even if I would: I absolutely have no intention to do so.
I extremely dislike hidden content forcing readers to register. And it doesn't work. You can't hide such things forever. Security by obscurity is a flawed concept.
You cannot make a system fool-proofed. Another flawed concept.

I simply wanted to express my opinion that we will see abuse of this gem and we should be prepared to handle it.
Title: Re: Portable ROM dumper
Post by: Ant123 on September 05, 2020, 06:35:59 PM
Quote from: Walter Schulz on September 05, 2020, 06:15:14 PM
I'm not a mod, I'm not an administrator.
You are in second on this forum by the number of posts, so I confused you with moderator. Sorry.
But aren't you tired of answering the same stupid newbie questions over and over again?

QuoteI simply wanted to express my opinion that we will see abuse of this gem and we should be prepared to handle it.
I also do not prefer "security by obscurity".
But forcing users to read the FAQ formally shifts the responsibility for the negative consequences of their actions on them.
Title: Re: Portable ROM dumper
Post by: yourboylloyd on September 05, 2020, 08:27:04 PM
Hey. Don't know how significant this is, but when I try to place the sd card with portable rom dumper into the R6, the camera locks up in slot 2 AND slot 1.

But when I run the basic dumper, the sd card only works in slot 2.
Title: Re: Portable ROM dumper
Post by: Nicolas Apud on September 10, 2020, 12:04:07 PM
Quote from: monmond on August 25, 2020, 08:12:31 AM
Hi, I'm newb here.
btw, I have 6D2 and this is the text on dumper screen

via 4gb sd card.

Could you explain how to do it? I have 2 6d2 and I want to help
Title: Re: Portable ROM dumper
Post by: Walter Schulz on September 10, 2020, 01:43:04 PM
Try
https://www.magiclantern.fm/forum/index.php?topic=24827.msg230622#msg230622
instead

Title: Re: Portable ROM dumper
Post by: Gurney255 on October 29, 2020, 09:57:05 AM
Hello guys.

My 60D is bricked. Not starting at all.  I went off and during shooting.
Display test works, Blink test works, ROM Dump successful. Still no luck to start the camera. I have tried various SDs. Now, after putting the battery in it just blinks, forever.
What should I do now?
Thank you all in advance for any suggestions. I really appreciate it.
Title: Re: Portable ROM dumper
Post by: Walter Schulz on November 14, 2020, 12:27:15 AM
Remove battery, remove card.
Insert battery.
Startup cam.

Status? Any activity?