[DONE] How do I dump 5D3 firmware 1.2.3?

Started by chris_overseas, January 05, 2014, 06:18:47 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

chris_overseas

I would like to try and update ML so it runs with firmware 1.2.3 on the 5D Mark III. My understanding is that I need to do the following:

  • Obtain a ROM dump of the 1.1.3 and 1.2.3 firmwares.
  • Load these dumps into ARMu.exe.
  • For each address is platform/5D3.113/stubs.S, find the equivalent address in the 1.2.3 firmware by comparing the disassemblies.
  • Create a new platform/5D3.123/stubs.S file using these newly discovered addresses.
  • Update various build/compile scripts as required.
I'm new to this however and I've hit a wall at the very first step. How do I get a dump of 1.2.3? I tried updating my 5D3 to 1.2.3 and then flashing 5D3-113-bootflag.fir. Perhaps not surprisingly, this dies with the following error: "Update file error. Please check the memory card and reload the battery and try again". I've also tried to run both the scripts here: http://chdk.wikia.com/wiki/Canon_Basic/Scripts/Dumper but nothing is produced. I'm not convinced the scripts even run. As instructed I set the "SCRIPT" string in the boot sector (using EOScard) and created the appropriate script.req and extend.m files - is there something else I need to do too or does Canon Basic scripting not work on DSLRs, just the point'n'shoots? Can I perhaps extract and decrypt the dump from Canon's 5D300123.FIR file instead? If I need to create my own .fir file for 1.2.3 to dump the ROM, how would I go about doing that - presumably it needs signing? Can I make an autoexec.bin to dump it? Is there another trick I can try?

Obviously various developers here know how to get a ROM dump but I haven't been able to find this process documented in detail anywhere. Any hints would be greatly appreciated!

Appealing to Pelican, coutts, 1%, a1ex, g3gg0, or anyone else who can help me learn more about this.

A couple of relvant threads:
http://www.magiclantern.fm/forum/index.php?topic=2602.msg73403#msg73403
http://www.magiclantern.fm/forum/index.php?topic=5695.msg61657#msg61657
EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II

mva

Good luck! Hope someone who can help you responds.

Quote from: chris_overseas on January 05, 2014, 06:18:47 PM
I would like to try and update ML so it runs with firmware 1.2.3 on the 5D Mark III. My understanding is that I need to do the following:

  • Obtain a ROM dump of the 1.1.3 and 1.2.3 firmwares.
  • Load these dumps into ARMu.exe.
  • For each address is platform/5D3.113/stubs.S, find the equivalent address in the 1.2.3 firmware by comparing the disassemblies.
  • Create a new platform/5D3.123/stubs.S file using these newly discovered addresses.
  • Update various build/compile scripts as required.
I'm new to this however and I've hit a wall at the very first step. How do I get a dump of 1.2.3? I tried updating my 5D3 to 1.2.3 and then flashing 5D3-113-bootflag.fir. Perhaps not surprisingly, this dies with the following error: "Update file error. Please check the memory card and reload the battery and try again". I've also tried to run both the scripts here: http://chdk.wikia.com/wiki/Canon_Basic/Scripts/Dumper but nothing is produced. I'm not convinced the scripts even run. As instructed I set the "SCRIPT" string in the boot sector (using EOScard) and created the appropriate script.req and extend.m files - is there something else I need to do too or does Canon Basic scripting not work on DSLRs, just the point'n'shoots? Can I perhaps extract and decrypt the dump from Canon's 5D300123.FIR file instead? If I need to create my own .fir file for 1.2.3 to dump the ROM, how would I go about doing that - presumably it needs signing? Can I make an autoexec.bin to dump it? Is there another trick I can try?

Obviously various developers here know how to get a ROM dump but I haven't been able to find this process documented in detail anywhere. Any hints would be greatly appreciated!

Appealing to Pelican, coutts, 1%, a1ex, g3gg0, or anyone else who can help me learn more about this.

A couple of relvant threads:
http://www.magiclantern.fm/forum/index.php?topic=2602.msg73403#msg73403
http://www.magiclantern.fm/forum/index.php?topic=5695.msg61657#msg61657

Magic 7D

+one on this, I'd like to help with any dirty work. Btw I know nothing about coding, but I could maybe help looking for stubs etc once we've got the dump.  :)
A MAGIC 7D and 5D III + Lenses in the range of 15 - 400mm. Man it's shining.. it's a Lantern oh it's Magic, Aha it's Magic Lantern! ;)

chris_overseas

No one has some hints for me? It seems a bit crazy to reinvent the wheel.
EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II

mva

Quote from: chris_overseas on January 16, 2014, 06:11:54 PM
No one has some hints for me? It seems a bit crazy to reinvent the wheel.
Chris, you've made this offer a number of times since early November. Clearly it's been seen, and those with the relevant knowledge have decided not to respond. Seems rude to me that they haven't responded at all -- you're after all not simply complaining about wanting some new feature, but willing to do the work, which the administrators and devs are always encouraging complainers to do -- but perhaps they have their reasons for not even telling you 'no'. Maybe try sending a PM.

chris_overseas

Quote from: mva on January 17, 2014, 11:55:22 AM
Chris, you've made this offer a number of times since early November. Clearly it's been seen, and those with the relevant knowledge have decided not to respond. Seems rude to me that they haven't responded at all -- you're after all not simply complaining about wanting some new feature, but willing to do the work, which the administrators and devs are always encouraging complainers to do -- but perhaps they have their reasons for not even telling you 'no'. Maybe try sending a PM.

Well I've done a bit of reverse engineering over the years but I'm new to cameras, so I feel that if I had the spare time I could probably figure at least some of it out for myself. The problem is I don't have a lot of spare time so I was hoping for a bit of a jump-start with some tips from those who already know the correct route to take. I suspect the problem is that this particular firmware upgrade is a lot more difficult to deal with than a normal one - coutts alluded to that with his comments on the 1.2.1 stubs he updated. Perhaps given my lack of knowledge I'll only end up bricking my camera and no one wants that responsibility :)

I'm not actively looking into this because I have other projects to work on for now. If someone does give me a few pointers though I'll gladly take a look and do as much as I can, including documenting the process for others. I'd really like to get ML + f/8 AF on my 5D3.
EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II

nanomad

Steps to be done:
- Dump the firmware (get in touch with a1ex, he may help)
- Match stubs / consts between the two ports (requires access to IDA)
- Fix bugs
- ???
- You have your new port.

EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

chris_overseas

Thanks nanomad. I tried asking the same question recently but didn't get much response (http://www.magiclantern.fm/forum/index.php?topic=9827.0).

I'll drop a1ex a note and see if he can point us in the right direction for getting that firmware dump.
EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II

VisualPursuit

Another one here.

I have also offered to put in a few hundred hours, provided someone
helps me get onto the horse.

I don't mind virtually copying a telephone book by hand, do the "dirty work"
as someone worded it. But I am no programmer, have long forgotten 99%
of the BASIC and PASCAL we had in school, back when there was no monitor
but a teletype instead.

About the rude answers (or no answers at all):
This is not a commercial project, nobody has any interest in customers,
and there is definitely no community management.

We have to take it like it is and get organised by ourselves.

The stupid thing though: Obviously most of what is asked will be read, even
if there is no answer or a rude answer. I myself have asked in three or four
places about proceedings for firmware 1.2.1 and have never received any
answer but instead a warning for posting duplicate questions.
I am now on moderated status and every time I contribute to this forum,
a moderator must make the effort to look into what I write and approve or decline.

Proper community management would have told me to dig out the stubs myself
after the first question, and point me to somebody who might be able and willing
to make me fit for the job.

Yes, your playground, your rules. But I think one could make life easier for
everybody and get more help in coming forward - even for those who are
not too much interested in firmware 1.2.3.

People hear about magic lantern, and whoever buys a 5D Mk3 now will have
1.2.3 on it. The questions will not die, they will grow. Let's work together
to relief the pressure from the kettle.

chris_overseas

Some progress. I managed to get a dump off my camera with help from a1ex and I've updated most of the stubs. I still trying to figure a few of them out and I have no idea how long it will take to get the stubs finished, changes tested and stable, but at least there is a clear path forward now.
EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II

nanomad

Good job! Looking forward for a pull request from you!
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

Jerone

@chris_overseas - Good luck, I really hope you will make it! :)

I upgraded from MKII to 5DMKIII today just to shoot full HD RAW and then found out the hard way that ML isn't supported with new MK3's at all... Would really like to help but don't know much about coding :/

Walter Schulz


chris_overseas

Pull request submitted: https://bitbucket.org/hudson/magic-lantern/pull-request/354/ported-ml-over-to-5d3-firmware-123/

It works but there are still a few bugs. I've submitted the pull request because I don't have the skills to progress it much further, plus I'm not going to be able to work on this for the next week or so anyway. I'm hoping others will be able to do as much testing as possible, and hopefully some of the more knowledgeable devs can help out with the bug fixing. Enjoy!
EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II

Greg

Quote from: chris_overseas on January 26, 2014, 10:38:05 PM
Pull request submitted: https://bitbucket.org/hudson/magic-lantern/pull-request/354/ported-ml-over-to-5d3-firmware-123/

It works but there are still a few bugs. I've submitted the pull request because I don't have the skills to progress it much further, plus I'm not going to be able to work on this for the next week or so anyway. I'm hoping others will be able to do as much testing as possible, and hopefully some of the more knowledgeable devs can help out with the bug fixing. Enjoy!

I think you should create a new folder 5D3.123

Audionut

Big thanks to Chris for getting this moving.

Those people interested in running ML on 1.2.3 should checkout the pull request:  https://bitbucket.org/hudson/magic-lantern/pull-request/354/ported-ml-over-to-5d3-firmware-123

pompeiisneaks

I'm wanting to test it out, but I have two questions.

From that last link you gave it seems like they had a folder named .123, but they merged or renamed it to .113 and my latest pull from the mercurial repo didn't have a .123 dir.  I built the .113 dir just fine, but is that for the 1.2.3 firmware?  And second question, what of the built files is the .fir file is it the magiclantern.bin file?

Or what is the file I use to create the .fir file used in the normal installs?

~Phil

Audionut

If you already have the bootflag enabled.

Format a SD/CF card in camera, install Canon firmware 1.2.3, clone chris_overseas repo:  https://bitbucket.org/chris_miller/magic-lantern/overview

Build it just like you would any other 5D3 build.

Leave feedback.

pompeiisneaks

Oh ok, I'll use his repo, thanks,  What file is the .fir firmware file?  Is it the magiclantern.bin?

I tried looking around a while here on the site and didn't see it.

pompeiisneaks

Well I got that built just fine too, but looking at the makefile there should be a make fir to make the firmware but that doesn't seem to work, I've tried a few other make targets to no avail.  ugh, how do you create the firmware?

~Phil

Audionut

Quote from: Audionut on February 04, 2014, 03:38:22 AM
If you already have the bootflag enabled.

Format a SD/CF card in camera, install Canon firmware 1.2.3, clone chris_overseas repo:  https://bitbucket.org/chris_miller/magic-lantern/overview

Build it just like you would any other 5D3 build.

Leave feedback.

You only need the developer firmware from the alpha thread to enable the bootflag (prior to 1.2.3).

I do,
make clean
make 5D3
change to module directory
make
change to platform directory (5D3.123)
make zip


Extract zip and copy contents to card.
Good to go.

One day I'll sort out how to get the VM to pick up the card directly.

pompeiisneaks

That was exactly what I needed, running it right now, seems great so far, focus peaking, exposure zebras, etc working.  I'm still pretty  new to ML so I'm playing around to see what I can do

~Phil

pompeiisneaks

Question, some of the modules, raw_rec for example, fail to build with this error:

/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.4.7/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.4.7/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
collect2: ld returned 1 exit status
make[3]: *** [raw2dng] Error 1

Is this because it wants a 32 bit gcc?  Can I not build some of these modules on a 64 bit os?

~Phil

dmilligan

It's not actually the module that is failing to build, it's the converter program raw2dng that is meant to run on your computer and convert the raw files that ML spits out into DNGs. I gues your standard gcc compiler (not the arm cross-compiler) is not setup as expected.

pompeiisneaks

I ended up just coying the modules from the last nightly build.

After more raw playing, I note every time I record raw I get:

Hack error at ff4acd4:
expected e3a0032, got 6e654c2c

Is this a known bug or something related to the new 123 build?

Its still recording raw, but with my slow card I only get a few seconds. 

~Phil