Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: dfort on April 23, 2017, 08:17:23 AM

Title: Porting a Canon firmware update
Post by: dfort on April 23, 2017, 08:17:23 AM
Porting a (minor) Canon firmware update

(https://c1.staticflickr.com/3/2872/33398209813_963fa9edf3_z.jpg)

Pretty much every week someone posts something like, "When is Magic Lantern going to be working with X.X.X version firmware?" The answer is usually, "Downgrade to X.X.X -1." Why doesn't ML work with the latest Canon firmware? There are reasons including:
However, there are also reasons for porting to the latest Canon firmware:
Let's put the emphasis on you can meaning anyone reading this article should be able to port ML to a Canon firmware update, though it probably shouldn't the first project you attempt. You don't need to know anything about programming or reverse engineering and quite frankly most of what is required to do is grunt work that the main developers aren't really interested in doing. The first thing you need is to set up a development system in order to compile Magic Lantern. There are several ways to do this and there are plenty of tutorials for compiling ML so we won't go into it here.

Note that this article is about doing a relatively minor firmware update like the type Canon released around October of 2016 to fix a problem that affected a couple of lenses. Major updates that add new features or porting to a new camera isn't covered here.

OK--first step is getting ML working on your camera with the supported firmware. This is necessary in order to get firmware dumps, set the camera boot flag and to set the boot flag on the memory cards that you'll be using. You'll find the ROM dumps in the ML/LOGS directory of the card. Save those files. We'll get back to them soon.

You should install ML on a few cards if for nothing else so you'll have cards with the boot flag set. You can also set the card boot flag using a utility like EOScard (http://pel.hu/eoscard/) for Windows and for Mac try MacBoot (http://www.zenoshrdlu.com/macboot/macboot.html). I recommend having various different cards with the boot flag set because soon we'll be running something that might not work on certain cards or might corrupt the file system. @a1ex gives a good explanation of what is happening and how to work around it in Reply #1 (http://www.magiclantern.fm/forum/index.php?topic=19417.msg183579#msg183579) below. This explains why my trusty 2GB Patriot SD card works like a champ while the much higher performing SanDisk Extreme PRO cards I tried usually failed.

Keep one card without the boot flag in order to do the actual Canon firmware update. Copy both the Canon firmware that is currently working with ML and the version you are upgrading to. That way with one card you can upgrade and downgrade the Canon firmware. If you have several different camera models you can keep all the firmware versions on that one card because the camera will pick out only updates that are valid for that model.

Note that on the 5D3 you can't downgrade from 1.3.X in camera, you need to use EOS Utility. However, what EOS Utility does is to copy the firmware to the card and the camera takes over from there. This process is to discourage users from downgrading. There seems to be a good reason for this because 1.3.X has some added features that might corrupt user settings when downgrading. Simply clearing the Canon settings when downgrading seems to clear up those corruption issues.

Updating the Canon firmware

Here we go--Canon firmware update on card without the boot flag set, check. Update the firmware. Note that this is an article that attempts to cover all ML enabled cameras so I'll be mixing up screenshots from various models.

(https://c1.staticflickr.com/3/2827/32705592071_2c9799fb7d.jpg)

Dumping the ROM

When you install ML it sets a boot flag on the camera and a boot flag on the card. The camera will always check first to see if the card has the boot flag set and if it is set, attempt to boot off an autoexec.bin file on the card. There are some special autoexec.bin files available on the download page under the heading of ROM dumpers (https://builds.magiclantern.fm/), scroll down the page to find them. You are most like going to need the one labeled Portable. Put it on one of your cards that has the boot flag set and insert the card in your camera. It will probably start working without even turn on the power. You should see something like this:

(https://c1.staticflickr.com/3/2929/33824808820_ddf72c9aa9.jpg)

Or this:

(https://c1.staticflickr.com/3/2870/34051496252_61003a07a6.jpg)

Didn't happen on your first try? Some times it is stubborn so try it again or switch to another card. Once again, check out the comment on Reply #1 (http://www.magiclantern.fm/forum/index.php?topic=19417.msg183579#msg183579) if you are having trouble getting it to work. The point of this is to get a ROM dump of the new firmware.

If you want to be meticulous about it you should run an MD5 check on your file. When I did it both attempts had the same MD5 value and the dumps matched each other in size so I was confident that I had a good dump to work with. Lazy me. Open up a terminal (Powershell on Windows), navigate to where your ROM dumps and MD5 files are saved and:

Linux

For Linux and probably Cygwin on Windows:
md5sum -c *.BIN

Windows

For Windows 8 or higher (thanks Walter):
powershell "get-filehash *.BIN -A MD5 | format-list"

Macintosh

A stock Macintosh doesn't come with md5sum installed so you can either create the md5 checksum and compare it with the one generated by the portable rom dumper:
md5 ROM1A.BIN
MD5 (ROM1A.BIN) = 8a4d0fbfa6e6759d41b833bf764748fb
cat ROM1A.MD5
8a4d0fbfa6e6759d41b833bf764748fb  ROM1A.BIN

Or you can install md5sum. I use Homebrew:
brew install md5sha1sum
This version of md5sum couldn't check all of the dumps at once:
md5sum -c *.MD5
Error: --check <filename> cannot be used with additional files

So you'll have to do them one at a time:
md5sum -c ROM1A.MD5
ROM1A.BIN: OK


Now you've got several sets of verified ROM dumps, make sure you know which one is which. In most cases you'll only need to work with the ROM1.BIN for the Canon firmware that you started from and the one for the Canon firmware you're porting to.

Congratulations, you just completed the most essential step towards porting Magic Lantern!

If you're in a hurry, curious, stupid or all of the above you'll be tempted to try your current version of ML on the new firmware. Guess which description I fit into?

(https://c1.staticflickr.com/3/2942/34078165241_8e580d46aa.jpg)

Disassembly

There are various methods of disassembling the ROM dumps in order to work with them. Everyone seems to have their favorite method. ARMu (http://pel.hu/getpage.php?pg=armu&ac=1) is quick but only works on Windows--ok it also works on Mac under wine but I haven't figured it out yet. ARM-console (http://magiclantern.wikia.com/wiki/GPL_Tools/ARM_console) is very difficult to set up and use but it is very powerful--this currently my favorite disassembler. There is also a commercial product called IDA (https://www.hex-rays.com/products/ida/) but in order to disassemble ARM code you need to buy a license. By far the easiest for beginners is disassemble.pl (http://chdk.wikia.com/wiki/GPL:disassemble.pl) which is a perl script that needs just a few tweaks to get running properly as explained in this excellent tutorial on finding stubs (http://www.magiclantern.fm/forum/index.php?topic=12177.msg117735#msg117735).

What about QEMU? That could be very useful, especially when porting major firmware updates and new cameras but we're doing just a minor firmware update that can be done with just a few basic tools and a lot of perseverance.

Now it might be tempting to upload and share these ROM dumps and disassemblies but DON'T DO IT! You're holding copyrighted material that is owned by a very large and powerful corporation. You bought the camera so you can look at what is inside it but you aren't supposed to publish "intellectual property" in an open forum like this one. I'll be editing the bits in this post so they don't perfectly match the real disassembly but it should be good enough for instructional purposes.

So let's do this. Assuming disassemble.pl is in the same directory as your ROM1A.BIN file:
perl disassemble.pl 0xFF000000 ROM1A.BIN
string dump
create elf file
label scan
0xffff5a4c 
disassemble and string lookup
0xfffffffc 
job complete!


The file you are interested in is ROM1A.BIN.dis. You'll be spending a lot of time together.

Preparing your local repository for the update.

There are lots of ways to work with your local repository. It really depends how you are most comfortable. If you plan to eventually do a pull request you should consider creating a "scratch" or development branch where you can experiment and post all your notes and then when everything is working, copy over your changes to a fresh branch to avoid cluttering up the revision history with trivial commits (https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/2/update-to-5d3134-wip/commits).

Take a look at some recent firmware updates so you'll know what you are getting yourself into:

EOS M 2.0.2 - 2.0.3 (https://bitbucket.org/hudson/magic-lantern/pull-requests/792/update-to-eosm-firmware-revision-203/diff)
700D 1.1.4 - 1.1.5 (https://bitbucket.org/hudson/magic-lantern/pull-requests/813/update-to-700d115/diff)

This one is somewhat different because it doesn't replace any previous firmware version and is a bit more complex:

5D3 1.3.4 (https://bitbucket.org/hudson/magic-lantern/pull-requests/816/update-to-5d3134/diff)

The first thing you'll notice is that it seems that we're throwing out the old and creating new files. That's not really the case, we're renaming the old files but don't do it with the operating system because it defeats the point of keeping everything under version control. Generally I like to work with the SourceTree application (https://www.sourcetreeapp.com/) but copying directories is one of those things that is probably best done on the command line. There are two directories and one file that needs to be renamed. For example, when going from 1.1.4 to 1.1.5 on the 700D:

The platform directory
cd magic-lantern/platform
hg rename 700D.114 700D.115


The installer directory
cd magic-lantern/installer
hg rename 700D.114 700D.115


Editing the Source Code

Now for the good news. This step has registered a lot of changes but you'll only need to modify a few of these files. Most of the changes will be happening in the stubs.S file. While it is challenging looking for new stubs or finding stubs for a new camera model, a minor firmware update is rather easy.

Let's start with a very easy change. In magic-lantern/installer/Makefile you'll see this line:

SUPPORTED_INSTALLERS := 1100D.105 500D.111 50D.109 550D.109 5D2.212 5D3.113 60D.111 600D.102 650D.104 6D.116 700D.114 EOSM.202 7D.203


Pretty easy to figure out what to change on that line. However, there's something to watch out for when editing a Makefile, you got to make sure that your text editor doesn't change the indents from tabs to spaces. A Makefile needs those tabs.

Now check the pull requests on bitbucket and you'll see several other places where you need to change from the old to the new firmware version. Don't forget to make changes in the modules: adtg_gui, dual_iso, mlv_rec and mlv_lite.

Another rather easy change to make is to remove the old ML-SETUP.FIR file. It won't work with the new firmware and a developer will have to create a new one for you. In the case of a minor firmware update if you did the Canon update with the boot flag set on the camera, you won't have any problems later when you test out your new port. Just don't rush it! There are a few things that need to be done in the right order before attempting to boot into your newly ported ML.

In fact if you want to be extra careful you could take the advice that is in property.c and disable the prop_request_change function. If you're a suspenders and belt kind of guy also go into your platform's internals.h and comment out '#define CONFIG_PROP_REQUEST_CHANGE'. This is highly recommended for new ports because you can do some serious damage. I didn't take that precaution because mine were very minor updates.

Finding where the stubs moved

Now we get into the nitty gritty of finding stubs. We aren't porting a new camera, just a minor firmware update so all the stubs that you should be able to easily look up in the old disassembled ROM1.BIN disassembly should also be in the new disassembly--only in a different location.

If you're working with a DIGIC V the first thing you will notice in the stubs.S file is something like this (700D.114):

#define RAM_OFFSET (0xFFA5E8B0-0x1900) // Nanomad: some functions are copied to RAM at around ff0c0098; they have to be called from RAM...

There is a very good explanation of what this RAM_OFFSET is all about in a1ex's Tutorial: finding stubs (http://www.magiclantern.fm/forum/index.php?topic=12177.0). In this case Nanomad left a clue so let's check out what line 0xff0c0098 says in the 700D.114 disassembly. Depending which disassembler you used it might look like this:

ff0c0098: e59f0048 ldr r0, [pc, #72] ; pointer to sub_FFA5E8B0
ff0c009c: e59f1048 ldr r1, [pc, #72] ; 0xff0c00ec: pointer to 0x1900


or like this:

ff0c0098: e59f0048 ldr r0, [pc, #72] ; ff0c00e8: (ffa5e8b0)
ff0c009c: e59f1048 ldr r1, [pc, #72] ; ff0c00ec: (00001900)


Now let's look at what the 700D.115 disassembly looks like:

ff0c0098: e59f0048 ldr r0, [pc, #72] ; ff0c00e8: (ffa5e8b8)
ff0c009c: e59f1048 ldr r1, [pc, #72] ; ff0c00ec: (00001900)


Hooray! RAM_OFFSET changed from 0xFFA5E8B0-0x1900 to 0xFFA5E8B8-0x1900.

Now start going through the rest of the stubs.S file. One trick I discovered is that if you're tackling one of those minor updates Canon pushed in October of 2016, any stub that doesn't use RAM_OFFSET remains unchanged and those with the offset move the same amount in the same direction that RAM_OFFSET changed--for the most part. A few will move. Why? Because there was a few lines of code that were changed somewhere that shifted things around slightly. In any case you should check the address to every stub to make sure. In some cases there's a descriptive string around the stub you're checking into, other times there is nothing to go by except if several lines "look" the same. It becomes sort of a pattern matching game like those cartoons that you try to find the difference between two drawings that look almost identical.

What about those stubs that aren't addresses?

/** Task info **/
NSTUB(   0x25024,  task_max)
NSTUB(0xFFA0A0D8 - RAM_OFFSET,  is_taskid_valid)            // AJ_task_trampoline_related_p10
NSTUB(   0x23E14,  current_task)
NSTUB(     0x674,  current_interrupt)                       // in interrupt handler (0x18), where MEM(C0201004) is stored


You can take a chance that it didn't change but it is best to look at the hints, search the forum, read the wiki or post a question how to find these values. Leave those challenging ones until the end because by the time you've gone through all of the other stubs you'll become more of an expert at sleuthing out those stubs.

There are some helpful scripts in the magic-lantern/contrib directory. Try running check-stubs.py to get a report on the differences between the old and new stubs.S files. I posted my results on the 700D.115 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/813/update-to-700d115/diff#chg-platform/700D.114/stubs.S).

Make sure you got all of the changed addresses

Not all of the addresses you need to check are in the stubs.S file. The source code for some of the modules also have references to firmware versions and/or addresses that need checking and probably changing. For example, adtg_gui.c, dual_iso.c, mlv_rec.c and mlv_lite.c. By now your pattern matching skills are probably pretty good so good luck hunting them down.

Getting the firmware signature

The next step involves something a little more exciting that pouring over endless lines of disassembly code. You'll need to find the new firmware's signature and for that you'll need to build Magic Lantern, but this isn't a working build--not yet.

Take a look at fw-signature.h. Your updated firmware version is probably not in the list. Now check out boot-hack.c and you'll find this near the beginning of the file:

#if defined(CONFIG_HELLO_WORLD)
#include "fw-signature.h"
#endif


That's right, we're going to welcome your firmware update to the world. In order to do that open up config-defines.h and look for this piece of code:

/**
* Enable these for early ports
*/

    /** If CONFIG_EARLY_PORT is defined, only a few things will be enabled (e.g. changing version string) */
    //~ #define CONFIG_EARLY_PORT

    /** Load fonts and print Hello World (disable CONFIG_EARLY_PORT); will not start any other ML tasks, handlers etc. */
    //~ #define CONFIG_HELLO_WORLD
   
    /** Create a developer FIR for enabling the bootflag and dumping the ROM. */
    //~ #define CONFIG_DUMPER_BOOTFLAG


Uncomment the hello world line:

#define CONFIG_HELLO_WORLD

Now build Magic Lantern, put it on one of your boot flag enabled cards, insert it into your camera and you should see something like this:

(https://c1.staticflickr.com/3/2832/33398273973_f7c157edbf_z.jpg)

Write it down, put that address in fw-signature.h, comment out '#define CONFIG_HELLO_WORLD' and...

Trying it out

Ok so it should work but something will probably go wrong the first time you try it out. On the 700D.115 some of the fonts were missing. Searching for fonts I found this:

consts.h
// http://magiclantern.wikia.com/wiki/Fonts
#define BFNT_CHAR_CODES    0xFFCF67E4
#define BFNT_BITMAP_OFFSET 0xFFCF972C
#define BFNT_BITMAP_DATA   0xFFCFC674


So it took a bit more effort to find those addresses but magic-lantern/contrib/indy/find_fnt.py found them automatically.

On the 5D3.134 there were some other things that needed attention in consts.h. You would think that constants by definition don't change but like the old saying goes, "the only thing constant is change."

Oh, one last thing--the ML-SETUP.FIR file. Usually that's one of the first things that you need if you're working on a new port but since we're only doing a minor firmware update we can wait until the end. Only one of the main developers can prepare this file for you so you'll have to ask.




Here are the firmware updates I've been tracking:

EOS 5D Mark III, firmware 1.2.3 -> 1.3.4 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/816)
EOS 5D Mark III, firmware 1.2.3 -> 1.3.5 in progress (https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/14/update-to-5d135-wip/diff)
EOS 6D, firmware 1.1.6 -> 1.1.8 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/926) See this post. (https://www.magiclantern.fm/forum/index.php?topic=15088.msg200062#msg200062)
EOS 7D, firmware 2.0.3 -> 2.0.6 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/878/update-to-7d206/diff) Still needs some work.
EOS 60D, firmware 1.1.1 -> 1.1.2 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/917/update-to-60d112/diff) See this post (https://www.magiclantern.fm/forum/index.php?topic=14739.msg199918#msg199918) Should be good to merge.
EOS 500D, firmware 1.1.1 -> 1.1.2 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/916/update-to-500d112/diff) See this post (https://www.magiclantern.fm/forum/index.php?topic=11864.msg199872#msg199872)
EOS 550D, firmware 1.0.9 -> 1.1.0 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/905/update-to-550d110/diff) See this post. (http://www.magiclantern.fm/forum/index.php?topic=19417.msg192667#msg192667) Some issues noted.
EOS 600D, firmware 1.0.2 -> 1.0.3 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/915/update-to-600d103/diff) See this post. (https://www.magiclantern.fm/forum/index.php?topic=15360.msg199663#msg199663) Should be good to merge.
EOS 650D, firmware 1.0.4 -> 1.0.5 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/914/update-to-650d105/diff) See this post. (https://www.magiclantern.fm/forum/index.php?topic=7473.msg199357#msg199357) Should be good to merge.
EOS 1100D, firmware 1.0.5 -> 1.0.6 pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/920/update-to-1100d106/diff) See this post. (https://www.magiclantern.fm/forum/index.php?topic=1009.msg200539#msg200539)
EOS M, firmware 2.0.2 -> 2.0.3 unified branch pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/792/update-to-eosm-firmware-revision-203/diff), crop_rec_4k branch pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/879/crop-rec-4k-eosm203/diff) Should be good to merge.
EOS M50, firmware 1.0.1 -> 1.0.2 digic6-dumper branch pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/955/m50-update-to-canon-firmware-version-102/diff), Still an early port but it should be good to merge.

Title: Re: Porting a Canon firmware update
Post by: a1ex on April 23, 2017, 11:15:34 AM
Very nice write-up (and very easy to follow!)

Some minor correction:

Quote from: dfort on April 23, 2017, 08:17:23 AM
Didn't happen on your first try? Some times it is stubborn so try it again or switch to another card. The point of this is to get a ROM dump of the new firmware. If you want to be meticulous about it you should run an MD5 check on your file.

Here, checking the MD5 is not about being meticulous; the bootloader file I/O routines *are* really stubborn. They may corrupt the filesystem, insert a few or a lot of wrong bytes, lock up the camera, refuse to run on any or all of your cards and so on.

So far, what worked every single time was formatting the card at a much lower capacity (I've used 256MB successfully in many cases). To do this, you can write the SD image that comes with QEMU (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/sd.img.xz) to your SD or CF card. This guide (https://thepihut.com/blogs/raspberry-pi-tutorials/17789160-backing-up-and-restoring-your-raspberry-pis-sd-card) is helpful; just don't forget to unzip the SD image. This image contains the portable display test (http://www.magiclantern.fm/forum/index.php?topic=14732) and is bootable (so, you can test it in the camera straight away).

Even in this case, checking the MD5 is mandatory. On Linux, this is as simple as:

md5sum -c *.MD5
Title: Re: Porting a Canon firmware update
Post by: Walter Schulz on April 23, 2017, 01:22:26 PM
Quote from: a1ex on April 23, 2017, 11:15:34 AM
Even in this case, checking the MD5 is mandatory. On Linux, this is as simple as:

md5sum -c *.MD5


Windows 8 (or higher):
powershell get-filehash *.MD5 -A MD5 | format-list
Title: Re: Porting a Canon firmware update
Post by: dfort on May 13, 2017, 02:36:35 AM
Ok--finished the tutorial. Next time someone asks when ML will work with the x.x.x firmware update, point them here (http://www.magiclantern.fm/forum/index.php?topic=19417.0).
Title: Re: Porting a Canon firmware update
Post by: dfort on May 13, 2017, 05:54:25 PM
Moved this post about firmware update ports in progress to the end of the original post (https://www.magiclantern.fm/forum/index.php?topic=19417.msg183571#msg183571).
Title: Re: Porting a Canon firmware update
Post by: ItsMeLenny on May 14, 2017, 04:14:20 AM
I should really attempt the 550D at some point.
This comment is also a note so I can refind this thread.
Title: Re: Porting a Canon firmware update
Post by: qqluqq on May 15, 2017, 12:46:01 AM
Quote from: ItsMeLenny on May 14, 2017, 04:14:20 AM
I should really attempt the 550D at some point.
This comment is also a note so I can refind this thread.

I already started on the 550D checking stubs atm
Title: Re: Porting a Canon firmware update
Post by: qqluqq on May 16, 2017, 08:43:23 PM
followed the guide
found all adresses

now trying to run hello world but screen just stays black when I try it,
any idea on where I should look to fix that?

code here (not working)
https://bitbucket.org/qqluqq/magic-lantern/branch/550D%20110

also, why do you replace old fw version instead of leaving both in?
Title: Re: Porting a Canon firmware update
Post by: Walter Schulz on May 16, 2017, 08:50:19 PM
Quote from: qqluqq on May 16, 2017, 08:43:23 PM
also, why do you replace old fw version instead of leaving both in?

?
Title: Re: Porting a Canon firmware update
Post by: qqluqq on May 16, 2017, 08:58:48 PM
Quote from: Walter Schulz on May 16, 2017, 08:50:19 PM
?

He renames the old version instead of copying, shouldn't both the new and old be in the source?
Title: Re: Porting a Canon firmware update
Post by: dfort on May 16, 2017, 09:03:40 PM
Quote from: qqluqq on May 16, 2017, 08:43:23 PM
also, why do you replace old fw version instead of leaving both in?

It is possible to include both Canon firmware versions but it is easier to support one version at a time. The 5D3 is an exception. The developers didn't want to remove 1.1.3 because 1.2.3 and up has some issues with ML. Minor Canon firmware updates like the one you're doing shouldn't introduce any new bugs, at least not in theory.
Title: Re: Porting a Canon firmware update
Post by: a1ex on May 16, 2017, 09:40:23 PM
@qqluqq: 550D can be debugged in QEMU (much easier than double-checking everything blindly, hoping to find a needle in the haystack).

Refer to this page (http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/) for installation and for checking what works and what not.
Title: Re: Porting a Canon firmware update
Post by: dfort on May 23, 2017, 01:38:13 AM
Thanks Walter -- edited original post.
Title: Re: Porting a Canon firmware update
Post by: dfort on November 01, 2017, 11:58:30 PM
Got my hands on a 7D so I thought hey, what the heck. No big deal, right? This one turned out the be quite a challenge because the portable dumper doesn't work on it. Well it works in QEMU but not on the camera.

My timing was pretty good because g3gg0 added a display dumper to the portable dumper in order to copy the ROM of some of the newer cameras. Turns out that it worked on the 7D. You can follow along the adventures of learning how to use the display dumper on the Portable ROM dumper topic (http://www.magiclantern.fm/forum/index.php?topic=16534.0) starting here (http://www.magiclantern.fm/forum/index.php?topic=16534.msg192016#msg192016).

I should mention that porting a firmware update almost never works the first time. First time I ran it on the camera it looked like this:

(https://farm5.staticflickr.com/4473/38073402161_f44a5583f2.jpg) (https://flic.kr/p/211q91i)

I was sure the font address was fine because I used contrib/indy/find_fnt.py to verify that it didn't change:

python find_fnt.py 7D.206/ROM1.BIN

Find bitmap fonts in Canon DSLR firmwares
Arm.Indy. based on work by Pel, Trammel Hudson and A1ex
Assume ROM file was dumped from 0xf8000000

0xf8d369b4: FNT
0xf8d369b8: (+0x04) 0xffd8
0xf8d369ba: (+0x06) font_width = 40
0xf8d369bc: (+0x08) charmap_offset = 0x24
0xf8d369c0: (+0x0c) charmap_size = 0x2bcc
0xf8d369c4: (+0x10) bitmap_size = 0x795fa
0xf8d369c8: (+0x14) font name = 'HCanonGothic///'
0xf8d369d8: (+0x24) char_codes[]. 2803 chars
0xf8d395a4: (+0x2bf0) offsets[]. Last offset value = 0x79528
0xf8d3c170: (+0x57bc) bitmaps[]
  0xf8db5698: (+0x7ece4) last bitmap
  +0x00: bitmap width = 36
  +0x02: bitmap height = 36
  +0x04: char width = 36
  +0x06: X offset = 0
  +0x08: Y offset = 0
    bitmap size = 0xb4

0xf8db576c: FNT
0xf8db5770: (+0x04) 0xffd8
0xf8db5772: (+0x06) font_width = 40
0xf8db5774: (+0x08) charmap_offset = 0x24
0xf8db5778: (+0x0c) charmap_size = 0x180
0xf8db577c: (+0x10) bitmap_size = 0x30c0
0xf8db5780: (+0x14) font name = 'CanonMonospace'
0xf8db5790: (+0x24) char_codes[]. 96 chars
0xf8db5910: (+0x1a4) offsets[]. Last offset value = 0x303e
0xf8db5a90: (+0x324) bitmaps[]
  0xf8db8ace: (+0x3362) last bitmap
  +0x00: bitmap width = 22
  +0x02: bitmap height = 22
  +0x04: char width = 22
  +0x06: X offset = 0
  +0x08: Y offset = 0
    bitmap size = 0x42


Running contrib/stub-checker/check-stubs.py there were a few suspicious jumps in the offset and after fixing those stubs, which used very similar code:

python check-stubs.py -n 7D.203/stubs.S 7D.206/stubs.S

Module termcolor missing, no color support will be available
STUB                                                    OLD           NEW     DELTA
current_interrupt                                    0x00000674 -> 0x00000674 [0x000]
next_task_ctx                                        0x0000067c -> 0x0000067c [0x000]
current_task_ctx                                     0x00000680 -> 0x00000680 [0x000]
task_dispatch_hook                                   0x00001934 -> 0x00001934 [0x000]
pre_isr_hook                                         0x0000193c -> 0x0000193c [0x000]
post_isr_hook                                        0x00001940 -> 0x00001940 [0x000]
current_task                                         0x00001a1c -> 0x00001a1c [0x000]
gui_main_struct                                      0x00001bf4 -> 0x00001bf4 [0x000]
sounddev                                             0x00001f54 -> 0x00001f54 [0x000]
dm_names                                             0x00002b18 -> 0x00002b18 [0x000]
task_max                                             0x00002b24 -> 0x00002b24 [0x000]
gui_timer_struct                                     0x00003624 -> 0x00003624 [0x000]
mvr_config                                           0x0000a39c -> 0x0000a39c [0x000]
additional_version                                   0x00016fb8 -> 0x00016fb8 [0x000]
gui_task_list                                        0x0001a904 -> 0x0001a904 [0x000]
cf_device                                            0x00022938 -> 0x00022938 [0x000]
bmp_vram_info                                        0x00029a60 -> 0x00029a60 [0x000]
LCD_Palette                                          0x00029a9c -> 0x00029a9c [0x000]
camera_engine                                        0xc0220000 -> 0xc0220000 [0x000]
cstart                                               0xff010fb0 -> 0xff010fb0 [0x000] [!!!]
create_init_task                                     0xff01559c -> 0xff0155a0 [0x004]
_malloc                                              0xff0162f0 -> 0xff0162f4 [0x004]
_free                                                0xff01634c -> 0xff016350 [0x004]
init_task                                            0xff018734 -> 0xff018738 [0x004]
gui_main_task                                        0xff021b44 -> 0xff021b48 [0x004]
GUI_Control                                          0xff021f04 -> 0xff021f08 [0x004]
GUI_ChangeMode                                       0xff022004 -> 0xff022008 [0x004]
gui_init_end                                         0xff022680 -> 0xff022684 [0x004]
SRM_AllocateMemoryResourceFor1stJob                  0xff02cd78 -> 0xff02cd4c [0x02c]
SRM_FreeMemoryResourceFor1stJob                      0xff02f098 -> 0xff02f06c [0x02c]
StartASIFDMAADC                                      0xff061a88 -> 0xff061a5c [0x02c]
StopASIFDMAADC                                       0xff061c08 -> 0xff061bdc [0x02c]
StartASIFDMADAC                                      0xff061c44 -> 0xff061c18 [0x02c]
StopASIFDMADAC                                       0xff061d20 -> 0xff061cf4 [0x02c]
SetNextASIFADCBuffer                                 0xff0621c4 -> 0xff062198 [0x02c]
SetNextASIFDACBuffer                                 0xff06227c -> 0xff062250 [0x02c]
sounddev_task                                        0xff063c70 -> 0xff063c44 [0x02c]
SoundDevActiveIn                                     0xff0640ec -> 0xff0640c0 [0x02c]
sounddev_active_in                                   0xff0640ec -> 0xff0640c0 [0x02c]
SoundDevShutDownIn                                   0xff064304 -> 0xff0642d8 [0x02c]
_prop_request_change                                 0xff067fac -> 0xff067f80 [0x02c]
prop_deliver                                         0xff06858c -> 0xff068560 [0x02c]
prop_register_slave                                  0xff068614 -> 0xff0685e8 [0x02c]
prop_unregister_slave                                0xff0686a4 -> 0xff068678 [0x02c]
_prop_cleanup                                        0xff068734 -> 0xff068708 [0x02c]
EnableImagePhysicalScreenParameter                   0xff06e8b0 -> 0xff06e884 [0x02c]
_audio_ic_read                                       0xff072f7c -> 0xff072f50 [0x02c]
_audio_ic_write                                      0xff0730c8 -> 0xff07309c [0x02c]
PowerAudioOutput                                     0xff07365c -> 0xff073630 [0x02c]
SetSamplingRate                                      0xff0736f4 -> 0xff0736c8 [0x02c]
SetAudioVolumeOut                                    0xff073944 -> 0xff073918 [0x02c]
LoadCalendarFromRTC                                  0xff075210 -> 0xff0751e4 [0x02c]
DryosDebugMsg                                        0xff0776ac -> 0xff077680 [0x02c]
dm_set_store_level                                   0xff077990 -> 0xff077964 [0x02c]
call                                                 0xff07842c -> 0xff078400 [0x02c]
SetHPTimerAfterNow                                   0xff078e6c -> 0xff078e40 [0x02c]
SetHPTimerNextTick                                   0xff078eec -> 0xff078ec0 [0x02c]
BulkOutIPCTransfer                                   0xff07a704 -> 0xff07a6d8 [0x02c]
BulkInIPCTransfer                                    0xff07a7d4 -> 0xff07a7a8 [0x02c]
QuickOutIPCTransfer                                  0xff07a954 -> 0xff07a928 [0x02c]
RegisterRPCHandler                                   0xff07aed4 -> 0xff07aea8 [0x02c]
RequestRPC                                           0xff07af84 -> 0xff07af58 [0x02c]
gui_timer_something                                  0xff07b708 -> 0xff07b6dc [0x02c]
create_named_semaphore                               0xff07b7c4 -> 0xff07b798 [0x02c]
take_semaphore                                       0xff07b924 -> 0xff07b8f8 [0x02c]
give_semaphore                                       0xff07ba0c -> 0xff07b9e0 [0x02c]
msleep                                               0xff07bdf8 -> 0xff07bdcc [0x02c]
task_create                                          0xff07beac -> 0xff07be80 [0x02c]
GetSizeOfMaxRegion                                   0xff07ca74 -> 0xff07ca48 [0x02c]
GetMemoryInformation                                 0xff07cac8 -> 0xff07ca9c [0x02c]
_AllocateMemory                                      0xff07cd24 -> 0xff07ccf8 [0x02c]
_FreeMemory                                          0xff07d0b4 -> 0xff07d088 [0x02c]
CreateMemoryChunk                                    0xff07d97c -> 0xff07d950 [0x02c]
GetMemoryAddressOfMemoryChunk                        0xff07dee8 -> 0xff07debc [0x02c]
DeleteMemorySuite                                    0xff07df5c -> 0xff07df30 [0x02c]
CreateMemorySuite                                    0xff07e12c -> 0xff07e100 [0x02c]
AddMemoryChunk                                       0xff07e3cc -> 0xff07e3a0 [0x02c]
GetFirstChunkFromSuite                               0xff07e4a0 -> 0xff07e474 [0x02c]
GetNextMemoryChunk                                   0xff07ec38 -> 0xff07ec0c [0x02c]
msg_queue_create                                     0xff07fc28 -> 0xff07fbfc [0x02c]
CreateRecursiveLock                                  0xff07fe0c -> 0xff07fde0 [0x02c]
dma_memcpy                                           0xff080224 -> 0xff0801f8 [0x02c]
AllocateMemoryResource                               0xff080e78 -> 0xff080e4c [0x02c]
AllocateContinuousMemoryResource                     0xff080ec4 -> 0xff080e98 [0x02c]
FreeMemoryResource                                   0xff080fdc -> 0xff080fb0 [0x02c]
SetTimerAfter                                        0xff082500 -> 0xff0824d4 [0x02c]
CancelTimer                                          0xff082700 -> 0xff0826d4 [0x02c]
task_trampoline                                      0xff084ca4 -> 0xff084c78 [0x02c]
is_taskid_valid                                      0xff087940 -> 0xff087914 [0x02c]
DispSensorStart                                      0xff08e350 -> 0xff08e324 [0x02c]
SetGUIRequestMode                                    0xff0a8738 -> 0xff0a870c [0x02c]
gui_massive_event_loop                               0xff0a9384 -> 0xff0a9358 [0x02c]
gui_change_mode                                      0xff0aabc4 -> 0xff0aab98 [0x02c]
gui_local_post                                       0xff0ab1c0 -> 0xff0ab194 [0x02c]
gui_other_post                                       0xff0ab5dc -> 0xff0ab5b0 [0x02c]
gui_post_10000085                                    0xff0ab858 -> 0xff0ab82c [0x02c]
gui_change_shoot_type_post                           0xff0ab8e4 -> 0xff0ab8b8 [0x02c]
gui_change_lcd_state_post                            0xff0ab97c -> 0xff0ab950 [0x02c]
gui_init_event                                       0xff0ac000 -> 0xff0abfd4 [0x02c]
ptpPropSetUILock                                     0xff15fc84 -> 0xff15fc58 [0x02c]
PtpDps_remote_release_SW1_SW2_worker                 0xff16004c -> 0xff160020 [0x02c]
mvrSetDefQScale                                      0xff1e08e0 -> 0xff1e08b4 [0x02c]
mvrSetFullHDOptSize                                  0xff1e0918 -> 0xff1e08ec [0x02c]
mvrSetGopOptSizeFULLHD                               0xff1e0b08 -> 0xff1e0adc [0x02c]
mvrFixQScale                                         0xff1e0e04 -> 0xff1e0dd8 [0x02c]
SetEDmac                                             0xff1f5360 -> 0xff1f5334 [0x02c]
ConnectWriteEDmac                                    0xff1f5418 -> 0xff1f53ec [0x02c]
ConnectReadEDmac                                     0xff1f54dc -> 0xff1f54b0 [0x02c]
StartEDmac                                           0xff1f5660 -> 0xff1f5634 [0x02c]
AbortEDmac                                           0xff1f57b0 -> 0xff1f5784 [0x02c]
RegisterEDmacCompleteCBR                             0xff1f5850 -> 0xff1f5824 [0x02c]
UnregisterEDmacCompleteCBR                           0xff1f5864 -> 0xff1f5838 [0x02c]
RegisterEDmacAbortCBR                                0xff1f5878 -> 0xff1f584c [0x02c]
UnregisterEDmacAbortCBR                              0xff1f58b8 -> 0xff1f588c [0x02c]
RegisterEDmacPopCBR                                  0xff1f5980 -> 0xff1f5954 [0x02c]
UnregisterEDmacPopCBR                                0xff1f5994 -> 0xff1f5968 [0x02c]
shamem_read                                          0xff1f664c -> 0xff1f6620 [0x02c]
_EngDrvOut                                           0xff1f675c -> 0xff1f6730 [0x02c]
_engio_write                                         0xff1f6b20 -> 0xff1f6af4 [0x02c]
CreateResLockEntry                                   0xff1f9694 -> 0xff1f9668 [0x02c]
LockEngineResources                                  0xff1f9bdc -> 0xff1f9bb0 [0x02c]
UnLockEngineResources                                0xff1f9d18 -> 0xff1f9cec [0x02c]
_FIO_OpenFile                                        0xff1fc224 -> 0xff1fc1f8 [0x02c]
_FIO_CreateFile                                      0xff1fc2e0 -> 0xff1fc2b4 [0x02c]
_FIO_RemoveFile                                      0xff1fc38c -> 0xff1fc360 [0x02c]
_FIO_ReadFile                                        0xff1fc434 -> 0xff1fc408 [0x02c]
FIO_SeekSkipFile                                     0xff1fc4e4 -> 0xff1fc4b8 [0x02c]
_FIO_WriteFile                                       0xff1fc5d4 -> 0xff1fc5a8 [0x02c]
FIO_CloseFile                                        0xff1fc684 -> 0xff1fc658 [0x02c]
_FIO_GetFileSize                                     0xff1fc800 -> 0xff1fc7d4 [0x02c]
_FIO_CreateDirectory                                 0xff1fccf4 -> 0xff1fccc8 [0x02c]
_FIO_FindFirstEx                                     0xff1fd2b8 -> 0xff1fd28c [0x02c]
FIO_FindNextEx                                       0xff1fd42c -> 0xff1fd400 [0x02c]
FIO_FindClose                                        0xff1fd554 -> 0xff1fd528 [0x02c]
LightMeasure_n_Callback_r0                           0xff202814 -> 0xff2027e8 [0x02c]
msg_queue_receive                                    0xff205c70 -> 0xff205c44 [0x02c]
msg_queue_post                                       0xff205e5c -> 0xff205e30 [0x02c]
msg_queue_count                                      0xff205e9c -> 0xff205e70 [0x02c]
AcquireRecursiveLock                                 0xff205fa8 -> 0xff205f7c [0x02c]
ReleaseRecursiveLock                                 0xff2060bc -> 0xff206090 [0x02c]
vsnprintf                                            0xff209670 -> 0xff209644 [0x02c]
_alloc_dma_memory                                    0xff20da50 -> 0xff20da24 [0x02c]
_free_dma_memory                                     0xff20da84 -> 0xff20da58 [0x02c]
GUI_SetRollingPitchingLevelStatus                    0xff236e9c -> 0xff236e70 [0x02c]
GUI_SetLvMode                                        0xff237724 -> 0xff2376f8 [0x02c]
Gui_SetSoundRecord                                   0xff237858 -> 0xff23782c [0x02c]
SetAudioVolumeIn                                     0xff237a90 -> 0xff237a64 [0x02c]
ptp_register_handler                                 0xff2885cc -> 0xff2885a0 [0x02c]
dialog_redraw                                        0xff300640 -> 0xff300614 [0x02c]
ErrForCamera_handler                                 0xff336d60 -> 0xff336d34 [0x02c]
LiveViewApp_handler                                  0xff347918 -> 0xff3478ec [0x02c]
LiveViewApp_handler_BL_JudgeBottomInfoDispTimerState 0xff347e6c -> 0xff347e40 [0x02c]
LiveViewApp_handler_end                              0xff348d94 -> 0xff348d68 [0x02c]
CancelDateTimer                                      0xff34e5b0 -> 0xff34e584 [0x02c]
PlayMain_handler                                     0xff3552dc -> 0xff3552b0 [0x02c]
ShootOlcApp_handler                                  0xff35df58 -> 0xff35df2c [0x02c]
dialog_set_property_str                              0xff361bdc -> 0xff361bb0 [0x02c]
LiveViewWbApp_handler                                0xff3674a4 -> 0xff367478 [0x02c]
GUI_SetCFnForTab4                                    0xff375998 -> 0xff37596c [0x02c]
GUI_GetCFnForTab4                                    0xff3759f0 -> 0xff3759c4 [0x02c]
fsuDecodePartitionTable                              0xff3d7798 -> 0xff3d7770 [0x028]
StopPlayProtectGuideApp                              0xff41dde0 -> 0xff41ddb8 [0x028]
StartPlayProtectGuideApp                             0xff41e17c -> 0xff41e154 [0x028]
PlayMovieGuideApp_handler                            0xff42b700 -> 0xff42b6d8 [0x028]
ErrCardForLVApp_handler                              0xff430250 -> 0xff430228 [0x028]
GetCFnData                                           0xff4c621c -> 0xff4c61f4 [0x028]
SetCFnData                                           0xff4c6424 -> 0xff4c63fc [0x028]
bzero32                                              0xff50cdec -> 0xff50cdc4 [0x028]
lv_path_struct                                       0xff5517b0 -> 0xff551788 [0x028]
audio_thresholds                                     0xff57b014 -> 0xff57afec [0x028]


it booted up fine.

(https://farm5.staticflickr.com/4492/38068235592_363362244c.jpg) (https://flic.kr/p/ZZXEaw)

Next, how about running some tests? The stub check test in the selftest module passed with flying colors:

(https://farm5.staticflickr.com/4457/38048783032_7db535284d.jpg) (https://flic.kr/p/ZYeXAy)

However, the lua api test caused all sorts of issues. The camera would hang requiring a battery pull and I seldom was able to get a test to run much less to complete. The most complete test I got was with an STM lens that apparently wasn't controllable via lua on the 7D. I'll publish these tests on separate posts because of the per post character limit on this forum.

Rather disappointing so I regressed to the old 2.0.3 firmware and found out that was also happening. Good to know I didn't break anything but strange nonetheless.

Speaking of strange, the 7D has two processors, a "master" and a "slave" and ML only runs on the slave. So what does the 7D_MASTER platform do? I could only get this out of it:

(https://farm5.staticflickr.com/4488/37362131024_74741b3bc6.jpg) (https://flic.kr/p/YVyFSQ)

Just for good form I updated the 7D_MASTER but since I couldn't figure out any of it, all the addresses are still for the 2.0.3 firmware.

This was an interesting exercise so I thought I'd share the experience and post a pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/878/update-to-7d206/diff).
Title: Re: Porting a Canon firmware update
Post by: dfort on November 02, 2017, 12:03:28 AM
Here is the output of the selftest module on the 7D.206 update:

STUBTEST.LOG
[Pass] is_play_mode() => 0x1
[Pass] src = fio_malloc(size) => 0x42104094
[Pass] dst = fio_malloc(size) => 0x429080a0
[Pass] memcmp(dst, src, 4097) => 0x93
[Pass] edmac_memcpy(dst, src, 4097) => 0x429080a0
[Pass] memcmp(dst, src, 4097) => 0x0
[Pass] edmac_memcpy(dst, src, 4097) => 0x429080a0
[Pass] memcmp(dst, src, size) => 0x30
[Pass] edmac_memcpy(dst, src, size) => 0x429080a0
[Pass] memcmp(dst, src, size) => 0x0
[Pass] memcmp(dst, src, size) => 0xffffff82
[Pass] edmac_memcpy_start(dst, src, size) => 0x429080a0
       dt => 0x3ac7
[Pass] copied => 0x401a80
[Pass] copied => 0x401a80
[Pass] copied => 0x401a80
[Pass] memcmp(dst, src, copied) => 0x0
[Pass] memcmp(dst, src, copied + 16) => 0xec
       edmac_memcpy_finish()
       free(src)
       free(dst)
Cache test A (EDMAC on BMP buffer)...
[Pass] bmp = bmp_load("ML/CROPMKS/CINESCO2.BMP", 1) => 0xc5508
[Pass] old => 0x0
[Pass] irq => 0xc0
[Pass] differences => 0x280
[Pass] old => 0x0
[Pass] irq => 0xc0
[Pass] differences => 0x0
Cache test B (FIO on 8K buffer)...
[Pass] tries[0] => 0xfa
[Pass] tries[1] => 0xea
[Pass] tries[2] => 0xfe
[Pass] tries[3] => 0x106
[Pass] failr[0] => 0x7c
[Pass] failw[0] => 0x4b
[Pass] failr[1] => 0x6f
[Pass] failw[1] => 0x0
[Pass] failr[2] => 0x0
[Pass] failw[2] => 0x4d
[Pass] failr[3] => 0x0
[Pass] failw[3] => 0x0
       times[0] / tries[0] => 0x47
       times[1] / tries[1] => 0x4e
       times[2] / tries[2] => 0x5a
       times[3] / tries[3] => 0x55
Cache tests finished.

[Pass] f = FIO_CreateFile("test.dat") => 0x3
[Pass] FIO_WriteFile(f, (void*)0xFF000000, 0x10000) => 0x10000
[Pass] FIO_WriteFile(f, (void*)0xFF000000, 0x10000) => 0x10000
       FIO_CloseFile(f)
[Pass] FIO_GetFileSize("test.dat", &size) => 0x0
[Pass] size => 0x20000
[Pass] p = (void*)_alloc_dma_memory(0x20000) => 0x4097f798
[Pass] f = FIO_OpenFile("test.dat", O_RDONLY | O_SYNC) => 0x3
[Pass] FIO_ReadFile(f, p, 0x20000) => 0x20000
       FIO_CloseFile(f)
       _free_dma_memory(p)
[Pass] count => 0x3a98
[Pass] buf = fio_malloc(0x1000000) => 0x42104094
[Pass] FIO_GetFileSize_direct("test.dat") => 0x82000000
[Pass] f = FIO_OpenFile("test.dat", O_RDWR | O_SYNC) => 0x3
[Pass] FIO_SeekSkipFile(f, 0, SEEK_END) => 0x82000000
[Pass] FIO_WriteFile(f, buf, 0x10) => 0x10
[Pass] FIO_SeekSkipFile(f, -0x20, SEEK_END) => 0x81fffff0
[Pass] FIO_WriteFile(f, buf, 0x30) => 0x30
[Pass] FIO_SeekSkipFile(f, 0x20, SEEK_SET) => 0x20
[Pass] FIO_SeekSkipFile(f, 0x30, SEEK_CUR) => 0x50
[Pass] FIO_SeekSkipFile(f, -0x20, SEEK_CUR) => 0x30
[Pass] FIO_GetFileSize_direct("test.dat") => 0x82000020
[Pass] is_file("test.dat") => 0x1
[Pass] FIO_RemoveFile("test.dat") => 0x0
[Pass] is_file("test.dat") => 0x0
[Pass] SetTimerAfter(0, timer_cbr, overrun_cbr, 0) => 0x15
[Pass] timer_func => 0x2
[Pass] SetTimerAfter(1000, timer_cbr, overrun_cbr, 0) => 0x1a8
       msleep(900)
[Pass] timer_func => 0x0
       msleep(200)
[Pass] timer_func => 0x1
[Pass] ABS((timer_time/1000 - t0) - 1000) => 0xb
[Pass] ABS((timer_arg - ta0) - 1000) => 0x14
[Pass] timer = SetTimerAfter(1000, timer_cbr, overrun_cbr, 0) => 0x1aa
       msleep(400)
       CancelTimer(timer)
[Pass] timer_func => 0x0
       msleep(1500)
[Pass] timer_func => 0x0
[Pass] SetHPTimerAfterNow(0, timer_cbr, overrun_cbr, 0) => 0x15
[Pass] timer_func => 0x2
[Pass] SetHPTimerAfterNow(100000, timer_cbr, overrun_cbr, 0) => 0x34e6
       msleep(90)
[Pass] timer_func => 0x0
       msleep(20)
[Pass] timer_func => 0x1
[Pass] ABS(DeltaT(timer_time, t0) - 100000) => 0x137
[Pass] ABS(DeltaT(timer_arg, ta0) - 100000) => 0x107
[Pass] ABS((get_us_clock_value() - t0) - 110000) => 0xffffffbc
[Pass] SetHPTimerAfterNow(90000, next_tick_cbr, overrun_cbr, 0) => 0x34f0
       msleep(80)
[Pass] timer_func => 0x0
       msleep(20)
[Pass] timer_func => 0x3
       msleep(80)
[Pass] timer_func => 0x3
       msleep(20)
[Pass] timer_func => 0x1
[Pass] ABS(DeltaT(timer_time, t0) - 300000) => 0x70
[Pass] ABS(DeltaT(timer_arg, ta0) - 300000) => 0x42
[Pass] ABS((get_us_clock_value() - t0) - 310000) => 0xffffffd5
       t0 = *(uint32_t*)0xC0242014 => 0xb9a5f
       msleep(250)
       t1 = *(uint32_t*)0xC0242014 => 0xf5792
[Pass] ABS(MOD(t1-t0, 1048576)/1000 - 250) => 0x5
       LoadCalendarFromRTC( &now )
       s0 = now.tm_sec => 0x7
       Date/time: 2017/10/31 21:00:07
       msleep(1500)
       LoadCalendarFromRTC( &now )
       s1 = now.tm_sec => 0x9
[Pass] MOD(s1-s0, 60) => 0x2
[Pass] MOD(s1-s0, 60) => 0x2
       m0 = MALLOC_FREE_MEMORY => 0x3e2c8
[Pass] p = (void*)_malloc(50*1024) => 0xc7258
[Pass] CACHEABLE(p) => 0xc7258
       m1 = MALLOC_FREE_MEMORY => 0x31ab8
       _free(p)
       m2 = MALLOC_FREE_MEMORY => 0x3e2c8
[Pass] ABS((m0-m1) - 50*1024) => 0x10
[Pass] ABS(m0-m2) => 0x0
       m0 = GetFreeMemForAllocateMemory() => 0x2be070
[Pass] p = (void*)_AllocateMemory(256*1024) => 0x97f758
[Pass] CACHEABLE(p) => 0x97f758
       m1 = GetFreeMemForAllocateMemory() => 0x27e064
       _FreeMemory(p)
       m2 = GetFreeMemForAllocateMemory() => 0x2be070
[Pass] ABS((m0-m1) - 256*1024) => 0xc
[Pass] ABS(m0-m2) => 0x0
       m01 = MALLOC_FREE_MEMORY => 0x3e2c8
       m02 = GetFreeMemForAllocateMemory() => 0x2be070
[Pass] p = (void*)_alloc_dma_memory(256*1024) => 0x4097f798
[Pass] UNCACHEABLE(p) => 0x4097f798
[Pass] CACHEABLE(p) => 0x97f798
[Pass] UNCACHEABLE(CACHEABLE(p)) => 0x4097f798
       _free_dma_memory(p)
[Pass] p = (void*)_shoot_malloc(24*1024*1024) => 0x42104074
[Pass] UNCACHEABLE(p) => 0x42104074
       _shoot_free(p)
       m11 = MALLOC_FREE_MEMORY => 0x3e2c8
       m12 = GetFreeMemForAllocateMemory() => 0x2be070
[Pass] ABS(m01-m11) => 0x0
[Pass] ABS(m02-m12) => 0x0
[Pass] suite = shoot_malloc_suite_contig(24*1024*1024) => 0xa5388
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0x1
[Pass] suite->size => 0x1800000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0xa53b0
[Pass] chunk->signature => 'MemChunk'
[Pass] chunk->size => 0x1800000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x42104070
[Pass] UNCACHEABLE(p) => 0x42104070
       shoot_free_suite(suite); suite = 0; chunk = 0;
[Pass] suite = shoot_malloc_suite_contig(0) => 0xa5388
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0x1
[Pass] suite->size => 0x1f80000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0xa53b0
[Pass] chunk->signature => 'MemChunk'
[Pass] chunk->size => 0x1f80000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4a000064
[Pass] UNCACHEABLE(p) => 0x4a000064
       shoot_free_suite(suite); suite = 0; chunk = 0;
[Pass] suite = shoot_malloc_suite(64*1024*1024) => 0xa5388
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0xc
[Pass] suite->size => 0x4000000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0xa53b0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x19f8000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x42104070
[Pass] UNCACHEABLE(p) => 0x42104070
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc430
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3978000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4a000064
[Pass] UNCACHEABLE(p) => 0x4a000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc468
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x39f4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x5bf800e4
[Pass] UNCACHEABLE(p) => 0x5bf800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc4a0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3a70000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x59f800e4
[Pass] UNCACHEABLE(p) => 0x59f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc4d8
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3aec000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x57f800e4
[Pass] UNCACHEABLE(p) => 0x57f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc510
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3b68000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x55f800e4
[Pass] UNCACHEABLE(p) => 0x55f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc548
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3be4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x53f800e4
[Pass] UNCACHEABLE(p) => 0x53f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc580
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3c60000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x51f800e4
[Pass] UNCACHEABLE(p) => 0x51f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc5b8
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3cdc000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4ff800e4
[Pass] UNCACHEABLE(p) => 0x4ff800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc5f0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3d58000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4df800e4
[Pass] UNCACHEABLE(p) => 0x4df800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc628
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3dd4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4bf800e4
[Pass] UNCACHEABLE(p) => 0x4bf800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc660
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x4000000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4c000064
[Pass] UNCACHEABLE(p) => 0x4c000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0x0
[Pass] total => 0x4000000
       shoot_free_suite(suite); suite = 0; chunk = 0;
[Pass] suite = shoot_malloc_suite(0) => 0xa5388
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0x13
[Pass] suite->size => 0x13700000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0xa53b0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x19f8000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x42104070
[Pass] UNCACHEABLE(p) => 0x42104070
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc430
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3978000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4a000064
[Pass] UNCACHEABLE(p) => 0x4a000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc468
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x39f4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x5bf800e4
[Pass] UNCACHEABLE(p) => 0x5bf800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc4a0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3a70000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x59f800e4
[Pass] UNCACHEABLE(p) => 0x59f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc4d8
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3aec000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x57f800e4
[Pass] UNCACHEABLE(p) => 0x57f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc510
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3b68000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x55f800e4
[Pass] UNCACHEABLE(p) => 0x55f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc548
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3be4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x53f800e4
[Pass] UNCACHEABLE(p) => 0x53f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc580
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3c60000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x51f800e4
[Pass] UNCACHEABLE(p) => 0x51f800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc5b8
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3cdc000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4ff800e4
[Pass] UNCACHEABLE(p) => 0x4ff800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc5f0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3d58000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4df800e4
[Pass] UNCACHEABLE(p) => 0x4df800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc628
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3dd4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4bf800e4
[Pass] UNCACHEABLE(p) => 0x4bf800e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc660
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x5d54000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4c000064
[Pass] UNCACHEABLE(p) => 0x4c000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc698
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x7cd4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4e000064
[Pass] UNCACHEABLE(p) => 0x4e000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc6d0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x9c54000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x50000064
[Pass] UNCACHEABLE(p) => 0x50000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc708
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0xbbd4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x52000064
[Pass] UNCACHEABLE(p) => 0x52000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc740
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0xdb54000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x54000064
[Pass] UNCACHEABLE(p) => 0x54000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc778
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0xfad4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x56000064
[Pass] UNCACHEABLE(p) => 0x56000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc7b0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x11a54000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x58000064
[Pass] UNCACHEABLE(p) => 0x58000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0xbc7e8
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x13700000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x5a000064
[Pass] UNCACHEABLE(p) => 0x5a000064
       chunk = GetNextMemoryChunk(suite, chunk) => 0x0
[Pass] total => 0x13700000
       shoot_free_suite(suite); suite = 0; chunk = 0;
[Pass] strlen("abc") => 0x3
[Pass] strlen("qwertyuiop") => 0xa
[Pass] strlen("") => 0x0
[Pass] strcpy(msg, "hi there") => 0x191c64
[Pass] msg => 'hi there'
[Pass] snprintf(a, sizeof(a), "foo") => 0x3
[Pass] snprintf(b, sizeof(b), "foo") => 0x3
[Pass] strcmp(a, b) => 0x0
[Pass] snprintf(a, sizeof(a), "bar") => 0x3
[Pass] snprintf(b, sizeof(b), "baz") => 0x3
[Pass] strcmp(a, b) => 0xfffffff8
[Pass] snprintf(a, sizeof(a), "Display") => 0x7
[Pass] snprintf(b, sizeof(b), "Defishing") => 0x9
[Pass] strcmp(a, b) => 0x4
[Pass] snprintf(buf, 3, "%d", 1234) => 0x2
[Pass] buf => '12'
[Pass] memcpy(foo, bar, 6) => 0x191c40
[Pass] foo => 'asdfghuiop'
[Pass] memset(bar, '*', 5) => 0x191c20
[Pass] bar => '*****hjkl;'
       bzero32(bar + 5, 5)
[Pass] bar => '****'
       EngDrvOut(LCD_Palette[0], 0x1234)
[Pass] shamem_read(LCD_Palette[0]) => 0x1234
       call("TurnOnDisplay")
[Pass] DISPLAY_IS_ON => 0x1
       call("TurnOffDisplay")
[Pass] DISPLAY_IS_ON => 0x0
       call("TurnOnDisplay")
[Pass] DISPLAY_IS_ON => 0x1
       task_create("test", 0x1c, 0x1000, test_task, 0) => 0x13e400bc
[Pass] test_task_created => 0x1
[Pass] get_current_task_name() => 'run_test'
[Pass] task_max => 0x68
[Pass] task_max => 0x68
[Pass] mq = mq ? mq : (void*)msg_queue_create("test", 5) => 0x13e60090
[Pass] msg_queue_post(mq, 0x1234567) => 0x0
[Pass] msg_queue_receive(mq, (struct event **) &m, 500) => 0x0
[Pass] m => 0x1234567
[Pass] msg_queue_receive(mq, (struct event **) &m, 500) => 0x9
[Pass] sem = sem ? sem : create_named_semaphore("test", 1) => 0x13e802da
[Pass] take_semaphore(sem, 500) => 0x0
[Pass] take_semaphore(sem, 500) => 0x9
[Pass] give_semaphore(sem) => 0x0
[Pass] take_semaphore(sem, 500) => 0x0
[Pass] give_semaphore(sem) => 0x0
[Pass] rlock = rlock ? rlock : CreateRecursiveLock(0) => 0x13ea0038
[Pass] AcquireRecursiveLock(rlock, 500) => 0x0
[Pass] AcquireRecursiveLock(rlock, 500) => 0x0
[Pass] ReleaseRecursiveLock(rlock) => 0x0
[Pass] ReleaseRecursiveLock(rlock) => 0x0
[Pass] ReleaseRecursiveLock(rlock) => 0xf
       SetGUIRequestMode(1); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x1
       SetGUIRequestMode(2); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x2
       SetGUIRequestMode(0); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x0
[Pass] display_idle() => 0x1
       GUI_Control(BGMT_PLAY, 0, 0, 0); msleep(1000);
[Pass] PLAY_MODE => 0x1
[Pass] MENU_MODE => 0x0
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(1000);
[Pass] MENU_MODE => 0x1
[Pass] PLAY_MODE => 0x0
[Pass] dialog->type => 'DIALOG'
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(500);
[Pass] MENU_MODE => 0x0
[Pass] PLAY_MODE => 0x0
       SW1(1,100)
[Pass] HALFSHUTTER_PRESSED => 0x1
       SW1(0,100)
[Pass] HALFSHUTTER_PRESSED => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x1
[Pass] is_pure_play_movie_mode() => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x1
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x0

Title: Re: Porting a Canon firmware update
Post by: dfort on November 02, 2017, 12:05:10 AM
And this is the most complete lua test I was able to pull off. Note that the other lua scripts seem to working fine, it is just that the 7D is having problems with the test using either the 2.0.3 or 2.0.6 firmware.

LUATEST.LOG

===============================================================================
ML/SCRIPTS/API_TEST.LUA - 2017-10-31 16:20:52
===============================================================================

Strict mode tests...
Strict mode tests passed.

Generic tests...
camera = table:
  shutter = table:
    raw = 112
    apex = 7.
    ms = 8
    value = 0.007812
  aperture = table:
    raw = 48
    apex = 5
    value = 5.599999
    min = table:
      raw = 22
      apex = 1.75
      value = 1.8
    max = table:
      raw = 80
      apex = 9
      value = 22.6
  iso = table:
    raw = 0
    apex = -4
    value = 0
  ec = table:
    raw = 0
    value = 0
  flash_ec = table:
    raw = 0
    value = 0
  kelvin = 5200
  mode = 3
  metering_mode = 3
  drive_mode = 0
  model = "Canon EOS 7D"
  model_short = "7D"
  firmware = "2.0.6"
  temperature = 147
  state = 0
  bulb = function: p
  shoot = function: p
  reboot = function: p
event = table:
  pre_shoot = nil
  post_shoot = nil
  shoot_task = nil
  seconds_clock = nil
  keypress = nil
  custom_picture_taking = nil
  intervalometer = nil
  config_save = nil
console = table:
  hide = function: p
  write = function: p
  show = function: p
  clear = function: p
lv = table:
  enabled = false
  paused = false
  running = false
  zoom = 1
  stop = function: p
  resume = function: p
  info = function: p
  pause = function: p
  wait = function: p
  start = function: p
lens = table:
  name = "EF50mm f/1.8 STM"
  focal_length = 0
  focus_distance = 14080
  hyperfocal = 0
  dof_near = 0
  dof_far = 0
  af = true
  af_mode = 0
  focus = function: p
display = table:
  idle = true
  height = 480
  width = 720
  off = function: p
  screenshot = function: p
  notify_box = function: p
  draw = function: p
  circle = function: p
  pixel = function: p
  rect = function: p
  load = function: p
  on = function: p
  line = function: p
  print = function: p
  clear = function: p
key = table:
  last = 0
  wait = function: p
  press = function: p
menu = table:
  visible = false
  new = function: p
  block = function: p
  open = function: p
  close = function: p
  set = function: p
  get = function: p
testmenu = userdata:
  value = 0
  name = "Script API tests"
  help = "Various tests for the Lua scripting API."
  help2 = "When adding new Lua APIs, tests for them should go here."
  advanced = 0
  depends_on = 0
  edit_mode = 0
  hidden = false
  icon_type = 5
  jhidden = false
  max = 0
  min = 0
  selected = true
  shidden = false
  starred = false
  submenu_height = 0
  submenu_width = 0
  unit = 0
  works_best_in = 0
  run_in_separate_task = 0
  select = function: p
  update = nil
  info = nil
  rinfo = nil
  warning = nil
movie = table:
  recording = false
  stop = function: p
  start = function: p
dryos = table:
  clock = 17
  ms_clock = 17016
  prefix = "IMG_"
  dcim_dir = table:
    exists = true
    create = function: p
    children = function: p
    files = function: p
    parent = table:
      exists = true
      create = function: p
      children = function: p
      files = function: p
      parent = table:
        exists = true
        create = function: p
        children = function: p
        files = function: p
        parent = nil
        path = "A:/"
      path = "A:/DCIM/"
    path = "A:/DCIM/100EOS7D/"
  config_dir = table:
    exists = true
    create = function: p
    children = function: p
    files = function: p
    parent = table:
      exists = true
      create = function: p
      children = function: p
      files = function: p
      parent = table:
        exists = true
        create = function: p
        children = function: p
        files = function: p
        parent = nil
        path = "A:/"
      path = "ML/"
    path = "ML/SETTINGS/"
  ml_card = table:
    cluster_size = 32768
    drive_letter = "A"
    file_number = 9972
    folder_number = 100
    free_space = 3865088
    type = "CF"
    path = "A:/"
    _card_ptr = userdata
  shooting_card = table:
    cluster_size = 32768
    drive_letter = "A"
    file_number = 9972
    folder_number = 100
    free_space = 3865088
    type = "CF"
    path = "A:/"
    _card_ptr = userdata
  date = table:
    month = 10
    hour = 16
    year = 2017
    day = 31
    min = 20
    yday = 304
    isdst = false
    sec = 53
    wday = 3
  directory = function: p
  call = function: p
  remove = function: p
interval = table:
  time = 10
  count = 0
  running = 0
  stop = function: p
battery = table:
  level = 69
  id = 0
  performance = 3
  time = 7452
  drain_rate = 33
task = table:
  create = function: p
  yield = function: p
property = table:
Generic tests completed.

Module tests...
Copy test: autoexec.bin -> tmp.bin
Copy test OK
Append test: tmp.txt
Append test OK
Testing exposure settings, module 'camera'...
Camera    : Canon EOS 7D (7D) 2.0.6
Lens      : EF50mm f/1.8 STM
Shoot mode: 3
Shutter   : Ç125 (raw 112, 0.007812s, 8ms, apex 7.)
Aperture  : Å5.6 (raw 48, f/5.599999, apex 5)
Av range  : Å1.8..Å22 (raw 22..80, f/1.8..f/22.6, apex 1.75..9)
ISO       : AutoISO (raw 0, 0, apex -4)
EC        : 0.0 (raw 0, 0 EV)
Flash EC  : 0.0 (raw 0, 0 EV)
Setting shutter to random values...
Setting ISO to random values...
Setting aperture to random values...
Please switch to Av mode.
Setting EC to random values...
Setting Flash EC to random values...
Exposure tests completed.

Testing module 'lv'...
LiveView is running; stopping...
Starting LiveView...
Setting zoom to x1...
Setting zoom to x5...
Setting zoom to x10...
Setting zoom to x5...
Setting zoom to x1...
Setting zoom to x10...
Setting zoom to x1...
Pausing LiveView...
Resuming LiveView...
Stopping LiveView...
LiveView tests completed.

Focus distance: 410
Focusing backward...
Focus distance: 620
Focusing forward with step size 3, wait=true...

Focus distance: 620
Focusing backward with step size 3, wait=true...

Focus distance: 620
Focus range: 0 steps forward, 0 steps backward.
Focusing forward with step size 3, wait=false...

Focus distance: 620
Focusing backward with step size 3, wait=false...

Focus distance: 620
Focus range: 0 steps forward, 0 steps backward.
Focusing forward with step size 2, wait=true...

Focus distance: 620
Focusing backward with step size 2, wait=true...

Focus distance: 620
Focus range: 0 steps forward, 0 steps backward.
Focusing forward with step size 2, wait=false...

Focus distance: 620
Focusing backward with step size 2, wait=false...

Focus distance: 620
Focus range: 0 steps forward, 0 steps backward.
Focusing forward with step size 1, wait=true...

Focus distance: 620
Focusing backward with step size 1, wait=true...

Focus distance: 620
Focus range: 0 steps forward, 0 steps backward.
Focusing forward with step size 1, wait=false...

Focus distance: 620
Focusing backward with step size 1, wait=false...

Focus distance: 620
Focus range: 0 steps forward, 0 steps backward.
Focus test completed.

Done!
Title: Re: Porting a Canon firmware update
Post by: dfort on November 03, 2017, 04:00:16 AM
So how about doing a firmware update without the camera?

qqluqq started doing a 550D.109 to 110 update but ran into some problems. Let's see what's possible. First of all since he published his work in progress (https://bitbucket.org/qqluqq/magic-lantern/branch/550D%20110#diff) it looked like it was only a matter of checking what wasn't working. He tried to keep both 550D.109 and 550D.110 working at the same time but that's a bit advanced and not really needed on this camera. A quick check with check-stubs.py turned up a few problems that were very easily resolved. Finally, a few constants changed -- remember, the only thing constant is change.

Here's the output of the fixed stubs in check-stubs.py:

python check-stubs.py /Users/rosiefort/magic-lantern-hudson/platform/550D.109/stubs.S /Users/rosiefort/magic-lantern/platform/550D.110/stubs.S

STUB                                                    OLD           NEW     DELTA
current_interrupt                                    0x00000674 -> 0x00000674 [0x000]
task_dispatch_hook                                   0x00001938 -> 0x00001938 [0x000]
current_task                                         0x00001a20 -> 0x00001a20 [0x000]
gui_main_struct                                      0x00001c04 -> 0x00001c04 [0x000]
sounddev                                             0x00001ed0 -> 0x00001ed0 [0x000]
dm_names                                             0x00002b74 -> 0x00002b74 [0x000]
task_max                                             0x00003080 -> 0x00003080 [0x000]
gui_timer_struct                                     0x00003ac8 -> 0x00003ac8 [0x000]
mvr_config                                           0x000067bc -> 0x000067bc [0x000]
additional_version                                   0x00015094 -> 0x00015094 [0x000]
gui_task_list                                        0x0001e638 -> 0x0001e638 [0x000]
cf_device                                            0x00026258 -> 0x00026258 [0x000]
sd_device                                            0x00026284 -> 0x00026284 [0x000]
LCD_Palette                                          0x0002cdb0 -> 0x0002cdb0 [0x000]
bmp_vram_info                                        0x0002e5b0 -> 0x0002e5b0 [0x000]
camera_engine                                        0xc0220000 -> 0xc0220000 [0x000]
cstart                                               0xff010ff4 -> 0xff010ff4 [0x000] [!!!]
create_init_task                                     0xff017518 -> 0xff017510 [0x008]
_malloc                                              0xff018270 -> 0xff018268 [0x008]
_free                                                0xff0182cc -> 0xff0182c4 [0x008]
init_task                                            0xff018d1c -> 0xff018d14 [0x008]
gui_main_task                                        0xff01ff94 -> 0xff01ff8c [0x008]
GUI_Control                                          0xff020350 -> 0xff020348 [0x008]
GUI_ChangeMode                                       0xff020450 -> 0xff020448 [0x008]
gui_init_end                                         0xff020acc -> 0xff020ac4 [0x008]
SRM_AllocateMemoryResourceFor1stJob                  0xff028590 -> 0xff028588 [0x008]
SRM_FreeMemoryResourceFor1stJob                      0xff02ad5c -> 0xff02ad54 [0x008]
StartASIFDMAADC                                      0xff051950 -> 0xff051948 [0x008]
StartASIFDMADAC                                      0xff051afc -> 0xff051af4 [0x008]
StopASIFDMADAC                                       0xff051bd0 -> 0xff051bc8 [0x008]
SetNextASIFADCBuffer                                 0xff052088 -> 0xff052080 [0x008]
SetNextASIFDACBuffer                                 0xff0521cc -> 0xff0521c4 [0x008]
sounddev_task                                        0xff05346c -> 0xff053464 [0x008]
sounddev_active_in                                   0xff0537d4 -> 0xff0537cc [0x008]
_prop_request_change                                 0xff056e38 -> 0xff056e30 [0x008]
prop_deliver                                         0xff057418 -> 0xff057410 [0x008]
prop_register_slave                                  0xff0574a0 -> 0xff057498 [0x008]
_prop_cleanup                                        0xff0575c0 -> 0xff0575b8 [0x008]
LoadCalendarFromRTC                                  0xff06366c -> 0xff063664 [0x008]
_audio_ic_read                                       0xff063aa8 -> 0xff063aa0 [0x008]
_audio_ic_write                                      0xff063bf4 -> 0xff063bec [0x008]
SetAudioVolumeOut                                    0xff063ea8 -> 0xff063ea0 [0x008]
PowerAudioOutput                                     0xff064114 -> 0xff06410c [0x008]
SetSamplingRate                                      0xff06419c -> 0xff064194 [0x008]
DryosDebugMsg                                        0xff0673ec -> 0xff0673e4 [0x008]
dm_set_store_level                                   0xff067748 -> 0xff067740 [0x008]
call                                                 0xff068178 -> 0xff068170 [0x008]
SetHPTimerAfterNow                                   0xff068bb8 -> 0xff068bb0 [0x008]
SetHPTimerNextTick                                   0xff068c38 -> 0xff068c30 [0x008]
create_named_semaphore                               0xff069d10 -> 0xff069d08 [0x008]
take_semaphore                                       0xff069e70 -> 0xff069e68 [0x008]
give_semaphore                                       0xff069f58 -> 0xff069f50 [0x008]
msleep                                               0xff06a344 -> 0xff06a33c [0x008]
task_create                                          0xff06a3f8 -> 0xff06a3f0 [0x008]
GetSizeOfMaxRegion                                   0xff06afc0 -> 0xff06afb8 [0x008]
GetMemoryInformation                                 0xff06b014 -> 0xff06b00c [0x008]
_AllocateMemory                                      0xff06b270 -> 0xff06b268 [0x008]
_FreeMemory                                          0xff06b600 -> 0xff06b5f8 [0x008]
CreateMemoryChunk                                    0xff06bec8 -> 0xff06bec0 [0x008]
GetMemoryAddressOfMemoryChunk                        0xff06c434 -> 0xff06c42c [0x008]
DeleteMemorySuite                                    0xff06c4a8 -> 0xff06c4a0 [0x008]
CreateMemorySuite                                    0xff06c678 -> 0xff06c670 [0x008]
AddMemoryChunk                                       0xff06c918 -> 0xff06c910 [0x008]
GetFirstChunkFromSuite                               0xff06c9ec -> 0xff06c9e4 [0x008]
GetNextMemoryChunk                                   0xff06d184 -> 0xff06d17c [0x008]
CreateRecursiveLock                                  0xff06e358 -> 0xff06e350 [0x008]
AllocateMemoryResource                               0xff06f3d8 -> 0xff06f3d0 [0x008]
AllocateContinuousMemoryResource                     0xff06f424 -> 0xff06f41c [0x008]
FreeMemoryResource                                   0xff06f53c -> 0xff06f534 [0x008]
SetTimerAfter                                        0xff070a60 -> 0xff070a58 [0x008]
CancelTimer                                          0xff070c60 -> 0xff070c58 [0x008]
task_trampoline                                      0xff0771bc -> 0xff0771b4 [0x008]
DispSensorStart                                      0xff07c6a0 -> 0xff07c698 [0x008]
SetGUIRequestMode                                    0xff092558 -> 0xff092550 [0x008]
gui_massive_event_loop                               0xff093108 -> 0xff093100 [0x008]
gui_change_mode                                      0xff094cf4 -> 0xff094cec [0x008]
gui_local_post                                       0xff0952f0 -> 0xff0952e8 [0x008]
gui_other_post                                       0xff095728 -> 0xff095720 [0x008]
mvrSetDefQScale                                      0xff1aa4a0 -> 0xff1aa498 [0x008]
mvrSetFullHDOptSize                                  0xff1aa4d8 -> 0xff1aa4d0 [0x008]
mvrSetGopOptSizeFULLHD                               0xff1aa6c8 -> 0xff1aa6c0 [0x008]
mvrFixQScale                                         0xff1aa9c4 -> 0xff1aa9bc [0x008]
SetEDmac                                             0xff1bff44 -> 0xff1bff3c [0x008]
ConnectWriteEDmac                                    0xff1bfffc -> 0xff1bfff4 [0x008]
ConnectReadEDmac                                     0xff1c00c0 -> 0xff1c00b8 [0x008]
StartEDmac                                           0xff1c024c -> 0xff1c0244 [0x008]
AbortEDmac                                           0xff1c0378 -> 0xff1c0370 [0x008]
RegisterEDmacCompleteCBR                             0xff1c0418 -> 0xff1c0410 [0x008]
UnregisterEDmacCompleteCBR                           0xff1c042c -> 0xff1c0424 [0x008]
RegisterEDmacAbortCBR                                0xff1c0440 -> 0xff1c0438 [0x008]
UnregisterEDmacAbortCBR                              0xff1c0454 -> 0xff1c044c [0x008]
RegisterEDmacPopCBR                                  0xff1c0468 -> 0xff1c0460 [0x008]
UnregisterEDmacPopCBR                                0xff1c047c -> 0xff1c0474 [0x008]
shamem_read                                          0xff1c1150 -> 0xff1c1148 [0x008]
_EngDrvOut                                           0xff1c1260 -> 0xff1c1258 [0x008]
_engio_write                                         0xff1c15cc -> 0xff1c15c4 [0x008]
CreateResLockEntry                                   0xff1c4074 -> 0xff1c406c [0x008]
UnLockEngineResources                                0xff1c46f8 -> 0xff1c46f0 [0x008]
LockEngineResources                                  0xff1c4a34 -> 0xff1c4a2c [0x008]
_FIO_OpenFile                                        0xff1c658c -> 0xff1c6584 [0x008]
_FIO_CreateFile                                      0xff1c6648 -> 0xff1c6640 [0x008]
_FIO_RemoveFile                                      0xff1c66f4 -> 0xff1c66ec [0x008]
_FIO_ReadFile                                        0xff1c679c -> 0xff1c6794 [0x008]
FIO_SeekSkipFile                                     0xff1c684c -> 0xff1c6844 [0x008]
_FIO_WriteFile                                       0xff1c693c -> 0xff1c6934 [0x008]
FIO_CloseFile                                        0xff1c69ec -> 0xff1c69e4 [0x008]
_FIO_GetFileSize                                     0xff1c6b68 -> 0xff1c6b60 [0x008]
_FIO_RenameFile                                      0xff1c7060 -> 0xff1c7058 [0x008]
_FIO_CreateDirectory                                 0xff1c71a0 -> 0xff1c7198 [0x008]
_FIO_FindFirstEx                                     0xff1c7760 -> 0xff1c7758 [0x008]
FIO_FindNextEx                                       0xff1c7854 -> 0xff1c784c [0x008]
FIO_FindClose                                        0xff1c7934 -> 0xff1c792c [0x008]
msg_queue_create                                     0xff1d2a60 -> 0xff1d2a58 [0x008]
msg_queue_receive                                    0xff1d2b70 -> 0xff1d2b68 [0x008]
msg_queue_post                                       0xff1d2d5c -> 0xff1d2d54 [0x008]
msg_queue_count                                      0xff1d2d9c -> 0xff1d2d94 [0x008]
AcquireRecursiveLock                                 0xff1d2ea8 -> 0xff1d2ea0 [0x008]
ReleaseRecursiveLock                                 0xff1d2fbc -> 0xff1d2fb4 [0x008]
vsnprintf                                            0xff1d6638 -> 0xff1d6630 [0x008]
_alloc_dma_memory                                    0xff1da26c -> 0xff1da264 [0x008]
_free_dma_memory                                     0xff1da2a0 -> 0xff1da298 [0x008]
GUI_SetRollingPitchingLevelStatus                    0xff2045e0 -> 0xff2045d8 [0x008]
ptp_register_handler                                 0xff250bf0 -> 0xff250be8 [0x008]
dialog_redraw                                        0xff2b67f4 -> 0xff2b67ec [0x008]
MirrorDisplay                                        0xff2cfa54 -> 0xff2cfa4c [0x008]
ReverseDisplay                                       0xff2cfa84 -> 0xff2cfa7c [0x008]
NormalDisplay                                        0xff2cfab4 -> 0xff2cfaac [0x008]
is_taskid_valid                                      0xff2e9ab0 -> 0xff2e9aa8 [0x008]
ErrForCamera_handler                                 0xff2f0cc8 -> 0xff2f0cc0 [0x008]
HideUnaviFeedBack_maybe                              0xff2fdef8 -> 0xff2fdef0 [0x008]
LiveViewApp_handler                                  0xff300ba0 -> 0xff300b98 [0x008]
LiveViewApp_handler_BL_JudgeBottomInfoDispTimerState 0xff3010f4 -> 0xff3010ec [0x008]
LiveViewApp_handler_end                              0xff30204c -> 0xff302044 [0x008]
CancelDateTimer                                      0xff3050a4 -> 0xff30509c [0x008]
PlayMain_handler                                     0xff30c590 -> 0xff30c588 [0x008]
ShootOlcApp_handler                                  0xff31971c -> 0xff319714 [0x008]
dialog_set_property_str                              0xff31cdf8 -> 0xff31cdf0 [0x008]
fsuDecodePartitionTable                              0xff386714 -> 0xff386714 [0x000] [!!!]
PlayMovieGuideApp_handler                            0xff3c0638 -> 0xff3c0638 [0x000] [!!!]
ErrCardForLVApp_handler                              0xff3c5150 -> 0xff3c5150 [0x000] [!!!]
GetCFnData                                           0xff42e15c -> 0xff42e15c [0x000] [!!!]
SetCFnData                                           0xff42e364 -> 0xff42e364 [0x000] [!!!]
bzero32                                              0xff44f8f4 -> 0xff44f8f4 [0x000] [!!!]
lv_path_struct                                       0xff4b39ac -> 0xff4b39ac [0x000] [!!!]
audio_thresholds                                     0xff4c466c -> 0xff4c466c [0x000] [!!!]


Ok, here we go. Will the firmware dump launch in QEMU?

(https://farm5.staticflickr.com/4509/38120650771_d9ed475c92.jpg) (https://flic.kr/p/215Ainv)

Yep, how about running a minimal autoexec.bin?

(https://farm5.staticflickr.com/4472/26351465169_cba4b4ec9d.jpg) (https://flic.kr/p/G9A9wn)

Let's get the firmware signature.

(https://farm5.staticflickr.com/4453/38073625106_646112fa52.jpg) (https://flic.kr/p/211rhhb)

And finally:

(https://farm5.staticflickr.com/4472/38073625286_388ab7044c.jpg) (https://flic.kr/p/211rhkh)

(https://farm5.staticflickr.com/4547/24275940178_102bc5bcbd.jpg) (https://flic.kr/p/CZbxfJ)
Title: Re: Porting a Canon firmware update
Post by: ItsMeLenny on November 05, 2017, 03:23:28 AM
Is it necessary to keep the 109 stuff in the code if a 110 upgrade is performed?
Title: Re: Porting a Canon firmware update
Post by: dfort on November 05, 2017, 04:36:35 AM
No, just make a clean break from 109 to 110. It is easy to go back and forth between the two if you feel like testing but don't want to commit until it is accepted into the main repository. I posted both of the firmware updates on my Bitbucket downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/) so testers don't have to search for them.

The only case that we've got more than one firmware version on a camera platform is with the 5D3 because 1.1.3 has fewer issues but 1.2.3 has some new features like full resolution HDMI output. The 1.3.4 update adds some improved AF performance in LiveView with wide angle lenses and a fix for lens aberration correction on the EF 70-300mm lens but it also makes it more difficult to downgrade the firmware.
Title: Re: Porting a Canon firmware update
Post by: orielsy on December 02, 2017, 05:34:34 AM
Hello all,

I last touched C++ 6 years ago after college (been doing Frontend development ever since). I was wondering what the community here thinks are my chances at porting ML to the EOS M2?

I would love some guidance on this as I would love to avoid wasting my holidays (in case it's impossible to port or something).

Cheers!
Title: Re: Porting a Canon firmware update
Post by: dfort on December 02, 2017, 05:53:02 PM
Here is a good case for updating all of the ML ports to the latest Canon firmware.

(https://farm5.staticflickr.com/4533/37901786105_4da04062ce_z.jpg) (https://flic.kr/p/ZKfywM)

That's what Pelican's website (http://pel.hu/) looks like when I went to look for some previous Canon firmware updates. Yeah, it is offline and it has been offline for at least a week.

As many of you know his archived collection of Canon firmware updaters is the go to place for anyone wanting to run ML on their camera but had the "wrong" firmware on their camera.
Title: Re: Porting a Canon firmware update
Post by: dfort on December 08, 2017, 02:50:31 AM
Ok--just discovered that the Developers have thought of everything, including the possibility of Pelican's site going down. If you go to the downloads page and click on the Useful Links button you'll get a link for the Canon firmware updater:

(https://farm5.staticflickr.com/4521/25032573038_c092f192ce.jpg) (https://flic.kr/p/E93tQS)

This links to web.archive.org instead of directly to pel.hu so even though the website is down you can still get the old Canon firmware updater.
Title: Re: Porting a Canon firmware update
Post by: dfort on December 08, 2017, 05:04:50 PM
Quote from: orielsy on December 02, 2017, 05:34:34 AM
I was wondering what the community here thinks are my chances at porting ML to the EOS M2?

@orielsy - sent you a PM when you posted this. Have you read through the ML on EOS-M2 topic (http://www.magiclantern.fm/forum/index.php?topic=15895.0) yet? That port should be very close but we got stuck trying to get the ML menu to come up in QEMU.
Title: Re: Porting a Canon firmware update
Post by: dfort on April 29, 2018, 06:33:33 PM
Just thought I'd bump this topic because it looks like all of the cameras that are currently behind Canon's latest firmware are now in progress. I've been updating Reply #4 (https://www.magiclantern.fm/forum/index.php?topic=19417.msg184578#msg184578) with the status of each of these cameras.
Title: Re: Porting a Canon firmware update
Post by: dgatwood on May 19, 2018, 11:29:53 PM
Beyond figuring out where all the function entry points are in the base firmware, what else is typically required when bringing up new hardware?  (I currently have a 5D Mark IV and a 6D Mark II, hence my interest.)  Does it require anything invasive like soldering JTAG headers to the main board, or is the bootloader sufficiently standardized to make it possible to do a software-only board bringup?
Title: Re: Porting a Canon firmware update
Post by: Walter Schulz on May 20, 2018, 09:21:42 AM
Offtopic. You are asking for ports for new hardware. This thread is about porting an existing ML version to higher firmware version.

See https://builds.magiclantern.fm/ -> "Your camera is not listed?" and linked threads for Digic VI and VII.
Esp. https://www.magiclantern.fm/forum/index.php?topic=19737.msg200799#msg200799
Below -> ROM dumpers for 5D4 and 6D2 available: https://builds.magiclantern.fm/ -> ROM dumpers
Title: Re: Porting a Canon firmware update
Post by: dfort on May 20, 2018, 05:17:46 PM
I'll add another link:

ML on EOS-M2
https://www.magiclantern.fm/forum/index.php?topic=15895.msg185084#msg185084

Note that even a camera that should be very similar to the cameras that have already been ported to ML can present quite a few challenges. Cameras like your 5D4 and 6D2 use Digic 6, 6+ and 7 processors. So far ML is working on cameras with Digic 4 and 5 processors.

Porting ML shouldn't require any physical modifications. In fact much of it can be done via QEMU once you're got a firmware dump.

Of course invasive hardware hacking is another way to figure out what makes digital cameras tick -- or rather click?

https://nada-labs.net/2014/finding-jtag-on-a-canon-elph100hs-ixus115/
Title: Re: Porting a Canon firmware update
Post by: giuliom92 on May 04, 2020, 11:25:36 AM
Hello dfort,
I'm having issues porting the firmware, for the EOS 2000d, everything's fine until I try to disassemble with the script, the script returns
Offset + filesize - 1 > 0xffffffff. We can't wrap around!

And I tried to adjust the path but it didn't work, what can I do?
Title: Re: Porting a Canon firmware update
Post by: dfort on May 05, 2020, 07:49:10 AM
The 2000D has a Digic 4+ processor. Not sure what offset is needed when disassembling.


Sent from my iPhone using Tapatalk