DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)

[Monet.C]

Got my SL2 on the way, would be happy to help out with testing. Read most of the treads, I have some first year programming knowledge (C, C++, JS) but it doesn't go further than that. Would be happy to help where it is possible

[LeptoN]

Another I want to help message.

I'm a software developer still in my senior year of college. I have experience with C, C++, 32 bit ARM assembly, and some reverse engineering (game hacking). I had an internship working with embedded systems helping developing firmware for a vehicle infotainment system.

I'd like to help speed up development for the 6D II but I'm not sure what is done, what needs work, and where I should start. I would like some advice.

[names_are_hard]

Hi!  No testing required yet, it's not got as far as menus.  Or working buttons.

[LeptoN]

How can I help?

[names_are_hard]

@LeptoN - was replying to Monet with the previous post, yours wasn't up at the time I started writing!  Your skillset suggests you could help a lot, if you're patient!

6D2 is Digic 7, like 200D.  That should mean you can run arbitrary code, including flashing LEDs and drawing to the screen.  However, it doesn't run full ML yet.  I'm (slowly) working on 200D, where I can build full ML, load it, and it crashes after some early init code.  I think some problem to do with tasks, but so far I've been unable to diagnose it.  For some crashes it will log stack traces to disk, but not this one.

The outline of what to do is this:
- make your cam bootable using special firmware update earlier in this thread
- get ML building (to test you have correct toolchain etc)
- dump firmware from your cam using digic6-dumper
- find required stubs and consts to get full ML building
- build, it will crash, diagnose why, fix, repeat (this is where I am on 200D)

To get started, I would recommend this branch, digic6-dumper, it's easier to build than ML:
https://bitbucket.org/hudson/magic-lantern/branch/digic6-dumper

You'll need to get that building - Linux is probably easiest but I know people can build on Mac and WSL as well.  You need an arm toolchain, obviously.  I build by cd'ing to platform/200D then "make zip".  This will fail if you have python3 first in your path.  If you think it's time for python2 to die, I can point you at the changes required for python3 to work.  Otherwise, just make sure "python" means "python2" on your system.

You'll need to get booting enabled on your cam.  This is a small, persistent change to firmware (a bit flip in flash, I think) that instructs the cam to try and do a firmware upgrade if a .FIR file run code from an autoexec.bin file if found on the card.  This is a one-off task, find the magic firmware for 6D2, somewhere earlier in this thread I expect.  ML hijacks the firmware update process to inject into the running OS - but note that no persistent changes are made; there's no firmware update, it's just how code gets injected into ram.

Build digic6-dumper for your cam.  See if it works.  If so, congratulations!  You can run arbitrary code on your cam!  Now it gets harder.  Digic6-dumper is very bare bones.  I think Alex found a minimum set of stubs and consts (more on those later) to get it working on most Digic 6/7/8 cams, but there's not enough to succeed loading full ML.

In order to hook the right functions, ML needs to know the fixed addresses where they are in ram.  ML calls these "stubs", they're in platform/XXX/stubs.S.  There are some scripts that try to guess these from a similar firmware, but I don't know how well these work since we don't have a working >= Digic 6 cam to base it on.  In any case, the scripts aren't perfect, so you'll want something like Ghidra or IDA Pro for finding the stubs for your cam from a firmware dump.  Digic6-dumper should have dropped that on the card for you.  You also want to find some consts, to put in consts.h.  For both these tasks it helps enormously if you a firmware dump from another model where the stubs are known.  The code is surprisingly similar between models.

After you have what you think is a good set of stubs and consts you can try and build full ML.  If you can't find some, you can ignore them, probably, and see what works and what is broken.  For some stuff it won't compile and you may have to fake stuff out, for other stuff you can use obviously bad values if it compiles, but you'll have to go back and fix it later.  You will also want to set some #defines so that ML doesn't try and dangerous code - this is important!  I can't remember the define right now but it stops ML attempting dangerous writes (they're fine if we understand a given model, but here we don't, so...).

Here's the line in internals.h that prevents some ways of breaking your cam.  You want it commented out as shown:

Code Select Expand


/** Properties are persistent (saved in NVRAM) => a mistake can cause permanent damage. Undefine this for new ports. */
//#define CONFIG_PROP_REQUEST_CHANGE


My repo here is probably the furthest along, but it's far from working:
https://bitbucket.org/stephen-e/ml_200d/src/master/

I've tried to #ifdef stuff in a way that it might work on all Digic 6/7/8 cams, but I only have a 200D to test on.  You can probably get some use out of reading my stubs.S as I comment my reasoning on how I found them.  Firmware between 6D2 and 200D may be similar.

Good luck, and do please let me know if you'd like help with any of those stages.  I think I'm the only person working on a Digic 6/7/8 port at the moment.  Ilia is considering starting on the 5Ds.  It would be nice to have people to share problems with!

[ArcziPL]

Very well written introduction. I'd only like to correct this detail:

You'll need to get booting enabled on your cam.  This is a small, persistent change to firmware (a bit flip in flash, I think) that instructs the cam to try and do a firmware upgrade if a .FIR file is found on the card.  This is a one-off task, find the magic firmware for 6D2, somewhere earlier in this thread I expect.  ML hijacks the firmware update process to inject into the running OS - but note that no persistent changes are made; there's no firmware update, it's just how code gets injected into ram.

It's slightly different. ML hijacks the firmware update process only in two cases:
1) ML "installator" to enable the boot flag
2) portable ROM dumper runs fully in this mode

Activated boot flag instructs the cam to look for autoexec.bin on a memory card at every boot and, if found, execute it.

[Walter Schulz]

And here is the boot process in all its glory:
https://www.magiclantern.fm/forum/index.php?topic=23641.msg213338#msg213338

[turtius]

IRC is a hassle. Can the smaller discussion be on discord or some sort?

[names_are_hard]

IRC is indeed a hassle.  The lack of chat history is very annoying.  No reason Discord couldn't be used, if you can convince people to use it.

What kind of discussion were you hoping would happen?

[turtius]

Minor thing about the ROM's and the layout of the canon's firmware and such.. planned stuff and more..

if you can convince people to use it

Ill just say use the web version... simple as pie
In-fact when the chat 'session' gets old one can just search the whole conversation without the need to ask again here....

[names_are_hard]

I admire your optimism and hope it is as simple as you think

[turtius]

its about time...
dm me on discord..
https://discord.gg/yBurP7a (discord server invite link.)
The server is pretty much built up and awaiting for users. Hopefully it shouldn't be that difficult

[names_are_hard]

Massive merge to a more modern code base didn't work.  It builds, but crashes with no feedback, very early, cam unresponsive.  I'm not very surprised, it was too much to merge in one go.  Still, I learnt a bunch about how the code is organised in doing it.

I think my next steps are to try porting the task code specifically (ideally, as a proper merge, if I can find where this more modern code came from) and to see if it's practical to port the tasks code into digic6-dumper, so I'm experimenting with tasks in a less complex environment.  I'm fairly sure now that tasks are more complex on Digic 7 than single-core cams.  My guess is that each CPU has a separate task queue, held in the per CPU region (0x0000-0xfff for one core, 0x1000-0x1fff the other), with coordination happening in the 0x4000 region.  I also need to dig into the existing code to see if it makes assumptions that would break under that system.  Oh and @jack001214 found task_dispatch_hook for 200D, so he's officially a stub-hunter now!

Separately I may try to merge together the known-important parts in Unified, but outside of Mercurial, and adapt for >= Digic 6.  There's been no official word on what's happening with the Mercurial repo, nobody uses Mercurial anymore, so I may as well merge stuff myself in Git.

[pgathana]

Hello, I have the 200D. I have no idea about code. Is there any way to help?

[Walter Schulz]

Yes, learn to code.

[pgathana]

Great idea..! I haven't thought about it...

[Walter Schulz]

Canon 200D was introduced about 3 years ago. If you - or anyone else - would have started learning to code back then (instead of waiting for something dev team was adamant about not being able to deliver because of lack of time and dev team constantly asking for additional support by anyone willing to start into coding) I think following calculation might be valid: Give 20 minutes a day to coding -> 3 x 365 x 0.33h = 365 hours. Assuming 39h per week for full time employees -> More than 9 weeks. This compares well to the time a student in STEM is given to work on a project from scratch to presentation.
Start now and give us a presentation in 2023, please.

[Isaac Martínez]

Hi, I have a t7i, and I want to help. Anyone can guide me or give me some tutorials for start?
Thanks, and sorry for my english...

[Walter Schulz]

Describe your skills and experiences. Good starting point for coders is making QEMU run. See autoexec_bin Twitter account -> Sticky tweet.
If you don't have any codings skills you might want to learn C programming language first. There are a few web-based online courses.

[Isaac Martínez]

I know how to program a little... But, what do I need to do to get started? Is anyone trying to get the magic lantern on the 800D? How can I contact them?
I have an arduino board and a Canon 800D. What can I do for progress?
Sorry for my engilsh.

[turtius]

Well perhaps try loading the ROM dumps into ghidra or ida and compare them with a port which has a port running. Currently i attempted to compare the 50D with 200D (as instructed by @names_are_hard) and found that the dumps are very similar.

[Isaac Martínez]

Okay, so first I need to install Ghidra, and second, where can I find the different ports and ROM dumps? The 800D's ROM dumper listed here is correct? https://www.magiclantern.fm/forum/index.php?topic=16534.0

[Isaac Martínez]

Hi?

[Walter Schulz]

Have you succeeded in loading ROM dumps into Ghidra?

[mfurseman]

Hi, This is my first post on the forum so please keep me inline with the etiquette!

I own a 6DII and find the limitations imposed by Canons software interface quite frustrating, in particular the lack of techniques for manual focus. I am terrified about bricking my new shiny Camera but would like to assist in the development of ML for this platform. I work as a professional software and controls engineer in scientific research, I've got a good knowledge of Python, C, C++ and software engineering practice in general, but only a paltry knowledge of disassembly and nothing but a desire to learn ARM assembly.

I wonder if anyone knows what the cheapest DIGIC 7 camera is so I can try and source a used / broken one for testing without risk. The best description I've found for getting started on porting is in this thread, if there are any better resources could someone point me to them?