DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)

[names_are_hard]

debug printfs - shameful, but we all use them (entirely excusable here, where you don't have a debugger!).  I've added a convenience printf that works like this:
bmp_printf_auto("some string with data: %d", 6")
It has static x and y which it increments for you.  Nice for lazily printing things in order.

The fixed assert handler has made it much more stable.  Lockups are rare.

This is probably too hopeful, but I wonder if the main thing stopping me getting into ML menus is I haven't defined the button to trigger it?  I think this is supposed to be the delete / bin / trash button, but I don't know where to find it in consts / stubs / code.  What's the name used?  Or, more generallly, what's the path that leads to ML menus, so I can debug it?

[MrDerrick]

Hi, i have an 800d, I can help you in any way?

[names_are_hard]

Digic7 are in very early stages.  Do you have experience with C or disassembly, or have the time to learn?

[nikfreak]

This is probably too hopeful, but I wonder if the main thing stopping me getting into ML menus is I haven't defined the button to trigger it?  I think this is supposed to be the delete / bin / trash button, but I don't know where to find it in consts / stubs / code.  What's the name used?  Or, more generallly, what's the path that leads to ML menus, so I can debug it?

Check platform dir and gui.h in it.
Might be a good idea to copy and use that from my 100d port. You can also grep source code for 100D

[names_are_hard]

Ah, I'd forgotten about gui.h, thank you.  I took a look at it, and I can see it's associated with button codes in gui_massive_event_loop, but comparing between 50D and 200D is difficult.  I think Ghidra has decompiled them in different ways, one part that I'd expect to be a switch is a messy if/else nest.  I think I will have to come back to this part.

Other problems I have: this crash log:

ASSERT: 0
at TouchUtility.c:132, task DispDCtrl
lv:0 mode:0

DispDCtrl stack: 28d100 [28d218-28c218]
0xUNKNOWN  @ 1b67db:28d100

Which I think is related to these debug messages:

19.697151   DispVCtrl:e054852d:42:02: GiveSemaphore : StateChangeWaitCBR 56
19.697188   DispDCtrl:e04e2e51:45:03: TCH_ProhibitTouch
19.697216   DispDCtrl:e055a3bb:45:03: JDI_LAM_ProhibitTouch
19.697245   DispDCtrl:e059bec3:45:06: TryPost Cannot be Done!!! 3   <--- I think this is strongly related to the assert

I can locate the function that triggers those messages but it's not obvious how I fix the crash.  I also don't know if I need to - the camera remains responsive.  Perhaps this is not a high priority?  Anyone know a general pattern for dealing with these kinds of logs?  To debug this further I think I would write a dynamic patching util to dump register contents and stacktrace when a given address was hit, which would be fun but likely slow to get working reliably (if it was x86 asm it would be easy ).

I also find that any attempt to vsnprintf() with %s in the format string has a high chance of causing a very fast crash, fast enough that it stops the string being displayed.  But, it's not a guarantee, just very likely.  Sometimes you get part of the string displayed before locking up.  I don't understand this fully and can't explain it.  It evens occurs with local static strings as the arg.  There is nothing in crash logs when this happens.  I'm mostly mentioning this in case it confuses other Digic7 porters, but if this makes sense to anybody, please let me know.

[nikfreak]

I would just try to disable touch in canon menu until you find the root cause (stub / const). It's at least possible on 100d. Do you have a pull request available? It might help others to chime in

[kitor]

Do you have a pull request available? It might help others to chime in

https://bitbucket.org/stephen-e/ml_200d/
I don't think that this will be ready for PR anytime soon 

[names_are_hard]

Disabling touch in the menus was a good idea I hadn't thought of, thanks.  It didn't help, still get the same crash.  I note that the debug messages look related to disabling touch, and the camera will crash if I halfshutter - perhaps it always tries to disable the touchscreen when the display is off?  It does give me the crash log if I insert card, turn on cam, do nothing, turn off cam - but that leads to a display off too.  It's a pattern worth investigating.

I'm happy to look at PRs, but they'd be Git ones, not HG.  Unless Kitor was simply saying my code is too painful to work with   I'll also throw stuff over the fence into HG land if people want me to (but I don't want to use it all the time).

[kitor]

I thought about PR into unified which this is based on
Obviously taking this as a new branch in ML repo is different story - and may encourage others to give it a try.

I really need to find some time and dig for stubs on R to try this. I hope for next month - depends mostly on how long will take me to migrate all stuff from server running in my closet to full-blown rack setup in my parents house ("just IT things") 
You know that you work on too many items at once when you have multiple kanban boards just for your own projects...
/ot

[mylandscapeshots]

Long time listener, first time caller.

Here's yet another "I have a 6D2 but have very little programming experience, what low-risk tasks can I do?" type of post. Is there anything I CAN do or just sit back until further instructions?

[names_are_hard]

A quick update: I am not dead, was just too busy with other stuff (including Blackhat / DEFCON, which was pretty cool).

I am feeling kind of stuck though, I don't know how to proceed with the assert from a few messages back.  If anyone could help diagnose or suggest changes / tests, please do.  Best guess I've come up with is blanking the back screen triggers Canon to disable the touchscreen, and disabling the touchscreen triggers a crash (but why?).  I haven't thought of an easy test for this theory.

For now I am doing a second pass over all the stubs I've found, and sorting them into high / middling / low confidence findings.  If anyone would like to help there, you could do that without running code.  In particular, looking at low confidence stubs and either finding better choices, or explaining why I'm right on the one I've found, so confidence is improved:
https://bitbucket.org/stephen-e/ml_200d/src/dev/platform/200D.101/stubs.S

[Jamh]

Hey guys, What's up?
I have a Rebel T7I/800d and wanna help

  Magic Lantern Rescue
----------------------------
- Model ID: 0x405 800D
- Camera: Canon EOS Rebel T7i / Kiss X9i
- Firmware version: 1.0.1 / 7.3.5 6D(33)
- IMG naming: 100CANON/IMG_0357.JPG
- User PS: CineStyle Jamh Natural Flat Colors
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- ROMBASEADDR: 0xE0040000
- boot_read/write_sector 106f45 107041
- 1018CB Card init => 2
- Dumping ROM0... 100%
- MD5: 7dd1e3b8c10211f086859effdb98b67f
- Dumping ROM1... 100%
- MD5: 5a9cf31fdd9789b1e27ace2a4a6b3ad2
- No serial flash.
- Saving RESCUE.LOG ...

[names_are_hard]

Hi Jamh - what kind of experience do you have?  The Digic 7 cams are in quite early stages.  ARM assembly, C and embedded systems experience would all be useful skills (or the time and inclination to learn them!).

[Jamh]

Hi, names_are_hard

Actually I'm not a programmer or an assembly expert
but I'm just started learning Python3 language + Arduino applications

[names_are_hard]

Cool - Python is a really useful and clean language to learn.  You can program Arduino in assembly and you might want to learn after a while - I read you can get significantly faster or smaller code than AVR compilers generate.  For tiny CPUs it can be important

For now I don't think there is much you'll be able to help with porting to 800D.  But the code is freely available if you'd like to learn from it!  Get comfortable with C and AVR assembly and you could even try to work on a port.

[lacek]

Hi Jamh - what kind of experience do you have?  The Digic 7 cams are in quite early stages.  ARM assembly, C and embedded systems experience would all be useful skills (or the time and inclination to learn them!).

Hi, I have 6d2, I have C/c++ experience (I code lots of numerics code) and some experience with "why this bloody hell does not link", but formally I'm not a programmer. I have some ASM experience, but I am "rusty" (last ASM code I wrote was 16bit code for Pentium 200 MMX and it was running in DOS...), though I guess this is not a problem, though it was of course single core unlike the digic 7? I have experience with embedded system, but very high-level (iOS apps development) and very low level (soldering mobile phones under the microsope), the mid levels (such as coding microcontrollers) is what I do not have. I work on Linux @home and @work.

And I have some time and lots of inclination to learn.

My motivation is to speed up having clean HDMI output on 6d2, and more importantly actually I am looking for new challenges - things like low-level take on embedded systems is something I always wanted to work on, but never dared to try....

How can I help? I guess the question  I want to ask is what is the next thing to do on 6d2. I have read though this thread -  it seems to me it is  "getting stubs" (is it right?) If yes, should I start with "getting stubs" tutorial and references therein? Is there a place where I can ask newbie questions?

I also read about the IRC channel where people communicate. Is it the same IRC (chat software) that was popular in early 90s?  If yes, what is the server?

[Walter Schulz]

STRG-f
Then type
irc

Welcome! And SCNR!

[names_are_hard]

Hi Lacek!

Stubs you can do first, you can also get a build environment working - both can be done at the same time.  You should be able to run the digic6-dumper branch:
https://bitbucket.org/hudson/magic-lantern/branch/digic6-dumper

That should run on your cam...  but it doesn't do very much.  It's still useful, gives some logs, and should prove you have a correct build env.  Unfortunately, the digic6-dumper code hasn't been officially merged back into the mainline "unified" branch.  I have made some progress with that, here:
https://bitbucket.org/stephen-e/ml_200d

That code acts a lot more like full Magiclantern, but it crashes quite early on and I'm a bit stuck diagnosing the cause.  Also I have only tried to make it work on 200D, but if you look at the diffs you might find it helpful.

You will probably find my stubs useful, the 6d2 is probably fairly similar.

[aprofiti]

Found stubs for 6d2 1.0.4 some times ago.

They need to be double checked by a1ex before
a bootflag enabler will be released to public, then hello world should be straightforward.

[lacek]

Found stubs for 6d2 1.0.4 some times ago.

They need to be double checked by a1ex before
a bootflag enabler will be released to public, then hello world should be straightforward.

Ok I see three things:
1) the stubs I have found so far (that Memory functions, but they are easy, still curious how did you identify that "create" function) are not be really useful, as I have 1.0.2 firmware
2) the stubs I have found so far are not really useful, as you have all of them as well
3) looking for stubs seemed much easier than debugging the code...

I will start by looking at the stubs you have found, after I update the camera...

[lacek]

Found stubs for 6d2 1.0.4 some times ago.

They need to be double checked by a1ex before
a bootflag enabler will be released to public, then hello world should be straightforward.

Actually I have three more questions:

1) I was following the "stubs tutorial". The Memory stubs I have identified were of the type : if(func()!=0) { "func() failed" } - the latter type discussed in the "stubs tutorial".  There are other types mentioned, for example "label followed by push command".   Do you write own, custom scripts that parse entire disassembly file and look for the occurrence of a label in line N and push command in line N+1 or are there some established scripts that help with stub searching?

2) I assume that stub searching is easier if you have an older camera with stubs identified: I assume that large % the low-level API stays unchanged from version to version, and one could look e.g for a similar sequence of commands or sth like that.  Am I making sense here?

3) the stubs.S files for other cameras have around 50-100 stubs that are identified. I am surprised that this number is actually so low. There way more functions mentioned in the string files for the ROM i downloaded. The ML code is not able to operate merely on these 50-100 functions right? How is that so?  does this: "THUMB_FN(0xE04E706A,  call)                          /* many functions called by name (lv_start, lv_stop etc) */" open the door to call everything else?

[aprofiti]

1) I was following the "stubs tutorial"...
....
Do you write own, custom scripts that parse entire disassembly file and look for the occurrence of a label in line N and push command in line N+1 or are there some established scripts that help with stub searching?

Most of the time looking for strings is a good way to have a looks in the right direction (or code segment), usually can be done by searching inside decompiled files if using dissasemble.pl script or by looking for xrefs in IDA.

check-stubs.py can also be useful when porting firmware update.

that Memory functions, but they are easy, still curious how did you identify that "create" function

Also using QEMU for some tricks (.idc file of function called during emulation and code blocks copied from ROM to RAM), helped me a lot to have more functions marked inside the disassembly or to retrieve the RAM address instead of the ROM address for a specific stub.

IDA will replace and display calls to RAM version of functions automatically if code segments are loaded into the project; the exact function can be found inside the disassembly without using them, but having RAM address instead come in handy when we need to easly patch them.

2) I assume that stub searching is easier if you have an older camera with stubs identified: I assume that large % the low-level API stays unchanged from version to version, and one could look e.g for a similar sequence of commands or sth like that.  Am I making sense here?

Definitely useful to have a reference camera which use same DIGIC generation, it makes looking for stubs much easier because much of the code is shared and usually unchanged between them.

But because most of the code is shared or evolved during years, also comparing disassembly from older cameras can reveal the same or similar code structure to compare with: If I remember correctly I also double checked using a disassembly from 50D (Digic IV) when working on 77D and others (Digic VII).

Unfortunately comparing too much older cameras with a more recent one, ie. 5DC with 50D or even 40D (only 1 generation newer) won't help much... while disassembling 5DC, a lot of stubs looks like 50% smaller or completely different from newer cameras... also error messages (ie. looking for error code pattern from "finding stubs") looks like are handled in a different way, a lot of strings missing... making things harder...

3) the stubs.S files for other cameras have around 50-100 stubs that are identified. I am surprised that this number is actually so low. There way more functions mentioned in the string files for the ROM i downloaded.

Some older camera have unused stubs which were commented because they were replaced by standard library instead of canon's version, or because some stubs required by some specific feature are not necessary anymore due to some refactor of ML code.

Obviously having lesser stubs to find makes porting much easier for us.

The ML code is not able to operate merely on these 50-100 functions right? How is that so?  does this: "THUMB_FN(0xE04E706A,  call)                          /* many functions called by name (lv_start, lv_stop etc) */" open the door to call everything else?

A much better answer to last question need to be provided by main devs or someone which saw how ML code evolved during years.

[lacek]

Thank you for very insightful reply. I will analyze it thoroughly.

[Argiziont]

Ok, i have 200d and i am close to programming (sort of)
I read some posts on this forum.
I read and practiced on python neural network (for those who know 'theano','openCV',' tensorflow') And for a now learning C/C++ inThe University. So i thought i could help us to faster release 200d firmware. But when i tried to set up Ubuntu then qemu and ML on it i faced with huge amount of problems, i said ok and launched my second OS Xubuntu and thied here but i still could not to run this F qemu canon firmware I thied again and again and killed about 20 hours.
I think it's too complicated. Now i really appriciate your work guys, it's really tough job.
Sory for terrible and aughfull english, I'm live in Ukraine and we have bad english education, but i do my best.
Thank you in advance for your future works.

[hkisgg]

Hello, I'm an owner of 77D and I didn't know anything about ML until I bought Canon 77D. I have been looking into posts on the development of Digic 7 especially 77D. After reading every post carefully I understood and installed a virtual machine with ubuntu, magic lantern and qemu, while writing this message, I have kept the battery of 77D for charging and will get rom files. I have done a few c programs and but I'm not a science background student. I have seen posts on finding stubs. I need help with that if you kindly show me how to get stubs file(s) and 1 example of finding stubs(if there are threads or posts). I can help with the development of ML for Canon 77D. (Totally noob, I'm sorry if anything wrong)