How to run Magic Lantern into QEMU?!...

[dfort]

Great, I'll get to it after work today. A few questions, I think you have my 1.3.4 ROM or is it really a 1.1.3? I remember something about caching the previous version number when dumping a ROM from a new firmware update.

Can you get a dm-spy log from photo mode, with LOG_INTERRUPTS enabled?

This isn't a startup log so what action do you want me to log? Note that on 1.3.4 there is still that issue that also affected 1.1.3 when opening the ML menu in photo mode, not in LiveView. There is a flicker showing the Canon menus before the ML menu comes up. This only happens the first time the ML menus are accessed.

- change property 0x204000D to 1

Where do I change that property?

[a1ex]

Yes, a startup log. However, now that I've narrowed down the issue, I can get it myself.

I've tested with:

Code: [Select]

    int value = 1;
    prop_request_change(PROP_LCD_BRIGHTNESS_MODE, &value, 4);

placed in my_big_init_task, somewhere after call_init_funcs.

Alternatively, you may use prop_diag from the recovery branch, find where that property is stored in the ROM, and patch it.

For 1.2.3, you need this patch:

Code: [Select]

--- a/contrib/qemu/eos/eos.c
+++ b/contrib/qemu/eos/eos.c
@@ -4442,4 +4442,11 @@
     switch (address & 0xFFF)
     {
+        case 0x014:
+        {
+            /* 5D3 1.2.3: expects 0x10 for built-in LCD and 0x4 for HDMI? */
+            ret = 0x10;
+            break;
+        }
+
         case 0x01C:
         case 0x31C:

Not tested on 1.3.x.

[dfort]

Yay!



Not working with 1.3.4 - yet.

I did find this small issue when rebuilding QEMU.

Code: [Select]

../../Makefile.setup:100: *** missing separator.  Stop.

I had to comment out the "undefine CONFIG_SITE" for it to work on a Mac.

Makefile.setup

Code: [Select]

# some recent Linux distros have this defined
# we don't use it, but the checks below will get upset and print a warning
# undefine CONFIG_SITE


[a1ex]

More updates:

- GUI works for 6D, 70D and 5D3 1.2.3
- the test suite was getting too slow for my taste, so I've refactored it to allow parallel execution
  (about 1 order of magnitude faster on parallelized tests, about 2-3 times faster overall)

Here's a puzzle where I need some help, if you are familiar with containers (the TODO at the end of the Parallel execution section).

edit: screenshots ready

[dfort]

Trying to get to the next level in this game.

I managed to get 1.3.3 and 1.3.4 into the Canon menu without loading ML "boot=0"





Kind of tricky to get these screenshots because the patch to navigate the menus seems to work only when ML is loaded and I don't have GUI emulation working on these firmware versions yet. The way I did it was by going to the Canon menu I want to show in the camera and dump the firmware in that state. Yeah, 133 and 134 are working in camera but not in QEMU.

The 1.3.3 port that chris_miller did a while back almost works:





but once I merged it into the latest patched QEMU branch it was no better than 134.

Here's how I've got the directory structure for the 5D3:



This brings up a question about debugmsg.gdb. There is a section specific to 5D3.123:

Code: [Select]

# 1.2.3
if *(int*)0xFF136C94 == 0xE92D403E
  b *0xFF13B674
  register_func_log
end

I'm not sure if I'm stressing over the small stuff. According to "Blame" - Alex  committed 29bab2b, "GDB scripts: disabled slow items by default (semaphores, message queues, MPU communication, ResLock, EDMAC)" I was able to find the equivalent address for 1.3.3 and 1.3.4. However, shouldn't the debugmsg.gdb for each of these firmware versions be inside of the appropriate directory and run with, for example:

Code: [Select]

./run_canon_fw.sh 5D3,firmware="134;boot=1" -d debugmsg -s -S & arm-none-eabi-gdb -x 5D3/134/debugmsg.gdb
An issue I bumped up against is when using this command from the qemu directory to compile a version:

Code: [Select]

make -C ../magic-lantern/platform/5D3.134 install_qemu

I'm often getting messages that the sd.img resource is busy and it won't copy ML onto the image file. However, mounting the virtual sd card and installing it that way works fine.

Several of the recent QEMU updates have to do with the run_ml_all_cams.sh script so I gave that a try and was able to get log files for all of the 5D3 firmware versions. As expected some problems are showing up on the 1.3.3 and 1.3.4 versions.

5D3.134.log

Code: [Select]

c
./run_canon_fw.sh 5D3,firmware=134;boot=1 -display none -monitor stdio
pidof: illegal option -- s
ps: Invalid process id: Help:
ps: illegal option -- k
usage: ps [-AaCcEefhjlMmrSTvwXx] [-O fmt | -o fmt] [-G gid[,gid...]]
          [-g grp[,grp...]] [-u [uid,uid...]]
          [-p pid[,pid...]] [-t tty[,tty...]] [-U user[,user...]]
       ps [-L]
ps: Invalid process id: Help:
ps: illegal option -- k
usage: ps [-AaCcEefhjlMmrSTvwXx] [-O fmt | -o fmt] [-G gid[,gid...]]
          [-g grp[,grp...]] [-u [uid,uid...]]
          [-p pid[,pid...]] [-t tty[,tty...]] [-U user[,user...]]
       ps [-L]
 &

DebugMsg=00005b90 (overriden)
QEMU 2.5.0 monitor - type 'help' for more information
(qemu) Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - DFFFFFFF: eos.iomem
[EOS] loading symbols from ../magic-lantern/platform/5D3.134//autoexec (800000-86CB40)
[EOS] loading symbols from ../magic-lantern/platform/5D3.134//magiclantern (69500-E7F14)
[EOS] loading './5D3/134/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './5D3/134/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[MPU] warning: non-empty spell #41 (PROP_VIDEO_MODE) has duplicate(s): #42
[MPU] warning: non-empty spell #74 (PROP_TFT_STATUS) has duplicate(s): #48 #49 #52 #56 #59 #63 #70 #79 #80 #85 #87 #92 #95 #100 #103 #108
[MPU] warning: non-empty spell #84 (Current Q position) has duplicate(s): #82 #89 #91
[MPU] warning: non-empty spell #93 (Current Q position) has duplicate(s): #97 #99
[MPU] warning: non-empty spell #98 (Current Q position) has duplicate(s): #51 #58 #65 #83 #90 #106
[MPU] warning: non-empty spell #101 (Current Q position) has duplicate(s): #105 #107
[MPU] warning: non-empty spell #113 (PROP_CARD1_STATUS) has duplicate(s): #8

[MPU] Available keys:
- Arrow keys   : Navigation
- Numpad keys  : Joystick (8 directions)
- Numpad 5     : Joystick center
- PgUp, PgDn   : Sub dial (rear scrollwheel)
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess (press only)
- M            : MENU (press only)
- P            : PLAY (press only)
- I            : INFO/DISP
- Q            : guess (press only)
- L            : LiveView (press only)
- W            : Pic.Style (press only)
- Shift        : Half-shutter
- B            : Open battery door
- C            : Open card door
- F10          : Power down switch
- F1           : show this help

Setting BOOTDISK flag to FFFFFFFF
FFFF0948: MCR p15,0,Rd,cr9,cr1,0: XSCALE_LOCK_ICACHE_LINE <- 0x40000006 (40000000 - 40000FFF, 0x1000)
FFFF0948: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x2078
FFFF0948: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0x12078   
FFFF2F8C: MCR p15,0,Rd,cr6,cr0,0:  946_PRBS0 <- 0x3F       (00000000 - FFFFFFFF, 0x100000000)
FFFF2F94: MCR p15,0,Rd,cr6,cr1,0:  946_PRBS1 <- 0x3D       (00000000 - 7FFFFFFF, 0x80000000)
FFFF2F9C: MCR p15,0,Rd,cr6,cr2,0:  946_PRBS2 <- 0xE0000039 (E0000000 - FFFFFFFF, 0x20000000)
FFFF2FA4: MCR p15,0,Rd,cr6,cr3,0:  946_PRBS3 <- 0xC0000039 (C0000000 - DFFFFFFF, 0x20000000)
FFFF2FAC: MCR p15,0,Rd,cr6,cr4,0:  946_PRBS4 <- 0xFF00002F (FF000000 - FFFFFFFF, 0x1000000)
FFFF2FB4: MCR p15,0,Rd,cr6,cr5,0:  946_PRBS5 <- 0x39       (00000000 - 1FFFFFFF, 0x20000000)
FFFF2FBC: MCR p15,0,Rd,cr6,cr6,0:  946_PRBS6 <- 0xF700002F (F7000000 - F7FFFFFF, 0x1000000)
FFFF2FC4: MCR p15,0,Rd,cr2,cr0,0: DCACHE_CFG <- 0x70       
FFFF2FCC: MCR p15,0,Rd,cr3,cr0,0:       DACR <- 0x70       
FFFF2FD0: MCR p15,0,Rd,cr2,cr0,1: ICACHE_CFG <- 0x70       
FFFF2FD4: MCR p15,0,Rd,cr5,cr0,0:    DATA_AP <- 0x3FFF     
FFFF2FDC: MCR p15,0,Rd,cr5,cr0,1:    INSN_AP <- 0x3FFF     
FFFF2FE0: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x12078
FFFF3000: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC001307D
FFFF0974: MCR p15,0,Rd,cr9,cr1,1: XSCALE_UNLOCK_ICACHE <- 0x6        (00000000 - 00000FFF, 0x1000)
FFFF0974: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC001307D
FFFF0974: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005307D
FFFF09A4: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005307D
FFFF09A4: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
SD LOAD OK.

Open file for read : AUTOEXEC.BIN

SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
File size : 0x6CB40

Now jump to AUTOEXEC.BIN!!

0010DCCC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010DCCC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
K285 READY

qququiquit
[MPU] WARNING: forced shutdown.

For clean shutdown, please use 'Machine -> Power Down'
(or 'system_powerdown' in QEMU monitor.)

pidof and ps on the Mac don't seem to have those options though I don't know if that is affecting the validity of the log.

In any case, even though the 1.3.3 and 1.3.4 ML ports are running in camera it looks like QEMU is showing some issues that need to be resolved. Now where to start?

[a1ex]

Didn't this work?

- set LCD brightness to Manual before dumping the ROM

Didn't look into light sensor emulation yet.

The only errors I've got about sd.img were if the card image was full (I was running the silent picture module and the card image got filled with dng's pretty fast), or when copying ML with qemu already running (this results in filesystem corruption; just restore from sd.img.xz).

Pushed some Mac fixes. The pidof/ps issue was cosmetic (just re-printing the commands after clearing the screen).

To make the log a bit more readable, you could either "cat" it to a terminal, then copy the result, or run it through ansi2txt to remove the color codes. Maybe also add "-d debugmsg" to the emulation.

[dfort]

Didn't this work?

- set LCD brightness to Manual before dumping the ROM

Nope - I just tried it and it did't get any further. Just to make sure we're talking apples to apples:

Reproduced with your 1.1.3 ROM.

I'm testing 1.3.4 which I passed to you. I also made dumps for 1.1.3, 1.2.3 and 1.3.3 on the same camera using the same settings but I don't believe I passed those to you. We had an interesting glitch with the very first 1.3.4 ROM dump which is the one you are probably using. Right after updating the firmware ML was still showing the firmware version it was updated from (1.1.3).

The logs are in color just like the QEMU output? No wonder I couldn't make sense of those logs. I can now see that the problem is very early in the process:

SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
SD LOAD OK.
Open file for read : AUTOEXEC.BIN
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
File size : 0x6C240
Now jump to AUTOEXEC.BIN!!
0010DCCC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010DCCC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
K285 READY
quit
[MPU] WARNING: forced shutdown.

The only errors I've got about sd.img were if the card image was full (I was running the silent picture module and the card image got filled with dng's pretty fast), or when copying ML with qemu already running (this results in filesystem corruption; just restore from sd.img.xz).

I think that the issue I'm seeing is a Mac problem. When installing from a different branch (not qemu) I mount/unmount the sd.img using the finder. This works fine except when I go back to the "make -C ../magic-lantern/platform/5D3.134 install_qemu" method. Seems that OSX doesn't release the resources when umounting via the finder.

[EDIT] Deleted most of my previous edit - turns out I was using the wrong firmware version.

Note that on several platforms I need to press the "M" key to invoke the Canon menu so the screenshots I'm getting with "run_ml_all_cams.sh" don't show anything. Again, maybe just a Mac problem?

Speaking of Mac problems, I recently discovered the excellent QEMU documentation. Why is this a Mac problem? Because none of the Mac apps I've got opens the README.rst file properly. The best way I found to view it on a Mac is on Bitbucket.

Another issue not necessarily Mac specific but probably with bash version 4.4 and newer when running the install.sh script:

Code: [Select]

   Note: Canon GUI emulation (menu navigation, no LiveView) only works on:
   ./install.sh: line 418: warning: command substitution: ignored null byte in input
5D2 5D3 6D 50D 60D 70D 450D 500D 550D 600D 650D 700D 100D 1000D 1100D 1200D EOSM EOSM2

I tried several options but haven't found anything that removes that warning.

[a1ex]

Nope - I just tried it and it did't get any further.

Really? I've patched property 0x204000D = PROP_LCD_BRIGHTNESS_MODE from 0 (auto) to 1 (manual) in your 1.3.4 ROM (offset 0xf6259c in your ROM1, likely different on other 5D3's) and Canon menus started to work.

However, rather than finding the offset (e.g. with prop_diag or by manually looking up that property), it would have been a LOT easier (but maybe more time-consuming) to change this in Canon menu, reboot, then dump the ROM again with this setting already configured to "manual". Hence my suggestion.

on several platforms I need to press the "M" key to invoke the Canon menu so the screenshots I'm getting with "run_ml_all_cams.sh" don't show anything.

They probably start with the main display turned off; try pressing M from the script, e.g.:

Code: [Select]

env QEMU_SCRIPT="sleep 10; echo sendkey m; sleep 1" \
    SCREENSHOT=1 \
    ML_PLATFORMS="5D3.113/ 5D3.123/ 5D3.134/" \
    ./run_ml_all_cams.sh


Speaking of Mac problems, I recently discovered the excellent QEMU documentation. Why is this a Mac problem? Because none of the Mac apps I've got opens the README.rst file properly. The best way I found to view it on a Mac is on Bitbucket.

Same here - it's meant to be viewed online, but you can convert it to other formats if you wish. For example, pdf:

Code: [Select]

rst2latex README.rst > README.tex
pdflatex README.tex

Formatting is not the best (the layout could use some tweaking), but it's a good starting point. The (now outdated) ML user guide used to be in this (source) format, and was rendered as wikia code (now broken since the new wiki is dokuwiki), pdf (for desktop viewing) and in-camera BMPs (a bit heavyweight, but back then we did not have proportional fonts).

Conversion to HTML works as well, but it also needs some CSS (by default, it doesn't look very well). Didn't dig deeper to find one - maybe it's good to render it during installation.

The README was linked a few times, including first post (also asked for some proof-reading).

[dfort]

Really? I've patched property 0x204000D = PROP_LCD_BRIGHTNESS_MODE from 0 (auto) to 1 (manual) in your 1.3.4 ROM (offset 0xf6259c in your ROM1, likely different on other 5D3's) and Canon menus started to work.

Yes, the Canon menus work. I was referring to this test:

Code: [Select]

env ML_PLATFORMS="5D3.134/" \
TIMEOUT=10 \
SCREENSHOT=1 \
./run_ml_all_cams.sh
This is how it ends:

Code: [Select]

SD LOAD OK.
Open file for read : AUTOEXEC.BIN
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
File size : 0x6C240
Now jump to AUTOEXEC.BIN!!
0010DCCC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010DCCC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
K285 READY
screendump 5D3.134.ppm
(qemu) quit
[MPU] WARNING: forced shutdown.

This is how the "screendump" looks:



Running the run_ml_all_cams.sh script with a BOOT=0 option will produce a more complete log but the screendump looks the same. When I run this command:

Code: [Select]

./run_canon_fw.sh 5D3,firmware='134;boot=0'
It also comes up with a grey screen but pressing the "m" key will invoke the Canon menu. [EDIT] Though pressing the left/right arrow keys will freeze the QEMU GUI.

The README was linked a few times, including first post (also asked for some proof-reading).

Oops. My only excuse is that I didn't read the whole thing, only the parts that I was having problems with. Promise I'll read the rest of it and post any proofreading notes.

By the way, I usually run "make clean" from the magic-lantern directory to clean up everything but it doesn't work with the qemu branch. I've got to run "make clean" in each individual directory.

[a1ex]

Same gray screen here - it appears to start with display turned off. Maybe it was configured that way before dumping the ROM.

BTW - managed to get a fairly decent local rendering of README.rst with rst2html5 from python3-docutils (unfortunately not in python2-docutils which our script already installs).

edit: there are two three different variants of rst2html5 - one from python3-docutils which gives a nice layout, but the overall look is a bit dull, and this rst2html5, which has a bunch of advanced options and styles, but gives bad layout with default settings, and there's also this one, which doesn't even like my source...

Installing any of these "third party" rst2html5 with pip3 breaks the rst2html5 from python3-docutils (even when running as rst2html5-docutils).

TLDR: auto-rendering the RST as HTML on user's PC from the install script may be a can of worms.

[dfort]

Same gray screen here - it appears to start with display turned off. Maybe it was configured that way before dumping the ROM.

Interesting. Wonder how the camera should be set before dumping the ROM. Of all the dumps I tested only the 700D starts with a non-grey screen.

TLDR: auto-rendering the RST as HTML on user's PC from the install script may be a can of worms.

I don't mind reading online. Maybe add a README.html in the qemu directory that just links to the online documentation?

README.html

Code: [Select]

<meta http-equiv="refresh" content="0; url=https://bitbucket.org/hudson/magic-lantern/src/4895777de907c24ffd6332bcee23a7608450f6bd/contrib/qemu/README.rst?at=qemu&fileviewer=file-view-default" />

[a1ex]

700D starts in movie mode (that's how you started it when you've got the MPU log).

Here's how various models start with my ROMs (look at *-menu.png, first image from the set):

https://builds.magiclantern.fm/jenkins/job/QEMU-tests/lastSuccessfulBuild/artifact/qemu/tests/

For 6D, Audionut uploaded 3 logs: movie, photo LV, photo without LV. I've used the last one for QEMU, but you can try the others as well. From the mpu_spells directory:

Code: [Select]

python extract_init_spells.py /path/to/6D-startup_movie_mode.LOG > 6D.h

and it will start in the same way as 700D.

Don't know the property for starting with the main info screen turned on - I believe you should press INFO until that screen appears, and make sure the next boot actually starts with that screen. Then dump the ROM. I didn't do anything special.

On some models, the firmware thinks the eye sensor near the LCD is active, so it turns off the display. For example, on 450D, I have to turn off this option from Canon menu, then the info screen appears.

[dfort]

Tricky stuff.

Code: [Select]

python extract_init_spells.py /Users/rosiefort/Desktop/6D_startup_movie_mode.LOG
Traceback (most recent call last):
  File "extract_init_spells.py", line 169, in <module>
    model = log_filename[:log_filename.index("-")]
ValueError: substring not found

Details details, the log file needs to be renamed to "6D-startup_movie_mode.LOG" and the output saved to "6D.h" like this:

Code: [Select]

python extract_init_spells.py /Users/rosiefort/Desktop/6D-startup_movie_mode.LOG > 6D.h
It doesn't always catch the menu in the screendump. Maybe that has to do with the setting of the TIMER option?

Code: [Select]

env ML_PLATFORMS="6D.116/" \
BOOT=1 \
TIMEOUT=10 \
SCREENSHOT=1 \
./run_ml_all_cams.sh
But yeah, eventually I did get that same screendump as the 700D.

So getting ML working in QEMU on the 5D3.133/134 is a combination of dumping the ROM with the camera at some certain setting that will bring up the Canon menu instead of a blank screen and getting a startup log file that captures the MPU messages?

[EDIT] Oh yeah, and just for good measure turn off the automatic LCD brightness.

[a1ex]

So getting ML working in QEMU on the 5D3.133/134 is a combination of dumping the ROM with the camera at some certain setting that will bring up the Canon menu instead of a blank screen and getting a startup log file that captures the MPU messages?

I've downloaded your 1.3.4 build (Nightly.2017Nov02.5D3134.zip), placed it on the virtual card, started QEMU, pressed DELETE on the blank screen and entered ML menu. That's expected - you can open ML menu if your main display is off. Also pressed INFO at startup a few times and got Canon's screen. ML menu works from there as well.

To have the emulation start with Canon's info screen, it's a matter of dumping the ROM with this setting enabled (I hope there are no other tricks). The MPU messages were already captured in this configuration, so you shouldn't have to change them. However, that's just a minor cosmetic issue.

ML emulation already works (with your 1.3.4 ROM, patched for manual LCD brightness as described above), so I'm not sure what your question is.


Noticed something weird: with ML loaded, if the first thing done after startup is pressing M twice, there is an error coming from a ML task, when calling some GUI function from Canon (maybe a bad stub?):

Code: [Select]

ASSERT : ./Dialog/Dialog.c, Task = debug_task, Line 1049

After this event, ML menu stops working. Repeatable.

The error doesn't happen without ML loaded (boot=0). With ML loaded (boot=1), it doesn't save a crash log (it should).

Pressing M twice after navigating ML menu works fine.

Does it match the behavior on real hardware?

edit: got a call stack (b *0x1900):

Code: [Select]

0x76250(0, 76250, 19980218, 19980218)                                            at [debug_task:de48:1ba5b8] (pc:sp)
 0x75C08(bf29d "ML/FONTS/", 0, 0, 69b84)                                         at [debug_task:76464:1ba530] (pc:sp)
  0x75A88(1ba3f8 "ML/FONTS/ARGHLF22.RBF", 0, 42, 1ba3ec)                         at [debug_task:75cac:1ba3f0] (pc:sp)
   0x756B0(4, 1ba374 "Reading ML/FONTS/ARGHLF22.RBF...", 42, 1ba364)             at [debug_task:75be4:1ba368] (pc:sp)
    0xBB4B8 -> 0xFF359384(0, 4, 1ba374 "Reading ML/FONTS/ARGHLF22.RBF...", 8181b4)
                                                                                 at [debug_task:756dc:1ba358] (pc:sp)
     0xFF4560CC(0, d, 1ba374 "Reading ML/FONTS/ARGHLF22.RBF...", 21)             at [debug_task:ff3593b8:1ba340] (pc:sp)
      0xFF455F18(0, 4, 1ba374 "Reading ML/FONTS/ARGHLF22.RBF...", 21)            at [debug_task:ff4560e0:1ba330] (pc:sp)
       0x1900(ff45433c "pDialog != NULL", ff454308 "./Dialog/Dialog.c", 419, 21) at [debug_task:ff455f34:1ba320] (pc:sp)

0xFF359384 is dialog_set_property_str; maybe ML thinks you are on the Format dialog? That's the only place where ML tries to change Canon's menu strings.

Please double-check DIALOG_MnCardFormatBegin in consts.h.

[dfort]

ML emulation already works (with your 1.3.4 ROM, patched for manual LCD brightness as described above), so I'm not sure what your question is.

Tracked down the problem. I merged 1.3.3 and 1.3.4 and qemu and it looks like it was a bad merge. I didn't think of running the build from my pull request. What I'm trying to figure out is a menu glitch issue that I posted in the Canon 5D Mark III / 5D3 / Firmware 1.3.4 topic.

Good to finally see ML on 5D3.134 working in QEMU.

Noticed something weird: with ML loaded, if the first thing done after startup is pressing M twice, there is an error coming from a ML task, when calling some GUI
...
Does it match the behavior on real hardware?

No problem when pressing the Menu button twice after startup with the Nov02 build on the camera. The issue I've got on camera is going into the ML menu (Trash button) after startup outside of LiveView. Maybe it is related?

Please double-check DIALOG_MnCardFormatBegin in consts.h.

Thanks for pointing that out. @chris_overseas got it right in his 1.3.3 port but I missed it. No change from 1.3.3 to 1.3.4 so that problem should be fixed. Updated the pull request and uploaded a new build.

[a1ex]

What I'm trying to figure out is a menu glitch issue that I posted in the Canon 5D Mark III / 5D3 / Firmware 1.3.4 topic.

The issue can be reproduced in QEMU here. There is some abnormal SD card activity from debug_task the first time you open ML menu (run with -d debugmsg) and the restore after format feature is still not working with today's build. Therefore, my advice would be to double-check the same stubs.

[dfort]

...my advice would be to double-check the same stubs.

Thanks, that's exactly what the problem was. Tested on camera.

[a1ex]

So there is a missing section in the  README telling you to also compile the sf_dump Module and putting it on the ML Card and activate it and then Reboot the camera and use the module from the Debug Menu of ML. (only found it through full text search on the whole ML directory)

Solved. The serial flash dumper should also be included at startup, as part of the usual ROM backup (maybe also in the installer).

3. In the README under DEBUGGING you also write to use "make CONFIG_qemu=y" and the "make install_qemu" which wouldn't compile in the unified branch for the 700D. I found out that the qemu (or no dm_spy_experiments) branch is needed to use these options, maybe stress that out a bit more in the section.

Solved. Soon we'll have QEMU in mainline as well.

Still, I often test old changesets in QEMU (usually for troubleshooting, maybe "hg bisect"), so it's helpful to know how to backport this rule whenever you need it.

Also been fixing a couple of minor things.

[a1ex]

Some progress on emulating DIGIC 6:

- no more startup patches needed!
- Dry-shell works
- serial flash emulation (needs SFDATA.BIN from a D5 camera)
- MPU spells guessed from 60D (only this is required to boot the GUI on most D3-D5 models)
- Omar firmware revealed (doesn't quite work yet)
- 80D starts a LOT of tasks, including some filesystem drivers and starts to initialize the image capture backend; other D6 models are catching up.
- edit: 80D file I/O works too (creates DCIM dir, saves debug logs on the SD image)

You will need a patched SFDATA.BIN (serial flash dump) from a 70D (preferred), or 700D, 650D, EOSM, 6D (not sure about 100D). If you don't have one, just comment it out in model_list.c; most of the stuff appears to work without it.

Fun stuff:

Code: [Select]

( sleep 3; echo "akashimorino";
  sleep 1; echo "SHM_SHOW_INFO";
  sleep 1; echo "SHM_SHOW_DIST_INFO";
) | ./run_canon_fw.sh 80D -serial stdio

To get an idea how far the emulation goes:

Code: [Select]

# with 46f2e6e1cbb0 (right before the above stuff):
(./run_canon_fw.sh 80D,firmware="102;boot=0" -d debugmsg -s -S & arm-none-eabi-gdb -x 80D/debugmsg.gdb) |& grep Notify.*Cur --text
[        init:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2018000, Flag = 0x10000)
[      SFRead:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2008000, Flag = 0x8000)

# with 7ea57e73c091 (the above stuff):
(./run_canon_fw.sh 80D,firmware="102;boot=0" -d debugmsg -s -S & arm-none-eabi-gdb -x 80D/debugmsg.gdb) |& grep --text Notify.*Cur
[        init:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2018000, Flag = 0x10000)
[      SFRead:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2008000, Flag = 0x8000)
[     RomRead:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2000000, Flag = 0x2000000)
[     Startup:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 1, 0x2, Flag = 0x2)
[     Startup:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x20420010, Flag = 0x20000000)
[      RscMgr:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x420010, Flag = 0x20000)
[     FileMgr:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x400010, Flag = 0x10)
[     FileMgr:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x400000, Flag = 0x400000)
[ShootCapture:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0xc0000, Flag = 0x40000)
[     Startup:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0x80000, Flag = 0x80000)

# 60D, which boots the GUI without any fuss
./run_canon_fw.sh 60D,firmware="boot=0" -d debugmsg |& grep --text Notify.*Cur
[        init:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x10000, Flag = 0x10000)
[    PowerMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 1, 0x20000002, Flag = 0x2)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 1, 0x20000000, Flag = 0x20000000)
[     FileMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x10, Flag = 0x10)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0xe0110, Flag = 0x40000)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0xa0110, Flag = 0x80000)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0x20110, Flag = 0x100)
[      RscMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0x20010, Flag = 0x20000)
[     FileMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0x10, Flag = 0x10)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 4, 0x110, Flag = 0x100)
[     FileMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 4, 0x10, Flag = 0x10)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 5, 0x80200200, Flag = 0x80000000)
[ GuiMainTask:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 5, 0x200200, Flag = 0x200000)
[       DpMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 5, 0x200, Flag = 0x200)

Queued: 1300D, 40D, 7D, property logging, docs on DryOS internals...

[Theta Sigma]

- 80D starts a LOT of tasks, including some filesystem drivers and starts to initialize the image capture backend; other D6 models are catching up.

Is shutter actuation data (shutter count) one of them?

[Ant123]

Some progress on emulating DIGIC 6:

I think it's possible to emulate simple drawing of text strings in case main CPU will send certain messages to MZRM core...
But on EOS M3  the camera controller still does not allow to start it normally and goes to shutdown.
What is the situation with DSLRs?

[a1ex]

Definitely - as long as these functions are called during the emulation. On DSLRs we don't know yet how the display buffer looks like - the above stuff was done without a camera, just by playing around with the ROM dump. The emulation doesn't seem to initialize Zico on 80D yet - does reach this stage on M3?

This snippet resembles TFT SIO registers:

Code: [Select]

(./run_canon_fw.sh EOSM3 -s -S -d io & arm-none-eabi-gdb -x EOSM3/debugmsg.gdb) |& grep --text -i -C 100 Backlight | grep --text -i -E "Backlight|TX|DIGIC6"

[     Startup:fc3587db ] (60) DispSwCon_TurnOnBackLight
[     Startup:fc14b9db ] (60) TurnOnBackLight
[DIGIC6] at Startup:FC32E2FE:FC1BB779 [0xD20B0D7C] <- 0xC       : ???
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0xB0      : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x104     : TX register
[DIGIC6] at Startup:FC32E25C:FC1BB78B [0xD20B0D7C] <- 0xC       : ???
[DIGIC6] at Startup:FC32E25C:FC1BB79F [0xD20B0D7C] <- 0xC       : ???
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0xC8      : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x100     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x10E     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x113     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x11A     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x119     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x117     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x117     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x126     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x125     : TX register

BTW, do you happen to have any notes on UTimer or Omar?

[Ant123]

The emulation doesn't seem to initialize Zico on 80D yet - does reach this stage on M3?

Code: [Select]

Set default DRAM parameter
#
DRYOS version 2.3, release #0055+p6
Copyright (C) 1997-2013 by CANON Inc.
[SDIO] Error
[SDIO] Error
[SDIO] Error
[SDIO] Error

StartDiskboot
Diskboot file not found
2.1.1
3.1.1
4.1.1
3.1.2:11,0,0,4,0
3.1.3:1
3.1.3:0
3.1.2:11,0,0,4,0
3.1.3:1
3.2.1:2
3.2.3:2
3.2.7:2
3.2.9:2
3.2.11:2
3.2.13:2
3.2.15:2
3.2.17:2
3.2.19:2
3.4.4.1:0,128,128,1
== PnlSync =========
  vwidth  : 494
  hwidth  : 909
  h_pre   : 723
  h_blank : 719
  vb_lt   : 490
  vb_st   : 11
  vp_lt   : 490
  vp_st   : 11
  vb_l    : 490
  vb_s    : 11
  vp_l    : 490
  vp_s    : 11
====================
3.4.1.1:720,480,1,10,1
3.3.2:fc5f95cc(32,32,32),1
3.3.3:fc5f95d0(0),1
3.3.4:fc5f95ec,(0,1),1
3.3.5:fc5f962c,1
3.3.7:fc5f965c,1
3.3.9:fc5f9664,1
3.3.10:fc5f9668,1
3.3.1:fc5f95cc,1
3.1.6:1,2,9,0
3.10.1:124,128,134,0,0,0,1
3.3.12:1,1
3.2.19:0
3.2.17:0
3.2.15:0
3.2.13:0
3.2.11:0
3.2.9:0
3.2.7:0
3.3.14:0,0,1
InitializeGraphicLog Addr:0x4112b000 Size:0x5000
DlphLog:Addr:0x4112b000, Size:0x1400
_FreeMsg   : ------ req:0 stt:32
_CreateMsg : 0xbff00500 size:12
_FreeMsg   : ------ req:0 stt:32
SendMsg   : 1
ZicoLog:Addr:0, Size:0x2800
InitializeGraphicLog SUCCESS
_FreeMsg   : ------ req:1 stt:33
_FreeMsg   : 0xbff00500 free:1
_CreateMsg : 0xbff00500 size:4
_FreeMsg   : ------ req:1 stt:33
SendMsg   : 2
_FreeMsg   : ------ req:2 stt:33
_CreateMsg : 0xbff00528 size:76
_FreeMsg   : ------ req:2 stt:33
SendMsg   : 3
_FreeMsg   : ------ req:3 stt:33
_CreateMsg : 0xbff00598 size:4
_FreeMsg   : ------ req:3 stt:33
SendMsg   : 4
_FreeMsg   : ------ req:4 stt:33
_CreateMsg : 0xbff005c0 size:0
_FreeMsg   : ------ req:4 stt:33
SendMsg   : 5
[GRYP]T: --- Initialize start ----------------
_FreeMsg   : ------ req:5 stt:37
_FreeMsg   : 0xbff00500 free:2
_FreeMsg   : 0xbff00528 free:3
_FreeMsg   : 0xbff00598 free:4
_FreeMsg   : 0xbff005c0 free:5
_CreateMsg : 0xbff00500 size:0
_FreeMsg   : ------ req:5 stt:37
SendMsg   : 6
[GRYP]T: InitializeGryp(Pri)    : Completed.
         Privilege Event handle : 0x02500050
_FreeMsg   : ------ req:6 stt:38
_FreeMsg   : 0xbff00500 free:6
_CreateMsg : 0xbff00500 size:0
_FreeMsg   : ------ req:6 stt:38
SendMsg   : 7
         GRYPHON revision       : 0x00000000
[GRYP]T: Initialize(Pri): Completed.
[GRYP]T: --- Initialize(Pri/Nml) Completed ---

_FreeMsg   : ------ req:7 stt:39
_FreeMsg   : 0xbff00500 free:7
_CreateMsg : 0xbff00500 size:12
_FreeMsg   : ------ req:7 stt:39
SendMsg   : 8
_FreeMsg   : ------ req:8 stt:40
_FreeMsg   : 0xbff00500 free:8
_CreateMsg : 0xbff00500 size:12
_FreeMsg   : ------ req:8 stt:40
SendMsg   : 9
_FreeMsg   : ------ req:9 stt:40
_CreateMsg : 0xbff00530 size:16

...



SendMsg   : 27
_FreeMsg   : ------ req:27 stt:59
_FreeMsg   : 0xbff00500 free:27
_CreateMsg : 0xbff00500 size:772
_FreeMsg   : ------ req:27 stt:59
SendMsg   : 28
3.4.1.4:5,00690f70,1,0,1e0
3.4.1.5:5,0,0,720,480,0,0,1
3.4.1.6:5,0,3,0,1
3.4.1.2:5,1,1
3.2.21:1,fc152451,00000000
3.2.22:1,1
3.2.3:0
3.3.15:1,1
At least it tries to draw something by sending JediDraw message (0xFC4BB8BA)
I've used this patch to display debug messages:

Code: [Select]

PatchDbgByte(0x00028698,0xFF);
PatchDbgByte(0x00028699,0xFF);
PatchDbgByte(0x0002869A,0xFF);
PatchDbgByte(0x0002869B,0xFF);

PatchDbgByte(0x000286A0,0x9D);
PatchDbgByte(0x000286A1,0xFD);
PatchDbgByte(0x000286A2,0x37);
PatchDbgByte(0x000286A3,0xFC);

PatchDbgByte(0x000286A8,0x9D);
PatchDbgByte(0x000286A9,0xFD);
PatchDbgByte(0x000286AA,0x37);
PatchDbgByte(0x000286AB,0xFC);

BTW, do you happen to have any notes on UTimer or Omar?

no

[dfort]

Noticed a message that I don't remember seeing on previous versions:

Code: [Select]

49:53: execution error: The variable qemu is not defined. (-2753)
Doesn't seem to hurt anything, just wondering if others are seeing it.

[a1ex]

Don't remember seeing it; when/where does it appear, and in what color?