Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: jplxpto on September 23, 2012, 08:29:02 PM

Title: How to run Magic Lantern into QEMU?!...
Post by: jplxpto on September 23, 2012, 08:29:02 PM
How to run Magic Lantern into QEMU?!...

I would like to know if anyone has ever launched Magic Lantern on QEMU. I've used it a few times, and it seemed to me that however limited it may be useful to conduct some tests.

If anyone has knowledge about this subject, thank you give me some tips that may also be useful for other developers.

Thank you.




Short answer: check the README (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst) and the developer guide (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/HACKING.rst).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nanomad on September 23, 2012, 09:10:22 PM
Last time I checked there was a patch for qemu in ML source code.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: jplxpto on September 23, 2012, 09:48:30 PM
Ok ... thank you...

One of these days I'll test it ...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: miyake on September 24, 2012, 10:50:23 AM
I'm now watching here.
http://chdk.wikia.com/wiki/GPL_Qemu

I wish this for something help for you.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: jplxpto on September 24, 2012, 02:21:25 PM
Quote from: miyake on September 24, 2012, 10:50:23 AM
I'm now watching here.
http://chdk.wikia.com/wiki/GPL_Qemu

I wish this for something help for you.

Thank you
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on February 08, 2013, 09:31:25 PM
With a bit of tweaking, that qemu patch seems to do something :)

(http://a1ex.magiclantern.fm/bleeding-edge/qemu.png)

(http://a1ex.magiclantern.fm/bleeding-edge/qemu2.png)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: scrax on March 17, 2013, 10:03:16 PM
I'm interested in running ML in QEMU for testing scripts, is the tweaked patch in the source?
What do I need to do compile a patched ML version and load it with QEMU?
I'm reading the page linked by miyake now.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on March 17, 2013, 10:29:30 PM
It's not yet ready for this. What's missing is task switching support; I've tried some hacks based on setjmp (this (http://homepage.cs.uiowa.edu/~jones/opsys/threads/) and this (https://www.cs.cmu.edu/~mihaib/articole/threads/)), but they weren't reliable at all, so I'm thinking to try something like FreeRTOS. A simple cooperative scheduler (to switch tasks when they call msleep) would be enough.

G3gg0 had some success in running the DryOS task switcher. My approach is a bit different, I'm trying to re-implement the stubs from scratch so only ML code is emulated - that is, menu, scripting engine, overlays etc.

The qemu patch is very rough and not yet published.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: jplxpto on March 23, 2013, 03:15:40 AM
I'm very interested in this subject.

I'll be glad to help you.

Maybe this can help us ... The eCos is another option (http://ecos.sourceware.org/about.html) and it already has support for GDB.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: mark.farnell on March 29, 2013, 09:51:25 PM
Quote from: jplxpto on March 23, 2013, 03:15:40 AM
I'm very interested in this subject.

I'll be glad to help you.

Maybe this can help us ... The eCos is another option (http://ecos.sourceware.org/about.html) and it already has support for GDB.

In this case, is eCos a replacement of qemu?  However it says it is an operating system.... so do you mean to replace DryOS with eCos altogether? 
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on March 29, 2013, 09:55:12 PM
Yes, only for emulation. I don't need to emulate the entire Canon firmware, just ML code.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: mark.farnell on March 29, 2013, 10:50:46 PM
Quote from: a1ex on March 29, 2013, 09:55:12 PM
Yes, only for emulation. I don't need to emulate the entire Canon firmware, just ML code.

So at this stage, is it possible to emulate the ML code with eCos?  If so, how?

Also will the ability of emulating the ML code make developing safer?  ( I mean detecting silly mistakes such as writing wrong values to NVRAM variables, that can potentially brick the camera)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on March 29, 2013, 11:10:23 PM
See my previous post. There was no progress since then.

Wrong NVRAM values depend only on how they are interpreted by Canon code, so emulating only ML code won't help. If you get permanent ERR70 after changing some setting, that's a clear sign that you have set an invalid value; there's no other way to tell this. Maybe full emulation could help here, but that's very difficult (of course, I won't be surprised if g3gg0 succeeds). Detecting memory leaks may be possible (maybe run ML under valgrind?)

I want to emulate ML so I can check how the menus look on each camera, without having to buy every single model. Also, it may be helpful when writing user scripts, or when working with graphics code, fonts etc.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: 1% on March 30, 2013, 12:26:45 AM
I got 600D booting in trix to the service/bootloader menu but didn't really know what to do with it afterwards.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on March 30, 2013, 12:53:32 AM
at the moment i am working on qemu to emulate the whole fimware.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Indy on March 30, 2013, 12:58:10 AM
Updater code (at least 7d one) can run partially on qemu.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on March 30, 2013, 01:07:34 AM
here are the changes i made to qemu:
http://upload.g3gg0.de/pub_files/7dc0800617416fdbeb1490dcc4a2164d/qemu_eos.7z

they contain a lot of hardware emulation.

booting fails imho due to the MPU that is not emulated. init routines expect a property that is not available.
i think the MPU would send it to the main firmware.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: jplxpto on April 04, 2013, 01:33:02 AM
Quote from: g3gg0 on March 30, 2013, 01:07:34 AM
here are the changes i made to qemu:
http://upload.g3gg0.de/pub_files/7dc0800617416fdbeb1490dcc4a2164d/qemu_eos.7z

they contain a lot of hardware emulation.

booting fails imho due to the MPU that is not emulated. init routines expect a property that is not available.
i think the MPU would send it to the main firmware.

Thank you! I will try this ..
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 08, 2013, 12:12:32 AM
Just merged my version with g3gg0's and pushed it on the main repo. Works on 60D, 600D, 500D, 5D2 and 650D.

In theory, you only have to run the install script (https://bitbucket.org/hudson/magic-lantern/src/tip/contrib/qemu/install.sh) - it will download QEMU 1.4.0, apply our modifications, and it will tell you what to do next.

(http://a1ex.magiclantern.fm/bleeding-edge/qemu.png)

If successful, you should get the hello world picture, and a log like this:


00000000 - 3FFFFFFF: eos.ram
40000000 - 7FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror_F1
F2000000 - F2FFFFFF: eos.rom0_mirror_F2
F3000000 - F3FFFFFF: eos.rom0_mirror_F3
F4000000 - F4FFFFFF: eos.rom0_mirror_F4
F5000000 - F5FFFFFF: eos.rom0_mirror_F5
F6000000 - F6FFFFFF: eos.rom0_mirror_F6
F7000000 - F7FFFFFF: eos.rom0_mirror_F7
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror_F9
FA000000 - FAFFFFFF: eos.rom1_mirror_FA
FB000000 - FBFFFFFF: eos.rom1_mirror_FB
FC000000 - FCFFFFFF: eos.rom1_mirror_FC
FD000000 - FDFFFFFF: eos.rom1_mirror_FD
FE000000 - FEFFFFFF: eos.rom1_mirror_FE
FF000000 - FFFFFFFF: eos.rom1_mirror_FF
C0000000 - CFFFFFFF: eos.iomem
[EOS] loading 'ROM-650D.BIN' to 0xF7000000-0xF8FFFFFF
[EOS] loading 'qemu-helper.bin' to 0x30000000-0x300088E7
[EOS] loading 'autoexec.bin' to 0x00800000-0x00855F4F
...
[GPIO] [0xC022C188] <- 0x138800 at pc=0x855E90
[FlashIF] at [0x000D4020]: 'Write enable' enabled
[Basic] at [0x000D4020] [0x00000000] <- [0xC0400008]
[Basic] at [0x000D4020] [0x00430005] -> [0xC0400008]
create_init_task(7e1ac)
*** init_task
[DebugMsg] (50,3) Magic Lantern v2.3.NEXT.2013Apr07.650D101 (781e0140ec5a+ (unified) tip)
[DebugMsg] (50,3) Built on 2013-04-07 20:02:47 by alex@thinkpad
...
Hello at QEMU console!


Let me know if it works for you (and what other dependencies you had to install).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: scrax on April 08, 2013, 05:26:24 AM
Quote from: a1ex on April 08, 2013, 12:12:32 AM
Just merged my version with g3gg0's and pushed it on the main repo. Works on 60D, 600D, 500D, 5D2 and 650D.

In theory, you only have to run the install script (https://bitbucket.org/hudson/magic-lantern/src/tip/contrib/qemu/install.sh) - it will download QEMU 1.4.0, apply our modifications, and it will tell you what to do next.

(http://a1ex.magiclantern.fm/bleeding-edge/qemu.png)

If successful, you should get the hello world picture, and a log like this:


00000000 - 3FFFFFFF: eos.ram
40000000 - 7FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror_F1
F2000000 - F2FFFFFF: eos.rom0_mirror_F2
F3000000 - F3FFFFFF: eos.rom0_mirror_F3
F4000000 - F4FFFFFF: eos.rom0_mirror_F4
F5000000 - F5FFFFFF: eos.rom0_mirror_F5
F6000000 - F6FFFFFF: eos.rom0_mirror_F6
F7000000 - F7FFFFFF: eos.rom0_mirror_F7
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror_F9
FA000000 - FAFFFFFF: eos.rom1_mirror_FA
FB000000 - FBFFFFFF: eos.rom1_mirror_FB
FC000000 - FCFFFFFF: eos.rom1_mirror_FC
FD000000 - FDFFFFFF: eos.rom1_mirror_FD
FE000000 - FEFFFFFF: eos.rom1_mirror_FE
FF000000 - FFFFFFFF: eos.rom1_mirror_FF
C0000000 - CFFFFFFF: eos.iomem
[EOS] loading 'ROM-650D.BIN' to 0xF7000000-0xF8FFFFFF
[EOS] loading 'qemu-helper.bin' to 0x30000000-0x300088E7
[EOS] loading 'autoexec.bin' to 0x00800000-0x00855F4F
...
[GPIO] [0xC022C188] <- 0x138800 at pc=0x855E90
[FlashIF] at [0x000D4020]: 'Write enable' enabled
[Basic] at [0x000D4020] [0x00000000] <- [0xC0400008]
[Basic] at [0x000D4020] [0x00430005] -> [0xC0400008]
create_init_task(7e1ac)
*** init_task
[DebugMsg] (50,3) Magic Lantern v2.3.NEXT.2013Apr07.650D101 (781e0140ec5a+ (unified) tip)
[DebugMsg] (50,3) Built on 2013-04-07 20:02:47 by alex@thinkpad
...
Hello at QEMU console!


Let me know if it works for you (and what other dependencies you had to install).

Will try on osx the script and report back, but first I need to clean up some space from my main hd.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: trsaunders on April 08, 2013, 09:37:44 AM
I'm trying to run this on Arch linux:
I had to build with these options because python 2 binary is called python2 and the docs building didn't work:
./configure --target-list=arm-softmmu --python=/usr/bin/python2 --disable-docs

when I try to launch:
➜  qemu  ./run_ml_5D3.sh
make: Entering directory `/home/tom/dev/software/qemu/qemu-1.4.0'
make: Leaving directory `/home/tom/dev/software/qemu/qemu-1.4.0'
make: Entering directory `/home/tom/dev/software/magic-lantern/platform/5D3.113'
[ VERSION  ]   ../../platform/5D3.113/version.c
[ CC       ]   version.o
[ MENU IDX ]   menuindexentries.h
No menuindex.txt not running "python2 menuindex.py"
[ CC       ]   menuindex.o
[ LD       ]   magiclantern
[ OBJCOPY  ]   magiclantern.bin
[ SYMBOLS  ]   magiclantern.sym
[ CC       ]   reboot.o
[ LD       ]   autoexec
autoexec.bin: 443984 bytes

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x06c550 0x000d5950 0x000d5950 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00069500 0x00069500 0x6c458 0x7bb6d RWE 0x100
[ OBJCOPY  ]   autoexec.bin
make: Leaving directory `/home/tom/dev/software/magic-lantern/platform/5D3.113'
make: Entering directory `/home/tom/dev/software/magic-lantern/platform/5D3.113'
make: `qemu-helper.bin' is up to date.
make: Leaving directory `/home/tom/dev/software/magic-lantern/platform/5D3.113'
rm: cannot remove 'vram.txt': No such file or directory
rm: cannot remove 'vram.png': No such file or directory
00000000 - 3FFFFFFF: eos.ram
40000000 - 7FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror_F1
F2000000 - F2FFFFFF: eos.rom0_mirror_F2
F3000000 - F3FFFFFF: eos.rom0_mirror_F3
F4000000 - F4FFFFFF: eos.rom0_mirror_F4
F5000000 - F5FFFFFF: eos.rom0_mirror_F5
F6000000 - F6FFFFFF: eos.rom0_mirror_F6
F7000000 - F7FFFFFF: eos.rom0_mirror_F7
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror_F9
FA000000 - FAFFFFFF: eos.rom1_mirror_FA
FB000000 - FBFFFFFF: eos.rom1_mirror_FB
FC000000 - FCFFFFFF: eos.rom1_mirror_FC
FD000000 - FDFFFFFF: eos.rom1_mirror_FD
FE000000 - FEFFFFFF: eos.rom1_mirror_FE
FF000000 - FFFFFFFF: eos.rom1_mirror_FF
C0000000 - CFFFFFFF: eos.iomem
eos_load_image: file not found 'ROM-5D3.BIN'
run_ml.sh: line 15: 25600 Aborted                 (core dumped) $QEMU_PATH/arm-softmmu/qemu-system-arm -M ML-$1
convert: unable to open image `vram.txt': No such file or directory @ error/blob.c/OpenBlob/2641.
convert: no images defined `vram.png' @ error/convert.c/ConvertImageCommand/3103.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 08, 2013, 09:46:22 AM
5D3 is not working yet, here it fails at create_init_task. You also have to get some Canon firmware dumps, see the install instructions.

You may have better luck if you add support for 50D (you need to add a ML_MACHINE definition and create a launch script).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: trsaunders on April 08, 2013, 10:13:39 AM
Sorry, I didn't read that well! I thought I had the 5D3-ROM in the correct location but I guess not. I now get as far as create_init_task so I assume that qemu is at least working. I'll have a go at adding support for 50D.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 08, 2013, 11:49:06 AM
5D3 works too, I was loading the old 1.1.2 dump by mistake.


create_init_task(695ac)
*** init_task
[DebugMsg] (50,3) Magic Lantern v2.3.NEXT.2013Apr08.5D3113 (087dd0afd6b8+ (unified) tip)
...


It fails at redraw, you will need to comment it out from ML code.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: trsaunders on April 08, 2013, 12:36:39 PM
(http://i.imgur.com/PXf5p5t.png)

50D worked without much effort - just copied 60D definitions, changed to 50D and added the appropriate run_ml script.

I tried to generate a patch for the changes but hg diff was producing a lot of spurious changes - maybe it doesn't like diffing a diff?!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 08, 2013, 12:55:14 PM
I've used a new hg tree under qemu to generate the diff. It's a bit ugly now; I'm thinking to modify the install script to store the new files as plain files, and use the patch only for modification to QEMU sources.

Edit: did these changes and some small additions:

- to emulate Canon firmware, without ML:
    ./run_canon_fw.sh 600D

- to generate a diff or commit changes (say you have modified eos.c or added some script):
    ./copy_back_to_contrib.sh
    then normal hg commands or gui in contrib/qemu

- to run the firmware in gdb:
    qemu-1.4.0/arm-softmmu/qemu-system-arm -M 5D3 -s -S in one terminal
    arm-elf-gdb -x gdbopts in another
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: 1% on April 09, 2013, 12:24:45 AM
6D is running in Qemu.. ML seemed to run but I did not see hello world in VNC.
Ran the canon FW and debugger and it opened up stopping at ff0c0008.
I guess this can be connected as debugger to ida or another disassembler?

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 09, 2013, 07:26:00 AM
Yes, you can connect from IDA or GDB to localhost:1234.

The display device is not implemented - it just saves a screenshot when you call dispcheck.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: 1% on April 10, 2013, 06:03:49 AM
I compiled on windows with mingw. It loads but get:

VNC server running on `::1:5900'

and can't connect to vnc or gdb.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on April 10, 2013, 12:31:35 PM
vnc isnt implemented for eos cameras, just gdb.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: 1% on April 10, 2013, 04:05:38 PM
Works in linux. You can connect over vnc and shut down qemu.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Marsu42 on April 10, 2013, 08:01:01 PM
Quote from: 1% on April 10, 2013, 06:03:49 AM
I compiled on windows with mingw. It loads but get

It would be nice to also get it running on Windows though - of course I have Linux/whatever vms, but for ml I'd like to get around them on my puny laptop just like there is no need to use Linux to compile ml.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: 1% on April 10, 2013, 11:18:38 PM
I can post the binary... the armmmu folder is only something like 20-30mb.... but I can't connect to it in windows, like it needs another patch to use the networking. The linux binary will only connect on localhost so I can't push it out to gdb over the network.

Thats the prob I'm having with QEMU.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Marsu42 on April 11, 2013, 01:35:48 AM
Quote from: 1% on April 10, 2013, 11:18:38 PM
The linux binary will only connect on localhost so I can't push it out to gdb over the network.

Use a ssh tunnel or network redirector?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: 1% on April 11, 2013, 02:52:02 AM
I turned UFW off and set some forwarding from lo to eth0... so far its working. Really would be nice to get it on one machine tho.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wolf on May 05, 2013, 10:16:49 AM
Is there a way to run qemu for a 550D?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: 1% on May 06, 2013, 12:32:41 AM
Yep, just edit the files and add 550D + values.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wolf on May 06, 2013, 02:32:19 AM
@1% Thanks. 

I changed the content of "run_ml_550D.sh" to "sh run_ml.sh 550D 109" (550 instead of 500)
but it still won't work. The 550D is not listed under supported machines. I don't know if it's important and failure is caused by something else, or caused by my installation? Well I have no vram.txt as you see. Do you have any clue?
$ ./run_ml_550D.sh
make: Entering directory `/home/wolf/qemu/qemu-1.4.0'
make: Leaving directory `/home/wolf/qemu/qemu-1.4.0'
make: Entering directory `/home/wolf/magic-lantern/platform/550D.109'
[ VERSION  ]   ../../platform/550D.109/version.c
[ CC       ]   version.o
[ MENU IDX ]   menuindexentries.h
No menuindex.txt not running "python2 menuindex.py"
[ CC       ]   menuindex.o
[ LD       ]   magiclantern
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 449232 bytes
[ SYMBOLS  ]   magiclantern.sym
[ CC       ]   reboot.o
[ LD       ]   autoexec

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x06dbc8 0x00cedbc8 0x00cedbc8 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00c80100 0x00c80100 0x6dad0 0x7d6d9 RWE 0x100
[ OBJCOPY  ]   autoexec.bin
[ STAT     ]   autoexec.bin
autoexec.bin: 449728 bytes
make: Leaving directory `/home/wolf/magic-lantern/platform/550D.109'
make: Entering directory `/home/wolf/magic-lantern/platform/550D.109'
make: `qemu-helper.bin' is up to date.
make: Leaving directory `/home/wolf/magic-lantern/platform/550D.109'
rm: cannot remove 'vram.txt': No such file or directory
rm: cannot remove 'vram.png': No such file or directory
Supported machines are:
none                 empty machine
collie               Collie PDA (SA-1110)
ML-50D               Magic Lantern on Canon EOS 50D
ML-60D               Magic Lantern on Canon EOS 60D
ML-600D              Magic Lantern on Canon EOS 600D
ML-500D              Magic Lantern on Canon EOS 500D
ML-5D2               Magic Lantern on Canon EOS 5D2
ML-5D3               Magic Lantern on Canon EOS 5D3
ML-650D              Magic Lantern on Canon EOS 650D
50D                  Canon EOS 50D
60D                  Canon EOS 60D
600D                 Canon EOS 600D
500D                 Canon EOS 500D
5D2                  Canon EOS 5D2
5D3                  Canon EOS 5D3
650D                 Canon EOS 650D
nuri                 Samsung NURI board (Exynos4210)
smdkc210             Samsung SMDKC210 board (Exynos4210)
connex               Gumstix Connex (PXA255)
verdex               Gumstix Verdex (PXA270)
highbank             Calxeda Highbank (ECX-1000)
integratorcp         ARM Integrator/CP (ARM926EJ-S) (default)
kzm                  ARM KZM Emulation Baseboard (ARM1136)
mainstone            Mainstone II (PXA27x)
musicpal             Marvell 88w8618 / MusicPal (ARM926EJ-S)
n800                 Nokia N800 tablet aka. RX-34 (OMAP2420)
n810                 Nokia N810 tablet aka. RX-44 (OMAP2420)
sx1                  Siemens SX1 (OMAP310) V2
sx1-v1               Siemens SX1 (OMAP310) V1
cheetah              Palm Tungsten|E aka. Cheetah PDA (OMAP310)
realview-eb          ARM RealView Emulation Baseboard (ARM926EJ-S)
realview-eb-mpcore   ARM RealView Emulation Baseboard (ARM11MPCore)
realview-pb-a8       ARM RealView Platform Baseboard for Cortex-A8
realview-pbx-a9      ARM RealView Platform Baseboard Explore for Cortex-A9
akita                Akita PDA (PXA270)
spitz                Spitz PDA (PXA270)
borzoi               Borzoi PDA (PXA270)
terrier              Terrier PDA (PXA270)
lm3s811evb           Stellaris LM3S811EVB
lm3s6965evb          Stellaris LM3S6965EVB
tosa                 Tosa PDA (PXA255)
versatilepb          ARM Versatile/PB (ARM926EJ-S)
versatileab          ARM Versatile/AB (ARM926EJ-S)
vexpress-a9          ARM Versatile Express for Cortex-A9
vexpress-a15         ARM Versatile Express for Cortex-A15
xilinx-zynq-a9       Xilinx Zynq Platform Baseboard for Cortex-A9
z2                   Zipit Z2 (PXA27x)
convert: unable to open image `vram.txt': No such file or directory @ error/blob.c/OpenBlob/2641.
convert: no images defined `vram.png' @ error/convert.c/ConvertImageCommand/3106.
$


Title: Re: How to run Magic Lantern into QEMU?!...
Post by: 1% on May 06, 2013, 02:53:45 AM
You have to add add it to eos.c too.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wolf on May 06, 2013, 09:06:40 PM
It's working. :-)
But is it possible to test a picoc script already?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: 1% on May 06, 2013, 10:34:23 PM
I wish.. I couldn't get anything useful for debugging either.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 31, 2014, 08:42:55 PM
QEMU working again :)

- runs DryOS task scheduler, semaphores, message queues (massive credits to g3gg0 for the low-level emulation code and insights)
- loads files from a local directory (sdcard or cfcard)
- able to load config files, modules, cropmarks...
- menu navigation working
- file manager working
- arkanoid working (playable!)
- properties not working
- most canon code is not working :P

(http://a1ex.magiclantern.fm/bleeding-edge/qemu-arkanoid.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu-menu.png)

https://bitbucket.org/hudson/magic-lantern/branch/qemu

Feel free to turn it into something useful... like script interpreter, testing server, source-level debugger, HDMI emulator, support for image buffers, add a nice GUI... or just port it for your camera. I've only tested it on 5D3 1.1.3, and it's been already used to debug the early 100D ML port in GDB.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 24, 2014, 05:39:25 PM

[DebugMsg] (139,22) Wait Master Wakeup
[GPIO] at [0xFF080090] [0x00000000] <- [0xC0220024]
....
[DebugMsg] (139,6) Wait Master Wakeup Timeout
[DebugMsg] (139,22) Master Wakeup


Other than that, emulating 7D ML in QEMU works quite well. Any hints about how to emulate the master processor and the communication between them?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nikfreak on October 02, 2014, 11:41:45 AM
Works fine here for 6D. Needed to do some additional stuff like filename capitalization etc. I just ran into one last problem. I have somehow no write access - at least there's no "magic.cfg" saved neither "bench.ppm" nor are ROM dumps from debug menu written etc. Though qemu doesn't report an error on qemu monitor. It's just like qemu has write access to some nirvana place in memory....


[???] [0x00000010] -> [0xC020006C] PC: 0xFF0C7C14
*** FIO_CreateFile('B:/ML/SETTINGS/magic.cfg') => 17
*** FIO_WriteFile(11, 231)
{{{
# Magic Lantern Nightly.2014Oct02.6D113 (b59a1ac5fbfc+ (qemu) tip)
# Built on 2014-10-02 08:56:42 UTC by magiclantern@magiclante
}}}
*** FIO_CloseFile(11)
[FIO wrapper] closefile() nothing open
Save configs...


Edit:
qemu-helper.c define R/W access. Do I have to change the values for 6D. Btw I compiled and tried via qemu branch
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 02, 2014, 12:19:44 PM
Correct, file writing is not implemented. The entire FIO implementation is weak (only single-task and very poor error checking) - that is, just barely enough to load ML :P.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 11, 2015, 12:26:12 PM
Just had success in emulating the display test (http://www.magiclantern.fm/forum/index.php?topic=14732) (on most cameras). On 6D, I had to emulate the bootloader as well (without it, the display init routine would get stuck).

(http://a1ex.magiclantern.fm/bleeding-edge/qemu-boot-6D.png)

To run the display test, look for the following "if (0)" and enable them:
- "bootloader config, 4 bpp" -> required to run all boot display tests
- "6D bootloader experiment" -> required for 6D; launch with ./run_canon_fw.sh 6D

edit: after some small changes, Linux works as well :)

(http://a1ex.magiclantern.fm/bleeding-edge/qemu-boot-linux.png)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: mk11174 on April 24, 2015, 03:48:31 PM
Anyone know what this HDMI USB thing is, I don't see this on the 550D or 700D when I play with them, this loops so much and makes it super slow to work the menus?
(http://s3.postimg.org/asramqo6r/Capture.jpg)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 24, 2015, 10:48:27 PM
Major progress: I'm able to launch Canon GUI under QEMU :)

( side note: Nikon Hacker guys achieved this step a long time ago (http://www.magiclantern.fm/forum/index.php?topic=8823.0), so we are just playing catch-up :P )

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/qemu-canon-gui.png)

More details:
- it launches most Canon tasks
- unmodified 60D firmware (without autoexec.bin or ROM patches) runs as well (and starts the GUI too)
- SD card emulation also works (it loads autoexec.bin and even creates the DCIM directory on startup)
- MPU emulation kinda works (it replays messages from a log file)
- sample log: 60D-qemu-canon-gui-and-sd.log (http://a1ex.magiclantern.fm/bleeding-edge/qemu/60D-qemu-canon-gui-and-sd.log)

Next steps:
- emulate unmodified autoexec.bin
- remove all those CONFIG_QEMU hacks
- implement key events as MPU messages
- CF emulation
- enable the emulation for other cameras
- do something about those huge logs
- make the code more QEMU-ish and less hackish
- write a quick start guide
- do something useful with it :)

What's the use?
- much easier to understand Canon firmware (you can see exactly what some piece of code does with the hardware)
- very useful in diagnosing soft-bricked cameras
- a way to debug your code (or Canon's) in a GUI (gdb or IDA)
- test bench for Lua scripting or for module development
- automated tests for the nightly builds (see also this proposal (http://www.magiclantern.fm/forum/index.php?topic=12396.msg119592#msg119592))

Some tips, until a more complete guide will be available:
- to load a SD card image, use something like: ./run_canon_fw.sh 60D -sd sd.img
- to display a trace of the firmware code, with disassembly, use: ./run_canon_fw.sh 60D -sd sd.img -d exec,int -singlestep
- there is a monitor console as well: ./run_canon_fw.sh 60D -sd sd.img -monitor stdio
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on May 25, 2015, 03:02:40 AM
good work!
a milestone, indeed :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 25, 2015, 06:50:34 AM
Thanks.

Quote from: mk11174 on April 24, 2015, 03:48:31 PM
Anyone know what this HDMI USB thing is [...] ?

That's Canon's HotPlug task reading those registers. I'll add an option to quiet them down ( -d ioport, as this one is standard in qemu).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nikfreak on May 25, 2015, 10:23:20 AM
Well done and cool stuff. I'll have a closer look once solved some 70D issues. The detailed sd command states from your posted log re-activated my interest in researching sd card related stuff (http://magiclantern.fm/forum/index.php?topic=12862.0). I would also like to test if I can re-produce the ML menu timeout bug now in QEMU.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dmilligan on May 30, 2015, 05:05:39 PM
Thanks for updating to qemu 2.3.0, I can get it to run now on OSX 10.10 (older qemu versions won't compile), well sort of. Things seem to boot:

K287 ICU Firmware Version 1.1.1 ( 3.3.7 )
[DebugMsg] (139,5)
ICU Release DateTime 2012.05.14 13:18:25
[DebugMsg] (0,3) [SEQ] CreateSequencer (Startup, Num = 6)
[DebugMsg] (0,2) [SEQ] NotifyComplete (Startup, Flag = 0x10000)
[DebugMsg] (0,3) [SEQ] NotifyComplete (Cur = 0, 0x10000, Flag = 0x10000)
[DebugMsg] (50,3) Magic Lantern Nightly.2015May30.60D111 (91e1ed550ec7 (qemu))
[DebugMsg] (50,3) Built on 2015-05-30 14:50:55 UTC by [email protected]
[*****] Starting task ff02b61c(3d2494) Startup
[DebugMsg] (0,5) [SEQ] seqEventDispatch (Startup, 0)
[GPIO]     at 0xFF012604     [0xC0222004] <- 0x12      : ???
[DebugMsg] (139,5) startupEntry
[*****] Starting task ff012584(0) Startup



[*****] Starting task 1fe07970(0) menu_task
[*****] Starting task 1fe09e24(0) menu_redraw_task
[*****] Starting task 1fe1ba88(0) bitrate_task
[*****] Starting task 1fe28180(0) focus_task
[*****] Starting task 1fe291a0(0) notifybox_task
[*****] Starting task 1fe2c124(0) fps_task
[*****] Starting task 1fe34244(0) shoot_task
[*****] Starting task 1fe2cd8c(0) clock_task
[*****] Starting task 1fe3b1bc(0) audio_common_task
[*****] Starting task 1fe43770(0) livev_hiprio_task
[*****] Starting task 1fe4245c(0) cls_task
[*****] Starting task 1fe452f0(0) beep_task
[*****] Starting task 1fe4d6d0(0) console_task
[*****] Starting task 1fe544dc(0) qemu_key_poll
[*****] Starting task 1fe0d7fc(0) debug_task
[*****] Starting task 1fe1d1e0(0) tweak_task



Load modules...
Linking..
Register modules...
Load configs...
Init modules...
Updating symbols...
  [i] 404: dual_iso_get_recovery_iso 1fe1fe00
  [i] 404: dual_iso_is_active 1fe1fe00
  [i] 404: auto_ettr_intervalometer_wait 1fe2cba0
  [i] 404: auto_ettr_intervalometer_warning 1fe2cba0
  [i] 404: auto_ettr_export_correction 1fe389d0
  [i] 404: mlv_snd_is_enabled 1fe3a8f0
  [i] 404: dual_iso_get_dr_improvement 1fe48da0
  [i] 404: dual_iso_get_recovery_iso 1fe48da0
Modules loaded


But I never get a screen image (the qemu window comes up, but it's just black), just this over and over again:


[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT
[GPIO]     at 0x1FE3B230     [0xC0220070] -> 0x1       : VIDEO CONNECT


I can send the whole log if it will help

I haven't tried booting Canon firmware yet, I need to get a hold of an SD card small enough I can make an image of it...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wolf on May 30, 2015, 09:04:11 PM
@dmilligan
I tried to load the lua module, but nothing did appear in the menu.
Anyway, the QEMU progress is impressive.

(http://i.imgur.com/vaTqI4H.png)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dmilligan on May 30, 2015, 10:39:18 PM
Did you have any scripts? There is no menu for lua itself, scripts can create or add their own menu items wherever they see fit. You can check the console for script loading/status/errors.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wolf on May 30, 2015, 10:51:26 PM
Quote from: dmilligan on May 30, 2015, 10:39:18 PM
Did you have any scripts? There is no menu for lua itself, scripts can create or add their own menu items wherever they see fit. You can check the console for script loading/status/errors.

Yes.
/qemu/sdcard/ML/SCRIPTS/{CALC.LUA,EDITOR.LUA,SOKOBAN.LUA}
It seems to me that QEMU doesn't "look" for the script folder.
I did run your lua module on my camera and it did work very well.

$grep SCRIPTS log.txt
[FIO wrapper] readdir() => SCRIPTS size=4096 mode=10 time=556420ee
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dmilligan on May 30, 2015, 11:01:19 PM
IDK, perhaps some FIO routines not emulated, or incomplete
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wolf on May 30, 2015, 11:27:32 PM
@ dmilligan
How did you compile QEMU version 2.3.0?
I've got 1.6.0.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dmilligan on May 31, 2015, 01:08:24 AM
The latest in the qemu branch uses 2.3.0. Just run the install script.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on August 19, 2015, 04:44:38 PM
Hello,

I'm trying to emulate 100D with qemu 2.3.0 and I'm running into some issues. I've got it to run quite far but there seems to be multiple issues and I'm not sure which one to try to fix next...

I've attached the log file for the longest I've got it running, and the major changes from what's in the repo is:

- Added addresses for 100D
- No boot flag, just canon stuff
- Asserts aren't fatal, they just print to serial and return to caller (GDB hack)
- DebugMsg prints to log (with color output, GDB/QEMU magic)
- Sequencer events are delayed if being propagated too early, to avoid errors (again some GDB magic)
- Added some SPI EEPROM code
- Grepped out the USB/HDMI/MIC clobber


So some questions:

Is there an SPI EEPROM in 60D?
The 100D firmware tries to load some properties from there and isn't happy with getting zeros. Not sure whether real data is necessary for the emulation, but it seems to be the reason why some asserts are raised.

There are some "unknown spells", should I try to figure out what they do? For the 60D, are the spell responses dumped from the camera or reversed from firmware?

It also complains about I2C and RTC at some points, but I don't think those are fatal errors.

Somehow it crashes to a $pc=4 state which I 'm trying to track down atm, but any thoughts on what to do to make it run further would be very appreciated. :)

This is the qemu log:
https://www.dropbox.com/s/y92sf7babtf3t2w/100D.log?dl=0
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nikfreak on November 10, 2015, 01:52:00 PM
a1ex if you read this:
May we expect an update to the qemu branch anytime soon? Sort of having issues like dmilligan encountered since you last updated qemu to 2.3.

Quote from: dmilligan on May 30, 2015, 05:05:39 PM
Thanks for updating to qemu 2.3.0, I can get it to run now on OSX 10.10 (older qemu versions won't compile), well sort of. Things seem to boot:
....
But I never get a screen image (the qemu window comes up, but it's just black), just this over and over again:

For me it just looks like this:
(http://i.imgur.com/P6uQbL8.jpg?1)

Qemu 1.6 worked great for 70D but with 2.3 I just get the black graphics shown as posted above whether I try 70D or 100D. For my use case it does the job for porting ROMs to see if they boot up but ofc I am hesitated to see the canon gui emulation which you presented on forums, too. The actual qemu branch also gives me a compile error in file "eos.c" at "eos_tio_handle" where I have to bypass "msg_arg1". So I just hope an updated branch might fix my problems or I simply am going to setup a new virtual machine with an older ubuntu release for qemu only. No rush...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nikfreak on December 28, 2015, 11:44:51 AM
Quote from: a1ex on May 24, 2015, 10:48:27 PM
Major progress: I'm able to launch Canon GUI under QEMU :)

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/qemu-canon-gui.png)

A1ex, how about an update to qemu? At it's actual state from bitbucket I always have to uncomment some lines to get qemu compiled. I just set-up a fresh 12.04 Ubuntu but had the exact same error on 14.04 Ubuntu. So quoting your post from May above I assume there's some local work on your build machine which needs to be merged:


/home/ml/qemu/qemu/qemu-2.3.0/hw/arm/eos.c:2716:9: error: variable 'msg_arg1' set but not used [-Werror=unused-but-set-variable]
cc1: all warnings being treated as errors
make[1]: *** [hw/arm/eos.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [subdir-arm-softmmu] Error 2


And once you read this: I would welcome the dmspy-experiments branch being merged into unified,too.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on February 26, 2016, 08:46:18 PM
Updated to run the Lua module on 60D.

(http://a1ex.magiclantern.fm/bleeding-edge/lua/qemu-lua-bug-scriptsource.png)

@nikfreak: I'm not getting this error about msg_arg1 (not even a warning), probably different cflags (I'm always (re-)installing it from contrib/qemu/install.sh script, but on an older Ubuntu). I've fixed that one anyway (it was, indeed, a bug).

@nkls: unfortunately, the MPU spells are valid only for 60D for now.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nikfreak on February 29, 2016, 12:30:53 PM
Really strange. As I didn't have the graphical glitch shown below in early days of 70D QEMU porting
Quote from: nikfreak on November 10, 2015, 01:52:00 PM
(http://i.imgur.com/P6uQbL8.jpg?1)

The only thing I can think of atm is a wrong conflicting merge of my 70D branch into qemu's as I already tried a new ubuntu installation and experience the same. Any clue when 70D might become part of unified and merged into qemu by you? Porting began Dec 2015 and I feel it's just about time to get it into unified...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on February 29, 2016, 01:09:54 PM
Yeah, the stuff in the QEMU branch expects Canon GUI to do all the display initializations (including color palette).

But that only works on 60D atm...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 28, 2016, 12:11:56 AM
Small progress: CF card emulation appears to work on 5D2 bootloader, enough to load autoexec.bin from a CF image file.

The 5D2 GUI isn't working yet though...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on May 11, 2016, 09:47:16 PM
I've pushed my changes to qemu to my bitbucket now, it's located at:

https://bitbucket.org/niklastisk/qemu (branches: eos-develop and 100D-testing)

It was a while since I touched the code but I hope parts of it can be useful to someone at least.

Things I recall I've done:
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 11, 2016, 10:13:25 PM
Very nice work! You seem pretty familiar with qemu internals (where I'm struggling).

I'll try to integrate it with the latest branch.

Regarding your questions (sorry, I somehow missed your post, so you probably got the answers meanwhile):

Quote from: nkls on August 19, 2015, 04:44:38 PM
Is there an SPI EEPROM in 60D?

There is an EEPROM that you can dump with this module (https://bitbucket.org/hudson/magic-lantern/commits/8c8e3329).

I didn't use this file in my emulation though.

Quote
There are some "unknown spells", should I try to figure out what they do? For the 60D, are the spell responses dumped from the camera or reversed from firmware?

They are taken from a startup log, from the dm-spy-experiments branch (CONFIG_DEBUG_INTERCEPT_STARTUP=y in Makefile.user), that contains calls to mpu_send/mpu_recv branch. The code is generated with this script (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/scripts/extract_init_spells.py).

They are quite incomplete - just enough to boot the main dialog. I know the buttons are also sent from MPU in the same way, and it's easy to get the pattern for them. I also figured out how to decode some properties, but but most other events are still a mystery to me.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on May 12, 2016, 09:58:12 PM
@nikls:
i am impressed and happy to see such rev.eng. skills :)
guessing right that you are using IDA?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on May 16, 2016, 04:50:26 PM
Thanks guys, and thanks for the info! I was trying to to get 100D working and I guess I learned a bit about the qemu internals on the way. I wouldn't call myself an expert on the system but I know my way around bigger C/linux projects like this. :)

I never realized the dm-spy-experiments dumped the mpu calls, I guess I have to make one of 100D when I get the time!

Yeah, I'm using IDA for the decompiling and code analysis.


About that serial flash though, I'm under the impression that it's only available in newer cameras (100D and 70D? (https://bitbucket.org/hudson/magic-lantern/issues/2535/70d-black-screen-menu#comment-27815246)), and that it is necessary to emulate them to get to the main dialog in qemu. But correct me if I'm wrong.

In 100D there are references to two flash chips PC28F128M29EWHF and K5C262GATMAF50 which are both 16MB iirc. There is a function "IsAddressSerialFlash" which is used by the property manager (or some flash access abstraction) to determine if the data is stored in flash rom (same as firmware) or serial flash rom (controlled over serial interface). I think I had it setup with DMA, interrupts and whatnot but I never made it to the main dialog. :( I'll give it another shot sometime but atm I'm too busy with other projects.

Here are some extracts from the factory menu, note the S and V options in FACTADJ and the SIO6 menu for accessing the serial rom.

FACTADJ

********** FACTORY ADJUSTMENT MENU VER 0.01 **********
0. Exit from Factory Adjustment
1. Leak Check
2. SDRAM Check
3. ROM Check
4. HDMI Check
5. Video Adjustment
6. ICU Version Check
7. Audio Check
8. Adjustment Data Display and Change
9. Check Flag Display and Initialization
A. ALL Check
B. ALLP Check
F. FROM ID Check
P. Power Domain Check
S. Serial Flash Check
V. Serial Flash Version Check
Z. Input_Device_Unique

S
Serial Flash Check.
346
Select OK or NG

V
SROM Version Check.
4.2.1
Select OK or NG


FROMUTILITY

************ FROMUTILITY MENU Ver 0.09 ************
[Type:346 Body:DC Revision:0.00]
0.Exit from FROM Menu
1.Erase Sector(0x20000)
2.Erase Chip
4.Write from card
5.Write from DRAM
6.Firm   flag 0xF8000000 0x00000000 ON
7.Boot   flag 0xF8000004 0x00000000 OFF
8.UpDate flag 0xF800000C 0xFFFFFFFF OFF
9.Create Boot Disk
A.Exec Program from card
B.ALL Block Unprotect
C.Connect card
G.Memory Dump
I.Write Data
J.Direct Jump
P.Power Domain
S.SROM Menu
U.Firm update
Z.RCbind.bin update

S

**** SROM(SIO6) Menu ****
0.Exit from SROM Menu
1.Erase Chip   0x01000000
2.Erase Block  0x0000F000
3.Erase Sector 0x00001000
4.Write Data
5.Write from Card
6.SROM Dump(SIO Read)
7.SROM Dump(QUAD Read)
8.Get Info
9.Get Version

8
8.Get Info 0x80000346

9
9.Get Version 4.2.1
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 16, 2016, 06:52:51 PM
Yes, experience with Linux definitely helps. I don't have that one either.

I tried to run your serial flash dumper on 5D3, but the serial flash appears unused here. There are no mentions of it in the startup log, and calling SF_CreateFlash locks up. Didn't try to debug further. The 5D3 firmware has these strings: PC28F128M29EWHF and MBM29LV640EBXEI70G.

There are plenty of serial flash messages on the 70D startup log, and from the errors, I think they are needed for emulation. Fortunately, the communication protocol doesn't seem too complicated.

70D startup log says:

[SF] InstallSerialFlash 4 0xc022002c 0x0 0x800000 1


and these strings are present: PC28F128M29EWHF and 64Mt8Kx8m64Kx126b8Kx8.




Quick question: when emulating the factory menu, how are you entering the input selection?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on May 16, 2016, 09:51:58 PM
The chip install code is run quite early in the init task, this is how it looks in IDA:
(http://i.imgur.com/0ENFsrm.png?1)

As you can see there is a flag (derived from FROM) which determines what chip to install. On a second thought, those routines must be for setting up writing to main flash.

The function call before "SerialFlash Initialize" is definitely for SROM though, I guess there would be some similar call in 70D but not in 5D3? Maybe the 5D3 has more space in main flash memory and thus no need for it?
In 100D the output is

[SF] InstallSerialFlash 6 0xc022c0d4 0xf0000000 0x1000000 1

so it's similar.

I think I've got most of the protocol working, take a look in this file in 100D-testing (https://bitbucket.org/niklastisk/qemu/src/ada7b6496bde7ebc6d7cc6b1557e9710156cc129/hw/eos/serial_flash.c?at=100D-testing&fileviewer=file-view-default). It gets messy once you try to emulate DMA since it is shared with the SD memory transfer.


Quick answer: (too much of a hack to push to bitbucket)

unsigned int eos_handle_tio ( unsigned int parm, EOSState *s, unsigned int address, unsigned char type, unsigned int value )
{
    unsigned int ret = 1;
    const char * msg = 0;
    int msg_arg1 = 0;

    switch(address & 0xFF)
    {
        case 0x00:
            if(type & MODE_WRITE)
            {
                if((value == 0x08 || value == 0x0A || value == 0x0D || (value >= 0x20 && value <= 0x7F)))
                {
                    printf("\x1B[31m%c\x1B[0m", value);
                    return 0;
                }
            }
            else
            {
                return 0;
            }
            break;

        case 0x04:
            msg = "Read byte: 0x%02X";
            msg_arg1 = s->tio_rxbyte & 0xFF;
            ret = s->tio_rxbyte & 0xFF;
            while (ret == 0 || ret == EOF) {
                ret = getchar();
            }
            return ret;
            break;
       
        case 0x08:
            /* quiet */
            return 0;

        case 0x14:
            if(type & MODE_WRITE)
            {
                if(value & 1)
                {
                    msg = "Reset RX indicator";
                    s->tio_rxbyte |= 0x100;
                    return 0; /* nkls: quiet */
                }
                else
                {
                    /* quiet */
                    return 0;
                }
            }
            else
            {
                if((s->tio_rxbyte & 0x100) == 0)
                {
                    msg = "Signalling RX indicator";
                    ret = 3;
                }
                else
                {
                    /* quiet */
                    return 3; /* nkls: signal something on the line */
                    //return 2;
                }
            }
            break;
    }

    io_log("TIO", s, address, type, value, ret, msg, msg_arg1, 0);
    return ret;
}

is there a hidden menu option (https://bitbucket.org/niklastisk/qemu/src/2d85ec1ee3a5feb3233183ee8180c8b10b04e5cd/hw/eos/eos_bufcon_100D.h?at=100D-testing&fileviewer=file-view-default) in other firmwares as well?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on May 17, 2016, 01:07:19 AM
one problem of qemu was, you couldnt emulate real flashes. (not SPI)
to write properties, the firmware is issuing flash state machine commands (writes to the flash base address)

QEMU cannot handle (passing writes to a driver) this. at least the version of 1.5 years ago wasn't able.
IIRC this has to do with the fact that the ARM instruction set isn't interpreted, but somewhat translated.

so it might be troublesome to get the whole camera hardware behave correctly using QEMU, if there is no workaround.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 19, 2016, 06:37:29 PM
@nkls: I think I've managed to integrate your code with my latest changes. It was a little difficult, and I'm not yet sure I did the merges right, so may I ask you to check whether it's still working fine on your side?

The DebugMsg hack is pretty cool - I just managed to boot the 60D GUI with full debug messages, without any additional guest code (no autoexec.bin loaded).

Didn't try the other stuff yet.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 21, 2016, 10:25:54 PM
Now that @nkls introduced me to the world of GDB scripting, here's a quick tip for tracing various functions around the firmware:


# task_create
br *0x98CC
commands
  silent
  printf "task_create(%s, prio=%x, stack=%x, entry=%x, arg=%x)\n", $r0, $r1, $r2, $r3, *(int*)$sp
  c
end

br *0xFF12CB14
commands
  silent
  printf "SearchFromProperty(%x,%x) from %x\n", $r0, $r1, $pc
  c
end


I also tried to implement the DebugMsg hook with pure gdb scripting, but I got stuck because gdb wants the exact number of arguments to the format string. For example, this works for a DebugMsg call with a single % in it, but fails otherwise.


  eval "printf \"[DebugMsg] (%%02x,%%02x) %s\n\", $r0, $r1, $r3", $r2


Any hints would be welcome.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on May 30, 2016, 10:26:16 PM
Nice to see things pushed to the main repo and that you've sorted out the open ends I left in there! It seems to work and looks correct, so I think it's all good.

I remember trying to get the debug messages showing through pure gdb for a while but then gave up and hacked it together in C instead. It might be possible to some extent but I bet it'll be ugly. Maybe count %-signs and branch to different cases?

The gdb-qemu link isn't ideal either, the current:
set *0xCF999001 = *0xCF999001
triggers the IO-handle when it reads from *0xCF999001 but not when it writes to it and I can't figure out why. Not that it matters much, but it would be nice to be able to send values to qemu through a single address.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 12, 2016, 12:47:44 PM
Quote from: nkls on May 30, 2016, 10:26:16 PM
Maybe count %-signs and branch to different cases?

Yes, that's exactly what I ended up with. I made a generic logging library for various functions around the firmware (tasks, semaphores, timers, interrupts, MPU communication), all in pure gdb:

https://bitbucket.org/hudson/magic-lantern/src/qemu-nkls/contrib/qemu/scripts/debug-logging.gdb

This file can be included in the model-specific GDB script, like this:
https://bitbucket.org/hudson/magic-lantern/src/qemu-nkls/contrib/qemu/scripts/5D3/debugmsg.gdb
https://bitbucket.org/hudson/magic-lantern/src/qemu-nkls/contrib/qemu/scripts/70D/debugmsg.gdb

and then get a log from both QEMU and GDB on the same terminal with a command like this:

./run_canon_fw.sh 5D3 -s -S & arm-none-eabi-gdb -x 5D3/debugmsg.gdb


The functionality is similar to the one in the dm-spy-experiments branch (http://www.magiclantern.fm/forum/index.php?topic=2388.msg113154#msg113154), but - as you pointed out - it works without changing the guest code or having to load autoexec.bin (for example, I could use it on 7D2).

Also, during the past two weeks (when I was without internet access), I made huge progress regarding emulation of 70D, 5D3, 7D2 and EOS M3. Canon GUI doesn't start yet, but it shouldn't be very far away.

I've also extended model_list to include other model-specific parameters, and made it easy to add new ones:

https://bitbucket.org/hudson/magic-lantern/src/qemu-nkls/contrib/qemu/eos/model_list.c

BTW, I suspect the serial flash contents are not fully correct. In your dumper, I changed the buffer to uncacheable (fio_malloc) and got slightly better results (property blocks were recognized), and I also had to do this change (https://bitbucket.org/hudson/magic-lantern/commits/097d66e6ba8da80736096feccb2184112a1ef603?at=qemu-nkls) when reading the serial flash contents via DMA. The data appears offset by half-byte, and on regular (non-DMA) reads, Canon code fixes it in the same way (in ReadBlockSerialFlash, for blocks smaller than 0x200). I guess the DMA engine is expected to apply the same "fix".

However, after these two changes, I still suspect data corruption (couldn't parse the property data structures completely - the first few properties are fine, and after a while it's pure gibberish, at least on the 70D SF dump from nikfreak). Guess we'll have to dump the RAM address where the serial flash contents is loaded (or maybe attempt to read the entire serial flash with a single call in the dumper).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Walter Schulz on June 12, 2016, 03:03:05 PM
Quote from: a1ex on June 12, 2016, 12:47:44 PM
Also, during the past two weeks (when I was without internet access), I made huge progress regarding emulation of 70D, 5D3, 7D2 and EOS M3.

EOS M3? Didn't see that coming ...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 12, 2016, 04:46:49 PM
Well, it's the only other DIGIC 6 camera for which I have a firmware (or did any other camera get a firmware update meanwhile?)

Sure, it's a PowerShot, but the DryOS core is the same, so I actually used it in order to understand the 7D2 code better. I only tried to emulate Canon firmware, didn't try to load any sort of custom code on it yet.

I'll post more details about the M3 on the relevant thread, since this emulation can also be useful for those who are porting CHDK on this camera.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on June 13, 2016, 10:57:16 PM
Wow, I'm impressed! Nice job! :) I won't be able to try it out in the upcoming two weeks, but I'll give it a go after that.

You are right about the sf data being manipulated in some way, I wondered what strange format would produce data like:

0000fff0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00010000: 0fff f000 0dcf 70f0 0000 0000 1d0f 70f0  ......p.......p.
00010010: 0000 0000 0040 3000 0000 0000 10c0 0000  .....@0.........
00010020: 0fff ffff f010 0000 10c0 0000 0fea fdcb  ................
00010030: a080 0000 1180 0000 0000 0000 0000 0000  ................
00010040: 0000 0000 0000 0000 0020 0000 1180 0000  ......... ......

but now that you say it it looks very half-byte shifted at 0x10000.

Interestingly, the first block contains the version string "4.2.1" which is not half-byte shifted.

00000000: 4603 0080 342e 322e 3100 0000 0000 0000  F...4.2.1.......
00000010: 0000 0000 ffff ffff ffff ffff ffff ffff  ................
00000020: ffff ffff ffff ffff ffff ffff ffff ffff  ................


Is it just me or is the write address not incremented in your fix (https://bitbucket.org/hudson/magic-lantern/commits/097d66e6ba8da80736096feccb2184112a1ef603?at=qemu-nkls)?

So wait, is the data still unshifted when you use uncacheable memory? What happens with the data block exactly -- is the first half-byte always set to zero, and the last half-byte discarded? Any indication of the higher-level calls requesting more data than asked for, or is the last byte of a read always assumed to be bogus? (note to self)

You could also try to force it to only use non-DMA/only-DMA with gdb and see if that works better, I recall getting different results when doing that.

Quote from: a1ex on June 12, 2016, 12:47:44 PM
Guess we'll have to dump the RAM address where the serial flash contents is loaded (or maybe attempt to read the entire serial flash with a single call in the dumper).
It'd be tricky to allocate 16MB(?) for the full flash wouldn't it? It would also be interesting to see how e.g. one 2kb read compares to two 1kb reads, maybe we just need to overlap the block reads by a few bytes and unshift them to make it work.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 14, 2016, 07:14:38 AM
Quote from: nkls on June 13, 2016, 10:57:16 PM
Is it just me or is the write address not incremented in your fix (https://bitbucket.org/hudson/magic-lantern/commits/097d66e6ba8da80736096feccb2184112a1ef603?at=qemu-nkls)?

Wow, great catch; it fixes a bunch of asserts in 70D log, regarding those properties located in serial flash.

Quote
So wait, is the data still unshifted when you use uncacheable memory?
Yes.

QuoteWhat happens with the data block exactly -- is the first half-byte always set to zero, and the last half-byte discarded?

Canon code does this:

if ( size < 0x200 )
{
      readSerialFlash(src, buf, 512);
      while ( offset < size )
      {
        *(_BYTE *)(out + offset) = 16 * *(_BYTE *)(buf + offset) | (*(_BYTE *)(buf + offset + 1) >> 4);
        ++offset;
      }
}
else
{
    v6 = readSerialFlashWithQuad(src, out, size);
}


Quote
It'd be tricky to allocate 16MB(?) for the full flash wouldn't it?

Nope, just try it.

(some background info here (http://www.magiclantern.fm/forum/index.php?topic=5071.msg166799#msg166799) and here (http://www.magiclantern.fm/forum/index.php?topic=8358.0))
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on June 19, 2016, 12:26:47 PM
I had another look at the flash memory yesterday, and I figured out a few things. Your code appears to load the serial flash correctly for the 100D without changes, and it might be that it's just the spells causing the property errors. (Still no canon gui.)

The chip in my camera is a Winbond W25Q128FV (datasheet (https://www.pjrc.com/teensy/W25Q128FV.pdf)), and not a Macronix as I thought before. The Manufacturer Code and device ID is {0xEF,0x40,0x18} which is what I got from SPI instruction 9Fh. This doesn't matter much since it works anyways, but it's good to have the actual data sheet to base the flash emulation of.

There are two high-level interfaces (with the same debug names) for the serial flash. One uses SPI-only transfer with "serial layout", while the other is "QUAD" read which reads data through DMA, or SPI if the read block is small enough. The DMA read has this shifted data layout, maybe due to physical wiring limitations or obfuscation or whatnot. The code you posted is used by the property manager, and is from the QUAD interface, which has to un-shift the SPI data to ensure proper data layout.

This explains why the version string in the first block is readable -- it is read and written by the SPI-only interface which don't expect DMA layout. I've used the address of the SPI-only interface in my dumper, so the data in the dump should be the same as in the actual flash.

I've also tried to change the block size, and there is no difference between the data received from a read-all-at-once dump file and a 1024-byte fio_malloc'ed dumper.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 19, 2016, 09:10:33 PM
Quote from: nkls on June 19, 2016, 12:26:47 PM
it might be that it's just the spells causing the property errors.

These should be easy to fix. Did you take the spells from another camera, or did you log the 100D startup with dm-spy-experiments?

Spells from another camera will cause issues.




BTW, I've included a small SD/CF image here (https://bitbucket.org/hudson/magic-lantern/commits/84b3f86ec6921eb65783764fa341801d88841209), hopefully this makes it easier to install and get started. The card image is bootable and includes a small autoexec.bin that does the display test (http://www.magiclantern.fm/forum/index.php?topic=14732.0), as this runs on most (if not all) DIGIC 4/5 models out of the box.

That means: run the install script, compile QEMU, copy your ROM from the ML card and it should be ready to go.

Still, it won't display the GUI on cameras other than 60D...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 20, 2016, 11:28:05 AM
Okay, I know you won't believe this one.

After about 2 weeks of intensive work on QEMU, I couldn't manage to get 70D and 5D3 working.

Today, after about half an hour of tinkering... 1200D Canon GUI boots.... with the MPU SPELLS from 60D!

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/1200D-qemu.png)

Emulation log: 1200D-qemu.log (http://a1ex.magiclantern.fm/bleeding-edge/qemu/1200D-qemu.log)

These were the changes I did to QEMU for 1200D: [1] (https://bitbucket.org/hudson/magic-lantern/commits/7241ce6bec50947082bcec05ad5078cf963baf08) [2] (https://bitbucket.org/hudson/magic-lantern/commits/507c96ed43b51c80d7776d70ddd12d00e366b0a5) [3] (https://bitbucket.org/hudson/magic-lantern/commits/008aaa6fa261beb45a27bdd08de919dbd3724fcb). And a GDB script to help me see what's going on: [4] (https://bitbucket.org/hudson/magic-lantern/commits/60225f0e05c09b551aeaef001237723c73f96668).

I tried the 60D spells just to see how it goes, but I didn't hope it would go that far.

(side note: currently we can see the MPU spells only after ML is ported, as they come from a secondary CPU whose firmware is pretty much impossible to understand, at least for me)

At this point, I think it's worth trying most other DIGIC 4 cameras with SD card. They probably require a MPU spell log (easy to get, as described here (http://magiclantern.fm/forum/index.php?topic=2864.msg166938#msg166938)), and probably a few other minor tweaks. Happy hacking!

P.S. looks like it took me more time to write this message, than to actually get the 1200D GUI booting :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 20, 2016, 07:59:02 PM
Some more:

1100D also ran out of the box with 60D spells:
(http://a1ex.magiclantern.fm/bleeding-edge/qemu/1100D-qemu.png)

550D is stubborn; I ended up with this after patching lvInit (not yet committed):
(http://a1ex.magiclantern.fm/bleeding-edge/qemu/550D-qemu.png)

600D needs its own MPU spell set (most likely because of the crop video mode settings, which are stored in the MPU), but I expect it to work without much trouble.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: mathias on June 24, 2016, 12:02:04 AM
alex, I am having a lot of errors like

/home/matias/qemu/qemu-2.5.0/hw/arm/../eos/eos_handle_serial_flash.c:38:5: error: 'for' loop initial declarations are only allowed in C99 mode
/home/matias/qemu/qemu-2.5.0/hw/arm/../eos/eos_handle_serial_flash.c:38:5: note: use option -std=c99 or -std=gnu99 to compile your code

I've fixed some, but seems that some config is missing, or i am wrong?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 24, 2016, 03:38:14 PM
Interesting; I used to have C99 errors before, but they were no longer present after upgrading to QEMU 2.5.0 and Ubuntu 15.10. I thought QEMU devs enabled C99 in newer versions, so I started using C99 constructs in my code.

Looks like the answer is here:

https://gcc.gnu.org/onlinedocs/gcc-4.8.0/gcc/Standards.html
Quote
The default, if no C language dialect options are given, is -std=gnu90

https://gcc.gnu.org/onlinedocs/gcc-5.2.0/gcc/Standards.html
Quote
The default, if no C language dialect options are given, is -std=gnu11

Try adding -std=gnu99 to CFLAGS in your QEMU Makefile, or when configuring it. For example:


/path/to/ml/qemu/qemu-2.5.0$ CFLAGS=-std=gnu99 ../configure_eos.sh


(this should be included in the installer, actually)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: mathias on June 24, 2016, 07:09:04 PM
Great,

I was able to run 1200D with Hello world in qemu 2.5.0
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 24, 2016, 11:48:28 PM
Interesting, after trying a couple of times (and getting different execution logs every time) I finally got your hello world running as well.

Looks like there is a race condition somewhere in the emulation.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: mathias on June 29, 2016, 05:50:04 AM
I don't know why but it's throwing an error if i remove hello world from define. the error is not clear (at least for my)

Error loading 'ML/MODULES/1200D_100.sym': File does not exist
while in the emulator I see some error like symbols not found (dissapears fast), if I hit del key screen goes black.

but tried in 1.6 version, no error and del key givesme ML menu.

I don't know how did you do to display GUI in qemu
(before testing port in my camera I always try to see it working in qemu)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 29, 2016, 07:31:07 AM
If the file does exist, ML cannot load it because its name is longer than 8 characters.

Were you able to get the GUI in QEMU 2.5.0 like this?

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/1200D-hello-qemu.png)

For me, it doesn't work every time; I have to start QEMU a couple of times to get this screen. This only happens when loading autoexec.bin; if I try to run plain Canon firmware (with bootflag disabled), it works every time.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nikfreak on June 29, 2016, 07:47:20 AM
Quote from: mathias on June 29, 2016, 05:50:04 AM
Error loading 'ML/MODULES/1200D_100.sym': File does not exist
while in the emulator I see some error like symbols not found (dissapears fast), if I hit del key screen goes black.

If 8.3 filenaming applies to qemu, too then you got your answer now: shorten to "x70_100.sym" or something like that.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 29, 2016, 12:23:03 PM
Quote from: nikfreak on June 29, 2016, 07:47:20 AM
If 8.3 filenaming applies to qemu

QEMU actually emulates a SD card device, with all the low-level communication (including DMA transfers), so filesystem behavior should match Canon's. For example, if you start the emulation on a formatted SD image, you will see Canon firmware creating the DCIM and MISC folders.

I expect formatting from Canon menu should work as well, just didn't try it. This needs the MPU spells (button codes and whatever other GUI events there might be) for navigating Canon menu.

For CF cards, emulation currently works properly in the bootloader (e.g. loading autoexec.bin from card), but not in the main firmware.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: mathias on June 30, 2016, 04:35:33 AM
Well tried to reinstall qemu (in case my fail) but same errors appears.
using 1200D branch, if I run it with HELLO_WORLD I get this: (no cannon GUI)
(https://s31.postimg.org/t6cdd17cn/Screenshot_from_2016_06_29_19_05_48.png) (https://postimg.org/image/t6cdd17cn/)

if i remove it i get file not found as I said before. (Double checked and the file exist, this model 1200D only has SD card so i am just mounting sd, notice the file in the screenshot)

(https://s31.postimg.org/rp8dhtvqf/Screenshot_from_2016_06_29_19_19_56.png) (https://postimg.org/image/rp8dhtvqf/)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 30, 2016, 06:45:11 AM
Copy all ML files (make install), not just autoexec ;)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 30, 2016, 06:17:01 PM
600D is just as stubborn as 550D, but easier to debug (more recent codebase).

With proper MPU spells, it gives the date/time screen, just like 550D.

With a small patch on the RTC init routine, it gives the sensor cleaning animation (real-time!)

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/600D-qemu-datetime.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/600D-qemu-sensorcleaning.png)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on July 01, 2016, 05:35:51 PM
500D :
0xFF18A884 - mpu_send
0xFF05C1F0 - mpu_recv
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 08, 2016, 10:58:19 PM
Some progress understanding MPU messages:

Button codes

They are encoded like this:

0x06, 0x05, 0x06, 0x00, btn_code, btn_code_arg


The button codes can be found from bindReceiveSwitch - this translates the MPU button codes (btn_code, btn_code_arg) into GUI button codes as used by GuiMainTask (the BGMT constants from gui.h). When the GUI button codes are sent to GuiMainTask, they appear in the debug log as "GUI_Control:%d 0x%x". When they are actually processed, they appear as "GUI_CONTROL:%d".

Since finding these button codes for each camera would be incredibly boring, I wrote a Python script (https://bitbucket.org/hudson/magic-lantern/commits/d40c4d99e99351e3b6ee77933885dee9f85977cb#chg-contrib/qemu/eos/mpu_spells/extract_button_codes.py) to get them automatically from the ROM, by directly emulating bindReceiveSwitch in unicorn (http://www.unicorn-engine.org), trying all usual input values and checking debug messages.

btn_code_arg meaning can be: press/unpress (1,0), scrollwheel direction (1, -1) and number of steps for very fast turns (2, -3 etc), or some buttons can be grouped under a single btn_code (for example, the direction pad).

Many button codes are common across all cameras (DIGIC 4 and 5): MENU (0,1), INFO (1,1), PLAY (3,1), DELETE (4,1), SET (12,1/0), scrollwheels (13 and 14), others are not.

btn_code = 30 is ServiceMenu on all cameras. Interesting string on 1200D: "Enter Secret mode Electric Shutter!!!".

Some cameras also use a generic event, GUICMD_PRESS_BUTTON_SOMETHING, which I don't know how to interpret (other than some button was pressed).

Side note: this research uncovered a few subtle bugs regarding button codes on 600D, 100D and EOS M, and not-so-subtle on 1100D (see this PR (https://bitbucket.org/hudson/magic-lantern/pull-requests/743/misc-cleanups-some-gui-button-codes/diff)).

GUI modes

On 600D, menu navigation looks somewhat like this ("spell" being data sent from ICU to MPU, and "reply" being the response from MPU):

    { 0x06, 0x05, 0x03, 0x19, 0x00, 0x00 }, {                   /* spell #44 */
        { 0x06, 0x05, 0x03, 0x17, 0x9a, 0x00 },                 /* reply #44.1 */
        { 0x06, 0x05, 0x06, 0x26, 0x01, 0x00 },                 /* reply #44.2, GUI_Control:76, bindReceiveSwitch(38, 1) */
        { 0x06, 0x05, 0x06, 0x00, 0x01, 0x00 },                 /* reply #44.3, BGMT_MENU, GUI_Control:6, bindReceiveSwitch(0, 1) */
        { 0x06, 0x05, 0x04, 0x0d, 0x00, 0x00 },                 /* reply #44.4 */
        { 0 } } }, {
    { 0x06, 0x05, 0x03, 0x19, 0x00, 0x00 }, {                   /* spell #45 */
        { 0x06, 0x05, 0x06, 0x26, 0x01, 0x00 },                 /* reply #45.1, GUI_Control:76, bindReceiveSwitch(38, 1) */
        { 0x06, 0x05, 0x06, 0x00, 0x01, 0x00 },                 /* reply #45.2, BGMT_MENU, GUI_Control:6, bindReceiveSwitch(0, 1) */
        { 0 } } }, {
    { 0x06, 0x05, 0x04, 0x00, 0x01, 0x00 }, {                   /* spell #46, NotifyGUIEvent(1) */
        { 0x06, 0x05, 0x06, 0x0a, 0x00, 0x00 },                 /* reply #46.1, BGMT_UNPRESS_ZOOMOUT_MAYBE, GUI_Control:17, bindReceiveSwitch(10, 0) */
        { 0x06, 0x05, 0x06, 0x09, 0x00, 0x00 },                 /* reply #46.2, BGMT_UNPRESS_ZOOMIN_MAYBE, GUI_Control:15, bindReceiveSwitch(9, 0) */
        { 0x06, 0x05, 0x04, 0x00, 0x01, 0x01 },                 /* reply #46.3 */
        { 0x0e, 0x0c, 0x0a, 0x08, 0x11, 0x00, 0x15, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00 },/* reply #46.4 */
        { 0 } } }, {
    { 0x08, 0x06, 0x00, 0x00, 0x04, 0x00, 0x00 }, {             /* spell #47, Complete WaitID = 0x80020000 */
        { 0 } } }, {
    { 0x06, 0x05, 0x03, 0x34, 0x00, 0x00 }, {                   /* spell #48 */
        { 0 } } }, {
    { 0x06, 0x05, 0x03, 0x19, 0x00, 0x00 }, {                   /* spell #49 */
        { 0x06, 0x05, 0x06, 0x26, 0x01, 0x00 },                 /* reply #49.1, GUI_Control:76, bindReceiveSwitch(38, 1) */
        { 0x06, 0x05, 0x06, 0x1a, 0x01, 0x00 },                 /* reply #49.2, BGMT_PRESS_RIGHT, GUI_Control:35, bindReceiveSwitch(26, 1) */
        { 0x06, 0x05, 0x06, 0x1a, 0x00, 0x00 },                 /* reply #49.3, BGMT_UNPRESS_RIGHT, GUI_Control:36, bindReceiveSwitch(26, 0) */
        { 0x06, 0x05, 0x06, 0x26, 0x01, 0x00 },                 /* reply #49.4, GUI_Control:76, bindReceiveSwitch(38, 1) */
        { 0x06, 0x05, 0x06, 0x1a, 0x01, 0x00 },                 /* reply #49.5, BGMT_PRESS_RIGHT, GUI_Control:35, bindReceiveSwitch(26, 1) */
        { 0x06, 0x05, 0x06, 0x1a, 0x00, 0x00 },                 /* reply #49.6, BGMT_UNPRESS_RIGHT, GUI_Control:36, bindReceiveSwitch(26, 0) */


NotifyGUIEvent (called by SetGUIRequestMode) sends a message like this:

0x06, 0x05, 0x04, 0x00, event_code, 0x00


The MPU is supposed to reply something, probably this:

0x06, 0x05, 0x04, 0x00, event_code, 0x01


Now the interesting part: if I enable the NotifyGUIEvent reply on 60D, and the other cameras that accept the same MPU spell set, they no longer boot the GUI: instead, they go to the date/time dialog. You can adjust the date/time in QEMU using the arrow keys, scrollwheels, and the spacebar (SET), but when pressing OK, the GUI freezes.

Obviously, with NotifyGUIEvent disabled, the GUI mode can't be changed (so you can't enter Canon menu, or playback mode, or whatever).

The question is: how to make the GUI mode switches work, so one can navigate the menu?




Note: with current implementation, you can already navigate ML menu without CONFIG_QEMU=y... if you define GUIMODE_ML_MENU = 0 (so it won't try to change the GUI mode, because that part doesn't work). One big step closer towards running unmodified ML in QEMU :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on July 18, 2016, 01:58:19 AM
500D :
(https://s31.postimg.org/aa32bvxff/500_D.png) (https://s32.postimg.org/teuuqtlvp/500d.gif) (https://s31.postimg.org/vxp1gewpn/500d1.png) (https://s31.postimg.org/5qnukgefv/500d2.png) (https://s31.postimg.org/ok9nhgcnv/500d3.png)
# ./run_canon_fw.sh 500D -s -S & arm-none-eabi-gdb -x 500D/debugmsg.gdb

source -v debug-logging.gdb

macro define CURRENT_TASK 0x1A74
macro define CURRENT_ISR  (*(int*)0x664 ? (*(int*)0x668) >> 2 : 0)

b *0xFF066A98
DebugMsg_log

b *0xFF069E2C
task_create_log

b *0xFF064520
load_default_date_time_log
macro define RTC_VALID_FLAG (*(int*)0x2BC4)

cont
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: budafilms on July 21, 2016, 12:09:46 PM
Hi everybody,
an optimistic question from someone without skills to use this: wich is the real utility/advantage of this?

(I mean, for example, more memory, more resolution, new language for coding, girls...)

Thanks!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Walter Schulz on July 21, 2016, 12:53:02 PM
Testing code without cam:
- Cam will not brick
- You have not to wear out gear (card, slot, cardreader, etc.) each time you have to replace binaries.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 21, 2016, 01:33:09 PM
Quote from: Walter Schulz on July 21, 2016, 12:53:02 PM
Testing code without cam:
- Cam will not brick
- You have not to wear out gear (card, slot, cardreader, etc.) each time you have to replace binaries.

- debugging your code like a PC program, by running it step by step (not just with printf's)
- you can debug Canon code as well, in order to understand what it does
- possible to implement automated tests for the nightly builds (see also http://magiclantern.fm/forum/index.php?topic=12396.0 )
- test Lua scripts in the emulator
- lower the entry barrier for new developers
- assist me in unbricking cameras (the main driving force behind this, sadly)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on July 21, 2016, 03:22:31 PM
500D, LV
mpu_send(06 04 09 00 00)
mpu_recv(3c 3a 09 00 3c 3c e0 00 3f 80 00 00 38 12 c0 00 b9 cb c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 08 11 10 50 49 02 59 88 88 00 32 00 00 00 00 00 01 00 00 00)
PROP_LV_LENS


0x32 - focal length
0x10 - aperture


Now we can read lens_info in Photo mode.
Just call mpu_send(06 04 09 00 00). CPU receives data and automatically overwrite property lens_info.

(https://s32.postimg.cc/3kk7nrbqd/500d.jpg) (https://s32.postimg.cc/hfii6865h/500d2.jpg)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 22, 2016, 11:27:57 AM
Nice trick.

Let's continue the discussion regarding MPU communication here: http://www.magiclantern.fm/forum/index.php?topic=17596
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on July 29, 2016, 10:33:43 AM
I've managed to get the dm-spy-extra branch to run for 100D (fw 100A) with full logging, sources are at:
https://bitbucket.org/niklastisk/magic-lantern/branch/100D-mpu-spy

Still no success at emulating the 100D though, just including the correct mpu messages did not work out.


On a side note, me and a1ex discovered some interesting things about the memory management of the 100D's 256M RAM when using the CONFIG_MARK_UNUSED_MEMORY_AT_STARTUP configuration:

      p1=&memtest; p3=UNCACHEABLE(p1);
      init:00b1c75c:00:00: POINTER   p1         p2         p3
      init:00b1c77c:00:00: ADDRESS   0x00B7F204 0x10B7F204 0x40B7F204
      init:00b1c7a8:00:00: (init)    0x00000123 0x00000123 0x00000123
      init:00b1c7e0:00:00: memtest++ 0x00000124 0x00000123 0x00000123
      init:00b1c818:00:00: (*p1)++   0x00000125 0x00000123 0x00000123
      init:00b1c850:00:00: (*p2)++   0x00000125 0x00000123 0x00000123
      init:00b1c888:00:00: (*p3)++   0x00000125 0x00000124 0x00000124

(source code: http://pastebin.com/vHWcH9ij)

My interpretation is that (0x10000000 | ptr) creates a pointer that wraps around the 256M ram, but bypasses the cache and is read-only such that write attempts fail silently. This obviously won't work for 512M cameras. :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on August 01, 2016, 11:04:47 AM
I've fixed the serial flash reads, turns out the QUAD mode discards the last 16 bytes in each 0x800-byte block. Now the property tables are read correctly.

https://bitbucket.org/niklastisk/qemu/branch/eos-develop-new

The spell sets for 100D are included as well, and I even got the GUI showing (once)! There still seems to be problems with the touch panel, but I don't know if they are critical or not.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 10, 2016, 11:54:09 PM
Just committed some updates, mostly written while I had limited network connectivity (which is why they weren't uploaded earlier):

- directory layout changed, you need to move the ROM files (see install script)
- 5D3 boots the GUI on 1.1.3 :)
- cache hack emulation (very incomplete, only tested on 60D dm-spy, which runs successfully and saves its log on the SD card image)
- SD emulation fix (card info is now read correctly and gets mounted on digic 5 models)
- experimental EDMAC emulation (didn't really play with it, but can be used to research the picture taking process, bit depth reduction tricks, eeko and lots of other low level stuff)
- initial support for 750D and 760D (nothing interesting, they crash in the same way as 80D)
- integrated nkls' changes about 100D, but not yet tested (they will certainly be helpful for 70D and most other recent cameras)

I'm also starting to get confident about emulating the TX19A in QEMU, thanks to leegong's findings, but I have nothing interesting to show yet.

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 26, 2016, 01:07:16 AM
Small updates:

- card LED emulation
- better SD emulation on DIGIC 6 (tested on EOS M3 - it loads DISKBOOT.BIN (https://chdk.setepontos.com/index.php?topic=12542.msg130078#msg130078) from the SD card image)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 26, 2016, 11:52:52 PM
We now have a test suite (https://bitbucket.org/hudson/magic-lantern/commits/99d05c437b26edd515342b03528c183f0f437e4c) for QEMU :)


./run_tests.sh
Compiling...

Testing bootloaders...
  5D2: K218 READY
  5D3: K285 READY
   6D: K302 READY
   7D: FAILED!
7D2M: K289M READY
7D2S: K289S READY
  50D: FAILED!
  60D: K287 READY
  70D: K325 READY
  80D: K350 READY
500D: K252 READY
550D: K270 READY
600D: K286 READY
650D: K301 READY
700D: K326 READY
750D: K393 READY
760D: K347 READY
100D: K346 READY
1100D: K288 READY
1200D: K327 READY
EOSM: K331 READY

Testing display from bootloader...
Setting up a temporary SD card image...
'../magic-lantern/contrib/qemu/sd.img.xz' -> './sd.img.xz'
  5D2: OK
  5D3: OK
   6D: OK
   7D: please check
7D2M: OK
7D2S: OK
  50D: please check
  60D: OK
  70D: OK
  80D: OK
500D: OK
550D: OK
600D: OK
650D: OK
700D: OK
750D: OK
760D: OK
100D: OK
1100D: OK
1200D: OK
EOSM: OK
Restoring your SD card image...

Testing Canon GUI...
  60D: OK
  5D3: OK
600D: OK
1200D: OK
1100D: OK


Can you recommend some testing framework that could work in this case? I feel like I'm reinventing the wheel with all those scripts, but at least it works. For now.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 30, 2016, 12:03:45 AM
Added support for a few more cameras (they all load autexec and run the display test):
- 5D, 40D, 400D, 450D, 1000D (oldies)
- 50D (minor fix)
- 5D4

This one just boots DryOS and starts a few tasks:
- A1100 (PowerShot)

Current test results:

Testing bootloaders...
   5D: ROM READY
  5D2: K218 READY
  5D3: K285 READY
  5D4: K349 READY
   6D: K302 READY
   7D: FAILED!
7D2M: K289M READY
7D2S: K289S READY
  40D: K190 READY : Ver 4.0.1
  50D: K261 READY
  60D: K287 READY
  70D: K325 READY
  80D: K350 READY
400D: InitializeIntercom
450D: K176 READY : Ver 4.0.3
500D: K252 READY
550D: K270 READY
600D: K286 READY
650D: K301 READY
700D: K326 READY
750D: K393 READY
760D: K347 READY
100D: K346 READY
1000D: K254 READY : Ver 3.7.5
1100D: K288 READY
1200D: K327 READY
EOSM: K331 READY

Setting up temporary SD/CF card images...
'../magic-lantern/contrib/qemu/sd.img.xz' -> './sd.img.xz'

Testing display from bootloader...
   5D: OK
  5D2: OK
  5D3: OK
  5D4: OK
   6D: OK
   7D: please check
7D2M: OK
7D2S: OK
  40D: OK
  50D: OK
  60D: OK
  70D: OK
  80D: OK
400D: OK
450D: OK
500D: OK
550D: OK
600D: OK
650D: OK
700D: OK
750D: OK
760D: OK
100D: OK
1000D: OK
1100D: OK
1200D: OK
EOSM: OK

Testing EOS M3...
SD boot: StartDiskboot
Display: TurnOnDisplay

Restoring your SD/CF card images...

Testing Canon GUI...
  60D: OK
  5D3: OK
600D: OK
1200D: OK
1100D: OK


Hope all of this will be useful to other developers  8)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wildstray on October 21, 2016, 09:24:29 PM
Hello, I'm new to ML, I'm trying it on an EOSM. I wish to develop a module, but it would be really difficult using the "physical" camera. So I'm trying to setup an emulation environment... unluckily I miss something... I'm stuck on qemu... I need some help.

/opt/qemu/bin/qemu-system-arm -machine EOSM
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 3FFFFFFF: eos.ram
40001000 - 7FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror_F1
F2000000 - F2FFFFFF: eos.rom0_mirror_F2
F3000000 - F3FFFFFF: eos.rom0_mirror_F3
F4000000 - F4FFFFFF: eos.rom0_mirror_F4
F5000000 - F5FFFFFF: eos.rom0_mirror_F5
F6000000 - F6FFFFFF: eos.rom0_mirror_F6
F7000000 - F7FFFFFF: eos.rom0_mirror_F7
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror_F9
FA000000 - FAFFFFFF: eos.rom1_mirror_FA
FB000000 - FBFFFFFF: eos.rom1_mirror_FB
FC000000 - FCFFFFFF: eos.rom1_mirror_FC
FD000000 - FDFFFFFF: eos.rom1_mirror_FD
FE000000 - FEFFFFFF: eos.rom1_mirror_FE
FF000000 - FFFFFFFF: eos.rom1_mirror_FF
C0000000 - CFFFFFFF: eos.iomem
[EOS] loading 'ROM-EOSM.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading 'ROM-EOSM.BIN' to 0xF8000000-0xF8FFFFFF (offset 0x1000000)
FIXME: no MPU spells for EOSM.

Available keys:
- Arrow keys   : Navigation
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- M            : MENU
- P            : PLAY
- I            : INFO/DISP
- L            : LiveView
- F1           : show this help

VNC server running on '127.0.0.1;5900'


I know that without MPU spells I cannot display Canon menu. But the only think I can display in VNC is the garbled screen. How to load ML?
I compiled EOSM platform with CONFIG_QEMU=y and I made qemu-helper.bin.

(https://s4.postimg.org/qvvqp774d/desktop10.png)

This is my directory setup (all uppercase as required?):


├── ROM-EOSM.BIN
└── sdcard
     ├── AUTOEXEC.BIN
     ├── ML
     │   ├── CROPMKS
     │   │   ├── CINESCO2.BMP
     │   │   ├── CRSSMTR2.BMP
     │   │   ├── PASSPORT.BMP
     │   │   ├── PHIPHOTO.BMP
     │   │   └── PHIVIDEO.BMP
     │   ├── DATA
     │   │   ├── APSC8P.LUT
     │   │   ├── APSC8R.LUT
     │   │   ├── FF8P.LUT
     │   │   └── FF8R.LUT
     │   ├── DOC
     │   ├── DOCS
     │   ├── FONTS
     │   │   ├── ARGHLF22.RBF
     │   │   ├── ARGNOR23.RBF
     │   │   ├── ARGNOR28.RBF
     │   │   ├── ARGNOR32.RBF
     │   │   ├── TERM12.RBF
     │   │   └── TERM20.RBF
     │   ├── MODULES
     │   │   ├── ADV_INT.MO
     │   │   ├── ARKANOID.MO
     │   │   ├── AUTOEXPO.MO
     │   │   ├── BENCH.MO
     │   │   ├── DEFLICK.MO
     │   │   ├── DUAL_ISO.MO
     │   │   ├── EOSM_202.SYM
     │   │   ├── ETTR.MO
     │   │   ├── FILE_MAN.MO
     │   │   ├── LUA.MO
     │   │   ├── MLV_PLAY.MO
     │   │   ├── MLV_REC.MO
     │   │   ├── MLV_SND.MO
     │   │   ├── PANO.MO
     │   │   ├── PIC_VIEW.MO
     │   │   ├── RAW_REC.MO
     │   │   ├── SELFTEST.MO
     │   │   └── SILENT.MO
     │   ├── README
     │   └── SCRIPTS
     ├── MAGICLANTERN
     ├── ML-SETUP.FIR
     └── QEMU-HELPER.BIN


Qemu 2.5.50 from niklastisk repo, branch eos-develop-new (./configure --disable-docs --target-list=arm-softmmu).

...anyway, I'm asking myself: is there a way to display ML menu at startup in the emulator? Otherwise it would be a bit difficult to recall it on EOSM... on the real camera I use double touch on the screen...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wildstray on October 21, 2016, 09:51:07 PM
PS: I had to comment out a code fragment from eos_init_common(). Otherwise, after loading ROM-EOSM.BIN, qemu exits with a "CF init failed".

    /* init CF card */
    DriveInfo *dj;
    dj = drive_get_next(IF_IDE);
    if (!dj) {
        printf("CF init failed\n");
        exit(1);
    }

    ide_bus_new(&s->cf.bus, sizeof(s->cf.bus), NULL, 0, 2);
    ide_init2(&s->cf.bus, s->interrupt);
    ide_create_drive(&s->cf.bus, 0, dj);


EOSM doesn't have CF. It fails also if I create a cfcard directory. And now I have the dubt that sdcard and cfcard directory are totally ignored...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 22, 2016, 01:37:07 PM
For now, I recommend working on a model that can display Canon menu (at the time of writing, 60D, 5D3, 1100D or 1200D).

There are two qemu branches, with different design goals:

- the one in unified, which only attempts to run some ML code:
   - it displays the ML menu for most models
   - requires several changes to ML code (lots of hacks - grep for CONFIG_QEMU)
   - runs only a tiny part of Canon code (some startup code, task scheduler and... that's pretty much it)
   - there is a filesystem emulation, but it's extremely hackish: ML file I/O calls are replaced with wrapped calls to the host system (in the sdcard/cfcard directories)
   - it also runs some LiveView and HDMI tests (also very hackish)
   - back then, emulating Canon GUI was a pipe dream
   - I don't remember touching it in the last 1-2 years, so it should probably go away.

- the one in the qemu branch, which attempts to run the complete Canon firmware and unmodified autoexec.bin:
   - ML menu only works on those models that can boot Canon GUI
   - to navigate it easily, define GUIMODE_ML_MENU as 0 in consts.h (that's because GUI mode switching doesn't work yet)
   - patching ML code is optional (it can load unmodified autoexec.bin files, but patching can still be useful)
   - storage emulation uses a SD or CF card image (a single big file)
   - SD/CF emulation works by emulating the low-level calls (Identify Drive, read/write sector, stuff like that)
   - bootloader emulation works on nearly all models (from DIGIC 2 to DIGIC 6), including loading autoexec.bin from the card image
   - file I/O calls from both Canon and ML code are expected to work well on SD (e.g. loading ML modules, saving settings, creating the DCIM and MISC directories from Canon code)
   - CF emulation is not very solid (there are nondeterministic errors)
   - changing most Canon settings (properties) doesn't work (it requires talking to MPU)
   - LiveView and HDMI tests are very hard to run (it requires hacking some very hackish code)
   - you can debug it and trace various calls in Canon firmware using gdb (very useful for reverse engineering)

Basically, that's the current state of the emulation. Not exactly straightforward for new developers, but slowly getting there.

It also serves as "executable documentation" for a very large part of our knowledge about Canon code.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wildstray on October 22, 2016, 02:17:48 PM
Thank for the clarifications, A1ex!

Ok, I could works indifferently on an emulated 60D, 5D3 or so. But the only ROM I have is the one dumped from my camera, so firstly I need to obtain another ROM (how?).

About ML branches, you recommend to use qemu branch for new developers? If it would be more reliable I'll use it. Actually, I wanted to use unified because of SD emulation wrapping on host fs... it would be simpler and more agile to work in ML directory without mounting and unmounting an image file.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 22, 2016, 03:28:18 PM
I think the newer branch is a lot more robust and the emulation is much closer to the real thing.

For mounting/unmounting, it should be similar to a real SD card: once you have set up your SD card image with kpartx (see mount.sh), you click on EOS_DIGITAL in your file browser, then you run "make install" from your camera or module directory, then you start QEMU. You can automate this as well, if you like.

FYI, the install script sets up an initial SD card image, cloned from a 256MB card, which is already bootable and contains the portable display test autoexec.bin (handy to test your initial installation).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: wildstray on October 22, 2016, 10:07:04 PM
Yeah! :D After several peregrinations between repositories and patches, and thanks to install.sh and run_canon_fw.sh scripts and the suggestions of a1ex, now I have a semi-working qemu environment. After seconds, I got the Canon menu...

(https://s17.postimg.org/4e4xu2dxr/desktop11.png)

But this is the maximum I can do. When I press a key, I can see only these warnings in qemu console...


Warning: no scancode found for keysym 112
Warning: no scancode found for keysym 112
Warning: no scancode found for keysym 112
Warning: no scancode found for keysym 112


Did I miss something about keycodes?

PS: I tried to run 60D and EOS ROMs... I seen the Magic Lantern Rescue screen for both. But the sd card image I prepared with ML installation is 60D.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 22, 2016, 10:19:37 PM
Looks fine (I get the same screen here).

The scancode error is internal to qemu; maybe it doesn't like some keyboard layouts?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 23, 2016, 12:47:28 AM
BTW, some small updates:

- initial support for 5D4 AECU aka K349AE (a secondary core, also ARM with DryOS)
- a test for GDB scripts (in the test suite)
- interactive UART support (you can navigate those bootloader menus)

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/fromutil-550D.png)

Here's an easy coding task for you: some models don't have a debugmsg.gdb script. Creating one requires just finding some stubs. Right now, those models are: 5D2, 6D, 7D, 50D, 500D, 650D, 100D, 1100D and EOS M.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on October 25, 2016, 10:26:31 PM
I don't see menu on Lubuntu 16.04:
(http://thumbnails116.imagebam.com/51158/249e29511571455.jpg) (http://www.imagebam.com/image/249e29511571455)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 01, 2016, 08:38:20 PM
Some more updates:

- emulate the picture taking process (60D and 1200D):
   - can "capture" a full-res silent picture; that includes:
     - EDMAC transfers and interrupts
     - image processing modules (http://magiclantern.wikia.com/wiki/Register_Map#Image_PreProcessing) (dummy emulation of ADKIZ and HIV for now)
     - HEAD timers (dummy emulation for now)
   - you will need a reference image at qemu/<camera>/VRAM/PH-QR/RAW-000.DNG (a full-res silent picture)
   - it cannot emulate the unmodified silent.mo because of some GUI calls
   - you can either patch silent.c & raw.c, or use a minimal implementation
- 550D shows the date/time screen (but refuses to show the main screen)
- heartbeat timer now works on VxWorks models (can switch tasks)
- 450D emulates file I/O from main firmware
- a few more self-tests:
   - portable ROM dumper test (SD I/O from bootloader; no CF yet)
   - file I/O test from main firmware (check whether the DCIM directory is created at startup; SD models only)
   - test for picture taking emulation (test code under minimal/qemu-frsp)

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/qemu-60D-frsp.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/qemu-1200D-frsp.png)

The first feature also reveals the register configurations required for taking a picture by directly driving the sensor with our own code :)

Full-res photo capture log: qemu-60D-FA_CaptureTestImage.log (http://a1ex.magiclantern.fm/bleeding-edge/qemu/qemu-60D-FA_CaptureTestImage.log)

Current self-test log:

Testing bootloaders...
   5D: ROM READY
  5D2: K218 READY
  5D3: K285 READY
  5D4: K349 READY
   6D: K302 READY
   7D: FAILED!
7D2M: K289M READY
  40D: K190 READY : Ver 4.0.1
  50D: K261 READY
  60D: K287 READY
  70D: K325 READY
  80D: K350 READY
400D: InitializeIntercom
450D: K176 READY : Ver 4.0.3
500D: K252 READY
550D: K270 READY
600D: K286 READY
650D: K301 READY
700D: K326 READY
750D: K393 READY
760D: K347 READY
100D: K346 READY
1000D: K254 READY : Ver 3.7.5
1100D: K288 READY
1200D: K327 READY
EOSM: K331 READY
5D4AE: K349AE AECU Firm Ver. 5.8.1(5.8.1)
7D2S: K289S READY

Testing Canon GUI...
  60D: OK
  5D3: OK
550D: OK
600D: OK
1200D: OK
1100D: OK

Testing GDB scripts...
   5D: [  tExcTask:ffb223cc ] task_create(CmdShell, prio=a, stack=0, entry=ffb22420, arg=0)
  5D2: 5D2/debugmsg.gdb not present
  5D3: [      init:ff0c32d4 ] task_create(Startup, prio=11, stack=400, entry=ff0c2928, arg=0)
  5D4: [      init:fe0e14c9 ] task_create(SFRead, prio=11, stack=400, entry=fe0e0ef7, arg=620008)
   6D: 6D/debugmsg.gdb not present
   7D: 7D/debugmsg.gdb not present
7D2M: [      init:00002207 ] task_create(OmarInit, prio=f, stack=400, entry=2183, arg=4ae1ec)
  40D: [ tTaskMain:ff812a60 ] task_create(HotPlug, prio=1e, stack=0, entry=ff812cb8, arg=0)
  50D: 50D/debugmsg.gdb not present
  60D: [   Startup:ff1dcc18 ] task_create(PropMgr, prio=14, stack=0, entry=ff1dcb24, arg=807b1c)
  70D: [      init:ff0c3360 ] task_create(TaskMain, prio=1d, stack=0, entry=ff0c28ac, arg=0)
  80D: [      init:fe0d449f ] task_create(TaskMain, prio=1d, stack=0, entry=fe0d3619, arg=0)
400D: [  tStartup:ffb1d1bc ] task_create(ImagePlayDriverTask, prio=19, stack=0, entry=ffa0ccd0, arg=0)
450D: [ tTaskMain:ffd0a664 ] task_create(PropMgr, prio=15, stack=0, entry=ffd0a9e4, arg=384e5c)
500D: 500D/debugmsg.gdb not present
550D: [   Startup:ff1d8b30 ] task_create(PropMgr, prio=14, stack=0, entry=ff1d8a3c, arg=726d20)
600D: [   Startup:ff1fbba8 ] task_create(PropMgr, prio=14, stack=0, entry=ff1fbab4, arg=757140)
650D: 650D/debugmsg.gdb not present
700D: [   Startup:ff0c38fc ] task_create(Startup2, prio=11, stack=400, entry=ff0c35b0, arg=0)
750D: [      init:fe0ce241 ] task_create(TaskMain, prio=1d, stack=0, entry=fe0cd4a9, arg=0)
760D: [      init:fe0ce445 ] task_create(TaskMain, prio=1d, stack=0, entry=fe0cd6ad, arg=0)
100D: FAILED!
1000D: [          :ff812eec ] task_create(HotPlug, prio=1e, stack=0, entry=ff813050, arg=0)
1100D: 1100D/debugmsg.gdb not present
1200D: [      K327:ff2b9bd8 ] task_create(PropMgr, prio=14, stack=0, entry=ff2b9ae4, arg=71a11c)
EOSM: EOSM/debugmsg.gdb not present
5D4AE: [      init:fe0a2aa1 ] task_create(TaskMain, prio=1d, stack=0, entry=fe0a2159, arg=0)
7D2S: 7D2S/debugmsg.gdb not present
EOSM3: [   Startup:010e17a9 ] task_create(SD1stInit, prio=18, stack=0, entry=10e1739, arg=0)
A1100: [   Startup:ffc3f6b0 ] task_create(SD1stInit, prio=18, stack=0, entry=ffc3f604, arg=0)

Setting up temporary SD/CF card images...
'../magic-lantern/contrib/qemu/sd.img.xz' -> './sd.img.xz'

Testing FA_CaptureTestImage...
  60D: OK
1200D: OK

Testing file I/O (DCIM directory)...
  60D: OK
  5D3: OK
550D: OK
600D: OK
1200D: OK
1100D: OK
100D: OK
450D: OK

Testing display from bootloader...
   5D: OK
  5D2: OK
  5D3: OK
  5D4: OK
   6D: OK
   7D: please check
7D2M: OK
  40D: OK
  50D: OK
  60D: OK
  70D: OK
  80D: OK
400D: OK
450D: OK
500D: OK
550D: OK
600D: OK
650D: OK
700D: OK
750D: OK
760D: OK
100D: OK
1000D: OK
1100D: OK
1200D: OK
EOSM: OK

Testing EOS M3...
SD boot: StartDiskboot
Display: TurnOnDisplay

Preparing portable ROM dumper...
Testing portable ROM dumper...
   5D: skipping
  5D2: skipping
  5D3: ROM0.BIN: OK ROM1.BIN: OK
  5D4: ROM1.BIN: OK
   6D: ROM0.BIN: OK ROM1.BIN: OK
   7D: ROMs not saved
7D2M: ROM1.BIN: OK
  40D: skipping
  50D: skipping
  60D: ROM0.BIN: OK ROM1.BIN: OK
  70D: ROM0.BIN: OK ROM1.BIN: OK
  80D: ROM1.BIN: OK
400D: skipping
450D: skipping
500D: skipping
550D: ROM0.BIN: OK ROM1.BIN: OK
600D: ROM0.BIN: OK ROM1.BIN: OK
650D: ROM0.BIN: OK ROM1.BIN: OK
700D: ROM0.BIN: OK ROM1.BIN: OK
750D: ROM1.BIN: OK
760D: ROM1.BIN: OK
100D: ROM0.BIN: OK ROM1.BIN: OK
1000D: skipping
1100D: ROM0.BIN: OK ROM1.BIN: OK
1200D: ROM0.BIN: OK ROM1.BIN: OK
EOSM: ROM0.BIN: OK ROM1.BIN: OK

Restoring your SD/CF card images...


It's getting close to being usable for testing the nightly builds :)




You may wonder why I'm looking at those old VxWorks models - they do a lot of things in a slightly different way, yet still compatible with the new models. This gives extra information for understanding hardware internals (e.g. different debug strings, or just exercising different code paths in QEMU).

For example, the photo taking code on 60D uses an old-style interface for the interrupt controller, but most other code uses the new one. There is a single interrupt controller, with two interfaces (see eos_handle_intengine and eos_handle_intengine_vx in eos.c). The VxWorks models use both interfaces (very old models probably use only the old interface).

Before looking at VxWorks models, I had no idea what the old-style registers might even be doing.

Will this extra knowledge translate into new features or new models ported? I hope so, but - as usual - can't promise anything. Feel free to jump in and get your hands dirty with the code.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on November 08, 2016, 11:59:15 PM
i still want 1k FPS :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on November 09, 2016, 07:13:35 PM
Quote from: g3gg0 on November 08, 2016, 11:59:15 PM
i still want 1k FPS :)
it's better to want a replacement for cache hacks on Digic 6  :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on November 09, 2016, 10:24:54 PM
what do you mean?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on November 10, 2016, 11:56:13 AM
I mean that cache debug operations are not found in Cortex-R4 Technical Reference Manual (http://infocenter.arm.com/help/topic/com.arm.doc.ddi0363g/CIHBJBAI.html)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on November 12, 2016, 03:57:50 PM
oh. didnt know. where was the discussion?
this one? http://www.magiclantern.fm/forum/index.php?topic=13746.msg170772#msg170772
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on January 18, 2017, 01:18:50 AM
More updates:

- 100D boots Canon GUI
- 70D shows date/time screen (but refuses to boot Canon GUI if date/time is patched)
- EOSM and 450D run most of the initialization and pass the DCIM dir test
- very basic (incomplete) support for Eeko and JPCORE
- photo capture emulation works on 5D3 as well (test doesn't fully pass yet though)
- memory protection registers are printed at startup
- full list here (https://bitbucket.org/hudson/magic-lantern/commits/branch/qemu).

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/100D-qemu.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/70D-qemu.png)

The test suite now runs on ML build server as well, here (http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on January 29, 2017, 09:58:38 PM
500D with spells from 550D has a new menu item :

(https://s23.postimg.org/l9ynkv557/qemux1.png) (https://s27.postimg.org/y44mzqjpv/qemux2.png) (https://s28.postimg.org/4i58gvjn1/qemux3.png)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on January 29, 2017, 11:03:34 PM
Very cool. I had this setting enabled on 550D, so it must be saved on the MPU side (probably EEPROM).

Old notes about it: https://groups.google.com/d/topic/ml-devel/ti8GyVqEZmo/discussion
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on January 30, 2017, 02:21:39 AM
Yes it is stored in the MPU.
Studio mode on  { 0x06, 0x05, 0x01, 0x42, 0x01, 0x00 },
Studio mode off { 0x06, 0x05, 0x01, 0x42, 0x00, 0x00 },
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on January 31, 2017, 01:01:49 AM
500D 30MB/s  :P
(https://s30.postimg.org/72oj0zpw1/qemubench.png)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on February 12, 2017, 05:59:45 PM
GUI switch progress :

(https://s12.postimg.org/72qetgkz1/qemu.png) (https://s17.postimg.org/5jxrwq0cv/qemu1.png) (https://s31.postimg.org/4qttjvtej/qemu2.png) (https://s31.postimg.org/vpxof1fvf/qemu3.png) (https://s10.postimg.org/4k248c1op/qemu4.png)

Live View :

(https://s12.postimg.org/4gfhq9j59/qemulv.png)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on February 14, 2017, 09:03:32 PM
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on February 16, 2017, 10:18:39 PM
i am still stunned how good the emulation works.. :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on February 17, 2017, 04:48:58 PM
Live View VRAM patch :

(https://s24.postimg.org/fry3mwyyt/qemu_lv.png)

[EDMAC#18] Starting transfer to 0x1B07800 from conn #4, 1440x424, flags=0x20000080
Loading photo raw data from ./500D/VRAM/PH-LV/LV-000.422...
[EDMAC#18] 610560 bytes written to 1B07800-1B9C900.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on February 18, 2017, 11:28:54 PM
500D menu navigation works here too, thanks Greg :D

Currently, this old camera is the only one that lets you navigate Canon menu. All other models show either a static GUI or the date/time screen.

Also committed:
- initial support for EOS M10 and M5 (for CHDK)
- an option to export function calls to IDA
- an experiment to group related MPU messages from timestamps (in the dm-spy-experiments branch)
- some auto comments regarding MPU messages
- minor fixes here and there.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on February 21, 2017, 01:46:02 AM
Formatting the virtual card works too, both from Canon and ML (of course, on 500D) :)

This is the first test in the suite that runs an unmodified ML binary. It actually downloads the current nightly build (at the time of writing) and checks both the GUI (expected screens) and the card contents (whether ML still boots after being restored).

Test log (http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/47/console)

This log should also contain useful info (what commands to run), should you want to reproduce these experiments on your PC. I should probably write a guide, other than the tips from the install script.

Menu screens currently covered by the test suite:

(http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/47/artifact/qemu/tests/menu.png)

(http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/47/artifact/qemu/tests/format.png)

(http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/47/artifact/qemu/tests/fmtrestore.png)

There are still some nondeterministic bugs (that's why some tests are retried a few times, until sucess); those will need fixing before using QEMU as a test platform for ML builds. Still, it already starts to be useful (for example, for getting menu screenshots).

At this stage, I think the old implementation (http://www.magiclantern.fm/forum/index.php?topic=2864.msg173682#msg173682) is no longer useful, so we may start thinking about merging the QEMU branch into unified. This will remove most of those CONFIG_QEMU hacks from the source code.

BTW, if you have experience with some testing framework, and you know a nicer way to implement the current tests (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/tests/run_tests.sh), I'd be interested in hearing from you.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Greg on February 26, 2017, 06:17:25 PM
Firmware update
ROM modified with hexeditor "DisableMainFirm" - http://magiclantern.wikia.com/wiki/Bootflags
500D 1.1.1 -> 1.1.2

(https://s27.postimg.org/9l5keyg83/qemu1.png) (https://s32.postimg.org/d6bg9eb91/qemu2.png) (https://s10.postimg.org/ao5qy413d/qemu3.png) (https://s27.postimg.org/537g4rl43/qemu4.png)

200:  5337.856 [UPD] Welcome to Update program
201:  5337.856 [UPD]   Program Ver.Slave 0.2.0
208:  5338.112 [UPD] ------------ Initialized
277:  5343.232 [UPD] CurrentVersion=1.1.1
278:  5343.232 [UPD] DS_MODELID =0x80000252
295:  5350.144 [MS] LOCK (1)
535:  8316.672 [UPD] StartFirmupProgress
569:  8319.232 [UPD] ERROR Do not read
571:  8358.912 [UPD] 0=UPD_VerifyFirmware
572:  8830.464 [UPD] 0=UPD_DecryptoFirmware
574:  8850.944 [UPD] CheckSum file=0xd960afec buffer=0xd960afec
575:  8947.968 [UPD] SAFEMODE
620: 35370.240 [UPD] ERR 1=updSpecificPartner


Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on March 07, 2017, 04:44:59 PM
First ML change I've tested in QEMU on all unified models:

https://bitbucket.org/hudson/magic-lantern/pull-requests/796/new-method-for-getting-current-task-names/diff

Latest update adds partial 7D support (slave CPU only, without IPC).

Test log (http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/48/console).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on March 24, 2017, 01:08:42 AM
A small change that unlocked Canon menu navigation on many models:

https://bitbucket.org/hudson/magic-lantern/commits/c881ba2

After some refactoring and porting the 500D MPU messages required for GUI, Canon menu navigation is now also working on...

60D, 550D, 600D, 700D, 100D, 1100D and 1200D!

Screenshots (guess the cam):

(https://builds.magiclantern.fm/jenkins/job/QEMU-tests/54/artifact/qemu/tests/100D-menu.png)
(https://builds.magiclantern.fm/jenkins/job/QEMU-tests/54/artifact/qemu/tests/1100D-menu.png)
(https://builds.magiclantern.fm/jenkins/job/QEMU-tests/54/artifact/qemu/tests/600D-menu.png)
(https://builds.magiclantern.fm/jenkins/job/QEMU-tests/54/artifact/qemu/tests/60D-format.png)

Test log (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/54/console).

All screenshots here (https://builds.magiclantern.fm/jenkins/job/QEMU-tests/54/) (click on Expand all).

This is a big breakthrough, as it effectively lets me review ML ports or code changes on cameras I don't own :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: DeafEyeJedi on March 24, 2017, 05:34:39 AM
Holy cow, @a1ex!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on March 25, 2017, 05:40:31 PM
please don't let jenkins test builds format the card. i am scared it will wipe our build server  :o

kidding :D
cool work!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nkls on March 27, 2017, 10:12:32 PM
Nice work! Finally some running cameras to toy around with.  :D

I've managed to merge the latest changes into qemu v2.9.0-rc1 (https://bitbucket.org/niklastisk/qemu/branch/eos-2.9.0-rc1-merge). It's probably not a perfect merge, the camera gui hangs up more often than on v2.5.0 for me.

The patches to qemu code merged quite well, but someone with more knowledge of them should probably review whether they are ok/redundant in 2.9.0.

A major difference is that I've replaced the interrupt thread with a QEMUTimer. This solved som iothread lock bug, but I guess it should work better than the thread since it's now synced to the guest clock and not the host system.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on March 27, 2017, 10:22:20 PM
Quote from: nkls on March 27, 2017, 10:12:32 PM
A major difference is that I've replaced the interrupt thread with a QEMUTimer. This solved som iothread lock bug, but I guess it should work better than the thread since it's now synced to the guest clock and not the host system.

Yay! That's what I wanted to do next, hoping it would solve the GUI lock-ups. I/O lock was another issue that I didn't know how to solve.

Quote
I've managed to merge the latest changes into qemu v2.9.0-rc1 (https://bitbucket.org/niklastisk/qemu/branch/eos-2.9.0-rc1-merge). It's probably not a perfect merge, the camera gui hangs up more often than on v2.5.0 for me.

My attempt to merge with 2.8.0 was not very successful (got stuck at making the serial port work, so ended up disabling it), but otherwise it seemed to run fairly well. Will try to integrate your changes and see how it goes.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 17, 2017, 07:05:01 PM
Quote from: nkls on March 27, 2017, 10:12:32 PM
A major difference is that I've replaced the interrupt thread with a QEMUTimer. This solved som iothread lock bug, but I guess it should work better than the thread since it's now synced to the guest clock and not the host system.

This solved the intermittent I/O lock-ups that I was unable to track down for a long time!!!

5D3 SD card test successfully ran 10 times in a row. Previously, I had to run it about 5 times to get one successful run...

All the menu navigation tests from test suite passed with flying colors, without any retries required!

Thanks nkls!!!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: DeafEyeJedi on April 17, 2017, 07:17:26 PM
Hell yeah and way to go @nkls!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 24, 2017, 10:18:01 PM
Added a couple of jobs on the build server:

- QEMU-dm-spy (http://builds.magiclantern.fm/jenkins/job/QEMU-dm-spy/): compiles the dm-spy-experiments branch and runs the binary in the emulator. These logs contain all Canon's debug messages (http://www.magiclantern.fm/forum/index.php?topic=2388.0), and optionally all MMIO activity. Should be useful for anyone who wants to understand the startup process.
- QEMU-boot-check (http://builds.magiclantern.fm/jenkins/job/QEMU-boot-check/): compiles ML from every camera model (from nightly) with CONFIG_QEMU=y and runs it for a few seconds in the emulator; this compilation flag enables additional debug info at startup, useful for checking the boot process (where autoexec.bin is loaded, how much memory it takes, what it does to reserve it and so on).
- QEMU-FA_CaptureTestImage (http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-FA_CaptureTestImage/): compiles a minimal autoexec.bin that calls FA_CaptureTestImage (therefore taking a full-res silent picture). All the debug messages from Canon and all the MMIO activity are logged. Might be useful for understanding the still photo capture process (http://www.magiclantern.fm/forum/index.php?topic=1915.0).
- QEMU-tests (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/): that's the test suite for QEMU (presented earlier in this thread)

All these tests have HTML logs (actually just plain text with colors) and screenshots (where it's the case).

I'm also thinking to run some basic tests on the nightly, on those models with functional GUI (tests such as menu screenshots, load each module, check memory usage, run some simple Lua scripts). The emulation is not there yet for more complex tests (for example, we cannot take a CR2 picture or go to LiveView).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 26, 2017, 09:25:05 PM
Some updates:
- upgraded to QEMU 2.9.0, thanks nkls (still experimental, as I had quite a bit of trouble with it, so it's in a different branch for now)
- fixed another (or maybe the same?) nondeterministic lock-up (see a few posts above)
- initial support for 1300D (http://www.magiclantern.fm/forum/index.php?topic=17969) (WIP)
- options to log memory accesses (aka memory tracing); run with "-d help" to get the list

The lock-up bug was showing up very rarely on 2.5.0 after the timer refactoring from nkls (let's say about 1 out of 100 runs was bad), but after upgrading to 2.9.0 it showed up in more than half of the test runs (or about 1/5 of the test runs if the log was redirected to file). Narrowed down to interrupt controller (from a change made many months ago to support 1000D and other VxWorks models).

I'm also experimenting with logging all memory accesses made by the guest firmware, on 2.5.0. Examples for 1300D:


./run_canon_fw.sh 1300D -d romw
...
Firm Jump RAM to ROM 0xFE0C0000
K404 READY
[rom1]     at 0x0001D54C:0001D54C [0xF8000000] <- 0x6       : 8-bit
[rom1]     at 0x0001D54C:0001D54C [0xF8000000] <- 0x6       : 8-bit
[rom1]     at 0x0001D54C:0001D54C [0xF8000000] <- 0xE9      : 8-bit
[DMA1] Copy [0xF8E60000] -> [0x402D4000], length [0x0026BBF8], flags [0x00030001]
[DMA1] OK
     0:    20.480 [STARTUP]



./run_canon_fw.sh 1300D -d ramw,romr
...
[rom1]     at 0xFE0C000C:001000EC [0xFEA7A270] -> 0xE92D4010
[ram]      at 0xFE0C000C:001000EC [0x00001900] <- 0xE92D4010
[rom1]     at 0xFE0C009C:001000EC [0xFEA7A274] -> 0xEB000BAB
[ram]      at 0xFE0C009C:001000EC [0x00001904] <- 0xEB000BAB
...


I know I'm (https://github.com/aclements/mtrace) almost (https://github.com/moyix/panda) certainly (https://github.com/panda-re/panda) reinventing (https://projects.gso.ac.upc.edu/projects/qemu-dbi/wiki) the (https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/cheri-qemu.html) wheel (http://web.eece.maine.edu/~vweaver/projects/qemu-trace/), but I had only limited success with these modified versions:
- mtrace uses a very very old QEMU
- panda 1.0 uses QEMU 1.0.1, examples work, lots of nice tools, but appears deprecated (shouldn't be hard to roll back our patches to the older version)
- panda 2.0 uses a very recent QEMU, but could not run any ARM examples (segmentation fault). Also, most of the cool tools from panda 1.0 are not ported yet.
- QEMU-DBI is "being upstreamed into QEMU", and a large part of it is already in 2.9.0 (the main reason I've upgraded). TODO: figure out how to use it...
- QEMU-CHERI is a mod for MIPS that also traces memory and instructions (nice to see how it works)
- the last one, QEMU-trace, is a very simple patch that showed me where to place the hooks in the QEMU codebase (also with this message (https://lists.nongnu.org/archive/html/qemu-devel/2014-04/msg02875.html) and this thread (https://lists.linaro.org/pipermail/linaro-dev/2012-March/010887.html) from mailing lists).

So, yeah, I still want to use the state-of-art method for logging memory accesses, just need to figure out how. Until then, my monkey-patched method appears to work pretty well (can rebuild the memory contents from the trace) and has very little overhead as long as I'm not printing each access to the console.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 01, 2017, 01:49:29 AM
Currently experimenting with a binary instrumentation tool similar to valgrind's memcheck (though a lot more primitive, as I'm reinventing the wheel again). It's written on top of the memory tracing (which is already committed) and a similar hook calling every time a new code block (TranslationBlock) is executed.

Quick example (don't click me):

    uint32_t * p = malloc(1234);
    qprintf("p=%x\n", p);
    p[100] = p[200] + 1;            /* use of uninitialized value (read) */
    free(p);
    qprintf("p freed\n");
    p[20] = p[30] + 1;              /* use after free (both read and write) */
    qprintf("test complete\n");


From emulation log:

p=fb440
[run_test:589c8:589c8] fb760 uninitialized
p freed
[run_test:589e4:589e4] fb4b8 read after free (0)
[run_test:589f0:589e4] fb490 written after free (1)
test complete


The current state is just a very rough proof of concept, but it already found a bunch of null pointer, uninitialized memory (https://bitbucket.org/hudson/magic-lantern/commits/branch/memcheck-fixes) and  thread safety (https://bitbucket.org/hudson/magic-lantern/commits/branch/thread-safety) bugs :D
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: eduperez on May 01, 2017, 10:21:10 PM
Quote from: a1ex on May 01, 2017, 01:49:29 AM
The current state is just a very rough proof of concept, but it already found a bunch of null pointer, uninitialized memory (https://bitbucket.org/hudson/magic-lantern/commits/branch/memcheck-fixes) and  thread safety (https://bitbucket.org/hudson/magic-lantern/commits/branch/thread-safety) bugs :D

For a moment I though you where talking about bugs in Canon's code...  :o
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 01, 2017, 10:29:41 PM
For your viewing pleasure:

http://builds.magiclantern.fm/jenkins/job/QEMU-memcheck/QEMU_memcheck_logs/500D.111-memchk.log.html

The analysis only has 500D stubs for now (though it's easy to add for other models).

The first bunch of TCM warnings can be ignored (these are the initialization sequence). The remaining TCM accesses from Canon tasks are probably bugs in Canon firmware, or in my emulation.

Here's an obvious one, if you look it up in the disassembly:

[FileMgr:ff3b5d38:ff3b5d38] address 0 written to TCM (12)
[FileMgr:ff3b5d48:ff3b5d48] address 0 written to TCM (2000)
[FileMgr:ff3b5d58:ff3b5d58] address 0 written to TCM (100)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 02, 2017, 09:08:07 PM
More updates:

- 50D boots the GUI! (figured it out from this log (http://www.magiclantern.fm/forum/index.php?topic=9852.msg184191#msg184191))
- 5D2 is very close (http://www.magiclantern.fm/forum/index.php?topic=11205.msg184202#msg184202)
- faster emulation (test suite about twice as fast)
- code coverage report (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-coverage/QEMU_code_coverage_(lcov)/)

(https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/98/artifact/qemu/tests/50D-menu.png)

Self-testing log (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/98/console)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kennetrunner on May 11, 2017, 05:07:41 PM
@a1ex   I have qemu 2.9, and tried to install the patch contrib/qemu/qemu-2.5.0.patch   - which fails :-(
Do you have an updated patch file ?

.. ken
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 11, 2017, 09:58:20 PM
Yes, on the qemu-2.9.0 branch.

However, some parts of the test suite fail in 2.9.0 (but are OK on 2.5.0), and 2.5.0 has a few more features on the EOS side (in particular, the memory tracing and related tools). Syncing them shouldn't be very difficult.

2.9.0 can be interesting for EOS M5, as it might emulate multi-core CPUs a little better (note: the self-test fails for this model) and for porting the code to other QEMU variants (e.g. Panda 2.0 for its binary instrumentation plugins, or Xilinx QEMU*) if you want to look into UHS), but for the moment I've switched back to 2.5.0 (simply because it works and I'm used to it).

*) Xilinx QEMU is currently based on 2.6.0, so the patch for it is probably something in-between.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kennetrunner on May 12, 2017, 02:48:58 PM
So, after a ton of hurdles I shelved my own attempts at getting qemu 2.9 running...
Instead I downloaded the vbox image from http://www.magiclantern.fm/forum/index.php?topic=7579.msg134989#msg134989 (http://www.magiclantern.fm/forum/index.php?topic=7579.msg134989#msg134989) and compiled qemu 2.5 and it **seemed** to complete successfully...

However, now when I run ./run_canon_fw.sh 550D I get qemu-system-arm: unsupported machine type

When I list the supported machine types, the only Canon one to show up is canon-a1100

Seems like I'm missing a step, somewhere ?... any pointers ?

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 12, 2017, 03:50:03 PM
The vanilla QEMU does not include our modifications.

See the install log (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/lastSuccessfulBuild/consoleFull) from the build server for reference.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 17, 2017, 03:08:44 PM
I just started into trying to get the qemu branch working on macOS Sierra 10.12.4.

The ML repository needs to be in the home (user) directory or it won't work. The script in magic-lantern/contrib/qemu/install.sh seems to run fine up until this point:

Next steps:
===========

1) Compile QEMU

   cd /Users/rosiefort/qemu/qemu-2.5.0
   ../configure_eos.sh
grep: /proc/cpuinfo: No such file or directory
   make -j


Running ../configure_eos.sh (without the cpuinfo) or "./configure --target-list=arm-softmmu --disable-docs --enable-sdl" results in:

ERROR: Cocoa and SDL UIs cannot both be enabled at once

QEMU can also be installed via Homebrew but of course it doesn't include the Magic Lantern modifications.

Maybe it is a matter of figuring out how to configure the "cpuinfo" for the Mac?
sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz


Though it is probably much more than just that. Maybe it is the clang compiler? Any hints?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 17, 2017, 03:11:35 PM
Comment out the affected line; plain make is fine.

Or, try something like this:


echo "   make -j`grep -c processor /proc/cpuinfo || sysctl -n hw.ncpu || echo 1`"


Compiling without SDL should be fine (it's used that way on the build server, where it runs without GUI).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 17, 2017, 03:52:07 PM
echo "   make -j`grep -c processor /proc/cpuinfo || sysctl -n hw.ncpu || echo 1`"
grep: /proc/cpuinfo: No such file or directory
   make -j8


Ah ha, so the processor on this machine is:
sysctl -n hw.ncpu
8


So I tried:
make -j8
Please call configure before running make!
make: *** No rule to make target `trace/generated-events.h', needed by `Makefile'.  Stop.
make: *** Waiting for unfinished jobs....
make: *** [config-host.mak] Error 1



Changing configure_eos.sh so it uses gcc-5 instead of clang gives the warning noted in the script but it still comes up with:
C++ compiler c++ does not work with C compiler gcc-5 --std=gnu99
Disabling C++ specific optional code

ERROR: Cocoa and SDL UIs cannot both be enabled at once


It looks like the issue is setting this up so the configure script will select only the SDL UI. Need to dig into it a bit more.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 17, 2017, 04:00:25 PM
To specify the C++ compiler, try this:


CC="clang" CXX="clang++" \
    ./configure --target-list=arm-softmmu --disable-docs --disable-sdl \
    --extra-cflags="-Wno-error=deprecated-declarations" $*


It almost compiles with clang, except for a tiny function which you can safely comment out, and some warnings. Will look into them.

For SDL, try --disable-sdl.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 17, 2017, 04:48:09 PM
Almost there. Are you seeing these same errors?
/Users/rosiefort/qemu/qemu-2.5.0/hw/arm/../eos/dbi/logging.c:196:9: error:
      function definition is not allowed here
        {
        ^
/Users/rosiefort/qemu/qemu-2.5.0/hw/arm/../eos/dbi/logging.c:200:16: error: use
      of undeclared identifier 'close_idc'; did you mean 'closedir'?
        atexit(close_idc);
               ^~~~~~~~~
               closedir
/usr/include/dirent.h:102:5: note: 'closedir' declared here
int closedir(DIR *) __DARWIN_ALIAS(closedir);
    ^
2 errors generated.
make[1]: *** [hw/arm/../eos/dbi/logging.o] Error 1
make[1]: *** Waiting for unfinished jobs....
/Users/rosiefort/qemu/qemu-2.5.0/hw/arm/../eos/dbi/memcheck.c:91:20: warning:
      unused function 'set_uninitialized' [-Wunused-function]
static inline void set_uninitialized(uint32_t addr)
                   ^
/Users/rosiefort/qemu/qemu-2.5.0/hw/arm/../eos/dbi/memcheck.c:99:20: warning:
      unused function 'set_freed' [-Wunused-function]
static inline void set_freed(uint32_t addr)
                   ^
/Users/rosiefort/qemu/qemu-2.5.0/hw/arm/../eos/dbi/memcheck.c:107:20: warning:
      unused function 'clr_freed' [-Wunused-function]
static inline void clr_freed(uint32_t addr)
                   ^
3 warnings generated.
1 warning generated.
make: *** [subdir-arm-softmmu] Error 2


Hint on where to find the tiny function that can be commented out?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 17, 2017, 05:01:06 PM
Yes, exactly there.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 17, 2017, 05:34:25 PM
Tried something different. Since cr2hdr doesn't compile on clang we need to install a different compiler on the Mac.
CC="gcc-5" \
    ./configure --target-list=arm-softmmu --disable-docs --disable-sdl \
    --extra-cflags="-Wno-error=deprecated-declarations" $*


Then ran "make -j8" like before and it compiled. There were some other errors but it finished compiling.

Got to run off to work now but hope to get back to this soon.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 18, 2017, 12:51:49 AM
These are the errors I got running the gcc-5 compiler from the Homebrew distribution on the Mac:
In file included from /usr/include/Availability.h:190:0,
                 from /usr/include/stdlib.h:61,
                 from /Users/rosiefort/qemu/qemu-2.5.0/include/qemu/osdep.h:35,
                 from /Users/rosiefort/qemu/qemu-2.5.0/include/qemu-common.h:15,
                 from block/raw-posix.c:24:
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:53:34: error: expected ',' or '}' before '__attribute__'
     kCFISO8601DateFormatWithYear API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0)) = (1UL << 0),
                                  ^
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:80:126: error: 'introduced' undeclared here (not in a function)
CFDateFormatterRef CFDateFormatterCreateISO8601Formatter(CFAllocatorRef allocator, CFISO8601DateFormatOptions formatOptions) API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0));
                                                                                                                              ^
  CC    crypto/cipher.o
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'deprecated' undeclared here (not in a function)
const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                       ^
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'message' undeclared here (not in a function)
const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));


It did finish compiling so did it work? Followed instructions with the rom dumps and mounted the sd card image and this is what happened:
./run_canon_fw.sh EOSM
./run_canon_fw.sh: line 10: losetup: command not found
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]
./run_canon_fw.sh: line 10: losetup: command not found
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]
CHK version_gen.h
  LINK  qemu-ga
  LINK  ivshmem-server
  CC    block/raw-posix.o
In file included from /usr/include/Availability.h:190:0,
                 from /usr/include/stdlib.h:61,
                 from /Users/rosiefort/qemu/qemu-2.5.0/include/qemu/osdep.h:35,
                 from /Users/rosiefort/qemu/qemu-2.5.0/include/qemu-common.h:15,
                 from block/raw-posix.c:24:
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:53:34: error: expected ',' or '}' before '__attribute__'
     kCFISO8601DateFormatWithYear API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0)) = (1UL << 0),
                                  ^
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:80:126: error: 'introduced' undeclared here (not in a function)
CFDateFormatterRef CFDateFormatterCreateISO8601Formatter(CFAllocatorRef allocator, CFISO8601DateFormatOptions formatOptions) API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0));
                                                                                                                              ^
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'deprecated' undeclared here (not in a function)
const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                       ^
/System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'message' undeclared here (not in a function)
const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                       ^
block/raw-posix.c: In function 'hdev_open':
block/raw-posix.c:2129:23: warning: variable 'kernResult' set but not used [-Wunused-but-set-variable]
         kern_return_t kernResult;
                       ^
make: *** [block/raw-posix.o] Error 1


Looks like the same error messages.

Note that I'm running macosx 10.12.4 and there doesn't seem to be a readily available losetup. Is this necessary?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 18, 2017, 12:57:19 AM
Searching this error message gives http://stackoverflow.com/questions/41143981/macos-sierra-corefoundation-error-while-compiling-wxwidgets-for-simspark

To me, it looks like qemu didn't finish compiling.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 22, 2017, 12:11:20 AM
QEMU on Mac -- Yay!

(https://c1.staticflickr.com/5/4169/34767851946_d027c59a4f_z.jpg)

This might be a real hack, here's what I did. First of all the CoreFoundation.framework issues affect the Homebrew gcc-5 compiler so I went back to the macos clang.

~/qemu/configure_eos.sh
CC="clang" CXX="clang++" \
    ./configure --target-list=arm-softmmu --disable-docs --disable-sdl \
    --extra-cflags="-Wno-error=deprecated-declarations" $*


The compilation errors we had with clang were because of this section of code, so I commented it out.

~/qemu/qemu-2.5.0/hw/eos/dbi/logging.c line#194
        /* QEMU is usually closed with CTRL-C, so call this when finished */
//        void close_idc(void)
//        {
//            fprintf(idc, "}\n");
//            fclose(idc);
//        }
//        atexit(close_idc);


Loaded up the EOSM ROM0.BIN, ROM1.BIN and had to create an SFDATA.BIN using the sf_dump module and placed them in ~/qemu/EOSM. Mounted the sd.img and ran:

./run_canon_fw.sh EOSM

Now just because it launches doesn't mean that it is working properly but I wanted to share my progress.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 22, 2017, 11:45:27 AM
Nice. This is the cocoa interface, right? Do you have menus that allow you to switch between serial console, VGA and so on?

BTW, after this commit (https://bitbucket.org/hudson/magic-lantern/commits/970d60759f1ea6251925c04874b854208885cc4a) it should install cleanly on clang and/or Mac.

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 23, 2017, 02:51:37 PM
Speaking for all of us Mac users -- Thanks!

Here are a few things Mac users who want to get this working should watch out for.

install.sh prints some instructions at the end of the process. On osx it prints out the grep error instead of keeping silent.
1) Compile QEMU

   cd /Users/rosiefort/qemu/qemu-2.5.0
   ../configure_eos.sh
grep: /proc/cpuinfo: No such file or directory
   make -j8


It is caused by this line:

install.shecho "   make -j`grep -c processor /proc/cpuinfo || sysctl -n hw.ncpu || echo 1`"


Switching the positions around eliminated the grep error on osx and created a sysctl command not found on Linux so I tried this to suppress the error messages and it worked:echo "   make -j`grep -c processor /proc/cpuinfo 2> /dev/null || sysctl -n hw.ncpu 2> /dev/null || echo 1`"



This instruction doesn't work for osx:
3) Mount the included SD (or CF) image (you may use mount.sh)
mount.sh calls "kpartx" which isn't available for osx. Mounting sd.img can be done by simply double clicking the sd.img icon but I'm not sure if that worked properly because when running qemu it shows this:
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
SD LOAD OK.
Open file for read : AUTOEXEC.BIN
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error


I looked for an alternate for kpartx and maybe hdiutil will work? I haven't figured out if there is some special way it needs to be invoked to mount the img files as loopback devices but either of these seem to work:hdiutil mount sd.img
or
hdiutil attach sd.img



Back to install.sh, at the end it is supposed to list some camera models but on osx it displays this:

   Note: Canon GUI emulation (menu navigation, no LiveView) only works on
   usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]


That problem is partially because the install script calls tests/run_tests.sh. Here is the output of running that script on osx:
https://pastebin.com/0T5GLRh5

Not sure if any of this would prevent qemu from running properly on osx. I'm just starting on this. Running the minimal autoexc.bin works on every ROM dump I've got but I haven't been able to go much beyond that.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 24, 2017, 10:05:48 AM
Quote from: dfort on May 23, 2017, 02:51:37 PM
SD: CMD12 in a wrong state
[SDIO] Error

That's fine, I get those too. I'm not sure if this indicates an emulation bug / incomplete model, or it's just how the (simplified) SD driver used in bootloader is supposed to behave (note: CMD12 is STOP_TRANSMISSION).

In the main firmware, you'll get a similar error about CMD1; this one is OK, as it appears to be the way Canon code probes for MMC cards. Regular SD cards are probably not supposed to reply to CMD1 outside the SPI transfer mode (at least that's my understanding), so the SD emulation backend prints some messages. The full conversation can be watched with -d sdcf (or -d sdcf,io for more details) and cross-checked with the SD specification (https://www.sdcard.org/downloads/pls/).

If the reader is familiar with SD protocol, I'd welcome any insights (in particular, for the UHS-I initialization sequence (http://www.magiclantern.fm/forum/index.php?topic=12862.0)).


Quote from: dfort on May 18, 2017, 12:51:49 AM
Note that I'm running macosx 10.12.4 and there doesn't seem to be a readily available losetup. Is this necessary?

To avoid corrupting the data on the SD image, it's best to prevent starting the emulation if the image is mounted by the user (as there will be two processes wanting to write on the same card image, without knowing about each other). This is done on Linux by checking /proc/mounts (losetup is used with "loopback" devices - that is, when mounting an image as a filesystem). Don't know how this works on Mac, and don't know whether my method is portable across other Linux distributions either.




The other issues appear to be (more or less) just annoyances (as they don't print what the user expects to see) and probably easy to fix.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 28, 2017, 03:29:02 PM
Some recent additions (some of them were covered in other threads):

* Cleaned up the self-test log (http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/lastSuccessfulBuild/console) to include installation instructions (the self-test begins with reinstalling QEMU from scratch, so you can also use it to check the expected output of the commands etc).

* Call stack reconstruction (inspired from Panda's callstack_instr (https://github.com/panda-re/panda/blob/master/panda/plugins/callstack_instr/callstack_instr.cpp), but heavily customized for EOS firmware). This makes the call stack available to other analysis tools (e.g. when memcheck wants to print an error). Enable with -d callstack . Works on DIGIC 4 to 6 and Eeko; almost there on DIGIC 2 and 3; for best results, .current_task_addr should also be defined (because each task has its own stack).

* Call/return trace. Every function call and return (that could be identified automatically) is logged on the console, including arguments and return values (enable with -d calls ). Works best on DIGIC 4 to 6 and Eeko, almost there on DIGIC 2 and 3. Interrupts are also handled (though it still has some trouble with corner cases). Example for Eeko (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/QEMU_self-testing_results/tests/5D3eeko/calls-fint-raw.log.html).

Tip: you can load these indented logs in your text editor, configure it for Python source code, and you've got collapse buttons :)

* Task switches. Make sure you have .current_task_addr defined in model_list.c and enable with -d tasks . Works on DIGIC 2 to 7 and Eeko. Example for EOS M2 (http://www.magiclantern.fm/forum/index.php?topic=15895.msg185280#msg185280).

* Memory blocks copied from ROM to RAM can be identified automatically (-d romcpy). They are listed on the self-test log (http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/lastSuccessfulBuild/console), and they should be a big time-saver when setting up the disassembler (usually you'll need to know what are the blocks worth disassembling, besides the ROM). Works on DIGIC 2 to 7. Example for EOS M5 (can be cross-checked with values found manually (https://chdk.setepontos.com/index.php?topic=13014.msg131205#msg131205)):

[ROMCPY] 0xE001AF2C -> 0xDF020000 size 0x3C0      at 0xE0005AA8
[ROMCPY] 0xE001B2E4 -> 0x4000     size 0xF1C      at 0xE000492C
[ROMCPY] 0xE115CF88 -> 0x8000     size 0x6054C    at 0xE002003C
[ROMCPY] 0xE11BD4D4 -> 0x1900000  size 0x1444     at 0xE0020060
[ROMCPY] 0xE11BE918 -> 0xDFFC4900 size 0x152A0    at 0xE0020084


* Symbol names from .elf files. QEMU has this feature built in, but had to customize it a bit (for example, it didn't recognize our stubs as function calls, because our build system doesn't mark them as such). Usage (requires bash):

. ./export_ml_syms.sh 550D.109
./run_canon_fw.sh 550D.109,firmware="boot=1" ...
...
Task switch to init:ff010470                                                     at [init:ff0164c4:ff076f18]
call 0x8B010 my_init_task(0, 8b010, 19980218, 19980218)                          at [init:ff0771dc:0]
call 0xFF018D1C init_task(0, 8b010, 19980218, 19980218)                         at [init:8b014:ff0771e0] (my_init_task)
  call 0xFF0108FC(0, 8b010, 19980218, 19980218)                                  at [init:ff018d20:8b018]


In other words, if you want the log to use the function names you have annotated in the disassembly, just add them to stubs.S and compile ML; the above script will load them. If you don't want to run autoexec.bin, you can probably just load stubs.o (which is an elf file), or you can probably create a stubs file just for analysis (one that will not be included in ML, because it will usually have a lot more names than you actually want to call).

There's a lot of room for improvement on this one (for example, right now function names are only listed in the call/return trace). Adding this info in more places is generally an easy coding task - find what parts of the log you would like to have function names rather than just raw addresses, and change the output strings accordingly). Automating symbol name export from your favorite disassembler should also be easy (and in many cases, the code is available online from other open source tools; just needs to be integrated here).

* Memcheck warnings closer to valgrind's (e.g. show where the affected memory block was allocated or freed).

* Basic ABI checking (e.g. R4-R11 and SP should be restored after function call). This is done while creating the call stack (-d callstack).

* The auto-generated IDC file should be much more accurate (because we finally have a sane call/return trace).

A walkthrough for some of these features can be found in the EOS M2 (http://www.magiclantern.fm/forum/index.php?topic=15895.msg185103#msg185103) topic, where, with dfort's help, I'd like to show in detail how to port Magic Lantern on a new camera from scratch.

Tip: to find all these analysis options, run QEMU with -d help. The ones interesting for us are:

int        show interrupts/exceptions in short format
exec       show trace before each executed TB (lots of logs)
nochain    do not chain compiled TBs so that "exec" and "cpu" show complete traces
io         EOS: log low-level I/O activity
mpu        EOS: log low-level MPU activity
sflash     EOS: log low-level serial flash activity
sdcf       EOS: log low-level SD/CF activity
uart       EOS: log low-level UART activity
ram        EOS: log all RAM reads and writes
rom        EOS: log all ROM reads and writes
ramr       EOS: log all RAM reads
romr       EOS: log all ROM reads
ramw       EOS: log all RAM writes
romw       EOS: log all ROM writes
ram_dbg    EOS: self-test for the RAM logging routines
callstack  EOS: reconstruct call stack (implies nochain,singlestep)
calls      EOS: log function calls (implies callstack,ramr,nochain,singlestep)
idc        EOS: export called functions to IDA (implies callstack,nochain,singlestep)
tasks      EOS: log task switches (implies callstack,nochain,singlestep)
romcpy     EOS: find memory blocks copied from ROM to RAM
memchk     EOS: check memory usage (malloc/free, uninitialized values)
v         
verbose    EOS: very detailed debug messages


Happy reversing!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on June 03, 2017, 06:26:55 PM
Just sharing some of my recent QEMU adventures on Mac -- and Cygwin.

It seems to be working fine on Mac but there are several issues with run_tests.sh. It doesn't really affect the compiling but it does show a message that makes it look like something went terribly wrong. I haven't worked it all out but so far most of the issues have to so with missing programs on the Mac.

The bash version that comes with the Mac is 3.2.57 but the command "declare -A" requires bash 4. Installing bash via Homebrew gives you version 4.4.12 but the script uses "#!/bin/bash" which points to the old version. Changing the script's shebang fixes that and this should be portable:
#!/usr/bin/env bash

Next problem is that the Mac doesn't have "ansi2txt" and there doesn't seem to be one on Homebrew either. I installed it from the source (https://sourceforge.net/projects/ansi2txt/). Because the Mac keeps you from installing programs in /bin I changed the Makefile like this:BINPATH = /usr/local/bin
MANPATH = /usr/local/share/man/man1


Yet another missing program is "timeout" and this can be installed via "brew install coreutils" but then you get gtimeout so to get the default names I had to (just run these commands to update the bash preference, no need to put it in the Makefile):PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"
MANPATH="/usr/local/opt/coreutils/libexec/gnuman:$MANPATH"


Almost there, there is a grep error that I mentioned before:   usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]


At this point I ran out of time but will revisit this later.

I also tried running QEMU on Cygwin. I didn't compile though "install.sh" did finish successfully. If you set up a Cygwin environment using my tutorial (http://www.magiclantern.fm/forum/index.php?topic=15894.msg#new) you'll need to install a few extra packages to get through the actual compiling.git
patch
gcc-g++


git needs to be set up:  git config --global user.email "[email protected]"
  git config --global user.name "Your Name"


The qemu-2.5.0 configure script supports Cygwin but it has a depreciated option:CYGWIN*)
  mingw32="yes"
  QEMU_CFLAGS="-mno-cygwin $QEMU_CFLAGS"
  audio_possible_drivers="sdl"
  audio_drv_list="sdl"
;;


Simply delete this line:
  QEMU_CFLAGS="-mno-cygwin $QEMU_CFLAGS"


I still didn't get it to compile and am stuck with this error:$ ../configure_eos.sh
Setting up QEMU on CYGWIN_NT-6.0...
Using gcc --std=gnu99 / g++ with -Wno-error=deprecated-declarations
Options:

ERROR: glib-2.22 gthread-2.0 is required to compile QEMU


Not sure which Cygwin packages are needed to satisfy those dependencies.

I opened a pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/833/preliminary-support-for-eosm2-and-ml-qemu/diff) that incorporates a few of these suggestions along with preliminary support for the EOSM2 in case any of this is useful.

[EDIT] That pull request had build system changes mixed with EOSM2 support. Best split them in separate pull requests. In the meantime I discovered that, "Builds with the normal Cygwin compiler are not supported. (http://wiki.qemu.org/Hosts/W32#Native_builds_with_Cygwin)" Even following those instructions I still haven't been able to build QEMU in Cygwin. In fact I've gotten just as far using the native (host) compiler. In both cases it ended like this:

/usr/include/w32api/winsock2.h:995:34: error: conflicting types for 'select'
   WINSOCK_API_LINKAGE int WSAAPI select(int nfds,fd_set *readfds,fd_set *writefds,fd_set *exceptfds,const PTIMEVAL timeout);
                                  ^
In file included from /usr/include/sys/types.h:68:0,
                 from /usr/include/time.h:28,
                 from /usr/include/glib-2.0/glib/gtypes.h:35,
                 from /usr/include/glib-2.0/glib/galloca.h:32,
                 from /usr/include/glib-2.0/glib.h:30,
                 from qga/commands-win32.c:14:
/usr/include/sys/select.h:73:5: note: previous declaration of 'select' was here
int select __P ((int __n, fd_set *__readfds, fd_set *__writefds,
     ^
qga/commands-win32.c: In function 'qmp_guest_file_open':


[EDIT] Put up pull requests for build system tweaks (https://bitbucket.org/hudson/magic-lantern/pull-requests/834/qemu-build-tweaks-1/diff) and EOSM2 (https://bitbucket.org/hudson/magic-lantern/pull-requests/835/eosm2-preliminary-setup/diff).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on June 04, 2017, 06:19:53 PM
Found out why the Mac is having problems with "install.sh" -- it is a well known issue with BSD grep that is installed on OSX not being able to use the "-P" option. Replacing it with GNU grep "fixes" the issue.

Another solution would be to rewrite the script so it doesn't use the "grep -P" option which might not be a bad idea because that is a "highly experimental" option:
       -P, --perl-regexp
              Interpret  the  pattern  as  a  Perl-compatible  regular  expression  (PCRE).  This is highly
              experimental and grep -P may warn of unimplemented features.


To replace grep on the Mac with Homebrew:brew tap homebrew/core; brew install grep --with-default-names


Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dmilligan on June 04, 2017, 08:57:34 PM
IIRC, if you can't use -P, you loose the vast majority of powerful parts of regex syntax (depending on what you're trying to do, it might not be possible to use grep at all, and so you would need to take on an extra dependency anyway). And I bet that statement about "highly experimental" was put in decades ago and never taken out. These more advanced parts of regular expression syntax are now quite standard.

There's also this: http://jlebar.com/2012/11/28/GNU_grep_is_10x_faster_than_Mac_grep.html
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 06, 2017, 11:56:47 PM
Some more updates:

- 5D3 menu navigation works*) (1.1.3 only)
- call/return trace: function arguments that look like strings are now identified
- experiment: include more debug info in ML elf files +)
- a few other minor tricks

*) To make this work, I had to port the GDB patch for date/time (easy to find in */debugmsg.gdb). On most (all?) other models, this patch is not really needed, because the date/time dialog is easy to bypass (just click OK or Cancel). Not so on 5D3; figure out why.

The 70D also seems to react to this trick, although menu navigation locks up very quickly on this model...

(http://builds.magiclantern.fm/jenkins/view/Experiments/job/QEMU-tests/ws/qemu/tests/5D3/menu10.png)

+) This enables translation from binary address to source code line, for example:

# ML must be compiled from the qemu branch (for now)
hg up qemu -C
cd magic-lantern/platform/500D.111
make clean; make

# pick some address above RESTARTSTART
eu-addr2line -s -S -e magiclantern 0x4F000
meminfo_display+0x2bc
mem.c:1149


More to come.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 07, 2017, 01:38:22 AM
Some more:

- parameterized MPU spells *)
- description of known MPU messages
- log timestamps are now used to help deciding the relationships between MPU spells +)
- build fixes for Mac (dfort)

*) Many MPU messages represent properties (for example: PROP_ICU_UILOCK is 06 05 04 01 nn 00, where "nn" is the value of the property). So, rather than including a "spell" for every possible value, it makes sense to define a template: in this case, whenever our emulated MPU receives 06 05 04 01 something 00, it should reply the same message back. To define such behavior, you may use ARG0 to ARG3 to define parameters; for example:

    { { 0x06, 0x05, 0x04, 0x01, ARG0, 0x00 }, {
        { 0x06, 0x05, 0x04, 0x01, ARG0, 0x00 },
        { 0 } } },


That's a trivial example, where the reply simply echoes the same message. Not all messages are the same; for example, for changing the GUI state:

    { { 0x06, 0x05, 0x04, 0x00, ARG0, 0x00 }, {                 /* NotifyGUIEvent(ARG0) */
        { 0x06, 0x05, 0x04, 0x00, ARG0, 0x01 },                 /* PROP_GUI_STATE(ARG0) */
        { 0 } } },


In other words, whenever it receives 06 05 04 00 something 00 (which is a request for changing GUI state), it replies back with 06 05 04 00 that_something 01 (which probably means there's green light for changing GUI state, or something like that).

+) Recent DebugMsg logs include a timestamp; this is now used to decide the relationship between MPU spells (previously, this was decided solely based on the sequence of the mpu_send/recv calls, with some exceptions for e.g. button messages). Only messages received shortly after a mpu_send call are now considered replies; messages that arrive later are assumed to be external inputs and commented out. Experimental; only 5D3, 5D2 and 50D have this enabled currently.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on June 07, 2017, 09:17:29 PM
Quote from: a1ex on June 07, 2017, 01:38:22 AM
- build fixes for Mac (dfort)

Thanks! I'm still working on getting the EOSM2 setup by following your QEMU tutorial (http://www.magiclantern.fm/forum/index.php?topic=15895.msg185089#msg185089) on that port -- you aren't going to let me get off easy are you?  :D

QEMU on Mac seems to be working. The most success I've been having is with the 550D (I don't have 500D ROMs):

(https://c1.staticflickr.com/5/4216/34352621253_0af2cae91d_z.jpg)

I looked into mount.sh and got it working on OS X (kpartx isn't available on Mac):

#!/usr/bin/env bash

echo "This will mount sd.img and cf.img as a loopback device."
echo "Please enter your password (of course, after reviewing what this script does)."

if [ $(uname -s) == "Darwin" ]; then
    sudo hdiutil attach sd.img
    sudo hdiutil attach cf.img
    echo "Done."
    echo "To remove the device mappings, run:"
    echo '   sudo hdiutil detach "/Volumes/EOS_DIGITAL"'
    echo '   sudo hdiutil detach "/Volumes/EOS_DIGITAL 1"'
else
    sudo kpartx -av sd.img
    sudo kpartx -av cf.img
    echo "Done."
    echo "To remove the device mappings, run:"
    echo "   sudo kpartx -dv sd.img"
    echo "   sudo kpartx -dv cf.img"
fi


I would submit a pull request for this but it seems that on the Mac it isn't necessary to mount as root. I also realized that when running QEMU via ./run_canon_fw.sh the first thing that script does is to make sure sd.img and cd.img aren't mounted. That script uses losetup which is yet another one that isn't available on the Mac. It displays an error but that doesn't seem to cause any problems. I've run QEMU with and without sd.img mounted and it doesn't seem to make any difference so maybe we just leave this alone for now?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 07, 2017, 09:22:47 PM
Quote from: dfort on June 07, 2017, 09:17:29 PM
I've run QEMU with and without sd.img mounted and it doesn't seem to make any difference

It will make a difference as soon as both you and the camera code will write something to the SD image during an emulation session. I did that by mistake a while ago and ended up with an unusable image (had to re-create it, copy ML again etc).

Additionally, at least under Linux, the data you write to some mounted directory is not flushed right away to disk. So, if you copy autoexec.bin or other files on the partition, you may find out they are not fully visible to the guest OS.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on June 08, 2017, 01:07:38 AM
I see, so we should check if /Volumes/EOS_DIGITAL* exists on OS X.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 11, 2017, 12:34:21 AM
More updates:

* Monitor console available by default as a UNIX socket; that means, during emulation you can interact with it with netcat (for quick commands or from a script), or with socat (for interactive console):

echo "log io" | nc -U qemu.monitor
socat - UNIX-CONNECT:qemu.monitor


* Log DebugMsg calls without GDB (very fast; credits go to @nkls - I've used a modified version of his initial DebugMsg hook).

./run_canon_fw.sh 5D3,firmware="boot=0" -d debugmsg


To use it on plain Canon firmware (any shell):
env QEMU_EOS_DEBUGMSG=0x5b90 ./run_canon_fw.sh 5D3,firmware="boot=0" -d debugmsg
or, with ML loaded (requires bash):
. ./export_ml_syms.sh 5D3.113
./run_canon_fw.sh 5D3,firmware="boot=1" -d debugmsg


* Verbose stack trace (to see where each message is coming from), for both -d debugmsg and GDB scripts (DIGIC 4-6). Example for the former:

./run_canon_fw.sh 5D3,firmware="boot=0" -d debugmsg,callstack,v

Current stack: [14ff80-14ef80] sp=14fed8                                         at [FileMgr:5b90:ff0f9684]
0x17B60(51ec48 &"TaskClass", 17b60, 19980218, 19980218)                          at [FileMgr:de48:14ff78] (pc:sp)
0xFF11B818(51ea28 &"FileMgr", 6, 0, 2)                                          at [FileMgr:17bbc:14ff50] (pc:sp)
  0x178B4(51ec1c &"StateObject", 51ea28 &"FileMgr", 6, 0)                        at [FileMgr:ff11b844:14ff38] (pc:sp)
   0x178EC(51ec1c &"StateObject", 51ea28 &"FileMgr", 6, 0)                       at [FileMgr:178e4:14ff28] (pc:sp)
    0xFF2C8F5C(51ea28 &"FileMgr", 0, 2, ff2c8f5c)                                at [FileMgr:1796c:14ff08] (pc:sp)
     0xFF0C5194(10, 0, 24, ff0c5194)                                             at [FileMgr:ff2c9050:14fef0] (pc:sp)
      0x5B90(0, 3, ff0f9784 "[SEQ] NotifyComplete (Cur = %d, %#x, Flag = %#x)", 4)
                                                                                 at [FileMgr:ff0f9680:14fed8] (pc:sp)
[     FileMgr:ff0f9680 ] (00:03) [SEQ] NotifyComplete (Cur = 4, 0x10, Flag = 0x10)


This tool is very powerful - rather than hunting for several minutes/hours to see where some error message might be coming from, you now get the answer in seconds (example (http://www.magiclantern.fm/forum/index.php?topic=17969.msg187160#msg187160)).

* Thorough consistency check to make sure the stack trace gives the same information as if you would follow the call/return trace manually.

More to come (regarding 1300D, digic 6, memory checking, automatic testing of ML builds). Most of these were written some time ago, but it takes a while to integrate everything and make sure they pass the test suite. Though slow, this approach does catch a lot of bugs very early, and I hope to have soon the tools to use a similar development approach for the main ML codebase.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 14, 2017, 01:14:29 PM
Some more:

- 650D GUI (come on guys, it was as simple as copying some 700D definitions and trying it...)
- EOSM2 GUI (see this walkthrough (http://www.magiclantern.fm/forum/index.php?topic=15895.10), thanks dfort for following it along)
- 1300D: minor updates (http://www.magiclantern.fm/forum/index.php?topic=17969.msg187160#msg187160)
- 70D menu tests (they don't work all the time, figure out why)
- 5D4 emulation updated for latest firmware

- splitgdb.sh updated to use cgdb (http://www.magiclantern.fm/forum/index.php?topic=15895.msg186245#msg186245) (installing it from source is recommended)

- run_ml_all_cams.sh can now be used to compile and run ML in QEMU with various options, very customizable

To get started: this compiles ML for 500D and 60D, copies each build to the virtual SD image, runs it for 10 seconds and takes a screenshot.

env ML_PLATFORMS="500D.111/ 60D.111/" TIMEOUT=10 SCREENSHOT=1 ./run_ml_all_cams.sh


Internally, this is how the emulator is invoked:

(
  sleep 10
  echo screendump 60D.111.ppm
  echo quit
) | (
  arm-none-eabi-gdb -x 60D/patches.gdb &
  ./run_canon_fw.sh 60D,firmware='boot=1' \
      -display none -monitor stdio  -s -S
) &> 60D.111.log


More examples: EOSM2 hello world (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-EOSM2/18/console), or running ML from the dm-spy-experiments branch in the emulator (QEMU-dm-spy (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-dm-spy/65/consoleFull)).




- track direct jumps in call stacks (widely used on DIGIC 6)

Example for 80D:

# in 80D/debugmsg.gdb
macro define PRINT_CALLSTACK 1

./run_canon_fw.sh 80D,firmware="boot=0" -s -S -d callstack & arm-none-eabi-gdb -x 80D/debugmsg.gdb
...
0xFE0D3385(0, fe0d3385, 19980218, 19980218)                                      at [init:8000173d:2e9108] (pc:sp)
0xFE533439(fe0d76f9, 0, fe0d76e5, fe0d76e7)                                     at [init:fe0d407d:2e90f0] (pc:sp)
  0xFE209159(fe0d76f9, 0, fe0d76e5, fe0d76e7)                                    at [init:fe53344f:2e90d8] (pc:sp)
   0xFE3FC0DC -> 0xFF1(fe209254 "PowerMgr", 20, 400, fe2090e1)                   at [init:fe209175:2e90c8] (pc:sp)
[      init:fe209175 ] task_create(PowerMgr, prio=20, stack=400, entry=fe2090e1, arg=0)


On the last line, 0xFE3FC0DC is an ARM wrapper (veneer) for 0xFF1 task_create. Wrappers like this are used in D6 code all over the place. Sometimes there are many different wrappers for the same functions, or even wrappers to wrappers to wrappers - these make the function calls very hard to track down.

Here's an example:

b *0xFE483D42
commands
  silent
  print_current_location_with_callstack
  printf "sei\n"
  c
end



Current stack: [2e9118-2e8118] sp=2e9000                                         at [init:fe483d42:fe483f2d]
0xFE0D3385(0, fe0d3385, 19980218, 19980218)                                      at [init:8000173d:2e9108] (pc:sp)
0xFE3FC204 -> 0x800035E1(8b, 16, fe0d4790 "\n%s ICU Firmware Version %s ( %s )", fe0d39a8)
                                                                                 at [init:fe0d43e9:2e90f0] (pc:sp)
  0x8000579B(3cc000, 0, 2e9044, 2e9048 "[STARTUP] \nK350 ICU Firmware Version 1.0.1 ( 6.2.2 )")
                                                                                 at [init:8000367f:2e9040] (pc:sp)
   0x800061F5(3cc084, 0, ffffffff, 3cc0d4)                                       at [init:8000581b:2e9020] (pc:sp)
    0x800051DC -> 0xFE483F0D(3cc084, 0, ffffffff, 3cc0d4)                        at [init:80006201:2e9010] (pc:sp)
     0xFE3FC00C -> 0xCA9 -> 0x168B -> 0xFE483D43(0, 96, ffffffff, 3cc0d4)        at [init:fe483f29:2e9000] (pc:sp)
[      init:fe483f29 ] sei


Look at those functions from the last line - they do nothing but jumping to the next one:

FE3FC00C 04 F0 1F E5                 LDR             PC, =(sub_CA8+1)

00000CA8 00 F0 EF BC                 B.W             sub_168A

0000168A 43 F6 43 5C+                MOV             R12, #0xFE483D43
00001692 60 47                       BX              R12 ; sub_FE483D42

FE483D42 EF F3 00 81                 MRS.W           R1, CPSR
FE483D46 21 F0 80 01                 BIC.W           R1, R1, #0x80
...


Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 15, 2017, 05:31:51 PM
More:

- "-d debugmsg" now just works; you no longer need to setup additional stuff, just run something like:


./run_canon_fw.sh 60D,firmware="boot=0" -d debugmsg


- stack trace also available (in a more limited format) without instrumentation; the full stack trace ("-d callstack") requires instrumentation to detect function calls, so it's slower, but also captures function arguments and handles difficult cases better.

To show the stack trace (with any of the two methods):
* it's always enabled for assert in GDB scripts
* "macro define PRINT_CALLSTACK 1" in GDB scripts enables it for all logged functions
* "print_current_location_with_callstack" at GDB prompt
* "-d debugmsg,v" or "-d debugmsg,callstack,v" on the command line (very verbose)
* from custom code, just call eos_callstack_print_verbose

Internally, the two methods were cross-checked to make sure they give the same results or fail gracefully, on every single function call until GUI boot.

The non-instrumented stack trace is also usable in ML, on the camera (e.g. in crash logs) (http://www.magiclantern.fm/forum/index.php?topic=19933.0).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on July 16, 2017, 06:41:51 AM
Got a really noob QEMU question.

Is it possible to bring up the ML menus using the Trash button? If so, where is the Trash button?

Maybe the Trash button is currently an unknown MPU spell? (If that's the right term.)

I've got the 550D and 1100D working with all sorts of builds including nightlies from the jenkins server but I haven't figured out how to bring up the ML menus.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dmilligan on July 16, 2017, 07:39:34 PM
You probably had trouble finding it b/c you are on a Mac right? Most Mac keyboards lack a dedicated forward delete key. Try: Fn + Delete
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on July 16, 2017, 08:51:54 PM
Hi @dmilligan -- Yep, I'm on a Mac. No combination of delete with a modifier key is showing a key event in QEMU. Though hitting random keys brought up this interesting screen:

(https://farm5.staticflickr.com/4306/35964513615_b3518f51c0.jpg)

That's trying to run a full ML on the 1100D and bring up the ML menus. I've been successful running a minimal "Hello World" build but a full ML build doesn't get all the way to the Canon menu. Oh well, Have fun!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 16, 2017, 09:44:19 PM
1100D doesn't have a dedicated "delete" button - it uses Av, which is not emulated in QEMU, and it's not even a regular button - the event it sends is actually a refresh of the info display (Canon calls this OLC). So, we use a very weird heuristic to detect short presses (and I'm not even sure how reliable it works).

PoC: https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-ML-menu/

(still has a few bugs in it)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on July 17, 2017, 06:02:06 PM
That is so cool:

700D in QEMU
(https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-ML-menu/lastSuccessfulBuild/artifact/qemu/700D.gif)

So is the next step adapting Canon EF lenses to the computer?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 18, 2017, 07:36:59 PM
Quote from: dfort on July 17, 2017, 06:02:06 PM
So is the next step adapting Canon EF lenses to the computer?

Joking aside, you've probably noticed the Debug -> Lens Info menu is actually usable on some models. This info comes from MPU messages.

That means, you can use the startup-log builds (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log/) while starting the camera with various lenses, and find out which parts of the log are lens-specific.

If we want to log things such as changing focus distance or focal length in LiveView (where these are reported back to the ARM CPU), we have a small issue: the log buffer fills up really quickly in LiveView. It may make sense to prepare a different set of builds, with reduced verbosity (or maybe skip logging unimportant messages once e.g. half of the buffer is full). Will look into that.

I should also make the mpu_send/recv stubs mandatory, as they are now known to work identically in all models, from digic 2 to digic 6 (not sure about 7 yet). And I'll need such logs from all cameras anyway.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: BBA on July 18, 2017, 08:21:49 PM
Sigma firmware updates can be downloaded for their lenses on Canon EF mount.... Don't know which processor they use...What do you think ?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 18, 2017, 08:29:40 PM
Sorry, no experience with Sigma lenses.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on August 07, 2017, 11:47:09 PM
More stuff:

- found a way to get deterministic execution traces (-icount option in QEMU); a lot of tests were failing because of timing variations and differences in host CPU speed
- a test covering the entire call/return trace, from start to booting the GUI
- option to identify tail function calls ("-d calls,tail" or "-d callstack,tail")
- incomplete 5D3 1.2.3 GUI emulation (boots with black screen, ML loads fully, but no menus)
- option to highlight certain MMIO registers (hardcoded (https://bitbucket.org/hudson/magic-lantern/commits/5792b77445568093f368a6e65f63322d684c03c9))
- helper to cross-check MMIO register values with the ones from actual hardware (see this commit (https://bitbucket.org/hudson/magic-lantern/commits/726806f3bc352c41bbd72bf40fdbab3c7245039d) for usage notes)
- experimental UART emulation in main firmware (Dry-shell, eventproc shell - like this (https://nada-labs.net/2014/finding-jtag-on-a-canon-elph100hs-ixus115/); examples for 5D3 (https://builds.magiclantern.fm/jenkins/view/Experiments/job/QEMU-tests/ws/qemu/tests/5D3/drysh.log), 70D (https://builds.magiclantern.fm/jenkins/view/Experiments/job/QEMU-tests/ws/qemu/tests/70D/drysh.log), 500D (https://builds.magiclantern.fm/jenkins/view/Experiments/job/QEMU-tests/ws/qemu/tests/500D/drysh.log)). See also on CHDK forum (https://chdk.setepontos.com/index.php?topic=11029).

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/drysh.png)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 13, 2017, 01:18:08 PM
As the current state is pretty much usable and most of the tests are passing (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/lastSuccessfulBuild/consoleFull), I'm ready to merge it into mainline. Besides the emulation (which is installed out of the ML tree), the "qemu" branch also provides a debugging API, explained below. I've used this API for the new DryOS task hooks (https://bitbucket.org/hudson/magic-lantern/pull-requests/672/dryos-task-hooks-for-newer-cameras-6d-70d/diff) (6D, 100D, 70D and EOS M2), so these new ports (all but 6D) depend on "qemu" being merged first.

I just need a second pair of eyes to look over it and make sure:
1) there's nothing broken in the main builds (build system, functionality);
2) the debugging API works as described below.




The main debugging function is qprintf (and its lightweight friends: qprint, qprintn and qdisas). Feel free to use them *anywhere* - they won't get compiled in regular builds (therefore they won't increase the executable size). These functions will print to QEMU console whenever ML (or a subset of it) is compiled with CONFIG_QEMU=y. Example:


cd platform/550D.109
make clean; make                  # regular build
make clean; make CONFIG_QEMU=y    # debug build for QEMU


It works for modules as well:

cd modules/lua
# add some qprintf call in lua_init for testing
make clean; make                  # regular build
make clean; make CONFIG_QEMU=y    # debug build for QEMU


(Side note: as we don't emulate ML shutdown properly yet, you'll have to enable Debug -> Load modules after crash. Solving this is a little above easy coding task level, but doable.)

You can also specify CONFIG_QEMU=y in Makefile.user - but as this is more likely to be used as a temporary option, the command line makes a little more sense to me.

The QEMU debugging API is header-only (qemu-util.h), auto-included by dryos.h. You can use it for either the entire ML, or just for a subset of it - e.g. the source file(s) you are currently editing, or only some modules. The lightweight functions can also be used in very early boot code, where you can't call vsnprintf or you may not even have a stack properly set up.

In a nutshell:

CONFIG_QEMU=n (default):
- regular build
- the executable works in QEMU (within the limitations of the emulation)
- no guest debugging code (no additional debugging facilities)

CONFIG_QEMU=y (optional, on the command line or in Makefile.user):
- debug build for QEMU only
- does not run on the camera (!)
- enables qprintf and friends to print on the QEMU console
- enables unlimited number of ROM patches - useful for dm-spy-experiments (in QEMU you can simply write to ROM as if it were RAM)
- may enable other workarounds for models that are not emulated very well

Additionally, you have better support for debugging ML at source level, in gdb (or any front-end you like). Uncomment the symbol-file line in your GDB script:

symbol-file ../magic-lantern/platform/80D.102/magiclantern

and you can now run ML code step by step in the debugger, or set breakpoints to any ML function:

b my_init_task


For debugging very early boot code (e.g. reboot.c), you'll have to use "autoexec" for symbol-file.

More details on debugging (http://www.magiclantern.fm/forum/index.php?topic=15895.msg186173#msg186173) on the EOS M2 thread.

Unfortunately I don't have a good solution for debugging modules in the same way...

Also started a README (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/) - proof-reading welcome :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on September 14, 2017, 02:14:09 AM
Hey, I'm happy!

700D.115 running in QEMU:

(https://farm5.staticflickr.com/4376/36400273823_3ee07e9fb5_z.jpg) (https://flic.kr/p/XsyVeZ)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kichetof on September 16, 2017, 04:02:25 PM
Hi guys, I'm happy too !

After trying for a long time to get QEMU working on my Mac, it works now!  8)
I finally find how to solve the pixman compilation error.. it was really simple  :o

pixman-mmx.c:100:20: error: constraint 'K' expects an integer constant expression
        : "y" (__A), "K" (__N)

Solved with: brew install pixman

50D 1.0.9
(https://s26.postimg.org/pt5djpiix/qemu_50_D_Magic_Lantern_Rescue_sucessful.png)
(https://s26.postimg.org/57888u7ex/qemu_50_D_unified_ML_homescreen.png)




5D3 1.1.3
(https://s26.postimg.org/3sp132zux/qemu_5_D3_113_Magic_Lantern_Rescue_sucessful.png)
Unable to play with Canon or ML menus, on startup I've "SD card test (0 --> 100)" and after test
(https://s26.postimg.org/tz7ug2oll/qemu_5_D3_113_bug_modules_loading_crop_rec_4k_ski.png)
Key has no effect. Try some branches and same bug, black screen (with 2 dots above "was not shut")
Output from terminal (https://pastebin.ubuntu.com/25547870/) (I'll find why grep give an error; line 50 from run_canon_fw.sh; same bug as @dfort found (http://www.magiclantern.fm/forum/index.php?topic=2864.msg185591#msg185591), grep on MacOS)

I've found why, you explain it here (http://www.magiclantern.fm/forum/index.php?topic=2864.msg185649#msg185649), I'll try with debugmsg.gdb




Could you try another keys for keyboards without a numpad ? On my MacBook Pro I don't have one and I can't found keys for Joystick (8 directions & center).

QEMU Readme give a great help to start with emulation! Thanks

Many thanks a1ex to make our DSLR and now our computer to be so cool  8)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 16, 2017, 06:42:02 PM
Very cool, glad to see it's working.

Ideally, those dependencies should be handled in the install script; will look into it.

You can customize the keys in mpu.c. To add alternate keys, see for example Shift, which accepts 2 key codes: left shift and right shift. My keyboard doesn't have a numpad either, but it does have the classic arrow keys - for menu navigation it's working well.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kichetof on September 16, 2017, 11:48:49 PM
Quote from: a1ex on September 16, 2017, 06:42:02 PM
Ideally, those dependencies should be handled in the install script; will look into it.

To fix bug with MacOS, we need to install GNU grep with brew install grep. Personally I don't want to replace MacOS version, so we need to use ggrep instead of grep when -P argument is required.



What do you think about specific cards by model? (to avoid some bugs with wrong build)

I customized run_canon_fw.sh like that:


SD="sd.img"
CF="cf.img"

if [ "$CAM" ] && [ -f sd-$CAM.img ]; then
    SD="sd-$CAM.img"
fi

if [ "$CAM" ] && [ -f cf-$CAM.img ]; then
    CF="cf-$CAM.img"
fi

# run the emulation
env QEMU_EOS_DEBUGMSG="$QEMU_EOS_DEBUGMSG" \
  $QEMU_PATH/arm-softmmu/qemu-system-arm \
    -drive if=sd,format=raw,file=$SD \
    -drive if=ide,format=raw,file=$CF \
    -chardev socket,server,nowait,path=qemu.monitor,id=monsock \
    -mon chardev=monsock,mode=readline \
    -M $*




Some custom on mpu.c (QWERTZ keyboard, simulate numpad with TZU/GHJ/VBN)


    { 0x0014,   BGMT_PRESS_UP_LEFT,     "T",            "Joystick Up Left",             },
    { 0x0094,   BGMT_UNPRESS_UDLR,                                                      },
    { 0x002C,   BGMT_PRESS_UP,          "Z",            "Joystick Up",                  },
    { 0x00AC,   BGMT_UNPRESS_UDLR,                                                      },
    { 0x0016,   BGMT_PRESS_UP_RIGHT,    "U" ,           "Joystick Up Right",            },
    { 0x0096,   BGMT_UNPRESS_UDLR,                                                      },
    { 0x0022,   BGMT_PRESS_LEFT,        "G",            "Joystick Left",                },
    { 0x00A2,   BGMT_UNPRESS_UDLR,                                                      },
    { 0x0023,   BGMT_JOY_CENTER,        "H",            "Joystick center",              },
    { 0x00A3,   BGMT_UNPRESS_UDLR,                                                      },
    { 0x0024,   BGMT_PRESS_RIGHT,       "J",            "Joystick Right",               },
    { 0x00A4,   BGMT_UNPRESS_UDLR,                                                      },
    { 0x002F,   BGMT_PRESS_DOWN_LEFT,   "V",            "Joystick Down Left",           },
    { 0x00AF,   BGMT_UNPRESS_UDLR,                                                      },
    { 0x0030,   BGMT_PRESS_DOWN,        "B",            "Joystick Down",                },
    { 0x00B0,   BGMT_UNPRESS_UDLR,                                                      },
    { 0x0031,   BGMT_PRESS_DOWN_RIGHT,  "N",            "Joystick Down Right",          },
    { 0x00B1,   BGMT_UNPRESS_UDLR,                                                      },


Nothing happen (50D) when I press Joystick center (I try to play with settings (aperture, speed, ISO,..), for testing :))
Key event: 23 -> 0b01
[MPU] Sending : 06 05 06 0b 01 00
[MPU] Received: 06 05 04 00 09 00  (NotifyGUIEvent - spell #44)
[MPU] Sending : 06 05 04 00 09 01
[MPU] Received: 08 06 00 00 04 00 00 00  (unknown spell)
  1212:  2740.224 [MC] PROP_GUI_STATE 9
  1231:  2740.224 [MC] cam event guimode comp. 9
  1278:  2745.600 [DISP] TurnOnDisplay (PUB) Type=0 fDisplayTurnOn=1
[MPU] Received: 06 05 03 19 00 00  (spell #37)
Key event: a3 -> 0b00
[MPU] Sending : 06 05 06 0b 00 00
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 17, 2017, 07:12:34 AM
Quote from: kichetof on September 16, 2017, 11:48:49 PM
What do you think about specific cards by model?

If you have enough free space on the disk, it's a good idea (so it's best to keep it optional). I prefer storing all model-specific files under the $CAM/ directory (where the ROMs are), but otherwise, your change looks fine. Maybe even enabling only one card in the emulation.

Quote
Nothing happen (50D) when I press Joystick center (I try to play with settings (aperture, speed, ISO,..), for testing :))

I don't expect this to work, as it requires MPU communication. If it's a simple protocol, such as sending something about ISO and replying back the same code, it can be solved with custom MPU spells (http://www.magiclantern.fm/forum/index.php?topic=2864.msg185653#msg185653); if it requires some state, it's more difficult, as the current mpu.c is suitable mostly for replaying known messages.

The first step would be to log Canon messages during these actions (using the startup-log build (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log/) or dm-spy-experiments branch) and understand what messages are for ISO, shutter and aperture, and how these should look like when changing them. Note the change may be initiated from the MPU (when pressing the buttons) or from the main CPU (when changing these parameters from ML menu).

These messages can be cross-checked with Leegong's notes (http://www.magiclantern.fm/forum/index.php?topic=17596).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kichetof on September 17, 2017, 09:52:28 AM
Quote from: a1ex on September 17, 2017, 07:12:34 AM
I prefer storing all model-specific files under the $CAM/ directory (where the ROMs are), but otherwise, your change looks fine. Maybe even enabling only one card in the emulation.

It will be more beautiful to store under $CAM directory and name it cf.img and sd.img. I'll adapt my script to check in $CAM dir.
To use only one card, we need to know which cam use only SD, SD+CF and only CF to send the right card. Or maybe we can simulate one or the other.

Quote from: a1ex on September 17, 2017, 07:12:34 AM
I don't expect this to work, as it requires MPU communication. If it's a simple protocol, such as sending something about ISO and replying back the same code, it can be solved with custom MPU spells (http://www.magiclantern.fm/forum/index.php?topic=2864.msg185653#msg185653); if it requires some state, it's more difficult, as the current mpu.c is suitable mostly for replaying known messages.

The first step would be to log Canon messages during these actions (using the startup-log build (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log/) or dm-spy-experiments branch) and understand what messages are for ISO, shutter and aperture, and how these should look like when changing them. Note the change may be initiated from the MPU (when pressing the buttons) or from the main CPU (when changing these parameters from ML menu).

These messages can be cross-checked with Leegong's notes (http://www.magiclantern.fm/forum/index.php?topic=17596).

Thanks for all these informations! I'll learn how to get and interpret these messages. Lot of stuff to read and learn  ;D
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 22, 2017, 07:20:28 PM
Updates:

- emulation is now used to run some automated tests on ML nightly builds (http://www.magiclantern.fm/forum/index.php?topic=20560) (just scratching the surface, but already found a couple of bugs in ML)
- clean shutdown on most models (menu: Machine -> Power Down); 70D is the black sheep here (not sure what the issue is)
- 450D and 1000D GUIs (based on logs from Ant123). For other VxWorks models, you'll have to customize this (https://bitbucket.org/hudson/magic-lantern/branch/vxworks-dm-spy) and get a startup log with mpu_send/recv calls (can be debugged in QEMU).
- 400D GUI starts without any MPU emulation (!), but it's stuck (does not react to any key presses)

(https://builds.magiclantern.fm/jenkins/job/QEMU-tests/183/artifact/qemu/tests/1000D-menu.png)

Still looking for some help with proof-reading the README (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/), to merge it into mainline (EOSM2, 100D and 70D are waiting for it (http://www.magiclantern.fm/forum/index.php?topic=15895.msg189856#msg189856)).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 27, 2017, 09:49:26 PM
Currently experimenting with an updated toolchain, in the qemu branch. I'm following this guide (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/) (actually I'm testing it on a fresh Ubuntu VM):


sudo apt install mercurial
hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg update qemu -C
cd contrib/qemu
./install.sh


This asked me for the sudo password to install the dependencies.

(http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/5-enter.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/18-type.png)

After installing some 800MB worth of packages, I've copy/pasted the QEMU compilation commands printed by install.sh (they will be different on your system):

cd /home/alex/qemu/qemu-2.5.0
../configure_eos.sh
make -j1


(http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/35-install-instructions.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/38-wait.png)

Copied the ROM files for 5D3 1.1.3, and ran:

./run_canon_fw.sh 5D3


(http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/54-type.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/61-qemu-5D3-test.png)

For Canon GUI, we need to run under GDB (patches.gdb):

./run_canon_fw.sh 5D3,firmware="boot=0" -s -S & arm-none-eabi-gdb -x 5D3/patches.gdb


(http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/71-qemu-5D3-gui.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/160-qemu-5D3-menu.png)


Then compiled ML and ran it in QEMU:

make -C ../magic-lantern/platform/5D3.113 install_qemu
./run_canon_fw.sh 5D3,firmware="boot=1" -s -S & arm-none-eabi-gdb -x 5D3/patches.gdb


(http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/91-qemu-5D3-ml-menu.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/140-right.png)

Enabled the Lua module, restarted the virtual camera cleanly (Machine -> Power Down) ...
(http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/111-delete.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/115-qemu-5D3-ml-poweroff.png)

... and ran the Hello World and Sokoban scripts:
(http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/139-wait.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/124-qemu-5D3-lua-sokoban.png)

Yes, it was that simple 8)

Animation (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/xenial.gif) (3MB)

Scripts used (should you want to re-create the above scenario, maybe on another OS):
qemu-demo-xenial.sh (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/qemu-demo-xenial.sh) and anim.py (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/anim.py)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Licaon_Kter on September 28, 2017, 12:43:40 PM
Two ideas:

1. Maybe download GCC from the new site: https://developer.arm.com/open-source/gnu-toolchain/gnu-rm/downloads
wget -c https://developer.arm.com/-/media/Files/downloads/gnu-rm/5_4-2016q3/gcc-arm-none-eabi-5_4-2016q3-20160926-$OS.tar.bz2 && \

2. Why GCC 5.4.1 and not the latest (available there) GCC 6.3.1 ? I build with it and have not encountered any issues, but then again I only build for EOS M :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 28, 2017, 12:50:05 PM
Noticed the new site after committing :D

GDB from the GCC 6.x package (only available as 64-bit for Linux) is unable to run some of the scripts required for booting the Canon GUI in QEMU; the above scenario only works with 32-bit GDB (available in the GCC 5.x package). Maybe it's time to report a bug to GDB (ideally we should find a way to reproduce with non-proprietary code and on vanilla QEMU).

This is an issue under Windows 10 - WSL (http://www.magiclantern.fm/forum/index.php?topic=20214.msg190537#msg190537) (where 32-bit Linux binaries do not work) and also on Mac (where only 64-bit builds are available).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: DeafEyeJedi on September 28, 2017, 09:16:59 PM
Rather than double posting (or perhaps should have posted in here first) but here's my first attempt at compiling/installing QEMU (http://www.magiclantern.fm/forum/index.php?topic=16012.msg190670#msg190670first%20attempt%20at%20compiling/installing%20QEMU) which seem to have failed under my stupidity since I am unable to empathize what 'please call configure before running make' means?

I'd really like to try and get this whole QEMU experiment under my belt. It's been long overdue. Thanks for any help!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 29, 2017, 01:08:18 PM
Playing with different toolchains on Ubuntu (Xenial 64-bit):

1) gdb-arm-none-eabi:i386 and gcc-arm-none-eabi from Ubuntu repo (gcc 4.9 64-bit, 32-bit gdb): animation (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/xenial1.gif) (3MB)
2) 32-bit gcc-arm-embedded (gcc-arm-none-eabi-5_4-2016q3): animation (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/xenial2.gif) (3MB)
3) gdb-arm-none-eabi and gcc-arm-none-eabi from Ubuntu repo (gcc 4.9, 64-bit): animation (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/xenial3.gif) (3MB) - using 60D without GDB
4) gcc-arm-embedded from ppa:team-gcc-arm-embedded/ppa (gcc 6.x, 64-bit): TODO (need to use a different camera - 5D3 requires 32-bit GDB)

Scripts used (should you want to re-create the above scenario, maybe on another OS):
qemu-demo-xenial1.sh (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/qemu-demo-xenial1.sh)
qemu-demo-xenial2.sh (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/qemu-demo-xenial2.sh)
qemu-demo-xenial3.sh (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/qemu-demo-xenial3.sh)
anim.py (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/anim.py) (to render the animation)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 30, 2017, 11:12:54 PM
Some good news for those affected by the 64-bit GDB bug:

- Found out why 5D3 GUI wasn't coming up without patching the date/time: it was waiting for... PROP_MPU_GPS (06 04 03 54 00 00 from MPU). After this change, 5D3 GUI booted without GDB (and the date/time dialog could be bypassed by clicking OK)!
- Even better - g3gg0 figured out how to emulate the real-time clock (https://bitbucket.org/hudson/magic-lantern/commits/0d654a0836629042a28289562ff83b44d1cfe270)! This change superseded a bunch of GDB scripts (https://bitbucket.org/hudson/magic-lantern/commits/93e57ce578026f50329a92f35b01a09b8474487a) - no more need to patch the date/time dialog!
- Exception: 5D2 and 50D appear to use a different RTC chip edit: g3gg0 just solved it!
- EOS M boots Canon GUI (with the same limitations as EOS M2 (http://www.magiclantern.fm/forum/index.php?topic=15895.msg187913#msg187913))
- EOS M and M2 have the date/time dialog left enabled on purpose (to prevent the camera from entering LiveView, which is not emulated)

That means:
- The good news: to boot the GUI, you no longer need GDB for most models (exceptions: M and M2)
- The date/time dialog at startup is gone! (exceptions: 5D2, 50D, M and M2)
- More Canon menus navigable without locking up on DIGIC 5 models (because of that GPS property...)
- The bad news: if you want to do actual debugging in GDB, you will need a 32-bit arm-none-eabi-gdb.

Current status:
- Models able to run the GUI and navigate Canon menu: 16 (most DIGIC 4 and 5, some DIGIC 3).
- Models with major GUI issues: 70D, 5D3 1.2.3.
- Models unable to run the GUI: 6D (help needed (https://www.magiclantern.fm/forum/index.php?topic=15088.msg186141#msg186141)), 7D (hard), all DIGIC 6 models, most VxWorks models.

Final note: if you have QEMU already set up, I recommend installing the new one from scratch - the install script will not delete old patches.gdb files. Or just delete these files manually.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kichetof on October 01, 2017, 12:04:38 AM
Guys, you're awesome!!  8)
Canon 5D3 113 on MacOS High Sierra! Happy!

(https://s25.postimg.org/bagc247nz/5_D3-113-mac_OS.gif)

@a1ex no module menu, need to enable Debug -> Modules debug -> Load modules after crash; reboot and module menu appear
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 01, 2017, 12:42:23 AM
@kichetof: "Common issues and workarounds" in the README (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/) (or watch my animations)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: DeafEyeJedi on October 01, 2017, 03:07:09 AM
Looking great out there @kichetof! Could use some of your beers. :P

Quote from: a1ex on September 30, 2017, 11:12:54 PM
Some good news for those affected by the 64-bit GDB bug:

Great progress @a1ex!

Quote from: a1ex on September 30, 2017, 11:12:54 PM
Final note: if you have QEMU already set up, I recommend installing the new one from scratch - the install script will not delete old patches.gdb files. Or just delete these files manually.

Could you pleaase shed some light on this? It seems I am pretty close to getting the QEMU to emulate on OS X 10.13 (I can see the black window popping up briefly before it disappears) with this message below:

./run_canon_fw.sh 100D
DebugMsg=0x4A74 (from GDB script)
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
./run_canon_fw.sh 100D,firmware=boot=1
DebugMsg=0x4A74 (from GDB script)
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
./run_canon_fw.sh 100D,firmware=boot=1
DebugMsg=0x4A74 (from GDB script)
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - DFFFFFFF: eos.iomem
[EOS] loading './100D/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './100D/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
Could not open ./100D/SFDATA.BIN
Seans-Mac-mini-385:qemu DeafEyeJedi$


Especially the 'Could not open ./100D/SFDATA.BIN' part? BTW I did make this virtual SD bootable w Macboot. So close I can feel it!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on October 01, 2017, 03:55:50 AM
Quote from: DeafEyeJedi on October 01, 2017, 03:07:09 AM
Could not open ./100D/SFDATA.BIN
Seans-Mac-mini-385:qemu DeafEyeJedi$


Especially the 'Could not open ./100D/SFDATA.BIN' part? BTW I did make this virtual SD bootable w Macboot. So close I can feel it!

Do you have your serial flash dump (SFDATA.BIN) next to your ROM0.BIN and ROM1.BIN files?

You don't need to make the virtual SD bootable with Macboot--where did you read that? Maybe it is corrupted now? No big deal, if you get into trouble or want to try out the latest QEMU changes simply delete your qemu directory (or rename it if you want to save it) and re-run the install.sh script in magic-lantern/contrib/qemu.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: DeafEyeJedi on October 01, 2017, 07:53:13 AM
Quote from: dfort on October 01, 2017, 03:55:50 AM
Do you have your serial flash dump (SFDATA.BIN) next to your ROM0.BIN and ROM1.BIN files?

Actually I do not. This is where I hit a wall. Not sure where I can actually get the serial flash dump -- care to refresh my memory in here, please?

(https://farm5.staticflickr.com/4494/23571161168_0fa35a2de6.jpg) (https://flic.kr/p/BUUn1u)

Quote from: dfort on October 01, 2017, 03:55:50 AM
You don't need to make the virtual SD bootable with Macboot--where did you read that? Maybe it is corrupted now? No big deal, if you get into trouble or want to try out the latest QEMU changes simply delete your qemu directory (or rename it if you want to save it) and re-run the install.sh script in magic-lantern/contrib/qemu.

Thought I read it somewhere that we had to make sure the virtual SD mount was bootable before running QEMU on it or no?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on October 01, 2017, 08:37:58 AM
You need to run the sf_dump module to get a SFDATA.BIN dump and the virtual sd card already has the boot flag set. At least I never needed to run anything special to make it bootable.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: DeafEyeJedi on October 01, 2017, 09:01:20 AM
Quote from: dfort on October 01, 2017, 08:37:58 AM
You need to run the sf_dump module to get a SFDATA.BIN dump...

I read that. Clearly. Guess what I should have asked was how to get this sf_dump module? Can't seem to find it anywhere. Or maybe looking at the wrong directories?

Quote from: dfort on October 01, 2017, 08:37:58 AM
...and the virtual sd card already has the boot flag set. At least I never needed to run anything special to make it bootable.

Gotcha. Thanks for the clarification.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nikfreak on October 01, 2017, 09:10:13 AM
You can find sf_dump in modules directory: Check:
https://bitbucket.org/hudson/magic-lantern/src/08720b28c58db49ae4de925c17240cf49050c74f/modules/sf_dump/?at=unified
Title: How to run Magic Lantern into QEMU?!...
Post by: DeafEyeJedi on October 01, 2017, 10:50:17 AM
Thanks for pointing me to that @nikfreak! Actually ended up getting a copy from @dfort that I shared w him from last year or so -- whew good save on that one, right?   ;)

Anyhow, here's my 2nd attempt (more like 3rd or 4th, ha) at emulating QEMU on OS X 10.12.6 below...



...notice I ended up with a 'Camera was not shut down cleanly - Skipping module loading' message -- perhaps I didn't shut down the Emulator properly earlier and is there a way to 'turn off the camera' without having to force quit QEMU-system-arm manually?

and lastly what would be the 'trash' button on this keyboard? 

Because I have placed the required ML files in the virtual SD mount or at least seem to think so :P

(https://farm5.staticflickr.com/4354/37379134306_b738b2eb42_o.png) (https://flic.kr/p/YX4QmS)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 01, 2017, 01:37:45 PM
Quote from: DeafEyeJedi on October 01, 2017, 10:50:17 AM
...notice I ended up with a 'Camera was not shut down cleanly - Skipping module loading' message -- perhaps I didn't shut down the Emulator properly earlier and is there a way to 'turn off the camera' without having to force quit QEMU-system-arm manually?

Maybe this needs reworded in a different way? (sorry, not native English speaker)

Quote from: a1ex on October 01, 2017, 12:42:23 AM
@kichetof: "Common issues and workarounds" in the README (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/) (or watch my animations)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kichetof on October 02, 2017, 02:12:14 PM
Quote from: a1ex on October 01, 2017, 01:37:45 PM
Maybe this needs reworded in a different way?

It's perfect, but, is it possible to keep the warning on the screen when you run it with QEMU ?
In my case, I didn't see the warning (too stuff in parallels :)))
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 02, 2017, 02:24:49 PM
Good point, will see if that can be done without changing QEMU core code (as the shutdown behavior is hardcoded in every single GUI backend...)

edit: got it working :) atexit to the rescue...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on October 02, 2017, 02:56:06 PM
Quote from: DeafEyeJedi on October 01, 2017, 10:50:17 AM
...notice I ended up with a 'Camera was not shut down cleanly - Skipping module loading' message -- perhaps I didn't shut down the Emulator properly earlier and is there a way to 'turn off the camera' without having to force quit QEMU-system-arm manually?

If you get that message remove remove the LOADING.LCK from the ML/modules directory and the modules will load next time you start QEMU. I had to do that every time on the EOSM2 because I couldn't find a way to do a proper camera shutdown. Development on the qemu is moving quite rapidly so make sure you update your local repository and re-install QEMU if you encounter an issue--it might already be fixed!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kichetof on October 02, 2017, 03:06:19 PM
Quote from: a1ex on October 02, 2017, 02:24:49 PM
edit: got it working :) atexit to the rescue...

You're the best!  8)
If anymore ever complains about it, make a psychedelic flash of the warning  :P
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 02, 2017, 07:06:39 PM
A little Easter egg (tested on 100D and 700D, but likely on all other D5 models):

Start the camera, go to PLAY mode (without any image present) and press DELETE. Watch the console output:


ON_ERASE
open B:/AUTOEXEC.SC
Not Found B:/AUTOEXEC.SC
ffffffff


Have fun discovering the language! (hint (https://www.defcon.org/images/defcon-18/dc-18-presentations/Isacson-Ortega/DEFCON-18-Isacson-Ortega-Exploiting-Digital-Cameras-WP.pdf))
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: nikfreak on October 02, 2017, 07:49:58 PM
tried to follow http://chdk.wikia.com/wiki/Canon_Basic and used EOScard to set SCRIPT flag but somehow failing for the moment. Did you get it to work outside QEMU?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on October 02, 2017, 07:59:55 PM
Didn't try, but I bet this is not PowerShot Basic (so there's no point in following their guide).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on October 06, 2017, 11:01:32 PM
Quote from: a1ex on September 30, 2017, 11:12:54 PM
- Even better - g3gg0 figured out how to emulate the real-time clock (https://bitbucket.org/hudson/magic-lantern/commits/0d654a0836629042a28289562ff83b44d1cfe270)! This change superseded a bunch of GDB scripts (https://bitbucket.org/hudson/magic-lantern/commits/93e57ce578026f50329a92f35b01a09b8474487a) - no more need to patch the date/time dialog!

for the record, got information from ricoh that the 5D3 chip is a R2262K (https://www.e-devices.ricoh.co.jp/en/products/product_rtc/3wire/r2262/r2262-e.pdf)
(thanks, guys!)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 04, 2017, 05:37:55 PM
This is a really crazy idea that will probably never work but I'll ask anyway.

Would it be possible to update the firmware in QEMU?

(https://farm5.staticflickr.com/4586/24313078518_aef264f1b5.jpg) (https://flic.kr/p/D3sTbW)

Here is what happens on the 500D when selecting the "OK" button with the space bar:


Key event: 39 -> 0c01
[MPU] Sending : 06 05 06 0c 01 00  (GUI_SWITCH)
[MPU] Received: 06 05 04 01 7f 00  (PROP_ICU_UILOCK - spell #40)
[MPU] Sending : 06 05 04 01 7f 00  (PROP_ICU_UILOCK)
FF012F50: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
FF012F50: MCR p15, ...          : CACHEMAINT x5773 (omitted)
FF012F50: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC0051079
FF012F84: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC0051079
FF012F70: MCR p15, ...          : CACHEMAINT x1 (omitted)
FF012F84: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC0050079
FF012ED0: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC0050079
FF012EBC: MCR p15, ...          : CACHEMAINT x1 (omitted)
FF012ED0: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005007D
FF012EFC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005007D
FF012EE8: MCR p15, ...          : CACHEMAINT x1 (omitted)
FF012EFC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
[MPU] Received: 06 05 03 11 02 00  (unknown - unnamed)
[MPU] Received: 06 04 04 07 00 00  (unnamed - spell #45)
[MPU] Sending : 06 05 02 0b 00 00  (unnamed)
   938: 11598.080 [TERM] SHUTDOWN_REQUEST
   951: 11598.080 [LVCFG] PROP_LV_LOCK PROHIBIT
   952: 11598.336 [LV] JudgeStartLV 0x1 0x0 0xFFFF 2 0 0 5145
Key event: b9 -> 0c00
[MPU] Sending : 06 05 06 0c 00 00  (GUI_SWITCH)
[MPU] Received: 06 05 03 19 01 00  (PROP_TFT_STATUS - spell #47)
[MPU] Received: 06 05 02 0b 02 00  (unnamed - spell #46)
[MPU] Shutdown requested.


Looks like it sets up some registers then reboots the camera but in QEMU it simply shuts down.

Why would we want to do this? Getting a ROM dump on a Canon firmware update without actually updating the firmware on the camera would be an obvious reason though that might get complicated. Testing out a new .FIR would be another use though only a few developers can create .FIR files so they probably already have a way of testing them out. So the only good reason I could come up with is like that old mountaineering saying, "Because It's There."

[EDIT] .FIR files need to be signed so to get this to work you need access to a key that will most likely open Pandora's box. So let's just call this post a muse rather than a feature request.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on November 05, 2017, 02:48:34 PM
Hi,
a few days back a1ex asked me in the 750D thread to proofread his new README for the qemu branch.
Well as time is a premium I was able to do some testing only the other day. So here are some remarks, in what is hopefully the right thread to post them.

1. I was first testing with my 750D ROM files, which are not booting fully at the moment as we need to find some more places to patch away the endless waiting loops but with the instructions given in the readme and a looooot of hitting step on gdb that will be a nice task for cold winter evenings ;).

2. I wanted to understand better what is going on and managed to get my hands on an EOS 700D, compiled and installed ML and copied the ROM0 and ROM1 as described to the qemu/700D115 directory. Tried to start but it wasn't booting  :( as it was missing a SFDATA.BIN file.
So there is a missing section in the  README telling you to also compile the sf_dump Module and putting it on the ML Card and activate it and then Reboot the camera and use the module from the Debug Menu of ML. (only found it through full text search on the whole ML directory)

3. In the README under DEBUGGING you also write to use "make CONFIG_qemu=y" and the "make install_qemu" which wouldn't compile in the unified branch for the 700D. I found out that the qemu (or no dm_spy_experiments) branch is needed to use these options, maybe stress that out a bit more in the section.

These are my comments so far. Hope it helps.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 05, 2017, 06:45:26 PM
I've got several firmware update projects that seem to be working fine except I don't have access to a ML-SETUP.FIR in order to set the camera bootflag on and off  so I need to downgrade the firmware, change the bootflag then upgrade firmware again to check and see if ML is affecting the camera. 

Of course we can do this when launching QEMU:

Bootflag off, starts Canon menus.
500D,firmware="boot=0"

(https://farm5.staticflickr.com/4499/24339288518_4d3a5d02eb_n.jpg) (https://flic.kr/p/D5Mdvu)

Bootflag on, starts Magic Lantern.
500D,firmware="boot=1"

(https://farm5.staticflickr.com/4576/38135901516_5fe73ff17c_n.jpg) (https://flic.kr/p/216WsTo)

QEMU actually patches the ROM1.BIN with the "boot=" command, I learned that when working on the ML on EOS-M2 (http://www.magiclantern.fm/forum/index.php?topic=15895.msg186043#msg186043) project.

I had some ROM dumps that were done without the bootflag set and when ML saves the ROM files the bootflag is set so I thought I'd look into where the bootflag was being set. I searched the forum and wiki for this and couldn't figure it out but once I looked at the ROM1.BIN files in a hex editor it didn't take long to find it.

Here is a ROM1.BIN that doesn't have the camera bootflag set. Note that only the ROM1.BIN needs to be changed.

(https://farm5.staticflickr.com/4565/38135901266_aa620d2a3b.jpg) (https://flic.kr/p/216WsP5)

And here it is with the camera bootflag set:

(https://farm5.staticflickr.com/4528/24339288118_538ef91bf3.jpg) (https://flic.kr/p/D5MdoA)

Pretty simple, huh? I could have figured it out by looking at the QEMU code but didn't think about that at the time. Looks like I'm not the only one reinventing the wheel.

eos.c
        /* fixme: reinventing the wheel */
        if (strstr(options, "boot=1") || strstr(options, "boot=0"))
        {
            /* change the boot flag */
            uint32_t flag = strstr(options, "boot=1") ? 0xFFFFFFFF : 0;
            fprintf(stderr, "Setting BOOTDISK flag to %X\n", flag);
            MEM_WRITE_ROM(s->model->bootflags_addr + 4, (uint8_t*) &flag, 4);
        }


The address to look for is "bootflags_addr + 4" so what is the bootflags_addr?

model_list.c
        /* defaults for DIGIC 4 cameras */
        .bootflags_addr         = 0xF8000000,
...
        /* defaults for DIGIC 5 cameras */
        .bootflags_addr         = 0xF8000000,
...
        /* defaults for DIGIC 6 cameras */
        .bootflags_addr         = 0xFC040000,


Just thought I'd share this for those of us who like to copy and paste the commands from debugmsg.gdb or patches.gdb without having to backspace to add the "boot=1" or "boot=0" depending on what we're checking out.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 06, 2017, 07:16:56 AM
This might be basic but I've been having problems navigating around the 5D3 GUI. The up/down arrow keys work fine when the Canon menu first comes up but as soon as I press the left/right arrow keys the GUI freezes up. This doesn't happen on other cameras I tried but it does happen all the firmware versions of the 5D3 that I tried.

Oh yeah, I'm using a MacBook Pro so using the "keypad" to simulate the joystick doesn't seem to be an option.

Is there some sort of a workaround for this?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 06, 2017, 08:49:51 AM
Reproduced with your 1.1.3 ROM.

Can you get a dm-spy log from photo mode, with LOG_INTERRUPTS enabled?

edit: it gets stuck into some routines related to LightMeasure, so it must be related to auto LCD brightness. Workarounds:
- set LCD brightness to Manual before dumping the ROM
- change property 0x204000D to 1

Confirmed the second workaround on your ROM.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 06, 2017, 06:32:37 PM
Great, I'll get to it after work today. A few questions, I think you have my 1.3.4 ROM or is it really a 1.1.3? I remember something about caching the previous version number when dumping a ROM from a new firmware update.

Quote from: a1ex on November 06, 2017, 08:49:51 AM
Can you get a dm-spy log from photo mode, with LOG_INTERRUPTS enabled?

This isn't a startup log so what action do you want me to log? Note that on 1.3.4 there is still that issue that also affected 1.1.3 when opening the ML menu in photo mode, not in LiveView. There is a flicker showing the Canon menus before the ML menu comes up. This only happens the first time the ML menus are accessed.

Quote from: a1ex on November 06, 2017, 08:49:51 AM
- change property 0x204000D to 1

Where do I change that property?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 06, 2017, 07:03:32 PM
Yes, a startup log. However, now that I've narrowed down the issue, I can get it myself.

I've tested with:


    int value = 1;
    prop_request_change(PROP_LCD_BRIGHTNESS_MODE, &value, 4);


placed in my_big_init_task, somewhere after call_init_funcs.

Alternatively, you may use prop_diag from the recovery branch, find where that property is stored in the ROM, and patch it.

For 1.2.3, you need this patch:

--- a/contrib/qemu/eos/eos.c
+++ b/contrib/qemu/eos/eos.c
@@ -4442,4 +4442,11 @@
     switch (address & 0xFFF)
     {
+        case 0x014:
+        {
+            /* 5D3 1.2.3: expects 0x10 for built-in LCD and 0x4 for HDMI? */
+            ret = 0x10;
+            break;
+        }
+
         case 0x01C:
         case 0x31C:


Not tested on 1.3.x.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 07, 2017, 06:26:07 AM
Yay!

(https://farm5.staticflickr.com/4475/38232995711_1d01b98e77.jpg) (https://flic.kr/p/21fw6Be)

Not working with 1.3.4 - yet.

I did find this small issue when rebuilding QEMU.

../../Makefile.setup:100: *** missing separator.  Stop.


I had to comment out the "undefine CONFIG_SITE" for it to work on a Mac.

Makefile.setup
# some recent Linux distros have this defined
# we don't use it, but the checks below will get upset and print a warning
# undefine CONFIG_SITE
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 11, 2017, 01:47:34 AM
More updates:

- GUI works for 6D, 70D and 5D3 1.2.3
- the test suite was getting too slow for my taste, so I've refactored it to allow parallel execution
  (about 1 order of magnitude faster on parallelized tests, about 2-3 times faster overall)

Here's a puzzle where I need some help (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst?fileviewer=file-view-default#rst-header-parallel-execution), if you are familiar with containers (the TODO at the end of the Parallel execution section).

edit: screenshots ready :)

(https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/lastSuccessfulBuild/artifact/qemu/tests/6D-menu.png)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 11, 2017, 01:53:24 AM
Trying to get to the next level in this game.

I managed to get 1.3.3 and 1.3.4 into the Canon menu without loading ML "boot=0"

(https://farm5.staticflickr.com/4580/38218498342_4ffe5f8102.jpg) (https://flic.kr/p/21eeN3A)

(https://farm5.staticflickr.com/4579/37543472954_297213c490.jpg) (https://flic.kr/p/ZcA7vQ)

Kind of tricky to get these screenshots because the patch to navigate the menus seems to work only when ML is loaded and I don't have GUI emulation working on these firmware versions yet. The way I did it was by going to the Canon menu I want to show in the camera and dump the firmware in that state. Yeah, 133 and 134 are working in camera but not in QEMU.

The 1.3.3 port that chris_miller did a while back almost works:

(https://farm5.staticflickr.com/4523/38199291866_f2b6d05bf6.jpg) (https://flic.kr/p/21cxmCs)

(https://farm5.staticflickr.com/4515/26546801649_52440216fd.jpg) (https://flic.kr/p/GrRifH)

but once I merged it into the latest patched QEMU branch it was no better than 134.

Here's how I've got the directory structure for the 5D3:

(https://farm5.staticflickr.com/4525/38222226332_3a8c6d430d.jpg) (https://flic.kr/p/21eyUfh)

This brings up a question about debugmsg.gdb. There is a section specific to 5D3.123:

# 1.2.3
if *(int*)0xFF136C94 == 0xE92D403E
  b *0xFF13B674
  register_func_log
end


I'm not sure if I'm stressing over the small stuff. According to "Blame" - Alex  committed 29bab2b, "GDB scripts: disabled slow items by default (semaphores, message queues, MPU communication, ResLock, EDMAC)" I was able to find the equivalent address for 1.3.3 and 1.3.4. However, shouldn't the debugmsg.gdb for each of these firmware versions be inside of the appropriate directory and run with, for example:

./run_canon_fw.sh 5D3,firmware="134;boot=1" -d debugmsg -s -S & arm-none-eabi-gdb -x 5D3/134/debugmsg.gdb

An issue I bumped up against is when using this command from the qemu directory to compile a version:

make -C ../magic-lantern/platform/5D3.134 install_qemu


I'm often getting messages that the sd.img resource is busy and it won't copy ML onto the image file. However, mounting the virtual sd card and installing it that way works fine.

Several of the recent QEMU updates have to do with the run_ml_all_cams.sh script so I gave that a try and was able to get log files for all of the 5D3 firmware versions. As expected some problems are showing up on the 1.3.3 and 1.3.4 versions.

5D3.134.log
c
./run_canon_fw.sh 5D3,firmware=134;boot=1 -display none -monitor stdio
pidof: illegal option -- s
ps: Invalid process id: Help:
ps: illegal option -- k
usage: ps [-AaCcEefhjlMmrSTvwXx] [-O fmt | -o fmt] [-G gid[,gid...]]
          [-g grp[,grp...]] [-u [uid,uid...]]
          [-p pid[,pid...]] [-t tty[,tty...]] [-U user[,user...]]
       ps [-L]
ps: Invalid process id: Help:
ps: illegal option -- k
usage: ps [-AaCcEefhjlMmrSTvwXx] [-O fmt | -o fmt] [-G gid[,gid...]]
          [-g grp[,grp...]] [-u [uid,uid...]]
          [-p pid[,pid...]] [-t tty[,tty...]] [-U user[,user...]]
       ps [-L]
&

DebugMsg=00005b90 (overriden)
QEMU 2.5.0 monitor - type 'help' for more information
(qemu) Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - DFFFFFFF: eos.iomem
[EOS] loading symbols from ../magic-lantern/platform/5D3.134//autoexec (800000-86CB40)
[EOS] loading symbols from ../magic-lantern/platform/5D3.134//magiclantern (69500-E7F14)
[EOS] loading './5D3/134/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './5D3/134/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[MPU] warning: non-empty spell #41 (PROP_VIDEO_MODE) has duplicate(s): #42
[MPU] warning: non-empty spell #74 (PROP_TFT_STATUS) has duplicate(s): #48 #49 #52 #56 #59 #63 #70 #79 #80 #85 #87 #92 #95 #100 #103 #108
[MPU] warning: non-empty spell #84 (Current Q position) has duplicate(s): #82 #89 #91
[MPU] warning: non-empty spell #93 (Current Q position) has duplicate(s): #97 #99
[MPU] warning: non-empty spell #98 (Current Q position) has duplicate(s): #51 #58 #65 #83 #90 #106
[MPU] warning: non-empty spell #101 (Current Q position) has duplicate(s): #105 #107
[MPU] warning: non-empty spell #113 (PROP_CARD1_STATUS) has duplicate(s): #8

[MPU] Available keys:
- Arrow keys   : Navigation
- Numpad keys  : Joystick (8 directions)
- Numpad 5     : Joystick center
- PgUp, PgDn   : Sub dial (rear scrollwheel)
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess (press only)
- M            : MENU (press only)
- P            : PLAY (press only)
- I            : INFO/DISP
- Q            : guess (press only)
- L            : LiveView (press only)
- W            : Pic.Style (press only)
- Shift        : Half-shutter
- B            : Open battery door
- C            : Open card door
- F10          : Power down switch
- F1           : show this help

Setting BOOTDISK flag to FFFFFFFF
FFFF0948: MCR p15,0,Rd,cr9,cr1,0: XSCALE_LOCK_ICACHE_LINE <- 0x40000006 (40000000 - 40000FFF, 0x1000)
FFFF0948: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x2078
FFFF0948: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0x12078   
FFFF2F8C: MCR p15,0,Rd,cr6,cr0,0:  946_PRBS0 <- 0x3F       (00000000 - FFFFFFFF, 0x100000000)
FFFF2F94: MCR p15,0,Rd,cr6,cr1,0:  946_PRBS1 <- 0x3D       (00000000 - 7FFFFFFF, 0x80000000)
FFFF2F9C: MCR p15,0,Rd,cr6,cr2,0:  946_PRBS2 <- 0xE0000039 (E0000000 - FFFFFFFF, 0x20000000)
FFFF2FA4: MCR p15,0,Rd,cr6,cr3,0:  946_PRBS3 <- 0xC0000039 (C0000000 - DFFFFFFF, 0x20000000)
FFFF2FAC: MCR p15,0,Rd,cr6,cr4,0:  946_PRBS4 <- 0xFF00002F (FF000000 - FFFFFFFF, 0x1000000)
FFFF2FB4: MCR p15,0,Rd,cr6,cr5,0:  946_PRBS5 <- 0x39       (00000000 - 1FFFFFFF, 0x20000000)
FFFF2FBC: MCR p15,0,Rd,cr6,cr6,0:  946_PRBS6 <- 0xF700002F (F7000000 - F7FFFFFF, 0x1000000)
FFFF2FC4: MCR p15,0,Rd,cr2,cr0,0: DCACHE_CFG <- 0x70       
FFFF2FCC: MCR p15,0,Rd,cr3,cr0,0:       DACR <- 0x70       
FFFF2FD0: MCR p15,0,Rd,cr2,cr0,1: ICACHE_CFG <- 0x70       
FFFF2FD4: MCR p15,0,Rd,cr5,cr0,0:    DATA_AP <- 0x3FFF     
FFFF2FDC: MCR p15,0,Rd,cr5,cr0,1:    INSN_AP <- 0x3FFF     
FFFF2FE0: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x12078
FFFF3000: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC001307D
FFFF0974: MCR p15,0,Rd,cr9,cr1,1: XSCALE_UNLOCK_ICACHE <- 0x6        (00000000 - 00000FFF, 0x1000)
FFFF0974: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC001307D
FFFF0974: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005307D
FFFF09A4: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005307D
FFFF09A4: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
SD LOAD OK.

Open file for read : AUTOEXEC.BIN

SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
File size : 0x6CB40

Now jump to AUTOEXEC.BIN!!

0010DCCC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010DCCC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
K285 READY

qququiquit
[MPU] WARNING: forced shutdown.

For clean shutdown, please use 'Machine -> Power Down'
(or 'system_powerdown' in QEMU monitor.)


pidof and ps on the Mac don't seem to have those options though I don't know if that is affecting the validity of the log.

In any case, even though the 1.3.3 and 1.3.4 ML ports are running in camera it looks like QEMU is showing some issues that need to be resolved. Now where to start?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 11, 2017, 10:02:55 AM
Didn't this work?

Quote from: a1ex on November 06, 2017, 08:49:51 AM
- set LCD brightness to Manual before dumping the ROM

Didn't look into light sensor emulation yet.

The only errors I've got about sd.img were if the card image was full (I was running the silent picture module and the card image got filled with dng's pretty fast), or when copying ML with qemu already running (this results in filesystem corruption; just restore from sd.img.xz).

Pushed some Mac fixes. The pidof/ps issue was cosmetic (just re-printing the commands after clearing the screen).

To make the log a bit more readable, you could either "cat" it to a terminal, then copy the result, or run it through ansi2txt to remove the color codes. Maybe also add "-d debugmsg" to the emulation.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 12, 2017, 06:45:17 PM
Quote from: a1ex on November 11, 2017, 10:02:55 AM
Didn't this work?

Quote from: a1ex on November 06, 2017, 08:49:51 AM
- set LCD brightness to Manual before dumping the ROM

Nope - I just tried it and it did't get any further. Just to make sure we're talking apples to apples:

Quote from: a1ex on November 06, 2017, 08:49:51 AM
Reproduced with your 1.1.3 ROM.

I'm testing 1.3.4 which I passed to you. I also made dumps for 1.1.3, 1.2.3 and 1.3.3 on the same camera using the same settings but I don't believe I passed those to you. We had an interesting glitch with the very first 1.3.4 ROM dump which is the one you are probably using. Right after updating the firmware ML was still showing the firmware version it was updated from (1.1.3).

The logs are in color just like the QEMU output? No wonder I couldn't make sense of those logs. I can now see that the problem is very early in the process:

QuoteSD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
SD LOAD OK.
Open file for read : AUTOEXEC.BIN
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
File size : 0x6C240
Now jump to AUTOEXEC.BIN!!
0010DCCC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010DCCC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
K285 READY
quit
[MPU] WARNING: forced shutdown.

Quote from: a1ex on November 11, 2017, 10:02:55 AM
The only errors I've got about sd.img were if the card image was full (I was running the silent picture module and the card image got filled with dng's pretty fast), or when copying ML with qemu already running (this results in filesystem corruption; just restore from sd.img.xz).

I think that the issue I'm seeing is a Mac problem. When installing from a different branch (not qemu) I mount/unmount the sd.img using the finder. This works fine except when I go back to the "make -C ../magic-lantern/platform/5D3.134 install_qemu" method. Seems that OSX doesn't release the resources when umounting via the finder.

[EDIT] Deleted most of my previous edit - turns out I was using the wrong firmware version.

Note that on several platforms I need to press the "M" key to invoke the Canon menu so the screenshots I'm getting with "run_ml_all_cams.sh" don't show anything. Again, maybe just a Mac problem?

Speaking of Mac problems, I recently discovered the excellent QEMU documentation (https://bitbucket.org/hudson/magic-lantern/src/4895777de907c24ffd6332bcee23a7608450f6bd/contrib/qemu/README.rst?at=qemu&fileviewer=file-view-default). Why is this a Mac problem? Because none of the Mac apps I've got opens the README.rst file properly. The best way I found to view it on a Mac is on Bitbucket.

Another issue not necessarily Mac specific but probably with bash version 4.4 and newer when running the install.sh script:

   Note: Canon GUI emulation (menu navigation, no LiveView) only works on:
   ./install.sh: line 418: warning: command substitution: ignored null byte in input
5D2 5D3 6D 50D 60D 70D 450D 500D 550D 600D 650D 700D 100D 1000D 1100D 1200D EOSM EOSM2


I tried several options but haven't found anything that removes that warning.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 12, 2017, 08:46:39 PM
QuoteNope - I just tried it and it did't get any further.

Really? I've patched property 0x204000D = PROP_LCD_BRIGHTNESS_MODE from 0 (auto) to 1 (manual) in your 1.3.4 ROM (offset 0xf6259c in your ROM1, likely different on other 5D3's) and Canon menus started to work.

However, rather than finding the offset (e.g. with prop_diag or by manually looking up that property), it would have been a LOT easier (but maybe more time-consuming) to change this in Canon menu, reboot, then dump the ROM again with this setting already configured to "manual". Hence my suggestion.

Quoteon several platforms I need to press the "M" key to invoke the Canon menu so the screenshots I'm getting with "run_ml_all_cams.sh" don't show anything.

They probably start with the main display turned off; try pressing M from the script, e.g.:

env QEMU_SCRIPT="sleep 10; echo sendkey m; sleep 1" \
    SCREENSHOT=1 \
    ML_PLATFORMS="5D3.113/ 5D3.123/ 5D3.134/" \
    ./run_ml_all_cams.sh


Quote from: dfort on November 12, 2017, 06:45:17 PM
Speaking of Mac problems, I recently discovered the excellent QEMU documentation (https://bitbucket.org/hudson/magic-lantern/src/4895777de907c24ffd6332bcee23a7608450f6bd/contrib/qemu/README.rst?at=qemu&fileviewer=file-view-default). Why is this a Mac problem? Because none of the Mac apps I've got opens the README.rst file properly. The best way I found to view it on a Mac is on Bitbucket.

Same here - it's meant to be viewed online, but you can convert it to other formats if you wish. For example, pdf:


rst2latex README.rst > README.tex
pdflatex README.tex


Formatting is not the best (the layout could use some tweaking), but it's a good starting point. The (now outdated) ML user guide used to be in this (source) format, and was rendered as wikia code (now broken since the new wiki is dokuwiki), pdf (for desktop viewing) and in-camera BMPs (a bit heavyweight, but back then we did not have proportional fonts).

Conversion to HTML works as well, but it also needs some CSS (by default, it doesn't look very well). Didn't dig deeper to find one - maybe it's good to render it during installation.

The README was (http://www.magiclantern.fm/forum/index.php?topic=2864.msg189925#msg189925) linked (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190254;topicseen#msg190254) a (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190596;topicseen#msg190596) few (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190831#msg190831) times (http://www.magiclantern.fm/forum/index.php?topic=16012.msg190744#msg190744), including first post (http://www.magiclantern.fm/forum/index.php?topic=2864.225) (also asked for some proof-reading).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 12, 2017, 09:30:06 PM
Quote from: a1ex on November 12, 2017, 08:46:39 PM
Really? I've patched property 0x204000D = PROP_LCD_BRIGHTNESS_MODE from 0 (auto) to 1 (manual) in your 1.3.4 ROM (offset 0xf6259c in your ROM1, likely different on other 5D3's) and Canon menus started to work.

Yes, the Canon menus work. I was referring to this test:

env ML_PLATFORMS="5D3.134/" \
TIMEOUT=10 \
SCREENSHOT=1 \
./run_ml_all_cams.sh


This is how it ends:

SD LOAD OK.
Open file for read : AUTOEXEC.BIN
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
File size : 0x6C240
Now jump to AUTOEXEC.BIN!!
0010DCCC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010DCCC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
K285 READY
screendump 5D3.134.ppm
(qemu) quit
[MPU] WARNING: forced shutdown.


This is how the "screendump" looks:

(https://farm5.staticflickr.com/4541/24498021198_0dc58648de_n.jpg) (https://flic.kr/p/DjNLcQ)

Running the run_ml_all_cams.sh script with a BOOT=0 option will produce a more complete log but the screendump looks the same. When I run this command:

./run_canon_fw.sh 5D3,firmware='134;boot=0'

It also comes up with a grey screen but pressing the "m" key will invoke the Canon menu. [EDIT] Though pressing the left/right arrow keys will freeze the QEMU GUI.

Quote from: a1ex on November 12, 2017, 08:46:39 PM
The README was (http://www.magiclantern.fm/forum/index.php?topic=2864.msg189925#msg189925) linked (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190254;topicseen#msg190254) a (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190596;topicseen#msg190596) few (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190831#msg190831) times (http://www.magiclantern.fm/forum/index.php?topic=16012.msg190744#msg190744), including first post (http://www.magiclantern.fm/forum/index.php?topic=2864.225) (also asked for some proof-reading).

Oops. My only excuse is that I didn't read the whole thing, only the parts that I was having problems with. Promise I'll read the rest of it and post any proofreading notes.

By the way, I usually run "make clean" from the magic-lantern directory to clean up everything but it doesn't work with the qemu branch. I've got to run "make clean" in each individual directory.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 12, 2017, 09:35:45 PM
Same gray screen here - it appears to start with display turned off. Maybe it was configured that way before dumping the ROM.

BTW - managed to get a fairly decent local rendering of README.rst with rst2html5 from python3-docutils (unfortunately not in python2-docutils which our script already installs).

edit: there are two three different variants of rst2html5 - one from python3-docutils which gives a nice layout, but the overall look is a bit dull, and this rst2html5 (https://marianoguerra.github.io/rst2html5/), which has a bunch of advanced options and styles, but gives bad layout with default settings, and there's also this one (https://pypi.python.org/pypi/rst2html5), which doesn't even like my source...

Installing any of these "third party" rst2html5 with pip3 breaks the rst2html5 from python3-docutils (even when running as rst2html5-docutils).

TLDR: auto-rendering the RST as HTML on user's PC from the install script may be a can of worms.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 12, 2017, 10:30:20 PM
Quote from: a1ex on November 12, 2017, 09:35:45 PM
Same gray screen here - it appears to start with display turned off. Maybe it was configured that way before dumping the ROM.

Interesting. Wonder how the camera should be set before dumping the ROM. Of all the dumps I tested only the 700D starts with a non-grey screen.

(https://farm5.staticflickr.com/4551/37655751364_5f6474901a_n.jpg) (https://flic.kr/p/ZnvyYu)

Quote from: a1ex on November 12, 2017, 09:35:45 PM
TLDR: auto-rendering the RST as HTML on user's PC from the install script may be a can of worms.

I don't mind reading online. Maybe add a README.html in the qemu directory that just links to the online documentation?

README.html
<meta http-equiv="refresh" content="0; url=https://bitbucket.org/hudson/magic-lantern/src/4895777de907c24ffd6332bcee23a7608450f6bd/contrib/qemu/README.rst?at=qemu&fileviewer=file-view-default" />
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 12, 2017, 10:43:10 PM
700D starts in movie mode (that's how you started it when you've got the MPU log).

Here's how various models start with my ROMs (look at *-menu.png, first image from the set):

https://builds.magiclantern.fm/jenkins/job/QEMU-tests/lastSuccessfulBuild/artifact/qemu/tests/

For 6D, Audionut uploaded 3 logs (https://bitbucket.org/hudson/magic-lantern/issues/1974/#comment-40853231): movie, photo LV, photo without LV. I've used the last one for QEMU, but you can try the others as well. From the mpu_spells directory:


python extract_init_spells.py /path/to/6D-startup_movie_mode.LOG > 6D.h


and it will start in the same way as 700D.

Don't know the property for starting with the main info screen turned on - I believe you should press INFO until that screen appears, and make sure the next boot actually starts with that screen. Then dump the ROM. I didn't do anything special.

On some models, the firmware thinks the eye sensor near the LCD is active, so it turns off the display. For example, on 450D, I have to turn off this option from Canon menu, then the info screen appears.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 13, 2017, 12:26:30 AM
Tricky stuff.

python extract_init_spells.py /Users/rosiefort/Desktop/6D_startup_movie_mode.LOG
Traceback (most recent call last):
  File "extract_init_spells.py", line 169, in <module>
    model = log_filename[:log_filename.index("-")]
ValueError: substring not found


Details details, the log file needs to be renamed to "6D-startup_movie_mode.LOG" and the output saved to "6D.h" like this:

python extract_init_spells.py /Users/rosiefort/Desktop/6D-startup_movie_mode.LOG > 6D.h

It doesn't always catch the menu in the screendump. Maybe that has to do with the setting of the TIMER option?

env ML_PLATFORMS="6D.116/" \
BOOT=1 \
TIMEOUT=10 \
SCREENSHOT=1 \
./run_ml_all_cams.sh


But yeah, eventually I did get that same screendump as the 700D.

So getting ML working in QEMU on the 5D3.133/134 is a combination of dumping the ROM with the camera at some certain setting that will bring up the Canon menu instead of a blank screen and getting a startup log file that captures the MPU messages?

[EDIT] Oh yeah, and just for good measure turn off the automatic LCD brightness.

(https://farm5.staticflickr.com/4559/26597364079_52404457c1_n.jpg) (https://flic.kr/p/GwjrGK)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 13, 2017, 10:09:29 AM
Quote from: dfort on November 13, 2017, 12:26:30 AM
So getting ML working in QEMU on the 5D3.133/134 is a combination of dumping the ROM with the camera at some certain setting that will bring up the Canon menu instead of a blank screen and getting a startup log file that captures the MPU messages?

I've downloaded your 1.3.4 build (Nightly.2017Nov02.5D3134.zip), placed it on the virtual card, started QEMU, pressed DELETE on the blank screen and entered ML menu. That's expected - you can open ML menu if your main display is off. Also pressed INFO at startup a few times and got Canon's screen. ML menu works from there as well.

To have the emulation start with Canon's info screen, it's a matter of dumping the ROM with this setting enabled (I hope there are no other tricks). The MPU messages were already captured in this configuration, so you shouldn't have to change them. However, that's just a minor cosmetic issue.

ML emulation already works (with your 1.3.4 ROM, patched for manual LCD brightness as described above), so I'm not sure what your question is.




Noticed something weird: with ML loaded, if the first thing done after startup is pressing M twice, there is an error coming from a ML task, when calling some GUI function from Canon (maybe a bad stub?):

ASSERT : ./Dialog/Dialog.c, Task = debug_task, Line 1049


After this event, ML menu stops working. Repeatable.

The error doesn't happen without ML loaded (boot=0). With ML loaded (boot=1), it doesn't save a crash log (it should).

Pressing M twice after navigating ML menu works fine.

Does it match the behavior on real hardware?

edit: got a call stack (b *0x1900):

0x76250(0, 76250, 19980218, 19980218)                                            at [debug_task:de48:1ba5b8] (pc:sp)
0x75C08(bf29d "ML/FONTS/", 0, 0, 69b84)                                         at [debug_task:76464:1ba530] (pc:sp)
  0x75A88(1ba3f8 "ML/FONTS/ARGHLF22.RBF", 0, 42, 1ba3ec)                         at [debug_task:75cac:1ba3f0] (pc:sp)
   0x756B0(4, 1ba374 "Reading ML/FONTS/ARGHLF22.RBF...", 42, 1ba364)             at [debug_task:75be4:1ba368] (pc:sp)
    0xBB4B8 -> 0xFF359384(0, 4, 1ba374 "Reading ML/FONTS/ARGHLF22.RBF...", 8181b4)
                                                                                 at [debug_task:756dc:1ba358] (pc:sp)
     0xFF4560CC(0, d, 1ba374 "Reading ML/FONTS/ARGHLF22.RBF...", 21)             at [debug_task:ff3593b8:1ba340] (pc:sp)
      0xFF455F18(0, 4, 1ba374 "Reading ML/FONTS/ARGHLF22.RBF...", 21)            at [debug_task:ff4560e0:1ba330] (pc:sp)
       0x1900(ff45433c "pDialog != NULL", ff454308 "./Dialog/Dialog.c", 419, 21) at [debug_task:ff455f34:1ba320] (pc:sp)


0xFF359384 is dialog_set_property_str; maybe ML thinks you are on the Format dialog? That's the only place where ML tries to change Canon's menu strings.

Please double-check DIALOG_MnCardFormatBegin in consts.h.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 13, 2017, 03:29:44 PM
Quote from: a1ex on November 13, 2017, 10:09:29 AM
ML emulation already works (with your 1.3.4 ROM, patched for manual LCD brightness as described above), so I'm not sure what your question is.

Tracked down the problem. I merged 1.3.3 and 1.3.4 and qemu and it looks like it was a bad merge. I didn't think of running the build from my pull request. What I'm trying to figure out is a menu glitch issue that I posted in the Canon 5D Mark III / 5D3 / Firmware 1.3.4 (http://www.magiclantern.fm/forum/index.php?topic=18966.msg193146#msg193146) topic.

Good to finally see ML on 5D3.134 working in QEMU.

(https://farm5.staticflickr.com/4531/38334592226_069c82e7c7_n.jpg) (https://flic.kr/p/21puNHh)

Quote from: a1ex on November 13, 2017, 10:09:29 AM
Noticed something weird: with ML loaded, if the first thing done after startup is pressing M twice, there is an error coming from a ML task, when calling some GUI
...
Does it match the behavior on real hardware?

No problem when pressing the Menu button twice after startup with the Nov02 build on the camera. The issue I've got on camera is going into the ML menu (Trash button) after startup outside of LiveView. Maybe it is related?

Quote from: a1ex on November 13, 2017, 10:09:29 AM
Please double-check DIALOG_MnCardFormatBegin in consts.h.

Thanks for pointing that out. @chris_overseas got it right in his 1.3.3 port but I missed it. No change from 1.3.3 to 1.3.4 so that problem should be fixed. Updated the pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/816/update-to-5d3134/diff) and uploaded a new build.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on November 13, 2017, 07:50:59 PM
Quote from: dfort on November 13, 2017, 03:29:44 PM
What I'm trying to figure out is a menu glitch issue that I posted in the Canon 5D Mark III / 5D3 / Firmware 1.3.4 (http://www.magiclantern.fm/forum/index.php?topic=18966.msg193146#msg193146) topic.

The issue can be reproduced in QEMU here. There is some abnormal SD card activity from debug_task the first time you open ML menu (run with -d debugmsg) and the restore after format feature is still not working with today's build. Therefore, my advice would be to double-check the same stubs.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on November 14, 2017, 07:49:58 AM
Quote from: a1ex on November 13, 2017, 07:50:59 PM
...my advice would be to double-check the same stubs.

Thanks, that's exactly what the problem was. Tested on camera (http://www.magiclantern.fm/forum/index.php?topic=18966.msg193182#msg193182).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on December 06, 2017, 10:50:44 PM
Quote from: t3r4n on November 05, 2017, 02:48:34 PM
So there is a missing section in the  README telling you to also compile the sf_dump Module and putting it on the ML Card and activate it and then Reboot the camera and use the module from the Debug Menu of ML. (only found it through full text search on the whole ML directory)

Solved (https://bitbucket.org/hudson/magic-lantern/commits/6fd16a7ce0d940291408f57c522981a24254dc11). The serial flash dumper should also be included at startup, as part of the usual ROM backup (maybe also in the installer).

Quote
3. In the README under DEBUGGING you also write to use "make CONFIG_qemu=y" and the "make install_qemu" which wouldn't compile in the unified branch for the 700D. I found out that the qemu (or no dm_spy_experiments) branch is needed to use these options, maybe stress that out a bit more in the section.

Solved (https://bitbucket.org/hudson/magic-lantern/commits/bc84c8579aa8cc1011fa1a40687a1b7b85c40355). Soon we'll have QEMU in mainline as well.

Still, I often test old changesets in QEMU (usually for troubleshooting, maybe "hg bisect"), so it's helpful to know how to backport this rule whenever you need it.

Also been fixing a couple of minor things.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on December 20, 2017, 12:49:51 AM
Some progress on emulating DIGIC 6:

- no more startup patches (https://bitbucket.org/hudson/magic-lantern/commits/2b0b39ae12ed1ca452e2293fb8af8f61be43faa7) needed!
- Dry-shell (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/HACKING.rst#rst-header-dryos-internals) works
- serial flash emulation (https://bitbucket.org/hudson/magic-lantern/commits/652133663c39d58dd58360b1a3154ffc3f66b871?at=qemu) (needs SFDATA.BIN from a D5 camera)
- MPU spells guessed from 60D (only this (https://bitbucket.org/hudson/magic-lantern/commits/391ba3364ae74964edd16178761e5f690309c9d8) is required to boot the GUI on most D3-D5 models)
- Omar firmware (http://www.magiclantern.fm/forum/index.php?topic=13408.msg194424#msg194424) revealed (doesn't quite work yet)
- 80D starts a LOT of tasks, including some filesystem drivers and starts to initialize the image capture backend; other D6 models are catching up.
- edit: 80D file I/O works too (http://www.magiclantern.fm/forum/index.php?topic=17360.msg194996#msg194996) (creates DCIM dir, saves debug logs on the SD image)

You will need a patched SFDATA.BIN (https://bitbucket.org/hudson/magic-lantern/commits/652133663c39d58dd58360b1a3154ffc3f66b871?at=qemu) (serial flash dump) from a 70D (preferred), or 700D, 650D, EOSM, 6D (not sure about 100D). If you don't have one, just comment it out in model_list.c; most of the stuff appears to work without it.

Fun stuff:

( sleep 3; echo "akashimorino";
  sleep 1; echo "SHM_SHOW_INFO";
  sleep 1; echo "SHM_SHOW_DIST_INFO";
) | ./run_canon_fw.sh 80D -serial stdio


To get an idea how far the emulation goes:

# with 46f2e6e1cbb0 (right before the above stuff):
(./run_canon_fw.sh 80D,firmware="102;boot=0" -d debugmsg -s -S & arm-none-eabi-gdb -x 80D/debugmsg.gdb) |& grep Notify.*Cur --text
[        init:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2018000, Flag = 0x10000)
[      SFRead:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2008000, Flag = 0x8000)

# with 7ea57e73c091 (the above stuff):
(./run_canon_fw.sh 80D,firmware="102;boot=0" -d debugmsg -s -S & arm-none-eabi-gdb -x 80D/debugmsg.gdb) |& grep --text Notify.*Cur
[        init:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2018000, Flag = 0x10000)
[      SFRead:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2008000, Flag = 0x8000)
[     RomRead:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2000000, Flag = 0x2000000)
[     Startup:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 1, 0x2, Flag = 0x2)
[     Startup:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x20420010, Flag = 0x20000000)
[      RscMgr:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x420010, Flag = 0x20000)
[     FileMgr:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x400010, Flag = 0x10)
[     FileMgr:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x400000, Flag = 0x400000)
[ShootCapture:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0xc0000, Flag = 0x40000)
[     Startup:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0x80000, Flag = 0x80000)

# 60D, which boots the GUI without any fuss
./run_canon_fw.sh 60D,firmware="boot=0" -d debugmsg |& grep --text Notify.*Cur
[        init:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x10000, Flag = 0x10000)
[    PowerMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 1, 0x20000002, Flag = 0x2)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 1, 0x20000000, Flag = 0x20000000)
[     FileMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x10, Flag = 0x10)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0xe0110, Flag = 0x40000)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0xa0110, Flag = 0x80000)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0x20110, Flag = 0x100)
[      RscMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0x20010, Flag = 0x20000)
[     FileMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 3, 0x10, Flag = 0x10)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 4, 0x110, Flag = 0x100)
[     FileMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 4, 0x10, Flag = 0x10)
[     Startup:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 5, 0x80200200, Flag = 0x80000000)
[ GuiMainTask:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 5, 0x200200, Flag = 0x200000)
[       DpMgr:ff02b9f8 ] (00:03) [SEQ] NotifyComplete (Cur = 5, 0x200, Flag = 0x200)


Queued: 1300D, 40D, 7D, property logging, docs on DryOS internals...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Theta Sigma on December 20, 2017, 02:54:04 PM
Quote from: a1ex on December 20, 2017, 12:49:51 AM
- 80D starts a LOT of tasks, including some filesystem drivers and starts to initialize the image capture backend; other D6 models are catching up.

Is shutter actuation data (shutter count) one of them?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on December 20, 2017, 03:52:38 PM
Quote from: a1ex on December 20, 2017, 12:49:51 AM
Some progress on emulating DIGIC 6:

I think it's possible to emulate simple drawing of text strings (https://chdk.setepontos.com/index.php?topic=12788.msg135622#msg135622) in case main CPU will send certain messages to MZRM core...
But on EOS M3  the camera controller still does not allow to start it normally and goes to shutdown.
What is the situation with DSLRs?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on December 20, 2017, 09:50:43 PM
Definitely - as long as these functions are called during the emulation. On DSLRs we don't know yet how the display buffer looks like - the above stuff was done without a camera, just by playing around with the ROM dump. The emulation doesn't seem to initialize Zico on 80D yet - does reach this stage on M3?

This snippet resembles TFT SIO registers (http://www.magiclantern.fm/forum/index.php?topic=21108.0):

(./run_canon_fw.sh EOSM3 -s -S -d io & arm-none-eabi-gdb -x EOSM3/debugmsg.gdb) |& grep --text -i -C 100 Backlight | grep --text -i -E "Backlight|TX|DIGIC6"

[     Startup:fc3587db ] (60) DispSwCon_TurnOnBackLight
[     Startup:fc14b9db ] (60) TurnOnBackLight
[DIGIC6] at Startup:FC32E2FE:FC1BB779 [0xD20B0D7C] <- 0xC       : ???
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0xB0      : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x104     : TX register
[DIGIC6] at Startup:FC32E25C:FC1BB78B [0xD20B0D7C] <- 0xC       : ???
[DIGIC6] at Startup:FC32E25C:FC1BB79F [0xD20B0D7C] <- 0xC       : ???
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0xC8      : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x100     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x10E     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x113     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x11A     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x119     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x117     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x117     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x126     : TX register
[SIO8]   at Startup:00000120:FC13AB6B [0xC0820818] <- 0x125     : TX register


BTW, do you happen to have any notes on UTimer or Omar?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on December 21, 2017, 08:19:47 AM
Quote from: a1ex on December 20, 2017, 09:50:43 PM
The emulation doesn't seem to initialize Zico on 80D yet - does reach this stage on M3?

Set default DRAM parameter
#
DRYOS version 2.3, release #0055+p6
Copyright (C) 1997-2013 by CANON Inc.
[SDIO] Error
[SDIO] Error
[SDIO] Error
[SDIO] Error

StartDiskboot
Diskboot file not found
2.1.1
3.1.1
4.1.1
3.1.2:11,0,0,4,0
3.1.3:1
3.1.3:0
3.1.2:11,0,0,4,0
3.1.3:1
3.2.1:2
3.2.3:2
3.2.7:2
3.2.9:2
3.2.11:2
3.2.13:2
3.2.15:2
3.2.17:2
3.2.19:2
3.4.4.1:0,128,128,1
== PnlSync =========
  vwidth  : 494
  hwidth  : 909
  h_pre   : 723
  h_blank : 719
  vb_lt   : 490
  vb_st   : 11
  vp_lt   : 490
  vp_st   : 11
  vb_l    : 490
  vb_s    : 11
  vp_l    : 490
  vp_s    : 11
====================
3.4.1.1:720,480,1,10,1
3.3.2:fc5f95cc(32,32,32),1
3.3.3:fc5f95d0(0),1
3.3.4:fc5f95ec,(0,1),1
3.3.5:fc5f962c,1
3.3.7:fc5f965c,1
3.3.9:fc5f9664,1
3.3.10:fc5f9668,1
3.3.1:fc5f95cc,1
3.1.6:1,2,9,0
3.10.1:124,128,134,0,0,0,1
3.3.12:1,1
3.2.19:0
3.2.17:0
3.2.15:0
3.2.13:0
3.2.11:0
3.2.9:0
3.2.7:0
3.3.14:0,0,1
InitializeGraphicLog Addr:0x4112b000 Size:0x5000
DlphLog:Addr:0x4112b000, Size:0x1400
_FreeMsg   : ------ req:0 stt:32
_CreateMsg : 0xbff00500 size:12
_FreeMsg   : ------ req:0 stt:32
SendMsg   : 1
ZicoLog:Addr:0, Size:0x2800
InitializeGraphicLog SUCCESS
_FreeMsg   : ------ req:1 stt:33
_FreeMsg   : 0xbff00500 free:1
_CreateMsg : 0xbff00500 size:4
_FreeMsg   : ------ req:1 stt:33
SendMsg   : 2
_FreeMsg   : ------ req:2 stt:33
_CreateMsg : 0xbff00528 size:76
_FreeMsg   : ------ req:2 stt:33
SendMsg   : 3
_FreeMsg   : ------ req:3 stt:33
_CreateMsg : 0xbff00598 size:4
_FreeMsg   : ------ req:3 stt:33
SendMsg   : 4
_FreeMsg   : ------ req:4 stt:33
_CreateMsg : 0xbff005c0 size:0
_FreeMsg   : ------ req:4 stt:33
SendMsg   : 5
[GRYP]T: --- Initialize start ----------------
_FreeMsg   : ------ req:5 stt:37
_FreeMsg   : 0xbff00500 free:2
_FreeMsg   : 0xbff00528 free:3
_FreeMsg   : 0xbff00598 free:4
_FreeMsg   : 0xbff005c0 free:5
_CreateMsg : 0xbff00500 size:0
_FreeMsg   : ------ req:5 stt:37
SendMsg   : 6
[GRYP]T: InitializeGryp(Pri)    : Completed.
         Privilege Event handle : 0x02500050
_FreeMsg   : ------ req:6 stt:38
_FreeMsg   : 0xbff00500 free:6
_CreateMsg : 0xbff00500 size:0
_FreeMsg   : ------ req:6 stt:38
SendMsg   : 7
         GRYPHON revision       : 0x00000000
[GRYP]T: Initialize(Pri): Completed.
[GRYP]T: --- Initialize(Pri/Nml) Completed ---

_FreeMsg   : ------ req:7 stt:39
_FreeMsg   : 0xbff00500 free:7
_CreateMsg : 0xbff00500 size:12
_FreeMsg   : ------ req:7 stt:39
SendMsg   : 8
_FreeMsg   : ------ req:8 stt:40
_FreeMsg   : 0xbff00500 free:8
_CreateMsg : 0xbff00500 size:12
_FreeMsg   : ------ req:8 stt:40
SendMsg   : 9
_FreeMsg   : ------ req:9 stt:40
_CreateMsg : 0xbff00530 size:16

...



SendMsg   : 27
_FreeMsg   : ------ req:27 stt:59
_FreeMsg   : 0xbff00500 free:27
_CreateMsg : 0xbff00500 size:772
_FreeMsg   : ------ req:27 stt:59
SendMsg   : 28
3.4.1.4:5,00690f70,1,0,1e0
3.4.1.5:5,0,0,720,480,0,0,1
3.4.1.6:5,0,3,0,1
3.4.1.2:5,1,1
3.2.21:1,fc152451,00000000
3.2.22:1,1
3.2.3:0
3.3.15:1,1


At least it tries to draw something by sending JediDraw message (0xFC4BB8BA)
I've used this patch to display debug messages:
PatchDbgByte(0x00028698,0xFF);
PatchDbgByte(0x00028699,0xFF);
PatchDbgByte(0x0002869A,0xFF);
PatchDbgByte(0x0002869B,0xFF);

PatchDbgByte(0x000286A0,0x9D);
PatchDbgByte(0x000286A1,0xFD);
PatchDbgByte(0x000286A2,0x37);
PatchDbgByte(0x000286A3,0xFC);

PatchDbgByte(0x000286A8,0x9D);
PatchDbgByte(0x000286A9,0xFD);
PatchDbgByte(0x000286AA,0x37);
PatchDbgByte(0x000286AB,0xFC);


QuoteBTW, do you happen to have any notes on UTimer or Omar?
no
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on December 23, 2017, 08:22:48 AM
Noticed a message that I don't remember seeing on previous versions:

49:53: execution error: The variable qemu is not defined. (-2753)

Doesn't seem to hurt anything, just wondering if others are seeing it.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on December 23, 2017, 08:25:16 AM
Don't remember seeing it; when/where does it appear, and in what color?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on December 24, 2017, 10:21:49 AM
Okay, sorted out, it was a typo when trying to bring QEMU window to foreground. This is still tricky, for example it doesn't work when starting multiple instances, but I'm probably doing this wrong - how do you open a window in foreground from command line on Mac?!

Also found a better way to tell whether the SD/CF images are mounted (or otherwise in use). On Mac, checking them with lsof is enough - this handles both images mounted with "hdiutil attach" and being in use by QEMU itself. On Linux (or at least on my system), lsof only handles the latter case, so checking with losetup (or grepping the output of "mount") is still needed to make sure the images are not in use by some other process when running the emulation.

BTW - already old news, but you can already navigate Canon menus on 1300D (http://www.magiclantern.fm/forum/index.php?topic=17969.msg195036#msg195036) and 40D (http://www.magiclantern.fm/forum/index.php?topic=1452.msg195051#msg195051) in QEMU.

Current state: 20 EOS models able to run the GUI in the emulator! These are:
5D2 5D3 6D 40D 50D 60D 70D 450D 500D 550D 600D 650D 700D 100D 1000D 1100D 1200D 1300D EOSM EOSM2.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on December 25, 2017, 12:09:33 PM
@a1ex asked about this one:
Quote from: dfort on December 23, 2017, 08:22:48 AM
Noticed a message that I don't remember seeing on previous versions:

49:53: execution error: The variable qemu is not defined. (-2753)

Doesn't seem to hurt anything, just wondering if others are seeing it.

Quote from: a1ex on December 23, 2017, 08:25:16 AM
Don't remember seeing it; when/where does it appear, and in what color?

it happens on the 750D around this time in the boot, I tried to recreate the colours as they were (normal colour is qemu and red is camera?):
(sorry the code tag and colours don't mix)

[SF] InstallSerialFlash 2 0xd20b0d8c 0x0 0x800000 1

[ROM-DMA0] Copy [0xFD13A000] -> [0x407FFFA0], length [0x00001363], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA0] Copy [0xFDCC0000] -> [0x40D6C000], length [0x0015B6AF], flags [0x11100003]
[XDMAC0] OK
     0:     3.328 [STARTUP]
K393 ICU Firmware Version 1.0.0 ( 8.7.2 )
     5:     4.864 [PROPAD] ERROR Not Exist Valid ComboPackages!! 0x20000
49:53: execution error: Die Variable ,,qemu" ist nicht definiert. (-2753)
[ROM-DMA0] Copy [0xFD200000] -> [0x408020C0], length [0x000205E3], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA0] Copy [0xFD320000] -> [0x40842160], length [0x0001AEDB], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA0] Copy [0xFD360000] -> [0x40862200], length [0x0001DCA7], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA1] Copy [0xFE744B88] -> [0xDFF00000], length [0x00002E77], flags [0x11100003]
[XDMAC1] OK
[MPU] Received: 06 04 02 00 00 00  (Init - spell #1)
[MPU] Sending : 2c 2a 02 00 03 03 03 04 03 00 00 48 00 00 00 14 50 00 00 00 00 81 06 00 00 04 06 00 00 04 06 00 00 04 01 01 00 00 00 00 4d 4b 01 00  (Init)
     9:    64.512 [PROPAD] ERROR SearchPropertyPackage DataType (0) = 0x01000000(L:3294)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on December 25, 2017, 10:20:12 PM
Quote from: a1ex on December 23, 2017, 08:25:16 AM
Don't remember seeing it; when/where does it appear, and in what color?

It was showing up here, in black:

./run_canon_fw.sh EOSM,firmware=boot=0 -s -S &
arm-none-eabi-gdb -x EOSM/patches.gdb
...
Setting BOOTDISK flag to 0
49:53: execution error: The variable qemu is not defined. (-2753)
0xffff0000 in ?? ()


However, this disappeared on the latest commits so -- nevermind!

By the way, the EOSM doesn't launch into the GUI so I'm not able to run the sf_dump you asked for (http://www.magiclantern.fm/forum/index.php?topic=17360.msg195135#msg195135)--unless there's another way of doing this without using the GUI.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on December 27, 2017, 01:11:24 PM
Hey,
I've done two minor changes to the qemu install script to make installation of an updated version a bit easier. That was the small step  :).

I hope I did everything right by forking the repository, creating a new branch, uploading the commit and creating the pull request on the emu branch (the old sticky thread here has lost all images so I was guessing from the remaining text).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on December 30, 2017, 12:58:19 AM
Applied, thanks.

More cool stuff:

- lots of MPU properties documented (http://www.magiclantern.fm/forum/index.php?topic=17596.msg195296#msg195296), cross-checking welcome
- mode dial emulation (not perfect, but...)
- you can change some MPU-based properties, as long as the change is initiated from the main CPU *)

These are:
- shutter, aperture, ISO
- exposure compensation, flash exposure compensation
- metering mode, drive mode, AF mode
- picture style, white balance
- ExpSim, ALO, HTP, MLU.

*) That means, from ML menus/scripts or from the Q dialog, but not directly from scrollwheels. Some properties (shutter, aperture, AF points) are changed from the MPU (that is, the MPU is expected to interpret the button presses on its own and decide to change these parameters, but this behavior is not emulated). This is a bit model-specific, e.g. you can change shutter speed on 6D or 100D (where the change is made from the main CPU), but not on most other models.

If you can identify other properties that cannot be changed, please report.

It now runs a large part of api_test.lua! (all except photo capture, autofocus, half-shutter and LiveView tests). It even handles the shooting mode switch and those random exposure loops (with animation!)


diff -r f37efb4d8d53 scripts/api_test.lua
--- a/scripts/api_test.lua
+++ b/scripts/api_test.lua
@@ -237,13 +237,6 @@
     assert(camera.gui.play == true)
     assert(camera.gui.mode == 1)

-    -- half-shutter should exit playback mode
-    key.press(KEY.HALFSHUTTER)
-    msleep(1000)
-    assert(camera.gui.play == false)
-    assert(camera.gui.mode == 0)
-    key.press(KEY.UNPRESS_HALFSHUTTER)
-
     -- randomly switch between PLAY, MENU and IDLE (with or without LiveView)
     for i = 1,100 do
         -- we can request MENU or PLAY mode from anywhere
@@ -281,32 +274,8 @@
                 assert(camera.gui.mode == 0)
             end
         end
-
-        -- also play around with LiveView
-        if camera.gui.menu == false and camera.gui.play == false then
-            if math.random(1,2) == 1 then
-                -- do something with LiveView, but not as often as switching MENU/PLAY
-                if not lv.enabled then
-                    printf("Start LiveView...\n");
-                    lv.start()
-                elseif lv.paused then
-                    printf("Resume LiveView...\n");
-                    lv.resume()
-                elseif math.random(1,10) < 9 then
-                    -- this gets taken less often than the next one, why?
-                    -- fixme: biased random?
-                    printf("Pause LiveView...\n");
-                    lv.pause()
-                else
-                    printf("Stop LiveView...\n");
-                    lv.stop()
-                end
-            end
-        end
     end

-    lv.stop()
-
     printf("Canon GUI tests completed.\n")
     printf("\n")
end
@@ -1341,14 +1310,9 @@
     test_io()
     test_camera_gui()
     test_menu()
-    test_camera_take_pics()
     msleep(1000)
     test_multitasking()
-    test_keys()
-    test_lv()
-    test_lens_focus()
     test_camera_exposure()
-    test_movie()
     
     printf("Done!\n")


(http://a1ex.magiclantern.fm/bleeding-edge/qemu/api_test.lua.modeswitch.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/api_test.lua.png)

8)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on January 02, 2018, 09:51:32 PM
Hi,
question:
When running qemu with the -d calls option is it possible to suppress certain calls? If I start a run into FROM Utility the poll serial io call fill up a log file with hundreds of MB quite fast.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on January 02, 2018, 11:03:47 PM
Not directly, but you may use grep (the -C option helps to show calls around some keyword, MMIO register, other function). Or, by hardcoding some custom filters in logging.c.

Most of the time, I find the call stack more helpful than the (huge) call trace. To use that one, run with -d callstack and print it for any function you wish, like this:


b *0x1234
commands
  silent
  print_current_location_with_callstack
  printf "whatever message\n"
  c
end


BTW, just committed a bunch of doc updates, mostly with DryOS internals, debugging tips and similar stuff.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: esas on January 04, 2018, 07:27:53 PM
Quote from: a1ex on November 12, 2017, 08:46:39 PM
The README was (http://www.magiclantern.fm/forum/index.php?topic=2864.msg189925#msg189925) linked (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190254;topicseen#msg190254) a (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190596;topicseen#msg190596) few (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190831#msg190831) times (http://www.magiclantern.fm/forum/index.php?topic=16012.msg190744#msg190744), including first post (http://www.magiclantern.fm/forum/index.php?topic=2864.225) (also asked for some proof-reading).

Think I found a small error. Under headline "Running Canon firmware" shouldn't there be a "-x" in there like this:
./run_canon_fw.sh EOSM,firmware="boot=0" -s -S & arm-none-eabi-gdb -x EOSM/patches.gdb

At least I couldn't get it to work before I found that comment inside the patches.gdb.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on January 04, 2018, 08:00:09 PM
Thanks - that was after I was pretty sure I've checked that section a couple of times.

Exactly that's why I've asked for proof-reading - to make sure it works for new users who were not familiar with QEMU before.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: escho on January 13, 2018, 11:14:32 PM
I´m trying to get ML running into qemu...
... but I run into some little problems

My system is openSUSE Thumbleweed, my cams are 6D and 600D

Starting install.sh from contrib/qemu tells me:

install.sh: Zeile 338: pip2: Kommando nicht gefunden.
install.sh: Zeile 339: pip2: Kommando nicht gefunden.
install.sh: Zeile 339: vncdotool: Kommando nicht gefunden.
install.sh: Zeile 339: pip2: Kommando nicht gefunden.

These messages correspond to these lines in the install script:

# install docutils (for compiling ML modules) and vncdotool (for test suite)
# only install if any of them is missing
pip2 list | grep docutils  || rst2html -h  > /dev/null || pip2 install docutils
pip2 list | grep vncdotool || vncdotool -h > /dev/null || pip2 install vncdotool


pip2 doesn´t exist on my system. So I would have to use pip or pip3. Which one should I prefer?

Since I compile ML, I have docutils installed via paketmanager (Yast). But what about vncdotool? I don´t find any pakets for it (not in the installed repos, not in openSUSE build service).
I´m a bit afraid to install vncdotool outside the paketmanager (using the pip-stuff in the script) and not be able to revert it, if something goes wrong.

Would you recommand to run a VM to play with this stuff?

A last question for the moment:

Why do you tell the user to add gcc/gdb-bins to PATH? Why not a little line in the script  for automation?:
export PATH=$PATH:~/$TOOLCHAIN/bin
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on January 13, 2018, 11:27:23 PM
That's interesting... I'm also running Tumbleweed right now and I have: pip pip2 pip2.7 pip3 pip3.6

Maybe you have to install python2? It should also work with python3/pip3.

You don't really need vncdotool, unless you want to run the test suite (which actually requires a patched vncdotool, I should submit a PR) or the examples that use it.

If you run the "export PATH" command in a bash script, it will only be valid within that script. To run the examples from the guide, one has to have arm-none-eabi-gdb in PATH (or modify the command lines).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: escho on January 14, 2018, 11:33:45 AM
Yes, interesting. "cnf pip2" tells me, that it´s in python2-pip, but python2-pip isn´t installed per default here in Thumbleweed. Got the script running with the two "pip2-lines" commented out.

But now, I´m hanging here:
./run_canon_fw.sh 6D,firmware="boot=1"
I will look later, whats going on.

Thanks for your help, Alex
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on January 17, 2018, 06:22:16 AM
Got a suggestion for the documentation.

After hours of struggling to get QEMU working again on Windows Subsystem for Linux (WSL) after a bunch of Microsoft and Ubuntu updates -- what is really important but not obvious is having to start the X server. BTW, VcXsrv isn't working here after the updates but Xming is and it is just as easy to install and run.

[EDIT] Oh yeah, don't forget this:

export DISPLAY=:0
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on January 22, 2018, 06:16:38 AM
My main development system is a Mac PowerBook and since much of what I've been running in QEMU is not on the qemu branch I've had to unmount the disk image manually. Doing "make install" would trigger the "Error: please unmount the SD image." even though the disk didn't appear as a mounted volumes. Turns out that the fix was to simply "eject" instead of "unmount" -- this works great with both physical cards and disk images. According to the hdiutil man page:

QuoteNOTE: unmount does NOT detach any disk image associated with the volume.
          Images are attached and detached; volumes are mounted and unmounted.

Pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/896/mac-os-x-eject-disk-image/diff) submitted on the qemu branch.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: esas on January 23, 2018, 06:47:42 PM
Quote from: dfort on January 17, 2018, 06:22:16 AM
[EDIT] Oh yeah, don't forget this:

export DISPLAY=:0

And maybe hint on putting this in the $HOME/.bashrc so you don't have to type it every time you launch WSL.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on February 02, 2018, 11:44:59 AM
Will do, need to revive somehow the Windows VM (http://www.magiclantern.fm/forum/index.php?topic=20214.msg195687#msg195687)...

Meanwhile, I did one bold change that I wanted before it gets into mainline: moved the installation directory to qemu-eos (less likely to conflict with your existing vanilla qemu repository, if you've got one).

Options for migrating:
- rename the existing qemu directory to qemu-eos, then run the install script (I prefer this one)
- install from scratch, then manually copy the ROM files and other stuff you may have
- "export QEMU_DIR=qemu" to install in the previous location (or, if you prefer another name...)

The first steps towards this come from an early 100D contributor who was very upset about the default installation path...

BTW, make install_qemu (https://bitbucket.org/hudson/magic-lantern/pull-requests/898) now works from modules as well, and the README also got some nice updates (split in two, since it got too big for bitbucket to render properly):

README.rst (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst) - user-level documentation (installation, how to run, how to use it for debugging ML)
HACKING.rst (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/HACKING.rst) - development and reverse engineering guide (also covering some introductory concepts I could not grasp a few years ago)

Next step: update the videos, fix the remaining OS-specific issues, and if all goes well, it's ready for mainline.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on February 02, 2018, 04:49:43 PM
Got a problem installing a new qemu environment on the Mac. Seems that Apple and/or Homebrew pushed some docutils changes on me.

Here's the fix:

brew upgrade python

Didn't need to update docutils via pip -- go figure. Anyway, if you get this error message try updating python.

./install.sh: line 298: rst2html: command not found
Collecting docutils
  Using cached docutils-0.14-py2-none-any.whl
Installing collected packages: docutils
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 345, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 316, in clobber
    ensure_dir(destdir)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/docutils'
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: g3gg0 on February 03, 2018, 01:59:23 AM
Quote from: a1ex on February 02, 2018, 11:44:59 AM
Will do, need to revive somehow the Windows VM (http://www.magiclantern.fm/forum/index.php?topic=20214.msg195687#msg195687)...
i set up a new VM on magicroot with WSL installed and prepared
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on April 12, 2018, 05:37:33 PM
Found some interesting things about the 50D worth sharing. This probably also applies to the 5D2.

The ROM dumps saved by ML are 0x800000 in size but it appears that the second half of it is a mirror of the first half. This is something that I noticed in the 5D2 disassembly but the 50D ROM dump I was using was only 0x400000 in size so I never noticed this on the 50D until I helped @Asiertxu with some questions about one of my Digic 4 experiments. Since I don't have a 50D and my wife will kill me if I bid on yet another old camera on ebay I thought I'd run the 50D in QEMU. The 0x400000 ROMs didn't work but the the fresh dumps from the camera worked great:

Hola!
(https://farm1.staticflickr.com/865/41368543452_47b411cb80.jpg) (https://flic.kr/p/262AAHb)

I made a QEMU beginner's mistake and loaded ML on the sd.img but of course the 50D only has a CF slot so make sure ML is loaded on the cf.img and voilà!

(https://farm1.staticflickr.com/887/40697114884_98cd59b088.jpg) (https://flic.kr/p/251gmoy)

So my question is why work with two 0x800000 sized firmware dumps when this camera is apparently using only one 0x400000 file? More of a theoretical question rather than a practical one because there's no need to fix something that is working.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 12, 2018, 06:47:55 PM
You can use the small dump if you also declare the right size in model_list.c. The current code was written for the one-size-fits-all ROM backup code that runs at ML startup, so you can use the files auto-saved into ML/LOGS on the card. However, that code is no longer valid for 1300D and D6 models (these have ROMs larger than 16MB), so it's probably a good idea to declare the ROM size, address and possibly RAM size as well, somewhere in platform directory (consts.h? internals.h?)

Unsure where the 8MB dumps come from though, as both the portable ROM dumper and the autobackup code from boot-hack.c save 16MB files, and ROM1 size is declared as 16MB for all DIGIC 4 models (one size fits all, as some models have smaller ROMs).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on April 13, 2018, 03:00:03 AM
Quote from: a1ex on April 12, 2018, 06:47:55 PM
Unsure where the 8MB dumps come from...

I might have gotten them from Silk Road before they were shut down  :P

Seriously though, I understand why someone would split a 5D2 or 50D ROM1.BIN in half before disassembling it. After all, those ROMs are 8MB, right? In addition, only the ROM1.BIN was archived because from my understanding on these and several other cameras the ROM0.BIN is just noise. Now we've got Digic 6 cameras, some with a big fat single ROM. Didn't realize any of this until I started playing around with QEMU.

I suppose that declaring the ROM size will also change the firmware signature. Did you see this issue with the firmware signature on the 600D firmware update (https://www.magiclantern.fm/forum/index.php?topic=15360.msg199718#msg199718)? QEMU to the rescue!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 13, 2018, 07:06:11 AM
OK, but since the ROM size is declared as 16MB in QEMU, you'll get a warning at startup and the memory layout won't be identical to the one from a real camera (the ROM copies will no longer be "complete"). In particular, with an 8MB ROM, no matter what you put into it, QEMU won't cover the bootloader!

BTW, g3gg0 has a long answer (https://www.magiclantern.fm/forum/index.php?topic=6785.msg58899#msg58899) why the ROM is mirrored like that.

On 5D2, ROM1 size (actually ROM0 in Canon strings; I just used g3gg0's notation) is 8MB (if you look at 16MB ROM, you see two mirrored copies, but if you trim it to 8MB, you get unique data in both halves). However, ROM0 is just 4MB on 5D2.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 01, 2018, 05:05:07 AM
Something I've been seeing on the Mac when compiling on some of the branches.

From the ~/qemu-eos directory I'm compiling this way:

make -C ../magic-lantern 6D_install_qemu

And this scrolls by:

[ SCRIPTS  ]   install_extra_data
for target in  CONFIG_QEMU_install  CONFIG_MODULES_install; do /Library/Developer/CommandLineTools/usr/bin/make $target; done
/Library/Developer/CommandLineTools/usr/bin/make -C qemu-eos
make: *** qemu-eos: No such file or directory.  Stop.
make[2]: *** [CONFIG_QEMU_compile] Error 2
/Library/Developer/CommandLineTools/usr/bin/make -C ../../modules


It doesn't seem to cause any problems but it does say Error 2 so thought I'd report it.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 01, 2018, 12:37:02 PM
Appears to be related to this commit (https://bitbucket.org/hudson/magic-lantern/commits/f3ad52ffc80f53ebf8c3639454f5e3f468f8be60?at=qemu).

P.S. emulation ready for DIGIC 7 EOS models (https://www.magiclantern.fm/forum/index.php?topic=19737.msg200799#msg200799) (most of the things were very similar to DIGIC 6).




Here's a proof of concept I'm experimenting with, written as a helper script to keep up with all these firmware updates. It finds a few common stubs from an initial test run, and creates an (updated) debugmsg.gdb.


#!/bin/bash
# Finds some common stubs from an initial test run of some EOS firmware in QEMU.
# Usage: ./find_stubs.sh 6D

CAM="$1"    # camera model
FW="$2"     # firmware version (optional)
DELAY=5     # time (in seconds) for the test run
GREP=${GREP:=grep}

[ "$FW" == "" ] && DEBUGMSG_GDB="$CAM/debugmsg.gdb" || DEBUGMSG_GDB="$CAM/$FW/debugmsg.gdb"

if [ $(uname) == "Darwin" ]; then
    if [[ -n $(which ggrep) ]]; then
        GREP=ggrep
    else
        echo
        echo "Error: you need GNU grep to run this script"
        echo "brew install grep"
        exit 1
    fi

    # Mac doesn't like piping the output of run_canon_fw.sh to other commands (why?!)
    echo "Test run..." 1>&2
    test_run=$( (sleep $DELAY; echo quit) | \
        ./run_canon_fw.sh $CAM,firmware="$FW;boot=0" -d calls,tail -display none -monitor stdio -serial file:uart.log \
        2>&1 )

    test_run=$( echo "$test_run" | ansi2txt )

else # not Mac

    # the above two-step piping would print a warning on Linux
    echo "Test run..." 1>&2
    test_run=$( (sleep $DELAY; echo quit) | \
        ./run_canon_fw.sh $CAM,firmware="$FW;boot=0" -d calls,tail -display none -monitor stdio -serial file:uart.log \
        2>&1 | ansi2txt )

fi

# print firmware version to stderr
( (cat uart.log | $GREP -a -m1 "Firmware Version") || (echo "$test_run" | $GREP -o -m1 '[^"]*Firmware Version [0-9][^"]*') ) 1>&2
echo 1>&2

function clear_line {
    echo -en "\r                                                                                \r"
}

# extract the called function from a line that looks like this:
# call 0x1234(...)
# call 0x1234 DebugMsg(...)
function extract_call {
    head -n1 | $GREP -oP '(?<=call ).*?(?=\()' | cut -d ' ' -f 1
}

# extract the address of a direct jump
# fixme: does not handle multiple direct jumps
# call 0x1234(...) at [init:4444:8888]
#  -> 0xFF1234     at [init:1234:4448]
function extract_jump {
    head -n2 | tail -n1 | $GREP -oP '(?<=-> ).*?(?= +at )' | cut -d ' ' -f 1
}

# looks for a function called with one of the arguments
# returns first match
function find_stub_from_strings {
    for str in "$@"; do

        # status indicator
        clear_line 1>&2
        echo -en "\r$str ..." 1>&2

        # any direct jump? use that
        local stub=$(echo "$test_run" \
                        | $GREP -a -m1 -A1 "$str" \
                        | extract_jump)
        if [ "$stub" != "" ]; then
            echo $stub
            return
        fi

        # regular call?
        local stub=$(echo "$test_run" \
                        | $GREP -a -m1 "$str" \
                        | extract_call)
        if [ "$stub" != "" ]; then
            echo $stub
            return
        fi
    done
}

# fallback - if a stub cannot be found, get it from the existing GDB script
function get_stub_from_gdb_script {
    cat $DEBUGMSG_GDB | $GREP ${1}_log -B 1 | $GREP -Pom1 "(?<=b \*)0x.*"
}

# find some common stubs
DebugMsg=$(find_stub_from_strings startupEntry startupEventDispatch DisablePowerSave)
task_create=$(find_stub_from_strings TaskMain "[a-z]Task" systemtask CmdShell EvShel HotPlug PowerMgr PowerMan)
register_interrupt=$(find_stub_from_strings ICAPCHx OC4_14 SIO3_ISR)
CreateStateObject=$(find_stub_from_strings DMState EMState PropState SRMState)
create_semaphore=$(find_stub_from_strings PropSem mallocLock stdioLock dm_lock)
create_msg_queue=$(find_stub_from_strings MainMessQueue QueueForDeviceIn SystemTaskMSGQueue)

register_func=$(echo "$test_run" \
    | $GREP -a flashwrite \
    | $GREP -B1 -m1 NameService \
    | extract_call)

SIO3_ISR=$(echo "$test_run" | $GREP -a -i SIO3_ISR \
    | $GREP -o SIO3_ISR.* \
    | cut -d ',' -f 3 | tr -d ' ')

MREQ_ISR=$(echo "$test_run" | $GREP -a -i MREQ_ISR \
    | $GREP -o MREQ_ISR.* \
    | cut -d ',' -f 3 | tr -d ' ')

# GDB breakpoints must be without the Thumb bit, if any
function clr_thumb {
    param=$1
    if [[ "$param" != 0x* ]]; then
        param="0x$param"
    fi
    printf "0x%X\n" $((param & ~1))
}

# print a GDB logging stub
# fall back by taking it from the existing stub
function print_logging_stub {
    param=$1

    if [ "${!param}" != "" ]; then
        echo "b *$(clr_thumb ${!param})"
        echo "${param}_log"
    else
        local stub=$(get_stub_from_gdb_script $param)
        if [ "$stub" != "" ]; then
            echo "# from $DEBUGMSG_GDB"
            echo "b *$stub"
            echo "${param}_log"
        else
            echo "# not found"
            echo "# b *0x..."
            echo "# ${param}_log"
        fi
    fi
    echo
}

function print_commented_stub {
    param=$1
    if [ "${!param}" != "" ]; then
        echo "# $(clr_thumb ${!param}) ${param}"
    fi
}

# clear any previous status message
clear_line 1>&2

# output the GDB script
echo                        1>&2
echo "$DEBUGMSG_GDB"        1>&2
echo "====================" 1>&2
echo                        1>&2

echo "# ./run_canon_fw.sh $1 -d debugmsg"
echo "# ./run_canon_fw.sh $1 -d debugmsg -s -S & arm-none-eabi-gdb -x $DEBUGMSG_GDB"
echo
echo "source -v debug-logging.gdb"
echo
cat $DEBUGMSG_GDB | $GREP "To get debugging symbols"
cat $DEBUGMSG_GDB | $GREP "symbol-file"
echo
cat $DEBUGMSG_GDB | $GREP "macro define"
echo

if [ "$DebugMsg" != "" ]; then
    echo "# GDB hook is very slow; -d debugmsg is much faster"
    echo "# ./run_canon_fw.sh will use this address, don't delete it"
    echo "# b *$(clr_thumb $DebugMsg)"
    echo "# DebugMsg_log"
    echo
fi

print_logging_stub task_create
print_logging_stub assert
print_logging_stub register_interrupt
print_logging_stub register_func
print_logging_stub mpu_send
print_logging_stub mpu_recv
print_logging_stub create_semaphore
print_logging_stub create_msg_queue
print_logging_stub CreateStateObject

print_commented_stub SIO3_ISR
print_commented_stub MREQ_ISR


# finished
echo
echo cont


These stubs include DebugMsg, task_create, register_func, register_interrupt and CreateStateObject. Run the firmware in GDB using the generated script to get a few hundreds (possibly thousands) named functions (https://www.magiclantern.fm/forum/index.php?topic=6785.msg200117#msg200117).

Refer to Initial firmware analysis (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/HACKING.rst?fileviewer=file-view-default#rst-header-initial-firmware-analysis) for a sample workflow (this script automates step 5).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 01, 2018, 01:33:53 PM
for us mac fanboys

replace |& with 2>&1 | in the script

go to source forge, download ansi2txt
untar and change the directories in Makefile to /usr/local/....
make && make installl

brew install grep
and replace grep with ggrep in the script

this should get the script running....
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 01, 2018, 01:51:49 PM
Quote from: t3r4n on May 01, 2018, 01:33:53 PM
for us mac fanboys
[...]
this should get the script running....

Thanks; edited the script to add these changes.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 01, 2018, 04:19:04 PM
Quote from: a1ex on May 01, 2018, 12:37:02 PM
Appears to be related to this commit (https://bitbucket.org/hudson/magic-lantern/commits/f3ad52ffc80f53ebf8c3639454f5e3f468f8be60?at=qemu).

Yup, that's it.

There seems to be more issues for Mac fanboys. The script uses ansi2txt which isn't available on the Mac. Ran into this problem before and it is easy enough to install from source (https://www.magiclantern.fm/forum/index.php?topic=2864.msg185552#msg185552). That's a rather long post, here's how to do it:

Quote from: dfort on June 03, 2017, 06:26:55 PM
Mac doesn't have "ansi2txt" and there doesn't seem to be one on Homebrew either. I installed it from the source (https://sourceforge.net/projects/ansi2txt/). Because the Mac keeps you from installing programs in /bin I changed the Makefile like this:BINPATH = /usr/local/bin
MANPATH = /usr/local/share/man/man1


I can see what the script is supposed to do and wow, maybe something like this could be used to automate firmware updates and possibly even speed up new ports. I tried running the script on the 6D.118 firmware because that's the one that is really challenging me but it isn't working on the Mac.

Running just the QEMU command:

./run_canon_fw.sh 6D,firmware="boot=0" -d calls,tail -display none -monitor stdio 2>&1 | ansi2txt

will hang the terminal. Simplifying it to this:

./run_canon_fw.sh 6D,firmware="boot=0" -d calls,tail -display none -monitor stdio

shows the output on the terminal but it doesn't end well.

return from interrupt to fd60 (old=907c)                                                                                              at [:630:387f0]
                                -> 0xFD60                                                  at [:907c:387f0]
R0 changed
Assertion failed: (0), function eos_callstack_log_exec, file /Users/rosiefort/qemu-eos/qemu-2.5.0/hw/arm/../eos/dbi/logging.c, line 1469.
./run_canon_fw.sh: line 153: 20953 Abort trap: 6           env QEMU_EOS_DEBUGMSG="$QEMU_EOS_DEBUGMSG" $QEMU_PATH/arm-softmmu/qemu-system-arm -drive if=sd,format=raw,file=sd.img -drive if=ide,format=raw,file=cf.img -chardev socket,server,nowait,path=qemu.monitor$QEMU_JOB_ID,id=monsock -mon chardev=monsock,mode=readline -name $CAM -M $*
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 01, 2018, 04:42:41 PM
Hey dfort,
hmm strange I see your error when I run without the 2>&1 | ansi2txt portion,
but it seems to work ok when I add it.
I had to increase the time in the script to 60s to get most of the breakpoints right (still missing mpg_rec_log and mph_send_log) but otherwise it is working out.
Question:
When I now start qemu with -s -S and attach the debugger with -x debugmsg.gdb, the gdb will create a temporary breakpoint for each semaphore, msg_queue and so on. These breakpoints can not be stepped by with c only by disabling them? This is not happening while it is running over e.g. the function_register_log, how can I prevent this other than removing it from the .gdb file?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 01, 2018, 06:53:13 PM
Quote from: t3r4n on May 01, 2018, 04:42:41 PM
Question:
When I now start qemu with -s -S and attach the debugger with -x debugmsg.gdb, the gdb will create a temporary breakpoint for each semaphore, msg_queue and so on. These breakpoints can not be stepped by with c only by disabling them? This is not happening while it is running over e.g. the function_register_log, how can I prevent this other than removing it from the .gdb file?

What we have found is that on the Mac with the terminal window default size we get this when running the debugger:

(https://farm1.staticflickr.com/950/40931127915_d211368309.jpg) (https://flic.kr/p/25mWJiZ)

Here's the explanation that a1ex gave on why this is happening:

https://www.magiclantern.fm/forum/index.php?topic=15895.msg196075#msg196075

I found that resizing the terminal window to something about twice as large it will skip that message and maybe it will resolve the issue you're having with the breakpoints.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 01, 2018, 07:01:19 PM
Quote from: t3r4n on May 01, 2018, 04:42:41 PM
Question:
When I now start qemu with -s -S and attach the debugger with -x debugmsg.gdb, the gdb will create a temporary breakpoint for each semaphore, msg_queue and so on. These breakpoints can not be stepped by with c only by disabling them? This is not happening while it is running over e.g. the function_register_log, how can I prevent this other than removing it from the .gdb file?

These temporary breakpoints are to log return values. They work here out of the box (with 32-bit arm-none-eabi-gdb from gcc-arm-none-eabi-5_4-2016q3), but could reproduce your issue with a 64-bit gdb (tried a couple of versions, latest being 8.0.50.20171008-git).

Try compiling a 32-bit GDB for Mac, then let's include that in the installation script.

Quote from: dfort on May 01, 2018, 04:19:04 PM
Assertion failed: (0), function eos_callstack_log_exec, file /Users/rosiefort/qemu-eos/qemu-2.5.0/hw/arm/../eos/dbi/logging.c, line 1469.

Repeatable? Can you find a value for -icount that would reproduce the issue? The -icount option ensures repeatable emulation (for example, timers and other interrupts will always fire at the same time, so you'll get the same execution trace every time). By default, emulation follows the real-time clock to some extent, but is not deterministic.

Here it boots just fine to Canon menu, with both 1.1.6 and 1.1.8, with or without -icount 5, but emulation is very slow.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 01, 2018, 07:06:44 PM
oh you mean the warning with 64 bits and issues and stuff  :D never heard from that  8)
I'll look how to get a 32 bit gdb to work. From what I've read in the past it might be time consuming.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 01, 2018, 09:03:54 PM
Quote from: a1ex on May 01, 2018, 07:01:19 PM
Repeatable?

Yes, as long as "-d calls" is included.

Quote from: a1ex on May 01, 2018, 07:01:19 PM
Can you find a value for -icount that would reproduce the issue?

Interesting--this works:

./run_canon_fw.sh 6D,firmware="boot=0" -icount 5 -d calls

So I tried different values and found that "-icount 7" will reproduce the issue. Other values I tried wouldn't reproduce that issue though "-icount 100" and "-icount 0" wouldn't bring up the Canon menu -- just testing the limits.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on May 02, 2018, 12:59:42 AM
Going to make another attempt to get QEMU running on my AMD64 , I tried to run my VM Ubuntu development VDI from amd 64 in my i7 laptop with latest version (5.2.10) of Oracle Virtualbox.
But no go , all I have access to is i386 (32bit) versions of Linux on the Intel i7 cpu , so it will not load the Ubuntu 64 vdi   >:(
For some reason I can't get a 64bit version of  Virtualbox keeps loading as i386 , Unless someone know a work around ?
I didn't what to load a fresh 32bit version of Ubuntu as my 64bit has all the Toolchains already enabled plus I has all my source code in there .

Not to sure if this is possible to run under Win7 (I just what to run the Emulator with my 5d2 rom) So I found the latest version ( 2.11.90) https://qemu.weilnetz.de/w64/2018/
it seem to be a experimental version for 64bit on windows platform ,
I see ARMS support , what would be the  differences between "arm.exe" & "armw.exe" 64bit ?
(https://image.ibb.co/hwtTES/qemu.png) (https://imgbb.com/)(https://preview.ibb.co/nKUtES/qemu4.png) (https://ibb.co/meAN8n)
does emulator need to be in 32bit mode ? is the Arm on canon 32 or 64bit ?
It looks to be all self contain , dll's etc. ... has a folder for rom's & bin files
(https://image.ibb.co/jTTZon/qemu1.png) (https://imgbb.com/)(https://image.ibb.co/eQF5Tn/qemu2.png) (https://imgbb.com/)

Do I need still need a tool chains etc. .... or other related software or is this not possible -- do I need to run this under Linux ? 


Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on May 02, 2018, 06:55:11 AM
run C:\Program Files\qemu>qemu-system-arm -machine help and got this list
interesting that out of the box it can emulate "canon-a1100----------Canon PowerShot A1100 IS"
not that I know what the hell I'm doing , but I guess gotta start some where  :)
maybe helpful with some info for reverse engineering

Supported machines are:
akita                Sharp SL-C1000 (Akita) PDA (PXA270)
ast2500-evb          Aspeed AST2500 EVB (ARM1176)
bast                 Simtec Electronics BAST (S3C2410A, ARM920T)
borzoi               Sharp SL-C3100 (Borzoi) PDA (PXA270)
canon-a1100          Canon PowerShot A1100 IS
cheetah              Palm Tungsten|E aka. Cheetah PDA (OMAP310)
collie               Sharp SL-5500 (Collie) PDA (SA-1110)
connex               Gumstix Connex (PXA255)
cubieboard           cubietech cubieboard
emcraft-sf2          SmartFusion2 SOM kit from Emcraft (M2S010)
highbank             Calxeda Highbank (ECX-1000)
imx25-pdk            ARM i.MX25 PDK board (ARM926)
integratorcp         ARM Integrator/CP (ARM926EJ-S)
kzm                  ARM KZM Emulation Baseboard (ARM1136)
lm3s6965evb          Stellaris LM3S6965EVB
lm3s811evb           Stellaris LM3S811EVB
mainstone            Mainstone II (PXA27x)
mcimx7d-sabre        Freescale i.MX7 DUAL SABRE (Cortex A7)
midway               Calxeda Midway (ECX-2000)
mps2-an385           ARM MPS2 with AN385 FPGA image for Cortex-M3
mps2-an505           ARM MPS2 with AN505 FPGA image for Cortex-M33
mps2-an511           ARM MPS2 with AN511 DesignStart FPGA image for Cortex-M3
musicpal             Marvell 88w8618 / MusicPal (ARM926EJ-S)
n800                 Nokia N800 tablet aka. RX-34 (OMAP2420)
n810                 Nokia N810 tablet aka. RX-44 (OMAP2420)
netduino2            Netduino 2 Machine
none                 empty machine
nuri                 Samsung NURI board (Exynos4210)
palmetto-bmc         OpenPOWER Palmetto BMC (ARM926EJ-S)
raspi2               Raspberry Pi 2
realview-eb          ARM RealView Emulation Baseboard (ARM926EJ-S)
realview-eb-mpcore   ARM RealView Emulation Baseboard (ARM11MPCore)
realview-pb-a8       ARM RealView Platform Baseboard for Cortex-A8
realview-pbx-a9      ARM RealView Platform Baseboard Explore for Cortex-A9
romulus-bmc          OpenPOWER Romulus BMC (ARM1176)
sabrelite            Freescale i.MX6 Quad SABRE Lite Board (Cortex A9)
smdk2443             smdk2443 (ARM920-T)
smdkc210             Samsung SMDKC210 board (Exynos4210)
spitz                Sharp SL-C3000 (Spitz) PDA (PXA270)
sx1                  Siemens SX1 (OMAP310) V2
sx1-v1               Siemens SX1 (OMAP310) V1
terrier              Sharp SL-C3200 (Terrier) PDA (PXA270)
tosa                 Sharp SL-6000 (Tosa) PDA (PXA255)
tt                   OpenTom (ARM920-T)
tt666                OpenTom (ARM920-T)
verdex               Gumstix Verdex (PXA270)
versatileab          ARM Versatile/AB (ARM926EJ-S)
versatilepb          ARM Versatile/PB (ARM926EJ-S)
vexpress-a15         ARM Versatile Express for Cortex-A15
vexpress-a9          ARM Versatile Express for Cortex-A9
virt-2.10            QEMU 2.10 ARM Virtual Machine
virt-2.11            QEMU 2.11 ARM Virtual Machine
virt                 QEMU 2.12 ARM Virtual Machine (alias of virt-2.12)
virt-2.12            QEMU 2.12 ARM Virtual Machine
virt-2.6             QEMU 2.6 ARM Virtual Machine
virt-2.7             QEMU 2.7 ARM Virtual Machine
virt-2.8             QEMU 2.8 ARM Virtual Machine
virt-2.9             QEMU 2.9 ARM Virtual Machine
xilinx-zynq-a9       Xilinx Zynq Platform Baseboard for Cortex-A9
z2                   Zipit Z2 (PXA27x)

So is the QEMU that is used here to run ML , different then what I have here ?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 02, 2018, 09:29:22 AM
- First post from this thread
- Main page -> Docs -> QEMU guide
- Main page -> big red QEMU link
- Latest commit (https://bitbucket.org/hudson/magic-lantern/commits/8b9b81bac630c5d9c4084b9f2cbe86cde7969cb5) (at the time of writing)
- Twitter -> sticky post -> video (https://twitter.com/autoexec_bin/status/913530810686418944)
- Getting started with development (https://www.magiclantern.fm/forum/index.php?topic=991)
- Mac guide (https://www.magiclantern.fm/forum/index.php?topic=16012) with video (https://www.magiclantern.fm/forum/index.php?topic=16012.msg191686#msg191686)
- Windows 10 guide (https://www.magiclantern.fm/forum/index.php?topic=20214)
- docs (https://www.magiclantern.fm/forum/index.php?topic=21311.msg194814#msg194814) linked (https://www.magiclantern.fm/forum/index.php?topic=17969.msg199888#msg199888) all (https://www.magiclantern.fm/forum/index.php?topic=1452.msg195051#msg195051) over (https://www.magiclantern.fm/forum/index.php?topic=2054.msg190271#msg190271) the (https://www.magiclantern.fm/forum/index.php?topic=19737.msg200799#msg200799) place (https://www.magiclantern.fm/forum/index.php?topic=17360.msg195002;topicseen#msg195002)

Should I go on?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on May 02, 2018, 10:40:12 AM
Quote from: a1ex on May 02, 2018, 09:29:22 AM
- Twitter -> sticky post -> video (https://twitter.com/autoexec_bin/status/913530810686418944)
Should I go on?

Please recommend the exact version of linux you are using(have used) for it.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 02, 2018, 11:09:19 AM
That was Ubuntu Xenial, but should work just as well in any recent distribution. Was your experience different?

(I'm currently using it on openSUSE Tumbleweed)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 02, 2018, 06:20:33 PM
@a1ex - just wondering if you ran the find_stubs.sh script on the 6D.118 and if it came up the same changes I made on 6D/debugmsg.gdb.

I take it that in order to get the full QEMU messages working on this platform I'll need to run a startup log through the scripts to make a new qemu-eos/qemu-2.5.0/hw/eos/mpu_spells/6D.h -- right?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 02, 2018, 07:28:46 PM
I didn't do any changes to the MPU spells, but the script found some different stubs. It still requires updating CURRENT_TASK / CURRENT_ISR manually.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 02, 2018, 09:33:51 PM
Hey a1ex,
its a bit rough for start but it works. Seems like gdb 8 resolves the issue with the temporary breakpoints.


#!/bin/bash

# Mirror of gnu.org to be used.
MIRROR=https://ftp.gnu.org/gnu

#create Directory
mkdir ~/crossgcc
cd ~/crossgcc

mkdir src
cd src

# get a bunch of stuff
wget -c $MIRROR/gdb/gdb-8.1.tar.xz

# let's unpack
tar jxf gdb-8.1.tar.xz

# now build
# read about CC='gcc -m32' but also that newer gdb should handle 32 bit fine ...
mkdir build-gdb
cd build-gdb
../gdb-8.1/configure --target=arm-none-eabi --prefix=$HOME/crossgcc/
make all install 2>&1 | tee make.log
echo "Done, please add: "
echo $HOME/crossgcc/bin
echo "to your PATH"

Can someone with a mac verify.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 02, 2018, 10:26:20 PM
@t3r4n -- Tried out your script and it worked perfectly. At least I think it did.

Quote from: a1ex on May 02, 2018, 07:28:46 PM
It still requires updating CURRENT_TASK / CURRENT_ISR manually.

I'm pretty sure I got CURRENT_TASK correct for the 6D.118 but have no idea where to look for CURRENT_ISR:

6D/debugmsg.gdb
macro define CURRENT_TASK 0x74C80
macro define CURRENT_ISR  (MEM(0x648) ? MEM(0x64C) >> 2 : 0)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on May 03, 2018, 04:35:38 AM
Quote from: a1ex on May 02, 2018, 09:29:22 AM
- First post from this thread
- Main page -> Docs -> QEMU guide
- Main page -> big red QEMU link
- Latest commit (https://bitbucket.org/hudson/magic-lantern/commits/8b9b81bac630c5d9c4084b9f2cbe86cde7969cb5) (at the time of writing)
- Twitter -> sticky post -> video (https://twitter.com/autoexec_bin/status/913530810686418944)
- Getting started with development (https://www.magiclantern.fm/forum/index.php?topic=991)
- Mac guide (https://www.magiclantern.fm/forum/index.php?topic=16012) with video (https://www.magiclantern.fm/forum/index.php?topic=16012.msg191686#msg191686)
- Windows 10 guide (https://www.magiclantern.fm/forum/index.php?topic=20214)
- docs (https://www.magiclantern.fm/forum/index.php?topic=21311.msg194814#msg194814) linked (https://www.magiclantern.fm/forum/index.php?topic=17969.msg199888#msg199888) all (https://www.magiclantern.fm/forum/index.php?topic=1452.msg195051#msg195051) over (https://www.magiclantern.fm/forum/index.php?topic=2054.msg190271#msg190271) the (https://www.magiclantern.fm/forum/index.php?topic=19737.msg200799#msg200799) place (https://www.magiclantern.fm/forum/index.php?topic=17360.msg195002;topicseen#msg195002)

Should I go on?
Yes I know ,
I don't have Win10 & I only use MAC for Video editing (I don't know MAC good enough to understand) (I'm windows sever2008 & win7 guy)
That's the reason I'm using a prebuilt VM(Ubuntu) from Quick Guide and Solutions for VirtualBox  (https://www.magiclantern.fm/forum/index.php?topic=7579.msg134989#msg134989)
I guess I should of mention that I tried to get it running mouths ago but run in to AMD cpu problem (I have a FX8350) and it needed some extra support files
that I could not get to work , I posted it in the thread Qemu problem  (https://www.magiclantern.fm/forum/index.php?topic=19080.msg180528#msg180528) I make back in Feb/'17 but couldn't resolve it so I moved on .
That's why I'm looking for other solutions , as I noted above I wanted to just run the VM (Ubuntu) on my i7 laptop but will not load the VDI as its 64bit
thou I laptop is only a few year old so it does support 64bit , I run MAC OS on it thought Multi Beast boot loader . (It's my Mac backup & portable video editing station)
VirtualBox keep loading the 32bit version of the software on my i7 laptop instead of the 64bit version that running on my AMD desktop .

I don't need it to compile ml just need to run qemu so I can get the final info for lossless
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 03, 2018, 07:44:27 AM
From the first section:
Quote
What does not work (yet):
...
- Native Windows build (QEMU can be compiled on Windows => contribution welcome).
...

Given that, I've assumed you jumped into trying unsupported stuff without reading the guide. Actually I'm still pretty sure about that, as the steps you have tried are in the opposite direction.

https://wiki.qemu.org/Hosts/W32 (caveat: I did not try it)

I've ran into a similar problem with some older Ubuntu machines - their repositories were moved to archive.ubuntu.org:

https://superuser.com/questions/339537/where-can-i-get-the-repositories-for-old-ubuntu-versions

However, I don't remember testing this on a 32-bit VM, so there may be surprises. Will give it a try later.

Edit: there were a few build errors, fixed. After that, installation on Xenial 32-bit was as straightforward as in the Twitter clip.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on May 03, 2018, 08:06:37 AM
 Ok , I'll keep at it -- thank
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 03, 2018, 08:11:05 PM
Okay thanks @dfort for testing.
Should I turn this script into a complete cross chain builder that in the end might even reside in the ML directory universe? That way we would have an reference compiler chain that should render the same results on all platforms (notice the should and not will ;) ).

But back to the original topic on finding stubs. @a1ex I now get a lot of functions with weird binary names in the idc file. Is that normal or do I need to investigate?

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 03, 2018, 08:26:32 PM
Also confirmed gdb 8.1 handles the temporary breakpoints well. However, latest precompiled toolchains are using older versions that do not have this fix.

Currently, I only use gdb with qemu, so it may be a good idea to have it in the install script. The same issue is on WSL; hopefully the steps for compiling gdb are the same.

Can you give an example of a function with weird binary names? On 750D, with the autogenerated GDB script I get 549 named functions. All of them appear valid to me at first sight. Some examples:

  MakeAutoNamedFunc(0xFE0FD5D1, "LoadScript");
  MakeAutoNamedFunc(0xFE27B4D7, "UTimerDriver_ISR");
  MakeAutoNamedFunc(0xFE1F82F1, "PowerMgr_task");
  MakeAutoNamedFunc(0xFE42ADE3, "SRMState_S00_I00");


The other idc, saved from QEMU with -d idc, does not have any named functions; that one only has functions called during execution (their start address).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 03, 2018, 09:45:54 PM
Okay I've started completely clean and I now get a lot more functions but still a lot of these:

  MakeAutoNamedFunc(0x00002404, "�F�F��V��_task");
  MakeAutoNamedFunc(0x0000240E, "�F�F��V��_task");
  MakeAutoNamedFunc(0x00002418, "�F�F��V��_task");
  MakeAutoNamedFunc(0x00002422, "�F�F��V��_task");

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 03, 2018, 10:07:58 PM
What stub did you get for task_create?

I've got:

b *0x1E44
task_create_log


then, in the console log:

[        init:fe1f8385 ] task_create(PowerMgr, prio=20, stack=400, entry=fe1f82f1, arg=0)
[        init:000022b7 ] task_create(DbgMgr, prio=1f, stack=0, entry=2233, arg=7fe800)
[        init:fe0d9fdb ] task_create(Startup, prio=19, stack=2800, entry=fe0d9ecd, arg=7feb3c)
[        init:fe0ce221 ] task_create(RomRead, prio=11, stack=400, entry=fe0cd53f, arg=0)
...


Do you get different things?

I get the same result with both the 64-bit GDB compiled with your script, and with my old 32-bit version.

Your addresses don't seem to be valid function start addresses btw. Where do you get the task_create call for these? Hopefully these are not tasks created by ML - run with boot=0 to be sure.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 04, 2018, 06:42:10 AM
It found the task create at b *0xFE172BB2
task_create_log

And no command line is:
./run_canon_fw.sh 750D,firmware="boot=0" -d debugmsg -s -S
Will try to reset to default task_create.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 04, 2018, 09:26:55 AM
May I see your call trace around the first occurence of TaskMain?


CAM=750D

test_run=$( (sleep 2; echo quit) | \
    ./run_canon_fw.sh $CAM,firmware="boot=0" -d calls,tail -display none -monitor stdio \
    2>&1 | ansi2txt )

echo "$test_run" | grep -m1 -C5 TaskMain


My result (relevant bits only):

  call 0xFE3CDF4C(fe0ce600 "TaskMain", 1d, 0, fe0cd4a9)                          at [init:fe0ce241:80001735]
   -> 0x1E45                                                                     at [init:fe3cdf4c:fe0ce245]


Address 0xFE172BB2/3 is not present in my call trace (?!)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 04, 2018, 06:41:27 PM
hmm ...

9:1eb5]
   return 0 to 0x1EB5                                                                               at [init:80000d23:fe
0ce235]
  return 48000e to 0xFE0CE235                                                                                at [init:1e
81:80001735]
  call 0xFE3CDF44(fe0ce600 "TaskMain", 1d, 0, fe0cd4a9)

looks different
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 04, 2018, 06:51:20 PM
Terminal window too small? Was grep -C5 used?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 04, 2018, 07:16:38 PM
Terminal full screen width.
And I've now even used ggrep and declared export TERM=ansi

      call 0xFE3CDE84(c0003, 60000053, 1, 5)
         at [SFRead:fe32bd4f:fe32a717]
       -> 0x186F                                                                           at [SFRead:fe3cde84:fe32bd53]

        call 0xFE3CDF94(90007, 0, 73, 0)
   at [TaskMain:fe1c1c7f:fe2eb9fb]
         -> 0x800020B3                                                                         at [TaskMain:fe3cdf94:fe1
c1c83]
         call 0x800056DC(2ee300, 0, 73,  0)
    at [TaskMain:800020bb:fe1c1c83]
          -> 0xFE172B85                                                                        at [TaskMain:800056dc:800


I thought I had left such problems back in 1998 ...  :(
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 04, 2018, 07:42:47 PM
Tried smaller font? Or writing to a log file and copying the output from a text editor?

The second snippet can't be the first occurrence of TaskMain, btw.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 04, 2018, 07:54:00 PM
font :
nope ..
here is the output of the find  stub script

b *0xFE172BB2
task_create_log

# from 750D/debugmsg.gdb
b *0xFE52F980
assert_log

# from 750D/debugmsg.gdb
b *0x1774
register_interrupt_log

# from 750D/debugmsg.gdb
b *0xFE445CB8
register_func_log

# from 750D/debugmsg.gdb
b *0x...
mpu_send_log

# from 750D/debugmsg.gdb
b *0x...
mpu_recv_log

b *0xFE3CDFE4
create_semaphore_log

b *0x1C18
create_msg_queue_log

b *0x211A
CreateStateObject_log

for today I'm done ... I'll grab a beer and the camera and enjoy the nice weather and sunset out at the lake ;)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 05, 2018, 09:05:44 AM
I was re-reading some of the posts and got excited when I realized that most of the lua API tests can run in QEMU as posted on Reply #254 (https://www.magiclantern.fm/forum/index.php?topic=2864.msg195347#msg195347). So I applied the patch to api_test.lua and tried out the 1100D.106 lua_fix build.

It got through almost everything on the first try:


===============================================================================
ML/SCRIPTS/API_TEST.LUA - 2017-9-30 12:15:00
===============================================================================

Strict mode tests...
Strict mode tests passed.

Generic tests...
arg = table:
  [0] = "API_TEST.LUA"
camera = table:
  shutter = table:
    raw = 104
    apex = 6.
    ms = 16
    value = 0.015625
  aperture = table:
    raw = 75
    apex = 8.375
    value = 18.2
    min = table:
      raw = 45
      apex = 4.625
      value = 4.9
    max = table:
      raw = 88
      apex = 10.
      value = 32
  iso = table:
    raw = 72
    apex = 5.
    value = 100
  ec = table:
    raw = 0
    value = 0
  flash = true
  flash_ec = table:
    raw = 0
    value = 0
  kelvin = 5200
  mode = 3
  metering_mode = 3
  drive_mode = 4
  model = "Canon EOS 1100D"
  model_short = "1100D"
  firmware = "1.0.6"
  temperature = 146
  gui = table:
    menu = false
    play = false
    play_photo = false
    play_movie = false
    qr = false
    idle = true
  shoot = function: p
  bulb = function: p
  reboot = function: p
  wait = function: p
  burst = function: p
event = table:
  pre_shoot = nil
  post_shoot = nil
  shoot_task = nil
  seconds_clock = nil
  keypress = nil
  custom_picture_taking = nil
  intervalometer = nil
  config_save = nil
console = table:
  hide = function: p
  write = function: p
  show = function: p
  clear = function: p
lv = table:
  enabled = false
  paused = false
  running = false
  zoom = 1
  overlays = false
  pause = function: p
  resume = function: p
  start = function: p
  wait = function: p
  info = function: p
  stop = function: p
lens = table:
  name = "EF-S18-55mm f/3.5-5.6 IS"
  focal_length = 0
  focus_distance = 14080
  hyperfocal = 0
  dof_near = 0
  dof_far = 0
  af = false
  af_mode = 3
  focus = function: p
  autofocus = function: p
display = table:
  idle = nil
  height = 480
  width = 720
  off = function: p
  print = function: p
  notify_box = function: p
  pixel = function: p
  screenshot = function: p
  draw = function: p
  load = function: p
  rect = function: p
  circle = function: p
  clear = function: p
  on = function: p
  line = function: p
key = table:
  last = 10
  wait = function: p
  press = function: p
menu = table:
  visible = false
  close = function: p
  block = function: p
  open = function: p
  get = function: p
  new = function: p
  select = function: p
  set = function: p
movie = table:
  recording = false
  start = function: p
  stop = function: p
dryos = table:
  clock = 5
  ms_clock = 5081
  image_prefix = "IMG_"
  dcim_dir = table:
    exists = true
    create = function: p
    children = function: p
    files = function: p
    parent = table:
      exists = true
      create = function: p
      children = function: p
      files = function: p
      parent = table:
        exists = true
        create = function: p
        children = function: p
        files = function: p
        parent = nil
        path = "B:/"
      path = "B:/DCIM/"
    path = "B:/DCIM/100CANON/"
  config_dir = table:
    exists = true
    create = function: p
    children = function: p
    files = function: p
    parent = table:
      exists = true
      create = function: p
      children = function: p
      files = function: p
      parent = table:
        exists = true
        create = function: p
        children = function: p
        files = function: p
        parent = nil
        path = "B:/"
      path = "ML/"
    path = "ML/SETTINGS/"
  ml_card = table:
    cluster_size = 16384
    drive_letter = "B"
    file_number = 9321
    folder_number = 100
    free_space = 215520
    type = "SD"
    _card_ptr = userdata
    path = "B:/"
  shooting_card = table:
    cluster_size = 16384
    drive_letter = "B"
    file_number = 9321
    folder_number = 100
    free_space = 215520
    type = "SD"
    _card_ptr = userdata
    path = "B:/"
  date = table:
    hour = 12
    yday = 1
    month = 9
    isdst = false
    sec = 0
    day = 30
    min = 15
    year = 2017
    wday = 2
  rename = function: p
  remove = function: p
  call = function: p
  directory = function: p
interval = table:
  time = 10
  count = 0
  running = false
  stop = function: p
battery = table:
function not available on this camera
stack traceback:
[C]: in ?
[C]: in for iterator 'for iterator'
ML/SCRIPTS/LIB/logger.lua:125: in function 'logger.serialize'
ML/SCRIPTS/API_TEST.LUA:36: in function <ML/SCRIPTS/API_TEST.LUA:35>
[C]: in function 'xpcall'
ML/SCRIPTS/API_TEST.LUA:35: in function 'print_table'
ML/SCRIPTS/API_TEST.LUA:81: in function 'generic_tests'
ML/SCRIPTS/API_TEST.LUA:1307: in function 'api_tests'
ML/SCRIPTS/API_TEST.LUA:1328: in main chunktask = table:
  yield = function: p
  create = function: p
property = table:
Generic tests completed.

Module tests...
Testing file I/O...
Copy test: autoexec.bin -> tmp.bin
Copy test OK
Append test: tmp.txt
Append test OK
Rename test: apple.txt -> banana.txt
Rename test OK
Rename test: apple.txt -> ML/banana.txt
Rename test OK
File I/O tests completed.

Testing Canon GUI functions...
Enter MENU mode...
Enter PLAY mode...
Exit PLAY mode...
Enter MENU mode...
Enter PLAY mode...
Enter MENU mode...
Enter PLAY mode...
Exit PLAY mode...
Enter PLAY mode...
Exit PLAY mode...
Enter PLAY mode...
Enter PLAY mode...
Enter MENU mode...
Enter PLAY mode...
Enter PLAY mode...
Enter PLAY mode...
Exit PLAY mode...
Enter MENU mode...
Enter MENU mode...
Exit MENU mode...
Enter PLAY mode...
Enter PLAY mode...
Exit PLAY mode...
Enter MENU mode...
Enter MENU mode...
Exit MENU mode...
Enter PLAY mode...
Enter MENU mode...
Enter PLAY mode...
Enter MENU mode...
Exit MENU mode...
Enter PLAY mode...
Enter PLAY mode...
Exit PLAY mode...
Enter PLAY mode...
Enter MENU mode...
Enter MENU mode...
Exit MENU mode...
Enter PLAY mode...
Exit PLAY mode...
Enter MENU mode...
Enter PLAY mode...
Enter PLAY mode...
Enter MENU mode...
Exit MENU mode...
Enter PLAY mode...
Exit PLAY mode...
Enter MENU mode...
Enter PLAY mode...
Enter MENU mode...
Enter MENU mode...
Enter MENU mode...
Enter MENU mode...
Exit MENU mode...
Enter PLAY mode...
Enter MENU mode...
Enter MENU mode...
Enter MENU mode...
Exit MENU mode...
Enter MENU mode...
Enter PLAY mode...
Enter MENU mode...
Enter MENU mode...
Enter MENU mode...
Enter PLAY mode...
Enter MENU mode...
Exit MENU mode...
Canon GUI tests completed.

Testing ML menu API...
Menu tests completed.

Testing multitasking...
Only one task allowed to interrupt...
Main task yielding.
Task C started.
Task C finished.
Main task back.
Main task yielding.
Task C started.
Task C finished.
Main task back.
Main task yielding.
Task C started.
Task C finished.
Main task back.
Main task yielding.
Task C started.
Task C finished.
Main task back.
Main task yielding.
Task C started.
Task C finished.
Main task back.
Main task yielding.
Task C started.
Task C finished.
Main task back.
Main task yielding.
Task C started.
Task C finished.
Main task back.
Main task yielding.
Task C started.
Task C finished.
Main task back.
Main task yielding.
Task C started.
Task C finished.
Main task back.
Main task yielding.
Task C started.
Task C finished.
Main task back.
Multitasking tests completed.

Testing exposure settings...
Camera    : Canon EOS 1100D (1100D) 1.0.6
Lens      : EF-S18-55mm f/3.5-5.6 IS
Shoot mode: 3
Shutter   : Ç60 (raw 104, 0.015625s, 16ms, apex 6.)
Aperture  : Å18 (raw 75, f/18.2, apex 8.375)
Av range  : Å4.9..Å32 (raw 45..88, f/4.9..f/32, apex 4.625..10.)
ISO       : Ä1600 (raw 104, 1600, apex 9.)
EC        : 0.0 (raw 0, 0 EV)
Flash EC  : 0.0 (raw 0, 0 EV)
Setting shutter to random values...



It was pretty cool watching it go through the tests in QEMU but it ended up like this--stuck on the exposure test:

(https://farm1.staticflickr.com/975/41857680052_9cdf6e6a07.jpg) (https://flic.kr/p/26LPxYQ)

Hum--is that a memory issue? This camera has very little memory.

So I tried running just the exposure test and got a bit further:

===============================================================================
ML/SCRIPTS/API_TEST.LUA - 2017-9-30 12:15:00
===============================================================================

Module tests...
Testing exposure settings...
Camera    : Canon EOS 1100D (1100D) 1.0.6
Lens      : EF-S18-55mm f/3.5-5.6 IS
Shoot mode: 3
Shutter   : Ç60 (raw 104, 0.015625s, 16ms, apex 6.)
Aperture  : Å18 (raw 75, f/18.2, apex 8.375)
Av range  : Å4.9..Å32 (raw 45..88, f/4.9..f/32, apex 4.625..10.)
ISO       : Ä100 (raw 72, 100, apex 5.)
EC        : 0.0 (raw 0, 0 EV)
Flash EC  : 0.0 (raw 0, 0 EV)
Setting shutter to random values...
Setting ISO to random values...
Setting aperture to random values...
Please switch to Av mode.


Now the problem is--how do you switch this camera to Av mode? It is done by the mode dial on the camera so ???
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 05, 2018, 10:13:12 AM
Hey a1ex,
I got it working.
- As written before export TERM=ansi
- font to 11 and fullscreen does only work up to a point and the output will wrap on the middle of screen.
- I had to eliminate the monitor and nodisplay options and run qemu into a file (this way I discovered it would crash, but it was enough to get the task and other calls except mpu).
- put something like :test_run=$(cat test_run.txt) in the script.

The weekend is full now but maybe I can post some progress on the 750D now on sunday night.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 05, 2018, 10:48:20 AM
Also got it working some minutes ago (ran the script in a Mac VM). Turns out:

- the output is broken if QEMU (run_canon_fw.sh) and ansi2txt are executed both in the same command => bad result
- the output is correctly formatted if the QEMU output is stored in a variable and then passed through ansi2txt => correct result

The same happens with bash 3.2 (that comes with Mac) and 4.4 (brew install bash).

The issue is present if piping to any other command (such as tr).

Quote from: dfort on May 05, 2018, 09:05:44 AM
Now the problem is--how do you switch this camera to Av mode? It is done by the mode dial on the camera so ???

Press F1:


[MPU] Available keys:
...
- 0/9          : Mode dial (press only)
- V            : Movie mode (press only)
...


Wait a minute, does the Mac have numeric keys? They seem to work in the VM, but the same is true for buttons in Canon firmware present on other models :D
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 05, 2018, 04:05:01 PM
Doh! Note to self--don't ask questions after midnight.

Quote from: a1ex on May 05, 2018, 10:48:20 AM
Wait a minute, does the Mac have numeric keys?

Yes, and the 0 and 9 keys do indeed go through the mode dial options. V switches to movie mode.

Been trying to follow the find_stubs.sh changes to get it working on Mac but still no luck over here.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 05, 2018, 05:52:02 PM
Updated the script to address the above issue; this time it should work on Mac.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 05, 2018, 07:55:25 PM
Hi a1ex,
it's a bit confusing for newbies like me that you are updating the original posts. I was looking in hg for the script and thought man what did I miss ... but found it on page 12 ;)
The alias for Mac doesn't work ... may I suggest the following

...
GREP=grep
if [ $(uname) == "Darwin" ]; then
    if [[ -n $(which ggrep) ]]; then
        export GREP=ggrep
    else
        echo
        echo "Error: you need GNU grep to run this script"
        echo "brew install grep"
        exit 1
    fi
...

and then a s/grep/\$GREP/g
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 05, 2018, 08:34:10 PM
Hum--The script on Reply #273 (https://www.magiclantern.fm/forum/index.php?topic=2864.msg200846#msg200846) is not working out of the box yet.

I was trying to figure out what is up with the alias command on the Mac when t3r4n posted his suggestion. Got past that hump but it is still not working. Thought I'd take a look at what is in the test_run variable like this:

    test_run=$( echo "$test_run" | ansi2txt )

    echo $test_run
    exit 1

else # not Mac


This can't be right:

./find_stubs.sh 6D
Test run...
CHK version_gen.h ./run_canon_fw.sh 6D,firmware=;boot=0 -d calls,tail -display none -monitor stdio -serial file:uart.log DebugMsg=0x6824 (from GDB script) QEMU 2.5.0 monitor - type 'help' for more information (qemu) Lockdown read 0 Lockdown read 0 Lockdown read 1 Lockdown read 1 Lockdown read 2 Lockdown read 2 Lockdown read 3 Lockdown read 3 Lockdown read 4 Lockdown read 4 00000000 - 00000FFF: eos.tcm_code 40000000 - 40000FFF: eos.tcm_data 00001000 - 1FFFFFFF: eos.ram 40001000 - 5FFFFFFF: eos.ram_uncached F0000000 - F0FFFFFF: eos.rom0 F1000000 - F1FFFFFF: eos.rom0_mirror F2000000 - F2FFFFFF: eos.rom0_mirror F3000000 - F3FFFFFF: eos.rom0_mirror F4000000 - F4FFFFFF: eos.rom0_mirror F5000000 - F5FFFFFF: eos.rom0_mirror F6000000 - F6FFFFFF: eos.rom0_mirror F7000000 - F7FFFFFF: eos.rom0_mirror F8000000 - F8FFFFFF: eos.rom1 F9000000 - F9FFFFFF: eos.rom1_mirror FA000000 - FAFFFFFF: eos.rom1_mirror FB000000 - FBFFFFFF: eos.rom1_mirror FC000000 - FCFFFFFF: eos.rom1_mirror FD000000 - FDFFFFFF: eos.rom1_mirror FE000000 - FEFFFFFF: eos.rom1_mirror FF000000 - FFFFFFFF: eos.rom1_mirror C0000000 - DFFFFFFF: eos.mmio [EOS] enabling code execution logging. [EOS] enabling memory access logging (R). [EOS] enabling singlestep. [EOS] loading './6D/ROM0.BIN' to 0xF0000000-0xF0FFFFFF [EOS] mirrored data; unique 0x800000 bytes repeated 0x2 times [EOS] loading './6D/ROM1.BIN' to 0xF8000000-0xF8FFFFFF [EOS] loading './6D/SFDATA.BIN' as serial flash, size=0x800000 [MPU] warning: non-empty spell #2 (Complete WaitID = 0x80000001 Mode group) has duplicate(s): #6 [MPU] warning: non-empty spell #52 (PROP_VIDEO_MODE) has duplicate(s): #53 [MPU] Available keys: - Arrow keys : Navigation - PgUp, PgDn : Sub dial (rear scrollwheel) - [ and ] : Main dial (top scrollwheel) - SPACE : SET - DELETE : guess (press only) - M : MENU (press only) - P : PLAY (press only) - I : INFO/DISP (press only) - Q : guess (press only) - L : LiveView (press only) - Z : Zoom in - Shift : Half-shutter - 0/9 : Mode dial (press only) - V : Movie mode (press only) - B : Open battery door - C : Open card door - F10 : Power down switch - F1 : show this help Setting BOOTDISK flag to 0 quit [MPU] WARNING: forced shutdown. For clean shutdown, please use 'Machine -> Power Down' (or 'system_powerdown' in QEMU monitor.)


It is all on one line so how can grep with with it? Also note that it didn't get very far before it shutdown.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 05, 2018, 10:17:39 PM
Heh, I must have installed ggrep to override the default grep, so that's why the script worked for me. Updated again.

That line looks OK, maybe you need to increase the delay. It's not just one line btw - the quotes are important.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 06, 2018, 02:49:57 AM
Part of the problem I'm having is probably that I'm trying to get the codes for 6D.118 and I might not have the correct CURRENT_TASK and CURRENT_ISR (oh what I'd give for the addresses you're using.) However, on the 1100D.106 it didn't work with the default 5 seconds but check out what happens when I increase the DELAY to 20 seconds:

./find_stubs.sh 1100D
Test run...
K288 ICU Firmware Version 1.0.6 ( 3.7.4 )

                                                                               
1100D/debugmsg.gdb
====================

# ./run_canon_fw.sh 1100D -d debugmsg
# ./run_canon_fw.sh 1100D -d debugmsg -s -S & arm-none-eabi-gdb -x 1100D/debugmsg.gdb

source -v debug-logging.gdb

# To get debugging symbols from Magic Lantern, uncomment one of these:
#symbol-file ../magic-lantern/platform/1100D.106/magiclantern
#symbol-file ../magic-lantern/platform/1100D.106/autoexec
#symbol-file ../magic-lantern/platform/1100D.106/stubs.o

macro define CURRENT_TASK 0x1a2c
macro define CURRENT_ISR  (MEM(0x670) ? MEM(0x674) >> 2 : 0)

# GDB hook is very slow; -d debugmsg is much faster
# ./run_canon_fw.sh will use this address, don't delete it
# b *0xFF06C91C
# DebugMsg_log

b *0xFF06FAFC
task_create_log

# not found
# b *0x...
# assert_log

b *0xFF1E8638
register_interrupt_log

b *0xFF06D708
register_func_log

# not found
# b *0x...
# mpu_send_log

# not found
# b *0x...
# mpu_recv_log

b *0xFF06F414
create_semaphore_log

b *0xFF1E8754
create_msg_queue_log

b *0xFF1EE188
CreateStateObject_log

# 0xFF1CAD94 SIO3_ISR
# 0xFF1CAD04 MREQ_ISR

cont


Interesting that increasing DELAY to 60 sec and it doesn't find any of the stubs.

Now don't kill the messenger but ansi2txt isn't available on the Mac or in Homebrew so I had build it from source. Wouldn't "cat" work as well for striping out the ascii control codes?

    test_run=$( echo "$test_run" | cat )


@t3r4n - don't know how you're doing it because I'm getting nothing on the 750D.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 06, 2018, 08:25:08 AM
@dfort:
well lets do a "watch check":

$ ansi2txt -v
ansi2txt - version 0.2.2, compiled on May  1 2018 at 13:15:18.

$ bash --version
bash --version
GNU bash, version 4.4.19(1)-release (x86_64-apple-darwin16.7.0)

$ ggrep -V
ggrep -V
ggrep (GNU grep) 3.1
Packaged by Homebrew


I noticed that I already have an alias grep=... in my bash_profile so that might be a reason.
If I run the script I need to have a delay greater 30 seconds to get the stubs.

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 06, 2018, 09:48:23 AM
Quote from: dfort on May 06, 2018, 02:49:57 AM
Part of the problem I'm having is probably that I'm trying to get the codes for 6D.118 and I might not have the correct CURRENT_TASK and CURRENT_ISR (oh what I'd give for the addresses you're using.)

I didn't try to find them, only noted the old ones won't work. The script does not use them; the logging backend does. I still have the ones for 1.1.6, so the context info (right column) in the test run is not correct, but the script doesn't look at it.


./find_stubs.sh 6D 118
Test run...
K302 ICU Firmware Version 1.1.8 ( 5.8.8 )
...
b *0x9798
task_create_log
...


These two were covered in the M2 topic (https://www.magiclantern.fm/forum/index.php?topic=15895.msg185228#msg185228), btw.

Quote
Interesting that increasing DELAY to 60 sec and it doesn't find any of the stubs.

Works here on the Mac VM, just very slow. Does it at least print the firmware version?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on May 06, 2018, 01:18:47 PM
Started to rework the script in Python.

After a painful struggle with pexpect (TLDR: broken output, 1998 problems), I've managed to get some clean output from QEMU with subprocess.

Does this work on Mac, or it's back to square one?


#!/usr/bin/env python2
from __future__ import print_function
import os, sys
import subprocess
import time
import re

string_stubs = {
    "DebugMsg"              : [ "startupEntry", "startupEventDispatch", "DisablePowerSave" ],
    "task_create"           : [ "TaskMain", 'Task"', "systemtask", "CmdShell", "EvShel", "HotPlug", "PowerMgr", "PowerMan" ],
    "register_interrupt"    : [ "ICAPCHx", "OC4_14", "SIO3_ISR" ],
    "CreateStateObject"     : [ "DMState", "EMState", "PropState", "SRMState" ],
    "create_semaphore"      : [ "PropSem", "mallocLock", "stdioLock", "dm_lock" ],
    "create_msg_queue"      : [ "MainMessQueue", "QueueForDeviceIn", "SystemTaskMSGQueue" ],
}

string_stubs_followed_by = {
    "register_func"         : ([ "flashwrite", "gpiowrite" ], ["NameService"])
}

def eprint(*args, **kwargs):
    print(*args, file=sys.stderr, **kwargs)

cam = sys.argv[1]
fw = sys.argv[2] if 2 in sys.argv else ""

eprint("Test run...")

cmd = ('./run_canon_fw.sh %s,firmware="%s;boot=0" -d calls,tail '
            '-display none -monitor stdio -serial file:uart.log' % (cam, fw))
eprint(cmd)
qemu = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

# tried pexpect, but ran into lots of problems
# including broken terminal, missing newlines, broken pipe messages
# todo: find a minimal example and file a bug report?
qemu.output = ""

def qexpect(strings, timeout):
    t0 = time.time()
    while time.time() - t0 < timeout:
        output = qemu.stderr.readline()
        qemu.output += output
        if output == '' and qemu.poll() is not None:
            eprint("QEMU not running!")
            break
        if any([str in output for str in strings]):
            eprint(output)
            break
    rc = qemu.poll()
    return any([str in output for str in strings])

if qexpect(string_stubs["task_create"], 60):
    eprint("Task found")
else:
    eprint("Task not found")

# let it run for 5 seconds
qexpect([], 5)

try: print("quit", file=qemu.stdin)
except: pass

q_stdout, q_stderr = qemu.communicate()
qemu.wait()
qemu.output += q_stderr
qemu.lines = qemu.output.split("\n")

with open("find_stubs.log", "w") as log:
    print(qemu.output, file=log)

# extract the called function from a line that looks like this:
# call 0x1234(...)
# call 0x1234 DebugMsg(...)
#   -> 0x5678                  # optional (direct jump)
#    -> 0xFFABCD               # also optional
def extract_call(lines):
    assert len(lines) == 3
    if " -> " in lines[1]:
        jump_line = lines[2] if " -> " in lines[2] else lines[1]
        m = re.search('(?<=-> )(.*?)(?= +at )', jump_line)
        if m:
            return int(m.groups()[0], 16)
    else:
        m = re.search('(?<=call )(.*?)(?=\()', lines[0])
        if m:
            return int(m.groups()[0].split(" ")[0], 16)

# strings       : list of strings to be found
#                 first string has the highest priority
# next_strings  : one of these should be on the next line (optional)
def find_stub_from_strings(strings, next_strings):
    lines = qemu.lines
    for s in strings:
        for i,l in enumerate(lines[:-3]):
            if s in l and "call " in l:
                if next_strings is None or any([ns in lines[i+1] for ns in next_strings]):
                    return extract_call(lines[i:i+3])

eprint("")

stubs_found = {}
for name, strings in string_stubs.iteritems():
    stub = find_stub_from_strings(strings, None)
    if stub:
        stubs_found[name] = stub
        eprint("%8X %s" % (stub, name))
    else:
        eprint("     ???", name)

for name, (strings, next_strings) in string_stubs_followed_by.iteritems():
    stub = find_stub_from_strings(strings, next_strings)
    if stub:
        stubs_found[name] = stub
        eprint("%8X %s" % (stub, name))
    else:
        eprint("     ???", name)


Good luck making this work in Python 3...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on May 06, 2018, 04:39:00 PM
Quote
Does this work on Mac, or it's back to square one?
Well no error and if this output is expected yes

./findstub.py 750D
Test run...
./run_canon_fw.sh 750D,firmware=";boot=0" -d calls,tail -display none -monitor stdio -serial file:uart.log
    call 0xFE3CDF44(fe1f845c "PowerMgr", 20, 400, fe1f82e9)                      at [init:fe1f837d:fe506533]

Task found

    211B CreateStateObject
    1775 register_interrupt
80001FC5 create_semaphore
    1E45 task_create
     ??? DebugMsg
     ??? create_msg_queue
FE445CB9 register_func
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 06, 2018, 06:09:38 PM
Here's what I'm getting:

$ python find_stubs.py 1100D
Test run...
./run_canon_fw.sh 1100D,firmware=";boot=0" -d calls,tail -display none -monitor stdio -serial file:uart.log
    call 0xFF06FAFC(ff1eaa10 "PowerMgr", 20, 400, ff1ea7dc)                      at [init:ff1ea910:ff072fe0]

Task found

FF1EE188 CreateStateObject
FF1E8638 register_interrupt
FF017630 create_semaphore
FF06FAFC task_create
FF06C91C DebugMsg
     ??? create_msg_queue
FF06D708 register_func


Noticed that create_semaphore is different (FF06F414 on the bash script) but the rest matches the values in Reply #313 (https://www.magiclantern.fm/forum/index.php?topic=2864.msg201060#msg201060).

Quote from: t3r4n on May 06, 2018, 08:25:08 AM
well lets do a "watch check":

$ ansi2txt -v
ansi2txt - version 0.2.2, compiled on May  6 2018 at 08:22:16.

$ bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)

$ ggrep -V
ggrep (GNU grep) 3.1
Packaged by Homebrew


The difference might be that you're running bash 4. I had that running for a while but one of the software updates reset it to Apple's default which is version 3. The find_stubs.sh script uses "#!/bin/bash" so it should be using Apple's bash instead of the environmental preference used in the other QEMU scripts -- "#!/usr/bin/env bash"

To test options for ansi2txt I removed it so I had to re-compile it to do our "watch check" -- ansi2txt isn't available for Mac unless you build it from source, right? ansi2txt is used in other ML scripts but not having it installed doesn't seem to be a problem.

Turned out that the reason I wasn't having any success with the 750D was because I don't have a SFDATA.BIN for it. Substituted one from the 700D and got the same results you got. (bash script working too.)

$ python find_stubs.py 750D
Test run...
./run_canon_fw.sh 750D,firmware=";boot=0" -d calls,tail -display none -monitor stdio -serial file:uart.log
    call 0xFE3CDF44(fe1f845c "PowerMgr", 20, 400, fe1f82e9)                      at [init:fe1f837d:fe506533]

Task found

    211B CreateStateObject
    1775 register_interrupt
80001FC5 create_semaphore
    1E45 task_create
     ??? DebugMsg
     ??? create_msg_queue
FE445CB9 register_func


RE: DELAY=60

Quote from: a1ex on May 06, 2018, 09:48:23 AM
Does it at least print the firmware version?

Just tried it again and yes, it prints the firmware version. It is very slow and you're re-coding this in python but thought I'd re-check it anyway and the 60 sec. delay worked this time. Go figure.

RE: CURRENT_TASK and CURRENT_ISR

Quote from: a1ex on May 06, 2018, 09:48:23 AM
These two were covered in the M2 topic (https://www.magiclantern.fm/forum/index.php?topic=15895.msg185228#msg185228), btw.

Yes, I know. I've been going over that part over and over trying to understand it and as far as I can see the values haven't changed but I'm obviously not looking hard enough.

BTW--been going back to that EOSM2 topic many times for reference. Someday maybe I'll be able to get that port working properly.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on May 07, 2018, 09:30:35 PM
Running the lua tests in QEMU is helping with the firmware updates I've been doing so I thought I'd try the lua tests on the EOSM2 but before going there I did a run with the EOSM. Besides some of the issues I posted about the lua test on a "real" EOSM (https://www.magiclantern.fm/forum/index.php?topic=9741.msg201090#msg201090), I've been getting this in QEMU:

(https://farm1.staticflickr.com/911/40151670090_f91877aa1f.jpg) (https://flic.kr/p/24b4NDY)

But I am in M mode! Note that it doesn't matter if I try to switch to M while running the tests or switch to M before running the tests, I get the same message.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on June 11, 2018, 02:38:14 AM
I Got problems again
Quote from: a1ex on September 27, 2017, 09:49:26 PM
Currently experimenting with an updated toolchain, in the qemu branch. I'm following this guide (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/) (actually I'm testing it on a fresh Ubuntu VM):


sudo apt install mercurial
hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg update qemu -C
cd contrib/qemu
./install.sh

following this , i get 2 error . after the "Install.sh" and stops and doesn't install the emulator
W: Failed to fetch http://ppa.launchpad.net/accessibility-dev/ppa/ubuntu/dists/trusty/main/binary-amd64/Packages  403  Forbidden

W: Failed to fetch http://ppa.launchpad.net/accessibility-dev/ppa/ubuntu/dists/trusty/main/binary-i386/Packages  403  Forbidden

i can't even get it manually and install it , still say    403  Forbidden :(
using 64bit VM "Ubuntu Mate 14.04 LTS"  on Win7pro i5 dell laptop though VirtualBox
from here Quick Guide and Solutions for VirtualBox (https://www.magiclantern.fm/forum/index.php?topic=7579.msg134989#msg134989) so is not fully supported ? Do i have to install a new updated version of Ubuntu ?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on June 13, 2018, 06:22:36 AM
Success :D !

(https://image.ibb.co/cLjted/SA95V9_0.png) (https://imgbb.com/)

A word to the wise , make sure your Linux OS is up to date  ::)
That was the problem all a long , I started with the per-configured VM Ubuntu Mate 14.04 LTS(see previous post) and had nothing but trouble with trying to get Qemu up and running.
After a short period of confusion i notice a window to upgrade to the latest version of Ubuntu Mate  :o  16.xx something , took about 2 hours to upgrade then ran the
./install.sh
after that everything when smoothly just like what a1ex posted here (https://www.magiclantern.fm/forum/index.php?topic=2864.msg190596#msg190596) now i hope i can get
the missing parts to 5d2/D4 Lossless compression  8)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on June 15, 2018, 06:57:29 AM
I'm getting just a grey screen on the 5d2 & 5d3 with this
/run_canon_fw.sh 5D3,firmware="boot=0" -s -S & arm-none-eabi-gdb -x 5D3/patches.gdb
for the 5d2 i just drop
-x 5D3/patches.gdb
and of course i changed 5d3 to 5d2
I can get the bootloader display test to come up like in the my previous post ,
with ./run_canon_fw.sh 5D3
is there some switches i need to add ?
or is it a 64bit Linux (Ubuntu 16.04) i though i read something about it or maybe it's dependencies  .

i ran this for the 5d3.113
make -C ../magic-lantern/platform/5D3.113 install_qemu
./run_canon_fw.sh 5D3,firmware="boot=1" -s -S & arm-none-eabi-gdb -x 5D3/patches.gdb


and got this , seem to hang after "mvrChangeAckCBR:Video ..... " see bottom screen shot

(https://preview.ibb.co/eNWCSy/SOLDR7_1.png) (https://ibb.co/g3qiZd)
(https://preview.ibb.co/fHc2Sy/small_SOLDR7_1.png) (https://ibb.co/irwJ0J)

what i'm i missing here ?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 15, 2018, 07:42:41 AM
The GUI should be in the VGA tab; is the serial0 displayed by default?

There's no more patches.gdb for 5D3; this is enough (same for 5D2):

./run_canon_fw.sh 5D3,firmware="boot=0"


If the above doesn't bring the GUI, I'd like to see the full log (with -d debugmsg).

For older Ubuntu, see reply #292 (https://www.magiclantern.fm/forum/index.php?topic=2864.msg200971#msg200971).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on June 16, 2018, 06:57:08 AM
Quote from: a1ex on June 15, 2018, 07:42:41 AM
The GUI should be in the VGA tab; is the serial0 displayed by default?
No it defaults to VGA,  i just had it in serial0 to see what happening
Quote from: a1ex on June 15, 2018, 07:42:41 AM
There's no more patches.gdb for 5D3; this is enough (same for 5D2):

./run_canon_fw.sh 5D3,firmware="boot=0"

still has a grey screen
OK , i ran
./run_canon_fw.sh 5D3,firmware="boot=0" -d debugmsg

i did see any log being saved , it did print out a lot of data like a log in terminal
[    Fstorage:ff1ae44c ] (9e:03) fssRegister (0, 0)
[         Mrk:ff4386bc ] (25:03) CreateObject : Handle = 0xa0082006
[         Mrk:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[         Mrk:ff11a758 ] (27:01) FC_IsThereHandle
[         Mrk:ff298c98 ] (27:01) _FC_LockCatalog (2)
[         Mrk:ff2a934c ] (27:01) _FC_IsThereHandle (2, 2, 512, 6)
[         Mrk:ff2a934c ] (27:01) _FC_IsThereHandle (2, 2, 512, 6)
[         Mrk:ff298cf4 ] (27:01) _FC_UnlockCatalog (2)
[         Mrk:ff4386e4 ] (25:03) CreateObject : File not exist
[         Mrk:ff2d9a54 ] (25:03) PrepareCallback : Success(Handle = 0xa0082006)
[         Mrk:ff2d9a54 ] (25:03) PrepareCallback : Success(Handle = 0xa0082006)
[         Mrk:ff2acae0 ] (27:01) FC_RegisterHandleCallback
[         Mrk:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[         Mrk:ff4386bc ] (25:03) CreateObject : Handle = 0xa0090006
[         Mrk:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[         Mrk:ff11a758 ] (27:01) FC_IsThereHandle
[         Mrk:ff298c98 ] (27:01) _FC_LockCatalog (2)
[         Mrk:ff2a934c ] (27:01) _FC_IsThereHandle (2, 2, 4096, 6)
[         Mrk:ff298cf4 ] (27:01) _FC_UnlockCatalog (2)
[         Mrk:ff4386e4 ] (25:03) CreateObject : File not exist
[         Mrk:ff2d9a54 ] (25:03) PrepareCallback : Success(Handle = 0xa0090006)
[         Mrk:ff2acae0 ] (27:01) FC_RegisterHandleCallback
[         Mrk:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[     MetaCtg:ff42b108 ] (27:03) fcmcOpenMetaCtg Start
[     MetaCtg:ff58a074 ] (27:03) FcmcMusicOpenMetaCtg
[     MetaCtg:ff58ae7c ] (27:03) Noting MUSIC Dir. (B:/DCIM/MUSIC)
[     MetaCtg:ff58ae7c ] (27:03) Noting MUSIC Dir. (B:/DCIM/MUSIC)
[     MetaCtg:ff58a138 ] (27:03) FcmcMusicOpenMetaCtg : Nothing music dir!
[     MetaCtg:ff58a138 ] (27:03) FcmcMusicOpenMetaCtg : Nothing music dir!
[     MetaCtg:ff6b16e0 ] (23:01) sdReadBlk: st=352, num=32, buf=0x402f6000
[     MetaCtg:ff6b16e0 ] (23:01) sdReadBlk: st=352, num=32, buf=0x402f6000
[     MetaCtg:ff6b0258 ] (23:01) sdDMAReadBlk: st=352, num=32
[    MainCtrl:ff0cea60 ] (9c:01) ID:8(19)
[MPU] Sending : 06 05 04 0e 01 00  (PROP 8002000D)
[    MainCtrl:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[    MainCtrl:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[    MainCtrl:ff0cea60 ] (9c:01) ID:8(19)
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[1][4][1]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[MPU] Sending : 08 06 01 04 03 00 00 00  (PROP_AF_MODE)
[    MainCtrl:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[    MainCtrl:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[4][4][0]
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[4][4][0]
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[4][4][0]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff1bbc34 ] (9f:01)     tv 58
[MPU] Sending : 06 05 03 37 00 00  (PROP_MIRROR_DOWN_IN_MOVIE_MODE)
[    MainCtrl:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[    MainCtrl:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[    MainCtrl:ff1bbc34 ] (9f:01)     tv 58
[    MainCtrl:ff1bbc50 ] (9f:01)     range tv a0,10
[    MainCtrl:ff1bbc50 ] (9f:01)     range tv a0,10
[    MainCtrl:ff1bbd90 ] (9f:01)     av 38
[    MainCtrl:ff1bbd90 ] (9f:01)     av 38
[    MainCtrl:ff1bbdac ] (9f:01)     range av 50,20
[MPU] Sending : 06 05 03 35 01 00  (PROP_BATTERY_REPORT_COUNTER)
[    MainCtrl:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[    MainCtrl:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[    MainCtrl:ff1bbdf8 ] (9f:01)     iso 70
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[5][4][0]
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[5][4][0]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff1bbb18 ] (9f:01)     aeb_count 0
[MPU] Sending : 1c 1b 03 1d 3f 00 00 00 00 72 00 4c 50 2d 45 36 00 00 00 00 00 01 00 ae 7e 3b 61 00  (PROP_BATTERY_REPORT)
[    MainCtrl:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[    MainCtrl:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[    MainCtrl:ff1bbb18 ] (9f:01)     aeb_count 0
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[2][4][0]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff1bbf04 ] (9f:01)     mirror_up 0
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[0][8][0]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff1bbf90 ] (9f:01) rmt_olc_com_gr8:0,0,0
[    MainCtrl:ff1bbf90 ] (9f:01) rmt_olc_com_gr8:0,0,0
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[3][4][0]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff1bbfbc ] (9f:01)     focusstatus 0,1
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[0][8][0]
[    MainCtrl:ff105150 ] (33:01) PD_NotifyOlcInfoChanged[0][8][0]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff104798 ] (33:01) SendPipeEvent [0][0][10]
[    MainCtrl:ff1bc13c ] (9f:01)     focusinfo 0,0,1
[    MainCtrl:ff0cc028 ] (89:03) bindReceiveNewTFTOLC
[    MainCtrl:ff0decb0 ] (85:03) GUI_Control:103 0x96fee0
[     PropMgr:ff104bac ] (33:01) ptpPropChangeEvCBR[80000004][2][3]
[     PropMgr:ff104bac ] (33:01) ptpPropChangeEvCBR[80000004][2][3]
[     PropMgr:ff104798 ] (33:01) SendPipeEvent [0][0][9]
[     PropMgr:ff104798 ] (33:01) SendPipeEvent [0][0][49]
[     PropMgr:ff1a4cec ] (83:01) changeCBR PropID(0x80000004)Parameter(3)Size(2)
[    MainCtrl:ff0cea60 ] (9c:01) ID:8000002C(20)
[     PropMgr:ff104798 ] (33:01) SendPipeEvent [0][0][49]
[     PropMgr:ff104798 ] (33:01) SendPipeEvent [0][0][49]
[     PropMgr:ff104798 ] (33:01) SendPipeEvent [0][0][49]
[     PropMgr:ff104798 ] (33:01) SendPipeEvent [0][0][49]
[     PropMgr:ff1a4cec ] (83:01) changeCBR PropID(0x80030034)Parameter(0)Size(4)
[         Gmt:ff170c8c ] (9a:02) gmtProperty ID=0x80000004(0x3)
[         Gmt:ff177a34 ] (9a:03) [WAKU] gmtChangeLensMFFace (state:0)
[         Gmt:ff177ab0 ] (9a:00) [WAKU] Center : gmtChangeLensMFFace (746 481)
[         Gmt:ff175770 ] (9a:03) [WAKU] gmtUpdateFrameWithModeChange (state:0 mani:22)
[         Gmt:ff175544 ] (9a:00) [WAKU] UpdateImageMagnify M:0 Z:65536 S:0 A:0
[         Gmt:ff1755dc ] (9a:00) [WAKU] Center : UpdateImageMagnify (746 481)
[         Gmt:ff175370 ] (9a:02) [WAKU] ZoomRatio:00010000 LvMode:0
[         Gmt:ff175394 ] (9a:02) [WAKU] Avail ImgW:H(5760:3840)->(5760:3840)
[         Gmt:ff1753b8 ] (9a:02) [WAKU] Avail WinW:H(1152:768)->(1152:768)
[         Gmt:ff1753dc ] (9a:02) [WAKU] Avail X:Y(0:0)->(0:0)
[         Gmt:ff173018 ] (9a:03) gmtChangeAvailableArea state:1
[         Gmt:ff0ddbec ] (9a:01) Event 23 Result State 1->1
[         Gmt:ff174f80 ] (9a:01) [WAKU] Event 8 Result State 0->6
[         Gmt:ff0ddbec ] (9a:01) Event 19 Result State 1->1 ID 0x80000004(3)
[     PropMgr:ff0f78bc ] (8f:02) dcsChangeAckCBR (0x80050007, 0x1680)
[     PropMgr:ff270ea0 ] (33:01) PropCBR camctrl:0x80050007,0x1680
[     PropMgr:ff1a4cec ] (83:01) changeCBR PropID(0x80050007)Parameter(5760)Size(124)
[ GuiMainTask:ff0df028 ] (84:01) GUI_CONTROL:103
[ GuiMainTask:ff347730 ] (84:03) copyOlcDataToStorage Length(66)group1(0xff)group2(0x1f)
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data02[0x01]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data03[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data04[0x01]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data05[0x01]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data06[0xa0]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data07[0x10]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data08[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data09[0x58]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data10[0x01]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data11[0x01]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data12[0x50]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data13[0x20]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data14[0x38]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data15[0x01]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data16[0x01]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data17[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data18[0x70]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data19[0x04]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data20[0x01]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data21[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data22[0xe3]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data23[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data24[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data25[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data26[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data27[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data28[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data29[0x01]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data30[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data31[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data32[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data33[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data34[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data35[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data36[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data37[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data38[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data39[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data40[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data41[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data42[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data43[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data44[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data45[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data46[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data47[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data48[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data49[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data50[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data51[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data52[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data53[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data54[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data55[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data56[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data57[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data58[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data59[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data60[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data61[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data62[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data63[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data64[0x00]
[ GuiMainTask:ff347758 ] (84:01) copyOlcDataToStorage data65[0x00]
[ GuiMainTask:ff1a46dc ] (83:03) GUI_SetLvMirrorDownStatus status=0 prestatus=0
[ GuiMainTask:ff1a7100 ] (84:01) gui control end
[ GuiMainTask:ff1a7124 ] (84:01) 0msec = 660 - 660
[ GuiMainTask:ff1a7140 ] (84:01) 0msec = 826624 - 826624
[     CtrlSrv:ff189d20 ] (84:01) copyDataToStorage eventID(0x80000004)Data(3)size(0)
[     CtrlSrv:ff354fec ] (83:03) MainEventHndler PROP_MIRROR_DOWN_IN_MOVIE_MODE(0)fUI_OK(1)
[     CtrlSrv:ff189d20 ] (84:01) copyDataToStorage eventID(0x80030034)Data(0)size(0)
[     CtrlSrv:ff18d594 ] (83:03) PROP_MIRROR_DOWN_IN_MOVIE_MODE [0]
[     CtrlSrv:ff189d20 ] (84:01) copyDataToStorage eventID(0x80050007)Data(9897288)size(124)
[     CtrlSrv:ff189d20 ] (84:01) copyDataToStorage eventID(0x80050007)Data(9897288)size(124)
[     CtrlSrv:ff189d20 ] (84:01) copyDataToStorage eventID(0x80050007)Data(9897288)size(124)
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 10, Param = 0
[      PtpDps:ff261158 ] (33:01) ptpPropertyOlcInfoChangeEvent[1][4][1]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[MPU] Sending : 06 04 03 36 00 00  (PROP_BATTERY_REPORT_FINISHED)
[      PtpDps:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[      PtpDps:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[      PtpDps:ff26c678 ] (33:01) ptpNotifyOlcInfoChanged:1,4,1
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 10, Param = 0
[      PtpDps:ff261158 ] (33:01) ptpPropertyOlcInfoChangeEvent[4][4][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff26c678 ] (33:01) ptpNotifyOlcInfoChanged:0,4,1
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff26b684 ] (33:01) ptpGetPropEvent FAILED! [d1c3][0]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 10, Param = 0
[      PtpDps:ff261158 ] (33:01) ptpPropertyOlcInfoChangeEvent[5][4][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff26c678 ] (33:01) ptpNotifyOlcInfoChanged:0,4,1
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff26b684 ] (33:01) ptpGetPropEvent FAILED! [d1c7][0]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 10, Param = 0
[      PtpDps:ff261158 ] (33:01) ptpPropertyOlcInfoChangeEvent[2][4][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff26c678 ] (33:01) ptpNotifyOlcInfoChanged:0,4,1
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff26b684 ] (33:01) ptpGetPropEvent FAILED! [d1bf][0]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 10, Param = 0
[      PtpDps:ff261158 ] (33:01) ptpPropertyOlcInfoChangeEvent[0][8][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff26c678 ] (33:01) ptpNotifyOlcInfoChanged:0,8,1
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff26b684 ] (33:01) ptpGetPropEvent FAILED! [d1d3][0]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 10, Param = 0
[      PtpDps:ff261158 ] (33:01) ptpPropertyOlcInfoChangeEvent[3][4][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff26c678 ] (33:01) ptpNotifyOlcInfoChanged:0,4,1
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff26b684 ] (33:01) ptpGetPropEvent FAILED! [d1c0][0]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 10, Param = 0
[      PtpDps:ff261158 ] (33:01) ptpPropertyOlcInfoChangeEvent[0][8][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff26c678 ] (33:01) ptpNotifyOlcInfoChanged:0,8,1
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff26b684 ] (33:01) ptpGetPropEvent FAILED! [d1d3][0]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 9, Param = 0
[      PtpDps:ff260f40 ] (33:01) ptpPropertyChangeEvent[80000004][2][3][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff26c88c ] (33:01) PROP:0x80000004,3
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff26b684 ] (33:01) ptpGetPropEvent FAILED! [d1d3][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff26b684 ] (33:01) ptpGetPropEvent FAILED! [d108][0]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 49, Param = 0
[      PtpDps:ff1ba75c ] (9f:03) rmtChangePtpParam [80000004][516df4][2]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 49, Param = 0
[      PtpDps:ff1ba75c ] (9f:03) rmtChangePtpParam [8000002c][65ef64][4]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 49, Param = 0
[      PtpDps:ff1ba75c ] (9f:03) rmtChangePtpParam [80000035][749db0][4]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 49, Param = 0
[      PtpDps:ff1ba75c ] (9f:03) rmtChangePtpParam [8000002d][749cb0][4]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 49, Param = 0
[      PtpDps:ff1ba75c ] (9f:03) rmtChangePtpParam [80000036][749e30][4]
[     MetaCtg:ff6b0258 ] (23:01) sdDMAReadBlk: st=352, num=32
   124:   583.168 [IMPP] H264E InitializeH264EncodeFor1080pDZoom
   125:   583.424 [IMPP] H264E InitializeH264EncodeFor1080p25fpsDZoom
   129:   613.376 [GUI] MainEventHandler PROP_QR_DIDNOT_EXECUTE(0)(0)
   132:   644.096 [MR_MOV] (Empty Func) MVW_RegisterXmpDataCallback
   133:   648.192 [HDMI] HDMI CEC Initialize
   134:   652.544 [MR] mvrChangeAckCBR : Video - Mode=0, Type=0, Rate=24, GOP=1
   135:   656.128 [HDMI] HPD OFF
   136:   656.384 [HDMI] [EDID] dwVideoCode = 0
   137:   656.384 [HDMI] [EDID] dwHsize = 0
   138:   656.640 [HDMI] [EDID] dwVsize = 0
   139:   657.408 [HDMI] [EDID] ScaningMode = EDID_NON_INTERLACE(p)
   140:   657.664 [HDMI] [EDID] VerticalFreq = EDID_FREQ_60Hz
   141:   657.920 [HDMI] [EDID] AspectRatio = EDID_ASPECT_4x3
   142:   657.920 [HDMI] [EDID] AudioMode = 0
   143:   658.176 [HDMI] [EDID] ColorMode = EDID_COLOR_RGB
   144:   660.480 [MR] mvrChangeAckCBR : Video - Mode=0, Type=0, Rate=24, GOP=1
   145:   663.296 [DISP] BrightnessControl from GUI 0
   146:   665.600 [HDMI] WARN [DISCON]HDMI is Already FALSE!!
   148:   668.928 [STARTUP] startupInitializeComplete
   149:   668.928 [MC] cam event guimode comp. 0
   150:   669.440 [MC] cam event guimode comp. 0
[     PropMgr:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[     PropMgr:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[     PropMgr:ff27c5c8 ] (36:01) ceresPropertyChangeCBR[80030019]
[     PropMgr:ff27c5e4 ] (00:01)  ceresPropertyChangeCBR ID[0x8003001d] Size[0x60]
[     PropMgr:ff27c664 ] (36:01) PROP_BATTERY_REPORT:1
[     PropMgr:ff27cdb4 ] (36:01) 1,0,3f,0
[     PropMgr:ff1a4cec ] (83:01) changeCBR PropID(0x8003001d)Parameter(1)Size(96)
[     CtrlSrv:ff3544fc ] (83:03) GuiMainEventHandler.c PROP_BATTERY_REPORT size<96>
[     CtrlSrv:ff19dab4 ] (83:03) GUI_GetNumOfBatteriesHistory: num<0>
[     CtrlSrv:ff189d20 ] (84:01) copyDataToStorage eventID(0x8003001d)Data(9895648)size(96)
[     CtrlSrv:ff18cf0c ] (83:03) PROP_BATTERY_REPORT copysize (96)<-(96)
[MPU] Sending : 06 05 03 16 00 00  (PROP_BATTERY_CHECK)
[    PowerMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[    PowerMgr:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[MPU] Sending : 06 05 08 06 00 00  (COM_FA_CHECK_FROM)
[     PropMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[     PropMgr:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[     PropMgr:ff104bac ] (33:01) ptpPropChangeEvCBR[80030013][4][0]
[     PropMgr:ff104bac ] (33:01) ptpPropChangeEvCBR[80030013][4][0]
[     PropMgr:ff104798 ] (33:01) SendPipeEvent [0][0][9]
[     PropMgr:ff1bd644 ] (90:05) MpuMonSpecificFromPartner : COM_FA_CHECK_FROM 0
[     PropMgr:ff1a4cec ] (83:01) changeCBR PropID(0x80030013)Parameter(0)Size(4)
[     PropMgr:ff1a4cec ] (83:01) changeCBR PropID(0x80030013)Parameter(0)Size(4)
[     CtrlSrv:ff4a44d0 ] (83:03) DlgActiveSweepExcute.c PROP_BATTERY_CHECK = OK
[     CtrlSrv:ff195548 ] (83:03) GUI_SetActiveSweepStatus (1)
[     PropMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[     PropMgr:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[     PropMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[     PropMgr:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[     CtrlSrv:ff189d20 ] (84:01) copyDataToStorage eventID(0x80030013)Data(0)size(0)
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 9, Param = 0
[      PtpDps:ff260f40 ] (33:01) ptpPropertyChangeEvent[80030013][4][0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff26c88c ] (33:01) PROP:0x80030013,0
[MPU] Received: 06 05 04 0d 01 00  (unknown - PROP_ACTIVE_SWEEP_STATUS)
[     MetaCtg:ff58a388 ] (27:03) FcmcMusicOpenMetaCtg : (ClusterSize:16384, Used:0)
[     MetaCtg:ff116904 ] (27:01) FC_GetRootObject
[     MetaCtg:ff298c98 ] (27:01) _FC_LockCatalog (2)
[     MetaCtg:ff2af724 ] (27:03) _FC_GetRootObject (2, 0, 0, 0)
[     MetaCtg:ff298cf4 ] (27:01) _FC_UnlockCatalog (2)
[     MetaCtg:ff116dec ] (27:01) FC_GetFirstDirHandle
[     MetaCtg:ff298c98 ] (27:01) _FC_LockCatalog (2)
[     MetaCtg:ff29671c ] (27:03) _FC_GetFirstDirHandle (2, 0, 0, 0)
[     MetaCtg:ff298cf4 ] (27:01) _FC_UnlockCatalog (2)
[     MetaCtg:ff42b490 ] (27:01) fcmcOpenMetaCtg : RemoveCtgFile start
[     MetaCtg:ff42b4f0 ] (27:01) fcmcOpenMetaCtg : RemoveCtgFile end
[     MetaCtg:ff42b52c ] (27:03) fcmcOpenMetaCtg : Dir(100)
[     MetaCtg:ff42b57c ] (27:03) fcmcOpenMetaCtg : Dir(100) is Empty
[     MetaCtg:ff116ea8 ] (27:01) FC_GetNextDirHandle
[     MetaCtg:ff298c98 ] (27:01) _FC_LockCatalog (2)
[     MetaCtg:ff296880 ] (27:03) _FC_GetNextDirHandle (2, 100, 0, 0)
[     MetaCtg:ff298cf4 ] (27:01) _FC_UnlockCatalog (2)
[     MetaCtg:ff42b664 ] (27:03) ##### Init Check Finish [OK](2)
[     MetaCtg:ff42b6d8 ] (27:03) fcmcOpenMetaCtg : (ClusterSize:16384, UsedCluster:0)
[     MetaCtg:ff42ba18 ] (27:03) fcmcOpenCompleteMetaCtg
[     MetaCtg:ff104abc ] (33:03) ptpDpsCtginfoCheckCompleteCBR[42][0][2]
[     MetaCtg:ff104798 ] (33:01) SendPipeEvent [0][0][67]
[      PtpDps:ff1044e0 ] (33:01) Dispatch : Cur = 0, Event = 67, Param = 0
[      PtpDps:ff26f8e8 ] (33:01) PtpPropGetStorageID 2 0x20001
[      PtpDps:ff263564 ] (33:03) ptpEventCtginfoCheckComplete
[      PtpDps:ff257dc4 ] (33:03) DSEvent:0xc1a4,0x20001
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[      PtpDps:ff2519d8 ] (32:01) GetConnectSessionFirst end[0][3]
[      PtpDps:ff251bcc ] (32:01) GetConnectSessionHandle 1 err[0][0]
[     MetaCtg:ff352cac ] (85:03) GuiMainEventhandler.c pNotifyCatalogCreateCBR err = 0, DriveNo = 2
[     CtrlSrv:ff35b404 ] (83:03) IDLEHandler LOCAL_COMP_CATALOG_CREATE(2)
[     CtrlSrv:ff188648 ] (83:03) guiSetCatalogCreateBusy(0)
[ GuiMainTask:ff0df23c ] (84:01) GUI_OTHEREVENT:11
[     MetaCtg:ff42ba90 ] (27:01) FreeUncacheableMemory(TempCache)
[     MetaCtg:ff42bab8 ] (27:01) DeleteMemorySuite(TempParse)
[     MetaCtg:ff42bae0 ] (27:01) FreeUncacheableMemory(TempParse)
[     MetaCtg:ff42bb08 ] (27:01) FreeUncacheableMemory(OldVerCache)
[    PowerMgr:ff144dd4 ] (9c:03) (delayed) 1
[    MainCtrl:ff0cea60 ] (9c:01) ID:FFFFFFFF(21)
Key event: 44 -> 1100
Key event: c4 -> 0e0e003f
[MPU] Sending : 06 05 06 11 00 00  (GUI_SWITCH)
[    PowerMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[    PowerMgr:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[    MainCtrl:ff0cea60 ] (9c:01) ID:11(22)
[    MainCtrl:ff144efc ] (9c:16) REQ : MultiShotTerminate !!!
[MPU] Sending : 06 05 02 0b 00 00  (PROP_TERMINATE_SHUT_REQ)
[    MainCtrl:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[    MainCtrl:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 0)
[    MainCtrl:ff144efc ] (9c:16) REQ : MultiShotTerminate !!!
[    MainCtrl:ff1451f8 ] (9c:03) notice Lock 0
[    MainCtrl:ff0cc1cc ] (89:03) bindReceiveSwitch (17, 0)
[    MainCtrl:ff0cc608 ] (89:03) LOCK (0)
[    MainCtrl:ff0cc608 ] (89:03) LOCK (0)
[    MainCtrl:ff0cc638 ] (89:03) LOCK (JOB = 0, GUI = 0)
[    MainCtrl:ff0decb0 ] (85:03) GUI_Control:95 0x2
[    MainCtrl:ff0decb0 ] (85:03) GUI_Control:89 0x0
[     PropMgr:ff218af4 ] (81:01) CustomSlaveCBR 0x80010001(0 0 1) 0x0
[     PropMgr:ff128694 ] (02:03) Compare FROMAddress (0) 0x40510e00 0xff060000 Size 2424
[     PropMgr:ff0cdbd4 ] (8c:03) terminateChangeCBR : SHUTDOWN (0)
[     PropMgr:ff0cdda0 ] (8c:16) SHUTDOWN_REQUEST
[     PropMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[     PropMgr:ff0f93c8 ] (00:03) [SEQ] CreateSequencer (Terminate, Num = 3)
[     PropMgr:ff0dec38 ] (85:03) GUI_Suspend(1)
[    MainCtrl:ff0cea60 ] (9c:01) ID:80010001(23)
[    MainCtrl:ff0dfd30 ] (00:04) < GUI Lock > GUILock_ProhibitLock (PUB)
[    MainCtrl:ff130d98 ] (82:02) DispConEmergencyStart
[    MainCtrl:ff0dece0 ] (85:03) QueueClearFlg Clear
[    MainCtrl:ff2ee89c ] (05:03) CtrlSrvCancelAllEvent NumOfCancelQueueRequest=1
[    MainCtrl:ff216830 ] (19:03) [ImgPlyDr] LockImagePlayDiriver
[    MainCtrl:ff216834 ] (19:03) [ImgPlyDr]  CancelCountUp 0x10001
[    MainCtrl:ff2166ec ] (19:03) [ImgPlyDr] CancelImagePlay PathState : 0
[    MainCtrl:ff2166f8 ] (19:03) [ImgPlyDr]  CancelCountUp 0x10002
[    MainCtrl:ff0cea60 ] (9c:01) ID:0(24)
[  ImgPlayDrv:ff215f18 ] (19:01) [ImgPlyDr]    ID_CANCEL
[  ImgPlayDrv:ff215f1c ] (19:03) [ImgPlyDr]  CancelCountDown 0x10001
[     PropMgr:ff0cd08c ] (89:16) ShutDownReq
[    MainCtrl:ff0cea60 ] (9c:01) ID:80050021(25)
[    MainCtrl:ff146dc8 ] (9c:03) PROP_LV_LOCK : LVLOCK_PROHIBIT
[         Gmt:ff170c8c ] (9a:02) gmtProperty ID=0x80050021(0)
[         Gmt:ff1707f4 ] (9a:03) gmtIgnore (state:1)
[         Gmt:ff0ddbec ] (9a:01) Event 4 Result State 1->1
[         Gmt:ff0ddbec ] (9a:01) Event 19 Result State 1->1 ID 0x80050021(0)
[ GuiMainTask:ff0df028 ] (84:01) GUI_CONTROL:95
[ GuiMainTask:ff1a64bc ] (84:03) GUICMD_START_AS_CHECK
[ GuiMainTask:ff1a7100 ] (84:01) gui control end
[ GuiMainTask:ff1a7124 ] (84:01) 0msec = 58960 - 58960
[ GuiMainTask:ff1a7140 ] (84:01) 0msec = 930560 - 930560
[ GuiMainTask:ff0df028 ] (84:01) GUI_CONTROL:89
[ GuiMainTask:ff1a64bc ] (84:03) GUICMD_LOCK_OFF
[ GuiMainTask:ff1a7100 ] (84:01) gui control end
[ GuiMainTask:ff1a7124 ] (84:01) 0msec = 58960 - 58960
[ GuiMainTask:ff1a7140 ] (84:01) 512msec = 930816 - 931328
[ GuiMainTask:ff0df23c ] (84:01) GUI_OTHEREVENT:21
[ GuiMainTask:ff0df1e4 ] (84:01) GUIOTHER_CANCEL_ALL_EVENT
[ GuiLockTask:ff0df854 ] (00:03) < GUI Lock > GUILockTask 0
[ GuiLockTask:ff133310 ] (82:03) [ME] GuiLockTask (0) [TurnOffDisplay] (OFF)
[  DisplayMgr:ff12f824 ] (82:02) ignore >>> TURNOFF
[  DisplayMgr:ff133358 ] (82:02) GUILockWaitCBR pParam=0
[     PropMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 2)
[     PropMgr:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 1)
[     PropMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 2)
[     PropMgr:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 1)
[     PropMgr:ff1a4cec ] (83:01) changeCBR PropID(0x80030015)Parameter(1)Size(4)
[     CtrlSrv:ff2ee268 ] (05:03) Queue Clear (2054) NumOfCancelQueueRequest=0
[     CtrlSrv:ff358090 ] (83:03) IDLEHandler START_AS_CHECK[2]
[     CtrlSrv:ff357c70 ] (83:03) IDLEHandler GUI_LOCK_OFF
[     CtrlSrv:ff35b748 ] (83:03) IDLEHandler OTHER_SUSPEND bLockOff(1)
[     CtrlSrv:ff0e0288 ] (00:04) < GUI Lock > GUILock_ShutDownSyncTurnOffDisplay (PUB)
[ GuiLockTask:ff0df854 ] (00:03) < GUI Lock > GUILockTask 3
[ GuiLockTask:ff133310 ] (82:03) [ME] GuiLockTask (0) [TurnOffDisplay] (OFF)
[  DisplayMgr:ff12f824 ] (82:02) ignore >>> TURNOFF
[  DisplayMgr:ff133358 ] (82:02) GUILockWaitCBR pParam=0
[     CtrlSrv:ff35b768 ] (83:03) IDLEHandler OTHER_SUSPEND RecordingState(0)
[     CtrlSrv:ff0f961c ] (00:02) [SEQ] NotifyComplete (Terminate, Flag = 0x200000)
[     CtrlSrv:ff0f9680 ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x200000, Flag = 0x200000)
[   Terminate:ff0f92dc ] (00:05) [SEQ] seqEventDispatch (Terminate, 0)
[   Terminate:ff0ce1c0 ] (8c:03) terminateShutReq (0x200000)
[   Terminate:ff10a298 ] (36:03)  ShutdownRequestAdapterControl
[   Terminate:ff0f961c ] (00:02) [SEQ] NotifyComplete (Terminate, Flag = 0x100)
[   Terminate:ff0f9680 ] (00:03) [SEQ] NotifyComplete (Cur = 1, 0x100, Flag = 0x100)
[   Terminate:ff0f92dc ] (00:05) [SEQ] seqEventDispatch (Terminate, 1)
[   Terminate:ff0ce1c0 ] (8c:03) terminateShutReq (0x100)
[   Terminate:ff2da6b0 ] (25:05) MRK_Terminate
[     CtrlSrv:ff189d20 ] (84:01) copyDataToStorage eventID(0x80030015)Data(1)size(0)
[     CtrlSrv:ff18e5e4 ] (83:01) PropertyData PROP_TFT_STATUS[0x1]
[         Mrk:ff0f961c ] (00:02) [SEQ] NotifyComplete (Terminate, Flag = 0x200)
[         Mrk:ff0f961c ] (00:02) [SEQ] NotifyComplete (Terminate, Flag = 0x200)
[MPU] Received: 06 05 03 19 01 00  (PROP_TFT_STATUS - spell #52)
[         Mrk:ff0f961c ] (00:02) [SEQ] NotifyComplete (Terminate, Flag = 0x200)
[         Mrk:ff0f9680 ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x200, Flag = 0x200)
[   Terminate:ff0f92dc ] (00:05) [SEQ] seqEventDispatch (Terminate, 2)
[   Terminate:ff0ce1c0 ] (8c:03) terminateShutReq (0x200)
[     PropMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 2)
[     PropMgr:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 1)
[     PropMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 2)
[     PropMgr:ff13b918 ] (00:01) [PM] EnablePowerSave (Counter = 1)
[   Terminate:ff0f9384 ] (00:05) [SEQ] seqEventDispatch (Terminate) : End
[         Mrk:ff43a458 ] (25:03) Terminate : Success
[       TOMgr:ff13b8a8 ] (00:01) [PM] DisablePowerSave (Counter = 2)
[       TOMgr:ff408cf4 ] (43:03)  tomCancelOrder (2, Dir = 0, File = 0, Type = 0x3e)
[       TOMgr:ff406020 ] (3b:01)  tomGetTOFEntryList (ID = 2)
[       TOMgr:ff406070 ] (3b:01)  tomGetTOFEntryList : Cache Not Hit (ID = 2)
[       TOMgr:ff4057cc ] (3b:01)  tomRefillCache (ID = 2)
[       TOMgr:ff541e0c ] (3c:01)  tomGetTOFInfo (ID = 2)
[MPU] Received: 06 05 02 0b 02 00  (PROP_TERMINATE_SHUT_REQ - spell #122)
[MPU] Shutdown requested.


Quote from: a1ex on June 15, 2018, 07:42:41 AM
For older Ubuntu, see reply #292 (https://www.magiclantern.fm/forum/index.php?topic=2864.msg200971#msg200971).
So what is the latest version i should using ?(i'm on 16.04) Or should i use a different version of Linux (Not Ubuntu i mean) ?



Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 16, 2018, 09:00:55 AM
Pretty sure the log doesn't start with "fssRegister"; there must be some more stuff on the terminal. Best guess, from the line with "BrightnessControl from GUI 0": you had automatic LCD brightness enabled in Canon menu; this part of the code is not emulated well. All my working ROMs have LCD brightness on Manual. I've got one ROM with automatic LCD brightness set, but that one does open Canon menu. I've got another non-working ROM with Control over HDMI enabled, but that one loops in some I2C-related loop.

If it doesn't work with LCD brightness set to Manual, please send me a copy of your ROMs (there might be some other Canon setting that's not emulated well).

You can use any Ubuntu version you wish, including the older one. Other Linux versions should also work.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on June 17, 2018, 03:13:55 PM
Hi guys,
didn't have much time lately. But nonetheless I got some PNs here regarding the use of radare2 for debugging. I thought I might share some of the things I use and some ideas I have for it on the qemu side and maybe someone joins in on it.
So first of all why radare2 its not the standard used by the other guys here. Well frankly IDA pro with ARM debugging cost more than a Camera which has features of MagicLantern available and on gdb I was never able to get a decent gui running and its missing some features I like its free and there are lots of articles on reversing with it. It can do ARM and ARM Thumb and has scripting (python, java ruby ...) interface. It has even an build in emulation which is quite capable of running emulating stuff without qemu.
So lets start.
First you can connect radare2 to QEMU like you would with gdb (-s -S or in the nc qemu.monitor and gdbserver)
The start r2 with the following:
r2 -i load_db.r2 -d gdb://localhost:1234
so what is load_db.r2? It is a file giving lots of commands for a good setup here are some of the things I put in it:

## Pretty stuff
# Solarized theme
eco solarized
# Use UTF-8 to show cool arrows
e scr.utf8 = true
e scr.utf8.curvy=true
# Show comments at right of disassembly if they fit in screen
e asm.cmtright=true
## Processor stuff
# set arch and cpu type
e io.va = true
e asm.arch = arm
e asm.bits = 16
e asm.cpu=cortex
# anal.armthumb (aae computes arm/thumb changes (lot of false positives ahead))
e anal.armthumb=true
# Shows pseudocode in disassembly. Eg mov eax, str.ok = > eax = str.ok
e asm.pseudo = true
# (Show ESIL instead of mnemonic)
# e asm.esil = true
# Selected: asm.describe (Show opcode description)
e asm.describe = false
#asm.emu (Run ESIL emulation analysis on disasm)
e asm.emu = true
e asm.section.sub = true
e io.va=true

that was quite generic and the comments should tell ya whats happening. The following is camera specific it sets up memory regions and gives names to these regions.

S 0x00000000 0x00000000 0x00003fff tcmcode mrwx #00000000 - 00003FFF: eos.tcm_code
S 0x00004000 0x00004000 0x1FFFC000 eosram mrw- #00004000 - 1FFFFFFF: eos.ram
S 0x40000000 0x40000000 0x00004000 eosramuncached0 mrw- #40000000 - 40003FFF: eos.ram_uncached0
S 0x40004000 0x40004000 0x1FFFC000 eosramuncached mrw- #40004000 - 5FFFFFFF: eos.ram_uncached
S 0x80000000 0x80000000 0x00010000 tcmram mrw- #80000000 - 8000FFFF: eos.tcm_data
S 0xBFE00000 0xBFE00000 0x00200000 eosramextra mrw- #BFE00000 - BFFFFFFF: eos.ram_extra
S 0xc0000000 0xc0000000 0x20000000 eosiomem mrw- #C0000000 - DFFFFFFF: eos.iomem
S 0xfc000000 0xfc000000 0x20000000 eosrom1 mr-x #FC000000 - FDFFFFFF: eos.rom1
S 0xfe000000 0xfe000000 0x20000000 eosrom1m mr-x#FE000000 - FFFFFFFF: eos.rom1_mirror

the next lines will setup analysis and define some flags in memory taken from debugmsgs.gdb. r2 uses flags for everything and if I understand documentation right functions follow a fcn.<name> scheme. I have so far not been able to use the afn command to create functions but more later. It would be possible to define these flags as breakpoints and put a modified version of a1ex script for indentifiying functions here. Speaking of which.
I used the script of a1ex to create as described above in this thread an .idc file.
Here I modified the header as follows:

#include "stubshelper.h"

int  main(void)
{
  MakeAutoNamedFunc(0xFE0FD5C9, "LoadScript");


and another file stubhelper.h 

#include <stdio.h>


void MakeAutoNamedFunc(unsigned int ea ,char name[])
{
  printf("f %s = 0x%0X\n",name,ea);
}

void NSTUB(unsigned int ea ,char name[])
{
  printf("f %s = 0x%0X\n",name,ea);
}

compile and pipe the output to your load_db.r2
inside radare2 you can now use

af @@@f
s fcn.<name>
Vpp
to inspect a function.

Ideas:
- Radare provides a scripting interface. Use python script to search through memory for e.g. Frambuffer
- define Names for IO areas to have them marked in the assembly
- is it possible to use the signaturez function of radare to help speedup new firmware ports or new ports.
- ...

Questions:
- anyone got a better idea on how to define functions

Further reading:
- i found this Video of a talk very helpful where the inventor of radare describes how to use it to reverse an ARM based radio: http://radare.org/r/talks.html (http://radare.org/r/talks.html) the talk in 2017. I have not yet looked into the possibility of emulation of io devices via the scripted breakpoints ...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 17, 2018, 04:07:42 PM
Very cool, maybe you should consider a sticky topic in the Reverse Engineering area. The memory map (and possibly an initial r2 script) should be autogenerated from QEMU, in a way similar to the IDCs.

One quick note:
Quote from: t3r4n on June 17, 2018, 03:13:55 PM
(aae computes arm/thumb changes (lot of false positives ahead))

Same with IDA. That's why the autogenerated .idc script (the one from -d idc, without function names) specifies whether a function is ARM or Thumb, based on how it was actually executed during emulation.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on June 19, 2018, 06:27:10 AM
Something weird is going on with QEMU , tried a new saved rom dump (5D2) (i made sure the lcd screen was set to manual)
./run_canon_fw.sh 5D2

(https://preview.ibb.co/fb7Uxy/SOMYLG_Y.png) (https://ibb.co/mHMYPd)

says in the top left connor "QEMU(5d2)" but try and read it as 5d3.113  ???

./run_canon_fw.sh 5D2
DebugMsg=0xFF86AF64 (from GDB script)
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
E8000000 - E8052FFF: eos.ram_extra
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - CFFFFFFF: eos.mmio
[EOS] loading './5D2/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] mirrored data; unique 0x400000 bytes repeated 0x4 times
[EOS] loading './5D2/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[EOS] mirrored data; unique 0x800000 bytes repeated 0x2 times
[MPU] warning: non-empty spell #5 (PROP_CARD3_STATUS) has duplicate(s): #16
[MPU] warning: non-empty spell #17 (PROP_CARD1_STATUS) has duplicate(s): #48
[MPU] warning: non-empty spell #31 (PROP 8003001A) has duplicate(s): #36
[MPU] warning: non-empty spell #37 (PROP_VIDEO_MODE) has duplicate(s): #38

[MPU] Available keys:
- Arrow keys   : Navigation
- Numpad keys  : Joystick (8 directions)
- Numpad 5     : Joystick center
- PgUp, PgDn   : Sub dial (rear scrollwheel)
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess (press only)
- M            : MENU (press only)
- P            : PLAY (press only)
- I            : INFO/DISP (press only)
- L            : LiveView (press only)
- W            : Pic.Style (press only)
- Z/X          : Zoom in/out
- Shift        : Half-shutter
- 0/9          : Mode dial (press only)
- V            : Movie mode (press only)
- B            : Open battery door
- C            : Open card door
- F10          : Power down switch
- F1           : show this help

[DMA3] Copy [0xF8760000] -> [0xE8000000], length [0x00053000], flags [0x00000001]
[DMA3] OK
FFFF2368: MCR p15,0,Rd,cr6,cr0,0:  946_PRBS0 <- 0x3F       (00000000 - FFFFFFFF, 0x100000000)
FFFF2370: MCR p15,0,Rd,cr6,cr1,0:  946_PRBS1 <- 0x3D       (00000000 - 7FFFFFFF, 0x80000000)
FFFF2378: MCR p15,0,Rd,cr6,cr2,0:  946_PRBS2 <- 0xE0000039 (E0000000 - FFFFFFFF, 0x20000000)
FFFF2380: MCR p15,0,Rd,cr6,cr3,0:  946_PRBS3 <- 0xC0000039 (C0000000 - DFFFFFFF, 0x20000000)
FFFF2388: MCR p15,0,Rd,cr6,cr4,0:  946_PRBS4 <- 0xFF80002D (FF800000 - FFFFFFFF, 0x800000)
FFFF2390: MCR p15,0,Rd,cr6,cr5,0:  946_PRBS5 <- 0x39       (00000000 - 1FFFFFFF, 0x20000000)
FFFF2398: MCR p15,0,Rd,cr6,cr6,0:  946_PRBS6 <- 0xF780002D (F7800000 - F7FFFFFF, 0x800000)
FFFF23A0: MCR p15,0,Rd,cr2,cr0,0: DCACHE_CFG <- 0x70       
FFFF23A8: MCR p15,0,Rd,cr3,cr0,0:       DACR <- 0x70       
FFFF23AC: MCR p15,0,Rd,cr2,cr0,1: ICACHE_CFG <- 0x70       
FFFF23B0: MCR p15,0,Rd,cr5,cr0,0:    DATA_AP <- 0x3FFF     
FFFF23B8: MCR p15,0,Rd,cr5,cr0,1:    INSN_AP <- 0x3FFF     
FFFF23BC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x2078
FFFF23BC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC000307D
FFFF05F8: MCR p15,0,Rd,cr9,cr1,1: XSCALE_UNLOCK_ICACHE <- 0x6        (00000000 - 00000FFF, 0x1000)
FFFF05F8: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC000307D
FFFF05F8: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC004307D
FFFF0634: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC004307D
FFFF0634: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC004107D
FFFF0634: MCR p15,0,Rd,cr9,cr1,0: XSCALE_LOCK_ICACHE_LINE <- 0x40000006 (40000000 - 40000FFF, 0x1000)
FFFF0634: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC004107D
FFFF0634: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
CF LOAD OK.
Open file for read : AUTOEXEC.BIN
Total_size=6D300
Now jump to AUTOEXEC.BIN!!
0010C08C: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010C08C: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D


I tried other roms from other cams (eosm , 6D, 5d3.123 , 100D
same thing .

I guess i could try re-installing QEMU and see if that corrects the issue.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on June 25, 2018, 01:12:16 PM
Quote from: t3r4n on May 02, 2018, 09:33:51 PM
Hey a1ex,
its a bit rough for start but it works. Seems like gdb 8 resolves the issue with the temporary breakpoints.

[...]

Can someone with a mac verify.

Included this in the install script. Please test, in particular on Mac, WSL, 32-bit Ubuntu and 64-bit Ubuntu.

I'm going to test it as well on some fresh VMs.




Edit - some test results from non-fresh systems (QEMU upgraded from previous installations):

OpenSuse 64-bit: the script installed gcc-arm-none-eabi-5_4-2016q3 and was happy with the gdb found there (32-bit, known to work well).
Mac VM: the script installed gcc 7-2014-q4, then compiled gdb 8.1 from source even though a (buggy) 64-bit gdb was already available from the gcc package. Looks like it's working!
Win10 WSL: seems to be working, still at "configure" after half an hour :P
Xenial 32-bit: OK, script happy with previous gcc/gdb.




Fresh Xenial 32-bit:

sudo apt install mercurial
hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg up qemu
cd contrib/qemu
./install.sh
y
1
y
y


Option 1 is gcc/gdb from package manager. Also tested with 2 (launchpad), 3 (ppa) and 4 (gcc from package manager, gdb 8.1 from source).

Fresh Artful 64-bit: tested options 1 and 4. Had to install makeinfo (https://sourceware.org/bugzilla/show_bug.cgi?id=18113) in order to compile gdb. (solved)

Fresh Bionic 64-bit (latest Ubuntu): tried options 1 (it no longer has the 32-bit GDB package; it has gdb-multiarch instead) and 4 (worked).

Nanomad VM: too old, it doesn't have arm-none-eabi-gcc prepackaged, pip not working, qemu compiles but doesn't run... not worth the hassle

Nikfreak VM: disk full, maybe later.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: t3r4n on June 25, 2018, 07:53:36 PM
Quote from: a1ex on June 25, 2018, 01:12:16 PM
Included this in the install script. Please test, in particular on Mac, WSL, 32-bit Ubuntu and 64-bit Ubuntu.


I tested it on Mac without the PATH to gdb set the first time it complained about the gdb being there and told me how to export the PATH. I did that.
Second run it complained about gdb being V7 it did install V8 then and told me to export PATH again.
Third time round it did the normal routine of compiling qemu.
So thumbs up from me.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 17, 2018, 04:28:55 AM
A new install of Ubuntu 18.04 made a new VDI in VirturailBox
then ran this

sudo apt install mercurial
hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg up qemu
cd contrib/qemu
./install.sh
y
4
y
y

Got this at the end .
*** Please add GDB binaries to your executable PATH, then run this script again.
*** Run this command, or paste it into your .profile and reopen the terminal:
    export PATH=/home/rdtv/gdb-arm-none-eabi-8_1/bin/:$PATH


Unfortunately this all very strange to me , being a windows 7 guy
can someone help out please ?

.profile ?
How do I add this ?  ???
here my install log/text file
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/install.txt
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 17, 2018, 05:37:07 AM
First results from some search engines:

http://www.theunixschool.com/2011/07/what-is-profile-file.html
https://www.stefaanlippens.net/bashrc_and_others/
https://unix.stackexchange.com/questions/40708/what-is-the-difference-between-profile-bashrc-bash-profile-gnomer

TLDR: generally, settings in *nix systems are changed by editing text files. To change the executable PATH (you have that one in Windows 7, too), you add commands like the above to your .profile file with a text editor. You could just paste that command the terminal, but the effect would be temporary (i.e. you'd have to do this again when opening a new terminal).

Here's some code to automate this step (didn't try yet): https://github.com/mitsuhiko/pipsi/pull/148/files

Option 1 (the recommended choice) didn't work?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 17, 2018, 06:19:58 AM
Thanks a1ex , I thought I need to use option #4 , I'll re-run with #1

Edit: Option#1 same to work Ok , it's compiling qemu as I write this
Didn't Compile , Not too sure why I may have to remake the vdi and start fresh
collect2: error: ld returned 1 exit status
Makefile:193: recipe for target 'qemu-system-arm' failed
make[1]: *** [qemu-system-arm] Error 1
Makefile:184: recipe for target 'subdir-arm-softmmu' failed
make: *** [subdir-arm-softmmu] Error 2

*** Compilation failed.
*** Please check what went wrong, try to fix it and report back.
rdtv@reddeercity:~/magic-lantern/contrib/qemu$

Log/txt file from the fail
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/failed%20qemu.txt
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 17, 2018, 07:10:19 AM
Fails on these lines from the errors i posted
line 193
(cd $(SRC_PATH)/pixman; autoreconf -v --install)
line 184
[$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" all,)
Is this still the path problem i had ?
Maybe best to start with  a fresh vm
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Audionut on July 17, 2018, 11:07:09 AM
Quote from: a1ex on July 17, 2018, 05:37:07 AM
To change the executable PATH (you have that one in Windows 7, too), you add commands like the above to your .profile file with a text editor. You could just paste that command the terminal, but the effect would be temporary (i.e. you'd have to do this again when opening a new terminal).

In windows GUI
(https://imgur.com/download/xB8zDjC)

The top path variable is for the current user only, the bottom path variable is system wide.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 17, 2018, 02:10:03 PM
Quote from: reddeercity on July 17, 2018, 06:19:58 AM
Didn't Compile , Not too sure why I may have to remake the vdi and start fresh

Tried again on my fresh Ubuntu Bionic VM, with option 1. No surprises.

No idea what's going on; expecting to see some sort of error message in the log. Some of the errors mention "make[1]: flex: Command not found", but I don't have this command either on the VM (and I also get these messages). Try running "make V=1" from qemu-eos/qemu-2.5.0:

cd ~/qemu-eos/qemu-2.5.0
make V=1


That should be a little more verbose, but normally compile errors should show up without any tweaking...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 18, 2018, 02:29:49 AM
cd ~/qemu-eos/qemu-2.5.0
make V=1

Yea that worked , thanks a1ex , There still some errors but work to compile QEMU ,
here the log/txt from that --> make_V=1log/txt (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/make_V=1.txt) , then to test I run ./run_canon_fw.sh 5D2
the log/txt file is here (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/qemu_first%20run.txt) , was successful .
Then tried
./run_canon_fw.sh 5D2,firmware="boot=0"
Log/txt file here (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/qemu_boot_to_firmware.txt) , was not successful got the same grey screen like before . I'll re-make my VM and start from fresh , there same problem there maybe from the all the updates .
I checked the version ,  Ubuntu Bionic Beaver 18.04. So I'll try again and report back .

Edit: Got this in the beginning for error , I noticed
Gtk-Message: 18:03:50.078: Failed to load module "canberra-gtk-module"
means something but not to me ,
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on July 18, 2018, 04:12:56 AM
Quote from: reddeercity on July 18, 2018, 02:29:49 AM
...got the same grey screen like before...

The 5D2 launches into a grey screen on my system too. Pressing the "M" key brings up the Canon menu. However, that error message looks like there might be a problem with your GTK (GIMP Tool Kit) installation. Try reinstalling libgtk2.0-dev. If that doesn't work, maybe try libsdl1.2-dev.

contrib/qemu/install.sh
if [  -n "$(lsb_release -i 2>/dev/null | grep Ubuntu)" ]; then
    # Ubuntu-based system? (including WSL)
    # install these packages, if not already
    # only request sudo if any of them is missing
    # instead of GTK (libgtk2.0-dev), you may prefer SDL (libsdl1.2-dev)
    packages="
        build-essential mercurial pkg-config libtool
        git libglib2.0-dev libpixman-1-dev zlib1g-dev
        libgtk2.0-dev xz-utils mtools netcat-openbsd
        python python-pip python-docutils"
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 18, 2018, 06:00:50 AM
Thanks @dfort , I got it to work now so far   :)) . I Re-made my VM (new install) with Ubuntu Bionic Beaver 18.04 , ran the update and got the latest updates for Ubuntu .
Then ran the install script

sudo apt install mercurial
hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg up qemu
cd contrib/qemu
./install.sh
y
1
y
y

Here (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Install_QEMU_full.txt) is the Log/txt file of the process . Yes I see , I pressed "m" and it's there  8)

(https://preview.ibb.co/kgn5yd/installed_QEMU.png) (https://ibb.co/dzY0Wy)

I'll try some other ROM's also to see if it's fully functional e.g. digic5 , if I'm not mistaken Liveview is not working for 5d2/digic4 right ?
I need it to figure out compressed raw & full res MJpeg at the moment plus other thing down the road .

Edit: Some screen shots of 5D2 with Liveview  ,most of the navigation works so far just need to understand how to use qemu commands .

(https://preview.ibb.co/fTcNdd/liveview_small1.png) (https://ibb.co/ev2oJd)(https://preview.ibb.co/cRXcBy/liveview_small2.png) (https://ibb.co/jcuqWy)
(https://preview.ibb.co/bsnury/liveview_small3.png) (https://ibb.co/iKKkyd)(https://preview.ibb.co/iD6ZPJ/liveview_small4.png) (https://ibb.co/fGdQWy)
(https://preview.ibb.co/mqpM4J/liveview_small5.png) (https://ibb.co/jYfiJd)(https://preview.ibb.co/nguAyd/liveview_small6.png) (https://ibb.co/kG1ZPJ)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 19, 2018, 07:48:51 AM
Got it running on my AMD Desktop(FX8350) , followed the same process as my Intel laptop
ran this
./run_canon_fw.sh 5D2 -d io,int
to get MMIO activity (registers) and interrupts , some very interesting information . Here (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2_MMIO%20activity%20(registers)%20and%20interrupts.txt) is the MMIO activity Log/txt file for the 5D2, too many enters to post .
I did see the same error as my laptop
Gtk-Message: 23:15:38.959: Failed to load module "canberra-gtk-module"
Quote from: dfort on July 18, 2018, 04:12:56 AM
However, that error message looks like there might be a problem with your GTK (GIMP Tool Kit) installation. Try reinstalling libgtk2.0-dev. If that doesn't work, maybe try libsdl1.2-dev.

contrib/qemu/install.sh
if [  -n "$(lsb_release -i 2>/dev/null | grep Ubuntu)" ]; then
    # Ubuntu-based system? (including WSL)
    # install these packages, if not already
    # only request sudo if any of them is missing
    # instead of GTK (libgtk2.0-dev), you may prefer SDL (libsdl1.2-dev)
    packages="
        build-essential mercurial pkg-config libtool
        git libglib2.0-dev libpixman-1-dev zlib1g-dev
        libgtk2.0-dev xz-utils mtools netcat-openbsd
        python python-pip python-docutils"

I'll have to try the fix @dfort suggests , but at this point I'm looking for logs files to help with reverse engineering  on digic iv
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 19, 2018, 10:05:18 AM
Nice, that's how far the emulation goes here as well.

In LiveView, you can look at the MMIO activity; it will show some initial configuration (ADTG, CMOS etc), and at some point it will expect some HEAD timer interrupts. These are not implemented, but I had a few attempts; will keep trying. For the emulation on the home page, I've loaded the image buffers manually.

That error about canberra is probably harmless: it appears to be a sound library, and I didn't try to implement anything audio-related yet.

https://stackoverflow.com/questions/20518346/gtk-message-failed-to-load-module-canberra-gtk-module
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 20, 2018, 08:11:07 AM
Thanks, I'm now try to do a "FACapture Test" i found this
./run_canon_fw.sh 5D2,firmware=boot=1 -display none -monitor stdio -d debugmsg,io,int -serial file:5D2.212-frsp-uart.log -s -S &
arm-none-eabi-gdb -x 5D2/debugmsg.gdb &

Didn't really work , thou it did make the "5D2.212-frsp-uart.log" but with "0" data , It complained about  arm-none-eabi-gdb that I don't have i guess , so i drop that & the -x 5D2/debugmsg.gdb & still no go .
From the HACKING.rst --  I then tried to run the Test suite - Bootloader code (to make sure AUTOEXEC.BIN is loaded from the card)
- Portable display test (all EOS models)
- Portable ROM dumper (EOS models with bootloader file write routines)
- Menu navigation (on supported models) — depends on user settings from the ROM
- Card formatting (and restoring ML)
- Call/return trace until booting the GUI (a rigid test that may have to be updated frequently)
- Call/return trace on bootloader (likely independent of firmware version and user settings)
- Callstack consistency with call/return trace (at every DebugMsg call)
- File I/O (whether the firmware creates a DCIM directory on startup)
- FA_CaptureTestImage (basic image capture process, without compression or CR2 output)
- HPTimer (difficult to get right)
- DryOS task information (current_task, current_interrupt)
- GDB scripts (just a few basics)
- DryOS shell (UART)
- PowerShot models (limited tests)
- Secondary DryOS cores (limited tests)
with ./run_tests.sh 5D2 got errors
dmiazga@reddeercity:~/qemu-eos/tests$ ./run_tests.sh 5D2
Using netcat: nc -N
Compiling...
/sbin/losetup
/sbin/losetup
/dev/loop12p1 /media/dmiazga/EOS_DIGITAL vfat ro,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro 0 0

Error: please unmount the CF image.


and tried ./run_tests.sh 5D2 menu calls-main drysh
got this dmiazga@reddeercity:~/qemu-eos/tests$ ./run_tests.sh 5D2 menu calls-main drysh
Using netcat: nc -N
Compiling...
/sbin/losetup
/sbin/losetup
/dev/loop12p1 /media/dmiazga/EOS_DIGITAL vfat ro,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro 0 0

Error: please unmount the CF image.

"please unmount the CF image"
So how do I unmount ?
I'm just trying to run the FA_CaptureTestImage , is there something I'm missing ? Do i have to have a full set of Image Vram dumps ?  Can that test be done with compression ? CR2 so i can get the info for compressed raw.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 20, 2018, 08:28:11 AM
In your install log, the short guide printed on the console should have told you to use gdb-multiarch instead of arm-none-eabi-gdb. They are interchangeable.

To unmount: click on the Eject icon in the file manager.

Will test these steps later in the VM, maybe there are some other quirks.

For FA_CaptureTestImage, you only need a reference DNG image (you'll see it in the error messages).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: critix on July 20, 2018, 03:08:27 PM
Quote from: a1ex on July 21, 2016, 01:33:09 PM
- debugging your code like a PC program, by running it step by step (not just with printf's)
How can debugging step by step?
Thanks
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 20, 2018, 04:30:18 PM
https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst#rst-header-debugging
https://www.magiclantern.fm/forum/index.php?topic=15895.msg186173#msg186173
https://wiki.osdev.org/Kernel_Debugging#Use_GDB_with_QEMU
https://doppioandante.github.io/2015/07/10/Simple-ARM-programming-on-linux.html
https://reverseengineering.stackexchange.com/questions/8829/cross-debugging-for-arm-mips-elf-with-qemu-toolchain
https://beej.us/guide/bggdb/

Maybe a video tutorial could be useful, but I'm afraid I'm not very good at explaining things in a way suitable for beginners.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 22, 2018, 02:42:51 AM
This looks to be the correct command for cr2 compression image test
To capture a full-res image  using a CR2  (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst?fileviewer=file-view-default#rst-header-debugging)
make -C ../magic-lantern/minimal/qemu-frsp MODEL=5D2 CONFIG_QEMU=y clean install_qemu
env QEMU_EOS_VRAM_PH_QR_RAW='/path/to/IMG_1234.CR2' ./run_canon_fw.sh 5D2,firmware="boot=1"

This was for 5d3  , I changed it to 5d2 . I haven't tried it yet , I guess the big differences from the "Test" in qemu I tried before  is this uses ML running with FRSP .
I also see a "envorment" path , I'll give it a go and see what happens .
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 22, 2018, 03:44:52 AM
Back to ~/qemu-eos/tests$ ./run_tests.sh 5D2
Using netcat: nc -N
Compiling...
/sbin/losetup
/sbin/losetup
/dev/loop13p1 /media/dmiazga/EOS_DIGITAL1 vfat ro,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro 0 0

Error: please unmount the CF image.

got further when I unmounted the CF card just as a1ex suggested in a post above ,
at first i couldn't find it . It ended up in the "Other Location" in Ubuntu  , It had 2 virtual cf cards I unmounted them both , run the ./run_tests.sh 5D2 again stopped at "Testing Canon menu ... " could not find the "sponge" command
dmiazga@reddeercity:~/qemu-eos/tests$ ./run_tests.sh 5D2
Using netcat: nc -N
Compiling...

Setting up temporary SD/CF card images...
'../magic-lantern/contrib/qemu/sd.img.xz' -> './sd.img.xz'

Testing Canon menu...
./run_tests.sh: line 398: sponge: command not found

    ( printf "%7s: " $CAM && test_$1 ) 2>&1 | sponge
What's the "sponge" command ?
is there some workaround here  , or ?

Edit: Ok got "sponge" to work i had to to install a tools set ubuntu/bionic/moreutils (https://packages.ubuntu.com/bionic/moreutils)
/usr/bin/chronic
.......
/usr/bin/sponge
/usr/bin/ts
/usr/bin/vidir
/usr/bin/vipe
/usr/bin/zrun
/usr/share/doc/moreutils/README
.....


Now it stopped at
~/qemu-eos/tests$ ./run_tests.sh 5D2
Using netcat: nc -N
Compiling...

Setting up temporary SD/CF card images...
'../magic-lantern/contrib/qemu/sd.img.xz' -> './sd.img.xz'

Testing Canon menu...
    5D2: ¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿...¿.¿.¿tests/check_grep.sh: line 2: ansi2txt: command not found
FAILED!

so i guess i need to find "ansi2txt" now
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 22, 2018, 06:05:08 AM
However , after checking the "Test"directory in the 5D2 folder i see there is 30 PNG images and a Menu.Log  :))
the LOG makes me so Excited ! I see very valuable info , it's like a startup Log but so much more , even has some "Resources" for compressed raw , not to sure how complete it is but i never seen this level of information .
I'll post the LOG file link shortly .
 
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: critix on July 22, 2018, 06:13:31 AM
How long does the script for a camera run? I ran for 1300D, but after 2 hours, I had to stop the script.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 22, 2018, 06:36:07 AM
Ok here the menu.Log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/menu.log) & menu.txt (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/menu.txt) just incase there a problem reading the .Log file plus here are menu screen shots that qemu made Menu_Screen_shots.rar (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Menu_Screen_shots.rar) there are some black image ,
can only assume that those are from thing that are not functional in qemu.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 22, 2018, 06:52:56 AM
The test suite doesn't exactly work out of the box; there are a few patches to vncdotool that I've installed manually on the build server. Need to handle these in the installer script.

For FRSP, looks like "make clean install_qemu" no longer works in one command, figure out why. Workaround: split the command into "make clean" and "make install_qemu".

In any case, FA_CaptureTestImage emulation doesn't work on 5D2 yet. It does work on 50D, 60D and other DIGIC 4 models though. The test suite only covers the following models:

Testing FA_CaptureTestImage...
    5D3: OK (no display)
   500D: OK
   550D: OK
    50D: OK
    60D: OK
  1100D: OK (no display)
  1200D: OK
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 22, 2018, 07:49:46 AM
Ok , I'll keep working at it . Thanks
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 23, 2018, 07:18:32 AM
Almost got ML to run in qemu , use this command  make -C ../magic-lantern/platform/5D2.212 install_qemu
./run_canon_fw.sh 5D2,firmware="boot=1" -s -S & arm-none-eabi-gdb -x 5D2/patches.gdb


(https://image.ibb.co/kzXMWy/Screenshot_from_2018_07_22_22_49_21_small.png) (https://imgbb.com/)
I cann't get in to the ml menu , is there a "trash can" key ?
tried "M" key nothing

So when i tried to shut qemu while ml was loaded i got a interesting result
(https://image.ibb.co/c4V8By/Screenshot_from_2018_07_22_23_00_47_small.png) (https://imgbb.com/)

Just kept looping until i closed terminal window , couldn't find any recording , never seen this before .

Edit: Found the recording , in the cf.img , just chick it to open it up and ML is installed in there plus a MVI_8611.MOV was saved but with "0" data
in the setting folder I found the magic.cfg file
# Magic Lantern Nightly.2018Jul22.5D2212 (cc1331663f9c+ (qemu))
# Built on 2018-07-23 04:42:22 UTC by dmiazga@reddeercity
# Configuration saved on 2017/09/30 12:15:00
disp.mode.x = 215


Hope I can get this to work right to test builds
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 24, 2018, 02:47:49 AM
Seem Intel cpu has a better time emulating ml in qemu  , the above screen shots are from my AMD desktop .
This one below is from my Intel i5 laptop

(https://image.ibb.co/jj4kto/Screenshot_from_2018_07_23_18_37_12_small.png) (https://imgbb.com/)

Edit: I can get a .MOV/h264 to start by pressing the space but can't stop it nor can I bring up the ml menu so I can't load any
modules , the only way to stop the recording it to power down then goes to the blue disc "recording ..." then I have to crash it to make it stop .
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 26, 2018, 07:04:23 AM
Back to the cr2 frsp test
make -C ../magic-lantern/minimal/qemu-frsp MODEL=5D2 CONFIG_QEMU=y clean install_qemu
env QEMU_EOS_VRAM_PH_QR_RAW='/path/to/IMG_1234.CR2' ./run_canon_fw.sh 5D2,firmware="boot=1"


dosen't work as noted before , it gets 2 errors
make -C ../magic-lantern/minimal/qemu-frsp MODEL=5D2 CONFIG_QEMU=y  install_qemu
make: Entering directory '/home/david/magic-lantern/minimal/qemu-frsp'
Using /usr/bin/arm-none-eabi-gcc (from PATH).
../../Makefile.inc:70: removing ../../platform/*/magiclantern.sym
[ RM dir   ]   /home/david/magic-lantern/minimal/qemu-frsp/zip/
[ RM       ]   ../../platform/*/magiclantern.sym
mkdir -p /home/david/magic-lantern/minimal/qemu-frsp/zip
make -C ../../installer/5D2.212 build_fir
make[1]: Entering directory '/home/david/magic-lantern/installer/5D2.212'
[ VERSION  ]   ../../platform/5D2.212/version.bin
[ VERSION  ]   ../../platform/5D2.212/version.c
[ CC       ]   version.o
[ LD       ]   magiclantern
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 30424 bytes
[ CC       ]   reboot.o
[ LD       ]   autoexec-fir
[ OBJCOPY  ]   autoexec-fir.bin
[ XOR_CHK  ]   autoexec-fir.bin
python ../../../dumper/build_fir.py -r ../../../dumper/5D200212.FIR autoexec-fir.bin ML-SETUP.FIR 0x80000218
python: can't open file '../../../dumper/build_fir.py': [Errno 2] No such file or directory
../../platform/Makefile.platform.extras:5: recipe for target 'build_fir' failed
make[1]: [build_fir] Error 2 (ignored)
make[1]: Leaving directory '/home/david/magic-lantern/installer/5D2.212'
cp ../../installer/5D2.212/ML-SETUP.FIR ML-SETUP.FIR
cp: cannot stat '../../installer/5D2.212/ML-SETUP.FIR': No such file or directory
../../platform/Makefile.platform.extras:46: recipe for target 'ML-SETUP.FIR' failed
make: *** [ML-SETUP.FIR] Error 1
make: Leaving directory '/home/david/magic-lantern/minimal/qemu-frsp'


it seem to crash on "ML-SETUP.FIR"
Any hints/helps  :D

If i run
- compile from minimal/qemu-frsp with "make MODEL=5D2

i get it to compile, with whole branch of file  plus a Autoexec
david@reddeercity:~/magic-lantern/minimal/qemu-frsp$ MODEL=5D2 make
Using /usr/bin/arm-none-eabi-gcc (from PATH).
[ VERSION  ]   ../../platform/5D2.212/version.bin
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ CC       ]   minimal.o
[ CC       ]   font_direct.o
[ CC       ]   raw.o
[ CC       ]   vram.o
[ CC       ]   propvalues.o
[ CC       ]   stdio.o
[ CC       ]   imgconv.o
[ CC       ]   dialog_test.o
[ AR       ]   strrchr.o
[ AR       ]   dietlibc.a
[ AR       ]   lib_a-setjmp.o
[ AR       ]   newlib-libc.a
[ CP       ]   newlib-libm.a
[ CP       ]   gcc-libgcc.a
[ LD       ]   magiclantern
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 35653 bytes
[ CC       ]   reboot.o
[ CC       ]   disp_direct.o
[ CC       ]   footer.o
[ LD       ]   autoexec
[ OBJCOPY  ]   autoexec.bin
[ XOR_CHK  ]   autoexec.bin

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x000060 0x0004e000 0x0004e000 0x08b45 0x0e524 RWE 0x10

but not sure what to do with it , Can i call that minimal build up in qemu ?
with out making the cf.img , plus how do load a nightly build in qemu
i'm kind of lost on the proper syntax . :)

Edit : I almost forgot i had to add a folder in  Home~/magic-lantern/minimal/5D2/Makefile
MODEL=5D2
include ../Makefile.minimal

there was no 5d2 make file so i add it ,
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 26, 2018, 08:23:29 AM
Unable to reproduce your error. Here it doesn't try to build any FIR files from the minimal target. edit: reproduced, here's a workaround:


touch ../magic-lantern/installer/5D2.212/ML-SETUP.FIR
make -C ../magic-lantern/minimal/qemu-frsp MODEL=5D2 clean
make -C ../magic-lantern/minimal/qemu-frsp MODEL=5D2 CONFIG_QEMU=y install_qemu


I also didn't have to create any directories; looking into it.

Once you get it to compile, run the emulation with -d debugmsg,io:

./run_canon_fw.sh 5D2,firmware=boot=1 -d debugmsg,io


and it will lock up here:

[GPIO] at ShootCapt:FF9B72E4:FFA360E0 [0xC022001C] -> 0x1       : GPIO_7


The cleanest way to get further is to get a MMIO log during photo capture, that includes this register, to see what values the firmware expects from there. You can do that from either dm-spy-experiments or io_trace_full, with CONFIG_DEBUG_INTERCEPT=y and CONFIG_MMIO_TRACE=y. In io_trace.c, use this definition:

static ASM_VAR uint32_t protected_region = REGION(0xC0220000, 0x1000);


and... pray it won't lock up...

If it locks up, there's the "old-school" way of adding manual logging hooks in dm-spy-extra.c. Run the emulation with -d io_log and you'll get this line:

    { 0xFF9B72E8, "0xC022001C", R(0), mmio_log },     /* [GPIO] GPIO_7 at ShootCapt:FF9B72E4 (0x1)*/


Set CONFIG_MMIO_TRACE=n in Makefile.user, copy that line into dm-spy-extra.c under CONFIG_5D2 and this time it should not lock up. However, this method only works with a small number of logging hooks at a time, while io_trace attempts to capture everything at once. Still couldn't figure out why it locks up on DIGIC 4.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on July 28, 2018, 07:28:00 AM
Thanks @a1ex for the help ! I'll give these workaround a try this weekend .  :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 28, 2018, 08:59:28 AM
Nice, also committed a fix, so the workaround should no longer be needed. Also, "make clean install_qemu" appears to work with -j1, but couldn't figure out how to specify the correct dependencies so it also works with -j8. Help welcome.

Currently working on emulating FA_CaptureTestImage on most other models. Current status (not yet committed):
- 5D2 almost working (showing Image Power Failure in logs; changing that register breaks other models)
- 6D almost working (need a reference test image (https://www.magiclantern.fm/forum/index.php?topic=15088.msg204627#msg204627), CR2 saves a smaller area than what is captured)
- 70D working
- 1100D almost working (unsure how the preview looks like (https://www.magiclantern.fm/forum/index.php?topic=1009.msg204473#msg204473))
- 600D working (the ROM I've got had Auto ISO by default, had to change it from the test code)
- 650D, 700D, 100D all failing in the same spot, didn't figure out yet first two working
- 1300D, EOSM, 1300D all failing early, didn't check yet
- VxWorks models not compiling the minimal example, didn't check yet
- DIGIC 6/7: image capture tasks not started, will leave these for later
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 02, 2018, 07:14:13 AM
Got busy on the weekend , just got around to trying this out again .
Thanks @a1ex for the workaround , I see there a commit to fix the issue but i just when with the work around .
Compiled Ok , then ran
./run_canon_fw.sh 5D2,firmware=boot=1 -d debugmsg,io
and as noted it locks up at
[GPIO] at ShootCapt:FF9B72E4:FFA360E0 [0xC022001C] -> 0x1       : GPIO_7
I tried to get same info before it locks up , I crashed qemu to get the terminal log
not too much there at least to me , (I'll post the log/text file shortly) there was some hopeful info when i ran the FA_CaptureTestImage , qemu first had a gray screen  (that's normal i guess) then it turn black and said "No Image" and stayed black and no key strokes could bring up the canon menu gui .
When i switch to serial0 i see this
(https://image.ibb.co/ezUODz/Screenshot_from_2018_08_01_20_12_23_small.png) (https://imgbb.com/)
Interesting thing , when i dropped the
-d debugmsg,io
It didn't lockup in  terminal
./run_canon_fw.sh 5D2,firmware=boot=1
DebugMsg=0xFF86AF64 (from GDB script)
Gtk-Message: 20:56:04.553: Failed to load module "canberra-gtk-module"
Lockdown read 1
Lockdown read 1
Lockdown read 0
Lockdown read 0
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
Lockdown read 5
Lockdown read 5
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
E8000000 - E8052FFF: eos.ram_extra
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - CFFFFFFF: eos.mmio
[EOS] loading './5D2/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] mirrored data; unique 0x400000 bytes repeated 0x4 times
[EOS] loading './5D2/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[EOS] mirrored data; unique 0x800000 bytes repeated 0x2 times
[MPU] warning: non-empty spell #5 (PROP_CARD3_STATUS) has duplicate(s): #16
[MPU] warning: non-empty spell #17 (PROP_CARD1_STATUS) has duplicate(s): #48
[MPU] warning: non-empty spell #31 (PROP 8003001A) has duplicate(s): #36
[MPU] warning: non-empty spell #37 (PROP_VIDEO_MODE) has duplicate(s): #38

[MPU] Available keys:
- Arrow keys   : Navigation
- Numpad keys  : Joystick (8 directions)
- Numpad 5     : Joystick center
- PgUp, PgDn   : Sub dial (rear scrollwheel)
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess (press only)
- M            : MENU (press only)
- P            : PLAY (press only)
- I            : INFO/DISP (press only)
- L            : LiveView (press only)
- W            : Pic.Style (press only)
- Z/X          : Zoom in/out
- Shift        : Half-shutter
- 0/9          : Mode dial (press only)
- V            : Movie mode (press only)
- B            : Open battery door
- C            : Open card door
- F10          : Power down switch
- F1           : show this help

Setting BOOTDISK flag to FFFFFFFF
[DMA3] Copy [0xF8760000] -> [0xE8000000], length [0x00053000], flags [0x00000001]
[DMA3] OK
FFFF2368: MCR p15,0,Rd,cr6,cr0,0:  946_PRBS0 <- 0x3F       (00000000 - FFFFFFFF, 0x100000000)
FFFF2370: MCR p15,0,Rd,cr6,cr1,0:  946_PRBS1 <- 0x3D       (00000000 - 7FFFFFFF, 0x80000000)
FFFF2378: MCR p15,0,Rd,cr6,cr2,0:  946_PRBS2 <- 0xE0000039 (E0000000 - FFFFFFFF, 0x20000000)
FFFF2380: MCR p15,0,Rd,cr6,cr3,0:  946_PRBS3 <- 0xC0000039 (C0000000 - DFFFFFFF, 0x20000000)
FFFF2388: MCR p15,0,Rd,cr6,cr4,0:  946_PRBS4 <- 0xFF80002D (FF800000 - FFFFFFFF, 0x800000)
FFFF2390: MCR p15,0,Rd,cr6,cr5,0:  946_PRBS5 <- 0x39       (00000000 - 1FFFFFFF, 0x20000000)
FFFF2398: MCR p15,0,Rd,cr6,cr6,0:  946_PRBS6 <- 0xF780002D (F7800000 - F7FFFFFF, 0x800000)
FFFF23A0: MCR p15,0,Rd,cr2,cr0,0: DCACHE_CFG <- 0x70       
FFFF23A8: MCR p15,0,Rd,cr3,cr0,0:       DACR <- 0x70       
FFFF23AC: MCR p15,0,Rd,cr2,cr0,1: ICACHE_CFG <- 0x70       
FFFF23B0: MCR p15,0,Rd,cr5,cr0,0:    DATA_AP <- 0x3FFF     
FFFF23B8: MCR p15,0,Rd,cr5,cr0,1:    INSN_AP <- 0x3FFF     
FFFF23BC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x2078
FFFF23BC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC000307D
FFFF05F8: MCR p15,0,Rd,cr9,cr1,1: XSCALE_UNLOCK_ICACHE <- 0x6        (00000000 - 00000FFF, 0x1000)
FFFF05F8: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC000307D
FFFF05F8: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC004307D
FFFF0634: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC004307D
FFFF0634: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC004107D
FFFF0634: MCR p15,0,Rd,cr9,cr1,0: XSCALE_LOCK_ICACHE_LINE <- 0x40000006 (40000000 - 40000FFF, 0x1000)
FFFF0634: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC004107D
FFFF0634: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
CF LOAD OK.
Open file for read : AUTOEXEC.BIN
Total_size=A3A0
Now jump to AUTOEXEC.BIN!!
0010C08C: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010C08C: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
0010BF60: MCR p15,0,Rd,cr7,cr5,0: FlushICache <- 0x0       
00809820: MCR p15, ...          : CACHEMAINT x770 (omitted)
00809820: MCR p15,0,Rd,cr7,cr5,0: FlushICache <- 0x0       
[boot] copy_and_restart 0x4e000 (319488)
0004E134: MCR p15, ...          : CACHEMAINT x257 (omitted)
0004E134: MCR p15,0,Rd,cr7,cr5,0: FlushICache <- 0x0       
K218 READY
[DMA1] Copy [0xF85B0000] -> [0x40302E00], length [0x001AD038], flags [0x00030001]
[DMA1] OK
[DMA2] Copy [0xF0330000] -> [0x404B2F00], length [0x0002F5AC], flags [0x00030001]
[DMA2] OK
     0:    26.112 [STARTUP] ICU Firmware Version 2.1.2 ( 6.9.8 )
[DMA2] Copy [0xF032E000] -> [0x404F3100], length [0x00000994], flags [0x00030001]
[DMA2] OK
[DMA2] Copy [0xF0390000] -> [0x404F4300], length [0x00018028], flags [0x00030001]
[DMA2] OK
[DMA2] Copy [0xF800E000] -> [0x40514500], length [0x000007E0], flags [0x00030001]
[DMA2] OK
[MPU] Received: 06 04 02 00 00 00  (Init - spell #1)
[MPU] Sending : 08 07 01 33 03 03 03 00  (PROP 80000029)
[MPU] Sending : 06 05 01 20 01 00  (PROP_CARD1_EXISTS)
[MPU] Sending : 06 05 01 21 00 00  (PROP_CARD2_EXISTS)
[MPU] Sending : 06 05 01 22 00 00  (PROP_CARD3_EXISTS)
[MPU] Sending : 06 05 03 0c 01 00  (PROP_CARD1_RECORD)
[MPU] Sending : 06 05 03 0d 01 00  (PROP_CARD2_RECORD)
[MPU] Sending : 06 05 03 0e 01 00  (PROP_CARD3_RECORD)
[MPU] Sending : 08 06 01 23 00 00 00 00  (PROP_CARD1_STATUS)
[MPU] Sending : 08 06 01 24 00 00 00 00  (PROP_CARD2_STATUS)
[MPU] Sending : 08 06 01 25 00 00 00 00  (PROP_CARD3_STATUS)
[MPU] Sending : 06 05 01 2e 01 00  (PROP_SAVE_MODE)
[MPU] Sending : 06 05 01 2c 01 00  (PROP_CURRENT_MEDIA)
[MPU] Sending : 06 05 03 20 01 00  (PROP_STARTUP_CONDITION)
[MPU] Sending : 06 05 01 3d 00 00  (PROP_TEMP_STATUS)
[MPU] Sending : 06 05 01 42 00 00  (PROP_PHOTO_STUDIO_MODE)
[MPU] Sending : 06 05 01 43 00 00  (PROP 80040017)
[MPU] Sending : 06 05 01 46 00 00  (PROP_PHOTO_STUDIO_ENABLE_ISOCOMP)
[MPU] Sending : 06 05 01 44 00 00  (PROP 80040018)
[MPU] Sending : 06 05 01 00 03 00  (PROP_SHOOTING_MODE)
[MPU] Sending : 2c 2a 02 00 03 03 05 00 00 00 00 48 00 00 09 17 70 00 00 fe 00 85 07 08 01 03 06 00 00 04 07 08 01 03 01 00 00 07 08 00 73 3b 01 00  (Init group)
[MPU] Sending : 06 05 01 37 00 00  (PROP_CARD_EXTENSION)
[MPU] Sending : 06 05 01 49 00 00  (PROP_LIVE_VIEW_AF_SYSTEM)
[MPU] Received: 08 06 00 00 02 00 00 00  (Complete WaitID = 0x80000001 Init - spell #2)
[MPU] Sending : 06 05 01 3e 00 00  (PROP_ELECTRIC_SHUTTER_MODE)
[MPU] Sending : 08 06 01 45 00 10 00 00  (PROP_METERING_TIMER_FOR_LV)
[MPU] Sending : 06 05 01 48 02 00  (PROP_LIVE_VIEW_MOVIE_SELECT)
[MPU] Sending : 06 05 01 4b 02 00  (PROP_LIVE_VIEW_VIEWTYPE_SELECT)
[MPU] Sending : 06 05 01 40 00 00  (PROP_STROBO_ETTLMETER)
[MPU] Sending : 06 05 01 41 00 00  (PROP_STROBO_CURTAIN)
[MPU] Sending : 06 05 01 3f 00 00  (PROP_FLASH_ENABLE)
[MPU] Sending : 16 14 01 4e 00 00 00 00 00 00 00 00 00 00 00 1e 00 00 00 0f 00 00  (PROP_VIDEO_MODE)
[MPU] Sending : 0e 0c 02 05 08 00 00 01 00 00 00 00 00 00  (PROP_CFN_1)
[MPU] Sending : 0c 0a 02 06 06 00 03 00 03 00 00 00  (PROP_CFN_2)
[MPU] Sending : 14 13 02 07 09 00 00 00 01 00 00 00 02 06 08 00 00 02 00 00  (PROP_CFN_3)
[MPU] Sending : 0e 0c 02 08 08 00 00 00 00 02 00 00 00 00  (PROP_CFN_4)
[MPU] Sending : 0a 08 03 2f 00 00 00 00 00 00  (PROP_SPECIAL_OPTION)
[MPU] Sending : 06 05 03 05 02 00  (PROP_POWER_LEVEL)
[MPU] Sending : 1e 1c 03 30 53 65 53 65 65 65 65 65 00 a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  (PROP 8003002A)
[MPU] Sending : 0e 0c 03 2e 00 00 7c 46 00 00 7e 44 00 00  (PROP_SHUTTER_COUNTER)
[MPU] Sending : 06 05 03 35 01 00  (PROP_BATTERY_REPORT_COUNTER)
[MPU] Sending : 1c 1b 03 1d 4f 03 00 00 00 18 00 4c 50 2d 45 36 00 00 00 00 00 01 00 7e 5b 1c 57 00  (PROP_BATTERY_REPORT)
[MPU] Sending : 06 04 03 36 00 00  (PROP_BATTERY_REPORT_FINISHED)
[MPU] Sending : 06 05 01 04 00 00  (PROP_AF_MODE)
[MPU] Sending : 06 05 01 06 3b 00  (PROP_APERTURE)
[MPU] Received: 06 05 03 0e 00 00  (PROP_CARD3_RECORD - spell #3)
[MPU] Received: 06 05 01 22 00 00  (PROP_CARD3_EXISTS - spell #4)
[MPU] Sending : 14 13 02 07 09 00 00 00 01 00 00 00 02 06 08 00 00 02 00 00  (PROP_CFN_3)
[MPU] Received: 08 06 01 25 00 00 00 00  (PROP_CARD3_STATUS - spell #5)
[MPU] Received: 06 05 01 37 00 00  (PROP_CARD_EXTENSION - spell #6)
[MPU] Sending : 06 05 03 23 08 00  (unnamed)
[MPU] Sending : 0e 0c 03 24 31 39 2d 33 35 6d 6d 00 00 00  (PROP_LENS_NAME)
[MPU] Received: 0a 08 03 06 00 00 00 00 00 00  (PROP_AVAIL_SHOT - spell #10)
[MPU] Sending : 06 04 03 25 00 00  (unnamed)
[MPU] Received: 06 04 03 10 00 00  (PROP 80030008 - spell #11)
[MPU] Sending : 06 05 01 04 00 00  (PROP_AF_MODE)
[MPU] Received: 06 05 03 07 ff 00  (PROP_BURST_COUNT - spell #12)
[MPU] Sending : 06 05 03 0e 00 00  (PROP_CARD3_RECORD)
[MPU] Sending : 06 05 01 22 00 00  (PROP_CARD3_EXISTS)
[MPU] Sending : 08 06 01 25 00 00 00 00  (PROP_CARD3_STATUS)
[MPU] Sending : 06 05 01 37 00 00  (PROP_CARD_EXTENSION)
[MPU] Sending : 0a 08 01 34 08 01 07 03 01 00  (PROP_CARD1_IMAGE_QUALITY)
[MPU] Received: 06 05 01 2e 01 00  (PROP_SAVE_MODE - spell #13)
[MPU] Sending : 06 05 01 2e 01 00  (PROP_SAVE_MODE)
[MPU] Received: 0a 08 03 0b 00 00 00 00 00 00  (PROP 80030007 - spell #14)
[MPU] Received: 08 06 00 00 01 34 00 00  (Complete WaitID = 0x8000002F PROP_CARD1_IMAGE_QUALITY - spell #7)
[MPU] Sending : 0a 08 01 35 00 00 06 04 01 00  (PROP_CARD2_IMAGE_QUALITY)
[MPU] Received: 08 06 00 00 01 35 00 00  (Complete WaitID = 0x80000030 PROP_CARD2_IMAGE_QUALITY - spell #8)
[MPU] Sending : 0a 08 01 36 08 01 07 03 01 00  (PROP_CARD3_IMAGE_QUALITY)
[MPU] Received: 08 06 00 00 01 36 00 00  (Complete WaitID = 0x80000031 PROP_CARD3_IMAGE_QUALITY - spell #9)
[MPU] Received: 06 05 04 0e 00 00  (PROP 8002000D - spell #15)
[MPU] Received: 08 06 01 25 00 00 00 00  (PROP_CARD3_STATUS - spell #16)
[MPU] Sending : 08 06 01 25 00 00 00 00  (PROP_CARD3_STATUS)
    28:    33.024 [STARTUP] Ceres Disappeared
   113:    51.968 [ENG] [ENGIO](Addr:0x5c640000, Data:0x   30000)
   137:    55.040 [FM] FM_RegisterSpaceNotifyCallback
   140:    55.296 [FM] FM_RegisterNumberNotifyCallback
   176:    58.624 [MC] PROP_GUI_STATE 0
   181:    58.880 [MC] JobState 0
   182:    58.880 [MC] HDMIConnect ---> (0)
   186:    59.648 [MC] regist master CardCover
   202:   183.040 [CF] ERROR GetRotatingDeviceInfo
   203:   183.040 [CF] ERROR GetMakerAndVersionTuple : SearchTuple (CISTPL_VERS_1)
   204:   183.040 [CF] ERROR GetFirstTuple: CISTPL_CONFIG
   205:   183.040 [CF] ERROR GetConfigurationTuple
[MPU] Received: 08 06 01 23 00 01 00 00  (PROP_CARD1_STATUS - spell #17)
[MPU] Sending : 08 06 01 23 00 01 00 00  (PROP_CARD1_STATUS)
[MPU] Received: 08 06 01 26 00 64 00 00  (PROP_CARD1_FOLDER_NUMBER - spell #18)
[MPU] Received: 08 07 01 29 21 a2 00 00  (PROP_CARD1_FILE_NUMBER - spell #21)
[MPU] Received: 06 05 03 07 08 00  (PROP_BURST_COUNT - spell #19)
[MPU] Received: 0a 08 03 06 00 00 00 0c 00 00  (unknown - PROP_AVAIL_SHOT)
[MPU] Received: 06 05 03 11 01 00  (PROP_ICU_AUTO_POWEROFF - spell #22)
[MPU] Received: 06 05 02 0a 00 00  (PROP_PERMIT_ICU_EVENT - spell #23)
[MPU] Sending : 06 05 01 2c 01 00  (PROP_CURRENT_MEDIA)
[MPU] Sending : 0a 08 03 00 a1 00 00 00 00 00  (PROP 80030000)
[MPU] Received: 06 05 03 19 01 00  (PROP_TFT_STATUS - spell #24)
   247:   263.168 [SND] Seq LPC fin
[MPU] Sending : 06 05 03 04 00 00  (PROP_POWER_KIND)
   280:   270.848 [PRP] M:A1 F:0 L:0 P:0
[MPU] Sending : 14 12 03 15 01 26 4f 00 a0 00 13 00 23 91 00 00 00 00 00 00  (PROP_LENS)
[MPU] Sending : 06 05 03 17 92 00  (PROP_EFIC_TEMP)
[MPU] Sending : 08 06 01 0a 00 01 00 00  (PROP_AFPOINT)
[MPU] Sending : 06 05 01 38 00 00  (PROP 80040005)
[MPU] Sending : 06 05 01 39 00 00  (PROP 80040006)
[MPU] Sending : 06 05 01 0f 00 00  (PROP 8000000F)
[MPU] Sending : 06 05 03 23 08 00  (unnamed)
[MPU] Sending : 0e 0c 03 24 31 39 2d 33 35 6d 6d 00 00 00  (PROP_LENS_NAME)
[MPU] Sending : 06 04 03 25 00 00  (unnamed)
[MPU] Sending : 06 05 01 3d 00 00  (PROP_TEMP_STATUS)
[MPU] Sending : 14 12 03 15 01 26 4f 00 a0 00 13 00 23 91 00 00 00 00 00 00  (PROP_LENS)
[MPU] Received: 06 05 09 11 01 00  (PROP_LV_DISPSIZE - spell #25)
[MPU] Received: 12 11 09 15 00 00 00 00 00 00 00 00 00 00 00 00 00 00  (PROP 80050020 - spell #26)
[MPU] Received: 26 24 09 17 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  (PROP_LV_FOCUS_DATA - spell #27)
[MPU] Received: 08 06 03 18 00 00 00 00  (PROP 8003000F - spell #28)
[MPU] Received: 08 06 03 1f 00 00 00 00  (PROP 80030019 - spell #29)
[MPU] Received: 06 05 03 13 00 00  (PROP_LOGICAL_CONNECT - spell #30)
[MPU] Received: 06 05 03 1e 00 00  (PROP 8003001A - spell #31)
[MPU] Sending : 06 05 03 35 01 00  (PROP_BATTERY_REPORT_COUNTER)
[MPU] Sending : 1c 1b 03 1d 4f 03 00 00 00 18 00 4c 50 2d 45 36 00 00 00 00 00 01 00 7e 5b 1c 57 00  (PROP_BATTERY_REPORT)
[MPU] Received: 08 07 01 3b ff ff 00 00  (PROP_USBDEVICE_CONNECT - spell #32)
[MPU] Received: 08 07 01 3b ff 00 00 00  (PROP_USBDEVICE_CONNECT - spell #33)
[MPU] Sending : 06 04 03 36 00 00  (PROP_BATTERY_REPORT_FINISHED)
[MPU] Received: 08 06 03 1f 00 00 00 00  (PROP 80030019 - spell #34)
[MPU] Received: 08 07 01 3b ff 00 00 00  (PROP_USBDEVICE_CONNECT - spell #35)
[MPU] Received: 06 05 03 1e 00 00  (PROP 8003001A - spell #36)
[MPU] Sending : 06 05 03 35 01 00  (PROP_BATTERY_REPORT_COUNTER)
   295:   312.576 [LVMD] Init RCh1=0, RCh2=0
   297:   313.088 [LVCFG] LV_Initialize Aug 16 2010
   302:   313.088 [LVMD] Set RCh1=a, RCh2=19
   306:   316.160 [LVCFG] PROP_TEMP_STATUS  Temp:0, FrameRate:1
   307:   316.160 [LVCFG] PROP_LV_ACTION STOP
   311:   316.160 [LVCFG] PROP_LV_LOCK PERIMIT
   315:   316.160 [LVCFG] PROP_SHOOTING_TYPE 0
   321:   317.440 [LV] PROP_LIVE_VIEW_FACE_AF
   328:   317.952 [LVCFG] PROP_LIVE_VIEW_VIEWTYPE_SELECT 0->2
   330:   317.952 [LVCFG] PROP_LIGHT_FALLOFF_COMP 0
   369:   328.704 [PTP] PhotoStudioMode:0
   429:   334.592 [PTP] PSI DisconnectViewFinder
   558:   340.992 [PTPCOM] SetPtpTransportResources:0,3199
   571:   340.992 WriteFROM Normal:0,0
[MPU] Sending : 1c 1b 03 1d 4f 03 00 00 00 18 00 4c 50 2d 45 36 00 00 00 00 00 01 00 7e 5b 1c 57 00  (PROP_BATTERY_REPORT)
[MPU] Sending : 06 04 03 36 00 00  (PROP_BATTERY_REPORT_FINISHED)
[MPU] Received: 16 14 01 4e 00 00 00 00 00 00 00 00 00 00 00 1e 00 00 00 0f 00 00  (PROP_VIDEO_MODE - spell #37)
[MPU] Sending : 16 14 01 4e 00 00 00 00 00 00 00 00 00 00 00 1e 00 00 00 0f 00 00  (PROP_VIDEO_MODE)
[MPU] Received: 16 14 01 4e 00 00 00 00 00 00 00 00 00 00 00 1e 00 00 00 0f 00 00  (PROP_VIDEO_MODE - spell #37)
[MPU] Sending : 16 14 01 4e 00 00 00 00 00 00 00 00 00 00 00 1e 00 00 00 0f 00 00  (PROP_VIDEO_MODE)
[MPU] Received: 06 05 03 19 01 00  (PROP_TFT_STATUS - spell #39)
[MPU] Received: 06 05 03 19 01 00  (PROP_TFT_STATUS - spell #39)
[MPU] Received: 06 05 02 0a 01 00  (PROP_PERMIT_ICU_EVENT - spell #41)
[MPU] Sending : 06 05 06 11 01 00  (GUI_SWITCH)
[MPU] Received: 06 05 03 16 06 00  (PROP_BATTERY_CHECK - spell #47)
[MPU] Sending : 06 05 06 12 00 00  (GUI_SWITCH)
[MPU] Sending : 42 41 0a 08 ff 1f 01 00 01 01 a0 10 00 73 01 01 50 25 3b 01 01 00 48 04 01 08 18 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  (PD_NotifyOlcInfoChanged)
   698:   368.128 [HDMI] HPD OFF
   748:   388.352 [LVCFG] PROP_OUTPUT_TYPE 0(0) / 0
   800:   391.936 [GUI] HDMI_VIDEO_CODE 0
   911:   340.480 [STARTUP] startupInitializeComplete
   918:   340.480 [MC] cam event guimode comp. 0
   936:   342.784 [MC] cam event guimode comp. 0
   939:   343.040 [DISP] TurnOffDisplay (PUB) Type=0 fDisplayTurnOn=0
   946:   343.552 [DISP] TurnOffDisplay (PUB) Type=0 fDisplayTurnOn=0
[MPU] Sending : 06 05 04 0e 01 00  (PROP 8002000D)
[MPU] Sending : 06 05 03 16 00 00  (PROP_BATTERY_CHECK)
[MPU] Received: 06 05 04 0d 01 00  (PROP_ACTIVE_SWEEP_STATUS - spell #42)
[MPU] Sending : 06 05 04 0d 00 00  (PROP_ACTIVE_SWEEP_STATUS)
[MPU] Sending : 06 05 04 15 00 00  (PROP_DL_ACTION)
[MPU] Sending : 06 05 03 17 92 00  (PROP_EFIC_TEMP)
[MPU] Received: 06 05 03 19 01 00  (PROP_TFT_STATUS - spell #72)
  1045:   380.160 [DL ERROR] StartDL : not PrepareDL
  1060:   386.048 [DISP] TurnOffDisplay (PUB) Type=0 fDisplayTurnOn=0
[MPU] Received: 06 05 04 00 01 00  (NotifyGUIEvent - spell #44)
[MPU] Sending : 06 05 04 00 01 01  (NotifyGUIEvent)
[MPU] Received: 08 06 00 00 04 00 00 00  (Complete WaitID = 0x80020000 - spell #45)
  1065:  1459.968 [MC] PROP_GUI_STATE 1
  1086:  1459.968 [MC] cam event guimode comp. 1
  1103:  1462.528 [DISP] TurnOnDisplay (PUB) Type=0 fDisplayTurnOn=0
  1114:  1404.416 [DISP] BackLightOn
[MPU] Received: 06 05 03 19 00 00  (PROP_TFT_STATUS - spell #43)
  1115:  2437.120 [FA] FA_CreateTestImage
  1116:  2437.120 [RSC] SRM_ChangeMemoryManagementForFactory
  1132:  2437.120 [FA] hJob(0x42000064)(tv=0x73,av=0x3b,iso=0x48)
  1133:  2437.120 [FA] FA_CreateTestImage Fin
  1134:  2437.120 [FA] FA_CaptureTestImage(hJob:0x42000064)
  1135:  2437.120 [SHTC] SCS_FaSetSkeltonJob(0x42000064)
  1136:  2437.120 [FA] faSetProperty ID=0x80040000 Size=4 Value=0x4
  1141:  2437.120 [FA] Property Value (0x4 -> 0x4)
[MPU] Received: 06 05 03 07 07 00  (unknown - PROP_BURST_COUNT)
[MPU] WARNING: forced shutdown.
For clean shutdown, please use 'Machine -> Power Down'
(or 'system_powerdown' in QEMU monitor.)


Edit: here the txt/logs with the " -d debugmsg,io"
FA_CaptureTestImage.txt (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/FA_CaptureTestImage.txt) , FA_CaptureTestImage_2.txt (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/FA_CaptureTestImage_2.txt) and this txt/log is without " -d debugmsg,io" FA_CaptureTestImage_3_without_-d debugmsg,io.txt (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/FA_CaptureTestImage_3_without_-d%20debugmsg-io.txt)

@a1ex so the other suggestions you made need to be run on camera right ?
I did some dm-spy-experiments log a while ago but the cr2 capture progress was incomplete . I'll give it another try tomorrow .

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 02, 2018, 08:01:49 AM
I guess there some "FA" stuff , from the 3th log I posted
1115:  2410.496 [FA] FA_CreateTestImage
  1116:  2410.496 [RSC] SRM_ChangeMemoryManagementForFactory
  1132:  2410.496 [FA] hJob(0x42000064)(tv=0x73,av=0x3b,iso=0x48)
  1133:  2410.496 [FA] FA_CreateTestImage Fin
  1134:  2410.496 [FA] FA_CaptureTestImage(hJob:0x42000064)
  1135:  2410.496 [SHTC] SCS_FaSetSkeltonJob(0x42000064)
  1136:  2410.496 [FA] faSetProperty ID=0x80040000 Size=4 Value=0x4
  1141:  2410.496 [FA] Property Value (0x4 -> 0x4)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on August 02, 2018, 09:51:08 PM
Right, the MMIO_TRACE experiment was meant to be run on the camera, in order to cross-check the MMIO values from emulation with those from actual hardware, and find out their meaning. However, I think I've figured it out by looking at what the firmware expects - there is one bit checked to make sure the sensor was powered on. If that bit is not right, the firmware prints "Image Power Failure"; after that, it will (on most models) or will not (on 500D and 5D2) proceed with image capture.

FA_CaptureTestImage is currently emulated well on the following models:


Testing FA_CaptureTestImage...
    5D2: OK
    5D3: OK
     6D: OK
    50D: OK
    60D: OK
    70D: OK
   500D: OK
   550D: OK
   600D: OK
   650D: OK
   700D: OK
  1100D: OK
  1200D: OK


Also tested manually (outside the automated test suite) on 450D and 1300D. Reason: the minimal testing code is not portable enough.

EOS M and M2 are not working: apparently they fail to allocate a buffer for image capture. They also request a larger image size, compared to other 18 megapixel models. Figure out why.

EOS M2 is also a bit unique: it attempts to call some functions from the MPU first. It's also the only model that prints FA_CaptureTestImageForML.

100D and EOS M2 have something called ADTGDMA, that expects interrupt 0x37 on completion. By emulating that interrupt, FA_CaptureTestImage works on 100D, but it breaks the LiveView screenshots on EOS M2 (they look washed out after that change), so it's not committed yet.

700D and 650D were expecting some other EDMAC transfers to complete, besides the 14-bit raw data. No idea what sort of data they were expecting; I've just returned zeros (https://bitbucket.org/hudson/magic-lantern/commits/51ea8e0ba322a13e29ec60dc2c88ef05cffa082d?at=qemu).

For 5D2, this (https://bitbucket.org/hudson/magic-lantern/commits/15586890c998b43d7ab608ba7dbef5721a34f7c0) was the changeset that fixed the emulation. Unlike most other camera models, which simply checked whether the sensor is powered on (ImgPowDet) after enabling its power signal, 5D2 firmware also checked whether the sensor is off before enabling the power. So, other models could get away with a simpler emulation of the ImgPowDet register (just returning 1 was enough), while 5D2 locked up during these checks. The changeset is large because I wanted to emulate it properly on all other models, as this code also serves as documentation of what Canon hardware does, and it helped me understand this topic (https://www.magiclantern.fm/forum/index.php?topic=22401.0) a little better.

The above emulation covers only the raw image capture, without any postprocessing. It won't be able to create a CR2.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 04, 2018, 06:21:15 AM
Sound good , I'll try to get the cr2 process thought dm-spy first ,
QuoteThe cleanest way to get further is to get a MMIO log during photo capture, that includes this register, to see what values the firmware expects from there. You can do that from either dm-spy-experiments
I did notice when i cloned ml & updated to the dm-spy-experiments branch the line of code you said to add was already there static ASM_VAR uint32_t protected_region = REGION(0xC0220000, 0x010000); @ line 38 , i did make the changes in "Makefile.user.default" as you mention CONFIG_DEBUG_INTERCEPT = yCONFIG_MMIO_TRACE = y I'll report back hopefully with a complete cr2 photo capture with the full "Resources" for Lossless .
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 04, 2018, 08:29:33 AM
Holy Cow ! there so much info I can't believe it  :))
I do think I have what I need for lossless  , just got to figure out which Resources to use for compressed raw .
I won't post the interesting parts or the Logs files , even exported the digic reg's Logs and there again see valuable info for 4k/UHD
e.g. c0f06084:    10036 & c0f06088:  4f40432 That's 1:1 by the way  8)
Here the LOG's from dm-spy
dm-0000_cr2_capture.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0000_cr2_capture.log) , dm-0001_cr2_capture.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0001_cr2_capture.log) , dm-0002_cr2_capture.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0002_cr2_capture.log) , digic00-dm-spy.LOG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/digic00-dm-spy.LOG) , digic01-dm-spy.LOG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/digic01-dm-spy.LOG)

Ok I know this not the right place for this but I can't resist , here what I think could be the lossless resources
EEAF8>     RscMgr:ff8b65c4:80:01: Allocate MEM3 0 0
EEB1F>     RscMgr:ff8b6670:80:01: OK AllocateMEM3 0xBAA4C0(876544)(1)
EEBD1> FrontShtDe:ff888cc0:96:05: sdfExecuteMem1ToRawPath(5742)
EEC5E> FrontShtDe:ff888ea8:96:05: sdfExecuteMem1ToRawPath(5742)��(SemOK)
EEC9A> FrontShtDe:ff888ee0:96:05: ProcessTwoInTwoOutJpegPath(R) Start(5742)
EECE9> FrontShtDe:000965bc:00:00: *** LockEngineResources(72bed4) x17 from ffa59b1c:
EED33> FrontShtDe:00096678:00:00:      1)    10002 (read channel 0xa)
EED62> FrontShtDe:00096678:00:00:      2)        3 (write channel 0x3)
EED88> FrontShtDe:00096678:00:00:      3)        4 (write channel 0x4)
EEDAE> FrontShtDe:00096678:00:00:      4)    30000 (read connection 0x0)
EEDD8> FrontShtDe:00096678:00:00:      5)    20005 (write connection 0x5)
EEE03> FrontShtDe:00096678:00:00:      6)    20016 (write connection 0x16)
EEE26> FrontShtDe:00096678:00:00:      7)    50003 (?)
EEE48> FrontShtDe:00096678:00:00:      8)    5000d (?)
EEE69> FrontShtDe:00096678:00:00:      9)    5000f (?)
EEE8B> FrontShtDe:00096678:00:00:     10)    5001a (?)
EEEAC> FrontShtDe:00096678:00:00:     11)    80000 (?)
EEECE> FrontShtDe:00096678:00:00:     12)    90000 (?)
EEEEF> FrontShtDe:00096678:00:00:     13)    a0000 (?)
EEF10> FrontShtDe:00096678:00:00:     14)   160000 (?)
EEF33> FrontShtDe:00096678:00:00:     15)   130003 (?)
EEF58> FrontShtDe:00096678:00:00:     16)   130004 (?)
EEF7C> FrontShtDe:00096678:00:00:     17)   130005 (?)
EF007> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF048> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF07C> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF0AE> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF0DD> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF10C> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF140> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF173> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF1A1> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF1CF> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF1FD> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF227> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF250> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF27E> FrontShtDe:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
EF2C1> FrontShtDe:ffa59694:16:03: [TTJ][150,5742,0] RAW(5792,3804,0,14)
EF366> FrontShtDe:00096224:00:00: *** ConnectReadEDmac(0xa, 0x0), from ffa59770
EF3C5> FrontShtDe:00096224:00:00: *** RegisterEDmacCompleteCBR(0xa, 0xffa593f8 "[TTJ][%d,%d,%d] Read1CompleteCBR", 0x0), from ffa59780
EF415> FrontShtDe:00096224:00:00: *** ConnectWriteEDmac(0x3, 0x5), from ffa597f8
EF4F6> FrontShtDe:00096224:00:00: *** RegisterEDmacCompleteCBR(0x3, 0xffa9285c "[PackMem] CompleteInterrupt In %d", 0x3), from ffa92be0
EF522> FrontShtDe:ff9a47e0:00:02: [ENG] RegisterEDmacAbortCBR(3)
EF572> FrontShtDe:00096224:00:00: *** ConnectWriteEDmac(0x4, 0x16), from ffa59830
EF5D1> FrontShtDe:00096224:00:00: *** RegisterEDmacCompleteCBR(0x4, 0xffa59340 "[TTJ][%d,%d,%d] Write2CompleteCBR", 0x0), from ffa59840
EF600> FrontShtDe:ffa59888:16:03: [TTJ] START WR1:0x200807c WR2:0x1acac0f0
EF639> FrontShtDe:ffa598b4:16:03: [TTJ] START RD1:0x10000048 RD2:0x124d1864


There other resources being logged also , this may not be it , I'll continue on the ProcessTwoInTwoOutLosslessPath thread
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 04, 2018, 08:36:49 AM
Screen shots from the memory patches
(https://image.ibb.co/bEibMK/VRAM0_small.png) (https://imgbb.com/)
(https://image.ibb.co/gCd48z/VRAM2_small.png) (https://imgbb.com/)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on August 04, 2018, 09:15:41 AM
Looks good!

Were these tests run from LiveView? That's a bit too verbose; it's best if you can run them in photo mode.

Since it didn't lock up, you may now increase the MMIO range to be logged. Most of the interesting stuff (possibly all on DIGIC 4) is from 0xC0000000 to 0xC0FFFFFF; that's defined by REGION(0xC0000000, 0x1000000). This is the default configuration on the io_trace_full branch.

I recommend using io_trace_full instead of dm-spy-experiments for this stuff; it has a slightly nicer log format (with decoded microsecond timestamps) and it's able to capture a lot more stuff in the same amount of RAM. Known regressions: it's not able to run with CONFIG_DEBUG_INTERCEPT_STARTUP & CONFIG_MMIO_TRACE yet (but CONFIG_DEBUG_INTERCEPT & CONFIG_MMIO_TRACE works fine), and if the MMIO events captured require more than 16MB of storage, you will lose them all.

TLDR: looking for the same logs with the io_trace_full branch instead of dm-spy-experiments, logging a still image capture outside LiveView.

I'm unable to run these tests right now, since I'm out for holidays, far away from the city.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 05, 2018, 08:55:38 AM
Quote from: a1ex on August 04, 2018, 09:15:41 AM
Looks good!
Were these tests run from LiveView? That's a bit too verbose; it's best if you can run them in photo mode.
Yes Liveview in photo mode without any modules loaded , I can re-run that one outside of live in photo mode .

Quote from: a1ex on August 04, 2018, 09:15:41 AM
I recommend using io_trace_full instead of dm-spy-experiments for this stuff; it has a slightly nicer log format
Ok , I ran it with "io_trace_full" and there of course is a lot of "MMIO" but I could not make much sense of it , there was no
cr2 capture process or resources like in dm-spy logs , even thou I took 2 cr2 photo (there still on the card)  here (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0000_io-trace-full.log) the io-trace-full log , I'll re-run this tomorrow I think I must of did something wrong .
I did run it in Liveview with photo mode , I see I should be outside of liveview in photo mode , so I'll try again later
I had CONFIG_DEBUG_INTERCEPT & CONFIG_MMIO_TRACE set to "Y" yes .




Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on August 05, 2018, 05:33:38 PM
Yes, please run them outside LiveView.

The default settings of what gets captured are different between the two branches. For example, io_trace_full has LOG_INTERRUPTS enabled, while dm-spy-experiments doesn't.

There is a line you should watch out for:

[MMIO] WARNING: lost data (try increasing buffer size)


If this line is present, I'm afraid the log is not good, since MMIO entries required more 16MB of storage and this case is not handled very well. If that happens, I just stop the experiment earlier. It's very easy to trigger this in LiveView, where you get lots of messages for every single frame, but I doubt it will happen outside LiveView, while capturing one single image.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on August 05, 2018, 05:37:56 PM
Nice work reddeercity.

So I take it that what's good for the gander is good for the goose? Meaning what you find on the 5D2, and how you find it, will also apply to the other Digic 4 cameras like the 7D, 50D, 500D, 550D, 600D, 1100D, 1200D and maybe even the 1300D?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 06, 2018, 04:14:00 AM
Quote from: a1ex on August 05, 2018, 05:33:38 PM
Yes, please run them outside LiveView.
Ok will do
Quote from: a1ex on August 05, 2018, 05:33:38 PM
There is a line you should watch out for:

[MMIO] WARNING: lost data (try increasing buffer size)

Yea I saw that for a belief second or 2 .

Quote from: dfort on August 05, 2018, 05:37:56 PM
Nice work reddeercity.
Thanks
Quote from: dfort on August 05, 2018, 05:37:56 PM
So I take it that what's good for the gander is good for the goose? ....
Yea sure , I never thought that what I was doing would apply to the newer D4+ cams .
I was only thinking 50D & 7D would benefit , the more the merrier I say !
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 06, 2018, 06:50:20 AM
Ran the io-trace-full outside of Liveview , here's the dm-0000_io-trace-full-outside-liveview-8-5-2018.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0000_io-trace-full-outside-liveview-8-5-2018.log) (for those just joining in it's from 5D2)
Looks much better ! closer to the dm-spy but many more MMIO logging , it took a long time just to quickly look though it . I did found some "resources" for what I think is lossless stuff
plus so much more , should help me with 4k/UHD dev.

6.834.203  FrontShtDe:ff9b9190:00:00: *** SDSFrontState: (1) --3--> (1)          ff88afcc (x=702ef0 z=b3e26c t=80000003)
6.834.236  FrontShtDe:ff88b008:96:05: sdsMem1ToJpegDevelop(5749)
6.834.254  FrontShtDe:ff8890a0:96:05: sdfExecuteMem1ToJpegPath(5749)
6.834.366  FrontShtDe:ff889290:96:05: ProcessTwoInTwoOutJpegPath(J) Start(5749)
6.834.434  FrontShtDe:ffa59b1c:00:00: *** LockEngineResources(72c400) x56:
6.834.465  FrontShtDe:000970ec:00:00:      1)    10002 (read channel 0xa)
6.834.485  FrontShtDe:000970ec:00:00:      2)    10003 (read channel 0xb)
6.834.500  FrontShtDe:000970ec:00:00:      3)        3 (write channel 0x3)
6.834.514  FrontShtDe:000970ec:00:00:      4)        4 (write channel 0x4)
6.834.533  FrontShtDe:000970ec:00:00:      5)    30000 (read connection 0x0)
6.834.552  FrontShtDe:000970ec:00:00:      6)    30021 (read connection 0x21)
6.834.570  FrontShtDe:000970ec:00:00:      7)    20005 (write connection 0x5)
6.834.587  FrontShtDe:000970ec:00:00:      8)    20003 (write connection 0x3)
6.834.599  FrontShtDe:000970ec:00:00:      9)    50003 (?)
6.834.610  FrontShtDe:000970ec:00:00:     10)    5000d (?)
6.834.622  FrontShtDe:000970ec:00:00:     11)    5000f (?)
6.834.633  FrontShtDe:000970ec:00:00:     12)    5001a (?)
...............................
6.835.178  FrontShtDe:000970ec:00:00:     55)   220022 (?)
6.835.191  FrontShtDe:000970ec:00:00:     56)   220023 (?)

another interesting thing
6.630.657  ShootPreDe:ff884154:95:05: WB RectH:(2960, 2992)
6.630.677  ShootPreDe:ff884174:95:05: WB RectV:(1911, 1943)

Slices ?

I'll run the dm-spy outside Liveview in photo mode also and post the log

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on August 06, 2018, 08:27:30 AM
Looks good!

Some tips - how to "skim" this file.

SCSState aka ShootCapture, i.e. Canon's image capture task (cross-check with these notes (https://www.magiclantern.fm/forum/index.php?topic=1915.0)):


cat dm-0000_io-trace-full-outside-liveview-8-5-2018.log |grep -a SCSState
6.209.844  ShootCaptu:ff9b9190:00:00: *** SCSState: (1) --1--> (2)          ff87f914 (x=640084 z=0 t=0)
6.229.970  ShootCaptu:ff9b9190:00:00: *** SCSState: (2) --2--> (4)          ff87fe5c (x=640084 z=40b2bfe0 t=25)
6.240.560  ShootCaptu:ff9b9190:00:00: *** SCSState: (4) --3--> (5)          ff880560 (x=640084 z=0 t=0)
6.273.068  ShootCaptu:ff9b9190:00:00: *** SCSState: (5) --4--> (6)          ff880600 (x=640084 z=0 t=0)
6.278.745  ShootCaptu:ff9b9190:00:00: *** SCSState: (6) --5--> (7)          ff8808cc (x=640084 z=0 t=0)
6.324.911  ShootCaptu:ff9b9190:00:00: *** SCSState: (7) --6--> (8)          ff880a1c (x=640084 z=0 t=0)
6.442.677  ShootCaptu:ff9b9190:00:00: *** SCSState: (8) --10--> (8)          ff88118c (x=640084 z=0 t=0)
6.562.736  ShootCaptu:ff9b9190:00:00: *** SCSState: (8) --7--> (1)          ff880bd8 (x=640084 z=0 t=0)


"FPS" timers during still image capture, also raw resolution registers:

cat dm-0000_io-trace-full-outside-liveview-8-5-2018.log | grep -a 0xC0F060
6.236.948  ShootCaptu:ff9a5630:MMIO : [0xC0F06008] <- 0x05DB05DB
6.236.950  ShootCaptu:ff9a5630:MMIO : [0xC0F0600C] <- 0x05DB05DB
6.236.951  ShootCaptu:ff9a5630:MMIO : [0xC0F06010] <- 0x000005DB
...
6.273.667  ShootCaptu:ff9a5630:MMIO : [0xC0F06084] <- 0x00010037
6.273.668  ShootCaptu:ff9a5630:MMIO : [0xC0F06088] <- 0x0EDD0B87
...
6.285.858  ShootCaptu:ff9a5630:MMIO : [0xC0F06014] <- 0x00000EDC


"Plain" diagnostic log (without MMIO and interrupts):

cat dm-0000_io-trace-full-outside-liveview-8-5-2018.log |grep -av 'MMIO\|>>> INT\|<<< INT'


IMGPOWDET register (0xC022001C) and InitializePcfgPort register (my guess: 0xC0F01010):

cat dm-0000_io-trace-full-outside-liveview-8-5-2018.log |grep -a '0xC022001C\|0xC0F01010'
6.210.352  ShootCaptu:ff9b72e4:MMIO : [0xC022001C] -> 0x00000020
6.219.347  ShootCaptu:ffa36154:MMIO : [0xC0F01010] <- 0x00200000
6.229.294  ShootCaptu:ff9b72e4:MMIO : [0xC022001C] -> 0x00000021


The emulation matches these values :D
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 07, 2018, 07:02:44 AM
That's great ! Just finish running "dm-spy-experimental" Cr2 capture Log from outside Liveview , notice there more Jpeg stuff going on .
Here the dm-0001_dm-spy_outside_liveview_cr2_capture.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0001_dm-spy_outside_liveview_cr2_capture.log)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on August 07, 2018, 08:34:54 AM
May I ask for a similar log while reviewing an image, with the io_trace_full branch? That should cover JPEG decoding and might be easier to understand.

Also getting closer to emulating a CR2 image capture. Emulation reaches sdfExecuteMem1ToRawPath (FrontShtDevelop), attempts to configure JPCORE and gets stuck requesting image data from EDMAC channel #3, connection 5. I believe that's where the firmware expects LJ92 data from the lossless encoder.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 07, 2018, 08:50:37 AM
Just for the hell of it I run the dm-spy (outside Liveview) again but I set the cam to sRaw
47755> FrontShtDe:ffa596e8:16:03: [TTJ][150,5756,0] MRAW(3872,2574,0,16)
Here the dm-0002_dm-spy-sRaw_Cr2_Capture_outside_liveview.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0002_dm-spy-sRaw_Cr2_Capture_outside_liveview.log)
Here the MRaw 3872x2574 Cr2_MG_8745.CR2 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/_MG_8745.CR2)

Thought while I was at it , I run the dm-spy in Liveview while recording H264 .mov
it seems base things from 5616x3744 , nice to bypass the rez re-size and have it  5.6k H264  ;)
dm-0003_dm-spy-H264_recording_in_Liveview.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0003_dm-spy-H264_recording_in_Liveview.log)

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 07, 2018, 08:54:30 AM
Quote from: a1ex on August 07, 2018, 08:34:54 AM
May I ask for a similar log while reviewing an image, with the io_trace_full branch? That should cover JPEG decoding and might be easier to understand.

Also getting closer to emulating a CR2 image capture. Emulation reaches sdfExecuteMem1ToRawPath (FrontShtDevelop), attempts to configure JPCORE and gets stuck requesting image data from EDMAC channel #3, connection 5. I believe that's where the firmware expects LJ92 data from the lossless encoder.

Sure , I'll do that right always .
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 07, 2018, 09:38:59 AM
Ok here it is , I take a Cr2 then reviewed it outside of Liveview dm-0001_io-trace-full_reveiwing_cr2_outside_liveview.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0001_io-trace-full_reveiwing_cr2_outside_liveview.log)
Found this , not too sure if this is what your are talking about
14.986.246  ImgPlayDrv:ffa3bf78:1a:02: DEC Jpeg Format:1 W:5616 H:3744
14.986.261  ImgPlayDrv:ffa3be08:1a:02: DEC CalculateRabbitParameter 895
14.986.276  ImgPlayDrv:ffa3bf04:1a:02: DEC CalculateRabbitParameter
14.986.295  ImgPlayDrv:ffa3bf2c:1a:02: DEC XA:160 XB:16 XN:35 YA:8 YB:8 YN:467
..................................
14.986.340  ImgPlayDrv:ffa3ba30:1a:02: DEC SetResampleParametersForJuno 991
14.986.353  ImgPlayDrv:ffa3ba60:1a:02: DEC XXA:8 XXB:3 YXA:8 YXB:3
............................................
14.986.571  ImgPlayDrv:ffa3bbc8:1a:02: DEC LES_H  XA:60 XB:6 XN:35 YA:8 YB:8 YN:467
14.986.595  ImgPlayDrv:ffa3bbf0:1a:02: DEC PFIL1  XA:60 XB:6 XN:35 YA:8 YB:8 YN:467
14.986.616  ImgPlayDrv:ffa3bc18:1a:02: DEC LES_V  XA:60 XB:6 XN:35 YA:3 YB:3 YN:467
14.986.637  ImgPlayDrv:ffa3bc40:1a:02: DEC P_RES  XA:60 XB:6 XN:35 YA:3 YB:3 YN:467
..............................................
14.986.722  ImgPlayDrv:ffa3c03c:00:00: *** ConnectWriteEDmac(0x3, 0x3)
.............................
14.986.834  ImgPlayDrv:ffa3c150:1a:02: DEC ResumeDecodeJpeg 436
14.986.858  ImgPlayDrv:ffa3c194:00:00: *** StartEDmac(0x3, 0x0)
14.986.903  ImgPlayDrv:ffa3c194:00:00:     addr d00000, ptr d00000, size (please load edmac.mo)


I also by mistake I reviewed a H264 .Mov in io-trace-full and logged , lot of resizing jpeg stuff , not sure if it will be of any help.
dm-0000_io-trace-full_reviewing_mov_h264.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dm-0000_io-trace-full_reviewing_mov_h264.log)


Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 08, 2018, 07:11:33 AM
Quote from: a1ex on August 07, 2018, 08:34:54 AM
Also getting closer to emulating a CR2 image capture. Emulation reaches sdfExecuteMem1ToRawPath (FrontShtDevelop), attempts to configure JPCORE and gets stuck requesting image data from EDMAC channel #3, connection 5. I believe that's where the firmware expects LJ92 data from the lossless encoder.
That's great ! I'm going to try to see what happen on cam.
Having problems compiling lossless silent.c , what branch should I be using ? "compressed_raw" , "crop_rec_4k" or ? I was using dfort's "crop_rec_4k_Digic4" but now that's giving me errors .
Was working a few mouths ago , I just clone it , so its fresh . I'll try tomorrow again.

@dfort , can you compile your "crop_rec_4k_Digic4" ?
these are changes I made to my lossless.c
      else if (is_camera("7D", "*") || is_camera("5D2", "*"))
    {
        uint32_t resources[] = {
            0x00000 | edmac_channel_to_index(edmac_write_chan),
            0x10002 | edmac_channel_to_index(edmac_read_chan),
            0x30000,    /* read  connection  0x0 */
            0x20005,    /* write connection  0x5 */
            0x20003,    /* write connection  0x3 */
            0x50003,
            0x5000d,
            0x5000f,
            0x5001a,
            0x80000,
            0x90000,
            0xa0000,
            0x160000,
            0x130003,
            0x130004,
            0x130005,
        };
/*       
1)    10002 (read channel 0xa)
2)    10003 (read channel 0xb)
3)        3 (write channel 0x3)
4)        4 (write channel 0x4)
5)    30000 (read connection 0x0)
6)    30021 (read connection 0x21)
7)    20005 (write connection 0x5)
8)    20003 (write connection 0x3)
9)    50003 (?)
10)    5000d (?)
11)    5000f (?)
12)    5001a (?)
13)    80000 (?)
14)    d0000 (?)
15)    a0000 (?)
16)    90000 (?)
17)    e0000 (?)
18)   200000 (?)
*/


Do I really need the piece of code that's between the 2 asterisk ?
" /*       
1)    10002 (read channel 0xa)
etc. ... "
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on August 08, 2018, 07:40:31 AM
Quote from: reddeercity on August 08, 2018, 07:11:33 AM
@dfort , can you compile your "crop_rec_4k_Digic4" ?

Only for the 7D and was trying just the silent module to see if I could get lossless compression working. Hit a wall a while back and put it aside.

Quote from: reddeercity on August 08, 2018, 07:11:33 AM
Do I really need the piece of code that's between the 2 asterisk ?
" /*       
1)    10002 (read channel 0xa)
etc. ... "

No, that's copied from my log file and commented out. I was using it as reference.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: reddeercity on August 09, 2018, 07:42:16 AM
Ok , fixed my compiling problem had to commented out in
lv-img-engio.c
//total_movie_gain *= _raw_lv_get_iso_post_gain();//
now i can add my code mod's to lossless.c
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: cedricb on September 13, 2018, 06:11:01 PM
Hi,

How do you run the EOS M6 into qemu ?  There is already a structure for the M5 therefore I've tried the CHDK dump from https://drive.google.com/open?id=0B08pqRtyrObjWVdWVGVwakVmcjQ#list  but there is only a PRIMARY.bin file; how do split it to have the expected ROM0.bin and ROM1.bin ?

Before going any further, can I use qemu to use the default Canon's basic scripting, instead of constantly moving the SD card in/out of the camera/laptop ?  I'm not going to emulate ML or CHDK.

So far I've installed qemu from the ML qemu branch (followed the README from the contrib folder) which is version 2.5.0. Is this version not too old?  ...there is a qemu-2.9.0 branch but it's not maintained since April last year.


Cheers,
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 13, 2018, 06:42:27 PM
I didn't look into M6 yet, as PowerShots are not exactly my primary focus. I won't be able to look into it during the next few days, so please find the general notes for emulating a new camera model (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/HACKING.rst?fileviewer=file-view-default#rst-header-adding-support-for-a-new-camera-model).

PRIMARY.BIN is one of the two ROMs; you will need to know its start address and specify it in model_list.c (or, if that address already matches some existing model, maybe M5, start from there).

The 2.9.0 branch had some issues that were not obvious how to fix, 2.5.0 just worked, so I went forward with that. I know it's behind, but...

Just FYI, srsa_4c was able to run the A2300 GUI in QEMU, based on some draft A2200 patches from me.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: cedricb on September 13, 2018, 10:23:37 PM
@alex: Thanks for the quick reply. I'll have a go and let you know if I struggle
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on September 19, 2018, 05:32:48 PM
Came up with a recurring problem -- how to get into the ML menus in QEMU. Depending on the camera there is a different keystroke combination. Right now I'm looking at the EOSM, the Canon menus come up fine and I can get into LiveView but for the life of me I can't find my notes on how to get into the ML menu.

May I suggest adding a section in the documentation on how to get into the ML menus:

https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst#rst-header-navigating-menus

Another suggestion for the forum moderators -- how about making "How to run Magic Lantern into QEMU?!..." a sticky topic?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 19, 2018, 06:23:54 PM
It's always the same key as with regular ML. For example, on 1100D, user has to press the Av button, which is shared with Delete. In photo mode, this button behaves like the regular Av button on other models (same button code coming from the MPU). So, in the emulator, you press the Av key, i.e. A.

On EOS M, that's a long press of the Delete button (i.e. down arrow), but as the LiveView emulation is not perfect, you'll need to press L a few times to work around some GUI mode switching issue. After a bit of trial and error, the menu should appear. Just tried: M M L Down(long) showed up the long-press indicator, then ML menu showed up. It also seems to work with the Delete key, as ML interprets that button code as well in a generic way (for all models), but I'm pretty sure the hardware doesn't send that button code in LiveView, so it's not very accurate.

EOS M also opens ML menu with a double tap on the screen or something like that. I've got draft patches for touchscreen emulation, just need to clean them up and commit.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on September 20, 2018, 03:29:21 AM
Thanks -- M M L Down(long) worked here though I didn't see the long-press indicator. The camera boots into LiveView and only requires Down(long) so maybe those extra steps required in QEMU could be added to the documentation and/or the model specific F1 help menu?

BTW--On the EOSM and other cameras (i.e. 100D) the SET and Q button are combined but in QEMU they behave differently.

EOSM F1 menu
[MPU] Available keys:
- Arrow keys   : Navigation
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess (press only)
- M            : MENU (press only)
- P            : PLAY (press only)
- I            : INFO/DISP (press only)
- Q            : guess
- L            : LiveView (press only)
- Shift        : Half-shutter
- 0/9          : Mode dial (press only)
- V            : Movie mode (press only)
- B            : Open battery door
- C            : Open card door
- F10          : Power down switch
- F1           : show this help


Maybe add that the down key is also the Trash button?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on September 20, 2018, 07:26:18 AM
Quote from: dfort on September 20, 2018, 03:29:21 AM
Maybe add that the down key is also the Trash button?

The MPU code only knows it's sending the Down key; it doesn't care how the main firmware interprets it later.

In PLAY mode or Canon menu, the MPU might send the Delete code (not 100% sure; you may check that with a MPU log (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log-mpu/) - look for "06 05 06"). Other than deleting images, is the Delete button used anywhere else in Canon menu, where I could test it?

Same for the SET/Q button (which is handled identically in 100D (https://www.magiclantern.fm/forum/index.php?topic=16040.msg196814#msg196814), EOSM and EOSM2): in some GUI modes it behaves like SET, in others it behaves like Q. QEMU doesn't know that - for now, you need to press the appropriate button, i.e. to know what button code the MPU is going to send. The MPU code is not that smart, it just does a simple mapping between PC keyboard codes and MPU button codes.

This reminds me I also need to add a definition for the Delete button for 1100D & co (which is shared with Av). In photo mode, that button behaves like Av. In Canon menu, e.g. in the Format dialog, that button apparently behaves like Delete (don't see how otherwise you could toggle the low level format option).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on December 31, 2018, 01:30:53 AM
Cleaned up and committed a bunch of minor changes I had locally. Most important:
- CPU info registers are now matching the logs from real hardware (https://www.magiclantern.fm/forum/index.php?topic=17714.0) fairly well (not perfect; you are welcome to double-check)
- all GDB helpers are documented (they appear in GDB's help)
- fixed a couple of button codes and cleaned them up a bit

Still waiting for info on how the Delete button behaves in Canon menu on EOS M/M2 (previous post). Capturing a MPU log (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log-mpu/) with the linked build should be enough, as long as you press the Delete key somewhere in Canon menu and somewhere outside it. No need to compile or do complicated black magic; anyone can do it.

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on January 06, 2019, 06:29:23 AM
Quote from: a1ex on December 31, 2018, 01:30:53 AM
Still waiting for info on how the Delete button behaves in Canon menu on EOS M/M2 (previous post).

Here's a log for the EOSM. The EOSM2 will require some more work. Looks like the builds you posted have some changes that were not committed.

https://www.dropbox.com/sh/fcgwoz3t9rmhx3h/AACPzAhF3Vt_7TNsiuIOXjn2a?dl=0

Held down the Trash (a.k.a. down key) in LiveView, which brings up the ML menu then did the same with the Canon menu.

I'm once again having problems getting the EOSM/EOSM2 running in the latest QEMU. Went over previous discussions and I believe I'm doing it right. Other cameras I tried (700D/1300D) seem to run fine.

On the Mac this will crash QEMU:

./run_canon_fw.sh EOSM2,firmware="boot=0" -d debugmsg -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb

Same with "boot=1"
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on January 06, 2019, 01:47:18 PM
Something went wrong after 2.267353 seconds; memory corruption? Reproducible?

Can you try with build #21 (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log-mpu/21/) as well? That's a little less verbose, so it would allow for longer experiment times, but it will only log the MPU messages and nothing else.

Reproduced the crash on the Mac VM. Workaround: comment out the call to find_rom_string in debug-logging.gdb, until I'll figure it out. For some reason, I'm unable to run QEMU under Valgrind on the Mac VM (possibly this bug (https://www.mail-archive.com/[email protected]/msg139407.html)); if you are able to do so, I'd like to see a log. From skimming that bug report, running QEMU under Valgrind might be possible on Mac OS older than Sierra (my Mac VM is High Sierra). Edit: after building valgrind from source and applying these (https://bugs.kde.org/show_bug.cgi?id=380269#c4) patches (https://bugs.kde.org/show_bug.cgi?id=380269#c7), it still doesn't work.

Compiling QEMU with -fsanitize=address (https://clang.llvm.org/docs/AddressSanitizer.html) helps and points to a stack buffer overflow error during GDB communication. Fixed by this commit (https://github.com/qemu/qemu/commit/9005774b27b).

BTW, for regular use (i.e. if you don't need to log all of that stuff from debugmsg.gdb), it's best to use patches.gdb, as it's much faster.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on January 06, 2019, 08:31:19 PM
Quote from: a1ex on January 06, 2019, 01:47:18 PM
Something went wrong after 2.267353 seconds; memory corruption? Reproducible?

Really? I didn't notice anything unusual with the camera but yeah--there is a problem. Tried starting in different modes and the only clean log is when starting in playback mode and leaving it in playback mode until it finishes logging.

Ran the test with Build #21 (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log-mpu/21/) and this one had a clean finish. Saved log in the same Dropbox folder (https://www.dropbox.com/sh/fcgwoz3t9rmhx3h/AACPzAhF3Vt_7TNsiuIOXjn2a?dl=0). If that works for you I'll try to do the same with the EOSM2.

Quote from: a1ex on January 06, 2019, 01:47:18 PM
Workaround: comment out the call to find_rom_string in debug-logging.gdb, until I'll figure it out.

Thanks, back up and running with the EOSM/EOSM2.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on January 06, 2019, 08:54:30 PM
Quote from: dfort on January 06, 2019, 08:31:19 PM
Ran the test with Build #21 (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log-mpu/21/) and this one had a clean finish. Saved log in the same Dropbox folder (https://www.dropbox.com/sh/fcgwoz3t9rmhx3h/AACPzAhF3Vt_7TNsiuIOXjn2a?dl=0). If that works for you I'll try to do the same with the EOSM2.

This works, with only one exception - I'm unable to identify the Delete press while in Canon menu:

0.725.114  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 11 01 00)                               ; GMT_GUICMD_LOCK_ON
0.725.332  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 12 00 00)                               ; GMT_GUICMD_CLOSE_SLOT_COVER
0.730.254     PropMgr:0000393c:00:00: *** mpu_send(06 05 04 00 00 00)                               ; NotifyGUIEvent
5.439.711  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 19 01 00)                               ; BGMT_PRESS_DOWN
6.001.290     PropMgr:0000393c:00:00: *** mpu_send(06 05 04 00 06 00)                               ; NotifyGUIEvent
6.035.215  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 04 00 06 01)                               ; NotifyGUIEvent
6.046.854  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 1c 00 00)                               ; Unknown GUI event
6.988.237  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 19 00 00)                               ; BGMT_UNPRESS_DOWN
7.760.202  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 00 01 00)                               ; BGMT_MENU
8.580.178  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 00 01 00)                               ; BGMT_MENU
9.175.421  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 04 00 00 01)                               ; NotifyGUIEvent
9.666.056  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 00 01 00)                               ; BGMT_MENU
9.669.594     PropMgr:0000393c:00:00: *** mpu_send(06 05 04 00 01 00)                               ; NotifyGUIEvent
9.679.593  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 04 00 01 01)                               ; NotifyGUIEvent
9.681.301  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 1c 00 00)                               ; Unknown GUI event
11.986.296  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 19 01 00)                               ; BGMT_PRESS_DOWN
15.268.982  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 19 00 00)                               ; BGMT_UNPRESS_DOWN


Maybe the code I'm looking for only shows up when you actually try to delete some picture? Can you try to log that action as well?

That unknown GUI event has the same code as "Unpress AV" on other models. Likely some internal event used during GUI mode switching.
Quote from: dfort on January 06, 2019, 08:31:19 PM
Tried starting in different modes and the only clean log is when starting in playback mode and leaving it in playback mode until it finishes logging.

Well, that's an issue I'd like to narrow down, then.

Quote from: dfort on January 06, 2019, 08:31:19 PM
Thanks, back up and running with the EOSM/EOSM2.

It should now work on Mac without any workarounds.

Been running various tests with sanitizers; they found quite a few bugs I had no idea they were present (which were not caught by valgrind in my previous tests). Huge progress, if you ask me (or, rather, it looks like I've been living under a rock) :D
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on January 07, 2019, 02:23:57 AM
Quote from: a1ex on January 06, 2019, 08:54:30 PM
Maybe the code I'm looking for only shows up when you actually try to delete some picture? Can you try to log that action as well?

Start in play mode, delete picture -- log saved in same Dropbox folder.

Quote from: a1ex on January 06, 2019, 08:54:30 PM
Huge progress, if you ask me (or, rather, it looks like I've been living under a rock) :D

LOL -- I thought we made some great progress on the EOSM2 but now can't get it working in QEMU though the original EOSM works fine now.

[EDIT] Found the problem.

Compiling like this:
make -C ../magic-lantern EOSM2_install_qemu

This works:
./run_canon_fw.sh EOSM2,firmware="boot=0" -d debugmsg -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb
./run_canon_fw.sh EOSM2,firmware="boot=1" -d debugmsg -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb


This works only on the qemu default minimal autoexec.bin
./run_canon_fw.sh EOSM2,firmware="boot=1" -d debugmsg

This doesn't work:
./run_canon_fw.sh EOSM2,firmware="boot=0" -d debugmsg

This doesn't work at all:
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S & arm-none-eabi-gdb -x EOSM2/patches.gdb -ex quit
./run_canon_fw.sh EOSM2,firmware="boot=1" -s -S & arm-none-eabi-gdb -x EOSM2/patches.gdb -ex quit


I should qualify that. By "work" I mean able to bring up the GUI. The EOSM2 starts up on a grey screen, pressing the "M" key brings up the Canon menu. Pressing the "M" key again gets out of the Canon menu and from there the down arrow key should bring up the ML menu but I can't seem to get it on the M2. It is finicky but possible on the M. Note that I had more luck with "fn delete" on the Mac keyboard.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on January 07, 2019, 09:10:48 AM
Quote from: dfort on January 07, 2019, 02:23:57 AM
Start in play mode, delete picture -- log saved in same Dropbox folder.


0.722.727  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 11 01 00)                               ; GMT_GUICMD_LOCK_ON
0.722.942  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 12 00 00)                               ; GMT_GUICMD_CLOSE_SLOT_COVER
0.727.044     PropMgr:0000393c:00:00: *** mpu_send(06 05 04 00 01 00)                               ; NotifyGUIEvent
0.757.264  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 04 00 01 01)                               ; NotifyGUIEvent
0.759.025  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 1c 00 00)                               ; Unknown GUI event
3.278.136  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 19 01 00)                               ; BGMT_PRESS_DOWN
3.834.397  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 19 00 00)                               ; BGMT_UNPRESS_DOWN
4.590.576  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 1a 01 00)                               ; BGMT_PRESS_RIGHT
4.732.795  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 1a 00 00)                               ; BGMT_UNPRESS_RIGHT
7.762.292  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 0c 01 00)                               ; BGMT_PRESS_SET
7.974.462  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 0c 00 00)                               ; BGMT_UNPRESS_SET
8.441.945  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 19 01 00)                               ; BGMT_PRESS_DOWN
11.193.172  **INT-36h*:0001dedc:00:00: *** mpu_recv(06 05 06 19 00 00)                               ; BGMT_UNPRESS_DOWN


Maybe the main firmware really re-interprets the Down key in PLAY mode (i.e. it performs the Delete action instead). Any luck with the verbose logger on the same action?

Quote from: dfort on January 07, 2019, 02:23:57 AM
This doesn't work at all:
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S & arm-none-eabi-gdb -x EOSM2/patches.gdb -ex quit
./run_canon_fw.sh EOSM2,firmware="boot=1" -s -S & arm-none-eabi-gdb -x EOSM2/patches.gdb -ex quit


I should qualify that. By "work" I mean able to bring up the GUI. The EOSM2 starts up on a grey screen, pressing the "M" key brings up the Canon menu. Pressing the "M" key again gets out of the Canon menu and from there the down arrow key should bring up the ML menu but I can't seem to get it on the M2. It is finicky but possible on the M. Note that I had more luck with "fn delete" on the Mac keyboard.

Reproduced the issue on EOS M2... after swapping SFDATA.BIN with an older one from 100D.

The good one (likely from you) starts with 55 03 00 80 in a hex editor (0x80000355 is the model ID (https://sno.phy.queensu.ca/~phil/exiftool/TagNames/Canon.html) for EOS M2) and has MD5 f8c4d7fa1d7ceb4cade8b98b3573c375 (likely not essential, but could be useful to narrow down). It also works with -icount 5 (which is deterministic, i.e. it will get exactly the same execution trace on all systems, as long as you use the same ROMs).

Otherwise, both commands are working fine in QEMU (including on the Mac VM). If ML menu doesn't show up, press L a few times (as LiveView emulation is not the best). Before pressing L, Canon overlays are gray, but afterwards they should turn black and that's when ML menu works.

I wonder if emulating "just" the video timer interrupts could be enough for fixing this quirk.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on January 07, 2019, 11:35:46 PM
Quote from: a1ex on January 07, 2019, 09:10:48 AM
Any luck with the verbose logger on the same action?

Nope -- log keeps borking at the same place.

Quote from: a1ex on January 07, 2019, 09:10:48 AM
Reproduced the issue on EOS M2... after swapping SFDATA.BIN with an older one from 100D.

The good one (likely from you) starts with 55 03 00 80 in a hex editor (0x80000355 is the model ID (https://sno.phy.queensu.ca/~phil/exiftool/TagNames/Canon.html) for EOS M2) and has MD5 f8c4d7fa1d7ceb4cade8b98b3573c375

Strange. Found that good SFDATA.BIN and tried it with the dumps I sent you, in fact tried it with all the dumps I have and can't get out of the grey screen. The default qemu autoexec.bin works fine with any combination.

Quote from: a1ex on January 07, 2019, 09:10:48 AM
I wonder if emulating "just" the video timer interrupts could be enough for fixing this quirk.

Know what to do with that information I do not.

Back to mpu logging--

For the EOSM2 I created a patch with the changes made on the "Project startup-log-mpu" EOSM by copying the information from the autoexec.bin file. That's a great feature by the way. The EOSM2 created a full log without borking. Same action as before--down key (trash) on LiveView (brings up ML menu) then down key in Canon menu.

https://www.dropbox.com/sh/dgyn5xzfgfn9xsx/AAAIZ9ButGKdWhoWYbdbmc1Ra?dl=0
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on January 19, 2019, 10:12:45 PM
Quote from: Ant123 on December 20, 2017, 03:52:38 PM
I think it's possible to emulate simple drawing of text strings (https://chdk.setepontos.com/index.php?topic=12788.msg135622#msg135622) in case main CPU will send certain messages to MZRM core...
But on EOS M3  the camera controller still does not allow to start it normally and goes to shutdown.

Got Canon menu navigation working on M3:
CtrlSrv -> SflwWrpDrawStringWithinRect [0000,0222]: No Image.
CtrlSrv -> SflwWrpDrawStringWithinRect [-6962,0434]: Memory card locked
CtrlSrv -> SflwWrpDrawStringWithinRect [0544,0060]: SETUP4
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0167]: Certification Logo Display
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0219]: Copyright Info
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0271]: Clear all camera settings
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0115]: Wi-Fi Settings
CtrlSrv -> SflwWrpDrawStringWithinRect [0544,0060]: PLAY1
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0115]: Transition Effect
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0115]: Fade
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0167]: Index Effect
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0167]: On
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0219]: Scroll Display
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0219]: On
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0271]: Auto Rotate
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0271]: On
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0323]: Resume
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0323]: Last seen
CtrlSrv -> SflwWrpDrawStringWithinRect [0544,0060]: PLAY2
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0167]: Magnify (approx.)
CtrlSrv -> SflwWrpDrawStringWithinRect [0410,0167]: 2x
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0115]: Playback information display
CtrlSrv -> SflwWrpDrawStringWithinRect [0544,0060]: SETUP1
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0219]: Format
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0271]: Video system
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0271]: PAL
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0323]: Electronic Level
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0115]: Create Folder
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0115]: Monthly
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0167]: File Numbering
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0167]: Continuous
CtrlSrv -> SflwWrpDrawStringWithinRect [0544,0060]: SETUP2
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0167]: Power Saving
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0271]: Time Zone
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0323]: Date/Time
CtrlSrv -> SflwWrpDrawStringWithinRect [0410,0323]: '19.01.13 20:00
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0219]: LCD Brightness
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0115]: Eco Mode
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0115]: Off
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0375]: Language
CtrlSrv -> SflwWrpDrawStringWithinRect [0410,0375]: English
CtrlSrv -> SflwWrpDrawStringWithinRect [0544,0060]: SETUP3
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0167]: Hints & Tips
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0167]: Off
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0115]: Beep
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0115]: On
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0219]: Touch Operation
CtrlSrv -> SflwWrpDrawStringWithinRect [0412,0219]: Standard
CtrlSrv -> SflwWrpDrawStringWithinRect [0544,0060]: SETUP4
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0167]: Certification Logo Display
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0219]: Copyright Info
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0271]: Clear all camera settings
CtrlSrv -> SflwWrpDrawStringWithinRect [0036,0115]: Wi-Fi Settings
CtrlSrv -> SflwWrpDrawStringWithinRect [0000,0222]: No Image.


Any idea how to render these strings on QEMU VGA screen?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on January 20, 2019, 11:23:53 AM
Awesome, looking forward to trying this one!

Some time ago I've extracted the VGA font routines (fixed-width, fixed-size) from QEMU console code. It won't look pretty, but at least you'll see something.


/* basic char display */
/* adapted from console.c */

#define FONT_HEIGHT 16
#define FONT_WIDTH 8

#include "ui/vgafont.h"

#define QEMU_RGB(r, g, b)                                               \
    { .red = r << 8, .green = g << 8, .blue = b << 8, .alpha = 0xffff }

static void putcharxy(DisplaySurface *surface, int x, int y, int ch)
{
    static pixman_image_t *glyphs[256];
    pixman_color_t fgcol = QEMU_RGB(0xaa, 0xaa, 0xaa);
    pixman_color_t bgcol = QEMU_RGB(0x00, 0x00, 0x00);

    if (!glyphs[ch]) {
        glyphs[ch] = qemu_pixman_glyph_from_vgafont(FONT_HEIGHT, vgafont16, ch);
    }
    qemu_pixman_glyph_render(glyphs[ch], surface->image,
                             &fgcol, &bgcol, x, y, FONT_WIDTH, FONT_HEIGHT);
}

static void putsxy(DisplaySurface *surface, int x, int y, const char * str)
{
    for (const char * c = str; *c; c++)
    {
        putcharxy(surface, x, y, *c);
        x++;
    }
}

static void printfxy(DisplaySurface *surface, int x, int y, const char * fmt, ...)
{
    char buf[256];
    va_list ap;
    va_start( ap, fmt );
    vsnprintf( buf, sizeof(buf)-1, fmt, ap );
    va_end( ap );
    putsxy(surface, x, y, buf);
}


These routines are meant to be used in the display callback procedure, somewhere before dpy_gfx_update. In any case, it's not in sync with the program execution. If you receive these strings from MMIO hooks, or from breakpoints into certain drawing functions, you will need to store them somewhere in a data structure, until the next display event comes up.

Otherwise, the display surface can be used directly, see e.g. our LED drawing code:

    uint32_t * dest = surface_data(surface);


but, of course, you will need some library to draw the fonts (some suggestions (https://stackoverflow.com/questions/366278/graphics-library-for-embedded-systems-without-linux), or maybe even CHDK/ML RBF engine).
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on January 20, 2019, 02:17:13 PM
Quote from: a1ex on January 20, 2019, 11:23:53 AM
If you receive these strings from MMIO hooks, or from breakpoints into certain drawing functions, you will need to store them somewhere in a data structure, until the next display event comes up.

I recieve these strings from eos_handle_digic6: mzrm_send function writes 1 into register 0xD20F0840 to inform Zico core about new message.

I was trying this code, but it doesn't work:

DisplaySurface *surface = qemu_console_surface(s->disp.con);
printfxy(surface, (pos_x >> 16), (pos_y >> 16), "%s", MZRM_str);

Do I need to update the display somehow?

Even calling printfxy from eos_update_display doesn't work.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on January 28, 2019, 06:39:38 PM
Some progress with Zico core emulation:
(https://i.ibb.co/ZT4MbY7/Lubuntu-16.png) (https://ibb.co/cTfrsCV)

But direct font taken from ML doesn't look beautiful...
But I found these fonts in M3 rom:
0xfd8a9640, 0x00021704
0xfd8eb030, 0x00024060
0xfd90f090, 0x00012e60
0xfd921ef0, 0x00056d70
0xfd978c60, 0x0009fe10
0xfda18a70, 0x0001f18c
0xfda37bfc, 0x00024cf4
0xfda5c8f0, 0x0000b9a0
0xfda682a0, 0x00000cc4
0xfda68f64, 0x00000f40
0xfda6aa8c, 0x00000cb8
0xfda6b744, 0x00000854
0xfda6bf98, 0x00000b88
0xfda6dc10, 0x00000cb0
0xfda6e8c0, 0x00000ad4
0xfda6f394, 0x0000113c


16 of them can be opened with FontForge (https://ru.wikipedia.org/wiki/FontForge)

Also there need to decode images from ROM used to draw icons and some texts in Viewfinder mode.
These are vector graphic objects. They can be found by signature "99 99 0C 00" in DSLR's ROMs too. The next 32bit word is a size of drawing object.
Probably there need to use OpenVG to render it.


Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on February 05, 2019, 04:09:16 PM
Quote from: Ant123 on January 28, 2019, 06:39:38 PM
But direct font taken from ML doesn't look beautiful...

RBF font looks better:

(https://i.ibb.co/Vj9XbTD/rbf-ML.png) (https://ibb.co/2nhXRdP)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 09, 2019, 02:25:06 PM
Narrowed down a bunch of bugs in the logging code, i.e. edge and corner cases when following DryOS or VxWorks task switches. Commit (https://bitbucket.org/hudson/magic-lantern/commits/9049426c8cceaf52d9dbfcc6f5f516742652010d).

Whew, this was hard! I kept bumping into all sorts of edge cases... but hey, it finally passes the tests!!! I can finally follow all those task switches!

New feature: print number of instructions executed by each function (-d calls,tasks -icount 5). Very useful for code optimization - I actually wrote it one month ago for profiling the drawing code for DIGIC 6/7/8, in particular for helping chris_overseas with the Hello World code for 5D4. It was actually this feature (i.e. a simple assertion) that revealed all the above bugs. Commit (https://bitbucket.org/hudson/magic-lantern/commits/a4af6594437fcb950de854793c2d81c75a7aa909).

Example - profiling Hello World on 60D.111:

call 0x1FE0FE18 bmp_printf(40201, 32, 32, 1fe589da "Hello, World!")             at [ml_init:1fe008b0:ff07b8d0] (my_big_init_task)
  call 0x1FE56018 __vsnprintf_veneer(1660b0, 7f, 1fe589da "Hello, World!", 166140)
                                                                                 at [ml_init:1fe0fe44:1fe008b4] (bmp_printf)
   -> 0xFF1DA530 vsnprintf                                                       at [ml_init:1fe56018:1fe0fe48] (__vsnprintf_veneer)
  return d to 0x1FE0FE48                                                         at [ml_init:ff1da8dc:1fe008b4]                    [icount 141]
  call 0x1FE0FD80 bmp_puts(40201, 1660a4, 1660a0, 1660b0 "Hello, World!")        at [ml_init:1fe0fe58:1fe008b4] (bmp_printf)
   call 0x1FE11D20 rbf_draw_string(96868, 32, 32, 1660b0 "Hello, World!")        at [ml_init:1fe0fdf8:1fe0fe5c] (bmp_puts)
    call 0x1FE114F8 rbf_draw_char(96868, 32, 32, 48)                             at [ml_init:1fe11d88:1fe0fdfc] (rbf_draw_string)
    ...
    return 15 to 0x1FE11D8C                                                      at [ml_init:1fe11678:1fe0fdfc] (rbf_draw_char)    [icount 12905]
    ...
    return 9 to 0x1FE11D8C                                                       at [ml_init:1fe11678:1fe0fdfc] (rbf_draw_char)    [icount 5993]
   return b8 to 0x1FE0FDFC                                                       at [ml_init:1fe121f0:1fe0fe5c] (rbf_draw_string)  [icount 116740]
  return b8 to 0x1FE0FE5C                                                        at [ml_init:1fe0fe0c:1fe008b4] (bmp_puts)         [icount 116776]
return b8 to 0x1FE008B4                                                         at [ml_init:1fe0fe68:ff07b8d0] (bmp_printf)       [icount 116938]

call 0x1FE0D620 info_led_blink(1, 1f4, 1f4, 218)                                at [ml_init:1fe008d8:ff07b8d0] (my_big_init_task)
  call 0x1FE56020 __msleep_veneer(1f4, 1f4, 1f4, 218)                            at [ml_init:1fe0d650:1fe008dc] (info_led_blink)
   -> 0xFF06EA08 msleep                                                          at [ml_init:1fe56020:1fe0d654] (__msleep_veneer)
   ... (other tasks) ...
return 0 to 0x1FE008DC                                                          at [ml_init:1fe0d668:ff07b8d0] (info_led_blink)   [icount 542]


Profiling minimal Hello World on 5D4 (doesn't work out of the box yet):

call 0x1CCCB0 font_draw(64, 4b, 1, 3)                                           at [run_test:1cca48:80001737] (hello_world)
  arg5 = 1d09a4 "Hello, World!"                                                  at [run_test:1ccccc:1cca4c] (font_draw)
  call 0x1CCAA8 disp_set_pixel(64, 4b, 1, 1ccaa8 disp_set_pixel)                 at [run_test:1ccd70:1cca4c] (font_draw)
   call 0x1CD0F0 rgb2yuv422(ff, ff, ff, 1cd0f0 rgb2yuv422)                       at [run_test:1ccb00:1ccd74] (disp_set_pixel)
    -> 0x1CCF6C rgb2yuv422_rec709                                                at [run_test:1cd0f0:1ccb04] (rgb2yuv422)
   return fe80fe80 to 0x1CCB04                                                   at [run_test:1cd024:1ccd74] (rgb2yuv422_rec709)   [icount 48]
  return 80fe80 to 0x1CCD74                                                      at [run_test:1ccb5c:1cca4c] (disp_set_pixel)      [icount 89]
...
return 0 to 0x1CCA4C                                                            at [run_test:1ccd8c:80001737] (font_draw)         [icount 555882]


It reveals how many instructions are required for the entire function, for a single pixel, where various overheads come from, and so on. Very useful for me.

I'd also like to clean it up a bit, make sure it works on all major operating systems (anyone had trouble with the install script?) and merge the current state into mainline, as pretty much all recent developments depend on this.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: scrax on April 09, 2019, 04:49:13 PM
To be sure. I've made (in qemu branch):


hg pull
cd contrib/qemu/
./install.sh


I'm on OsX 10.13.6 and seems to work ok with clang
GCC: gcc-arm-none-eabi-7-2017-q4-major
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: calle2010 on April 09, 2019, 11:33:41 PM
A good oppportunity to reprovision my Vagrant environment. The install script works with latest Ubuntu Bionic

arm-none-eabi-gcc: gcc version 6.3.1 20170620 (15:6.3.1+svn253039-1build1)
gdb-multiarch: GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 10, 2019, 12:08:21 AM
Sounds good. I've tested on an older Win10 WSL VM (based on Xenial) and downloaded a newer one, hopefully based on Bionic; not expecting any surprises, other than minor usability quirks.

Also preparing to install Mojave in a VM to try that as well; High Sierra appears to work fine (well, with the "1998" Mac quirks discussed earlier in the thread).

Noticed a quirk - in the "qemu" branch I'm compiling ML with -ggdb3; this option adds debug information usable for QEMU+GDB (so you can step through ML code at source level). For some reason, this also generates slightly different binary code (aside from the additional debug information, which doesn't end up in autoexec.bin anyway). Likely minor, as this change was used in experimental builds for quite some time, without issues.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on April 10, 2019, 12:29:44 AM
I'm using QEMU on Mojave, macOS 10.14.4 using an actual (not virtual) PowerBook and it seems (almost) as good as Linux. Haven't checked to see if that 1998 quirk (https://www.magiclantern.fm/forum/index.php?topic=2864.msg201023#msg201023) is still there.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 10, 2019, 09:11:36 AM
Installed Mojave, downloaded the zip archive of the QEMU branch (from here (https://bitbucket.org/hudson/magic-lantern/downloads/?tab=branches)) with Safari (i.e. without first installing hg & co), ran the install script (cd contrib/qemu; ./install.sh) and... worked out of the box! It installed brew and other dependencies, compiled gdb 8.2.1 from source et voilà! QEMU and ML development environment installed and ready to use!

Problems:
- precompiled toolchain (8-2018-q4 (https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads) at the time of writing) includes gdb 8.0.x 8.2.50.20181213-git, which... doesn't work with our scripts (e.g. EOSM/debugmsg.gdb)
- guess what - gdb 8.2.1 "stable" is not working either! Why?! I remember testing 8.1 successfully before...
- back to gdb 8.1 from source -> working! Regression in GDB?!
- previous precompiled toolchain (7-2018-q2-update) uses gdb 8.1.0.20180315-git, which IS WORKING with EOSM/debugmsg.gdb, so I'll just use that in the install script.

Minor wrinkles:
- user has to modify PATH manually (can be solved, see e.g. this PR (https://github.com/mitsuhiko/pipsi/pull/148) for another project)
- need to allow Terminal to interact with System Events or something like that

Mac quirks:
- how do you type a tilde on Mac?! (OK, found it at the end of a google search)
- why is scrolling in the opposite direction?! (and why it's asking me for a USB mouse in order to change this setting?!)
- how do you open a new terminal?! (clicking the icon just activates the old one)
- how do you navigate to your Home directory? how do you cut & paste a file? and so on...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Danne on April 10, 2019, 09:27:05 AM
Quote from: a1ex on April 10, 2019, 09:11:36 AM

Mac quirks:
- how do you open a new terminal?! (clicking the icon just activates the old one)
- how do you navigate to your Home directory? how do you cut & paste a file? and so on...

- cmd + n for new terminal window while having terminal up front
- Cut not working but cmd + c(copy) then cmd + v(paste)
- Shift + cmd + g(will open 'Go to the folder' field) then paste tilde sign and enter

Other stuff:
- shift + cmd + .(punctuation) will unhide all hidden folders. Run the same command to go back to hidden mode.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: calle2010 on April 10, 2019, 03:55:17 PM
Quote from: a1ex on April 10, 2019, 09:11:36 AM
- why is scrolling in the opposite direction?!

Because this direction is the same as the direction you would use on a touchpad with a two-finger-gesture to move the screen content. See the scroll wheel as a primitive one-dimensional touchpad.

The Windows scroll wheel direction comes from the "I click&drag a scrollbar to move the content" times (1990s or so :) ). The scrollbar moves into the opposite direction (down if the content moves up).
Fortunately my Logitech mouse on Windows can change this weird Windows quirk.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: dfort on April 10, 2019, 04:05:39 PM
Quote from: a1ex on April 10, 2019, 09:11:36 AM
- how do you navigate to your Home directory?

All of these work on the Mac:

cd
cd ~
cd ~/
cd $HOME
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 10, 2019, 10:00:54 PM
I mean, the Mac has a reputation of being easy to use, right? I get all sorts of animated popups, OK, maybe that's what users like, but... for pretty much every single basic thing (like moving a file, or going to the Home directory in the GUI, or... typing the ~ character) I need to google how to do it. And it's not the first time I've used a Mac.

Win10 has its own share of quirks (e.g. the WSL home directory is hidden quite deep in the host filesystem), but not nearly as many.

Good news - precompiled toolchain 7-2018-q2-update appears to work fine! The installation script now defaults to that on both Mac and WSL.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: scrax on April 10, 2019, 10:07:12 PM
Quote from: a1ex on April 10, 2019, 10:00:54 PM
I mean, the Mac has a reputation of being easy to use, right? I get all sorts of animated popups, OK, maybe that's what users like, but... for pretty much every single basic thing (like moving a file, or going to the Home directory in the GUI, or... typing the ~ character) I need to google how to do it. And it's not the first time I've used a Mac.

Win10 has its own share of quirks (e.g. the WSL home directory is hidden quite deep in the host filesystem), but not nearly as many.

Good news - precompiled toolchain 7-2018-q2-update appears to work fine! The installation script now defaults to that on both Mac and WSL.
It's easy for who never used a PC before :P
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: scrax on April 21, 2019, 11:59:00 AM
I don't know if it something that need attention or not, but when emulating 600D if I press L (for live view) i got this:

   163:  8499.968 [MC] PROP_GUI_STATE 0
   164:  8499.968 [PRP] ERROR TWICE ACK REQUEST L:846
   165:  8499.968 [PRP] this->dwWaitAckID = 0x80020000(0x80040004)
   166:  8499.968 [LV] [LVAE] EP_SetControlBv() >> EP_ControlBv:1
   167:  8500.992 WARN [LVDS] First Get DTS_GetAllRandomData
   168:  8501.248 [LV] [PATH] GetPathDriveInfo[0]
   169:  8501.248 WARN [LVDS] First Get DTS_GetAllRandomData
   170:  8501.760 WARN [LVDS] First Get DTS_GetAllRandomData
   171:  8502.272 WARN [LVDS] First Get DTS_GetAllRandomData
   172:  8502.272 WARN [LVDS] First Get DTS_GetAllRandomData
   173:  8502.528 [MC] cam event guimode comp. 0
   175:  8506.880 [GUI] ERROR ***** Lv GetMovieFrameRateIcon S (81)
   176:  8507.136 [GUI] ERROR ***** Lv GetMovieZoomIcon S (88)
   177:  8507.136 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   178:  8508.672 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   179:  8508.672 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   180:  8508.928 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   181:  8508.928 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   182:  8510.208 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   183:  8510.208 [DISP] UpdateReverseTFT(off) Current=1 Target=1
   184:  8558.336 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   185:  8558.336 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   186:  8558.592 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   187:  8558.592 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   188:  8558.848 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   189:  8616.704 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   190:  8616.704 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   191:  8616.704 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
   192:  8616.704 [GUI] ERROR ***** Lv IsMovieZoomSetting(88)
... this Lv IsMovieZoomSetting(88) continues till I get out of LV...


(I'm using last qemu branch with python3 and new toolchain)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kitor on April 23, 2019, 04:37:56 PM
I installed QEMU from qemu branch, however I don't see any definitions for M50 or R here. What do I need to do to emulate those?
Any hacks in arm-softmmu/hw/eos, or I should use some other camera for now?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on April 23, 2019, 04:39:43 PM
There is a patch for M50 here (https://www.magiclantern.fm/forum/index.php?topic=23296.msg210088#msg210088); IIRC it worked on R with minimal changes. Will clean them up for committing, but I think it will happen after bringing the current state into mainline.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: heder on May 28, 2019, 03:51:49 PM
Hi.

I'm trying to get QEMU running with my 500D rom on Ubuntu 14.04 but I get only white noise ...

Branch: "qemu"
HOST: QEMU (installed using contrib/qemu/install.sh)
TARGET: Canon 500D
GCC: 5.4.1

Inorder to get the 500D platform to compile I had to change "static void my_gui_main_task(void)" into "void ml_gui_main_task(void)" in platform/500D.111/gui.c as 500D is still using it's own gui.c. After that it compiles. Then I ran ./run_ml_500D.sh

Running the firmware does however only give me white noise in QEMU. I'v noticed in the log that ml_init_task is not started and I rather get the standard init task. Somehow the hijack in boot-hack is not working. I also defined CONFIG_HELLO_WORLD, but nothing happens.

Here is my log.


make: Entering directory `/home/ml/qemu/qemu-1.6.0'
CHK version_gen.h
make: Leaving directory `/home/ml/qemu/qemu-1.6.0'
make: Entering directory `/home/ml/magic-lantern/platform/500D.111'
[ VERSION  ]   ../../platform/500D.111/version.bin
[ VERSION  ]   ../../platform/500D.111/version.c
[ CC       ]   version.o
make -C ../../tcc
make[1]: Entering directory `/home/ml/magic-lantern/tcc'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/ml/magic-lantern/tcc'
[ LD       ]   magiclantern
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 473052 bytes
[ CC       ]   reboot.o
../../src/reboot.c: In function 'cstart':
../../src/reboot.c:100:6: warning: #warning Signature Checking bypassed!! Please use a proper signature [-Wcpp]
     #warning Signature Checking bypassed!! Please use a proper signature
      ^
[ LD       ]   autoexec

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x0738d4 0x000c07d4 0x000c07d4 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x0004d000 0x0004d000 0x737dc 0x86ccc RWE 0x100
[ OBJCOPY  ]   autoexec.bin
[ STAT     ]   autoexec.bin
autoexec.bin: 473552 bytes
[ SYMBOLS  ]   magiclantern.sym
[ CP       ]   500D_111.sym
make: Leaving directory `/home/ml/magic-lantern/platform/500D.111'
make: Entering directory `/home/ml/magic-lantern/platform/500D.111'
make: `qemu-helper.bin' is up to date.
make: Leaving directory `/home/ml/magic-lantern/platform/500D.111'
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 3FFFFFFF: eos.ram
40001000 - 7FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror_F1
F2000000 - F2FFFFFF: eos.rom0_mirror_F2
F3000000 - F3FFFFFF: eos.rom0_mirror_F3
F4000000 - F4FFFFFF: eos.rom0_mirror_F4
F5000000 - F5FFFFFF: eos.rom0_mirror_F5
F6000000 - F6FFFFFF: eos.rom0_mirror_F6
F7000000 - F7FFFFFF: eos.rom0_mirror_F7
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror_F9
FA000000 - FAFFFFFF: eos.rom1_mirror_FA
FB000000 - FBFFFFFF: eos.rom1_mirror_FB
FC000000 - FCFFFFFF: eos.rom1_mirror_FC
FD000000 - FDFFFFFF: eos.rom1_mirror_FD
FE000000 - FEFFFFFF: eos.rom1_mirror_FE
FF000000 - FFFFFFFF: eos.rom1_mirror_FF
C0000000 - CFFFFFFF: eos.iomem
[EOS] loading 'ROM-500D.BIN' to 0xF0000000-0xF1FFFFFF
[EOS] loading 'ROM-500D.BIN' to 0xF8000000-0xF9FFFFFF
[EOS] loading 'autoexec.bin' to 0x00800000-0x008739CF
[EOS] loading 'qemu-helper.bin' to 0x30000000-0x30008C8F
[QEMU_HELPER] stub ff066a98 -> 300000a0 (e92d000f)
[QEMU_HELPER] stub ff197378 -> 3000072c (e92d41f0)
[QEMU_HELPER] stub ff1974a4 -> 30000784 (e92d41f0)
[QEMU_HELPER] stub ff1975a4 -> 300000f4 (e92d4070)
[QEMU_HELPER] stub ff196a88 -> 30000224 (e92d4010)
[QEMU_HELPER] stub ff1963ec -> 30000118 (e92d4070)
[QEMU_HELPER] stub ff196338 -> 3000026c (e92d41f0)
[QEMU_HELPER] stub ff196724 -> 30000328 (e92d41f0)
[QEMU_HELPER] stub ff1968d4 -> 30000148 (e92d41f0)
[QEMU_HELPER] stub ff196494 -> 300001a8 (e92d4070)
[QEMU_HELPER] stub ff1967d4 -> 30000820 (e92d43f8)
[QEMU_HELPER] stub ff059868 -> 30000050 (e92d4ff8)
[QEMU_HELPER] stub ff059988 -> 3000006c (e92d43f8)
[QEMU_HELPER] stub ff19384c -> 30000088 (e92d47f0)
[GPIO] at [0x00873910] [0x00000046] -> [0xC0220134]
[FlashIF] at [0x000C0920]: 'Write enable' enabled
[???] [0x00000001] -> [0xC020010C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020000C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020001C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020002C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020003C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020004C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020005C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020006C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020007C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020008C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC020009C] PC: 0x000C0920
[???] [0x000000FF] -> [0xC02000AC] PC: 0x000C0920
[???] [0x000000FF] -> [0xC02000BC] PC: 0x000C0920
[???] [0x000000FF] -> [0xC02000CC] PC: 0x000C0920
[???] [0x000000FF] -> [0xC02000DC] PC: 0x000C0920
[???] [0x000000FF] -> [0xC02000EC] PC: 0x000C0920
[???] [0x000000FF] -> [0xC02000FC] PC: 0x000C0920
[Basic] at [0x000C0920] [0x00430005] -> [0xC0400008]
[???] [0x00000000] <- [0xC0242010] PC: 0x000C0920
[???] [0x00000001] -> [0xC0242010] PC: 0x000C0920
[Int] Write to Int space [0x43210DCB] -> [0xC0201100] PC: [0xFF0128D8]
[Int] Write to Int space [0xFEA98765] -> [0xC0201104] PC: [0xFF0128D8]
[Int] Write to Int space [0x00000001] -> [0xC0201200] PC: [0xFF0128D8]
[???] [0x00000008] -> [0xC0203008] PC: 0xFF0129A8
[Int] Enabled interrupt ID 0x0A PC: [0xFF012928]
[Basic] at [0xFF010370] [0x00000000] -> [0xC0400018]
[Basic] at [0xFF012A64] [0x00000000] <- [0xC0400008]
[Basic] at [0xFF012A64] [0x00001000] -> [0xC0400008]
[Basic] at [0xFF010380] [0x00000000] <- [0xC0400000]
[Timer] at [0xFF0129E8] [0x80000000] -> [0xC0210200]
[Timer] at [0xFF012A08] [0x00000002] -> [0xC0210204]
[Timer] at [0xFF012A08] [0x00000003] -> [0xC0210214]
[Timer] at [0xFF0103B0] [0x0000270F] -> [0xC0210208]
[Timer] at [0xFF012A40] [0x00000001] -> [0xC0210210]
[Timer] at [0xFF012A40] Starting triggering
[EOS] trigger int 0x0A (delayed!)
[Timer] at [0xFF012A40] [0x00000001] -> [0xC0210200]
[Int] Enabled interrupt ID 0x2E PC: [0xFF012908]
[Int] Enabled interrupt ID 0x3A PC: [0xFF012908]
[???] [0x00000000] <- [0xC05000D0] PC: 0xFF012418
[???] [0x00000000] -> [0xC05000D0] PC: 0xFF012418
[???] [0x40024680] -> [0xC05000C0] PC: 0xFF012418
[???] [0x00000080] -> [0xC05000C4] PC: 0xFF012418
[???] [0x40024680] -> [0xC05000C8] PC: 0xFF012418
[???] [0x00000080] -> [0xC05000CC] PC: 0xFF012418
[???] [0x00000000] -> [0xC05000D4] PC: 0xFF012418
[???] [0x00000017] -> [0xC05000D8] PC: 0xFF012418
[???] [0x00010023] -> [0xC05000D0] PC: 0xFF012418
[TIO] Reset RX indicator
K252 READY
[GPIO] at [0xFF01321C] [0x00000010] -> [0xC0222004]
[GPIO] at [0xFF0638E4] [0x00800C00] -> [0xC022D074]
[GPIO] at [0xFF0638E4] [0x00000000] -> [0xC0221110]
[GPIO] at [0xFF0638E4] [0x00000050] -> [0xC0221104]
[GPIO] ERASE SW OFF read at [0xFF063910]
[GPIO] at [0xFF063910] [0x00000000] -> [0xC022110C]
[GPIO] at [0xFF063924] [0x0000001F] -> [0xC0221200]
[GPIO] at [0xFF063930] [0x0000000E] -> [0xC0221204]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221208]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC022120C]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221210]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221214]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221218]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC022121C]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221220]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221224]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221228]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC022122C]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221230]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221234]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221238]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC022123C]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221240]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221244]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221248]
[GPIO] at [0xFF063930] [0x00000006] -> [0xC022124C]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221250]
[GPIO] at [0xFF063930] [0x0000001D] -> [0xC0221254]
[GPIO] at [0xFF063930] [0x00000034] -> [0xC0221258]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC022125C]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221260]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221264]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221268]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC022126C]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221270]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221274]
[GPIO] at [0xFF063930] [0x0000001F] -> [0xC0221278]
[GPIO] at [0xFF063944] [0x00000002] -> [0xC0221000]
[GPIO] at [0xFF063944] [0x00000048] -> [0xC0220000]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC0220004]
[GPIO] at [0xFF063968] [0x00000046] -> [0xC0220008]
[GPIO] at [0xFF063968] [0x00000008] -> [0xC022000C]
[GPIO] at [0xFF063954] [0x0000001F] -> [0xC0221004]
[GPIO] at [0xFF063954] [0x00000044] -> [0xC0220010]
[GPIO] /VSW_ON read at [0xFF063968]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC0220018]
[GPIO] at [0xFF063968] [0x00000000] -> [0xC022001C]
[GPIO] at [0xFF063968] [0x00000001] -> [0xC0221008]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC0220020]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC0220024]
[GPIO] at [0xFF063968] [0x00000046] -> [0xC0220028]
[GPIO] at [0xFF063968] [0x00000028] -> [0xC022002C]
[GPIO] at [0xFF063968] [0x0000001C] -> [0xC022100C]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC0220030]
[GPIO] USB CONNECT read at [0xFF063968]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC0220038]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC022003C]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC0221010]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC0220040]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC0220044]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC0220048]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC022004C]
[GPIO] /VSW_ON read at [0xFF063968]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC0220050]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC0220054]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC0220058]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC022005C]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC0221018]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC0220060]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC0220064]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC0220068]
[GPIO] at [0xFF063968] [0x00000004] -> [0xC022006C]
[GPIO] at [0xFF063968] [0x00000008] -> [0xC022101C]
[GPIO] VIDEO CONNECT read at [0xFF063968]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC0220074]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC0220078]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC022007C]
[GPIO] at [0xFF063968] [0x00000003] -> [0xC0221020]
[GPIO] at [0xFF063968] [0x00000000] -> [0xC0220080]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC0220084]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC0220088]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC022008C]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC0221024]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC0220090]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC0220094]
[GPIO] at [0xFF063968] [0x00000046] -> [0xC0220098]
[GPIO] at [0xFF063968] [0x00000046] -> [0xC022009C]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC0221028]
[GPIO] at [0xFF063968] [0x00000020] -> [0xC02200A0]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC02200A4]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC02200A8]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC02200AC]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC022102C]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC02200B0]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC02200B4]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC02200B8]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC02200BC]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC0221030]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC02200C0]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC02200C4]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC02200C8]
[GPIO] at [0xFF063968] [0x00000000] -> [0xC02200CC]
[GPIO] USB CONNECT read at [0xFF063968]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC02200D0]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC02200D4]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC02200D8]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC02200DC]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC0221038]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC02200E0]
[GPIO] at [0xFF063968] [0x00000040] -> [0xC02200E4]
[GPIO] MIC CONNECT read at [0xFF063968]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC02200EC]
[GPIO] at [0xFF063968] [0x00000015] -> [0xC022103C]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC02200F0]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC02200F4]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC02200F8]
[GPIO] at [0xFF063968] [0x00000048] -> [0xC02200FC]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC0221040]
[GPIO] at [0xFF063968] [0x00000046] -> [0xC0220100]
[GPIO] at [0xFF063968] [0x00000000] -> [0xC0220104]
[GPIO] ERASE SW OFF read at [0xFF063968]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC022010C]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC0221044]
[GPIO] at [0xFF063968] [0x00000046] -> [0xC0220110]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC0220114]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC0220118]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC022011C]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC0221048]
[GPIO] at [0xFF063968] [0x00000046] -> [0xC0220120]
[GPIO] at [0xFF063968] [0x00000046] -> [0xC0220124]
[RTC] CS reset at [0xFF063968]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC0220128]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC022012C]
[GPIO] at [0xFF063968] [0x0000001F] -> [0xC022104C]
[GPIO] at [0xFF063968] [0x00000046] -> [0xC0220130]
[GPIO] at [0xFF063968] [0x00000044] -> [0xC0220134]
[GPIO] HDMI CONNECT read at [0xFF063968]
[GPIO] at [0xFF063968] [0x00000020] -> [0xC022013C]
[GPIO] at [0xFF06398C] [0x00000025] -> [0xC0221300]
[???] [0x00000004] -> [0xC020302C] PC: 0xFF06398C
[GPIO] at [0xFF0639A0] [0x00000024] -> [0xC0221304]
[???] [0x00000004] -> [0xC0203030] PC: 0xFF0639A0
[GPIO] at [0xFF0639A0] [0x00000007] -> [0xC0221308]
[GPIO] at [0xFF0639A0] [0x0000004F] -> [0xC022130C]
[???] [0x00000004] -> [0xC0203038] PC: 0xFF0639A0
[GPIO] at [0xFF0639A0] [0x0000004E] -> [0xC0221310]
[GPIO] at [0xFF0639A0] [0x0000001E] -> [0xC0221314]
[GPIO] at [0xFF0639A0] [0x0000001C] -> [0xC0221318]
[???] [0x00000004] -> [0xC0203044] PC: 0xFF0639A0
[GPIO] at [0xFF0639A0] [0x00000041] -> [0xC022131C]
[Int] Write to Int space [0x43210DCB] -> [0xC0201100] PC: [0xFF0128D8]
[Int] Write to Int space [0xFEA98765] -> [0xC0201104] PC: [0xFF0128D8]
[Int] Write to Int space [0x00000001] -> [0xC0201200] PC: [0xFF0128D8]
[???] [0x00000000] -> [0xC0243110] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243214] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243214] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC024321C] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC024321C] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC0243120] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243224] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243224] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC024322C] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC024322C] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC0243130] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243234] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243234] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC024323C] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC024323C] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC0243140] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243244] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243244] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC024324C] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC024324C] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC0243150] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243254] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243254] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC024325C] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC024325C] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC0243160] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243264] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243264] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC024326C] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC024326C] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC0243170] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243274] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243274] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC024327C] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC024327C] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC0243180] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243284] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243284] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC024328C] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC024328C] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC0243190] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243294] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243294] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC024329C] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC024329C] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC02431A0] PC: 0xFF19F098
[???] [0x00000000] -> [0xC02432A4] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC02432A4] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC02432AC] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC02432AC] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC02431B0] PC: 0xFF19F098
[???] [0x00000000] -> [0xC02432B4] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC02432B4] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC02432BC] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC02432BC] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC02431C0] PC: 0xFF19F098
[???] [0x00000000] -> [0xC02432C4] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC02432C4] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC02432CC] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC02432CC] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC02431D0] PC: 0xFF19F098
[???] [0x00000000] -> [0xC02432D4] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC02432D4] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC02432DC] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC02432DC] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC02431E0] PC: 0xFF19F098
[???] [0x00000000] -> [0xC02432E4] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC02432E4] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC02432EC] PC: 0xFF19F0FC
[???] [0x00000000] <- [0xC02432EC] PC: 0xFF19F0FC
[???] [0x00000000] -> [0xC0202000] PC: 0xFF19F138
[???] [0x00000000] -> [0xC0242200] PC: 0xFF19F160
[???] [0x00000000] -> [0xC0242204] PC: 0xFF19F170
[???] [0x00000000] -> [0xC0202004] PC: 0xFF19F138
[???] [0x00000000] -> [0xC0242210] PC: 0xFF19F160
[???] [0x00000000] -> [0xC0242214] PC: 0xFF19F170
[???] [0x00000000] -> [0xC0202008] PC: 0xFF19F138
[???] [0x00000000] -> [0xC0242220] PC: 0xFF19F160
[???] [0x00000000] -> [0xC0242224] PC: 0xFF19F170
[???] [0x00000000] -> [0xC020200C] PC: 0xFF19F138
[???] [0x00000000] -> [0xC0242230] PC: 0xFF19F160
[???] [0x00000000] -> [0xC0242234] PC: 0xFF19F170
[???] [0x00000000] -> [0xC0202010] PC: 0xFF19F138
[???] [0x00000000] -> [0xC0242240] PC: 0xFF19F160
[???] [0x00000000] -> [0xC0242244] PC: 0xFF19F170
[???] [0x00000000] -> [0xC0202014] PC: 0xFF19F138
[???] [0x00000000] -> [0xC0242250] PC: 0xFF19F160
[???] [0x00000000] -> [0xC0242254] PC: 0xFF19F170
[???] [0x00000000] -> [0xC0202018] PC: 0xFF19F138
[???] [0x00000000] -> [0xC0242260] PC: 0xFF19F160
[???] [0x00000000] -> [0xC0242264] PC: 0xFF19F170
[???] [0x00000000] -> [0xC020201C] PC: 0xFF19F138
[???] [0x00000000] -> [0xC0242270] PC: 0xFF19F160
[???] [0x00000000] -> [0xC0242274] PC: 0xFF19F170
[???] [0x00000000] -> [0xC0243700] PC: 0xFF19F200
[???] [0x00000000] -> [0xC0243710] PC: 0xFF19F200
[???] [0x00000001] -> [0xC024311C] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC024312C] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC024313C] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC024314C] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC024315C] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC024316C] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC024317C] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC024318C] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC024319C] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC02431AC] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC02431BC] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC02431CC] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC02431DC] PC: 0xFF19F1A4
[???] [0x00000001] -> [0xC02431EC] PC: 0xFF19F1A4
[???] [0x00000000] -> [0xC0243800] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243804] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243808] PC: 0xFF19F214
[???] [0x00000000] -> [0xC024380C] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243810] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243814] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243818] PC: 0xFF19F214
[???] [0x00000000] -> [0xC024381C] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243820] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243824] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243828] PC: 0xFF19F214
[???] [0x00000000] -> [0xC024382C] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243830] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243834] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243838] PC: 0xFF19F214
[???] [0x00000000] -> [0xC024383C] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243840] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243844] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243848] PC: 0xFF19F214
[???] [0x00000000] -> [0xC024384C] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243850] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243854] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243858] PC: 0xFF19F214
[???] [0x00000000] -> [0xC024385C] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243860] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243864] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243868] PC: 0xFF19F214
[???] [0x00000000] -> [0xC024386C] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243870] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243874] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243878] PC: 0xFF19F214
[???] [0x00000000] -> [0xC024387C] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243880] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243884] PC: 0xFF19F214
[???] [0x00000000] -> [0xC0243888] PC: 0xFF19F214
[???] [0x00000000] -> [0xC024388C] PC: 0xFF19F214
[Int] Enabled interrupt ID 0x1A PC: [0xFF012908]
[Int] Enabled interrupt ID 0x1B PC: [0xFF012908]
[Int] Enabled interrupt ID 0x1C PC: [0xFF012908]
[Int] Enabled interrupt ID 0x1D PC: [0xFF012908]
[Int] Enabled interrupt ID 0x1E PC: [0xFF012908]
[Int] Enabled interrupt ID 0x1F PC: [0xFF012908]
[Int] Enabled interrupt ID 0x10 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x20 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x21 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x22 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x23 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x24 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x25 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x26 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x27 PC: [0xFF012908]
[???] [0x00000000] -> [0xC0243160] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243264] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243264] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC0243170] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243274] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243274] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC0243180] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243284] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243284] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC0243190] PC: 0xFF19F098
[???] [0x00000000] -> [0xC0243294] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC0243294] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC02431A0] PC: 0xFF19F098
[???] [0x00000000] -> [0xC02432A4] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC02432A4] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC02431B0] PC: 0xFF19F098
[???] [0x00000000] -> [0xC02432B4] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC02432B4] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC02431C0] PC: 0xFF19F098
[???] [0x00000000] -> [0xC02432C4] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC02432C4] PC: 0xFF19F0E8
[???] [0x00000000] -> [0xC02431D0] PC: 0xFF19F098
[???] [0x00000000] -> [0xC02432D4] PC: 0xFF19F0E8
[???] [0x00000000] <- [0xC02432D4] PC: 0xFF19F0E8
[???] [0x40000401] -> [0xC0242010] PC: 0xFF19F000
[Basic] at [0xFF06DAB8] [0x00000000] <- [0xC0400008]
[Basic] at [0xFF06DAB8] [0x00200000] -> [0xC0400008]
[Basic] at [0xFF06E18C] [0x00000000] <- [0xC0400008]
[Basic] at [0xFF06E18C] [0x01000000] -> [0xC0400008]
[DMA1] [0x00000001] -> [0xC0A10000]
[DMA2] [0x00000001] -> [0xC0A20000]
[DMA3] [0x00000001] -> [0xC0A30000]
[DMA4] [0x00000001] -> [0xC0A40000]
[DebugMsg] (0,1) [PM] DisablePowerSave (Counter = 1)
[DebugMsg] (139,22)
K252 ICU Firmware Version 1.1.1 ( 3.6.4 )
[DebugMsg] (139,5)
ICU Release DateTime 2011.03.04 10:18:15
[DebugMsg] (0,3) [SEQ] CreateSequencer (Startup, Num = 6)
[DebugMsg] (0,2) [SEQ] NotifyComplete (Startup, Flag = 0x10000)
[DebugMsg] (0,3) [SEQ] NotifyComplete (Cur = 0, 0x10000, Flag = 0x10000)
[DebugMsg] (50,3) Magic Lantern Nightly.2019May28.500D111 (a8a501d5773e+ (qemu))
[DebugMsg] (50,3) Built on 2019-05-28 13:05:28 UTC by ml@ml-pc
BMP buffer (LCD): raw=40302f6c hdmi=40303008 lcd=4030a100 real=4030a100 idle=4038a100

Open Console K252[1]>...

K252[1]>[DebugMsg] (0,5) [SEQ] seqEventDispatch (Startup, 0)
[GPIO] at [0xFF013774] [0x00000012] -> [0xC0222004]
[DebugMsg] (139,5) startupEntry
[GPIO] at [0xFF013580] [0x00000012] -> [0xC0222000]
[DMA1] [0x00000000] -> [0xC0A10004]
[DMA1] [0x00000000] -> [0xC0A10010]
[DMA1] [0xF8A00000] -> [0xC0A10018]
[DMA1] [0x40405B00] -> [0xC0A1001C]
[DMA1] [0x00196794] -> [0xC0A10020]
[DMA1] [0x00000007] -> [0xC0A10014]
[Int] Enabled interrupt ID 0x2F PC: [0xFF012908]
[DMA1] Copy [0xF8A00000] -> [0x40405B00], length [0x00196794], flags [0x00030001]
[DMA1] OK
[EOS] trigger int 0x2F
[Int] Requested int reason [0x000000BC] <- [0xC0201004] PC: [0x00000500]
[DMA1] [0x00000000] -> [0xC0A10010]
[DebugMsg] (2,6) SearchFromProperty DataType = 0x00000000(L:1581)
ASSERT : Startup\Startup.c, Task = Startup, Line 334
[DebugMsg] (139,6) ASSERT : Startup\Startup.c, Task = Startup
[DebugMsg] (139,6) ASSERT : Line 334
[DebugMsg] (139,6) ASSERT : err == SUCCESS
*** _prop_request_change(80030002)
[DebugMsg] (139,5) startupPropAdminMain : End
[DebugMsg] (0,2) [SEQ] NotifyComplete (Startup, Flag = 0x20000000)
[DebugMsg] (0,3) [SEQ] NotifyComplete (Cur = 1, 0x20000002, Flag = 0x20000000)
[GPIO] at [0xFF013640] [0x00000010] -> [0xC0222000]
[DMA1] [0x00000000] -> [0xC0A10004]
[DMA1] [0x00000000] -> [0xC0A10010]
[DMA1] [0xF8910000] -> [0xC0A10018]
[DMA1] [0x406C5D00] -> [0xC0A1001C]
[DMA1] [0x000323DC] -> [0xC0A10020]
[DMA1] [0x00000007] -> [0xC0A10014]
[DMA1] Copy [0xF8910000] -> [0x406C5D00], length [0x000323DC], flags [0x00030001]
[DMA1] OK
[EOS] trigger int 0x2F
[Int] Requested int reason [0x000000BC] <- [0xC0201004] PC: [0x00000500]
[DMA1] [0x00000000] -> [0xC0A10010]
[DMA1] [0x00000000] -> [0xC0A10004]
[DMA1] [0x00000000] -> [0xC0A10010]
[DMA1] [0xF88F7000] -> [0xC0A10018]
[DMA1] [0x40705F00] -> [0xC0A1001C]
[DMA1] [0x00000864] -> [0xC0A10020]
[DMA1] [0x00000007] -> [0xC0A10014]
[DMA1] Copy [0xF88F7000] -> [0x40705F00], length [0x00000864], flags [0x00030001]
[DMA1] OK
[EOS] trigger int 0x2F
[Int] Requested int reason [0x000000BC] <- [0xC0201004] PC: [0x00000500]
[DMA1] [0x00000000] -> [0xC0A10010]
[DMA1] [0x00000000] -> [0xC0A10004]
[DMA1] [0x00000000] -> [0xC0A10010]
[DMA1] [0xF89B0000] -> [0xC0A10018]
[DMA1] [0x40707100] -> [0xC0A1001C]
[DMA1] [0x0000F3BC] -> [0xC0A10020]
[DMA1] [0x00000007] -> [0xC0A10014]
[DMA1] Copy [0xF89B0000] -> [0x40707100], length [0x0000F3BC], flags [0x00030001]
[DMA1] OK
[EOS] trigger int 0x2F
[Int] Requested int reason [0x000000BC] <- [0xC0201004] PC: [0x00000500]
[DMA1] [0x00000000] -> [0xC0A10010]
[DMA1] [0x00000000] -> [0xC0A10004]
[DMA1] [0x00000000] -> [0xC0A10010]
[DMA1] [0xF8EB0000] -> [0xC0A10018]
[DMA1] [0x40727200] -> [0xC0A1001C]
[DMA1] [0x00008DDC] -> [0xC0A10020]
[DMA1] [0x00000007] -> [0xC0A10014]
[DMA1] Copy [0xF8EB0000] -> [0x40727200], length [0x00008DDC], flags [0x00030001]
[DMA1] OK
[EOS] trigger int 0x2F
[Int] Requested int reason [0x000000BC] <- [0xC0201004] PC: [0x00000500]
[DMA1] [0x00000000] -> [0xC0A10010]
[Basic] at [0xFF18A804] [0x00000000] <- [0xC0400008]
[Basic] at [0xFF18A804] [0x00200000] -> [0xC0400008]
[???] [0x0000000C] -> [0xC020302C] PC: 0xFF18A804
[Int] Enabled interrupt ID 0x50 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x36 PC: [0xFF012908]
[DebugMsg] (0,1) [PM] DisablePowerSave (Counter = 2)
[DebugMsg] (0,1) [PM] EnablePowerSave (Counter = 1)
[GPIO] at [0xFF18A904] [0x00000001] <- [0xC022009C]
[GPIO] at [0xFF18A904] [0x00000001] -> [0xC022009C]
[GPIO] VIDEO CONNECT read at [0xFF01DFE4]
[???] [0x0000001C] -> [0xC0203044] PC: 0xFF01E000
[GPIO] USB CONNECT read at [0xFF01E000]
[???] [0x00000018] -> [0xC020301C] PC: 0xFF01E000
[Int] Enabled interrupt ID 0x56 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x44 PC: [0xFF012908]
[Int] Enabled interrupt ID 0x54 PC: [0xFF012908]
[DebugMsg] (138,3) CreateTask Master End
*** prop_register_slave(12fae0)
[GPIO] VIDEO CONNECT read at [0xFF01DCCC]
[GPIO] USB CONNECT read at [0xFF01DD78]
[GPIO] HDMI CONNECT read at [0xFF01DDD8]
[GPIO] VIDEO CONNECT read at [0xFF01DCE0]
[GPIO] USB CONNECT read at [0xFF01DCF0]
[GPIO] HDMI CONNECT read at [0xFF01DCF0]
[GPIO] VIDEO CONNECT read at [0xFF01DE58]
[GPIO] USB CONNECT read at [0xFF01DE58]
[GPIO] HDMI CONNECT read at [0xFF01DE58]
[GPIO] VIDEO CONNECT read at [0xFF01DE58]
[GPIO] USB CONNECT read at [0xFF01DE58]
[GPIO] HDMI CONNECT read at [0xFF01DE58]
[GPIO] VIDEO CONNECT read at [0xFF01DE58]


Any clues to what is wrong ?

Title: Re: How to run Magic Lantern into QEMU?!...
Post by: heder on May 30, 2019, 12:12:14 AM
Quote from: heder on May 28, 2019, 03:51:49 PM
Hi.

I'm trying to get QEMU running with my 500D rom on Ubuntu 14.04 but I get only white noise ...
...
Any clues to what is wrong ?

:o

I found the issue myself tonight, I was using the old qemu system. After "make 500D.111 install_qemu" it's working.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Jip-Hop on July 09, 2019, 11:30:58 AM
Quote from: a1ex on April 10, 2019, 09:11:36 AM
Installed Mojave, downloaded the zip archive of the QEMU branch (from here (https://bitbucket.org/hudson/magic-lantern/downloads/?tab=branches)) with Safari (i.e. without first installing hg & co), ran the install script (cd contrib/qemu; ./install.sh) and... worked out of the box! It installed brew and other dependencies, compiled gdb 8.2.1 from source et voilà! QEMU and ML development environment installed and ready to use!

I'm following these instructions to get QEMU working with EOSM, but I'm stuck.
If I run ./run_canon_fw.sh EOSM,firmware=boot=1 I get this message:
Could not open ./EOSM/SFDATA.BIN

So I need to get my hands on a compiled version of the sf_dump module.
But when I run make in this folder:
/magic-lantern/modules/sf_dump

I get this output:

abort: no repository found in '/magic-lantern/modules/sf_dump' (.hg not found)!

(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x10dca43f8>)
[ CC       ]   sf_dump.o
In file included from sf_dump.c:3:0:
../../src/module.h:344:10: fatal error: module_strings.h: No such file or directory
#include "module_strings.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [sf_dump.o] Error 1

Where can I find a compiled version of this module?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 09, 2019, 01:47:54 PM
The portable ROM dumper (https://www.magiclantern.fm/forum/index.php?topic=16534.0) should do the trick (I should update the docs).

The error means you should clone the repository (with hg clone), rather than just downloading the sources. Reason: the compiled modules (i.e. *.mo files) also include some version info, and any source code changes since last commit.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Jip-Hop on July 09, 2019, 06:11:05 PM
Thanks a1ex! Used portable rom dumper, got the SFDATA.BIN file now.
What should I do next to get GDB working?

./run_canon_fw.sh EOSM,firmware="boot=0" -s -S & arm-none-eabi-gdb -x EOSM/patches.gdb -ex quit
[1] 1311
-bash: arm-none-eabi-gdb: command not found


QEMU starts but all I see is a grey screen.
This is the terminal output:

./run_canon_fw.sh EOSM,firmware=boot=0 -s -S &

DebugMsg=0x40D4 (from GDB script)
Lockdown read 4
Lockdown read 4
Lockdown read 5
Lockdown read 5
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 0FFFFFFF: eos.ram
40001000 - 4FFFFFFF: eos.ram_uncached
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - DFFFFFFF: eos.mmio
[EOS] loading './EOSM/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[EOS] loading './EOSM/SFDATA.BIN' as serial flash, size=0x800000
[MPU] warning: non-empty spell #30 (PROP 80030019) has duplicate(s): #34
[MPU] warning: non-empty spell #40 (PROP_VIDEO_MODE) has duplicate(s): #41

[MPU] Available keys:
- Arrow keys   : Navigation
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess (press only)
- M            : MENU (press only)
- P            : PLAY (press only)
- I            : INFO/DISP (press only)
- Q            : guess
- L            : LiveView (press only)
- Z/X          : Zoom in/out
- Shift        : Half-shutter
- 0/9          : Mode dial (press only)
- V            : Movie mode (press only)
- B            : Open battery door
- C            : Open card door
- F10          : Power down switch
- F1           : show this help

Setting BOOTDISK flag to 0


Seems like GDB didn't install at all:
-bash: gdb: command not found
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 09, 2019, 06:21:30 PM
If the installer asked you to type some PATH commands during installation, and you closed the terminal since then, you will have to type them again. The installation doesn't start unless you have a valid arm-none-eabi-gdb installed.

Normally, these paths should be added to your Bash profile, but I didn't want to mess with user's config files.

TODO: add this (https://github.com/mitsuhiko/pipsi/pull/148) to the install script.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Jip-Hop on July 09, 2019, 08:54:30 PM
Aah yes that was it!

If I try to run Canon firmware, following this guide (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/#rst-header-opening-the-battery-door), I still get a mostly grey screen. It also has a grey circle on the bottom right now and some black lines on the top left.

./run_canon_fw.sh EOSM,firmware="boot=0" -s -S & arm-none-eabi-gdb -x EOSM/patches.gdb -ex quit

Here's the log (https://www.dropbox.com/s/3nuwbr9xnrrp2ff/eos_m_qemu_log.txt?dl=0).

Trying to run ML with boot=1 I get a message to remove the battery.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on July 10, 2019, 03:51:11 PM
Sounds about right - press M to open Canon menu. LiveView emulation is quite tricky and has plenty of bugs.

To run ML with boot=1, you will have to run "make install_qemu" from platform/EOSM.202 first. Then, to open ML menu, I need to press "L" a few times, as a workaround (not exactly deterministic, sometimes works, sometimes doesn't). Things are better on cameras that don't start in LiveView by default.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Jip-Hop on July 10, 2019, 05:28:16 PM
Thanks, I'll try it next time. For now I think I'll stick to testing my Lua scripts on my EOS M. That's probably the quickest.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: heder on August 02, 2019, 08:50:08 PM
Help  :(

I'v got a strange problem using qemu from qemu branch (freshly compiled without problems).

My system is ubuntu studio 18.02, using vnc from win10 to the ubuntu studio server (not qemu itself)

My qemu keys are not the correct onces, for instance instead of menu "M" I need to use "U"
and all other keys are mapped to other keys. My problem is that I can't locate the arrow keys.

Anyone experience this problem before ?


Title: Re: How to run Magic Lantern into QEMU?!...
Post by: a1ex on August 02, 2019, 09:22:52 PM
That's an interesting bug, but didn't encounter it yet. I couldn't reproduce it on a Bionic 18.04 VM with Mate interface, also accessed via VNC, but from an openSUSE desktop.

Workaround: the button codes are hardcoded in mpu.c, so you can change them as needed. Would be nice to reproduce the issue and fix it properly, though.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: heder on August 04, 2019, 03:57:20 PM
The key are correct when running with local display/keyboard on my Ubuntu server, but when running via vnc the
keys are incorrect.  So the problem is not qemu/linux but properly tightvnc/windows.  Not that big a issue,
I'll just remap the key to something useable.

5-8-2019
Was using vncserver, changed to x11vnc on display :0, problem gone  :)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: felix_ on June 18, 2020, 02:26:16 AM
Is the tutorial out of date? I dont get the qemu-2.5.0 folder and the eos-qemu folder and the configure.sh script
here is a tree of my install folder https://pastebin.com/eTtkg5s1 The last command I ran successfully was ./install.sh  Where do I get the missing files and folders?
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: felix_ on June 19, 2020, 01:02:37 PM
Never mind, I solved it
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: timbytheriver on July 08, 2020, 01:19:43 PM
First attempt at installing QEMU in my local environment.

Install itself seems to be ok, but get error screen when attempting to run:

/path/to/qemu-eos$  ./run_canon_fw.sh 5D3,firmware="boot=1"


(https://i.ibb.co/4tvxSJL/qemu.png) (https://ibb.co/C8CF9Wy)

Why the message about 1.2.3? The code I copied to the sd.img was 1.1.3 So...

1) I mounted my camera SD card (with my current build on it) and copied the files to the sd.img as usual. Is this correct?

2) "For models that use a serial flash, you may have to dump its contents using the sf_dump module, then copy SFDATA.BIN as well." Should I be doing this for 5D3?

All I have in my QEMU/5D3 folder is:

debugmsg.gdb
ROM0.BIN
ROM1.BIN

Thanks.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: timbytheriver on July 10, 2020, 10:10:46 AM
*UPDATE*

Decided to start over again. Deleted qemu-eos folder.

Followed these excellent video tuts: https://www.magiclantern.fm/forum/index.php?topic=16012.msg191686#msg191686 and https://www.magiclantern.fm/forum/index.php?topic=2864.msg190851#msg190851

Installed with no error messages.

Partial success: I can now get to the qemu canon screen. But the keystrokes do not control navigation. I can see that the keystrokes are received at the terminal – but no menu navigation happens. I read an @a1ex post which suggested adding the Terminal to Security panel. This allowed me to control the menus with the arrow keys one time – but subsequently froze, and hasn't worked again.

Ideally I'd like to be able run qemu from a custom image of my physical SD card– but am having trouble making this work also.

Any suggestions?

I have read the thread, the README and searched.
Mac OSX Mojave 10.14.6
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: SubZeroz on October 15, 2020, 08:08:38 PM
Hi,

BitBucket is not working anymore...
Is there anywhere else I can get the script to patch QEMU?

Thanks!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: names_are_hard on October 15, 2020, 09:25:03 PM
Please tell us which patch you are referring to.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: yourboylloyd on October 16, 2020, 01:07:24 AM
Quote from: SubZeroz on October 15, 2020, 08:08:38 PM
Hi,

BitBucket is not working anymore...
Is there anywhere else I can get the script to patch QEMU?

Thanks!

You have to do some manual link editing. The QEMU migration solution is here and the first post of that thread is the original tutorial:

https://www.magiclantern.fm/forum/index.php?topic=991.msg230220#msg230220
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on May 29, 2021, 12:38:04 AM
Some progress with GUI emulation on EOS M3:

(https://i.ibb.co/9trP3kZ/Virtual-Box-Lubuntu-16-04-4-29-05-2021-01-09-10.png) (https://ibb.co/hfm01JY)


Is it possible to utilize more than one host's core by QEMU?
For example for display redrawing.
My current implementation converts zico's rgba buffer to yuv bitmap+opacity, then converts yuv image and yuv bitmap buffers to rgb, mix them using opacity buffer and put the result to QEMU's output buffer. This is a job for the second core.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: names_are_hard on May 29, 2021, 03:33:50 AM
Cool!  Which Qemu are you using?

The steps you describe sound sequential to me, which is often not improved by running in parallel.  Qemu does support parallelisation, but I've never used it myself.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on May 29, 2021, 08:36:58 AM
qemu-2.5.0
These steps are performed on special hardware(zico and display controller) independently of the main (ARM) core. So they could be emulated by the second thread independently...
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kitor on May 29, 2021, 09:01:05 AM
QuoteMy current implementation converts zico's rgba buffer to yuv bitmap+opacity, then converts yuv image and yuv bitmap buffers to rgb, mix them using opacity buffer and put the result to QEMU's output buffer. This is a job for the second core.

Static source and destination addresses or you somehow parsed XimrContext?
Good to see a progress on that.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on May 29, 2021, 09:38:39 AM
Quote from: kitor on May 29, 2021, 09:01:05 AM
Static source and destination addresses or you somehow parsed XimrContext?
The second one. ATM I use only one RGBA buffer. It's should be enough for playback mode

QuoteGood to see a progress on that.
It is bad that A1ex and other 'guru' developers  show little activity now. There are less than 20 commits for two years.  :(
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kitor on May 29, 2021, 10:42:55 AM
QuoteThe second one.
Good and bad at the same time.
Bad - considering XimrContext struct changed multiple times in Digic6 gen and then for each next generation. It will require work for multiple models.

Good - I noticed this week that GUI code on R swaps buffer addresses inside VRAM struct used for 1st layer (GUI) somewhere on early boot. If I grab GUI VRAM struct fast enough, I get pBitmap address of overlays buffer instead of GUI. Thus hardcoded addresses would fail quickly.

@coon messed around eeprom code (d7/8 EOS won't boot due to that), but I'm not sure if he implemented it eventually.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: Ant123 on May 29, 2021, 11:05:34 AM
Quote from: kitor on May 29, 2021, 10:42:55 AM
Good and bad at the same time.
Bad - considering XimrContext struct changed multiple times in Digic6 gen and then for each next generation. It will require work for multiple models.
It's not a big problem. Look at CHDK: they support 160 cameras and 320+ firmwares.

QuoteGood - I noticed this week that GUI code on R swaps buffer addresses inside VRAM struct used for 1st layer (GUI) somewhere on early boot. If I grab GUI VRAM struct fast enough, I get pBitmap address of overlays buffer instead of GUI. Thus hardcoded addresses would fail quickly.

The current YUV Bitmap address can be obtained from MMIO register  (https://foss.heptapod.net/magic-lantern/magic-lantern/-/blob/branch/qemu/contrib/qemu/eos/eos.c#L5480)of display controller.
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: names_are_hard on May 29, 2021, 06:14:27 PM
Quote from: Ant123 on May 29, 2021, 09:38:39 AM
It is bad that A1ex and other 'guru' developers  show little activity now. There are less than 20 commits for two years.  :(

Yes, this is not ideal.  But I have a fork that is reasonably active: https://github.com/reticulatedpines/magiclantern_simplified/commits/dev

The primary purpose of my repo is to try and get Digic 6, 7, 8 cams supported.  We are making good progress and have menus etc working.

That repo has Qemu 4.2.1, and is a merge of lua_fix, unified, qemu and digic6-dumper, so for most things you can work on it directly rather than needing to keep swapping branches.  I've updated the build system a little so it works with more modern Linux tools, it works with Debian Testing for me.  Qemu 4 is not fully tested and not perfectly equivalent to 2, but it's pretty good - can boot e.g. 50D to full GUI including ML GUI.  Qemu 4 is slightly out of date now.  It was supported when I ported ML patchset, but isn't anymore, so it wants updating to 5.

I'd be interested in seeing your changes and happy to consider PRs etc!
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: kitor on May 30, 2021, 06:30:01 PM
Quote from: Ant123 on May 29, 2021, 11:05:34 AM
The current YUV Bitmap address can be obtained from MMIO register  (https://foss.heptapod.net/magic-lantern/magic-lantern/-/blob/branch/qemu/contrib/qemu/eos/eos.c#L5480)of display controller.

I was writing about RGBA 'input layer', not YUV 'output chunk'. EOS firmware have a nice pointer from WINSYS that points to VRAM struct that happens to be 1st layer in XimrContext. In EOS R, inside that VRAM struct, pointer to buffer changes somewhere during early boot.

That's why I said it is good, as if you parse XimrContext - it doesn't matter ;)
Title: Re: How to run Magic Lantern into QEMU?!...
Post by: names_are_hard on June 21, 2021, 12:37:28 AM
With Ant's help I have duplicated this - can run M3 to GUI locally.  Thank you Ant for being patient with my stupid questions!

I have a lot of qemu tasks that I can work on.  Largely these need C and linux make skills, not ARM asm or ML experience.  So if anyone would like to volunteer, these are more accessible tasks.
- get local qemu-eos test suite for ML working in 2 and 4, so they can be compared
- as needed, improve tests or qemu-eos 4 accuracy, so that 4 is at parity with 2 for ML tests
- update qemu-eos from 4 to 5: 4 is no longer supported (6 is very new, I think 5 is a better target)
- using M3 as a guide, extend support to other modern cams (this will need ML and asm knowledge!)