How to run Magic Lantern into QEMU?!...

[a1ex]

I've used a new hg tree under qemu to generate the diff. It's a bit ugly now; I'm thinking to modify the install script to store the new files as plain files, and use the patch only for modification to QEMU sources.

Edit: did these changes and some small additions:

- to emulate Canon firmware, without ML:
    ./run_canon_fw.sh 600D

- to generate a diff or commit changes (say you have modified eos.c or added some script):
    ./copy_back_to_contrib.sh
    then normal hg commands or gui in contrib/qemu

- to run the firmware in gdb:
    qemu-1.4.0/arm-softmmu/qemu-system-arm -M 5D3 -s -S in one terminal
    arm-elf-gdb -x gdbopts in another

[1%]

6D is running in Qemu.. ML seemed to run but I did not see hello world in VNC.
Ran the canon FW and debugger and it opened up stopping at ff0c0008.
I guess this can be connected as debugger to ida or another disassembler?

[a1ex]

Yes, you can connect from IDA or GDB to localhost:1234.

The display device is not implemented - it just saves a screenshot when you call dispcheck.

[1%]

I compiled on windows with mingw. It loads but get:

VNC server running on `::1:5900'

and can't connect to vnc or gdb.

[g3gg0]

vnc isnt implemented for eos cameras, just gdb.

[1%]

Works in linux. You can connect over vnc and shut down qemu.

[Marsu42]

I compiled on windows with mingw. It loads but get

It would be nice to also get it running on Windows though - of course I have Linux/whatever vms, but for ml I'd like to get around them on my puny laptop just like there is no need to use Linux to compile ml.

[1%]

I can post the binary... the armmmu folder is only something like 20-30mb.... but I can't connect to it in windows, like it needs another patch to use the networking. The linux binary will only connect on localhost so I can't push it out to gdb over the network.

Thats the prob I'm having with QEMU.

[Marsu42]

The linux binary will only connect on localhost so I can't push it out to gdb over the network.

Use a ssh tunnel or network redirector?

[1%]

I turned UFW off and set some forwarding from lo to eth0... so far its working. Really would be nice to get it on one machine tho.

[wolf]

Is there a way to run qemu for a 550D?

[1%]

Yep, just edit the files and add 550D + values.

[wolf]

@1% Thanks. 

I changed the content of "run_ml_550D.sh" to "sh run_ml.sh 550D 109" (550 instead of 500)
but it still won't work. The 550D is not listed under supported machines. I don't know if it's important and failure is caused by something else, or caused by my installation? Well I have no vram.txt as you see. Do you have any clue?

Code Select Expand

$ ./run_ml_550D.sh
make: Entering directory `/home/wolf/qemu/qemu-1.4.0'
make: Leaving directory `/home/wolf/qemu/qemu-1.4.0'
make: Entering directory `/home/wolf/magic-lantern/platform/550D.109'
[ VERSION  ]   ../../platform/550D.109/version.c
[ CC       ]   version.o
[ MENU IDX ]   menuindexentries.h
No menuindex.txt not running "python2 menuindex.py"
[ CC       ]   menuindex.o
[ LD       ]   magiclantern
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 449232 bytes
[ SYMBOLS  ]   magiclantern.sym
[ CC       ]   reboot.o
[ LD       ]   autoexec

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x06dbc8 0x00cedbc8 0x00cedbc8 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00c80100 0x00c80100 0x6dad0 0x7d6d9 RWE 0x100
[ OBJCOPY  ]   autoexec.bin
[ STAT     ]   autoexec.bin
autoexec.bin: 449728 bytes
make: Leaving directory `/home/wolf/magic-lantern/platform/550D.109'
make: Entering directory `/home/wolf/magic-lantern/platform/550D.109'
make: `qemu-helper.bin' is up to date.
make: Leaving directory `/home/wolf/magic-lantern/platform/550D.109'
rm: cannot remove 'vram.txt': No such file or directory
rm: cannot remove 'vram.png': No such file or directory
Supported machines are:
none                 empty machine
collie               Collie PDA (SA-1110)
ML-50D               Magic Lantern on Canon EOS 50D
ML-60D               Magic Lantern on Canon EOS 60D
ML-600D              Magic Lantern on Canon EOS 600D
ML-500D              Magic Lantern on Canon EOS 500D
ML-5D2               Magic Lantern on Canon EOS 5D2
ML-5D3               Magic Lantern on Canon EOS 5D3
ML-650D              Magic Lantern on Canon EOS 650D
50D                  Canon EOS 50D
60D                  Canon EOS 60D
600D                 Canon EOS 600D
500D                 Canon EOS 500D
5D2                  Canon EOS 5D2
5D3                  Canon EOS 5D3
650D                 Canon EOS 650D
nuri                 Samsung NURI board (Exynos4210)
smdkc210             Samsung SMDKC210 board (Exynos4210)
connex               Gumstix Connex (PXA255)
verdex               Gumstix Verdex (PXA270)
highbank             Calxeda Highbank (ECX-1000)
integratorcp         ARM Integrator/CP (ARM926EJ-S) (default)
kzm                  ARM KZM Emulation Baseboard (ARM1136)
mainstone            Mainstone II (PXA27x)
musicpal             Marvell 88w8618 / MusicPal (ARM926EJ-S)
n800                 Nokia N800 tablet aka. RX-34 (OMAP2420)
n810                 Nokia N810 tablet aka. RX-44 (OMAP2420)
sx1                  Siemens SX1 (OMAP310) V2
sx1-v1               Siemens SX1 (OMAP310) V1
cheetah              Palm Tungsten|E aka. Cheetah PDA (OMAP310)
realview-eb          ARM RealView Emulation Baseboard (ARM926EJ-S)
realview-eb-mpcore   ARM RealView Emulation Baseboard (ARM11MPCore)
realview-pb-a8       ARM RealView Platform Baseboard for Cortex-A8
realview-pbx-a9      ARM RealView Platform Baseboard Explore for Cortex-A9
akita                Akita PDA (PXA270)
spitz                Spitz PDA (PXA270)
borzoi               Borzoi PDA (PXA270)
terrier              Terrier PDA (PXA270)
lm3s811evb           Stellaris LM3S811EVB
lm3s6965evb          Stellaris LM3S6965EVB
tosa                 Tosa PDA (PXA255)
versatilepb          ARM Versatile/PB (ARM926EJ-S)
versatileab          ARM Versatile/AB (ARM926EJ-S)
vexpress-a9          ARM Versatile Express for Cortex-A9
vexpress-a15         ARM Versatile Express for Cortex-A15
xilinx-zynq-a9       Xilinx Zynq Platform Baseboard for Cortex-A9
z2                   Zipit Z2 (PXA27x)
convert: unable to open image `vram.txt': No such file or directory @ error/blob.c/OpenBlob/2641.
convert: no images defined `vram.png' @ error/convert.c/ConvertImageCommand/3106.
$


[1%]

You have to add add it to eos.c too.

[wolf]

It's working. :-)
But is it possible to test a picoc script already?

[1%]

I wish.. I couldn't get anything useful for debugging either.

[a1ex]

QEMU working again

- runs DryOS task scheduler, semaphores, message queues (massive credits to g3gg0 for the low-level emulation code and insights)
- loads files from a local directory (sdcard or cfcard)
- able to load config files, modules, cropmarks...
- menu navigation working
- file manager working
- arkanoid working (playable!)
- properties not working
- most canon code is not working



https://bitbucket.org/hudson/magic-lantern/branch/qemu

Feel free to turn it into something useful... like script interpreter, testing server, source-level debugger, HDMI emulator, support for image buffers, add a nice GUI... or just port it for your camera. I've only tested it on 5D3 1.1.3, and it's been already used to debug the early 100D ML port in GDB.

[a1ex]

Code Select Expand


[DebugMsg] (139,22) Wait Master Wakeup
[GPIO] at [0xFF080090] [0x00000000] <- [0xC0220024]
....
[DebugMsg] (139,6) Wait Master Wakeup Timeout
[DebugMsg] (139,22) Master Wakeup


Other than that, emulating 7D ML in QEMU works quite well. Any hints about how to emulate the master processor and the communication between them?

[nikfreak]

Works fine here for 6D. Needed to do some additional stuff like filename capitalization etc. I just ran into one last problem. I have somehow no write access - at least there's no "magic.cfg" saved neither "bench.ppm" nor are ROM dumps from debug menu written etc. Though qemu doesn't report an error on qemu monitor. It's just like qemu has write access to some nirvana place in memory....

Code Select Expand


[???] [0x00000010] -> [0xC020006C] PC: 0xFF0C7C14
*** FIO_CreateFile('B:/ML/SETTINGS/magic.cfg') => 17
*** FIO_WriteFile(11, 231)
{{{
# Magic Lantern Nightly.2014Oct02.6D113 (b59a1ac5fbfc+ (qemu) tip)
# Built on 2014-10-02 08:56:42 UTC by magiclantern@magiclante
}}}
*** FIO_CloseFile(11)
[FIO wrapper] closefile() nothing open
Save configs...


Edit:
qemu-helper.c define R/W access. Do I have to change the values for 6D. Btw I compiled and tried via qemu branch

[a1ex]

Correct, file writing is not implemented. The entire FIO implementation is weak (only single-task and very poor error checking) - that is, just barely enough to load ML .

[a1ex]

Just had success in emulating the display test (on most cameras). On 6D, I had to emulate the bootloader as well (without it, the display init routine would get stuck).



To run the display test, look for the following "if (0)" and enable them:
- "bootloader config, 4 bpp" -> required to run all boot display tests
- "6D bootloader experiment" -> required for 6D; launch with ./run_canon_fw.sh 6D

edit: after some small changes, Linux works as well

[mk11174]

Anyone know what this HDMI USB thing is, I don't see this on the 550D or 700D when I play with them, this loops so much and makes it super slow to work the menus?

[a1ex]

Major progress: I'm able to launch Canon GUI under QEMU

( side note: Nikon Hacker guys achieved this step a long time ago, so we are just playing catch-up )



More details:
- it launches most Canon tasks
- unmodified 60D firmware (without autoexec.bin or ROM patches) runs as well (and starts the GUI too)
- SD card emulation also works (it loads autoexec.bin and even creates the DCIM directory on startup)
- MPU emulation kinda works (it replays messages from a log file)
- sample log: 60D-qemu-canon-gui-and-sd.log

Next steps:
- emulate unmodified autoexec.bin
- remove all those CONFIG_QEMU hacks
- implement key events as MPU messages
- CF emulation
- enable the emulation for other cameras
- do something about those huge logs
- make the code more QEMU-ish and less hackish
- write a quick start guide
- do something useful with it

What's the use?
- much easier to understand Canon firmware (you can see exactly what some piece of code does with the hardware)
- very useful in diagnosing soft-bricked cameras
- a way to debug your code (or Canon's) in a GUI (gdb or IDA)
- test bench for Lua scripting or for module development
- automated tests for the nightly builds (see also this proposal)

Some tips, until a more complete guide will be available:
- to load a SD card image, use something like: ./run_canon_fw.sh 60D -sd sd.img
- to display a trace of the firmware code, with disassembly, use: ./run_canon_fw.sh 60D -sd sd.img -d exec,int -singlestep
- there is a monitor console as well: ./run_canon_fw.sh 60D -sd sd.img -monitor stdio

[g3gg0]

good work!
a milestone, indeed

[a1ex]

Thanks.

Anyone know what this HDMI USB thing is [...] ?

That's Canon's HotPlug task reading those registers. I'll add an option to quiet them down ( -d ioport, as this one is standard in qemu).