How to run Magic Lantern into QEMU?!...

Started by jplxpto, September 23, 2012, 08:29:02 PM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

jplxpto

How to run Magic Lantern into QEMU?!...

I would like to know if anyone has ever launched Magic Lantern on QEMU. I've used it a few times, and it seemed to me that however limited it may be useful to conduct some tests.

If anyone has knowledge about this subject, thank you give me some tips that may also be useful for other developers.

Thank you.




Short answer: check the README and the developer guide.

nanomad

Last time I checked there was a patch for qemu in ML source code.
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

jplxpto

Ok ... thank you...

One of these days I'll test it ...

miyake


jplxpto


a1ex

With a bit of tweaking, that qemu patch seems to do something :)




scrax

I'm interested in running ML in QEMU for testing scripts, is the tweaked patch in the source?
What do I need to do compile a patched ML version and load it with QEMU?
I'm reading the page linked by miyake now.
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

a1ex

It's not yet ready for this. What's missing is task switching support; I've tried some hacks based on setjmp (this and this), but they weren't reliable at all, so I'm thinking to try something like FreeRTOS. A simple cooperative scheduler (to switch tasks when they call msleep) would be enough.

G3gg0 had some success in running the DryOS task switcher. My approach is a bit different, I'm trying to re-implement the stubs from scratch so only ML code is emulated - that is, menu, scripting engine, overlays etc.

The qemu patch is very rough and not yet published.

jplxpto

I'm very interested in this subject.

I'll be glad to help you.

Maybe this can help us ... The eCos is another option (http://ecos.sourceware.org/about.html) and it already has support for GDB.

mark.farnell

Quote from: jplxpto on March 23, 2013, 03:15:40 AM
I'm very interested in this subject.

I'll be glad to help you.

Maybe this can help us ... The eCos is another option (http://ecos.sourceware.org/about.html) and it already has support for GDB.

In this case, is eCos a replacement of qemu?  However it says it is an operating system.... so do you mean to replace DryOS with eCos altogether? 

a1ex

Yes, only for emulation. I don't need to emulate the entire Canon firmware, just ML code.

mark.farnell

Quote from: a1ex on March 29, 2013, 09:55:12 PM
Yes, only for emulation. I don't need to emulate the entire Canon firmware, just ML code.

So at this stage, is it possible to emulate the ML code with eCos?  If so, how?

Also will the ability of emulating the ML code make developing safer?  ( I mean detecting silly mistakes such as writing wrong values to NVRAM variables, that can potentially brick the camera)

a1ex

See my previous post. There was no progress since then.

Wrong NVRAM values depend only on how they are interpreted by Canon code, so emulating only ML code won't help. If you get permanent ERR70 after changing some setting, that's a clear sign that you have set an invalid value; there's no other way to tell this. Maybe full emulation could help here, but that's very difficult (of course, I won't be surprised if g3gg0 succeeds). Detecting memory leaks may be possible (maybe run ML under valgrind?)

I want to emulate ML so I can check how the menus look on each camera, without having to buy every single model. Also, it may be helpful when writing user scripts, or when working with graphics code, fonts etc.

1%

I got 600D booting in trix to the service/bootloader menu but didn't really know what to do with it afterwards.

g3gg0

at the moment i am working on qemu to emulate the whole fimware.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Indy

Updater code (at least 7d one) can run partially on qemu.

g3gg0

here are the changes i made to qemu:
http://upload.g3gg0.de/pub_files/7dc0800617416fdbeb1490dcc4a2164d/qemu_eos.7z

they contain a lot of hardware emulation.

booting fails imho due to the MPU that is not emulated. init routines expect a property that is not available.
i think the MPU would send it to the main firmware.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

jplxpto

Quote from: g3gg0 on March 30, 2013, 01:07:34 AM
here are the changes i made to qemu:
http://upload.g3gg0.de/pub_files/7dc0800617416fdbeb1490dcc4a2164d/qemu_eos.7z

they contain a lot of hardware emulation.

booting fails imho due to the MPU that is not emulated. init routines expect a property that is not available.
i think the MPU would send it to the main firmware.

Thank you! I will try this ..

a1ex

Just merged my version with g3gg0's and pushed it on the main repo. Works on 60D, 600D, 500D, 5D2 and 650D.

In theory, you only have to run the install script - it will download QEMU 1.4.0, apply our modifications, and it will tell you what to do next.



If successful, you should get the hello world picture, and a log like this:


00000000 - 3FFFFFFF: eos.ram
40000000 - 7FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror_F1
F2000000 - F2FFFFFF: eos.rom0_mirror_F2
F3000000 - F3FFFFFF: eos.rom0_mirror_F3
F4000000 - F4FFFFFF: eos.rom0_mirror_F4
F5000000 - F5FFFFFF: eos.rom0_mirror_F5
F6000000 - F6FFFFFF: eos.rom0_mirror_F6
F7000000 - F7FFFFFF: eos.rom0_mirror_F7
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror_F9
FA000000 - FAFFFFFF: eos.rom1_mirror_FA
FB000000 - FBFFFFFF: eos.rom1_mirror_FB
FC000000 - FCFFFFFF: eos.rom1_mirror_FC
FD000000 - FDFFFFFF: eos.rom1_mirror_FD
FE000000 - FEFFFFFF: eos.rom1_mirror_FE
FF000000 - FFFFFFFF: eos.rom1_mirror_FF
C0000000 - CFFFFFFF: eos.iomem
[EOS] loading 'ROM-650D.BIN' to 0xF7000000-0xF8FFFFFF
[EOS] loading 'qemu-helper.bin' to 0x30000000-0x300088E7
[EOS] loading 'autoexec.bin' to 0x00800000-0x00855F4F
...
[GPIO] [0xC022C188] <- 0x138800 at pc=0x855E90
[FlashIF] at [0x000D4020]: 'Write enable' enabled
[Basic] at [0x000D4020] [0x00000000] <- [0xC0400008]
[Basic] at [0x000D4020] [0x00430005] -> [0xC0400008]
create_init_task(7e1ac)
*** init_task
[DebugMsg] (50,3) Magic Lantern v2.3.NEXT.2013Apr07.650D101 (781e0140ec5a+ (unified) tip)
[DebugMsg] (50,3) Built on 2013-04-07 20:02:47 by alex@thinkpad
...
Hello at QEMU console!


Let me know if it works for you (and what other dependencies you had to install).

scrax

Quote from: a1ex on April 08, 2013, 12:12:32 AM
Just merged my version with g3gg0's and pushed it on the main repo. Works on 60D, 600D, 500D, 5D2 and 650D.

In theory, you only have to run the install script - it will download QEMU 1.4.0, apply our modifications, and it will tell you what to do next.



If successful, you should get the hello world picture, and a log like this:


00000000 - 3FFFFFFF: eos.ram
40000000 - 7FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror_F1
F2000000 - F2FFFFFF: eos.rom0_mirror_F2
F3000000 - F3FFFFFF: eos.rom0_mirror_F3
F4000000 - F4FFFFFF: eos.rom0_mirror_F4
F5000000 - F5FFFFFF: eos.rom0_mirror_F5
F6000000 - F6FFFFFF: eos.rom0_mirror_F6
F7000000 - F7FFFFFF: eos.rom0_mirror_F7
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror_F9
FA000000 - FAFFFFFF: eos.rom1_mirror_FA
FB000000 - FBFFFFFF: eos.rom1_mirror_FB
FC000000 - FCFFFFFF: eos.rom1_mirror_FC
FD000000 - FDFFFFFF: eos.rom1_mirror_FD
FE000000 - FEFFFFFF: eos.rom1_mirror_FE
FF000000 - FFFFFFFF: eos.rom1_mirror_FF
C0000000 - CFFFFFFF: eos.iomem
[EOS] loading 'ROM-650D.BIN' to 0xF7000000-0xF8FFFFFF
[EOS] loading 'qemu-helper.bin' to 0x30000000-0x300088E7
[EOS] loading 'autoexec.bin' to 0x00800000-0x00855F4F
...
[GPIO] [0xC022C188] <- 0x138800 at pc=0x855E90
[FlashIF] at [0x000D4020]: 'Write enable' enabled
[Basic] at [0x000D4020] [0x00000000] <- [0xC0400008]
[Basic] at [0x000D4020] [0x00430005] -> [0xC0400008]
create_init_task(7e1ac)
*** init_task
[DebugMsg] (50,3) Magic Lantern v2.3.NEXT.2013Apr07.650D101 (781e0140ec5a+ (unified) tip)
[DebugMsg] (50,3) Built on 2013-04-07 20:02:47 by alex@thinkpad
...
Hello at QEMU console!


Let me know if it works for you (and what other dependencies you had to install).

Will try on osx the script and report back, but first I need to clean up some space from my main hd.
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

trsaunders

I'm trying to run this on Arch linux:
I had to build with these options because python 2 binary is called python2 and the docs building didn't work:
./configure --target-list=arm-softmmu --python=/usr/bin/python2 --disable-docs

when I try to launch:
➜  qemu  ./run_ml_5D3.sh
make: Entering directory `/home/tom/dev/software/qemu/qemu-1.4.0'
make: Leaving directory `/home/tom/dev/software/qemu/qemu-1.4.0'
make: Entering directory `/home/tom/dev/software/magic-lantern/platform/5D3.113'
[ VERSION  ]   ../../platform/5D3.113/version.c
[ CC       ]   version.o
[ MENU IDX ]   menuindexentries.h
No menuindex.txt not running "python2 menuindex.py"
[ CC       ]   menuindex.o
[ LD       ]   magiclantern
[ OBJCOPY  ]   magiclantern.bin
[ SYMBOLS  ]   magiclantern.sym
[ CC       ]   reboot.o
[ LD       ]   autoexec
autoexec.bin: 443984 bytes

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x06c550 0x000d5950 0x000d5950 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00069500 0x00069500 0x6c458 0x7bb6d RWE 0x100
[ OBJCOPY  ]   autoexec.bin
make: Leaving directory `/home/tom/dev/software/magic-lantern/platform/5D3.113'
make: Entering directory `/home/tom/dev/software/magic-lantern/platform/5D3.113'
make: `qemu-helper.bin' is up to date.
make: Leaving directory `/home/tom/dev/software/magic-lantern/platform/5D3.113'
rm: cannot remove 'vram.txt': No such file or directory
rm: cannot remove 'vram.png': No such file or directory
00000000 - 3FFFFFFF: eos.ram
40000000 - 7FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror_F1
F2000000 - F2FFFFFF: eos.rom0_mirror_F2
F3000000 - F3FFFFFF: eos.rom0_mirror_F3
F4000000 - F4FFFFFF: eos.rom0_mirror_F4
F5000000 - F5FFFFFF: eos.rom0_mirror_F5
F6000000 - F6FFFFFF: eos.rom0_mirror_F6
F7000000 - F7FFFFFF: eos.rom0_mirror_F7
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror_F9
FA000000 - FAFFFFFF: eos.rom1_mirror_FA
FB000000 - FBFFFFFF: eos.rom1_mirror_FB
FC000000 - FCFFFFFF: eos.rom1_mirror_FC
FD000000 - FDFFFFFF: eos.rom1_mirror_FD
FE000000 - FEFFFFFF: eos.rom1_mirror_FE
FF000000 - FFFFFFFF: eos.rom1_mirror_FF
C0000000 - CFFFFFFF: eos.iomem
eos_load_image: file not found 'ROM-5D3.BIN'
run_ml.sh: line 15: 25600 Aborted                 (core dumped) $QEMU_PATH/arm-softmmu/qemu-system-arm -M ML-$1
convert: unable to open image `vram.txt': No such file or directory @ error/blob.c/OpenBlob/2641.
convert: no images defined `vram.png' @ error/convert.c/ConvertImageCommand/3103.
50D, 5D3

a1ex

5D3 is not working yet, here it fails at create_init_task. You also have to get some Canon firmware dumps, see the install instructions.

You may have better luck if you add support for 50D (you need to add a ML_MACHINE definition and create a launch script).

trsaunders

Sorry, I didn't read that well! I thought I had the 5D3-ROM in the correct location but I guess not. I now get as far as create_init_task so I assume that qemu is at least working. I'll have a go at adding support for 50D.
50D, 5D3

a1ex

5D3 works too, I was loading the old 1.1.2 dump by mistake.


create_init_task(695ac)
*** init_task
[DebugMsg] (50,3) Magic Lantern v2.3.NEXT.2013Apr08.5D3113 (087dd0afd6b8+ (unified) tip)
...


It fails at redraw, you will need to comment it out from ML code.

trsaunders



50D worked without much effort - just copied 60D definitions, changed to 50D and added the appropriate run_ml script.

I tried to generate a patch for the changes but hg diff was producing a lot of spurious changes - maybe it doesn't like diffing a diff?!
50D, 5D3