Author Topic: How to run Magic Lantern into QEMU?!...  (Read 100259 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10716
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #250 on: December 24, 2017, 10:21:49 AM »
Okay, sorted out, it was a typo when trying to bring QEMU window to foreground. This is still tricky, for example it doesn't work when starting multiple instances, but I'm probably doing this wrong - how do you open a window in foreground from command line on Mac?!

Also found a better way to tell whether the SD/CF images are mounted (or otherwise in use). On Mac, checking them with lsof is enough - this handles both images mounted with "hdiutil attach" and being in use by QEMU itself. On Linux (or at least on my system), lsof only handles the latter case, so checking with losetup (or grepping the output of "mount") is still needed to make sure the images are not in use by some other process when running the emulation.

BTW - already old news, but you can already navigate Canon menus on 1300D and 40D in QEMU.

Current state: 20 EOS models able to run the GUI in the emulator! These are:
5D2 5D3 6D 40D 50D 60D 70D 450D 500D 550D 600D 650D 700D 100D 1000D 1100D 1200D 1300D EOSM EOSM2.

t3r4n

  • New to the forum
  • *
  • Posts: 20
Re: How to run Magic Lantern into QEMU?!...
« Reply #251 on: December 25, 2017, 12:09:33 PM »
@a1ex asked about this one:
Noticed a message that I don't remember seeing on previous versions:

Code: [Select]
49:53: execution error: The variable qemu is not defined. (-2753)
Doesn't seem to hurt anything, just wondering if others are seeing it.

Don't remember seeing it; when/where does it appear, and in what color?

it happens on the 750D around this time in the boot, I tried to recreate the colours as they were (normal colour is qemu and red is camera?):
(sorry the code tag and colours don't mix)

[SF] InstallSerialFlash 2 0xd20b0d8c 0x0 0x800000 1

[ROM-DMA0] Copy [0xFD13A000] -> [0x407FFFA0], length [0x00001363], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA0] Copy [0xFDCC0000] -> [0x40D6C000], length [0x0015B6AF], flags [0x11100003]
[XDMAC0] OK
    0:     3.328 [STARTUP]
K393 ICU Firmware Version 1.0.0 ( 8.7.2 )
     5:     4.864 [PROPAD] ERROR Not Exist Valid ComboPackages!! 0x20000

49:53: execution error: Die Variable „qemu“ ist nicht definiert. (-2753)
[ROM-DMA0] Copy [0xFD200000] -> [0x408020C0], length [0x000205E3], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA0] Copy [0xFD320000] -> [0x40842160], length [0x0001AEDB], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA0] Copy [0xFD360000] -> [0x40862200], length [0x0001DCA7], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA1] Copy [0xFE744B88] -> [0xDFF00000], length [0x00002E77], flags [0x11100003]
[XDMAC1] OK
[MPU] Received: 06 04 02 00 00 00  (Init - spell #1)
[MPU] Sending : 2c 2a 02 00 03 03 03 04 03 00 00 48 00 00 00 14 50 00 00 00 00 81 06 00 00 04 06 00 00 04 06 00 00 04 01 01 00 00 00 00 4d 4b 01 00  (Init)
    9:    64.512 [PROPAD] ERROR SearchPropertyPackage DataType (0) = 0x01000000(L:3294)

dfort

  • Hero Member
  • *****
  • Posts: 2321
Re: How to run Magic Lantern into QEMU?!...
« Reply #252 on: December 25, 2017, 10:20:12 PM »
Don't remember seeing it; when/where does it appear, and in what color?

It was showing up here, in black:

Code: [Select]
./run_canon_fw.sh EOSM,firmware=boot=0 -s -S &
arm-none-eabi-gdb -x EOSM/patches.gdb
...
Setting BOOTDISK flag to 0
49:53: execution error: The variable qemu is not defined. (-2753)
0xffff0000 in ?? ()

However, this disappeared on the latest commits so -- nevermind!

By the way, the EOSM doesn't launch into the GUI so I'm not able to run the sf_dump you asked for--unless there's another way of doing this without using the GUI.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

t3r4n

  • New to the forum
  • *
  • Posts: 20
Re: How to run Magic Lantern into QEMU?!...
« Reply #253 on: December 27, 2017, 01:11:24 PM »
Hey,
I've done two minor changes to the qemu install script to make installation of an updated version a bit easier. That was the small step  :).

I hope I did everything right by forking the repository, creating a new branch, uploading the commit and creating the pull request on the emu branch (the old sticky thread here has lost all images so I was guessing from the remaining text).

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10716
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #254 on: December 30, 2017, 12:58:19 AM »
Applied, thanks.

More cool stuff:

- lots of MPU properties documented, cross-checking welcome
- mode dial emulation (not perfect, but...)
- you can change some MPU-based properties, as long as the change is initiated from the main CPU *)

These are:
- shutter, aperture, ISO
- exposure compensation, flash exposure compensation
- metering mode, drive mode, AF mode
- picture style, white balance
- ExpSim, ALO, HTP, MLU.

*) That means, from ML menus/scripts or from the Q dialog, but not directly from scrollwheels. Some properties (shutter, aperture, AF points) are changed from the MPU (that is, the MPU is expected to interpret the button presses on its own and decide to change these parameters, but this behavior is not emulated). This is a bit model-specific, e.g. you can change shutter speed on 6D or 100D (where the change is made from the main CPU), but not on most other models.

If you can identify other properties that cannot be changed, please report.

It now runs a large part of api_test.lua! (all except photo capture, autofocus, half-shutter and LiveView tests). It even handles the shooting mode switch and those random exposure loops (with animation!)

Code: [Select]
diff -r f37efb4d8d53 scripts/api_test.lua
--- a/scripts/api_test.lua
+++ b/scripts/api_test.lua
@@ -237,13 +237,6 @@
     assert(camera.gui.play == true)
     assert(camera.gui.mode == 1)
 
-    -- half-shutter should exit playback mode
-    key.press(KEY.HALFSHUTTER)
-    msleep(1000)
-    assert(camera.gui.play == false)
-    assert(camera.gui.mode == 0)
-    key.press(KEY.UNPRESS_HALFSHUTTER)
-
     -- randomly switch between PLAY, MENU and IDLE (with or without LiveView)
     for i = 1,100 do
         -- we can request MENU or PLAY mode from anywhere
@@ -281,32 +274,8 @@
                 assert(camera.gui.mode == 0)
             end
         end
-
-        -- also play around with LiveView
-        if camera.gui.menu == false and camera.gui.play == false then
-            if math.random(1,2) == 1 then
-                -- do something with LiveView, but not as often as switching MENU/PLAY
-                if not lv.enabled then
-                    printf("Start LiveView...\n");
-                    lv.start()
-                elseif lv.paused then
-                    printf("Resume LiveView...\n");
-                    lv.resume()
-                elseif math.random(1,10) < 9 then
-                    -- this gets taken less often than the next one, why?
-                    -- fixme: biased random?
-                    printf("Pause LiveView...\n");
-                    lv.pause()
-                else
-                    printf("Stop LiveView...\n");
-                    lv.stop()
-                end
-            end
-        end
     end
 
-    lv.stop()
-
     printf("Canon GUI tests completed.\n")
     printf("\n")
 end
@@ -1341,14 +1310,9 @@
     test_io()
     test_camera_gui()
     test_menu()
-    test_camera_take_pics()
     msleep(1000)
     test_multitasking()
-    test_keys()
-    test_lv()
-    test_lens_focus()
     test_camera_exposure()
-    test_movie()
     
     printf("Done!\n")



8)

t3r4n

  • New to the forum
  • *
  • Posts: 20
Re: How to run Magic Lantern into QEMU?!...
« Reply #255 on: January 02, 2018, 09:51:32 PM »
Hi,
question:
When running qemu with the -d calls option is it possible to suppress certain calls? If I start a run into FROM Utility the poll serial io call fill up a log file with hundreds of MB quite fast.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10716
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #256 on: January 02, 2018, 11:03:47 PM »
Not directly, but you may use grep (the -C option helps to show calls around some keyword, MMIO register, other function). Or, by hardcoding some custom filters in logging.c.

Most of the time, I find the call stack more helpful than the (huge) call trace. To use that one, run with -d callstack and print it for any function you wish, like this:

Code: [Select]
b *0x1234
commands
  silent
  print_current_location_with_callstack
  printf "whatever message\n"
  c
end

BTW, just committed a bunch of doc updates, mostly with DryOS internals, debugging tips and similar stuff.

esas

  • New to the forum
  • *
  • Posts: 20
  • 70D.112, 650D.104, EOSM.202
Re: How to run Magic Lantern into QEMU?!...
« Reply #257 on: January 04, 2018, 07:27:53 PM »
The README was linked a few times, including first post (also asked for some proof-reading).

Think I found a small error. Under headline "Running Canon firmware" shouldn't there be a "-x" in there like this:
Code: [Select]
./run_canon_fw.sh EOSM,firmware="boot=0" -s -S & arm-none-eabi-gdb -x EOSM/patches.gdb
At least I couldn't get it to work before I found that comment inside the patches.gdb.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10716
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #258 on: January 04, 2018, 08:00:09 PM »
Thanks - that was after I was pretty sure I've checked that section a couple of times.

Exactly that's why I've asked for proof-reading - to make sure it works for new users who were not familiar with QEMU before.

escho

  • Senior
  • ****
  • Posts: 482
Re: How to run Magic Lantern into QEMU?!...
« Reply #259 on: January 13, 2018, 11:14:32 PM »
I´m trying to get ML running into qemu...
... but I run into some little problems

My system is openSUSE Thumbleweed, my cams are 6D and 600D

Starting install.sh from contrib/qemu tells me:

i
Code: [Select]
nstall.sh: Zeile 338: pip2: Kommando nicht gefunden.
install.sh: Zeile 339: pip2: Kommando nicht gefunden.
install.sh: Zeile 339: vncdotool: Kommando nicht gefunden.
install.sh: Zeile 339: pip2: Kommando nicht gefunden.
These messages correspond to these lines in the install script:

Code: [Select]
# install docutils (for compiling ML modules) and vncdotool (for test suite)
# only install if any of them is missing
pip2 list | grep docutils  || rst2html -h  > /dev/null || pip2 install docutils
pip2 list | grep vncdotool || vncdotool -h > /dev/null || pip2 install vncdotool

pip2 doesn´t exist on my system. So I would have to use pip or pip3. Which one should I prefer?

Since I compile ML, I have docutils installed via paketmanager (Yast). But what about vncdotool? I don´t find any pakets for it (not in the installed repos, not in openSUSE build service).
I´m a bit afraid to install vncdotool outside the paketmanager (using the pip-stuff in the script) and not be able to revert it, if something goes wrong.

Would you recommand to run a VM to play with this stuff?

A last question for the moment:

Why do you tell the user to add gcc/gdb-bins to PATH? Why not a little line in the script  for automation?:
Code: [Select]
export PATH=$PATH:~/$TOOLCHAIN/bin

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10716
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #260 on: January 13, 2018, 11:27:23 PM »
That's interesting... I'm also running Tumbleweed right now and I have: pip pip2 pip2.7 pip3 pip3.6

Maybe you have to install python2? It should also work with python3/pip3.

You don't really need vncdotool, unless you want to run the test suite (which actually requires a patched vncdotool, I should submit a PR) or the examples that use it.

If you run the "export PATH" command in a bash script, it will only be valid within that script. To run the examples from the guide, one has to have arm-none-eabi-gdb in PATH (or modify the command lines).

escho

  • Senior
  • ****
  • Posts: 482
Re: How to run Magic Lantern into QEMU?!...
« Reply #261 on: January 14, 2018, 11:33:45 AM »
Yes, interesting. "cnf pip2" tells me, that it´s in python2-pip, but python2-pip isn´t installed per default here in Thumbleweed. Got the script running with the two "pip2-lines" commented out.

But now, I´m hanging here:
Code: [Select]
./run_canon_fw.sh 6D,firmware="boot=1"I will look later, whats going on.

Thanks for your help, Alex

dfort

  • Hero Member
  • *****
  • Posts: 2321
Re: How to run Magic Lantern into QEMU?!...
« Reply #262 on: January 17, 2018, 06:22:16 AM »
Got a suggestion for the documentation.

After hours of struggling to get QEMU working again on Windows Subsystem for Linux (WSL) after a bunch of Microsoft and Ubuntu updates -- what is really important but not obvious is having to start the X server. BTW, VcXsrv isn't working here after the updates but Xming is and it is just as easy to install and run.

[EDIT] Oh yeah, don't forget this:

Code: [Select]
export DISPLAY=:0
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

dfort

  • Hero Member
  • *****
  • Posts: 2321
Re: How to run Magic Lantern into QEMU?!...
« Reply #263 on: Today at 06:16:38 AM »
My main development system is a Mac PowerBook and since much of what I've been running in QEMU is not on the qemu branch I've had to unmount the disk image manually. Doing "make install" would trigger the "Error: please unmount the SD image." even though the disk didn't appear as a mounted volumes. Turns out that the fix was to simply "eject" instead of "unmount" -- this works great with both physical cards and disk images. According to the hdiutil man page:

Quote
NOTE: unmount does NOT detach any disk image associated with the volume.
          Images are attached and detached; volumes are mounted and unmounted.

Pull request submitted on the qemu branch.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103