Author Topic: How to run Magic Lantern into QEMU?!...  (Read 84926 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10326
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #200 on: September 29, 2017, 01:08:18 PM »
Playing with different toolchains on Ubuntu (Xenial 64-bit):

1) gdb-arm-none-eabi:i386 and gcc-arm-none-eabi from Ubuntu repo (gcc 4.9 64-bit, 32-bit gdb): animation (3MB)
2) 32-bit gcc-arm-embedded (gcc-arm-none-eabi-5_4-2016q3): animation (3MB)
3) gdb-arm-none-eabi and gcc-arm-none-eabi from Ubuntu repo (gcc 4.9, 64-bit): animation (3MB) - using 60D without GDB
4) gcc-arm-embedded from ppa:team-gcc-arm-embedded/ppa (gcc 6.x, 64-bit): TODO (need to use a different camera - 5D3 requires 32-bit GDB)

Scripts used (should you want to re-create the above scenario, maybe on another OS):
qemu-demo-xenial1.sh
qemu-demo-xenial2.sh
qemu-demo-xenial3.sh
anim.py (to render the animation)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10326
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #201 on: September 30, 2017, 11:12:54 PM »
Some good news for those affected by the 64-bit GDB bug:

- Found out why 5D3 GUI wasn't coming up without patching the date/time: it was waiting for... PROP_MPU_GPS (06 04 03 54 00 00 from MPU). After this change, 5D3 GUI booted without GDB (and the date/time dialog could be bypassed by clicking OK)!
- Even better - g3gg0 figured out how to emulate the real-time clock! This change superseded a bunch of GDB scripts - no more need to patch the date/time dialog!
- Exception: 5D2 and 50D appear to use a different RTC chip edit: g3gg0 just solved it!
- EOS M boots Canon GUI (with the same limitations as EOS M2)
- EOS M and M2 have the date/time dialog left enabled on purpose (to prevent the camera from entering LiveView, which is not emulated)

That means:
- The good news: to boot the GUI, you no longer need GDB for most models (exceptions: M and M2)
- The date/time dialog at startup is gone! (exceptions: 5D2, 50D, M and M2)
- More Canon menus navigable without locking up on DIGIC 5 models (because of that GPS property...)
- The bad news: if you want to do actual debugging in GDB, you will need a 32-bit arm-none-eabi-gdb.

Current status:
- Models able to run the GUI and navigate Canon menu: 16 (most DIGIC 4 and 5, some DIGIC 3).
- Models with major GUI issues: 70D, 5D3 1.2.3.
- Models unable to run the GUI: 6D (help needed), 7D (hard), all DIGIC 6 models, most VxWorks models.

Final note: if you have QEMU already set up, I recommend installing the new one from scratch - the install script will not delete old patches.gdb files. Or just delete these files manually.

kichetof

  • Senior
  • ****
  • Posts: 455
  • Take a beer and enjoy it!
Re: How to run Magic Lantern into QEMU?!...
« Reply #202 on: October 01, 2017, 12:04:38 AM »
Guys, you're awesome!!  8)
Canon 5D3 113 on MacOS High Sierra! Happy!



@a1ex no module menu, need to enable Debug -> Modules debug -> Load modules after crash; reboot and module menu appear

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10326
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #203 on: October 01, 2017, 12:42:23 AM »
@kichetof: "Common issues and workarounds" in the README (or watch my animations)

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3031
  • 5D3 / M1 / 7D / 70D / SL1
Re: How to run Magic Lantern into QEMU?!...
« Reply #204 on: October 01, 2017, 03:07:09 AM »
Looking great out there @kichetof! Could use some of your beers. :P

Some good news for those affected by the 64-bit GDB bug:

Great progress @a1ex!

Final note: if you have QEMU already set up, I recommend installing the new one from scratch - the install script will not delete old patches.gdb files. Or just delete these files manually.

Could you pleaase shed some light on this? It seems I am pretty close to getting the QEMU to emulate on OS X 10.13 (I can see the black window popping up briefly before it disappears) with this message below:

Code: [Select]
./run_canon_fw.sh 100D
DebugMsg=0x4A74 (from GDB script)
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
./run_canon_fw.sh 100D,firmware=boot=1
DebugMsg=0x4A74 (from GDB script)
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
./run_canon_fw.sh 100D,firmware=boot=1
DebugMsg=0x4A74 (from GDB script)
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - DFFFFFFF: eos.iomem
[EOS] loading './100D/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './100D/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
Could not open ./100D/SFDATA.BIN
Seans-Mac-mini-385:qemu DeafEyeJedi$

Especially the 'Could not open ./100D/SFDATA.BIN' part? BTW I did make this virtual SD bootable w Macboot. So close I can feel it!
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

dfort

  • Hero Member
  • *****
  • Posts: 1974
Re: How to run Magic Lantern into QEMU?!...
« Reply #205 on: October 01, 2017, 03:55:50 AM »
Code: [Select]
Could not open ./100D/SFDATA.BIN
Seans-Mac-mini-385:qemu DeafEyeJedi$

Especially the 'Could not open ./100D/SFDATA.BIN' part? BTW I did make this virtual SD bootable w Macboot. So close I can feel it!

Do you have your serial flash dump (SFDATA.BIN) next to your ROM0.BIN and ROM1.BIN files?

You don't need to make the virtual SD bootable with Macboot--where did you read that? Maybe it is corrupted now? No big deal, if you get into trouble or want to try out the latest QEMU changes simply delete your qemu directory (or rename it if you want to save it) and re-run the install.sh script in magic-lantern/contrib/qemu.
EOSM.202 EOSM.203 EOSM2.103 700D.115 5D3.*

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3031
  • 5D3 / M1 / 7D / 70D / SL1
Re: How to run Magic Lantern into QEMU?!...
« Reply #206 on: October 01, 2017, 07:53:13 AM »
Do you have your serial flash dump (SFDATA.BIN) next to your ROM0.BIN and ROM1.BIN files?

Actually I do not. This is where I hit a wall. Not sure where I can actually get the serial flash dump -- care to refresh my memory in here, please?



You don't need to make the virtual SD bootable with Macboot--where did you read that? Maybe it is corrupted now? No big deal, if you get into trouble or want to try out the latest QEMU changes simply delete your qemu directory (or rename it if you want to save it) and re-run the install.sh script in magic-lantern/contrib/qemu.

Thought I read it somewhere that we had to make sure the virtual SD mount was bootable before running QEMU on it or no?
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

dfort

  • Hero Member
  • *****
  • Posts: 1974
Re: How to run Magic Lantern into QEMU?!...
« Reply #207 on: October 01, 2017, 08:37:58 AM »
You need to run the sf_dump module to get a SFDATA.BIN dump and the virtual sd card already has the boot flag set. At least I never needed to run anything special to make it bootable.
EOSM.202 EOSM.203 EOSM2.103 700D.115 5D3.*

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3031
  • 5D3 / M1 / 7D / 70D / SL1
Re: How to run Magic Lantern into QEMU?!...
« Reply #208 on: October 01, 2017, 09:01:20 AM »
You need to run the sf_dump module to get a SFDATA.BIN dump...

I read that. Clearly. Guess what I should have asked was how to get this sf_dump module? Can't seem to find it anywhere. Or maybe looking at the wrong directories?

...and the virtual sd card already has the boot flag set. At least I never needed to run anything special to make it bootable.

Gotcha. Thanks for the clarification.
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 1026
70D.112 & 100D.101

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3031
  • 5D3 / M1 / 7D / 70D / SL1
How to run Magic Lantern into QEMU?!...
« Reply #210 on: October 01, 2017, 10:50:17 AM »
Thanks for pointing me to that @nikfreak! Actually ended up getting a copy from @dfort that I shared w him from last year or so -- whew good save on that one, right?   ;)

Anyhow, here's my 2nd attempt (more like 3rd or 4th, ha) at emulating QEMU on OS X 10.12.6 below...


...notice I ended up with a 'Camera was not shut down cleanly - Skipping module loading' message -- perhaps I didn't shut down the Emulator properly earlier and is there a way to 'turn off the camera' without having to force quit QEMU-system-arm manually?

and lastly what would be the 'trash' button on this keyboard? 

Because I have placed the required ML files in the virtual SD mount or at least seem to think so :P

5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10326
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #211 on: October 01, 2017, 01:37:45 PM »
...notice I ended up with a 'Camera was not shut down cleanly - Skipping module loading' message -- perhaps I didn't shut down the Emulator properly earlier and is there a way to 'turn off the camera' without having to force quit QEMU-system-arm manually?

Maybe this needs reworded in a different way? (sorry, not native English speaker)

@kichetof: "Common issues and workarounds" in the README (or watch my animations)

kichetof

  • Senior
  • ****
  • Posts: 455
  • Take a beer and enjoy it!
Re: How to run Magic Lantern into QEMU?!...
« Reply #212 on: October 02, 2017, 02:12:14 PM »
Maybe this needs reworded in a different way?

It's perfect, but, is it possible to keep the warning on the screen when you run it with QEMU ?
In my case, I didn't see the warning (too stuff in parallels :)))

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10326
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #213 on: October 02, 2017, 02:24:49 PM »
Good point, will see if that can be done without changing QEMU core code (as the shutdown behavior is hardcoded in every single GUI backend...)

edit: got it working :) atexit to the rescue...

dfort

  • Hero Member
  • *****
  • Posts: 1974
Re: How to run Magic Lantern into QEMU?!...
« Reply #214 on: October 02, 2017, 02:56:06 PM »
...notice I ended up with a 'Camera was not shut down cleanly - Skipping module loading' message -- perhaps I didn't shut down the Emulator properly earlier and is there a way to 'turn off the camera' without having to force quit QEMU-system-arm manually?

If you get that message remove remove the LOADING.LCK from the ML/modules directory and the modules will load next time you start QEMU. I had to do that every time on the EOSM2 because I couldn't find a way to do a proper camera shutdown. Development on the qemu is moving quite rapidly so make sure you update your local repository and re-install QEMU if you encounter an issue--it might already be fixed!
EOSM.202 EOSM.203 EOSM2.103 700D.115 5D3.*

kichetof

  • Senior
  • ****
  • Posts: 455
  • Take a beer and enjoy it!
Re: How to run Magic Lantern into QEMU?!...
« Reply #215 on: October 02, 2017, 03:06:19 PM »
edit: got it working :) atexit to the rescue...

You're the best!  8)
If anymore ever complains about it, make a psychedelic flash of the warning  :P

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10326
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #216 on: October 02, 2017, 07:06:39 PM »
A little Easter egg (tested on 100D and 700D, but likely on all other D5 models):

Start the camera, go to PLAY mode (without any image present) and press DELETE. Watch the console output:

Code: [Select]
ON_ERASE
open B:/AUTOEXEC.SC
Not Found B:/AUTOEXEC.SC
ffffffff

Have fun discovering the language! (hint)

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 1026
Re: How to run Magic Lantern into QEMU?!...
« Reply #217 on: October 02, 2017, 07:49:58 PM »
tried to follow http://chdk.wikia.com/wiki/Canon_Basic and used EOScard to set SCRIPT flag but somehow failing for the moment. Did you get it to work outside QEMU?
70D.112 & 100D.101

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10326
  • 5D Mark Free
Re: How to run Magic Lantern into QEMU?!...
« Reply #218 on: October 02, 2017, 07:59:55 PM »
Didn't try, but I bet this is not PowerShot Basic (so there's no point in following their guide).

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3040
Re: How to run Magic Lantern into QEMU?!...
« Reply #219 on: October 06, 2017, 11:01:32 PM »
- Even better - g3gg0 figured out how to emulate the real-time clock! This change superseded a bunch of GDB scripts - no more need to patch the date/time dialog!

for the record, got information from ricoh that the 5D3 chip is a R2262K
(thanks, guys!)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!