Author Topic: Compiling Magic Lantern on a Macintosh  (Read 39167 times)

Danne

  • Hero Member
  • *****
  • Posts: 3530
Re: Compiling Magic Lantern on a Macintosh
« Reply #150 on: September 29, 2017, 04:19:08 PM »
I see. .profile or manually copy paste. Won't be in here for a while. Maybe dfort takes a look before me.
Thanks.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3056
  • 5D3 / M1 / 7D / 70D / SL1
Re: Compiling Magic Lantern on a Macintosh
« Reply #151 on: September 29, 2017, 05:46:59 PM »
Great progress guys on troubleshooting this QEMU bad boy down. Here's a quick summary after my 2nd attempt at installing QEMU...

Code: [Select]
==> Summary
🍺  /usr/local/Cellar/isl/0.18: 80 files, 3.8MB
==> Installing mingw-w64
==> Downloading https://homebrew.bintray.com/bottles/mingw-w64-5.0.2_3.sierra.bo
######################################################################## 100.0%
==> Pouring mingw-w64-5.0.2_3.sierra.bottle.tar.gz
🍺  /usr/local/Cellar/mingw-w64/5.0.2_3: 7,834 files, 669MB


compiling qemu
0 files updated, 0 files merged, 0 files removed, 0 files unresolved

This will setup QEMU for emulating Magic Lantern.
Thou shalt not be afraid of compiling stuff on Linux ;)
Continue? [y/n] y

*** Setting up QEMU in /Users/SMJ/qemu...

--2017-09-29 08:42:03--  http://wiki.qemu-project.org/download/qemu-1.6.0.tar.bz2
Resolving wiki.qemu-project.org... 172.99.69.163
Connecting to wiki.qemu-project.org|172.99.69.163|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://wiki.qemu.org/download/qemu-1.6.0.tar.bz2 [following]
--2017-09-29 08:42:04--  http://wiki.qemu.org/download/qemu-1.6.0.tar.bz2
Resolving wiki.qemu.org... 172.99.69.163
Reusing existing connection to wiki.qemu-project.org:80.
HTTP request sent, awaiting response... 302 Found
Location: https://wiki.qemu.org/download/qemu-1.6.0.tar.bz2 [following]
--2017-09-29 08:42:06--  https://wiki.qemu.org/download/qemu-1.6.0.tar.bz2
Connecting to wiki.qemu.org|172.99.69.163|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://download.qemu.org/qemu-1.6.0.tar.bz2 [following]
--2017-09-29 08:42:07--  https://download.qemu.org/qemu-1.6.0.tar.bz2
Resolving download.qemu.org... 172.99.69.163
Connecting to download.qemu.org|172.99.69.163|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

    The file is already fully retrieved; nothing to do.

../magic-lantern/contrib/qemu/scripts/copy_back_to_contrib.sh -> ./copy_back_to_contrib.sh
../magic-lantern/contrib/qemu/scripts/gdbopts -> ./gdbopts
../magic-lantern/contrib/qemu/scripts/run_canon_fw.sh -> ./run_canon_fw.sh
../magic-lantern/contrib/qemu/scripts/run_ml.sh -> ./run_ml.sh
../magic-lantern/contrib/qemu/scripts/run_ml_100D.sh -> ./run_ml_100D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_500D.sh -> ./run_ml_500D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_50D.sh -> ./run_ml_50D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_550D.sh -> ./run_ml_550D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_5D2.sh -> ./run_ml_5D2.sh
../magic-lantern/contrib/qemu/scripts/run_ml_5D3.sh -> ./run_ml_5D3.sh
../magic-lantern/contrib/qemu/scripts/run_ml_5D3_123.sh -> ./run_ml_5D3_123.sh
../magic-lantern/contrib/qemu/scripts/run_ml_600D.sh -> ./run_ml_600D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_60D.sh -> ./run_ml_60D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_650D.sh -> ./run_ml_650D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_6D.sh -> ./run_ml_6D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_7D.sh -> ./run_ml_7D.sh
../../magic-lantern/contrib/qemu/hw/eos.c -> hw/arm/eos.c
../../magic-lantern/contrib/qemu/hw/eos.h -> hw/arm/eos.h
patching file hw/arm/Makefile.objs
patching file memory.c
patching file target-arm/cpu.c

Next steps:
===========

1) Compile QEMU

   cd /Users/SMJ/qemu/qemu-1.6.0
   ./configure --target-list=arm-softmmu --disable-docs --enable-sdl
grep: /proc/cpuinfo: No such file or directory
   make -j

2) Grab a copy of the Canon firmware from your own camera
   (don't request one and don't share it online - it's copyrighted)

   Look on your SD card - you should find ML/LOGS/ROM0.BIN and ROM1.BIN
   Copy those in /Users/SMJ/qemu/ and then run (for 60D):

   cat ROM0.BIN ROM1.BIN > ROM-60D.BIN

3) Enable CONFIG_QEMU=y in your Makefile.user
   from magic-lantern directory, then run 'make clean' to make sure
   you will rebuild ML from scratch.

4) Create a subdirectory named 'sdcard' or 'cfcard' and copy ML files there.
   Make sure all ML files are UPPERCASE (just copy them from a FAT32 card).
   You should get something like:
   /Users/SMJ/qemu/sdcard/ML/AUTOEXEC.BIN
   /Users/SMJ/qemu/sdcard/ML/FONTS/ARGNOR32.RBF
   /Users/SMJ/qemu/sdcard/ML/MODULES/ARKANOID.MO
   and so on.

5) Start emulation with:

   cd /Users/SMJ/qemu/
   ./run_ml_60D.sh

   (this will recompile ML and QEMU - handy if you edit the sources often)

Enjoy!
Seans-Mac-mini-385:~ DeafEyeJedi$



Does this mean it's some kind of a success?

Uh-oh...

Code: [Select]
Seans-Mac-mini-385:~ DeafEyeJedi$ cd /Users/SMJ/qemu/qemu-1.6.0
Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$    ./configure --target-list=arm-softmmu --disable-docs --enable-sdl

ERROR: User requested feature sdl
       configure was not able to find it

Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$ grep: /proc/cpuinfo: No such file or directory
-bash: grep:: command not found
Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$    make -j
make: *** No rule to make target `/qemu-options.hx', needed by `qemu-options.def'.  Stop.
make: *** Waiting for unfinished jobs....
Please call configure before running make!
make: *** [config-host.mak] Error 1
Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$

Did I miss a step in here?
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10543
  • 5D Mark Free
Re: Compiling Magic Lantern on a Macintosh
« Reply #152 on: September 29, 2017, 06:01:44 PM »
When all else fails, read the instructions (and replies #132, #149, or this or latest commits or the latest tweet...)

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: Compiling Magic Lantern on a Macintosh
« Reply #153 on: September 30, 2017, 05:04:48 AM »
@DeafEyeJedi -- You keep running the error message. Don't just blindly copy and paste. Read what it says first--grep is trying to tell you something is wrong!

Code: [Select]
1) Compile QEMU

   cd /Users/SMJ/qemu/qemu-1.6.0
   ./configure --target-list=arm-softmmu --disable-docs --enable-sdl
grep: /proc/cpuinfo: No such file or directory
   make -j

That said, the reason this isn't going well for you is because things have changed recently and you are trying to install QEMU without having the latest ML Mac development environment. Let's start over:

Code: [Select]
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
cd /usr/local
ls

[EDIT - removed the suggestion to use "rm -rf *"] Got anything left in there? It is ok to get rid of it if you're going to do a new Homebrew installation.

I'd do it a step at a time, you don't want to force remove everything if you're in the wrong directory!

Let's also delete the ML repository and the gcc toolchain. (That "cd" is to get you into your home directory, it is a shortcut for "cd ~/")

Code: [Select]
cd
rm -rf magic-lantern
rm -rf gcc-arm-none-eabi*

Now run the quick install script on the first post--I keep updating it with the latest suggestions.

If you answer "Y" to the QEMU install prompt it won't work the first time. The script installs a newer version of the gcc-arm-none-eabi toolchain and ends up with this message:

Code: [Select]
*** Please add gcc binaries to your executable PATH.
*** Run this command, or paste it into your .profile and reopen the terminal:
    export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** Please set up a valid arm-none-eabi-gdb before continuing.

So do exactly what that message says:

Code: [Select]
export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH
At this point my script has done all that it could and invoked the QEMU install.sh script so:

Code: [Select]
cd ~/magic-lantern
hg update qemu -C
cd contrib/qemu/
./install.sh

You will probably see something like this:

Code: [Select]
*** Will download gcc-arm-none-eabi-5_4-2016q3 from:
    https://developer.arm.com/open-source/gnu-toolchain/gnu-rm

*** Toolchain already installed in:
    ~/gcc-arm-none-eabi-5_4-2016q3

*** Please add gcc binaries to your executable PATH.
*** Run this command, or paste it into your .profile and reopen the terminal:
    export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** WARNING: 64-bit GDB is known not to work.
Continue anyway? [y/N] y

Go for it, it works on the Mac. Well there maybe some things that don't work but this should be good enough to get you started so study those links a1ex gave you and head on over to the QEMU topic and join the fun. Here's a quick tip--since you have a 100D, find the folder for that camera in your ~/qemu directory and put your firmware dumps in there, ROM0.BIN, ROM1.BIN and SFDATA.BIN - yes, you need to dump your serial flash but you know how to do that, right? Then in your terminal:

Code: [Select]
cd ~/qemu
./run_canon_fw.sh 100D

Well? Did it work?
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

Danne

  • Hero Member
  • *****
  • Posts: 3530
Re: Compiling Magic Lantern on a Macintosh
« Reply #154 on: September 30, 2017, 07:38:12 AM »
I wouldn't run rm -rf * on anything. What if the folder usr/local doesn't exist even?

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: Compiling Magic Lantern on a Macintosh
« Reply #155 on: September 30, 2017, 07:53:19 AM »
usr/local

Make sure you're in "/usr/local" -- in any case I edited my post even though I did put in a warning about using "rm -rf *"
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3056
  • 5D3 / M1 / 7D / 70D / SL1
Re: Compiling Magic Lantern on a Macintosh
« Reply #156 on: September 30, 2017, 08:31:49 AM »
Thanks @Danne for the suggestions (especially on not using rm -rf * on anything) and @dfort for the comprehensive details. I will report my findings once this shitstorm of mine settles down this weekend.

This also reminds me that I should be looking for an old used MB Air to do this kind of work where there are no important files on the system for future references.  :P
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10543
  • 5D Mark Free
Re: Compiling Magic Lantern on a Macintosh
« Reply #157 on: October 15, 2017, 08:28:37 PM »
Latest qemu branch on a fresh Mac VM:

Virtualization  8)

Code: [Select]
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install mercurial

hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg update qemu -C
cd contrib/qemu
./install.sh



Thanks kichetof 8)

kichetof

  • Senior
  • ****
  • Posts: 464
  • Take a beer and enjoy it!
Re: Compiling Magic Lantern on a Macintosh
« Reply #158 on: October 15, 2017, 08:42:55 PM »
Thanks kichetof 8)

Glad to help you  8)  minor contribution for all your great works! :)

Happy to see Mac development on top level!

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: Compiling Magic Lantern on a Macintosh
« Reply #159 on: October 16, 2017, 01:03:11 AM »
Looks like it is time to rebuild my QEMU environment. Thanks for making ML development on Mac so easy--well setting up the environment is easy, the rest is still pretty hard.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

henrus

  • New to the forum
  • *
  • Posts: 3
Re: Compiling Magic Lantern on a Macintosh
« Reply #160 on: November 16, 2017, 11:58:14 PM »
Thanks for all the effort setting up this tutorial. You guys are heros. I ran into an issue on my iMac running High Sierra (10.13.1). (Xcode 9.1 (9B55) installed but not used. From the terminal, after I proceeded with this step:

    brew tap homebrew/dupes/; brew install homebrew/versions/gcc5

I did receive the following error, though suspect it may not be a big deal:

    Error: Invalid tap name 'homebrew/dupes/'

The script went along fine, but then at this step:

    Hunk #1 succeeded at 228 with fuzz 2 (offset 12 lines).
    ==> ../configure --build=x86_64-apple-darwin17.2.0 --prefix=/usr/local/Cellar/[email protected]/5.5.0 --libdir=/usr/local/Cellar/[email protected]/5.5.0/lib/gcc/5 --enable-la
    ==> make bootstrap

everything seemed kind of stalled. The titlebar of my Terminal window was looping through multiple commands, but this kept going for like 10+ minutes. I wasn't sure if this was normal behavior or if it was stuck in some loop. I can post a screen video grab if needed. I ended up stopping all the processes and killing them. I read on a prior post here that someone said something took 37 minutes to complete, so perhaps it wasn't looping but I just need to wait for it to complete? If so, it would be useful to update the install instructions and let folks know that it may take a while to complete this step. Any confirmation would be helpful. Thanks.

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: Compiling Magic Lantern on a Macintosh
« Reply #161 on: November 17, 2017, 12:55:49 AM »
Looks like there was another change at Homebrew:

Code: [Select]
Warning: Use [email protected] instead of deprecated homebrew/versions/gcc5
I'll need to update the tutorial.

The stall you're experiencing is the compiler compiling a compiler from scratch. It takes a long time but if you let it keep running it should eventually finish and work. There should be a binary version available so that shouldn't be necessary. I'll update this topic when I get things working properly again.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

henrus

  • New to the forum
  • *
  • Posts: 3
Re: Compiling Magic Lantern on a Macintosh
« Reply #162 on: November 17, 2017, 01:01:14 AM »
Awesome. Thank you. By the way, while I was puttering around, I decided to try running the full script at the bottom of the article and that seemed to complete everything with no issues and without the long wait. Not sure why (perhaps most of the work was completed the first time through). All good now. Thanks for the quick response.

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: Compiling Magic Lantern on a Macintosh
« Reply #163 on: November 17, 2017, 05:13:11 PM »
Glad you got it working. I reworked the script to accommodate that [email protected] change in Homebrew and while I was at it improved the QEMU installation. The script can be run even if you have a development environment already setup, it won't clobber your current installation (I hope).  :P
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

kichetof

  • Senior
  • ****
  • Posts: 464
  • Take a beer and enjoy it!

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: Compiling Magic Lantern on a Macintosh
« Reply #165 on: November 17, 2017, 06:58:06 PM »
You're right -- removed the symlink. Thanks for the update.

Have you noticed that the script can now install QEMU from soup to nuts?
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3056
  • 5D3 / M1 / 7D / 70D / SL1
Re: Compiling Magic Lantern on a Macintosh
« Reply #166 on: November 17, 2017, 07:42:16 PM »
Ran the latest scripted installation from OP while on 10.13 w this MBP that already had previous compiling environments.

Love it that it can also install QEMU as an option towards the end.  8)


Seems all good to me even w the PIP update command that I ran (hence the in video above) afterwards w ease.

Thanks @dfort for the quick fix after @henrus's report. Good catch on that!
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: Compiling Magic Lantern on a Macintosh
« Reply #167 on: November 17, 2017, 08:53:15 PM »
Great video @DeafEyeJedi - I like your choice of colors.  :P
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

kichetof

  • Senior
  • ****
  • Posts: 464
  • Take a beer and enjoy it!
Re: Compiling Magic Lantern on a Macintosh
« Reply #168 on: November 17, 2017, 10:56:33 PM »
Have you noticed that the script can now install QEMU from soup to nuts?

Yeah! works well!  8)