Author Topic: Compiling Magic Lantern on a Macintosh  (Read 46377 times)

Danne

  • Hero Member
  • *****
  • Posts: 3816
Re: Compiling Magic Lantern on a Macintosh
« Reply #150 on: September 29, 2017, 04:19:08 PM »
I see. .profile or manually copy paste. Won't be in here for a while. Maybe dfort takes a look before me.
Thanks.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3081
  • 5D3 / M1 / 7D / 70D / SL1
Re: Compiling Magic Lantern on a Macintosh
« Reply #151 on: September 29, 2017, 05:46:59 PM »
Great progress guys on troubleshooting this QEMU bad boy down. Here's a quick summary after my 2nd attempt at installing QEMU...

Code: [Select]
==> Summary
🍺  /usr/local/Cellar/isl/0.18: 80 files, 3.8MB
==> Installing mingw-w64
==> Downloading https://homebrew.bintray.com/bottles/mingw-w64-5.0.2_3.sierra.bo
######################################################################## 100.0%
==> Pouring mingw-w64-5.0.2_3.sierra.bottle.tar.gz
🍺  /usr/local/Cellar/mingw-w64/5.0.2_3: 7,834 files, 669MB


compiling qemu
0 files updated, 0 files merged, 0 files removed, 0 files unresolved

This will setup QEMU for emulating Magic Lantern.
Thou shalt not be afraid of compiling stuff on Linux ;)
Continue? [y/n] y

*** Setting up QEMU in /Users/SMJ/qemu...

--2017-09-29 08:42:03--  http://wiki.qemu-project.org/download/qemu-1.6.0.tar.bz2
Resolving wiki.qemu-project.org... 172.99.69.163
Connecting to wiki.qemu-project.org|172.99.69.163|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://wiki.qemu.org/download/qemu-1.6.0.tar.bz2 [following]
--2017-09-29 08:42:04--  http://wiki.qemu.org/download/qemu-1.6.0.tar.bz2
Resolving wiki.qemu.org... 172.99.69.163
Reusing existing connection to wiki.qemu-project.org:80.
HTTP request sent, awaiting response... 302 Found
Location: https://wiki.qemu.org/download/qemu-1.6.0.tar.bz2 [following]
--2017-09-29 08:42:06--  https://wiki.qemu.org/download/qemu-1.6.0.tar.bz2
Connecting to wiki.qemu.org|172.99.69.163|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://download.qemu.org/qemu-1.6.0.tar.bz2 [following]
--2017-09-29 08:42:07--  https://download.qemu.org/qemu-1.6.0.tar.bz2
Resolving download.qemu.org... 172.99.69.163
Connecting to download.qemu.org|172.99.69.163|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

    The file is already fully retrieved; nothing to do.

../magic-lantern/contrib/qemu/scripts/copy_back_to_contrib.sh -> ./copy_back_to_contrib.sh
../magic-lantern/contrib/qemu/scripts/gdbopts -> ./gdbopts
../magic-lantern/contrib/qemu/scripts/run_canon_fw.sh -> ./run_canon_fw.sh
../magic-lantern/contrib/qemu/scripts/run_ml.sh -> ./run_ml.sh
../magic-lantern/contrib/qemu/scripts/run_ml_100D.sh -> ./run_ml_100D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_500D.sh -> ./run_ml_500D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_50D.sh -> ./run_ml_50D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_550D.sh -> ./run_ml_550D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_5D2.sh -> ./run_ml_5D2.sh
../magic-lantern/contrib/qemu/scripts/run_ml_5D3.sh -> ./run_ml_5D3.sh
../magic-lantern/contrib/qemu/scripts/run_ml_5D3_123.sh -> ./run_ml_5D3_123.sh
../magic-lantern/contrib/qemu/scripts/run_ml_600D.sh -> ./run_ml_600D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_60D.sh -> ./run_ml_60D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_650D.sh -> ./run_ml_650D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_6D.sh -> ./run_ml_6D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_7D.sh -> ./run_ml_7D.sh
../../magic-lantern/contrib/qemu/hw/eos.c -> hw/arm/eos.c
../../magic-lantern/contrib/qemu/hw/eos.h -> hw/arm/eos.h
patching file hw/arm/Makefile.objs
patching file memory.c
patching file target-arm/cpu.c

Next steps:
===========

1) Compile QEMU

   cd /Users/SMJ/qemu/qemu-1.6.0
   ./configure --target-list=arm-softmmu --disable-docs --enable-sdl
grep: /proc/cpuinfo: No such file or directory
   make -j

2) Grab a copy of the Canon firmware from your own camera
   (don't request one and don't share it online - it's copyrighted)

   Look on your SD card - you should find ML/LOGS/ROM0.BIN and ROM1.BIN
   Copy those in /Users/SMJ/qemu/ and then run (for 60D):

   cat ROM0.BIN ROM1.BIN > ROM-60D.BIN

3) Enable CONFIG_QEMU=y in your Makefile.user
   from magic-lantern directory, then run 'make clean' to make sure
   you will rebuild ML from scratch.

4) Create a subdirectory named 'sdcard' or 'cfcard' and copy ML files there.
   Make sure all ML files are UPPERCASE (just copy them from a FAT32 card).
   You should get something like:
   /Users/SMJ/qemu/sdcard/ML/AUTOEXEC.BIN
   /Users/SMJ/qemu/sdcard/ML/FONTS/ARGNOR32.RBF
   /Users/SMJ/qemu/sdcard/ML/MODULES/ARKANOID.MO
   and so on.

5) Start emulation with:

   cd /Users/SMJ/qemu/
   ./run_ml_60D.sh

   (this will recompile ML and QEMU - handy if you edit the sources often)

Enjoy!
Seans-Mac-mini-385:~ DeafEyeJedi$



Does this mean it's some kind of a success?

Uh-oh...

Code: [Select]
Seans-Mac-mini-385:~ DeafEyeJedi$ cd /Users/SMJ/qemu/qemu-1.6.0
Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$    ./configure --target-list=arm-softmmu --disable-docs --enable-sdl

ERROR: User requested feature sdl
       configure was not able to find it

Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$ grep: /proc/cpuinfo: No such file or directory
-bash: grep:: command not found
Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$    make -j
make: *** No rule to make target `/qemu-options.hx', needed by `qemu-options.def'.  Stop.
make: *** Waiting for unfinished jobs....
Please call configure before running make!
make: *** [config-host.mak] Error 1
Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$

Did I miss a step in here?
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11208
  • 5D Mark Free
Re: Compiling Magic Lantern on a Macintosh
« Reply #152 on: September 29, 2017, 06:01:44 PM »
When all else fails, read the instructions (and replies #132, #149, or this or latest commits or the latest tweet...)

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: Compiling Magic Lantern on a Macintosh
« Reply #153 on: September 30, 2017, 05:04:48 AM »
@DeafEyeJedi -- You keep running the error message. Don't just blindly copy and paste. Read what it says first--grep is trying to tell you something is wrong!

Code: [Select]
1) Compile QEMU

   cd /Users/SMJ/qemu/qemu-1.6.0
   ./configure --target-list=arm-softmmu --disable-docs --enable-sdl
grep: /proc/cpuinfo: No such file or directory
   make -j

That said, the reason this isn't going well for you is because things have changed recently and you are trying to install QEMU without having the latest ML Mac development environment. Let's start over:

Code: [Select]
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
cd /usr/local
ls

[EDIT - removed the suggestion to use "rm -rf *"] Got anything left in there? It is ok to get rid of it if you're going to do a new Homebrew installation.

I'd do it a step at a time, you don't want to force remove everything if you're in the wrong directory!

Let's also delete the ML repository and the gcc toolchain. (That "cd" is to get you into your home directory, it is a shortcut for "cd ~/")

Code: [Select]
cd
rm -rf magic-lantern
rm -rf gcc-arm-none-eabi*

Now run the quick install script on the first post--I keep updating it with the latest suggestions.

If you answer "Y" to the QEMU install prompt it won't work the first time. The script installs a newer version of the gcc-arm-none-eabi toolchain and ends up with this message:

Code: [Select]
*** Please add gcc binaries to your executable PATH.
*** Run this command, or paste it into your .profile and reopen the terminal:
    export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** Please set up a valid arm-none-eabi-gdb before continuing.

So do exactly what that message says:

Code: [Select]
export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH
At this point my script has done all that it could and invoked the QEMU install.sh script so:

Code: [Select]
cd ~/magic-lantern
hg update qemu -C
cd contrib/qemu/
./install.sh

You will probably see something like this:

Code: [Select]
*** Will download gcc-arm-none-eabi-5_4-2016q3 from:
    https://developer.arm.com/open-source/gnu-toolchain/gnu-rm

*** Toolchain already installed in:
    ~/gcc-arm-none-eabi-5_4-2016q3

*** Please add gcc binaries to your executable PATH.
*** Run this command, or paste it into your .profile and reopen the terminal:
    export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** WARNING: 64-bit GDB is known not to work.
Continue anyway? [y/N] y

Go for it, it works on the Mac. Well there maybe some things that don't work but this should be good enough to get you started so study those links a1ex gave you and head on over to the QEMU topic and join the fun. Here's a quick tip--since you have a 100D, find the folder for that camera in your ~/qemu directory and put your firmware dumps in there, ROM0.BIN, ROM1.BIN and SFDATA.BIN - yes, you need to dump your serial flash but you know how to do that, right? Then in your terminal:

Code: [Select]
cd ~/qemu
./run_canon_fw.sh 100D

Well? Did it work?
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

Danne

  • Hero Member
  • *****
  • Posts: 3816
Re: Compiling Magic Lantern on a Macintosh
« Reply #154 on: September 30, 2017, 07:38:12 AM »
I wouldn't run rm -rf * on anything. What if the folder usr/local doesn't exist even?

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: Compiling Magic Lantern on a Macintosh
« Reply #155 on: September 30, 2017, 07:53:19 AM »
usr/local

Make sure you're in "/usr/local" -- in any case I edited my post even though I did put in a warning about using "rm -rf *"
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3081
  • 5D3 / M1 / 7D / 70D / SL1
Re: Compiling Magic Lantern on a Macintosh
« Reply #156 on: September 30, 2017, 08:31:49 AM »
Thanks @Danne for the suggestions (especially on not using rm -rf * on anything) and @dfort for the comprehensive details. I will report my findings once this shitstorm of mine settles down this weekend.

This also reminds me that I should be looking for an old used MB Air to do this kind of work where there are no important files on the system for future references.  :P
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11208
  • 5D Mark Free
Re: Compiling Magic Lantern on a Macintosh
« Reply #157 on: October 15, 2017, 08:28:37 PM »
Latest qemu branch on a fresh Mac VM:

Virtualization  8)

Code: [Select]
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install mercurial

hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg update qemu -C
cd contrib/qemu
./install.sh



Thanks kichetof 8)

kichetof

  • Senior
  • ****
  • Posts: 464
  • Take a beer and enjoy it!
Re: Compiling Magic Lantern on a Macintosh
« Reply #158 on: October 15, 2017, 08:42:55 PM »
Thanks kichetof 8)

Glad to help you  8)  minor contribution for all your great works! :)

Happy to see Mac development on top level!

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: Compiling Magic Lantern on a Macintosh
« Reply #159 on: October 16, 2017, 01:03:11 AM »
Looks like it is time to rebuild my QEMU environment. Thanks for making ML development on Mac so easy--well setting up the environment is easy, the rest is still pretty hard.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

henrus

  • New to the forum
  • *
  • Posts: 3
Re: Compiling Magic Lantern on a Macintosh
« Reply #160 on: November 16, 2017, 11:58:14 PM »
Thanks for all the effort setting up this tutorial. You guys are heros. I ran into an issue on my iMac running High Sierra (10.13.1). (Xcode 9.1 (9B55) installed but not used. From the terminal, after I proceeded with this step:

    brew tap homebrew/dupes/; brew install homebrew/versions/gcc5

I did receive the following error, though suspect it may not be a big deal:

    Error: Invalid tap name 'homebrew/dupes/'

The script went along fine, but then at this step:

    Hunk #1 succeeded at 228 with fuzz 2 (offset 12 lines).
    ==> ../configure --build=x86_64-apple-darwin17.2.0 --prefix=/usr/local/Cellar/gcc@5/5.5.0 --libdir=/usr/local/Cellar/gcc@5/5.5.0/lib/gcc/5 --enable-la
    ==> make bootstrap

everything seemed kind of stalled. The titlebar of my Terminal window was looping through multiple commands, but this kept going for like 10+ minutes. I wasn't sure if this was normal behavior or if it was stuck in some loop. I can post a screen video grab if needed. I ended up stopping all the processes and killing them. I read on a prior post here that someone said something took 37 minutes to complete, so perhaps it wasn't looping but I just need to wait for it to complete? If so, it would be useful to update the install instructions and let folks know that it may take a while to complete this step. Any confirmation would be helpful. Thanks.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: Compiling Magic Lantern on a Macintosh
« Reply #161 on: November 17, 2017, 12:55:49 AM »
Looks like there was another change at Homebrew:

Code: [Select]
Warning: Use gcc@5 instead of deprecated homebrew/versions/gcc5
I'll need to update the tutorial.

The stall you're experiencing is the compiler compiling a compiler from scratch. It takes a long time but if you let it keep running it should eventually finish and work. There should be a binary version available so that shouldn't be necessary. I'll update this topic when I get things working properly again.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

henrus

  • New to the forum
  • *
  • Posts: 3
Re: Compiling Magic Lantern on a Macintosh
« Reply #162 on: November 17, 2017, 01:01:14 AM »
Awesome. Thank you. By the way, while I was puttering around, I decided to try running the full script at the bottom of the article and that seemed to complete everything with no issues and without the long wait. Not sure why (perhaps most of the work was completed the first time through). All good now. Thanks for the quick response.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: Compiling Magic Lantern on a Macintosh
« Reply #163 on: November 17, 2017, 05:13:11 PM »
Glad you got it working. I reworked the script to accommodate that gcc@5 change in Homebrew and while I was at it improved the QEMU installation. The script can be run even if you have a development environment already setup, it won't clobber your current installation (I hope).  :P
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

kichetof

  • Senior
  • ****
  • Posts: 464
  • Take a beer and enjoy it!

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: Compiling Magic Lantern on a Macintosh
« Reply #165 on: November 17, 2017, 06:58:06 PM »
You're right -- removed the symlink. Thanks for the update.

Have you noticed that the script can now install QEMU from soup to nuts?
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3081
  • 5D3 / M1 / 7D / 70D / SL1
Re: Compiling Magic Lantern on a Macintosh
« Reply #166 on: November 17, 2017, 07:42:16 PM »
Ran the latest scripted installation from OP while on 10.13 w this MBP that already had previous compiling environments.

Love it that it can also install QEMU as an option towards the end.  8)


Seems all good to me even w the PIP update command that I ran (hence the in video above) afterwards w ease.

Thanks @dfort for the quick fix after @henrus's report. Good catch on that!
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: Compiling Magic Lantern on a Macintosh
« Reply #167 on: November 17, 2017, 08:53:15 PM »
Great video @DeafEyeJedi - I like your choice of colors.  :P
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

kichetof

  • Senior
  • ****
  • Posts: 464
  • Take a beer and enjoy it!
Re: Compiling Magic Lantern on a Macintosh
« Reply #168 on: November 17, 2017, 10:56:33 PM »
Have you noticed that the script can now install QEMU from soup to nuts?

Yeah! works well!  8)

dariSSight

  • Member
  • ***
  • Posts: 135
Re: Compiling Magic Lantern on a Macintosh
« Reply #169 on: December 23, 2017, 10:46:32 AM »
HELP! I’m trying to sink my teeth into compiling but I’m hitting a snag, I’m at the part where you ask to modify Makefile.user.default. But I don’t see that file or all the different filed you ask us to edit as we go, so I’m kind of stuck after “brew install docutils”. I think I’m going to start over again but I just wanted some advice.
Canon 5D Mark II

Danne

  • Hero Member
  • *****
  • Posts: 3816
Re: Compiling Magic Lantern on a Macintosh
« Reply #170 on: December 23, 2017, 11:08:33 AM »
Are you running the quick installation script from the bottom of the first post?
The makefile you're referring to is in the root folder. Just copy it and rename it to Makefile.user and put in the suggested changes.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: Compiling Magic Lantern on a Macintosh
« Reply #171 on: December 23, 2017, 07:26:41 PM »
Actually, you don't even need to do that. The quick installation script does it for you:

Code: [Select]
    cat <<'EOT' > Makefile.user
#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)
EOT

Here's a little secret -- you don't even need this for compiling ML. However, sooner or later you'll want to compile cr2hdr and you'll tear your hair out trying to figure out what is going on and it will always end up that you are missing this Makefile.user file.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: Compiling Magic Lantern on a Macintosh
« Reply #172 on: April 07, 2018, 03:07:43 AM »
Danne just published Compiler.app, a menu driven app that simplifies setting up and running an ML development environment.

He included the quick install script and started tweaking it to make it more user friendly. So instead of having the quick install script located at the bottom of the first post in this topic it is now on Bitbucket which means it is under full revision control and will be managed just like a "real" program. It can still be run as a stand alone script. Check the bottom of the first post to read how to download and run the script using the terminal.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112