Need help compiling on mac

Started by stevefal, August 28, 2013, 07:53:28 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

stevefal

Hallelujah, that worked. Thanks! I was able to build core ML in a snap.

However, even though CONFIG_MODULES = y, modules don't seem to get built. I've also tried in make in /modules, but I get "make: Nothing to be done for `all'."

Here's the output for making for 5D3:

make
make -C  /Users/steve/ml/magic-lantern/platform/5D3.113
[ VERSION  ]   ../../platform/5D3.113/version.bin
[ VERSION  ]   ../../platform/5D3.113/version.c
[ CC       ]   version.o
make -C ../../tcc
make[2]: Nothing to be done for `all'.
[ LD       ]   magiclantern
dietlibc.a(vsscanf.o): In function `vsscanf':
vsscanf.c:(.text+0x5c): warning: warning: the scanf functions add several kilobytes of bloat.
dietlibc.a(sprintf.o): In function `sprintf':
sprintf.c:(.text+0x14): warning: warning: Avoid *sprintf; use *snprintf. It is more secure.
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 435000 bytes
[ CC       ]   reboot.o
[ LD       ]   autoexec

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x06a430 0x000d3830 0x000d3830 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00069500 0x00069500 0x6a338 0x7d17c RWE 0x100
[ OBJCOPY  ]   autoexec.bin
[ STAT     ]   autoexec.bin
autoexec.bin: 435584 bytes
[ SYMBOLS  ]   magiclantern.sym
[ CP       ]   5D3_113.sym
Steve Falcon

a1ex

In Makefile.modules.user, select the modules you want to build.

Mine looks like this:

ML_MODULES_DYNAMIC = raw_rec file_man pic_view ettr dual_iso adtg_gui script

stevefal

Steve Falcon

mkrjf

Does anyone have a complete documented process for building on Mac?
A few people I talked to offline said they could not get it to work.

Also - the virtu of suggestion - can someone share a virtual machine image that has successfully compiled a recent nightly? That would result in immediate zero pain access!
I mean assuming people want to share their efforts
Mike

stevefal

I'm happy to give it a shot right here and keep editing until we get it right. I went down many false paths before getting it working. I also ended up installing lots of stuff that was ultimately unnecessary, at least for my purposes. I'll be back soon with my first take.
Steve Falcon

stevefal

Here goes - stevefal's brain-dead MacOS setup:

Getting the Code:

1) Install SourceTree from http://www.sourcetreeapp.com. This lets you clone the repository locally, without learning the Mercurial source control system command line interface.
2) Get an account on BitBucket.org. This will be used when you interact with the ML repository via SourceTree.
3) Click "Clone" on https://bitbucket.org/hudson/magic-lantern, and choose 'Clone in SourceTree'. Accept the defaults, and you will have the tree at /users/<you>/magic-lantern/ (hereafter shown as .../magic-lantern/

Getting required compilers, etc:

4) Make the folder /users/<you>/mltools/, and copy http://tinyurl.com/ovzk7rp into it. This 'truncate' tool is needed by the makefile.
5) Make the folder /users/<you>/mltools/gcc/ and copy the contents of  http://tinyurl.com/p2brjme (4 folder e.g. bin, lib ) into it. This is the ARM toolchain.

Setting up your PATH to include the tools:

6) Open Terminal and enter open -a TextEdit .profile to edit your environment. Use .bash_profile instead if you are using bash.
7) Add ~mltools:~/mltools/gcc/bin:~/mltools/gcc/tools to the "PATH=" statement, if it has one. If it doesn't have one, add the following two lines:
       PATH=~mltools/:~/mltools/gcc/bin:~/mltools/gcc/tools
       export PATH

8 ) Quit Terminal completely (important)

Setting up your make files:

9 ) Copy the file .../magic-lantern/Makefile.user.default and rename to Makefile.user. This is your personal make file.
10) Edit the new .../magic-lantern/Makefile.user lines:
      ~19:          SUPPORTED_MODELS:= <your camera model>
      ~25:          ARM_PATH=~/mltools/gcc
11) Create .../magic-lantern/modules/Makefile.modules.user, with the following line to build modules you want:
      ML_MODULES_DYNAMIC = <module1> <module2>     (e.g. My line reads "ML_MODULES_DYNAMIC = raw_rec")

Building:

12) To build ML, re-open Terminal, cd to .../magic-lantern and enter make. Expect messy output. Successful building produces the following files:
       .../magic-lantern/platforms/<camera>/autoexec.bin
       .../magic-lantern/platforms/<camera>/<camera>.sym
13) To build modules, cd to .../magic-lantern/modules and enter make. Module files are in each module folder:
       .../magic-lantern/modules/<module>/<module>.mo

Comments welcome and I'll edit this to correct mistakes.
Steve Falcon

dmilligan

you might be interested in this:

http://www.magiclantern.fm/forum/index.php?topic=8199.msg74756#msg74756

I also prefer brew, and would recommend using that to install ML build dependencies (I use it for a bunch of other stuff too). I didn't have any issues like your 12) and 13) I think b/c I used brew to install truncate, python, etc.

You'll have to modify the PATH setting in that Xcode project for your setup since you didn't use brew (Xcode does not setup the PATH from your bash profile).
I also set all the environment variables for everything that is typically in Makefile.user in the xcode build options, so that that file wouldn't be necessary, but Makefile.user.default does not check before overwriting the values, so some of them do not "stick" and you still need the Makefile.user. Maybe a dev would be willing to modify Makefile.user.default to not overwrite things if they are already defined in environment variables.

I came up with this little script that should do everything:

#!/bin/bash
cd
ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"
curl -OL https://launchpad.net/gcc-arm-embedded/4.7/4.7-2013-q2-update/+download/gcc-arm-none-eabi-4_7-2013q2-20130614-mac.tar.bz2
bzip2 -d gcc-arm-none-eabi-4_7-2013q2-20130614-mac.tar.bz2
tar -x -f gcc-arm-none-eabi-4_7-2013q2-20130614-mac.tar
brew install hg
hg clone -b unified https://bitbucket.org/hudson/magic-lantern
brew install truncate


I may have left off something ML needs that I already had installed on my system, if so just do

brew install name_of_missing_tool

stevefal

Although my setup has the silly fixes, it doesn't depend on installing python, ruby, brew or the hg cli. I think it will work as explained on a virgin mac. Correct me if I'm wrong.

Perhaps the build could eliminate the truncate '-s' parameters if they are unnecessary, and remove the dependency on an empty menuindexentries.h file.

My initial challenge was explanations that require prior familiarity with the toolchain. I have no experience with python, ruby, brew, yagarto, gcc, macports, and any number of tools and acronyms that can pop up along the way. So once I was building, I tried to eliminate all the seeming unnecessary stuff I'd accumulated along the way. Those steps are the result.

Maybe you can write up beginner instructions for the more robust environment.
Steve Falcon

dmilligan

Quote from: stevefal on September 10, 2013, 11:22:19 PM
Although my setup has the silly fixes, it doesn't depend on installing python, ruby, brew or the hg cli. I think it will work as explained on a virgin mac. Correct me if I'm wrong.

Yes your setup should work fine, but installing brew is so easy, and it's even scriptable, then you can script the installation of everything else. I can't really test that script b/c I don't have a "virgin" mac, but theoretically all you should have to do is install xcode from the app store and then run that script. The only thing missing is a Makefile.user, so I'll put that up on my bitbucket so that the script can download it, so now we have:

#!/bin/bash
cd
ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"
curl -OL https://launchpad.net/gcc-arm-embedded/4.7/4.7-2013-q2-update/+download/gcc-arm-none-eabi-4_7-2013q2-20130614-mac.tar.bz2
bzip2 -d gcc-arm-none-eabi-4_7-2013q2-20130614-mac.tar.bz2
tar -x -f gcc-arm-none-eabi-4_7-2013q2-20130614-mac.tar
brew install hg
hg clone -b unified https://bitbucket.org/hudson/magic-lantern
brew install truncate
cd magic-lantern
curl -OL https://bitbucket.org/dmilligan/magic-lantern/downloads/Makefile.user
make clean
make

dmilligan

So here's my 3 step instructions:

1) Install Xcode from the Mac App Store (free, but will require AppleID)

2) Install the command line tools from the Xcode Preferences

3) Open the Terminal and run this:

bash <(curl -L https://bitbucket.org/dmilligan/magic-lantern/downloads/SetupML.sh)


You should end up with a compiled magic lantern at the end of it.
(Assuming my script does everything right, I need some help testing it since I don't have a "virgin" system)

stevefal

I'm in the process of fresh-installing another mac, so I'll give it a try later.
Steve Falcon

stevefal

Almost made it. Choked on truncate:

...
Error: No available formula for truncate
...


...
make[1]: truncate: No such file or directory
...


Output: http://popspring.com/mldrop/installML1.rtf
Steve Falcon

dmilligan

ah, looks like truncate is not it's own brew package but part of coreutils

try this:

brew install coreutils


if it works I'll update the script

stevefal

I did that alone, and truncate is still not found.
Steve Falcon

dmilligan

hmmm, did brew say everything went ok? it should automatically setup your PATH for new stuff it installs

what do you get when you type this

echo $PATH


it also should have symlinks to truncate and the other stuff here:
/usr/local/opt/coreutils/libexec/gnubin

dmilligan

I think I know what's wrong, after first installing brew I think you need to run
brew doctor
and then
brew update
try that and then install coreutils again

stevefal

falconmacmini:magic-lantern stevefal$ ECHO $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

falconmacmini:magic-lantern stevefal$ brew doctor
Your system is ready to brew.
falconmacmini:magic-lantern stevefal$ brew update
Already up-to-date.
falconmacmini:magic-lantern stevefal$ brew install coreutils
Warning: coreutils-8.21 already installed

falconmacmini:magic-lantern stevefal$ make
make -C  /Users/stevefal/magic-lantern/platform/60D.111
[ VERSION  ]   ../../platform/60D.111/version.bin
make[1]: truncate: No such file or directory
make[1]: *** [../../platform/60D.111/version.bin] Error 1
make: *** [60D] Error 2
falconmacmini:magic-lantern stevefal$
Steve Falcon

dmilligan

Looks like brew didn't set the PATH, I thought that it should, maybe it didn't b/c brew doctor wasn't run b4 installing. I'm going to assume that's the case and fix the script as such, maybe somebody else can try it on a fresh system. You can try brew uninstall coreutils then brew install coreutils, if that still doesn't work set the path by hand. It may be the case that I have to add that to the script. I'm not at my computer so I can't test it but this should work:

export PATH="$PATH:$(brew --prefix coreutils)/libexec/gnubin"


(type that in terminal and also add to your bash profile)

I'm not 100% that the libexec/gnubin part is correct so double check those are the correct directory names.

stevefal

I updated my instructions to eliminate the silly fixes: http://www.magiclantern.fm/forum/index.php?topic=7936.msg74962#msg74962

I finally got the correct truncate tool, which solved all my build problems.
Steve Falcon

dmilligan

for step 7 I would recommend doing this for the PATH in your profile (this appends to PATH rather than overwriting it, also makes it only one line):

export PATH="$PATH:~mltools/:~/mltools/gcc/bin:~/mltools/gcc/tools"


so then for step 6 you can add this line to your profile with one command via the >> pipe (rather than the command to open it with text edit, then having to manually edit it):

echo "export PATH=\"\$PATH:~mltools/:~/mltools/gcc/bin:~/mltools/gcc/tools\"" >> ~/.bash_profile

k1w1

I had this problem on my OSX Mavericks MacBookPro too, and solved it by installing Python and ImageMagick under MacPorts (see http://www.imagemagick.org/script/binary-releases.php#macosx and http://astrofrog.github.io/macports-python/).

I then ran into more issues (e.g. missing MnSymbols.sy fonts), and resolved this using info found at http://codealamode.blogspot.co.nz/2012/08/installing-minionpro-for-latex-on-mac.html.

I'm now a lot closer to getting a fully error-free ML 'make docq' or 'make docs' compile finished, but still have at least one more issue to fix. ("! LaTeX Error: File `titlesec.sty' not found.").

Seems that a lot more persistence than should be necessary is needed!

Part of my problem (I think) is that I haven't found a complete (-enough) description anywhere of what the detailed pre-requisites for setting up a successful dev environment are. Some authors have described what they did to get theirs going, but I'm not aware of anything that documents exactly what is assumed to be loaded/running from the outset? Anyone who knows what that is and can spare time to write it up could save other would-be devs some time to get themselves going!? Having said that, it has been an instructive, if time-consuming, way of learning!

EOS 7DII, EOS 70D, EF 70-200 f/2.8L USM, Sigma 150-600mm Sport zoom, EF-S 60 f/2.8 macro USM, EF-S 18-135 f/3.5-5.6 IS, EF10-20mm wide angle.

k1w1

Quote from: k1w1 on August 10, 2014, 07:15:38 AM
Ok, got past that by setting in Makefile.user:

  PYTHON=python2.7

Installed montage with "sudo port install montage" but still:

  make docq
  cd /Users/steve/magic-lantern/platform/all; make docq
  cd ../../doc && RST2LATEX=rst2latex.py python2.7 mkdoc.py
  sh: montage: command not found
  make[1]: *** [docq] Error 1
  make: *** [docq] Error 2




I had this problem on my OSX Mavericks MacBookPro too, and solved it by installing ImageMagick and Python (see http://www.imagemagick.org/script/binary-releases.php#macosx and http://astrofrog.github.io/macports-python/).

I then ran into more issues (e.g. missing MnSymbols.sy fonts), and resolved this using info found at http://codealamode.blogspot.co.nz/2012/08/installing-minionpro-for-latex-on-mac.html.

I'm now a lot closer to getting a fully error-free ML 'make docq' or 'make docs' compile finished, but still have at least one more issue to fix. ("! LaTeX Error: File `titlesec.sty' not found.").

Seems that a lot more persistence than should be necessary is needed!

Part of my problem (I think) is that I haven't found a complete (-enough) description anywhere of what the detailed pre-requisites for setting up a successful dev environment are. Some authors have described what they did to get theirs going, but I'm not aware of anything that documents exactly what is assumed to be loaded/running from the outset? Anyone who knows what that is and can spare time to write it up could save other would-be devs some time to get themselves going!? Having said that, it has been an instructive, if time-consuming, way of learning!
EOS 7DII, EOS 70D, EF 70-200 f/2.8L USM, Sigma 150-600mm Sport zoom, EF-S 60 f/2.8 macro USM, EF-S 18-135 f/3.5-5.6 IS, EF10-20mm wide angle.