Magic Lantern Forum

For the terminally impatient try Danne's Magic lantern development compiler tool(Mac OS) (https://www.magiclantern.fm/forum/index.php?topic=21882.0). It includes the quick install script originally written for this tutorial. You can still get just the install script and skip this tutorial--scroll down to the bottom of this first post.

Newsflash
Apple released macOS Catalina, 10.15 so these instructions no longer work. However, there are some fixes in the works (https://www.magiclantern.fm/forum/index.php?topic=16012.msg221238#msg221238).

Before we begin did you know that all Macs have development tools already installed yet they are not available to the user? If you have a "virgin" Mac that you have never used for development try this, open up a terminal (located in Applications under Utilities) and type:

which gcc

Chances are that it will return /usr/bin/gcc, gcc is the compiler to build Mac binaries and one of the tools that we'll use to build Magic Lantern. So we're all set, right? Ok, not that easy try typing in gcc to launch the compiler:


We don't really need the full Xcode development environment so simply click the highlighted "Install" button, accept the licence to activate the developer tools. If you have already installed Xcode you might still need to install the command tools. Use this terminal command:

xcode-select --install

This will install the developer tools in /Library/Developer/CommandLineTools/. These tools need to play nicely with other developer tools required by Magic Lantern. If you are using OS 10.14 (Mojave) or newer there's another step we need to take so all these tools work nicely together:

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.*.pkg

Next, there are some additional tools needed to compile Magic Lantern and several ways to install them. The simplest is to use one of the popular package managers like Fink (http://www.finkproject.org/) MacPorts (https://www.macports.org/) or Homebrew (http://brew.sh/) Homebrew is the newest of the bunch so we'll be using that. Homebrew installs the tools so they are available in /usr/local/bin which happens to be included in every new Mac's default path. However, your Mac may or may not have a /usr/local directory. Another possibility is that you have a /usr/local but don't have permissions to install anything in there. Do a directory listing to see where you stand:

QuoteRosies-Air:~ Fort$ ls -la /usr
total 8
drwxr-xr-x@   12 root  wheel    408 Oct 12 11:45 .
drwxr-xr-x    31 root  wheel   1122 Sep 30 21:56 ..
drwxr-xr-x     5 root  wheel    170 Aug 22 18:51 X11
lrwxr-xr-x     1 root  wheel      3 Sep 30 21:22 X11R6 -> X11
drwxr-xr-x     3 root  wheel    102 Aug 26 19:17 adic
drwxr-xr-x  1061 root  wheel  36074 Sep 30 21:28 bin
drwxr-xr-x   258 root  wheel   8772 Oct 12 11:45 include
drwxr-xr-x   285 root  wheel   9690 Oct 12 11:45 lib
drwxr-xr-x   183 root  wheel   6222 Sep 30 21:22 libexec
drwxr-xr-x   243 root  wheel   8262 Sep 30 21:22 sbin
drwxr-xr-x    46 root  wheel   1564 Oct 12 11:45 share
drwxr-xr-x     4 root  wheel    136 Sep 17 00:03 standalone

Notice that I used "ls -la" to get a detailed view of everything in the /usr directory. This system doesn't have a /usr/local directory so we'll have to create one. If you have a /usr/local directory but it is owned by root, you need to change permissions. Try this first:

sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local

Don't be intimidated by long command lines, you're on a Mac so simply copy and paste it into the terminal window. Note that you will have to enter your password.

If it didn't work you are probably on OS-X 10.11 "El Capitan" or 10.12 "Sierra" or even something newer so you need to temporarily turn off disable System Integrity Protection (SIP) in order to create a /usr/local you can use.

1. Restart the computer, while booting hold down Command-R to boot into recovery mode.
2. Once booted, navigate to the "Utilities > Terminal" in the top menu bar.
3. Enter csrutil disable in the terminal window and hit the return key.
4. Restart the machine and System Integrity Protection will now be disabled.


Now in the terminal enter this command:

sudo mkdir /usr/local && sudo chflags norestricted /usr/local && sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local

What you are looking for is a /usr/local directory with your name on it:

Quotedrwxr-xr-x     2 Fort  admin     68 Oct 12 12:52 local

Don't skip this next step -- go back into recovery mode and this time type csrutil enable in the terminal.

Now that we've got a /usr/local that we can use, install Homebrew. Ok--if you are really itching to get up and running you can now just skip to the bottom of this post and run the quick installation script. Go!

If you want to understand what we're doing any why, keep following this tutorial:

First of all this command will download and run a ruby script to install a basic Homebrew setup:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Next start adding some of the tools needed to build Magic Lantern.

brew install python wget mercurial

What's the deal with Python, isn't it already installed? Yes, but getting the Apple version configured to work with the ML install scripts is more trouble than it is worth so we'll just use the Homebrew version. In addition, we're going to get some python scripts using the package manager that Apple left out of their version of Python.

pip install docutils
pip2 install docutils
pip3 install docutils


That's not a mistake--we want all three versions of docutils.

If for some reason you'd rather use Apple's version of Python, I've included instructions on how to set it up at the bottom of this post.

Although the compiler that comes with the Mac can compile ML there are reasons to use a different version. Technically it isn't really gcc but another compiler called "clang" and if you try using it to compile ML you'll probably get a bunch of clang error messages. In addition, Homebrew switched to version 6 of gcc and that's causing some issues with ML so we'll go with good old version 5.

brew install gcc@5

Two versions of gcc? Won't that confuse the system? Homebrew is designed to work with the system so the gcc that you just installed is called gcc-5. This requires some special handling that we'll get to a little later.

We're not not done with compilers, we'll need yet another version of gcc to compile binaries that will run on the ARM architecture used in our Canon cameras. To figure out which ARM compiler to get refer to the ML code, specifically: Makefile.user.default -- ARM_PATH=~/gcc-arm-none-eabi-4_8-2013q4 notice that little squiggly mark after the equal sign? That means that the path starts in your home directory. Ok, so much for the lecture let's get it and put everything in its place.

cd ~ && wget -c https://launchpad.net/gcc-arm-embedded/5.0/5-2016-q3-update/+download/gcc-arm-none-eabi-5_4-2016q3-20160926-mac.tar.bz2 &&  tar -jxf gcc-arm-none-eabi-5_4-2016q3-20160926-mac.tar.bz2 &&  rm gcc-arm-none-eabi-5_4-2016q3-20160926-mac.tar.bz2

Note that some of the branches will be looking for a different version of the ARM compiler. You can install multiple versions and the build scripts will search out the preferred version. Let's get 2013-q4 because several of the development branches use that version:

cd ~ && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && tar -jxf gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && rm gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2

Before doing this next step make sure you are in the directory where you want to save the magic-lantern source code.

Finally, time to grab the ML source code and start building. There are plenty of resources on the forum but basically, to get the latest unified branch:

hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg update unified

We're almost ready to build Magic Lantern but first we've got to make one small change first. Go into the magic-lantern directory, find the file named Makefile.user.default and you'll see this block of code:

#
# Host compiler settings
#
HOST_CC=$(shell which gcc)
HOST_LD=$(shell which ld)
HOST_AR=$(shell which ar)


Now make a new file called Makefile.user and put in just these lines:

#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)


Warning: DON'T EDIT Makefile.user.default -- Put the changes you want to make to Makefile.user.default in your Makefile.user file. Think of this as a way to customize the Magic Lantern build environment without modifying the code.

Another area you might want to customize is which modules to include when making a new build. The modules that the developers decided you probably want are in the modules directory in a file called Makefile.modules.default but maybe you'd like to add another one, take for example bulb_nd. To do this without modifying any Magic Lantern code create a file in the modules directory called Makefile.modules.user and put this in it:

ML_MODULES_DYNAMIC += bulb_nd

Now let's build Magic Lantern. Here's the way I do it for the EOSM:

cd ~/magic-lantern/platform/EOSM.202/
make clean && make zip

So you want to start hacking away on the code? There's lots of choices when it comes to text editors. You can even use TextEdit though I wouldn't recommend it because you have to turn off all the "smart" features so it doesn't destroy your source code. A better choice is Aquamacs (http://aquamacs.org/). I have no financial interest if you get it or not. Hey, it is free. What about the editor that comes with Xcode? You can certainly install the full Xcode app that includes a very capable editor though it is quite a beast to master. It is also quite a heavy download. Of course if you are a real propeller head there's all the old favorites like vi, emacs, pico and nano already installed for you.

Some other software you might want to look into if you are going to get serious and contribute back to the Magic Lantern project is SourceTree (https://www.sourcetreeapp.com/). It supports the bitbucket repositories used by Magic Lantern and it has a nice graphical interface showing the various branches in ML. You're on a Mac, not everything has to be done on the terminal.

Added bonus --  Compiling Magic Lantern command line tools a.k.a. "Module Helpers"

The Magic Lantrn source code comes with some applications that you can run in the terminal or incorporate them in your own programs. Both the cr2hdr Lightroom plugin and Danne's cr2hdr-r workflows use--surprise, cr2hdr which is a part of the dual_iso module. There is also a raw2dng, mlv_dump and a few others. Your system is already configured to compile these tools. In fact if you make a build for your camera the Makefile script builds a few of them for you. For example, in the terminal go to magic-lantern/modules/dual_iso and type:

./cr2hdr

We didn't give it any instructions so it simply lists a short list of the available options that it has available. It is beyond the scope of this tutorial how to use the various command line tools but search through the forum and wiki for instructions on how to use the command line tools.

If you find something interesting in the modules you'd like to try out like say raw2dng.c and there isn't a binary for it. Check out the Makefile and see if there is a "rule" to build it. In the case of raw2dng simply type:

make raw2dng

and presto, you have a Mac binary to play with.

Extra Added bonus --  Cross compile to Windows

Let you in on a little Magic Lantern secret. Most Windows binaries aren't compiled in Windows. Sure, there are developers using Windows systems but they are either running a Linux distribution inside of a "VirtualBox" or Cygwin which is a self contained environment that functions within Windows. Only a MinGW or MinGW-64 Magic Lantern development setup might be considered Windows native. In fact if you want the best environment to work with open source projects like Magic Lantern set up a Linux system but one issue is that although a Linux system can be easily set up to compile Windows binaries it is not so easy to do Mac binaries. Anyway, to make Windows binaries on a Mac you need yet another gcc compiler like the ones maintained by the MinGW project. MinGW is short for Minimalistic GNU for Windows. What does GNU stand for? GNU is a recursive acronym for GNU's Not Unix. If you check out Apple documentation you will see that they claim that OS-X is Unix. Geek alert - instead of getting caught up in techno-babble how about setting up a cross compiler on your Mac already and start building Windows binaries.

You will need a darwin (OS-X) build of the MinGW compiler. Homebrew to the rescue.

brew install mingw-w64

This give you a full 32-bit and 64-bit Windows cross compiling setup ready to go. Now to make a Windows binary all you need to do is add the ".exe" file extension when you invoke make. For example:

Quotemake cr2hdr.exe

Being able to create Windows binaries on a Mac is so easy that I am including it on the quick installation script at the bottom of this post, though you'll need to uncomment that line first.

Tip - Removing pesky "._" files from your card

Filenames beginning with a dot are "invisible" and normally you won't even know they are there. Every time you copy files from a Mac file system to another type of filesystem it creates these "dot" files to store some information that is not normally used in other filesystems. If you are using lua on a memory card that is formatted as a FAT16 or FAT32 filesystem (usually cards under 64GB), you may see some strange messages on your screen when your camera starts up with the lua module enabled. It isn't a big deal and dmilligan put in some code to ignore files that begin with a dot but it is rather annoying. Apple actually included a utility to clean up this mess. Assuming your card is named EOS_DIGITAL, here's how to run it:

dot_clean -m /Volumes/EOS_DIGITAL

The "-m" option also removes files that start with an underbar.

Tip - Uninstalling Homebrew
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
You might also want to clean out your /usr/local directory for any stragglers. Just remember that if you are on "El Capitan" or "Sierra" and you delete /usr/local you will have to do the Command-R csrutil enable/disable song and dance to get it back.

Tip - Configuring Apple's Python to work with ML

Note: I don't use this and things might have changed so you're on your own if you want to experiment with this.

Makefile.user.default specifies that PYTHON=python2. Of course you can change that or make an alias for python2 but if you are comfortable disabling System Integrity Protection (SIP) go ahead and do that then:

ln -s /usr/bin/python /usr/bin/python2

The reason for installing the Homebrew version of Python is because Apple left out a Python utility called "pip" that makes it easy to install docutils. Follow these steps to install pip and docutils:

wget https://bootstrap.pypa.io/get-pip.py
sudo -H python get-pip.py
sudo -H pip install docutils

I like to do some cleanup. The docutil scripts are owned by root so let's change that and make sure that we've got permissions for everything in /usr/local:
sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local

You're done with the get-pip.py
rm get-pip.py

Quick installation script.

If you have a Mac with a /usr/local directory and would like to get everything up and running in a hurry, you can use Danne's menu driven compiler tool (https://www.magiclantern.fm/forum/index.php?topic=21882.0) or download just the quick install script from his repository.

Let's download and run the script using only the terminal. First we'll use curl (copy URL) because it is available on a vanilla Mac. This command copies the script from Danne's repository and saves it to a file named mac_ml.sh

curl https://bitbucket.org/Dannephoto/compiler/raw/default/Compiler.app/Contents/mac_ml.sh > mac_ml.sh

In order to make the file executable you need to change the file's properties. Here's an easy way to do it:

chmod +x mac_ml.sh

Now it is ready to run. To run it from the current current directory you can start it this way:

./mac_ml.sh

You'll need to answer a few questions, enter your password and make decisions like if you want to install the Windows cross compiler and/or QEMU.

Happy coding!

That,s it. I, m going for the El Capitano upgrade. This is too good not to. Thanks for sharing this very comprehensive description.

Damn Daniel... How'd you just all of a sudden get me into this mode when I'm just literally like "That's it -- I'm gonna jump onto the Captain's boat and just sail along with y'all" and THANK YOU for your contributions on this yet another remarkable achievement of yours for this wonderful community.

This is fairly new stuff so I've been making changes to the tutorial. After doing the section on the Widows cross compiler I realized that the stock Apple gcc compiler that comes with "El Capitan" can be used to build ML. It saves a few minutes setting up the system and avoids conflicts between the various compiler versions.

The built in Apple compiler is clang/llvm. Last I checked, it would not compile most of the command line tools (cr2hdr, raw2dng, etc.) because those tools use non-byte aligned (aka 'bit field') structs (for processing 14-bit raw data). Supposedly bit-field structs are deprecated in clang/llvm. I'm surprised you actually got it to work. Are you sure they actually compiled and are functioning correctly with clang?

Maybe the 10.11 update addressed that issue? I'm having no problems compiling those command line tools and I just checked cr2hdr and it seems to be working properly.

I've been doing some further testing and although I should have probably stopped once I had a working system I wanted to see how much I could simplify setting up the development environment. This probably isn't for experienced developers like you but rather for someone who just wants to try their hand at compiling ML. The setup requires no changes to Makefile.user.default.

There are a few more adjustments I'd like to make. I found out how to get docutils working without having to install python from Homebrew so the only packages needed are wget and mercurial.

Somewhere along the line I'm getting myself in a situation where everything works, even things that aren't supposed to. On the EOSM the dot_tune module is supposed to fail but it is building without so much as a warning message. Seems that the platform specific features are being ignored?

Figured it out. If you make other platforms that can use dot_tune then compile a platform that cannot use dot_tune it will give you a warning but still give you a dot_tune for the platform that cannot use it. Doing a "make clean" in the magic-lantern root directory fixes this problem.

Finally. I have all three binaries compiling, cr2hdr, mlv_dump, raw2dng. What a thrill! dfort, you,re the man. Thanks dmilligan for shedding light on clang. Exactly my problem also here with El Capitan.

So, I followed dfort steps but skipped the csrutil disable part after running this
sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
Gave admin rights and access right away to usr/local/bin and suddenly I can see files in usr/bin
Don,t think I can place binaries in usr/bin though. Tried copy over raw2dng. Still, compiling worked so does,t matter.
*copying files over to usr/local/bin works but not to usr/bin. I went from Yosemite to El Capitan so usr/local/bin was already there. Don,t know how a fresh El Capitan install behave.

To get rid of this
(http://s3.postimg.org/fd6cq8agf/Sk_rmavbild_2015_10_15_kl_21_25_32.jpg) (http://postimg.org/image/fd6cq8agf/)

I had to run this. Thanks dfort!
brew tap homebrew/dupes; brew install gcc --use-llvm && for f in /usr/local/bin/*-5; do mv "$f" "${f/-5/}"; done

When you get to the warning:

Code: [Select]
override r-xr-xr-x  Fort/admin for /usr/local/bin/gfortran? (y/n [n])

Couldn,t overwrite though it chose not overwritten by itself. Doesn,t matter. Compiling worked anyway, yey!

Now, how will I remember these steps in a month  8) :P ;D

Well I'm certainly not going to argue with dmilligan over which compiler to use or ignore Danne's success. I changed the instructions back to using Homebrew's latest gcc compiler.

I also included a tip on how to use the Python that is already installed with the Mac if there are any users out there that might be Apple purists.

Added a quick install script. The whole installation, including downloading the Magic Lantern source code should take about 2 minutes. Scroll to the bottom of the OP.

I also did some small changes to smooth out the installation process.

Very nice with an optional script solution . Thanks for putting it up.

Curious how long it takes to build the Windows cross-compiler? Here are the times it took for each component on a 1.4 GHz Intel Core i5 MacBook Air. It isn't nearly as painful on a more powerful system. Things will get better if the gcc-cross-mingw32 (i686) gets merged and they host a pre-compiled version. As far a I know Homebrew is also a volunteer project so like Magic Lantern, somebody has to do it an post a pull request. In the meantime if you plan on making Windows binaries on your Mac you'll just have to wait until it finishes building or someone builds it for you.


Quite a difference from the 2 minute quick install script.

... this is why I love Macintosh -- even tho no one says it anymore.

Quote from: dfort on October 15, 2015, 07:45:35 PM
Maybe the 10.11 update addressed that issue? I'm having no problems compiling those command line tools and I just checked cr2hdr and it seems to be working properly.

I just tried again compiling with clang and as expected I got this:
$ make -C modules/dual_iso/ cr2hdr
Updated HGVERSION
[ README   ]   module_strings.h
[ gcc      ]   cr2hdr
clang: error: unknown argument: '-mno-ms-bitfields'
make: *** [cr2hdr] Error 1


$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.0 (clang-700.0.72)
Target: x86_64-apple-darwin15.0.0
Thread model: posix


Perhaps you though you were compiling with clang when in fact you were using gcc?

Quote from: dmilligan on October 18, 2015, 07:59:06 PM
Perhaps you though you were compiling with clang when in fact you were using gcc?

That could very well be. I was able to test it on a virgin Mac that never had any development tools installed on it and got the same results as you. I changed to gcc a while back on your recommendation and had not had any problems with it since. I guess if someone is only going to be building ML and none of the command line tools it wouldn't matter because (AFAIK) the only thing that the host system compiler is building is xor_chk.

Added some error checking to the quick install script. Just checks to see if there is a /usr/local directory before starting the installation.

Safety first :). Well done.

Finally got it tackled down on the new MBP running El Captain and Thanks Daniel for holding up with me throughout the day!

Here are my very first Compiles (https://www.dropbox.com/sh/46f69mkmd8aqomb/AACy3Yk0p3OfvLbHCVKW0d1ka?dl=0Compiles) ...

(https://farm6.staticflickr.com/5768/22296111220_0c50f9a4db_o.png) (https://flic.kr/p/zYep5G)

... and yet more to come later!

8)

Hello! Great! Could you also make a compile for 5Dmk3 123 :))

Actually I just tried it and it wouldn't produce -- claiming that this file doesn't exist.

(https://farm1.staticflickr.com/573/22290438378_119500e7ab_n.jpg) (https://flic.kr/p/zXJjK5)

Went over to the Magic Lantern Folder in my root and here's what I noticed...

(https://farm1.staticflickr.com/680/22489139261_c7a242d0aa_n.jpg) (https://flic.kr/p/AghHAn)

So it looks like until 5D3.123 are back in the Nightlies then I can start compiling them for you or anyone else.

You need to switch to the 5D3.123 branch. Look up the Mercurial command or easier still, download the Sourcetree app and switch branches with a double click on the branch you want to work with.

Ah, I figured that it would probably had to do with needing to switch to a different branch but TBH I wasn't quite sure how exactly to tackle that part just yet.

However, I was able to change the forks and able to use yours in order to compile for EOS-M which worked well.

Thanks for that, D!

@swinxx let's say you followed instructions and cloned the main repository:
hg clone -r unified https://bitbucket.org/hudson/magic-lantern

Then you went into the platform directory--
cd magic-lantern/platform/

and found out that there is no 5Dmk3 123
Rosies-Air:platform Fort$ ls
1100D.105 7D_MASTER.203
500D.111 EOSM.202
50D.109 Makefile
550D.109 Makefile.platform
5D2.212 Makefile.platform.base
5D3.113 Makefile.platform.extras
600D.102 Makefile.platform.map
60D.111 README-unmaintained.txt
650D.104 all
6D.116 firmsum.py
700D.114 unmaintained
7D.203


But if you look at the branches in the main repository, it is there - https://bitbucket.org/hudson/magic-lantern/branches/

As long as you are somewhere in the magic-lantern directory you can switch branches:
hg update 5D3-123

If you were doing some experiments and edited a piece of code you'll get this error message:
abort: uncommitted changes
(commit or update --clean to discard changes)

so just do what it says to discard your changes:
Rosies-Air:platform Fort$ hg update --clean 5D3-123
168 files updated, 0 files merged, 53 files removed, 0 files unresolved
Rosies-Air:platform Fort$ ls
1100D.105 7D.203
500D.111 7D_MASTER.203
50D.109 EOSM.202
550D.109 Makefile
5D2.212 Makefile.platform
5D3.113 Makefile.platform.base
5D3.123 Makefile.platform.extras
600D.102 Makefile.platform.map
60D.111 README-unmaintained.txt
650D.104 all
6D.113 firmsum.py
700D.113 unmaintained


Now you can cd into the 5D3.123 directory and compile it.
make clean && make zip

Of course if you are asking someone else to do it for you, that's not the point of this topic--this is for those who want to compile ML themselves. Just download the nightly which is currently:
https://builds.magiclantern.fm/jenkins/job/5D3.123/41/artifact/platform/5D3.123/magiclantern-Nightly.2015Apr28.5D3123.zip

That's right--no changes since April. You want changes? Fork the main repository, apply some of the changes others are doing to see if they work on your platform and if they do then submit a pull request to get that branch updated. You can also try and merge the 5D-123 into unified and see what happens. Once you get something that works, please share it.

How to do all of that is a bit beyond the scope of this topic but that's basically how you can get things get done--instead of asking someone else to do it for you.

Well.. Well.. Well.. How can I say this?

I feel like I've just flown back in Time after spending hours in Terminaland refreshing my memories of how artful those codings can be!

Dreadfully Artful, that is. But hey I admit it was definitely worth the hassle and now I can see why @dfort is the man he is today. His wife is a lucky lady!  8)

He's an incredible human being that I admire him so much as if he were my 8th grade teacher from Computer Science whom I should've kept in touch with over the years.

But he joined onto this community on February 25, 2014 and I reckon his user name @dfort popping up on certain threads that were rather important so I decided to keep a close eye on him and lord behold he made the threads more valuable to not only to himself but to all of us as well.

I truly appreciate the dedication & eagerness that you proactively provide into this community.

Okay enough of my babbling already and back on topic -- I decided to stick with your latest fork (Thanks for that!) since it has all the "bells & whistles" from the branch that you referred to by PM earlier.

Here's what I was able to accomplished in order to get 5D3.123 to work (somewhat):

Added your Fork in Terminal ...

hg clone -r EOSM__working https://bitbucket.org/daniel_fort/magic-lantern

Then I typed this to pull the 5D3.123's Branch ...

hg pull && hg update 5D3-123

Then I typed this to make the Compile ...

cd magic-lantern/platform/5D3.123/
make clean && make zip

... and VOLIA!!!

Seans-Mac-mini-6:~ SMJ$ cd magic-lantern/platform/5D3.123/
Seans-Mac-mini-6:5D3.123 SMJ$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  5D3_123.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:79: remove /Users/SMJ/magic-lantern/platform/5D3.123/zip
[ RM dir ]  /Users/SMJ/magic-lantern/platform/5D3.123/zip/
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/5D3.123/stubs.o
[ VERSION  ]   ../../platform/5D3.123/version.c
[ CC       ]   version.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate-5d3.o
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ CC       ]   menuindex.o
[ CC       ]   focus.o
[ CC       ]   notify_box.o
[ CC       ]   bootflags.o
[ CC       ]   dialog_test.o
[ CC       ]   vram.o
[ CC       ]   greenscreen.o
[ CC       ]   fps-engio.o
[ CC       ]   shoot.o
[ CC       ]   hdr.o
[ CC       ]   lv-img-engio.o
[ CC       ]   state-object.o
[ CC       ]   tasks.o
[ CC       ]   vsync-lite.o
[ CC       ]   tskmon.o
[ CC       ]   battery.o
[ CC       ]   imgconv.o
[ CC       ]   histogram.o
[ CC       ]   falsecolor.o
[ CC       ]   audio-ak.o
[ CC       ]   zebra.o
../../src/zebra.c:1134:12: warning: 'zebra_digic_dirty' defined but not used [-Wunused-variable]
static int zebra_digic_dirty = 0;
            ^
[ CC       ]   vectorscope.o
[ CC       ]   beep.o
[ CC       ]   crop-mode-hack.o
[ CC       ]   ph_info_disp.o
[ CC       ]   flexinfo.o
[ CC       ]   screenshot.o
[ CC       ]   fileprefix.o
[ CC       ]   lvinfo.o
[ CC       ]   builtin-enforcing.o
[ CC       ]   ml-cbr.o
[ CC       ]   raw.o
[ CC       ]   chdk-dng.o
[ CC       ]   edmac-memcpy.o
[ CC       ]   console.o
[ CC       ]   tcc-glue.o
make -C ../../tcc
make[1]: Nothing to be done for `all'.
[ CC       ]   module.o
[ CC       ]   video_hacks.o
[ CC       ]   misc.o
[ CC       ]   afma.o
[ CC       ]   asm.o
[ AR       ]   strrchr.o
[ AR       ]   dietlibc.a
[ AR       ]   lib_a-setjmp.o
[ AR       ]   newlib-libc.a
[ CP       ]   newlib-libm.a
[ CP       ]   gcc-libgcc.a
[ LD       ]   magiclantern
[ SYMBOLS  ]   magiclantern.sym
[ CP       ]   5D3_123.sym
[ MKDIR    ]   install
[ CP       ]   install
[ VERSION  ]   ../../platform/5D3.123/version.bin
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 471120 bytes
[ CC       ]   reboot.o
[ LD       ]   autoexec

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x073148 0x000dc548 0x000dc548 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00069500 0x00069500 0x73050 0x870f4 RWE 0x100
[ OBJCOPY  ]   autoexec.bin
[ STAT     ]   autoexec.bin
autoexec.bin: 471760 bytes
cp autoexec.bin /Users/SMJ/magic-lantern/platform/5D3.123/zip/
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/fonts
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/settings
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/data
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/cropmks
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/scripts
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/logs
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/doc
cp ../../data/fonts/*.rbf /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/fonts/
cp ../../data/vram/*.lut /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/data/
cp ../../data/cropmks/*.bmp /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/cropmks/
cp ../../scripts/*.lua /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/scripts/
cp ../../scripts/*.c /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/scripts/
cp ../../scripts/*.py /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/scripts/
make -C ../../modules

Building module raw_rec...

Building module file_man...

Building module pic_view...

Building module ettr...

Building module dual_iso...
Updated HGVERSION
[ README   ]   module_strings.h
[ CC       ]   dual_iso.o
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
0000136c dual_iso_calc_dr_improvement
000013c4 dual_iso_set_enabled
000013e4 dual_iso_is_enabled
000013f4 dual_iso_is_active
00001414 dual_iso_get_dr_improvement
00001594 dual_iso_get_recovery_iso
000015b4 dual_iso_set_recovery_iso
[ DEPENDS  ]   dual_iso.dep
Will load on:
    5D3.123
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 5D3.113, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
[ gcc      ]   cr2hdr
clang: error: unknown argument: '-mno-ms-bitfields'
make[4]: *** [cr2hdr] Error 1

********************************************************
WARNING: module dual_iso failed to build, deleting
********************************************************

[ RM ]  dual_iso.o dual_iso.mo dual_iso.sym dual_iso.dep module_strings.h *.o *.d *.dep *.sym hgstamp
[ RM ]  cr2hdr cr2hdr.exe

Building module autoexpo...

Building module arkanoid...

Building module silent...

Building module dot_tune...

Building module deflick...

Building module mlv_play...

Building module mlv_rec...

Building module mlv_snd...
make -C ../../modules install

Building module raw_rec...

Building module file_man...

Building module pic_view...

Building module ettr...

Building module dual_iso...
Updated HGVERSION
[ README   ]   module_strings.h
[ CC       ]   dual_iso.o
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
0000136c dual_iso_calc_dr_improvement
000013c4 dual_iso_set_enabled
000013e4 dual_iso_is_enabled
000013f4 dual_iso_is_active
00001414 dual_iso_get_dr_improvement
00001594 dual_iso_get_recovery_iso
000015b4 dual_iso_set_recovery_iso
[ DEPENDS  ]   dual_iso.dep
Will load on:
    5D3.123
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 5D3.113, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
[ gcc      ]   cr2hdr
clang: error: unknown argument: '-mno-ms-bitfields'
make[4]: *** [cr2hdr] Error 1

********************************************************
WARNING: module dual_iso failed to build, deleting
********************************************************

[ RM ]  dual_iso.o dual_iso.mo dual_iso.sym dual_iso.dep module_strings.h *.o *.d *.dep *.sym hgstamp
[ RM ]  cr2hdr cr2hdr.exe

Building module autoexpo...

Building module arkanoid...

Building module silent...

Building module dot_tune...

Building module deflick...

Building module mlv_play...

Building module mlv_rec...

Building module mlv_snd...
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/modules || echo "no problem"
cp ../modules/*/*.mo /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/modules/ || echo "no problem"
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/doc
cp -v ../../doc/cam/* /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/doc
cp: ../../doc/cam/*: No such file or directory
make: [installdoc] Error 1 (ignored)
cp ../../doc/README.user /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/README
cp ../../doc/INSTALL.pdf /Users/SMJ/magic-lantern/platform/5D3.123/zip || echo "warning Unable to copy INSTALL.pdf"
cp: ../../doc/INSTALL.pdf: No such file or directory
warning Unable to copy INSTALL.pdf
cp ../../doc/UserGuide.pdf /Users/SMJ/magic-lantern/platform/5D3.123/zip || echo "warning Unable to copy UserGuide.pdf"
cp: ../../doc/UserGuide.pdf: No such file or directory
warning Unable to copy UserGuide.pdf
cp ../../src/logo.bmp /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/docs || echo "warning Unable to copy logo.bmp"
[ LD       ]   autoexec-fir
[ OBJCOPY  ]   autoexec-fir.bin
[ STAT     ]   autoexec-fir.bin
autoexec-fir.bin: 471760 bytes
python ../../../dumper/build_fir.py -r ../../../dumper/5D300123.FIR autoexec-fir.bin ML5D3123.FIR
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file '../../../dumper/build_fir.py': [Errno 2] No such file or directory
make: [build_fir] Error 2 (ignored)
[ RM ]  magiclantern-Nightly.2015Oct27.5D3123.zip
chmod -x /Users/SMJ/magic-lantern/platform/5D3.123/zip/autoexec.bin
cd /Users/SMJ/magic-lantern/platform/5D3.123/zip; zip -z ../magiclantern-Nightly.2015Oct27.5D3123.zip < \
../../../src/zip.txt -r .\
@printf "[ RM dir ] %s\n" " /Users/SMJ/magic-lantern/platform/5D3.123/zip/";
zip warning: name not matched: @printf
zip warning: name not matched: [ RM dir ] %s\n
zip warning: name not matched:  /Users/SMJ/magic-lantern/platform/5D3.123/zip/
  adding: autoexec.bin (deflated 49%)
  adding: ML/ (stored 0%)
  adding: ML/cropmks/ (stored 0%)
  adding: ML/cropmks/CineSco2.bmp (deflated 87%)
  adding: ML/cropmks/CrssMtr2.bmp (deflated 94%)
  adding: ML/cropmks/Passport.bmp (deflated 72%)
  adding: ML/cropmks/PhiPhoto.bmp (deflated 62%)
  adding: ML/cropmks/PhiVideo.bmp (deflated 68%)
  adding: ML/data/ (stored 0%)
  adding: ML/data/apsc8p.lut (deflated 17%)
  adding: ML/data/apsc8r.lut (deflated 7%)
  adding: ML/data/ff8p.lut (deflated 17%)
  adding: ML/data/ff8r.lut (deflated 11%)
  adding: ML/doc/ (stored 0%)
  adding: ML/docs (deflated 94%)
  adding: ML/fonts/ (stored 0%)
  adding: ML/fonts/arghlf22.rbf (deflated 90%)
  adding: ML/fonts/argnor23.rbf (deflated 84%)
  adding: ML/fonts/argnor28.rbf (deflated 84%)
  adding: ML/fonts/argnor32.rbf (deflated 86%)
  adding: ML/fonts/term12.rbf (deflated 64%)
  adding: ML/fonts/term20.rbf (deflated 81%)
  adding: ML/logs/ (stored 0%)
  adding: ML/modules/ (stored 0%)
  adding: ML/modules/5D3_123.sym (deflated 65%)
  adding: ML/modules/arkanoid.mo (deflated 51%)
  adding: ML/modules/autoexpo.mo (deflated 58%)
  adding: ML/modules/deflick.mo (deflated 53%)
  adding: ML/modules/dot_tune.mo (deflated 51%)
  adding: ML/modules/ettr.mo (deflated 56%)
  adding: ML/modules/file_man.mo (deflated 52%)
  adding: ML/modules/mlv_play.mo (deflated 62%)
  adding: ML/modules/mlv_rec.mo (deflated 53%)
  adding: ML/modules/mlv_snd.mo (deflated 56%)
  adding: ML/modules/pic_view.mo (deflated 48%)
  adding: ML/modules/raw_rec.mo (deflated 52%)
  adding: ML/modules/silent.mo (deflated 52%)
  adding: ML/README (deflated 61%)
  adding: ML/scripts/ (stored 0%)
  adding: ML/scripts/brack.c (deflated 52%)
  adding: ML/scripts/clock.c (deflated 60%)
  adding: ML/scripts/dump.lua (deflated 11%)
  adding: ML/scripts/dumpcfg.lua (deflated 28%)
  adding: ML/scripts/hello.c (deflated 27%)
  adding: ML/scripts/hello.py (deflated 42%)
  adding: ML/scripts/img.c (deflated 53%)
  adding: ML/scripts/keys.c (deflated 55%)
  adding: ML/scripts/print.lua (deflated 62%)
  adding: ML/scripts/sokoban.c (deflated 73%)
  adding: ML/scripts/test.c (deflated 69%)
  adding: ML/scripts/wb_shoot.lua (deflated 47%)
  adding: ML/settings/ (stored 0%)
  adding: ML5D3123.FIR (deflated 0%)
enter new zip file comment (end with .):
Seans-Mac-mini-6:5D3.123 SMJ$

So now I am not in the mood to upgrade my 5D3 to 123 because I am just too happy being on 113 so for those who are curious to try... Feel free to check my Dropbox if you like!

However, I noticed the "Dual-ISO Didn't Build" errors so I suspect that this is probably due to change of forks? <-- this only applies to the 5D3.123 build that I just made.

*edit* I just tested it again on another fork (unified branch) and still encountered this "Dual-ISO didn't build" error. Is there something else going on that I don't know about?

It all worked fine few days ago and now this is such a trip to me. Both Mac Mini (Yosemite) and MBP (El Captain) are producing the "Dual-ISO Didn't Build" errors from both branches... Really Strange?!

Sorry guys I'll dig into this some more tomorrow after finally catching up on my much needed sleep! I

You are using the clang compiler. When I was writing this tutorial I was experimenting to see if I could get by without having to install yet another compiler and thought it was working. You might have set up your system when I took out that step but dmilligan, got the same error that you just reported:

Quote from: dmilligan on October 18, 2015, 07:59:06 PM
I just tried again compiling with clang and as expected I got this:

Code Select Expand

$ make -C modules/dual_iso/ cr2hdr
Updated HGVERSION
[ README   ]   module_strings.h
[ gcc      ]   cr2hdr
clang: error: unknown argument: '-mno-ms-bitfields'
make: *** [cr2hdr] Error 1

Code Select Expand


$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.0 (clang-700.0.72)
Target: x86_64-apple-darwin15.0.0
Thread model: posix


Perhaps you though you were compiling with clang when in fact you were using gcc?

So make sure you get it:
brew tap homebrew/dupes; brew install gcc --use-llvm && for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}"; done

As long as you are sending out kudos--dmilligan is doing some amazing work and moving ML to a whole new level. And his development platform is a Mac.

Right. Figured. Well this is what I was dealing with all night last night and yet again this morning after following your suggestions (I did @dmilligan's instructions numerously of times) to no avails and it was rather more confusing than frustrating...

(https://farm1.staticflickr.com/755/22548955555_960e589c94_n.jpg) (https://flic.kr/p/AmzhTM)

SO I have decided to just delete the GCC and Magic-Lantern folders and just rn the script that @Danne created for you in order to start from scratch again to see if that would help clear up the problem or did the recent update for El Captain from Apple somehow caused these errors?

*edit*

Went ahead and deleted those two Folders in root and ran the script from OP which is now giving me this... How exactly can I run Software Update through Terminal?

Seans-Mac-mini-6:~ DeafEyeJedi$ cat <<'EOF' > mac_ml.sh
> #!/bin/sh
> set -e
>
> #  mac_ml.sh
> # 
> #  Script to almost automatically install
> #  and configure a Magic Lantern development
> #  environment on a Macintosh.
> # 
> #  Reference:
> #  http://magiclantern.fm/forum/index.php?topic=16012.0
> #
> #  Daniel Fort (dfort) - 2015-10-15
> #  Thanks to Danne for his input
>
> if ! test -d /usr/local; then
>   echo "/usr/local directory not found."
>   echo "Follow the steps at the beginning of"
>   echo "tutorial to create /usr/local and"
>   echo "then re-run this script."
>   exit 1
> fi
>
> echo "/usr/local found -- "
> echo "continuing with installation."
>
> cd ~
> sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
> sudo mkdir -p /usr/local/ && sudo chflags norestricted /usr/local/ && sudo chown $(whoami):admin /usr/local/ && echo sudo chown -R $(whoami):admin /usr/local/
> xcode-select --install
> ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
> ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
> brew install python wget mercurial
> pip install docutils
> brew tap homebrew/dupes; brew install gcc --use-llvm && for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}"; done
> cd ~ && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && tar -jxf gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && rm gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2
> for f in /usr/local/bin/*.py; do ln -s $f "${f/.py/}"; done
> hg clone -r unified https://bitbucket.org/hudson/magic-lantern
> exit 0
> EOF
Seans-Mac-mini-6:~ DeafEyeJedi$ bash mac_ml.sh && rm mac_ml.sh
/usr/local found --
continuing with installation.

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

Password:
sudo chown -R DeafEyeJedi:admin /usr/local/
xcode-select: error: command line tools are already installed, use "Software Update" to install updates
Seans-Mac-mini-6:~ DeafEyeJedi$


Thanks again for the follow-up, D!

The script is mostly dforts work. I merely lined up what is already in the first post. Get back on how it worked :)

Quote from: Danne on October 28, 2015, 06:19:08 PM
The script is mostly dforts work. I merely lined up what is already in the first post. Get back on how it worked :)

Still nice of you to contribute on your end as well!  :D

Anyway here's what I have done so far rather than running the script I just did everything manually step by step again from the OP.

Last login: Wed Oct 28 10:16:24 on ttys000
Seans-Mac-mini-6:~ DeafEyeJedi$ xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" to install updates
Seans-Mac-mini-6:~ DeafEyeJedi$ sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
Password:
Sorry, try again.
Password:
Seans-Mac-mini-6:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
It appears Homebrew is already installed. If your intent is to reinstall you
should do the following before running this installer again:
    ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
Seans-Mac-mini-6:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
Warning: This script will remove:
/usr/local/.git/
/usr/local/.gitignore
/usr/local/.yardopts
/usr/local/CODEOFCONDUCT.md
/usr/local/CONTRIBUTING.md
/usr/local/Cellar/
/usr/local/LICENSE.txt
/usr/local/Library/
/usr/local/README.md
/usr/local/SUPPORTERS.md
/usr/local/bin/brew
/usr/local/share/doc/homebrew/
/usr/local/share/man/man1/brew.1
Are you sure you want to uninstall Homebrew? [y/N] y
==> Removing Homebrew installation...
==> Removing empty directories...
==> Homebrew uninstalled!
The following possible Homebrew files were not deleted:
/usr/local/bin/
/usr/local/cuda/
/usr/local/etc/
/usr/local/foundry/
/usr/local/include/
/usr/local/lib/
/usr/local/share/
You may consider to remove them by yourself.
You may want to restore /usr/local's original permissions
  sudo chmod 0755 /usr/local
  sudo chgrp wheel /usr/local

Seans-Mac-mini-6:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
==> This script will install:
/usr/local/bin/brew
/usr/local/Library/...
/usr/local/share/man/man1/brew.1

Press RETURN to continue or any other key to abort
==> /usr/bin/sudo /bin/mkdir /Library/Caches/Homebrew
Password:
==> /usr/bin/sudo /bin/chmod g+rwx /Library/Caches/Homebrew
==> /usr/bin/sudo /usr/sbin/chown DeafEyeJedi /Library/Caches/Homebrew
==> Downloading and installing Homebrew...
remote: Counting objects: 3800, done.
remote: Compressing objects: 100% (3642/3642), done.
remote: Total 3800 (delta 40), reused 506 (delta 22), pack-reused 0
Receiving objects: 100% (3800/3800), 3.24 MiB | 4.98 MiB/s, done.
Resolving deltas: 100% (40/40), done.
From https://github.com/Homebrew/homebrew
* [new branch]      master     -> origin/master
Checking out files: 100% (3804/3804), done.
HEAD is now at a3e97b0 ldc: update 0.16.1 bottle.
==> Installation successful!
==> Next steps
Run `brew help` to get started
Seans-Mac-mini-6:~ DeafEyeJedi$ brew install python wget mercurial
==> Installing dependencies for python: readline, sqlite, gdbm, openssl
==> Installing python dependency: readline
==> Downloading https://homebrew.bintray.com/bottles/readline-6.3.8.yosemite.bot
######################################################################## 100.0%
==> Pouring readline-6.3.8.yosemite.bottle.tar.gz
==> Caveats
This formula is keg-only, which means it was not symlinked into /usr/local.

OS X provides the BSD libedit library, which shadows libreadline.
In order to prevent conflicts when programs look for libreadline we are
defaulting this GNU Readline installation to keg-only.


Generally there are no consequences of this for you. If you build your
own software and it requires this formula, you'll need to add to your
build variables:

    LDFLAGS:  -L/usr/local/opt/readline/lib
    CPPFLAGS: -I/usr/local/opt/readline/include

==> Summary


You're on Yosemite--at least Homebrew says you are. I didn't test this stuff out on 10.10 but it should work. All you needed to do was to install the gcc compiler. Didn't that work for you?

I did install the gcc compiler and it worked if I use nikfreak's fork but not yours for some reason?

Here I was able to compile fine for 5D3.113 but not for 5D3.123 while using his fork...

(https://farm1.staticflickr.com/736/22363691029_d9d7ac2c64_o.png) (https://flic.kr/p/A5cLcP)

However this is the error I was getting after trying to compile for 5D3.123...

Last login: Wed Oct 28 11:12:10 on ttys000
Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/platform/5D3.123/
Seans-Mac-mini-6:5D3.123 DeafEyeJedi$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  5D3_123.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:79: remove /Users/SMJ/magic-lantern/platform/5D3.123/zip
[ RM dir ]  /Users/SMJ/magic-lantern/platform/5D3.123/zip/
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/5D3.123/stubs.o
[ VERSION  ]   ../../platform/5D3.123/version.c
[ CC       ]   version.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate-5d3.o
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ CC       ]   menuindex.o
[ CC       ]   focus.o
[ CC       ]   notify_box.o
[ CC       ]   bootflags.o
[ CC       ]   dialog_test.o
[ CC       ]   vram.o
[ CC       ]   greenscreen.o
[ CC       ]   fps-engio.o
[ CC       ]   shoot.o
[ CC       ]   hdr.o
[ CC       ]   lv-img-engio.o
[ CC       ]   state-object.o
[ CC       ]   tasks.o
[ CC       ]   vsync-lite.o
[ CC       ]   tskmon.o
[ CC       ]   battery.o
[ CC       ]   imgconv.o
[ CC       ]   histogram.o
[ CC       ]   falsecolor.o
[ CC       ]   audio-ak.o
[ CC       ]   zebra.o
../../src/zebra.c:1133:12: warning: 'zebra_digic_dirty' defined but not used [-Wunused-variable]
static int zebra_digic_dirty = 0;
            ^
[ CC       ]   vectorscope.o
[ CC       ]   beep.o
[ CC       ]   crop-mode-hack.o
[ CC       ]   ph_info_disp.o
[ CC       ]   flexinfo.o
[ CC       ]   screenshot.o
[ CC       ]   fileprefix.o
[ CC       ]   lvinfo.o
[ CC       ]   builtin-enforcing.o
[ CC       ]   ml-cbr.o
[ CC       ]   raw.o
[ CC       ]   chdk-dng.o
[ CC       ]   edmac-memcpy.o
[ CC       ]   console.o
[ CC       ]   tcc-glue.o
make -C ../../tcc
make[1]: Nothing to be done for `all'.
[ CC       ]   module.o
[ CC       ]   video_hacks.o
[ CC       ]   misc.o
[ CC       ]   afma.o
[ CC       ]   asm.o
[ AR       ]   strrchr.o
[ AR       ]   dietlibc.a
[ AR       ]   lib_a-setjmp.o
[ AR       ]   newlib-libc.a
[ CP       ]   newlib-libm.a
[ CP       ]   gcc-libgcc.a
[ LD       ]   magiclantern
[ SYMBOLS  ]   magiclantern.sym
[ CP       ]   5D3_123.sym
[ MKDIR    ]   install
[ CP       ]   install
[ VERSION  ]   ../../platform/5D3.123/version.bin
make: truncate: No such file or directory
make: *** [../../platform/5D3.123/version.bin] Error 1
Seans-Mac-mini-6:5D3.123 DeafEyeJedi$


*edit#2*

Sorry guys after uninstalling and reinstalling I notice this pattern of when I successfully compile the first One and then when make clean from that one Body then I try to Compile another body which would end up giving me errors... Okay time for me to take a break and cook some eggs since I am losing protein over this battle from the past 24 hours.

Last login: Wed Oct 28 11:27:06 on ttys000
Seans-Mac-mini-6:~ DeafEyeJedi$ hg clone -r unified https://bitbucket.org/nikfreak/magic-lantern/
destination directory: magic-lantern
adding changesets
adding manifests
adding file changes                                                             
added 11300 changesets with 25057 changes to 2724 files                         
updating to branch unified
1569 files updated, 0 files merged, 0 files removed, 0 files unresolved
Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/platform/
Seans-Mac-mini-6:platform DeafEyeJedi$ cd
Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/
Seans-Mac-mini-6:magic-lantern DeafEyeJedi$ hg pull && hg update 5D3-123
pulling from https://bitbucket.org/nikfreak/magic-lantern/
searching for changes
adding changesets
adding manifests
adding file changes
added 1662 changesets with 3167 changes to 532 files (+21 heads)                                                                           
(run 'hg heads' to see heads)
254 files updated, 0 files merged, 96 files removed, 0 files unresolved
Seans-Mac-mini-6:magic-lantern DeafEyeJedi$ cd magic-lantern/platform/5D3.123/
-bash: cd: magic-lantern/platform/5D3.123/: No such file or directory
Seans-Mac-mini-6:magic-lantern DeafEyeJedi$ cd
Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/platform/5D3.123/
Seans-Mac-mini-6:5D3.123 DeafEyeJedi$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  5D3_123.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:79: remove /Users/SMJ/magic-lantern/platform/5D3.123/zip
[ RM dir ]  /Users/SMJ/magic-lantern/platform/5D3.123/zip/
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/5D3.123/stubs.o
[ VERSION  ]   ../../platform/5D3.123/version.c
[ CC       ]   version.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate-5d3.o
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ MENU IDX ]   ../../src/menuindexentries.h

No menuindex.txt! The "Complete user guide" menu is empty!
Run make docq to update menuindex.txt.

/bin/sh: truncate: command not found
make: *** [../../src/menuindexentries.h] Error 127
Seans-Mac-mini-6:5D3.123 DeafEyeJedi$ cd
Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/platform/EOSM.202/
Seans-Mac-mini-6:EOSM.202 DeafEyeJedi$ cd
Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/platform/5D3.123/
Seans-Mac-mini-6:5D3.123 DeafEyeJedi$ make clean
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  5D3_123.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM dir ]  zip
[ RM ]  *.zip
Seans-Mac-mini-6:5D3.123 DeafEyeJedi$ cd
Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/platform/EOSM.202/
Seans-Mac-mini-6:EOSM.202 DeafEyeJedi$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  EOSM_202.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:79: remove /Users/SMJ/magic-lantern/platform/EOSM.202/zip
[ RM dir ]  /Users/SMJ/magic-lantern/platform/EOSM.202/zip/
mkdir -p /Users/SMJ/magic-lantern/platform/EOSM.202/zip
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/EOSM.202/stubs.o
[ VERSION  ]   ../../platform/EOSM.202/version.c
[ CC       ]   version.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate.o
In file included from ../../src/bitrate.c:5:0:
../../src/bitrate.c: In function 'time_indicator_show':
../../src/bitrate.c:397:81: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         int fnts = FONT(FONT_SMALL, COLOR_WHITE, mvr_config.actual_qscale_maybe == -16 ? COLOR_RED : COLOR_BLACK);
                                                                                 ^
../../src/bmp.h:174:13: note: in definition of macro 'FONT'
         | ((bg) & 0xFF) << 8 \
             ^
../../src/bitrate.c:402:23: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
             mvr_config.actual_qscale_maybe < 0 ? "-" : "+",
                       ^
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ MENU IDX ]   ../../src/menuindexentries.h

No menuindex.txt! The "Complete user guide" menu is empty!
Run make docq to update menuindex.txt.

/bin/sh: truncate: command not found
make: *** [../../src/menuindexentries.h] Error 127
Seans-Mac-mini-6:EOSM.202 DeafEyeJedi$


*edit#3*

I think I give up with Yosemite on my Mac Mini -- kind of annoying to have two different OS in between devices so I'm currently upgrading to El Captain on the Mac Mini then will give another go at it.

Are there reasons why each user forks have different error results as well as random times that sometimes it works and sometimes not. Is that due to the rapid updates in the branches?

We've been PM'ing about this but others might see the same errors. For example:

Quotemake: truncate: No such file or directory

Whoa, that's some old code in that branch. Ok--so you really want to compile it? I did some searching and testing so here you go:

brew install https://raw.githubusercontent.com/artw/homebrew/master/Library/Formula/truncate.rb

That will give you truncate. It isn't a part of the main Homebrew repository so I had to figure out how to get it from the pull request. From there you should be able to do the old "make clean && make zip" that you are already familiar with. Homebrew, El Capitan and some of the ML code is somewhat new and untested but you might as well get on board with the latest--in a few weeks it will become the new norm and a few weeks after that it will be obsolete.

Haha love this stuff, D!

Ok finally updated the Mac Mini to El Captain. I then ran Terminal and this is what I am getting now... Looks like the same errors?

Last login: Thu Oct 29 00:41:13 on ttys000
Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/platform/EOSM.202/
Seans-Mac-mini-6:EOSM.202 DeafEyeJedi$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  EOSM_202.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM ]  ../../build_tools/xor_chk ../../build_tools/xor_chk.exe
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:79: remove /Users/SMJ/magic-lantern/platform/EOSM.202/zip
[ RM dir ]  /Users/SMJ/magic-lantern/platform/EOSM.202/zip/
mkdir -p /Users/SMJ/magic-lantern/platform/EOSM.202/zip
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/EOSM.202/stubs.o
[ VERSION  ]   ../../platform/EOSM.202/version.c
[ CC       ]   version.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate.o
In file included from ../../src/bitrate.c:5:0:
../../src/bitrate.c: In function 'time_indicator_show':
../../src/bitrate.c:400:81: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         int fnts = FONT(FONT_SMALL, COLOR_WHITE, mvr_config.actual_qscale_maybe == -16 ? COLOR_RED : COLOR_BLACK);
                                                                                 ^
../../src/bmp.h:174:13: note: in definition of macro 'FONT'
         | ((bg) & 0xFF) << 8 \
             ^
../../src/bitrate.c:405:23: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
             mvr_config.actual_qscale_maybe < 0 ? "-" : "+",
                       ^
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ CC       ]   menuindex.o
[ CC       ]   focus.o
[ CC       ]   notify_box.o
[ CC       ]   bootflags.o
[ CC       ]   dialog_test.o
[ CC       ]   vram.o
[ CC       ]   greenscreen.o
[ CC       ]   fps-engio.o
../../src/fps-engio.c:237:6: error: #error fixme: FPS_TIMER_B_MIN and FPS_TIMER_B_MIN are plain wrong
     #error fixme: FPS_TIMER_B_MIN and FPS_TIMER_B_MIN are plain wrong
      ^
../../src/fps-engio.c:295:12: warning: 'fps_timer_b_method' defined but not used [-Wunused-variable]
static int fps_timer_b_method = 0;
            ^
make: *** [fps-engio.o] Error 1
Seans-Mac-mini-6:EOSM.202 DeafEyeJedi$

[/quote]z

QuoteThe definition of insanity is doing the same thing and expecting different results.

../../src/fps-engio.c:237:6: error: #error fixme: FPS_TIMER_B_MIN and FPS_TIMER_B_MIN are plain wrong     #error fixme: FPS_TIMER_B_MIN and FPS_TIMER_B_MIN are plain wrong
      ^

You are trying to compile the EOSM platform from the main repository. It is intentionally blocked until the a1ex has the time to check out the timer values from this pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/649/fps_timer_a_min-and-fps_timer_b_min-fixed/diff). If you want to get it to compile it is actually very simple. Go to magic-lantern/src/fps-engio.c and look for this starting on line 232. See that last line? The one that starts with "#error" ? Simply delete that line and it will compile.

#elif defined(CONFIG_EOSM)
    #define TG_FREQ_BASE 32000000
    #define FPS_TIMER_A_MIN 520
    #undef FPS_TIMER_B_MIN
    #define FPS_TIMER_B_MIN MIN(fps_timer_b_orig, 1970)
    #error fixme: FPS_TIMER_B_MIN and FPS_TIMER_B_MIN are plain wrong


That said, it is much better compiling from my version because it has the fixes that haven't been merged in yet.

hg clone -r EOSM__working https://bitbucket.org/daniel_fort/magic-lantern

Alright here's the deal... First of all Thanks for the clarifications once again even tho you have already mentioned it to me. I must have gone insanity myself!

If so, I apologize for this but hey I got three of my cameras compiled with ease this morning from your fork branch.

(https://farm6.staticflickr.com/5788/22391030270_44c939bc0e_o.png) (https://flic.kr/p/A7BTd3)

Then once I added...

hg pull && hg update 5D3-123

That's when shit goes loose... Here's the code I am encountering no matter which camera build I try to compile even 5D3.113 that compiled just fine before this pull update... Could that very well be the culprit of this whole thing I have been trying to do?

Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/
Seans-Mac-mini-6:magic-lantern DeafEyeJedi$ hg pull && hg update 5D3-123
pulling from https://bitbucket.org/daniel_fort/magic-lantern
searching for changes
adding changesets
adding manifests
adding file changes
added 1523 changesets with 2869 changes to 642 files (+30 heads)                                                                                             
(run 'hg heads' to see heads)
170 files updated, 0 files merged, 53 files removed, 0 files unresolved
Seans-Mac-mini-6:magic-lantern DeafEyeJedi$ cd
Seans-Mac-mini-6:~ DeafEyeJedi$ cd magic-lantern/platform/5D3.113/
Seans-Mac-mini-6:5D3.113 DeafEyeJedi$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  5D3_123.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM ]  ../../build_tools/xor_chk
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:79: remove /Users/SMJ/magic-lantern/platform/5D3.113/zip
[ RM dir ]  /Users/SMJ/magic-lantern/platform/5D3.113/zip/
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.113/zip
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/5D3.123/stubs.o
[ VERSION  ]   ../../platform/5D3.123/version.c
[ CC       ]   version.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate-5d3.o
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ CC       ]   menuindex.o
[ CC       ]   focus.o
[ CC       ]   notify_box.o
[ CC       ]   bootflags.o
[ CC       ]   dialog_test.o
[ CC       ]   vram.o
[ CC       ]   greenscreen.o
[ CC       ]   fps-engio.o
[ CC       ]   shoot.o
[ CC       ]   hdr.o
[ CC       ]   lv-img-engio.o
[ CC       ]   state-object.o
[ CC       ]   tasks.o
[ CC       ]   vsync-lite.o
[ CC       ]   tskmon.o
[ CC       ]   battery.o
[ CC       ]   imgconv.o
[ CC       ]   histogram.o
[ CC       ]   falsecolor.o
[ CC       ]   audio-ak.o
[ CC       ]   zebra.o
../../src/zebra.c:1134:12: warning: 'zebra_digic_dirty' defined but not used [-Wunused-variable]
static int zebra_digic_dirty = 0;
            ^
[ CC       ]   vectorscope.o
[ CC       ]   beep.o
[ CC       ]   crop-mode-hack.o
[ CC       ]   ph_info_disp.o
[ CC       ]   flexinfo.o
[ CC       ]   screenshot.o
[ CC       ]   fileprefix.o
[ CC       ]   lvinfo.o
[ CC       ]   builtin-enforcing.o
[ CC       ]   ml-cbr.o
[ CC       ]   raw.o
[ CC       ]   chdk-dng.o
[ CC       ]   edmac-memcpy.o
[ CC       ]   console.o
[ CC       ]   tcc-glue.o
make -C ../../tcc
make[1]: Nothing to be done for `all'.
[ CC       ]   module.o
[ CC       ]   video_hacks.o
[ CC       ]   afma.o
[ CC       ]   asm.o
[ AR       ]   strrchr.o
[ AR       ]   dietlibc.a
[ AR       ]   lib_a-setjmp.o
[ AR       ]   newlib-libc.a
[ CP       ]   newlib-libm.a
[ CP       ]   gcc-libgcc.a
[ LD       ]   magiclantern
[ SYMBOLS  ]   magiclantern.sym
[ CP       ]   5D3_123.sym
[ MKDIR    ]   install
[ CP       ]   install
[ VERSION  ]   ../../platform/5D3.123/version.bin
make: truncate: No such file or directory
make: *** [../../platform/5D3.123/version.bin] Error 1
Seans-Mac-mini-6:5D3.113 DeafEyeJedi$


Lesson learned for me at least atm is to stay away from the

hg pull && hg update 5D3-123
... Such an annoying POS Monster in the making!

Thanks, D!

And round and around we go:
http://www.magiclantern.fm/forum/index.php?topic=16012.msg156286#msg156286

The 5D3-123 branch is only for that one platform--switch branches for other platforms.

Okay NOW I fully understand that you basically need to switch in between branches if I want to compile JUST for 5D3.123.

Then switch back for the rest.

Now I hence the reason why you like to use the SourceTree app -- which I downloaded recently (Thanks for the tip!) and do I just copy/paste the link to these hg clone fork branches within the app?

If so, how exactly?

P.S. Sorry for such a request but I'm currently on the road to Abel Cine to pick up New Gears for work.

Just go to bitbucket.org, create an account if you don't already have one and fork the repositories you'd like to work with. You only need to fork each repository once and you'll get all the branches. Then when you use SourceTree you can copy the repositories to your hard drive, work on them, commit your changes and push them back to bitbucket. dmilligan has several tips (http://magiclantern.fm/forum/index.php?topic=9524.0) on how to keep things organized but if you are just learning there's no harm in deleting your repositories and starting over if you make a mistake. Once you get serious and start submitting pull requests you need to keep your branches in order because anything you do to that branch will automatically show up on the pull request once you push it to bitbucket.

All this will become much clearer once you start using SourceTree. Oh, and maybe any discussion about SourceTree should be its own topic.

Thanks for the tips regarding bitbucket.org and SoureTree. I'll dig into this some more and perhaps start another topic on it if there isn't one.  :)

SO back on topic:

Decided to give it another go on the MBP just to have a clean compiler similar to how I got it working earlier this morning on my Mac Mini running El Captain.

I also understand that it worked earlier on this MBP until I messed around with the "5D3.123" poisonous branch so I am unable to run the uninstallation of HomeBrew because it says it is unable to locate it (but then when I tried installing it claims that I already have it forcing me to uninstall) but the it wont let me uninstall either because it is unable to locate Home Brew.

... and yes of course I tried running the infamous Command-R csrutil enable/disable song and dance to get it back again but that didn't seem to help either?

Last login: Thu Oct 29 16:27:20 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
Password:
Apples-Macintosh-10:~ DeafEyeJedi$
  [Restored Oct 29, 2015, 4:31:36 PM]
Last login: Thu Oct 29 16:31:36 on console
Restored session: Thu Oct 29 16:29:05 PDT 2015
Apples-Macintosh-10:~ DeafEyeJedi$
  [Restored Oct 29, 2015, 4:35:02 PM]
Last login: Thu Oct 29 16:35:01 on console
Restored session: Thu Oct 29 16:32:12 PDT 2015
Apples-Macintosh-10:~ DeafEyeJedi$ sudo mkdir /usr/local && sudo chflags norestricted /usr/local && sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
Password:
mkdir: /usr/local: File exists
Apples-Macintosh-10:~ DeafEyeJedi$
  [Restored Oct 29, 2015, 4:38:02 PM]
Last login: Thu Oct 29 16:38:00 on console
Restored session: Thu Oct 29 16:35:54 PDT 2015
Apples-Macintosh-10:~ DeafEyeJedi$
  [Restored Oct 29, 2015, 4:40:10 PM]
Last login: Thu Oct 29 16:40:08 on console
Restored session: Thu Oct 29 16:38:14 PDT 2015
Apples-Macintosh-10:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
It appears Homebrew is already installed. If your intent is to reinstall you
should do the following before running this installer again:
    ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
Apples-Macintosh-10:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
Failed to locate Homebrew!
Apples-Macintosh-10:~ DeafEyeJedi$


Meanwhile, I'll dig through the Home-Brew's website to see if there are other ways to uninstall properly. Thanks, D!


*edit#1*

This is exactly what I wanted to show you whether I try installing or uninstalling Home Brew... This feels like a game of Cat & Mouse!

Last login: Thu Oct 29 18:56:58 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
Password:
Apples-Macintosh-10:~ DeafEyeJedi$ sudo mkdir /usr/local && sudo chflags norestricted /usr/local && sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
mkdir: /usr/local: File exists
Apples-Macintosh-10:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
It appears Homebrew is already installed. If your intent is to reinstall you
should do the following before running this installer again:
    ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
Apples-Macintosh-10:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
Failed to locate Homebrew!
Apples-Macintosh-10:~ DeafEyeJedi$

It looks like the uninstall script is failing. To completly remove Homebrew delete everything in /usr/local/bin and /Library/Caches/Homebrew

Then try installing it again with:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Of course you'll also have to re-install the tools to compile ML.

brew install python wget mercurial
brew tap homebrew/dupes; brew install gcc --use-llvm && for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}"; done

And maybe:

pip install docutils

Followed those instructions (Thanks for that) and here's what I encountered...

https://vimeo.com/144198397

Last login: Fri Oct 30 23:15:27 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
It appears Homebrew is already installed. If your intent is to reinstall you
should do the following before running this installer again:
    ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
Apples-Macintosh-10:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
Failed to locate Homebrew!
Apples-Macintosh-10:~ DeafEyeJedi$


Now I find this really interesting because I managed to get it working on Mac-Mini with the same instructions but for some reason it won't let me on this MBP even though they are both running on latest El Captain.

Goodnight ... Over and Out!

*edit#1*

What the ... after researching some more about Home Brew on the internet (Google is your best friend) and here's what I found...

Apples-Macintosh-10:~ DeafEyeJedi$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install )"
==> This script will install:
/usr/local/bin/brew
/usr/local/Library/...
/usr/local/share/man/man1/brew.1

Press RETURN to continue or any other key to abort
==> Downloading and installing Homebrew...
remote: Counting objects: 3803, done.
remote: Compressing objects: 100% (3645/3645), done.
remote: Total 3803 (delta 39), reused 513 (delta 22), pack-reused 0
Receiving objects: 100% (3803/3803), 3.25 MiB | 4.66 MiB/s, done.
Resolving deltas: 100% (39/39), done.
From https://github.com/Homebrew/homebrew
* [new branch]      master     -> origin/master
HEAD is now at 5ddbedc emacs_requirement: set $EMACS
==> Installation successful!
==> Next steps
Run `brew help` to get started
Apples-Macintosh-10:~ DeafEyeJedi$


Apparently this code was the cure of this whole mess I got myself into on the MBP...

rm -rf /usr/local/Cellar /usr/local/.git && brew cleanup
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install )"

Thought it'd be worth sharing with you, D as well as for others who may run into this down the road.

*edit#2*

Not so fast... I'm trying to understand what this means here?

[code]GCC has been built with multilib support. Notably, OpenMP may not work:
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60670
If you need OpenMP support you may want to
  brew reinstall gcc --without-multilib
==> Summary

I haven't had any problems with that but if you want you could try:

brew tap homebrew/dupes; brew install gcc --use-llvm --without-multilib && for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}"; done

You might need to uninstall gcc first.

Of course if it works you don't need to fix it. Paying attention to all the warning messages will drive you crazy--if you aren't there already.

Ran the uninstallation script for GCC brew uninstall gccand then ran the Brew Install script that you suggested (Thanks again for that) and here's what I got so far even tho Terminal is still "rendering" for the longest I've ever seen (in fact it's still running and I see my disk drive space is being used) but not sure where is all of this going into... I don't see GCC in the Root as of yet...

Last login: Sat Oct 31 09:50:10 on console
Apples-Macintosh-10:~ DeafEyeJedi$ brew uninstall gcc
Uninstalling /usr/local/Cellar/gcc/5.2.0... (1353 files, 248M)
Apples-Macintosh-10:~ DeafEyeJedi$ brew tap homebrew/dupes; brew install gcc --use-llvm --without-multilib && for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}"; done
Warning: homebrew/dupes already tapped!
==> Downloading http://ftpmirror.gnu.org/gcc/gcc-5.2.0/gcc-5.2.0.tar.bz2
==> Downloading from http://gnu.mirror.vexxhost.com/gcc/gcc-5.2.0/gcc-5.2.0.tar.
######################################################################## 100.0%
==> Patching
patching file gcc/jit/Make-lang.in
patching file gcc/jit/jit-playback.c
Hunk #1 succeeded at 2459 with fuzz 2 (offset 43 lines).
==> ../configure --build=x86_64-apple-darwin15.0.0 --prefix=/usr/local/Cellar/gc
==> make bootstrap

I know this is all my fault from the very beginning as soon as I took the GCC Folder from user and deleted into Trash which now I learned that it is a BIG NO NO ... Oops I did it again!

Question -- Am I allowed to closed the Terminal window even while it's still "running" because I don't think this is right?

*edit#1*

Closed the Terminal Window (oops!) and then opened a new Terminal window and ran "Brew Uninstall" again and now this is coming up...

Last login: Sat Oct 31 10:04:28 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ brew uninstall gcc
Error: No such keg: /usr/local/Cellar/gcc
Apples-Macintosh-10:~ DeafEyeJedi$

Damn it!

It looks like the "--without-multilib" option isn't in the binary gcc distribution so it is building it from the source code. It does take a long time--a very long time. Though not as long as the times I posted (http://www.magiclantern.fm/forum/index.php?topic=16012.msg155570#msg155570) for building the cross compiler. Patience is the key word here.

Quote from: DeafEyeJedi on October 31, 2015, 06:16:00 PM
Question -- Am I allowed to closed the Terminal window even while it's still "running" because I don't think this is right?

Actually it was right. Closing the terminal will kill the process. There is a way of running a process in the background by appending an ampersand "&" at the end of a command. You won't be able to see the progress but you'll be able to continue using the terminal. I believe that on OS-X closing the terminal with background processes running will also kill those processes. Anyway, there are lots of tricks you can do with the command line but getting back to the original issue the problem is that the user (that's you DeafEyeJedi) was having a staring contest with the terminal and blinked first.

Last time I did it, it took about 3-4 hours to compile gcc.

Ahh, Thanks for the tip @dfort & @dmilligan!

Stared at the screen for literally 24 minutes this time without blinking so somehow I beat Terminal.  8)

Last login: Sun Nov  1 10:04:35 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ brew tap homebrew/dupes; brew install gcc --use-llvm --without-multilib && for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}"; done
Warning: homebrew/dupes already tapped!
==> Downloading http://ftpmirror.gnu.org/gcc/gcc-5.2.0/gcc-5.2.0.tar.bz2
Already downloaded: /Library/Caches/Homebrew/gcc-5.2.0.tar.bz2
==> Patching
patching file gcc/jit/Make-lang.in
patching file gcc/jit/jit-playback.c
Hunk #1 succeeded at 2459 with fuzz 2 (offset 43 lines).
==> ../configure --build=x86_64-apple-darwin15.0.0 --prefix=/usr/local/Cellar/gc
==> make bootstrap
==> make install
/usr/local/Cellar/gcc/5.2.0: 1285 files, 215M, built in 24.6 minutes


Then once I tried to compile for 7D.203 this is what I'm getting...

Apples-Macintosh-10:~ DeafEyeJedi$ cd magic-lantern/platform/7D.203/
Apples-Macintosh-10:7D.203 DeafEyeJedi$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  7D_203.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM ]  ../../build_tools/xor_chk ../../build_tools/xor_chk.exe
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:79: remove /Users/appleuser/magic-lantern/platform/7D.203/zip
[ RM dir ]  /Users/appleuser/magic-lantern/platform/7D.203/zip/
mkdir -p /Users/appleuser/magic-lantern/platform/7D.203/zip
[ CPP      ]   magiclantern.lds
/bin/sh: /Users/appleuser/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3: No such file or directory
make: *** [magiclantern.lds] Error 1
Apples-Macintosh-10:7D.203 DeafEyeJedi$

I noticed the gcc wasn't in the Root user so now I was able to locate it through 'Go To Folder' -- now my question is am I allowed to drag the gcc folder into the root directory?

(https://farm1.staticflickr.com/634/22699774451_416b7155bd_n.jpg) (https://flic.kr/p/AzUh6z)

Thanks for the follow-up guys!

You are looking in the wrong directory. Look at what the error message is saying--

/bin/sh: /Users/appleuser/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3: No such file or directory

You don't have the cross-compiler installed. It is supposed to be in your home directory. Did you miss this step?

cd ~ && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && tar -jxf gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && rm gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2

The screenshot you posted is pointing to a completely different compiler in a completely different location--don't move it!

Just to make sure you are running the correct gcc type:

which gcc

Make sure that it is the one in /usr/local/bin and you might also try:

gcc --version

It should return something like:

gcc (Homebrew gcc 5.2.0) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


If you get:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.0 (clang-700.1.76)
Target: x86_64-apple-darwin15.0.0
Thread model: posix


That's the Apple (clang) compiler.

Well done, Daniel!  :D

Apples-Macintosh-10:~ DeafEyeJedi$ cd ~ && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && tar -jxf gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && rm gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2
--2015-11-02 08:08:30--  https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2
Resolving launchpad.net... 91.189.89.222, 91.189.89.223
Connecting to launchpad.net|91.189.89.222|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://launchpadlibrarian.net/162333029/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 [following]
--2015-11-02 08:08:32--  https://launchpadlibrarian.net/162333029/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2
Resolving launchpadlibrarian.net... 91.189.89.228, 91.189.89.229
Connecting to launchpadlibrarian.net|91.189.89.228|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 65325285 (62M) [application/x-tar]
Saving to: 'gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2'

gcc-arm-none-eabi-4_8-2013 100%[=========================================>]  62.30M  2.51MB/s   in 51s   

2015-11-02 08:09:25 (1.22 MB/s) - 'gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2' saved [65325285/65325285]

Apples-Macintosh-10:~ DeafEyeJedi$ which gcc
/usr/local/bin/gcc
Apples-Macintosh-10:~ DeafEyeJedi$ gcc --version
gcc (Homebrew gcc 5.2.0 --without-multilib) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


So to understand this more thoroughly -- do I still keep the first gcc that i installed from the source yesterday (the one that took 24 minutes) in their respective places as well as keeping this one gcc I just installed 5 minutes ago in the root directory at the same time?

I'm only asking because the first time i got gcc working before I messed up -- I don't recall myself having to install gcc from the source or was that done automatically?

Sorry for the questions but hey it's def better than having to assume for the worse... right?  :P

Thank you very much once again, sir!

There are two different compilers, but they are both 'gcc'. One of them is to compile stuff for your computer (which has an intel processor), and one of them is to compile stuff for the camera (which has an ARM processor). You need both of them, and you need to keep them both where they are. Do not move stuff around.

Okay thanks for the clarifications, David, now this leads me to another unexpected question only because when I set to compile a certain camera model (by using Daniel's fork) and now every Module is showing this...

********************************************************
WARNING: module arkanoid failed to build, deleting
********************************************************

[ RM ]  arkanoid.o arkanoid.mo arkanoid.sym arkanoid.dep module_strings.h *.o *.d *.dep *.sym hgstamp

********************************************************
WARNING: module silent failed to build, deleting
********************************************************

[ RM ]  silent.o silent.mo silent.sym silent.dep module_strings.h *.o *.d *.dep *.sym hgstamp


********************************************************
WARNING: module mlv_play failed to build, deleting
********************************************************

[ RM ]  mlv_play.o video.bmp.rsc mlv_play.mo mlv_play.sym mlv_play.dep module_strings.h *.o *.d *.dep *.sym hgstamp

... and so on for the rest -- does this mean anything to you guys?

(https://farm1.staticflickr.com/605/22690693656_e9661f6086_n.jpg) (https://flic.kr/p/Az6JG9)

Also I am unable to post the entire Terminal report because it's more than the limited requirement for this Forum to allow such posting.

That is one of the most common issues--
rst2html: command not found.
Are you sure you followed all of my instructions?
Try this:
which rst2html
If you get nothing -- Did you install docutils? That's where we get rst2html.
pip install docutils
and then make soft links to all the python scripts that end in .py with:
for f in /usr/local/bin/*.py; do ln -s $f "${f/.py/}"; done

That's weird because I actually went back to the OP of this thread AFTER I successfully got the 2nd gcc installed into the root user.

Here's what I did...

Last login: Mon Nov  2 08:25:57 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ which rst2html
Apples-Macintosh-10:~ DeafEyeJedi$ pip install docutils
Requirement already satisfied (use --upgrade to upgrade): docutils in /usr/local/lib/python2.7/site-packages
Apples-Macintosh-10:~ DeafEyeJedi$ pip install docutils --upgrade
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(54, 'Connection reset by peer'))': /simple/docutils/
Requirement already up-to-date: docutils in /usr/local/lib/python2.7/site-packages
Apples-Macintosh-10:~ DeafEyeJedi$

Is this common to you at all? I haven't seen this one before myself...

Anyway after all is said and done... Went ahead and tried to compile again from your fork which still shows "Error, Module did not Build -- Deleted" for each and every Module... I find this strange. I wonder if it had anything to do with the Firewall through Wifi that I am using at work.

Will give it a 2nd try using my Hotspot from iPhone to see if I can bypass those so called errors...

*edit#1*

Doesn't seem to matter which Network I am using (as expected) and now even tho I did install "pip install docutils" which claims I already have them installed. However, May I just go-ahead & deleted those and reinstall them if that would to help?

Apples-Macintosh-10:~ DeafEyeJedi$ cd magic-lantern/platform/EOSM.202/
Apples-Macintosh-10:EOSM.202 DeafEyeJedi$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  EOSM_202.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM ]  ../../build_tools/xor_chk ../../build_tools/xor_chk.exe
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:79: remove /Users/appleuser/magic-lantern/platform/EOSM.202/zip
[ RM dir ]  /Users/appleuser/magic-lantern/platform/EOSM.202/zip/
mkdir -p /Users/appleuser/magic-lantern/platform/EOSM.202/zip
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/EOSM.202/stubs.o
[ VERSION  ]   ../../platform/EOSM.202/version.c
[ CC       ]   version.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate.o
In file included from ../../src/bitrate.c:5:0:
../../src/bitrate.c: In function 'time_indicator_show':
../../src/bitrate.c:400:81: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         int fnts = FONT(FONT_SMALL, COLOR_WHITE, mvr_config.actual_qscale_maybe == -16 ? COLOR_RED : COLOR_BLACK);
                                                                                 ^
../../src/bmp.h:174:13: note: in definition of macro 'FONT'
         | ((bg) & 0xFF) << 8 \
             ^
../../src/bitrate.c:405:23: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
             mvr_config.actual_qscale_maybe < 0 ? "-" : "+",
                       ^
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ CC       ]   menuindex.o
[ CC       ]   focus.o
[ CC       ]   notify_box.o
[ CC       ]   bootflags.o
[ CC       ]   dialog_test.o
[ CC       ]   vram.o
[ CC       ]   greenscreen.o
[ CC       ]   fps-engio.o
../../src/fps-engio.c:294:12: warning: 'fps_timer_b_method' defined but not used [-Wunused-variable]
static int fps_timer_b_method = 0;
            ^
[ CC       ]   shoot.o
[ CC       ]   hdr.o
[ CC       ]   lv-img-engio.o
[ CC       ]   state-object.o
[ CC       ]   tasks.o
[ CC       ]   vsync-lite.o
[ CC       ]   tskmon.o
[ CC       ]   battery.o
[ CC       ]   imgconv.o
[ CC       ]   histogram.o
[ CC       ]   falsecolor.o
[ CC       ]   audio-ak.o
[ CC       ]   zebra.o
../../src/zebra.c:173:12: warning: 'transparent_overlay_hidden' defined but not used [-Wunused-variable]
static int transparent_overlay_hidden = 0;
            ^
[ CC       ]   vectorscope.o
[ CC       ]   beep.o
[ CC       ]   crop-mode-hack.o
[ CC       ]   ph_info_disp.o
[ CC       ]   flexinfo.o
[ CC       ]   screenshot.o
[ CC       ]   fileprefix.o
[ CC       ]   lvinfo.o
[ CC       ]   builtin-enforcing.o
[ CC       ]   ml-cbr.o
[ CC       ]   raw.o
../../src/raw.c: In function 'raw_update_params_work':
../../src/raw.c:539:10: warning: #warning FIXME: are these values correct for 720p and crop modes? [-Wcpp]
         #warning FIXME: are these values correct for 720p and crop modes?
          ^
[ CC       ]   chdk-dng.o
[ CC       ]   edmac-memcpy.o
[ CC       ]   console.o
[ CC       ]   tcc-glue.o
make -C ../../tcc
make[1]: Nothing to be done for `all'.
[ CC       ]   module.o
[ AR       ]   strrchr.o
[ AR       ]   dietlibc.a
[ AR       ]   lib_a-setjmp.o
[ AR       ]   newlib-libc.a
[ CP       ]   newlib-libm.a
[ CP       ]   gcc-libgcc.a
[ LD       ]   magiclantern
[ SYMBOLS  ]   magiclantern.sym
[ CP       ]   EOSM_202.sym
[ MKDIR    ]   install
[ CP       ]   install
[ VERSION  ]   ../../platform/EOSM.202/version.bin
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 441336 bytes
[ CC       ]   reboot.o
[ CC       ]   disp_direct.o
[ CC       ]   font_direct.o
[ CC       ]   footer.o
[ LD       ]   autoexec

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x06bcd0 0x00109dd0 0x00109dd0 0x00008 0x00008 R   0x4
  LOAD           0x0000e0 0x0009e1e0 0x0009e1e0 0x6bbf8 0x7f20c RWE 0x100
[ XOR_CHK  ]   ../../build_tools/xor_chk
[ OBJCOPY  ]   autoexec.bin
[ OBJCOPY  ]   autoexec.bin
[ XOR_CHK  ]   autoexec.bin
cp autoexec.bin /Users/appleuser/magic-lantern/platform/EOSM.202/zip/
mkdir -p /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML
mkdir -p /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/fonts
mkdir -p /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/settings
mkdir -p /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/data
mkdir -p /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/cropmks
mkdir -p /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/scripts
mkdir -p /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/logs
mkdir -p /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/doc
cp ../../data/fonts/*.rbf /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/fonts/
cp ../../data/vram/*.lut /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/data/
cp ../../data/cropmks/*.bmp /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/cropmks/
cp ../../scripts/*.lua /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/scripts/
cp ../../scripts/*.c /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/scripts/
cp ../../scripts/*.py /Users/appleuser/magic-lantern/platform/EOSM.202/zip/ML/scripts/
make -C ../../modules

Building module raw_rec...
Updated HGVERSION
[ README   ]   module_strings.h
$//"README.rst | grep -v -E "^:([^:])+:.+$" | rst2html --no-xml-declaration | python ../html2text.py -b 700 | sed "s/



/bin/sh: rst2html: command not found

(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x100c72cf8>)
make[4]: *** [module_strings.h] Error 1

********************************************************
WARNING: module raw_rec failed to build, deleting
********************************************************

[ RM ]  raw_rec.o raw_rec.mo raw_rec.sym raw_rec.dep module_strings.h *.o *.d *.dep *.sym hgstamp
[ RM ]  raw2dng raw2dng.exe dng2raw dng2raw.exe

It looks like docutils is installed, those error messages had to do with the "--upgrade" option that you didn't really have to do. As far as your development environment not working yet---

/bin/sh: rst2html: command not found
Same problem. Did you do as I told you?
which rst2html
If that comes up empty try:
which rst2html.py
If you get:
/usr/local/bin/rst2html.py
That means you skipped one of my steps:
for f in /usr/local/bin/*.py; do ln -s $f "${f/.py/}"; done

I have to thank you, dfort! Installation on Hackintosh following your clear and detailed instructions worked like a charm. Was able to compile a1ex' latest branch (see here (https://bitbucket.org/hudson/magic-lantern/pull-requests/665/650d-700d-eos-m-cleaner-method-for-hiding/diff#comment-11575541)).

You're welcome--and thank you WalterSchulz for doing what you do.

It worked on a Hacintosh? Great! I've got some other tutorials for Windows users too.

I wanted to give something back to the ML community and since there were several requests for special builds I felt there was a need to demystify compiling. Now if someone could demystify reverse engineering....

Hey guys! I upgraded to El Capitan on my main computer and installed your fine compilation through script. Unfortunately I get the "clang" related error. I can,t really remember how I fixed it on my other computer, hmm

Maybe the compiler didn't install properly. Try running this:

brew tap homebrew/dupes; brew install gcc --use-llvm && for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}"; done

Tried that a few times. Too new build? Check it.

Warning: Tap homebrew/dupes already tapped.
Warning: gcc-5.3.0 already installed
mv: rename /usr/local/bin/*-5 to /usr/local/bin/*: No such file or directory

What are you trying to compile that is giving you the clang errors? Also, what exactly are those error messages trying to tell you?

Are you getting this when you enter "gcc --version"?

$ gcc --version
gcc (Homebrew gcc 5.2.0) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Oops--looks like I didn't update my system for a while, you're on gcc-5.3.0.

Should I not update the gcc then after all?

There,s that clang thingy again. I should dive in a little here.

gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin15.2.0
Thread model: posix

So when trying make on mlv_dump

make mlv_dump
[ HOST_CC  ]   mlv_dump.host.o
clang: error: unknown argument: '-mno-ms-bitfields'
make: *** [mlv_dump.host.o] Error 1

Had the exact same error on mavericks before. Suppose my upgrade to El Capitan kept my problems. Solution here? Uninstall and reinstall something?

Make cr2hdr gives following

make cr2hdr
Updated HGVERSION
[ README   ]   module_strings.h
cat README.rst | grep -v -E "^:([^:])+:.+$" | rst2html --no-xml-declaration | py$//" ../html2text.py -b 700 | sed "s/



Traceback (most recent call last):
  File "/opt/local/bin/rst2html", line 17, in <module>
    from docutils.core import publish_cmdline, default_description
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/docutils/core.py", line 20, in <module>
    from docutils import frontend, io, utils, readers, writers
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/docutils/frontend.py", line 41, in <module>
    import docutils.utils
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/docutils/utils/__init__.py", line 20, in <module>
    import docutils.io
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/docutils/io.py", line 18, in <module>
    from docutils.utils.error_reporting import locale_encoding, ErrorString, ErrorOutput
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/docutils/utils/error_reporting.py", line 47, in <module>
    locale_encoding = locale.getlocale()[1] or locale.getdefaultlocale()[1]
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 543, in getdefaultlocale
    return _parse_localename(localename)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 475, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x10e4cd320>)
make: *** [module_strings.h] Error 1


make raw2dng

make raw2dng
Makefile:27: warning: overriding commands for target `raw2dng.exe'
Makefile:22: warning: ignoring old commands for target `raw2dng.exe'
[ HOST_CC  ]   raw2dng
[ HOST_CC  ]   raw2dng
clang: error: unknown argument: '-mno-ms-bitfields'
make: *** [raw2dng] Error 1

I prefer not to replace the clang gcc with gnu gcc. By default I think homebrew actually installs gnu gcc to something like gcc5 or maybe gcc-5 (just type gcc then hit tab twice to find out) and leaves clang gcc alone. This is a good idea. All you need to do is change your makefile.user thusly:

HOST_CC = gcc5


After hitting tab twice I get this
gcc         gcc-ar      gcc-nm      gcc-ranlib


Tried changing in the Makefile.user.default first to as suggested to HOST_CC = gcc5

and then to any of the other gcc names that entered in terminal.

make mlv_dump
[ HOST_CC  ]   mlv_dump.host.o
/bin/sh: gcc5: command not found
make: *** [mlv_dump.host.o] Error 127
Daniels-MacBook-Pro:mlv_rec Daniel$ make mlv_dump
[ HOST_CC  ]   mlv_dump.host.o
gcc-ar: Cannot find plugin 'liblto_plugin.so'
make: *** [mlv_dump.host.o] Error 1
Daniels-MacBook-Pro:mlv_rec Daniel$ make mlv_dump
[ HOST_CC  ]   mlv_dump.host.o
clang: error: unknown argument: '-mno-ms-bitfields'
make: *** [mlv_dump.host.o] Error 1
Daniels-MacBook-Pro:mlv_rec Daniel$ make mlv_dump
[ HOST_CC  ]   mlv_dump.host.o
gcc-nm: Cannot find plugin 'liblto_plugin.so'
make: *** [mlv_dump.host.o] Error 1
Daniels-MacBook-Pro:mlv_rec Daniel$ make mlv_dump
[ HOST_CC  ]   mlv_dump.host.o
gcc-ranlib: Cannot find plugin 'liblto_plugin.so'
make: *** [mlv_dump.host.o] Error 1
Daniels-MacBook-Pro:mlv_rec Daniel$

This:

for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}";

is probably not a good idea. I'm guessing the purpose of this command is to rename the homebrew gcc from gcc-5 to just gcc. It's really not a good idea to try and outsmart your package manager, things will only get screwed up worse (plus this command could potentially touch other stuff unrelated to gcc, that you didn't really wanted moved around).

My guess for what is going on here, Danne, is that you have multiple gccs on your path since homebrew is reporting it has already installed gcc. You have the clang gcc from apple, and you have the gnu gcc from homebrew, renamed from gcc-5 to just gcc. Because of your specific path order, the clang gcc is being found first. Try this:

/usr/local/bin/gcc --version


I would suggest moving the gcc executables in /usr/local/bin/ back to gcc-5, and change your Makefile.user (not Makefile.user.default) to this

HOST_CC = gcc-5


If you do this, you won't have to try and outsmart homebrew and run into these kinds of issues.

Basically you want this:

$gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin15.2.0
Thread model: posix

$gcc-5 --version
gcc (Homebrew gcc 5.3.0) 5.3.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


On a clean system it should be as simple as installing gcc via homebrew the normal way, and updating Makefile.user to use gcc-5.

I got it to compile a couple of hours ago. I will put up a thorough description later. Thanks again for your help dmilligan and dfort.

First I unistalled and reinstalled gcc using brew
brew uninstall gcc
brew install gcc

Checking my /usr/local/bin and /usr/local/Cellar/gcc/5.3.0/bin/ directory it seemed ok.

(http://s15.postimg.org/xzyv7k7a3/usr_local_bin.png)

(http://s15.postimg.org/50925be1n/usr_local_Cellar_gcc_5_3_0_bin.png)

Looking for gcc versions I found only
gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin15.2.0
Thread model: posix

When looking for below version it found nothing
gcc-5 --version

(http://s15.postimg.org/mc9eqr7iz/01_magic_lantern.png)

Had to create a makefile.user since I only have makefile.user.default. Changed to HOST=gcc-5
Tried running make mlv_dump still it couldn,t find the correct gcc-5.

I asked dfort and another idea from him about my path not being correct and some commands I could try so here is what made it finally work with the new makefile.user document.

Solution

Tip from dfort. I checked my paths
echo $PATH
/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

Then I wrote
cat ~/.profile
Which gave me this
# MacPorts Installer addition on 2015-10-02_at_10:36:46: adding an appropriate PATH variable for use with MacPorts.
export PATH="/opt/local/bin:/opt/local/sbin:$PATH"
# Finished adapting your PATH environment variable for use with MacPorts.

Then I decided to fool around with a path command I use in MLP
mlv_rec Daniel$ export PATH="/usr/local/bin:$PATH" make mlv_dump
Daniels-MacBook-Pro:mlv_rec Daniel$ make mlv_dump

And finally tried this and all worked. After this it compiles and it finds the gcc-5 --version fine in terminal
Daniels-MacBook-Pro:mlv_rec Daniel$ export PATH="/usr/local/bin:$PATH" make mlv_dump
Daniels-MacBook-Pro:mlv_rec Daniel$ make mlv_dump
[ HOST_CC  ]   mlv_dump.host.o
mlv_dump.c: In function 'chroma_smooth':
mlv_dump.c:946:31: warning: passing argument 1 of 'chroma_smooth_2x2' from incompatible pointer type [-Wincompatible-pointer-types]
             chroma_smooth_2x2(aux, aux2, raw2ev, ev2raw);


Conclusion. Not finding the path to gcc-5 but somehow a path fiddling got the paths straight. This seems to have been my issue also in mavericks.
Not sure if it was the cat ~/.profile command or the export PATH="/usr/local/bin:$PATH" that fixed the issue.
Glad that I finally got it right. Thanks again guys!

Glad you got it working Danne. I've got to take another look into that hack that I've got in my setup instructions. (I'll admit it is a hack, as in an inelegant solution.) The reason I did that is because I've had too many issues with the Apple compiler and setting it up this way would also allow me compile ML without touching one line of code. I don't have much coding experience so that appealed to me. The default path for a new Mac has /usr/local/bin ahead of /usr/bin but Danne's system has that reversed and that's why the hack didn't work.

dmilligan is right when he said that it isn't a good idea. This actually bit me last night when I updated Homebrew and though it updated gcc to 5.3.0 my gcc was still pointing to 5.2.0. That's not really a big deal for someone who just wants to compile ML. Once you got a system that can compile ML there shouldn't be any need to update the development environment unless the source code is changed to require newer development tools. There is work underway that will update the ARM compiler to gcc-arm-none-eabi-4_9-2015q3, we're still using the one from 2013q4 but that hasn't been merged into the unified branch yet so don't update the ARM compiler just yet.

In any case, I'll change the setup instructions with some gcc caveats and options after I chew on it a while.

Am glad you got it working on your end as well ... However I decided to give it another go on my MBP and now I am back to square one with errors looking like this... Here's a portion of it.

Apples-Macintosh-10:~ DeafEyeJedi$ cd magic-lantern/platform/5D3.113/
Apples-Macintosh-10:5D3.113 DeafEyeJedi$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  5D3_113.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM ]  ../../build_tools/xor_chk ../../build_tools/xor_chk.exe
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:80: remove /Users/appleuser/magic-lantern/platform/5D3.113/zip
[ RM dir ]  /Users/appleuser/magic-lantern/platform/5D3.113/zip/
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/5D3.113/stubs.o
[ VERSION  ]   ../../platform/5D3.113/version.c
[ CC       ]   version.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate-5d3.o
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ CC       ]   menuindex.o
[ CC       ]   focus.o
[ CC       ]   notify_box.o
[ CC       ]   bootflags.o
[ CC       ]   dialog_test.o
[ CC       ]   vram.o
[ CC       ]   greenscreen.o
[ CC       ]   fps-engio.o
[ CC       ]   shoot.o
[ CC       ]   hdr.o
[ CC       ]   lv-img-engio.o
[ CC       ]   state-object.o
[ CC       ]   tasks.o
[ CC       ]   vsync-lite.o
[ CC       ]   tskmon.o
[ CC       ]   battery.o
[ CC       ]   imgconv.o
[ CC       ]   histogram.o
[ CC       ]   falsecolor.o
[ CC       ]   audio-ak.o
[ CC       ]   zebra.o
[ CC       ]   vectorscope.o
[ CC       ]   beep.o
[ CC       ]   crop-mode-hack.o
[ CC       ]   ph_info_disp.o
[ CC       ]   flexinfo.o
[ CC       ]   screenshot.o
[ CC       ]   fileprefix.o
[ CC       ]   lvinfo.o
[ CC       ]   builtin-enforcing.o
[ CC       ]   ml-cbr.o
[ CC       ]   raw.o
[ CC       ]   chdk-dng.o
[ CC       ]   edmac-memcpy.o
[ CC       ]   console.o
[ CC       ]   tcc-glue.o
make -C ../../tcc
make[1]: Nothing to be done for `all'.
[ CC       ]   module.o
[ CC       ]   video_hacks.o
[ CC       ]   afma.o
[ CC       ]   asm.o
[ AR       ]   strrchr.o
[ AR       ]   dietlibc.a
[ AR       ]   lib_a-setjmp.o
[ AR       ]   newlib-libc.a
[ CP       ]   newlib-libm.a
[ CP       ]   gcc-libgcc.a
[ LD       ]   magiclantern
[ SYMBOLS  ]   magiclantern.sym
[ CP       ]   5D3_113.sym
[ MKDIR    ]   install
[ CP       ]   install
[ VERSION  ]   ../../platform/5D3.113/version.bin
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 484288 bytes
[ CC       ]   reboot.o
[ CC       ]   disp_direct.o
[ CC       ]   font_direct.o
[ CC       ]   footer.o
[ LD       ]   autoexec

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x0764b8 0x000df8b8 0x000df8b8 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00069500 0x00069500 0x763c0 0x8a0f4 RWE 0x100
[ XOR_CHK  ]   ../../build_tools/xor_chk
[ OBJCOPY  ]   autoexec.bin
[ OBJCOPY  ]   autoexec.bin
[ XOR_CHK  ]   autoexec.bin
cp autoexec.bin /Users/appleuser/magic-lantern/platform/5D3.113/zip/
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/fonts
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/settings
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/data
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/cropmks
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/scripts
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/scripts/lib
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/logs
mkdir -p /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/doc
cp ../../data/fonts/*.rbf /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/fonts/
cp ../../data/vram/*.lut /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/data/
cp ../../data/cropmks/*.bmp /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/cropmks/
cp ../../scripts/*.lua /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/scripts/
cp ../../scripts/lib/*.lua /Users/appleuser/magic-lantern/platform/5D3.113/zip/ML/scripts/lib/
make -C ../../modules

Building module raw_rec...
Updated HGVERSION
[ README   ]   module_strings.h
cat README.rst | grep -v -E "^:([^:])+:.+$" | rst2html --no-xml-declaration | py$//" ../html2text.py -b 700 | sed "s/



/bin/sh: rst2html: command not found

(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x10f30fcf8>)
make[4]: *** [module_strings.h] Error 1

********************************************************
WARNING: module raw_rec failed to build, deleting
********************************************************

[ RM ]  raw_rec.o raw_rec.mo raw_rec.sym raw_rec.dep module_strings.h *.o *.d *.dep *.sym hgstamp
[ RM ]  raw2dng raw2dng.exe dng2raw dng2raw.exe

Building module file_man...
Updated HGVERSION
[ README   ]   module_strings.h
cat README.rst | grep -v -E "^:([^:])+:.+$" | rst2html --no-xml-declaration | py$//" ../html2text.py -b 700 | sed "s/



/bin/sh: rst2html: command not found

(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x103a85cf8>)
make[4]: *** [module_strings.h] Error 1

********************************************************
WARNING: module file_man failed to build, deleting
********************************************************

[ RM ]  file_man.o file_man.mo file_man.sym file_man.dep module_strings.h *.o *.d *.dep *.sym hgstamp

Building module pic_view...
Updated HGVERSION
[ README   ]   module_strings.h
cat README.rst | grep -v -E "^:([^:])+:.+$" | rst2html --no-xml-declaration | py$//" ../html2text.py -b 700 | sed "s/



/bin/sh: rst2html: command not found


Yes I know it's because you're probably thinking I 'skipped' the directions on your OP but no it isn't it ... I just haven't compiled ever since I last had it working on this same MBP about a month or two ago ... so I doubt it has anything to do with me needing to update the gcc?

Sorry for being a pain in the ass, guys!

Quote from: DeafEyeJedi on December 19, 2015, 03:04:31 AM

Code Select Expand

/bin/sh: rst2html: command not found

Roll back to Reply #50 on: November 02, 2015:

QuoteThat is one of the most common issues--

Code Select Expand

rst2html: command not found.
Are you sure you followed all of my instructions?
Try this:

Code Select Expand

which rst2html
If you get nothing -- Did you install docutils? That's where we get rst2html.

Code Select Expand

pip install docutils
and then make soft links to all the python scripts that end in .py with:

Code Select Expand

for f in /usr/local/bin/*.py; do ln -s $f "${f/.py/}"; done

You sure you didn't mess around with the system that caused this to break?

Unlike the gcc hack, making a link from rst2html.py to rst2html is something that dmilligan recommends (http://www.magiclantern.fm/forum/index.php?topic=2899.msg125816#msg125816) so I take it that this is a good idea.

I made a few changes to the Homebrew installation instructions. The biggest change is that I got rid of my ugly gcc hack. The downside is that you need to make a small modification in the Magic Lantern code.

Go into the magic-lantern directory, find the file named Makefile.user.default and make a copy named Makefile.user next open up Makefile.user with your preferred text editor and find this block of code:

#
# Host compiler settings
#
HOST_CC=$(shell which gcc)
HOST_LD=$(shell which ld)
HOST_AR=$(shell which ar)


Simply change the lines to this:

#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=$(shell which ld)
HOST_AR=$(shell which ar)

What if you liked the hack because you didn't have to make any changes to the ML code? There's a better way to handle it provided your system is setup so that /usr/local/bin is listed before /usr/bin in your path. To find out what's in your path do this:

echo $PATH

Mine looks like this:

/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

Here's the hack, make symbolic links to the gcc-5 binaries in /usr/local/bin. If you look closely you'll see that they are already symbolic links so we're making symbolic links to symbolic links. Let's say we just want to have the system default to the Homebrew gcc installation instead of the Apple Clang compiler:

ln -s /usr/local/bin/gcc-5 /usr/local/bin/gcc


Now when you enter "which gcc" in the terminal it will point to /usr/local/bin/gcc. Want to switch to the Apple Clang compiler? "brew uninstall gcc" will do it. The symbolic link no longer works and "which gcc" now points to /usr/bin/gcc. Switch back to Homebrew again? "brew install gcc" since it has already been downloaded it is a fairly quick install.

There was another slight change, the quick install script now checks to see if you already have Xcode command line tools and deals with it properly.

I recently updated from El Capitan to Sierra and came up with some errors when trying to compile Magic Lantern. The fix was simply to re-install the X-code command line tools:

xcode-select --install

Another issue with the Sierra update. Seems like some of the symlinks disappeared. I got them back by re-running this from the install script:

for f in /usr/local/bin/*.py; do ln -s $f "${f/.py/}"; done

I just realized that I haven't tested Windows in a while and found out that my cross compiler is broken. Darn it--another Sierra issue?

Ok--seems that the issues with Sierra are pretty simple to fix. Simply rerun the installation steps on the first post. When you invoke Homebrew it will see that you've already got it installed but it will also recognize that you've got Sierra and move files to a new location and change permissions. After that things seem to be working as they should. If you installed the cross compiler, rerun those installation steps too. Don't mind the messages that you've already got something installed, just go through the steps.

I've been playing around with some of the development branches of Magic Lantern. The focus branch is going to allow lua scripts to have more control over the lens.

You might these warnings when compiling the newer lua development branches:

[ LUAC     ]   syntax_check.log
/bin/sh: luac: command not found


luac will compile lua scripts into binaries. Will we be using compiled lua for faster in-camera performance? Cool. That means the lua scripts will probably have to be compiled using the ARM cross compiler but for now I thought I'd get rid of the warnings by installing luac. Simple enough, it is a part of the homebrew lua package:

brew install lua

So is someone working on compiling lua scripts?  :)

You could but that's not what it's for, and I don't see much point. It's simply to make sure your Lua scripts are free of compilation errors before running them on camera.

Lua doesn't compile to native code, it compiles to its own bytecode format which is still run by the Lua interpreter (same way Java works), so it basically would only save a small amount of startup time where the scripts are loaded. Execution performance is unaffected (scripts are compiled in camera to the exact same bytecode).

As far as homebrew installation goes, what you wrote may have issues because I think homebrew is still on Lua 5.2.x and we are using 5.3.x. So you'll need to specifically tell brew you want Lua 5.3.

Thanks for the explanation dmilligan.

Quote from: dmilligan on October 25, 2016, 09:52:49 PM
I think homebrew is still on Lua 5.2.x and we are using 5.3.x. So you'll need to specifically tell brew you want Lua 5.3.

You're right--of course. Homebrew is using 5.2.4 as the default so I did this:

brew uninstall lua
brew install lua53

That installs 5.3.3 but then we're back to the same issue with the missing luac. However, using the --universal option did install luac:

brew install lua53 --universal

Maybe this because I'm on a 64-bit MacBook Pro and the install script only works on 32-bit systems? In any case, things seem to be working now.

Maybe off topic ?
For new developers like me, it is interesting to detect as many errors as possible before run time in camera.

Well if it means having to add lua to the install script it is very much on topic. Guess it is time to revisit all the tutorials that I posted.

I meant it can be useful to find errors before execution time.
Luac maybe is one way to do that but it is not the only one.
This may be better discussed in the "Lua scripting" topic.

Don't change all your tutorials just for that.
IMHO they are very well made for now.. Thanks for that.

I think the brew instructions you shared are well enough for people having such particular needs.

Ever since I saw @rbrune shared (Thanks for that) his branch in this thread (http://www.magiclantern.fm/forum/index.php?topic=5601.msg174579#msg174579thread) which I went ahead and wanted to give you (@dfort) a helping hand in getting this compiling environment to work on my system in order for me to make the builds for testers to keep our projects moving forward without hesitation.

Here's what I am getting now after getting the branch installed with no problems (that I know of) until I tried building a 7D.203 (same xcrun error with EOSM, 5D3) which I haven't seen before. Have you?

Last login: Tue Nov  8 11:14:36 on ttys000
Seans-Mac-mini-385:~ DeafEyeJedi$ hg clone -r unified https://bitbucket.org/rbrune/magic-lantern/branch/crop_rec_raw_video_10bit_12bit
destination directory: crop_rec_raw_video_10bit_12bit
adding changesets
adding manifests
adding file changes                                                             
added 12169 changesets with 26665 changes to 2912 files                         
updating to branch unified
1729 files updated, 0 files merged, 0 files removed, 0 files unresolved
Seans-Mac-mini-385:~ DeafEyeJedi$ cd ~/magic-lantern/platform/7D.203/
Seans-Mac-mini-385:7D.203 DeafEyeJedi$ make clean && make zip
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
Seans-Mac-mini-385:7D.203 DeafEyeJedi$

*edit*

After I decided that this could very well be related to my Mac Mini so tried this same routine again on my MBP which had compiling working previously (w the exception of Dual-ISO being broken -- not sure why yet!) and this time I spat out 7D.203 build just fine with no Dual-ISO module included.

(https://c5.staticflickr.com/6/5691/30233342244_67980b0ffe.jpg) (https://flic.kr/p/N4BMks)

So at least it seems to be OK on this MBP but not on my Mac Mini -- will have to investigate further more before I can go ahead and build all platforms from @rbrune's crop_rec branch (w 10/12-bit options as well) so please stand by guys until I get this cleared up with the great @dfort!

*EDIT #2*

Sorry for the mess I've made but felt this was worth sharing as I ran both builds (7D and EOSM) which then weren't including neither the crop_rec.mo nor the 10/12-bit options under raw_rec.mo & now I am suspect that this could be related to using 'unified' while installing the branch so I decided to try this again by not including the terminology 'unified' and now I am being prompted with questions and not sure which one to choose from?

Last login: Tue Nov  8 12:24:37 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ hg clone -r https://bitbucket.org/rbrune/magic-lantern/branch/crop_rec_raw_video_10bit_12bit
hg clone: invalid arguments
hg clone [OPTION]... SOURCE [DEST]

make a copy of an existing repository

options ([+] can be repeated):

-U --noupdate          the clone will include an empty working directory
                        (only a repository)
-u --updaterev REV     revision, tag, or branch to check out
-r --rev REV [+]       include the specified changeset
-b --branch BRANCH [+] clone only the specified branch
    --pull              use pull protocol to copy metadata
    --uncompressed      use uncompressed transfer (fast over LAN)
-e --ssh CMD           specify ssh command to use
    --remotecmd CMD     specify hg command to run on the remote side
    --insecure          do not verify server certificate (ignoring web.cacerts
                        config)

(use "hg clone -h" to show more help)
Apples-Macintosh-10:~ DeafEyeJedi$ destination directory: crop_rec_raw_video_10bit_12bit


Ok, you almost got it. Let's do this the way I would--not that I'm an expert but this seems to work for me.

We're going to clone rbrune's magic-lantern repository--hope that's ok with you Rafael. The reason we're cloning instead of forking is because we want to see his updates in SourceTree as he pushes his commits. Also, we won't be making any pull requests. That's another topic.

(https://c7.staticflickr.com/6/5822/30564650470_146fd8ca80.jpg)


Go to rburne's bitbucket page and copy the URL.
https://bitbucket.org/rbrune/magic-lantern

(https://c8.staticflickr.com/6/5456/30866025255_ca179e82f7.jpg)


Paste the address into SourceTree.

(https://c3.staticflickr.com/6/5736/30829575946_3733ff1610.jpg)


Note that it will create a local repository with the same name as rbrune's repository name, magic-lantern. This could be confusing if you are cloning and forking various repositories so in order to keep things organized we'll append rburne to your local repository.

(https://c6.staticflickr.com/6/5824/30866025565_3304257b16.jpg)


When you open up his repository in SourceTree you'll see his latest commits at the top but caution because you aren't in the right branch yet, you're in the default branch--check the panel on the left.

(https://c3.staticflickr.com/6/5613/30829576306_c413868fbc.jpg)


Double click either on the panel on the left or the graphic display of his workflow to get into the branch that you want. In this case you'll want his crop_rec_raw_video_10bit_12bit branch.

(https://c2.staticflickr.com/6/5662/30866025785_ed6f72505d.jpg)


Ok, now you can navigate through the local repository with the terminal, saved in your home directory if you followed this tutorial.

I've got an idea what's going on with gcc-5 because of something that happened to me just this morning. Try this first:

which gcc-5

If you get nothing, check that it is installed:

brew search gcc

This is what I got:

Quotecosmo0920/mingw_w64/i686-w64-mingw32-gcc ✔
cosmo0920/mingw_w64/x86-64-w64-mingw32-gcc
gcc ✔
homebrew/dupes/apple-gcc42 ✔
homebrew/versions/gcc43
homebrew/versions/gcc44
homebrew/versions/gcc45
homebrew/versions/gcc46
homebrew/versions/gcc47
homebrew/versions/gcc48
homebrew/versions/gcc49 ✔
homebrew/versions/gcc5 ✔
homebrew/versions/gcc6
homebrew/versions/llvm-gcc28
Caskroom/cask/gcc-arm-embedded

Note that the check marks show what's installed. If you have a check mark on gcc6 and gcc that means that you're linked to gcc6. That isn't working with ML at this time so link to gcc5:

brew link gcc5

Now try this again:

which gcc-5

You should get this:

Quote/usr/local/bin/gcc-5

Got all that? Ok--let's see if you can compile all those platforms. They should all work except for 1100D.105 and don't bother with 7D_MASTER.203, not sure what that one is for.

Good Morning Dan...  :)

Quote from: dfort on November 08, 2016, 10:04:59 PM
Ok, you almost got it. Let's do this the way I would--not that I'm an expert but this seems to work for me.

You'll always be the expert at least to me and few of us (you know who you are) and please don't ever change your sense of humor!

Quote from: dfort on November 08, 2016, 10:04:59 PM
We're going to clone rbrune's magic-lantern repository--hope that's ok with you Rafael. The reason we're cloning instead of forking is because we want to see his updates in SourceTree as he pushes his commits. Also, we won't be making any pull requests. That's another topic.

(https://c7.staticflickr.com/6/5822/30564650470_146fd8ca80.jpg)


Go to rburne's bitbucket page and copy the URL.

Code Select Expand

https://bitbucket.org/rbrune/magic-lantern

(https://c8.staticflickr.com/6/5456/30866025255_ca179e82f7.jpg)


Paste the address into SourceTree.

(https://c3.staticflickr.com/6/5736/30829575946_3733ff1610.jpg)


Note that it will create a local repository with the same name as rbrune's repository name, magic-lantern. This could be confusing if you are cloning and forking various repositories so in order to keep things organized we'll append rburne to your local repository.

(https://c6.staticflickr.com/6/5824/30866025565_3304257b16.jpg)

Done. Whoa that was easy. Now I wished I had gotten the hang of using this application sooner rather than later.  :-X

Quote from: dfort on November 08, 2016, 10:04:59 PM
When you open up his repository in SourceTree you'll see his latest commits at the top but caution because you aren't in the right branch yet, you're in the default branch--check the panel on the left.

(https://c3.staticflickr.com/6/5613/30829576306_c413868fbc.jpg)


Double click either on the panel on the left or the graphic display of his workflow to get into the branch that you want. In this case you'll want his crop_rec_raw_video_10bit_12bit branch.

(https://c2.staticflickr.com/6/5662/30866025785_ed6f72505d.jpg)


Ok, now you can navigate through the local repository with the terminal, saved in your home directory if you followed this tutorial.

Nice. Now I see that that I am in it after double clicking on his workflow. Really cool actually!

Quote from: dfort on November 08, 2016, 10:04:59 PM
I've got an idea what's going on with gcc-5 because of something that happened to me just this morning. Try this first:

Code Select Expand

which gcc-5

If you get nothing, check that it is installed:

Code Select Expand

brew search gcc

This is what I got:

Note that the check marks show what's installed. If you have a check mark on gcc6 and gcc that means that you're linked to gcc6. That isn't working with ML at this time so link to gcc5:

Code Select Expand

brew link gcc5

Now try this again:

Code Select Expand

which gcc-5

This is what I'm getting...

Last login: Tue Nov  8 12:25:43 on ttys001
Apples-Macintosh-10:~ DeafEyeJedi$ which gcc-5
Apples-Macintosh-10:~ DeafEyeJedi$ brew search gcc
gcc ✔                                    homebrew/dupes/apple-gcc42
homebrew/versions/gcc43                  homebrew/versions/gcc49
homebrew/versions/gcc44                  homebrew/versions/gcc5
homebrew/versions/gcc45                  homebrew/versions/gcc6
homebrew/versions/gcc46                  homebrew/versions/llvm-gcc28
homebrew/versions/gcc47                  Caskroom/cask/gcc-arm-embedded
homebrew/versions/gcc48
Apples-Macintosh-10:~ DeafEyeJedi$ brew link gcc5
Error: No such keg: /usr/local/Cellar/gcc5

I have a feeling that this may be due to the fact that I may have done a "brew update" few weeks ago when I was having compiling issues so could this be very well related? Should I re-install GCC for your OP in this thread?

Quote from: dfort on November 08, 2016, 10:04:59 PM
Got all that? Ok--let's see if you can compile all those platforms. They should all work except for 1100D.105 and don't bother with 7D_MASTER.203, not sure what that one is for.

I did got all that. Thanks again for this BUT I just need to know whether I should delete the GCC file and reinstall this again?

I can feel it that I am so close to getting this to work perfectly...  8)

*edit*

After researching through your OP -- should I go ahead and do this installation again?

brew tap homebrew/dupes; brew install homebrew/versions/gcc5

*edit 2*

Well I went ahead and did that then after typing up which gcc I would be running...

Apples-Macintosh-10:~ DeafEyeJedi$ which gcc-5
Apples-Macintosh-10:~ DeafEyeJedi$ brew search gcc
gcc ✔                                    homebrew/versions/gcc47
homebrew/dupes/apple-gcc42               homebrew/versions/gcc48
homebrew/versions/gcc43                  homebrew/versions/gcc49
homebrew/versions/gcc44                  homebrew/versions/gcc5 ✔
homebrew/versions/gcc45                  homebrew/versions/gcc6
homebrew/versions/gcc46                  homebrew/versions/llvm-gcc28
Caskroom/cask/gcc-arm-embedded
Apples-Macintosh-10:~ DeafEyeJedi$

So that looks good to me now, right? I'm gonna go ahead and do the rest from your post above... Stand by Dan!

@DeafEyeJedi

Make sure gcc-5 is linked:

which gcc-5

You should get this for an answer:

Quote/usr/local/bin/gcc-5

If you don't then:

brew link gcc5

And once again:

which gcc-5

Note when to use the dash and when not to use the dash.

Looks like I'll have to wait.... though it has been 20 minutes since.

Last login: Wed Nov  9 09:18:28 on ttys004
Apples-Macintosh-10:~ DeafEyeJedi$ which gcc-5
Apples-Macintosh-10:~ DeafEyeJedi$ brew link gcc5
Error: Operation already in progress for gcc5
Another active Homebrew process is already using gcc5.
Please wait for it to finish or terminate it to continue.
Apples-Macintosh-10:~ DeafEyeJedi$

btw just for good news (at least a little) managed to spit out EOSM, 7D builds with no problems (includes Rafael's fix w crop_rec.mo and options for 10/12-bit) on both builds with the exception of Dual-ISO being broken once again. What gives?

(https://c2.staticflickr.com/6/5707/30792917961_af8c3b1223_n.jpg) (https://flic.kr/p/NV4KGD) (https://c6.staticflickr.com/6/5583/30793094381_daa54ed411_n.jpg) (https://flic.kr/p/NV5E9n)

*edit*

Actually I am glad I didn't quit that brew install because it literally took 37 minutes to install according to this...

🍺  /usr/local/Cellar/gcc5/5.4.0: 1,391 files, 246.6M, built in 37 minutes 31 seconds
Apples-Macintosh-10:~ DeafEyeJedi$

and here's the better news of all...

Apples-Macintosh-10:~ DeafEyeJedi$ which gcc-5
/usr/local/bin/gcc-5
Apples-Macintosh-10:~ DeafEyeJedi$

Well by now I think I should try compiling again and see if Dual-ISO is still broken or not (shouldn't be because now I got gcc-5 linked, right?) well guess we'll find out below.

*edit 2*

Nope. Dual-ISO is still broken. Could this small portion of error be related to this?

cr2hdr.c:2548:59: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2574:21: error: function definition is not allowed here
                    {
                    ^


Ideally I beleive I shall wait until I get Dual-ISO module to work before going ahead and compile the crop_rec.mo + 10/12-bit merge from @rbrune's branch, right guys?  :o

I don't think that's the error.

I might have skipped an important step--did you make your Makefile.user file? If it isn't in there it will use the Apple clang compiler.

Makefile.user
#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)


Quote from: dfort on November 09, 2016, 07:31:46 PM
I don't think that's the error.

I might have skipped an important step--did you make your Makefile.user file? If it isn't in there it will use the Apple clang compiler.

Makefile.user

Code Select Expand

#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)


Yes of course I followed your instructions and made the new Makefile.user within magic-lantern-rbune directory ... It's already becoming a good habit!  :D

(https://c3.staticflickr.com/6/5699/30846206066_0776b00b5d.jpg) (https://flic.kr/p/NZLSp5)

*edit 2*

Sorry this is all becoming too addicting to give up on figuring out why Dual-ISO module is still broken (won't build) and here's a log portion directly to Dual-ISO module and maybe there are hints? Could the culprit be related to a certain cr2hdr that I am using?

Building module dual_iso...
Updated HGVERSION
[ README   ]   module_strings.h
Help page 1: Quick start
Help page 2: Tips and tricks
[ CC       ]   dual_iso.o
[ HGDIFF   ]   hgdiff.tmp
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ OBJCOPY  ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
0000132c dual_iso_calc_dr_improvement
00001384 dual_iso_set_enabled
000013a4 dual_iso_is_enabled
000013b4 dual_iso_is_active
000013d4 dual_iso_get_dr_improvement
00001554 dual_iso_get_recovery_iso
00001574 dual_iso_set_recovery_iso
[ DEPENDS  ]   dual_iso.dep
Will load on:
    5D3
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
[ gcc      ]   cr2hdr
cr2hdr.c:1351:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist rggb.m"));
                                             ^
cr2hdr.c:1351:46: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:1489:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist bddb.m"));
                                             ^
cr2hdr.c:1489:46: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:1765:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist iso-curve.m"));
                                                  ^
cr2hdr.c:1765:51: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2201:55: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist fullres-curve.m"));
                                                      ^
cr2hdr.c:2201:55: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2535:56: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edges.dng"));
                                                       ^
cr2hdr.c:2535:56: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2548:59: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edge-map.dng"));
                                                          ^
cr2hdr.c:2548:59: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2574:21: error: function definition is not allowed here
                    {
                    ^
cr2hdr.c:2588:31: warning: implicit declaration of function 'edge_interp' is
      invalid in C99 [-Wimplicit-function-declaration]
                    int pi0 = edge_interp(dir);
                              ^
cr2hdr.c:2841:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist mix-curve.m"));
                                                  ^
cr2hdr.c:2841:51: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:3325:55: warning: if statement has empty body [-Wempty-body]
            if(system("octave --persist soft-film.m"));
                                                      ^
cr2hdr.c:3325:55: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:3669:34: warning: if statement has empty body [-Wempty-body]
        if(system("octave wb.m"));
                                 ^
cr2hdr.c:3669:34: note: put the semicolon on a separate line to silence this
      warning
10 warnings and 1 error generated.
dcraw-bridge.c:101:3: warning: unused variable 'unique'
      [-Wunused-const-variable]
} unique[] = {
  ^
1 warning generated.
amaze_demosaic_RT.c:983:10: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                            ^
amaze_demosaic_RT.c:983:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:983:35: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                                                     ^
amaze_demosaic_RT.c:983:35: note: use function 'fabs' instead
  ...if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;...
                                  ^~~~~
                                  fabs
amaze_demosaic_RT.c:1215:10: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                            ^
amaze_demosaic_RT.c:1215:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:1215:35: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                                                     ^
amaze_demosaic_RT.c:1215:35: note: use function 'fabs' instead
  ...if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
                                  ^~~~~
                                  fabs
4 warnings generated.
make[4]: *** [cr2hdr] Error 1

********************************************************
WARNING: module dual_iso failed to build, deleting
********************************************************

[ RM ]  dual_iso.o dual_iso.mo dual_iso.sym dual_iso.dep module_strings.h hgdiff.tmp *.o *.d *.dep *.sym hgstamp
[ RM ]  cr2hdr cr2hdr.exe dcraw dcraw.c dcraw.exe exiftool.exe exiftool.tar.gz exiftool exiftool.zip cr2hdr.zip cr2hdr-win.zip cr2hdr-win_exiftool-perl-script.zip

In any case if anyone wants me to go ahead and compile certain builds to test out the crop_rec.mo + 10/12-bit together then please look at this thread. (http://www.magiclantern.fm/forum/index.php?topic=5601.msg174659#msg174659please%20look%20at%20this%20thread)

Also if anyone insist to have this Dual-ISO module on their cams then are we allowed just to grab the Dual-ISO module from the nightlies and bundle it together or is that a no-no?

*edit* Finally got it working w Dual-ISO. Now 10/12 bit raw_rec.mo + crop_rec.mo builds can be downloaded from this post (http://www.magiclantern.fm/forum/index.php?topic=5601.msg174726#msg174726post). Please remember to keep all bug reports related to this in that particular thread during testing. Thanks!

Seems like we tried everything. I have pretty much the same setup and things are working fine over here.

Try navigating into the dual_iso directory and compile cr2hdr. Do it one step at a time and watch for error messages.

In the terminal, change to the dual_iso directory.
cd ~/magic-lantern-rbrune/modules/dual_iso

Clean out any prior builds.
make clean

Now compile just cr2hdr
make cr2hdr

See how that works.

I'm not sure what was the rational for compiling cr2hdr every time you want to build ML for your camera but that's where the problem seems to be. Everything else uses the ARM cross-compiler but the command line tools that obviously need to be built with the host compiler. If you're feeling adventurous try this. Open the Makefile that is in the modules/dual_iso directory with a text editor:

Makefile
# this is a very simple makefile for modules
TOP_DIR=../..

# define the module name - make sure name is max 8 characters
MODULE_NAME=dual_iso
MODULE_OBJS=dual_iso.o

# include modules environment
include $(TOP_DIR)/modules/Makefile.modules

# include cr2hdr makefile rules
include Makefile.cr2hdr


And comment out that last line:

# include Makefile.cr2hdr

That will skip building cr2hdr and the dual_iso module will compile for your camera.

Good morning Dan once again... Such a wonderful feeling to wake up knowing that ML will be alive and well for us to dig through and be happy for the day!  8)

Quote from: dfort on November 10, 2016, 05:17:55 AM
Try navigating into the dual_iso directory and compile cr2hdr. Do it one step at a time and watch for error messages.

In the terminal, change to the dual_iso directory.

Code Select Expand

cd ~/magic-lantern-rbrune/modules/dual_iso

Clean out any prior builds.

Code Select Expand

make clean

Now compile just cr2hdr

Code Select Expand

make cr2hdr

See how that works.

Done. Though there was nothing to clean up literally. Besides there was one error that didn't seem to be worrisome. Here are the logs:

Apples-Macintosh-10:~ DeafEyeJedi$ cd ~/magic-lantern-rbrune/modules/dual_iso
Apples-Macintosh-10:dual_iso DeafEyeJedi$ make clean
[ RM ]  dual_iso.o dual_iso.mo dual_iso.sym dual_iso.dep module_strings.h hgdiff.tmp *.o *.d *.dep *.sym hgstamp
[ RM ]  cr2hdr cr2hdr.exe dcraw dcraw.c dcraw.exe exiftool.exe exiftool.tar.gz exiftool exiftool.zip cr2hdr.zip cr2hdr-win.zip cr2hdr-win_exiftool-perl-script.zip
rm -rf lib
Apples-Macintosh-10:dual_iso DeafEyeJedi$ make cr2hdr
Updated HGVERSION
[ README   ]   module_strings.h
Help page 1: Quick start
Help page 2: Tips and tricks
[ gcc      ]   cr2hdr
cr2hdr.c:1351:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist rggb.m"));
                                             ^
cr2hdr.c:1351:46: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:1489:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist bddb.m"));
                                             ^
cr2hdr.c:1489:46: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:1765:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist iso-curve.m"));
                                                  ^
cr2hdr.c:1765:51: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2201:55: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist fullres-curve.m"));
                                                      ^
cr2hdr.c:2201:55: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2535:56: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edges.dng"));
                                                       ^
cr2hdr.c:2535:56: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2548:59: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edge-map.dng"));
                                                          ^
cr2hdr.c:2548:59: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2574:21: error: function definition is not allowed here
                    {
                    ^
cr2hdr.c:2588:31: warning: implicit declaration of function 'edge_interp' is
      invalid in C99 [-Wimplicit-function-declaration]
                    int pi0 = edge_interp(dir);
                              ^
cr2hdr.c:2841:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist mix-curve.m"));
                                                  ^
cr2hdr.c:2841:51: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:3325:55: warning: if statement has empty body [-Wempty-body]
            if(system("octave --persist soft-film.m"));
                                                      ^
cr2hdr.c:3325:55: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:3669:34: warning: if statement has empty body [-Wempty-body]
        if(system("octave wb.m"));
                                 ^
cr2hdr.c:3669:34: note: put the semicolon on a separate line to silence this
      warning
10 warnings and 1 error generated.
dcraw-bridge.c:101:3: warning: unused variable 'unique'
      [-Wunused-const-variable]
} unique[] = {
  ^
1 warning generated.
amaze_demosaic_RT.c:983:10: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                            ^
amaze_demosaic_RT.c:983:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:983:35: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                                                     ^
amaze_demosaic_RT.c:983:35: note: use function 'fabs' instead
  ...if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;...
                                  ^~~~~
                                  fabs
amaze_demosaic_RT.c:1215:10: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                            ^
amaze_demosaic_RT.c:1215:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:1215:35: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                                                     ^
amaze_demosaic_RT.c:1215:35: note: use function 'fabs' instead
  ...if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
                                  ^~~~~
                                  fabs
4 warnings generated.
make: *** [cr2hdr] Error 1
Apples-Macintosh-10:dual_iso DeafEyeJedi$


Not sure what that means? However since I was feeling a bit adventurous and went ahead with these suggestions below...

Quote from: dfort on November 10, 2016, 05:17:55 AM
I'm not sure what was the rational for compiling cr2hdr every time you want to build ML for your camera but that's where the problem seems to be. Everything else uses the ARM cross-compiler but the command line tools that obviously need to be built with the host compiler. If you're feeling adventurous try this. Open the Makefile that is in the modules/dual_iso directory with a text editor:

Makefile

Code Select Expand

# this is a very simple makefile for modules
TOP_DIR=../..

# define the module name - make sure name is max 8 characters
MODULE_NAME=dual_iso
MODULE_OBJS=dual_iso.o

# include modules environment
include $(TOP_DIR)/modules/Makefile.modules

# include cr2hdr makefile rules
include Makefile.cr2hdr


And comment out that last line:

Code Select Expand

# include Makefile.cr2hdr

That will skip building cr2hdr and the dual_iso module will compile for your camera.

Indeed that did the trick. Now it seems I was able to get it compiled for 5D3.113 from @rbrune's branch with your latest fix included (raw_rec.mo + 10/12-bit options) and this time Dual-ISO spat out just fine. Everything looks normal within the logs as well as the build (see attachment below):

Building module dual_iso...
[ HGDIFF   ]   hgdiff.tmp
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ OBJCOPY  ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
0000132c dual_iso_calc_dr_improvement
00001384 dual_iso_set_enabled
000013a4 dual_iso_is_enabled
000013b4 dual_iso_is_active
000013d4 dual_iso_get_dr_improvement
00001554 dual_iso_get_recovery_iso
00001574 dual_iso_set_recovery_iso
[ DEPENDS  ]   dual_iso.dep
Will load on:
    5D3
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 600D, 60D, 650D, 6D, 700D, 7D, EOSM

Building module silent...
[ HGDIFF   ]   hgdiff.tmp
[ MODULE   ]   silent.mo
[ STRIP    ]   silent.mo
[ OBJCOPY  ]   silent.mo
[ EXPORTS  ]   silent.sym
[ DEPENDS  ]   silent.dep
Will load on:
    5D3
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 600D, 60D, 650D, 6D, 700D, 7D, EOSM


(https://c7.staticflickr.com/6/5577/30864624646_321176ae24.jpg) (https://flic.kr/p/P2pgAN)

Although I notice there was this one error message that I thought I should double check and share with you just to be sure it's nothing cirtical:

[ XOR_CHK  ]   ../../build_tools/xor_chk
../../build_tools/xor_chk.c:48:88: warning: format specifies type
      'unsigned long' but the argument has type 'uint64_t' (aka
      'unsigned long long') [-Wformat]
  ...error (expected 0x%lX, got 0x%lX)\n", 0xCCCCCCCCE12FFF13, footer_magic);
                                  ~~~                          ^~~~~~~~~~~~
                                  %llX
1 warning generated.


...and this one from the lua.mo if by all means anything just in case:

Building module lua...
[ HGDIFF   ]   hgdiff.tmp
[ MODULE   ]   lua.mo
lua/lauxlib.o: In function `panic':
lauxlib.c:(.text+0x360): warning: warning: your code uses stdio (7+k bloat).
dietlibc.a(vfprintf.o): In function `vfprintf':
vfprintf.c:(.text+0x34): warning: warning: the printf functions add several kilobytes of bloat.


Whew that felt so good. Thanks again @dfort for your wonderful contributions and I very much enjoy my learning experience with you on this. Now I have to first downgrade from 123 to 113 for the 5D3 to be tested with this build (unless I should give it a try to find a 5D3.123 branch that can be merged with this crop_rec.rec + 10/12-bit options) or is that going to be a long shot for me to get it compiled?

I don't mind downgrading but when I go out on paid gig's on weekends occasionally (upgrade to 5D3.123 is a must) for me to be able to have clean HDMI out on set. I know what a pain but like I said I don't mind. Just would prefer to be able to compile if I'm allowed to with these latest gems?

Anyway enough with the questions now I'm gonna go ahead and compile the rest of DSLR's for you, Dan and perhaps I should use bitbucket instead of Dropbox.com or Mega.nz?

Actually I just made this download site of my own from Bitbucket (https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads) and if you don't mind me asking what would be the best way to keep these files organized in terms of which branch was it from? What's included? What's not included? I mean just so that way our testers will know which one is which and not to be confused with bug reports coming from these builds versus the unified nightlies, right?

Perhaps I should change the file name to include 10/12-bit + crop_rec.mo from @rbrune's branch, ya?

Finally!

Well, there's still a problem. We should figure out why cr2hdr isn't compiling on your system. Those other problems that you pointed out might be benign. How are the builds running on your cameras? You might want to try the unified branch just to see if it works the same as a downloaded nightly.

I like renaming the zip file from Nightly to something more descriptive. Otherwise you'll have a bunch of files labeled Nightly and you won't be able to keep them organized.

I'm thinking about putting in a pull request that skips building cr2hdr. Partially because of the problems it is causing for people that are first trying to compile ML and partially so that it works the same way as mlv_dump and raw2dng.

So it seems that one of the most common problems when setting up a development system is that cr2hdr is picky about which gcc you're using so dual_iso fails to compile. I'm not sure what the logic was behind having cr2hdr compile when all you want is ML for your camera so I put in a pull request that changes the rules so it works more like mlv_dump and raw2dng. a1ex already approved it but hasn't merged it yet.

https://bitbucket.org/hudson/magic-lantern/pull-requests/770/removed-all-rule-so-cr2hdr-isnt-built/diff

The change is really simple, just delete the "all::" rule at the end of Makefile.cr2hdr.

By the way, did you know that the build script can also download dcraw and exiftool? Change into the modules/dual_iso directory enter "make cr2hdr.zip" in the terminal and it will download and build dcraw, download exiftool, build cr2hdr and package everything in a .zip file. Pretty cool, huh?

Been having a real blast with my workstation just never got compiling to work but it seems finally I solved a long ongoing problem.

Had python messages showing why builds were failing and the end part showed:
ValueError: unknown locale: UTF-8

The fix was easy
https://fcojperez.wordpress.com/2014/08/18/valueerror-unknown-locale-utf-8-cuando-instalamos-awscli/
In terminal write
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8

You can add those export lines straight into your bash profile.
https://coderwall.com/p/-k_93g/mac-os-x-valueerror-unknown-locale-utf-8-in-python

One other problem is raw_rec failing to build because of the makefile. Simply erase:
all:: raw2dng inside Makefile in raw_rec folder.

Quote from: Danne on November 27, 2016, 10:55:40 PM
One other problem is raw_rec failing to build because of the makefile. Simply erase:

Code Select Expand

all:: raw2dng
inside Makefile in raw_rec folder.

I submitted a pull request for that along with a change that allows cross compiling a Windows binary on a Mac as described on the first post of this topic.

https://bitbucket.org/hudson/magic-lantern/pull-requests/773/raw2dng-make-rules-changes/diff

Nice work @Danne!  ;D

@dfort -- is this something that we are required to do on our Mac's even if we aren't getting the raw_rec failing?

Quote from: DeafEyeJedi on November 30, 2016, 09:48:50 AM
...is this something that we are required to do on our Mac's even if we aren't getting the raw_rec failing?

No, it isn't required but it isn't just about raw_rec failing. If you can't compile the command line tools, mlv_dump, raw2dng and cr2hdr, that means your system isn't setup properly and there's probably an issue with your host platform's gcc compiler.

What this is about is that generally you are either building ML for your camera or are building the command line tools but you usually don't need to build them both at the same time. Many of us didn't even know that raw2dng was being built every time we compiled a build for our cameras. The build rules are still there for the command line tools when you need them.

Excuse me!
I am using Canon 1DX Mark ii, I hope I understood correctly (!)
possibile shooting in raw my canon 1dx mark ii?  :(

ML dev team decided some time ago not to touch Canon's pro gear. This includes 1D and C series.
And it is not a matter of discussion.

@Danne found an issue with cosmo0920/mingw_w64/i686-w64-mingw32-gcc -- it currently won't install.

I made the developer aware of this, hopefully it will be fixed soon.

Error: i686-w64-mingw32-gcc contains conflicting version recursive dependencies:
  isl, [email protected]

Got a reply.

QuoteThanks for your report!
But recently homebrew-core got merged mingw-64 Formula which installs cross gcc and binutils at once.

Could you tell to use mingw-w64 instead of my formula?

So now it should be much easier. Simply:

brew install mingw-w64

and you have a full 32-bit and 64-bit Windows cross compiling setup ready to go. I'll update the tutorial so it will work with this latest mingw-64 formula.

That was too easy. Works great. Thanks dfort.

Yay! Goodby ugly monster Cygwin...

I just wanna say thanks so so much dfort/Danne for the install script at the end :)
Couldn't get anything to work on my fresh macOS install until I just tried that!

Worked on 10.13 high sierra almost flawlessly, only had to make this small change for it to work:
pip install docutils
changed to this:
sudo pip install docutils

... and a suggestion: maybe don't automatically install mingw in the script, as most people wont be compiling this stuff for other platforms,
instead maybe add a note on the first post on how to for those who want to (please? :D) Also space is precious on a 240gb SSD (real reason for my concern) ;)

Great Ilia3101. This is thread is a lifesaver  :P. Went back in here yesterday after reinstalling python versions and whatnot. Suddenly lost docutils and then cr2hdr wouldn´t compile.
For docutils I did:
sudo pip2 install docutils
and when cr2hdr would complain I changed makefile as suggested in first post:

Now make a new file called Makefile.user and put in just these lines:
#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)
Seems that only cr2hdr needs this change nowadays.

Worth mentioning, after installing docutils this wasn´t needed anymore:
for f in /usr/local/bin/*.py; do ln -s $f "${f/.py/}"; done

I made the mingw-w64 Windows cross compiler option in the quick install script and started looking into the pip issue. Yeah, looks like something has changed and my installation no longer works as it should. Glad you guys could figure it out on your systems but I'd like to get the quick install script installing quickly again. Need to find a virgin Mac to try it out.

Quote from: dfort on September 24, 2017, 08:36:44 AM
Need to find a virgin Mac to try it out.

Virtualization  8)
I did it for you!

---

Compiling Magic Lantern on macOS High Sierra _{BETA (10.13 17A362a)}

(https://s26.postimg.org/v7nju00p5/mac_OS_info_640x480.png)
It's an adaptation of @dfort commands, thanks to you !

Compiling ML + Modules + tools (like cr2hdr, mlv_dump, ..)

Quote
macos-high-sierra:~ tof$ xcode-select --install
macos-high-sierra:~ tof$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
macos-high-sierra:~ tof$ brew install python wget mercurial
macos-high-sierra:~ tof$ pip2 install docutils
macos-high-sierra:~ tof$ brew tap homebrew/dupes; brew install homebrew/versions/gcc5
macos-high-sierra:~ tof$ cd ~ && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && tar -jxf gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && rm gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2
macos-high-sierra:~ tof$ hg clone https://bitbucket.org/hudson/magic-lantern
macos-high-sierra:~ tof$ cd magic-lantern
macos-high-sierra:magic-lantern tof$ hg update unified
macos-high-sierra:magic-lantern tof$ cat <<'EOT' > Makefile.user
#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)
EOT
# OPTIONAL cross-compile
macos-high-sierra:~ tof$ brew install mingw-w64

Compiling QEMU

Quote
macos-high-sierra:magic-lantern tof$ brew install xz grep pkg-config glib automake libtool pixman mtools
macos-high-sierra:magic-lantern tof$ hg update qemu
macos-high-sierra:magic-lantern tof$ cd contrib/qemu/
# "Fix" for QEMU on macOS (need custom install.sh : change grep -P to ggrep -P)
# need to PR a great solution not a workarround, like PYTHON=python2, GREP=ggrep
# I don't want to replace macOS grep with GNU grep (prevent eventual bug), if you want: @dfort solution (http://www.magiclantern.fm/forum/index.php?topic=2864.msg185591#msg185591) and ignore last command
macos-high-sierra:qemu tof$ sed -i '' 's/grep/ggrep/g' install.sh
macos-high-sierra:qemu tof$ sed -i '' 's/grep/ggrep/g' scripts/run_canon_fw.sh
macos-high-sierra:qemu tof$ ./install
# Follow next steps from output

We're ready for tomorrow  8)

Wow, thanks @kichetof

So the whole "pip" issue was because Homebrew switched over from pip to pip2. No sudo needed and that line to make symbolic links on the docutils scripts is still needed.

In any case, I updated the install script and it is working again.

Need to do some more testing. Walter suggested a "virtual virgin" setup. (Sure that's not a virtuous virgin?  ;) )

Nice work on the QEMU setup. I was stumbling around in the dark when I came across the Mac grep issue. It looks like kichetof has a better solution. Does it require some tweaks in the QEMU install script?

[EDIT] Just a thought, @Danne you're good at bash scripting--how about adding user interactive options at the end of the script to add Windows cross compiling and another for setting up a QEMU ready development environment?

QuoteEDIT] Just a thought, @Danne you're good at bash scripting--how about adding user interactive options at the end of the script to add Windows cross compiling and another for setting up a QEMU ready development environment?

Sure thing. Shouldn't be hard at all. Will take a look soon.
Great action @Kichetof.

This should work, check OPTIONAL 1 and OPTIONAL 2:

Modified following:
Added
./install.sh to the script (OPTION 2) and erased set -e at the top.
Took the liberty to erase:
sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
sudo mkdir -p /usr/local/ && sudo chflags norestricted /usr/local/ && sudo chown $(whoami):admin /usr/local/ && echo sudo chown -R $(whoami):admin /usr/local/
Also erased:
for f in /usr/local/bin/*.py; do ln -s $f "${f/.py/}"; done

cat <<'EOF' > mac_ml.sh
#!/bin/sh

#  mac_ml.sh
#
#  Script to almost automatically install
#  and configure a Magic Lantern development
#  environment on a Macintosh.
#
#  Reference:
#  http://magiclantern.fm/forum/index.php?topic=16012.0
#
#  Daniel Fort (dfort) - 2015-10-15
#  Modified 2016-03-17 - took out ugly gcc hack
#  Modified 2017-06-27 - added mingw-w64 for cross compiling Windows binaries
#  Modified 2017-06-28 - check if magic-lantern already exists
#  Modified 2017-07-12 - write a Makefile.user file
#  Modified 2017-09-23 - made mingw-w64 optional
#  Modified 2017-09-24 -
#
#  Thanks to Danne and kichetof for their input

if ! test -d /usr/local; then
  echo "/usr/local directory not found."
  echo "Follow the steps at the beginning of"
  echo "tutorial to create /usr/local and"
  echo "then re-run this script."
  exit 1
fi

echo "/usr/local found -- "
echo "continuing with installation."

cd ~
if [ ! -d "/Library/Developer/CommandLineTools" ]; then xcode-select --install; fi
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install python wget mercurial

#
# Uncomment the following line if you want to cross compile Windows binaries.
# brew install mingw-w64
#

pip2 install docutils

brew tap homebrew/dupes; brew install homebrew/versions/gcc5
cd ~ && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && tar -jxf gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && rm gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2

if ! test -d ~/magic-lantern; then
    hg clone https://bitbucket.org/hudson/magic-lantern
    cd magic-lantern
    hg update unified
    cat <<'EOT' > Makefile.user
#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)
EOT
fi

#OPTIONAL 1
clear
read -p $(tput bold)"Would you also like to install the windows cross compiler mingw-w64?$(tput setaf 1)

Y/N?"$(tput sgr0) -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
brew install mingw-w64
fi

#OPTIONAL 2
clear
read -p $(tput bold)"How about compiling QEMU?$(tput setaf 1)

Y/N?"$(tput sgr0) -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
cd magic-lantern
brew install xz grep pkg-config glib automake libtool pixman mtools
hg update qemu
cd contrib/qemu/
sed -i '' 's/gggrep/ggrep/g' install.sh
./install.sh
fi

cd ~
exit 0
EOF
bash mac_ml.sh && rm mac_ml.sh

A couple of things.

1 - The script really works best with a "virgin mary" machine. As soon as you are trying to reinstall things the script breaks telling you what is already installed. This means you won´t be able to rerun the script since it will break at the same place each time.Erase set -e

A solution: As the script breaks it could fall back into a bash menu again interacting with the user telling what is already installed and then ask user to continue anyway. Even better if files are already installed simply continue automatically.
*Update. Just checked and erase set -e to let the script run without breaks.

2 - compiling Qemu OPTION 2, last line:
sed -i '' 's/grep/ggrep/g' install.sh
replaces to ggrep in install.sh. Next step, run the install.sh script:
Simply finish with this?
./install.sh


The two interactive user options should then become:
#OPTIONAL 1
clear
read -p $(tput bold)"Would you also like to install the windows cross compiler mingw-w64?$(tput setaf 1)

Y/N?"$(tput sgr0) -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
brew install mingw-w64
fi

#OPTIONAL 2
clear
read -p $(tput bold)"How about compiling QEMU?$(tput setaf 1)

Y/N?"$(tput sgr0) -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
cd magic-lantern
brew install xz grep pkg-config glib automake libtool pixman mtools
hg update qemu
cd contrib/qemu/
sed -i '' 's/gggrep/ggrep/g' install.sh
./install.sh
fi
And then the quemu install.sh script will tell you:
This will setup QEMU for emulating Magic Lantern.
Thou shalt not be afraid of compiling stuff on Linux ;)
Continue? [y/n]



Still using the for loop although running pip2?
pip2 install docutils
for f in /usr/local/bin/*.py; do ln -s $f "${f/.py/}"; done

I forgot to add change for run_canon_fw.sh
edited my last post with changes

@Danne I didn't change permissions on /usr/local and didn't copy *.py

---

@a1ex what do you think about this change ? (for install.sh and run_canon_fw.sh)

tof-mbp:qemu tof$ diff run_canon_fw_tof.sh run_canon_fw.sh
3d2
< GREP=${GREP:=grep}
11c10
<     SD_DEV=`losetup -j $1 | $GREP -Po "(?<=/dev/)[^ ]*(?=:)"`
---
>     SD_DEV=`losetup -j $1 | grep -Po "(?<=/dev/)[^ ]*(?=:)"`
13c12
<         if cat /proc/mounts | $GREP /dev/mapper/$SD_DEV; then
---
>         if cat /proc/mounts | grep /dev/mapper/$SD_DEV; then
21c20
<     if [[ -n $(ls /Volumes | $GREP EOS_DIGITAL*) ]]; then
---
>     if [[ -n $(ls /Volumes | grep EOS_DIGITAL*) ]]; then
26,34d24
<     
<     if [[ -n $(which ggrep) ]]; then
<         GREP=ggrep
<     else
<         echo
<         echo "Error: you need GNU grep to run this script"
<         echo "brew install grep"
<         exit 1
<     fi
60c50
<     QEMU_EOS_DEBUGMSG=`cat $CAM/debugmsg.gdb | $GREP DebugMsg_log -B 1 | $GREP -Pom1 "(?<=b \*)0x.*"`
---
>     QEMU_EOS_DEBUGMSG=`cat $CAM/debugmsg.gdb | grep DebugMsg_log -B 1 | grep -Pom1 "(?<=b \*)0x.*"`

QuoteI didn't change permissions on /usr/local and didn't copy *.py

Yeah, me neither, could probably be deleted...

Quote from: Danne on September 24, 2017, 09:09:02 PM
could probably be deleted...

On macOS Sierra and High Sierra for me yes

Updated the quick install script.

A few notes:

• You are right Danne, the symbolic links are no longer required for docutils. It was fixed on this merge (https://bitbucket.org/hudson/magic-lantern/commits/2c2a11e319741dfff4d4d8276b5a4e62c8351d06).
• All user input requires a carriage return but the options in the script will drop you out of the script if you hit the carriage return.
• So not only did you setup the system to install QEMU you are also running the QEMU installer script. Cool--but:

./install.sh: line 38: cd: .git: No such file or directory
Initialized empty Git repository in /Users/Fort/qemu/qemu-2.5.0/.git/

*** Please tell me who you are.

Run

  git config --global user.email "[email protected]"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'Fort@Rosies-Air.(none)')


Quite a few Macs are probably set up this way. Given enough time I could probably fix these issues but I'm using my wife's Mac to run these tests and she wants it back!

Sick progress guys. Now I'm anxious to try the script installation for QEMU. Probably should run the first script just to be sure my compiling environment is up to date. Thanks!

Looks like that git error is because the install.sh script is initializing a git repository in order to track changes. Maybe this could be a user option in install.sh or the Mac quick install script should verify that auto-detecting the email address works before getting into the QEMU installation?

# get qemu
wget --progress=dot:giga -c http://wiki.qemu-project.org/download/$QEMU_NAME.tar.bz2
tar jxf $QEMU_NAME.tar.bz2

# initialize a git repo, to make it easy to track changes to QEMU source
cd $QEMU_NAME
cd .git && cd .. || (git init && git add . && git commit -q -m "$QEMU_NAME vanilla")
cd ..


Sorted it out - the same issue was present on a fresh Ubuntu.

Also updated the install script to set up the dependencies as well - WIP.

Excellent! Updated the Mac quick install script to take advantage of the latest changes in QEMU install.sh.

FYI for users wanting to install QEMU with the Mac quick install script--this is still a WIP and the QEMU installation currently ends with:

*** Please add gcc binaries to your executable PATH:
    PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** Please set up arm-none-eabi-gdb before continuing.


You'll have to find your way from there. The journey I took to get QEMU running on the Mac started here:

http://www.magiclantern.fm/forum/index.php?topic=2864.msg184773#msg184773

For those who have updated from macOS Sierra to macOS High Sierra, you need to reinstall gcc-5

brew reinstall gcc@5

take some beers and wait!  ;D

Thanks for the heads up @kichetof as I'm about to update mine as well.

Alright guys here we go with me about to embarrass myself for beneficial reasons.  :P

My first attempt at installing QEMU after running the latest Quick Installation Script (Thanks @Danne!) which then shows what needed to be updated even tho most of the compiling tools were already installed previously awhile ago on this grumpy old Mac Mini running 10.12.6 (decided not to update this one yet to Sierra High 10.13 since after experiencing random issues when updating this on my MBP) and finally this is where I started w my attempt at installing QEMU...



What is it that I could be doing incorrectly and sorry for my stupidity. Since I am unable to empathize what 'please call configure before running make' means?

Here's how it should work in Ubuntu; should be the same on Mac, except for the 64-bit GDB bug (so you may not be able to launch the 5D3 GUI - try 100D instead):

http://www.magiclantern.fm/forum/index.php?topic=2864.msg190596#msg190596

(I have a hard time reading your error messages, even when watching in full-screen...)

This animation (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/xenial.gif) might be useful (although I should probably add some captions to show the commands edit: DONE!).

You have a grep error message:

(https://farm5.staticflickr.com/4482/23525525668_a7735de267_o.png) (https://flic.kr/p/BQStaw)

Then you ran the error message--that won't get you anywhere. What grep attempted to do was to give you configure instructions for your system. I ran into that same problem a while back:

http://www.magiclantern.fm/forum/index.php?topic=2864.msg184773#msg184773

After installing the dependencies using Homebrew and downloading the ARM compiler the install script should compile QEMU. I'm surprised you're getting as far as you got, I'm getting:

*** Please add gcc binaries to your executable PATH:
    PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** Please set up arm-none-eabi-gdb before continuing.


And copy-pasting the PATH command doesn't help?

(the script cannot change environment variables outside it, and I didn't dig too far to find a better and cross-platform way to solve this)

Quote from: a1ex on September 28, 2017, 09:59:56 PM
And copy-pasting the PATH command doesn't help?

Nope. Though arm-none-eabi-gdb is in the path:

which arm-none-eabi-gdb
/Users/rosiefort/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gdb

Ah, it's because the script figures out the 64-bit GDB won't work...

Does the Mac run 32-bit binaries? If yes, can you try compiling a 32-bit GDB from source and place that one in PATH?

edit: found one here, does it help? https://acroname.com/software/arm-eabi-gcc-toolchain-mac-osx-macos-x-32bit

The Mac can run 32-bit binaries. Downloaded and placed in the right path but--

*** Installing dependencies for Mac...


*** WARNING: arm-none-eabi-gdb is not installed.
*** Downloading gcc-arm-embedded toolchain and installing it without the package manager.
*** Will be installed in your home directory (Makefile.user.default expects it there).

*** Will download the recommended 5.4-mac from gcc-arm-embedded.

*** Please add gcc binaries to your executable PATH:
    PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** Please set up arm-none-eabi-gdb before continuing.
RosieFoComputer:qemu rosiefort$ which arm-none-eabi-gdb
/Users/rosiefort/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gdb


It didn't download anything but it didn't recognize the installed arm-none-eabi-gdb even though as can see it is in the path.

Quote from: a1ex on September 28, 2017, 10:27:29 PM
Does the Mac run 32-bit binaries? If yes, can you try compiling a 32-bit GDB from source and place that one in PATH?

Until the next (2018? 2019?) release of MacOS it can run 32 bit without "compromise" https://developer.apple.com/news/?id=06282017a

@DeafEyeJedi could you copy past the commands and output from terminal, hard to read :) and could you edit manually contrib/qemu/install.sh and look at the end the grep -Po and if you don't have 'ggrep' the quick script failed to edit it. On my MacOS High Sierra VM freshly installed no problem with all commands ( try to copy past manually (http://www.magiclantern.fm/forum/index.php?topic=16012.msg190369#msg190369))

What does this print?


arm-none-eabi-gdb -v


Best guess: if both the 64-bit and the 32-bit toolchain are in PATH, the first one will be used.

Deafeyejedi´s problem isn´t grep related. He doesn´t get passed make -j cause processor info isn´t found(ubuntu command). I changed install.sh to:

echo "1) Compile QEMU"
echo
echo "   cd `pwd`/${QEMU_NAME}"
echo "   ./configure --target-list=arm-softmmu --disable-docs --enable-sdl"
echo "   make -j $(sysctl -n hw.physicalcpu)"

then running configure in qemu-1.6.0:
./configure --target-list=arm-softmmu --disable-docs --enable-sdl
   make -j 2

Compiles a lot but stops here:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libqemuutil.a(generated-tracers.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libqemuutil.a(host-utils.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libqemuutil.a(cache-utils.o) has no symbols
  LINK  qemu-ga
  LINK  qemu-nbd
Undefined symbols for architecture x86_64:
  "_use_rt_clock", referenced from:
      _bdrv_acct_start in block.o
      _bdrv_acct_done in block.o
      _qemu_get_clock_ns in qemu-timer.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [qemu-nbd] Error 1
make: *** Waiting for unfinished jobs....
dans-MacBook-Pro:qemu-1.6.0 dan$

*Error above seems related to qemu itself:
https://github.com/espes/xqemu/issues/85

Do we need version qemu1.6?
Seems latest version is qemu2.10.0

Also note that dfort didn´t update the script with this line from kichetof although I´m not sure it´s needed:
sed -i '' 's/grep/ggrep/g' install.sh

Also tried path(my gcc in my path):
PATH=~/gcc-arm-none-eabi-4_8-2013q4/bin:$PATH
and ran configure again but no difference.

Quote from: Danne on September 29, 2017, 07:43:00 AM
echo "   make -j $(sysctl -n hw.physicalcpu)"

echo "   make -j`$GREP -c processor /proc/cpuinfo 2> /dev/null || sysctl -n hw.ncpu 2> /dev/null || echo 1`"
Works well on my MBP (Sierra and High Sierra), I don't have it with me to paste output

ohhh... update your local repo!! :)

Quote from: Danne on September 29, 2017, 08:09:36 AM

Code Select Expand

sed -i '' 's/grep/ggrep/g' install.sh

That's the workarround, and don't forget:

sed -i '' 's/grep/ggrep/g' scripts/run_canon_fw.sh

Everywhere grep run with -P argument, we need to replace it with ggrep (not implemented in macOS grep)

dfort script needs updating to fit sed replacements to ggrep.

This is in install.sh, doesn´t play well with mac:
./configure --target-list=arm-softmmu --disable-docs --enable-sdl"
echo "   make -j`grep -c processor /proc/cpuinfo`"

Your line probably works for both linux and mac. Works on mac at least, just tried:
echo "   make -j`$GREP -c processor /proc/cpuinfo 2> /dev/null || sysctl -n hw.ncpu 2> /dev/null || echo 1`"

I also wonder if qemu needs update? I get the following error:
https://github.com/espes/xqemu/issues/85

Do we need version qemu1.6?
Seems latest version is qemu2.10.0

Are you running qemu fully working on your mac kichetof?


@dfort
Could you update bottom part of the script to:
read -p $(tput bold)"How about compiling QEMU?$(tput setaf 1)

Y/N?"$(tput sgr0) -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
cd magic-lantern
hg update qemu
cd contrib/qemu/
sed -i '' 's/grep/ggrep/g' install.sh
sed -i '' 's/grep/ggrep/g' scripts/run_canon_fw.sh
./install.sh
fi

Kichetof. Why even $GREP? Empty variable. This will do:
echo "   make -j`-c processor /proc/cpuinfo 2> /dev/null || sysctl -n hw.ncpu 2> /dev/null || echo 1`"

And then this will also work:
echo "   make -j`sysctl -n hw.ncpu`"

Updated the script but didn't have time to test it yet. I trust you have.

It would be nice if your prompts could accept carriage returns. If you type "Y [return]" on the cross compiler installation you'll never get to the QEMU option.

@Danne 50D works well and 5D3 only show SD card test (32 bit bug)

Don't forget to update your repo to qemu branch

hg update qemu -C

And the install.sh look like that (https://bitbucket.org/hudson/magic-lantern/src/1a0c8cc7da9250e7b440806a0b5207883f1ae94f/contrib/qemu/install.sh?at=qemu)

Oops. THanks Kichetof for the latest  :P.
Anyway. Stops at the same place as dfort
*** Installing dependencies for Mac...


*** WARNING: arm-none-eabi-gdb is not installed.
*** Downloading gcc-arm-embedded toolchain and installing it without the package manager.
*** Will be installed in your home directory (Makefile.user.default expects it there).

*** Will download the recommended 5.4-mac from gcc-arm-embedded.

*** Please add gcc binaries to your executable PATH:
    PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** Please set up arm-none-eabi-gdb before continuing.
dans-MacBook-Pro:

I guess if we´re updating to hg update qemu -C which seems needed there´s no need for these?:
sed -i '' 's/grep/ggrep/g' install.sh
sed -i '' 's/grep/ggrep/g' scripts/run_canon_fw.sh
?

As a matter of fact after running sed replace brew complains when running the install.sh scrip.
Error: No available formula with the name "ggrep"

Oh I forgot, my workarround with sed grep to ggrep was solved with latest update from a1ex.

Quote from: Danne on September 29, 2017, 09:44:33 AM
I guess if we´re updating to hg update qemu -C which seems needed there´s no need for these?:

Code Select Expand

sed -i '' 's/grep/ggrep/g' install.sh
sed -i '' 's/grep/ggrep/g' scripts/run_canon_fw.sh
?

Yes, say good by to my workarround :)

Now we use $GREP and set it to grep for linux and ggrep for macOS (https://bitbucket.org/hudson/magic-lantern/src/1a0c8cc7da9250e7b440806a0b5207883f1ae94f/contrib/qemu/install.sh?at=qemu&fileviewer=file-view-default#install.sh-49)

In the script you could try to do that before running install.sh:


echo 'export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH' >> ~/.bash_profile
. ~/.bash_profile


Quote from: Danne on September 29, 2017, 09:44:33 AM
As a matter of fact after running sed replace brew complains when running the install.sh scrip.

Code Select Expand

Error: No available formula with the name "ggrep"

brew list grep
which ggrep
grep -V

Great.
Latest take dfort:
Y/N?"$(tput sgr0) -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
cd magic-lantern
hg update qemu -C
cd contrib/qemu/
./install.sh
fi

Just updated the script to use a 32-bit gdb for Mac (untested) and a little more verbosity.

It also allows using a 64-bit GDB if you don't mind the bug (but asks for confirmation).

Tested latest:
Continue? [y/n] y

*** Installing dependencies for Mac...


*** WARNING: a valid arm-none-eabi-gdb could not be found.
*** Downloading a toolchain and installing it without the package manager.
*** Will be installed in your home directory (Makefile.user.default expects it there).

*** Will download macx_i386_gcc_arm_none_eabi_4.8.2 from:
    https://acroname.com/software/arm-eabi-gcc-toolchain-mac-osx-macos-x-32bit

--2017-09-29 09:58:48--  https://acroname.com/system/files/software/macx_i386_gcc_arm_none_eabi_4.8.2.tgz
Resolving acroname.com... 192.155.81.18
Connecting to acroname.com|192.155.81.18|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24839733 (24M) [application/x-gtar]
Saving to: 'macx_i386_gcc_arm_none_eabi_4.8.2.tgz'

macx_i386_gcc_arm_n 100%[===================>]  23.69M  2.49MB/s    in 17s     

2017-09-29 09:59:07 (1.42 MB/s) - 'macx_i386_gcc_arm_none_eabi_4.8.2.tgz' saved [24839733/24839733]

/Users/dan/magic-lantern/contrib/qemu
*** Please add gcc binaries to your executable PATH:
    PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** Please set up a valid arm-none-eabi-gdb before continuing.
dans-MBP:qemu dan$

Where is gcc versions downloaded? Before it went straight to my user folder but not now.
Noticed it downloads but is deleted before extracted.

Committed a fix - the printed PATH command was incorrect.

Did the trick. However, there´s no gdb to be found in the bin like with gcc 5.4:
macx_i386_gcc_ar 100%[==========>]  23.69M  2.32MB/s    in 14s     

2017-09-29 10:47:28 (1.72 MB/s) - 'macx_i386_gcc_arm_none_eabi_4.8.2.tgz' saved [24839733/24839733]

/Users/dan/magic-lantern/contrib/qemu
*** Please add gcc binaries to your executable PATH:
    PATH=~/cortex/4.8.2/bin:$PATH

*** Please set up a valid arm-none-eabi-gdb before continuing.
dans-MBP:qemu dan$

While we´re at it I renamed my version gcc-arm-none-eabi-4_8-2013q4 to gcc-arm-none-eabi-4_8-2013q3 which has gdb and the follow up to that is:

*** Installing dependencies for Mac...

*** WARNING: 64-bit GDB is known not to work.

*** WARNING: a valid arm-none-eabi-gdb could not be found.
*** Downloading a toolchain and installing it without the package manager.
*** Will be installed in your home directory (Makefile.user.default expects it there).

*** Will download macx_i386_gcc_arm_none_eabi_4.8.2 from:
wget: missing URL
Usage: wget [OPTION]... [URL]...

Try `wget --help' for more options.
/Users/dan/magic-lantern/contrib/qemu
*** Please add gcc binaries to your executable PATH:
    PATH=~//bin:$PATH

*** WARNING: 64-bit GDB is known not to work.
Continue anyway? [y/N] y


*** Using GDB: /Users/dan/gcc-arm-none-eabi-4_8-2013q3/bin/arm-none-eabi-gdb
GNU gdb (GNU Tools for ARM Embedded Processors) 7.6.0.20131129-cvs

*** Using GCC: /Users/dan/gcc-arm-none-eabi-4_8-2013q3/bin/arm-none-eabi-gcc
/Users/dan/magic-lantern/contrib/qemu/install.sh: line 286: syntax error near unexpected token `&'
dans-MBP:qemu dan$

It´s this line that has to be changed:
arm-none-eabi-gcc -v |& grep "gcc version"
I erase that line(why do we need it?

Script finishes and then:
cd /Users/dan/qemu/qemu-2.5.0
   ../configure_eos.sh
   make -j4
'
Succes!
Ending lines.
CC    arm-softmmu/target-arm/gdbstub.o
  CC    arm-softmmu/target-arm/psci.o
6 warnings generated.
  CC    arm-softmmu/target-arm/crypto_helper.o
  GEN   trace/generated-helpers.c
  CC    arm-softmmu/gdbstub-xml.o
  CC    arm-softmmu/trace/generated-helpers.o
  LINK  arm-softmmu/qemu-system-arm
  REZ   arm-softmmu/qemu-system-arm
  SETFILE arm-softmmu/qemu-system-arm
dans-MBP:qemu-2.5.0 dan$

Got it, back to the 64-bit one for now...

If you are able to compile a 32-bit GDB, would be nice to use that one.

Got it working. Check updated post above :).

We should use this version:
gcc-arm-none-eabi-4_8-2013q4
https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2

erase or rewrite this line:
arm-none-eabi-gcc -v |& grep "gcc version"

Already done - any issues with 5.4? (4.8 is also 64-bit on Mac)

I don´t see no reason for it not to work with 5.4. I´ve been focused on reported errors.

The issue why it´s not working is simply because after the new PATH is set it will not find gdb since it scripted like this:
arm-none-eabi-gdb -v
Our Path i set so now it has to be this:
gdb

e.g:
function valid_arm_gdb {
    if ! gdb -v &> /dev/null; then
        # not installed, or not able to run for any reason
        return 1
    fi

At least it seems to work here to get past all the brakes.

Full script:

#!/usr/bin/env bash

set -e

QEMU_NAME=${QEMU_NAME:=qemu-2.5.0}
ML=${ML:=magic-lantern}
GREP=${GREP:=grep}

echo
echo "This will setup QEMU for emulating Magic Lantern."
echo "Thou shalt not be afraid of compiling stuff on Linux ;)"
echo -n "Continue? [y/n] "
read answer
if test "$answer" != "Y" -a "$answer" != "y"; then exit 0; fi
echo

function install_gdb {
    echo
    echo "*** Will download gcc-arm-none-eabi-5_4-2016q3 from:"
    echo "    https://developer.arm.com/open-source/gnu-toolchain/gnu-rm"
    echo

    TOOLCHAIN=gcc-arm-none-eabi-5_4-2016q3
    DOWNLOAD=https://developer.arm.com/-/media/Files/downloads/gnu-rm/5_4-2016q3/
    UNTAR="tar -jxf"

    if [ $(uname) == "Darwin" ]; then
        TARBALL=gcc-arm-none-eabi-5_4-2016q3-20160926-mac.tar.bz2
    else
        TARBALL=gcc-arm-none-eabi-5_4-2016q3-20160926-linux.tar.bz2
    fi

    if [ ! -f ~/$TOOLCHAIN/bin/arm-none-eabi-gdb ]; then
        cd ~
        wget -c $DOWNLOAD$TARBALL && $UNTAR $TARBALL && rm $TARBALL
        cd -
    else
        echo "*** Toolchain already installed in:"
        echo "    ~/$TOOLCHAIN"
        echo
    fi

    echo "*** Please add gcc binaries to your executable PATH:"
    echo "    PATH=~/$TOOLCHAIN/bin:\$PATH"
    echo
}


function valid_arm_gdb {
    if ! gdb -v &> /dev/null; then
        # not installed, or not able to run for any reason
        return 1
    fi

    if gdb -v | grep -q "host=x86_64"; then
        # 64-bit version - doesn't work
        # fixme: this may get printed more than once
        echo "*** WARNING: 64-bit GDB is known not to work."
        return 1
    fi

    # assume it's OK
    # todo: check version number
    return 0
}

function valid_arm_gcc {
    if ! gcc -v &> /dev/null; then
        # not installed, or not able to run for any reason
        return 1
    fi

    echo "#include <stdlib.h>" > arm-gcc-test.c
    if ! gcc -c arm-gcc-test.c; then
        echo "*** WARNING: your gcc is unable to compile a simple program."
        rm arm-gcc-test.c
        return 1
    fi

    rm arm-gcc-test.c
    return 0
}

if [ $(uname) == "Darwin" ]; then
    echo "*** Installing dependencies for Mac..."
    echo
    # fixme: don't these require sudo?
    # can we check whether they are already installed, as on Ubuntu?
    if ! xcode-select -p &> /dev/null; then
        xcode-select --install
    fi
    # brew is "The missing package manager for macOS"
    # https://brew.sh
    if ! brew -v &> /dev/null; then
        ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
    fi
   
    packages="python wget mercurial xz grep pkg-config glib automake libtool pixman mtools"
    for pkg in $packages; do
        brew list $pkg &> /dev/null || brew install $pkg
    done
   
    GREP=ggrep
fi

if apt-get -v &> /dev/null; then
    # apt-based system?
    # install these packages, if not already
    # only request sudo if any of them is missing
    # instead of GTK (libgtk2.0-dev), you may prefer SDL (libsdl1.2-dev)
    packages="
        build-essential mercurial pkg-config libtool
        git libglib2.0-dev libpixman-1-dev zlib1g-dev
        libgtk2.0-dev xz-utils mtools netcat-openbsd
        python python-pip python-docutils"

    # if a valid arm-none-eabi-gcc/gdb is already in PATH, try to use that
    # otherwise, we'll try to install something
    if ! valid_arm_gdb || ! valid_arm_gcc; then
        echo "*** You do not seem to have an usable arm-none-eabi-gcc and/or gdb installed."
        echo "*** 64-bit GDB is known not to work, so you'll have to install a 32-bit one for now."
        echo
        echo "*** You have a few options:"
        echo
        echo "1 - Install gdb-arm-none-eabi:i386 and gcc-arm-none-eabi from Ubuntu repo (recommended)"
        echo "    This will install 32-bit binaries - will not work under Windows Subsystem for Linux."
        echo
        echo "2 - Download a 32-bit gcc-arm-embedded and install it without the package manager."
        echo "    Will be installed in your home directory; to move it, you must edit the Makefiles."
        echo "    This will install 32-bit binaries - will not work under Windows Subsystem for Linux."
        echo
        echo "3 - Install gdb-arm-none-eabi from Ubuntu repository (64-bit)"
        echo "    WARNING: this will not be able to run all our GDB scripts."
        echo
        if dpkg -l binutils-arm-none-eabi 2>/dev/null | grep -q '^.i'; then
            echo "4 - Remove Ubuntu toolchain and install the one from gcc-arm-embedded PPA (gcc 6.x)"
            echo "    This will:"
            echo "    - sudo apt-get remove gcc-arm-none-eabi gdb-arm-none-eabi \\"
            echo "           binutils-arm-none-eabi libnewlib-arm-none-eabi"
        else
            echo "4 - Install the toolchain from gcc-arm-embedded PPA (gcc 6.x)"
            echo "    This will:"
        fi
        echo "    - sudo add-apt-repository ppa:team-gcc-arm-embedded/ppa"
        echo "    - install the gcc-arm-embedded package."
        echo "    WARNING: this will not be able to run all our GDB scripts."
        echo
        echo "5 - Manually install arm-none-eabi-gdb from https://launchpad.net/gcc-arm-embedded"
        echo "    or any other source, make sure it is in PATH, then run this script again."

        if ! gdb -v &> /dev/null; then
            echo
        else
            echo
            echo "6 - Just use the current 64-bit toolchain."
            echo "    WARNING: this will not be able to run all our GDB scripts."
        fi

        echo
        echo -n "Your choice? "
        read answer
        echo
        case $answer in
            1)
                # Ubuntu's 32-bit arm-none-eabi-gdb works fine
                # gcc-arm-none-eabi:i386 does not include libnewlib - Ubuntu bug?
                packages="$packages gdb-arm-none-eabi:i386 "
                packages="$packages gcc-arm-none-eabi libnewlib-arm-none-eabi"
                ;;
            2)
                # 32-bit gdb will be downloaded after installing these packages
                packages="$packages libc6:i386 libncurses5:i386"
                ;;
            3)
                # Ubuntu's 64-bit arm-none-eabi-gdb works... sort of
                # it's unable to run 5D3 1.1.3 GUI and maybe others
                packages="$packages gdb-arm-none-eabi:amd64"
                packages="$packages gcc-arm-none-eabi:amd64 libnewlib-arm-none-eabi:amd64"
                ;;
            4)
                # gcc-arm-embedded conflicts with gcc-arm-none-eabi
                # but the dependencies are not configured properly
                # so we have to fix the conflict manually...
                if dpkg -l binutils-arm-none-eabi 2>/dev/null | grep -q '^.i'; then
                    echo
                    echo "*** Please double-check - the following might remove additional packages!"
                    echo
                    sudo apt-get remove gcc-arm-none-eabi gdb-arm-none-eabi \
                         binutils-arm-none-eabi libnewlib-arm-none-eabi
                fi
                packages="$packages gcc-arm-embedded"
                echo
                echo "*** Adding the team-gcc-arm-embedded PPA repository..."
                echo "    sudo add-apt-repository ppa:team-gcc-arm-embedded/ppa"
                echo
                sudo add-apt-repository ppa:team-gcc-arm-embedded/ppa
                ;;
            5)
                # user will install arm-none-eabi-gdb and run the script again
                exit 0
                ;;
            6)
                # use the installed version, even though it's known not to work
                ;;
            *)
                # invalid choice
                exit 1
                ;;
        esac
    else
        echo "*** You have a valid ARM GCC/GDB already installed - using that one."
    fi

    echo "*** Checking dependencies for Ubuntu..."
    echo
    # https://wiki.debian.org/ListInstalledPackages
    # dpkg -l also returns packages that are not installed
    deps_installed=yes
    for package in $packages; do
        if ! dpkg -l $package 2>/dev/null | grep -q '^.i'; then
            echo Not installed: $package
            deps_installed=no
        fi
    done

    if [ "$deps_installed" == "no" ]; then
        echo
        echo "*** Installing dependencies for Ubuntu..."
        echo
        sudo apt-get update
        sudo apt-get install $packages
        echo
    fi

    deps_installed=yes
    for package in $packages; do
        if ! dpkg -l $package 2>/dev/null | grep -q '^.i'; then
            echo Not installed: $package
            deps_installed=no
        fi
    done

    if [ "$deps_installed" == "no" ]; then
        echo
        echo "*** Error: Ubuntu dependencies could not be installed."
        echo
        exit 1
    fi
fi

# all systems (including Mac, or Ubuntu if the installation from repositories failed)
# this one works on old systems as well, but it won't work under WSL
if ! valid_arm_gdb; then
    echo
    echo "*** WARNING: a valid arm-none-eabi-gdb could not be found."
    echo "*** Downloading a toolchain and installing it without the package manager."
    echo "*** Will be installed in your home directory (Makefile.user.default expects it there)."
    echo
    install_gdb
fi

# make sure we have a valid arm-none-eabi-gdb (regardless of operating system)
if ! valid_arm_gdb; then
    if ! arm-none-eabi-gdb -v &> /dev/null; then
        echo "*** Please set up a valid arm-none-eabi-gdb before continuing."
        exit 1
    else
        # valid_arm_gdb will print why the current one is not good
        echo -n "Continue anyway? [y/N] "
        read answer
        if test "$answer" != "Y" -a "$answer" != "y"; then exit 1; fi
        echo
    fi
fi

# same for arm-none-eabi-gcc
if ! valid_arm_gcc; then
    echo "*** Please set up a valid arm-none-eabi-gcc before continuing."
    exit 1
fi

echo
echo -n "*** Using GDB: "
command -v arm-none-eabi-gdb
arm-none-eabi-gdb -v | head -n1
echo
echo -n "*** Using GCC: "
command -v arm-none-eabi-gcc
arm-none-eabi-gcc -v 2>&1 | grep "gcc version"
echo

# install docutils (for compiling ML modules) and vncdotool (for test suite)
# only install if any of them is missing
pip2 list | grep docutils  || rst2html -h  > /dev/null || pip2 install docutils
pip2 list | grep vncdotool || vncdotool -h > /dev/null || pip2 install vncdotool

function die { echo "${1:-"Unknown Error"}" 1>&2 ; exit 1; }

pwd | grep $ML/contrib/qemu > /dev/null || die "error: we should be in $ML/contrib/qemu"

# go to the parent of magic-lantern folder
cd ../../..
ls | $GREP $ML > /dev/null || die "error: expecting to find $ML here"

mkdir -p qemu
cd qemu

echo
echo "*** Setting up QEMU in `pwd`..."
echo

# get qemu
wget -q --show-progress --progress=dot:giga -c http://wiki.qemu-project.org/download/$QEMU_NAME.tar.bz2
echo
tar jxf $QEMU_NAME.tar.bz2
echo

# initialize a git repo, to make it easy to track changes to QEMU source
cd $QEMU_NAME
if [ ! -d .git ]; then
  git init
  # git requires a valid email; if not setup, add one for this directory only
  git config user.email || git config user.email [email protected]
  git add .
  git commit -q -m "$QEMU_NAME vanilla"
fi
cd ..

echo "Copying files..."

# copy our helper scripts
cp -r ../$ML/contrib/qemu/scripts/* .
chmod +x *.sh

# copy our testing scripts
mkdir -p tests
cp -r ../$ML/contrib/qemu/tests/* tests/
chmod +x tests/*.sh

# apply our patch
cd ${QEMU_NAME}
mkdir -p hw/eos
cp -r ../../$ML/contrib/qemu/eos/* hw/eos/
cp -r ../../$ML/src/backtrace.[ch] hw/eos/dbi/
patch -N -p1 < ../../$ML/contrib/qemu/$QEMU_NAME.patch
cd ..

# setup the card image
if [ ! -f "sd.img" ]; then
    echo "Setting up SD card image..."
    cp -v ../$ML/contrib/qemu/sd.img.xz .
    unxz -v sd.img.xz
else
    echo "SD image already exists, skipping."
fi

if [ ! -f "cf.img" ]; then
    echo "Setting up CF card image..."
    cp -v sd.img cf.img
else
    echo "CF image already exists, skipping."
fi

echo ""
echo "Next steps:"
echo "==========="
echo
echo "1) Compile QEMU"
echo
echo "   cd `pwd`/${QEMU_NAME}"
echo "   ../configure_eos.sh"
echo "   make -j`$GREP -c processor /proc/cpuinfo 2> /dev/null || sysctl -n hw.ncpu 2> /dev/null || echo 1`"
echo
echo "2) Grab a copy of the Canon firmware from your own camera"
echo "   (don't request one and don't share it online - it's copyrighted)"
echo
echo "   Look on your SD card - you should find ML/LOGS/ROM0.BIN and ROM1.BIN"
echo "   Copy them under your camera model's subdirectory, for example:"
echo "   `pwd`/60D/"
echo
echo "   For models that use a serial flash, you may have to dump its contents"
echo "   using the sf_dump module, then copy SFDATA.BIN as well."
echo
echo "3) Mount the included SD (or CF) image (you may use mount.sh)"
echo "   and install ML on it, as usual. The card image must be bootable as well."
echo
echo "   The included card image is bootable and contains a small autoexec.bin"
echo "   that runs on all DIGIC 4/5 cameras and prints some basic info."
echo
echo "   To create your own SD/CF image, you need to copy the raw contents"
echo "   of the entire card, not just one partition. For example:"
echo "   dd if=/dev/mmcblk0 of=sd.img"
echo
echo "4) Start emulation with:"
echo
echo "   cd `pwd`/"
echo "   ./run_canon_fw.sh 60D"
echo
echo "   This will recompile QEMU, but not ML."
echo
echo "   Note: Canon GUI emulation (menu navigation, no LiveView) only works on"
echo -n "   "; $GREP --color=never -oP "(?<=GUI_CAMS=\( ).*(?=\))" tests/run_tests.sh;
echo
echo "5) Tips & tricks:"
echo "   - to enable or disable the boot flag in ROM, use something like:"
echo "     ./run_canon_fw.sh 60D,firmware=\"boot=1\""
echo "   - to use multiple firmware versions, place the ROMs under e.g. 5D3/113/ROM*.BIN and run:"
echo "     ./run_canon_fw.sh 5D3,firmware=\"113;boot=1\""
echo "   - to show MMIO activity (registers) and interrupts, use:"
echo "     ./run_canon_fw.sh 60D -d io,int"
echo "   - to show the executed ASM code, step by step, use:"
echo "     ./run_canon_fw.sh 60D -d exec,int -singlestep"
echo "   - to trace debug messages and various functions in the firmware, use:"
echo "     ./run_canon_fw.sh 60D -s -S & arm-none-eabi-gdb -x 60D/debugmsg.gdb"
echo "   - if the above is too slow, compile the dm-spy-experiments branch "
echo "     with CONFIG_QEMU=y and CONFIG_DEBUG_INTERCEPT_STARTUP=y and try:"
echo "     ./run_canon_fw.sh 60D,firmware=\"boot=1\" -d io,int"
echo "   - some camera models require GDB patches to bypass tricky code sequences:"
echo "     ./run_canon_fw.sh 700D -s -S & arm-none-eabi-gdb -x 700D/patches.gdb"
echo "   - to trace all function calls and export them to IDA:"
echo "     ./run_canon_fw.sh 60D -d calls -singlestep"
echo "   - you may enable additional debug code (such as printing to QEMU console)"
echo "     by compiling ML with CONFIG_QEMU=y in your Makefile.user (also run make clean)."
echo "   - caveat: you cannot run autoexec.bin compiled with CONFIG_QEMU on the camera."

echo
echo "Enjoy!"

arm-none-eabi-gdb is correct - double-check the path. It should be:


PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH


(http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/85-type.png) (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/92.png)

You´re right. I was calling system specific gdb. However. The issue is that the checking command doesn´t return that it finds gdb. Will check more.

Ok, been through a few rounds now but it seems to boil down to the path not being properly set:
    echo "   PATH=~/$TOOLCHAIN/bin:\$PATH"

Only echoing the path won´t work. The backslash breaks it as well.

Usually I would do:
export PATH=~/$TOOLCHAIN/bin/:$PATH

This works:
    echo "   PATH=~/$TOOLCHAIN/bin/:$PATH"
    PATH=~/$TOOLCHAIN/bin/:$PATH

And this:
    echo "   PATH=~/$TOOLCHAIN/bin/:$PATH"
    export PATH=~/$TOOLCHAIN/bin/:$PATH

Here is dfort script. Should get you right into qemu after all else is installed:
cat <<'EOF' > mac_ml.sh
#!/bin/sh

#  mac_ml.sh
#
#  Script to almost automatically install
#  and configure a Magic Lantern development
#  environment on a Macintosh.
#
#  Reference:
#  http://magiclantern.fm/forum/index.php?topic=16012.0
#
#  Daniel Fort (dfort) - 2015-10-15
#  Modified 2016-03-17 - took out ugly gcc hack
#  Modified 2017-06-27 - added mingw-w64 for cross compiling Windows binaries
#  Modified 2017-06-28 - check if magic-lantern already exists
#  Modified 2017-07-12 - write a Makefile.user file
#  Modified 2017-09-23 - made mingw-w64 (Windows cross compiler) optional
#  Modified 2017-09-24 - added interactive options for mingw-w64 and QEMU
#  Modified 2017-09-26 - QEMU install.sh now installs dependencies so no need to do it here
#                                   now using gcc-arm-none-eabi-5_4-2016q3-20160926-mac.tar.bz2
#
#  Thanks to Danne and kichetof and of course a1ex for their input

if ! test -d /usr/local; then
  echo "/usr/local directory not found."
  echo "Follow the steps at the beginning of"
  echo "tutorial to create /usr/local and"
  echo "then re-run this script."
  exit 1
fi

echo "/usr/local found -- "
echo "continuing with installation."

cd ~
if [ ! -d "/Library/Developer/CommandLineTools" ]; then xcode-select --install; fi
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install python wget mercurial

#
# Uncomment the following line if you want to cross compile Windows binaries.
# brew install mingw-w64
#

pip2 install docutils

brew tap homebrew/dupes; brew install homebrew/versions/gcc5

cd ~ && \
    [ ! -f gcc-arm-none-eabi-4_8-2013q4 ] &&
    wget -c https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && \
    tar -jxf gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && \
    rm gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2

if ! test -d ~/magic-lantern; then
    hg clone https://bitbucket.org/hudson/magic-lantern
    cd magic-lantern
    hg update unified
    cat <<'EOT' > Makefile.user
#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)
EOT
    cd ~
fi

#OPTIONAL 1
clear
read -p $(tput bold)"Would you also like to install the windows cross compiler mingw-w64?$(tput setaf 1)

Y/N?"$(tput sgr0) -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
clear
read -p $(tput bold)"Would you also like to compile QEMU?$(tput setaf 1)
Y/N?"$(tput sgr0) -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
clear
echo "
installing mingw-w64"
brew install mingw-w64 || :
#compiling qemu
clear
echo "
compiling qemu"
cd magic-lantern
hg update qemu -C
cd contrib/qemu/
TOOLCHAIN=gcc-arm-none-eabi-5_4-2016q3
export PATH=~/$TOOLCHAIN/bin/:$PATH
./install.sh
else
clear
echo "
installing mingw-w64"
brew install mingw-w64 || :
fi
else
#OPTIONAL 2
clear
read -p $(tput bold)"How about compiling QEMU?$(tput setaf 1)

Y/N?"$(tput sgr0) -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
cd magic-lantern
hg update qemu -C
cd contrib/qemu/
TOOLCHAIN=gcc-arm-none-eabi-5_4-2016q3
export PATH=~/$TOOLCHAIN/bin/:$PATH
./install.sh
fi
fi

cd ~
exit 0
EOF
bash mac_ml.sh && rm mac_ml.sh

Of course, echoing the path won't work, you need to copy/paste that command and run it outside the script (or place it into your .profile if you want). And I don't want to echo the entire path, that's the point of "\$".

The script itself is not able to set the path outside it (it won't change the environment of its parent shell (https://stackoverflow.com/questions/15541321/set-a-parent-shells-variable-from-a-subshell)). That's why I'm echoing the command.

The script itself has no need to run arm-none-eabi-gdb/gcc - it only makes sure *you* are going to be able to run them, as described in the QEMU guide (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/).

See my screenshots above and this animation (http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/xenial2.gif).

I see. .profile or manually copy paste. Won't be in here for a while. Maybe dfort takes a look before me.
Thanks.

Great progress guys on troubleshooting this QEMU bad boy down. Here's a quick summary after my 2nd attempt at installing QEMU...

==> Summary
🍺  /usr/local/Cellar/isl/0.18: 80 files, 3.8MB
==> Installing mingw-w64
==> Downloading https://homebrew.bintray.com/bottles/mingw-w64-5.0.2_3.sierra.bo
######################################################################## 100.0%
==> Pouring mingw-w64-5.0.2_3.sierra.bottle.tar.gz
🍺  /usr/local/Cellar/mingw-w64/5.0.2_3: 7,834 files, 669MB


compiling qemu
0 files updated, 0 files merged, 0 files removed, 0 files unresolved

This will setup QEMU for emulating Magic Lantern.
Thou shalt not be afraid of compiling stuff on Linux ;)
Continue? [y/n] y

*** Setting up QEMU in /Users/SMJ/qemu...

--2017-09-29 08:42:03--  http://wiki.qemu-project.org/download/qemu-1.6.0.tar.bz2
Resolving wiki.qemu-project.org... 172.99.69.163
Connecting to wiki.qemu-project.org|172.99.69.163|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://wiki.qemu.org/download/qemu-1.6.0.tar.bz2 [following]
--2017-09-29 08:42:04--  http://wiki.qemu.org/download/qemu-1.6.0.tar.bz2
Resolving wiki.qemu.org... 172.99.69.163
Reusing existing connection to wiki.qemu-project.org:80.
HTTP request sent, awaiting response... 302 Found
Location: https://wiki.qemu.org/download/qemu-1.6.0.tar.bz2 [following]
--2017-09-29 08:42:06--  https://wiki.qemu.org/download/qemu-1.6.0.tar.bz2
Connecting to wiki.qemu.org|172.99.69.163|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://download.qemu.org/qemu-1.6.0.tar.bz2 [following]
--2017-09-29 08:42:07--  https://download.qemu.org/qemu-1.6.0.tar.bz2
Resolving download.qemu.org... 172.99.69.163
Connecting to download.qemu.org|172.99.69.163|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

    The file is already fully retrieved; nothing to do.

../magic-lantern/contrib/qemu/scripts/copy_back_to_contrib.sh -> ./copy_back_to_contrib.sh
../magic-lantern/contrib/qemu/scripts/gdbopts -> ./gdbopts
../magic-lantern/contrib/qemu/scripts/run_canon_fw.sh -> ./run_canon_fw.sh
../magic-lantern/contrib/qemu/scripts/run_ml.sh -> ./run_ml.sh
../magic-lantern/contrib/qemu/scripts/run_ml_100D.sh -> ./run_ml_100D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_500D.sh -> ./run_ml_500D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_50D.sh -> ./run_ml_50D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_550D.sh -> ./run_ml_550D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_5D2.sh -> ./run_ml_5D2.sh
../magic-lantern/contrib/qemu/scripts/run_ml_5D3.sh -> ./run_ml_5D3.sh
../magic-lantern/contrib/qemu/scripts/run_ml_5D3_123.sh -> ./run_ml_5D3_123.sh
../magic-lantern/contrib/qemu/scripts/run_ml_600D.sh -> ./run_ml_600D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_60D.sh -> ./run_ml_60D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_650D.sh -> ./run_ml_650D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_6D.sh -> ./run_ml_6D.sh
../magic-lantern/contrib/qemu/scripts/run_ml_7D.sh -> ./run_ml_7D.sh
../../magic-lantern/contrib/qemu/hw/eos.c -> hw/arm/eos.c
../../magic-lantern/contrib/qemu/hw/eos.h -> hw/arm/eos.h
patching file hw/arm/Makefile.objs
patching file memory.c
patching file target-arm/cpu.c

Next steps:
===========

1) Compile QEMU

   cd /Users/SMJ/qemu/qemu-1.6.0
   ./configure --target-list=arm-softmmu --disable-docs --enable-sdl
grep: /proc/cpuinfo: No such file or directory
   make -j

2) Grab a copy of the Canon firmware from your own camera
   (don't request one and don't share it online - it's copyrighted)

   Look on your SD card - you should find ML/LOGS/ROM0.BIN and ROM1.BIN
   Copy those in /Users/SMJ/qemu/ and then run (for 60D):

   cat ROM0.BIN ROM1.BIN > ROM-60D.BIN

3) Enable CONFIG_QEMU=y in your Makefile.user
   from magic-lantern directory, then run 'make clean' to make sure
   you will rebuild ML from scratch.

4) Create a subdirectory named 'sdcard' or 'cfcard' and copy ML files there.
   Make sure all ML files are UPPERCASE (just copy them from a FAT32 card).
   You should get something like:
   /Users/SMJ/qemu/sdcard/ML/AUTOEXEC.BIN
   /Users/SMJ/qemu/sdcard/ML/FONTS/ARGNOR32.RBF
   /Users/SMJ/qemu/sdcard/ML/MODULES/ARKANOID.MO
   and so on.

5) Start emulation with:

   cd /Users/SMJ/qemu/
   ./run_ml_60D.sh

   (this will recompile ML and QEMU - handy if you edit the sources often)

Enjoy!
Seans-Mac-mini-385:~ DeafEyeJedi$


(https://farm5.staticflickr.com/4502/23540578708_43935b326a.jpg) (https://flic.kr/p/BScBUG)

Does this mean it's some kind of a success?

Uh-oh...

Seans-Mac-mini-385:~ DeafEyeJedi$ cd /Users/SMJ/qemu/qemu-1.6.0
Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$    ./configure --target-list=arm-softmmu --disable-docs --enable-sdl

ERROR: User requested feature sdl
       configure was not able to find it

Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$ grep: /proc/cpuinfo: No such file or directory
-bash: grep:: command not found
Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$    make -j
make: *** No rule to make target `/qemu-options.hx', needed by `qemu-options.def'.  Stop.
make: *** Waiting for unfinished jobs....
Please call configure before running make!
make: *** [config-host.mak] Error 1
Seans-Mac-mini-385:qemu-1.6.0 DeafEyeJedi$


Did I miss a step in here?

When all else fails, read the instructions (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/#rst-header-installation) (and replies #132, #149, or this (http://www.magiclantern.fm/forum/index.php?topic=2864.msg190596#msg190596) or latest commits (https://bitbucket.org/hudson/magic-lantern/commits/all) or the latest tweet (https://twitter.com/autoexec_bin/status/913530810686418944)...)

@DeafEyeJedi -- You keep running the error message. Don't just blindly copy and paste. Read what it says first--grep is trying to tell you something is wrong!

1) Compile QEMU

   cd /Users/SMJ/qemu/qemu-1.6.0
   ./configure --target-list=arm-softmmu --disable-docs --enable-sdl
grep: /proc/cpuinfo: No such file or directory
   make -j


That said, the reason this isn't going well for you is because things have changed recently and you are trying to install QEMU without having the latest ML Mac development environment. Let's start over:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
cd /usr/local
ls


[EDIT - removed the suggestion to use "rm -rf *"] Got anything left in there? It is ok to get rid of it if you're going to do a new Homebrew installation.

I'd do it a step at a time, you don't want to force remove everything if you're in the wrong directory!

Let's also delete the ML repository and the gcc toolchain. (That "cd" is to get you into your home directory, it is a shortcut for "cd ~/")

cd
rm -rf magic-lantern
rm -rf gcc-arm-none-eabi*

Now run the quick install script on the first post--I keep updating it with the latest suggestions.

If you answer "Y" to the QEMU install prompt it won't work the first time. The script installs a newer version of the gcc-arm-none-eabi toolchain and ends up with this message:

*** Please add gcc binaries to your executable PATH.
*** Run this command, or paste it into your .profile and reopen the terminal:
    export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** Please set up a valid arm-none-eabi-gdb before continuing.


So do exactly what that message says:

export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

At this point my script has done all that it could and invoked the QEMU install.sh script so:

cd ~/magic-lantern
hg update qemu -C
cd contrib/qemu/
./install.sh


You will probably see something like this:

*** Will download gcc-arm-none-eabi-5_4-2016q3 from:
    https://developer.arm.com/open-source/gnu-toolchain/gnu-rm

*** Toolchain already installed in:
    ~/gcc-arm-none-eabi-5_4-2016q3

*** Please add gcc binaries to your executable PATH.
*** Run this command, or paste it into your .profile and reopen the terminal:
    export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** WARNING: 64-bit GDB is known not to work.
Continue anyway? [y/N] y


Go for it, it works on the Mac. Well there maybe some things that don't work but this should be good enough to get you started so study those links a1ex gave you and head on over to the QEMU topic and join the fun. Here's a quick tip--since you have a 100D, find the folder for that camera in your ~/qemu directory and put your firmware dumps in there, ROM0.BIN, ROM1.BIN and SFDATA.BIN - yes, you need to dump your serial flash but you know how to do that, right? Then in your terminal:

cd ~/qemu
./run_canon_fw.sh 100D

Well? Did it work?

I wouldn't run rm -rf * on anything. What if the folder usr/local doesn't exist even?

Quote from: Danne on September 30, 2017, 07:38:12 AM
usr/local

Make sure you're in "/usr/local" -- in any case I edited my post even though I did put in a warning about using "rm -rf *"

Thanks @Danne for the suggestions (especially on not using rm -rf * on anything) and @dfort for the comprehensive details. I will report my findings once this shitstorm of mine settles down this weekend.

This also reminds me that I should be looking for an old used MB Air to do this kind of work where there are no important files on the system for future references.  :P

Latest qemu branch on a fresh Mac VM:

Quote from: kichetof on September 24, 2017, 03:44:08 PM
Virtualization  8)


ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install mercurial

hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg update qemu -C
cd contrib/qemu
./install.sh


(http://a1ex.magiclantern.fm/bleeding-edge/qemu/install/mac.gif)

Thanks kichetof 8)

Quote from: a1ex on October 15, 2017, 08:28:37 PM
Thanks kichetof 8)

Glad to help you  8)  minor contribution for all your great works! :)

Happy to see Mac development on top level!

Looks like it is time to rebuild my QEMU environment. Thanks for making ML development on Mac so easy--well setting up the environment is easy, the rest is still pretty hard.

Thanks for all the effort setting up this tutorial. You guys are heros. I ran into an issue on my iMac running High Sierra (10.13.1). (Xcode 9.1 (9B55) installed but not used. From the terminal, after I proceeded with this step:

    brew tap homebrew/dupes/; brew install homebrew/versions/gcc5

I did receive the following error, though suspect it may not be a big deal:

    Error: Invalid tap name 'homebrew/dupes/'

The script went along fine, but then at this step:

    Hunk #1 succeeded at 228 with fuzz 2 (offset 12 lines).
    ==> ../configure --build=x86_64-apple-darwin17.2.0 --prefix=/usr/local/Cellar/gcc@5/5.5.0 --libdir=/usr/local/Cellar/gcc@5/5.5.0/lib/gcc/5 --enable-la
    ==> make bootstrap

everything seemed kind of stalled. The titlebar of my Terminal window was looping through multiple commands, but this kept going for like 10+ minutes. I wasn't sure if this was normal behavior or if it was stuck in some loop. I can post a screen video grab if needed. I ended up stopping all the processes and killing them. I read on a prior post here that someone said something took 37 minutes to complete, so perhaps it wasn't looping but I just need to wait for it to complete? If so, it would be useful to update the install instructions and let folks know that it may take a while to complete this step. Any confirmation would be helpful. Thanks.

Looks like there was another change at Homebrew:

Warning: Use gcc@5 instead of deprecated homebrew/versions/gcc5

I'll need to update the tutorial.

The stall you're experiencing is the compiler compiling a compiler from scratch. It takes a long time but if you let it keep running it should eventually finish and work. There should be a binary version available so that shouldn't be necessary. I'll update this topic when I get things working properly again.

Awesome. Thank you. By the way, while I was puttering around, I decided to try running the full script at the bottom of the article and that seemed to complete everything with no issues and without the long wait. Not sure why (perhaps most of the work was completed the first time through). All good now. Thanks for the quick response.

Glad you got it working. I reworked the script to accommodate that gcc@5 change in Homebrew and while I was at it improved the QEMU installation. The script can be run even if you have a development environment already setup, it won't clobber your current installation (I hope).  :P

@dfort no need to symlink ggrep (https://bitbucket.org/hudson/magic-lantern/src/4895777de907c24ffd6332bcee23a7608450f6bd/contrib/qemu/install.sh?at=qemu&fileviewer=file-view-default#install.sh-103)

And if grep need -P, update script to use in same way  8)

Brew checks if Xcode commandline are already installed https://bitbucket.org/hudson/magic-lantern/commits/b145d5f6489d3d960d2a8d17a031748c5980684d?at=qemu

You're right -- removed the symlink. Thanks for the update.

Have you noticed that the script can now install QEMU from soup to nuts?

Ran the latest scripted installation from OP while on 10.13 w this MBP that already had previous compiling environments.

Love it that it can also install QEMU as an option towards the end.  8)



Seems all good to me even w the PIP update command that I ran (hence the in video above) afterwards w ease.

Thanks @dfort for the quick fix after @henrus's report. Good catch on that!

Great video @DeafEyeJedi - I like your choice of colors.  :P

Quote from: dfort on November 17, 2017, 06:58:06 PM
Have you noticed that the script can now install QEMU from soup to nuts?

Yeah! works well!  8)

HELP! I'm trying to sink my teeth into compiling but I'm hitting a snag, I'm at the part where you ask to modify Makefile.user.default. But I don't see that file or all the different filed you ask us to edit as we go, so I'm kind of stuck after "brew install docutils". I think I'm going to start over again but I just wanted some advice.

Are you running the quick installation script from the bottom of the first post?
The makefile you're referring to is in the root folder. Just copy it and rename it to Makefile.user and put in the suggested changes.

Actually, you don't even need to do that. The quick installation script does it for you:

    cat <<'EOT' > Makefile.user
#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)
EOT


Here's a little secret -- you don't even need this for compiling ML. However, sooner or later you'll want to compile cr2hdr and you'll tear your hair out trying to figure out what is going on and it will always end up that you are missing this Makefile.user file.

Danne just published Compiler.app (https://www.magiclantern.fm/forum/index.php?topic=21882.0), a menu driven app that simplifies setting up and running an ML development environment.

He included the quick install script and started tweaking it to make it more user friendly. So instead of having the quick install script located at the bottom of the first post in this topic it is now on Bitbucket which means it is under full revision control and will be managed just like a "real" program. It can still be run as a stand alone script. Check the bottom of the first post to read how to download and run the script using the terminal.

Ok I screwed my compiling environment up  :o  and got it working again  :P

But just wanted to check if this is going te be fine or if I can encounter new problems.

This happened
Yesterday I found this interesting program called 'butterflow' it can interpolate intermediate frames.
I wanted to test it how it works on 12.5 FPS UHD file, and if I can get it to 25fps with this program.
I'm not that savvy with this home-brew python stuff...
I'm on a Mac, so I typed this line in terminal:
brew install brewsci/science/butterflow
A lot got installed on the way and in the end I could not get the programming to work.
Couldn't even get the help menu, this happens with 'butterflow -h' command:
Traceback (most recent call last):
  File "/usr/local/Cellar/butterflow/0.2.3_1/libexec/bin/butterflow", line 9, in <module>
    load_entry_point('butterflow==0.2.3', 'console_scripts', 'butterflow')()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 476, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2700, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2318, in load
    return self.resolve()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2324, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/Cellar/butterflow/0.2.3_1/libexec/lib/python2.7/site-packages/butterflow/cli.py", line 11, in <module>
    from butterflow import ocl, avinfo, motion, ocl
ImportError: dlopen(/usr/local/Cellar/butterflow/0.2.3_1/libexec/lib/python2.7/site-packages/butterflow/avinfo.so, 2): Library not loaded: /usr/local/opt/ffmpeg/lib/libavcodec.57.dylib
  Referenced from: /usr/local/Cellar/butterflow/0.2.3_1/libexec/lib/python2.7/site-packages/butterflow/avinfo.so
  Reason: image not found


Too bad that I couldn't get it to work, BUT when I wanted to compile a new crop_rec module file from magic lantern, I got a new error, and it didn't compile the module.
/bin/sh: python2: command not found

After a lot of trial and error and reinstalling gcc and python (which gave messages that it was already there) I found out that there wasn't a file named 'python2' in my usr/bin/ directory.
and changed this line in the make.user.default file from:
PYTHON=python2
changed to:
PYTHON=python

And now I can compile again, but the weird thing is, the butterflow installation couldn't mess with the make.user.file.
So apparently I was running on python 2 and the butterflow installation messed things up, maybe overwritten python files with newer or older versions ?
Is this fix I did a good solution, or is it a bit messy ?

Butterflow probably changed your python paths. What do you see when running this?
python -c "import sys; print('\n'.join(sys.path))"

Tried reinstalling python 2?
brew install python2
Might ask to relink or so if somethig was messed up.

Quote
What do you see when running this?

Code Select Expand

python -c "import sys; print('\n'.join(sys.path))"

I see this:

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
/Users/magic_lantern/Library/Python/2.7/lib/python/site-packages
/usr/local/lib/python2.7/site-packages
/usr/local/Cellar/numpy/1.14.5/libexec/nose/lib/python2.7/site-packages
/Library/Python/2.7/site-packages
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC


Haven't try to reinstall python 2, is it better or needed for compiling ?

It seems to be there, must have been paths.
Well, I would play around with butterflow and then reinstall python2. Probably more futureproof.

Couldn't get butterflow to work (gonna give it a try on a new user account, so when things get messy again, I delete the user account  :P)

So I reinstalled python 2, needed to do some forced relinking, changed the default.user.file back to python2 and ML compiling is working again with python2, thanks  :D

Yeah, I also had issues when installing dependencies for various applications. Good to hear you got python2 working again.

For those of us who like to live on the bleeding edge--Apple released a major OS update called Mojave (10.14). After running the update I had to re-install the xcode command line developer tools. There's also a Homebrew update for Mojave. Seems I did everything right but I'm getting this error when compiling ML:

[ XOR_CHK  ]   ../../build_tools/xor_chk
In file included from /usr/local/Cellar/gcc@5/5.5.0_2/lib/gcc/5/gcc/x86_64-apple-darwin17.3.0/5.5.0/include/stdint.h:9:0,
                 from ../../build_tools/xor_chk.c:2:
/usr/local/Cellar/gcc@5/5.5.0_2/lib/gcc/5/gcc/x86_64-apple-darwin17.3.0/5.5.0/include-fixed/stdint.h:27:32: fatal error: sys/_types/_int8_t.h: No such file or directory
compilation terminated.
make: *** [../../build_tools/xor_chk] Error 1

I also tried wiping out brew and starting over with Danne's Compiler.app (https://www.magiclantern.fm/forum/index.php?topic=21882.0) but I'm still getting this error.

There's also problems building the ML command line tools:

make mlv_dump
Using ~/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gcc (preferred).
[ HOST_CC  ]   mlv_dump.host.o
In file included from mlv_dump.c:22:0:
/usr/local/Cellar/gcc@5/5.5.0_2/lib/gcc/5/gcc/x86_64-apple-darwin17.3.0/5.5.0/include-fixed/math.h:45:23: fatal error: sys/cdefs.h: No such file or directory
compilation terminated.
make: *** [mlv_dump.host.o] Error 1

The MinGW compiler works fine so there seems to be an issue with Mojave and the Homebrew gcc compiler.

[EDIT] Workaround is to remove the Makefile.user section that points to the Homebrew gcc-5 compiler. This works for compiling ML but not for the command line tools, mlv_dump and cr2hdr.

Could you try uninstall command line tools and reinstall them?

Quote from: Danne on October 29, 2018, 04:17:33 PM
Could you try uninstall command line tools and reinstall them?

Been there, done that. Seems to be a problem with the Homebrew version of gcc-5 not being compatible with Mojave.

Just upgraded command line tools and now facing the same issues as you describe.

Ok, fix seems this:
https://stackoverflow.com/questions/52509602/cant-compile-c-program-on-a-mac-after-upgrade-to-mojave

Install macOS_SDK_headers_for_macOS_10.14.pkg by finding it here:
/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Conclusion: command line tools will not install the same way anymore. Will fix the script later.
Compiles platform build
Compiles both mlv_dump and cr2hdr

Ok--this command fixed it:

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Should rework the script to do proper reinstalls/updates.

Compiler.app updated with fix for Mojave:
https://bitbucket.org/Dannephoto/compiler/commits/5ccdf722a114040229c8680d2d97ce660ca831bd

[ CP       ]   newlib-libm.a
[ CP       ]   gcc-libgcc.a
[ CC       ]   disp_direct.o
[ CC       ]   font_direct.o
[ CC       ]   footer.o
[ XOR_CHK  ]   ../../build_tools/xor_chk
/bin/sh: cc-5: command not found
make[1]: *** [../../build_tools/xor_chk] Error 127
make[1]: *** Waiting for unfinished jobs....
make: *** [600D.102] Error 2


that is my problem compiling ML following this guide, on Osx ElCapitain 10.11.16

had to change the Makefile in build_tools/ to have correct gcc-5 instead of cc-5 (like in the error)

from this:
$(call build,XOR_CHK,$(HOST_CC) $< -o $@)
to this:
$(call build,XOR_CHK,gcc-5 $< -o $@)

Great catch @scrax and thanks for the much needed reminder... Gonna give this script of @Danne's another go while on Mojave 10.14.3!   :)

Did you make a Makefile.user file? The advantage of doing this is that it isn't tracked so you can customize your build environment without making any changes to the source code. You can put in several options in there and uncomment them as needed. Here's what's in mine:

Makefile.user.default
#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)

# CONFIG_QEMU = y
# LOG_INTERRUPTS = y
# CONSOLE_DEBUG = y
# CONFIG_DEBUGMSG = y
# CONFIG_DEBUG_INTERCEPT_STARTUP = y
# CONFIG_DEBUG_INTERCEPT = y
# CONFIG_GDB      = y
# CONFIG_GDBSTUB  = y
# CONFIG_MMIO_TRACE=y

# Recovery branch options:
# CONFIG_BOOT_FULLFAT=y
# CONFIG_BOOT_DUMPER=y
# CONFIG_BOOT_SROM_DUMPER=y

# VERSION = $(call eval_once,VERSION,raw_video_10bit_12bit_LVState.$(shell LC_TIME=EN date +'%Y%b%d').$(MODEL)$(FW_VERSION))


You can do the same with the modules. Here, I was testing a bunch of modules in one of the bleeding edge branches to see if they will compile.

Makefile.modules.user
ML_MODULES += \
mlv_rec \
pic_view \
ettr \
dot_tune \
autoexpo \
arkanoid \
deflick \
img_name \
bench \
selftest \
adv_int \
edmac \
ime_base \
ime_null \
ime_rot \
ime_std \
img_name \
io_crypt \
iso_regs \
mem_spy \
mpu_dump \
plot \
raw_diag \
sf_dump \
trace \
bulb_nd \
bolt_rec \
adtg_gui \
raw_twk \
edmac \
filepref \


Quote from: dfort on February 25, 2019, 10:18:11 PM
Did you make a Makefile.user file? The advantage of doing this is that it isn't tracked so you can customize your build environment without making any changes to the source code. You can put in several options in there and uncomment them as needed. Here's what's in mine:

Makefile.user.default

Code Select Expand

#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)

# CONFIG_QEMU = y
# LOG_INTERRUPTS = y
# CONSOLE_DEBUG = y
# CONFIG_DEBUGMSG = y
# CONFIG_DEBUG_INTERCEPT_STARTUP = y
# CONFIG_DEBUG_INTERCEPT = y
# CONFIG_GDB      = y
# CONFIG_GDBSTUB  = y
# CONFIG_MMIO_TRACE=y

# Recovery branch options:
# CONFIG_BOOT_FULLFAT=y
# CONFIG_BOOT_DUMPER=y
# CONFIG_BOOT_SROM_DUMPER=y

# VERSION = $(call eval_once,VERSION,raw_video_10bit_12bit_LVState.$(shell LC_TIME=EN date +'%Y%b%d').$(MODEL)$(FW_VERSION))


You can do the same with the modules. Here, I was testing a bunch of modules in one of the bleeding edge branches to see if they will compile.

Makefile.modules.user

Code Select Expand

ML_MODULES += \
mlv_rec \
pic_view \
ettr \
dot_tune \
autoexpo \
arkanoid \
deflick \
img_name \
bench \
selftest \
adv_int \
edmac \
ime_base \
ime_null \
ime_rot \
ime_std \
img_name \
io_crypt \
iso_regs \
mem_spy \
mpu_dump \
plot \
raw_diag \
sf_dump \
trace \
bulb_nd \
bolt_rec \
adtg_gui \
raw_twk \
edmac \
filepref \


I have a Makefile.user made by editing the default one and on the host compiler settings part it's same as yours

HOST_CC=gcc-5
HOST_LD=gcc-5

but when compiling $(HOST_CC) became "cc-5" instead of "gcc-5" so had to put manually in the build_tools/Makefile