Compiling and setting up ML command line tools for Macs [cr2hdr and raw2dng]

[Francis]

It seems like lots of people are using wine to run the command line tools provided with Magic Lantern. Here is how to compile and create symbolic links to the binary to make the workflow a little bit simpler .

I'm assuming you are fairly comfortable using a command line interface and navigating the filesystem with commands like cd. If you aren't then this is not the resource for you, you have plenty of other reading to do before you will be ready for this. Edit: for emphasis.

You're going to need a few tools for this: MacPorts, hg, dcraw, and a pre-compiled arm toolchain.

First step is download and install MacPorts, a command line-based package system similar to APT for linux. After installation is complete you are going to want to install a few packages.

In terminal type 'sudo port install mercurial' to download install hg', which we will use to download the source. You'll be prompted for your administrator acct. password. Follow any warnings or suggestions on updating the package list.

Next we're going to download dcraw, which is needed by cr2hdr and raw2dng for conversion purposes. Again in terminal type 'sudo port install dcraw'.

We're done with MacPorts for now and can use hg to download the source tree. Navigate in Terminal to whereever you wish to download the ML source files and type 'hg clone -r unified https://bitbucket.org/hudson/magic-lantern';. This will create a folder ./magic-lantern/ which contains all the source files.

Now that we have the source we need to get a pre-compiled toolchain to compile the code. Save and extract.

Now we have to tell the ML build files where they can fine the compiler. open ./magic-lantern/makefile.user.default in a text editor and and change the following line to indicate where you extracted the toolchain.

Change

Code Select Expand

ARM_PATH=~/gcc-arm-none-eabi-4_7-2012q4

to

Code Select Expand

ARM_PATH=~/path/to/compiler

For me its

Code Select Expand

ARM_PATH=~/Downloads/Magic_Lantern/gcc

Be sure to not end your path with '/'

Save this new file as makefile.user (dropping the .default extension). Now we are ready to build.

First, you must compile the autoexec.bin and associated .sym file for your camera.

Now, in terminal, go to your source ./magic-lantern/modules/raw_rec

Type make. You will get some warnings about not having compiled the source for any model but these instructions are purely for getting the utilities to compile. You'll end up with our first utility, raw2dng, amongst other things like the module files.

Now we are going to make a link to raw2dng so that it can be run as a command in terminal no matter what directory you are in.

Type 'sudo ln -s raw2dng /usr/bin/raw2dng' This creates a symbolic link to the original file.

Now we are going to do the same for cr2hdr. Enter ./magic-lantern/modules/dual_iso

Type 'make' then 'sudo ln -s cr2hdr /usr/bin/cr2hdr'.

Voila now these 2 utilities are compiled and ready to be run from any directory through the terminal. For example with dual ISO images, enter the folder with the images and type 'cr2hdr IMG_0000.cr2'

To be totally honest there are probably some bumps you are going to run into and files you may have to edit to get everything compiling. I'll do my best to help but I have very limited programming experience and mostly have gotten through this with the help of Coutts 'developing on Mac' wiki article and a lot of trial and error.

[Danne]

Supercool. I, ll get right on it. Thank you very much

[REDcineX]

I am getting an error because it cannot find the file.

"/bin/sh: /Users/Admin/Downloads/magic-lantern/gcc/bin/arm-none-eabi-gcc-4.7.3: No such file or directory"

[REDcineX]

I looked up the file in the directory and found that on mac it is a .alias instead of an executable that surround it.




EDIT: And it doesn't link to an original file..

[REDcineX]

After downloading a few versions of the toolchain, I found an .exe of the 4.7.3 file. Now Terminal tells me that it cannot find the -ld file. I found the file and it is, as previously stated, a .alias file as well. It points no where.


"Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ make
[ MODULE   ]   dual_iso.mo
/bin/sh: /Users/Admin/gcc-arm-none-eabi-4_7-2012q4/bin/arm-none-eabi-ld: No such file or directory
make: *** [dual_iso.mo] Error 127"

Proof that file is in directory:


Makefile.user:

[Francis]

You downloaded the windows version of the toolchain. That is why they are all .exes

[REDcineX]

I have tried the windows, linux, and mac version. All with the same results. It just happened to be the windows one I took a screenshot of.

[Francis]

That screenshot doesn't show the full path but I'm assuming you removed the Windows files and put these in the same place. First in your makefile.user you have gcc version =4.7.4 instead of 4.7.3 but that probably isn't your only problem. Did you download the toolchain from the page I linked to? If so why use an older version? The current download links on that main page are 2013q2. That probably isn't a problem either.

I'm not sure what to tell you but if your first impulse when it didn't work was to try the Windows version instead, you're probably not ready for this. Just download this binary and then make the symbolic link again. You probably should delete the one you've possibly already made.

Edit: Actually the 4.7.4 line probably is most of your problem. Change that to 4.7.3 and download the current toolchain. In my version they are all unix executables, none of them are links.

[REDcineX]

My first urge wasn't to use the windows version. I have parallels installed so I was going to see if I could use anything on windows. I rendered it pointless as others have made the mac version work.

I have tried both the new version of the toolchain and old versions because depending on which one I download I get more .alias (link) files. That is exactly how it looks after extracting the mac .tar.bz2

EDIT: I downloaded the current toolchain and changed it back to 4.7.3. My problem is still "/bin/sh: /Users/Admin/Downloads/gcc-arm-none-eabi-4_7-2013q2/bin/arm-none-eabi-ld: No such file or directory"

Because it appears as a link/alias and goes nowhere. I don't understand how it is doing that..

[REDcineX]

Fixed the problem of the .alias files. It was my unarchiver.

Now it still won't work, but at least that problem above is out of the way.

Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ make
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
[ DEPENDS  ]   dual_iso.dep
Not checked (compile ML for these cameras first):
    100D, 1100D, 40D, 500D, 50D, 550D, 5D2, 5D3, 5DC, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
make: *** [dual_iso.dep] Error 1
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ sudo ln -s cr2hdr /usr/bin/cr2hdr
Password:
ln: /usr/bin/cr2hdr: File exists


Is my problem that the file is already there so it won't overwrite it? Where can I actually find that file and erase it?

[Francis]

Delete it with the rm command. You don't have to use sudo (which switches to the admin/root account to perform a command) if you are already logged in as the admin.

To delete type
rm /usr/bin/cr2hdr

Then link again. If it says you don't have permission then add the sudo to the beginning of it.

[zappa1976]

Francis,

sorry if I'am asking again your help but i have some errors.

I link a screenshoot with errors I have.



I attached also 2 other screen shoot to show my settings.





Please can you check if are they ok?

Many thanks in advance!
Saverio

[REDcineX]

Hehe. One step at a time. Successfully deleted the previous file. Unfortunately, it still won't recognize "cr2hdr" as a command.

Macbook-Pro-i7-26gHz-2012:~ Admin$ rm /usr/bin/cr2hdr
rm: /usr/bin/cr2hdr: Permission denied
Macbook-Pro-i7-26gHz-2012:~ Admin$ sudo rm /usr/bin/cr2hdr
Password:
Macbook-Pro-i7-26gHz-2012:~ Admin$ cd magic-lantern/modules/dual_iso
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ make
[ DEPENDS  ]   dual_iso.dep
Not checked (compile ML for these cameras first):
    100D, 1100D, 40D, 500D, 50D, 550D, 5D2, 5D3, 5DC, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
make: *** [dual_iso.dep] Error 1
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ sudo ln -s cr2hdr /usr/bin/cr2hdr
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ sudo ln -s cr2hdr /usr/bin/cr2hdr
ln: /usr/bin/cr2hdr: File exists
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ cd ..
Macbook-Pro-i7-26gHz-2012:modules Admin$ cd ..
Macbook-Pro-i7-26gHz-2012:magic-lantern Admin$ cd ..
Macbook-Pro-i7-26gHz-2012:~ Admin$ cd desktop
Macbook-Pro-i7-26gHz-2012:desktop Admin$ cd untitledfolder
Macbook-Pro-i7-26gHz-2012:untitledfolder Admin$ cr2hdr _MG_9518.CR2
-bash: cr2hdr: command not found
Macbook-Pro-i7-26gHz-2012:untitledfolder Admin$

[REDcineX]

Francis,

sorry if I'am asking again your help but i have some errors.

I link a screenshoot with errors I have.



I attached also 2 other screen shoot to show my settings.





Please can you check if are they ok?

Many thanks in advance!
Saverio

Can you post a screenshot of what is in "bin"?

[REDcineX]

I believe you need to remove the "/Users/Saverio" in the ARM path on your makefile.user.

[zappa1976]

I believe you need to remove the "/Users/Saverio" in the ARM path on your makefile.user.

I tried but I have the same error..

[Francis]

I tried but I have the same error..

You don't need to change anything in your makefile.user. You've got it built. Now the only problem is making the link.

Try typing out the whole path when you are making the link.

sudo rm /usr/bin/cr2hdr <--to get rid of the old link
sudo ln -s /Users/Saverio/magic-lantern/modules/dual_iso/cr2hdr /usr/bin/cr2hdr

It's working for me with those exact same steps.

[zappa1976]

I believe you need to remove the "/Users/Saverio" in the ARM path on your makefile.user.

Something now it is working (I download the second toolchain "gcc-arm-none-eabi-4_7-2013q1"  with 4.7.3 instead of "gcc-arm-none-eabi-4_7-2013q2" with 4.7.4...is it right??)

I attached a screenshoot with the Error 1 I still have..



I don't know how to solve the error..

[REDcineX]

@Francis Do you know why I am still getting a "command not found" error?

Look at reply #12 for the terminal

[Francis]

Here is what I did, and have redone multiple times:

First start with this

Code Select Expand

Francis-Danforths-MacBook-Pro:~ danforthf$ sudo rm /usr/bin/cr2hdr
Password:
Francis-Danforths-MacBook-Pro:~ danforthf$ cd magic-lantern/modules/dual_iso/
Francis-Danforths-MacBook-Pro:dual_iso danforthf$ make clean
[ RM ]  dual_iso.o dual_iso.mo *.o *.d
[ RM ]  cr2hdr
Francis-Danforths-MacBook-Pro:dual_iso danforthf$


This basically resets you to scratch.


Then -->


I don't know what else to tell you guys. The hard part is done. Both of you have compiled the source. Maybe google 'making symbolic links in OSX' or '/usr/bin Mac' or something like that.

[Danne]

Here,s another one
I did all the steps an cleaned up also. When I hit make this is what I get?

[REDcineX]

I get the same thing as Danne below. And I would imagine the problem is the in the toolchain as I mentioned before, the file says 4.7.4 (not 4.7.3). So we are calling a file that is not there. Of course, it can't find it. If I change it to 4.7.4 in the makefile.user, I get past that error, but I still do not get cr or cr2hdr as a command at the end.

Macbook-Pro-i7-26gHz-2012:~ Admin$ sudo rm /usr/bin/cr2hdr
Password:
Macbook-Pro-i7-26gHz-2012:~ Admin$ cd magic-lantern/modules/dual_iso/
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ make clean
[ RM ]  dual_iso.o dual_iso.mo *.o *.d
[ RM ]  cr2hdr
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ make
[ GCC      ]   cr2hdr
[ GCC      ]   cr2hdr
[ GCC      ]   cr2hdr
~/Downloads/gcc-arm-none-eabi-4_7-2013q2/bin/arm-none-eabi-gcc-4.7.3  -Os -mthumb-interwork -march=armv5te -D__ARM__ -Wp,-MMD,./dual_iso.d -Wp,-MT,dual_iso.o -nostdlib -fomit-frame-pointer -fno-strict-aliasing -I../../include  -Wall -W -Wno-unused-parameter -Wno-unused-function -Wno-implicit-function-declaration -Wno-missing-field-initializers -Wno-format -Wdouble-promotion -ffast-math -fno-trapping-math -fsingle-precision-constant -fshort-double -std=gnu99 -Winline -I../../platform/all -I../../platform/all/include -I../../src -I../../picoc   -DMODULE -DMODULE_NAME=dual_iso    -c -o dual_iso.o dual_iso.c
/bin/sh: /Users/Admin/Downloads/gcc-arm-none-eabi-4_7-2013q2/bin/arm-none-eabi-gcc-4.7.3: No such file or directory
make: *** [dual_iso.o] Error 127


Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ make
~/Downloads/gcc-arm-none-eabi-4_7-2013q2/bin/arm-none-eabi-gcc-4.7.4  -Os -mthumb-interwork -march=armv5te -D__ARM__ -Wp,-MMD,./dual_iso.d -Wp,-MT,dual_iso.o -nostdlib -fomit-frame-pointer -fno-strict-aliasing -I../../include  -Wall -W -Wno-unused-parameter -Wno-unused-function -Wno-implicit-function-declaration -Wno-missing-field-initializers -Wno-format -Wdouble-promotion -ffast-math -fno-trapping-math -fsingle-precision-constant -fshort-double -std=gnu99 -Winline -I../../platform/all -I../../platform/all/include -I../../src -I../../picoc   -DMODULE -DMODULE_NAME=dual_iso    -c -o dual_iso.o dual_iso.c
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
[ DEPENDS  ]   dual_iso.dep
Not checked (compile ML for these cameras first):
    100D, 1100D, 40D, 500D, 50D, 550D, 5D2, 5D3, 5DC, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
make: *** [dual_iso.dep] Error 1
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ ls
Makefile   cr2hdr      cr2hdr.o   dual_iso.d   dual_iso.o   optmed.h
chdk-dng.o   cr2hdr.c   dual_iso.c   dual_iso.mo   dual_iso.sym   qsort.h
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ sudo ln -s /Users/Admin/magic-lantern/module/dual_iso/cr2hdr /usr/bin/cr2hdr
Password:
Macbook-Pro-i7-26gHz-2012:dual_iso Admin$ cd ~/Desktop
Macbook-Pro-i7-26gHz-2012:Desktop Admin$ cd untitledfolder
Macbook-Pro-i7-26gHz-2012:untitledfolder Admin$ ls
M12-1656.RAW   _MG_9518.CR2   _MG_9521.CR2   _MG_9523.CR2   _MG_9524.CR2   _MG_9526.CR2   _MG_9527.CR2
Macbook-Pro-i7-26gHz-2012:untitledfolder Admin$ cr
-bash: cr: command not found
Macbook-Pro-i7-26gHz-2012:untitledfolder Admin$ cr2hdr _MG_9524.CR2
-bash: cr2hdr: command not found

[Danne]

Nice to see I,m not alone

[Francis]

You can check to see if the link is made by doing the following

Code Select Expand

Francis-Danforths-MacBook-Pro:dual_iso danforthf$ cd /usr/bin
Francis-Danforths-MacBook-Pro:bin danforthf$ ls -al
total 291648
drwxr-xr-x  1209 root   wheel       41106 Aug 14 12:05 .
drwxr-xr-x@   14 root   wheel         476 Feb  8  2012 ..
..............
-rwxr-xr-x     1 root   wheel      377872 Oct 24  2010 cpp-4.0
-rwxr-xr-x     1 root   wheel      166128 Oct 24  2010 cpp-4.2
-rwxr-xr-x     1 root   wheel        1540 Aug  1  2009 cpuwalk.d
lrwxr-xr-x     1 root   wheel          54 Aug 14 12:05 cr2hdr -> /Users/danforthf/magic-lantern/modules/dual_iso/cr2hdr
-rwxr-xr-x     1 root   wheel         807 May 19  2009 crc32
-r-xr-xr-x     1 root   wheel         816 May 19  2009 crc325.10.0
-r-xr-xr-x     1 root   wheel         815 May 19  2009 crc325.8.9
................
Francis-Danforths-MacBook-Pro:bin danforthf$


Obviosly you have to scroll a good ways up to see the line.

Next check the file /etc/paths by opening in a text editor to see if '/usr/bin' is in there. If it isn't add it as the top line. Be sure to save as plain text.

[Danne]

O, ll check into this when I have a little time.
Big thanks Francis for helping out.