Compiling Magic Lantern on a Macintosh

[dmilligan]

You could but that's not what it's for, and I don't see much point. It's simply to make sure your Lua scripts are free of compilation errors before running them on camera.

Lua doesn't compile to native code, it compiles to its own bytecode format which is still run by the Lua interpreter (same way Java works), so it basically would only save a small amount of startup time where the scripts are loaded. Execution performance is unaffected (scripts are compiled in camera to the exact same bytecode).

As far as homebrew installation goes, what you wrote may have issues because I think homebrew is still on Lua 5.2.x and we are using 5.3.x. So you'll need to specifically tell brew you want Lua 5.3.

[dfort]

Thanks for the explanation dmilligan.

I think homebrew is still on Lua 5.2.x and we are using 5.3.x. So you'll need to specifically tell brew you want Lua 5.3.

You're right--of course. Homebrew is using 5.2.4 as the default so I did this:

Code Select Expand

brew uninstall lua
brew install lua53

That installs 5.3.3 but then we're back to the same issue with the missing luac. However, using the --universal option did install luac:

Code Select Expand

brew install lua53 --universal

Maybe this because I'm on a 64-bit MacBook Pro and the install script only works on 32-bit systems? In any case, things seem to be working now.

[BBA]

Maybe off topic ?
For new developers like me, it is interesting to detect as many errors as possible before run time in camera.

[dfort]

Well if it means having to add lua to the install script it is very much on topic. Guess it is time to revisit all the tutorials that I posted.

[BBA]

I meant it can be useful to find errors before execution time.
Luac maybe is one way to do that but it is not the only one.
This may be better discussed in the "Lua scripting" topic.

Don't change all your tutorials just for that.
IMHO they are very well made for now.. Thanks for that.

I think the brew instructions you shared are well enough for people having such particular needs.

[DeafEyeJedi]

Ever since I saw @rbrune shared (Thanks for that) his branch in this thread which I went ahead and wanted to give you (@dfort) a helping hand in getting this compiling environment to work on my system in order for me to make the builds for testers to keep our projects moving forward without hesitation.

Here's what I am getting now after getting the branch installed with no problems (that I know of) until I tried building a 7D.203 (same xcrun error with EOSM, 5D3) which I haven't seen before. Have you?

Code Select Expand

Last login: Tue Nov  8 11:14:36 on ttys000
Seans-Mac-mini-385:~ DeafEyeJedi$ hg clone -r unified https://bitbucket.org/rbrune/magic-lantern/branch/crop_rec_raw_video_10bit_12bit
destination directory: crop_rec_raw_video_10bit_12bit
adding changesets
adding manifests
adding file changes                                                             
added 12169 changesets with 26665 changes to 2912 files                         
updating to branch unified
1729 files updated, 0 files merged, 0 files removed, 0 files unresolved
Seans-Mac-mini-385:~ DeafEyeJedi$ cd ~/magic-lantern/platform/7D.203/
Seans-Mac-mini-385:7D.203 DeafEyeJedi$ make clean && make zip
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
Seans-Mac-mini-385:7D.203 DeafEyeJedi$

*edit*

After I decided that this could very well be related to my Mac Mini so tried this same routine again on my MBP which had compiling working previously (w the exception of Dual-ISO being broken -- not sure why yet!) and this time I spat out 7D.203 build just fine with no Dual-ISO module included.



So at least it seems to be OK on this MBP but not on my Mac Mini -- will have to investigate further more before I can go ahead and build all platforms from @rbrune's crop_rec branch (w 10/12-bit options as well) so please stand by guys until I get this cleared up with the great @dfort!

*EDIT #2*

Sorry for the mess I've made but felt this was worth sharing as I ran both builds (7D and EOSM) which then weren't including neither the crop_rec.mo nor the 10/12-bit options under raw_rec.mo & now I am suspect that this could be related to using 'unified' while installing the branch so I decided to try this again by not including the terminology 'unified' and now I am being prompted with questions and not sure which one to choose from?

Code Select Expand

Last login: Tue Nov  8 12:24:37 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ hg clone -r https://bitbucket.org/rbrune/magic-lantern/branch/crop_rec_raw_video_10bit_12bit
hg clone: invalid arguments
hg clone [OPTION]... SOURCE [DEST]

make a copy of an existing repository

options ([+] can be repeated):

-U --noupdate          the clone will include an empty working directory
                        (only a repository)
-u --updaterev REV     revision, tag, or branch to check out
-r --rev REV [+]       include the specified changeset
-b --branch BRANCH [+] clone only the specified branch
    --pull              use pull protocol to copy metadata
    --uncompressed      use uncompressed transfer (fast over LAN)
-e --ssh CMD           specify ssh command to use
    --remotecmd CMD     specify hg command to run on the remote side
    --insecure          do not verify server certificate (ignoring web.cacerts
                        config)

(use "hg clone -h" to show more help)
Apples-Macintosh-10:~ DeafEyeJedi$ destination directory: crop_rec_raw_video_10bit_12bit


[dfort]

Ok, you almost got it. Let's do this the way I would--not that I'm an expert but this seems to work for me.

We're going to clone rbrune's magic-lantern repository--hope that's ok with you Rafael. The reason we're cloning instead of forking is because we want to see his updates in SourceTree as he pushes his commits. Also, we won't be making any pull requests. That's another topic.




Go to rburne's bitbucket page and copy the URL.

Code Select Expand

https://bitbucket.org/rbrune/magic-lantern




Paste the address into SourceTree.




Note that it will create a local repository with the same name as rbrune's repository name, magic-lantern. This could be confusing if you are cloning and forking various repositories so in order to keep things organized we'll append rburne to your local repository.




When you open up his repository in SourceTree you'll see his latest commits at the top but caution because you aren't in the right branch yet, you're in the default branch--check the panel on the left.




Double click either on the panel on the left or the graphic display of his workflow to get into the branch that you want. In this case you'll want his crop_rec_raw_video_10bit_12bit branch.




Ok, now you can navigate through the local repository with the terminal, saved in your home directory if you followed this tutorial.

I've got an idea what's going on with gcc-5 because of something that happened to me just this morning. Try this first:

Code Select Expand

which gcc-5

If you get nothing, check that it is installed:

Code Select Expand

brew search gcc

This is what I got:

cosmo0920/mingw_w64/i686-w64-mingw32-gcc ✔
cosmo0920/mingw_w64/x86-64-w64-mingw32-gcc
gcc ✔
homebrew/dupes/apple-gcc42 ✔
homebrew/versions/gcc43
homebrew/versions/gcc44
homebrew/versions/gcc45
homebrew/versions/gcc46
homebrew/versions/gcc47
homebrew/versions/gcc48
homebrew/versions/gcc49 ✔
homebrew/versions/gcc5 ✔
homebrew/versions/gcc6
homebrew/versions/llvm-gcc28
Caskroom/cask/gcc-arm-embedded

Note that the check marks show what's installed. If you have a check mark on gcc6 and gcc that means that you're linked to gcc6. That isn't working with ML at this time so link to gcc5:

Code Select Expand

brew link gcc5

Now try this again:

Code Select Expand

which gcc-5

You should get this:

/usr/local/bin/gcc-5

Got all that? Ok--let's see if you can compile all those platforms. They should all work except for 1100D.105 and don't bother with 7D_MASTER.203, not sure what that one is for.

[DeafEyeJedi]

Good Morning Dan... 

Ok, you almost got it. Let's do this the way I would--not that I'm an expert but this seems to work for me.

You'll always be the expert at least to me and few of us (you know who you are) and please don't ever change your sense of humor!

We're going to clone rbrune's magic-lantern repository--hope that's ok with you Rafael. The reason we're cloning instead of forking is because we want to see his updates in SourceTree as he pushes his commits. Also, we won't be making any pull requests. That's another topic.




Go to rburne's bitbucket page and copy the URL.

Code Select Expand

https://bitbucket.org/rbrune/magic-lantern




Paste the address into SourceTree.




Note that it will create a local repository with the same name as rbrune's repository name, magic-lantern. This could be confusing if you are cloning and forking various repositories so in order to keep things organized we'll append rburne to your local repository.

Done. Whoa that was easy. Now I wished I had gotten the hang of using this application sooner rather than later. 

When you open up his repository in SourceTree you'll see his latest commits at the top but caution because you aren't in the right branch yet, you're in the default branch--check the panel on the left.




Double click either on the panel on the left or the graphic display of his workflow to get into the branch that you want. In this case you'll want his crop_rec_raw_video_10bit_12bit branch.




Ok, now you can navigate through the local repository with the terminal, saved in your home directory if you followed this tutorial.

Nice. Now I see that that I am in it after double clicking on his workflow. Really cool actually!

I've got an idea what's going on with gcc-5 because of something that happened to me just this morning. Try this first:

Code Select Expand

which gcc-5

If you get nothing, check that it is installed:

Code Select Expand

brew search gcc

This is what I got:

Note that the check marks show what's installed. If you have a check mark on gcc6 and gcc that means that you're linked to gcc6. That isn't working with ML at this time so link to gcc5:

Code Select Expand

brew link gcc5

Now try this again:

Code Select Expand

which gcc-5

This is what I'm getting...

Code Select Expand

Last login: Tue Nov  8 12:25:43 on ttys001
Apples-Macintosh-10:~ DeafEyeJedi$ which gcc-5
Apples-Macintosh-10:~ DeafEyeJedi$ brew search gcc
gcc ✔                                    homebrew/dupes/apple-gcc42
homebrew/versions/gcc43                  homebrew/versions/gcc49
homebrew/versions/gcc44                  homebrew/versions/gcc5
homebrew/versions/gcc45                  homebrew/versions/gcc6
homebrew/versions/gcc46                  homebrew/versions/llvm-gcc28
homebrew/versions/gcc47                  Caskroom/cask/gcc-arm-embedded
homebrew/versions/gcc48
Apples-Macintosh-10:~ DeafEyeJedi$ brew link gcc5
Error: No such keg: /usr/local/Cellar/gcc5

I have a feeling that this may be due to the fact that I may have done a "brew update" few weeks ago when I was having compiling issues so could this be very well related? Should I re-install GCC for your OP in this thread?

Got all that? Ok--let's see if you can compile all those platforms. They should all work except for 1100D.105 and don't bother with 7D_MASTER.203, not sure what that one is for.

I did got all that. Thanks again for this BUT I just need to know whether I should delete the GCC file and reinstall this again?

I can feel it that I am so close to getting this to work perfectly... 

*edit*

After researching through your OP -- should I go ahead and do this installation again?

Code Select Expand

brew tap homebrew/dupes; brew install homebrew/versions/gcc5

*edit 2*

Well I went ahead and did that then after typing up which gcc I would be running...

Code Select Expand

Apples-Macintosh-10:~ DeafEyeJedi$ which gcc-5
Apples-Macintosh-10:~ DeafEyeJedi$ brew search gcc
gcc ✔                                    homebrew/versions/gcc47
homebrew/dupes/apple-gcc42               homebrew/versions/gcc48
homebrew/versions/gcc43                  homebrew/versions/gcc49
homebrew/versions/gcc44                  homebrew/versions/gcc5 ✔
homebrew/versions/gcc45                  homebrew/versions/gcc6
homebrew/versions/gcc46                  homebrew/versions/llvm-gcc28
Caskroom/cask/gcc-arm-embedded
Apples-Macintosh-10:~ DeafEyeJedi$

So that looks good to me now, right? I'm gonna go ahead and do the rest from your post above... Stand by Dan!

[dfort]

@DeafEyeJedi

Make sure gcc-5 is linked:

Code Select Expand

which gcc-5

You should get this for an answer:

/usr/local/bin/gcc-5

If you don't then:

Code Select Expand

brew link gcc5

And once again:

Code Select Expand

which gcc-5

Note when to use the dash and when not to use the dash.

[DeafEyeJedi]

Looks like I'll have to wait.... though it has been 20 minutes since.

Code Select Expand

Last login: Wed Nov  9 09:18:28 on ttys004
Apples-Macintosh-10:~ DeafEyeJedi$ which gcc-5
Apples-Macintosh-10:~ DeafEyeJedi$ brew link gcc5
Error: Operation already in progress for gcc5
Another active Homebrew process is already using gcc5.
Please wait for it to finish or terminate it to continue.
Apples-Macintosh-10:~ DeafEyeJedi$

btw just for good news (at least a little) managed to spit out EOSM, 7D builds with no problems (includes Rafael's fix w crop_rec.mo and options for 10/12-bit) on both builds with the exception of Dual-ISO being broken once again. What gives?



*edit*

Actually I am glad I didn't quit that brew install because it literally took 37 minutes to install according to this...

Code Select Expand

🍺  /usr/local/Cellar/gcc5/5.4.0: 1,391 files, 246.6M, built in 37 minutes 31 seconds
Apples-Macintosh-10:~ DeafEyeJedi$

and here's the better news of all...

Code Select Expand

Apples-Macintosh-10:~ DeafEyeJedi$ which gcc-5
/usr/local/bin/gcc-5
Apples-Macintosh-10:~ DeafEyeJedi$

Well by now I think I should try compiling again and see if Dual-ISO is still broken or not (shouldn't be because now I got gcc-5 linked, right?) well guess we'll find out below.

*edit 2*

Nope. Dual-ISO is still broken. Could this small portion of error be related to this?

Code Select Expand

cr2hdr.c:2548:59: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2574:21: error: function definition is not allowed here
                    {
                    ^


Ideally I beleive I shall wait until I get Dual-ISO module to work before going ahead and compile the crop_rec.mo + 10/12-bit merge from @rbrune's branch, right guys? 

[dfort]

I don't think that's the error.

I might have skipped an important step--did you make your Makefile.user file? If it isn't in there it will use the Apple clang compiler.

Makefile.user

Code Select Expand

#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)


[DeafEyeJedi]

I don't think that's the error.

I might have skipped an important step--did you make your Makefile.user file? If it isn't in there it will use the Apple clang compiler.

Makefile.user

Code Select Expand

#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)


Yes of course I followed your instructions and made the new Makefile.user within magic-lantern-rbune directory ... It's already becoming a good habit! 



*edit 2*

Sorry this is all becoming too addicting to give up on figuring out why Dual-ISO module is still broken (won't build) and here's a log portion directly to Dual-ISO module and maybe there are hints? Could the culprit be related to a certain cr2hdr that I am using?

Code Select Expand

Building module dual_iso...
Updated HGVERSION
[ README   ]   module_strings.h
Help page 1: Quick start
Help page 2: Tips and tricks
[ CC       ]   dual_iso.o
[ HGDIFF   ]   hgdiff.tmp
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ OBJCOPY  ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
0000132c dual_iso_calc_dr_improvement
00001384 dual_iso_set_enabled
000013a4 dual_iso_is_enabled
000013b4 dual_iso_is_active
000013d4 dual_iso_get_dr_improvement
00001554 dual_iso_get_recovery_iso
00001574 dual_iso_set_recovery_iso
[ DEPENDS  ]   dual_iso.dep
Will load on:
    5D3
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
[ gcc      ]   cr2hdr
cr2hdr.c:1351:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist rggb.m"));
                                             ^
cr2hdr.c:1351:46: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:1489:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist bddb.m"));
                                             ^
cr2hdr.c:1489:46: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:1765:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist iso-curve.m"));
                                                  ^
cr2hdr.c:1765:51: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2201:55: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist fullres-curve.m"));
                                                      ^
cr2hdr.c:2201:55: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2535:56: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edges.dng"));
                                                       ^
cr2hdr.c:2535:56: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2548:59: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edge-map.dng"));
                                                          ^
cr2hdr.c:2548:59: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2574:21: error: function definition is not allowed here
                    {
                    ^
cr2hdr.c:2588:31: warning: implicit declaration of function 'edge_interp' is
      invalid in C99 [-Wimplicit-function-declaration]
                    int pi0 = edge_interp(dir);
                              ^
cr2hdr.c:2841:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist mix-curve.m"));
                                                  ^
cr2hdr.c:2841:51: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:3325:55: warning: if statement has empty body [-Wempty-body]
            if(system("octave --persist soft-film.m"));
                                                      ^
cr2hdr.c:3325:55: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:3669:34: warning: if statement has empty body [-Wempty-body]
        if(system("octave wb.m"));
                                 ^
cr2hdr.c:3669:34: note: put the semicolon on a separate line to silence this
      warning
10 warnings and 1 error generated.
dcraw-bridge.c:101:3: warning: unused variable 'unique'
      [-Wunused-const-variable]
} unique[] = {
  ^
1 warning generated.
amaze_demosaic_RT.c:983:10: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                            ^
amaze_demosaic_RT.c:983:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:983:35: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                                                     ^
amaze_demosaic_RT.c:983:35: note: use function 'fabs' instead
  ...if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;...
                                  ^~~~~
                                  fabs
amaze_demosaic_RT.c:1215:10: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                            ^
amaze_demosaic_RT.c:1215:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:1215:35: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                                                     ^
amaze_demosaic_RT.c:1215:35: note: use function 'fabs' instead
  ...if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
                                  ^~~~~
                                  fabs
4 warnings generated.
make[4]: *** [cr2hdr] Error 1

********************************************************
WARNING: module dual_iso failed to build, deleting
********************************************************

[ RM ]  dual_iso.o dual_iso.mo dual_iso.sym dual_iso.dep module_strings.h hgdiff.tmp *.o *.d *.dep *.sym hgstamp
[ RM ]  cr2hdr cr2hdr.exe dcraw dcraw.c dcraw.exe exiftool.exe exiftool.tar.gz exiftool exiftool.zip cr2hdr.zip cr2hdr-win.zip cr2hdr-win_exiftool-perl-script.zip

In any case if anyone wants me to go ahead and compile certain builds to test out the crop_rec.mo + 10/12-bit together then please look at this thread.

Also if anyone insist to have this Dual-ISO module on their cams then are we allowed just to grab the Dual-ISO module from the nightlies and bundle it together or is that a no-no?

*edit* Finally got it working w Dual-ISO. Now 10/12 bit raw_rec.mo + crop_rec.mo builds can be downloaded from this post. Please remember to keep all bug reports related to this in that particular thread during testing. Thanks!

[dfort]

Seems like we tried everything. I have pretty much the same setup and things are working fine over here.

Try navigating into the dual_iso directory and compile cr2hdr. Do it one step at a time and watch for error messages.

In the terminal, change to the dual_iso directory.

Code Select Expand

cd ~/magic-lantern-rbrune/modules/dual_iso

Clean out any prior builds.

Code Select Expand

make clean

Now compile just cr2hdr

Code Select Expand

make cr2hdr

See how that works.

I'm not sure what was the rational for compiling cr2hdr every time you want to build ML for your camera but that's where the problem seems to be. Everything else uses the ARM cross-compiler but the command line tools that obviously need to be built with the host compiler. If you're feeling adventurous try this. Open the Makefile that is in the modules/dual_iso directory with a text editor:

Makefile

Code Select Expand

# this is a very simple makefile for modules
TOP_DIR=../..

# define the module name - make sure name is max 8 characters
MODULE_NAME=dual_iso
MODULE_OBJS=dual_iso.o

# include modules environment
include $(TOP_DIR)/modules/Makefile.modules

# include cr2hdr makefile rules
include Makefile.cr2hdr


And comment out that last line:

Code Select Expand

# include Makefile.cr2hdr

That will skip building cr2hdr and the dual_iso module will compile for your camera.

[DeafEyeJedi]

Good morning Dan once again... Such a wonderful feeling to wake up knowing that ML will be alive and well for us to dig through and be happy for the day! 

Try navigating into the dual_iso directory and compile cr2hdr. Do it one step at a time and watch for error messages.

In the terminal, change to the dual_iso directory.

Code Select Expand

cd ~/magic-lantern-rbrune/modules/dual_iso

Clean out any prior builds.

Code Select Expand

make clean

Now compile just cr2hdr

Code Select Expand

make cr2hdr

See how that works.

Done. Though there was nothing to clean up literally. Besides there was one error that didn't seem to be worrisome. Here are the logs:

Code Select Expand

Apples-Macintosh-10:~ DeafEyeJedi$ cd ~/magic-lantern-rbrune/modules/dual_iso
Apples-Macintosh-10:dual_iso DeafEyeJedi$ make clean
[ RM ]  dual_iso.o dual_iso.mo dual_iso.sym dual_iso.dep module_strings.h hgdiff.tmp *.o *.d *.dep *.sym hgstamp
[ RM ]  cr2hdr cr2hdr.exe dcraw dcraw.c dcraw.exe exiftool.exe exiftool.tar.gz exiftool exiftool.zip cr2hdr.zip cr2hdr-win.zip cr2hdr-win_exiftool-perl-script.zip
rm -rf lib
Apples-Macintosh-10:dual_iso DeafEyeJedi$ make cr2hdr
Updated HGVERSION
[ README   ]   module_strings.h
Help page 1: Quick start
Help page 2: Tips and tricks
[ gcc      ]   cr2hdr
cr2hdr.c:1351:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist rggb.m"));
                                             ^
cr2hdr.c:1351:46: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:1489:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist bddb.m"));
                                             ^
cr2hdr.c:1489:46: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:1765:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist iso-curve.m"));
                                                  ^
cr2hdr.c:1765:51: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2201:55: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist fullres-curve.m"));
                                                      ^
cr2hdr.c:2201:55: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2535:56: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edges.dng"));
                                                       ^
cr2hdr.c:2535:56: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2548:59: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edge-map.dng"));
                                                          ^
cr2hdr.c:2548:59: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:2574:21: error: function definition is not allowed here
                    {
                    ^
cr2hdr.c:2588:31: warning: implicit declaration of function 'edge_interp' is
      invalid in C99 [-Wimplicit-function-declaration]
                    int pi0 = edge_interp(dir);
                              ^
cr2hdr.c:2841:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist mix-curve.m"));
                                                  ^
cr2hdr.c:2841:51: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:3325:55: warning: if statement has empty body [-Wempty-body]
            if(system("octave --persist soft-film.m"));
                                                      ^
cr2hdr.c:3325:55: note: put the semicolon on a separate line to silence this
      warning
cr2hdr.c:3669:34: warning: if statement has empty body [-Wempty-body]
        if(system("octave wb.m"));
                                 ^
cr2hdr.c:3669:34: note: put the semicolon on a separate line to silence this
      warning
10 warnings and 1 error generated.
dcraw-bridge.c:101:3: warning: unused variable 'unique'
      [-Wunused-const-variable]
} unique[] = {
  ^
1 warning generated.
amaze_demosaic_RT.c:983:10: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                            ^
amaze_demosaic_RT.c:983:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:983:35: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fab...
                                                                     ^
amaze_demosaic_RT.c:983:35: note: use function 'fabs' instead
  ...if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;...
                                  ^~~~~
                                  fabs
amaze_demosaic_RT.c:1215:10: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                            ^
amaze_demosaic_RT.c:1215:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:1215:35: warning: absolute value function 'fabsf' given an
      argument of type 'double' but has parameter of type 'float' which may
      cause truncation of value [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fab...
                                                                     ^
amaze_demosaic_RT.c:1215:35: note: use function 'fabs' instead
  ...if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
                                  ^~~~~
                                  fabs
4 warnings generated.
make: *** [cr2hdr] Error 1
Apples-Macintosh-10:dual_iso DeafEyeJedi$


Not sure what that means? However since I was feeling a bit adventurous and went ahead with these suggestions below...

I'm not sure what was the rational for compiling cr2hdr every time you want to build ML for your camera but that's where the problem seems to be. Everything else uses the ARM cross-compiler but the command line tools that obviously need to be built with the host compiler. If you're feeling adventurous try this. Open the Makefile that is in the modules/dual_iso directory with a text editor:

Makefile

Code Select Expand

# this is a very simple makefile for modules
TOP_DIR=../..

# define the module name - make sure name is max 8 characters
MODULE_NAME=dual_iso
MODULE_OBJS=dual_iso.o

# include modules environment
include $(TOP_DIR)/modules/Makefile.modules

# include cr2hdr makefile rules
include Makefile.cr2hdr


And comment out that last line:

Code Select Expand

# include Makefile.cr2hdr

That will skip building cr2hdr and the dual_iso module will compile for your camera.

Indeed that did the trick. Now it seems I was able to get it compiled for 5D3.113 from @rbrune's branch with your latest fix included (raw_rec.mo + 10/12-bit options) and this time Dual-ISO spat out just fine. Everything looks normal within the logs as well as the build (see attachment below):

Code Select Expand

Building module dual_iso...
[ HGDIFF   ]   hgdiff.tmp
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ OBJCOPY  ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
0000132c dual_iso_calc_dr_improvement
00001384 dual_iso_set_enabled
000013a4 dual_iso_is_enabled
000013b4 dual_iso_is_active
000013d4 dual_iso_get_dr_improvement
00001554 dual_iso_get_recovery_iso
00001574 dual_iso_set_recovery_iso
[ DEPENDS  ]   dual_iso.dep
Will load on:
    5D3
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 600D, 60D, 650D, 6D, 700D, 7D, EOSM

Building module silent...
[ HGDIFF   ]   hgdiff.tmp
[ MODULE   ]   silent.mo
[ STRIP    ]   silent.mo
[ OBJCOPY  ]   silent.mo
[ EXPORTS  ]   silent.sym
[ DEPENDS  ]   silent.dep
Will load on:
    5D3
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 600D, 60D, 650D, 6D, 700D, 7D, EOSM




Although I notice there was this one error message that I thought I should double check and share with you just to be sure it's nothing cirtical:

Code Select Expand

[ XOR_CHK  ]   ../../build_tools/xor_chk
../../build_tools/xor_chk.c:48:88: warning: format specifies type
      'unsigned long' but the argument has type 'uint64_t' (aka
      'unsigned long long') [-Wformat]
  ...error (expected 0x%lX, got 0x%lX)\n", 0xCCCCCCCCE12FFF13, footer_magic);
                                  ~~~                          ^~~~~~~~~~~~
                                  %llX
1 warning generated.


...and this one from the lua.mo if by all means anything just in case:

Code Select Expand

Building module lua...
[ HGDIFF   ]   hgdiff.tmp
[ MODULE   ]   lua.mo
lua/lauxlib.o: In function `panic':
lauxlib.c:(.text+0x360): warning: warning: your code uses stdio (7+k bloat).
dietlibc.a(vfprintf.o): In function `vfprintf':
vfprintf.c:(.text+0x34): warning: warning: the printf functions add several kilobytes of bloat.


Whew that felt so good. Thanks again @dfort for your wonderful contributions and I very much enjoy my learning experience with you on this. Now I have to first downgrade from 123 to 113 for the 5D3 to be tested with this build (unless I should give it a try to find a 5D3.123 branch that can be merged with this crop_rec.rec + 10/12-bit options) or is that going to be a long shot for me to get it compiled?

I don't mind downgrading but when I go out on paid gig's on weekends occasionally (upgrade to 5D3.123 is a must) for me to be able to have clean HDMI out on set. I know what a pain but like I said I don't mind. Just would prefer to be able to compile if I'm allowed to with these latest gems?

Anyway enough with the questions now I'm gonna go ahead and compile the rest of DSLR's for you, Dan and perhaps I should use bitbucket instead of Dropbox.com or Mega.nz?

Actually I just made this download site of my own from Bitbucket (https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads) and if you don't mind me asking what would be the best way to keep these files organized in terms of which branch was it from? What's included? What's not included? I mean just so that way our testers will know which one is which and not to be confused with bug reports coming from these builds versus the unified nightlies, right?

Perhaps I should change the file name to include 10/12-bit + crop_rec.mo from @rbrune's branch, ya?

[dfort]

Finally!

Well, there's still a problem. We should figure out why cr2hdr isn't compiling on your system. Those other problems that you pointed out might be benign. How are the builds running on your cameras? You might want to try the unified branch just to see if it works the same as a downloaded nightly.

I like renaming the zip file from Nightly to something more descriptive. Otherwise you'll have a bunch of files labeled Nightly and you won't be able to keep them organized.

I'm thinking about putting in a pull request that skips building cr2hdr. Partially because of the problems it is causing for people that are first trying to compile ML and partially so that it works the same way as mlv_dump and raw2dng.

[dfort]

So it seems that one of the most common problems when setting up a development system is that cr2hdr is picky about which gcc you're using so dual_iso fails to compile. I'm not sure what the logic was behind having cr2hdr compile when all you want is ML for your camera so I put in a pull request that changes the rules so it works more like mlv_dump and raw2dng. a1ex already approved it but hasn't merged it yet.

https://bitbucket.org/hudson/magic-lantern/pull-requests/770/removed-all-rule-so-cr2hdr-isnt-built/diff

The change is really simple, just delete the "all::" rule at the end of Makefile.cr2hdr.

By the way, did you know that the build script can also download dcraw and exiftool? Change into the modules/dual_iso directory enter "make cr2hdr.zip" in the terminal and it will download and build dcraw, download exiftool, build cr2hdr and package everything in a .zip file. Pretty cool, huh?

[Danne]

Been having a real blast with my workstation just never got compiling to work but it seems finally I solved a long ongoing problem.

Had python messages showing why builds were failing and the end part showed:

Code Select Expand

ValueError: unknown locale: UTF-8

The fix was easy
https://fcojperez.wordpress.com/2014/08/18/valueerror-unknown-locale-utf-8-cuando-instalamos-awscli/
In terminal write

Code Select Expand

export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8

You can add those export lines straight into your bash profile.
https://coderwall.com/p/-k_93g/mac-os-x-valueerror-unknown-locale-utf-8-in-python

One other problem is raw_rec failing to build because of the makefile. Simply erase:

Code Select Expand

all:: raw2dng inside Makefile in raw_rec folder.

[dfort]

One other problem is raw_rec failing to build because of the makefile. Simply erase:

Code Select Expand

all:: raw2dng
inside Makefile in raw_rec folder.

I submitted a pull request for that along with a change that allows cross compiling a Windows binary on a Mac as described on the first post of this topic.

https://bitbucket.org/hudson/magic-lantern/pull-requests/773/raw2dng-make-rules-changes/diff

[DeafEyeJedi]

Nice work @Danne! 

@dfort -- is this something that we are required to do on our Mac's even if we aren't getting the raw_rec failing?

[dfort]

...is this something that we are required to do on our Mac's even if we aren't getting the raw_rec failing?

No, it isn't required but it isn't just about raw_rec failing. If you can't compile the command line tools, mlv_dump, raw2dng and cr2hdr, that means your system isn't setup properly and there's probably an issue with your host platform's gcc compiler.

What this is about is that generally you are either building ML for your camera or are building the command line tools but you usually don't need to build them both at the same time. Many of us didn't even know that raw2dng was being built every time we compiled a build for our cameras. The build rules are still there for the command line tools when you need them.

[serosi1]

Excuse me!
I am using Canon 1DX Mark ii, I hope I understood correctly (!)
possibile shooting in raw my canon 1dx mark ii? 

[Walter Schulz]

ML dev team decided some time ago not to touch Canon's pro gear. This includes 1D and C series.
And it is not a matter of discussion.

[dfort]

@Danne found an issue with cosmo0920/mingw_w64/i686-w64-mingw32-gcc -- it currently won't install.

I made the developer aware of this, hopefully it will be fixed soon.

Code Select Expand

Error: i686-w64-mingw32-gcc contains conflicting version recursive dependencies:
  isl, [email protected]

[dfort]

Got a reply.

Thanks for your report!
But recently homebrew-core got merged mingw-64 Formula which installs cross gcc and binutils at once.

Could you tell to use mingw-w64 instead of my formula?

So now it should be much easier. Simply:

Code Select Expand

brew install mingw-w64

and you have a full 32-bit and 64-bit Windows cross compiling setup ready to go. I'll update the tutorial so it will work with this latest mingw-64 formula.

[Danne]

That was too easy. Works great. Thanks dfort.