Compiling Magic Lantern on a Macintosh

[names_are_hard]

I think I confused things by failing to read Tim's output properly [EDIT: Tim didn't post the actual error first time ].  The uint64_t thing is the last line, but only a warning as Danne says.  The few lines before that are the actual error:

[ XOR_CHK  ]   ../../build_tools/xor_chk
cp ../../data/fonts/*.rbf /Users/xxxx/ML repos/magic-lantern_danne_9Dec2019/platform/5D3.113/zip/ML/fonts/
cp: /Users/xxxx/ML is a directory (not copied).
make: *** [install_extra_data] Error 1
make: *** Waiting for unfinished jobs....

I think the problem could be the space in "ML repos"?  I would rename that dir so it doesn't have a space in it.  Spaces in filenames are evil

[timbytheriver]

I would rename that dir so it doesn't have a space in it.

Great catch @names_are_hard ! That fixed it. Thanks plenty.

[Levas]

ok, screwed up my system again.

When I do the command make, within a module directory, I get this (and lots of more errors which do look like this one):

Code: [Select]

make
Using ~/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gcc (preferred).
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):

Short story how I screwed up, which may help to a solution 
I wanted to make use of Pyflow (https://www.magiclantern.fm/forum/index.php?topic=20999.0, and I got a error messages in there.
So I messed around with installing opencv2 and python2.7, although I didn't succeed in fixing the problem  .
To make sure compiling Magic Lantern still works, I tried the make command in a module and found out that I messed up my development environment...

[Levas]

Got Magic Lantern working again 

Here I found the answer:
https://stackoverflow.com/questions/59269208/errorrootcode-for-hash-md5-was-not-found-when-using-any-hg-mercurial-command

[DeafEyeJedi]

Hi everyone. Been awhile since I've been in this situation. Sold my 2012 MBP and just scooped up a decent 2015 MBP.

Beginning with this vanilla Catalina build I decided to run @Danne's Compiler.app script just to be sure I have all the bells and whistles working.

Then I tried to follow @dfort's suggestion on how to get this MLV_App_Compiler to work on 10.15.5 to no avail. Still getting the same 20 Fatal errors messages.

Suspect it has something to do with Clang and instead I should be using Apple's compiler so I followed the instructions in reply #208 to no avail.

What gives? 

[names_are_hard]

Please paste exact error messages.

[DeafEyeJedi]

Thanks @names_are_hard for your prompt response. Here's what I get in the beginning upon opening MLV_App_Compiler:

https://ibb.co/DpBhBxC

Here are the infamous 20 fatal errors (I also suspect this may very well have to do with GitHub's message from above's screenshot):

https://drive.google.com/file/d/12DMOC8-bYUGEwcNLbabRg-7VCO_Fu_pF/view?usp=sharing

I'll continue to try and work on @Levas' trick on fixing the openssl although I doubt that I have this installed. Thanks again!

[Danne]

Access denied on 2nd link. Then again you are screening Mlv_App.compiler and not the magic lantern compiler which this thread is about .

[DeafEyeJedi]

Fixed. Had restrictions enabled. I thought about posting this on the MLV_App Thread but there's already so much going on that and didn't want to derail them.

I'm also trying to get my compiling environment up and running on this vanilla MBP as well so figured I could get by in this thread. Sorry @Danne! 

[Danne]

Try:

Code: [Select]

chsh -s /bin/bashrestart terminal

Then:

Code: [Select]

brew install libressl,
brew install openssl
Then maybe these too:

Code: [Select]

xcode-select --install
Depending on environment:

Code: [Select]

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.*.pkg

[DeafEyeJedi]

Thanks for the follow up @Danne. I've went ahead and follow the instructions and this is what I've got so far.

Code: [Select]

Last login: Thu Jun  4 12:45:37 on ttys000

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
Macs-MacBook-Pro:~ mac$ brew install libressl,
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> New Formulae
vlang
==> Updated Formulae
adios2                     faudio                     logstash
asciidoc                   gauge                      nexus
aws-cdk                    gnutls                     octant
axel                       grafana                    odin
docker-compose             jenkins                    sslyze
docker-compose-completion  klavaro
fastlane                   libwebsockets

Error: No available formula with the name "libressl,"
==> Searching for a previously deleted formula (in the last month)...
Error: No previously deleted formula found.
==> Searching for similarly named formulae...
This similarly named formula was found:
libressl
To install it, run:
  brew install libressl
==> Searching taps...
==> Searching taps on GitHub...
Error: No formulae found in taps.
Macs-MacBook-Pro:~ mac$ brew install openssl
Warning: openssl@1.1 1.1.1g is already installed and up-to-date
To reinstall 1.1.1g, run `brew reinstall openssl@1.1`
Macs-MacBook-Pro:~ mac$ xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" to install updates
Macs-MacBook-Pro:~ mac$ open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.*.pkg
The file /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.*.pkg does not exist.
Macs-MacBook-Pro:~ mac$
After this I decided rerun the updates for the command line just to be sure I didn't miss anything.

Then I ran MLV_App_Compiler which again shows the same infamous 20 fatal errors (Take 2):

https://drive.google.com/file/d/12DMOC8-bYUGEwcNLbabRg-7VCO_Fu_pF/view?usp=sharing

I'll wait for @names_are_hard to see if he can give any hints. Thanks again!

Btw what's our plan going forward once Bitbucket shuts down on us. Do we plan on going to Github full time?

[names_are_hard]

This is almost certainly "Apple decided they wanted to break all your scripts by forcing you onto zsh and libressl", but beyond that I can only guess, because I don't have a Mac (partly because they pull shit like this all the time!).

Things to try:
<reboot laptop>
brew install curl-openssl
brew upgrade git
<reboot laptop>
<try MLV_App_Compiler again>

[DeafEyeJedi]

Most definitely to do w Apple decided to make things rather less relevant for us or those who enjoy having environmental builds on their systems.

Thanks for your hints. I did the brew install curl-openssl which gave me this:

Code: [Select]

Last login: Thu Jun  4 14:56:39 on ttys001

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
Macs-MacBook-Pro:~ mac$ brew install curl-openssl
Warning: curl-openssl 7.70.0 is already installed and up-to-date
To reinstall 7.70.0, run `brew reinstall curl-openssl`
Macs-MacBook-Pro:~ mac$ brew reinstall curl-openssl
==> Downloading https://homebrew.bintray.com/bottles/curl-openssl-7.70.0.catalin
Already downloaded: /Users/mac/Library/Caches/Homebrew/downloads/a66b27f37e4acebbe48ead1f40397229240a9adc414140de664caba036e996ce--curl-openssl-7.70.0.catalina.bottle.tar.gz
==> Reinstalling curl-openssl
==> Pouring curl-openssl-7.70.0.catalina.bottle.tar.gz
==> Caveats
curl-openssl is keg-only, which means it was not symlinked into /usr/local,
because macOS provides curl.

If you need to have curl-openssl first in your PATH run:
  echo 'export PATH="/usr/local/opt/curl-openssl/bin:$PATH"' >> ~/.profile

For compilers to find curl-openssl you may need to set:
  export LDFLAGS="-L/usr/local/opt/curl-openssl/lib"
  export CPPFLAGS="-I/usr/local/opt/curl-openssl/include"

==> Summary
🍺  /usr/local/Cellar/curl-openssl/7.70.0: 457 files, 3.4MB
Macs-MacBook-Pro:~ mac$

Then I ran brew upgrade git which gave me this:

Code: [Select]

🍺  /usr/local/Cellar/curl-openssl/7.70.0: 457 files, 3.4MB
Macs-MacBook-Pro:~ mac$ brew upgrade git
Warning: git 2.27.0 already installed
Macs-MacBook-Pro:~ mac$

Rebooted MBP and reopen MLV_App_Compiler again which then gave me this message before running it:

Code: [Select]

Last login: Thu Jun  4 14:57:25 on ttys001

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
/Users/mac/Desktop/Mlv_App_compiler.app/Contents/main.command ; exit;
Macs-MacBook-Pro:~ mac$ /Users/mac/Desktop/Mlv_App_compiler.app/Contents/main.command ; exit;
updating MLV-App-master source
warning: Pulling without specifying how to reconcile divergent branches is
discouraged. You can squelch this message by running one of the following
commands sometime before your next pull:

  git config pull.rebase false  # merge (the default strategy)
  git config pull.rebase true   # rebase
  git config pull.ff only       # fast-forward only

You can replace "git config" with "git config --global" to set a default
preference for all repositories. You can also pass --rebase, --no-rebase,
or --ff-only on the command line to override the configured default per
invocation.

Already up to date.
Recent commits:
23a46a0 (HEAD -> master, origin/master, origin/HEAD) Update README.md
af9caa8 Too much invisible...
87370c9 Added Transfer Function LineEdit for testing
a04936c remove all puts and prints
6277c35 Transfer function as a string. Mostly working. Only BMDFilm needs figuring out.
f56f841 Make tinyexpr compile with MLVApp
65a6d70 Added macro's for custom wb
c10890f Added tinyexpr
811d810 Added error message into copy action
4a08a9c Export/Copy the current receipt if nothing selected


----------------
MLV App compiler(thanks to Ilia3101,masc,bouncyball)
----------------

current branch: master

(c)   compile MLV App
(op)  compile MLV App with openmp(macOS 10.10 and onwards)
(b)   check out branches
(U)   update dependencies(if compiling fails)
(m)   make clean

(q)   quit this compiler

Please enter your selection number below and press enter:
Ran MLV_App_Compiler gives me those errors as it seems to be a problem with not having SDK Headers for MacOS 10.* per @Danne's suggestion from earlier :

Code: [Select]

Macs-MacBook-Pro:~ mac$ open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.*.pkg
The file /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.*.pkg does not exist.
Macs-MacBook-Pro:~ mac$
Take 3 20 Fatal Error Messages:
https://drive.google.com/file/d/1KJk1IeM-BNFlUkL7ExKOO1idO2qUMxgP/view?usp=sharing

If all else fails and I would rather downgrade this MBP to MoJave than putting you guys on this situation although some of you have environmental builds running on Catalina w no issues.

I guess I should also look into disabling the SIP trick as we used to do this with older Mac OS X and not sure if this still applies. 

[names_are_hard]

Those messages are much more encouraging, because before you weren't able to contact git.  Now it's succeeding at that step and failing during the compile.  This is real progress.

The error looks something to do with either clang or OSX headers so I can't help any further.  Possibly Danne, or somebody with OSX experience, can help.

[DeafEyeJedi]

I noticed that. It finally connected to git. That's indeed some progress. Still can't figure out how I'm still using Clang as oppose to Apple's version or whether it has to do w OS X headers.

Nonetheless, I went ahead and tried to run 'U' to ensure I had all the bells and whistles up to date but I keep getting those messages re: Fatal w git repo on bottom:

Code: [Select]

----------------
MLV App compiler(thanks to Ilia3101,masc,bouncyball)
----------------

current branch: master

(c)   compile MLV App
(op)  compile MLV App with openmp(macOS 10.10 and onwards)
(b)   check out branches
(U)   update dependencies(if compiling fails)
(m)   make clean

(q)   quit this compiler

Please enter your selection number below and press enter:
U



Updating!
xcode-select: error: command line tools are already installed, use "Software Update" to install updates
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to raw.githubusercontent.com:443
Warning: qt 5.14.2 is already installed and up-to-date
To reinstall 5.14.2, run `brew reinstall qt`
Warning: qt5 5.14.2 already installed
fatal: not a git repository (or any of the parent directories): .git

----------------
MLV App compiler(thanks to Ilia3101,masc,bouncyball)
----------------

current branch:

(c)   compile MLV App
(op)  compile MLV App with openmp(macOS 10.10 and onwards)
(b)   check out branches
(U)   update dependencies(if compiling fails)
(m)   make clean

(q)   quit this compiler

Please enter your selection number below and press enter:
U

Updating!
xcode-select: error: command line tools are already installed, use "Software Update" to install updates
Warning: The Ruby Homebrew installer is now deprecated and has been rewritten in
Bash. Please migrate to the following command:
  /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

Password:
==> This script will install:
/usr/local/bin/brew
/usr/local/share/doc/homebrew
/usr/local/share/man/man1/brew.1
/usr/local/share/zsh/site-functions/_brew
/usr/local/etc/bash_completion.d/brew
/usr/local/Homebrew

Press RETURN to continue or any other key to abort
==> Downloading and installing Homebrew...
HEAD is now at 49324ad17 Merge pull request #7689 from Moisan/fix_audit_release
Updated 1 tap (homebrew/core).
==> New Formulae
airshare
==> Updated Formulae
dita-ot           gopass            jolie             vim
==> Installation successful!

==> Homebrew has enabled anonymous aggregate formulae and cask analytics.
Read the analytics documentation (and how to opt-out) here:
  https://docs.brew.sh/Analytics
No analytics data has been sent yet (or will be during this `install` run).

==> Homebrew is run entirely by unpaid volunteers. Please consider donating:
  https://github.com/Homebrew/brew#donations

==> Next steps:
- Run `brew help` to get started
- Further documentation:
    https://docs.brew.sh
Warning: qt 5.14.2 is already installed and up-to-date
To reinstall 5.14.2, run `brew reinstall qt`
Warning: qt5 5.14.2 already installed
fatal: not a git repository (or any of the parent directories): .git

Regardless I still appreciate your time @names_are_hard and will let you now if I come across something to fix this. Thanks again!

[names_are_hard]

Oh, Apple makes it so that if you type "gcc", it runs clang.  Because fuck you, that's why.  I'm not sure if that's part of this problem, because that's a long standing silly thing Apple does so I'd guess Danne already works around it.

[Danne]

Are you searching your errors @deafeyejedi?

Try:

Code: [Select]

brew reinstall curl
I would suggest you find a way to uninstall command line tools and try to reinstall. Or try software update. Google this and report back how you try and solve it.

Someone mention antivirus(Kaspersky) issues around this https://superuser.com/questions/1264498/homebrew-gives-ssl-error-ssl-error-syscall-on-home-network.

Did you run this too?

Code: [Select]

chsh -s /bin/bashAre you back in bash?

Clang should be taken care of if all was set up as it should on your machine. I ran this app on two catalina macs and they were working fine.
In the script it will look like this:

Code: [Select]

cd Mlv_app_master && \
$(ls -d /usr/local/Cellar/qt/5* | head -1 | tr -d ':')/bin/qmake ~/MLV-App-master/platform/qt/MLVApp.pro QMAKE_CC=/usr/local/opt/llvm@7/bin/clang QMAKE_CXX=/usr/local/opt/llvm@7/bin/clang++ QMAKE_LINK=/usr/local/opt/llvm@7/bin/clang++ QMAKE_CFLAGS+=-fopenmp QMAKE_CXXFLAGS+=-fopenmp INCLUDEPATH+=-I/usr/local/opt/llvm@7/include LIBS+=-L/usr/local/opt/llvm@7/lib LIBS+=-lgomp QMAKE_MACOSX_DEPLOYMENT_TARGET=10.8 -spec macx-clang CONFIG+=x86_64 CONFIG+=release && /usr/bin/make -j4 && $(ls -d /usr/local/Cellar/qt/5* | head -1 | tr -d ':')/bin/macdeployqt ~/MLV-App-master/platform/Mlv_app_master/MLV\ App.app && \
make clean && \
If you run this:

Code: [Select]

/usr/local/opt/llvm@7/bin/clangYou should get this to verify it is installed:

Code: [Select]

clang-7: error: no input filesAre you?

[DeafEyeJedi]

Thanks @Danne I've been trying to. Just did a software update (10.15.5) where it also updated the command line tools.

I actually came across that site re: ssl error and I even tried to use my hotspot just to see if it'll help to no avail.

Anyhow, I've went ahead and followed your instructions from above and here are my findings below:

After running brew reinstall curl this is what I've got:

Code: [Select]

Last login: Thu Jun  4 23:45:30 on ttys000

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
Macs-MacBook-Pro:~ mac$ brew reinstall curl
==> Downloading https://homebrew.bintray.com/bottles/curl-7.70.0.catalina.bottle
Already downloaded: /Users/mac/Library/Caches/Homebrew/downloads/0dd6194ae8605b961d6c4d53d4fade5e478c2d6b45e59f64c83df221fc609920--curl-7.70.0.catalina.bottle.tar.gz
==> Reinstalling curl
==> Pouring curl-7.70.0.catalina.bottle.tar.gz
==> Caveats
curl is keg-only, which means it was not symlinked into /usr/local,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.

If you need to have curl first in your PATH run:
  echo 'export PATH="/usr/local/opt/curl/bin:$PATH"' >> ~/.profile

For compilers to find curl you may need to set:
  export LDFLAGS="-L/usr/local/opt/curl/lib"
  export CPPFLAGS="-I/usr/local/opt/curl/include"


zsh completions have been installed to:
  /usr/local/opt/curl/share/zsh/site-functions
==> Summary
🍺  /usr/local/Cellar/curl/7.70.0: 459 files, 3.2MB
Macs-MacBook-Pro:~ mac$
Ran 'chsh -s /bin/bash' which got me to this:

Code: [Select]

Macs-MacBook-Pro:~ mac$ chsh -s /bin/bash
Changing shell for mac.
Password for mac:
chsh: no changes made
Macs-MacBook-Pro:~ mac$
Upon running this...

Code: [Select]

cd Mlv_app_master && \
$(ls -d /usr/local/Cellar/qt/5* | head -1 | tr -d ':')/bin/qmake ~/MLV-App-master/platform/qt/MLVApp.pro QMAKE_CC=/usr/local/opt/llvm@7/bin/clang QMAKE_CXX=/usr/local/opt/llvm@7/bin/clang++ QMAKE_LINK=/usr/local/opt/llvm@7/bin/clang++ QMAKE_CFLAGS+=-fopenmp QMAKE_CXXFLAGS+=-fopenmp INCLUDEPATH+=-I/usr/local/opt/llvm@7/include LIBS+=-L/usr/local/opt/llvm@7/lib LIBS+=-lgomp QMAKE_MACOSX_DEPLOYMENT_TARGET=10.8 -spec macx-clang CONFIG+=x86_64 CONFIG+=release && /usr/bin/make -j4 && $(ls -d /usr/local/Cellar/qt/5* | head -1 | tr -d ':')/bin/macdeployqt ~/MLV-App-master/platform/Mlv_app_master/MLV\ App.app && \make clean && \
...it would then give me the same exact 20 fatal error messages as previously stated. Odd.

After running '/usr/local/opt/llvm@7/bin/clang' I can confirm this:

Code: [Select]

clang-7: error: no input files

[Danne]

You need to add your password when changing to bash.

I think you need to uninstall command line tools and reinstall it.

Well. Good luck for now. If you summarize your errors and try to solve them one by one it should work. Also compare the compiling string for "c" option which seems to work.

[DeafEyeJedi]

You need to add your password when changing to bash.

I did and it states that 'no changes made'

I think you need to uninstall command line tools and reinstall it.

I just manually deleted the CommandLine Folder and then installed a fresh new one through your Compiler.app 

Well. Good luck for now. If you summarize your errors and try to solve them one by one it should work. Also compare the compiling string for "c" option which seems to work.

Okay I'll do that. We def know that 'c' option works just fine but it isn't cooperating when selecting 'op' for the faster exports.

Will let you know my findings. Trying to refrain myself from downgrading this MBP to MoJave. Thanks again @Danne!

[dfort]

Trying to refrain myself from downgrading this MBP to MoJave.

Shouldn't need to do that, these instructions should work fine on Catalina, at least it works fine over here without having to change from the new default zsh shell.

What seems to be the problem is that all of these instructions are for compiling ML and you are struggling to compile MLV App.

[Levas]

How/where is handled which modules are included in a platform zip file build ?

So I have a Magic-lantern build on my computer.
Open terminal and go to the directory of my camera (Magic-lantern/Platform/6d.116/)
In here I do "make clean && make zip" command.
And now I have a zip, with some of the available modules in the build.
How can I get, for example, also the adtg_gui module in the zip file ?

[Danne]

Check for this file in modules folder. Makefile.modules.default
For eosm i have it looking like this:

Code: [Select]

MODULE_CFLAGS += -I../../src/
ML_MODULES ?= \
mlv_lite \
mlv_play \
mlv_snd \
dual_iso \
silent \
lua \
crop_rec \
sd_uhs \
file_man \You can add adtg_gui in there to build for your platform. That is you have adtg_gui already working in your branch.

[Levas]

Thanks 
adtg_gui already works over here, but until now, I run command "make" in the adtg_gui module directory itself and copy the module file on an existing build on my SD card.