Automatic, cross-platform Magiclantern building via Docker

[names_are_hard]

I thought I would try and make it easier for people to build Magiclantern.  This is a work in progress and only brave people should help me test - but I do want testers!  This should work on Linux, Mac or Windows 10, I have only tested on Linux.

The idea is we'd only need one set of instructions for building on any OS, and as a bonus everyone would be building with the same build tools, which is nice for debugging problems.  I hope the instructions can be much simpler than the current process.  There are some downsides but I think they're manageable.

To help, you will need git, and be happy to run command-line stuff.  Do this:
git clone https://bitbucket.org/stephen-e/ml_docker_builder
Then follow the instructions in the README.txt.

Please use README.txt (I want feedback on those instructions so that I can improve them).  However, so that people can see what I'm trying to do, the process is like this:

<install docker>
<become root or admin>
<copy-paste the following lines...>
docker build -t ml_build .
docker rm ml_build_output
docker create --name ml_build_output ml_build https://bitbucket.org/hudson/magic-lantern/branch/unified 5D3.113
docker start --attach ml_build_output
docker cp ml_build_output:/home/ml_builder/ml_build/autoexec.bin .

You should now have autoexec.bin.  You can change the repo or camera version to get different autoexec.bin.  It works with both Mercurial and Git repos (but this is pretty crude, I'm sure there are cases I haven't considered).

I *think* that is a fairly easy way to get started building ML?  If it isn't helpful, please let me know.  If there are obvious things that should be added, also let me know. I guess I want some ability to make the zipfile?  I don't know what most people use to create the files they need.

I also don't have a cam that works with ML, so I can't test this is currently a good build.  I know ML can build but with broken output with some compiler versions, etc - at the moment I just want to know if autoexec.bin builds for different people on different OS.  If you want to try the autoexec.bin, that's up to you!  I give no guarantees!

[timbytheriver]

Thanks! But I think we need the complete folder/file set – not just autoexec.bin…

Usually after the unzip we have (in root)

Code: [Select]

autoexec.bin
ML (folder)
ML-SETUP.FIR


[names_are_hard]

Yup - autoexec.bin was simply the easiest proof of concept.  It's easy to change it to copy out the zip file instead.

Did you try it and did it work for autoexec.bin?  If that does work on Mac I'll bother making it output the zip.  Or if you want you can take a look at ml_build_setup.py, which does the work inside the container.

[names_are_hard]

I take it back, I'm not going to make it output a zip.  Trying to do so lead me to find that the build process is a) dependent on Python 2, b) broken if you try and run make zip on my system.  Don't know why.  Not going to investigate before all the Python is made v3 compliant.

[DeafEyeJedi]

Thanks for this @names_are_hard and out of respect I would prefer to wait until I get my compiling environment fixed on Catalina first before I can test this one out. Nevertheless, it looks intriguing and I may very well jump on the gun with my older Mac if need.

[names_are_hard]

I have it working locally to create zip files, but my hack fix to the build process probably only works on Linux and I don't use this process as I have a working normal build environment.

Technical details should anyone happen across this in the future: ML has multiple .py files that don't have #!/usr/bin/env lines, and within the scripts shell out to "python".  So if python 3 is your "python", these scripts fail.  If you only add #!/usr/bin/env python2 lines, they *still* fail, because of the shelling out to plain "python".  But I know on at least Windows and I think Mac, the installer doesn't create a "python3" and/or "python2" in your path.  The clean solution is to port all scripts to python3 and never shell out to "python"; instead, if we must shell out to python, query the running python and use that directly for consistency.  But really, stop shelling out to python from within python, that's horrid.  If you want to run python code from within python, import it!

This has the inline python2 changes that likely break cross-platform builds:
https://bitbucket.org/stephen-e/ml_200d/commits/30bab0cf0b8b7a26b9d7704ff7dd64ed46f37774
Previous commit has the #!/usr/bin/env changes, which shouldn't break on Windows (it ignores #! lines), might break on Mac.