Compiling Magic Lantern with Cygwin/MinGW-64

[dfort]

Setting up a Magic Lantern development environment with Cygwin/MinGW-64

Quick start for the terminally impatient:

Download the Cygwin installer. We're going to use the 32-bit installer. Don't worry if you have a 64-bit machine, a 32-bit Magic Lantern development environment works better than a 64-bit Cygwin installation.

setup-x86.exe

Assuming it is in your Downloads directory open a Command Prompt. Note: Don't use the PowerShell, it seems to have a problem with the long command we're using in this tutorial.

Run the installer in the Command Prompt - just copy and paste the following line and it will automatically install the packages needed to compile Magic Lantern.

Code Select Expand

"%UserProfile%"\Downloads\setup-x86.exe -s http://cygwin.mirrors.pair.com -q -P mingw64-i686-gcc-core,gcc-core,make,mercurial,python37-docutils,perl,wget,unzip,zip,diffutils

Open a Cygwin Terminal from your start menu.

Code Select Expand

cd && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip && mkdir gcc-arm-none-eabi-4_8-2013q4 && unzip -n gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip -d gcc-arm-none-eabi-4_8-2013q4 && rm gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip

That was the ARM toolchain that the unified branch is using but some of the experimental branches are looking for a newer version so let's get that one to.

Code Select Expand

cd && wget https://launchpad.net/gcc-arm-embedded/5.0/5-2016-q3-update/+download/gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip && mkdir gcc-arm-none-eabi-5_4-2016q3 && unzip -n gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip -d gcc-arm-none-eabi-5_4-2016q3 && rm gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip

Now get the Magic Lantern source.

Code Select Expand

hg clone https://bitbucket.org/hudson/magic-lantern && cd magic-lantern && hg update unified && cd

Go!

Cygwin is probably the quickest and easiest way to set up a ML development environment on Windows. Note that you won't be able to use all of the features of the ML build scripts as you can with Linux or to an extent on a Macintosh, though you can still compile an ML nightly zip file and all of command line tools. It used to be that you couldn't create Windows native cr2dng, mlv_dump or other ML command line tools. However, ML Supporter, Hero Member and Cygwin advocate Marsu42 pointed out that it is possible to use the MinGW-w64 gcc compilers with Cygwin to in effect cross compile Windows native binaries.

One caveat before you start. If you've got a space in your Windows user name it doesn't work. I tried coming up with a workaround but nothing worked 100% of the time. The best advice is to create a user name without spaces.

Get the Cygwin Installer, setup-x86.exe (just click on the link to save a few steps), these are the same installers that are on the official Cygwin website.

Just accept the default options for the installer.

The easiest way to find what you want is to type it in the Search field. For example, to get the 32-bit version of the MinGW gcc package which is key to cross compiling the Magic Lantern command line tools:

That's the mingw64-i686-gcc-core package. What if you have a 64-bit system?

Note that most module exe helpers are targeted for x86 code, i.e. even if you run cygwin x64 (which you should on a x64 Windows) you need to use the i686 mingw toolchain.

Please leave feedback if you get mingw64-x86-64-gcc-core working and on which command line tools--it is going to need some tweaks to the ML source code. In the meantime I'm recommending setting up a 32-bit Cygwin even on 64-bit machines. Why? because the command line tools like mlv_dump don't build properly in the Cygwin64 environment.

You will also need some other packages:

Devel

• gcc-core (Yes, you will need both the MinGW and Cygwin compilers)
• make
• mercurial

Python

• python37-docutils

Perl

• perl

Web

• wget

Archive

• unzip
• zip

Utils

• diffutils

There may be some other packages that you might want to install but this should be enough to build the unified branch which is what gets pushed out to the nightly builds.

Once the installer is finished, let it make shortcuts to your startup menu.



Go ahead and launch the Cygwin Terminal.

We're almost there. Canon cameras are obviously not Windows computers so we will need one more compiler to build something that will run on the ARM Architecture used in the camera. To figure out which ARM compiler to get refer to the ML code, specifically: Makefile.user.default -- ARM_PATH=~/gcc-arm-none-eabi-4_8-2013q4 notice that little squiggly mark after the equal sign? That means that the path starts in your home directory. Let's make it easy, just copy/paste the following into your Cygwin Terminal:

Code Select Expand

cd && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip && mkdir gcc-arm-none-eabi-4_8-2013q4 && unzip -n gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip -d gcc-arm-none-eabi-4_8-2013q4 && rm gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip

Some of the newer experimental branches prefer a newer ARM toolchain so get that one too:

Code Select Expand

cd && wget https://launchpad.net/gcc-arm-embedded/5.0/5-2016-q3-update/+download/gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip && mkdir gcc-arm-none-eabi-5_4-2016q3 && unzip -n gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip -d gcc-arm-none-eabi-5_4-2016q3 && rm gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip


Finally, time to grab the ML source code and start building. There are plenty of resources on the forum but basically, to get the latest unified branch:

Code Select Expand

hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg update unified

Here's the way I do it for the EOSM:

Code Select Expand

cd magic-lantern/platform/EOSM.202/
make zip

So you want to start hacking away on the code? There's lots of choices when it comes to text editors, I would recommend using Notepad++. It is designed to work with source code and you can check if the source has Linux, Mac or Windows line endings--don't mix them up! Here is a screenshot of the interface, notice that posix.c has Windows (CR+LF) line endings.

Also consider using the excellent SourceTree app to keep track of your changes and easily switch between branches.

Saving the best for last.

If you want to build one of the command line tools, let's say cr2hdr, and run it on another Windows computer you might look into the ~/magic-lantern/modules/dual_iso directory and maybe (if you already compiled ML for your camera) find a cr2hdr.exe file already built. If you run it from the Cygwin Terminal it will work fine but move it somewhere else and run it with cmd.exe and it will not work if it cannot find the cygwin1.dll. The way around this is to cross compile to Windows. It may seem strange, after all aren't you already in Windows? Not really, Cygwin sets up its own environment to mimic a posix (a.k.a. Linux) system.

Here's how to compile mlv_dump:

Code Select Expand

cd ~/magic-lantern/modules/mlv_rec

Code Select Expand

make mlv_dump.exe

By adding the ".exe" file extension the Magic Lantern Makefile(s) will use the MinGW gcc compiler and create a native Windows binary. I have tested this with cr2hdr, mlv_dump and raw2dng. There are other command line tools, some of which I don't know what they do and can't get a working build for them. That seems to be pretty much standard operating procedure in open source development. Happy hacking!



It may seem that creating Windows programs that do not rely on the cygwin1.dll was either not possible or not permitted because of this F.A.Q. item in the official Cygwin website:

• Can I build a Cygwin program that does not require cygwin1.dll at runtime?
  
  No. If your program uses the Cygwin API, then your executable cannot run without cygwin1.dll. In particular, it is not possible to statically link with a Cygwin library to obtain an independent, self-contained executable.
  
  If this is an issue because you intend to distribute your Cygwin application, then you had better read and understand https://cygwin.com/licensing.html, which explains the licensing options. Unless you purchase a special commercial license from Red Hat, then your Cygwin application must be Open Source.

ML command line tools aren't really using the Cygwin API (Application Programming Interface) and MinGW has been using the Win32 API for years so there shouldn't be any legal issues using a setup like this as long as everything is kept open source. However, keep in mind that I'm not a legal expert.

[chris_overseas]

Thanks a bunch for this! A Cygwin/MinGW environment something I've been meaning to try to get set up for a while now because developing/building from an old 32 bit Linux VM (my current setup) is a pain at times. When I get a chance I'll follow your instructions and let you know if I hit any problems along the way. In the meantime though, it looks like your commands to run setup-x86.exe and setup-x86_64.exe should be prefixed with the user's home directory, eg:

Code Select Expand

%USERPROFILE%\Downloads\setup-x86.exe -K http://cygwinports.org/ports.gpg

Thanks again, hopefully I can give some better feedback soon.

[Marsu42]

Glad to be of service :-) and I hope cygwin will be recognized and maintained as a viable way to quickly compile a full ML (including docs and all) under windows with minimal disc space requirements, i.e. w/o running a full Linux vm. The basic compiler (arm gcc from launchpad) should give the same results in both Linux and Windows (though the newest 4.9 version generates buggy ML autoexec.bin for many cams for unknown reasons).

Note that most module exe helpers are targeted for x86 code, i.e. even if you run cygwin x64 (which you should on a x64 Windows) you need to use the i686 mingw toolchain.

* Btw you should select the "exp" radio button when running the cygwin setup using the cygports packages, this way you'll get more recent versions of the packages. You can always select older individual packages by switching through the versions, but the official cygwin policy is rather conservative so the non-"exp" repo often contains rather old versions.

* Fyi the one disadvantage of the Windows kernel is the lack of fork(), for example this results in a crawling automake ./configure so in these cases running a Linux vm is actually way quicker than the windows cygwin emulation layer - but this doesn't concern ML.

* Suggestion: Some time ago I posted a 7zip of a cygwin setup dir with all the required packages for ML already added and a working Makefile.user (cygwin is portable by default, just drop the unpacked dir anywhere). Amazingly, the required size was only a few megs. It just needed the launchpad gcc dropped in the respective dir. Maybe someone could refresh this, it would spare people going through the motions of selecting the packages by themselves as I don't know of any method to feed a list of packages to the cygwin installer.

[dfort]

In the meantime though, it looks like your commands to run setup-x86.exe and setup-x86_64.exe should be prefixed with the user's home directory, eg:

Code Select Expand

%USERPROFILE%\Downloads\setup-x86.exe -K http://cygwinports.org/ports.gpg

Great tip. I incorporated it in the tutorial and gave you credit.

...a viable way to quickly compile a full ML (including docs and all) under windows...

I've compiled ML under OS-X, Linux and three different Windows setups and have never been able to get the docs working. They are also missing in the nightly builds so I take it that until someone is willing to update the docs users are going to have to search for the documentation--I found that it is spread around various sites, especially some of the newer modules.

Note that most module exe helpers are targeted for x86 code, i.e. even if you run cygwin x64 (which you should on a x64 Windows) you need to use the i686 mingw toolchain.

* Btw you should select the "exp" radio button when running the cygwin setup using the cygports packages, this way you'll get more recent versions of the packages. You can always select older individual packages by switching through the versions, but the official cygwin policy is rather conservative so the non-"exp" repo often contains rather old versions.

Thanks, I incorporated those tips in the tutorial.

Lately I've been having some problems with the installer verifying the cygwinports signatures:

Code Select Expand

Mirror Error: Setup.ini signature
ftp://ftp.cygwinports.org/pub/cygwinports/x86/setup.bz2.sig from
ftp://ftp.cygwinports.org/pub/cygwinports/ failed to verify.
Possible corrupt mirror? Setup.ini rejected.
and

Code Select Expand

Mirror Error: Setup.ini signature
ftp://ftp.cygwinports.org/pub/cygwinports/x86/setup.ini.sig from
ftp://ftp.cygwinports.org/pub/cygwinports/ failed to verify.
Possible corrupt mirror? Setup.ini rejected.
Note the issue is with setup.bz2.sig and setup.ini.sig. However it seems to run fine after dismissing the error windows.

I saw your post with the Cygwin download but that would be too easy and doesn't require a tutorial

Seriously, I was thinking the same thing, deleting all the extra packages until it was down to the bare essentials to compile ML but then isn't it a problem updating the installation? It would also take lots of time and testing to make sure nothing breaks.

Gee, how did I get myself here--I'm a Mac user!

[Marsu42]

Seriously, I was thinking the same thing, deleting all the extra packages until it was down to the bare essentials to compile ML but then isn't it a problem updating the installation? It would also take lots of time and testing to make sure nothing breaks.

No problem there, if you de-select all packages on installation and then carefully select only those required for ML (some trial and error is involved) the cygwin installer takes care of the (minimal) dependencies and thus no problem on updating exists. You might want to de-select all manual/docs packes and even delete the /share/doc folder to save space - on installation the docs just get installed again.

If you found my old posts, there is even a simple script included that pulls the ml repo, copies Makefile.user and compiles the whole thing - I don't quite remember how elaborate it was.

Last not least, the docs do work - I got it to work using the cygwin tex, but you have to get all dependencies right and it needs a lot of disk space so including it into a "bare bones" zip package might not be feasible.

[dfort]

Wow, look what I found:

Code Select Expand

Does Setup accept command-line arguments?

Yes, the full listing is written to the setup.log file when you run setup-x86.exe --help or setup-x86_64.exe --help. The current options are:

Command Line Options:
-D --download                     Download from internet
-L --local-install                Install from local directory
-s --site                         Download site
-O --only-site                    Ignore all sites except for -s
-R --root                         Root installation directory
-x --remove-packages              Specify packages to uninstall
-c --remove-categories            Specify categories to uninstall
-P --packages                     Specify packages to install
-C --categories                   Specify entire categories to install
-p --proxy                        HTTP/FTP proxy (host:port)
-a --arch                         architecture to install (x86_64 or x86)
-q --quiet-mode                   Unattended setup mode
-M --package-manager              Semi-attended chooser-only mode
-B --no-admin                     Do not check for and enforce running as
                                   Administrator
-h --help                         print help
-l --local-package-dir            Local package directory
-r --no-replaceonreboot           Disable replacing in-use files on next
                                   reboot.
-X --no-verify                    Don't verify setup.ini signatures
-n --no-shortcuts                 Disable creation of desktop and start menu
                                   shortcuts
-N --no-startmenu                 Disable creation of start menu shortcut
-d --no-desktop                   Disable creation of desktop shortcut
-K --pubkey                       URL of extra public key file (gpg format)
-S --sexpr-pubkey                 Extra public key in s-expr format
-u --untrusted-keys               Use untrusted keys from last-extrakeys
-U --keep-untrusted-keys          Use untrusted keys and retain all
-g --upgrade-also                 also upgrade installed packages
-o --delete-orphans               remove orphaned packages
-A --disable-buggy-antivirus      Disable known or suspected buggy anti virus
                                   software packages during execution.


So you should be able to do:

Code Select Expand

setup-x86.exe -q -P packagename1,packagename2
and get just the essential packages.

Still need to figure out how to use the --site option for the cygwinports packages.

[dfort]

Found out you don't need to go to cygwinports.org to get the MinGW-64 tool chain. I also found out how to do a nearly automatic install:

Code Select Expand

%USERPROFILE%\Downloads\setup-x86.exe -s http://cygwin.mirrors.pair.com -q -P mingw64-i686-gcc-core,gcc-core,make,mercurial,python-docutils,wget,unzip,zip,diffutils
One gotcha--it doesn't work if you are in the root directory when you run the command because it can't change to the root directory. Go figure.

I'll add this as an option but will keep the rest of it the same just in case people have trouble with it.

[Marsu42]

Wow, look what I found

I saw these command line options, too, but didn't have much luck with them - for example I was unable to find a way to update all packages from the "exp" repo w/o going through the gui, essentially what you can do in any Linux distro with rpm or apt-get. It seems the cygwin installer isn't optimized for command line use, but feel free to try the options and post suggestions to the cygwin mailing list - as cygwin is actively maintained there's a good chance any reasonable requests might be incorporated quickly.

[Marsu42]

When you get to cross compiling the command line tools some tools look for the MinGW gcc compiler in the home directory so we need to create a directory structure and make a symbolic link to that compiler in there:

The much better option would be to modify all exe helper Makefiles to use one central compiler path that is found in the Makefile.user ... symlinking stuff works, but I don't like the approach.

Here's a hack to try out--trick the system to use the 64-bit gcc compiler instead of the 32-bit one. Please leave feedback so we know whether or not this works and on which tools.

The issue is not setting up the mingw windows compiler to x64 (simply use x86_64-w64-mingw32-gcc.exe instead of i686-w64-mingw32-gcc.exe) but at least cr2hdr definitely doesn't work with 64bit integers ... a x64 c compiler isn't a drop-in replacement for x86 but often needs adapted code. As there's really little reason to have the exe helpers as x64 (well, cr2hdr might be a bit faster (or not)) imho just stick with x86 to prevent platform-specific bugs to confuse debugging ml even further.

[dfort]

The much better option would be to modify all exe helper Makefiles to use one central compiler path that is found in the Makefile.user ... symlinking stuff works, but I don't like the approach.

Yes, I know. I wrote this tutorial to get someone up and running without any source code tweaking. The Makefile shouldn't be looking into the home directory for anything other than the ARM compiler--though I think it should also be moved out of the home directory.

As there's really little reason to have the exe helpers as x64 (well, cr2hdr might be a bit faster (or not)) imho just stick with x86 to prevent platform-specific bugs to confuse debugging ml even further.

I was suggesting a rather ugly hack--I'll take it out. By the way, cr2hdr is one of the exe helpers that looks for the cross compiler in the home directory. Hopefully that will be changed when the 20-bit version of cr2hdr is merged. Then maybe we can have it use the 64-bit compiler if possible?

There is no end to this is there?

[Marsu42]

Hopefully that will be changed when the 20-bit version of cr2hdr is merged. Then maybe we can have it use the 64-bit compiler if possible?

The 20bit version doesn't compile with x64 either, and alex has stated that this is known and most likely "wontfix" because the code changes are not worth the possible benefits.

There is no end to this is there?

You probably spend a lot of time figuring out msys and mingw :-) ... but actually cygwin works quite nicely recently, it used to be much worse. Basically it's just the compiler paths for the module exe helpers and some other smallish stuff. Given the nearly stalled ml development, I don't expect any major ml build system changes soon that would break cygwin again.

And in any case, very few people will hack the source themselves and the need for "compile it yourself" is basically gone with the nightly builds. So the one real use for compiling ml is the integration of some branch or pull request you cannot wait for to be merged (like nss audio on 6d), and again I doubt many people will do this. The download counter for my former cygwin zip package was extremely low, go figure.

[dfort]

Thanks for the info.

I started getting serious building ML when the EOSM platform was disabled from the nightly builds. I got on Windows when testers wanted Windows binaries of some of the exe helpers from dmilligan's ml_dng branch. What I would really like to have happen is to get the EOSM shutter-bug resolved because that directly affects me. If more users can build ML maybe someone will come up with a solution? Oh, and the EOSM hasn't been updated in the nightlies since April--not to complain because I'm getting quite an education diving into the source code. I'm actually happy that I was able to get MinGW working simply because it was a challenge and it looks like nobody else was able to do it. Sort of an ego booster.

I don't know how many people are going to follow this tutorial. From your low download counter on something that was much easier to follow I'd say even fewer will give this a try.

[garry23]

@dfort

I for one truly hope that this becomes a functioning toolset.

I played around with compiling in the cloud and created my own 'versions' of ML using this approach.

I personally believe this is the real value of being able to compile.

I haven't yet tried the Cygwin/MinGW-64 approach, but will do so once it is clearer how I can take the latest nightly, tweak or add to the code and then compile.

As I read the above, this isn't fully clear to me yet: and this is a criticism of me, not you :-)

[dfort]

Hi @garry23

I also played around with compiling in the cloud--seems that was the most popular method for non-developers to get into rolling their own ML builds. Too bad that it doesn't work for new users or maybe it does with a paid subscription.

If you created your own versions of ML in the cloud you should already know how to take the latest nightly, tweak and compile--it works pretty much the same way in Cygwin/MinGW-64 only better because you have more control over the development environment.

Here are a few tricks you can do without having to code.

Want to check out all the hoopla over lua?

Code Select Expand

hg clone -r lua https://bitbucket.org/hudson/magic-lantern

Got an EOSM and want to get something newer that the April version that is available in nightlies?

Code Select Expand

hg clone -r EOSM__working https://bitbucket.org/daniel_fort/magic-lantern

How about going bleeding edge and peeking into some cool DNG development work being done by David Milligan:

Code Select Expand

hg clone -r ml_dng https://bitbucket.org/dmilligan/magic-lantern

You don't really need much in the way of coding skills to contribute. My first merged pull request was simply replacing [DELETE] with [Trash to change]:
EOS-M Format Card - Trash toggles keep or delete ML - UI Clarification That was also my first bug fix: EOS-M can't TAP to switch keep ML when formatting

Is it fully clear yet? Maybe a little less "cloudy" ?

[garry23]

@dfort

I think the question I had in my mind is, with Cygwin/MinGW-64 approach are you provided a tool to change code.

That is, can I access the ML code and the parsing etc is handled correctly?

One final thought is that if someone is using this approach: would they be able to video a demo for us non-coders who are trying to get into all this 'coding stuff'.

Cheers

Garry

[Marsu42]

One final thought is that if someone is using this approach: would they be able to video a demo for us non-coders who are trying to get into all this 'coding stuff'.

Imho the simplest way is to provide a bash shell script that people just would need to run after installing (or unzipping) cygwin - this would automatically download the latest ml code, copy a working Makefile user, compile everything and then throw the autoexec.bin & modules .mo into a folder. People would just have to look at this shell script to understand how it works.

You don't really need much in the way of coding skills to contribute.

Personally, I'd advise caution to prevent frustration. If you can code C and know the basics of distributed software development (i.e. Mercurial) then it's easy to get started.

But if you want to change non-trivial things a lot of learning about how the Canon fw and the ML code works is involved, and there are always things that simply don't work as you expect because ML depends on what Canon provides. And for trivial changes, in the future you can just use user scripting with lua and use ML's lua api which has the sandbox advantage of not being able to brick the camera.

Last not least, real embedded hardware hacking is another matter altogether and far beyond my skills - an example would be the "new sound system" branch.

[dfort]

Imho the simplest way is to provide a bash shell script that people just would need to run after installing (or unzipping) cygwin - this would automatically download the latest ml code, copy a working Makefile user, compile everything and then throw the autoexec.bin & modules .mo into a folder. People would just have to look at this shell script to understand how it works.

You mean something like this?

Code Select Expand

#!/bin/sh

# Uncomment the appropriate line
# to build to your preferred
# Magic Lantern camera platform
#
# Uncomment means to simply delete the
# '#" sign from the beginning of the line
# DO THIS FOR ONLY ONE CAMERA AT A TIME!
#
#CAMERA=5D2.212
#CAMERA=5D3.113
#CAMERA=6D.116
#CAMERA=7D.203
#CAMERA=50D.109
#CAMERA=500D.111
#CAMERA=550D.109
#CAMERA=600D.102
#CAMERA=650D.104
#CAMERA=700D.114
#CAMERA=1100D.105
#CAMERA=EOSM.202

cd ~/magic-lantern
hg pull && hg update
cd platform/$CAMERA
make clean && make zip
mv *.zip ~/
make clean
cd ~



I didn't go as far as unzipping, moving the files to the card and making some toast and coffee--don't know how far to take it. I did test it and it works fine. I might be a little overzealous with the "make clean" though.

Personally, I'd advise caution to prevent frustration. If you can code C and know the basics of distributed software development (i.e. Mercurial) then it's easy to get started.

I knew nothing about any of this stuff when I started a few months ago. Well, I did take a BASIC and a Pascal programming class in the '80's but forgot most of it. I also tinkered around with the terminal in Linux and OS-X and made some very simple shell scripts but I never even wrote a "hello world" in C. What I did do was follow instructions on how to compile open source programs and found many needed some sort of little tweak besides ./configure make and make install.

But if you want to change non-trivial things a lot of learning about how the Canon fw and the ML code works is involved, and there are always things that simply don't work as you expect because ML depends on what Canon provides. And for trivial changes, in the future you can just use user scripting with lua and use ML's lua api which has the sandbox advantage of not being able to brick the camera.

Of course users don't have to wait until the lua pull request is merged. I have already used it and some debugging tools found in other branches to take a stab at killing the EOS-M shutter bug. No luck but I haven't bricked a camera--yet.

By the way, you can brick a camera with a lua script. Take a closer look at dmilligan's code:

Code Select Expand

// !!!DANGER WILL ROBINSON!!!
//#define LUA_PROP_REQUEST_CHANGE

Last not least, real embedded hardware hacking is another matter altogether and far beyond my skills - an example would be the "new sound system" branch.

Wrestling with the Shutter-bug forced me to search for a new "stub" so I had to disassembe my ROM1.BIN (link) and wow--how did they figure this stuff out? If I ever run into g3gg0 or a1ex I may just fall to my knees and scream, "I'm not worthy!"

[DeafEyeJedi]

I hear you, @dfort! 

[Marsu42]

You mean something like this?

Yeah, something like this ... here's the end of my current "just works" compile script. I see no sense to use make zip locally as I instantly want to use the files, thus I copy the stuff over a (recent) official nightly version in case I missed some files and then copy everything to the card (if it's available). You'd need to add that part to yours and use a variable for my hardcoded 6d.116. The "copy all modules" line should be of special interest, took me a while to get that shell line right :-p

Code Select Expand


mkdir -p ../dist/ML/modules
rm -f ../dist/ML/modules/*.mo
mv platform/6D.116/autoexec.bin ../dist/
mv platform/6D.116/6D_116.sym ../dist/ML/modules/
cp modules/*/*.mo ../dist/ML/modules
cp modules/dual_iso/*.exe ../dist/ML/
cp modules/raw_rec/*.exe ../dist/ML/
cp modules/mlv_rec/*.exe ../dist/ML/
cp modules/io_crypt/*.exe ../dist/ML/
mkdir -p ../dist/ML/fonts
cp data/fonts/* ../dist/ML/fonts


By the way, you can brick a camera with a lua script. Take a closer look at dmilligan's code

Ugh, I didn't realize and thought there were some additional safeguards with lua. So note: When in doubt, use new ML code in a C mode :-> ... beats having a heart attack every time your camera not turning itself on (mostly b/c the battery is empty) :-o

If I ever run into g3gg0 or a1ex I may just fall to my knees and scream, "I'm not worthy!"

These two basically carry the ml development, and finding devs with this ability and time on their hands is the reason why there's always the danger of the trunk getting stalled. Lucky us ml is working ok right now and essential features (like dual_iso & raw video) are production ready.

As for frustration with development: You might be lucky and and everything works as you want to, but there's always the possibility the Canon fw doesn't play ball (for example b/c a button doesn't send an event code) and you end up with endless sessions of trial & error, swapping the card between computer and camera. And sometimes, you have to realize it won't work at all properly - like reading the exposure from EC in M mode, w/o direct hardware access it'll be always dodgy.

[dfort]

...You'd need to add that part to yours...

Like I said, there's no end to this. I was thinking of copying over the files to the user's desktop so there's no need to even look at the Cygwin home directory. On your script you might think about adding ML/scripts especially if/when lua is merged and ML/doc if you have your docs working.

BTW--I never looked into io_crypt, didn't know ML could do that. I've got an EOS-M so I can't play around with it, I also don't shoot subjects that would require encryption in case I get captured and tortured to show my photos. (Usually it is the other way around.)

Ugh, I didn't realize and thought there were some additional safeguards with lua.

Actually dmilligan has that line commented out so it isn't possible to change properties on a normal lua build. I posted an EOSM lua build on the shutter-bug issue page and struggled with the decision whether or not to turn on that option. To fix the shutter-bug you probably need the ability to change properties but if a user can't compile ML then turning on that option is giving him just enough rope to hang himself.

I may be straying a bit off topic but having the ability to compile your own custom version of ML with lua and turning on the hangman option is definitely on topic.

These two basically carry the ml development, and finding devs with this ability and time on their hands is the reason why there's always the danger of the trunk getting stalled. Lucky us ml is working ok right now and essential features (like dual_iso & raw video) are production ready.

Yes but at least with the platform that I'm on there has been some bug fixes that aren't getting pushed out to the users. That's one of the reasons why I started building my own ML versions. It turned out to be one of those, "when life hands you lemons make lemonade" situations.

And sometimes, you have to realize it won't work at all properly - like reading the exposure from EC in M mode, w/o direct hardware access it'll be always dodgy.

EC in M mode? Had to look look it up in my ML to English dictionary. The autoexposure module is something else I haven't played around with so I turned it on in my camera and whoa!



Don't know if I want to go down that rabbit hole at the moment.

In any case, compiling ML has opened up a whole new world for me and a deeper understanding of how the camera and Magic Lantern works. It also made me aware of just how little I know.

[Marsu42]

I also don't shoot subjects that would require encryption in case I get captured and tortured to show my photos. (Usually it is the other way around.)

:-p ... The more probable real-life situation is that you shoot an event, and the organizer pressures you to "just give him your shots" which a pro photog would never do. If your files are encrypted, you can just say "be my guest" and state that you have no idea why his whatever-he's-using-raw-converter doesn't work :->

EC in M mode? Had to look look it up in my ML to English dictionary. The autoexposure module is something else I haven't played around with so I turned it on in my camera and whoa!

My (now private) auto_iso modules also contains ec in m, it's very useful if shoot in changing lighting conditions and still want to pre-set the exposure (esp. for flash or capturing motion) and aperture (depth of field). Nikon has it, and Canon graced their 1dx with this feature in a recent fw update, but other Canon cams are too dumb.

Anyway, it's just an example when something nice doesn't quite work due to the limited Canon fw - the EC indicator in M mode is just made for rough visual confirmation of over/underexposure, not an exact or fast meter you can use reliably by ML code ... we'd need direct hardware access to the metering system for that.

[dfort]

The much better option would be to modify all exe helper Makefiles to use one central compiler path that is found in the Makefile.user ... symlinking stuff works, but I don't like the approach.

Made a pull request for that:

https://bitbucket.org/hudson/magic-lantern/pull-requests/659/eliminate-need-for-mingw-w32-bin-i686-w64/diff

Now comes the waiting game.

I didn't work on cross compiling dng2raw because I'm not really sure what it does and all my attempts at cross compiling it resulted in errors so I left it pretty much alone.

[Marsu42]

Made a pull request for that

I approved it on bitbucket just for the enthusiasm of yours :-) ... but my advice from experience is to open a thread in the general dev forum and discuss promoting cygwin to a "supported" compilation environment. Imho there are good reasons to use cygwin on windows instead of a Linux vm, and it lowers the barrier for quick compilation and patching.

Do note: if the main devs don't really want this, your pull requests will have little luck and you're in for much frustration and a lifetime of cygwin patching as every new code addition will break cygwin again. Better get everyone to agree to take care of the little cygwin needs in the first place (other like for example throwing the arm gcc compiler in the home dir as the only option).

I didn't work on cross Now comes the waiting game.

Well, as you see this could take months atm so if you've got time on your hands make sure this pull requests covers full cywin/win-mingw support for everything (including make zip & make docs). I'd advise to use the cygwin version of the tool required (tex, mercurial, ...) to prevent script bugs because of different unix/win paths.

I didn't work on cross compiling dng2raw because I'm not really sure what it does and all my attempts at cross compiling it resulted in errors so I left it pretty much alone.

I didn't look at that, but that's the very reason for the existence of Cygwin (as msys doesn't have a recent compiler for the emulation environment): some sources need *nix POSIX functionality (like gettext) that Windows simply lacks. Some things can be done with little patches like an additional include line or a more precise var type, but some code would need to be completely rewritten so it's simpler just to use the emulation and be done with it.

[DimaAffovaPax]

I was wondering if I can keep working on my .wxs filelike the graphics interface, which is created when I was using wxwidgets with c/c, in wxpython then?

[dfort]

I was wondering if I can keep working on my .wxs filelike the graphics interface, which is created when I was using wxwidgets with c/c, in wxpython then?

If you are using a Cygwin environment then it should work otherwise you might try MSYS2/MinGW-w64 for a Windows native development environment--or switch to Linux if possible.