Magic Lantern Forum

Developing Magic Lantern => General Development Discussion => Topic started by: dfort on September 22, 2015, 02:04:07 AM

Title: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 22, 2015, 02:04:07 AM
Setting up a Magic Lantern development environment with Cygwin/MinGW-64

(https://farm1.staticflickr.com/679/21420052290_920ce961c2_c.jpg)

Quick start for the terminally impatient:
Download the Cygwin installer. We're going to use the 32-bit installer. Don't worry if you have a 64-bit machine, a 32-bit Magic Lantern development environment works better than a 64-bit Cygwin installation.

setup-x86.exe (http://cygwin.com/setup-x86.exe)

Assuming it is in your Downloads directory open a Command Prompt. Note: Don't use the PowerShell, it seems to have a problem with the long command we're using in this tutorial.

Run the installer in the Command Prompt - just copy and paste the following line and it will automatically install the packages needed to compile Magic Lantern.

Code: [Select]
"%UserProfile%"\Downloads\setup-x86.exe -s http://cygwin.mirrors.pair.com -q -P mingw64-i686-gcc-core,gcc-core,make,mercurial,python3-docutils,perl,wget,unzip,zip,diffutils
Open a Cygwin Terminal from your start menu.
Code: [Select]
cd && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip && mkdir gcc-arm-none-eabi-4_8-2013q4 && unzip -n gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip -d gcc-arm-none-eabi-4_8-2013q4 && rm gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip
That was the ARM toolchain that the unified branch is using but some of the experimental branches are looking for a newer version so let's get that one to.
Code: [Select]
cd && wget https://launchpad.net/gcc-arm-embedded/5.0/5-2016-q3-update/+download/gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip && mkdir gcc-arm-none-eabi-5_4-2016q3 && unzip -n gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip -d gcc-arm-none-eabi-5_4-2016q3 && rm gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip
Now get the Magic Lantern source.
Code: [Select]
hg clone https://bitbucket.org/hudson/magic-lantern && cd magic-lantern && hg update unified && cd
Go!

Cygwin is probably the quickest and easiest way to set up a ML development environment on Windows. Note that you won't be able to use all of the features of the ML build scripts as you can with Linux or to an extent on a Macintosh (http://www.magiclantern.fm/forum/index.php?topic=16012.0), though you can still compile an ML nightly zip file and all of command line tools. It used to be that you couldn't create Windows native cr2dng, mlv_dump or other ML command line tools. However, ML Supporter, Hero Member and Cygwin advocate Marsu42 pointed out that it is possible to use the MinGW-w64 gcc compilers with Cygwin to in effect cross compile Windows native binaries.

One caveat before you start. If you've got a space in your Windows user name it doesn't work. I tried coming up with a workaround but nothing worked 100% of the time. The best advice is to create a user name without spaces.

Get the Cygwin Installer, setup-x86.exe (http://cygwin.com/setup-x86.exe) (just click on the link to save a few steps), these are the same installers that are on the official Cygwin website (https://cygwin.com/).

Just accept the default options for the installer.

(https://c1.staticflickr.com/5/4276/34551209980_8fc14f4569_z.jpg)

The easiest way to find what you want is to type it in the Search field. For example, to get the 32-bit version of the MinGW gcc package which is key to cross compiling the Magic Lantern command line tools:

(https://farm1.staticflickr.com/583/21422047439_067c88b273_c.jpg)

That's the mingw64-i686-gcc-core package. What if you have a 64-bit system?
Note that most module exe helpers are targeted for x86 code, i.e. even if you run cygwin x64 (which you should on a x64 Windows) you need to use the i686 mingw toolchain.
Please leave feedback if you get mingw64-x86-64-gcc-core working and on which command line tools--it is going to need some tweaks to the ML source code. In the meantime I'm recommending setting up a 32-bit Cygwin even on 64-bit machines. Why? because the command line tools like mlv_dump don't build properly in the Cygwin64 environment.

You will also need some other packages:

Devel
Python
Perl
Web
Archive
Utils

There may be some other packages that you might want to install but this should be enough to build the unified branch which is what gets pushed out to the nightly builds.

Once the installer is finished, let it make shortcuts to your startup menu.

(https://c1.staticflickr.com/5/4251/34104340044_2cc74ec5a4.jpg)

Go ahead and launch the Cygwin Terminal.

We're almost there. Canon cameras are obviously not Windows computers so we will need one more compiler to build something that will run on the ARM Architecture used in the camera. To figure out which ARM compiler to get refer to the ML code, specifically: Makefile.user.default -- ARM_PATH=~/gcc-arm-none-eabi-4_8-2013q4 notice that little squiggly mark after the equal sign? That means that the path starts in your home directory. Let's make it easy, just copy/paste the following into your Cygwin Terminal:

Code: [Select]
cd && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip && mkdir gcc-arm-none-eabi-4_8-2013q4 && unzip -n gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip -d gcc-arm-none-eabi-4_8-2013q4 && rm gcc-arm-none-eabi-4_8-2013q4-20131204-win32.zip
Some of the newer experimental branches prefer a newer ARM toolchain so get that one too:

Code: [Select]
cd && wget https://launchpad.net/gcc-arm-embedded/5.0/5-2016-q3-update/+download/gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip && mkdir gcc-arm-none-eabi-5_4-2016q3 && unzip -n gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip -d gcc-arm-none-eabi-5_4-2016q3 && rm gcc-arm-none-eabi-5_4-2016q3-20160926-win32.zip

Finally, time to grab the ML source code and start building. There are plenty of resources on the forum but basically, to get the latest unified branch:

Code: [Select]
hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg update unified

Here's the way I do it for the EOSM:

Code: [Select]
cd magic-lantern/platform/EOSM.202/
make zip

So you want to start hacking away on the code? There's lots of choices when it comes to text editors, I would recommend using Notepad++. It is designed to work with source code and you can check if the source has Linux, Mac or Windows line endings--don't mix them up! Here is a screenshot of the interface, notice that posix.c has Windows (CR+LF) line endings. :o

(https://farm1.staticflickr.com/638/21544791975_b9af3bb5f7.jpg)

Also consider using the excellent SourceTree app (https://www.sourcetreeapp.com/) to keep track of your changes and easily switch between branches.



Saving the best for last.

If you want to build one of the command line tools, let's say cr2hdr, and run it on another Windows computer you might look into the ~/magic-lantern/modules/dual_iso directory and maybe (if you already compiled ML for your camera) find a cr2hdr.exe file already built. If you run it from the Cygwin Terminal it will work fine but move it somewhere else and run it with cmd.exe and it will not work if it cannot find the cygwin1.dll. The way around this is to cross compile to Windows. It may seem strange, after all aren't you already in Windows? Not really, Cygwin sets up its own environment to mimic a posix (a.k.a. Linux) system.

Here's how to compile mlv_dump:

Code: [Select]
cd ~/magic-lantern/modules/mlv_rec
Code: [Select]
make mlv_dump.exe
By adding the ".exe" file extension the Magic Lantern Makefile(s) will use the MinGW gcc compiler and create a native Windows binary. I have tested this with cr2hdr, mlv_dump and raw2dng. There are other command line tools, some of which I don't know what they do and can't get a working build for them. That seems to be pretty much standard operating procedure in open source development. Happy hacking!



It may seem that creating Windows programs that do not rely on the cygwin1.dll was either not possible or not permitted because of this F.A.Q. item (https://www.cygwin.com/faq.html#faq.programming.static-linking) in the official Cygwin website:

ML command line tools aren't really using the Cygwin API (Application Programming Interface) and MinGW has been using the Win32 API for years so there shouldn't be any legal issues using a setup like this as long as everything is kept open source. However, keep in mind that I'm not a legal expert.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: chris_overseas on September 22, 2015, 10:30:55 AM
Thanks a bunch for this! A Cygwin/MinGW environment something I've been meaning to try to get set up for a while now because developing/building from an old 32 bit Linux VM (my current setup) is a pain at times. When I get a chance I'll follow your instructions and let you know if I hit any problems along the way. In the meantime though, it looks like your commands to run setup-x86.exe and setup-x86_64.exe should be prefixed with the user's home directory, eg:

Code: [Select]
%USERPROFILE%\Downloads\setup-x86.exe -K http://cygwinports.org/ports.gpg
Thanks again, hopefully I can give some better feedback soon.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Marsu42 on September 22, 2015, 04:10:34 PM
Glad to be of service :-) and I hope cygwin will be recognized and maintained as a viable way to quickly compile a full ML (including docs and all) under windows with minimal disc space requirements, i.e. w/o running a full Linux vm. The basic compiler (arm gcc from launchpad) should give the same results in both Linux and Windows (though the newest 4.9 version generates buggy ML autoexec.bin for many cams for unknown reasons).

Note that most module exe helpers are targeted for x86 code, i.e. even if you run cygwin x64 (which you should on a x64 Windows) you need to use the i686 mingw toolchain.

* Btw you should select the "exp" radio button when running the cygwin setup using the cygports packages, this way you'll get more recent versions of the packages. You can always select older individual packages by switching through the versions, but the official cygwin policy is rather conservative so the non-"exp" repo often contains rather old versions.

* Fyi the one disadvantage of the Windows kernel is the lack of fork(), for example this results in a crawling automake ./configure so in these cases running a Linux vm is actually way quicker than the windows cygwin emulation layer - but this doesn't concern ML.

* Suggestion: Some time ago I posted a 7zip of a cygwin setup dir with all the required packages for ML already added and a working Makefile.user (cygwin is portable by default, just drop the unpacked dir anywhere). Amazingly, the required size was only a few megs. It just needed the launchpad gcc dropped in the respective dir. Maybe someone could refresh this, it would spare people going through the motions of selecting the packages by themselves as I don't know of any method to feed a list of packages to the cygwin installer.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 22, 2015, 09:25:36 PM
In the meantime though, it looks like your commands to run setup-x86.exe and setup-x86_64.exe should be prefixed with the user's home directory, eg:

Code: [Select]
%USERPROFILE%\Downloads\setup-x86.exe -K http://cygwinports.org/ports.gpg

Great tip. I incorporated it in the tutorial and gave you credit.

...a viable way to quickly compile a full ML (including docs and all) under windows...

I've compiled ML under OS-X, Linux and three different Windows setups and have never been able to get the docs working. They are also missing in the nightly builds so I take it that until someone is willing to update the docs users are going to have to search for the documentation--I found that it is spread around various sites, especially some of the newer modules.

Note that most module exe helpers are targeted for x86 code, i.e. even if you run cygwin x64 (which you should on a x64 Windows) you need to use the i686 mingw toolchain.

* Btw you should select the "exp" radio button when running the cygwin setup using the cygports packages, this way you'll get more recent versions of the packages. You can always select older individual packages by switching through the versions, but the official cygwin policy is rather conservative so the non-"exp" repo often contains rather old versions.

Thanks, I incorporated those tips in the tutorial.

Lately I've been having some problems with the installer verifying the cygwinports signatures:
Code: [Select]
Mirror Error: Setup.ini signature
ftp://ftp.cygwinports.org/pub/cygwinports/x86/setup.bz2.sig from
ftp://ftp.cygwinports.org/pub/cygwinports/ failed to verify.
Possible corrupt mirror? Setup.ini rejected.
and
Code: [Select]
Mirror Error: Setup.ini signature
ftp://ftp.cygwinports.org/pub/cygwinports/x86/setup.ini.sig from
ftp://ftp.cygwinports.org/pub/cygwinports/ failed to verify.
Possible corrupt mirror? Setup.ini rejected.
Note the issue is with setup.bz2.sig and setup.ini.sig. However it seems to run fine after dismissing the error windows.

I saw your post with the Cygwin download (http://www.magiclantern.fm/forum/index.php?topic=9517.msg110364#msg110364) but that would be too easy and doesn't require a tutorial :)

Seriously, I was thinking the same thing, deleting all the extra packages until it was down to the bare essentials to compile ML but then isn't it a problem updating the installation? It would also take lots of time and testing to make sure nothing breaks.

Gee, how did I get myself here--I'm a Mac user!
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Marsu42 on September 22, 2015, 09:43:55 PM
Seriously, I was thinking the same thing, deleting all the extra packages until it was down to the bare essentials to compile ML but then isn't it a problem updating the installation? It would also take lots of time and testing to make sure nothing breaks.

No problem there, if you de-select all packages on installation and then carefully select only those required for ML (some trial and error is involved) the cygwin installer takes care of the (minimal) dependencies and thus no problem on updating exists. You might want to de-select all manual/docs packes and even delete the /share/doc folder to save space - on installation the docs just get installed again.

If you found my old posts, there is even a simple script included that pulls the ml repo, copies Makefile.user and compiles the whole thing - I don't quite remember how elaborate it was.

Last not least, the docs do work - I got it to work using the cygwin tex, but you have to get all dependencies right and it needs a lot of disk space so including it into a "bare bones" zip package might not be feasible.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 22, 2015, 11:00:27 PM
Wow, look what I found:

Code: [Select]
Does Setup accept command-line arguments?

Yes, the full listing is written to the setup.log file when you run setup-x86.exe --help or setup-x86_64.exe --help. The current options are:

Command Line Options:
 -D --download                     Download from internet
 -L --local-install                Install from local directory
 -s --site                         Download site
 -O --only-site                    Ignore all sites except for -s
 -R --root                         Root installation directory
 -x --remove-packages              Specify packages to uninstall
 -c --remove-categories            Specify categories to uninstall
 -P --packages                     Specify packages to install
 -C --categories                   Specify entire categories to install
 -p --proxy                        HTTP/FTP proxy (host:port)
 -a --arch                         architecture to install (x86_64 or x86)
 -q --quiet-mode                   Unattended setup mode
 -M --package-manager              Semi-attended chooser-only mode
 -B --no-admin                     Do not check for and enforce running as
                                   Administrator
 -h --help                         print help
 -l --local-package-dir            Local package directory
 -r --no-replaceonreboot           Disable replacing in-use files on next
                                   reboot.
 -X --no-verify                    Don't verify setup.ini signatures
 -n --no-shortcuts                 Disable creation of desktop and start menu
                                   shortcuts
 -N --no-startmenu                 Disable creation of start menu shortcut
 -d --no-desktop                   Disable creation of desktop shortcut
 -K --pubkey                       URL of extra public key file (gpg format)
 -S --sexpr-pubkey                 Extra public key in s-expr format
 -u --untrusted-keys               Use untrusted keys from last-extrakeys
 -U --keep-untrusted-keys          Use untrusted keys and retain all
 -g --upgrade-also                 also upgrade installed packages
 -o --delete-orphans               remove orphaned packages
 -A --disable-buggy-antivirus      Disable known or suspected buggy anti virus
                                   software packages during execution.

So you should be able to do:
Code: [Select]
setup-x86.exe -q -P packagename1,packagename2and get just the essential packages.

Still need to figure out how to use the --site option for the cygwinports packages.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 23, 2015, 02:20:59 AM
Found out you don't need to go to cygwinports.org to get the MinGW-64 tool chain. I also found out how to do a nearly automatic install:
Code: [Select]
%USERPROFILE%\Downloads\setup-x86.exe -s http://cygwin.mirrors.pair.com -q -P mingw64-i686-gcc-core,gcc-core,make,mercurial,python-docutils,wget,unzip,zip,diffutilsOne gotcha--it doesn't work if you are in the root directory when you run the command because it can't change to the root directory. Go figure.

I'll add this as an option but will keep the rest of it the same just in case people have trouble with it.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Marsu42 on September 23, 2015, 04:28:21 AM
Wow, look what I found

I saw these command line options, too, but didn't have much luck with them - for example I was unable to find a way to update all packages from the "exp" repo w/o going through the gui, essentially what you can do in any Linux distro with rpm or apt-get. It seems the cygwin installer isn't optimized for command line use, but feel free to try the options and post suggestions to the cygwin mailing list - as cygwin is actively maintained there's a good chance any reasonable requests might be incorporated quickly.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Marsu42 on September 23, 2015, 02:23:21 PM
When you get to cross compiling the command line tools some tools look for the MinGW gcc compiler in the home directory so we need to create a directory structure and make a symbolic link to that compiler in there:

The much better option would be to modify all exe helper Makefiles to use one central compiler path that is found in the Makefile.user ... symlinking stuff works, but I don't like the approach.

Here's a hack to try out--trick the system to use the 64-bit gcc compiler instead of the 32-bit one. Please leave feedback so we know whether or not this works and on which tools.

The issue is not setting up the mingw windows compiler to x64 (simply use x86_64-w64-mingw32-gcc.exe instead of i686-w64-mingw32-gcc.exe) but at least cr2hdr definitely doesn't work with 64bit integers ... a x64 c compiler isn't a drop-in replacement for x86 but often needs adapted code. As there's really little reason to have the exe helpers as x64 (well, cr2hdr might be a bit faster (or not)) imho just stick with x86 to prevent platform-specific bugs to confuse debugging ml even further.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 23, 2015, 04:26:01 PM
The much better option would be to modify all exe helper Makefiles to use one central compiler path that is found in the Makefile.user ... symlinking stuff works, but I don't like the approach.

Yes, I know. I wrote this tutorial to get someone up and running without any source code tweaking. The Makefile shouldn't be looking into the home directory for anything other than the ARM compiler--though I think it should also be moved out of the home directory.

As there's really little reason to have the exe helpers as x64 (well, cr2hdr might be a bit faster (or not)) imho just stick with x86 to prevent platform-specific bugs to confuse debugging ml even further.

I was suggesting a rather ugly hack--I'll take it out. By the way, cr2hdr is one of the exe helpers that looks for the cross compiler in the home directory. Hopefully that will be changed when the 20-bit version of cr2hdr is merged. Then maybe we can have it use the 64-bit compiler if possible?

There is no end to this is there?
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Marsu42 on September 23, 2015, 06:23:35 PM
Hopefully that will be changed when the 20-bit version of cr2hdr is merged. Then maybe we can have it use the 64-bit compiler if possible?

The 20bit version doesn't compile with x64 either, and alex has stated that this is known and most likely "wontfix" because the code changes are not worth the possible benefits.

There is no end to this is there?

You probably spend a lot of time figuring out msys and mingw :-) ... but actually cygwin works quite nicely recently, it used to be much worse. Basically it's just the compiler paths for the module exe helpers and some other smallish stuff. Given the nearly stalled ml development, I don't expect any major ml build system changes soon that would break cygwin again.

And in any case, very few people will hack the source themselves and the need for "compile it yourself" is basically gone with the nightly builds. So the one real use for compiling ml is the integration of some branch or pull request you cannot wait for to be merged (like nss audio on 6d), and again I doubt many people will do this. The download counter for my former cygwin zip package was extremely low, go figure.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 23, 2015, 10:17:11 PM
Thanks for the info.

I started getting serious building ML when the EOSM platform was disabled from the nightly builds. I got on Windows when testers wanted Windows binaries of some of the exe helpers from dmilligan's ml_dng branch. What I would really like to have happen is to get the EOSM shutter-bug resolved because that directly affects me. If more users can build ML maybe someone will come up with a solution? Oh, and the EOSM hasn't been updated in the nightlies since April--not to complain because I'm getting quite an education diving into the source code. I'm actually happy that I was able to get MinGW working simply because it was a challenge and it looks like nobody else was able to do it. Sort of an ego booster.

I don't know how many people are going to follow this tutorial. From your low download counter on something that was much easier to follow I'd say even fewer will give this a try.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: garry23 on September 23, 2015, 10:33:18 PM
@dfort

I for one truly hope that this becomes a functioning toolset.

I played around with compiling in the cloud and created my own 'versions' of ML using this approach.

I personally believe this is the real value of being able to compile.

I haven't yet tried the Cygwin/MinGW-64 approach, but will do so once it is clearer how I can take the latest nightly, tweak or add to the code and then compile.

As I read the above, this isn't fully clear to me yet: and this is a criticism of me, not you :-)
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 23, 2015, 11:31:31 PM
Hi @garry23

I also played around with compiling in the cloud--seems that was the most popular method for non-developers to get into rolling their own ML builds. Too bad that it doesn't work for new users or maybe it does with a paid subscription.

If you created your own versions of ML in the cloud you should already know how to take the latest nightly, tweak and compile--it works pretty much the same way in Cygwin/MinGW-64 only better because you have more control over the development environment.

Here are a few tricks you can do without having to code.

Want to check out all the hoopla over lua?
Code: [Select]
hg clone -r lua https://bitbucket.org/hudson/magic-lantern
Got an EOSM and want to get something newer that the April version that is available in nightlies?
Code: [Select]
hg clone -r EOSM__working https://bitbucket.org/daniel_fort/magic-lantern
How about going bleeding edge and peeking into some cool DNG development work being done by David Milligan:
Code: [Select]
hg clone -r ml_dng https://bitbucket.org/dmilligan/magic-lantern
You don't really need much in the way of coding skills to contribute. My first merged pull request was simply replacing [DELETE] with [Trash to change]:
EOS-M Format Card - Trash toggles keep or delete ML - UI Clarification (https://bitbucket.org/hudson/magic-lantern/pull-requests/640/eos-m-format-card-trash-toggles-keep-or/diff) That was also my first bug fix: EOS-M can't TAP to switch keep ML when formatting (https://bitbucket.org/hudson/magic-lantern/issues/2078/eos-m-cant-tap-to-switch-keep-ml-when)

Is it fully clear yet? Maybe a little less "cloudy" ? :D
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: garry23 on September 24, 2015, 11:13:56 AM
@dfort

I think the question I had in my mind is, with Cygwin/MinGW-64 approach are you provided a tool to change code.

That is, can I access the ML code and the parsing etc is handled correctly?

One final thought is that if someone is using this approach: would they be able to video a demo for us non-coders who are trying to get into all this 'coding stuff'.

Cheers

Garry
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Marsu42 on September 24, 2015, 09:08:18 PM
One final thought is that if someone is using this approach: would they be able to video a demo for us non-coders who are trying to get into all this 'coding stuff'.

Imho the simplest way is to provide a bash shell script that people just would need to run after installing (or unzipping) cygwin - this would automatically download the latest ml code, copy a working Makefile user, compile everything and then throw the autoexec.bin & modules .mo into a folder. People would just have to look at this shell script to understand how it works.

You don't really need much in the way of coding skills to contribute.

Personally, I'd advise caution to prevent frustration. If you can code C and know the basics of distributed software development (i.e. Mercurial) then it's easy to get started.

But if you want to change non-trivial things a lot of learning about how the Canon fw and the ML code works is involved, and there are always things that simply don't work as you expect because ML depends on what Canon provides. And for trivial changes, in the future you can just use user scripting with lua and use ML's lua api which has the sandbox advantage of not being able to brick the camera.

Last not least, real embedded hardware hacking is another matter altogether and far beyond my skills - an example would be the "new sound system" branch.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 25, 2015, 12:33:43 AM
Imho the simplest way is to provide a bash shell script that people just would need to run after installing (or unzipping) cygwin - this would automatically download the latest ml code, copy a working Makefile user, compile everything and then throw the autoexec.bin & modules .mo into a folder. People would just have to look at this shell script to understand how it works.

You mean something like this?

Code: [Select]
#!/bin/sh

# Uncomment the appropriate line
# to build to your preferred
# Magic Lantern camera platform
#
# Uncomment means to simply delete the
# '#" sign from the beginning of the line
# DO THIS FOR ONLY ONE CAMERA AT A TIME!
#
#CAMERA=5D2.212
#CAMERA=5D3.113
#CAMERA=6D.116
#CAMERA=7D.203
#CAMERA=50D.109
#CAMERA=500D.111
#CAMERA=550D.109
#CAMERA=600D.102
#CAMERA=650D.104
#CAMERA=700D.114
#CAMERA=1100D.105
#CAMERA=EOSM.202

cd ~/magic-lantern
hg pull && hg update
cd platform/$CAMERA
make clean && make zip
mv *.zip ~/
make clean
cd ~


I didn't go as far as unzipping, moving the files to the card and making some toast and coffee--don't know how far to take it. I did test it and it works fine. I might be a little overzealous with the "make clean" though.

Personally, I'd advise caution to prevent frustration. If you can code C and know the basics of distributed software development (i.e. Mercurial) then it's easy to get started.

I knew nothing about any of this stuff when I started a few months ago. Well, I did take a BASIC and a Pascal programming class in the '80's but forgot most of it. I also tinkered around with the terminal in Linux and OS-X and made some very simple shell scripts but I never even wrote a "hello world" in C. What I did do was follow instructions on how to compile open source programs and found many needed some sort of little tweak besides ./configure make and make install.

But if you want to change non-trivial things a lot of learning about how the Canon fw and the ML code works is involved, and there are always things that simply don't work as you expect because ML depends on what Canon provides. And for trivial changes, in the future you can just use user scripting with lua and use ML's lua api which has the sandbox advantage of not being able to brick the camera.

Of course users don't have to wait until the lua pull request is merged. I have already used it and some debugging tools found in other branches to take a stab at killing the EOS-M shutter bug (https://bitbucket.org/hudson/magic-lantern/issues/1893/eos-m-shutter-bug). No luck but I haven't bricked a camera--yet.

By the way, you can brick a camera with a lua script. Take a closer look at dmilligan's code:

Code: [Select]
// !!!DANGER WILL ROBINSON!!!
//#define LUA_PROP_REQUEST_CHANGE

Last not least, real embedded hardware hacking is another matter altogether and far beyond my skills - an example would be the "new sound system" branch.

Wrestling with the Shutter-bug forced me to search for a new "stub" so I had to disassembe my ROM1.BIN (link (http://www.magiclantern.fm/forum/index.php?topic=12177.msg117735#msg117735)) and wow--how did they figure this stuff out? If I ever run into g3gg0 or a1ex I may just fall to my knees and scream, "I'm not worthy!"
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: DeafEyeJedi on September 25, 2015, 05:28:26 AM
I hear you, @dfort!  ;)
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Marsu42 on September 25, 2015, 10:25:07 AM
You mean something like this?

Yeah, something like this ... here's the end of my current "just works" compile script. I see no sense to use make zip locally as I instantly want to use the files, thus I copy the stuff over a (recent) official nightly version in case I missed some files and then copy everything to the card (if it's available). You'd need to add that part to yours and use a variable for my hardcoded 6d.116. The "copy all modules" line should be of special interest, took me a while to get that shell line right :-p

Code: [Select]
mkdir -p ../dist/ML/modules
rm -f ../dist/ML/modules/*.mo
mv platform/6D.116/autoexec.bin ../dist/
mv platform/6D.116/6D_116.sym ../dist/ML/modules/
cp modules/*/*.mo ../dist/ML/modules
cp modules/dual_iso/*.exe ../dist/ML/
cp modules/raw_rec/*.exe ../dist/ML/
cp modules/mlv_rec/*.exe ../dist/ML/
cp modules/io_crypt/*.exe ../dist/ML/
mkdir -p ../dist/ML/fonts
cp data/fonts/* ../dist/ML/fonts

By the way, you can brick a camera with a lua script. Take a closer look at dmilligan's code

Ugh, I didn't realize and thought there were some additional safeguards with lua. So note: When in doubt, use new ML code in a C mode :-> ... beats having a heart attack every time your camera not turning itself on (mostly b/c the battery is empty) :-o

If I ever run into g3gg0 or a1ex I may just fall to my knees and scream, "I'm not worthy!"

These two basically carry the ml development, and finding devs with this ability and time on their hands is the reason why there's always the danger of the trunk getting stalled. Lucky us ml is working ok right now and essential features (like dual_iso & raw video) are production ready.

As for frustration with development: You might be lucky and and everything works as you want to, but there's always the possibility the Canon fw doesn't play ball (for example b/c a button doesn't send an event code) and you end up with endless sessions of trial & error, swapping the card between computer and camera. And sometimes, you have to realize it won't work at all properly - like reading the exposure from EC in M mode, w/o direct hardware access it'll be always dodgy.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 26, 2015, 07:40:11 PM
...You'd need to add that part to yours...

Like I said, there's no end to this. I was thinking of copying over the files to the user's desktop so there's no need to even look at the Cygwin home directory. On your script you might think about adding ML/scripts especially if/when lua is merged and ML/doc if you have your docs working.

BTW--I never looked into io_crypt, didn't know ML could do that. I've got an EOS-M so I can't play around with it, I also don't shoot subjects that would require encryption in case I get captured and tortured to show my photos. (Usually it is the other way around.)

Ugh, I didn't realize and thought there were some additional safeguards with lua.

Actually dmilligan has that line commented out so it isn't possible to change properties on a normal lua build. I posted an EOSM lua build on the shutter-bug issue page and struggled with the decision whether or not to turn on that option. To fix the shutter-bug you probably need the ability to change properties but if a user can't compile ML then turning on that option is giving him just enough rope to hang himself.

I may be straying a bit off topic but having the ability to compile your own custom version of ML with lua and turning on the hangman option is definitely on topic.

These two basically carry the ml development, and finding devs with this ability and time on their hands is the reason why there's always the danger of the trunk getting stalled. Lucky us ml is working ok right now and essential features (like dual_iso & raw video) are production ready.

Yes but at least with the platform that I'm on there has been some bug fixes that aren't getting pushed out to the users. That's one of the reasons why I started building my own ML versions. It turned out to be one of those, "when life hands you lemons make lemonade" situations.

And sometimes, you have to realize it won't work at all properly - like reading the exposure from EC in M mode, w/o direct hardware access it'll be always dodgy.

EC in M mode? Had to look look it up in my ML to English dictionary. The autoexposure module is something else I haven't played around with so I turned it on in my camera and whoa!

(https://farm1.staticflickr.com/775/21743241261_f2946c1265.jpg)

Don't know if I want to go down that rabbit hole at the moment.

In any case, compiling ML has opened up a whole new world for me and a deeper understanding of how the camera and Magic Lantern works. It also made me aware of just how little I know.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Marsu42 on September 26, 2015, 08:38:27 PM
I also don't shoot subjects that would require encryption in case I get captured and tortured to show my photos. (Usually it is the other way around.)

:-p ... The more probable real-life situation is that you shoot an event, and the organizer pressures you to "just give him your shots" which a pro photog would never do. If your files are encrypted, you can just say "be my guest" and state that you have no idea why his whatever-he's-using-raw-converter doesn't work :->


EC in M mode? Had to look look it up in my ML to English dictionary. The autoexposure module is something else I haven't played around with so I turned it on in my camera and whoa!

My (now private) auto_iso modules also contains ec in m, it's very useful if shoot in changing lighting conditions and still want to pre-set the exposure (esp. for flash or capturing motion) and aperture (depth of field). Nikon has it, and Canon graced their 1dx with this feature in a recent fw update, but other Canon cams are too dumb.

Anyway, it's just an example when something nice doesn't quite work due to the limited Canon fw - the EC indicator in M mode is just made for rough visual confirmation of over/underexposure, not an exact or fast meter you can use reliably by ML code ... we'd need direct hardware access to the metering system for that.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on September 27, 2015, 10:03:32 PM
The much better option would be to modify all exe helper Makefiles to use one central compiler path that is found in the Makefile.user ... symlinking stuff works, but I don't like the approach.

Made a pull request for that:

https://bitbucket.org/hudson/magic-lantern/pull-requests/659/eliminate-need-for-mingw-w32-bin-i686-w64/diff

Now comes the waiting game.

I didn't work on cross compiling dng2raw because I'm not really sure what it does and all my attempts at cross compiling it resulted in errors so I left it pretty much alone.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Marsu42 on September 29, 2015, 08:12:05 AM
Made a pull request for that

I approved it on bitbucket just for the enthusiasm of yours :-) ... but my advice from experience is to open a thread in the general dev forum and discuss promoting cygwin to a "supported" compilation environment. Imho there are good reasons to use cygwin on windows instead of a Linux vm, and it lowers the barrier for quick compilation and patching.

Do note: if the main devs don't really want this, your pull requests will have little luck and you're in for much frustration and a lifetime of cygwin patching as every new code addition will break cygwin again. Better get everyone to agree to take care of the little cygwin needs in the first place (other like for example throwing the arm gcc compiler in the home dir as the only option).

I didn't work on cross Now comes the waiting game.

Well, as you see this could take months atm so if you've got time on your hands make sure this pull requests covers full cywin/win-mingw support for everything (including make zip & make docs). I'd advise to use the cygwin version of the tool required (tex, mercurial, ...) to prevent script bugs because of different unix/win paths.

I didn't work on cross compiling dng2raw because I'm not really sure what it does and all my attempts at cross compiling it resulted in errors so I left it pretty much alone.

I didn't look at that, but that's the very reason for the existence of Cygwin (as msys doesn't have a recent compiler for the emulation environment): some sources need *nix POSIX functionality (like gettext) that Windows simply lacks. Some things can be done with little patches like an additional include line or a more precise var type, but some code would need to be completely rewritten so it's simpler just to use the emulation and be done with it.
Title: Compiling Magic Lantern with Cygwin/MinGW 64
Post by: DimaAffovaPax on March 21, 2016, 07:54:14 PM
I was wondering if I can keep working on my .wxs filelike the graphics interface, which is created when I was using wxwidgets with c/c, in wxpython then?
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on March 22, 2016, 01:40:41 AM
I was wondering if I can keep working on my .wxs filelike the graphics interface, which is created when I was using wxwidgets with c/c, in wxpython then?

If you are using a Cygwin environment then it should work otherwise you might try MSYS2/MinGW-w64 for a Windows native development environment (http://magiclantern.fm/forum/index.php?topic=15845.msg154356#msg154356)--or switch to Linux if possible.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dmilligan on March 22, 2016, 03:47:07 AM
I'm pretty sure that's a spambot
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: vstrglv on November 22, 2016, 07:54:32 AM
I am trying to compile with
 "hg clone -r unified https://bitbucket.org/hudson/magic-lantern
cd magic-lantern/platform/5D3.113/
make clean && make zip",
but there is an error for raw_rec.mo:
Code: [Select]
[ GCC      ]   raw2dng
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lm
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc_s.dll.a when searching for -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc_s.dll.a when searching for -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc_s.dll.a when searching for -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc_s.dll.a when searching for -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lcygwin
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libadvapi32.a when searching for -ladvapi32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libadvapi32.a when searching for -ladvapi32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libadvapi32.a when searching for -ladvapi32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -ladvapi32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libshell32.a when searching for -lshell32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libshell32.a when searching for -lshell32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libshell32.a when searching for -lshell32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lshell32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libuser32.a when searching for -luser32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libuser32.a when searching for -luser32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libuser32.a when searching for -luser32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -luser32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libkernel32.a when searching for -lkernel32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libkernel32.a when searching for -lkernel32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/w32api/libkernel32.a when searching for -lkernel32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lkernel32
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc_s.dll.a when searching for -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc_s.dll.a when searching for -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc_s.dll.a when searching for -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc_s.dll.a when searching for -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lgcc_s
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0//libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/libgcc.a when searching for -lgcc
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lgcc
collect2: error: ld returned 1 exit status
Makefile:16: recipe for target 'raw2dng' failed
make[4]: *** [raw2dng] Error 1

********************************************************
WARNING: module raw_rec failed to build, deleting
********************************************************
What is wrong?
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on November 22, 2016, 09:17:20 AM
That's odd, it is building raw2dng.exe. As I recall that doesn't build properly with a 64-bit compiler.

I can't reproduce the error because my PC is a very slow and old 32-bit laptop. Here are some things you can try.

Go into the modules/lv_rec directory and open up the file named Makefile with a text editor. Find this line:

Code: [Select]
all:: raw2dng
and comment it out like this:

Code: [Select]
# all:: raw2dng
That doesn't solve the issue with being able to build raw2dng with a 64-bit compiler. For that you can look for this block of code:

Code: [Select]
raw2dng.exe: FORCE
$(call build,MINGW,$(MINGW_GCC) -c $(SRC_DIR)/chdk-dng.c -m32 -mno-ms-bitfields -O2 -Wall -I$(SRC_DIR))
$(call build,MINGW,$(MINGW_GCC) -c raw2dng.c -m32 -mno-ms-bitfields -O2 -Wall -I$(SRC_DIR) -D_FILE_OFFSET_BITS=64)
$(call build,MINGW,$(MINGW_GCC) raw2dng.o chdk-dng.o -o raw2dng.exe -lm -m32)

and change that second "call build" line to include the -std=c99 switch:

Code: [Select]
$(call build,MINGW,$(MINGW_GCC) -c raw2dng.c -m32 -mno-ms-bitfields -O2 -Wall -I$(SRC_DIR) -D_FILE_OFFSET_BITS=64 -std=c99)
If neither of these works you can delete your 64-bit Cygwin installation and install a 32-bit version in its place.

Please post which "fix" worked for you.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: a1ex on November 22, 2016, 09:35:03 AM
Searching for (parts of) the error message is a good idea. Just be careful with dashes, e.g. search for:
Code: [Select]
x86_64-pc-cygwin/bin/ld: "cannot find -lgcc"

Without quotes, -lgcc in the search string means "return pages that do not contain lgcc".

http://stackoverflow.com/questions/30119573/compile-32bit-code-from-cygwin64
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: vstrglv on November 22, 2016, 05:33:18 PM
Thank you for reply dfort and alex. dfort, i have tried your fixes, but it does not work. No raw2dng.exe, no raw_rec.mo. May be i have to install cygwin32.
Now commented # all:: raw2dng in modules/raw_rec and raw_rec.mo appeared!
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on November 22, 2016, 08:11:57 PM
I don't have a 64-bit PC to test out these issues. On the tutorial I included this:

Quote
...What if you have a 64-bit system?
Note that most module exe helpers are targeted for x86 code, i.e. even if you run cygwin x64 (which you should on a x64 Windows) you need to use the i686 mingw toolchain.
Please leave feedback if you get mingw64-x86-64-gcc-core working and on which command line tools--it is going to need some tweaks to the ML source code.

That might be a little confusing but basically it means that if you are going to compile the command line tools, like raw2dng.exe, it has been tested to work with the 32-bit compiler (i686) but not with the 64-bit compiler (x64).

I wasn't aware that building a platform automatically builds raw2dng but now that I see that does. It was the same with cr2hdr until just recently with this commit (https://bitbucket.org/hudson/magic-lantern/commits/21ac498913cfd789ddca3bf852d5832b1544e3fa?at=unified). The other popular command line tool, mlv_dump, isn't built when a platform is compiled so I thought that was preferred method. Build rules for raw2dng are in both raw_rec and lv_rec so perhaps we should look into that.

Of course this doesn't resolve another issue which is that raw2dng.exe can't be built with a 64-bit compiler. I'm not sure if this is true for all 64-bit compilers but it is a problem when I cross compile a Windows binary on a Mac:

Code: [Select]
[ MINGW    ]   raw2dng.exe
raw2dng.c: In function 'find_and_fix_cold_pixels':
raw2dng.c:1213:9: error: 'for' loop initial declarations are only allowed in C99 or C11 mode
         for (int y = 0; y < h; y++)
         ^
raw2dng.c:1213:9: note: use option -std=c99, -std=gnu99, -std=c11 or -std=gnu11 to compile your code
raw2dng.c:1215:13: error: 'for' loop initial declarations are only allowed in C99 or C11 mode
             for (int x = 0; x < w; x++)
             ^
raw2dng.c:1233:5: error: 'for' loop initial declarations are only allowed in C99 or C11 mode
     for (int p = 0; p < cold_pixels; p++)
     ^
raw2dng.c:1243:9: error: 'for' loop initial declarations are only allowed in C99 or C11 mode
         for (int i = -4; i <= 4; i++)
         ^
raw2dng.c:1245:13: error: 'for' loop initial declarations are only allowed in C99 or C11 mode
             for (int j = -4; j <= 4; j++)
             ^
make: *** [raw2dng.exe] Error 1

Simply adding the "-std=c99" switch worked for me but it should be tested on other 64-bit systems.

Thank you for reply dfort and alex. dfort, i have tried your fixes, but it does not work. No raw2dng.exe, no raw_rec.mo. May be i have to install cygwin32.
Now commented # all:: raw2dng in modules/raw_rec and raw_rec.mo appeared!

I'm a bit confused with that comment. So commenting out the "all" rule worked for you? This should build the raw_rec module but not raw2dng.exe.

What was it that didn't work, adding the "-std=c99" switch?

@alex - Which compiler are you using on Linux and are you cross compiling Windows binaries? Would adding the "-std=c99" switch to the raw2dng make rules have any negative consequences? It seems to work fine with the Windows binaries that I have compiled on my Mac.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: vstrglv on November 22, 2016, 08:28:55 PM
So commenting out the "all" rule worked for you? This should build the raw_rec module but not raw2dng.exe.

What was it that didn't work, adding the "-std=c99" switch?


Yes, if i comment "all", raw_rec module  is built but not raw2dng.exe
If add "-std=c99" switch, raw_rec module  is not built.
But it is possible to compile raw2dng.exe by "make raw2dng.exe" from modules/raw_rec folder. The same for cr2hdr.exe from modules/dual_iso
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: a1ex on November 22, 2016, 08:29:37 PM
@alex - Which compiler are you using on Linux and are you cross compiling Windows binaries?

The one from Makefile. The nightly build server compiles cr2hdr.exe and mlv_dump.exe, but raw2dng binaries weren't updated lately (maybe I should set up a job for it as well).

Quote
Would adding the "-std=c99" switch to the raw2dng make rules have any negative consequences?

Nope.

BTW, gcc 5.x uses -std=gnu11 (https://gcc.gnu.org/gcc-5/changes.html) by default, whereas gcc 4.x uses -std=gnu89.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on November 22, 2016, 09:27:45 PM
Ok--looks like @vstrglv is able to compile ML so I'll test some Makefile changes that should keep these issues from coming up again.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: vstrglv on December 06, 2016, 04:20:38 PM
Quote
Ok--looks like @vstrglv is able to compile ML so I'll test some Makefile changes that should keep these issues from coming up again.
Thank you. Now this error has gone.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on May 27, 2017, 02:06:11 AM
A while back a user came up with a problem: Unable to build ML with Cygwin 32 (http://www.magiclantern.fm/forum/index.php?topic=19117.msg180793#msg180793). I'm now helping another user having problems with this tutorial so I decided to delete my Cygwin installation and rebuild it by following the tutorial. The installation seemed to go fine but I got the same compile errors that the other user reported:

Code: [Select]
make[1]: Leaving directory '/home/Dan/magic-lantern/tcc'
[ CC       ]   module.o
[ AR       ]   strrchr.o
[ AR       ]   dietlibc.a
[ AR       ]   lib_a-setjmp.o
[ AR       ]   newlib-libc.a
[ CP       ]   newlib-libm.a
[ CP       ]   gcc-libgcc.a
[ LD       ]   magiclantern
module.o: In function `module_load_symbols.constprop.2':
module.c:(.text+0x428): undefined reference to `tcc_add_symbol'
module.o: In function `_module_load_all':
module.c:(.text+0x498): undefined reference to `tcc_new'
module.c:(.text+0x4a4): undefined reference to `tcc_set_options'
module.c:(.text+0x4c8): undefined reference to `tcc_delete'
module.c:(.text+0x504): undefined reference to `tcc_delete'
module.c:(.text+0x720): undefined reference to `tcc_delete'
module.c:(.text+0x75c): undefined reference to `tcc_add_file'
module.c:(.text+0x7bc): undefined reference to `tcc_relocate'
module.c:(.text+0x7e8): undefined reference to `tcc_relocate'
module.c:(.text+0x850): undefined reference to `tcc_delete'
module.c:(.text+0x8bc): undefined reference to `tcc_get_symbol'
module.c:(.text+0x8e4): undefined reference to `tcc_get_symbol'
module.c:(.text+0x90c): undefined reference to `tcc_get_symbol'
module.c:(.text+0x934): undefined reference to `tcc_get_symbol'
module.c:(.text+0x95c): undefined reference to `tcc_get_symbol'
module.o:module.c:(.text+0xc4c): more undefined references to `tcc_get_symbol' follow
module.o: In function `_module_load_all':
module.c:(.text+0xcac): undefined reference to `tcc_delete'
module.o: In function `module_load_task':
module.c:(.text+0xf7c): undefined reference to `tcc_load_offline_section'
module.o: In function `module_load':
module.c:(.text+0x1040): undefined reference to `tcc_new'
module.c:(.text+0x104c): undefined reference to `tcc_set_options'
module.c:(.text+0x1070): undefined reference to `tcc_delete'
module.c:(.text+0x1084): undefined reference to `tcc_add_file'
module.c:(.text+0x1098): undefined reference to `tcc_relocate'
module.c:(.text+0x10a8): undefined reference to `tcc_delete'
module.o: In function `module_get_symbol':
module.c:(.text+0x10c8): undefined reference to `tcc_get_symbol'
module.o: In function `module_exec':
module.c:(.text+0x10f0): undefined reference to `tcc_get_symbol'
module.o: In function `module_unload':
module.c:(.text+0x126c): undefined reference to `tcc_delete'
make: *** [../../src/Makefile.src:195: magiclantern] Error 1

I'm at a loss to figure out what is going on here and how to fix it. Just before deleting my previous Cygwin installation I downloaded the latest ML and compiled just fine. Of course I no longer have the previous installation--ugh!

[EDIT]

For what it is worth turning off the module configuration allows compilation to complete but of course no modules are built.

Makefile.user.default
Code: [Select]
CONFIG_MODULES      = n
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: a1ex on May 27, 2017, 09:46:33 AM
Full log?

Best guess: I'm using a trick to strip most TCC symbols - everything but the API we use - in tcc/Makefile near localsyms. This file should contain all TCC's symbols except those listed in the error message. Some of the tools used to create this file might not be available on Windows.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on May 27, 2017, 07:33:48 PM
Full log (https://pastebin.com/r1Fata9R)

I'm guessing that between September 2015 when I wrote this tutorial and March 2017 when this issue was reported something changed in Cygwin. Maybe one of the tools used in tcc/Makefile? Long shot, grep was updated to 3.0.

[EDIT] It looks like the "symbols"command is missing in Cygwin while OSX has it. I never heard of that command.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: a1ex on May 27, 2017, 08:05:40 PM
Try:
Code: [Select]
cd tcc   # from magic-lantern root dir
make clean
make
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on May 27, 2017, 08:12:33 PM
Results:

Code: [Select]
Dan@Laptop-PC ~/magic-lantern/tcc
$ make
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3.exe -o libtcc.o -c libtcc.c -DTCC_TARGET_ARM -DWITHOUT_LIBTCC -I.  -Os -fno-strict-aliasing -Wno-pointer-sign -Wno-sign-compare -Wno-unused-result
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3.exe -o tccpp.o -c tccpp.c -DTCC_TARGET_ARM -DWITHOUT_LIBTCC -I.  -Os -fno-strict-aliasing -Wno-pointer-sign -Wno-sign-compare -Wno-unused-result
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3.exe -o tccgen.o -c tccgen.c -DTCC_TARGET_ARM -DWITHOUT_LIBTCC -I.  -Os -fno-strict-aliasing -Wno-pointer-sign -Wno-sign-compare -Wno-unused-result
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3.exe -o tccelf.o -c tccelf.c -DTCC_TARGET_ARM -DWITHOUT_LIBTCC -I.  -Os -fno-strict-aliasing -Wno-pointer-sign -Wno-sign-compare -Wno-unused-result
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3.exe -o tccasm.o -c tccasm.c -DTCC_TARGET_ARM -DWITHOUT_LIBTCC -I.  -Os -fno-strict-aliasing -Wno-pointer-sign -Wno-sign-compare -Wno-unused-result
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3.exe -o tccrun.o -c tccrun.c -DTCC_TARGET_ARM -DWITHOUT_LIBTCC -I.  -Os -fno-strict-aliasing -Wno-pointer-sign -Wno-sign-compare -Wno-unused-result
tccrun.c: In function 'tcc_relocate_ex':
tccrun.c:190:18: warning: #warning FIXME: why does it overflow without this extra RAM when loading the big adtg_gui? [-Wcpp]
                 #warning FIXME: why does it overflow without this extra RAM when loading the big adtg_gui?
                  ^
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3.exe -o arm-gen.o -c arm-gen.c -DTCC_TARGET_ARM -DWITHOUT_LIBTCC -I.  -Os -fno-strict-aliasing -Wno-pointer-sign -Wno-sign-compare -Wno-unused-result
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-ld -r -o libtcctmp.o libtcc.o tccpp.o tccgen.o tccelf.o tccasm.o tccrun.o arm-gen.o
#~ ~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-objcopy libtcctmp.a libtcc.a --localize-symbols localsyms
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-objcopy libtcctmp.o libtccx.o --localize-symbols localsyms
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: a1ex on May 27, 2017, 08:22:47 PM
Hm, no error here...

Code: [Select]
cat localsyms
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-readelf libtcctmp.o -Ws
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-readelf libtccx.o -Ws
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on May 28, 2017, 06:14:08 PM
Here are my results (https://pastebin.com/2kazAx4A) of those commands.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: a1ex on May 28, 2017, 06:31:14 PM
Okay, tcc_new and the other missing symbols should *not* be in localsyms, but for some reason, they are. You could try running the localsyms rule from Makefile manually (maybe step by step) to see where it fails. Outside Makefile, it would look somewhat like this:
Code: [Select]
readelf libtcctmp.o -Ws | awk "{print \$8}" | sort | uniq | grep -v tcc_new

The output should contain all other symbols, except tcc_new.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on May 29, 2017, 01:25:33 AM
Ran that command:

Code: [Select]
readelf libtcctmp.o -Ws | awk "{print \$8}" | sort | uniq | grep -v tcc_new
through each of these listed in the Makefile:

Code: [Select]
# I'm sure there's a cleanest solution, but... I just don't know how
localsyms: libtcctmp.o
@$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
| grep -v "^tcc_new$$" \
| grep -v "^tcc_delete$$" \
| grep -v "^tcc_add_file$$" \
| grep -v "^tcc_relocate$$" \
| grep -v "^tcc_get_symbol$$" \
| grep -v "^tcc_add_symbol$$" \
| grep -v "^tcc_set_options$$" \
| grep -v "^tcc_load_offline_section$$" \
> $@

In every case the one I ran did not appear in the list. Just to be sure I also tried the ARM version of readelf with the same results:

Code: [Select]
~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-readelf libtcctmp.o -Ws | awk "{print \$8}" | sort | uniq | grep -v tcc_new
I can see those tcc_* appearing in the Cygwin tcc/localsyms but are not in the Mac tcc/localsyms. So I take it that might be the problem? By the way running "make clean" will not remove the localsyms file though that probably isn't an issue.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: a1ex on May 29, 2017, 01:59:47 AM
The results of readelf ... grep commands were the same on both Mac and Cygwin?

As a workaround, you can manually delete these symbols from "localsyms", then run "make" again. The localsyms file will not be re-created (unless you also edit some TCC sources), but libtccx.o (which is the one used by ML) will be re-created using your modified localsyms.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on May 29, 2017, 02:33:44 AM
The results of readelf ... grep commands were the same on both Mac and Cygwin?

I didn't run the readelf ... grep commands on the Mac. All I did was compare the two localsyms files.

Found the problem and a solution!

The problem is with grep in the current Cygwin distribution. I haven't seen any reports on it but this must have been an issue for a while. Switching to another version will clear up the issue. At one point I was suspecting grep but struck it out because it seemed highly unlikely:

I'm guessing that between September 2015 when I wrote this tutorial and March 2017 when this issue was reported something changed in Cygwin. Maybe one of the tools used in tcc/Makefile? Long shot, grep was updated to 3.0.

Here's what users following this tutorial should do until grep is fixed in Cygwin. Re-run the installer, choose the "Up To Date" view option, search for "grep" then click on the circle with the arrows where it says "Keep" until you find the previous version of grep.

(https://c1.staticflickr.com/5/4225/34915788806_eb4c08c073_o.jpg)

Cygwin setup will replace grep and it should work. You might have to delete tcc/localsyms file. Note that after you successfully build ML at least once you can go back to that buggy version of grep and you'll be able to continue building ML because the localsyms file doesn't get overwritten with each build or even by running "make clean" this may or may not be a good thing. It drove me nuts while trying to get this Cygwin tutorial working again.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on May 29, 2017, 09:25:01 PM
Got some answers from the Cygwin mailing list. To sum it up, there's a change coming in grep version 3 that will also affect other platforms that affects the Windows platform. The fix for ML is to do this:
Code: [Select]
-    @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
+    @$(READELF) $< -Ws | tr -d '\r' | $(AWK) "{print \$$8}" | sort | uniq \

If you're interested in the nitty gritty details:

https://sourceware.org/ml/cygwin-announce/2017-02/msg00035.html

And the responses from the mailing list:
Code: [Select]
cygwin Digest 29 May 2017 17:14:45 -0000 Issue 10265
...
---------- Forwarded message ----------
From: Daniel Fort <dan@digiola.com>
To: cygwin@cygwin.com
Cc:
Bcc:
Date: Sun, 28 May 2017 23:34:36 -0700
Subject: grep-3.0-2 issues within Makefile
grep-3.0-2 binary will not function as expected when the -v option is
used in a Makefile.

Resolution - downgrade to grep-3.0-1.

When using Cygwin to build Magic Lantern users stated reporting a
build errors on new Cygwin installs around November 2016. The
resolution was to downgrade grep to the previous version.

The discussion and instructions on how to prepare Cygwin to compile
Magic Lantern are in this forum topic:

http://www.magiclantern.fm/forum/index.php?topic=15894.msg154435#msg154435

Using a Cygwin install that includes the default grep-3.0-1 will
result in errors when running the follow Makefile code:

localsyms: libtcctmp.o
    @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
        | grep -v "^tcc_new$$" \
        | grep -v "^tcc_delete$$" \
        | grep -v "^tcc_add_file$$" \
        | grep -v "^tcc_relocate$$" \
        | grep -v "^tcc_get_symbol$$" \
        | grep -v "^tcc_add_symbol$$" \
        | grep -v "^tcc_set_options$$" \
        | grep -v "^tcc_load_offline_section$$" \
        > $@


---------- Forwarded message ----------
From: Marco Atzeri <marco.atzeri@gmail.com>
To: cygwin@cygwin.com
Cc:
Bcc:
Date: Mon, 29 May 2017 10:52:11 +0200
Subject: Re: grep-3.0-2 issues within Makefile
On 29/05/2017 08:34, Daniel Fort wrote:
grep-3.0-2 binary will not function as expected when the -v option is
used in a Makefile.

Please note the last grep announcement
https://sourceware.org/ml/cygwin-announce/2017-02/msg00035.html

and the changes between text and binary mounts.

...
--

and what is the error ?

Regards
Marco


---------- Forwarded message ----------
From: Eric Blake <eblake@redhat.com>
To: cygwin@cygwin.com
Cc:
Bcc:
Date: Mon, 29 May 2017 06:39:53 -0500
Subject: Re: grep-3.0-2 issues within Makefile
On 05/29/2017 03:52 AM, Marco Atzeri wrote:
> On 29/05/2017 08:34, Daniel Fort wrote:
>> grep-3.0-2 binary will not function as expected when the -v option is
>> used in a Makefile.
>
> Please note the last grep announcement
> https://sourceware.org/ml/cygwin-announce/2017-02/msg00035.html
>
> and the changes between text and binary mounts.
>

>> Using a Cygwin install that includes the default grep-3.0-1 will
>> result in errors when running the follow Makefile code:
>>
>> localsyms: libtcctmp.o
>>     @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \

Most likely, $(READELF) is producing \r\n-terminated output. The
solution, then, is to rewrite the line to:

$(READELF) $< -Ws | tr -d '\r' | $(AWK) ...

>
> and what is the error ?

Most likely, grep is not filtering as expected, because now that it is
treating your data as binary rather than text, your explicit $ anchor is
only matching \n instead of \r\n.

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



---------- Forwarded message ----------
From: Eric Blake <eblake@redhat.com>
To: cygwin@cygwin.com
Cc:
Bcc:
Date: Mon, 29 May 2017 06:44:07 -0500
Subject: Re: grep-3.0-2 issues within Makefile
On 05/29/2017 06:39 AM, Eric Blake wrote:

>>> localsyms: libtcctmp.o
>>>     @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
>
> Most likely, $(READELF) is producing \r\n-terminated output.

That said, WHAT is $(READELF) actually expanding to? If it is the cygwin
binutils version, it should NOT be outputting \r\n in the first place.
Generally, you don't get \r\n output unless you are mixing non-cygwin
programs into the pipeline.

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



---------- Forwarded message ----------
From: Brian Inglis <Brian.Inglis@systematicsw.ab.ca>
To: cygwin@cygwin.com
Cc:
Bcc:
Date: Mon, 29 May 2017 09:11:02 -0600
Subject: Re: grep-3.0-2 issues within Makefile
On 2017-05-29 05:44, Eric Blake wrote:
> On 05/29/2017 06:39 AM, Eric Blake wrote:
>>>> localsyms: libtcctmp.o
>>>>     @$(READELF) $< -Ws | $(AWK) "{print \$$8}" | sort | uniq \
>> Most likely, $(READELF) is producing \r\n-terminated output.
> That said, WHAT is $(READELF) actually expanding to? If it is the cygwin
> binutils version, it should NOT be outputting \r\n in the first place.
> Generally, you don't get \r\n output unless you are mixing non-cygwin
> programs into the pipeline.

Cygwin .o files are not ELF format and not recognized as such by readelf

$ readelf src/astro/sofa/20160503_a/c/build/zr.o -Ws
readelf: Error: Not an ELF file - it has the wrong magic bytes at the start

or file

$ file src/astro/sofa/20160503_a/c/build/zr.o
src/astro/sofa/20160503_a/c/build/zr.o: data

running Cygwin readelf on cross builds which produce ELF .o don't
generate "\r":

$ file util/*.o
util/ntp-keygen.o:      ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info
util/ntp-keygen-opts.o: ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info
util/ntptime.o:         ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info
util/tickadj.o:         ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info
util/version.o:         ELF 32-bit LSB relocatable, ARM, EABI5 version 1
(SYSV), not stripped, with debug_info

$ readelf util/*.o -Ws | wc -lwcL
    460    3479   26678      84
$ readelf util/*.o -Ws | grep $'\r' | wc -lwcL
      0       0       0       0

and it's not the Cygwin mingw binutils

$ /usr/bin/x86_64-w64-mingw32-readelf util/*.o -Ws | grep $'\r' | wc -lwcL
      0       0       0       0

and just to prove this detects "\r"

$ echo $'\r' | grep $'\r' | wc -lwcL
      1       0       2       0

so culprit must be native Mingw binutils readelf.

--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada


I need to make sure this is working on previous versions of grep and other platforms but it looks like it is a rather benign change.

[EDIT] This issue should only affect the Windows platform and is caused by the Windows line endings created by the pre-compiled ARM toolchain. Mixing non-Cygwin packages in a Cygwin environment is not recommended but the only other option is to build the ARM toolchain from scratch--unless the Linux binary works in the Cygwin environment? Never mind, that didn't work.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on May 30, 2017, 06:35:48 PM
So it looks like some of these issues are Déjà vu.

Removing localsyms was something we did for the lua module:
https://bitbucket.org/hudson/magic-lantern/pull-requests/692/remove-localsyms-in-modules-lua/diff

And we needed to make some tweaks for Magic Lantern to compile on Windows:
https://bitbucket.org/hudson/magic-lantern/pull-requests/660/cr2hdr-build-rules/diff
https://bitbucket.org/hudson/magic-lantern/pull-requests/655/lv_rec-use-fseeko-fseeko64-depending-on/diff
https://bitbucket.org/hudson/magic-lantern/pull-requests/658/build_tools-for-windows/diff

As far as the best way to "fix" the issue of mixing a Windows native ARM toolchain in a Cygwin environment--I've been trying to compile the ARM toolchain in Cygwin but haven't quite got it yet. The easier method would be to just deal with the Windows line endings in the Makefile in a way that doesn't affect other platforms which is something that we already have figured out.

Finally, suggestions for how to improve one line of code keeps coming. This one is from Brian Inglis on the Cygwin mailing list.
Code: [Select]
I've found moving functionality into awk runs faster than creating one
additional pipe and another process, so to avoid creating multiple
additional pipes and processes, you could change your awk script to do
all the filtering (with appropriate line breaks allowed and quoting
required by your make environment inserted), saving 8 pipes and greps
(some of which could also be saved by a single egrep (grep -E) using "|"
or basic grep using "\|") e.g. (all one-liners)

egrep -v
'/^tcc_(new|delete|add_file|relocate|get_symbol|add_symbol|set_options|load_offline_section)$/'

OR

grep -Ev
'/^tcc_(new|delete|add_file|relocate|get_symbol|add_symbol|set_options|load_offline_section)$/'

OR

grep -v
'/^tcc_\(new\|delete\|add_file\|relocate\|get_symbol\|add_symbol\|set_options\|load_offline_section\)$/'

OR awk

$8 ~ /\r$/ { sub( /\r$/, "", $8)}; $8 !~
/^$|^tcc_(new|delete|add_file|relocate|get_symbol|add_symbol|set_options|load_offline_section)$/
{print $8}

with a bit more work, it can generate a unique list, using associative
array indices, saving 1 pipe and uniq, and a lot of input to and work
for sort (some of which could also be saved using "sort -u" instead of
uniq):

$8 ~ /\r$/ { sub( /\r$/, "", $8)}; $8 !~
/^$|^tcc_(new|delete|add_file|relocate|get_symbol|add_symbol|set_options|load_offline_section)$/)
{++a[$8]}; END {for (i in a) print i}

and if you are using gawk, also sort the indices, giving a sorted array
with the original indices as values, having sequential numeric indices,
saving 1 pipe and uniq (which could also be saved using "sort -u"):

$8 ~ /\r$/ { sub( /\r$/, "", $8)}; $8 !~
/^$|^tcc_(new|delete|add_file|relocate|get_symbol|add_symbol|set_options|load_offline_section)$/)
{++a[$8]}; END {n = asorti(a); for (i = 1; i <= n; ++i) print a[i]}

HTH
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: a1ex on May 30, 2017, 07:33:42 PM
Very nice. To me, the second version looks the most readable. The awk versions are a lot more complex than ARM assembler :P

And regarding execution time, on a 9-year old laptop:
Code: [Select]
time make localsyms
real 0m0.040s
user 0m0.004s
sys 0m0.008s

so I don't see a valid reason for optimizing it (it's only called once in the build process).
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 01, 2017, 04:33:48 AM
Made a pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/832/tcc-makefile-tweaks/diff) that will fix the Cygwin grep 3.0 issue with precompiled Windows ARM toolchain. Also added some other tweaks to the tcc Makefile. Tested on Cygwin and Mac development environments.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: garry23 on June 01, 2017, 07:16:04 AM
@dfort

Just to say I really appreciate your efforts here.

As soon as I am able to give it some time, and know things are 'fixed', I will take the compile plunge  ;)

Cheers

Garry
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 01, 2017, 03:30:19 PM
@garry23

Things are fixed! a1ex merged the changes into the unified branch so you should merge unified from the main repository into your working branches.

I just ran through the tutorial and everything worked as expected. From bare metal to a fully functional ML build took less than an hour on a very slow 10 year old laptop.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 01, 2017, 07:16:52 PM
I ran through the tutorial on a much faster 64-bit Windows 7 desktop system and went from zero to finished ML build in 15 minutes. Also learned a few things along the way like it is better to setup a 32-bit Cygwin installation even though the computer has a 64-bit processor. Why? Because the command line tools like mlv_dump won't build properly with the 64-bit gcc used in Cygwin. That's only important if you want to build Cygwin native applications, you really should be using MinGW-64 to build Windows native applications.

Updated the tutorial so Windows 64-bit users should have a better experience.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 01, 2017, 11:16:56 PM
Tried compiling the command line tools. mlv_dump, raw2dng, built fine but there's a problem with cr2hdr:

Code: [Select]
$ make cr2hdr.exe
Updated HGVERSION
[ README   ]   module_strings.h
Help page 1: Quick start
Help page 2: Tips and tricks
CROSS=1 make cr2hdr.exe
make[1]: Entering directory '/home/dfort/magic-lantern/modules/dual_iso'
[ i686-w64-mingw32-gcc ]   cr2hdr.exe
cr2hdr.c:544:20: error: conflicting types for ‘module_get_string’
 static const char* module_get_string(const char* name)
                    ^
In file included from cr2hdr.c:56:0:
../../src/module.h:268:13: note: previous declaration of ‘module_get_string’ was here
 const char* module_get_string(int mod_number, const char* name);
             ^
make[1]: *** [Makefile.cr2hdr:18: cr2hdr.exe] Error 1
make[1]: Leaving directory '/home/dfort/magic-lantern/modules/dual_iso'
make: *** [Makefile.cr2hdr:21: cr2hdr.exe] Error 2
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Danne on June 01, 2017, 11:34:02 PM
Seems a problem on mac too. If you erase the conflicting string
Code: [Select]
const char* module_get_string(int mod_number, const char* name); from module.h cr2hdr will compile. Not the solution but some baby step direction  :P

Here´s some more info.
https://stackoverflow.com/questions/5691650/conflicting-types-error-when-compiling-c-program-using-gcc


You could rename module_get_string(two places) in cr2hdr.c to something like  my_module_get_string and get rid of the conflict as well.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 02, 2017, 12:18:30 AM
Interesting -- this must be affecting Linux also. Can anyone verify?
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Danne on June 02, 2017, 03:04:22 PM
I noticed another error in crop_rec_4k branch when compiling cr2hdr. Complexity of bleeding edge?

Code: [Select]
[ gcc-5    ]   cr2hdr
cr2hdr.c:544:20: error: conflicting types for 'module_get_string'
 static const char* module_get_string(const char* name)
                    ^
In file included from cr2hdr.c:56:0:
../../src/module.h:274:13: note: previous declaration of 'module_get_string' was here
 const char* module_get_string(int mod_number, const char* name);
             ^
cr2hdr.c: In function 'module_get_string':
cr2hdr.c:546:34: error: '__module_strings_MODULE_NAME' undeclared (first use in this function)
     module_strpair_t *strings = &__module_strings_MODULE_NAME[0];
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 02, 2017, 03:30:03 PM
Created bug report: cr2hdr fails to build (https://bitbucket.org/hudson/magic-lantern/issues/2727/cr2hdr-fails-to-build)
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Danne on June 02, 2017, 03:42:44 PM
https://bitbucket.org/hudson/magic-lantern/commits/37426e9f8f9f5c6e1bcd69b7b2f7f85eafa6f456?at=crop_rec_4k
 :P
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 02, 2017, 09:31:16 PM
Thanks -- works now. Should get that change into the unified branch.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Danne on June 02, 2017, 09:35:13 PM
Not the same changes exactly but that minor conflicting string nedds to be fixed in unified. Glad it works on my mac now. Thanks g3gg0.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 03, 2017, 03:31:35 PM
Thanks g3gg0!

Fixed (https://bitbucket.org/hudson/magic-lantern/commits/4126b560c22b9a7e577e0b8862e29adb4e44f1ef).
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Danne on June 25, 2017, 05:51:19 PM
Had some nice progress wth building a cygwin environment inside a windows 10 os set up in virtual box on my mac. The first 4 commands in first post was all it took to get compiling working.
Then I pulled down some branches I work with on my mac and then got the errors recognized here.
http://www.magiclantern.fm/forum/index.php?topic=17099.msg165820#msg165820

Code: [Select]
Updated HGVERSION
[ README   ]   module_strings.h
Traceback (most recent call last):
  File "../readme2modulestrings.py", line 113, in <module>
    txt = run('cat README.rst | grep -v -E "^:([^:])+:.+$" | ' + rst2htmlCommand + ' | python ../html2text.py -b 700 | sed "s/\r$//"')
TypeError: cannot concatenate 'str' and 'NoneType' objects
../Makefile.modules:39: recipe for target 'module_strings.h' failed
make: *** [module_strings.h] Error 1

Solution usually is to symlink python like following but this failed
for f in /usr/bin/*.py; do ln -s $f "${f/.py/}"; done

Reading further down the forum post link above user Escho install python3. This is also what worked with cygwin. Adding the packages for python3 solved the issues compiling mlv_dump and also cr2hdr on windows 10.

All in all a great tutorial dfort.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Danne on June 27, 2017, 04:46:22 PM
Been digging around cygwin so gonna share my latest findings.

FIrst script should change python2-docutils to python3-docutils.
Code: [Select]
%UserProfile%\Downloads\setup-x86.exe -s http://cygwin.mirrors.pair.com -q -P mingw64-i686-gcc-core,gcc-core,make,mercurial,python3-docutils,perl,wget,unzip,zip,diffutils
This won´t be needed anymore
Code: [Select]
for f in /usr/bin/*.py; do ln -s $f "${f/.py/}"; done

I was a bit sloppy building mlv_dump without the extension .exe which made mlv_dump dependent on cygwin1.dll. After a few reinstallations I actually noticed dfort wrote about this at the bottom of the first post, hmm.

Alway do e.g make mlv_dump.exe or make cr2hdr.exe, not make mlv_dump
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 27, 2017, 04:56:00 PM
If you stay within the Cygwin environment you'll never know.

Best to always compile using make mlv_dump.exe because that binary will run both inside and outside of Cygwin.

If anyone wants a real challenge try getting QEMU running in Cygwin.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 27, 2017, 10:00:46 PM
Whoa--this is really weird.

Traced down the problem to what seems to be an issue with Cygwin's docutils. This affects both python2 and 3.

This is with 0.13.1-1 of python2-docutils

Code: [Select]
$ ls -la /usr/bin/*.py
-rwxr-xr-x 1 Dan None  6488 Mar 14 21:30 /usr/bin/msgfmt.py
-rwxr-xr-x 1 Dan None 22097 Mar 14 21:30 /usr/bin/pygettext.py
-rwxr-xr-x 1 Dan None   594 Mar 13 23:12 /usr/bin/rst2html.py
-rwxr-xr-x 1 Dan None  1140 Mar 13 23:12 /usr/bin/rst2html5.py
-rwxr-xr-x 1 Dan None   791 Mar 13 23:12 /usr/bin/rst2latex.py
-rwxr-xr-x 1 Dan None   600 Mar 13 23:12 /usr/bin/rst2man.py
-rwxr-xr-x 1 Dan None   764 Mar 13 23:12 /usr/bin/rst2odt.py
-rwxr-xr-x 1 Dan None  1698 Mar 13 23:12 /usr/bin/rst2odt_prepstyles.py
-rwxr-xr-x 1 Dan None   601 Mar 13 23:12 /usr/bin/rst2pseudoxml.py
-rwxr-xr-x 1 Dan None   637 Mar 13 23:12 /usr/bin/rst2s5.py
-rwxr-xr-x 1 Dan None   871 Mar 13 23:12 /usr/bin/rst2xetex.py
-rwxr-xr-x 1 Dan None   602 Mar 13 23:12 /usr/bin/rst2xml.py
-rwxr-xr-x 1 Dan None   670 Mar 13 23:12 /usr/bin/rstpep2html.py
-rwxr-xr-x 1 Dan None 18545 Mar 14 21:29 /usr/bin/smtpd.py

And this is with the current 0.13.1-2

Code: [Select]
Dan@Laptop-PC /usr/bin
$ ls -la /usr/bin/*.py
-rwxr-xr-x 1 Dan None  6488 Mar 14 21:30 /usr/bin/msgfmt.py
-rwxr-xr-x 1 Dan None 22097 Mar 14 21:30 /usr/bin/pygettext.py
-rwxr-xr-x 1 Dan None 18545 Mar 14 21:29 /usr/bin/smtpd.py

Where did rst2html go? No wonder my symlink trick stopped working. You can still go back one version on the Cygwin installer but the default is to upgrade you to the latest.

(https://c1.staticflickr.com/5/4218/35575068925_a9c356c25b_z.jpg)
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Danne on June 27, 2017, 10:13:06 PM
Leaves us with both cygwin and crosscompiler mingw-64 broken. At least cygwin let's you compile mlv_dump :)
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 27, 2017, 10:40:11 PM
...crosscompiler mingw-64 broken. At least cygwin let's you compile mlv_dump :)

mingw64-i686-gcc isn't broken, at least not in Cygwin. There's a problem on the Mac with the Homebrew package of mingw64-i686-gcc that I'm using in the tutorial.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on June 27, 2017, 11:02:03 PM
The fix for now is to launch the Cygwin installer, find python2-binutils, revert to 0.13.1-1 then run my one-liner:

Code: [Select]
for f in /usr/bin/*.py; do ln -s $f "${f/.py/}"; done
That should get you back up and running.

I had some other issues trying to work this out because I had the Python for Windows installed and Cygwin would try to run the rst2html script from that package and it kept failing.

Sent a message to the Cygwin mailing list to see if there is a reason those docutils scripts vanished.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on July 23, 2017, 02:58:39 PM
It looks like python2-binutils won't be fixed anytime soon. Turns out that python2-binutils and python3-binutils use the same scripts and for some reason they can't be in both packages at the same time.

python2-binutils 0.13.1-1 works
python3-binutils 0.13.1-1 broken

python2-binutils 0.13.1-2 broken
python3-binutils 0.13.1-2 works

Long story short -- I switched the installation so it uses python3-binutils. I didn't do this before because I didn't want to move to python3 but as it turns out Cygwin defaults to using python2 with the way we're installing the ML development environment. Go figure.

Code: [Select]
$ python --version
Python 2.7.13

So the installation on the first post is working again. If you start with a clean slate just delete C:\cygwin and run the tutorial again.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Danne on July 23, 2017, 03:10:47 PM
Another tip coming from Bouncyball is to strip mlv_dump.exe from unneeded stuff.
make mlv_dump.exe && strip mlv_dump.exe

Worked with MinGW-64 but only after I replaced mac strip with the one from MinGW-64 into the xcode version here:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip

If not wanting to replace strip following should work when running make on mlv_dump:
make mlv_dump.exe && /usr/local/Cellar/mingw-w64/5.0.2_1/toolchain-i686/i686-w64-mingw32/bin/strip mlv_dump.exe



Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on July 24, 2017, 12:09:21 AM
Those paths are for Mac. Haven't tried it on Cygwin yet but it should work.


Sent from my iPad using Tapatalk
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Danne on July 24, 2017, 12:31:38 AM
Oops. Thought MinGW-64 was for mac only...
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on July 24, 2017, 02:44:21 AM
MinGW is Minimalist GNU for Windows and it is available for Mac, Linux and Windows development environments though the binaries will most likely only run on Windows.

I tried that trick on Cygwin and it works. Either version of strip seems to do the trick:

Code: [Select]
strip mlv_dump.exe
Code: [Select]
i686-w64-mingw32-strip mlv_dump.exe
Before: 631,091 mlv_dump.exe
After:  154,112 mlv_dump.exe
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: Audionut on July 25, 2017, 02:55:28 AM
Thanks dfort.  Nice and easy.

Now just to work my head around SourceTree.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on July 30, 2017, 02:01:33 AM
Just ran into an issue where if the user name has a space in it the cmd script won't work. The fix was to quote the %UserProfile%:

Code: [Select]
"%UserProfile%"
Go figure. Fixed it in the tutorial.

[EDIT] Sorry, there are also issues with the build scripts so even though you can setup a Cygwin development environment on a Windows system that has spaces in the user name, ML won't build.

Code: [Select]
make[1]: Leaving directory '/home/new user/magic-lantern/tcc'
...
cp: target 'gcc-libgcc.a' is not a directory
make: *** [../../src/Makefile.src:320: gcc-libgcc.a] Error 1

Best advice for now is not to put spaces in your Windows user name.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: chris_overseas on October 01, 2017, 09:42:30 AM
It looks like Visual Studio now has support for building using MinGW-64 and Cygwin:

https://blogs.msdn.microsoft.com/vcblog/2017/07/19/using-mingw-and-cygwin-with-visual-cpp-and-open-folder/
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on October 02, 2017, 07:21:45 PM
Looks like Microsoft is opening up to open source. On Windows 10 it is possible to install Ubuntu.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: theBilalFakhouri on February 10, 2018, 03:07:06 PM
Hey @dfort
I have followed all the steps, but when I am trying to compile a module this what happening in ML Menu:
(https://preview.ibb.co/iLaWi7/VRAM1.png) (https://ibb.co/iwScO7)
What should I do or update to get API 7?
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on February 10, 2018, 06:38:01 PM
You compiled only the module? Some modules are sensitive to which branch and/or changeset you're using.

Compile ML then go back and compile the module -- though it looks like you want to compile the dual_iso module which is already in the Makefile.modules.default list so it will automatically get built when you compile ML. If you want a module that isn't on the default list you can also add it in a file named Makefile.modules.user like this:

magic-lantern/modules/Makefile.modules.user
Code: [Select]
ML_MODULES_DYNAMIC += adtg_gui

BTW--the adtg_gui needs some other options turned on so you should check the link to that quote if you're interested in that one.

[EDIT] After posting this I saw another problem:

Code: [Select]
dual_i~1.mo
That shouldn't happen. You must have already had a previous version of the dual_iso module on your card. Delete everything on your card before copying over the new build.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: theBilalFakhouri on February 10, 2018, 07:01:08 PM
[EDIT] After posting this I saw another problem:
Code: [Select]
dual_i~1.mo
No, I renamed the original dual_iso so I don't miss it.
but sorry, how can I compile ML build?
and how I can know from which branch I am compiling ML?
If you can give some useful links and tips I will be appreciate it so much!
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on February 10, 2018, 07:46:47 PM
Assuming you have the repository in your home directory and it is named magic-lantern:

Code: [Select]
# Change into your home directory
cd
# Now go to the platform you want to compile
cd magic-lantern/platform/700D.115
# Which branch do you want?
hg branches
# Lots of branches to choose from!
# You'll probably see a ":" at the bottom, press the spacebar to keep scrolling or "q" to get out of that.
q
# Let's say you want to see which branch you are on:
hg branch
# How about switching to the crop_rec_4k branch
hg update crop_rec_4k
# Let's make sure you've got the latest changes
hg pull
hg update
# Check that it builds cleanly
make
# All good? Let's zip up a "Nightly" build
make zip
# copy it someplace that is easy to find like your home directory
cp *.zip ~/.
# clean up the mess
make clean

Of course there is a lot more but that should get you started.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: theBilalFakhouri on February 10, 2018, 08:43:00 PM
Thanks, it was really helpful!
Code: [Select]
$ make
Using ~/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gcc (preferred).
[ VERSION  ]   ../../platform/700D.115/version.bin
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/700D.115/stubs.o
[ VERSION  ]   ../../platform/700D.115/version.c
[ CC       ]   version.o
[ CC       ]   backtrace.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate.o
In file included from ../../src/bitrate.c:5:0:
../../src/bitrate.c: In function 'time_indicator_show':
../../src/bitrate.c:374:81: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         int fnts = FONT(FONT_SMALL, COLOR_WHITE, mvr_config.actual_qscale_maybe == -16 ? COLOR_RED : COLOR_BLACK);
                                                                                 ^
../../src/bmp.h:174:13: note: in definition of macro 'FONT'
         | ((bg) & 0xFF) << 8 \
             ^
../../src/bitrate.c:379:44: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
             mvr_config.actual_qscale_maybe < 0 ? "-" : "+",
                                            ^
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ CC       ]   menuindex.o
[ CC       ]   af_patterns.o
[ CC       ]   focus.o
[ CC       ]   notify_box.o
[ CC       ]   bootflags.o
[ CC       ]   dialog_test.o
[ CC       ]   vram.o
[ CC       ]   greenscreen.o
[ CC       ]   fps-engio.o
../../src/fps-engio.c: In function 'get_current_shutter_reciprocal_x1000':
../../src/fps-engio.c:533:6: warning: #warning FIXME: consider defining FRAME_SHUTTER_BLANKING_READ [-Wcpp]
     #warning FIXME: consider defining FRAME_SHUTTER_BLANKING_READ
      ^
../../src/fps-engio.c: At top level:
../../src/fps-engio.c:316:12: warning: 'fps_timer_b_method' defined but not used [-Wunused-variable]
 static int fps_timer_b_method = 0;
            ^
[ CC       ]   shoot.o
[ CC       ]   hdr.o
[ CC       ]   lv-img-engio.o
[ CC       ]   state-object.o
[ CC       ]   tasks.o
[ CC       ]   vsync-lite.o
[ CC       ]   tskmon.o
../../src/tskmon.c: In function 'null_pointer_check':
../../src/tskmon.c:242:66: warning: signed and unsigned type in conditional expression [-Wsign-compare]
             int id = tskmon_last_task ? tskmon_last_task->taskId : -1;
                                                                  ^
[ CC       ]   battery.o
[ CC       ]   imgconv.o
[ CC       ]   histogram.o
[ CC       ]   falsecolor.o
[ CC       ]   audio-ak.o
[ CC       ]   zebra.o
[ CC       ]   vectorscope.o
[ CC       ]   beep.o
[ CC       ]   crop-mode-hack.o
[ CC       ]   ph_info_disp.o
[ CC       ]   flexinfo.o
[ CC       ]   screenshot.o
[ CC       ]   fileprefix.o
[ CC       ]   lvinfo.o
[ CC       ]   builtin-enforcing.o
[ CC       ]   powersave.o
[ CC       ]   ml-cbr.o
[ CC       ]   raw.o
[ CC       ]   chdk-dng.o
../../src/chdk-dng.c:209:16: warning: 'cam_apex_shutter' defined but not used [-Wunused-variable]
 static int32_t cam_apex_shutter[2]          = { 0, 96 };            // Shutter speed in APEX units
                ^
../../src/chdk-dng.c:210:16: warning: 'cam_apex_aperture' defined but not used [-Wunused-variable]
 static int32_t cam_apex_aperture[2]         = { 0, 96 };            // Aperture in APEX units
                ^
[ CC       ]   edmac-memcpy.o
../../src/edmac-memcpy.c: In function 'edmac_memcpy_init':
../../src/edmac-memcpy.c:91:57: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
     static int AbortEDmac_check __attribute__((used)) = &AbortEDmac;
                                                         ^
[ CC       ]   patch.o
[ CC       ]   console.o
[ CC       ]   tcc-glue.o
make -C ../../tcc
make[1]: Entering directory '/home/Bilal Fakhouri/magic-lantern/tcc'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/Bilal Fakhouri/magic-lantern/tcc'
[ CC       ]   module.o
[ AR       ]   strrchr.o
[ AR       ]   dietlibc.a
[ AR       ]   lib_a-setjmp.o
[ AR       ]   newlib-libc.a
[ CP       ]   newlib-libm.a
make: *** No rule to make target 'fakhouri/gcc-arm-none-eabi-5_4-2016q3/bin/../lib/gcc/arm-none-eabi/5.4.1/libgcc.a', needed by 'gcc-libgcc.a'.  Stop.

Bilal Fakhouri@WORKS ~/magic-lantern/platform/700D.115
$ make zip
Using ~/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gcc (preferred).
../../Makefile.inc:81: removing ../../platform/*/magiclantern.sym
[ RM dir   ]   /home/Bilal Fakhouri/magic-lantern/platform/700D.115/zip/
[ RM       ]   ../../platform/*/magiclantern.sym
mkdir -p /home/Bilal Fakhouri/magic-lantern/platform/700D.115/zip
[ MKDIR    ]   ML directory structure...
[ VERSION  ]   ../../platform/700D.115/version.c
[ CC       ]   version.o
make -C ../../tcc
make[1]: Entering directory '/home/Bilal Fakhouri/magic-lantern/tcc'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/Bilal Fakhouri/magic-lantern/tcc'
make: *** No rule to make target 'fakhouri/gcc-arm-none-eabi-5_4-2016q3/bin/../lib/gcc/arm-none-eabi/5.4.1/libgcc.a', needed by 'gcc-libgcc.a'.  Stop.

Bilal Fakhouri@WORKS ~/magic-lantern/platform/700D.115
$
Is that correct? or it didn't compile?
If yes, I lost here "cp *.zip ~/." Should I change  something here like the name instaed of "*" or add directory(Example please).
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on February 11, 2018, 12:46:14 AM
It didn't compile properly.

Code: [Select]
# Go back to your magic-lantern directory
# ~ is a shortcut for your home directory
cd ~/magic-lantern
# Clean it up
make clean

Now try it again. You might have switched branches without running "make clean" or maybe you made changes without committing them and switched branches. You can discard your changes when switching branches like this:

Code: [Select]
hg up -C unified
Note that up is a shortcut for "update" there are lots of shortcuts.

If you find yourself in a big mess just delete your local repository and re-clone it.
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: theBilalFakhouri on February 11, 2018, 01:28:45 AM
I tried, it didn't work. I can compile the modules separately  or the mlv_dump with no problem.
Any Idea why & how to fix it?
Can give me what should I get in the terminal please?
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: dfort on February 11, 2018, 07:11:44 AM
I think I found the problem:

Code: [Select]
Bilal Fakhouri@WORKS ~/magic-lantern/platform/700D.115
There's a space in your username. That will throw it off:

Code: [Select]
make: *** No rule to make target 'fakhouri/...
Easiest "fix" is to make another user without spaces in the username and reinstall Cygwin for that user. You might be able to get away with just renaming the Cygwin directory but that probably won't work. According to the FAQ you need to use the mkpasswd command.

https://cygwin.com/faq.html#faq.setup.name-with-space

Odd that everything else works. Maybe the tcc makefile could be modified so it will work like the others?
Title: Re: Compiling Magic Lantern with Cygwin/MinGW-64
Post by: theBilalFakhouri on February 11, 2018, 09:38:41 AM
Thanks@dfort! it worked.
I thought the username's space issue was solved.