Compiling Magic Lantern with Cygwin/MinGW-64

Started by dfort, September 22, 2015, 02:04:07 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

garry23

@dfort

Just to say I really appreciate your efforts here.

As soon as I am able to give it some time, and know things are 'fixed', I will take the compile plunge  ;)

Cheers

Garry

dfort

@garry23

Things are fixed! a1ex merged the changes into the unified branch so you should merge unified from the main repository into your working branches.

I just ran through the tutorial and everything worked as expected. From bare metal to a fully functional ML build took less than an hour on a very slow 10 year old laptop.

dfort

I ran through the tutorial on a much faster 64-bit Windows 7 desktop system and went from zero to finished ML build in 15 minutes. Also learned a few things along the way like it is better to setup a 32-bit Cygwin installation even though the computer has a 64-bit processor. Why? Because the command line tools like mlv_dump won't build properly with the 64-bit gcc used in Cygwin. That's only important if you want to build Cygwin native applications, you really should be using MinGW-64 to build Windows native applications.

Updated the tutorial so Windows 64-bit users should have a better experience.

dfort

Tried compiling the command line tools. mlv_dump, raw2dng, built fine but there's a problem with cr2hdr:

$ make cr2hdr.exe
Updated HGVERSION
[ README   ]   module_strings.h
Help page 1: Quick start
Help page 2: Tips and tricks
CROSS=1 make cr2hdr.exe
make[1]: Entering directory '/home/dfort/magic-lantern/modules/dual_iso'
[ i686-w64-mingw32-gcc ]   cr2hdr.exe
cr2hdr.c:544:20: error: conflicting types for 'module_get_string'
static const char* module_get_string(const char* name)
                    ^
In file included from cr2hdr.c:56:0:
../../src/module.h:268:13: note: previous declaration of 'module_get_string' was here
const char* module_get_string(int mod_number, const char* name);
             ^
make[1]: *** [Makefile.cr2hdr:18: cr2hdr.exe] Error 1
make[1]: Leaving directory '/home/dfort/magic-lantern/modules/dual_iso'
make: *** [Makefile.cr2hdr:21: cr2hdr.exe] Error 2

Danne

Seems a problem on mac too. If you erase the conflicting string const char* module_get_string(int mod_number, const char* name); from module.h cr2hdr will compile. Not the solution but some baby step direction  :P

Here´s some more info.
https://stackoverflow.com/questions/5691650/conflicting-types-error-when-compiling-c-program-using-gcc


You could rename module_get_string(two places) in cr2hdr.c to something like  my_module_get_string and get rid of the conflict as well.

dfort

Interesting -- this must be affecting Linux also. Can anyone verify?

Danne

I noticed another error in crop_rec_4k branch when compiling cr2hdr. Complexity of bleeding edge?

[ gcc-5    ]   cr2hdr
cr2hdr.c:544:20: error: conflicting types for 'module_get_string'
static const char* module_get_string(const char* name)
                    ^
In file included from cr2hdr.c:56:0:
../../src/module.h:274:13: note: previous declaration of 'module_get_string' was here
const char* module_get_string(int mod_number, const char* name);
             ^
cr2hdr.c: In function 'module_get_string':
cr2hdr.c:546:34: error: '__module_strings_MODULE_NAME' undeclared (first use in this function)
     module_strpair_t *strings = &__module_strings_MODULE_NAME[0];




dfort

Thanks -- works now. Should get that change into the unified branch.

Danne

Not the same changes exactly but that minor conflicting string nedds to be fixed in unified. Glad it works on my mac now. Thanks g3gg0.


Danne

Had some nice progress wth building a cygwin environment inside a windows 10 os set up in virtual box on my mac. The first 4 commands in first post was all it took to get compiling working.
Then I pulled down some branches I work with on my mac and then got the errors recognized here.
http://www.magiclantern.fm/forum/index.php?topic=17099.msg165820#msg165820

Updated HGVERSION
[ README   ]   module_strings.h
Traceback (most recent call last):
  File "../readme2modulestrings.py", line 113, in <module>
    txt = run('cat README.rst | grep -v -E "^:([^:])+:.+$" | ' + rst2htmlCommand + ' | python ../html2text.py -b 700 | sed "s/\r$//"')
TypeError: cannot concatenate 'str' and 'NoneType' objects
../Makefile.modules:39: recipe for target 'module_strings.h' failed
make: *** [module_strings.h] Error 1


Solution usually is to symlink python like following but this failed
for f in /usr/bin/*.py; do ln -s $f "${f/.py/}"; done

Reading further down the forum post link above user Escho install python3. This is also what worked with cygwin. Adding the packages for python3 solved the issues compiling mlv_dump and also cr2hdr on windows 10.

All in all a great tutorial dfort.

Danne

Been digging around cygwin so gonna share my latest findings.

FIrst script should change python2-docutils to python3-docutils.

%UserProfile%\Downloads\setup-x86.exe -s http://cygwin.mirrors.pair.com -q -P mingw64-i686-gcc-core,gcc-core,make,mercurial,python3-docutils,perl,wget,unzip,zip,diffutils


This won´t be needed anymore
for f in /usr/bin/*.py; do ln -s $f "${f/.py/}"; done


I was a bit sloppy building mlv_dump without the extension .exe which made mlv_dump dependent on cygwin1.dll. After a few reinstallations I actually noticed dfort wrote about this at the bottom of the first post, hmm.

Alway do e.g make mlv_dump.exe or make cr2hdr.exe, not make mlv_dump

dfort

If you stay within the Cygwin environment you'll never know.

Best to always compile using make mlv_dump.exe because that binary will run both inside and outside of Cygwin.

If anyone wants a real challenge try getting QEMU running in Cygwin.

dfort

Whoa--this is really weird.

Traced down the problem to what seems to be an issue with Cygwin's docutils. This affects both python2 and 3.

This is with 0.13.1-1 of python2-docutils

$ ls -la /usr/bin/*.py
-rwxr-xr-x 1 Dan None  6488 Mar 14 21:30 /usr/bin/msgfmt.py
-rwxr-xr-x 1 Dan None 22097 Mar 14 21:30 /usr/bin/pygettext.py
-rwxr-xr-x 1 Dan None   594 Mar 13 23:12 /usr/bin/rst2html.py
-rwxr-xr-x 1 Dan None  1140 Mar 13 23:12 /usr/bin/rst2html5.py
-rwxr-xr-x 1 Dan None   791 Mar 13 23:12 /usr/bin/rst2latex.py
-rwxr-xr-x 1 Dan None   600 Mar 13 23:12 /usr/bin/rst2man.py
-rwxr-xr-x 1 Dan None   764 Mar 13 23:12 /usr/bin/rst2odt.py
-rwxr-xr-x 1 Dan None  1698 Mar 13 23:12 /usr/bin/rst2odt_prepstyles.py
-rwxr-xr-x 1 Dan None   601 Mar 13 23:12 /usr/bin/rst2pseudoxml.py
-rwxr-xr-x 1 Dan None   637 Mar 13 23:12 /usr/bin/rst2s5.py
-rwxr-xr-x 1 Dan None   871 Mar 13 23:12 /usr/bin/rst2xetex.py
-rwxr-xr-x 1 Dan None   602 Mar 13 23:12 /usr/bin/rst2xml.py
-rwxr-xr-x 1 Dan None   670 Mar 13 23:12 /usr/bin/rstpep2html.py
-rwxr-xr-x 1 Dan None 18545 Mar 14 21:29 /usr/bin/smtpd.py


And this is with the current 0.13.1-2

Dan@Laptop-PC /usr/bin
$ ls -la /usr/bin/*.py
-rwxr-xr-x 1 Dan None  6488 Mar 14 21:30 /usr/bin/msgfmt.py
-rwxr-xr-x 1 Dan None 22097 Mar 14 21:30 /usr/bin/pygettext.py
-rwxr-xr-x 1 Dan None 18545 Mar 14 21:29 /usr/bin/smtpd.py


Where did rst2html go? No wonder my symlink trick stopped working. You can still go back one version on the Cygwin installer but the default is to upgrade you to the latest.



Danne

Leaves us with both cygwin and crosscompiler mingw-64 broken. At least cygwin let's you compile mlv_dump :)

dfort

Quote from: Danne on June 27, 2017, 10:13:06 PM
...crosscompiler mingw-64 broken. At least cygwin let's you compile mlv_dump :)

mingw64-i686-gcc isn't broken, at least not in Cygwin. There's a problem on the Mac with the Homebrew package of mingw64-i686-gcc that I'm using in the tutorial.

dfort

The fix for now is to launch the Cygwin installer, find python2-binutils, revert to 0.13.1-1 then run my one-liner:

for f in /usr/bin/*.py; do ln -s $f "${f/.py/}"; done

That should get you back up and running.

I had some other issues trying to work this out because I had the Python for Windows installed and Cygwin would try to run the rst2html script from that package and it kept failing.

Sent a message to the Cygwin mailing list to see if there is a reason those docutils scripts vanished.

dfort

It looks like python2-binutils won't be fixed anytime soon. Turns out that python2-binutils and python3-binutils use the same scripts and for some reason they can't be in both packages at the same time.

python2-binutils 0.13.1-1 works
python3-binutils 0.13.1-1 broken

python2-binutils 0.13.1-2 broken
python3-binutils 0.13.1-2 works

Long story short -- I switched the installation so it uses python3-binutils. I didn't do this before because I didn't want to move to python3 but as it turns out Cygwin defaults to using python2 with the way we're installing the ML development environment. Go figure.

$ python --version
Python 2.7.13


So the installation on the first post is working again. If you start with a clean slate just delete C:\cygwin and run the tutorial again.

Danne

Another tip coming from Bouncyball is to strip mlv_dump.exe from unneeded stuff.
make mlv_dump.exe && strip mlv_dump.exe

Worked with MinGW-64 but only after I replaced mac strip with the one from MinGW-64 into the xcode version here:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip

If not wanting to replace strip following should work when running make on mlv_dump:
make mlv_dump.exe && /usr/local/Cellar/mingw-w64/5.0.2_1/toolchain-i686/i686-w64-mingw32/bin/strip mlv_dump.exe




dfort

Those paths are for Mac. Haven't tried it on Cygwin yet but it should work.


Sent from my iPad using Tapatalk

Danne

Oops. Thought MinGW-64 was for mac only...

dfort

MinGW is Minimalist GNU for Windows and it is available for Mac, Linux and Windows development environments though the binaries will most likely only run on Windows.

I tried that trick on Cygwin and it works. Either version of strip seems to do the trick:

strip mlv_dump.exe
i686-w64-mingw32-strip mlv_dump.exe

Before: 631,091 mlv_dump.exe
After:  154,112 mlv_dump.exe

Audionut

Thanks dfort.  Nice and easy.

Now just to work my head around SourceTree.