Author Topic: Compiling Magic Lantern in the Cloud  (Read 43386 times)

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3126
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #25 on: April 10, 2015, 12:50:42 PM »
The 'last' question I have now is, how do I change the source, say in lens.c, ie to create a personal tweak of lens.c and compile it in the cloud..

Simply edit the file, save it, then run make again. Codio has a builtin code editor, all your files should appear on the left side bar, click on one and it should open as a tab in the codio editor. Alternatively you can use vim or emacs (these are command line based text editors) from the terminal (e.g. vim filename.c).

Licaon_Kter

  • Senior
  • ****
  • Posts: 494
  • M
Re: Compiling Magic Lantern in the Cloud
« Reply #26 on: April 10, 2015, 01:41:49 PM »

Another thing I noticed is that in the module folder there are no modules, only .c files.
Did you compile the modules by running make all_modules too?


My compile command line is:
Code: [Select]
make clean && make && make all_modules && cd platform/EOSM.202 && make zip

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #27 on: April 10, 2015, 02:46:16 PM »
@dmilligan and Licaon_Kter

Thanks to both of you for the education. I can see I'm going to have fun this week end.

I think I read earlier that the download from Codio has 'everything' in it 

Thus what I will need to do is trawl through the download and compare with a normal ML download, to ensure I match the right files that are required.

Any other pointers for a Codio virgin :-)

Cheers

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3126
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #28 on: April 10, 2015, 05:19:15 PM »
Thus what I will need to do is trawl through the download and compare with a normal ML download, to ensure I match the right files that are required.
The download will have the entire contents of your codio workspace. Look for the zip file inside that corresponds to the zip you generated with ML (should be in the platform folder for your specific camera).

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3366
  • I have a 1Dx, I'm awesome.
Re: Compiling Magic Lantern in the Cloud
« Reply #29 on: April 10, 2015, 07:57:49 PM »
My compile command line is:
Code: [Select]
make clean && make && make all_modules && cd platform/EOSM.202 && make zip

Make zip in the camera platform directory should be all that's needed.  Make clean is good practice though.

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #30 on: April 10, 2015, 09:23:07 PM »
@All

I have made progress but get compile errors, eg modules not compiled, and the resultant zip downloaded is obviously incomplete.

I have used the following terminal commands in codio:

wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern

make clean && make && make all_modules && cd platform/magic-lantern && make zip

Can anyone see my 'mistake'?

surami

  • Member
  • ***
  • Posts: 103
Re: Compiling Magic Lantern in the Cloud
« Reply #31 on: April 10, 2015, 09:33:17 PM »
I wanted to compile today a fresh unified build on the Codio, but I got the same errors as garry23 wrote, eg.:
Code: [Select]
Building module mlv_rec...                                                                                                                                                                                                                 
Updated HGVERSION                                                                                                                                                                                                                         
[ README   ]   module_strings.h                                                                                                                                                                                                           
cat README.rst | grep -v -E "^:([^:])+:.+$" | rst2html --no-xml-declaration | python ../html2text.py -b 700                                                     
                                                                                                                                                                                                                                           
/bin/sh: 1: rst2html: not found                                                                                                                                                                                                           
grep: write error: Broken pipe                                                                                                                                                                                                             
                                                                                                                                                                                                                                           
(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x2b27e9af0f38>)                                                                                                                                                     
make[4]: *** [module_strings.h] Error 1                                                                                                                                                                                                   
                                                                                                                                                                                                                                           
********************************************************                                                                                                                                                                                   
WARNING: module mlv_rec failed to build, deleting                                                                                                                                                                                         
********************************************************

I used the same line from the OP:
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash
Code: [Select]
cd magic-lantern/platform/550D.109/
Code: [Select]
make zip
550D + nightly ML

surami

  • Member
  • ***
  • Posts: 103
Re: Compiling Magic Lantern in the Cloud
« Reply #32 on: April 10, 2015, 10:33:09 PM »
@garry23: Make a new project and if you are at install software step, where you have to install zip, install the python3 too.

Somebody knows, how to solve the gcc 32bit issue? (dual_iso, dot_tune, etc. modules couldn't be compiled)
550D + nightly ML

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #33 on: April 10, 2015, 11:06:57 PM »
@surami

Tried that but still get the error that every module fails.

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #34 on: April 10, 2015, 11:17:24 PM »
Postscript

Could someone who has successfully used this approach simply lay out the exact terminal commands they used.

Cheers

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #35 on: April 11, 2015, 03:11:12 AM »
PPS

I have continue to try all advice, eg Audionut's.

Here are my terminal commands in codio

wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern/platform/5D3.113/

make clean && make zip

And what follows is module compile errors and the process fails.

I think I'll give up until some expert hints at where I'm going wrong :-)


dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3126
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #36 on: April 11, 2015, 06:20:47 AM »
It doesn't actually fully fail, you should at least have some stuff built. It's just a couple of modules that fail to build (mainly just dual_iso I think), and the reason is that they are trying to build the desktop tools (cr2hdr) and most ML desktop tools require 32 bit compilation, while these VM s are 64bit and don't have 32bit gcc libraries available.

I have a PR in the queue that will fix the 64 bit compilation issue.

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #37 on: April 11, 2015, 02:33:51 PM »
@dmilligan

Thanks for the update.

Can you confirm my command line inputs above are the correct ones to use.

And if they are, I simplt ignore all the errors and download the zip.

Cheers

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #38 on: April 11, 2015, 03:36:48 PM »
@dmilligan

I can report every module fails, eg

/bin/sh: 1: rst2html: not found                                                                                                                   
grep: write error: Broken pipe                                                                                                                   
                                                                                                                                                 
(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x2b10a697dfc8>)                                                           
make[4]: *** [module_strings.h] Error 1                                                                                                           
                                                                                                                                                 
********************************************************                                                                                         
WARNING: module mlv_snd failed to build, deleting                                                                                                 
********************************************************           

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #39 on: April 11, 2015, 05:06:01 PM »
Another post from me as I'm trying to provide feedback to help developers.

I have tried once again, and now get this error sett:
Welcome to the Codio Terminal!                                                                                                                 
*                                                                                                                                                 
* README: https://codio.com/s/docs/boxes                                                                                                         
*                                                                                                                                                 
* Your Codio Box domain: design-jazz.codio.io[:1024-9999]                                                                                         
* External ssh connection: ssh:[email protected]:2140                                                                                       
*                                                                                                                                                 
[email protected]:~/workspace$ wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash                                         
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                                                   
                                 Dload  Upload   Total   Spent    Left  Speed                                                                     
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0                                                                   
100 57.5M  100 57.5M    0     0  4184k      0  0:00:14  0:00:14 --:--:-- 5168k                                                                   
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                                                   
                                 Dload  Upload   Total   Spent    Left  Speed                                                                     
100   551  100   551    0     0   5027      0 --:--:-- --:--:-- --:--:-- 12244                                                                   
                                                                                                                                                 
gzip: docutils-0.12.tar.gz: not in gzip format                                                                                                   
tar: docutils-0.12.tar: Cannot open: No such file or directory                                                                                   
tar: Error is not recoverable: exiting now                                                                                                       
bash: line 9: cd: docutils-0.12: No such file or directory                                                                                       
python: can't open file 'setup.py': [Errno 2] No such file or directory                                                                           
cp: cannot stat `.local/bin/rst2html.py': No such file or directory                                                                               
destination directory: magic-lantern                                                                                                             
adding changesets                                                                                                                                 
adding manifests                                                                                                                                 
adding file changes                                                                                                                               
added 11013 changesets with 24585 changes to 2672 files                                                                                           
updating to branch unified                                                                                                                       
1554 files updated, 0 files merged, 0 files removed, 0 files unresolved                                                                           
[email protected]:~/workspace$   

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3126
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #40 on: April 11, 2015, 09:36:29 PM »
Looks like docutils failed to download correctly. Maybe the server was down or the url changed. I'll check on it.

BTW, it will be easier to read stuff like console output on the forum if you put it inside code tags. Also prevents posts from getting too long, and makes it more clear the difference between what is output and what is just you talking.

Licaon_Kter

  • Senior
  • ****
  • Posts: 494
  • M
Re: Compiling Magic Lantern in the Cloud
« Reply #41 on: April 12, 2015, 01:32:46 AM »
To skip docs build (errors) edit /magic-lantern/Makefile

Code: [Select]

./Makefile-88-# We must build the docs first to use fresh doc/menuindex.txt
./Makefile:89:# during 'make all'. We can't write 'zip: all docs' because
./Makefile-90-# of possible problem in case of parallel build.
./Makefile-91-# (see make's '-j' option documentation)
./Makefile:92:zip: docs
./Makefile-93-  $(MAKE) all
./Makefile:94:  cd $(PLATFORM_PATH)/all; $(MAKE) zip
./Makefile-95-
And on line 92 delete the word docs

To skip 32-bit apps build, edit /magic-lantern/modules/dual_iso/Makefile
Code: [Select]
./Makefile-6-MODULE_OBJS=dual_iso.o
./Makefile-7-
./Makefile-8-# include modules environment
./Makefile-9-include $(TOP_DIR)/modules/Makefile.modules
./Makefile-10-
./Makefile:11:# include cr2hdr makefile rules
./Makefile:12:include Makefile.cr2hdr
and remove line 12


dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3126
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #42 on: April 12, 2015, 02:51:30 AM »
You still need docutils installed to compile the modules, even if you skip 'docs' build (it's used to automatically generate header files that are compiled into modules).

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3126
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #43 on: April 12, 2015, 03:03:09 AM »
@garry23

I don't know why your docutils download failed, it seems to be working now when I try it, so maybe it was just a fluke. Try running these commands:
Code: [Select]
cd
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #44 on: April 12, 2015, 03:47:50 AM »
@dmilligan

Thanks.

The issue is that I just can't get a full zip.

When I use the following I get errors.

wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern

make clean

cd platform/5D3.113/

make zip

Any ideas?



Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3366
  • I have a 1Dx, I'm awesome.
Re: Compiling Magic Lantern in the Cloud
« Reply #45 on: April 12, 2015, 05:30:21 AM »
What errors?

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #46 on: April 12, 2015, 05:39:55 AM »
@Audionut

As I have said above I use the following

Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern/platform/5D3.113/

make clean && make zip

When I look in the download .zip on the platform folder there is just one module.

Every module has an error like this:

Code: [Select]
                                                                                                                                                 
Building module raw_rec...                                                                                                                       
Updated HGVERSION                                                                                                                                 
[ README   ]   module_strings.h                                                                                                                   
cat README.rst | grep -v -E "^:([^:])+:.+$" | rst2html --no-xml-declaration | python ../html2text.py -b 700                                       
                                                                                                                                                 
                                                                                                                                                 
                                                                                                                                                 
/bin/sh: 1: rst2html: not found                                                                                                                   
grep: write error: Broken pipe                                                                                                                   
                                                                                                                                                 
(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x2b3b4aead1b8>)                                                           
make[4]: *** [module_strings.h] Error 1                                                                                                           
                                                                                                                                                 
********************************************************                                                                                         
WARNING: module raw_rec failed to build, deleting                                                                                                 
********************************************************                                                                                         
                                                         

I really am trying to make this work, but I can see what I'm doing wrong.

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3366
  • I have a 1Dx, I'm awesome.
Re: Compiling Magic Lantern in the Cloud
« Reply #47 on: April 12, 2015, 05:46:16 AM »
Looks like you didn't follow this?

@garry23

I don't know why your docutils download failed, it seems to be working now when I try it, so maybe it was just a fluke. Try running these commands:
Code: [Select]
cd
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html

P.s.  You nearly got the code tags right.  Hit the modify post button on your post and check how I formatted it.  ;)

garry23

  • Hero Member
  • *****
  • Posts: 1074
Re: Compiling Magic Lantern in the Cloud
« Reply #48 on: April 12, 2015, 05:55:30 AM »
Isn't this the same?

Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

Licaon_Kter

  • Senior
  • ****
  • Posts: 494
  • M
Re: Compiling Magic Lantern in the Cloud
« Reply #49 on: April 12, 2015, 08:53:48 AM »
It should be the same but if you follow it step by step maybe you can identify where the issue arises.