Magic Lantern Forum

Developing Magic Lantern => General Development Discussion => Topic started by: dmilligan on March 14, 2015, 04:08:42 AM

Title: Compiling Magic Lantern in the Cloud
Post by: dmilligan on March 14, 2015, 04:08:42 AM
Compiling Magic Lantern in the Cloud

Just found this neat solution for compiling and editing ML "in the cloud". There are several different online services that provide free linux VMs for coding. I managed to get ML to compile in one of these with minimal setup.

Here's a simple step by step guide

Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash


Now you are all set to compile, cd to your specific camera's directory, for example:
Code: [Select]
cd platform/550D.109/

Then:
Code: [Select]
make clean && make zip

Once you finished building, to download your files from the VM to your computer go to Project > Export as Zip

For more info on how to checkout other branches, merge and do other version control stuff see: http://hginit.com and also http://www.magiclantern.fm/forum/index.php?topic=9524.0

Happy Coding!


Next Steps: figure out how to setup QEMU in there


*here's the original script for reference:
Code: [Select]
#!/bin/sh
cd
curl -OL https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
bzip2 -d gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
tar -x -f gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html
cd workspace
hg clone -r unified https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
Title: Re: Compiling Magic Lantern in the Cloud
Post by: DeafEyeJedi on March 14, 2015, 05:41:27 AM
Sweet Stuff @dmilligan -- thanks for sharing a much needed program to make it easier for some of us to compile.

However, when trying to do this for 5D3.113 -- I get the following error via Terminal:

(https://farm8.staticflickr.com/7282/16601143387_aa6e19e5b5_n.jpg) (https://flic.kr/p/rhZcy8)

Take 2:
(https://farm8.staticflickr.com/7649/16807363612_d157236ce4_n.jpg) (https://flic.kr/p/rBd8Df)

Take 3:
https://vimeo.com/122166125 (https://vimeo.com/122166125)

Perhaps, am I missing something?

Thanks!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: g3gg0 on March 14, 2015, 09:42:53 AM
good idea, dmilligan.

@DeafEyeJedi
Code: [Select]
cd magic-lantern
make 5D3


that bootstrap script from dmilligan in short (pastebin):
Code: [Select]
    wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash
Title: Re: Compiling Magic Lantern in the Cloud
Post by: DeafEyeJedi on March 18, 2015, 08:27:35 AM
Thanks @g3gg0 -- I'll get on it and once I get the hang of it...will let you guys know how it goes!

*edit*

I got it to work and exported as a zip file... then the unzipped file itself looks quite a bit of a mess or no?

Codio:
(https://farm9.staticflickr.com/8709/16825919906_3171e5b9ff_n.jpg) (https://flic.kr/p/rCReMm)

Zip file (unzipped):
(https://farm8.staticflickr.com/7588/16229405714_3bde960e6d_n.jpg) (https://flic.kr/p/qJ8WTU)

Is this normal?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: vertigopix on March 18, 2015, 08:49:22 AM
it's the entire source tree.

In "platform" you can get the autoexec.bin and in "modules" you gan get ...modules !   ;)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on March 18, 2015, 11:47:22 AM
Is this normal?
Yes. I couldn't figure out a way to just download specific files, so you just download the entire thing. You should see a standard zip similar to the nightly build zip in platform/[cam]
Title: Re: Compiling Magic Lantern in the Cloud
Post by: DeafEyeJedi on March 18, 2015, 05:13:13 PM
Okay, I guess that makes sense. Thanks @vertigopix & @dmilligan for clarifying this. I'll give it another go.

Question -- how exactly can I grab the FRSP module that's been updated in the hudson branch recently by @a1ex? Do I just copy the whole module script from there and pasted it into the module's folder?
(i.e. -- I went ahead & copied the silent.c from hudson branch and just pasted it into silent.c within codio before exporting a zip file, is this correct?) [check video]

Also I'm not seeing autoexec.bin in platform -- hmmm?
 
I don't mind getting more familiar in advance with this so I can start compiling certain modules with the latest nightlies on a daily rate when possible.

Thanks again guys!

*edit*

Here's what I was referring to earlier...
https://vimeo.com/122552072 (https://vimeo.com/122552072)

I don't think I'm seeing the 'standard' zip similar to the nightly build in platform (cam) folders... Unless I'm doing something wrong?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on March 18, 2015, 07:20:35 PM
Question -- how exactly can I grab the FRSP module that's been updated in the hudson branch recently by @a1ex?

FSRP is in it's own branch: "full-res-silent-pictures", you need to checkout that branch. Just coping silent.c will not work correctly.

For more info on how to checkout other branches, merge and do other version control stuff see: http://hginit.com
Title: Compiling Magic Lantern in the Cloud
Post by: DeafEyeJedi on March 18, 2015, 07:40:38 PM
Love the read so far regarding hg.

Definitely gonna dig deep on this to get the most of out it - thanks for the reminder w the link.

Until then...
Title: Re: Compiling Magic Lantern in the Cloud
Post by: guisquil on March 20, 2015, 08:30:23 AM
Compiling Magic Lantern in the Cloud

Thanks so much for doing this I was able to compile the 7D "fullres-silent-pics" branch
Title: Re: Compiling Magic Lantern in the Cloud
Post by: guisquil on March 21, 2015, 03:01:21 AM
Some modules failed to build I'm following the OP steps:

Building module dual_iso...

/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/libgcc.a when searching for -lgcc                                         
/usr/bin/ld: cannot find -lgcc                                                                                                                 
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/libgcc_s.so when searching for -lgcc_s                                     
/usr/bin/ld: cannot find -lgcc_s                                                                                                               
collect2: error: ld returned 1 exit status                                                                                                     
make[4]: *** [cr2hdr] Error 1
                                                                                                                     
********************************************************                                                                                       
WARNING: module dual_iso failed to build, deleting                                                                                             
********************************************************
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on March 21, 2015, 03:22:48 AM
Yeah, cr2hdr and raw2dng have to be compiled in 32 bit mode, and this VM is 64 bit and doesn't have the 32bit gcc libraries installed. Without sudo privliges, I'm not sure installing them is possible. The only solution really is to modify the makefile for dual_iso module so that it doesn't try to compile cr2hdr (the same for raw_rec which tries to compile raw2dng).
Title: Re: Compiling Magic Lantern in the Cloud
Post by: guisquil on March 21, 2015, 10:17:50 AM
Thanks dmilligan for the info,
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 09, 2015, 10:22:50 PM
I'm not a coder but tried this 'in the cloud' approach.

I'm afraid I failed at step one, ie trying to duplicate dmilligan's instructions.

Has anyone successfully used this approach and is willing to give a little more insight into the workflow or steps to take.

I will be happy to simply get a compiled version, ie without changing anything.

Cheers
Title: Re: Compiling Magic Lantern in the Cloud
Post by: surami on April 09, 2015, 10:50:48 PM
Which ML build version?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 09, 2015, 10:52:13 PM
I'm directing all my learning at my 5DIII.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: surami on April 09, 2015, 11:03:09 PM
If you are not a coder, it's better to wait for the right person. I compiled my first build on this week, so I'm not a pro in this.

Ask for help here (http://www.magiclantern.fm/forum/index.php?topic=12608.0).
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 09, 2015, 11:10:08 PM
Let me correct myself.

I'm not an ML coder or more correctly someone who understands how to compile ML .

I do know how to code, it's just the steps to take using the 'in the cloud' approach.

Any help gratefully received :-)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 10, 2015, 12:10:30 AM
Where are you getting stuck? Were you able to make a codio account and create a new project?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 10, 2015, 12:18:13 AM
Truly, I don't want you to waste your time, however, I did stumble at the start:

Head over to http://www.codio.com and setup an account (you get a VM with 2GB of hard drive and 256MB of RAM for free). - DID THIS OK

Create a new project with the default "stack". I THINK I DID THIS OK

When the project opens, goto Tools > Install Software and install 'zip': DID NOT UNDERSTAND INSTALL ZIP

Now go to Tools > Terminal DIDNT GET THIS FAR
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 10, 2015, 01:41:52 AM
You see the tools menu? Go to "install software" you will see a lot of things you can install. Down near the end is "zip". Install it.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 10, 2015, 02:24:53 AM
@dmilligan

Many thanks: progress.

It is exciting to see ML compile in front of you.

Having said that I stumbled again, as clearly I don't know how to use the 'cd' command.

Also, I really need to see the workflow of someone who has taken the ML 'source' and tweaked it and compiled it in the cloud.

BTW I'm not asking you to tell me or show me.

But I help some kind ML traveler who has worked out the 'cloud secrets' will share their method sometime.

Once again, thanks for getting me this far: I sense there is hope for me :-)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 10, 2015, 04:21:48 AM
http://www.linfo.org/cd.html
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 10, 2015, 04:45:27 AM
@dmilligan

I owe you a beer ;-)

I managed to complete the full process. Thanks for your help.

The 'last' question I have now is, how do I change the source, say in lens.c, ie to create a personal tweak of lens.c and compile it in the cloud..

Is this too complicated.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 10, 2015, 05:10:11 AM
Another thing I noticed is that in the module folder there are no modules, only .c files.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 10, 2015, 12:50:42 PM
The 'last' question I have now is, how do I change the source, say in lens.c, ie to create a personal tweak of lens.c and compile it in the cloud..

Simply edit the file, save it, then run make again. Codio has a builtin code editor, all your files should appear on the left side bar, click on one and it should open as a tab in the codio editor. Alternatively you can use vim or emacs (these are command line based text editors) from the terminal (e.g. vim filename.c).
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Licaon_Kter on April 10, 2015, 01:41:49 PM

Another thing I noticed is that in the module folder there are no modules, only .c files.
Did you compile the modules by running make all_modules too?


My compile command line is:
Code: [Select]
make clean && make && make all_modules && cd platform/EOSM.202 && make zip
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 10, 2015, 02:46:16 PM
@dmilligan and Licaon_Kter

Thanks to both of you for the education. I can see I'm going to have fun this week end.

I think I read earlier that the download from Codio has 'everything' in it 

Thus what I will need to do is trawl through the download and compare with a normal ML download, to ensure I match the right files that are required.

Any other pointers for a Codio virgin :-)

Cheers
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 10, 2015, 05:19:15 PM
Thus what I will need to do is trawl through the download and compare with a normal ML download, to ensure I match the right files that are required.
The download will have the entire contents of your codio workspace. Look for the zip file inside that corresponds to the zip you generated with ML (should be in the platform folder for your specific camera).
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Audionut on April 10, 2015, 07:57:49 PM
My compile command line is:
Code: [Select]
make clean && make && make all_modules && cd platform/EOSM.202 && make zip

Make zip in the camera platform directory should be all that's needed.  Make clean is good practice though.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 10, 2015, 09:23:07 PM
@All

I have made progress but get compile errors, eg modules not compiled, and the resultant zip downloaded is obviously incomplete.

I have used the following terminal commands in codio:

wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern

make clean && make && make all_modules && cd platform/magic-lantern && make zip

Can anyone see my 'mistake'?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: surami on April 10, 2015, 09:33:17 PM
I wanted to compile today a fresh unified build on the Codio, but I got the same errors as garry23 wrote, eg.:
Code: [Select]
Building module mlv_rec...                                                                                                                                                                                                                 
Updated HGVERSION                                                                                                                                                                                                                         
[ README   ]   module_strings.h                                                                                                                                                                                                           
cat README.rst | grep -v -E "^:([^:])+:.+$" | rst2html --no-xml-declaration | python ../html2text.py -b 700                                                     
                                                                                                                                                                                                                                           
/bin/sh: 1: rst2html: not found                                                                                                                                                                                                           
grep: write error: Broken pipe                                                                                                                                                                                                             
                                                                                                                                                                                                                                           
(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x2b27e9af0f38>)                                                                                                                                                     
make[4]: *** [module_strings.h] Error 1                                                                                                                                                                                                   
                                                                                                                                                                                                                                           
********************************************************                                                                                                                                                                                   
WARNING: module mlv_rec failed to build, deleting                                                                                                                                                                                         
********************************************************

I used the same line from the OP:
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash
Code: [Select]
cd magic-lantern/platform/550D.109/
Code: [Select]
make zip
Title: Re: Compiling Magic Lantern in the Cloud
Post by: surami on April 10, 2015, 10:33:09 PM
@garry23: Make a new project and if you are at install software step, where you have to install zip, install the python3 too.

Somebody knows, how to solve the gcc 32bit issue? (dual_iso, dot_tune, etc. modules couldn't be compiled)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 10, 2015, 11:06:57 PM
@surami

Tried that but still get the error that every module fails.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 10, 2015, 11:17:24 PM
Postscript

Could someone who has successfully used this approach simply lay out the exact terminal commands they used.

Cheers
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 11, 2015, 03:11:12 AM
PPS

I have continue to try all advice, eg Audionut's.

Here are my terminal commands in codio

wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern/platform/5D3.113/

make clean && make zip

And what follows is module compile errors and the process fails.

I think I'll give up until some expert hints at where I'm going wrong :-)

Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 11, 2015, 06:20:47 AM
It doesn't actually fully fail, you should at least have some stuff built. It's just a couple of modules that fail to build (mainly just dual_iso I think), and the reason is that they are trying to build the desktop tools (cr2hdr) and most ML desktop tools require 32 bit compilation, while these VM s are 64bit and don't have 32bit gcc libraries available.

I have a PR in the queue that will fix the 64 bit compilation issue.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 11, 2015, 02:33:51 PM
@dmilligan

Thanks for the update.

Can you confirm my command line inputs above are the correct ones to use.

And if they are, I simplt ignore all the errors and download the zip.

Cheers
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 11, 2015, 03:36:48 PM
@dmilligan

I can report every module fails, eg

/bin/sh: 1: rst2html: not found                                                                                                                   
grep: write error: Broken pipe                                                                                                                   
                                                                                                                                                 
(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x2b10a697dfc8>)                                                           
make[4]: *** [module_strings.h] Error 1                                                                                                           
                                                                                                                                                 
********************************************************                                                                                         
WARNING: module mlv_snd failed to build, deleting                                                                                                 
********************************************************           
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 11, 2015, 05:06:01 PM
Another post from me as I'm trying to provide feedback to help developers.

I have tried once again, and now get this error sett:
Welcome to the Codio Terminal!                                                                                                                 
*                                                                                                                                                 
* README: https://codio.com/s/docs/boxes                                                                                                         
*                                                                                                                                                 
* Your Codio Box domain: design-jazz.codio.io[:1024-9999]                                                                                         
* External ssh connection: ssh://codio@ssh2.codio.com:2140                                                                                       
*                                                                                                                                                 
codio@design-jazz:~/workspace$ wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash                                         
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                                                   
                                 Dload  Upload   Total   Spent    Left  Speed                                                                     
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0                                                                   
100 57.5M  100 57.5M    0     0  4184k      0  0:00:14  0:00:14 --:--:-- 5168k                                                                   
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                                                   
                                 Dload  Upload   Total   Spent    Left  Speed                                                                     
100   551  100   551    0     0   5027      0 --:--:-- --:--:-- --:--:-- 12244                                                                   
                                                                                                                                                 
gzip: docutils-0.12.tar.gz: not in gzip format                                                                                                   
tar: docutils-0.12.tar: Cannot open: No such file or directory                                                                                   
tar: Error is not recoverable: exiting now                                                                                                       
bash: line 9: cd: docutils-0.12: No such file or directory                                                                                       
python: can't open file 'setup.py': [Errno 2] No such file or directory                                                                           
cp: cannot stat `.local/bin/rst2html.py': No such file or directory                                                                               
destination directory: magic-lantern                                                                                                             
adding changesets                                                                                                                                 
adding manifests                                                                                                                                 
adding file changes                                                                                                                               
added 11013 changesets with 24585 changes to 2672 files                                                                                           
updating to branch unified                                                                                                                       
1554 files updated, 0 files merged, 0 files removed, 0 files unresolved                                                                           
codio@design-jazz:~/workspace$   
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 11, 2015, 09:36:29 PM
Looks like docutils failed to download correctly. Maybe the server was down or the url changed. I'll check on it.

BTW, it will be easier to read stuff like console output on the forum if you put it inside code tags. Also prevents posts from getting too long, and makes it more clear the difference between what is output and what is just you talking.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Licaon_Kter on April 12, 2015, 01:32:46 AM
To skip docs build (errors) edit /magic-lantern/Makefile

Code: [Select]

./Makefile-88-# We must build the docs first to use fresh doc/menuindex.txt
./Makefile:89:# during 'make all'. We can't write 'zip: all docs' because
./Makefile-90-# of possible problem in case of parallel build.
./Makefile-91-# (see make's '-j' option documentation)
./Makefile:92:zip: docs
./Makefile-93-  $(MAKE) all
./Makefile:94:  cd $(PLATFORM_PATH)/all; $(MAKE) zip
./Makefile-95-
And on line 92 delete the word docs

To skip 32-bit apps build, edit /magic-lantern/modules/dual_iso/Makefile
Code: [Select]
./Makefile-6-MODULE_OBJS=dual_iso.o
./Makefile-7-
./Makefile-8-# include modules environment
./Makefile-9-include $(TOP_DIR)/modules/Makefile.modules
./Makefile-10-
./Makefile:11:# include cr2hdr makefile rules
./Makefile:12:include Makefile.cr2hdr
and remove line 12

Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 12, 2015, 02:51:30 AM
You still need docutils installed to compile the modules, even if you skip 'docs' build (it's used to automatically generate header files that are compiled into modules).
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 12, 2015, 03:03:09 AM
@garry23

I don't know why your docutils download failed, it seems to be working now when I try it, so maybe it was just a fluke. Try running these commands:
Code: [Select]
cd
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 12, 2015, 03:47:50 AM
@dmilligan

Thanks.

The issue is that I just can't get a full zip.

When I use the following I get errors.

wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern

make clean

cd platform/5D3.113/

make zip

Any ideas?


Title: Re: Compiling Magic Lantern in the Cloud
Post by: Audionut on April 12, 2015, 05:30:21 AM
What errors?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 12, 2015, 05:39:55 AM
@Audionut

As I have said above I use the following

Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern/platform/5D3.113/

make clean && make zip

When I look in the download .zip on the platform folder there is just one module.

Every module has an error like this:

Code: [Select]
                                                                                                                                                 
Building module raw_rec...                                                                                                                       
Updated HGVERSION                                                                                                                                 
[ README   ]   module_strings.h                                                                                                                   
cat README.rst | grep -v -E "^:([^:])+:.+$" | rst2html --no-xml-declaration | python ../html2text.py -b 700                                       
                                                                                                                                                 
                                                                                                                                                 
                                                                                                                                                 
/bin/sh: 1: rst2html: not found                                                                                                                   
grep: write error: Broken pipe                                                                                                                   
                                                                                                                                                 
(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x2b3b4aead1b8>)                                                           
make[4]: *** [module_strings.h] Error 1                                                                                                           
                                                                                                                                                 
********************************************************                                                                                         
WARNING: module raw_rec failed to build, deleting                                                                                                 
********************************************************                                                                                         
                                                         

I really am trying to make this work, but I can see what I'm doing wrong.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Audionut on April 12, 2015, 05:46:16 AM
Looks like you didn't follow this?

@garry23

I don't know why your docutils download failed, it seems to be working now when I try it, so maybe it was just a fluke. Try running these commands:
Code: [Select]
cd
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html

P.s.  You nearly got the code tags right.  Hit the modify post button on your post and check how I formatted it.  ;)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 12, 2015, 05:55:30 AM
Isn't this the same?

Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Licaon_Kter on April 12, 2015, 08:53:48 AM
It should be the same but if you follow it step by step maybe you can identify where the issue arises.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 12, 2015, 03:21:50 PM
@licaon_Kter

I tried adding the addtional code from Audionut but it confirmed docutils was already loaded.

Once again I tried the following:

Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern/platform/5D3.113/

make clean && make zip

Every module appears to fail like this:

Code: [Select]
/bin/sh: 1: rst2html: not found                                                                                                                   
grep: write error: Broken pipe                                                                                                                   
                                                                                                                                                 
(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x2b710424cfc8>)                                                           
make[4]: *** [module_strings.h] Error 1                                                                                                           
                                                                                                                                                 
********************************************************                                                                                         
WARNING: module mlv_snd failed to build, deleting                                                                                                 
********************************************************   

When I look in the downloaded zip, in the platform zip file it is obviously not built, ie no modules present.

What the hell am I doing wrong !!!!!!!!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 12, 2015, 03:37:28 PM
Please run the commands I posted exactly as is (do not run the wget command!), and show the output.

The modules aren't getting built because a python script called "rst2html" is not present. It is not present because the download and setup of docutils fails for some strange reason. So I need you to run the commands that set up docutils so I can see what is happening. If you don't do this, there's nothing else I can do, b/c the script works just fine for me.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 12, 2015, 03:56:10 PM
@dmilligan

I will do as you say, now.

BTW do I have to type in each line on the terminal, or is there an easy way to past in all the lines at once?

Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 12, 2015, 04:05:58 PM
@dmilligan

OK spotted a problem, once I have followed your commands, which I have done, what do I do now, ie to compile ML.

BTW how do you copy all the text on a terminal?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 12, 2015, 06:54:48 PM
You should be able to simply select the text, copy and paste it. If not, just take a screen shot.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 12, 2015, 07:06:44 PM
@dmilligan

I'm sorry to appear to be dragging this out, but I can assure you I'm not!

After inputting your command, the screen terminal text gives pages, but in windows I can only grab a page worth at a time, ie I can't drag to select.

Unless you have any suggestion, I will have to labourosly copy bits at a time.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Walter Schulz on April 12, 2015, 07:09:09 PM
Mark text -> Open context menu and select "copy" (or just hit CTRL-C) -> Go to codio's terminal window and drag by CTRL-V

And I typed the whole text, without copy-paste ...
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 17, 2015, 05:58:03 PM
I've been away from this for a week, but now trying again.

I have followed instructions above to the letter, ie using these commands:
Code: [Select]
cd
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html
cd workspace
hg clone -r unified https://bitbucket.org/hudson/magic-lantern

I then did this:
Code: [Select]
cd magic-lantern/platform/5D3.113/
make zip

But all I get is this:
Code: [Select]
codio@santana-dynasty:~/workspace/magic-lantern/platform/5D3.113$ make zip                                                           
../../Makefile.inc:79: remove /home/codio/workspace/magic-lantern/platform/5D3.113/zip                                               
[ RM dir ]  /home/codio/workspace/magic-lantern/platform/5D3.113/zip/                                                                 
mkdir -p /home/codio/workspace/magic-lantern/platform/5D3.113/zip                                                                     
[ CPP      ]   magiclantern.lds                                                                                                       
/bin/sh: 1: /home/codio/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3: not found                                           
make: *** [magiclantern.lds] Error 1                                                                                                 
codio@santana-dynasty:~/workspace/magic-lantern/platform/5D3.113$   

Can anyone with knowledge help me please.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: surami on April 17, 2015, 06:17:10 PM
You leaved out this 3 lines, run them before docutils.

Code: [Select]
curl -OL https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
bzip2 -d gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
tar -x -f gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 17, 2015, 06:27:41 PM
@surami

Tried that, ie used this set of commands:
Code: [Select]
cd
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
curl -OL https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
bzip2 -d gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
tar -x -f gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"cd
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html
cd workspace
hg clone -r unified https://bitbucket.org/hudson/magic-lantern

Then did a make clean and a make zip, but got this:
Code: [Select]
codio@cricket-rio:~/workspace/magic-lantern/platform/5D3.113$ make zip                                                                                                                                                                     
../../Makefile.inc:79: remove /home/codio/workspace/magic-lantern/platform/5D3.113/zip                                                                                                                                                     
[ RM dir ]  /home/codio/workspace/magic-lantern/platform/5D3.113/zip/                                                                                                                                                                     
mkdir -p /home/codio/workspace/magic-lantern/platform/5D3.113/zip                                                                                                                                                                         
[ CPP      ]   magiclantern.lds                                                                                                                                                                                                           
/bin/sh: 1: /home/codio/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3: not found                                                                                                                                               
make: *** [magiclantern.lds] Error 1                                                                                                                                                                                                       
codio@cricket-rio:~/workspace/magic-lantern/platform/5D3.113$             

Any ideas?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: surami on April 17, 2015, 06:48:14 PM
Read this again (http://www.magiclantern.fm/forum/index.php?topic=14942.msg144870#msg144870) and don't forget, that where the many code lines are, there is a scrollbar too.

I just tested and it works for me.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 17, 2015, 07:04:25 PM
@surami

Many thanks for staying with me :-)

I followed your instructions to the letter and it worked!!!

I have one last question, if I may?

At the moment I have to enter every command line individually. Do you do this as well?

If you don't, how do you use pastebin to run the command in batch mode? I have a pastebin account.

Once again, many thanks for helping me  :)

Title: Re: Compiling Magic Lantern in the Cloud
Post by: surami on April 17, 2015, 08:20:08 PM
:)

You can get your own Pastebin URL without an account too:

0. Copy this code. I make it universal, so people with another camera can use this too:
Code: [Select]
cd
curl -OL https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
bzip2 -d gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
tar -x -f gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html
cd workspace
hg clone -r unified https://bitbucket.org/hudson/magic-lantern
1. Go to the Pastebin site (http://pastebin.com/) and paste the code to the New Paste area.
2. Copy the (I say now) platform code line to the end of the previous code lines at the Pastebin site:
Code: [Select]
cd magic-lantern/platform/PLATFORM.FOLDER.NAME/You can get your platform folder name from the nightly builds download site (https://builds.magiclantern.fm/#/). Eg. "5D3.113":
Code: [Select]
cd magic-lantern/platform/5D3.113/3. Write this to the end of the code lines at the Pastebin site:
Code: [Select]
make zip4. Submit
5. Captcha -> Submit
6. Open a Notepad and copy this code to there:
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=HERE_COMES_THE_LAST_8_CHARACHTER | sed "s/\r//g" | bash7. Copy the last 8 charachter from your browser address line, you will see something like this
Code: [Select]
http://pastebin.com/ABCDEFGH, so you need "ABCDEFGH":
8. Replace the "HERE_COMES_THE_LAST_8_CHARACHTER" with that last 8 characher, Eg.:
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=ABCDEFGH | sed "s/\r//g" | bash
Copy this code line and paste to the Codio Terminal and hit enter.

This way you can get a unified build, but (till now) without the dot_tune, dual_iso and raw_rec module (gcc 32bit issue).

You can make two Pastebin URLs too. One for cloning the repository and one for compiling the build. In between you can make your unique modifications with your own responsibility.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 17, 2015, 11:35:10 PM
@surami

Many thanks...once again  :)

Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 18, 2015, 05:01:05 PM
@surami

Maybe I spoke too soon.

When I use your command lines and enter them one at a time, all works fine.

If I use the pastebin approach I just get one module compiled

Have you, or anyone else, seen this behavio?

Cheers

PS here is my pastebin: http://pastebin.com/6NUPS8ff (http://pastebin.com/6NUPS8ff)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: surami on April 18, 2015, 11:36:18 PM
@garry23

I don't know what is the problem now. Sometimes the compiling works, sometimes not. Maybe it's a Codio service issue?

Here are the two lines (+pastebin links), what I tried:
1. Cloning the unified repository (click here to see the code (http://pastebin.com/1Fh8eyhz)):
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=1Fh8eyhz | sed "s/\r//g" | bash2. Compiling the unified build for 5D Mark III (1.1.3) (click here to see the code (http://pastebin.com/vRDmTAT6)):
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=vRDmTAT6 | sed "s/\r//g" | bash
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 18, 2015, 11:45:03 PM
@surami

I think you are confirming what I found, ie using the pastebin approach doesn't work every time.

Thanks for looking into things.

BTW my biggest problem is that I simple can't get sqrt to work in the C++ ML environment. It always returns zero, even when I know the input is a double variable.

Do you, or anyone else (any developers) know if sqrt function is supported?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 19, 2015, 01:11:27 AM
ML is not C++, it's plain C.

The ARM processor in these cameras doesn't have floating point hardware. You can do some floating point stuff, but it's very slow and you should try to avoid it (if you do use floating point, use float, not double). It's better to use integers or fixed point instead (you'll notice that ML makes extensive use of fixed point, and there is very little floating point). See imath.h for some integer based math functions and helper macros. Remember to be careful about the order of operations to avoid propagating round off errors.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 19, 2015, 01:34:07 AM
@dmilligan

Many thanks for your insight.

I am determined to learn how to tweak the ML code and have spent all day 'educating' myself, eg looking at bits of ML.

Clearly sqrtf works but, as you say, I need to be careful.

Once again: thanks.

Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on April 19, 2015, 08:54:52 PM
I'm adding this post to hopefully give confidence to others.

Before trying the 'compile in the cloud' approach, I knew nothing about C or compiling C for ML. Thanks to great support from several (see above) I have now reached the stage of successfully personalising ML: which I just wrote about on my blog http://photography.grayheron.net/2015/04/customising-magic-lantern.html (http://photography.grayheron.net/2015/04/customising-magic-lantern.html)

The test case I chose was to tweak the ML DoF feedback to account for diffraction.

After 'educating' myself about C and its syntax and rules, I managed to successfully tweak ML to now give me DoF and account for diffraction.

In terms of the way forward: I'm afraid until the compile in the cloud is fully running, ie all modules compiled, I will have to put further efforts to personalise ML on hold.

The message I wanted to get out, is that, if I can do it, so can you!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Audionut on April 20, 2015, 05:00:10 AM
The next step is to create a pull request to push your changes back to the main repository.

http://www.magiclantern.fm/forum/index.php?topic=7940.0
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Zaimon on April 20, 2015, 03:38:27 PM
Hi,

I have compiled for the 600D.102 according to your instructions.
Everything went fine but I can not find any lua.mo in modules.

Am I doing something wrong?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on April 20, 2015, 05:06:46 PM
The lua stuff is not merged to the main branch ('unified') yet, it's in a separate branch. You need to switch to the lua branch (this guide is for compiling ML in general, not specifically for compiling lua branch).

From 'magic-lantern' directory, run:
Code: [Select]
hg pull -u -b lua

You also need to explicitly build the lua.mo (I didn't add it to the modules makefile yet):
Code: [Select]
make -C modules/lua

However, IIRC the 600D is like 1100D in regards to the fairly small autoexec.bin size limit (someone please correct me if I'm thinking of another camera). All of the extra libc symbols needed by lua push the autoexec.bin size over the limit (at least on 1100D). So you will probably have to disable some stuff in platform/600D.102/features.h to get it to run (to disable a feature, simply #undef the macro for it; for a list, see all_features.h). Look for the MemSiz column in the build output, it should be less than 0x80000 (512KB).

You're looking for something like this:
Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x073918 0x00cf3918 0x00cf3918 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00c80100 0x00c80100 0x73820 0x7ff0c RWE 0x100


For 1100D I usually disable features that take up a lot of space, that I don't need like: DEFISHING_PREVIEW, ANAMORPHIC_PREVIEW, MOTION_DETECT, TRAP_FOCUS, FOLLOW_FOCUS, etc.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Zaimon on April 21, 2015, 12:11:18 PM
@dmilligan

I built the lua.mo and as you pointed out I needed to reduce the size of the autoexec.bin.
The module seems to work correctly.

Thanks a lot!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dfort on June 09, 2015, 05:28:52 AM
Thank you so much for this. I finally got a way to compile ML from source and had my first pull request merged into unified. Whoo Hoo!

A while back I wanted to compile raw2dng for use on a Macbook Pro and got tripped up because I was following instructions to compile ML. Finally got around it by pulling out just the raw2dng code. Now with dmilligan's guidance we got a simple way to compile ML from source but I am getting tripped up trying to get the raw_rec and dual_iso modules because of this:

http://www.magiclantern.fm/forum/index.php?topic=14725.msg142856#msg142856

Quote from: dmilligan
Yeah, cr2hdr and raw2dng have to be compiled in 32 bit mode, and this VM is 64 bit and doesn't have the 32bit gcc libraries installed. Without sudo privliges, I'm not sure installing them is possible. The only solution really is to modify the makefile for dual_iso module so that it doesn't try to compile cr2hdr (the same for raw_rec which tries to compile raw2dng).

I tried to work my way around it but have failed miserably. Oh well, everything else seems to work fine.

So my question is--is it ok to copy the raw_rec and dual_iso modules from the ML/modules directory of the nightly builds? I tried it and it seems to work but thought I'd ask.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Danne on June 09, 2015, 07:51:02 AM
Interesting. Could I ask how you added changes in the raw2dng codes? I tried but my changes didn, t work. Maybe a step by step guide if you have some time?
Thanks.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Licaon_Kter on June 09, 2015, 09:39:44 AM

Now with dmilligan's guidance we got a simple way to compile ML from source but I am getting tripped up trying to get the raw_rec and dual_iso modules because of this:
http://www.magiclantern.fm/forum/index.php?topic=14725.0 (http://www.magiclantern.fm/forum/index.php?topic=14725.0)
Link is wrong? But I guess you need this patch: https://bitbucket.org/hudson/magic-lantern/diff/src/chdk-dng.c?diff2=6aa15daa7bc0&at=fullres-silent-pics ?



Title: Re: Compiling Magic Lantern in the Cloud
Post by: Danne on June 09, 2015, 10:52:08 AM
Been a while since I worked on this but I tried changing things like metadata. Could have been raw_rec i fiddled with. I got to take some time to learn more deeply.
Noob question. What would it take to change metadata information in raw2dng and get a compiled binary for mac? Is the 64bit issue solved?
Thanks.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dfort on June 09, 2015, 09:10:09 PM
@Licaon_Kter

Fixed the link and quoted what dmilligan wrote. Thanks for the patches.

Interesting. Could I ask how you added changes in the raw2dng codes? I tried but my changes didn, t work. Maybe a step by step guide if you have some time?
Thanks.

I was able to compile raw2dng but that was quite a while ago.

http://magiclantern.fm/forum/index.php?topic=7645.msg111168#msg111168

(Hope I posted the right link this time!)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dfort on June 11, 2015, 12:04:09 PM
Darn it. I thought I fixed that link. This is the issue I'm talking about:


Yeah, cr2hdr and raw2dng have to be compiled in 32 bit mode, and this VM is 64 bit and doesn't have the 32bit gcc libraries installed. Without sudo privliges, I'm not sure installing them is possible. The only solution really is to modify the makefile for dual_iso module so that it doesn't try to compile cr2hdr (the same for raw_rec which tries to compile raw2dng).
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Zaimon on June 15, 2015, 07:33:04 PM
Dear all,

I compiled both ML and the  LUA.mo in the cloud for a 600D, as described in the post:
http://www.magiclantern.fm/forum/index.php?topic=14725.msg145456#msg145456

I wrote a LUA script that takes a picture when the camera is switched on and I installed both the build and the script on a SD card whose size is 4GB, on the 600D.
I substituted the camera battery with an external temporized power supply that provides the power at a certain time interval and for a certain amount of time.
Everything worked fine. The camera starts, takes a picture and shut down the number of time set by the temporized external power supply.

Then I installed the ML equipped with the LUA.mo and the script on a SD card whose size is 32GB. If I use the camera battery and I manually switch on and off the camera everything works fine: the camera switch on, take a picture and when I switch the camera off it shut down normally. If I substitute the camera battery with the external temporized power supply, seems that a problem occur during the shut down. At the first bootstrap after the first shut down the console shows the following message error:
"Camera was not shut down cleanly. Skipping module loading.". The same problem do not occur with the 4GB SD card.

Does this problem is caused by the SD card size (32GB instead of 4GB)? If yes, do I have to make a partition as in the case of the CHDK?
Is there a way to automatically force the module loading even if the camera was not shut down cleanly?
Do you have any suggestion to overcome this problem?

Thanks a lot!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Walter Schulz on June 15, 2015, 07:50:38 PM
Does this problem is caused by the SD card size (32GB instead of 4GB)?

No.

Is there a way to automatically force the module loading even if the camera was not shut down cleanly?
Debug tab -> Modules debug -> Load modules after crash ON
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Zaimon on June 16, 2015, 05:51:03 PM
@Walter
thank you very much. It works perfectly.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: jipo on June 22, 2015, 11:44:26 PM
hi,

sorry for a stupid question, but i can't go through the simple Codio setup...
1) i created an account....OK
2) i created a new project...ok
3) i failed to install sw or go to the terminal, with a mesage "Installing Software and Box access is only available if you have an active subscription."

Am I missing something, it is not for free now?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: PhotoCat on June 24, 2015, 06:41:30 PM
Interesting discussion! I have successfully compiled ML once
following this thread:

http://www.magiclantern.fm/forum/index.php?topic=6783.0

But that was how far I went... sadly...

Just wondering what is the advantage of cloud compiling versus the "old" way?
Sorry I am out to lunch...   pls enlighten me so that I can choose a better path
in learning how to tweak some ML codes. Tks!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on June 24, 2015, 06:49:55 PM
@PhotoCat

The gurus will comment I'm sure, however, from a novice perspective I think 'compile in the cloud' is great. It has opened up ML for me.

Before CITC all I could do was hope a guru would tweak ML for me.

I read about compiling on my PC, but it was al 'too much'.

With CITC I was able to tweak the ML depth of field code and prove it out, before the gurus authorized its use.

As a further example, with CITC I was able to take the latest nightly and compile a version for my IR converted 50D, ie IR diffraction.

The only 'negative' observation I have is that you need to tweak some of the batch coding, ie to force certain modules. But others have helped me here.

Bottom line: I love it!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: PhotoCat on June 24, 2015, 07:13:04 PM
Wow thanks garry23!
I will try it then...  so I don't need to setup the Virtual Machine on my PC any more?

My biggest road block was that I got ML to compiled and got the ML binaries but I
had no confidence to install it in my camera! So I was stuck sadly...   
Not knowing how the revision control software works (pull/push request??) is another problem...

I love to move forward if someone can help me check my binaries :)   
I know some C but very very rusty.

Thanks for all the encouragements garry23 & I really appreciate it!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on June 24, 2015, 07:24:06 PM
@PhotoCat

You are welcome :-)

Having been positive, I will say a few personal 'negatives'.

Like you, I consider myself someone who is able to code, BUT I'm not a C programmer. As I tried to understand ML C code, I became aware that you really need to understand C 'linguistics' before writing your own code. I still don't understand C well enough.

Also, ML code, although often commented, is not always readable, eg unless you wrote the code in the first place. This is not a moan, but a simple observation: ML is complex, ie with bits here and bits there.

I believe an alternative avenue is opening up, and one I have yet to try. That is script writing. I would also suggest you look at this.

Cheers

Garry


Title: Re: Compiling Magic Lantern in the Cloud
Post by: PhotoCat on June 24, 2015, 09:10:32 PM
Tks Garry23 once again for the pointer!
Script writing as in DOS script or AWK/Perl script? I have played with them a little bit but far from being proficient!
For now, I don't dream of coding ML in C. Just modifying some constants and get a working binary in my camera
will excite me very much LOL!
I also heard someone booted Linux on a Canon DSLR...   Not sure how that will change
ML development work flow in the future...
Anyway thanks for your help once again and I will see if I can get this cloud compile to work first :)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: PhotoCat on June 24, 2015, 11:31:10 PM
now I know what jipo is taking about:

 "Installing Software and Box access is only available if you have an active subscription."

I guess "Installing software" and "terminal" functions are not free any more...

I signed in via bitbucket... Can that be a problem?  Tks!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on June 25, 2015, 04:02:29 AM
https://codio.com/docs/boxes/installsw/box-parts/
Quote
As of June 2015 a subscription is required to install software packages. Existing users are not affected.

So you won't be able install the zip package, but everything else should work, and you still be able to compile ML (you just won't be able to 'make zip'). You can manually install zip though (compile it from source).

They have not taken away the "terminal", that would essentially defeat the purpose of having codio at all.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: PhotoCat on June 25, 2015, 04:24:18 AM
Thank you for your info dmilligan but for my beginner level, I am really at a lost here.
Perhaps I should just go back to the good old way of compiling ML on my computer with VM,
which I was successful at one time.

I think what I really need to know is how I could tell if the compiled binary file is working and not corrupted,
due to any compilation mistakes I may have.   Any tools available to test ML binary file without
risking our own camera?   Thanks a lot!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on June 25, 2015, 01:02:48 PM
I think what I really need to know is how I could tell if the compiled binary file is working and not corrupted,
due to any compilation mistakes I may have.
Try to load it on the camera.

This scenario is extremely unlikely anyway. 99.999% of the time, if you made a "compilation mistake" then the compilation would simply fail. If there were no errors during compilation and you successfully ended up with an autoexec.bin, the probability that it is somehow corrupted is extremely low. Further, even if it is "corrupted" the most likely thing to happen will be that it simply won't boot, and you'll have to pull the card and the battery. Pretty much the only thing you can do wrong is compile for the wrong camera, and if you compiled for the wrong camera, there is now an error message that shows on the screen and tells you what to do.

Any tools available to test ML binary file without risking our own camera?
There is QEMU (http://www.magiclantern.fm/forum/index.php?topic=2864.0)

Just wondering what is the advantage of cloud compiling versus the "old" way?
Nothing much. It's just probably easier to set up for some folks. It doesn't require setting up a VM or the toolchain on your own computer, and there's no requirements for you own computer other simply having a web browser (so it is actually possible to do this from a tablet or smart phone). If you already have a VM setup on your computer, then that is probably the way to go.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: PhotoCat on June 25, 2015, 05:14:16 PM
Kindly thanks dmilligan for your pointers and encouragement once again! OK I will try to setup the VM environment agn on my win7 machine...
and compile...
Hopefully I will have the courage to load the binaries into my cam!

btw, I will follow this video this time:

http://www.magiclantern.fm/forum/index.php?topic=6425.msg55525#msg55525
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Licaon_Kter on June 25, 2015, 07:03:21 PM
I did not see the youtube clip, but I advise you to use this compiler: https://launchpad.net/gcc-arm-embedded/+milestone/4.8-2014-q3-update (https://launchpad.net/gcc-arm-embedded/+milestone/4.8-2014-q3-update) -> gcc-arm-none-eabi-4_8-2014q3-20140805-linux.tar.bz2
you unpack it in /home/<username>/arm-toolchain

Makefile.user (to put in the root of magic-lantern)

Code: [Select]
SUPPORTED_MODELS:=EOSM
SUPPORTED_UNIFIED_MODELS:=
ARM_ABI=none-eabi
ARM_PATH=~/arm-toolchain
GCC_VERSION=-4.8.4
ARM_BINPATH=$(ARM_PATH)/bin
CONFIG_TCC          = y
CONFIG_MODULES      = y
CONFIG_PICOC        = y

Replace EOSM with your camera model, see these two lists, leave only your targets:
Code: [Select]
SUPPORTED_MODELS:=50D 60D 550D 600D 500D 5D2 5D3 7D 650D 700D EOSM 1100D 6D
SUPPORTED_UNIFIED_MODELS:=50D 60D 550D 600D 500D 5D2 1100D

in the magic-lantern folder run:
Code: [Select]
make clean && make && make all_modules && cd platform/<yourcamera> && make zip(yeah yeah it's a bit redundant but works ok and you make sure it's a clean slate before re-compile ;) )
Title: Re: Compiling Magic Lantern in the Cloud
Post by: PhotoCat on June 25, 2015, 09:07:01 PM
Thanks Licaon_Kter for the good info!  I think last time I only used gcc!
Wow, this gcc-arm-embedded one sound wonderful!  Tks!!   :)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: josepvm on June 25, 2015, 10:00:46 PM
The default gcc compiler for a Linux distro builds binary code intended to run in your computer, usually with a x86 compatible processor. You cannot run code built this way on a Canon camera, that uses an embedded ARM processor. So you need the gcc-arm-embedded compiler to build ML and get binary code that your camera can execute, there is not another way.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: PhotoCat on June 25, 2015, 10:13:13 PM
thks josepvm for the good info and I am getting more confident to give it a 2nd try...    :)
Title: Re: Compiling Magic Lantern in the Cloud
Post by: PhotoCat on June 30, 2015, 04:53:19 AM
Yay! I did it  :)   Thanks for all your encouragements! Got the ML codes to compile and loaded the autoexec.bin into my camera... no smoke...  8)

Got Virtualbox 4.3.28 and installed new toolchain 4.8.3 as per A1ex's recommendation.

https://launchpad.net/gcc-arm-embedded/+download

gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2 (md5)   Linux installation tarball

(not tried 4.8.4 yet)

Mathew's tutorial was great:

But somehow probably due to the newer Virtualbox version, I was unable to get the
copy-and-paste from host to VM function to work. Trying to install this function as per the video will crash Virtualbox.
So I skipped this part of the video but everything else seemed to work ok.

BTW, sorry to hijack this thread!  Now what??  Should I open a new thread or post in an existing thread?
How do I get help from now on as I am ready to do some very very minor mods to the codes?
Also I need to know the procedure to submit my new codes (if any) to the ML team... absolutely no clues...
Any pointers would be appreciated! Thanks!
Title: Re: Compiling Magic Lantern in the Cloud
Post by: DeafEyeJedi on June 30, 2015, 05:39:38 AM
Hello everyone -

I've decided to bite the bullet once again and get myself into this whole world of CITC... In effort to keep the development alive in the ML community. I ain't no coder but am pretty computer savvy and definitely have the patience for troubleshooting down bugs and whatnot.

Here's my 2nd take and since it has quite been a while since I last played with this... so I got to this point where it says:

(https://c1.staticflickr.com/1/284/19097048438_3c82b81bd4_n.jpg) (https://flic.kr/p/v6xnaQ)

Here's the clip of the entire process:

 

Perhaps it has to do with 'abort: destination 'magic-lantern' is not empty' according to Terminal?

Any suggestions or thoughts?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: Licaon_Kter on June 30, 2015, 09:45:44 AM
Now what??  Should I open a new thread or post in an existing thread?[/size][/font][/size]How do I get help from now on as I am ready to do some very very minor mods to the codes?Also I need to know the procedure to submit my new codes (if any) to the ML team... absolutely no clues...
Start a thread here: http://www.magiclantern.fm/forum/index.php?board=25.0 I guess

@DeafEyeJedi:
montage
is missing, says so right there, looking for it like this might help: https://packages.debian.org/search?searchon=contents&keywords=montage (https://packages.debian.org/search?searchon=contents&keywords=montage)
so install imagemagick?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dfort on July 01, 2015, 07:43:03 AM
I'm finding that Codio is running a little spotty. Sometimes it works, other times it dumps. However, I was able to get familiar with what is needed to work with and compile ML and I was finally able to get it working on my MacBook--no virtual box necessary.

In any case, compiling in the cloud is a good starting point for users that don't have a complete development platform to work with. Too bad that Codio started locking out new users from some of the feature needed to compile ML.
Title: Re: Compiling Magic Lantern in the Cloud
Post by: stoopkid on August 23, 2015, 05:36:58 AM
"Terminal and Box access is only available if you have an active subscription."

It doesn't look like this works anymore. Or am I missing something?
Title: Re: Compiling Magic Lantern in the Cloud
Post by: eatbuckshot on September 24, 2015, 09:53:22 AM
There are many free online IDE's with a linux vm, I frequently use https://c9.io or https://nitrous.io
Title: Re: Compiling Magic Lantern in the Cloud
Post by: garry23 on March 11, 2016, 06:37:44 PM
Has anyone recently and successfully used Codio for compiling in the Cloud?

I have followed the instructions on page 1, which worked for me about a year ago, but the Codio UI has changed since then and I'm getting nowhere.

If someone is currently using Codio, I would be grateful if they could share their workflow.

Cheers

Garry
Title: Re: Compiling Magic Lantern in the Cloud
Post by: dmilligan on March 11, 2016, 07:32:03 PM
It's probably worth scrapping this idea and moving on. Just use a VM like I've mentioned in other threads. If we can get a Linux VM setup easily, maybe we can replace this topic with instructions for doing that.

Try this:
Install VirtualBox: https://virtualbox.org/wiki/Downloads
Install a prebuilt Ubuntu VM: http://virtualboxes.org/images/ubuntu/
Run the bootstrap script from the OP in a command prompt
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash
Title: Re: Compiling Magic Lantern in the Cloud
Post by: hiser1944 on April 08, 2016, 09:26:37 AM
I'm not a coder but tried this 'in the cloud' approach.

I failed at step one,
Has anyone successfully used this approach and is willing to give a little more insight into the workflow or steps to take.