Author Topic: Compiling Magic Lantern in the Cloud  (Read 60853 times)

garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #50 on: April 12, 2015, 03:21:50 PM »
@licaon_Kter

I tried adding the addtional code from Audionut but it confirmed docutils was already loaded.

Once again I tried the following:

Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=jfVXzw1a | sed "s/\r//g" | bash

cd magic-lantern/platform/5D3.113/

make clean && make zip

Every module appears to fail like this:

Code: [Select]
/bin/sh: 1: rst2html: not found                                                                                                                   
grep: write error: Broken pipe                                                                                                                   
                                                                                                                                                 
(<type 'exceptions.SystemExit'>, SystemExit(1,), <traceback object at 0x2b710424cfc8>)                                                           
make[4]: *** [module_strings.h] Error 1                                                                                                           
                                                                                                                                                 
********************************************************                                                                                         
WARNING: module mlv_snd failed to build, deleting                                                                                                 
********************************************************   

When I look in the downloaded zip, in the platform zip file it is obviously not built, ie no modules present.

What the hell am I doing wrong !!!!!!!!

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3204
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #51 on: April 12, 2015, 03:37:28 PM »
Please run the commands I posted exactly as is (do not run the wget command!), and show the output.

The modules aren't getting built because a python script called "rst2html" is not present. It is not present because the download and setup of docutils fails for some strange reason. So I need you to run the commands that set up docutils so I can see what is happening. If you don't do this, there's nothing else I can do, b/c the script works just fine for me.

garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #52 on: April 12, 2015, 03:56:10 PM »
@dmilligan

I will do as you say, now.

BTW do I have to type in each line on the terminal, or is there an easy way to past in all the lines at once?


garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #53 on: April 12, 2015, 04:05:58 PM »
@dmilligan

OK spotted a problem, once I have followed your commands, which I have done, what do I do now, ie to compile ML.

BTW how do you copy all the text on a terminal?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3204
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #54 on: April 12, 2015, 06:54:48 PM »
You should be able to simply select the text, copy and paste it. If not, just take a screen shot.

garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #55 on: April 12, 2015, 07:06:44 PM »
@dmilligan

I'm sorry to appear to be dragging this out, but I can assure you I'm not!

After inputting your command, the screen terminal text gives pages, but in windows I can only grab a page worth at a time, ie I can't drag to select.

Unless you have any suggestion, I will have to labourosly copy bits at a time.

Walter Schulz

  • Hero Member
  • *****
  • Posts: 5761
Re: Compiling Magic Lantern in the Cloud
« Reply #56 on: April 12, 2015, 07:09:09 PM »
Mark text -> Open context menu and select "copy" (or just hit CTRL-C) -> Go to codio's terminal window and drag by CTRL-V

And I typed the whole text, without copy-paste ...

garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #57 on: April 17, 2015, 05:58:03 PM »
I've been away from this for a week, but now trying again.

I have followed instructions above to the letter, ie using these commands:
Code: [Select]
cd
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html
cd workspace
hg clone -r unified https://bitbucket.org/hudson/magic-lantern

I then did this:
Code: [Select]
cd magic-lantern/platform/5D3.113/
make zip

But all I get is this:
Code: [Select]
[email protected]:~/workspace/magic-lantern/platform/5D3.113$ make zip                                                           
../../Makefile.inc:79: remove /home/codio/workspace/magic-lantern/platform/5D3.113/zip                                               
[ RM dir ]  /home/codio/workspace/magic-lantern/platform/5D3.113/zip/                                                                 
mkdir -p /home/codio/workspace/magic-lantern/platform/5D3.113/zip                                                                     
[ CPP      ]   magiclantern.lds                                                                                                       
/bin/sh: 1: /home/codio/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3: not found                                           
make: *** [magiclantern.lds] Error 1                                                                                                 
[email protected]:~/workspace/magic-lantern/platform/5D3.113$   

Can anyone with knowledge help me please.

surami

  • Member
  • ***
  • Posts: 103
Re: Compiling Magic Lantern in the Cloud
« Reply #58 on: April 17, 2015, 06:17:10 PM »
You leaved out this 3 lines, run them before docutils.

Code: [Select]
curl -OL https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
bzip2 -d gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
tar -x -f gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar
550D + nightly ML

garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #59 on: April 17, 2015, 06:27:41 PM »
@surami

Tried that, ie used this set of commands:
Code: [Select]
cd
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
curl -OL https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
bzip2 -d gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
tar -x -f gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"cd
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html
cd workspace
hg clone -r unified https://bitbucket.org/hudson/magic-lantern

Then did a make clean and a make zip, but got this:
Code: [Select]
[email protected]:~/workspace/magic-lantern/platform/5D3.113$ make zip                                                                                                                                                                     
../../Makefile.inc:79: remove /home/codio/workspace/magic-lantern/platform/5D3.113/zip                                                                                                                                                     
[ RM dir ]  /home/codio/workspace/magic-lantern/platform/5D3.113/zip/                                                                                                                                                                     
mkdir -p /home/codio/workspace/magic-lantern/platform/5D3.113/zip                                                                                                                                                                         
[ CPP      ]   magiclantern.lds                                                                                                                                                                                                           
/bin/sh: 1: /home/codio/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3: not found                                                                                                                                               
make: *** [magiclantern.lds] Error 1                                                                                                                                                                                                       
[email protected]:~/workspace/magic-lantern/platform/5D3.113$             

Any ideas?

surami

  • Member
  • ***
  • Posts: 103
Re: Compiling Magic Lantern in the Cloud
« Reply #60 on: April 17, 2015, 06:48:14 PM »
Read this again and don't forget, that where the many code lines are, there is a scrollbar too.

I just tested and it works for me.
550D + nightly ML

garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #61 on: April 17, 2015, 07:04:25 PM »
@surami

Many thanks for staying with me :-)

I followed your instructions to the letter and it worked!!!

I have one last question, if I may?

At the moment I have to enter every command line individually. Do you do this as well?

If you don't, how do you use pastebin to run the command in batch mode? I have a pastebin account.

Once again, many thanks for helping me  :)


surami

  • Member
  • ***
  • Posts: 103
Re: Compiling Magic Lantern in the Cloud
« Reply #62 on: April 17, 2015, 08:20:08 PM »
:)

You can get your own Pastebin URL without an account too:

0. Copy this code. I make it universal, so people with another camera can use this too:
Code: [Select]
cd
curl -OL https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
bzip2 -d gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar.bz2
tar -x -f gcc-arm-none-eabi-4_8-2013q4-20131204-linux.tar
curl -OL http://prdownloads.sourceforge.net/docutils/docutils-0.12.tar.gz
gzip -d docutils-0.12.tar.gz
tar -x -f docutils-0.12.tar
cd docutils-0.12
python setup.py install --prefix=~/.local
cd
export PATH="$HOME/.local/bin:$PATH"
echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> .bash_profile
cp .local/bin/rst2html.py .local/bin/rst2html
cd workspace
hg clone -r unified https://bitbucket.org/hudson/magic-lantern
1. Go to the Pastebin site and paste the code to the New Paste area.
2. Copy the (I say now) platform code line to the end of the previous code lines at the Pastebin site:
Code: [Select]
cd magic-lantern/platform/PLATFORM.FOLDER.NAME/You can get your platform folder name from the nightly builds download site. Eg. "5D3.113":
Code: [Select]
cd magic-lantern/platform/5D3.113/3. Write this to the end of the code lines at the Pastebin site:
Code: [Select]
make zip4. Submit
5. Captcha -> Submit
6. Open a Notepad and copy this code to there:
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=HERE_COMES_THE_LAST_8_CHARACHTER | sed "s/\r//g" | bash7. Copy the last 8 charachter from your browser address line, you will see something like this
Code: [Select]
http://pastebin.com/ABCDEFGH, so you need "ABCDEFGH":
8. Replace the "HERE_COMES_THE_LAST_8_CHARACHTER" with that last 8 characher, Eg.:
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=ABCDEFGH | sed "s/\r//g" | bash
Copy this code line and paste to the Codio Terminal and hit enter.

This way you can get a unified build, but (till now) without the dot_tune, dual_iso and raw_rec module (gcc 32bit issue).

You can make two Pastebin URLs too. One for cloning the repository and one for compiling the build. In between you can make your unique modifications with your own responsibility.
550D + nightly ML

garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #63 on: April 17, 2015, 11:35:10 PM »
@surami

Many thanks...once again  :)


garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #64 on: April 18, 2015, 05:01:05 PM »
@surami

Maybe I spoke too soon.

When I use your command lines and enter them one at a time, all works fine.

If I use the pastebin approach I just get one module compiled

Have you, or anyone else, seen this behavio?

Cheers

PS here is my pastebin: http://pastebin.com/6NUPS8ff

surami

  • Member
  • ***
  • Posts: 103
Re: Compiling Magic Lantern in the Cloud
« Reply #65 on: April 18, 2015, 11:36:18 PM »
@garry23

I don't know what is the problem now. Sometimes the compiling works, sometimes not. Maybe it's a Codio service issue?

Here are the two lines (+pastebin links), what I tried:
1. Cloning the unified repository (click here to see the code):
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=1Fh8eyhz | sed "s/\r//g" | bash2. Compiling the unified build for 5D Mark III (1.1.3) (click here to see the code):
Code: [Select]
wget -q -O - http://pastebin.com/raw.php?i=vRDmTAT6 | sed "s/\r//g" | bash
550D + nightly ML

garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #66 on: April 18, 2015, 11:45:03 PM »
@surami

I think you are confirming what I found, ie using the pastebin approach doesn't work every time.

Thanks for looking into things.

BTW my biggest problem is that I simple can't get sqrt to work in the C++ ML environment. It always returns zero, even when I know the input is a double variable.

Do you, or anyone else (any developers) know if sqrt function is supported?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3204
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #67 on: April 19, 2015, 01:11:27 AM »
ML is not C++, it's plain C.

The ARM processor in these cameras doesn't have floating point hardware. You can do some floating point stuff, but it's very slow and you should try to avoid it (if you do use floating point, use float, not double). It's better to use integers or fixed point instead (you'll notice that ML makes extensive use of fixed point, and there is very little floating point). See imath.h for some integer based math functions and helper macros. Remember to be careful about the order of operations to avoid propagating round off errors.

garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #68 on: April 19, 2015, 01:34:07 AM »
@dmilligan

Many thanks for your insight.

I am determined to learn how to tweak the ML code and have spent all day 'educating' myself, eg looking at bits of ML.

Clearly sqrtf works but, as you say, I need to be careful.

Once again: thanks.


garry23

  • Hero Member
  • *****
  • Posts: 1314
Re: Compiling Magic Lantern in the Cloud
« Reply #69 on: April 19, 2015, 08:54:52 PM »
I'm adding this post to hopefully give confidence to others.

Before trying the 'compile in the cloud' approach, I knew nothing about C or compiling C for ML. Thanks to great support from several (see above) I have now reached the stage of successfully personalising ML: which I just wrote about on my blog http://photography.grayheron.net/2015/04/customising-magic-lantern.html

The test case I chose was to tweak the ML DoF feedback to account for diffraction.

After 'educating' myself about C and its syntax and rules, I managed to successfully tweak ML to now give me DoF and account for diffraction.

In terms of the way forward: I'm afraid until the compile in the cloud is fully running, ie all modules compiled, I will have to put further efforts to personalise ML on hold.

The message I wanted to get out, is that, if I can do it, so can you!

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3444
  • Blunt and to the point
Re: Compiling Magic Lantern in the Cloud
« Reply #70 on: April 20, 2015, 05:00:10 AM »
The next step is to create a pull request to push your changes back to the main repository.

http://www.magiclantern.fm/forum/index.php?topic=7940.0

Zaimon

  • New to the forum
  • *
  • Posts: 5
Re: Compiling Magic Lantern in the Cloud
« Reply #71 on: April 20, 2015, 03:38:27 PM »
Hi,

I have compiled for the 600D.102 according to your instructions.
Everything went fine but I can not find any lua.mo in modules.

Am I doing something wrong?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3204
  • 60Da / 1100D / EOSM
Re: Compiling Magic Lantern in the Cloud
« Reply #72 on: April 20, 2015, 05:06:46 PM »
The lua stuff is not merged to the main branch ('unified') yet, it's in a separate branch. You need to switch to the lua branch (this guide is for compiling ML in general, not specifically for compiling lua branch).

From 'magic-lantern' directory, run:
Code: [Select]
hg pull -u -b lua

You also need to explicitly build the lua.mo (I didn't add it to the modules makefile yet):
Code: [Select]
make -C modules/lua

However, IIRC the 600D is like 1100D in regards to the fairly small autoexec.bin size limit (someone please correct me if I'm thinking of another camera). All of the extra libc symbols needed by lua push the autoexec.bin size over the limit (at least on 1100D). So you will probably have to disable some stuff in platform/600D.102/features.h to get it to run (to disable a feature, simply #undef the macro for it; for a list, see all_features.h). Look for the MemSiz column in the build output, it should be less than 0x80000 (512KB).

You're looking for something like this:
Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x073918 0x00cf3918 0x00cf3918 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00c80100 0x00c80100 0x73820 0x7ff0c RWE 0x100


For 1100D I usually disable features that take up a lot of space, that I don't need like: DEFISHING_PREVIEW, ANAMORPHIC_PREVIEW, MOTION_DETECT, TRAP_FOCUS, FOLLOW_FOCUS, etc.

Zaimon

  • New to the forum
  • *
  • Posts: 5
Re: Compiling Magic Lantern in the Cloud
« Reply #73 on: April 21, 2015, 12:11:18 PM »
@dmilligan

I built the lua.mo and as you pointed out I needed to reduce the size of the autoexec.bin.
The module seems to work correctly.

Thanks a lot!

dfort

  • Hero Member
  • *****
  • Posts: 2091
Re: Compiling Magic Lantern in the Cloud
« Reply #74 on: June 09, 2015, 05:28:52 AM »
Thank you so much for this. I finally got a way to compile ML from source and had my first pull request merged into unified. Whoo Hoo!

A while back I wanted to compile raw2dng for use on a Macbook Pro and got tripped up because I was following instructions to compile ML. Finally got around it by pulling out just the raw2dng code. Now with dmilligan's guidance we got a simple way to compile ML from source but I am getting tripped up trying to get the raw_rec and dual_iso modules because of this:

http://www.magiclantern.fm/forum/index.php?topic=14725.msg142856#msg142856

Quote from: dmilligan
Yeah, cr2hdr and raw2dng have to be compiled in 32 bit mode, and this VM is 64 bit and doesn't have the 32bit gcc libraries installed. Without sudo privliges, I'm not sure installing them is possible. The only solution really is to modify the makefile for dual_iso module so that it doesn't try to compile cr2hdr (the same for raw_rec which tries to compile raw2dng).

I tried to work my way around it but have failed miserably. Oh well, everything else seems to work fine.

So my question is--is it ok to copy the raw_rec and dual_iso modules from the ML/modules directory of the nightly builds? I tried it and it seems to work but thought I'd ask.
EOSM.202 EOSM.203 EOSM2.103 700D.115 5D3.*