Author Topic: Compiling and changing the source code of Magic Lantern  (Read 8242 times)

Dunc101

  • Freshman
  • **
  • Posts: 92
Compiling and changing the source code of Magic Lantern
« on: January 01, 2015, 04:45:08 PM »
Hi,

I am trying to compile the source code for the 600D to make some small changes to the code from an old build.
I have downloaded Hg and Virtual Box but not quite sure the steps to getting and changing the source code.

I managed to do a autoexec.bin compile but can't work out how to make changes to the code.

I have a build with some specific features on it from 2013 that works on the 550D but I need to make it work on the 600D.
Any help on this would be great,

Thanks

D

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Compiling and changing the source code of Magic Lantern
« Reply #1 on: January 01, 2015, 07:52:05 PM »
but can't work out how to make changes to the code.
Well, you just open it in a text editor and change it. But, you sort of have to know how to code first. I assume that if you did know how to code, you probably wouldn't be asking this. Learning how to code is fairly easy. But a lot of ML stuff is rather complicated and requires much more than basic coding knowledge, and may also require reverse engineering. And even an advanced coder it make take some time just to get familiar with the ML code base (which is rather large and not particularly well documented). It just depends on what you're trying to do.

I have a build with some specific features on it from 2013 that works on the 550D but I need to make it work on the 600D.
Depending on what it is, this could be trivial or extremely difficult, or even impossible. Porting features from one camera to another often requires some reverse engineering (or at least finding stubs). Perhaps you should specify what specifically you are trying to do.

DigitalVeil

  • Member
  • ***
  • Posts: 110
  • 70D, 700D
Re: Compiling and changing the source code of Magic Lantern
« Reply #2 on: January 02, 2015, 02:58:19 AM »
Well, you just open it in a text editor and change it. But, you sort of have to know how to code first. I assume that if you did know how to code, you probably wouldn't be asking this. Learning how to code is fairly easy. But a lot of ML stuff is rather complicated and requires much more than basic coding knowledge, and may also require reverse engineering. And even an advanced coder it make take some time just to get familiar with the ML code base (which is rather large and not particularly well documented). It just depends on what you're trying to do.

How about something small like changing the maximum h264 bit rate value that can be chosen? I only have basic coding knowledge and I haven't even looked at the ML source code yet, but that would be the first thing I'd try. Would that be a particularly difficult thing for a noob programmer to alter without breaking everything?
Glass: EF-S 24mm f/2.8 STM, EF 50mm f/1.8, EF-S 55-250mm IS STM, EF-S 18-55mm IS STM

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Compiling and changing the source code of Magic Lantern
« Reply #3 on: January 02, 2015, 03:12:17 AM »
I imagine that the highest bit rate ML allows is not arbitrary, but rather, it is the largest that the Canon firmware/hardware encoder will accept (without throwing an error or just not working).

In general, doing things very closely tied to the camera hardware and Canon firmware are the most difficult (b/c they require reverse engineering and working with undocumented firmware and hardware).

The easy stuff and where to get your feet wet would be things like these:
http://www.magiclantern.fm/forum/index.php?topic=9625.0

Dunc101

  • Freshman
  • **
  • Posts: 92
Re: Compiling and changing the source code of Magic Lantern
« Reply #4 on: January 05, 2015, 10:53:27 PM »
Hi Dmilligan

Thanks for all this, I basically just need to add some extra times to the stop after feature and that is already on my version of the 600D version.

What do you actually edit in the text editor, the autoexec.bin?
What text editor do you use?

Cheers
D

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Compiling and changing the source code of Magic Lantern
« Reply #5 on: January 06, 2015, 02:28:49 AM »
You edit the source code files (the .c and .h files) in a text editor. You can use any text editor you like, one with syntax highlighting and auto-indent is nice (like notepad++, or jedit).

Then after you make your changes, you recompile and the compiler spits out a new autoexec.bin.

Dunc101

  • Freshman
  • **
  • Posts: 92
Re: Compiling and changing the source code of Magic Lantern
« Reply #6 on: January 07, 2015, 12:16:09 PM »
Can you convert an autoexec.bin into source code?

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5906
Re: Compiling and changing the source code of Magic Lantern
« Reply #7 on: January 07, 2015, 01:18:45 PM »
I like this thread. Since dmilligan is on it I,d like to ask some noob questions myself.
Editing .c-files, like dng.c in mlv_dump or raw2dng, cr2hdr etc. Will the workflow be the same as editing the auto.exe?
You gave me an answer to edit the chdk metadata info and apply your cdng info instead. Would this simply be a matter of copy paste your cdng implementation to a1ex and g3gg0 binaries or how would one proceed to get this done?
I read about using xcode but do you have any kind of recommendation how recompiling in mac environment?
I get the impression changing chdk to cdng metadata is complicated since it hasn,t been done yet by anyone in the original binaries.

Thanks
/D

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Compiling and changing the source code of Magic Lantern
« Reply #8 on: January 07, 2015, 01:55:29 PM »
Can you convert an autoexec.bin into source code?
No, not back to the original C code. You can decompile it into assembly language, but that is more or less pointless (you might as well start from scratch analyzing the Canon firmware).

You gave me an answer to edit the chdk metadata info and apply your cdng info instead. Would this simply be a matter of copy paste your cdng implementation to a1ex and g3gg0 binaries or how would one proceed to get this done?
No, it will be a fair amount of work. I have actually been working on this myself for some time. I think I have it mostly done for in camera (though not tested) as a module, but I would also like to update post processing tools like cr2hdr and mlv_dump, which I haven't gotten to yet (one big issue is that neither of these will compile with Apple's C compiler, and last time I tried to compile GCC, it failed and I gave up).

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5906
Re: Compiling and changing the source code of Magic Lantern
« Reply #9 on: January 07, 2015, 02:04:19 PM »
Thanks for clarifying.

Dunc101

  • Freshman
  • **
  • Posts: 92
Re: Compiling and changing the source code of Magic Lantern
« Reply #10 on: January 07, 2015, 02:31:16 PM »
Thanks for this Dmilligan,

Ok, Would it be possible for someone to help me add an existing feature that was working successfully on the T2i and get it onto the T3i? Quite a few of you helped I believe back last year.

Cheers
D

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3307
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Compiling and changing the source code of Magic Lantern
« Reply #11 on: January 07, 2015, 09:12:51 PM »
@dmilligan Did it not work well with MagHg?

@Danne -- which text editor do you normally use for OS X?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5906
Re: Compiling and changing the source code of Magic Lantern
« Reply #12 on: January 07, 2015, 09:16:00 PM »
I use the native text editor in mavericks  :P

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3307
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Compiling and changing the source code of Magic Lantern
« Reply #13 on: January 07, 2015, 09:18:49 PM »
Yeah I figured but to pull and compile which do you use?

I'm starting to regret upgrading to Yosemite...

Ugh!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5906
Re: Compiling and changing the source code of Magic Lantern
« Reply #14 on: January 07, 2015, 09:28:02 PM »
Can, t compile. Tried once with xcode and so on but never got it working. I run bash commands (terminal) in conjunction with automator. Badabim badaboom :)

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Compiling and changing the source code of Magic Lantern
« Reply #15 on: January 07, 2015, 10:40:25 PM »
Yeah I figured but to pull and compile which do you use?
You need a different program to pull, to edit, and to compile.

I use SourceTree to do version control things (pull, track changes, commit, push, etc.). There are other mercurial GUI alternatives, and you can always just use the command line.

I use Xcode as a text editor to edit source files (because it has nice stuff like syntax highlighting, autocomplete, jump to definition, etc.).

You have to use 'make' to compile everything (and the compiling is actually done by the GCC ARM 'toolchain', which you must have setup correctly).

MacHG is not a compiler, nor is it really a text editor. It's simply for doing version control stuff.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3307
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Compiling and changing the source code of Magic Lantern
« Reply #16 on: January 08, 2015, 08:27:35 PM »
Thanks for your response @dmilligan -- I apologiZe for the confusion between these two apps.

I'll look into SourceTree as soon as I get back from being on the road for work. Should be off this weekend to do so.

However, I do also own Xcode but I'm afraid the my GCC ARM 'tool chain' may not have been set Up properly because I believe I got a few errors in Terminal when trying to run/install from a guide I found through ML wiki (forgot the link but will get back to you on that) unless you know of a better step by step guide?

So I can confirm that it is in fact set  properly before going ahead with it...

Heck I may need to call one of my old buddies from HS to come by and look into the codings.

I know I'm lame like that! :)
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5906
Re: Compiling and changing the source code of Magic Lantern
« Reply #17 on: January 15, 2015, 07:35:54 PM »
I think I have it mostly done for in camera (though not tested) as a module,

@dmilligan
Are you saying you programmed a module which creates cdng files straight to memory card in camera?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Compiling and changing the source code of Magic Lantern
« Reply #18 on: January 15, 2015, 10:10:12 PM »
Yes, but they won't be Premiere/Resolve compatible because they will 14 bit. Really, I'm just slightly improving the existing dng code.

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5906
Re: Compiling and changing the source code of Magic Lantern
« Reply #19 on: January 15, 2015, 11:04:38 PM »
Nice work David.
Resolve works with mlv_dump, raw2dng 16-bit dng files. Should it not work with 14-bit too? Too bad it,s not for premiere but then again there is mlvfs :).
Do you by any chance know a way to rewrite metadata with cdng structure replacing the dng (subifd etc with ifdo, tiff tags) directly to a 16-bit dng? Exiftool is a bit pecky about what to erase and not. 

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Compiling and changing the source code of Magic Lantern
« Reply #20 on: January 15, 2015, 11:55:31 PM »
Nice work David.
Resolve works with mlv_dump, raw2dng 16-bit dng files. Should it not work with 14-bit too?
I'm not really sure if Resolve can read 14-bit files or not. I've never tried, but I assumed it was the same as Premiere, but maybe it can.

Do you by any chance know a way to rewrite metadata with cdng structure replacing the dng (subifd etc with ifdo, tiff tags) directly to a 16-bit dng? Exiftool is a bit pecky about what to erase and not. 
Do you mean convert a 14 bit DNG to a 16 bit CDNG? AFAIK, not possible without writing a lot of code to read and write the DNGs manually.

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5906
Re: Compiling and changing the source code of Magic Lantern
« Reply #21 on: January 16, 2015, 05:09:53 AM »
Thanks. Resolve works with the original dng files produced by mlv_dump and raw2dng so the module will probably be very handy for this workflow. They,re still handled as 10-bit I think.
Thanks for answer on the dng.

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5906
Re: Compiling and changing the source code of Magic Lantern
« Reply #22 on: January 16, 2015, 08:19:35 AM »
Actually I was thinking more in terms of going to 12 or 10 bit dng to have them work in premiere. I guess it is deeper coding knowledge needed since exiftool can, t erase certain meta data info.

N/A

  • Hero Member
  • *****
  • Posts: 576
  • Dreaming in 14 bit
Re: Compiling and changing the source code of Magic Lantern
« Reply #23 on: February 11, 2015, 05:58:20 PM »
(one big issue is that neither of these will compile with Apple's C compiler, and last time I tried to compile GCC, it failed and I gave up).
How did you manage to get it working? I had success with "brew install gcc49" and simply symlinked it to /usr/bin/. To compile dcraw 9.23 for mac I found a command to build with "llvm-gcc" and just changed it to "gcc". Runs pretty well, considering its a 2009 MacBook.
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Compiling and changing the source code of Magic Lantern
« Reply #24 on: February 11, 2015, 10:54:28 PM »
Yeah I used brew, I think when I tried it before there was some issue with gcc that they have since fixed. Either that, or I may have tried with macports which tried to compile it from source, which failed. I'm pretty sure brew uses precompiled binaries for most stuff. At any rate, on my system now I have 'gcc' = llvm compiler (there by default with Xcode), 'gcc-4.9' = actual gcc.