Magic Lantern Forum

Using Magic Lantern => General Help Q&A => Topic started by: Dunc101 on January 01, 2015, 04:45:08 PM

Title: Compiling and changing the source code of Magic Lantern
Post by: Dunc101 on January 01, 2015, 04:45:08 PM
Hi,

I am trying to compile the source code for the 600D to make some small changes to the code from an old build.
I have downloaded Hg and Virtual Box but not quite sure the steps to getting and changing the source code.

I managed to do a autoexec.bin compile but can't work out how to make changes to the code.

I have a build with some specific features on it from 2013 that works on the 550D but I need to make it work on the 600D.
Any help on this would be great,

Thanks

D
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: dmilligan on January 01, 2015, 07:52:05 PM
Quote from: Dunc101 on January 01, 2015, 04:45:08 PM
but can't work out how to make changes to the code.
Well, you just open it in a text editor and change it. But, you sort of have to know how to code first. I assume that if you did know how to code, you probably wouldn't be asking this. Learning how to code is fairly easy (http://code.org). But a lot of ML stuff is rather complicated and requires much more than basic coding knowledge, and may also require reverse engineering. And even an advanced coder it make take some time just to get familiar with the ML code base (which is rather large and not particularly well documented). It just depends on what you're trying to do.

Quote from: Dunc101 on January 01, 2015, 04:45:08 PM
I have a build with some specific features on it from 2013 that works on the 550D but I need to make it work on the 600D.
Depending on what it is, this could be trivial or extremely difficult, or even impossible. Porting features from one camera to another often requires some reverse engineering (or at least finding stubs (http://www.magiclantern.fm/forum/index.php?topic=12177.0)). Perhaps you should specify what specifically you are trying to do.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: DigitalVeil on January 02, 2015, 02:58:19 AM
Quote from: dmilligan on January 01, 2015, 07:52:05 PM
Well, you just open it in a text editor and change it. But, you sort of have to know how to code first. I assume that if you did know how to code, you probably wouldn't be asking this. Learning how to code is fairly easy (http://code.org). But a lot of ML stuff is rather complicated and requires much more than basic coding knowledge, and may also require reverse engineering. And even an advanced coder it make take some time just to get familiar with the ML code base (which is rather large and not particularly well documented). It just depends on what you're trying to do.

How about something small like changing the maximum h264 bit rate value that can be chosen? I only have basic coding knowledge and I haven't even looked at the ML source code yet, but that would be the first thing I'd try. Would that be a particularly difficult thing for a noob programmer to alter without breaking everything?
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: dmilligan on January 02, 2015, 03:12:17 AM
I imagine that the highest bit rate ML allows is not arbitrary, but rather, it is the largest that the Canon firmware/hardware encoder will accept (without throwing an error or just not working).

In general, doing things very closely tied to the camera hardware and Canon firmware are the most difficult (b/c they require reverse engineering and working with undocumented firmware and hardware).

The easy stuff and where to get your feet wet would be things like these:
http://www.magiclantern.fm/forum/index.php?topic=9625.0
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Dunc101 on January 05, 2015, 10:53:27 PM
Hi Dmilligan

Thanks for all this, I basically just need to add some extra times to the stop after feature and that is already on my version of the 600D version.

What do you actually edit in the text editor, the autoexec.bin?
What text editor do you use?

Cheers
D
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: dmilligan on January 06, 2015, 02:28:49 AM
You edit the source code files (the .c and .h files) in a text editor. You can use any text editor you like, one with syntax highlighting and auto-indent is nice (like notepad++, or jedit).

Then after you make your changes, you recompile and the compiler spits out a new autoexec.bin.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Dunc101 on January 07, 2015, 12:16:09 PM
Can you convert an autoexec.bin into source code?
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Danne on January 07, 2015, 01:18:45 PM
I like this thread. Since dmilligan is on it I,d like to ask some noob questions myself.
Editing .c-files, like dng.c in mlv_dump or raw2dng, cr2hdr etc. Will the workflow be the same as editing the auto.exe?
You gave me an answer to edit the chdk metadata info and apply your cdng info instead. Would this simply be a matter of copy paste your cdng implementation to a1ex and g3gg0 binaries or how would one proceed to get this done?
I read about using xcode but do you have any kind of recommendation how recompiling in mac environment?
I get the impression changing chdk to cdng metadata is complicated since it hasn,t been done yet by anyone in the original binaries.

Thanks
/D
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: dmilligan on January 07, 2015, 01:55:29 PM
Quote from: Dunc101 on January 07, 2015, 12:16:09 PM
Can you convert an autoexec.bin into source code?
No, not back to the original C code. You can decompile it into assembly language (http://en.wikipedia.org/wiki/Assembly_language), but that is more or less pointless (you might as well start from scratch analyzing the Canon firmware).

Quote from: Danne on January 07, 2015, 01:18:45 PM
You gave me an answer to edit the chdk metadata info and apply your cdng info instead. Would this simply be a matter of copy paste your cdng implementation to a1ex and g3gg0 binaries or how would one proceed to get this done?
No, it will be a fair amount of work. I have actually been working on this myself for some time. I think I have it mostly done for in camera (though not tested) as a module, but I would also like to update post processing tools like cr2hdr and mlv_dump, which I haven't gotten to yet (one big issue is that neither of these will compile with Apple's C compiler, and last time I tried to compile GCC, it failed and I gave up).
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Danne on January 07, 2015, 02:04:19 PM
Thanks for clarifying.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Dunc101 on January 07, 2015, 02:31:16 PM
Thanks for this Dmilligan,

Ok, Would it be possible for someone to help me add an existing feature that was working successfully on the T2i and get it onto the T3i? Quite a few of you helped I believe back last year.

Cheers
D
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: DeafEyeJedi on January 07, 2015, 09:12:51 PM
@dmilligan Did it not work well with MagHg?

@Danne -- which text editor do you normally use for OS X?
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Danne on January 07, 2015, 09:16:00 PM
I use the native text editor in mavericks  :P
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: DeafEyeJedi on January 07, 2015, 09:18:49 PM
Yeah I figured but to pull and compile which do you use?

I'm starting to regret upgrading to Yosemite...

Ugh!
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Danne on January 07, 2015, 09:28:02 PM
Can, t compile. Tried once with xcode and so on but never got it working. I run bash commands (terminal) in conjunction with automator. Badabim badaboom :)
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: dmilligan on January 07, 2015, 10:40:25 PM
Quote from: DeafEyeJedi on January 07, 2015, 09:18:49 PM
Yeah I figured but to pull and compile which do you use?
You need a different program to pull, to edit, and to compile.

I use SourceTree to do version control things (pull, track changes, commit, push, etc.). There are other mercurial GUI alternatives, and you can always just use the command line.

I use Xcode as a text editor to edit source files (because it has nice stuff like syntax highlighting, autocomplete, jump to definition, etc.).

You have to use 'make' to compile everything (and the compiling is actually done by the GCC ARM 'toolchain', which you must have setup correctly).

MacHG is not a compiler, nor is it really a text editor. It's simply for doing version control stuff.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: DeafEyeJedi on January 08, 2015, 08:27:35 PM
Thanks for your response @dmilligan -- I apologiZe for the confusion between these two apps.

I'll look into SourceTree as soon as I get back from being on the road for work. Should be off this weekend to do so.

However, I do also own Xcode but I'm afraid the my GCC ARM 'tool chain' may not have been set Up properly because I believe I got a few errors in Terminal when trying to run/install from a guide I found through ML wiki (forgot the link but will get back to you on that) unless you know of a better step by step guide?

So I can confirm that it is in fact set  properly before going ahead with it...

Heck I may need to call one of my old buddies from HS to come by and look into the codings.

I know I'm lame like that! :)
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Danne on January 15, 2015, 07:35:54 PM
Quote from: dmilligan on January 07, 2015, 01:55:29 PM
I think I have it mostly done for in camera (though not tested) as a module,

@dmilligan
Are you saying you programmed a module which creates cdng files straight to memory card in camera?
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: dmilligan on January 15, 2015, 10:10:12 PM
Yes (https://bitbucket.org/hudson/magic-lantern/pull-request/603/dng-module-wip/diff), but they won't be Premiere/Resolve compatible because they will 14 bit. Really, I'm just slightly improving the existing dng code.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Danne on January 15, 2015, 11:04:38 PM
Nice work David.
Resolve works with mlv_dump, raw2dng 16-bit dng files. Should it not work with 14-bit too? Too bad it,s not for premiere but then again there is mlvfs :).
Do you by any chance know a way to rewrite metadata with cdng structure replacing the dng (subifd etc with ifdo, tiff tags) directly to a 16-bit dng? Exiftool is a bit pecky about what to erase and not. 
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: dmilligan on January 15, 2015, 11:55:31 PM
Quote from: Danne on January 15, 2015, 11:04:38 PM
Nice work David.
Resolve works with mlv_dump, raw2dng 16-bit dng files. Should it not work with 14-bit too?
I'm not really sure if Resolve can read 14-bit files or not. I've never tried, but I assumed it was the same as Premiere, but maybe it can.

Quote from: Danne on January 15, 2015, 11:04:38 PM
Do you by any chance know a way to rewrite metadata with cdng structure replacing the dng (subifd etc with ifdo, tiff tags) directly to a 16-bit dng? Exiftool is a bit pecky about what to erase and not. 
Do you mean convert a 14 bit DNG to a 16 bit CDNG? AFAIK, not possible without writing a lot of code to read and write the DNGs manually.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Danne on January 16, 2015, 05:09:53 AM
Thanks. Resolve works with the original dng files produced by mlv_dump and raw2dng so the module will probably be very handy for this workflow. They,re still handled as 10-bit I think.
Thanks for answer on the dng.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: Danne on January 16, 2015, 08:19:35 AM
Actually I was thinking more in terms of going to 12 or 10 bit dng to have them work in premiere. I guess it is deeper coding knowledge needed since exiftool can, t erase certain meta data info.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: N/A on February 11, 2015, 05:58:20 PM
Quote from: dmilligan on January 07, 2015, 01:55:29 PM
(one big issue is that neither of these will compile with Apple's C compiler, and last time I tried to compile GCC, it failed and I gave up).
How did you manage to get it working? I had success with "brew install gcc49" and simply symlinked it to /usr/bin/. To compile dcraw 9.23 for mac I found a command to build with "llvm-gcc" and just changed it to "gcc". Runs pretty well, considering its a 2009 MacBook.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: dmilligan on February 11, 2015, 10:54:28 PM
Yeah I used brew, I think when I tried it before there was some issue with gcc that they have since fixed. Either that, or I may have tried with macports which tried to compile it from source, which failed. I'm pretty sure brew uses precompiled binaries for most stuff. At any rate, on my system now I have 'gcc' = llvm compiler (there by default with Xcode), 'gcc-4.9' = actual gcc.
Title: Re: Compiling and changing the source code of Magic Lantern
Post by: N/A on February 12, 2015, 11:47:34 AM
Port didn't work for me either, got to the building gcc status and never progressed after 2 hours. On another note, I compiled ml_dng for 7d, I'll pm you when I put it through the paces. Had to list dng before silent in the makefile.modules.default to get silent to build, a bug or something I messed up?