Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: miyake on September 06, 2012, 06:28:40 PM

Title: [MERGED] Makefile with user prefs
Post by: miyake on September 06, 2012, 06:28:40 PM
@all deveropper
Sometime we developer push each environment specific Makefile valiable.
So I think
-separate User prefs to another file.
-load from Makefile.inc
-Do not add Userprefs.inc to repo
-Only add Userprefs.inc.default
  -> all user cp Userprefs.inc.default to Userprefs.inc and edit it each environment.
Then , We never touch Makfile values on the repository.

patch added

diff -r 048012c0f7ad Makefile.inc
--- a/Makefile.inc      Tue Sep 04 12:37:49 2012 +0200
+++ b/Makefile.inc      Fri Sep 07 01:28:45 2012 +0900
@@ -1,50 +1,5 @@
#generic makefile
-
-# for yagarto or the official ARM toolchain use ARM_ABI=none-eabi otherwise use ARM-ABI=elf
-ARM_ABI=none-eabi
-
-ARM_PATH=~/arm-toolchain462
-ARM_BINPATH=$(ARM_PATH)/bin
-GCC_VERSION=4.6.2
-CC=$(ARM_BINPATH)/arm-$(ARM_ABI)-gcc-$(GCC_VERSION)
-OBJCOPY=$(ARM_BINPATH)/arm-$(ARM_ABI)-objcopy
-AR=$(ARM_BINPATH)/arm-$(ARM_ABI)-ar
-RANLIB=$(ARM_BINPATH)/arm-$(ARM_ABI)-ranlib
-READELF=$(ARM_BINPATH)/arm-$(ARM_ABI)-readelf
-ARM_LIBC_A = $(ARM_PATH)/arm-$(ARM_ABI)/lib/libc.a
-LD=$(CC)
-HOST_CC=gcc
-HOST_CFLAGS=-g -O3 -W -Wall
-PYTHON=python2
-
-# Naming convention for Magic Lantern builds:
-# General rules:
-# - Always specify the camera and its firmware version number in the build name (e.g. 550d.fw109)
-# - For non-release builds, specify the build date and author's (nick)name.
-# - For experimental builds, add a short keyword indicating the particular feature tested.
-
-# Examples for experimental builds:
-# magiclantern-2010dec07.550d.fw108.cropmarks.a1ex.zip
-# magiclantern-2010nov23.550d.fw108.selectable-audio.piers.zip
-
-# Example for pre-release builds:
-# magiclantern-2010dec17.550d.fw109.PRERELEASE.alex.zip
-
-# Release builds:
-# magiclantern-0.2.0.rc1.550d.fw109.zip
-#~ VERSION=0.2.0.rc1.550d.fw109
-BUILDVER=$(shell whoami).$(shell hg id -i -r .)
-VERSION:=v2.3.NEXT.$(shell LC_TIME=EN date +'%Y%b%d').$(MODEL)$(FW_VERSION)
-
-CONFIG_PYMITE          = n
-CONFIG_RELOC           = n
-CONFIG_TIMECODE                = n
-CONFIG_PTP                     = n
-CONFIG_PTP_CHDK        = n
-CONFIG_PTP_ML          = n
-CONFIG_PLUGINS                 = n
-CONFIG_CONSOLE         = n
-CONFIG_DEBUGMSG        = 0
+include ../../Userprefs.inc

UNAME:=$(shell uname)

diff -r 048012c0f7ad Userprefs.inc.default
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/Userprefs.inc.default     Fri Sep 07 01:28:45 2012 +0900
@@ -0,0 +1,48 @@
+# for yagarto or the official ARM toolchain use ARM_ABI=none-eabi otherwise use ARM-ABI=elf
+ARM_ABI=none-eabi
+
+ARM_PATH=~/arm-toolchain462
+ARM_BINPATH=$(ARM_PATH)/bin
+GCC_VERSION=4.6.2
+CC=$(ARM_BINPATH)/arm-$(ARM_ABI)-gcc-$(GCC_VERSION)
+OBJCOPY=$(ARM_BINPATH)/arm-$(ARM_ABI)-objcopy
+AR=$(ARM_BINPATH)/arm-$(ARM_ABI)-ar
+RANLIB=$(ARM_BINPATH)/arm-$(ARM_ABI)-ranlib
+READELF=$(ARM_BINPATH)/arm-$(ARM_ABI)-readelf
+ARM_LIBC_A = $(ARM_PATH)/arm-$(ARM_ABI)/lib/libc.a
+LD=$(CC)
+HOST_CC=gcc
+HOST_CFLAGS=-g -O3 -W -Wall
+PYTHON=python2
+
+# Naming convention for Magic Lantern builds:
+# General rules:
+# - Always specify the camera and its firmware version number in the build name (e.g. 550d.fw109)
+# - For non-release builds, specify the build date and author's (nick)name.
+# - For experimental builds, add a short keyword indicating the particular feature tested.
+
+# Examples for experimental builds:
+# magiclantern-2010dec07.550d.fw108.cropmarks.a1ex.zip
+# magiclantern-2010nov23.550d.fw108.selectable-audio.piers.zip
+
+# Example for pre-release builds:
+# magiclantern-2010dec17.550d.fw109.PRERELEASE.alex.zip
+
+# Release builds:
+# magiclantern-0.2.0.rc1.550d.fw109.zip
+#~ VERSION=0.2.0.rc1.550d.fw109
+BUILDVER=$(shell whoami).$(shell hg id -i -r .)
+VERSION:=v2.3.NEXT.$(shell LC_TIME=EN date +'%Y%b%d').$(MODEL)$(FW_VERSION)
+
+CONFIG_PYMITE          = n
+CONFIG_RELOC           = n
+CONFIG_TIMECODE                = n
+CONFIG_PTP                     = n
+CONFIG_PTP_CHDK        = n
+CONFIG_PTP_ML          = n
+CONFIG_PLUGINS                 = n
+CONFIG_CONSOLE         = n
+CONFIG_DEBUGMSG        = 0
+
+
+#Additional CFLAGS here
\ No newline at end of file

Title: Re: Makefile with user prefs
Post by: nanomad on September 06, 2012, 09:14:57 PM
We're on the same line. Too bad you wasted time since I already did it :P

I'll have a look at your patch but I bet it's identical to my fork

edit: actually, mine is a little better since it has a default options file (that should stay in the repo) and an optionally customized one (that it's excluded by mercurial updates). Any options missing from the customized will be loaded from the default one.

Thanks anyway :)
Title: Re: Makefile with user prefs
Post by: miyake on September 07, 2012, 04:27:37 AM
Why didn't you push it yet?
I think you have a permission on main repo.
It's really easy/small-time things.

Title: Re: Makefile with user prefs
Post by: scrax on September 07, 2012, 06:34:37 AM
That's a really good idea.
+1
Title: Re: Makefile with user prefs
Post by: nanomad on September 07, 2012, 09:43:01 AM
I actually forgot to merge it since i had to inform a1ex about it  ::)
Title: Re: Makefile with user prefs
Post by: nanomad on September 07, 2012, 10:01:09 AM
I Went ahead and pushed it. Copy Makefile.user.defaults to Makefile.user and keep only the options that you want to customize
Title: Re: [MERGED] Makefile with user prefs
Post by: miyake on September 07, 2012, 10:32:08 AM
Confirmed .Thankyou  and good work!!
Title: Re: Makefile with user prefs
Post by: Marsu42 on September 07, 2012, 02:27:12 PM
Quote from: nanomad on September 07, 2012, 10:01:09 AM
I Went ahead and pushed it. Copy Makefile.user.defaults to Makefile.user and keep only the options that you want to customize

You should do the same to plugins/Makefile.top, or users still have to change that for the compiler path. Or even better, make the plugins directory pull the compiler variables from the main Makefile.user
Title: Re: [MERGED] Makefile with user prefs
Post by: nanomad on September 07, 2012, 03:11:50 PM
Will do, thanks for the suggestion  ;)

Fixed in rev 5afa67f1c4a7
Title: Re: [MERGED] Makefile with user prefs
Post by: miyake on September 07, 2012, 03:54:19 PM
@nanomad

One more .  Now we can add CFLAGS like this.

diff -r cac1f529216b Makefile.inc
--- a/Makefile.inc      Fri Sep 07 22:43:03 2012 +0900
+++ b/Makefile.inc      Fri Sep 07 22:51:21 2012 +0900
@@ -130,6 +130,8 @@
        -I$(PLATFORM_INC) \
        -I$(SRC_DIR) \

+CFLAGS += $(CFLAG_USER)
+
ifeq ($(CONFIG_PYMITE),y)
CFLAGS += $(PYMITE_CFLAGS)
endif
diff -r cac1f529216b Makefile.user.default
--- a/Makefile.user.default     Fri Sep 07 22:43:03 2012 +0900
+++ b/Makefile.user.default     Fri Sep 07 22:51:21 2012 +0900
@@ -46,3 +46,6 @@
CONFIG_PLUGINS                 = n
CONFIG_CONSOLE         = n
CONFIG_DEBUGMSG        = 0
+
+#CFLAG_USER = -mlong-calls \
+


Title: Re: [MERGED] Makefile with user prefs
Post by: Marsu42 on September 07, 2012, 04:00:45 PM
Quote from: miyake on September 07, 2012, 03:54:19 PM
One more .  Now we can add CFLAGS like this.

+1 .. but global CFLAGS don't seem to be that useful, I tried a compile with global -O3 and it was much larger & didn't run.

Alex suggested different flags only for some modules like this:...

# compile some modules with -O3 for increased speed
zebra.o: $(SRC_DIR)/zebra.c
$(call build,CC,$(CC) $(CFLAGS) -O3 -c -o $@ $<)


... but I still don't know where to put this block (anywhere I tried it was ignored) - any hints :-p ? And of course it would be esp. nice if such compile exceptions would go into Makefile.user, too, so the repository files are untouched locally.
Title: Re: [MERGED] Makefile with user prefs
Post by: nanomad on September 07, 2012, 04:09:57 PM
Merged, thanks

The truth is, ML build system could use some love. I dunno if everything in there is necessary or could be rewritten to be more flexible
Title: Re: [MERGED] Makefile with user prefs
Post by: miyake on September 07, 2012, 04:26:00 PM
Quote from: Marsu42 on September 07, 2012, 04:00:45 PM
+1 .. but global CFLAGS don't seem to be that useful, I tried a compile with global -O3 and it was much larger & didn't run.

-O3 is optimized flag. So I think it will provide "loop spread" and so on. So binary  will bigger...
Anyway, You can add alex's code like this....Maybe....
diff -r cac1f529216b Makefile.inc
--- a/Makefile.inc      Fri Sep 07 22:43:03 2012 +0900
+++ b/Makefile.inc      Fri Sep 07 23:23:32 2012 +0900
@@ -158,6 +161,8 @@
%.bin: %
        $(call build,OBJCOPY,$(OBJCOPY) -O binary $< $@)

+zebra.o: $(SRC_DIR)/zebra.c
+$(call build,CC,$(CC) $(CFLAGS) -O3 -c -o $@ $<)

zip: prepare_zip magiclantern-$(VERSION).zip




Anyway, I'm using -DCONFIG_600D_AUDIO_DEBUG flag. It's very good to adding a specific code all over the repository.
Title: Re: [MERGED] Makefile with user prefs
Post by: Marsu42 on September 07, 2012, 05:13:27 PM
Quote from: miyake on September 07, 2012, 04:26:00 PM
Anyway, You can add alex's code like this....Maybe....

... or maybe not :-p as this was the first thing I tried, it's ignored (check the output w/ V=1). Actually I'm used to use cross-compilers because I hack OpenWrt frequently, but ml's Makefile is really horrible in comparison to OpenWrt's buildroot.

Concerning the new benchmark: How it that supposed to work? On my 60d it just goes into play mode, displays "0 sec = .... fps" and that's that. If I get it to work I'm happy to tell you if my compile with Linaro 4.7 is any faster than the standard arm.
Title: Re: [MERGED] Makefile with user prefs
Post by: a1ex on September 07, 2012, 05:17:06 PM
Enable focus peaking first ;)
Title: Re: [MERGED] Makefile with user prefs
Post by: Marsu42 on September 07, 2012, 05:37:39 PM
Quote from: a1ex on September 07, 2012, 05:17:06 PM
Enable focus peaking first ;)

Ah, well, that's more like it :-> ... but for a benchmark the timing is much too rough, it should have at least some decimal places behind sec and fps.

And to ask for the n-th time, sorry to be a bother: Where do I put the -O3 optimization lines for zebra.o and so on for them to be recognized? If I can't compile an optimized version, providing benchmark results hardly makes any sense.
Title: Re: [MERGED] Makefile with user prefs
Post by: 1% on September 07, 2012, 05:43:21 PM
Did the O3 flags actually optimize anything? I know they were taken out a little while ago. i don't mind the binary being bigger and need a little more speed for the new peaking to be usable.
Title: Re: [MERGED] Makefile with user prefs
Post by: Marsu42 on September 07, 2012, 05:47:00 PM
Quote from: 1% on September 07, 2012, 05:43:21 PM
Did the O3 flags actually optimize anything?

The -O3 I compiled produced an err70 on my 60d, so I can't tell. But according to Alex only the 50d had problems, but for other cameras compiling at least time-critical modules should improve speed. If I knew where to put the -O3 directives in the Makefile, that is - argh.
Title: Re: [MERGED] Makefile with user prefs
Post by: 1% on September 07, 2012, 06:31:33 PM
I put it like this in makefile.inc... I don't know how to add this to makefile.user... is it even taking the flags out of there. I put -Wcomment to try to turn it off but I think I'm doing something wrong:

%.bin: %
   $(call build,OBJCOPY,$(OBJCOPY) -O binary $< $@)
zebra.o: $(SRC_DIR)/zebra.c
   $(call build,CC,$(CC) $(CFLAGS) -O3 -c -o $@ $<)

I guess you can keep adding different modules at the end the same way, but I'm not sure what modules will benefit.
Title: Re: [MERGED] Makefile with user prefs
Post by: miyake on September 07, 2012, 06:42:12 PM
%.o: $(SRC_DIR)/%.c
        $(call build,CC,$(CC) $(CFLAGS) -c -O3 -o $@ $<)


Then I used platform/600D.102/autoexec.bin.

[miyake@MLdev32 magic-lantern600daudio]$ ll -h platform/all/autoexec.bin
-rwxrwxr-x. 1 miyake miyake 2.4M Sep  8 01:18 platform/all/autoexec.bin
[miyake@MLdev32 magic-lantern600daudio]$ ll -h platform/600D.102/autoexec.bin
-rwxrwxr-x. 1 miyake miyake 420K Sep  8 01:17 platform/600D.102/autoexec.bin


result
23sec => 43fps

Without -O3
[miyake@MLdev32 magic-lantern600daudio]$ ll -h platform/all/autoexec.bin
-rwxrwxr-x. 1 miyake miyake 1.8M Sep  8 01:39 platform/all/autoexec.bin
[miyake@MLdev32 magic-lantern600daudio]$ ll -h platform/600D.102/autoexec.bin
-rwxrwxr-x. 1 miyake miyake 308K Sep  8 01:38 platform/600D.102/autoexec.bin


result
25sec=>40fps
Title: Re: [MERGED] Makefile with user prefs
Post by: 1% on September 07, 2012, 07:34:40 PM
I'll try it for all modules since it seems to help.

%.o: $(PLATFORM_DIR)/%.c
$(call build,CC,$(CC) $(CFLAGS) -c -O3 -o $@ $<)


Also did that, trying it. Bin is 400K

24sec -> 41 FPS

Just the core modules better? Same, no difference? Everything does seem zippier.

Does it only test old peaking methods?
Title: Re: [MERGED] Makefile with user prefs
Post by: Marsu42 on September 08, 2012, 12:14:20 AM
Quote from: 1% on September 07, 2012, 06:31:33 PM
I put it like this in makefile.inc... I don't know how to add this to makefile.user... is it even taking the flags out of there. I put -Wcomment to try to turn it off but I think I'm doing something wrong

Just adding zebra.o with -O3 to Makefile.inc does nothing - do make V=1 and check what parameters reach gcc. The only thing that does work is editing Makefile.inc and change the global flags, and the whole -O3 compile crashes my 60d.

edit: I created a new topic for benchmarks, since this one was really about user prefs in the makefile ... see http://www.magiclantern.fm/forum/index.php?topic=2609.0
Title: Re: [MERGED] Makefile with user prefs
Post by: miyake on September 08, 2012, 09:59:11 AM
@nanomad
Anyway, Thankyou , I'm deleted all difference  in Makefile.inc on our 600D audio repos.

Title: Re: [MERGED] Makefile with user prefs
Post by: miyake on September 16, 2012, 08:22:01 PM
@nanomad

I don't know why, But current position of CFLAG_USER doesn't work.
Please move CFLAG_USER to the final line in Makefile.user.default.


Title: Re: [MERGED] Makefile with user prefs
Post by: nanomad on September 19, 2012, 02:52:57 PM
Should be fixed in the latest revision, thanks.
Keep in mind that Makefile.user should contain only values that are different from default (and not everything)