[MERGED] Makefile with user prefs

Started by miyake, September 06, 2012, 06:28:40 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

miyake

@all deveropper
Sometime we developer push each environment specific Makefile valiable.
So I think
-separate User prefs to another file.
-load from Makefile.inc
-Do not add Userprefs.inc to repo
-Only add Userprefs.inc.default
  -> all user cp Userprefs.inc.default to Userprefs.inc and edit it each environment.
Then , We never touch Makfile values on the repository.

patch added

diff -r 048012c0f7ad Makefile.inc
--- a/Makefile.inc      Tue Sep 04 12:37:49 2012 +0200
+++ b/Makefile.inc      Fri Sep 07 01:28:45 2012 +0900
@@ -1,50 +1,5 @@
#generic makefile
-
-# for yagarto or the official ARM toolchain use ARM_ABI=none-eabi otherwise use ARM-ABI=elf
-ARM_ABI=none-eabi
-
-ARM_PATH=~/arm-toolchain462
-ARM_BINPATH=$(ARM_PATH)/bin
-GCC_VERSION=4.6.2
-CC=$(ARM_BINPATH)/arm-$(ARM_ABI)-gcc-$(GCC_VERSION)
-OBJCOPY=$(ARM_BINPATH)/arm-$(ARM_ABI)-objcopy
-AR=$(ARM_BINPATH)/arm-$(ARM_ABI)-ar
-RANLIB=$(ARM_BINPATH)/arm-$(ARM_ABI)-ranlib
-READELF=$(ARM_BINPATH)/arm-$(ARM_ABI)-readelf
-ARM_LIBC_A = $(ARM_PATH)/arm-$(ARM_ABI)/lib/libc.a
-LD=$(CC)
-HOST_CC=gcc
-HOST_CFLAGS=-g -O3 -W -Wall
-PYTHON=python2
-
-# Naming convention for Magic Lantern builds:
-# General rules:
-# - Always specify the camera and its firmware version number in the build name (e.g. 550d.fw109)
-# - For non-release builds, specify the build date and author's (nick)name.
-# - For experimental builds, add a short keyword indicating the particular feature tested.
-
-# Examples for experimental builds:
-# magiclantern-2010dec07.550d.fw108.cropmarks.a1ex.zip
-# magiclantern-2010nov23.550d.fw108.selectable-audio.piers.zip
-
-# Example for pre-release builds:
-# magiclantern-2010dec17.550d.fw109.PRERELEASE.alex.zip
-
-# Release builds:
-# magiclantern-0.2.0.rc1.550d.fw109.zip
-#~ VERSION=0.2.0.rc1.550d.fw109
-BUILDVER=$(shell whoami).$(shell hg id -i -r .)
-VERSION:=v2.3.NEXT.$(shell LC_TIME=EN date +'%Y%b%d').$(MODEL)$(FW_VERSION)
-
-CONFIG_PYMITE          = n
-CONFIG_RELOC           = n
-CONFIG_TIMECODE                = n
-CONFIG_PTP                     = n
-CONFIG_PTP_CHDK        = n
-CONFIG_PTP_ML          = n
-CONFIG_PLUGINS                 = n
-CONFIG_CONSOLE         = n
-CONFIG_DEBUGMSG        = 0
+include ../../Userprefs.inc

UNAME:=$(shell uname)

diff -r 048012c0f7ad Userprefs.inc.default
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/Userprefs.inc.default     Fri Sep 07 01:28:45 2012 +0900
@@ -0,0 +1,48 @@
+# for yagarto or the official ARM toolchain use ARM_ABI=none-eabi otherwise use ARM-ABI=elf
+ARM_ABI=none-eabi
+
+ARM_PATH=~/arm-toolchain462
+ARM_BINPATH=$(ARM_PATH)/bin
+GCC_VERSION=4.6.2
+CC=$(ARM_BINPATH)/arm-$(ARM_ABI)-gcc-$(GCC_VERSION)
+OBJCOPY=$(ARM_BINPATH)/arm-$(ARM_ABI)-objcopy
+AR=$(ARM_BINPATH)/arm-$(ARM_ABI)-ar
+RANLIB=$(ARM_BINPATH)/arm-$(ARM_ABI)-ranlib
+READELF=$(ARM_BINPATH)/arm-$(ARM_ABI)-readelf
+ARM_LIBC_A = $(ARM_PATH)/arm-$(ARM_ABI)/lib/libc.a
+LD=$(CC)
+HOST_CC=gcc
+HOST_CFLAGS=-g -O3 -W -Wall
+PYTHON=python2
+
+# Naming convention for Magic Lantern builds:
+# General rules:
+# - Always specify the camera and its firmware version number in the build name (e.g. 550d.fw109)
+# - For non-release builds, specify the build date and author's (nick)name.
+# - For experimental builds, add a short keyword indicating the particular feature tested.
+
+# Examples for experimental builds:
+# magiclantern-2010dec07.550d.fw108.cropmarks.a1ex.zip
+# magiclantern-2010nov23.550d.fw108.selectable-audio.piers.zip
+
+# Example for pre-release builds:
+# magiclantern-2010dec17.550d.fw109.PRERELEASE.alex.zip
+
+# Release builds:
+# magiclantern-0.2.0.rc1.550d.fw109.zip
+#~ VERSION=0.2.0.rc1.550d.fw109
+BUILDVER=$(shell whoami).$(shell hg id -i -r .)
+VERSION:=v2.3.NEXT.$(shell LC_TIME=EN date +'%Y%b%d').$(MODEL)$(FW_VERSION)
+
+CONFIG_PYMITE          = n
+CONFIG_RELOC           = n
+CONFIG_TIMECODE                = n
+CONFIG_PTP                     = n
+CONFIG_PTP_CHDK        = n
+CONFIG_PTP_ML          = n
+CONFIG_PLUGINS                 = n
+CONFIG_CONSOLE         = n
+CONFIG_DEBUGMSG        = 0
+
+
+#Additional CFLAGS here
\ No newline at end of file


nanomad

We're on the same line. Too bad you wasted time since I already did it :P

I'll have a look at your patch but I bet it's identical to my fork

edit: actually, mine is a little better since it has a default options file (that should stay in the repo) and an optionally customized one (that it's excluded by mercurial updates). Any options missing from the customized will be loaded from the default one.

Thanks anyway :)
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

miyake

Why didn't you push it yet?
I think you have a permission on main repo.
It's really easy/small-time things.


scrax

I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

nanomad

I actually forgot to merge it since i had to inform a1ex about it  ::)
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

nanomad

I Went ahead and pushed it. Copy Makefile.user.defaults to Makefile.user and keep only the options that you want to customize
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

miyake


Marsu42

Quote from: nanomad on September 07, 2012, 10:01:09 AM
I Went ahead and pushed it. Copy Makefile.user.defaults to Makefile.user and keep only the options that you want to customize

You should do the same to plugins/Makefile.top, or users still have to change that for the compiler path. Or even better, make the plugins directory pull the compiler variables from the main Makefile.user

nanomad

Will do, thanks for the suggestion  ;)

Fixed in rev 5afa67f1c4a7
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

miyake

@nanomad

One more .  Now we can add CFLAGS like this.

diff -r cac1f529216b Makefile.inc
--- a/Makefile.inc      Fri Sep 07 22:43:03 2012 +0900
+++ b/Makefile.inc      Fri Sep 07 22:51:21 2012 +0900
@@ -130,6 +130,8 @@
        -I$(PLATFORM_INC) \
        -I$(SRC_DIR) \

+CFLAGS += $(CFLAG_USER)
+
ifeq ($(CONFIG_PYMITE),y)
CFLAGS += $(PYMITE_CFLAGS)
endif
diff -r cac1f529216b Makefile.user.default
--- a/Makefile.user.default     Fri Sep 07 22:43:03 2012 +0900
+++ b/Makefile.user.default     Fri Sep 07 22:51:21 2012 +0900
@@ -46,3 +46,6 @@
CONFIG_PLUGINS                 = n
CONFIG_CONSOLE         = n
CONFIG_DEBUGMSG        = 0
+
+#CFLAG_USER = -mlong-calls \
+



Marsu42

Quote from: miyake on September 07, 2012, 03:54:19 PM
One more .  Now we can add CFLAGS like this.

+1 .. but global CFLAGS don't seem to be that useful, I tried a compile with global -O3 and it was much larger & didn't run.

Alex suggested different flags only for some modules like this:...

# compile some modules with -O3 for increased speed
zebra.o: $(SRC_DIR)/zebra.c
$(call build,CC,$(CC) $(CFLAGS) -O3 -c -o $@ $<)


... but I still don't know where to put this block (anywhere I tried it was ignored) - any hints :-p ? And of course it would be esp. nice if such compile exceptions would go into Makefile.user, too, so the repository files are untouched locally.

nanomad

Merged, thanks

The truth is, ML build system could use some love. I dunno if everything in there is necessary or could be rewritten to be more flexible
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

miyake

Quote from: Marsu42 on September 07, 2012, 04:00:45 PM
+1 .. but global CFLAGS don't seem to be that useful, I tried a compile with global -O3 and it was much larger & didn't run.

-O3 is optimized flag. So I think it will provide "loop spread" and so on. So binary  will bigger...
Anyway, You can add alex's code like this....Maybe....
diff -r cac1f529216b Makefile.inc
--- a/Makefile.inc      Fri Sep 07 22:43:03 2012 +0900
+++ b/Makefile.inc      Fri Sep 07 23:23:32 2012 +0900
@@ -158,6 +161,8 @@
%.bin: %
        $(call build,OBJCOPY,$(OBJCOPY) -O binary $< $@)

+zebra.o: $(SRC_DIR)/zebra.c
+$(call build,CC,$(CC) $(CFLAGS) -O3 -c -o $@ $<)

zip: prepare_zip magiclantern-$(VERSION).zip




Anyway, I'm using -DCONFIG_600D_AUDIO_DEBUG flag. It's very good to adding a specific code all over the repository.

Marsu42

Quote from: miyake on September 07, 2012, 04:26:00 PM
Anyway, You can add alex's code like this....Maybe....

... or maybe not :-p as this was the first thing I tried, it's ignored (check the output w/ V=1). Actually I'm used to use cross-compilers because I hack OpenWrt frequently, but ml's Makefile is really horrible in comparison to OpenWrt's buildroot.

Concerning the new benchmark: How it that supposed to work? On my 60d it just goes into play mode, displays "0 sec = .... fps" and that's that. If I get it to work I'm happy to tell you if my compile with Linaro 4.7 is any faster than the standard arm.

a1ex


Marsu42

Quote from: a1ex on September 07, 2012, 05:17:06 PM
Enable focus peaking first ;)

Ah, well, that's more like it :-> ... but for a benchmark the timing is much too rough, it should have at least some decimal places behind sec and fps.

And to ask for the n-th time, sorry to be a bother: Where do I put the -O3 optimization lines for zebra.o and so on for them to be recognized? If I can't compile an optimized version, providing benchmark results hardly makes any sense.

1%

Did the O3 flags actually optimize anything? I know they were taken out a little while ago. i don't mind the binary being bigger and need a little more speed for the new peaking to be usable.

Marsu42

Quote from: 1% on September 07, 2012, 05:43:21 PM
Did the O3 flags actually optimize anything?

The -O3 I compiled produced an err70 on my 60d, so I can't tell. But according to Alex only the 50d had problems, but for other cameras compiling at least time-critical modules should improve speed. If I knew where to put the -O3 directives in the Makefile, that is - argh.

1%

I put it like this in makefile.inc... I don't know how to add this to makefile.user... is it even taking the flags out of there. I put -Wcomment to try to turn it off but I think I'm doing something wrong:

%.bin: %
   $(call build,OBJCOPY,$(OBJCOPY) -O binary $< $@)
zebra.o: $(SRC_DIR)/zebra.c
   $(call build,CC,$(CC) $(CFLAGS) -O3 -c -o $@ $<)

I guess you can keep adding different modules at the end the same way, but I'm not sure what modules will benefit.

miyake

%.o: $(SRC_DIR)/%.c
        $(call build,CC,$(CC) $(CFLAGS) -c -O3 -o $@ $<)


Then I used platform/600D.102/autoexec.bin.

[miyake@MLdev32 magic-lantern600daudio]$ ll -h platform/all/autoexec.bin
-rwxrwxr-x. 1 miyake miyake 2.4M Sep  8 01:18 platform/all/autoexec.bin
[miyake@MLdev32 magic-lantern600daudio]$ ll -h platform/600D.102/autoexec.bin
-rwxrwxr-x. 1 miyake miyake 420K Sep  8 01:17 platform/600D.102/autoexec.bin


result
23sec => 43fps

Without -O3
[miyake@MLdev32 magic-lantern600daudio]$ ll -h platform/all/autoexec.bin
-rwxrwxr-x. 1 miyake miyake 1.8M Sep  8 01:39 platform/all/autoexec.bin
[miyake@MLdev32 magic-lantern600daudio]$ ll -h platform/600D.102/autoexec.bin
-rwxrwxr-x. 1 miyake miyake 308K Sep  8 01:38 platform/600D.102/autoexec.bin


result
25sec=>40fps

1%

I'll try it for all modules since it seems to help.

%.o: $(PLATFORM_DIR)/%.c
$(call build,CC,$(CC) $(CFLAGS) -c -O3 -o $@ $<)


Also did that, trying it. Bin is 400K

24sec -> 41 FPS

Just the core modules better? Same, no difference? Everything does seem zippier.

Does it only test old peaking methods?

Marsu42

Quote from: 1% on September 07, 2012, 06:31:33 PM
I put it like this in makefile.inc... I don't know how to add this to makefile.user... is it even taking the flags out of there. I put -Wcomment to try to turn it off but I think I'm doing something wrong

Just adding zebra.o with -O3 to Makefile.inc does nothing - do make V=1 and check what parameters reach gcc. The only thing that does work is editing Makefile.inc and change the global flags, and the whole -O3 compile crashes my 60d.

edit: I created a new topic for benchmarks, since this one was really about user prefs in the makefile ... see http://www.magiclantern.fm/forum/index.php?topic=2609.0

miyake

@nanomad
Anyway, Thankyou , I'm deleted all difference  in Makefile.inc on our 600D audio repos.


miyake

@nanomad

I don't know why, But current position of CFLAG_USER doesn't work.
Please move CFLAG_USER to the final line in Makefile.user.default.



nanomad

Should be fixed in the latest revision, thanks.
Keep in mind that Makefile.user should contain only values that are different from default (and not everything)
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5