Script languages for embedded systems

[jplxpto]

Python : https://code.google.com/p/tinypy/

Java Sript : https://code.google.com/p/tiny-js/

Lua : http://www.eluaproject.net/overview

[a1ex]

Here's my opinon on these script languages, after a short experience with a few of them:

PicoC (we use it and already have a small API and some scripts)
+ quite small (~50K) => could fit in nightly builds without much trouble
+ easily hackable (the code is easy to understand and modify)
+ it's pretty much plain C with relaxed rules (e.g. you can hack pointers, examine Canon code)
- slow (around 1000-5000x slower than gcc). Not a problem for simple bracketing scripts, but don't even think about image processing at pixel level
- not very well tested. I've ran around 100 problems from Google Code Jam and a bunch of random tests with CSmith, found and fixed a few bugs, and now it's pretty much OK.
- C might be a bit harder for novice programmers
- development is pretty much stalled, the author just merges patches from contributors

TCC (almost ported)
+ this is a real ANSI C compiler (C90, many C99 extensions, can compile Linux kernel)
+ very small for what it does (~200K, only required at compile time, can be unloaded after that)
+ built-in ELF loader (we use it for modules)
+ fast (you can do image processing, at least in playback mode). On desktop it's pretty much as fast as gcc -O0. Image processing test (720x480) was almost instant, on PicoC took 1 minute.
+ compilation is instant
+ calling existing C code just works (no extra wrapper functions needed)
+ active development, active mailing list
- needs a bit of work to get it running with all our memory constraints
- existing PicoC scripts will have to be rewritten (TCC is more strict, it requires a main function)
- still has some compiler bugs (passed most tests that I've ran on PicoC/gcc, but not all)
- source code is a bit difficult to understand (see also http://bellard.org/otcc/ )

TinyPy (I've ported it these days, more like a test for the new module system; can be loaded as a module and can print Hello World)
+ fairly small (~150K)
+ in-camera compiler and interpreter
+ pretty much as fast as regular Python (it uses a virtual machine inspired from Lua, but not as efficient)
+ the parser/compiler is implemented in Python and runs in the camera as bytecode (so, fairly good test coverage just from this)
+ clean syntax
- memory hungry (the current hello world example from the repo needs 16K of stack and ~1MB heap, with very aggressive GC)
- with default GC settings, with 2 simple loops, e.g. while(1) for (1 to 1000), it fills gigabytes of RAM (!)
- very few built-in tests (I expect many quirks in slightly more complex programs)
- no function prototypes: you can write sin(1,2,3,4,5) without any warning
- batteries not included: not even a basic printf...
- code base was not touched since 2009 (!)

PyMite
+ quite small (~50K)
- you have to compile the scripts on the PC (it reuses the full-blown Python compiler)
=> why not just use gcc?

Lua (I've tried it in early 2011, on 550D, when memory limits were not known; Sztupy also tried it on 60D in early 2012 - and it's still in ML tree as a plugin, but nobody maintained it)
+ fairly small (~150K)
+ fast (only ~20x slower than gcc)
+ mature language, widely used
+ nice docs, e.g. http://www.lua.org/doc/jucs05.pdf
+ used by CHDK
- memory usage was a bit too high for some cameras; now with the module system, this is no longer an issue
- minor syntax quirks, e.g. print(string.format("%d", 5)), or x ~= y, or... all variables are global by default

eLua (didn't try this one)
+ the Lua advantages
+ some nice modules, e.g. PWM, UART, interactive console, see http://www.eluaproject.net/doc/v0.8/en_status.html
+ optimizations for low-memory environments: Lua Tiny RAM , Emergency GC
- I'm just not familiar with it, so can't say too much here.

tiny-js
+ small (2000 lines, no idea about binary size)
+ web developers may find it easy
- who else uses javascript?
- it executes directly from source, like PicoC, so I expect it to be just as slow
- it's written in C++, so the integration effort may be higher (no idea about RAM usage)

ubasic
+ tiny
+ used by CHDK
- even slower than PicoC

Winners: TCC and LUA?

Other scripting engines worth considering?

With the module system, we can even support two scripting engines. For example, we can keep the existing GUI, and use the existing API from PicoC for both languages. This is pretty much similar to GCC having a few frontends. Worth the trouble?

I'm also considering full double support (no floats, just plain 64-bit doubles), so we don't have to struggle with integers. In current implementations, PicoC uses 32-bit floats and Lua uses 32-bit integers.

[engardeknave]

I think you have to go for the fastest option. Couldn't people who don't have enough memory for TCC compile on their PC?

[wolf]

If tcc is like picoc but faster, I prefer tcc. Don't mind the stricter syntax.
I tried some lua scripting with CHDK and it was quite good but I like more the C syntax.

What about BrainF***? 

[a1ex]

I've tried to write a DIGIC dumper for CHDK in Lua (got an A2200 for a friend and played a bit with FPS override). I had to call the garbage collector manually, otherwise got out of memory:

Code Select Expand


f = io.open("A/digic.log", "w")
for i = 0, 0x10000, 4 do
    addr = digic_ram_mirror + i
    val = peek(addr)
    if val ~= 0 then
        reg = bitand(addr, 0xFFFF) + 0xC0F00000
        print(string.format("%x => %x", reg, val));
        f:write(string.format("%x %x => %x\n", addr, reg, val));
        collectgarbage("collect") -- out of memory otherwise
    end
end
f:close()


[jplxpto]

Alex,

thank you for your summary.
You did a good job.

For those who know C programming, scripts TCC are a very interesting option but,
for ordinary mortals the Lua and uBasic scripts should be an option to consider.

I have no experience in any of these languages​​ but,
I think we can maintain support for TCC and Lua,
each language for a different purpose and audience.

I think that CHDK's users will like!

What do you think?

[jplxpto]

http://balau82.wordpress.com/2011/06/07/pypy-as-a-faster-alternative-for-python/