Proposal: SCRIPT engine [External compile scripts as modules?]

[gerk.raisen]

Hello,

What about the possibility to have some APIs to create scripts and compile it "like a module" so that we can circumvent the big problem of the impossibility of TCC in-camera compiling?
The basic ideas is that it can be a sort of template module where you only need to call the APIs and then compile it like other modules.
After that they can be simply runs as a modules.
In order to maintain a simple one-click run maybe at the start of that script-modules it ask immediately to confirm (with "SET") or edit (with another button) the built-in script parameters (like intervals, etc).
I don't remember if still they are problems to unload modules but probably for not very complex scripts the memory used it isn't a real problem. (I hope)
Maybe it's not so flexible as real scripts (ex. you can't edit it on camera, except for the parameters) but it would be a big step forward.

[Marsu42]

What about the possibility to have some APIs to create scripts and compile it "like a module" so that we can circumvent the big problem of the impossibility of TCC in-camera compiling?

I don't understand that - could you elaborate?

Afaik the old picoc scripting engine is to be updated for tcc sooner or later, but alex had more interesting things to do so far - but seeing how many requests here would be self-serving if they could be scripted by the user, imho it's gonna happen sooner or later.

[gerk.raisen]

The idea is basically to encapsulate scripts in a blank template module, so that you can directly run it (instead of the pure script) in-camera.
It will also have some advantages over a PicoC port ... for example it will not affected from PicoC slowness http://www.magiclantern.fm/forum/index.php?topic=5014.0 because is already compiled.

[Marsu42]

The idea is basically to encapsulate scripts in a blank template module, so that you can directly run it (instead of the pure script) in-camera.

So if I understand correctly the intended target audience are people who can write scripts, cannot code C (or don't want to learn writing a real module), but want C-like speed?

[dmilligan]

The plan for TCC scripting engine is that it will compile scripts on the camera and then run them, speed would be native. There's no need to encapsulate scripts as modules, the current plan is to simply finish porting TCC and the scripting API. Like Marsu said, pretty much just waiting on a1ex to get around to it.

If you want to wrap a script as a module and compile it on a computer, there's nothing stopping you. If you know enough C to write a script, then you know enough C to write a module (there are plenty of examples).

[gerk.raisen]

From what I read in the last A1lex post here:
http://www.magiclantern.fm/forum/index.php?topic=4877.0

I'm not optimist at all,
TCC for scripts seems to be a dead rail. 

[engardeknave]

That's really old. I think it was working but not complete, and then a bunch of other stuff like raw video and dual ISO took development priority.

[Marsu42]

If you know enough C to write a script, then you know enough C to write a module (there are plenty of examples).

Quite - but tbh you need much more ML knowledge to write a module, even if the code itself might not be very hard to do.

First off, the menu structure and variables (numeric, string, ...) and different menu triggers need some getting used to. And then you cannot simply hook whatever you're doing into the shoot task, I guess if it takes a longer time you should create an  own task that is run one after triggering the function from the menu. Last not least, if you write a module you can brick your camera while the script engine hopefully should you protect you a bit more.

All in all, I would be very hesitant to advise "just quickly write a module for what you want" to anyone new to ML.

[gerk.raisen]

@Marsu42,
You hit the mark

I think for example with my knowledge I can try to write some simple scripts but writing modules is more complicated...
Creation of module is a little too difficult for me, but with my idea of a module template script and some APIs I can start to try something.
The APIs migration from PicoC can be also a step in the right direction to port it for a (maybe) future TCC module.

@engardeknave

The last message by A1lex is dated 16 August 2013, is not so old.
Maybe I forgot something in the forum but after that message I didn't read anything about TCC progress and message itself seems a almost a stop of effort to port TCC for insuperable (at the moment) difficulty.

[dmilligan]

AFAIK the lua scripting is still working, have you looked at that (I haven't)?

[Marsu42]

AFAIK the lua scripting is still working, have you looked at that (I haven't)?

I never got that one working on 60d as it's consuming too much memory (that's why it was replaced with picoc), also it's said to be horribly slow and certainly lacks the api from the picoc version. Bottom line: If anyone wants scripting, try to find the latest version where picoc worked (I don't know when that was though).

[engardeknave]

The last message by A1lex is dated 16 August 2013, is not so old.

No, that's really old in a software community where new versions are compiled daily and new features are introduced weekly (when a1ex is around).

[gerk.raisen]

@dmilligan

Like Marsu42 the Lua Scripting never worked on my 60D, and the old PicoC version it's now too old, too many big feature are missing..RAW video for first.



@engardeknave

I check the Bitbucket committs daily and compile it almost everytime something newer is added, so I agree with you that 3 months can be an ethernity in the ML world

I would be pleased to hear good or simply WIP news about that but unfurtunately, in this case, from the last words wrote from A1lex (no further message about script engine porting later), we can't hope in any happy-end for the TCC port.

Pleased to hear something different from everyone...(A1lex included of course)

[Marsu42]

I would be pleased to hear good or simply WIP news about that but unfurtunately, in this case, from the last words wrote from A1lex (no further message about script engine porting later), we can't hope in any happy-end for the TCC port.

It's like writing docs - once you understand it, you have little reason to write it down. With scripting, everyone who would be able to implement it doesn't need it anymore but can write a real C module - so as it's not paid work only an overwhelming altruistic urge could do it :-p ... the first scripting versions were obviously fun and a proof of concept, but the tcc port would be long and boring work including a lot of testing and debugging.

Let's face it: Given the current dev resources (manpower and knowledge) it's absolutely stunning ML is working at all like it does across an amazing variety of cameras. and there are so many more interesting features in the queue than scripting (yawn).

[a1ex]

Basically, I see these possible ways for scripting:

- PicoC as a module (very easy, should work pretty much out of the box, moderate C knowledge needed, but is it worth it?)
- TCC (one has to implement software floating point - strong C and ARM ASM knowledge needed, it's beyond my current skills, but g3gg0 has some ideas)
- Lua (straightforward, no more memory problems after the new backend, moderate C knowledge needed, one has to tweak it and write and maintain a nice API, maybe close to CHDK)

See also this comparison.

The scripting GUI module is pretty much engine-independent (one could use it as a front-end for any scripting engine you may think of). So, if you want to port Forth or Lisp or whatever other language, you should be able to integrate it in the same GUI.

I still hope somebody will implement one of these before I'll be able to; the proof-of-concept code is already there.

[Marsu42]

- PicoC as a module (very easy, should work pretty much out of the box, moderate C knowledge needed, but is it worth it?)

Probably yes (not that I'm volunteering) - a lot of work has already gone into the scripting api, it can in principle it can solve some requests out there (like the latest 1/2 & 2/3 bracketing) and lets people w/o intimate C/ML knowledge automate their camera which might make more people contribute. Automating a dslr is simply a great leap in usability, and it's a loss that it isn't here anymore.

Once PicoC would be running again, the api could get more work or feature requests to make it do what people want it to do, and this work wouldn't be lost if TCC would be solved - but after your description I don't see the latter in the near future.

[gerk.raisen]

I agree, PicoC seems the simplest solution to start with, now we only need a volunteer 

@g3gg0
Is A1lex say right that you have some aces in the hole for TCC software floating point?

[engardeknave]

TCC doesn't really seem necessary. People who know enough to write something that goes beyond the limitations of PicoC will definitely be able to figure out how to compile a module. I was able to do this and I don't know anything.

What I need and I think most people need is basic script functionality. My concern is that hooking buttons and shooting brackets will be slow and not really practical to use. But I don't think that's a PicoC limitation.

[wolf]

- PicoC as a module (very easy, should work pretty much out of the box, moderate C knowledge needed, but is it worth it?)

I think yes.
One thing or disadvantage is, the speed of PicoC. That's most of the time not that important and it isn't too slow to write games like a1ex did.
The other thing is that it's far beyond my skills making a module with a scripting language.

[Marsu42]

Basically, I see these possible ways for scripting

POLL: Would you like to get PicoC back as a module?

http://www.magiclantern.fm/forum/index.php?topic=9424.0