[proposal] mod_mgr.mo Module Manager / Online Module Repository

[g3gg0]

as modules got really useful lately and some versioning and updating issues come up, we should
think about handling variants of modules, different revisions and automated module updating.

a year ago, every dev released customized autoexec.bin, today we all use basically the same autoexec.bin, but vary the modules.
this is a really good transition and was making us think more modular and separating concerns into separate modules.

but before it gets a mess with modules now, it would make sense to introduce a repository with all the latest modules and even branches or variant of standard modules.
i have no 100% clear view about how to set up the repository and the branch system exactly, i still have a bit highlevel view on this.

implementation level:
- heavy duty: using hg (bitbucket) and a separate repository called e.g. ml-modules
   pro:
     - perfect branch / fork mechanism
     - perfect revisioning mechanism
     - basically no server cost, administration and implementation effort
     - updater client just needs the base URL, which even can be a fork (just like we already do with source)
   con:
     - dependency on bitbucket
     - getting revision lists etc might be not as simple as just downloading an URL (is there a JSON interface?)

- light weight: simple upload system with some bash scripts...

functionality level:
- show a list of all modules possible to install
- get description of a module (extracted from .mo?)
   - contains dependencies, revision, hg link to source
   - contains a "provides" field? (like in all package managers to detect if two modules are doing the same thing)
- get revision list and branches of a module
- get module at specific revision / branch

so if we have this repository, we need an updater for the modules on the card.
why do i come up with this idea right now?
see this thread.

i want a mod_mgr.mo which uses the transcend wifi card to update modules automatically or on demand.
as soon the wifi module in 6D and 70D is understood enough, this interface can also be used,
so owners of many camera models can make use of the repository system - even while being on a journey using their cell phones.


of course we will first have to implement a windows/linux/mac updater (or one of those) which is the reference application and that uses disk access.


any comments?
anyone who will check if this is doable using bitbucket?

[dmilligan]

What about a separate repo per module? This would allow anyone to make a repo for their own module, then they could "register" their repo somehow with a module manager client or whatever and any user could easily install it (this client perhaps could just be a web page on the ml site that auto pulls data from "registered" module repos and makes it easy for users to find and download .mo files for example). Then you don't have to deal with stuff like game modules being in any "main" repos. The advantage with branching is that you don't have to pick and choose modules out of different branches in a combined repo to get the versions you want. Your client could present the option of which branch for a particular module they would like to install. Your branches are a lot cleaner too, b/c you won't have a bunch of different branches for different modules all cluttering up a "main" modules repo. Also I think this helps with permissions. Module developers won't have to submit pull requests to get their modules merged into the modules repo. It will be their repo that they own and they can push commits to it whenever.

I think coming up with a way to deal with versioning is also going to be very important going foreword. And the solution for that my greatly impact how you want to deal with module repos

Anyways just thinking out loud here as someone who is interested in developing modules for ML, there may be better options IDK

[Marsu42]

but before it gets a mess with modules now, it would make sense to introduce a repository with all the latest modules and even branches or variant of standard modules

I cannot really comment on the tech side of things, but concerning policy it'd be important that ml is delivered (nightly builds, releases) with as many modules as possible with an "enabled by default list" for the most popular/stable ones and then let the users make the choice what to use.

If any tech implementation would lead to a fragmentation into "first class" and "second class" modules, imho it would hinder module development because if someone takes the time to polish a module up to ml coding standard and add a nice gui it should be left to rot in some source repository, not matter how "important" or "valuable" it is considered at first sight.

Once this issue is addressed, different repos per module probably would be a nice idea since there would be zero admin burden after adding the repo to the list, it's compliant with the distributed source idea and no need for an "official" acknowledgment of changes. If a module is broken after some core changes, well, it just wouldn't compile or be added until fixed.

Edit: Also the issue of module-specific callback requirements to get model-specific variables isn't solved yet - what gets added where by whom?

[stevefal]

Given the module system, ML is now a platform that allows people to write closed-source add-ons. This will likely lead to proprietary paid modules, which goes against the mission "to provide technical means for executing open source software on Canon DSLR cameras." This brings up philosophical and legal questions.

When developers realize they can commercialize a large ML installed base, some will be tempted to go closed-source and charge for modules. That is within their right, but it could diminish sharing of code and expertise. And if such closed modules lead to claims of harm, legal implications could spread to the ML project.

Do ML leaders want to implement technical and legal safeguards to minimize the commercial opportunity and risk? Or do you throw caution to the wind and let it be an unrestricted platform?

Feel free to fork the thread - I thought I'd ask while you were here.

[Marsu42]

Feel free to fork the thread - I thought I'd ask while you were here.

Hmmm, I haven't thought of that - but with ml being gpl, and any modules being built upon the minimal module sample and using the ml calls, isn't it so that any module author also has to release the source code?

If not, maybe a license addition is in order to prevent commercial modules exploiting the ml work - imho ml should be "gpl", not "lgpl" ... some might say it's better to have commercial modules rather than non at all or people keeping their changes for themselves, but I wish that anyone that builds upon ml also returns the favor by releasing non-trivial enhancements.

Like with gpl, of course anyone should be free to make money from supporting users to use ml like publishing tutorials or coding specific features for money that are afterwards added to the official build.

[stevefal]

...any modules being built upon the minimal module sample and using the ml calls...

Calling an API doesn't normally bind you into licensing terms. Copyright is about copying protected work, so if you can write a module without copying protected material, you are not bound by a typical license. There are more aggressive licenses, but GPL is not one of them.

I don't understand the module architecture enough to know whether significant ML source or object code must be copied into a module implementation for it to work. If not, then ML is just as open to exploitation as Canon cameras are .

In fact, engineering the module interface such that every implementation must derive from an ML-licensed work is one of a handful of ways to protect ML's open-source mission. I think there are other ways as well.

But the question is whether the ML team wants protections in the first place.