External library integration in module

Started by novovaccum, March 16, 2023, 09:00:42 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

novovaccum

Hello,

I am a student in my last year of engineering school. For my diploma work I thought of using ML to develop a module to encrypt an image before it is written on the card. The idea is to use the https://doc.libsodium.org library to do so.

I've been working on this idea for a few weeks now, and I finally managed to link the cross-compiled library, but I'm now stuck with the calls related to the C standard library. (malloc, free, read, close, etc....).


Will NOT load on:
    5D2 (fcntl, malloc, read, free, abort, close, explicit_bzero, open)


I referred to this old post on the forum for a similar question about adding external libraries: https://www.magiclantern.fm/forum/index.php?topic=18897.msg179492#msg179492 Unfortunately most of the links are not available anymore.

I tried to follow the example in the LUA module with Dietlibc, but even if I reproduce the configuration and link the ".o" produced by Dielibc in the Makefile of the module in question, I cannot seem to solve the problem.

Or do I have to re-implement all methods in a wrapper such as "ml-lua-shim.c" ? Or is there a better way?

As I don't have any real knowledge in embedded system, I'm a bit stuck. So I apologize in advance if my questions do not seem relevant.

Thank you in advance for your help

names_are_hard

Hi!

ML doesn't include the full stdlib.  There are multiple reasons for this, but one is that DryOS already has a stdlib, so we can use that in some cases and avoid having to package our own.  Your module requests (via imports) functions that magiclantern does not provide as exports.  You can fix this in various ways, some easier than others.

We provide malloc() via a macro in src/mem.h.  If you include that header in your libsodium module it should fix that problem.  That will get you free() as well.  You'll be able to do similar with some of the other calls.

Some you'll need to modify the library.  We don't provide explicit_bzero(), for example, but we do provide bzero32() - probably this is good enough for your needs.  Change libsodium to use bzero32(), if you agree with me; I don't know exactly what you're doing so only you can really tell.  File ops are normally prefixed with FIO_ in DryOS, so we have FIO_ReadFile() that *probably* does something similar to read(), but you'll need to check the semantics are the same before modifying libsodium.

Fcntl may be a bigger problem.  This is POSIX I think.  DryOS is not POSIX and probably doesn't support fcntl.  I suspect you'll need to work out how to remove usage from libsodium (or maybe they have a non-POSIX variant for embedded usage?).

The compiled size of libsodium is relatively large, larger than ML itself.  You may find this is too large to load into memory on the cam.  Since you probably don't need or want to support every option available, you might want to cut down the library so it only supports the crypto that you've chosen to use.

You can test these things in Qemu, hopefully you are doing this already.  It's faster and safer than using a physical cam!

names_are_hard

Oh yeah, also worth mentioning, a similar module has been written before: https://www.magiclantern.fm/forum/index.php?topic=10279.0

You may want to reference this in your diploma, related work or something like that.

novovaccum

Thank you so much for your very detailed answer!  :)

I will see how I can carefully modify the Libsodium sources where needed so that it can best integrate with Dryos.
As suggested, I will also see how I can reduce the weight of the library based on the required usage in my situation.
I hope all this will work :)

And of course, if I succeed, I will mention the "io_crypt" module as the starting point to explore the possibility of encrypting the image with a library.

I let you know if it succeeded

Thanks again for all your answers. It really helps me to move forward.