Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: bpv5P on August 08, 2017, 09:48:11 AM

Title: Code static analysis and optimization tools
Post by: bpv5P on August 08, 2017, 09:48:11 AM
I don't know if the dev team is aware (probably is), but maybe some tools could help with ML testing and optimization. For example, the LLVM static analysis, Frama-C, splint, Valgrind (for memory) and Coverity:
http://www.frama-c.com/
http://www.splint.org/
http://valgrind.org/
https://www.coverity.com/

Got those tips from BCHS:
https://learnbchs.org/tools.html

Also, a question: I know it's trivial, but why use Bitbucket? Wouldn't it be a good practice to setup a project's own git? Since it has few commits, there's no need to pay a dedicated server, any laptop could run it...
https://git.zx2c4.com/cgit/about/


PS: sorry, if it's a duplicated thread please merge, I've not searched on this forum for previous threads on this subject.
PS: also, sorry for flooding this sub-board with two consecutive threads... too much coffee, I think. Wish I could help with this project, instead of giving awful ideas.
Title: Re: Code static analysis and optimization tools
Post by: a1ex on August 08, 2017, 10:35:05 AM
Valgrind: good luck porting it (http://www.valgrind.org/info/platforms.html). I had an attempt to write a similar tool in QEMU (http://www.magiclantern.fm/forum/index.php?topic=2864.msg184125;topicseen#msg184125) (-d memchk), and already found a couple of bugs. It's currently working only on 500D.

Very easy coding task: port it to other models (it's mostly a matter of copy/paste and testing, with some stubs to find). Sorry, time is an issue on my side.

I've also played a bit with code coverage:
- with gcov: the size overhead is significant (autoexec.bin would grow very large, but it's probably doable for small modules)
- with QEMU: I've got a code coverage report from an execution trace (with qemu-etrace and Xilinx QEMU); also tried Project Couverture (http://www.open-do.org/projects/couverture/) but could not get it to work.

I've also tried to use LLVM's thread safety analysis (https://www.magiclantern.fm/forum/index.php?topic=18907.0), but I need help (I'm not grasping these concepts very well).

Another suggestion I've got was to use openQA (http://open.qa/) for automating ML testing in QEMU. I've got the backend installed (os-autoinst), but didn't figure out the web interface yet; help is welcome with this one as well (it's probably straightforward, but takes time). The emulator already runs unmodified ML binaries on most cameras. A long time ago, when we could barely emulate ML menu navigation (without much Canon code), I've suggested a visual diff for menu screenshots (http://magiclantern.fm/forum/index.php?topic=12396.0), but apparently it fell on deaf ears.

I'm not familiar with the other tools, but contributions (on applying them to ML) are welcome.
Title: Re: Code static analysis and optimization tools
Post by: g3gg0 on August 08, 2017, 04:54:11 PM
about bitbucket: it's free for us iirc.
Title: Re: Code static analysis and optimization tools
Post by: a1ex on August 08, 2017, 05:38:53 PM
Right - Trammell used Bitbucket when he started the project, and we had no good reason to change it.
Title: Re: Code static analysis and optimization tools
Post by: bpv5P on August 10, 2017, 05:50:19 AM
Quote from: a1ex on August 08, 2017, 10:35:05 AM
Valgrind: good luck porting it (http://www.valgrind.org/info/platforms.html). I had an attempt to write a similar tool in QEMU (http://www.magiclantern.fm/forum/index.php?topic=2864.msg184125;topicseen#msg184125) (-d memchk), and already found a couple of bugs. It's currently working only on 500D.

I'll see if I can find an alternative...

Quote
I'm not familiar with the other tools, but contributions (on applying them to ML) are welcome.

The "splint" linked above is really simple to use., it may be worth your time:
http://www.splint.org/
Title: Re: Code static analysis and optimization tools
Post by: a1ex on August 10, 2017, 10:37:00 AM
Quote from: bpv5P on August 10, 2017, 05:50:19 AM
The "splint" linked above is really simple to use., it may be worth your time:
http://www.splint.org/


... magic-lantern/modules/mlv_lite $ splint mlv_lite.c -I../../src -I../../platform/all -I.
...
/usr/include/bits/types.h:30:8: Parse Error: Non-function declaration:
    __BEGIN_DECLS : int. (For help on parse errors, see splint -help
    parseerrors.)
*** Cannot continue.


How to get past that?
Title: Re: Code static analysis and optimization tools
Post by: bpv5P on August 11, 2017, 05:48:04 AM
Quote from: a1ex on August 10, 2017, 10:37:00 AM
How to get past that?

Hi a1ex, I'll see if I can solve that.