Just finished something that I had in mind for a long time: a new unified memory backend.
https://bitbucket.org/hudson/magic-lantern/src/tip/src/mem.hhttps://bitbucket.org/hudson/magic-lantern/src/tip/src/mem.cBackground: in each camera, there are a bunch of memory pools which have some unused space (Canon code allocates memory from there, but they are not fully used - there's a bit of free space left, and ML uses it).
Of course, Canon code doesn't know about ML (so it expects there's enough RAM for whatever it wants to do), and ML should be careful not to take away all that RAM (otherwise Canon code will fail and will usually throw ERR70).
Until now, we have used malloc and AllocateMemory for general-purpose stuff, and shoot_malloc for very large stuff (e.g. raw video frames, or storing ML files for restoring them after format). The decision whether to use malloc or AllocateMemory was hardcoded (most cameras have more free RAM in AllocateMemory, but some of them have more in malloc). A table with free RAM for each memory pool is here:
https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdFFIcFp1d0R5TzVPTVJXOEVyUndteGc#gid=2Now, we can just call "malloc" and the backend will decide what buffer to use (based on free space in each of them).
Advantages:- you just call malloc without thinking too much about it (the backend decides what pool to use; you will only see a large heap of RAM that just works)
- traceable: you can see in menu how much RAM was allocated, current/peak usage, a list of blocks that also shows source file, line, task, also a history of RAM usage...
- extensible: if you find some more methods for allocating RAM, simply create a malloc-like interface and add it to the list of allocators
- you can run huge modules if you really want that (didn't try, but should work)
More RAM sources that were not used: task stack space (most cameras have 500K free, some have 1MB) and unused RscMgr memory (e.g. on 60D). For these, you can take PicoC's heap routines (that behave like malloc), add them to the allocator list in mem.c, and the existing code will start using them right away.
Notes:- shoot_malloc is a bit special: Canon code expects it to be free when you change certain settings (e.g. noise reduction on 60D). Therefore, if we use it, we must free it quickly, before the user goes to Canon menu).
- error handling: malloc always returns 0
- for new code and refactoring existing code, I suggest using:
- plain malloc/free for most cases
- fio_malloc/fio_free if the memory is used for reading files (this goes to alloc_dma_memory or shoot_malloc)
- tmp_malloc/tmp_free if the memory will be freed quickly (so the backend will prefer shoot_malloc)
A bit related: after loading modules, TCC will be unloaded. This will free another big chunk of RAM.
Screenshots:
5D3, after loading a bunch of modules:
60D, no modules loaded:
(the peak is still from module backend btw - turn it off in debug menu and it will disappear)
Caveat: all ports will require renaming the malloc stubs (add an underscore); otherwise, compiling will fail (so
nightly builds will be broken until somebody with code skills will take a look at each port, update the stub names and see if it works). I know, it's not very nice, but I prefer a completely broken build (because the users can just download an older build) instead of a something that compiles and has a big chance to fail.
So, all you have to do is to:
- rename malloc-related stubs in stubs.h
- get the two screenshots and upload them here
- create a pull request (so I shouldn't have to edit each stubs file)
Now you have enough memory to run OpenCV on the camera
Credits go to g3gg0 for memcheck (this code is based on it).
