Some graphs for RscMgr memory map, from sMemShowFix logs (this is the function g3gg0 investigated
earlier in this thread):
You may ask - why I'm still investigating this?
The small allocators are sometimes too small for our needs, for example when loading too many modules, or when loading some memory-intensive ones. On small cameras like 1100D, you don't even need to do that.
Okay, but we have two large allocators (shoot_malloc and SRM), right?
Yes, but these have some restrictions:
- you can't keep them allocated forever:
- srm_malloc: you can no longer take pictures if anything is allocated from there (so it's useless as general-purpose memory)
- shoot_malloc: Canon code expects it to be completely free when doing certain things (for example, in-camera raw development, or enabling certain noise reduction modes, or - on 60D - simply switching between photo and movie mode). It works most of the time, but those rarely-used actions will fail (ERR70 or camera lockup).
- you can't allocate too many blocks from there (about 1000 from shoot_malloc, or a very small number from SRM).
- for this reason, the memory backend tries to avoid them - until the small allocators get full
In particular, Lua is memory-intensive, allocating thousands of small blocks using realloc. Our current allocators are not ready for that - loading many scripts gives ERR70, even if you have plenty of memory available, because of fragmentation.
I managed to get better results
here, by pre-allocating a fairly large block (512K) and managing it with a custom allocator optimized for small blocks. However, that large block may end up in shoot_malloc, which I prefer to avoid.
Other reasons for investigating the RscMgr:
On 70D, the RscMgr "allocates" (with hardcoded address) memory for IMGPLAY overlapping the first raw buffer right after a still photo was captured (which interferes with raw overlays). For that case, I'm seriously thinking to change the memory map somehow, so these two buffers no longer overlap. (
discussion)
Finally, some cameras still have memory that cannot be allocated by shoot+SRM. For example, 5D3 can allocate 438MB outside LiveView, but only 284MB in LiveView. I kinda doubt the LiveView actually uses 150MB. On 60D, the numbers are 436 outside LV, and 374 in LV - a little better. That extra memory could be useful for raw recording, but unfortunately it's not visible on these maps.
From the memory maps above, on 60D on the maps above, you can see a 7MB block at the end that appears unused (similar to the one g3gg0 found on 7D). Currently, autoexec.bin is loaded there, but it has some side effects - you cannot run the dm-spy-experiments branch, and not all this memory is available for ML. I'm thinking to move autoexec.bin to AllocateMemory, and use the 7MB block as general-purpose memory for ML, using some custom allocator (maybe Canon's).
I've got logs from 5D3, 60D, and also from 70D (nikfreak). For other cameras, I'd like you to get the log output for sMemShowFix, so I can generate similar graphs. To do that:
- find the stub (
tutorial here; look
here for strings to look for)
- compile the dm-spy-experiments branch, with CONFIG_DEBUG_INTERCEPT=y in Makefile.user
- put this in don't click me (example for 70D):
void debug_intercept();
debug_intercept();
void (*smemShowFix)(void) = 0xFF202DE8;
smemShowFix();
debug_intercept();
- run this code and upload dm.log from your card
Thanks in advance.
