Author Topic: New memory backend  (Read 27775 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11720
  • 5D Mark Free
New memory backend
« on: September 17, 2013, 05:51:03 PM »
Just finished something that I had in mind for a long time: a new unified memory backend.

https://bitbucket.org/hudson/magic-lantern/src/tip/src/mem.h
https://bitbucket.org/hudson/magic-lantern/src/tip/src/mem.c

Background: in each camera, there are a bunch of memory pools which have some unused space (Canon code allocates memory from there, but they are not fully used - there's a bit of free space left, and ML uses it).

Of course, Canon code doesn't know about ML (so it expects there's enough RAM for whatever it wants to do), and ML should be careful not to take away all that RAM (otherwise Canon code will fail and will usually throw ERR70).

Until now, we have used malloc and AllocateMemory for general-purpose stuff, and shoot_malloc for very large stuff (e.g. raw video frames, or storing ML files for restoring them after format). The decision whether to use malloc or AllocateMemory was hardcoded (most cameras have more free RAM in AllocateMemory, but some of them have more in malloc). A table with free RAM for each memory pool is here: https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdFFIcFp1d0R5TzVPTVJXOEVyUndteGc#gid=2

Now, we can just call "malloc" and the backend will decide what buffer to use (based on free space in each of them).

Advantages:
- you just call malloc without thinking too much about it (the backend decides what pool to use; you will only see a large heap of RAM that just works)
- traceable: you can see in menu how much RAM was allocated, current/peak usage, a list of blocks that also shows source file, line, task, also a history of RAM usage...
- extensible: if you find some more methods for allocating RAM, simply create a malloc-like interface and add it to the list of allocators
- you can run huge modules if you really want that (didn't try, but should work)

More RAM sources that were not used: task stack space (most cameras have 500K free, some have 1MB) and unused RscMgr memory (e.g. on 60D). For these, you can take PicoC's heap routines (that behave like malloc), add them to the allocator list in mem.c, and the existing code will start using them right away.

Notes:
- shoot_malloc is a bit special: Canon code expects it to be free when you change certain settings (e.g. noise reduction on 60D). Therefore, if we use it, we must free it quickly, before the user goes to Canon menu).
- error handling: malloc always returns 0
- for new code and refactoring existing code, I suggest using:
    - plain malloc/free for most cases
    - fio_malloc/fio_free if the memory is used for reading files (this goes to alloc_dma_memory or shoot_malloc)
    - tmp_malloc/tmp_free if the memory will be freed quickly (so the backend will prefer shoot_malloc)

A bit related: after loading modules, TCC will be unloaded. This will free another big chunk of RAM.

Screenshots:

5D3, after loading a bunch of modules:


60D, no modules loaded:


(the peak is still from module backend btw - turn it off in debug menu and it will disappear)

Caveat: all ports will require renaming the malloc stubs (add an underscore); otherwise, compiling will fail (so nightly builds will be broken until somebody with code skills will take a look at each port, update the stub names and see if it works). I know, it's not very nice, but I prefer a completely broken build (because the users can just download an older build) instead of a something that compiles and has a big chance to fail.

So, all you have to do is to:
- rename malloc-related stubs in stubs.h
- get the two screenshots and upload them here
- create a pull request (so I shouldn't have to edit each stubs file)

Now you have enough memory to run OpenCV on the camera ;)

Credits go to g3gg0 for memcheck (this code is based on it).

deletedAcc.0021

  • Member
  • ***
  • Posts: 238
  • 600d x 4
Re: New memory backend
« Reply #1 on: September 17, 2013, 05:59:26 PM »
Damn A1ex,  great news.   This is when I wish I had coding skills to lend a hand.

I wouldn't worry about broken nightly builds, like you say, there are plenty of other builds to use.

Good job dude ....

Greg

  • Hero Member
  • *****
  • Posts: 550
Re: New memory backend
« Reply #2 on: September 17, 2013, 07:12:37 PM »
500D, no modules loaded: :


1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: New memory backend
« Reply #3 on: September 17, 2013, 07:44:09 PM »
So far no memory related crashes + lots of ram free.

Defish never gives up what it allocates but now it isn't a problem. Will have to check 600D soon, it + 6D were the main crash on start offenders.

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3123
Re: New memory backend
« Reply #4 on: September 18, 2013, 12:46:14 AM »
good job, thanks!

should we push the module major number to make sure the users will get an error message about outdated modules,
or should we add stubs for the old calls which dont have the additional preprocessor based additional information? (__LINE__, etc)

first: clear cur, latter: legacy mode
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11720
  • 5D Mark Free
Re: New memory backend
« Reply #5 on: September 18, 2013, 08:04:58 AM »
I'd push the major number only when there are data structure changes not caught by the linker (e.g. if adding some field to menu structure => size changes, modules no longer binary compatible).

Right now the linker will just complain about missing symbols. If a module does not allocate anything dynamically, I think it will still work.

Warning: OldAPI modules are still loaded and code executed from them. Need to fix that.

mk11174

  • Hero Member
  • *****
  • Posts: 809
Re: New memory backend
« Reply #6 on: September 18, 2013, 11:19:51 AM »
550D Ettr, Raw_Rec, and Dual_ISO


550D No Modules
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

  • Hero Member
  • *****
  • Posts: 809
Re: New memory backend
« Reply #7 on: September 19, 2013, 01:00:53 AM »
I noticed for some reason on my 550D and the 600D after using the cameras in live view and shooting Dual ISO the live view freezes on screen and I have to take out battery to reset. I need to do more tests but so far with the latest update with the memory code this is happening to both cameras.

The 600D freezes when trying to get details of the memory, when it finishes its refresh it freezes live view?

The 550D mostly freezes using the modules, but have not used camera long enough without modules loaded to see if it just freezes without them.

Just wanted to post a result update.
500D/T1i  550D/T2i  600D/T3i  700D/T5i

Doyle4

  • Hero Member
  • *****
  • Posts: 690
  • Canon 5DRAWii & 600Dual-Iso
Re: New memory backend
« Reply #8 on: September 20, 2013, 03:25:44 PM »
Here is the 600D memory image you asked for A1ex.


a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11720
  • 5D Mark Free
Re: New memory backend
« Reply #9 on: September 20, 2013, 03:33:45 PM »
That's an ancient one...

Doyle4

  • Hero Member
  • *****
  • Posts: 690
  • Canon 5DRAWii & 600Dual-Iso
Re: New memory backend
« Reply #10 on: September 20, 2013, 03:45:17 PM »
Yeahh.. was 17th build, as all other builds said failed at the time. Im now on latest 20th build.. and heres the results, - Screen shot no longer crashes 600D



Doyle4

  • Hero Member
  • *****
  • Posts: 690
  • Canon 5DRAWii & 600Dual-Iso
Re: New memory backend
« Reply #11 on: September 20, 2013, 04:01:40 PM »
Nice work on the new memory stuff btw, ML seems more responsive than ever.

could this allow Small Hack to work better or is that something completely different?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11720
  • 5D Mark Free
Re: New memory backend
« Reply #12 on: September 20, 2013, 04:07:03 PM »
Placebo? The new fonts are twice as slow (try running the menu benchmark).

This update is mostly about not failing with err70 when you load a lot of things.

Doyle4

  • Hero Member
  • *****
  • Posts: 690
  • Canon 5DRAWii & 600Dual-Iso
Re: New memory backend
« Reply #13 on: September 20, 2013, 04:27:01 PM »
Think its because iv jumped from TragicLantern to this... and just seems smoother, i do prefer old font though (from 15th sept build) was much more clear and bold to read.

Will run a benchmark now.


Latest Nightly build 20th Sept: Results: Elapsed time 16300ms
TragicLantern Aug13th: Results: 11179ms

Doyle4

  • Hero Member
  • *****
  • Posts: 690
  • Canon 5DRAWii & 600Dual-Iso
Re: New memory backend
« Reply #14 on: September 20, 2013, 04:39:46 PM »
After doing Menu Benchmark i now have this, rebooted and still here.



Update: Formatting card from camera has cleared it.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11720
  • 5D Mark Free
Re: New memory backend
« Reply #15 on: September 20, 2013, 04:56:48 PM »
This means the memory fragmentation is bad, so the backend needs to know the size of the largest block (not just the free space).

Or a fallback mechanism (if one allocator fails, try another one).

Or both.

Doyle4

  • Hero Member
  • *****
  • Posts: 690
  • Canon 5DRAWii & 600Dual-Iso
Re: New memory backend
« Reply #16 on: September 20, 2013, 05:28:43 PM »
Other than that all seems ok, good work, ill keep trying the nightly's with my 600D.

deletedAcc.0021

  • Member
  • ***
  • Posts: 238
  • 600d x 4
Re: New memory backend
« Reply #17 on: September 20, 2013, 07:19:09 PM »
Alex, did this memory fix affect raw recording at all?  I was pleasantly surprised that I'm no longer getting pink frames with the latest nightly. 

Or is this the placebo effect again ... lol

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11720
  • 5D Mark Free
Re: New memory backend
« Reply #18 on: September 20, 2013, 07:29:16 PM »
I did not even try it...

What I did at some point was to use the fast zebras by default in LiveView instead of slow raw ones. This could have had an influence over pink frames.

deletedAcc.0021

  • Member
  • ***
  • Posts: 238
  • 600d x 4
Re: New memory backend
« Reply #19 on: September 20, 2013, 07:39:17 PM »
I did not even try it...

What I did at some point was to use the fast zebras by default in LiveView instead of slow raw ones. This could have had an influence over pink frames.

Once the build bot compiles tonight's build I will test raw further, I see you fixed the arrow shortcuts already.  Thanks!

tin2tin

  • Member
  • ***
  • Posts: 155
Re: New memory backend
« Reply #20 on: October 05, 2013, 11:25:48 AM »
What's the status on this? Would the potentially released RAM could be used to shoot longer RAW recordings?
EyeFrame Converter transcode RAW to DNxHD, ProRes, Mpeg2 I Frame HD and MJPEG with T/C and R/N
DVD slideshow GUI - freeware multi-purpose slideshow studio.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11720
  • 5D Mark Free
Re: New memory backend
« Reply #21 on: October 05, 2013, 12:49:21 PM »
Maybe if you solder some memory chips.

dlrpgmsvc

  • Senior
  • ****
  • Posts: 398
  • The 7D and 50D Robin Hood
Re: New memory backend
« Reply #22 on: October 05, 2013, 02:13:02 PM »
What's the status on this? Would the potentially released RAM could be used to shoot longer RAW recordings?

Before memory backend, I managed to record 1600 frames of raw video at 1920x1080 23,9fps on 50D, now I manage to record 2500 frames at the same settings ! 64Gb KomputerBay card. 81Mb/s - 82Mb/s boundary writing speeds, 81.5Mb/s mean value.But I don't want this to be a placebo effect...  ;-)
If you think it's impossible, you have lost beforehand

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11720
  • 5D Mark Free
Re: New memory backend
« Reply #23 on: October 05, 2013, 03:06:57 PM »
I bet it's unrelated.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: New memory backend
« Reply #24 on: October 05, 2013, 04:17:43 PM »
I have(some cams had) issues with it not freeing unless using TMP_malloc/fio_malloc... it takes but it won't give back till memory dwindles to nothing.