SRM job memory buffers

[MA Visuals]

Does this mean that the newly found SRM memory will be automatically be made available transparently to raw recording with no liveview resetting?  I was never able take advantage of the memory hack on the 5D3 because the long delay of resetting liveview caused my hdmi monitor to miss the initial 4 seconds of recording as the monitor reset itself.  This alone would have a huge impact for hdmi shooters.  Also, will small hacks still be available/useful?

[a1ex]

Would love to test if the extra memory hack can be used in combination with the SRM code, but I don't have any coding skills.

Try compiling 3d6a945, or ask Audionut here to do it for you.

Does this mean that the newly found SRM memory will be automatically be made available transparently to raw recording with no liveview resetting?

Yes.

[josepvm]

I have been testing the recent additions to the srm-memory branch on 500D, today's build.

Shooting with mlv-rec module, using a low resolution, 864x486, to allow continuous recording on SD card.


I have noticed this issue: when using SRM memory, AF (with back focus button, "*") does not work at all during recording.

With exactly the same settings but disabling SRM memory use, AF works again.


So, maybe SRM memory is used for autofocusing by the Canon firmware?

[a1ex]

Does it help if you comment out the gui_uilock calls from exmem.c?

(ML blocks the shutter button so you can't take pictures by mistake and get err70)

[josepvm]

Does it help if you comment out the gui_uilock calls from exmem.c?

(ML blocks the shutter button so you can't take pictures by mistake and get err70)

Yes, it helps.  AF works now with SRM memory enabled.

Thanks, A1ex.

(some additional info: I have CFn functions adjusted to focus always with back focus button, not the shutter button)
 

[josepvm]

I have tested the new Nightly Build for 500D (Nightly.2014July16).

It still has no option to use SRM memory for raw video, but the shutter button lock is active, and AF does not work for video.

With the srm-memory branch build you have the option to disable SRM and then have AF working. In this nightly there is no such option, and no AF 


I wonder if a better solution would be possible here, to avoid taking pictures during video shooting but allowing AF with backfocus button.

[a1ex]

Autofocus works just fine here with the nightly. I didn't include the SRM yet, will do tonight.

The safeguard is not really to prevent taking pictures while shooting video; these routines are general-purpose, other code could make use of them, and I want the backend to prevent the race conditions with picture taking code (that's why the "don't click me" test tells you to take pictures while it's allocating and freeing memory).

Food for thought: when allocating the entire SRM memory, Canon code already locks the UI (it prints BUSY), but the autofocus still works. Figure out how it does that...

edit: I think it's solved, tested on 5D3: https://bitbucket.org/hudson/magic-lantern/commits/92536576a20d

[josepvm]

edit: I think it's solved, tested on 5D3: https://bitbucket.org/hudson/magic-lantern/commits/92536576a20d

Tested this solution on 500D, using srm-memory branch to build, works Ok.

I need to disable "Extra hacks" to get the AF working. When disabling the hacks, the AF works fine with SRM enabled.

I have done the "Don't click me" test with this changes, as you suggested, and I am able to take shots, record video and AF during the test.
But I don't know exactly how this test works and how it should be used, sorry    In first place, I do not know if the test ends after a successfull run, or if it runs continously until I shut down the camera, as I did   


Disabling the "Extra hacks" is the key, also, for the Nightly build I tested earlier.  Now I've tested it again, disabling the Extra hacks, and AF works (no SRM, of course).

When I tested the srm-memory branch yesterday, I disabled the hacks also, because some days ago, testing mlv-rec, I got errors when starting video recording, and Greg suggested me to disable them.

But today, whith the new Nightly build, I saw mlv-rec worked without disabling the hacks, so I left them enabled ... but that's the reason for AF not working.

[a1ex]

The test simply checks if these routines play nice with Canon picture taking process. If you would comment out the gui lock tricks in exmem.c, you would get either lockups or err70 during the test.

[Levas]

Try compiling 3d6a945, or ask Audionut here to do it for you.

@ Alex,
Audionut compiled 3d6a945 for me to test on the 6d.

Both options(mem hack and SRM) together didn't result in any more  memory... or actually I mean 

Did some testing and looked at the MLV file size
Both options = the same file size as Memory hack only 
SRM option only = slightly higher file size vs memory hack option and both options.
So it  looks like both options results in gaining the memory from the memory hack, but doesn't give us any SRM memory.

So you were right 

This means, the benefit for 6D is just cleaner code and a smaller delay before starting to record.

[a1ex]

Alright folks, the SRM memory changes are now in the nightly builds. The good old memory hack is gone, as well as a bunch of older code hacks.

On 5D3 and 6D you have the same total memory as before (+/- a few MB), but without having to blank the LiveView. On other cameras, you no longer need to switch to small JPEG or similar configurations. And if you are lucky, you should get a nice improvement in recording times at certain resolutions.

I don't know the exact figures - you tell me.

This is not the end of memory research, but I'll stop here for now. Feel free to investigate the factory mode, or try to find some more allocators from the resource manager.

Thanks to all the contributors from this thread!

[dubarry]

EVERYTHING IS AWESOME!!!  (Yep I'm a parent)

60D    1728x736 2.35:1  24fps 254 frames @ 10 seconds
          1728x972 16x9    24fps 170 frames @ 6 seconds

Honestly 10 seconds has been my dream for about what 2.5/3 years now.

That's bloody spectacular
thank you, thank you, thank you all.

[MA Visuals]

I can confirm a big increase in performance after the SRM update.  Was getting 15-20 seconds 30fps... now getting 30-40 seconds.  Double the recording time.  At least for me, this makes 30fps a more stable and reliable frame rate.  Also, 48 fps (squashed mode) went from 8 to 15 seconds. Again, much more dependable where before, it was borderline for me. These new times are with raw_rec.  I get similar relative increases with mlv_rec, just shorter actual recording times due to the extra overhead of the mlv format.   

Changes like this really do make a big difference.  For many HDMI monitor users, this eliminates the Liveview blackout that happens with the old memory hack prior to recording.  I could never deal with that during a production so never realized the benefits of that hack.  Thank you Alex and for any others who contributed to this.

The above was using a 5D3 and a SmallHD DP4 monitor.

[N/A]

Today's nighly build for 7d failed. 

Changeset ab901b1

[dmilligan]

Today's nighly build for 7d failed. 

looks like some missing stubs:

Code: [Select]

exmem.c:(.text+0x748): undefined reference to `CreateMemorySuite'
exmem.c:(.text+0x780): undefined reference to `CreateMemoryChunk'
exmem.c:(.text+0x7a8): undefined reference to `AddMemoryChunk'
exmem.c:(.text+0x88c): undefined reference to `DeleteMemorySuite'


[dannyqt18]

simple MLV test on a 5D MARK ii with a 32gb 1000x Lexar:

NO SRM: shot at 1856x1044, 23.976fps for 16 sec

WITH SRM: shot at 1856x1044, 23.976fps for 39 sec (first time) 43 sec (second time)

very big leap for me, will test other resolutions and FPS times in the coming days.

Awesome job!!

[N/A]

looks like some missing stubs:

Code: [Select]

exmem.c:(.text+0x748): undefined reference to `CreateMemorySuite'
exmem.c:(.text+0x780): undefined reference to `CreateMemoryChunk'
exmem.c:(.text+0x7a8): undefined reference to `AddMemoryChunk'
exmem.c:(.text+0x88c): undefined reference to `DeleteMemorySuite'


Gotcha, how does one go about finding these stubs? I'll try do it tonight while I'm not busy.
Nevermind, found the tutorial. Yeah, it's going to take a lot longer than one night to learn that process. Poo.

[Pelican]

Alex already committed these stubs...

[N/A]

Yup, just noticed that. But I did learn how to compile, so that's a plus.

372 MB buffer, 13 seconds of 2496x1044 MLV footage. GD off, no sound. Not too shabby! 

Card write speeds on Lexar 1000x are still a bit inconsistent, on some recordings I get a steady 79-81 MB/s, and sometimes it drops to 65-75 MB/s. Maybe a freshly formatted card would help.

[N/A]

Is SRM also beneficial for h.264 recording? I would think that all the extra buffer memory would assist in using higher bitrates, but anything over 100 mbps instantly buffers out. Noticing a lot of instances of the buttons becoming unresponsive after switching back and forth between the ML menu and LV, although the display is still active. Requires a battery pull to fix.

[dmilligan]

Is SRM also beneficial for h.264 recording?

Canon is going to already be using all the memory they need for h.264.

Just to clarify something: we did not find new or extra memory in the cameras, we found a new memory allocator. SRM is just the name Canon gave one of their memory allocator functions (a memory allocator basically manages and keeps track of what tasks are using what memory).

In other words we found a new way to reserve memory for ourselves (ML) and prevent Canon tasks from using it (we can't use memory unless we can be sure that another task isn't going to try to use it, for obvious reasons). Before now, we "knew" about this memory (i.e. we know what valid range of RAM address are), we just weren't sure if/when Canon was using it, and we didn't know how to reserve it for ourselves (so we couldn't use it). Canon tasks certainly already know about this allocator and make use of it as they please.

[N/A]

That's what I had assumed, even though I was hoping otherwise. Cheers on the explanation.

[x4kep]

Canon 50D, card Transcend 16GB 600x TS16GCF600
Magic Lantern build jul 18 2014
Raw_rec (mlv_rec always gives the worst result)
Crop mode 1920x1080 24fps

SRM on
no raw, s-jpg2
GD on - 19s
GD off - 25s

s-raw2, no jpg
GD on - 19s
GD off - 27s

SRM off
no raw, s-jpg2
GD on - 11s
GD off - 15s

s-raw2, no jpg
GD on - 10s
GD off - 13s
always two pink frame each shot

Tragic Lantern build mar 3 2014 (no SRM)
Max Resolution
full mode - 1584x1056 (16:9 1584x892)
crop mode - 2000x1080

Raw_rec (mlv_rec always gives the worst result)
Crop mode 1920x1080 24fps
no raw, s-jpg2
GD on - 13s
GD off - 17s
no pink frames or other artifacts

ML raw module with SRM (jul 18 2014) give better recording time, but have pink frames and lower max resolution, than TL raw module (mar 3 2014)