Author Topic: SRM job memory buffers  (Read 118029 times)

Rewind

  • Senior
  • ****
  • Posts: 356
Re: SRM job memory buffers
« Reply #50 on: July 08, 2014, 04:25:54 PM »
Thank you, mk11174 for your findings!
I'd prefer to know about some search 'algorithms' to find the stubs by myself, but i may suppose it is not what's easy to say in two words )

Anyway, now I get the same strange results as yours on 650D:
The test from 'don't click me' shows only 1x31 Mb buffer and dimilligan's mlv_rec only reports 123MB
But mlv patch from gg3g0 still works and gives 154 Mb.

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: SRM job memory buffers
« Reply #51 on: July 08, 2014, 04:48:57 PM »
Thank you, mk11174 for your findings!
I'd prefer to know about some search 'algorithms' to find the stubs by myself, but i may suppose it is not what's easy to say in two words )

Anyway, now I get the same strange results as yours on 650D:
The test from 'don't click me' shows only 1x31 Mb buffer and dimilligan's mlv_rec only reports 123MB
But mlv patch from gg3g0 still works and gives 154 Mb.

This way works to and makes more sense from the comments in the 5d3 stubs file
For 650D
NSTUB(    0x835C,  CreateMemoryChunk)        // called from CreateMemorySuite
NSTUB(    0x8E58,  AddMemoryChunk)           // called before " 32(Src,bytes,Unit)=(%#lx,%#lx,%#lx)" in many places; see also hAddSuite


For 700D written this way acts a bit flakey, so I guess best to use the way I wrote it for 650D which works solid
NSTUB(0xFFA651FC,  CreateMemoryChunk)        // called from CreateMemorySuite
NSTUB(0xFFA65CF8,  AddMemoryChunk)           // called before " 32(Src,bytes,Unit)=(%#lx,%#lx,%#lx)" in many places; see also hAddSuite


Also this way also keep the debug test going after picture takes.

Ah, cool and now it shows 2x31 also and 154MB again in mlv_rec

First time I tried it I had no change, but decided to do a full make clean then recompiled and now shows correct



I use IDA to search, this is where I found them.


500D/T1i  550D/T2i  600D/T3i  700D/T5i

Rewind

  • Senior
  • ****
  • Posts: 356
Re: SRM job memory buffers
« Reply #52 on: July 08, 2014, 05:10:13 PM »
Yes, the 8E58 seems to be correct.
Also, now i get 4x31 MiB outside LV, but 2x31 in LV.
Don't actually understand that. Is it possible to use all 4x31 buffers for mlv recording?

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: SRM job memory buffers
« Reply #53 on: July 08, 2014, 05:17:27 PM »
Yes, the 8E58 seems to be correct.
Also, now i get 4x31 MiB outside LV, but 2x31 in LV.
Don't actually understand that. Is it possible to use all 4x31 buffers for mlv recording?
I have no clue how it works, and cool, never tried outside LV, lol
500D/T1i  550D/T2i  600D/T3i  700D/T5i

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: SRM job memory buffers
« Reply #54 on: July 08, 2014, 05:24:05 PM »
Don't actually understand that. Is it possible to use all 4x31 buffers for mlv recording?
No, LV is using 2 of the buffers itself, that's why you only get 2 in LV. When not in LV you get more b/c you get buffers that LV isn't using. The other buffers are needed by the Canon tasks that take a picture, so if you use them you can't take pictures (a1ex already mentioned this), until you release them again. So basically we are stealing memory from Canon's LV and picture taking tasks. We can use them so long as we are not trying to do one of the things we stole them from.

Rewind

  • Senior
  • ****
  • Posts: 356
Re: SRM job memory buffers
« Reply #55 on: July 08, 2014, 05:46:37 PM »
No, LV is using 2 of the buffers itself, that's why you only get 2 in LV.

Now it's clear. Thank you.
Dmilligan, can you please help us to adopt these changes to legacy raw_rec module?

I am asking because its speed still perceptibly higher than mlv_rec, and that matters for low end cameras.
For example, with my old raw_rec build i've got about 80 seconds of 1536x640 at 24 fps.
But with mlv_rec even with new SRM job buffers only about 50 secs.

This is because raw_rec speed hovers around 38.4 MB/s, but mlv_rec only about 37.
If we'll be able to make raw_rec to use new srm buffers, it will be a huge addition to recording times!

Greg

  • Contributor
  • Hero Member
  • *****
  • Posts: 607
Re: SRM job memory buffers
« Reply #56 on: July 08, 2014, 07:03:58 PM »
That's correct stubs?

500D :
Code: [Select]
NSTUB(0xFF07B90C,  CreateMemoryChunk)
NSTUB(0xFF07C3A8,  AddMemoryChunk)

Rainmanwalking

  • New to the forum
  • *
  • Posts: 13
Re: SRM job memory buffers
« Reply #57 on: July 08, 2014, 11:12:39 PM »
With the latest changes that a1ex made to get a mem_suite from SRM, you can actually use SRM the same way both raw_rec and mlv_rec allocate memory using mem_suite. Basically you just need call srm_malloc_suite(0) to get a mem_suite for SRM, and then you can use it the same way as shoot_malloc_suite(0); This is how I did it for mlv_rec it should be basically the same for raw_rec.
I don't understand much of what your saying but, What I think I'm hearing is these latest changes should also benefit video shot in raw_rec which I use primarily and record sound externally.  That would be great news since a few more seconds could make a world of difference. 

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: SRM job memory buffers
« Reply #58 on: July 09, 2014, 03:30:48 AM »
Dmilligan, can you please help us to adopt these changes to legacy raw_rec module?
https://bitbucket.org/dmilligan/magic-lantern/commits/2c86608a549a7a3b1e85fd17b9acd8ed9214d8c0

Warning: I did not test this at all

dubarry

  • New to the forum
  • *
  • Posts: 32
Re: SRM job memory buffers
« Reply #59 on: July 09, 2014, 05:54:31 AM »
Thanks for that dmilligan (didn't even think legacy raw_rec mo was even used or supported anymore.)

After the bots do their thing I'll def try this out.

Glad you asked Rewind, glad to see I'm not the only old school raw_rec mo around here!

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: SRM job memory buffers
« Reply #60 on: July 09, 2014, 09:53:20 AM »
https://bitbucket.org/dmilligan/magic-lantern/commits/2c86608a549a7a3b1e85fd17b9acd8ed9214d8c0

Warning: I did not test this at all
Confirmed working, on 700D everything is doubled in record frame amount! Nice job dmilligan!
500D/T1i  550D/T2i  600D/T3i  700D/T5i

Rewind

  • Senior
  • ****
  • Posts: 356
Re: SRM job memory buffers
« Reply #61 on: July 09, 2014, 11:13:10 AM »
Confirmed working on 650D. Thank you, dmilligan!

Results at 23.976 fps:

Resolution                     Old rec. time no SRM (GD off)       New rec. time with SRM (GD on)
1568 x 656 (2.39:1)      32 s                                              53 s
1600 x 672 (2.38:1)      18 s                                              32 s
1728 x 720 (2.40:1)      7 s                                                12 s

So we have x1.7 recording time increase even with Global Draw on

NedB

  • Freshman
  • **
  • Posts: 85
Re: SRM job memory buffers
« Reply #62 on: July 09, 2014, 11:51:45 AM »
@mk11174: These results on the 650D from Rewind are very encouraging. If the recording times on the 550D for near-HD (like 1600x672, etc.) start getting up in the 10-20 seconds range, that changes things a lot. I'm turning to you because you were the driving force last year behind getting raw up and running on the 550D.

Am I correct in viewing the current situation as regards the increased SRM?:
1. Various contributors are busy finding stubs and compiling ML and associated modules for particular cameras
2. These compiled ML versions and modules are being tested by these contributors and they are reporting their results
3. Right now, if you want to try out and/or test these cutting-edge versions, you have to compile them yourself

Have you been able to work on the 550D in this context? Do you have an ML version/modules you could share for testing purposes?  Thanks very much in advance and Cheers!
550D - Kit Lens | EF 50mm f/1.8 | Zacuto Z-Finder Pro 2.5x | SanDisk ExtremePro 95mb/s | Tascam DR-100MkII

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 1197
Re: SRM job memory buffers
« Reply #63 on: July 09, 2014, 12:05:33 PM »
Anyone able to compile for 6D and provide it for 6D owners to test recording time increase on 6D? Indeed, increased results posted here are very encouraging. I posted necessary stubs some days ago but am still trying to decide if I buy a 6D or move with something else like 7D. This could help me in making a decision and I think other 6D owners would like to test this, too.

So now see following. These are my findings for 6D_113:

Code: [Select]
NSTUB(0xFF0F0098,  SRM_AllocateMemoryResourceFor1stJob)
NSTUB(0xFF0F40C8,  SRM_FreeMemoryResourceFor1stJob)
NSTUB(0xFFCCE714 - RAM_OFFSET,  CreateMemoryChunk)
NSTUB(0xFFCCF210 - RAM_OFFSET,  AddMemoryChunk)

If you need others then just say so....
70D.112 & 100D.101

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: SRM job memory buffers
« Reply #64 on: July 09, 2014, 12:16:07 PM »
I posted necessary stubs some days ago but am still trying to decide if I buy a 6D

It would be great if you finally decide on a 6D, since this port is all but unmaintained currently, and you seem capable of keeping that port in an active state.  :)

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 1197
Re: SRM job memory buffers
« Reply #65 on: July 09, 2014, 12:52:51 PM »
It would be great if you finally decide on a 6D, since this port is all but unmaintained currently, and you seem capable of keeping that port in an active state.  :)

Hrrrr dunno dunno. I know I can maintain 6D platform and keep nightlies up2date. I did this for android device trees, too  :P
Will make a decision soon and ofc whichever canon body that will be I am going to contribute as much as I can to help this great project. If anyone knows where to grab a 6D for good price (also used) in Germany or Europe then PM me.
70D.112 & 100D.101

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: SRM job memory buffers
« Reply #66 on: July 09, 2014, 02:00:54 PM »
Anyone able to compile for 6D and provide it for 6D owners to test recording time increase on 6D? Indeed, increased results posted here are very encouraging. I posted necessary stubs some days ago but am still trying to decide if I buy a 6D or move with something else like 7D. This could help me in making a decision and I think other 6D owners would like to test this, too.
I have no idea if it even works since I don't have a 6D, but it compiled, so you can test it if you want. I put in the raw_rec and mlv_rec modules with the SRM changes. And added the stubs from this thread.
https://bitbucket.org/mk11174/magic-lantern/downloads/magiclantern-Nightly.2014Jul09.6D113.zip
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: SRM job memory buffers
« Reply #67 on: July 09, 2014, 02:03:51 PM »
Have you been able to work on the 550D in this context? Do you have an ML version/modules you could share for testing purposes?  Thanks very much in advance and Cheers!
Hi, sorry, I have not been testing the 550D with this mem module yet, I think alex was and said he had 31megs extra, not sure if the stubs are added for that one yet, I would have to go find them first then it should be good to test, unless someone already knows them?
500D/T1i  550D/T2i  600D/T3i  700D/T5i

Rewind

  • Senior
  • ****
  • Posts: 356
Re: SRM job memory buffers
« Reply #68 on: July 09, 2014, 02:27:40 PM »
Stubs for 550D (tested. working)

/** ExMem **/
NSTUB(0xFF06C678,  CreateMemorySuite)
NSTUB(0xFF06c4A8,  DeleteMemorySuite)

/** SRM JobMem **/
NSTUB(0xFF028590,  SRM_AllocateMemoryResourceFor1stJob)
NSTUB(0xFF02AD5C,  SRM_FreeMemoryResourceFor1stJob)
NSTUB(0xFF06BEC8,  CreateMemoryChunk)
NSTUB(0xFF06C918,  AddMemoryChunk)

Rewind

  • Senior
  • ****
  • Posts: 356
Re: SRM job memory buffers
« Reply #69 on: July 09, 2014, 02:29:33 PM »
Some quick tests with raw_rec for 550D at 23.976 fps:

Res.               Before               Now
1280x720      4 s                    8 s
1280x592      7 s                    14 s

So basically near the 2x rec. time increase.

But mlv_rec allocates 134 MiB — don't know, if it's new maximum for the 550d?

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: SRM job memory buffers
« Reply #70 on: July 09, 2014, 03:05:42 PM »
Have you been able to work on the 550D in this context? Do you have an ML version/modules you could share for testing purposes?  Thanks very much in advance and Cheers!
Here is a working version with the srm_memory update, and it def works!
https://bitbucket.org/mk11174/magic-lantern/downloads/magiclantern-Nightly.2014Jul09.550D109.zip
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: SRM job memory buffers
« Reply #71 on: July 09, 2014, 03:07:59 PM »

But mlv_rec allocates 134 MiB — don't know, if it's new maximum for the 550d?
I think it is because the shoot total is like 20MB less then the 650D and 700D
500D/T1i  550D/T2i  600D/T3i  700D/T5i

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 1197
Re: SRM job memory buffers
« Reply #72 on: July 09, 2014, 03:16:19 PM »
I have no idea if it even works since I don't have a 6D, but it compiled, so you can test it if you want. I put in the raw_rec and mlv_rec modules with the SRM changes. And added the stubs from this thread.
https://bitbucket.org/mk11174/magic-lantern/downloads/magiclantern-Nightly.2014Jul09.6D113.zip

Yeah I knew that it can campile as a1ex already mentioned it after posting the first stubs for 6d. Now I need to post this in 6D dev thread to get feedback. thanx for your efforts.
70D.112 & 100D.101

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1741
  • 6d - Nightly build user
Re: SRM job memory buffers
« Reply #73 on: July 09, 2014, 04:47:20 PM »
I have no idea if it even works since I don't have a 6D, but it compiled, so you can test it if you want. I put in the raw_rec and mlv_rec modules with the SRM changes. And added the stubs from this thread.
https://bitbucket.org/mk11174/magic-lantern/downloads/magiclantern-Nightly.2014Jul09.6D113.zip

Downloaded this nightly build for 6d, it worsk, but no extra seconds of MLV recording.
In photo mode it has:
Shoot total of 255MB
SRM job total of 4 x 34MB

In movie mode:
Shoot total of 151MB
SRM job total of 3 x 34MB

So we're missing some memory in movie mode, but potentially it has 255MB + 3 x 34MB ?

Rewind

  • Senior
  • ****
  • Posts: 356
Re: SRM job memory buffers
« Reply #74 on: July 09, 2014, 05:14:27 PM »
Downloaded this nightly build for 6d, it worsk, but no extra seconds of MLV recording.

This might be helpful:
Quote from: mk11174
Also reformat card from camera for both test and see if mlv_rec allocates the 154MB before first recording. If you dont reformat you wont see it because it is already created.