SRM job memory buffers

Started by a1ex, July 02, 2014, 08:55:00 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Rewind

Thank you, mk11174 for your findings!
I'd prefer to know about some search 'algorithms' to find the stubs by myself, but i may suppose it is not what's easy to say in two words )

Anyway, now I get the same strange results as yours on 650D:
The test from 'don't click me' shows only 1x31 Mb buffer and dimilligan's mlv_rec only reports 123MB
But mlv patch from gg3g0 still works and gives 154 Mb.

mk11174

Quote from: Rewind on July 08, 2014, 04:25:54 PM
Thank you, mk11174 for your findings!
I'd prefer to know about some search 'algorithms' to find the stubs by myself, but i may suppose it is not what's easy to say in two words )

Anyway, now I get the same strange results as yours on 650D:
The test from 'don't click me' shows only 1x31 Mb buffer and dimilligan's mlv_rec only reports 123MB
But mlv patch from gg3g0 still works and gives 154 Mb.

This way works to and makes more sense from the comments in the 5d3 stubs file
For 650D
NSTUB(    0x835C,  CreateMemoryChunk)        // called from CreateMemorySuite
NSTUB(    0x8E58,  AddMemoryChunk)           // called before " 32(Src,bytes,Unit)=(%#lx,%#lx,%#lx)" in many places; see also hAddSuite


For 700D written this way acts a bit flakey, so I guess best to use the way I wrote it for 650D which works solid
NSTUB(0xFFA651FC,  CreateMemoryChunk)        // called from CreateMemorySuite
NSTUB(0xFFA65CF8,  AddMemoryChunk)           // called before " 32(Src,bytes,Unit)=(%#lx,%#lx,%#lx)" in many places; see also hAddSuite


Also this way also keep the debug test going after picture takes.

Ah, cool and now it shows 2x31 also and 154MB again in mlv_rec

First time I tried it I had no change, but decided to do a full make clean then recompiled and now shows correct



I use IDA to search, this is where I found them.


500D/T1i  550D/T2i  600D/T3i  700D/T5i

Rewind

Yes, the 8E58 seems to be correct.
Also, now i get 4x31 MiB outside LV, but 2x31 in LV.
Don't actually understand that. Is it possible to use all 4x31 buffers for mlv recording?

mk11174

Quote from: Rewind on July 08, 2014, 05:10:13 PM
Yes, the 8E58 seems to be correct.
Also, now i get 4x31 MiB outside LV, but 2x31 in LV.
Don't actually understand that. Is it possible to use all 4x31 buffers for mlv recording?
I have no clue how it works, and cool, never tried outside LV, lol
500D/T1i  550D/T2i  600D/T3i  700D/T5i

dmilligan

Quote from: Rewind on July 08, 2014, 05:10:13 PM
Don't actually understand that. Is it possible to use all 4x31 buffers for mlv recording?
No, LV is using 2 of the buffers itself, that's why you only get 2 in LV. When not in LV you get more b/c you get buffers that LV isn't using. The other buffers are needed by the Canon tasks that take a picture, so if you use them you can't take pictures (a1ex already mentioned this), until you release them again. So basically we are stealing memory from Canon's LV and picture taking tasks. We can use them so long as we are not trying to do one of the things we stole them from.

Rewind

Quote from: dmilligan on July 08, 2014, 05:24:05 PM
No, LV is using 2 of the buffers itself, that's why you only get 2 in LV.

Now it's clear. Thank you.
Dmilligan, can you please help us to adopt these changes to legacy raw_rec module?

I am asking because its speed still perceptibly higher than mlv_rec, and that matters for low end cameras.
For example, with my old raw_rec build i've got about 80 seconds of 1536x640 at 24 fps.
But with mlv_rec even with new SRM job buffers only about 50 secs.

This is because raw_rec speed hovers around 38.4 MB/s, but mlv_rec only about 37.
If we'll be able to make raw_rec to use new srm buffers, it will be a huge addition to recording times!

Greg

That's correct stubs?

500D :
NSTUB(0xFF07B90C,  CreateMemoryChunk)
NSTUB(0xFF07C3A8,  AddMemoryChunk)

Rainmanwalking

Quote from: dmilligan on July 08, 2014, 12:08:18 AM
With the latest changes that a1ex made to get a mem_suite from SRM, you can actually use SRM the same way both raw_rec and mlv_rec allocate memory using mem_suite. Basically you just need call srm_malloc_suite(0) to get a mem_suite for SRM, and then you can use it the same way as shoot_malloc_suite(0); This is how I did it for mlv_rec it should be basically the same for raw_rec.
I don't understand much of what your saying but, What I think I'm hearing is these latest changes should also benefit video shot in raw_rec which I use primarily and record sound externally.  That would be great news since a few more seconds could make a world of difference. 

dmilligan


dubarry

Thanks for that dmilligan (didn't even think legacy raw_rec mo was even used or supported anymore.)

After the bots do their thing I'll def try this out.

Glad you asked Rewind, glad to see I'm not the only old school raw_rec mo around here!

mk11174

Quote from: dmilligan on July 09, 2014, 03:30:48 AM
https://bitbucket.org/dmilligan/magic-lantern/commits/2c86608a549a7a3b1e85fd17b9acd8ed9214d8c0

Warning: I did not test this at all
Confirmed working, on 700D everything is doubled in record frame amount! Nice job dmilligan!
500D/T1i  550D/T2i  600D/T3i  700D/T5i

Rewind

Confirmed working on 650D. Thank you, dmilligan!

Results at 23.976 fps:

Resolution                     Old rec. time no SRM (GD off)       New rec. time with SRM (GD on)
1568 x 656 (2.39:1)      32 s                                              53 s
1600 x 672 (2.38:1)      18 s                                              32 s
1728 x 720 (2.40:1)      7 s                                                12 s

So we have x1.7 recording time increase even with Global Draw on

NedB

@mk11174: These results on the 650D from Rewind are very encouraging. If the recording times on the 550D for near-HD (like 1600x672, etc.) start getting up in the 10-20 seconds range, that changes things a lot. I'm turning to you because you were the driving force last year behind getting raw up and running on the 550D.

Am I correct in viewing the current situation as regards the increased SRM?:
1. Various contributors are busy finding stubs and compiling ML and associated modules for particular cameras
2. These compiled ML versions and modules are being tested by these contributors and they are reporting their results
3. Right now, if you want to try out and/or test these cutting-edge versions, you have to compile them yourself

Have you been able to work on the 550D in this context? Do you have an ML version/modules you could share for testing purposes?  Thanks very much in advance and Cheers!
550D - Kit Lens | EF 50mm f/1.8 | Zacuto Z-Finder Pro 2.5x | SanDisk ExtremePro 95mb/s | Tascam DR-100MkII

nikfreak

Anyone able to compile for 6D and provide it for 6D owners to test recording time increase on 6D? Indeed, increased results posted here are very encouraging. I posted necessary stubs some days ago but am still trying to decide if I buy a 6D or move with something else like 7D. This could help me in making a decision and I think other 6D owners would like to test this, too.

Quote from: nikfreak on July 03, 2014, 01:52:13 PM
So now see following. These are my findings for 6D_113:

NSTUB(0xFF0F0098,  SRM_AllocateMemoryResourceFor1stJob)
NSTUB(0xFF0F40C8,  SRM_FreeMemoryResourceFor1stJob)
NSTUB(0xFFCCE714 - RAM_OFFSET,  CreateMemoryChunk)
NSTUB(0xFFCCF210 - RAM_OFFSET,  AddMemoryChunk)


If you need others then just say so....
[size=8pt]70D.112 & 100D.101[/size]

Audionut

Quote from: nikfreak on July 09, 2014, 12:05:33 PM
I posted necessary stubs some days ago but am still trying to decide if I buy a 6D

It would be great if you finally decide on a 6D, since this port is all but unmaintained currently, and you seem capable of keeping that port in an active state.  :)

nikfreak

Quote from: Audionut on July 09, 2014, 12:16:07 PM
It would be great if you finally decide on a 6D, since this port is all but unmaintained currently, and you seem capable of keeping that port in an active state.  :)

Hrrrr dunno dunno. I know I can maintain 6D platform and keep nightlies up2date. I did this for android device trees, too  :P
Will make a decision soon and ofc whichever canon body that will be I am going to contribute as much as I can to help this great project. If anyone knows where to grab a 6D for good price (also used) in Germany or Europe then PM me.
[size=8pt]70D.112 & 100D.101[/size]

mk11174

Quote from: nikfreak on July 09, 2014, 12:05:33 PM
Anyone able to compile for 6D and provide it for 6D owners to test recording time increase on 6D? Indeed, increased results posted here are very encouraging. I posted necessary stubs some days ago but am still trying to decide if I buy a 6D or move with something else like 7D. This could help me in making a decision and I think other 6D owners would like to test this, too.
I have no idea if it even works since I don't have a 6D, but it compiled, so you can test it if you want. I put in the raw_rec and mlv_rec modules with the SRM changes. And added the stubs from this thread.
https://bitbucket.org/mk11174/magic-lantern/downloads/magiclantern-Nightly.2014Jul09.6D113.zip
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

Quote from: NedB on July 09, 2014, 11:51:45 AM
Have you been able to work on the 550D in this context? Do you have an ML version/modules you could share for testing purposes?  Thanks very much in advance and Cheers!
Hi, sorry, I have not been testing the 550D with this mem module yet, I think alex was and said he had 31megs extra, not sure if the stubs are added for that one yet, I would have to go find them first then it should be good to test, unless someone already knows them?
500D/T1i  550D/T2i  600D/T3i  700D/T5i

Rewind

Stubs for 550D (tested. working)

/** ExMem **/
NSTUB(0xFF06C678,  CreateMemorySuite)
NSTUB(0xFF06c4A8,  DeleteMemorySuite)

/** SRM JobMem **/
NSTUB(0xFF028590,  SRM_AllocateMemoryResourceFor1stJob)
NSTUB(0xFF02AD5C,  SRM_FreeMemoryResourceFor1stJob)
NSTUB(0xFF06BEC8,  CreateMemoryChunk)
NSTUB(0xFF06C918,  AddMemoryChunk)

Rewind

Some quick tests with raw_rec for 550D at 23.976 fps:

Res.               Before               Now
1280x720      4 s                    8 s
1280x592      7 s                    14 s

So basically near the 2x rec. time increase.

But mlv_rec allocates 134 MiB — don't know, if it's new maximum for the 550d?

mk11174

Quote from: NedB on July 09, 2014, 11:51:45 AM
Have you been able to work on the 550D in this context? Do you have an ML version/modules you could share for testing purposes?  Thanks very much in advance and Cheers!
Here is a working version with the srm_memory update, and it def works!
https://bitbucket.org/mk11174/magic-lantern/downloads/magiclantern-Nightly.2014Jul09.550D109.zip
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

Quote from: Rewind on July 09, 2014, 02:29:33 PM

But mlv_rec allocates 134 MiB — don't know, if it's new maximum for the 550d?
I think it is because the shoot total is like 20MB less then the 650D and 700D
500D/T1i  550D/T2i  600D/T3i  700D/T5i

nikfreak

Quote from: mk11174 on July 09, 2014, 02:00:54 PM
I have no idea if it even works since I don't have a 6D, but it compiled, so you can test it if you want. I put in the raw_rec and mlv_rec modules with the SRM changes. And added the stubs from this thread.
https://bitbucket.org/mk11174/magic-lantern/downloads/magiclantern-Nightly.2014Jul09.6D113.zip

Yeah I knew that it can campile as a1ex already mentioned it after posting the first stubs for 6d. Now I need to post this in 6D dev thread to get feedback. thanx for your efforts.
[size=8pt]70D.112 & 100D.101[/size]

Levas

Quote from: mk11174 on July 09, 2014, 02:00:54 PM
I have no idea if it even works since I don't have a 6D, but it compiled, so you can test it if you want. I put in the raw_rec and mlv_rec modules with the SRM changes. And added the stubs from this thread.
https://bitbucket.org/mk11174/magic-lantern/downloads/magiclantern-Nightly.2014Jul09.6D113.zip

Downloaded this nightly build for 6d, it worsk, but no extra seconds of MLV recording.
In photo mode it has:
Shoot total of 255MB
SRM job total of 4 x 34MB

In movie mode:
Shoot total of 151MB
SRM job total of 3 x 34MB

So we're missing some memory in movie mode, but potentially it has 255MB + 3 x 34MB ?

Rewind

Quote from: Levas on July 09, 2014, 04:47:20 PM
Downloaded this nightly build for 6d, it worsk, but no extra seconds of MLV recording.

This might be helpful:
Quote from: mk11174Also reformat card from camera for both test and see if mlv_rec allocates the 154MB before first recording. If you dont reformat you wont see it because it is already created.