### Author Topic: 550D/T2i raw video recording port official thread  (Read 978862 times)

#### a1ex

• Hero Member
• Posts: 12564
##### Re: 550D raw video recording port official thread
« Reply #625 on: June 19, 2013, 07:36:11 PM »
The 9x8 looks close to optimal for this case. I've got only 1910 frames with the variable buffer method in simulation (compared to 1565 for 9x8MB).

A question for you: what write speed are you getting if you only write one frame at a time? (that is, split the buffers in very small chunks equal to frame_size - and as many as you can - should be 81 stars for the numbers above). This will be helpful in fine-tuning the speed curve.

Edit: current simulation results (1280x426, 23.976, speed tuned to 21.16MB/s to match 1565 frames):

8MB + 2x32MB - 317 frames:

9x8MB - 1566 frames:

Variable buffering, starting from 8MB + 2x32MB - 1910 frames:

X: time in seconds; Y: number of buffered frames.

All 3 cases with the same write speed and the same amount of total memory.

Warning: this is pure theory, I didn't try this on any camera. But you can see the mathematical background behind raw recording, study the algorithms and suggest improvements.

Note: if write speed wouldn't vary with buffer size, the optimal algorithm would be trivial: just write one frame at a time.

#### qsara

• Guest
##### Re: 550D raw video recording port official thread
« Reply #626 on: June 19, 2013, 07:53:53 PM »
to my experience, sandisk extreme cards not only capable of writing sequentially but also in parallel. I think we can push more data into card with that strategy.

#### Kim.dh

• New to the forum
• Posts: 43
##### Re: 550D raw video recording port official thread
« Reply #627 on: June 19, 2013, 07:59:06 PM »
This is pretty awesome. Would this mean a few seconds of 1080p is possible?

#### a1ex

• Hero Member
• Posts: 12564
##### Re: 550D raw video recording port official thread
« Reply #628 on: June 19, 2013, 08:12:41 PM »
No. At 1920x1080, the simulation says:
current method: 19 frames
optimized by splitting and reordering buffers: 27 frames
variable buffering: 25

At 1728x972: 24/34/34 frames.

#### MaKsOZ

• New to the forum
• Posts: 16
##### Re: 550D raw video recording port official thread
« Reply #629 on: June 19, 2013, 08:17:14 PM »
No. At 1920x1080, the simulation says:
current method: 19 frames
optimized by splitting and reordering buffers: 27 frames
variable buffering: 25

At 1728x972: 24/34/34 frames.

And at 720p what is the max frames ?

Also if we shoot with the 2.35 ratio, how many frame @1920 ?

Thanks Alex

#### a1ex

• Hero Member
• Posts: 12564
##### Re: 550D raw video recording port official thread
« Reply #630 on: June 19, 2013, 08:20:56 PM »
1280x720: 50/96/101.
1728x736: 31/50/51.

Try Xaint's zip, see if the theory makes any sense.

#### qsara

• Guest
##### Re: 550D raw video recording port official thread
« Reply #631 on: June 19, 2013, 08:32:38 PM »
No. At 1920x1080, the simulation says:
current method: 19 frames
optimized by splitting and reordering buffers: 27 frames
variable buffering: 25

At 1728x972: 24/34/34 frames.

just tried xaint's code, I can get

60 frames 1920x480
64 frames 1728x576
114 frames 1728x432
280 frames 1728x346
196 frames 1600x400
837 frames 1600x320
126 frames 1472x490
600 frames 1472x368
151 frames 1344x584
242 frames 1344x448
88 frames 1280x720
106 frames 1280x640
114 frames 1280x582
151 frames 1280x544
138 frames 1280x536
196 frames 1280x512
227 frames 1280x480
471 frames 1280x426
continuous recording 1280x320

all at 23.976fps with Sandisk Extreme 30MB/s 8GB

#### mk11174

• Contributor
• Hero Member
• Posts: 809
##### Re: 550D raw video recording port official thread
« Reply #632 on: June 19, 2013, 09:16:12 PM »
Me too
DUDE!!!! That is much much BETTER!!!!! WOW!!!
500D/T1i  550D/T2i  600D/T3i  700D/T5i

#### Rewind

• Senior
• Posts: 356
##### Re: 550D raw video recording port official thread
« Reply #633 on: June 19, 2013, 09:26:40 PM »
Yes, the method of 8Mb+15Mb+15Mb+16Mb+16Mb gives me 164 frames with 1280x536 at 23.976 fps for now. It's quite a step forward.

For example, recently i could get only 192 frames at 1152x482 at 24fps, now it is more than 730 (usually 739).

#### dougie

• New to the forum
• Posts: 3
##### Re: 550D raw video recording port official thread
« Reply #634 on: June 19, 2013, 09:44:29 PM »
great job, now getting continuous shooting @ 1152x432 on a 64GB sandisk extreme pro sdxc 95MB/s and the last rar posted by Xaint

other numbers are:

190f @ 1152x576 2:1
295f @ 1152x524 2.20:1
544f @ 1152x490 2.35:1
310f @ 1344x448 3:1
164f @ 1344x504 2.67:1
126f @ 1344x538 2.5:1
114f @ 1344x562 2.39:1
114f @ 1344x562 2.39:1
114f @ 1344x572 2.35:1
106f @ 1344x610 2.20:1
80f @ 1344x672 2:1
62f @ 1344x756 16:9
80f @ 1280x720 16:9
151f @ 1280x544 2.35:1
1010 @ 1280x426 3:1
280 @ 1280x480 2.67:1
666 @ 960x576 5:3
280 @ 960x640 3:2

#### marekk

• Senior
• Posts: 331
• www.oddajmisie.com
##### Re: 550D raw video recording port official thread
« Reply #635 on: June 19, 2013, 09:57:02 PM »
what should I change in raw_rec.c to allocate 16MB memory chunks instead of 32 ?

#### xaint

• New to the forum
• Posts: 46
##### Re: 550D raw video recording port official thread
« Reply #636 on: June 19, 2013, 10:04:21 PM »
The 9x8 looks close to optimal for this case. I've got only 1910 frames with the variable buffer method in simulation (compared to 1565 for 9x8MB).

A question for you: what write speed are you getting if you only write one frame at a time? (that is, split the buffers in very small chunks equal to frame_size - and as many as you can - should be 81 stars for the numbers above). This will be helpful in fine-tuning the speed curve.

Edit: current simulation results (1280x426, 23.976, speed tuned to 21.16MB/s to match 1565 frames):

Warning: this is pure theory, I didn't try this on any camera. But you can see the mathematical background behind raw recording, study the algorithms and suggest improvements.

Note: if write speed wouldn't vary with buffer size, the optimal algorithm would be trivial: just write one frame at a time.

I think it's not that easy

With frame_size sized buffers, it says 19.8Mb/sec  (~1100 frames). So I think we can rule this theory out

As I can see it now, it not just depends on the buffer size, but also on the resolution.    Here's how:

With 1152x460 the 1x8Mb + 4x16Mb setup gives better results:

 Sorting Buffers Frames Seconds asc 9x8Mb 4371 ~182 asc 1x8Mb + 4x16Mb 5168 ~215

But in 1280x426, inversely, the 9x8Mb gives the better results:

 Sorting Buffers Frames Seconds asc 9x8Mb 1524 ~63 asc 1x8Mb + 4x16Mb 1395 ~58

The write speeds are better with the 1x8Mb + 4x16Mb setup, in BOTH cases! It says constant 21.0Mb/sec...
With 9x8Mb however it says 20.8-20.9Mb/sec in both cases!

All tests were performed twice, gives almost the same results! (but there may be some randomness... )

Maybe I'm stupid, but isn't it possible, that the 16Mb buffers are faster than the 8Mb ones?
If I believe in this diagram:

I've lost track, help me out!!!

EDIT: I think the write speed vary. A lot.

#### notdabod

• New to the forum
• Posts: 29
##### Re: 550D raw video recording port official thread
« Reply #637 on: June 19, 2013, 10:09:32 PM »

To be honest , I also thought, that the more buffers are better, even if the overall size isn't bigger, I don't know why I haven't tried so far: D, but here it is:

Some really quick test results here, on 550D with SanDisk ExtremePro 16GB, 1280x426 @ 24fps:

 Sorting Buffers Frames Seconds asc 8Mb+31Mb+31Mb 317 ~13 desc 16Mb+16Mb+15Mb+15Mb+8Mb 1326 ~55 asc 8Mb+15Mb+15Mb+16Mb+16Mb 1395 ~58 none 15Mb+15Mb+16Mb+16Mb+8Mb 1326 ~55

Try and let me know:

Great findings by the way a1ex, I'm still experimenting with this, and with your speedsym.py script.. looks like it worth the try thanks!

EDIT (9 x 8Mb):

 Sorting Buffers Frames Seconds - 8Mb+8Mb+8Mb+8Mb+8Mb+8Mb+8Mb+8Mb+8Mb 1565 ~65

With this new xaint I get twice the number of frames!

8Mb+15Mb+15Mb+16Mb+16Mb

was getting:

80 frames at 1152x460 with 23.976 fps with my Kingston sd card which is the same speed as the sandisk 45 mb/s.

now getting

162 frames at 1152x460 with 23.976 fps with my Kingston sd card which is the same speed as the sandisk 45 mb/s.

#### xaint

• New to the forum
• Posts: 46
##### Re: 550D raw video recording port official thread
« Reply #638 on: June 19, 2013, 10:12:45 PM »
what should I change in raw_rec.c to allocate 16MB memory chunks instead of 32 ?

I don't know whether I'm doing it well or not, a1ex will tell, but:

Code: [Select]
`                       buffers[buffer_count].ptr = (void*)(((intptr_t)ptr + 4095) & ~4095);                buffers[buffer_count].size = size/2 - 8192;                buffers[buffer_count].used = 0;                buffers[buffer_count+1].ptr = (void*)(((intptr_t)ptr+size/2 + 4095) & ~4095);                buffers[buffer_count+1].size = size/2 - 8192;                buffers[buffer_count+1].used = 0;                buffer_count+=2;                total += size;`

#### a1ex

• Hero Member
• Posts: 12564
##### Re: 550D raw video recording port official thread
« Reply #639 on: June 19, 2013, 10:24:22 PM »
Okay guys, got some homework for you.

Take a look here: http://www.magiclantern.fm/forum/index.php?topic=5471

and run the same test (Debug - Card buffer benchmark), but this time in movie mode. The camera will shutdown after half an hour, so touch the half-shutter every now and then. Use any of the autoexecs from this thread.

Then post the logs.

#### xaint

• New to the forum
• Posts: 46
##### Re: 550D raw video recording port official thread
« Reply #640 on: June 19, 2013, 10:32:05 PM »
Okay guys, got some homework for you.

Take a look here: http://www.magiclantern.fm/forum/index.php?topic=5471

and run the same test (Debug - Card buffer benchmark), but this time in movie mode. The camera will shutdown after half an hour, so touch the half-shutter every now and then. Use any of the autoexecs from this thread.

Then post the logs.

As you wish sir!

I think there is a lot of factors, that affects the write speed, including the card itself, the buffer distribution, the order of the buffers, the frame_size, (the varying cpu load maybe?) and so on.. So good luck a1ex to find the best solution, what works best in all cases!
I'll try to help if I can!

#### qsara

• Guest
##### Re: 550D raw video recording port official thread
« Reply #641 on: June 19, 2013, 11:00:33 PM »
I've noticed sth about sandisk extreme 45mb/s cards. after 6 months of intense use write rates drops about 1mb/s, because I've recently bought 3 new 45MB/s cards and they are all faster than my older 2 45MB/s cards. I'm surprised to see the result, because I also have a sandisk extreme III 20MB/s edition 6 yr old, works at same performance all along.

anyone noticed sth like that?

and @a1ex, is it possible to partition eos m's memory like that?

#### marekk

• Senior
• Posts: 331
• www.oddajmisie.com
##### Re: 550D raw video recording port official thread
« Reply #642 on: June 19, 2013, 11:09:25 PM »
60D hangs with this

I don't know whether I'm doing it well or not, a1ex will tell, but:

Code: [Select]
`                       buffers[buffer_count].ptr = (void*)(((intptr_t)ptr + 4095) & ~4095);                buffers[buffer_count].size = size/2 - 8192;                buffers[buffer_count].used = 0;                buffers[buffer_count+1].ptr = (void*)(((intptr_t)ptr+size/2 + 4095) & ~4095);                buffers[buffer_count+1].size = size/2 - 8192;                buffers[buffer_count+1].used = 0;                buffer_count+=2;                total += size;`

#### dougie

• New to the forum
• Posts: 3
##### Re: 550D raw video recording port official thread
« Reply #643 on: June 19, 2013, 11:11:22 PM »

With this new xaint I get twice the number of frames!

8Mb+15Mb+15Mb+16Mb+16Mb

was getting:

80 frames at 1152x460 with 23.976 fps with my Kingston sd card which is the same speed as the sandisk 45 mb/s.

now getting

162 frames at 1152x460 with 23.976 fps with my Kingston sd card which is the same speed as the sandisk 45 mb/s.

That's strange, I get 2242 frames @ 1152x460 23.976fps on a sandisk 95MB/s but the real bottle neck is the ~20MB/s 550d SD hw

#### marregi

• New to the forum
• Posts: 8
##### Re: 550D raw video recording port official thread
« Reply #644 on: June 19, 2013, 11:41:00 PM »
Did my homework
Code: [Select]
`Buffer size experimentML v2.3.NEXT.2013Jun19.550D109, f3631da68349+Mode: movie 1920x1080 24p, Global Draw: OFFSandisk 16GB 45MB/s10312704 19515885312 20280896 698949760 20128661760 20416264192 20524521728 20016424960 20313148160 20115619072 2021658880 18821001216 2018548352 20319760128 2025965824 19624906752 20213968384 20116770048 2004542464 19713063168 20533263616 20230312448 20216713728 2033887104 19821841920 20212846080 20416192512 2053333120 1976719488 19717313792 20623374848 2012865152 19232488448 20018588672 2054186112 199821248 1793661824 20018359296 20119875840 20122754304 20513665280 20224957952 20328555264 20333442816 20510708992 20020242432 2038774656 19911216896 20323156736 20112750848 20112369920 2042157568 18815279104 20329943808 20030321664 20419004416 206268288 12923778304 204467968 1521905664 19432368640 20419829760 20326281984 20024153088 20411364352 20232160768 2045944320 19924118272 20428511232 20431526912 20515956992 2031732608 19313881344 2056106112 19817367040 20421693440 20030835712 2011139712 17713698048 20519388416 2031532928 18616333824 20315963136 20223540736 20523770112 2057317504 2016019072 20113303808 20232039936 20413137920 19924311808 20125442304 2048986624 20225421824 2042720768 19118526208 20212448768 2026514688 1996632448 19917897472 20226569728 20522568960 20511912192 20022007808 20430651392 20229615104 20418367488 20321049344 20115778816 20324951808 2025359616 19923476224 20117125376 2013318784 19412782592 20430965760 2021974272 19432021504 20624463360 20228246016 20623352320 20427161600 20325191424 20510841088 20322846464 20219111936 20126234880 20432610304 20214385152 20218029568 20311080704 2013933184 19722414336 2018314880 20027988992 20420073472 20523487488 20330233600 20425299968 2022406400 19328059648 2042145280 19027503616 20120705280 20528731392 2022458624 19119662848 2046198272 19921655552 20532225280 20121788672 20216669696 20212007424 2024321280 19614878720 20410118144 1992836480 19121092352 20217086464 2021707008 19217509376 20528161024 20418457600 20116290816 20110758144 20317766400 20328011520 20319835904 20111879424 20318191360 20212581888 20115906816 2056525952 19930545920 200934912 1786738944 20118920448 20527171840 20319916800 20628598272 20323926784 20210470400 20420392960 2017282688 2024824064 19711820032 20130988288 20326221568 20613527040 20430140416 2036785024 20025762816 20123324672 20326011648 2034475904 19833500160 2068732672 20229461504 20212411904 20212294144 20311481088 20012786688 19915858688 20516142336 20429415424 20532114688 20218460672 20315672320 20312832768 20123447552 20318734080 20125832448 20220690944 20227086848 20624894464 20332919552 20330015488 2042870272 19127643904 20325778176 2024189184 20113642752 2017861248 20021999616 2064232192 19812140544 2003162112 19727608064 20318791424 2055019648 20015958016 20220426752 206685056 16418709504 2026188032 2016668288 20330684160 2009438208 20420264960 2051707008 1924702208 20227417600 20131681536 2041326080 1839743360 20129618176 20428645376 20121629952 20721515264 211 `

#### mk11174

• Contributor
• Hero Member
• Posts: 809
##### Re: 550D raw video recording port official thread
« Reply #645 on: June 19, 2013, 11:51:54 PM »
WOW!!! 1725 frames at 1280x426x23.976  Very nice!!!
500D/T1i  550D/T2i  600D/T3i  700D/T5i

#### 1%

• Developer
• Hero Member
• Posts: 5936
• 600D/6D/50D/EOSM/7D
##### Re: 550D raw video recording port official thread
« Reply #646 on: June 19, 2013, 11:58:02 PM »
I did this to 50D and write speed fell... but idle time did too.

Well thing is size/2 makes ~13mb buffers so maybe having them at 16MB would be ideal.

#### xaint

• New to the forum
• Posts: 46
##### Re: 550D raw video recording port official thread
« Reply #647 on: June 20, 2013, 12:14:40 AM »
I did this to 50D and write speed fell... but idle time did too.

Well thing is size/2 makes ~13mb buffers so maybe having them at 16MB would be ideal.

What do you mean that size/2 makes ~13Mb buffers? How big the size then? I don't know the 50D, in 550D we have 2x31Mb, and the remaining 7-8Mb from the 75Mb shoot_malloc.

#### xaint

• New to the forum
• Posts: 46
##### Re: 550D raw video recording port official thread
« Reply #648 on: June 20, 2013, 12:24:39 AM »
Homework

(The cam overheated, sorry. Today was the hottest day of the summer)

Mode: movie 1920x1080 25p, Global Draw: OFF
SanDisk ExtremePro 16GB

Code: [Select]
`10312704 20715885312 20880896 868949760 20728661760 20816264192 20624521728 20916424960 20713148160 20815619072 2061658880 19521001216 2088548352 20619760128 2085965824 20424906752 20713968384 20716770048 2074542464 20213063168 20633263616 20830312448 20816713728 2083887104 20321841920 20612846080 20716192512 2073333120 2026719488 20417313792 20723374848 2062865152 19832488448 20818588672 2074186112 203821248 1853661824 20418359296 20819875840 20822754304 20813665280 20624957952 20828555264 20633442816 20810708992 20420242432 2078774656 20611216896 20423156736 20812750848 20512369920 2092157568 19515279104 20529943808 20930321664 20619004416 203268288 14523778304 209467968 1651905664 19832368640 20919829760 20626281984 20724153088 20711364352 20532160768 2075944320 20624118272 20728511232 20931526912 20715956992 2081732608 19613881344 2086106112 20417367040 20921693440 20730835712 2091139712 18813698048 20319388416 2081532928 19516333824 20515963136 20823540736 20723770112 2087317504 2066019072 20413303808 20632039936 20713137920 20624311808 20725442304 2088986624 20625421824 2082720768 20018526208 20912448768 2066514688 2036632448 20517897472 20626569728 20622568960 20811912192 20622007808 20630651392 20929615104 20818367488 20721049344 20615778816 20624951808 2085359616 20023476224 20217125376 2083318784 20112782592 20530965760 2091974272 19832021504 20924463360 20528246016 20823352320 20327161600 20825191424 20610841088 20622846464 20819111936 20726234880 20732610304 20914385152 20718029568 20811080704 2053933184 20122414336 2088314880 20427988992 20920073472 20723487488 20830233600 20525299968 2082406400 19628059648 2082145280 19627503616 20720705280 20828731392 2072458624 19819662848 2086198272 20421655552 20832225280 20621788672 20716669696 20612007424 2054321280 20114878720 20710118144 2052836480 20021092352 20717086464 2051707008 19417509376 20528161024 20818457600 20616290816 20710758144 20417766400 20428011520 20819835904 20211879424 20518191360 20812581888 20315906816 2086525952 20330545920 206934912 1866738944 20418920448 20427171840 20819916800 20728598272 20723926784 20710470400 20620392960 2067282688 2014824064 20211820032 20730988288 20526221568 20413527040 20730140416 2046785024 20225762816 20723324672 20526011648 2054475904 20033500160 2058732672 20629461504 20712411904 20312294144 20711481088 20512786688 20515858688 20616142336 20329415424 20632114688 20818460672 20515672320 20612832768 20723447552 20618734080 20625832448 20620690944 20727086848 20624894464 20832919552 20730015488 2082870272 19727643904 20825778176 2064189184 20213642752 2047861248 20321999616 2024232192 20212140544 2063162112 20127608064 20818791424 2045019648 20315958016 20820426752 207685056 17418709504 2066188032 2046668288 20530684160 2079438208 20420264960 2061707008 1924702208 20427417600 20831681536 2061326080 1929743360 20729618176 20628645376 20821629952 20621515264 20612170240 20529555712 2062771968 19830593024 20923996416 20631484928 208`

#### intrinsic

• New to the forum
• Posts: 8
##### Re: 550D raw video recording port official thread
« Reply #649 on: June 20, 2013, 01:53:14 AM »
Is this buffering aspect has anything to do with increasing card write speed at lower FPS using FPS override? Or is it CPU problem?
I've first noticed that on 5dMkII, and 550D shows the same, it goes up to 22Mb/s at the beginning of the recording (than drops around 21), @FPS 8-12