Author Topic: Card benchmark - what's the optimal write buffer size?  (Read 124377 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Card benchmark - what's the optimal write buffer size?
« Reply #50 on: May 18, 2013, 09:52:17 AM »
Thanks, looks like randomizing doesn't help.


Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: Card benchmark - what's the optimal write buffer size?
« Reply #51 on: May 18, 2013, 10:03:27 AM »
Wow, quite significant. Need to try it.

An old benchmark from the card before alignment.



And one done just now after alignment.



Couple of extra write MB/s and a heap of read MB/s for free  ;D

Yomommassis

  • New to the forum
  • *
  • Posts: 42
Re: Card benchmark - what's the optimal write buffer size?
« Reply #52 on: May 18, 2013, 10:10:51 AM »
EOS 5D MkIII (ML alpha 3)| EOS 5D MkII (ML v2.3) | EOS 7D (ML alpha 2) | EOS T2i (ML v2.3)
f/2.8 70-200MM L USM | f/1.4 50mm USM | f/1.4 35mm L USM | T/1.5 24mm
Zoom H4n | Rode NTG2

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: Card benchmark - what's the optimal write buffer size?
« Reply #53 on: May 18, 2013, 10:23:19 AM »
https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/ML/BENCHRND.LOG

5D3 - Sandisk 16gb extreme 60MB/s CF.  8 mins (forgot to disable power save).
I can do a better test later tonight if you're still interested.

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3176
Re: Card benchmark - what's the optimal write buffer size?
« Reply #54 on: May 18, 2013, 10:53:04 AM »
can some of you with a LEXAR 1000x post the VRAMx.BMP that is being saved with this plugin?
http://upload.g3gg0.de/pub_files/40763cb6132afa2b257321834ad10154/cf_acc.mo

Debug -> "Read CF details (MAY CAUSE ERR)"
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

nanomad

  • Administrator
  • Hero Member
  • *****
  • Posts: 2918
  • All your websites are belong to us
Re: Card benchmark - what's the optimal write buffer size?
« Reply #55 on: May 18, 2013, 11:20:11 AM »
EOS 650D still running, but the results so far are very very consistent.
Sandisk UHS-I 95MB/s, as long as the buffer is bigger than 15M and it's odd the write speed is 41.4/41.5 MB/S
edit: give it a buffer bigger than 29 MB and it will decrease the speed
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

Atrakt

  • New to the forum
  • *
  • Posts: 11
Re: Card benchmark - what's the optimal write buffer size?
« Reply #56 on: May 18, 2013, 12:05:52 PM »
Here is my Buffer Benchmark on 550D / T2i with SanDisk 8Gb SDHC-I Class 10 95MB/s
Code: [Select]
Buffer size experiment
ML v2.3.NEXT.2013May18.550D109, 1d5cc48c11db (unified) tip
Mode: photo, Global Draw: OFF
10312704 205
15885312 207
80896 102
8949760 207
28661760 207
16264192 207
24521728 208
16424960 208
13148160 206
15619072 207
1658880 204
21001216 208
8548352 207
19760128 208
5965824 206
24906752 207
13968384 207
16770048 208
4542464 205
13063168 206
33263616 209
30312448 207
16713728 208
3887104 207
21841920 209
12846080 206
16192512 208
3333120 203
6719488 208
17313792 205
23374848 207
2865152 204
32488448 208
18588672 207
4186112 206
821248 197
3661824 208
18359296 208
19875840 208
22754304 208
13665280 207
24957952 207
28555264 208
33442816 208
10708992 207
20242432 206
8774656 206
11216896 208
23156736 208
12750848 208
12369920 210
2157568 200
15279104 205
29943808 210
30321664 208
19004416 207
268288 172
23778304 206
467968 179
1905664 204
32368640 209
19829760 207
26281984 207
24153088 208
11364352 208
32160768 207
5944320 207
24118272 208
28511232 208
31526912 209
15956992 209
1732608 203
13881344 208
6106112 205
17367040 210
21693440 207
30835712 209
1139712 197
13698048 207
19388416 208
1532928 200
16333824 207
15963136 207
23540736 208
23770112 207
7317504 207
6019072 208
13303808 206
32039936 208
13137920 209
24311808 207
25442304 208
8986624 207
25421824 208
2720768 203
18526208 207
12448768 208
6514688 206
6632448 206
17897472 207
26569728 208
22568960 206
11912192 207
22007808 207
30651392 208
29615104 207
18367488 208
21049344 207
15778816 207
24951808 208
5359616 206
23476224 208
17125376 205
3318784 203
12782592 207
30965760 208
1974272 204
32021504 206
24463360 207
28246016 208
23352320 205
27161600 211
25191424 208
10841088 206
22846464 208
19111936 206
26234880 208
32610304 209
14385152 209
18029568 208
11080704 207
3933184 205
22414336 207
8314880 207
27988992 208
20073472 209
23487488 209
30233600 211
25299968 208
2406400 202
28059648 209
2145280 201
27503616 208
20705280 208
28731392 208
2458624 202
19662848 209
6198272 207
21655552 207
32225280 209
21788672 206
16669696 209

Hope this help !

EDIT : I'm trying with extFAT this afternoon!

Yomommassis

  • New to the forum
  • *
  • Posts: 42
Re: Card benchmark - what's the optimal write buffer size?
« Reply #57 on: May 18, 2013, 12:09:12 PM »
Messing with partition alignment and noticing some speed increases
can anyone else confirm that the 5D mark 2 cant read exFat formatted cards?
or am I the only one having trouble with this

5D3 and T2i read exFat fine but my 5D2 wont
EOS 5D MkIII (ML alpha 3)| EOS 5D MkII (ML v2.3) | EOS 7D (ML alpha 2) | EOS T2i (ML v2.3)
f/2.8 70-200MM L USM | f/1.4 50mm USM | f/1.4 35mm L USM | T/1.5 24mm
Zoom H4n | Rode NTG2

AndreasK

  • New to the forum
  • *
  • Posts: 42
Re: Card benchmark - what's the optimal write buffer size?
« Reply #58 on: May 18, 2013, 02:24:33 PM »
Alright I need a 1000x so I can help. From the charts I read here the transcend 1000x 64GB should be enough to record 1920x1080@25p is that correct? If so I'm gonny order one from Amazon immediately...

Yomommassis

  • New to the forum
  • *
  • Posts: 42
Re: Card benchmark - what's the optimal write buffer size?
« Reply #59 on: May 18, 2013, 02:44:30 PM »
For the record I only think I did realignment right
I set the partition alignment and allocation to 4096k, but the 5D Mark 2 refused to read any of my cards in any format other than FAT32

550D
SanDisk Extreme 32GB 45MB/s - exFAT 4096 align/allocation - 2 Hours
http://pastebin.com/8a4AUDm8

5D Mark 2
Transcend 32 GB 400x - FAT32 - Unknown alignment (Default: 16k allocation) - 2 Hours
http://pastebin.com/WiSw5WJY

5D Mark 3
Komputer Bay 64 GB 1000x - exFAT 4096 align/allocation - 2 Hours
http://pastebin.com/RARBn4ye
EOS 5D MkIII (ML alpha 3)| EOS 5D MkII (ML v2.3) | EOS 7D (ML alpha 2) | EOS T2i (ML v2.3)
f/2.8 70-200MM L USM | f/1.4 50mm USM | f/1.4 35mm L USM | T/1.5 24mm
Zoom H4n | Rode NTG2

Stedda

  • Hero Member
  • *****
  • Posts: 504
Re: Card benchmark - what's the optimal write buffer size?
« Reply #60 on: May 18, 2013, 02:56:56 PM »
Alright I need a 1000x so I can help. From the charts I read here the transcend 1000x 64GB should be enough to record 1920x1080@25p is that correct? If so I'm gonny order one from Amazon immediately...

Works fine for me...
5D Mark III -- 7D   SOLD -- EOS M 22mm 18-55mm STM -- Fuji X-T1 18-55 F2.8-F4 & 35 F1.4
Canon Glass   100L F2.8 IS -- 70-200L F4 -- 135L F2 -- 85 F1.8 -- 17-40L --  40 F2.8 -- 35 F2 IS  Sigma Glass  120-300 F2.8 OS -- 50 F1.4 -- 85 F1.4  Tamron Glass   24-70 2.8 VC   600EX-RT X3

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Card benchmark - what's the optimal write buffer size?
« Reply #61 on: May 18, 2013, 03:07:23 PM »
Quote
edit: give it a buffer bigger than 29 MB

What is the largest contig chunk size... around there? 600D has 31MB... 32MB writes seemed to help/didn't hurt. 6D has 30 MB.. 32MB writes slowed things down.

I'm going to check alignment with the utility, in windows the starting offset was divisible by 4096 so I guess aligned already?

All my disks were aligned. Sd cars too: http://www.freewarefiles.com/Disk-Alignment-Test_program_71311.html and checked with msinfo.

pio_hyun

  • New to the forum
  • *
  • Posts: 37
Re: Card benchmark - what's the optimal write buffer size?
« Reply #62 on: May 18, 2013, 04:08:10 PM »
Here is my Buffer Benchmark on 6D with SanDisk Extreme Pro 32GB 95MB/s...
Code: [Select]
Buffer size experiment
ML v2.3.NEXT.2013May18.6D113, 25d36d43a82c (unified) tip
Mode: playback, Global Draw: ON
10312704 407
15885312 419
80896 145
8949760 409
28661760 411
16264192 411
24521728 413
16424960 415
13148160 411
15619072 411
1658880 379
21001216 411
8548352 407
19760128 408
5965824 404
24906752 414
13968384 410
16770048 412
4542464 402
13063168 409
33263616 412
30312448 417
16713728 412
3887104 399
21841920 409
12846080 416
16192512 409
3333120 395
6719488 404
17313792 413
23374848 409
2865152 393
32488448 412
18588672 411
4186112 402
821248 349
3661824 400
18359296 408
19875840 409
22754304 414
13665280 410
24957952 412
28555264 414
33442816 411
10708992 408
20242432 410
8774656 406
11216896 412
23156736 411
12750848 410
12369920 415
2157568 386
15279104 409
29943808 416
30321664 412
19004416 409
268288 266
23778304 409
467968 312
1905664 383
32368640 410
19829760 411
26281984 410
24153088 412
11364352 407
32160768 409
5944320 402
24118272 407
28511232 414
31526912 414
15956992 413
1732608 380
13881344 410
6106112 403
17367040 411
21693440 408
30835712 417
1139712 307

CFP

  • Member
  • ***
  • Posts: 185
Re: Card benchmark - what's the optimal write buffer size?
« Reply #63 on: May 18, 2013, 06:32:38 PM »
Hi. I guess since we already know that the EOS 600D wont't be able to write more then 21 MB/s this isn't really helpfull. But here is my test:

I used: Canon EOS 600D | Yesterday's Nightly Build | 16 GB 45 MB/s SanDisk Extreme SDHC

Code: [Select]
Buffer size experiment
ML v2.3.NEXT.2013May17.600D102, cd52bc1653e9 tip
Mode: photo, Global Draw: OFF
10312704 211
15885312 211
80896 81
8949760 209
28661760 210
16264192 211
24521728 212
16424960 210
13148160 211
15619072 211
1658880 201
21001216 211
8548352 209
19760128 211
5965824 209
24906752 213
13968384 209
16770048 210
4542464 207
13063168 210
33263616 212
30312448 212
16713728 210
3887104 207
21841920 212
12846080 211
16192512 211
3333120 205
6719488 208
17313792 212
23374848 211
2865152 203
32488448 212
18588672 210
4186112 209
821248 186
3661824 207
18359296 211
19875840 211
22754304 211
13665280 209
24957952 211
28555264 212
33442816 212
10708992 210
20242432 211
8774656 209
11216896 210
23156736 211
12750848 210
12369920 212
2157568 200
15279104 209
29943808 211
30321664 211
19004416 209
268288 145
23778304 211
467968 168
1905664 201
32368640 209
19829760 211
26281984 211
24153088 210
11364352 210
32160768 212
5944320 208
24118272 210
28511232 211
31526912 212
15956992 211
1732608 201
13881344 211
6106112 208
17367040 212
21693440 212
30835712 212
1139712 193
13698048 210
19388416 212
1532928 197
16333824 211
15963136 211
23540736 211
23770112 212
7317504 209
6019072 209
13303808 209
32039936 212
13137920 210
24311808 212
25442304 212
8986624 210
25421824 210
2720768 203
18526208 211
12448768 210
6514688 206
6632448 208
17897472 210
26569728 212
22568960 211
11912192 211
22007808 212
30651392 211
29615104 210
18367488 211
21049344 211
15778816 210
24951808 212
5359616 208
23476224 212
17125376 211
3318784 205
12782592 211
30965760 212
1974272 203
32021504 211
24463360 212
28246016 212
23352320 211
27161600 212
25191424 208
10841088 210
22846464 211
19111936 212
26234880 211
32610304 213
14385152 211
18029568 212
11080704 210
3933184 207
22414336 210
8314880 210
27988992 212
20073472 210
23487488 212
30233600 213
25299968 211
2406400 200
28059648 210
2145280 200
27503616 208
20705280 212
28731392 211
2458624 202
19662848 213
6198272 209
21655552 211
32225280 212
21788672 212
16669696 211
12007424 211
4321280 206
14878720 209
10118144 209
2836480 204
21092352 211
17086464 211
1707008 200
17509376 211
28161024 211
18457600 212
16290816 211
10758144 210
17766400 211
28011520 212
19835904 212
11879424 209
18191360 211
12581888 213
15906816 210
6525952 209
30545920 210
934912 189
6738944 208
18920448 210
27171840 210
19916800 212
28598272 212
23926784 212
10470400 209
20392960 212
7282688 206
4824064 207
11820032 211
30988288 212
26221568 212
13527040 210
30140416 211
6785024 209
25762816 211
23324672 211
26011648 211
4475904 206
33500160 213
8732672 210
29461504 211
12411904 211
12294144 210
11481088 200
12786688 210
15858688 212
16142336 211
29415424 212
32114688 212
18460672 212
15672320 210
12832768 210
23447552 212
18734080 211
25832448 212
20690944 211
27086848 211
24894464 211
32919552 212
30015488 210
2870272 203
27643904 212
25778176 212
4189184 207
13642752 211
7861248 210
21999616 212
4232192 204
12140544 212
3162112 208
27608064 212
18791424 210
5019648 208
15958016 212
20426752 211
685056 180
18709504 211
6188032 208
6668288 208
30684160 212
9438208 211
20264960 210
1707008 200
4702208 210
27417600 211
31681536 212
1326080 195
9743360 206
29618176 213
28645376 212
21629952 212
21515264 210
12170240 210
29555712 212
2771968 203
30593024 211
23996416 212
31484928 211
24702976 212
24359936 211
32328704 209
21560320 211
18289664 211
26721280 211
33334272 212
29471744 212
30886912 212
10810368 210
16393216 212
4559872 206
17182720 211
251904 142
10129408 208
18564096 211
30530560 213
5036032 207
10642432 210
1025024 190
20564992 211
17504256 211
19713024 211
12450816 209
596992 176
20088832 211
3368960 205
12530688 211
29465600 211
10443776 210
23064576 213
32175104 211
9240576 212
30198784 211
11613184 210
7633920 210
22871040 211
22456320 211
16783360 211
1076224 192
1472512 197
53248 65
22145024 211
13445120 211
15488000 210
17851392 212
8222720 205
3790848 207
31733760 212
4009984 207
29513728 209
29906944 211
25694208 212
16442368 211
3623936 206
29339648 212
27055104 212
12034048 211
32011264 212
32172032 209
22600704 211
13225984 211
28665856 211
11716608 211
28250112 211
5085184 208
28856320 211
218112 134
21444608 211
30626816 213
7068672 208
31740928 209
27124736 212
17964032 211
23683072 212
10264576 211
23499776 212
23206912 211
29933568 211
13522944 211
28731392 212
13669376 210
18453504 211
8773632 209
31521792 212
32563200 212
14050304 210
23192576 211
2580480 204
12610560 211
8542208 209
21291008 211
1245184 206
9519104 211
29709312 212
14656512 210
19511296 211
17680384 211
31774720 212
17035264 211
16637952 212
24604672 211
23776256 212
12297216 211
31228928 211
19137536 211
11517952 211
25170944 211
10852352 209
14022656 213
27422720 212
2764800 200
31773696 212
25122816 212
6719488 208
20889600 213
13719552 207
26656768 212
27925504 212
25307136 212
30016512 210
15267840 211
1992704 202
7454720 211
13520896 211
19296256 212
15080448 210
2733056 203
6084608 208
7767040 209
19521536 211
11791360 206
4573184 203
31201280 202

If you find a way to overclock the SD-Controller, then do it. I don't care if my camera explodes if I can record 1280 X 720 for longer then 3 seconds :p

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Card benchmark - what's the optimal write buffer size?
« Reply #64 on: May 18, 2013, 06:51:11 PM »
No obvious overclock for controller... maybe somehow a few mhz at best.

Colemar

  • Freshman
  • **
  • Posts: 52
Re: Card benchmark - what's the optimal write buffer size?
« Reply #65 on: May 18, 2013, 11:15:31 PM »
can some of you with a LEXAR 1000x post the VRAMx.BMP that is being saved with this plugin?

here's one from a 32GB CF card g3ggo: https://docs.google.com/file/d/0B-UavK-PpYzwQk9xc0F2TUZTMWc/edit?usp=sharing

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3176
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

KMA_WWC

  • Freshman
  • **
  • Posts: 70
Re: Card benchmark - what's the optimal write buffer size?
« Reply #67 on: May 19, 2013, 12:17:15 AM »
Ok, I started the test of the Hoodman Steel 1000x 64gb card (from 5min benchmark tests I get writing speed up to 116mb/s and reading up to 143mb/s). I will run the test until my battery dies and post the results here. Hope it helps :)
Canon 5D Mark III

Hazer

  • Freshman
  • **
  • Posts: 56
Re: Card benchmark - what's the optimal write buffer size?
« Reply #68 on: May 19, 2013, 02:00:32 AM »
Hi guys.

Interested in trying partition alignment on my Lexar 1000x 32GB.  If this results in repeatable speed boost across the board this may be worthy of a thread in its own right with clear instructions for Win/Mac.  For the mean time...

I'm on Mac OS X Mountain Lion.  Here's what gdisk reports about the Lexar:

--

Disk /dev/disk2: 62521200 sectors, 29.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 401BD3D8-8D0B-494B-A00D-87E8368CE132
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 62521166
Partitions will be aligned on 1-sector boundaries
Total free space is 1004 sectors (502.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              63        62520191   29.8 GiB    0700  Microsoft basic data

--

I believe I can use iPartition to adjust the start sector of the card.  Ideally what sector should it be?  Or is the card currently aligned?

Thanks!

skrull

  • New to the forum
  • *
  • Posts: 15
Re: Card benchmark - what's the optimal write buffer size?
« Reply #69 on: May 19, 2013, 03:18:05 AM »
Hi guys.

Interested in trying partition alignment on my Lexar 1000x 32GB.  If this results in repeatable speed boost across the board this may be worthy of a thread in its own right with clear instructions for Win/Mac.  For the mean time...

I'm on Mac OS X Mountain Lion.  Here's what gdisk reports about the Lexar:

--

Disk /dev/disk2: 62521200 sectors, 29.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 401BD3D8-8D0B-494B-A00D-87E8368CE132
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 62521166
Partitions will be aligned on 1-sector boundaries
Total free space is 1004 sectors (502.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              63        62520191   29.8 GiB    0700  Microsoft basic data

--

I believe I can use iPartition to adjust the start sector of the card.  Ideally what sector should it be?  Or is the card currently aligned?

Thanks!

Set 2048 as start sector.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Card benchmark - what's the optimal write buffer size?
« Reply #70 on: May 19, 2013, 03:23:24 AM »
All of the drives I see are aligned x*(4096).. my HDDs and SD cards.

Yomommassis

  • New to the forum
  • *
  • Posts: 42
Re: Card benchmark - what's the optimal write buffer size?
« Reply #71 on: May 19, 2013, 03:35:06 AM »
All of the drives I see are aligned x*(4096).. my HDDs and SD cards.

My Komputer Bay 64 GB 1000x wasn't aligned, maybe the Komputer Bay 128 GB 1000x isn't aligned and that's why it has such slow benchmarks?
EOS 5D MkIII (ML alpha 3)| EOS 5D MkII (ML v2.3) | EOS 7D (ML alpha 2) | EOS T2i (ML v2.3)
f/2.8 70-200MM L USM | f/1.4 50mm USM | f/1.4 35mm L USM | T/1.5 24mm
Zoom H4n | Rode NTG2

platu

  • Member
  • ***
  • Posts: 101
  • 5D3, 60D
Re: Card benchmark - what's the optimal write buffer size?
« Reply #72 on: May 19, 2013, 03:38:52 AM »
My Komputer Bay 64 GB 1000x wasn't aligned, maybe the Komputer Bay 128 GB 1000x isn't aligned and that's why it has such slow benchmarks?

My Komputer 128GB wasn't aligned either.  After alignment there was no change in performance in benchmarks.

Yomommassis

  • New to the forum
  • *
  • Posts: 42
Re: Card benchmark - what's the optimal write buffer size?
« Reply #73 on: May 19, 2013, 03:44:51 AM »
I'm still not 100% on what the "correct" or ideal settings are
what to set the align offset at or the chunk size
EOS 5D MkIII (ML alpha 3)| EOS 5D MkII (ML v2.3) | EOS 7D (ML alpha 2) | EOS T2i (ML v2.3)
f/2.8 70-200MM L USM | f/1.4 50mm USM | f/1.4 35mm L USM | T/1.5 24mm
Zoom H4n | Rode NTG2

Hazer

  • Freshman
  • **
  • Posts: 56
Re: Card benchmark - what's the optimal write buffer size?
« Reply #74 on: May 19, 2013, 04:22:19 AM »
Set 2048 as start sector.

Ok, gave that a shot.  That seemed to drop the write speeds a few MB/s.  I was around 59-60 MB/s before the 2048 setting, now I'm at 57-58 MB/s.  Reformatting the card in the camera restored the original partition settings and the write speeds are now about 59-60 again.

So, couple questions:

One, how exactly is the correct start sector on a FAT32 volume calculated, and are we sure 2048 is correct?

Two, is there any reason CF card performance would degrade over time from recording data at rates like these?

Thanks!