UHS-I / SD cards investigation

[theBilalFakhouri]

5D3:

192MHz and 240 MHz presets do work but it has same problem as some other DIGIC 5 cameras, it either switch to safe mode which is SDR50 @ 48 MHz, or the camera make the SD card inaccessible resulting in "Card full" issue when recording RAW video, or camera lock up or crash when trying to capture Still photo or record H.264 video with error message until you restart the camera by pulling the battery out

And these happens when camera try to make a read process to SD card (This is my guess, I talked about this earlier in this thread), as long as there is no reading happening, the presets just work:

5D3 113 @ 240 MHz in PLAY mode (Sandisk Extreme PRO UHS-I U3 170 MB/s):


Of course currently these presets are unpractical right now, How I made it work:

1- First you need an empty SD card, only ML files on it (Sometimes it works with some MLV files)
2- You need to apply the patch after the camera fully startup (after red led become off), applying the patch during startup won't work unless you are very lucky (because -I think- during camera startup there are reading processes happening like loading ML files into memory, or Canon firmware checking SD card content . . etc)

(What about trying these steps on EOS M, 240 MHz might work, but won't be practical, just for testing)

Well, when 240 MHz preset works you get:


~81 MB write speed in video mode

Who needs CF card?

-I didn't test it with CF card yet (with Card spanning), waiting the CF to arrive . .

You won't get anything from this post, further research is required . . but it's cool to showcase how 240 MHz preset would perform when it works.

[Grognard]

-I didn't test it with CF card yet (with Card spanning), waiting the CF to arrive . .

You won't get anything from this post, further research is required . . but it's cool to showcase how 240 MHz preset would perform when it works.

Great. Wait and see.
Does it mean you finally got a 5diii?

[theBilalFakhouri]

It seems CF + SD card spanning maxes at ~150 MB/s (in PLAY mode) using all presets 160 MHz, 192 MHz and 240 MHz, so I think we hit memory limit here (is there a workaround for this?).

I couldn't test 192 MHz and 240 MHz in LiveView with card spanning I expect to be same as 160 MHz with card spanning which is maxes at ~135 MB/s in LiveView when recording RAW (uncompressed RAW, 23.976 FPS)
-Lossless compression make a bottleneck BTW due to extra processing/memory cycle, which make write speed with card spanning below 135 MB/s .

Does it mean you finally got a 5diii?

Yes, thanks to "unknown soldiers" who gave me the opportunity to get a 5D3 and work on it .

[Jonneh]

Yes, thanks to "unknown soldiers" who gave me the opportunity to get a 5D3 and work on it .

Just when I was thinking it was about time we pooled some resources to buy one, the soldier stepped forward. Good on him or her. Enjoy it! :-)

Regarding the stability issues with the faster overclocking presets, the explanation you propose is interesting. It would be a boon if there turns out to be a workaround, but only if the data rate bottleneck can be overcome, naturally.

Regarding this apparent 135/150 MB/s bottleneck (I'd noticed the 135 MB/s limit before when the total rate with a fast CF + SD didn't benefit from SD overclocking in terms of total data rate---doing so does give more equal write ratios, of course, which is good in itself, since the CF and SD cards have the same capacity. Good to know that it's due to the lossless compression overhead.), I suppose working around that might be a big ask, but I confess to major ignorance on the matter.

On this topic, I'm slightly confused about what you mean by there being a 150MB/s bottleneck in PLAY mode. Are you referring to a read bottleneck, or rather a write bottleneck with LiveView off, or perhaps something else?

Great stuff all around, either way!

[mlrocks]

I just benchmarked my sd cards with different capacity in 650D. All sd cards are 170MB/s Sandisk Extreme Pro. There are two 64 GB, one 128 GB, one 256 GB. All were bought within a year. All of them were overclocked to 240 hz in the 650 D July 24, 2021 ML version.
The two 64 GB and one 128 GB cards benchmarked for one minute, and got the write speed of 60 MB/s, and the read speed of 70 MB/s. The 256 GB card got the write speed of 45 MB/s, and the read speed of 45 MB/s. Yet the 256 GB card could continuously record at 55-60 MB/s. When it was over 60 MB/s, then it could only record several seconds. Not sure if it is the card issue, or the benchmark module is not tuned for larger size cards like 256 GB or 512 GB.

[Walter Schulz]

Benchmark doesn't care about card size. It won't even check for sufficient free space.
Your numbers for all your cards seems to be a bit low. Try a run in photo mode. And just for fun: What are the numbers you get benchmarking on your PC? Esp. for the 256 GB variety.

[theBilalFakhouri]

Also try to format the cards in camera (Low level format) before running benchmarks to get constant write/read speed.

[mlrocks]

Benchmark doesn't care about card size. It won't even check for sufficient free space.
Your numbers for all your cards seems to be a bit low. Try a run in photo mode. And just for fun: What are the numbers you get benchmarking on your PC? Esp. for the 256 GB variety.

For the 256 gb card, I did overclocking to 240 hz, then formatted it at low level using the Canon Menu, then benchmarked using the 1 minute test in photo and video mode. In photo mode, I got read about 40 MB/s, write 36 MB/s. In video mode, for several times I consistently got 45 MB/s for read and write. Turning off modules like LUA EDMAC etc did not help.
On computer, using Sandisk card reader USB 3.1 (computer USB adapter may be just USB 2.0 or USB 3.0, but not USB 3.1.), the read speed was 160 MB/s consistently, the write speed was 98 MB/s consistently.
I can record continuously as long as the data flow was about 55 MB/s. Over 60 MB/s, it will record several seconds, then stop.

[theBilalFakhouri]

Have you restarted the camera after selecting 240 Mhz preset?

[mlrocks]

Have you restarted the camera after selecting 240 Mhz preset?

Yes, I first chose the 240 Mhz preset, then restarted the camera. Then I reformatted the card in camera. The ML still worked after the low level in camera format.

[Walter Schulz]

These are different things:
- ML will save its own data during format and write back. Has nothing to do with overclocking.

The numbers you get by benchmarking in cam are pretty low. You should get around the same numbers (or better) without overclocking.

[mlrocks]

I just found out the root cause of this issue. I was in 650D crop new 2.35 1x3 1600x1800 mode, testing all these benchmark. Now I am in crop UHD 1x3 1280x2160 mode, benchmark showed 66 MB/s writing speed. So the benchmark is actually preset dependent.

[Danne]

Turn off raw video and do the test in play mode.

[mlrocks]

Turn off raw video and do the test in play mode.

Turn off raw video and in photo mode (play mode), mode: Play-Unk LCD, global draw: on, benchmark 1 minute test showed:
93 MB/s write
105 MB/s read

Turn off raw video and in photo mode (play mode), mode: Play-Unk LCD, global draw: off, benchmark 1 minute test showed:
94 MB/s write
106 MB/s read

[mlrocks]

I just did a stress test checking how the overheating of the SD card overclocking at 240Mhz behaves on 650D.

In the old but reliable mode of movie crop, 10-bit color depth, 14-bit lossless compression, 1800x1012, AR 16:9, fps 29.97, dual ISO and auto ETTR on, global draw on, the 650D recorded with a green dot most of the time, the left time was orange, only at the beginning several seconds it was red.

At the ambient temperature of 20 C, the camera temperature went up to 50 C in the first 30 minutes, then maintained at 50 C for the rest of the time. With a 256 gb card, the recording time was continuous for 69 minutes totally. I changed the batteries and added another 64 gb card, to see how far the little 650D can go before shutting down due to overheating. After recording for about 5 minutes, the recording stopped automatically. I started it again, the temperature was still at 48 C. Another stop happened after 3 minutes' recording. Another stop happened after 2 minutes' recording. I started it again, then the camera recorded until this 64 gb card was full. The camera temperature was at 48 C. I changed to another 64 gb card, and hit the record button again. It was a 20-minutes continuous recording until the card was full. The camera temperature was still at 48 C.  In total, after going through three sd cards with a total capacity of 384 gb and about 100 minutes recording time in a continuous raw recording mode, the 650D temperature was maintained at 50 C. With live view on all the time, the camera took 2 hours to go back from 50 C to 42 C. It may take several hours more to go back to below 30 C.

It seems that the camera has an embedded cooling system within kicks in when the camera temperature reaches 50 C. Heat may store somewhere in the camera body relative safely and can be released gradually after the camera is turned off. This kind of temperature control is good enough for an 8-hour full day shooting when sd card overclocking is turned on.

[Walter Schulz]

"Embedded cooling system" is a bit misleading. It's just a temperature threshold and cam turns off if overstepped. No timer involved as in EOS R series, for example.
Heat cannot be "stored" inside a camera. Heat goes from "hot" to "cold" by radiation, conduction and convection until equilibrium is established.

[mlrocks]

"Embedded cooling system" is a bit misleading. It's just a temperature threshold and cam turns off if overstepped. No timer involved as in EOS R series, for example.
Heat cannot be "stored" inside a camera. Heat goes from "hot" to "cold" by radiation, conduction and convection until equilibrium is established.

The camera maintained at 50 C for about 1 hour recording raw without shutting down. If there is no way to transfer the heat from the cpu, like fans in a desktop computer, the camera should have shut down several minutes after it reached 50 C. Or because the ambient temperature was 20 C? Probably someone can do a similar stress test under the noon sun to check how long the camera can record when the ambient temperature is high, like 35-40 C.

[ainu]

@theBilalFakhouri

Can you please give me a link to the sh_uhs module with the ability to enter values in the menu to accelerate the recording speed?

I have a Canon 70D camera. For the Kingston 64GB U3 card (label 90mb/s read, 80 mb/s write), only the 160 MHz preset works in your automatic module, and the 192 and 240 presets do not work.
However, when using the version of the module from Levas (https://www.magiclantern.fm/forum/index.php?topic=12862.msg228576#msg228576), the benchmark shows a recording speed of 80-85 mb/s and I can record raw video with a stream of about 75 mb/s in video mode. Unfortunately, this module does not work stably - the ability to write with a stream of 70-75 mb/s can turn into 50-30-20 mb/s.
Therefore, before buying sd card sandisk extreme pro 256gb 170mb/s (ArcziPL confirmed the operability of the preset 240mgz on 70d - https://www.magiclantern.fm/forum/index.php?topic=25841.msg234506#msg234506), I decided to try to select parameters for my existing card.

Thank you in advance.

@all
Or may be anyone else have mosldule and can give me link to it please?

[theBilalFakhouri]

Hi @ainu

Can you please give me a link to the sh_uhs module with the ability to enter values in the menu to accelerate the recording speed?

Do you mean this? :
https://www.magiclantern.fm/forum/index.php?topic=25841.msg234228#msg234228

Or this:
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228367#msg228367 ?

(I don't recommend to play with it)

Levas's preset in this link is the same as in this sd_uhs version, it's 240 MHz preset.

Not much people tested "Kingston 64GB U3" cards with SD overclocking . . some cards won't accept the overclock for some reason, but I can see the 240 MHz preset worked using your Kingston U3 card, but it's seems not stable.

Have you tried "Low level format" in camera? this might solve the drop speed issue, if it sill switch to 20 MB/s then currently 192/240 MHz presets aren't practical/stable for 70D, even with Sandisk Extreme PRO U3 UHS-I 170MB/s cards.

I can see in Levas's sd_uhs.c he isn't using "SDR-104" hack, I will update my version with ability to turn it off for testing purposes, it might be the reason why my sd_uhs version didn't work on your Kingston card at 192/240 MHz.

[ainu]

Have you tried "Low level format" in camera? this might solve the drop speed issue.

@theBilalFakhouri
About card format: yes, I tried much more.
I format (do it at PC) card to FAT32 with cluster size start of 32KB to 128KB, exFat with cluster size start of 4KB to 16MB and on each combination of file system and cluster size (include low level format in camera with sd_uhs on at 160Mhz and off) run benchmark on stable preset 160Mhz ("old" sd_uhs module with SDR104 hack enabled "on").
So, at all combinations:
- in Play-UNC LCD mode write speed about 63-67mb/s, read speed mostly is 71.4mb/s
- in MV-1080 mode write speed about 56-59mb/s, read speed ~59mb/s.
To sum it all: file system and cluster size does not matter.

About test "new" sd_uhs with SDR104 on/off switch: on 160 MHz no difference between "on" and "off", 192 Mhz both modes (on and off) still not work for my SD card (write/read speed 20mb/s), 240Mhz in both modes write speed drop (first write speed ~85mb/s, then read/write/read speeds is ~20Mb/s.
At 240Mhz in video mode some lucky times I shoot raw video at ~60-65mb/s continues. In times when I can shoot raw video continues if I switch to photo mode and again to video mode speed drops to 20-8-2mb/s.
To sum it all: at this moment with current presets 240Mhz for Kingston sd card is working "particular". May be some sd cards need other presets.

Therefore I want to try this module https://www.magiclantern.fm/forum/index.php?topic=12862.msg228367#msg228367

In near future I plan to buy Sandisk Extreme Pro 256GB 170mb/s read speed. When it happens - I will test it with SDR104 hack off mode for understanding does it matter or not.

[theBilalFakhouri]

@ainu

It seems Kingston cards doesn't provide stable write/read speeds, in this area Sandisk Extreme PRO UHS-I U3 170MB/s is better, it always have constant write/read speeds without dropping in 160 MHz preset (also for 192/240 MHz, tested on 650D/700D/6D) after low level formatting the card in camera.

To sum it all: file system and cluster size does not matter.

I remember testing these, yes it's true it didn't have any effect on write speed or stability.

Okay using 192/240 MHz on 70D isn't practical currently, 5D3 share the same story, 160 MHz is fine on 5D3 but 192/240 MHz you can record some clips then instead of switching to 21 MB/s mode it disable the access to card completely.

EOS M also suffering from switching to 21 MB/s (48 MHz) when using 240 MHz overclock, but 192 MHz preset on EOS M works without any problem.

On 700D/650D/6D 192/240 MHz presets works fine too without any problem (Sandisk Extreme PRO UHS-I U3 170MB/s).

I spent much time yesterday to investigate to see what's difference among 5D3-700D, nothing important yet.

Therefore I want to try this module https://www.magiclantern.fm/forum/index.php?topic=12862.msg228367#msg228367

Link was sent via PM, and good luck

[ainu]

Thank you theBilalFakhouri.

I made a small selection of values (table here).
Luck with the current sd card NOT smiled.
However, there was some understanding of how to set values. But the found preset from Levas (which works in Play mode always, and in Video mode times from time to time) does not fit into this understanding.
It turns out that in Play mode, the card and camera can record 80mb\s and read 105mb\s stably, and in video mode, the recording speed is kept only until the first reading test.
Perhaps the problem is not the preset - the future will show.

Edit.
Another note: after entering settings in the menu, during their application, the diode near the cover sd card lights up twice. By the duration of the second flash (if it is longer than ~ 2 seconds) you can understand the settings applied or the camera returned to the "48 MHz - 20.4mb\s" mode. Thus, you can not start benchmark (with a long flash, it always shows 20.4mb\s) and reduce the search time.

[theBilalFakhouri]

@ainu

Perhaps the problem is not the preset - the future will show.

240 MHz works fine on 650D/700D/6D, I have been using it for a year, never switched to 48 MHz mode using Sandisk Extreme PRO UHS-I U3 170 MB/s cards, never had data corruption with it.

-My my expectations are:

1. Canon has a function which traces card controllers speed clocks/or something makes Canon think something isn't usual is happening regarding how card is being configured which doesn't match what Canon code is expecting --> Switches to safe mode (48 MHz).

We already know there is something like that, but this was a different case:
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228485#msg228485
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228510#msg228510

I don't think it's related to 70D issue, what was happening there is my *Sandisk Extreme PRO UHS-I U3 95 MB/s was giving up (because the card had 99 MB/s write speed with 240 MHz overclock, probably it couldn't handle this really high write speed) or **SD card reader in camera was overheating/signal isn't clean. (My Sandisk Extreme PRO UHS-I U3 170 MB/s cards have fixed write speed of ~90 MB/s, which is probably made the signal cleaner/normal).

*Also my Sandisk Extreme PRO UHS-I U3 95 MB/s have 45 MB/s read/write speed using the card reader on PC, while the 170 MB/s version was giving ~85 MB/s write speed with same card reader, newer cards from Sandisk is more fine tuned I guess.
**I put the camera in fridge with 95 MB/s Sandisk inside the camera (240 MHz) as @names_are_hard said it might be overheating issue, and made some tests, the card never switched to 48 MHz mode but it had speed drops, and a recording was stopping suddenly.

2. A setting needs to be changed on EOS M/70D/5D3 to accept these high clocks speeds, little experiment on 700D:

-I called sdSoftReset (which switches the card to 48 MHz mode), write speed becomes 21 MB/s (from ~40 MB/s)
-I changed registers values to 96 MHz preset (96 MHz preset is the default preset on 700D and all other DIGIC 5 cameras except for 5D3), write speed became 36 MB/s which a little lower than default 96 MHz mode
-It turned out AccessMode was changed from 7 (default) to 3 (after sdSoftReset), I re-patched AccessMode to 7 and got ~40 MB/s write speed which is the correct one

-So now we are in default 96 MHz mode after a sdSoftReset, let's overclock the values again to 160MHz/192MHz/240 MHz --> Card full; card can't be accessed (another safe mode but safer from Canon)
-That's why I believe there is a setting we are missing, finding it out might solve the problem for 5D3 and probably for EOS M/70D

3. Third expectation: it might just the wrong 240 MHz values

[Walter Schulz]

*Also my Sandisk Extreme PRO UHS-I U3 95 MB/s have 45 MB/s read/write speed using the card reader on PC, while the 170 MB/s version was giving ~85 MB/s write speed with same card reader, newer cards from Sandisk is more fine tuned I guess.

Unable to reproduce because I don't own this cardreader. Own tests with Kingston FCR-HS3 and FCR-HS4 and all cardreader tested by CameraMemorySpeed.com showed write rates > 80 MByte/s.
I think you have either a counterfeit card or a cardwriter issue.

[theBilalFakhouri]

Unable to reproduce because I don't own this cardreader. Own tests with Kingston FCR-HS3 and FCR-HS4 and all cardreader tested by CameraMemorySpeed.com showed write rates > 80 MByte/s.
I think you have either a counterfeit card or a cardwriter issue.

I have tested my Sandisk Extreme PRO UHS-I U3 95 MB/s card on PC using three different card readers (RDF5 -RDF8 - Laptop card reader) on Windows, it gave same speed around ~45 MB/s, however on camera:
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228555#msg228555

That's was the (Sandisk Extreme PRO UHS-I U3 95 MB/s)

Pretty sure it's not fake, fake cards write speeds maxes out at 20-25 MB/s, my card works greatly with 192 MHz preset.

Also my friend got same SD card version from same local store nearly the same time, his card appear to maxes out @ 55 MB/s on PC, on camera with the overclock it gives the full speed.
I think only some of Sandisk Extreme PRO UHS-I U3 95 MB/s cards have this issue, probably I will get the maximum write/read speed on PC using a Sandisk card reader or on another OS.