UHS-I / SD cards investigation

[Jip-Hop]

SanDisk 64gb 170 MB/s Low Level Format @ 160mhz


SanDisk 64gb 170 MB/s Low Level Format @ 192mhz


SanDisk 64gb 170 MB/s Low Level Format @ 240mhz


SanDisk 64gb 170 MB/s Low Level Format @ Overclock Off


Interestingly the fastest speed I've seen was with 240mhz. Other than that, both of my cards (SanDisk 64gb 170 MB/s and SanDisk 256gb 95 MB/s) are responding the same to the overclock settings.

[70MM13]

is everyone sitting down?

i have some shocking news!

i just tested my $5 (CDN) kingston 128GB card (microSD with supplied adapter) and it WORKS at 95MB/S

simply installed danne's build from today, enabled overclock with the 95MB/S option, rebooted the camera, and started recording!

no additional steps.  no formatting, no nothing!

i recorded 2880*1226 @ 24FPS, 14bit lossless using the uhd preset.  NO CF card installed, recorded only to the $5 128GB card, and the overexposed scene had a green indicator!

i think i am dreaming...

[yourboylloyd]

i just tested my $5 (CDN) kingston 128GB card (microSD with supplied adapter) and it WORKS at 95MB/S

simply installed danne's build from today, enabled overclock with the 95MB/S option, rebooted the camera, and started recording!

i recorded 2880*1226 @ 24FPS, 14bit lossless using the uhd preset.  NO CF card installed, recorded only to the $5 128GB card, and the overexposed scene had a green indicator!

i think i am dreaming...

This is crazy! MICROSD?!?

What resolution/framerate can you get shooting in 10bit?

[theBilalFakhouri]

bilal, i'm using a sandisk extreme pro 170mb/s on my 5d3.
..

3616*1536 @ 24fps 14 bits lossless, extremely overexposed scene: continuous recording with a green icon!

...

My mind was tricked sorry, I thought you are on 70D

Great result!

i just tested my $5 (CDN) kingston 128GB card (microSD with supplied adapter) and it WORKS at 95MB/S

Picture of the card? Maybe a online link to it, to see the card specs, Could you run benchmark on this card in play mode ?

Your claiming shocks 

[70MM13]

i'm sorry everyone, but it all just came crashing down...

i was doing a bunch of recordings at various resolutions @ 10 bit, 1 minute each.

on approximately the 10th recording, i got the dreaded "card full" error (NOT FULL).

so i did a low level format in camera, reinstalled magic lantern, and did a benchmark in photo mode.

it was "down" to 86 MB/S, with several tests.

then i switched to video mode,  and first i ran the benchmark again, and this time it was about 70 MB/S with global draw and raw recording OFF.

as soon as i hit record, i got "card full".



i will keep experimenting, but it looks like maybe the card can't handle it.

isn't it strange that the benchmark still works though?

[Danne]

ANd if you run lower like 192Mhz? I think one gets card full when sd controller spins off to oblivion. I had it when benchmarking around 5000Mb/s by manipulating sd_uhs code. Maybe this can happen with code somehow otherwise too. It seems related to this part:

Code: [Select]

/* called right before the case switch in sd_setup_mode (not at the start of the function!) */
static void sd_setup_mode_in_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    qprintf("sd_setup_mode switch(mode=%x) en=%d\n", regs[sd_setup_mode_reg], sd_setup_mode_enable);
   
    if (sd_setup_mode_enable && regs[sd_setup_mode_reg] == 4)   /* SDR50? */
    {
        /* set our register overrides */
        for (int i = 0; i < COUNT(uhs_regs); i++)
        {
            MEM(uhs_regs[i]) = uhs_vals[i];
        }
       
        /* set some invalid mode to bypass the case switch
         * and keep our register values only */
        regs[sd_setup_mode_reg] = 0x13;
    }
}If I do something like:

Code: [Select]

if (sd_setup_mode_enable && regs[sd_setup_mode_reg])   /* SDR50? */This gets called all the time and caused the card to spin off. Needless to say. Don´t try this at home.

[70MM13]

this was at 160 mhz. (the 95 mb/s option)

i was thinking about trying 192 mhz after lunch

i was also wondering if maybe this will somehow work with card spanning.  just a crazy thought that is worth testing

[Danne]

Uploaded new builds for 5D3. The 160Mhz preset will now use the older patch, more stable.
Tested 240Mb/s, it´s rock solid.

[theBilalFakhouri]

Uploaded new builds for 5D3. The 160Mhz preset will now use the older patch, more stable.
Tested 240Mb/s, it´s rock solid.

Good, you mean 240MHz ?
Could we have benchmark in PLAY Mode with the used Card @ 240 MHz, Also how this preset perform in 5D3 ?

[70MM13]

it no longer works for me with my sandisk 170 card

i don't have the first version of today's build, it got overwritten due to identical filename...

could you reupload the earlier one so i can keep using it?  i might not be the only one who needs it

[Danne]

Good, you mean 240MHz ?
Could we have benchmark in PLAY Mode with the used Card @ 240 MHz, Also how this preset perform in 5D3 ?

Hehe, yes 

Thing is. In photo mode when benchmarking It´s better to turn off RAW video. So I just uploaded new 5D3 builds again which do that now. Turns off RAW video when patching in photo mode. This is probably causing instabilities for movie mode too but some patches seems more stable than others. Or at least seems like it.

70mm13 - Builds are not different. Could you test again with this upload? Just put up two new versions. Minor change.

Some patches(170Mb/s sandisk extreme pro card):

Code: [Select]

0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x101,      0x101,        0x1

Code: [Select]

0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3

Code: [Select]

0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4

[70MM13]

second attempt from scratch, this time at 192 overclock before low level format, still fails...

i'm getting concerned the card/camera might be damaged?

can you send me the earlier version from today just to try?

[Bender@arsch]

I tested your new upload and found a issue, but while writing your upload a new version again^^.
So i tested again, but the issue is the same:

Can't play movie with sd patch at 192mhz and 240mhz --> "File ends prematurely during block header"
With 160mhz all is working.

[Danne]

Hm, yes. I tested now too and just running sd card alone will not work with 192Mhz and 240Mhz. I was under the impression this was tested working before? Maybe not.
I will remove the builds for now. They are benchmarking ok but recording is not legit. 160Mhz uses old patch proven working.

EDIT: Uploaded back builds with only 160Mhz included. Back to drawing board.

[Danne]

Ok, here we go again(5D3). THink I found the issue. I added some fix including a function interfering with patching. Back again with builds all included.
Manual lenses is a hassle as it stall screen for a while. This can lead to file creation errors but mostly it´s ok. I usually open up liveview manually when starting cam instead of waiting for the process to finish.
Please report back if this works again.
Builds here:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/

[Bender@arsch]

You are to fast for me^^

first update: same issue
second update: same issue

but switching back to 160mhz, i can play all files!

and

If i load, after restart, first time the sd patch in photo mod, switch to video mod and pres rec. it stopped automatical.  I need a restart again for working record.
(UHD mode, 4K with global draw, and card spanning at 10bit)

[70MM13]

benchmarks are exactly as they were on today's first version, but i cannot record on the sd card.  file create error or card full error every time.

reformatting the sd card using my pc to start absolutely fresh...  another report soon.

trying to stay optimistic!

[Danne]

Crap. I was wrong. Back to stable 160Mhz only for 5D3. Uploaded last builds for today. This needs special attention. 5D3 is much more sensitive than eosm. I recommend ditch any older builds and redownload the 160Mhz version only for now.
AND. I need a break...

No use asking for earlier builds. Nothing vital changed and I tested already older stuff. If it was somehow working before it´s pure luck or I must be missing something essential.

[70MM13]

ok, i hear you loud and clear...

meanwhile, here's the results of my testing on that triple play build:

only 160 mhz was working at all, and it was rock solid with SD card only.  as soon as i restarted with both cards in and tried spanning, "file create error" and hard lock requiring battery pull.

i will now grab your latest version and take it for a spin...  fingers crossed!

[names_are_hard]

So...  my guess here is that the drop to a slow speed is some kind of back off or safety feature.  There are many factors that might trigger it, but if it triggers it latches on until something is reset (the whole camera?  Some register?).

One of the factors that is likely to make a card glitch (or perhaps it's some SD related HW) and get put into slow mode is heat.  That's going to be variable, and affected by what you're doing.  It's fairly plausible that the highest speed you can reach will never be stable for long periods, as heat builds up (or, some other random factor happens to occur, an unlikely cache spill or whatever).  I see three obvious routes for making this a usable feature (and it's too cool not to try to make it usable!):
 - detect the drop to slow speeds and work out how to reset the safety feature (good if you're okay with needing many takes or only short takes)
 - improve the reliability of benchmarking the card to determine the stable overclock (and ideally recording this onto the individual card so you only need to benchmark each card once)
 - using the existing benchmarking, select a max speed below the fastest achievable and hope you left enough margin

Adding active cooling to the card might also work but doesn't seem very practical.  Might be interesting to see if overclocking is more reliable inside a fridge!  We might learn if heat is a real factor here.  Drill holes into the battery / card compartment in advance for best testing

[Danne]

How do you explain eosm, and the rest of the cameras working without fallback?
On 5D3 it's not holding up because of when raw is enabled for one. It is semiworking as write benching works but read mode flips. I'd say it's some registry tweak needed. Or even patching routine. Code is somewhat different from the rest. But hey, a1ex probably knows more here. Malloc issues.
Heat etc could be problematic once we get continuous recording working but for know it's more like breakdown .

[70MM13]

ok, it's working again!

obviously the speed is not the same as it was a few hours ago

but it's working and my camera isn't screwed up!

with spanning, the highest resolution i get a green indicator on is now 3248*1382 @24fps and 14 bits.

fingers crossed for an eventual return to those incredible heights!!

i don't think heat is a factor.  the failures were at camera startup and everything was cool.  i'm testing in a cold room also...

[names_are_hard]

Danne: I haven't been trying to find a pattern between models.  Could it be that more modern models are more likely to be stable?  Or, physically larger cameras?  Heat is just a theory, and it's hard to say one way or the other, since the individual card certainly plays a role too.

It's very *plausible* that heat matters, as that's a common reason why components are driven under spec, heat will make electronics unreliable, plus controlling heat to within regulations for consumer devices is quite challenging.  As I say, it's my guess, I don't have a strong opinion or even much evidence.

[Danne]

I am speculating it might work better when both cf and sd working together. Pressure becomes less for the sd. Did you do a lot of testing with solely sd card?
Did you playback the card spanning files after testing before? If card was wrongly patched files should be problematic to open.

Edit: Needs more testing and tinkering @names_are_hard. Would be nice to see it through .
But heat, hm. How come bench works? And works better with raw video off. Patch, sd_uhs code, my shit, all needs to come together.
Also "card full" must be sd loosing complete controll hypotethically filling the sd card in an instant. Also reveals complete breakdown when bench read numbers skyrockets.

[70MM13]

i did tests with sd alone as well as both cards together.

one thing that stands out immediately in spanning mode is the return to "lopsided" data writing.  when it was working at 240mhz, it was very evenly divided.  now it was about 70 for the CF and about 50 for the SD.  also, lower overall data rate.  it was 131, and now it is more like 120.

strange, considering the CF has no problems doing 90... but i guess it is the headroom issue.  it was definitely better at 240 in all cases!