UHS-I / SD cards investigation

[Walter Schulz]

Exactly what version ?

https://drive.google.com/file/d/1wRLfWoMDCxB2SAZXRVoKM-9plDBAvxIq/view?usp=sharing

CRC32: 132155EA
CRC64: A01BAE741060715B
SHA256: 5870A0D507329669EDAAA373007704E9CE2CA161C8B99C2F2F9484989FA01962
SHA1: 77C19D9788B24E0808A5F3DE86A22D27F5D77469
BLAKE2sp: C97394E35805B91796F0DFA84643A11AB0077DAD7208CA03F0A3C37846ADC290

[Danne]

This right?
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228576#msg228576

Just want to keep track of the patch sequence, assume this:

Code: [Select]

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3 }

[Walter Schulz]

Unable to do any compiling. I'm packing for a road trip by motorbike.

[Levas]

There goes my digic5 vs digic5+ theory 

So apparently the 650d also can make use of the same sd_uhs settings as 6d and 5d3.
But 700d and EOSM don’t benefit from the same sd_uhs settings 

[zandette974]

Hi,
Tested on 700d with bleeding edge crop rec 4k isogain in 5k anamorphic mode at 1488x1900.
When i run the script it ask me to "enable sd_uhs.mo, but the  module is already enabled
I didn't have this message with normal sd hack. I tried with both versions Levas posted today.
My card is a Sandisk extrême pro 95 mb/s. When i hit record i have 58mb/s and continuous recording but i think it could be better

Laurent.

[70MM13]

it just means there is an even greater discovery waiting around the corner! 

[theBilalFakhouri]

But 700d and EOSM don’t benefit from the same sd_uhs settings 

700D does benefit from the same sd_uhs settings, I managed to record in all mods 1x3, x5, mv720, mv1080 etc . . more than 20 Clips, I filled 32 GB card many times, but after recording some clips, or recording @ Maximum overclock speed e.g. 68 MB/s in video mode then it's mostly will switch to 48 MHz

Waiting 70D, 100D testers (@IDA_ML , @OlRivrRat  where are you? )

I am interested in this preset more:

Code: [Select]

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x3,        0x3,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 }
On 700D max write speed @ ~ 71 MB/s and it's very constant, I filled up 32 GB SD card recording at 69.1 MB/s all the time , the recording didn't stop until the card is full, pretty good result

[Danne]

On my eosm it will not even patch. It starts really slow. BUT, I can´t complain. The other overclock setting and the working sd patch for 5D3 must be something to keep most of us smiling the rest of the summer .

Bitbucket still didn´t delete my shit. Maybe they´re watching  .

[Levas]

If you think about it, it is ridiculous what these old cameras can do.
Imagine if Canon gave these write speeds and raw video recording at the introduction date...most SD  cards at the time couldn’t probably even handle the speed though 
But they could have introduced a paid firmware upgrade 3 years after introduction date and it would still be interesting for many users.

[70MM13]

but it is also canon's worst nightmare...

i am absolutely not interested in buying a new camera!

why would i?  this is wonderful for us, but not so good for manufacturers

but it's good for the planet

[Levas]

Now the whole 8K raw thing in the upcoming Eos R5 makes sense.

Dialogue:
Canon marketing: ok time for our next camera. 4k video must be enough...H.264 will do.

Canon technicians: eh, Magic Lantern has found out how to use crop mode. They now almost have 4k resolution in raw mode.

Canon Marketing: No problem, 5d3 can’t handle that much data.

Canon technicians: Well...technically the 5d3 can handle that much data...it’s only a matter of time until they find out how...
plus there will be the risk they find out how to overclock sensor readout speed and unlock 5k raw on the 8 year old 5d3.

Canon Marketing: Ok we go for 8k raw in our next cam ( 5d3 can’t do that in any possible way...right?)
Ok 8K raw it will be.
And make sure to only use  inferior parts this time .

[70MM13]

that last line is so true and so heartbreaking...

oh well, at least there are a few more newer models for us to unlock that might have incredible potential before we hit the R5 brick wall of inferior components!

cheers!

[yourboylloyd]

If you think about it, it is ridiculous what these old cameras can do.
Imagine if Canon gave these write speeds and raw video recording at the introduction date

I disagree with this statement. I don't think that Canon knew what these cameras were capable of. I think you guys are just better engineers than Canon's team.

[70MM13]

i did some comparison recordings using the "old" and new sd overclock versions, and i am left with a question, but first the results:

with the new one, it was fairly even between the two cards, in the mid 60 MB/s range, and very stable during recording.

with the old one, it was not even between the cards.  the CF was recording at a higher rate, sometimes even hitting 90 MB/s, while the SD was limited to about 51 MB/s, sometimes as low as 40 (when the cf was at 90)

but...  in both cases, it was almost as if it is "pinned" at about 131 MB/s between the two cards.

even though the total is basically the same, the new one seems to record longer, perhaps because it is so evenly balanced thanks to the faster SD clocking?

and this makes me wonder...  is it possible to make the card spanning use "brute force" during recording to alternate between the slots on a per-frame basis?  my reasoning is thus:  i'm wondering if the seeming hard limit of about 131  MB/S is because of cpu overhead, and would it potentially be reduced within the card spanning routine if it was "dumb" and was hardcoded to alternate frames (or some arbitrary chunks) between the slots?

just a question for those in the know!

[theBilalFakhouri]

After some more searching in the LOG 0 I found this, maybe it's where the problem starts:

Code: [Select]

1.300.435      Fwrite:ff74c4dc:23:26: sdDMAWriteBlk: Transfer Block Size Invalid(4809<->6167)
 1.300.462      Fwrite:ff74c540:23:06: sdDMAWriteBlk(SDSTS_ED)(0x2:0x8001:0x40:0x2)

Do you remember this error that causes the switch to 48 MHz, Canon has two registers for Transfer Block Size:

1- 0xC0C2007C
2- 0xC0C20080

There is a pattern, it starts with the first register (0xC0C2007C), it will set a value then the second register (0xC0C20080) will show up and will set a value which should be identical to the first register value, Now Canon compares the two values:

If it were identical --> Continue
If it weren't identical --> It breaks
 

From same LOG 0:

Register 0xC0C2007C set a value which is 0x1817 = 6167

Code: [Select]

1.639.620      Fwrite:ff74c410:MMIO : [0xC0C2007C] <- 0x00001817
Register 0xC0C20080 set a value which is 0x12C9 = 4809

Code: [Select]

1.300.233  **INT172h*:ff74c11c:MMIO : [0xC0C20080] -> 0x000012C9
Oh shit it's not identical, there is a problem:

Code: [Select]

1.300.435      Fwrite:ff74c4dc:23:26: sdDMAWriteBlk: Transfer Block Size Invalid(4809<->6167)
 1.300.462      Fwrite:ff74c540:23:06: sdDMAWriteBlk(SDSTS_ED)(0x2:0x8001:0x40:0x2)     ----------------------- what is this ?

Try the first register value?

Code: [Select]

1.300.732      Fwrite:ff74c7b4:23:26: sdWriteBlk: Retry:0 102694 6167
Then sdSoftReset( 0 )

In the normal situations the two registers is always identical, you may see LOG 1.

Now these registers represent Transfer Block Size at the current write speed ? I don't know what really it is (if someone can explain . . @a1ex we missed you), it doesn't have constant values, so how register one get it value, same for register two? and what makes register two has a different value, in short what is causing this ?

If it's okay to have two different values and if it not dangerous to have them we may patch the comparing process (ignoring it if this possible somehow), Or can we override them, if it's only reporting a values without effects on reading/writing process. We should find some informations about it before trying doing anything.

[Levas]

Could it be that the switch to lower speeds is sort of initiated by the SD card.
If the card can't handle the speed (maybe also for a 5 second speed drop), the SD controller switches to a safe 20Mb/s.
There are plenty of cards out there with different speeds, and the SD unit has no idea what card it has to write on.

Maybe the two registers are some sort of time stamp or time limit in which the card has to respond back or something ?

[Levas]

in both cases, it was almost as if it is "pinned" at about 131 MB/s between the two cards.

Maybe 130Mb/s is the speed limit of the buffer memory ?

[70MM13]

i just ran the benchmark you provided and after running patch2 the sd card results are 95 MB/S write and 5 MB/S read

yes, you read that correctly...

edit:  i didn't read it correctly!!  it actually says 5752 MB/S for the read speed, but my mind refused to accept it and edited it to say bytes per second instead.

obviously there's something wrong with this...  i will run the benchmark again in video mode.

benchmark in video mode, write speed is 60.5, and there is a malloc error for the read tests (malloc error: buffer=16777216

i ran it twice.  same both times.

global draw OFF increases the write speed to 78 MB/S.  malloc error on read persists.

[Levas]

oeps, almost sounds like the first broken sd card with these new settings 

Have you tried reformatting it in camera (and keep ML) ?
How does the card behave on a computer, normal read speed or not ?
Have you run benchmark without the sd_uhs module, how are read/write speeds without the module activated ?

EDIT: Could this have something to do with both use of hack and card spanning ?

EDIT2: The benchmark is from "crop_rec_4k_mlv_snd_isogain_1x3_presets" branch, guess it should work normal on 5d3, but not sure 

[theBilalFakhouri]

it actually says 5752 MB/S for the read speed, but my mind refused to accept it and edited it to say bytes per second instead.

Sometimes the problem could be from Bench.mo, once it happened to me when I removed maybe the write speed test from the code, but this also caused a died card in the past.

and there is a malloc error for the read tests (malloc error: buffer=16777216

Turn off RAW video

[70MM13]

i tried your suggestion of formatting the card in camera.
it then said "restoring magic lantern" and rebooted the camera.

but it never woke up again.

after a battery pull, it still won't start.

any suggestions?

it boots with other cards.

[theBilalFakhouri]

@70MM13

Take SD Card and CF card out of the camera, Pull the battery out, put the battery again and start the camera without any cards. feedback

/Edit: okay it boots./

Does the SD Card work fine in PC card reader, write/read speed ?

[Levas]

it boots with other cards.

Ok, now it really sounds like something is wrong with the card.
Can you format the card on a computer ?

[70MM13]

ok, i got it sorted out!

since the camera is working with other cards, i reformatted this card on the pc, with a different blocksize to make sure things have changed

then the camera booted with this card.  reinstalled magic lantern and everything is fine again

i just ran the bench again:

photo mode, global draw off (MB/S results) 96, 451, 96, 5785
video mode, raw video off (thanks bilal): 77, 3710, 77, 3778

no more error, and the card is fine!

[Levas]

Cool, you got it working again.

Also found this post about checking if SD/CF card is beyond repair:
https://www.magiclantern.fm/forum/index.php?topic=11428.msg167530#msg167530

photo mode, global draw off (MB/S results) 96, 451, 96, 5785
video mode, raw video off (thanks bilal): 77, 3710, 77, 3778

You got some strange reading speeds there, are you sure this is ok, is that a 5d3 in combination with the benchmark module thing ?
I would disable the benchmark module and not using it ever again with those strange read speeds 
(And reformat the card in computer again and fix it, just to be sure)