Author Topic: UHS-I / SD cards investigation (Read 231233 times)

Levas · « **Reply #375 on:** June 29, 2020, 07:58:33 PM »

Oh boy

Now to make this usable, there must be found a way or trick to make this work every single time.

theBilalFakhouri · « **Reply #376 on:** June 29, 2020, 08:00:27 PM »

Quote from: Levas on June 29, 2020, 07:35:13 PM

Far from usable as it is uncertain when speed drop comes.

Yes there is a lot of speed drops when switching to Video Mode

In PLAY Mode the result is constant

theBilalFakhouri · « **Reply #377 on:** June 29, 2020, 08:03:14 PM »

Maybe a1ex had something for this

Quote from: a1ex on April 03, 2018, 12:55:26 PM

Caveat: SDR104 requires tuning the sampling point (not implemented, not performed by Canon firmware, but doable). That might be required to avoid random errors, speed drops, or higher frequency - if the controller supports it. From 0xC0400610/20, the frequencies are 80, 96, 120, 160 and 240 - the latter is probably too high.

theBilalFakhouri · « **Reply #378 on:** June 29, 2020, 08:15:41 PM »

mv1080 benchmark on 700D:

Effective Write Speed ~70.5MB/s

Danne · « **Reply #379 on:** June 29, 2020, 08:30:10 PM »

On eosm this works. Changing:

Code: [Select]

static uint32_t sdr_160MHz[]   = {        0x2,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };   /* overclocked

To:

Code: [Select]

static uint32_t sdr_160MHz[]   = {        0x3,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };   /* overclocked

The ones shown for 700D and 6D will not work on eosm.

I get 66mb/s in photo mode. The rest 19mb/s
Eh, second benchmarking gives 76Mb/s(I had fps override set to 11fps)

Levas · « **Reply #380 on:** June 29, 2020, 08:45:08 PM »

Battery is full again.
Can’t reproduce again. Write speed stays at 20Mb/s.
How do you start cam, in photo or video mode?

Walter Schulz · « **Reply #381 on:** June 29, 2020, 08:51:32 PM »

Quote from: Danne on June 29, 2020, 08:30:10 PM

Eh, second benchmarking gives 76Mb/s(I had fps override set to 11fps)

It's a gain of about 7.6 MByte/s compared to previous experiments (which gave persistent results):
https://www.magiclantern.fm/forum/index.php?topic=12862.msg199250#msg199250
Way to go, keep it up!

theBilalFakhouri · « **Reply #382 on:** June 29, 2020, 09:03:06 PM »

Quote from: Levas on June 29, 2020, 08:45:08 PM

Battery is full again.
Can’t reproduce again. Write speed stays at 20Mb/s.
How do you start cam, in photo or video mode?

I don't use automated sd_uhs, it didn't work well for me, I am using sd_uhs with a submenu and I run it manually, I will PM you my version, change the registers manually from the submenu, you can run it in Photo Mode and run Bench.mo or Enable it in Video mode with RAW Video turned off, sometimes it works sometimes it drops.

Edit: after changing the values, run sd_uhs only for once then press half-shutter to close the console then you are ready, when changing the value of this register 0xC0400614, the first digit from the left will disappear it's okay that's my C skills

Levas · « **Reply #383 on:** June 29, 2020, 09:32:49 PM »

Did a quick try with your build.
Changed the settings in submenu and loaded it. Still doesn't work

slow 20Mb/s.
Have to dive into this, probably changed some other setting that's conflicting with it or something ?

Already tried the old build, SD card is still ok, can get ~59Mb/s with the "old" sd-uhs hack.

If we can get this working, then it's huge, that's about ~20Mb/s more write speed on top of the current SD-UHS hack

theBilalFakhouri · « **Reply #384 on:** June 29, 2020, 09:43:18 PM »

Hmmm
Format you Card in camera and try again in Photo Mode, make sure you don't have anything in DCIM (only ML files on the card) and load only sd_uhs and bench.mo

Levas · « **Reply #385 on:** June 29, 2020, 10:05:36 PM »

Have to go off to work now (nightshift).
Will be continued...

nikfreak · « **Reply #386 on:** June 29, 2020, 10:32:10 PM »

You guys better try this on spare cards cause some will burn

Levas · « **Reply #387 on:** June 30, 2020, 05:56:10 PM »

You mean our camera's are on fire

As in figure of speech, not literally (yet

)

Levas · « **Reply #388 on:** June 30, 2020, 05:59:46 PM »

Doing some more testing, and if the high write speed is activated is a hit or a miss, can't figure out why

Sometimes it works for one or more clips, sometimes it won't work for even one clip.

Still not sure what triggers it to go into 20Mb/s mode

Could it be some Canon routine in the background, maybe some Magic lantern routine in the background ?
Tried disabling all non essential modules, still no luck, it's a hit or miss at current state.

70MM13 · « **Reply #389 on:** June 30, 2020, 06:20:21 PM »

does anything you have discovered potentially apply to the CF interface on cameras with both?

Levas · « **Reply #390 on:** June 30, 2020, 06:59:45 PM »

I just messed with the "new" settings theBilalFakhouri found. Trial and error.

But it is common believe that ~100Mb/s is the max write speed for these Canon camera's, because that's the max of the UHS-I spec.
See this post from theBilalFakhouri
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228370#msg228370

So theBilalFakhouri started messing with these registers, because it was very likely that higher write speeds could be achieved.

Don't know anything about CF card specs.

theBilalFakhouri · « **Reply #391 on:** June 30, 2020, 07:12:59 PM »

By capturing some logs from io_trace_full branch, I can confirm the camera switches to 48 MHz, this a part from the log captured by Canon 700D:

It shows SD UHS default registers in 48 MHz same as 5D3 values, the overclocking was working @ 240MHz before the switch:

Code: [Select]

1.639.422      Fwrite:ff337a90:MMIO : [0xC040062C] <- 0x00000001
 1.639.432      Fwrite:ff3377a0:MMIO : [0xC040046C] <- 0x00000001
 1.639.435      Fwrite:ff33790c:MMIO : [0xC0400600] <- 0x00000003
 1.639.438      Fwrite:ff337914:MMIO : [0xC0400610] <- 0x00000009
 1.639.439      Fwrite:ff33791c:MMIO : [0xC0400614] <- 0x1D000601
 1.639.441      Fwrite:ff337920:MMIO : [0xC0400618] <- 0x00000000
 1.639.445      Fwrite:ff337950:MMIO : [0xC0400624] <- 0x00000504
 1.639.446      Fwrite:ff337954:MMIO : [0xC0400628] <- 0x00000504
 1.639.447      Fwrite:ff337958:MMIO : [0xC040061C] <- 0x00000100
 1.639.448      Fwrite:ff3378e0:MMIO : [0xC0400620] <- 0x00000009
 1.639.450      Fwrite:ff3378e4:MMIO : [0xC0400604] <- 0x00000003

It starts with sdSoftReset( 0 )

Code: [Select]

 1.483.311      Fwrite:ff74aea0:23:01: sdSoftReset( 0 )
 1.483.324      Fwrite:ff337a90:MMIO : [0xC040062C] <- 0x00000001
 1.483.335      Fwrite:ff3377a0:MMIO : [0xC040046C] <- 0x00000001
 1.483.341      Fwrite:ff3377ec:MMIO : [0xC0400600] <- 0x00000008
 1.483.343      Fwrite:ff3377f4:MMIO : [0xC0400610] <- 0x0000017F
 1.483.344      Fwrite:ff3377fc:MMIO : [0xC0400614] <- 0x1D004101
 1.483.347      Fwrite:ff337800:MMIO : [0xC0400618] <- 0x00000000
 1.483.348      Fwrite:ff337808:MMIO : [0xC0400624] <- 0x0000403F
 1.483.350      Fwrite:ff33780c:MMIO : [0xC0400628] <- 0x0000403F
 1.483.352      Fwrite:ff337814:MMIO : [0xC040061C] <- 0x0000007F
 1.483.354      Fwrite:ff337818:MMIO : [0xC0400620] <- 0x0000007F
 1.483.356      Fwrite:ff33781c:MMIO : [0xC0400604] <- 0x00000000

Then it goes with multiple commands from Canon (same as when the camera initialize the SD card during startup, e.g in 5D3 startup log)

Code: [Select]

sdSoftReset SUCCESS
sdTrySendCommand1 Start

sdSoftReset( 0 )

sdSoftReset SUCCESS
sdIdentifyDrive Start
sdSendIFCondition Start

sdSendIFCondition End
sdSendOCR Start

sdSendIFCondition End
sdSendOCR Start

sdSendOCR End
sdAllSendCID Start

sdAllSendCID End
sdSendRelativeAddress Start

sdSendRelativeAddress End
sdSendCSD Start

sdSendCSD End
sdSendCID Start

sdSendCID: MID = 0x03, PDN = 0x534c
sdSendCID End
sdSelectDeselectCard Start

.... etc

But I noticed it doesn't switch the frequency directly to 48 MHz or 96 MHz, it goes from multiple mods from the lowest frequency to the highest desired one (a1ex figured out this and he wrote the settings in sd_uhs.c 5D3 Mode 0 .. etc)

So I tried to start from the first values during SD card initialization to the next one to 24 MHz , 48 MHz , 96 MHz , 160 MHz and 240 MHz using sd_uhs.mo, I wrote it in order to emulate what Canon does . . . aaand . . . same result it switch back to 48 MHz

Sorry it didn't work

I don't know if we should call sdSoftReset or others commands, but also I got something maybe the problem from sd_uhs module itself, Because after Canon switch back to 48 MHz, theoretically I should able to run sd_uhs overclock again and get back 240 MHz overclocked values, but It doesn't patch the settings again only after a startup, and this problem from sd_uhs module (maybe) so if we could e.g. clear ROM patches by switching off sd_uhs like adtg_gui, or simply force the overclocked values and disable Canon patching (I think this type of code is used in crop_rec.c and adtg_gui.c)

I am not sure if this a good way, or even if it will work, maybe I messing

Good news: SDR104 @ 240 MHz is stable when it's working, the write speeds doesn't drops when recording and, it's constant and there is no corrupted frames or corrupted data, but I noticed the card and SD card door got warm but not too hot, my PC USB card reader gets a lot hotter when writing @ 90 MB/s

700D Logs:

How I made the tests:

LOG 0: Overclocked values to 240 MHz then I took a photo in Photo Mode, after taking the picture it starts to switch to 48 MHz mode.
LOG 1: Without Overclocking, I got into Photo Mode and took a picture, everything is normal in this LOG.

Note: DebugMsg (io_trace_full build) didn't capture the overclocked values @240 MHz correctly instead it showing like this:

Code: [Select]

 1.009.343      Fwrite:00af7544:MMIO : [0xC0400600] <- 0xEEEEEEEE
 1.009.345      Fwrite:00af7544:MMIO : [0xC0400604] <- 0xEEEEEEEE
 1.009.346      Fwrite:00af7544:MMIO : [0xC0400610] <- 0xEEEEEEEE
 1.009.347      Fwrite:00af7544:MMIO : [0xC0400614] <- 0xEEEEEEEE
 1.009.349      Fwrite:00af7544:MMIO : [0xC0400618] <- 0xEEEEEEEE
 1.009.350      Fwrite:00af7544:MMIO : [0xC0400624] <- 0xEEEEEEEE
 1.009.352      Fwrite:00af7544:MMIO : [0xC0400628] <- 0xEEEEEEEE
 1.009.353      Fwrite:00af7544:MMIO : [0xC040061C] <- 0xEEEEEEEE
 1.009.354      Fwrite:00af7544:MMIO : [0xC0400620] <- 0xEEEEEEEE

Same as when making overclock @ 160 MHz it also shows 0xEEEEEEEE, so this a bug from io_full_trace as I think, 0xEEEEEEEE value represents these registers have patched by ML.

It took me many hours to make sd_uhs work in io_trace_full since it's working in top of crop_rec_4k branches, io_trace_full not supported.

Danne · « **Reply #392 on:** June 30, 2020, 08:02:39 PM »

Bravo theBilalFakhouri!

Levas · « **Reply #393 on:** June 30, 2020, 08:40:29 PM »

I smell something burning

Ok, not sure why write test worked here, did nothing new.

What I did, disabled crop_rec and raw recording in canon video tab (Modules were still enabled)
Then I started cam in photo mode and used theBilalFakhouri custom sd-uhs module and dialed in the settings which gave us sdr104 mode.

Code: [Select]

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }; 
static uint32_t sdr_???MHz[]   = {        0x3,        0x3,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 };

Activated the settings and run benchmark in photo mode.

So I did nothing special here, and there is 107Mb/s read speed

After the benchmark I enabled crop_rec and raw recording in video tab. Switched to video mode on cam and started recording...speed drop...write speed is already 20Mb/s

Levas · « **Reply #394 on:** June 30, 2020, 08:44:11 PM »

I did some more testing today, and most succes I had when switching crop_rec and raw recording to off in video menu tab.
Start the camera in video mode, switch on crop_rec and raw recording in video tab and start recording.
Most of the times, the first clip is recorded in high speed, sometimes, the speed is already dropped before any clip is recorded.

So it looks like there is a conflict between Sd-UHS hack and crop_rec and/or raw recording ?

theBilalFakhouri · « **Reply #395 on:** June 30, 2020, 08:50:21 PM »

Nice results!

I think it conflicts somehow with ROM/RAM patches, keep crop_rec turned off, disable SRM memory from mlv_lite and disable small hacks, run some tests in x5 mode, I don't guarantee if will be some improvement probably not . .

theBilalFakhouri · « **Reply #396 on:** July 01, 2020, 12:57:41 AM »

Maybe I got the problem, my theory, In Short: there are another addresses Canon are using them for patching SD Card registers back to 48 MHz.

Explanation:
If you enable the overclock @ 160 MHz or less, you can also disable it by un-patching the addresses, and you will get back to the default speed, unpatching addresses (registers) are done by The following two lines:

Code: [Select]

	unpatch_memory(sd_setup_mode);
	unpatch_memory(sd_setup_mode_in);

I put these lines a in new module I called sd_kill it's safe to use you can get it

, enabling it will un-patch the SD UHS registers (same as when you turn off crop_rec when using higher resolution you will get back to default res), as I said you can get back to the default speed e.g. if you overclocked sd card to 160 MHz, by using unpatch function you will get back to 96 MHz, unlike 240 MHz, if the disable the patches (after the switch to 48 MHz) you will not get to the default speed, you will stuck at 21 MB/s

And if you tried to run the overclock again (to patch the addresses again) nothing will happen it's look like the current addresses are unusable and Canon has switched to another ones.

It's clearly Canon uses another addresses for that, I don't think I can get addresses from io_trace_full when running DebugMsg? if we could and if it was true we will able to get them from DebugMsg LOG and it will be Camera specific addresses . .

I could get addresses by adtg_gui log, but unfortunately I am not sure how to show sd_uhs registers from adtg_gui.

Levas · « **Reply #397 on:** July 01, 2020, 03:49:08 PM »

Can't get it stable

Found a settings that is little bit slower but is activated at least every single time at startup with the automatic sd_uhs module (the other setting is a random hit or miss)

Code: [Select]

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x2,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };

I get about 74MB/s in video mode with the above setting.
But unfortunately the setting can break quickly in 20Mb/s mode.

I can't figure out why it breaks, it can even break during recording, so first 5 seconds the speed is 74/Mb/s and then it suddenly drops to 20MB/s

Other times it breaks when switching between 1x and 5 x zoom modes or it can even break by starting up in photo mode and then switching to video mode.

It breaks with the use of crop_rec and it breaks without crop_rec activated...

theBilalFakhouri · « **Reply #398 on:** July 01, 2020, 04:47:30 PM »

After some more searching in the LOG 0 I found this, maybe it's where the problem starts:

Code: [Select]

 1.300.435      Fwrite:ff74c4dc:23:26: sdDMAWriteBlk: Transfer Block Size Invalid(4809<->6167)
 1.300.462      Fwrite:ff74c540:23:06: sdDMAWriteBlk(SDSTS_ED)(0x2:0x8001:0x40:0x2)

Then it will try something

Code: [Select]

sdWriteBlk: Retry:0
Canon detected something unusual, after this it will do sdSoftReset which will initialize SD Controllers in 48 MHz mode (Safe Mode ?), this error isn't presented in LOG 1 which there is no overclock applied there, I am not fully sure, I will try to call it and see if it will switch SD Controllers to 48 MHz mode without the overclock.

Levas · « **Reply #399 on:** July 01, 2020, 05:35:43 PM »

Some new finding:

Here are 3 different settings that all give about 75MB/s write speed in video mode:

Code: [Select]

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x101,      0x101,        0x1 }
{        0x2,        0x2,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 }
{        0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 }

The first settings is from one of your post theBilalFakhouri
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228368#msg228368

Now, the first one, stays stable, I've tried my best, but it never goes to 20Mb/s

(I get about 75Mb/s in video mode

)
But the second and third settings, which show the same speed when enabled in your custom sd_uhs module, will break within a minute or so to 20Mb/s.

So it could be that not all options are valid, or according to SD standards and will forced to 20MB/s by Canon firmware.
So my guess, we'll have to look further for some magic settings that also give 100MB/s speed, but don't drop to 20Mb/s...

News:

Author Topic: UHS-I / SD cards investigation (Read 231233 times)