Magic Lantern Forum

Developing Magic Lantern => Reverse Engineering => Topic started by: nikfreak on July 30, 2014, 05:46:56 PM

Title: UHS-I / SD cards investigation
Post by: nikfreak on July 30, 2014, 05:46:56 PM
Any idea how to identify which hi-speed mode gets used in e.g. photo mode or while raw recording. I attached an image of ARMu so you can see what I mean.

(http://web.archive.org/web/20190706115049im_/http://i60.tinypic.com/ir2h41.jpg)
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on July 30, 2014, 10:46:25 PM
Spend some hours now to read through standarized specifications. Found a really interesting document here https://www.sdcard.org/downloads/pls/simplified_specs/part1_410.pdf (https://www.sdcard.org/downloads/pls/simplified_specs/part1_410.pdf).

To sum it up UHS-I modes and speeds are controlled by different commands for e.g. cmd6, cmd11, cmd55 and so on. Those control and set block lengths, speed modes, bus widths, voltages and much more. As canon specifies cams like 6D as UHS-I capable I was able to verify in 6D ROm with ARMu that we use "1.8V signaling" and there' also command for "voltageswitch" and blocklength control or set"buswidth" stuff and also that earlier stated commands cmd6, cmd11, cmd55 etc. I also found out that the default voltage is 0.72w for UHS-I and it's stated in the linked document above that for maximum performance standardized 1.44W should be used and as I understood can be used riskfree as it is standardized for UHS-I spec. But i have no clue how to use earlier stated "voltage" sw.
Document above also states that somehow in idle mode of a card when it's powered on again the bus width i 1bit and needs first to be set to 4bit again via commands. This could explain while we need some kind of warm up for our cards before we reach max performance in benchmarking? If anyone has some more ideas please help. i am free to try experiments with my 6D. in ARmu for eg I think I found some stubs for voltageswitch, transfer commands and so on. But for e.g. I have also no clue how to disassemble "./card/STGinitialsetting.c " which i see in ARMu comments.

(http://web.archive.org/web/20190705220029im_/http://i61.tinypic.com/2yzc6yc.jpg)
(http://web.archive.org/web/20190706191105im_/http://i59.tinypic.com/r0sboy.jpg)
(http://web.archive.org/web/20190705221017im_/http://i62.tinypic.com/2eftens.jpg)
(http://web.archive.org/web/20190706192245im_/http://i62.tinypic.com/2reikup.jpg)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 30, 2014, 11:09:09 PM
What is it that you're looking for ?
Do you think you can optimize speed performance so we have 50MB/s speed ? (at the moment 40MB/s is max what I get with the 6d)
Or do you want to know if the 6d is capable of the 104MB/s mode ?
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on July 31, 2014, 08:41:40 AM
Actually I guess Host type for 6D according to the above table is SDR50. If we are lucky then it also could be SDR104. Dunno the technical details of my cam  :o
And yes I get 40MB max, too. This could be influenced by some of those variables / commands stated above. Good thing is it's a standardized spec which Canon has to follow as long as they state that a cam is UHS capable. They do so I found lots of stuff in ARMu. It's not only squeezing out the max performance for 6D I am looking for but also for 650D, 700D, 70D and future cams. Investigating all the threads on the forum I realized there's simply no other chance actually to get more write performance. I tried setting higher buffer size, reducing fps and or resolution for raw recording. I don't know if it is possible but I simply think and hope that Canon crippled max. performance by commands / software.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on July 31, 2014, 09:28:43 AM
SD card initialization messages from 5D3:

        CSMgrTask:ff5c4634:1e:01: CSMgrTask: pMessage=0x361a4, pLStorage=0x647a5c
        CSMgrTask:ff5c457c:1e:01: MapLogToPhysic: pLStorage=0x647a5c
        CSMgrTask:ff5c45a0:1e:01: MapLogToPhysic: SUCCESS(ID=1)
        CSMgrTask:ff5c457c:1e:01: MapLogToPhysic: pLStorage=0x647a5c
        CSMgrTask:ff5c45a0:1e:01: MapLogToPhysic: SUCCESS(ID=1)
        CSMgrTask:ff6bda48:23:03: ---- SDEventHandler(ID=1:Event=8) ----
        CSMgrTask:ff6bcb14:23:01: SD_DeviceCreate: StorageID=1, WP=0
        CSMgrTask:ff6bcc20:23:01: sdIdentifyDrive Start
        CSMgrTask:ff6baeb0:23:01: sdSoftReset( 0 )
        CSMgrTask:ff6baf24:23:01: sdSoftReset SUCCESS
        CSMgrTask:ff6baf40:23:01: sdTrySendCommand1 Start
        CSMgrTask:ff6baf88:23:03: sdTrySendCommand1: Timeout
        CSMgrTask:ff6baeb0:23:01: sdSoftReset( 0 )
        CSMgrTask:ff6baf24:23:01: sdSoftReset SUCCESS
        CSMgrTask:ff6baac0:23:01: sdIdentifyDrive Start
        CSMgrTask:ff6bd6e4:23:01: sdSendIFCondition Start
        CSMgrTask:ff6bd8f4:23:01: sdSendIFCondition End
        CSMgrTask:ff6ba724:23:01: sdSendOCR Start
        CSMgrTask:ff6ba864:23:05: sdSendOCR: Hi-Capacity Card
        CSMgrTask:ff6ba888:23:05: sdSendOCR: 1.8V Signaling Card
        CSMgrTask:ff6baaa4:23:01: sdSendOCR End
        CSMgrTask:ff6bab48:23:01: sdAllSendCID Start
        CSMgrTask:ff6bac7c:23:01: sdAllSendCID End
        CSMgrTask:ff6ba5b0:23:01: sdSendRelativeAddress Start
        CSMgrTask:ff6ba6f8:23:01: sdSendRelativeAddress End
        CSMgrTask:ff6ba1cc:23:01: sdSendCSD Start
        CSMgrTask:ff6ba288:23:03: CsdStructVersion = 1
        CSMgrTask:ff6ba2a8:23:03: MMCSpecVersion = 0
        CSMgrTask:ff6ba370:23:03: C_SIZE=0x0001d0df READ_BL_LEN=0x0009
        CSMgrTask:ff6ba3ac:23:03: *** EraseBlks = 128 ***
        CSMgrTask:ff6ba3d0:23:05: CARD CAPACITY is 59504Mbyte( 121864192Sec )
        CSMgrTask:ff6ba438:23:01: sdSendCSD End
        CSMgrTask:ff6b9df8:23:01: sdSendCID Start
        CSMgrTask:ff6b9eac:23:01: ***********************************
        CSMgrTask:ff6b9ec4:23:01: MID: 0x41
        CSMgrTask:ff6b9ee8:23:01: OID: 0x3432
        CSMgrTask:ff6b9f18:23:01: PNM: SD64G
        CSMgrTask:ff6b9f38:23:01: PRV: 3.0
        CSMgrTask:ff6b9f64:23:01: PSN: 0x02e81ab2
        CSMgrTask:ff6b9f8c:23:01: MDT: 2013/05
        CSMgrTask:ff6b9fa4:23:01: CRC: 0x9b
        CSMgrTask:ff6b9fb4:23:01: ***********************************
        CSMgrTask:ff6ba038:23:03: sdSendCID: MID = 0x41, PDN = 0x5344
        CSMgrTask:ff6ba048:23:01: sdSendCID End
        CSMgrTask:ff6b9bf0:23:01: sdSelectDeselectCard Start
        CSMgrTask:ff6b9ddc:23:01: sdSelectDeselectCard End
        CSMgrTask:ff6b9a78:23:01: sdSetClearCardDetect Start
        CSMgrTask:ff6b9bd4:23:01: sdSetClearCardDetect End
        CSMgrTask:ff6b9708:23:01: sdSendStatus Start
        CSMgrTask:ff6b98bc:23:03: sdSendStatus: SD_CARD_TYPE_H  = 0x0000
        CSMgrTask:ff6b98d8:23:03: sdSendStatus: SD_CARD_TYPE_L  = 0x0000
        CSMgrTask:ff6b9958:23:01: sdSendStatus End
        CSMgrTask:ff6b95fc:23:01: sdSendSCR Start
        CSMgrTask:ff6b9690:23:03: sdSendSCR: SCR=0x03804502, SpecVersion=2
        CSMgrTask:ff6b96e8:23:01: sdSendSCR End
        CSMgrTask:ff6bd384:23:01: sdCheckFunction Start
        CSMgrTask:ff6bd40c:23:01: sdCheckFunction End
        CSMgrTask:ff6b8fec:23:01: sdSetFunction( 16776961 ) Start
        CSMgrTask:ff6b91c8:23:01: sdSetFunction( 16776961 ) End
        CSMgrTask:ff6bae58:23:01: sdIdentifyDrive End(2)
        CSMgrTask:ff6bcd20:23:01: pBlkDev=0x647b64
        CSMgrTask:ff6bcd40:23:03: nBlocks=121864192, blksPerTrack=0, nHeads=0
        CSMgrTask:ff6bcec0:23:03: ERASE_TIMEOUT:2a
        CSMgrTask:ff6bcee0:23:03: ERASE_OFFSET:2
        CSMgrTask:ff6bcf08:23:03: ERASE_SIZE:100
        CSMgrTask:ff6bcf28:23:03: AU_SIZE:f
        CSMgrTask:ff6bcf3c:23:03: dwEraseSectorSize:128
        CSMgrTask:ff6bcf5c:23:03: dwEraseBlks:80000 CSD:1
        CSMgrTask:ff6bdb44:23:05: SD_GetAccessMode=3
        CSMgrTask:ff6bdb7c:23:05: Set Hi-Speed Mode( 48MHz )
        CSMgrTask:ff6bc87c:23:01: sdReadBlk: st=0, num=1, buf=0x40647d4c
        CSMgrTask:ff6bb3f4:23:01: sdDMAReadBlk: st=0, num=1
        CSMgrTask:ff5c592c:21:01: fsuGetPart: default 1
        CSMgrTask:ff5c5948:21:01:             nBlocks = 121864192, blkOffset = 32768


Code from dm-spy-experiments branch, "CONFIG_DEBUG_INTERCEPT_STARTUP = y" in Makefile.user, and the following code changes:


diff -r 3f8197bdca6a src/dm-spy.c
--- a/src/dm-spy.c Thu Jul 31 10:23:06 2014 +0300
+++ b/src/dm-spy.c Thu Jul 31 10:23:37 2014 +0300
@@ -24,7 +24,7 @@
extern void dm_spy_extra_install();
extern void dm_spy_extra_uninstall();

-#define BUF_SIZE (1024*1024)
+#define BUF_SIZE (128*1024)
static char* buf = 0;
static volatile int len = 0;

@@ -86,6 +86,8 @@
/* careful, 1MB may be too much for your camera */
static char staticbuf[BUF_SIZE];

+#include "cache_hacks.h"
+
// call this from boot-hack.c
void debug_intercept()
{
@@ -95,12 +97,9 @@
     {
         buf = staticbuf;
         //~ dm_spy_extra_install();                     /* not exactly working, figure out why */
-        patch_memory(
-            DebugMsg_addr,                              /* hook on the first instruction in DebugMsg */
-            MEM(DebugMsg_addr),                         /* do not do any checks; on 5D2 it would be e92d000f, not sure if portable */
-            B_INSTR(DebugMsg_addr, my_DebugMsg),        /* replace all calls to DebugMsg with our own function (no need to call the original) */
-            "dm-spy: log all DebugMsg calls"
-        );
+        cache_lock();
+        cache_fake(DebugMsg_addr + 0xFF9F07C0, B_INSTR(DebugMsg_addr, &my_DebugMsg), TYPE_DCACHE);
+       
     }
     else // subsequent call, uninstall the hook and save log to file
     {
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on July 31, 2014, 10:00:27 AM
Thanx for this a1ex. Will try and see what I get from my 6D but my initial thoughts according to your card initiliazation messages on 5D3 should be true:

6D / 5D3 (and maybe other UHS-I capable cams) use at least SDR50 or DDR50 bus speed with 1.8V signal voltage and a possible max. power of 1.44W. 5D3 is set at 48Mhz. card initialization messages give no details about max. power or stuff like Buswidth (UHS-I can only use 1bit or 4bit).
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 01, 2014, 12:36:30 AM
Title: Re: UHS-I / SD cards investigation
Post by: Audionut on August 01, 2014, 12:48:13 AM
Enable GDB in the makefile.

That's the debugger.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on August 01, 2014, 06:12:52 AM
You should also add a call to debug_intercept() somewhere early in the boot process; the 6D uses a different boot method than the one I've tested. I think putting it right before init_task should be OK.

There may be some more quirks (like who knows what is clearing the cache, or you run out of RAM), and to debug these, LED blinking (with busy waiting, not msleep) is pretty much the only tool you have. Take a look in blink.c.

I'll post a decoder for these blinks soon, so you can read the startup messages (encoded as LED blinks) with a second camera.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on August 01, 2014, 09:14:45 AM
Just updated the dm-spy-experiments branch, so you should no longer need to apply any code changes (CONFIG_DEBUG_INTERCEPT_STARTUP = y should be enough). Enabling the hook right before init_task seems to be the most portable way, and there we already have the DryOS task scheduler running (so msleep, task info routines and some basic stuff like snprintf are now available). There are no interesting debug mesages before init_task anyway.
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 01, 2014, 09:21:58 AM
cool i will try and report once i get home later
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on August 01, 2014, 01:08:10 PM
Here's a quick hack:


...
if sd_device.off_0x10 /*0x39970*/ != 0:
    sd_use_1_8V_signaling()
    if ret_sd_use_1_8V_signaling_FF6BAB0C != 0:
        RAM:DebugMsg(35, 5, msg='Set 1.8V Signaling')
        ...

NSTUB(0xff48446c, sd_use_1_8V_signaling):
ff48446c: e3a00000 mov r0, #0
ff484470: e12fff1e bx lr


=> when setting the DebugMsg hook:

+        patch_memory(0xff48446c, 0xe3a00000, 0xe3a00001, "SD 1.8V");
+        patch_memory(0x39970, 0, 1, "SD 1.8V");


=>


Set 1.8V Signaling
sdSendVoltageSwitch Start
sdSendVoltageSwitch End
sdAllSendCID Start
sdAllSendCID End
sdSendRelativeAddress Start
sdSendRelativeAddress End
sdSendCSD Start
CsdStructVersion = 1
MMCSpecVersion = 0
C_SIZE=0x0001d0df READ_BL_LEN=0x0009
*** EraseBlks = 128 ***
CARD CAPACITY is 59504Mbyte( 121864192Sec )
sdSendCSD End
sdSendCID Start
***********************************
MID: 0x41
OID: 0x3432
PNM: SD64G
PRV: 3.0
PSN: 0x02e81ab2
MDT: 2013/05
CRC: 0x9b
***********************************
sdSendCID: MID = 0x41, PDN = 0x5344
sdSendCID End
sdSelectDeselectCard Start
sdSelectDeselectCard End
sdSetClearCardDetect Start
sdSetClearCardDetect End
sdSendStatus Start
sdSendStatus: SD_CARD_TYPE_H  = 0x0000
sdSendStatus: SD_CARD_TYPE_L  = 0x0000
sdSendStatus End
sdSendSCR Start
sdSendSCR: SCR=0x03804502, SpecVersion=2
sdSendSCR End
sdCheckFunction Start
sdCheckFunction End
sdSetFunction( 16711682 ) Start
sdSetFunction( 16711682 ) End
sdIdentifyDrive End(2)
pBlkDev=0x647c44
nBlocks=121864192, blksPerTrack=0, nHeads=0
ERASE_TIMEOUT:2a
ERASE_OFFSET:2
ERASE_SIZE:100
AU_SIZE:f
dwEraseSectorSize:128
dwEraseBlks:80000 CSD:1
SD_GetAccessMode=7
Set Hi-Speed Mode( 96MHz )
sdReadBlk: st=0, num=1, buf=0x40648130
sdDMAReadBlk: st=0, num=1
fsuGetPart: default 1
            nBlocks = 121864192, blkOffset = 32768


With a regular (non-UHS) card, this command fails. I can still browse the photos on the card :D

That's it for now, see you next week.
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 01, 2014, 02:17:22 PM
Yeah only UHS-I (II/III) cards use 1.8V. Older cards operate at 3.3V. So I assume we can play with settings like bus-width, block length, hispeed-mode (96MHz) too and see if we can reach some performance increase as long as stubs for 'em are idenified. I think I found some interesting ones while browsing through ARMu. Just need to do it again and note them down.

Edit: Errrmm

Quote
SD_GetAccessMode=7
Set Hi-Speed Mode( 96MHz )

Compared to your earlier post you now got 96MHz? Same card as earlier where debug posted earlier showed 48MHz???  :D :D :D
Title: Re: UHS-I / SD cards investigation
Post by: Greg on August 01, 2014, 06:24:41 PM
500D log

        CSMgrTask:ff329c70:1e:01: CSMgrTask: pMessage=0x1d548, pLStorage=0x7c67f8
        CSMgrTask:ff329bb0:1e:01: MapLogToPhysic: pLStorage=0x7c67f8
        CSMgrTask:ff329bd4:1e:01: MapLogToPhysic: SUCCESS(ID=1)
        CSMgrTask:ff329bb0:1e:01: MapLogToPhysic: pLStorage=0x7c67f8
        CSMgrTask:ff329bd4:1e:01: MapLogToPhysic: SUCCESS(ID=1)
        CSMgrTask:ff393d60:23:03: ---- SDEventHandler(ID=1:Event=8) ----
        CSMgrTask:ff393148:23:01: SD_DeviceCreate: StorageID=1, TotalBlks=0, HiddenBlk=0
        CSMgrTask:ff39315c:23:01:                  WriteProtect=0
        CSMgrTask:ff393244:23:01: sdIdentifyDrive Start
        CSMgrTask:ff392180:23:01: sdSoftReset( 0 )
        CSMgrTask:ff3921e8:23:01: sdSoftReset SUCCESS
        CSMgrTask:ff392204:23:01: sdTrySendCommand1 Start
        CSMgrTask:ff39223c:23:03: sdTrySendCommand1: Timeout
        CSMgrTask:ff392180:23:01: sdSoftReset( 0 )
        CSMgrTask:ff3921e8:23:01: sdSoftReset SUCCESS
        CSMgrTask:ff391d84:23:01: sdIdentifyDrive Start
        CSMgrTask:ff393b68:23:01: sdSendIFCondition Start
        CSMgrTask:ff393c34:23:01: sdSendIFCondition End
        CSMgrTask:ff391a54:23:01: sdSendOCR Start
        CSMgrTask:ff391b68:23:03: sdSendOCR: Hi-Capacity Card
        CSMgrTask:ff391b68:23:03: sdSendOCR: Hi-Capacity Card
        CSMgrTask:ff391b68:23:03: sdSendOCR: Hi-Capacity Card
        CSMgrTask:ff391b68:23:03: sdSendOCR: Hi-Capacity Card
        CSMgrTask:ff391b68:23:03: sdSendOCR: Hi-Capacity Card
        CSMgrTask:ff391b68:23:03: sdSendOCR: Hi-Capacity Card
        CSMgrTask:ff391b68:23:03: sdSendOCR: Hi-Capacity Card
        CSMgrTask:ff391d68:23:01: sdSendOCR End
        CSMgrTask:ff391de0:23:01: sdAllSendCID Start
        CSMgrTask:ff391f40:23:01: sdAllSendCID End
        CSMgrTask:ff3918ec:23:01: sdSendRelativeAddress Start
        CSMgrTask:ff391a28:23:01: sdSendRelativeAddress End
        CSMgrTask:ff3915a8:23:01: sdSendCSD Start
        CSMgrTask:ff391650:23:03: CsdStructVersion = 1
        CSMgrTask:ff39166c:23:03: MMCSpecVersion = 0
        CSMgrTask:ff391730:23:03: C_SIZE=0x000076b2 READ_BL_LEN=0x0009
        CSMgrTask:ff391770:23:03: *** EraseBlks = 128 ***
        CSMgrTask:ff391794:23:05: CARD CAPACITY is 15193Mbyte( 31116288Sec )
        CSMgrTask:ff3917ac:23:01: sdSendCSD End
        CSMgrTask:ff39133c:23:01: sdSendCID Start
        CSMgrTask:ff3913ec:23:01: **************************************************
        CSMgrTask:ff391400:23:01: 0x03534453
        CSMgrTask:ff391414:23:01: 0x55313647
        CSMgrTask:ff391428:23:01: 0x80160927
        CSMgrTask:ff39143c:23:01: 0x9300c22d
        CSMgrTask:ff39144c:23:01: **************************************************
        CSMgrTask:ff3914d4:23:03: sdSendCID: MID = 0x03, PDN = 0x5355
        CSMgrTask:ff3914e4:23:01: sdSendCID End
        CSMgrTask:ff3938e4:23:01: sdSelectDeselectCard Start
        CSMgrTask:ff3939b0:23:01: sdSelectDeselectCard End
        CSMgrTask:ff3910c4:23:01: sdSetClearCardDetect Start
        CSMgrTask:ff391320:23:01: sdSetClearCardDetect End
        CSMgrTask:ff390e90:23:01: sdSendStatus Start
        CSMgrTask:ff391050:23:01: sdSendStatus: SD_CARD_TYPE  = 0x0000
        CSMgrTask:ff3910a8:23:01: sdSendStatus End
        CSMgrTask:ff390c74:23:01: sdSendSCR Start
        CSMgrTask:ff390e48:23:03: sdSendSCR: SCR=0x03803502, SpecVersion=2
        CSMgrTask:ff390e70:23:01: sdSendSCR End
        CSMgrTask:ff390b6c:23:01: sdSwitchFunction( 0 ) Start
        CSMgrTask:ff390c28:23:03: sdSwitchFunction( 0 ) Status=0x80000003, AccessMode=3
        CSMgrTask:ff390c54:23:01: sdSwitchFunction( 0 ) End
        CSMgrTask:ff391ed0:23:03: Support Hi Speed Mode
        CSMgrTask:ff390b6c:23:01: sdSwitchFunction( 1 ) Start
        CSMgrTask:ff390c28:23:03: sdSwitchFunction( 1 ) Status=0x80000003, AccessMode=3
        CSMgrTask:ff390c54:23:01: sdSwitchFunction( 1 ) End
        CSMgrTask:ff39202c:23:01: sdIdentifyDrive End(2)
        CSMgrTask:ff39344c:23:01: pBlkDev=0x7c684c
        CSMgrTask:ff39346c:23:03: nBlocks=31116288, blksPerTrack=0, nHeads=0
        CSMgrTask:ff393e8c:23:05: SD_GetAccessMode=3
        CSMgrTask:ff393ea8:23:03: Set Hi-Speed Mode( 48MHz )
        CSMgrTask:ff29000c:21:01: [fsuDeviceAdd] (1)
        CSMgrTask:ff392e18:23:01: sdReadBlk: st=0, num=1, ofst=0
        CSMgrTask:ff3928a0:23:01: sdDMAReadBlk: st=0, num=1
        CSMgrTask:ff32fc0c:21:01: fsuGetPart: default 1
        CSMgrTask:ff32fc28:21:01:             nBlocks = 31116288, blkOffset = 8192
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 02, 2014, 11:58:16 AM
Quote from: a1ex on August 01, 2014, 06:12:52 AM
You should also add a call to debug_intercept() somewhere early in the boot process; the 6D uses a different boot method than the one I've tested. I think putting it right before init_task should be OK.

There may be some more quirks (like who knows what is clearing the cache, or you run out of RAM), and to debug these, LED blinking (with busy waiting, not msleep) is pretty much the only tool you have. Take a look in blink.c.

I'll post a decoder for these blinks soon, so you can read the startup messages (encoded as LED blinks) with a second camera.

Can you be more precise? The only log (pastebin ful log (http://pastebin.com/ERDYi0RS)) I can get is this and it contains nothing about sdcard:
             init:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
             init:ff146a60:00:01: [PM] DisablePowerSave (Counter = 1)
             init:ff0c32b0:8b:16:
K302 ICU Firmware Version 1.1.3 ( 5.7.4 )
             init:ff0c32c4:8b:05:
ICU Release DateTime 2013.02.28 18:02:17
             init:ff0fc40c:00:03: [SEQ] CreateSequencer (Startup, Num = 6)
             init:ff0fc660:00:02: [SEQ] NotifyComplete (Startup, Flag = 0x10000)
             init:ff0fc6c4:00:03: [SEQ] NotifyComplete (Cur = 0, 0x10000, Flag = 0x10000)
          Startup:ff133ca0:02:16: PROPAD_CreateFROMPropertyHandle DRAMAddr 0x41744000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0x10000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0x10000
          Startup:ff1472f8:00:01: [SF] CreateSerial : 0x80a00408 0xc0820438 0xc0820404
          Startup:ff1471a0:00:01: [SF] readSerialFlash Dest 0x4044f9dc Src 0x10000 Size 512
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x44
          Startup:ff147168:00:01: [SF] SendCommandWithAddress : 0x3 0x1 0x0 0x0
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
          Startup:ff130d58:02:16: SerialFlash Packages!! 0x10000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0x10000
          Startup:ff1472f8:00:01: [SF] CreateSerial : 0x80a00408 0xc0820438 0xc0820404
          Startup:ff1471a0:00:01: [SF] readSerialFlash Dest 0x4044f9dc Src 0x10000 Size 512
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x44
          Startup:ff147168:00:01: [SF] SendCommandWithAddress : 0x3 0x1 0x0 0x0
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
          Startup:ff1472f8:00:01: [SF] CreateSerial : 0x80a00408 0xc0820438 0xc0820404
          Startup:ff147c88:00:01: [SF] readSerialFlashWithQuad Dest 0x41744000 Src 0x10000 Size 1813912

          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x44
          Startup:ff149cd8:00:01: [SF] SendCommandToSerialFlash : 0x9f
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
          Startup:ff147c50:00:03: [SF] readManufactureCodeSerialFlash 0xc2
          Startup:ff147cbc:00:03: [SF] MACRONIX
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x44
          Startup:ff149cd8:00:01: [SF] SendCommandToSerialFlash : 0x6
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x44
          Startup:ff149cd8:00:01: [SF] SendCommandToSerialFlash : 0x5
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x44
          Startup:ff147bc8:00:01: [SF] SendCommandToSerialFlashWithData : 0x1 0x40 0xf0
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x44
          Startup:ff149cd8:00:01: [SF] SendCommandToSerialFlash : 0x5
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x44
          Startup:ff147168:00:01: [SF] SendCommandWithAddress : 0x6b 0x1 0x0 0x0
          Startup:ff0fc320:00:05: [SEQ] seqEventDispatch (Startup, 0)
          Startup:ff0c3664:8b:05: startupEntry
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf00c0000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf00c0000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf00c0000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf01c0000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf8080000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf8080000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf80ac000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf80ae000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf0060000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf0060000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf0080000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf00a0000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf01e0000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf01e0000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf01e0000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf0200000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf0020000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf0020000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf0020000
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0xf0021000
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
          Startup:ff149b8c:00:01: [SF] IsAddressSerialFlash 0x490000
          Startup:ff1472f8:00:01: [SF] CreateSerial : 0x80a00408 0xc0820438 0xc0820404
          Startup:ff1471a0:00:01: [SF] readSerialFlash Dest 0x4059df3c Src 0x10000 Size 512
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x44
          Startup:ff147168:00:01: [SF] SendCommandWithAddress : 0x3 0x1 0x0 0x0
          Startup:ff149c04:00:01: [SF] SetCSSerialFlash : 0xc022002c 0x46
          Startup:ff130d58:02:16: SerialFlash Packages!! 0x7
          Startup:ff1311d4:02:03: pFromProperty->pNextWriteAddress 0x10000
          Startup:ff0c298c:8b:05: startupPropAdminMain : End
          Startup:ff0fc660:00:02: [SEQ] NotifyComplete (Startup, Flag = 0x20000000)
          Startup:ff0fc830:00:16: [SEQ ERROR] NotifyComplete (Cur = 1, 0x2, Flag = 0x20000000)
          Startup:ff2293b4:81:03: Initialize Adjective & Situation
          ....
          ....
          PropMgr:ff311d2c:01:03: LV_CFilter : kind 0 hv 0
          PropMgr:ff311d60:01:03: Level 0 0 0 0 0 0 0 0
          PropMgr:ff0cb72c:8a:03: HotPlug WifiSetting = 0
          PropMgr:ff3104fc:01:03: Complete WaitID = 0x8000003F, 0x00000000(0)
          PropMgr:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
          PropMgr:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          PropMgr:ff0f7f28:8d:03: emSlaveChangeCBR : AUTO_POWEROFF (1)
          PropMgr:ff0f7f58:8d:03: emSlaveChangeCBR : UILOCK (0x0)
          PropMgr:ff3104fc:01:03: Complete WaitID = 0x80000045, 0x00000000(0)
          PropMgr:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
          PropMgr:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          Startup:ff0fc320:00:05: [SEQ] seqEventDispatch (Startup, 1)
          Startup:ff0c43b8:8b:05: startupPrepareProperty
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          PropMgr:ff0c4378:8b:03: PROP_GPS_AUTO_TIME_SETTING 0
          PropMgr:ff3104fc:01:03: Complete WaitID = 0x8003006F, 0x00000000(0)
          PropMgr:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
          PropMgr:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          Startup:ff0fc660:00:02: [SEQ] NotifyComplete (Startup, Flag = 0x20000000)
          Startup:ff0fc6c4:00:03: [SEQ] NotifyComplete (Cur = 2, 0x20420110, Flag = 0x20000000)
          Startup:ff0c4400:8b:03: startupPrepareCapture : CLASSID_FLAG_PROPAD
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          Startup:ff128384:24:03: FM_Initialize (0, 1, 0)
          PropMgr:ff1281cc:24:03: fmResultCBR (0x5a16ac)
          PropMgr:ff128210:24:03: PROP_HDD_DCIM_PATH (/)
          PropMgr:ff0fce14:81:03: dwNewAeModeDial = 2
          PropMgr:ff228b50:81:01: CustomSlaveCBR 0x80000000(0 0 0) 0x2
          PropMgr:ff228f14:81:03: ChangeAEMode -> 2 @AE_MODE_DIAL
          PropMgr:ff228834:81:03: !! Convert End !!
          PropMgr:ff228b50:81:01: CustomSlaveCBR 0x80000044(0 0 0) 0x0
          PropMgr:ff228b50:81:01: CustomSlaveCBR 0x80000001(0 0 0) 0x2
          PropMgr:ff30d23c:01:03: Deliv WaitID = 0x80000001, 0xFF228B1C(1)
          PropMgr:ff228b50:81:01: CustomSlaveCBR 0x205000e(0 0 0) 0x1
          PropMgr:ff3104fc:01:03: Complete WaitID = 0x80000001, 0xFF228B1C(0)
          PropMgr:ff310570:01:03: SpecialComplete ID = 0x80000001, 0x80000001 2478
          PropMgr:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
          PropMgr:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          Startup:ff12260c:27:03: FC_Initialize [drive:3][ClassID:39]
          Startup:ff141348:00:03: [RTC] PROPAD_GetPropertyData : PROP_TIME_ZONE 19
          Startup:ff1414f4:00:03: [RTC] PROPAD_GetPropertyData : PROP_SUMMERTIME_SETTING 0
          PropMgr:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
          PropMgr:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          PropMgr:ff14108c:00:07: [RTC] ChangePropertyCBR 0x0, 0x6000
          Startup:ff14d6d8:00:03: [SND] Seq LPC 5-0
          Startup:ff14d708:00:03: [SND]  HARB_ARBMODE  Before:0x00000001 Current:0x00000000
          Startup:ff14d740:00:03: [SND]  HARB_HARBCTRL Before:0x2AAAAAE8 Current:0x2AAAAAE8
          Startup:ff14d750:00:03: [SND] Seq LPC 5-1
          Startup:ff14d778:00:03: [SND] Seq LPC 5-2
          Startup:ff14d79c:00:03: [SND] Seq LPC 5-3
          Startup:ff14d7d4:00:03: [SND] Seq LPC 5-4
          Startup:ff14d7e4:00:03: [SND] Seq LPC 5-5
          Startup:ff14d80c:00:03: [SND] Seq LPC 5-6
          Startup:ff14d820:00:03: [SND] Seq LPC 5-7
          Startup:ff0c44dc:00:16: [SND] Seq LPC fin
          Startup:ff0ef8ec:80:16: hMemoryQueue (0x800010) hStorageQueue (0x820012)
          PropMgr:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
          PropMgr:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          PropMgr:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
          PropMgr:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          PropMgr:ff0ef054:80:16: MemoryStatusMasterResultCBR
          PropMgr:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
          PropMgr:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
    **INTERRUPT**:ff146a60:00:01: [PM] DisablePowerSave (Counter = 2)
    **INTERRUPT**:ff146ad0:00:01: [PM] EnablePowerSave (Counter = 1)
          Startup:ff1fb614:80:16: ### LIST 8 0 8 ###
          Startup:ff1fb744:80:03: AllocateMemoryUnit For ExMem1
          Startup:ff1fb744:80:03: AllocateMemoryUnit For ExMem1
          Startup:ff1fb744:80:03: AllocateMemoryUnit For ExMem1
          Startup:ff1fb744:80:03: AllocateMemoryUnit For ExMem1
          Startup:ff1fb950:80:03: AllocateMemoryUnit For OnlyMem1
          Startup:ff1fb950:80:03: AllocateMemoryUnit For OnlyMem1
          Startup:ff1fb950:80:03: AllocateMemoryUnit For OnlyMem1
          Startup:ff1fb950:80:03: AllocateMemoryUnit For OnlyMem1
          Startup:ff1fc1f8:80:03: #Add [ FreeUnit 4 AddNum 0 ]
          Startup:ff1fc244:80:03: #Add1[ 0-> Unit 006c2e60 Address 52400000 SubUnit 15]
          Startup:ff1fc330:80:03: #AddB[ 0-> Unit 006c2e60 Address 52400000 ]
          Startup:ff1fc244:80:03: #Add1[ 1-> Unit 006c2e44 Address 50000000 SubUnit 15]
          Startup:ff1fc330:80:03: #AddB[ 1-> Unit 006c2e44 Address 50000000 ]
          Startup:ff1fc244:80:03: #Add1[ 2-> Unit 006c2e28 Address 4d400000 SubUnit 15]
          Startup:ff1fc330:80:03: #AddB[ 2-> Unit 006c2e28 Address 4d400000 ]
          Startup:ff1fc244:80:03: #Add1[ 3-> Unit 006c2e0c Address 4b000000 SubUnit 15]
          Startup:ff1fc330:80:03: #AddB[ 3-> Unit 006c2e0c Address 4b000000 ]
          Startup:ff206b70:80:03: #Add2[ FreeUnit 4 ]
          Startup:ff206c1c:80:03: MemMgr(IMAGE) 0 1
          Startup:ff206c1c:80:03: MemMgr(IMAGE) 0 0
          Startup:ff206c1c:80:03: MemMgr(IMAGE) 0 6
          Startup:ff206c1c:80:03: MemMgr(IMAGE) 0 0
          Startup:ff1fa384:80:03: ---- MODE[0 0 0]
          Startup:ff201f30:80:03: RearrangeMemMgr 0 3
          Startup:ff201fec:80:03: PROP_MEMORY_STATUS(SELF) (10->)13 (OTHER) 13
          Startup:ff202020:80:03: Deliver PROP_MEMORY_STATUS 13
          Startup:ff1fd844:80:01: ###### AllocateMemoryFromShootMemoryObject 1
          Startup:ff1fd920:80:01: ###### 0 0x6C2FA0 0x6C2FA0
          Startup:ff1fd920:80:01: ###### 1 0x6C2FA8 0x6C2D80
          Startup:ff1fd948:80:01: ###### 1 pMemoryUnit 0x6C2D80 0xDBF50
          Startup:ff1fd9b4:80:01: AllocatableSizeOfPackHeap 4194192, 16384, 16384
          Startup:ff1fdc5c:80:01: !!AllocateMemory 16384
          Startup:ff1fda94:80:01: AllocateMemoryFromShootMemoryObject 1 4194192 16384 0x000dbfa0
          Startup:ff1fdab0:80:01: ###### VirtualXXXFreeSize1 4194192 0
          Startup:ff1fde70:80:01: ###### VirtualXXXFreeSize2 4177796 0
          Startup:ff1fde84:80:01: ###### !! AllocateMemoryFromShootMemoryObject hMemSuite 0xDBFA0
          Startup:ff201b74:80:01: OK AllocDev[0] DBFA0 4000 50 1 3FBF84 3FBF84 3FFF90
          Startup:ff1f5f9c:80:03: ClearBusy(0x1) 0x40->0x40,0x0(0x40)[0,0]
          Startup:ff2019d8:80:03: ### SRMExMem1_1State Event 0 Result State 0->6 ###
           RscMgr:ff0ef380:80:02: srmEventDispatch: Current = 0, dwEventID = 15, dwParam = 4 S
           RscMgr:ff0ef588:80:02: srmEventDispatch: Current = 0, dwEventID = 15, dwParam = 4 E
           RscMgr:ff0ef380:80:02: srmEventDispatch: Current = 0, dwEventID = 15, dwParam = 4 S
           RscMgr:ff0ef588:80:02: srmEventDispatch: Current = 0, dwEventID = 15, dwParam = 4 E
           RscMgr:ff0ef380:80:02: srmEventDispatch: Current = 0, dwEventID = 15, dwParam = 4 S
           RscMgr:ff2019d8:80:03: ### SRMExMem1_1State Event 4 Result State 6->6 ###
           RscMgr:ff1fe40c:80:01: ###### VirtualFreeMemoryToShootMemoryObject 0xDBFA0
           RscMgr:ff1fe0dc:80:01: VirtualAllocFreeMemory 0xDBFA0 0x6C2FA0 0x6C2F68
           RscMgr:ff1fe16c:80:01: ###### ChunkSize 16396
           RscMgr:ff1fe198:80:01: ###### VIRTUAL_FREE pMemoryUnit->VirtualFreeSize[0] 4194192
           RscMgr:ff1fe1c4:80:01: ######    +pMemoryObject->Virtual1stFreeSize 4194192
           RscMgr:ff1fe378:80:01: ######    +pMemoryObject->Virtual2ndFreeSize 0
           RscMgr:ff1fd7b8:80:01: FreeMemoryToShootMemoryObject 4194192 16384 0xDBFA0
           RscMgr:ff1fd7cc:80:01: ###### FreeMemoryToShootMemoryObject 0xDBFA0
           RscMgr:ff201dac:80:01: FMem3 DBFA0 4000 9AD6A00 9AD6A00 9AD6A00 3FFF90 3FFF90 3FFF90
           RscMgr:ff1fa384:80:03: ---- MODE[0 0 0]
           RscMgr:ff1fd844:80:01: ###### AllocateMemoryFromShootMemoryObject 1
           RscMgr:ff1fd920:80:01: ###### 0 0x6C2FA0 0x6C2FA0
           RscMgr:ff1fd920:80:01: ###### 1 0x6C2FA8 0x6C2D80
           RscMgr:ff1fd948:80:01: ###### 1 pMemoryUnit 0x6C2D80 0xDBF50
           RscMgr:ff1fd9b4:80:01: AllocatableSizeOfPackHeap 4194192, 16384, 16384
           RscMgr:ff1fdc5c:80:01: !!AllocateMemory 16384
           RscMgr:ff1fda94:80:01: AllocateMemoryFromShootMemoryObject 1 4194192 16384 0x000dbfa0
           RscMgr:ff1fdab0:80:01: ###### VirtualXXXFreeSize1 4194192 0
           RscMgr:ff1fde70:80:01: ###### VirtualXXXFreeSize2 4177796 0
           RscMgr:ff1fde84:80:01: ###### !! AllocateMemoryFromShootMemoryObject hMemSuite 0xDBFA0
   ....
        .....
          ....
          .....
          ....


edit: I tried to set debug_intercept(); earlier in boot-hack.c but had no luck. lol those fake cache hacks in there for 6D are irritating btw. Is this the reason why you once said 6D doesn't use gui_massive_eventloop? Just got the idea that those could also be related to the ML menu disappearing bug...
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on August 04, 2014, 08:36:18 AM
Yeah, that's the log; probably it's full and you either need a larger size, or just print the interesting messages only - for example, only log those messages from CSMgrTask.

I've ran some benchmarks meanwhile: 7.9 MB/s write / 10.9 read with the hack, 11.1 / 21.4 without.
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 04, 2014, 11:45:51 AM
Ok so you got 96MHz Hi-SpeedMode set? Do we have a possibility to check buswidth set in this mode, too (e.g. 1bit or 4bit)? Which result would a downclock set at 24MhZ produce? ANother question: did you run that bench on something like 600D or 5d3. the results look slow to me..
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 05, 2014, 12:46:12 PM
Looks like this needs more time to study - at least for me. As far as I understood a complete set of commands (CMD) have to be executed step by step to get the desired result. Some commands have default values which will be used if they are not set properly (see 2nd post the graphics showing function groups with default values like "power limit 0.72W / driver strength default B"). This may result in unexpected low results by using defaults.
I dunno where to interrupt this in our case and send own commands or even if this would be possible without re-initiating the whole steps.

ACMD6 is required to set 4bit mode.
CMD6 still looks most interesting to me as this will define UHS-I modes, power limits and driver strengths.
I have still no clue if SDR104 mode is supported or we are using SDR50 mode.

https://www.sdcard.org/developers/overview/host_controller/simple_spec/Simplified_SD_Host_Controller_Spec.pdf (https://www.sdcard.org/developers/overview/host_controller/simple_spec/Simplified_SD_Host_Controller_Spec.pdf)

(http://web.archive.org/web/20190706134319im_/http://i59.tinypic.com/2508i36.jpg)

Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 05, 2014, 08:05:32 PM
Ok and here are some stats from a log where my 6D crashed with an ERR70. Summed up: 6D uses 96MHz Hi-SpeedMode and as long 6D can't use more than 96MHz (more possible? less maybe more effective {downclock}?)  the only things which could now improve write performance on 6D could be adjusting values for driver strength, bus speed, bus width and/or power limit. Still "guess & hope" so if anyone has more knowledge give a shout plz:


[SD] ---- SDEventHandler(ID=1:Event=8) ----
    751:   479.188 [SD] sdTrySendCommand1: Timeout
    752:   581.824 [SD] sdSendOCR: CCS=1, S18A=1
    753:   581.940 [SD] Set 1.8V Signaling
    754:   616.888 [SD] CsdStructVersion = 1
    755:   616.912 [SD] MMCSpecVersion = 0
    756:   616.941 [SD] C_SIZE=0x000077bf READ_BL_LEN=0x0009
    757:   616.960 [SD] *** EraseBlks = 128 ***
    758:   616.988 [SD] CARD CAPACITY is 15328Mbyte( 31391744Sec )
    759:   618.293 [SD] sdSendCID: MID = 0x01, PDN = 0x5355
    760:   625.440 [SD] sdSendStatus: SD_CARD_TYPE_H  = 0x0000
    761:   625.457 [SD] sdSendStatus: SD_CARD_TYPE_L  = 0x0000
    762:   628.260 [SD] sdSendSCR: SCR=0x03803502, SpecVersion=2
    763:   629.946 [SD] nBlocks=31391744, blksPerTrack=0, nHeads=0
    764:   629.968 [SD] ERASE_TIMEOUT:1
    765:   629.979 [SD] ERASE_OFFSET:3
    766:   629.992 [SD] ERASE_SIZE:8
    767:   630.001 [SD] AU_SIZE:9
    768:   630.013 [SD] dwEraseSectorSize:128
    769:   630.028 [SD] dwEraseBlks:2000 CSD:1
    770:   630.054 [SD] SD_GetAccessMode=7
    771:   630.066 [SD] Set Hi-Speed Mode( 96MHz )
    772:   631.102 [FSU] fsuGetPart: Block(8192, 31383552, 31391744)
    773:   634.495 [FM] EV_INSERTION_COMPLETE : ID = 2, stat = 8192
    774:   634.561 [FM] fmLateMountCard
    775:   634.584 [FM] fmPrepareShooting (Drive = 2)
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 07, 2014, 03:50:27 PM
Found out that 4bit bus width is mandatory for UHS-I in Hi-Speed-Mode. So forget about that - at least for 650D / 700D / 70D and 6D. So the only things left are : bus speed, driver strength and power limit. From what I can definitely say that our UHS-I capable cams support at least "sdio spec V3.0". And in conjunction with that there are so called CCCR registers (Card Common Control Registers). Very interesting are adresses 14h (DDR50 / SDR50 or SDR104 bus modes) and 15h (driver strength). See also page 32 of the official specification V3 (https://www.sdcard.org/downloads/pls/simplified_specs/archive/partE1_300.pdf).

I guess that all of those are somehow related to "STGInitialSetting.c" or those SUBs seen around that in ARMu. See following picture.

(http://web.archive.org/web/20190707224601im_/http://i62.tinypic.com/33c9bbb.jpg)
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 07, 2014, 03:56:25 PM
Btw earlier post was done because I simply hope that we can switch some day to SDR104 because while googling around all host controllers I found also supported SDR104 (not only SDR50 / DDR50). I think incam controller could be same.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on August 07, 2014, 04:47:20 PM
Would be insane if the controllers support SDR104  :D
But I doubt they would, why would Canon not make use of the speed  :-\
I can imagine that Canon slows down the max speed of 50MbitMb to 40MbitMb for better lifetime or more stable performance.
But why would you slow down to 40MbitMb if the controller supports 100MbitMb

But don't let me stop you from investigation, who knows what will be discovered  8)
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 07, 2014, 05:01:02 PM
Quote from: Levas on August 07, 2014, 04:47:20 PM
But I doubt they would, why would Canon not make use of the speed  :-\
I can imagine that Canon slows down the max speed of 50Mbit to 40Mbit for better lifetime or more stable performance.
But why would you slow down to 40Mbit if the controller supports 100Mbit...
...
No clue myself just guessing why they could slow it down:

But ofc you may be right that they use simply crippled and cheap to produce components for cheap cameras including a fullframe 6D.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on August 07, 2014, 05:10:27 PM
Didn't think about the CF card versus SD card speed theory.
Sounds like a good enough reasons for Canon to slow down the SD controllers to make a speed difference statement between CF and SD camera's.

Hmm,  :D   would be insane if you can manage to get speeds above 50Mb!  (Although getting actually 50Mb output speed (instead of the 40Mb) would be a big break through too!)

Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 07, 2014, 05:17:26 PM
Actually I am still freshman but documenting here what I find. reverse enginnering is for other guys. maybe some day I can manage to do this on my own but for now other have to do some work or lead me into the right direction. As long as my camera has warranty I am willing to risk everything  8)
And yes even if it's only 50MB/s we theroretically still have 10MB/s lost somewhere. So it's worth it in my eyes to spend some work into this. In real worl practice it should be something about 5-7MB/s which I assume we can gain if it's 50MB/s only...
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 11, 2014, 03:21:25 PM
just wanted to add that by downclocking (for e.g. EOS 6D is 96MHz and could be downclocked to 12/  24 / 48MHz) we might achieve ofc less write speed but this may become useful for cases where continuous write speed is not much important (time lapses?). Advantage is less power consumption in theory... According to the sdcard specifications this should happen if some old card (mmc in formware) is used. Should run at max. 48MHz automatically.
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on August 20, 2014, 03:14:34 PM
UHS-I capable cameras need to be switched to UHS104 & SDR104 (bus speed 104MB) by using CMD6 function group 1.
I found some CMD6 commands as stated earlier in ARMu. maybe someone can help to accomplish setting my 6D or any other UHS-I capable cam to the new bus speed by a CMD6 code switch?
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on September 03, 2014, 05:31:01 PM
Ok need definitely help from one of you ML dev gurus on this. I spent again some hours to read through all official SD specification PDFs from V1.10 til 4.X and I can say CMD6 is all that is needed to set bus speed modes (SDR50 SDR104 DDR50 ...). There's one CMD6 SwitchCommand available in my 6D rom so it may be worth a try. Ofc this only works for UHS-I cards. I also simply guess that 5D3 is set to SDR25. a1ex tried once (see 1st page) but if something is done wrong then sd spec says that afterwards settings are defaulted and that's then SDR12. 

(http://web.archive.org/web/20190709025536im_/http://i59.tinypic.com/ip741i.png)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on September 06, 2014, 08:45:53 PM
Wish I could help you...but have no idea how.

"There's one CMD6 SwitchCommand available in my 6D rom"

Sounds promising...
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on September 15, 2014, 06:03:22 PM
Quote from: a1ex on August 04, 2014, 08:36:18 AM
....for example, only log those messages from CSMgrTask.

Can someone give me a detailed instruction based on dm-spy-experiments (https://bitbucket.org/hudson/magic-lantern/commits/branch/dm-spy-experiments) on how to log only "CSMgrTask" + "SdioDrv" +"SdioTsk". I bought a faster Sandisk card which is rated 60MB/s in write speed (before this I had panasonic Gold with a rating of max 45MB/s.). I wanna compare the logs I get from both cards and yes I am still trying to find a way where buss speed modes are set. Googling around I found this here (http://www.dooroos.org/dooroos.RealTime/WebHelp/sdiodrv.h.htm) which contains stuff like

#define MMC_CAP_UHS_SDR12       (1 << 15)       /* Host supports UHS SDR12 mode */

        #define MMC_CAP_UHS_SDR25       (1 << 16)       /* Host supports UHS SDR25 mode */

        #define MMC_CAP_UHS_SDR50       (1 << 17)       /* Host supports UHS SDR50 mode */

        #define MMC_CAP_UHS_SDR104      (1 << 18)       /* Host supports UHS SDR104 mode */

        #define MMC_CAP_UHS_DDR50       (1 << 19)       /* Host supports UHS DDR50 mode */


and maybe SdioDrv (http://magiclantern.wikia.com/wiki/IDAPython/Tracing_calls/CreateTask) can show what's going on in 6D logs. So some hints where i can define what is going to be logged would be helpful.
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on September 25, 2014, 04:14:46 PM
Please have a look at this example sub / stub. Looks like I understood myself what I can achieve with dm-spy-experiments but plz at least give me a hint if that would be ok:

(http://web.archive.org/web/20190707093507im_/http://i58.tinypic.com/2l57bs.png)

Now I take /src/dm-spy-extra.c and in there I would add like this:


#ifdef CONFIG_6D
    { 0xyzblabla, "StateTransition", 4 , state_transition_log },
    { 0xFF78D814, "SDCheckStatus", WhatToPutHere??? }
#endif


Would that work for startup / "debug (don't click me?) and log some stuff related to SDCheckStatus or doesn't this make any sense to identify what's going on for other stubs?
Title: Re: UHS-I / SD cards investigation
Post by: kuga0509 on February 05, 2015, 06:37:45 AM
Any luck on this?  Before I even read this thread, the only logical limiting factor I could find seemed to be that the software was limiting the hardware.  Since the hardware should support the SDR104 standard, and since the voltage and all of the other requirements would remain the same, it really seems that simple.  I hope this works out.  :)

The only other solution would be to load a different h264 profile to change the color space.  I know that isn't raw, but it may at least get the desired bit depth.  However, that would still require this same issue to be resolved since it would likely require a higher bus speed...
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on June 18, 2017, 11:54:16 PM
Revisiting this.

At the suggestion of Ant123, I've ran ML under the Xilinx version of QEMU, which also emulates UHS. This revealed the missing bits from my previous patch: on 5D3, although it was printing a message about 96MHz, the card interface was set up in the same way as for... 24 MHz (which explains the previous benchmark results (http://www.magiclantern.fm/forum/index.php?topic=12862.msg124516#msg124516)).

Took the missing register configurations from 700D and...

(http://a1ex.magiclantern.fm/bleeding-edge/UHS/5D3-uhs.png)

8)




Now the details.

Xilinx QEMU (http://www.wiki.xilinx.com/QEMU) includes UHS emulation (along with some other nice stuff) and is based on QEMU 2.6.x (at the time of writing). Currently we have patches for QEMU 2.5.0 and 2.9.0, and for the Xilinx version, it will be something in-between. If there is interest, I can commit the patches.

First, I've ran 5D3 1.2.3 from the dm-spy-experiments branch on both the camera and QEMU 2.5.0. Result (http://a1ex.magiclantern.fm/bleeding-edge/UHS/5D3-123-before-cam-vs-qemu.html).

The problem:

CSMgrTask:ff6bdb7c:23:05: Set Hi-Speed Mode( 48MHz )                ; real hardware
[   CSMgrTask:ff6bdb9c ] (23:05) Set Normal-Speed Mode( 24MHz )     ; QEMU 2.5.0


That's where the Xilinx QEMU comes in. Result (http://a1ex.magiclantern.fm/bleeding-edge/UHS/5D3-123-before-cam-vs-qemu-xilinx.html): after a minor patch to the SCR structure (first field - SpecVersion - changed from 0 to 1), both the log from camera and Xilinx QEMU now have:

CSMgrTask:ff6bdb44:23:05: SD_GetAccessMode=3
CSMgrTask:ff6bdb7c:23:05: Set Hi-Speed Mode( 48MHz )


Also, besides a minor difference in handling CMD1, and the card capacity, the emulation matches the reality pretty well. There's even some nice debug info if you uncomment DEBUG_SD in hw/sd/sd.c and use the "-d sd" switch on the command line. Result (http://a1ex.magiclantern.fm/bleeding-edge/UHS/5D3-123-before-cam-vs-qemu-xilinx-sd.html) - in particular, these lines:

CSMgrTask:ff6b8fec:23:01: sdSetFunction( 16776961 ) Start
CSMgrTask:000aea50:00:00: *** sd_setup_mode(0x2), from ff484738
sd: CMD16 0x00000040 state 4
sd: Response: 00 00 09 00 state 4
CSMgrTask:000aea50:00:00: *** sd_setup_mode(0x2), from ff6b8300
sd: CMD6 0x80ffff01 state 4
sd: Function default selected (fn grp 2)
sd: Function high-speed/SDR25 selected (fn grp 1)
sd: Response: 00 00 09 00 state 5
sd: CMD13 0x45670000 state 4
sd: Response: 00 00 09 00 state 4
sd: CMD16 0x00000200 state 4
sd: Response: 00 00 09 00 state 4
CSMgrTask:ff6b91c8:23:01: sdSetFunction( 16776961 ) End


Now we can analyze how the SD initialization code configures the hardware (MMIO registers). This is as easy as specifying "-d io" in QEMU's command-line. Result (http://a1ex.magiclantern.fm/bleeding-edge/UHS/5D3-123-before-cam-vs-qemu-xilinx-io.html).

At this point I've applied the old patch (http://www.magiclantern.fm/forum/index.php?topic=12862.msg124289#msg124289) and tried to figure out whether there's any obvious trouble. Result (http://a1ex.magiclantern.fm/bleeding-edge/UHS/5D3-123-qemu-xilinx-io-before-vs-oldhack.html):

Without hack:

CSMgrTask:ff6bdb7c:23:05: Set Hi-Speed Mode( 48MHz )
CSMgrTask:000aeed0:00:00: *** sd_set_mode(0x1, 0x3), from ff6bdb88
(some registers)
CSMgrTask:000aeed0:00:00: *** sd_setup_mode(0x3), from ff484738
(some more registers)
CSMgrTask:ff6bc87c:23:01: sdReadBlk: st=0, num=1, buf=0x40983808


With hack:

CSMgrTask:ff6bdb5c:23:05: Set Hi-Speed Mode( 96MHz )
CSMgrTask:000aeef0:00:00: *** sd_set_mode(0x1, 0x4), from ff6bdb88
(some registers)
CSMgrTask:000aeef0:00:00: *** sd_setup_mode(0x4), from ff484738
(a few more registers, but many of them missing!)
CSMgrTask:ff6bc87c:23:01: sdReadBlk: st=0, num=1, buf=0x409837bc


In other words, sd_setup_mode(4) appears to skip some hardware configuration it might be supposed to perform. With other arguments, it sets a bunch of registers, and the argument appears to be related to SD clock speed (3 = 48MHz, 2 = 24MHz, 4 = 96MHz).

Let's see what it does on 700D. Result (http://a1ex.magiclantern.fm/bleeding-edge/UHS/5D3-123-oldhack-vs-700D.html) (also with a zoomed-in view (http://a1ex.magiclantern.fm/bleeding-edge/UHS/hi-speed-issue.html)).

Notice some additional registers on 700D (highlighted in red on the zoomed-in view). What's up with them?

The 0xC04004xx range is never set on 5D3, so these registers are probably specific to 700D hardware. Same for 0xC040063x and 0xC040064x. I didn't touch them.

The remaining registers, 0xC04006[012]x, are also set on 5D3, at other speed modes. In these other modes, their values are the same on 700D. You can see them here: sd_setup_mode(2) (http://a1ex.magiclantern.fm/bleeding-edge/UHS/sd_setup_mode_2.html), sd_setup_mode(4) (http://a1ex.magiclantern.fm/bleeding-edge/UHS/sd_setup_mode_4.html).

My hypothesis was that 5D3's SD controller is UHS-capable, but for some unknown reason (could be even problems during the initial tests), Canon decided not to include it in the firmware. As a result, some of the UHS initialization code (hopefully a small part) was optimized out.

So I've tried to take the missing register configurations from 700D.

Therefore, the patch for 5D3 1.2.3 becomes:

/* in dm-spy.c, right before dm_spy_extra_install() */
patch_instruction(0xff48446c, 0xe3a00000, 0xe3a00001, "SD 1.8V");

/* in dm-spy-extra.c */
static void sd_setup_mode_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    /* log the original call as usual */
    generic_log(regs, stack, pc);

    if (regs[0] == 4)
    {
        MEM(0xC0400600) = 3;
        MEM(0xC0400610) = 4;
        MEM(0xC0400614) = 0x1D000301;
        MEM(0xC0400618) = 0;
        MEM(0xC0400624) = 0x201;
        MEM(0xC0400628) = 0x201;
        MEM(0xC040061C) = 0x100;
        MEM(0xC0400620) = 4;
        MEM(0xC0400604) = 3;
    }
}

    /* under CONFIG_5D3_123 */
    { 0xFF4844A0, "sd_setup_mode", 1, sd_setup_mode_log },


If you decide to try it, make sure you don't have any important data on your card. Otherwise, you will be playing Russian Roulette with your data (just like with my other SD patch (https://www.magiclantern.fm/forum/index.php?topic=13983.0)).

Does this apply to DIGIC 4 cameras?

I'm afraid not - the hardware configuration of these cameras is different (and a lot simpler). You now know where to look, so you can play with it, attempt to change the clock speed and report your findings.

Can the clock speed be pushed even further?

I have no idea. Feel free to play with these registers, run the benchmarks and report.

Can this be included in a module, to be used on a regular ML build?

That's hard, because the hack must be applied before the SD card gets initialized by Canon firmware (in other words, before loading any module, and also before loading the config file). So, even if we include it in ML core, it will be hard to create an option for it.

At the moment, the easiest way to try it would be a custom build. Probably best to start from the crop_rec_4k branch, as the backend support is there, and there is little reason to try this hack outside that branch.

It might be possible to switch the SD to a higher speed on the fly. Didn't investigate this approach.

How's this useful in practice?

Other than raw video recording on both CF and SD at the same time (aka card spanning), it's probably not very useful.

What's the maximum total speed (CF+SD)?

Load the benchmarks module (bench.mo) to find out.

How can I get similar results in QEMU?

Take a look at this post (http://www.magiclantern.fm/forum/index.php?topic=2388.msg183164#msg183164). In a nutshell, it's the dm-spy-experiments branch compiled with CONFIG_DEBUG_INTERCEPT_STARTUP=y for the camera, and additionally with CONFIG_QEMU=y for running it under the emulator. These two will give logs that can be directly compared, and with the logging options from QEMU, you can get additional details. Then, look for CSMgrTask in the logs, compare them and try to understand what it does. Also refer to SD docs (summarized nicely by nikfreak earlier in this thread) to understand the initialization protocol. That's it. If you get stuck, just ask (here or on IRC).

If there is interest, I can commit the patch for Xilinx QEMU and write a walkthrough similar to this one (http://www.magiclantern.fm/forum/index.php?topic=15895.msg185103#msg185103).

Please note I no longer have the UHS card to do more tests (it wasn't mine), so from now on, what will happen with this hack is entirely up to you.
Title: Re: UHS-I / SD cards investigation
Post by: aschille84 on June 19, 2017, 08:03:27 AM
Wow, so this means we can get another 30mb/s write speed. I would like to try it out.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 19, 2017, 10:38:24 AM
Nice finding.

One question about this one:

Can the clock speed be pushed even further?

I have no idea. Feel free to play with these registers, run the benchmarks and report.


How is this in the other cams, like the 6d. I've always thought that UHS-I, 50 MByte/s (SDR50) was the limit here ?
Or is there still a small chance that it can reach UHS-I, 104 MByte/s (SDR104) ?
Title: Re: UHS-I / SD cards investigation
Post by: Ant123 on June 19, 2017, 11:31:21 AM
Quote from: a1ex on June 18, 2017, 11:54:16 PM
My hypothesis was that 5D3's SD controller is UHS-capable, but for some unknown reason (could be even problems during the initial tests), Canon decided not to include it in the firmware. As a result, some of the UHS initialization code (hopefully a small part) was optimized out.

If you can not enter UHS mode with ML but it works without, read this topic (https://chdk.setepontos.com/index.php?topic=13089.30).
Conclusion:
Most SD cards need to be reinitialized by switching off SD power if they already were in UHS mode.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 01, 2018, 10:23:19 PM
Quote from: a1ex on June 18, 2017, 11:54:16 PM
Can the clock speed be pushed even further?

Let's try some overclocking: sd_uhs.mo (https://a1ex.magiclantern.fm/bleeding-edge/uhs/sd_uhs.mo) (to be loaded on top of crop_rec_4k branch). Got the idea after looking into the EOS M shutter bug and understanding how clocks work (https://www.magiclantern.fm/forum/index.php?topic=21728.msg198529#msg198529) - to some extent. See the source below for RE notes.

Before and after. This is a slower card, compared to previous post (https://www.magiclantern.fm/forum/index.php?topic=12862.msg185968#msg185968).

(https://a1ex.magiclantern.fm/bleeding-edge/uhs/before.png) (https://a1ex.magiclantern.fm/bleeding-edge/uhs/after.png)

Source: module_hginfo_dump.sh sd_uhs.mo

5D3 only for now, tested on 1.1.3 with 2 UHS cards and 2 regular ones.

Other D5 models on the todo list. Do not try pattern-matching the stubs for other models - it won't work. Only SD_ReConfiguration is generic code, the rest is 5D3-specific.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 01, 2018, 10:57:57 PM
 :o

Is this the real april first surprise  8)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 01, 2018, 11:10:59 PM
Do you think there's room left in the cameras that already have about 40Mb writing speed, like the, random pick, canon 6d  :P
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on April 01, 2018, 11:14:31 PM
No way man! Wow! Is this Day of Surprises movie? :D
This is very large hope for doing continues 3K or just continues Slow mo on small cameras!

And is there benefit by merging both SD + CF writes speeds for continues UHD and 4K  for 5D3 ?
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on April 02, 2018, 08:33:20 AM
So this is SDR50 at how much speed? Guess beyond 96MHz?
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 02, 2018, 08:47:37 AM

module_hginfo_dump.sh sd_uhs.mo



static uint32_t regs[] = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sd50[] = {        0x3,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 };   /* SDR50 values from 700D */
static uint32_t unck[] = {        0x3,        0x3,                             0x5, 0x1D000401,        0x0,      0x201,      0x201,      0x100,        0x5 };   /* underclocked values */
static uint32_t ovck[] = {        0x3,        0x3,                             0x3, 0x1D000201,        0x0,      0x201,      0x201,      0x100,        0x3 };   /* overclocked values */
static uint32_t twak[] = {        0xF,        0xF,                             0xF, 0x00000F00,        0x0,      0xF0F,      0xF0F,        0x0,        0xF };   /* what can be tweaked? */
// mode 0                                                                      17F  0x1D004101           0        403F        403F          7F          7F
// mode 1 16MHz                     3           3          1          1         1D  0x1D001001           0       0xF0E       0xF0E          1D          1D
// mode 2 24MHz                     3           3          1          1         13  0x1D000B01           0       0xA09       0xA09          13          13
// mode 3 48MHz                     3           3          1          1          9  0x1D000601           0       0x504       0x504       0x100           9
// mode 4 96MHz SDR50 700D          3           3          1          1          4  0x1D000301           0       0x201       0x201       0x100           4
// mode 5 serial flash?             3                                            7  0x1D000501           0       0x403       0x403       0x403           7
// mode 6                           7                                           13  0x1D000B01           0       0xA09       0xA09          13          13


Look at 0xC0400610 and 0xC0400620:

16 vs 24 MHz => 0x13 vs 0x1D. Exact ratio match with 0x14 vs 0x1E (setting registers to X-1 is a common trick used in Canon hardware).
24 vs 48 MHz => 0x9 vs 0x13. Exact match with 0x10 vs 0x20.
48 vs 96 MHz => 0x4 vs 0x9. Exact match with 0x5 vs 0x10.
Overclocked: 0x4 vs 0x3 => 125% theoretical speedup.
Benchmark result (read column): 54.4/43.1 = 126%. Check!

=> SDR50 @ 120 MHz.

0xC0400624 hi: exact match at 16/24 and 24/48, rounded at 48/96. 5/2/1.25 = 2 => exact match for the overclocked version?
0xC040061C ?!

Underclocking test: 0x4 vs 0x5 => 83.33% speed => 35.9MB/s expected (matches my benchmarks).

Porting notes will follow. Basic idea: place a logging hook right after these registers are set, to be able to override them. 5D3 does not configure these registers in  UHS mode (likely optimized out in Canon firmware), which is why pattern-matching the stub won't work on other models. Registers are refreshed on every sdReadBlk/sdWriteBlk. SD_ReConfiguration will reset the card, including power cycling. My call to that function is not thread safe - do not run the overclocking tests while other tasks are accessing the card. Debug with qemu -d io,sdcf (or xilinx-qemu, but I need to cleanup and publish the patch for that).
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on April 02, 2018, 09:05:34 AM
woot.
I always thought SDR50 was limited by host to 100MHz according to the screenshots I posted on page1.
120MhZ rather sounds like we have SDR104 - at least if canon followed official specs. Anyways as always great job. I encourage 5D3 owners to test stability and report their findings - ofc everyone wanna have this. If you got a unique way to handle / port this for Digic 5 cameras then don't hesitate to post it @a1ex.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 02, 2018, 09:11:53 AM
Actually, one of my UHS cards (a slow one, that writes at 15MB/s after the hack) refuses the overclocked settings, but handles the regular SDR50 and the underclocked one.

I did try to change the function to SDR104, but did not make a difference with any of my cards. 5D3 1.1.3 dm-spy-experiments:


static void sd_set_function_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    /* log the original call as usual */
    generic_log(regs, stack, pc);

    /* force UHS-I SDR104 */
    regs[0] = 0xff0003;
}


    { 0xFF6ADE34, "sdSetFunction", 1, sd_set_function_log },
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on April 02, 2018, 09:58:28 AM
Yay tricky one.
Ant123 states to switch power off if already in UHS mode (https://www.magiclantern.fm/forum/index.php?topic=12862.msg185992#msg185992). Maybe he can comment on it. Just a guess but once we find out how to reinitialize the card it could probably also help with the eosm shutter bug - while extending boot times due to reinitialization.

In 2014 while googling I didn't find host controllers (https://www.magiclantern.fm/forum/index.php?topic=12862.msg124894#msg124894) which only support one of both modes.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 02, 2018, 11:08:28 AM
OK, I was wrong. Applied the SDR104 hack again and got another speed improvement!

(https://a1ex.magiclantern.fm/bleeding-edge/uhs/after-sdr104-120mhz.png)


static void sd_set_function_log(uint32_t* arm_regs, uint32_t* stack, uint32_t pc)
{
    /* UHS-I SDR50? */
    if (arm_regs[0] == 0xff0002)
    {
        /* force UHS-I SDR104 */
        arm_regs[0] = 0xff0003;
    }
}


Module updated (same link).
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 02, 2018, 11:22:10 AM
Nice one  :D
Does the SD card you're using for this claim a read speed, like the sandisks, 45MB/s, 80Mb/s, 90Mb or 95Mb/s ?
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 02, 2018, 11:26:56 AM
It doesn't claim anything special, but noticed it can do about 70MB/s in the card reader, using dd.


sudo dd if=/dev/mmcblk0 of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.5176 s, 74.0 MB/s


No luck with 160MHz yet.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on April 02, 2018, 11:49:39 AM
A1ex,

Is it possible to implement card spanning with the losslessly compressed 4K-crop recording modes on the 5D3?  That would be a break through for this camera, I think, since it will provide much larger recording times at high resolutions.
Title: Re: UHS-I / SD cards investigation
Post by: Ant123 on April 02, 2018, 12:18:17 PM
Quote from: nikfreak on April 02, 2018, 09:58:28 AM
Yay tricky one.
Ant123 states to switch power off if already in UHS mode (https://www.magiclantern.fm/forum/index.php?topic=12862.msg185992#msg185992). Maybe he can comment on it.

What can I comment? Just read CHDK forum from here (https://chdk.setepontos.com/index.php?topic=13089.msg132542#msg132542).
On powershots the card is already in UHS mode while loading CHDK. So most cards can't be switched to UHS mode two times.
Maybe on DSLRs the card is not in UHS mode while loading autoexec.bin and you don't need to turn off SD power.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 03, 2018, 12:55:26 PM
Ran a brute force (random) search for the above registers, and...

SDR104 @ 160MHz 8)


static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 };   /* SDR50 values from 700D (96MHz) */
static uint32_t sdr_80MHz[]    = {        0x3,        0x3,                             0x5, 0x1D000401,        0x0,      0x201,      0x201,      0x100,        0x5 };   /* underclocked values: 80MHz = 96*(4+1)/(5+1) */
static uint32_t sdr_120MHz[]   = {        0x3,        0x3,                             0x3, 0x1D000201,        0x0,      0x201,      0x201,      0x100,        0x3 };   /* overclocked values: 120MHz = 96*(4+1)/(3+1) */
static uint32_t sdr_132MHz[]   = {        0x2,        0x2,                             0x2, 0x1D000201,        0x0,      0x100,      0x100,      0x100,        0x2 };   /* overclocked values: 132MHz?! (found by brute-forcing) */
static uint32_t sdr_160MHz[]   = {        0x2,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };   /* overclocked values: 160MHz = 96*(4+1)/(2?+1) (found by brute-forcing) */


Before and after (5D3 1.1.3):
(https://a1ex.magiclantern.fm/bleeding-edge/uhs/before.png) (https://a1ex.magiclantern.fm/bleeding-edge/uhs/after-sdr104-160mhz.png)

I'd expect even higher speeds with faster SD cards.

Caveat: SDR104 requires tuning the sampling point (not implemented, not performed by Canon firmware, but doable). That might be required to avoid random errors, speed drops, or higher frequency - if the controller supports it. From 0xC0400610/20, the frequencies are 80, 96, 120, 160 and 240 - the latter is probably too high. Frequency appears tweakable from other registers - if any of you can figure out how to explain the 132MHz preset, that might be helpful for getting other intermediate values.


Download: sd_uhs.mo (https://a1ex.magiclantern.fm/bleeding-edge/uhs/sd_uhs.mo) (same link), to be loaded on top of crop_rec_4k branch (link removed; one SD card died during the experiment).
Source: module_hginfo_dump.sh sd_uhs.mo (sd_uhs (https://bitbucket.org/hudson/magic-lantern/branch/sd_uhs) branch, based on crop_rec_4k)

Supported models: 5D3 1.1.3/1.2.3, 700D 1.1.5, 6D 1.1.6 (edit: 650D 1.0.4, 100D 1.0.1, 70D 1.1.2, EOS M 2.0.2).
Card must be at least 2GB (not checked).
Porting to other DIGIC 5 models should be as easy as finding the stubs.

Brute-forcing: press shutter halfway during the initial tests, until it starts running some more. Press shutter halfway again to stop (infinite loop).
No guarantees of success, no guarantees of safety, no guarantees of data integrity. You have been warned.

Please post logs and benchmarks.
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 03, 2018, 01:51:24 PM
Too dumb to work out how to take a screen shot but here is a photo of 700D 1.1.5 ..Sandisk Extreme pro V30 card usually gives 37MBs Write  8)





(https://thumb.ibb.co/gs2RhH/test_SD_card.jpg) (https://ibb.co/gs2RhH)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 03, 2018, 01:53:16 PM
Sandisk extreme class 10 UHS-I - claiming 45mb/s read speed - on 6d
Before hack - Read 44Mb/s Write 41Mb/s
After hack - Read 61Mb/s Write 46Mb/s

Looks like SDR50 @120 Mhz and SDR104 @ 132Mhz are the sweet spot for this SD card

(https://farm1.staticflickr.com/798/41207230481_4707f4f8e7_b.jpg)
(https://farm1.staticflickr.com/883/41207230221_70fe056687_b.jpg)

EDIT:
Canon's default on 6d is SDR104@96Mhz
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 03, 2018, 03:21:26 PM
Quote from: a1ex on April 03, 2018, 12:55:26 PM
Source: module_hginfo_dump.sh sd_uhs.mo
Wanted to take a look at the source code, but can't find it ?

Would expect to find it here, what am I doing wrong  ???
https://bitbucket.org/hudson/magic-lantern/src/bb163f78aa6a76d97af777f729d12e32640d5dc8/modules/module_hginfo_dump.sh?at=crop_rec_4k&fileviewer=file-view-default (https://bitbucket.org/hudson/magic-lantern/src/bb163f78aa6a76d97af777f729d12e32640d5dc8/modules/module_hginfo_dump.sh?at=crop_rec_4k&fileviewer=file-view-default)
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 03, 2018, 03:36:38 PM
Run that command in the terminal.

BTW, how fast is this card in the card reader? The last test might indicate the need for sampling point tuning.

@Tony: Debug -> Screenshot. Also run the card benchmarks from bench.mo (after closing the console).
Title: Re: UHS-I / SD cards investigation
Post by: Sapporo on April 03, 2018, 03:42:53 PM
Canon 6D and 64GB Sandisk Extreme Pro UHS-I 95MB/s

2018/04/03 15:37:47
===================
Before the hack: r:44MB/s w:42MB/s  W:42MB/s R:44MB/s  : )  [best 42MB/s]
SDR50 @ 96MHz  : r:44MB/s w:40MB/s  W:42MB/s R:44MB/s  meh [best 42MB/s]
SDR50 @ 96MHz  : r:44MB/s w:43MB/s  W:41MB/s R:44MB/s  : )  [best 43MB/s]
SDR50 @ 80MHz  : r:37MB/s w:35MB/s  W:36MB/s R:37MB/s  meh [best 43MB/s]
SDR50 @ 80MHz  : r:37MB/s w:36MB/s  W:35MB/s R:37MB/s  meh [best 43MB/s]
SDR50 @ 120MHz : r:54MB/s w:53MB/s  W:53MB/s R:55MB/s  : )  [best 53MB/s]
SDR50 @ 120MHz : r:54MB/s w:51MB/s  W:51MB/s R:54MB/s  meh [best 53MB/s]
SDR104 @ 96MHz : r:44MB/s w:40MB/s  W:43MB/s R:44MB/s  meh [best 53MB/s]
SDR104 @ 96MHz : r:44MB/s w:43MB/s  W:41MB/s R:44MB/s  meh [best 53MB/s]
SDR104 @ 80MHz : r:37MB/s w:35MB/s  W:36MB/s R:37MB/s  meh [best 53MB/s]
SDR104 @ 80MHz : r:37MB/s w:36MB/s  W:35MB/s R:37MB/s  meh [best 53MB/s]
SDR104 @ 120MHz: r:54MB/s w:51MB/s  W:52MB/s R:55MB/s  meh [best 53MB/s]
SDR104 @ 120MHz: r:54MB/s w:53MB/s  W:51MB/s R:54MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:60MB/s w:58MB/s  W:59MB/s R:60MB/s  : )  [best 58MB/s]
SDR104 @ 132MHz: r:60MB/s w:54MB/s  W:57MB/s R:60MB/s  meh [best 58MB/s]
SDR104 @ 160MHz: r:72MB/s w:69MB/s  W:69MB/s R:72MB/s  : )  [best 69MB/s]
SDR104 @ 160MHz: r:72MB/s w:69MB/s  W:68MB/s R:72MB/s  : )  [best 69MB/s]

Done.
Please run THOROUGH tests before using!!!
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 03, 2018, 03:51:53 PM
Quote from: a1ex on April 03, 2018, 03:36:38 PM
BTW, how fast is this card in the card reader? The last test might indicate the need for sampling point tuning.

In the SD card slot of my computer the card write speeds varies between 44-52Mb/s


Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on April 03, 2018, 04:42:12 PM
Crossing fingers for stability. Thanks for taking time to achieve this @a1ex. You're awesome.  ;D
Title: Re: UHS-I / SD cards investigation
Post by: dfort on April 03, 2018, 04:43:52 PM
Quote from: Levas on April 03, 2018, 03:21:26 PM
Wanted to take a look at the source code, but can't find it ?

Quote from: a1ex on April 03, 2018, 03:36:38 PM
Run that command in the terminal.

I took a few more steps.

Step 1 - put a copy of the sd_uhs.mo file you downloaded into the magic-lantern/modules directory
Step 2 - navigate your terminal to that directory: "cd ~/magic-lantern/modules"
Step 3 - this runs under crop_rec_4k so switch to that branch: "hg up -C crop_rec_4k" (the "-C" deletes any uncommitted changes)
Step 3 (optional) - you might want to make a new branch e.g.: "hg branch sd_uhs"
Step 4 - run that command, like this: "./module_hginfo_dump.sh sd_uhs.mo > sd_uhs.patch"
Step 5 - now import that patch you just created: "hg import sd_uhs.patch"
Step 6 - clean up files you no longer need: "rm sd_uhs.mo sd_uhs.patch"

Now you've got the source code for the module in your repository ready to compile.
Title: Re: UHS-I / SD cards investigation
Post by: dfort on April 03, 2018, 05:19:17 PM
Stubs for EOSM:

    if (is_camera("EOSM", "2.0.2"))
    {
        sd_setup_mode       = 0xFF338D40;
        sd_setup_mode_in    = 0xFF338DC8;
        sd_setup_mode_reg   = 1;
        sd_set_function     = 0xFF63EF60;
        SD_ReConfiguration  = (void *) 0xFF641314;
    }


Using a SanDisk Extreme Pro 32GB (the no shutter-bug card)

before
(https://farm1.staticflickr.com/897/40496638864_8f65b91033.jpg) (https://flic.kr/p/24GxRRs)

after
(https://farm1.staticflickr.com/897/41165781182_54213c8b45.jpg) (https://flic.kr/p/25HFoxY)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 03, 2018, 05:24:11 PM
Thanks Dfort, got the source (sd_uhs.c) on my computer now  :D

And why does everyone here has a quicker card than I have  :P
Waiting for stability reports and then go after a new SD card  ;D
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 03, 2018, 05:46:18 PM
Modified the module so it skips the 160Mhz, which is better for the card I'm using (Sandisk extreme 64GB 45Mb/s)
Before:
(https://farm1.staticflickr.com/786/39401007800_a8cd1cb9e4_o.jpg)

After: (Sorry for the messy screen  :P)
(https://farm1.staticflickr.com/801/40497255884_1ac0d92a3b_o.jpg)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 03, 2018, 06:25:15 PM
Quote from: a1ex on April 03, 2018, 03:36:38 PM
BTW, how fast is this card in the card reader? The last test might indicate the need for sampling point tuning.
Googling about sampling point tuning.

Has to be done only one time and is card specific right?
@Alex, do you think this could help or could it be that the card is at max at 45Mb/s write speed.
The benchmarks of the others show no need for sampling point tuning   ???
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 03, 2018, 06:27:11 PM

(https://thumb.ibb.co/fzV5bc/bench0.jpg) (https://ibb.co/fzV5bc)

(https://thumb.ibb.co/iabdwc/bench1.jpg) (https://ibb.co/iabdwc)


700D Sandisk V30, wooooooooo does this mean I will be able to do 3k on a 700D ?  :D
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on April 03, 2018, 06:53:50 PM
Levas if the card is labelled 45Mb/s you alread reached max performance. You will need to get a faster one for e.g. Sandisk Extreme Pro
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on April 03, 2018, 06:54:27 PM
Amazing Stuff > Any chance that this will work on SL1?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 03, 2018, 07:02:53 PM
Quote from: nikfreak on April 03, 2018, 06:53:50 PM
Levas if the card is labelled 45Mb/s you alread reached max performance. You will need to get a faster one for e.g. Sandisk Extreme Pro

The labeled speed on my card is read speed, not write speed as far as I know.
But in this case I also think my card is maxed out at 45Mb/s write speed.

If this stuff is stable enough to record raw video with, I'll be happy looking for a new SD card  :D

Quote from: a1ex on April 03, 2018, 12:55:26 PM
From 0xC0400610/20, the frequencies are 80, 96, 120, 160 and 240 - the latter is probably too high.

Anyone already had the balls to try out 240 Mhz  :P
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 03, 2018, 07:10:06 PM
Press shutter halfway during these tests (until it starts running some more), then let it run overnight on the power adapter, then press shutter halfway again to stop. It will try random changes to these registers, starting from the fastest configuration found. That's an extremely crude version of a genetic algorithm, but it's how I've found the presets above 120MHz (as manual tweaking did not exactly work out of the box).
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 03, 2018, 07:14:45 PM
So that's needed to find the 208Mhz for SDR104.
About 100Mb/s should be what we're aiming for, right   :D

Don't have a SD card to test it out, I'll let this over to the people with fast SD cards

EDIT:
Or does it keep/save a log file and can I help with this too ?
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 03, 2018, 07:26:33 PM
700D
2520x1072
2:35
12 bit lossless
Continuous recording  !!! @58MB/s

Pretty impressive
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on April 03, 2018, 07:43:09 PM
EDIT: wrong link posted.

That's correct one:
hint, hint... (https://www.magiclantern.fm/forum/index.php?topic=19300.msg183286#msg183286)
Title: Re: UHS-I / SD cards investigation
Post by: dfort on April 03, 2018, 08:05:46 PM
Quote from: OlRivrRat on April 03, 2018, 06:54:27 PM
Any chance that this will work on SL1?

100D stubs - not tested
    if (is_camera("100D", "1.0.1"))
    {
        sd_setup_mode       = 0xFF3355B0;
        sd_setup_mode_in    = 0xFF335648;
        sd_setup_mode_reg   = 1;
        sd_set_function     = 0xFF6530A4;
        SD_ReConfiguration  = (void *) 0xFF655458;
    }


[EDIT] And the rest of the supported Digic 5 cameras -- not tested:
    if (is_camera("650D", "1.0.4"))
    {
        sd_setup_mode       = 0xFF334C4C;
        sd_setup_mode_in    = 0xFF334CD4;
        sd_setup_mode_reg   = 1;
        sd_set_function     = 0xFF73FD20;
        SD_ReConfiguration  = (void *) 0xFF7420D4;
    }

    if (is_camera("70D", "1.1.2"))
    {
        sd_setup_mode       = 0xFF33E078;
        sd_setup_mode_in    = 0xFF33E100;
        sd_setup_mode_reg   = 1;
        sd_set_function     = 0xFF7CE4B8;
        SD_ReConfiguration  = (void *) 0xFF7D086C;
    }
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on April 03, 2018, 08:37:14 PM
Holy sh!* , Epic!
I was right when I chose Canon over Nikon, thanks god for that  :)

Quote from: Tony Weller on April 03, 2018, 07:26:33 PM700D
2520x1072
2:35
12 bit lossless
Continuous recording  !!! @58MB/s
@Tony Weller

Amazing!
Can you test it with 720p @ 60fps with crop_rec 3x3?
The default write speed in mv720 is lower than mv1080 so let's see what will happen, I hope the camera was with me  :'(
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 03, 2018, 09:12:01 PM
@theBilalFakhouri

41.2 MB/s and 47MB/s x5 crop mode record indicator green throughout

Hope that helps, it's not a mode I use but think I did it right 1280x720 @60fps
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on April 03, 2018, 09:36:15 PM
@Tony Weller
Thanks, but I meant set 720p from canon menu and enable crop_rec after loading it, this will gives you 1736x696 @ 60fps Raw video in ML menu, and test it continuous or not  :D
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 03, 2018, 09:47:32 PM
@theBilalFakhouri

Yes I did set it from the Canon menu and I had to change to NTSC to get 60fps BUT I did have frame rate limit on to 23.976 (shall I try again with it off)?

And it gave me 1736x696 easy.

OK tried with 60 fps and got 3 seconds @1736x696 BUT the recording thingy was only stating 38MB/s maybe that mode isn't working yet?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on April 03, 2018, 10:12:12 PM
@Tony Weller

Yes it's strange, run benchmark (bench.mo) in that mode at same settings, maybe something happening or just the module dosen't support this mode right now.

@a1ex take a look  ::)
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 03, 2018, 10:24:12 PM
@theBilalFakhouri

Yeah the benchmark showing the overclocked (68MB/s) speed so something not connected to the module in 720 3x3 ? someone much more knowledgeable than me will be along soon  ;D
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on April 03, 2018, 10:36:49 PM
Nice  :D
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 03, 2018, 10:39:04 PM
Raw recording 101: for maximum speed, disable global draw, enable frozen LV.

5D3 720p60 1792x672 (same area), lossless compression 50%: 13 seconds, 44.7 MB/s (with a card that scores ~50MB/s in benchmark).

Memory bandwidth is usually the bottleneck in this mode; stopping non-essential image streams (such as preview and overlays) helps.
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 03, 2018, 10:40:22 PM

(https://thumb.ibb.co/b1B1jx/sd.png) (https://ibb.co/b1B1jx)


This image may help choosing a SD card for the new turbo overclocking-ness

I chose a V30 as it says 4k but I'm certain they mean 4k compressed ?

I was going to get a V60 but now is unnecessary as the 700 seems to be at it's limit with a bit of overhead on the SD card writes

Joy  :)
Title: Re: UHS-I / SD cards investigation
Post by: jai554 on April 04, 2018, 10:04:53 AM
Quote from: a1ex on June 18, 2017, 11:54:16 PM
Does this apply to DIGIC 4 cameras?

I'm afraid not - the hardware configuration of these cameras is different (and a lot simpler). You now know where to look, so you can play with it, attempt to change the clock speed and report your findings.

hi,
so it is still possible to make it work on digic 4 cameras?
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 04, 2018, 11:52:19 AM
I doubt - D4 uses only 2 bits (http://magiclantern.wikia.com/wiki/Register_Map#Timer.2FClock_Module) for clock speed (0xC0400004). Feel free to prove me wrong (possibly by brute-forcing the other bits).

1300D has a string that explains these 2 bits: SetSdClock %d ->0:210K 1:16MHz 2:24MHz 3:48MHz
Title: Re: UHS-I / SD cards investigation
Post by: saulbass on April 04, 2018, 03:08:25 PM
Just tried a1ex's sd_uhs.mo on my venerable 650D.
Came up with a FIXME unsupported model, but carried on...
The reported Benchmarks on a fast 95mb SD card were as follows:


(https://thumb.ibb.co/m5r5Bc/bench650_D.jpg) (https://ibb.co/m5r5Bc)


suggesting that perhaps the module isnt configured for the 650D. If any kind soul has the time to set this up for the 650D I will check it.
Title: Re: UHS-I / SD cards investigation
Post by: dfort on April 04, 2018, 05:25:47 PM
@saulbass - I posted a sd_uhs.mo that should work with your 650D on my downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/). It should also work on all the ML supported Digic 5 cameras.

I looked at the stubs for the 7D and found just one. Strange thing is that camera only has one CF slot and no SD slot yet it has "SD_ReConfiguration" -- wonder what it does?
Title: Re: UHS-I / SD cards investigation
Post by: andy kh on April 04, 2018, 06:19:04 PM
no luck on 70D
Title: Re: UHS-I / SD cards investigation
Post by: dfort on April 04, 2018, 06:27:51 PM
Hum--it should work. Make sure sd_uhs is the only active module. The 70D isn't working with lossless compresssion yet and mlv_lite defaults to 14-bit lossless.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 04, 2018, 06:33:15 PM
I'm impressed! 650D results with 128 GB Sandisk Extreme Pro
===================
2018/04/04 17:29:44
===================
Before the hack: r:43MB/s w:42MB/s  W:40MB/s R:43MB/s  :)  [best 42MB/s]
SDR50 @ 96MHz  : r:43MB/s w:40MB/s  W:42MB/s R:43MB/s  meh [best 42MB/s]
SDR50 @ 96MHz  : r:43MB/s w:42MB/s  W:41MB/s R:43MB/s  meh [best 42MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 42MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 42MB/s]
SDR50 @ 120MHz : r:54MB/s w:52MB/s  W:50MB/s R:54MB/s  :)  [best 52MB/s]
SDR50 @ 120MHz : r:54MB/s w:51MB/s  W:51MB/s R:54MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:43MB/s w:39MB/s  W:41MB/s R:43MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:43MB/s w:42MB/s  W:42MB/s R:43MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 52MB/s]
SDR104 @ 120MHz: r:54MB/s w:52MB/s  W:51MB/s R:54MB/s  :)  [best 52MB/s]
SDR104 @ 120MHz: r:54MB/s w:50MB/s  W:51MB/s R:54MB/s  meh [best 52MB/s]
SDR104 @ 132MHz: r:50MB/s w:48MB/s  W:48MB/s R:50MB/s  meh [best 52MB/s]
SDR104 @ 132MHz: r:50MB/s w:48MB/s  W:47MB/s R:50MB/s  meh [best 52MB/s]
SDR104 @ 160MHz: r:71MB/s w:64MB/s  W:68MB/s R:71MB/s  :)  [best 64MB/s]
SDR104 @ 160MHz: r:71MB/s w:69MB/s  W:65MB/s R:71MB/s  :)  [best 69MB/s]

Done.
Please run THOROUGH tests before using!!!


Same card, 100D:
===================
2018/04/04 20:30:52
===================
Before the hack: r:43MB/s w:42MB/s  W:39MB/s R:43MB/s  :)  [best 42MB/s]
SDR50 @ 96MHz  : r:43MB/s w:41MB/s  W:41MB/s R:43MB/s  meh [best 42MB/s]
SDR50 @ 96MHz  : r:43MB/s w:40MB/s  W:42MB/s R:43MB/s  meh [best 42MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 42MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 42MB/s]
SDR50 @ 120MHz : r:54MB/s w:51MB/s  W:50MB/s R:54MB/s  :)  [best 51MB/s]
SDR50 @ 120MHz : r:54MB/s w:52MB/s  W:52MB/s R:54MB/s  :)  [best 52MB/s]
SDR104 @ 96MHz : r:43MB/s w:40MB/s  W:41MB/s R:43MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:43MB/s w:41MB/s  W:42MB/s R:43MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 52MB/s]
SDR104 @ 120MHz: r:54MB/s w:51MB/s  W:52MB/s R:54MB/s  meh [best 52MB/s]
SDR104 @ 120MHz: r:54MB/s w:52MB/s  W:50MB/s R:54MB/s  meh [best 52MB/s]
SDR104 @ 132MHz: r:50MB/s w:46MB/s  W:48MB/s R:50MB/s  meh [best 52MB/s]
SDR104 @ 132MHz: r:50MB/s w:48MB/s  W:46MB/s R:50MB/s  meh [best 52MB/s]
SDR104 @ 160MHz: r:71MB/s w:66MB/s  W:67MB/s R:71MB/s  :)  [best 66MB/s]
SDR104 @ 160MHz: r:71MB/s w:66MB/s  W:66MB/s R:72MB/s  meh [best 66MB/s]

Done.
Please run THOROUGH tests before using!!!
Title: Re: UHS-I / SD cards investigation
Post by: Kharak on April 04, 2018, 06:57:43 PM
@a1ex do you think that with CF+SD recording on the 5d3, we could reach ~130 MB/s
Title: Re: UHS-I / SD cards investigation
Post by: andy kh on April 04, 2018, 07:04:00 PM
Quote from: dfort on April 04, 2018, 06:27:51 PM
Hum--it should work. Make sure sd_uhs is the only active module. The 70D isn't working with lossless compresssion yet and mlv_lite defaults to 14-bit lossless.

this is what i get. i activate only sd_uhs this time
(https://thumb.ibb.co/bH6LcH/70_D_SD_UHS.jpg) (https://ibb.co/bH6LcH)
Title: Re: UHS-I / SD cards investigation
Post by: loknar on April 04, 2018, 07:41:00 PM
Oh, WOW, this is amazing :).

On EOS M 2.02 somehow i max out at ~45mb/s,  two different Sandisk Extreme Pro cards, two different builds (second is dfort's build from yesterday).
@dfort have you got some special edition cards? :) Better stickers or something?

(https://thumb.ibb.co/gM7hrc/VRAM0.jpg) (https://ibb.co/gM7hrc)

(https://thumb.ibb.co/nO0fcH/VRAM01.jpg) (https://ibb.co/nO0fcH)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on April 04, 2018, 07:53:27 PM
@andy kh
You must run a custom build from crop_rec_4k branch, you will find it in dfort's download page https://bitbucket.org/daniel_fort/magic-lantern/downloads/
Title: Re: UHS-I / SD cards investigation
Post by: andy kh on April 04, 2018, 08:37:53 PM
Quote from: theBilalFakhouri on April 04, 2018, 07:53:27 PM
@andy kh
You must run a custom build from crop_rec_4k branch, you will find it in dfort's download page https://bitbucket.org/daniel_fort/magic-lantern/downloads/

my bad. i was using the nightly build. now it works
(https://thumb.ibb.co/ikr3Ex/70_D_sd_hs_working.jpg) (https://ibb.co/ikr3Ex)

sandisk extreme pro 64gb
Title: Re: UHS-I / SD cards investigation
Post by: dfort on April 04, 2018, 08:44:34 PM
Quote from: loknar on April 04, 2018, 07:41:00 PM
@dfort have you got some special edition cards? :) Better stickers or something?

Ha ha. Try loading the bench module and run the card benchmark test. Run it in Playback mode so the LiveView doesn't wipe out the printout. It will save an image file of the final results on your card's root directory.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 04, 2018, 08:56:42 PM
Not needed. There is a log: ML\LOGS\SD_UHS.LOG
Title: Re: UHS-I / SD cards investigation
Post by: andy kh on April 04, 2018, 09:02:40 PM
70D

===================
2018/04/04 23:44:35
===================
Before the hack: r:40MB/s w:36MB/s  W:37MB/s R:40MB/s  :)  [best 36MB/s]
SDR50 @ 96MHz  : r:40MB/s w:37MB/s  W:36MB/s R:40MB/s  :)  [best 37MB/s]
SDR50 @ 96MHz  : r:39MB/s w:37MB/s  W:37MB/s R:39MB/s  meh [best 37MB/s]
SDR50 @ 80MHz  : r:34MB/s w:32MB/s  W:32MB/s R:33MB/s  meh [best 37MB/s]
SDR50 @ 80MHz  : r:34MB/s w:32MB/s  W:32MB/s R:34MB/s  meh [best 37MB/s]
SDR50 @ 120MHz : r:48MB/s w:44MB/s  W:41MB/s R:48MB/s  :)  [best 44MB/s]
SDR50 @ 120MHz : r:48MB/s w:45MB/s  W:44MB/s R:48MB/s  :)  [best 45MB/s]
SDR104 @ 96MHz : r:40MB/s w:35MB/s  W:36MB/s R:39MB/s  meh [best 45MB/s]
SDR104 @ 96MHz : r:39MB/s w:38MB/s  W:37MB/s R:39MB/s  meh [best 45MB/s]
SDR104 @ 80MHz : r:34MB/s w:32MB/s  W:31MB/s R:34MB/s  meh [best 45MB/s]
SDR104 @ 80MHz : r:34MB/s w:31MB/s  W:32MB/s R:34MB/s  meh [best 45MB/s]
SDR104 @ 120MHz: r:48MB/s w:44MB/s  W:43MB/s R:48MB/s  meh [best 45MB/s]
SDR104 @ 120MHz: r:48MB/s w:44MB/s  W:43MB/s R:46MB/s  meh [best 45MB/s]
SDR104 @ 132MHz: r:52MB/s w:46MB/s  W:46MB/s R:53MB/s  :)  [best 46MB/s]
SDR104 @ 132MHz: r:52MB/s w:48MB/s  W:48MB/s R:52MB/s  :)  [best 48MB/s]
SDR104 @ 160MHz: r:61MB/s w:52MB/s  W:52MB/s R:61MB/s  :)  [best 52MB/s]
SDR104 @ 160MHz: r:61MB/s w:55MB/s  W:54MB/s R:62MB/s  :)  [best 55MB/s]

Done.
Please run THOROUGH tests before using!!!

sandisk extreme pro 64gb

Title: Re: UHS-I / SD cards investigation
Post by: andy kh on April 04, 2018, 11:05:48 PM

===================
2018/04/05 02:17:38
===================
Before the hack: r:40MB/s w:18MB/s  W:32MB/s R:39MB/s  :)  [best 18MB/s]
SDR50 @ 96MHz  : r:39MB/s w:18MB/s  W:31MB/s R:39MB/s  :)  [best 18MB/s]
SDR50 @ 96MHz  : r:40MB/s w:19MB/s  W:35MB/s R:39MB/s  :)  [best 19MB/s]
SDR50 @ 80MHz  : r:34MB/s w:30MB/s  W:30MB/s R:34MB/s  :)  [best 30MB/s]
SDR50 @ 80MHz  : r:33MB/s w:31MB/s  W:30MB/s R:34MB/s  :)  [best 31MB/s]
SDR50 @ 120MHz : r:48MB/s w:42MB/s  W:41MB/s R:48MB/s  :)  [best 42MB/s]
SDR50 @ 120MHz : r:48MB/s w:42MB/s  W:37MB/s R:47MB/s  :)  [best 42MB/s]
SDR104 @ 96MHz : r:39MB/s w:34MB/s  W:20MB/s R:39MB/s  meh [best 42MB/s]
SDR104 @ 96MHz : r:39MB/s w:20MB/s  W:18MB/s R:39MB/s  meh [best 42MB/s]
SDR104 @ 80MHz : r:33MB/s w:18MB/s  W:31MB/s R:34MB/s  meh [best 42MB/s]
SDR104 @ 80MHz : r:34MB/s w:30MB/s  W:30MB/s R:34MB/s  meh [best 42MB/s]
SDR104 @ 120MHz: r:47MB/s w:42MB/s  W:41MB/s R:47MB/s  meh [best 42MB/s]
SDR104 @ 120MHz: r:48MB/s w:41MB/s  W:42MB/s R:48MB/s  meh [best 42MB/s]
SDR104 @ 132MHz: r:52MB/s w:44MB/s  W:44MB/s R:51MB/s  :)  [best 44MB/s]
SDR104 @ 132MHz: r:52MB/s w:44MB/s  W:44MB/s R:52MB/s  :)  [best 44MB/s]
SDR104 @ 160MHz: r:59MB/s w:46MB/s  W:47MB/s R:60MB/s  :)  [best 46MB/s]
SDR104 @ 160MHz: r:60MB/s w:48MB/s  W:46MB/s R:59MB/s  :)  [best 48MB/s]

Done.
Please run THOROUGH tests before using!!!

sony sdhc 94mb/s 32gb
https://www.amazon.in/Sony-SDHC-UHS-I-32GB-Memory/dp/B0081R14LI?tag=googinhydr18418-21&tag=googinkenshoo-21&ascsubtag=60b8e748-bc59-42ca-b386-1d3a558c323d
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 04, 2018, 11:14:06 PM
There are/were compatibility issues with this card type in some Canon cams. See http://www.cameramemoryspeed.com/reviews/sd-cards/sony-32gb-sdhc-memory-card/
18 MByte/s vs. 48 MByte/s? Wow!
Title: Re: UHS-I / SD cards investigation
Post by: domasa on April 04, 2018, 11:56:27 PM
QuoteRan a brute force (random) search for the above registers, and...

...what about brute force search registers for CompactFlash?  ;)
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 04, 2018, 11:56:51 PM
Updated sd_uhs.mo (same link (https://www.magiclantern.fm/forum/index.php?topic=12862.msg199224#msg199224)) with support for EOS M, 100D, 650D and 70D (thanks dfort), some minor fixes and a new experiment.

Noticed one of my cards was a bit unstable in LiveView when using the 160MHz mode (sometimes worked, sometimes the tests resulted in error) and managed to get consistent results after changing driver strength (default is 0, got better results with 1). The new module tries all possible values (0 to 3).

Committed the source (https://bitbucket.org/hudson/magic-lantern/branch/sd_uhs), too.

@Levas: please retry the test on the same (old) card.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 05, 2018, 12:14:28 AM
Sandisk Extreme Pro, 128 GB.
100D and 650D in photo mode, only sd_uhs.mo loaded.
===================
2018/04/05 00:31:02
===================
Before the hack: r:43MB/s w:42MB/s  W:35MB/s R:43MB/s  8)  [best 42MB/s]
SDR50 @ 96MHz  : r:43MB/s w:39MB/s  W:40MB/s R:43MB/s  ::) [best 42MB/s]
SDR50 @ 96MHz  : r:43MB/s w:41MB/s  W:42MB/s R:43MB/s  ::) [best 42MB/s]
SDR50 @ 80MHz  : r:36MB/s w:33MB/s  W:35MB/s R:36MB/s  meh [best 42MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 42MB/s]
SDR50 @ 120MHz : r:54MB/s w:48MB/s  W:51MB/s R:54MB/s  8)  [best 48MB/s]
SDR50 @ 120MHz : r:54MB/s w:48MB/s  W:50MB/s R:54MB/s  ::) [best 48MB/s]
SDR104 @ 96MHz : D0 D0 r:43MB/s w:41MB/s  W:42MB/s R:43MB/s  meh [best 48MB/s]
SDR104 @ 96MHz : D1 D1 r:43MB/s w:39MB/s  W:41MB/s R:43MB/s  meh [best 48MB/s]
SDR104 @ 80MHz : D0 D0 r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 48MB/s]
SDR104 @ 80MHz : D1 D1 r:36MB/s w:35MB/s  W:34MB/s R:36MB/s  meh [best 48MB/s]
SDR104 @ 120MHz: D0 D0 r:54MB/s w:47MB/s  W:52MB/s R:54MB/s  ::) [best 48MB/s]
SDR104 @ 120MHz: D1 D1 r:54MB/s w:51MB/s  W:51MB/s R:54MB/s  :)  [best 51MB/s]
SDR104 @ 132MHz: D0 D0 r:50MB/s w:45MB/s  W:47MB/s R:50MB/s  meh [best 51MB/s]
SDR104 @ 132MHz: D1 D1 r:50MB/s w:47MB/s  W:48MB/s R:50MB/s  ::) [best 51MB/s]
SDR104 @ 160MHz: D0 D0 r:72MB/s w:66MB/s  W:62MB/s R:71MB/s  8)  [best 66MB/s]
SDR104 @ 160MHz: D1 D1 r:71MB/s w:60MB/s  W:67MB/s R:71MB/s  meh [best 66MB/s]
SDR104 @ 160MHz: D2 D2 r:71MB/s w:67MB/s  W:67MB/s R:71MB/s  :)  [best 67MB/s]
SDR104 @ 160MHz: D3 D3 r:71MB/s w:62MB/s  W:68MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D0 D0 r:71MB/s w:65MB/s  W:63MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D1 D1 r:72MB/s w:61MB/s  W:68MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D2 D2 r:71MB/s w:66MB/s  W:61MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D3 D3 r:71MB/s w:66MB/s  W:67MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D0 D0 r:71MB/s w:62MB/s  W:65MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D1 D1 r:71MB/s w:67MB/s  W:67MB/s R:71MB/s  :)  [best 67MB/s]
SDR104 @ 160MHz: D2 D2 r:71MB/s w:66MB/s  W:68MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D3 D3 r:71MB/s w:61MB/s  W:65MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D0 D0 r:72MB/s w:61MB/s  W:67MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D1 D1 r:72MB/s w:62MB/s  W:65MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D2 D2 r:71MB/s w:66MB/s  W:66MB/s R:71MB/s  ::) [best 67MB/s]
SDR104 @ 160MHz: D3 D3 r:71MB/s w:66MB/s  W:61MB/s R:72MB/s  ::) [best 67MB/s]
Best: D1 D1 r:71MB/s w:62MB/s  W:68MB/s R:72MB/s  ::) [best 67MB/s]
Best: D1 D1 r:71MB/s w:61MB/s  W:67MB/s R:72MB/s  ::) [best 67MB/s]

Done.
Please run THOROUGH tests before using!!!

===================
2018/04/04 23:12:54
===================
Before the hack: r:43MB/s w:42MB/s  W:37MB/s R:43MB/s  8)  [best 42MB/s]
SDR50 @ 96MHz  : r:43MB/s w:42MB/s  W:39MB/s R:43MB/s  ::) [best 42MB/s]
SDR50 @ 96MHz  : r:43MB/s w:41MB/s  W:42MB/s R:43MB/s  ::) [best 42MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 42MB/s]
SDR50 @ 80MHz  : r:36MB/s w:33MB/s  W:34MB/s R:36MB/s  meh [best 42MB/s]
SDR50 @ 120MHz : r:54MB/s w:51MB/s  W:51MB/s R:54MB/s  8)  [best 51MB/s]
SDR50 @ 120MHz : r:54MB/s w:48MB/s  W:49MB/s R:54MB/s  ::) [best 51MB/s]
SDR104 @ 96MHz : D0 D0 r:43MB/s w:41MB/s  W:42MB/s R:43MB/s  meh [best 51MB/s]
SDR104 @ 96MHz : D1 D1 r:43MB/s w:41MB/s  W:42MB/s R:43MB/s  meh [best 51MB/s]
SDR104 @ 80MHz : D0 D0 r:36MB/s w:33MB/s  W:34MB/s R:36MB/s  meh [best 51MB/s]
SDR104 @ 80MHz : D1 D1 r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 51MB/s]
SDR104 @ 120MHz: D0 D0 r:54MB/s w:51MB/s  W:48MB/s R:54MB/s  ::) [best 51MB/s]
SDR104 @ 120MHz: D1 D1 r:54MB/s w:47MB/s  W:51MB/s R:54MB/s  ::) [best 51MB/s]
SDR104 @ 132MHz: D0 D0 r:50MB/s w:47MB/s  W:48MB/s R:50MB/s  ::) [best 51MB/s]
SDR104 @ 132MHz: D1 D1 r:50MB/s w:45MB/s  W:46MB/s R:50MB/s  meh [best 51MB/s]
SDR104 @ 160MHz: D0 D0 r:72MB/s w:66MB/s  W:68MB/s R:72MB/s  8)  [best 66MB/s]
SDR104 @ 160MHz: D1 D1 r:72MB/s w:66MB/s  W:63MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D2 D2 r:72MB/s w:61MB/s  W:66MB/s R:70MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D3 D3 r:71MB/s w:66MB/s  W:67MB/s R:71MB/s  :)  [best 66MB/s]
SDR104 @ 160MHz: D0 D0 r:72MB/s w:61MB/s  W:66MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D1 D1 r:72MB/s w:62MB/s  W:65MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D2 D2 r:72MB/s w:61MB/s  W:68MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D3 D3 r:71MB/s w:66MB/s  W:63MB/s R:72MB/s  :)  [best 66MB/s]
SDR104 @ 160MHz: D0 D0 r:72MB/s w:66MB/s  W:67MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D1 D1 r:72MB/s w:62MB/s  W:65MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D2 D2 r:71MB/s w:61MB/s  W:67MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D3 D3 r:71MB/s w:66MB/s  W:68MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D0 D0 r:71MB/s w:61MB/s  W:64MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D1 D1 r:71MB/s w:66MB/s  W:68MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D2 D2 r:72MB/s w:62MB/s  W:65MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D3 D3 r:72MB/s w:66MB/s  W:63MB/s R:71MB/s  ::) [best 66MB/s]
Best: D3 D3 r:72MB/s w:66MB/s  W:62MB/s R:71MB/s  ::) [best 66MB/s]
Best: D3 D3 r:72MB/s w:61MB/s  W:67MB/s R:72MB/s  ::) [best 66MB/s]

Done.
Please run THOROUGH tests before using!!!
Title: Re: UHS-I / SD cards investigation
Post by: saulbass on April 05, 2018, 02:29:01 AM

Similar results to Walter - just a smidge slower! - but nearly double the old speed of 35Mbs.

Amazing find a1ex
and thanks to dfort too.

SanDisk 95Mbs Extreme Pro 32Gb

Can someone explain if I can now use these speeds to up the record performance on my 650D?
Is this already working?



===================
2018/04/05 02:10:21
===================
Before the hack: r:43MB/s w:41MB/s  W:36MB/s R:42MB/s  8)  [best 41MB/s]
SDR50 @ 96MHz  : r:43MB/s w:41MB/s  W:40MB/s R:43MB/s  ::) [best 41MB/s]
SDR50 @ 96MHz  : r:43MB/s w:41MB/s  W:39MB/s R:44MB/s  ::) [best 41MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:34MB/s R:36MB/s  meh [best 41MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:34MB/s R:36MB/s  meh [best 41MB/s]
SDR50 @ 120MHz : r:53MB/s w:50MB/s  W:36MB/s R:54MB/s  8)  [best 50MB/s]
SDR50 @ 120MHz : r:53MB/s w:50MB/s  W:48MB/s R:54MB/s  :)  [best 50MB/s]
SDR104 @ 96MHz : D0 D0 r:43MB/s w:41MB/s  W:41MB/s R:44MB/s  meh [best 50MB/s]
SDR104 @ 96MHz : D1 D1 r:43MB/s w:41MB/s  W:40MB/s R:43MB/s  meh [best 50MB/s]
SDR104 @ 80MHz : D0 D0 r:36MB/s w:35MB/s  W:34MB/s R:36MB/s  meh [best 50MB/s]
SDR104 @ 80MHz : D1 D1 r:36MB/s w:35MB/s  W:33MB/s R:36MB/s  meh [best 50MB/s]
SDR104 @ 120MHz: D0 D0 r:52MB/s w:50MB/s  W:49MB/s R:54MB/s  ::) [best 50MB/s]
SDR104 @ 120MHz: D1 D1 r:53MB/s w:50MB/s  W:49MB/s R:54MB/s  :)  [best 50MB/s]
SDR104 @ 132MHz: D0 D0 r:49MB/s w:47MB/s  W:30MB/s R:49MB/s  ::) [best 50MB/s]
SDR104 @ 132MHz: D1 D1 r:49MB/s w:47MB/s  W:44MB/s R:50MB/s  ::) [best 50MB/s]
SDR104 @ 160MHz: D0 D0 r:69MB/s w:65MB/s  W:65MB/s R:72MB/s  8)  [best 65MB/s]
SDR104 @ 160MHz: D1 D1 r:69MB/s w:65MB/s  W:64MB/s R:72MB/s  :)  [best 65MB/s]
SDR104 @ 160MHz: D2 D2 r:69MB/s w:66MB/s  W:63MB/s R:72MB/s  :)  [best 66MB/s]
SDR104 @ 160MHz: D3 D3 r:69MB/s w:65MB/s  W:61MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D0 D0 r:69MB/s w:65MB/s  W:63MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D1 D1 r:69MB/s w:65MB/s  W:63MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D2 D2 r:69MB/s w:65MB/s  W:41MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D3 D3 r:69MB/s w:65MB/s  W:64MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D0 D0 r:69MB/s w:65MB/s  W:64MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D1 D1 r:69MB/s w:65MB/s  W:63MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D2 D2 r:69MB/s w:65MB/s  W:60MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D3 D3 r:69MB/s w:65MB/s  W:63MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D0 D0 r:70MB/s w:66MB/s  W:62MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D1 D1 r:69MB/s w:65MB/s  W:63MB/s R:72MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D2 D2 r:70MB/s w:65MB/s  W:41MB/s R:71MB/s  ::) [best 66MB/s]
SDR104 @ 160MHz: D3 D3 r:69MB/s w:65MB/s  W:63MB/s R:72MB/s  ::) [best 66MB/s]
Best: D2 D2 r:69MB/s w:65MB/s  W:63MB/s R:72MB/s  ::) [best 66MB/s]
Best: D2 D2 r:70MB/s w:65MB/s  W:61MB/s R:71MB/s  ::) [best 66MB/s]

Done.
Please run THOROUGH tests before using!!!



Title: Re: UHS-I / SD cards investigation
Post by: saulbass on April 05, 2018, 02:39:06 AM
Using magiclantern-crop_rec_4k.2018Mar10.650D104 & the most recent sd_uhs.mo

I think I am getting the same level of performance as before the recent sd_uhs.mo update.

1736x976 14bit lossless - 365 frames before it crashes out

2520x1078 14bit lossless - 102 frames before it crashes out

but at least things still working.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on April 05, 2018, 09:06:30 AM
After overclocking the write speed, is this mean the H.264 proxy will becomes more stable? And it will work fine soon if this depending on write speed?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 05, 2018, 09:46:45 AM
Doesn't help for my card, also not sure if the test works flawlessly:
(https://farm1.staticflickr.com/801/26375418167_317ebbdfa5_b.jpg)

Whole test goes normal, only with SDR104 @ 160 Mhz it shows the first line normal, with the D0's, it's slower than best, so I expect a, meh as comment, but it gives ? ? ? as comment.
And after that it goes on with the test, without showing the D settings ???

Already ordered a brand new shiny Sandisk Extreme Pro card, according to their website it should do 90Mb/s write speed, so in case the 208Mhz settings are found, I'm good  :D
The card I'm using here is a Sandisk Extreme from 2013, the current Extreme line up does 60Mb/s write speed, but I guess the 2013 Extreme line up does 45Mb/s.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 05, 2018, 10:49:38 AM
Indeed, D settings not showing is a bad sign. Can you enable the printf from sd_set_function_log? (replace qprintf with log_printf). The former only goes to QEMU console, the latter goes to the screen and in the log file, too. Attach the full log file (from ML/LOGS).
Title: Re: UHS-I / SD cards investigation
Post by: Kharak on April 05, 2018, 11:36:53 AM
Just wondering if this is potentially damaging to the SD controller, this is messing with hardware controllers, right? I mean doubling hz, is it not causing more heat?
Title: Re: UHS-I / SD cards investigation
Post by: Markus on April 05, 2018, 12:27:49 PM
Just tried this with Sandisk extreme Pro SD 512GB rated at 95/mb sec but I'm just getting "malloc error". The camera can see the 512Gb card fine but seems like the module cant handle the big card? Any ideas? Also noticed that the camera could not format the card even though it can see it.

5D3 FW 123

Would be really cool to get this working with this card =).
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 05, 2018, 01:13:47 PM
... turn off raw video.
Title: Re: UHS-I / SD cards investigation
Post by: Markus on April 05, 2018, 01:25:00 PM
Ok, that fixed it.
write 35mb / read 39 Best
Not much luck on higher speeds. Any idea why?

5D3 fw123 "4K raw video recording; lossless compression" version of ML

CrystalDiskMark bench for reference:
Seq Read: 81.47MB/s Write 77.89MB/s

===================
2018/04/05 12:15:25
===================
Before the hack: r:20MB/s w:20MB/s  W:20MB/s R:21MB/s  8)  [best 20MB/s]
SDR50 @ 96MHz  : r:35MB/s w:34MB/s  W:35MB/s R:38MB/s  8)  [best 34MB/s]
SDR50 @ 96MHz  : r:35MB/s w:35MB/s  W:35MB/s R:38MB/s  :)  [best 35MB/s]
SDR50 @ 80MHz  : r:31MB/s w:30MB/s  W:30MB/s R:33MB/s  meh [best 35MB/s]
SDR50 @ 80MHz  : r:30MB/s w:30MB/s  W:30MB/s R:33MB/s  meh [best 35MB/s]
SDR50 @ 120MHz : r:err [SAFE]  [BACK]
SDR50 @ 120MHz : r:err [SAFE]  [BACK]
SDR104 @ 96MHz : D0 D0 r:35MB/s w:34MB/s  W:35MB/s R:38MB/s  ::) [best 35MB/s]
SDR104 @ 96MHz : D1 D1 r:34MB/s w:34MB/s  W:35MB/s R:39MB/s  ::) [best 35MB/s]
SDR104 @ 80MHz : D0 D0 r:30MB/s w:30MB/s  W:30MB/s R:33MB/s  meh [best 35MB/s]
SDR104 @ 80MHz : D1 D1 r:30MB/s w:30MB/s  W:30MB/s R:33MB/s  meh [best 35MB/s]
SDR104 @ 120MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 120MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 132MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 132MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 160MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D2 D2 r:err [SAFE] D0  [BACK] D2
SDR104 @ 160MHz: D3 D3 r:err [SAFE] D0  [BACK] D3
SDR104 @ 160MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 160MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D2 D2 r:err [SAFE] D0  [BACK] D2
SDR104 @ 160MHz: D3 D3 r:err [SAFE] D0  [BACK] D3
SDR104 @ 160MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 160MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D2 D2 r:err [SAFE] D0  [BACK] D2
SDR104 @ 160MHz: D3 D3 r:err [SAFE] D0  [BACK] D3
SDR104 @ 160MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 160MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D2 D2 r:err [SAFE] D0  [BACK] D2
SDR104 @ 160MHz: D3 D3 r:err [SAFE] D0  [BACK] D3

Best: D0 D0 r:35MB/s w:34MB/s  W:34MB/s R:38MB/s  ::) [best 35MB/s]
Best: D0 D0 r:35MB/s w:34MB/s  W:35MB/s R:39MB/s  ::) [best 35MB/s]

-----------------------------------------------------------------------
CrystalDiskMark 5.1.0 x64
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

Sequential Read (Q= 32,T= 1) :    81.474 MB/s
Sequential Write (Q= 32,T= 1) :    77.886 MB/s

(https://encrypted-tbn1.gstatic.com/shopping?q=tbn:ANd9GcSxfug0O3GvOTuRMNYJBIdCVNcZorbpLY6xa6BBCAOfWatVNGfMpF3RMCBRb1yFgvEqf44yWgzz&usqp=CAc)
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 05, 2018, 03:47:25 PM
Hm, for some reason, this card doesn't accept the overclocked settings at all. Wonder if it's the power limit (I did not change it), sampling point (no idea how to adjust it, but at least I was able to call CMD19) or something else. You could try on 1.1.3, but I don't expect improvements.

Does this happen on smaller cards from the same series?




edit: here's something you can try:

Quote from: a1ex on April 03, 2018, 07:10:06 PM
Press shutter halfway during these tests (until it starts running some more), then let it run overnight on the power adapter, then press shutter halfway again to stop.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 05, 2018, 03:54:23 PM
32 GB in 650D (saulbass) and 128 GB in 650D/100D (mine) working. About 65-67 MByte/s (write).

Don't know if controller type matches with 512 GB variety. Took Sandisk some time to force that much cells into SD format and haven't found reliable info about their controllers.
Markus' card seems to give a bit low numbers with CrystalDiskMark. May be caused by PC or cardreader used, though.
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 05, 2018, 04:00:58 PM
Quote from: a1ex on April 05, 2018, 03:47:25 PM

Does this happen on smaller cards from the same series?


(https://thumb.ibb.co/fUsjNH/sandisk.png) (https://ibb.co/fUsjNH)


No ! might be something to do with those 'V' settings I posted earlier.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 05, 2018, 04:03:15 PM
Those V "settings" are just labels.
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 05, 2018, 04:06:17 PM
Quote from: Walter Schulz on April 05, 2018, 04:03:15 PM
Those V "settings" are just labels.

OK, same card then apart from the size obviously.
If there is any test I could run to find the difference I'm happy to do so.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 05, 2018, 04:10:36 PM
Quote from: Tony Weller on April 05, 2018, 04:06:17 PM
OK, same card then apart from the size obviously.

Nothing is obvious in SD-card business. See Lexar Professional 800x (CF-card):
32 GB: About 46 MByte/s (write)
64 GB: About 78 MByte/s (write)
Same card type, different controllers -> different.
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 05, 2018, 04:15:35 PM
OK, was just answering Alex1s question...

Thanks for the info.
Title: Re: UHS-I / SD cards investigation
Post by: Markus on April 05, 2018, 04:44:06 PM
I'll Downgrade to 113 test it there.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 05, 2018, 04:45:36 PM
Quote from: a1ex on April 05, 2018, 10:49:38 AM
Indeed, D settings not showing is a bad sign. Can you enable the printf from sd_set_function_log? (replace qprintf with log_printf). The former only goes to QEMU console, the latter goes to the screen and in the log file, too. Attach the full log file (from ML/LOGS).

So, finally made it, can't get used to cloning magic lantern and stuff...
Made the changes and run it, the log file:

===================
2018/04/05 15:36:04
===================
Before the hack: r:42MB/s w:40MB/s  W:40MB/s R:44MB/s  8)  [best 40MB/s]
SDR50 @ 96MHz  : r:42MB/s w:24MB/s  W:39MB/s R:44MB/s  meh [best 40MB/s]
SDR50 @ 96MHz  : r:42MB/s w:40MB/s  W:39MB/s R:43MB/s  :)  [best 40MB/s]
SDR50 @ 80MHz  : r:35MB/s w:34MB/s  W:34MB/s R:36MB/s  meh [best 40MB/s]
SDR50 @ 80MHz  : r:35MB/s w:34MB/s  W:34MB/s R:36MB/s  meh [best 40MB/s]
SDR50 @ 120MHz : r:52MB/s w:44MB/s  W:43MB/s R:54MB/s  :)  [best 44MB/s]
SDR50 @ 120MHz : r:52MB/s w:43MB/s  W:43MB/s R:54MB/s  ::) [best 44MB/s]
SDR104 @ 96MHz : sd_set_function(0xff0002)
D0 sd_set_function(0xff0002)
D0 r:42MB/s w:40MB/s  W:40MB/s R:43MB/s  ::) [best 44MB/s]
SDR104 @ 96MHz : sd_set_function(0xff0002)
D1 sd_set_function(0xff0002)
D1 r:42MB/s w:40MB/s  W:40MB/s R:43MB/s  ::) [best 44MB/s]
SDR104 @ 80MHz : sd_set_function(0xff0002)
D0 sd_set_function(0xff0002)
D0 r:35MB/s w:34MB/s  W:34MB/s R:36MB/s  meh [best 44MB/s]
SDR104 @ 80MHz : sd_set_function(0xff0002)
D1 sd_set_function(0xff0002)
D1 r:36MB/s w:34MB/s  W:34MB/s R:36MB/s  meh [best 44MB/s]
SDR104 @ 120MHz: sd_set_function(0xff0002)
D0 sd_set_function(0xff0002)
D0 r:51MB/s w:44MB/s  W:43MB/s R:54MB/s  :)  [best 44MB/s]
SDR104 @ 120MHz: sd_set_function(0xff0002)
D1 sd_set_function(0xff0002)
D1 r:52MB/s w:44MB/s  W:43MB/s R:54MB/s  ::) [best 44MB/s]
SDR104 @ 132MHz: sd_set_function(0xff0002)
D0 sd_set_function(0xff0002)
D0 r:57MB/s w:44MB/s  W:43MB/s R:59MB/s  ::) [best 44MB/s]
SDR104 @ 132MHz: sd_set_function(0xff0002)
D1 sd_set_function(0xff0002)
D1 r:57MB/s w:44MB/s  W:43MB/s R:60MB/s  ::) [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xff0002)
D0 sd_set_function(0xff0002)
D0 r:58MB/s w:sd_set_function(0xffff01)
15MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:20MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:15MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:20MB/s w:21MB/s  W:20MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:20MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:14MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:21MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
SDR104 @ 160MHz: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:22MB/s w:21MB/s  W:20MB/s R:22MB/s  ??? [best 44MB/s]
Best: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:22MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 44MB/s]
Best: sd_set_function(0xffff01)
sd_set_function(0xffff01)
r:22MB/s w:21MB/s  W:20MB/s R:22MB/s  ??? [best 44MB/s]

Done.
Please run THOROUGH tests before using!!!
Title: Re: UHS-I / SD cards investigation
Post by: saulbass on April 05, 2018, 04:47:21 PM

Following on from Tony Weller - this is my card - fwiw I've had it quite a few years now - originally from Amazon - I think at one time there was a problem with re-badged counterfeit cards being sold - not sure how you can check though.

Also wondering if your card needs to be formatted as an Ex Fat? Mine is.


(https://thumb.ibb.co/gdt8hH/IMG_4757.jpg) (https://ibb.co/gdt8hH)

Title: Re: UHS-I / SD cards investigation
Post by: Markus on April 05, 2018, 06:12:24 PM
No more luck with the 512Gb Sandisk Extreme Pro with fw 113.
Same errors.  :'(

===================
2018/04/05 17:02:51
===================
Before the hack: r:20MB/s w:20MB/s  W:20MB/s R:21MB/s  8)  [best 20MB/s]
SDR50 @ 96MHz  : r:36MB/s w:35MB/s  W:34MB/s R:39MB/s  8)  [best 35MB/s]
SDR50 @ 96MHz  : r:35MB/s w:35MB/s  W:35MB/s R:39MB/s  :)  [best 35MB/s]
SDR50 @ 80MHz  : r:31MB/s w:31MB/s  W:30MB/s R:34MB/s  meh [best 35MB/s]
SDR50 @ 80MHz  : r:31MB/s w:31MB/s  W:30MB/s R:33MB/s  meh [best 35MB/s]
SDR50 @ 120MHz : r:err [SAFE]  [BACK]
SDR50 @ 120MHz : r:err [SAFE]  [BACK]
SDR104 @ 96MHz : D0 D0 r:36MB/s w:35MB/s  W:35MB/s R:39MB/s  ::) [best 35MB/s]
SDR104 @ 96MHz : D1 D1 r:36MB/s w:35MB/s  W:35MB/s R:39MB/s  ::) [best 35MB/s]
SDR104 @ 80MHz : D0 D0 r:31MB/s w:31MB/s  W:30MB/s R:33MB/s  meh [best 35MB/s]
SDR104 @ 80MHz : D1 D1 r:31MB/s w:30MB/s  W:29MB/s R:34MB/s  meh [best 35MB/s]
SDR104 @ 120MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 120MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 132MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 132MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 160MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D2 D2 r:err [SAFE] D0  [BACK] D2
SDR104 @ 160MHz: D3 D3 r:err [SAFE] D0  [BACK] D3
SDR104 @ 160MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 160MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D2 D2 r:err [SAFE] D0  [BACK] D2
SDR104 @ 160MHz: D3 D3 r:err [SAFE] D0  [BACK] D3
SDR104 @ 160MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 160MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D2 D2 r:err [SAFE] D0  [BACK] D2
SDR104 @ 160MHz: D3 D3 r:err [SAFE] D0  [BACK] D3
SDR104 @ 160MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
SDR104 @ 160MHz: D1 D1 r:err [SAFE] D0  [BACK] D1
SDR104 @ 160MHz: D2 D2 r:err [SAFE] D0  [BACK] D2
SDR104 @ 160MHz: D3 D3 r:err [SAFE] D0  [BACK] D3
Best: D0 D0 r:36MB/s w:36MB/s  W:35MB/s R:39MB/s  :)  [best 36MB/s]
Best: D0 D0 r:36MB/s w:33MB/s  W:35MB/s R:38MB/s  ::) [best 36MB/s]
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 05, 2018, 06:55:40 PM
@Markus
Are you running the test in photo mode as when I tried it in video mode I got loads of crazy stuff coming up.

Just a thought..
Title: Re: UHS-I / SD cards investigation
Post by: Markus on April 05, 2018, 07:16:04 PM
Quote@Markus
Are you running the test in photo mode as when I tried it in video mode I got loads of crazy stuff coming up.

Just a thought..

I got a little bit faster results on the modes that worked but error on the same ones as when I tested in video-mode.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 05, 2018, 07:48:47 PM
Are you able to test the same card on another DIGIC 5 model? (anything other than 5D3).

Also, brute-forcing (via half-shutter) might be useful. Even without a power adapter, after an hour or so I'd expect it to find something and give some more clues. Run it in play or photo mode to avoid the additional load (LiveView is a lot heavier on CPU and RAM).
Title: Re: UHS-I / SD cards investigation
Post by: Markus on April 05, 2018, 08:54:03 PM
I can try that tomorrow. Would be awsome to get it to work on this card.  8) :D
Title: Re: UHS-I / SD cards investigation
Post by: andy kh on April 05, 2018, 09:43:33 PM
70D
test with the updated sd_uhs.mo
sandisk extreme pro 64 gb remains the same with dfort's build but i can see difference with the sony card 94mb/s 32 gb card

below is the test with sony card
===================
2018/04/06 00:52:28
===================
Before the hack: r:39MB/s w:25MB/s  W:28MB/s R:40MB/s  8)  [best 25MB/s]
SDR50 @ 96MHz  : r:40MB/s w:18MB/s  W:34MB/s R:40MB/s  meh [best 25MB/s]
SDR50 @ 96MHz  : r:40MB/s w:19MB/s  W:30MB/s R:39MB/s  meh [best 25MB/s]
SDR50 @ 80MHz  : r:34MB/s w:13MB/s  W:27MB/s R:34MB/s  meh [best 25MB/s]
SDR50 @ 80MHz  : r:34MB/s w:17MB/s  W:12MB/s R:34MB/s  meh [best 25MB/s]
SDR50 @ 120MHz : r:47MB/s w:13MB/s  W:35MB/s R:47MB/s  meh [best 25MB/s]
SDR50 @ 120MHz : r:48MB/s w:20MB/s  W:18MB/s R:47MB/s  meh [best 25MB/s]
SDR104 @ 96MHz : D0 D0 r:39MB/s w:35MB/s  W:35MB/s R:39MB/s  8)  [best 35MB/s]
SDR104 @ 96MHz : D1 D1 r:39MB/s w:19MB/s  W:29MB/s R:39MB/s  meh [best 35MB/s]
SDR104 @ 80MHz : D0 D0 r:34MB/s w:17MB/s  W:26MB/s R:34MB/s  ??? [best 35MB/s]
SDR104 @ 80MHz : D1 D1 r:34MB/s w:17MB/s  W:28MB/s R:33MB/s  ??? [best 35MB/s]
SDR104 @ 120MHz: D0 D0 r:48MB/s w:20MB/s  W:22MB/s R:48MB/s  meh [best 35MB/s]
SDR104 @ 120MHz: D1 D1 r:48MB/s w:28MB/s  W:39MB/s R:47MB/s  meh [best 35MB/s]
SDR104 @ 132MHz: D0 D0 r:52MB/s w:8.3MB/s  W:20MB/s R:51MB/s  ??? [best 35MB/s]
SDR104 @ 132MHz: D1 D1 r:52MB/s w:11MB/s  W:21MB/s R:47MB/s  ??? [best 35MB/s]
SDR104 @ 160MHz: D0 D0 r:60MB/s w:21MB/s  W:46MB/s R:60MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D1 D1 r:61MB/s w:21MB/s  W:37MB/s R:60MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D2 D2 r:60MB/s w:22MB/s  W:26MB/s R:60MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D3 D3 r:60MB/s w:18MB/s  W:20MB/s R:60MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D0 D0 r:61MB/s w:12MB/s  W:15MB/s R:60MB/s  ??? [best 35MB/s]
SDR104 @ 160MHz: D1 D1 r:60MB/s w:22MB/s  W:14MB/s R:61MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D2 D2 r:61MB/s w:13MB/s  W:41MB/s R:61MB/s  ??? [best 35MB/s]
SDR104 @ 160MHz: D3 D3 r:52MB/s w:20MB/s  W:17MB/s R:61MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D0 D0 r:61MB/s w:18MB/s  W:40MB/s R:60MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D1 D1 r:61MB/s w:22MB/s  W:26MB/s R:59MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D2 D2 r:61MB/s w:31MB/s  W:28MB/s R:60MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D3 D3 r:60MB/s w:18MB/s  W:17MB/s R:60MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D0 D0 r:60MB/s w:21MB/s  W:13MB/s R:61MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D1 D1 r:60MB/s w:22MB/s  W:41MB/s R:60MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D2 D2 r:60MB/s w:21MB/s  W:26MB/s R:61MB/s  meh [best 35MB/s]
SDR104 @ 160MHz: D3 D3 r:61MB/s w:6.6MB/s  W:26MB/s R:61MB/s  ??? [best 35MB/s]
Best: D0 D0 r:40MB/s w:11MB/s  W:11MB/s R:39MB/s  ??? [best 35MB/s]
Best: D0 D0 r:39MB/s w:18MB/s  W:18MB/s R:39MB/s  meh [best 35MB/s]

Done.
Please run THOROUGH tests before using!!!


Title: Re: UHS-I / SD cards investigation
Post by: Hans_Punk on April 06, 2018, 02:43:42 AM

(https://thumb.ibb.co/m3zXSH/IMG_3412.jpg) (https://ibb.co/m3zXSH)




Benchmark from brand new SanDisk Extreme Pro 64GB 95 MB/s card - 3849.6 MB/s Write, 2836.5 MB/s Read!
Benchmark was done in a blink of an eye!!!

(After sd_uhs.mo was run in photo mode with only Benchmark and sd_uhs.mo loaded within 5D3 fw123 "4K raw video recording; lossless compression" version of ML).


But...

Tried a test recording out of curiosity but was unable to...the camera displayed 'Card Full' message and refused to initiate recording.

After quickly realising that something was not right (or probably healthy for the card), I tried a clean install procedure (format SD card to Exfat, clean install of Crop 4k 123 5D3 build with SD_uhs.mo in module root). As before, I was getting solid 20MB/s R&W before overclock...then up to 45MB/s with overclock but with apparent errors (like a couple have had so far with this card type):

SDR104 @ 120MHz: D0 D0 r:err [SAFE] D0  [BACK] D0
until the end...
SDR104 @ 160MHz: D3 D3 r:err [SAFE] D0  [BACK] D3

I was able to record at approx 45MB/s speed using the improved overclock speed.

Did another ML card benchmark...and R&W speeds are same as above screenshot. Tried a test recording again> 'Card Full' message again> Restarted camera> camera did not turn on.
SD card now appears to be dead as a dodo. Cannot get it to be discovered on win PC through disk management.

Removed SD card, camera turned on and booted fine with my slower Sandisk SD card with ML (all back to normal).
Foolishly I should have quit earlier to recover the logs from the SD card, but alas that Sandisk appears to be toast, hence the lack of precise data - just my anecdotal report.

So, the Sandisk extreme pro 64 gb 95MB/s card might not be a wise choice right now (for some) Unless I overlooked instructions from the thread about not running ML benchmark speed test after SD overclock?

SD Card was new item from Amazon reputable seller - 2017 version of the card according to the box.

I'm happy to take one for the team on such an early offshoot discovery (and potentially look stupid)...but figured it best to report in case it is of any information of value.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 06, 2018, 08:47:46 AM
Ouch, that's not good. Can you try the card on a Linux PC (possibly an Ubuntu Live CD/USB (https://tutorials.ubuntu.com/tutorial/tutorial-create-a-usb-stick-on-windows)) and show the last lines from "dmesg" after inserting it?

Those huge numbers sound like an uncaught error in the benchmark code.

Going to order one of these (https://www.sparkfun.com/products/11468) to check the signals, to make sure the controller is not somehow switching back to 3.3V while the card is in UHS mode (1.8V). Meanwhile, found out what driver strength means (http://lkml.iu.edu/hypermail/linux/kernel/1502.2/00270.html), and reverted the module to the previous changeset, just in case.

Do not try on expensive cards until we figure out what happened!
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 06, 2018, 09:01:03 AM
Quote from: a1ex on April 06, 2018, 08:47:46 AMThose huge numbers sound like an uncaught error in the benchmark code.

Seen this error before. Looks like benchmark not checking card for unused size required. Giving those incorrect numbers after filling card.

If this card was unused (=not filled with footage) I suspect a counterfeited card. I strongly recommend to verify card integrity for each and every card after purchasing. Respected seller or not.
h2testw for Windows or F3/F3X for OS X/macOS.
And CrystalDiskMark (or others) for Windows or Blackmagic Disk Speed Test for OS X/macOS and a decent cardreader.
Title: Re: UHS-I / SD cards investigation
Post by: Markus on April 06, 2018, 01:05:11 PM
Trying to activate brute force mode for testning on the Sandisk Extreme pro 512GB bit it does not seem to react to me pushing halfshutter when the test starts?

I think i accedently got it runnig the first time i tried this module but now it just does the normal tests.
Title: Re: UHS-I / SD cards investigation
Post by: Hans_Punk on April 06, 2018, 04:17:29 PM
Quote from: a1ex on April 06, 2018, 08:47:46 AM
Ouch, that's not good. Can you try the card on a Linux PC (possibly an Ubuntu Live CD/USB (https://tutorials.ubuntu.com/tutorial/tutorial-create-a-usb-stick-on-windows)) and show the last lines from "dmesg" after inserting it?

Those huge numbers sound like an uncaught error in the benchmark code.

Going to order one of these (https://www.sparkfun.com/products/11468) to check the signals, to make sure the controller is not somehow switching back to 3.3V while the card is in UHS mode (1.8V). Meanwhile, found out what driver strength means (http://lkml.iu.edu/hypermail/linux/kernel/1502.2/00270.html), and reverted the module to the previous changeset, just in case.

Do not try on expensive cards until we figure out what happened!

Cheers a1ex!

Unfortunatly I am unable to test on Linux machine...or get the SD card to register as showing up as a device at all. Shows all the signs of being completely dead (tried numerous device detection methods on PC).
It would make sense that the SD card could have been subjected to over-voltage in UHS mode, as the the card appears to be totally non-responsive (cooked) - not even 'recognised' as a non-recognised device to attempt a repair or recovery.

It seemed that running the ML benchmark test after running sd_uhs.mo is what somehow triggered an error - maybe resulting in voltage change that cooked or 'knocked-out' the card when in UHS mode?
Before running the ML Card Benchmark test, I was successfully recording raw video at around the improved 45MB/s speeds (from 20MB/s original) which falls into similar results others have recently reported on Sandisk Extreme pro cards.

I purposely bought the SD card at full price from the top rated seller on Amazon - just in case something like this happened. Not a foolproof way to ensure a non-counterfeit product, but more reliable than some other online sources...at least an easy refund should be possible :)

Here is a video of the benchmark speeds on second attempt to check the card after clean re-install - It was obvious something was wrong, but part of me really wished it was a real stable write speed from the SD card that could be harnessed by ML
I can dream can't I? :)

https://www.youtube.com/watch?v=dRHMtZIq-J4&feature=youtu.be

RIP Sandisk
 
Title: Re: UHS-I / SD cards investigation
Post by: loknar on April 06, 2018, 11:17:55 PM
This is just amazing  :o
5 years old camera that i bought for $150 can do continuous 2.5K (2520x1072 - 2.35:1)@24 fps video in 12-bit compressed RAW (~56MB/s), wow, just wow  :D

Sandisk Extreme (microSD, not even Pro) results (in replay mode)
===================
2018/04/06 22:29:28
===================
Before the hack: r:37MB/s w:34MB/s  W:32MB/s R:37MB/s  :)  [best 34MB/s]
SDR50 @ 96MHz  : r:37MB/s w:32MB/s  W:33MB/s R:37MB/s  meh [best 34MB/s]
SDR50 @ 96MHz  : r:37MB/s w:19MB/s  W:33MB/s R:38MB/s  meh [best 34MB/s]
SDR50 @ 80MHz  : r:32MB/s w:30MB/s  W:28MB/s R:32MB/s  meh [best 34MB/s]
SDR50 @ 80MHz  : r:36MB/s w:34MB/s  W:35MB/s R:37MB/s  :)  [best 34MB/s]
SDR50 @ 120MHz : r:54MB/s w:52MB/s  W:52MB/s R:55MB/s  :)  [best 52MB/s]
SDR50 @ 120MHz : r:54MB/s w:50MB/s  W:52MB/s R:55MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:42MB/s  W:42MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:41MB/s  W:42MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:35MB/s  W:36MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:35MB/s  W:35MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 120MHz: r:55MB/s w:53MB/s  W:52MB/s R:55MB/s  :)  [best 53MB/s]
SDR104 @ 120MHz: r:54MB/s w:51MB/s  W:50MB/s R:55MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:51MB/s w:48MB/s  W:48MB/s R:51MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:50MB/s w:48MB/s  W:48MB/s R:51MB/s  meh [best 53MB/s]
SDR104 @ 160MHz: r:72MB/s w:70MB/s  W:67MB/s R:73MB/s  :)  [best 70MB/s]
SDR104 @ 160MHz: r:72MB/s w:68MB/s  W:64MB/s R:73MB/s  meh [best 70MB/s]

Done.
Please run THOROUGH tests before using!!!

Title: Re: UHS-I / SD cards investigation
Post by: saulbass on April 07, 2018, 04:24:21 AM
running magiclantern-crop_rec_4k.2018Mar10.650D104 with a1ex's sd_uhs module loaded
doesn't seem to give a performance improvement on the 650D.104 despite the card benchmarks matching others improvements using a 700D.
running crop mode 1920x1076 24fps 12bit LossLess gives 144 frames on multiple runs filming same shot whether to not the sd_uhs module is running or not. Tested multiple times.
Title: Re: UHS-I / SD cards investigation
Post by: mk11174 on April 07, 2018, 09:39:15 AM
Hey, been out of the loop for a long time, found new hobby with drones, LOL! Saw this very exciting module and thought I would share my results for the Devs. Here are the only 2 cards I have, my older 16gig and my newer 32gig I use with my drone mostly, but also use in the camera as well.

Tests were run in Photomode, no Live view.

Used the latest update of the module as of April 6th.

Keep up the great work guys!

Card Used

(https://thumb.ibb.co/gkBfhH/IMG_20180407_032503.jpg) (https://ibb.co/gkBfhH)

Before Bench

(https://thumb.ibb.co/dLAj9x/Before_Big_Card.jpg) (https://ibb.co/dLAj9x)

After Bench

(https://thumb.ibb.co/n8fj9x/After_Big_Card.jpg) (https://ibb.co/n8fj9x)


Log

===================
2018/04/07 03:11:29
===================
Before the hack: r:43MB/s w:41MB/s  W:39MB/s R:43MB/s  8)  [best 41MB/s]
SDR50 @ 96MHz  : r:43MB/s w:41MB/s  W:40MB/s R:44MB/s  ::) [best 41MB/s]
SDR50 @ 96MHz  : r:43MB/s w:41MB/s  W:40MB/s R:43MB/s  :)  [best 41MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:34MB/s R:36MB/s  meh [best 41MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:34MB/s R:36MB/s  meh [best 41MB/s]
SDR50 @ 120MHz : r:53MB/s w:47MB/s  W:45MB/s R:54MB/s  8)  [best 47MB/s]
SDR50 @ 120MHz : r:53MB/s w:47MB/s  W:45MB/s R:54MB/s  ::) [best 47MB/s]
SDR104 @ 96MHz : r:43MB/s w:23MB/s  W:22MB/s R:43MB/s  ??? [best 47MB/s]
SDR104 @ 96MHz : r:43MB/s w:23MB/s  W:22MB/s R:43MB/s  ??? [best 47MB/s]
SDR104 @ 80MHz : r:36MB/s w:21MB/s  W:20MB/s R:36MB/s  ??? [best 47MB/s]
SDR104 @ 80MHz : r:36MB/s w:21MB/s  W:20MB/s R:36MB/s  ??? [best 47MB/s]
SDR104 @ 120MHz: r:45MB/s w:25MB/s  W:25MB/s R:46MB/s  meh [best 47MB/s]
SDR104 @ 120MHz: r:45MB/s w:25MB/s  W:25MB/s R:46MB/s  meh [best 47MB/s]
SDR104 @ 132MHz: r:45MB/s w:24MB/s  W:24MB/s R:46MB/s  meh [best 47MB/s]
SDR104 @ 132MHz: r:45MB/s w:24MB/s  W:24MB/s R:46MB/s  meh [best 47MB/s]
SDR104 @ 160MHz: r:45MB/s w:15MB/s  W:20MB/s R:22MB/s  ??? [best 47MB/s]
SDR104 @ 160MHz: r:22MB/s w:21MB/s  W:21MB/s R:22MB/s  ??? [best 47MB/s]
Best: r:22MB/s w:21MB/s  W:20MB/s R:22MB/s  ??? [best 47MB/s]
Best: r:22MB/s w:21MB/s  W:20MB/s R:22MB/s  ??? [best 47MB/s]

Done.
Please run THOROUGH tests before using!!!


Card Used

(https://thumb.ibb.co/kguEsH/IMG_20180407_032619.jpg) (https://ibb.co/kguEsH)

Before Bench

(https://thumb.ibb.co/gAZKRc/Before_Small_Card.jpg) (https://ibb.co/gAZKRc)

After Bench

(https://thumb.ibb.co/i81cCH/After_Small_Card.jpg) (https://ibb.co/i81cCH)

Log
===================
2018/04/07 02:52:24
===================
Before the hack: r:44MB/s w:41MB/s  W:41MB/s R:44MB/s  8)  [best 41MB/s]
SDR50 @ 96MHz  : r:44MB/s w:40MB/s  W:41MB/s R:44MB/s  ::) [best 41MB/s]
SDR50 @ 96MHz  : r:44MB/s w:41MB/s  W:41MB/s R:44MB/s  ::) [best 41MB/s]
SDR50 @ 80MHz  : r:37MB/s w:34MB/s  W:35MB/s R:37MB/s  meh [best 41MB/s]
SDR50 @ 80MHz  : r:37MB/s w:33MB/s  W:34MB/s R:37MB/s  meh [best 41MB/s]
SDR50 @ 120MHz : r:55MB/s w:50MB/s  W:49MB/s R:55MB/s  8)  [best 50MB/s]
SDR50 @ 120MHz : r:55MB/s w:52MB/s  W:50MB/s R:55MB/s  :)  [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:40MB/s  W:41MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:40MB/s  W:41MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:34MB/s  W:35MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:36MB/s  W:35MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 120MHz: r:55MB/s w:49MB/s  W:50MB/s R:55MB/s  ::) [best 52MB/s]
SDR104 @ 120MHz: r:55MB/s w:48MB/s  W:52MB/s R:55MB/s  ::) [best 52MB/s]
SDR104 @ 132MHz: r:51MB/s w:46MB/s  W:48MB/s R:51MB/s  meh [best 52MB/s]
SDR104 @ 132MHz: r:51MB/s w:49MB/s  W:47MB/s R:51MB/s  ::) [best 52MB/s]
SDR104 @ 160MHz: r:73MB/s w:64MB/s  W:65MB/s R:72MB/s  8)  [best 64MB/s]
SDR104 @ 160MHz: r:72MB/s w:61MB/s  W:67MB/s R:72MB/s  ::) [best 64MB/s]
Best: r:73MB/s w:64MB/s  W:66MB/s R:72MB/s  :)  [best 64MB/s]
Best: r:72MB/s w:68MB/s  W:64MB/s R:72MB/s  :)  [best 68MB/s]

Done.
Please run THOROUGH tests before using!!!
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 07, 2018, 04:35:39 PM
@mk11174: please try this module on both cards:

sd_uhs_d.mo (https://a1ex.magiclantern.fm/bleeding-edge/uhs/sd_uhs_d.mo) (rename to sd_uhs.mo, otherwise it doesn't work for some reason)

Supported models for this diagnostic version: 5D3 1.1.3 and 700D 1.1.5.

700D: it won't perform any overclocking (will only run some tests to print card capabilities returned by CMD6).
5D3 1.1.3: it will only install the basic version of the hack (with 700D parameters) and will run some tests to print card capabilities.

I'd like the same test to be run on SanDisk cards from 5D3 1.1.3, if you don't mind the possibility of damaging the card (risk hopefully minimized, but unfortunately I cannot be 100% certain).
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 07, 2018, 05:06:12 PM
My new shiny Sandisk Extreme Pro card was delivered today.
Ran the first version of the SD_UHS module on it, yes 70Mb write speed on the 6d  :D :D :D


===================
2018/04/07 16:05:21
===================
Before the hack: r:44MB/s w:42MB/s  W:42MB/s R:44MB/s  :)  [best 42MB/s]
SDR50 @ 96MHz  : r:44MB/s w:40MB/s  W:41MB/s R:44MB/s  meh [best 42MB/s]
SDR50 @ 96MHz  : r:43MB/s w:43MB/s  W:43MB/s R:43MB/s  :)  [best 43MB/s]
SDR50 @ 80MHz  : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 43MB/s]
SDR50 @ 80MHz  : r:36MB/s w:36MB/s  W:36MB/s R:36MB/s  meh [best 43MB/s]
SDR50 @ 120MHz : r:54MB/s w:53MB/s  W:51MB/s R:54MB/s  :)  [best 53MB/s]
SDR50 @ 120MHz : r:54MB/s w:52MB/s  W:53MB/s R:54MB/s  meh [best 53MB/s]
SDR104 @ 96MHz : r:44MB/s w:41MB/s  W:41MB/s R:43MB/s  meh [best 53MB/s]
SDR104 @ 96MHz : r:43MB/s w:43MB/s  W:42MB/s R:44MB/s  meh [best 53MB/s]
SDR104 @ 80MHz : r:36MB/s w:35MB/s  W:35MB/s R:36MB/s  meh [best 53MB/s]
SDR104 @ 80MHz : r:36MB/s w:36MB/s  W:36MB/s R:36MB/s  meh [best 53MB/s]
SDR104 @ 120MHz: r:54MB/s w:53MB/s  W:51MB/s R:54MB/s  :)  [best 53MB/s]
SDR104 @ 120MHz: r:54MB/s w:52MB/s  W:53MB/s R:54MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:60MB/s w:59MB/s  W:56MB/s R:60MB/s  :)  [best 59MB/s]
SDR104 @ 132MHz: r:60MB/s w:58MB/s  W:58MB/s R:60MB/s  meh [best 59MB/s]
SDR104 @ 160MHz: r:72MB/s w:68MB/s  W:67MB/s R:72MB/s  :)  [best 68MB/s]
SDR104 @ 160MHz: r:72MB/s w:70MB/s  W:69MB/s R:72MB/s  :)  [best 70MB/s]

Done.
Please run THOROUGH tests before using!!!

Title: Re: UHS-I / SD cards investigation
Post by: domasa on April 07, 2018, 05:47:02 PM
I tested five slow SD cards with 5D Mark III :
(https://thumb.ibb.co/drQz9x/20180407_174036.jpg) (https://ibb.co/drQz9x)

SanDisk Ultra 64GB (white/gray) : 33 MB/s
(https://thumb.ibb.co/bQcU9x/20180407_164142.jpg)

Before the hack: r:22MB/s w:19MB/s  W:21MB/s R:22MB/s  8)  [best 19MB/s]
SDR50 @ 96MHz  : r:43MB/s w:23MB/s  W:18MB/s R:43MB/s  8)  [best 23MB/s]
SDR50 @ 96MHz  : r:43MB/s w:19MB/s  W:16MB/s R:43MB/s  meh [best 23MB/s]
SDR50 @ 80MHz  : r:36MB/s w:12MB/s  W:19MB/s R:36MB/s  meh [best 23MB/s]
SDR50 @ 80MHz  : r:36MB/s w:16MB/s  W:12MB/s R:36MB/s  meh [best 23MB/s]
SDR50 @ 120MHz : r:54MB/s w:19MB/s  W:12MB/s R:54MB/s  meh [best 23MB/s]
SDR50 @ 120MHz : r:54MB/s w:17MB/s  W:15MB/s R:54MB/s  meh [best 23MB/s]
SDR104 @ 96MHz : r:43MB/s w:20MB/s  W:15MB/s R:43MB/s  meh [best 23MB/s]
SDR104 @ 96MHz : r:43MB/s w:24MB/s  W:21MB/s R:43MB/s  :)  [best 24MB/s]
SDR104 @ 80MHz : r:36MB/s w:18MB/s  W:21MB/s R:36MB/s  meh [best 24MB/s]
SDR104 @ 80MHz : r:36MB/s w:33MB/s  W:12MB/s R:36MB/s  8)  [best 33MB/s]
SDR104 @ 120MHz: r:54MB/s w:16MB/s  W:17MB/s R:ERR
SDR104 @ 120MHz: r:err [SAFE]  [BACK]
SDR104 @ 132MHz: r:60MB/s w:45MB/s  W:27MB/s R:ERR
SDR104 @ 132MHz: r:err [SAFE]  [BACK]
SDR104 @ 160MHz: r:71MB/s w:18MB/s  W:14MB/s R:ERR
SDR104 @ 160MHz: r:err [SAFE]  [BACK]
Best: r:36MB/s w:13MB/s  W:24MB/s R:36MB/s  ??? [best 33MB/s]
Best: r:36MB/s w:16MB/s  W:21MB/s R:36MB/s  ??? [best 33MB/s]

Done.
Please run THOROUGH tests before using!!!


SanDisk Ultra 64GB (red/gray) : 37 MB/s
(https://thumb.ibb.co/bRA5hH/20180407_164623.jpg)

Before the hack: r:22MB/s w:16MB/s  W:19MB/s R:22MB/s  8)  [best 16MB/s]
SDR50 @ 96MHz  : r:43MB/s w:22MB/s  W:19MB/s R:43MB/s  8)  [best 22MB/s]
SDR50 @ 96MHz  : r:43MB/s w:21MB/s  W:26MB/s R:43MB/s  ::) [best 22MB/s]
SDR50 @ 80MHz  : r:36MB/s w:20MB/s  W:26MB/s R:36MB/s  ::) [best 22MB/s]
SDR50 @ 80MHz  : r:36MB/s w:23MB/s  W:26MB/s R:36MB/s  :)  [best 23MB/s]
SDR50 @ 120MHz : r:54MB/s w:24MB/s  W:29MB/s R:54MB/s  :)  [best 24MB/s]
SDR50 @ 120MHz : r:54MB/s w:37MB/s  W:39MB/s R:54MB/s  8)  [best 37MB/s]
SDR104 @ 96MHz : r:43MB/s w:30MB/s  W:22MB/s R:43MB/s  meh [best 37MB/s]
SDR104 @ 96MHz : r:43MB/s w:26MB/s  W:27MB/s R:43MB/s  meh [best 37MB/s]
SDR104 @ 80MHz : r:36MB/s w:29MB/s  W:33MB/s R:36MB/s  meh [best 37MB/s]
SDR104 @ 80MHz : r:36MB/s w:36MB/s  W:26MB/s R:36MB/s  ::) [best 37MB/s]
SDR104 @ 120MHz: r:err [SAFE]  [BACK]
SDR104 @ 120MHz: r:err [SAFE]  [BACK]
SDR104 @ 132MHz: r:err [SAFE]  [BACK]
SDR104 @ 132MHz: r:err [SAFE]  [BACK]
SDR104 @ 160MHz: r:err [SAFE]  [BACK]
SDR104 @ 160MHz: r:err [SAFE]  [BACK]
Best: r:err [SAFE]  [BACK]
Best: r:err [SAFE]  [BACK]

Done.
Please run THOROUGH tests before using!!!


Lexar 633x 16GB : 19 MB/s
(https://thumb.ibb.co/bWy2Ux/20180407_171917.jpg)

Before the hack: r:22MB/s w:18MB/s  W:16MB/s R:22MB/s  8)  [best 18MB/s]
SDR50 @ 96MHz  : r:43MB/s w:18MB/s  W:19MB/s R:43MB/s  :)  [best 18MB/s]
SDR50 @ 96MHz  : r:43MB/s w:18MB/s  W:19MB/s R:43MB/s  ::) [best 18MB/s]
SDR50 @ 80MHz  : r:36MB/s w:18MB/s  W:18MB/s R:36MB/s  ::) [best 18MB/s]
SDR50 @ 80MHz  : r:36MB/s w:18MB/s  W:18MB/s R:36MB/s  ::) [best 18MB/s]
SDR50 @ 120MHz : r:54MB/s w:19MB/s  W:19MB/s R:54MB/s  :)  [best 19MB/s]
SDR50 @ 120MHz : r:54MB/s w:19MB/s  W:19MB/s R:53MB/s  :)  [best 19MB/s]
SDR104 @ 96MHz : r:43MB/s w:18MB/s  W:19MB/s R:43MB/s  ::) [best 19MB/s]
SDR104 @ 96MHz : r:43MB/s w:19MB/s  W:18MB/s R:43MB/s  ::) [best 19MB/s]
SDR104 @ 80MHz : r:36MB/s w:17MB/s  W:17MB/s R:36MB/s  meh [best 19MB/s]
SDR104 @ 80MHz : r:36MB/s w:16MB/s  W:16MB/s R:36MB/s  meh [best 19MB/s]
SDR104 @ 120MHz: r:54MB/s w:18MB/s  W:18MB/s R:54MB/s  ::) [best 19MB/s]
SDR104 @ 120MHz: r:54MB/s w:18MB/s  W:18MB/s R:54MB/s  ::) [best 19MB/s]
SDR104 @ 132MHz: r:59MB/s w:18MB/s  W:18MB/s R:59MB/s  ::) [best 19MB/s]
SDR104 @ 132MHz: r:60MB/s w:18MB/s  W:18MB/s R:59MB/s  ::) [best 19MB/s]
SDR104 @ 160MHz: r:71MB/s w:18MB/s  W:18MB/s R:70MB/s  ::) [best 19MB/s]
SDR104 @ 160MHz: r:71MB/s w:18MB/s  W:18MB/s R:70MB/s  ::) [best 19MB/s]
Best: r:54MB/s w:18MB/s  W:18MB/s R:54MB/s  ::) [best 19MB/s]
Best: r:54MB/s w:18MB/s  W:18MB/s R:54MB/s  ::) [best 19MB/s]

Done.
Please run THOROUGH tests before using!!!


Kingston 64GB : 9 MB/s
(https://thumb.ibb.co/kYnJ2H/20180407_170101.jpg)

Before the hack: r:22MB/s w:6.1MB/s  W:6.8MB/s R:22MB/s  8)  [best 6.1MB/s]
SDR50 @ 96MHz  : r:43MB/s w:9.2MB/s  W:8.2MB/s R:43MB/s  8)  [best 9.2MB/s]
SDR50 @ 96MHz  : r:43MB/s w:9.1MB/s  W:5.8MB/s R:43MB/s  ::) [best 9.2MB/s]
SDR50 @ 80MHz  : r:36MB/s w:8.9MB/s  W:12MB/s R:36MB/s  ::) [best 9.2MB/s]
SDR50 @ 80MHz  : r:36MB/s w:9.0MB/s  W:12MB/s R:36MB/s  ::) [best 9.2MB/s]
SDR50 @ 120MHz : r:err [SAFE]  [BACK]
SDR50 @ 120MHz : r:err [SAFE]  [BACK]
SDR104 @ 96MHz : r:43MB/s w:ERR [SAFE] !!! [BACK]
SDR104 @ 96MHz : r:35MB/s w:ERR [SAFE] !!! [BACK]
SDR104 @ 80MHz : r:33MB/s w:ERR [SAFE] !!! [BACK]
SDR104 @ 80MHz : r:33MB/s w:ERR [SAFE] !!! [BACK]
SDR104 @ 120MHz: r:err [SAFE]  [BACK]
SDR104 @ 120MHz: r:err [SAFE]  [BACK]
SDR104 @ 132MHz: r:err [SAFE]  [BACK]
SDR104 @ 132MHz: r:err [SAFE]  [BACK]
SDR104 @ 160MHz: r:err [SAFE]  [BACK]
SDR104 @ 160MHz: r:err [SAFE]  [BACK]
Best: r:34MB/s w:ERR [SAFE] !!! [BACK]
Best: r:34MB/s w:ERR [SAFE] !!! [BACK]

Done.
Please run THOROUGH tests before using!!!


Transcend 64GB : 11 MB/s
(https://thumb.ibb.co/hut2Ux/20180407_170739.jpg)

Before the hack: r:21MB/s w:11MB/s  W:20MB/s R:19MB/s  8)  [best 11MB/s]
SDR50 @ 96MHz  : r:22MB/s w:11MB/s  W:21MB/s R:21MB/s  :)  [best 11MB/s]
SDR50 @ 96MHz  : r:22MB/s w:11MB/s  W:10MB/s R:21MB/s  ::) [best 11MB/s]
SDR50 @ 80MHz  : r:22MB/s w:11MB/s  W:21MB/s R:20MB/s  ::) [best 11MB/s]
SDR50 @ 80MHz  : r:22MB/s w:11MB/s  W:11MB/s R:20MB/s  ::) [best 11MB/s]
SDR50 @ 120MHz : r:22MB/s w:11MB/s  W:21MB/s R:20MB/s  :)  [best 11MB/s]
SDR50 @ 120MHz : r:22MB/s w:11MB/s  W:11MB/s R:20MB/s  :)  [best 11MB/s]
SDR104 @ 96MHz : r:22MB/s w:11MB/s  W:11MB/s R:21MB/s  ::) [best 11MB/s]
SDR104 @ 96MHz : r:22MB/s w:11MB/s  W:21MB/s R:21MB/s  :)  [best 11MB/s]
SDR104 @ 80MHz : r:22MB/s w:11MB/s  W:21MB/s R:20MB/s  ::) [best 11MB/s]
SDR104 @ 80MHz : r:22MB/s w:11MB/s  W:11MB/s R:20MB/s  ::) [best 11MB/s]
SDR104 @ 120MHz: r:22MB/s w:11MB/s  W:11MB/s R:20MB/s  ::) [best 11MB/s]
SDR104 @ 120MHz: r:22MB/s w:11MB/s  W:11MB/s R:20MB/s  ::) [best 11MB/s]
SDR104 @ 132MHz: r:22MB/s w:11MB/s  W:21MB/s R:20MB/s  ::) [best 11MB/s]
SDR104 @ 132MHz: r:22MB/s w:11MB/s  W:21MB/s R:20MB/s  ::) [best 11MB/s]
SDR104 @ 160MHz: r:22MB/s w:11MB/s  W:11MB/s R:21MB/s  ::) [best 11MB/s]
SDR104 @ 160MHz: r:22MB/s w:11MB/s  W:11MB/s R:20MB/s  ::) [best 11MB/s]
Best: r:22MB/s w:11MB/s  W:21MB/s R:20MB/s  ::) [best 11MB/s]
Best: r:22MB/s w:11MB/s  W:21MB/s R:20MB/s  ::) [best 11MB/s]

Done.
Please run THOROUGH tests before using!!!
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 07, 2018, 06:11:43 PM
Some of these cards had trouble enabling higher speeds. Mind running the diagnostic version (right above your post) on all these cards? You will need firmware 1.1.3 for that test. Just to check whether the card capabilities can predict the results, to some extent.
Title: Re: UHS-I / SD cards investigation
Post by: domasa on April 07, 2018, 07:13:53 PM
I downgraded firmware to 1.1.3. But I have problem with load module sd_uhs_d:

(https://thumb.ibb.co/hhuZRc/sd_uhs_d.gif) (https://ibb.co/hhuZRc)


I have ML on CF card.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 07, 2018, 07:23:07 PM
That's weird; renaming the module to sd_uhs.mo solves the issue. Will check later to see why that happens.
Title: Re: UHS-I / SD cards investigation
Post by: domasa on April 07, 2018, 07:34:12 PM
SanDisk Ultra 64GB (white/gray)

Before the hack: r:22MB/s w:14MB/s  W:10MB/s R:22MB/s  8)  [best 14MB/s]
SDR50 @ 96MHz  : sd_check_function =>
00C8 8001 8001 800F 800F 8001 801F 0000
8001 800F 800F 8001 801F 0000 0100 0000
800F 8001 801F 0000 0100 0000 0000 0000
801F 0000 0100 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     1 (supported 0,1,2,3,4,F)
sd_set_function =>
0190 8001 8001 800F 800F 8001 801F 0000
8001 800F 800F 8001 801F 0000 0200 0000
800F 8001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 400 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:18MB/s  W:18MB/s R:43MB/s  8)  [best 18MB/s]
SDR50 @ 96MHz  : sd_set_function =>
0190 8001 8001 800F 800F 8001 801F 0000
8001 800F 800F 8001 801F 0000 0200 0000
800F 8001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 400 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:18MB/s  W:18MB/s R:43MB/s  :)  [best 18MB/s]
Best: sd_set_function =>
0190 8001 8001 800F 800F 8001 801F 0000
8001 800F 800F 8001 801F 0000 0200 0000
800F 8001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 400 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:18MB/s  W:17MB/s R:43MB/s  :)  [best 18MB/s]
Best: sd_set_function =>
0190 8001 8001 800F 800F 8001 801F 0000
8001 800F 800F 8001 801F 0000 0200 0000
800F 8001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 400 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:17MB/s  W:17MB/s R:43MB/s  ::) [best 18MB/s]

Done.
Please run THOROUGH tests before using!!!


SanDisk Ultra 64GB (red/gray)

Before the hack: r:22MB/s w:19MB/s  W:12MB/s R:22MB/s  8)  [best 19MB/s]
SDR50 @ 96MHz  : sd_check_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0100 0000
800F C001 801F 0000 0100 0000 0000 0000
801F 0000 0100 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     1 (supported 0,1,2,3,4,F)
sd_set_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0200 0000
800F C001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:22MB/s  W:32MB/s R:43MB/s  8)  [best 22MB/s]
SDR50 @ 96MHz  : sd_set_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0200 0000
800F C001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:33MB/s  W:24MB/s R:43MB/s  8)  [best 33MB/s]
Best: sd_set_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0200 0000
800F C001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:21MB/s  W:25MB/s R:43MB/s  meh [best 33MB/s]
Best: sd_set_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0200 0000
800F C001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:24MB/s  W:25MB/s R:43MB/s  meh [best 33MB/s]

Done.
Please run THOROUGH tests before using!!!


Kingston 64GB

Before the hack: r:22MB/s w:6.4MB/s  W:6.9MB/s R:22MB/s  8)  [best 6.4MB/s]
SDR50 @ 96MHz  : sd_check_function =>
0064 8001 8001 800F 800F 8001 8017 0000
8001 800F 800F 8001 8017 0000 0100 0000
800F 8001 8017 0000 0100 0000 0000 0000
8017 0000 0100 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 100 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     1 (supported 0,1,2,4,F)
sd_set_function =>
0064 8001 8001 800F 800F 8001 8017 0000
8001 800F 800F 8001 8017 0000 0200 0000
800F 8001 8017 0000 0200 0000 0000 0000
8017 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 100 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,4,F)
r:39MB/s w:7.2MB/s  W:8.2MB/s R:43MB/s  8)  [best 7.2MB/s]
SDR50 @ 96MHz  : sd_set_function =>
0064 8001 8001 800F 800F 8001 8017 0000
8001 800F 800F 8001 8017 0000 0200 0000
800F 8001 8017 0000 0200 0000 0000 0000
8017 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 100 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,4,F)
r:39MB/s w:7.2MB/s  W:5.7MB/s R:43MB/s  :)  [best 7.2MB/s]
Best: sd_set_function =>
0064 8001 8001 800F 800F 8001 8017 0000
8001 800F 800F 8001 8017 0000 0200 0000
800F 8001 8017 0000 0200 0000 0000 0000
8017 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 100 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,4,F)
r:39MB/s w:7.2MB/s  W:13MB/s R:42MB/s  :)  [best 7.2MB/s]
Best: sd_set_function =>
0064 8001 8001 800F 800F 8001 8017 0000
8001 800F 800F 8001 8017 0000 0200 0000
800F 8001 8017 0000 0200 0000 0000 0000
8017 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 100 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,4,F)
r:39MB/s w:7.2MB/s  W:13MB/s R:42MB/s  :)  [best 7.2MB/s]

Done.
Please run THOROUGH tests before using!!!

Title: Re: UHS-I / SD cards investigation
Post by: mk11174 on April 08, 2018, 12:29:30 AM

(https://thumb.ibb.co/kguEsH/IMG_20180407_032619.jpg) (https://ibb.co/kguEsH)

===================
2018/04/07 18:22:55
===================
Before the hack: r:43MB/s w:42MB/s  W:40MB/s R:43MB/s  8)  [best 42MB/s]
SDR50 @ 96MHz  : sd_check_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0100 0000
800F C001 801F 0000 0100 0000 0000 0000
801F 0000 0100 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     1 (supported 0,1,2,3,4,F)
sd_set_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0200 0000
800F C001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:39MB/s  W:41MB/s R:43MB/s  ::) [best 42MB/s]
SDR50 @ 96MHz  : sd_set_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0200 0000
800F C001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:41MB/s  W:41MB/s R:43MB/s  ::) [best 42MB/s]
Best: sd_set_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0200 0000
800F C001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:40MB/s  W:41MB/s R:43MB/s  ::) [best 42MB/s]
Best: sd_set_function =>
00C8 8001 8001 800F 800F C001 801F 0000
8001 800F 800F C001 801F 0000 0200 0000
800F C001 801F 0000 0200 0000 0000 0000
801F 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,2,3,F)
Function group 3 (Driver Strength) 0 (supported 0,1,2,3,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,3,4,F)
r:43MB/s w:40MB/s  W:41MB/s R:43MB/s  ::) [best 42MB/s]

Done.
Please run THOROUGH tests before using!!!



(https://thumb.ibb.co/gkBfhH/IMG_20180407_032503.jpg) (https://ibb.co/gkBfhH)

===================
2018/04/07 18:26:14
===================
Before the hack: r:43MB/s w:41MB/s  W:39MB/s R:44MB/s  8)  [best 41MB/s]
SDR50 @ 96MHz  : sd_check_function =>
00C8 8001 8001 8003 8001 C001 8017 0000
8001 8003 8001 C001 8017 0000 0100 0000
8001 C001 8017 0000 0100 0000 0000 0000
8017 0000 0100 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,F)
Function group 3 (Driver Strength) 0 (supported 0,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     1 (supported 0,1,2,4,F)
sd_set_function =>
00C8 8001 8001 8003 8001 C001 8017 0000
8001 8003 8001 C001 8017 0000 0200 0000
8001 C001 8017 0000 0200 0000 0000 0000
8017 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,F)
Function group 3 (Driver Strength) 0 (supported 0,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,4,F)
r:42MB/s w:41MB/s  W:40MB/s R:43MB/s  ::) [best 41MB/s]
SDR50 @ 96MHz  : sd_set_function =>
00C8 8001 8001 8003 8001 C001 8017 0000
8001 8003 8001 C001 8017 0000 0200 0000
8001 C001 8017 0000 0200 0000 0000 0000
8017 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,F)
Function group 3 (Driver Strength) 0 (supported 0,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,4,F)
r:43MB/s w:41MB/s  W:40MB/s R:43MB/s  ::) [best 41MB/s]
Best: sd_set_function =>
00C8 8001 8001 8003 8001 C001 8017 0000
8001 8003 8001 C001 8017 0000 0200 0000
8001 C001 8017 0000 0200 0000 0000 0000
8017 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,F)
Function group 3 (Driver Strength) 0 (supported 0,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,4,F)
r:43MB/s w:41MB/s  W:40MB/s R:43MB/s  :)  [best 41MB/s]
Best: sd_set_function =>
00C8 8001 8001 8003 8001 C001 8017 0000
8001 8003 8001 C001 8017 0000 0200 0000
8001 C001 8017 0000 0200 0000 0000 0000
8017 0000 0200 0000 0000 0000 0000 0000
Switch Function Status version 0
Max current: 200 mA
Function group 6 (Reserved)        0 (supported 0,F)
Function group 5 (Reserved)        0 (supported 0,F)
Function group 4 (Power Limit)     0 (supported 0,1,F)
Function group 3 (Driver Strength) 0 (supported 0,F)
Function group 2 (Command System)  0 (supported 0,E,F)
Function group 1 (Access Mode)     2 (supported 0,1,2,4,F)
r:43MB/s w:41MB/s  W:40MB/s R:43MB/s  :)  [best 41MB/s]

Done.
Please run THOROUGH tests before using!!!
Title: Re: UHS-I / SD cards investigation
Post by: goldenchild9to5 on April 08, 2018, 06:15:15 PM
Will it be possible to overclock the "CF" interface as well so we can get higher rez on the 5D Mark III?  Awesome work @a1ex & the whole magic lantern team. 
Title: Re: UHS-I / SD cards investigation
Post by: domasa on April 08, 2018, 10:14:25 PM
Quote from: goldenchild9to5 on April 08, 2018, 06:15:15 PM
Will it be possible to overclock the "CF" interface...?
I had same question ☺
Quote...what about brute force search registers for CompactFlash?  ;)
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 09, 2018, 10:52:26 PM
Of course. Managed to get 130MB/s in benchmarks (compared to 110 in default configuration).

However, while running some low-level benchmarks (without overclocking), my CF card stopped working (I can no longer format it). I might have overwritten some important parts of the card firmware by mistake, as it no longer reports the correct version.

Before:

2.193.870   CSMgrTask:ff6aa2f8:22:05: [ID:Firmware Revision] = 20121203
2.193.882   CSMgrTask:ff6aa30c:22:05: [ID:Model Number] = SILICONMOTION SM2236AC   
2.193.904   CSMgrTask:ff6aa3b4:22:05: IDE = 4, PCMCIA = 80, UDMA = 7
2.193.927   CSMgrTask:ff6aa58c:22:05: 48-bit LBA:0x00000000_03ba3e70
2.193.953   CSMgrTask:ff6aa604:22:03: Cyl=62041, Hds=16, Trk=63, nSec=62537328


After (same card):

0.592.929   CSMgrTask:ff6aa2f8:22:05: [ID:Firmware Revision] =         
0.592.940   CSMgrTask:ff6aa30c:22:05: [ID:Model Number] = SM2236-AB                               
0.592.962   CSMgrTask:ff6aa3b4:22:05: IDE = 4, PCMCIA = 0, UDMA = 0
0.592.989   CSMgrTask:ff6aa604:22:03: Cyl=65, Hds=16, Trk=63, nSec=65520


The card still reacts somewhat to I/O requests, but reports bogus values when trying to read sectors from it. It simply repeats the LBA address modulo 256, 512 times.

Quote from: KB TechSupport on May 23, 2016, 10:47:19 AM
If the software reports that the card is less than 64MB (not GB) then the card is corrupted  and is unrecoverable using card reader.

65520*512 = 32MB => RIP KB 32GB 1000x.

If you have experience with reflashing CF card firmware, or other low-level diagnostics, please get in touch.

Camera still works fine with another (slower) CF card.
Title: Re: UHS-I / SD cards investigation
Post by: domasa on April 10, 2018, 12:30:40 AM
Quote from: a1ex on April 09, 2018, 10:52:26 PM
However, while running some low-level benchmarks (without overclocking), my CF card stopped working

I have broken KomputerBay 64GB 1000x. When I was recording to RAW a few years ago: camera crash and CF contains file with strange name... Slow format was not possible (stop after +-60%).

I tried program "SD Card Formatter" today... Result? Crash and cappacity is 31,9 MB... => absolute RIP
Title: Re: UHS-I / SD cards investigation
Post by: Kharak on April 10, 2018, 12:55:11 AM
Maybe this is as good a time as any to tell how I corrupted a 128 GB 1000x Lexar card.

This is about 3 years ago, I was looking in to doing Spanned recordings to increase recording time with Vanilla MLV in 1920x1280. I had tried it before, but it was a long time before. So I did the mistake of going in to Canon Menu and set record to CF+SD Card and did the same in ML RAW submenu, I should have set it only in ML Menu, this resulted in an immediate crash when I pressed record and the card was inaccessible via camera or card reader. But when you buy Lexar cards, you get free serial key for their Card Recovery program, with the program I managed to repair the card to function again. Some pictures and MLV's were corrupted, but I can still use the card today, until there is about 20 GB of space left, then recording times drop to a few seconds. the SD card was fine though.

Just a FYI tl:dr. Dont set Canon Menu to record to CF+SD combined with ML Menu Spanned Recording ON, this will ruin your CF card.
Title: Re: UHS-I / SD cards investigation
Post by: reddeercity on April 10, 2018, 07:27:06 AM
Sorry to go off topic a bit .
Quote from: goldenchild9to5 on April 08, 2018, 06:15:15 PM
Will it be possible to overclock the "CF" interface as well so we can get higher rez ......
Don't think so , I had a look at 5D2 log for CF action and notice it auto detect the UDMA mode , in my case UDMA 6
but that kind of puzzling at that mode , it's writes speed is max 133MB/s according to Wikipedia (https://en.wikipedia.org/wiki/UDMA) (bench marks says on 5d2 around 79MB/s on a udma7 card 1066x) can't believe there that much over head .
and udma7 writes at 167MB/s and I do think 5d3 is around 120MB/s .
Is the controller really limited or is there a software limit , being 5d2 never really needed high write speeds like the 5d3 for video (720p60 &1080p30)

dm_log from 5d2
825FE>  CSMgrTask:00095f98:00:00: *** register_interrupt("CFDriver", 0x82, 0xffb8b8cc, 0x0), from ffb8bb58
      .............
834A0>  CSMgrTask:ffb8a92c:22:05: [ID:Model Number] = LEXAR ATA FLASH CARD                   
834C8>  CSMgrTask:ffb8aabc:22:05: IDE = 4, PCMCIA = 80, UDMA = 6
      .............
83572>  CSMgrTask:ffbdb8a0:22:01: cfDecideTiming: UDMA Mode 6 (CFA4.0)
      .............
83592>  CSMgrTask:ffbdbb3c:22:03: CF_GetAccessTiming : DatTim = 3, DatMod = 6

can we fool the controller to switch to UDMA7 on D4 camera CF cards?
Can the "DatTim" be changed to 4? & "DatMod" to 7 ? assuming that means "UDMA"
or though some register in the "CFDriver" .  Thou I have no idea how to try and change it , just a few thoughts  :D


Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 10, 2018, 08:12:13 AM
I wouldn't be surprised if these were coincidental failures, except there's pretty much no way to tell.

Quote from: Kharak on April 10, 2018, 12:55:11 AM
Just a FYI tl:dr. Dont set Canon Menu to record to CF+SD combined with ML Menu Spanned Recording ON, this will ruin your CF card.

When card spanning is enabled (mlv_rec), it ignores Canon setting (main stream goes to CF, secondary stream goes to SD). ML cannot even tell whether CF+SD is selected in Canon menu (it only recognizes the card selected for playback), so I'm pretty sure it wasn't that. BTW, can you record H.264 on both SD and CF at the same time? (it doesn't seem to work here).

I also have a Lexar 512MB SD that now reports 256MB after a crash while copying something on it, some years ago. Maybe I should try SD Card Formatter on it.

With the dead CF, I was doing sector-level benchmarks with Canon routines from bootflags.c, filling the entire card with 0xFFFFFFFF, 0x00000000, 0x55555555, 0xAAAAAAAA, 0x5A5A5A5A etc, to see if that makes any difference in benchmarks. It didn't, but some blocks were written at ~120MB/s, others at ~85MB/s, and a very small percentage of them were written at lower speeds. That pattern wasn't exactly repeatable, but the histograms were pretty much the same. I've saved logs from 3 such runs, so here are some plots:

(https://a1ex.magiclantern.fm/bleeding-edge/cf/dots.png) (https://a1ex.magiclantern.fm/bleeding-edge/cf/hist.png)

Right before the card died, I accidentally wrote something to sector -32768 (LBA underflow?)

@reddeercity: more details after I'll get a new card (it *is* possible to overclock the 5D2 CF interface).
Title: Re: UHS-I / SD cards investigation
Post by: goldenchild9to5 on April 10, 2018, 08:01:59 PM
Awesome work @a1ex..
Title: Re: UHS-I / SD cards investigation
Post by: Kharak on April 10, 2018, 08:27:44 PM
@a1ex

I always thought it was the Double Spanning from Canon and ML causing it, but if you say they dont interfere with each other, then I don't know what happened. I set the settings and hit record and the camera crashed and I got an Error code and the card was bust. Never happened before or after, never tried it again and will never try it again :)

I have not tried h264 to two separate cards, I never even considered it, but just to be clear, you mean Recording H264 to both cards simultaneously, is that an ML feature?. I also couldn't figure out how to set h264 proxy to record to SD while MLV went to CF.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 10, 2018, 08:34:52 PM
I've tried it before posting; no surprises. When you record raw video, Canon's LiveView remains idle (their code doesn't even know the record button was pressed, since the raw recorder catches that event and doesn't pass it back to Canon firmware). Remember the LiveView used to be turned off after 30 minutes, since Canon firmware believed the camera was idle (while it was actually recording raw video). For the card selection setting, ML only sees 0 = CF or 1 = SD, there are no other possible values for that property. ML does not (and did not) see the advanced options from Canon menu, like CF+SD.

I thought recording H.264 on two cards is a feature from Canon firmware (after you said you can "set Canon Menu to record to CF+SD").

In crop_rec_4k builds, H.264 goes to the card selected in Canon menu, while raw video (mlv_lite) always goes to CF if there's one inserted. In main builds and in mlv_rec from any builds (regular recording without H.264 proxy), both will go to the card selected in Canon menu. In mlv_rec with spanning enabled, main stream goes to CF and secondary stream goes to SD, regardless of Canon setting.
Title: Re: UHS-I / SD cards investigation
Post by: reddeercity on April 11, 2018, 05:23:47 AM
Quote from: a1ex on April 10, 2018, 08:12:13 AM
@reddeercity: more details after I'll get a new card (it *is* possible to overclock the 5D2 CF interface).
:o Really , cool that's better then trying to change to udma 7
Increase the "bus speed" ? can't wait to read the details .
I saw something in the 5d2 disassembly that look interesting
"<K218 Board> SystemCLK 132MHz":

looks to be part of the system check before boot loader .
Title: Re: UHS-I / SD cards investigation
Post by: reddeercity on April 11, 2018, 06:48:32 AM
search for "cfDecideTiming" in the rom, got this
*"cfGetRegisterTiming: I/O 250nS (PIO Mode4)"
so "Mode4" =250nS I/O there was also a
*"cfGetRegisterTiming: I/O 120nS"
but not "Mode"
this got me interested in the CF events , would be nice to have more write speed .

Edit: look like that the slowest after some searching , would need to be around 30 or 20ns for 100MB/s
Title: Re: UHS-I / SD cards investigation
Post by: reddeercity on April 11, 2018, 07:28:39 AM
Is this kind of what you can do @a1ex "Double Transition Clocking"
http://www.pcguide.com/intro/fun/clockDouble-c.html
If not it's still interesting reading  :)
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 11, 2018, 07:48:49 AM
There is the DDR50 mode for SD cards, easily enabled on the card side from sdSetFunction, but... good luck configuring Canon controller to use that. Brute-forcing the "known" registers didn't help.

You can send ATA commands to the CF card (QEMU emulates them), so if the only difference between UDMA 6 and 7 is timing, it might even be possible to put the card in UDMA7.
Title: Re: UHS-I / SD cards investigation
Post by: Ant123 on April 11, 2018, 02:06:19 PM
Quote from: a1ex on April 11, 2018, 07:48:49 AM
good luck configuring Canon controller to use that. Brute-forcing the "known" registers didn't help.

Why do you think that camera controller has support of DDR mode?
In accordance with debugging messages inside D6 & D7 firmwares the fastest mode is SDR104 and there are no signs of DDR...
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 11, 2018, 02:57:34 PM
I don't know whether the controller supports DDR or not, but - as you can see in previous logs - there are cards that support DDR50, but not SDR104. That's why I've assumed DDR50 might be an older standard.
Title: Re: UHS-I / SD cards investigation
Post by: Ant123 on April 11, 2018, 04:30:18 PM
I did not found significant changes in the standard.
Look at the paragraph 3.9.3 of "Physical Layer Simplified Specification (https://www.sdcard.org/downloads/pls/)" (from v3.01 to v6.00). There are three host types.
Title: Re: UHS-I / SD cards investigation
Post by: reddeercity on April 12, 2018, 10:29:06 PM
Quote from: a1ex on April 11, 2018, 07:48:49 AM
You can send ATA commands to the CF card (QEMU emulates them), so if the only difference between UDMA 6 and 7 is timing, it might even be possible to put the card in UDMA7.
Ok , yet another reason to get QEMU going
Title: Re: UHS-I / SD cards investigation
Post by: David_Hugh on April 13, 2018, 11:06:29 PM
This is insane. It really works, just like that. Did the test on a gold sandisk extreme 32gb and a 70D. This card now effectively records raw at 49Mb/s instead of 40, resulting in a noticeable bump in resolution/aspect ratio. For those saying they dont see the "real world" results, I ran the test in video mode and mlv_rec was already turned on. I dont know whether turning the cam off kills the overclocking for now, or if running it in video mode was what did it, but this combo worked for me. Also, the test showed a siginificantly different result in photo/video mode (53 vs. 43) and although it only showed 43 in video the card writes 49 continous.

I did notice however that the card got quit warm relatively quickly. Again. This is insane. You guys just made every cam that can run this like 10 times better.
Title: Re: UHS-I / SD cards investigation
Post by: 50mm1200s on April 14, 2018, 12:18:11 AM
@a1ex, since some people got the cards killed by this experiment, wouldn't it be better to remove it from downloading, for now? Some people might use it for real world production and get the card crashed in the middle of the work...
I think it's a good idea, though. I can barely get 16:9 FullHD in crop_rec (50D), with this patch (if it works in CF too) it would possibly make it to continuous FullHD recording.
Do you need any help to get disposable cards that you can "fry" as you want? I could buy some cheap ones for you and send here from Brazil.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on April 14, 2018, 06:19:28 AM
The pre-built module was already rolled back to a hopefully safer version and had a stronger warning, but OK, removed the download. Nobody tested it this week anyway. The diagnostic version (https://www.magiclantern.fm/forum/index.php?topic=12862.msg199589#msg199589) (which does only the basic version of the hack on 5D3 and only prints some capabilities on 700D) is still available.

BTW, when N = 1, you may want to use the singular form (i.e. card instead of cards).
Title: Re: UHS-I / SD cards investigation
Post by: dfort on April 14, 2018, 07:37:19 AM
I removed my test build too.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 14, 2018, 01:42:47 PM
Just wanted to note, I'm still heavily testing with this and not messed up my card  :D
As far as I could tell, only the second build with driver strength options, messed up card(s).

For practical use and testing, I changed the code of the original build a little (removed the line where it starts showing the console text, and removed the tests and only kept the line with 160Mhz settings)
This way, now I can go to photo mode, click the SD overclock in the debug menu and within a second I'm ready to go. (No console text, just a few card reader led blinks)
Last night I photographed a live performance of 2 music bands and also did some video tests with the SD_overclock module.
I used the same SD card the whole time (2,5 hours) and activated the SD_overclock module multiple times, recorded some video and moved on taking photos (with
SD_overclock still activated)
I even recorded, successfully, a 5 minute length clip of 8558 frames in 2560 x 958 resolution in 14 bit lossless...on a 6d that is  :D
The MLV file was 21Gb in size  ;D

Title: Re: UHS-I / SD cards investigation
Post by: mk11174 on April 14, 2018, 04:56:59 PM
Same here, working very nice on my 32gig micro SD card, definitely an improvement with no signs of card dieing yet.
Title: Re: UHS-I / SD cards investigation
Post by: loknar on April 14, 2018, 08:16:20 PM
As instructed I did some extensive tests of sd_uhs module and:
First of all, thank you guys, this really extends usability of my EOS M far beyond of my expectations.
I've been using two cards, SD Sandisk Extreme 16 GB and uSD Sandisk Extreme 32 GB.
I've recorded over 30 GB of data during 4 hours, cards are OK, camera is OK, Data are OK (I'm using 3rd April dforts build for EOS M and crop rec build for EOS M from 10th of March).
I shot at 2520x1072 with fps override to 24 fps (HighTV) in 12-bit lossless. Resulting Bandwidth around 55 MB/s.
In most of scenes recording could be continuous, but in some settings I've got only around 3-10 seconds, didn't found reason for this (other than maybe complexity of the scene or temperature of the processor).
Also I don't know how it is supposed to behave but in my case it was: new ML installation enable SD hack, benchmark:
===================
2018/04/06 22:29:28
===================
Before the hack: r:37MB/s w:34MB/s  W:32MB/s R:37MB/s  :)  [best 34MB/s]
SDR50 @ 96MHz  : r:37MB/s w:32MB/s  W:33MB/s R:37MB/s  meh [best 34MB/s]
SDR50 @ 96MHz  : r:37MB/s w:19MB/s  W:33MB/s R:38MB/s  meh [best 34MB/s]
SDR50 @ 80MHz  : r:32MB/s w:30MB/s  W:28MB/s R:32MB/s  meh [best 34MB/s]
SDR50 @ 80MHz  : r:36MB/s w:34MB/s  W:35MB/s R:37MB/s  :)  [best 34MB/s]
SDR50 @ 120MHz : r:54MB/s w:52MB/s  W:52MB/s R:55MB/s  :)  [best 52MB/s]
SDR50 @ 120MHz : r:54MB/s w:50MB/s  W:52MB/s R:55MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:42MB/s  W:42MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:41MB/s  W:42MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:35MB/s  W:36MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:35MB/s  W:35MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 120MHz: r:55MB/s w:53MB/s  W:52MB/s R:55MB/s  :)  [best 53MB/s]
SDR104 @ 120MHz: r:54MB/s w:51MB/s  W:50MB/s R:55MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:51MB/s w:48MB/s  W:48MB/s R:51MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:50MB/s w:48MB/s  W:48MB/s R:51MB/s  meh [best 53MB/s]
SDR104 @ 160MHz: r:72MB/s w:70MB/s  W:67MB/s R:73MB/s  :)  [best 70MB/s]
SDR104 @ 160MHz: r:72MB/s w:68MB/s  W:64MB/s R:73MB/s  meh [best 70MB/s]

Done.
Please run THOROUGH tests before using!!!

Then I could record videos with high bitrate. Once i switched camera off and on again memory patch wasn't (obviously) applied, so i tried to enable SD hack again, but i've got this:
===================
2018/04/10 08:29:10
===================
Before the hack: malloc error
[best 66MB/s]
SDR50 @ 96MHz  : malloc error
[best 66MB/s]
SDR50 @ 96MHz  : malloc error
[best 66MB/s]
SDR50 @ 80MHz  : malloc error
[best 66MB/s]
SDR50 @ 80MHz  : malloc error
[best 66MB/s]
SDR50 @ 120MHz : malloc error
[best 66MB/s]
SDR50 @ 120MHz : malloc error
[best 66MB/s]
SDR104 @ 96MHz : malloc error
[best 66MB/s]
SDR104 @ 96MHz : malloc error
[best 66MB/s]
SDR104 @ 80MHz : malloc error
[best 66MB/s]
SDR104 @ 80MHz : malloc error
[best 66MB/s]
SDR104 @ 120MHz: malloc error
[best 66MB/s]
SDR104 @ 120MHz: malloc error
[best 66MB/s]
SDR104 @ 132MHz: malloc error
[best 66MB/s]
SDR104 @ 132MHz: malloc error
[best 66MB/s]
SDR104 @ 160MHz: malloc error
[best 66MB/s]
SDR104 @ 160MHz: malloc error
[best 66MB/s]

Done.
Please run THOROUGH tests before using!!!

Strangely enough, after this memory patch have been applied and i could continue recording with high bitrate.
I wanted to ask if is it possible to streamline usage of the hack (like add menu "apply saved SD clock values", or to avoid opening console), but since you retracted the module, i don't know whether you intend to explore this more. (And as far as i'm concerned i'm not giving this module back :D )

If you are interested in result, i posted it on youtube


Title: Re: UHS-I / SD cards investigation
Post by: ahmedvienna on April 15, 2018, 02:33:53 PM
I am trying to implement sd_uhs on my 6d. using the latest nightbuild.  have an error when I do it.  can anyone assist me.
here's the pic of the error.

https://pbs.twimg.com/media/DabZ9QVWsAA7zD-.jpg:large

thanks !
Ahmed
Title: Re: UHS-I / SD cards investigation
Post by: Danne on April 15, 2018, 07:44:11 PM
Wow, quite something. My Sandisk extreme pro 95mb/s shows 69mb/s in the test. Runs nicely on my 100D. Thanks Levas for feedback.
Regarding malloc error on initial testing I run the module with liveview closed. When opened malloc errors..
Title: Re: UHS-I / SD cards investigation
Post by: dfort on April 15, 2018, 08:32:40 PM
Looks like loknar, Danne, mk11174 and Levas are on the Plateau of Productivity with the sd_uhs module while some of us are stuck in the Trough of Disillusionment.

(https://upload.wikimedia.org/wikipedia/commons/thumb/9/94/Gartner_Hype_Cycle.svg/559px-Gartner_Hype_Cycle.svg.png)

@Levas, looks like you got a good working version. Could you share the changes you made on the module?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 15, 2018, 08:59:53 PM
This stuff is too good not to be used, thanks Alex  :D
This and the lossless option, makes continuous recording available for the SD cams  :D

The build I use is based on Alex first build, I only deleted a few lines, didn't add any. (I'm not that savvy in programming, but I had to have something more practical)
I deleted one line with 'show console' and I deleted a bunch off the lines where most of the settings are tested.
I only kept the highest 160Mhz setting in it.

Here's the altered source file I'm using:
https://drive.google.com/drive/folders/1A_OVNAucbHYfUXfPT9kxBy1IiUk-012U?usp=sharing (https://drive.google.com/drive/folders/1A_OVNAucbHYfUXfPT9kxBy1IiUk-012U?usp=sharing)
Title: Re: UHS-I / SD cards investigation
Post by: mk11174 on April 15, 2018, 09:01:33 PM
Quote from: loknar on April 14, 2018, 08:16:20 PM

Then I could record videos with high bitrate. Once i switched camera off and on again memory patch wasn't (obviously) applied, so i tried to enable SD hack again, but i've got this:
===================
2018/04/10 08:29:10
===================
Before the hack: malloc error
[best 66MB/s]
SDR50 @ 96MHz  : malloc error
[best 66MB/s]
SDR50 @ 96MHz  : malloc error
[best 66MB/s]
SDR50 @ 80MHz  : malloc error
[best 66MB/s]
SDR50 @ 80MHz  : malloc error
[best 66MB/s]
SDR50 @ 120MHz : malloc error
[best 66MB/s]
SDR50 @ 120MHz : malloc error
[best 66MB/s]
SDR104 @ 96MHz : malloc error
[best 66MB/s]
SDR104 @ 96MHz : malloc error
[best 66MB/s]
SDR104 @ 80MHz : malloc error
[best 66MB/s]
SDR104 @ 80MHz : malloc error
[best 66MB/s]
SDR104 @ 120MHz: malloc error
[best 66MB/s]
SDR104 @ 120MHz: malloc error
[best 66MB/s]
SDR104 @ 132MHz: malloc error
[best 66MB/s]
SDR104 @ 132MHz: malloc error
[best 66MB/s]
SDR104 @ 160MHz: malloc error
[best 66MB/s]
SDR104 @ 160MHz: malloc error
[best 66MB/s]

Done.
Please run THOROUGH tests before using!!!

Strangely enough, after this memory patch have been applied and i could continue recording with high bitrate.
I wanted to ask if is it possible to streamline usage of the hack (like add menu "apply saved SD clock values", or to avoid opening console), but since you retracted the module, i don't know whether you intend to explore this more. (And as far as i'm concerned i'm not giving this module back :D )

I only get memory error if I am in movie mode with MLV-Lite or MLV-REC turned ON, make sure these modules are off before you run this card speed module, once it is working, then ok to turn ON the MLV modules you want to use.

Obviously if you turn off camera, you will need to go through same procedure in same order again next time you turn camera on if you want the card speed boost.

Not sure if this memory error is normal, or how it was expected to work, but, thats when I get the memory error, if I do it in the right order, all is good.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 15, 2018, 09:17:02 PM
I have the module standard loaded at startup.
When I want to use it, I switch to photomode(no liveview) and start the module in Debug menu.
Switch back to video mode and ready to use it.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on April 15, 2018, 09:22:43 PM
Photo mode did the trick here too :)
Title: Re: UHS-I / SD cards investigation
Post by: mk11174 on April 15, 2018, 09:34:17 PM
Yes, using it in Photomode, non live view will def work fine, but, when in Movie mode and Raw enabled somehow, its important to note, the module will cause memory error.

I normally start in Photomode myself, but wanted to point out to the other user about maybe why he was getting the malloc error in case he might have been using movie mode with MLV On.
Title: Re: UHS-I / SD cards investigation
Post by: mk11174 on April 15, 2018, 09:43:44 PM
Quote from: ahmedvienna on April 15, 2018, 02:33:53 PM
I am trying to implement sd_uhs on my 6d. using the latest nightbuild.  have an error when I do it.  can anyone assist me.

You need to be using the crop_rec_4k build, not latest nightly.
Title: Re: UHS-I / SD cards investigation
Post by: dfort on April 15, 2018, 09:51:53 PM
Thanks @Levas -- here are your changes compared to the latest @a1ex commit (https://bitbucket.org/daniel_fort/magic-lantern/commits/979c1fc3c37ac8913d8f6ca605889734bf100a67). I branched off of the latest in order to include all the Digic 5 cameras and to see if maybe some of the latest changes should be put back in.

@Danne -- I think this might be the version you were looking for.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 15, 2018, 10:04:40 PM
Wow, good work Dfort, lot's of changes compared to the latest build  :o
I checked my build and Alex first build, and I only deleted 30 lines of code.

Title: Re: UHS-I / SD cards investigation
Post by: Danne on April 15, 2018, 10:57:11 PM
Quote from: dfort on April 15, 2018, 09:51:53 PM
Thanks @Levas -- here are your changes compared to the latest @a1ex commit (https://bitbucket.org/daniel_fort/magic-lantern/commits/979c1fc3c37ac8913d8f6ca605889734bf100a67). I branched off of the latest in order to include all the Digic 5 cameras and to see if maybe some of the latest changes should be put back in.

@Danne -- I think this might be the version you were looking for.

Actually, I use a compiled version from a1ex sd_uhs branch(latest code). I'll check into Levas stuff tomorrow. Anyway, this hack is really something else. I hope it won't be abandoned.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on April 16, 2018, 09:00:23 AM
Adding 100D and eosm to the early modified code from Levas seems to work:
    if (is_camera("EOSM", "2.0.2"))
    {
        sd_setup_mode       = 0xFF338D40;
        sd_setup_mode_in    = 0xFF338DC8;
        sd_setup_mode_reg   = 1;
        sd_set_function     = 0xFF63EF60;
        SD_ReConfiguration  = (void *) 0xFF641314;
    }

    if (is_camera("100D", "1.0.1"))
    {
        sd_setup_mode       = 0xFF3355B0;
        sd_setup_mode_in    = 0xFF335648;
        sd_setup_mode_reg   = 1;
        sd_set_function     = 0xFF6530A4;
        SD_ReConfiguration  = (void *) 0xFF655458;
    }

Latest code seems more dependent on test runs so I can´t really say why Levas "test bypass" seems to work but it does, at least on my eos 100D.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on April 18, 2018, 12:33:27 PM
Hello Danne,

What write speeds did you achieve with the 100D?  Is there a safe module available for download?  Could you post a download link?

Thanks.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on April 18, 2018, 12:45:51 PM
It´s pretty fast on 100D. 69-70mb/s when tests are performed with a sandisk extreme pro 95mb card. Only card I tested.
If you know how to compile you can create the sd_uhs module form inside modules folder once you are in sd_uhs branch. Then you simply add that module to your crop_rec_4k build.
If you´re not compiling already I can recommend this to get started:
https://www.magiclantern.fm/forum/index.php?topic=21882.msg199370#msg199370

About safety. My card still works, that´s all can refer to.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on April 18, 2018, 04:05:08 PM
Wow, that's a pretty impressive speed!  This means "continuous recording with sound" at 2520x1080 resolution. 
Title: Re: UHS-I / SD cards investigation
Post by: Danne on April 18, 2018, 04:10:31 PM
It sure does. Not sure if the card records over 55mb in real life though at least when I tested on my 100D.
Now I wanted to record dual iso 12bit lossless this way and process in Mlv app but that seems to be another can of worms to deal with. 14bit works fine but 12bit lossless do not unfortunately. Anybody wants to tweak dual iso code:
https://bitbucket.org/hudson/magic-lantern/src/7e752d614f7332541d2a5bd3baf0335e99863235/modules/dual_iso/cr2hdr.c?at=qemu&fileviewer=file-view-default
  :P
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on April 18, 2018, 05:34:39 PM
How about overheating, Danne?  My 100D reaches 51 deg. C when continuously recording at 41 MB/s.  I can imagine, that it gets substantially warmer at 69 or 55 MB/s. 

As far as Dual ISO at 2520x1080 resolution is concerned, I remember that I got pretty decent results with no aliasing and moire but you get ugly artifacts in slightly overexposed areas.  I am not sure if those are caused by focus pixels but in most cases I liked the non Dual ISO recordings better.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on April 18, 2018, 05:39:27 PM
Overheating ey. I never get to film more than a minute here and there but if you check this thread there are several users that hard tested their set up.
Title: Re: UHS-I / SD cards investigation
Post by: _DK_ on April 18, 2018, 10:33:38 PM
This looks promising. But honestly have no clue on how to test it on my 650D. Can anyone give me a hint, that gets me started? Or a link maybe with the needed module, since the links have been removed...
Thanks
Title: Re: UHS-I / SD cards investigation
Post by: saulbass on April 19, 2018, 04:04:12 PM
@ _DK_ - things are gradually developing - afaik presently there are no 650D builds that support these fast card speeds but I'm sure something stable will come eventually..
Title: Re: UHS-I / SD cards investigation
Post by: Danne on April 19, 2018, 04:23:50 PM
Actually, 650D seems supported in the code. I added the sd_uhs.mo module here but try with caution:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/

Module to be put in modules folder with crop_rec_4k build.

There are more cameras this module works on. Check the sd_uhs branch and the code for what cameras.
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on April 19, 2018, 06:05:47 PM
      @Danne

   If going to test "sd_uhs.mo" on SL1, should the "sd_uhs.mo" that is within

"magiclantern-Nightly.2018Apr18.100D101_sd_uhs" be used or the "sd_uhs.mo"

that You have posted this Morn' be used ?

                     ORR ~ DeanB
Title: Re: UHS-I / SD cards investigation
Post by: saulbass on April 19, 2018, 06:24:51 PM
@Danne - thanks!

the sd_uhs build that you posted seems to work on the 650D - I managed two passes of 3100 frames each lossless 14bit 1920x1076 - the best I've managed yet! (both old glass and 50/1.8) - so that continuous ever so nearly FULL HD 14bit raw from a 2013 camera...
Title: Re: UHS-I / SD cards investigation
Post by: canneloni on April 20, 2018, 01:49:19 PM
I also tested it on my 100Dwith the SanDisk extreme Pro and I get a strange bug. While running the new module it first gets really high transfer speeds but suddenly it gets lower than without the module to around 24mb/s.  After it finished I run the benchmark test and get the same speeds. Any idea why ?
Title: Re: UHS-I / SD cards investigation
Post by: saulbass on April 20, 2018, 01:55:01 PM
perhaps you switched the phone off or (I can't be certain) perhaps you went back to photo mode - I think at present the patch is not persistent - no doubt this will change in time.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on April 20, 2018, 02:14:26 PM
I have an 5 year old sandisk extreme, which gives about 46Mb/s write speed at most settings, but at highest settings(160Mhz) it drops to 21Mb/s.
Seems that not every card out there can handle the higher speeds.
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 21, 2018, 11:18:52 PM
Quote from: Levas on April 15, 2018, 08:59:53 PM
This stuff is too good not to be used, thanks Alex  :D
I deleted one line with 'show console' and I deleted a bunch off the lines where most of the settings are tested.
I only kept the highest 160Mhz setting in it.

Is this for download as a module as I cant-wont-couldn't compile C unless it's Arduino  ::)

I'm ready to go if someone would be so kind.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 21, 2018, 11:38:31 PM
Read reply #188
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on April 22, 2018, 10:22:53 AM
Thanks, missed that !
Works a treat, thanks Danne  :)
Title: Re: UHS-I / SD cards investigation
Post by: ahmedvienna on April 22, 2018, 10:29:36 AM
so after trying it out for 1 week. the overclock was able to give me a result of 66-67mb/sec.  However, due to stability reasons I decided to downgrade to 60mb/sec which gave me a resolution of 1824x830.  It's pretty great I have to say.  so far very stable.  I think I recorded RAW easily 100GB.  No problems whatsoever with the card or the camera.  It used the following:

SD104 @ 160 MHz: r 69mb/s w: 67mb/s . W: 64mb/s R: 72mb/s

of course the drain on the battery is quite amazing, but understandable.  Overall I am really quite happy about this. it gave my camera (canon 6d + sand disk extreme pro 95mb/sec) another life. 

I wonder when this will be part of the mainstream built.  I just hate having to run this overclock every time I run the camera.
Title: Re: UHS-I / SD cards investigation
Post by: whothe on April 23, 2018, 05:02:24 PM
Hey everyone,
I ran a test with an EOS M and a new Sandisk 128GB ExtremePRO.
I used "magiclantern-Nightly.2018Apr17.EOSM202_sd_uhs.zip"/2018-04-17 and "sd_uhs.mo"/2018-04-19 from Danne.


(https://thumb.ibb.co/h3OZ6x/UHS_Module_test.jpg) (https://ibb.co/h3OZ6x)


(https://thumb.ibb.co/bvpS9H/benchmark.jpg) (https://ibb.co/bvpS9H)


And I found these Memory patches... though I dont really know if it is of any value...

(https://thumb.ibb.co/gotn9H/IMG_20180423_172846.jpg) (https://ibb.co/gotn9H)


However I did not quite catch how to enable the overclock for the actuall recording -

I think this was already disclaimed, but I got an error message when trying to run the test whilste a couple other modules where still enabled.
Do we already know which modules produce this error?

Which other stepps need to be taken next to move this module forward?
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on April 25, 2018, 08:51:49 AM
Quote from: whothe on April 23, 2018, 05:02:24 PM
However I did not quite catch how to enable the overclock for the actuall recording

The procedure is described here in detail:

https://www.magiclantern.fm/forum/index.php?topic=16040.1175
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on April 25, 2018, 08:59:34 AM
UHS-I/SD.mo Benchmark on 64GB SanDisk ExtremePro in SL1


(https://thumb.ibb.co/hX8Zg7/w_A6vw_Knj_JLp_ZUc_Dx6_PO1_B2_N1w2q_Xc7m_Az_jn_XVAw_Ygfhg_Mkj_GSf8ej_Thnmkn_Q5_Sz0tnkt_QKDQAp_Fxn47fq0rd_Bhjgq_Ctcd_Sy_Zm_Rglubi6q_KAim533mm_Pak_Lz_MRx8dk2tmfq_An_KKA_w600_h315_p_k.jpg) (https://ibb.co/hX8Zg7)


https://photos.app.goo.gl/8s9d36uXVhxsiAyp1
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on May 12, 2018, 09:43:36 PM
I tried Danne's build from 17th April for the EOS M and it works nicely. I'm using a Sandisk extreme pro 95mb/s 64gb card, which goes up to 55 mb/s write speed max. Seems a bit low for this card but still allows continuous recording.

Here's a test video I've done with it:



Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on May 13, 2018, 10:54:53 AM
This SD-interface overclocking feature is amazing!  I have been using it on a regular basis to record more than 60 GB of footage at 12-bit lossless, 2520x1080 resolution and 24 fps where I get continuous recording with sound on the 100D.  In my opinion, this is one of the most revolutionary developments of Magic Lantern since its introduction almost 7 years ago.  Here is a another test video that I shot on the 100D:

https://vimeo.com/266871797

If the development could be set forth towards implementation into all ML-supported cameras and improving stability, this will be well worth the effort keeping in mind the clean moire-free video and its superb quality at 5x-magnification, comparable to the video quality of high-end professional RAW-video cameras.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 13, 2018, 11:08:09 AM
At least hardcode memory patches upon startup, maybe on 100D and eosm to start with? Is it safe enough? Seems like it. I also tested the hardcoded version on a slow card and it was skipping patching so that is good if running fast cards and suddenly changing to a slow card.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on May 13, 2018, 11:46:40 AM
Danne, I tested only with the SanDisk Extreme Pro (95 MB/s), since I do not have a slower card at the moment.  Your latest hardcoded build that allows up to 42 fps works very well if I do not switch modes too often.   As soon as I switch from 5x-magnification to Movie Crop Mode for example, the 100D freezes, I have to pull battery out, restart camera, reload modules, run the overclock test and only then I can continue working. 

As far as safety is concerned, I cannot comment on that but I never noticed a problem.  Even after a crash, all my previously recorded clips remained perfectly intact on the card.   I never lost a clip, unless the crash happened during recording.  This happened only once or twice and I cannot reproduce it but the reason may have been a battery level close to running out of power where supply voltage may fluctuate and drop below critical level for short periods of time during recording.   

Overall, I am very satisfied with the overclocking feature.  Again, I think, it's revolutionary!
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on May 13, 2018, 12:31:37 PM
Thanks @IDA_ML - your instructions on how to set this up on the 100D were really helpful and got me up and running on the EOS M.

I've also only tested the Sandisk extreme pro 95 MB/s on eos m and in my case haven't suffered a crash or any problems with footage yet, though I'm only shooting 24fps. Maybe for safety it would be nice to make a list of cards that work fine with this build... and recommend that those cards be used?

@Danne I agree it would be nice to focus on the 100D and eos m first. They're the cheaper cameras and people seem to be having success with them.

I'm particularly interested in the eos m because it's the only magic lantern camera on which we'll be able to use a speed booster soon (Viltrox seem to be planning to release a proper EF-M to EF focal reducer in the summer). That means that in 5x zoom mode, it will be possible to go wide angle using aps-c lenses + speed booster.

If we want to go further, in the "crop_rec on steroids" thread, @theBilalFakhouri spoke about bypassing the resolution limitation in the 5x zoom mode. See here: https://www.magiclantern.fm/forum/index.php?topic=19300.msg197870#msg197870
Now with SD interface overclocking, reaching those higher resolutions seems more feasible? It would be nice to be able to shoot closer to 16:9 in 2.5k. He was testing on a 700D and one of the resolutions he was able to get working was 2520x1386 at 23.976fps, which sounds incredible.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 13, 2018, 03:40:21 PM
@alpicat
Did you try any changes with adtg module to increase resolution? Big post and sure would take some time and fiddling to get it right.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on May 13, 2018, 06:08:57 PM
Quote from: alpicat on May 13, 2018, 12:31:37 PM
If we want to go further, in the "crop_rec on steroids" thread, @theBilalFakhouri spoke about bypassing the resolution limitation in the 5x zoom mode. See here: https://www.magiclantern.fm/forum/index.php?topic=19300.msg197870#msg197870
Now with SD interface overclocking, reaching those higher resolutions seems more feasible? It would be nice to be able to shoot closer to 16:9 in 2.5k. He was testing on a 700D and one of the resolutions he was able to get working was 2520x1386 at 23.976fps, which sounds incredible.

I fully agree with you, Alpicat.  A resolution ot 2520x1386 would be a dream come true for the 100D which has the same senzor size and resolution as the 700D.  I have been using the 7D at 2520x1200 resolution a lot lately and believe it or not, the 120 pixel larger vertical resolution of this camera, compared to the 100D, makes a huge difference in the overall vision of the video.  A 16:9 vision at 2520x1386 would be so much better ... But I am dreaming again.

I have a question for you.  You mentioned the speed booster for increased view angle with EF-S lenses on APSC-sensor based mirrorless.  Would it be possible to use such speed boosters for the same purpose but with 1,6x crop sensor DSLRs operating in the LifeView (video recording) mode, with full-frame lenses, of course?  If not, why not?  And if yes, are such speed boosters available?  Sorry for the stupid question but I have never used and even seen a speed booster sofar.
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on May 13, 2018, 08:24:50 PM
Quote from: Danne on May 13, 2018, 03:40:21 PM
@alpicat
Did you try any changes with adtg module to increase resolution? Big post and sure would take some time and fiddling to get it right.

I haven't tried any of this since I don't have any technical knowledge. However, will research this adtg module and see if I can understand it. Looks like there's quite a bit of info about it in the crop_rec on steriods thread. @theBilalFakhouri seemed to get quite far with this on his 700D, and wondering if what he discovered would also apply to the EOS M and 100D, since they're pretty similar cameras... although maybe this discussion is a bit off topic in this thread!

Quote from: IDA_ML on May 13, 2018, 06:08:57 PM
I have a question for you.  You mentioned the speed booster for increased view angle with EF-S lenses on APSC-sensor based mirrorless.  Would it be possible to use such speed boosters for the same purpose but with 1,6x crop sensor DSLRs operating in the LifeView (video recording) mode, with full-frame lenses, of course?  If not, why not?  And if yes, are such speed boosters available?  Sorry for the stupid question but I have never used and even seen a speed booster sofar.

If you're shooting with the full aps-c sensor area, adding a speed booster would give the EOS M the same effective field of view and DOF as a full frame sensor, whilst increasing the lens aperture by 1 stop. At the moment you can get cheap Chinese speed boosters on ebay which have poor optics and no electronic contacts. However, Viltrox told me in an email that they will be releasing a proper speed booster for EF-M mount in around 3 months or so. That's exciting. It'll turn the EOS M into a full frame camera.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on May 14, 2018, 02:49:55 PM
Quote from: alpicat on May 13, 2018, 08:24:50 PM
It'll turn the EOS M into a full frame camera.

Well, my question was actually about APSC-sensor based DSLRs.  Can a speed booster turn them into full-frame cameras when operating in Life View with full-frame lenses?  Do such speed boosters exist and if yes, does anyone on this forum have any experience with them? 
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on May 14, 2018, 03:25:39 PM
Quote from: IDA_ML on May 14, 2018, 02:49:55 PM
Well, my question was actually about APSC-sensor based DSLRs.  Can a speed booster turn them into full-frame cameras when operating in Life View with full-frame lenses?  Do such speed boosters exist and if yes, does anyone on this forum have any experience with them? 

Sorry I misread your question. Unfortunately that's optically impossible. The flange distance (between sensor and EF mount) on EOS DSLRs is 44mm so there's no space for a speedbooster. You could hack one and put it inside the camera housing, but you'd have to take your camera apart and remove the mirror mechanism. On the other hand, EF-M mount flange distance is only 18mm (as it's mirrorless), so you can add a speedbooster in front of that mount.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on May 14, 2018, 03:43:23 PM
I tend to disagree. With same logic applied you may prove teleconverters are impossible to use on Canon cams. IMO you mix up two concepts: Extending flange distance without optical elements (extenders, bellows) and inserting an optical element (teleconverter, speedbooster, mount adapters with lenses) into optical path.

It's another question if there is a company seeing a market for such an adapter. My 2 cents: Nope.
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on May 14, 2018, 05:44:47 PM
@Walter Schulz, that's a very good point!
Title: Re: UHS-I / SD cards investigation
Post by: Licaon_Kter on May 18, 2018, 10:54:48 AM
Keeping an eye on this too
Title: Re: UHS-I / SD cards investigation
Post by: ArcziPL on May 18, 2018, 12:46:11 PM
Hello, does anyone have a Samsung Evo Plus microSD card and did or could do the benchmarking? It seems to be a promising alternative. Generally it is also able to achieve approx. 90MB/s bulk write speeds but at much lower cost than SanDisk Extreme Pro. As an example: 128GB Samsung Evo Plus is at Amazon equally priced to 64GB SanDisk Extreme Pro (39€ vs. 38,99€). Both are sold by Amazon, so the risk of counterfeit should be equally low.

Funnily, according to some sources (https://www.allesbeste.de/test/die-beste-micro-sd-karte/), the transfer drops by roughly a half when using the provided MicroSD -> SD adapter. If another adapter can overcome this bottleneck and if similarly high speeds can be achieved with an EOS camera it for me unknown so far. It will decide if it's really an alternative. If yes, it would be a bargain.
Title: Re: UHS-I / SD cards investigation
Post by: Sapporo on May 18, 2018, 05:56:12 PM
Quote from: ArcziPL on May 18, 2018, 12:46:11 PM
Hello, does anyone have a Samsung Evo Plus microSD card and did or could do the benchmarking? It seems to be a promising alternative. Generally it is also able to achieve approx. 90MB/s bulk write speeds but at much lower cost than SanDisk Extreme Pro. As an example: 128GB Samsung Evo Plus is at Amazon equally priced to 64GB SanDisk Extreme Pro (39€ vs. 38,99€). Both are sold by Amazon, so the risk of counterfeit should be equally low.

Funnily, according to some sources (https://www.allesbeste.de/test/die-beste-micro-sd-karte/), the transfer drops by roughly a half when using the provided MicroSD -> SD adapter. If another adapter can overcome this bottleneck and if similarly high speeds can be achieved with an EOS camera it for me unknown so far. It will decide if it's really an alternative. If yes, it would be a bargain.
Samsung PRO 32GB here. 41MB/s default with 6D. 65MB/s overclocked.
Title: Re: UHS-I / SD cards investigation
Post by: kye on May 19, 2018, 07:42:08 AM
Quote from: ArcziPL on May 18, 2018, 12:46:11 PM
Hello, does anyone have a Samsung Evo Plus microSD card and did or could do the benchmarking? It seems to be a promising alternative. Generally it is also able to achieve approx. 90MB/s bulk write speeds but at much lower cost than SanDisk Extreme Pro. As an example: 128GB Samsung Evo Plus is at Amazon equally priced to 64GB SanDisk Extreme Pro (39€ vs. 38,99€). Both are sold by Amazon, so the risk of counterfeit should be equally low.

Funnily, according to some sources (https://www.allesbeste.de/test/die-beste-micro-sd-karte/), the transfer drops by roughly a half when using the provided MicroSD -> SD adapter. If another adapter can overcome this bottleneck and if similarly high speeds can be achieved with an EOS camera it for me unknown so far. It will decide if it's really an alternative. If yes, it would be a bargain.

I have Samsung Evo Plus 256Gb microSD card (in the adapter) and 700D running the crop_rec from 10th March.  I am willing to do some testing but haven't got a clue how to install the UHS module, so would need a lot of hand-holding!
I also have a few adapters that I could try if someone is able to help me set it up.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on May 19, 2018, 08:03:48 AM
@kye:
First question is if there is a penalty involved using SD-card adapters. If you own a decent cardreader you can run Crystaldiskmark (Windows) or Blackmagic Disk Speed Test (MacOS, OS X) to find out.
Don't forget sd_uhs.mo is able to kill memory cards.
Title: Re: UHS-I / SD cards investigation
Post by: kye on May 19, 2018, 08:32:06 AM
Quote from: Walter Schulz on May 19, 2018, 08:03:48 AM
@kye:
First question is if there is a penalty involved using SD-card adapters. If you own a decent cardreader you can run Crystaldiskmark (Windows) or Blackmagic Disk Speed Test (MacOS, OS X) to find out.
Don't forget sd_uhs.mo is able to kill memory cards.

Ok, figured it out!

BM Disk Speed test in Transcend SD card reader:
Samsung EVO Plus with no adapter (directly in microSD slot in card reader) ~83MB/s
Samsung EVO Plus with Red Sandisk adapter ~83MB/s
Samsung EVO Plus with Black Sandisk adapter (from Sandisk Ultra card) ~83MB/s
Samsung EVO Plus with Black Kingston adapter ~83MB/s

Looks like no differences between those adapters anyway.

Log from sd_uhs.mo in 700D:
Quote
===================
Before the hack: r:27MB/s w:31MB/s  W:27MB/s R:38MB/s  8)  [best 31MB/s]
SDR50 @ 96MHz  : r:38MB/s w:33MB/s  W:31MB/s R:38MB/s  :)  [best 33MB/s]
SDR50 @ 96MHz  : r:38MB/s w:33MB/s  W:31MB/s R:37MB/s  ::) [best 33MB/s]
SDR50 @ 80MHz  : r:33MB/s w:29MB/s  W:29MB/s R:32MB/s  meh [best 33MB/s]
SDR50 @ 80MHz  : r:33MB/s w:29MB/s  W:29MB/s R:32MB/s  meh [best 33MB/s]
SDR50 @ 120MHz : r:42MB/s w:32MB/s  W:32MB/s R:45MB/s  ::) [best 33MB/s]
SDR50 @ 120MHz : r:42MB/s w:32MB/s  W:32MB/s R:45MB/s  ::) [best 33MB/s]
SDR104 @ 96MHz : D0 D0 r:37MB/s w:33MB/s  W:31MB/s R:37MB/s  :)  [best 33MB/s]
SDR104 @ 96MHz : D1 D1 r:38MB/s w:33MB/s  W:31MB/s R:37MB/s  ::) [best 33MB/s]
SDR104 @ 80MHz : D0 D0 r:32MB/s w:29MB/s  W:29MB/s R:32MB/s  meh [best 33MB/s]
SDR104 @ 80MHz : D1 D1 r:32MB/s w:30MB/s  W:29MB/s R:26MB/s  meh [best 33MB/s]
SDR104 @ 120MHz: D0 D0 r:42MB/s w:32MB/s  W:33MB/s R:45MB/s  ::) [best 33MB/s]
SDR104 @ 120MHz: D1 D1 r:41MB/s w:32MB/s  W:32MB/s R:45MB/s  ::) [best 33MB/s]
SDR104 @ 132MHz: D0 D0 r:40MB/s w:32MB/s  W:32MB/s R:43MB/s  ::) [best 33MB/s]
SDR104 @ 132MHz: D1 D1 r:40MB/s w:32MB/s  W:32MB/s R:42MB/s  ::) [best 33MB/s]
SDR104 @ 160MHz: D0 D0 r:56MB/s w:36MB/s  W:47MB/s R:42MB/s  :)  [best 36MB/s]
SDR104 @ 160MHz: D1 D1 r:42MB/s w:47MB/s  W:46MB/s R:42MB/s  8)  [best 47MB/s]
SDR104 @ 160MHz: D2 D2 r:55MB/s w:37MB/s  W:47MB/s R:41MB/s  meh [best 47MB/s]
SDR104 @ 160MHz: D3 D3 r:56MB/s w:46MB/s  W:47MB/s R:41MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D0 D0 r:43MB/s w:46MB/s  W:47MB/s R:42MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D1 D1 r:46MB/s w:46MB/s  W:47MB/s R:42MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D2 D2 r:44MB/s w:46MB/s  W:47MB/s R:42MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D3 D3 r:45MB/s w:47MB/s  W:46MB/s R:40MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D0 D0 r:41MB/s w:48MB/s  W:47MB/s R:42MB/s  :)  [best 48MB/s]
SDR104 @ 160MHz: D1 D1 r:42MB/s w:45MB/s  W:47MB/s R:42MB/s  ::) [best 48MB/s]
SDR104 @ 160MHz: D2 D2 r:56MB/s w:47MB/s  W:47MB/s R:42MB/s  ::) [best 48MB/s]
SDR104 @ 160MHz: D3 D3 r:42MB/s w:46MB/s  W:47MB/s R:43MB/s  ::) [best 48MB/s]
SDR104 @ 160MHz: D0 D0 r:42MB/s w:47MB/s  W:47MB/s R:41MB/s  ::) [best 48MB/s]
SDR104 @ 160MHz: D1 D1 r:42MB/s w:48MB/s  W:45MB/s R:41MB/s  ::) [best 48MB/s]
SDR104 @ 160MHz: D2 D2 r:43MB/s w:47MB/s  W:46MB/s R:41MB/s  ::) [best 48MB/s]
SDR104 @ 160MHz: D3 D3 r:43MB/s w:46MB/s  W:46MB/s R:42MB/s  ::) [best 48MB/s]
Best: D0 D0 r:42MB/s w:46MB/s  W:47MB/s R:40MB/s  ::) [best 48MB/s]
Best: D0 D0 r:42MB/s w:45MB/s  W:47MB/s R:40MB/s  ::) [best 48MB/s]

Done.
Please run THOROUGH tests before using!!!

Edit: forgot to mention my Samsung card didn't come with an adapter.
Here's the equipment from this test:

(https://thumb.ibb.co/csrc5o/IMG_1700.jpg) (https://ibb.co/csrc5o)


Edit 2: ran it again and it seemed to change its mind on which settings were the best?  Not sure if this is just the test results varying normally?  (the BM tests would jump around between about 80-83MB/s so I assume it's just what happens with throughput testing.

QuoteBefore the hack: r:38MB/s w:35MB/s  W:32MB/s R:38MB/s  8)  [best 35MB/s]
SDR50 @ 96MHz  : r:38MB/s w:34MB/s  W:34MB/s R:38MB/s  ::) [best 35MB/s]
SDR50 @ 96MHz  : r:38MB/s w:34MB/s  W:33MB/s R:38MB/s  ::) [best 35MB/s]
SDR50 @ 80MHz  : r:33MB/s w:29MB/s  W:29MB/s R:33MB/s  meh [best 35MB/s]
SDR50 @ 80MHz  : r:32MB/s w:30MB/s  W:30MB/s R:32MB/s  meh [best 35MB/s]
SDR50 @ 120MHz : r:45MB/s w:40MB/s  W:39MB/s R:46MB/s  8)  [best 40MB/s]
SDR50 @ 120MHz : r:45MB/s w:38MB/s  W:39MB/s R:43MB/s  ::) [best 40MB/s]
SDR104 @ 96MHz : D0 D0 r:38MB/s w:33MB/s  W:34MB/s R:35MB/s  meh [best 40MB/s]
SDR104 @ 96MHz : D1 D1 r:37MB/s w:34MB/s  W:34MB/s R:37MB/s  meh [best 40MB/s]
SDR104 @ 80MHz : D0 D0 r:33MB/s w:30MB/s  W:29MB/s R:32MB/s  meh [best 40MB/s]
SDR104 @ 80MHz : D1 D1 r:32MB/s w:30MB/s  W:30MB/s R:32MB/s  meh [best 40MB/s]
SDR104 @ 120MHz: D0 D0 r:46MB/s w:40MB/s  W:38MB/s R:42MB/s  ::) [best 40MB/s]
SDR104 @ 120MHz: D1 D1 r:46MB/s w:36MB/s  W:40MB/s R:41MB/s  meh [best 40MB/s]
SDR104 @ 132MHz: D0 D0 r:43MB/s w:38MB/s  W:36MB/s R:42MB/s  ::) [best 40MB/s]
SDR104 @ 132MHz: D1 D1 r:43MB/s w:33MB/s  W:33MB/s R:43MB/s  meh [best 40MB/s]
SDR104 @ 160MHz: D0 D0 r:58MB/s w:45MB/s  W:45MB/s R:55MB/s  8)  [best 45MB/s]
SDR104 @ 160MHz: D1 D1 r:55MB/s w:47MB/s  W:47MB/s R:57MB/s  :)  [best 47MB/s]
SDR104 @ 160MHz: D2 D2 r:57MB/s w:47MB/s  W:47MB/s R:57MB/s  :)  [best 47MB/s]
SDR104 @ 160MHz: D3 D3 r:53MB/s w:47MB/s  W:47MB/s R:57MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D0 D0 r:58MB/s w:47MB/s  W:47MB/s R:57MB/s  :)  [best 47MB/s]
SDR104 @ 160MHz: D1 D1 r:56MB/s w:47MB/s  W:47MB/s R:58MB/s  :)  [best 47MB/s]
SDR104 @ 160MHz: D2 D2 r:57MB/s w:46MB/s  W:46MB/s R:54MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D3 D3 r:57MB/s w:45MB/s  W:46MB/s R:54MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D0 D0 r:55MB/s w:47MB/s  W:47MB/s R:57MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D1 D1 r:56MB/s w:44MB/s  W:43MB/s R:55MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D2 D2 r:57MB/s w:42MB/s  W:46MB/s R:54MB/s  meh [best 47MB/s]
SDR104 @ 160MHz: D3 D3 r:58MB/s w:47MB/s  W:47MB/s R:56MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D0 D0 r:56MB/s w:44MB/s  W:43MB/s R:57MB/s  ::) [best 47MB/s]
SDR104 @ 160MHz: D1 D1 r:54MB/s w:48MB/s  W:46MB/s R:45MB/s  :)  [best 48MB/s]
SDR104 @ 160MHz: D2 D2 r:55MB/s w:45MB/s  W:38MB/s R:55MB/s  ::) [best 48MB/s]
SDR104 @ 160MHz: D3 D3 r:56MB/s w:47MB/s  W:46MB/s R:55MB/s  ::) [best 48MB/s]
Best: D1 D1 r:55MB/s w:47MB/s  W:46MB/s R:58MB/s  ::) [best 48MB/s]
Best: D1 D1 r:57MB/s w:43MB/s  W:47MB/s R:46MB/s  meh [best 48MB/s]
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 19, 2018, 10:22:35 AM
I have put up a hardcoded 160MHZ 700D build in my repository to be tested. It´s the same as for 100D and eosm. If anybody tests this, please feedback here what´s working or not. Fried cards or not etc...
On top of downloads page:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/
Title: Re: UHS-I / SD cards investigation
Post by: kye on May 19, 2018, 11:52:52 AM
Quote from: Danne on May 19, 2018, 10:22:35 AM
I have put up a hardcoded 160MHZ 700D build in my repository to be tested. It´s the same as for 100D and eosm. If anybody tests this, please feedback here what´s working or not. Fried cards or not etc...
On top of downloads page:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/

I think I might have stuffed up..  do I need to be in photo mode when I run the module?  I've just run it in there and gotten much better results than I posted before!   ::)

Also, should I replace your previous module with the hard-coded one?  What's the difference?

Also also, with the non-hard-coded one do I have to run it every time I turn on the camera?  or does it remember the best settings from last time?  I tried a couple of tests and couldn't get a straight answer.

Sorry for all the questions...  hopefully I'm helping more than getting in the way  :D
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 19, 2018, 12:14:21 PM
Good questions. This topic has started and the module is tested atm. If it's here to stay, who knows.
This version has some early code based on some code erased in a1ex earlier code. As you say, run the module in photo mode or with raw_rec turned off. Switch back to movie mode and you should be able to record at higher speeds. You have to run this set up every time you restart your camera unfortunately. Maybe if A1ex thinks it safe we could have the patch hard coded upon start up...
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on May 19, 2018, 12:24:36 PM
To verify overclocking is working in movie mode: Load Bench.mo, too, run sd_uhs in photo mode, switch to movie mode and run card benchmark (debug tab). Numbers will be lower in movie mode, though.
Title: Re: UHS-I / SD cards investigation
Post by: ArcziPL on May 19, 2018, 08:14:59 PM
Quote from: Sapporo on May 18, 2018, 05:56:12 PM
Samsung PRO 32GB here. 41MB/s default with 6D. 65MB/s overclocked.
Quote from: kye on May 19, 2018, 08:32:06 AM
Ok, figured it out!

BM Disk Speed test in Transcend SD card reader:
Samsung EVO Plus with no adapter (directly in microSD slot in card reader) ~83MB/s
Samsung EVO Plus with Red Sandisk adapter ~83MB/s
Samsung EVO Plus with Black Sandisk adapter (from Sandisk Ultra card) ~83MB/s
Samsung EVO Plus with Black Kingston adapter ~83MB/s

Looks like no differences between those adapters anyway.

Quote from: kye on May 19, 2018, 08:32:06 AM
Log from sd_uhs.mo in 700D:
Before the hack: r:27MB/s w:31MB/s  W:27MB/s R:38MB/s  8)  [best 31MB/s]
(...)
SDR104 @ 160MHz: D1 D1 r:42MB/s w:47MB/s  W:46MB/s R:42MB/s  8)  [best 47MB/s]
Sapporo, kye, thank you very much for sharing your results! So there is no problem with adapters, Samsung Evo Plus is a decent card but only twice more expensive Samsung Pro is on par with SanDisk Extreme Pro when using with 700D. Good to know!
Title: Re: UHS-I / SD cards investigation
Post by: Tullen on May 20, 2018, 12:49:36 AM
Quote from: IDA_ML on May 14, 2018, 02:49:55 PM
Well, my question was actually about APSC-sensor based DSLRs.  Can a speed booster turn them into full-frame cameras when operating in Life View with full-frame lenses?  Do such speed boosters exist and if yes, does anyone on this forum have any experience with them?

Actually I already made a thread about it. I have a speedboaster, an extra 50D and some ideas about how it can be done, just haven't got to it yet. https://www.magiclantern.fm/forum/index.php?topic=13957.msg134615#msg134615
Title: Re: UHS-I / SD cards investigation
Post by: kye on May 20, 2018, 01:41:12 PM
Why is there a difference in the write speed between photo and video modes?

My Samsung Evo Plus gets ~47MB/s in video mode but ~68MB/s in photo mode in my 700D.

Is there a CPU overhead in video mode perhaps?  The reason I ask is that it seems like it might be something that can be optimised for performance improvement.
Title: Re: UHS-I / SD cards investigation
Post by: bakersdozen on May 23, 2018, 02:54:40 AM
Quote from: Danne on May 19, 2018, 10:22:35 AM
I have put up a hardcoded 160MHZ 700D build in my repository to be tested. It´s the same as for 100D and eosm. If anybody tests this, please feedback here what´s working or not. Fried cards or not etc...
On top of downloads page:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/

Hi @Danne, thanks for your work on hardcoding the overclock. Really saves time for testing purposes and then to commit to shooting - well done. Any chance you could do the same for 70D? (combine dfort's crop_rec_4k.57614b3.2018Mar14.70D112.zip and your 160MHZ hardcode)

Such exciting times for ML with these constant advancements.
Title: Re: UHS-I / SD cards investigation
Post by: andy kh on May 23, 2018, 05:41:36 AM
Quote from: bakersdozen on May 23, 2018, 02:54:40 AM
Any chance you could do the same for 70D? (combine dfort's crop_rec_4k.57614b3.2018Mar14.70D112.zip and your 160MHZ hardcode)

dfort's crop-rec doesnt work yet
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 23, 2018, 12:17:34 PM
Since hardcoding sd_uhs.mo seems a bit far off here is a lua snippet which will start the SD overclock patching upon starting the camera. Lua module has to be enabled and the script needs to be set to Autorun to ON.
Put following in a file with the extension .lua and put it in the scripts folder.
-- enable SD overclocking

    if menu.get("Debug", "SD overclock", "OFF")
    then
    console.show()
    print("This will enable SD overclock")
    msleep(2000)
    menu.set("Debug", "SD overclock", "ON")
    else
    console.show()
    print("Please enable sd_uhs.mo before running this script again.")
    msleep(5000)
    console.hide()
    return
    end


I put up two new builds (23may) for eosm and eos100D. Use with caution!:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/
Title: Re: UHS-I / SD cards investigation
Post by: bakersdozen on May 23, 2018, 12:24:18 PM
Quote from: andy kh on May 23, 2018, 05:41:36 AM
dfort's crop-rec doesnt work yet

Working fine for me on 70D so far, need to get the 95 sandisk SD to keep pushing the limits - I've only got the 90 MicroSD with adapter currently. Still able to get some good speeds out of it though! And I can change resolution higher than 1728, so definitely working!

Quote from: Danne on May 23, 2018, 12:17:34 PM
Since hardcoding sd_uhs.mo seems a bit far off here is a lua snippet which will start the SD overclock patching upon starting the camera. Lua module has to be enabled and the script needs to be set to Autorun to ON.
Put following in a file with the extension .lua and put it in the scripts folder.
-- enable SD overclocking

    if menu.get("Debug", "SD overclock", "OFF")
    then
    console.hide()
    menu.set("Debug", "SD overclock", "ON")
    else
    console.show()
    print("Please enable sd_uhs.mo before running this script again.")
    msleep(5000)
    console.hide()
    return
    end



Thanks @Danne ! exactly what I needed for now. Much appreciated and kind regards. More benchmarks from 70D to follow once new card arrives.
I will also try your new EOSM build tomorrow.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 23, 2018, 12:34:38 PM
My builds are specifically for faster cards and I don't know how the lua script will react to sd_uhs.mo with full test run procedure(full code).
Anyhow, needs testing/reporting and so on...

Edit:moved console.hide() to after patching.
Title: Re: UHS-I / SD cards investigation
Post by: bakersdozen on May 23, 2018, 01:03:09 PM
@Danne, the .lua script you added works fine with Autorun on startup. No problems there, thanks! Just have to remember to be in photo mode when turning on the camera.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 23, 2018, 01:14:49 PM
Cool.
I think it worked on my 100D going straight to movie mode. Did you try?
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 23, 2018, 02:19:14 PM
Changed lua to following:
-- enable SD overclocking

    if menu.get("Debug", "SD overclock", "OFF")
    then
    console.show()
    print("This will enable SD overclock")
    msleep(2000)
    menu.set("Debug", "SD overclock", "ON")
    else
    console.show()
    print("Please enable sd_uhs.mo before running this script again.")
    msleep(5000)
    console.hide()
    return
    end


Better to have some sort of indication what´s going on when patching...

In the provided builds enable lua.mo then run then script enable SD overclocking and set it to Autorun ON for patching upon starting your camera.
New uploads:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/
Title: Re: UHS-I / SD cards investigation
Post by: andy kh on May 23, 2018, 03:33:10 PM
Quote
Working fine for me on 70D so far

it never work on mine. u should report in 70D forum. dfort wil be happy to know that. he worked so hard but no luck for most of us
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 23, 2018, 04:27:14 PM
@andy_kh
Above quote is coming from bakersdozen, not danne(me).
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on May 23, 2018, 06:57:53 PM
Not lua Savvy here so guess I'm missing something > Can't get it to work. The only thing showing under "Scripts" is "Show Console" .
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on May 23, 2018, 07:07:04 PM
       Also just noticed that the M'L' Menu no longer does the "Off/On" routine while in LiveView on Movie Modes. When did that get resolved. Guess I haven't been paying attention as well as I thought. Must be getting old > Oh Wait > I am Old .
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 23, 2018, 07:11:00 PM
If you use any of the provided builds on my downloads page the script is already included.
Activate lua.mo, mlv_lite, sd_uhs.mo etc, restart camera and go find the lua script called enable SD overclocking and run it and set it to Autorun ON.

If you create the script yourself paste the info to a file and give it the extension .lua Put the script in scripts. The folder exists in magic lantern build folder ML -> scripts -> SD_overclock.lua.

Alex fixed the flicker(on/off screen) issue but did not include it(I think) so I put it in this build as well. Good catch.
Commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/5ee19676453e3f12dbd6cfa6399db4219f1d6463?at=crop_rec_4k_mlv_lite_snd_sd_uhs_HDR_extended

I´m working this branch:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_lite_snd_sd_uhs_HDR_extended
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on May 23, 2018, 07:19:47 PM
       Using "magiclantern-Nightly.2018May23.100D101_sd_uhs" Mod's loaded as You describe

& can't find "lua script called SD_overclock.lua" .
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on May 23, 2018, 07:38:19 PM
       The "SD_overclock.lua" Script is in the "ML" Folder but does not appear in the "Scripts" Menu

on Cam. Only script showing is "Show console"

       Just clicked "Show console" & dialog @ bot' of screen says >

"skipping SD_overclock.lua (file name to long)"
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 23, 2018, 07:39:56 PM
Sorry, it will say "enable SD overclocking" under Scripts menu.
It's the title coming from inside the lua script. Just enable that and set it to Autorun ON...
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on May 23, 2018, 07:41:13 PM
Changed name to SDOclock.lua & now it shows up ~
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 23, 2018, 07:45:05 PM
Strange. What cam are you using? Eosm or 100D?
Title: Re: UHS-I / SD cards investigation
Post by: wyrlyn on May 23, 2018, 07:45:25 PM
@Danne Could you make me a build for 700D? I want to test it. Thank you.
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on May 23, 2018, 07:49:09 PM
100D
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 23, 2018, 08:02:51 PM
Quote from: wyrlyn on May 23, 2018, 07:45:25 PM
@Danne Could you make me a build for 700D? I want to test it. Thank you.

Out at the moment but grab the sd_uhs.mo from the 100D and replace existing one. Also grab the script from the scripts folder and put in your recent 700D build. Should work...

Edit: actually, you only need the lua script, no need to replace sd_uhs.mo.
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on May 23, 2018, 11:36:28 PM
Quote from: OlRivrRat on May 23, 2018, 07:41:13 PM
Changed name to SDOclock.lua & now it shows up ~

@Danne  I've just tried your new build on my eos m (with Sandisk extreme pro) and it works. Thanks so much for this! I initially had the same issue as @OlRivRat where I had to change the name of the lua file to get it to appear in the scripts menu, but that was easy to fix.

I'm wondering if we still need to run the benchmark test every time the camera is turned on or is this not necessary?
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on May 23, 2018, 11:51:21 PM
Quote from: alpicat on May 23, 2018, 11:36:28 PM
I'm wondering if we still need to run the benchmark test every time the camera is turned on or is this not necessary?

This was never necessary, but the quick tests performed during patching are not sufficient to make sure the preset is reliable.

Besides this, you have to make sure there will be absolutely no other card access during the initial tests. The code is not thread safe and any kind of card access during the inital tests, from either Canon firmware or ML, will lock up the camera and may result in a corrupted filesystem. Should this happen (camera locking up), you must format the card (even if it appears to work fine at first sight, it's almost certainly not OK and some of your files are probably already lost at that point).

I'm not saying this just for kicks; I've actually experienced corrupted filesystems many times while testing this module. If in doubt, please review the code before running it, and make sure you are not using it for anything important. Not joking. There is a reason why this is not available as a regular download.

P.S. more stuff coming soon (in particular, better compatibility with certain Sandisk cards). Still experimenting. No success with thread safety yet.

P.P.S. Not all 95MB/s cards are compatible with the hardcoded 160MHz preset. I've just got two of these.
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on May 24, 2018, 07:14:24 AM
   Not sure if this is a SD SpeedUp issue or an SL1 Issue but today I got some time to do some full testing,

ie, Rec & Process & the resulting images have a Problem >

   Using "magiclantern-Nightly.2018May23.100D101_sd_uhs" on My SL1 I shot several Min's of 2520x1080

& the Resulting DNGs & Vid only have 2352 Useable Width. The Right Hand 168Pxls are Garbled (Horizontal

Streaks & Bands)


(https://thumb.ibb.co/mm3L1T/M23_1803_1_2018_05_23_0001_C0000_000686.jpg) (https://ibb.co/mm3L1T)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 24, 2018, 07:58:40 AM
Quote from: a1ex on May 23, 2018, 11:51:21 PM
This was never necessary, but the quick tests performed during patching are not sufficient to make sure the preset is reliable.

Besides this, you have to make sure there will be absolutely no other card access during the initial tests. The code is not thread safe and any kind of card access during the inital tests, from either Canon firmware or ML, will lock up the camera and may result in a corrupted filesystem. Should this happen (camera locking up), you must format the card (even if it appears to work fine at first sight, it's almost certainly not OK and some of your files are probably already lost at that point).

I'm not saying this just for kicks; I've actually experienced corrupted filesystems many times while testing this module. If in doubt, please review the code before running it, and make sure you are not using it for anything important. Not joking. There is a reason why this is not available as a regular download.

P.S. more stuff coming soon (in particular, better compatibility with certain Sandisk cards). Still experimenting. No success with thread safety yet.

P.P.S. Not all 95MB/s cards are compatible with the hardcoded 160MHz preset. I've just got two of these.

Thanks for shedding some light on this topic, especially on this
Quotemake sure there will be absolutely no other card access during the initial tests.
Looking forward to what you have up your sleeve regarding sandisk cards :).
QuoteP.S. more stuff coming soon (in particular, better compatibility with certain Sandisk cards). Still experimenting. No success with thread safety yet.


Anyone with lua knowledge that could shed light on this?
Quote from: alpicat on May 23, 2018, 11:36:28 PM
@Danne...I initially had the same issue as @OlRivRat where I had to change the name of the lua file to get it to appear in the scripts menu, but that was easy to fix.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on May 24, 2018, 09:48:32 AM
Quote from: a1ex on May 23, 2018, 11:51:21 PM
Besides this, you have to make sure there will be absolutely no other card access during the initial tests. The code is not thread safe and any kind of card access during the inital tests, from either Canon firmware or ML, will lock up the camera and may result in a corrupted filesystem. Should this happen (camera locking up), you must format the card (even if it appears to work fine at first sight, it's almost certainly not OK and some of your files are probably already lost at that point).

That one is new for me, until now, nothing bad happened here.
Does Canon firmware or ML do much random accessing the card ?
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on May 24, 2018, 10:01:00 AM
Generally speaking, card access is quite predictable - look at the LED blinks. For example, if you switch to photo mode during the overclocking tests, Canon firmware may want to read from the card in order to display the last photo. Another example: ML settings are saved every time after you change some setting in ML menu. Or, if you start recording before the tests are completed.

Finally, if you use the Lua script and there are other scripts that happen to auto-load (or read their config files or whatever) during the overclocking tests, you may have a little surprise.

Being a race condition, the outcome is not deterministic; it doesn't mean that you will get a lockup every single time a card access is made during these tests. However, the low-level tests are not thread-safe (we have to tell Canon code not to use the card during these tests, and I don't know how to do that). If the other card access happens in the middle of the low-level write test, for example, the camera will lock up and the sectors written during that test (which may or may not overlap existing files or directories on the card) will contain garbage.
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on May 24, 2018, 12:36:25 PM
@a1ex  Thanks for your work on this and the warning about card access, that's very useful information. So maybe it's safer not to use the Lua script (to autorun the overclock test) as I guess it adds even more to the risk.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 24, 2018, 04:34:55 PM
Added three new builds(eosm, 100D, 700D) top of downloads section:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/

I noticed my embarrasing lua script wasn´t even doing what it should do so rewrote the lot of it. Maybe a little less embarrasing today  :P:
-- enable SD overclocking

    console.hide()
    if menu.get("Debug", "SD overclock", "MAY CAUSE DATA LOSS") == "MAY CAUSE DATA LOSS"
    then
    if menu.get("Movie", "RAW video", "") == "OFF"
    then
    display.notify_box("enabling of SD overclocking")
    menu.set("Debug", "SD overclock", "ON")
    msleep(1000)
    display.notify_box("please wait...")
    msleep(2000)
    display.notify_box("patching...")
    msleep(2000)
    display.notify_box("done!")
    else
    menu.set("Movie", "RAW video", "OFF")
    display.notify_box("enabling of SD overclocking")
    menu.set("Debug", "SD overclock", "ON")
    msleep(1000)
    display.notify_box("please wait...")
    msleep(2000)
    display.notify_box("patching...")
    msleep(3000)
    menu.set("Movie", "RAW video", "ON")
    display.notify_box("done!")
    end
    else
    display.notify_box("Please enable sd_uhs.mo")
    msleep(2000)
    display.notify_box("Please enable sd_uhs.mo")
    msleep(2000)
    return
    end


The script will now look if RAW video is set to off or not and will temporarily turn the setting off while patching, then reeanble RAW video again when done.
I tried meet a condition while RAW video was set to ON but lua seems rather picky here. For instance:
if menu.get("Movie", "RAW video", "") == "ON"
will not work since it also needs what else is written after ON:
if menu.get("Movie", "RAW video", "") == "ON, 1736x584"
will work. Is there some other way to check for when a menu item is on or not?   

Title: Re: UHS-I / SD cards investigation
Post by: a1ex on May 25, 2018, 05:08:18 PM
Lots of examples on this one in api_test.lua (https://bitbucket.org/hudson/magic-lantern/src/lua_fix/scripts/api_test.lua?#api_test.lua-314).
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 25, 2018, 05:20:46 PM
Thank you. Examples needed.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 26, 2018, 02:39:12 PM
Posted new builds for these cams:
magiclantern-Nightly.2018May26.6D116sd_uhs.zip
magiclantern-Nightly.2018May26.70D112sd_uhs.zip
magiclantern-Nightly.2018May26.100D101sd_uhs.zip
magiclantern-Nightly.2018May26.650D104sd_uhs.zip
magiclantern-Nightly.2018May26.700D115sd_uhs.zip
magiclantern-Nightly.2018May26.EOSM202sd_uhs.zip


Find it here(includes enabling SD overclocking patch lua script):
https://bitbucket.org/Dannephoto/magic-lantern/downloads/

As always. Any risks included are your responsiblity, not mine. And the more feedback the merrier...
Title: Re: UHS-I / SD cards investigation
Post by: Tony Weller on May 26, 2018, 03:21:58 PM
magiclantern-Nightly.2018May26.EOSM202sd_uhs.zip

Working fine  :)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on May 26, 2018, 07:27:33 PM
It's also working flawlessly on 700D. Thanks @Danne!
Title: Re: UHS-I / SD cards investigation
Post by: dfort on May 27, 2018, 06:31:29 AM
I also tested Danne's build on the 700D. Super easy to set it up.
It adds to the startup time but other than that it just runs in the background without having to think about it.

Been a while since we've seen card benchmarks so here is a set using the SanDisk Extreme Pro 32GB UHS-I (95/90MB/s read/write speed, video speed C10, U3, V30)

Photo Mode (normal)
(https://farm2.staticflickr.com/1751/40569320930_bbfa72a7d1.jpg) (https://flic.kr/p/24NYnGw)

Photo Mode (with sd_uhs module)
(https://farm2.staticflickr.com/1736/40569321520_c7b2433e1b.jpg) (https://flic.kr/p/24NYnSG)

Movie Mode (normal)
(https://farm2.staticflickr.com/1731/40569321950_92fc9e9560.jpg) (https://flic.kr/p/24NYo17)

Movie Mode (with sd_uhs module)
(https://farm2.staticflickr.com/1732/40569322530_54076a1271.jpg) (https://flic.kr/p/24NYob7)

Of course the real test is how it performs in the wild so I'm loading up Danne's build on all my cards and taking it out on a three week trip. I usually just shoot stills when on vacation but I'll try shooting some raw video this time. Hope to come home with something to share.

[EDIT] Well maybe I won't set this up on all the cards. A 128GB SanDisk card showed pretty much the same speed improvements as the 32GB card of the same make/model but a couple of Komputerbay 64GB cards are showing something very different.

komputerbay 64GB Photo Mode (normal)
(https://farm2.staticflickr.com/1753/28505091188_1a65b9fe5c.jpg) (https://flic.kr/p/KqU3vs)

komputerbay 64GB Photo Mode (with sd_uhs module)
(https://farm2.staticflickr.com/1754/28505019728_e50f421fee.jpg) (https://flic.kr/p/KqTFgo)

The write speed does improve but it is almost half the speed of the SanDisk card. Sorry, another example of you get what you pay for.
Title: Re: UHS-I / SD cards investigation
Post by: andy kh on May 27, 2018, 07:56:04 AM
i think the previous build works better for me on 70D. latest build crash twice. i shoot for an hour(not continues) yesterday with the 25May build without crashing. i wil do more test and report
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 27, 2018, 08:44:57 AM
@andy_kh
Would be nice to know a little more about your crashes. What settings used etc. Reproduceable or not? Any logs?
It´s more or less the same build except for changing this:
https://bitbucket.org/Dannephoto/magic-lantern/commits/19ca4bea793d0fcdb3756ddd21b41b326f4d9c99
By the way. Can you send me your 25May build to look at?
Cannot see anything that would affect the 70D build but who knows.

@dfort
Have a nice trip. Right now I added some more stuff to my own lua script for faster action. Since I mostly play around with the 5x zoom in 2520x1080p setting I put this at the bottom of the script. On my eosm it´s a bit of a hassle getting the 5x zoom, so easy letting lua do it instead:
while camera.mode ~= MODE.MOVIE do
   display.notify_box("enable MOVIE mode")
   msleep(1000)
end

    menu.set("Movie", "RAW video", "ON")   
    camera.shutter.value = 1/50
    camera.iso.value=400
-- ready to enter x5 crop mode
    lv.zoom = 5
-- fixme: doesn't seem to work if you have fine-tuned it
    menu.set("RAW video", "Resolution", 2560)
    menu.set("Overlay", "Global Draw", "OFF")
    menu.set("Display", "Clear overlays", "OFF")
    menu.set("RAW video", "Data format", "12-bit lossless")


Noticed I set resolution to 2560? That´s because my cams don´t have the 2520 in the scroll down menu.

I then manually set FPS override to around 24. Would really like to get that menu item to work with lua too but it seems very unstable. For instance:
menu.set("FPS override", "Desired FPS", "24.002")
Will set 16.003 fps on my cam. Maybe it´s something in cam not updating correctly when going through lua. Needs some more understanding.
https://www.magiclantern.fm/forum/index.php?topic=14828.msg199769#msg199769

Title: Re: UHS-I / SD cards investigation
Post by: 50mm1200s on May 28, 2018, 05:24:03 AM
So, I got a Lexar Pro (1066x) 32GB, and it's able to record only 4 seconds on MLV at 50D's maximum res (without crop, I think it's 1556x10xx or something). I don't know why, but I was not able to record more than these 4 seconds, while my Komputerbay 64GB is able to record up to 2min in the same resolution.

I'm posting this here because I could try anything on this card, since I have no much use for it. If @a1ex or @dford want to fry this one in some crazy experiment, feel free to send me a module for 50D and I'll do it.
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on May 28, 2018, 05:45:46 AM
                    @Danne

           My SL1 has also had a few "Soft Bricks" with 26May Build. Can't say for sure what was in process when it happens >

It's random ~ I'll try to pay closer attention.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on May 28, 2018, 02:19:43 PM
Maybe not really the right topic, BUT I think it fits best here, for the user group.

Danne compiled some nice builds with the simplified SD_UHS overclock module.
And my guess is that the users of this build would be happy if they could select 10bit lossless for all iso's.

Few months ago I altered the MLV_lite module a little so I could select 8/9/10 or 11 bit on every iso.
I was just curious as hell how 8 bit raw looks like on iso 100  ;D :P
And since then I know, shadows can look horrible posterised.
So disclaimer, Alex is right in how he implemented the lossless bit options, that is for maximum quality.
But I think 10 bit lossless is usable for video on all iso's, it's worse then 14 bit ofcourse, but 10 bit raw video isn't a bad option in video world.
I even think the 8 bit lossless can be used for modern art like projects or video clips, it has some posterised style which some could like  :P

So here's the source of this MLV_Lite
https://drive.google.com/file/d/1VcIMktzPpt_c5m1uIDtuU9pf8WlQ1Zow/view?usp=sharing (https://drive.google.com/file/d/1VcIMktzPpt_c5m1uIDtuU9pf8WlQ1Zow/view?usp=sharing)
And here's a compiled version:
https://drive.google.com/file/d/1CJ7hXialyM7dYra6CUYSwyh8Dkl5bkm7/view?usp=sharing (https://drive.google.com/file/d/1CJ7hXialyM7dYra6CUYSwyh8Dkl5bkm7/view?usp=sharing)
And here is how the menu looks like.
(https://farm1.staticflickr.com/882/40599254050_72e2b93463.jpg)

Not sure if it works on all cams, I guess so, maybe Danne could check it out and help you out.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 28, 2018, 02:48:00 PM
Nice Levas. Will check it out. You should work from bitbucket branch stuff for easier overview. Even the sd_uhs speed up patching is coming from your code to begin with :). What branch did you add your changes in mlv_lite.c by the way? The one with audio working hopefully?

If you want to check 8bit output you can actually do that directly with mlv_dump(on steroids) since buncyball got that feature working a while ago with -b option. So just pass a 14bit file and select -b 8 and you´ll see how shitty it looks ;).

Edit: checking hg diff it seems you worked from the audio branch...
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 28, 2018, 03:18:19 PM
Bunch of new builds uploaded, thanks to Levas. Check two posts above. Not something I missed but could be good to have in the future:
magiclantern-Nightly.2018May28.6D116sd_uhs_allbits.zip
magiclantern-Nightly.2018May28.70D112sd_uhs_allbits.zip
magiclantern-Nightly.2018May28.100D101sd_uhs_allbits.zip
magiclantern-Nightly.2018May28.650D104sd_uhs_allbits.zip
magiclantern-Nightly.2018May28.700D115sd_uhs_allbits.zip
magiclantern-Nightly.2018May28.EOSM202sd_uhs_allbits.zip

All builds in this downloads section:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/

Commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/9b53eaef80680d3f131d1309ab62184267f654b1

new branch:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_lite_snd_sd_uhs_HDR_ext_all_bits

I encourage testers to compile for themselves. For mach users I simplified compiling with this tool:
Compiler.app
https://www.magiclantern.fm/forum/index.php?topic=21882.msg199370#msg199370
Title: Re: UHS-I / SD cards investigation
Post by: andy kh on May 28, 2018, 04:23:26 PM
Lossless isnt working for the 70D yet. But stil i wil test the new build
Title: Re: UHS-I / SD cards investigation
Post by: Levas on May 28, 2018, 09:05:53 PM
@Danne, wow you're quick  8)
If I'm not mistaken I altered the mlv_lite in March, from the crop_rec_4K build, so indeed with audio.
It should be almost the same as the crop_rec_4K builds on the experiments page here on the ML site (which are from March 10th 2018).

Quote from: Danne on May 28, 2018, 02:48:00 PM
If you want to check 8bit output you can actually do that directly with mlv_dump(on steroids) since buncyball got that feature working a while ago with -b option. So just pass a 14bit file and select -b 8 and you´ll see how shitty it looks ;).

Damn, there was an easier way of checking how 8 bit raw looks in iso 100  :o

I must check on the options in MLV_dump more often, I recently find out that you could also export in lossless dng's instead of the full size dng's, gonna be saving a lot of diskspace here  :)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 28, 2018, 09:28:23 PM
Yes, lossless option works great. If I'm not mistaken much of the code is derived from mlrawviewer compression code A Baldwin created. Bouncyball refined some code implementation from "footage app" guy and then polishing, polishing...
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on May 29, 2018, 10:11:46 PM
Quote from: a1ex on April 03, 2018, 12:55:26 PM
Ran a brute force (random) search for the above registers, and...

SDR104 @ 160MHz 8)


static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 };   /* SDR50 values from 700D (96MHz) */
static uint32_t sdr_80MHz[]    = {        0x3,        0x3,                             0x5, 0x1D000401,        0x0,      0x201,      0x201,      0x100,        0x5 };   /* underclocked values: 80MHz = 96*(4+1)/(5+1) */
static uint32_t sdr_120MHz[]   = {        0x3,        0x3,                             0x3, 0x1D000201,        0x0,      0x201,      0x201,      0x100,        0x3 };   /* overclocked values: 120MHz = 96*(4+1)/(3+1) */
static uint32_t sdr_132MHz[]   = {        0x2,        0x2,                             0x2, 0x1D000201,        0x0,      0x100,      0x100,      0x100,        0x2 };   /* overclocked values: 132MHz?! (found by brute-forcing) */
static uint32_t sdr_160MHz[]   = {        0x2,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };   /* overclocked values: 160MHz = 96*(4+1)/(2?+1) (found by brute-forcing) */


Maybe a dumb question

@a1ex Can you explain what the math of this calculations ?
e.g. How in 120 MHz mode did you calculated the values from the registers ?  /* overclocked values: 120MHz = 96*(4+1)/(3+1) */
Which register have the value 96 ? and why did you multiply it with (4+1) ? and what is the registers for (4+1) ? etc..

Can you explain what the math ? How and what's happening ?
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on May 29, 2018, 10:40:59 PM
See reply #41. At first, things appear to make sense, but once you start tweaking some more registers, things are no longer obvious and frequency was guessed from maximum read speed.

The presets I've tried last week (take with a grain of salt; not yet double-checked):

static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4,               0    };   /* SDR50 values from 700D (96MHz) */
static uint32_t sdr_96MHz_b[]  = {        0x3,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,        0x0,        0x4,               0    };

static uint32_t sdr_80MHz_a[]  = {        0x3,        0x3,                             0x5, 0x1D000401,        0x0,      0x201,      0x201,        0x0,        0x5,               0    };   /* underclocked values: 80MHz = 96*(4+1)/(5+1) */

static uint32_t sdr_120MHz_a[] = {        0x3,        0x3,                             0x3, 0x1D000201,        0x0,      0x201,      0x201,      0x100,        0x3,               0    };   /* overclocked values: 120MHz = 96*(4+1)/(3+1) */
static uint32_t sdr_120MHz_b[] = {        0x3,        0x3,                             0x3, 0x1D000201,        0x0,      0x201,      0x201,        0x0,        0x3,               0    };

static uint32_t sdr_132MHz_a[] = {        0x2,        0x2,                             0x2, 0x1D000201,        0x0,      0x100,      0x100,      0x100,        0x2,               0    };   /* overclocked values: 132MHz?! */
static uint32_t sdr_132MHz_b[] = {        0x5,        0x2,                             0x2, 0x1D000001,        0x0,      0x201,      0x301,      0x100,        0x2,               0    };   /* overclocked values: this one works on SanDisk Extreme Pro 32GB 95MB/s V30 (5D3) */
static uint32_t sdr_132MHz_c[] = {        0x2,        0x2,                             0x2, 0x1D000200,        0x1,      0x101,      0x100,    0x10100,    0x10003,               0    };

static uint32_t sdr_160MHz_a[] = {        0x2,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1,               0    };   /* overclocked values: 160MHz = 96*(4+1)/(2?+1) (found by brute-forcing) */
static uint32_t sdr_160MHz_b[] = {        0x3,        0x3,                             0x2, 0x1D000200,        0x1,      0x301,      0x100,    0x10100,    0x10003,               0    };
static uint32_t sdr_160MHz_c[] = {        0x3,        0x3,                             0x2, 0x1D000200,        0x1,      0x301,      0x100,    0x10000,    0x10003,               0    };
static uint32_t sdr_160MHz_d[] = {        0x3,        0x3,                             0x2, 0x1D000200,        0x1,      0x301,      0x100,        0x0,    0x10003,               0    };
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on May 31, 2018, 09:31:22 AM
@a1ex

I'd like to play with values without changing it in the code and recompiling it every time.

Can I adjust these registers by adtg_gui? if not, Can anyone knows the coding better than me make these registers adjustable via sd_uhs with a submenu ? then we can run an overclocking with the settings we did  :o
Title: Re: UHS-I / SD cards investigation
Post by: dinobike on June 07, 2018, 09:05:29 PM
Had anyone tried the hack with this one??
(https://thumb.ibb.co/mqy5ho/memoria_sandisk_32gb_extreme_pro_sdhc_uhs_ii.jpg) (https://ibb.co/mqy5ho)

Title: Re: UHS-I / SD cards investigation
Post by: Sapporo on June 08, 2018, 09:47:25 AM
Quote from: dinobike on June 07, 2018, 09:05:29 PM
Had anyone tried the hack with this one??
(https://thumb.ibb.co/mqy5ho/memoria_sandisk_32gb_extreme_pro_sdhc_uhs_ii.jpg) (https://ibb.co/mqy5ho)

UHS-II, SanDisk and Canon? Slow combination.
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on June 08, 2018, 06:32:08 PM
           @Sapporo

Why is that?
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on June 08, 2018, 06:53:58 PM
@OlRivrRat  That's a UHS-II card. Canon cameras use UHS-I, so there's no point trying an expensive UHS-II card - I'm guessing they'd end up being slower in a Canon than a fast UHS-I card. Although I don't know if anyone's tested this.

@Sapporo  Wondering if there's a faster alternative to Sandisk cards, in particular to their Extreme Pro 95mb/s V30 UHS-I SD cards? I've tested two of those and the max write speed I can get on an EOS M is 55mb/s. I've seen people get faster benchmark results but unsure exactly which cards they were using.
Title: Re: UHS-I / SD cards investigation
Post by: Sapporo on June 08, 2018, 08:39:44 PM
Quote from: OlRivrRat on June 08, 2018, 06:32:08 PM
           @Sapporo

Why is that?
Check yourself http://www.cameramemoryspeed.com/canon-7d-mark-ii/fastest-sd-cf-card-comparison/

SanDisk Extreme Pro 280MB/s UHS-II 32GB 39.8MB/s average speed with the 7D II.
Title: Re: UHS-I / SD cards investigation
Post by: Sapporo on June 08, 2018, 08:40:39 PM
Quote from: alpicat on June 08, 2018, 06:53:58 PM
@OlRivrRat  That's a UHS-II card. Canon cameras use UHS-I, so there's no point trying an expensive UHS-II card - I'm guessing they'd end up being slower in a Canon than a fast UHS-I card. Although I don't know if anyone's tested this.

@Sapporo  Wondering if there's a faster alternative to Sandisk cards, in particular to their Extreme Pro 95mb/s V30 UHS-I SD cards? I've tested two of those and the max write speed I can get on an EOS M is 55mb/s. I've seen people get faster benchmark results but unsure exactly which cards they were using.

I got 71MB/s with 6D and SanDisk Extreme Pro 95MB/s 64GB.
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on June 08, 2018, 09:13:17 PM
Quote from: Sapporo on June 08, 2018, 08:40:39 PM
I got 71MB/s with 6D and SanDisk Extreme Pro 95MB/s 64GB.

That's very impressive - maybe it's something to do with the EOS M rather than the card. I have this same card, and also tried another one which was the same.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 08, 2018, 10:05:36 PM
@alpicat

You are maybe running the benchmark in video mode -- same thing in the other cameras about ~55mb/s  write speed.
But in Play mode you should got at least 65mb/s write speed with Sandisk Extreme Pro 95mb/s U3 32GB which I use.

However -- Using fps override at 12 fps or less in video mode you will got a higher write speeds like play mode --> So more FPS I think it will make a CPU overhead which will affect in the write speeds especially like 720p @ 60 fps ~40mb/s.
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on June 09, 2018, 07:15:57 PM
      @Sapporo & Alpicat

   Can certainly understand that the UHS-II SDs are unnecessary in Our EOSs.

I was curious as to why the "Combo" of "UHS-II, SanDisk and Canon" would be thought to be "Slow".
Title: Re: UHS-I / SD cards investigation
Post by: Sapporo on June 10, 2018, 01:02:37 AM
Quote from: OlRivrRat on June 09, 2018, 07:15:57 PM
      @Sapporo & Alpicat

   Can certainly understand that the UHS-II SDs are unnecessary in Our EOSs.

I was curious as to why the "Combo" of "UHS-II, SanDisk and Canon" would be thought to be "Slow".
You have the write speed in the link. Lexar UHS-II has not the same issue.
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on June 11, 2018, 06:23:39 PM
@theBilalFakhouri thanks for this - running the benchmark with fps override at less than 12fps now gives me around 62mb/s write speed. Unsure how to use benchmark testing whilst in Play mode on the EOS M though, since I can't access the magic lantern menu in that mode.

Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 12, 2018, 09:01:26 PM
@alpicat
Run the benchmark then press PLAY button.
Title: Re: UHS-I / SD cards investigation
Post by: alpicat on June 13, 2018, 12:07:14 AM
@theBilalFakhouri aha, that's better. Now getting 68.7mb/s write speed (72mb/s read speed). Thanks so much!
Title: Re: UHS-I / SD cards investigation
Post by: Lolignon on July 13, 2018, 08:35:23 PM
Hi,
I tried the build magiclantern-Nightly.2018Jun27.600D102HDR_ext.zip from https://bitbucket.org/Dannephoto/magic-lantern/downloads/ (https://bitbucket.org/Dannephoto/magic-lantern/downloads/),
especially looking to use the sd_uhs module on my 600D, but when I selected it to be loaded on next reboot,
it gave me the following message: "FIXME: unsupported model" on boot,
and despite having no problem recording video in the regular video mode (default Canon's mode),
when the mlv_lite module was enabled and while trying to rec some raw footage, the console showed up with an error about data corruption and compression error

Some pics:

(https://thumb.ibb.co/md7ibo/IMG_20180713_201837_HHT.jpg) (https://ibb.co/md7ibo)

(https://thumb.ibb.co/jAEUGo/IMG_20180713_202120.jpg) (https://ibb.co/jAEUGo)

(https://thumb.ibb.co/h0OpGo/IMG_20180713_202527.jpg) (https://ibb.co/h0OpGo)

(https://thumb.ibb.co/iusc2T/IMG_20180713_202540.jpg) (https://ibb.co/iusc2T)


PS:I'm using a Sandisk extreme pro 95mb/s uhs-1 sd card
and sorry for my bad English
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 14, 2018, 10:48:38 AM
The 600D isn't supported when using the sd_uhs module.
Title: Re: UHS-I / SD cards investigation
Post by: Lolignon on July 15, 2018, 09:03:36 PM
Ok, will be supported?
Title: Re: UHS-I / SD cards investigation
Post by: A.Werfhorst on July 19, 2018, 01:13:34 PM
Quote from: Lolignon on July 15, 2018, 09:03:36 PM
Ok, will be supported?
Guys, correct me if I am wrong, as I am mostly a reader who follows this with interest.

If I am correct the investigation currently mostly focuses on the SD hardware that is available in the 650D and newer models. So for optimizing write speeds in older models someone should start a similar investigation, if we want any progress for those.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 19, 2018, 02:34:12 PM
Best you can do is check into the sd_uhs code. First check the bottom what cameras are supported and take it from there. Probably possible to find registers needed but level of difficulty I have no idea about...
Title: Re: UHS-I / SD cards investigation
Post by: dfort on July 19, 2018, 03:30:00 PM
Quote from: A.Werfhorst on July 19, 2018, 01:13:34 PM
...So for optimizing write speeds in older models someone should start a similar investigation, if we want any progress for those.

It might be possible but the method will probably be different.

Quote from: a1ex on April 04, 2018, 11:52:19 AM
I doubt - D4 uses only 2 bits (http://magiclantern.wikia.com/wiki/Register_Map#Timer.2FClock_Module) for clock speed (0xC0400004). Feel free to prove me wrong (possibly by brute-forcing the other bits).

1300D has a string that explains these 2 bits: SetSdClock %d ->0:210K 1:16MHz 2:24MHz 3:48MHz

We often group all Digic 4 cameras in the "older models" category but the 1300D has a Digic 4+ processor that is newer than the Digic 5. In fact if the introduction dates listed on Wikipedia (https://en.m.wikipedia.org/wiki/DIGIC) can be trusted the Digic 4+ is even newer than the Digic 6.

Anyway, I looked for the stubs on an old Digic 4 but came up a little short:

Quote from: dfort on April 04, 2018, 05:25:47 PM
I looked at the stubs for the 7D and found just one. Strange thing is that camera only has one CF slot and no SD slot yet it has "SD_ReConfiguration" -- wonder what it does?

As far as I know UHS-I doesn't apply to CF cards so that was probably going to be a dead end anyway but I also came up empty searching for these stubs on other Digic 4 cameras.
Title: Re: UHS-I / SD cards investigation
Post by: barbona on August 10, 2018, 07:13:49 PM
Hello, could anyone point me to the correct link to download the correct "sd_uhs.mo" module for a 6d. I've tried one (dump the module file in the july 3 nightly build) but it gives a error.

Sorry for my english. Thanks!
Title: Re: UHS-I / SD cards investigation
Post by: reddeercity on September 11, 2018, 07:54:48 AM
Looking at CF card timing again , now that the 5d2 can record 3k but not continuous .
dm-spy.log

825FE>  CSMgrTask:00095f98:00:00: *** register_interrupt("CFDriver", 0x82, 0xffb8b8cc, 0x0), from ffb8bb58
834A0>  CSMgrTask:ffb8a92c:22:05: [ID:Model Number] = LEXAR ATA FLASH CARD                   
834C8>  CSMgrTask:ffb8aabc:22:05: IDE = 4, PCMCIA = 80, UDMA = 6
83572>  CSMgrTask:ffbdb8a0:22:01: cfDecideTiming: UDMA Mode 6 (CFA4.0)
83592>  CSMgrTask:ffbdbb3c:22:03: CF_GetAccessTiming : DatTim = 3, DatMod = 6


as seen the cf card is only being access @ UDMA 6 which limit to 80MB/s thou the card is capable of UDMA 7 ,
so can I assume that  DatMod = 6 is (UDMA = 6) ? Would this be a parameter that can change to "7"  if indeed that what it means .

Quote from: a1ex on April 10, 2018, 08:12:13 AM
....  but some blocks were written at ~120MB/s, others at ~85MB/s, and a very small percentage of them were written at lower speeds.
@reddeercity: more details after I'll get a new card (it *is* possible to overclock the 5D2 CF interface).

Any info on this @a1ex ?
How is something like this coded , as a module or just code in ML source ?

Quote from: a1ex on April 11, 2018, 07:48:49 AM
You can send ATA commands to the CF card (QEMU emulates them), so if the only difference between UDMA 6 and 7 is timing, it might even be possible to put the card in UDMA7.
How would I do this in QEMU ?

Is there any special Log file I can generate for the CF Card interface ?

Edit: found this for 7D & 5D3  -- need to find this for 5D2
http://magiclantern.wikia.com/wiki/Register_Map#CFDMA
Title: Re: UHS-I / SD cards investigation
Post by: Audionut on September 11, 2018, 12:28:03 PM
Quote from: reddeercity on September 11, 2018, 07:54:48 AM
How is something like this coded , as a module or just code in ML source ?

sd_uhs.mo (5D3 only)
https://www.magiclantern.fm/forum/index.php?topic=12862.msg199158#msg199158

Doesn't look like the source is public yet.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 11, 2018, 01:38:32 PM
Source?
https://bitbucket.org/hudson/magic-lantern/branch/sd_uhs

Still seems there's some unpublished stuff around this but then again it's not the safest hack in the toolshed it seems.
Title: Re: UHS-I / SD cards investigation
Post by: allemyr on September 13, 2018, 11:23:58 PM
Quote from: reddeercity on September 11, 2018, 07:54:48 AM

as seen the cf card is only being access @ UDMA 6 which limit to 80MB/s thou the card is capable of UDMA 7,

The reel bottleneck of this isnt the cf card, its the camera in all cases 5D2, 5D3 and 5D4.

UDMA6 is 133mb/s not 80? UDMA7 is 167mb/s.

I don't know what the 5D2 writespeed is, 80 as you say but thats because of the cameras interface. As same with the 5D3 maxing out at around 90-100mb/s write, even with a 160mb/s cf card inside. The 5D4 maxes out at 105mb/s. In 14 bit raw all those cameras will be limited to around 1080p cause of bandwith in camera interface. Shure 10 bit and 12 bit get you to higher resolution, but max of all these cameras will be different and 5D4 won't go above 105mb/s when solved.

Just my thought, worth looking into, if iam not totally wrong.
Title: Re: UHS-I / SD cards investigation
Post by: waza57 on September 14, 2018, 08:51:48 AM
I used DIGIC POKE to simply modify this values I found in  http://magiclantern.wikia.com/wiki/Register_Map#CFDMA (http://magiclantern.wikia.com/wiki/Register_Map#CFDMA) :

0xC062850C --> 0x1213 and I obtain UDMA 0 speed (around 15MB/s)
                     --> ...........
                     --> ...........
                     -->0x0202 and I obtain UDMA 6 speed (around 80MB/s) i suppose original value
                     -->0x0101 and sadly I obtain UDMA 6 speed  too :(
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on September 14, 2018, 09:00:55 AM
My raw notes:


CF 5D3:
0.932.277   CSMgrTask:ff6aa02c:MMIO : [0xC0628504] <- 0x04060503
0.932.285   CSMgrTask:ff6aa050:MMIO : [0xC0628508] <- 0x010A1105
0.932.291   CSMgrTask:ff6a9e4c:MMIO : [0xC062850C] <- 0x00000202

    0xC0628504 0xC0628508 0xC062850C
u7: 0x04060503 0x010A1105 0x00000202
u6: 0x04060503 0x010A1105 0x00000303
u5: 0x04060503 0x010A1105 0x00000303
u4: 0x04060503 0x010A1405 0x01000505
u3: 0x04060503 0x030A1405 0x01000808
u2: 0x04060503 0x060A1405 0x01000B0B
u1: 0x04060503 0x090A1905 0x01000F0E
u0: 0x05040402 0x04060503 0x01001616


Can you try 102 or 201?
Title: Re: UHS-I / SD cards investigation
Post by: allemyr on September 14, 2018, 09:42:26 AM
Quote from: waza57 on September 14, 2018, 08:51:48 AM
                     -->0x0202 and I obtain UDMA 6 speed (around 80MB/s) i suppose original value     

UDMA 6 is 133 megabytes per second, you can read it all over the internet. The limit is something else then UDMA6, UDMA7 will therefore not help much.
Title: Re: UHS-I / SD cards investigation
Post by: 12georgiadis on September 14, 2018, 09:54:25 AM
Maybe we can change the name of the thread if we have to include CF cards investigation. And thank you all for the ressearch effort. And @reddercity, you are very helpful and you've participated a lot to the progress of D4 Raw. Let's try talking here about SD / CF investigation. If it's the chaos, I hope that someone from the forum moderation would split again the topic if it's necessary.
Title: Re: UHS-I / SD cards investigation
Post by: waza57 on September 15, 2018, 11:39:57 PM
@A1ex
QuoteCan you try 102 or 201?
yes but nothing change.

But with the value of  1000102,  I have get a big improvement of around 30% speed write.
Sadly , this corrupt the card filesystem .  ( i can't read mlv file in card and i must format the card if i try to delete it)


........hey , reddeercity, do you hear this?  ;)
Title: Re: UHS-I / SD cards investigation
Post by: aprofiti on September 27, 2018, 03:01:56 AM
Quote from: waza57 on September 14, 2018, 08:51:48 AM
I used DIGIC POKE to simply modify this values I found in  http://magiclantern.wikia.com/wiki/Register_Map#CFDMA (http://magiclantern.wikia.com/wiki/Register_Map#CFDMA) :

0xC062850C --> 0x1213 and I obtain UDMA 0 speed (around 15MB/s)
                     --> ...........
                     --> ...........
                     -->0x0202 and I obtain UDMA 6 speed (around 80MB/s) i suppose original value
                     -->0x0101 and sadly I obtain UDMA 6 speed  too :(

Does this address refers to Word 88 of CF Software Interface specifications?

if so, bit 14 is for selection of udma6 and bit 15 is for udma7 (just a guess but can be right):
0x8CF  Select UDMA4 and support from 0 to 4
0x403F Select UDMA6 and support from 0 to 6
0x807F Select UDMA7 and support from 0 to 7

Quote from: a1ex on September 14, 2018, 09:00:55 AM

CF 5D3:
0.932.277   CSMgrTask:ff6aa02c:MMIO : [0xC0628504] <- 0x04060503
0.932.285   CSMgrTask:ff6aa050:MMIO : [0xC0628508] <- 0x010A1105
0.932.291   CSMgrTask:ff6a9e4c:MMIO : [0xC062850C] <- 0x00000202

    0xC0628504 0xC0628508 0xC062850C
u7: 0x04060503 0x010A1105 0x00000202
u6: 0x04060503 0x010A1105 0x00000303
u5: 0x04060503 0x010A1105 0x00000303
u4: 0x04060503 0x010A1405 0x01000505
u3: 0x04060503 0x030A1405 0x01000808
u2: 0x04060503 0x060A1405 0x01000B0B
u1: 0x04060503 0x090A1905 0x01000F0E
u0: 0x05040402 0x04060503 0x01001616


Can you try 102 or 201?

Maybe 403 for udma6 and 807 for udma7???


825FE>  CSMgrTask:00095f98:00:00: *** register_interrupt("CFDriver", 0x82, 0xffb8b8cc, 0x0), from ffb8bb58

Does this initialise supported features of the reader? (Word 82 Features/command sets supported from specifications)
Or maybe is Word 130 (Vendor unique bytes). Does this changes in logs?
Title: Re: UHS-I / SD cards investigation
Post by: waza57 on October 01, 2018, 04:33:56 PM
I noticed that the speed of writing (under the red icon of the top) varies according to the frame rate.
I do not use the card benchmarks speed test yet, but is it normal that the speed of writing depends on the frame rate?
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on October 01, 2018, 04:42:56 PM
Yes, it's normal.

If capture speed is greater than writing speed (i.e. recording is not continuous): memory bus is the main limit. If you capture at a higher data rate (e.g. higher resolution and/or FPS), there won't be enough bandwidth left for file I/O at full speed. I'm not sure how DMA priorities are working, or whether there's any way to adjust them. This is why the hacks that freeze LiveView are able to achive slightly better speed - they turn off some "useless" image streams, freeing some memory bandwidth.

Otherwise... the buffer will be mostly empty and the speed will be limited by image capture speed. In this case, mlv_lite will try to "benchmark" only when the card is actively writing, and may print the "idle" time as well. However, this method is just an approximation: there will be some overhead caused by small buffer sizes (as the raw recording task starts writing to card as soon as it has completed frames in the buffer, rather than waiting for a huge contiguous chunk to be completed). No big deal, as in this case, recording will be continuous anyway. It may affect the estimated time for the next recording.
Title: Re: UHS-I / SD cards investigation
Post by: waza57 on October 01, 2018, 05:05:09 PM
OK , thanks.
I just use the cf card benchmark and I get between 30-40 Mb/s while the 5d2 is rather 70-80 Mb/s. (value under red icon foe example)
I wonder what's going on?
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on October 01, 2018, 06:35:11 PM
Card benchmark does not apply any of the LiveView hacks used during raw recording. Try benchmarking outside LiveView, although that would overestimate recording speed by quite a bit.

30-40 Mb/s sounds a bit too small though...
Title: Re: UHS-I / SD cards investigation
Post by: mothaibaphoto on October 07, 2018, 01:26:10 PM
How to safely activate some default preset (without any testing) during camera startup?
I mean, to place that code at the module initialization is bad idea (I tried :) ).
Need some delayed activation, when everything is loaded and no more card activity...
Title: Re: UHS-I / SD cards investigation
Post by: Danne on October 07, 2018, 02:12:04 PM
Lua?
https://bitbucket.org/Dannephoto/lua_magic/src/default/SDoverclock.lua
Title: Re: UHS-I / SD cards investigation
Post by: mothaibaphoto on October 07, 2018, 03:47:38 PM
And how do you run that Lua script? Manually? What the point of that script then?
There is no problem to run some code.
The problem is to choose right time to do that.
And, by the way, no need to disable RAW video mode to enable overclock itself.
Problem with allocating buffer for write tests only. 
Title: Re: UHS-I / SD cards investigation
Post by: Danne on October 07, 2018, 04:11:02 PM
Set the script to autorun on start up. Basic function if you worked any lua script.
Did you modify code to run with raw video set to on? Right now I use a version which partly skips the test and it works better with raw video set to off. Feel free to modify.
Title: Re: UHS-I / SD cards investigation
Post by: mothaibaphoto on October 07, 2018, 04:34:16 PM
This is the starting point of my message: by running that code at startup you risk to corrupt your card.
I did.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on October 07, 2018, 05:11:57 PM
Well, so what do you want then? One way or the other you will need to execute the code. Lua script is one way, maybe not the safest but a script could interact and wait for things to settle etc and easily done.
Safer code, ask a1ex...
Title: Re: UHS-I / SD cards investigation
Post by: mothaibaphoto on October 08, 2018, 04:35:55 AM
I want to find the answer on my question :)
And i want to warn others, experimenting with code from sd_uhs:
1. Card overclock and reinitialisation should be performed when there is no card activity.
2. Last version corrupts filesystem(record several files with one filename) when "driver strength" is changed.
Title: Re: UHS-I / SD cards investigation
Post by: a1ex on October 08, 2018, 08:00:23 AM
Quote from: mothaibaphoto on October 08, 2018, 04:35:55 AM
2. Last version corrupts filesystem(record several files with one filename) when "driver strength" is changed.

Are there any driver strength values that are "better" (less likely to cause issues) than others?

Just to cross-check with my own findings.
Title: Re: UHS-I / SD cards investigation
Post by: mothaibaphoto on October 08, 2018, 10:22:21 AM
Wow, you ask me as if i know, what it's all about :)
This is why i can to some extent be sure, that the "strength" is the source of problem:
First, I experimented with precompiled version, and has no any problems.
Than i compiled the version from repo, and got the corrupted filesystem.
I commented out all the SDR104 testing  - and no more corruptions.
I have only one SDR50 card, it records fine at 120Mhz. 12bit FHD losless continuous.
For now, i'm happy with that.
If you eventually will deсide to reimplement card spanning, that would be fantastic :)
Title: Re: UHS-I / SD cards investigation
Post by: Bropa on November 17, 2018, 07:01:40 PM
FYI : 700D/1.1.5 using Danne's Sept11 build with SDoverclock slipstreamed and SD_UHS hack

For reference, I purchased a ProGrade 64GB SDXC V90 UHS-3 Card - looking to exceed the 50-60 MBps Write numbers ive seen posted recently.

Numbers are shown here in the Image. Very impressive results. I'll perform video shooting this weekend and report back my results


(https://thumb.ibb.co/gJ92FL/Pro-Grade-SDXC.jpg) (https://ibb.co/gJ92FL)
Title: Re: UHS-I / SD cards investigation
Post by: ludzik3d on November 25, 2018, 05:50:00 PM
Hi! Is Samsung 128GB microSDXC Evo Plus good choice for Canon Eos M with UHS Hack? Sorry for my english
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on November 25, 2018, 05:56:54 PM
Write rate (without overclocking) is 23 MByte/s according to Cameramemoryspeed.com (https://www.cameramemoryspeed.com/reviews/micro-sd-cards/).
Title: Re: UHS-I / SD cards investigation
Post by: ludzik3d on November 25, 2018, 06:22:53 PM
Thank you, in specs it have write 90MB/s. Could it reach 60 MB/s (actually limit with overclocking for EosM from what I was saw) with UHS Hack? Is someone using this model?
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on November 25, 2018, 06:35:12 PM
Actually there are 2 versions named "Samsung EVO plus"

old (slow): MB-MC128DA/EU
new (fast): MB-MC128GA/EU

The new one replaces "Samsung EVO pro" and is in fact quite fast.
Make sure to get the newer model.
Title: Re: UHS-I / SD cards investigation
Post by: ludzik3d on November 25, 2018, 06:37:15 PM
Walter Shulz - Thanks, I will check this models.

I think also about safety with overclocking on not used by this way SD/microSD cards.
Title: Re: UHS-I / SD cards investigation
Post by: vincent pierre on December 02, 2018, 01:29:32 PM
Hello everyone

I read all the pages of the topic. I did some testing with the "hack sd overclock" a few weeks ago.

1. First : thanks you so much. Your work is amazing. :P

2. I try to sumarise this topic adapting it for my case. :o

3. My goal : in 2019 i will shoot a short film. at this moment i try to validate my workflow.

4. My config : canon eos 700d / T5i. canon 50mm STM. canon 10-18mm STM. With Magic Lantern Firmware  :) :) :P

5. I would shoot in : raw 14b lossless or 12b lossless or 10b lossless. 2.5k. 24fps.

6. Final delivery: at a minimum: 1080p 24fps.

7. Video Apps : MLV App  :P + Da Vinci Resolve.

8. I would like to extend by a few mega per second the writing capacity of my camera. ;)

9. I would like to use the experimental branch: "4K raw video recording; lossless compression / crop_rec module with higher resolutions"


This is my two question :

A. which version of "hack module" I should use and where I can download it. thanks for your recommendations.

B. what is the card (128gb, is the more adapted i presume for RAW) that is the more fiable and compatible with the hack ? thanks for your recommendations.

Thank you in advance
And really really: BRAVO for your work.
From France
Best regards
Title: Re: UHS-I / SD cards investigation
Post by: Bropa on February 09, 2019, 07:39:03 PM
Quote from: Bropa on November 17, 2018, 07:01:40 PM
FYI : 700D/1.1.5 using Danne's Sept11 build with SDoverclock slipstreamed and SD_UHS hack

For reference, I purchased a ProGrade 64GB SDXC V90 UHS-3 Card - looking to exceed the 50-60 MBps Write numbers ive seen posted recently.

Numbers are shown here in the Image. Very impressive results. I'll perform video shooting this weekend and report back my results


(https://thumb.ibb.co/gJ92FL/Pro-Grade-SDXC.jpg) (https://ibb.co/gJ92FL)


Update here;

My goal was to find a way to shoot 720P RAW with the 700D/T5i, what I discovered with the single sample unit I had access to from Prograde was, the camera would shoot 5-10 seconds of footage, and then stop. It would intermittently drop the memory controller after being pushed to the 50 MBps + threshold. I tried this in 2 of my 4 T5i's and it happened on both cameras. Ultimately proving to be unstable even with this top-end SD Memory card.

Having said that, with the SD Hack, this is the fastest SD Card access I have been able to achieve with this Camera.
Title: Re: UHS-I / SD cards investigation
Post by: slurpies on February 16, 2019, 07:38:37 PM
Could anybody tell me how to get this SD_UHS module for the 6d?  I've downloaded the LUA experimental build and the 4k Experimental build, along with the newest nightly build, and I can't find the module in the camera sections, the zip files, or anywhere online. 

Thanks.
Title: Re: UHS-I / SD cards investigation
Post by: dfort on February 16, 2019, 08:21:19 PM
It is very experimental and could possibly damage your card so there are no official builds. You need to compile it from the sd_uhs branch or ask someone to compile it for you.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on February 17, 2019, 03:05:45 PM
@Slurpies
Here's one of the earlier full builds, compiled for 6d.
https://drive.google.com/open?id=1N1MnP276bHw24dG6MTu8oIaGWb0Vogg1 (https://drive.google.com/open?id=1N1MnP276bHw24dG6MTu8oIaGWb0Vogg1)

Never experienced any troubles with it, but to be sure nothing bad happens, it's best to not press any buttons or ML menu stuff etc, while this module is setting up (as long as the red led is blinking ;) ).

It works as follows.
The module must be put in the Magic lantern module folder on your SD card.
Now when starting your camera up, you can activate this module in the module tab.
After a new camera startup the module is loaded, but not activated by default.
Activation is done in the Debug tab from the ML menu, there is a new option there called SD Overclock, highlight this option and press the set button.
If all is right, you see text appearing on the screen doing some writing and reading tests on your card, you'll see the red led blinking, as long as the blinking goes on, it's best to do nothing and just wait.
At the end, it shows the best setting and has set this setting, you can close console(those box with text) in the debug menu with option show console, turn it on and off.
As long as you don't turn off the camera, the SD overclock settings are set.
Once you turn off the camera, SD overclock setting is gone, you have to run this module every time you want to use it.

If the last setting the module tests during setup, is the fastest for your card, I have another altered version of SD-UHS module, which is a little quicker and user friendly, it just setups the SD interface to highest setting.

Title: Re: UHS-I / SD cards investigation
Post by: slurpies on February 22, 2019, 01:02:19 AM
Sorry for the delay, I didn't have my notifications turned on. 
Thank you a great deal for the information, I didn't realize this had to be run each time the camera was booted.
Still possibly worth it.  I've a UHS-3 Card and I'm getting 30 second shots at great resolution now in 14-bit lossless raw.
Love what you guys are doing.
Thanks again.
Title: Re: UHS-I / SD cards investigation
Post by: Aperture Science on February 22, 2019, 07:34:59 AM
Quote from: IDA_ML on May 13, 2018, 06:08:57 PM
I fully agree with you, Alpicat.  A resolution ot 2520x1386 would be a dream come true for the 100D which has the same senzor size and resolution as the 700D.  I have been using the 7D at 2520x1200 resolution a lot lately and believe it or not, the 120 pixel larger vertical resolution of this camera, compared to the 100D, makes a huge difference in the overall vision of the video.  A 16:9 vision at 2520x1386 would be so much better ... But I am dreaming again.

I have a question for you.  You mentioned the speed booster for increased view angle with EF-S lenses on APSC-sensor based mirrorless.  Would it be possible to use such speed boosters for the same purpose but with 1,6x crop sensor DSLRs operating in the LifeView (video recording) mode, with full-frame lenses, of course?  If not, why not?  And if yes, are such speed boosters available?  Sorry for the stupid question but I have never used and even seen a speed booster sofar.


For a DSLR, it is not possible to make a "speed booster" because of the "flange distance". Flange Distance is the Distance between the cmos surface and the lens. Because of the mirror less camera have less Flange distance (Because there is no mirror on the way). When putting a DSLR lens on it, there is a long distance in the mid of the cmos and the lens, speed booster is basically adding a convex lens in between to "zoom out" the image field into a approximately APS_C image field. For DSLR, there is no space between the mirror and cmos (Because there is a mirror for viewfinder, you can't put anything in that space or you want to break your camera loll). Have fun with mirrorless camera on video shooting will be the best.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on March 08, 2019, 10:29:23 PM
Soo
Hey what's up ?

No I didn't get a higher write speed but as you Know (actually you don't know) I am thinking about where is the bottleneck comes from?

Now as you know Canon 50D has DIGIC 4 CPU and it can perform up to 72-82 MB/S write speed it uses CF Card instead of SD card , and our DIGIC 5 cameras can't reach that only ~55MB/S effective when using sd_uhs from 68MB/S write speed . . Okay so the CPU isn't the bottleneck here there is something else causing this .

By using FPS override to lower frame rates like 12FPS the performance of write speed increases,  but why? It's a DIGIC 5 and DIGIC 4 have more performance ?

Is the buffer size is related somehow to the write speed? I am thinking we can somehow enhance the effective write speed , but we should know where the bottleneck comes from? Maybe by disabling some tasks . . Anyone have any idea?

Maybe there is something technical between CF and SD controllers? We should dig into 80D which has 80MB/s write speed and it uses SD card slot.
Title: Re: UHS-I / SD cards investigation
Post by: koljanych on March 09, 2019, 01:28:11 PM
Quote from: theBilalFakhouri on March 08, 2019, 10:29:23 PM
By using FPS override to lower frame rates like 12FPS the performance of write speed increases,  but why?

memory access takes many cycles. the less access to memory, the more cycles for disk operations
Title: Re: UHS-I / SD cards investigation
Post by: DOP on April 08, 2019, 05:41:15 AM
Quote from: dfort on February 16, 2019, 08:21:19 PM
It is very experimental and could possibly damage your card so there are no official builds. You need to compile it from the sd_uhs branch or ask someone to compile it for you.

Specifically for the 6D I am attempting to build this myself using the Compiler.app (https://www.magiclantern.fm/forum/index.php?topic=21882.msg199370#msg199370) that Danne mentioned but running into some hurdles.

Compiler.app runs beautifully (thanks Danne!) but when I compile the the sd_uhs branch there is no sd_uhs module in the resulting .zip file. I looked into compiling the sd_uhs module on it's own but I get the following output...


$ pwd
/Users/username/magic-lantern/modules/sd_uhs
$ make Makefile
Using ~/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gcc (preferred).
make: Nothing to be done for `Makefile'.


Suggestions?

PS I did catch part of the discussion about using some .lua related stuff to load / autoload the sd_ush module and maybe having that stuff baked into newer builds but I assume the module itself still has to be there.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on April 08, 2019, 06:11:19 AM
You need to be in the correct branch. Check first post, bottom half since the example points to the branch in question. Continue discussion in that  thread too:
https://www.magiclantern.fm/forum/index.php?topic=21882.msg199370#msg199370
Title: Re: UHS-I / SD cards investigation
Post by: DOP on April 08, 2019, 06:24:39 AM
Quote from: Danne on April 08, 2019, 06:11:19 AM
You need to be in the correct branch. Check first post, bottom half since the example points to the branch in question. Continue discussion in that  thread too:
https://www.magiclantern.fm/forum/index.php?topic=21882.msg199370#msg199370

Thanks, followed those instructions. Will continue this in that thread.
Title: Re: UHS-I / SD cards investigation
Post by: sheleviy on May 31, 2019, 02:46:15 AM
Quote from: Levas on February 17, 2019, 03:05:45 PM
...
I have another altered version of SD-UHS module, which is a little quicker and user friendly, it just setups the SD interface to highest setting.

Hi,
can you share a link for this version? Stock version that you compiled before works fine on my 6D, but since it resets not just after restarting the camera, but also after returning from standby mode, I spend a lot of time running the speed tests. My highest speed is usually last or second last setting.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on May 31, 2019, 11:18:28 AM
Compile the module from this branch:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets
Title: Re: UHS-I / SD cards investigation
Post by: Levas on May 31, 2019, 09:47:49 PM
@sheleviy
Can imagine you're not able to compile, in that case, here's a compiled version of the SD_UHS module:
https://drive.google.com/file/d/1SJHrA75UXobTzBcsEY3lqbHxhNdoksPb/view?usp=sharing (https://drive.google.com/file/d/1SJHrA75UXobTzBcsEY3lqbHxhNdoksPb/view?usp=sharing)

Just replace this one with the one on your card.
This one, when activated in Magic lantern module tab, loads automatically at camera  startup.
Title: Re: UHS-I / SD cards investigation
Post by: sheleviy on June 01, 2019, 02:11:27 AM
Quote from: Levas on May 31, 2019, 09:47:49 PM
@sheleviy
Can imagine you're not able to compile, in that case, here's a compiled version of the SD_UHS module:
https://drive.google.com/file/d/1SJHrA75UXobTzBcsEY3lqbHxhNdoksPb/view?usp=sharing (https://drive.google.com/file/d/1SJHrA75UXobTzBcsEY3lqbHxhNdoksPb/view?usp=sharing)

Just replace this one with the one on your card.
This one, when activated in Magic lantern module tab, loads automatically at camera  startup.

Thanks for the link!
I haven't tried compiling anything yet – it does look a little complex. I am new to ML but I am very excited about scripting possibilities, so I guess I'll get there in some time!

Update: tried it, works great! You mentioned it was more user-friendly, but it's kind of perfect since it loads the UHS settings on startup, no action needed from me. Amazing!
Title: Re: UHS-I / SD cards investigation
Post by: sheleviy on June 01, 2019, 02:12:41 AM
Quote from: Danne on May 31, 2019, 11:18:28 AM
Compile the module from this branch:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets

Too noob for this right now :(
Title: Re: UHS-I / SD cards investigation
Post by: Danne on June 01, 2019, 11:52:24 AM
Well gold thing levas is helping out. Nice to hear it's working.
Title: Re: UHS-I / SD cards investigation
Post by: jakeymort on June 20, 2019, 07:16:47 PM
according to the raw calculator I need 54.7mb to record continuously with my desired settings, can this overclock get a 650d from its 40mb up to 55mb+?
Title: Re: UHS-I / SD cards investigation
Post by: Danne on June 20, 2019, 07:26:48 PM
Yes. Here´s a version with sd_uhs:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Apr29.650D104.zip
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on June 23, 2019, 05:47:15 PM
           Danne

Any chance that this SpeedUp Trick can be made to work for 5DM2 CF Cards ?

I Apologize if this has been discussed before ~
Title: Re: UHS-I / SD cards investigation
Post by: Danne on June 23, 2019, 05:55:24 PM
On a sidenote. Could you test the latest build for 100D I uploaded? I fixed some sd_uhs stuff that seems to work better for 100D.
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on June 23, 2019, 06:23:05 PM
           @Danne

I'll give it a shot soon & let You know ~
Title: Re: UHS-I / SD cards investigation
Post by: OlRivrRat on June 23, 2019, 07:10:41 PM
           @ Danne

Don't get any 'Formal' notification that sd_uhs is running & Bench'Test are inconsistent >

On 4 seperate BnchTsts Writes are from 42.1 - 60.9 ~

1st BnchTst I ran Showed 1st Write @ 47.8 & 2nd @ 60.9 & never got that high again ~
Title: Re: UHS-I / SD cards investigation
Post by: Danne on June 26, 2019, 10:04:26 PM
What card? Did you test the latest build from my bitbucket download section?
Title: Re: UHS-I / SD cards investigation
Post by: papkee on August 16, 2019, 11:18:23 PM
Hi Danne

Using your latest 650D build .zip, both raw_rec and mlv_rec appear to be broken.

When mlv_rec is loaded on boot, I receive this error:
tcc: error: undefined symbol 'get_us_clock_value'
tcc: error: undefined symbol 'raw_force_aspect_ratio_1to1'
    [E] failed to link modules


And raw_rec results in:
tcc: error: undefined symbol 'raw_force_aspect_ratio_1to1'
tcc: error: undefined symbol 'get_ms_clock_value'
    [E] failed to link modules


Help would be appreciated.
Title: Re: UHS-I / SD cards investigation
Post by: Solomon on August 18, 2019, 11:46:53 PM
Hello, can anybody tell me if it's possible to run the module on a 600D?
Title: Re: UHS-I / SD cards investigation
Post by: walter_schulz on August 19, 2019, 08:30:22 PM
Not supported within Digic 4 cams and no one working on it.
Title: Re: UHS-I / SD cards investigation
Post by: clubsoda on March 12, 2020, 05:12:49 PM
Quote from: Levas on May 31, 2019, 09:47:49 PM
@sheleviy
Can imagine you're not able to compile, in that case, here's a compiled version of the SD_UHS module:
https://drive.google.com/file/d/1SJHrA75UXobTzBcsEY3lqbHxhNdoksPb/view?usp=sharing (https://drive.google.com/file/d/1SJHrA75UXobTzBcsEY3lqbHxhNdoksPb/view?usp=sharing)

Just replace this one with the one on your card.
This one, when activated in Magic lantern module tab, loads automatically at camera  startup.

What is the newest magic lantern build for this module and my 6d to use with?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on March 12, 2020, 09:25:20 PM
For 6d the newest build is this one:
www.magiclantern.fm/forum/index.php?topic=15088.msg223031#msg223031 (http://www.magiclantern.fm/forum/index.php?topic=15088.msg223031#msg223031)

The SD_UHS module in the build from the link above is autmatically started at startup (when SD_UHS module is enabled in module tab ofcourse)
You don't see a message on screen or something.
If your card can handle the speed, you will see better recording times(or continuous)

And some slightly older builds here, probably also with the SD_UHS module in it:
https://bitbucket.org/Levas_EOS/magic-lantern/downloads/ (https://bitbucket.org/Levas_EOS/magic-lantern/downloads/)
Title: Re: UHS-I / SD cards investigation
Post by: clubsoda on March 17, 2020, 09:17:18 PM
Thank you! :)
Title: Re: UHS-I / SD cards investigation
Post by: Whr on March 23, 2020, 03:46:55 AM
The RAW video really shocked me, but the device I used was 500d of the digtal4 processor, which can only record 3-5 seconds. I hope the sd card can be overclocking on the 500d. I can provide my device for the test.Thanks


https://www.youtube.com/embed/se5E7f_Mi4c
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 25, 2020, 03:23:41 PM
Quote from: theBilalFakhouri on May 31, 2018, 09:31:22 AM
I'd like to play with values without changing it in the code and recompiling it every time.

Can I adjust these registers by adtg_gui? if not, Can anyone knows the coding better than me make these registers adjustable via sd_uhs with a submenu ? then we can run an overclocking with the settings we did  :o

I did it yesterday
(https://i.ibb.co/tpjNKBh/SD-UHS.png) (https://ibb.co/QvSqQ9c)

If someone want to play with fire I will PM him sd_uhs.mo with adjustable uhs values in a submenu and the source code if needed,I will not post it to public Because it's dangerous for your CARD and Camera

I am trying to get more read/write speeds (more than 160 MHz) and understand how these registers affects the bandwidth together, But I think I will end up with a developed version for tweaking these registers in automated way (https://www.magiclantern.fm/forum/index.php?topic=12862.msg199224#msg199224), saving time and effort

Something I have noticed, sometimes when I tweak the values and end up with 2.6 MB/s write speed then 20 to 21 MB/s in the second test, and after I put the values which I tweaked back to default values the write speed stays 21MB/S Instead of 40MB/S (700D default) and I need to restart the camera to get the my write speed back.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 25, 2020, 04:38:18 PM
I got some small improvement, The tests below in play mode, Card Sandisk Extreme Pro 170MB/S 64GB:

Before:
Write Speed 66.1 MB/S
Read Speed 71.8 MB/S

After:
Write Speed 68.5 MB/S
Read Speed 74.7 MB/S

The write speed some users have achieved it at 68MB/S in the past, But the read speed never got above 72MB/S in the past, now it is  :D

Preset Settings:

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr_160MHz[]   = {        0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x101,      0x101,        0x1 };   /*  (found by brute-forcing and modified by trial and error) */

Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 25, 2020, 07:41:15 PM
Hello @a1ex

According to sdcard.org
https://www.sdcard.org/press/past_evens/pdf/SD_Standards_and_Technology_GWTaipei_Oct2014.pdf

(https://i.ibb.co/SdXyyHt/sdr104specs.png) (https://ibb.co/wwNgg8S)

In SDR50: 100MHz gives 50MB/S , so every 2 MHz gives 1 MB/s, in this case your presests names in sd_uhs.c is not accurete ?,
Like: "SDR50 values from 700D (96MHz)" it's actually 80MHz which is 40MB/S , and the 160MHz prseset name should be 144MHz (it gives 72MB/s)

Edit: the presets names (48 MHz and 96 MHz . .) are from camera ROM or DebugMsg logs , so it's from Canon
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on June 25, 2020, 08:10:09 PM
Quote from: a1ex on April 03, 2018, 12:55:26 PM
Ran a brute force (random) search for the above registers, and...

SDR104 @ 160MHz 8)

......
Brute-forcing: press shutter halfway during the initial tests, until it starts running some more. Press shutter halfway again to stop (infinite loop).
No guarantees of success, no guarantees of safety, no guarantees of data integrity. You have been warned.

Please post logs and benchmarks.

Anyone ever been able to find or brute force 208 Mhz for SDR104 ??? ??
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 25, 2020, 08:22:54 PM
a1ex told me before he tried 8 hours on AC Adapter power, I tried once but the battery ran out :P
And I don't think there is someone else, if there, Hello somone else how are you ? did you get 208MHz :D

Some ideas to Speed-UP the Process like a turbo

Quote from: theBilalFakhouri on September 30, 2018, 07:48:55 AM
I think brute forcing is a better solution but if you can .. :P make sd_uhs realize the last settings before the battery goes off and skip the tested values (time saver) and no AC adapter required (it can help with AC adapter also) & anybody can help with this without AC adapter.

Also making the tested values general or save it to the logs or something (In this way we can share in the forum the values we have tested to each other and speed up the process) then I can copy it to my card and sh_uhs will skip it.

But it need C programmer volunter, My skills aren't there yet
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on June 25, 2020, 08:31:06 PM
errrrm a1ex, couldn't this (=you  ;D) be brute forced within qemu?? No? We could need some volunteers to try this on their cams  otherwise. Should be worth it.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on June 25, 2020, 09:26:45 PM
I got an adapter an eosm and 100D. @farouk.. I really am out of time atm but could put a camera running. However. My attempts on the eosm was a bit wacky. Maybe the 100d works better.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 25, 2020, 11:29:09 PM
BEST RESULT so far on Sandisk Extreme Pro 32 GB 95MB/S
(https://i.ibb.co/Jyww0zD/BEST-RESULT.png) (https://ibb.co/dLww1Q8)

Read speed is always the same at 74.8 MB/s it's very constant I ran more than 10 tests, it always keep it up at 74.8 MB/s but Write speed it's varies between 68.9 MB/s to 71.3 MB/s , so maybe your results will not the same, but hey it reached 71.3 MB/s at some point.

Same preset above can do this result but I tweaked 0xC0400600 to 0x2 instead of 0x3, didn't notice a difference after tweaking it.

-Continuous 3K?
-No

-Actual Performance?
-Not on my camera

-Hmmm
-Yes

-Overclocking the Memory (RAM) instead ?
-Maybe it will be more effective
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 29, 2020, 10:54:34 AM
Quote from: a1ex on April 03, 2018, 12:55:26 PM
SDR104 @ 160MHz 8)

meh

SDR104 @ 192MHz  8) 2020

(https://i.ibb.co/j8nYTp9/196-MHz-Bench.png) (https://ibb.co/x2xbD4K)

Write Speed 81.3 MB/s


(https://i.ibb.co/Jj35whC/196-MHz-Read-Speed.png) (https://ibb.co/LdNnXGv)

Read Speed 86 MB/s (I am expecting Higher than that in Bench.mo, but . . .)


(https://i.ibb.co/ZgqNs0S/mv1080-196-MHz-Bench.png) (https://ibb.co/JpDCL9q)

Effective Write Speed in mv1080 61.6 MB/s before it was 52 MB/s on 700D

Explanation for above results:
As you saw in the first picture the write speed started at 81.3 MB/s then when the read speed benchmark started, the speed dropped for both write and read speeds, okay it's actually not a drop, I think after a little experiments SD UHS controllers switched back to 48 MHz mode and this happen when only performing reading action by the camera at 192 MHz (like viewing images in PLAY mode or a background tasks), (maybe it's a safeguard?) As Long as there is no reading action happening @ 192 MHz from the sd card the write speed stays @ 81.3 MB/s

Good to mention: when messing around with sd_uhs registers and perform a messed up values the camera also switch to 48 MHz mode and you can't get the default speed even if you bring registers values back to original values, you need to restart the camera (safeguard also?)

sd_uhs.mo performs lo and hi write/read speed test (see sd_uhs.c), I got 86MB/s read speed result from the "hi" read/write speed test, "lo" makes read speed drops to 15 MB/s then 21 MB/s

Okay the settings :D here you go:
static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr_192MHz[]   = {        0x4,        0x3,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 };  /*found by trial and error*/


How did I get above values? by thinking how to get them

Original values from 700D is:
static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 };   /* SDR50 values from 700D (96MHz) */


0xC0400610 Register is the main guy here to make an overclock, lower its value by one step to 0x3 (also override  0xC0400620 which have same value) this gives 55 MB/s , easy ha

Override 0xC0400610 & 0xC0400620 again to 0x2 (Now you need to tweak 0xC0400614 from 0x1D000301 to 0x1D000201 to make the overclock work, if you don't the camera will switch to 48 MHz mode 21MB/s (messed values) )
That gives 72 MB/s

Okay now you think overriding it to 0x1 should give more speed ? let's try
Override 0xC0400610 & 0xC0400620 to 0x1
and 0xC0400614 to 0x1D000001
now you need to override either 0xC0400600 from 0x3 to 0x2 , now these values will give same 72 MB/s no improvement
or 0xC0400604 from 0x3 to 0x2 , that will give 74.8 MB/s which is meh and lowering 0xC0400610 to 0x0 will not work (at least for now)

These two registers 0xC0400600 & 0xC0400604 are related somehow to overclocking, this 0xC0400600 if you increase its value from 0x3 to 0x4 will underclock the speed,

I thought let's underclock from 0xC0400600 , then re-overclock by 0xC0400610 , and it worked and we got the benefit from lowering 0xC0400610 to 0x1



Quote from: theBilalFakhouri on September 29, 2018, 04:43:08 PM
maybe in one week we will have 208MHz

Quote from: a1ex on September 29, 2018, 06:32:21 PM
I wouldn't rush to say this,

it took 5 days :D
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 29, 2020, 05:55:31 PM
Quote from: theBilalFakhouri on June 29, 2020, 10:54:34 AM

Explanation for above results:
As you saw in the first picture the write speed started at 81.3 MB/s then when the read speed benchmark started, the speed dropped for both write and read speeds, okay it's actually not a drop, I think after a little experiments SD UHS controllers switched back to 48 MHz mode and this happen when only performing reading action by the camera at 192 MHz (like viewing images in PLAY mode or a background tasks), (maybe it's a safeguard?) As Long as there is no reading action happening @ 192 MHz from the sd card the write speed stays @ 81.3 MB/s

Good to mention: when messing around with sd_uhs registers and perform a messed up values the camera also switch to 48 MHz mode and you can't get the default speed even if you bring registers values back to original values, you need to restart the camera (safeguard also?)


Cool finding  8)

Can confirm it works on the 6d, but as you mention, it drops back to really slow speeds very quick. Most of the times I could record one or 2 clips with higher speed, then it drops to 20MB/s  :P
So if a way is found to prevent the speed drop, then this is really useful :D

Normal SD-UHS 160Mhz write speed in video mode on 6d is about 59MB/s, with the above settings I got 66Mb/s (quick benchmark in video mode)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 29, 2020, 06:08:26 PM
Quote from: theBilalFakhouri on June 29, 2020, 10:54:34 AM
Original values from 700D is:
static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 };   /* SDR50 values from 700D (96MHz) */


I always thought, these SD-UHS hack settings are the same on each digit 5 SD card camera, but standard settings on 6d for SD-UHS overclock are:

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr_160MHz[]   = {        0x2,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };   /* overclocked values: 160MHz = 96*(4+1)/(2?+1) (found by brute-forcing) */


How come, different values are used across different cameras ?
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on June 29, 2020, 06:24:10 PM
good job, please continue !!!!  ;D
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 29, 2020, 06:28:52 PM
Quote from: Levas on June 29, 2020, 06:08:26 PM
I always thought, these SD-UHS hack settings are the same on each digit 5 SD card camera

Yes you are right, overclockling values are the same across all D5 Cameras, I Mentioned the default values from 700D without the hack which is "sdr50_700D" mode . . not "sdr104_160MHz"

sdr104_160MHz is not a 6D standard values, the standard values for 6D will be same as sdr50_700D which performs ~43 MB/s read/write speeds
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 29, 2020, 06:45:44 PM
Ah, so these were the default canon values ;)

Can you try and change the first register value ( 0xC0400600 ) to 0x3 ?

I did that, and got 79.5Mb/s write speed in VIDEO mode  :D

Couldn't test it long, battery is empty now. 79.5Mb/s benchmark in 5x zoom video mode.  Recorded a 1.5Gb mlv file in 2880x1200@25fps in 14 bit lossless.

Tried to benchmark it again, without raw recording enabled, so benchmark screenshot is saved. But didn't succeed. One time, the speed dropped halfway during the benchmark Write test... Battery is empty now.

Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 29, 2020, 07:01:44 PM
@Levas

Wow

Can you share your the full set of registers values ?

Which preset should I change 0xC0400600 to 0x3 ? sdr104_160MHz or sdr104_160MHz?

I remember I tried it before in both presets, there is no improvement, but I will check again when I get the camera
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 29, 2020, 07:19:12 PM

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr_160MHz[]   = {        0x3,        0x3,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 };
Title: Re: UHS-I / SD cards investigation
Post by: Danne on June 29, 2020, 07:23:19 PM
Hm, tried 0x3 before on eosm but no cegar. Strange. Worth digging into again it seems. And Levas. Keep that 2880x1200@25fps. Only one of its kind on the 6D recorded for that long  :P.

Maybe try patching without raw video on? Or in photo mode.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 29, 2020, 07:35:13 PM
Unfortunately, the 1.5Gb mlv isn't that interesting to watch  :P

Taking a look at it now on the computer, exported it to lossless dng's.
410 frames @ 25fps. The framesize varies from 3.9Mb to 4.6Mb (4.6 Mb when I point it to a window, so overexposed highlights).

410 frames at 25fps ->1.55Gb file...thats 94.5 Mb/s (with use of 250Mb buffer memory ofcourse, so the 79.5Mb/s seems true  :D )
Will try again if battery is full.

When trying to save benchmark screenshot, it looked like it only works in 5x zoom mode and not normal 1080p mode.
Benchmarking in 1080p mode gave 20Mb/s and trying again in 5x zoom mode started fast, but halfway write test it dropped.

So have to take a proper look at this.
Far from usable as it is uncertain when speed drop comes.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 29, 2020, 07:54:55 PM
Quote from: Levas on June 29, 2020, 07:19:12 PM

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr_160MHz[]   = {        0x3,        0x3,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 };


You Nailed it

SDR104 @ 240 MHz  8)
(https://i.ibb.co/cF5T490/SDR104-240-MHz.png) (https://ibb.co/hW583Nb)

Same problem, it switch to 48 MHz , I think this is the reason why brute-forcing didn't get it because of write/speed test makes the camera switch to 48 MHz
Title: Re: UHS-I / SD cards investigation
Post by: Danne on June 29, 2020, 07:57:35 PM
Just crazy!
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 29, 2020, 07:58:33 PM
Oh boy  :D :D :D

Now to make this usable, there must be found a way or trick to make this work every single time.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 29, 2020, 08:00:27 PM
Quote from: Levas on June 29, 2020, 07:35:13 PM
Far from usable as it is uncertain when speed drop comes.

Yes there is a lot of speed drops when switching to Video Mode

In PLAY Mode the result is constant :D
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 29, 2020, 08:03:14 PM
Maybe a1ex had something for this

Quote from: a1ex on April 03, 2018, 12:55:26 PM
Caveat: SDR104 requires tuning the sampling point (not implemented, not performed by Canon firmware, but doable). That might be required to avoid random errors, speed drops, or higher frequency - if the controller supports it. From 0xC0400610/20, the frequencies are 80, 96, 120, 160 and 240 - the latter is probably too high.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 29, 2020, 08:15:41 PM
mv1080 benchmark on 700D:
(https://i.ibb.co/wCt52GG/SDR104-240-MHz-mv1080.png) (https://ibb.co/Bjk0YWW)

Effective Write Speed ~70.5MB/s
Title: Re: UHS-I / SD cards investigation
Post by: Danne on June 29, 2020, 08:30:10 PM
On eosm this works. Changing:
static uint32_t sdr_160MHz[]   = {        0x2,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };   /* overclocked
To:
static uint32_t sdr_160MHz[]   = {        0x3,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };   /* overclocked
The ones shown for 700D and 6D will not work on eosm.

I get 66mb/s in photo mode. The rest 19mb/s
Eh, second benchmarking gives 76Mb/s(I had fps override set to 11fps)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 29, 2020, 08:45:08 PM
Battery is full again.
Can't reproduce again. Write speed stays at 20Mb/s.
How do you start cam, in photo or video mode?
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on June 29, 2020, 08:51:32 PM
Quote from: Danne on June 29, 2020, 08:30:10 PM
Eh, second benchmarking gives 76Mb/s(I had fps override set to 11fps)

It's a gain of about 7.6 MByte/s compared to previous experiments (which gave persistent results):
https://www.magiclantern.fm/forum/index.php?topic=12862.msg199250#msg199250
Way to go, keep it up!
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 29, 2020, 09:03:06 PM
Quote from: Levas on June 29, 2020, 08:45:08 PM
Battery is full again.
Can't reproduce again. Write speed stays at 20Mb/s.
How do you start cam, in photo or video mode?

I don't use automated sd_uhs, it didn't work well for me, I am using sd_uhs with a submenu and I run it manually, I will PM you my version, change the registers manually from the submenu, you can run it in Photo Mode and run Bench.mo or Enable it in Video mode with RAW Video turned off, sometimes it works sometimes it drops.

Edit: after changing the values, run sd_uhs only for once then press half-shutter to close the console then you are ready, when changing the value of this register 0xC0400614, the first digit from the left will disappear it's okay that's my C skills
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 29, 2020, 09:32:49 PM
Did a quick try with your build.
Changed the settings in submenu and loaded it. Still doesn't work  ??? slow 20Mb/s.
Have to dive into this, probably changed some other setting that's conflicting with it or something ?

Already tried the old build, SD card is still ok, can get ~59Mb/s with the "old" sd-uhs hack.

If we can get this working, then it's huge, that's about ~20Mb/s more write speed on top of the current SD-UHS hack  :o 8)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 29, 2020, 09:43:18 PM
Hmmm
Format you Card in camera and try again in Photo Mode, make sure you don't have anything in DCIM (only ML files on the card) and load only sd_uhs and bench.mo
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 29, 2020, 10:05:36 PM
Have to go off to work now (nightshift).
Will be continued...
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on June 29, 2020, 10:32:10 PM
You guys better try this on spare cards cause some will burn  8) ;D
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 30, 2020, 05:56:10 PM
You mean our camera's are on fire  ;D 8)
As in figure of speech, not literally (yet  :P)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 30, 2020, 05:59:46 PM
Doing some more testing, and if the high write speed is activated is a hit or a miss, can't figure out why  ???
Sometimes it works for one or more clips, sometimes it won't work for even one clip.

Still not sure what triggers it to go into 20Mb/s mode  ???
Could it be some Canon routine in the background, maybe some Magic lantern routine in the background ?
Tried disabling all non essential modules, still no luck, it's a hit or miss at current state.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on June 30, 2020, 06:20:21 PM
does anything you have discovered potentially apply to the CF interface on cameras with both?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 30, 2020, 06:59:45 PM
I just messed with the "new" settings theBilalFakhouri found. Trial and error.

But it is common believe that ~100Mb/s is the max write speed for these Canon camera's, because that's the max of the UHS-I spec.
See this post from theBilalFakhouri
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228370#msg228370 (https://www.magiclantern.fm/forum/index.php?topic=12862.msg228370#msg228370)

So theBilalFakhouri started messing with these registers, because it was very likely that higher write speeds could be achieved.

Don't know anything about CF card specs.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 30, 2020, 07:12:59 PM
By capturing some logs from io_trace_full branch, I can confirm the camera switches to 48 MHz, this a part from the log captured by Canon 700D:

It shows SD UHS default registers in 48 MHz same as 5D3 values, the overclocking was working @ 240MHz before the switch:
1.639.422      Fwrite:ff337a90:MMIO : [0xC040062C] <- 0x00000001
1.639.432      Fwrite:ff3377a0:MMIO : [0xC040046C] <- 0x00000001
1.639.435      Fwrite:ff33790c:MMIO : [0xC0400600] <- 0x00000003
1.639.438      Fwrite:ff337914:MMIO : [0xC0400610] <- 0x00000009
1.639.439      Fwrite:ff33791c:MMIO : [0xC0400614] <- 0x1D000601
1.639.441      Fwrite:ff337920:MMIO : [0xC0400618] <- 0x00000000
1.639.445      Fwrite:ff337950:MMIO : [0xC0400624] <- 0x00000504
1.639.446      Fwrite:ff337954:MMIO : [0xC0400628] <- 0x00000504
1.639.447      Fwrite:ff337958:MMIO : [0xC040061C] <- 0x00000100
1.639.448      Fwrite:ff3378e0:MMIO : [0xC0400620] <- 0x00000009
1.639.450      Fwrite:ff3378e4:MMIO : [0xC0400604] <- 0x00000003


It starts with sdSoftReset( 0 )
1.483.311      Fwrite:ff74aea0:23:01: sdSoftReset( 0 )
1.483.324      Fwrite:ff337a90:MMIO : [0xC040062C] <- 0x00000001
1.483.335      Fwrite:ff3377a0:MMIO : [0xC040046C] <- 0x00000001
1.483.341      Fwrite:ff3377ec:MMIO : [0xC0400600] <- 0x00000008
1.483.343      Fwrite:ff3377f4:MMIO : [0xC0400610] <- 0x0000017F
1.483.344      Fwrite:ff3377fc:MMIO : [0xC0400614] <- 0x1D004101
1.483.347      Fwrite:ff337800:MMIO : [0xC0400618] <- 0x00000000
1.483.348      Fwrite:ff337808:MMIO : [0xC0400624] <- 0x0000403F
1.483.350      Fwrite:ff33780c:MMIO : [0xC0400628] <- 0x0000403F
1.483.352      Fwrite:ff337814:MMIO : [0xC040061C] <- 0x0000007F
1.483.354      Fwrite:ff337818:MMIO : [0xC0400620] <- 0x0000007F
1.483.356      Fwrite:ff33781c:MMIO : [0xC0400604] <- 0x00000000


Then it goes with multiple commands from Canon (same as when the camera initialize the SD card during startup, e.g in 5D3 startup log (https://www.magiclantern.fm/forum/index.php?topic=2388.msg197313#msg197313))
sdSoftReset SUCCESS
sdTrySendCommand1 Start

sdSoftReset( 0 )

sdSoftReset SUCCESS
sdIdentifyDrive Start
sdSendIFCondition Start

sdSendIFCondition End
sdSendOCR Start

sdSendIFCondition End
sdSendOCR Start

sdSendOCR End
sdAllSendCID Start

sdAllSendCID End
sdSendRelativeAddress Start

sdSendRelativeAddress End
sdSendCSD Start

sdSendCSD End
sdSendCID Start

sdSendCID: MID = 0x03, PDN = 0x534c
sdSendCID End
sdSelectDeselectCard Start

.... etc


But I noticed it doesn't switch the frequency directly to 48 MHz or 96 MHz, it goes from multiple mods from the lowest frequency to the highest desired one (a1ex figured out this and he wrote the settings in sd_uhs.c 5D3 Mode 0 .. etc)

So I tried to start from the first values during SD card initialization to the next one to 24 MHz , 48 MHz , 96 MHz , 160 MHz and 240 MHz using sd_uhs.mo, I wrote it in order to emulate what Canon does . . . aaand . . . same result it switch back to 48 MHz :P Sorry it didn't work

I don't know if we should call sdSoftReset or others commands, but also I got something maybe the problem from sd_uhs module itself, Because after Canon switch back to 48 MHz, theoretically I should able to run sd_uhs overclock again and get back 240 MHz overclocked values, but It doesn't patch the settings again only after a startup, and this problem from sd_uhs module (maybe) so if we could e.g. clear ROM patches by switching off sd_uhs like adtg_gui, or simply force the overclocked values and disable Canon patching (I think this type of code is used in crop_rec.c and adtg_gui.c)

I am not sure if this a good way, or even if it will work, maybe I messing

Good news: SDR104 @ 240 MHz is stable when it's working, the write speeds doesn't drops when recording and, it's constant and there is no corrupted frames or corrupted data, but I noticed the card and SD card door got warm but not too hot, my PC USB card reader gets a lot hotter when writing @ 90 MB/s

700D Logs:

How I made the tests:

LOG 0 (https://drive.google.com/file/d/1HqODsxhvNu0YtPvW2iMj63nDvaDdAbhw/view?usp=sharing/view?usp=sharing): Overclocked values to 240 MHz then I took a photo in Photo Mode, after taking the picture it starts to switch to 48 MHz mode.
LOG 1 (https://drive.google.com/file/d/15HQYqV7RmXjKhE-WjO7T8ZGj_KGHhXRT/view?usp=sharing): Without Overclocking, I got into Photo Mode and took a picture, everything is normal in this LOG.

Note: DebugMsg (io_trace_full build) didn't capture the overclocked values @240 MHz correctly instead it showing like this:
1.009.343      Fwrite:00af7544:MMIO : [0xC0400600] <- 0xEEEEEEEE
1.009.345      Fwrite:00af7544:MMIO : [0xC0400604] <- 0xEEEEEEEE
1.009.346      Fwrite:00af7544:MMIO : [0xC0400610] <- 0xEEEEEEEE
1.009.347      Fwrite:00af7544:MMIO : [0xC0400614] <- 0xEEEEEEEE
1.009.349      Fwrite:00af7544:MMIO : [0xC0400618] <- 0xEEEEEEEE
1.009.350      Fwrite:00af7544:MMIO : [0xC0400624] <- 0xEEEEEEEE
1.009.352      Fwrite:00af7544:MMIO : [0xC0400628] <- 0xEEEEEEEE
1.009.353      Fwrite:00af7544:MMIO : [0xC040061C] <- 0xEEEEEEEE
1.009.354      Fwrite:00af7544:MMIO : [0xC0400620] <- 0xEEEEEEEE


Same as when making overclock @ 160 MHz it also shows 0xEEEEEEEE, so this a bug from io_full_trace as I think, 0xEEEEEEEE value represents these registers have patched by ML.

It took me many hours to make sd_uhs work in io_trace_full since it's working in top of crop_rec_4k branches, io_trace_full not supported.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on June 30, 2020, 08:02:39 PM
Bravo theBilalFakhouri!
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 30, 2020, 08:40:29 PM
I smell something burning  :P

(https://live.staticflickr.com/65535/50062626621_d4e02ed673_o.png)

Ok, not sure why write test worked here, did nothing new.

What I did, disabled crop_rec and raw recording in canon video tab (Modules were still enabled)
Then I started cam in photo mode and used theBilalFakhouri custom sd-uhs module and dialed in the settings which gave us sdr104 mode.

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };
static uint32_t sdr_???MHz[]   = {        0x3,        0x3,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 };


Activated the settings and run benchmark in photo mode.

So I did nothing special here, and there is 107Mb/s read speed  ???

After the benchmark I enabled crop_rec and raw recording in video tab. Switched to video mode on cam and started recording...speed drop...write speed is already 20Mb/s  ::)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on June 30, 2020, 08:44:11 PM
I did some more testing today, and most succes I had when switching crop_rec and raw recording to off in video menu tab.
Start the camera in video mode, switch on crop_rec and raw recording in video tab and start recording.
Most of the times, the first clip is recorded in high speed, sometimes, the speed is already dropped before any clip is recorded.

So it looks like there is a conflict between Sd-UHS hack and crop_rec and/or raw recording ?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 30, 2020, 08:50:21 PM
Nice results!

I think it conflicts somehow with ROM/RAM patches, keep crop_rec turned off, disable SRM memory from mlv_lite and disable small hacks, run some tests in x5 mode, I don't guarantee if will be some improvement probably not . .
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 01, 2020, 12:57:41 AM
Maybe I got the problem, my theory, In Short: there are another addresses Canon are using them for patching SD Card registers back to 48 MHz.

Explanation:
If you enable the overclock @ 160 MHz or less, you can also disable it by un-patching the addresses, and you will get back to the default speed, unpatching addresses (registers) are done by The following two lines:
unpatch_memory(sd_setup_mode);
unpatch_memory(sd_setup_mode_in);


I put these lines a in new module I called sd_kill (https://drive.google.com/file/d/1t9j30la6jNdNIZayzHmEeG2pV6EQWieb/view?usp=sharing) it's safe to use you can get it :P, enabling it will un-patch the SD UHS registers (same as when you turn off crop_rec when using higher resolution you will get back to default res), as I said you can get back to the default speed e.g. if you overclocked sd card to 160 MHz, by using unpatch function you will get back to 96 MHz, unlike 240 MHz, if the disable the patches (after the switch to 48 MHz) you will not get to the default speed, you will stuck at 21 MB/s

And if you tried to run the overclock again (to patch the addresses again) nothing will happen it's look like the current addresses are unusable and Canon has switched to another ones.

It's clearly Canon uses another addresses for that, I don't think I can get addresses from io_trace_full when running DebugMsg? if we could and if it was true we will able to get them from DebugMsg LOG and it will be Camera specific addresses . .

I could get addresses by adtg_gui log, but unfortunately I am not sure how to show sd_uhs registers from adtg_gui.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 03:49:08 PM
Can't get it stable  :P
Found a settings that is little bit slower but is activated at least every single time at startup with the automatic sd_uhs module (the other setting is a random hit or miss)

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x2,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };


I get about 74MB/s in video mode with the above setting.
But unfortunately the setting can break quickly in 20Mb/s mode.

I can't figure out why it breaks, it can even break during recording, so first 5 seconds the speed is 74/Mb/s and then it suddenly drops to 20MB/s  ::)
Other times it breaks when switching between 1x and 5 x zoom modes or it can even break by starting up in photo mode and then switching to video mode.

It breaks with the use of crop_rec and it breaks without crop_rec activated...
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 01, 2020, 04:47:30 PM
After some more searching in the LOG 0 I found this, maybe it's where the problem starts:

1.300.435      Fwrite:ff74c4dc:23:26: sdDMAWriteBlk: Transfer Block Size Invalid(4809<->6167)
1.300.462      Fwrite:ff74c540:23:06: sdDMAWriteBlk(SDSTS_ED)(0x2:0x8001:0x40:0x2)


Then it will try something
sdWriteBlk: Retry:0

Canon detected something unusual, after this it will do sdSoftReset which will initialize SD Controllers in 48 MHz mode (Safe Mode ?), this error isn't presented in LOG 1 which there is no overclock applied there, I am not fully sure, I will try to call it and see if it will switch SD Controllers to 48 MHz mode without the overclock.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 05:35:43 PM
Some new finding:

Here are 3 different settings that all give about 75MB/s write speed in video mode:

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x101,      0x101,        0x1 }
{        0x2,        0x2,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 }
{        0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 }


The first settings is from one of your post theBilalFakhouri
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228368#msg228368 (https://www.magiclantern.fm/forum/index.php?topic=12862.msg228368#msg228368)

Now, the first one, stays stable, I've tried my best, but it never goes to 20Mb/s  :D (I get about 75Mb/s in video mode  8) )
But the second and third settings, which show the same speed when enabled in your custom sd_uhs module, will break within a minute or so to 20Mb/s.

So it could be that not all options are valid, or according to SD standards and will forced to 20MB/s by Canon firmware.
So my guess, we'll have to look further for some magic settings that also give 100MB/s speed, but don't drop to 20Mb/s...
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 01, 2020, 05:41:41 PM
I tested some with your new combination Levas.
About other, faster tests not patching this might be the fix:
https://bitbucket.org/Dannephoto/magic-lantern_jip-hop/commits/e495cbc9685fbb81a8284984cbdffa17ca70352a

These are routines I use with 5D3 turning off and on raw video during patching. Eosm seems not needing it but other cameras do. So there is still a chance for automation with even faster tests :). Also your other combos might work better here.

Now I can´t remember if menu in 6D and 700D is this:
menu_set_str_value_from_script("Movie", "RAW video", "ON", 1);
or this:
menu_set_str_value_from_script("Movie", "raw video", "ON", 1);
Think it´s the first and then the code should apply correctly as is in the commit.

When will bb delete my repo.....

Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 01, 2020, 06:04:21 PM
@Levas

Good catch! I didn't notice there is an improvement on 700D, Because using Benchmark in PLAY mode I only got 70 MB/s max write speed and sometimes even 68.5 MB/s , brute-forcing 160 MHz preset can go up to 68.4 MB/s write speed in PLAY mode, so I thought will be no difference in video mode but there is, On video mode now I can get 59 MB/s write speed , before (brute-forcing 160MHz preset) it was 52 MB/s write speed.

Quote from: Levas on July 01, 2020, 05:35:43 PM

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x101,      0x101,        0x1 }


Quote from: Levas on July 01, 2020, 05:35:43 PM
Now, the first one, stays stable, I've tried my best, but it never goes to 20Mb/s  :D (I get about 75Mb/s in video mode  8) )

But
Hmmm something I am missing, Could you provide a Benchmark in PLAY mode using above settings you mentioned (The first one), because I have only 70 MB/s write speed in PLAY mode, and you can get up to 75 MB/s write speed in video mode, doesn't make sense, all the previous presets we have tested the write speed in PLAY Mode were identical to all DIGIC 5 Cameras

Edit: What is the card you are using ?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 06:31:14 PM
With the above settings I get with benchmark in photo mode:

Write 84.0Mb/s
Read 90.3Mb/s
Write 84.0Mb/s
Read 90.3Mb/s

The card is a 64Gb Sandisk Extreme Pro (95MB/s claim on label)
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 01, 2020, 06:34:39 PM
These are very fine numbers and if they remain to be consistent: Chapeau!
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 06:38:08 PM
Current settings from theBilalFakhouri:

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x101,      0x101,        0x1 }


This combination works too(only changing 0xc0400614), also stable, but just a little slower (2Mb/s slower write speed, read speed the same)

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x101,      0x101,      0x101,        0x1 }


Benchmark in photo mode:
Write 82.3Mb/s
Read 90.3Mb/s
Write 82.5Mb/s
Read 90.3Mb/s
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 01, 2020, 06:40:18 PM
Ongoing patch:
static uint32_t sdr_160MHz[]   = {        0x2,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };
(https://i.postimg.cc/prH0jL0G/bench1-ppm-500px.png)

static uint32_t sdr_160MHz[]   = {        0x2,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };

(https://i.postimg.cc/8cmZmg91/bench0-ppm-500px.png)

Other "safe values" can+t take me any higher but hey, increase with 2-3Mb/s, always welcome.


Also tested Levas patch:
(http://static%20uint32_t%20sdr_160mhz[%5D %20%20=%20%7B %20 %20 %20 %200x3, %20 %20 %20 %200x2, %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20%200x1,%200x1D000001, %20 %20 %20 %200x0, %20 %20 %200x100, %20 %20 %200x101, %20 %20 %200x101, %20 %20 %20 %200x10x1%20%7D;)
But results exactly like with the fastest above. Al tests on my eosm and a 128gb extreme pro 170MB/s card.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 01, 2020, 06:42:42 PM
Same settings, Same Card:
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x101,      0x101,        0x1 }


Canon 6D
Quote from: Levas on July 01, 2020, 06:31:14 PM
With the above settings I get with benchmark in photo mode:

Write 84.0Mb/s
Read 90.3Mb/s
Write 84.0Mb/s
Read 90.3Mb/s

The card is a 64Gb Sandisk Extreme Pro (95MB/s claim on label)

Vs

700D
(https://i.ibb.co/4ZFD6hB/another-192-MHz.png) (https://ibb.co/vxcr702)

This is the strangest thing so far, so this preset is 192 MHz (at least on 6D)

Which sd_uhs are you are using, is sdr104 mode enabled also (from sd_uhs.c)?
I am confused :P
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 01, 2020, 06:45:05 PM
Are you formatting with exfat maybe? Strange Levas, getting that in between goodies.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 06:52:19 PM
I'm using the custom made sd_uhs.mo from theBilalFakhour (the one you send today)
I've also put the same settings in the automated sd_uhs module from Danne. Same results.

Could it be that the 6d has some more cpu overhead ?
The same reason as video mode is always ~10Mb/s slower as photo mode ?

6d has digic 5+ most others have digic 5 without the plus.
According to wikipedia the 70d also has digic 5+, anyone with a 70d here to test read and write speed ?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 06:56:13 PM
Pics or it didn't happen  :P

(https://live.staticflickr.com/65535/50065991787_bb8002b41e_o.jpg)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 01, 2020, 07:00:49 PM
Quote from: Danne on July 01, 2020, 06:45:05 PM
Are you formatting with exfat maybe?

It's actually 32GB, FAT32, Sandisk Extreme Pro 95MB/s, Okay I have just formatted it to ExFat, and made a test, same speed . .

Quote from: Danne on July 01, 2020, 06:45:05 PM
Strange Levas, getting that in between goodies.

Yeah, what about 5D3, 100D and 70D? I am wondering :P

Quote from: Levas on July 01, 2020, 06:52:19 PM
Could it be that the 6d has some more cpu overhead ?

Not sure, but I don't think this is the case, In video mode yes it might be a CPU overhead or maybe Memory Bandwidth same on all D5 Cameras, but the other presets gave the same results like 240 MHz, we got 99 MB/s in PLAY mode both on 6D and 700D
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 01, 2020, 07:03:19 PM
So 6D effectively gets 75Mb/s recording speed right now whereas eosm makes out in 55-57Mb/s? I´d say that is a significant change for the 6D. What was it before. Around 60Mb/s tops on the 6D?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 07:05:53 PM
Yep, max I've seen is 63MB/s
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 01, 2020, 07:12:27 PM
Brilliant.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 07:14:38 PM
This is the source sd_uhs.c which gives me 75Mb/s write speed, probably very much the same as yours:
https://drive.google.com/file/d/1hRtjmh_--NVYXkouGNN0_jNDWhPEO2_H/view?usp=sharing (https://drive.google.com/file/d/1hRtjmh_--NVYXkouGNN0_jNDWhPEO2_H/view?usp=sharing)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 01, 2020, 08:25:07 PM
It´s my old sd_uhs code tweaks. Do try this as it will probably work better generally:
https://bitbucket.org/Dannephoto/magic-lantern_jip-hop/src/e495cbc9685fbb81a8284984cbdffa17ca70352a/modules/sd_uhs/sd_uhs.c
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 09:05:07 PM
Just wondering, could it be that our settings are right for max sdr104 speed, but our cards can't handle the speed ?

I had the same problem with the sandisk extreme pro 45Mb/s cards when first version of sd_uhs hack was introduced. Those cards also switched to 20Mb/s...

Maybe we should looking for better sd cards  ;D
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 01, 2020, 09:23:13 PM
i'm happy to try with my sandisk extreme pro 170mb/s if you think the 5d3 can play along ;)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 09:48:21 PM
Would be interesting since the 5d3 is also digic 5+.
I also have one sandisk extreme pro 170mb card. But it's unfortunately no different to my 95mb sandisk card.
Same write speed.

But if the 5d3 can do 75Mb/s on SD that would be huge if both CF and SD are used for recording.

Not sure if anything is different for 5d3. So I'm not sure if I can upload my sd_uhs.mo file ?
I don't want to ruin a 5d3 or a good Sandisk card :P
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 01, 2020, 09:52:15 PM
5D3 has an older sd slot, worse than eosm.
Anyway. Feel free to test this module:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/sd_uhs.mo

I assume you run my experimental build so stick to that.

I changed this:
static uint32_t sdr_160MHz[]   = {        0x2,        0x3,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };
to this:
static uint32_t sdr_160MHz[]   = {        0x2,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };

Let´s start with that. Bleeding edge. Anything happens, what can I say. You get to keep the burnt card ;).
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 01, 2020, 11:32:54 PM
i tried it, and it worked, at least in terms of enabling it, but i can't get bench.mo to work...  i'm getting errors enabling the module.  do i need a special version to use with your latest build?

while i await your response i will do some recording tests with overclock on and see if it is reliable...
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 01, 2020, 11:36:31 PM
If you disable global draw on the overlay tab and set REC indicator to instant bitrate on the movie tab, you can see the current writing speed while recording. No benchmark needed.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 01, 2020, 11:40:43 PM
false start!  sd overclock turns "ON" but as soon as i hit record, it says "file create error".

looks like it isn't working... yet!
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 01, 2020, 11:46:05 PM
Run only sd card. Not with cf card.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 12:02:45 AM
yup, it was sd only... :(
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 12:06:17 AM
Only one try?
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 12:09:59 AM
i tried a few times.  the error was a little different one time...  "card full"

empty freshly formatted card, of course.

i had the same behaviour a few months back when we were trying to overclock crappy old cards...  i guess it means the card can't handle it?
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 06:37:35 AM
Yes, probably not. Thanks for testing. Let´s leave that one for now.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 02, 2020, 08:47:16 AM
1.Canon 700D:
(https://i.ibb.co/kDzBfPX/New-Preset-1.png) (https://ibb.co/0912Szr)

2.Canon 700D:
(https://i.ibb.co/Lz5cTR5/New-Preset-2.png) (https://ibb.co/C52Yj82)


Thanks for Watching Nailing registers Part II
You're Welcome

Preset for the first Picture:
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 }


It didn't switch to 48 MHz during my tests, there is a little of speed drop I have noticed, when it drops it also increase.

New discovery: You can make an overclock by changing only one register 0xC0400600 from 0x3 to 0x8, you got 81 MB/s in play mode End of Story, Video Mode around 58 to 61 Write Speed maybe sometimes a little more on 700D.

Preset for the second Picture:
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3 }


Continued from the first one, I set 0xC0400610 & 0xC0400620 from 0x4 to 0x3, that's it, write speed in Video Mode Ah let me show you:
(https://i.ibb.co/5TMQkc1/Preset-2-Video-Mode.png) (https://ibb.co/JnBYqkc)

Unfortunately it switch back to 48 MHz, but you can record some clips before the switch and at least I could a benchmark in video mode. Maybe there is a sweet spot I didn't play so much with this new combination of the registers, make your tests and feedback ;D
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 09:00:07 AM
checking these new found settings.
Work on 6d too, benchmark in photo mode:

Write speed 97.9Mb/s
Read speed 21.7Mb/s
Write speed 21.0Mb/s
Read speed 21.9Mb/s

So indeed still switches back to 20Mb/s mode...but this could mean there is another combination giving the same speed and will work.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 09:08:15 AM
Try these settings  ;D

(https://live.staticflickr.com/65535/50068032457_f588197754_o.jpg)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 09:35:32 AM
Haven't been able to break the above settings to 20Mb/s  8)

In video mode, recording bitrate starts at about 82Mb/s, saw it climb up to 88Mb/s during recording  :D

Benchmark in video mode:
Write speed 80.0 Mb/s
Read speed 83.7 Mb/s
Write speed 80.0 Mb/s
Read speed 83.5 Mb/s
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 10:07:37 AM
Well,well,well  :). From theBilalFakhouri numbers on eosm:
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }

{        0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x100,      0x201,      0x201,        0x4 }

(https://i.postimg.cc/HnPF6Z9Z/bench3-ppm-500px.png)


EDIT:
This also works:
0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 02, 2020, 10:27:08 AM
Oh, this was a typo, it should be like your edit, I will edit the presets to the correct values.

Quote from: Danne on July 02, 2020, 10:07:37 AMEDIT:
This also works:
0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 10:37:19 AM
So cool. I am getting full sensor readout(anamorphic) in 1736x3256 14fps write speed 68Mb/s on my eosm. Prior was 10fps maximum.
Title: Re: UHS-I / SD cards investigation
Post by: masc on July 02, 2020, 10:46:34 AM
This is soo good! Well done guys!
@Danne: would it be possible to record full width anamorphic 1736x2216 (2.35:1) @24fps on EOS M? At 10bit I expect something about 66MB/s, at 12bit around 80MB/s, if my calculation is correct.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 02, 2020, 10:50:11 AM
Sandisk wants to know your location!
Title: Re: UHS-I / SD cards investigation
Post by: mineralof on July 02, 2020, 10:57:17 AM
Quote from: Levas on July 02, 2020, 09:35:32 AM
Haven't been able to break the above settings to 20Mb/s  8)

In video mode, recording bitrate starts at about 82Mb/s, saw it climb up to 88Mb/s during recording  :D

Benchmark in video mode:
Write speed 80.0 Mb/s
Read speed 83.7 Mb/s
Write speed 80.0 Mb/s
Read speed 83.5 Mb/s
unbelievable!
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 10:58:18 AM
Quote from: Walter Schulz on July 02, 2020, 10:50:11 AM
Sandisk wants to know your location!
Say what?

@masc
The card speed depends on fps. Even if the card theoretically seems to work for this it seems with 24 fps it maxes out around 57Mb/s. This is not the case with full resolution preset which runs with a reduced framerate.
Title: Re: UHS-I / SD cards investigation
Post by: mineralof on July 02, 2020, 11:14:48 AM
is it already possible to download a new module for testing on canon 6d?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 11:35:08 AM
@mineralof, other stuff to do now, but I will upload the module and/or build for the 6d within 3 hours.
Curious what recording speeds :D other people get
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 11:40:06 AM
@Danne and @theBilalFakhouri
Are your settings stable or does it still switch to 20Mb/s after a while ?
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 11:51:52 AM
Seems stable over here(eosm).
I recommend you run my latest sd_uhs code.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 12:54:56 PM
It is already stable over here.
Downloaded your improved sd_uhs source file.
Works fine, although I like the old source better (no waiting times and messages on screen  :P )
Never experienced any trouble with the old build.

So if I'm correct you're using

0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4


Works on the 6d, but recording speed is ~72Mb/s

With this one I get ~85Mb/s:

0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3


The last one looks stable to me.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 01:00:07 PM
Tried your 0x3 one but not working on eosm. First one does :).
Wait for the magician. I think he returns with 90Mb/s  :P.

EDIT: Seems eosm can handle the patching without the delays and comments so out it goes ;).
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 01:29:42 PM
Quote from: Danne on July 02, 2020, 01:00:07 PM
Wait for the magician. I think he returns with 90Mb/s  :P.

Either that, or some men in black just put him in the back of a van with tinted windows by now  ;D
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 01:58:05 PM
Ok, here it goes.
One compiled sd_uhs.mo file.

I used these settings, which gave me the highest recording speeds on 6d and seems stable (doesn't switch to 20Mb/s, but not tested extensively):

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3 }


Tested on 2 different sd cards on the 6d (write speed varies depending on card, one card does ~80Mb/s and the other one ~85Mb/s)

Use at your own risk, only tested on 6d, but the sd_uhs patching is probably also done on EOSM, 100D, 700D, 70D, 650D and 5d3 with this build.
Although Danne already reports this setting doesn't work for EOSM.
When settings don't work, or your card can't handle it, I expect you get 20Mb/s write speed.
In that case, turn off the sd_uhs module in the module tab, restart your camera and everything should be normal again.

Copy this file in your ML module directory (replace old sd_uhs.mo file if you're already using one)
Should work on crop_rec_4K branches I guess ?

The patching is done at startup of the camera, you won't get any messages or feedback with this one.
Only thing you would see is some more activity from your SD card unit (red led blinks a few more times at startup)

Here is the link to the compiled module file:
https://drive.google.com/file/d/1wRLfWoMDCxB2SAZXRVoKM-9plDBAvxIq/view?usp=sharing (https://drive.google.com/file/d/1wRLfWoMDCxB2SAZXRVoKM-9plDBAvxIq/view?usp=sharing)

Here's a link to the source:
https://drive.google.com/file/d/1w8E1IPGSyJbYIb3MiZNic-jGI0yQTwJv/view?usp=sharing (https://drive.google.com/file/d/1w8E1IPGSyJbYIb3MiZNic-jGI0yQTwJv/view?usp=sharing)

To know if it works, try some raw recording and see what write speeds are displayed or see if you get longer recording times.

If you want to do benchmark test, here's the benchmark module for the crop_rec_4K branch:
https://drive.google.com/file/d/1dbqHccpJK52hxguy8QGvFUhVVdYe3S_G/view?usp=sharing (https://drive.google.com/file/d/1dbqHccpJK52hxguy8QGvFUhVVdYe3S_G/view?usp=sharing)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 02, 2020, 02:24:28 PM
Quote from: Levas on July 02, 2020, 11:40:06 AM
@Danne and @theBilalFakhouri
Are your settings stable or does it still switch to 20Mb/s after a while ?

This preset works fine and doesn't switch to 48 MHz, tested on 700D
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 }


This one it also works on 700D and I could record tens of video clips in all video mods mv720, mv1080, x5, 1x3 etc . . but at some point it will switch back to 48 MHz on 700D
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3 }



However the write speed on these two presets are not constant, it drops especially on Sandisk Extreme Pro 170 MB/s, only the Write Speed drops , but read speed is constant during benchmark, I got it 105.4 MB/s read speed :D , we need Bench.mo with graph overtime , if someone could implement it, that will be helpful.



I tried to call sdDMAWriteBlk: Transfer Block Size Invalid I end up with a Card not working with Camera, it couldn't startup again with card I made the call with, on PC the card works just fine, I formatted it many times in different types, the camera refused to work with the card, until I got SD Memory Card Formatter from SD Association, that resolved the problem like magic after a quick format . .

Not sure if I did the call right :P
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 02:36:56 PM
Quote from: theBilalFakhouri on July 02, 2020, 02:24:28 PM
However the write speed on these two presets are not constant, it drops especially on Sandisk Extreme Pro 170 MB/s

Same over here, noticed yesterday with the 128Gb Sandisk Extreme Pro 170 MB/s that the write speeds fluctuate.
The 64Gb Sandisk extreme pro 95Mb/s I'm having seems far more stable with write speeds.

Although I didn't try formatting the 128GB card. So could be that after reformatting it performs the same ?
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 03:05:32 PM
levas, i tried your module on my 5d3, and it gave me 20mb/s  with my sandisk 170 :(

i await the next version :)
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 03:28:33 PM
PS: should i have seen the overclock menu item?  i did not.  i guessed it was because you have hardcoded it, but maybe it just failed to load, although there was no error message...
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 05:16:38 PM
You can check if the patch is done on the Debug tab in ML menu.
The menu item 'memory patches'
should show 3 rom patches.

Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 05:29:58 PM
0 patches :(

rebooted several times.  always 0.

edit: should i be running a specific build?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 05:35:30 PM
Should work in the crop_rec 4k build from the experimental downloadspage.

But if you haven't got a compatible build, I expect you get an module error message at startup ?
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 05:39:18 PM
i'm using danne's very latest, june 27.  no error messages.
i'll try the standard nightly build now...
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 05:42:16 PM
Don't use the automated sd_uhs.mo for eosm and 6d. That is why I custom build the module for 5d3. I can build a few versions tonight for testing.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 05:50:21 PM
thanks danne,

here's the error i got using the nightly build

(https://imgur.com/a/90kFpBv)

https://imgur.com/a/90kFpBv
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 06:29:04 PM
Below for 5D3 with my experimental builds. The ongoing patches from bilalFarouk...Obviously you need to erase prefix patch_1_ and prefix patch_2_ before adding it in modules folder.
patch 1
https://bitbucket.org/Dannephoto/magic-lantern/downloads/patch_1_sd_uhs.mo
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 }



patch 2
https://bitbucket.org/Dannephoto/magic-lantern/downloads/patch_2_sd_uhs.mo
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3 }
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 06:59:41 PM
patch 2 WORKED!!!!!!!!!!!!!

it recorded at 80.x MB/S for 3 recordings in excess of 1 minute each, strictly to the sd card.

the 4th recording didn't work, with the "card full" error (card wasn't full)

i looked at the resulting video files and they were perfect.  no lost frames, no issues of any kind!

this is VERY exciting stuff, wow!
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 07:21:58 PM
So... ~80Mb/s on the sd card unit only of the 5d3.
How fast was the compactflash drive again ? ~100Mb/s.

Does this mean, that with card spanning, the 5d3 could do ~180Mb/s  :o (which ofcourse should be ridiculous  :P )
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 07:27:21 PM
Quote from: 70MM13 on July 02, 2020, 06:59:41 PM
patch 2 WORKED!!!!!!!!!!!!!

it recorded at 80.x MB/S for 3 recordings in excess of 1 minute each, strictly to the sd card.

the 4th recording didn't work, with the "card full" error (card wasn't full)

i looked at the resulting video files and they were perfect.  no lost frames, no issues of any kind!

this is VERY exciting stuff, wow!
Unexpected since card slot in 5d3 is a slow story. Did you test card spanning?
@Ilia3101, check this out...
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 07:29:46 PM
Not sure, but isn't it expected that the 5d3 has the same sd card slot as the 6d (or vice versa) ?
Both were introduced in 2012 and within half a year of each other.
Both cameras are also digic5+

Edit: Patch 2 are the same settings used on the 6d and give ~80Mb/s on the 6d (Climbs up to 88Mb/s)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 07:33:14 PM
Maybe so Levas. Didn´t bother too much around sd slot but if that´s true card spanning might get a real push forward here. Around 130-140Mb/s. Too bad I don´t have my 5D3 with me. Only carry my eosm everywhere.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 07:49:31 PM
i just tried dual card mode...

3616*1536*14 bits lossless, continuous!!!

i recorded 5 clips in a row with no errors.

131 MB/S, very evenly divided, both slots in the 60's

damn, this is getting exciting!
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 07:50:51 PM
Ok, I'll admit, I'm jealous now  ;D
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 08:22:44 PM
Not bad at all!
Title: Re: UHS-I / SD cards investigation
Post by: nikfreak on July 02, 2020, 08:36:51 PM
this will drive some nice traffic to the website once stable.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 08:53:11 PM
@Nikfreak, do you still have a 70d ? (as stated at the bottom of your posts)

Because 70d is also digic5+.
So curious if the sd card slot behaves like the ones on a 6d and 5d3. If so, 70d probaly also has 80Mb/s write speed.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 02, 2020, 08:55:55 PM
650D user here waiting for compatible module.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 09:10:07 PM
Here's the sd_uhs module that is automaticcaly activated at startup:
This one is with the settings that work for 700d and EOSM:
https://drive.google.com/file/d/1xwOEDFVRqXMFbSY_wPvoxFcVbm73kGPX/view?usp=sharing (https://drive.google.com/file/d/1xwOEDFVRqXMFbSY_wPvoxFcVbm73kGPX/view?usp=sharing)

Edit: link to source:
https://drive.google.com/file/d/1zQSUOCjR_ptSpPja2cS16kyQFhHotE3_/view?usp=sharing (https://drive.google.com/file/d/1zQSUOCjR_ptSpPja2cS16kyQFhHotE3_/view?usp=sharing)

If the above doesn't work, you can try the module I posted earlier today, that one uses the settings that work for 6d, but the module should do the patching on 100d, 700d, 650d, 70d, 6d and EOSM.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 09:16:13 PM
I already added the latest sd_uhs module to the eosm build. The usual place.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 02, 2020, 09:22:53 PM
Your latest module with 650D crop_rec_4k.2018Jul22.650D104
Read runs like a clock and gives around 85 MByte/s.
Write not so much. It begins very fast but hits a break ("break" point varies in different runs) and is seriously slowed down. Accelerating after and result is in the 68 MByte/s range.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 02, 2020, 09:36:30 PM
Tested previous version. I have no words ...
(https://abload.de/img/bench30fxjrs.png)
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 09:37:50 PM
wow, this is turning into a magic lantern history day!
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 09:42:40 PM
Quote from: Walter Schulz on July 02, 2020, 09:36:30 PM
Tested previous version. I have no words ...
Exactly what version 8)?
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 02, 2020, 09:45:32 PM
Quote from: Danne on July 02, 2020, 09:42:40 PM
Exactly what version 8)?
https://drive.google.com/file/d/1wRLfWoMDCxB2SAZXRVoKM-9plDBAvxIq/view?usp=sharing

CRC32: 132155EA
CRC64: A01BAE741060715B
SHA256: 5870A0D507329669EDAAA373007704E9CE2CA161C8B99C2F2F9484989FA01962
SHA1: 77C19D9788B24E0808A5F3DE86A22D27F5D77469
BLAKE2sp: C97394E35805B91796F0DFA84643A11AB0077DAD7208CA03F0A3C37846ADC290
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 09:50:31 PM
This right?
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228576#msg228576

Just want to keep track of the patch sequence, assume this:
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3 }
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 02, 2020, 09:52:34 PM
Unable to do any compiling. I'm packing for a road trip by motorbike.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 10:00:30 PM
There goes my digic5 vs digic5+ theory  :P

So apparently the 650d also can make use of the same sd_uhs settings as 6d and 5d3.
But 700d and EOSM don't benefit from the same sd_uhs settings  ???
Title: Re: UHS-I / SD cards investigation
Post by: zandette974 on July 02, 2020, 10:04:07 PM
Hi,
Tested on 700d with bleeding edge crop rec 4k isogain in 5k anamorphic mode at 1488x1900.
When i run the script it ask me to "enable sd_uhs.mo, but the  module is already enabled ???
I didn't have this message with normal sd hack. I tried with both versions Levas posted today.
My card is a Sandisk extrême pro 95 mb/s. When i hit record i have 58mb/s and continuous recording but i think it could be better ;D

Laurent.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 02, 2020, 10:09:50 PM
it just means there is an even greater discovery waiting around the corner!  :)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 02, 2020, 10:20:56 PM
Quote from: Levas on July 02, 2020, 10:00:30 PM
But 700d and EOSM don't benefit from the same sd_uhs settings  ???

700D does benefit from the same sd_uhs settings, I managed to record in all mods 1x3, x5, mv720, mv1080 etc . . more than 20 Clips, I filled 32 GB card many times, but after recording some clips, or recording @ Maximum overclock speed e.g. 68 MB/s in video mode then it's mostly will switch to 48 MHz

Waiting 70D, 100D testers (@IDA_ML , @OlRivrRat  where are you? ) :D



I am interested in this preset more:
{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 }
{        0x3,        0x3,                             0x1, 0x1D000001,        0x0,      0x201,      0x201,      0x100,        0x1 }


On 700D max write speed @ ~ 71 MB/s and it's very constant, I filled up 32 GB SD card recording at 69.1 MB/s all the time , the recording didn't stop until the card is full, pretty good result
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 02, 2020, 10:25:19 PM
On my eosm it will not even patch. It starts really slow. BUT, I can´t complain. The other overclock setting and the working sd patch for 5D3 must be something to keep most of us smiling the rest of the summer ;).

Bitbucket still didn´t delete my shit. Maybe they´re watching  :).
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 02, 2020, 11:03:03 PM
If you think about it, it is ridiculous what these old cameras can do.
Imagine if Canon gave these write speeds and raw video recording at the introduction date...most SD  cards at the time couldn't probably even handle the speed though  :P
But they could have introduced a paid firmware upgrade 3 years after introduction date and it would still be interesting for many users.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 12:23:15 AM
but it is also canon's worst nightmare...

i am absolutely not interested in buying a new camera!

why would i?  this is wonderful for us, but not so good for manufacturers ;)

but it's good for the planet :)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 03, 2020, 12:44:43 AM
Now the whole 8K raw thing in the upcoming Eos R5 makes sense.

Dialogue:
Canon marketing: ok time for our next camera. 4k video must be enough...H.264 will do.

Canon technicians: eh, Magic Lantern has found out how to use crop mode. They now almost have 4k resolution in raw mode.

Canon Marketing: No problem, 5d3 can't handle that much data.

Canon technicians: Well...technically the 5d3 can handle that much data...it's only a matter of time until they find out how...
plus there will be the risk they find out how to overclock sensor readout speed and unlock 5k raw on the 8 year old 5d3.

Canon Marketing: Ok we go for 8k raw in our next cam ( 5d3 can't do that in any possible way...right?)
Ok 8K raw it will be.
And make sure to only use  inferior parts this time .
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 12:49:05 AM
that last line is so true and so heartbreaking...

oh well, at least there are a few more newer models for us to unlock that might have incredible potential before we hit the R5 brick wall of inferior components!

cheers!
Title: Re: UHS-I / SD cards investigation
Post by: yourboylloyd on July 03, 2020, 01:06:22 AM
Quote from: Levas on July 02, 2020, 11:03:03 PM
If you think about it, it is ridiculous what these old cameras can do.
Imagine if Canon gave these write speeds and raw video recording at the introduction date

I disagree with this statement. I don't think that Canon knew what these cameras were capable of. I think you guys are just better engineers than Canon's team.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 02:31:23 AM
i did some comparison recordings using the "old" and new sd overclock versions, and i am left with a question, but first the results:

with the new one, it was fairly even between the two cards, in the mid 60 MB/s range, and very stable during recording.

with the old one, it was not even between the cards.  the CF was recording at a higher rate, sometimes even hitting 90 MB/s, while the SD was limited to about 51 MB/s, sometimes as low as 40 (when the cf was at 90)

but...  in both cases, it was almost as if it is "pinned" at about 131 MB/s between the two cards.

even though the total is basically the same, the new one seems to record longer, perhaps because it is so evenly balanced thanks to the faster SD clocking?

and this makes me wonder...  is it possible to make the card spanning use "brute force" during recording to alternate between the slots on a per-frame basis?  my reasoning is thus:  i'm wondering if the seeming hard limit of about 131  MB/S is because of cpu overhead, and would it potentially be reduced within the card spanning routine if it was "dumb" and was hardcoded to alternate frames (or some arbitrary chunks) between the slots?

just a question for those in the know!
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 03, 2020, 06:47:26 AM
Quote from: theBilalFakhouri on July 01, 2020, 04:47:30 PM
After some more searching in the LOG 0 I found this, maybe it's where the problem starts:

1.300.435      Fwrite:ff74c4dc:23:26: sdDMAWriteBlk: Transfer Block Size Invalid(4809<->6167)
1.300.462      Fwrite:ff74c540:23:06: sdDMAWriteBlk(SDSTS_ED)(0x2:0x8001:0x40:0x2)



Do you remember this error that causes the switch to 48 MHz, Canon has two registers for Transfer Block Size:

1- 0xC0C2007C
2- 0xC0C20080

There is a pattern, it starts with the first register (0xC0C2007C), it will set a value then the second register (0xC0C20080) will show up and will set a value which should be identical to the first register value, Now Canon compares the two values:

If it were identical --> Continue
If it weren't identical --> It breaks


From same LOG 0:

Register 0xC0C2007C set a value which is 0x1817 = 6167
1.639.620      Fwrite:ff74c410:MMIO : [0xC0C2007C] <- 0x00001817

Register 0xC0C20080 set a value which is 0x12C9 = 4809
1.300.233  **INT172h*:ff74c11c:MMIO : [0xC0C20080] -> 0x000012C9

Oh shit it's not identical, there is a problem:
1.300.435      Fwrite:ff74c4dc:23:26: sdDMAWriteBlk: Transfer Block Size Invalid(4809<->6167)
1.300.462      Fwrite:ff74c540:23:06: sdDMAWriteBlk(SDSTS_ED)(0x2:0x8001:0x40:0x2)     ----------------------- what is this ?



Try the first register value?
1.300.732      Fwrite:ff74c7b4:23:26: sdWriteBlk: Retry:0 102694 6167

Then sdSoftReset( 0 )

In the normal situations the two registers is always identical, you may see LOG 1.

Now these registers represent Transfer Block Size at the current write speed ? I don't know what really it is (if someone can explain . . @a1ex we missed you), it doesn't have constant values, so how register one get it value, same for register two? and what makes register two has a different value, in short what is causing this ?

If it's okay to have two different values and if it not dangerous to have them we may patch the comparing process (ignoring it if this possible somehow), Or can we override them, if it's only reporting a values without effects on reading/writing process. We should find some informations about it before trying doing anything.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 03, 2020, 10:23:01 AM
Could it be that the switch to lower speeds is sort of initiated by the SD card.
If the card can't handle the speed (maybe also for a 5 second speed drop), the SD controller switches to a safe 20Mb/s.
There are plenty of cards out there with different speeds, and the SD unit has no idea what card it has to write on.

Maybe the two registers are some sort of time stamp or time limit in which the card has to respond back or something ?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 03, 2020, 10:26:37 AM
Quote from: 70MM13 on July 03, 2020, 02:31:23 AM
in both cases, it was almost as if it is "pinned" at about 131 MB/s between the two cards.

Maybe 130Mb/s is the speed limit of the buffer memory ?
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 12:10:36 PM
i just ran the benchmark you provided and after running patch2 the sd card results are 95 MB/S write and 5 MB/S read

yes, you read that correctly...

edit:  i didn't read it correctly!!  it actually says 5752 MB/S for the read speed, but my mind refused to accept it and edited it to say bytes per second instead.

obviously there's something wrong with this...  i will run the benchmark again in video mode.

benchmark in video mode, write speed is 60.5, and there is a malloc error for the read tests (malloc error: buffer=16777216

i ran it twice.  same both times.

global draw OFF increases the write speed to 78 MB/S.  malloc error on read persists.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 03, 2020, 12:43:04 PM
oeps, almost sounds like the first broken sd card with these new settings  :o

Have you tried reformatting it in camera (and keep ML) ?
How does the card behave on a computer, normal read speed or not ?
Have you run benchmark without the sd_uhs module, how are read/write speeds without the module activated ?

EDIT: Could this have something to do with both use of hack and card spanning ?

EDIT2: The benchmark is from "crop_rec_4k_mlv_snd_isogain_1x3_presets" branch, guess it should work normal on 5d3, but not sure  ???
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 03, 2020, 12:55:27 PM
Quote from: 70MM13 on July 03, 2020, 12:10:36 PM
it actually says 5752 MB/S for the read speed, but my mind refused to accept it and edited it to say bytes per second instead.

Sometimes the problem could be from Bench.mo, once it happened to me when I removed maybe the write speed test from the code, but this also caused a died card in the past.

Quote from: 70MM13 on July 03, 2020, 12:10:36 PM
and there is a malloc error for the read tests (malloc error: buffer=16777216

Turn off RAW video
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 01:07:55 PM
i tried your suggestion of formatting the card in camera.
it then said "restoring magic lantern" and rebooted the camera.

but it never woke up again.

after a battery pull, it still won't start.

any suggestions?

it boots with other cards.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 03, 2020, 01:15:55 PM
@70MM13

Take SD Card and CF card out of the camera, Pull the battery out, put the battery again and start the camera without any cards. feedback

/Edit: okay it boots./

Does the SD Card work fine in PC card reader, write/read speed ?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 03, 2020, 01:24:24 PM
Quote from: 70MM13 on July 03, 2020, 01:07:55 PM
it boots with other cards.

Ok, now it really sounds like something is wrong with the card.
Can you format the card on a computer ?
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 01:26:54 PM
ok, i got it sorted out!

since the camera is working with other cards, i reformatted this card on the pc, with a different blocksize to make sure things have changed ;)

then the camera booted with this card.  reinstalled magic lantern and everything is fine again :)

i just ran the bench again:

photo mode, global draw off (MB/S results) 96, 451, 96, 5785
video mode, raw video off (thanks bilal): 77, 3710, 77, 3778

no more error, and the card is fine!
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 03, 2020, 01:35:23 PM
Cool, you got it working again.

Also found this post about checking if SD/CF card is beyond repair:
https://www.magiclantern.fm/forum/index.php?topic=11428.msg167530#msg167530 (https://www.magiclantern.fm/forum/index.php?topic=11428.msg167530#msg167530)

Quote
photo mode, global draw off (MB/S results) 96, 451, 96, 5785
video mode, raw video off (thanks bilal): 77, 3710, 77, 3778

You got some strange reading speeds there, are you sure this is ok, is that a 5d3 in combination with the benchmark module thing ?
I would disable the benchmark module and not using it ever again with those strange read speeds  ???
(And reformat the card in computer again and fix it, just to be sure)
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 01:40:17 PM
yeah, no more benchmark for now!
i'll just record and look at the readout :)

i thought this would be more convenient - ha!

the card is definitely fine.  i just copied a video to/from it on the pc and it worked perfectly, and the read speed is definitely super fast!  1GB file copied before the speed could be determined ;)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 03, 2020, 01:44:19 PM
Formatting card with both cards in is probably not good. Guessing but probably it tries to find stuff on cf card.
Also formatting sd card when patched in cam seems even worse. Reports here with broken cards as result.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 01:58:56 PM
there was no cf card inserted at the time, but you're right, formatting while patched was probably a bad idea ;)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 03, 2020, 02:04:08 PM
I guess then that patch automation will interfere upon reinstall. Should look into fixing that.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 02:07:32 PM
excellent guess!
i have never done this procedure before under any circumstances, but it did seem like something went wrong during the automated magic lantern reinstallation.

sorry, i don't want to repeat the "test"! ;)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 03, 2020, 02:38:31 PM
Too curious, had to check.

I'm using the first sd_uhs module I uploaded yesterday. (based on older automatic version of Danne's sd_uhs module, no pauzes no messages on screen)
Now go into Canon menu, select format card, keep ML.
And go  :D

Ok, works, card got formatted, ML got restored and camera restarts and behaves normal, can record raw video with ~80Mb/s.
So the automatic version without pauzes and messages seems safe to have enabled while formatting card in cam.

Although A1ex did mention somewhere, that during the patching process, there must be absolutely no card access, otherwise you could end up with card corruption.
So Danne is probably right that it went bad during formatting and using patch 2 on the 5d3  ???
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 03, 2020, 03:20:23 PM
bravery!
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 03, 2020, 09:16:26 PM
I undefined restoring after format for the 5D3 and put up new builds.

Regarding eosm I heard conflicting reports about sandisk extreme pro 95Mb/s cards that don´t work with latest patch. masc reports not working, jiphop on the other hand that it works. So far sandisk extreme pro 170Mb/s seems stable.

Sorry guys, very little time to put into this atm...
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 03, 2020, 09:30:52 PM
Idea for the future of sd_uhs module, a submenu with different presets.
Option 1 = classic sd-uhs settings
Option 2 = Patch 1 settings
Option 3 = Patch 2 settings

Choice is saved with ML settings and patch is automatically done at startup.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 03, 2020, 09:39:35 PM
Cool. That shouldn´t be too hard to do.
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on July 03, 2020, 10:18:37 PM
If you wanted to get fancy, you could store the verified acceptable speeds for a given card on the card itself.  Then, when the module is activated, read the settings for the patch from the card.  Since it's likely that different cards fail at different speeds.  Have a benchmark mode that tests then records the best settings onto the card.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 03, 2020, 10:51:07 PM
Not that easy. Different reports with 95Mb extreme pro cards. One working the other didn´t.

@Levas. Here´s a little something selecting and storing different patches:
(https://i.postimg.cc/CMvZc6zw/VRAM0-PPM-500px.png)

Commit:
https://bitbucket.org/Dannephoto/magic-lantern_jip-hop/commits/aa513d18485cc6d20d4d19771b902c880c38a8fd

Eosm build updated, usual location.

EDIT: Updated again so it starts with 160Mhz patch upon install.
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on July 03, 2020, 10:56:01 PM
Yeah, that's what I guessed.  The clean way is to measure each card to determine what works, then record the measurements onto the card itself, so you know what works for next time.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 03, 2020, 10:58:13 PM
But even so, measuring includes trying to patch and then run code that sometimes works, sometimes not. Testing which also includes some safety numbers canon reverts to reducing card to around 20Mb/s etc requiring restarting cam. Many variables and buggy to say the least. But I get your point.
Title: Re: UHS-I / SD cards investigation
Post by: masc on July 03, 2020, 11:14:41 PM
Thanks for the build, Danne. With my SanDisk Extreme Pro 64GB 95MB/s and EOSM I get the following:
- 160MHz
(https://i.ibb.co/MDDx0NF/bench1.png)

- 192MHz
(https://i.ibb.co/Jp1xRVC/bench2.png)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 03, 2020, 11:31:05 PM
Yeah. No luck with that card. Jiphop got it better. Well good with reports.
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on July 03, 2020, 11:39:11 PM
Yeah, it's always a lot more work to do it cleanly.  It's a suggestion only, since I won't be doing the work :)

If you start the testing with the slowest speed, it doesn't sound too hard to detect if the "safety feature" kicks in, and if so record the prior speed as "probably good".  Then maybe power cycle the cam.  Might be more reliable if you make the benchmark longer, too.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 03, 2020, 11:48:49 PM
Thanks but I am not doing it either. Not worth it imo. But Levas maybe want to get his hands dirty.
Besides. All is wip. Numbers will probably change again soon.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 12:08:48 AM
Quote from: masc on July 03, 2020, 11:14:41 PM
Thanks for the build, Danne. With my SanDisk Extreme Pro 64GB 95MB/s and EOSM I get the following:
- 192MHz
(https://i.ibb.co/Jp1xRVC/bench2.png)

Quote from: Danne on July 03, 2020, 11:31:05 PM
Yeah. No luck with that card. Jiphop got it better. Well good with reports.

Just tested Danne sd_uhs.mo @ 192 MHz on 700D, same problem, commenting out these lines solved the problem, please re-test with these changes
    /* power-cycle and reconfigure the SD card */
    //SD_ReConfiguration();
   
    /* enable SDR104 */
    patch_hook_function(sd_set_function, MEM(sd_set_function), sd_set_function_log, "SDR104");
    //SD_ReConfiguration();
   
    if (sd_overclock == 1) memcpy(uhs_vals, sdr_160MHz, sizeof(uhs_vals));
    if (sd_overclock == 2) memcpy(uhs_vals, sdr_192MHz, sizeof(uhs_vals));
    if (sd_overclock == 3) memcpy(uhs_vals, sdr_240MHz, sizeof(uhs_vals));


Comment out both SD_ReConfiguration();
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 12:25:31 AM
New build up according to BilalFakhouri suggestion. Maybe works better @masc?
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_raw_only_2020Jul04.EOSM202.zip
Title: Re: UHS-I / SD cards investigation
Post by: 743v04 on July 04, 2020, 01:54:16 AM
Tested working in 170MB/s mode with the Sandisk Extreme Pro 128GB 170MB/s version on EOSM. Ran the 2.5k preset in an overexposed frame for a solid 30 seconds or so. I can run a benchmark if anybody would be so kind to let me know how to do that  :D
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 02:07:43 AM
Quote from: 743v04 on July 04, 2020, 01:54:16 AM
I can run a benchmark if anybody would be so kind to let me know how to do that  :D

Load Bench.mo module, restart the camera, now you have "Benchmarks" in "Debug" Menu press it --> Card Benchmarks --> Quick R/W benchmark (1 min), then press PLAY button to get the best results, feedback :D
Title: Re: UHS-I / SD cards investigation
Post by: 743v04 on July 04, 2020, 02:18:04 AM
Thanks theBilalFakhouri but unfortunately it looks like the build Danne provided does not allow for additional modules to be included. Adding the bench.mo from the EOSM nightly build seems to not allow any of the modules to load correctly. I saw Walter Schultz's mention of another method but I don't see that option available either. I guess I will just wait for now  :)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 02:34:54 AM
Ah, Danne has removed bench.mo from his build

Grab bench.mo from this build:
https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/80/artifact/platform/EOSM.202/magiclantern-crop_rec_4k.2018Jul22.EOSM202.zip
Title: Re: UHS-I / SD cards investigation
Post by: 743v04 on July 04, 2020, 03:08:01 AM
Each of these were done with the 192MHz mode enabled using the same 128GB Sandisk Extreme Pro 170MB/s card:

(https://i.imgur.com/zhyTyFq.png)

(https://i.imgur.com/1CPvgXM.png)

(https://i.imgur.com/poOO1z7.png)


And for reference, here is a benchmark done with the build I had been using prior to this newest one with that same card:

(https://i.imgur.com/GovWhYh.png)

Solid improvement, amazing work!  :)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 04:12:51 AM
240MHz now stable on 700D using Sandisk Extreme PRO 64GB 170MB/s , on  Sandisk Extreme PRO 32GB 95MB/s It's not !
Based on sd_uhs from Danne, here is it (https://drive.google.com/file/d/1AYagqlTpyP-fIVghjchLpU-TknoiWLY3/view?usp=sharing) but only for 700D, source (https://drive.google.com/file/d/1zbnZsJwaIoYi3d7TJfLRVeMzB5kjIiVH/view?usp=sharing)

Write Speed in video mode ~68 MB/s , No speed drops, I have managed to fill 64GB of MLVs in all mods, no problems

1736x976 14-bit uncompressed @ 23.976 fps is now possible on 700D, what a dream came true :)


(https://i.ibb.co/LC2HBSK/First-14-bit-Uncompressed-on-700-D-MLVApp.png) (https://imgbb.com/)


(https://i.ibb.co/hWW5Bj9/First-14-bit-Uncompressed-on-700-D.png) (https://ibb.co/xjjv10L)

Benchmarks:
(https://i.ibb.co/d47HRcp/Constant-240-MHz.png) (https://ibb.co/v4DMSBd)

It's Constant!

Quote from: a1ex on April 01, 2017, 11:15:41 AM
What else could you wish for?

:)



I will make more tests . . to reassure my heart.
Title: Re: UHS-I / SD cards investigation
Post by: reddeercity on July 04, 2020, 05:35:04 AM
Great Job !!  8)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 06:06:07 AM
How Sandisk Extreme Pro 64 GB 170MB/s became usable @ 240 MHz ? (at least for me on 700D, all benchmark I am talking about made in PLAY Mode)

In the first benchmark I was getting serious write speed drops in 240 MHz (last preset),
The benchmarks were 58 MB/s write speed then 72 MB/s write speed

In other tests it were 80 MB/s write speed then 70 write speed MB/s

But accidentally I formatted the card in "Low level format" in the camera and the overclock was ON @ 240 MHz, after the restart for restoring ML files, I made another restart (sd_uhs hack turns on during startup), I made new benchmark and I got:

Quote from: theBilalFakhouri on July 04, 2020, 04:12:51 AM
(https://i.ibb.co/d47HRcp/Constant-240-MHz.png) (https://ibb.co/v4DMSBd)

Wow this is cool !

So I managed to have another Sandisk Extreme Pro 64 GB 170MB/s , again I made a benchmark before "Low level format" in the camera, I had serious write speed drops again, however after doing the steps above it becomes 90 MB/s constant write speed, and it seems 170MB/s Sandisk cards handles 240 MHz pretty well.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 07:16:15 AM
Tried low level format but still no 240Mhz for the eosm  :P.
Added back bench.mo into my eosm build.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 07:36:20 AM
Low level format help to prevent speeds drop, not for making 240 MHz work :P
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 08:09:04 AM
Too bad it isn´t ;).
By the way. This is needed for the 5DIII but not for eom. Go figure.   
/* power-cycle and reconfigure the SD card */
    SD_ReConfiguration();
Title: Re: UHS-I / SD cards investigation
Post by: masc on July 04, 2020, 11:11:04 AM
Quote from: Danne on July 04, 2020, 12:25:31 AM
New build up according to BilalFakhouri suggestion. Maybe works better @masc?
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_raw_only_2020Jul04.EOSM202.zip
Oh, thanks. This seems to work better! In 192MHz I see a speed drop between 55/100 and 65/100, all other numbers are shown a shorter time. That's the result for 192MHz:
(https://i.ibb.co/GHZFcKW/bench0.png)

And that's the result for 160MHz:
(https://i.ibb.co/BZyCqzK/bench1.png)

These numbers again for EOSM and SanDisk Extreme Pro 64GB 95MB/s.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 11:16:52 AM
Speed drop?
So 95Mb/s cards is working now? Did yoy need any low level formatting in cam?
Title: Re: UHS-I / SD cards investigation
Post by: masc on July 04, 2020, 11:27:20 AM
Quote from: Danne on July 04, 2020, 11:16:52 AM
Speed drop?
So 95Mb/s cards is working now? Did yoy need any low level formatting in cam?
After starting the benchmark you see the numbers "running" from 1 to 100. This speed for changing the numbers is always the same. Only between 55 and 65 it is way slower.

Yes, can confirm 95MB/s card works now at 192MHz. I did not format in cam. I just installed the new build. Should I format in cam? How is low level format in cam done? With standard Canon formating feature?
Title: Re: UHS-I / SD cards investigation
Post by: masc on July 04, 2020, 11:33:28 AM
Wicked! Now we're talking! Found the low level format. "Formatierung in niedriger Stufe" in german... WTF?!
Speed drop is gone. Here the new numbers for 192MHz:
(https://i.ibb.co/HPrt2B6/bench0.png)
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 12:36:34 PM
i saw the new 5d3 build and grabbed it to try the nice new menu options.  looks great!

i ran the bench and saw the dreaded speed drop during the second run.  looks like it drops right down to 20 about halfway through.  the result was 95 for the first and about 60 for the second.

next i tried the low level format trick, but there was no restoration of magic lantern.  so while it was still alive i ran the benchmark again, and it was perfect this time.

so before i do anything else i thought i would ask for advice!
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 12:45:07 PM
I removed restoring ml when formatting for now. Will damage your card with sd hack enabled. Until I know how to fix it you need to reinstall if you want to do low level format.
Title: Re: UHS-I / SD cards investigation
Post by: Tullen on July 04, 2020, 12:50:02 PM
Sandisk Extreme Pro 128GB 95MB/s (Card Nr 1)

Low level format in camera (Menu button -> First tool icon -> Format card -> tick in low level format and press OK)

A few tests runs of the benchmark as well as taking a picture, so card was not totally empty when I run the below benchmarks.


Modules tab -> Bench.mo ON -> Restart -> Debug tab -> sd overclock -> 160MHz -> Restart -> Debug tab ->  Benchmarks -> Card Benchmarks -> Quick R/W benchmark (1 min) -> press play

160MHz
(https://i.ibb.co/3Tq4zQ3/Play-160mhz.jpg) (https://ibb.co/mRwb8gj)

Debug tab -> sd overclock -> 192MHz -> Restart -> Debug tab ->  Benchmarks -> Card Benchmarks -> Quick R/W benchmark (1 min) -> press play

192MHz
(https://i.ibb.co/LYchJ3G/Play-192mhz.jpg) (https://ibb.co/jbQZWSp)

240MHz gave around 20 MB/s and Global Draw Off made no difference.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 01:04:00 PM
I am curios what is the default preset for 6D and EOS M from Canon without overclocking, I want to take a look especially for these registers:

0xC0400624
0xC0400628
0xC040061C
0xC0400614

so Levas, Danne if you could capture some logs, here is io_trace_full build for 6D (https://drive.google.com/file/d/16uUMpM3c2pxbjLXs-S93a5zWQyo9GB9G/view?usp=sharing) and for EOS M it didn't compile correctly maybe some missing stubs or something, I remember this problem reported before by @dfort , maybe there is a working build which used by dfort to capture some logs in the past for EOS M, try to find one

How to capture a LOG ?
From Debug menu you will find DebugMsgs something like that, press it, now it will starts logging, take a picture and press DebugMsgs again to stop it, it's printing the LOG now, you will find it on SD card root, that's it



Quote from: Danne on July 04, 2020, 12:45:07 PM
I removed restoring ml when formatting for now. Will damage your card with sd hack enabled. Until I know how to fix it you need to reinstall if you want to do low level format.

Why it will damage the card? isn't it the same when you record RAW video, simple read/write speed action, I did a lot of low level format for about a year with sd_uhs enabled all time, maybe because of after restarting it does write/read while is sd_uhs is being enabled ?
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 01:05:59 PM
i reinstalled manually after the low level format at 240mhz, and everything went smoothly.

did 5 benchmark tests in a row, with no speed drops.  76 MB/S in video mode with global draw off.

excellent!

you guys are miracle workers!

i'll test card spanning now.  it looks like this will open new possibilities for my next music video!  thanks so much!
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 01:11:55 PM
Great.
Will check trace stuff when I can. Tomorrow maybe.
5D3 differs in patch behaviour. I do my best to work around the issues. Formatting restoring seriously messes things up. Probably simply need to wait after all has been restored but with my c skill levels it's gonna take a week tinkering  :P.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 01:13:21 PM
it's still a very beautiful thing!
thanks :)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 01:26:35 PM
@70MM13

Nice, what is your card?

if it switched to 48 MHz at some point from 240 MHz, please feedback, also if you could provide a log from 70D that's will be helpful, here is a io_trace_full build for 70D (https://drive.google.com/file/d/15ldGXxtNl6jxj5zDySLsyUMz4e87GXXV/view?usp=sharing),

How to capture the log:
Quote from: theBilalFakhouri on July 04, 2020, 01:04:00 PM
From Debug menu you will find DebugMsgs something like that, press it, now it will starts logging, take a picture and press DebugMsgs again to stop it, it's printing the LOG now, you will find it on SD card root, that's it



Quote from: Danne on July 04, 2020, 01:11:55 PM
5D3 differs in patch behaviour. I do my best to work around the issues. Formatting restoring seriously messes things up. Probably simply need to wait after all has been restored but with my c skill levels it's gonna take a week tinkering  :P.

Aha, 5D3 is different in many things :P
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 01:36:37 PM
bilal, i'm using a sandisk extreme pro 170mb/s on my 5d3.

the card spanning video tests are complete, with beautiful results!

3616*1536 @ 24fps 14 bits lossless, extremely overexposed scene: continuous recording with a green icon!

seriously, guys...  this is so amazing!

several recordings made, all perfect with no speed drops.

this will open some new possibilities for my next music video!  thanks so much!
Title: Re: UHS-I / SD cards investigation
Post by: Bender@arsch on July 04, 2020, 02:55:37 PM
@70MM13

How you set this resolution(can't set between 3840 and 3520), is this a custom mode? I don't understand how it works (no programmer, bad English).
How fast is writespeed? Is this with life view, or other special settings?

In my tests with 5D3 I see no difference between the old module.
Old Module:
3840x1440 x 10bit, at 23,976fps continuous with liveview > up to~125MB/s
3840x 1536 x 10bit, 23,976fps continuous without liveview > up to~129MB/s
--> maximum settings, on bright conditions, continuous.

New modules:
No difference at 192Mhz and 240mhz.

I have low level format SD card and I have a 170mb/s card.
Title: Re: UHS-I / SD cards investigation
Post by: Jip-Hop on July 04, 2020, 03:06:56 PM
Awesome stuff! Did some benchmarks with my 95 MB/s card, but it works best with 192mhz. So 192mhz is not only for the 170MB/s cards.

SanDisk 256gb 95 MB/s Low Level Format @ 160mhz - Test 1
(https://i.ibb.co/WkffFgT/256gb-card-95-MBs-low-level-format-160mhz.png)

SanDisk 256gb 95 MB/s Low Level Format @ 160mhz - Test 2
(https://i.ibb.co/yVJr6nY/256gb-card-95-MBs-low-level-format-160mhz-2.png)

SanDisk 256gb 95 MB/s Low Level Format @ 192mhz
(https://i.ibb.co/BLqRtfS/256gb-card-95-MBs-low-level-format-192mhz.png)

SanDisk 256gb 95 MB/s Low Level Format @ 240mhz
(https://i.ibb.co/sbQM6FJ/256gb-card-95-MBs-low-level-format-240mhz.png)

SanDisk 256gb 95 MB/s Low Level Format @ Overclock Off
(https://i.ibb.co/M6vbHpb/256gb-card-95-MBs-low-level-format-overclock-off.png)

I've got a 64GB 170MB/s card too. Will report benchmarks from that one later.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 03:55:39 PM
@ bender,

i'm using uhd mode, and selecting the closest resolution by menu item and then use the scrollwheel to step up/down to the right resolution.  2.35:1

the maximum speed using card spanning will NOT increase, but the difference is more stable/longer duration recording because the SD card is as fast as the CF.

i am wishing we will find out what is stopping the camera from recording at faster than 131MB/S.  i already suggested maybe a simpler spanning option that does even distribution between the two cards, and i am also wondering if we can disable some functionality to free up clock cycles?  maybe an "ultra lite" recording module?

but i am NOT complaining!!  this is incredible already!!

just some ideas to maybe one day make it even better :)

tim inspired me to try some of my slower cards with this new low level formatting trick...  i'll report if there are any successes :)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 04, 2020, 04:09:15 PM
Coming back from work, missed a lot here I see.

@Danne, great job on the menu item in the module!
One sd-uhs module, to rule them all  8)



EDIT: @Danne, looking at the source now, why did you delete all other cams ? Only cam in the source is EOS-M ?
https://bitbucket.org/Dannephoto/magic-lantern_jip-hop/commits/aa513d18485cc6d20d4d19771b902c880c38a8fd#chg-modules/sd_uhs/sd_uhs.c (https://bitbucket.org/Dannephoto/magic-lantern_jip-hop/commits/aa513d18485cc6d20d4d19771b902c880c38a8fd#chg-modules/sd_uhs/sd_uhs.c)
Am I missing something here  ???

Title: Re: UHS-I / SD cards investigation
Post by: Bender@arsch on July 04, 2020, 04:50:34 PM
@70MM13
thx, so easy...

OK, but I can't rec. This res. With 14bit lossless continues. Writespeed preview say me up to 180MB/s needed (before rec.).
Maybe you have a better condition (->better compression) with lossless.

I tested some benchmarks with 5d3:

192mhz photo mod
Write 80mb/s
Read 5000... Something

240mhz photo mod
Write 96mb/s
Read 5000... Something

240mhz video mod ~63mb/s and malloc error

Benchmark with both cards photo mod:
CF 80mb/s
Sd 76mb/s

= 156mb/s
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 04:56:51 PM
the 131 MB/S i am referring to is actual recording speed in video mode.

make sure you have global draw OFF during recording.  this increases video recording rate.

benchmarks in video mode seem to require raw recording OFF.  best to "bench" by just recording some video!

make sure it is overexposed!
Title: Re: UHS-I / SD cards investigation
Post by: Jip-Hop on July 04, 2020, 05:03:25 PM
SanDisk 64gb 170 MB/s Low Level Format @ 160mhz
(https://i.ibb.co/xGgfRKH/64gb-card-170-MBs-low-level-format-160mhz.png)

SanDisk 64gb 170 MB/s Low Level Format @ 192mhz
(https://i.ibb.co/zPvWmTC/64gb-card-170-MBs-low-level-format-192mhz.png)

SanDisk 64gb 170 MB/s Low Level Format @ 240mhz
(https://i.ibb.co/MZ9np2D/64gb-card-170-MBs-low-level-format-240mhz.png)

SanDisk 64gb 170 MB/s Low Level Format @ Overclock Off
(https://i.ibb.co/zSSL53f/64gb-card-170-MBs-low-level-format-overclock-off.png)

Interestingly the fastest speed I've seen was with 240mhz. Other than that, both of my cards (SanDisk 64gb 170 MB/s and SanDisk 256gb 95 MB/s) are responding the same to the overclock settings.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 05:36:50 PM
is everyone sitting down?

i have some shocking news!

i just tested my $5 (CDN) kingston 128GB card (microSD with supplied adapter) and it WORKS at 95MB/S

simply installed danne's build from today, enabled overclock with the 95MB/S option, rebooted the camera, and started recording!

no additional steps.  no formatting, no nothing!

i recorded 2880*1226 @ 24FPS, 14bit lossless using the uhd preset.  NO CF card installed, recorded only to the $5 128GB card, and the overexposed scene had a green indicator!

i think i am dreaming...
Title: Re: UHS-I / SD cards investigation
Post by: yourboylloyd on July 04, 2020, 05:39:35 PM
Quote from: 70MM13 on July 04, 2020, 05:36:50 PM

i just tested my $5 (CDN) kingston 128GB card (microSD with supplied adapter) and it WORKS at 95MB/S

simply installed danne's build from today, enabled overclock with the 95MB/S option, rebooted the camera, and started recording!

i recorded 2880*1226 @ 24FPS, 14bit lossless using the uhd preset.  NO CF card installed, recorded only to the $5 128GB card, and the overexposed scene had a green indicator!

i think i am dreaming...

This is crazy! MICROSD?!?

What resolution/framerate can you get shooting in 10bit?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 05:48:55 PM
Quote from: 70MM13 on July 04, 2020, 01:36:37 PM
bilal, i'm using a sandisk extreme pro 170mb/s on my 5d3.
..

3616*1536 @ 24fps 14 bits lossless, extremely overexposed scene: continuous recording with a green icon!

...

My mind was tricked sorry, I thought you are on 70D :P

Great result!

Quote from: 70MM13 on July 04, 2020, 05:36:50 PM

i just tested my $5 (CDN) kingston 128GB card (microSD with supplied adapter) and it WORKS at 95MB/S


Picture of the card? Maybe a online link to it, to see the card specs, Could you run benchmark on this card in play mode ?

Your claiming shocks  :D
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 06:10:58 PM
i'm sorry everyone, but it all just came crashing down...

i was doing a bunch of recordings at various resolutions @ 10 bit, 1 minute each.

on approximately the 10th recording, i got the dreaded "card full" error (NOT FULL).

so i did a low level format in camera, reinstalled magic lantern, and did a benchmark in photo mode.

it was "down" to 86 MB/S, with several tests.

then i switched to video mode,  and first i ran the benchmark again, and this time it was about 70 MB/S with global draw and raw recording OFF.

as soon as i hit record, i got "card full".

:(

i will keep experimenting, but it looks like maybe the card can't handle it.

isn't it strange that the benchmark still works though?
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 06:13:31 PM
ANd if you run lower like 192Mhz? I think one gets card full when sd controller spins off to oblivion. I had it when benchmarking around 5000Mb/s by manipulating sd_uhs code. Maybe this can happen with code somehow otherwise too. It seems related to this part:
/* called right before the case switch in sd_setup_mode (not at the start of the function!) */
static void sd_setup_mode_in_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    qprintf("sd_setup_mode switch(mode=%x) en=%d\n", regs[sd_setup_mode_reg], sd_setup_mode_enable);
   
    if (sd_setup_mode_enable && regs[sd_setup_mode_reg] == 4)   /* SDR50? */
    {
        /* set our register overrides */
        for (int i = 0; i < COUNT(uhs_regs); i++)
        {
            MEM(uhs_regs[i]) = uhs_vals[i];
        }
       
        /* set some invalid mode to bypass the case switch
         * and keep our register values only */
        regs[sd_setup_mode_reg] = 0x13;
    }
}

If I do something like:
if (sd_setup_mode_enable && regs[sd_setup_mode_reg])   /* SDR50? */
This gets called all the time and caused the card to spin off. Needless to say. Don´t try this at home.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 06:21:56 PM
this was at 160 mhz. (the 95 mb/s option)

i was thinking about trying 192 mhz after lunch :)

i was also wondering if maybe this will somehow work with card spanning.  just a crazy thought that is worth testing ;)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 06:54:59 PM
Uploaded new builds for 5D3. The 160Mhz preset will now use the older patch, more stable.
Tested 240Mb/s, it´s rock solid.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 04, 2020, 07:03:38 PM
Quote from: Danne on July 04, 2020, 06:54:59 PM
Uploaded new builds for 5D3. The 160Mhz preset will now use the older patch, more stable.
Tested 240Mb/s, it´s rock solid.

Good, you mean 240MHz ?
Could we have benchmark in PLAY Mode with the used Card @ 240 MHz, Also how this preset (https://www.magiclantern.fm/forum/index.php?topic=12862.msg228519#msg228519) perform in 5D3 ?
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 07:26:37 PM
it no longer works for me with my sandisk 170 card :(

i don't have the first version of today's build, it got overwritten due to identical filename...

could you reupload the earlier one so i can keep using it?  i might not be the only one who needs it :(
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 07:29:44 PM
Quote from: theBilalFakhouri on July 04, 2020, 07:03:38 PM
Good, you mean 240MHz ?
Could we have benchmark in PLAY Mode with the used Card @ 240 MHz, Also how this preset (https://www.magiclantern.fm/forum/index.php?topic=12862.msg228519#msg228519) perform in 5D3 ?
Hehe, yes  :P

Thing is. In photo mode when benchmarking It´s better to turn off RAW video. So I just uploaded new 5D3 builds again which do that now. Turns off RAW video when patching in photo mode. This is probably causing instabilities for movie mode too but some patches seems more stable than others. Or at least seems like it.

70mm13 - Builds are not different. Could you test again with this upload? Just put up two new versions. Minor change.

Some patches(170Mb/s sandisk extreme pro card):
0x3,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x101,      0x101,        0x1
(https://i.postimg.cc/s2WZkXLw/bench2-ppm-500px.png)

0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3
(https://i.postimg.cc/DwHXpsxS/bench3-ppm-500px.png)

0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4
(https://i.postimg.cc/9FxTTsTt/bench4-ppm-500px.png)
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 07:45:10 PM
second attempt from scratch, this time at 192 overclock before low level format, still fails...

i'm getting concerned the card/camera might be damaged?

can you send me the earlier version from today just to try?
Title: Re: UHS-I / SD cards investigation
Post by: Bender@arsch on July 04, 2020, 07:51:51 PM
I tested your new upload and found a issue, but while writing your upload a new version again^^.
So i tested again, but the issue is the same:

Can't play movie with sd patch at 192mhz and 240mhz --> "File ends prematurely during block header"
With 160mhz all is working.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 08:11:50 PM
Hm, yes. I tested now too and just running sd card alone will not work with 192Mhz and 240Mhz. I was under the impression this was tested working before? Maybe not.
I will remove the builds for now. They are benchmarking ok but recording is not legit. 160Mhz uses old patch proven working.

EDIT: Uploaded back builds with only 160Mhz included. Back to drawing board.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 08:47:22 PM
Ok, here we go again(5D3). THink I found the issue. I added some fix including a function interfering with patching. Back again with builds all included.
Manual lenses is a hassle as it stall screen for a while. This can lead to file creation errors but mostly it´s ok. I usually open up liveview manually when starting cam instead of waiting for the process to finish.
Please report back if this works again.
Builds here:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/
Title: Re: UHS-I / SD cards investigation
Post by: Bender@arsch on July 04, 2020, 09:11:27 PM
You are to fast for me^^

first update: same issue
second update: same issue

but switching back to 160mhz, i can play all files!

and

If i load, after restart, first time the sd patch in photo mod, switch to video mod and pres rec. it stopped automatical.  I need a restart again for working record.
(UHD mode, 4K with global draw, and card spanning at 10bit)
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 09:13:07 PM
benchmarks are exactly as they were on today's first version, but i cannot record on the sd card.  file create error or card full error every time.

reformatting the sd card using my pc to start absolutely fresh...  another report soon.

trying to stay optimistic!
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 09:13:16 PM
Crap. I was wrong. Back to stable 160Mhz only for 5D3. Uploaded last builds for today. This needs special attention. 5D3 is much more sensitive than eosm. I recommend ditch any older builds and redownload the 160Mhz version only for now.
AND. I need a break...

No use asking for earlier builds. Nothing vital changed and I tested already older stuff. If it was somehow working before it´s pure luck or I must be missing something essential.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 09:52:50 PM
ok, i hear you loud and clear...

meanwhile, here's the results of my testing on that triple play build:

only 160 mhz was working at all, and it was rock solid with SD card only.  as soon as i restarted with both cards in and tried spanning, "file create error" and hard lock requiring battery pull.

i will now grab your latest version and take it for a spin...  fingers crossed!
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on July 04, 2020, 09:55:17 PM
So...  my guess here is that the drop to a slow speed is some kind of back off or safety feature.  There are many factors that might trigger it, but if it triggers it latches on until something is reset (the whole camera?  Some register?).

One of the factors that is likely to make a card glitch (or perhaps it's some SD related HW) and get put into slow mode is heat.  That's going to be variable, and affected by what you're doing.  It's fairly plausible that the highest speed you can reach will never be stable for long periods, as heat builds up (or, some other random factor happens to occur, an unlikely cache spill or whatever).  I see three obvious routes for making this a usable feature (and it's too cool not to try to make it usable!):
- detect the drop to slow speeds and work out how to reset the safety feature (good if you're okay with needing many takes or only short takes)
- improve the reliability of benchmarking the card to determine the stable overclock (and ideally recording this onto the individual card so you only need to benchmark each card once)
- using the existing benchmarking, select a max speed below the fastest achievable and hope you left enough margin

Adding active cooling to the card might also work but doesn't seem very practical.  Might be interesting to see if overclocking is more reliable inside a fridge!  We might learn if heat is a real factor here.  Drill holes into the battery / card compartment in advance for best testing ;)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 10:13:03 PM
How do you explain eosm, and the rest of the cameras working without fallback?
On 5D3 it's not holding up because of when raw is enabled for one. It is semiworking as write benching works but read mode flips. I'd say it's some registry tweak needed. Or even patching routine. Code is somewhat different from the rest. But hey, a1ex probably knows more here. Malloc issues.
Heat etc could be problematic once we get continuous recording working but for know it's more like breakdown ;).
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 10:18:45 PM
ok, it's working again!

obviously the speed is not the same as it was a few hours ago :(

but it's working and my camera isn't screwed up!

with spanning, the highest resolution i get a green indicator on is now 3248*1382 @24fps and 14 bits.

fingers crossed for an eventual return to those incredible heights!!

i don't think heat is a factor.  the failures were at camera startup and everything was cool.  i'm testing in a cold room also...
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on July 04, 2020, 10:36:20 PM
Danne: I haven't been trying to find a pattern between models.  Could it be that more modern models are more likely to be stable?  Or, physically larger cameras?  Heat is just a theory, and it's hard to say one way or the other, since the individual card certainly plays a role too.

It's very *plausible* that heat matters, as that's a common reason why components are driven under spec, heat will make electronics unreliable, plus controlling heat to within regulations for consumer devices is quite challenging.  As I say, it's my guess, I don't have a strong opinion or even much evidence.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 04, 2020, 10:38:39 PM
I am speculating it might work better when both cf and sd working together. Pressure becomes less for the sd. Did you do a lot of testing with solely sd card?
Did you playback the card spanning files after testing before? If card was wrongly patched files should be problematic to open.

Edit: Needs more testing and tinkering @names_are_hard. Would be nice to see it through :).
But heat, hm. How come bench works? And works better with raw video off. Patch, sd_uhs code, my shit, all needs to come together.
Also "card full" must be sd loosing complete controll hypotethically filling the sd card in an instant. Also reveals complete breakdown when bench read numbers skyrockets.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 04, 2020, 10:42:02 PM
i did tests with sd alone as well as both cards together.

one thing that stands out immediately in spanning mode is the return to "lopsided" data writing.  when it was working at 240mhz, it was very evenly divided.  now it was about 70 for the CF and about 50 for the SD.  also, lower overall data rate.  it was 131, and now it is more like 120.

strange, considering the CF has no problems doing 90... but i guess it is the headroom issue.  it was definitely better at 240 in all cases!
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on July 05, 2020, 12:58:46 AM
The bench is quite short, I think?  I've never run it.  And the bench doesn't stress as many components as real video.

"Card full" could be a lot of things.  I think fundamentally it must be some function call returning a bad value, that's interpreted as "card full because it errored", but if you're stressing the card in unusual ways it could easily error in ways Canon didn't anticipate.  The card won't actually be full, since it's impossible to write to it that fast.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 05, 2020, 01:49:32 AM
yeah, the benchmark writes 1 GB to the card and then reads the same amount, then repeats both steps.

it's definitely nothing like the stress of actual video, that's correct!  but i guess it's a great way to compare performance without such a huge pile of variables :)
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on July 05, 2020, 08:39:05 AM
Right now I am on vacation with my family at the beautiful Southern Black Sea Coast.  I have been filming a lot with the EOS-M at 5k anamorphic (1736x2928), 10-bits lossless and 14 fps using an older build without the latest overclocking implementations.  At these settings recording is continuous and I get a very beautiful motion blur of the moving water.  After postprocessing  I then interpolate the footage to 24 fps in Resolve with the optical flow function.  Image quality is breath taking, video is very smooth and there are no signs of aliasing whatsoever !!!  The only problems with jello effects due to 14 fps I get with close-up scenes with fast lateral motion.  If, with these latest and very exciting overclocking developments, the fps could be raised to at least 18 fps, I think, this would be fantastic and the problem with jello in problematic scenes would be eased significantly.

Since I only have one 95 MHz card with me here, I do not dare to test the latest overclocking builds because, if that card dies, it will be over with filming.  However, I am watching the latest overclocking efforts with great excitement and keep my thumbs pressed that you guys succeed!  So please, keep up the good work!  If you succeed, this will be a huge step forward!

Danne and all other guys involved, please accept my greatest appreciation for the genious work that you are doing with further improving ML.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 05, 2020, 09:14:13 AM
14 fps possible with 1736x3256 with latest build. I shouldn't worry about card but I hear you.
Maximum 15fps with that resolution regardless of sd overclocking then image breaks.

Edit: Actually seems possible to reach 17fps with full resolution but no way continuous. Might be useful as a burst mode setting.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on July 05, 2020, 09:50:25 AM
Quote from: Danne on July 05, 2020, 09:14:13 AM
Edit: Actually seems possible to reach 17fps with full resolution but no way continuous. Might be useful as a burst mode setting.

17 fps would be more than welcome even if recording is limited to say 5-10 sec.  This would ease interpolation and jello in problematic scenes significantly.  I remember filming quite a lot at 17 fps with the 5D3 in the past and footage really came out beautifully.  You even implemented a 17 fps mode in that camera, remember?  From the practical filming point of view, some attention to that mode in the EOS-M and 100D would be well worth the effort with the new overclocking features.

By the way, 10-bit lossless is God sent for high-resolution scene filming in the full sensor readout 5k anamorphic mode.  With brightly lit scenes, there is no noticeable image quality degradation compared to 12 or 14 bits but you never have to worry that recording will stop prematurely even with high-contrast, very bright and even overexposed scenes.  Absolutely fantastic!   
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 05, 2020, 10:11:49 AM
1736x2930 16fps now available on eosm. If you want 17fps you can reduce reg 6014 in sub menu but not continuous in 17fps:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg228768#msg228768
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on July 05, 2020, 10:19:14 AM
Man, you are moving fast!  Very tempting, really!  Will test as soon as possible.  Thank you so much, Danne!
Title: Re: UHS-I / SD cards investigation
Post by: rinski on July 05, 2020, 10:21:55 AM
Hello, I have used in twixtor plugin with resolve 16 studio for 5d3 14 bits 1920x3240 15.003 anamorphic and it does not improve just in the interpolation at 24 fps, some advice to edit with this resolution.
Thank you.
Title: Re: UHS-I / SD cards investigation
Post by: ZEEK on July 05, 2020, 12:08:13 PM
Thanks Danne! 😎👍
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 05, 2020, 05:22:56 PM
Quote from: Danne on July 04, 2020, 05:54:25 PM
Exactly. That´s why benchmarks are so fast when in play mode. Higher fps reduces the effect.

Didn't know that, but can confirm now
I have got a [email protected] crop preset and write speed in this preset is 97Mb/s, green indicator and a message besides: 3% idle  :D

But some less good news, I have had some speed drops in 240Mhz. It doesn't happen that often, but it does happen.
Not sure what triggers it, but I think it's triggered by the patching done in crop_rec module (not 100% sure though  ???).
But yesterday I had 2 speed drops, both while extensively switching between different crop presets and recording clips and playback.
Today I had a speed drop, 2 or 3 times in a row (after camera restart) while trying out the same specific crop_rec preset.
Switched between some other presets and now I can't trigger it again in that specific preset...strange.

So I doubt if it is really the card that causes the speed drop, could be that it is triggered by the patching done in crop_rec module.

Edit:
So it doesn't look like a 5d3 only thing.
Does it on 5d3 happen with certain crop_rec presets, or while extensively switching between presets ?
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 05, 2020, 05:31:27 PM
for me, it is a little different...
i was getting speed drops until i did the in-camera low level format after patching.  no more speed drops since!

but the big problem with the 5d3 seems to be inability to work at all at speeds over 160mhz without a very lucky module :)

have you tried doing the low level format after patching?
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 05, 2020, 05:38:06 PM
Card has been low level formatted in camera several times last few days.

Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 05, 2020, 05:49:07 PM
5d3 is completely unreliable. Only 160Mhz is stable.
If using card spanning one won't notice because main recording is done to cf. I doubt the footage will be ok though on the sd card.
On the eosm 192Mhz seems rock solid. 240Mhz not working at all. Jip-Hop reported one time only write mode around 95Mb/s but read speed 20Mb/s on eosm. Not repeatable.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 05, 2020, 06:21:48 PM
Quote from: a1ex on April 03, 2018, 12:55:26 PM
Caveat: SDR104 requires tuning the sampling point (not implemented, not performed by Canon firmware, but doable). That might be required to avoid random errors, speed drops, or higher frequency - if the controller supports it.

Try disabling SDR104 Patch, it *might* solve some problems e.g 240 MHz on EOS M, Card full on 5D3, Speed drop on 6D maybe, give it a try:

Comment it out
/* enable SDR104 */
//patch_hook_function(sd_set_function, MEM(sd_set_function), sd_set_function_log, "SDR104");


Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 05, 2020, 06:26:03 PM
You might be right with that.
Disabled the SDR104 patching and see if speed drops again.  8)
Title: Re: UHS-I / SD cards investigation
Post by: Aleks N on July 05, 2020, 11:12:54 PM
Hi! I can test on 650D.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 05, 2020, 11:44:24 PM
For 650d you could try the build posted by me on page 18 of this topic (Reply #446 on: July 02, 2020, 01:58:05 PM)

The module in that post should work on 650d in combination with crop_rec_4K build from the experimental builds downloadpage on this site.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 06, 2020, 02:09:50 AM
good news for 5d3 owners:

i just used danne's new july 5 build to shoot an episode of my new series "mindful moments" at 3616*1536 @ 24fps/12 bit lossless and it went perfectly!  continuous recording for 13 minutes until one of my cards filled up :(
fortunately it was close enough to the end that i just stopped there...
i overexposed it slightly (3 solidly clipped light bulbs in the scene) just to push it really hard, and it handled it beautifully!
this was with spanning, of course, and the load was more even between the two than before, thanks to the slightly faster clock.  if 240mhz eventually works, this could mean 14 bits :)

3616*1536 is the highest resolution possible at 2.35:1 so this is maxed out, baby!

link will be in a "share your videos" thread as soon as i upload, which takes quite a while with my snail paced cellphone internet ;)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 06, 2020, 09:14:49 AM
I got some info regarding the switching problem ( to 48 MHz), not fully sure but it's likely true, In short: Instability issues for 192 MHz and 240 MHz presets are because **we are still in SDR50 Mode not SDR104,

After the switch to 48 MHz I was Unable to apply the overclock again even after un-patching the memory and re-enabling the overclocking task, it stuck on 21MB/s

By changing 4 to 3 in this line:
if (sd_setup_mode_enable && regs[sd_setup_mode_reg] == 4)   /* SDR50? */
to if (sd_setup_mode_enable && regs[sd_setup_mode_reg] == 3)   /* SDR50? */

And enabling un-patch memory for clearing previous overclock, and re-overclocking by the change above, now I can get ~41 MB/s without restarting the camera and after it switched to 48 MHz

I set the registers values to the default ones from Canon which are:
static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4,               0    };


I re-overclocked it a to 120 MHz or 132 MHz (the one below 160 MHz) and it gave 53 MB/s write speed in play mode, these are the settings:
static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3,               0    };

Overclocking more than that is not possible, gives Card full error, and in benchmark gives ~4657 MB/s in read/write , these values appears in benchmark when the Card full error is presented and when you can't read on write on the card,

So if it was this SDR50, [sd_setup_mode_reg] == 4)   /* SDR50? */

then is't this SDR25 ? [sd_setup_mode_reg] == 3)

And if this code able to bring SDR104:

static void sd_set_function_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    qprintf("sd_set_function(0x%x)\n", regs[0]);

    /* UHS-I SDR50? */
   // if (regs[0] == 0xff0002)
  //  {

     regs[0] = 0xff0003;

   // }

..
..

patch_hook_function(sd_set_function, MEM(sd_set_function), sd_set_function_log, "SDR104");


Why it can't bring SDR25 which will not accept a preset above 132 MHz ?

Maybe it could accept 160 MHz preset and that happen when I brought SDR104 peace of code again to the code but with some changes:
static void sd_set_function_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    qprintf("sd_set_function(0x%x)\n", regs[0]);

    /* UHS-I SDR50? */
   // if (regs[0] == 0xff0002)
  //  {

     regs[0] = 0xff0004;  /*now it's 0xff0004 instead of 0xff0003*/

   // }

..
..

patch_hook_function(sd_set_function, MEM(sd_set_function), sd_set_function_log, "SDR104");


And I applied 160 MHz preset (after the swtich) which is:
static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x2, 0x1D000201,        0x0,      0x201,      0x201,      0x100,        0x2,               0    };


Now it accepted preset more than 132 MHz, and I got 62 MB/s Write Speed, but not constant it gives Card full error after some seconds . .

Conclusion
I don't think SDR104 patch is an SDR104 patch

**Aren't we already in SDR104 by default ? and maybe the switch to 48 MHz mode is the SDR50 mode (according to SD Association and all the internet, max speed @ SDR104 is 104 MB/s , and SDR50 @ 50 MB/s so my results above because it switched to SDR50 not SDR25, and this make a lot of sense in the results above, and in the previous results I always commented out SDR104 patch, and I always got 160 MHz , 192 MHz, 240 MHz working fine, and with the patch enabled I had no difference in speed at all, so as a1ex said it's need some tuning to avoid random errors)

Card full error happens when you apply high frequency preset in not supported mode like SDR25, and SDR25 can't switch to 48 MHz because it is 48 MHz already so it gives Card full error

You can make a change from:
if (sd_setup_mode_enable && regs[sd_setup_mode_reg] == 4)   /* SDR50? */

to

if (sd_setup_mode_enable)   /* SDR50? */

This will apply any given registers values in any mode and the registers values will not change after the switch or sdSoftReset, so now instead of 21 MB/s speed when it switch to 48 MHz mode, it will give Card full error, now we know how to lock the values . . but how lock SDR50 or SD104 mode ?

I am not saying my assumptions are all true, but we are missing something . . maybe.
Title: Re: UHS-I / SD cards investigation
Post by: Leopold LJ on July 06, 2020, 12:47:44 PM
Quote from: rinski on July 05, 2020, 10:21:55 AM
Hello, I have used in twixtor plugin with resolve 16 studio for 5d3 14 bits 1920x3240 15.003 anamorphic and it does not improve just in the interpolation at 24 fps, some advice to edit with this resolution.
Thank you.

You don't have to use twixt plugin in resolve.
1- Use Retime process with Optical flow && Motion estimation Enhanced better --> Good
2- Use Retime process with Optical flow && sppped warp (AI engine that do much than interpolation but use neuronal network to imagine frames 16 to 24, 18 to 24 ...(very very heavy processing for the graphic card)
Title: Re: UHS-I / SD cards investigation
Post by: Aleks N on July 06, 2020, 06:28:21 PM
Quote from: Levas on July 05, 2020, 11:44:24 PM
For 650d you could try the build posted by me on page 18 of this topic (Reply #446 on: July 02, 2020, 01:58:05 PM)

The module in that post should work on 650d in combination with crop_rec_4K build from the experimental builds downloadpage on this site.

It works on 650D!

Everything started the first time on the "crop_rec_4k" firmware.
Results 60 MB/s before recording clips. I recorded two clips in ~ 1 minute in 14bit Losless. During recording, the indicator was always green, occasionally yellow. Recording did not stop even when moving the camera (previously interrupted).
After stopping the recording and restarting the camera, I made another benchmark. Result 65MB/s!

Сard: SanDisk Extreme Pro 128GB, 95/170 MB/s

https://youtu.be/IeNsbb8lSHg

I think this is enough for me so far) Thank you guys!


Link to this firmware (for 650D):
https://drive.google.com/file/d/1_W9yKHI6JtWufq1jhrXQMQ4KXh9lljSL/view?usp=sharing

or

https://yadi.sk/d/kihVgN_bFPCiVw
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 06, 2020, 07:52:29 PM
I used an alternative adress for uhs_regs and it works with 192Mhz as well:
static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC040060C, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr_192MHz[]   = {        0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 };


This:
static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };
To this:
static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC040060C, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };
Title: Re: UHS-I / SD cards investigation
Post by: Levas on July 06, 2020, 08:38:42 PM
If I'm not mistaken, register 0xC0400610 is set to 0x4 by default(Canon settings).
So looks like you're doing 'normal'   192Mhz setting with an extra register, 0xC040060c set to 0x4.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 06, 2020, 09:33:15 PM
Yes, this register 0xC040060C has no effect, in this preset all the values are the default ones from Canon, except 0xC0400600, it's original value is 0x3, by tweaking it to 0x8 gives the 192 MHz Preset, without doing any another changes.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 06, 2020, 09:41:37 PM
Yes, it's probably not used so disables that reg. Made 240Mhz work giving same write speed as 192Mhz. 160Mhz patch won't work with this.
Title: Re: UHS-I / SD cards investigation
Post by: RawCutz on July 07, 2020, 02:05:36 AM
Hi guys I totally appreciate what y'all are doing.

@theBilalFakhouri is it possible for you to compile a build for my 700D with the latest sd_uhs developments?

I have been using an old build compiled by you for some time now but it struggles with 14bit @ 1736x972 unless I under expose.

I used on old build from @Danne as well and it worked just fine but the same issue.

I have the Sandisk Extreme Pro 64GB 170mb/s.
Title: Re: UHS-I / SD cards investigation
Post by: RawCutz on July 07, 2020, 02:46:58 AM
@theBilalFakhouri

Your build is the only one that have worked with sd_uhs.

I have never had any success with the builds from Danne and Dfort.

Most likely because I dont know how to compile.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 07, 2020, 04:17:07 AM
Maybe we should look into:
https://bitbucket.org/hudson/magic-lantern/src/sd_uhs/modules/sd_uhs/sd_uhs.c#lines-59
drv_strength

Also note, if this isn´t applied:
    /* enable SDR104 */
    //patch_hook_function(sd_set_function, MEM(sd_set_function), sd_set_function_log, "SDR104");
    //SD_ReConfiguration();


We can skip this:
//static void sd_set_function_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
//{
  //  qprintf("sd_set_function(0x%x)\n", regs[0]);
   
    /* UHS-I SDR50? */
    //if (regs[0] == 0xff0002)
    //{
        /* force UHS-I SDR104 */
      //  regs[0] = 0xff0003;
   // }
// }


EDIT:
Also, basically what is needed for the eosm is this(reduced the code):
/**
* Experimental SD UHS overclocking.
*/

#include <module.h>
#include <dryos.h>
#include <patch.h>
#include <console.h>
#include <config.h>
#include <lens.h>

/* camera-specific parameters */
static uint32_t sd_setup_mode = 0;
static uint32_t sd_setup_mode_in = 0;
static uint32_t sd_setup_mode_reg = 0xFFFFFFFF;
static uint32_t sd_set_function = 0;

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr_160MHz[]   = {        0x2,        0x2,                             0x1, 0x1D000001,        0x0,      0x100,      0x100,      0x100,        0x1 };   /* overclocked values: 160MHz = 96*(4+1)/(2?+1) (found by brute-forcing) */
static uint32_t sdr_192MHz[]   = {        0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 };
static uint32_t sdr_240MHz[]   = {        0x8,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3 };

static uint32_t uhs_vals[COUNT(uhs_regs)];  /* current values */
static int sd_setup_mode_enable = 0;
static int turned_on = 0;
static CONFIG_INT("sd.sd_overclock", sd_overclock, 2);

/* start of the function */
static void sd_setup_mode_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    qprintf("sd_setup_mode(dev=%x)\n", regs[0]);
   
    /* this function is also used for other interfaces, such as serial flash */
    /* only enable overriding when called with dev=1 */
    sd_setup_mode_enable = (regs[0] == 1);
}

/* called right before the case switch in sd_setup_mode (not at the start of the function!) */
static void sd_setup_mode_in_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    qprintf("sd_setup_mode switch(mode=%x) en=%d\n", regs[sd_setup_mode_reg], sd_setup_mode_enable);
   
    if (sd_setup_mode_enable && regs[sd_setup_mode_reg] == 4)   /* SDR50? */
    {
        /* set our register overrides */
        for (int i = 0; i < COUNT(uhs_regs); i++)
        {
            MEM(uhs_regs[i]) = uhs_vals[i];
        }
       
        /* set some invalid mode to bypass the case switch
         * and keep our register values only */
        regs[sd_setup_mode_reg] = 0x13;
    }
}

static void sd_overclock_task()
{
   
    /* install the hack */
    if (sd_overclock == 1) memcpy(uhs_vals, sdr_160MHz, sizeof(uhs_vals));
    if (sd_overclock == 2) memcpy(uhs_vals, sdr_192MHz, sizeof(uhs_vals));
    if (sd_overclock == 3) memcpy(uhs_vals, sdr_240MHz, sizeof(uhs_vals));
    patch_hook_function(sd_setup_mode, MEM(sd_setup_mode), sd_setup_mode_log, "SD UHS");
    patch_hook_function(sd_setup_mode_in, MEM(sd_setup_mode_in), sd_setup_mode_in_log, "SD UHS");

}

static struct menu_entry sd_uhs_menu[] =
{
    {
        .name   = "sd overclock",
        .priv   = &sd_overclock,
        .max    = 3,
        .choices = CHOICES("OFF", "160MHz", "192MHz", "240MHz(feeling lucky)"),
        .help   = "Select a patch and restart camera. Disable with OFF and restart",
        .help2  = "Proven working with 95Mb/s and 170Mb/s cards",
    }
};

static struct menu_entry sd_uhs_menu1[] =
{
    {
        .name   = "sd overclock",
        .priv   = &sd_overclock,
        .max    = 3,
        .choices = CHOICES("OFF", "160MHz", "192MHz", "240MHz (feeling lucky)"),
        .help   = "Select a patch and restart camera. Disable with OFF and restart",
        .help2  = "Proven working with 95Mb/s and 170Mb/s cards",
    }
};

static unsigned int sd_uhs_init()
{
   
    //needed with manual lenses cause it stalls liveview. Maybe helps for cams like 6D. To be tested.
    while (is_movie_mode() && !lv)
    {
        msleep(100);
    }
   
    if (is_camera("EOSM", "2.0.2"))
    {
        sd_setup_mode       = 0xFF338D40;
        sd_setup_mode_in    = 0xFF338DC8;
        sd_setup_mode_reg   = 1;
        sd_set_function     = 0xFF63EF60;
        if (sd_overclock)
        {
            sd_overclock_task();
            turned_on = 1;
        }
    }
   
    menu_add("Movie", sd_uhs_menu, COUNT(sd_uhs_menu));
    menu_add("Debug", sd_uhs_menu1, COUNT(sd_uhs_menu1));
   
    return 0;
}

static unsigned int sd_uhs_deinit()
{
    return 0;
}

MODULE_INFO_START()
MODULE_INIT(sd_uhs_init)
MODULE_DEINIT(sd_uhs_deinit)
MODULE_INFO_END()

MODULE_CONFIGS_START()
MODULE_CONFIG(sd_overclock)
MODULE_CONFIGS_END()



Title: Re: UHS-I / SD cards investigation
Post by: Bender@arsch on July 07, 2020, 01:43:05 PM
Because of the 131mb/s writespeed limit, I tested just for fun the full resolution mode (5d3, 5784x3254 7,4fps at 10bit) with card spanning and the 160mhz module.

writspeed:
CF: 105MB/s, 42% idle
SD: 65MB/s, 12% idle

=170MB/s with green icon ;)
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 07, 2020, 01:45:44 PM
Hehe, not bad! Reduced fps maximizes card speed.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 07, 2020, 01:52:13 PM
doesn't this point to cpu overhead within the magic lantern code?
slower fps reduces the burden on the cpu, no?
it would be great to know where the most cpu resources are going during usage.  is there a way to measure this, akin to "process explorer" in windows that shows exactly how much cpu resources are being used by each running process?
Title: Re: UHS-I / SD cards investigation
Post by: Bender@arsch on July 07, 2020, 01:58:30 PM
OK I think this is a failure:
3840x1440 x 23,976fps = ~130 mega pixel
5784x3254 x 7,4fps = ~140 mega pixel

I need ~125mb/s for 4K at 10bit, so 170mb/s is not correct I think.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 07, 2020, 02:02:58 PM
hopefully the write speed indicator is a true readout of the rate of data being written to the cards.
Title: Re: UHS-I / SD cards investigation
Post by: Bender@arsch on July 07, 2020, 02:07:38 PM
OK, I checked it out:
~1600mb (cf 830, sd 768), 102
frames =~14sec
1600/14 =114mb/s :(
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 07, 2020, 02:55:00 PM
can you try the same comparison between rates that are reported versus recorded :P with both full sensor recording and cropped modes?  let's see if the same discrepancy exists...
Title: Re: UHS-I / SD cards investigation
Post by: Bender@arsch on July 07, 2020, 04:20:20 PM
Ok. Here are some short tests:

Full res. 5784x3254 7,4fps 10bit
CF: 105 42% idle
SD: 65 12%idle
=170MB/s
-> 1600 (cf 830, sd 768), 102 frames =~14sec
1600/14 =114mb/s

4K 3840x1536 23,976fps 10bit
CF: 70,4, 2% idle
SD: 50,4 6% idle
=120,8MB/s
-> 2791 (cf 1619, sd 1172), 558 frames =~23sec
2791/23 =121,3mb/s

4K half 4096x2304 11,988fps 10bit
CF: 108   22% idle
SD: 61,5 80% idle
=169,5MB/s
-> 1531 (cf 1340, sd 197), 191 frames =~16sec
1531/16 =95,6mb/s

Anamorphic 1808x2300 23,976fps 10bit
CF: 93 30% idle
SD: 53 66% idle
=146MB/s
-> 1397 (cf 1039, sd 358), 401 frames =~17sec
1397/17 =82,17mb/s

Anamorphic 1808x2300 23,976fps 12bit
CF: 90 30% idle
SD: 53 50% idle
=143MB/s
-> 1271 (cf 910, sd 361), 334 frames =~14sec
1271/14 =90,7mb/s

Anamorphic 1808x2300 23,976fps 14bit
CF: 77 15% idle
SD: 53 20% idle
=130MB/s
-> 1714 (cf 991, sd 723), 388 frames =~16sec
1714/16 =107mb/s
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 07, 2020, 04:47:51 PM
well, isn't that interesting!

looks like we need to stick to only one preset for accurate speed reporting during recording...

hopefully someone familiar with the codebase can figure out what's going on!

nice catch...
Title: Re: UHS-I / SD cards investigation
Post by: rinski on July 07, 2020, 09:56:02 PM


Full anamorphic 1.1 1920x3760 14 bits 15,003 fps, 6 minutes 240 mhz with little drop in speed 91 MB / s cf lexar 1066x 74.5 MB / s, sdxc sandisk extreme pro 170 MB / s. Thanks for your work.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 07, 2020, 11:42:26 PM
Canon 700D with Sandisk Extreme Pro 170 MB/s @ 240 MHz:
2560x1280 10-bit lossless @ 23.976 FPS - Continuous
2560x1440 10-bit lossless @ 23.976 FPS - Mostly Continuous (Depending on scene with Frozen Preview)

Canon 700D with Sandisk Extreme Pro 95 MB/s @ 240 MHz:
2560x1440 10-bit lossless @ 23.976 FPS  - Continuous (Sometimes with Frozen Preview)

-However 240 MHz overclock with Sandisk Extreme Pro 95 MB/s isn't stable.
-2560x1280 is 2:1 Aspect Ratio
Title: Re: UHS-I / SD cards investigation
Post by: yourboylloyd on July 08, 2020, 12:43:46 AM
I want to test on my t3i. I'm going to buy a bestbuy today to get a faster sd card. Is there a test build/module for that one?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 08, 2020, 12:49:57 AM
Quote from: yourboylloyd on July 08, 2020, 12:43:46 AM
I want to test on my t3i. I'm going to buy a bestbuy today to get a faster sd card. Is there a test build/module for that one?

Nope, t3i is a DIGIC 4 camera, many features aren't there yet for t3i, like lossless compression, higher resolutions and sd overclock, I do not recommended it for ML use with these features.
Title: Re: UHS-I / SD cards investigation
Post by: reddeercity on July 08, 2020, 02:20:58 AM
It funny that you are talking about t3i/d4  cam , last night i was reading though my 5d2 rom disassembly
Looking for CF card udma string to set udma speed (e.g. udma6 5d2/50d to umda7 (120Mb) ) and Lossless stuff
I found the SD_SoftReset etc. ... all the stuff that you guy are playing around in the D5 Cams e.g. 700D etc. ... in the d4 rom
So the max speed in the rom for SD card is 48MHz plus there  is a 24MHz mode also , not sure what speed the T3i records at ?
but I think it's doable  to apply this code to t3i's if there not at max already , to be sure t3i can't go over 48MHz unless
the bus speed is over clocked . & Hi res crop_rec  is only a matter of some one following what i did to the 50D .
It would be the same , i totally expect crop_rec to work on the t3i , but i don't have the time to do it my self (i will help thou :) )

Any how found a couple of string for CF
ff38b1f8: STRING:  'cfIdentifyDrive: Set UDMA( Mode=%d )'
ff38b1b0: 128f20dc addne r2, pc, #220 ; *'CF_DeviceCreate: SoftReset
ffbdbb48: STRING:  'CF_GetAccessTiming : DatTim = %d, DatMod = %d'
 

I hoping to set CF to UDMA7 from 6 by code ,

I wondering are you able to set the band width when you are switch to hi-speed in the sd card on 700d ?
I see this in d4 rom
ff395544: STRING:  'SD_SetBusWidth: Width(Width=%d)'
ff395564: STRING:  'SD_SetBusWidth(CMD55)'
ff3955b0: STRING:  'SD_SetBusWidth(ACMD6)'


Also when over clocking to a higher speed dose the voltage increase to support the data output ?
Not sure how you would check that , it seems to me that same of the unstable results are some missing configuration or reg 
my 2cents worth  :)
Here a older (2001) Document on SD card I/O ,  nice read
http://ugweb.cs.ualberta.ca/~c274/resources/hardware/SDcards/SD_SDIO_specsv1.pdf

Great Work on this @theBilalFakhouri ,

Edit:
found this in the 700d 115 rom , i have
ff74aafc: e28f2e26 add r2, pc, #608 ; ff74ad64: (20746553)  *"Set 1.8V Signaling"
ff74d684: e28f20fc add r2, pc, #252 ; ff74d788: (65536473)  *"sdSendVoltageSwitch"

interesting ! i think the voltage for the older cards are around 3.3Vdc , or unless the newer cards use less voltage .
maybe other cam are having a voltage switching problem ? just a thought
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 08, 2020, 03:36:18 AM
@reddeercity

I also have the most if not all of CF stuff in my 700D ROM, probably Canon has generic sections in their cameras firmware.

Quote from: reddeercity on July 08, 2020, 02:20:58 AM
Also when overclocking to a higher speed dose the voltage increase to support the data output ?

Nope, the voltage stays at 1.8 V for UHS-I cards
(https://i.ibb.co/wwNgg8S/sdr104specs.png) (https://ibb.co/wwNgg8S)

5D3 needs 1.8 Volt patch maybe to enable the overclocking, see sd_uhs.c .

For SD_SetBusWidth I don't know if it possible to call it I even don't know what it does, I need more Reverse Engineering and Assembly language skills in addition to C skills, Last time I tried to call sdSoftReset with Sandisk Extreme PRO 95 MB/s the card didn't boot in the camera again I thought I broke it or modified its firmware, fortunately after a quick format using SD Formatter from SD Association it worked again.

All what I did to achieve higher clock speeds is playing with the registers which control the clock speed, but reverse engineering work and figuring out how to call these registers and changing its values are done by a1ex , I am working to improve my skills to do some tasks on my own :D

Quote from: reddeercity on July 08, 2020, 02:20:58 AM
I hoping to set CF to UDMA7 from 6 by code ,

Good Luck with this! if it technically possible it might be doable.

Quote from: reddeercity on July 08, 2020, 02:20:58 AM
it seems to me that same of the unstable results are some missing configuration or reg 

Yeah that what's I am thinking too, maybe some more calls are needed.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 14, 2020, 08:56:18 AM
what do you all think about the 5d3 beyond 160 mhz?

160 has been rock solid, but not quite fast enough to do 3616*1536 with no sudden stops.  i'm wondering if 192 might be just fast enough!

has anyone tried faster cards than the sandisk 170mb/s? 
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 14, 2020, 09:07:04 AM
Quote from: 70MM13 on July 14, 2020, 08:56:18 AM
has anyone tried faster cards than the sandisk 170mb/s? 

I suppose there are no faster cards. UHS-II cards have to run in UHS-I mode and all conventional tests (without overclocking) are showing a bit lower numbers. Fastest results are with SanDisk's properietary overclocking UHS-I cards (labeled > 95 MB/s) and SanDisk cardreaders supporting overclocking protocol. See https://cameramemoryspeed.com

Examples:
Fastest UHS-I card in SanDisk reader (supporting proprietary protocol): ~100 MByte/s. But in all other cardreaders: Up to ~85 MByte/s.
Fastest UHS-I card in conventional readers: ~93 MByte/s.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 14, 2020, 09:34:19 AM
nice link, thanks...

i looked at the eos r test, which seemed to be the newest one:
https://www.cameramemoryspeed.com/canon-eos-r/best-sd-cards/

"The camera is also compatible with UHS-I cards and supports SDR104 (up to 104 MB/s). The fastest UHS-I cards averaged up to 75.6 MB/s write speed in the EOS R."

it was done in 2018, so the fastest uhs-1 card they tested was the 95mb/s version.  it would be interesting to see if the 170 fares any better.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on July 14, 2020, 12:02:40 PM
Quote from: Bender@arsch on July 07, 2020, 01:43:05 PM
Because of the 131mb/s writespeed limit, ...

I was wondering where does this number come from?  And if the 5D3 cannot write faster, what is this SD-overclocking effort on the 5D3 all about?  Can't we get this write speed just by using Card spanning?  The 5D3 can write at 120 MB/s on the CF card and 21 MB/s on the SD card.  Adding these numbers results in 141 MB/s write speed which is still by 10 MB/s more that 131 MB/s. 

If someone could explain this to me, I should greatly appreciate that.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 14, 2020, 12:25:24 PM
this is a question that is largely unanswered at this time.

there's overhead involved that changes the way it performs when spanning is enabled.  immediately the CF writing is slowed down as a result.  and in actual usage, the CF rarely goes far above 90 mb/s on it's own to begin with.  once spanning is enabled, it drops from there...

in my usage of spanning and overclock combined, i find that the faster the sd is clocked, the better the results overall.  it gets closer to 50/50 writing as the sd clock goes up.  when it was briefly working at 240 mhz, recording was very robust at the highest resolution and bit depth with very close to 50/50 writing.

i think there is a bottleneck in the software loop that is resulting in the 131 mb/s total speed limit, but it is only a guess.  if anyone is interested in testing it, i would suggest disabling components in mlv_lite that are not needed for video recording (such as sound recording and anything else you can think of) to see if this holds true. it would be interesting to see the results.

maybe higher framerates would be achievable...

edit: let me rephrase that last part to avoid potential confusion: i don't mean disabling them by the user in the magic lantern menus, rather disabling them in the code and recompiling the actual module, to tighten the main loop.  that may be too general of a description, and i don't know enough about the codebase for a better guess about what can be done to tighten it.  it could possibly be only related to i/o routines, or even more specific still.

or it could simply be a hardware limitation creating the 131 mb/s limit.

i haven't done programming in 36 years ( 6809 assembly) and all this high level language stuff is not my domain.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on July 14, 2020, 08:52:54 PM
Thanks a lot 70MM13,

This 131 MB/s number is pretty precise.  I wonder how this number has been measured or otherwise determined.  Can anyone elaborate on that?
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on July 14, 2020, 09:23:04 PM
for me it is just from experience.  no matter how high the overclocking went during that short time it was working here, the total never exceeded 131...
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on July 15, 2020, 04:11:14 AM
Got it.  Thanks again.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on July 16, 2020, 06:37:23 AM
Hi and thank you all working hard on the new overclocking feature on our cameras.

I am writing to ask for some help in getting the 5D3 working here.  Eager to test this exciting new overclocking development, I borrowed the 5D3 and tried to get it to work with Sandisk Extreme Pro cards (64 MB/160 MB/s for the CF card and 64 MB/95MB/s for the SD card).  I did the following:

1) Formatted both cards with NTFS on the PC using a card reader.  I tried also formatting the cards in ExFAT.  These are the two formatting options that I have on my Win7x64 OS.

2) Formatted both cards in the camera

3) Performed a fresh install of the latest July 14-th build by Danne

4) With only the SD-card in camera, I activated SD-overclocking and restarted the camera.  Patch was successful - that's what the message on the screen says.

5) In the video mode, without touching anything else, I start recording with the default settings as set in the build after fresh install.  Camera stops recording immediately and nothing gets recorded on the SD card.  I get the same behavior also with both cards in the camera.

6) I tried in-camera format after running the SD-overclocking patch.  I get the "Card full" message and the SD card becomes unaccessible.  It needs to be reformatted in the PC to continue working again.  I get this behavior with both: the NTFS and ExFat format options.

I should also mention that, without SD-overclocking, everything works fine, even Card spanning.  If I do not overexpose, with the 3k 1:1 mode, I get continuous recording at 3072x1728/24 fps/10 bits lossless when I activate Card spanning with both cards in the camera.  In this case, camera writes nicely on both cards at an overall speed of 114 MB/s and footage opens in MLVApp without problems.  No corrupt frames either.  This makes me think that the overclocking patch seems to be the trouble maker.

I know, some of you got it working.  If you could give me some advice on what am I doing wrong and what else could I try to get the 5D3 working, (step by step instructions if possible), I should greatly appreciate that.

Thank you in advance.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 16, 2020, 06:52:28 AM
Ntfs? Either fat32 or possibly exfat. Also low level format in camera. If it's a sandisk extreme pro card it should work. If not working maybe defective or weak card. Some reports of 95mb not working stable but 170Mb/s seems to work.
Also. Why are you formatting card in camera after sd patching? Seems dangerous to me. Instead format and then reinstall magic lantern from scratch. Before applying the sd patch?
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on July 16, 2020, 07:52:37 AM
Just tried that again, this time with the July 15-th build.  I first ExFat formatted the SD-card in the PC, then, without ML installed, performed a low-level format on it in the camera.  Then I copied the files from the build onto the card in the PC.  Then I installed ML and ran the sd-patch in the photo mode.  Then I ran the benchmark test in the photo mode and got some very strange numbers for read and write speeds - 5005,3 MB/s and similar.  Then I switched to video mode, activated RAW video and without touching anything else, started recording. I got immediately the "Card full" message and then card was unaccessible.  Right now, I am formatting it again in the PC using the ExFat file system with the Quick option UNchecked.

My SD card seems to be OK.  It is a genuine SanDisk indicating 90 and 95MB/s write and read speeds, respectively, when running the ATO benchmark test. 

Thank God, it's working again, without overclocking, of course.  I am stopping the overclocking experiments for now.  It indeed seems too dangerous, to me too.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 16, 2020, 07:59:12 AM
I will need a screen recording of both your card and full workflow. I have no reports describing your issues with extreme pro cards: I suspect user error or fake card. Sorry, but text descriptions will not help in this case.
Title: Re: UHS-I / SD cards investigation
Post by: IDA_ML on July 16, 2020, 08:11:47 AM
I don't have another card to try but the procedure I described in my post #628 is perfectly reproducible with the one that I have.  I tried it at least 5 times with both - the July 14 and 15 builds.  Sorry, I cannot provide screen recordings at the moment.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 16, 2020, 08:24:07 AM
Well. Whenever you can. In this case needed to exclude user error. Meanwhile others might wanna report how overclocking works with their cards. Especially problematic patching.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 16, 2020, 08:58:21 AM
Depending on card size cam format will apply FAT32 or ExFAT . You have to check which filesystem is actually in use. If you want to use ExFAT with smaller cards you have to avoid formatting cards in cam.
a1ex coded forced ExFAT formatting but it is not included in any build.
Title: Re: UHS-I / SD cards investigation
Post by: LeandroFreitas on July 16, 2020, 12:37:43 PM
Quote from: Walter Schulz on July 02, 2020, 09:36:30 PM
Tested previous version. I have no words ...
(https://abload.de/img/bench30fxjrs.png)

What card did you use?
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 16, 2020, 12:39:53 PM
SanDisk Extreme Pro 95 MB/s 128 GB.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 16, 2020, 03:02:43 PM
Quote from: Walter Schulz on July 16, 2020, 12:39:53 PM
SanDisk Extreme Pro 95 MB/s 128 GB.

Does the speed drop to 21 MB/s when doing some RAW recording e.g 1736x976 @ 23.976 14-bit uncompressed ?

Could you make a test with above settings (and other settings require 67 MB/s at lest) by filling the card up to 32 GB or more?

I am wondering if Sandisk Extreme Pro 95MB/s with higher capacity than 32 GB could handle the 240 MHz overcock or not.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 17, 2020, 07:27:02 AM
Quote from: theBilalFakhouri on July 16, 2020, 03:02:43 PM
Does the speed drop to 21 MB/s when doing some RAW recording e.g 1736x976 @ 23.976 14-bit uncompressed ?

It drops. Benchmark in video mode: ~21 MByte/s.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 18, 2020, 10:57:04 AM
Quote from: theBilalFakhouri on July 16, 2020, 03:02:43 PM
Could you make a test with above settings (and other settings require 67 MB/s at lest) by filling the card up to 32 GB or more?

Used another card and recorded 9 and 10 minutes at around 67.9 MByte/s. Files 36.4 GB and 39.8 GB.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 18, 2020, 11:51:10 AM
Interesting. What card exactly?
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 18, 2020, 02:02:44 PM
SanDisk Extreme 128 GB microSDXC
V30 U3 A2. Red-gold Label.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 18, 2020, 02:05:05 PM
Micro sd. Nice. Have one for my drone. Will give that one a try tomorrow.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 18, 2020, 02:16:37 PM
Quote from: Walter Schulz on July 18, 2020, 10:57:04 AM
Used another card and recorded 9 and 10 minutes at around 67.9 MByte/s. Files 36.4 GB and 39.8 GB.

Good results, and does the speed drop to 21 MB/s after recording some clips in different mods?
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 21, 2020, 01:14:27 PM
Having troubles to get reproducable results. More test runs needed.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on August 09, 2020, 02:30:38 PM
check out this guy's CF thermal throttling information.  i wonder if it also applies to SD cards.

could it be the reason for the slowdowns?

https://alikgriffin.com/canon-r5-overheating-with-cfexpress-card-are-the-cards-to-blame/
Title: Re: UHS-I / SD cards investigation
Post by: Dmitriy84759 on September 14, 2020, 10:05:26 AM
Advice me please. I'm about SD overclocking and card spanning.  I have 5dm3 with Sandisk Extreme Pro CompactFlash 256Gb (160/140 Mb/s) CF card.
Should I buy the fastest SanDisk Extreme Pro SDXC UHS-I U3 V30 (170/90 MB/s) SD card or it enough just SanDisk Extreme SDXC Class 10 UHS-I U3 V30 (150/70 MB/s) to have best perfomance with overclocking and card spanning with CF card?
Should I buy same size of SD card (256Gb) as my main CF card or it will enough 128Gb or less?
Title: Re: UHS-I / SD cards investigation
Post by: ilguercio on September 15, 2020, 11:23:32 AM
Hi guys, haven't been around for a long time.
I am a bit confused at to what the latest stable build with SD card overclock is at the moment.
The one i downloaded is missing this module.
Is there any build that contains the lot and is the most updated?
Cheers :)
Title: Re: UHS-I / SD cards investigation
Post by: Levas on September 16, 2020, 10:26:46 AM
Not sure for which camera you need it ?
For 6d you can find the latest complete build with sd_uhs module in this post:

https://www.magiclantern.fm/forum/index.php?topic=15088.msg229094#msg229094 (https://www.magiclantern.fm/forum/index.php?topic=15088.msg229094#msg229094)
Title: Re: UHS-I / SD cards investigation
Post by: ilguercio on September 16, 2020, 06:00:26 PM
Quote from: Levas on September 16, 2020, 10:26:46 AM
Not sure for which camera you need it ?
For 6d you can find the latest complete build with sd_uhs module in this post:

https://www.magiclantern.fm/forum/index.php?topic=15088.msg229094#msg229094 (https://www.magiclantern.fm/forum/index.php?topic=15088.msg229094#msg229094)
Cheers, it's for a 6D indeed. Looks like i got to catch up with the latest developments :)
Thank you.
Title: Re: UHS-I / SD cards investigation
Post by: tmilardo on September 23, 2020, 11:35:14 AM
I'm able to bench ~91MB/s write on my EOS M at 240MHz once in a while. Just for the first pass of the Quick R/W benchmark. Sandisk Extreme Pro 170MB/s 256GB microSD. I can normally do about 78MB/s on 192MHz. Is there any sort of log or something I can share to help development? Would love to have access to higher bitrates.
Title: Re: UHS-I / SD cards investigation
Post by: tmilardo on September 27, 2020, 10:57:39 AM
First, my 192MHz benchmarks:

Play mode
(https://images2.imgbox.com/8a/b1/Lr16F8Ei_o.png)

Photo mode
(https://images2.imgbox.com/fd/6e/nDQbLZbj_o.png)

Movie mode
(https://images2.imgbox.com/1f/00/rYrnnlRN_o.png)



My 240MHz benchmarks:

Play mode
(https://images2.imgbox.com/ae/8e/wwNMEhwX_o.png)

Photo mode
(https://images2.imgbox.com/fc/51/aeg0oSFq_o.png)

Movie mode
(https://images2.imgbox.com/6f/86/prKelFBx_o.jpg)

Movie mode won't print a benchx.ppm. There is a memory error in the debug menu:
(https://images2.imgbox.com/2c/75/m6KRi9G2_o.jpg)

This is what I see when entering said menu:
(https://images2.imgbox.com/f8/e6/PQXL3sLE_o.jpg)

From here the device is frozen. I have to physically remove the battery to power off.

All live view benchmarks are done with the camera looking at an overexposed scene. Note that I can't even match my 240MHz speeds on my SD card reader.

The second write pass of the 240MHz movie benchmark goes at full speed until around 80/100, at which it goes down to a speed similar to the ~20MB/s passes. This behaviour (reproducible) is interesting, as other modes never seem to change speeds during a pass, and never sustain the 240MHz speed past the first pass.


Merged your posts. The direct image links appear to be broken, but the link to the folder works and shows all of the images. //Audionut
Found the edit button. Fixed. //tmilardo
Title: Re: UHS-I / SD cards investigation
Post by: ZEEK on October 07, 2020, 12:43:50 PM
Testing the 512GB SanDisk Extreme PRO again @240Hz Patch, best card I've used to date. Tested in 1080 Mode 16:9, 10 Bit on the EOS M.

Photo Mode SD Benchmark:
Read: 105.9 MB/s
Write: 88.5 MB/s
https://drive.google.com/file/d/10bUxjQG-p3K2bGQ2KmTHoj50aPIrEx2U/view?usp=sharing

Video Mode SD Benchmark:
Read: 105.9 MB/s
Write: 89.3 MB/s
https://drive.google.com/file/d/1HBabqM5HxDGd7aefe27YuTPlm12Jgu-n/view?usp=sharing

*Note, I could also record 2.8K RAW on the EOS M @2.35:1 with green signal for over a minute.
https://drive.google.com/file/d/1SQQzYvFAt1p0heYqJu_JS3AGb7l9Qz5u/view?usp=sharing
Title: Re: UHS-I / SD cards investigation
Post by: 2blackbar on October 07, 2020, 12:51:13 PM
Thanks for info, its at the price of the camera itself so ill pass on it for now but its interesting, it could maybe record in 3K 10 bit ?
Ill try to find cheaper alternative with similar specs
Title: Re: UHS-I / SD cards investigation
Post by: ZEEK on October 07, 2020, 12:56:35 PM
If you're a deciated ML user who travels a lot and shoots high resolution Video, it will definitely come in handy. I was hesitant at first, but now no regrets with this purchase.

Unfortunately 3K Raw on the M is max 18/20fps. 2.8K is the best Resolution so far.

With the 192Mhz patch I get about 5-12 Seconds, with the 240MHz patch, 1 Minute & above. Not sure about the consequences of using the highest patch and how taxing it can be on the card in long term use. Don't think it should be too bad.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on October 07, 2020, 01:06:11 PM
Beautiful numbers Zeek.
Could we have a 5D3 tester with this card? Let me know if so.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on October 07, 2020, 01:44:51 PM
A user reached to me last month with Sandisk Extreme PRO 64 GB 280 MB/s (The following photo sent by him):

(https://i.ibb.co/CJXXkr6/Photo-2020-10-07-14-10-11.jpg) (https://imgbb.com/)

-Unfortunately benchmarking this card at 240 MHz on 700D the speed would drop to 21 MB/s.

At 192 MHz it's fine:
(https://i.ibb.co/5TwHjrW/Photo-2020-10-07-14-14-58.jpg) (https://imgbb.com/)

Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on October 07, 2020, 01:58:41 PM
One must avoid comparing different cards but this is what the 32 GB variety does:
https://www.cameramemoryspeed.com/reviews/sd-cards/sandisk-extreme-pro-280-mbs-uhs-ii-32gb-sdhc-memory-card/
One of the early UHS-II cards with serious speed limit in UHS-I compatibility mode. Not able to beat 50 MByte/s in UHS-I write mode.

If you are able to use a decent UHS-I reader you may be able to verify/falsify.
Title: Re: UHS-I / SD cards investigation
Post by: smasry on October 07, 2020, 02:40:05 PM
Quote from: Danne on October 07, 2020, 01:06:11 PM
Beautiful numbers Zeek.
Could we have a 5D3 tester with this card? Let me know if so.

Hi Danne, I have the 512Gb x170 Sandisk SD and a 5d3 running 1.2.6, and am happy to test.

Just point me to the right build and tell me what to do, and I'll post the results up here.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on October 07, 2020, 03:00:34 PM
Great.
I need to prepare a build later tonight. However. You need firmware 1.1.3 or 1.2.3 to even be able and install magic lantern. Ever installed before?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on October 08, 2020, 10:30:11 AM
Quote from: names_are_hard on July 04, 2020, 09:55:17 PM
Adding active cooling to the card might also work but doesn't seem very practical.  Might be interesting to see if overclocking is more reliable inside a fridge!  We might learn if heat is a real factor here.  Drill holes into the battery / card compartment in advance for best testing ;)

Tested today Sandisk Extreme PRO 32 GB 95 MB/s with Canon 700D inside the fridge:

This card benchmarks 99 MB/s write speed in PLAY Mode compared to Sandisk Extreme PRO 170 MB/s card which benchmarks 90 MB/s write speed.

The speed never dropped to 21 MB/s , but at longer recording times the speed would drop and the recording would stop automagically, of course this is temporary drop, it didn't switch to safe mode (48 MHz mode) which gives 21 MB/s , starting another recording you will have full speed again until at some point it will stop automagically.

So yeah I think the card struggle with high frequency such as 240Mhz even if the temperature wouldn't be that high.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on October 08, 2020, 10:42:49 AM
Tests with a 512gb card should be performed on 240Mb/s also on other cams then eos m.
Title: Re: UHS-I / SD cards investigation
Post by: smasry on October 08, 2020, 02:25:50 PM
Quote from: Danne on October 07, 2020, 03:00:34 PM
Great.
I need to prepare a build later tonight. However. You need firmware 1.1.3 or 1.2.3 to even be able and install magic lantern. Ever installed before?

Sorry, Danne, forgot my firmware version... it is 1.2.3.

And yes, I've been using ML for years now, am presently on your Nightly.2020Jul05.5D3123, as reported by the Magic Lantern launch menu.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on October 08, 2020, 02:56:08 PM
Ok. Work only with sd card in your camera.

You are gonna need this build:
https://bitbucket.org/Dannephoto/magic-lantern_dannephoto_git/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2020Sep14.5D3123.zip

Replace the sd_uhs.mo to the one below:
https://bitbucket.org/Dannephoto/raw2mlv/downloads/sd_uhs.mo

Enable bench.mo. The rest of the modules will be enabled automatically.

1 - Enable the sd uhs patch called 240Mb/s and restart camera.
2 - Enter photo mode and run the 1 minute benchmark test. Are you getting high readings? Should be bitmap file on your sd card recorded.
3 - Now take a photo, then rerun benchmark test. Are you still having high numbers? If not your card will not respond well with 240Mb/s. If results still are high move on to step 4
4 - Enter movie movie mode. Start recording in anamorphic mode. Do about three to 5 takes. Everything orking as expected? If so, this card should be working with the patch.
Title: Re: UHS-I / SD cards investigation
Post by: smasry on October 08, 2020, 05:20:53 PM
Quote from: Danne on October 08, 2020, 02:56:08 PM
Ok. Work only with sd card in your camera.

You are gonna need this build:
https://bitbucket.org/Dannephoto/magic-lantern_dannephoto_git/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2020Sep14.5D3123.zip

Replace the sd_uhs.mo to the one below:
https://bitbucket.org/Dannephoto/raw2mlv/downloads/sd_uhs.mo

Enable bench.mo. The rest of the modules will be enabled automatically.

1 - Enable the sd uhs patch called 240Mb/s and restart camera.
2 - Enter photo mode and run the 1 minute benchmark test. Are you getting high readings? Should be bitmap file on your sd card recorded.
3 - Now take a photo, then rerun benchmark test. Are you still having high numbers? If not your card will not respond well with 240Mb/s. If results still are high move on to step 4
4 - Enter movie movie mode. Start recording in anamorphic mode. Do about three to 5 takes. Everything orking as expected? If so, this card should be working with the patch.

Ok, Danne, done 1, results of step 2 are:

- Following step 2 from scratch: https://imgur.com/LoVts4q

3.
- Tried to take a photo, got a warning about card 2 being unavailable, followed by an Err 2 crash. Upon restart, deleted the photo (which *did* save), and reran step 2 with higher benchmark results: https://imgur.com/slAHOrB
- Just in case, I restarted the camera, deleted the new images, ran the benchmark, took two more photos, and reran the test with equally high results: https://imgur.com/bYnbncT

4.
- Entered movie mode, tried to record in anamorphic mode, got 'CARD FULL' error, over and over again. Tried restarting the camera, trying to record video failed again with the same message: https://imgur.com/8M5XF2T
- Trying to start a video after the card full failure message results in: https://imgur.com/BQJB1GO

- Eventually, after a few restarts, I was able to shoot a video for around 10 seconds, which then auto-stopped. Trying to take another video after resulted in the same 'CARD FULL' message.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on October 08, 2020, 05:45:16 PM
Thanks. Confirmed not working. No further tests needed.
Title: Re: UHS-I / SD cards investigation
Post by: tmilardo on October 10, 2020, 06:11:27 AM
Quote from: ZEEK on October 07, 2020, 12:43:50 PM*Note, I could also record 2.8K RAW on the EOS M @2.35:1 with green signal for over a minute.
https://drive.google.com/file/d/1SQQzYvFAt1p0heYqJu_JS3AGb7l9Qz5u/view?usp=sharing

Wow, great to hear that recording with 240MHz is possible. Shame it requires such an expensive card. I watch your videos Zeek, I see that you have multiple EOSMs. Does this card work at 240MHz on multiple cameras?

Quote from: ZEEK on October 07, 2020, 12:56:35 PMWith the 192Mhz patch I get about 5-12 Seconds, with the 240MHz patch, 1 Minute & above. Not sure about the consequences of using the highest patch and how taxing it can be on the card in long term use. Don't think it should be too bad.

These cards do have a lifetime warranty (https://kb.sandisk.com/app/warranty/a_id/22478), don't be shy about frying your card for science :D
Title: Re: UHS-I / SD cards investigation
Post by: ZEEK on October 10, 2020, 09:29:50 AM
Quote from: tmilardo on October 10, 2020, 06:11:27 AM
Wow, great to hear that recording with 240MHz is possible. Shame it requires such an expensive card. Does this card work at 240MHz on multiple cameras?
Yes indeed! I'm no rich guy, but after being an EOS M user for a few yeas now, I knew it was finally worth the jump ;) Yes, it works on all my EOS M Cams.

Quote from: tmilardo on October 10, 2020, 06:11:27 AM
These cards do have a lifetime warranty (https://kb.sandisk.com/app/warranty/a_id/22478), don't be shy about frying your card for science :D
Haha, better not lose the receipt ;)
Title: Re: UHS-I / SD cards investigation
Post by: smith Os on December 20, 2020, 02:09:50 AM
Just bought a 700d for an extremely good price so I instantly set out to try and get raw video out of it. And I also just made my own account on this wonderful forum... I did so because I am having a pretty hard time getting the "sd_uhs" module to work on my t5i/700d. I tried everything, from installing other firmwares, to just plain formatting and starting fresh. I've been trying for about 6 hours just today just scouring various forums and different youtube videos, so I finally decided to cut the snake off at the head and just make an account.

If someone could give me the latest firmware for the 700d known to work with "sd_uhs.md" with download links, then that would be greatly appreciated!!

Best regards,
-Eli
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on December 20, 2020, 02:55:05 AM
https://www.magiclantern.fm/forum/index.php?topic=5951.msg231434#msg231434
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on January 11, 2021, 10:16:15 PM
An EOS M user tested Sandisk Extreme PRO 512 GB 170 MB/s with 240 MHz overclock, the 240 MHz overclock didn't work on that Card, benchmarks showed 20 MB/s

Some cards are lucky, like as ZEEK's Sandisk Extreme PRO 512 GB 170 MB/s card, unfortunately buying an Sandisk Extreme PRO 170 MB/s with 512 GB capacity isn't a guarantee to make 240MHz preset work on EOS M.

A link to his post on "MLV RAW ON EOS M" Facebook group:
https://www.facebook.com/groups/375018026655820/permalink/868207530670198/?comment_id=868301957327422&reply_comment_id=868493133974971

Title: Re: UHS-I / SD cards investigation
Post by: Danne on January 11, 2021, 10:22:31 PM
Yes, and after some more testing ZEEK also had some not so stable recordings with his 512gb card. He did some tests in a pm I sent.
I noticed also I am getting a few frames occasionally with 16:9 preset with full real time preview anamorphic. It only needs a little reducing of height to get fixed so I´ll test some more when I get the chance.
Title: Re: UHS-I / SD cards investigation
Post by: Audionut on January 28, 2021, 03:14:22 AM
I'm not entirely sure of what the status of this project is at currently. I don't have time to read 27 pages sorry.

I've moved this thread into the new section. I would appreciate if someone wants to take a leadership role with the creation of fancy OP, with updated build links, a short summary, that sort of thing.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on February 13, 2021, 02:58:12 PM
Managed to call "sdSoftReset":

static void (*SD_SoftReset)() = 0xFF74AE78;     /* 700D.115 */
    SD_SoftReset();


LOG:
3.405.991    run_test:ff74aea0:23:01: sdSoftReset( 0 )
..
3.406.540    run_test:ff74af14:23:01: sdSoftReset SUCCESS
..
4.154.844   menu_task:ff74aea0:23:01: sdSoftReset( 0 )
..
4.155.645   menu_task:ff74af14:23:01: sdSoftReset SUCCESS
4.155.656   menu_task:ff74af30:23:01: sdTrySendCommand1 Start
..
4.157.496   menu_task:ff74af78:23:03: sdTrySendCommand1: Timeout
..
etc


Write/Read speed after that ~21 MB/s
Title: Re: UHS-I / SD cards investigation
Post by: Danne on February 13, 2021, 03:59:40 PM
Hehe, let the hazard begin  8).  Great stuff.
Title: Re: UHS-I / SD cards investigation
Post by: dinobike on February 15, 2021, 03:28:25 PM
I've been a little out of touch with new advancements in ML, so i tried the latest uhs.mo improvements and was surprised with my "old" Sandisk extreme Pro 300mbs uhs-II that hits 80mb/s write speed @192mhz. With the previous version of uhs.mo the card was almost useless for raw video recording, but now im just so happy and still trying to figure out the new crop and binning records.
For now i'll be stuck with "normal" resolutions 1736x976, that looks incredible and gave new life to my 700D. :)

The benchmark was in photo mode.

(https://i.ibb.co/CVgLRkR/memoria-sandisk-32gb-extreme-pro-sdhc-uhs-ii.jpg)

(https://i.ibb.co/cgshmSL/BENCH6.jpg)
Title: Re: UHS-I / SD cards investigation
Post by: wmtr on July 03, 2021, 05:24:15 PM
New to the forum, here to post some findings. Just got my first Eos M and I have some sd/micro sd cards laying around, before I buy a new one I thought I would test them out first.

Samsung EVO select microsd 128GB,   160Mhz gives best results @ 54.5MB/s, which is enough for 1080p at 14bit, which is the only mode I need, this card runs about $20 on amazon.
Samsung EVO select microsd 256GB,   192Mhz gives best results @ 56MB/s, which is enough for 1080p at 14bit, this card currently sells at $35 on amazon, I got it when it was on sale at $25.
sandisk extreme Pro 95MB/s 32Gb,      192Mhz gives best results @ 44MB/s, just about barely enough for 1080p at 14bit.
SanDisk Ultra 64GB MicroSDXC, this card is a no no, it barely goes above 20MB/s, so not enough for magic lantern.
Patriot LX Series 64GB, slightly better than Sandisk Ultra, goes to about 28MB/s, still not enough for magic lantern.

So shooting at 1080p 16:9 at 14bit, you are gonna need at least 41 MB/S writing speed. A 64GB card is gonna give you around 20minutes of footage, 128GB 40min, 256GB 80min and so on so forth. Based on my personal experience, a 8-10 hours shooting day(story based short film), I usually get about 60-75 mins of actual footage on camera. So for me, 256GB is the minimum.

So if you are like me looking for a cheaper alternatives to the much praised but more expensive SanDisk Extreme PRO, samsung evo select is a pretty good contender, they are not as fast as Sandisk extreme pro, but  good enough for what I need.


Currently I'm waiting for a 256GB Samsung pro plus full size SD card to be delivered,  I got it when the price is marked down to $30 for 256GB size, not sure it's temporary or permanent. It's supposedly designed for 'professional' use so I expect it to be more durable and writes fast than the consumer grade EVO select. I will report back when I receive it.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 10, 2021, 05:51:26 PM
5D3:

192MHz and 240 MHz presets do work but it has same problem as some other DIGIC 5 cameras, it either switch to safe mode which is SDR50 @ 48 MHz, or the camera make the SD card inaccessible resulting in "Card full" issue when recording RAW video, or camera lock up or crash when trying to capture Still photo or record H.264 video with error message until you restart the camera by pulling the battery out

And these happens when camera try to make a read process to SD card (This is my guess, I talked about this earlier in this thread), as long as there is no reading happening, the presets just work:

5D3 113 @ 240 MHz in PLAY mode (Sandisk Extreme PRO UHS-I U3 170 MB/s):
(https://i.ibb.co/3dLf4QT/5-D3-240-MHz.png)

Of course currently these presets are unpractical right now, How I made it work:

1- First you need an empty SD card, only ML files on it (Sometimes it works with some MLV files)
2- You need to apply the patch after the camera fully startup (after red led become off), applying the patch during startup won't work unless you are very lucky (because -I think- during camera startup there are reading processes happening like loading ML files into memory, or Canon firmware checking SD card content . . etc)

(What about trying these steps on EOS M, 240 MHz might work, but won't be practical, just for testing)

Well, when 240 MHz preset works you get:
(https://i.ibb.co/ctBnzMT/5-D3-240-MHz.jpg)

~81 MB write speed in video mode :D

Who needs CF card? :P

-I didn't test it with CF card yet (with Card spanning), waiting the CF to arrive . .



You won't get anything from this post, further research is required . . but it's cool to showcase how 240 MHz preset would perform when it works.
Title: Re: UHS-I / SD cards investigation
Post by: Grognard on July 11, 2021, 09:57:29 PM
Quote from: theBilalFakhouri on July 10, 2021, 05:51:26 PM


-I didn't test it with CF card yet (with Card spanning), waiting the CF to arrive . .



You won't get anything from this post, further research is required . . but it's cool to showcase how 240 MHz preset would perform when it works.

Great. Wait and see.
Does it mean you finally got a 5diii?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 14, 2021, 02:10:46 AM
It seems CF + SD card spanning maxes at ~150 MB/s (in PLAY mode) using all presets 160 MHz, 192 MHz and 240 MHz, so I think we hit memory limit here (is there a workaround for this?).

I couldn't test 192 MHz and 240 MHz in LiveView with card spanning I expect to be same as 160 MHz with card spanning which is maxes at ~135 MB/s in LiveView when recording RAW (uncompressed RAW, 23.976 FPS)
-Lossless compression make a bottleneck BTW due to extra processing/memory cycle, which make write speed with card spanning below 135 MB/s .



Quote from: Grognard on July 11, 2021, 09:57:29 PM
Does it mean you finally got a 5diii?

Yes, thanks to "unknown soldiers" who gave me the opportunity to get a 5D3 and work on it :) .
Title: Re: UHS-I / SD cards investigation
Post by: Jonneh on July 15, 2021, 04:59:40 PM
Quote from: theBilalFakhouri on July 14, 2021, 02:10:46 AMYes, thanks to "unknown soldiers" who gave me the opportunity to get a 5D3 and work on it :) .

Just when I was thinking it was about time we pooled some resources to buy one, the soldier stepped forward. Good on him or her. Enjoy it! :-)

Regarding the stability issues with the faster overclocking presets, the explanation you propose is interesting. It would be a boon if there turns out to be a workaround, but only if the data rate bottleneck can be overcome, naturally.

Regarding this apparent 135/150 MB/s bottleneck (I'd noticed the 135 MB/s limit before when the total rate with a fast CF + SD didn't benefit from SD overclocking in terms of total data rate---doing so does give more equal write ratios, of course, which is good in itself, since the CF and SD cards have the same capacity. Good to know that it's due to the lossless compression overhead.), I suppose working around that might be a big ask, but I confess to major ignorance on the matter.

On this topic, I'm slightly confused about what you mean by there being a 150MB/s bottleneck in PLAY mode. Are you referring to a read bottleneck, or rather a write bottleneck with LiveView off, or perhaps something else?

Great stuff all around, either way!
Title: Re: UHS-I / SD cards investigation
Post by: mlrocks on July 28, 2021, 08:28:38 AM
I just benchmarked my sd cards with different capacity in 650D. All sd cards are 170MB/s Sandisk Extreme Pro. There are two 64 GB, one 128 GB, one 256 GB. All were bought within a year. All of them were overclocked to 240 hz in the 650 D July 24, 2021 ML version.
The two 64 GB and one 128 GB cards benchmarked for one minute, and got the write speed of 60 MB/s, and the read speed of 70 MB/s. The 256 GB card got the write speed of 45 MB/s, and the read speed of 45 MB/s. Yet the 256 GB card could continuously record at 55-60 MB/s. When it was over 60 MB/s, then it could only record several seconds. Not sure if it is the card issue, or the benchmark module is not tuned for larger size cards like 256 GB or 512 GB.

Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 28, 2021, 09:09:18 AM
Benchmark doesn't care about card size. It won't even check for sufficient free space.
Your numbers for all your cards seems to be a bit low. Try a run in photo mode. And just for fun: What are the numbers you get benchmarking on your PC? Esp. for the 256 GB variety.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 28, 2021, 12:11:17 PM
Also try to format the cards in camera (Low level format) before running benchmarks to get constant write/read speed.
Title: Re: UHS-I / SD cards investigation
Post by: mlrocks on July 28, 2021, 05:25:00 PM
Quote from: Walter Schulz on July 28, 2021, 09:09:18 AM
Benchmark doesn't care about card size. It won't even check for sufficient free space.
Your numbers for all your cards seems to be a bit low. Try a run in photo mode. And just for fun: What are the numbers you get benchmarking on your PC? Esp. for the 256 GB variety.

For the 256 gb card, I did overclocking to 240 hz, then formatted it at low level using the Canon Menu, then benchmarked using the 1 minute test in photo and video mode. In photo mode, I got read about 40 MB/s, write 36 MB/s. In video mode, for several times I consistently got 45 MB/s for read and write. Turning off modules like LUA EDMAC etc did not help.
On computer, using Sandisk card reader USB 3.1 (computer USB adapter may be just USB 2.0 or USB 3.0, but not USB 3.1.), the read speed was 160 MB/s consistently, the write speed was 98 MB/s consistently.
I can record continuously as long as the data flow was about 55 MB/s. Over 60 MB/s, it will record several seconds, then stop.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on July 28, 2021, 06:02:12 PM
Have you restarted the camera after selecting 240 Mhz preset?
Title: Re: UHS-I / SD cards investigation
Post by: mlrocks on July 28, 2021, 09:30:49 PM
Quote from: theBilalFakhouri on July 28, 2021, 06:02:12 PM
Have you restarted the camera after selecting 240 Mhz preset?

Yes, I first chose the 240 Mhz preset, then restarted the camera. Then I reformatted the card in camera. The ML still worked after the low level in camera format.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 28, 2021, 09:47:16 PM
These are different things:
- ML will save its own data during format and write back. Has nothing to do with overclocking.

The numbers you get by benchmarking in cam are pretty low. You should get around the same numbers (or better) without overclocking.
Title: Re: UHS-I / SD cards investigation
Post by: mlrocks on July 28, 2021, 09:57:13 PM
I just found out the root cause of this issue. I was in 650D crop new 2.35 1x3 1600x1800 mode, testing all these benchmark. Now I am in crop UHD 1x3 1280x2160 mode, benchmark showed 66 MB/s writing speed. So the benchmark is actually preset dependent.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on July 28, 2021, 10:34:26 PM
Turn off raw video and do the test in play mode.
Title: Re: UHS-I / SD cards investigation
Post by: mlrocks on July 29, 2021, 12:40:52 AM
Quote from: Danne on July 28, 2021, 10:34:26 PM
Turn off raw video and do the test in play mode.

Turn off raw video and in photo mode (play mode), mode: Play-Unk LCD, global draw: on, benchmark 1 minute test showed:
93 MB/s write
105 MB/s read

Turn off raw video and in photo mode (play mode), mode: Play-Unk LCD, global draw: off, benchmark 1 minute test showed:
94 MB/s write
106 MB/s read
Title: Re: UHS-I / SD cards investigation
Post by: mlrocks on July 29, 2021, 04:46:26 AM
I just did a stress test checking how the overheating of the SD card overclocking at 240Mhz behaves on 650D.

In the old but reliable mode of movie crop, 10-bit color depth, 14-bit lossless compression, 1800x1012, AR 16:9, fps 29.97, dual ISO and auto ETTR on, global draw on, the 650D recorded with a green dot most of the time, the left time was orange, only at the beginning several seconds it was red.

At the ambient temperature of 20 C, the camera temperature went up to 50 C in the first 30 minutes, then maintained at 50 C for the rest of the time. With a 256 gb card, the recording time was continuous for 69 minutes totally. I changed the batteries and added another 64 gb card, to see how far the little 650D can go before shutting down due to overheating. After recording for about 5 minutes, the recording stopped automatically. I started it again, the temperature was still at 48 C. Another stop happened after 3 minutes' recording. Another stop happened after 2 minutes' recording. I started it again, then the camera recorded until this 64 gb card was full. The camera temperature was at 48 C. I changed to another 64 gb card, and hit the record button again. It was a 20-minutes continuous recording until the card was full. The camera temperature was still at 48 C.  In total, after going through three sd cards with a total capacity of 384 gb and about 100 minutes recording time in a continuous raw recording mode, the 650D temperature was maintained at 50 C. With live view on all the time, the camera took 2 hours to go back from 50 C to 42 C. It may take several hours more to go back to below 30 C.

It seems that the camera has an embedded cooling system within kicks in when the camera temperature reaches 50 C. Heat may store somewhere in the camera body relative safely and can be released gradually after the camera is turned off. This kind of temperature control is good enough for an 8-hour full day shooting when sd card overclocking is turned on.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 29, 2021, 09:34:32 AM
"Embedded cooling system" is a bit misleading. It's just a temperature threshold and cam turns off if overstepped. No timer involved as in EOS R series, for example.
Heat cannot be "stored" inside a camera. Heat goes from "hot" to "cold" by radiation, conduction and convection until equilibrium is established.
Title: Re: UHS-I / SD cards investigation
Post by: mlrocks on July 29, 2021, 03:45:26 PM
Quote from: Walter Schulz on July 29, 2021, 09:34:32 AM
"Embedded cooling system" is a bit misleading. It's just a temperature threshold and cam turns off if overstepped. No timer involved as in EOS R series, for example.
Heat cannot be "stored" inside a camera. Heat goes from "hot" to "cold" by radiation, conduction and convection until equilibrium is established.

The camera maintained at 50 C for about 1 hour recording raw without shutting down. If there is no way to transfer the heat from the cpu, like fans in a desktop computer, the camera should have shut down several minutes after it reached 50 C. Or because the ambient temperature was 20 C? Probably someone can do a similar stress test under the noon sun to check how long the camera can record when the ambient temperature is high, like 35-40 C.
Title: Re: UHS-I / SD cards investigation
Post by: ainu on August 25, 2021, 08:22:47 PM
@theBilalFakhouri

Can you please give me a link to the sh_uhs module with the ability to enter values in the menu to accelerate the recording speed?

I have a Canon 70D camera. For the Kingston 64GB U3 card (label 90mb/s read, 80 mb/s write), only the 160 MHz preset works in your automatic module, and the 192 and 240 presets do not work.
However, when using the version of the module from Levas (https://www.magiclantern.fm/forum/index.php?topic=12862.msg228576#msg228576), the benchmark shows a recording speed of 80-85 mb/s and I can record raw video with a stream of about 75 mb/s in video mode. Unfortunately, this module does not work stably - the ability to write with a stream of 70-75 mb/s can turn into 50-30-20 mb/s.
Therefore, before buying sd card sandisk extreme pro 256gb 170mb/s (ArcziPL confirmed the operability of the preset 240mgz on 70d - https://www.magiclantern.fm/forum/index.php?topic=25841.msg234506#msg234506), I decided to try to select parameters for my existing card.

Thank you in advance.

@all
Or may be anyone else have mosldule and can give me link to it please?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on August 26, 2021, 08:10:53 AM
Hi @ainu

Quote from: ainu on August 25, 2021, 08:22:47 PM
Can you please give me a link to the sh_uhs module with the ability to enter values in the menu to accelerate the recording speed?

Do you mean this? :
https://www.magiclantern.fm/forum/index.php?topic=25841.msg234228#msg234228

Or this:
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228367#msg228367 ?

(I don't recommend to play with it)

Levas's preset in this link (https://www.magiclantern.fm/forum/index.php?topic=12862.msg228576#msg228576) is the same as in this sd_uhs version (https://www.magiclantern.fm/forum/index.php?topic=25841.msg234228#msg234228), it's 240 MHz preset.

Not much people tested "Kingston 64GB U3" cards with SD overclocking . . some cards won't accept the overclock for some reason, but I can see the 240 MHz preset worked using your Kingston U3 card, but it's seems not stable.

Have you tried "Low level format" in camera? this might solve the drop speed issue, if it sill switch to 20 MB/s then currently 192/240 MHz presets aren't practical/stable for 70D, even with Sandisk Extreme PRO U3 UHS-I 170MB/s cards (https://www.magiclantern.fm/forum/index.php?topic=25841.msg235469#msg235469).



I can see in Levas's sd_uhs.c he isn't using "SDR-104" hack, I will update my version (https://www.magiclantern.fm/forum/index.php?topic=25841.msg234228#msg234228) with ability to turn it off for testing purposes, it might be the reason why my sd_uhs version didn't work on your Kingston card at 192/240 MHz.
Title: Re: UHS-I / SD cards investigation
Post by: ainu on August 26, 2021, 10:40:32 PM
Quote from: theBilalFakhouri on August 26, 2021, 08:10:53 AM
Have you tried "Low level format" in camera? this might solve the drop speed issue.

@theBilalFakhouri
About card format: yes, I tried much more.
I format (do it at PC) card to FAT32 with cluster size start of 32KB to 128KB, exFat with cluster size start of 4KB to 16MB and on each combination of file system and cluster size (include low level format in camera with sd_uhs on at 160Mhz and off) run benchmark on stable preset 160Mhz ("old" sd_uhs module with SDR104 hack enabled "on").
So, at all combinations:
- in Play-UNC LCD mode write speed about 63-67mb/s, read speed mostly is 71.4mb/s
- in MV-1080 mode write speed about 56-59mb/s, read speed ~59mb/s.
To sum it all: file system and cluster size does not matter.

About test "new" sd_uhs with SDR104 on/off switch: on 160 MHz no difference between "on" and "off", 192 Mhz both modes (on and off) still not work for my SD card (write/read speed 20mb/s), 240Mhz in both modes write speed drop (first write speed ~85mb/s, then read/write/read speeds is ~20Mb/s.
At 240Mhz in video mode some lucky times I shoot raw video at ~60-65mb/s continues. In times when I can shoot raw video continues if I switch to photo mode and again to video mode speed drops to 20-8-2mb/s.
To sum it all: at this moment with current presets 240Mhz for Kingston sd card is working "particular". May be some sd cards need other presets.

Therefore I want to try this module https://www.magiclantern.fm/forum/index.php?topic=12862.msg228367#msg228367

In near future I plan to buy Sandisk Extreme Pro 256GB 170mb/s read speed. When it happens - I will test it with SDR104 hack off mode for understanding does it matter or not.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on August 27, 2021, 07:51:05 PM
@ainu

It seems Kingston cards doesn't provide stable write/read speeds, in this area Sandisk Extreme PRO UHS-I U3 170MB/s is better, it always have constant write/read speeds without dropping in 160 MHz preset (also for 192/240 MHz, tested on 650D/700D/6D) after low level formatting the card in camera.

Quote from: ainu on August 26, 2021, 10:40:32 PM
To sum it all: file system and cluster size does not matter.
I remember testing these, yes it's true it didn't have any effect on write speed or stability.

Okay using 192/240 MHz on 70D isn't practical currently, 5D3 share the same story, 160 MHz is fine on 5D3 but 192/240 MHz you can record some clips then instead of switching to 21 MB/s mode it disable the access to card completely.

EOS M also suffering from switching to 21 MB/s (48 MHz) when using 240 MHz overclock, but 192 MHz preset on EOS M works without any problem.

On 700D/650D/6D 192/240 MHz presets works fine too without any problem (Sandisk Extreme PRO UHS-I U3 170MB/s).

I spent much time yesterday to investigate to see what's difference among 5D3-700D, nothing important yet.

Quote from: ainu on August 26, 2021, 10:40:32 PM
Therefore I want to try this module https://www.magiclantern.fm/forum/index.php?topic=12862.msg228367#msg228367

Link was sent via PM, and good luck ;D
Title: Re: UHS-I / SD cards investigation
Post by: ainu on August 29, 2021, 05:35:24 PM
Thank you theBilalFakhouri.

I made a small selection of values (table here (https://docs.google.com/spreadsheets/d/1cxg5vYyX9NXudR8ca6exKbEjXbCoXM6b/edit?usp=sharing&ouid=104998444646372989223&rtpof=true&sd=true)).
Luck with the current sd card NOT smiled.
However, there was some understanding of how to set values. But the found preset from Levas (which works in Play mode always, and in Video mode times from time to time) does not fit into this understanding.
It turns out that in Play mode, the card and camera can record 80mb\s and read 105mb\s stably, and in video mode, the recording speed is kept only until the first reading test.
Perhaps the problem is not the preset - the future will show.

Edit.
Another note: after entering settings in the menu, during their application, the diode near the cover sd card lights up twice. By the duration of the second flash (if it is longer than ~ 2 seconds) you can understand the settings applied or the camera returned to the "48 MHz - 20.4mb\s" mode. Thus, you can not start benchmark (with a long flash, it always shows 20.4mb\s) and reduce the search time.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on August 30, 2021, 01:41:22 AM
@ainu

Quote from: ainu on August 29, 2021, 05:35:24 PM
Perhaps the problem is not the preset - the future will show.

240 MHz works fine on 650D/700D/6D, I have been using it for a year, never switched to 48 MHz mode using Sandisk Extreme PRO UHS-I U3 170 MB/s cards, never had data corruption with it.

-My my expectations are:

1. Canon has a function which traces card controllers speed clocks/or something makes Canon think something isn't usual is happening regarding how card is being configured which doesn't match what Canon code is expecting --> Switches to safe mode (48 MHz).

We already know there is something like that, but this was a different case:
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228485#msg228485
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228510#msg228510

I don't think it's related to 70D issue, what was happening there is my *Sandisk Extreme PRO UHS-I U3 95 MB/s was giving up (because the card had 99 MB/s write speed with 240 MHz overclock, probably it couldn't handle this really high write speed) or **SD card reader in camera was overheating/signal isn't clean. (My Sandisk Extreme PRO UHS-I U3 170 MB/s cards have fixed write speed of ~90 MB/s, which is probably made the signal cleaner/normal).

*Also my Sandisk Extreme PRO UHS-I U3 95 MB/s have 45 MB/s read/write speed using the card reader on PC, while the 170 MB/s version was giving ~85 MB/s write speed with same card reader (https://www.transcend-info.com/Products/No-396), newer cards from Sandisk is more fine tuned I guess.
**I put the camera in fridge with 95 MB/s Sandisk inside the camera (240 MHz) as @names_are_hard said it might be overheating issue, and made some tests, the card never switched to 48 MHz mode but it had speed drops, and a recording was stopping suddenly.

2. A setting needs to be changed on EOS M/70D/5D3 to accept these high clocks speeds, little experiment on 700D:

-I called sdSoftReset (which switches the card to 48 MHz mode), write speed becomes 21 MB/s (from ~40 MB/s)
-I changed registers values to 96 MHz preset (96 MHz preset is the default preset on 700D and all other DIGIC 5 cameras except for 5D3), write speed became 36 MB/s which a little lower than default 96 MHz mode
-It turned out AccessMode was changed from 7 (default) to 3 (after sdSoftReset), I re-patched AccessMode to 7 and got ~40 MB/s write speed which is the correct one

-So now we are in default 96 MHz mode after a sdSoftReset, let's overclock the values again to 160MHz/192MHz/240 MHz --> Card full; card can't be accessed (another safe mode but safer from Canon)
-That's why I believe there is a setting we are missing, finding it out might solve the problem for 5D3 and probably for EOS M/70D

3. Third expectation: it might just the wrong 240 MHz values :-\
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on August 30, 2021, 03:44:49 AM
Quote from: theBilalFakhouri on August 30, 2021, 01:41:22 AM
*Also my Sandisk Extreme PRO UHS-I U3 95 MB/s have 45 MB/s read/write speed using the card reader on PC, while the 170 MB/s version was giving ~85 MB/s write speed with same card reader (https://www.transcend-info.com/Products/No-396), newer cards from Sandisk is more fine tuned I guess.

Unable to reproduce because I don't own this cardreader. Own tests with Kingston FCR-HS3 and FCR-HS4 and all cardreader tested by CameraMemorySpeed.com (https://www.cameramemoryspeed.com/reviews/sd-cards/sandisk-extreme-pro-95mbs-uhs-i-u3-128gb-sdxc-card/) showed write rates > 80 MByte/s.
I think you have either a counterfeit card or a cardwriter issue.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on August 30, 2021, 09:02:05 AM
Quote from: Walter Schulz on August 30, 2021, 03:44:49 AM
Unable to reproduce because I don't own this cardreader. Own tests with Kingston FCR-HS3 and FCR-HS4 and all cardreader tested by CameraMemorySpeed.com (https://www.cameramemoryspeed.com/reviews/sd-cards/sandisk-extreme-pro-95mbs-uhs-i-u3-128gb-sdxc-card/) showed write rates > 80 MByte/s.
I think you have either a counterfeit card or a cardwriter issue.

I have tested my Sandisk Extreme PRO UHS-I U3 95 MB/s card on PC using three different card readers (RDF5 -RDF8 - Laptop card reader) on Windows, it gave same speed around ~45 MB/s, however on camera:
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228555#msg228555

That's was the (Sandisk Extreme PRO UHS-I U3 95 MB/s)

Pretty sure it's not fake, fake cards write speeds maxes out at 20-25 MB/s, my card works greatly with 192 MHz preset.

Also my friend got same SD card version from same local store nearly the same time, his card appear to maxes out @ 55 MB/s on PC, on camera with the overclock it gives the full speed.
I think only some of Sandisk Extreme PRO UHS-I U3 95 MB/s cards have this issue, probably I will get the maximum write/read speed on PC using a Sandisk card reader or on another OS.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on August 30, 2021, 09:51:09 AM
Quote from: theBilalFakhouri on August 30, 2021, 09:02:05 AM

Pretty sure it's not fake, fake cards write speeds maxes out at 20-25 MB/s

Source?

Quote from: theBilalFakhouri on August 30, 2021, 09:02:05 AM
Also my friend got same SD card version from same local store nearly the same time, his card appear to maxes out @ 55 MB/s on PC, on camera with the overclock it gives the full speed.

Which proves what?

Quote from: theBilalFakhouri on August 30, 2021, 09:02:05 AM
I think only some of Sandisk Extreme PRO UHS-I U3 95 MB/s cards have this issue, probably I will get the maximum write/read speed on PC using a Sandisk card reader or on another OS.

Source?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on August 30, 2021, 02:14:47 PM
Quote from: Walter Schulz on August 30, 2021, 09:51:09 AM
Source?

I had three fake Sandisk Extreme PRO cards (https://www.magiclantern.fm/forum/index.php?topic=24315.0) before, all of them had ~20-25 MB/s write speed (or a bit more), source is coming from my own tests :D
Also the fake cards didn't accept the overclock.

I am not sure if there are fake Sandisk Extreme PRO cards which have same performance as authentic ones (at least on camera). If this is the situation here, that's would be another story . .

Quote from: Walter Schulz on August 30, 2021, 09:51:09 AM
Which proves what?

The local seller have imported many affected cards in the same time, probably Sandisk had bunch of these affected cards at that time.

Quote from: Walter Schulz on August 30, 2021, 09:51:09 AM
Source?

Idea from a mind, I am guessing.
Title: Re: UHS-I / SD cards investigation
Post by: z56667895 on September 02, 2021, 07:45:04 PM
Any setting errors cause slow writing speed?

My RAW recording can only record about 2 seconds in any mode

I don't know what caused it
My card is Sandisk Extreme PRO 170MB/s 128G

No problem testing the memory card on the computer

(https://imgur.com/I24YlVt)
(https://imgur.com/pNa4VuS)
(https://imgur.com/undefined)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 04, 2021, 05:16:27 PM
This card appears to accept 240 MHz overclock (tested on 700D by a user interacted with me on FB)

(https://i.ibb.co/SBPwtm0/Sandisk-Ultra-120-MB-s.jpg)

-No overclock:
Write 35.2 MB/s
Read 43 MB/s

-160 MHz:
Write 52 MB/s
Read 74 MB/s

-192 MHz:
Write 58 MB/s
Read ~84 MB/s

-240 MHz:
Write ~62 MB/s
Read 106 MB/s

-All previous tests was in play mode, we didn't test write speed using card reader on PC.
-We recorded many clips, it didn't switch to 48 MHz, seems like a stable card with 240 MHz overclock.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on September 04, 2021, 06:04:37 PM
There is a bar code on the rear of the blister pack and a string like "SDSQUSC-128G-ANCMA" above. This is the only identifier giving valid information about type and revision. Are you able to retrieve this string? Would be helpful to get a grip on working cards.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 04, 2021, 10:37:43 PM
Quote from: theBilalFakhouri on September 04, 2021, 05:16:27 PM
This card appears to accept 240 MHz overclock (tested on 700D by a user interacted with me on FB)

(https://i.ibb.co/SBPwtm0/Sandisk-Ultra-120-MB-s.jpg)

-No overclock:
Write 35.2 MB/s
Read 43 MB/s

-160 MHz:
Write 52 MB/s
Read 74 MB/s

-192 MHz:
Write 58 MB/s
Read ~84 MB/s

-240 MHz:
Write ~62 MB/s
Read 106 MB/s

-All previous tests was in play mode, we didn't test write speed using card reader on PC.
-We recorded many clips, it didn't switch to 48 MHz, seems like a stable card with 240 MHz overclock.

Behaviour on 5d3?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 05, 2021, 11:18:16 AM
Quote from: Walter Schulz on September 04, 2021, 06:04:37 PM
There is a bar code on the rear of the blister pack and a string like "SDSQUSC-128G-ANCMA" above. This is the only identifier giving valid information about type and revision. Are you able to retrieve this string? Would be helpful to get a grip on working cards.

That was:
SDSDUN4-128G-GN6IN
(https://www.amazon.com/SanDisk-128GB-Ultra-UHS-I-Memory/dp/B08GY61ZZN)
Yeah, it's better to list these cards in a spreadsheet in Google Drive or ML Wiki

Quote from: Danne on September 04, 2021, 10:37:43 PM
Behaviour on 5d3?

That card was tested by a user with his 700D, not me, he doesn't have 5D3
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 17, 2021, 07:32:58 AM
I did some experiments lately with SD overlocking stuff,

After calling sdSoftReset the camera switch from 96 MHz to 48 MHz mode, and AccessMode from 7 to 3, resulting in 21 MB/s write speed, changing preset to 96 MHz makes the write speed ~36 MB/s instead of ~41 MB/s , changing AccessMode from 3 back to 7 makes the write speed ~41 MB/s (now we got back original write speed), from there I could overclock to 160 MHz without a problem, but doing an overclock to 192 MHz/240 MHz makes the camera switches back to 48 MHz (21 MB/s)

After some research I found something (probably related to Bus speed configuration) which changes its value from 0 to 1 after doing sdSoftReset and especially after hitting recording button (after doing write/read processes), I tracked what part of code does that in 700D ROM, and found out it's coming from sdReadBlk and sdWriteBlk, well let's force that "something" value from 1 to 0 using this code (for 700D):

patch_instruction(0xff74c814, 0x13a01001, 0x13a01000, "0xff74c814"); // sdWriteBlk
patch_instruction(0xff74cae0, 0x13a01001, 0x13a01000, "0xff74cae0"); //sdReadBlk


-->I could overclock to 192/240 MHz presets (after a sdSoftReset/after switching to 48 MHz) :D (didn't blow up the card or camera . . managed to record several clips and +50 GB of data, everything was stable)

I am not really sure what that "something" does, do you remember these initial 192/240 MHz presets (which were and still not stable):
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228447#msg228447
https://www.magiclantern.fm/forum/index.php?topic=12862.msg228460#msg228460

Above patch forced these none-stable presets to work, and it didn't switch to 48 MHz . . but I experienced speed drops then --> Card can't be accessed a.k.a "Card full" issue, well we knew those presets weren't stable and we are not using them, but we know current 192/240 MHz presets (https://www.magiclantern.fm/forum/index.php?topic=12862.msg228555#msg228555) are stable (on 700D/6D)

--> So from above experiment and results it might be there some Hope to force 240 MHz on EOS M, and 192/240 MHz on 70D, please try the following sd_uhs.mo (for 70D/EOS M):

Download: https://drive.google.com/file/d/1DGa2nhSdgu8tSY2Yy22tZQR3PmYpwB4i/view?usp=sharing

If it worked --> you will have stable 192/240 MHz overclock
If it didn't work --> you might experience speed drops while recording and emergency stops, then "Card full" issue might appear (maybe something else?)

-Do this test at your own risk
-If it worked, it's might not be the correct way to force these presets

Edit: Doesn't work well (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236755#msg236755), not stable at 240 MHz for EOS M, don't use it.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 17, 2021, 07:39:06 AM
Hehe, nice work man. Will test in the evening  :)
Title: Re: UHS-I / SD cards investigation
Post by: masc on September 17, 2021, 07:30:09 PM
I tried your module on my EOSM with SanDisk Extreme Pro 95MB/s 64GB:
240MHz: 21MB/s (R/W)
192MHz: 86MB/s (R) 79MB/s (W)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 17, 2021, 07:41:19 PM
@masc

Hmmm, could you share a picture for "Memory patches" from "Debug" tab
Title: Re: UHS-I / SD cards investigation
Post by: masc on September 17, 2021, 07:50:13 PM
Quote from: theBilalFakhouri on September 17, 2021, 07:41:19 PM
could you share a picture for "Memory patches" from "Debug" tab
This is booted with 240MHz setting.
(https://i.ibb.co/zZcx2BW/VRAM0.jpg)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 17, 2021, 08:03:46 PM
It seems the patch didn't apply, will check why
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 17, 2021, 08:08:14 PM
Okay, it was a typo, updated the link, please try this:
https://drive.google.com/file/d/1DGa2nhSdgu8tSY2Yy22tZQR3PmYpwB4i/view?usp=sharing
Title: Re: UHS-I / SD cards investigation
Post by: masc on September 17, 2021, 08:34:36 PM
I tried again. With 240MHz setting, I get a console-like output when starting the cam. Recording 12bit 5K frtp works without problem. The benchmark is... strange...:
W 96.1MB/s
R 889.6MB/s
W 134.7MB/s
R 4785.0MB/s
LOL... really? First try had even bigger values.  :P (While the benchmark screenshot is all black.)
Recording 14bit lossless 5K frtp 1440x1860 was working with green indicator all the time.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 17, 2021, 08:49:26 PM
Cool, it didn't switch . . but

These big values happens probably due to instability, could you try recording 1736x976 @ 23.976 FPS in 14-bit uncompressed, press Info to toggle to Canon Preview, try to record as long as possible, what is the reported write speed? is there an emergency stops? do you have "Card full" issue?
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 17, 2021, 08:53:36 PM
Also verify by checking the recorded files even after getting those wacky numbers. Shouldn't give you normal files but who knows what works with this hack :).
Title: Re: UHS-I / SD cards investigation
Post by: masc on September 17, 2021, 08:56:29 PM
Quote from: theBilalFakhouri on September 17, 2021, 08:49:26 PM
... could you try recording 1736x976 @ 23.976 FPS in 14-bit uncompressed, press Info to toggle to Canon Preview, try to record as long as possible, what is the reported write speed?
Sorry... stupid question...: how to record uncompressed? I use Danne's Feb build (I think it is not possbile here) - do I need the standard nightly for that? And where to find the "reported write speed"?
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 17, 2021, 09:04:44 PM
I think I turned off the uncompressed alternative in my version. But do recordings work in general with 240mhz patch?
Title: Re: UHS-I / SD cards investigation
Post by: masc on September 17, 2021, 09:13:44 PM
Quote from: Danne on September 17, 2021, 09:04:44 PM
I think I turned off the uncompressed alternative in my version. But do recordings work in general with 240mhz patch?
Yes, worked fine. I recorded 14bit lossless 1440x1836 for over a minute, indicator always green. File worked fine in MLVApp. Did not see any pink frame or something else...
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 17, 2021, 09:15:45 PM
Interesting. And it kept working even after you were getting those wacky test numbers?
Title: Re: UHS-I / SD cards investigation
Post by: masc on September 17, 2021, 09:18:59 PM
Quote from: Danne on September 17, 2021, 09:15:45 PM
Interesting. And it kept working even after you were getting those wacky test numbers?
Tried again for this: after the benchmark I get a "File create error".
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 17, 2021, 09:21:31 PM
I see. So maybe it´s the usual lucky first recording and then it breaks. Too bad I have no battery in my cam and charger at home.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 17, 2021, 09:30:14 PM
Quote from: masc on September 17, 2021, 08:56:29 PM
Sorry... stupid question...: how to record uncompressed? I use Danne's Feb build (I think it is not possbile here) - do I need the standard nightly for that? And where to find the "reported write speed"?

Not sure how to do it from Danne build, use the official crop_rec_4k build:
https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/80/artifact/platform/EOSM.202/magiclantern-crop_rec_4k.2018Jul22.EOSM202.zip

You can change bit-depth from mlv_lite submenu, maybe you need to use Movie crop mode (lower the resolution to 1736x976) because official build doesn't have mv1080 patch. I mean reported write speed by mlv_lite, like this:

(https://i.ibb.co/ctBnzMT/5-D3-240-MHz.jpg)

I want you test 1736x976 @ 23.976 FPS in 14-bit uncompressed because it has fixed write speed which is 67.8 MB/s , and recording should be continuous only with 240 MHz preset, look out for speed drops during recording . .
Title: Re: UHS-I / SD cards investigation
Post by: masc on September 17, 2021, 09:31:34 PM
Tested a little more: recording as long as possible with 5K frtp setting from above:
1st file stopped after 11.8GB, 4272frames, 2:58min. No error visible in file.
2nd file stopped ("card full", while 36GB are still left) after 11.3GB. I can't open the file, because of error "Invalid blocksize '0', corrupted file".

Edit: 11,8GB*1024/178s = 67,88MB/s.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 17, 2021, 09:41:27 PM
This version of mlv_lite.mo should work with 14 bit uncompressed in my build:
https://bitbucket.org/Dannephoto/switch_mini-git/downloads/mlv_lite.mo
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 17, 2021, 09:47:49 PM
Quote from: masc on September 17, 2021, 09:31:34 PM
Tested a little more: recording as long as possible with 5K frtp setting from above:
1st file stopped after 11.8GB, 4272frames, 2:58min. No error visible in file.
2nd file stopped ("card full", while 36GB are still left) after 11.3GB. I can't open the file, because of error "Invalid blocksize '0', corrupted file".

Okay, that's is good enough to know, thanks for testing!, bad news: don't use this patch :P

The patch seems to prevent SD card from switching to 48 MHz, but won't solve the real issue, the "instability", at least at this point we might know where to look to be able fix the instability issue



Here is the source code I used if someone needs it:
https://drive.google.com/file/d/1uvDeQeKemtfD_a9m4HOPU30DJYnI7vv2/view?usp=sharing

These lines are the patch for EOS M:
if (is_camera("EOSM", "2.0.2"))
{
patch_instruction(0xff642a98, 0x13a01001, 0x13a01000, "0xff642a98");
patch_instruction(0xff6427cc, 0x13a01001, 0x13a01000, "0xff6427cc");
}
Title: Re: UHS-I / SD cards investigation
Post by: ainu on September 17, 2021, 10:56:22 PM
Quote from: theBilalFakhouri on September 17, 2021, 09:47:49 PM
bad news: don't use this patch :P

I tested the module (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236743#msg236743) on two sd cards:
1. Kingston SDA3 64GB, on which previously only the 160 MHz preset worked steadily
2. Sandisk Extreme Pro 256GB (SDSDXXY-256G-GN4IN) all presets work well.

Results:
1. 192 and 240 MHz presets work stably for recording. The benchmark shows a recording speed of ~ 80 Mb\sec (240MHz) in play mode and ~ 71 Mb\sec in video mode, mlv files are recorded and played normally (start recording without run benckmark. If first run benckmark and then try record raw video then "card full" error).
At the same time, the read speed in the benchmark is ~ 850mb\sec (192) and ~ 1100 mb\sec (240). Everything was tested with no photos and videos on the card.
If you take a photo, then when you start the benchmark, a pause appears and the photo is not displayed correctly as the background of the benchmark. However, the write speed is the same 71\80 Mb\sec.
At the same time, the read speed AND write speed are displayed correctly on the 160 MHz preset.
2. Everything works with this card in the same way as before.

In light of this, I assume that some sd cards need other presets and value locks.
I do not believe that with the normal recording of files, an error in their reading (abnormal speed in the benchmark, inability to read a photo from the card) is the fault of the card itself.
Title: Re: UHS-I / SD cards investigation
Post by: ainu on September 19, 2021, 08:20:40 PM
I tried to pick up a preset, better than 240 MHz.
I found it:)
https://drive.google.com/file/d/13WHC6LE9u5OFMuLL1dSbbv4zZIICWo1z/view
register 0xC0400614 = 0x1d000201

Look at 620, I set it to 0x0 and its color changed to gray.
Without this change, it was impossible to obtain a stable reading speed (after the first write cycle in the benchmark, the speed of the first read cycle immediately fell).

Results on sd sandisk 256gb extreme pro (170mb\s) with this preset:
Play mode Write 97.4 Read 140 mb\s
https://drive.google.com/file/d/1IU_VMWSVvxCWgftdihUygrNn266wxWw7/view?usp=sharing
Video mode Write 82,2 Read 105,5 mb\s
https://drive.google.com/file/d/1kDcZVdm-Y5UW2qtTqRVdBiQmMnTUpYmg/view?usp=sharing

At the same time, I can record raw videos and files open normally.
Record indicator https://drive.google.com/file/d/13vY8fDGzSZOxx5Z2Mm9c_xiSdnl1ivyI/view?usp=sharing
Raw video settings https://drive.google.com/file/d/18t2nmtp4DPVM8c0COOiHOCXnaf0StwMt/view?usp=sharing
MLV lite screenshot https://drive.google.com/file/d/1cl0Ti0vUkjsC9-yoP4f0unDKOdh0gpMs/view?usp=sharing

Prior to this, 240 mhz speeds were in play mode W 95, R 105.5 mb\s and in video mode W 76 R ~ 85 mb\s.

With kingston sda3 64GB, this preset does not work, because in the current version of the module, where you can change settings, there is no blocking of values.
And as I found out earlier, for this sd card, blocking values ​ ​ is necessary (see the previous post).

Please check this preset on other cameras and sd cards where the 240 mfz preset works normally. You may see improved write and read speeds. If so, share the results on which camera and with which sd card it turned out.


@theBilalFakhouri, can you say when you set the value of the register to 0x0, does it really correspond to this value, or does the register simply remain unchanged and the patch does not apply to it? This is important for understanding what is really happening.

In the process of selecting register values ​ ​ for the preset, I noticed a number of laws that I will try to describe further. These patterns are only assumptions, but you can confirm or refute them, as well as share your experience in selecting values ​ ​ for presets.
The register 0xC0400600 can take values from 0 (?) to F, among which 3 groups are allocated (from faster write speed to lower):
- 0x8, 9, a, b, c, d, e, f
- 0x3
- 0x1, 2, 4, 5, 6, 7
Within each group, if you change the register, the write rate remains the same.

The register 0xC0400604 can take values from 0 (?) to F, among which 4 groups are allocated (from faster write speed to lower):
- 0x3, 7, b, f
- 0x2, 6, a, e
- 0x1, 5, 9, d
- 0x0, 4, 8, c
Within each group, if you change the register, the write rate remains the same.

The 0xC0400610 register can take values from 0 (?) to F, with the following distribution according to the recording speeds in video mode (values for sandisk extreme pro 256gb):
0x1 - 20.7mb\s is a possibly incorrect value that returns the write rate to "48mhz."
0x2 - 82 mb\s
0x3 - 77,1 mb\s
0x4 - 67 mb\s
0x5 - 57,8 mb\s
0x6 - 51,3 mb\s
... - recording speed drops as value increases
0xf - 26,2 mb\s

The 0xC0400614 register can take a lot of values, but the very speed of writing and reading is not directly affected. It affects in general whether the preset will work correctly, or the read/write speed will drop by 20.7-21 mb\s aka "48 mhz mode."
The value of this register may be associated with the value of the 0xC0400610 register.
I will explain by example:
At value of register 0xC0400610 equal 0x2, any value of register 0xC0400614 excellent from 0x1d000201 does prest not working (speed falls up to 20.7 mb\s).
A similar picture in the case of a register value 0xC0400610 0x1 - for it you need 0x1d000001 in the register 0xC0400614.
At the same time, if the register value 0xC0400610 equal to 0x3, 4, etc., the values ​ ​ of the register 0xC0400614, at which the preset will work (the speed does not fall to 20.7 mb\s), there can be a set (without any pattern of distribution).

The register 0xC0400614 can take values from 0 (?) to F. In this case, if the value of the register 0xC0400614 does not match the value of the register 0xC0400610 (with the exception of 0x0), then the write\read speed drops to 20.7 mb\s, making the preset inoperative.

Register 0xC0400618 (I tried all values from 0 to F in different combinations with other registers), registers 0xC0400624, 0xC0400628, 0xC040061C probably have no effect on the write speed\read and the operation of the preset.
For example, I made the following settings (for registers 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C) and got the same recording speed in video mode 82 mb\s, as if I left the default values.
https://drive.google.com/file/d/1HjVij4y-vICXrqGdpfRnANIQP7qLdEnp/view?usp=sharing

I ask you to check whether these patterns are observed on other cameras and sd cards and publish the results here.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 19, 2021, 09:46:10 PM
Vey nice findings @ainu 8).
Tested your numbers on the eos m but couldn´t make it work. Should test further on 5d3 as well. Hopefully @theBilalFakhouri will jump into your findings soon.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 19, 2021, 10:38:14 PM
Interesting numbers :), unfortunately these presets didn't work in 700D:

Quote from: ainu on September 19, 2021, 08:20:40 PM
https://drive.google.com/file/d/13WHC6LE9u5OFMuLL1dSbbv4zZIICWo1z/view
....
https://drive.google.com/file/d/1HjVij4y-vICXrqGdpfRnANIQP7qLdEnp/view?usp=sharing
(it switches to 48 MHz)

I need you to double check your values (just in case if you changed it by accident while making your tests), and perform one more test, your preset in the first link is:


static uint32_t uhs_regs[]       = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };
static uint32_t sdr_240MHzPlus[] = {        0x8,        0x3,                             0x2, 0x1D000201,        0x0,      0x201,      0x201,      0x100,        0x0 };


I added it into sd_uhs.mo with name of "240MHz +", here is the module compiled:
https://drive.google.com/file/d/1KQHxJfKnSHP5jy83dIutPW3ZIXeQUEJS/view?usp=sharing

Please select "240MHz +" preset, restart the camera, and run benchmarks in Play mode and share the result (use your Sandisk card).

Here is the source code (just in case):
https://drive.google.com/file/d/1mKRqyzHrGS0xKcRM26tHb4HCIN12ZyKu/view?usp=sharing



@Levas

How "240MHz +" preset performs on your 6D, could you make a test?



@ainu

Good job by experimenting with these, I will take a closer look into the patterns you have founded (currently no much time), keep it up!
Title: Re: UHS-I / SD cards investigation
Post by: ainu on September 20, 2021, 06:19:27 AM
Quote from: theBilalFakhouri on September 19, 2021, 10:38:14 PM
Please select "240MHz +" preset, restart the camera, and run benchmarks in Play mode and share the result (use your Sandisk card).

I tested it many times. Results is the same.
Photo mode W 97.5 R 140.5
https://drive.google.com/file/d/14D-tuIO91JulJ5EuGAwRzpRom4KLfSgL/view?usp=sharing
Video mode W 82.5 R 105,4
https://drive.google.com/file/d/1YzaUZtFihtPa_RJUesfsG3ONuFGOhqvK/view?usp=sharing

@theBilalFakhouri, can you add 240+ preset to module with value locks? I want to test it on Kingston SDA3. I think that this sd card can achive comparable read\write speed.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 20, 2021, 07:30:09 AM
Tested numbers on 5d3 but getting the 5000mb/s write and read speed so what's nicely delimiting 70D speed isn't working with 5d3 seemingly. Still like to see more tests though.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on September 20, 2021, 08:24:23 AM
I'm sure you didn't but have you accidently tested using a full card?
Title: Re: UHS-I / SD cards investigation
Post by: Danne on September 20, 2021, 08:32:36 AM
Sandisk 64 gb 170Mb/s card was tested. Plenty space left.
@theBilalFakhouri could you send me a build like the one ainu is using? Also source code for this build since patching routines also seems significant for the tests.

EDIT: Tested with ainus sd_uhs.mo build but still not working better so I guess 70D is the fastest cam with sd slot at the moment.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 20, 2021, 07:19:00 PM
Ok, cool, thanks for the retesting!

It seems we have Sandisk card reader (https://www.amazon.com/SanDisk-UHS-I-Card-Reader-Sddr-C531-Gnann/dp/B07HB866XN) in our cameras :P

240MHz Plus preset founded by @ainu on his 70D:

(https://i.ibb.co/2FfPhHV/240-MHz-Plus.jpg)

(I reposted this above picture from @ainu from this reply (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236770#msg236770))

And yes, it's:
- 97.5 MB/s write speed
- 140.5 MB/s read speed!

-Preset seems stable on 70D from the benchmarks

Quote from: ainu on September 20, 2021, 06:19:27 AM
@theBilalFakhouri, can you add 240+ preset to module with value locks? I want to test it on Kingston SDA3. I think that this sd card can achive comparable read\write speed.

Some SD cards (like your Kingston card) doesn't accept these high speeds, it mostly won't work, but I can add it (maybe tomorrow) I will PM you . .




Quote from: Danne on September 20, 2021, 07:30:09 AM
Tested numbers on 5d3 but getting the 5000mb/s write and read speed so what's nicely delimiting 70D speed isn't working with 5d3 seemingly. Still like to see more tests though.

5000MB/s or these very high numbers in benchmarks means card can't be accessed, or can't be accessed normally which means the overclock didn't work (mostly it's a Bench.mo bug, can't deal in these type of cases) so there no actual write process happening on the card.

Quote from: Danne on September 20, 2021, 08:32:36 AM
@theBilalFakhouri could you send me a build like the one ainu is using? Also source code for this build since patching routines also seems significant for the tests.

ainu is probably using ArcziPL's custom build (https://www.magiclantern.fm/forum/index.php?topic=25786.0), source code same as what we have (crop_rec_4k_mlv_lite_snd), only the difference a working lossless compression for 70D and some fixes for FPS code for 70D, nothing related to patches routine, at some point I asked ArcziPL for his source code, and I got it, will try to find it and will PM you



Quote from: ainu on September 19, 2021, 08:20:40 PM
Raw video settings https://drive.google.com/file/d/18t2nmtp4DPVM8c0COOiHOCXnaf0StwMt/view?usp=sharing

@ainu
You know that you can use 14-bit lossless compression instead of 14-bit uncompressed and this would cut data rate to nearly to half, you don't have to use 14-bit uncompressed (unless you are making write speed tests)
Title: Re: UHS-I / SD cards investigation
Post by: ainu on September 20, 2021, 09:02:56 PM
Yes, I use using ArcziPL's custom build. At 14 bit lossless I can record about 2240x... with 240mhz+ preset.

Quoteat some point I asked ArcziPL for his source code, and I got it, will try to find it and will PM you
This will be very good! Please PM to me too. I want to make a working movie log when shooting Raw video (current work only on non raw video or raw + h.264 proxy).

May be I'm wrong about registers 0xC0400624, 0xC0400628
I test value 0x1 in 0xC0400620 with 0x2 in 0xC0400610 - all speed drops after first write test (0xC0400624 and 0xC0400628 = 0x201), but if I change both registers 0xC0400624, 0xC0400628 to  0x301, then I achive in Play mode W 96,6 R105,5 mb\s, Video mode W 20,7 R 20,7 mb\s.
static uint32_t uhs_regs[]       = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };
static uint32_t sdr_test1[] = {        0x8,        0x3,                             0x2, 0x1D000201,        0x0,      0x301,      0x301,      0x100,        0x1 };


If I set to 0x301 only 0xC0400628, then I achive in Play mode W 96,6 R 140,4 mb\s, Video Mode W 82 R 20.7 mb\s.
static uint32_t uhs_regs[]       = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };
static uint32_t sdr_test2[] = {        0x8,        0x3,                             0x2, 0x1D000201,        0x0,      0x201,      0x301,      0x100,        0x1 };
Title: Re: UHS-I / SD cards investigation
Post by: ainu on September 21, 2021, 08:57:02 PM
Oh, man's, you don't belive me...
https://drive.google.com/file/d/1JmaM36AY8KLrB3WaIx4fEqvDIVW9LfDq/view?usp=drivesdk
Play mode W 96.2 R 192.1 mb/s and photo files write and read correctly.
Video mode 20.7  :(
at settings
https://drive.google.com/file/d/1KXFelYhaihiDB28fPYnrOzXyjlqsFjks/view?usp=drivesdk

Edit:Update settings image! Previous picture was wrong.
Title: Re: UHS-I / SD cards investigation
Post by: Levas on September 23, 2021, 05:21:37 PM
Tested the build Bilal posted on the 6d.

Quickly benchmarked the 240+ preset in both play and video mode...but both my sandisk cards (95mb/s and 170mb/s) drop to ~20mb/s read/write speed  :-\

Will look into the source and test a little further but I think these 240+ settings won't work for 6d  :-\
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 24, 2021, 12:20:44 AM
This preset (240MHz +) appears to work (partially) in 700D:

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };
{        0x8,        0x3,                             0x2, 0x1D000201,        0x0,      0x100,      0x100,      0x100,        0x2 };


(I applied this patch (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236735#msg236735) to prevent the switch to 48 MHz, without it switches to 48 MHz)

-Benchmarks:
Write speed: 91.6 MB/s
Read speed: +800 MB/s (there is a problem when reading to card with this preset, card can't be accessed, it a bit strange because write speed benchmarks are working fine)
Write speed: 91.8 MB/s
Read speed: +4000 MB/s (same thing)
(.ppm file didn't save correctly)

-Benchmarks with regular 240 MHz preset:
Max write speed: ~90.3 MB/s

So there is ~1 MB/s improvement, I think this is the maximum write speed could be achieved with my Sandisk 170 MB/s SD card.

I think above preset may work with 70D too normally as this preset (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236769#msg236769) (which was founded by ainu), but not sure, a test is required, could you try it @ainu?

Currently 240 MHz + presets aren't stable for other DIGIC 5 cameras, except for 70D:

TO DO:
Figure out why :)

It might be from:
-ainu's SD card version, his card might just accept these overclocked values out of the box
-SD overlock registers
-other SD controller parameters (e.g card current (same as power limit?), driver strength... other things?) ــــــــــــــــــــــــــــــــــــــــ> (I am trying to understand these, and how to control it)



If a1ex was around, I would ask him for these two useful things which might help us a lot to understand what is happening:

Quote from: a1ex on June 18, 2017, 11:54:16 PM
Xilinx QEMU (http://www.wiki.xilinx.com/QEMU) includes UHS emulation (along with some other nice stuff) and is based on QEMU 2.6.x (at the time of writing). Currently we have patches for QEMU 2.5.0 and 2.9.0, and for the Xilinx version, it will be something in-between. If there is interest, I can commit the patches.

And source code of:
Quote from: a1ex on April 07, 2018, 04:35:39 PM
sd_uhs_d.mo (https://a1ex.magiclantern.fm/bleeding-edge/uhs/sd_uhs_d.mo) (rename to sd_uhs.mo, otherwise it doesn't work for some reason)
Supported models for this diagnostic version: 5D3 1.1.3 and 700D 1.1.5.

700D: it won't perform any overclocking (will only run some tests to print card capabilities returned by CMD6).
5D3 1.1.3: it will only install the basic version of the hack (with 700D parameters) and will run some tests to print card capabilities.

-Currently getting 240MHz + preset to work stably is low priority for me (so currently I won't make further tests with it), I will just stick with regular 240 MHz preset or at least 192 MHz and my interest to make it work on 5D3 (or get 240MHz working on EOS M).



ainu has mentioned that the regular 192/240 MHz presets are working fine with his Sandisk Extreme PRO UHS-I U3 170 MB/s (SDSDXXY-256G-GN4IN):
Quote from: ainu on September 17, 2021, 10:56:22 PM
2. Sandisk Extreme Pro 256GB (SDSDXXY-256G-GN4IN) all presets work well.
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on September 24, 2021, 05:06:06 AM
From strings in the mo file, it might be this commit: https://foss.heptapod.net/magic-lantern/magic-lantern/-/commit/1d9bdf74695ad9439d8062480366b84e533dfc02

Possibly it is based on that with unpushed changes.
Title: Re: UHS-I / SD cards investigation
Post by: ainu on September 24, 2021, 07:05:44 PM
Quote from: theBilalFakhouri on September 24, 2021, 12:20:44 AM
This preset (240MHz +) appears to work (partially) in 700D:

{ 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };
{        0x8,        0x3,                             0x2, 0x1D000201,        0x0,      0x100,      0x100,      0x100,        0x2 };


a test is required, could you try it @ainu?
I try, but no luck - R/W speed 20.7 mb.

Within a few days, I want to get the result of the sandisk 256gb card recording speed test (in the cardrider and usb 3.0 slot) to understand the maximum above which not to step.

Quote from: theBilalFakhouri on September 24, 2021, 12:20:44 AM
Currently 240 MHz + presets aren't stable for other DIGIC 5 cameras, except for 70D:
TO DO:
Figure out why :)
...
-other SD controller parameters (e.g card current (same as power limit?), driver strength... other things?)

Is it possible to determine these parameters by measuring the supply voltage, current, etc. on the contacts of the card?
I can assemble an extender from the microSD->SD adapter and SD card slot so that I can access the card contacts while using it in the camera.


Quote from: names_are_hard on September 24, 2021, 05:06:06 AM
From strings in the mo file, it might be this commit
Thank you. I will see it later. May be it help me to underdstand what is happening.
Title: Re: UHS-I / SD cards investigation
Post by: srsa on September 24, 2021, 07:26:38 PM
Quote from: theBilalFakhouri on September 24, 2021, 12:20:44 AM
And source code of:
Quote from: a1ex on April 07, 2018, 04:35:39 PM
sd_uhs_d.mo (https://a1ex.magiclantern.fm/bleeding-edge/uhs/sd_uhs_d.mo) (rename to sd_uhs.mo, otherwise it doesn't work for some reason)
To get the source code of a binary module, follow this guide (https://www.magiclantern.fm/forum/index.php?topic=19336.msg215912#msg215912).
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on September 24, 2021, 08:10:40 PM
Huh, interesting.  Seems weird compared to listing the commit hash so you can lookup the whole thing.

Due to the forced repo change, the hash in this module can't be used to find the base commit.  So you get the diff but it's not that useful (unless there's some way to map Heptapod hashes to bitbucket ones?  Last I remember there wasn't).

My local bitbucket hg copy doesn't have the hash so I suspect this is some local branch of Alex's that never got pushed.  But I suck at hg so I could easily be doing something wrong.  Does hg sync everything?  I vaguely recall it doesn't so maybe there was a public branch but I never had it locally?
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 24, 2021, 08:13:38 PM
Quote from: ainu on September 24, 2021, 07:05:44 PM
I try, but no luck - R/W speed 20.7 mb.

Okay, thanks for testing!
(We should really understand how these registers works and what it affect)

Quote from: names_are_hard on September 24, 2021, 05:06:06 AM
From strings in the mo file, it might be this commit: https://foss.heptapod.net/magic-lantern/magic-lantern/-/commit/1d9bdf74695ad9439d8062480366b84e533dfc02

Possibly it is based on that with unpushed changes.

Nah, unfortunately it's not that commit, yeah it's unpushed changes

Quote from: srsa on September 24, 2021, 07:26:38 PM
To get the source code of a binary module, follow this guide (https://www.magiclantern.fm/forum/index.php?topic=19336.msg215912#msg215912).

Thanks for mentioning this! that guide did the trick (I can see the changes, but didn't test it yet), awesome hack to hack the hack
Title: Re: UHS-I / SD cards investigation
Post by: srsa on September 24, 2021, 08:37:04 PM
Quote from: names_are_hard on September 24, 2021, 08:10:40 PM
Due to the forced repo change, the hash in this module can't be used to find the base commit.
I have a working copy of the old repo, the output of hg log does list the following changeset:
changeset:   17218:b2db3dc9dd4f
branch:      sd_uhs
user:        alex@thinkpad
date:        Wed Apr 04 22:32:31 2018 +0200
summary:     sd_uhs: minor fixes and fine-tunings
The hash also appears (https://foss.heptapod.net/magic-lantern/magic-lantern/-/commit/1d9bdf74695ad9439d8062480366b84e533dfc02) as one of the last commits of the sd_uhs branch on heptapod. Searching for that hash or the commit msg (on the heptapod web interface) gets me no results though.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on September 24, 2021, 09:17:25 PM
Quote from: theBilalFakhouri on September 24, 2021, 12:20:44 AM
And source code of:
Quote from: a1ex on April 07, 2018, 04:35:39 PM
sd_uhs_d.mo (https://a1ex.magiclantern.fm/bleeding-edge/uhs/sd_uhs_d.mo) (rename to sd_uhs.mo, otherwise it doesn't work for some reason)

Quote from: srsa on September 24, 2021, 07:26:38 PM
To get the source code of a binary module, follow this guide (https://www.magiclantern.fm/forum/index.php?topic=19336.msg215912#msg215912).

Source code has been restored: sd_uhs_d.c (https://bitbucket.org/bilal_fakhouri/magic-lantern/downloads/sd_uhs_d.c), compiled the module and made a test, output appears to be the same as the sd_uhs_d.mo provided by a1ex
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on September 24, 2021, 09:40:21 PM
Yay, problem sorted!  Turns out it was the commit I already suggested (but couldn't prove because heptapod didn't preserve commit hashes).

Not very relevant, but my mistake was guessing that "hg log b2db3dc9dd4f" would tell me about that commit (because searching for how to do this didn't get me any useful results...).  Instead it silently does nothing.  "hg log|grep b2db3dc9dd4f" does work.  Hg is weird and I'm glad I don't have to use it.
Title: Re: UHS-I / SD cards investigation
Post by: Teamsleepkid on October 04, 2021, 01:07:31 AM
I bought that sd card that was pictured. I hope It helps. I get a lot of drop outs using 5k frtp. Be careful doing some of the stuff in this thread I somehow bricked my camera. Then I freaked out and was going to beg people to help me fix it. But then I found a thread on fixing bricked cameras and fixed mine myself somehow lol. Anyone getting 5k frtp to run reliably? Im on the eos m. Anyway thanks for your work. Oh also it did seem to work really well when I loaded the module. Before a bricked my cam. Thanks for your work:)
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on October 10, 2021, 11:11:21 PM
@Teamsleepkid
http://www.catb.org/~esr/faqs/smart-questions.html#beprecise

Quote from: Teamsleepkid on October 04, 2021, 01:07:31 AM
I bought that sd card that was pictured. I hope It helps. I get a lot of drop outs using 5k frtp.

Could you share a picture for the card?
Have you tried running card benchmarks, result?

Quote from: Teamsleepkid on October 04, 2021, 01:07:31 AM
Be careful doing some of the stuff in this thread I somehow bricked my camera. Then I freaked out and was going to beg people to help me fix it. But then I found a thread on fixing bricked cameras and fixed mine myself somehow lol.

How did you brick your camera, what happened? how did you fix your *bricked* camera?

Quote from: Teamsleepkid on October 04, 2021, 01:07:31 AM
Anyone getting 5k frtp to run reliably? Im on the eos m.

It should work fine, if the problem is only early recording stop then you should run benchmarks in camera for your card, and see if write/read speeds are stable or not, you may want to format the card in camera (low level format) at least for once to get stable write speed, is there another problem with 5k frtp?

You may consider using lower bit-depths for 5K FRTP, 14-bit lossless might not be continuous. it's off-topic here, report 5k frtp problems in the dedicated thread (https://www.magiclantern.fm/forum/index.php?topic=25781.0) for Danne's builds for EOS M.

Quote from: Teamsleepkid on October 04, 2021, 01:07:31 AM
Anyway thanks for your work. Oh also it did seem to work really well when I loaded the module. Before a bricked my cam. Thanks for your work:)

Which module? could give me a link for it.
Title: Re: UHS-I / SD cards investigation
Post by: lightspeed on October 13, 2021, 08:15:11 AM
I just want to confirm. Is 240 working on the eos m?

I tried the module and it seemed to work at first but was very buggy.
Title: Re: UHS-I / SD cards investigation
Post by: Teamsleepkid on October 14, 2021, 05:32:19 AM
Sorry so vague. So I used this link

https://drive.google.com/file/d/1KQHxJfKnSHP5jy83dIutPW3ZIXeQUEJS/view?usp=sharing

and put the module in my magic lantern folder. Replacing the existing one. Like I said somehow bricked my camera somehow fixed it. I really forget what I did or what it said it was like a week ago. I got some screen to come up. Loaded something Alex made to save cameras. It worked.

The card was the Sandisk extreme 120MB card
Title: Re: UHS-I / SD cards investigation
Post by: Teamsleepkid on October 14, 2021, 05:34:28 AM
Quote from: theBilalFakhouri on September 04, 2021, 05:16:27 PM
This card appears to accept 240 MHz overclock (tested on 700D by a user interacted with me on FB)

(https://i.ibb.co/SBPwtm0/Sandisk-Ultra-120-MB-s.jpg)

I ran a few benchmarks I'm getting like 56MB write speeds which is ok to get yellow running 5kfrtp so pretty happy way less drop outs than my 95mb extreme pro. Thank you. Sorry I'm so bad at using the forum. And I always use 10 bit I don't use 12 or 14 because I don't want to choke the card.
Title: Re: UHS-I / SD cards investigation
Post by: 70MM13 on December 22, 2021, 05:22:17 PM
Hi all,
I'm feeling the itch to buy an SD card for my good old 5d3 if there's anything that might be worth testing for overclocking...
The sandisk 170MB/s extreme pro I am using now is excellent and always reliable, so I don't "need" a new card, but if there's something new with the right udma characteristics that may be faster, I'd order one to try it out!
Has anyone been keeping abreast of anything new and interesting?
Title: Re: UHS-I / SD cards investigation
Post by: Frayfray on December 24, 2021, 11:35:18 PM
there are  three cards that i want to try Lexar professional silver series 1066x uhs1 120 MB/s write speed 160 MB/s read ,new Samsung PRO PLUS same specs . and Kingston canvas go plus but has lower specs 90MB/s write but claims 170 MB/S read speed ...note that these are claims that they are making . not me although i did order new card readers by kingston sandisk to test on computer ..i think i read some were that they are using  DDR208 controller on the UHS 1 pins to get higher read speeds
Title: Re: UHS-I / SD cards investigation
Post by: lightspeed on April 23, 2022, 12:55:54 AM
240mhz with the new build and the old sd hack is working for me on eos M.

Anyone else have it working?
Title: Re: UHS-I / SD cards investigation
Post by: ZEEK on April 23, 2022, 03:00:42 PM
Quote from: lightspeed on April 23, 2022, 12:55:54 AM
240mhz with the new build and the old sd hack is working for me on eos M.
Anyone else have it working?
Stops immediately after a second on my Sandisk Extreme PRO 170 MB/s with 240 MHz
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 23, 2022, 03:36:31 PM
Quote from: lightspeed on April 23, 2022, 12:55:54 AM
240mhz with the new build and the old sd hack is working for me on eos M.

Anyone else have it working?

Little more details, please!
Which old "SD hack"? Link, please!
And card in use.
Title: Re: UHS-I / SD cards investigation
Post by: lightspeed on April 23, 2022, 06:06:39 PM
I used the one with the patch that prevents it from going into safe mode. And I disabled lua script and it works for me.
Title: Re: UHS-I / SD cards investigation
Post by: lightspeed on April 23, 2022, 06:08:37 PM
Tlapse.lua script
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on April 23, 2022, 06:52:16 PM
Quote from: Walter Schulz on April 23, 2022, 03:36:31 PM
Which old "SD hack"? Link, please!

Quote from: lightspeed on April 23, 2022, 06:06:39 PM
I used the one with the patch that prevents it from going into safe mode. And I disabled lua script and it works for me.

@lightspeed means this:
https://www.magiclantern.fm/forum/index.php?topic=12862.msg236735#msg236735

-Edit: I wrote my notes on it, it wasn't really stable, see masc's (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236745#msg236745) replies (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236755#msg236755), one of his clips got corrupted with this sd_uhs version.
Title: Re: UHS-I / SD cards investigation
Post by: lightspeed on April 23, 2022, 07:07:23 PM
Mine was buggy to.  It the. I disabled that tlapse script and it seems to work good now.
Title: Re: UHS-I / SD cards investigation
Post by: rinski on April 25, 2022, 06:41:51 PM
Hello, could you explain to us how the lua script is disabled, it gives me an error and the version of the sd patch is that of September 17, 2021?
Thank you.
Title: Re: UHS-I / SD cards investigation
Post by: rinski on April 28, 2022, 08:11:21 PM
card full
load error 0:ML/CROPMKS/16_9.bmp
Title: Re: UHS-I / SD cards investigation
Post by: lightspeed on May 01, 2022, 01:31:40 AM
it seems to work if you disable the aspect ratio as well.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 12, 2022, 07:48:19 PM
I did some tests on 100D, it doesn't accept 240 MHz overclock at all (*both 5D3 and EOSM does for some time @ 240 MHz but not practical), tried with only write benchmark and with *forcing patch (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236735#msg236735)*, no sign of working 240 MHz overclock on 100D.

*on 5D3 240 MHz does work but only for write operations, read operations won't be possible. on EOS M it does work with *forcing patch* but not stable until the card become inaccessible.
BTW, 240 MHz does work on 650D/700D/6D/70D without a problem (with compatible card).

Personally I won't touch UHS stuff again until I get the following:
Quote from: a1ex on June 18, 2017, 11:54:16 PM
Xilinx QEMU (http://www.wiki.xilinx.com/QEMU) includes UHS emulation (along with some other nice stuff) and is based on QEMU 2.6.x (at the time of writing). Currently we have patches for QEMU 2.5.0 and 2.9.0, and for the Xilinx version, it will be something in-between. If there is interest, I can commit the patches.

It might be easier to spot the differencees among e.g. 100D and 700D regarding SD configurations using QEMU.
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on June 12, 2022, 08:01:37 PM
Qemu 4 also looks to have UHS-I support.
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on June 12, 2022, 08:16:00 PM
QEMU 4 doesn't have DebugMsg I think?
I am asking for something ready to work with :P QEMU 4 might need some patches too :-\

If you managed to run card benchmarks using QEMU 4 with e.g. DIGIC 5 or 7/8 model ROMs and if benchmarks matched reality benchmarks numbers, please let me know :D
Title: Re: UHS-I / SD cards investigation
Post by: names_are_hard on June 12, 2022, 08:24:56 PM
What would demonstrate support for (or lack of) DebugMsg in Qemu 4?

I'd never expect emulated hardware to match benchmark numbers on a physical cam.  What are you trying to check for?
Title: Re: UHS-I / SD cards investigation
Post by: lightspeed on June 12, 2022, 11:53:10 PM
Quote from: rinski on April 25, 2022, 06:41:51 PM
Hello, could you explain to us how the lua script is disabled, it gives me an error and the version of the sd patch is that of September 17, 2021?
Thank you.


I just disabled it so it didn't run at startup. once I did that it seemed to work pretty good. but only on a 64 gb card. not sure why it doesn't work on others all the time.
Title: Re: UHS-I / SD cards investigation
Post by: lightspeed on June 19, 2022, 10:38:52 PM
it also gets a little hot. 50 c.

can the camera run at 50? will that harm it?
Title: Re: UHS-I / SD cards investigation
Post by: Skinny on June 20, 2022, 09:53:52 AM
I used to run 550D at 60 degrees or even more for hours.. for timelapses in video mode with slow fps. Red thermometer sign was there, but camera didn't turn off. Although it should at some point of overheat. I still use this camera sometimes without any problems, didn't even noticed any increased sensor noise (it was noisy before, it is noisy now, lol)
Title: Re: UHS-I / SD cards investigation
Post by: lightspeed on July 02, 2022, 03:57:53 AM
if you format your card on Mac with the following it seems to make it work.

sudo news_exfat -c 4 disk2s1


replace disk2s1 with whatever disk you have.
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on July 02, 2022, 09:35:22 AM
Are we talking about emulation here?
We have an open issue with illegal directory/folder attributes with ExFAT and Big Sur (or up). https://www.magiclantern.fm/forum/index.php?topic=25656
Title: Re: UHS-I / SD cards investigation
Post by: theBilalFakhouri on August 13, 2022, 12:27:05 AM
Reproduced ainu's (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236770#msg236770) finding (https://www.magiclantern.fm/forum/index.php?topic=12862.msg236775#msg236775) on 700D (I had to make some tweaks):

(https://i.ibb.co/F35MxpX/280MHz.png) (https://ibb.co/Jtv4BZq)

If we calculated the frequency from read speed, it's ~140 (MB/s) * 2 = ~280 MHz

Notes:
-This preset not stable in LiveView, it drops to 21 MB/s.
-ainu's SD card variant apparently faster than mine, look at write speed (ainu's card vs my card).
So ainu probably was lucky to have faster SD card and the preset worked out of the box (without the tweaks I did, or maybe 70D SD configuration is different from the one in 700D).

-This preset is useless for now, unless if it works with new Sandisk 200 MB/s 140MB/s SD cards. The tweaks I used to make the preset work could be useful.
-Details will be shared in future (once it will be organized, and once I can share it), will recap all SD stuff too in one reply.



I will have some news soon for 5D3/EOS M/100D users regarding 240 MHz SD overclocking. I was digging into SD stuff in the past few weeks :) . if you was on Discord (https://discord.gg/uaY8akC) You could have known some info.
Title: Re: UHS-I / SD cards investigation
Post by: Danne on August 13, 2022, 09:00:58 AM
Quote from: theBilalFakhouri on August 13, 2022, 12:27:05 AM
I will have some news soon for 5D3/EOS M/100D users regarding 240 MHz SD overclocking. I was digging into SD stuff in the past few weeks :) . if you was on Discord (https://discord.gg/uaY8akC) You could have known some info.
Not bad  8).
Title: Re: UHS-I / SD cards investigation
Post by: SebastianC on August 13, 2022, 11:01:13 PM
This is good news!!
Title: Re: UHS-I / SD cards investigation
Post by: vastunghia on August 14, 2022, 08:24:32 AM
Can't wait :P
Title: Re: UHS-I / SD cards investigation
Post by: Icaab on August 21, 2022, 04:29:44 PM
Hello. I wanted to buy a new memory card, but there is no possibility to take extreme pro 170MByte/s.
The dilemma between:
Kingston canvas go plus 170MByte/s(64 and 128)
SanDisk Ultra 120MByte/s (128) or SanDisk Ultra microSDHC 120MB/s (128)
And some Samsung EVO Plus
No table where the comparison of different cards?
Or are there no analogs extreme and it is better to take it from the hands ? (Amazon, etc.)
Title: Re: UHS-I / SD cards investigation
Post by: Icaab on August 21, 2022, 04:36:23 PM
I'm sorry to repeat, but I'll write it here
Memory card needed for camera 650d 14 bit Crop Mode UHD
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on August 21, 2022, 04:58:49 PM
Still Gold Standard:
Sandisk Extreme Pro R170/W90

Tad slower:
Sandisk Extreme Pro 95 MB/s

My tests with Sandisk Extreme Plus R170/W90 were good, too. Same performance as Extreme Pro R170/W90. Same price, too.

Other cards tested failed at 240 MHz overclocking.
Just ordered a Sandisk Extreme R190/W130. In a week (or later) we may find out how it performs.
Title: Re: UHS-I / SD cards investigation
Post by: Icaab on August 21, 2022, 07:57:53 PM
Quote from: Walter Schulz on August 21, 2022, 04:58:49 PM
Still Gold Standard:
Sandisk Extreme Pro R170/W90

Tad slower:
Sandisk Extreme Pro 95 MB/s

My tests with Sandisk Extreme Plus R170/W90 were good, too. Same performance as Extreme Pro R170/W90. Same price, too.

Other cards tested failed at 240 MHz overclocking.
Just ordered a Sandisk Extreme R190/W130. In a week (or later) we may find out how it performs.
As I understood correctly - the speed of the memory card Extreme Pro 95 is not enough to record UHD, because when my test the video shooting was quickly interrupted, and sometimes worked steadily as an extreme pro 170 (and it turns out that the Sandisk Extreme Plus R170/W90 will also behave in this way and for 4K speed is required 170)
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on August 21, 2022, 08:05:29 PM
May I ask which write rate you are trying to achieve?
I just ran some tests and got around 80 MByte/s for video recording.
Title: Re: UHS-I / SD cards investigation
Post by: Icaab on August 21, 2022, 08:33:24 PM
Quote from: Walter Schulz on August 21, 2022, 08:05:29 PM
May I ask which write rate you are trying to achieve?
I just ran some tests and got around 80 MByte/s for video recording.
I meant the speed that is written on the memory card box. I just need to be able to shoot with Crop mode in 4K.
With Sandisk Extreme Pro R170/W90 I was capable to do this. Sandisk Extreme Pro 95 MB/s did give me problems (low-level formatting was)
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on August 21, 2022, 08:51:10 PM
It may (!) be caused by running on the edge and the small difference in card performance indeed makes a difference. Speculation, of course.
Title: Re: UHS-I / SD cards investigation
Post by: bobolee on April 29, 2023, 05:13:12 AM
Can you make the cameras compatible with ddr225? :D
https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/collateral/tech-brief/tech-brief-sandisk-quickflow-technology.pdf
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on April 29, 2023, 08:18:17 AM
Overclocking abilities (implemented):
160
192
240

Overclocking abilites (in theory):
280
384

There were first experiments with 280 but never got stable.
384 ... we don't really think there is any UHS-I card compatible wth this mode. Status ATM: Pipe dream.
Title: Re: UHS-I / SD cards investigation
Post by: 123kid on September 27, 2023, 11:48:40 PM
Hey,

i dont if somebody has this issue, but i can only record with on my 5dmarkiii about 18mb/s write speed.
i used several san disk pro extreme sd cards..but i cant get more than 20mb/s write speed.

i have the 5dmarkiii with firmware 1.2.3 and the latest magic lantern version on the sd cards...

any help?
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on September 27, 2023, 11:56:04 PM
Are you using Danne's build? Overclocking is only available there.
You have to activate sd_uhs.mo module. Then access Debug tab and enable overclocking.
Title: Re: UHS-I / SD cards investigation
Post by: 123kid on October 19, 2023, 12:10:49 PM
I was able to load the sduhs module on my 5d mark iii.
and get with sd card around 97 write mb/s and 70 read mb/s.

but anyone get around 170mb/s with a V60 or V90 sd card?

Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on October 19, 2023, 12:30:34 PM
No.
V60 and V90 are UHS-II cards. Look it up: There is no UHS-I card with this label and never can be because of rules made by SD Association.

UHS-II will run in UHS-I compatibility mode and may perform well @240 MHz (compared to fastests UHS-I cards). Or not.

Initial experiments with 280 MHz oc were not successful.
Title: Re: UHS-I / SD cards investigation
Post by: 123kid on October 19, 2023, 12:37:16 PM
Quote from: Walter Schulz on October 19, 2023, 12:30:34 PM
No.
V60 and V90 are UHS-II cards. Look it up: There is no UHS-I card with this label and never can be because of rules made by SD Association.

UHS-II will run in UHS-I compatibility mode and may perform well @240 MHz (compared to fastests UHS-I cards). Or not.

Initial experiments with 280 MHz oc were not successful.

what about a micro sd card which is able to write 120mb/s with an cf card adapter? I mean you can put the micro sd card into a cf adapter..
Title: Re: UHS-I / SD cards investigation
Post by: Walter Schulz on October 19, 2023, 12:40:58 PM
Have fun chasing the unicorn!