Author Topic: UHS-I / SD cards investigation  (Read 20511 times)

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 966
Re: UHS-I / SD cards investigation
« Reply #25 on: August 11, 2014, 03:21:25 PM »
just wanted to add that by downclocking (for e.g. EOS 6D is 96MHz and could be downclocked to 12/  24 / 48MHz) we might achieve ofc less write speed but this may become useful for cases where continuous write speed is not much important (time lapses?). Advantage is less power consumption in theory... According to the sdcard specifications this should happen if some old card (mmc in formware) is used. Should run at max. 48MHz automatically.
70D.112 & 100D.101

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 966
Re: UHS-I / SD cards investigation
« Reply #26 on: August 20, 2014, 03:14:34 PM »
UHS-I capable cameras need to be switched to UHS104 & SDR104 (bus speed 104MB) by using CMD6 function group 1.
I found some CMD6 commands as stated earlier in ARMu. maybe someone can help to accomplish setting my 6D or any other UHS-I capable cam to the new bus speed by a CMD6 code switch?
70D.112 & 100D.101

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 966
Re: UHS-I / SD cards investigation
« Reply #27 on: September 03, 2014, 05:31:01 PM »
Ok need definitely help from one of you ML dev gurus on this. I spent again some hours to read through all official SD specification PDFs from V1.10 til 4.X and I can say CMD6 is all that is needed to set bus speed modes (SDR50 SDR104 DDR50 ...). There's one CMD6 SwitchCommand available in my 6D rom so it may be worth a try. Ofc this only works for UHS-I cards. I also simply guess that 5D3 is set to SDR25. a1ex tried once (see 1st page) but if something is done wrong then sd spec says that afterwards settings are defaulted and that's then SDR12. 

70D.112 & 100D.101

Levas

  • Hero Member
  • *****
  • Posts: 870
  • 6d - Nightly build user
Re: UHS-I / SD cards investigation
« Reply #28 on: September 06, 2014, 08:45:53 PM »
Wish I could help you...but have no idea how.

"There's one CMD6 SwitchCommand available in my 6D rom"

Sounds promising...

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 966
Re: UHS-I / SD cards investigation
« Reply #29 on: September 15, 2014, 06:03:22 PM »
....for example, only log those messages from CSMgrTask.

Can someone give me a detailed instruction based on dm-spy-experiments on how to log only "CSMgrTask" + "SdioDrv" +"SdioTsk". I bought a faster Sandisk card which is rated 60MB/s in write speed (before this I had panasonic Gold with a rating of max 45MB/s.). I wanna compare the logs I get from both cards and yes I am still trying to find a way where buss speed modes are set. Googling around I found this here which contains stuff like
Code: [Select]
#define MMC_CAP_UHS_SDR12       (1 << 15)       /* Host supports UHS SDR12 mode */

        #define MMC_CAP_UHS_SDR25       (1 << 16)       /* Host supports UHS SDR25 mode */

        #define MMC_CAP_UHS_SDR50       (1 << 17)       /* Host supports UHS SDR50 mode */

        #define MMC_CAP_UHS_SDR104      (1 << 18)       /* Host supports UHS SDR104 mode */

        #define MMC_CAP_UHS_DDR50       (1 << 19)       /* Host supports UHS DDR50 mode */

and maybe SdioDrv can show what's going on in 6D logs. So some hints where i can define what is going to be logged would be helpful.
70D.112 & 100D.101

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 966
Re: UHS-I / SD cards investigation
« Reply #30 on: September 25, 2014, 04:14:46 PM »
Please have a look at this example sub / stub. Looks like I understood myself what I can achieve with dm-spy-experiments but plz at least give me a hint if that would be ok:



Now I take /src/dm-spy-extra.c and in there I would add like this:

Code: [Select]
#ifdef CONFIG_6D
    { 0xyzblabla, "StateTransition", 4 , state_transition_log },
    { 0xFF78D814, "SDCheckStatus", WhatToPutHere??? }
#endif

Would that work for startup / "debug (don't click me?) and log some stuff related to SDCheckStatus or doesn't this make any sense to identify what's going on for other stubs?
70D.112 & 100D.101

kuga0509

  • New to the forum
  • *
  • Posts: 3
Re: UHS-I / SD cards investigation
« Reply #31 on: February 05, 2015, 06:37:45 AM »
Any luck on this?  Before I even read this thread, the only logical limiting factor I could find seemed to be that the software was limiting the hardware.  Since the hardware should support the SDR104 standard, and since the voltage and all of the other requirements would remain the same, it really seems that simple.  I hope this works out.  :)

The only other solution would be to load a different h264 profile to change the color space.  I know that isn't raw, but it may at least get the desired bit depth.  However, that would still require this same issue to be resolved since it would likely require a higher bus speed...

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10064
  • 5D Mark Free
Re: UHS-I / SD cards investigation
« Reply #32 on: June 18, 2017, 11:54:16 PM »
Revisiting this.

At the suggestion of Ant123, I've ran ML under the Xilinx version of QEMU, which also emulates UHS. This revealed the missing bits from my previous patch: on 5D3, although it was printing a message about 96MHz, the card interface was set up in the same way as for... 24 MHz (which explains the previous benchmark results).

Took the missing register configurations from 700D and...



8)



Now the details.

Xilinx QEMU includes UHS emulation (along with some other nice stuff) and is based on QEMU 2.6.x (at the time of writing). Currently we have patches for QEMU 2.5.0 and 2.9.0, and for the Xilinx version, it will be something in-between. If there is interest, I can commit the patches.

First, I've ran 5D3 1.2.3 from the dm-spy-experiments branch on both the camera and QEMU 2.5.0. Result.

The problem:
Code: [Select]
CSMgrTask:ff6bdb7c:23:05: Set Hi-Speed Mode( 48MHz )                ; real hardware
[   CSMgrTask:ff6bdb9c ] (23:05) Set Normal-Speed Mode( 24MHz )     ; QEMU 2.5.0

That's where the Xilinx QEMU comes in. Result: after a minor patch to the SCR structure (first field - SpecVersion - changed from 0 to 1), both the log from camera and Xilinx QEMU now have:
Code: [Select]
CSMgrTask:ff6bdb44:23:05: SD_GetAccessMode=3
CSMgrTask:ff6bdb7c:23:05: Set Hi-Speed Mode( 48MHz )

Also, besides a minor difference in handling CMD1, and the card capacity, the emulation matches the reality pretty well. There's even some nice debug info if you uncomment DEBUG_SD in hw/sd/sd.c and use the "-d sd" switch on the command line. Result - in particular, these lines:
Code: [Select]
CSMgrTask:ff6b8fec:23:01: sdSetFunction( 16776961 ) Start
CSMgrTask:000aea50:00:00: *** sd_setup_mode(0x2), from ff484738
sd: CMD16 0x00000040 state 4
sd: Response: 00 00 09 00 state 4
CSMgrTask:000aea50:00:00: *** sd_setup_mode(0x2), from ff6b8300
sd: CMD6 0x80ffff01 state 4
sd: Function default selected (fn grp 2)
sd: Function high-speed/SDR25 selected (fn grp 1)
sd: Response: 00 00 09 00 state 5
sd: CMD13 0x45670000 state 4
sd: Response: 00 00 09 00 state 4
sd: CMD16 0x00000200 state 4
sd: Response: 00 00 09 00 state 4
CSMgrTask:ff6b91c8:23:01: sdSetFunction( 16776961 ) End

Now we can analyze how the SD initialization code configures the hardware (MMIO registers). This is as easy as specifying "-d io" in QEMU's command-line. Result.

At this point I've applied the old patch and tried to figure out whether there's any obvious trouble. Result:

Without hack:
Code: [Select]
CSMgrTask:ff6bdb7c:23:05: Set Hi-Speed Mode( 48MHz )
CSMgrTask:000aeed0:00:00: *** sd_set_mode(0x1, 0x3), from ff6bdb88
(some registers)
CSMgrTask:000aeed0:00:00: *** sd_setup_mode(0x3), from ff484738
(some more registers)
CSMgrTask:ff6bc87c:23:01: sdReadBlk: st=0, num=1, buf=0x40983808

With hack:
Code: [Select]
CSMgrTask:ff6bdb5c:23:05: Set Hi-Speed Mode( 96MHz )
CSMgrTask:000aeef0:00:00: *** sd_set_mode(0x1, 0x4), from ff6bdb88
(some registers)
CSMgrTask:000aeef0:00:00: *** sd_setup_mode(0x4), from ff484738
(a few more registers, but many of them missing!)
CSMgrTask:ff6bc87c:23:01: sdReadBlk: st=0, num=1, buf=0x409837bc

In other words, sd_setup_mode(4) appears to skip some hardware configuration it might be supposed to perform. With other arguments, it sets a bunch of registers, and the argument appears to be related to SD clock speed (3 = 48MHz, 2 = 24MHz, 4 = 96MHz).

Let's see what it does on 700D. Result (also with a zoomed-in view).

Notice some additional registers on 700D (highlighted in red on the zoomed-in view). What's up with them?

The 0xC04004xx range is never set on 5D3, so these registers are probably specific to 700D hardware. Same for 0xC040063x and 0xC040064x. I didn't touch them.

The remaining registers, 0xC04006[012]x, are also set on 5D3, at other speed modes. In these other modes, their values are the same on 700D. You can see them here: sd_setup_mode(2), sd_setup_mode(4).

My hypothesis was that 5D3's SD controller is UHS-capable, but for some unknown reason (could be even problems during the initial tests), Canon decided not to include it in the firmware. As a result, some of the UHS initialization code (hopefully a small part) was optimized out.

So I've tried to take the missing register configurations from 700D.

Therefore, the patch for 5D3 1.2.3 becomes:
Code: [Select]
/* in dm-spy.c, right before dm_spy_extra_install() */
patch_instruction(0xff48446c, 0xe3a00000, 0xe3a00001, "SD 1.8V");

/* in dm-spy-extra.c */
static void sd_setup_mode_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    /* log the original call as usual */
    generic_log(regs, stack, pc);

    if (regs[0] == 4)
    {
        MEM(0xC0400600) = 3;
        MEM(0xC0400610) = 4;
        MEM(0xC0400614) = 0x1D000301;
        MEM(0xC0400618) = 0;
        MEM(0xC0400624) = 0x201;
        MEM(0xC0400628) = 0x201;
        MEM(0xC040061C) = 0x100;
        MEM(0xC0400620) = 4;
        MEM(0xC0400604) = 3;
    }
}

    /* under CONFIG_5D3_123 */
    { 0xFF4844A0, "sd_setup_mode", 1, sd_setup_mode_log },

If you decide to try it, make sure you don't have any important data on your card. Otherwise, you will be playing Russian Roulette with your data (just like with my other SD patch).

Does this apply to DIGIC 4 cameras?

I'm afraid not - the hardware configuration of these cameras is different (and a lot simpler). You now know where to look, so you can play with it, attempt to change the clock speed and report your findings.

Can the clock speed be pushed even further?

I have no idea. Feel free to play with these registers, run the benchmarks and report.

Can this be included in a module, to be used on a regular ML build?

That's hard, because the hack must be applied before the SD card gets initialized by Canon firmware (in other words, before loading any module, and also before loading the config file). So, even if we include it in ML core, it will be hard to create an option for it.

At the moment, the easiest way to try it would be a custom build. Probably best to start from the crop_rec_4k branch, as the backend support is there, and there is little reason to try this hack outside that branch.

It might be possible to switch the SD to a higher speed on the fly. Didn't investigate this approach.

How's this useful in practice?

Other than raw video recording on both CF and SD at the same time (aka card spanning), it's probably not very useful.

What's the maximum total speed (CF+SD)?

Load the benchmarks module (bench.mo) to find out.

How can I get similar results in QEMU?

Take a look at this post. In a nutshell, it's the dm-spy-experiments branch compiled with CONFIG_DEBUG_INTERCEPT_STARTUP=y for the camera, and additionally with CONFIG_QEMU=y for running it under the emulator. These two will give logs that can be directly compared, and with the logging options from QEMU, you can get additional details. Then, look for CSMgrTask in the logs, compare them and try to understand what it does. Also refer to SD docs (summarized nicely by nikfreak earlier in this thread) to understand the initialization protocol. That's it. If you get stuck, just ask (here or on IRC).

If there is interest, I can commit the patch for Xilinx QEMU and write a walkthrough similar to this one.

Please note I no longer have the UHS card to do more tests (it wasn't mine), so from now on, what will happen with this hack is entirely up to you.

aschille84

  • Freshman
  • **
  • Posts: 66
Re: UHS-I / SD cards investigation
« Reply #33 on: June 19, 2017, 08:03:27 AM »
Wow, so this means we can get another 30mb/s write speed. I would like to try it out.

Levas

  • Hero Member
  • *****
  • Posts: 870
  • 6d - Nightly build user
Re: UHS-I / SD cards investigation
« Reply #34 on: June 19, 2017, 10:38:24 AM »
Nice finding.

One question about this one:

Can the clock speed be pushed even further?

I have no idea. Feel free to play with these registers, run the benchmarks and report.


How is this in the other cams, like the 6d. I've always thought that UHS-I, 50 MByte/s (SDR50) was the limit here ?
Or is there still a small chance that it can reach UHS-I, 104 MByte/s (SDR104) ?

Ant123

  • Freshman
  • **
  • Posts: 68
Re: UHS-I / SD cards investigation
« Reply #35 on: June 19, 2017, 11:31:21 AM »
My hypothesis was that 5D3's SD controller is UHS-capable, but for some unknown reason (could be even problems during the initial tests), Canon decided not to include it in the firmware. As a result, some of the UHS initialization code (hopefully a small part) was optimized out.

If you can not enter UHS mode with ML but it works without, read this topic.
Conclusion:
Most SD cards need to be reinitialized by switching off SD power if they already were in UHS mode.