UHS-I / SD cards investigation

[names_are_hard]

The bench is quite short, I think?  I've never run it.  And the bench doesn't stress as many components as real video.

"Card full" could be a lot of things.  I think fundamentally it must be some function call returning a bad value, that's interpreted as "card full because it errored", but if you're stressing the card in unusual ways it could easily error in ways Canon didn't anticipate.  The card won't actually be full, since it's impossible to write to it that fast.

[70MM13]

yeah, the benchmark writes 1 GB to the card and then reads the same amount, then repeats both steps.

it's definitely nothing like the stress of actual video, that's correct!  but i guess it's a great way to compare performance without such a huge pile of variables

[IDA_ML]

Right now I am on vacation with my family at the beautiful Southern Black Sea Coast.  I have been filming a lot with the EOS-M at 5k anamorphic (1736x2928), 10-bits lossless and 14 fps using an older build without the latest overclocking implementations.  At these settings recording is continuous and I get a very beautiful motion blur of the moving water.  After postprocessing  I then interpolate the footage to 24 fps in Resolve with the optical flow function.  Image quality is breath taking, video is very smooth and there are no signs of aliasing whatsoever !!!  The only problems with jello effects due to 14 fps I get with close-up scenes with fast lateral motion.  If, with these latest and very exciting overclocking developments, the fps could be raised to at least 18 fps, I think, this would be fantastic and the problem with jello in problematic scenes would be eased significantly.

Since I only have one 95 MHz card with me here, I do not dare to test the latest overclocking builds because, if that card dies, it will be over with filming.  However, I am watching the latest overclocking efforts with great excitement and keep my thumbs pressed that you guys succeed!  So please, keep up the good work!  If you succeed, this will be a huge step forward!

Danne and all other guys involved, please accept my greatest appreciation for the genious work that you are doing with further improving ML.

[Danne]

14 fps possible with 1736x3256 with latest build. I shouldn't worry about card but I hear you.
Maximum 15fps with that resolution regardless of sd overclocking then image breaks.

Edit: Actually seems possible to reach 17fps with full resolution but no way continuous. Might be useful as a burst mode setting.

[IDA_ML]

Edit: Actually seems possible to reach 17fps with full resolution but no way continuous. Might be useful as a burst mode setting.

17 fps would be more than welcome even if recording is limited to say 5-10 sec.  This would ease interpolation and jello in problematic scenes significantly.  I remember filming quite a lot at 17 fps with the 5D3 in the past and footage really came out beautifully.  You even implemented a 17 fps mode in that camera, remember?  From the practical filming point of view, some attention to that mode in the EOS-M and 100D would be well worth the effort with the new overclocking features.

By the way, 10-bit lossless is God sent for high-resolution scene filming in the full sensor readout 5k anamorphic mode.  With brightly lit scenes, there is no noticeable image quality degradation compared to 12 or 14 bits but you never have to worry that recording will stop prematurely even with high-contrast, very bright and even overexposed scenes.  Absolutely fantastic!   

[Danne]

1736x2930 16fps now available on eosm. If you want 17fps you can reduce reg 6014 in sub menu but not continuous in 17fps:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg228768#msg228768

[IDA_ML]

Man, you are moving fast!  Very tempting, really!  Will test as soon as possible.  Thank you so much, Danne!

[rinski]

Hello, I have used in twixtor plugin with resolve 16 studio for 5d3 14 bits 1920x3240 15.003 anamorphic and it does not improve just in the interpolation at 24 fps, some advice to edit with this resolution.
Thank you.

[ZEEK]

Thanks Danne! 😎👍

[Levas]

Exactly. That´s why benchmarks are so fast when in play mode. Higher fps reduces the effect.

Didn't know that, but can confirm now
I have got a [email protected] crop preset and write speed in this preset is 97Mb/s, green indicator and a message besides: 3% idle 

But some less good news, I have had some speed drops in 240Mhz. It doesn't happen that often, but it does happen.
Not sure what triggers it, but I think it's triggered by the patching done in crop_rec module (not 100% sure though  ).
But yesterday I had 2 speed drops, both while extensively switching between different crop presets and recording clips and playback.
Today I had a speed drop, 2 or 3 times in a row (after camera restart) while trying out the same specific crop_rec preset.
Switched between some other presets and now I can't trigger it again in that specific preset...strange.

So I doubt if it is really the card that causes the speed drop, could be that it is triggered by the patching done in crop_rec module.

Edit:
So it doesn't look like a 5d3 only thing.
Does it on 5d3 happen with certain crop_rec presets, or while extensively switching between presets ?

[70MM13]

for me, it is a little different...
i was getting speed drops until i did the in-camera low level format after patching.  no more speed drops since!

but the big problem with the 5d3 seems to be inability to work at all at speeds over 160mhz without a very lucky module

have you tried doing the low level format after patching?

[Levas]

Card has been low level formatted in camera several times last few days.

[Danne]

5d3 is completely unreliable. Only 160Mhz is stable.
If using card spanning one won't notice because main recording is done to cf. I doubt the footage will be ok though on the sd card.
On the eosm 192Mhz seems rock solid. 240Mhz not working at all. Jip-Hop reported one time only write mode around 95Mb/s but read speed 20Mb/s on eosm. Not repeatable.

[theBilalFakhouri]

Caveat: SDR104 requires tuning the sampling point (not implemented, not performed by Canon firmware, but doable). That might be required to avoid random errors, speed drops, or higher frequency - if the controller supports it.

Try disabling SDR104 Patch, it *might* solve some problems e.g 240 MHz on EOS M, Card full on 5D3, Speed drop on 6D maybe, give it a try:

Code Select Expand

Comment it out
/* enable SDR104 */
//patch_hook_function(sd_set_function, MEM(sd_set_function), sd_set_function_log, "SDR104");

[Levas]

You might be right with that.
Disabled the SDR104 patching and see if speed drops again. 

[Aleks N]

Hi! I can test on 650D.

[Levas]

For 650d you could try the build posted by me on page 18 of this topic (Reply #446 on: July 02, 2020, 01:58:05 PM)

The module in that post should work on 650d in combination with crop_rec_4K build from the experimental builds downloadpage on this site.

[70MM13]

good news for 5d3 owners:

i just used danne's new july 5 build to shoot an episode of my new series "mindful moments" at 3616*1536 @ 24fps/12 bit lossless and it went perfectly!  continuous recording for 13 minutes until one of my cards filled up
fortunately it was close enough to the end that i just stopped there...
i overexposed it slightly (3 solidly clipped light bulbs in the scene) just to push it really hard, and it handled it beautifully!
this was with spanning, of course, and the load was more even between the two than before, thanks to the slightly faster clock.  if 240mhz eventually works, this could mean 14 bits

3616*1536 is the highest resolution possible at 2.35:1 so this is maxed out, baby!

link will be in a "share your videos" thread as soon as i upload, which takes quite a while with my snail paced cellphone internet

[theBilalFakhouri]

I got some info regarding the switching problem ( to 48 MHz), not fully sure but it's likely true, In short: Instability issues for 192 MHz and 240 MHz presets are because **we are still in SDR50 Mode not SDR104,

After the switch to 48 MHz I was Unable to apply the overclock again even after un-patching the memory and re-enabling the overclocking task, it stuck on 21MB/s

By changing 4 to 3 in this line:

Code Select Expand

if (sd_setup_mode_enable && regs[sd_setup_mode_reg] == 4)   /* SDR50? */
to

Code Select Expand

if (sd_setup_mode_enable && regs[sd_setup_mode_reg] == 3)   /* SDR50? */

And enabling un-patch memory for clearing previous overclock, and re-overclocking by the change above, now I can get ~41 MB/s without restarting the camera and after it switched to 48 MHz

I set the registers values to the default ones from Canon which are:

Code Select Expand

static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4,               0    };


I re-overclocked it a to 120 MHz or 132 MHz (the one below 160 MHz) and it gave 53 MB/s write speed in play mode, these are the settings:

Code Select Expand

static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x3, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x3,               0    };

Overclocking more than that is not possible, gives Card full error, and in benchmark gives ~4657 MB/s in read/write , these values appears in benchmark when the Card full error is presented and when you can't read on write on the card,

So if it was this SDR50,

Code Select Expand

[sd_setup_mode_reg] == 4)   /* SDR50? */

then is't this SDR25 ?

Code Select Expand

[sd_setup_mode_reg] == 3)

And if this code able to bring SDR104:

Code Select Expand

static void sd_set_function_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    qprintf("sd_set_function(0x%x)\n", regs[0]);

    /* UHS-I SDR50? */
   // if (regs[0] == 0xff0002)
  //  {

     regs[0] = 0xff0003;

   // }

..
..

patch_hook_function(sd_set_function, MEM(sd_set_function), sd_set_function_log, "SDR104");

Why it can't bring SDR25 which will not accept a preset above 132 MHz ?

Maybe it could accept 160 MHz preset and that happen when I brought SDR104 peace of code again to the code but with some changes:

Code Select Expand

static void sd_set_function_log(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    qprintf("sd_set_function(0x%x)\n", regs[0]);

    /* UHS-I SDR50? */
   // if (regs[0] == 0xff0002)
  //  {

     regs[0] = 0xff0004;  /*now it's 0xff0004 instead of 0xff0003*/

   // }

..
..

patch_hook_function(sd_set_function, MEM(sd_set_function), sd_set_function_log, "SDR104");

And I applied 160 MHz preset (after the swtich) which is:

Code Select Expand

static uint32_t sdr50_700D[]   = {        0x3,        0x3,                             0x2, 0x1D000201,        0x0,      0x201,      0x201,      0x100,        0x2,               0    };


Now it accepted preset more than 132 MHz, and I got 62 MB/s Write Speed, but not constant it gives Card full error after some seconds . .

Conclusion
I don't think SDR104 patch is an SDR104 patch

**Aren't we already in SDR104 by default ? and maybe the switch to 48 MHz mode is the SDR50 mode (according to SD Association and all the internet, max speed @ SDR104 is 104 MB/s , and SDR50 @ 50 MB/s so my results above because it switched to SDR50 not SDR25, and this make a lot of sense in the results above, and in the previous results I always commented out SDR104 patch, and I always got 160 MHz , 192 MHz, 240 MHz working fine, and with the patch enabled I had no difference in speed at all, so as a1ex said it's need some tuning to avoid random errors)

Card full error happens when you apply high frequency preset in not supported mode like SDR25, and SDR25 can't switch to 48 MHz because it is 48 MHz already so it gives Card full error

You can make a change from:

Code Select Expand

if (sd_setup_mode_enable && regs[sd_setup_mode_reg] == 4)   /* SDR50? */

to

Code Select Expand

if (sd_setup_mode_enable)   /* SDR50? */

This will apply any given registers values in any mode and the registers values will not change after the switch or sdSoftReset, so now instead of 21 MB/s speed when it switch to 48 MHz mode, it will give Card full error, now we know how to lock the values . . but how lock SDR50 or SD104 mode ?

I am not saying my assumptions are all true, but we are missing something . . maybe.

[Leopold LJ]

Hello, I have used in twixtor plugin with resolve 16 studio for 5d3 14 bits 1920x3240 15.003 anamorphic and it does not improve just in the interpolation at 24 fps, some advice to edit with this resolution.
Thank you.

You don't have to use twixt plugin in resolve.
1- Use Retime process with Optical flow && Motion estimation Enhanced better --> Good
2- Use Retime process with Optical flow && sppped warp (AI engine that do much than interpolation but use neuronal network to imagine frames 16 to 24, 18 to 24 ...(very very heavy processing for the graphic card)

[Aleks N]

For 650d you could try the build posted by me on page 18 of this topic (Reply #446 on: July 02, 2020, 01:58:05 PM)

The module in that post should work on 650d in combination with crop_rec_4K build from the experimental builds downloadpage on this site.

It works on 650D!

Everything started the first time on the "crop_rec_4k" firmware.
Results 60 MB/s before recording clips. I recorded two clips in ~ 1 minute in 14bit Losless. During recording, the indicator was always green, occasionally yellow. Recording did not stop even when moving the camera (previously interrupted).
After stopping the recording and restarting the camera, I made another benchmark. Result 65MB/s!

Сard: SanDisk Extreme Pro 128GB, 95/170 MB/s

https://youtu.be/IeNsbb8lSHg

I think this is enough for me so far) Thank you guys!


Link to this firmware (for 650D):
https://drive.google.com/file/d/1_W9yKHI6JtWufq1jhrXQMQ4KXh9lljSL/view?usp=sharing

or

https://yadi.sk/d/kihVgN_bFPCiVw

[Danne]

I used an alternative adress for uhs_regs and it works with 192Mhz as well:

Code Select Expand

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC040060C, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };   /* register addresses */
static uint32_t sdr_192MHz[]   = {        0x8,        0x3,                             0x4, 0x1D000301,        0x0,      0x201,      0x201,      0x100,        0x4 };

This:

Code Select Expand

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC0400610, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };
To this:

Code Select Expand

static uint32_t uhs_regs[]     = { 0xC0400600, 0xC0400604,/*C0400608, C040060C*/0xC040060C, 0xC0400614, 0xC0400618, 0xC0400624, 0xC0400628, 0xC040061C, 0xC0400620 };

[Levas]

If I'm not mistaken, register 0xC0400610 is set to 0x4 by default(Canon settings).
So looks like you're doing 'normal'   192Mhz setting with an extra register, 0xC040060c set to 0x4.

[theBilalFakhouri]

Yes, this register 0xC040060C has no effect, in this preset all the values are the default ones from Canon, except 0xC0400600, it's original value is 0x3, by tweaking it to 0x8 gives the 192 MHz Preset, without doing any another changes.

[Danne]

Yes, it's probably not used so disables that reg. Made 240Mhz work giving same write speed as 192Mhz. 160Mhz patch won't work with this.