Author Topic: UHS-I / SD cards investigation  (Read 121101 times)

Kharak

  • Hero Member
  • *****
  • Posts: 984
Re: UHS-I / SD cards investigation
« Reply #150 on: April 10, 2018, 08:27:44 PM »
@a1ex

I always thought it was the Double Spanning from Canon and ML causing it, but if you say they dont interfere with each other, then I don't know what happened. I set the settings and hit record and the camera crashed and I got an Error code and the card was bust. Never happened before or after, never tried it again and will never try it again :)

I have not tried h264 to two separate cards, I never even considered it, but just to be clear, you mean Recording H264 to both cards simultaneously, is that an ML feature?. I also couldn't figure out how to set h264 proxy to record to SD while MLV went to CF.
once you go raw you never go back

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12505
Re: UHS-I / SD cards investigation
« Reply #151 on: April 10, 2018, 08:34:52 PM »
I've tried it before posting; no surprises. When you record raw video, Canon's LiveView remains idle (their code doesn't even know the record button was pressed, since the raw recorder catches that event and doesn't pass it back to Canon firmware). Remember the LiveView used to be turned off after 30 minutes, since Canon firmware believed the camera was idle (while it was actually recording raw video). For the card selection setting, ML only sees 0 = CF or 1 = SD, there are no other possible values for that property. ML does not (and did not) see the advanced options from Canon menu, like CF+SD.

I thought recording H.264 on two cards is a feature from Canon firmware (after you said you can "set Canon Menu to record to CF+SD").

In crop_rec_4k builds, H.264 goes to the card selected in Canon menu, while raw video (mlv_lite) always goes to CF if there's one inserted. In main builds and in mlv_rec from any builds (regular recording without H.264 proxy), both will go to the card selected in Canon menu. In mlv_rec with spanning enabled, main stream goes to CF and secondary stream goes to SD, regardless of Canon setting.

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2267
Re: UHS-I / SD cards investigation
« Reply #152 on: April 11, 2018, 05:23:47 AM »
@reddeercity: more details after I'll get a new card (it *is* possible to overclock the 5D2 CF interface).
:o Really , cool that's better then trying to change to udma 7
Increase the "bus speed" ? can't wait to read the details .
I saw something in the 5d2 disassembly that look interesting
Code: [Select]
"<K218 Board> SystemCLK 132MHz":
looks to be part of the system check before boot loader .

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2267
Re: UHS-I / SD cards investigation
« Reply #153 on: April 11, 2018, 06:48:32 AM »
search for "cfDecideTiming" in the rom, got this
Code: [Select]
*"cfGetRegisterTiming: I/O 250nS (PIO Mode4)"so "Mode4" =250nS I/O there was also a
Code: [Select]
*"cfGetRegisterTiming: I/O 120nS"but not "Mode"
this got me interested in the CF events , would be nice to have more write speed .

Edit: look like that the slowest after some searching , would need to be around 30 or 20ns for 100MB/s

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2267
Re: UHS-I / SD cards investigation
« Reply #154 on: April 11, 2018, 07:28:39 AM »
Is this kind of what you can do @a1ex "Double Transition Clocking"
http://www.pcguide.com/intro/fun/clockDouble-c.html
If not it's still interesting reading  :)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12505
Re: UHS-I / SD cards investigation
« Reply #155 on: April 11, 2018, 07:48:49 AM »
There is the DDR50 mode for SD cards, easily enabled on the card side from sdSetFunction, but... good luck configuring Canon controller to use that. Brute-forcing the "known" registers didn't help.

You can send ATA commands to the CF card (QEMU emulates them), so if the only difference between UDMA 6 and 7 is timing, it might even be possible to put the card in UDMA7.

Ant123

  • Contributor
  • Member
  • *****
  • Posts: 162
Re: UHS-I / SD cards investigation
« Reply #156 on: April 11, 2018, 02:06:19 PM »
good luck configuring Canon controller to use that. Brute-forcing the "known" registers didn't help.

Why do you think that camera controller has support of DDR mode?
In accordance with debugging messages inside D6 & D7 firmwares the fastest mode is SDR104 and there are no signs of DDR...

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12505
Re: UHS-I / SD cards investigation
« Reply #157 on: April 11, 2018, 02:57:34 PM »
I don't know whether the controller supports DDR or not, but - as you can see in previous logs - there are cards that support DDR50, but not SDR104. That's why I've assumed DDR50 might be an older standard.

Ant123

  • Contributor
  • Member
  • *****
  • Posts: 162
Re: UHS-I / SD cards investigation
« Reply #158 on: April 11, 2018, 04:30:18 PM »
I did not found significant changes in the standard.
Look at the paragraph 3.9.3 of "Physical Layer Simplified Specification" (from v3.01 to v6.00). There are three host types.

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2267
Re: UHS-I / SD cards investigation
« Reply #159 on: April 12, 2018, 10:29:06 PM »
You can send ATA commands to the CF card (QEMU emulates them), so if the only difference between UDMA 6 and 7 is timing, it might even be possible to put the card in UDMA7.
Ok , yet another reason to get QEMU going

David_Hugh

  • Freshman
  • **
  • Posts: 80
Re: UHS-I / SD cards investigation
« Reply #160 on: April 13, 2018, 11:06:29 PM »
This is insane. It really works, just like that. Did the test on a gold sandisk extreme 32gb and a 70D. This card now effectively records raw at 49Mb/s instead of 40, resulting in a noticeable bump in resolution/aspect ratio. For those saying they dont see the "real world" results, I ran the test in video mode and mlv_rec was already turned on. I dont know whether turning the cam off kills the overclocking for now, or if running it in video mode was what did it, but this combo worked for me. Also, the test showed a siginificantly different result in photo/video mode (53 vs. 43) and although it only showed 43 in video the card writes 49 continous.

I did notice however that the card got quit warm relatively quickly. Again. This is insane. You guys just made every cam that can run this like 10 times better.

50mm1200s

  • Member
  • ***
  • Posts: 177
Re: UHS-I / SD cards investigation
« Reply #161 on: April 14, 2018, 12:18:11 AM »
@a1ex, since some people got the cards killed by this experiment, wouldn't it be better to remove it from downloading, for now? Some people might use it for real world production and get the card crashed in the middle of the work...
I think it's a good idea, though. I can barely get 16:9 FullHD in crop_rec (50D), with this patch (if it works in CF too) it would possibly make it to continuous FullHD recording.
Do you need any help to get disposable cards that you can "fry" as you want? I could buy some cheap ones for you and send here from Brazil.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12505
Re: UHS-I / SD cards investigation
« Reply #162 on: April 14, 2018, 06:19:28 AM »
The pre-built module was already rolled back to a hopefully safer version and had a stronger warning, but OK, removed the download. Nobody tested it this week anyway. The diagnostic version (which does only the basic version of the hack on 5D3 and only prints some capabilities on 700D) is still available.

BTW, when N = 1, you may want to use the singular form (i.e. card instead of cards).

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: UHS-I / SD cards investigation
« Reply #163 on: April 14, 2018, 07:37:19 AM »
I removed my test build too.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1671
  • 6d - Nightly build user
Re: UHS-I / SD cards investigation
« Reply #164 on: April 14, 2018, 01:42:47 PM »
Just wanted to note, I'm still heavily testing with this and not messed up my card  :D
As far as I could tell, only the second build with driver strength options, messed up card(s).

For practical use and testing, I changed the code of the original build a little (removed the line where it starts showing the console text, and removed the tests and only kept the line with 160Mhz settings)
This way, now I can go to photo mode, click the SD overclock in the debug menu and within a second I'm ready to go. (No console text, just a few card reader led blinks)
Last night I photographed a live performance of 2 music bands and also did some video tests with the SD_overclock module.
I used the same SD card the whole time (2,5 hours) and activated the SD_overclock module multiple times, recorded some video and moved on taking photos (with
SD_overclock still activated)
I even recorded, successfully, a 5 minute length clip of 8558 frames in 2560 x 958 resolution in 14 bit lossless...on a 6d that is  :D
The MLV file was 21Gb in size  ;D


mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: UHS-I / SD cards investigation
« Reply #165 on: April 14, 2018, 04:56:59 PM »
Same here, working very nice on my 32gig micro SD card, definitely an improvement with no signs of card dieing yet.
500D/T1i  550D/T2i  600D/T3i  700D/T5i

loknar

  • Freshman
  • **
  • Posts: 78
  • 550D, EOS M
Re: UHS-I / SD cards investigation
« Reply #166 on: April 14, 2018, 08:16:20 PM »
As instructed I did some extensive tests of sd_uhs module and:
First of all, thank you guys, this really extends usability of my EOS M far beyond of my expectations.
I've been using two cards, SD Sandisk Extreme 16 GB and uSD Sandisk Extreme 32 GB.
I've recorded over 30 GB of data during 4 hours, cards are OK, camera is OK, Data are OK (I'm using 3rd April dforts build for EOS M and crop rec build for EOS M from 10th of March).
I shot at 2520x1072 with fps override to 24 fps (HighTV) in 12-bit lossless. Resulting Bandwidth around 55 MB/s.
In most of scenes recording could be continuous, but in some settings I've got only around 3-10 seconds, didn't found reason for this (other than maybe complexity of the scene or temperature of the processor).
Also I don't know how it is supposed to behave but in my case it was: new ML installation enable SD hack, benchmark:
Code: [Select]
===================
2018/04/06 22:29:28
===================
Before the hack: r:37MB/s w:34MB/s  W:32MB/s R:37MB/s  :)  [best 34MB/s]
SDR50 @ 96MHz  : r:37MB/s w:32MB/s  W:33MB/s R:37MB/s  meh [best 34MB/s]
SDR50 @ 96MHz  : r:37MB/s w:19MB/s  W:33MB/s R:38MB/s  meh [best 34MB/s]
SDR50 @ 80MHz  : r:32MB/s w:30MB/s  W:28MB/s R:32MB/s  meh [best 34MB/s]
SDR50 @ 80MHz  : r:36MB/s w:34MB/s  W:35MB/s R:37MB/s  :)  [best 34MB/s]
SDR50 @ 120MHz : r:54MB/s w:52MB/s  W:52MB/s R:55MB/s  :)  [best 52MB/s]
SDR50 @ 120MHz : r:54MB/s w:50MB/s  W:52MB/s R:55MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:42MB/s  W:42MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:41MB/s  W:42MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:35MB/s  W:36MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:35MB/s  W:35MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 120MHz: r:55MB/s w:53MB/s  W:52MB/s R:55MB/s  :)  [best 53MB/s]
SDR104 @ 120MHz: r:54MB/s w:51MB/s  W:50MB/s R:55MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:51MB/s w:48MB/s  W:48MB/s R:51MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:50MB/s w:48MB/s  W:48MB/s R:51MB/s  meh [best 53MB/s]
SDR104 @ 160MHz: r:72MB/s w:70MB/s  W:67MB/s R:73MB/s  :)  [best 70MB/s]
SDR104 @ 160MHz: r:72MB/s w:68MB/s  W:64MB/s R:73MB/s  meh [best 70MB/s]

Done.
Please run THOROUGH tests before using!!!
Then I could record videos with high bitrate. Once i switched camera off and on again memory patch wasn't (obviously) applied, so i tried to enable SD hack again, but i've got this:
Code: [Select]
===================
2018/04/10 08:29:10
===================
Before the hack: malloc error
 [best 66MB/s]
SDR50 @ 96MHz  : malloc error
 [best 66MB/s]
SDR50 @ 96MHz  : malloc error
 [best 66MB/s]
SDR50 @ 80MHz  : malloc error
 [best 66MB/s]
SDR50 @ 80MHz  : malloc error
 [best 66MB/s]
SDR50 @ 120MHz : malloc error
 [best 66MB/s]
SDR50 @ 120MHz : malloc error
 [best 66MB/s]
SDR104 @ 96MHz : malloc error
 [best 66MB/s]
SDR104 @ 96MHz : malloc error
 [best 66MB/s]
SDR104 @ 80MHz : malloc error
 [best 66MB/s]
SDR104 @ 80MHz : malloc error
 [best 66MB/s]
SDR104 @ 120MHz: malloc error
 [best 66MB/s]
SDR104 @ 120MHz: malloc error
 [best 66MB/s]
SDR104 @ 132MHz: malloc error
 [best 66MB/s]
SDR104 @ 132MHz: malloc error
 [best 66MB/s]
SDR104 @ 160MHz: malloc error
 [best 66MB/s]
SDR104 @ 160MHz: malloc error
 [best 66MB/s]

Done.
Please run THOROUGH tests before using!!!
Strangely enough, after this memory patch have been applied and i could continue recording with high bitrate.
I wanted to ask if is it possible to streamline usage of the hack (like add menu "apply saved SD clock values", or to avoid opening console), but since you retracted the module, i don't know whether you intend to explore this more. (And as far as i'm concerned i'm not giving this module back :D )

If you are interested in result, i posted it on youtube


ahmedvienna

  • New to the forum
  • *
  • Posts: 5
Re: UHS-I / SD cards investigation
« Reply #167 on: April 15, 2018, 02:33:53 PM »
I am trying to implement sd_uhs on my 6d. using the latest nightbuild.  have an error when I do it.  can anyone assist me.
here's the pic of the error.

https://pbs.twimg.com/media/DabZ9QVWsAA7zD-.jpg:large

thanks !
Ahmed
Canon 6D + 50 USM + 35 IS USM
Sandisk extreme pro 95mb/sec 32GB

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7018
Re: UHS-I / SD cards investigation
« Reply #168 on: April 15, 2018, 07:44:11 PM »
Wow, quite something. My Sandisk extreme pro 95mb/s shows 69mb/s in the test. Runs nicely on my 100D. Thanks Levas for feedback.
Regarding malloc error on initial testing I run the module with liveview closed. When opened malloc errors..

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: UHS-I / SD cards investigation
« Reply #169 on: April 15, 2018, 08:32:40 PM »
Looks like loknar, Danne, mk11174 and Levas are on the Plateau of Productivity with the sd_uhs module while some of us are stuck in the Trough of Disillusionment.



@Levas, looks like you got a good working version. Could you share the changes you made on the module?
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1671
  • 6d - Nightly build user
Re: UHS-I / SD cards investigation
« Reply #170 on: April 15, 2018, 08:59:53 PM »
This stuff is too good not to be used, thanks Alex  :D
This and the lossless option, makes continuous recording available for the SD cams  :D

The build I use is based on Alex first build, I only deleted a few lines, didn't add any. (I'm not that savvy in programming, but I had to have something more practical)
I deleted one line with 'show console' and I deleted a bunch off the lines where most of the settings are tested.
I only kept the highest 160Mhz setting in it.

Here's the altered source file I'm using:
https://drive.google.com/drive/folders/1A_OVNAucbHYfUXfPT9kxBy1IiUk-012U?usp=sharing

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: UHS-I / SD cards investigation
« Reply #171 on: April 15, 2018, 09:01:33 PM »

Then I could record videos with high bitrate. Once i switched camera off and on again memory patch wasn't (obviously) applied, so i tried to enable SD hack again, but i've got this:
Code: [Select]
===================
2018/04/10 08:29:10
===================
Before the hack: malloc error
 [best 66MB/s]
SDR50 @ 96MHz  : malloc error
 [best 66MB/s]
SDR50 @ 96MHz  : malloc error
 [best 66MB/s]
SDR50 @ 80MHz  : malloc error
 [best 66MB/s]
SDR50 @ 80MHz  : malloc error
 [best 66MB/s]
SDR50 @ 120MHz : malloc error
 [best 66MB/s]
SDR50 @ 120MHz : malloc error
 [best 66MB/s]
SDR104 @ 96MHz : malloc error
 [best 66MB/s]
SDR104 @ 96MHz : malloc error
 [best 66MB/s]
SDR104 @ 80MHz : malloc error
 [best 66MB/s]
SDR104 @ 80MHz : malloc error
 [best 66MB/s]
SDR104 @ 120MHz: malloc error
 [best 66MB/s]
SDR104 @ 120MHz: malloc error
 [best 66MB/s]
SDR104 @ 132MHz: malloc error
 [best 66MB/s]
SDR104 @ 132MHz: malloc error
 [best 66MB/s]
SDR104 @ 160MHz: malloc error
 [best 66MB/s]
SDR104 @ 160MHz: malloc error
 [best 66MB/s]

Done.
Please run THOROUGH tests before using!!!
Strangely enough, after this memory patch have been applied and i could continue recording with high bitrate.
I wanted to ask if is it possible to streamline usage of the hack (like add menu "apply saved SD clock values", or to avoid opening console), but since you retracted the module, i don't know whether you intend to explore this more. (And as far as i'm concerned i'm not giving this module back :D )

I only get memory error if I am in movie mode with MLV-Lite or MLV-REC turned ON, make sure these modules are off before you run this card speed module, once it is working, then ok to turn ON the MLV modules you want to use.

Obviously if you turn off camera, you will need to go through same procedure in same order again next time you turn camera on if you want the card speed boost.

Not sure if this memory error is normal, or how it was expected to work, but, thats when I get the memory error, if I do it in the right order, all is good.
500D/T1i  550D/T2i  600D/T3i  700D/T5i

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1671
  • 6d - Nightly build user
Re: UHS-I / SD cards investigation
« Reply #172 on: April 15, 2018, 09:17:02 PM »
I have the module standard loaded at startup.
When I want to use it, I switch to photomode(no liveview) and start the module in Debug menu.
Switch back to video mode and ready to use it.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7018
Re: UHS-I / SD cards investigation
« Reply #173 on: April 15, 2018, 09:22:43 PM »
Photo mode did the trick here too :)

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: UHS-I / SD cards investigation
« Reply #174 on: April 15, 2018, 09:34:17 PM »
Yes, using it in Photomode, non live view will def work fine, but, when in Movie mode and Raw enabled somehow, its important to note, the module will cause memory error.

I normally start in Photomode myself, but wanted to point out to the other user about maybe why he was getting the malloc error in case he might have been using movie mode with MLV On.
500D/T1i  550D/T2i  600D/T3i  700D/T5i