UHS-I / SD cards investigation

[Kharak]

@a1ex

I always thought it was the Double Spanning from Canon and ML causing it, but if you say they dont interfere with each other, then I don't know what happened. I set the settings and hit record and the camera crashed and I got an Error code and the card was bust. Never happened before or after, never tried it again and will never try it again

I have not tried h264 to two separate cards, I never even considered it, but just to be clear, you mean Recording H264 to both cards simultaneously, is that an ML feature?. I also couldn't figure out how to set h264 proxy to record to SD while MLV went to CF.

[a1ex]

I've tried it before posting; no surprises. When you record raw video, Canon's LiveView remains idle (their code doesn't even know the record button was pressed, since the raw recorder catches that event and doesn't pass it back to Canon firmware). Remember the LiveView used to be turned off after 30 minutes, since Canon firmware believed the camera was idle (while it was actually recording raw video). For the card selection setting, ML only sees 0 = CF or 1 = SD, there are no other possible values for that property. ML does not (and did not) see the advanced options from Canon menu, like CF+SD.

I thought recording H.264 on two cards is a feature from Canon firmware (after you said you can "set Canon Menu to record to CF+SD").

In crop_rec_4k builds, H.264 goes to the card selected in Canon menu, while raw video (mlv_lite) always goes to CF if there's one inserted. In main builds and in mlv_rec from any builds (regular recording without H.264 proxy), both will go to the card selected in Canon menu. In mlv_rec with spanning enabled, main stream goes to CF and secondary stream goes to SD, regardless of Canon setting.

[reddeercity]

@reddeercity: more details after I'll get a new card (it *is* possible to overclock the 5D2 CF interface).

Really , cool that's better then trying to change to udma 7
Increase the "bus speed" ? can't wait to read the details .
I saw something in the 5d2 disassembly that look interesting

Code: [Select]

"<K218 Board> SystemCLK 132MHz":
looks to be part of the system check before boot loader .

[reddeercity]

search for "cfDecideTiming" in the rom, got this

Code: [Select]

*"cfGetRegisterTiming: I/O 250nS (PIO Mode4)"so "Mode4" =250nS I/O there was also a

Code: [Select]

*"cfGetRegisterTiming: I/O 120nS"but not "Mode"
this got me interested in the CF events , would be nice to have more write speed .

Edit: look like that the slowest after some searching , would need to be around 30 or 20ns for 100MB/s

[reddeercity]

Is this kind of what you can do @a1ex "Double Transition Clocking"
http://www.pcguide.com/intro/fun/clockDouble-c.html
If not it's still interesting reading 

[a1ex]

There is the DDR50 mode for SD cards, easily enabled on the card side from sdSetFunction, but... good luck configuring Canon controller to use that. Brute-forcing the "known" registers didn't help.

You can send ATA commands to the CF card (QEMU emulates them), so if the only difference between UDMA 6 and 7 is timing, it might even be possible to put the card in UDMA7.

[Ant123]

good luck configuring Canon controller to use that. Brute-forcing the "known" registers didn't help.

Why do you think that camera controller has support of DDR mode?
In accordance with debugging messages inside D6 & D7 firmwares the fastest mode is SDR104 and there are no signs of DDR...

[a1ex]

I don't know whether the controller supports DDR or not, but - as you can see in previous logs - there are cards that support DDR50, but not SDR104. That's why I've assumed DDR50 might be an older standard.

[Ant123]

I did not found significant changes in the standard.
Look at the paragraph 3.9.3 of "Physical Layer Simplified Specification" (from v3.01 to v6.00). There are three host types.

[reddeercity]

You can send ATA commands to the CF card (QEMU emulates them), so if the only difference between UDMA 6 and 7 is timing, it might even be possible to put the card in UDMA7.

Ok , yet another reason to get QEMU going

[David_Hugh]

This is insane. It really works, just like that. Did the test on a gold sandisk extreme 32gb and a 70D. This card now effectively records raw at 49Mb/s instead of 40, resulting in a noticeable bump in resolution/aspect ratio. For those saying they dont see the "real world" results, I ran the test in video mode and mlv_rec was already turned on. I dont know whether turning the cam off kills the overclocking for now, or if running it in video mode was what did it, but this combo worked for me. Also, the test showed a siginificantly different result in photo/video mode (53 vs. 43) and although it only showed 43 in video the card writes 49 continous.

I did notice however that the card got quit warm relatively quickly. Again. This is insane. You guys just made every cam that can run this like 10 times better.

[50mm1200s]

@a1ex, since some people got the cards killed by this experiment, wouldn't it be better to remove it from downloading, for now? Some people might use it for real world production and get the card crashed in the middle of the work...
I think it's a good idea, though. I can barely get 16:9 FullHD in crop_rec (50D), with this patch (if it works in CF too) it would possibly make it to continuous FullHD recording.
Do you need any help to get disposable cards that you can "fry" as you want? I could buy some cheap ones for you and send here from Brazil.

[a1ex]

The pre-built module was already rolled back to a hopefully safer version and had a stronger warning, but OK, removed the download. Nobody tested it this week anyway. The diagnostic version (which does only the basic version of the hack on 5D3 and only prints some capabilities on 700D) is still available.

BTW, when N = 1, you may want to use the singular form (i.e. card instead of cards).

[dfort]

I removed my test build too.

[Levas]

Just wanted to note, I'm still heavily testing with this and not messed up my card 
As far as I could tell, only the second build with driver strength options, messed up card(s).

For practical use and testing, I changed the code of the original build a little (removed the line where it starts showing the console text, and removed the tests and only kept the line with 160Mhz settings)
This way, now I can go to photo mode, click the SD overclock in the debug menu and within a second I'm ready to go. (No console text, just a few card reader led blinks)
Last night I photographed a live performance of 2 music bands and also did some video tests with the SD_overclock module.
I used the same SD card the whole time (2,5 hours) and activated the SD_overclock module multiple times, recorded some video and moved on taking photos (with
SD_overclock still activated)
I even recorded, successfully, a 5 minute length clip of 8558 frames in 2560 x 958 resolution in 14 bit lossless...on a 6d that is 
The MLV file was 21Gb in size 

[mk11174]

Same here, working very nice on my 32gig micro SD card, definitely an improvement with no signs of card dieing yet.

[loknar]

As instructed I did some extensive tests of sd_uhs module and:
First of all, thank you guys, this really extends usability of my EOS M far beyond of my expectations.
I've been using two cards, SD Sandisk Extreme 16 GB and uSD Sandisk Extreme 32 GB.
I've recorded over 30 GB of data during 4 hours, cards are OK, camera is OK, Data are OK (I'm using 3rd April dforts build for EOS M and crop rec build for EOS M from 10th of March).
I shot at 2520x1072 with fps override to 24 fps (HighTV) in 12-bit lossless. Resulting Bandwidth around 55 MB/s.
In most of scenes recording could be continuous, but in some settings I've got only around 3-10 seconds, didn't found reason for this (other than maybe complexity of the scene or temperature of the processor).
Also I don't know how it is supposed to behave but in my case it was: new ML installation enable SD hack, benchmark:

Code: [Select]

===================
2018/04/06 22:29:28
===================
Before the hack: r:37MB/s w:34MB/s  W:32MB/s R:37MB/s  :)  [best 34MB/s]
SDR50 @ 96MHz  : r:37MB/s w:32MB/s  W:33MB/s R:37MB/s  meh [best 34MB/s]
SDR50 @ 96MHz  : r:37MB/s w:19MB/s  W:33MB/s R:38MB/s  meh [best 34MB/s]
SDR50 @ 80MHz  : r:32MB/s w:30MB/s  W:28MB/s R:32MB/s  meh [best 34MB/s]
SDR50 @ 80MHz  : r:36MB/s w:34MB/s  W:35MB/s R:37MB/s  :)  [best 34MB/s]
SDR50 @ 120MHz : r:54MB/s w:52MB/s  W:52MB/s R:55MB/s  :)  [best 52MB/s]
SDR50 @ 120MHz : r:54MB/s w:50MB/s  W:52MB/s R:55MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:42MB/s  W:42MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 96MHz : r:44MB/s w:41MB/s  W:42MB/s R:44MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:35MB/s  W:36MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 80MHz : r:37MB/s w:35MB/s  W:35MB/s R:37MB/s  meh [best 52MB/s]
SDR104 @ 120MHz: r:55MB/s w:53MB/s  W:52MB/s R:55MB/s  :)  [best 53MB/s]
SDR104 @ 120MHz: r:54MB/s w:51MB/s  W:50MB/s R:55MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:51MB/s w:48MB/s  W:48MB/s R:51MB/s  meh [best 53MB/s]
SDR104 @ 132MHz: r:50MB/s w:48MB/s  W:48MB/s R:51MB/s  meh [best 53MB/s]
SDR104 @ 160MHz: r:72MB/s w:70MB/s  W:67MB/s R:73MB/s  :)  [best 70MB/s]
SDR104 @ 160MHz: r:72MB/s w:68MB/s  W:64MB/s R:73MB/s  meh [best 70MB/s]

Done.
Please run THOROUGH tests before using!!!Then I could record videos with high bitrate. Once i switched camera off and on again memory patch wasn't (obviously) applied, so i tried to enable SD hack again, but i've got this:

Code: [Select]

===================
2018/04/10 08:29:10
===================
Before the hack: malloc error
 [best 66MB/s]
SDR50 @ 96MHz  : malloc error
 [best 66MB/s]
SDR50 @ 96MHz  : malloc error
 [best 66MB/s]
SDR50 @ 80MHz  : malloc error
 [best 66MB/s]
SDR50 @ 80MHz  : malloc error
 [best 66MB/s]
SDR50 @ 120MHz : malloc error
 [best 66MB/s]
SDR50 @ 120MHz : malloc error
 [best 66MB/s]
SDR104 @ 96MHz : malloc error
 [best 66MB/s]
SDR104 @ 96MHz : malloc error
 [best 66MB/s]
SDR104 @ 80MHz : malloc error
 [best 66MB/s]
SDR104 @ 80MHz : malloc error
 [best 66MB/s]
SDR104 @ 120MHz: malloc error
 [best 66MB/s]
SDR104 @ 120MHz: malloc error
 [best 66MB/s]
SDR104 @ 132MHz: malloc error
 [best 66MB/s]
SDR104 @ 132MHz: malloc error
 [best 66MB/s]
SDR104 @ 160MHz: malloc error
 [best 66MB/s]
SDR104 @ 160MHz: malloc error
 [best 66MB/s]

Done.
Please run THOROUGH tests before using!!!Strangely enough, after this memory patch have been applied and i could continue recording with high bitrate.
I wanted to ask if is it possible to streamline usage of the hack (like add menu "apply saved SD clock values", or to avoid opening console), but since you retracted the module, i don't know whether you intend to explore this more. (And as far as i'm concerned i'm not giving this module back )

If you are interested in result, i posted it on youtube

[ahmedvienna]

I am trying to implement sd_uhs on my 6d. using the latest nightbuild.  have an error when I do it.  can anyone assist me.
here's the pic of the error.

https://pbs.twimg.com/media/DabZ9QVWsAA7zD-.jpg:large

thanks !
Ahmed

[Danne]

Wow, quite something. My Sandisk extreme pro 95mb/s shows 69mb/s in the test. Runs nicely on my 100D. Thanks Levas for feedback.
Regarding malloc error on initial testing I run the module with liveview closed. When opened malloc errors..

[dfort]

Looks like loknar, Danne, mk11174 and Levas are on the Plateau of Productivity with the sd_uhs module while some of us are stuck in the Trough of Disillusionment.



@Levas, looks like you got a good working version. Could you share the changes you made on the module?

[Levas]

This stuff is too good not to be used, thanks Alex 
This and the lossless option, makes continuous recording available for the SD cams 

The build I use is based on Alex first build, I only deleted a few lines, didn't add any. (I'm not that savvy in programming, but I had to have something more practical)
I deleted one line with 'show console' and I deleted a bunch off the lines where most of the settings are tested.
I only kept the highest 160Mhz setting in it.

Here's the altered source file I'm using:
https://drive.google.com/drive/folders/1A_OVNAucbHYfUXfPT9kxBy1IiUk-012U?usp=sharing

[mk11174]

Then I could record videos with high bitrate. Once i switched camera off and on again memory patch wasn't (obviously) applied, so i tried to enable SD hack again, but i've got this:

Code: [Select]

===================
2018/04/10 08:29:10
===================
Before the hack: malloc error
 [best 66MB/s]
SDR50 @ 96MHz  : malloc error
 [best 66MB/s]
SDR50 @ 96MHz  : malloc error
 [best 66MB/s]
SDR50 @ 80MHz  : malloc error
 [best 66MB/s]
SDR50 @ 80MHz  : malloc error
 [best 66MB/s]
SDR50 @ 120MHz : malloc error
 [best 66MB/s]
SDR50 @ 120MHz : malloc error
 [best 66MB/s]
SDR104 @ 96MHz : malloc error
 [best 66MB/s]
SDR104 @ 96MHz : malloc error
 [best 66MB/s]
SDR104 @ 80MHz : malloc error
 [best 66MB/s]
SDR104 @ 80MHz : malloc error
 [best 66MB/s]
SDR104 @ 120MHz: malloc error
 [best 66MB/s]
SDR104 @ 120MHz: malloc error
 [best 66MB/s]
SDR104 @ 132MHz: malloc error
 [best 66MB/s]
SDR104 @ 132MHz: malloc error
 [best 66MB/s]
SDR104 @ 160MHz: malloc error
 [best 66MB/s]
SDR104 @ 160MHz: malloc error
 [best 66MB/s]

Done.
Please run THOROUGH tests before using!!!Strangely enough, after this memory patch have been applied and i could continue recording with high bitrate.
I wanted to ask if is it possible to streamline usage of the hack (like add menu "apply saved SD clock values", or to avoid opening console), but since you retracted the module, i don't know whether you intend to explore this more. (And as far as i'm concerned i'm not giving this module back )

I only get memory error if I am in movie mode with MLV-Lite or MLV-REC turned ON, make sure these modules are off before you run this card speed module, once it is working, then ok to turn ON the MLV modules you want to use.

Obviously if you turn off camera, you will need to go through same procedure in same order again next time you turn camera on if you want the card speed boost.

Not sure if this memory error is normal, or how it was expected to work, but, thats when I get the memory error, if I do it in the right order, all is good.

[Levas]

I have the module standard loaded at startup.
When I want to use it, I switch to photomode(no liveview) and start the module in Debug menu.
Switch back to video mode and ready to use it.

[Danne]

Photo mode did the trick here too

[mk11174]

Yes, using it in Photomode, non live view will def work fine, but, when in Movie mode and Raw enabled somehow, its important to note, the module will cause memory error.

I normally start in Photomode myself, but wanted to point out to the other user about maybe why he was getting the malloc error in case he might have been using movie mode with MLV On.