New 240 MHz preset for 5D3 / EOS M / 100D (Download available)

[theBilalFakhouri]

Edit 20/9/2022:

Latest sd_uhs which works perfectly on 5D3 / EOS M / 100D and other DIGIC 5 models can be found in:

SD Overclocking - DIGIC 5 only

Info 1
Info 2



Original OP: (keeping it for documenting some of the info is outdated)

Hi!

One of my goals was to get stable 240 MHz SD overclocking preset across all DIGIC 5 models, so I was digging into SD stuff again for few weeks. and Thanks to 100D (where 240 MHz doesn't work) I could compare it directly to my 700D (where 240 MHz does work flawlessly).

I did find new things (related to SD configuration) which helped to move 240 MHz on 100D from "it doesn't work at all" to it works partially, but not really usable.

After a lot of tests I noticed a pattern, and I thought . . and came up with a magic trick . . current state with the Magic Trick:

-5D3:

1. So far managed to record 512 GB of data at ~86 MB/s write speed (hacks enabled) with 240 MHz SD overclocking using Sandisk Extreme PRO UHS-I U3 170 MB/s SD card without problems.

2. We don't need to worry anymore about read/write operations while running overclocking process, I found a way to pause read/write operations --> Which mean better stability while overclocking and "Keep ML" option will work fine after formatting SD card in camera.


-100D / EOS M:

1. You will be able to record around **4 Minutes and 30 Seconds on average at ~80 MB/s write speed (hacks enabled), after that it will drop to 21 MB/s (switch to 48 MHz).

2. That's better than nothing.

3. On one try I could fill 64 GB SD card with 240 MHz at ~80 MB/s write speed without dropping to 21 MB/s on 100D.

4. After some more tests I noticed that when SD card becomes hot, the drop to 21 MB/s begin to happen, example (I think ambience was around ~31° C):

● First try I could record around 5 minutes then it dropped to 21 MB/s
● I restarted the camera and got another ~5 minutes then it dropped to 21 MB/s (SD card is hot at this stage)
● I restarted the camera again, this time recording was limited to ~3 minutes then it dropped to 21 MB/s
● I restarted the camera again, recording time was limited to ~2 minutes then it dropped to 21 MB/s
● I restarted the camera again, recording time was under 2 minutes then it dropped to 21 MB/s

High SD card/SD controller temperature seems a stability factor here for 100D and probably the same thing for EOS M.
To verify: I put 100D inside the Fridge, I managed to fill 64 GB SD card twice at 240 MHz @ ~80 MB/s without problems (~25 minutes of recording), it didn't drop to 21 MB/s.

I don't think the heat is the only factor for this issue, it might be there a some related SD configuration tweaks which we are not aware of might solve this problem,
I mean solves instability at 240 MHz at high temps. I doubt if it a hardware limit. And it seems only cams with shared Card/Battery door are effected by this issue.

5. Bonus: Also the new things I found allows some SD card to work at higher OC, my Sandisk Extreme PRO 95 MB/s U3 UHS-I now works on 100D at 192 MHz (instead of 160 MHz).
   I am expecting that more cards should work in general at higher OC for 100D/EOS M, no guarantee though.

**It seems to depends on SD controller/card temps!


-It's not for free:
Yes, you have to pay in order to get it.

Reverse engineering work takes a lot of hours and effort, *initially I wanted ML users to give back their support to ML project in someway since they are getting our support for free.
My other ML goals are to work on lossless, SD overclocking, Crop modes on a DIGIC 4 model . . which is 600D, and give some of my support to DIGIC 4 models.

The price is based on averaged used 600D cost here (in my country).

*Yeah, that was my initial plan a month ago, but things changed a little. Unfortunately I started to loss my energy on ML project, I still don't know if this a temporary thing.
But I am still selling 240 MHz at 600D cost, I might not buy a 600D or might buy it, take a look into it then sell it again.

Selling forks under GPL is completely normal, getting a fair price for a dev efforts is okay too.

-Price:
Total of 265 USD (Edit: Paid)
Download available

-How to pay:
● Contact me here on the forum by sending a PM or on Discord (@thebilalfakhouri#1803) or on Reddit, Telegram, Facebook, Instagram or Twitter.
● PayPal and Cryptocurrencies are accepted.
● 265 USD can be split across all interested users or supporters.  When the total is reached, all code will be released to everyone
● I will update the collected amount frequently after each payment.


-Notes:
● Tests were done using Sandisk Extreme PRO U3 UHS-I 170 MB/s
● There is no 100% guarantee of success, don't blame me
● No refund
● Both source code and the new sd_uhs.mo would be released after we collect the full amount (265 USD)
  You will have to wait after you pay . . until we reach the goal, nothing would be released if we didn't reach the goal.
  What if there were no enough buyers? Let's wait 30 days at least. Then we will talk.

● The trick I used could be risky, and might damage SD card or SD controller . . but to be fair, I don't know. So far I didn't notice issues.
● The new 240 MHz preset has been tested by me (100D/5D3), ilia3101 (5D3/EOS M) and Walter (EOS M). Their reports were similar to as I described in "current state" above.

● Will you need a fast CF card anymore for 5D3? --> you might save some money by relying on SD cards and slower CF cards.
● Worst case scenario if new 240 MHz failed on 5D3 during recording, the card will become inaccessible and you might the lose the recorded clip. Until now that didn't happen to me (managed to record 512 GB) of MLV clips without issues.
   More tests are needed by the community (by you) to verify that on 5D3.

-Old vs New sd_uhs :

Old:  192 MHz and 240 MHz doesn't work on 5D3, you are limited to 160 MHz (~63 MB/s write speed).
         240 MHz doesn't work on EOS M / 100D, you are limited to 192 MHz (~70 MB/s write speed).

New: 192 MHz and 240 MHz does work on 5D3, 240 MHz goes up to ~86 MB/s write speed.
         240 MHz does work on EOS M / 100D (as described above), 240 MHz gives ~80 MB/s write speed.


Currently I am away from cameras, will post demo videos soon! (for 5D3/100D).

Feel free to ask questions, chat and throw suggestions!


-Collected amount:
  265 USD from 12 people.

-Remaining amount:
  0 USD.

-Goal:
  265 USD.

Goal reached!


Download:

The compiled version of the new module + source code have been posted in Reply #1:

TAKE ME THERE!

Happy testing!

Results:
Users reports recap

[theBilalFakhouri]

Credits:

Thanks to to buyers who helped to bring the new module to everyone:

Code: [Select]

TrincaEspinhas
gabriielangel
Danne
vastunghia
David Khoirul
notthesame
AceCom
Grognard
ilia3101
revelations
SebastianC
th0ma5
-Special thanks to ilia3101 for handling PayPal!


The module:

Source code and compiled version of the module can be found here:

New_240_MHz_and_sd_uhs.zip (Outdated)

Latest one + source code in SD Overclocking - DIGIC 5 only topic

Important infos in first post:
Take me to the first post



Details:

1.New GPIO registers:

Explaining will be added soon!
(Edit: Explaining was added in 9-1-2023)

These registers seem to be related to SD controller registers?
Their values change depending on clock speed in the function which toggle among default clock speeds e.g. like 48 MHz and 96 MHz (on 700D the function is ff337b78).

on 650D/700D these registers change from:

Code: [Select]

(used in default 96 MHz)
0xC022C634              = 0x577
0xC022C638 and others   = 0x544

to

(used in defualt 48 MHz)
0xC022C634              = 0x555
0xC022C638 and others   = 0x555
I called these registers GPIO because that what QEMU calls them, which are:

Code: [Select]

0xC022C634 (Has either same value as the rest of registers or has a value higher than them)

0xC022C638 (This register and below always share the same value)
0xC022C63C
0xC022C640
0xC022C644
0xC022C648
all of the following values are default values from Canon in default 96 MHz preset, except for 5D3 default clock is 48 MHz):

650D/700D:
0xC022C634                 = 0x577
0xC022C638 and others = 0x544

6D: all of them 0x555

70D:
0xC022C634                 = 0x566
0xC022C638 and others = 0x555

100D:
0xC022C634                 = 0x655 (240 MHz never worked with these values)
0xC022C638 and others = 0x733

EOS M:
0xC022C634                 = 0x734
0xC022C638 and others = 0x733

5D3: (tweaking them didn't have an effect on 5D3?)
0xC022C634                 = 0x755
0xC022C638 and others = 0x755

If you look closely in DIGIC 5 models where 240 MHz works out of the box like 650D/700D/6D/70D, you will see the good GPIO range for high OC (like 240 MHz) is around 0x5XX.

in source code I called:
0xC022C634 as                  GPIO1
0xC022C638 and others as  GPIO2

for my 100D:
0x599 for GPIO1, 0x555 GPIO2 worked, and it seems the best for 240 MHz preset.

0x588 for GPIO1, 0x555 GPIO2 works too on 100D
0x599 for GPIO1, 0x566 GPIO2 works too on 100D
0x577 for GPIO1, 0x544 GPIO2 (700D values) don't work very well.

Without tweaking these registers, 240 MHz never been stable on 100D/EOS M, so they are required and they are a part of SD puzzle.

2.The Magic Trick:
How the trick works:

On 5D3, 240 MHz is stable but only with SD write operations, as soon as the camera does read operation it make the card inaccessible (Block Transfer error), I made tests around a year ago on 5D3 and made sure and the problem was instability only with read operations (I wrote this info in forum, nothing new).

Same applies e.g. on 100D (and other models at some settings), when running benchmarks, write benchmark would be completed normally, then when benchmarks start reading test it either switch to 21 MB/s or make the card inaccessible.

The trick is to switch to 160 MHz before reading operation happens, then switch to 240 MHz before write operation happens, so whenever card is reading the clock speed would be 160 MHz (has stable read speed), and whenever card is doing writing it would be 240 MHz because the write speed is stable at 240 MHz, that's for 5D3. For EOS M / 100D same thing except for using 192 MHz for read operations.

No idea if it risky, well, in general the CPU changes its clock speed all the time, no idea if it comparable to storage clock speed.
I am wondering if some of SD card readers does that too for some cards.

During my tests, write speed performance was very stable, didn't notice any side effects, it seems the trick works flawlessly.

Think about it as downclocking, what if we used default 96 MHz (~40 MB/s), and underclocked it for read operations to 48 MHz (21 MB/s), would you think it's risky?
I don't think so, but who knows.

[Danne]

Nice work! Will donate soon 

[vastunghia]

Definitely going to pay.

Tbh as a very personal (and somewhat marketing) note I would be encouraged to pay more if I knew that this may cover future development of clean HDMI output on 5D3… that would make the 5D3 a really complete and powerful RAW camera. Why DIGIC 4? But hey, it is a matter of personal interest!

So again, going to pay.

Bilal, are you planning to provide a frequent update on the progress of the campaign? I think that would be helpful.

Thanks

Sergio

[gabriielangel]

Hello, good idea and I will donate some as soon as I get your Paypal address.

Like vastunghia, if some of the Devs pledge to tackle Centering of presets, HDMI and other Hard to do fixes on EOS M, I would definitely chip in again (whether they succeed or not).
I think it is a good initiative, since all the devs here have been dedicated, they deserve a little something for their time, test gear, etc.

[theBilalFakhouri]

Thanks folks!

Tbh as a very personal (and somewhat marketing) note I would be encouraged to pay more if I knew that this may cover future development of clean HDMI output on 5D3… that would make the 5D3 a really complete and powerful RAW camera. Why DIGIC 4? But hey, it is a matter of personal interest!

Please note the payment will go only for new 240 MHz preset at its current state!
Also, please don't pay if you are expecting other stuff.. Sorry, currently I don't have plans to work on 5D3 on other aspects.

off-topic but 5D3 already have clean HDMI output using 1.2.3 firmware, what are you looking for?
Maybe you referring to real-time preview at high resolution preset?

Re DIGIC 4: I wanted to work on it around two years ago for many reasons, it's not a really new interest . . working on certain model or features depend on dev inspiration, and on many factors

Bilal, are you planning to provide a frequent update on the progress of the campaign? I think that would be helpful.

Of course! already wrote that in OP:

-How to pay:
...
● I will update the collected amount frequently after each payment.

[theBilalFakhouri]

Like vastunghia, if some of the Devs pledge to tackle Centering of presets, HDMI and other Hard to do fixes on EOS M, I would definitely chip in again (whether they succeed or not).
I think it is a good initiative, since all the devs here have been dedicated, they deserve a little something for their time, test gear, etc.

It's a good idea yeah to support and say thanks to dev for their work (speaking in general).
Unfortunately I don't have new plans for working on other aspects for now, perhaps other devs might be interested.

off-topic:
On 100D/650D/700D using my build all crop modes is centered on sensor. There is a Hardware limit for centering RAW image on CMOS, so it's not 100% perfect but very close to center.
On EOS M I don't know, it should be very easy to adjust centering of crop modes presets there. If it's reached the limit, then nothing to do.

[vastunghia]

Please note the payment will go only for new 240 MHz preset at its current state!

Duly noted, thanks for clarifying.

Maybe you referring to real-time preview at high resolution preset?

Correct. HDMI out is pretty unusable with 5D3 in crop modes.

Of course! already wrote that in OP:

Sorry I missed that part.

Thank you!

[theBilalFakhouri]

Update: 10 USD have been collected from one person.

[gabriielangel]

It's a good idea yeah to support and say thanks to dev for their work (speaking in general).
Unfortunately I don't have new plans for working on other aspects for now, perhaps other devs might be interested.

off-topic:
On 100D/650D/700D using my build all crop modes is centered on sensor. There is a Hardware limit for centering RAW image on CMOS, so it's not 100% perfect but very close to center.
On EOS M I don't know, it should be very easy to adjust centering of crop modes presets there. If it's reached the limit, then nothing to do.

It was only a general suggestion. I'll donate for the 240MHz, no problem. Don't forget to msg me your paypal address.

[theBilalFakhouri]

@gabriielangel

Your suggestion and any suggestions are welcome, I just talked about myself and described my current state regarding ML.
Even if things got a little off-topic, that's okay for this thread, we can move to other threads if you want to continue the discussion.

PayPal address has been sent.

[70MM13]

excellent news!
regarding the 5d3, was any testing done to determine if the maximum recording bitrate using card spanning and overclock increases beyond the "limit" we found when overclocking to 160MHz?

[theBilalFakhouri]

@70MM13

Thanks!

Nope, write speed limit with Card Spanning is still about ~133 MB/s in LiveView with hacks enabled. Higher SD OC like 192 MHz and 240 MHz won't improve this if you already reached ~133 MB/s limit.
But due to faster SD card bandwidth (@ 240 MHz), the write speed almost would be splitted equally among CF and SD, ~50% of total write speed will go to CF and ~50% will go to SD, which means both cards would fill almost equally.

[theBilalFakhouri]

Update: 21 USD have been collected from two people.
Remaining amount: 244 USD.

[Grognard]

 That could seems strange to pay for a ML feature. But we had already asked to find a way to help ML devs! So I Will contribute for sure to thank you for what you achieved Bilal even if this feature will not be very useful for me.

I know you don't want to work on 5D3 to unblock correct framing liveview. but if you change your mind, you could ask money for the price of a new R5

thank you.

[Dmytro_ua]

I know you don't want to work on 5D3 to unblock correct framing liveview. but if you change your mind, you could ask money for the price of a new R5

That's for sure!! 

[vastunghia]

So I Will contribute for sure to thank you for what you achieved Bilal even if this feature will not be very useful for me.

Exactly the same here. I must confess that Bilal's achievement at first gave me the false hope of being able to achieve overall (CF+SD) higher write speeds... but I will pay nevertheless, even though the feature does not seem to be very relevant to me (it may even turn out to be counter-productive actually, as I currently have a 256 GB CF card and a 128 GB SD card, so my optimal recording ratio is 2/3 on CF and 1/3 on SD, which is approximatively the one I'm achieving now w/o SD overclock.)

Nope, write speed limit with Card Spanning is still about ~133 MB/s in LiveView with hacks enabled.

Mmh in my experience I can reach far higher rates than 133 MB/s with Global Draw = OFF. Check the following shot:



152 MB/s as in the above image is pretty on the edge, very close to automagical stop, but I think something closer to 145 MB/s is definitely compatible with continuous recording on my 5D3. Should check better with the latest chunk of MLV files I took during August holiday trips, but I'm pretty sure.

Is this higher rate I'm observing simply enabled by the deactivation of Global Draw? So once again, just to be sure, is it confirmed that there is no effective write speed enhancement expected to come from the enabling of SD overclock?

I know you don't want to work on 5D3 to unblock correct framing liveview. but if you change your mind, you could ask money for the price of a new R5

+1 here!

Sergio

[Walter Schulz]

The numbers I get during recording on EOS M (and other cams) are misleading.
I recommend to use actual file numbers and recording durations to determine actual data write rates instead. Because lossless compression may cause a high variance I recommend to use non-lossless modes for benchmarking.
And because buffering may have some influence, too: Just recordings durations 4 minutes or above for good measurement.

[theBilalFakhouri]

That could seems strange to pay for a ML feature.

Is it strange or we are not used to it . many open source projects actually do that.
BTW, ML before 2013 was charging for new builds releases.

But we had already asked to find a way to help ML devs! So I Will contribute for sure to thank you for what you achieved Bilal even if this feature will not be very useful for me.

Thanks!

I know you don't want to work on 5D3 to unblock correct framing liveview.

I don't want? Nope!

I already tried many times during the last year on 5D3 but without success, improving preview in general is one my goals for DIGIC 5 models, that's why I started LiveView Investigation two years ago. LiveView is really complex, working on it requires a lot of time and effort and also some good experience. I might get back to it at some point, no idea when. or may never get back, I don't know currently . .

..you could ask money for the price of a new R5

R5 sounds cool
I was thinking about R7 or R10 for a price of preview, R5 sounds too much

That's for sure!! 

+1 here!

Holy $h!t! getting a R5 would be an enough reason to let me leave ML, why would you do that?
Thanks for your support!

[vastunghia]

Holy $h!t! getting a R5 would be an enough reason to let me leave ML, why would you do that?

C'mon dude, you would start porting Doom to the R5 as soon as you had it in your hands!

[theBilalFakhouri]

I must confess that Bilal's achievement at first gave me the false hope of being able to achieve overall (CF+SD) higher write speeds.

When using Card Spanning (with fast cards), we will hit Memory (RAM) limit:

in 5D3, 240 MHz overclock used for SD, Play mode:
My CF benchmarks ~120 MB/s Write
My SD benchmarks ~90 MB/s Write

So in theory with Card Spanning it should be 90 + 120 = 210 MB/s but that won't happen if you run CF + SD benchmarks, total write speed with card spanning benchmarks would be limited to ~150 MB/s.

In LiveView it's around ~133 MB/s.

My CF: Sandisk Extreme PRO 64GB 160 MB/s UDMA7
My SD: Sandisk Extreme PRO 64GB UHS-I U3 170 MB/s

152 MB/s as in the above image is pretty on the edge, very close to automagical stop, but I think something closer to 145 MB/s is definitely compatible with continuous recording on my 5D3.

Reported write speed during recording isn't accurate, I don't rely on it, it will always be higher in most cases.
And accurate way to measure max write speed in LiveView is by running benchmarks with hacks enabled, already did that in LiveView hacks (write speed improvement), here.

[theBilalFakhouri]

Update: 95 USD have been collected from four people.
Remaining amount: 170 USD.

Huh, 95 and 170 . . remind me of something

[theBilalFakhouri]

Note: It's not 265 USD per user, but it's total of 265 USD for one time can be paid by by multiple interested users or supporters. You only need to pay a part of 265 USD, not all of it.
Check the end of first post for current state:

-Collected amount:
  95 USD from four people.

-Remaining amount:
  170 USD.

-Goal:
  265 USD.

[koopg]

will join effort tomorow, whats the atatus of it anyway?
and i dont understand, we pay but wint be able to het it untill 265 reached ?

Sent from my SM-N975F using Tapatalk

[theBilalFakhouri]

will join effort tomorow, whats the atatus of it anyway?

Thanks, status of what exactly?

and i dont understand, we pay but wint be able to het it untill 265 reached ?

I didn't get your question . . but nothing would be released until we reach 265 USD goal. If you paid and if we didn't hit the goal yet, you will have to wait.
Already wrote that in first post.

Do you mean something else?