Magic Lantern Forum

Experimental builds (WIP) => Other experimental builds => Topic started by: theBilalFakhouri on February 12, 2021, 10:32:15 PM

Title: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on February 12, 2021, 10:32:15 PM
(New OP 21/9/2022)

Experimental SD Overclocking module with selectable presets (160/192/240 MHz) for DIGIC 5 models only (5D3 1.13 / 5D3.123 / 6D.116 / 70D.112 / 700D.115 / 650D.104 / 100D.101 / EOSM.202 / EOSM2 1.0.3).

Download (https://bitbucket.org/bilal_fakhouri/magic-lantern/downloads/sd_uhs_compiled.zip) / Source code (https://github.com/bilalfakhouri/magiclantern_hg_02/tree/crop_rec_4k_mlv_snd/modules/sd_uhs) / Technical discussion (https://www.magiclantern.fm/forum/index.php?topic=12862.0)

(https://i.ibb.co/WWPYcJL/Menu1.png) (https://i.ibb.co/HGbT2FD/Menu2.png)




Stock 700D160MHz192MHz240MHz
(https://i.ibb.co/TTBssHV/Stock.png)(https://i.ibb.co/8b8P4s9/160MHz.png)(https://i.ibb.co/KF9jxMH/192MHz.png)(https://i.ibb.co/kx9B5Sm/240MHz.png)


How to:
-You need a build based on crop_rec_4k branch (Experiments (https://builds.magiclantern.fm/experiments.html) / crop_rec builds (https://www.magiclantern.fm/forum/index.php?board=66.0))
-Download sd_uhs_compiled.zip (https://bitbucket.org/bilal_fakhouri/magic-lantern/downloads/sd_uhs_compiled.zip), then extract it somewhere on PC
-Copy sd_uhs.mo to your SD card in "ML/modules" folder
-After loading the module in camera, select a preset (160/192/240) then restart the camera
-Run "Benchmarks" test from Bench.mo in Photo mode
-Share the result, your card name, selected preset.

SD card recommendations:

-Any SD card with W90 label (capable of 90 MB/s write speed) should work with 240 MHz or lower.
-List of SD cards which were tested and they work fine at 240 MHz (pay attention to SD card capacity, lower capacities may not be capable of 90 MB/s write speed):

-SD cards which doesn't work well at 240 MHz:

Notes:
-SDR104 is selected by default which should work on all fast SD cards (UHS-I W90). Only try SDR50 when SDR104 fails. Like with Sandisk Extreme PRO 95 MB/s (https://www.magiclantern.fm/forum/index.php?topic=26634.msg240421#msg240421) case (https://www.magiclantern.fm/forum/index.php?topic=26634.msg240433#msg240433).

(https://i.ibb.co/3F6DrNt/Access-Mode.png)


-Some models will have "(H)" beside some presets which indicates that Hybrid clock speed will be used; clock speeds for read operations will be different (slower) from the one for write operations.
This won't affect write speed performance. Read speed benchmark will not represent the actual read speed for an SD card at an selected frequency (only in (H) case).

(https://i.ibb.co/KF4VDD9/Hyprid-clock-speed.png)


-Some SD info can be found in "Show CID info" option.

(https://i.ibb.co/gmmLhY6/CID-1.png) (https://i.ibb.co/pL9byqN/CID-2.png)


-Perform "Low Level Format" in camera if you had speed drops, run benchmarks again.

Before (https://www.magiclantern.fm/forum/index.php?topic=26634.msg240433#msg240433):                                                                                                         After (https://www.magiclantern.fm/forum/index.php?topic=26634.msg240439#msg240439):

(https://i.ibb.co/YhN4y01/bench0.png) (https://i.ibb.co/GFyPhQm/bench0.jpg)


Title: Re: SD Overclocking - DIGIC 5 only
Post by: Danne on February 12, 2021, 11:30:41 PM
Eosm max out on 192Mhz, 5D3 no more than 160Mhz if I recall correctly.
I also don´t trust the same patch routines here for 5D3. I do other stuff when patching 5D3 so beware.
https://bitbucket.org/Dannephoto/magic-lantern_dannephoto_git/src/master/modules/sd_uhs/sd_uhs.c
Title: Re: SD Overclocking - DIGIC 5 only
Post by: ArcziPL on March 01, 2021, 11:36:38 PM
Confirmed: 70D can do 240MHz as well.

(https://i.postimg.cc/dtCnKkYT/70d-240-MHz-bench.png)

The card was Sandisk Extreme PRO 170MB/s 64GB.
With the older 95MB/s version of the card the 192MHz preset works well (results like on 700D) but the 240MHz falls back to 20MB/s.

Thumbs up Bilal! Great achievement!
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on March 02, 2021, 12:39:36 AM
Nice results! Thanks for testing!

Could you do one more test (Benchmarks) with Sandisk Extreme PRO 170MB/s @ 240 MHz in Video Mode (1080p24 from Canon menu), in that mode on 700D the max write speed ~68 MB/s .

I'd like to see what 70D would perform, it has DIGIC 5+ (700D has DIGIC 5) and more RAM (maybe also faster RAM than 700D).
Title: Re: SD Overclocking - DIGIC 5 only
Post by: ArcziPL on March 02, 2021, 12:53:16 AM
(https://i.postimg.cc/fLNPScfX/70d-240-MHz-bench-mv1080.png)
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on March 02, 2021, 11:58:10 AM
Thanks!

Cool results! 70D could handle continuous recording in higher resolutions better than the entry level cameras like 700D and EOS M.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: mynamesriz on May 12, 2021, 11:03:41 AM
Hi there,
I'm trying to figure out how to get this working on my 70D, for some reason only ArcziPL's mar19 build is working on my model, his newer builds aren't. Interestingly it worked for a week and then randomly stopped.

(Heads up I don't know ASM so I'm so sorry if this is a stupid question but I guess we've all got to start somewhere and I'm really hoping I can contribute to the community)


I'm struggling to figure out a few things:


Also If you have any recommendations on resources that would help me understand the above I'd really appreciate it.

Thanks in advance.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on May 12, 2021, 11:26:36 AM
Hello @mynamesriz

Strange. Have you tried another 170 MB/s Sandisk Extreme PRO card? Also try to do "Low level format" in camera, make sure there is no important data in the card.

I am not very familiar with 70D, so can't make/do tests, but ArcziPL's build should work fine . .
Have you tried sd_uhs.mo posted in this thread (https://www.magiclantern.fm/forum/index.php?topic=25841.msg234228#msg234228)? download it and put it in ML/modules folder in SD card (you may want to replace it, use latest ArcziPL's build), 192 MHz and 240 MHz should work fine, 160 MHz may not work.

AFAIK sd_uhs.mo in this thread is identical to sd_uhs.mo in ArcziPL's build except for 160 MHz preset (mine doesn't work in 70D, but ArcziPL's 160 MHz version works, I need to update it . .), Anyway try 192 MHz and 240 MHz, do some benchmarks in mv1080 and report back.

These registers founded by looking in QEMU and camera startup LOGs, we don't know what exactly these registers mean, overclocked values founded initially by brute-forcing then by trial and error for 192 MHz and 240 MHz presets.

Long answer, you may want to read in this thread:
https://www.magiclantern.fm/forum/index.php?topic=12862.msg185968#msg185968
Title: Re: SD Overclocking - DIGIC 5 only
Post by: mynamesriz on May 12, 2021, 12:36:23 PM
Thanks @theBilalFakhouri

I've actually tried two 170MB/s SD cards a 64GB and 128GB both with the same result.

I just tried again:
It's a similar story for 192Mhs preset.

Tried the same as above with your module with the same results.

And thanks I'll take a look at that link. :)


edit: i'm actually getting memory allocation errors in mv1080 read benchmarks, so about 20MB/s writes mem alloc error on read
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on July 14, 2021, 03:14:36 AM
Quote from: Danne on February 12, 2021, 11:30:41 PM
I also don´t trust the same patch routines here for 5D3. I do other stuff when patching 5D3 so beware.
https://bitbucket.org/Dannephoto/magic-lantern_dannephoto_git/src/master/modules/sd_uhs/sd_uhs.c

Correct.



-New version:
Added 5D3 sd_uhs patching approach from Danne, added original 160 MHz preset from a1ex (70D didn't work with my 160 MHz preset) now 160 MHz should work on 70D.

Same download links above.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on July 14, 2021, 03:17:31 AM
@mynamesriz

192MHz and 240MHz presets might be unpractical for 70D, same for 5D3 . . nothing to do currently.

Quote from: mynamesriz on May 12, 2021, 12:36:23 PM
...
...
edit: i'm actually getting memory allocation errors in mv1080 read benchmarks, so about 20MB/s writes mem alloc error on read

Turn OFF RAW video before running benchmarks in LiveView.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on August 26, 2021, 10:45:05 AM
-New version:
Added an option to turn ON/OFF SDR104 patch, highlight "SD Overclock" and press Q button.
You can test SD overclock presets with and without SDR104 patch, if you found any difference then report back.

-After you turn ON/OFF SDR104 patch, you have to restart the camera to apply the new setting.
-This option is on by default (in all previous version of sd_uhs it was on).

(https://i.ibb.co/SfbD3gq/SDR104-Patch.png)

Same download links in the first post (https://www.magiclantern.fm/forum/index.php?topic=25841.msg234228#msg234228).
Title: Re: SD Overclocking - DIGIC 5 only
Post by: z56667895 on September 02, 2021, 07:50:20 PM
Any setting errors cause the writing speed to slow down?

My RAW recording can only record about 2 seconds in any mode

I don't know what caused it
My card is Sandisk Extreme PRO 170MB/s 128G

Test the memory card on the computer is no problem

Sorry i don't know how to upload pictures
So I used other websites to upload


https://imgur.com/I24YlVt
https://imgur.com/pNa4VuS
https://imgur.com/undefined
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Dmitriy84759 on September 24, 2021, 07:02:36 PM
Is it possible to overclock 5dm3 SD-card with this hint or not?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on September 24, 2021, 08:42:01 PM
Quote from: Dmitriy84759 on September 24, 2021, 07:02:36 PM
Is it possible to overclock 5dm3 SD-card with this hint or not?

Best way to find out is to make a test, anyway:

Yes you can, 160 MHz is stable with Sandisk Extreme PRO UHS-I U3 95/170 MB/s cards. (Don't format SD card when the overlock is on)
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on September 21, 2022, 02:20:39 PM
Updated the first post with a new sd_uhs version:

- Contain latest Universal sd_uhs code
- Added show CID info option
- Little fixes and modifications

Check it out (https://www.magiclantern.fm/forum/index.php?topic=25841.msg234228#msg234228)

- Test "Show CID info" on all models (on 5D3 / 100D / 700D it seem to work fine)
- The module should work fine, report back if there is regression
- Test the module on EOS M2 1.0.3
Title: Re: SD Overclocking - DIGIC 5 only
Post by: GullRaDriel on September 21, 2022, 07:04:53 PM
Working fine on my 5D3.
I can see a difference between OFF/160/192/240, but no difference between SD50 & SD104 modes.
Looks like at best that Lexar card can only hit up to 72MB, but now nearly constantly.
At least I'm now able to record using 3k5 centered live view with sound continously :-)

Quote(Don't format SD card when the overlock is on)
I have no problem doing so on my setup, overclocked @240 MHz (yes, I re enabled it in my copy of Danne's source)
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on September 21, 2022, 07:16:08 PM
@GullRaDriel

You don't need to change "Access Mode" if SDR104 is working fine with your SD card (already wrote when to change this option in first post). This setting will not affect clock speed.
Formatting SD card with "Keep ML" option with overclocking ON is no longer a problem on 5D3, I have solved this issue in the new versions of sd_uhs :)

Thanks for the test!
Title: Re: SD Overclocking - DIGIC 5 only
Post by: GullRaDriel on September 21, 2022, 09:48:01 PM
You're welcome :-)
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on September 21, 2022, 09:54:31 PM
Working Fine on my EOS M, but I had to use it with your Jul 31 Build to see the CID (Otherwise it showed a bunch of zeroes). Maybe it needed another restart?

1- Is is ok to post the show CID Info on the forum?
2- Is there a way for me to get the error messages displayed on screen to a file? It used to work a while back, but I don't know how I got it to do it back then.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on September 21, 2022, 10:04:07 PM
@gabriielangel

The build shouldn't matter, CID info should be shown when the overclocking is ON (when you select a preset then restart camera). Could you double check?

1- CID info doesn't include copyrighted material, so it's totally okay. Well, CID info shows also Serial number (PSN) for SD card, I don't know if PSN should be kept in private.
2- Could you share more context? which error message do you mean?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on September 21, 2022, 10:16:04 PM
1- Your're right. It works now regardless of the build. The overclock isn't on by default.

2-I am trying to come up with a repeatable way of testing the record capabilities of a card at various resolutions, as the speed shown in the benchmarks doesn't mean that the camera will record at a certain resolution.
I found out that with certain types of images (Notably if I focus on the LCD monitor too closely) The amount of detail seems to be too much for the Lossless compressor / Data Rate (It just won't record).

I get a bunch of frame corruption and compression messages, but those are displayed too fast for me to read. It would be easier if those messages were saved on the SD card.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on September 22, 2022, 02:53:49 AM
Quote from: gabriielangel on September 21, 2022, 10:16:04 PM
I get a bunch of frame corruption and compression messages, but those are displayed too fast for me to read. It would be easier if those messages were saved on the SD card.

Sure but you should be able to record display messages with another device and replay them frame by frame.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on September 22, 2022, 03:12:51 AM
If that's the only way, I will take that route then.
It is a different kind of endeavour, as I will need a second tripod, etc.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on September 22, 2022, 07:15:30 AM
Don't over-engineer. We are not going for IQ. Table setup, some cardboard, some tape, smartphone, low IQ ... anything will do.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Danne on September 22, 2022, 07:53:31 AM
Quote from: gabriielangel on September 22, 2022, 03:12:51 AM
If that's the only way, I will take that route then.
It is a different kind of endeavour, as I will need a second tripod, etc.
Most likely compression errors if text appear while recording. Check this place in mlv_lite.c:
        if (OUTPUT_COMPRESSION)
        {
            /* PackMem appears to require stricter memory alignment */
            ASSERT(((uint32_t)out_ptr & 0x3F) == 0);
            ASSERT((max_frame_size & 0xFFF) == 0);
            struct memSuite * outSuite = CreateMemorySuite(out_ptr, max_frame_size, 0);
            ASSERT(outSuite);

            int compressed_size = lossless_compress_raw_rectangle(
                outSuite, fullSizeBuffer,
                raw_info.width, (skip_x + 7) & ~7, skip_y & ~1,
                res_x, res_y
            );

            /* only report compression errors while recording
             * some of them appear during video mode switches
             * unlikely to cause actual trouble - silence them for now */
            if (compressed_size < 0 && !RAW_IS_IDLE)
            {
                printf("Compression error %d at frame %d\n", compressed_size, frame_count-1);
                ASSERT(0);
            }

            DeleteMemorySuite(outSuite);
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on September 22, 2022, 03:12:55 PM
I did a quick record this morning. I will be able to tell you about the data rates of the different sections in the mlv file this evening.

In the video I start with a soft focus of my test image. I stop recording myself;
Second trial I focus a little more, It stops recording itself;
Third trial I get sharp focus. You see the errors (As Danne said, while recording) I let it go a little then stop recording.

Video Demonstration: https://bit.ly/3Uxkkcs

Test conditions to replicate easily:
2.5k 1:1 Preset
Lens at f3.5
Iso 400 (Makes it easier to replicate)
Lens at 66 cm from the screen
Exposure as seen on screen (It will still happen without the Overlays, I left those on for ease)
Record the still first frame of the test file (The rest is to test something else): https://bit.ly/3LwH4W9

This is not limited to Lab settings, I ran into this problem during the summer when shooting certain foliage and animal closeups (Fur) under harsh sun, but could not figure it out.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Danne on September 23, 2022, 09:35:59 AM
Compression errors can be fixed with tweaking regs. Often reg 713 and reg 7150. Seems this could be needed in this preset.

EDIT: Then again when looking closer I see liveview addons like histogram? If so, test is not valid. No extras to reduce overhead. Use only framing or real time previev, nothing else selected.
Generally when speed indicator goes in and out of orange red there could be a compression error risk.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on September 23, 2022, 05:00:12 PM
@Danne, if you read closely, I said that I left the overlays on for ease, because exposure is an important parameter (I specified, because we had an exchange about this).
I ran the test with and without the Histogram and spot meter, and it did it too. I am able to replicate the test results on demand.

I think that because it is easier to make it crap out 10 times out of 10 when ISO is set to 400, I suspect that the extra noise (Extra details) is one of the reasons.
But I could be wrong.

Edit: Do you know what reg_713c and reg_7150 Stand for? The values are -2000 to +2000. That's a lot of combinations to test :)
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Danne on September 23, 2022, 05:16:12 PM
You can check in crop_rec.c. Unfortunately I can't help atm but you could try lowering 7150 a little and raise 713c or lower. No more than 5-15 up and down 713c.
7150 not sure if needed.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on September 23, 2022, 08:39:03 PM
This is within my capabilities, no problem.
Maybe someone in the know could chime in, if they have the explanation handy(Actual purpose of each reg, so I can take potential problems into consideration)?

BTW, for those reading this out of context, this problem can be mitigated by:

1-Using a lower ISO (Less noise to encode)
2-Underexposition (Which results in a larger number of darker patches, easier to encode)
3-Avoiding tiny little details across the frame. For example, if I need to film a medium shot with blowing tree leaves without a subject in the foreground (Blurs the background a bit), I use the soft lens.

When I run into this problem, it is often because I move the camera, and the reflexions I tamed with the polarizer reappear suddently because of the change of angle.

I will move any findings, if any, in your main thread, to keep this one less cluttered.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: dimas1md on September 29, 2022, 01:15:49 AM
Guys i have a problem, my 6D doesn't want to load patch, i have this errore

https://drive.google.com/drive/folders/1f4U_yzcbjYxZhoEm2s1ZQ5WQ0uTdGqxM?usp=sharing

Can you help me to make my old horse run like a young one? )))
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on September 29, 2022, 04:04:41 AM
Quote from: dimas1md on September 29, 2022, 01:15:49 AM
Guys i have a problem, my 6D doesn't want to load patch, i have this errore

You should always tell which build you are using. I suppose you are not using a crop_4k build as requested. See https://builds.magiclantern.fm/experiments.html or https://www.magiclantern.fm/forum/index.php?topic=25782.0
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on February 22, 2023, 11:02:26 PM
I added the new sd_uhs source code (https://github.com/bilalfakhouri/magiclantern_hg_02/tree/crop_rec_4k_mlv_snd/modules/sd_uhs) to my ML github fork (https://github.com/bilalfakhouri/magiclantern_hg_02/tree/crop_rec_4k_mlv_snd), also I have clean it up a bit, added more comments and split it to few commits.

-Also fixed a 6D issue, the card become inaccessible  (https://discord.com/channels/671072748985909258/844581352082898975/1062746627112190013)while running the overclock, pausing write/read operations is required too same as 5D3. Commit (https://github.com/bilalfakhouri/magiclantern_hg_02/commit/b0d76fc1a66e2cd37b2f059ef7f93e1f1f4e1183).
-Links are updated in first post, please report for any issues!

Edit: Will delete other sd_uhs versions which were posted by me (in other posts/topics) soon, and will redirect sd_uhs download to this topic.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on February 23, 2023, 06:29:29 PM
I just updated SD card recommendations in first post:

Quote from: theBilalFakhouri on February 12, 2021, 10:32:15 PM
SD card recommendations:

-Any SD card with W90 label (capable of 90 MB/s write speed) should work with 240 MHz or lower.
-List of SD cards which were tested and they work fine at 240 MHz (pay attention to SD card capacity, lower capacities may not be capable of 90 MB/s write speed):

  • Sandisk Extreme PRO   95 MB/s  (64 GB or higher, SDR50, 32 GB isn't stable at 240 MHz)
  • Sandisk Extreme PRO 170 MB/s  (64 GB or higher, does 32 GB work?)
  • Sandisk Extreme PRO 200 MB/s  (64 GB or higher, does 32 GB work?)
  • Sandisk Extreme R190/W130      (256 GB, do lower capacities work?)
  • Sandisk Extreme Plus R170/W90 (256 GB, do lower capacities work?)
  • Samsung PRO Plus  (128 GB or higher, does 64 GB work?)
  • Samsung EVO Plus (256 GB)
  • Samsung EVO Select (256 GB)
  • Lexar UHS-II V90
  • Lexar Professional 256GB 1066x SDXC UHS-I (Silver Series)

-SD cards which doesn't work well at 240 MHz:

  • Kingston Canvas Go!

Feel free to suggest edits to update the list.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: iaburn on February 23, 2023, 08:17:07 PM
I would add the Kingston CANVAS Select Plus UHS-I, U3, V30 (https://www.kingston.com/es/memory-cards/canvas-select-plus-sd-card), the ones over 128GB.
Slightly faster than The Sandisk Extreme PRO (tested 128 and 256GB), very stable on long recordings and reliable so far.
(https://i.ibb.co/7Cqqv09/ktc-hero-sd-canvas-select-plus-lg.jpg) (https://imgbb.com/)

I can record RAW at 30fps on the BMPCC with no drop frames on this card, that says a lot  :D
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on February 23, 2023, 10:35:34 PM
Thanks for the suggestion iaburn, I have added it in the list.

Quote from: iaburn on February 23, 2023, 08:17:07 PM
Slightly faster than The Sandisk Extreme PRO (tested 128 and 256GB), very stable on long recordings and reliable so far.

Could you share some benchmarks at 240 MHz in Play mode for both Sandisk Extreme PRO and Kingston CANVAS Select Plus?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: iaburn on February 23, 2023, 11:07:30 PM
Quote from: theBilalFakhouri on February 23, 2023, 10:35:34 PM
Could you share some benchmarks at 240 MHz in Play mode for both Sandisk Extreme PRO and Kingston CANVAS Select Plus?

Sure, just did it with the 128GB ones, and I was actually wrong. I formatted both cards before the benchmark and the Sandisk got 94.4 MB/s max, I think it was around 85MB/s last time I checked. The Kingston tops at 89.8 MB/s:

Kingston CANVAS Select Plus 128GB:
(https://i.ibb.co/nLg2zZj/bench0-kingston-128.png) (https://ibb.co/yQfTWGq)

Sandisk Extreme PRO (170MB/s) 128GB:
(https://i.ibb.co/w7NLVLm/bench0-sandisk-128.png) (https://ibb.co/KDjGdGv)
Title: Re: SD Overclocking - DIGIC 5 only
Post by: LesV on April 29, 2023, 10:20:16 PM
Hi guys,
Could you help me to understand why my sd card's performance is so poor? Can I do something (different settings) to reach that level I can see on the images above? Or I should buy a new one from the list you refer above?

The camera: Canon EOS 700D
The build I'm using: crop_rec_4k.2022May25.700D115
The SD card: Kingston CANVAS Go! Plus, 128 GB, 170 MB/s, U3, V30
Please have a look at the link below. The result is shockingly poor, even though overclocked it according the description in here, setting it to 240MHz.

Thanks in advance.

https://drive.google.com/file/d/158xEtdns-19YnRmXdtTX3a04hkZdicqT/view?usp=share_link
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on April 30, 2023, 12:53:10 AM
@LesV

The SD card you are using "Kingston CANVAS Go!" was tested before and it doesn't work fine with SD Overclocking (see first post).
However sd_uhs version in crop_rec_4k.2022May25.700D115 build is outdated, download the latest sd_uhs version from first post, replace the old sd_uhs.mo with the new one.

Or use crop mood build (https://www.magiclantern.fm/forum/index.php?topic=26851.0) which include latest sd_uhs too (I will no longer maintain my old 650D / 700D builds (https://www.magiclantern.fm/forum/index.php?topic=25784.0)).

Test 160, 192 and 240 MHz with the latest sd_uhs then feedback please.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: pepper_chip on April 30, 2023, 01:20:05 AM
Quote from: LesV on April 29, 2023, 10:20:16 PM
Hi guys,
Could you help me to understand why my sd card's performance is so poor? Can I do something (different settings) to reach that level I can see on the images above? Or I should buy a new one from the list you refer above?

The camera: Canon EOS 700D
The build I'm using: crop_rec_4k.2022May25.700D115
The SD card: Kingston CANVAS Go! Plus, 128 GB, 170 MB/s, U3, V30
Please have a look at the link below. The result is shockingly poor, even though overclocked it according the description in here, setting it to 240MHz.

Thanks in advance.

https://drive.google.com/file/d/158xEtdns-19YnRmXdtTX3a04hkZdicqT/view?usp=share_link

Something went wrong, have two same Micro sd's and it can easy get around 78 mb/s when I'm recording
Title: Re: SD Overclocking - DIGIC 5 only
Post by: LesV on May 01, 2023, 12:18:02 PM
Quote from: theBilalFakhouri on April 30, 2023, 12:53:10 AM
@LesV

The SD card you are using "Kingston CANVAS Go!" was tested before and it doesn't work fine with SD Overclocking (see first post).
However sd_uhs version in crop_rec_4k.2022May25.700D115 build is outdated, download the latest sd_uhs version from first post, replace the old sd_uhs.mo with the new one.

Or use crop mood build (https://www.magiclantern.fm/forum/index.php?topic=26851.0) which include latest sd_uhs too (I will no longer maintain my old 650D / 700D builds (https://www.magiclantern.fm/forum/index.php?topic=25784.0)).

Test 160, 192 and 240 MHz with the latest sd_uhs then feedback please.

@theBilalFakhouri

Hi, thanks for the super quick reply. Apologies, I had a very busy day yesterday and I wanted to try also what you advised me to. I didn't mention that for test, I used already THAT sd_uhs module you recommended. I combined it with the build I mentioned.

Now I installed the latest build (see the image on the link below) but got exactly the same result. Absolutely disappointing (not the build!). I also watched your video (Crop mood - Build Walkthrough) on YT and tried to apply the same settings you were doing but I couldn't completely (vid resolutions). The raw video options I can only choose, getting green (that is working continuously and properly), are just ridiculously poor and at very low bit rates. 

The benchmark result: https://drive.google.com/file/d/1FCzVvijUBFPP7G_NO-AdAnqbLNquNhbW/view?usp=sharing

Is there ANY chance that the camera itself (by a setting I might have done somewhere) can cause this poor result? For goodness sake, the actual speed is only the half of the factory maximum. Or can my Kingston SD be faulty?

Any other ideas?

I already ordered another one (SanDisk Extreme Pro 128GB 200 MB/s) because I need more storage capacity anyway. Once I have it, will run a test, and will confirm if it works as it expected.

Thanks a lot.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: LesV on May 01, 2023, 04:01:24 PM
Quote from: pepper_chip on April 30, 2023, 01:20:05 AM
Something went wrong, have two same Micro sd's and it can easy get around 78 mb/s when I'm recording

@pepper_chip

Hi,

Thanks for your reply too. The fact that you wrote about the micro version of this sd card (from the same brand) should not effect speed, I mean yours and mine should work at pretty much the same speed. But, unfortunately, it's not the case. So far no clue what the problem might be. I'm waiting for a new card (ordered online) from a different brand, and will see the difference.

Thanks
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on May 01, 2023, 04:49:40 PM
Quote from: LesV on May 01, 2023, 12:18:02 PM
Is there ANY chance that the camera itself (by a setting I might have done somewhere) can cause this poor result? For goodness sake, the actual speed is only the half of the factory maximum. Or can my Kingston SD be faulty?

We have limited knowledge about SD registers, if the card support 90 MB/s write speed and didn't work with 240 MHz (dropped to 21 MB/s) that mostly mean that the SD card didn't accept the overclock because of our SD overlocking configuration.

Run benchmarks for the kingston SD card using fast card reader in PC, if it's close to marketing SD speed numbers then the SD card is fine.
Try a recommended SD card in camera with 240 MHz, if it works then your SD card reader in camera is fine too.

You didn't do anything wrong regarding camera/ML settigns. Try lower frequencies like 160 and 192 MHz, might work with your kingston card.
Anyway, Kingston Canvas Go! cards are not recommended with our current SD Ovreclocking version until someone figure out how to make it work.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: names_are_hard on May 01, 2023, 04:51:50 PM
I don't think anyone has mentioned it so far: the reported 20MB/s results are what happens when a card fails to overclock.  The cam drops back to a slow, safe default (this is not ML but the cam / card itself).

With this card, you may get higher benchmark results if you disable all overclocking.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on May 01, 2023, 04:55:49 PM
Yes, that's right. I have mentioned "The cam drops back to a slow, safe default" part in several posts in the past.
Will include it them in first post, it's helpful.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: LesV on May 01, 2023, 09:18:31 PM
Quote from: theBilalFakhouri on May 01, 2023, 04:49:40 PM
You didn't do anything wrong regarding camera/ML settigns. Try lower frequencies like 160 and 192 MHz, might work with your kingston card.
Anyway, Kingston Canvas Go! cards are not recommended with our current SD Ovreclocking version until someone figure out how to make it work.

Thanks for getting me back. OK, I got it. It's 'good' to know that it's not my or my camera's fault but a 'tough' card's 'fault' that cannot be overclocked.

Right after I sent my previous comment, I tried the card at 192, but got the same result.

The new card however, that I ordered, is on your list, so it should be fine.

Thanks
Title: Re: SD Overclocking - DIGIC 5 only
Post by: LesV on May 01, 2023, 09:26:35 PM
Quote from: names_are_hard on May 01, 2023, 04:51:50 PM
I don't think anyone has mentioned it so far: the reported 20MB/s results are what happens when a card fails to overclock.  The cam drops back to a slow, safe default (this is not ML but the cam / card itself).

With this card, you may get higher benchmark results if you disable all overclocking.

Ah! Brilliant. I deactivated the module and indeed, now it works better. At least I have the factory potential...

Thanks!
Title: Re: SD Overclocking - DIGIC 5 only
Post by: LucasFlores on June 09, 2023, 10:09:03 PM
Hi everyone. I have tested my new Sandisk Extreme Pro, upgraded version - READ 200 and WRITE 140MB/s and I am using the crop_mood.2023May09.700D115 in my 700D camera.
As you can see in the photos below, it's able to perform 95.0 MB/s in photo mode using the 240MHz overclock preset...
(https://i.ibb.co/b68hDKV/benchmark.jpg) (https://imgbb.com/)

but when I try to start recording RAW video using the preset 4.4k, medium, it performs very bad as you can see in the screenshot below. What I am doing wrong? I formatted the card at low-level to see if that is the issue, nothing. Also I tried other presets, the same issue. I tried changing the overclock presets but the results are the same and worse.
(https://i.ibb.co/wJJVWvw/recording-speed.jpg) (https://ibb.co/LQQmtGJ)
<a target='_blank' href='https://es.imgbb.com/'>fotos temporales</a>

This is my setup:
(https://i.ibb.co/8XNSxyx/my-setup.jpg) (https://ibb.co/GFnbMZM)

This is my card:

(https://i.ibb.co/4p0j35g/Extreme-Pro-spec.jpg) (https://ibb.co/WfRcjYp)
(https://i.ibb.co/8Kd56Dj/Extreme-Pro-spec-v2.jpg) (https://ibb.co/1vdq8f9)
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on June 11, 2023, 02:25:05 PM
@LucasFlores

Interesting, we need to find a better 240 MHz configuration in this case, unfortunately I can't work on it since I don't have an affected SD card.
Your SD card seems fine, our knowledge is still fairly limited about SD registers, as result not all of SD cards will be stable with current 240 MHz.

Have you tried 192 MHz, does it work fine?

I think it could be a good idea to add debug menu in SD Overclock submenu, I can suggest some values for some SD overclocking registers.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: bobolee on June 12, 2023, 04:00:15 PM
Quote from: LucasFlores on June 09, 2023, 10:09:03 PM
Hi everyone. I have tested my new Sandisk Extreme Pro, upgraded version - READ 200 and WRITE 140MB/s and I am using the crop_mood.2023May09.700D115 in my 700D camera.
...
Have you changed the photo shooting settings to raw format in original Canon menu?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on June 12, 2023, 04:15:31 PM
Quote from: theBilalFakhouri on June 11, 2023, 02:25:05 PM
Interesting, we need to find a better 240 MHz configuration in this case, unfortunately I can't work on it since I don't have an affected SD card.

Can do test runs on 650D if it helps.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on June 12, 2023, 11:21:38 PM
@Walter

Does your Sandisk Extreme PRO 200 MB/s 256GB drop to 21 MB/s when recording (OC at 240 MHz)?
To find out try recording at 80 MB/s maybe until you fill the card, try peviewing the clips between recording sessions.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on June 13, 2023, 09:00:07 PM
Benchmark in mode PLAY-PH LCD
w 94.5
r 105-7
w 95.2
r 106.1

Running record tests now.

EDIT: Looks like unstable with 240 MHz and Access mode SDR 104. Recording will stop with message "expected more frames ...) and fallback to 20 MByte/s. SDR50 not having this problem.
First test, may have to do more thorough tests.

EDITEDIT: Correction: More stable. A lot more stable. But will fallback to 20 MByte/s and stopping recording. Got a 30 GB file and 8:26 recording -> Around 60 MByte/s.

Q: Next steps?

PS: Ah, have to redo tests. Not configured correctly. Will retest tomorrow.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on June 15, 2023, 04:12:20 PM
@Walter

Thanks for the tests, so it does drop to 21 MB/s after some time with SDR104?
Could you try the same test but this time on EOS M instead of 650D using the same SD card? Is it stable there?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on June 15, 2023, 05:09:42 PM
650D: It stops after mere seconds with SDR104.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: bobolee on June 17, 2023, 02:32:14 AM
(https://i.ibb.co/XXYzP7s/QQ-20230429231802.jpg)(https://i.ibb.co/y4v6PM8/IMG-20230617-075634.jpg)(https://i.ibb.co/26qJVtg/IMG-20230616-170315.jpg)
This micro sd card was getting 20MB/S and recording halt randomly when using standard sdr105/240mhz overclocking,but yesterday,I find that sdr50/240mhz mode got a full card recording without any problem,just like recording with my old 128GB 95MB/s sd card in sdr105/240mhz mode does.
Tested in 30+ ℃ ambient temperature.3x3 1080p 16:9 30fps ,global draw all disabled to minimize CPU usage.Will test more in the future to make sure it is 100% reliable even in harsh environment .
Another conclusion is that SanDisk Mico sd and sd are equal for the use of ML cameras,both works the same,but micro sd version of extreme pro is about half the price of SD version.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on June 20, 2023, 11:05:19 AM
Bilal, you checked my results with Samsung PRO Plus 256GB "K" Series? https://www.magiclantern.fm/forum/index.php?topic=26932.msg244103#msg244103
No issues with SDR 104 at all.

Now testing Sandisk Extreme Pro R200/W140 and EOS M.

Benchmark looks fine with 95 MByte/s for write in PLAY-UNK.

SDR104 and SDR50 act quite similar: Forced stop after some minutes. 17GB and 8.x GB in first run.


It looks like Samsung is performing a bit better!
Title: Re: SD Overclocking - DIGIC 5 only
Post by: theBilalFakhouri on June 20, 2023, 12:59:21 PM
Thanks for the test, so EOS M with Sandisk Extreme Pro R200/W140 at 240 MHz didn't drop to 21 MB/s in all tests unlike 650D (https://www.magiclantern.fm/forum/index.php?topic=25841.msg244025#msg244025)?
If yes, that's because 240 MHz preset configuration on EOS M is different from the one on 650D:

1- Hyprid method was used on EOS M i.e 240 MHz for write and 192 MHz for read while on 650D it's 240 MHz in both write/read
2- 240 MHz preset on EOS M is different from the one on 650D (I mean registers values in this case).

I think it worth adding an option to toggle among "240 MHz" and "240 MHz (H)" presets for 650D / 700D, "240 MHz (H)" could be more stable on some SD cards.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on June 20, 2023, 01:22:13 PM
I have *no* problems on both cams running benchmarks! Both cards perform well with SDR 104.

I can do at least 6 minutes (haven't tested longer sessions yet) with Samsung PRO Plus and 1280p mode. For both cams.

Have to redo some tests to see if fallback mode is present after breaks for Sandisk card.

EDIT:
=======
Nope!
EOS M. This time Sandisk took 10:03 in 1280p mode to break down after going "red" a few times. SDR 50 mode.
And it *does not* fallback to 20 MByte/s. Benchmark in movie mode is around 77 MByte/s after forced stop.

Calculated 83.3 MByte/s for recording.

=======
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Damiafix on July 02, 2023, 04:31:48 PM
Hi everyone,

I would like to share my experience with a card which is not listed as a good choice for 240Mhz overclock, but it works to me.

Camera:  Canon 5d Mark III
Firmware: 1.1.3
ML settings:  card spanning and 240mhz overclock activated.
ML build: Danne's crop rec 4k_mlv_snd_isogain_1x3_presets_2023Feb03.5D3113
SD card: Lexar Professional 1667x SD card 128 GB, SDXC UHS-II, up to 250 MB/s. (camera low-level formatting)
(https://i.ibb.co/stNgKsr/1.jpg) (https://ibb.co/stNgKsr)
(https://i.ibb.co/2yYtGbS/2.jpg) (https://ibb.co/2yYtGbS)

CF card: Lexar Professional CF 128 GB 1066x (160 MB/s) UDMA 7 (camera
low-level formatting)
PC card reader: Trascend TS-RDF9K USB 3.1/3.0

Benchmarks in video mode

Sd card benchmark:
(https://i.ibb.co/QJcdfp3/bench10.jpg) (https://ibb.co/QJcdfp3)
Sd and CF card benchmark:
(https://i.ibb.co/fnCmFVK/bench11.jpg) (https://ibb.co/fnCmFVK)

Benchmarks in photo mode:
Sd card benchmark:
(https://i.ibb.co/J52HsL2/bench12.jpg) (https://ibb.co/J52HsL2)
Sd and CF card benchmark:
(https://i.ibb.co/sJm9FSp/bench13.jpg) (https://ibb.co/sJm9FSp)

CrystalDisk Mark benchmark:
(https://i.ibb.co/8NB04fT/Crystal-Benchmark.jpg) (https://ibb.co/8NB04fT)

According to the list of Sd 'working' cards under 240Mhz overclock, I bought a SanDisk 128 GB Extreme PRO, SDXC up to 200 MB/s, UHS-I, Class10, U3, V30. Unfortunately overclock doesn't work properly, in my case. I mean it works for a while and I can run some benchmarks:

Benchmarks in video mode:

160Mhz overclock
(https://i.ibb.co/vmMrdxf/bench12.jpg) (https://ibb.co/vmMrdxf)
192Mhz overclock
(https://i.ibb.co/grbWPVW/bench13.jpg) (https://ibb.co/grbWPVW)
240Mhz overclock
(https://i.ibb.co/RBBPY8N/bench14.jpg) (https://ibb.co/RBBPY8N)

If I try to record a video, under 240Mhz overclock, the camera becomes unstable. After that, if I run any benchmars I get unreal speed readings:

240Mhz
(https://i.ibb.co/MDNng3g/240-Mhz-instabile.jpg) (https://ibb.co/MDNng3g)

Nothing to do even lowering the overclock to 160Mhz
160Mhz
(https://i.ibb.co/zm7NPRN/160-Mhz-instabile.jpg) (https://ibb.co/zm7NPRN)

The camera, paired to the overclocked SandDisk, isn't stable at all. The situation doesn't change switching from SDR 104  to SDR 50 or warming  the card up.  Peraphs the problem is just limited to such a card. 

My SanDisk 128 GB Extreme Pro:
(https://i.ibb.co/WyT3sSH/1.jpg) (https://ibb.co/WyT3sSH)
(https://i.ibb.co/NKkX8bw/2.jpg) (https://ibb.co/NKkX8bw)
<a href="https://ibb.co/FKSNtfd">(https://i.ibb.co/FKSNtfd/3.jpg)

CrystalDisk Mark benchmark:
</a> (https://i.ibb.co/Y05r1NV/Crystal-Mark-benchmarkl.jpg) (https://ibb.co/Y05r1NV)

Under PC, values seem to be coherent with the results of other users. 

Any possible setting before returning it to the seller (bought by Amazon) and buying the 170mb/s version? It would be warmly appreciated.

Thanks for your support.

All the best.

Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on July 02, 2023, 06:14:42 PM
Can you share a screenshot with Sandisk Extreme Pro R200/W90 showing those read numbers?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: bobolee on July 03, 2023, 01:44:39 AM
It seems that this SanDisk card has no problem in EOS M/700D,but 5DM3
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Damiafix on July 03, 2023, 09:35:12 AM
Quote from: bobolee on July 03, 2023, 01:44:39 AM
It seems that this SanDisk card has no problem in EOS M/700D,but 5DM3

Hi Bobolee,

thanks for writing. It is a bit weird since there are users (https://www.magiclantern.fm/forum/index.php?topic=26928.0) who have a similar setup as mine, and they can overclock the SanDisk extreme pro 200mb/s keeping the camera stable. I am trying to understand if it is a problem of settings or the card itself.

Title: Re: SD Overclocking - DIGIC 5 only
Post by: names_are_hard on July 03, 2023, 11:21:23 PM
The name written on the card doesn't mean much.  Sometimes they're fake, sometimes they're legit but the same name is used for a different production run with different characteristics.

Bobolee is speculating and doesn't really know if there is some camera / card incompatibility.  Personally I think it's unlikely.

Listen to Walter, he has lots of experience in this area.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Damiafix on July 04, 2023, 11:30:31 AM
Quote from: names_are_hard on July 03, 2023, 11:21:23 PM
The name written on the card doesn't mean much.  Sometimes they're fake, sometimes they're legit but the same name is used for a different production run with different characteristics.

Bobolee is speculating and doesn't really know if there is some camera / card incompatibility.  Personally I think it's unlikely.

Listen to Walter, he has lots of experience in this area.

Thanks so much, it makes total sense. Just to share some other infos about my piece of SanDisk Extreme Pro, It follows what ML CID Info reports.

The card seems to be made by SanDisk:
(https://i.ibb.co/fxGDQw1/CID-info.jpg) (https://ibb.co/fxGDQw1)

And I have to say that the package seems to be genuine too:
(https://i.ibb.co/jMqvLQr/1.jpg) (https://ibb.co/jMqvLQr)
(https://i.ibb.co/ZXNrCGG/2.jpg) (https://ibb.co/ZXNrCGG)
(https://i.ibb.co/g4S2wYL/3.jpg) (https://ibb.co/g4S2wYL)

Quote from: Walter Schulz on July 02, 2023, 06:14:42 PM
Can you share a screenshot with Sandisk Extreme Pro R200/W90 showing those read numbers?
Walter are you referring to me? I am not sure.

Best.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on July 05, 2023, 07:07:23 AM
Quote from: Damiafix on July 04, 2023, 11:30:31 AM
Walter are you referring to me? I am not sure.

Sorry, missed the pics you shared. I see the problem now.
I'm a bit confused TBH. It doesn't look like an error I seen before.
Are there any other modules loaded? If so: Please re-test with only bench.mo and sd_uhs.mo loaded.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Damiafix on July 07, 2023, 02:41:24 PM
Quote from: Walter Schulz on July 05, 2023, 07:07:23 AM
Sorry, missed the pics you shared. I see the problem now.
I'm a bit confused TBH. It doesn't look like an error I seen before.
Are there any other modules loaded? If so: Please re-test with only bench.mo and sd_uhs.mo loaded.

Hi Walter,

thanks. The only modules I am using are sd_uhs.mo and bench.mo.
(https://i.ibb.co/nRgxxLm/Modules.jpg) (https://ibb.co/nRgxxLm)
(https://i.ibb.co/HX3WGcv/Overclock-160-Mhz.jpg) (https://ibb.co/HX3WGcv)
The card does accept just SDR104, with SDR50 the access to the card itself is compromised and ML doesn't work properly.
(https://i.ibb.co/PGzYwzx/Overclock-Settings-2-SDR104.jpg) (https://ibb.co/PGzYwzx)

Anyway on yesterday, after turning all the modules off/on by "Debug/Disable all modules"and rebooting ML a couple of times I progressively set the overclock to 160Mhz, 192Mhz and finally to 240Mhz and I have been able to run some benchmarks and record some videos outside. It seemed to work for a limited period of time, but the camera became unstable forcing to reboot it frequently. It happened with all the overclock configurations.

I think that with such a card the main problem is the 'read' and not the 'write', even though the camera is too unstable.

In fact If I try to play a recorded video I frequently get error messages. After that rebooting ML is the only solution.
It follows one of the errors which I get during the play:
(https://i.ibb.co/qBYqc7t/Editato.jpg) (https://ibb.co/qBYqc7t)

They follow the VIDEO settings, with all the hacks activated (with or without hacks makes no difference):

(https://i.ibb.co/WxtDhJk/IMG-20230707-WA0012.jpg) (https://ibb.co/WxtDhJk)
(https://i.ibb.co/7Ws79B1/IMG-20230707-WA0013.jpg) (https://ibb.co/7Ws79B1)
(https://i.ibb.co/C0zR0j0/IMG-20230707-WA0014.jpg) (https://ibb.co/C0zR0j0)
(https://i.ibb.co/H2pXRXd/IMG-20230707-WA0015.jpg) (https://ibb.co/H2pXRXd)
(https://i.ibb.co/VWHKSyv/IMG-20230707-WA0016.jpg) (https://ibb.co/VWHKSyv)

Canon settings. I don't know if they can affect the stability, but with the SD Lexar Professional 1667x SD card 128 GB, SDXC UHS-II, up to 250 MB/s and the same settings I have no problems.
(https://i.ibb.co/9w9dFF2/Canon-Settings.jpg) (https://ibb.co/9w9dFF2)

Anyway I wish confirming the usability of the Lexar Professional CF 128 GB 1066x, which even under 240Mhz overclock gives back very good numbers (and it works very well with the Lexar SD):
(https://i.ibb.co/F88VC8B/Bench-Mark-240-Mhz.jpg) (https://ibb.co/F88VC8B)

The benchmark SD (SanDisk Extreme Pro 200mb/s) + CF under 240Mhz, when the camera is still a bit stable, is good too:
(https://i.ibb.co/7z9rSJ5/Bench-Mark-SD-San-Disk-and-CF-Lexar.jpg) (https://ibb.co/7z9rSJ5)

I don't understand what's the issue with the SanDisk Extreme Pro. I will probably return it if I can't find a solution.

Thanks for any advice. Best.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: 123kid on September 27, 2023, 11:53:09 PM
Hey,

i dont if somebody has this issue, but i can only record with on my 5dmarkiii about 18mb/s write speed.
i used several san disk pro extreme sd cards..but i cant get more than 20mb/s write speed.

i have the 5dmarkiii with firmware 1.2.3 and the latest magic lantern version on the sd cards...

any help?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on September 27, 2023, 11:58:55 PM
Please do not crosspost/spam.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: 123kid on October 15, 2023, 02:50:41 PM
Someone testet a sd/tf card with a Cf Card Adapter??

i will test it if there are any speed changes..

https://www.amazon.de/Yunseity-Speicherkarte-Kartenleser-Speicherkartenadapter-Hochgeschwindigkeitskamera-Schwarz/dp/B09GG2959L/ref=sr_1_11?__mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=1EXMKPGFIILBI&keywords=cf%2Bcard%2Badapter%2Btf&qid=1697374166&s=computers&sprefix=cf%2Bcard%2Badapter%2Btf%2Ccomputers%2C95&sr=1-11&th=1
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on October 15, 2023, 03:04:32 PM
No support by ML project.
https://wiki.magiclantern.fm/faq#ml_won_t_work_with_my_sd-to-cf_adapter
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 04, 2024, 04:47:40 AM
Why Benchmarks are not enough.

I saw a few posts on the fb ML group, people reporting problems recording with certain cards.
Hopefully, the following will help a little.

Let's take those 3 cards:

Samsung Pro Plus 128GB (MDT:2023/05) - item "G" in the picture below
Samsung Evo Plus 128GB (MDT:2023/04) -  item "H" in the picture below
Sandisk Extreme Pro 256GB (MDT:2022/06) - item "B" in the picture below

(https://i.ibb.co/G5QgbN0/Cards-Tested.jpg) (https://ibb.co/0GyPdvc)

Looking at the individual benchmarks:
(240 MHz / SDR104)

Samsung Pro Plus 128GB (MDT:2023/05)
(https://i.ibb.co/KV2jKwy/Samsung-Pro-Plus-180-128-GB-m-SD-240-MHz-SDR104-Long.jpg) (https://ibb.co/F85bHV7)

Samsung Evo Plus 128GB (MDT:2023/04)
(https://i.ibb.co/jRKY9qv/Samsung-EVO-Plus-130-128-GB-m-SD-240-MHz-SDR104-Long-V2.jpg) (https://ibb.co/RynKGtp)

Sandisk Extreme Pro 256GB (MDT:2022/06)
(https://i.ibb.co/1MBsfp8/Sandisk-Extreme-Pro-200-256-GB-SD-240-MHz-SDR104-Long.jpg) (https://ibb.co/gdQvMXz)

Now, let's see how those benchmarks compare to real world results.

Edit: I found out that some cards seem to have some kind of built-in thermal management. After recording for a while, those cards enter into safe mode and stop recording until you restart the cam. The harder you push it (Data Rate wise), the sooner the problem appears.
So, it would be advisable to record a few clips (About 5-10, more details later) where the recording light alternates between orange and red before testing. This would warm up the card and hopefully expose the problem sooner, if present...

Edit 2: I also benchmarked the cards to get a "Continuous" Green value (Those numbers are included in the posts where I test individual cards)

Updated Method: (Open the aperture to the maximum and use the slowest shutter speed available. Set "Kill Global Draw" to ON on the Camera.)

I recorded the Test Chart linked at the end of the post on a monitor at 2.8k, 2.39:1 AR, 23.976fps; and used a variable ND filter on a bright lens (ef-m 32mm f1.4).
You may need an aperture of at least f2.8, because the Monitor may not be bright enough to yield high Data Rates.
Focus slightly off-focus until all the moiré on screen disappears (Moiré and pixels will make the compressor work unnaturally hard).

1)  Max Orange Speed

First record slightly overexposed at 14bits, and then slowly close the ND filter in small increments, and record again until the camera manages to record continuously for 1 minute.
This is usually the point where the light is orange and briefly turns red once and a while.
If it can't reach the 1 minute mark, and you closed the ND filter to the point where the image is too dark to be useful, lower the bit depth to 12bits and try again.

Once it reaches 1 minute for 2-3 recordings, Open the ND filter in tiny increments, record, and repeat  until it can't reach the 1 minute mark anymore.

And finally close the ND filter in small increments again, and repeat until the camera records for 1 minute.

Record a few clips. If those all record 1 minute or more, this is the speed for "Maximum Orange" (Max. Speed).

2) Max Green Speed ("Continuous")

First record slightly overexposed at 14bits, and then slowly close the ND filter until you get a steady Green light.
You will need to do several recordings in a row to get to the highest possible Green point. You may get Green, and still get Orange on the next recording.

After pressing Record, The light must become Green before or at second 2 onwards. If you still see Red or Orange at second 2, you're still too high.

Once you get 3 consecutive recordings where you get Green at, or before second 2, and it stays that way for 1 minute; you've got it. This is the speed for "Maximum Green" ("Continuous").

3)  Max Red Speed (This is to check a card's resiliency to sudden changes in Data Rates)

First record slightly overexposed at 14bits, and then slowly close the ND filter in small increments, and record again until the camera manages to stop recording by itself after about 15 seconds.

With some cards, it is very hard to get repeatable results because we are at the card's upper limit, and that's what we are trying to find out.

Record a few clips. Once you reach a point where it always stops by itself around the same number of seconds, (On some cards, that point was between 15 and 18 seconds);
take the lowest MB/s number (Or the most frequent) and this is the speed for "Maximum Red" (Peak Speed).

This test chart generates files about 5 MB/s faster than the most I usually get when shooting outdoors on a bright day, with similar histogram / false colors values
(I usually expose with histogram between 1.1 and 0.8, some red, but nothing showing black on false colors).
I chose 2.8k/14, as running a single test gives you an idea of what you'd get at 3k 1:1/11; 5.2k 3x1/11; 4.8k 3x1/12; and everything below at 14bits.

Samsung Pro Plus 128GB (MDT:2023/05)
(https://i.ibb.co/6sFRwhD/Samsung-Pro-Plus180-128-G-MLVApp.png) (https://ibb.co/VBTW9yM)

This card is not stable at 240MHz. In order to get the results above, I had to restart the camera after each take! Trying to record twice in a row results in the camera going into safe mode (The 240MHz in the Overclocking section of the camera becomes dim) A camera restart is required to go back to 240MHz. When overclock is at 192MHz, the card behaves normally, but at slower speeds. One of the reasons why it is important to do more than 1 recording!

Samsung Evo Plus 128GB (MDT:2023/04)
(https://i.ibb.co/KLJFTCk/Samsung-Evo-Plus130-128-G-MLVapp.png) (https://ibb.co/wwfM5DX)

This card would not record more than a few seconds at 2.8k, even when severely underexposed.
It managed to record 2.5k at 62 MB/s (True to benchmark).

Sandisk Extreme Pro 256GB (MDT:2022/06)
(https://i.ibb.co/30gXnKt/Sandisk-Extreme-Pro200-256-G-MLVapp.png) (https://ibb.co/brchktp)

Although this card had a lower number than the Pro Plus in the benchmarks, it gave the best performance at 86 MB/s without any throttling.

When looking at the Data Rate Graph on a PC (Not included here), some cards have momentary dropouts (and even Bursts) of up to 20 MB/s! The card with the highest sustained speed / fewer (or Shallower) dropouts wins.

Now, although this has been said before, it's always nice to have everything in the same place:
Why does the camera stops recording "Sometimes"?

Excluding camera related hiccups, because we're recording in Lossless, Scene complexity and exposure have an effect on Data Rate; and so does scene composition!
A closeup with a big blurry background has a significantly lower Data Rate than a landscape where everything is in focus.
Any change in brightness (Shiny object crossing the frame), complexity (tree branches) or change of focus while recording can change the Data Rate and put it above the Maximum the Cam / Card can handle.

If you look at the examples below, every take was recorded outside on the same day, within about 1.5 hour. The Data Rates are all over the place. When you see 14bits followed by a 12bits take, it's because I had to lower the bitrate to be able to record the exact same scene.

(https://i.ibb.co/fDNR7xX/Data-Rate-Examples1.png) (https://ibb.co/MB1TLCg)
(https://i.ibb.co/1Td45r6/Data-Rate-Examples2.png) (https://ibb.co/pdRk8fy)

You can also find more examples in a previous post: https://www.magiclantern.fm/forum/index.php?topic=25781.msg240610#msg240610

If you want to test a card with the chart I used, here it is: (https://i.ibb.co/dGztVLM/Alignment-Chart.png) (https://ibb.co/WgY0MP5)

EDIT: One thing to keep in mind is that, because I were looking for the absolute peak data rate I could get for 1 minute, framing isn't too important. But depending how close or far you are from the screen, the same data rate will be achieved with more and less exposure respectively, as when you are further away (Up to a point) the details become smaller and are harder to compress (But when you are closer, you get more brightness from the screen so...!). If someone were to compare data rates across presets, framing would need to be identical and exposures would need to be matched...

Tested on Eos m, with magiclantern-crop_mood.2023Aug19.EOSM202.zip, Small Hacks set to more, Camera in af mode.

The official 240Mhz Overclock cards compatibility page: https://wiki.magiclantern.fm/cards_240mhz











Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on January 04, 2024, 06:19:22 AM
Thanks! Will try to reproduce ASAP and will esp. retest with my Samsung Pro Plus 128 GB (2021 edition)

Questions/Remarks:
Samsung Pro Plus in blue exists in 2 different varieties: 2021 and 2023 edition. You will find a couple of strings on card's backside. "MB-MD128K" or "MB-MD128S". Which one are we talking about here?

I tested EVO Plus 128 GB (2021 edition) and it failed @240 MHz and it is marked as failed in linked chart.
Just out of curiosity: Can you post a pic of card's back?

(EDIT: I know not to trust benchmarks. I have cards with stable benchmarks @280 (sic!) MHz  but unable to record. My test routine is not as sophisticated as yours but include record test as well. Needs to be refined, though!)

EDIT2: Any luck with "Sandisk Extreme Pro 95 MB/s"? It worked perfectly with older oc modules but my specimen fails with new one in 650D. And because of mechanical issues it can't be used in my M. 
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 04, 2024, 04:49:30 PM
Quote from: Walter Schulz on January 04, 2024, 06:19:22 AM
Thanks! Will try to reproduce ASAP and will esp. retest with my Samsung Pro Plus 128 GB (2021 edition)

Questions/Remarks:
Samsung Pro Plus in blue exists in 2 different varieties: 2021 and 2023 edition. You will find a couple of strings on card's backside. "MB-MD128K" or "MB-MD128S". Which one are we talking about here?
MB-MD128S

I tested EVO Plus 128 GB (2021 edition) and it failed @240 MHz and it is marked as failed in linked chart.
Just out of curiosity: Can you post a pic of card's back? (Can't connect anything atm, so:)

MB-MD128K (with a little checkmark after)
MBMCDGVGDBCW-CW-5
KPVK119T2317

Made in Phillipines

EDIT2: Any luck with "Sandisk Extreme Pro 95 MB/s"? It worked perfectly with older oc modules but my specimen fails with new one in 650D. And because of mechanical issues it can't be used in my M.
I don't have that card, but will let you know when I find one. I am also curious about the Lexar x1667 UHS-II. If someone who owns one wants to chime in... :)

Answers in red in the quote above.
I will add the other cards specs as I get to properly test those.

EDIT: @Walter, when you test the EvoPlus128GB, try recording the first take for 5 minutes. Yesterday, the card seemed fine at  first. I suspect there is some kind of thermal throttling going on...
Title: Re: SD Overclocking - DIGIC 5 only
Post by: names_are_hard on January 04, 2024, 06:14:53 PM
Some (fairly reliable) people have also reported low-level format in cam restoring a slow card back to fast.  Possibly, this is better if done at the overclock you intend to run the card at.  I'm not convinced by that last part, but it might be worth testing.  The low-level format restoring a slow card I've seen multiple people report good results with.

I'm not sure why a low-level format should help.  Possibly, because writing to erased blocks is fast, but erasing blocks is slow?  If the drive controller knows which blocks are erased, this might avoid the need for erases during recording.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 05, 2024, 05:18:11 AM
Speed Test: Lexar SILVER Series 1066x microSDXC UHS-I 256GB
R160/W120
Product Number: LMS1066256G-BNANU
MDT: 2023/07

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 88.12 MB/s @ 2.8k/14bits
Highest CropMood speed reached for "Continuous"   (Steady Green): 77.88 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 90.8 MB/s @ 2.8k/14bits
240MHz / SDR104

(Item "F" in the picture below)
(https://i.ibb.co/G5QgbN0/Cards-Tested.jpg) (https://ibb.co/0GyPdvc)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/Ry7WZYJ/Lexar-1066x-256-GB-m-SD-240-MHz-SDR104-Long.jpg) (https://ibb.co/dg7y1jH)

(https://i.ibb.co/wyw0G4y/Lexar1066x-256-GB-MLVApp.png) (https://ibb.co/4WYmvpW)

The method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

I recorded more clips this time, to hopefully expose any Thermal Throttling or Burst Speed Wizardry implemented by the manufacturer. I recorded about 10 clips of various lengths (Light orange / red) prior to beginning the test (Not in the picture, to save on transfer time)

This card did not exhibit any problems and was consistent.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on January 05, 2024, 01:55:53 PM
Thanks! Added to chart.

You know you can edit ML wiki and add card info?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 05, 2024, 07:44:09 PM
Quote from: Walter Schulz on January 05, 2024, 01:55:53 PM
Thanks! Added to chart.

You know you can edit ML wiki and add card info?

I didn't! I will have a look once I am done with the other cards on the picture. Thanks.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 06, 2024, 12:08:37 AM
Speed Test: Sandisk Extreme PLUS microSDXC UHS-I 128GB
R200/W90
Product Number: SDSQXBD-128G-CN6MA
MDT: 2023/06

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 79.41 MB/s @ 2.8k/12bits
Highest CropMood speed reached for "Continuous"   (Steady Green): 72.61 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 83 MB/s @ 2.8k/14bits
240MHz / SDR104

(Item "C" in the picture below)
(https://i.ibb.co/G5QgbN0/Cards-Tested.jpg) (https://ibb.co/0GyPdvc)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/0f54jPS/Sandisk-Extreme-Plus-200-128-GB-m-SD-240-MHz-SDR50-Long.jpg) (https://ibb.co/7GhMJ75)

(https://i.ibb.co/4F86sfW/Sandisk-Extreme-Plus-128-GB-MLVApp.png) (https://ibb.co/8mrWzBM)

The updated method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

The card could not handle 2.8k/14bits unless the image was quite underexposed and would not reach 1 minute,
but worked well at 2.8k/12bits.
This card did not exhibit any problems and was consistent.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 06, 2024, 12:29:20 AM
Speed Test: Sandisk Extreme PLUS microSDXC UHS-I 256GB
R200/W140
Product Number: SDSQXBD-256G-CN6MA
MDT: 2023/09

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 83.16 MB/s @ 2.8k/12bits -OR- 83.85 MB/s @ 2.5k/14bits
Highest CropMood speed reached for "Continuous"   (Steady Green): 76.68 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 87 MB/s @ 2.8k/14bits
240MHz / SDR104

(Item "D" in the picture below)
(https://i.ibb.co/G5QgbN0/Cards-Tested.jpg) (https://ibb.co/0GyPdvc)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/rf065Nn/Sandisk-Extreme-Plus-200-256-GB-m-SD-240-MHz-SDR104-Long.jpg) (https://ibb.co/x8LjzdV)

(https://i.ibb.co/vkgngFY/Sandisk-Extreme-Plus256-GB-MLVApp.png) (https://ibb.co/bXq9q0J)

The updated method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

The card could not handle 2.8k/14bits unless the image was quite underexposed and would not reach 1 minute,
but worked well at 2.8k/12bits.
This card was consistent, but went into safe mode when pushed in the red region.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on January 06, 2024, 06:49:26 AM
There are a few reports about SanDisk R200/W140 being unstable with access mode SDR104. Mine included.
After switching to SDR50 cards seem to be stable, though.
Can you retest in that mode?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 06, 2024, 06:53:17 PM
Sure, which one?
I have Sandisk Extreme Pro 256GB 200/140 SD and Sandisk Extreme Plus 256GB 200/140 Micro SD
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on January 06, 2024, 07:06:32 PM
Why not both? ;-)
But it is Extreme Pro I'm most interested (and others, I suppose).
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 06, 2024, 07:12:57 PM
I'll eventually post all of those, but will test the one you care about a little harder...
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 07, 2024, 02:12:25 AM
Speed Test: Sandisk Extreme PRO SDXC UHS-I 256GB
R200/W140
Product Number: SDSSDXXD-256G-GN4IN
MDT: 2022/06

240MHz / SDR104
Highest CropMood speed reached for a 1 minute Clip (Peak Orange):  86.08 MB/s @ 2.8k/14bits. (Inconsistent)
Highest CropMood speed reached for a 2 minutes Clip (Peak Orange): 85.42 MB/s @ 2.8k/14bits  (Inconsistent)
                                                                                                     83.57 MB/S @ 2.8k/14bits (Consistent)
Highest CropMood speed reached for "Continuous"    (Steady Green): 77.74 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 88.6 MB/s @ 2.8k/14bits

240MHz / SDR50
Highest CropMood speed reached for a 2 minutes Clip: 83.72 MB/s @ 2.8k/14bits  (Consistent)

(Item "B" in the picture below)
(https://i.ibb.co/G5QgbN0/Cards-Tested.jpg) (https://ibb.co/0GyPdvc)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/1MBsfp8/Sandisk-Extreme-Pro-200-256-GB-SD-240-MHz-SDR104-Long.jpg) (https://ibb.co/gdQvMXz)

(https://i.ibb.co/6Z9nQKG/Sandisk-Extreme-Pro256-GB-MLVApp-V2.png) (https://ibb.co/9vK8PxB)

The updated method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

The card is mostly consistent (Succeeded at 1 data rate but then failed at the same data rate in another take, at around 85 MB/s.)
The Card was consistent at a  slightly lower data rate (83 MB/s). See Picture.
I don't think the 2 MB/s difference is relevant as it could be related to the camera (In line with the other cards tested, and I did a lot more passes this time)

I did not get any Safe Mode trigger or corrupted Frames.
The Max consistent speed is the same, whether in SDR 104 or SDR 50 mode.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: mlrocks on January 07, 2024, 02:38:49 AM
<MOD: Full quote deleted.>


"then back to the camera to make it bootable."

how can you make sd card bootable in camera?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 07, 2024, 02:57:43 AM
Quote from: mlrocks on January 07, 2024, 02:38:49 AM


"then back to the camera to make it bootable."

how can you make sd card bootable in camera?

Once you copied the ML files to the card, You put it in the camera and do a firmware update (It won't do a firmware update, but it will make the card boot to ML)
You follow the instructions and restart when asked.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 07, 2024, 03:19:51 AM
Quote from: names_are_hard on January 04, 2024, 06:14:53 PM
Some (fairly reliable) people have also reported low-level format in cam restoring a slow card back to fast.  Possibly, this is better if done at the overclock you intend to run the card at.  I'm not convinced by that last part, but it might be worth testing.  The low-level format restoring a slow card I've seen multiple people report good results with.

I'm not sure why a low-level format should help.  Possibly, because writing to erased blocks is fast, but erasing blocks is slow?  If the drive controller knows which blocks are erased, this might avoid the need for erases during recording.

I don't know about the magnitude of the speed gain when the blocks are pre-erased (format), but I know this:
Each time you insert the card in a different system, the file system is modified. The Mac writes a bunch of invisible dot files, Android writes extra folders both visible and invisible, etc.

After enough going back and forth (You need to transfer the files to your Mac or PC), the FAT gets corrupted (well, modified differently by a different OS, especially if you use the computer to erase the files)  and you slowly begin to get errors and corruptions, until the card needs a complete re-format and re-copying of all the files.

Ever since I started to Low-Level format before each recording session (There is a option to keep the ML files on the card. I don't know how they did it, but it works) I stopped having those kinds of problems. This is anecdotal of course, but I recorded about  7 TB over the course of the summer, most of it on the same 256GB card. So I guess this is a large enough sample.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: mlrocks on January 07, 2024, 03:58:54 AM
Quote from: gabriielangel on January 07, 2024, 02:57:43 AM
Once you copied the ML files to the card, You put it in the camera and do a firmware update (It won't do a firmware update, but it will make the card boot to ML)
You follow the instructions and restart when asked.
thx
you think this new way will increase sandisk extreme pro 256gb to 85MB/s? mine is about 66 MB/s now.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 07, 2024, 04:52:54 PM
There's only one way to find out! :)
But maybe you could post your card's CID, along with those who reported the same issue, so we can see if those cards are like mine. Prior to the latest Overclock module, I had two Extreme Pro 128's, and only one of those worked.

There is something else I don't have a way to test, but maybe some cards draw more current than others? I use a Dummy Battery and power Bank which outputs  2A @ 9V, 22.5W.
Have you tried with the original Canon battery with a full charge?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: mlrocks on January 08, 2024, 05:29:49 AM
Quote from: gabriielangel on January 07, 2024, 04:52:54 PM
There's only one way to find out! :)
But maybe you could post your card's CID, along with those who reported the same issue, so we can see if those cards are like mine. Prior to the latest Overclock module, I had two Extreme Pro 128's, and only one of those worked.

There is something else I don't have a way to test, but maybe some cards draw more current than others? I use a Dummy Battery and power Bank which outputs  2A @ 9V, 22.5W.
Have you tried with the original Canon battery with a full charge?

yes, i used original canon batteries in battery grip on 650d. but 100d i use only one fully charged battery.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: mlrocks on January 08, 2024, 05:34:16 AM
Quote from: gabriielangel on January 07, 2024, 04:52:54 PM
There's only one way to find out! :)
But maybe you could post your card's CID, along with those who reported the same issue, so we can see if those cards are like mine. Prior to the latest Overclock module, I had two Extreme Pro 128's, and only one of those worked.

There is something else I don't have a way to test, but maybe some cards draw more current than others? I use a Dummy Battery and power Bank which outputs  2A @ 9V, 22.5W.
Have you tried with the original Canon battery with a full charge?

so the new way is
1. format the card on pc/mac as a new card.
2. low level format the card in ml camera with 240 mhz sd card overclock module activated.
3. on pc/mac copy the ml files to the sd card.
4. firmware update to make the sd card bootable
5. reformat the sd card before everytime use it

then it can reach 85mb/s for sandisk extreme pro 256gb.

do i miss anything here? thx. 85 mb/s is a big jump.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 08, 2024, 03:33:42 PM
Quote from: mlrocks on January 08, 2024, 05:34:16 AM
so the new way is
1. format the card on pc/mac as a new card.
2. low level format the card in ml camera with 240 mhz sd card overclock module activated.
3. on pc/mac copy the ml files to the sd card.
4. firmware update to make the sd card bootable
5. Reboot the camera at least twice before the first use.
6. reformat the sd card before everytime use it (with 240 mhz sd card overclock module activated)

then it can reach 85mb/s for sandisk extreme pro 256gb.

do i miss anything here? thx. 85 mb/s is a big jump.

Nowhere did I (Or anyone else)  say that this method will make the card work. This method avoids the problems related to card corruption. And based on a whole summer's worth of almost daily shoots, it works.
But In your case, it could be that your card is from a different batch. (See my previous post about the 128GB Extreme Pro)

So the first thing you should do is to try the method yourself.
Use Danne's version of crop mood https://bitbucket.org/Dannephoto/magiclantern_hg_02/downloads/ (https://bitbucket.org/Dannephoto/magiclantern_hg_02/downloads/)  you won't have to scratch your head or watch the video to make it work.
You copy the files and it's ready to shoot.
(You may have to enable the bench.mo module manually though)
The whole process should take you less than 10 minutes.


Edit: I just realized that you are not using the eos m, so Danne's custom build won't work for now.
You'll need to use bilal's original version. All the info you need to set it up is in the first few posts in this thread: https://www.magiclantern.fm/forum/index.php?topic=26851.msg242974#msg242974 (https://www.magiclantern.fm/forum/index.php?topic=26851.msg242974#msg242974)

I added : 5. Reboot the camera at least twice before the first use.
Some settings are quite persistent on the Eos M (So maybe it's the same thing for the 650D). And for some reason, the new settings don't always get transferred to the cam's non-volatile memory on the first try.  (Maybe someone who knows why that is could chime in??)

Also, after the low-level format, you'll see the light flashing for a few seconds. You should wait for it to stop flashing before going further.

Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on January 08, 2024, 03:39:49 PM
After step 1 you will find it impossible to proceed with step 2 as specified because you can't use ML's oc without ML running and ML cannot run because ML was deleted in step 1.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 08, 2024, 06:44:11 PM
Speed Test: Sandisk Extreme PRO SDXC UHS-I 128GB
R170/W90
Product Number: SDSSDXXY-128G-GN4IN
MDT: 2021/11

240MHz / SDR104
Highest CropMood speed reached for a 1 minute Clip (Peak Orange):  86.73 MB/s @ 2.8k/14bits. (Reached 87.1 MB/s once)
Highest CropMood speed reached for "Continuous"   (Steady Green): 76.53 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 83 MB/s @ 2.8k/14bits


(Item "A" in the picture below)
(https://i.ibb.co/G5QgbN0/Cards-Tested.jpg) (https://ibb.co/0GyPdvc)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/6Yyt6hy/Sandisk-Extreme-Pro-170-128-GB-SD-240-MHz-SDR104-Long.jpg) (https://ibb.co/b16vph6)

(https://i.ibb.co/ZTFZqns/Sandisk-Extreme-Pro128-GB-MLVApp-V2.png) (https://ibb.co/7yZ9swc)

The updated method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

The card is consistent.
I did not get any Safe Mode trigger or corrupted Frames.

After a few tests, I realized that, although the "Frame Order Error" message doesn't result in corrupted clips when it is displayed, in such a case the possible recording length of the clip will be shortened.
Recording another clip right after will work as expected.

If there is a module enabling the logging of everything  available, let me know. I would then be able to zero in on the issue.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 09, 2024, 01:00:40 AM
Speed Test: Sandisk Extreme microSDXC UHS-I 256GB
R160/W90  Ref. Here (https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/sandisk/product/memory-cards/extreme-uhs-i-microsd/data-sheet-extreme-uhs-i-microsd.pdf)
(The newer ones are rated R190/W130) Ref. Here (https://www.westerndigital.com/en-ca/products/memory-cards/sandisk-extreme-uhs-i-microsd?sku=SDSQXAV-256G-GN6MA)

Product Number: SDSQXA1-256G-GN6MN
PNM: SN256
PRV: 8.6
MDT: 2021/12

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 80.29 MB/s @ 2.8k/12bits (Mostly Consistent)
Highest CropMood speed reached for "Continuous"   (Steady Green): 71.17 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 82 MB/s @ 2.8k/14bits
240MHz / SDR104

(Item "E" in the picture below)
(https://i.ibb.co/G5QgbN0/Cards-Tested.jpg) (https://ibb.co/0GyPdvc)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/ry8c8WJ/Sandisk-Extreme-256-GB-m-SD-240-MHz-SDR104-Long.jpg) (https://ibb.co/ggqPqnC)

(https://i.ibb.co/nBMHjbN/Sandisk-Extreme256-GB-MLVApp-V2.png) (https://ibb.co/hcFPVyJ)

The updated method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

The card could not handle 2.8k/14bits, but worked well at 2.8k/12bits.
The card was mostly consistent, and did no go into safe mode.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: names_are_hard on January 09, 2024, 02:07:41 AM
Quote from: gabriielangel on January 08, 2024, 06:44:11 PM
If there is a module enabling the logging of everything  available, let me know. I would then be able to zero in on the issue.

Can you define "everything available"?  I might be able to add logging if I knew what you wanted to be logged.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 09, 2024, 03:54:55 AM
Quote from: names_are_hard on January 09, 2024, 02:07:41 AM
Can you define "everything available"?  I might be able to add logging if I knew what you wanted to be logged.

You (And the other Devs) will need to validate what I am saying here, as I do not code.

So I will describe the problem: You record let's say 10 clips, then all of a sudden, there is a "Frame Order Error slot XXX frame XXX expected XXX". This stops recording prematurely, but usually recording again will work for a while until the error happens again.

What happens between recordings is checking for focus (x10 zoom), toggling false colors on/off, changing shutter or aperture; or going inside the ML menu.
But I just tried now, and even if I only hit the record button on and off, and nothing else, until it happens, it will happen.

It seems to happen more often when you are close to, or when you hit the red light (Recording Indicator Green, Orange, Red)

So I'd say logging SD Card Access, compression tasks, live view tasks and keystrokes, as certain keys trigger a lot of state changes. I use Kill Global Draw on, so besides the recording counter, nothing else shows up on screen.

I don't know exactly which task triggers the error, as no error log is written when this particular error occurs.

When the error occurs, it lasts about 5 seconds before recording stops. So If I could hit a key as soon as it happens. It could write some sort of flag? This would narrow the search area.

Maybe writing the CPU usage of each task to disk as soon as I hit the key could help zero in on the offending module?
Title: Re: SD Overclocking - DIGIC 5 only
Post by: mlrocks on January 09, 2024, 04:26:37 AM
Quote from: gabriielangel on January 08, 2024, 03:33:42 PM
Nowhere did I (Or anyone else)  say that this method will make the card work. This method avoids the problems related to card corruption. And based on a whole summer's worth of almost daily shoots, it works.
But In your case, it could be that your card is from a different batch. (See my previous post about the 128GB Extreme Pro)

So the first thing you should do is to try the method yourself.
Use Danne's version of crop mood https://bitbucket.org/Dannephoto/magiclantern_hg_02/downloads/ (https://bitbucket.org/Dannephoto/magiclantern_hg_02/downloads/)  you won't have to scratch your head or watch the video to make it work.
You copy the files and it's ready to shoot.
(You may have to enable the bench.mo module manually though)
The whole process should take you less than 10 minutes.

I added : 5. Reboot the camera at least twice before the first use.
Some settings are quite persistent on the Eos M (So maybe it's the same thing for the 650D). And for some reason, the new settings don't always get transferred to the cam's non-volatile memory on the first try.  (Maybe someone who knows why that is could chime in??)

Also, after the low-level format, you'll see the light flashing for a few seconds. You should wait for it to stop flashing before going further.

thx a lot. will try. exciting.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: names_are_hard on January 09, 2024, 04:29:27 AM
Probably all you need to try is low-level format card in cam, and choose to keep ML.

All the other steps, you don't care about.  You're not trying to make comparable benchmarks each time.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: names_are_hard on January 09, 2024, 04:42:59 AM
Quote from: gabriielangel on January 09, 2024, 03:54:55 AM
You (And the other Devs) will need to validate what I am saying here, as I do not code.

So I will describe the problem: You record let's say 10 clips, then all of a sudden, there is a "Frame Order Error slot XXX frame XXX expected XXX". This stops recording prematurely, but usually recording again will work for a while until the error happens again.

As you describe it, this is too complicated for me, I don't know the mlv_lite module well and don't want to spend a lot of time in this area.

The "frame order error" occurs here, in mlv_lite.c:

3538                 if (slots[slot_index].frame_number != last_processed_frame + 1)
3539                 {
3540                     bmp_printf( FONT_MED, 30, 110,
3541                         "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number,      last_processed_frame + 1
3542                     );
3543                     beep();
3544                 }
3545                 last_processed_frame++;


This is in a loop designed to mark frames as free so the memory can be re-used.  It suggests there is a logic error around which frames have been written to disk, and can thus be freed.  Annoyingly, the exact same error text is used in a second place, so that's also possible (never use the same error message twice :( ).

I wouldn't expect the recording to stop because of this, it looks non-fatal.  You say it can take 5 seconds before recording stops.  I think the recording stopping is not *directly* related to this error.  It could easily still be a symptom of whatever the root cause is.  If a dev with more raw video experience wants to look at this, working out why the slot ends up being freed when it shouldn't is the obvious place to start.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 09, 2024, 11:57:39 AM
Quote from: mlrocks on January 09, 2024, 04:26:37 AM
thx a lot. will try. exciting.

I overlooked one thing: You seem to be using the 650D and 100D exclusively, So you won't be able to use Danne's custom build for now.

Therefore, You'll have to use bilal's original build.
You can follow the instructions here to enable all the modules: https://www.youtube.com/watch?v=V0M7n2cAHMM

Sorry about that!
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 09, 2024, 12:08:20 PM
Quote from: names_are_hard on January 09, 2024, 04:42:59 AM


I wouldn't expect the recording to stop because of this, it looks non-fatal.  You say it can take 5 seconds before recording stops.  I think the recording stopping is not *directly* related to this error.  It could easily still be a symptom of whatever the root cause is.  If a dev with more raw video experience wants to look at this, working out why the slot ends up being freed when it shouldn't is the obvious place to start.

Thanks for looking into it.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Danne on January 09, 2024, 02:32:57 PM
Quote from: names_are_hard on January 09, 2024, 04:42:59 AM
As you describe it, this is too complicated for me, I don't know the mlv_lite module well and don't want to spend a lot of time in this area.

The "frame order error" occurs here, in mlv_lite.c:

3538                 if (slots[slot_index].frame_number != last_processed_frame + 1)
3539                 {
3540                     bmp_printf( FONT_MED, 30, 110,
3541                         "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number,      last_processed_frame + 1
3542                     );
3543                     beep();
3544                 }
3545                 last_processed_frame++;


This is in a loop designed to mark frames as free so the memory can be re-used.  It suggests there is a logic error around which frames have been written to disk, and can thus be freed.  Annoyingly, the exact same error text is used in a second place, so that's also possible (never use the same error message twice :( ).

I wouldn't expect the recording to stop because of this, it looks non-fatal.  You say it can take 5 seconds before recording stops.  I think the recording stopping is not *directly* related to this error.  It could easily still be a symptom of whatever the root cause is.  If a dev with more raw video experience wants to look at this, working out why the slot ends up being freed when it shouldn't is the obvious place to start.

I commented out this from eos m. It never caused any real life issues to me.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: names_are_hard on January 09, 2024, 03:37:49 PM
Printing the warning doesn't stop recording.  So yes, I would not expect commenting it out to cause a *problem*.

But gabriielangel finds that recording is stopped some time after the warning.  You won't have any clues, because you hid it.  So it's not the best idea, even though it doesn't directly cause a problem.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Danne on January 09, 2024, 03:53:55 PM
Probably not. I did not investigate this further unfortunately so can't say much about it.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: mlrocks on January 10, 2024, 03:51:48 AM
Quote from: gabriielangel on January 09, 2024, 11:57:39 AM
I overlooked one thing: You seem to be using the 650D and 100D exclusively, So you won't be able to use Danne's custom build for now.

Therefore, You'll have to use bilal's original build.
You can follow the instructions here to enable all the modules: https://www.youtube.com/watch?v=V0M7n2cAHMM

Sorry about that!

i don't have m. thx for the video.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 11, 2024, 12:20:16 AM
A Little Primer On Exposure

In order to come up with a representative enough chart for possible record times for a given resolution (To help choose the right card for a given budget),
I needed to establish a few things first. As data rates are heavily influenced by the average exposure of the scene, we need to have a good idea of how low we can go before getting too much noise.

So, the first thing is to find out where the sweet spot is; the goal being to get the lowest decent Data Rate while keeping noise at bay.

Once you factor in all the hours needed to really understand those cams (Eos m, in my case), it's definitely not a $300 cam anymore! Hopefully, those posts will help those beginning getting straight to making movies...

DISCLAIMER: This is not an absolute Dynamic Range test. I did not have access to the proper tools to get to that level. So I used a computer monitor, which already has some sort of polarizer applied to it, plus the Variable ND on the lens, which is 2 stacked polarizers. So the illumination is uneven.
But for the present purposes, it is fine.

The chart I used:
(https://i.ibb.co/pd64jMD/00a-DR-Chart-Scope4-Clean.png) (https://ibb.co/2d0c3C2)
I created the file on Photoshop Mac. If you use a PC, you'll need to adjust your brightness and contrast a little, as the gamma is different (This matters only if you plan on replicating the test.)
There is 1/2 stop difference between each level on top, and 1/4 stop difference at the bottom.

The values when using the RGB Spotmeter on the EOS M (In Red):
(https://i.ibb.co/g6cSxnr/00b-DR-Chart-Spot-Meter-Values.png) (https://ibb.co/hCTBSqg)
I recorded the chart slightly off-focus, to eliminate the Moiré (it is an LCD Monitor right?!)
Those are the values you get when exposing the Chart ETTR without any highlight clipping.

The Marshall False Color Key:
(https://i.ibb.co/9hjLQjt/00c-Marshall-False-Color-Key.jpg) (https://ibb.co/PM8nq8Q)
To help decide how low of an exposure we can use to get a good Signal-To-Noise Ratio (SNR) we'll use the False Colors on the EOS M, set to Marshall. In that case, The Pink has been replaced by Orange, and Black means your Highlights are clipped. Magenta (Fushia on that picture) means your Blacks are clipped. Orange is the ballpark for well exposed skintones.

So given this scene:This is taken with a phone. I will put the actual MLV output in the next post.
(https://i.ibb.co/c1gtWvy/01-On-Screen.jpg) (https://ibb.co/KX07tG5)

With False Colors applied, we get this:
(https://i.ibb.co/RjXHtVX/03-ETTR.jpg) (https://ibb.co/z63fC03)
(Exposed ETTR, whithout any clipping)

If we were to clip the highlights:
(https://i.ibb.co/qNX8SMk/02-Clipped-Highlights.jpg) (https://ibb.co/McWHvDM)
If you do this on the EOS M, you'll only get a few seconds of recording time. You can have Red, but Black is a no-no.
Black means your highlights are clipped.

With a slight underexposure:
(https://i.ibb.co/KX8C0rG/04-Slight-Under-Exposure.jpg) (https://ibb.co/6mq6nWZ)
In that case, anything in the area below the 9 1/4 Stop will give you an ugly kind of noise where the Magenta dots appear,  if you try to do any shadow recovery (Actual examples in next post).

Heavily Underexposed:
(https://i.ibb.co/k5xNg8Y/05-Too-Under-Exposed.jpg) (https://ibb.co/QpfyFNs)
Here, anything lower than 8 stops below the highlights won't look nice if boosted in post.
So, you need to leave Magenta for only the darkest blacks you know you'll never try to recover!

As a last resort:
(https://i.ibb.co/gWNX7zN/06-Lowest-Iwould-Ever-Go.jpg) (https://ibb.co/kDjryQj)
This would give you 8 clean stops of Dynamic Range, plus a full stop of shadows you can recover relatively cleanly. If you get a little Magenta in a very dark area (For example, under a tree) no one will notice. If I do something serious, I'll add some kind of reflector or light to bring the shadows up a little. If I just walk by for fun, I may just choose a better spot :)

At night:
(https://i.ibb.co/Pw70DVV/07-Last-Resort.jpg) (https://ibb.co/qktVJ22)
If this was a nightly or dimly-lit scene, it could still look good, as long as the subject is around the 2nd stop and you crush the lowest shadows a little.

So the next step is to find out when the noise becomes obtrusive... In the next post.

Here's a very good short Youtube link if you want to see false colors in action, in a more natural context: https://www.youtube.com/watch?v=nncazai0Ei4 (https://www.youtube.com/watch?v=nncazai0Ei4)






Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 11, 2024, 03:58:23 AM
Exposure vs Noise (EOS M)

(It will be easier if you download all the images first. The file names are descriptive.)

The end goal here is to come up with a realistic minimum average exposure, to establish an approximate Data Rate for any given Resolution / Bitrate / Framerate.
That way, someone could decide if they want to invest $100 for a lexar card, or settle for a $15-$20 Samsung if it gets the job done at one's preferred resolution.

So, to find out how low we can underexpose to save some bandwidth, while keeping a good image quality...

Given that the chart I used in the previous post:
(https://i.ibb.co/pd64jMD/00a-DR-Chart-Scope4-Clean.png) (https://ibb.co/2d0c3C2)

Is exposed like this:
(https://i.ibb.co/RjXHtVX/03-ETTR.jpg) (https://ibb.co/z63fC03)
(Exposed ETTR, whithout any clipping)

We get images like these:
(https://i.ibb.co/m5yyS97/09a-All-Files.png) (https://ibb.co/7tppbnF)

14bits
(https://i.ibb.co/g3RW6mR/09-14bits.png) (https://ibb.co/g3RW6mR)

12bits
(https://i.ibb.co/4gLXdTR/10-12bits.png) (https://ibb.co/4gLXdTR)

11bits
(https://i.ibb.co/K6z0zyT/11-11bits.png) (https://ibb.co/K6z0zyT)

10bits
(https://i.ibb.co/YLV8qky/12-10bits.png) (https://ibb.co/YLV8qky)

14 bits, with exposure lowered in MLV app to check highlights.
(https://i.ibb.co/hFVMWP5/13-14bits-Low-Exposure.png) (https://ibb.co/hFVMWP5)

10 bits, with exposure lowered in MLV app to check highlights. Highlights are comparable to 14 bits.
(https://i.ibb.co/1dgYsZv/16-10bits-Low-Exposure.png) (https://ibb.co/1dgYsZv)

So far, we can see that without any shadow recovery, noise starts being visible around Stop 6. Slightly before 6 at 10bits, but barely (Not easy to judge with a static frame)
This will vary according to the color of course.
A slight red tint appears in the low level noise at 11bit, and becomes obvious at 10bits.

Let's investigate the shadows. We'll add a shadow boost of 100 in MLV app:

14bits with Shadows +100
(https://i.ibb.co/br7czwG/17-14bits-100-Shadows.png) (https://ibb.co/br7czwG)

12bits with Shadows +100
(https://i.ibb.co/D9ZqW6B/18-12bits-100-Shadows.png) (https://ibb.co/D9ZqW6B)

11bits with Shadows +100
(https://i.ibb.co/9wY9M2J/19-11bits-100-Shadows.png) (https://ibb.co/9wY9M2J)

10bits with Shadows +100
(https://i.ibb.co/0Xfs8kf/20-10bits-100-Shadows.png) (https://ibb.co/0Xfs8kf)

Now we see a huge difference in the quality of the noise that is recovered along with the shadows.
at 14 bits, the noise starts to become really coarse at 9 3/4 below Highlight level.
at 12 bits, the coarseness starts at around 9, but there is a clear difference starting around  Stop 7 compared to 14 bits.
at 11 bits, the difference compared to 12 bits starts at Stop 7, but the coarseness start being visible at 8.
at 10 bits, noise becomes quite visible at Stop 6, at Stop 8 there is a lot more noise than at 11bits, and what is at 9 and below is uncomfortably coarse. Also, there is a red tint all over the shadows.

Let's see if using Darkframe subtraction improves the noise a bit.
I filmed clips with the cap on, one dark clip per bit rate. I exported those dark clips with the exact same 4 second length as MLV fast pass and then as MLV Averaged frame (To get same length averages).

14bits, shadows+100, Darkframe subtraction:
(https://i.ibb.co/V3JF3Rr/21-14bits-100-Shadows-Darkframe.png) (https://ibb.co/V3JF3Rr)

12bits, shadows+100, Darkframe subtraction:
(https://i.ibb.co/zSJD4Ry/22-12bits-100-Shadows-Darkframe.png) (https://ibb.co/zSJD4Ry)

11bits, shadows+100, Darkframe subtraction:
(https://i.ibb.co/Nj5gQCH/23-11bits-100-Shadows-Darkframe.png) (https://ibb.co/Nj5gQCH)

10bits, shadows+100, Darkframe subtraction:
(https://i.ibb.co/wL2n6wP/24-10bits-100-Shadows-Darkframe.png) (https://ibb.co/wL2n6wP)

10bits, shadows+60, Darkframe subtraction: (To see what it looks like in a slightly more realistic context)
(https://i.ibb.co/4NzqFQD/28-10bits-60-Shadows-Darkframe.png) (https://ibb.co/4NzqFQD)

At 14bits and 12 bits, the difference is subtle, but still noticeable as the red tint is removed.
But at 11bits, slight vertical lines begin to appear and at 10bits, those vertical lines become quite obvious.

So, what does this all means?

1st thing is to avoid shooting at 10bits if you plan on making any color or exposure modifications in post. I would start with 11bits minimum and prefer at least 12bits.
(This has been stated before by several members on this forum).
Because exposing close to ETTR maximizes the amount of light the sensor receives, and the bit reduction happens once the signal is digital Edit: This is not 100% exact, it appears that the bit depth reduction is obtained by gain reduction at the analog stage and a 14bit container is always used. (This is one of the reasons why the quality of the noise degrades rapidly at the lower end of the scale, as bits of the shadows sharply get crushed to pure black. You end up with larger black blocks once you bring it all up);
I am under the impression that adding a 13 bits option would be a good idea to help gain a few MB/s, as this would still allow to ETTR while allowing a slightly lower Data Rate, as opposed to significantly Underexposing 14bits while recording.

Now, we want to make sure that Skintones are always above the visible noise level, which is between Stop 5 and Stop 6 below highlights;
But we also want to keep the grainy parts as low as possible...

The EOS M is rated at 11.2 Stops of DR according to DxOMark :https://www.dxomark.com/Cameras/Canon/EOS-M---Measurements (https://www.dxomark.com/Cameras/Canon/EOS-M---Measurements)
We definitely want anything important enough to be seen to be above Stop 9, because of the coarse noise happening below that point.

Based on the false color images above, the average showing up on the Histogram should be above 2.6 to minimize the grainy noise in the visible parts.
I know, by experience, that outside on a sunny day, I can shoot 30 seconds + whatever, at 2.8k/14bits with histogram showing 1.2 (up to 0.8 every now and then) As long as the highlights are mostly yellow with a hint of red (On the false colors).

So the target seems to be somewhere between Histogram 2.6 and 1.0 (Histogram set to ETTR Hint).

Again, the goal here is to come up with a realistic minimum average exposure, to establish an approximate Data Rate for any given Resolution / Bitrate / Framerate.

Your comments are very welcome... :)






Title: Re: SD Overclocking - DIGIC 5 only
Post by: benoit on January 11, 2024, 09:43:55 AM
Thanks a lot for a such good detailled information !   :)
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on January 13, 2024, 07:20:40 PM
Because Samsung PRO Plus 128 GB has been reported as unstable (in "real life", benchmark looks good) both cards (2021 and 2023 edition) got a "NO" flag in Wiki card list (https://wiki.magiclantern.fm/cards_240mhz).
Better save than sorry.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 19, 2024, 07:10:54 AM
Here is a tentative chart for the recording possibilities of different cards at various resolutions, for the EOS M.
If you own one of the cards I tested (Listed in the second sheet), you could try recording a scene or two, to see if the predictions are robust.
All the instructions are in the spreadsheet.

To download it :  Here (https://bit.ly/42rumA9)

EDIT: I added a Scene complexity Dropdown menu, to have an idea of how scene complexity can affect data rates. The chart is a lot more complex than I would have liked to see it, if I were to look at it for the first time, but I think it beats having to learn this by trial-and-error :)
EDIT2:There is now a dropdown menu to choose cards instead of entering the numbers manually.

Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 24, 2024, 03:13:26 PM
Speed Test: Samsung Evo Plus microSDXC UHS-I 512GB
R130/W130
Product Number: MB-MC512KA/CA
MDT: 2023/07

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 86.67 MB/s @ 2.8k/14bits
Highest CropMood speed reached for "Continuous"   (Steady Green): 76.79 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 89.3 MB/s @ 2.8k/14bits
240MHz / SDR104

(https://i.ibb.co/vXVf3sR/Samsung-Evo-Plus130-512-GB.jpg) (https://ibb.co/VwJbgH5)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/S57fh8j/Samsung-Evo-Plus130-512-GB-m-SD-240-MHz-SDR104-Long.jpg) (https://ibb.co/FmqHSMr)

(https://i.ibb.co/TgjCFj7/Samsung-Evo-Plus130-512-GB-MLVApp.png) (https://ibb.co/RDr5Vrs)

The method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

I recorded more clips this time, to hopefully expose any Thermal Throttling or Burst Speed Wizardry implemented by the manufacturer. I recorded about 10 clips of various lengths (Light orange / red) prior to beginning the test (Not in the picture, to save on transfer time)

This card did not exhibit any problems and was consistent. Unlike the other cards tested so far, this card did not trigger a single "Frame Error" message during the tests.
This card is about the same price as a Lexar Silver Series 1066x 256GB, for TWICE the size, and the performances seem comparable so far.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on January 24, 2024, 03:24:49 PM
R130/W130 for EVO Plus?

There are no decent spec sheets by Samsung for sure!
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 24, 2024, 03:56:30 PM
Quote from: Walter Schulz on January 24, 2024, 03:24:49 PM
R130/W130 for EVO Plus?

There are no decent spec sheets by Samsung for sure!
At least they wrote something :)  Some cards say "Transfer Speeds of up to 120 MB/s!" Then after looking for write speed everywhere on the package, what you find somewhere on the back:
Numbers stated are for read speeds of up to 120MB/s, when using a compatible reader. Write speeds lower.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Walter Schulz on January 25, 2024, 08:29:49 AM
Please check wiki entry: K means 2021 edition. 2023 has S suffix. Date in third culumn doesn't refer to production or purchase.
A (or other suffixes like EU, US) refer to market region and should be ommited in this column.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 25, 2024, 04:34:27 PM
Quote from: Walter Schulz on January 25, 2024, 08:29:49 AM
Please check wiki entry: K means 2021 edition. 2023 has S suffix. Date in third culumn doesn't refer to production or purchase.
A (or other suffixes like EU, US) refer to market region and should be ommited in this column.

Well, I took the MDT  from the CID Info Menu (2023/07).
The PRV (Revision Number) is: 3.0
The number written behind the microSD card is MB-MC512K
The Number written next to the UPC code (See on the picture): MB-MC512KA/CA .
I made the assumption that the CA (Canada) was the market part, as there is also a model called MB-MC512KA/EU (!)

But you're right, it is 2021 according to this link: https://www.samsung.com/uk/memory-storage/memory-card/evo-plus-512gb-microsd-card-mb-mc512ka-eu/

Will fix shortly, thanks.

Have a look at this (I could not find the rev3.0), maybe this could be of use to those who know how to tweak the controller to get more out of the card?

https://download.semiconductor.samsung.com/resources/data-sheet/2023_samsung_data_sheet_microsd_card_evo_plus_rev.1.0.pdf (https://download.semiconductor.samsung.com/resources/data-sheet/2023_samsung_data_sheet_microsd_card_evo_plus_rev.1.0.pdf)

At the very bottom (Fine print) it says:
2)SDR104: 1.8V Signaling, Frequency up to 208 MHz, up to 104MB/sec, Max. Current Consumption 800mA (varies by test conditions)
3) DDR200 : 1.8V Signaling, Frequency up to 208 MHz, up to 200MB/sec, Max. Current Consumption 800mA (varies by test conditions)


Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on January 30, 2024, 11:06:56 PM
Speed Test: Samsung EVO Plus microSDXC UHS-I 128GB
R130/W130
Product Number: MB-MC128KA / MB-MD128S (Important precision, thanks @walter)
MDT: 2023/05

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 62.64 MB/s @ 2.8k/14bits
Highest CropMood speed reached for "Continuous"   (Steady Green): 55.13 MB/s @ 2.8k/12bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 66.4 MB/s @ 2.8k/14bits
240MHz / SDR104

(Item "H" in the picture below)
(https://i.ibb.co/G5QgbN0/Cards-Tested.jpg) (https://ibb.co/0GyPdvc)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/h9gmHJZ/Samsung-EVO-Plus-130-128-GB-m-SD-240-MHz-SDR104-Long-V2.jpg) (https://ibb.co/Sy6304d)

(https://i.ibb.co/yh61kwJ/Samsung-Evo-Plus130-128-G-MLVapp-V2.png) (https://ibb.co/mv9wyL7)
(https://i.ibb.co/LpdJMzr/Data-Rate-Tests-for-Red-All-Cards1.png) (https://ibb.co/LpdJMzr) (https://i.ibb.co/n8XzjDz/Data-Rate-Tests-for-Red-All-Cards2.png) (https://ibb.co/n8XzjDz)

The method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

To hopefully expose any Thermal Throttling or Burst Speed Wizardry implemented by the manufacturer, I recorded about 10 clips of various lengths (Light orange / red) prior to beginning the test (Not in the picture, to save on transfer time)

*Something new, I added a "Red Speed" to the test, which is the maximum speed which a card can sustain in the red for 15 seconds.
This doesn't make much of a difference when recording at 12 bit, but with the high resolution presets at 14 bit, it shows if the card can tolerate sudden changes in the scene (Like confetti exploding or anything entering the scene that's brighter than when you first hit record) I updated the other card tests as well.

This card exhibited a weird behaviour in which every other recording gave a "Frame Order" error as soon as I recorded clips over 30 seconds, in the Orange zone !?
If I recorded in the Green zone, no problem. This happened recording at 2.5k 2.39:1, 14bits and 12 bits, 1 minute clips and clips over 30 seconds.
I tried at 240MHz, 192MHz, SDR104 and SDR50; No difference.

As @walter pointed out, the model number written on the card itself makes a difference, but this number isn't written anywhere on the package, which looks identical to the package of K variants, at least here in Canada. So there is no way to know which variant (K or S) you are getting before opening the package.

So far, the Samsung cards (Along with the Lexar) have the steadiest, constantly repeatable across takes speed.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on February 06, 2024, 03:05:23 AM
I updated my EOS M Data Rate Estimator Chart: Here (https://bit.ly/42rumA9)
You need to save it locally or copy it to your google drive in order to use it properly.

No need to enter the numbers manually, you can now select one of the tested SD/micro SD cards using a Dropdown Menu.
Hopefully, more people will test their cards so that we can have a larger database.

What I have gathered so far is that, unless one plans on using only the 3x3 Modes, a card which can remain in the Green Zone at 76 MB/s or more
is required to be able to use most presets comfortably @23.976 fps. The higher resolution presets require one of the fastest cards.
It appears that some manufacturers have some cards run with a series of short "Bursts", so that those look good on benchmarks.
Although ok for file transfers, what we need here are steady sustained speeds! So, a different kind of benchmark is necessary.

If the quality of the image is important, 14 and 12 bit are to be preferred. At 12 bit, due to the way the bit depth is reduced,
we are already running almost 2 stops below optimal exposure. So trying to underexpose further to lower the data rate is not ideal (Would progressively make the image noisier).
I have seen a few posts on the facebook group where people are complaining about the noise, not knowing that recording at 10 bits is not ideal, and requires a very bright scene to look decent.
In such cases, choosing a faster card / higher bit depth is the best cure. This may all be obvious for most people on this forum, but it is definitely not so for someone first putting their hands on the cam!
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on February 13, 2024, 07:06:49 AM
By how much can we reduce the Data Rate with Underexposure?
Another chapter in the quest for Data Rate Optimization on Eos m.

Several people report long recording times on the various social media, but after close inspection, one sees that the footage is often severely underexposed.
So, how much gain (or rather, loss) can we expect, while keeping the quality as high as possible? See the attached animated .png below.

at 2.8k 1:1 14bit, The Data Rate can be reduced by 7%, by underexposing 1 Stop; and 13% by underexposing 2 stops (Will vary slightly depending on the scene being shot, of course).
Because the Bit Depth reduction is obtained by reducing the Analog Gain in the camera, Underexposing 2 stops equals switching to 12bit (3 Stops=11bit, 4 Stops=10bit).
BUT, as you can see in the image below, Underexposing is a lot noisier than reducing the bits with Analog Gain(!)
Therefore, as soon as you need to underexpose significantly, it is better to lower the Bit Depth to the closest match instead.

The color chart was lit by 2 LED panels, and was exposed so that the white square was right below the clipping point.
I recorded 1 clip per aperture, in 1/3 stop decrements. I then boosted the exposure accordingly in MLV App to normalize the smaller aperture clip's brightness against the first clip (Wide Open).
This clearly shows that the clip recorded at 12bit (12bit-ML) is cleaner than the clip recorded 2 stops underexposed (12bit-Equivalent). The same goes for 10 and 11bit at the transition points.
Also, the Data Rate is lower for the clips recorded with Analog Gain Bit Depth reduction (When you change the Bit Depth in the Movie / Crop Mood menus), because of the lower noise.
You can also see that at image 13, 4 Stops below, the noise is already quite apparent.

So, again, it would be a good idea to add a 13bit option, as this would allow a significant reduction of the Data Rate, while having almost no impact on image quality.
This would also give about 1 extra stop of cleaner exposure (Compared to underexposing by 1 Stop).
With a good card, it would be easier to reap the benefits of recording at higher Resolutions / Bit Depth / Exposures.
(Click on the image to load the full resolution animated version)


(https://i.ibb.co/3mPLwBY/ML-Exposure-Datarate2.png) (https://ibb.co/CKSTqhs)

All the images in a .zip file, if you get dizzy: https://bit.ly/3SB7FoP (https://bit.ly/3SB7FoP)
Title: Re: SD Overclocking - DIGIC 5 only
Post by: Danne on February 13, 2024, 08:35:28 AM
Interesting.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on March 27, 2024, 01:32:48 AM
Speed Test: Lexar Professional 1667x SDXC UHS-II 256GB
R250/W120
Product Number: LSD256CBNA1667 / A0185-V60-256BSL A (Written behind the Card)
MDT: 2020/12
PRV: 1.0

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 88.8 MB/s @ 2.8k/14bits
Highest CropMood speed reached for "Continuous"  (Steady Green): 78 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 91 MB/s @ 2.8k/14bits
240MHz / SDR104

(https://i.ibb.co/DR7CpzP/Lexar1667x-SD256-GB-Front.jpg) (https://ibb.co/QK6Ndp7)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/VY7tkkT/Lexar-1667x-256-G-SD-240-Mhz-SDR104-Long.png) (https://ibb.co/qYtd22p)

(https://i.ibb.co/DWSRS6z/Lexar-1667x-256-G-SD-240-Mhz-SDR104-MLVApp.png) (https://ibb.co/DWSRS6z)

The testing method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694


This card did not exhibit any problems and was consistent. 
The card triggered a few "Frame Error" messages during the test.

This card has a high enough Data Rate to handle good brightness at 2.5k 14bits / 2.8k 12bits 1:1.

So far, I have noted that the difference in data rate between the camera stopping by itself at around 1 minute, and it going past the 2 minutes mark is often of 1-2 MB/s for most cards.

Unless you plan on recording clips at 3K 1:1/12bits (or Bright 2.8k 1:1), Samsung EVO Plus, Sandisk Extreme Pro and Lexar Silver 1066x are a better value, as this is one of the most expensive cards.
Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on April 10, 2024, 09:39:23 PM
Speed Test: Samsung EVO Plus mSDXC UHS-I 256GB
R130/W130
Product Number: MB-MC256KA (MB-MC256K written behind the Card)
MDT: 2023/10
PRV: 3.0

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 87 MB/s @ 2.8k/14bits
Highest CropMood speed reached for "Continuous"  (Steady Green): 76.6 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 88.9 MB/s @ 2.8k/14bits
240MHz / SDR104 (Benchmark is identical with SDR50)

(https://i.ibb.co/87VnGBk/Samsung-Evo-Plus-256-G-MB-MC-256-K.jpg) (https://ibb.co/qrwLhn6)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/dB0pSsR/Samsung-Evo-Plus-256-GB-m-SD-SDR104-Long.jpg) (https://ibb.co/tBM2Sw9)
(https://i.ibb.co/cJQHvxk/Samsung-Evo-Plus-256-G-MB-MC-256-K-MLVa.png) (https://ibb.co/cJQHvxk)  (https://i.ibb.co/Yt9DB3C/Samsung-Evo-Plus-256-G-MB-MC-256-K-MLVb.png) (https://ibb.co/Yt9DB3C)   (https://i.ibb.co/3NcdPTx/Samsung-Evo-Plus-256-G-MB-MC-256-K-MLVc.png) (https://ibb.co/3NcdPTx)

The testing method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694


This time, I filled up the whole card's capacity with tests. This gave me more data: (The cam stops recording by itself)

30 Seconds: 87.6 MB/a
23 Seconds: 88.2 MB/s
18 Seconds: 88.6 MB/s
10 Seconds: 91 MB/s

The card is not as consistent as the Lexars in the red zone, but is very consistent in the Orange and Green Zones.
The performance is on par with the Sandisk Extreme Pro 256GB R200/W140, and costs 30% less even at regular price.
No Safe Mode triggers, no throttling, only a few sporadic "Frame Order" error messages (4 messages out of 90+ clips recorded), where the recording stops early.

Regarding the Benchmark:

The benchmark runs between 8-10 seconds and the number it gives is close to the 10 seconds results I got in the test. In this area, there is a lot of variability, except for the most expensive cards.
As discussed previously, a 1-2 MB/s difference can double the possible recording time at the higher end, and add up to several minutes in the low orange zone.

Most compatible "Pro" cards can perform reliably in the Green Zone (76-78 MB/s), but around this data rate, 1x3 4.4k/12bit and 1:1 2.5k/12bit are the highest settings where you'll get reliable recordings.

So, the most interesting zone to test is the Orange zone, where the cam can record continuously for 1 minute +. This is where the door opens to the highest resolutions.
1 minute is plenty to record about everything I see posted as examples on the facebook group...

Very few people will go through the trouble of testing the cards manually, and most people return the cards immediately if those don't work as expected. This makes it hard to know what can be done with a particular card, and those vary, sometimes substantially, between revisions.

So it would be nice if the Benchmark could test for SUSTAINED speeds for a given length of time,
which would make it a lot more useful and make it easier to build a reliable Card Database, as reporting would be simplified.




Title: Re: SD Overclocking - DIGIC 5 only
Post by: gabriielangel on April 26, 2024, 04:59:07 AM
Speed Test: SanDisk Extreme Pro SDXC UHS-I Memory Card 95 MB/s 128GB
R95/W90
Product Number: ?
MDT: 2018/11
PRV: 8.0
PNM: SR128

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 84.1 MB/s @ 2.8k/14bits
Highest CropMood speed reached for "Continuous"  (Steady Green): 73.7 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 86.4 MB/s @ 2.8k/14bits
240MHz / SDR104 (Benchmark is similar with SDR50)

(https://i.ibb.co/H7HGm4D/Sandisk-Extreme-Pro-128-GB-95-MBs.jpg) (https://ibb.co/VVqMX9W)

SDR104:
(https://i.ibb.co/tL6JrP6/Sandisk-Extreme-Pro-128-GB-95-MBs-SDR104-Long.jpg) (https://ibb.co/KKgrvzg)
SDR50:
(https://i.ibb.co/TvKpf5d/Sandisk-Extreme-Pro-128-GB-95-MBs-SDR50-Long.jpg) (https://ibb.co/6nvVM5L)

Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.

(https://i.ibb.co/Ry8N2Bz/Sandisk-Extreme-Pro-128-GB-95-MBs-MLVAppa.png) (https://ibb.co/Ry8N2Bz) (https://i.ibb.co/XzrNLVJ/Sandisk-Extreme-Pro-128-GB-95-MBs-MLVAppb.png) (https://ibb.co/XzrNLVJ)

The testing method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

There are several versions of this card; on some, the "V30" logo is absent on the label.
This card gave a single "Frame Order" error message for the whole testing session.
The card's behaviour is not very consistent when recording in the red zone, because a difference of .09 MB/s can vary the recording length between 45s and 1m30s.
The card didn't go into safe mode. The recording times are similar in SDR104 and SDR50 modes.