Best SD card for Canon EOS M + Magic Lantern

[wyup]

BTW, I tried these on my Sandisk Extreme PRO 64GB UHS-I U3 170 MB/s and it performed worse when I ran benchmarks (using 240 MHz overclock on 700D):

Write speed:
1024 bytes: 82 MB/s
2048 bytes: 88 MB/s
4096 bytes: 89 MB/s
8192 bytes: 90 MB/s (matches 128 KB)

Read speed:
All was ~106 MB/s, no differences here.

Thanks Bilal. I don't know which bench.mo you're using and in which mode.

As for me, at 8192 bytes sector, recording speeds are the same as always: 58-59 Mb/s max sustained. I can't record in 2.8k mode at 10-bit.

I see it's the same Digic 5 chip your 700D and EOS M, and released about the same year (2012/13). Why are your benchmarks so high? 90 MB/s compared to max 76 MB/s with my 200 MB/s card, 70 MB/s in Liveview/Movie mode. Are there many differences between your build and Danne's that can affect card writing performance?

I'd like to know what recording speeds do people get with their cards, specially Extreme 170 MB/s, to know if my mine is underperforming.

[wyup]

You don't get 72Mb/s in 192Mhz and 240Mhz don't work on eos m. Around 60 seems right.
If you do benchmark stay in photo mode and also push play button just before bench marking starts.

Anyway, now your result matches 100D/700D with (Sandisk 170MB/s) at the same settings. RAW video recording should get up to 70 MB/s too (at least this happens on 100D/700D).
And again, your SD card seems fine.

I don't know if this an issue related to what things Danne have changed in his custom build which prevent getting 70 MB/s write speed, it could be his implementation/changes to small/new hacks (hacks don't work in some cases while recording RAW video?), or maybe other things?.

Why do Danne and Bilal implementations differ so much with the same Digic5 chip to yield a ~10 MB/s actual recording speed difference between M and 100/700D, when we get the same Bilal's-modified benchmark speeds? Default benchmark speeds also differ a ~10-13 MB/s. Can we expect to equal out?

[theBilalFakhouri]

@wyup

Try this, it should give you the extra 10 MB/s when using 192 MHz overclock:

crop_rec_4k_mlv_snd_raw_only_2022Jul31.EOSM202.zip

Enable the hacks from RAW video submenu, then report back.

[Danne]

Share code please. Random builds without code is very bad practice.

[theBilalFakhouri]

It was online since 25/05/2022 . . check my implementation for the new hacks in 650D/700D source code, it's the same (you need to add EOS M addresses).

I don't keep the source code for these kind of tests, anyway I did this:

Code: [Select]

static REQUIRES(RawRecTask)
void hack_liveview_more()
{
/* Exlude Movie Crop Mode from the new hacks */
if (video_mode_crop)
{
return;
}

    if (more_hacks) /* excludes mcm mode on eosm */
{
void (*aewbSuspend)() =
cam_eos_m   ? 0xff2606f4 :
cam_5d3_113 ? 0xff23bc60 :
cam_5d3_123 ? 0xff23ff10 :
0;

void (*lvfaceEnd)() =
cam_eos_m   ? 0xff177ff8 :
cam_5d3_113 ? 0xff16d77c :
cam_5d3_123 ? 0xff16e318 :
0;

lvfaceEnd();

if (more_hacks == 2)
{
aewbSuspend();
}
}

/* Causes freeze on eosm
if (one_more_hack)
{
void (*CartridgeCancel)() =
cam_eos_m ? 0xffa7e7d8 :
cam_5d3_113 ? 0xff17fd68 :
cam_5d3_123 ? 0xff181340 :
0;
CartridgeCancel();
msleep(10);
}
*/
}
Also reverted back "Small hacks" to official code to make sure all hacks would work for this test:

Code: [Select]

static REQUIRES(RawRecTask)
void hack_liveview(int unhack)
{
/* auto sets kill global draw on faster fps (EOSM) in real time preview mode */
    if (kill_gd || (preview_mode == 1 && (shamem_read(0xC0F06804) == 0x2f701d4 || shamem_read(0xC0F06804) == 0x30701d4)))
    {
        if (!unhack)
        {
            idle_globaldraw_dis();
            clrscr();
        }
        else
        {
            idle_globaldraw_en();
        }
    }
 
if (small_hacks)
    {
        /* disable canon graphics (gains a little speed) */
        static int canon_gui_was_enabled;
        if (!unhack)
        {
            canon_gui_was_enabled = !canon_gui_front_buffer_disabled();
            canon_gui_disable_front_buffer();
        }
        else if (canon_gui_was_enabled)
        {
            canon_gui_enable_front_buffer(0);
            canon_gui_was_enabled = 0;
        }

        /* disable auto exposure and auto white balance */
        call("aewb_enableaewb", unhack ? 1 : 0);  /* for new cameras */
        call("lv_ae",           unhack ? 1 : 0);  /* for old cameras */
        call("lv_wb",           unhack ? 1 : 0);
       
        /* change dialog refresh timer from 50ms to 8192ms */
        uint32_t dialog_refresh_timer_addr = /* in StartDialogRefreshTimer */
            cam_50d ? 0xffa84e00 :
            cam_5d2 ? 0xffaac640 :
            cam_5d3_113 ? 0xff4acda4 :
            cam_5d3_123 ? 0xFF4B7648 :
            cam_550d ? 0xFF2FE5E4 :
            cam_600d ? 0xFF37AA18 :
            cam_650d ? 0xFF527E38 :
            cam_6d   ? 0xFF52C684 :
            cam_eos_m ? 0xFF539C1C :
            cam_700d ? 0xFF52BB60 :
            cam_7d  ? 0xFF345788 :
            cam_60d ? 0xff36fa3c :
            cam_70d ? 0xFF558FF0 :
            cam_100d ? 0xFF542580 :
            cam_500d ? 0xFF2ABEF8 :
            cam_1100d ? 0xFF373384 :
            /* ... */
            0;
        uint32_t dialog_refresh_timer_orig_instr = 0xe3a00032; /* mov r0, #50 */
        uint32_t dialog_refresh_timer_new_instr  = 0xe3a00a02; /* change to mov r0, #8192 */

        if (dialog_refresh_timer_addr)
        {
            if (!unhack) /* hack */
            {
                int err = patch_instruction(
                    dialog_refresh_timer_addr, dialog_refresh_timer_orig_instr, dialog_refresh_timer_new_instr,
                    "raw_rec: slow down Canon dialog refresh timer"
                );
               
                if (err)
                {
                    NotifyBox(1000, "Hack error at %x:\nexpected %x, got %x", dialog_refresh_timer_addr, dialog_refresh_timer_orig_instr, *(volatile uint32_t*)dialog_refresh_timer_addr);
                    beep_custom(1000, 2000, 1);
                }
            }
            else /* unhack */
            {
                unpatch_memory(dialog_refresh_timer_addr);
            }
        }
    }
}

Code: [Select]

            {
                .name = "Small hacks",
                .priv = &small_hacks,
                .max = 1,
                .choices = CHOICES("OFF", "ON"),
                .help  = "Slow down Canon GUI, disable auto exposure and af, white balance...",
                .help2  = "Set af on to enable auto focus but loose som overhead.",
                .advanced = 1,
            },

Did this crop_rec_4k_mlv_snd_raw_only_2022Jul31.EOSM202.zip work for you?

Did anyone get 70 MB/s write speed now on EOS M?


I think the problem in your current implementation for the new hacks in the following line:

Code: [Select]

if (more_hacks && shamem_read(0xc0f383d4) != 0x4f0010) /* excludes mcm mode on eosm */
But I am not sure, can't test, you should do some tests and figure out

The implementation seems broken also in your 5D3 builds (tested UHD 1:1 preset and 1920x1280 @ 30 FPS at 14-bit uncompressed, the new hacks didn't work).

[Danne]

Only way for me to check against bugs is checking code. Any open published test build should include sources. Normally this change would be done in a branch derived from an original fork.
Not many players in here follow gpl basics which is a shame really. Code is much more interesting then the closed out fix.
Just run your code in git and all is good.

[theBilalFakhouri]

Only way for me to check against bugs is checking code.

Hey, You did have the working code in first place (I PMed you my initial implementation back then when new hacks were discovered), my newer changes were included in my 650D/700D source code and it was online for months . . your changes broke the new hacks, have you tested your changes and made sure it's working before pushing your code? You should have noticed the problem, beside the working copy was online, you could compare it.

Any open published test build should include sources. Normally this change would be done in a branch derived from an original fork.

Sorry, but I don't have forks for your repos, and I don't want to *support them, but I can report bugs and give some hints like what I did here.

*I am not a fan of your heavy ML modifications, most of them feels wrong, I don't want to support that (making commits) . . personally I think -to some point- you are following 1% fork style , and it seems like a dead-end.
  Notes from side (shared it on Discord a few days ago, a user was asking for a merging possibility for your EOS M code to this reop):

Danne code for his EOS M build should be re-written from scratch, merging it at current state isn't a good idea, some notes from my side:
-Some of modules are loaded by default (you can't unload them), I see this as a downside
-Heavy ML UI changes (in particular Movie tab), and unfortunately to me he made it complex (not user friendly)
-crop_rec code needs major clean up, and again I think he made the code complex
-Several bugs which are easy to fix but not fixed yet
-Probably other things too

Not many players in here follow gpl basics which is a shame really. Code is much more interesting then the closed out fix.

Again, my code is online, I didn't restrict the fix or any info.

Just run your code in git and all is good.

I will run git, but as I mentioned, that does not mean I would work on your forks in my git repo.
If you prefer seeing the changes on git . . sorry, but that's not my problem, my code is online (on Google Drive, use SourceTree to see commits), does the GPL basics says I must use git?

Anyway, I said before I would rewrite my 650D/700D/100D code in a clean way and make the repo has native git support (no needs for hg / mercurial), and actually re-writing in progress and it might take some time:

https://github.com/bilalfakhouri/magiclantern_hg_02/commits/crop_rec_4k_mlv_snd

My goal to include the latest achievements for all DIGIC 5 models in a clean way in one branch (includes 100D/650D/700D/5D3/EOS M and probably 6D and 70D). Beside a basic support for DIGIC 4 models.

Sorry if my reply did have a high tone.

[Danne]

You're missing the whole point. Strongly suggest you read up about gpl philosophy and workflow.
Couldn't care less about your support. Only want this. If you share builds or testing fixes out in the open to always share sources and clearly where to find it. That's it.
Also. No matter what you're opinions are you used a lot a lot of my work in your builds. Even if a lot of stuff is ugly code there's still the possibility to pick out the good stuff and build something new. That's how this works.

[theBilalFakhouri]

Okay, next time I wouldn't share test builds in public in these kind of cases.

But did you get ~70 MB/s write speed anyway?

[theBilalFakhouri]

Also. No matter what you're opinions are you used a lot a lot of my work in your builds. Even if a lot of stuff is ugly code there's still the possibility to pick out the good stuff and build something new. That's how this works.

You didn't get the point, it's not about my "opinions", well:
-Due to your 5D3/EOS M builds, new users are getting more confused due to increased use complexity. I think this makes ML shape looks bad.
-How would contribute back to the main repo with your source code? Even if I decided to take the good stuff (to port EOS M to main repo), due to code complexity I might not able to do it.

I care about ML here.

Yeah there are a lot of shared code among us, you did that too . . that's about collaboration. If I saw some of good code I will use it, and I will throw out the not good stuff.

[Danne]

Sorry but I did my best. It was even more confusing before.
As with any other piece of art you never finish it, you just leave it.
In this case it is fully possible for anybody to take my fork, create a branch, and start polishing whatever. My time is very limited for a long time to come .
If a1ex or anyone with true coding abilities rather see that the fork is closed I probably consider that but so far this didn't happen. Personally I consider the code "safe" enough for using/playing/developing.

[names_are_hard]

Anyway, I said before I would rewrite my 650D/700D/100D code in a clean way and make the repo has native git support (no needs for hg / mercurial), and actually re-writing in progress and it might take some time:

https://github.com/bilalfakhouri/magiclantern_hg_02/commits/crop_rec_4k_mlv_snd

My goal to include the latest achievements for all DIGIC 5 models in a clean way in one branch (includes 100D/650D/700D/5D3/EOS M and probably 6D and 70D). Beside a basic support for DIGIC 4 models.

My main ML repo has git support already.  The commits that do that work are here:
https://github.com/reticulatedpines/magiclantern_simplified/commit/4a29a347f29276d4bec0876412cec4bb12b5a28c
https://github.com/reticulatedpines/magiclantern_simplified/commit/3163d41cee35fb54b4308e1f32318ff87de53645
(I think there might be a third commit near that time, too.  But that should be most of it).

In theory, that repo should work on Digic 4 and 5 cams.  It's been tested, a little...  I'd be interested to learn if your modules work with it - but test in qemu first!  You'll need to copy your module code to my repo and build there, because I made changes to how modules are compiled.  The official repo has a serious bug that can make modules jump to incorrect address when doing calls, triggering bad crashes.

Do you know if crop_rec_4k_mlv_snd is very different from lua_fix?  My repo has lua_fix already (and some other branches, merged together), so you may find it's not that hard to merge your changes into mine directly.  You're welcome to take the hg removal commits whatever you choose.  I think even better would be to recognise if the repo is hg *or* git, and work correctly for both.  It's nice to have the commit tag in the build info screen, which mine currently doesn't do.

[Danne]

I don't think bilal is working from crop_rec_4k_mlv_snd branch.
Not sure if g3gg0 mlv_snd code is included  into lua_fix but maybe it is. No time to check unfortunately.

[theBilalFakhouri]

Sorry but I did my best. It was even more confusing before.
As with any other piece of art you never finish it, you just leave it.
In this case it is fully possible for anybody to take my fork, create a branch, and start polishing whatever. My time is very limited for a long time to come .
If a1ex or anyone with true coding abilities rather see that the fork is closed I probably consider that but so far this didn't happen. Personally I consider the code "safe" enough for using/playing/developing.

Fair enough

For now, I am considering the Github repos as the way to go (Heptapod is dead currently), in particular:

magiclantern_simplified --> Currently used for porting ML to D678, beside D45 code works (needs more tests), this is based on unified branch, other branches have been removed to simplify things.
magiclantern_hg_02 --> This same as official ML mercurial repo, but imported to git, it include all branches, in order to build these changes are required, I will be using a fork of it for now and will re-write the code there, and hopefully it would be included in either in here or here.

[theBilalFakhouri]

My main ML repo has git support already.

Yes, I know, but it lacks crop_rec_4k_mlv_snd branch which is the one I am interested in.
This commit was enough to let crop_rec_4k_mlv_snd builds without issues, I did it here, from initial tests everything seems to work without problems, I will follow your other commits for removing hg requirements.

In theory, that repo should work on Digic 4 and 5 cams.  It's been tested, a little...  I'd be interested to learn if your modules work with it - but test in qemu first!  You'll need to copy your module code to my repo and build there, because I made changes to how modules are compiled.  The official repo has a serious bug that can make modules jump to incorrect address when doing calls, triggering bad crashes.

Sure, I can do tests later, but I am not very interested in unified branch which is the main one you are using in magiclantern_simplified.

Do you know if crop_rec_4k_mlv_snd is very different from lua_fix?

Yes, they are different, I don't how much but e.g. lua_fix doesn't have lossless compression or low bit-depths in uncompressed RAW, probably mlv_lite.mo doesn't work with mlv_snd.mo too.
Further more, adtg_gui.mo doesn't work in lua_fix (iirc OldAPI error?), but works in both crop_rec_4k_mlv_snd and iso-research branches.

I think lua_fix is similar to unified.

My repo has lua_fix already ..

Which repo exactly?

... so you may find it's not that hard to merge your changes into mine directly.

You mean to magiclantern_simplified? , I think we need to import crop_rec_4k_mlv_snd in magiclantern_simplified repo, then we need to find a way to merge dev branch and crop_rec_4k_mlv_snd correctly, but I think these two branches have a lot of differences, things might break, if that works I have no problem to work in your main branch.

For now I think the best way is to stay in my fork of magiclantern_hg_02, and port the new D45 crop mode stuff in one place, and probably merging other branches if it make sense, then importing the updated crop_rec_4k_mlv_snd to magiclantern_simplified then dealing with merging it to e.g. dev branch (unless if you have other plans).

[Danne]

Nice one Bilal 👍

[names_are_hard]

Sure, I can do tests later, but I am not very interested in unified branch which is the main one you are using in magiclantern_simplified.

Nope - unified is too old.  Mine is unified + lua_fix + qemu + some other ones (I forget...  there are so many open branches on official, I wanted to get away from that).  I want one long-lived branch only (and stable / release branches, no work happens there).  Other branches are for feature work only, before merging into dev (the main branch).  I nuked a lot of history because it was too confusing to understand, hundreds of branches.  I made that magiclantern_hg_02 repo to keep all history from all branches preserved in case it was needed.

I think lua_fix is similar to unified.

I don't really know, because I don't know of any documentation for what the different branches are for (other than commit messages), but in practice, lua_fix was the dev branch for several years.  It has many changes that are unrelated to lua.  I think the intention was to merge to unified, but this never happened.

I think we need to import crop_rec_4k_mlv_snd in magiclantern_simplified repo, then we need to find a way to merge dev branch and crop_rec_4k_mlv_snd correctly, but I think these two branches have a lot of differences, things might break, if that works I have no problem to work in your main branch.

I can look at this and see how different they are.

For now I think the best way is to stay in my fork of magiclantern_hg_02, and port the new D45 crop mode stuff in one place, and probably merging other branches if it make sense, then importing the updated crop_rec_4k_mlv_snd to magiclantern_simplified then dealing with merging it to e.g. dev branch (unless if you have other plans).

That would also work for me.  I warn you that the different official branches are painful to merge.  I had to inspect several 40k line diffs.  Fun.  Make sure you have some free time

[names_are_hard]

crop_rec_4k_mlv_snd is very close to lua_fix, lua_fix was merged into it "only" 2 years ago:
https://foss.heptapod.net/magic-lantern/magic-lantern/-/network/branch%2Fcrop_rec_4k_mlv_snd
https://foss.heptapod.net/magic-lantern/magic-lantern/-/compare/branch%2Fcrop_rec_4k_mlv_snd...branch%2Flua_fix?from_project_id=321

Merging these would be easy, so merging to mine shouldn't be too hard; I've already done the hard lua_fix + unified merge.

But that's enough cluttering up this SD card thread!  Let me know if you get the merge working, and I'll let you know if I do

[Danne]

@wyup

Try this, it should give you the extra 10 MB/s when using 192 MHz overclock:

crop_rec_4k_mlv_snd_raw_only_2022Jul31.EOSM202.zip

Enable the hacks from RAW video submenu, then report back.

I tested my own version and your hacks works just fine. ran 2.8k without hack 14bit lossless getting around 160 frames. Added the more hacks lvface + awb and getting continuous 2.8k 24 fps 14 bit lossless at around 69Mb/s so nothing wrong with my implementation. I suggest you test the 5d3 version a little more since I suspect it works just fine too.

Will add this to my code though. Much nicer than running shamem.

Code: [Select]

/* Exlude Movie Crop Mode from the new hacks */
if (video_mode_crop)
{
return;
}

[wyup]

@wyup

Try this, it should give you the extra 10 MB/s when using 192 MHz overclock:

crop_rec_4k_mlv_snd_raw_only_2022Jul31.EOSM202.zip

Enable the hacks from RAW video submenu, then report back.

I tried your build (overwrote files in card with yours) but I didn't notice improvement in actual recording speed. I get max around 60 MB/s with my 200MB/s Sandisk card.

[Danne]

You need to push existing presets as they are restricted to around 60Mb/s right now. Anyway, I already tested code and is working.

[wyup]

If I'm not mistaken, according to some reports people with Sandisk 170 MB/s get about 3MB/s faster than mine in both actual recording speed as benchmarks. So I'm not sure presets are restricted 60 MB/s, since as far as I know they get 63 MB/s.

[Danne]

Not restricted. Of you run the two hacks enabled with 2.8k preset and set bitdepth to OFF it will shoot into 68-69Mb/s.

[wyup]

I did. it doesn't make any difference. It ramps to 61-62 Mb/s briefly only to stop recording after 2-3-4 seconds.
Both hacks enabled, 2.8k Bit depth off.

[Walter Schulz]

I have the feeling you may want to purchase a Sandisk Extreme Pro R170/W90 to answer your questions.
AFAIK they come in all kind of sizes performing quite similiar and a 64 GByte specimen should do. Not like R200 variety which offers different write performance depending on size. Which one is yours, btw?