Eos M Data Rate Estimation, SD Cards and Optimizations

Started by gabriielangel, April 02, 2024, 02:13:06 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.


This post will be very useful for anyone wanting to get the best image-quality-to-performance ratio, and newcomers who would rather make movies than spending countless hours browsing through too many posts. It may seem daunting at first and will probably require a few readings for some, but it will get you to the proper image quality level in a fraction of the time it would otherwise take you to absorb that much info.
I think that time can be better spent learning about proper lighting and exposure techniques than fighting with a camera for purely technical reasons!
So, here is a summary of everything I learned about ML for Eos M in the past 2 years.


Where do those numbers come from?
1- Getting a proper SD Card.
2- Prepping the Card.
3- Using optimal Camera settings.
4- Optimal Scene Exposure Method (With False Colors examples).
5- Useful Links

Where do those numbers come from?

You can find an interactive version of the Data Rate Estimator here ;
You can read about the tests and procedure which lead to the above Estimator here (This has been written for a completely different purpose, but can still be very useful for those wanting to know more about the lossless compressor's efficiency);
You can read more about the way I tested some SD cards and detailed SD cards tests from this post onwards.

1- Getting a proper SD Card.

First, let's have a look at the Data Rate requirements for Lossless Raw Recording. Here is a Data Rate Estimation chart with a Lexar Silver Series 1066x 256GB Card, which is one of the fastest cards I have tested so far:

The numbers you see above are what you will get when exposing the image according to the method I will describe in Section 4. If you were to use the ettr (Expose as close to the clipping point as possible) method, those numbers would be even higher!
The Data Rate is highly dependant on scene complexity and brightness (More on this later). Therefore there are two numbers for each Bit Depth, for a particular preset:Normal and Complex.
The Normal number is the average Data Rate you will get for a scene with a moderate amount of details and some background blur, and the Complex number is the average Data Rate you will get for a scene with a lot of tiny details (Tree branches, blowing leaves, snow or tiny textures) and / or several reflective or shiny objects.
If you follow the guidelines explained later in this post, your Data Rate will be between those two numbers.

For example, you can see in the chart above that with a Lexar Silver Series 1066x 256GB Card, you can record comfortably (With less chances of the Camera stopping in the middle of a recording) at 1:1 2.5K 2.39:1 14bit (Data Rate between 69.4 and 81.2 MB/s) and lower; or 1x3 4.2K 16:9 12bit (Data Rate between 72.6 and 84.9 MB/s) or lower.
You could also record at 1:1 2.8k 2.39:1 12bit (Data Rate between 76.1 and 89.1) by exposing slightly below what is recommended. Underexposing by 1 stop reduces the Data Rate by 6-7%.

If you need to underexpose by 2 Stops  or more to get a low enough Data Rate, it is preferable to select the next lower bit depth (i.e. go from 14bit to 12bit).
I did a detailed test on that topic here.

The Lexar Silver Series 1066x 256GB Card is a very fast card capable of sustaining 88 MB/s for 1 minute or more and short bursts of up to 90.8 MB/s for 15 seconds. This is definitely not the case for the lower-end cards! (Those tend to yield inconsistent results when recording in the High Orange / Red Recording Indicator zone).
What the people on Youtube (Those who like to Hype things up) fail to disclose is that when they claim the camera can record continuously, it is often because they underexpose the image significantly, record very short clips or use the lower resolution and Aliasing-Prone 3x3 "1080p" mode. This is all fine if image quality is not your main concern, but definitely not if you want a clean image.

If someone recommends a Card saying "A lot of people said it works", ask them for a short clip and at which resolution they recorded. If you like what you see, then it is a good card for you!
The ML Benchmark fails to report certain subtleties that make a card suitable for longer recordings. There is no direct correlation between the numbers reported and the actual recording capabilities. Some cards just won't work despite showing good numbers! Therefore, I had to manually test a few cards to get a better idea of what's required. There is a list of tested cards in the Data Rate Estimator tool I linked above.

What I recommend is to get the fastest Card you can afford, otherwise, you will eventually have to buy another one...

Also, for applications where you just can't afford to redo a take (Weddings, some interviews, etc.) it would be wiser to use something like a Sony a6400. It will give you a sharper picture, full autofocus, ready-to-use files, better low-light performance and guaranteed recording times. In such cases, the less vibrant colors and banding issues are of lesser importance than the obligation not to miss a single moment!

The Eos M with ML will give you rich 12-14bit banding-free colors and the ability to deeply work those. You can modify white balance in post and apply LUTS to your footage with a lot less chances of getting a blocky image. But it requires more planning and is better suited for a "Scene-by-Scene" recording approach. Even if recording 30 minutes-long takes is possible, it is just not the way to go.

2- Prepping the Card.

Take the habit of doing a low-level format (Use the "Keep ML Files" option) before each filming session. For this, I will quote someting I said in a previous post:

I don't know about the magnitude of the speed gain when the blocks are pre-erased (format), but I know this:
Each time you insert the card in a different system, the file system is modified. The Mac writes a bunch of invisible dot files, Android writes extra folders both visible and invisible, etc.

After enough going back and forth (You need to transfer the files to your Mac or PC), the FAT gets corrupted (well, modified differently by a different OS, especially if you use the computer to erase the files)  and you slowly begin to get errors and corruptions, until the card needs a complete re-format and re-copying of all the files.

Ever since I started to Low-Level format before each recording session (There is a option to keep the ML files on the card. I don't know how they did it, but it works) I stopped having those kinds of problems. This is anecdotal of course, but I recorded about  7 TB over the course of the summer, most of it on the same 256GB card. So I guess this is a large enough sample.

3- Using optimal Camera settings.

For those who prefer using the higher resolution / high bit depth Presets, or just extend recording times, the following is recommended:
Kill Global Draw must be set to On;
ISO / Aperture assignments to the custom buttons must be avoided while recording. (On Danne's build, More Hacks must be set to "Allow" in the Customize Buttons section).
Also, it is preferable not to use shutter fine-tuning. You can find a very close match to the 180° rule by setting the shutter range to "Full range" in the ML menu, if what you get with the "Original" settings is too far off. I usually set it to 1/46 for 23.976 fps and it gives the expected motion blur.
You can read about the Data Rate impact of certain settings in this post.

To insure that the False Colors display correctly, use the Neutral Picture Style. You can change the sharpness, but not the other values. (I use 3,0,0,0). Those settings have no effect on the recording, but change the way the image appears in the Live View.

4- Optimal Scene Exposure Method.

Because Data Rate is Scene and Exposure (Brightness) dependant, it is important not to overexpose the image, if one seeks to use Higher Resolutions and Bit Depths. So, I did some tests to determine how to expose to maintain low noise levels and keep the Data Rates within the current Card's / Overclock capabilities. I did some preliminary testing from this post onwards.
The easiest way to reach an acceptable compromise is to use a combination of the False Colors and Histogram functions in ML.

By exposing the frame with the highlights showing mostly Yellow and only hints of Dark Red on the False Colors display, and by having the histogram (Which displays the rough average brightness of the frame) at 1.3 or below (The number is usually between 2.4 and 1.3 for 1:1 modes, mostly 2.0 when recording in 1x3 modes); you will reach the Data Rate figures found in the Data Rate Estimator Chart.

Here are a few examples where I recorded scenes with most of the frame in focus and plenty of details: (There are reflexions in the false colors images, as it was a sunny day)

2.8k 2.39:1 12bit, 87.07 MB/s. (The chart says 76.1-89.1 MB/s, we're quite close to the Complex number).

The same Scene, but 1 stop underexposed, 80.17 MB/s. (a reduction of 8% in Data Rate)

2.8k 2.39:1 12bit, 82.71 MB/s. (The chart says 76.1-89.1 MB/s, we're within range)

The same Scene, but 1 stop underexposed, 77.39 MB/s. (a reduction of 6.5% in Data Rate)

2.8k 2.39:1 12bit, 83.93 MB/s. (The chart says 76.1-89.1 MB/s, we're within range)

The same Scene, but 1 stop underexposed, 77.47 MB/s. (a reduction of 7.7% in Data Rate)

2.5k 2.39:1 14bit, 78.38 MB/s. (The chart says 69.4-81.2 MB/s, we're within range)
Here, I were able to push the clouds into the Bright Red Region, because there is very little of the frame in the highlights region (Yellow and above).

2.5k 2.39:1 14bit, 80.49 MB/s. (The chart says 69.4-81.2 MB/s, we're still within range, but close to the upper limit)
Here I pushed the clouds very close to clipping (Black spots in False colors). This needs to be avoided unless a large portion of the frame is well below the highlights region, otherwise the Data Rate will be too high. Note that the histogram is still below 1.3, this is why we are within range nonetheless.

The key is to have as little Red as possible in the false colors, just enough to keep as much of the frame as possible in the low-noise range.

When working with people, it is always better to use a Reflector or a powerful enough Led Light to avoid having to underexpose your talent because of a very bright environment.
Skin tones and the subject in the frame must be around the Orange False Color (Or higher) to be as clean as possible.

Here is an example of the noise level in the different False Colors areas when doing shadow recovery: (Click to see the animation)

There are some more Scene / Data Rate examples in this post;
And a primer on using False Colors here.

5- Useful Links.

Bilal's Crop Mood repository Link.
Danne's Custom Crop Mood repository Link.
Canon Eos M Manual Link.
ML Camera Help page (Useful for some functions) Link.


MOD: Please leave this post blank, as I will add more at a later date!


This is an amazing, precious and well thought out compendium. Thanks mate!


Thank you, full of useful information. However, you didn't mention that false colors are linked with the image profile and level of contrast used, so it's not accurate in RAW.


Quote from: Grognard on April 02, 2024, 11:13:30 PMThank you, full of useful information. However, you didn't mention that false colors are linked with the image profile and level of contrast used, so it's not accurate in RAW.

May I ask why you would want to change anything other than the sharpness in the picture profile? In RAW, it isn't recorded, so you would have no idea of what you are actually recording...


Quote from: gabriielangel on April 02, 2024, 11:48:14 PMMay I ask why you would want to change anything other than the sharpness in the picture profile? In RAW, it isn't recorded, so you would have no idea of what you are actually recording...

Yes, it isn't recorded. Raw is raw. But we still need to use a picture style. Which picture style do you use? Neutral?
In framing, false colors are very slow but more accurate in showing what the raw will really look like.

However, we can make the liveview close to framing with some tweaks (negative gain, less contrast ...)


I use the Neutral 0,0,0,0 profile. I will add a note about it.



Raw 12 bits lossless

Contrast - 1
ML digital ISO gain + 1.0EV.

It's not perfect but closer to framing exposure.