Messages

Thanks for your findings.
I consider 12bit as standard for all presets.
Corrupted frames in 1620p. Might be overhead histogram or similar. What was enabled? What SD card is used? Run 12bit and don't add global draw items, manual focus. Still corrupted frames?
Thanks for all your tests. Valuable too many.

Sandisk Extreme Pro 128GB R170W/W90 (Freshly Low-Level formatted)
Kill Global Draw On (So Nothing Was On Screen)
Manual Focus (Feb02, Feb04 and Feb05. The tests I did before that were all on AF)
U/D Buttons Off
INFO Button x10 Zoom
SET Button False Colors
TAP Screen Off
(I always use the same setup)

Today, running the Feb05 Build,
Recording at 12bit, I got a corrupted frame using 1376x2322 (See Image)

Although anything is possible, I have gone through the preset thing at least 6 times since Jan 1, so that's 25+ Presets + the 3x3 Presets, times 6.
9 of those presets have a higher Megapixel Count. So I'd lean more towards a recent change.
I am unable to reproduce it on demand though. So a few more tests are required.

I will record some this evening and let you know.

Tested magiclantern-crop_mood.2024Feb04_Custom.EOSM202.zip

Tested every resolution, no more black / red bars at the bottom of any preset.
No corrupted frames, except for the new preset. This one is strange, because out of 3 recordings, 2 showed glitches on screen while I were recording, but only 1 file had corrupted frames in it.
And 1736x868 48fps gave a corrupted frame even in manual mode.
The image is free of artifacts.

About the 3k preset:
After recording yesterday and last time with the watch scene, I realized that the 3k preset cannot work at 14 bits. The data rate is way too high,
and you need to underexpose 2 stops (Which is equivalent to switching to 12 bits) for it to work.

See the scene I recorded for the tests below. You can see that it is reasonably exposed and not overly bright. Plenty of shadow areas and it still needs 94 MB/s.

If you look at the screenshot, you'll see that 20, 21 and 22fps brings it within reach for the fastest cards.
So, it would be a good idea to take the same approach you took with the Full-Res 1x3 presets, whereas 14bit uses a lower frame rate.
This would make 14bit usable until faster cards are found.

New build:

Could you test this build @iaburn?

- Adjusting powersavetiming registry probably causing random image tearing. To be tested.

commit
https://bitbucket.org/Dannephoto/magiclantern_hg_02/commits/2e041f5e51fbcdbd931399b1e31109c1db45ca67

I just tested again, will post results tomorrow pm as it is still compressing as I am writing this, but the new 1620p preset gives corrupted frames right from the start (I can see it onscreen as it records).
I switched to 2.8k to see, no problem there.
Then back to 1620p and it did it again.
This seems to be recent, as when I recorded last time (Feb 2, Jan 29), it did not happen.

[New build uploaded:]

New build uploaded:
- Fixed following presets, height tuning.
1280x2160, 1472x2208, 1600x2400, 3072x1308, 2520x1080, 1920x1280, 1736x1160, HFR: 1736x976, 1736x868, 1736x790, 1736x738.
- Added new pixel maps to Mlv App
- Reworked 3K 1x1 so that it is now 2.39:1 instead of 2.35:1.

Thanks @gabriielangel
If we find anymore pixelated presets please let me know which ones.

Since you asked so nicely 

I tested all the presets while I were at it. For some reason GitHub isn't cooperating, so I can't check the focus pixels. But only the following stilll have issues:

1736x976 HFR: Corrupted Frame (But this is the one showing the black preview. I found that HFR 16:9 and 2:1 show ok if you move the focus box Up by 4 units. Other ratios remain black);
1472x2208 Black line at bottom
5208x3478 Black or Red line at bottom, depending on exposure.
1920x1280 Needs Focus Pix
2560x1440 Black line at bottom
3072x1284 Needs Focus Pix
1552x2118 Needs Focus Pix
1600x2400 Needs Focus Pix



I filmed a white card this time to make sure I wouldn't miss anything, but I'll film a real scene on your next release to make sure the image didn't get hurt.

Try enabling "Preview toggle" on the RAW video menu, and also remove any action assigned to the Half-Shutter on "Customize buttons". If you do that, half shutter press and (*) will toggle between real-time and actual framing. Not sure why half shutter affects (*) though, maybe is not possible to assign an action just to (*).

The * is assigned in the Canon Menu's Custom Functions(C.fn). In C.Fn IV menu 5, some settings transfer Auto-Focus to the * Button.

But Just enabling "Preview Toggle" Did the trick! You can then use the * Button to see the whole frame.

I had a nice setup today, so I took the opportunity to test the latest build while things were still in place: magiclantern-crop_mood.2024Jan29_Custom.EOSM202

The image quality is better than before.
I have tested all the resolutions below:


And I haven't found a single one of these (bad scan lines):

Which would appear every now and then in the previous builds. This is quite nice.
No corrupted frames either.

The following resolutions still have remnants of scanlines at the bottom, but those do not export to prores:

1280x2160, 1472x2208, 1600x2400, 3072x1308, 2520x1080, 1920x1280, 1736x1160, HFR: 1736x976, 1736x868, 1736x790, 1736x738.

The 726p 3x3 HFR Preset will show a black preview if the camera is set to AF. It shows ok when set to MF.

The 1:1 full-res LV only shows a portion of the frame, and it's not possible to press the * button to switch to framing.
This preset only worked well on bilal's first release of crop mood, where you would see a cropped preview, but you could press * to see the framing. (the 3k also worked this way back then)

I Like the order of the new Movie menu: Everything you use most often is right there, and 1 push up, goes at the bottom, so you can get to the rest with less scrolling.

The new preset is nice, You can post on TikTok or other Vertical / Square delivery platforms without wasting the side pixels like before.

I have 2 suggestions:

1- It would be nice if the 3k preset got the 2.39:1 treatment. The little extra room would give it more recording time on a bright day.

2- It would also be nice if MLV App could have an option to map clipped pixels to white. Because of the pink highlights, it is very challenging to shoot a scene like I did today, without underexposing a lot (Because of the reflective stainless steel on the watch).

[Speed Test: Samsung EVO Plus microSDXC UHS-I 128GB]

Speed Test: Samsung EVO Plus microSDXC UHS-I 128GB
R130/W130
Product Number: MB-MC128KA / MB-MD128S (Important precision, thanks @walter)
MDT: 2023/05

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 62.64 MB/s @ 2.8k/14bits
Highest CropMood speed reached for "Continuous"   (Steady Green): 55.13 MB/s @ 2.8k/12bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 66.4 MB/s @ 2.8k/14bits
240MHz / SDR104

(Item "H" in the picture below)


Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.






The method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

To hopefully expose any Thermal Throttling or Burst Speed Wizardry implemented by the manufacturer, I recorded about 10 clips of various lengths (Light orange / red) prior to beginning the test (Not in the picture, to save on transfer time)

*Something new, I added a "Red Speed" to the test, which is the maximum speed which a card can sustain in the red for 15 seconds.
This doesn't make much of a difference when recording at 12 bit, but with the high resolution presets at 14 bit, it shows if the card can tolerate sudden changes in the scene (Like confetti exploding or anything entering the scene that's brighter than when you first hit record) I updated the other card tests as well.

This card exhibited a weird behaviour in which every other recording gave a "Frame Order" error as soon as I recorded clips over 30 seconds, in the Orange zone !?
If I recorded in the Green zone, no problem. This happened recording at 2.5k 2.39:1, 14bits and 12 bits, 1 minute clips and clips over 30 seconds.
I tried at 240MHz, 192MHz, SDR104 and SDR50; No difference.

As @walter pointed out, the model number written on the card itself makes a difference, but this number isn't written anywhere on the package, which looks identical to the package of K variants, at least here in Canada. So there is no way to know which variant (K or S) you are getting before opening the package.

So far, the Samsung cards (Along with the Lexar) have the steadiest, constantly repeatable across takes speed.

Thx!

Here are some examples I recorded last year. I left the images at their original resolutions, so I suggest you do like @iaburn did, and resize all the pictures to a common format. I did my best to keep a similar framing at every resolution. (Click on images to download full-size)

0- An example of 1x3 aliasing / Stairstepping on contrasting edges. Those "Animate" when you move the camera.


1- 5208x2108 Original Resolution


2-Same as above, but with CA Desaturation applied, to get rid of the rainbows in the hair (I forgot to do this when I sent the image to the upscaler)


3-2880x1206 Original Resolution


4-Same as above with CA Desaturation applied


5-2520x1054 Original Resolution


6-3072x1286 Original Resolution


7-4416x1846 Original Resolution


8-4800x2008 Original Resolution


9-2880x1206 upscaled with Ai to 3840x1608


10-5208x2178-DownTo1920x804 then Upscaled with ai algo 1 to 3840x1606 (The downscaling cuts the processing time by a factor of 4, and yields a slightly sharper image)


11-Same as above, but with a different Algorithm


I don't have a 3x3 example, but I was a little surprised of the 3x3 sharpness in @iaburn's example.

In the examples above, I left the upscaling to default values, but I would have normally dialed down the sharpness a bit.
I think the debayer could use a little help, to better deal with hair, fur and tiny details.

Before recording to the SD Card, the bit depth is reduced to 12,11 or 10 bits when you select those bit depths.
I think they lower the gain to achieve it, because the brightness of the image changes when you select a different bit depth.
So I would like to know the formula they use to go from 14 to the other bit depths.

Hello, I am writing a little guide to maximize recording on EOS M and I have 2 more questions:

Could someone tell me the actual video buffer size we have access to while recording?
I would also like to know what the formula used in crop mood is, when reducing the bit depth.

Thanks.

Thanks for factual numbers @gabriielangel. Down sampling 1x3 and then scale up after down sample? Will the output differ or am I talking out of my ass?

A lot of up-sampling is done with AI nowadays (DaVinci SuperScale Neural Engine, Topaz Video AI, etc.). If you start with a better quality image, you will get smoother results.

If you take 1736x2178 and bring it down to something like 3472x1452 (x2/3 instead of x3) You could post it as-is, no need to go to 4k;
If you bring it to its "Real Resolution" 1736x726 or, to 1080p at 1920x804, you give the AI the resolution it was trained to upscale from.

In both cases:
Because you Oversampled by scanning a larger sensor area, you get extra details;
You gained significant real vertical resolution;
You gained some horizontal resolution because the in-camera binning averages 3 "Real" pixels into one (At the expense of some Stairstepping/Aliasing);
The closer you get to the "Real Resolution" the lesser the stairstepping effect's appearance
(There is a point of diminishing returns in 1x3 mode, where too much width res. is lost while not much vertical res. is gained and aliasing is introduced, vs a 1:1 mode, which I haven't tested).

So you end up giving the AI upscaler a cleaner image to begin with (There is an option for this in Topaz's offering called CG or Anti-aliasing if I remember correctly. I don't have access to the software ATM).

@2blackbar, try different Debayers in MLVApp, you will see some significant differences in the text portion of your tests.

I used the term "Real Resolution" here for lack of a better one. Finding the sweet spot would require some testing.
Edit: The term "Native Resolution" would be more appropriate.

Why not take the time to post some actual screenshots every now and then? Those would turn opinions into facts

The "4.8k" 1x3 preset at 1600x2008 gives you 3.21 Megapixels;
The "5.2k" 1x3 preset at 1736x2178 gives you 3.78 Megapixels;
The 2.8k 1:1 Preset at 2800x1206 gives you 3.47 Megapixels.

Because the 3x1 modes give you a larger sensor area, you can  get closer to the subject to get a similar framing, which allows you to capture more details and this in turn gives you an apparent higher resolution. But these modes are plagued with aliasing (a lot less so with 5.2k). The debayer is not perfect, and it will show a lot of stairstepping on sharp curved edges and diagonal lines above a certain angle.
In order to get the full benefits of the 1x3 modes, you need to scale down (As opposed to the default in MLV App, which is to x3 the width), or rather, stay as close as possible to the native resolution's megapixels

3.47MP>3.21MP and that's that. If you select a wider angle lens for the 2.8k so that you can get to the exact same distance from your subject, and maybe accept more image distortion depending on the lens and distance, you will always get a cleaner image than any of the 1x3 modes.

Please check wiki entry: K means 2021 edition. 2023 has S suffix. Date in third culumn doesn't refer to production or purchase.
A (or other suffixes like EU, US) refer to market region and should be ommited in this column.

Well, I took the MDT  from the CID Info Menu (2023/07).
The PRV (Revision Number) is: 3.0
The number written behind the microSD card is MB-MC512K
The Number written next to the UPC code (See on the picture): MB-MC512KA/CA .
I made the assumption that the CA (Canada) was the market part, as there is also a model called MB-MC512KA/EU (!)

But you're right, it is 2021 according to this link: https://www.samsung.com/uk/memory-storage/memory-card/evo-plus-512gb-microsd-card-mb-mc512ka-eu/

Will fix shortly, thanks.

Have a look at this (I could not find the rev3.0), maybe this could be of use to those who know how to tweak the controller to get more out of the card?

https://download.semiconductor.samsung.com/resources/data-sheet/2023_samsung_data_sheet_microsd_card_evo_plus_rev.1.0.pdf

At the very bottom (Fine print) it says:
2)SDR104: 1.8V Signaling, Frequency up to 208 MHz, up to 104MB/sec, Max. Current Consumption 800mA (varies by test conditions)
3) DDR200 : 1.8V Signaling, Frequency up to 208 MHz, up to 200MB/sec, Max. Current Consumption 800mA (varies by test conditions)

R130/W130 for EVO Plus?

There are no decent spec sheets by Samsung for sure!

At least they wrote something   Some cards say "Transfer Speeds of up to 120 MB/s!" Then after looking for write speed everywhere on the package, what you find somewhere on the back:
Numbers stated are for read speeds of up to 120MB/s, when using a compatible reader. Write speeds lower.

[Speed Test: Samsung Evo Plus microSDXC UHS-I 512GB]

Speed Test: Samsung Evo Plus microSDXC UHS-I 512GB
R130/W130
Product Number: MB-MC512KA/CA
MDT: 2023/07

Highest CropMood speed reached for a 1 minute Clip (Peak Orange): 86.67 MB/s @ 2.8k/14bits
Highest CropMood speed reached for "Continuous"   (Steady Green): 76.79 MB/s @ 2.8k/14bits
Highest CropMood speed reached for a 15 seconds clip (Red Speed): 89.3 MB/s @ 2.8k/14bits
240MHz / SDR104



Card has been formatted in camera (EOS M), moved to a Mac to copy the CropMood files, then back to the camera to make it bootable.





The method is outlined here: https://www.magiclantern.fm/forum/index.php?topic=25841.msg245694#msg245694

I recorded more clips this time, to hopefully expose any Thermal Throttling or Burst Speed Wizardry implemented by the manufacturer. I recorded about 10 clips of various lengths (Light orange / red) prior to beginning the test (Not in the picture, to save on transfer time)

This card did not exhibit any problems and was consistent. Unlike the other cards tested so far, this card did not trigger a single "Frame Error" message during the tests.
This card is about the same price as a Lexar Silver Series 1066x 256GB, for TWICE the size, and the performances seem comparable so far.

I tested the Samsung Evo Plus 128GB 2023/04 thoroughly today (Will post the results soon).
I discovered that with that particular card, I had a clear pattern regarding the "Frame Order Error": One clip records fine, One clip records with frame order error, and so on.
I restarted the camera a few times and that pattern remained constant. This happened recording at 2.5k 2.39:1, 14bits and 12 bits, 1 minute clips.
SDR104 and SDR50 yielded the same results and behavior.
With Bilal's latest build for EOS m, Camera set tp AF, Small Hacks more.

EDIT: I also tested the Samsung Evo Plus 512GB MB-MC512KA. 2023/07, same parameters, and this card did not give me a single Frame Order Error over the course of the session.

Played with the new presets (so many, probably not all):



Edit: 1736x2322 has 2.24:1 AR. Can't reproduce that. Is there a typo?

Sorry @masc, yes typo. It is 1376x2322

I re-checked:
1736x2178 , it doesn't work
1736x3478, it doesn't work

1736x738: works fine here also
1736x726: works fine here also


I recorded everything with the Jan 19 build, and I think Danne uploaded 2 versions on Jan 20.
As 1736x2178 works when I use bilal's original version, maybe check to make sure there won't be any interference?

My computer isn't internet connected (MLV App cannot auto-download), would you mind giving me the repo's address?

Those are the focus maps I have in my app, and I can see it getting rid of the focus pixels on files I recorded Last Week at 2880x1206.
This was before @danne added the 2.39:1 preset, so I changed the ratio from the Raw Video submenu when recording. Don't know if that makes a difference.
Edit:It does make a difference.

Those are the rez where my focus pix don't work here:
1736x2178
1736x2322
1504x2538
1736x3478
2880x1206

Here is a tentative chart for the recording possibilities of different cards at various resolutions, for the EOS M.
If you own one of the cards I tested (Listed in the second sheet), you could try recording a scene or two, to see if the predictions are robust.
All the instructions are in the spreadsheet.

To download it :  Here

EDIT: I added a Scene complexity Dropdown menu, to have an idea of how scene complexity can affect data rates. The chart is a lot more complex than I would have liked to see it, if I were to look at it for the first time, but I think it beats having to learn this by trial-and-error
EDIT2:There is now a dropdown menu to choose cards instead of entering the numbers manually.

Thanks! I just needed to know, to make sure I wouldn't introduce too much bias in my calculations.
You both gave me what I need: LJ92 scales well and I can take the whole data rate reported in MLV app to make comparisons across resolutions.

New build:
When up/down or SET or INFO buttons are assigned to ISO or Aperture More hacks is bypassed. If up/down button is disabled and other functions to SET and INFO are assigned than ISO and Aperture More hacks is active.

I also think this is a sound compromise. Those recording with higher resolution presets will be able to get longer recording times.

Making preview 14bit allows for more precise exposure monitoring, as false colors and Histogram are more representative. Those were always slightly off when recording below 14 bits before, so you had a greater chance of clipping footage. The pink clouds you see in iaburn's example are the result of the highlights being clipped. It's not always possible to get rid of those after.

I have 2 questions:

Is the audio in the MLV file aslo compressed with lossless, or is it a regular .wav file?

How do the devs estimate the theoretical resulting Data Rate for a given resolution, given that you don't know ahead of time how efficient the lossless compression will be? Do you have a compression ratio window?

I was using SET for x10 Zoom but now using for Aperture + so was hoping for an alternative for x10 Zoom that toggles on/off

If you set your camera to Manual Focus (MF), in the canon menu, or on the lens if you are using an ef-s lens on an adapter.

You can go to the customize buttons menu and set it as:

Half-Shutter: Zoom x10
SET Button: Aperture +
INFO Button: Aperture -

Then, pressing the shutter halfway (Be careful not to push it too far!) will toggle x10 Zoom on off.
If you press it all the way in, it may take a photo, but sometimes it freezes the camera. So you need to pull the battery and restart it.

EDIT: I remember what I did to stop that annoyance:
Go to the Canon Menu
In the 5th tab
Set Video snapshot to: ENABLE
From now on, when in Video mode, the full Shutter Button won't work at all.
And it will work as usual in Photo Mode.
No Video Snapshot seems to be recorded on the card.
I haven't tested if it affects performance though. Let me know.



If your camera (Or ef-s lens) is set to AF

You can go to the customize buttons menu and set it as:

INFO Button: Zoom x10 (Preferred)
or
SET Button: Zoom x10 (If you do this, you won't be able to recenter the focus box automatically if you move it. You will have to use the arrows on screen to put it back in the center)

Then you use the wheel to change the Aperture.
If you use the wheel directly, it will change the shutter speed.
If you push right button, then turn the wheel, you will be able to adjust Aperture + -

Using the Tap screen fo x10Zoom would not work, as you wouldn't be able to turn it off once you're in, because of the Nav arrows on screen;
Using the U/D buttons seems to have a performance cost (I don't know if it's the buttons themselves, or what's currently assigned to them);
L/R buttons cannot be assigned in eos m ATM (From what I have read so far).

[Exposure vs Noise (EOS M)]

Exposure vs Noise (EOS M)

(It will be easier if you download all the images first. The file names are descriptive.)

The end goal here is to come up with a realistic minimum average exposure, to establish an approximate Data Rate for any given Resolution / Bitrate / Framerate.
That way, someone could decide if they want to invest $100 for a lexar card, or settle for a $15-$20 Samsung if it gets the job done at one's preferred resolution.

So, to find out how low we can underexpose to save some bandwidth, while keeping a good image quality...

Given that the chart I used in the previous post:


Is exposed like this:

(Exposed ETTR, whithout any clipping)

We get images like these:


14bits


12bits


11bits


10bits


14 bits, with exposure lowered in MLV app to check highlights.


10 bits, with exposure lowered in MLV app to check highlights. Highlights are comparable to 14 bits.


So far, we can see that without any shadow recovery, noise starts being visible around Stop 6. Slightly before 6 at 10bits, but barely (Not easy to judge with a static frame)
This will vary according to the color of course.
A slight red tint appears in the low level noise at 11bit, and becomes obvious at 10bits.

Let's investigate the shadows. We'll add a shadow boost of 100 in MLV app:

14bits with Shadows +100


12bits with Shadows +100


11bits with Shadows +100


10bits with Shadows +100


Now we see a huge difference in the quality of the noise that is recovered along with the shadows.
at 14 bits, the noise starts to become really coarse at 9 3/4 below Highlight level.
at 12 bits, the coarseness starts at around 9, but there is a clear difference starting around  Stop 7 compared to 14 bits.
at 11 bits, the difference compared to 12 bits starts at Stop 7, but the coarseness start being visible at 8.
at 10 bits, noise becomes quite visible at Stop 6, at Stop 8 there is a lot more noise than at 11bits, and what is at 9 and below is uncomfortably coarse. Also, there is a red tint all over the shadows.

Let's see if using Darkframe subtraction improves the noise a bit.
I filmed clips with the cap on, one dark clip per bit rate. I exported those dark clips with the exact same 4 second length as MLV fast pass and then as MLV Averaged frame (To get same length averages).

14bits, shadows+100, Darkframe subtraction:


12bits, shadows+100, Darkframe subtraction:


11bits, shadows+100, Darkframe subtraction:


10bits, shadows+100, Darkframe subtraction:


10bits, shadows+60, Darkframe subtraction: (To see what it looks like in a slightly more realistic context)


At 14bits and 12 bits, the difference is subtle, but still noticeable as the red tint is removed.
But at 11bits, slight vertical lines begin to appear and at 10bits, those vertical lines become quite obvious.

So, what does this all means?

1st thing is to avoid shooting at 10bits if you plan on making any color or exposure modifications in post. I would start with 11bits minimum and prefer at least 12bits.
(This has been stated before by several members on this forum).
Because exposing close to ETTR maximizes the amount of light the sensor receives, and the bit reduction happens once the signal is digital Edit: This is not 100% exact, it appears that the bit depth reduction is obtained by gain reduction at the analog stage and a 14bit container is always used. (This is one of the reasons why the quality of the noise degrades rapidly at the lower end of the scale, as bits of the shadows sharply get crushed to pure black. You end up with larger black blocks once you bring it all up);
I am under the impression that adding a 13 bits option would be a good idea to help gain a few MB/s, as this would still allow to ETTR while allowing a slightly lower Data Rate, as opposed to significantly Underexposing 14bits while recording.

Now, we want to make sure that Skintones are always above the visible noise level, which is between Stop 5 and Stop 6 below highlights;
But we also want to keep the grainy parts as low as possible...

The EOS M is rated at 11.2 Stops of DR according to DxOMark :https://www.dxomark.com/Cameras/Canon/EOS-M---Measurements
We definitely want anything important enough to be seen to be above Stop 9, because of the coarse noise happening below that point.

Based on the false color images above, the average showing up on the Histogram should be above 2.6 to minimize the grainy noise in the visible parts.
I know, by experience, that outside on a sunny day, I can shoot 30 seconds + whatever, at 2.8k/14bits with histogram showing 1.2 (up to 0.8 every now and then) As long as the highlights are mostly yellow with a hint of red (On the false colors).

So the target seems to be somewhere between Histogram 2.6 and 1.0 (Histogram set to ETTR Hint).

Again, the goal here is to come up with a realistic minimum average exposure, to establish an approximate Data Rate for any given Resolution / Bitrate / Framerate.

Your comments are very welcome...