Bilal's crop_rec_4k experiments for 100D / SL1

[GianlucaM83]

Load bench.mo in modules tab. Change overclocking frequency to 160 MHz. Restart camera.
Access Debug tab -> Benchmarks -> Card benchmarks -> Quick R/W benchmark (1 min) and start it.
If card defaults to 19-21 MByte/s -> Lower frequency, restart and retest.

Thanks for the help
In the tests I get a write and read speed of about 21 mb/s in all three modes "160 MHz - 192 MHz - 240 MHz"
If I do the test in video mode I get these errors: Alloc error: buffer=16777216

[theBilalFakhouri]

@GianlucaM83

160MHz overclocking is working fine here using my Sandisk Extreme PRO 32GB UHS-I U3 95MB/s.
You must restart your camera after changing SD overclocking frequency, if you didn't do that the setting won't be applied.

Let's start fresh:
-Unload all modules, load only bench.mo, restart the camera.
-Switch to Photo mode (no LiveView)
-Run cards benchmarks, share the result.
-Load sd_uhs.mo, restart the camera.
-Select 160 MHz, restart the camera.
-Run cards benchmarks, share the result.

If I do the test in video mode I get these errors: Alloc error: buffer=16777216

Turn off RAW video before running benchmarks in movie mode.

[GianlucaM83]

@GianlucaM83

160MHz overclocking is working fine here using my Sandisk Extreme PRO 32GB UHS-I U3 95MB/s.
You must restart your camera after changing SD overclocking frequency, if you didn't do that the setting won't be applied.

Let's start fresh:
-Unload all modules, load only bench.mo, restart the camera.
-Switch to Photo mode (no LiveView)
-Run cards benchmarks, share the result.
-Load sd_uhs.mo, restart the camera.
-Select 160 MHz, restart the camera.
-Run cards benchmarks, share the result.

Turn off RAW video before running benchmarks in movie mode.

ok. I followed your instructions. 
these are the results:

only bench.mo loaded:
Write speed 41.8 mb/s
Read speed 43.6 mb/s



sd_uhs.mo Loaded (restarted camera), 160 MHz selected (restarted camera)
Write speed 21.1 mb/s
Read speed 21.8 mb/s



I cannot understand the reason for this result

further info I can provide
the sd is formatted in FAT32 and I did a deep format in camera before installing magiclantern

[theBilalFakhouri]

Okay, thanks for re-testing.

In my custom 100D build I am using my 160 MHz preset which worked fine using my SD cards, let's test a1ex's 160 MHz preset with this build:
crop_rec_4k.2022Jun19.100D101_160MHz.zip

Replace all ML files with the provided build (or make a clean install, format SD card in PC then install the provided build), test 160 MHz preset and see if it will drop to 21 MB/s or not, report back please.

[GianlucaM83]

Thanks for your patience 
I did a clean install
unfortunately i get the same speed even with this version.
i think the problem is just my SD, maybe it is damaged.
i will buy a new one and come back with new information.

[Walter Schulz]

Maybe giving this version a try first:
https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/80/artifact/platform/100D.101/magiclantern-crop_rec_4k.2018Jul22.100D101.zip
There are several sd_uhs.mo variants to try.
Just copy it to ML\modules directory.

[theBilalFakhouri]

@GianlucaM83

One last test from my side, test 160 MHz again with this build:
crop_rec_4k.2022Jun19.100D101_160MHz_SD_Configuration.zip

i think the problem is just my SD, maybe it is damaged.

Maybe your 95 MB/s variant is different from mine . . or requires different overclock settings.

i will buy a new one and come back with new information.

Make sure it's Sandisk Extreme PRO 170 MB/s UHS-I U3 version which works with 192 MHz overclock.

Maybe giving this version a try first:
https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/80/artifact/platform/100D.101/magiclantern-crop_rec_4k.2018Jul22.100D101.zip

Let's test that build with official sd_uhs from a1ex:
https://drive.google.com/file/d/1xWYubqqr0DceaqMhXqiIovyF_7ep5vfc/view?usp=sharing

After running a1ex's sd_uhs version, a log would be saved in ML/LOGS folder in SD card (SD_UHS.LOG), please upload it somewhere.

[GianlucaM83]

Ok done. this is the .LOG file
https://drive.google.com/file/d/1sP4CJKMZlBXwItMUhhkYXB1fLYEjs23E/view?usp=sharing

[GianlucaM83]

I used the sd_uhs from a1ex.  because i hadn't seen the first link with your build, sorry.

[theBilalFakhouri]

Cool, so 160 MHz does actually work with your SD card. You need to check if it stable (doesn't drop to 21 MB/s), you can do this by recording RAW video or run Benchmarks multiple times in movie mode and photo mode.

Last thing I need you to test is:

One last test from my side, test 160 MHz again with this build:
crop_rec_4k.2022Jun19.100D101_160MHz_SD_Configuration.zip

If that worked, then I would know what causing the issue (the drop to 21 MB/s when using 160 MHz in my custom build).

[GianlucaM83]

I installed the latest test build you provided. Something has changed. it looks better now

Write speed 67.3 MB/s
Read speed 71.9 MB/s

[theBilalFakhouri]

Nice, please make some RAW recording tests and see if it stable (doesn't drop to 21 MB/s).
Will post more details later.

[GianlucaM83]

ok I'm back 
I recorded two raw videos. duration one minute per video, because the battery was getting low.
It's really stable, I think it could keep shooting continuously.

These are the settings

-UHD 1X3 1280x2160
12 bit Lossless, 23.973 fps
Frames 1504, Duration 1 Min, But I think it's continuous

-4K 1x3: 1x3 Binning 1360x1976
12 bit Lossless, 23.973 fps
Frames 1493, Duration 1 Min, But I think it's continuous

Bilal you are great 

[GianlucaM83]

I had the opportunity to try the build more thoroughly.
The speed of the SD seems constant. I can record for several minutes without problems.
One problem I noticed is that, in every RAW video I have recorded, on the top of the frame only, there is some kind of intermittent noise and strange vertical stripes.
This is most visible in outdoor footage, instead in the recordings made at home the problem is very little noticeable.
This happens in both mlvapp and davinci resolve using the uncompressed dng files extracted with mlvapp.
The problem is also visible by opening the mlvs files with the FastcinemaDNG program.
I upload some MLVS test files.
Note that. The mlvs files I have uploaded contain only a few frames extracted from the original mlvs files.
I exported them from mlvapp using the MLV flatpass method. this is because the original files are too big to upload directly to google drive.
if you need other files or other tests I can do it
Thanks for the help

https://drive.google.com/drive/folders/1-COgSnAwgprsIif1iA1tzZ2iQcoonMWL?usp=sharing

[theBilalFakhouri]

@GianlucaM83

Can you cut MLV files even further? file sizes are still a little huge to me.
2 Seconds per clip should be enough.

[GianlucaM83]

Yes of course, sorry
Link to the same folder but with smaller files:
https://drive.google.com/drive/folders/1-COgSnAwgprsIif1iA1tzZ2iQcoonMWL?usp=sharing

[theBilalFakhouri]

Thanks for reporting, it's CMOS 7 issue, a little tweak should fix the problem, I will release a new build soon.

Do you know if this problem happens in other presets? or when using 25 FPS with some presets?

[GianlucaM83]

No problem.
unfortunately i only got to try these two presets at 23.973 fps. Maybe later or tomorrow i will be able to try the other presets at 25 fps too and I will report the results 

[theBilalFakhouri]

Thanks, it would be cool if you test every preset so we can fix all issues if there are ones.
I did only test presets indoor, didn't notice problems. as you said, it seems to happen (the artifacts) at certain lighting conditions.

Here is a new build, hopefully will fix artifacts for 4K/UHD 1x3 presets:
crop_rec_4k.2022Jun29.100D101.zip

Please re-test mentioned presets, no need to hurry, take your time.

[GianlucaM83]

I did some more tests with new build. I shot some raw video under the same lights conditions as last time.
Used only UHD 1X3 and 4k 1x3 presets in both 25fps and 23,973 fps.
Unfortunately, the results are the same as last time. On the top of the frame I have that strange effect. videos shot indoors have no problem.
I noticed a few videos with corrupted frames at the beginning.
I share the link to the folder where I uploaded all the mlvs files. Inside it you will find subfolders with the name of the used presets.
Take a look when you can
If you need more tests just ask 
https://drive.google.com/drive/folders/145oB6eJZbft6W7xLb768njG7Il9LAoUR?usp=sharing

[theBilalFakhouri]

Weird, I am unable to reproduce these artifacts on my 100D, tried with a lot of lighting conditions, there are no artifacts.

Could you share your *camera and ML settings? does these artifacts appear in LiveView too?
Are you using FPS override? or any other settings?

Is focus box centered on screen? e.g. turn off crop mode, press zoom button, make sure x1 printed on screen (by Canon overlays), press SET button to center the focus box, select UHD or 4K preset, artifacts are still there?

...videos shot indoors have no problem.

It actually does have the same problem, look closely at all indoor clips (above silver Coca Cola), in some areas the artifacts are there but they are not very visible.

I noticed a few videos with corrupted frames at the beginning.

How to reproduce the issue? .. I never had corrupted frames . .
*Your camera/ML settings please?

*For ML settings, please zip SETTINGS folder inside ML folder in SD card and upload it somewhere, but first make sure your settings do produce the artifacts.

I think I have another solution for the artifacts, used by Danne on EOS M for the same presets which is changing vertical image offset a little to the top, this way both UHD/4K 1x3 presets won't have accurate centering on sensor (I am trying to avoid this option) but at least it will clear the artifacts . . let's try this option as last resource, will share images show the current offset and the new offset if we decided to do it this way.


@100D users:
Do you have these artifacts when using 4K/UHD 1x3 presets?

It looks like this (vertical lines in the top of frame):

[GianlucaM83]

I am not using any particular setting.
Almost all settings are the default ones except these:
I only enabled the "focus peack" and " raw histogram" in the overlay section. In the RAW video section I activated the two options "small hack" and "more hacks lvface+aewb". Fps ovveride is not active.
In the canon menu I set the video format to PAL and the recording format to 1920x1080 25 fps.
I have tried disabling crop mode, putting the focus rectangle in the center in 1x mode and re-enabled crop mode but the problem remains.
I noticed that these artifacts are also visible in the preview on the live view, before I start recording and while I am recording.
These are the modules I have enabled:
crop_rec - dual_iso - mlv_lite - mlv_play - mlv_snd - sd_uhs.

This is the configuration file:
https://drive.google.com/file/d/1H-n_Q9_rn-ioQV_99a5FQO4MHXrV3TyP/view?usp=sharing

Thanks

[Danne]

Disable histogram.

[GianlucaM83]

@Danne
I did it but nothing changes. I also disabled everything in the overlay section. I also tried a fresh installation but the artifacts remain.
I leave a short clip with the operation I usually do to record raw video. to see if I missed something.

One more thing, I tried the 1920x1280p 1:1 preset it seems there are no artifacts with it.
https://drive.google.com/file/d/1YUVXB5CODWrKc8VtWEDrEE326ojRB6hn/view?usp=sharing


I personally think it would be helpful if some other users would also provide their feedback on this.

[theBilalFakhouri]

Thanks for the video, I will provide some more tests to run later, but for now please try:

-Set White Balance to specific value instead of Auto.
-Switch to Photo mode, go to ML settings, Expo tab and turn on ExpSim.
-Restart camera.

Now try UHD or 4K preset, artifacts are still there?
If yes, I am going to provide another set of tests.

-Does 4.5K and 4.3K 1x3 presets work without these artifacts? try 23.976 and 25 FPS.
-If you want to upload MLV clips, please cut them to 5 frames instead of two seconds, that would be enough too . .

I started to suspect that it could be your camera sensor is slightly different from mine (slight hardware differences?), I could be wrong of course . . BTW, where was your 100D manufactured? (you can read this on sticker in the bottom of camera)