Bilal's crop_rec_4k experiments for 100D / SL1

Started by theBilalFakhouri, June 06, 2022, 12:50:10 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

theBilalFakhouri

Hello @soundfan
Thanks for your words!

Quote from: soundfan on February 22, 2023, 03:50:40 PM
I managed to successfully install ML, update it with this crop_rec build from June 19th (is this the latest, recommended build?)..

Yes, that build should work for you. If you have artifacts in 1x3 presets, try this build.

Quote from: soundfan on February 22, 2023, 03:50:40 PM
But many crop settings result in slow camera response, corrupted live view, reduced field of view and even freeze the camera.
Sometimes a mode seems to work, I can access higher resolutions and make continuous video. The next time I try to do the same it says max resolution limited to xy, no matter what I try.

Could you share a video shows what are you doing then upload it on Youtube?
If you are using "Crop mode" and some high resolution presets like 1440p and 3K, you need to turn off then on RAW video multiple time to apply the new preset:

Quote from: theBilalFakhouri on June 15, 2022, 12:03:36 PM
Note: You will need to turn off and on RAW video multiple times after selecting 1440p or 3K presets, to let RAW video update its parameters!!

Will try to fix this issue later this year.


Quote from: soundfan on February 22, 2023, 06:39:23 PM
Beside that it records only 2 seconds now, although my SD supports up to 85MB/s @240MHz. Is the higher write rate only supported in the experimental crop mode?

I was trying to do 4K recording with the crop_rec module. And this caused the problems mentioned before.

Which preset are you using which give you 2 seconds?

You need to pay attention to required write speed, if it was too high (higher than the camera can actually writes) you will have limited recording times.
You can use hacks under RAW video submenu, also use lower bit-depths (from Crop mode submenu, such as 12/11/10 bit-depths) and lossless compression (always select 14-bit lossless in RAW video submenu) to reduce required write speed and increase write speed in LiveView.

Also, set Picture quality to RAW from Canon menu, then restart the camera (for once), this will free 31 MB of memroy. All of these tips are covered in 650D / 700D topic.

Again, if you could share a video shows what are you doing that would be helpful to me.


Quote from: soundfan on February 22, 2023, 06:39:23 PM
Also I recognized that the camera heats up very fast when testing with live view and playing with the ML menu. Is that normal?

Quote from: soundfan on February 22, 2023, 08:02:59 PM
Only thing I recognized is that write speed is only stable as long as camera temp is below ~45°C. Beyond that anything can happen. Now that I don't need to try 'n error for hours anymore I guess that overheating will no longer be an issue.

I don't what "very fast" means, that depends on many factors like ambient temp, camera power type (Battery, AC adapter, power bank) . . Does it heat up in same way without ML?
Usually iirc my 100D / 700D are around 45 to 55 when using Battery, ~21 ambient.

But I have noticed that with SD overclocking especially at 240 MHz makes SD card heat up faster (also the surrounded components), on 100D at some point this affect write/read speeds (when SD card get too hot):

Quote from: theBilalFakhouri on September 15, 2022, 08:28:05 AM
-Side effect I noticed (at least using my card and 100D):

● It happens after SD controller/card become too hot (I mean it) write speed performance will drop from 90 MB/s to 72 MB/s in benchmarks (Play mode), or from ~80 MB/s to ~62 MB/s for RAW video recording.
● After a camera restart SD card will restore full write speed (90 MB/s in Play mode, ~80 MB/s in RAW video recording) and you should be able to record more minuets until it drop again to 72 MB/s in Play mode or 62 MB/s while RAW video recording.

On 700D / 5D3 no such issue there. You can make sure if this happening to you by running benchmarks in Play mode when you notice the drop, compare it to when SD card is at cool temp.
Other speed drop issue: perform low level format in camera, at least for once.

theBilalFakhouri

Quote from: soundfan on February 22, 2023, 08:02:59 PM
QuoteGeneral notes:
-You can load both crop_rec and crop_new modules, but don't activate them at the same time (use "Crop mode" or "Crop mode V2").
-Use maximum aspect ratio and and resolution from mlv_lite when using crop_rec presets with real-time correct preview, otherwise mlv_lite won't crop the RAW data correctly.

With this pre-settings everything works now as expected.

The note in red color should only affect how RAW data is being cropped, shouldn't be related to issues which you are having.

soundfan

Hi Bilal,

thanks for your answers.

I'm not sure what I'm doing wrong. I wanted to make a video showing my problems and at that point everything worked. No matter which resolution or bit depth. Always continuous recording.

Next minute write speed drops to 20MB/s and nothing works anymore. I got the feeling that the sd_uhs module is not working stable.

Here are two videos anyway. Perhaps you see something I don't.

https://youtu.be/xScs1yYDfcw

https://youtu.be/Jqk7S25dpmw


Many thanks for your support in advance.
EOS 100D... just beginning

soundfan

one more thing I found when bench testing SD: It now runs first write test @83.4MB/s but during first run of read test speed dros at frame 59 to 20MB/s, also for the 2nd read and write cycle.

I tried to copy sd_uhs module again, but this didn't work. Somehow I think the crop high res writing is not stable on my camera.

Is there something I can do to support debugging?
EOS 100D... just beginning

theBilalFakhouri

Thanks for sharing the videos, everything seem fine, except for SD card speed drop. Yes, it seems 240 MHz OC isn't stable for you, that's strange.

Which sd_uhs version are you using?

soundfan

Actually I used your latest one just yesterday:

https://www.magiclantern.fm/forum/index.php?topic=25841.msg234228#msg234228

What I didn't try yet is a low level format of SD inside the camera before testing. This I will try next this afternoon.
EOS 100D... just beginning

theBilalFakhouri

Hmmm, okay
low level format won't help in case if speed drop to 21 MB/s (switch to 48 MHz).

Things to try:

1- Benchmark SD card in PC using fast card reader, share the result
2- Try 192 MHz in camera, run benchmarks, does it work? (don't forget to restart the camera after selecting 192 MHz)
3- Redo step 2 but with 160 MHz
4- Switch AccessMode to SDR50 in camera, select 240 MHz, restart camera, run benchamrks, does it work?

soundfan

Yes, I will try those again.

Thing is benchmark worked already with 84MB/s write speed and 86MB/s read speed couple of times. Trying again after several tests of different resolutions leads to the drop to 21MB/s. Unfortunately I'm not yet able to put a handle on it.

Anyway, I will continue trying this afternoon for sure.

And thanks for your suggestions and continuous support!  8)
EOS 100D... just beginning

Danne


soundfan

EOS 100D... just beginning

soundfan

Okay, I did a lot of test now but results don't make any sense (at least to me).

When I started (cold) camera and run benchmark twice at 240MHz, everything was fine, see pictures below.

https://drive.google.com/file/d/1vW79Px5pmOV-1GP6scQEY2XvRNvK2Qqe/view?usp=sharing

https://drive.google.com/file/d/1xTXHToLQFlgfWJ3HLdjtO8910EFV3jh-/view?usp=sharing

After that I made a recording @ 4k/14bit. It recorded continuously. Then I changed resolution to 4.5K and it went back to 20MB/s. No matter what I tried next (other resolution, different SD speed, turning off and on camera, even remove battery in between), no change.

Next I tried 240Mhz at SDR50 . This resulted in the best benchmark ever, see picture.

https://drive.google.com/file/d/19eeOJvfZhc60fhVAj0CR_RChjPnrw_k-/view?usp=sharing

With this setting I managed to record continuously 4K/14bit, 4.5K/14bit, UHD/14bit without problem. Then I tried another benchmark. First write was 90MB/s again, but already the next reading cycle dropped to 20MB/s again. I switched off camera, unplugged battery, waited a while. Restarted benchmark with this result:

https://drive.google.com/file/d/1AxaCyyTnZCiCbBnMWAmoAkC6ucepC4Eb/view?usp=sharing

At the second reading cycle it dropped to 23MB/s again.

I start to run out of ideas... :'(

EOS 100D... just beginning

theBilalFakhouri

Thanks for the tests, but you missed the first three steps here, waiting the results :)

Quote from: theBilalFakhouri on February 24, 2023, 07:14:57 AM
Things to try:

1- Benchmark SD card in PC using fast card reader, share the result
2- Try 192 MHz in camera, run benchmarks, does it work? (don't forget to restart the camera after selecting 192 MHz)
3- Redo step 2 but with 160 MHz
..

Try recording with 192 MHz and 160 MHz, does it also drop to 21 MB/s after some time?

soundfan

Unfortunately I don't have a fast SD card reader for PC. The one integrated in my laptop was telling me like 26MB/s for the Sandisk Extreme Pro 128GB 200MB/s with the disk speed test from your earlier post. This is kind of ridiculous.

Card benchmark with

160MHz (SDR50): 64.3/71.9/64.8/71.9
190MHz (SDR50): 75.1/drop to 22.7/20.8/21.8

redo 160MHz (SDR50): 63.8/71.9/64.1/71.9

Another test with SDR104:
160MHz: 67.0/71.9/66.8/71.8
192MHz: 78.4/drop to 22.9/20.8/21.7

redo 160MHz: 66.9/71.9/66.7/71.9

160MHz seems to be the only stable frequency right now. No matter if SDR50 or SDR104.

Does this generate any new ideas?
EOS 100D... just beginning

soundfan

Recording in 160MHz seems the be continuous as long as predicted data rate is adequate.

Recording in 192MHz drops almost right away to ~20MB/s. Back to 160MHz it works continuous again immediately.

At least I now know a mode I can work with. Would be nice to get the last MB/s as well, but so far it works.
EOS 100D... just beginning

Walter Schulz


soundfan

EOS 100D... just beginning

theBilalFakhouri

Thanks for the tests, it seems to me it's a SD card issue, but not quite sure.
Tweaking GPIO registers in sd_uhs.c might help or might have no effect.

It's also possible that your SD card needs some kind of configuration which I am not aware of.
Do you have access to other SD card or other DIGIC 5 camera?

Could you share CID info from "SD Overlock" submenu?

For now, please try formatting SD card using SD Memory Card Formatter on PC.
That might help fixing SD card issues if there were any.

Then redo tests at 192 MHz or 240 MHz, does it work better after formatting with mentioned tool?

I will provide sd_uhs.mo with tweakable GPIO values from submenu later, so you can try some values.

Danne

In my build I think it needed some pause when applied on 100d. Might be some patch routine similar to 5diii here. I know because I tried some faster patching that works on eos m but didn't on 100d.

theBilalFakhouri

@Danne

Have you tried my latest sd_uhs on your 100D with your SD cards?
Latest sd_uhs works fine on my 100D, also other 100D users reported the same thing in eariler replies.

soundfan

Did the formatting with SD Memory Card Formatter. Same result as before. Cold camera works for a while @240MHz with 84MB/s, then drops to 20MB/s.

@160MHz works perfectly again, benchmark same as recording.

Here the CID info from that SD card:
MID: 0x03
OID: 0x5344
PNM: SR128
PRV: 8.6
PSN: 0xc0512c54
MDT: 2023/01
CRC: 0x89

whatever that means...  ::)

Tried another card (Intenso SDHC UHS-I Preimium 45MB/s 32GB)

standard speed: 22.0/43.5/22.1/43.5
SDR104:
160MHz: 24.5/71.9/26.7/71.9
192MHz: 15.4/21.7/15.4/21.7  Card seems not to support higher frequencies

SDR50:
160MHz: 26.8/71.4/26.7/71.4
192MHz: 15.3/21.7/15.3/21.7  Card seems not to support higher frequencies

I know that this card is not usable for RAW video recording, therefore I bought the fastest available (Sandisk Extreme Pro 128GB 200MB/s).
Perhaps it is useful to point into the right direction anyway.

Unfortunately I don't have another camera on hand (yet).
EOS 100D... just beginning

Danne

Quote from: theBilalFakhouri on February 24, 2023, 08:31:20 PM
@Danne

Have you tried my latest sd_uhs on your 100D with your SD cards?
Latest sd_uhs works fine on my 100D, also other 100D users reported the same thing in eariler replies.
No time for my baby 100d :(.

soundfan

Back with some more test results:

SD card seems not to be the problem.

I just bought a new Sandisk Extreme Pro 64GB 170MB/s, the most recommended SD card (becomes pretty difficult to get these days).

First benchmarks were promising, see below:
160MHz:
https://drive.google.com/file/d/13MapLJ4ZCe9JpHnSjvgEEl2MLnbhc5qE/view?usp=share_link

192MHz:
https://drive.google.com/file/d/160_bb9xJS6dwOqaOWUloEs5PE5ORBLkT/view?usp=share_link

240MHz (first attempt):
https://drive.google.com/file/d/1K5kmn8IfyB23vPMsJQNU7J33EnteDCP6/view?usp=share_link

Here the CID info from that new SD card:
MID: 0x03
OID: 0x5344
PNM: SR65G
PRV: 8.6
PSN: 0x3fa84F98
MDT: 2022/02
CRC: 0x76

After a few test and camera getting just a little warmer (~38°C, no idea if that is relevant), the recording stopped again and benchmark shows drop to 20MB/s again:

240MHz (second attempt):
https://drive.google.com/file/d/1eKaKcPtwb8Q41pEUbGrxWZEezmRXOomh/view?usp=share_link

So actually the same behaviour like with the other Sandisk Extreme Pro 128GB 200MB/s card.

BTW: How can I post pictures directly in my posts? Most forums allow upload of pictures. I couldn't find any option here. No problem though, can upload to Google drive anyway, but only can share the link.
EOS 100D... just beginning

theBilalFakhouri

Okay, it seems like your 100D could be a little different from mine, either from SD card controller in camera, or some SD configuration.
I need to look into your 100D ROM files, also I need a DebugLog from your 100D:

1- zip ROM0.BIN and ROM1.BIN in ML/LOGS folder and upload it somewhere (like Google Drive) then please send a link to me via personal message (don't post the link here).

2- DebugLog:
    Make a clean install (fromat SD card in PC first, then format it in camera) and install io_trace_full.2023Mar01.100D101.zip build on Sandisk 170 or 200 MB/s SD card, switch the camera to photo mode (no LiveView),
    click on "Don't click me!" from "Debug" tab in ML menu, wait until it finish, it will print "saved" and name of log file on screen, then turn off the camera, you will find the log file in SD card,
    upload it somewhere and share it please.

-The camera will take a few seconds with LED ON until it turn on, that's normal with this io_trace build.


Quote from: soundfan on March 01, 2023, 05:30:45 PM
BTW: How can I post pictures directly in my posts? Most forums allow upload of pictures. I couldn't find any option here. No problem though, can upload to Google drive anyway, but only can share the link.

You need to convert .ppm files first to either PNG or JPEG using some kind of image processing software, Photoshop can do it for example.
Then you will need to upload the PNG or JPEG to image host website like https://imgbb.com/

After uploading it there click on the picture from host website, so you can find these:

embe" border="0

Copy either Full or Thumbnail image BBCode to your post here, then you will have a posted picture on the forum.

soundfan

Thanks again for your help.

Registered to imgbb.com

Test:


Sent you pm.
EOS 100D... just beginning

theBilalFakhouri

Thanks for the LOG and the ROMs, SD configuration (i.e registers which covered in sd_uhs.c) seems identical to my 100D.
Perhaps there is other configuration which work better for 192 MHz and 240 MHz across 100D bodies (assuming it's not a hardware difference), let's start with simple thing:

1. Downclocking read speed to 160 MHz instead of 192 MHz in 192 MHz / 240 MHz presets:
    Please try this sd_uhs.mo, try 192 MHz and 240 MHz, does write speed drop to 21 MB/s?