Bilal's crop_rec_4k experiments for 650D / 700D (T4i / T5i)

[theBilalFakhouri]

New builds are posted in the first post for 650D/700D

Changes:

crop_rec:
-Switching between presets now is faster
-Fixed a conflict with crop_new which made "Crop mode V2" in most cases has no effect
-Some code optimizations

RAW video with HDMI:

-Fixed a regression introduced back in "11 Dec, 2016" by this pull request, regression symptoms:
(These symptoms happen when the camera is connected to HDMI monitor, no matter what's the output):

-LiveView freezes after recording some RAW video clips, or when toggling between mv1080 and x5 modes when RAW video is enabled
-Weird flickering/glitches in LiveView when the camera startups up with RAW video enabled

This issue is still presented in official Nightly builds, and probably all of DIGIC 5 cameras are affected, maybe except 5D3, (this issue isn't related to HDMI overhead things).

Some details:
ML use EDMAC write channel #18 for RAW data stream (something related to RAW video recording), back then a1ex asked the developers to find a none-used channel by Canon firmware for each model, channel #18 was available (and worked across all models except for 5D3 1.2.3), so we have used it for years . . without a problem, until some days ago I found out this channel is being used by Canon when connecting the camera (my 700D) to HDMI monitor and causes the issues above . .

Well, I started looking for another available EDMAC write channel, I found channel #3 being the best, not used by Canon in anything? (Not used in HDMI, capturing still images, recording H.264, and so) beside channel #3 didn't introduce corrupted frames, same performance as #18 channel in addition to no issues with HDMI connection .

[Danne]

Cool. You've been busy lately . Would be nice adding some of this to eos m. How would I change edmac channel and hdmi fixes?

[Icaab]

New builds are posted in the first post for 650D/700D

Changes:

crop_rec:
-Switching between presets now is faster
-Fixed a conflict with crop_new which made "Crop mode V2" in most cases has no effect
-Some code optimizations

RAW video with HDMI:

-Fixed a regression introduced back in "11 Dec, 2016" by this pull request, regression symptoms:
(These symptoms happen when the camera is connected to HDMI monitor, no matter what's the output):

-LiveView freezes after recording some RAW video clips, or when toggling between mv1080 and x5 modes when RAW video is enabled
-Weird flickering/glitches in LiveView when the camera startups up with RAW video enabled

This issue is still presented in official Nightly builds, and probably all of DIGIC 5 cameras are affected, maybe except 5D3, (this issue isn't related to HDMI overhead things).

Some details:
ML use EDMAC write channel #18 for RAW data stream (something related to RAW video recording), back then a1ex asked the developers to find a none-used channel by Canon firmware for each model, channel #18 was available (and worked across all models except for 5D3 1.2.3), so we have used it for years . . without a problem, until some days ago I found out this channel is being used by Canon when connecting the camera (my 700D) to HDMI monitor and causes the issues above . .

Well, I started looking for another available EDMAC write channel, I found channel #3 being the best, not used by Canon in anything? (Not used in HDMI, capturing still images, recording H.264, and so) beside channel #3 didn't introduce corrupted frames, same performance as #18 channel in addition to no issues with HDMI connection .

Thank you very much for such a fast feedback. I downloaded the firmware and tried it. Yes, the switching problem has gone away, but the main problem I’ve always had to worry about with the cropping regime has not been solved:
There is no convenient height in the frame when you preview. This can be eliminated by turning on another preview mode, but for the filming of important work, this is not all and will only cause total discomfort.
For convenience, I flooded the phone footage of the camera settings. There’s a preview comparison between the first crop of fashion and the second one, and you can also see the problem of the shutter value, which when changed on the screen remains in terms of its numerical value the same
https://mega.nz/file/F9AzmAwK#2Lx44NfjflSKbrUJMFdB4bjzfavNFw2OQ9KrxCOMvWM

[theBilalFakhouri]

Hello @Danne

I have updated the source code, you will find it in the first post if you want to take a look there, it's really easy,

To change EDMAC write channel for RAW stream from edmac-memcpy.c file:

Code: [Select]

#if defined(CONFIG_5D3)
uint32_t raw_write_chan = 0x4;  /* 0x12 gives corrupted frames on 1.2.3, http://www.magiclantern.fm/forum/index.php?topic=10443 */
#elif defined(EVF_STATE)
uint32_t raw_write_chan = 0x12; /* 60D and newer, including all DIGIC V */
#endif
Change it from:

Code: [Select]

uint32_t raw_write_chan = 0x12; /* 0x12 = channel #18 */ to

Code: [Select]

uint32_t raw_write_chan = 0x3; /* 0x3 = channel #3 */
Not sure if channel 0x3 would work fine on EOS M you need to make tests, check channel activity with edmac.mo module, unload all modules, load edmac.mo, if channel #3 stays like this "0:     <w!>" what ever you do (Capture still photo, record H.264 video, connect HDMI monitor) then this channel isn't being used by Canon at these cases (that's what we need), now compile a build with 0x3 channel if you didn't, try recording RAW video in high frame rates like 1736x586 @ 60 FPS in lossless, try other crop modes, and look if there a corrupted frames or not compared to when using channel #18

Some channels appear to produce more corrupted frames than others, luckily channel #3 have same performance as channel #18 and without HDMI and RAW video issues .

[theBilalFakhouri]

..but the main problem I’ve always had to worry about with the cropping regime has not been solved:
There is no convenient height in the frame when you preview. This can be eliminated by turning on another preview mode.

From the first post:

-Crop mode V2 module (crop_new) with Arbitrary Resolutions:
...
...
-This module doesn't include any recent preview fixes.

Currently if you want to use 1x3 modes with real-time correct preview you need to use crop_rec "Crop mode", and unfortunately there is no 5K option (*5208x2214), there is 4.3K (*4320x1836) 2.35:1 AR.

*After stretching.

Implementing real-time correct preview in crop_new "Crop mode V2" requires further research and time, processing 1736x2214 in LiveView might not be possible, even if it was possible we may have another problems like corrupted frames due to hardware limitations and limited memory or limited memory bandwidth.

[theBilalFakhouri]

..and you can also see the problem of the shutter value, which when changed on the screen remains in terms of its numerical value the same..

Okay, I will release two builds for 650D like 700D, this issue affects some 650D users , will try to release a second new build for 650D ASAP solves shutter issue.

[Icaab]

From the first post:

Currently if you want to use 1x3 modes with real-time correct preview you need to use crop_rec "Crop mode", and unfortunately there is no 5K option (*5208x2214), there is 4.3K (*4320x1836) 2.35:1 AR.

*After stretching.

Implementing real-time correct preview in crop_new "Crop mode V2" requires further research and time, processing 1736x2214 in LiveView might not be possible, even if it was possible we may have another problems like corrupted frames due to hardware limitations and limited memory or limited memory bandwidth.

Thank you. Sorry to bother you with the same question. I wanted to finally wait for the correction of this moment to finally start shooting what I wanted). But I guess I’ll have to work with 4.3k in crop mode 1. And I fully believe that crop mode 2 is more likely to have a good source than the first )

[mlrocks]

Alex and you all are magicians, making 10-year old consumer level still cameras competing with the latest crop of professional video cameras, disrupting the whole Hollywood business. Kudos to you.
I want to not only compete with the big brothers like Red Weapon and Alexa LF, but also to beat them. I am using Alex's crop2 to get 1x3 5200x1160, i.e., 1736x1160, so it has an aspect ratio of 5:1. The 650D can do this at 10-bit raw 23.975 fps continuously, about 57 MB/s. The preview is good, I can use either raw video module to get the wide and narrow correct frame view or use raw video mlv module to get the full screen anamorphic preview which I think is better in this case because it uses the whole screen space. I like the aspect ratio 5 footage. The problem is the color taint. There is severely color taint in footage. Hopefully the color taint can be removed somehow to make this mode practical. Thanks a lot.

[mlrocks]

I did some concept testing. Using Alex's crop2 new module, set full width 1x3, 1736x1947, i.e., 5200x1947, aspect ratio: 2.67. Bit rate: 10. Using raw video module, 14 bit lossless. It is about 64 MB/s. If ISO is set to 100 to 200, this mode can be continuous. If 400-800, can be over 1 minute. It seems that even going up to 6400, the digital noise does not increase drastically. This is different from 5d3, which has a flat low digital noise at <ISO 3200, but increases rapidly at 6400, and drastically so at 12800 and 25600, rendering the ISO >6400 practically not usable.
Hopefully, color taint can be removed in this mode. Also, using Alex's crop2 new module, the AF does not need to be disabled. However, the crop module requires the lens to turn off the AF.

[mlrocks]

I tested in Alex's crop new module full width 1:3 1736x1736 aspect ratio 3. I can do 10 bit 14-bit lossless 24 p continuously. The data is about 56 MB/s. There is color taint, sometimes red in darkness sometimes green in darkness.

[theBilalFakhouri]

@mlrocks

There is only color tint/taint in LiveView when using lower bit-depths from crop_rec/crop_new submenu, this in normal and doesn't affect RAW footage

Let me check what you are having by providing a sample MLV file which has the taint issue, 5 seconds are enough, you can shorten MLV clip in MLVApp (export as MLV fast pass) then upload it somewhere and share the link

ISOs which are above 3200 are digital on 650D/700D and won't improve shadows it will just make noise more visible, there is a tiny (not noticeable) improvement between ISO 1600 and 3200 ISO, it better to keep ISO lower than 1600 for the best results in general on these cameras

[theBilalFakhouri]

@Icaab

It seems only your 650D has shutter value issue (first reported 650D), so we need to check shutter blanking addresses from your camera, but before doing that I want you to check if Danne crop_rec_4k build for 650D have the same issue on your 650D, try this build from Danne (outdated build, no longer maintained), but it would be useful for our test:
crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Apr29.650D104.zip

Test it and feedback, does it have shutter issue?

Also, does Dual ISO work on your 650D? or it give ISOless errors?

[bern047]

I have the shutter problem as well on the Canon 700D and Iso here in the UK, but with the right build it works really well, thanks Bilal

[alt1]

I use autofocus and I have no ability use the crop_mode in the last version with autofocus, but I could use it in the previous one (crop_rec_4k.2021May24.650D104.zip )
Bilal, would you please add the trigger anywhere in submenu to enable crop_mode menu with autofocus, maybe unchecked by default.

[Icaab]

@Icaab

It seems only your 650D has shutter value issue (first reported 650D), so we need to check shutter blanking addresses from your camera, but before doing that I want you to check if Danne crop_rec_4k build for 650D have the same issue on your 650D, try this build from Danne (outdated build, no longer maintained), but it would be useful for our test:
crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Apr29.650D104.zip

Test it and feedback, does it have shutter issue?

Also, does Dual ISO work on your 650D? or it give ISOless errors?

Yes, I have problems with double ISO, therefore I do not use it (isoless error)

I put the firmware you suggested. The Dual ISO started to work, and the problem with shutter value - did not disappear (this manifested when included modules for shooting in RAW ).

[alt1]

@Icaab
Also, does Dual ISO work on your 650D? or it give ISOless errors?

I have the ISOless PH err(2) error with Dual ISO on my 650d, but it seems there is the solution

-The ISOless PH err(2) is still there when i activate Dual ISO in M Mode. Starting in P mode and then moving to M mode works.

[Icaab]

I have the ISOless PH err(2) error with Dual ISO on my 650d, but it seems there is the solution

Thank you for your decision. I hope they will correct this error in the new update.

[Icaab]

Hello. I have a question: I understand perfectly well how to use the modules for shooting in Crop mode and RAW, but again, I would like to clarify the profile for the camera:
When shooting with such modules - the profiles do not work and we get the image in RAW as you see when the standard style is turned on ,but there is a certain sharpness there.
I take it I have to put it down at the very least, so the video can be even better for post-production? Or does it not make any sense and in RAW format still comes a certain sharpness?
https://ibb.co/NTDHPs3

[Icaab]

And one more question:
In my artistic design, it is necessary to shoot in the scene a computer monitor turned on.
When you look at the footage, you see a little aliasing on the monitor.
(I shoot and in 1x3 and procedure in MLVApp, as is required, so that in other cases the muar and aliasing do not manifest)
Can do anything other than buy optical low pass filter?

[theBilalFakhouri]

I have the shutter problem as well on the Canon 700D and Iso here in the UK, but with the right build it works really well, thanks Bilal

Welcome

I use autofocus and I have no ability use the crop_mode in the last version with autofocus, but I could use it in the previous one (crop_rec_4k.2021May24.650D104.zip )
Bilal, would you please add the trigger anywhere in submenu to enable crop_mode menu with autofocus, maybe unchecked by default.

Yeah, I disabled the ability to use autofocus when crop_rec "Crop mode" is ON due to stability reasons, which modes you want to use autofocus with?

Autofocus might be stable only with 1440p 1:1 and 3K 1:1 modes, so I may re-enable autofocus with these modes.

Yes, I have problems with double ISO, therefore I do not use it (isoless error)

I put the firmware you suggested. The Dual ISO started to work, and the problem with shutter value - did not disappear (this manifested when included modules for shooting in RAW ).

Okay, we need to check your camera shutter registers addresses, I will write another post regarding this and how to it.

Hello. I have a question: I understand perfectly well how to use the modules for shooting in Crop mode and RAW, but again, I would like to clarify the profile for the camera:
When shooting with such modules - the profiles do not work and we get the image in RAW as you see when the standard style is turned on ,but there is a certain sharpness there.
I take it I have to put it down at the very least, so the video can be even better for post-production? Or does it not make any sense and in RAW format still comes a certain sharpness?
https://ibb.co/NTDHPs3

Picture style doesn't affect RAW data, only H.264 video and LiveView

And one more question:
In my artistic design, it is necessary to shoot in the scene a computer monitor turned on.
When you look at the footage, you see a little aliasing on the monitor.
(I shoot and in 1x3 and procedure in MLVApp, as is required, so that in other cases the muar and aliasing do not manifest)
Can do anything other than buy optical low pass filter?

Shooting a monitor is a nightmare for cameras, popular solution is to make the monitor out of focus (using camera lens) a bit, but in your case there are many subjects in the frame, so you probably need to change monitor position, or use a wider aperture . .

1:1 modes (have higher crop) might help to reduce aliasing/moire on monitor . . I have no idea if optical low pass filter can deal with monitor aliasing/moire . .

[Icaab]

Welcome

Yeah, I disabled the ability to use autofocus when crop_rec "Crop mode" is ON due to stability reasons, which modes you want to use autofocus with?

Autofocus might be stable only with 1440p 1:1 and 3K 1:1 modes, so I may re-enable autofocus with these modes.

Okay, we need to check your camera shutter registers addresses, I will write another post regarding this and how to it.

Picture style doesn't affect RAW data, only H.264 video and LiveView

Shooting a monitor is a nightmare for cameras, popular solution is to make the monitor out of focus (using camera lens) a bit, but in your case there are many subjects in the frame, so you probably need to change monitor position, or use a wider aperture . .

1:1 modes (have higher crop) might help to reduce aliasing/moire on monitor . . I have no idea if optical low pass filter can deal with monitor aliasing/moire . .

Thank you.
I’m going to do a different composition, and I’m going to use focus advice to shoot the monitor.

[Icaab]

Hello. I have a problem: if you try to record a video with a microphone (boya by-m1) connected, the recording of the sound is disabled during the recording. Without a microphone, the sound is recorded and included in the raw settings.
(It’s probably not a firmware problem, but I don’t have any way to test it with another device right now. That’s why I’m interested, just in case.)

The problem was easily resolved. You can delete this unnecessary message .

[elenhil]

Bilal, sorry to ping you in the wrong thread, but did you get any luck obtaining ArcziPL's source code? I would really love to see your work combined for a better 70D experience.

[theBilalFakhouri]

@Icaab

Okay, cool , well, I can't really delete replies or posts for other users, that's a moderator role, no problem with keeping your reply.

Bilal, sorry to ping you in the wrong thread, but did you get any luck obtaining ArcziPL's source code? I would really love to see your work combined for a better 70D experience.

Hello @elenhil, feel free to ask anytime . .

I got the source code from ArcziPL yes, but I can't really do much for 70D without having the hardware, at some point I was going to borrow 70D for some time, but plans changed, I don't mind working on 70D if I got the body.

I think higher resolutions in 1:1 and 1x3 are possible on 70D, but I have no idea if we can get real-time correct preview for 1x3 modes, I need to take a closer look, e.g 5D3 is different from 700D which means 5D3 need a new understanding about how preview works, 70D might be similar to 700D (easy) or 5D3 (currently difficult).

[Icaab]

Bilal, hey.
Tell me: Is there such a opportunity that in the future in the firmware 650d/700d it will be possible to shoot timelapse with video so that all settings are flexibly adjusted ( shutter speed, for example) as on EOS M?