Crop mode recording (crop_rec.mo) (1:1, RAW/H.264, 25/30/50/60 fps)

Started by a1ex, April 01, 2016, 08:37:14 AM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

a1ex

During the last few weeks I have finally managed to sit down and implement the 3x crop mode discovered by Greg a while ago, and summarized here. This feature could be very useful for wild life, astro, or just for bragging on the forums about how cool your camera is :)

How it works?

It modifies Canon's 1080p and 720p video modes by altering the sensor area that is going to be captured. Resolution and nearly all other parameters are left unchanged.

That means:
- it works with both H.264 and RAW
- works at all usual frame rates: 24/25/30/50/60 fps (with some quirks at high FPS)
- preview, sound, overlays, HDMI out... most of the stuff should just work as expected.

Samples?

update: here's one from kgv5


Not yet, but I have a feeling DeafEyeJedi is already on it :) scroll down :)

Quirks

On 5D3 (other cameras may be different, we'll see):

- framing almost centered (only roughly checked by zooming on a test subject on the camera screen)
- 720p aspect ratio:
   - at 720p (50/60fps), we are sampling the sensor at 1:1 crop, but Canon uses a 5x3 pixel binning
   - that means, H.264 video will be squashed - resize the video in post at 1280x432 or 1920x648
   - however, raw video will have 1:1 pixel ratio (not squashed, just very wide - up to 1920x632)
- there is a small black border at the top of the frame, if you record at max resolution in RAW
- it may have side effects such as sensor overheating, camera exploding or displaying BSODs.

As usual - if it breaks, you get to keep both pieces.

Compatibility

Current implementation only works on 5D3, and I've tested it only on 1.1.3. The module is not yet compatible with current nightlies, so you need a full package (not just the module).

As you can see if you scroll down, it is possible to port this on many other cameras. It's just not very straightforward. But, on the bright side, Maqs is already eager to port it to 6D, and I'm sure others will follow.

Note that 600D and 70D already have this feature from Canon, and 650D, 700D and EOS M already have it in ML with a little hack. All other cameras could already use the crop mode when recording RAW from the 5x zoom view, but with some quirks (mainly bad preview and off-center image). So, this is nothing really new - maybe just a little more usable.

Why a separate build is needed? It's because this module uses an experimental patching library, which seemed to work fine while I wrote the code, but as soon as I took it outside (about one month ago), it crashed almost every time I used ETTR + Dual ISO. I've fixed the bug since then, but you can imagine you don't want this level of "stability" in the nightly builds.

However, this library paves the way to implementing the long-awaited ISO tweaks (with real ISOs lower than 100, including a small dynamic range boost). I've also used this library as a backend for low-level tweaks such as choosing FAT32 or exFAT when formatting a card from the camera. So, let's test it and iron out all the quirks!

Downloads

- source code
- 5D3 1.1.3: magiclantern-crop3x.2016Apr01.5D3113.zip (build log)
- 5D3 1.2.3: magiclantern-crop3x.2016Apr01.5D3123.zip (build log) (confirmed by Hans_Punk)
- other cameras: hopefully coming soon

TODO

- port it to other cameras
- merge into nightly builds

Hacking

- grab adtg_gui.mo and raw_diag.mo from the ISO research thread, then:
- try to understand what those registers do, and which ones need to be changed to achieve various effects
- check black bars with raw_diag, option OB zones (trigger with long half-shutter press in LV)
- optional: check DR, SNR curve, full well and read noise with raw_diag, option "SNR curve (2 shots)", trigger with "Dummy Bracket"
- take your time to read and experiment; it's very time-consuming, but once you get the hang of it, be careful - it's addictive.

Porting checklist

- clean image (without weird artifacts)
- clean turning on and off, in all the supported video modes
- clean switch to/from other modes (5x/10x zoom, other video modes, photo mode - these should not be affected)
- black bars should be larger than or equal to the values assumed in raw.c (check with raw_diag OB zones)
- centered image: put the focus box in the centered image and zoom in; the subject should not move
- menu: if there is any mode where the patch is not working, it should print a warning

Credits

Greg - original findings on 500D
Levas - for finding the equivalent registers for 6D
mothaibaphoto - for finding the 5D3 register values for 30/50/60 fps
Maqs - for the lightweight code hooks used in the backend
g3gg0 - for laying out the foundation about ADTG registers, ENGIO registers and other low-level stuff that tends to be forgotten once it's up and running.

Hans_Punk

Thank you!

Works great, and can confirm crop mode works with external monitor (smallhd 501) on build 1.2.3 perfectly.
If sensor area remapping has been 'cracked' - could we ever see a APSC sub-mode crop option?

Having ability to use 'APSC only' or super 35 cine lenses without vignette on full frame without significant loss of quality would be truly outstanding.

p.s - I also noticed the April 1st screen flicker gag again  :D

DeafEyeJedi

Quote from: a1ex on April 01, 2016, 08:37:14 AM
Not yet, but I have a feeling DeafEyeJedi is already on it :)

Just tested this on 5D3.123 (w Lexus 32GB 1000x) and confirmed all to be working. Quick video below. It's literally 2 am in here.

https://vimeo.com/161157449

More in depth testing to come (w KB 128GB 1066x) and will probably post results along with some more screen flow as well!

As to be expected, I can also confirm the infamous April Fool's flicker gag once again ... There's an temporary easy-workaround to it.  ;)

Happy April, Everyone!

*edit*

Here's one shot entirely in 30p H.264 (Faithful 0,1,2,0) while shooting in crop_rec.mo w 5D3, 7D and mixed in w a rented C100II for slo-mo (Standard EOS) as well for this event:

https://vimeo.com/167454180 (Sorry fellas the couple decided to make their wedding video private)

5D3 Non Crop-Mode 35L @ ISO 3200


5D3 Crop-Mode (1:1) 35L @ ISO 3200


Where's DeafEyeJedi at? 7D + 6.5mm @ ISO 6400


From where I was standing and just to be able to push a few buttons (preferably one) in order to get into crop-mode is priceless.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

a1ex

Quote from: Hans_Punk on April 01, 2016, 10:25:05 AM
If sensor area remapping has been 'cracked' - could we ever see a APSC sub-mode crop option?

If one figures out how to do a 2x2 pixel binning...

I remember ADTG[0x800C], on 60D and 600D IIRC, does this (see adtg_gui source, where they are documented):

    {DST_ADTG, 0x800C, 0, "Line skipping factor (2 = 1080p, 4 = 720p, 0 = zoom)"},


Let's say you adjust this line skipping register to 1 (that is, read every two lines). Now you have a little problem: you only captured two Bayer colors (either red/green or green/blue). Something like this:

Quote from: Greg on January 05, 2015, 11:45:35 PM



You would need a smarter binning/skipping algorithm for 2x2 - that is, read two lines, skip two lines. Or, readout at 1:1 (good luck doing that at 24 fps) and resize in hardware (same).

And I'm afraid the binning patterns might be hardcoded in the sensor chip.

Of course, nothing stops you from fiddling with these registers, just be sure to take the battery out as soon as you suspect a burning smell coming from the sensor ;)

squig

Cool. Thx a1ex.

>:( That flicker's been driving me nuts all day. I was ready to fix it with a sledgehammer.

GutterPump

Amazing feature ! Thanks to all developers :)

I expected this news for a while..

I will ask my troll question but for future builds, will we be able to go beyond 1080?

a1ex

After tinkering a bit with the registers, I've got a 3x3 binning in 720p (50/60 fps) on 5D3. This one is very easy to apply with adtg_gui.

CMOS[1] = 0x748         
ADTG*[0x800C] = 2


This gives square pixels in RAW without any crop (just 3x3 binning), so for slow motion, you would have 1920x664 RAW that does not require any de-squeezing. It would look just like 1080p RAW, with vertical crop to 664 instead of 1280.

It should be also be possible to have 1x3 or 3x1 binning (with really squashed image), probably interesting for anamorphic lens users.

Anyone interested in any of those presets?

DeafEyeJedi

Absolutely, @a1ex and I am all for this.

Just about to get home from work and ready to start the weekend with your latest gems.

Speaking of I was able to take the 5D3 to work earlier and shot some in H264 and MLV.

Curious to see how it looks when de-stretching these ugly looking squashed footage ... However I'm looking forward to this next step with you all!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Quentin

If there were not some links, I would have thought that was April's fools lie.
What an incredible tool.
I tried it on my 5dm3 1.1.3
I managed to record 60FPS up to 9 seconds and then it stopped.
50FPS was possible without interruption.
I also noticed the vertical shorter size and the black bar at the top.

Considering that we are on early stage, I am confident that ML developers will find the magic spell :D
Thank you very much for making our camera so powerful.

a1ex

Updated source code with 3 new presets:

- 3x3 binning in 720p (unsquashed slow motion at 1920x664 in RAW, image cropped only vertically; preview and H.264 are squashed)
- 1x3 binning: read all lines, bin every 3 columns (extremely squashed image)
- 3x1 binning: bin every 3 lines, read all columns (doesn't work well, image has artifacts)

The last preset could be useful for anamorphic lenses, but unfortunately I couldn't get it work without image artifacts. It might be possible to fix it by postprocessing, or it might be possible to find some register values that fixes it, I don't know. If you want to try, edit the source and put .max = 4 in the menu description. If you want to edit registers manually, use this configuration in adtg_gui:

CMOS[2] = 0x10E               ; read every column, centered crop
ADTG*[0x800C] = 2             ; vertical binning factor = 3
ADTG2[0x8806] = 0x6088        ; artifacts worse without it


then switch video mode to 720p and back to 1080p to apply all the changes.

So, my best guess for solving these artifacts would be by tweaking ADTG2[0x8806] and maybe also ADTG4[0x8806]. I wasn't able to find a suitable value, so if you have lots of time in your hands, you are welcome to try. Here's a tip: enable the full-screen magic zoom to see the artifacts.

Build coming later (I hope to fix the black bar issue as well); meanwhile, you can compile the sources.

Levas

Wow, do you guys say that there is finally a way to get 50 or 60 fps raw MLV without squashed image  :D
3 x crop mode at 50 fps and square pixels, nice!

andy kh

5D Mark III - 70D

a1ex

Quote from: Levas on April 02, 2016, 02:33:15 PM
Wow, do you guys say that there is finally a way to get 50 or 60 fps raw MLV without squashed image  :D
3 x crop mode at 50 fps and square pixels, nice!

That's right - 3x and also 1x with the latest update.

New builds:
- 5D3 1.1.3: magiclantern-crop3x.2016Apr02.5D3113.zip (build log)
- 5D3 1.2.3: magiclantern-crop3x.2016Apr02.5D3123.zip (build log)

Presets:
- 1:1 (3x) for 1080p and 720p (same as first post)
- 3x3 binning in 720p (square pixels in RAW 50/60 FPS, full sensor width, vertical crop)
- 1x3 binning: read all lines, bin every 3 columns (extremely squashed image)
- 3x1 binning, image has artifacts, setting hidden (you can edit crop_rec.cfg if you really want to try it without recompiling)


kgv5

Its unbelievable, wow. Thank you for that, soooo useful now, this is huge!
www.pilotmovies.pl   5D Mark III, 6D, 550D

Danne

This is great. Thanks for putting this in my cam. Awesome stuff.

I did a some short test footage comparisons exported with ffmpeg prores4444 out of MLP. Looking at the last dual iso clip, the one called 1:1 3xcrop 50fps 720p(mode) DUALISO I noticed the dual iso artifacts are a lot less noticeable than with the clip before called 1:1 3xcrop 25fps 1080p(mode) DUALISO.Not valid test, not equal settings on camera.

dualiso dng files for comparison.(white balance might be a little off)
1:1 3xcrop 50fps 720p(mode) DUALISO
https://drive.google.com/file/d/0B4tCJMlOYfirXzFFODVCb3UtM2M/view?usp=sharing
1:1 3xcrop 25fps 1080p(mode) DUALISO
https://drive.google.com/file/d/0B4tCJMlOYfiraEtUdkRLXy1HdVE/view?usp=sharing



bouncyball

@Danne: Yep, looks like aliasing of the window frame is almost gone

beauchampy

Unfortunately I'm still getting corrupt frames on this build using 1.2.3 +MLV on 5d3. There's already a thread on this, so if you want to help solve this, we're on it here:
http://www.magiclantern.fm/forum/index.php?topic=5473.msg165242#msg165242

a1ex

Quote from: Danne on April 03, 2016, 10:29:02 AM
Looking at the last dual iso clip, the one called 1:1 3xcrop 50fps 720p(mode) DUALISO I noticed the dual iso artifacts are a lot less noticeable than with the clip before called 1:1 3xcrop 25fps 1080p(mode) DUALISO.

Apples with oranges. Repeat with a static scene, with identical focus in both frames ;)

Danne

hehe, ok, thanks for fighting the placebo.

original DNG files
https://drive.google.com/file/d/0B4tCJMlOYfirRHBCeUh0X19CZGs/view?usp=sharing
https://drive.google.com/file/d/0B4tCJMlOYfirUmxWSnhoSGc1aDg/view?usp=sharing

Quick comparison. Couldn,t get super exact match. Not sure if subtle variations is due to cr2hdr or if I missed out on shutter. Think I nailed it. Anyhow. Can,t check more right now but any variations can be anything. Now it seems 25fps is a little less aliased in the blinds but could be coincidence.

1:1 3x crop 25fps 1080p setting


1:1 3x crop 50fps 720p setting


1:1 3x crop 25fps 1080p setting(cropped 400%)


1:1 3x crop 50fps 720p setting(cropped 400%)

axelcine

@a1ex & Co - I had it on my good ol' 700d and I loved it. And now... same wonderful functionality on my new babies. You're just awesome. Thank you once again for ML; what you're doing is very much appreciated. Yesterday we had a wonderful sunset. I was filming near a lake with some exciting bird life with a long lens, and all the time I said to myself: should have brought the ol' trusted 700d - 1.6 factor + crop mode - o, I wish I could do this with my 5d's.
And today I wake up - and it's there. Magic. Pure m a g i c  l a n t e r n.
EOS RP, 5dIII.113/Batt.grip, 5dIII.123, 700d/Batt.Grip/VF4 viewfinder + a truckload of new and older Canon L, Sigma and Tamron glass

budafilms

@a1ex

Dear a1ex, which is the settings to test this module: should I put in Canon Menu 1280x720 50? Anything else? Thanks.


Sorry, I saw de @DeafEyeJedi video! thanks.

Hans_Punk

@A1ex

Quote
- 3x3 binning in 720p (unsquashed slow motion at 1920x664 in RAW, image cropped only vertically; preview and H.264 are squashed)
- 1x3 binning: read all lines, bin every 3 columns (extremely squashed image)
- 3x1 binning: bin every 3 lines, read all columns (doesn't work well, image has artifacts)

The last preset could be useful for anamorphic lenses, but unfortunately I couldn't get it work without image artifacts. It might be possible to fix it by postprocessing, or it might be possible to find some register values that fixes it, I don't know.

This is incredible, having a fix for 3x1 would be outstanding for those (like me and many others) who shoot anamorphic.

dmilligan

It's probably going to be important to have metadata about the crop mode stored in the MLV file (and also in sidecars for H.264 for people who might want to write auto-desqueeze scripts). Up until now, you could guess the crop mode based on the raw_info dimensions, but with this module, I assume that will no longer be possible.

Danne

+10
QuoteIt's probably going to be important to have metadata about the crop mode stored in the MLV file...