crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

Started by a1ex, April 01, 2017, 11:15:41 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

a1ex

As you probably have guessed from the latest developments (QEMU, EDMAC graphs, JPCORE, EEKO), our understanding on how LiveView works has improved considerably. Finally, all my fiddling with QEMU, at first sight with little or no purpose for the everyday users, starts paying off.

Today, Magic Lantern proudly announces new ground-breaking features that were previously thought impossible or very hard to achieve.

We proudly present....


4K RAW Video Recording!



DOWNLOAD

Twitter announcement





On the 5D Mark III, you now have the following new resolutions:

* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning) - continuous*)
* 1920x800 @ 60p (same as above)  - continuous*)
* 1920x1080 @ 45p and 48p (3x3 binning)  - continuous at 45p*)
* 1920x1920 @ 24p (1:1 square crop) - continuous*)
* 3072x1920 @ 24p (1:1 crop)
* 3840x1536 @ 24p (1:1 crop) (corrupted frames at 1600)
* 4096x2560 @ 12.5p (1:1 crop) - continuous*) at 8 FPS
* 4096x1440 @ 25p (1:1 crop)
* Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter) - continuous*) at 5 FPS!
* Full-width LiveView - decrease vertical resolution in the crop_rec submenu, all the way to 5796x400 @ 48 fps :)

The last feature complements the well-known full-resolution silent pictures - the new implementation will be usable at fast shutter speeds, without the exposure gradient - but with rolling shutter (just like regular LiveView frames).

*) Continuous recording for the above resolutions can be achieved as long as you can get a LJ92 compression ratio (compressed / 14-bit uncompressed) of about 50-55%, with preview set to Frozen LV (previously known as Hacked Preview) for an additional speed boost. Otherwise, you'll have to reduce the resolution or the frame rate.

The following table shows how compression rate changes with ISO and bit depth; please check the figures for your particular scene in the raw video submenu, as they can vary a lot, depending on the scene content.

Quote from: a1ex on April 15, 2017, 01:12:36 PM
Bits per pixel      14  12  11  10   9   8
ISO  100 1/100     61% 53% 50% 48% 46% 43%
ISO  200 1/200     62% 54% 51% 49% 47% 44%
ISO  400 1/400    63% 54% 51% 49% 47% 45%
ISO  800 1/800     65% 55% 52% 50% 48% 46%
ISO 1600 1/1600    67% 56% 53% 50% 48% 46%
ISO 3200 1/3200    70% 57% 53% 50% 49% 47%
ISO 6400 1/6250    76% 60% 55% 52% 50% 48%
ISO 12800 1/12500  79% 63% 57% 53% 50% 49%

Credits: Greg (full-width LiveView), g3gg0 (video timer, DIGIC registers documentation and lots of other low-level insights).

Complete list of new video modes:

                                /*   24p   25p   30p   50p   60p */
    [CROP_PRESET_3X_TALL]       = { 1920, 1728, 1536,  960,  800 }, /* 1920 */
    [CROP_PRESET_3x3_1X]        = { 1290, 1290, 1290,  960,  800 }, /* 1920 */
    [CROP_PRESET_3x3_1X_48p]    = { 1290, 1290, 1290, 1080, 1080 }, /* 1920; 1080p45/48 <- 50/60p in menu */
    [CROP_PRESET_3K]            = { 1920, 1728, 1504,  760,  680 }, /* 3072 */
    [CROP_PRESET_UHD]           = { 1536, 1472, 1120,  640,  540 }, /* 3840 */
    [CROP_PRESET_4K_HFPS]       = { 2560, 2560, 2500, 1440, 1200 }, /* 4096 half-FPS */
    [CROP_PRESET_FULLRES_LV]    = { 3870, 3870, 3870, 3870, 3870 }, /* 5796 */


What else could you wish for?




FAQ

Where's the catch?

This is only a very rough proof of concept. It has not been battle-tested and has many quirks. Some of them may be easy to fix, others not so. In particular:

* It feels quite buggy. I'm still hunting the issues one by one, but it's hard, as Canon's LiveView implementation is very complex, and our understanding on how it works is still very limited.
* Write speeds are high. For example, 10-bit 4096x2500 at 15 fps requires 180 MB/s. 1080p45 should be a little more manageable at 111 MB/s.
* Canon preview is broken in most modes; you need to use the grayscale preview in the raw recording module.
* High-resolution modes (in particular, full-res LiveView) may cause trouble with memory management. This is very tricky to solve, as we only get 3 full-resolution buffers in LiveView, with restrictions on the order in which they must be freed, and lots of other quirks.
* Since these settings were pushed to limit, the risk of corrupted frames is high. If it happens, decrease the vertical resolution a bit (from the crop_rec submenu).
* When refreshing LiveView settings, the camera might lock-up (no idea why). Pressing MENU twice appears to fix it.

May I fine-tune the new modes?

Yes! I've included some of the knobs on the user interface. Normally you shouldn't need to touch these buttons, but if you do, you might be able to squeeze a few more pixels.

Does it work with FPS override?

Sort of. It's not reliable at this point, so it's best not to try yet.

Overheating?

During my tests, I didn't manage to get a sensor temperature higher than 60 degrees. Your mileage may vary.


Risks?

This mod changes some low-level sensor parameters that are not well understood. They were all figured by trial and error, and there are no guarantees about the safety of these changes.

As usual, if it breaks, it's your fault, sorry.

Will it work on other camera models?

I hope so; however, this is an area where I hope to get contributions from others (yes, from you). If these new features don't motivate you to look into it, I wonder what else will.

I'll explain how all this works in the coming days or weeks.

Is it difficult to port to other camera models?

So far, the 3x3 720p mode from crop_rec was ported to EOS M (rbrune), 700D (dfort) and 100D (nikfreak). So it shouldn't be that hard...

Will you port this to my camera model, please?

No, sorry. I have better things to do - such as, preparing the April 1st prank for next year :)

Wait a minute, didn't you say you are primarily a still photo user? Why are you even doing this?

If you look close, the usefulness for video is fairly limited, as the write speeds (and therefore the recording times) are not practical.

But the full-resolution LiveView is - in my opinion - very useful for still photo users. Although the current implementation is not very polished (it's just a proof of concept), I hope you'll like the idea of a 7.4 FPS burst mode, 100% silent, without shutter actuations.

Right now, you can take the mlv_lite module with pre-recording and half-shutter trigger: at 10 bits per pixel, you get 5 frames pre-recorded, and saved to card as soon as you touch the half-shutter button. Or, you can capture one frame for each half-shutter press, with negative shutter lag! (since the captured frame will always be pre-recorded).

And if a burst at 7.4 fps is not enough, you may also look at the 4K modes (12-15 fps).

(I know, I know, GH4 already does this, at much higher frame rates...)

The help menu for full-res LiveView says 5796x3870, but MLV Lite only records 5784x3856. What's going on?

The raw recording modules have a couple of alignment constraints (e.g. can only start cropping from a multiple of 8 pixels, and the size of the cropped area (that goes into the MLV file) must be multiple of 16 bytes (that is, W*bpp/8 + H mod 16 must be 0).

To capture the full resolution, you may use the silent picture module. However, this module is not the best when it comes to memory management and buffering. Currently, you'll get an impressive buffer of 2 frames in burst mode :)

But hey - it outputs lossless DNG!

What about that lossless compression routine?

It's included, although I didn't manage to test it much. There is a lot of room for improvement, but for a proof of concept, it seems to work.

update: also got lossless compression at reduced bit depths (8...12-bit).




P.S. The initial announcement was disguised as an April Fools joke, just like the original crop_rec.

Twitter announcement

From original April Fools post:
Quote

With our latest achievements in wizardry with ARM programming and DIGIC reverse engineering, we can speak of a new era of raw video recording.

On models like the 5D Mark III, the next upcoming releases will feature an improved version of our crop_rec module that delivers the following new resolutions:

* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning)
* 1920x800 @ 60p (same as above)
* 1920x1080 @ 45p and 48p (3x3 binning)
* 1920x1920 @ 24p (1:1 square crop)
* 3072x1920 @ 24p (1:1 crop)
* 3840x1600 @ 24p (1:1 crop)
* 4096x2560 @ 12.5p (1:1 crop)
* Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter).

The last feature complements the well-known full-resolution silent pictures - the new implementation will be usable at fast shutter speeds, without the exposure gradient - but with rolling shutter (just like regular LiveView frames).

Please understand that providing the source code for those highly DIGIC optimized routines is a bit troublesome and will need some extra legal care. After this step is taken and as soon we are finished with ensuring the product quality you are used from Magic Lantern, we will upload the code to our repository.

Consider this being a huge leap towards our next mind boggling goal:

8K RAW Video Recording!



Sample DNG from 5D Mark III, to show that our proof of concept is working:

8k.dng

Stay tuned for more information!


Walter Schulz

Who is ML's officiail bookie? I want to risk some Euro ...

nikfreak

[size=8pt]70D.112 & 100D.101[/size]

g3gg0

this is wonderful! i cant wait to test it on my camera at home!
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Wayne H

INCREDIBLE!!!!! Wonderful job alex, i really appreciate all the hard work you and and the other developers have put in over the years to get magic lantern to where it's at right now.

ULTRA HD RAW 24p on 5d mark iii wooah :) ,so the most obvious question is when will we be able to test out this new update.   

Feels like xmas. :)

Alex you are a genius.

Danne



nikfreak

Nope, I would rather say it's the whole truth as long as no one talks about expects continuous recording:

http://www.magiclantern.fm/forum/index.php?topic=18300.0

8) :P 8) :P :D :D ;D 8) 8)
[size=8pt]70D.112 & 100D.101[/size]

ilia3101

KooooooooL  8) any chance of anything for the mark 2 or other cameras?

beauchampy

Please guys, I can't take the heart ache. Is this an April fools? Specifically the 960px 3x3?

Levas

I love how you get a 8K DNG sample from a 5d mark III, there's some real magic going on there  ;D

Walter Schulz

Quote from: beauchampy on April 01, 2017, 01:29:47 PMIs this an April fools?

You don't expect an answer without tricks, traps, false mirrors and double meaning, do you? No today, not today ...

Greg


Frank7D

Thank you so much for everything you've done! Now, not to be demanding, but 16K is obviously doable so when can we expect it?


a1ex

Committed some of the sources; still playing with it, but those who can compile from source can already give it a try.

hyalinejim

a1ex, which of the crop modes offer a real-time, colour preview?

a1ex

Only the simplest crop mode (1920x1080) has good real-time preview. The modes with higher vertical resolutions have color preview too, but it's not centered (only the top of the frame is shown).

In the regular crop_rec branch, there's also centered 5x zoom. Here I've removed it temporarily, as it increased the code complexity quite a bit, but I'll probably add it later when things will settle.

reddeercity

I read all this and thought Great Joke ! :P but then I read the Source Code Holy Cow  :o :o :o :o
There goes next few days  , I'll be living in front of my computer exploring this and hopefully I can compile for 5d2  :D 

ilia3101

reddeercity do you think that's possible yet? Wouldn't it require tons more reverse engineering?
Damn it I just happen to be away from proper computers that can compile ML for a week right now so I won't be able to try it if it is possible :'(

beauchampy

Build for 5d3 available? I just spent a week shooting 50p 3x: so would love to put it through its paces

Walter Schulz

Top of page -> Downloads
Download page -> Downloads (drop-down menu) -> Experiments

reddeercity

Quote from: Ilia3101 on April 02, 2017, 12:11:53 AM
reddeercity do you think that's possible yet? Wouldn't it require tons more reverse engineering?
Will not sure yet but I'm looking in to it .
If you read this post Full-resolution silent pictures there's info about 5D2 and lead me to believe it maybe possible  8)
I'm currently try to compile the code now , but I get some error's .
some tweak.o stuff , I working of older code , so I think that could be the problem so I'll update to the latest source code
and try again . I don't think 4k is a reality on 5d2 but maybe 3K as the bandwidth for 4K is around 150MB from what I read.
Since we are limited to max. 80MB/s  and by your Yet Another RAW Video Calculator   at 10bit 3x crop 2496x1134 @ 2:2.1 A.R. is 80MB/s
unless Lossless compression is used and that's another thing by it's self .

a1ex

Greg's original proof of concept (full-width LiveView) was done on 500D (same generation as 5D2 and 50D).

Sure, it probably needs a bunch of reverse engineering, but it probably boils down to changing some ADTG registers and video timers.

I've actually got these presets working in adtg_gui first. A proof of concept for 3K was committed back in 2016 (also mentioned in the "classic" crop_rec thread).

The registers interesting for changing resolution are documented either on the Register Map wiki page, or in adtg_gui source. Interesting commits: 8bbce9c, 7333869, 4a66a48, 6c62a6b, 38e78de.