Author Topic: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView  (Read 60911 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 9727
  • 5D Mark Free
As you probably have guessed from the latest developments (QEMU, EDMAC graphs, JPCORE, EEKO), our understanding on how LiveView works has improved considerably. Finally, all my fiddling with QEMU, at first sight with little or no purpose for the everyday users, starts paying off.

Today, Magic Lantern proudly announces new ground-breaking features that were previously thought impossible or very hard to achieve.

We proudly present....

4K RAW Video Recording!



DOWNLOAD

Twitter announcement

On the 5D Mark III, you now have the following new resolutions:

* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning) - continuous*)
* 1920x800 @ 60p (same as above)  - continuous*)
* 1920x1080 @ 45p and 48p (3x3 binning)  - continuous at 45p
* 1920x1920 @ 24p (1:1 square crop) - continuous*)
* 3072x1920 @ 24p (1:1 crop)
* 3840x1536 @ 24p (1:1 crop) (corrupted frames at 1600)
* 4096x2560 @ 12.5p (1:1 crop) - continuous*) at 8 FPS
* 4096x1440 @ 25p (1:1 crop)
* Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter) - continuous*) at 5 FPS!

The last feature complements the well-known full-resolution silent pictures - the new implementation will be usable at fast shutter speeds, without the exposure gradient - but with rolling shutter (just like regular LiveView frames).

*) Continuous recording is achieved in 14-bit lossless compression mode (LJ92), preview set to Frozen LV (previously known as Hacked Preview), low ISOs (about 55% ratio). With complex (highly detailed) scenes and/or higher ISO, the image doesn't compress very well, so recording may no longer be continuous.

Credits: Greg (full-width LiveView), g3gg0 (video timer, DIGIC registers documentation and lots of other low-level insights).

Complete list of new video modes:
Code: [Select]
                                /*   24p   25p   30p   50p   60p */
    [CROP_PRESET_3X_TALL]       = { 1920, 1728, 1536,  960,  800 }, /* 1920 */
    [CROP_PRESET_3x3_1X]        = { 1290, 1290, 1290,  960,  800 }, /* 1920 */
    [CROP_PRESET_3x3_1X_48p]    = { 1290, 1290, 1290, 1080, 1080 }, /* 1920; 1080p45/48 <- 50/60p in menu */
    [CROP_PRESET_3K]            = { 1920, 1728, 1504,  760,  680 }, /* 3072 */
    [CROP_PRESET_UHD]           = { 1536, 1472, 1120,  640,  540 }, /* 3840 */
    [CROP_PRESET_4K_HFPS]       = { 2560, 2560, 2500, 1440, 1200 }, /* 4096 half-FPS */
    [CROP_PRESET_FULLRES_LV]    = { 3870, 3870, 3870, 3870, 3870 }, /* 5796 */

What else could you wish for?


Screenshots:





FAQ

Where's the catch?

This is only a very rough proof of concept. It has not been battle-tested and has many quirks. Some of them may be easy to fix, others not so. In particular:

* It feels quite buggy. I'm still hunting the issues one by one, but it's hard, as Canon's LiveView implementation is very complex, and our understanding on how it works is still very limited.
* Write speeds are high. For example, 10-bit 4096x2500 at 15 fps requires 180 MB/s. 1080p45 should be a little more manageable at 111 MB/s.
* Canon preview is broken in most modes; you need to use the grayscale preview in the raw recording module.
* High-resolution modes (in particular, full-res LiveView) may cause trouble with memory management. This is very tricky to solve, as we only get 3 full-resolution buffers in LiveView, with restrictions on the order in which they must be freed, and lots of other quirks.
* Since these settings were pushed to limit, the risk of corrupted frames is high. If it happens, decrease the vertical resolution a bit (from the crop_rec submenu).
* When refreshing LiveView settings, the camera might lock-up (no idea why). Pressing MENU twice appears to fix it.

May I fine-tune the new modes?

Yes! I've included some of the knobs on the user interface. Normally you shouldn't need to touch these buttons, but if you do, you might be able to squeeze a few more pixels.

Does it work with FPS override?

Sort of. It's not reliable at this point, so it's best not to try yet.

Overheating?

During my tests, I didn't manage to get a sensor temperature higher than 60 degrees. Your mileage may vary.


Risks?

This mod changes some low-level sensor parameters that are not well understood. They were all figured by trial and error, and there are no guarantees about the safety of these changes.

As usual, if it breaks, it's your fault, sorry.

Will it work on other camera models?

I hope so; however, this is an area where I hope to get contributions from others (yes, from you). If these new features don't motivate you to look into it, I wonder what else will.

I'll explain how all this works in the coming days or weeks.

Is it difficult to port to other camera models?

So far, the 3x3 720p mode from crop_rec was ported to EOS M (rbrune) and 700D (dfort). So it shouldn't be that hard...

Will you port this to my camera model, please?

No, sorry. I have better things to do - such as, preparing the April 1st prank for next year :)

Wait a minute, didn't you say you are primarily a still photo user? Why are you even doing this?

If you look close, the usefulness for video is fairly limited, as the write speeds (and therefore the recording times) are not practical.

But the full-resolution LiveView is - in my opinion - very useful for still photo users. Although the current implementation is not very polished (it's just a proof of concept), I hope you'll like the idea of a 7.4 FPS burst mode, 100% silent, without shutter actuations.

Right now, you can take the mlv_lite module with pre-recording and half-shutter trigger: at 10 bits per pixel, you get 5 frames pre-recorded, and saved to card as soon as you touch the half-shutter button. Or, you can capture one frame for each half-shutter press, with negative shutter lag! (since the captured frame will always be pre-recorded).

And if a burst at 7.4 fps is not enough, you may also look at the 4K modes (12-15 fps).

(I know, I know, GH4 already does this, at much higher frame rates...)

The help menu for full-res LiveView says 5796x3870, but MLV Lite only records 5784x56. What's going on?

The raw recording modules have a couple of alignment constraints (e.g. can only start cropping from a multiple of 8 pixels, and the size of the cropped area (that goes into the MLV file) must be multiple of 16 bytes (that is, W*bpp/8 + H mod 16 must be 0).

To capture the full resolution, you may use the silent picture module. However, this module is not the best when it comes to memory management and buffering. Currently, you'll get an impressive buffer of 2 frames in burst mode :)

But hey - it outputs lossless DNG!

What about that lossless compression routine?

It's included, although I didn't manage to test it much. There is a lot of room for improvement, but for a proof of concept, it seems to work.



P.S. The initial announcement was disguised as an April Fools joke, just like the original crop_rec.

Twitter announcement

From original April Fools post:
Quote

With our latest achievements in wizardry with ARM programming and DIGIC reverse engineering, we can speak of a new era of raw video recording.

On models like the 5D Mark III, the next upcoming releases will feature an improved version of our crop_rec module that delivers the following new resolutions:
 
* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning)
* 1920x800 @ 60p (same as above)
* 1920x1080 @ 45p and 48p (3x3 binning)
* 1920x1920 @ 24p (1:1 square crop)
* 3072x1920 @ 24p (1:1 crop)
* 3840x1600 @ 24p (1:1 crop)
* 4096x2560 @ 12.5p (1:1 crop)
* Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter).

The last feature complements the well-known full-resolution silent pictures - the new implementation will be usable at fast shutter speeds, without the exposure gradient - but with rolling shutter (just like regular LiveView frames).

Please understand that providing the source code for those highly DIGIC optimized routines is a bit troublesome and will need some extra legal care. After this step is taken and as soon we are finished with ensuring the product quality you are used from Magic Lantern, we will upload the code to our repository.

Consider this being a huge leap towards our next mind boggling goal:

8K RAW Video Recording!



Sample DNG from 5D Mark III, to show that our proof of concept is working:

8k.dng

Stay tuned for more information!


Walter Schulz

  • Hero Member
  • *****
  • Posts: 5380
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #1 on: April 01, 2017, 11:22:04 AM »
Who is ML's officiail bookie? I want to risk some Euro ...

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 935
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #2 on: April 01, 2017, 12:12:57 PM »
Shut up and take my money!
70D.112 & 100D.101


Wayne H

  • New to the forum
  • *
  • Posts: 5
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #4 on: April 01, 2017, 12:13:52 PM »
INCREDIBLE!!!!! Wonderful job alex, i really appreciate all the hard work you and and the other developers have put in over the years to get magic lantern to where it's at right now.

ULTRA HD RAW 24p on 5d mark iii wooah :) ,so the most obvious question is when will we be able to test out this new update.   

Feels like xmas. :)

Alex you are a genius.

Danne

  • Hero Member
  • *****
  • Posts: 2788
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #5 on: April 01, 2017, 12:24:51 PM »
Incredible. It,s art.

jagnje

  • New to the forum
  • *
  • Posts: 40
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #6 on: April 01, 2017, 12:32:31 PM »
april 1st

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 935
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #7 on: April 01, 2017, 12:40:17 PM »
Nope, I would rather say it's the whole truth as long as no one talks about expects continuous recording:

http://www.magiclantern.fm/forum/index.php?topic=18300.0

 8) :P 8) :P :D :D ;D 8) 8)
70D.112 & 100D.101

Ilia3101

  • Freshman
  • **
  • Posts: 79
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #8 on: April 01, 2017, 01:18:10 PM »
KooooooooL  8) any chance of anything for the mark 2 or other cameras?
5D2

beauchampy

  • Senior
  • ****
  • Posts: 270
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #9 on: April 01, 2017, 01:29:47 PM »
Please guys, I can't take the heart ache. Is this an April fools? Specifically the 960px 3x3?

Levas

  • Hero Member
  • *****
  • Posts: 853
  • 6d - Nightly build user
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #10 on: April 01, 2017, 01:50:51 PM »
I love how you get a 8K DNG sample from a 5d mark III, there's some real magic going on there  ;D

Walter Schulz

  • Hero Member
  • *****
  • Posts: 5380
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #11 on: April 01, 2017, 02:02:02 PM »
Is this an April fools?

You don't expect an answer without tricks, traps, false mirrors and double meaning, do you? No today, not today ...

Greg

  • Hero Member
  • *****
  • Posts: 544
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #12 on: April 01, 2017, 02:06:12 PM »
WOW!  :o
8K is a bit noisy.

Frank7D

  • Senior
  • ****
  • Posts: 444
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #13 on: April 01, 2017, 08:17:34 PM »
Thank you so much for everything you've done! Now, not to be demanding, but 16K is obviously doable so when can we expect it?

nikki

  • Freshman
  • **
  • Posts: 95
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #14 on: April 01, 2017, 08:23:19 PM »
 ..it works perfect

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 9727
  • 5D Mark Free
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #15 on: April 01, 2017, 11:31:34 PM »
Committed some of the sources; still playing with it, but those who can compile from source can already give it a try.

hyalinejim

  • Freshman
  • **
  • Posts: 79
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #16 on: April 01, 2017, 11:50:36 PM »
a1ex, which of the crop modes offer a real-time, colour preview?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 9727
  • 5D Mark Free
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #17 on: April 01, 2017, 11:56:32 PM »
Only the simplest crop mode (1920x1080) has good real-time preview. The modes with higher vertical resolutions have color preview too, but it's not centered (only the top of the frame is shown).

In the regular crop_rec branch, there's also centered 5x zoom. Here I've removed it temporarily, as it increased the code complexity quite a bit, but I'll probably add it later when things will settle.

reddeercity

  • Hero Member
  • *****
  • Posts: 1276
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #18 on: April 02, 2017, 12:06:54 AM »
I read all this and thought Great Joke ! :P but then I read the Source Code Holy Cow  :o :o :o :o
There goes next few days  , I'll be living in front of my computer exploring this and hopefully I can compile for 5d2  :D 

Ilia3101

  • Freshman
  • **
  • Posts: 79
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #19 on: April 02, 2017, 12:11:53 AM »
reddeercity do you think that's possible yet? Wouldn't it require tons more reverse engineering?
Damn it I just happen to be away from proper computers that can compile ML for a week right now so I won't be able to try it if it is possible :'(
5D2

beauchampy

  • Senior
  • ****
  • Posts: 270
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #20 on: April 02, 2017, 01:00:53 AM »
Build for 5d3 available? I just spent a week shooting 50p 3x: so would love to put it through its paces

Walter Schulz

  • Hero Member
  • *****
  • Posts: 5380
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #21 on: April 02, 2017, 01:04:05 AM »
Top of page -> Downloads
Download page -> Downloads (drop-down menu) -> Experiments

reddeercity

  • Hero Member
  • *****
  • Posts: 1276
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #22 on: April 02, 2017, 01:19:01 AM »
reddeercity do you think that's possible yet? Wouldn't it require tons more reverse engineering?
Will not sure yet but I'm looking in to it .
If you read this post Full-resolution silent pictures there's info about 5D2 and lead me to believe it maybe possible  8)
I'm currently try to compile the code now , but I get some error's .
some tweak.o stuff , I working of older code , so I think that could be the problem so I'll update to the latest source code
and try again . I don't think 4k is a reality on 5d2 but maybe 3K as the bandwidth for 4K is around 150MB from what I read.
Since we are limited to max. 80MB/s  and by your Yet Another RAW Video Calculator   at 10bit 3x crop 2496x1134 @ 2:2.1 A.R. is 80MB/s
unless Lossless compression is used and that's another thing by it's self .

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 9727
  • 5D Mark Free
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #23 on: April 02, 2017, 01:38:02 AM »
Greg's original proof of concept (full-width LiveView) was done on 500D (same generation as 5D2 and 50D).

Sure, it probably needs a bunch of reverse engineering, but it probably boils down to changing some ADTG registers and video timers.

I've actually got these presets working in adtg_gui first. A proof of concept for 3K was committed back in 2016 (also mentioned in the "classic" crop_rec thread).

The registers interesting for changing resolution are documented either on the Register Map wiki page, or in adtg_gui source. Interesting commits: 8bbce9c, 7333869, 4a66a48, 6c62a6b, 38e78de.

reddeercity

  • Hero Member
  • *****
  • Posts: 1276
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #24 on: April 02, 2017, 02:21:58 AM »
Thank you  :)