Author Topic: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView  (Read 884322 times)

pc_bel

  • Member
  • ***
  • Posts: 132
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #75 on: April 04, 2017, 10:13:46 AM »
Hi,
I don't know if it will be useful or not...

Testing latest build (April 3th crop_rec) on 5D3 113, for now only this mode because is which I find more useful for me: 1920x872 50p 3x3 (aspect ratio 2.20:1) 14bit lossless.

Yesterday I was able to record without problems in continuous with normal preview, in color.

Today, with the same settings as yesterday, it has been impossible to boot the camera. In the SD card there are text files called ASERT, one for each boot attempt. In them the following appears:
ML ASSERT:0at mlv_lite.c:1935 (compress_task), task compress_tasklv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr03.5D3113
Mercurial changeset   : 7b0cb9532629 (crop_rec_4k) tip
Built on 2017-04-03 14:54:32 UTC by jenkins@nightly.
Free Memory  : 173K + 3807K


If I change SD card for another with 2017Feb11th 5D3113 (crop_rec proxy) all works again.

Hope it helps!!

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #76 on: April 04, 2017, 10:14:03 AM »
Are you guys running ML off an SD card?

The best I can do @ 100 ISO is 3.3K 2.39:1. ML is on the CF card.


pc_bel

  • Member
  • ***
  • Posts: 132
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #77 on: April 04, 2017, 10:28:34 AM »

Quote
Hi,
I don't know if it will be useful or not...

Testing latest build (April 3th crop_rec) on 5D3 113, for now only this mode because is which I find more useful for me: 1920x872 50p 3x3 (aspect ratio 2.20:1) 14bit lossless.

Yesterday I was able to record without problems in continuous with normal preview, in color.

Today, with the same settings as yesterday, it has been impossible to boot the camera. In the SD card there are text files called ASERT, one for each boot attempt. In them the following appears:
ML ASSERT:0at mlv_lite.c:1935 (compress_task), task compress_tasklv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr03.5D3113
Mercurial changeset   : 7b0cb9532629 (crop_rec_4k) tip
Built on 2017-04-03 14:54:32 UTC by jenkins@nightly.
Free Memory  : 173K + 3807K

If I change SD card for another with 2017Feb11th 5D3113 (crop_rec proxy) all works again.

Hope it helps!!


I've erased all in SD card and copied clean April3th Crop_rec_4K build.

All works ok again like yesterday.

pc_bel

  • Member
  • ***
  • Posts: 132
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #78 on: April 04, 2017, 11:01:33 AM »
Same again:

ML ASSERT:0at mlv_lite.c:1935 (compress_task), task compress_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr03.5D3113
Mercurial changeset   : 7b0cb9532629 (crop_rec_4k) tip
Built on 2017-04-03 14:54:32 UTC by jenkins@nightly.
Free Memory  : 177K + 3923K

Need to pull battery, and then camera doesn't boot anymore :(

hyalinejim

  • Member
  • ***
  • Posts: 131
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #79 on: April 04, 2017, 11:20:55 AM »
Are you guys running ML off an SD card?

The best I can do @ 100 ISO is 3.3K 2.39:1. ML is on the CF card.



Yes. I was under the impression this frees up more bandwidth for recording to CF. Also, when you change CF cards your settings don't change.

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #80 on: April 04, 2017, 12:35:16 PM »
OK cool, I'll give it a shot.

Seeing as we're all madly trying to squeeze every last MB out of the oldfella to get closer to 4K, would it be possible to add card spanning to these experimental builds?

garry23

  • Contributor
  • Hero Member
  • *****
  • Posts: 2218
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #81 on: April 04, 2017, 01:47:28 PM »
I just downloaded the latest experimental build, but didn't get far.

I would welcome a little help, as I'm keen to test, but, I don't do video. That is I'm not up to speed on MLV etc.

If there is someone out there who has the time, maybe they would create a 1 page 'here is how to test', i.e. The steps.

In particular, how do I set up, run and process the full Res 'photo mode'. Once I know how to get things running, I can test.

Cheers

Garry

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #82 on: April 04, 2017, 05:52:37 PM »
So what is the bottle neck for transfer speeds in the 5D3? Is it still the card reader in the camera? I thought cameras card reader was limited even with really fast cards?
Is spanning the trick for UHD?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #83 on: April 04, 2017, 05:55:45 PM »
ML ASSERT:0at mlv_lite.c:1935 (compress_task)

This one indicates frame corruption, and should also print this (unless obscured by other messages):
Code: [Select]
                printf("\nCompressed size higher than uncompressed - corrupted frame?\n");
                printf("Please reboot, then decrease vertical resolution in crop_rec menu.\n\n");

1080p 60FPS(buggy, stops recording quickly) gray preview was glitchy and lagged.

Took a quick look - after the latest fixes (see below), the following resolutions are continuous (1.2.3, lossless ~55%, Preview: Frozen LV, with grayscale preview):

- 1920x800 60fps
- 1920x960 50fps
- 1920x1080 45fps

1920x1080 48 FPS is a bit unreliable (corrupted frames with Global Draw on, but appears OK with it off). This mode is really pushed to the limit (and that's the reason I've also added a 45-fps option). Feel free to play with the knobs from crop_rec submenu, if you think you can find the sweet spot without corrupted frames.

I would expect the 1.1.3 firmware to have a little more headroom (maybe even allow a few more pixels), but didn't test it. You can fine-tune vertical resolution from the crop_rec submenu (and if you have corrupted frames, decrease it from there).

New build posted. Short summary:

- fixes for high-FPS modes with Frozen LV (first frame was corrupted, which didn't play nice with lossless compression)
- continuous full-res recording with lossless compression (about 4-5 FPS, depending on ISO and scene complexity)
- print current FPS and rolling shutter in the FPS override menu (visible from main Movie menu tab)
- two experiments with a small risk:
   - single-buffering (risk of tearing, alongside with graphical checks)
   - use-after-free for SRM memory (to fix half-shutter quirks)
- minor fixes here and there.

GutterPump

  • Member
  • ***
  • Posts: 185
  • 5DIII/5DII/550D
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #84 on: April 04, 2017, 06:11:34 PM »
Thanks for the update A1ex, today it's my birthday, it's a nice gift from you :)

I believe this has already been said, but the best way to use the lossless 14bits is to forget to shoot in ETTR.
Do you think there is way to take the control on the compression codec ?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #85 on: April 04, 2017, 06:25:18 PM »
In particular, how do I set up, run and process the full Res 'photo mode'. Once I know how to get things running, I can test.

Simplest way:
- crop_rec.mo, silent.mo
- crop mode: full-res LV; silent mode: simple, any file format
- press half-shutter
- for preview, also load mlv_lite.mo mlv_rec.mo, enable raw recording and global draw.
- you can review 14-bit regular/lossless DNG (pic_view) or MLV (mlv_play); for 10-bit full-res MLV, raw_twk runs out of RAM

Continuous full-res recording at 4-5 FPS:
- crop_rec.mo, mlv_lite.mo
- crop mode: full-res LV; raw recording: max resolution, lossless compression
- FPS override: 4 or 5 FPS, low light (important)
- press record
- mlv_dump foobar.mlv --dng

Fast half-shutter trigger (with negative shutter lag):
- crop_rec.mo, mlv_lite.mo, configured as above
- raw recording: enable REC trigger (half-shutter: pre only; without pre-recording enabled, that means only 1 frame)
- no need to use FPS override, as we are only saving the frames triggered by half-shutter
- start recording
- press half-shutter every now and then to capture a frame
- stop recording

Capture unpredictable action (animals, kids etc):
- crop_rec.mo, mlv_lite.mo
- crop mode: full-res LV or 4K half-FPS
- raw recording: max res, 10-bit (here, pre-recording is a bit confused by variable frame sizes, so let's not be adventurous)
- pre-recording: 1 or 2 seconds (actually "up to")
- half-shutter trigger: "hold" (which will pre-buffer up to about half of available RAM), or "pre only" (which will pre-buffer all the available RAM)
- start recording, press half-shutter etc
- caveat: preview is not working at all with 10-bit (try 14-bit)
- to review in camera: 4K 10-bit can be played back with mlv_play + raw_twk, full-res 14-bit with mlv_play only.

I believe this has already been said, but the best way to use the lossless 14bits is to forget to shoot in ETTR.
Do you think there is way to take the control on the compression codec ?

Very little.

I wonder if there is a potential for ordinary resolution recording but with higher frame rates.

First post:
Quote
* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning)
* 1920x800 @ 60p (same as above)
* 1920x1080 @ 45p and 48p (3x3 binning)

In particular, 1080p48 is quite pushed to the limit, which is why I've also included a 45p preset (hopefully a bit more reliable).

1/ 1080x1080  (from 3x3 bining using the 3240x3240 sensor size). for a big anamorphic s65 lens.

Already possible on regular nightly.

Quote
2 / in the same way of the 14 bit losless mjpeg compression is an 8 bit mjpeg compression possible?

Yes...

Quote
so we can do low file size and get the real sharpness the 5d3 can offer (clean HDMI recorder on prores does not get any better sharpness than the in camera h264 file - whereas the raw files is always resolving much more sharpness)

... but it would simply compress Canon's YUV422 buffer (with their color science, picture styles, sharpness etc).

Quote
3/ finally how far would you say you are from a working preview for the mode 3K, UHD, or zoom x5 and is it even possible?

Far, far away. The trick I've used to center the 5x zoom preview in the regular (non-4K) crop_rec branch didn't work for the new modes :(

Also, I have no idea how to configure image scaling. Adtg_gui doesn't reveal any difference between x5 and x10 zoom modes (both use identical raw buffer configurations - just the preview differs).

I can try to include the centered x5 zoom mode (which you can already use in the "classic" crop_rec builds), if that helps.

Teamsleepkid

  • Member
  • ***
  • Posts: 248
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #86 on: April 04, 2017, 06:44:01 PM »
Anyone trying to turn the eos m into the blackmagic cinema camera? 2.5k possible? Maybe... hopefully...
EOS M

togg

  • Senior
  • ****
  • Posts: 444
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #87 on: April 04, 2017, 06:51:03 PM »
This is amazing for movie still photographers!

garry23

  • Contributor
  • Hero Member
  • *****
  • Posts: 2218
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #88 on: April 04, 2017, 07:25:36 PM »
@a1ex

Many thanks for the simple insight, for a simple still photographer  ;) That is:

Quote
Simplest way:
- crop_rec.mo, silent.mo
- crop mode: full-res LV; silent mode: simple, any file format
- press half-shutter
- for preview, also load mlv_lite.mo, enable raw recording and global draw.
- caveats:
  - preview doesn't work

Because of the preview limitation, I simply composed with crop_rec and silent switched off, used ETTR, at 0%, and switched them back on after the ETTR.

Grabbed a DNG which I inspected in LR.

Fantastic!!!

OK, I know you know the limitations at the moment, eg no EXIF data, but what I step forward for silent full res photography, ie in those churches I go to etc.

One thing I have noticed is that the ETTR exposure at 0% highlights (SNRs off/0) does show over exposure. Not sure if this is real or important at this stage of the developments.

As I said, I'm keen to help test the still photography side of the experiment: I'll stay clear of the MLV world...for now.

Cheers and, once again, thanks for this new step forward for ML.

Garry

lostfeliz

  • Freshman
  • **
  • Posts: 54
  • 5diii. 60d.
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #89 on: April 04, 2017, 08:57:20 PM »
I'm having trouble using mlv_dump to decode the compression. Any help appreciated. (sorry, I know this is a development forum. what I am I doing here if I can't use Terminal properly?) :)

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3413
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #90 on: April 04, 2017, 10:13:28 PM »
Assuming you're on OS X -- Have you even tried @Danne's latest cr2hdr.app which includes the modified mlv_dump yet @lostfeliz?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

etienne

  • New to the forum
  • *
  • Posts: 14
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #91 on: April 04, 2017, 10:36:40 PM »
@a1ex

Hello thanx for taking the time to answering the question this is highly appreciated. i'm on the curve to go to URSA 4.6K but this new fonctions let me think my olds 5d3's have still some life to live.

regarding the fact that mlv mjpeg 8 bit compression
Quote
would simply compress Canon's YUV422 buffer (with their color science, picture styles, sharpness etc)


Are you absolutely sure regarding the sharpness. I'm asking because this is a fact that the clean hdmi as absolutely zero additionnal sharpness from the h264 files i have tested this thousands times. But the raw definitly resolves way more sharpness similar to what produces an A7sii, gh4 or gh5 in HD so i was sure that the 5d breaks something in between what you catch with rec_mod and what you get as h264 or hdmi out.

Now what i'm undestanding from your saying is that the compress Canon's YUV422 buffer is what is sent to the hdmi out right? If it's the case it's useless for sure but can it be something else. Can you think of any other way to compress to something like 15~25% even if it means we loose all the color depth from the raw but just keep that sharpness from it. I'm insisting on this because Personnally I would be so much be happy with an MLV without any color depth additional value but which would just have the real sharpness we can get from the current raw and that could allow like 60~90 minutes on a 128GB card.

Also I want to congrats on the slowmotion mods all options are awesome i just did a 1080 60P in 2:35 flawlessly. As mentionned the 1080 48P as a few corrupted frames and its preview is also a little off towards the down of the image compared to the canon preview but appart from that we could say this is 99% usable.

Finnally. The 3K Mode and UHD are absolutely awesome. I personnally don't see how we could use over 3360x1528 with spanning and sound in the future so any preview that would be x2 would just be the ultimate thing to make this usable.

I undestand that for now it's complete unknow hope someone will find a solution one day otherwise in regards of you're sayinf:
Quote
I can try to include the centered x5 zoom mode (which you can already use in the "classic" crop_rec builds), if that helps.
Sure It's definitly better than nothing to have access to canon x5 center preview again instead of pink frame eventhough it will not help with framing but it will help to pull focus correctly durin shooting.

Bravo again keep the good work. Looking very forward to integration to MLV2 with spanning and sound enabled...

Good Luck.


 

hyalinejim

  • Member
  • ***
  • Posts: 131
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #92 on: April 04, 2017, 10:57:37 PM »
Hi a1ex, thanks for the new build.

I noticed that the preview framing in UHD seems to be too far up. I tried fiddling with the crop options, to no avail.

Also, for anybody concerned about low light shooting, it looks like I can get continuous 16:9 with around 2.4k horizontal resolution at ISO 3200.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #93 on: April 04, 2017, 11:24:03 PM »
Can you attach a screenshot + DNG?

(I did the same, but couldn't notice any mismatch...)


photofreak126

  • New to the forum
  • *
  • Posts: 5
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #94 on: April 04, 2017, 11:46:15 PM »
Hi! What great stuff is happening here!

One question: I try to avoid raw because of inconvenience.

The 1:1 1920 and 1:1T work awesomely fine for me with crop_rec enabled and raw disabled. It would be so good to have 1920 60p (1920x800) in non raw mode but if I choose either 720 60p or any other Canon frame rate/resolution combo, I just get the standard rates.... Is there a trick to it or is the crop_rec just for ml_raw ??

hyalinejim

  • Member
  • ***
  • Posts: 131
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #95 on: April 04, 2017, 11:54:01 PM »
So this is full frame mode, just to show the framing:


And this is the DNG from UHD mode:

https://www.dropbox.com/s/xoxmrxn0hcj19qo/M04-2243_000000.dng?dl=0

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #96 on: April 05, 2017, 12:08:23 AM »
You mean, the recorded area is not centered? That's probably true, as it can only be adjusted in rough increments. But the preview should be OK.

You can fine-tune the centering (in both directions) from the crop_rec submenu. Start values for vertical centering:
Code: [Select]
            case CROP_PRESET_UHD:
                cmos_new[1] =
                    (video_mode_fps == 24) ? PACK12(4,9)  :
                    (video_mode_fps == 25) ? PACK12(4,9)  :
                    (video_mode_fps == 30) ? PACK12(5,8)  :
                    (video_mode_fps == 50) ? PACK12(12,9) :
                    (video_mode_fps == 60) ? PACK12(13,9) :

Then, you can check with raw_diag (Downloads -> Modules) using the OB zones tool, to see whether the optical black areas are clean and useful data is present in all edges.

If you (or anyone else) finds better values, I'll include them.

goldenchild9to5

  • Hero Member
  • *****
  • Posts: 534
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #97 on: April 05, 2017, 01:00:40 AM »
WOW!!!!!  :o just saw this new development are you serious 4K on the old 5D Mark III.  Like I've said before Magic Lantern Team never sleeps, thank you @a1ex and the whole Magic Lantern Team for those awesome new features, and previously impossible features  :D

hyalinejim

  • Member
  • ***
  • Posts: 131
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #98 on: April 05, 2017, 01:14:35 AM »
Actually, the preview is off as well as the file. In 3K mode framing is correct in preview:



But in 4K it's totally off (in the last build it was fine)



I tried altering CMOS[1] lo hi values. At lo 10 the image was centered vertically but with a weird overlay on half of the screen:



It's a transparent overlay of what's actually beneath the frame. I tried playing with different values for CMOS[1] lo but with no luck. In the previous build this problem cropped up (excuse the pun!) from time to time, but I could restart the camera and it was back to normal. With this build UHD is vertically shifted whereas 3K is correct.

TANGENTIAL SUGGESTION: If there's some way to reliably control the positioning of the crop area, this is a great way to turn a wide angle lens into a digital shift lens, for those interested in architectural stuff and precise composition.

Another quick suggestion: It would be nice to have more control of the vertical resolution, if that's possible. Let's say I wanted to shoot in 4:3 using a 2x anamorphic. The max res I can get is 2048 x 1536, even though my card could handle a higher resolution.

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #99 on: April 05, 2017, 01:50:34 AM »
I got that overlay when I messed with the 1920 tall positioning on the 1st or 2nd build.

I shot standard 1080p raw vs 2.8K raw at the same ISO and aperture, taking a few steps to get close to the same framing; the 1:1 2.8K crop appears to be about half a stop brighter than the 3x3 binned 1080p. Somebody else might wanna test that to confirm.