crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

Started by a1ex, April 01, 2017, 11:15:41 AM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

Lars Steenhoff

that makes sense because the more noise of 6400 iso would result in less compression

Kharak

Happy Aprils fools day :)

How do I go about processing these mlv files with the mlv_dump included? I can't figure out how to spit dng's out
once you go raw you never go back


Launch

I salute the dedication but I wish squeezing every last pixel of out of the camera would be secondary to allowing audio with simple RAW. I would trade every last improvement for 1080p RAW with audio. For some reason, this worked for me once, but it just randomly stopped. It used to create a WAV file along with the RAW file.

squig

100 ISO - black screen ~49%
6400 ISO - black screen ~60%
6400 ISO - busy scene, fast movement ~69%

Levas

Ah that explains a lot.
High iso less compression, low iso high compression.

So for low iso shooting, compressed 14 bit is more efficiƫnt then shooting in 10 bit

Kharak

once you go raw you never go back

a1ex

Quote from: squig on April 03, 2017, 05:15:03 AM
I need to figure out what resolution will give the card enough headroom to write at that resolution continuously at least 90-95% of the time

Latest update estimates compression ratio on the fly, from the current scene and exposure settings. Also a few minor bug-fixes.

To get the best possible compression, all you need is to set ISO to maximum, shutter to 1/FPS, aperture wide-open and use a bright scene. You'll get close to 35% (apparently the limit for this codec, or maybe just for Canon's implementation).

Generally, if pixel values are predictable (from their neighbors), the image will compress well. That usually means low noise levels (low ISO, shadows, fully clipped highlights) and few details (large out-of-focus and/or low-contrast areas). Note that absolute noise values are higher in highlights, and the SNR is higher too, so exposing to the right will give higher file sizes (but that shouldn't stop you from using it).

Fast motion probably gives better compression, because of motion blur (didn't test, but that's my understanding with this codec).

Invalid image data (that includes e.g. 14-bit raw data interpreted as 10-bit, or pure noise, or machine code if you want) will give compressed output larger than initial size. In current implementation, this gives memory overflow. For some unknown reason (alignment issues? no idea), setting a maximum size for compressed output causes camera lock-up at certain resolutions, with no apparent logic (see this commit; minimal example here).

limey

For 1.23 only? I don't have a crop mode option in 1.13.

hyalinejim

Did you get the latest experimental build, and load the crop_rec module?

Quentin

I wonder if there is a potential for ordinary resolution recording but with higher frame rates.
Something like 100 FPS ?

etienne

One question to the developpers do you know if the 3 things below would be possible one day:

1/ 1080x1080  (from 3x3 bining using the 3240x3240 sensor size). for a big anamorphic s65 lens.

2 / in the same way of the 14 bit losless mjpeg compression is an 8 bit mjpeg compression possible? so we can do low file size and get the real sharpness the 5d3 can offer (clean HDMI recorder on prores does not get any better sharpness than the in camera h264 file - whereas the raw files is always resolving much more sharpness)

3/ finally how far would you say you are from a working preview for the mode 3K, UHD, or zoom x5 and is it even possible?

Also you probably have thought this but the basic full sensor preview with blacktape on the sceen could make it and if the 1:1 sensor readout cannot show original preview then maybe a x3 preview instead of the x5. Would not be perfect but would definitly make it usable.

Thanx for the work it's a dream. Where is the TIP JAR already?

Bests.

magnesiumnitrate

I started testing 14 bit lossless with 1080P 24 because thats what I use the most. File sizes are much smaller, thats nice. Something funky is happening though. Using the histogram overlay with 80% or more of my exposure on the right half I'm getting horribly, horribly underexposed results in davinci. The live view looks great, histogram looks great, but then it hits davinici and the footage looks awful. Not sure what to make of it, but thought you guys might want to know.

Filipe Samora

So after a few hours playing around with A1ex's second release of the 1.1.3 firmware, here's what I'm getting with 14bit lossless option:

3520 x 1320 = max. 1314 frames - ISO within 200-400 range - scales up to UHD with no perceived loss of quality

4096 x 1862 = max. 507 frames @ 12.5 fps - Not sure how useful this would be for landscape work or slow shots. Again ISO within 200-400 range.

Previews: At the moment, not fantastic but definitely better than a black screen. I was able to rack focus with a moderate degree of success.

Camera crashes persistently when using 50/60p modes.

I'm using Komputerbay cards 64GB 1066x

IQ, even shooting at home with crap light, is just beyond words.

Kharak

This is mighty fun to play with,

I was able to shoot up to 19 seconds at 3504x1312 at regular crop with Framed preview and Compressed Raw 14 bi, lots of movement, I was panning slowly downwards. But I only got so much recorded this one time, I never achieved the same length of recording again.

Is there a way of centering the crop? It looks weird not being centered.
once you go raw you never go back

hyalinejim

2 things I learned:

1. If you half press shutter button while recording the preview stays in colour rather than alternating between colour and greyscale. Frame rate still lags but it makes things a hell of a lot easier to know what's going on in front of the camera.

2. Filming the moire pattern of a computer screen's pixel array and refresh rate will absolutely kill the compression and stop the recording, even at lower resolutions!

Quote from: Kharak on April 04, 2017, 01:48:08 AM

Is there a way of centering the crop? It looks weird not being centered.

It should already be centered. Are you loading the crop_rec module of the latest experimental build and then selecting 3K or UHD? Framing is off in 5x but should be centred in these modes. Sometimes 3x is shifted up for me. I just restart the camera and it's fine.

bobbyshots2

Why doesnt the latest package have the mlvrec 2.0 module .. its only has light. for 5d3123.

bobbyshots2

Hi,
Ive been testing the latest build (April 03)for 5dIII123.
So far what i noticed there is no mlv snd module
or mlv_rec. Also there is no gray scale preview.

limey

Yeh I figured it out. I guess I was blind while looking at the modules or I wrote the files to the wrong sd card from my lexar workflow hub.

Getting pink frames and crashes in the 50fps modes.

Getting unreadable files in some other modes like the 3k and UHD. It was just an initial test, I will test more tomorrow. I am using 1066x 256gig lexar cards. However, they aren't formatted and have a lot of shots on them. I will format one and see what difference that makes.

hindra

Shooting with the Canon 5d3, 1.2.3 firmware and using the latest experimental build(April 3rd 2017) with Lexar 128gb 1066x cf card.

1080p 48FPS seems to be working (It seemed to have recorded fine, but I must have deleted the file.)
3520x1320 24FPS(ISO 100-200 iso seems to be continuous and at a 5x crop mode) ISO 400 recording stops after a few seconds. In no crop mode preview was colorful noise. in 5x crop preview worked great. I recorded 1510 frames and I ended it.
3584x1320 24FPS I recorded 1443 frames and I ended it.
1080p 60FPS(buggy, stops recording quickly) gray preview was glitchy and lagged.

3k (3072x1920@24fps) I got 66 frames.
UHD (3840x1536@24fps, couldnt get 3840x1600) I got 66 frames as well. It states its 120mb/sec, this might require the lossless compression with 10bit?
Both 3k and UHD frames are good, used MLV_DUMP from the page I downloaded the firmware to extract these.

I am also very tired, I will try and play around with them some more.
SL1 100D.100A - 5D - 7D2 - 5D3 1.2.3

budafilms


DeafEyeJedi

Ah, a little late to yet another party once again and after playing around in the past 48 hours or so I've finally come to a conclusion to be able to confirm that 3584x1320 in crop_rec (3K 1:1) can be recorded continuously if shot in between ISO 100-200 and more often than so in 400 depending on scene.

This was all tested with both 128gb 1066x & 256gb 1066x CF cards from KomputerBay on two separate 5D3's running 2017April03.5D3123 builds.



Seems that ISO 800 and higher doesn't get along well with 14-bit lossless compression which makes sense as previously stated by others in here.

Here's an example of what it looks like upon running this beautiful hack via LiveView from 5D3 below and watch what happens when I push it up to ISO 800 (and higher) towards the end:



I was able to convert all 14-bit lossless files with ease from @Danne's latest cr2hdr.app_4k (includes the required certain mlv_dump which was modified directly from @a1ex) so I am more than flabbergasted by the fact that all of this is storming down on us like there's no tomorrow.

Love it and will continue to play with and hopefully run into some more bugs! 

Also I can confirm that if I point the camera at a computer screen while recording and it'll crash once I focus upon the pixealated monitor (indeed this is quite strange but good find @hyalinejim) and I am actually loving the half shutter press for FULL COLOR LIVE DISPLAY or whatever that's called (plus it zooms in a bit more to be sure everything looks spot on) before recording which is very handy.

Thank you @a1ex for yet another petrifying update (in a good way!) and don't ever change who you are! Ha. 8)
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

beauchampy

Quote from: DeafEyeJedi on April 04, 2017, 09:12:19 AM

Here's an example of what it looks like upon running this beautiful hack via LiveView from 5D3 below and watch what happens when I push it up to ISO 800 (and higher) towards the end:

https://vimeo.com/211439899



I cant seem to get that lovely liveview on 113. Mine is low fps, desaturated and choppy.
Running crop_rec_4k.2017.Apr03.5D3113 and I copied your settings..

Levas

I was just wondering...

Who has twixtor or something alike and creates some stunning 6K - 24p video out of that 7 fps full resolution silent video mode  ;D 8)

DeafEyeJedi

Quote from: beauchampy on April 04, 2017, 09:36:31 AM
I cant seem to get that lovely liveview on 113. Mine is low fps, desaturated and choppy.
Running crop_rec_4k.2017.Apr03.5D3113 and I copied your settings..

Hmmm then perhaps I should downgrade to 113 and try to reproduce this?

Did you press 5x zoom after using ML menu or before? Sometimes it goes into a weird phase which may seem like having its own quirks but a simple restart (on/off) of the camera would help at least it does oh my end.

But it could very well be a bug for 113 and I'll let you know once I've downgraded.

Thought not too long ago @a1ex made a pull request for both firmwares (113 & 123) to be merged together after each update or does this only apply to the Nightlies and not experimental?

Well I'm actually tempted to check out the good old speedy 113... Just been using 123 as of late for HDMI output on gig's.

Actually this one trick may be worth to try (Thanks @dfort for the tips) perhaps reset your settings to default (from Canon menu) not the ML version and let me know if that helps either way.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109