crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

Started by a1ex, April 01, 2017, 11:15:41 AM

Previous topic - Next topic

0 Members and 3 Guests are viewing this topic.

reddeercity

Quote from: Danne on June 12, 2017, 04:23:53 AM
One workaround previewing any MLV footage is going through MLVFS. Once the virtual dng files are created MlRawViever will be able to read the dng sequence. Just right click a dng file and select MlRawViewer.
Yea I know  , but way to slow especially with UHD/4K it even lags in full HD with mlvfs .
On PC's best option for compressed raw is to convert to .raw legacy and use the existing PC apps .
There's more reasons to use MLVViewSharp then just for viewing the files .
MLVProducer developer was thinking of supporting compressed raw in his app , until then work arounds 

pc_bel


GutterPump

Quote from: reddeercity on June 12, 2017, 05:19:58 AM
Just use MLVBrowersSharp , right click on DNG+wave then it open to a window to save , (see next image)

1
by RedDeerCityTV, on Flickr

Select which drive or folder you want.

2
by RedDeerCityTV, on Flickr



Thanks for your answer, but as you can see in my picture :



all my DNG sequence are mixed in a same folder; i see you are using windows 7, maybe this issue is only about windows 10.

bouncyball

Quote from: reddeercity on June 12, 2017, 05:31:01 AM
Yea I know  , but way to slow especially with UHD/4K it even lags in full HD with mlvfs .
On PC's best option for compressed raw is to convert to .raw legacy and use the existing PC apps .
There's more reasons to use MLVViewSharp then just for viewing the files .
MLVProducer developer was thinking of supporting compressed raw in his app , until then work arounds 
On a decent hardware there is no lag on PC (w/o antivirus installed) and there is no need to use legacy RAW, you can just uncompress (with appropriate mlv_dump) to MLV again and use whatever tool you desire (mlrawviewer etc)

Danne

Eh, I totally forgot about decompression of MLV files although I made use of it in cr2hdr.app  8).
Thanks for the reminder.

sylvain_c

Sorry to annoy you,

I don't know what I'have done with the latest build (pretty sure a bad choice in bit depth in the hurry) but half of my converted mlv's are just blacks dng (latest mlv dump).
I'm asking just to be sure, but I think this shots are lost?

A sample of the problem:
https://mega.nz/#!vxkkGQJT!8GjEWu1dzUq3yVL7siRQI_u7TuPnFIm-vDmFCELu594
5D MKIII 1.1.3

Danne

Black or just darker? You´re uploaded file has a dark preview image but opens just fine in acr or when converted to a tiff with dcraw. I get those darker files as well. Not sure why but it seems related to 10/12bit and only preview.

sylvain_c

Really, it open fine in acr??I was pretty upset by the black preview I come here too quickly...
(You're right, all open fine in acr, it's my viewer, picasa viewer the culprit)!
Thanks for the quick reply Danne!
5D MKIII 1.1.3

Quentin

@g3gg0

Yesterday I did some shooting with latest build on 5dmk3
1920x800 @ 60FPS

https://drive.google.com/open?id=0B-d8ARtc7xwWQU9kVkktTW9NMEk

I cannot decompress some MLVs, here is a sample.

Use your wizardry
Thanks in advance

Walter Schulz


Danne

Do this.
mlv_dump --dng --relaxed
Fixes the issue on mac.

If you use cr2hdr.app you find the relaxed switch under (14) autopsy settings

g3gg0

as i am on low bandwidth, what is the problem with the file?

which block fails?

can you extract the troublesome block using:

mlv_dump m14-0847.mlv --extract <blocknumber> --autopsy-file bad.bin

and then upload the bad.bin
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Danne


g3gg0

thanks!

FYI
when fetching the latest mlv_dump and some recent mlv_lite / mlv_rec, you can use this to determine the exact build version:


./mlv_dump <mlv_file>  --extract-type VERS --payload-only --ascii

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Danne


Quentin

@Walter
@g3gg0

I get -1 error.
I use the mlv_dump that is in the Experiments page.
:(

Walter Schulz

mlv_dump --dng --relaxed
processes 381 out of 400 frames.

Quentin

Quote from: Walter Schulz on June 15, 2017, 09:34:53 PM
mlv_dump --dng --relaxed
processes 381 out of 400 frames.

Yes, I was about to report that.
The relaxed switch does the trick.
Gratitude

Kharak

@a1ex / @g3gg0

Could you look in to the mlv_dump from experimental. When I convert 60P 3x3 Footage I still have a lot of Vertical Stripes in highlights, even though mlv_dump is doing Vertical Stripe Fix, I am wondering if the code is not fitting for the 60p 3x3 mode or maybe there is an offset which enhances vertical stripes?, Maybe because of the 4 dead vertical rows on the left of the frame (also a bug btw). Especially if I have lens flare or direct sunlight flaring in the lens, then the Vertical Stripes are more visible than anything I've recorded with normal 1920x1080 MLV 3x3. What could be the cause of it?

When I play the footage in mlvrawviewer and turn on Vertical stripe fix in that program, sometimes that will get completely rid of the stripes and other times it will make it worse. Its really weird to work with.

The heavier the grade, the more visible the stripes. To make MLV footage look really good, you have to push footage so much that it crosses that threshold where the stripes become really visible. Temporal Noise reduction will also at times enhance it even more, as the algorithms "think" its important information to keep.

Here is a cropped screenshot from mlvrawviewer with a LUT pushing the image from a log curve, to show you the severity of the stripes.

https://mega.nz/#!8YIHGIJB!hzewH2JBX55vELsy2MPdp3gPSo2mTzlNKe_CQIVqu_o

As you can see the, flaring enhances the vertical stripes a lot and in this shot its only part of the image, sometimes the entire image will be engulfed in flare and everything turns to Stripes.. I could not find my shot where I had a full flare, aber its egal, same thing.


Also, it would be really nice if you could add Allow Global Draw - OFF during recording to MLV lite as in the vanilla MLV. Because when I am pressing half shutter to remove overlays, I am getting vastly improved recording times, almost double the time and sometimes continues (depending on scene).

and another bug to report from 60p 3x3, when selecting ISO 100 the Raw Zebras become disabled and the Histogram maxes out at about 90% the way to the right, the channels don't go beyond it, leading the operator to believe that one is well within ETTR range. The non-raw spotmeter also maxes out at 96% (not sure if also at other iso's).

Also a general question, why is it when hitting record button, that the recording starts so "sluggish", like 1-2 secs after, does it need to write information to a buffer or something, is it not something that could be done before hitting record, I mean just by entering the mode and keeping it Armed for recording? Also when recording stops, it takes about 4-6 seconds before I can hit record again, maybe longer.. It is quite counter productive, when one wants to shoot a lot of bursts at times and having to wait these combined almost 10 seconds per clip. Is it something one can expect some improvements on in the future ?

123 Fir
June 3rd build (I think it was, or the one thereabouts)




Thank you for your time.
once you go raw you never go back

a1ex

Quotehttps://mega.nz/#!8YIHGIJB!hzewH2JBX55vELsy2MPdp3gPSo2mTzlNKe_CQIVqu_o

In this particular case, I believe the vertical stripes are there because - just a guess - the first frame probably did not have relevant highlights to be analyzed.

Another approach that I didn't implement, but only documented the backend support for it: it would require pointing the camera to something like a gray wall and running a calibration routine that would adjust the column gain registers (8 multipliers) until the artifact disappears. Might be worth trying.

Quote
Allow Global Draw - OFF during recording to MLV lite as in the vanilla MLV

This one was a major source of confusion, so my previous answer is still valid.

Quoterecording starts so "sluggish", like 1-2 secs after

Sorry, not sure what you are talking about. Keep in mind I'm not a regular user of this mode (actually I don't remember touching the camera in the last few weeks...)

Is it the delay between you pressing the record button and the first recorded frame? Or the lack of responsiveness on the user interface?

QuoteI mean just by entering the mode and keeping it Armed for recording?

Pre-recording does exactly this.

Of course, the preparations can be done in background, with extra care when switching video modes, or when turning off raw recording. It's doable, but will increase the code complexity (although, probably not a bad idea, since some of the preparations are done in background in order to estimate recording times).

Quoteit takes about 4-6 seconds before I can hit record again, maybe longer

Set "Show graph: Buffers" to see what happens. All the frames that were recorded to RAM before you pressed Stop are now being saved to card (about 300MB iirc).

Quote
Is it something one can expect some improvements on in the future ?

If, in the future, you will select a resolution that gives continuous recording, then yes.

If we find some way to allocate more memory (the 5D3 has 512MB), this duration will increase (along with the recording time).

One can get a minor improvement (maybe speeding this up by 1-2 seconds) by pausing LiveView during this process. The silent picture module uses this trick, so it's an extremely easy coding task for those interested.

Quoteanother bug to report from 60p 3x3, when selecting ISO 100 the Raw Zebras become disabled

Will look into this one.

Kharak

Quote from: a1ex on June 16, 2017, 09:38:25 PM
In this particular case, I believe the vertical stripes are there because - just a guess - the first frame probably did not have relevant highlights to be analyzed.

Another approach that I didn't implement, but only documented the backend support for it: it would require pointing the camera to something like a gray wall and running a calibration routine that would adjust the column gain registers (8 multipliers) until the artifact disappears. Might be worth trying.


You are exactly right, first few seconds of that clip has no flaring.

I'll take a look in to the rest of your reply tomorrow and the calibration.
once you go raw you never go back

okanergoknil

I keep trying to install to firmware 1.2.3- 5d mark 3 however I cannot get the 4K tool. Can you please give the installation details? We tried on 2 different cameras but its not working  at all :(

Quentin

Any chance to get a flashing warning when card is almost full ?
Not during recording but when idle

Kharak

QuoteAnother approach that I didn't implement, but only documented the backend support for it: it would require pointing the camera to something like a gray wall and running a calibration routine that would adjust the column gain registers (8 multipliers) until the artifact disappears. Might be worth trying.

Read the thread again and I've been so interested in Darkframe Averaging and flatframe since the talk started here a long time ago, but as a Windows user there is no GUI converter with these options. Unfortunately MLVFS does not work properly with AE and AE is at the core of my workflow. Can you link me to a thread that explains how to type in the different codes/commands for Darkframing and Flatframing in mlv_dump, I cannot find any good relevant thread or post, there are so many search results of mlv_dump, but if you know where I should look, that would be nice, if not I'll keep looking.

QuoteThis one was a major source of confusion, so my previous answer is still valid.

Yeah, I just thought this 'Allow Global Draw - Off' was a specific "component" easy to port. I know what "Allow Global Draw - Off" means because I know what it does, but the wording of it makes no sense to me and even more confusing for newcomers.

QuoteSorry, not sure what you are talking about. Keep in mind I'm not a regular user of this mode (actually I don't remember touching the camera in the last few weeks...)

Is it the delay between you pressing the record button and the first recorded frame? Or the lack of responsiveness on the user interface?

Pre-recording does exactly this.

Of course, the preparations can be done in background, with extra care when switching video modes, or when turning off raw recording. It's doable, but will increase the code complexity (although, probably not a bad idea, since some of the preparations are done in background in order to estimate recording times).

Yes, recording starts 1-2 seconds after pressing record. I've tried with Pre-Recording and it does counter it but only after it has made the Pre-Recording which in it self takes 1-2 seconds to start recording aswell. On top of that, I have to activate the Pre-recording and then I have to press Record again, something that in the "heat of the battle" can get in to kind of an opposite effect where I record when I think I stopped recording or I stop a recording when I want to start it (happened a bit too much for me in 24p Mode.. and I only found out when I checked the footage  ;D ) I don't know, maybe I just need to get more used to it.

But with Pre-Recording I am getting less recording time, I usually get around 400/500 to 700 Frames and with pre-record 1 sec I get 300-450's. I use the lowest amount 1 sec pre-record (60 Frames), maybe you could add 1 frame pre-recording specific for the 60/50p mode as a standard, just so that recording starts immediately as an "easy" workaround? Not sure if easy or not. The Background thing you mentioned, sounds even better.

I can't give any technical explanation on why the recording times are lower, maybe I've just been unlucky with the compression, I must add that I did not test it extensively. Just done a few shots with pre-record in scenes where I was expecting longer recording times than the 300 to 400s.

EDIT: Just did a few more tests, switching between Pre-Recording and no Pre-Recording. The recording time is almost halved with Pre-Recording 1 Sec.

And you not touching your camera or this mode, you really have to try out the 60p mode, it is absolutely amazing! It has given my camera a new spirit. I thought I would use it more professionally but for filming family and those speedy little kids, its amazing.

QuoteOne can get a minor improvement (maybe speeding this up by 1-2 seconds) by pausing LiveView during this process. The silent picture module uses this trick, so it's an extremely easy coding task for those interested.

I don't think I would mind the LV freezing during that period, I am unable to do anything in that time anyway, so if it shortens the wait time it sounds good to me. Though I would suggest a small text appearing right before the freeze saying "Writing from Memory" or something like that, so users wont experience frustration over their camera freezing after every take and not knowing why if they didn't read this thread or pull.

Thanks again for your time.
once you go raw you never go back

Danne

@Kharak
Darkframe processing

1 - Film a few seconds with caps lock on your lens(totally black). Let´s call it your input_darkfootage.MLV.

2 - Perform averaging on this darkframe file like so:
mlv_dump -o averaged.MLV -a input_darkfootage.MLV

3 - Now use your averaged.MLV with corresponding normal footage.( Note! Use same settings when filming darkframe as corresponding footage). in command prompt do:
mlv_dump --dng -o OUTPUT INPUT.MLV -s averaged.MLV

INPUT.MLV is your nomal filmed footage
OUTPUT is your end result(dng files).

*note
If you leave out --dng you´ll get a MLV file instead of dng´s.
mlv_dump -o OUTPUT.MLV INPUT.MLV -s averaged.MLV