Messages

I think it is a bit wrong. Look at the 1st post of this thread. a1ex showed the 5d3 and all other (lower) camera bining methods. The averaging is done not exactly by 6 raw line pixels but a next pixel for every color (R,G+1 for first line and G,B+1 for second line). 5D3 is a complete another story with 3 by 3 binning.

I thought about what you've showed above and the algo you proposed, and according to the initial and the end values and their differences this is very inaccurate approach. Anyway after averaging the information is lost forever and it is impossible to recover it except only very basic cases.

Well... I'm gonna think about it more, but... maybe a1ex could suggest, does it worth a try or not.

regards
bb

If somehow the raw file can be reconstructed to a new raw file according to the binning pattern, then you have a good starting point:

I really dig the unbining idea. I need to understand more the bining theory and need some visual examples.

I have no better idea how to get frame count of all MLVs.

Yeah that's right. Even if you will only read start of the file and get frame count from MLV header block (it can be a bit off +-15frames for older MLVs recorded by mlv_rec.mo) the value represents frame amount of the current chunk only

For getting the total amount of session frames I do not have other solution either...

This is why I feel that it would be more logical to have it done on import.

One more clarification: It is done exactly on import.

Now what does it mean. When one MLV is to be loaded it needs indexing, e.g. find all video and audio frame offsets to sort by time and easily access post loading. Indexing is pricey operation because it needs the MLV to be checked frame by frame (block header by block header) from the beginning to the end! The MAPP is the all of the found info sorted and written into separate file followed by audio data at the end if such exists. Second time when same MLV is imported MAPP presence is checked and if exists all info loaded from it not the huge MLV. This saves a lot of time on slower HDDs (even SSDs if file is big enough).

So what we do is when selected clips are more than one, all but the last clip are read in just preview mode (MLV searching stops after 1st video frame found) to generate only thumbnail. The last imported MLV in the session list is the default selected one and it will be read wholly.

hope this helped
bb

@masc: during export for ETA calculation do you really read all MLVs twice?!

@togg

Also I've noticed that at the beginning of a batch all the MAPP files are created. It shouldn't be hard to extrapolate this as a feature, right?

Nope, they aren't. They created when each clip is loading.

E.g. when batch processing:
1st clip loaded, if MAPP not present it created, clip is processed, finished with this clip (closed)
2nd clip loaded, if MAPP not present it created, clip is processed, finished with this clip (closed)
and so on...

Anyway the error present itself in files that seems to have a black frame at the end, thing is than when exporting to lossless dng everything goes fine. I'll try to upload the mlv to test.

Yes, upload that clip somewhere please.

bb

@DeafEyeJedi

If so, mind if I ask how exactly did you do DF avg for this?

Sure.

1. Load M26-2323.MLV source clip into mlvapp then go to export settings and select Codec "MLV" with option (lower dropbox) "Averaged Frame". Export result somewhere (with changed name. E.g M26-2323_DF.MLV or something). Just temporarily Import this averaged MLV as a _regular_ clip to check whether it crashes app or not.
2. Use this averaged MLV as a dark frame in RAW Correction "Darkframe Subtraction" section (load it with yellow folder button).

One more thing. Could you share your already averaged MLV. I mean made by switch (hence w/mlv_dump) which crashes mlvapp?

That's all
bb

Hm, I can not reproduce the crash with uploaded file, sorry... it loads and plays flawlessly.

Perhaps I could/should have recorded all DF files in 14-bit uncompressed MLV's and STILL be able to apply these files in conjunction with 12-bit lossless MLV's, correct?

I believe the best is to record DF clip with the exact same settings. However it can be recorded with different bit depth as far as video mode/resolution is matched. Compressed/uncompressed does not matter.

For me that sounds like there happens something strange with Switch when creating darkframe MLVs... maybe metadata still tells there is audio, but it isn't, or something like that. If you create darkframe MLVs in MLVApp... happens the same?

Exactly that could be the root of this issue.

@DeafEyeJedi: Hey man! Give me the unmodified file which crashes mlvapp. I'll take a closer look.

BTW guys, if you got the error message "Can not use lossless MLV as a dark frame" this does not mean you can not use dark frames for lossless footage. This means that properly averaged dark frame (the one frame MLV) can not be lossless. Both mlvapp and mlv_dump (I believe) exporting dark frame MLV not compressed. I regret I did not introduce new dark frame file format for mlvapp, I just did it for the compatibility sake with mlv_dump.

Anyway MLV recorded as a dark frame source to be averaged to one frame DARKFRAME can be either lossless or uncompressed. Also the source which this dark frame was created for can be either lossless or uncompressed, so there is none of restriction applied.

[Zoom 200%]

Can you add a Zoom 200% option in the viewer?

You can zoom in/out with mouse scroll wheel. There is no problem beyond the 100% zoom until one pixel covers whole screen .

Even better with mlvapp's HSL correction and gradient tool



I guess left side of the sky is really unrecoverable.

BTW I've added MLV decompression option to mlvapp recently. So you can easily uncompress any MLV and play it even with good old unmodified MLRawViewer. I can play 3K+ MLVs in real time w/o issues.

@togg

Yes I'm trying to save to losslessly compressed DNG from 14bit compressed MLV. Also note that I'm talking about a 250GB total export batch, around 10 takes total.
Is MLV App trying multiple times if a frame give out errors? Could this be related to drive I/O errors? Weird though, since hard drives were still connected.

Lossless compressor fails sometimes (and there is nothing I can do about it). The encoder spits out error and it means the data can not be encoded efficiently (e.g. compressed data is even larger then original )
And yes this error also occurs due to I/O issues.
Please try to export uncompressed DNGs (it is working every time) and report back.

Also the opening with MAPP file is still a little bit long I'd say, 1 second and something? But I guess it can't be brought down easely.

Well... if the clip is long enough and includes audio then yes (w/o audio almost no time needed).

@megapolis

Thanks for the clarification I like your cool app and it works smoothly on my GTX1080. I just miss the Linux version.

Hi guys,

@togg

1) I really need that MLV to reproduce this error. I wrote this error handling and sadly never could reproduce this exact error in real life so I'm really interested to play with it. BTW are you trying to save losslessly compressed DNGs?

2) Yeah... MAPP is created when MLV fully read not just opened to generate preview thumb. After loading the MLV just once, accessing next time(s) to this exact clip is very fast. Actually batch saving of the MAPPs, as a separate action in the menu, is doable. Maybe we'll think about it with @masc.

3) Yeah we do not use GPU. Reason: if we do so we have to implement all processing pipeline (not only debayer) also on GPU, it's tough! Look at "fastcinemadng" it uses expensive external library which does lots of stuff on GPU in pure Nvidia's CUDA. Also look at the evolution of the MLRawViewer. I mean compare 1st commit to the last one and you will see how it grew from just one opengl shader to 6 or 7 including basic processing and minimalistic but cool GUI

regards
bb

Sad. There is still no Linux version available...

The reason why I like them separate windows is that I can color correct faster that way, I don't need to click r g b separate and can make quick adjustments between the curves.

This can be circumvented by keyboard shortcuts. e.g. 1,2,3,4 or similar. I don't see real advantage of occupying x4 more space for it.

but still, I got minor artifacts like red and white dots it appears like for 1 frame long.

Try turning off the bad pixel removal. This might help.

So older(no metadata) dualiso files will still work if force button is used?

Yup, it's a new feature in v1.3 .

Yes, its usage is a little tricky - it requires a very careful exposure to the right (ETTR) without blowing up the highlights.  But, once you get is right, the results are really gorgeous.

I never said dual iso is not good . a1ex did incredible job on it! And I'm using it for still shooting since the invention. But for motion pictures diso processing generates lot of aliasing for my taste, and that is where future 1x3 crop might be handy...

I just said it needs specific approach but did not want to go into deep explanation. I'm glad you got there by yourself! Now this IS a good approach to dual iso shooting. Never go above 1600 recovery ISO it is almost makes no sense.

I took a look at your latest MLV and it looks a LOT more groovy.

regards
bb

Note that the settings were extreme - 400/3200.  Without Dual ISO, the result would have been much worse in both: the highlights and the shadows.

Well, I doubt it , but anyway... have a nice test drive!

This MLV looks OK if white level is set to 5000-ish. No black level adjustment required.

Edit: I bet it is recorded by latest bleeding edge build with support of 1x3 crop and lowered gain registers, with very dark output and incorrect white level.
Edit2: look at the walls in shade and the unrecoverable lightness outside. What is the purpose of using dual iso if highlights are gone and in shadows there are lots of dancing ugly color noise dots?

...if I understood right (@bb?).

Yeah, but for some rare clips it happens (black level is not optimal).

I think everyone should understand that just point and shoot method is not right thing for obtaining usable dual iso footage with good DR. For some situations dual iso not a friend but enemy.

One thing that should be kept in mind with Dual ISO files is that you may get weird green casts in the darkest areas of your Dual ISO footage after applying the Dual ISO function in the RAW menu.  These can be completely cured by carefully adjusting the White level slider to a lower value.  The same applies also to some pink casts in the highlights.
ore hints on part of the developers on how to handle DualISO files for preserving highest quality would be very welcome.

Exactly and this is the one of the main reasons why those sliders are there.

Yes (thanks to @Danne) no special recipe is needed. Turn on dual iso with "AMaZE" and "Alias Map" on and export dngs (sometimes dual iso lossless dngs can not be created and error shows up, so your best bet is uncompressed ones). That's all

Note: do not use "CinemaDNG Fast Pass" it will completely bypass dual iso processing.

@ibrahim

Well actually exporting raw from raw (MLV -> DNG) is absolutely lossless and identical quality.

There are 2 options thought:
1. Just pass through all untouched raw data from MLV to DNG
    It's very fast but has downside. All original flaws of the raw image, e.g vertical stripes, bad pixels, focus pixels (only for affected cameras) etc are there also untouched
    and unprocessed.
2. Do raw processing and get rid of all original raw flaws. It's slower but also maintains highest quality no matter what e.g. exported either as uncompressed or lossless DNGs.

regards
bb