Author Topic: MLV App 1.8 - All in one MLV Video Post Processing App [Windows, Mac and Linux]  (Read 260571 times)

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 1271
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2450 on: January 06, 2019, 03:14:04 PM »
Also I've noticed that at the beginning of a batch all the MAPP files are created. It shouldn't be hard to extrapolate this as a feature, right?
MAPP files are only created when clips are loaded. In export they are loaded in the beginning to get the frame count - ETA calculation needs this.

... Actually it never went I way, I had thought to have manage to export the problematic clip (from mlv to compressed mlv) but I was mislead by the fact than when you click once a video in the left panel it actually doesn't do anything, you have to duble click to select. Shouldn't this be changed? ...
No, otherwise it won't be possible to export more than one and less than all clips at once. Selection would be broken.

edit: And now I understand why we had to do that very convoluted thing in order to use highlight recovery!! It is because the lighten curve in standard/film/tonemapping is actually crushing the image so in order to get that dinamic range back you have to do all that procedure. We were fighting the curve, now I get it. :)
The highlight is crushed before (-> when recording in the camera). But in relation to slider settings it is visible or not.

5D2.212 | EOSM.202

togg

  • Senior
  • ****
  • Posts: 401
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2451 on: January 06, 2019, 03:34:37 PM »
@togg
Nope, they aren't. They created when each clip is loading.

MAPP files are only created when clips are loaded. In export they are loaded in the beginning to get the frame count - ETA calculation needs this.

Then is it possible to extrapolate this feature as a batch MAPP creation ?

Ilia3101

  • Contributor
  • Hero Member
  • *****
  • Posts: 582
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2452 on: January 06, 2019, 05:09:44 PM »
1) as expected rec709 in resolve (as selected on the raw panel) and in MLV App looks completelly different. The one in Resolve is way darker : https://imgur.com/a/RkNOEY0

This should be down to the fact that 0 exposure in MLV App is actually +1.2 stops, did this because all traditional raw converters like lightroom seem to do this to enhance the feeling of recoverable highlights (I think it even matches the in camera preview image more).

If you set exposure down to -1.2 (approximately) it should look much closer than current result, maybe even almost identical. Way different doesn't need to be expected.
5D2, MLV App

togg

  • Senior
  • ****
  • Posts: 401
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2453 on: January 06, 2019, 05:21:40 PM »
Yes, upload that clip somewhere please.

bb

Here it is, uncompressed MLV causing the error ""Could not read VIDF image data from:  /Volumes/DNG/etc etc/RAW/M30-1357.MLV" when exporting to compressed MLV.

https://www.mediafire.com/file/grs8chz690p7q1p/M30-1357.MLV/file

But even more important I've found on my library a problematic MLV (uncompressed) that causes that terrible error of a few pages ago "could not export frame" etc etc. Only this time it is consistent, it isn't for sure a IO error and there's no way to actually export those frames. I can't even cut the MLV, it crashes if I try to do so. It is 1.4 GB, I'll see where I can load it.

This should be down to the fact that 0 exposure in MLV App is actually +1.2 stops, did this because all traditional raw converters like lightroom seem to do this to enhance the feeling of recoverable highlights.

If you set exposure down to -1.2 (approximately) it should look much closer than current result, maybe even almost identical. Way different doesn't need to be expected.

Ooh, I see. I guess that most non pro app are noticing the general way of underexposure on digital images (caused by a multitude of reasons) and give this boost.

togg

  • Senior
  • ****
  • Posts: 401
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2454 on: January 06, 2019, 06:29:03 PM »
An update from my workflow.

I went for a Master export (ProRes 444) from Resolve with bmd gamma, -3 sat, highlight recovery. I have to do this since my roundtrip from fcpx doesn't work. Using MLV App exclusevelly for dng extraction.

Then I'm using a bmd to cinema camera v2 rec709 conversion as a last node on the ProRes files in davinci. Works extremelly well!

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 1271
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2455 on: January 06, 2019, 07:40:12 PM »
Then is it possible to extrapolate this feature as a batch MAPP creation ?
Sure: export all clips to any format. Press abort as soon as ETA displays a time. Ready. Now you've only created MAPP files.

For your clip: last frame seems to be corrupted. If you look at last frame, all is black - no picture. If you export without last frame it works perfectly. How data looks exactly in this clip I have to take a look...

Edit: VIDF blocksize and all other variables = 0 ... no data for last frame available?!
5D2.212 | EOSM.202

togg

  • Senior
  • ****
  • Posts: 401
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2456 on: January 06, 2019, 08:33:42 PM »
Sure: export all clips to any format. Press abort as soon as ETA displays a time. Ready. Now you've only created MAPP files.

I'll do that :) Still not very user friendly for one of the most important features to view daily at the end of a shooting day!

For your clip: last frame seems to be corrupted. If you look at last frame, all is black - no picture. If you export without last frame it works perfectly. How data looks exactly in this clip I have to take a look...

Edit: VIDF blocksize and all other variables = 0 ... no data for last frame available?!


This is a big issue though. I have multiple MLV files that have one last empty frame. I can't check all of them and change out point etc just to do a batch export. The software should be able to notice one last black frame at the end, skip it, and move on.

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 1271
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2457 on: January 06, 2019, 08:49:27 PM »
I'll do that :) Still not very user friendly for one of the most important features to view daily at the end of a shooting day!

This is a big issue though. I have multiple MLV files that have one last empty frame. I can't check all of them and change out point etc just to do a batch export. The software should be able to notice one last black frame at the end, skip it, and move on.
One of the most important features?! ... we are talking about a processing app (main feature).

MLVApp is correct here, telling you that you're file is corrupted. A corrupted frame can be everywhere in a clip, not only at the end. Automating such issues(behavior with corrupted files) and doing something nobody really knows is no solution in my eyes. A round trip also can't work error free when doing "something". You better should try another magic lantern build and test if you produce less corrupted clips/frames with your camera, or correcting your corrupted clips manually. That's the only solution to really get what you want.
5D2.212 | EOSM.202

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 1271
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2458 on: January 06, 2019, 09:39:19 PM »
New most important feature added: MAPP batch creation.  :P
5D2.212 | EOSM.202

togg

  • Senior
  • ****
  • Posts: 401
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2459 on: January 06, 2019, 10:04:55 PM »
One of the most important features?! ... we are talking about a processing app (main feature).

MLVApp is correct here, telling you that you're file is corrupted. A corrupted frame can be everywhere in a clip, not only at the end. Automating such issues(behavior with corrupted files) and doing something nobody really knows is no solution in my eyes. A round trip also can't work error free when doing "something". You better should try another magic lantern build and test if you produce less corrupted clips/frames with your camera, or correcting your corrupted clips manually. That's the only solution to really get what you want.

Wait, maybing I'm doing things differently than any other, no prob then, really :P But how are you viewing your file before conversion? Are you doing it or you wait for the ProRes to come out (ideal, I agree)?

This is a standard shooting day for me, both documentary and fiction wise.

I shoot doring they day then I come back home and need to view the footage again to understand if everything went well, either catching a moment or see if the actress found her line in one of the 4 takes etc etc I need to cycle between takes on those moments, it is important to have like an uninterrupted viewing. Or maybe you are still using mlrviewer for that? Point is, as you said, MLV App needs the MAPP files even for processing! So basically MAPP files are needed for everything! You can't even touch the files without the associate MAPP file, now that I think of it they should almost be created on import. From my perspective it would be the most userfriendly and logical thing, but then again I'm not aware of other people setup, it's not like I'm judging it, I'm curious to know when people *don't* need MAPP files. Maybe the frame previewed on the left is enough for people to understand what to export and what not?


For how to deal with errors I disagree with you. Yes I've changed build a few months ago, went the compressed route which is more stable, but I still feel Magic Lantern outputs a lot of corrupted frames and such. It was always the case and probably will always be. I had a fairly recent build on the project of that file, still there're multiple instances where a black frame is present at the end. Reviewing every single files to be sure that your batch export will not get stuck during the night it's not really feasible imho. It would be wonderfull if Magic Lantern wasn't so proned to corruption but it is, at least if you shoot thousand of GBs of footage.


edit: I've a doubt, which camera don't suffer from vertical stripes? Is it camera dependent or build, resolution or something other? I was wondering why it is off by default in MLV App, I am under the impression that it is a standard problem for Canon sensors.

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 1271
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2460 on: January 06, 2019, 10:21:29 PM »
But how are you viewing your file before conversion? Are you doing it or you wait for the ProRes to come out (ideal, I agree)?

Yes, exactly. First I render proxy of all files (MLVApp). Then I cut (NLE). Then I only correct the used files (import via fcpxml import feature) (MLVApp) and relink in NLE.

Point is, as you said, MLV App needs the MAPP files even for processing!

No, it doesn't. MLVApp creates MAPP files if option is enabled and a clip is loaded. This happens on export for all files in a session if all files were marked. For processing it isn't needed at all.

So basically MAPP files are needed for everything!

No, MAPP files are only needed for faster loading of large files. No other feature.

You can't even touch the files without the associate MAPP file, now that I think of it they should almost be created on import.

?! Sure you can. I nearly always have MAPP disabled and still can use all features of MLVApp. I only enable it if I have clips >5min or if clips are on old slow HDDs for faster loading. If the clips are smaller or I work on SSD I feel no difference.

For how to deal with errors I disagree with you. Yes I've changed build a few months ago, went the compressed route which is more stable, but I still feel Magic Lantern outputs a lot of corrupted frames and such. It was always the case and probably will always be.

That is sad. I don't remember having a corrupted frame over 5 years using ML on my 5D2.

You now can compile latest commit version with batch MAPP creation feature. Maybe someone else is happy about it...
5D2.212 | EOSM.202

togg

  • Senior
  • ****
  • Posts: 401
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2461 on: January 06, 2019, 10:38:30 PM »
No, MAPP files are only needed for faster loading of large files. No other feature.

You're right, sorry I got confused, but when MAPP file are not created it means that MLV App is indexing and storing the info in the background right?I mean an indexing is needed, either wrote on a MAPP file or invisible. This is why I feel that it would be more logical to have it done on import. But for me the batch MAPP (hacked with the export thing or as a feature) will be enough :P It is mostly a concern for new users.

Got it for your workflow, problem for me is that the export of a day of shooting takes multiple hours, so basically I have to let it run during the night. Which would be similar to film processing workflows where dailies where watched on next morning :)

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 1271
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2462 on: January 06, 2019, 10:48:24 PM »
You're right, sorry I got confused, but when MAPP file are not created it means that MLV App is indexing and storing the info in the background right?I mean an indexing is needed, either wrote on a MAPP file or invisible. This is why I feel that it would be more logical to have it done on import. But for me the batch MAPP (hacked with the export thing or as a feature) will be enough :P It is mostly a concern for new users.

Got it for your workflow, problem for me is that the export of a day of shooting takes multiple hours, so basically I have to let it run during the night. Which would be similar to film processing workflows where dailies where watched on next morning :)
Indexes will - no matter what you do - be loaded into RAM, with or without MAPP file. Without MAPP file indexes have to be searched in MLV, with MAPP file they can be read out from MAPP file. You mustn't be worry about "export hack" anymore... feature is now there ;)

If you need to watch your clips immediately in realtime, you should do this with MlRawViewer. This is why there is the "Open in external application" feature. If you just like to check if the focus or exposure is correct, you should be able to do that in MLVApp without export.
5D2.212 | EOSM.202

togg

  • Senior
  • ****
  • Posts: 401
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2463 on: January 06, 2019, 11:08:38 PM »
OK perfet then! Thanks ^^

(I was trying to ditch mlrawviewer since it is so outdated and unreliable with audio etc.)

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 780
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2464 on: January 07, 2019, 01:33:05 PM »
This is why I feel that it would be more logical to have it done on import.
One more clarification: It is done exactly on import.

Now what does it mean. When one MLV is to be loaded it needs indexing, e.g. find all video and audio frame offsets to sort by time and easily access post loading. Indexing is pricey operation because it needs the MLV to be checked frame by frame (block header by block header) from the beginning to the end! The MAPP is the all of the found info sorted and written into separate file followed by audio data at the end if such exists. Second time when same MLV is imported MAPP presence is checked and if exists all info loaded from it not the huge MLV. This saves a lot of time on slower HDDs (even SSDs if file is big enough).

So what we do is when selected clips are more than one, all but the last clip are read in just preview mode (MLV searching stops after 1st video frame found) to generate only thumbnail. The last imported MLV in the session list is the default selected one and it will be read wholly.

hope this helped
bb

@masc: during export for ETA calculation do you really read all MLVs twice?!

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 1271
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2465 on: January 07, 2019, 03:01:19 PM »
@masc: during export for ETA calculation do you really read all MLVs twice?!
Yes, I read it once for ETA calcluation (read the frame count) in the beginning and then a second time to render all frames. I have no better idea how to get frame count of all MLVs.
5D2.212 | EOSM.202

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 780
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2466 on: January 07, 2019, 07:03:12 PM »
I have no better idea how to get frame count of all MLVs.
Yeah that's right. Even if you will only read start of the file and get frame count from MLV header block (it can be a bit off +-15frames for older MLVs recorded by mlv_rec.mo) the value represents frame amount of the current chunk only :(

For getting the total amount of session frames I do not have other solution either...

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5698
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2467 on: January 11, 2019, 12:20:58 PM »
Here´s a dualiso attempt coming from eosm and it´s 1x3 mode setting. Recorded in:
10bit,
sd_uhs,
24 fps,
dualiso 100/1600,
1600x2040(2.35:1)


mlv sample:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/1x3_dualiso_eosm_M11-1209.MLV








masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 1271
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2468 on: January 11, 2019, 12:31:08 PM »
Wow! That looks awesome! How long are you able to record with the EOS M using these settings?
5D2.212 | EOSM.202

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5698
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2469 on: January 11, 2019, 12:38:51 PM »
I tested a few seconds but seemed continuous. If not lowering res a bit will be without interrupts.
I ran framing preview on so turning that off should be even better.

togg

  • Senior
  • ****
  • Posts: 401
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2470 on: January 16, 2019, 06:58:06 PM »
Are we sure vertical stripe removal actually work? I'm consolidating the doubt that it doesn't, I have to test it more but looking at my clips doesn't seems like the parameter is kickin in, should check hot pixel as well mmm

Ilia3101

  • Contributor
  • Hero Member
  • *****
  • Posts: 582
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2471 on: January 16, 2019, 07:25:26 PM »
I have the stripiest camera in the world and I can confirm it works.
5D2, MLV App

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 1271
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2472 on: January 16, 2019, 08:15:26 PM »
And I have the 2nd stripiest camera here... works fine.
5D2.212 | EOSM.202

70MM13

  • Member
  • ***
  • Posts: 245
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2473 on: January 16, 2019, 10:12:36 PM »
i've gotta say, it never works for me either.  i stopped trying long ago.

it introduces strange lines and generally looks bad.  i put it down to not being intended for the stripes i get from the gain reduction hacking i do...

or maybe it doesn't work properly with the 5d3?

or maybe just mine  :P

togg

  • Senior
  • ****
  • Posts: 401
Re: MLV App 1.4 - All in one MLV Video Processing App [Windows, Mac and Linux]
« Reply #2474 on: January 16, 2019, 10:38:54 PM »
I have a 5D3 as well, will test.