MLV App - All in one MLV Raw Video Processing App [Windows, Mac and Linux]

[masc]

Filmmakers
¿Any tip for someone who has a 5D MK2 (mlv 2.0) and wants to shoot a shortfilm in raw and then post-edit in Adobe Premiere? In terms of LUT's order..if it's more important to apply a LUT in MLVAPP or export (proRes 4444) and apply the LUT in Premiere?

¿Which LUT for general color-grading do you guys apply? MLVAPP or in a video editing software (such as Resolve, Final Cut, Premiere..). I have dozens of LUT's, but don't know when to apply them, or if there are some very common on this community.

I would add the LUT at the stage where you color correct / color grade. So if you do this in MLVApp, add it in MLVApp, if you do this in Premiere, do it in Premiere.

[50mm1200s]

@masc since we're already talking about encoding, about our discussion some time ago on color management... no improvements? I see advanced color management can't be done with ffmpeg, but at least it supports bt2020 out of the box...

[masc]

@masc since we're already talking about encoding, about our discussion some time ago on color management... no improvements? I see advanced color management can't be done with ffmpeg, but at least it supports bt2020 out of the box...

Hmm... it supports not really... it only supports adding the metadata, but that does not mean bt2020 is inside. You'll need a matrix conversion inside the ffmpeg command. But that won't "bring" more quality, because the input & encoding is still the same. There is this colormatrix=... command - this one works in 8bit - so you'll loose more than you'll get. You also can adjust colorspace via scale filter, this one works with higher bitdepth. Atm we convert and export to bt709 in this way. Otherwise you can export with AVFoundation, if you don't like bt709, or you must color correct in another software if thats not enough - you're right.

[50mm1200s]

Filmmakers
¿Any tip for someone who has a 5D MK2 (mlv 2.0) and wants to shoot a shortfilm in raw and then post-edit in Adobe Premiere? In terms of LUT's order..if it's more important to apply a LUT in MLVAPP or export (proRes 4444) and apply the LUT in Premiere?

¿Which LUT for general color-grading do you guys apply? MLVAPP or in a video editing software (such as Resolve, Final Cut, Premiere..). I have dozens of LUT's, but don't know when to apply them, or if there are some very common on this community.

If you're going to apply a LUT, do it directly on MLVApp if possible. But, you might find MLVApp limited for color grading, since once it's encoded you can't change it (destructive workflow). So what I do is this:
- Import on MLVApp. Put the clips on Alexa Log-C. Fix exposure and WB. Export to ProRes 444
- Import on Premiere. Use Logarist to convert from LogC to Rec.709.
- Do color grading with Lumetri.

This way you have a somewhat solid color grading. I sometimes use ImpulZ LUT's (the "FPE" type, that is, the film color, but not density), but they have many issues on dark midtones (some color shift towards magenta).

You would get better results using a CinemaDNG on Resolve if you will be doing heavy color grading...

[50mm1200s]

Hmm... it supports not really... it only supports adding the metadata, but that does not mean bt2020 is inside. You'll need a matrix conversion inside the ffmpeg command. But that won't "bring" more quality, because the input & encoding is still the same. There is this colormatrix=... command - this one works in 8bit - so you'll loose more than you'll get. You also can adjust colorspace via scale filter, this one works with higher bitdepth. Atm we convert and export to bt709 in this way. Otherwise you can export with AVFoundation, if you don't like bt709, or you must color correct in another software if thats not enough - you're right.

Ok, I get it now. Thanks.

[IDA_ML]

Hello Masc,

Here are two 2K files from the 100D shortened to 5 frames each and exported with MLVApp to MLV according to your instructions.  The export option is Fast Pass.  The focus pixels on the videos are clearly seen, also there should be slightly overexposed areas on the files for you to check Highlight reconstruction.

https://we.tl/IDENw3zZHe

The link will be active for 7 days.  Please let me know if these files work for you and if yes, I will send you also other ones at higher resolutions. 

If you could fix the focus pixel issue on the latest 4k-crop versions of the Rebels and include this FP cleaning in the next version of MLVApp, the MLV export function in combination with cleaning focus pixels and Cut-in and Cut-out, will make an extremely useful function of MLVApp to remove unwanted sections in the clips, reduce their file size and prepare clean footage for further work with an NLE as DaVinci Resolve.  That would be GOLDEN !!!

[dfort]

@IDA_ML -- here's something you can try - no coding skills needed 

Not sure if you're on Mac or Windows but I believe you mentioned that you have MLVFS working on your system. That's the first app that used the focus pixel map files and what I still use when experimenting with new resolutions. Dig into the app and look for a file named, "80000346_2592x1108.fpm" now make a copy and name it "80000346_2592x1332.fpm" and try it on your MLV files.

Long answer.

If you extract the DNG's without chroma smoothing then process a DNG frame with dcraw using the -4 -E options you can clearly see the focus pixels.



and this is what it looks like in color:



Looking at the sample files with mlv_dump using the -v option we find this:

Code Select Expand

Block: RAWI
  Offset: 0x00000034
  Number: 1
    Size: 180
    Time: 0.910000 ms
    Res:  2096x1180 <--- image size
    raw_info:
      api_version      0x00000001
      height           1332 <--- full raw buffer is 2592x1332
      width            2592


That's the same full raw buffer width as in zoom mode but with increased vertical resolution. Now the question is if the crop and pan metadata works the same? This is what is needed to line up the focus pixel map file--the map files cover the full raw buffer so they can be used with all of the various image size and aspect ratio options.

It works!



Well almost. Focus pixels show up at the bottom of the frame because the focus pixel map file that is currently in MLVFS doesn't completely cover the new full raw buffer.

So all that is necessary is to extend the zoom mode focus pixel map file to cover the additional vertical resolution and we should be able to eliminate the focus pixels on this new crop_mode resolution.

[EDIT] Here are before and after images showing the MLVFS hack. You can zoom into them in flickr without having to download the file to take a close look at the focus pixels.

Before:


After:

[masc]

Thx @IDA_ML for the files. Highlight reconstruction works for this files. In the appartment file in the furniture window you can see a difference when toggling, in the city clip I only see that something happens in the histogram.

Thx @dfort for your explanation, but I don't get that to work. All focus pixels are still there. I tried exactly what you wrote, but MLVApp and MLVFS won't handle the focus pixels. MLVApp works with the same fpm files as MLVFS normally.

@IDA_ML: the focus pixels issue not really an issue - all is working - all we need are the focus pixel maps for these resolutions. These have to be found... if we/you have them, the MLVApp version you have now will handle it already.

@bouncyball: are we able with our algorithm for focus pixels to calculate the map somehow? Or do we need to know the pixels first?

[dfort]

Thx @dfort for your explanation, but I don't get that to work. All focus pixels are still there. I tried exactly what you wrote, but MLVApp and MLVFS won't handle the focus pixels. MLVApp works with the same fpm files as MLVFS normally.

That's strange. I updated my fpm.sh bash script to handle the new resolution and it is working fine over here in MLVFS:



Notice that the focus pixels that were on the bottom of the previous post are gone. I converted the DNG to TIFF using dcraw because that tends to show focus pixels more than other apps like DaVinci Resolve and Adobe Camera Raw.

Here is the new focus pixel map file for the 100D:

https://bitbucket.org/daniel_fort/ml-focus-pixels/src/default/focus_pixel_map_files/80000346_2592x1332.fpm

[masc]

@dfort: thx, this works perfectly, also with MLVApp!
@bouncyball: can you also update the focus pixel calcuation for such resolutions for EOS 100D, please?

[bouncyball]

@dfort: Yay! Now it works perfectly. Thanks for a quick response on my humble request

@masc: Sure

[Danne]

Working perfectly @dfort

[bouncyball]

@masc: Done! You can remove 80000346_2592x1332.fpm from repo.

Edit: "fpmutil" also updated.

[masc]

All unused focus pixel maps removed. Thx bouncyball & dfort - your work works great!

[IDA_ML]

A truly spectacular development.  Thank you, Dfort, Bouncyball, Masc and whoever may be involved.  If this developments continues at that speed, very soon our little cameras will become as powerful as Ari Alexa for RAW video.

And since the interest in the focus pixel subject seems to be substantial, I have prepared more test files from the 100D that everyone interested can download from here in the next 7 days:

https://we.tl/nTSisEjsan

These include 2520x1304; 3096x1320 and 4056x2280 resolutions.  Some of them include a few overexposed areas for you to check white levels that in my opinion, do not fully work yet.  I still notice a slight pink cast when I reduce the exposure or Highlights in MLVApp.  I have also included two 4K files shot in Dual ISO 100/800 which now again becomes very relevant, interesting and important for smooth timelapse videos.  Note the absence of noise in the dark areas of these Dual ISO shots.  This is really spectacular!

Dfort, please have a look at the shot where my daughter wipes out the table after breakfast.  In my opinion, there are still focus dots on that clip.

[FBongcam]

Hi,

I was wondering if there would be any way to process the older .RAW files with this app? I know the format probably is a bit outdated now...

As I'm able to process them in mlrawviewer I thought maybe it's not impossible to incorporate support in this app.
If it's out of the question, would there be any way to convert .RAW files to .mlv?

I have a bunch of old footage I'd really like to process with the workflow that this app offers.

Thank you for the continued development!

[Danne]

On mac you can use "Switch". Or else compile raw2dng. Bouncyball implemented raw to mlv conversion. Works very good.

[bouncyball]

Yeah you have to follow Danne's advice. We are not going to add .RAW support to MLV App.

If it's out of the question, would there be any way to convert .RAW files to .mlv?
I have a bunch of old footage I'd really like to process with the workflow that this app offers.

I also had bunch of old .RAW files from 60D, that's why I decided to implement RAW to MLV converter into raw2dng utility.

Download it here for any OS (it is command line utility, prints out help)

[masc]

These include 2520x1304; 3096x1320 and 4056x2280 resolutions.  Some of them include a few overexposed areas for you to check white levels that in my opinion, do not fully work yet.  I still notice a slight pink cast when I reduce the exposure or Highlights in MLVApp.  I have also included two 4K files shot in Dual ISO 100/800 which now again becomes very relevant, interesting and important for smooth timelapse videos.  Note the absence of noise in the dark areas of these Dual ISO shots.  This is really spectacular!

Dfort, please have a look at the shot where my daughter wipes out the table after breakfast.  In my opinion, there are still focus dots on that clip.

Thanks again for the clips!
Where do you see focus dots in the clip with your daughter? I can't find any... The pink highlights were gone in all shots, except for the last one. Note that I did not test with v0.16, I tested with the latest revision from the repos. For the last clip I changed dual iso highlight reconstruction in a way it works also for this clip now. For dual iso clips it is much more difficult than for standard clips. I know that my algorithm for dual iso is not perfect: if there is a bright pink in the picture which is no clipped highlight (e.g. a flower), it is filtered as well.

For the higher resolutions than 2.5k there is no focus pixel map yet there... so it has to be found again.

[IDA_ML]

Masc,

Thank you for checking out the files that I provided.  Right now, I am using the 016 version from the first page but work is still in progress.  As long as we are clear about all resolutions and center offsets, I will be more than happy to provide more test files as needed. In this way, it will be possible to refine the focus pixel maps at all resolutions.

[FBongcam]

Yeah you have to follow Danne's advice. We are not going to add .RAW support to MLV App.
I also had bunch of old .RAW files from 60D, that's why I decided to implement RAW to MLV converter into raw2dng utility.

Download it here for any OS (it is command line utility, prints out help)

Thanks, it worked. I do however get weird artifacts on clips where the black level is wrong. When corrected with mlv_dump I don't get these.
It's like pink highlights/dots around edges etc.

[masc]

Thanks, it worked. I do however get weird artifacts on clips where the black level is wrong. When corrected with mlv_dump I don't get these.
It's like pink highlights/dots around edges etc.

Could you provide a MLV (shortend to some frames is enough for analysing) please?

[masc]

Masc,
Thank you for checking out the files that I provided.  Right now, I am using the 016 version from the first page but work is still in progress.  As long as we are clear about all resolutions and center offsets, I will be more than happy to provide more test files as needed. In this way, it will be possible to refine the focus pixel maps at all resolutions.

It is always good to have newest stuff to test... thx again! v0.17 is in work for everybody and will be another step forward We just have to get some little things nice atm.

[FBongcam]

Could you provide a MLV (shortend to some frames is enough for analysing) please?

This is the mlv file converted from .raw with Switch to .mlv file. Shortened using MLV export in MLVApp.

https://www.dropbox.com/s/lafak8wh13xopmj/M01-2016.MLV?dl=0

[bouncyball]

I do however get weird artifacts on clips where the black level is wrong. When corrected with mlv_dump I don't get these.
It's like pink highlights/dots around edges etc.

Hmm... and where are those artifacts in the clip posted above? Cleanest image ever, IMHO