Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter (Read 959029 times)

dmilligan · « **Reply #825 on:** January 20, 2016, 03:57:10 PM »

AFAIK, the 7D's vertical stripes and the 5D3's are slightly different in nature. MLVFS incorporates the vertical stripes fix created by a1ex for 5D3. MLRV has a different algorithm developed by baldand who was a 7D user and created it mainly for 7D.

People with 5D3s who use MLRV actually have the opposite problem you're having.

Chroma smoothing will not fix vertical stripes (a luminance issue).

It might be a good idea to post and example file.

DeafEyeJedi · « **Reply #826 on:** January 20, 2016, 06:24:01 PM »

Here are some DNG's from 7D recently...

https://mega.nz/#F!6okHxJIZ!Mu8084Iw31wxf9Cut9ryEw

johannsebastianbach · « **Reply #827 on:** January 20, 2016, 07:20:25 PM »

What do you guy's think about a resizing option? I already love that MLVFS is automatically stretching the DNG's from 1280-mode in the right aspect ratio and it could be useful in conforming the size already before cutting (e.g. for multicam shots or if you need to downsize the framing to get more fps or so), right?

dmilligan · « **Reply #828 on:** January 20, 2016, 08:29:20 PM »

You can't really resize raw data until after you demosaic it (you could downsample it, but that would potentially introduce aliasing, and why do that anyway?) and then it's not raw data anymore. Pixel aspect ratio is just metadata. We are telling the raw processing software that it needs to do the stretch, but we can't actually do the stretch ourselves (we don't demosaic raw data, and I don't plan on adding any such feature).

Only additional thing that would be possible is specifying a custom pixel aspect ratio for anamorphic shooters.

johannsebastianbach · « **Reply #829 on:** January 20, 2016, 10:12:16 PM »

Quote from: dmilligan on January 20, 2016, 08:29:20 PM

You can't really resize raw data until after you demosaic it (you could downsample it, but that would potentially introduce aliasing, and why do that anyway?) and then it's not raw data anymore. Pixel aspect ratio is just metadata. We are telling the raw processing software that it needs to do the stretch, but we can't actually do the stretch ourselves (we don't demosaic raw data, and I don't plan on adding any such feature).

Only additional thing that would be possible is specifying a custom pixel aspect ratio for anamorphic shooters.

That makes sense, learned something new, thanks

PaulHarwood856 · « **Reply #830 on:** January 21, 2016, 01:48:18 AM »

Hey dmilligan and DeafEyeJedi,

I think I found a solution in MLVFS. I turned off Chroma Smoothing, kept Vertical Stripes fix on, and I switch Bad Pixel Fix to Aggressive. This seemed to eliminate most of the vertical stripes. I could use MLRawviewer to export cdngs, but MLVFS is so fast and convenient. Do you know what ISO settings I should try to stay at? And is there a way to set the EV and Black Levels in the camera to avoid vertical stripes all together? I read this online, but I don't quite understand it. Any more help would be greatly appreciated, and thank you for all that you have already done.

- Paul Harwood

dmilligan · « **Reply #831 on:** January 21, 2016, 02:43:12 AM »

You need to be more specific about the type of vertical stripes you are talking about because it is not clear (which is why I asked you to upload a sample).

I'm not sure what you mean by "EV" which is typically used as a unit of measurement based on a logarithmic scale. Perhaps you mean "EC" (exposure compensation)?

Black level is not something you can set or adjust either. It is piece of metadata need by the raw processing software (e.g. ACR, Resolve, etc.) to correctly interpret the raw data. It is either correct (matches the actual physical black level of the sensor) or incorrect (does not match). If it is wrong, your image will look totally screwy and have extreme color casts. This is something that you shouldn't need to fool with and should be totally transparent to you as an end user.

There are posts in this forum about how to "fix the black level". This is only necessary if you have footage with incorrect black level metadata (as a result of an ML bug). But this does not sound like your issue.

Discussion about choosing ISO or optimizing exposure is really outside the scope of this thread.

PaulHarwood856 · « **Reply #832 on:** January 21, 2016, 09:09:49 PM »

Hello dmilligan,

Thanks for your reply. Here is a link to the vertical stripes issue I'm having with the 7D: https://drive.google.com/open?id=0B7-vwCkJCer5TmRFVnBiOWJsX0E

If you could take a look and provide any input that would be awesome. Thanks.

- Paul Harwood

PaulHarwood856 · « **Reply #833 on:** January 21, 2016, 09:39:31 PM »

Sorry everyone, here is the link, but now it is shareable to all of you: https://drive.google.com/file/d/0B7-vwCkJCer5TmRFVnBiOWJsX0E/view?usp=sharing

bwinter88 · « **Reply #834 on:** January 29, 2016, 01:43:09 AM »

I'm having trouble loading After Effects projects with MLVFS image sequences in them. I've been sitting here for 10 minutes and the Open Project progress bar is at 32%. mlvfs process in activity monitor shows 99% CPU. In fact, anytime I try to open sequences in Premiere or AE, the program hangs and MLVFS process revs up the CPU for 1-2 minutes before the program resumes. Is this normal behavior? I'm on a Mac Pro 2010 with OS X 10.11.2.

I updated mlvfs and FUSE to the latest version a couple days ago. I think something changed? Although performance has always been a little slow. For instance every time I want to modify source settings of an MLVFS image sequence (open up ACR dialog in AE for instance) I have to wait about 60 seconds until the window appears.

I know it's not a hard drive latency issue because it's on an external RAID that blackmagic disk speed test gives a 240MB/sec read speed. I've also accessed these files before from an external USB 3 drive and performance was actually better then. (!)

PaulHarwood856 · « **Reply #835 on:** January 29, 2016, 06:21:38 PM »

Hey bwinter88,

This might seem silly, but I think it's worth trying. Try uninstalling Fuse and MLVFS, and reinstalling. But, use the versions that worked before you experienced crashes. To be honest, I've had to downgrade from El Capitan to Mavericks because I was experiencing so many crashes. It works a lot better. But, uninstalling and reinstalling the previous versions of FUSE and MLVFS might just do the trick. I hope this helps!

- Paul Harwood

aschille84 · « **Reply #836 on:** February 02, 2016, 10:04:09 PM »

Is MLVFS resizing 1280 footage in DR12? Seems like I have to stretch the footage or is there any other way to do it?

aschille84 · « **Reply #837 on:** February 02, 2016, 10:20:24 PM »

Hi johannsebastianbach, maybe you know something about this?

Anyway, found a way. In "Input scaling presets" in project settings, chose "stretch to fit corners", and voila! It stretches to the timeline resolution.

johannsebastianbach · « **Reply #838 on:** February 03, 2016, 01:15:18 AM »

Interesting, thanks!!

At this point, I would like to ask you something guys:

I read quite often in this forum (also here: http://www.magiclantern.fm/forum/index.php?topic=16542.msg161275#msg161275) and also have myself the problem, that even with global draw off, the maximum performance I can do with MLV's is 1920x490 on 50p. Everything above is making me ocasional (60p) and extremely often (GD or higher res.) pink frames which are ruining the clips. RAW Module on the other hand works flawlessly even with 67fps!

While I shoot slow motion most of the time and without sound anyway I don't need MLV, but there is a huge advantage - it can be opened by MLVFS for my so beloved super space-saving workflow. It really makes a difference, if you have a 1 Terabyte project going on and don't have to copy your footage to CDNG's before formatting to ProRes for example. I even tried renaming footage.RAW to footage.MLV and opening it with MLVFS hehehe Would it somehow be possible to do so

?

dmilligan · « **Reply #839 on:** February 03, 2016, 02:56:29 PM »

I won't support the RAW format as is. I don't believe it should be used in its current state for anything other than development purposes. However, MLVFS is open source, so you are free to modify it as you see fit and add support for whatever format yourself.

My goals and intentions for working on this project are probably at lot less pragmatic than most general users (who simply want something that works and gets the job done). My goal is to help drive the project forward: improving the quality of the code, increasing my understanding of ML and the internals of the underlying cameras (and helping others do the same). By not supporting the RAW format, I hope to encourage users not to take the easy way out, and actually try and solve the problem and move the project forward.

Faced with your problem, which of the two following paths to a solution do you think helps to move the project forward?
1. Just use the old raw_rec
2. Investigate and fix the pink frame issue with mlv_rec or propose better, alternate solutions

Since it does seem that there is a real resource constraint when using mlv_rec at high FPS or while using CPU intensive ML overlays, perhaps there is something that can be done without incurring any extra overhead that solves the following issues I have with the old RAW format:
1. The file extension should not be .RAW (this is way too generic, and users think they can open it with something like photoshop), it should be something unique
2. File header information and metadata should be stored at the beginning of the file, not at the end
3. There should be more metadata than is currently provided (at the very least, camera model, ML version, etc.)
4. There should be header information in every file if it is split into chunks

I see two possible solutions to address these issues:
1. Create a whole new file format
2. Create a special extension to the MLV format that allows us to write the extra metadata and whatnot we need at the beginning of the file (using existing defined MLV block structure), and then after that simply dump frames sequentially into the file like the raw format (maybe we need a different file extension if we do this?).

Danne · « **Reply #840 on:** February 03, 2016, 04:12:47 PM »

Quote

2. Create a special extension to the MLV format that allows us to write the extra metadata and whatnot we need at the beginning of the file (using existing defined MLV block structure), and then after that simply dump frames sequentially into the file like the raw format (maybe we need a different file extension if we do this?).

+1
And if this could be used straight in MLVFS it would be great.

dmilligan · « **Reply #841 on:** February 06, 2016, 05:13:09 AM »

Exciting news for Windows users: https://bitbucket.org/dmilligan/mlvfs/issues/7/dokany-support

We're also close to having the focus pixel issue solved. New builds soon.

PaulHarwood856 · « **Reply #842 on:** February 06, 2016, 06:31:07 AM »

Hello everyone,

I have had some issues today with handling my .mlv files with MLVFS. I am running OS X Fuse 2.8.3.

For some reason, After Effects CC 2015 (Version 13.7 - most recent) is not communicating well with my Macbook Pro, Late 2013, with a Nvidia GeForce GT 750M Graphics Card, and MLVFS.

I removed OS X Fuse in System Preferences, and reinstalled, but I still had the same issue.

(A few weeks ago I had downgraded from OS X El Capitan 10.11.3 to OS X Mavericks 10.9.5. The main reasons were because of many crashes, force quits, and Nvidia Cuda wasn't communicating with Premiere Pro CC 2015 when editing. - wasn't showing effects/dissolves, and I was forced to use OpenCL by OS X El Capitan. So I had downgraded to Mavericks since when I bought my Macbook Pro that was the operating system that was installed. Apparently the operating system the computer is initially installed with works the best, from what I've read.

So after still having After Effects freeze with MLVFS, I called Adobe, and they told me the recent After Effects updates require OS X Yosemite 10.10.5. (This is odd since last week they told me Mavericks 10.9.5 was the minimal required version.)

This afternoon I did a clone of my internal drive, made a bootable OS X Yosemite drive with Disk Maker X, did a clean install of 10.10.5, and installed my programs and files from the clone I made of my internal drive.

After all of this, I still had the same exact issue, now with OS X Yosemite 10.10.5.

So I called Adobe, and they told me I might need to convert the dng files to TIFs, and then import those. This completely disregards the point of MLVFS and an online directory's convenience. I have done this in the past, and it is quite time consuming, and takes up a lot of hard drive space.

I asked Adobe if a clean install of Adobe Creative Cloud and the programs would help. They said it was worth a try.

So, I uninstalled all Adobe applications and the Creative Cloud Desktop Manager. I couldn't get the desktop manager to reinstall, so I had to reinstall OS X Yosemite 10.10.5.

I now have all my Adobe programs working and Creative Cloud Desktop Manager as well. However, I am still having After Effects give the spinning beach ball with MLVFS.

The only other possible reasons I can think of that may have caused MLVFS to act odd are:

Using OnyX software to clean my computer, possibly deleting important files (prior to this morning).

The folder's name was changed and moving it back and forth between the internal drive and a G Drive Thunderbolt/USB 3.0, possibly confused files.

But what's weird is, MLVFS loads the files rather quickly. Google Chrome and Safari are pretty equal in speed, so using one over the other wasn't the issue. But I cannot copy the MLVFS files and put them on another hard drive now, which I could do before (I can but it says it will take 18 hours for 64 gb, which makes no sense).

I'm using Bad Pixel Fix at Aggressive, Vertical Lines Fix On, and Fix Noise Pattern On.

Bottom line, something's not right. If anyone has any input on what I can do to make MLVFS and After Effects run smoother I would greatly appreciate it. I apologize if what I typed is a bit confusing, I tried to make it as clear as possible. I just wanted to make sure I included everything I tried today in order to pinpoint what might be causing the problem. Thanks so much.

- Paul Harwood

dmilligan · « **Reply #843 on:** February 06, 2016, 01:49:34 PM »

Pattern noise fix is still somewhat experimental and VERY slow. The bad pixel fix will slow things down a little too.

Bad pixel fix and focus pixel fix are two different things. Focus pixel fix is pretty fast and is default to on for cameras that need it.

PaulHarwood856 · « **Reply #844 on:** February 06, 2016, 03:38:33 PM »

Hey dmilligan,

I changed the settings on MLVFS to just Vertical Stripes fix, and everything is so much faster now! Thank you so much, I'm so relieved!

- Paul Harwood

andy kh · « **Reply #845 on:** February 06, 2016, 11:28:58 PM »

very exciting as a window user

hjfilmspeed · « **Reply #846 on:** February 07, 2016, 01:41:58 AM »

I've been using on windows without a hitch. I had some issues with ACR but that's it. I have been mounting the MLVs right on the card, then compressing them with slimraw raw right from the mounted DNGs! So basically I now turn my transfer stage into a convert to DNG, compress, and transfer to hard drive all it one step! Saves soooooo much time!

dmilligan · « **Reply #847 on:** February 07, 2016, 04:38:55 AM »

The latest MLVFS is now available for Windows!

Thanks to g3gg0 and the dokany project, MLVFS now works on Windows just like the Mac/Linux FUSE version.

Download: MLVFS_x86.zip
You'll need to install Dokany

andy kh · « **Reply #848 on:** February 07, 2016, 05:21:03 AM »

thank you so much@dmilligan & g3ggo for this wonderful mlvfs. love it

Markus · « **Reply #849 on:** February 07, 2016, 08:32:29 AM »

Cool!

Need to check this out. Hot pixel fix plus pattern noise reduction avaliable for windows now?

News:

Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter (Read 959029 times)