Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter (Read 960155 times)

dmilligan · « **Reply #1100 on:** March 03, 2016, 01:55:23 AM »

I intended to just have everything in the same directory to avoid cross platform discrepancies (like different path separators and whatnot). I realize it's confusing for those things to be in the "data" directory in the source tree. I'm open to suggestions.

QuickHitRecord · « **Reply #1101 on:** March 04, 2016, 09:39:08 AM »

After updating to the latest nightly build today, I also installed the newest version of MLVFS and decided to do another body cap ISO test (see the video description for more details):

I'm not convinced that changing the boxes in the web interface is doing anything. The clips with "Bad Pixel Fix" checked don't seem to be any different than those without it. What am I doing wrong? I restarted Resolve after making the changes to the web interface so that they would take.

DeafEyeJedi · « **Reply #1102 on:** March 04, 2016, 09:56:31 AM »

Quote from: QuickHitRecord on March 04, 2016, 09:39:08 AM

I'm not convinced that changing the boxes in the web interface is doing anything. The clips with "Bad Pixel Fix" checked don't seem to be any different than those without it. What am I doing wrong? I restarted Resolve after making the changes to the web interface so that they would take.

Maybe you forgot to 'refresh' the Web interface after making your adjustments checked? I've had this happened to me occasionally whenever I forget. [emoji6]

@dmilligan - I ran a few test MLV_Lite files with latest MLVFS (Thanks for the update) and it works like normal with these files.

However, when I try running a folder with regular MLV files which then shows me the web gui error. Strange?

Link to MLVFS Log ... https://mega.nz/#F!alU0DTJa!9Z-kho7OPZPg7msyltc9-g

*edit*

Ah, Ha ... Looks like I managed to figured it out on my end (not sure why this wasn't an issue in the past) but I decided to rename my Folders that didn't have any (spaces or ' , _ etc) and reran latest MLVFS with all of my old MLV files from varies of cameras and all seems to be working like normal again. Whew!

Sorry for the False alarm even tho I could swear that Ive used Spaces or Punctuation Marks with no problems (according to my memory) or no?

QuickHitRecord · « **Reply #1103 on:** March 04, 2016, 09:17:28 PM »

Quote from: DeafEyeJedi on March 04, 2016, 09:56:31 AM

Maybe you forgot to 'refresh' the Web interface after making your adjustments checked? I've had this happened to me occasionally whenever I forget. [emoji6]

You were right. Thanks for the tip. I re-did the test, and expanded on it:

Some takeaways from this exercise:

If I don't hit refresh on the web interface (even with none of the boxes checked), I seem to get crazy dancing bad pixels and just more noise in general.
The 5D Mark III seems to be a native ISO 400 camera. There is a lot of additional noise introduced at ISO 800.
Bad pixel fix works great. Aggressive bad pixel fix didn't seem to yield any different results. Maybe this is useful for the Rebel and EOS-M lines that seem to have more bad pixel issues.
Fix pattern noise helps a lot, and seems to eliminate vertical stripes as well.
Vertical stripe fix doesn't seem to do anything.
Chroma smoothing adds a lot of noise. I don't think that I will be using this.
RawMagic displays vertical stripes, even with vertical stripe correction on.
Footage from RawMagic is about a stop brighter by default.

dmilligan · « **Reply #1104 on:** March 04, 2016, 10:27:40 PM »

Quote from: DeafEyeJedi on March 04, 2016, 09:56:31 AM

Sorry for the False alarm even tho I could swear that Ive used Spaces or Punctuation Marks with no problems (according to my memory) or no?

Spaces, quotes, and all sorts of manner of strange characters work just fine in the file system. It appears that some characters cause files not to show up in the webgui, this is probably b/c the json/url requests aren't being escaped properly.

dmilligan · « **Reply #1105 on:** March 05, 2016, 02:02:52 AM »

Think it should be fixed now.

voblaunsane · « **Reply #1106 on:** March 05, 2016, 09:49:24 PM »

Hi all,

I love the concept, but I cant get Premiere CC to properly deal with the files. After importing the sequence, scrubbing works nowhere (media bowser/bins/timeline), hence I initialy thought it does not work at all, BUT it plays back with no problem after a pausing a moment. Mind you that I have no such problems with dng extracted using raw2cdng 1.7.4

Alas, the biggest issue remains rendering, where I get "error compiling movie. unknown error" and therefore being unable to export at all! Again, this does not occur with dng's extracted with raw2cdng 1.7.4.. Here's what I get in console after export attempt:

Code: [Select]

H:\DCIM\100EOS5D/M04-1511.MLV: Error reading frame headers: vidf block for frame 0 was not found
H:\DCIM\100EOS5D/M04-1511.MLV: Error reading frame headers: vidf block for frame 0 was not found
H:\DCIM\100EOS5D/M04-1511.MLV: Error reading frame headers: vidf block for frame 0 was not found
H:\DCIM\100EOS5D/M04-1511.MLV: Error reading frame headers: vidf block for frame 0 was not found
mlvfs_wrap_read(1497): caught exception 0xC0000005 at address 0x7061D72E

I'm running this on Win7 SP1

Context Menu for Win

I would also like to make a small contribution to windows users adding a context menu option "Mount with MLVFS" to your folders. Technically registry entry only should suffice, but I found that webGUI loads blank and console outputs fopen errors without setting command path to mlvfs folder of residence before executing, hence the bat file..
Obviously, you'll have to change paths to suit your requirements.

mlvfs.reg (execute this file to add context menu option for your folders):

Code: [Select]

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\MLVFS]
@="Mount with MLVFS"

[HKEY_CLASSES_ROOT\Directory\shell\MLVFS\command]
@="\"C:\\Program Files (x86)\\MLVFS_x86\\mlvfs.bat\" \"%1\""

mlvfs.bat (place this next to mlvfs.exe)

Code: [Select]

@echo off
c:
cd "c:\Program Files (x86)\MLVFS_x86"
mlvfs.exe x:\ --mlv_dir=%1

bouncyball · « **Reply #1107 on:** March 06, 2016, 02:52:20 PM »

Thanks. Context menu is nice.

I've tried this method some time ago and mlvfs path was a problem for webgui as you described and I ended up with .bat command file as you did

. Besides I always run mlvfs with admin rights b/c dokan sometimes could not properly unmount the drive. I've tested with mlvfs in compatibility mode as admin (opens 2 console windows

if executed from bat) and with "runas.exe /user:Administrator ..." command and it sorta worked the way I'd like but it complicated things a bit (enter password etc) and I gave up and just use the shortcut with "run as administrator" checked on or just run it from FarManager console session.

I think the best way to solve this problem for all platforms is to hardcode the html and js resource into mlvfs binary itself (early versions were this way if I'm not mistaken). However, I guess, David had his own reasons for current implementation.

bb

dmilligan · « **Reply #1108 on:** March 06, 2016, 07:02:50 PM »

Constantly recompiling to fix html, css, and ajax is annoying. Writing all the html as string literals is also quite cumbersome, and I don't know of a good, simple, reliable, cross-platform way of inserting text documents into binaries. Some particular method might work well for gcc/make but not LLVM or VS and vice versa. Much simpler to just load them. With the whole focus pixel thing we have the need for additional resource files as well, so it was starting to get a little out of hand to keep everything in the binary. Plus it means if/when new focus pixel cameras are added, pixel maps can be added without recompiling, and people can make their own maps to fix things like troublesome hot/dead pixels.

g3gg0 · « **Reply #1109 on:** March 06, 2016, 07:53:32 PM »

the only proper solution is making some kind of "resource" (i.e. zipping all files) that is either linked (depends on which linker you use) or converted to a .c with some "char resourceData[] = { 0xaa, 0xbb, ... }".
but thats a bit annoying as you need additional helpers.

voblaunsane · « **Reply #1110 on:** March 07, 2016, 03:37:00 PM »

any idea about the unhandheld (haha, sorry/not sorry) exceptions and errors reading frame headers, which I am guessing are the causes of problematic handling by premiere?...

I've described those few posts above..
http://www.magiclantern.fm/forum/index.php?topic=13152.msg163728#msg163728

Please let me know if there is any troubleshooting or testing I could do in helping to improve it!

Thanks.

Roberto Mena · « **Reply #1111 on:** March 08, 2016, 10:34:45 AM »

Hi Guys. Thanks for all the amazing work. I downloaded FUSE and MLVFS but I am not getting the "Services" option when I right click a folder containing mlv files, showing at the bottom of the window near the "Tags" section. What am I doing wrong? Thanks.

bouncyball · « **Reply #1112 on:** March 08, 2016, 01:45:49 PM »

Quote from: voblaunsane on March 07, 2016, 03:37:00 PM

any idea about the unhandheld (haha, sorry/not sorry) exceptions and errors reading frame headers, which I am guessing are the causes of problematic handling by premiere?...

I never experienced frame header error with premiere or anything else. You should double check that particular mlv file in other tools.

mlvfs_wrap_read(1497): caught exception 0xC0000005 - was very common (in early windows mlvfs builds) until I disabled antivirus real time protection.

bb

g3gg0 · « **Reply #1113 on:** March 09, 2016, 12:55:14 AM »

Quote from: voblaunsane on March 07, 2016, 03:37:00 PM

any idea about the unhandheld (haha, sorry/not sorry) exceptions and errors reading frame headers, which I am guessing are the causes of problematic handling by premiere?...

can you check if the file with vidf errors reads fine using mlv_dump?
can you run mlvfs.exe with parameter -d and upload the whole log? (append "> log.txt" to log into a file)

USADavid · « **Reply #1114 on:** March 10, 2016, 01:29:38 AM »

Hey guys,

long time creep, but never had a reason to post anything. Great work everyone is doing here. Love the simplicity and cross-platform capability of MLVS.
Unfortunately, I've had issues with the sound for a while now, blamed it on me first, but after trying a lot of different things never seem to be able to fix it.

I'm running the latest MLVFS on my Mac (2016-03-04) and all my MLVs which are around > 2GB have an empty wav file. The wav files still show up even with the alleged size, but there's no sound. If I open up the MLVs in MlRawViewer, it plays just fine. I'm on the latest Fuse and even tried the Beta, also the ~./mlvfs.log file doesn't offer any clues on why this occurs.

Any help would be greatly appreciated.

voblaunsane · « **Reply #1115 on:** March 10, 2016, 05:48:46 AM »

Quote from: g3gg0 on March 09, 2016, 12:55:14 AM

can you check if the file with vidf errors reads fine using mlv_dump?
can you run mlvfs.exe with parameter -d and upload the whole log? (append "> log.txt" to log into a file)

Thanks!

I tried all combinations of commands I could've possibly ran, alas was unable to reproduce, very strange because I received those all first day of use and then next day it was gone like nothing ever happened! Also deleted the files before reading you post, aww..

Yet main issue being unable to render persists! So I ran mlvfs with -d 2>log.txt mounting a folder with one sequence (3 files mlv/m00/idx), created new premiere project, located sequence in media browser, dropped it straight on the timeline and hit render. This produced "error compiling movie. unknown error" and 54.6MB logfile here: https://drive.google.com/file/d/0Bzfm11RAqkBzaXgzOTUwQ1dIYUU/view?usp=sharing. Not knowing what I'm looking for tried to do some detective work and the following seems to be all over the place:

Code: [Select]

CreateFile status = 0 - lastError = 2
This mlv was recorded with no sound, but I see the following in report alot.

Code: [Select]

CreateFile : \M04-1513.MLV\M04-1513.wav
I left an instance of mlvfs with -d running while typing this post, new log file is nearly 2GB - I'm scared

g3gg0 · « **Reply #1116 on:** March 10, 2016, 08:19:15 PM »

if you extract the files using mlvfs and let premiere work on the extracted files, does that work?

voblaunsane · « **Reply #1117 on:** March 10, 2016, 08:44:05 PM »

Found a few things here:

1. Finally I managed to render mlvfs on premiere! I found that 25fps material works flawlessly at a lighting speed, when 23.976fps is impossible to scrub and it doesn't render "error compiling movie. unknown error"... when that same 23.976fps mlv, extracted with raw2cdng works just fine. Any ideas?

I logged the activity with separately mounted differing fps files here: https://drive.google.com/file/d/0Bzfm11RAqkBzN0dnV05kMGpmSms/view?usp=sharing

2. Mounting a full CF results in empty mlv folders. localhost:8000 reports 0 frames in files found. Freeing some space resolves the issue. It could be nice have mlvfs reporting "not enough disk space".. Otherwise it gives the impression that files are damaged..

Thanks!

voblaunsane · « **Reply #1118 on:** March 10, 2016, 08:49:47 PM »

Quote from: g3gg0 on March 10, 2016, 08:19:15 PM

if you extract the files using mlvfs and let premiere work on the extracted files, does that work?

no. same issues with 23.976. Works fine, same as on the fly with 25fps..

Using mounted 23.976 material renders always fail at a particular stage, until I drag the clip to another sequence, then it gets a new fail point. More often then not fail point is 99%.. The extract from mlvfs, even in new sequence, always fails at the similar point.. but same material extracted again, gets a new fail point .. makes sense?

Also caught the exception, actually all 6 of them, but was running many tests and can not reproduce, here's the log: https://drive.google.com/file/d/0Bzfm11RAqkBzb0xwbHpTT3JqSEE/view?usp=sharing

Markus · « **Reply #1119 on:** March 12, 2016, 07:00:53 PM »

I've installed dokan to try it out on win10 64bit but can't get it to function properly. I can mount the files and se the dng-files inside but if i try to open anything i just get: "load_chunks: fopen error: No such file or directory"

Anyone have any idea of what could be the problem?

v. 0.8.0 of dokan and MLVFSx86 from first post link

Ottoga · « **Reply #1120 on:** March 13, 2016, 05:35:33 AM »

@Markus

First try using the latest supported versions of Dokan and MLVFS.

Links to these can be found here: http://www.magiclantern.fm/forum/index.php?topic=13152.msg163041#msg163041

Note that you will need to uninstall Dokan first before reinstalling the latest version. The MLVFS download totally replaces your existing install instance.

Also, do the paths to your MLV files contain spaces? If so, create a path without spaces and try again.

If it still fails after that, report back with some screen shots demonstrating successful Loading of Dokan, File mounts (A localhost one) and the error message when trying to access a specific DNG.

What application are you using to access the DNG's. The issue may be with the application and not MLVFS. what happens if you just double click on a DNG to view the image with windows default viewer.

@DMilligan - Might be worthwhile to update the links in the 1st post to reflect the latest supported versions.

Ottoga · « **Reply #1121 on:** March 13, 2016, 05:50:19 AM »

@Developers

Just a note to advise that Dokan v1.0.0-RC2 is available and, at least on my W10 system is working error free. Tested with single and multi-chunk MLV's.

bouncyball · « **Reply #1122 on:** March 13, 2016, 12:04:04 PM »

@Ottoga: Thanks for pointing it out!

@g3gg0, dmilligan: delete/copy file/dir in passthrough mode is OK! for RC2

Markus · « **Reply #1123 on:** March 14, 2016, 12:17:22 PM »

@Ottoga Thanks for reply! I'll try it out as soon as possible. Excited of getting this to work instead of pismo. Fix pattern noise seems extremely useful.
Does anyone know if it is possible to process dng - sequences for fixed pattern noise if you haven't saved the mlv's?

Ottoga · « **Reply #1124 on:** March 14, 2016, 12:30:03 PM »

@Markus

Quote

Does anyone know if it is possible to process dng - sequences for fixed pattern noise if you haven't saved the mlv's?

From memory, you might be able to use mlv_dump to recreate an MLV file from a DNG series. One of the Devs may be able to confirm that.

If this is confirmed, then the answer to your question is likely to be "yes".

Oh, I also uninstalled Pismo and rebooted my PC before installing Dokan. I didn't see any point having two services running in the background that are essentially trying to do the same thing. Just something else to consider.

News:

Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter (Read 960155 times)