[deprecated] MLVFS windows client

[g3gg0]

Update 06.02.2016:

Please use MLVFS FUSE driver linked below, it will soon have builds for win32 that use dokany VFS driver.
dokany is open source, actively developed and allows to use FUSE drivers without API change.
(see: https://github.com/dokan-dev/dokany )

as long there is no official release, you can use this build after you installed dokany.

BR,
g3gg0

---


Hello,

inspired from the brilliant idea of the FUSE MLVFS driver from dmilligan and ayshih, that allows you to mount MLV files as directories,
i wondered how to make their code/idea available to us windows users too.
The underlying system they use, called FUSE, is a file system extender for unix-like systems and wraps normal file calls so that an application can do arbritraty stuff with it.
In the case of MLVFS they simulate that MLV files are directories that contain DNGs. If you read the DNG from that virtual directory, they create it on-the-fly.
Unfortunately there is no real alternative for windows users to load the MLVFS daemon that is designed for FUSE.

But i found a simple way to give windows users the same experience as unix users get with MLVFS.
Back in my symbian OS days, i created a WebDAV server that allows symbian phones to mount directories on your windows computer. (see my old site)
So i could use my symbian phone to browse directories on my computer at home (MP3s and such).

I've added MLV support and browsing it as virtual folders! Should work from WinXP up to Win8.

Supported (generated) file types:
 - 16 bit DNG
 - JPEG for previews
 - WAV in case of audio-MLV
 - RAW FITS with metadata for astral photography (monochrome raw bayer mode) for e.g. DeepSkyStacker
 - a text file containing all important metadata in human/script readable form

Features:
 - you can select your MLV folder on HDD or memory card and browse it just like a normal directory - as soon there is a MLV, its simulated as directory
 - any write access is redirected into a separate subfolder (<mlv_filename>.MLD), just like with the original FUSE driver
 - overwriting and modifying the virtual files also possible - files get copied into virtual folder then
 - deleting all files in the virtual folder will remove only the files in the .MLD subdirectory, so you will have a clean MLV again
 - MLV files are *never* modified when doing stuff with files in the directory, except you delete the directory from its parent folder
 - you can enable/disable any file type separately

Caching:
 - it is currently disabled, due to memory issues
   (i cannot catch out of memory exception properly, as it may happen anywhere)



Select a drive letter that is free and press "Map", it will connect the share to a network drive.
You can also do it manually using the shell by typing:
net use x: \\127.0.0.1@8080 (change letter and port accordingly)

If you close the window, it will minimize into systray, showing a star icon and run in background.

It also supports that windows computers mount the shares as a local network drive.
So i extended this tool to act a bit more responsive and added MLV support using MLVViewSharp and the DNG code from dmilligans MLVFS daemon.
Now you can browse the MLV files as they were directories, showing you the frames as DNGs, JPGs for preview and WAV if it contains audio.



You also can save files and folders "into" that MLV file. all files get redirected into a separate directory named like the MLV file itself, with an extra "_store" suffix


Just like with the unix-version, you can use (hopefully) all your tools with that mapped network drive.
For instance here i import the DNG frames using LightRoom:




Running WebDAVServer as Windows Service:
You can install the tool as a Windows Service which will automatically start on system boot.
To do this, first start the tool as Administrator (rightclick -&amp;gt; Run as Administrator).
First set up all options like Path, Port and Auth – dont forget to press the “Write” button to save a default config.
Now you have written a default config that is always loaded whenever the server starts (both as service and as normal app)
To install the service, simply press “Install”.

If this was successful, the “Install” button goes inactive and the “Uninstall” button activates.
The buttons “Start” and “Stop” are for starting and stopping the service.

Since the service has no GUI, sometimes it makes sense to stop the service and use the normal mode instead.

Web Browser Access:
You can access the server with your web browser and browse the contents of your share as the phone would see it.
There are some debug and log views too (check the links on top). If authentication is required (username/pass) the log/debug view is crippled to prevent abuse.
Accessing MLV content using the web browser is not implemented yet. Anyone who needs it?

Download:
Download the current version of the "MLV WebDAV Server" here
Download the source code on bitbucket

Important Hints:
Windows is per default very sluggish when accessing the WebDAV shares.
Please disable "Automatic proxy detection" in your internet explorer, like microsoft suggests here
If you dont do that, accessing the mounted drive is very slow. Its a problem with windows itself.
And yes, its important also for chrome and firefox users

If you want to use authentication on Windows Vista and above, you have to apply a registry patch that enables user/pass authentication.
Please install the fix from the microsoft article here.

If you get DLL errors, you might have to install the MSVCRT runtime libraries from here


Legals:
This program is licensed under the GPL v2 license.
This code contains GPL code from MLVFS, a GPLed FUSE library for accessing MLV files.
To be specific, the whole RAW-&amp;gt;DNG code was taken from there.

[BartvR]

Wow really cool, I'll check it out!

[Canon eos m]

Hope i can make this work. last time danne had to run a special tut for me to make me going  i am technology challenged

[g3gg0]

quickstart:
 - start the exe
 - click "Setup" tab
 - click "Path" button
 - select your card drive
 - press "Write" button
 - press "Map" button
 - now you have a drive letter associated

you can close the window, it is running in background (systray, star icon).
rightclick star icon and press close to exit the tool

if you want it to autostart on reboot
 - close the tool
 - run it "as Administrator"
 - press "Install"
 - exit the tool (systray, rightclick, quit)
 - now it will start on every reboot with the same settings

disable autostart on reboot
 - close the tool
 - run it "as Administrator"
 - press "Uninstall"
 - exit the tool (systray, rightclick, quit)
 - now its gone

When running as administrator, pressing "Map" will hang the tool, so dont do it.
Some Windows issue, didnt check yet what the reason is.

[surami]

I just tested as you described at the quickstart, I can't open the DNGs:
https://www.dropbox.com/s/8ddsb0cxdejjcbi/mlv_webdavserver_error_01.jpg?dl=0

What's wrong?

SOLVED, read the reply from g3gg0, #14.

[Canon eos m]

Awesome. Got it working. First try failed since for the path I chose G:/ (my card drive) and came back with an error. Then chose the actual MLV file and wow!

[g3gg0]

Surami, can you upload such dng file?

Canon eos m, you should also be able to choose your memory card's root folder.
Which error did you get?

[surami]

Surami, can you upload such dng file?

If I copy a DNG from the mounted "drive", its filesize will be 0 byte, so there is nothing inside.
https://www.dropbox.com/s/ol2r5lpmzxxqb02/M18-1424_000000.dng?dl=0

Now I'm uploading the source MLV file.

SOLVED, read the reply from g3gg0, #14.

[g3gg0]

redownload the server please and try again.

[surami]

With the new build I see only empty MLV directories and the IDX files + at the WebDAVServer Log messages there are much from this kind of lines:
'System.DllNotFoundException' in connection from 127.0.0.1:50279 (HandleContent/HandleRequest)
Connection from 127.0.0.1:50280 (PROPFIND)

Here is the mentioned MLV:
https://www.dropbox.com/s/latxxeebzzufgr4/M18-1424.MLV?dl=0

SOLVED, read the reply from g3gg0, #14.

[g3gg0]

Did you extract all the files from the zip?
It looks like you extracted only the exe..

[surami]

Yes, I downloaded the new build from here and overwrote the files.

SOLVED, read the reply from g3gg0, #14.

[g3gg0]

Where did you save it to?
How did you start it?
Which windows version?

[surami]

Where did you save it to?

I saved it to another HDD (D:\), not on the same where my MLV files are (E:\)

How did you start it?

Just double clicked on the WebDAVServer.exe. (I have administrator rights)

Which windows version?

Windows 7 (64 bit)

Makes the WebDAVServer other files/modifications outside the (eg. D:\WebDAVServer) directory? I mean WebDAVServer settings etc. not the MLV folders, IDXs.
May I delet or clear something somewhere on the system?

I found the cache files, they are at this location: %systemdrive%\windows\ServiceProfiles\LocalService\AppData\Local\Temp\TfsStore (source)

SOLVED, read the reply from g3gg0, #14.

[g3gg0]

1) redownload again, the DLL shoould be from 22:05 and try again

2) if the problem persists, try installing the MSVCRT runtime from here


WebDAVServer only writes a WebDAVServer.cfg file with its settings in its startup directory.
Please start it from there using a doubleclick, if this was the problem (should not be)
it does not require any write permission - only if you want to save files on your shared folders.

[surami]

Many thanks g3gg0 and of course thanks for dmilligan, ayshih, a1ex, chmee, baldand and other developers too.

I got proper DNGs on the virtual drive with the new build (22:05) + installed MSVCRT runtime '13 + after manually cleaning the cache files, what were generated by the older build.

I can't open the sequence directly with PP CS6, but dynamic linked through AE CS6 it works.
Resolve 11 Lite crashes the WebDAVServer.

[g3gg0]

i've optimized the DNG bitshifting code, its a bit faster. redownload please.
i also added SSE2 optimized libraries, just copy the one from the "SSE2-Library" subdirectory into the main dir.
yes, overwrite the existing one, you can copy it from the "Standard-Library" directory later again if you need.

the SSE2 one requires about 5 ms per image, where the standard unoptimized one took 17 ms.
but all the network stuff still takes ~45 ms, will check what i can optimize there.

@surami:
why does it crash?
any more information?

you can use your browser and navigate to http://127.0.0.1:8080/log (replace with your port)
there you see "Request messages" and "[Start]" link below. dont click it yet.

1) start your tool that crashes
2) do everything so its just a buttonpress away from crashing
3) press the [Start] link in your browser
4) make the tool crash
5) press F5 in browser
6) copy all messages to pastebin.com or anything like that

[surami]

I used now the SSE2 optimized version, it's a bit really faster, but this crashes too in Resolve Lite.

why does it crash?
any more information?

I mounted 6 MLVs (2-3 seconds long shots, each around 150 MB) to the virtual drive.
Started Resolve Lite and on the Library tab I started to play them back one after another (the playback isn't realtime, it's choppy).
The first was ok, the second too, but at the third came the window, that WebDAVServer crashed (.

I could log the things till the crash, but after that I can't refresh it (because the server stopped).
So I can't get the information from the log to send for you.

If you can implement such kind of thing, which saves the log in txt file, when the server stops, then I can send something.

[g3gg0]

ah okay misunderstood you. thought your resolve crashes.
thats a different story, my wrong.

what error message? just exiting? screenshot?
does it only happen with the *same* file, or ist it "the third one" being played?

redownload and repeat my instructions.
if it suddenly crashes, check for a "WebDAVServer.log" and upload it.

if it doesnt crash, check the log messages for exceptions.

[Canon eos m]

Surami, can you upload such dng file?

Canon eos m, you should also be able to choose your memory card's root folder.
Which error did you get?

Hi g3gg0 sorry for not responding earlier. My problem was solved following your instructions - as always clear 

What I meant (sorry for not being clear) is that when you sent me the instruction to link to the path, I kept trying to link to the root but soon figured out to link to one of the MLV files in the target sub folder. In fact that is why I qualified my post by saying 'awesome' and 'wow' 

[Canon eos m]

Will redo load and try now. I already feel like an expert on this to topic!

[g3gg0]

So you can choose e.g. G:\ as folder in WebDAVServer and it works?
Thats how I use it.

[surami]

redownload and repeat my instructions.
if it suddenly crashes, check for a "WebDAVServer.log" and upload it.

if it doesnt crash, check the log messages for exceptions.

It doesn't crashes now (SSE2 version), there aren't any exceptions at the log tab of WebDAVServer and there isn't any "WebDAVServer.log" file generated, so I think it works well now with Resolve 11 Lite. I didn't made the test in PP CS6 dynamic linked with AE CS6.

Another 3 things what I discovered:
1. I constantly watched the cache folder (C:\Windows\ServiceProfiles\LocalService\AppData\Local\Temp\TfsStore\Tfs_DAV) at playbacks and the stange thing is here:
https://www.dropbox.com/s/18s62rphzej0e3x/mlv_webdavserver_error_02.jpg?dl=0

As you can see, on the left side (in cache directory) the ending numbers of DNGs aren't correct, they are 7 digit long, there is an extra "0" at the end + there are some uncorrectly generated DNGs at the end of the directory. Right side is the mounted server (virtual drive).

2. Previously I mentioned, that In Resolve 11 Lite the playback isn't smooth. But now I discovered, that it plays back smoothly if the actual file in the middle box is played and on the left box an another directory is selected.
https://www.dropbox.com/s/vojt2rqxwc5ek1x/mlv_webdavserver_error_03.jpg?dl=0

[Canon eos m]

So you can choose e.g. G:\ as folder in WebDAVServer and it works?
Thats how I use it.

No, if I choose plain G:/ for example it doesn't work gives an error. Will try to replicate when I am home. Worked for sure when I chose one of the MLV files in the sub folder on the card and then all filed were converted.

[g3gg0]

@surami:
1. i dont have to do anything with the cache folder. its windows-internal.

2. interesting. is this repeatable?

@Canon eos m:
screenshot please, there should be no error. of course you have to select a valid drive letter..