MLVFS - Magic Lantern Video File System
What is it?
An MLV "converter" that provides a virtual file system of converted data 'on the fly' using F
ilesystem in Use
rspace (FUSE). It allows you to "mount" an MLV file(s) which shows up as a directory of converted CDNGs. The data for these CDNGs is provided 'on the fly' by MLFVS as the data is being requested by whatever raw editor or post processing software you are using.
It's fast and efficient, and I've managed to get real-time
playback in Premiere/SpeedGrade (on my 2012 MacBook Air!).
The code is written in C and works on Linux, Mac (via the OSX port of FUSE http://osxfuse.github.io/
), and Windows (via the dokany project: https://github.com/dokan-dev/dokany
thanks to g3gg0
- You don't have to convert first, you more or less "instantly" have "converted" DNGs
- You don't have to choose between keeping the original MLVs or converted DNGs or doubling disk usage to keep both.
- There's no need for a GUI, there already is one provided by the OS (i.e. the OS's file browser).
- Possible to vary the quality, pre-processing, bit depth, etc, of the DNGs on the fly
Working Proof of Concept => There's much more work to be done (see below), but there is a working proof of concept, and a Mac build is available.2014-09-08
Audio is working!2014-09-09
White balance mostly working2014-09-20
AE working! and filesystem is now read/write (files are stored in .MLD directory in the real filesystem)2014-09-26
Vertical banding correction2014-09-27
Bad pixel correction2014-10-09
Embedded mongoose webserver for configuring options at runtime2014-10-13
Full quality dual ISO working (cr2hdr-20bit)2014-10-30
Auto-linked audio in DaVinci Resolve2015-02-05
Compressed file support2015-02-17
Animated GIF Previews2016-02-05
Fix for Focus Pixels2016-02-07
Full Windows Support thanks to g3gg0
Double click the "MLVFS.workflow" file and you should see the following prompt:
Now, you can right click a folder (with MLV files in it) and go to "Services" > "MLVFS"
You will be prompted to select a mount directory (it must be empty)
To stop mlvfs, simply click the eject button that shows up next to the mount point
If you get a security error, you may need to change your security settings to allow unsigned/downloaded programs to runLinux
Install FUSE via the normal way for your distribution. It's probably called something like libfuse-dev
apt-get install libfuse-dev
Download the code with git and compile using makeWindows
Start it via the command line like this:
mlvfs.exe <mount point> --mlv_dir=<directory with MLV files>
Web based GUI
Known issues and things not yet implemented
Raw image pre-processing
hot/cold pixel correction
vertical banding correction
Support for compressed MLV files
- I'm sure there's more stuff...
Mac Download: CompressMLV.dmg
Install the same way as MLVFS.
Current Status: For Testing only!
Only MLVFS or the version of mlv_dump included inside the .workflow package contents will be able to decompress the results. The included version of mlv_dump includes various new features that have not yet been merged, and some custom tweaks by me. So please don't use this for real archival purposes.
The code is here
ML developers (esp. g3gg0
for designing the MLV format and providing reference converters)
Developers of open source MLV=>CDNG converters: chmee
=> your work and source code was a great help in understanding the CDNG format and requirements
Some helpful tutorials from @reddeercity and @DeafEyeJedi
So, what would be the 'best' or 'easiest' possible workflow for MLV? Native support for MLV format in whatever raw video processing tool you use (via open source code!), obviously.
There are ways to do this, many programs provide an API for third party plugins to implement something like this, but any implementation would be a lot of work and limited to working with one single app (as APIs are going to vary). Then I had the thought, what if we went down to a lower level of abstraction that all apps will have in common: the file system itself. So I started looking around the internet for an easy way to create virtual file systems, and found this awesome API that allows you to do just this in a user land program: FUSE
So the basic idea is that you can "mount" an MLV file(s). The converter program provides info and converted data to the OS "on demand" for this virtual file system. This is sort of like how you can mount a disk image or traverse into zip files in windows without actually extracting it.
Here's how it works: you just start the program, point it to some 'live' directory (where you can drop MLV files) and give it a mount point. The mount point directory then shows up as having folders for each MLV file containing CDNGs for each frame.
So I have started working on this and I have
some sort of VERY preliminary a proof of concept working. It is by no means usable yet, there are all kinds issues and things not implemented yet, but I can at least setup the file system and provide some semblance of the data that's going to be in the DNG (not readable yet, b/c I haven't created the DNG header with all the correct tags and such, and there are probably issues with my bit unpacking code) so that I can test speed and such. Basic DNG conversion is working and it seems to be pretty fast, so far it's not really noticeable that you're using a virtual file system.
If you're a coder and would like to help, that'd be fantastic, esp if you have a good knowledge of the DNG spec (I don't and I'm learning all of this as I go).