Magic Lantern Forum

Using Magic Lantern => Raw Video => Raw Video Postprocessing => Topic started by: dmilligan on August 31, 2014, 02:01:24 AM

Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 31, 2014, 02:01:24 AM
MLVFS - Magic Lantern Video File System



What is it?
An MLV "converter" that provides a virtual file system of converted data 'on the fly' using Filesystem in Userspace (FUSE). It allows you to "mount" an MLV file(s) which shows up as a directory of converted CDNGs. The data for these CDNGs is provided 'on the fly' by MLFVS as the data is being requested by whatever raw editor or post processing software you are using.

It's fast and efficient, and I've managed to get real-time playback in Premiere/SpeedGrade (on my 2012 MacBook Air!).

The code is written in C and works on Linux, Mac (via the OSX port of FUSE http://osxfuse.github.io/ ), and Windows (via the dokany project: https://github.com/dokan-dev/dokany thanks to g3gg0)




Advantages
- You don't have to convert first, you more or less "instantly" have "converted" DNGs
- You don't have to choose between keeping the original MLVs or converted DNGs or doubling disk usage to keep both.
- There's no need for a GUI, there already is one provided by the OS (i.e. the OS's file browser).
- Possible to vary the quality, pre-processing, bit depth, etc, of the DNGs on the fly




Updates
2014-09-02 Working Proof of Concept => There's much more work to be done (see below), but there is a working proof of concept, and a Mac build is available.
2014-09-08 Audio is working!
2014-09-09 Mac installer/service
2014-09-19 White balance mostly working
2014-09-20 AE working! and filesystem is now read/write (files are stored in .MLD directory in the real filesystem)
2014-09-26 Vertical banding correction
2014-09-27 Chroma smoothing
2014-10-08 Bad pixel correction
2014-10-09 Embedded mongoose webserver for configuring options at runtime
2014-10-13 Full quality dual ISO working (cr2hdr-20bit)
2014-10-30 Auto-linked audio in DaVinci Resolve
2015-02-05 Compressed file support
2015-02-17 Animated GIF Previews
2016-02-05 Fix for Focus Pixels
2016-02-07 Full Windows Support thanks to g3gg0 (via dokany)
2017-05-05 Lossless JPEG Support thanks to bouncyball



Source Code
https://bitbucket.org/dmilligan/mlvfs




Download
Mac
Install OSXFUSE (http://osxfuse.github.io/)
Download MLVFS.dmg (https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS.dmg)
Double click the "MLVFS.workflow" file and you should see the following prompt:

(https://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs_service_install.png)

(Click Install)

Now, you can right click a folder (with MLV files in it) and go to "Services" > "MLVFS"

(https://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs_service.png)

You will be prompted to select a mount directory (it must be empty)
To stop mlvfs, simply click the eject button that shows up next to the mount point

If you get a security error, you may need to change your security settings to allow unsigned/downloaded programs to run (http://stackoverflow.com/questions/19551298/app-cant-be-opened-because-it-is-from-an-unidentified-developer)

Linux
Install FUSE via the normal way for your distribution. It's probably called something like libfuse-dev

apt-get install libfuse-dev

Download the code with git and compile using make

Windows
Install Dokany (https://github.com/dokan-dev/dokany/releases/download/v0.8.0/DokanInstall_0.8.0.exe)
Download MLVFS_x86 (https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS_x86.zip)

Usage
Start it via the command line like this:

cd <mlvfs_exe_dir>
mlvfs.exe <mount point> --mlv_dir=<directory with MLV files>





Web based GUI
(http://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs_gui.png)




Known issues and things not yet implemented
- Audio
- Raw image pre-processing
    - hot/cold pixel correction
    - vertical banding correction
    - chroma-smoothing
- Support for compressed MLV files
- I'm sure there's more stuff...




Contributors
ayshih
dmilligan
g3gg0 (Windows port)

Credits
ML developers (esp. g3gg0 and a1ex for designing the MLV format and providing reference converters)
Developers of open source MLV=>CDNG converters: chmee and baldand => your work and source code was a great help in understanding the CDNG format and requirements




Some helpful tutorials from @reddeercity and @DeafEyeJedi



https://vimeo.com/168433918




Original Post
Quote
So, what would be the 'best' or 'easiest' possible workflow for MLV? Native support for MLV format in whatever raw video processing tool you use (via open source code!), obviously.

There are ways to do this, many programs provide an API for third party plugins to implement something like this, but any implementation would be a lot of work and limited to working with one single app (as APIs are going to vary). Then I had the thought, what if we went down to a lower level of abstraction that all apps will have in common: the file system itself. So I started looking around the internet for an easy way to create virtual file systems, and found this awesome API that allows you to do just this in a user land program: FUSE (http://fuse.sourceforge.net/)

So the basic idea is that you can "mount" an MLV file(s). The converter program provides info and converted data to the OS "on demand" for this virtual file system. This is sort of like how you can mount a disk image or traverse into zip files in windows without actually extracting it.

Here's how it works: you just start the program, point it to some 'live' directory (where you can drop MLV files) and give it a mount point. The mount point directory then shows up as having folders for each MLV file containing CDNGs for each frame.

So I have started working on this and I have some sort of VERY preliminary a proof of concept working. It is by no means usable yet, there are all kinds issues and things not implemented yet, but I can at least setup the file system and provide some semblance of the data that's going to be in the DNG (not readable yet, b/c I haven't created the DNG header with all the correct tags and such, and there are probably issues with my bit unpacking code) so that I can test speed and such. Basic DNG conversion is working and it seems to be pretty fast, so far it's not really noticeable that you're using a virtual file system.

If you're a coder and would like to help, that'd be fantastic, esp if you have a good knowledge of the DNG spec (I don't and I'm learning all of this as I go).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: Audionut on August 31, 2014, 02:16:57 AM
Looking forward to see where this ends up.  Nice work dmilligan.

Something useful for windows here?  http://sourceforge.net/p/fuse/wiki/OperatingSystems/#windows
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: ayshih on August 31, 2014, 05:04:54 AM
It sounds like a promising approach!

I'm interested in helping, but it'd be nice if there was a way for it to work on Windows.  Unfortunately, it looks like the only actively developed implementation may be DokanX (https://github.com/BenjaminKim/dokanx/), and I don't think it uses the FUSE API.  (See the list of relevant projects on this Wikipedia page (http://en.wikipedia.org/wiki/Filesystem_in_Userspace).)  But, I do have an OS X machine and Linux VMs, so it's not an non-starter.

I've only started reading up on this stuff, but I'll point out that OSXFUSE says that it provides an API that is a superset of the FUSE API, so you may want to make sure you're using just the FUSE API to ensure Linux portability.  Of course, at such an early stage, that's a very low-priority concern.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: dmilligan on August 31, 2014, 05:44:27 AM
Great! I'm pretty sure I've only used the basic FUSE API (I've only included fuse.h, which I assume is the basic API), and I only use basic C libs and general posix stuff (no apple specific libs), so it should be totally portable to linux. Most of the FUSE related stuff is pretty well implemented (we could do more advanced things later on with it), the real areas that needs help is:

Bit unpacking - I didn't want to use the bit fields struct stuff like ML and mlv_dump use, since this is non-standard compiler stuff (and not supported by Apple's compiler), so I wrote a more generic one, but IDK if it's really correct or efficient

DNG generation - I didn't want to use chdk-dng.c b/c that code is an absolute mess, also I need to write specific sections of the file to buffers on demand, not the entire file at once (that's the way the FUSE api works, a specific section of the file is requested, and you write that section of the file into a buffer). Also, it's not thread safe (nor is the vast majority of stuff from mlv_dump), which this needs to be since the FUSE API is multithreaded and requests for data can be concurrent (multiple programs accessing different or even the same file at the same time, etc.).

there are also areas of mlv parsing that will need to be improved: there's no support for chunks yet, it should generate and use index files to make finding the frame blocks faster, no audio support, etc.

If you have any questions about the code I've written so far, I'll be happy answer them.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: chmee on August 31, 2014, 07:38:38 AM
@dmilligan. good idea.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: surami on August 31, 2014, 08:04:48 PM
@dmilligan: awsome thought.

I know it's too early to write about it, but I think it would be also nice to add and option to apply DNG Camera Profiles (.dcp files) too. I mean the index file should contain the information, that during the virtual file system upbuilding which .dcp file should be applied to the DNGs. There is a utility, the dcpTool (http://dcptool.sourceforge.net/Introduction.html) which could be a good starting point if a xml file is needed for this.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: dmilligan on September 02, 2014, 11:08:46 PM
okay, proof of concept is basically working, big thanks to ayshih who has done a lot of work (and fixed my bugs!)

ACR and Resolve can both correctly read converted DNGs. Resolve playback is slow, but it's slow on my system anyway, I think it's the GPU. Hopefully I can get Premiere/SpeedGrade working, I think it's just a matter of getting all the right DNG tags, any thoughts @chmee?


david$ ./dng_validate -v /Users/david/mlvfs/M23-1157.MLV/00000000.DNG
Validating "/Users/david/mlvfs/M23-1157.MLV/00000000.DNG"...

Uses little-endian byte order
Magic number = 42

IFD 0: Offset = 8, Entries = 33

NewSubFileType: Main Image
ImageWidth: 1728
ImageLength: 972
BitsPerSample: 16
Compression: Uncompressed
PhotometricInterpretation: CFA
FillOrder: 1
Make: "Canon"
Model: "Canon EOS 60D"
StripOffsets: Offset = 65536
Orientation: 1 - 0th row is top, 0th column is left
SamplesPerPixel: 1
RowsPerStrip: 972
StripByteCounts: Count = 16896
PlanarConfiguration: 1
Software: "MLVFS"
*** Warning: IFD 0 DateTime has unexpected count (1) ***
*DateTime: ""
CFARepeatPatternDim: Rows = 2, Cols = 2
CFAPattern:
    Red      Green
    Green    Blue
ExifIFD: 410
DNGVersion: 1.4.0.0
UniqueCameraModel: "Canon EOS 60D"
LinearizationTable: 16383 entries
    Table [0] = 0
    Table [1] = 1
    Table [2] = 2
    Table [3] = 3
    Table [4] = 4
    Table [5] = 5
    Table [6] = 6
    Table [7] = 7
    Table [8] = 8
    Table [9] = 9
    Table [10] = 10
    Table [11] = 11
    Table [12] = 12
    Table [13] = 13
    Table [14] = 14
    Table [15] = 15
    Table [16] = 16
    Table [17] = 17
    Table [18] = 18
    Table [19] = 19
    Table [20] = 20
    Table [21] = 21
    Table [22] = 22
    Table [23] = 23
    Table [24] = 24
    Table [25] = 25
    Table [26] = 26
    Table [27] = 27
    Table [28] = 28
    Table [29] = 29
    Table [30] = 30
    Table [31] = 31
    Table [32] = 32
    Table [33] = 33
    Table [34] = 34
    Table [35] = 35
    Table [36] = 36
    Table [37] = 37
    Table [38] = 38
    Table [39] = 39
    Table [40] = 40
    Table [41] = 41
    Table [42] = 42
    Table [43] = 43
    Table [44] = 44
    Table [45] = 45
    Table [46] = 46
    Table [47] = 47
    Table [48] = 48
    Table [49] = 49
    Table [50] = 50
    Table [51] = 51
    Table [52] = 52
    Table [53] = 53
    Table [54] = 54
    Table [55] = 55
    Table [56] = 56
    Table [57] = 57
    Table [58] = 58
    Table [59] = 59
    Table [60] = 60
    Table [61] = 61
    Table [62] = 62
    Table [63] = 63
    Table [64] = 64
    Table [65] = 65
    Table [66] = 66
    Table [67] = 67
    Table [68] = 68
    Table [69] = 69
    Table [70] = 70
    Table [71] = 71
    Table [72] = 72
    Table [73] = 73
    Table [74] = 74
    Table [75] = 75
    Table [76] = 76
    Table [77] = 77
    Table [78] = 78
    Table [79] = 79
    Table [80] = 80
    Table [81] = 81
    Table [82] = 82
    Table [83] = 83
    Table [84] = 84
    Table [85] = 85
    Table [86] = 86
    Table [87] = 87
    Table [88] = 88
    Table [89] = 89
    Table [90] = 90
    Table [91] = 91
    Table [92] = 92
    Table [93] = 93
    Table [94] = 94
    Table [95] = 95
    Table [96] = 96
    Table [97] = 97
    Table [98] = 98
    Table [99] = 99
    ... 16283 more entries
BlackLevel: 2047.00
WhiteLevel: 15000
DefaultCropOrigin: H = 0.00 V = 0.00
DefaultCropSize: H = 1728.00 V = 972.00
ColorMatrix1:
      0.6719  -0.0994  -0.0925
     -0.4408   1.2426   0.2211
     -0.0887   0.2129   0.6051
AsShotNeutral: 0.4736 1.0000 0.6240
BaselineExposure: +0.00
ActiveArea: T = 0 L = 0 B = 972 R = 1728
*Tag51044: SRational = 24000/1001
BaselineExposureOffset: +0.00
NextIFD = 0

Exif IFD: Offset = 410, Entries = 5

ExposureTime: 1/30.0 sec
FNumber: f/1.80
ISOSpeedRatings: 400
SensitivityType: ISO Speed
ExifVersion: 2.30
NextIFD = 0

*** Warning: Too little padding on left edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on top edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on right edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on bottom edge of CFA image (possible interpolation artifacts) ***
Raw image read time: 0.008 sec
Linearization time: 0.005 sec
Interpolate time: 0.041 sec
Validation complete
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: g3gg0 on September 03, 2014, 10:58:08 AM
just... wow :)
amazing!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: dmilligan on September 03, 2014, 09:50:29 PM
Premiere/SpeedGrade are working and playback is realtime!!!!!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: Danne on September 03, 2014, 09:53:23 PM
 :o
holy crap!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: LRF on September 03, 2014, 10:48:54 PM
Guys, you never cease to amaze me  :D

If people with such skills and attitude were working in one company... Damn, you would turn the market inside-out within six months!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: dmilligan on September 03, 2014, 11:17:28 PM
Okay, now that it is basically working I have posted a preliminary binary for people to try. You'll need to be somewhat familiar the command line to get it started, but eventually I plan on adding some kind of installer or setup utility. See the OP for details. Also please keep in mind that there are all sorts of things not implemented yet (like audio, white balance, processing options such as banding correction and chroma smoothing, etc.).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: reddeercity on September 03, 2014, 11:36:40 PM
This look very good ! would this work with other Software like FCPX or is this just for Adobe Workflow ?
Never the less I will give it a try .
Thanks   8)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: Frank7D on September 03, 2014, 11:58:06 PM
But not for Windows (yet), right?
Edit: Never mind, I read the OP.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: tin2tin on September 03, 2014, 11:58:35 PM
Pismo is crossplatform (mac,linux, win): http://www.pismotechnic.com/
Can it be used as an alternative to fuse?

(It was used for a virtual filesystem for Avisynth: http://www.turtlewar.org/avfs/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: chmee on September 04, 2014, 01:02:21 AM
@dmilligan
the original bmcc-linearization-table is inside my cdng's. only copy/paste the tag and the values. or pipe the dng_validate -v output into a file and parse them into a binary. (or look into my sources (https://bitbucket.org/chmee/raw2cdng/src/3b267af91e6caaf81bbe47131773a85e4500d09d/csharp2010_code/raw2cdng_v2/?at=master). There's the dng-header named dngtemplate20.dng.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to DNG converter (WIP)
Post by: ayshih on September 04, 2014, 04:08:43 AM
Quote from: tin2tin on September 03, 2014, 11:58:35 PM
Pismo is crossplatform (mac,linux, win): http://www.pismotechnic.com/
Can it be used as an alternative to fuse?

(It was used for a virtual filesystem for Avisynth: http://www.turtlewar.org/avfs/)
Interesting!  It looks Pismo's file mounting may actually be only for Windows, but it's not a problem to have platform-specific code.  I'll definitely look into this.

Edit: Okay, there is cross-platform support for Pismo, but it may still be preferable to stick with fuse for OS X and Linux.

Alternatively, it hasn't been explicitly stated, but the MLVFS proof of concept works on Linux as well.  You'll have to grab the MLVFS code and compile your own executable, but you probably already have fuse installed.  I imagine one could configure a Linux VM in such a way as to achieve Windows support.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: baldand on September 05, 2014, 06:42:17 PM
I just tested mlvfs on Linux (Ubuntu 14.04). Nice work! :-)

I tried to playback the DNGs with MlRawViewer. There were a couple of small problems.

I had to mount using full path names, not with ~ in the mlv_argument. It didn't give an error when mounting, but the mount dir could not be opened.

The BaselineExposure tag gave a rational value of 0/0 which caused a divide by zero for me. Easy for me to workaround with a test though.

Another problem was that because the dir is read only, I can't write my per-clip config files into the DNG directory, so it's not possible to store any parameters or LUT settings for use when exporting. Export to MOV works, but with limited functionality. Well, not much to be done about that one.

I also noticed that CPU usage was quite high in the mlvfs process - something like 70% of one i7 core. Playing back through mlvfs was a bit heavier overall than direct playback, but it was still real time.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 05, 2014, 07:39:46 PM
This is simply amazing! Just created a new preference folder in davinci resolve 10 and the mlv files plays in realtime. No conversion. Dng:s magically appears in the osxfuse mount :o :o. Dmilligan breakthrough. All creds to participant coders and so on.

Couldn,t get sequence to work with After effects cs 6 nor to play in mlrawviewer, tried version 1.2.3, 1.1.7 and 1.1.6. (noticed Baldands reply above now regarding full pathnames)
Smooth as butter in davinci resolve. I,m on a macbook pro 15 inch 2.6 ghz, OSX mavericks.

Tried to convert some dual iso movie files but it wouldn,t sing. Tried Alex lates cr2hdr20bit binary and in lightroom. Got the following:

Output file     : /Users/Daniel/Desktop/mlv/M05-1941.MLV/00000005.DNG (already exists, overwriting)
Bus error: 10

Only info and of low priority of course.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 05, 2014, 08:15:59 PM
Thanks for the feedback!

I'll fix the BaselineExposure tag (interestingly, dng_validate didn't complain about that).

I'm not sure what the deal is with ~ is, but I have noticed that too. It seems that the stdio.h functions don't accept it.

About it being readonly: I can fix that (and I plan to), but it's going to be some work. Basically I can do a loopback to the original directory and store the actual data there (FUSE supports both read and write), but it means implementing all the FUSE API's write callbacks (since it's read only right now I only need to implement the read functions) and doing some type of path wrangling to manage where the files are put and how they map between the two file systems. It's certainly possible, but a lot of work.

The readonly thing seems to also be the reason AE isn't working, it tries to write it's own XMP settings, which it can't b/c the filesystem is readonly.

CPU usage is probably from the 14 =>16 bit conversion. There may be some possibility for performance improvement there (IDK how efficient my bit unpacking code is or could be, but it works), but if it's already realtime, it doesn't seem particularly important :P



Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 05, 2014, 08:29:21 PM
ah, ok read only. Fun is i can open the dng:s and store changes made in acr and photoshop.
Very promising this indeed.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on September 06, 2014, 05:39:40 AM
Will be watching this thread but will stay away from trying this awesome sw just yet since the discussion is too technical at the moment. Hope you folks can release the installer soon.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 06, 2014, 10:49:20 AM
Cool! When I right click a dual iso movie file I can give the file read and right priviligies and the files can convert fine with cr2hdr20 bit and also open in after effects.

I have to drag the folders to desktop to be able to change permissions.


Apply permissions to all items in a folder or a disk

    Select a folder or a disk, then choose File > Get Info.
    Click the lock icon to unlock it, then enter an administrator's name and password.
    Choose "Apply to enclosed items" from the Action pop-up menu (looks like a gear).

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on September 06, 2014, 04:54:15 PM
I will experiment with the Linux (Ubuntu 14.04.1) in a Virtual Box (https://www.virtualbox.org/) on Windows 7 in the evening. It should work in theory, I will report back:
1. Run the Linux in Virtual Box on Windows 7.
2. Run the MLVFS in the virtualized Linux system.
3. Open the DNG sequence in Windows 7.

Update: I can't get it working, the latest stable (4.3.15) Virtual Box drops error for me, but I'm on that to solve somehow.
Update2: I could get it working with the new stable 4.3.16 VB. It's a very long journey to do, but it works. :) As I have time I write a step by step "how-to". ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 06, 2014, 11:25:48 PM
Quote from: baldand on September 05, 2014, 06:42:17 PM
I had to mount using full path names, not with ~ in the mlv_argument. It didn't give an error when mounting, but the mount dir could not be opened.
Paths now have typical shell expansion (e.g., ~), and there is error handling for bad paths.  I'll let dmilligan update the binaries for those who can't compile.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 07, 2014, 02:54:33 PM
Created a short video tutorial on how to mount the fuse app according to the terminal commands provided in the first post from David Milligan.
Got some questions about this in my inbox.
This workflow will likely change soon due to the early development stages. But for now...


*Update Dmilligan created a automated gui through services in the first post. Terminal workflow on mac not needed anymore.

Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: nigel on September 07, 2014, 05:45:28 PM
Oh wow. Is it time to get a Mac? Hahah
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 08, 2014, 07:21:16 AM
Quote from: Danne on September 06, 2014, 10:49:20 AM
Cool! When I right click a dual iso movie file I can give the file read and right priviligies and the files can convert fine with cr2hdr20 bit and also open in after effects.
It's easy enough to understand why After Effects complains; the virtual filesystem is configured to not have write permission, and you work around that.  What I don't understand is your statement about being able to convert with cr2hdr.  The filesystem doesn't have any actual implementation of the write call.  So, what exactly happens when you run cr2hdr?  Where do the converted DNGs go?  I don't have a dual-ISO video to try this out myself.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 08, 2014, 10:42:36 AM
- Drag the dual iso dng folders to my desktop, then give the folder read, write permissions
- Convert the files using cr2hdr20bit as usual. Original dng files are being overwritten to the converted one.

Question. How do you explain acr being able to store write changes directly to dng files in the original fuse folder but not in after effects which needs permissions? Is acr overriding the write, read permission structure?

Thanks
/D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 08, 2014, 02:18:59 PM
Quote from: Danne on September 08, 2014, 10:42:36 AM
- Drag the dual iso dng folders to my desktop, then give the folder read, write permissions
- Convert the files using cr2hdr20bit as usual. Original dng files are being overwritten to the converted one.
Ah, okay.  To be clear, when you drag the folders to your desktop, you are converting the virtual DNGs to actual DNGs.

Quote from: Danne on September 08, 2014, 10:42:36 AM
Question. How do you explain acr being able to store write changes directly to dng files in the original fuse folder but not in after effects which needs permissions? Is acr overriding the write, read permission structure?
When you use ACR on files in a read-only directory (e.g., on a CD), it can't create the sidecar XMP files (which is also its fallback for read-only DNG files), so the changes are instead stored in a central Camera Raw database.  That's how ACR can work with the (current) read-only nature of the MLVFS filesystem.

I don't know how files are distinguished in the Camera Raw database, so it's possible you may lose your ACR modifications if you mounted the MLV file in a different location.  Certainly, the ACR modifications would not persist if you mounted the MLV file on a different computer.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 08, 2014, 02:58:38 PM
Theoretically, dual-ISO support would be built into this tool (that would be the best case scenario). However, I don't see this as a viable option. I don't think there's really any way to make dual-ISO conversion fast enough to be useful with this on-the-fly way of doing things (unless somebody wants to implement dual-ISO conversion in hardware on an FPGA or something :P), and make it fit in with the way the FUSE API works (processing small chunks of a frame on demand, rather than the whole frame at once).

I think for dual-ISO you're just going to need to go a more standard processing route, permanently converting the DNGs and then permanently converting those with cr2hdr (you can use this tool for permanent conversion, by copying the DNGs out of the virtual filesystem, but then there's no real advantage to using this tool over any of the other ones).

It would be possible for this tool to generate "low-res proxies" of dual-ISO clips for editing or previewing (such as by just skipping the bright rows in the output) though.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 08, 2014, 03:28:25 PM
Quote from: ayshih on September 08, 2014, 02:18:59 PM
Ah, okay.  To be clear, when you drag the folders to your desktop, you are converting the virtual DNGs to actual DNGs.
Yes, that must be the case. But I am simply copying them from the mount.

Quote from: ayshih on September 08, 2014, 02:18:59 PMWhen you use ACR on files in a read-only directory (e.g., on a CD), it can't create the sidecar XMP files (which is also its fallback for read-only DNG files), so the changes are instead stored in a central Camera Raw database.  That's how ACR can work with the (current) read-only nature of the MLVFS filesystem.
Thanks for claryfiyng.

Quote from: dmilligan on September 08, 2014, 02:58:38 PM
I think for dual-ISO you're just going to need to go a more standard processing route, permanently converting the DNGs and then permanently converting those with cr2hdr (you can use this tool for permanent conversion, by copying the DNGs out of the virtual filesystem, but then there's no real advantage to using this tool over any of the other ones).
It is till faster to copy the dng files from the fuse mount than exporting from a converter right?
So, there won,t be a possibility to overwrite the mounted dng:s since they,re only virtual ones from the orginal mlv files?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 08, 2014, 05:29:30 PM
Quote from: Danne on September 08, 2014, 03:28:25 PM
It is till faster to copy the dng files from the fuse mount than exporting from a converter right?
Theoretically, no. When you copy from the FUSE mount, you essentially turn this program into a traditional converter, so it should take about the same amount of time as a traditional converter, unless my conversion implementation is more efficient. Which might actually be the case right now, since I don't do any pre-processing on the files like banding correction or chroma smoothing, and also MLFVS supports multithreading via the FUSE API, so you might be getting more CPU usage. However, I imagine speed is mostly I/O limited, not CPU limited, and if this is the case, multithreading would have no effect. Feel free to do a speed comparison, though, I'd be interested to know (hint: use time (http://en.wikipedia.org/wiki/Time_(Unix)) to time a cp (http://en.wikipedia.org/wiki/Cp_(Unix)) of the virtual DNGs to the real filesystem).

Quote from: Danne on September 08, 2014, 03:28:25 PM
So, there won,t be a possibility to overwrite the mounted dng:s since they,re only virtual ones from the orginal mlv files?
No, that's not really possible, though I guess theoretically you could implement something where the CDNGs are writeable and the written data goes back into the MLV file VIDF blocks, modifying the original MLV file. That sounds VERY hard to implement.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on September 08, 2014, 05:33:28 PM
I could install Ubuntu (14.04.1) inside the VirtualBox (test build 4.3.15) (https://www.virtualbox.org/ticket/13187#comment:57) but the MLVFS isn't working. I think it's because I'm trying inside a virtual machine, so there is too much virtualisation "layer" for the system. Now I will try the Cygwin (http://cygwin.com/) (never used it), maybe it will work.

Update: I could get it working with the new stable 4.3.16 VB. It's a very long journey to do, but it works. :) As I have time I write a step by step "how-to". ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 08, 2014, 06:33:35 PM
MLVFS works fine in a virtual machine, but just within that virtual machine's local filesystem.  The way that VirtualBox and Parallels (at least) provide support for shared folders, they are local folders in the host machine, and thus MLVFS can't create its virtual filesystem there.

Instead, you'd probably have to use more traditional methods of sharing the MLVFS mount point: share the folder over a network as if they were two separate computers (e.g., using SMB).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on September 08, 2014, 07:25:32 PM
@ayshih: Thanks for your explanation, so I should use Samba server. I will try, but I never did this yet, so it will be a long journey for me. Cygwin is not for this job.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 08, 2014, 08:59:37 PM
So, compared the mlv_dump v 1.0 which ran vertical stripes correction to the speed of the mlvfs conversion. Used the "hints" (time, cp) commands from links provided from dmilligan (thanks).

The file is 9.52gb big.

The mlvfs conversion

real   0m35.285s
user   0m0.043s
sys   0m12.761s

The mlv_dump conversion

real   2m4.544s
user   1m36.526s
sys   0m10.662s

I also did some batch conversion around 38gb comparing mlrawviewer and mlvfs and the mlvfs was about 30 seconds faster around 5min30sec while mlrawviewer finished around 6min. Not whole lot of difference there.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 08, 2014, 10:58:03 PM
cool, thanks for the tests...


Audio is working!!!!

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: senzazn12 on September 09, 2014, 03:52:44 AM
Praying this gets ported to Windows somehow.  ::)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 09, 2014, 04:51:57 AM
Quote from: ayshih on September 08, 2014, 06:33:35 PM
Instead, you'd probably have to use more traditional methods of sharing the MLVFS mount point: share the folder over a network as if they were two separate computers (e.g., using SMB).
Oof, it's not for the faint of heart, but I've figured out how to get this approach working for Windows users.  You have to jump through a couple of extra hoops, and I'm sure performance will suffer because of the layers.

In the Linux VM:
mlvfs <mount point> --mlv_dir=<directory with MLV files> -o allow_other
In the Windows host, connect to the shared folder, e.g.,
\\<Linux hostname>\<mount point share>

Quote from: senzazn12 on September 09, 2014, 03:52:44 AM
Praying this gets ported to Windows somehow.  ::)
I'm still looking into using Pismo File Mount to provide native support for Windows, because the above is painful.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 09, 2014, 11:15:43 AM
oh i just remind of my good old symbian days.
back then i made a webdav server in C#.

using that tool we could get a cheap and easy-to-use MLVFS pendant on windows.
let me see....
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 09, 2014, 12:50:10 PM
updated the OP with the Windows instructions
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 09, 2014, 05:46:51 PM
short update:

i patched my mini WebDAVServer (http://upload.g3gg0.de/pub_files/d224e3b20042ab2ecce9a0912a347cf2/WebDAVServer.zip) to virtualize MLV files just like MLVFS does.

on windows you just have to mount the file system using
> net use x: \\localhost@8080
where 8080 is the port you specified in configuration dialog.
then drive X: shows you the content of the shared directory/card and allows you to go into MLV files.
you can even mount it via network. please dont use auth string, as windows confuses it with NTLM auth crap.

it only supports
- MLV->JPG
- MLV->WAV
- experimental saving files in MLV folder (creates a <mlv-file>_store directory)
- no DNG support

i will create a separate post for that and ask people to help.
creating sane DNGs is not really fun and takes its time to test and tweak.

@chmee:
interested in adding your C# DNG code to this tool?
will post it on bitbucket.org
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 09, 2014, 11:51:56 PM
There's now a simple installer and service for Mac, see the OP (and let me know that it works)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: jose_ugs on September 10, 2014, 12:55:40 AM
wow! mounting MLVs! keep it up, and hopefully you can figure out a Win7 solution
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 10, 2014, 01:07:14 AM
Works perfect! With audio aswell. How did you do that. Amazing work. Very convenient to have integration with services.
Folder names can,t contain spaces on mount point folder. One word only.
cool stuff.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: menoc on September 10, 2014, 01:34:55 AM
I installed it and works pretty well on my mac pro. Only thing is, for some reason Premiere  Pro CC 2014 won't let me import the dngs as an image sequence - the check box is greyed out. Anyone knows how to fix it?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: chmee on September 10, 2014, 02:24:46 AM
Menoc. You dont need to. It will be recognised AS sequence.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 10, 2014, 02:34:27 AM
Quote from: Danne on September 10, 2014, 01:07:14 AM
Folder names can,t contain spaces on mount point folder. One word only.
should be fixed now

Quote from: menoc on September 10, 2014, 01:34:55 AM
I installed it and works pretty well on my mac pro. Only thing is, for some reason Premiere  Pro CC 2014 won't let me import the dngs as an image sequence - the check box is greyed out. Anyone knows how to fix it?
Just select the first DNG and click import, the whole thing will be imported (don't worry about the check box). Or you can just use the "Media Browser", the DNGs should show up as a single clip and you can drag/drop it into your timeline.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 10, 2014, 07:23:18 AM
Folder names working.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: KurtAugust on September 10, 2014, 08:55:35 AM
Seeing this evolve so fast is like watching the Big Bang from your couch with a lovely cup of coffee. Very clever thinking!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: timbytheriver on September 10, 2014, 10:37:19 AM
Thank-you @dmilligan et al for this breakthrough work.

After following your new .dmg based instructions I'm getting an error running on Mac OSX 10.7.5

Operation fails after after selecting a mount folder:

=====

! The action "Run Shell Script" encountered an error.

I click > Show Workflow

The workflow appears > I click it. The workflow dialogue boxes appear, containing:

~/Library/Services/MLVFS.workflow/Contents/mlvfs "$2" --mlv_dir="$1"

Could you help please?

Cheers

tim
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 10, 2014, 12:44:08 PM
Quote from: timbytheriver on September 10, 2014, 10:37:19 AM
Mac OSX 10.7.5
I have not tested it on anything but Mavericks (10.9). You may need to compile it yourself if you want it to work on older versions.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: timbytheriver on September 10, 2014, 01:20:37 PM
Thank-you! I'd better learn to compile then... ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: menoc on September 11, 2014, 01:50:09 AM
QuoteJust select the first DNG and click import, the whole thing will be imported (don't worry about the check box). Or you can just use the "Media Browser", the DNGs should show up as a single clip and you can drag/drop it into your timeline.

Great thanks guys! That works in Premiere and Media Encoder. It does not work in After Effects CC 2014 though- I get an error
(https://www.dropbox.com/s/67k7r7mjhfxgn8x/Screen%20Shot%202014-09-10%20at%207.28.46%20PM.png?dl=0)
https://www.dropbox.com/s/67k7r7mjhfxgn8x/Screen%20Shot%202014-09-10%20at%207.28.46%20PM.png?dl=0
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 11, 2014, 02:16:59 AM
read this thread starting at reply #18
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mg_dev on September 11, 2014, 05:38:44 PM
Hi guys. I started tracing this topic some time ago, and I decided to share my conclusion :)

First, I'm interesting this topic from developer point of view rather than end user. I tried to convert MLV movie to set od DNGs using modified (fork) raw2cdng. I've prapared NVidia CUDA depth converters (in dynamic linked library). Sadly, summary processing time were not reduce too much because of hard drive bottleneck.. Creating 100 or 1000s  1-2Mb DNGs give huge overhead for disk-writing efficiency. 

Second, We all know - the idea of mounting MLV movies as a transparent 'drive' or folder is really good. But my approach is different a little bit - I will try to store DNGs in .ISO or .UDF (universal disk format). Output can be mounted on every platform. Writing huge data to one file is often more efficient than little data to  many files..
Or maybe someone tested this solution?..

I know that "ISO" access to DNGs won't be easier than mounting MLV in MLVFS, but it is kind of solution:)

I'll let You know, if I get better processing results :). And of course, I'll share my project if You want.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 13, 2014, 04:19:06 AM
I'm happy to announce that I've gotten an analogous implementation of MLVFS running on Windows using Pismo File Mount Audit Package (http://www.pismotechnic.com/pfm/ap/).  After installing PFMAP, download the MLV formatter DLL from Bitbucket (https://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs.dll) and then register it with PFMAP by entering on a command prompt:

pfm register mlvfs.dll

You can then mount any individual MLV video by right-clicking on it and selecting "Quick Mount", which will turn that video into a virtual folder containing the virtual DNGs.  The virtual folder also has support for creating and modifying additional files (e.g., XMP sidecar files), but the files are not preserved when the video is unmounted.

It's far better than the workaround with the Linux VM, but I don't know if it's better than g3gg0's WebDAV approach because I haven't tried it. 
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 13, 2014, 05:30:55 AM
Hi ayshih,
I tried your pismo method with no success.
I have been using pismo to mount avisynth files for years now, so I already had PFMAP (I believe).
I downloaded the file (but it was just "mlvfs", with no ".dll" on the end).
I changed the named to add ".dll" on the end.
I opened a command prompt, went to the directory with the file, and registered the file per your instructions.
I tried to quick mount an mlv file, but got a message saying it was an unsupported file format.

Edit:  Never mind, the problem was all me; I was using an old build of PFMAP. Once I installed the latest one I was fine.

Thanks for this great option! Can't wait to play with it!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 13, 2014, 06:54:42 AM
Using pismo, I am getting no wav files in the virtual folders.

Also, the folders are read-only by default, so I have to right-click them and make them writable in order to import the virtual DNGs into After Effects.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 13, 2014, 07:16:59 AM
Quote from: Frank7D on September 13, 2014, 06:54:42 AM
Using pismo, I am getting no wav files in the virtual folders.
Yup, I haven't implemented audio yet, but it shouldn't be too difficult since it's up and running on the FUSE side.

Quote from: Frank7D on September 13, 2014, 06:54:42 AM
Also, the folders are read-only by default, so I have to right-click them and make them writable in order to import the virtual DNGs into After Effects.
Hmm, the folder itself should already be writable, but I guess After Effects really wants the DNGs to be writable.  Be careful when making the DNGs writable; things could break in weird and fantastic ways because I didn't protect all of the possible calls (on the presumption that the DNGs were read-only).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 13, 2014, 04:03:15 PM
One other thing (using pismo): When I open the virtual DNGs in ACR or After Effects with white balance set to "As Shot", the "Temperature" is incorrect. For example, my camera (7D) was set to 5100 and the DNG opened with 2600.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 13, 2014, 04:22:14 PM
The DNG writing code for white balance needs work, because it doesn't have model-specific or preset-specific white-balance multipliers.  Perhaps you should create an issue on Bitbucket!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 13, 2014, 05:21:40 PM
Someone explain to me the math behind how to get from temperature to RGB multipliers, and I'll fix it :)

QuoteAsShotNeutral specifies the selected white balance at time of capture, encoded as the coordinates of a perfectly neutral color in linear reference space values.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 13, 2014, 07:08:58 PM
Quote from: dmilligan on September 13, 2014, 05:21:40 PM
Someone explain to me the math behind how to get from temperature to RGB multipliers, and I'll fix it :)
https://bitbucket.org/hudson/magic-lantern/src/e12ad3d84e7bc5b61e27c9e9e5a7c0423b39598c/contrib/g3gg0-tools/MLVViewSharp/DebayerBase.cs?at=unified

basically this:

            /* get the XYZ --> cone reference whites for requested temperatures */
            Matrix dst = coneDomain * KelvinToXYZ(ColorTemperature);
            /* verify: do all our matrices that came from dcraw really convert raw to D65? */
            Matrix src = coneDomain * KelvinToXYZ(6500);

            /* scale coordinates in cone color space */
            Matrix xyzScale = new Matrix(3, 3);
            xyzScale[0, 0] = dst[0] / src[0];
            xyzScale[1, 1] = dst[1] / src[1];
            xyzScale[2, 2] = dst[2] / src[2];

            /* finally scale colors */
            Matrix xyzKelvinWb = coneDomain.Inverse() * xyzScale * coneDomain;

            /* now combine the whole thing to get RAW --> RAW-WB --> XYZ --> Kelvin-WB --> XYZ --> (s)RGB --> RGB-WB */
            CamToRgbMatrix = WhiteBalanceMatrix * RGBToXYZMatrix.Inverse() * xyzKelvinWb * XYZToCamMatrix.Inverse() * WhiteBalanceMatrixRaw;
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 13, 2014, 11:05:25 PM
EDIT: okay, I think I've basically got it

What about G/M shift?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 14, 2014, 02:19:02 AM
never tried.

would set up a function that gives you the RGB values for the G/M shift, which is quite simple.  ([R, G, B] = [128-x, 128+x, 128-x])
then transform it to XYZ and straight forward to bradford cone domain.
then just do the same scaling as with kelvin.

basically you could do this with any white point in any color space.

so:

            Matrix gmShiftRef = new Matrix(3, 1);
            gmShiftRef [0] = 0.5;
            gmShiftRef [1] = 0.5;
            gmShiftRef [2] = 0.5;

            Matrix gmShiftRgb = new Matrix(3, 1);
            gmShiftRgb[0] = 0.5-gmShift;
            gmShiftRgb[1] = 0.5+gmShift;
            gmShiftRgb[2] = 0.5-gmShift;

            Matrix dst = RGBToXYZMatrix* gmShiftRgb;
            Matrix src = RGBToXYZMatrix* gmShiftRef;

            Matrix coneScale = new Matrix(3, 3);
            coneScale [0, 0] = dst[0] / src[0];
            coneScale [1, 1] = dst[1] / src[1];
            coneScale [2, 2] = dst[2] / src[2];

            Matrix xyzGMShift = XYZBradford.Inverse() * coneScale * XYZBradford;

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 14, 2014, 06:19:10 AM
MLVFS for Windows using PFM now extracts audio as well (DLL download (https://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs.dll)).  Let me know if you encounter any bugs!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mk11174 on September 14, 2014, 06:27:25 AM
This is pretty cool, can the RAW files somehow be added to work with this. Or is it only possible with MLV?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mario1000 on September 14, 2014, 07:23:14 AM
Quote from: ayshih on September 14, 2014, 06:19:10 AM
MLVFS for Windows using PFM now extracts audio as well (DLL download (https://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs.dll)).  Let me know if you encounter any bugs!

Hi ayshih
thanks a lot for your work but the download link does not work (it says that it found a dead link). Could you please fix this.
Thanks in advance

Edit: download link works!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 14, 2014, 07:41:47 AM
Quote from: mk11174 on September 14, 2014, 06:27:25 AM
This is pretty cool, can the RAW files somehow be added to work with this. Or is it only possible with MLV?
I don't think there's any limitation precluding implementing RAW support, although the lack of metadata is a little annoying.  However, I myself am more interested in making sure we fully support MLV files rather than worrying about the "older" format.  I encourage any interested developers to contribute to the code, of course!

Quote from: mario1000 on September 14, 2014, 07:23:14 AM
thanks a lot for your work but the download link does not work (it says that it found a dead link). Could you please fix this.
Thanks for letting me know!  Looks like Bitbucket got a little confused, so I've re-uploaded it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 14, 2014, 06:41:48 PM
(using pismo)
Now I have audio (.wav) files also; thanks!
One thing I notice: when I import the (virtual) sequence into After Effects, an .xmp (sidecar) file appears in the virtual folder. I believe these files normally contain the ACR settings used to import the sequence.
However, when I unmount the mlv and then mount it again, the xmp is gone. Maybe this is a good thing, since it implies AFX is not altering the mlv file.
If I want to retain the AFX/ACR import settings more permanently, I can save an xmp manually in the ACR settings folder (or maybe just drag the xmp from the virtual folder into a real folder).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 14, 2014, 07:03:37 PM
(using pismo)
What controls the naming of the (virtual) DNGs and WAVs?
All my DNG names are of the form ffffffff.DNG
All the WAVs are "_AUDIO.WAV"
This can be confusing when dealing with multiple MLV files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 14, 2014, 07:40:00 PM
Quote from: Frank7D on September 14, 2014, 06:41:48 PM
However, when I unmount the mlv and then mount it again, the xmp is gone. Maybe this is a good thing, since it implies AFX is not altering the mlv file.
If I want to retain the AFX/ACR import settings more permanently, I can save an xmp manually in the ACR settings folder (or maybe just drag the xmp from the virtual folder into a real folder).
Yup, while the virtual filesystem allows the creation of real files (e.g., XMP sidecars), I haven't yet implemented any preservation of those files after unmounting.  Dragging/copying the XMP files out of the virtual filesystem should work fine.

Quote from: Frank7D on September 14, 2014, 07:03:37 PM
What controls the naming of the (virtual) DNGs and WAVs?
The naming scheme, which is the same as on the FUSE versions, can certainly be changed.  I presume you want the filename of the MLV video prepended to the individual DNG files and WAV file?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 14, 2014, 07:45:19 PM
Is this xmp creation present on mac as well working with ae or is it a windows thing?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 14, 2014, 07:45:57 PM
"I presume you want the filename of the MLV video prepended to the individual DNG files and WAV file?"

That would be perfect (for me at least).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 14, 2014, 07:56:10 PM
Quote from: Danne on September 14, 2014, 07:45:19 PM
Is this xmp creation present on mac as well working with ae or is it a windows thing?
For now, just on the Windows side.  Because of the differences between FUSE and PFM and the rapid nature of MLVFS development, I imagine this won't be the last time you'll see features show up on one version before they show up on the other versions.

Quote from: Frank7D on September 14, 2014, 07:45:57 PM
"I presume you want the filename of the MLV video prepended to the individual DNG files and WAV file?"

That would be perfect (for me at least).
Does anyone else have any thoughts?  I don't want to make this change if there turns out to be a compelling reason to keep it as it currently is (or to change it to some third option).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 14, 2014, 08:06:17 PM
Cool, would it be able to store a new folder as well containing copies of dng;s like storing an xmp? I,m thinking Kitchehof lightroom app workflow in first hand. This would entirely skip the first converting step when working dual iso movie files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on September 14, 2014, 08:55:35 PM
Quote from: ayshihDoes anyone else have any thoughts?  I don't want to make this change if there turns out to be a compelling reason to keep it as it currently is (or to change it to some third option).
I prefer this kind of file naming, source MLV: M18-1424.MLV, DNG: M18-1424-00000.DNG ... M18-1424-00250.DNG
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 14, 2014, 09:07:45 PM
99,999 frames at 23.976 fps would give about 69.5 minutes of video, which should be more than enough for one take, so I agree with surami's suggestion.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 15, 2014, 01:01:14 AM
I chose "_AUDIO.WAV" so that the audio would appear first. I'm fine with adding the MLV filename as a prefix, but we probably need to decide something and then not change it anymore, as it would mess up any projects already created that point to specific filenames. The other option is making it configurable with various options from MLV metadata (such as cam name/serial, text/take tags, date/time, etc). Some kind of string format could be a command line parameter. Ex: "$CAMERA_$TAKE_$TIME_" => 7D_003_07-30_00000000.DNG. The advantage to making it customizable is that we can avoid breaking old projects that depend on specific paths. Of course the disadvantage is this is significantly more complex.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 15, 2014, 01:17:50 AM
ive chosen M26-1335_000000.dng in my windows MLV client.
thought thats a good choice. frame count should be enough and it contains the MLV name as prefix.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 15, 2014, 07:05:13 AM
I've changed the naming scheme in the FUSE and Windows/PFM versions to match g3gg0's scheme.  (dmilligan: please update the OS X binaries)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 15, 2014, 07:40:20 AM
So right now, the audio file is named like "M13-1837_audio.wav"
Could it just be "M13-1837.wav"?
It would work better with the After Effects script I am using ("Smart Import 2.jsx").
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 15, 2014, 08:10:33 AM
scripts can be simply changed ;)
main question is, if it makes sense to remove _audio or to keep it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 15, 2014, 08:25:48 AM
I agree. Is there a reason to have the "_audio" in the name?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 15, 2014, 01:32:06 PM
I don't really see a reason to have "_audio" in the name anymore using the prefixes. Before, when not using prefixes, it had to have some kind of name so I just named it what it was, "audio", and the "_" makes it show up first. Now, I see no reason not to just use "M12-3456.WAV"
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 15, 2014, 03:47:17 PM
Yeah, there really doesn't appear to be any benefit for having "_audio" in the filename when it already has the extension ".wav", so the cleaner approach would be to remove the "_audio".
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 15, 2014, 03:52:08 PM
will do it the same as MLVFS does. so you both decide.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 15, 2014, 05:39:40 PM
With no objections to removing "_audio", I have made that change.  New Windows/PFM DLL uploaded.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 15, 2014, 10:40:49 PM
mac binary updated
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 16, 2014, 06:04:36 AM
Quick note/tip for pismo method:
If having difficulty importing the (virtual) DNG sequences into your favorite software, try:
1. Making each virtual MLV folder writable.
2. Deleting the IDX files that get created alongside the folders when you mount the MLV.

I had to do both of the above when using the smart import scripts in After Effects.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 18, 2014, 12:02:58 AM
@g3gg0,

Well, I'm sort of stuck, this math seems to be over my head. I just sort of blindly implemented your C# code, but mathematically, I have no idea what's going on or what's supposed to be happening. I'm pretty sure I've computed xyzKelvinWb matrix correctly, but I don't really know where to go from there to get AsShotNeutral. If you don't mind, take a look: https://bitbucket.org/dmilligan/mlvfs/branch/white_balance
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on September 18, 2014, 12:14:12 AM
If it helps, I've extracted the ufraw kelvin routines in a separate C file:
https://bitbucket.org/hudson/magic-lantern/src/cr2hdr-20bit/modules/dual_iso/kelvin.c
https://bitbucket.org/hudson/magic-lantern/src/cr2hdr-20bit/modules/dual_iso/kelvin.h

Usage examples in cr2hdr.c (look for AsShotNeutral and kelvin): https://bitbucket.org/hudson/magic-lantern/src/cr2hdr-20bit/modules/dual_iso/cr2hdr.c
For initialization, call adobe_coeff("Canon", "EOS ...") - I'm doing this in dcraw-bridge.c.

Also, a good reference is this: http://users.soe.ucsc.edu/~rcsumner/rawguide/RAWguide.pdf

If you'll still be stuck, I can sit down and figure it out.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 18, 2014, 01:52:45 AM
a very informative site is here: http://www.brucelindbloom.com/index.html?Eqn_XYZ_to_T.html
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 18, 2014, 09:19:10 AM
a good XYZ/xyY/wavelength/kelvin converter is there: https://www.sylvania.com/en-us/tools-and-resources/Pages/led-color-calculator.aspx
it's from osram and meant for LED gamut calculation
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 18, 2014, 10:08:19 AM
if you want to debug your code, here the results of my code:


            /* get the XYZ --> cone reference whites for requested temperatures */
            Matrix temp = KelvinToXYZ(ColorTemperature);
            /* verify: do all our matrices that came from dcraw really convert raw to D65? */
            Matrix tempD65 = KelvinToXYZ(6500);

            Matrix dst = coneDomain * temp;
            Matrix src = coneDomain * tempD65;


(http://s1.postimg.org/3y18b7b0f/matrix.png)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 18, 2014, 06:25:44 PM
Thanks for the link a1ex, that's the most helpful thing I've read. I think I've started to make some progress in my understanding. Please correct me where I'm wrong:

Compute XYZKelvinWB matrix using the "chromatic adaptation" techniques in the Bradford cone domain (I'm pretty sure now, that I've got this part right). Then use the XYZKelvinWB matrix to transform "white" (1,1,1) to some different color temperature (I assume this is what the XYZKelvinWB matrix does, but I'm not totally sure). Then convert this new "white" back to the camera raw color space using the inverse of the "ColorMatrix1" (which if I understand correctly this matrix is a Cam->XYZ). Now we have a neutral color in camera raw coordinates right? So then the multipliers (aka AsShotNeutral) are simply {R/G, 1, B/G}. Mathematically this would be:

CamNeutral = [ColorMatrix1]-1 * ([XYZKelvinWB] * [1,1,1])
AsShotNeutral = [ CamNeutral R / CamNeutral G, 1, CamNeutral B / CamNeutral G ]

Is this correct, or am I totally off?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 19, 2014, 10:44:35 PM
well none of that worked, but I just refactored kelvin.c from c2hdr20bit to work with mlvfs (had to get rid of some global variables) and its working, yay!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 20, 2014, 09:18:09 AM
Nice work.
Hi David. Love this workflow. Any thoughts on including raw format and maybe also some "write" possibilities(direct use in after effects)? Noticed some of this going on in the windows workflow but I have no idea how complicated this will be for mac mlvfs.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 20, 2014, 05:00:05 PM
For Windows users who can't directly edit the cdngs, I posted a workflow for making proxies or intermediates with After Effects using some batch files to simplify things:
http://www.magiclantern.fm/forum/index.php?topic=13354.0 (http://www.magiclantern.fm/forum/index.php?topic=13354.0)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: senzazn12 on September 20, 2014, 06:07:37 PM
When I try to register the MLVFS.DLL in command prompt I get an error saying this. Anybody know what I'm doing wrong I'm doing wrong?

It says ERROR: Unable to register formater "mlvfs.dll"
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 20, 2014, 06:13:30 PM
Are you sure you are typing the command in the correct directory (i.e., where you downloaded mlvfs.dll to)?  You can type "dir mlvfs.dll" to make sure that the DLL is present in current directory.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: senzazn12 on September 20, 2014, 06:21:45 PM
It is in my downloads folder.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 20, 2014, 06:28:37 PM
What I did was:
1. Renamed the file to add the ".dll" on the end.
2. Copied the file to my Windows folder.
3. Left-clicked somewhere in the Windows folder.
4. Held down the shift key while right-clicking, and chose "Open Command Window Here."
5. Typed the command per ayshih.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: senzazn12 on September 20, 2014, 06:32:39 PM
Thanks ayshih and Frank7D

That worked perfect for me! I'm not too literate with CMD so thanks for helping me out. I'll be trying your workflow next. :D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 20, 2014, 11:28:59 PM
Quote from: Danne on September 20, 2014, 09:18:09 AM
Any thoughts on including raw format and maybe also some "write" possibilities(direct use in after effects)? Noticed some of this going on in the windows workflow but I have no idea how complicated this will be for mac mlvfs.
Well, I've mostly got a read/write filesystem working via a loopback to the real filesystem (files to show up in the xxx.MLV directory are stored in a xxx.MLD directory in the real filesystem). However, After Effects still wouldn't open the DNG sequence because it wanted to write to the DNGs themselves. I was hoping that AE would fall back to using an XMP sidecar if the DNGs were readonly, but this didn't happen and it appears there's no way force it to (unless you can find one). There's certainly no way to make the DNGs themselves writeable, so it looks like an AE workflow is not going to be possible. IMO this is very poor design on Adobe's part. Maybe Adobe will bring native CDNG support (like in Premiere) to AE soon.

It might still be possible to use Lr or Photoshop.

Though, personally, I much prefer workflows that don't involve ACR. There seem to be a lot of folks that swear by ACR, but I don't buy it, it's just not good for video. There are much better video specific tools out there.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 20, 2014, 11:52:25 PM
Quote from: dmilligan on September 20, 2014, 11:28:59 PM
I was hoping that AE would fall back to using an XMP sidecar if the DNGs were readonly, but this didn't happen and it appears there's no way force it to (unless you can find one). There's certainly no way to make the DNGs themselves writeable, so it looks like an AE workflow is not going to be possible.
I don't have or use AE, but Frank7D's workflow (http://www.magiclantern.fm/forum/index.php?topic=13354.0) uses the Windows/PFM version of MLVFS, and those virtual DNGs aren't writable either.  He turns off the read-only attribute on the virtual DNGs, and then AE can open the sequence.  If AE actually tries to write to the virtual DNGs, that implementation will return a type of error that is actually soft-ish (i.e., AE might not actually realize that the writes failed), which may also be helping.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 20, 2014, 11:55:17 PM
interesting, though it sounds kind of hackish to make the DNGs writable, I would prefer that the DNGs attributes remain readonly, if they are in fact going to be readonly, but I'll try it
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 21, 2014, 12:13:43 AM
okay well, that worked, so it's now working with AE, binaries updated
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on September 21, 2014, 12:39:38 AM
okay, will update the code to also use .MLD directories.
just to make our tools behave the same ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 21, 2014, 08:59:31 AM
Hi!
Tried the latest update and the workflow crashes. I all seems going good at first sight but when entering the mlv folders and trying opening the dng:s either makes the folders dissapear or the files won,t open in both photoshop acr and AE acr. This is how it looks. I,m on macbook pro os X mavericks adobe cs 6.

[img=http://s22.postimg.org/7u7tqsxf1/Screen_Shot_2014_09_21_at_08_51_11.png] (http://postimg.org/image/7u7tqsxf1/)

[img=http://s22.postimg.org/52ok06ywd/Screen_Shot_2014_09_21_at_08_54_10.png] (http://postimg.org/image/52ok06ywd/)

[img=http://s22.postimg.org/6gg6vhy5p/Screen_Shot_2014_09_21_at_08_54_53.png] (http://postimg.org/image/6gg6vhy5p/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mageye on September 21, 2014, 11:06:38 AM
I am getting the same thing. The last version was working.

The latest version appears to mount the files and as soon as you go to browse - the mounted folder disappears? The folders seem to have disappeared into the ether? However the folders are visible from the Terminal only (they are not visible in Finder).

What's happening?

I'm running Mavericks 10.9.5 with latest OSXFUSE (v2.71)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 21, 2014, 03:29:56 PM
Something's wrong with the latest OS X MLVFS, now that there is a write implementation, and I haven't figured it out yet.  It may have to do with the "._*" files that OS X wants to create for all of the files.  Sorry guys: we'll get this fixed as soon as possible.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 21, 2014, 07:04:53 PM
I can't figure out what's going on either. It was working fine, I even got it to work in AE, but now nothing works, so I've reverted the dmg to the previous working version until we figure it out.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 21, 2014, 09:21:03 PM
I believe I've fixed the problem.  dmilligan, please confirm and update.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 22, 2014, 12:49:49 AM
No, it's still really screwy, I get file not found errors. I think there are more FUSE API functions that need to be implemented for it to work (like statfs and getattr). The strange thing is I can't figure out how I ever got it to work in the first place.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 22, 2014, 12:59:46 AM
Hmm, that's odd.  I'm no longer getting errors on my OS X side, so I don't know if I can help you track down the problems.  If you haven't already, I recommend nuking everything (object files, .MLD directories, any files starting with a period) to make sure everything builds and runs as cleanly as possible.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 22, 2014, 03:44:37 AM
yeah I already tried that, also rebooted and installed the latest xcode updates, everything was the same. I have sort of narrowed it down though, it seems to be an issue with flush, when I get rid of it everything works. If you don't think there's anything wrong with getting rid of flush I'll push the commit.

I have updated the mac binary so please try it everyone.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 22, 2014, 07:12:41 AM
Holy mother of all crap. It is working! What is also working is direct conversion of dual iso moviefiles  :D :D :D. The mld directorys takes the files perfectly. I ran the cr2hdr binary. What a morning man. Awesomeness.
Biggest thanks David and Ayshih.

I you drag and drop several folders at once in after effects, see to it it doesn,t contain any wav file cause it won,t open then. Open the folders one by one works as usual. This is not related to the MLVFS app.
[img=http://s7.postimg.org/d6sbcun9j/Screen_Shot_2014_09_22_at_06_47_49.png] (http://postimg.org/image/d6sbcun9j/)

Dual iso creation!
[img=http://s7.postimg.org/8m64xx3k7/Screen_Shot_2014_09_22_at_07_00_42.png] (http://postimg.org/image/8m64xx3k7/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on September 22, 2014, 01:11:54 PM
A quick question: I have the MLVFS version downloaded from 10 days ago. Do I have to uninstall this from the Mac and reinstall the newer version on my laptop. Any safeguards or precautions to follow while do so?

Are the steps the same as they were before when Danne / Dimiligan guided me through the process.

I appears that the 10 day old version is working fine on AE 2014 for me. However, cannot say the same for the g3gg0's version on my Windows 8 machine. This maybe be because I am doing something horribly wrong.

Will try this files on Photoshop, Premier Pro and Speedgrade to ensure all is working fine and as intended.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 22, 2014, 01:20:03 PM
I,m on mac and I simply install the new version and services asks if I want to replace the old one and then it is good to go. Cannot help on windows.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on September 22, 2014, 01:35:41 PM
thanks but i am getting the message that mlvfs cannot open since it is from an unidentified source. sorry i am new to the osx environment .. would have never asked such a stupid question if i was no the windows environment  :(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 22, 2014, 01:41:41 PM
So if you,re on a mac and you get the unidentified developer notification you have to change settings in your,re OS. Try this or google for more info on the subject. Or are you on windows?
http://support.apple.com/kb/PH14369

http://stackoverflow.com/questions/19551298/app-cant-be-opened-because-it-is-from-an-unidentified-developer

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 22, 2014, 01:42:38 PM
Quote from: Canon eos m on September 22, 2014, 01:11:54 PM
I appears that the 10 day old version is working fine on AE 2014 for me. However, cannot say the same for the g3gg0's version on my Windows 8 machine. This maybe be because I am doing something horribly wrong.
For Windows, you can try the MLVFS version that uses Pismo File Mount Audit Package (see Frank7D's post on his AE workflow (http://www.magiclantern.fm/forum/index.php?topic=13354.0).  The MLVFS formatter can be downloaded here: https://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs.dll

@dmilligan: can you add a download link for the DLL to the first post?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 22, 2014, 02:01:34 PM
done, also added the stackoverflow link from Danne about mac security settings
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DavidSh on September 22, 2014, 04:20:03 PM
beautiful.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 22, 2014, 08:01:26 PM
Dual iso movie conversion the fast way. Thanks for all efforts. This is massive.

*Issues with dng exiftool samelevel command in lightroom. Still gets converted though but dng tags doesn,t get written. See if Kitchehof finds some useful info on this.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 26, 2014, 12:52:33 AM
New version of the Windows/PFM MLVFS DLL (https://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs.dll): like the other implementations, XMP sidecar files are now preserved in an associated .MLD directory.  Other types of files can be created, but they will be discarded upon unmount.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on September 26, 2014, 02:37:38 AM
Tried activating the Pismo, etc. but gave up.
Find the g3gg0 solution better for now at least. There is so much happening on this forum and so fast difficult to keep up  :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 27, 2014, 12:52:55 AM
ayshih, the "smart import 2" script didn't seem to have a problem with the latest mlvfs.dll, but I ran into another issue:
1. Mounted the MLVs
2. Created an AE project
3. Imported the sequence(s)
4. Closed and saved the project
5. Unmounted and then mounted the MLVs again
6. Opened the AE project (so far, so good)
7. Did a Ctrl+H on a sequence (replaces footage; this is a useful way to adjust the import settings after the fact)
8. ACR opened again; when I said "OK" to open in AE, I got this message:
(http://i62.tinypic.com/1dypgo.jpg)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 27, 2014, 04:29:21 AM
I've implemented vertical banding correction code from raw2dng. Please test if you have a 5D3 (I don't).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 27, 2014, 02:55:52 PM
Hi David!
I,m yet to see vertical banding on my 5d mark 3. I think I encountered one occasion with banding that was fixed with mlrawviewer.
Tried to reproduce a mlv file with banding cause the one I got is a raw file.
Just for you to clarify. Do you intend to include raw 1.0 to this workflow or is the philosophy here to stick strictly to mlv conversion? The reason I use raw is purely 3x mode and 2240x1260 24fps hacked preview which gives my several minutes of filming. Mlv is not really getting that speed.

Ok, so my test couldn,t be done cause of errors. It seems the files are getting different information in them with the latest binary. SOme shows with thumbnails, some don,t. I managed to get one dng out on my desktop, only to open up a completely black picture in acr. The mount randomly unmounts, very unstable.
OS X Mavericks macbook pro 15 inch

Here,s the dng sample
https://drive.google.com/file/d/0B4tCJMlOYfirU254U0JxR0l2aU0/edit?usp=sharing

A few error codes when I tried either open a file or drag/drop onto desktop.

I upload some pictures here:
[img=http://s28.postimg.org/l8whed015/Screen_Shot_2014_09_27_at_14_42_23.png] (http://postimg.org/image/l8whed015/)

translation: Photoshop could not follow you request...
[img=http://s28.postimg.org/4b2269g15/Screen_Shot_2014_09_27_at_14_42_55.png] (http://postimg.org/image/4b2269g15/)

[img=http://s28.postimg.org/jii1qm7vt/Screen_Shot_2014_09_27_at_14_43_53.png] (http://postimg.org/image/jii1qm7vt/)

[img=http://s28.postimg.org/p9884cfvt/Screen_Shot_2014_09_27_at_14_44_06.png] (http://postimg.org/image/p9884cfvt/)

[img=http://s28.postimg.org/534q5gk89/Screen_Shot_2014_09_27_at_14_44_15.png] (http://postimg.org/image/534q5gk89/) 

On a sidenote. Could view mlvfiles fine in mlrawviewer with the former mlvfs build but exports to mov don, t work. Is this expected behavior?

Thanks for good work!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on September 27, 2014, 04:06:56 PM
hello,
i have tried the latest mlvfs with my mac 10.9.5, i copied the mlv dir on the desktop also the empty dir. then i mounted the dir with the mlv file, it mounted the files to the empty one, then after 10 seconds it suddenly disappears,..
tried different mlv files, its the same with all other files.
tested the same procedure on an external drive, also not working.

thx. sw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 27, 2014, 05:19:56 PM
I think this is a multithreading issue. You can use the -s option to force single threaded mode. I've updated a new dmg that uses the -s option, so try that, there may still be other stability issues.

We're going to need to do some thread synchronization for this to work properly in multithreaded mode, not sure of a good multi-platform, generic way to do this since semaphores and locks are usually OS specific, you got any suggestions @ayshih?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 27, 2014, 05:36:18 PM
@dmilligan: I've only encountered a few problems with cross-platform support for POSIX threads, and even those problems were localized to thread attributes and handling (joining, canceling, etc.).  As long as we're not talking about instantiating threads ourselves, I don't believe we'd encounter any weirdness at all with just mutexes.

@Frank7D: I think I may have tracked down the problem with the Window/PFM version.  Stay tuned.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mageye on September 27, 2014, 05:50:44 PM
And on my hackintosh ... Mavericks 10.9.5 ... I am not experiencing any problems since the 22/09/2014 update of MLVFS.

(@ Danne) It's interesting you are getting those errors? ::) That is what I was getting pre 22/09/2014 (the unmounting and all error messages if you tried to import to After Effects.)

I must say now I am very happy with it as part of the workflow. I can mount, then open, adjust and export with relative ease and stability.

Oh. One thing I haven't really had a chance to try out yet is how it works with Davinci Resolve. I guess that will be fine though?

Thank you so much for all who have made this possible. It's genius! :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 27, 2014, 06:08:25 PM
New test.

Didn,t notice a different in banding with former versions. Probably because my pics don,t have banding?
With the latest mlvfs I think the vertical banding part is causing some irregular behaviour. It goes like this.
Open a dng within the mount folder and a thin pixelborder appears on top. When closing down and opening again, the border issue is gone.
When dragging a dng to desktop a big pixelated border is present on top. See included pics. These are only a sample from the latest mlvfs. These are crops and overexposed.

Straight from mlvfs mount
[img=http://s2.postimg.org/vg8m71bfp/Screen_Shot_2014_09_27_at_18_00_36.png] (http://postimg.org/image/vg8m71bfp/)
after closing down and opening again
[img=http://s2.postimg.org/7qj6icd2d/Screen_Shot_2014_09_27_at_18_00_52.png] (http://postimg.org/image/7qj6icd2d/)
dragged to desktop
[img=http://s2.postimg.org/vishtvf39/Screen_Shot_2014_09_27_at_18_03_24.png] (http://postimg.org/image/vishtvf39/)

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 27, 2014, 06:43:16 PM
@Frank7D: I've uploaded a new version that fixed any crashing on my side, but I'm not using AE.  Can you try it with your workflow?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on September 27, 2014, 07:05:19 PM
It works! Another step eliminated; thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 27, 2014, 07:20:10 PM
@Danne, I think I fixed your issues with the bad data at the top of the file, try the latest.

Quote from: Danne on September 27, 2014, 06:08:25 PM
Didn,t notice a different in banding with former versions. Probably because my pics don,t have banding?
Probably. The correction should be identical to the correction provided by raw2dng/mlv_dump, and the algorithm does not do any correction unless the coefficients are above a specific threshold.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 27, 2014, 08:19:15 PM
Confirmed working! Fast progress. Amazing.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 27, 2014, 10:35:29 PM
okay, I used pthreads_mutex, seems to work pretty well, so the stripe correction now supports multithreading mode. I made a LOCK() and UNLOCK() macro, so just redefine these if you need to use something else on a different platform.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 29, 2014, 12:05:16 AM
Some big improvements! Chroma smoothing is now implemented (I'm not getting real-time playback with it, but it is reasonably fast ~10 fps, perhaps could be realtime on higher end systems), and this has also paved the way for easily implementing dual ISO processing, by implementing a way to process an entire frame all at once. And this has provided a fix for the issue with exFAT on OS X (b/c we avoid constantly opening and closing the MLV file, which was slow on exFAT for some unknown reason).

I haven't posted a binary yet, b/c I'm not quite sure how to provide a way to easily enable/disable the chroma smoothing options. The possible options are to provide two different automator workflows (one with chroma smooth and one without), or to prompt the user to select options (which is some extra mouse clicks). Any preference, or other ideas?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on September 29, 2014, 12:09:44 AM
Whow Great!
What about a choosing option when mounting??
Greets sw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 29, 2014, 12:18:48 AM
Did I hear dual iso processing  :o : :D :D

Could the setting be stored and than changed back again as desired than I would go for the click option but otherwise different  automator workflows is cool with me.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on September 29, 2014, 01:17:13 AM
Quote from: dmilligan on September 29, 2014, 12:05:16 AM
The possible options are to provide two different automator workflows (one with chroma smooth and one without), or to prompt the user to select options (which is some extra mouse clicks). Any preference, or other ideas?
I'm limited on the Windows/PFM side because I can't actually supply any parameters for the mounting.  My likely solution is to have a configuration text file (e.g., "config.ini") in the MLV's directory that can be parsed for such parameters on the fly.

Note that with the "config.ini" saved in the associated .MLD directory, this approach would make possible per-MLV settings (e.g., one file needs more chroma smoothing than others) that are persistent.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 29, 2014, 04:15:28 AM
Mac binaries updated, I just made multiple workflows for now, install the main one and any of the cs ones you want to use. If there are more options in the future, we may have to come up with a better way to select them. (some kind of embedded webserver might be cool)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 29, 2014, 07:37:34 AM
Cool!
Tested on my macbook pro 2.6 ghz 16gb ram 15 inch. The chroma smoothing is giving a not a super smooth playback(tested in mlrawviewer)but I,m not complaining :).
Just to make sure. There is still the exiftool temp issue as before converting dual iso files? Converted as before, still the poop temp file :)
Thanks for nice work David
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 02, 2014, 03:15:12 AM
I've implemented a very experimental dual ISO conversion (I really prefer to think of it more as a preview). It's nowhere near the quality of cr2hdr and probably not all that useable for real work yet, but it's realtime :)

It will try to convert dual ISO if the path name contains "DUAL", so put your dual ISO mlv files in a folder called DUAL or rename them with the word DUAL in them, to try it out.

There are all sorts of issues (like strange color artifacts, banding, bad aliasing, etc.) b/c of all the short cuts I take compared to cr2hdr (and b/c I'm no a1ex when it comes to this computational image processing stuff, I need to read the dual ISO pdf about 10 more times), but I feel like it might at least be somewhat useful for previewing and doing quick edits.

Anyway, I'm putting it out there b/c I don't really have much good test footage, so please do some tests and post some comparisons, and let me know what you think.

Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: senzazn12 on October 02, 2014, 05:25:20 AM

Quote from: dmilligan on October 02, 2014, 03:15:12 AM
I've implemented a very experimental dual ISO conversion (I really prefer to think of it more as a preview). It's nowhere near the quality of cr2hdr and probably not all that useable for real work yet, but it's realtime...

Where can we try this out? Thanks for providing this to us.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 02, 2014, 07:39:57 AM
Hi David. A realtime viewer on dual iso moviefiles. Not bad at all! Mlvfs should be in camera as well ;).
I did a fast comparison between cr2hdr and mlvfs. Don,t know if it,s relevant. Just two original dng;s and then applied the same settings lifting shadows and such.
I provide the dng;s in two links. Also linking to the original mlv file (about 350mb) if anyone wants to play around.
Off to work then.

screencaptures:
cr2hdr
(http://s30.postimg.org/dcwg4gc3x/0_1_cr2hdr.png) (http://postimg.org/image/dcwg4gc3x/)
mlvfs
(http://s30.postimg.org/7tyi0z4gd/0_1_mlvfs.png) (http://postimg.org/image/7tyi0z4gd/)
cr2hdr
(http://s30.postimg.org/60qzywtvx/0_2_cr2hdr.png) (http://postimg.org/image/60qzywtvx/)
mlvfs
(http://s30.postimg.org/lowsjfyvh/0_2_mlvfs.png) (http://postimg.org/image/lowsjfyvh/)

dng files:
https://drive.google.com/file/d/0B4tCJMlOYfirUnBtOHNDdGllTjg/edit?usp=sharing
https://drive.google.com/file/d/0B4tCJMlOYfirWFJaenVhTG50eWc/edit?usp=sharing

Mlv original dual iso file:
https://drive.google.com/file/d/0B4tCJMlOYfirMDg0VV84VlplV28/edit?usp=sharing
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 02, 2014, 12:59:22 PM
Quote from: senzazn12 on October 02, 2014, 05:25:20 AMWhere can we try this out?
Download links are in the first post. Mac version has been updated, ayshih will need to update the Windows version.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: escho on October 02, 2014, 08:46:17 PM
I tried to compile, but I run into an undefinied reference:

edgar@linux-indh:~/mlvfs/mlvfs> make
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 dng.c -o dng.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 index.c -o index.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 wav.c -o wav.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 kelvin.c -o kelvin.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 stripes.c -o stripes.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 cs.c -o cs.o
gcc -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 main.c dng.o index.o wav.o kelvin.o stripes.o cs.o -pthread -lfuse -lm -o mlvfs
/tmp/ccayy0ec.o: In function `mlvfs_read':
main.c:(.text+0x1b8b): undefined reference to `hdr_convert_data'
collect2: error: ld returned 1 exit status
make: *** [mlvfs] Fehler 1


What did I wrong?

OpenSuse13.1 64bit
Newest sources

Edgar
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on October 02, 2014, 09:11:00 PM
@escho: The Makefile had not been updated for compilation on Linux.  Try again now.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: escho on October 02, 2014, 09:14:25 PM
OK, compiles now.
ThankYou

Edgar
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on October 03, 2014, 03:30:10 AM
Quote from: dmilligan on October 02, 2014, 03:15:12 AM
I've implemented a very experimental dual ISO conversion (I really prefer to think of it more as a preview).
Quote from: dmilligan on October 02, 2014, 12:59:22 PM
ayshih will need to update the Windows version.
The Windows/PFM version of MLVFS now has the same (experimental) dual ISO support.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 04, 2014, 04:10:45 AM
some stability improvements and fixed the exiftool issue with cr2hdr --same-levels, Mac binary updated
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 04, 2014, 07:14:05 AM
Quotesome stability improvements and fixed the exiftool issue with cr2hdr --same-levels, Mac binary updated

Wow. Just like that :). Confirmed working. Tested on mac binary cr2hdr20bit. This would be a killer now with kitchehof lightroom plugin.
Can I ask you of a terminal command. Lets say I want to convert a bunch of dual iso folder and use some drag and drop command to terminal instead of selecting all files within folders and dragging those to terminal window. I tried a little with recursive command but I really don,t get it.

For now I 1) drag cr2hdr20bit to terminal 2) enter the folder, selects all pictures, drag those to terminal 3) enter.
What I like is able to select all folders and drag those to terminal.
Thanks David!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 04, 2014, 02:48:45 PM
You can use 'find' and 'xargs' to recurse into subdirectories:

find /path/to/parent/directory -name '*.dng' | xargs /path/to/cr2hdr --any-cr2hdr-options-here

This will convert all dngs anywhere inside /path/to/parent/directory. Is that what you're looking for?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 04, 2014, 03:15:51 PM
Thanks a lot, gonna check into this later tonight.
Dual iso conversion is a breeze now since you fixed exiftool. Great work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 05, 2014, 01:13:12 AM
Awesome!! With your script command I,m only one click away now. Created a bashscript to a folder I will use on my desktop. Thank you a lot. Most grateful.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 05, 2014, 04:13:00 PM
A little monkey wrenching on a late sunday afternoon :)
Scriptbased batchconversion of dual iso dng files.
http://youtu.be/J5JR3oCYcFE?list=UUomeOeghS6wanMOCQ8BtH_A
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 07, 2014, 01:05:26 AM
hello,
i really like the idea and workflow of this tool, but have just realized, that the free version of davinci resolve 11 has no hot cold pixel correction for raw files (as far as i know). so i wonder if  - hot/cold pixel correction will be possilbe in the near future?
i ask because i had a video shoot the other day, where i recognized a bad pixel in the middle of the screen and it is really annoying.

thank you for your time and engagenment. greets. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 07, 2014, 02:57:31 PM
should be doable, it's on my todo list
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 07, 2014, 06:08:21 PM
@dmilligan sounds great!!

thank you swinxx

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 08, 2014, 11:19:20 PM
bad pixel correction is implemented, please test

it's kinda slow (~2fps on my machine), IDK if there's much way around that, there's a couple more possible optimizations, but I doubt it will ever be realtime
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 08, 2014, 11:52:47 PM
thank you dmilligan, i will test it at the weekend, you are great!
that should help a lot!
so i see a seperate bad pix script, is there chroma smoothing and/or dual iso implemented too?

btw.. i was download number 1 :))

greets. sw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 09, 2014, 02:29:36 AM
Yes, chroma smoothing is implemented. There is also a dual ISO "preview", quality is not very good. It will run if you have "DUAL" in the path name.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 10, 2014, 03:11:32 AM
Well, there were getting to be too many options, and I wanted there to be a way to turn them off and on without restarting mlvfs. So I have embedded a little webserver (mongoose (https://github.com/cesanta/mongoose)) in mlvfs for configuring options at runtime (this also eliminates the need for all those different automator workflows). The new mac workflow will automatically popup the configuration page in the web browser when you start it. You can always get to it at http://localhost:8000 when mlvfs is running. This will be a great help in the future as options start to proliferate, it also may be useful for displaying status/error information.

@ayshih, mongoose should work on windows, but this may not be as useful since you have to multiple instances (and the webservers would conflict), so you may want to simply disable this with macros
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 10, 2014, 09:26:44 AM
Very clever. Working nicely storing settings.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on October 10, 2014, 11:37:35 AM
Sounds promising. Thank you for your hard work. Im looking forward to try the new version at the weekend;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 10, 2014, 10:00:09 PM
Hi dmilligan!

i have tried the new badpixel option, but here in my specific shot my bad pixel is not gone.
so i see no difference in on and off position.

please take a look here:
https://copy.com/LjWpLtXeqIrO0AtK

should i contact a1ex, or is my problem not a bad pixel?

thank you very much..

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 10, 2014, 11:00:11 PM
That pixel doesn't appear to be bad enough to trigger correction based on the current algorithm, because I've assumed a value for dark noise rather than calculating it (for speed), and this pixel was just barely below that threshold (so the algorithm basically thinks it's dark noise, not a hot pixel). The threshold was conservatively high, so I'll lower it a little. Thanks for the feedback.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 10, 2014, 11:54:51 PM
thanks dmilligan!

i´m looking forward for the update.

greets. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 11, 2014, 11:39:40 PM
I added the "aggressive" hot pixel correction, and lowered the dark noise estimate a little, hopefully this will pick up your bad pixels.

Also cleaned up the web GUI a little and added a metadata display table for MLV files.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on October 12, 2014, 08:22:36 AM
Great. Could there be a downside when using agressive hox pixel correction with other material?
I will try today. Thx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 13, 2014, 07:11:29 PM
@swinxx,
IDK

@all,
Full dual-ISO conversion is now working!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kgv5 on October 13, 2014, 07:18:25 PM
Wow, cannot wait to test it  :D  Is the first page download link up to date (for windows)?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 13, 2014, 07:20:25 PM
Quote from: kgv5 on October 13, 2014, 07:18:25 PM
Is the first page download link up to date (for windows)?
No, Windows version has not yet been updated, only the Mac version
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 13, 2014, 08:00:48 PM
@dmilligan

first, let me say thank you very much for your time and knowhow to provide a brilliant solution for converting the mlv files. this is a really great way to speed up the working process.

but for the first time some questions came up after installing it.. perhaps you could explain in a few words.

.) when i select the file and mount the drive (with an empty folder which i have created) can i swap and change options on the fly..? i mean now i see a new web interface (which looks really nice btw :))

so the configuration options are applied at the same time when selecting them? or when i remount them? when i have a dual iso file here, i can not see any difference between "OFF", "PREVIEW" and "FULL"?? i mean, i see a distorted image, please have a look at my posted link. thx

UPDATE: Ok, it confused me cause i tried with dual iso first and there are some issues for me? perhaps because of the resolution aspect ratio?

.) I have tried it with a mlv dual iso file, but had problems (image looks strange)
here is the posted image after conversion:
https://copy.com/WHHtwGsDkm84v8rl

.) is it possible to add a "APPLY" button, that we can be sure the settings are applied?

UPDATE: Ok, after more testing i see, they are applied on the fly :=)

.) althought the web interface shows shutter time 19ms i am quite sure that i shot with 50ms..?
could this be a tag issue? or my fault?

.) in the link from above, there is a directory with 2 tiff images, where i sadly can see, that the agressive hot/bad pixel algo is doing some weird things.. perhaps a1ex can take a quick look at it.


all in all, great tool!! dmilligan you are a goal getter :)
thank you,
greets. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on October 13, 2014, 09:28:41 PM
Quote from: kgv5 on October 13, 2014, 07:18:25 PM
Wow, cannot wait to test it  :D  Is the first page download link up to date (for windows)?
I haven't been able to work on the Windows/PFM version lately, so it's fallen a bit behind.  Unfortunately, that's likely to continue to be the case for the next couple of weeks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 13, 2014, 09:43:06 PM
Quote from: swinxx on October 13, 2014, 08:00:48 PM
.) althought the web interface shows shutter time 19ms i am quite sure that i shot with 50ms..?
could this be a tag issue? or my fault?
Are you sure you didn't mean 1/50s?
1/50s = 20ms
(it's 19 probably because of either rounding error, or 1/50 only being the approximate shutter speed)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 13, 2014, 09:50:49 PM
ah yes of course..  :D
1/50 for 24fps.. nearly 180 degree :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 13, 2014, 10:04:36 PM
Quote from: swinxx on October 13, 2014, 08:00:48 PM
.) I have tried it with a mlv dual iso file, but had problems (image looks strange)
It looks more like what I would expect the preview to look like. Perhaps you had the preview option enabled and loaded the image and it got "cached", so when you switched to "full" it just retrieved the cached preview. Just unmount and try again, select the "full" option before doing anything else.

If that's not it, could you upload the first little bit of the original mlv file?
hint: you can use mlv_dump to trim an mlv file to just a few frames:
mlv_dump -o output.mlv -f 5 input.mlv
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 13, 2014, 10:21:40 PM
thank you,
i will have a quick look now, and get back as soon in some minutes..

greets.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 13, 2014, 10:33:05 PM
Yeah man! This is nice. It is a cache thing. Got the dual iso "view" files converted looking fishy but after remounting I could start conversion with dual iso 20bit converter.
Are files hardcoded with 60 000 whitelevel for same level correction? How is that done? Where can I view the code? Curious noob I am.
Compared converting with my cr2hdr-r script but altough my computer was sweating like a pig your conversion was 20 seconds faster with the same laptop and it didn,t make a sound. Go figure  :P.
Wishing for raw support purely for dual iso purposes and until then monkeywrenching i still a good thing ;).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 13, 2014, 10:35:41 PM
hello dmilligan,

so i have tried it, and yes my fault, i had to reboot the computer, now it is working, although, when you take a quick look (link posted above) i have some strange artefacts in the converted file (in the window) green and pinkish..
the same file converted with the lr plugin has no artefacts.
thx. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 13, 2014, 11:08:57 PM
Quote from: Danne on October 13, 2014, 10:33:05 PM
Are files hardcoded with 60 000 whitelevel for same level correction? How is that done?
white level is just the one from metadata * 4

Quote from: Danne on October 13, 2014, 10:33:05 PM
Where can I view the code? Curious noob I am.
https://bitbucket.org/dmilligan/mlvfs/src/
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 13, 2014, 11:14:45 PM
Thank you. Heavy reading. A lot of stuff. Thanks for all your work on this.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 14, 2014, 01:47:14 PM
Quote from: Danne on October 13, 2014, 10:33:05 PM
Compared converting with my cr2hdr-r script but altough my computer was sweating like a pig your conversion was 20 seconds faster with the same laptop and it didn,t make a sound. Go figure  :P.
It's probably the fact that I'm not using the AMaZE based interpolation. I just wanted to get cr2hdr working internally as simple and stripped down as possible, then add stuff back in.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 14, 2014, 02:18:18 PM
Could be the case. It looked good though but have to test some more clips. Wasn, t amaze "fixing" jagged lines a little better?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 15, 2014, 11:12:08 PM
update: some performance improvements

(mostly if you have multiple cores and are using the processing heavy options, also requires the software accessing the files to try to load multiple files at once, in SpeedGrade there is an option you can set for this, the default is 2 frames at once, a Finder copy only does one file at a time, so to see improvement do multiple copies at the same time)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 16, 2014, 04:22:57 PM
whow. great thank you . will try tonight.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on October 16, 2014, 09:21:23 PM
Just tested out the latest update of MLVFS -- it is in fact quite impressive especially with the new GUI layout as well as the DUAL-ISO 20-bit option...

I tried to leave everything alone as much as possible beside 'Bad Pixel Fix' and 'Dual-ISO 20-bit' and the DNG's are coming out quite weird...

(https://farm4.staticflickr.com/3932/14930543134_47896f3511_b.jpg) (https://flic.kr/p/oKmWiu)
(https://farm6.staticflickr.com/5603/15364673089_dd9602bc90_b.jpg) (https://flic.kr/p/ppHY1H)

Definitely better than the previous version but it's not quite the same quality as if you were just to export as unconverted DNG's and then use a stand alone app (cr2hdr-r) through with Automator OR the 20-bit plug-in for LR5... either one of these gets the job well.

**ALSO when trying export files directly from the MLVFS mount folder -- it freezes up the Photoshop app and have to force to close BUT it seems to work just fine with LR5... just have to wait for it to finish load which takes forever! is there a reason for this?

(https://farm6.staticflickr.com/5601/14930627194_9c50f343c1_b.jpg) (https://flic.kr/p/oKnnhN)

The only question I have regarding this is whether or not should I apply the same levels through command because I'm not sure if MLVFS does this? (this is why I love the new cr2hdr-r workflow the @Danne created as well @dmilligan's merged into this 'Service' similar to MLVFS.

Unless I am missing something regarding the newly updated MLVFS...

THANKS ALL!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 16, 2014, 10:02:33 PM
@dmilligan

hi,
when i mount an folder it works but safari gives me this error:
https://www.dropbox.com/s/qiylsho8d5gor2u/Screenshot%202014-10-16%2021.59.12.png?dl=0

greets.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 16, 2014, 10:27:01 PM
@DeafEyeJedi, those look like focus pixels, you need to turn on chroma smoothing to get rid of them. Right now you might get some weird artifacts with chroma smoothing, I need to incorporate it into the correct sequence within cr2hdr stuff. There are also several cr2hdr options that I have not implemented yet. These options, when implemented, should improve image quality greatly, and match the result of the standard cr2hdr.

It's slow because Lr is trying to load the DNG, which means MLVFS has to convert it first. Conversion can take several seconds per frame. If you're going to use LR, I would recommend copying the DNGs out of the MLVFS mount into the real filesystem (permanently converting them).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on October 16, 2014, 10:32:43 PM
Note that focus pixels with dual ISO are still unsolved, at least in my implementation (didn't try MLVFS yet, but will definitely do it). So try not using both at the same time - I know you can, from your signature :P
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 17, 2014, 12:08:03 AM
Okay I implemented the chroma smoothing the same way it's in cr2hdr (chroma smooth fullres and halfres before final blending). It helps with the focus pixels, but doesn't remove them completely. If you do the chroma smoothing after cr2hdr is completely finished, the focus pixels are completely gone, but there are strange green blob artifacts.

I haven't updated the binaries yet (I'll wait till some more updates)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on October 17, 2014, 08:49:45 AM
copy that @dmilligan & @a1ex -- I'm off to bed and will get back on the boat w you guys over the weekend!

Thanks guys!

SJ
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PowerMac84 on October 18, 2014, 09:38:43 AM
Quite often OS X 10.10 during Convert with that AE script the temporary mount point of MLVFS disappears. I usually name a folder like "MLVFS temp folder". This looks like a volume which acts like a folder on another volume. But after some time it just disappears and also the Webserver is not more there. I then recreated it and let AE reconnect the files. After a while it happened again and the AE batch export gave me errors and exported error clips. Any ideas?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PowerMac84 on October 18, 2014, 09:43:33 AM
Well the mount point of that virtual MLVFS volume also appears under the devices of my computer. Which volume should be used for working with the CDNG files? That mount point under devices or the concrete folder (virtual volume) on a other volume?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 18, 2014, 03:19:46 PM
I've only just started testing on 10.10. If the mount point disappears then that means the program crashed.

I've never seen the FUSE mount show up under "devices", but I suppose it doesn't make any difference (it's probably still actually pointing to the same place, the folder where you mounted it to).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 18, 2014, 04:40:12 PM
@swinxx,
I think I fixed the issue with the "%@" error popup (this was an issue with automator in osx 10.10)

@PowerMac84,
I have also some stability updates, please try the latest version and let me know (I also recommend updating FUSE to the latest version, 2.7.1, if you're not already on the latest version). If it's still crashing, let me know all the settings you were using.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on October 19, 2014, 06:23:19 AM
Just downloaded the latest update few mins ago.. Thanks @dmilligan! Will let you know how it goes..

Question -- is it always reccommended to take the files out from the mount folder and into the file system (desktop, external drives, etc) as oppose to try running apps within the mount folder itself?

Perhaps it would run faster running off the system rather than or does that not matter?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on October 19, 2014, 06:26:22 AM
Copying the files out and not using the mount itself kind of defeats the whole purpose of this program.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on October 19, 2014, 09:28:30 AM
Hmm. @dmilligan:
I dont think so, how can i work with davinci resolve instead??
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 19, 2014, 09:42:39 AM
You can work with resolve straight from the mount. Worked last time I tried.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 19, 2014, 08:05:27 PM
yes, but when i start with a project, i would like to have all my working files collected in a special folder...
greets.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 19, 2014, 08:14:28 PM
Collect all mlv files in folders on your harddrive and go from there?
Greets
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: delrosestephen on October 20, 2014, 07:39:25 AM
I just wanted to say thank you for this. I recently built a Hackintosh and MLVFS saves me so much time with post. Davinci Resolve works beautifully with this :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 20, 2014, 08:51:55 PM
hello dmilligan!

i updated to yosemite yesterday, and now i can not mount any mlv files anymore..?
have tried it from my desktop, from another hard drive, no chance..
when i try to start the service, my safari web browser pops up but with no web server, there is the message that it is not able to load the page..?

update:
ok, i had a problem with some writing privileges.. ? don´t know why but before the update everything was fine. now i have corrected those things and my issues are gladly gone..

thx.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: yanone on October 21, 2014, 09:11:01 AM
Hi,

I just downloaded and tried MLVFS for the first time. The initial post claims that audio is working, but not for me.

An audio file appears in the file system, but is reported at 0 Bytes size and therefore doesn't play back.
Also I have just tried the MLV module for the first time, coming from RAW, because I need the sound, and have no experience with MLV yet. But I turned the sound module on, and hey, something is there, yet empty.

Any ideas? Thank you!

UPDATE:
Please forgive my post. I have just found that there is another setting in the sound menu that will activate the sound recording. I thought loading the module would already achieve this. Cheers...

Tiny work flow enhancement:
Would be fantastic if MLVFS could remember the mount path, or maybe its parent directory.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 21, 2014, 10:57:32 AM
@dmilligan

do you think that it would be possible to link audio and video - especially in resolve, cause i have here the folders only and must click into all the folders, where else other programs like rawmagic (i know that there is a license problem) links the files that when i import it into resolve audio and video files are linked and i dont see them seperatly :)

thank you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 21, 2014, 01:12:35 PM
From what I've been able to gather, this requires that the file and folder names conform to some special naming scheme and also the audio file has some special metadata in it. The metadata in the wav file shouldn't be too hard, but the naming scheme presents somewhat of a challenge because of the way mlvfs and the fuse api works. Naming the DNGs something different isn't a problem at all, it's the parent folder that is a problem because mlvfs uses the parent folder's name to find the original mlv in the real filesystem. Instead of a simple translation of the path name, we'll have to have some kind of complicated lookup table.

It's certainly possible, and something I'd like to get working, but it will be a lot of work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kgv5 on October 21, 2014, 01:55:55 PM
Guys, i am little confused, could you please tell me where is the most recent version for windows - is it the one mlvfs.dll in the first page? I know that windows version is behind the mac but i am going to test it a little bit and would like to work on the latest version available. Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 21, 2014, 02:01:53 PM
Quote from: kgv5 on October 21, 2014, 01:55:55 PM
is it the one mlvfs.dll in the first page?
yes
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kgv5 on October 21, 2014, 02:41:11 PM
Ok, great, thanks dmilligan  :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 23, 2014, 11:57:34 PM
Update: automatic audio linking in Resolve should now work (you need to select the "DaVinci Resolve" naming option). Even if you don't have audio, if you use the resolve option then your dng folders themselves should show up as "movies" or whatever in the Resolve file browser instead of showing up as a folder containing a dng sequence, which is handy. There is one drawback: you can't use the MLD folders or write anything into the mount (like .xmp files) so AE won't work in this mode (yet).
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on October 24, 2014, 07:33:19 AM
Dmilligan, you arr the man!! Thank you 1000x
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 24, 2014, 11:04:55 AM
That is cool.
Have a strange thing going on. When I select da vinci resolve in the interface the files vanishes from the folder. However. Tried running the mlv file with audio straight in da vinci resolve without selecting for it in the mlvfs interface and it worked anyway straight out of the box? Audio and picture. Maybe it is set to da vinci as default?
I used Da vinci relove lite 10, OSX maverick mac book pro 15 inch.
NIce work on this. Gotta start using da vinci resolve more often though.

Tried also installing the next latest update and audio wasn,t merged as with latest to rule out audio working before
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 24, 2014, 02:15:03 PM
QuoteThere is one drawback: you can't use the MLD folders or write anything into the mount (like .xmp files) so AE won't work in this mode (yet).

Actually, I can use both after effects(storing xmp to MLD folder) and Da Vinci Resolve with audio without selecting da vinci resolve mode in mlvfs. SWEET!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on October 24, 2014, 04:51:14 PM
The audio seems to show up either way, but if you use the naming scheme, the directories themselves just show up as clips (so you don't have to navigate down into them)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 24, 2014, 05:52:41 PM
Ok. Well, then the DaVinci Resolve selection isn,t working on my computer. The mount empties when selected unfortunately.

With davinci option checked
(http://s9.postimg.org/jmgtyxr6z/Screen_Shot_2014_10_24_at_17_46_46.png) (http://postimg.org/image/jmgtyxr6z/)

Entering folder with davinci resolve checked(mount also empty)
(http://s9.postimg.org/5hb0x4i5n/Screen_Shot_2014_10_24_at_17_46_53.png) (http://postimg.org/image/5hb0x4i5n/)


(http://s9.postimg.org/5wmaq542z/Screen_Shot_2014_10_24_at_17_47_12.png) (http://postimg.org/image/5wmaq542z/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 25, 2014, 10:22:50 AM
hello,
i do not understand.. so what´s the problem with mlvfs and davinci now? should i use the option "resolve" in the webserver? or not?
is it also working without it? sorry for asking but i can not test it at the moment.
thx sw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 25, 2014, 01:50:11 PM
With resolve option selected folder content is suppose to show the moviefile without entering any folder.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 25, 2014, 02:57:48 PM
Okay, there seems to be some sort of strange issue with Resolve. I noticed you're in the thumbnail view. When I try thumbnail view, Resolve actually completely crashes (it's not MLVFS crashing, in fact MLVFS is still running fine afterwards). In the list view everything is fine (which is what I was using when testing), and the folders show up as clips. Could you verify that in thumbnail view after it disappears for you that MLVFS is still properly running?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 25, 2014, 03:47:56 PM
Hi David. Still no change. MLVFS mount comes up empty upon selecting da vinci resolve mode. Davinci app no change when in list view.

some pics

Selected
(http://s15.postimg.org/byxxcemon/Screen_Shot_2014_10_25_at_15_43_45.png) (http://postimg.org/image/byxxcemon/)

Empty due to davinci selected
(http://s15.postimg.org/yc5nz7nmf/Screen_Shot_2014_10_25_at_15_42_33.png) (http://postimg.org/image/yc5nz7nmf/)

List view emptiness
(http://s15.postimg.org/izlx56ogn/Screen_Shot_2014_10_25_at_15_43_12.png) (http://postimg.org/image/izlx56ogn/)

List view emptiness 2
(http://s15.postimg.org/eezqq94rb/Screen_Shot_2014_10_25_at_15_43_33.png) (http://postimg.org/image/eezqq94rb/)


Folder reappears when davinci resolve option is not selected
(http://s15.postimg.org/7sd2x2n2v/Screen_Shot_2014_10_25_at_15_44_11.png) (http://postimg.org/image/7sd2x2n2v/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 25, 2014, 06:39:31 PM
found and fixed the problem, try again
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on October 25, 2014, 06:43:42 PM
Will try out tonight. Thx so much for all your hard work
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 25, 2014, 06:56:20 PM
You most certainly did :) That is so great!

(http://s30.postimg.org/4n4dovuhp/Screen_Shot_2014_10_25_at_18_53_15.jpg) (http://postimg.org/image/4n4dovuhp/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on October 26, 2014, 07:58:35 AM
Great news regarding Davinci Resolve. I ran out today to shoot a recreational soccer game with 5D3 and shot it all in RAW (few Dual-ISO's) just for the hell of it and it came out quite well, I believe.

I am currently using MLVFS to use  the DNG's in PP_CC(2014) which can read within from the mount itself (which is nice) and can actually playback smoothly in the sequence layout.

(https://farm4.staticflickr.com/3954/15629684635_449ba4ab81_c.jpg) (https://flic.kr/p/pP9dHi)

My only question now... how come audio isn't being linked together? Do i have to drag the audio folder into it or something?

Thanks again, David!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: spnsir on October 26, 2014, 06:47:51 PM
Hey guys,

Resolve audio linked is great but is not working for me. All of the files are loading with a linked audio file, but most of the audio is blank, and only some of my longer clips have the audio start towards the end of the file. I don't know if it makes a difference, but all the clips are shot at 50p. When I play each clip in the source window in Resolve, the audio is linked and works perfectly, but once I load it into the project there's no more audio, or there is but only in a few clips and it starts towards the end of the clip//isn't in sync.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 26, 2014, 07:15:06 PM
Must be a timecode issue. I never actually went so far as adding the clips to projects when I was testing this. I assumed that if they were correct in the browser they'd show up correct once you actually add them. I'll look into it, though, resolve is super slow on my system so it's hard to even tell if the audio is correct because playback is so slow, so testing is a pita.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 26, 2014, 07:42:11 PM
hello,

@dmilligan: thank you so much for the resolve update!!

however, i converted the same sequence 2 times,
first time, audio was right :)
second time, audio was blank in resolve (as described) but i dont know why, cause it was the same file. perhaps different conversion options

if i find out why it was different, i will post my findings..

UPDATE:
in resolve -> Media Tab -> click on the video file -> Play it back -> Sound is here perfectly.
Import into the Media Pool -> Double Click -> Preview with the play button -> Sound has gone..

thats really strange, so when i preview the video file before the import, i can hear the sound, when i import it and play it back from the media directory -> the file is silent.


greets. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 27, 2014, 10:13:54 PM
@dmilligan

i also found out that in resolve after importing the footage into the media pool, the audio is no longer linked.
when i put the video file into a timeline, a wrong audio file (which is empty) is also placed within the video file onto the timeline, but it is not linked to the video file (it should)
normally i can not take the audio file and drag it to another position - cause it is locked to the video footage.
so to sum it up, at the moment audio files are not imported correctly into resolve, and they are also not linked.
the second interesting obervation is here:
https://www.dropbox.com/s/jnqym9j6wawkl3p/Screenshot%202014-10-27%2021.26.57.png?dl=0
https://www.dropbox.com/s/2ix9qfusvvqmrzb/Screenshot%202014-10-27%2021.26.47.png?dl=0

there are some dots jumping around in playback, they are strange and only in the black areas, but not in all clips.
my options within mlvfs were: resolve naming/Vertical Stripes Correction On/ Chroma Smoothing 2x2

thank you so much, greets. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 27, 2014, 10:30:04 PM
i can confirm that with rawmagic (sorry but great converter) audio works in resolve as expected.
rawmagic is a little bit brighter, and it has no jumping dots,
mlvfs is darker (dont know which one is right here), and has jumping dots in some darker areas of some (not all) shots

here is the link with 5 dngs from rawmagic (vertical stripe correction on) and from mlvfs (resolve on, vertical stripes correction on, 2x2 on)
perhaps you can find the missing information there..

https://copy.com/YOHU3WPaXbE7YT3q

greets. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 27, 2014, 11:12:10 PM
The audio thing does appear to be a timecode issue. The DNGs from RAWMagic has the timecode tag. This must need to match the timecode in the wav file. I have no idea how to do this. The CDNG specification merely references "SMPTE 331M-2004", which is not freely available, so I do not know how this field is supposed to work. Maybe @chmee can shed some light?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on October 28, 2014, 01:04:31 AM
Definitely makes sense with the timecode tag...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Audionut on October 28, 2014, 02:04:52 AM
Page 10 of this (http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/cinemadng/pdfs/CinemaDNG_Format_Specification_v1_1.pdf) gives an example.  :-\

Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on October 28, 2014, 02:53:32 AM
Yeah I know, that's the CDNG spec I'm talking about, no real helpful information just a reference to the smpte spec and some meaningless table. None of the things in that table are explained, it is assumed you know what they mean from the smpte spec, which I don't have and isn't freely available. If you can explain it to me feel free...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Audionut on October 28, 2014, 05:16:21 AM
Figured as much, but that table looked so pretty, I figured someone smarter then myself might be able to decode it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chmee on October 28, 2014, 08:23:35 AM
as described in this thread (http://www.magiclantern.fm/forum/index.php?topic=11520.0) (and this (http://www.magiclantern.fm/forum/index.php?topic=10861.0)), there are some more metadata in the wav-file to give it a timecode. cdng has its timecode, yes, and this is quite easy to transform (look on main code raw2cdng, line 1088 (https://bitbucket.org/chmee/raw2cdng/src/772e76f0a55f18a0c83fb0ac5c45d88b2324ea98/csharp2010_code/raw2cdng_v2/MainWindow.xaml.cs?at=master) and calc.cs), but the wavfile needs a timecode in units of samplerate*frames(daytime)
long timeRef = (long)(mData.data.audioData.audioSamplingRate * time2frame);
the last function in my code io.cs (https://bitbucket.org/chmee/raw2cdng/src/772e76f0a55f18a0c83fb0ac5c45d88b2324ea98/csharp2010_code/raw2cdng_v2/io.cs?at=master).. my converted files does NOT behave like blackmagic-files, which a path looks like a mediafile - our cdng-folders look like folders with audio and video inside - but they react according on auto-sync.

regards chmee
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 29, 2014, 01:19:20 PM
thanks @chmee, I think I figured out the DNG timecode, Resolve is still not working though, perhaps I'm just not setting them to the correct thing. The DNG timecode doesn't need the date part, correct (last 4 bytes)? And it should be set to the time of day? or is it just relative to the start of the sequence?

perhaps you have an example DNG header and wave file header that work correctly (just a screenshot of the hex is fine)?

Quote from: chmee on October 28, 2014, 08:23:35 AM
my converted files does NOT behave like blackmagic-files, which a path looks like a mediafile - our cdng-folders look like folders with audio and video inside - but they react according on auto-sync.
does this mean that auto-sync doesn't work if I try to make them show up like blackmagic files? or are you just saying that you don't do it like that?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chmee on October 29, 2014, 01:50:16 PM
(will search for examples..) So, yes, i take the created/modified-time of file (this is the datem/time of recording) as startpoint, the timecode for first picture.
Quote..doesn't need the date part..
i didnt used it. maybe its important for datechange (midnight)..

[Resolve][MediaExplorer] cant tell you. my knowledge so far is simple. if i use according naming-convention [name]_C001_0000_[framenumber] and path [name]_C001_0000, it will show up as a mediafile instead of a folder BUT only if theres no audiofile inside. just wanted to say, my convertions are auto-syncable, but they dont show up as mediafile. something i couldnt figure out.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 29, 2014, 03:06:23 PM
Quote from: chmee on October 29, 2014, 01:50:16 PM
it will show up as a mediafile instead of a folder BUT only if theres no audiofile inside. just wanted to say, my convertions are auto-syncable, but they dont show up as mediafile. something i couldnt figure out.
well, I guess I figured it out, though I'm not really sure what exactly makes it work, but I use the naming convention and the audio files and DNG sequence show up as a single mediafile and play correctly in the media browser, however, when you actually import them to the timeline, the audio doesn't work.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: chmee on October 29, 2014, 03:37:27 PM
Enlighten us :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: spnsir on October 29, 2014, 03:55:53 PM
from my experience, it seemed like the audio is properly linked in the timeline, just the timing is not right. it is starting up late, towards the end of the clip. but not sure why the audio timing is correct in the source window.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: spnsir on October 29, 2014, 05:43:46 PM
In Resolve, if you right click and select "Add into Media Pool With Offset" and then enter in a value around 3500, the audio should be close to sync. I'm not sure if 3500 will work for you guys but it worked with multiple clips for me.

Note: it doesn't put the audio perfectly in sync so this isn't a usable workaround unless you want to fiddle with finding the exact offset for each clip, but I think this may help dmilligan get closer to figuring out what's causing this.

Screenshot:
(https://www.dropbox.com/s/ocrwztlbvcais28/Screenshot%202014-10-29%2012.39.39.png?dl=1)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on October 29, 2014, 11:06:29 PM
Okay well it seems to work if I just use 0 for the start time code in the DNG and WAV. Mac download updated so try it and let me know if it works.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: spnsir on October 30, 2014, 05:50:15 AM
So far so good, working here! Thank you for the quick fix David!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 30, 2014, 07:06:54 AM
YES!! Great! It WORKS!!! BBIIGG TTHHHUUMMBBSS UP!! This is really great and a big step forward.

btw. i have found out that chroma smoothing (especially 2x2) causes those jumping hot? pixels in the dark areas..
can there anything be improved? perhaps from a1ex code?
thx swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on October 30, 2014, 08:10:26 AM
Nice work :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chmee on October 30, 2014, 11:06:40 AM
could someone show me a screenshot with the file and path? am i right, the filenaming-convention for

path is [name]_1_yyyy-mm-dd_0050_C0000
dng is [name]_1_yyyy-mm-dd_0050_C0000_[sequencenumber].dng
wav is [name]_1_yyyy-mm-dd_0050_C0000.wav

@swinxx
i assume chroma smoothing and banding are time consuming, so it wont be realtime? prove me wrong. if it is rt (and the dng's look right in premiere as well) its time to stop developing my tool. because the filesystem-idea is much better than the converting-thing.

regards chmee
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 30, 2014, 01:16:42 PM
Actually the filename convention seems to have nothing to do with the audio showing up correctly, even if I don't use the filename convention at all, the audio and dng sequence still show up as one media file. The only thing the filename convention really does is make the folder itself show up as a media file. I think it has to do with what the extra metadata in the wav file is. See my code (wav.c) for what I'm doing. I actually got most of that information from you (the bext and iXML stuff), so I'm surprised to hear it doesn't work for you.

chroma smoothing is quite slow, but I think banding is realtime or almost realtime if your computer is fast enough (it only calculates the coefficients once for one frame and then reuses them for the rest of the frames, it's the calculating that's slow, applying the correction is quite fast, all we have to do is subtract-multiply-add integers).

One thing you can do though, is to simply not use any of slow options until final render. So you can edit, CC, etc. until you are completely finished, then turn on the options, start the render and let it chug away (just make sure to clear any render caches when doing this).
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on October 30, 2014, 01:27:38 PM
@chmee
I have a fast computer;) so it runs nice also with chroma smoothing options on, the only thing is that thise pixels are spinning around, so it is not possible to really use it. I wonder that no one else had seen them before..?

@dmilligan:
So if i have a directory, with mixed mlv files (normal and dual iso ones) how would i mount them? All together or dual isos seperately?  Would it be a smart addition to add the possibility for making a file dependent dual iso on off option in the webserver??
Thx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 30, 2014, 02:00:05 PM
It should be able to detect pretty quickly that it's not dual ISO. It may hurt performance slightly to have dual ISO turned on for non-dual ISO files, but I think it will probably still be realtime. Just try it and let us know.

I don't really understand how the chroma smooth algorithm actually works because I've never dug into the code. I just implemented the code straight from cr2hdr/mlv_dump. So I have no idea what could be causing the artifacts you're seeing or how to fix it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chmee on October 30, 2014, 02:53:40 PM
@dmilligan
thanks. by now the main difference - it seems - is, you are starting on 00:00:00 with timecode, will do some tests.

@swinnx
[fast pc] thats good for you, but it limits the usability for the slower pc owners. indeed, it should be an option, enabling cs and vb.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 31, 2014, 08:57:54 AM
@chmee.
yes fast read speeds (i have a raid system) are very useful for working with raw video. the write speed doesn´t matter, cause when delivering a video or playback a sequence, the limiting factors are gpu and cpu. and i have trained myself to cut before denoising and cc.

what i do not understand is, why do so less people download that tool... i think the problem is that they simply are afraid of a complicated workflow. all i can say to all the magic lantern users out there.. just try that program, the workflow is sooo simple and soo fast! the best program out there..

what i do not understand when working with resolve is that how do you handle the mounting points? do you apply the service at the main video folder where all the different directories with the mlv video files are placed into or do you apply more than one mlvfs service (one for every project, or one for every scene?) i think when this mystey is solved mlvfs will probably my solution with no proir conversion. so far i always copy the files out of the mounting point to a dedicated directory. - i like to be on the safe side..

@dmilligan
why are there 2 different naming schemes? isn´t it possible to work with resolve naming scheme in ae or premiere?
I have a scene of the moon (will update this post with the link to dngs as soon as possilbe).. there are 3 white pixels which are bad or hot or whatever pixels.. with option 1 (Bad Pixel Handling) they simply do not go away.. but it is a black scene and the pixel is white..
when i use agressive Bad pixel handling the pixels go away but the moon surface gets really strange.. the algo destroys the moon surface.. so is it possible to adjust the threshold a bit? perhaps with my delivere dng..?

All in All Thank You for you HARD WORK!!
I LOVE THIS TOOL SO MUCH! Everything is working so fine now (except ChromaSmoothing and Bad Pixel handling)

Now MLV is a breeze to work with.

@TO ALL PEOPLE -> https://bitbucket.org/dmilligan/mlvfs/downloads (https://bitbucket.org/dmilligan/mlvfs/downloads) AND TRY THAT TOOL.. ITS AWESOME!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on October 31, 2014, 08:58:28 AM
Can somebody help me ?

I can't compile MLVFS under Mint 17 x64...
When i try, i get a lot of errors.

gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 stripes.c -o stripes.o
In file included from mlvfs.h:25:0,
                 from stripes.c:26:
raw.h:167:5: error: unknown type name 'uint32_t'
     uint32_t api_version;           // increase this when changing the structure
     ^
In file included from mlvfs.h:26:0,
                 from stripes.c:26:
mlv.h:42:5: error: unknown type name 'uint8_t'
     uint8_t     blockType[4];
     ^
mlv.h:43:5: error: unknown type name 'uint32_t'
     uint32_t    blockSize;
     ^
mlv.h:44:5: error: unknown type name 'uint64_t'
     uint64_t    timestamp;
     ^
mlv.h:48:5: error: unknown type name 'uint8_t'
     uint8_t     fileMagic[4];    /* Magic Lantern Video file header */
     ^
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on October 31, 2014, 09:26:07 AM
Oops.  For Linux, in "mlvfs.h", you need to add an include prior to the include for "raw.h":
#include <stdint.h>
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on October 31, 2014, 10:08:39 AM
Better !  :)
But...   :(

gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 dng.c -o dng.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 index.c -o index.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 wav.c -o wav.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 kelvin.c -o kelvin.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 stripes.c -o stripes.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 cs.c -o cs.o
gcc -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64   -c -o amaze_demosaic_RT.o amaze_demosaic_RT.c
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 hdr.c -o hdr.o
hdr.c: In function 'identify_bright_and_dark_fields':
hdr.c:553:13: warning: variable 'changed' set but not used [-Wunused-but-set-variable]
         int changed = 0;
             ^
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 histogram.c -o histogram.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 mongoose.c -o mongoose.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 webgui.c -o webgui.o
gcc -c -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 resource_manager.c -o resource_manager.o
gcc -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 main.c dng.o index.o wav.o kelvin.o stripes.o cs.o amaze_demosaic_RT.o hdr.o histogram.o mongoose.o webgui.o resource_manager.o -pthread -lfuse -lm -o mlvfs
main.c:33:18: fatal error: fuse.h: No such file or directory
#include <fuse.h>
                  ^
compilation terminated.
make: *** [mlvfs] Error 1
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on October 31, 2014, 10:58:30 AM
Found !
Need to add libfuse-dev

Now it compile.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on October 31, 2014, 12:23:06 PM
But webserver doesn't work under Linux.  (http://localhost:8000)
nmap report the port open 8000/tcp open  http-alt
but nothing...

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 31, 2014, 08:34:43 PM
@swnixx,

There are two different naming schemes in case anyone was using the old naming scheme and had imported stuff into whatever projects => with a different naming scheme none of their paths would be correct anymore and they would have to re-link everything. It's also much more compact, those huge long filenames are kind of annoying if you don't need them.

Quote
what i do not understand when working with resolve is that how do you handle the mounting points?
Handle it however you like, you can run multiple MLVFS instances if you wish. If you like a project to be contained entirely within a directory you could do something like this:

~/projects/project1/                                         <-root dir of project
~/projects/project1/project1.prprj                     <-project file for whatever NLE
~/projects/project1/other_media/
~/projects/project1/mlv/                                  <-original mlvs here
~/projects/project1/mlv/scene1/M00-0001.MLV
~/projects/project1/mlv/scene1/M00-0002.MLV
~/projects/project1/mlv/scene2/M00-0003.MLV
~/projects/project1/mlv/scene3/M00-0004.MLV
~/projects/project1/mlvfs/                               <- mount point

Whenever you're working on this project, mount "~/projects/project1/mlv/" to "~/projects/project1/mlvfs/" and import from "~/projects/project1/mlvfs/..." directly inside your NLE.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on October 31, 2014, 10:36:45 PM
@dmilligan
thank you for your detailed explanation. that makes sense.
will try it.
greets. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on November 02, 2014, 08:25:42 PM
Quote from: vertigopix on October 31, 2014, 12:23:06 PM
But webserver doesn't work under Linux.  (http://localhost:8000)
Sorry, I'm still out of commission with respect to working on ML or MLVFS for at least another week, so I haven't been able to maintain the Linux version.  I encourage you (or others) to try to figure out why it's not working!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 05, 2014, 11:23:55 PM
updates: hopefully improved hot pixel detection, try it out and let me know, also if you have some MLV samples with hot pixels that would be helpful (post them here: http://www.magiclantern.fm/forum/index.php?topic=11899.0)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on November 06, 2014, 03:47:14 AM
Thanks @dmilligan for the update -- will definitely try them out and let you know!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on November 06, 2014, 09:06:40 AM
@dmilligan:
I had a problem where only the bad pix slgo worked, so i will try it out again.
Thank you!! That is awesome.

swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 06, 2014, 08:57:53 PM
hello dmilligan!

i have found a bunch of old raw files and wanted to throw them into a mlvfs service.. but they are not displayed. now i realized that the "mlvfs" only supports mlv files..
will your service ever support raw files (the old v1 mlv) or does anyone know a possibility to convert some raw files to mlvs.

thx in advance, swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 06, 2014, 10:47:08 PM
hello dmilligan again.

i would have another question regarding your newest update with the new bad pixel adaptation:
when i deselect the bad pixel option in the webserver, will the service try to remove some pixels anyway?
i ask because my video seems clean now, without anything selected :=)

thank you very much
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 06, 2014, 11:23:22 PM
Quote from: swinxx on November 06, 2014, 10:47:08 PM
will the service try to remove some pixels anyway?
No

Quote from: swinxx on November 06, 2014, 08:57:53 PM
will your service ever support raw files (the old v1 mlv)
IDK. Probably not. I want to encourage people to use MLV and not RAW. It would also be a lot of work. There's lots of important metadata missing too.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bennyray1 on November 07, 2014, 01:05:25 AM
Not sure exactly what happened but only up until this latest update MLVS is not showing up in my services after right clicking on my .MLV files...is anyone else experiencing this? @dmilligan is there away I can start over with a fresh install, instead of replacing MLVS? Running on OS X 10.9.3,I even updated OSX Fuse.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 07, 2014, 02:15:06 AM
In the Finder menu click on Go > Go to Folder... and type ~/Library/Services/
You should see the MLVFS.workflow there. You can delete it and try installing it again. I sort of doubt it will help.

Quote from: bennyray1 on November 07, 2014, 01:05:25 AM
after right clicking on my .MLV files...
Just to be sure, you are right-clicking a folder right? and not the actual files? You mount a folder containing files, not the files themselves.

(I'm not seeing any issues with the automator service on my end)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bennyray1 on November 07, 2014, 04:18:21 AM
Thank you sir, I was simply trying it on the file its self, it had been a while since I used it and forgot to click on the folder and not the file...thank you very much, working like a charm sir..thank you for the update, everything is working perfect sir.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: silvertonesx24 on November 08, 2014, 08:32:30 PM
Would it be possible to integrate this workflow with split files- like M00, M01, M02, etc?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on November 08, 2014, 10:04:11 PM
Should already be working with split files automatically no problem (and has been since very early on)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 08, 2014, 10:58:44 PM
hello dmilligan:

i just wanna report that your tool rocks. everything works as expected and better :)
no problems so far, the only thing which would make it better would be a possibility to save my preferences..

e.g.: i would like to always have davinci and vertical stripes correction turned on :)
is that possible?

thx. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 08, 2014, 11:15:34 PM
You can modify the automator service and add command line options. Open the MLVFS.workflow with Automator and add options in the "Run Shell Script". So you would change the default:

nohup ~/Library/Services/MLVFS.workflow/Contents/mlvfs "$2" --mlv_dir="$1" -f > /dev/null 2>&1 &


to add whatever options you want after the -f:

nohup ~/Library/Services/MLVFS.workflow/Contents/mlvfs "$2" --mlv_dir="$1" -f --resolve-naming --stripes > /dev/null 2>&1 &


here's the complete list of options (in main.c (https://bitbucket.org/dmilligan/mlvfs/src/e30abe780435edaee54c94a762bbc4173f88e303/mlvfs/main.c?at=master)):

static const struct fuse_opt mlvfs_opts[] =
{
    { "--mlv_dir=%s", offsetof(struct mlvfs, mlv_path), 0 },
    { "--port=%s", offsetof(struct mlvfs, port), 0 },
    { "--resolve-naming", offsetof(struct mlvfs, name_scheme), 1 },
    { "--cs2x2", offsetof(struct mlvfs, chroma_smooth), 2 },
    { "--cs3x3", offsetof(struct mlvfs, chroma_smooth), 3 },
    { "--cs5x5", offsetof(struct mlvfs, chroma_smooth), 5 },
    { "--bad-pix", offsetof(struct mlvfs, fix_bad_pixels), 1 },
    { "--really-bad-pix", offsetof(struct mlvfs, fix_bad_pixels), 2 },
    { "--stripes", offsetof(struct mlvfs, fix_stripes), 1 },
    { "--dual-iso-preview", offsetof(struct mlvfs, dual_iso), 1 },
    { "--dual-iso", offsetof(struct mlvfs, dual_iso), 2 },
    { "--amaze-edge", offsetof(struct mlvfs, hdr_interpolation_method), 0 },
    { "--mean32", offsetof(struct mlvfs, hdr_interpolation_method), 1 },
    { "--no-alias-map", offsetof(struct mlvfs, hdr_no_alias_map), 1 },
    { "--alias-map", offsetof(struct mlvfs, hdr_no_alias_map), 0 },
    FUSE_OPT_END
};


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 08, 2014, 11:31:50 PM
thx. that worked flawlessly :=)

greets. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mageye on November 09, 2014, 12:07:31 PM
I feel pretty dumb :( ::). I'm getting the same problem as I got some time ago (http://www.magiclantern.fm/forum/index.php?topic=13152.msg129070#msg129070 (http://www.magiclantern.fm/forum/index.php?topic=13152.msg129070#msg129070)) where you go through the normal procedure of mounting the the MLV folder and assigning mount point.

Then it appears to have mounted the folder and shows the server on local host etc. But as soon as I click on the folder it it disappears into the ether? Also the mount point folder disappears?

This is with the latest MLVFS from the first page here. Running on Mavericks 10.9.5 with FUSE for OSX 2.7.2

Could you be so kind as to tell me where I am going wrong? It's probably something I am doing. Also is it likely that Paragon NTFS for Mac could, in any way, interfere  with this?

I apologise for my stupidity! :D
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on November 09, 2014, 02:10:30 PM
If the mount disappears, that means MLVFS crashed. Perhaps it's a particular MLV file that it's having trouble with. Can you try and narrow it down some? If you can't, I can work on a version that logs some diagnostic info to a file so we can figure out why it's crashing.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mageye on November 09, 2014, 02:50:43 PM
Thanks for the reply. :)

Well first I thought that I should get rid of the *.idx and *.MLD files from the folder so as to allow these to be rebuilt if necessary. So I tried with a folder with lots of *.MLV files and when I mounted it rebuilt the *.idx files.

After that I went to browse and I get the same problem :(.

So I decided to try it with another folder with just a few *.MLV files contained. Unfortunately the same again. :( ???

Yes I think perhaps a way of logging this would be useful to see what's going on here?

EDIT: I found some crash logs for MLVFS in folder /Users/[USERNAME]/Library/Logs/DiagnosticReports/

Do these help?

https://mega.co.nz/#!MoMxGBab!4-4a_ffgtY1T5I9uW25X3spt-TF8C2K5ZxXv52vbqa0
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 09, 2014, 08:10:52 PM
Thanks, the crash log is very helpful. The issue appears to be related to some file path. What is the exact path to the directory you are trying to mount (is the full path really long)? Also, could you please type the following in the terminal:

ls -la /path/to/your/mlv/folder/

and send me the output[/code]
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mageye on November 09, 2014, 09:36:08 PM
OK I don't have a particularly long and convoluted path it's: /Volumes/Storage2.5TB/Media/MLV/ (in fact quite logical I think!)

Anyway here's the output from the terminal:

https://mega.co.nz/#!8wFhlRpJ!Ogjh0zzojGIspGFOBjcjc_dDFQGSx-bksllsRgew_jc (https://mega.co.nz/#!8wFhlRpJ!Ogjh0zzojGIspGFOBjcjc_dDFQGSx-bksllsRgew_jc)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 09, 2014, 10:22:37 PM
Might have something to do with the "." in your file path. Try a source path that doesn't have a dot in it. Doesn't appear to be anything wrong with the files themselves, but there are a whole lot of them. If it is an issue with a particular file there's no way to know which one. Try with just one file in a clean directory with no dots in any of the source paths. I think I've found and fixed the issue but I just want to make sure it is in fact the dot that's causing the problems.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mageye on November 09, 2014, 10:39:11 PM
Well aint that interesting!

I just dropped the dot from the drive name and now it's staying mounted and appears to be working. Thank you for your time on this! :D

Anyway why I think it's interesting is that my drive has been called that for some time now and I haven't had that problem before? (apart from the one time I mentioned ::)).

Also, yes there are lots of files in there, but that's not been a problem before either? I did try earlier with just a few of those copied to a folder and I got the same thing, so it wasn't to do with the amount or crappy files.

But what is good is that it works finally! ;D :D

So thanks again. I owe you a beer ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 10, 2014, 01:17:11 AM
Okay I uploaded a new version, it should handle dots in the file path properly and also very long filenames (even though that doesn't seem to be the issue here). Please let me know if it works.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: epyonxero on November 11, 2014, 12:15:09 AM
I just went to give this app a try on my Mac running Yosemite. Installed FUSE and created a new, empty folder on my desktop to be the container. When I select the folder containing MLV files the folder becomes a FUSE drive and my browser tries to open the LocalHost but the browser window fails and the FUSE drive is empty.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on November 11, 2014, 12:22:53 AM
Sounds like maybe you did it backwards. Right click on the folder with MLV files in it and run the service, when the dialog pops up to select a folder, pick an empty one to be the mount point.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on November 11, 2014, 12:38:05 AM
Hello dmilligan!
Everything works so great with your provided service. I wonder if there are any other features planned for future updates?
Greets swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mageye on November 11, 2014, 12:46:04 AM
@dmilligan

Yes thanks again :). It's all good now. There were a few other programs which referenced the Volume name with the dots in, so I'm glad you made that possible!  ;D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 13, 2014, 09:07:15 PM
hello dmilligan.

omg. suddenly my mlvfs mounting procedure is not working anymore? the browser opens and says can not connect to site bla bla bla.
i have made a complete transfer with all my media data to a new raid system, but now i do not get it to work anymore. the system itself has not changed.
any idea?
how can i completely remove and install it again?

Update:
works on the desktop..
but does not work on my new raid drive.. name is in capitals.. nothing else changed.
the old working path was /Volumes/Daten/\[FILM\ SCHNITT\]/01_CODED_2014-06-15_MONOBIT/2_MEDIA/Raw/Kurhaus\ MLV
the new one is /Volumes/THUNDER/\[FILM\ SCHNITT\]/01_CODED_2014-06-15_MONOBIT/2_MEDIA/Raw/Kurhaus\ MLV

as you can see the only change was from drivename "Daten" to drive name "THUNDER"

UPDATE 2: puh, now it works, had to make it read and writeable for everyone..


thx. sw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 13, 2014, 10:58:44 PM
Hi!
Did a quick check and gotta ask this. So, if using dual iso I can actually preview the files in real time and cut, edit etc and before export I check the full cr2hdr20bit and start my exports? Like proxy editing? I mean, I thought I first had to convert my dual iso dng files to regular dng? This is amazing really if it works straight on the timeline. Can David or Swinxx confirm. Or I try more tomorrow. THis sure would be good news.

*So it seems christmas is early this year :).
Quick question. Is it not possible to export to prores through Mlrawviewer in conjunction with mlvfs? I hit the E button but nothing. What are other experiencing?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 13, 2014, 11:22:55 PM
Quote from: Danne on November 13, 2014, 10:58:44 PM
preview the files in real time and cut, edit etc and before export I check the full cr2hdr20bit and start my exports? Like proxy editing?
That's the idea. Assuming that the dual ISO preview is fast enough, and hopefully the preview is close enough to the full dual ISO output that it's useful (albeit at a much lower quality and having issues/artifacts). Let me know if it works for you like this.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 13, 2014, 11:29:15 PM
Did a quick test in davinci and the preview worked really nice. Only had a short clip so gonna collect some more files to check on this more thoroughly. Crazy good stuff. Opens up some real oppurtunitys with dual iso workflow.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 14, 2014, 03:46:46 PM
@dmilligan:
what i have realized while comparing some dual iso shots with danne´s tool cr2hdr-r is, that converted dualiso dngs do not get displayed properly with the finder preview on mac osx 10.10 ( don´t know if it works on 10.9)
you can take a look at the dual iso dngs after mlvfs conversion here:
https://copy.com/wWXYBHbJwMHyi4Z2

there is a pinkish cast in the frame of the window. although davinci does a great job, it is not able to cancel that out completely. all in all it was much more present in the past conversion attemps with other tools like a1ex command line tool for dual iso conversion.

in the following link i made a short clip, which represents the major problem with my dual iso shot, which was used in a problematic situation (i know)...
https://copy.com/0nLwfWmR7Ebxy9FW
dmilligan, perhaps you or a1ex could tweak the conversion code cause that cast is flickering.. perhaps it can be cancelled out completely. that would be a major step forward, especially for dualiso movie mode.

greets.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on November 14, 2014, 05:00:31 PM
This is super cool =). I've been trying it out on windows with pismo file mount program. Is it possible to control hotpixel fix and stripe removal in windows?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 14, 2014, 06:57:14 PM
Hi David!
I played a little with the dual iso functionality and as I am a little lazy learning davinci and still wait to get my license for latest adobe premiere I have no deep analysis here.
So. I might be the only one creaming out prores files through AE and so I tried this today using mlvfs. It opens up fine but what I noticed it was lagging my computer considerably. I then started a short export and it was very slow. Also tried drag folders to desktop and it took a long while for export. Is it the cdng process taking som extra time or something else? Amaze imlementation? Maybe my computer? I,m on a macbook pro retina 15,5, 16gb ram ssd harddrive.
I havn,t done a proper comparison yet with a1ex original binary.
Thanks for all your help around here.
/D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 14, 2014, 07:28:09 PM
For full dual ISO processing it usually takes about 1.5 seconds per frame on my computer. If you turn AMaZE off, it's usually about half that or faster. CDNG doesn't take any extra time. I also haven't done a speed comparison with cr2hdr. The dual ISO preview should be realtime or close to realtime.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 14, 2014, 07:30:43 PM
Ok, thanks. It seems about right. I,ll try to do some comparison. The issue would be the lagging effect which I,m gonna try to narrow down.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 15, 2014, 04:19:27 PM
The reason it's "lagging" is because the Finder tries to load the file to show you a preview, which causes MLVFS to do a full render. This takes several seconds if you have dual ISO "Full" on. I would recommend using a Finder view that doesn't show previews, or avoid browsing around in the Finder once you turn the dual ISO on "Full".

I did some investigating on the speed issues as well. On a test image MLVFS is taking about 6.5 seconds to render a single frame, while cr2hdr20bit only takes about 3. Of the 6.5 seconds, 5.2 is spent doing the AMaZE interpolation. If you use mean32 instead of AMaZE, it only takes 1.5 seconds and the mean32 interpolation part only takes 0.2s. I don't really know why MLVFS's implementation is so much slower, could be a compiler optimization thing, who knows.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 15, 2014, 04:32:40 PM
Thanks for sharing. Maybe implement an cr2hdr20bit export option with possibility to multithread? Might be messy and not in philosophy of the mlvfs workflow but it might speed up things a bit.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 15, 2014, 10:11:11 PM
@dmilligan

would it be possible to implement an "EJECT" button in the webserver?
that would be handy, cause in the finder that eject is very laggy and non responsive.

Update:
Sorry Mate, i have not understood the dual iso thing completely.. so my question:
would it be a good idea, to let mlvfs search for a folder called "dualiso" and let only those mlvs be processed like dual iso mlvs? i ask this because dual iso conversion is a slow process and when those shots are placed in a special folder within the mounted folder, mlvfs could speed up conversion process...? no?

thank you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Audionut on November 15, 2014, 11:19:23 PM
Are the SSE2 additions to AMaZE working as expected?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on November 15, 2014, 11:59:22 PM
While AMaZE itself is somewhat slower, the vast majority of the slowdown appears to be coming from cr2hdr code that does stuff with the AMaZE output, not AMaZE itself. IDK if that stuff is being vectorized or can be. Perhaps I should try compiling MLVFS with GCC and see if it makes any difference.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andrzej on November 17, 2014, 06:05:02 AM
Hi, i am geting: "safari can't connect to server"
sys wer 10.7.5 security settings changed.
safari 6.1.6
any suggestions?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on November 17, 2014, 08:02:46 AM
Perhaps a problem with write permissions of the drive with the mounting point.. I had that problem after updating mac osx to yosemite, just change it that everyone can read and write. Greets
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on November 17, 2014, 01:43:03 PM
Quote from: Markus on November 14, 2014, 05:00:31 PM
This is super cool =). I've been trying it out on windows with pismo file mount program. Is it possible to control hotpixel fix and stripe removal in windows?
Not yet, no, but the other demands on my attention have largely subsided, so I should be able to spend time on ML/MLVFS again.  Stay tuned!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 17, 2014, 01:44:34 PM
Quote from: Andrzej on November 17, 2014, 06:05:02 AM
Hi, i am geting: "safari can't connect to server"
Is it just the webserver that doesn't work? (does the mount still show up?) Or does it not work at all?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andrzej on November 17, 2014, 03:31:25 PM
Thanks for trying to help :)

Quote from: swinxx on November 17, 2014, 08:02:46 AM
Perhaps a problem with write permissions of the drive with the mounting point.. I had that problem after updating mac osx to yosemite, just change it that everyone can read and write. Greets
Changed that, still the same.

Quote from: dmilligan on November 17, 2014, 01:44:34 PM
Is it just the webserver that doesn't work? (does the mount still show up?) Or does it not work at all?
not work at all. MLVFS service is visible under "services"

Maybe i do something wrong with the installation proces? Is this method correct? https://www.youtube.com/watch?v=vNO4ks5Xgn8
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: silvertonesx24 on November 17, 2014, 04:26:08 PM
I also get the cannot "connect to server error as well", quite often. The problem seems to be when the operation chokes on certain MLV files. There is no indication of why it chokes on these files, and the only way I can tell is by adding groups of MLV files to a folder to find the one that eventually crashes FUSE.

A crash log is below, let me know if you need any more
http://1drv.ms/1wO7Ny6
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on November 17, 2014, 07:50:21 PM
ayshih:
Really looking forward to the possibility to get the webserver options controlable on Windows
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 17, 2014, 10:40:02 PM
@silvertonesx24
divide by zero error on this line:

result = sizeof(struct wav_header) + (uint64_t)mlv_wavi_hdr.bytesPerSecond * (uint64_t)mlv_file_hdr.sourceFpsDenom * (uint64_t)mlv_get_frame_count(path) / (uint64_t)mlv_file_hdr.sourceFpsNom;


Which means that the FPS metadata in the file is either wrong, or being read incorrectly. I can prevent the error, but your FPS is still going to be wrong. It's worth investigating *why* the data is bad rather than just suppressing the error (could be an ML issue). Is there anyway you can send a file that has this issue? or perhaps at least the output of mlv_dump -v
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 18, 2014, 11:18:27 PM
updated with some bug fixes (hopefully)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 18, 2014, 11:51:11 PM
thank you.
will test tomorrow.

greets.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on November 19, 2014, 02:47:22 AM
Will test it out tonight as soon as I get home from work. Thanks again @dmilligan!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 19, 2014, 09:28:10 PM
@dmilligan:
i have just converted some DI Shots with MLVFS, and wanted to inform you that in mac osx 10.10 the finder does not show the image correctly. here is a link to one mlvfs converted di image.
thank you very much, great app.
https://copy.com/aCMp94TfSCCF8jK2

sw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on November 19, 2014, 10:20:57 PM
Quote from: swinxx on November 19, 2014, 09:28:10 PMfinder does not show the image correctly
Can you elaborate?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 19, 2014, 10:26:48 PM
of course.
when hitting the space bar, finder shows a preview.
this previewed dng is pink/white. there is a screenshot in the upper link.
thx.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 19, 2014, 10:45:37 PM
Not much to do about that. It's a known bug with the Mac Finder, it uses a hard coded black level of 2048 and ignores the black level in the file (which is different because of dual ISO processing that actually utilizes 16 bit). Tell Apple to fix their software.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 20, 2014, 02:32:06 PM
ah. ok.
i just wanted to report, cause all other previewed files work as expected

thx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 23, 2014, 09:15:09 PM
Hi David! I have a question.
I somehow managed to create a davinci resolve naming scheme to cr2hdr-r following your example and naming scheme in MLVFS. Without audio the files appears as it should with no need to enter the folders but when including audio of course you have to enter the folder and audio and movie files are separated.
using exiftool I can put in the TimeCodes: 00:00:00.00 in my DNG:s but that alone don,t seem to do the trick. When I view the RIFF info in your wav file I see TimeReference: 0 amongst other info that I don,t find in my wav file. Trying to changing/adding metadata in RIFF section in my wav file seems pretty hard. Don,t really get exiftool to apply here. Would you know how maybe?
Would be really nice to hear what you have to say about this metadata issue and maybe point me in some direction how to solve this?
Using your audio file coming from MLVFS works together with my converted DNG files so changing metadata in wav somehow should do the trick. But how :)?
Update* Used a tool called BWF Metaedit which lets me change metadata but couldn,t really copy the order of things. Still something missing but can,t find out what, hmm
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on November 24, 2014, 12:13:30 AM
IDK if you'll be able to do it any way besides actually writing code to modify the WAV file itself directly. For reference on what all you need in the WAV file and how to do it see wav.c in the MLVFS source.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 24, 2014, 06:55:47 AM
Thanks, will check on this.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 24, 2014, 05:47:10 PM
Thanks David. Got audio and 16-bit dual iso DNG working by shortening audio with sox and adding the ixml info from your wav.c using bwfmetaedit. Also added TimeCode 00:00:00.00 with exiftool. Will wrench it together and put it up in cr2hdr-r in a while. Thanks again.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on November 25, 2014, 08:19:37 AM
Great news, Danne!

Dual ISO MLV would be sweet!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 28, 2014, 03:16:25 AM
Some updates:

I added a "prefetch" option. This will cause MLVFS to preemptively load a certain number of frames to allow utilization of multiple CPU cores even if the program accessing MLVFS isn't loading multiple files at once (this should give a fair amount of speed improvement if you're using processing heavy options with apps that don't already preload a certain number of frames, like a Finder copy). MLVFS was already "multithreaded" but this was only taken advantage of if the accessing program was actually trying to load multiple files at the same time. Now MLVFS will load future frames in expectation that they will be requested at some point soon. Let me know how well this works for you and if you find any bugs/crashing. Probably a good idea not to put a number in there higher than the number of CPU cores you have, or you will just incur extra threading overhead. Some programs like SpeedGrade already do this themselves (and you can set how many frames in the settings), if that's the case, I would recommend using that instead of MLVFS's prefetch.

Added the Adobe calibration matrices (thanks Andy! (http://www.magiclantern.fm/forum/index.php?topic=13948.0)) for all ML cameras to the DNG metadata. This should help improve color "correctness" in non-Adobe programs. Please test (see the linked thread for more info).
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on November 28, 2014, 06:50:08 AM
@dmilligan:
Thank you very much! Should the new update speed up the appereance of the webgui when mounting a big set of mlv files? Nice.
And should it also improve dualiso conversion?
Would it be possible to add an eject button in the webgui? Had some problems with unmounting it in the viewer..


Big thumbs up and a big Thank You for your improvements, swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kgv5 on November 28, 2014, 08:04:33 AM
Quote from: DeafEyeJedi on November 25, 2014, 08:19:37 AM
Dual ISO MLV would be sweet!

Oh yes, that would be great
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on November 28, 2014, 09:37:18 AM
i have now tried out the new mlvfs color matrix.

in the following example i seperated the white of the shirt and set it to the center of the vectorscope in both shots

https://copy.com/LvhkenDVEDPDkyj2

to my suprise the blue color changed dramatically.
left side is the old one, right side the new one.

i would prefere the left side, so perhaps dmilligan: would it be possilbe to put the new matix to the webgui as an optional setting?

thx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 28, 2014, 02:16:27 PM
Quote from: swinxx on November 28, 2014, 09:37:18 AM
would it be possilbe to put the new matix to the webgui as an optional setting?
It's actually not possible to make this an option, the size of the CDNG IFD is hardcoded. I wouldn't want to make something like this an option anyway, there's probably something else going on that needs to be fixed, and that's the real problem that should be addressed. It might be that the AsShotNeutral is getting computed wrong because it's using routines from dcraw that use the old dcraw matrices. I'll update them and see if it makes any difference.

Quote from: swinxx on November 28, 2014, 06:50:08 AM
Should the new update speed up the appereance of the webgui when mounting a big set of mlv files?
Probably not.
Quote from: swinxx on November 28, 2014, 06:50:08 AM
And should it also improve dualiso conversion?
Yes, especially since you're using the Finder, which only tries to read the files one by one.
Quote from: swinxx on November 28, 2014, 06:50:08 AM
Would it be possible to add an eject button in the webgui?
Probably. You can always type "umount /path/to/mlvfs/mount" in the terminal. (Note there is no "n" in "umount")


Quote from: DeafEyeJedi on November 25, 2014, 08:19:37 AM
Dual ISO MLV would be sweet!
Quote from: kgv5 on November 28, 2014, 08:04:33 AM
Oh yes, that would be great
Dual ISO conversion has been implemented in MLVFS for some time now.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on November 28, 2014, 04:51:09 PM
Thanks @dmilligan and that quote was actually for @Danne -- my fault on the confusion and again thanks for your kind update!

Enjoy your Holidays!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 28, 2014, 05:22:27 PM
Cool stuff David. Will test speed when time is on my hands.
@deafeyejedi. Cr2hdr-r has been able to convert mlv dualiso files from the very beginning. Check the first post in the cr2hdr-r thread.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on November 28, 2014, 07:30:11 PM
I knew that I guess what I meant was regarding audio directly.

Happy Thanksgiving to you all and everyone else for never stopping the magic!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: baldavenger on November 28, 2014, 11:49:03 PM
The new matrix has seemingly added a red/magenta bias to my images now.  I'm using a 5D Mark III, bought in March this year.  Would it be possible to download an older build of MLVFS so I can make comparisons between before and after?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andy600 on November 29, 2014, 02:33:41 AM
Sorry guys,

After looking at the previous matrices I think I might have got the matrix order wrong. I'm gonna check a few shots and if it's what I think I will fork the repo, change the code and submit a pull req.


Edit: Yes, it's my fault  :-[ There is an extra digit in ColorMatrix2 that shouldn't be there (green-green) and the forward matrices are incorrect. I will recheck everything.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andy600 on November 29, 2014, 03:11:32 AM
@dmilligan - submitted a pull req that should fix 5D mark III footage. I'll submit a separate pull req for the other cams later. Sorry for the hassle  :-[


Edit: pull req submitted for all cams. See note on Bitbucket.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on November 29, 2014, 07:17:43 AM
Thx andy, i'll give it a try when it will be updated ;)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: swinxx on November 29, 2014, 08:09:34 AM
And thx dmilligan for the fast update!! You are great!!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 29, 2014, 04:19:42 PM
Updated the binaries with Andy's changes. Let us know if it's better now.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 29, 2014, 04:46:41 PM
Hi David!
Dragged a folder with dual iso files to desktop but can,t view the files in acr.
Here is one of the converted dng:s
https://sprend.com/download.htm?C=4085435b91c746f7aebb0ad4821d3baa

Message from AE
(http://s18.postimg.org/ojl658cbp/Screen_Shot_2014_11_29_at_16_48_01.jpg) (http://postimg.org/image/ojl658cbp/)

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 29, 2014, 05:16:23 PM
reproduced..., this is what the Adobe DNG SDK has to say about it:

*** Error: ForwardMatrix1 does not map equal camera values to XYZ D50 ***


here are the relevant DNG tags:

Model: "Canon EOS 60D"
ColorMatrix1:
      0.7428  -0.1897  -0.0491
     -0.3505   1.0963   0.2929
     -0.0337   0.1242   0.6413
ColorMatrix2:
      0.6719  -0.0994  -0.0925
     -0.4408   1.2426   0.2211
     -0.0887   0.2129   0.6051
AsShotNeutral: 0.6215 1.0000 0.4568
CalibrationIlluminant1: Standard light A
CalibrationIlluminant2: D65
ForwardMatrix1:
      0.7550   0.0645   0.1448
      0.2138   0.8936   0.1075
     -0.0005  -0.4306   1.2562
ForwardMatrix2:
      0.7286   0.1385   0.0972
      0.2600   0.9468  -0.2068
     -0.0093  -0.2268   1.0426


any thoughts @Andy600?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 29, 2014, 06:25:05 PM
found the problem, some wrong negative signs, I'll check them for all cameras and then update it
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 29, 2014, 06:42:38 PM
Fixed problems in the 5D3, 5D2, and 60D forward matrices (and verified 550D is correct). Please test (especially for other cameras, which I haven't checked).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andy600 on November 29, 2014, 11:09:36 PM
I'll check the others. Looks like I made some typos on Bitbucket  ::)

edit: checked and double checked all cams - submitted pull req
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kgv5 on November 30, 2014, 10:15:35 AM
I would like to make sure - i am on windows 7 - does updating mlvfs requires to replace mlvfs.dll only or do i need to make something more (like registering in cmd every time for example)? Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on December 06, 2014, 09:17:50 AM
*feedbacks*

I just ran the latest MLFVS w a Dual-ISO MLV file and seems to look better with the thumbnails looking cleaner than ever! Great job on that @dmilligan -- however I ran into this odd pop up window after trying to export through with smartscript2 in AE...

(https://farm9.staticflickr.com/8582/15956889782_b1c74d6971_n.jpg) (https://flic.kr/p/qj4epm)

(https://farm8.staticflickr.com/7470/15931765046_19d5795859_n.jpg) (https://flic.kr/p/qgQsGf)

same AE error-code as @danne posted from earlier BUT this MLV file was produced directly from an EOS-M body...

thoughts?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on December 06, 2014, 03:49:37 PM
That error means the color matrices in the DNG metadata are wrong/invalid. I'll double check them for EOSM (make absolutely sure you have the latest version).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 06, 2014, 05:09:19 PM
They appear to be correct. Try updating again.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on December 06, 2014, 08:54:49 PM
Thanks @dmilligan for the quick follow up on the update -- will check as soon as I get home from work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 06, 2014, 10:01:42 PM
Thanks dmilligan for fine work. I havn,t really tested dual iso lately but will in time.
Curious about cdng implementation. Answer if you like. Curious how and where to look for recoding cr2hdr20bit to create cdng files. How is it done?
Shortcut question. Would it be possible to get some kind of mac binary of the cr2dr20bit from you with your code implemented or is this fuse workflow a totally different beast? Like to use it in cr2hdr-r workflow if possible.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on December 06, 2014, 11:02:30 PM
This is purely a genius type of an idea...

This definitely would be a nice addition as well...

:D
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on December 07, 2014, 12:08:58 AM
Best way would be to rewrite chdk_dng.c (which is used by ML itself, cr2hdr, raw2dng, and mlv_dump) using code from MLVFS's dng.c
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 07, 2014, 07:28:59 AM
What a challenge. Thanks for pointers. I, ll see where it takes me.Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on December 12, 2014, 05:36:17 AM
Final got around to Trying "MLVFS"  It seem to work very well so far, maybe its because I have a PC Video card in my Mac but I have pink cast image
in finder, but Looks OK in After Effect with ACR  ::)
I'm on maverick 10.9.3 with a GTX760 video card ,I will post a image tomorrow. 
I can see how this will greatly speed up A.E. workflow.
Thanks for the great work on "MLVFS" :D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Audionut on December 12, 2014, 09:37:51 AM
Known issue.  The finder uses an fixed black level.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on December 18, 2014, 04:14:36 AM
Hi,
the web interface at http://localhost:8000 is not working for me. I've tried restarting after installing but it is hung up on "waiting for localhost" indefinitely in firefox, chrome and safari. The FUSE disk mounts, but there are no files inside.

Edit: just realized it's still busy unpacking the mlvs and creating idx files. I will wait until it is finished and see if the webpage is accessible then.

edit 2--just finished, looks like it's all working fine.

Is there by chance the possibility of implementing a function that will fix the aspect ratio for slo-mo clips?

Also, is there a way to make these volumes searchable by Spotlight? Right now Spotlight can't see them, I think it's a permissions issue.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on December 18, 2014, 02:36:10 PM
Not sure if this has been mentioned but the latest version of PFMAP-171-win works fine in W8.1 (64bit) for .mlv files.  It doesn't/can't? successfully mount any .M00, M01 etc files.

All the files that I've tested with didn't have any embedded sound so I don't know if it works with an MLV with Audio file. Something to try another day and report the outcome.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arrinkiiii on December 18, 2014, 04:32:35 PM


Will ayshih implement the other stuff on the windows version, like strips correction?  Im having problems with g3gg0 windows version... empty folders =(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 18, 2014, 04:48:11 PM
Quote from: bwinter88 on December 18, 2014, 04:14:36 AM
Is there by chance the possibility of implementing a function that will fix the aspect ratio for slo-mo clips?
If only the DNG spec supported a "pixel aspect ratio". It doesn't, so there's basically no way to do it. Re-scaling must be done after demosaicing, so for now you'll have to do the stretch manually in your NLE.

Side Note: It's very strange to me that Adobe would not have support for this in the DNG spec. It would also be very useful for anamorphic lenses.

Quote from: bwinter88 on December 18, 2014, 04:14:36 AM
Also, is there a way to make these volumes searchable by Spotlight? Right now Spotlight can't see them, I think it's a permissions issue.
Probably not, if it doesn't work like it is now. Certain folders pretty much have to remain read only.
Using ls (http://en.wikipedia.org/wiki/Ls) and grep (http://en.wikipedia.org/wiki/Grep) works pretty well though (and regular expressions (http://www.regular-expressions.info/) are way better for searching if you know how to use them, also if you don't like Apple to know about everything you search for (http://arstechnica.com/security/2014/10/mac-os-x-yosemite-reportedly-leaks-location-search-data/)):
ls -R | grep "search expression"

Quote from: arrinkiiii on December 18, 2014, 04:32:35 PM
Will ayshih implement the other stuff on the windows version, like strips correction?  Im having problems with g3gg0 windows version... empty folders =(
http://www.magiclantern.fm/forum/index.php?topic=13152.msg133827#msg133827
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on December 18, 2014, 06:08:24 PM
Quote from: dmilliganQuote from: bwinter88 on Today at 04:14:36 AM
Quote from: bwinter88Also, is there a way to make these volumes searchable by Spotlight? Right now Spotlight can't see them, I think it's a permissions issue.
Probably not, if it doesn't work like it is now. Certain folders pretty much have to remain read only.
Using ls and grep works pretty well though (and regular expressions are way better for searching if you know how to use them, also if you don't like Apple to know about everything you search for):
ls -R | grep "search expression"

Thanks for the reply. I've read that to enable Spotlight on FUSE volumes you can use something called the "allow_other" option at mount. I won't pretend to understand FUSE or coding at all but what do you think? would it be possible to add this without affecting anything?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arrinkiiii on December 18, 2014, 06:17:45 PM
Thanks dmilligan =)    Today i have read the whole post and see that but since it's from 14 of November... Hope ayshih can continuous this app for windows. 
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on December 18, 2014, 09:34:27 PM
Tried this on my MacBook a couple months ago and couldn't get it working at all. I installed it again last night on my iMac and everything worked fine, right from the start. Going to try it again on the MacBook tonight or tomorrow and see if it was a technical issue, or just a user malfunction  ;)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on December 19, 2014, 02:25:05 AM
Will try this converter again on a new Mac mini that is supposed to arrive tomorrow -- should be quicker than my current 2006 MBP.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 20, 2014, 07:24:11 PM
Quote from: bwinter88 on December 18, 2014, 06:08:24 PM
Thanks for the reply. I've read that to enable Spotlight on FUSE volumes you can use something called the "allow_other" option at mount. I won't pretend to understand FUSE or coding at all but what do you think? would it be possible to add this without affecting anything?
Simply pass the FUSE options you want when you start MLVFS, it will recognize them.

I have also updated the automator workflow to use the necessary options (-o allow_other -o local). So MLVFS will now show up as a 'local' drive (hopefully this won't cause any issues, there's a warning against doing this on the OSXFUSE FAQ, so let me know), and spotlight search should work (I found you actually need both of these for spotlight to work). I also figured out how to make the name show up different in the Finder. So it's no longer "OSXFUSE ....", but it will simply keep the name of the mount directory and have "(MLVFS Mount)" at the end.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on December 20, 2014, 07:51:15 PM
So far, so good. The local drive is a nice feature, running fine now on my MacBook too. One thing though, did I miss the detail or does this not support .raw, only .mlv?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on December 20, 2014, 08:39:14 PM
MLV only. I want to discourage people from using RAW format. There's very vital metadata missing amongst other things.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on December 20, 2014, 11:46:11 PM
Ahh, I see. Had some .raw's from right after raw video was first developed to see how much nicer the converters are now. And then I spent 5 hours looking at different ways to edit and save exif/xmp information so that the .dng's would be universally compatible .   ::)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on December 21, 2014, 08:19:45 AM
@Danne -- just got my mac mini installed and upgraded with two 8gb of RAM (16gb total) along with a 360gb SSD. Looking forward to your cdng implementation test in the near future I hope.

Can't wait to test out MLVFS on this tiny beast...

Hopefully it won't get too HOT!

::)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on December 21, 2014, 05:42:20 PM
@dmilligan:
hello, so 2 days ago you commited a silent service update :)
nice. thank you.

great.
greets sw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on December 22, 2014, 03:52:39 AM
Expanding on my previous post about exif metadata, most, if not all, video/image editing programs claim to be able to interpret exif/xmp data but it seems like there are so many exif tags for most of the current digital cinema cameras that our poor little ML Raw files aren't supported very well.
ACR does a great job of adding and interpreting exif, so my idea was to open the mounted .dng's in Bridge ACR, make my adjustments, save the exif/xmp to the images and then import into Resolve or FCPX to skip the transcoding and get the best Raw processing possible. Bridge wouldn't write new exif/xmp to the images or the camera raw database for some reason (parsing error), so is there another way to go about this? I looked into .MXF files but haven't tried them yet.
On another note, Black Magic is developing a CDNG wrapper so that quicktime plays the files natively, hopefully that's something we can have implemented as well.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 22, 2014, 01:03:09 PM
Quote from: N/A on December 22, 2014, 03:52:39 AM
then import into Resolve or FCPX to skip the transcoding and get the best Raw processing possible.
What kind of stuff are you trying to add, that makes Resolve or FCPX work better? The CDNGs from MLVFS should already have all the necessary metadata to open correctly. Thanks to Andy600, all the color calibration matrices from Adobe are also now included. AFAIK, the vast majority of XMP data is ACR specific. XMP is really just an XML schema that Adobe uses to store non-destructive raw adjustments (i.e. the values of ACR sliders). You can also put EXIF tags/info and various other stuff in the XMP format, but there's no reason to, because you can put EXIF tags as IFD entries (TIFF headers) directly into the DNG.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on December 22, 2014, 02:02:01 PM
Color management. Compatibility. A thorough record of information for archiving, transcoding and sharing footage across programs, platforms and OS's. I've always transcoded to Prores before editing but since MLVFS makes it so much quicker to access the dng's, a raw workflow makes more sense now.

I'm not saying that the files lack anything, just offering a different perspective. I have a decade of audio experience but I've only been shooting and editing a few years. I want my mix and master files to sound clean, full and powerful on any speaker or program its played back with, and I've found that video files can be wildly unpredictable. Basically, a universal wrapper or data file.

Just a "bigger picture" point of view, really.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 22, 2014, 04:28:24 PM
Quote from: N/A on December 22, 2014, 02:02:01 PM
Color management. Compatibility. A thorough record of information for archiving, transcoding and sharing footage across programs, platforms and OS's.
Well, that's sort of the point of the DNG specification, and all that stuff is already there => all the information needed to properly and consistently interpret the DNG is already provided. If certain programs do not abide the DNG spec and ignore particular metadata (e.g. the Apple Finder ignores the blacklevel metadata), which is quite rampant (even for Adobe who created the DNG spec), then there's nothing I can really do about that.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on December 24, 2014, 06:38:26 PM
Yeah I definitely understand, hopefully we'll see a lot more support for CDng in the future.

Interestingly enough, I found a Quicktime and quicklooks component for playing back Dng's, but unfortunately its been discontinued.

http://hamburgpromedia.com/products/mxf4mac/codecs/dng-qt.php (http://hamburgpromedia.com/products/mxf4mac/codecs/dng-qt.php)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Piotruchna on December 28, 2014, 02:59:53 PM
Hello,
I am not sure if anybody had this problem but Ill just type it out:

If you use latin extended characters (for me polish) in names of the folders you will not be able to mount .mlv files. Error window will come out. Just changing name from "kręcić" to "krecic" will make everything work.

Also thanks!
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 28, 2014, 04:08:41 PM
Makes sense, the standard C libraries do not have multi-byte character support (UTF-8 ), only ASCII. This may or may not be a limitation that can be overcome depending on the FUSE library, I'm not sure.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwv656 on January 03, 2015, 07:31:31 PM
I just want to say Thank You to all of you made this possible.
I couldn't believe my eyes watching non-converted MLV playing real-time in resolve.

I haven't tried it on PC side yet but the OSXFUSE+MLVFS works incredibly well.
Thank you so much once again!!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: limey on January 04, 2015, 05:20:46 AM
Help me understand where I'm messing this up.

I'm using Windows.

I've installed PFMAP.
I've registered the DLL.
In a windows explorer window I find a MLV file and use Quick Mount.

I then goto Premiere and import the first image of the image sequence on that quick mounted MLV folder that is created.

When it gets imported the file is black and only the audio works.

**initially was mounting MLV files on an external drive, but wanted to rule out that it wasn't a loading issue and copied the files to the desktop in its own folder (MLV Test). Didn't fix it.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on January 04, 2015, 09:44:41 AM
Can you post a DNG?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: limey on January 04, 2015, 04:21:18 PM
The M21-0244 DNG is the one that comes out black.

I tried another MLV after quick mounting it and importing to Premiere and it has a Pink overlay. This is (M10-0244)

Both files are totally fine in MLVRawViewer and I've used them both in movies doing different methods before, ie export to DNG, or straight export to .mov file.

https://www.dropbox.com/sh/6sih3x1alfcme9o/AACQj3yySqm72gQ7Cc-z2Nn6a?dl=0

Thanks for your help dmilligan.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 04, 2015, 05:03:48 PM
There's nothing wrong with either DNG you posted. They both appear fine in Premiere and all the metadata looks correct. It may be some kind of issue with how Premiere is reading the data from MLVFS, but I can't really be much help with the Windows port as I don't own a Windows PC.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: limey on January 04, 2015, 05:42:40 PM
I appreciate the time you took to look into it dmilligan.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: limey on January 04, 2015, 07:08:02 PM
As a further sidenote.... When I export from MLVRawViewer to DNG, those will not import to premiere as a sequence, but MLV mounting does (despite the pink or black overlay). Not sure if that helps diagnose anything.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on January 04, 2015, 07:37:59 PM
You are running the latest Premiere CC? Have you tried SpeedGrade or Resolve or some other program?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on January 05, 2015, 04:33:45 PM
Resolve works for me on pc. Still on premiere cs6 though so I can't try to import them there.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andy kh on January 05, 2015, 05:34:59 PM
unable to register mlvfs.dll. getin this eror msg "the module mlvfs.dll was loaded but the entry-point DllRegisterServer was not found Make sure that mlvfs.dll is a valid DLL" can someone here help me
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on January 05, 2015, 06:05:19 PM
These are the steps I followed that worked:
1. Downloaded the file and renamed it to add the ".dll" on the end.
2. Copied the file to my Windows folder.
3. Left-clicked somewhere in the Windows folder.
4. Held down the shift key while right-clicking, and chose "Open Command Window Here."
5. Typed the command per ayshih.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: timbytheriver on January 14, 2015, 10:05:12 AM
Hello

I've installed binary on Mac OSX 10.8.5 as per instructions.

I'm not getting the services MLVFS option showing on my right-click menu > services > MLVFS

Have rebooted, repaired prefs. Can anyone help me with a solution please?

Thanks

Tim

PS I have checked in the main Preferences cp and it lists MLVFS as a ticked option for the menu. What gives? :S
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 14, 2015, 01:47:00 PM
Are you right clicking on a folder?

Quote from: timbytheriver on January 14, 2015, 10:05:12 AM
I've installed binary on Mac OSX 10.8.5 as per instructions.
I've never tested MLVFS on 10.8 so it could be this, nor do I really have the capability (or desire) to support old OS versions. I would recommend updating to the latest (10.10)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kgv5 on January 14, 2015, 01:53:46 PM
Guys, is vertical banding correction works on windows version? Seems not but maybe i missed something.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: timbytheriver on January 14, 2015, 02:57:20 PM
Quote from: dmilligan on January 14, 2015, 01:47:00 PM
Are you right clicking on a folder?
I've never tested MLVFS on 10.8 so it could be this, nor do I really have the capability (or desire) to support old OS versions. I would recommend updating to the latest (10.10)

@dmilligan I was clicking on the mlv itself. Doh!  :o  It's working fine. My bad, sorry.

PS And thank-you for an inspirational approach in this app.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on January 16, 2015, 01:02:58 PM
I just started to play again with the MLVFS on my freshly reinstalled system.
- OS: Windows 7 Ultimate 64bit
- Pismo File Mount version: 2014.7.11
- mlvfs.dll version: latest from bitbucket (2014-10-03)
- AE CC 2014 version: 13.2.0.49 (2014.2 Release)
- ML: Nightly.2015.Jan15.550D109, mercurial changeset 4fff9f6cb33f (unified) tip

I can't import the DNG sequence into AE. After I adjust the settings in ACR (version 8.7.1) during the import process and hit OK, it gives me an error, saying:
"After Effects error: The file format module could not parse the file 45::35"

This happens only with my new MLV file (https://www.dropbox.com/s/pac7egqarj34kko/M16-0963.MLV?dl=0) (recorded today). If I import an older MLV file (https://www.dropbox.com/s/k1ut4s3oc4da2ny/M18-1424.MLV?dl=0), which was recorded with a nightly build from previous year (I don't know the date), then everything works fine.

Somebody could help me out with checking the MLV files?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 16, 2015, 10:12:56 PM
They both work fine on the current Mac version, and there doesn't appear to be much difference between the two. I guess you'll have to wait until somebody with Windows and some programming skills updates the Windows version.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on January 16, 2015, 10:50:40 PM
Is it possible to speed up dual iso conversion (on mac) ?? Thx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 17, 2015, 03:56:20 AM
Yes. Use mean32 interpolation instead of AMaZE, or figure out how to optimize this code from cr2hdr, 75% of the time is spent here (I don't even understand how it works/what its doing, much less being able to optimize it, so good luck):

        for (int y = 5; y < h-5; y ++)
        {
            int s = (is_bright[y%4] == is_bright[(y+1)%4]) ? -1 : 1;    /* points to the closest row having different exposure */
            for (int x = 5; x < w-5; x ++)
            {
                int e_best = INT_MAX;
                int d_best = d0;
                int dmin = 0;
                int dmax = COUNT(edge_directions)-1;
                int search_area = 5;
               
                /* only use high accuracy on the dark exposure where the bright ISO is overexposed */
                if (!BRIGHT_ROW)
                {
                    /* interpolating bright exposure */
                    if (fullres_curve[raw_get_pixel32(x, y)] > fullres_thr)
                    {
                        /* no high accuracy needed, just interpolate vertically */
                        not_shadow++;
                        dmin = d0;
                        dmax = d0;
                    }
                    else
                    {
                        /* deep shadows, unlikely to use fullres, so we need a good interpolation */
                        deep_shadow++;
                    }
                }
                else if (raw_get_pixel32(x, y) < white_darkened)
                {
                    /* interpolating dark exposure, but we also have good data from the bright one */
                    not_overexposed++;
                    dmin = d0;
                    dmax = d0;
                }
                else
                {
                    /* interpolating dark exposure, but the bright one is clipped */
                    semi_overexposed++;
                }
               
                if (dmin == dmax)
                {
                    d_best = dmin;
                }
                else
                {
                    for (int d = dmin; d <= dmax; d++)
                    {
                        int e = 0;
                        for (int j = -search_area; j <= search_area; j++)
                        {
                            int dx1 = edge_directions[d].ack.x + j;
                            int dy1 = edge_directions[d].ack.y * s;
                            int p1 = raw2ev[gray[x+dx1 + (y+dy1)*w]];
                            int dx2 = edge_directions[d].a.x + j;
                            int dy2 = edge_directions[d].a.y * s;
                            int p2 = raw2ev[gray[x+dx2 + (y+dy2)*w]];
                            int dx3 = edge_directions[d].b.x + j;
                            int dy3 = edge_directions[d].b.y * s;
                            int p3 = raw2ev[gray[x+dx3 + (y+dy3)*w]];
                            int dx4 = edge_directions[d].bck.x + j;
                            int dy4 = edge_directions[d].bck.y * s;
                            int p4 = raw2ev[gray[x+dx4 + (y+dy4)*w]];
                            e += ABS(p1-p2) + ABS(p2-p3) + ABS(p3-p4);
                        }
                       
                        /* add a small penalty for diagonal directions */
                        /* (the improvement should be significant in order to choose one of these) */
                        e += ABS(d - d0) * EV_RESOLUTION/8;
                       
                        if (e < e_best)
                        {
                            e_best = e;
                            d_best = d;
                        }
                    }
                }
               
                edge_direction[x + y*w] = d_best;
            }
        }


My dual ISO preview code only takes about 10ms per frame :P The cr2hdr stuff I didn't write, nor do I totally understand everything it does.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andy kh on January 17, 2015, 09:10:58 AM
Quote from: surami on January 16, 2015, 01:02:58 PM

This happens only with my new MLV file (https://www.dropbox.com/s/pac7egqarj34kko/M16-0963.MLV?dl=0) (recorded today). If I import an older MLV file (https://www.dropbox.com/s/k1ut4s3oc4da2ny/M18-1424.MLV?dl=0), which was recorded with a nightly build from previous year (I don't know the date), then everything works fine.

i also have problem with new mlv files recorded @ 23.97 fps which i recorded recently. i am unable to import xml in resolve. older mlv files which i recorded last year wirth t2i  works fine.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 17, 2015, 08:55:57 PM
Updates:
fixed issues causing MLVFS to not work with full res silent pictures MLVs
enabled SSE2 extensions for AMaZE (AMaZE itself is faster, but the cr2hdr stuff overall isn't really any faster)
ability to override the framerate
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on January 18, 2015, 02:01:53 AM
Thanks for the update @dmilligan!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on January 18, 2015, 09:47:00 AM
@dmilligan: Thanks for checking the files and for the explanation.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on January 18, 2015, 09:37:09 PM
great. thx dmilligan.
so we now only need silentpic fullres working for canon 5d mk3 123 :))))
5 month no progression.. hope alex is back soon..
greets.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on January 19, 2015, 08:33:36 AM
Quotegreat. thx dmilligan.
so we now only need silentpic fullres working for canon 5d mk3 123 (http://www.magiclantern.fm/forum/Smileys/aaron/laugh.gif)))
5 month no progression.. hope alex is back soon..

And the 7D.....  ;)

Cheers
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on January 21, 2015, 09:04:41 AM
@dmilligan -- any idea why the MLV files that are produced from adv_intervalometer during time-lapse in the newly updated FRSP module  (http://www.magiclantern.fm/forum/index.php?topic=12523.msg137957#msg137957module) that I used on 7D 2.0.3 w latest nightly (Jan 15) aren't corresponding with latest MLVFS ... it just keeps crashing on me few mins after I enable '20-bit cr2hdr' and sometimes it would just become unresponsive. (Same even if I don't select any of the options and leave all as default)

Like it would start with a few MLV's (shows DNG's) and then the rest of MLV's (w empty folder -- no dog's) when I thought it may be a long process.... waited for a quite while and still no avail?

(https://farm8.staticflickr.com/7531/16331757545_46d6d49186_n.jpg) (https://flic.kr/p/qTbwwD) (https://farm8.staticflickr.com/7526/16145875207_8e6a0d94c2_n.jpg) (https://flic.kr/p/qAKQbK) (https://farm8.staticflickr.com/7472/16145875167_e25c8173fb_n.jpg) (https://flic.kr/p/qAKQb4)

Heck sometimes the mount folder would just disappear on its own after a while and then I would have to manually restart my Mac Mini to get the 'Mount Folder' back... so strange!?

I even deleted the IDX files just to be sure.

This only applies to MLV's from time-lapses (both non & Dual-ISO) with FRSP enabled -- regular RAW/MLV work fine atm.

This was done on OS X Yosemite 10.10.1 in case you were wondering.

Thoughts?

Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: poromaa on January 21, 2015, 10:10:32 AM
Hi and thanks for yet another great utility for us film makers!

I guess everybody hate requests, so don't see it as one, but I am filming raw with 5D2 and editing on the fly with my macbook air (!) yes, it works. And MLVFS is really what makes it possible.

Just one thought, would it be possible to have a setting for virtual dngs image resolution? I'm thinking it would work like a proxy and be faster than debayering the whole thing? Or is it not working that way at all?

If possible that would enabling having the real footage in low resolution while editing, and then just change settings before render/grading.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dubzeebass on January 21, 2015, 01:49:37 PM
That's a good idea!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on January 21, 2015, 06:49:58 PM
Good thinking @poromaa!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on January 21, 2015, 07:48:52 PM
Expanding on poromaa's suggestion, maybe have an option to have a full-size embedded jpeg preview, and use it as the proxy? Low rez proxies would work too but might get complicated when we have to edit our project settings for different resolutions.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on January 21, 2015, 08:28:26 PM
Point taken @N/A...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 21, 2015, 09:04:56 PM
Quote from: poromaa on January 21, 2015, 10:10:32 AM
Just one thought, would it be possible to have a setting for virtual dngs image resolution? I'm thinking it would work like a proxy and be faster than debayering the whole thing? Or is it not working that way at all?
Do you not already get real-time playback?

Premiere, SpeedGrade, and Resolve can all playback this stuff in real-time with GPU acceleration (and MLRawViewer too). The main limiting factor is usually hard drive speed. MLVFS would still have to read the entire image data from the MLV file anyway. And also this is a problem:
Quote from: N/A on January 21, 2015, 07:48:52 PM
might get complicated when we have to edit our project settings for different resolutions.

Quote from: N/A on January 21, 2015, 07:48:52 PM
Expanding on poromaa's suggestion, maybe have an option to have a full-size embedded jpeg preview, and use it as the proxy?
CDNG doesn't allow embedded previews (only 1 main IFD is allowed). Though, you could make it an option where MLVFS just outputs JPEGs or TIFFs, and I've thought about this, but it opens up a whole new can of worms that I really have no desire to get into, since you're no longer dealing with raw data: you've got to debayer (and deal with GPU acceleration needed for that), white balance, colorspace transformations, highlight recovery, etc. There are already professional tools out there that do this stuff, better/faster than I ever could. MLVFS does what it needs to do: provide these tools with the raw data in a format they can understand.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 21, 2015, 11:28:34 PM
Quote from: DeafEyeJedi on January 21, 2015, 09:04:41 AM
it just keeps crashing on me few mins after I enable '20-bit cr2hdr' and sometimes it would just become unresponsive. (Same even if I don't select any of the options and leave all as default)
Do you see any mlvfs crash logs in ~/Library/Logs/DiagnosticReports

Quote from: DeafEyeJedi on January 21, 2015, 09:04:41 AM
Heck sometimes the mount folder would just disappear on its own after a while and then I would have to manually restart my Mac Mini to get the 'Mount Folder' back... so strange!?
If that happens you can type 'umount path/to/mount/folder' in the terminal (note the lack of an 'n' in umount, that is not a typo)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mufanito on January 22, 2015, 12:11:40 PM
Hi there,

I´m having trouble getting a timeline in Resolve with dual-iso and regular iso clips. I cannot select the dual-iso option for the dual-iso files on the webserver without afecting the non dual-iso files. So, when mixing the 2 kind of files on a timeline i either get a correct image on the dual-iso files and a weird looking image on the non dual, or if i don´t select the dual-iso option on the webserver, the dual-iso files have that interlaced look and the non dual are correct. I´ve tried to put the two files types on separate folders and mount each one, but as soon as i mount the second one the first webserver tab gives place to a new one and it´s settings override the ones i set previously.

So, any tips on how to get dual-iso and non dual-iso files on the same timeline?

Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 22, 2015, 04:10:06 PM
You can start the webserver for the 2nd instance with a different port number with the --port=[port#] option. Then you can have two separate webguis. If you used for example 8000 and 8001 for port numbers, you'll use http://localhost:8000 and http://localhost:8001 to control the different instances.

However, if you have a non-dual ISO shot even if cr2hdr is turned on, it should detect that is not dual ISO and leave it alone (this check will hurt performance somewhat though). What we really need is dual ISO metadata provided in camera, so we know for sure whether to skip or apply dual ISO processing. I believe the MLV format has a provision for this, it's just not implemented. I'll look into implementing this, it shouldn't be too hard.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mufanito on January 22, 2015, 08:31:01 PM
Thanks!

"However, if you have a non-dual ISO shot even if cr2hdr is turned on, it should detect that is not dual ISO and leave it alone (this check will hurt performance somewhat though)." - i´ve tried this but to no avail.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: silenttapes on January 22, 2015, 09:33:23 PM
Ive tried to search this thread for the answer but I can't seem to find it.

MLVFS does not show up in the Davinci Resolve library for me (11 lite).  In OSX I can mount the files fine and they appear in the sidebar of finder. I have selected 'naming scheme Davinci resolve'.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: silenttapes on January 22, 2015, 10:05:26 PM
Nevermind I figured it out somewhat. It doesn't look like other peoples screen shots but I can access the mounted folders.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 22, 2015, 11:00:55 PM
Quote from: mufanito on January 22, 2015, 08:31:01 PM
"However, if you have a non-dual ISO shot even if cr2hdr is turned on, it should detect that is not dual ISO and leave it alone (this check will hurt performance somewhat though)." - i´ve tried this but to no avail.
I found the problem. The issue was that the metadata was being written as if the file had been dual ISO processed even if it had been skipped. The image data was correct, but the black level was wrong (cr2hdr processing changes the blacklevel). I'll post a new build shortly.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 23, 2015, 12:11:32 AM
okay, I've uploaded a build with the fix
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on January 23, 2015, 12:16:26 AM
@dmilligan -- sorry I've been on the road and will be back home tonight to check for MLVFS crash logs per your request.

Also would you think the latest update from today would solve my issue?

Ether way I'll still hope to locate the crash logs for you.

Thanks again for the update, David!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mufanito on January 23, 2015, 08:12:32 PM
Thanks dmilligan! Gonna try the new build now.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on January 25, 2015, 07:59:10 AM
Excuse me for the delay...

Here are the recent crash logs per your request from earlier. (I included a brief-test FRSP made MLV file as well just for you to check out)

'For @dmilligan' (http://https://www.dropbox.com/sh/euxlizop58cwz9r/AADwkMslhGNfzuPDfjtmqXOHa?dl=0'For%20dmilligan') <-- fixed this link (sorry)

currently updating Davinci Resolve and will let you know how it goes.

Thanks, again!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 25, 2015, 03:08:54 PM
Quote from: DeafEyeJedi on January 25, 2015, 07:59:10 AM
http://www.dropbox.com/home/Public/For_dmilligan (http://www.dropbox.com/home/Public/For_dmilligan)
That's not the right link. You need to get the public link from dropbox (only you can access this link you gave me).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on January 25, 2015, 06:58:03 PM
Sorry dmilligan -- guess I was half asleep last night...

here's the updated link...

'For dmilligan' (http://https://www.dropbox.com/sh/euxlizop58cwz9r/AADwkMslhGNfzuPDfjtmqXOHa?dl=0'For%20@dmilligan')
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 25, 2015, 08:24:51 PM
The crashes are because you had some MLV files that either didn't have the 'bits per pixel' metadata correct, or MLVFS didn't read it right (my guess would be the former).

The problem you are having with the FRSP dual ISO is actually due to the FUSE library timing out. A FUSE client app (MLVFS) has only so long to respond to a request before the FUSE library will timeout and kill the client and the mount. By default this is set to I think 30 seconds. It's taking the cr2hdr code longer than this to process a single frame, so it times out. You can override the timeout by supplying -o daemon_timeout=0

I would probably recommend against using MLVFS for processing full res dual ISO shots since it takes so long. It's better to use a more tradition conversion approach where you permanently process the DNG frames (so you don't have to do it again). The other reason is that the MLVFS mount is going to appear incredibly unresponsive, because it's have to work so hard to process frames.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on January 27, 2015, 07:00:32 AM
hi,

@dmilligan:
mlvfs worked so fine the last couple of month, its a great tool.
today i opened my working folder to remount some of the older mlvfs dirs, but suddenly safari can not link to the webhost, i get the message that the page could not be loaded..?
what happened. the files i try to mount are working normally..?
i run the latest fuse 2.7.4 drivers and the latest mlvfs

update: puh, i dont know why, but i had to "ignore the rights of the owner of the hdd" to make it work again. so if others have the same problems, just check the box of the information window of your drive.

best wishes, sw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on January 27, 2015, 07:07:53 AM
Quote from: dmilligan on January 25, 2015, 08:24:51 PM
I would probably recommend against using MLVFS for processing full res dual ISO shots since it takes so long. It's better to use a more tradition conversion approach where you permanently process the DNG frames (so you don't have to do it again). The other reason is that the MLVFS mount is going to appear incredibly unresponsive, because it's have to work so hard to process frames.

I second that now... From here on out I will try to refrain from using Dual-ISO's during Timelapse FRSP (unless if I need to) in order to use your app more often when it comes to TL-FRSP within MLV files.

Thanks again for your time on this!


Quote from: swinxx on January 27, 2015, 07:00:32 AM
update: puh, i dont know why, but i had to "ignore the rights of the owner of the hdd" to make it work again. so if others have the same problems, just check the box of the information window of your drive.

was this done on a Mac?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on January 27, 2015, 08:43:21 AM
I've been prompted to update my Nvidia CUDA and went ahead with it.

Now whenever I try to open Davinci Resolve 11 (with DR enabled within MLVFS) and once I start to scroll through REGULAR MLV's (not Dual-ISO's)... it would then become unresponsive and crash.

I would then get this window...

(https://farm8.staticflickr.com/7313/15756352154_4f5b46426f_n.jpg) (https://flic.kr/p/q1kqy3)

Not sure if this is related to MLVFS because the local web host is still active and this custom-modified Mac Mini sure seems to be up to standards for this type of stuff... (& Yes AE [w Smartimport2] as well as CC PP 2014 works just fine w MLVFS in case you were wondering)

(https://farm9.staticflickr.com/8605/16191198878_a26c95a134_n.jpg) (https://flic.kr/p/qEL8km) (https://farm8.staticflickr.com/7368/16352856826_0f6c42b02a_n.jpg) (https://flic.kr/p/qV3EBm) (https://farm8.staticflickr.com/7292/16377933842_782e4bf287_n.jpg) (https://flic.kr/p/qXgc8G) (https://farm8.staticflickr.com/7355/16377933782_2a9ae90fe3_n.jpg) (https://flic.kr/p/qXgc7E)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 27, 2015, 12:52:38 PM
Sounds like a problem with Resolve or the graphics driver. MLVFS does not use the GPU in any way.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on January 27, 2015, 09:01:26 PM
Okay thanks -- just thought it would because of the CUDA update that I just did.

Anyone else out here have a similar system OS X with the graphic card that I have?

Anyone experiencing in lagging during playback with MLVFS mounted files?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 27, 2015, 09:05:49 PM
Sorry, that was supposed to say GPU, MLVFS does not use the GPU or CUDA, so it's not anything related to MLVFS.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on January 27, 2015, 09:57:20 PM
Okay gotcha -- so would the graphic card be a factor then?

Strange it worked fine before the CUDA update.

Otherwise should I just downgrade DR back to 10 or 9 -- would this be even worth to try? I think not.

Hopefully I'll get to the bottom of this sooner rather than later.

Thanks again @dmilligan for your efforts!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 27, 2015, 11:13:06 PM
CUDA is a driver for the graphics card that allows programs like Resolve to take advantage of some special types of parallel processing that the GPU is good at. This is typically used for debayering. It sounds like there's some kind of issue between the CUDA driver update you installed and Resolve (google is your friend!). This would have nothing to do with MLVFS.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: niklas.seehausen on January 31, 2015, 11:40:18 AM
The question is pretty dump. But is audio "glued" to the video or do I get 1 audio file and a folder with  cdngs ? :-\
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Lars Steenhoff on January 31, 2015, 06:18:10 PM
Is there a way to change the default settings?

for example I would like to have bad pixel correction and stripe correction always on
What would I need to change to make this the default setting when starting MLVFS?

Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 31, 2015, 08:23:42 PM
Quote from: niklas.seehausen on January 31, 2015, 11:40:18 AM
The question is pretty dump. But is audio "glued" to the video or do I get 1 audio file and a folder with  cdngs ? :-\
I'm not sure what you mean by "glued", they are not in the same file, but in when viewed in media browsers for most apps (like Premiere of Resolve) they show up and import as one single clip with audio.

Quote from: Lars Steenhoff on January 31, 2015, 06:18:10 PM
Is there a way to change the default settings?
Yes. You can edit the MLVFS service in Automator. One of the "steps" is a shell script that actually invokes MLVFS. You can specify what ever additional command line options you would like there. MLVFS has its own command line options that control the various settings you talk about (see near the bottom of main.c in the source code for a list), there are also a lot of OSXFUSE built in options that control how the mount works. See the OSXFUSE documentation for more info about those.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Lars Steenhoff on January 31, 2015, 08:59:49 PM
Thanks dmilligan

I found the ones that I wanted:
--stripes
--bad-pix
--resolve-naming


Saves me a few clicks every time  :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DanHaag on February 01, 2015, 11:30:01 AM
Hi Lars, can you copy&paste the shell script with your adjustments in it so I can see what it looks like?  :) Need those changes, too. But I know nothing about using Automator or writing code.  :-[
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on February 01, 2015, 12:01:32 PM
Any work being done on the ported "Pismo" version for Windows? I find it preferable to extracting the DNGs as I did before and it's really easy to use. It's very fast and responsive when you have mounted the mlv's.  However the mounting time takes quite a while for large files. Mounting like 256gb of mlv's from the harddrive must be somewhere  around 30mins or so. As I understand it its almost instant on the original fuse mac version?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Lars Steenhoff on February 01, 2015, 05:56:07 PM
@ DANHAAG


nohup ~/Library/Services/MLVFS.workflow/Contents/mlvfs "$2" --stripes --bad-pix --resolve-naming --mlv_dir="$1" -f -o volname="$(basename $2) (MLVFS Mount)" -o local -o allow_other > /dev/null 2>&1 &


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DanHaag on February 01, 2015, 07:08:09 PM
Thank you, Lars! Would never have figured that out myself, helped a lot!  :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 01, 2015, 07:14:01 PM
Quote from: Markus on February 01, 2015, 12:01:32 PM
Any work being done on the ported "Pismo" version for Windows?
There is no one working on it ATM.

Quote from: Markus on February 01, 2015, 12:01:32 PM
However the mounting time takes quite a while for large files.
That's because MLVFS needs to build an index file (.IDX file). Basically it searches through the MLV file and builds a table of where all the data is, that way it can access the data quickly. This may take some time for extremely large files. It does this on Mac too, but only "on demand", so the Mac mount will show up instantly, but you may have to wait for the index files to get built before you can actually browse or use the mount. You only have to build the index files once though. So the next time you mount those files, it should be much faster (as long as you don't delete the .IDX file). If you playback the video in camera, the camera does this too, and it's the same format (actually much the same code), that builds that file, and MLVFS can actually use those index files. So if you playback in camera, make sure to copy the .IDX files, and MLVFS should mount faster.

Having "random access" storage (SSD, SD, CF, etc.) will likely make building the index files a lot faster as well. To build the table we don't have to read very much data, but we do have read a little data from all over then entire file. Spinning disks are not so good at this because of "seek time" required to move the head around to different areas. With solid state, we can read from anywhere in the same amount of time (hence "random access").
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on February 01, 2015, 07:36:16 PM
Thanks for your great reply! I guessed it had to do with indexing. I can see I have the Idx files saved next to mlv's I have mounted.  I'll have to test remounting some of them and  see if it's faster.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 02, 2015, 07:03:28 PM
Once the indexing files (.IDX files) are made for each MLV files -- then the next time you mount them with MLVFS it basically would be on the fly.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johnwangphoto on February 03, 2015, 12:34:29 AM
HI,

I got the MLVFS to work on my iMac and imported the DMG to Resolve 11. The problem I have is that all the raw clips turned way too blue and I have to adjust the white balance quite a bit. i don't know why as if I converted the MLV with MIRawViewer the white balance is normal. Could you help?

Thanks!

John
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 03, 2015, 01:26:44 AM
Well, it's not particularly important, typically you'll want to make adjustments to the WB anyway, that is one of the advantages to shooting raw.

What did you have the WB set to in camera? Does that not match what MLVFS is producing? I think MLRawViewer ignores the WB metadata in the file, so perhaps the default WB MLRawViewer is using just happens to be correct, while you actually had the WB from the camera set incorrectly and that is the WB MLVFS is using.

If you still think it's a problem with MLVFS, can you upload a sample MLV file that exhibits the problem?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johnwangphoto on February 03, 2015, 04:17:44 AM
Thanks for the reply. I used the Canon 5D Mark III and I set the WB as Auto. The WB should be around 5900K but the file from MLVFS has 3200K when it is opened in Resolve. I can adjust it for sure but just feel something wrong in the setting of the MLVFS. I would love to upload the MLV file for you to check and just tell me how to do so.

John
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on February 03, 2015, 07:12:03 AM
@johnwangphoto Dropbox would do?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: silenttapes on February 03, 2015, 01:37:49 PM
Hey!

Thanks for this app! - it has certainly saved me many hours of conversions and wondering wether to keep originals etc.

Can anyone recommend a work flow for deleting unused clips with in Davinci Resolve? I used to update usage data then delete one by one from my folders, but now they are virtual folders so the files arn't real :D

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 05, 2015, 11:46:05 PM
Compressed (LZMA) MLV file support added. You probably won't see realtime playback speeds for compressed MLV files, I would recommend simply lowering the bit depth of the MLV file if you want compression (to 12 or 10 bits), with that method you still get speedy playback. You'll probably see almost no IQ loss converting to 12 bit if you want to save some space with your MLV files. The LZMA compressed MLV files are not much smaller than 12 bit ones, so I'd drop the bit depth before I tried compression.

There is one issue if you use mlv_dump to change the bit depth of the MLV file and that is: it doesn't also update the black and white levels appropriately (if we drop bits off the image data values, we also need to drop bits off of the black and white levels). I have a fix (https://bitbucket.org/dmilligan/magic-lantern/commits/a22302855d05c6cb587336943c8b5fb623a96bb4) for this, until it gets merged, you can try this mlv_dump (https://bitbucket.org/dmilligan/magic-lantern/downloads/mlv_dump).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 06, 2015, 08:14:12 AM
I've given up after the fact that I learned about how I didn't actually need NVIDIA/CUDA installed in my Mac Mini and now that I've uninstalled it... It still LAGS playback!

Doubtful that I had done it completely. Tried in both Terminal and through Library but couldn't find "Framework" folder in Yosemite. Used to be able to in Mavericks.

I miss using MLVFS on the fly ever since my stupidity in updating the CUDA/NVIDIA unknowing that Ill get myself into this pit.

Not sure what's to do at this point other than trying to find a older version of CUDA/NVIDIA but after learning that I don't need it with the graphic card that I'm using...

Feeling  ???  as of lately...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johnwangphoto on February 06, 2015, 07:31:10 PM
I just found out if I set the 5DIII WB other than Auto the MLVFS can read the set WB correctly but when I set it to Auto WB for some reason the DNG file from MLVFS always has the WB 3150K (very blue) no matter what the actual WB is. Can MLVFS understand Auto WB or I miss something?

Thanks for any comments.

John
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 06, 2015, 07:35:39 PM
Quote from: dmilligan on February 03, 2015, 01:26:44 AM
can you upload a sample MLV file that exhibits the problem?

or at least the output of mlv_dump -v
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johnwangphoto on February 06, 2015, 09:06:45 PM
OK. Here is the Dropbox link for the original MLV clip and a dng file from MLVFS. The clips was shot on a sunny day with Auto WB. Let me know what can you figure out.

https://www.dropbox.com/sh/lrtgf2xnej7vpjk/AABGgmmVvKY472grMEBeGhu-a?dl=0

Thanks!

John
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 06, 2015, 10:56:28 PM

Block: WBAL
  Offset: 0x00000340
    Size: 48
    Time: 405.718000 ms
     Mode:   0
     Kelvin:   3100
     Gain R:   491
     Gain G:   1024
     Gain B:   638
     Shift GM:   0
     Shift BA:   0



        unsigned                wb_mode;  // see property.h for possible values
        unsigned                kelvin;   // wb temperature; only used when wb_mode = WB_KELVIN
        unsigned                WBGain_R; // only used when wb_mode = WB_CUSTOM
        unsigned                WBGain_G; // only used when wb_mode = WB_CUSTOM
        unsigned                WBGain_B; // only used when wb_mode = WB_CUSTOM


This is actually sort of an ML issue. If you have the WB set to "auto", then ML doesn't put any information about the actual WB in the metadata. MLVFS uses the kelvin value in the MLV file even when the mode is "auto", but this value doesn't actually contain the actual auto WB (but there's nothing else MLVFS can do!). In your case it was set to 3100. Don't ask me why. AFAIK, we may not even know how to read the Canon auto WB.

I suppose MLVFS could compute an auto WB itself, but this would be quite complicated, and I don't really see the point. WB is something your NLE/CC software should handle.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johnwangphoto on February 06, 2015, 11:48:07 PM
Thanks for the effort to figure it out and well like you said that is the beauty of the raw shooting: I can adjust to whatever I want. Maybe shooting a color chart and do the match in Resolve will be the best solution.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: poromaa on February 08, 2015, 03:15:26 PM
If I pull the DNGs directly into Premiere, I get a-lot of aliasing problems at sharp edges. However, if I import the DNG trough ACR (Lightroom/Photoshop) I get a totally different result. Even turning off every single setting (noise reduction, color noise etc) I get a clearly better result from the "ACR"-version.

What am I doing wrong here?

Below is a screenshot from within Premiere, where i have stacked the two versions on top of each other.
(http://s17.postimg.org/aw643780f/Screen_Shot_2015_02_08_at_15_10_24.png)
As one can see the aliasing gets really horrible on the file loaded in Premiere (to the right)

I have checked for field order etc but can't find any problems with it. There does not seem to be any debayering-settings in Premiere either, so I don't know what it's doing. I have also played around with the chroma smoothing, but this does not fix it.

Any ideas?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dubzeebass on February 08, 2015, 03:38:21 PM
Is that dualiso and what camera?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 08, 2015, 03:57:19 PM
Quote from: poromaa on February 08, 2015, 03:15:26 PM
If I pull the DNGs directly into Premiere, I get a-lot of aliasing problems at sharp edges. However, if I import the DNG trough ACR (Lightroom/Photoshop) I get a totally different result. Even turning off every single setting (noise reduction, color noise etc) I get a clearly better result from the "ACR"-version.
Well this is to be expected. Premiere does not use ACR. ACR is a very good, custom, CPU based demosaicing algorithm. It's really slow as a result, making it not so good for video. Premiere uses a more generic algorithm that can be hardware accelerated, so that playback is realtime. Therefore it is not as good at removing defects like this.

This really doesn't have anything to do with MLVFS. It's an issue with Adobe (and also the design of your camera, which uses line skipping in video mode, which causes the issue in the first place)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: poromaa on February 08, 2015, 04:41:26 PM
Yeah, the camera (5Dmk2) has some awful aliasing and does not even support progressive dual-iso in video-mode, but I guess I can live with it if using ACR.

My workaround is simply to use daVinci Resolve instead, since that debayering currently looks more like ACR, but gives realtime playback. Drawback with that is that Resolve does not have any free denoise and noisy footage tend to break if color-correcting in Resolve (for me, I guess I have some to learn here).

Well, thanks for the info.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dubzeebass on February 08, 2015, 08:32:37 PM
I wish I could get realtime playback! I have an early 2013 iMac loaded to the max and I have to use node rendering to work quickly.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 08, 2015, 09:55:13 PM
Quote from: dubzeebass on February 08, 2015, 08:32:37 PM
I wish I could get realtime playback! I have an early 2013 iMac loaded to the max and I have to use node rendering to work quickly.

@dubzeebass -- is this currently one of the only few solutions?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dubzeebass on February 09, 2015, 03:51:17 AM
Quote from: DeafEyeJedi on February 08, 2015, 09:55:13 PM
@dubzeebass -- is this currently one of the only few solutions?

That or basic white balance and export as Cinelog-C then re-import the files and apply the LUTs. I often find myself skipping Cinelog-C when it's only a few clips and going right from BMD Film to Hunter's LUT. Not terribly fast but this is a good product... well, both are.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 16, 2015, 09:01:31 PM
I'm currently going to use MLVFS via PP (DR11 stills crashes) just for basic dailies since I have no time for grading plus client likes how I shot it as it is in RAW/MLV (it was well exposed for the most part) so after playing around with @Danne's cr2hdr-r (betas and non betas) which gave me some real decent footage but today I've decided to give it a short run with your app on the "fly" -- it's currently .IDX'ing right now so hoping for good things to come.

Is it normal for it take awhile on a spanned MLV file? (up to 8 spanned files for a single MLV) it took about 3-4 minutes for to spit out an .IDX) lol...

Thanks again @dmilligan!
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 16, 2015, 09:27:30 PM
Also here's what I've BEEN dealing with regarding DR11 for some time now whenever I use MLVFS (I know you mentioned this is solely DR's problem and not with your app) but thought it was worth sharing...

https://vimeo.com/119791222 (https://vimeo.com/119791222)

Another reason why it can be rather difficult to use @Andy600's Cinelogs LUTS in conjunction w MLVFS if DR11 cannot hold up it's performance on my supposedly POS late 2012 modified MacMini!

Perhaps, any one out there have similar experiences?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 17, 2015, 07:05:12 AM
Hi David! How doyou find kelvin values in the mlv files? Are they not to be shown in exiftool efter export with mlv_dump? All I can see is asShotneutral values. Also looking for time code values which alse seems to show somewhere?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 17, 2015, 08:47:11 AM
Ah, found the metadata through mlv_dump.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 17, 2015, 12:50:15 PM
DNG metadata only stores white balance as RGB multipliers (AsShotNeutral). So if you have kelvin in the MLV file, you have to convert it to RGB multipliers for the DNG, and converting to/from kelvin is rather complicated and involves using the cam->XYZ matrices and doing a bunch of matrix transformations.

http://www.magiclantern.fm/forum/index.php?topic=13152.msg128304#msg128304
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 17, 2015, 11:33:40 PM
I have implemented some experimental GIF previews. They are 1/4 res, black and white (from the green pixels so we don't have to debayer; and also because GIF is limited to a 256 color palette, which is further limited to 128 colors b/c of the simplistic way I needed to implement LZW, so it's easiest to just have the color palette be 127 shades of gray), and loop through 10 frames of the MLV file.

These will show up as _PREVIEW.gif in the mount and also in the file tables in the webgui. This is somewhat experimental and needs further testing (you will very likely run into bugs), so I'm not updating the .dmg download in the first post, instead use this link if you would like to test it out: https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS_GIFPreviews.dmg

Let me know what you think.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 18, 2015, 12:04:40 AM
Nice work.
I click the name and got them bigger. Good. Press the back button and you,re back in thumbnail mode. Very handy.

(http://s27.postimg.org/8in99sqr3/Screen_Shot_2015_02_18_at_00_14_33.png) (http://postimg.org/image/8in99sqr3/)

(http://s27.postimg.org/ofhfsrtrj/Screen_Shot_2015_02_18_at_00_14_45.png) (http://postimg.org/image/ofhfsrtrj/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chainick on February 19, 2015, 07:23:55 AM
Can I get auto mount mlv-files in PFMAP (http://www.pismotechnic.com/pfm/ap/) on startup Windows?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Pedr on February 19, 2015, 10:35:57 AM
Am I right in thinking that you need to mount each directory separately?

For example if my directory structure is:

- shoots
  - 15_01_07
  - 16_01_07
  - 19_01_07

If I run the MLVFS service on shoots, I just get it's children as empty dirs in the mounted dir. The only way to get CDNGs from the contents of these dirs is to mount them individually. Is there any way to mount a whole directory structure so that any mlvs within child directories are mounted as CDNGs? I Need to use shots from a large number of directories in a project, so individually mounting each directory every time is a bit of an issue.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 19, 2015, 11:26:00 AM
Love the thumbnails preview @dmilligan... looking forward to its progress!

Possible to have the backspace button work as a "Back" as well as oppose to just coming from right-click only?

Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 19, 2015, 01:24:51 PM
Quote from: Pedr on February 19, 2015, 10:35:57 AM
Am I right in thinking that you need to mount each directory separately?
No. You shouldn't need to. You can mount an entire directory structure, it should work just fine.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 19, 2015, 01:28:23 PM
Quote from: DeafEyeJedi on February 19, 2015, 11:26:00 AM
Possible to have the backspace button work as a "Back" as well as oppose to just coming from right-click only?
Are you talking about in the web browser? I can't do anything about the way your web browser works.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Pedr on February 19, 2015, 06:33:31 PM
@dmilligan That's definitely not working for me. If I mount a directory that contains sub directories, the web UI does show the subdirs, but then locks up when I try and navigate into one. DR is the same - it shows the top-level dirs, then locks up (spinning beachball) for a while, then the directories disappear and the mounted MLVFS drive unmounts.

I can mount individual dirs by themselves though.

If I can help you debug - send logs etc just ask.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 19, 2015, 07:12:42 PM
Quote from: Pedr on February 19, 2015, 06:33:31 PM
If I can help you debug - send logs etc just ask.
you will find crash logs here: ~/Library/Logs/DiagnosticReports
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Pedr on February 19, 2015, 10:21:56 PM
@dmilligan Got plenty of mlrawviewer .crash files, but nothing relating to MLVFS.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on February 20, 2015, 01:44:24 AM
If the mount disappears that means MLFVS crashed, and if it crashed there should be a crash report.

I've never seen or had any issues with subdirectories, I use them all the time, so my guess is it's something else besides the fact that you are using subdirectories that is the real problem. Make sure you don't have any strange characters in any file or folder names. Try different combinations of MLV files and folder structures. For example, if you have a directory that mounts correctly by itself, put it in another directory and mount that.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 20, 2015, 07:49:47 AM
@dmilligan:

I was referring to when you use mouse and right-click then scroll down to 'back' to get back into thumbnail mode.

Regardless of which web browser -- Is this the only way?

And not the backspace button - correct?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 20, 2015, 01:17:14 PM
Quote from: DeafEyeJedi on February 20, 2015, 07:49:47 AM
I was referring to when you use mouse and right-click then scroll down to 'back' to get back into thumbnail mode.
I don't really know what you mean by "thumbnail mode". If you are right clicking on something and bringing up a context menu, then that has absolutely nothing to do with MLVFS, because there are no context menus or "back" or anything like that in MLVFS. If you click on a hyperlink in the webgui and it takes you to another page and you want to get back to the previous page, then how you do that depends on the browser you are using and the various ways it lets you go back to the previous page. The button in the top left corner of the window with an icon of a back arrow is the back button on almost all browsers, and there are numerous other ways that are browser specific. Here's a list of keyboard shortcuts for Chrome: https://support.google.com/chrome/answer/157179?hl=en&ref_topic=25799 I'm sure you can find a similar list for whatever browser you are using.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Pedr on February 20, 2015, 07:16:41 PM
@dmilligan That makes sense. I'll try and narrow down the issue later today. Anyway. It's a great approach. Thanks for all your work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 20, 2015, 08:25:42 PM
@dmilligan -- you were right it had nothing to do with MLVFS so my fault on that.

However, I did figured it out by going through the advance settings within Safari's.

Thank you for the clarification again.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 21, 2015, 09:08:07 PM
I had some time to test and clear up some bugs with the gif previews, it seems to be pretty stable now, so I have updated the main MLVFS.dmg download with the new gif previews feature. (if you were testing the gif previews, I recommend d/l the latest, lots of bugs/crashes have been fixed).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chainick on February 22, 2015, 09:16:40 AM
On Windows 7 for recursive mount mlv-files I use item in context menu on folder

(https://img-fotki.yandex.ru/get/16156/6697399.0/0_f12d0_4a5d5316_orig.png)


Create AddItemToMenu.reg file and run it.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\PismoRecursiveMount]
"MUIVerb"="Pismo recursive mount"
"Position"="Top"
"Icon"="C:\\Program Files\\Pismo File Mount Audit Package\\pfmcontrol.exe"

[HKEY_CLASSES_ROOT\Directory\shell\PismoRecursiveMount\command]
@="cmd.exe /c for /r \"%1\" %%i in (*.mlv) do pfm mount \"%%i\""

[HKEY_CLASSES_ROOT\Directory\shell\PismoRecursiveUnmount]
"MUIVerb"="Pismo recursive unmount"
"Position"="Top"
"Icon"="C:\\Program Files\\Pismo File Mount Audit Package\\pfmcontrol.exe"

[HKEY_CLASSES_ROOT\Directory\shell\PismoRecursiveUnmount\command]
@="cmd.exe /c for /r \"%1\" /d %%i in (*.mlv) do pfm unmount \"%%i\""
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on February 22, 2015, 02:25:27 PM
Quote from: chainick on February 22, 2015, 09:16:40 AM
On Windows 7 for recursive mount mlv-files I use item in context menu

Whats the difference between recursive mount and quickmount? I already have quick mount option to mount mlvs when right-clicking files?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chainick on February 22, 2015, 04:43:23 PM
What will you do if folders nested? For example see next structure of folders:
MyProject
  Day1
    CamA
    СamB
  Day2
    CamA
    CamB


I propose on MyProject folder call to context menu  and run recursive mount.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: sgofferj on February 24, 2015, 03:16:12 PM
Could we have a userspace umount option please? :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 24, 2015, 06:17:46 PM
does:

umount /path/to/mlvfs/mount/point/

not work?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: sgofferj on February 25, 2015, 12:29:09 PM
With sudo, yes. But that kinda defeats the idea of userspace tools, doesn't it? :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 25, 2015, 12:49:25 PM
no, userspace vs. kernel is not the same thing as root vs. standard user, there are plenty of userspace programs that must be run as root.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on February 27, 2015, 11:02:15 AM
Any news for a Windows update? I can't import new MLVs (DNG sequences) into AE... the interesting things is, that old MLVs (DNG sequences) are working properly. Is this because of the version differenc of MLs? What can cause this thing?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on February 27, 2015, 04:04:11 PM
surami,

Using Pismo, I assume? (as opposed to g3gg0's MLVS Windows Client)
Which ML version?
What symptoms when try to import into AE?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on February 27, 2015, 10:37:32 PM
@Frank7D

Yes, I'm using Pismo File Mount + the latest mlvfs.dll (by ayshih). As I remember the old MLV (https://www.dropbox.com/s/k1ut4s3oc4da2ny/M18-1424.MLV?dl=0) was created with a nightly build from previous year, somewhere in april and the new MLV (https://www.dropbox.com/s/pac7egqarj34kko/M16-0963.MLV?dl=0) is created with a build from this year, january. Are you using Pismo too? If yes, could you try the files.

I get this error in AE "the file format module could not parse the file" 45::35, when i try to import the new MLV's DNG sequence. I can adjust the settings in ACR, but after click on OK, the error window comes into my face.

In theory the mlv_rec modul should be the problematic thing or not? I think this because everything works perfectly with the old MLV.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on February 27, 2015, 11:16:53 PM
surami,

Using your mlv file, I reproduced this error.
Actually, this had happened to me as well, using "magiclantern-v2.3.NEXT.2014Feb27.7D203" build.
It only happened the first time I used the build (for several clips). Since then, no problem.

The problem only affects the first frame of the clip; if you advance to the second frame there is no error.

Not sure the best way to handle the problem clips you already have. In my case, I set the work area to start at the second frame and rendered starting there.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on February 28, 2015, 09:21:50 AM
@Frank7D

Thanks for your time to check the MLVs.

This mentioned trick isn't working for me. Another converter like raw2cdng produces good DNG sequence, what I can import into AE, but it would be better not to store unnecessarily the files.
Ayshih, please come back for an update :) or where lies the rub?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 28, 2015, 05:05:26 PM
the files work fine in the current mac version
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on February 28, 2015, 09:06:18 PM
@dmilligan:

hi,

may i report a bug within mlvfs which i found today:

today i had a shoot with some slowmo takes. 50fps/5d mk3
i tried to span it over 2 cards while recording to squeeze everything out of the slots :) (CF as main and SD as secondary recording card.)

now i am at home and when i mount those files with mlvfs i get some missing frames within the take. it looks like those sd card frames are missing.
when i convert it with mlrawviewer instead, the frames are all there and it looks ok.


i stumbled upon another problem.
when the take was longer ml splitted the files on the cf cards. so i have a mlv and a m00 files.
on the sd card i also have a m00 file from the same take. when i copy them together onto the hdd the sd card files got renamed (cause they are the same m00 as on the cf card.)

on the hdd the files are: mlv, m00 and m00 2.
when mounted with mlvfs those takes are also weired.

those files are also weired (frames skipping) with mlrawviewer.

can you please take a quick look if there is something to tune within mlvfs, or how i should name my files to convert them correctly with mlvfs.

i also noticed that mlrawiewer converted dngs are bout 1.3mb and mlvfs dngs are 2.5mb

havent you implemented 2 compressing feature formats?
where can i find that, cause in the webinterface there is no compressing feature.

thank you very much. and hope to hear from you soon,

swin
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on February 28, 2015, 11:36:08 PM
For the spanning issue, can you send me example files? I have no way to test spanning myself.

Rename spanned files so that you have .mlv, .m00, .m01, .m02, .m03, etc. it doesn't matter which files get which number, you just need to have every number sequential.

I did not implement compressed output, nor do I ever plan on that, there's no point. I implemented MLVFS being able read MLV files that have been compressed (you can use mlv_dump to compress MLV files).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on March 01, 2015, 09:06:54 AM
@dmilligan:

thx for your help.

i tried some files with mlrawviewer with success - not all..
within mlvfs there are missing frames (i think these frames are from the sd card which do not find the way back to the source)

i am uploading the files now:
https://copy.com/ulsYvGWuzA80BCIz

in the directory 1 for example mlrawviewer works fine, but with mlvfs there are frames missing.
in the dir 2 the recording stopped and ml created many files from mlv to m08 on the cf card. on the sd card there are also many files from the same take m00,.. m08. i renamed the second charge to m09-m16 and within mlrawviewer this is also working well.


Thank you so much, swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 01, 2015, 08:39:52 PM
Delete your IDX files and let MLVFS re-create them and it works just fine (same with mlv_dump).

My guess what happened is that you tried to playback in camera or you tried to process the files without them named correctly first, which triggered the IDX file to be created, but this IDX files does not contain info about the missing frames in the spanned file. Then when you copied the spanned file and gave it the correct name, the index did not get rebuilt with the frames from the spanned file (because the IDX was already created).

Also, for the situation where you had a whole bunch of  m0X files because recording stopped, you can just delete all of those extra files (as long as the file is not larger than a couple MB, there's nothing it). Those are just created by mlv_rec when recording starts and the unused ones are normally cleaned up, but they don't get cleaned up if recording stops prematurely.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on March 01, 2015, 08:44:18 PM
oh. thank you a lot.
i will try that and will get back as soon as possilbe.

best wishes. swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on March 02, 2015, 10:34:23 PM
@dmilligan:

can you please give me an example of an mlv_dump compression terminal command which is working with mlvfs, cause i tried to compress a mlv but with no success.. (the file got smaller, nearly 45% but mlvfs can not convert it anymore.)
my command was:

mlv_dump -o Compressed.mlv -v -e -c -b 16 -l 9 -x <filenamexyz.mlv>

thank you very much.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 03, 2015, 12:20:18 AM
delta encode mode (-e switch) is not supported by MLVFS, also mlv_dump doesn't handle white and black level properly when changing bit depth (IMO), so if you use -b 16, the output black (and white) level from MLVFS will be wrong. mlv_dump will actually output the correct black level if you change bit depth, but that is because it assumes the black level is 14 bit (IOW, it assumes the original data is 14 bit). I don't like to make that assumption.

Here's a fix: https://bitbucket.org/dmilligan/magic-lantern/commits/a22302855d05c6cb587336943c8b5fb623a96bb4

The other thing is that LZMA is quite slow, and you'll never get realtime playback with it. That's why I also implemented the new compression scheme from here: https://bitbucket.org/hudson/magic-lantern/branch/mlv_rec_lj92
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on March 03, 2015, 10:00:34 AM
hello dmilligan

thank you for the explanation.
so how can i compress them with the new option?
is there a simple batch possibility to compress - lets say a complete directory..

thank you very much.
sw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 03, 2015, 11:14:14 PM
Quote from: swinxx on March 03, 2015, 10:00:34 AM
so how can i compress them with the new option?
use this mlv_dump (https://bitbucket.org/dmilligan/mlvfs/downloads/mlv_dump)
like this

./mlv_dump -o <output.mlv> -c <input.mlv>
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on March 04, 2015, 10:02:34 AM
Finally managed to record a timelapse with fullres silent pics (nothing spectacular, just something to try if it works), and I installed MLVFS to postprocess it. Wanted to try it for a long time, and this was the perfect occasion (since I don't really record raw videos anyway).

Already did some minor tweaks, to address the small installation difficulties I had:
https://bitbucket.org/dmilligan/mlvfs/pull-request/5/minor-tweaks-for-command-line-interface/diff

Problems found:
- an invalid MLV crashes MLVFS right away SOLVED
- the web interface doesn't work, browser gets stuck at "Request sent" SOLVED
- running cr2hdr in the virtual FS crashes after processing a few files (will try to fix, but I think I need more time)
- copying DNGs from the virtual FS to the real FS (same HDD drive, not SSD) is about twice as slow as a straight file copy (~20MB/s vs ~40MB/s, will look into it)
- exiftool can't change black level of the DNGs created by MLVFS (minor, probably some bad header)
- with spanned files, there is a black frame for each chunk (!), most likely the last frame, need to double-check (mlv_dump handles these frames properly)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Pyriphlegethon on March 06, 2015, 12:01:02 AM
null
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 08, 2015, 08:59:34 PM
Quote from: a1ex on March 04, 2015, 10:02:34 AM
- the web interface doesn't work, browser gets stuck at "Request sent", not sure why (tried to change port to 80000, no luck)
I think this might have to do with background mode and the threading model. Try running with -f. (I always use -f so I can debug and watch console output, so I've never noticed this issue). When I tried without -f on OSX, the webgui didn't work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on March 08, 2015, 11:10:59 PM
Correct. When I run it with -f, I get "Segmentation fault" right when I connect to a web browser on 127.0.0.1:8000.

That's the only message I get printed on the console.

Valgrind log:

==26190== Memcheck, a memory error detector
==26190== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==26190== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==26190== Command: ./mlvfs MLVFS -f --mlv-dir=/path/to/mlvs
==26190==
webgui start
webgui run
webgui poll
webgui poll
webgui poll
webgui poll
webgui poll
webgui poll
webgui poll
==26190== Thread 2:
==26190== Invalid read of size 8
==26190==    at 0x409059: xref_sort (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x4098A7: make_index (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x409B0C: build_index (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x409E23: force_index (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x409E79: get_index (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x409F15: mlv_get_frame_count (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x4375D5: webgui_generate_mlv_html (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x437F1E: webgui_generate_html (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x4385A0: webgui_handler (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x42AE5A: call_user (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x42FD78: call_request_handler (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x4309DF: call_request_handler_if_data_is_buffered (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==26190==
==26190==
==26190== Process terminating with default action of signal 11 (SIGSEGV)
==26190==  Access not within mapped region at address 0x0
==26190==    at 0x409059: xref_sort (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x4098A7: make_index (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x409B0C: build_index (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x409E23: force_index (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x409E79: get_index (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x409F15: mlv_get_frame_count (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x4375D5: webgui_generate_mlv_html (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x437F1E: webgui_generate_html (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x4385A0: webgui_handler (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x42AE5A: call_user (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x42FD78: call_request_handler (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==    by 0x4309DF: call_request_handler_if_data_is_buffered (in /home/alex/src/ml/mlvfs/mlvfs/mlvfs/mlvfs)
==26190==  If you believe this happened as a result of a stack
==26190==  overflow in your program's main thread (unlikely but
==26190==  possible), you can try to increase the size of the
==26190==  main thread stack using the --main-stacksize= flag.
==26190==  The main thread stack size used in this run was 8388608.
==26190==
==26190== HEAP SUMMARY:
==26190==     in use at exit: 868,724 bytes in 105 blocks
==26190==   total heap usage: 1,096 allocs, 991 frees, 1,604,352 bytes allocated
==26190==
==26190== LEAK SUMMARY:
==26190==    definitely lost: 917 bytes in 34 blocks
==26190==    indirectly lost: 0 bytes in 0 blocks
==26190==      possibly lost: 1,088 bytes in 4 blocks
==26190==    still reachable: 866,719 bytes in 67 blocks
==26190==         suppressed: 0 bytes in 0 blocks
==26190== Rerun with --leak-check=full to see details of leaked memory
==26190==
==26190== For counts of detected and suppressed errors, rerun with: -v
==26190== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)


edit: it was caused by an invalid MLV (created by latest fullres silent code). Deleting that MLV solved the crash.

edit2: fix commited, thanks dmilligan for write access.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on March 09, 2015, 08:46:06 PM
@dmilligan

would it be possible to compile a new mac version with the latest code changes at the bitbucket download section :)

thank you very much swinxx
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 10, 2015, 01:42:33 AM
done
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 10, 2015, 08:56:21 AM
Thanks for the update regarding the latest cr2hdr_20-bit from @a1ex into your app @dmilligan!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 10, 2015, 11:54:09 AM
huh? I have not updated the cr2hdr code. It is based on, but slightly different from the main cr2hdr code. The main differences are:
it doesn't use dcraw, the data comes internally straight from the MLV file
it doesn't use exiftool, dng header data is written by mlvfs itself
it's thread safe
some gcc specific language features were removed so it would compile with llvm
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 10, 2015, 06:33:16 PM
Okay gotcha -- my fault for assuming that you got the updated cr2hdr compiled for Mac by N/A that he released just few few days ago...

Will give it another test run w MLVFS with a quick Dual-ISO MLV footage (hopefully it won't come out pinkish again) although it still comes out in different kind of pink cast for non Dual-ISO MLV's as well.

I also updated the Adobe DNG to the latest firmware in case you were wondering.

Thanks again!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on March 10, 2015, 08:56:41 PM
I think dmilligan knows how to compile cr2hdr as well  ;)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on March 10, 2015, 10:34:06 PM
Ofc, I would assume so.

Doesn't hurt to ask.

Thx.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on March 13, 2015, 04:37:30 AM
Getting crazy vertical stripes and green tint occasionally with  the latest mac mlvfs. Doesn't happen all the time and not every mlv either, seems to happen when I first log on and mount my folder. Ejecting the drive, closing my browser and re-mounting it seems to fix it. Toggling vertical stripe correction and bad pixel fix doesn't fix it. Going to try to replicate the issue and see if its random or if certain mlv's and settings cause it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on March 13, 2015, 02:21:10 PM
Just want to share my "MLVFS server based converter"


So now, from my W7 PC i can put any MLV file in the mlv-in share and get the DNGs in the mlv-out folder share

Perfect !  :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mario1000 on March 13, 2015, 04:34:43 PM
Quote from: vertigopix on March 13, 2015, 02:21:10 PM
Just want to share my "MLVFS server based converter"


  • Server is clearos-community-6.6-x86_64
  • Installed FUSE (from clearos-core repo)
  • Compiled MLVFS from BitBucket
  • Created 2 shares (Flexshare) mlv-in and mlv-out
  • Created the file /etc/fuse.conf with user_allow_other argument
  • Finely lauched  mlvfs -f /var/flexshare/shares/magiclantern/mlv-out --mlv_dir=/var/flexshare/shares/magiclantern/mlv-in -o allow_other &

So now, from my W7 PC i can put any MLV file in the mlv-in share and get the DNGs in the mlv-out folder share

Perfect !  :)

Do I understand it right that this is a solution for windows users to use the latest update of dmilligan? If this is the case could you please provide a more detailed step by step guidance for users. What I mean is: does your solution needs linux or just windows... How do I setup Flexshare?

Sorry if this question seems to be a little stupid but I am a user and not a very skilled programmer.

Thanks in advance
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 13, 2015, 05:02:58 PM
You need a Linux box to do this (or a VM running Linux).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arrinkiiii on March 13, 2015, 09:26:04 PM

....almost !!! grrrrrrrr ...for windows users!!!   :'(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on March 13, 2015, 09:35:50 PM
Arrinkiii: Running a VM is not that complicated. Try VMware Player.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 13, 2015, 09:46:17 PM
I've ran VMware on PC @ work since they don't have Mac's and MLVFS works quite well as per @dmilligan's suggestion.

However, I prefer to do it solely Mac's system alone.

*feedbacks*:

@dmilligan -- Based on my recent quick tests from earlier it seems I'm still getting strange pink frames from dual-ISO MLV files. Looks like it doesn't complete process during render before exporting from PP) Am I to export from the mount itself (with dual ISO and cs 2x2 enabled ON) or do I need to drag files onto system before I export it from PP?

Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 14, 2015, 02:22:35 PM
I'm not completely sure what you are asking. When I have time, I'll look into the dual ISO issue and see if I can duplicate it (I don't use dual ISO very much)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arrinkiiii on March 14, 2015, 04:31:34 PM

Thank you guys, i will try the VMware in my PC =)) 

Two days a go i bought one MackBook Pro, second hand, dual-core processor, do you guys think is ok for the MLVFS  or is to old/slow ?? 

Thanks for all  :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 14, 2015, 04:43:31 PM
The speed of your storage media is much more important than CPU specs, when it comes to MLVFS. If it's a SSD then you should be good. I get realtime performance on a 2012 macbook air as long as I use a SSD. (MLVFS typically only uses around 30-70% of one core).

A separate consideration is the ability of the computer in regards to your NLE. Typically you'll need a decent GPU for realtime playback of raw material. You didn't specify how old your MBP is, so no way to know the specs. Your NLE will be much more demanding than MLVFS (MLVFS's job is pretty easy, all the real "work" it does is bit-unpacking).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arrinkiiii on March 14, 2015, 04:53:24 PM

Thanks dmilligan, its the first time that i got mac... don't no sh** abou this but im learning. I must say that i bought this because i want to try the MLVFS. It don't have SSD but i will get one and change it.

My MBP is:

-HDD -> will change for SSD
-4 GB 1067 MHz DDR3 -> will change for 8GB
-NVIDIA GeForce 320M 256 MB
-2,4 GHz Intel Core 2 Duo   (will change for i5 or i7 one day lol )

I use Premiere, do you think with the extra memory (8GB total) plus the SSD i will get realtime playback in premiere?




Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dubzeebass on March 14, 2015, 09:11:03 PM
I've had to stop using this program because the drive disappears when I try to read the files in Resolve, though the mount points still shows up on my desktop. When I try to open it nothing is there, if I restart my iMac and then reload MLVFS it will work for a bit but then I'll try and view the files in the folder again and it will stop working. The only thing that works for a while is uninstalling OS X fuse and reinstalling it and then it stops working again after some short time. What logs do you need to troubleshoot?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 14, 2015, 11:57:54 PM
If you open up the "Console" app, on the left side there's a section "User Diagnostic Reports", there should be mlfvs crash reports there. Also, you can just go to ~/Library/Logs/DiagnosticReports
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 15, 2015, 02:18:54 AM
Congrats to @arrinkiiii for jumping onto the Mac Wagon. You'll never regret it!

As for the SSD -- it was rather easy to DIY on swapping HDD for SSD by watching YouTube videos online as I did with my 2006 MBP (upgraded to 360GB SSD as well 8GB memory ram) in which definitely made a difference and worth the effort! It also played a role in using MLVFS because I couldn't get smooth playback with the original specs in this particular MBP.

SSD have also gotten cheaper in prices lately respectively. Just make sure you have the proper tools as I had to get these in order to swap them properly. Most usually come with Clone-adapter Cable Kit.

Have fun! [emoji106]
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arrinkiiii on March 15, 2015, 01:20:01 PM

Thanks  DeafEyeJedi =DD Yes, i saw it in youtube, also you can have 2 disks, removing the cd-rom and use a caddy for other disk, it will be more slow than the main one but it's good for data/stuff. Need money  :P

-Clone adapter cable? For cloning the disk? i Think will install all again =))

With your BMP (ssd + 8gb) can't you have real play in Premiere?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on March 16, 2015, 03:54:37 PM
QuoteDo I understand it right that this is a solution for windows users to use the latest update of dmilligan? If this is the case could you please provide a more detailed step by step guidance for users. What I mean is: does your solution needs linux or just windows... How do I setup Flexshare?

Sorry if this question seems to be a little stupid but I am a user and not a very skilled programmer.

Thanks in advance

Sorry to be late to answer,

ClearOS is a SOHO linux server. I use it for my backup, dlna, vpn server, and so on.
With this distro, it's possible to install fuse and to compile mlvfs without problems.
Flexshare is a file sharing system included in ClearOS (http://www.clearcenter.com/support/documentation/user_guide/flexshare)

You can install it on a small pc or a VM like VirtualBox. I have tried the same (without Flexshare) in Xubuntu 14.04 with success.

So learn to install linux in a VM (it's easy !) and i can help you with mlvfs.

Tips : VM : https://www.virtualbox.org/ Xubuntu : http://xubuntu.org/getxubuntu/

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mario1000 on March 16, 2015, 06:40:43 PM
Quote from: vertigopix on March 16, 2015, 03:54:37 PM
Sorry to be late to answer,

ClearOS is a SOHO linux server. I use it for my backup, dlna, vpn server, and so on.
With this distro, it's possible to install fuse and to compile mlvfs without problems.
Flexshare is a file sharing system included in ClearOS (http://www.clearcenter.com/support/documentation/user_guide/flexshare)

You can install it on a small pc or a VM like VirtualBox. I have tried the same (without Flexshare) in Xubuntu 14.04 with success.

So learn to install linux in a VM (it's easy !) and i can help you with mlvfs.

Tips : VM : https://www.virtualbox.org/ Xubuntu : http://xubuntu.org/getxubuntu/



Hi vertigopix,

thank you very much up to this point. I downloaded and installed VM and Xubuntu as you have suggested and everthing runs without any problem. What are the next steps to install mlvfs?

Thanks in advance!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 16, 2015, 10:55:01 PM
Quote from: dmilligan on March 14, 2015, 02:22:35 PM
I'm not completely sure what you are asking. When I have time, I'll look into the dual ISO issue and see if I can duplicate it (I don't use dual ISO very much)

Here's to what I was referring to earlier regarding Dual-ISO (and regular MLV as well) showing pinkish effect after being exported.

MLVFS:
(https://farm8.staticflickr.com/7607/16837641825_bdf23b40b2_n.jpg) (https://flic.kr/p/rDTjip)

Dual-ISO MLV (Dual-ISO + cxs 2x2 enabled ON) via MLVFS > PP:
https://vimeo.com/122361687 (https://vimeo.com/122361687)

Dual-ISO MLV (@Danne's cr2hdr-r (http://www.magiclantern.fm/forum/index.php?topic=13512.msg130562#msg130562cr2hdr-r) ):
https://vimeo.com/122219543 (https://vimeo.com/122219543)

Non-Dual-ISO MLVFS > PP:
https://vimeo.com/122361688 (https://vimeo.com/122361688)

Non-Dual-ISO MLVFS > AE (graded w ACR):
https://vimeo.com/122363848 (https://vimeo.com/122363848)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 17, 2015, 02:05:33 AM
I can't reproduce. My dual ISO clips look fine in Premiere. Does it only happen when you export? (Do they look correct just viewing them?) Can you share the original MLV file for a short clip that exhibits the problem.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marekk on March 17, 2015, 02:07:59 PM
@dmilligan - is there a possibility to add option to preload some number of frames not only from current file but also from next available file ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 17, 2015, 02:16:32 PM
how would mlvfs be able to figure out what the "next" file is?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marekk on March 17, 2015, 04:08:41 PM
In current version there is an option to preload some frames. When I add all MLV files to Resolve timeline for grading and exporting to prores current CDNG is played smoothly. At the beginning of next file there is a 1-2 seconds period of interruption. So.. when MLVFS is currently working with FILE_1.MLV, we can preload some frames from FILE_2.MLV. Switch from FILE_1.MLV to FILE_2.MLV can be done smoothly. It will work only if files in timeline are sorted by name.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 17, 2015, 04:44:14 PM
Quote from: marekk on March 17, 2015, 04:08:41 PM
there is a 1-2 seconds period of interruption
There must be something else going on, and I don't think what you are asking for will help the situation. Processing a frame should be on the order of 40ms (or less for realtime playback), so if it's taking 1-2 seconds, there's something else going on, that pre-fetching is not going to help. The solution is to figure out what is actually causing the delay, and fixing that.

The pre-fetch is really only useful for when heavy processing options (read: dual ISO) are enabled so that more CPU cores can be utilized. It can actually hurt performance, esp. if you use too high of a number of frames. If you're not at 100% CPU utilization for MLVFS for a single core, then I wouldn't use it at all, it's likely to only make things worse.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on March 17, 2015, 09:41:11 PM
Any way to add an option to webgui to disable gif previews? It's not too bad when there are less than 50 GB of MLV's to preview, but working with 100+ GB of MLV's and a not-so-powerful gpu can slow things down pretty significantly.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 18, 2015, 02:28:23 AM
@N/A: That's a Great idea regarding disabling gif previews!

@dmilligan: Here's the Dual-ISO MLV file from 5D3 per your request.

MLV (https://www.dropbox.com/s/x9muqqodosmz25b/17980007.MLV?dl=0MLV)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on March 18, 2015, 10:29:12 AM
@mario1000

Open Terminal Emulator and type :

sudo apt-get install git
sudo apt-get install libfuse-dev

git clone https://bitbucket.org/dmilligan/mlvfs

cd mlvfs
cd mlvfs   (yes, 2 times !)

make clean
make

sudo cp mlvfs /usr/bin


Create the fuse.conf file and add the 'user_allow_other' value into it

echo "user_allow_other" | sudo tee -a /etc/fuse.conf


If everything is ok, you can create 2 folders (eg. mlv-in and mlv-out)

mkdir /home/<YOUR_NAME_HERE>/mlv-in
mkdir /home/<YOUR_NAME_HERE>/mlv-out


and now, you can launch mlvfs !

mlvfs -f /home/<YOUR_NAME_HERE>/mlv-out --mlv_dir=/home/<YOUR_NAME_HERE>/mlv-in -o allow_other

Don't close the terminal, open a browser and type : 'http://127.0.0.1:8000'

Magic as a lantern !  :)

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mario1000 on March 18, 2015, 12:16:53 PM
Hurra! It works! Thanks a lot for your help!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on March 18, 2015, 02:23:31 PM
and when a new version exist, go to mlvfs base directory and type
git pull

after this
make clean
make


And you can copy the new version to /usr/bin (you need to kill the process before updating)

How to kill  ;)
ps -e | grep mlvfs
sudo kill <process number>
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 19, 2015, 02:06:45 AM
Quote from: DeafEyeJedi on March 18, 2015, 02:28:23 AM
@dmilligan: Here's the Dual-ISO MLV file from 5D3 per your request
Thanks, I do see some issues, not sure what the cause is yet, will have to investigate further.

Quote from: vertigopix on March 18, 2015, 02:23:31 PM
How to kill  ;)
ps -e | grep mlvfs
sudo kill <process number>

A better way would be

sudo fusermount -u /home/<YOUR_NAME_HERE>/mlv-out
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: vertigopix on March 19, 2015, 07:34:45 AM
@dmilligan,

You're right, my mistake... ;-)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arrinkiiii on March 20, 2015, 01:20:32 PM

I must write this... One week ago i bought 2nd hand MacBook Pro  dual-core, five days ago i have change for a 2nd hand MBP i5, from my girlfriend lol, since she just use it for internet, email, etc... 

I thought that being an i5 and a Mac it would have a disk that work at 7200rpm but no, it come with a 5400rpm, cant understand why they didn't do that... for a small extra of money you could have a better disk, at least 7200rpm.

I format the disk that have, 500GB and made a partition, so 2 x 250GB. Install Yosemite OS OXFUSE and MLVFS, also Adobe CC and an CUDA 5.0 drive for work good with adobe.

Put some ML files in a folder on the disk, Ctrl+Click and chose MLVFS for that folder. In that moment it prompt to chose one folder where he will release/convert the ML files to DNG's. Immediately he open a web page with the settings that you want to change.The end!!!  So simple as that. Im amazing!!! 

I threw some DNG's to Premiere and it works good. In my MBP it skip some, a few... frames. For sure that is because of the 5400rpm disk.

Just my thoughts: Wend im in the web page it would be better if the previews didn't star automatically. It's possible to have bigger view of the preview windows? Yes, you can. Just click in the file name of the MLV and you got all the meta data regarding that file and with bigger preview window. Neat =)

I think this app is just for sidemont/convert the ML files and use it with Premiere, Resolve, etc... but would be good if could convert for ProRes and with a LUT of chose. I use MlViewer and i love it, i think each one of them have there one way/purpose of working with MLV's. I like to work with DNG's but for some works ProRes is more then fine.

Saying this, i need to get one SSD and more memory to my MBP lol  Going to read this threat all again for see if i miss something.

Resume: It works good with Premiere. Thans for this =))

Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on March 20, 2015, 02:55:16 PM
ProRes + LUTS = cr2hdr-r
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johnwangphoto on March 20, 2015, 11:05:00 PM
After I converted MLV files with MLVFS the dng files are only available in the MLVFS Mount folder for a little while then they all disappear so i have to do the conversion again. Is it normal?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 20, 2015, 11:50:54 PM
If the mount disappears, that means MLVFS crashed. There should be a log in ~/Library/Logs/DiagnosticReports. Or you can look in the 'Console.app' (located in Applications/Utilities).

Please send the log.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 21, 2015, 08:28:45 PM
Quote from: N/A on March 17, 2015, 09:41:11 PM
Any way to add an option to webgui to disable gif previews?
Done. Previews don't load until you click "Load" (or unless you click on a specific MLV file).
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 21, 2015, 08:36:55 PM
Thanks for the Great feature modification regarding previews @dmilligan!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on March 22, 2015, 12:49:24 AM
Quote from: dmilligan on March 21, 2015, 08:28:45 PM
Done. Previews don't load until you click "Load" (or unless you click on a specific MLV file).
Awesome, thank you. Got a shoot next weekend 5 hours away from home, having a fast, stable and reliable program like MLVFS is a lifesaver. Much appreciated.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on March 22, 2015, 08:13:47 PM
Hey David. Been researching a little on color matrix stuff. Question. I see forward matrix 1 and 2 are static for 5d mark 3. Where did you get these numbers? Also, how can numbers for other cams be found?
ah, see them now, part of the dng.c structure. Thanks!

WOuld you know if the model tags are the same in metadata as for "Canon 5d mark 3"? "Canon Eos 7D" and so on? Plan on implementing with exiftool but of course don,t have all the cameras in my possession. Need to differ them accordingly with camera name.
(http://s9.postimg.org/ijimocf17/Screen_Shot_2015_03_22_at_20_17_13.jpg) (http://postimg.org/image/ijimocf17/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 22, 2015, 10:05:17 PM
Andy600 pulled them from Adobe DNG converter. All you need to do is run a DNG through Adobe DNG converter without those tags and it will add them. Though, you really don't even need to do that, they're all hard-coded in the MLVFS source code.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on March 22, 2015, 10:17:51 PM
Yeah, I got them from looking at your file. Your workflow is benchmark, still I got my own little cr2hdr-r project going on so would like to test some stuff out.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 27, 2015, 12:32:51 AM
FYI: http://www.magiclantern.fm/forum/index.php?topic=14803.msg143311#msg143311

(I've updated this OP with that utility)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: TKF Productions on March 28, 2015, 07:07:13 PM
I'm trying to convert the .mlv file to something useable.  I downloaded the FUSE MLV to cDNG converter and everything worked fine and I got the DNG files, however, when I tried to import them into After Effects I get the following error:

"After Effects error: the file format module could not parse the file. ( 45 :: 35 )

Anybody know how to fix this or how to get the .MLV files properly converted?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 28, 2015, 10:45:08 PM
Just downloaded and updated to the latest MLVFS -- after selecting a mount folder on desktop and once the web host site opens up it says "Local Host not found" I've seen this issue before but don't remember how to get around it? (I've tried both Safari and Firefox)

I tried restarting MBP and reinstalled MLVFS to no avail as well as deleting .IDX files... any suggestions?

*edit*

Tried again after hard reboot (not sure if I needed to do this on MBP) but it kind of helped a little... once the web host shows up with proper details but the mount folder doesn't get 'mounted' for some reason?

(https://farm8.staticflickr.com/7633/16959037581_f7c336a7b3_n.jpg) (https://flic.kr/p/rQBv2F)

hence the 'Mount Folder' highlighted on the desktop screen behind it... strange?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on March 29, 2015, 10:36:06 AM
*update* MLVFS works fine on my Mac Mini but not MBP.

Will look into it some more tmw!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on March 29, 2015, 08:14:39 PM
@TKF Productions,
Can you send me a sample file?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 30, 2015, 08:57:33 PM
Regarding AE (SmartImport2) with MLVFS... (I set the project as 25fps but then when I try to render them it says 30fps and I have to manually custom the FPS to 25 for each before I export in AE -- is this normal?)

https://vimeo.com/123651925 (https://vimeo.com/123651925)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on March 30, 2015, 09:35:38 PM
DeafEyeJedi,

Did you set AE to default to 25 fps for image sequences?
See: http://www.magiclantern.fm/forum/index.php?topic=13292.msg128368#msg128368 (http://www.magiclantern.fm/forum/index.php?topic=13292.msg128368#msg128368)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 30, 2015, 10:02:56 PM
I'm not familiar with that script, but seems to me like it would be an issue with the script not MLVFS.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 30, 2015, 10:17:01 PM
@Frank7D:

Thanks for the inputs. Will look into it.

@dmilligan:

so you wouldn't recommend using MLVFS and AE together then? (even without the SmartImport2 -- MLVFS alone won't bring Audio in AE, right?)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on March 30, 2015, 10:30:13 PM
MLVFS can be used with AE and will have audio; it's easy if you use the script. That's what I do.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 30, 2015, 10:46:19 PM
You speaking of the SmartImport2 script?

Then how come the audio seems strange in this footage I put together after rendering them in AE within MLVFS?

Holden's Key (http://www.magiclantern.fm/forum/index.php?topic=14840.msg143620#msg143620Holden's%20Key)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: TKF Productions on March 30, 2015, 10:55:13 PM
Quote from: dmilligan on March 29, 2015, 08:14:39 PM
@TKF Productions,
Can you send me a sample file?

Thanks for offering to take a look.  I will send you a sample file tonight.  What would be the best way to send to you (doesn't looks like I can attach a file through PM)?  Also, do you just want one of the DNG files?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on March 30, 2015, 10:57:37 PM
DeafEyeJedi,

I'm not at a place where I can listen to the audio right now. Can you describe the issue? Actually, maybe we should have this discussion here: http://www.magiclantern.fm/forum/index.php?topic=12422.msg119832#msg119832 (http://www.magiclantern.fm/forum/index.php?topic=12422.msg119832#msg119832)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on March 31, 2015, 12:31:38 AM
@TKF Productions,
Any file sharing service like dropbox, google drive, etc. I might could diagnose the issue from just a DNG, but I might need to have an actual MLV, and it will make things faster anyway. It can a short/small as you like as long as it exhibits the issue.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: anderSF on April 02, 2015, 06:00:05 PM
Hi!

I'm having a problem with MLVFS where footage shot with a shutter speed of 1/100 (at night, with tungsten lighting) shows up in ACR as 1/110. This seems to cause flicker in the shot, weirdly enough. One should think as long as the footage was actually shot at 1/100, the metadata shouldn't matter.... But that does not seem to be the case, because when I conform the shots through MlRawViewer (which reads the shutter speed correctly), they come out flicker free.

Any ideas?

PS: I'm not doing highlight-adjustments or anything like that in ACR. Nothing that could cause flicker.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 02, 2015, 08:48:30 PM
ACR can cause flicker on it's own, without any adjustments. I really doubt it's the metadata. If you want to test it, you can copy the DNGs out of the mount and change the metadata with exiftool.

I'd also be interested to see the output of exiftool -v for DNGs from MLVFS vs. MlRawViewer, to figure out why it's 1/110 instead of 1/100. The output of mlv_dump -v would be helpful too.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: anderSF on April 02, 2015, 09:37:01 PM
Thanks for the quick reply! But I haven't really used any non-GUI software before, so I would have to look into that first....
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 02, 2015, 10:26:25 PM
okay, then just send me DNGs from both, and/or the mlv file
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on April 03, 2015, 12:38:32 AM
*FEEDBACKS*

From my previous post
Quote from: DeafEyeJedi on March 29, 2015, 10:36:06 AM
*update* MLVFS works fine on my Mac Mini but not MBP.

Will look into it some more tmw!

After all, I still don't understand how come MLVFS all of a sudden doesn't work on this MBP (when it previously has with no problems in the past!) still even after deleting all .IDX files...
(https://farm8.staticflickr.com/7618/16827741379_b7861db4f7_n.jpg) (https://flic.kr/p/rD1zf4)

Strange that even with WebHost not working properly it still spits out 2-3 .IDX files in the Mount Folder but not for all... not sure if noting this would help?

(https://farm8.staticflickr.com/7609/17012741162_4cc5b6a125_n.jpg) (https://flic.kr/p/rVmKeu)

MBP Specs:
(https://farm8.staticflickr.com/7585/17012517382_ed19db6180_n.jpg) (https://flic.kr/p/rVkAHd)


Hoping there's something that I may have miss out on solving this issue...

Thanks!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on April 03, 2015, 01:24:43 AM
So, does the mount still work okay? Just the webgui doesn't come up? Can you try running MLVFS from the command line? You can also try using a different port.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on April 03, 2015, 02:42:48 AM
No, the mount does NOT show up on desktop at all on MBP as well as the webgui -- I'll try running MLVFS from command line.

I think you have the directions on how to use Command Line on your first post, correct?

Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eriklieders on April 03, 2015, 10:05:41 PM
Hi all! I have the strange problem... I made some changes with cDNG sequence in ACR After Effects, BUT! I cant see any changes with any frames in Virtual folder (mlv folder). I want to make Color Correction in AE, then unmount and edit corrected MLV video through direct import (with Drastic help) in Premiere with color corrected frames. What im doing wrong? Why i cant see any ACR changes (1st frame) in windows explorer (or when i use ACDSEE). Adobe dng codec 2.0 is installed...

Windows 8.1 64. Thanks a lot!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on April 04, 2015, 12:57:38 AM
ACR tries to write it's settings into the DNGs as metadata or into sidecar xmp files. The mount is going to be read only, so that won't work. I've added support for allowing things to write files in the mount in the Mac/Linux version, so ACR can create and save its xmp sidecar files. But the Windows version is way behind and it may not even be technically possible to implement something like that.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eriklieders on April 04, 2015, 01:54:16 AM
But i thought ACR uses database or xmp. I switched to DB. It must be not write any data to dng
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on April 04, 2015, 03:43:03 AM
The windows/pismo file mount version of MLVFS does allow xmp sidecar files to be saved (in .MLD folders), so your ACR changes will be saved.
However, programs other than ACR may not be able to read these files.
Here's a thread with an overview of one possible workflow: http://www.magiclantern.fm/forum/index.php?topic=13354.0 (http://www.magiclantern.fm/forum/index.php?topic=13354.0)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eriklieders on April 04, 2015, 04:25:10 AM
I dont know how, but previously (before reinstall os) all changes were in 1st dng file. With xmp... Its good, but for why i need it? I cant apply xmp to mlv without mounting. I just want to colorize mlv with ACR and work without recoding to Intermediate format (mxf, prores e.t.c). 1. Edit with arc (wb). 2. Edit corrected mlv direct in premiere. Is it possible?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on April 04, 2015, 04:36:45 AM
I don't know how you can colorize the mlv with ACR and then edit it without mounting it. I think if you're using ACR you need to transcode to something else in order to keep the color. You can edit mlv (mounted to virtual CDNGs) directly in Premiere CC but then you can't use ACR (I believe).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 04, 2015, 06:19:31 AM
Yeah, sorry I misread the original question. You cannot use ACR settings with anything other than ACR. Premiere will not apply ACR settings, because Premiere uses a different raw engine that is not ACR. And MLVFS is certainly not capable of applying ACR settings back into a MLV file (which is impossible anyway). If you want to use ACR, you must render out to an intermediate format from either AE or photoshop (something that uses ACR).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eriklieders on April 04, 2015, 06:27:10 AM
With a help of Drastic software i can direct edit mlv without transcoding. But the question is - how to colorize dng inside mlv and get the corrected video even after unmount? I mean mlv video must be colorized after manipulations with dng inside. All we need - to have abilities to write dngs. This workflow can save 80% of time...
Edited: somehow couple month ago i got working changes in mlv mounted folder (i changed color and exposure). First frame was exactly like i made in ACR. ...Confused....
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on April 04, 2015, 06:45:38 AM
I don't use Adobe CC so not sure if this would work, but have you tried:

1. Mount MLV with MLVFS.
2. Edit (virtual) CDNG sequences in Premiere CC, then save project.
3. Open Premiere project in After Effects CC, then do a "replace footage" for each clip so that they open in ACR, where you can color them.

This would mean coloring the footage after it is edited, rather than before.
And once you're in After Effects, you can do a lot of other cool stuff too, of course.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eriklieders on April 04, 2015, 06:50:58 AM
... For each clip... Its about 9500 clips... (7 lovestorys and 2 wedding videos + 2 music videos)... Thats why i need more easy and faster workflow... Or buy raw camcorder :(

Ok, anyway, thank you for the tips!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on April 04, 2015, 06:52:50 AM
But...you still have to color every individual clip if you use ACR, no matter how you do it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eriklieders on April 04, 2015, 06:57:50 AM
Yes. But without transcoding. It saves a lot of time.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on April 04, 2015, 07:22:12 AM
If my 3-step proposed workflow works (and I don't know for sure that it does) then there is no transcoding involved.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 04, 2015, 07:16:10 PM
Quote from: eriklieders on April 04, 2015, 06:27:10 AM
But the question is - how to colorize dng inside mlv and get the corrected video even after unmount?
Impossible. As I have already stated. You cannot apply ACR settings back into an MLV file. MLVFS gives the impression that an MLV file is just composed of a bunch of DNGs. That is not the case. MLVFS transcodes from the MLV format to DNG on the fly, making it appear like there are just a bunch of DNGs inside. There are not actually a bunch of DNGs inside. The MLV format does not support any sort of method of storing ACR metadata inside.

I don't really know anything about how exactly drastic's software works, but I don't think it uses ACR, so even if you could put ACR metadata inside an MLV file (which you can't), your plugin wouldn't be able to use it anyway.

Like I said previously, the only software capable of applying ACR settings is ACR itself. So if you want to use ACR, you HAVE to transcode out of AE. There is no way around this.

If this is too slow, then I suggest dropping ACR from your workflow. It is not intended for video anyway, and there are much better tools out there for video CC, that are actually intended for that purpose.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on April 04, 2015, 08:50:04 PM
MLVFS > AE (ACR) > NLE (FCPX, PP, etc) does your body good!

@eriklieders: you can try saving a preset (.xmp file) with your first file and then just paste them for the remaining files within ACR via AE.

(if you keep the exposure/WB the same during a shoot which you normally should when possible) then this workflow should suit you well enough.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on April 05, 2015, 08:11:41 AM
For those of us using the PISMO file mount utility in Windows for our MLVS files:  It works fine for normal MLVS videos but won't mount/open an MLVS file created via FRSP Silent_Pics.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chmee on April 05, 2015, 01:56:14 PM
@dmilligan
i updated/added the 500D with DNG-Data. line 77 io.cs raw2cdng (https://bitbucket.org/chmee/raw2cdng/src/b67ee70dc91aa7143df66f00bb770156b6bae8a2/csharp2010_code/raw2cdng_v2/io.cs?at=master)

regards chmee
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on April 05, 2015, 10:52:17 PM
Thanks chmee.

@Ottoga,
Yeah the Windows port is pretty far behind because there's no developer with Windows that is keeping it up to date. The Mac/Linux version was fixed to work with FRSP a while ago.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on April 06, 2015, 05:35:58 AM
@dmilligan

I have an old Pentium 3 at home. Might have to dust it off and make a Linux box out of it. (slightly off topic.... any recommended distro?)

The good news for now is that I managed to extract the DNG's using MLV_Dump.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 06, 2015, 06:02:48 AM
I sort of doubt a pentium 3 could handle MLVFS, esp. the full res stuff (just 1080p24 on my 2ghz core i5 uses 60-70% cpu). You would have much better performance with a VM. There are some descriptions of setting this up a couple pages back in this thread.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on April 06, 2015, 11:51:21 AM
@dmilligan,

Thanks.  I'll look into the vm option. never set one up before, so a first time for everything.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: janoschsimon on April 22, 2015, 11:27:55 PM
will the 70D matrix coefficients be added in dng.c? https://bitbucket.org/dmilligan/mlvfs/src/d961f752f326856081a42f294e22ffca301df875/mlvfs/dng.c?at=master

cheers and thx janosch :-)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on April 22, 2015, 11:46:44 PM
If somebody tells me what the coefficients are, or sends me a DNG from a 70D with the matrices in it, I'll do it.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on April 23, 2015, 02:01:10 AM
Also got some 70D footage to test out with MLVFS -- first I need to know if you already have a copy of DNG from 70D?

If not, I can send you one... Dual-ISO or non Dual-ISO or Cdng?

Just let me know...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on April 23, 2015, 07:48:52 AM
Here,s a dng
https://drive.google.com/file/d/0B4tCJMlOYfirbndxRkt0cy0tX0U/view?usp=sharing

Got it from here. They share test cr2 files for most if not all canon cameras.
http://www.photographyblog.com/previews/canon_eos_70d_photos/
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: janoschsimon on April 23, 2015, 08:06:55 AM
ah you need a photo dng or one from the video cdng? :-)

cheers janosch
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: silenttapes on April 23, 2015, 08:13:26 PM
I have started to experience a problem with MLVFS.

MLVFS has worked great for a few months. Now every so often when I go to load a large folder (500GB) of MLV for us in Davinci it just won't load. The MLFVS folder keeps ejecting its self, or Davinci doesn't see that the files are in davinci mode and instead see's empty folders. I end up restarting my computer about 10 times, reloading the folder over and over and eventually the problem is solved. It's then permanently solved and I can load and unload the footage repeatedly no problem. It seems to happen when have round tripped back to Davinci having not loaded the MLFVS for a while.

Does anyone else have this problem? Does anyone know how to overcome it in a timely manner?

Cheers,

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 23, 2015, 08:40:31 PM
are there any MLVFS logs in ~/Library/Logs/DiagnosticReports?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chmee on April 24, 2015, 03:17:21 PM
@dmilligan here (https://bitbucket.org/chmee/raw2cdng/src/ef55e16df51fc66249f2215d0fffe269f0807766/csharp2013_code/raw2cdng_v2/io.cs?at=master) you go. 70D is in. (oops, i'm wondering why they re as 6D, have to doublecheck)

regards
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on April 25, 2015, 10:56:41 PM
@dmilligan:

Any progress towards an update for MLVFS to be able to read 70D files yet?

I have a bunch of small few seconds test clips (MLV/RAW) in ISO 800 and ISO 1600 (where you get to see a lot of hot pixels I suppose if that would help you, just let me know!

Thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on April 28, 2015, 04:53:23 PM
Quote@dmilligan here you go. 70D is in. (oops, i'm wondering why they re as 6D, have to doublecheck)

regards

Viewing exif on two different 70D raw-dng files seems 6D and 70D share the exact same matrix specification so the numbers seems right.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on April 28, 2015, 05:56:01 PM
Is that to be expected especially coming from Canon?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on April 28, 2015, 08:30:46 PM
On further notice seems relying on dcraw color matrix 1 is recommended for now. The matrix reported as from 6D to th 70D are wrong and will give greenish pics. Don,t think adobe software is updated with this cam yet.

*updated greenish prores files. Not the dng files. This needs more investigation.

*update. Seems I could be wrong on the prores transcoding. I put the wrong numbers in. The chmee matrices might be right.

*update- Now also ProRes transcoding in cr2hdr-r seems to work. Conclusion. Chmee 70D/6D matrices seems right. Only did brief tests.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on April 29, 2015, 09:50:48 AM
I can confirm that regarding 70D/6D matrices seems similar since the ProRes files came out well from @Danne's latest cr2hdr-r for both MLV & RAW.

Looking forward to using these files for MLVFS as well...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 03, 2015, 04:57:43 AM
I have updated MLVFS with the 70D matricies (they are the same as 6D)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on May 03, 2015, 06:46:19 AM
Sweeeeeet can't wait to test it out -- Thanks, David!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: janoschsimon on May 03, 2015, 09:56:17 AM
thx david will give it a try later :-)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: masc on May 05, 2015, 06:34:18 PM
Quote from: silenttapes on April 23, 2015, 08:13:26 PM
I have started to experience a problem with MLVFS.

MLVFS has worked great for a few months. Now every so often when I go to load a large folder (500GB) of MLV for us in Davinci it just won't load. The MLFVS folder keeps ejecting its self, or Davinci doesn't see that the files are in davinci mode and instead see's empty folders. I end up restarting my computer about 10 times, reloading the folder over and over and eventually the problem is solved. It's then permanently solved and I can load and unload the footage repeatedly no problem. It seems to happen when have round tripped back to Davinci having not loaded the MLFVS for a while.

Does anyone else have this problem? Does anyone know how to overcome it in a timely manner?

I think I have the same problem. For smaller projects around 50GB it works great.

Now I have a bigger project with around 400GB (around 350 mlv files). Mostly the folder (mlvfs drive) mounts first, but in the browser I can't see a connection to the page which should pop up. If I click on the folder, it will be away up to the next system restart - no way to eject it or to access.

If I take only a few files of the project, around 150GB (100 mlv files) all seems to work fine. With only some more files the problem begins: the page will first be shown in the browser and I can access the mounted folder, after some minutes the page is away and the folder too.

/Library/Logs/DiagnosticReports is empty.

Thanks for your help in advance!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 05, 2015, 07:08:18 PM
It's probably the index files.

MLVFS creates indexes of all your MLV files (if they don't already exist). These files are named the same as your MLV file, but end in .IDX. This indexing has to happen before MLVFS can give any information about a particular MLV file (this is because metadata and/or frames can appear anywhere in an MLV file, and the MLV format does not require that a frame count be in the first header).

If you put 500GB of files all in the same directory and then mount that directory with MLVFS, when the webgui comes up it wants to display information about all those files (same thing if you browse to the folder). So it tries to index all of them. For 500GB this will take a while. My guess is your issue has something to do with this (maybe you just didn't wait long enough, or there was some kind of timeout that cause MLVFS to crash).

Watch the folder where your original MLV files are located when you launch MLVFS. You should see these files starting to get created (if they are not already existing). Also keep an eye on disk, CPU, and memory usage of MLVFS. Then let me know what is going on.

The issue also could be related to corrupt index files. I would suggest you delete all the .IDX files before trying this. Then watch what happens as MLVFS creates them back.

One solution is to split up your files into separate subdirectories, that way MLVFS doesn't try to index all of them immediately all at once (it will only try to index files in a particular sub-directory if you browse to it).

Once all the index files are created, you won't have to wait anymore. My guess is this is why @silenttapes eventually got it to work. Each time he tried, MLVFS created a couple more index files, and eventually all of them got created.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: masc on May 05, 2015, 08:53:33 PM
Thank you for your fast answer!
What I did now: I deleted all IDX files. Then I mounted again. As you said it builds the IDX files again. And you are right - I was to fast - that takes for this amount of data on my hardware around 15min (the drive is slow, I know...)! If I click in this time on any option on the webgui, mlvfs stops immediatelly creating IDX files. So I closed my browser when it pops up to not disturb mlvfs. RAM und CPU consumption is very low. Only <20MB RAM and <25% CPU. After all IDX are created I tried to start the browser watching the webgui... the gui is not there (server not found). Clicking on the mounted drive makes it away... mlvfs is listed no longer in the OSX activity window then.
After a system restart I tried mounting again. I waited until the CPU usage of mlvfs is listed with 0%. Opening the browser and going onto the webgui brings mlvfs to 100% CPU usage (on one core) and around 110MB RAM. There seems to be no access on the hard drive where the MLV files are located, the disk LED is not blinking. I can access the mounted folder and I see the DNGs, but I can't access the webgui. I am waiting now another 20min and nothing changes... the browser says "waiting for localhost...".
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: masc on May 05, 2015, 11:55:57 PM
Now I tried your second tip - subfolders. This seems to work. Thanks for your help! With that little workaround I can live. :-)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on May 06, 2015, 12:33:42 AM
Quote from: dmilligan on May 03, 2015, 04:57:43 AM
I have updated MLVFS with the 70D matricies (they are the same as 6D)

Thanks again for the quick update for 70D color materices.

However, on my MBP it opens up web gui just fine and shows info.

But it the mount folder itself won't show up (even after spitting out the .IDX files)

(https://farm9.staticflickr.com/8806/16761752664_7c7f1ecc8c_n.jpg) (https://flic.kr/p/rxbn6A)

This was done on my 2006 MBP... any ideas?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on May 13, 2015, 10:21:03 AM
Hi dmilligan! Tried out your mlv_dump for archiving purposes. Very nice indeed. However. When extracting dng files both compressed and non compressed files it doesn,t add white and black level info? Could this be fixed? What I really like is that you rewrote the dng tiff tags completely for mlv to dng files.

Here is a file developed straight from an mlv file with your mlv_dump from first post.

https://drive.google.com/file/d/0B4tCJMlOYfirbXMxaVRCV0RiT1k/view?usp=sharing

*tried adding white/black level with exiftool. Doesn,t work. Is exiftool not working/writing to cdng files?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on May 19, 2015, 09:32:09 AM
*feedbacks*

Some few days ago, I decided to run your app with an old footage I shot from 3-4 months ago during a BTS shoot...

I enabled 20-bit Dual-ISO in web host config (although not all files were shot in this format) so is it problematic for me to have both normal and DI's files together in the mix? Do they need to be run seprately?

Also I noticed while rendering in AE CC 12.1, seems to be really slow (even for regular MLV's) is this normal?

It's literally almost like over an hour for each 13-15 second clips?

https://vimeo.com/128229218 (https://vimeo.com/128229218)

https://vimeo.com/128229544 (https://vimeo.com/128229544)

Thanks again, @dmilligan!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 19, 2015, 12:50:20 PM
Quote from: DeafEyeJedi on May 19, 2015, 09:32:09 AM
Also I noticed while rendering in AE CC 12.1, seems to be really slow (even for regular MLV's) is this normal?
There is no metadata in the MLV that indicates whether or not the file is dual ISO (IMO, there should be). So even on the non-dual ISO files we have to actually try to do a dual-ISO conversion and fail to figure it out, b/c we don't know if they are dual-ISO or not and there is no other way to figure it out.

I suppose there are some things I could do to speed things up a bit for the non-dual-ISO case, perhaps keep track of if frames in an MLV file fail to convert and don't try again once one or two frames in a file fail.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on May 19, 2015, 03:37:47 PM
Can you use an if condition? I use it when differing dual iso from non dual iso searching for the line "'Edge-directed interpolation..." when doing a test conversion. That way it will only try to (test)convert one file instead of testing all files for dual iso. Sending out the process of the first file to a log file with tee command.


find . -name ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.dng -print0 | xargs -0 /usr/bin/cr2hdr20bit | tee /tmp/magic_l/cr2hdr20bit_MLV.log ;
if grep -q 'Edge-directed interpolation...' tee /tmp/magic_l/cr2hdr20bit_MLV.log
then
echo yes
else
something else
fi


*
QuoteI suppose there are some things I could do to speed things up a bit for the non-dual-ISO case, perhaps keep track of if frames in an MLV file fail to convert and don't try again once one or two frames in a file fail.

You,re already on it I see :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ddelreal on May 19, 2015, 06:12:32 PM
Thanks so much for this, really like using it for my 6D files. Question though: with regular raw files using raw2dng (on Mac) ACR opens the converted files very flat looking and I can adjust accordingly or I can open in Resolve and set to BMD and do the same thing. Works pretty much the same in Resolve when using MLV converted to CDNG using MLVFS but if I use ACR, it looks like REC 709 already. Is that correct?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on May 20, 2015, 08:19:31 AM
Just got home from work and I find that MLVFS somehow froze up after spending the last few days rendering the shit outta my Mac Mini (all good the usual stuff) and apparently it thinks it ran out of diskspace but I looked up and see that I have 36 GB of free space.

Also Dual-ISO footage still comes out a bit darker than it normally would with other converters for some reason (I think we discussed this before) but not sure if it has been resolved or do i just need to boost up the exposure by a few stops just for these certain files only while using MLVFS?

Anyway, no big deal since I was't too pleased with the results of my footage. I'll have to work on some more with my color grading skills (if any) lol -- guess everything happens for a reason. Good thing I didn't waste a whole bunch of ProRes files for nothing.  :-X

Looking forward to your next update in regards to MLV/Dual-ISO files "edge-directed interpolation" system... it'll definitely be worth having this feature to speed up the workflow (even if its just a little bit) but perhaps not enough for you to redo all the codings?   :D

Thanks again, Dave!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on May 20, 2015, 06:27:37 PM
*updates*

Decided to try MLV spanning files again from a 5D3. Here's what I've encountered... One of the MLV's were recorded longer than 49 seconds (as stated in MLVFS web host) and when viewing playback the same original MLV's in MLRV (which playbacks fine the entire clip whereas one of the clip actually goes over 4 minutes long) but not after exporting from MLVFS. Strange?

https://vimeo.com/128391858 (https://vimeo.com/128391858)

Ah, so I'm not sure where to look into this so called spanning culprit? Do I need to rename the spanning files to make it recognizable? (Will have to try these spanning files again with another converter such as cr2hdr-r to see if problem still persist and if so then it may not be your app's issue) so I'll get back to you on that shortly.

Have you ran into similar situation or anyone else out there?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ddelreal on May 24, 2015, 11:03:29 PM
QuoteThanks so much for this, really like using it for my 6D files. Question though: with regular raw files using raw2dng (on Mac) ACR opens the converted files very flat looking and I can adjust accordingly or I can open in Resolve and set to BMD and do the same thing. Works pretty much the same in Resolve when using MLV converted to CDNG using MLVFS but if I use ACR, it looks like REC 709 already. Is that correct?

Sorry to quote my own post but was just wondering about my question...
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on May 25, 2015, 03:57:55 AM
MLVFS outputs RAW sensor data in the form of DNGs. How various software handles and interprets that data is up to the particular software. There may be slight differences in the metadata output between raw2dng and MLVFS, causing various softwares to interpret them slightly differently by default. However, the RAW data is the same, so once you make adjustments, you should see no difference.

I can't really comment on what ACR specifically is doing because I don't use it for video (and I recommend against using it for video), but I highly doubt it's outputting REC 709 by default as that is a video standard and ACR is intended mainly for photography and print and was never really intended for video.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on June 06, 2015, 10:33:51 PM
@dmilligan:

Any suggestions on how to prevent MLVFS from disappearing from desktop during rendering within AE? It has happened to me three times in the last 48 hours on this Mac Mini running Yosemite...

(https://c1.staticflickr.com/1/376/18537058731_3d29bf4d23_n.jpg) (https://flic.kr/p/uf4gJT) (https://c1.staticflickr.com/9/8846/17914435473_5df64a4375_n.jpg) (https://flic.kr/p/ti3ax8)

It's connected to an external drive via USB 3.0 -- tried this routine again with destination export on OS system instead & issue still persist. Is this normal when working with more than 50 files at once?

Thanks, David!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: goran on June 07, 2015, 01:59:34 PM
Just a quick question since I couldn't find an answer, how do I remove the MLVFS from services should I wish to do so?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 07, 2015, 02:15:14 PM
Username/Library/services

You might have to unhide your library folder via terminal.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: goran on June 07, 2015, 03:01:51 PM
Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DanHaag on June 08, 2015, 05:56:50 PM
Noticed MLV audio playing as it should in DaVinci Resolve when using your awesome MLVFS. Got some old CDNGs converted with RawMagic, their video and audio are badly out of sync (audio starting way after the video). Is there some metadata MLVS is adding that I might apply to my old files as well using ExifTool to fix this? I think I saw somebody writing about this a while ago but can't find those postings anymore.  :(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on June 08, 2015, 06:03:33 PM
DanHaag,

If you have cDNGs then I assume you got a WAV file for the audio also. Can you just trim the WAV so it is correct? I don't think metadata in the cDNGs (if any) is going to fix your problem.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DanHaag on June 08, 2015, 06:38:59 PM
I've done that for parts of the footage in FCPX. Can't do it directly in Resolve since it would not consider this kind of edit when outputting single clips i.e. as proxy files for editing (tried that as well, audio just goes back where it originally was after export). Manually moving the audio track in my NLE messes up my workflow since what I usually do is take the proxies from Resolve, sync them with external audio recordings in Plural Eyes and send this timeline to FCPX for editing without having to worry about that part of the process anymore. There's a whole lot of footage involved in that project I've started working on before MLVFS was around (wasn't able to keep the original MLVs, crazy amount of TB...) - Long story short: I can work around the problem but it's a huge effort and if there was any way to fix the problem with correcting metadata I'd be more than happy to know.  :) It just looks like DaVinci reads incorrect information on which timecode to put beginning of each audio track. Maybe there's no solution for this, other than fixing it the hard way.  :'(
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on June 08, 2015, 06:47:09 PM
It's more than just trimming the WAV file. There is special time code metadata that MLVFS puts in both the DNG and the WAV file. I don't think it would really be possible to add this with exiftool to the DNGs and there's no way at all, that I know of, to add it to the WAV file. You would have to write a program to do it. For more info see dng.c and wav.c in the MLVFS source, or have a look at raw2cdng code which also adds this metadata.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DanHaag on June 08, 2015, 06:56:36 PM
Thanks for your clarification.  :) If there was a way to fix the files, it's totally out of my reach.  :'( But now at least I can focus on working around the audio sync manually without worrying if there's an easier way for me to do it. After that project is done I'll stick with MLVFS for sure.  ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 08, 2015, 07:23:28 PM
Try bwf meta edit. I use it to add the black magic cam info from dmilligan wav info in cr2hdr-r.(thanks dmilligan)
I trim audio with sox and add meta time code info with exiftool. You can pm me a file with out of sync audio. Might be able to help.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: alexidoia on June 09, 2015, 03:21:04 PM
Hello,

I did not know where to post this, but I have discovered what I think is a bug from MLVFS and I wanted to report it.
I first used mlrawviewer to convert my mlvs to dngs but as I use the cinelog LUT and that they advise to use MLVFS because of its respect of metadata blablabla, I did another convertion from mlv to dng using mlvfs. And that point is important because I could have never found out about the bug if I had not kept the first export from mlrawviewer.

As I were editing, I found a shot of only one frame. I found that strange so I decided to compare that dng (produced by MLVFS) to the dng produced from mlrawviewer. I found out that the shot was actualy a minute long. I did another export from MLVFS just in case and it did produced the same one-shot DNG. A few hours laters similar problem with a shoot a few second long whereas I remembered having at least 2 minutes of ongoing action. I exported the DNG again from MLVFS and had the same shot with the same short length. I compared with the dng from mlrawviewer and found out that my clip was -as I thought it would be- 2minutes long.

My conclusion is that for some reason (that have nothing to do with the computer performance while exporting) MLVFS produced shorted DNGs. This looks clearly like a bug because if I repeat the export, the produce DNG is exactly the same, it is not a random stop.

I thought some people would know about it. This is very bad since, you really don't know if your shot is shorted or not.

Thanks.
Alex
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on June 09, 2015, 03:45:23 PM
So you have only one file in the mlvfs output folder when you navigate there? No GIF? No WAV? Screenshot of the folder contents?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: alexidoia on June 09, 2015, 06:22:43 PM
QuoteSo you have only one file in the mlvfs output folder when you navigate there? No GIF? No WAV? Screenshot of the folder contents?
heu... that was not for me right ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on June 09, 2015, 07:24:13 PM
Yes it is...
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on June 09, 2015, 07:51:41 PM
@alexidoia

Take a screenshot of the web interface for those shortened clips, upload an example MLV file somewhere that exhibits the problem or at least send me the output of mlv_dump -v
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on June 09, 2015, 07:56:54 PM
I've reported this somewhat similar issue (http://www.magiclantern.fm/forum/index.php?topic=13152.msg147987#msg147987issue) but no one got back to me about it as of yet...

@alexidoia:  was yours done with spanning files or no?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on June 09, 2015, 11:14:23 PM
For those using the Pismo File Mount version of MLVFS on Windows, I know that it hasn't been updated in eight(!) months, and it's far behind @dmilligan's Mac/Linux version.  I took a very long break from ML or MLVFS development, and I'm starting to get back into it.  I still have to figure out the best way to "catch up", not least of which is figuring out how to implement the equivalent to the webgui.  Let me know if there are specific features that should be the highest priority (e.g., FRSP support).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on June 10, 2015, 07:52:32 AM
ayshih: Super! I use it all the time despite lack of Web gui. I just mark drowes files in total commander select quick mount and it mounts all the files I selected at once.
I've been wondering if pismo can take advantage of existing idx files. It seems to take about the same time to mount with or without existing idx.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: alexidoia on June 12, 2015, 03:30:45 PM
I'm sorry,
I did not get back to you, as I have a movie to edit and a baaad deadline.
I don't have time to put the files online nor to test anyfurther, but basicaly,

I have a directory of .mlv
I right click on that directory to choose service>MVLFS et choose an empty directory. For 265 .mlv I have 265 .MLV directories which contains
1 _PREVIEW.GIF file
X .dng files
1 .log file
1 .wav file

That is to answer part of your previous questions.
The problem is that for one .mlv that last about a minute in mlrawviewer, I have only 1 .dng produced
and for some others I have the wrong amount of .dng files.

I did move the .mlv files that had not been converted correctly in a separate directory. and I mounted the directory via the services again. To my surprise, the file that used to output only one .dng had about the right amount of .dng this time. I am saying about because when comparing with the output of mlrawviewer, some .dng where still missing.

So I don't know. It changed something to isolate them into a different directory but it didn't really solve the problem.
I am not running mvlfs through the terminal so I am not really sure how to ouput the verbose version of it.

I'll get back to that when I have more time but right now I can't be diverted too much from my editing.
Alex
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on June 12, 2015, 08:53:51 PM
Seriously... I can't take this anymore... I REALLY cannot fathom with this any longer in allowing myself to use MLVFS again with BIG projects (regardless of how nice and awesome it is) it just totally won't render/export from AE for some reason (At first it says its rendering and I check the destination folder that's on the same external drive where I can see the file being processed) and leave my MacMini on all overnight (with screensaver and disk sleep enabled 'NEVER' within setting in OS X -- because I feel these can corrupt the rendering if it goes to sleep overnight):

(https://c1.staticflickr.com/1/495/18748713141_1ff533bc54_n.jpg) (https://flic.kr/p/uyL4dR) https://vimeo.com/130583139

It's driving me INSANE, this supposedly only happens if I use an external drive plugged in via USB 3.0 (big projects are usually over 160-200GB worth of MLV's) so I can't obviously use my Mac Mini's disk space which is limited to about 120gb of free disk space.

However, if I run small projects with only 10-20 MLV's then I can run MLVFS within my system on Mac Mini with export destination set on my desktop for faster rendering (Mac Mini has built in SSD) which works GREAT! I mean really fast and gets the job done well!

BUT for some reason if I use my external drives (which aren't SSD) seems to be running slower (which is to be expected) but then once I go to bed and wake up the next morning I find myself encountering with these nightmares on several occasions as previously reported:

(https://c1.staticflickr.com/1/508/18740826902_6cdb7f07a3_n.jpg) (https://flic.kr/p/uy4CVd) (https://c1.staticflickr.com/1/445/18748178761_1cd5116147_n.jpg) (https://flic.kr/p/uyHjnp) (https://c1.staticflickr.com/1/285/18123102844_869bb0e80f_n.jpg) (https://flic.kr/p/tBtD5q)

Also worth noting is the fact that it appears to me that I may need to purchase a SSD external drive with USB 3.0 in order to prevent this from happening again? What I mean is what can I do (if any) to prevent from having MLVFS mount files disappear in the middle of the night thus I believe is what causing these errors in AE during rendering?

I don't have this problem if I run everything solely on this Mac Mini (small projects) but if I want to work with big projects then I need to be able to use my external drives without hesitation of any kind. I thought I used to be able to do this with no problems in the past but as of lately is has been and I'm not sure why or what has changed?

The two things that I think could be the culprit for me is the fact that I have shot both in Non-Dual ISO MLV as well as in Normal MLV and there are all mixed up in order -- so is it not a good idea to have CS2x2 & Dual-ISO (Full 20-bit) both enabled on in webGUI?

I don't mind the slow process but am I allowed to do it this way?

Or do I need to separate them like batch all the Dual-ISO MLV's into one Folder and Regular MLV into another Folder?

Forcing myself to run TWO separate Projects to possibility prevent from this so called culprit occurring again that I've been encountering as of lately - hmm?

Or would it be ideal to try and run two external drives (one 3.0 and one Firewire 800) with one being used to provide BIG PROJECTS and one being used for the destination point for ProRes exports?

Sorry to bring this to you, David but I feel that I've tried all possibilities other than considering myself to purchase a SSD external drive w thunderbolt support. Thoughts on that?

Thanks for reading this.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 14, 2015, 02:52:24 AM
Issues like this are going to be very difficult for me to duplicate and fix, I would suggest trying to narrow it down as much as you can and figure out how to reliably duplicate the issue. I just don't have time to try and replicate issues involving unpredictability and vast amounts data/processing time.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on June 14, 2015, 05:41:30 AM
Okay no problem! I'll see what I can do over the weekend and will do the best I can.

FYI - I managed to get MLVFS to finally work again on my 2006 MBP by updating the Fuse to the latest and all is fine and dandy -- so please disregard this particular issue!

Thanks for your time, David!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kgv5 on June 18, 2015, 05:23:53 PM
I anyone of windows users who recently installed PFMap from the link on the first page have problems with MLVFS and importing files to premiere - pay attention on PFMap build - now is 177, i had some serious problems with it (couldnt import to premiere) and figured out that older build 171 is the one to go. Seems that 177 has some changes and because of that it wasn't working for me.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on June 19, 2015, 05:43:26 AM
Good to know!  I still haven't been able to put back real time into MLVFS again, but it looks like there have been quite a few recent changes to Pismo File Mount.  Here is the link to build 171:

http://www.pismotechnic.com/download/archive/pfmap-171-win.exe
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 19, 2015, 12:54:11 PM
updated the download link in the OP with that one
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on June 21, 2015, 11:22:01 AM
Hi, Dmilligan, I'm a mlv_dump long time user. Some feedback on my attempt to try MLVFS under windows with pismo(1.7.1) driver with ACR and AE.
1. It works, fast and flawlessly.
2. WB temp "as shot" always 2550, mlv_dump produces dng sequence with correct WB from camera.
3. I find no way to use command line options.
4. DLL file seems not updated from october 2014. Both mlv_dump and cr2hdr has updates/bug fixes since then.
5. Something wrong with MLVFS CDNG files format itself - Capture One, for example, didn't recognize it. DNG from mlv_dump and CDNG from raw2cdng are OK, but CDNG from MLVFS - not.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 21, 2015, 11:50:04 AM
Curious.
Are you sure mlv_dump is translating to correct wb to as shot neutral values? The mac binary doesn, t seem to at least from what I can see.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on June 21, 2015, 01:04:34 PM
Yes, mlv_dump generates DNGs which looks absolutely the same as CR2 if I shot the same scene with same camera settings. Actually, wrong WB is not a big problem.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 21, 2015, 01:32:15 PM
Oh, gotta check for mac. Actually I personally prefer to nail wb before actually shooting. Saves a lot time.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ayshih on June 21, 2015, 04:07:15 PM
@mothaibaphoto I wrote the PFM version of MLVFS, and as you've already noted, it is out of date because I took a long personal break from development.  I hope to fix it up with @dmilligan's improvements, but it will take some time.

Quote from: mothaibaphoto on June 21, 2015, 11:22:01 AM
2. WB temp "as shot" always 2550, mlv_dump produces dng sequence with correct WB from camera.
Good to know.  What camera?  And are you shooting normal images or dual-ISO images?

Quote from: mothaibaphoto on June 21, 2015, 11:22:01 AM
3. I find no way to use command line options.
Given how the PFM version works – mounting a specific MLV file in Explorer, typically via the right-click menu – it's non-obvious what the best approach is to feed in options.  I'm open to suggestions!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on June 21, 2015, 06:58:28 PM
Hi, ayshih!!! It's so great that you have plans to update MLVFS on windows! It seems MLVFS becomes very popular among Mac and Linux users, and its simplicity tempting us, windows users.

Quote from: ayshih on June 21, 2015, 04:07:15 PM
Good to know.  What camera?  And are you shooting normal images or dual-ISO images?
5D MarkIII, normal video
Quote from: ayshih on June 21, 2015, 04:07:15 PM
it's non-obvious what the best approach is to feed in options.  I'm open to suggestions!
It could be some predefined ini-file in folder with *.MLV files. When any MLV file mounts, DLL looks for parameters in this file in same folder. Ideally, this file could contain path to *.Mxx files. This makes possible not to copy files from cards to one folder first when shoot in spanned mode.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on June 21, 2015, 10:43:06 PM
mlv_dump does not write correct WB in DNG files. It uses a hard coded AsShotNeutral for Daylight (5500). There is a PR of mine to replace the old CHDK DNG writing code in mlv_dump with the DNG code I wrote for MLVFS that does handle WB correctly (with some kelvin to RGB multipliers code from UFRAW). Are you using that version or something? Otherwise I don't see how you're getting correct WB from mlv_dump unless you always shoot in daylight and never noticed it was wrong for other lighting situations.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on June 22, 2015, 06:14:10 AM
Sorry, dmilligan and everyone confused with my wrong sentences about mlv_dump. It really use 5500 K Temp. I didn't notice it, i prefer to shoot in good light indeed :) Just checked again with file shot at 4500 K in camera:
MLRawViewer 1.4.3 - 5500 K
mlv_dump (last one, by the way, how to check its version?) - 5500 K
raw2cdng.1.6.5 - 4500 K (wow, the winner !!!)
mlvfs.dll 0.4.0.0. - 2500 K :(
Maybe the obsolete dll is the only problem?
And, please, pay attention: MLVFS output file - the only not recognised by Capture One.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: morrisonmike on June 25, 2015, 02:01:04 AM
Sorry, I'm new to Linux. Can you give me the command line entries for this?

"Install FUSE via the normal way for your distribution (it may already be included)
Download the code with git and compile using make"

I get the following error:

mike@GP532UC-ABA:~/mlvfs/mlvfs$ make
gcc -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 main.c dng.o index.o wav.o stripes.o cs.o amaze_demosaic_RT.o hdr.o histogram.o mongoose/mongoose.o webgui.o resource_manager.o lj92.o gif.o LZMA/7zAlloc.o LZMA/7zBuf.o LZMA/7zBuf2.o LZMA/7zCrc.o LZMA/7zCrcOpt.o LZMA/7zDec.o LZMA/7zFile.o LZMA/7zIn.o LZMA/7zStream.o LZMA/Alloc.o LZMA/Bcj2.o LZMA/Bra.o LZMA/Bra86.o LZMA/BraIA64.o LZMA/CpuArch.o LZMA/Delta.o LZMA/LzFind.o LZMA/Lzma2Dec.o LZMA/Lzma2Enc.o LZMA/Lzma86Dec.o LZMA/Lzma86Enc.o LZMA/LzmaDec.o LZMA/LzmaEnc.o LZMA/LzmaLib.o LZMA/Ppmd7.o LZMA/Ppmd7Dec.o LZMA/Ppmd7Enc.o LZMA/Sha256.o LZMA/Xz.o LZMA/XzCrc64.o -pthread -lfuse -lm -o mlvfs
main.c:33:18: fatal error: fuse.h: No such file or directory
#include <fuse.h>
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on June 25, 2015, 02:10:33 AM
Use your distro packaging tools to find what package provides fuse.h, some thing like libfuse-dev  on Debian.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: morrisonmike on June 25, 2015, 02:30:31 AM
Thanks, but "Use your distro packaging tools to find what package provides fuse.h, some thing like libfuse-dev  on Debian" doesn't mean anything to me... Isn't there just some commands I can cut-n-paste into the command prompt?
-MM
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on June 25, 2015, 02:34:25 AM
No, because without telling which distribution you are using we can't tell.
Google
install fuse <insert name of your Linux distribution>
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 25, 2015, 03:53:39 AM
assuming you are using a debian flavored distribution (like ubuntu) something like

sudo apt-get install libfuse-dev

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: morrisonmike on June 25, 2015, 09:25:04 PM
Thanks everyone. I'm up and running now.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on June 27, 2015, 08:19:42 PM
Just tried running MLVFS this morning on a test run and it randomly disappeared from my services list on MBP running Yosemite.

I should note that this happened after I tried to create a shortcut button for MLVFS -- could this be the culprit? (though I turned it off and it still doesn't show up on the services list after right click) and I also notice now it just opens Automator every time I want to reinstall MLVFS -- I must have pushed some buttons somewhere not realizing it within Automator?

https://vimeo.com/131978677

How do I fix this and make it install and overwrite it as well as running MLVFS like normal again?

However, it still shows under settings preferences. See attachments below.

(https://c4.staticflickr.com/4/3751/19022560018_b3b0756d9f_n.jpg) (https://flic.kr/p/uYXAmJ) (https://c1.staticflickr.com/1/406/19022817280_3641ae708f_n.jpg) (https://flic.kr/p/uYYUQh)

Anyone else out see a similar situation? I've also tried uninstalling/reinstalling both MLVFS and FUSE to no avail.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 27, 2015, 10:13:24 PM
You have to right click on a folder. You can only mount folders, not individual files.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on June 28, 2015, 05:18:39 AM
Actually I apologize for the false alarm this morning. Must have been either really tired or a bit hangover from being in the post world for quite a bit!

FYI regarding previous post (http://www.magiclantern.fm/forum/index.php?topic=13152.msg149498#msg149498post) that I've noticed MLVFS hasn't disappeared or any problems when rendering overnight as of lately is most likely due to the fact that I had updated to the latest FUSE.

Also is it a good idea to use their beta version of FUSE or best to stick with non-beta's?

Thanks again, @dmilligan!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: benyonker on July 30, 2015, 04:55:26 PM
I just found this a few days ago, and I've been using it re-process some MLVs that were full of bad pixels. The "bad pixel fix" feature alone is going to save me hours of post on this music video, so thank you!! I have two questions that I haven't been able to find in the forums:


Neither of these is a big deal; I'm obviously color-correcting anyway, and the dates aren't critical. I'm just curious.

Thanks again!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on July 30, 2015, 05:59:23 PM
1. A DNG is a DNG, the raw data is going to be the same between the two. "Darker" is just a result of how the raw development application (ACR, Pemiere, DaVinci, etc.) chooses to treat the file (you can't actually 'see' true raw data or levels, it must be processed, demosaiced, white balanced, and converted to some real colorspace to display on your monitor for you to even view it, there are going to be arbitrarily chosen default values for parameters used in these processes).

Simply increase the exposure in your post processing app and they will look exactly the same. The difference is likely due to some different metatdata in the DNG that causes the raw development program to interpret the raw data differently by default. There is a DNG metadata field for 'exposure offset' or 'exposure' or something like that. There have been a lot of people who complain about the DNGs not matching the camera's LCD screen and 'appearing darker'  (this complaint is sort of silly for the reason given above), my guess is that RAWMagic developer utilized this field to make the DNGs appear brighter to match the LCD screen (to appease his paying customers). If you'd like to find out for sure, try comparing the metadata between the two with exiftool.

2. The struct tm has a field tm_mon, on some systems this is [0,11] on others it is [1,12]. That is probably the issue.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: benyonker on July 30, 2015, 11:48:26 PM
Thank you for the quick reply. Your answer to #1 makes perfect sense. I agree that the complaint is sort of silly given your explanation. As for #2, I don't even know what to make of that or how I can alter the tm_mon field. I'm on a Mac. Is this something I would have to do in Xcode? Terminal?

One final question: I shot this footage at 29.97, but I want it to play back at 23.98. In the MLVFS window, I see an option to override the framerate. When I change it to 23.98 (or 23.976), PPro interprets the footage to be 23.000fps. If I use 24 in MLVFS, PPro interprets it at 24. That could work, but I'd still rather have it be 23.98. Am I better off leaving it at 0 and doing Interpret Footage in PPro? Will that cause issues in Resolve?

Thanks so much, and again I apologize if these are basic questions.

**Correction to above: Resolve sees the footage as 23.98. When I render proxies, PPro also sees them as 23.98. Looks like we're OK here; just a glitch with the way PPro sees the original DNGs.**
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on July 31, 2015, 01:15:16 AM
For the month stuff, I will have to look at the code and fix it
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Slasheal on August 01, 2015, 10:19:50 PM
Some info:
I tried to compare MLV MYSTIC / MIRAWVIEWVER / MLVFS

Size:
MLV MYSTIC : 3mo
MIRAWVIEWVER : 1.8mo
MLVFS : 3.4mo


White balance:
MLV MYSTIC : 5100K (even if manual WB)
MIRAWVIEWVER : 5000K (even if manual WB)
MLVFS: in AWB mode, it will take the value stored in manual WB.
If you set to a WB preset (ex: tungsten light), it will take the WB preset.


In short: DO NOT USE AWB with MLVFS.
Still, it's a good thing to be able to change WB before grading.
Most grading softwares don't have "photo" WB control. I don't want to waste my time on camera raw.

So yep, actually for me, MLVFS is THE best solution on OSX to manage raw video :).
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on August 02, 2015, 02:45:22 AM
I'm not 100% sure, but this may actually be an ML issue (AWB is not captured correctly in camera to the MLV format). Need to check. I'm pretty sure MLVFS handles WB as correctly as possible, at least according to the MLV format specification. It was a while ago that I implemented all the WB stuff, so I'm a bit fuzzy.

You can check yourself by using mlv_dump -v to see all the metadata from the MLV file.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on August 02, 2015, 08:09:01 AM
Some new observations regarding WB and different convertors. I shot Dual ISO silent timelapse in MLV yesterday. WB in camera was set to AWB. I usually process this kind of files with mlv_dump+cr2hdr routinely. This time I decided to try different converters while keeping eye on WB. First surprise - mlv_dump+cr2hdr produces DNG with correct WB - 4100 "As shot" in ACR. Second - I can't try anything else - raw2cdng.1.6.5 just crashes, mlvfs.dll mounts empty folders and crashes when I unmount such empty folder.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 02, 2015, 09:55:13 AM
Do you mean Mlvfs+cr2hdr? Mlv_dump hardcode wb settings.
Mlvfs delivers correct wb for me.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on August 02, 2015, 10:23:21 AM
Quote from: Danne on August 02, 2015, 09:55:13 AM
Do you mean Mlvfs+cr2hdr?
I mean Mlvfs doesn't work at all with this kind of files, at least on Windows.

Quote from: Danne on August 02, 2015, 09:55:13 AM
Mlv_dump hardcode wb settings.
I know it. But this setup wonderfully produces correct WB :)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on August 02, 2015, 07:14:40 PM
The Windows version is way behind the Mac/Linux version and does not handle WB correctly. I believe cr2hdr recomputes the WB, b/c the processing it does has an effect on WB (I could be wrong on this point).

I would not consider hardcoded WB that just so happens to be the same as the scene you shot "correct". More like "lucky".
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Marsu42 on August 04, 2015, 12:04:24 AM
Quote from: dmilligan on August 02, 2015, 07:14:40 PM
The Windows version is way behind the Mac/Linux version and does not handle WB correctly.

Doh - any chance to get the win version updated as the webdav solution isn't available anymore (and less straight forward than mounting) ... or are the code bases so different the win version will lag behind?

Quote from: dmilligan on August 02, 2015, 07:14:40 PMI believe cr2hdr recomputes the WB, b/c the processing it does has an effect on WB (I could be wrong on this point).

I dunno if it's supposed to recompute wb, but it definitely has an effect on wb at least with my cams (60d, 6d) so with the adobe raw converter I usually have to do some manual tint/temperature adjustments.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on August 04, 2015, 06:20:12 AM
Maybe cr2hdr will produce more accurate result dealing with input DNG files with actual, not "hardcoded" WB?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 04, 2015, 01:55:03 PM
Quote from: Marsu42 on August 04, 2015, 12:04:24 AM
Doh - any chance to get the win version updated as the webdav solution isn't available anymore (and less straight forward than mounting) ... or are the code bases so different the win version will lag behind?
Main issue is I don't own a Windows box.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: sggvideo on August 06, 2015, 10:09:51 AM
Hi. I was very excited to discover and use MLVFS with my mac, and it has been a charm!!

I recently noticed when I try to mount a folder with my entire 63.98gb of .MLV data I sometimes lose frames in the .DNG's. In the browser window that pops up it also says there are less frames than I recorded. BUT, If I put one .MLV file (and all of its parts .M01, .M02..) into a folder alone and run MLVFS it gives me all the frames!

Anyone else had this happen?

If it's been answered I apologize, I read some of the discussion then skipped to the end  :o
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Slasheal on August 08, 2015, 06:28:41 PM
Another problem I have, I will test later if it's related to MLVFS or not:


The DNGs created with MLVFS, when opened with ACR, doesn't allow me tu properly use lens correction:

https://forums.adobe.com/message/7823545#7823545
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 09, 2015, 02:29:32 AM
I replied to the Adobe thread. Basically, it's going to be a challenge figuring out what the metadata needs to be for ACR to interpret the FOV correctly, so it can apply the lens corrections appropriately. Feel free to grab exiftool and the DNG spec (http://www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf) and start fiddling with the metadata. ACR is a bit of a black box, so figuring this stuff out requires some trial and error (something, I don't really have time to do myself).

When I was looking into this, I stumbled upon a DNG tag I had apparently previously overlooked, it's called DefaultScale. Basically it's pixel aspect ratio (the name makes no sense, which is why I had never found it before). Filling in the correct value here should allow us to re-scale the squeezed 50/60fps automatically! ACR seems to respect this value, I haven't tried others (Premiere, Resolve, etc.). I'll need some sample MLV files of squeezed footage from various cameras for testing, so please upload it (post it here: http://www.magiclantern.fm/forum/index.php?topic=11899.0).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 09, 2015, 10:28:54 PM
Updates:

I have implemented the automatic de-squeezing for 50/60 fps (and EOSM) 720p footage. Please test (and make sure also that it doesn't try to de-squeeze any sort of normal footage). Also, I have only tested ACR (which works correctly), other programs may or may not ignore this tag, so also let us know what programs work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 10, 2015, 12:25:54 AM
Gonna test this out tomorrow with my 5d mark III. Tried messing with exiftool yesterday but couldn, t get the tags in there. Thanks for sharing dng spec and really good find I must say.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on August 10, 2015, 02:42:43 AM
Very nice update, looks like it works correctly with squashed 7d footage. This seems to give us better scaling quality than what AE includes, presumably since it's the raw data that's being scaled. Do you have any insight on this, dmilligan? AE's detail-preserving upscale works well but only allows for the footage to be scaled to match the comp width or height, making this a great time-saver as well.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on August 10, 2015, 03:00:49 AM
It makes sense the quality might be slightly better since it allows the ACR debayer algorithm to be aware of the "non-square" pixels, and so it can take that into account when interpolating, resulting in slightly better interpolation.

Do you mind posting some 100% crop comparisons between the two methods for the pixel-peepers?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 10, 2015, 01:35:53 PM
Did a test with three mlv clips filmed in 48, 50 and 60 fps with my 5d mark III. ACR interpretes the default scale value accordingly but couldn,t get the reading in Davinci resolve nor Premiere pro CC 2014.
Nice to use in an ACR workflow. Good one.

ACR
(http://s15.postimg.org/ll46ql11z/Screen_Shot_2015_08_10_at_13_15_55.png) (http://postimg.org/image/ll46ql11z/)

Davinci resolve
(http://s15.postimg.org/4yroemefr/Screen_Shot_2015_08_10_at_13_22_17.png) (http://postimg.org/image/4yroemefr/)

Premiere pro CC
(http://s15.postimg.org/rji00tk0n/Screen_Shot_2015_08_10_at_13_31_46.png) (http://postimg.org/image/rji00tk0n/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andy600 on August 10, 2015, 02:44:45 PM
Premier and Resolve likely ignore the tag as it's not in CinemaDNG specs.

In Premier you can add custom pixel aspect ratios by editing a text file called 'Interpretation Rules.txt' (found in the Premier/Plugins directory - likely in a region specific folder i.e. en_US etc)

The aspect ratios must be integer values so if the unscaled image requires vertical scaling of say 1.67 you would scale by 100 and enter 100/167 in the text file - I haven't tried this BTW so you will have to experiment with the numbers.

Resolve is more limited as it has preset pixel aspect ratios so you would scale the frame size instead.

I'm not sure if either of the options above have any direct affect on the debayer algorithm itself but set Resolve's debayer to highest quality (and I suggest using the 'smoother' debayer option for these type of clips).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 10, 2015, 02:55:55 PM
Premiere state you can edit interpretation rules in the txt file. Me and several others have tried with no sucess. In Davinci resolve it,s very easy to unstretch the dng files. I think it,s "stretch footage to corners" or something like that.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andy600 on August 10, 2015, 03:11:02 PM
re: Resolve - that will only work correctly if the unstretched pixel aspect ratio correctly matches the output frame dimensions (i.e. 16:9 etc even if the actual stretched image is letterboxed) - a lot of users shoot odd frame sizes. It's better to choose 'Scale entire image to fit" in the main settings then scale the vertical dimension using the scale/framing controls in the color panel.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 10, 2015, 03:14:34 PM
Scale image to fit. That was what I meant :). Too bad however that the default scale tag isn, t read in cdng spec though...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 10, 2015, 03:52:23 PM
Quote from: Andy600 on August 10, 2015, 02:44:45 PM
Premier and Resolve likely ignore the tag as it's not in CinemaDNG specs.
Well technically, it is in the CinemaDNG spec, since the cDNG spec is basically: reference to the DNG spec + some extra tags + file/folder naming convention. The real reason it's ignored is b/c these programs don't really try to conform to the spec, they simply implement it for the specific 'flavors' of cDNG produced by specific cameras, and there probably aren't any cameras they support with non-square pixels (though if using anamorphic lenses, any camera can have non-square pixels). But it's not like anybody even fully supports the DNG spec either (and Adobe, who wrote the spec, is often the worst offender).

Quote from: Andy600 on August 10, 2015, 02:44:45 PM
The aspect ratios must be integer values so if the unscaled image requires vertical scaling of say 1.67 you would scale by 100 and enter 100/167 in the text file - I haven't tried this BTW so you will have to experiment with the numbers.
The exact aspect ratio is 3:5, so wouldn't it be better to use 3/5?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: N/A on August 10, 2015, 04:47:34 PM
Quote from: dmilligan on August 10, 2015, 03:00:49 AM
Do you mind posting some 100% crop comparisons between the two methods for the pixel-peepers?
Sure, I have a couple good examples. It may take a while though, my internet connection is pretty spotty right now.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andy600 on August 10, 2015, 04:53:24 PM
Yes, it's all down to what the devs choose to support beyond the mandatory tags.

I totally agree about Adobe. Their logic is more understandable when dealing with raw stills/print etc via ACR, Lightroom, Photoshop but I don't understand why they restrict Premier CDNG debayering to a display referred colorspace. It takes away some fundamental control of the raw image. Hopefully Resolve 12 has shown them how better to deal with colorspace management. I hope they move to something similar and also drop their reliance on ICC profiles - but I doubt it.

re: Pixel aspect ratio - of course, if that's the exact ratio it should be used.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on August 11, 2015, 12:24:00 AM
So far, I have been able to confirm with slo-mo files to be working fine with the latest updated MLVFS for both EOSM and 70D... it scales up nicely with ACR (SmartImport2) workflow in AE.

(https://farm1.staticflickr.com/522/20307155489_0dc320f90f_n.jpg) (https://flic.kr/p/wWttQv) (https://farm1.staticflickr.com/549/20485042732_ea030b4a37_n.jpg) (https://flic.kr/p/xdccvW) (https://farm1.staticflickr.com/325/20305784328_0c19aa479e_n.jpg) (https://flic.kr/p/wWmseN) (https://farm1.staticflickr.com/332/20307156969_792a71e2f4_n.jpg) (https://flic.kr/p/wWtuh2)

https://vimeo.com/136005690

I can't speak for DR but can confirm that PP does NOT read the tags for proper scaling.

Up next is the 5D3 slo-mo files.

Thanks, @dmilligan for the kind update!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Slasheal on August 15, 2015, 04:27:50 PM
Hello.

MIRAWviewver allow lens correction
MLVFS doesn't yet
MLVMystic doesn't yet too

Maybe you should ask for help.
You've got and answer on adobe forum:

Quote"Camera Raw / Lightroom needs to know the sensor area (or crop factor) to map the pixels of the source image to the physical model of the lens profile. ACR calculates this from the EXIF (not DNG) metadata. You can do this in one of two ways:

1. Use the FocalPlaneXResolution, FocalPlaneYResolution, and FocalPlaneResolutionUnit tags.

2. Use the FocalLength and FocalLengthIn35mm tags.

Of these two options, I recommend #1 because you'll have more precision. (For #2, FocalLength has tag type rational, which is fine, but FocalLengthIn35mm is an integer. That's an issue for short focal lengths.)"

Hope it will help.
If any more samples needed, don't hesitate to specify.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on August 15, 2015, 05:07:05 PM
I know, I saw that, it's the information I needed. When I have time I will work on it, it's a lot of work (figuring out what the focal plane resolution actually is, is not trivial, there are 3 different video resolution modes, and many supported ML cameras that all have different pixel pitches). Anybody is welcome to contribute code at any time.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: gmfotografie on August 15, 2015, 11:19:30 PM
after convertingi get the dng files in the specified folder.
my browser then opens and gives me a error message ... localhost:8000 -> could not opend.

using osx system
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cmccullum on August 17, 2015, 03:49:28 AM
I can't get this to work at all  :-\ @deafeyejedi has been trying to help me troubleshoot the problem, but still no luck. I keep getting safari can't connect to server "localhost". error

As far as I can tell, I've followed all of the correct steps, but I'm afraid I'm missing something

Running
OSX 10.7.5
OSXFuse 2.8.0

Any help would be great
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on August 17, 2015, 04:38:32 AM
Quote from: cmccullum on August 17, 2015, 03:49:28 AM
I keep getting safari can't connect to server "localhost". error
As far as I can tell, I've followed all of the correct steps,
but I'm afraid I'm missing something
Running
OSX 10.7.5
OSXFuse 2.8.0
Any help would be great
Not to sure about mlvfs running on "Lion" OSX 10.7.5 , I know it runs on Maverick(10.9.x) & Yosemite (10.10.x)
You may have to up-grade your OSX.

Even thou you don't have a web interface working  do get a mounted mlvfs folder ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cmccullum on August 17, 2015, 07:06:43 AM
@reddeercity nope, I don't get a mounted folder either. Anyone else think I might need to upgrade? I'd like to avoid this if possible
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on August 17, 2015, 07:20:05 AM
Another thing I just thought of , I'm using a older version(2.7.3) of osxfuse and a older build of MLVFS back from Nov/2014
Maybe try a older version of OSXFuse & or see if there is older build of mlvfs still available .

If I have some time in the next few days I have a old MacPro 1.1 that's on "Lion" OSX10.7.5 I see if I can get it working on it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cmccullum on August 17, 2015, 07:37:34 AM
If you could check that out, that'd be great. I tried older versions of OSXFuse, but didn't have any luck. I didn't think an older mlvfs would make any difference since it's a set of Automator actions. I'm thinking there's something I'm missing with OSXFuse or some deeper issue
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 17, 2015, 02:55:14 PM
MLVFS is not set of automator actions. The automator actions are there just make it easier for users who don't know much about command line to run it. There is an executable named 'mlvfs' embedded inside the automator package. That is what does all the work. The automator stuff just fills in the correct command line parameters and calls mlvfs for you, and then opens up the web browser to the correct address for the webgui.

If you want to figure out what is going on, you need to run that executable from the command line and see the output (the automator actions just throw away the standard output). If MLVFS is crashing, there should also be logs in ~/Logs/DiagnosticReports.

I would not recommend using an older version of MLVFS, it's not going to make a difference, other than probably being less stable and having fewer features (also, please do not share older builds here, I delete them for a reason). I would highly recommend upgrading to the latest version of the OS if you can; I'm definitely not going to be able to support old OS versions. If you really want to use an old OS version, you may have to compile MLVFS yourself.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cmccullum on August 17, 2015, 11:00:08 PM
@dmilligan sorry about that. I was half asleep when I wrote that response and had forgotten that I did open the Automator package and find the executable [facepalm] I don't know how to run it from the command line though :\
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on August 17, 2015, 11:58:56 PM
Run it with no parameters and it should tell you what the options are.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cmccullum on August 18, 2015, 04:42:34 AM
Not sure I'm doing it right, but I get:

Illegal instruction: 4
logout

[Process completed]
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 18, 2015, 12:42:02 PM
http://stackoverflow.com/questions/14268887/what-is-the-illegal-instruction-4-error-and-why-does-mmacosx-version-min-10

You will have to compile MLVFS yourself or update to a more recent version of OSX.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on August 19, 2015, 01:58:42 AM
Update to at least OS X 10.8.x and upward...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cmccullum on August 20, 2015, 03:54:10 AM
Updated my system, and everything is working now. Thanks everyone for your patience and help!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 23, 2015, 02:10:33 AM
RE: automatic lens corrections

I have implemented (https://bitbucket.org/dmilligan/mlvfs/commits/f026ffedf411ce5613daa694808d6fe913ed30e6) the tags necessary for this (FocalPlaneResolutionX, FocalPlaneResolutionY, FocalPlaneResolutionUnit), but only for my camera (60D).
I need the default values for these for all other cameras.

Here's how to get them:
1. Download and install exiftool (http://www.sno.phy.queensu.ca/~phil/exiftool/)
2. Take a normal raw still with your camera (CR2)
3. Run this command:

exiftool -v <image_filename.CR2>

(where <image_filename.CR2> is the actual filename of your CR2)
4. In the output look for the FocalPlaneResolutionX, FocalPlaneResolutionY, FocalPlaneResolutionUnit tags. It should look something like this:

  | | 18) FocalPlaneXResolution = 5728.176796 (5184000/905)
  | | 19) FocalPlaneYResolution = 5808.403361 (3456000/595)
  | | 20) FocalPlaneResolutionUnit = 2


I need the two ratios (ex: 5184000/905, 3456000/595) and the unit.
Once I get these values for all the cameras, I will update the code and release a new build.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on August 23, 2015, 02:46:37 AM
5D3:
  | | 26) FocalPlaneXResolution = 3942.505133 (5760000/1461)
  | | 27) FocalPlaneYResolution = 3950.617284 (3840000/972)
  | | 28) FocalPlaneResolutionUnit = 2


70D:
  | | 26) FocalPlaneXResolution = 6086.76307 (5472000/899)
  | | 27) FocalPlaneYResolution = 6090.15025 (3648000/599)
  | | 28) FocalPlaneResolutionUnit = 2


EOSM:
  | | 26) FocalPlaneXResolution = 5798.657718 (5184000/894)
  | | 27) FocalPlaneYResolution = 5788.944724 (3456000/597)
  | | 28) FocalPlaneResolutionUnit = 2
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Slasheal on August 26, 2015, 01:04:24 AM
7D:

Quote| | 26) FocalPlaneXResolution = 5715.545755 (5184000/907)
  | | 27) FocalPlaneYResolution = 5808.403361 (3456000/595)
  | | 28) FocalPlaneResolutionUnit = 2


Great!

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Mehmet Kozal on August 26, 2015, 07:23:12 AM
650D:

  | | 26) FocalPlaneXResolution = 5798.657718 (5184000/894)
  | | 27) FocalPlaneYResolution = 5788.944724 (3456000/597)
  | | 28) FocalPlaneResolutionUnit = 2
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on August 26, 2015, 08:28:19 AM
6D :

| | 26) FocalPlaneXResolution = 3810.584958 (5472000/1436)
| | 27) FocalPlaneYResolution = 3815.899582 (3648000/956)
| | 28) FocalPlaneResolutionUnit = 2


Thank you !
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 26, 2015, 09:05:40 AM
Should be for the rest of the bunch. Otherwise please tell.
*updated with a complete list of cameras.


5D mark 2

| | 24) FocalPlaneXResolution = 3849.211789 (5616000/1459)
  | | 25) FocalPlaneYResolution = 3908.141962 (3744000/958)
  | | 26) FocalPlaneResolutionUnit = 2


50D

| | 24) FocalPlaneXResolution = 5315.436242 (4752000/894)
  | | 25) FocalPlaneYResolution = 5306.532663 (3168000/597)
  | | 26) FocalPlaneResolutionUnit = 2


500D

| | 24) FocalPlaneXResolution = 5315.436242 (4752000/894)
  | | 25) FocalPlaneYResolution = 5342.32715 (3168000/593)
  | | 26) FocalPlaneResolutionUnit = 2


550D

| | 24) FocalPlaneXResolution = 5728.176796 (5184000/905)
  | | 25) FocalPlaneYResolution = 5808.403361 (3456000/595)
  | | 26) FocalPlaneResolutionUnit = 2


600D

| | 26) FocalPlaneXResolution = 5728.176796 (5184000/905)
  | | 27) FocalPlaneYResolution = 5808.403361 (3456000/595)
  | | 28) FocalPlaneResolutionUnit = 2


650D

| | 26) FocalPlaneXResolution = 5798.657718 (5184000/894)
  | | 27) FocalPlaneYResolution = 5788.944724 (3456000/597)
  | | 28) FocalPlaneResolutionUnit = 2


700D

| | 26) FocalPlaneXResolution = 5798.657718 (5184000/894)
  | | 27) FocalPlaneYResolution = 5788.944724 (3456000/597)
  | | 28) FocalPlaneResolutionUnit = 2


1100D

| | 26) FocalPlaneXResolution = 4720.441989 (4272000/905)
  | | 27) FocalPlaneYResolution = 4786.554622 (2848000/595)
  | | 28) FocalPlaneResolutionUnit = 2


5d mark 3

| | 26) FocalPlaneXResolution = 3942.505133 (5760000/1461)
  | | 27) FocalPlaneYResolution = 3950.617284 (3840000/972)
  | | 28) FocalPlaneResolutionUnit = 2


6D

| | 26) FocalPlaneXResolution = 3810.584958 (5472000/1436)
  | | 27) FocalPlaneYResolution = 3815.899582 (3648000/956)
  | | 28) FocalPlaneResolutionUnit = 2


7D

| | 24) FocalPlaneXResolution = 5715.545755 (5184000/907)
  | | 25) FocalPlaneYResolution = 5808.403361 (3456000/595)
  | | 26) FocalPlaneResolutionUnit = 2


60D

| | 26) FocalPlaneXResolution = 5728.176796 (5184000/905)
  | | 27) FocalPlaneYResolution = 5808.403361 (3456000/595)
  | | 28) FocalPlaneResolutionUnit = 2


70D

| | 26) FocalPlaneXResolution = 6086.76307 (5472000/899)
  | | 27) FocalPlaneYResolution = 6090.15025 (3648000/599)
  | | 28) FocalPlaneResolutionUnit = 2


eos m

| | 26) FocalPlaneXResolution = 5798.657718 (5184000/894)
  | | 27) FocalPlaneYResolution = 5788.944724 (3456000/597)
  | | 28) FocalPlaneResolutionUnit = 2
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: vertigopix on August 26, 2015, 09:06:23 AM
5D2 :

| | 24) FocalPlaneXResolution = 3849.211789 (5616000/1459)
| | 25) FocalPlaneYResolution = 3908.141962 (3744000/958)
| | 26) FocalPlaneResolutionUnit = 2
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 26, 2015, 10:55:53 PM
Thanks everyone. New build is available.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on August 26, 2015, 11:42:40 PM
Thank You!

*edit*

Just downloaded and installed... Ran a few test files and kept getting the same errors that I've never seen before. I just ran this same file last night and worked fine but that was before I updated to the latest MLVFS.

I also deleted .IDX files and tried again with no luck.

https://vimeo.com/137434601

(https://farm6.staticflickr.com/5718/20722743888_f42e89ab34_n.jpg) (https://flic.kr/p/xzctMJ)

Anyone else encountering similar errors?

*edit #2*

I just downgraded to an earlier version of MLVFS and ran into the same error -- now that most likely tells me that it's definitely not due your app and probably has to do with the pathway of the codec that I just installed DNxHR recently (although it worked fine last night) but I'll confirm this with you all once I get to the bottom of this.

*edit #3*

Got home, downloaded latest MLVFS on Mac Mini, Ran test file in AE and exported all Codec's like normal. Confirmed it has nothing to do with your app, once again I apologize for the false alarm. It definitely has to do with the pathway somehow the codecs got misplaced.

Thanks again, David for your great work so my question now is does the latest updates with automatic lens corrections affect the quality a little similar to how it does if we were to do the same in ACR? If so, will there be a way to disable it? (not that I wanted to but just a thought worth asking).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on September 01, 2015, 06:34:28 AM
Hello, couple things.

I've done this aspect ratio editing before and was successful in getting Ppro to recognize the new AR in stretched slo-mo footage from a 5D3. Unfortunately I can't remember exactly how I did it. I could have sworn I opened up a DNG in TextWrangler but maybe it was something else. However this is what I do remember:

-I used the DefaultScale option and calculated the PAR needed to counteract the "stretch"
-It was only one value I changed
-I only needed to edit the first file in the sequence. It seems Ppro gets its information from that first file and ignores the metadata in the rest. maybe this is common knowledge just thought I would mention

Also for those of you having trouble getting the localhost:8000 site to work, make sure you wait long enough. The webpage won't load until all IDX files have been created next to every MLV file you're trying to mount, and for me this took upwards of 10-20 minutes for all my files. I was pulling my hair out until I realized I just had to be patient.

Many thanks to the author. This functionality is primo, top shelf, keep up the great work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 01, 2015, 08:17:56 AM
How could you edit defaultscale tag when it hasn, t been introduced since recently in mlvfs by dmilligan? Or is the tag included in mlv_dump, raw2dng?
Would be cool if you remember what you did exactly :).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on September 01, 2015, 06:30:35 PM
It's part of the DNG standard and has a default value of 1. I really thought I opened it up in TextWrangler and it was all there in XML format. Now when I open up a file all I see is gibberish. I can remember everything I did except how I edited the value in the first place! How are you all editing EXIF values anyway? Exif editor doesn't show the advanced metadata categories, defaultscale included.

Edit: So, I think the DefaultScale tag is misunderstood here. I'm not sure if MLVFS is doing this but I took a look at my 60p footage and the tag has values of H=1 V=1.6667. Not sure where these numbers are coming from. What we want to do is change this value to counter the "stretch" of 1728x606 footage, which is an aspect ratio of 2.85. We want an aspect ratio of 1.7778 (16:9) so the pixels will need to be taller than they are wide, does this make sense? That means the correct DefaultScale values should be H=1 V=0.624.

Problem is, exiftool returns "1 image files unchanged" every time I try to write the file, can somebody help?

exiftool -DefaultScale='1 0.624' test.DNG
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on September 01, 2015, 08:11:19 PM
Can you edit this file and test it in adobe premiere pro.
https://drive.google.com/file/d/0B4tCJMlOYfirOUlSdDR4Q043NHc/view?usp=sharing

I didn,t get it to work in Ppro but it worked in adobe camera raw.

I suggest you write exiftool -DefaultScale='1 1.6667' test.DNG to get correct values.


By the way here is the file with inserted defaultscale value set at DefaultScale='1 1.6667'
https://drive.google.com/file/d/0B4tCJMlOYfirdlJicS0yX1l3Z1U/view?usp=sharing
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on September 01, 2015, 08:48:50 PM
Danne not sure if you misread my post but it's editing files in general that I'm having trouble with.

This is my command
exiftool -DefaultScale='1 0.624' test.DNG

and I'm getting
Nothing changed in test.DNG
    0 image files updated
    1 image files unchanged


Any idea why this is?

Again, the 1 1.6667 are the wrong values to use to "de-stretch," that's why we want to change it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 01, 2015, 09:09:51 PM
Quote from: bwinter88 on September 01, 2015, 08:48:50 PM
Again, the 1 1.6667 are the wrong values to use to "de-stretch," that's why we want to change it.
Nope, those are the correct values. The stretch is due to pixel skipping. The pixel skipping is every 3rd column and every 5th row. You can only skip odd numbers of rows or columns or you'd end up with missing colors (either blue or red) due to the bayer array. This means the ratio is exactly 3:5.

I think you have your horizontal and vertical mixed up. Horizontal (x) axis is always listed first. That means 1, 1.6667 is the correct value (3:5 = 1:1.6667). The equivalent of that would be 0.6:1, which would also work, and I think it's close to what you are going for (0.624 is wrong though, should be 0.6 exactly b/c like I explain above, we know the exact stretch ratio is 3:5), but I figure it's better to stretch things larger than to compress them smaller.

I have tested this and it de-stretches correctly in ACR (but not PPro, which just ignores it).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on September 01, 2015, 10:35:59 PM
Ah, my mistake, thanks for the correction dmilligan, you are right. However I'm not sure I understand the 3rd column/5th row skipping reasoning for the PAR of 0.6. Regardless of what pixels the camera skips, the output (at least for my 5D3) is 1728x606 which is a 2.85 AR not 3.5. A PAR of .6:1 or 1:1.667 would give a 1.71 AR, not quite the 1.7778 we want, no?

Also, does anyone have any idea why I can't change any exif info using exif tool? I keep getting "1 files unchanged" returned. Verbose mode just says "nothing changed in test.DNG"
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on September 01, 2015, 11:12:45 PM
The amount the image is stretched is exact (it's 3:5 and I just proved why that is the case). Just because you are getting 1728x606 out doesn't mean that image is supposed to be 16:9 when "de-stretched" (which is the assumption you are making). The original recorded image area before stretch is not exactly 16:9 equivalent.

To get to a specific aspect ratio, you should "de-stretch" by the exact amount and then crop to get to the exact aspect ratio you want, otherwise you are slightly distorting the image either vertically or horizontally.

If you still don't believe me that the exact PAR is 3:5, then feel free to shoot a resolution chart (or something with a perfect square on it) and measure it exactly for yourself.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on September 02, 2015, 12:57:25 AM
Ah! That makes sense. so I'm losing a tiny bit more resolution at these slo-mo rates. Bummer. Anyway I can recall that I also changed either a crop size or an image size tag to "1920x1080" have you tried that? It's so bizarre because Ppro is clearly reading some kind of PAR info from the file as evidenced by the Interpret footage dialog. I wish I could remember what I did to make it work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: poromaa on September 04, 2015, 04:47:32 PM
Hi, Is there anyway to enable saving a file in the virtual folder? Maybe a setting in the FUSE-thing? This would have been great because then you could grade first frame of the footage in lightroom and save the metadata in the virtual folder. Then load it in after effects and have the grading done.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 04, 2015, 05:52:34 PM
I have implemented it so that you can save separate files into the virtual folder such as XMP sidecar files (when you do this, they will appear in a ".MLD" directory next to the original MLV file). Overwriting the DNGs themselves is impossible.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on September 04, 2015, 06:14:56 PM
Quote from: poromaa on September 04, 2015, 04:47:32 PM
Hi, Is there anyway to enable saving a file in the virtual folder? Maybe a setting in the FUSE-thing? This would have been great because then you could grade first frame of the footage in lightroom and save the metadata in the virtual folder. Then load it in after effects and have the grading done.
Perhaps why not just do it all solely in AE with SmartImport2 while running MLVFS?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: festr on September 10, 2015, 09:50:50 AM
Hi Im experiencing vertical banding on my 5d mk3 but it is not corrected by the latest mlv fuse. I can get rid of them.only when I shoot to raw and use raw2dng utility which detects them in debug output and dng is clear. I even see vertical bands in mlvviewer. Is this known issue? What I can do?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on September 10, 2015, 10:27:20 AM
Did you forget to "refresh" web browser after clicking on Vertical stripes fix in order for MLVFS to take effect prior to rendering otherwise?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: festr on September 10, 2015, 11:49:19 AM
Oh I totally forgot that there is web settings and so as you suggested it is not enabled by default probably. I will check it once I get to my pc again. Thank you for the hint.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: iheartbmx on September 29, 2015, 09:07:24 PM
I apologize if this has been asked, but its hard to track down - Right-clicking my folders brings up nothing for services. My MLV's are located on an external SSD if that makes any difference.

- Macbook Pro / 10.9.5
- I've installed the latest versions of FUSE and MLVFS (Also deleted previous versions)
- Attempted directly from CF and also from external SSD

Thanks in advance!

*Edit - Services is now working, however - WebGUI opens, navigate to the footage directory, WebGUI page does not exist. I attempt again, WebGUI fails entirely. Double clicking the mounted folder does not work either, but it shows up on the desktop.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on September 29, 2015, 11:25:43 PM
How many GB of MLV in total? If lots of footage then give it some time for MLVFS to generate all the required .IDX files for them to show up on the WebGUI.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: iheartbmx on September 30, 2015, 08:08:51 AM
Quote from: DeafEyeJedi on September 29, 2015, 11:25:43 PM
How many GB of MLV in total? If lots of footage then give it some time for MLVFS to generate all the required .IDX files for them to show up on the WebGUI.

64GB's; Ill try again and give it some time. Thx for the help.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on October 06, 2015, 06:39:50 AM
Is anyone else having issues using the files from MLVFS in Adobe Premiere? It takes several seconds for the image to update in the source/preview monitors and I get the "media pending" screen often. When I convert using another method, RAWMagic for instance, playback and workflow is smooth.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: budafilms on October 07, 2015, 09:12:42 AM
- I have installed MLFVS in OSX Yosemite. I upgrade to EL Capitan, and everything is working with out resin tall again the scripts.

- The only confusing thing for me using this tool is the steps and pop up windows from navigator. I choose the option that I think (Resolve, Vertical Corrections) and I reload the page. After that I drag and drop the files to a local disk. I supposed that settings is applied in my DNGs footage.

- The other question is if exist a DNG compress, to save space.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: silenttapes on October 07, 2015, 11:50:48 AM
Please accept my apologies if this has already been asked, I have attempted to search but to no avail.

Is there any chance of MLFVS being able to reference a pool of dark frames (1 for each ISO), and then subtract it on the fly?

I understand I can do it per clip with MLV_dump but would rather use a more efficient method. I am having a tough time bringing in FPN removal to my work flow. My subtract experiments work much better in AE (Davinci Resolve seems to massively darken image instead, maybe i'm doing it wrong). AE isn't so great for my .xml workflow and I would much rather be working Davinci.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 07, 2015, 01:04:17 PM
Quote from: budafilms on October 07, 2015, 09:12:42 AM
- The other question is if exist a DNG compress, to save space.
The DNGs are virtual, so they already take up 0 space. "Compressing" them would only slow down MLVFS and they would still take up 0 space. However, it sounds like you are using this tool to permanently convert the DNGs ("After that I drag and drop the files to a local disk"). This sort of defeats the whole intended purpose of MLVFS, therefore I see no reason to do all the work to implement something like this. If you are permanently converting the DNGs anyway, there are plenty of tools out there than can compress them, and there are other MLV converters that provide compressed DNGs.

Quote from: silenttapes on October 07, 2015, 11:50:48 AM
Is there any chance of MLFVS being able to reference a pool of dark frames (1 for each ISO), and then subtract it on the fly?
Yes it's possible, and something I'd like to see added. Can't say when I might have time to implement it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: budafilms on October 09, 2015, 07:25:23 AM
@dmilligan

It's true what you said. I'm permantently converting and dragging DNG. I tried all the converters under OSX, and the differences in quality from this to other are big. Not only in the size, I just tried to export with the same setting and color post processsiong one DNG from MLVS and one from other converter and the difference it's huge in color, in noise, even in death pixels from one to other.

The way to check and clean unusual shots in my case it's after copy DNG to my local disk, I open Resolve and see the take. I can leave or permanently delete with a tool in Resolve - with the other click of the mouse over the shot. I know lot of people use MLViewer to do this but I get repeat freeze from that software and lot of problems.

My question to compress is in the local disk. As I understood, DNG could be compress and the quality loss it's not percieve at all. Other sofware do this, but as I said, this is the more stable for me.

Thanks for your tme and sorry for my english, I talk italian or spanish.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on October 17, 2015, 10:40:08 PM
My default browser is Chrome and after installing El Capitan I noticed that instead of the _PREVIEW.gif I'm getting a broken image icon. Don't know if it has to do with the system but it used to work fine.

Anyone else experiencing this?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 18, 2015, 01:58:41 AM
Click the 'Load' button. Previews no longer load by default.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on October 18, 2015, 07:32:11 AM
Doh!

Works fine. Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: templar on October 19, 2015, 05:17:28 PM
@dmilligan: great work and cool idea to mount and use the stuff on-the-fly

Do you know how far behind the windows version is?
Has someone actually tested this on windows with Davinci Resolve?
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on October 19, 2015, 05:32:26 PM
@templar:

http://www.magiclantern.fm/forum/index.php?topic=13268.0
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 19, 2015, 06:58:31 PM
There are two different windows implementations, one written by g3gg0 that uses WebDAV, that is somewhat different from the FUSE version in terms of implementation and features, and one written by Ayshih that uses PFMAP that is similar and has a lot of code in common with the FUSE version. However, they are both rather far behind the FUSE implementation (Mac/Linux version). I think in both cases it's probably been a year since they've been updated.

There is another alternative that is rather complex, and involves running the Linux version of MLVFS in a virtual machine:
http://magiclantern.fm/forum/index.php?topic=13152.msg142308#msg142308
http://magiclantern.fm/forum/index.php?topic=13152.msg142641#msg142641
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on October 20, 2015, 03:56:22 AM
Is anyone having trouble with importing with MLVFS? Programs are hanging indefinitely when I try the open file dialog box for browsing the filesystem when an MLVFS disk is mounted. As soon as I eject the image the problem goes away. I've reinstalled MLVFS as well as FUSE. I'm on OSX.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on October 20, 2015, 04:51:28 AM
Which OS X are you running?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: templar on October 20, 2015, 01:41:58 PM
@dmilligan thanks for your reply: HM, ok I see.
I guess the bottom line is - since I am now running win10 and the win version of the mounting scripts are more than a year old it's not an option anymore for us windows users.

have you heard of any plans to update the mounting sw?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 20, 2015, 07:45:50 PM
I have no plans to update the Windows version, as I don't own a Windows computer (and don't plan to). This project is open source, so anyone is more than welcome to update it themselves. Ayshih was the one who did the PFMAP version originally. We haven't heard from him in a while...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on October 21, 2015, 01:35:28 AM
@Templar , Why don't you use The PC version of mlvfs ?

Install PFMAP (http://www.pismotechnic.com/download/archive/pfmap-171-win.exe)
Download mlvfs.dll (https://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs.dll)
Then register the MLV formatter DLL:
pfm register mlvfs.dll
Usage: right-click on an MLV file and select "Quick Mount".

Work great , just doesn't have the fancy web GUI in the Mac version.
I use MLVFS on MAC & PC , mind you if you need chroma smoothing , bad pixel , vertical stripes fix & dual ISO
not sure if PC version as it as I have a 5d2 and don't need any of those extra setting
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on October 21, 2015, 07:09:19 AM
I was testing out MLVFS with a camera that has the focus pixel issue with raw video. No problem dealing with this on straight MLV by simply applying Chroma Smoothing 2x2 and turning on Bad Pixel Fix but when I got into dual_iso footage I just couldn't seem to get rid of all the focus pixels. I'm going to resurrect a discussion from a year ago starting here (http://www.magiclantern.fm/forum/index.php?topic=13152.msg131584#msg131584) and hopefully someone could shed some light on the current situation.

Quote from: dmilligan@DeafEyeJedi, those look like focus pixels, you need to turn on chroma smoothing to get rid of them. Right now you might get some weird artifacts with chroma smoothing, I need to incorporate it into the correct sequence within cr2hdr stuff. There are also several cr2hdr options that I have not implemented yet. These options, when implemented, should improve image quality greatly, and match the result of the standard cr2hdr.
Quote from: a1exNote that focus pixels with dual ISO are still unsolved, at least in my implementation (didn't try MLVFS yet, but will definitely do it). So try not using both at the same time - I know you can, from your signature :P
Quote from: dmilliganOkay I implemented the chroma smoothing the same way it's in cr2hdr (chroma smooth fullres and halfres before final blending). It helps with the focus pixels, but doesn't remove them completely. If you do the chroma smoothing after cr2hdr is completely finished, the focus pixels are completely gone, but there are strange green blob artifacts.

I haven't updated the binaries yet (I'll wait till some more updates)

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on October 21, 2015, 07:44:20 AM
Thanks for bringing this up again @dfort!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Mehmet Kozal on October 21, 2015, 09:10:18 AM
Not about dual-iso but ,my post about this issue with regular MLV files and MLVFS | Mystic http://www.magiclantern.fm/forum/index.php?topic=15808.msg153823#msg153823
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: templar on October 21, 2015, 10:22:46 AM
@reddeercity thanks for the hint, did you test this on win10?

I assume, BSOD mean Blue Screen Of Death...  ;)
http://www.magiclantern.fm/forum/index.php?topic=13268.125
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 21, 2015, 03:10:12 PM
@dfort,
One possibility is mapping the focus pixels for each camera and then interpolating out those pixels (rather than just chroma-smoothing the entire image). This might fix the dual ISO / focus pixel issue, IDK. It should be faster than chroma-smoothing, and not have any artifacts. It's a lot of work though, you need to create a focus pixel map for each camera and video mode.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on October 21, 2015, 06:04:33 PM
Sounds like a good plan regardless of how long this will take to make. I'm happy to help, @dfort!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on October 21, 2015, 06:42:54 PM
I think that was the method used by the PinkDotRemover (http://www.magiclantern.fm/forum/index.php?topic=6658.0) tool. Never could get that tool to work for me.

Danne is also looking into some sort of focus pixel masking or mapping solution.

This is an "interesting" problem because it looks like chroma smoothing removes the focus pixels on non dual_iso raw footage but it doesn't seem to work with dual_iso. I was able to remove the focus pixels in MLVFS on dual_iso footage with the "Bad Pixel Fix" set to "Aggressive" but it also pretty much destroyed the image. The following frame was processed using the cr2hdr Lightroom plugin but the results were the same in MLVFS.

Focus Pixels showing up on EOSM MLV Dual_ISO cropped mode 1280x720
(https://farm6.staticflickr.com/5765/22373110971_36823003be_z.jpg)

Same frame using --really-bad-pix option ("Bad Pixel Fix" set to "Aggressive" in MLVFS)
(https://farm1.staticflickr.com/671/22362420585_229bb60771_z.jpg)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on October 21, 2015, 11:51:31 PM
Quote from: templar on October 21, 2015, 10:22:46 AM
@reddeercity thanks for the hint, did you test this on win10?
Your welcome,
Yes , and to sure I just Tested it now and no problems.
(Windows 10 Pro on SSD Hyper-X, AMD FX8350-4.9Ghz , 8GB Ram 1600MHz, 2xGTX580-3GB Vram
On Large Files it may take a few minutes so be patient, make sure you are on the fastest drive e.g. SSD or raid etc.....
Quote from: templar on October 21, 2015, 10:22:46 AM
I assume, BSOD mean Blue Screen Of Death...  ;)
Yes ,
MLVFS windows client is not really working well right now , I would just stick with the simple "PFMAP" Version
for Windows 10 .
I wrote a post for Windows 10 + Magic Lantern Raw Video Tools (http://www.magiclantern.fm/forum/index.php?topic=15654.msg152335#msg152335) have a look at that so see what supported   
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: templar on October 22, 2015, 09:12:44 AM
cool, sounds great!

However, AMD processor tz tz tz tz  :)

yeah I do have a SSD. I will test this on my win10 machine with *wonder wonder* including a Intel CPU  8)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on October 22, 2015, 11:27:26 PM
Quote from: dmilligan on October 21, 2015, 03:10:12 PM
@dfort,
One possibility is mapping the focus pixels for each camera and then interpolating out those pixels (rather than just chroma-smoothing the entire image). This might fix the dual ISO / focus pixel issue, IDK. It should be faster than chroma-smoothing, and not have any artifacts. It's a lot of work though, you need to create a focus pixel map for each camera and video mode.

Thanks again dmilligan - I'm taking off with that suggestion!

http://www.magiclantern.fm/forum/index.php?topic=16054.msg155923#msg155923
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: templar on November 04, 2015, 05:58:57 PM
Ok tested on win10 with davinci resolve. Works well.

Thank you! This is brilliant!!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: silenttapes on November 04, 2015, 07:46:30 PM
Hey!

I've been searching for information regarding CompressMLV.dmg and its integration into MLVFS:
-it says testing on the 1st post of this page, whats your experience of using it? (i do want to compress for archival and general space saving during project use)
-how much slower is it?
-would you recommend it?

A final unrelated question: has anyone worked out a way to keep your project MLVs only (auto delete unused ones, preferably via davinci)?

Cheers,
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: iheartbmx on November 07, 2015, 10:12:15 PM
Where could I download older versions? Thanks
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on November 08, 2015, 12:36:42 AM
Nowhere. Why?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: iheartbmx on November 08, 2015, 01:40:05 AM
Quote from: dmilligan on November 08, 2015, 12:36:42 AM
Nowhere. Why?

Well I'm having issues with MLVFS:
-The web GUI fails when I navigate to the folder containing the MLV's
-If I try running MLVFS directly on the folder with MLV's it fails
-The mount on my desktop is acting like an icon and thats about it (clicking it does not bring up a folder)

I thought an older version may fix this issue. I made sure the directory is a blank folder, I tried the previous version of OSXFuse, and I've just upgraded to El Capitan.. but still no luck.

Any help would be much appreciated, Thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 08, 2015, 02:34:08 AM
Things you can try:

- figure out if it is a specific MLV file that is causing the problem (try mounting various combinations of the MLV files you have, a binary search would narrow it down to a single file the quickest)
- check for crash logs in ~/Library/Logs/DiagnosticReports
- run mlvfs from the command line and send me any output

Also you could be a little more specific about what you mean by 'it fails'. Do you get an error message, does it crash, does it just sit there, what exactly happens. If you have lots of MLV files, or really large ones, it make take a while for mlvfs to index them, so it may sit there for a while until the indexing is finished. You can check the folder with the MLV files in it for .IDX files being created.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: iheartbmx on November 08, 2015, 08:29:46 AM
Quote from: dmilligan on November 08, 2015, 02:34:08 AM
Things you can try:

- figure out if it is a specific MLV file that is causing the problem (try mounting various combinations of the MLV files you have, a binary search would narrow it down to a single file the quickest)
- check for crash logs in ~/Library/Logs/DiagnosticReports
- run mlvfs from the command line and send me any output

Also you could be a little more specific about what you mean by 'it fails'. Do you get an error message, does it crash, does it just sit there, what exactly happens. If you have lots of MLV files, or really large ones, it make take a while for mlvfs to index them, so it may sit there for a while until the indexing is finished. You can check the folder with the MLV files in it for .IDX files being created.

Thanks, The web GUI works as normal until I click the folder name containing the MLV's, It brings me to a blank page like I'm not connected to the web.

I have not tried running it from the command line as I'm not familiar with it. I pulled up the log but I'm not sure what to look for (It did say that thread#2 crashed). I copied one MLV to an external drive and ran MLVFS, and that worked fine. My assumption is that I filled the 32gb cf card to the point that MLVFS is not able to write anything else like the index files - is that even possible?.. My second 64gb card (that I didn't fill) worked fine with MLVFS. I should also mention the 32gb card was acting odd and has also done so in the past, and I though it was corrupt. I tried moving all the contents of the card to an external drive and was getting multiple error codes (-36, -43, -1407). Maybe my card is the issue here..? ugh..

I moved all the files to an SSD and then ran it through MLVFS and it worked fine.

Thanks again for all the help!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on November 08, 2015, 02:28:42 PM
Definitely sounds like the card is the problem. Can you please send me the crash log(s).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Fer5275 on November 30, 2015, 06:58:25 PM
Thank You!!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: monky on December 05, 2015, 11:11:26 PM
Hey Guys, im using mlvfs and its working like a charme for me only 1 thing seems to be strange.
The White Balance is not like it was in the shot its really blueish.
(http://fs5.directupload.net/images/151205/temp/mefnjacp.jpg)
Am i doing something wrong? Or better is there an easy Fix for that? 
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 06, 2015, 12:34:47 AM
Could you have been shooting in awb? I think you can use all modes except awb, and if you use awb it will rely on whatever kelvin your camera i set on. A really good way to nail white balance is via the Auto adjust Kelvin + G/M in ML menu. http://www.magiclantern.fm/forum/index.php?topic=16283.msg158136#msg158136
Unfortunately GM and BA numbers aren,t computed in any converter that I am aware of at the moment.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: monky on December 06, 2015, 11:59:01 AM
yes i was shooting in awb. But it seems linke is just a problem in mlvfs the footage is showen correctly in mlvrawviewer rawmagic and so on.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 06, 2015, 01:16:44 PM
It,s not correct. It goes to default 5500k in those converters.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on December 06, 2015, 07:32:06 PM
Some updates:

Loading pages in the webgui is now asynchronous, so if the index files have not generated yet, the page will come up immediately, and the rows will say "loading" until MLVFS processes the corresponding file.

If you have a large set of files, help test the feature by deleting existing .IDX files and letting MLVFS regenerate them. The webgui should come up immediately, and the rows should load asynchronously.

Also I've added an experimental "deflickering" feature that will match the exposure of each frame by equalizing the median pixel values of each frame to some specified value. This hasn't been extensively tested, so please let me know if/how well it works. It also depends on your raw processor actually respecting the BaselineExposure tag, so if it doesn't work, this may be the reason. AFAIK, ACR seems to respect this tag, but it typically applies its own non-linear processing to images, making deflickering more unpredictable.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: monky on December 06, 2015, 09:32:51 PM
The new Version is a little problematic for me. At the Gui the Videoinformation didnt show up. In Davinci i get an error message of a full gpu drive and wrong previews
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on December 06, 2015, 10:05:16 PM
Could you be a little more specific, provide more detailed information and perhaps post some screenshots?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 06, 2015, 11:16:18 PM
Quote from: monky on December 06, 2015, 09:32:51 PM
At the Gui the Videoinformation didnt show up.
ok, think I fixed this part, try new download
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: monky on December 07, 2015, 07:02:45 PM
Loading inside the Gui is now okay for me.

But here are my Davinci Errors:
(http://fs5.directupload.net/images/151207/j2csfozj.jpg) (http://fs5.directupload.net/images/151207/yhieenis.png)
I run it on a Macbook Pro 2013 with an Geforce 650 Videocard.

*Edit*


Seems like its working again now.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 07, 2015, 09:17:01 PM
Very cool stuff, deflickering. Just tried with a HDR file to see the reaction(dforts suggestion) value set to 5000 . ACR evens out the exposures quite good actually between iso 100-1600. I can see how good this will be with timelapses and frsp. Too bad dcraw doesn,t read these dng values.
(http://s12.postimg.org/6cq12aaft/Screen_Shot_2015_12_07_at_21_16_24.jpg) (http://postimg.org/image/6cq12aaft/)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on December 07, 2015, 10:17:10 PM
@monky,
I don't see how resolve running out of GPU memory could have anything to do with MLVFS.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: monky on December 07, 2015, 11:54:49 PM
hm dont know. But its happaning since i updated mlvfs and there are glitchy thumbnails of the video itself.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: anzecokl on December 12, 2015, 12:45:22 PM
I am having same problems with white balance as were described some time ago using Premiere Pro CC. Playing real time no problem, sound is in sync, everything just works, EXCEPT white balance, which is completely off no matter how i set it (awb, any of the presets or manual kelvin input). What am i missing?
Using MLVFS for me at least beats any other workflow!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on December 12, 2015, 01:25:32 PM
Post an MLV file that exhibits the problem
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johannsebastianbach on December 24, 2015, 03:29:13 AM
I'm sorry if this has been asked here before, but is there any chance to open .RAW files with it?
It's the only way for me to shoot 60p (MLV gives out pink frames) and MLVFS is essential if you don't want or can't have double gb for projects  ::) :)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on December 24, 2015, 11:26:03 PM
Perhaps give @Danne's MLP a chance on your Mac if you really want RAW instead of MLV? [emoji6]
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lanternman on December 25, 2015, 04:32:22 AM
hi guys,

Is there a way to batch import all the MLV clips into PP CC without selecting the first DNG file of every MLV clip? I'm using 6D ML and can only record 6s per clip, hence it would be extremely tedious if I have to import each clip individually.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johannsebastianbach on December 31, 2015, 11:15:27 AM
There is a AE Script to do so. You probably would need to convert your cDNG footage in a editable format like ProRes or dnxhd first anyway: http://www.magiclantern.fm/forum/index.php?topic=12422.msg119832#msg119832
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 03, 2016, 07:40:42 PM
Updates:

Implemented (https://bitbucket.org/dmilligan/mlvfs/commits/753d7ea1ccf269ecd000f04b3e1fad0849d5de1a) a1ex's pattern noise reduction algo: http://www.magiclantern.fm/forum/index.php?topic=11787.msg158766#msg158766
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on January 03, 2016, 08:13:13 PM
Cool, looks like it worked with minimal changes.

Samples welcome, especially if the algorithm struggles (either if it fails to fix the pattern noise in some images, or if it actually introduces more stripes).
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on January 03, 2016, 08:51:13 PM
Hell Yeah ... Thanks for the update and Prosperous New Years to y'all!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 05, 2016, 02:26:31 AM
Quote from: a1ex on January 03, 2016, 08:13:13 PM
Cool, looks like it worked with minimal changes.
All I really had to do was switch the assumed bayer order, but it would be fairly easy to just read the bayer order metadata and adjust accordingly.

Quote from: a1ex on January 03, 2016, 08:13:13 PM
Samples welcome, especially if the algorithm struggles (either if it fails to fix the pattern noise in some images, or if it actually introduces more stripes).
Didn't try to throw anything tough at it, just something simple with really apparent pattern noise so I could just make sure it was actually doing something.

https://bitbucket.org/dmilligan/mlvfs/downloads/52020000_000000_orig.dng
https://bitbucket.org/dmilligan/mlvfs/downloads/52020000_000000.dng

100% crop, ACR, exposure +5 ev
(https://bitbucket.org/dmilligan/mlvfs/downloads/patternnoise.jpg)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: baldavenger on January 05, 2016, 02:54:01 AM
I had a go with some noisy high ISO footage. In this case it introduced horizontal anomalies.

Original:

(http://i.imgur.com/AfDhg4fh.png)

With Fix Pattern Noise selected:

(http://i.imgur.com/ao5b4QOh.png)

I tested on the same footage that had a dark frame mlv subtracted via mlv_dump. The dark frame subtraction worked well, but the horizontal anomalies where the same as before once Fix Pattern Noise was selected.

This download contains the 4 dngs:

https://www.dropbox.com/s/ai343f6xo7a6zpq/pattern%20noise%20test.zip?dl=0

I hope this can be of some help to you. As always, I really appreciate the great work you're doing :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: hjfilmspeed on January 05, 2016, 05:33:55 AM
Whoa!!! Is there anyway to test this on windows!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on January 05, 2016, 06:17:46 AM
http://magiclantern.fm/forum/index.php?topic=16422.msg159671#msg159671
QuotePlus I show how to install MLVFS for windows including register the MLV formatter .dll

Or try running OS X with standard PC hardware. AKA Hackintosh. 2 machines running here.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on January 05, 2016, 08:37:54 AM
Or you can get a Linux VM (try a lighter one like Lubuntu) started in VirtualBox, extract the DNGs with MLVFS and put them in the shared folder.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Lars Steenhoff on January 05, 2016, 11:12:36 AM
Great progress !
The example from dmilligan looks super clean !
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: swinxx on January 06, 2016, 03:56:06 PM
hello!

i have a problem with the new version of mlvfs.. when i try to mount a dir with mlv files, i select the mountpoint (same hdd as ever) and then the files start to load (the dir is without idx files) then suddenly the mlvfs mounting points disappear suddenly? is my system broken?
thx
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on January 07, 2016, 01:04:52 AM
logs? example files that cause the problem? any sort of clues at all you can give me?

http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 08, 2016, 04:12:03 AM
@baldavenger,
I think those artifacts are due to a bug on my part, not a result of a1ex's algorithm. I think I fixed it, will compile a new build soon. Working on some optimizations to make it faster.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: budafilms on January 08, 2016, 09:09:11 AM
I don't know if this helps to Devs and research: in the open source soft called HandBreak, in settings, are a couple of new function about Noise Reduction and works very well, results very similar as the noise reduction from Davinci Resolve Studio.

;)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on January 08, 2016, 09:36:31 AM
You mean for NR to be implemented into MLVFS, @budafilms?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: MortenSJ on January 19, 2016, 04:19:50 AM
Quick question. Am i supposed to go into the mounted folder/drive and copy the MLV folders to another folder?

I get the mounted drive and when i click it, i see my folders, but i'm not really sure what to do after that?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on January 19, 2016, 11:03:39 AM

Quote from: MortenSJ on January 19, 2016, 04:19:50 AMQuick question. Am i supposed to go into the mounted folder/drive and copy the MLV folders to another folder?
You do what you want, you can copy the DNG files somewhere else and use those (keep MLV or not), or use them straight from that folder (realtime decoding FTW!).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ArthurX on January 19, 2016, 03:32:48 PM
Hi All,

Thanks for the great work you are all doing. I use MagicLantern for a long time. But I am new to Raw ML video. I use Debian and try to install mlvfs,
but get this error message:
Main point can not  find fuse.h and
hdiutil

main.c:33:18: fatal error: fuse.h: No such file or directory
#include <fuse.h>
                  ^
compilation terminated.
Makefile:35: recipe for target 'mlvfs' failed
make: *** [mlvfs] Error 1
make: Leaving directory '/usr/local/src/mlvfs/mlvfs'
cp: cannot stat 'mlvfs/mlvfs': No such file or directory
/usr/local/src/mlvfs/build_installer.sh: 22: /usr/local/src/mlvfs/build_installer.sh: hdiutil: not found
/usr/local/src/mlvfs/build_installer.sh: 23: /usr/local/src/mlvfs/build_installer.sh: hdiutil: not found
/usr/local/src/mlvfs/build_installer.sh: 28: /usr/local/src/mlvfs/build_installer.sh: hdiutil: not found
/usr/local/src/mlvfs/build_installer.sh: 29: /usr/local/src/mlvfs/build_installer.sh: hdiutil: not found


I looked around to find the hdiutil package, couldnt find it myself.

thanks in advance,
Greetings Arthur
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on January 19, 2016, 04:00:53 PM

Quote from: ArthurX on January 19, 2016, 03:32:48 PM
main.c:33:18: fatal error: fuse.h: No such file or directory

-> https://packages.debian.org/search?searchon=contents&keywords=fuse.h&mode=path&suite=stable&arch=any (https://packages.debian.org/search?searchon=contents&keywords=fuse.h&mode=path&suite=stable&arch=any) -> apt-get install libfuse-dev

Quote from: ArthurX on January 19, 2016, 03:32:48 PM
/usr/local/src/mlvfs/build_installer.sh: 22: /usr/local/src/mlvfs/build_installer.sh: hdiutil: not found

No you don't run the .SH, is for Mac (I guess), anyway, after a successful compile, put mlvfs in /usr/local/bin/ and that's it
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on January 19, 2016, 04:03:37 PM
apt-get install libfuse-dev
Or something like that.

The build_installer.sh is only for Mac. It creates a Automator workflow and Mac dmg using a Mac utility called hdiutil.

To "install" MLVFS on Linux, just 'make' and then copy the executable somewhere on your path.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ArthurX on January 19, 2016, 06:47:56 PM
Milligan and Licaon,
Thanks a lot. It works like a charm. :) :D
Have to start using it now. The video is playing in the localhost:8000
Arthur
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ibrahim on January 19, 2016, 07:27:39 PM
I've installed the files pfm-178-win-exe and mlvf.dll and put them in one and the same folder. From the thread I learned that the v.1.7.8 is more stable for my win 7.

Might be a stupid question, but how exactly do I carry out the following step:

QuoteThen register the MLV formatter DLL:
Code: [Select]
pfm register mlvfs.dll
Unlike the FUSE approach on the other platforms, you mount each MLV file individually: right-click on an MLV file and select "Quick Mount".
If you wish, you can build the MLV formatter DLL yourself by installing the Windows SDK and running nmake using the SDK command prompt in the mlvfs/win directory.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on January 19, 2016, 08:28:27 PM
Quote from: ibrahim on January 19, 2016, 07:27:39 PM
I've installed the files pfm-178-win-exe and mlvf.dll and put them in one and the same folder. From the thread I learned that the v.1.7.8 is more stable for my win 7.
Might be a stupid question, but how exactly do I carry out the following step:

Read this post , should help New PC Tools to replace ACR & A.E. with same or better results (Without Resolve) (http://www.magiclantern.fm/forum/index.php?topic=16422.msg159671#msg159671)

FYI , I use pfm-171-win-exe on my Win7 pro Laptop & Desktop Workstation works just fine.
If you have your MLV files on a SSD then mount it , it will work a lot better then off the memory card .
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ibrahim on January 19, 2016, 11:15:28 PM
Thanks.
I have installed pfm-178-win-exe and followed the first part of the video tutorial (installed the DNG codec and registered the dll).

When I mount the .MLV RAW footage the DNGs are exported into c drive > volume. However, I still see horizontal lines (dual ISO); it hasn't been processed. What's the missing step?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on January 19, 2016, 11:21:59 PM
Windows version is way behind and does not support dual ISO conversion.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on January 19, 2016, 11:55:09 PM

Quote from: dmilligan on January 19, 2016, 11:21:59 PM
Windows version is way behind and does not support dual ISO conversion.
Virtualbox (https://www.virtualbox.org/wiki/Downloads) -> Lubuntu (https://help.ubuntu.com/community/Lubuntu/GetLubuntu) -> MLVFS -> Shared folder (http://virtuatopia.com/index.php/VirtualBox_Shared_Folders) -> better?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ibrahim on January 20, 2016, 12:07:20 AM
Quote from: Licaon_Kter on January 19, 2016, 11:55:09 PM
Virtualbox (https://www.virtualbox.org/wiki/Downloads) -> Lubuntu (https://help.ubuntu.com/community/Lubuntu/GetLubuntu) -> MLVFS -> Shared folder (http://virtuatopia.com/index.php/VirtualBox_Shared_Folders) -> better?

Is this the workflow to handle dual iso in windows or in linux?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on January 20, 2016, 12:22:22 AM
Quote from: ibrahim on January 20, 2016, 12:07:20 AM
Is this the workflow to handle dual iso in windows or in linux?
You will export the processed dual-iso files in Linux and you can copy them to Windows.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ibrahim on January 20, 2016, 12:27:13 AM
Using linux on my windows and then return to windows?... :D Sounds confusing. :(
Is there another way to remove noise from my footages without using dual iso+ETTR? That is why I am trying to use dual iso+ETTR.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on January 20, 2016, 01:04:47 AM

Quote from: ibrahim on January 20, 2016, 12:27:13 AM
Using linux on my windows and then return to windows?
It gets the job done.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on January 20, 2016, 07:17:25 AM
Hello everyone,

     I'm having a difficult time with MLVFS. I have vertical banding in my shots, and even with the vertical stripes fix on, the dng files are not changing. I am shooting on a 7D. Is there a step I'm missing? It seems when the directory pops up on Safari everything is fine, but even with Chroma Smoothing 2x2 I get weird purple and green artifacts, almost like dead pixels. In MlRawviewer (I know it's not developed anymore) I don't get vertical stripes. I'm guessing since there is less detail in this program and because the vertical stripe fix works in MLRawviewer. Can anyone help me? Thanks.

- Paul Harwood
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on January 20, 2016, 03:57:10 PM
AFAIK, the 7D's vertical stripes and the 5D3's are slightly different in nature. MLVFS incorporates the vertical stripes fix created by a1ex for 5D3. MLRV has a different algorithm developed by baldand who was a 7D user and created it mainly for 7D.

People with 5D3s who use MLRV actually have the opposite problem you're having.

Chroma smoothing will not fix vertical stripes (a luminance issue).

It might be a good idea to post and example file.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on January 20, 2016, 06:24:01 PM
Here are some DNG's from 7D recently...

(https://farm2.staticflickr.com/1528/23876614673_c46996cff2_n.jpg) (https://flic.kr/p/CnTTGe) (https://farm2.staticflickr.com/1646/24503424065_d2ac307289_n.jpg) (https://flic.kr/p/DkhshH)

https://mega.nz/#F!6okHxJIZ!Mu8084Iw31wxf9Cut9ryEw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johannsebastianbach on January 20, 2016, 07:20:25 PM
What do you guy's think about a resizing option? I already love that MLVFS is automatically stretching the DNG's from 1280-mode in the right aspect ratio and it could be useful in conforming the size already before cutting (e.g. for multicam shots or if you need to downsize the framing to get more fps or so), right?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on January 20, 2016, 08:29:20 PM
You can't really resize raw data until after you demosaic it (you could downsample it, but that would potentially introduce aliasing, and why do that anyway?) and then it's not raw data anymore. Pixel aspect ratio is just metadata. We are telling the raw processing software that it needs to do the stretch, but we can't actually do the stretch ourselves (we don't demosaic raw data, and I don't plan on adding any such feature).

Only additional thing that would be possible is specifying a custom pixel aspect ratio for anamorphic shooters.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johannsebastianbach on January 20, 2016, 10:12:16 PM
Quote from: dmilligan on January 20, 2016, 08:29:20 PM
You can't really resize raw data until after you demosaic it (you could downsample it, but that would potentially introduce aliasing, and why do that anyway?) and then it's not raw data anymore. Pixel aspect ratio is just metadata. We are telling the raw processing software that it needs to do the stretch, but we can't actually do the stretch ourselves (we don't demosaic raw data, and I don't plan on adding any such feature).

Only additional thing that would be possible is specifying a custom pixel aspect ratio for anamorphic shooters.

That makes sense, learned something new, thanks :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on January 21, 2016, 01:48:18 AM
Hey dmilligan and DeafEyeJedi,

     I think I found a solution in MLVFS. I turned off Chroma Smoothing, kept Vertical Stripes fix on, and I switch Bad Pixel Fix to Aggressive. This seemed to eliminate most of the vertical stripes. I could use MLRawviewer to export cdngs, but MLVFS is so fast and convenient. Do you know what ISO settings I should try to stay at? And is there a way to set the EV and Black Levels in the camera to avoid vertical stripes all together? I read this online, but I don't quite understand it. Any more help would be greatly appreciated, and thank you for all that you have already done.

- Paul Harwood
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on January 21, 2016, 02:43:12 AM
You need to be more specific about the type of vertical stripes you are talking about because it is not clear (which is why I asked you to upload a sample).

I'm not sure what you mean by "EV" which is typically used as a unit of measurement based on a logarithmic scale. Perhaps you mean "EC" (exposure compensation)?

Black level is not something you can set or adjust either. It is piece of metadata need by the raw processing software (e.g. ACR, Resolve, etc.) to correctly interpret the raw data. It is either correct (matches the actual physical black level of the sensor) or incorrect (does not match). If it is wrong, your image will look totally screwy and have extreme color casts. This is something that you shouldn't need to fool with and should be totally transparent to you as an end user.

There are posts in this forum about how to "fix the black level". This is only necessary if you have footage with incorrect black level metadata (as a result of an ML bug). But this does not sound like your issue.

Discussion about choosing ISO or optimizing exposure is really outside the scope of this thread.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on January 21, 2016, 09:09:49 PM
Hello dmilligan,

     Thanks for your reply. Here is a link to the vertical stripes issue I'm having with the 7D: https://drive.google.com/open?id=0B7-vwCkJCer5TmRFVnBiOWJsX0E

If you could take a look and provide any input that would be awesome. Thanks.

- Paul Harwood
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on January 21, 2016, 09:39:31 PM
Sorry everyone, here is the link, but now it is shareable to all of you: https://drive.google.com/file/d/0B7-vwCkJCer5TmRFVnBiOWJsX0E/view?usp=sharing
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bwinter88 on January 29, 2016, 01:43:09 AM
I'm having trouble loading After Effects projects with MLVFS image sequences in them. I've been sitting here for 10 minutes and the Open Project progress bar is at 32%. mlvfs process in activity monitor shows 99% CPU. In fact, anytime I try to open sequences in Premiere or AE, the program hangs and MLVFS process revs up the CPU for 1-2 minutes before the program resumes. Is this normal behavior? I'm on a Mac Pro 2010 with OS X 10.11.2.

I updated mlvfs and FUSE to the latest version a couple days ago. I think something changed? Although performance has always been a little slow. For instance every time I want to modify source settings of an MLVFS image sequence (open up ACR dialog in AE for instance) I have to wait about 60 seconds until the window appears.

I know it's not a hard drive latency issue because it's on an external RAID that blackmagic disk speed test gives a 240MB/sec read speed. I've also accessed these files before from an external USB 3 drive and performance was actually better then. (!)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on January 29, 2016, 06:21:38 PM
Hey bwinter88,

     This might seem silly, but I think it's worth trying. Try uninstalling Fuse and MLVFS, and reinstalling. But, use the versions that worked before you experienced crashes. To be honest, I've had to downgrade from El Capitan to Mavericks because I was experiencing so many crashes. It works a lot better. But, uninstalling and reinstalling the previous versions of FUSE and MLVFS might just do the trick. I hope this helps!

- Paul Harwood
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: aschille84 on February 02, 2016, 10:04:09 PM
Is MLVFS resizing 1280 footage in DR12? Seems like I have to stretch the footage or is there any other way to do it?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: aschille84 on February 02, 2016, 10:20:24 PM
Hi johannsebastianbach, maybe you know something about this?

Anyway, found a way. In "Input scaling presets" in project settings, chose "stretch to fit corners", and voila! It stretches to the timeline resolution.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johannsebastianbach on February 03, 2016, 01:15:18 AM
Interesting, thanks!!


At this point, I would like to ask you something guys:

I read quite often in this forum (also here: http://www.magiclantern.fm/forum/index.php?topic=16542.msg161275#msg161275) and also have myself the problem, that even with global draw off, the maximum performance I can do with MLV's is 1920x490 on 50p. Everything above is making me ocasional (60p) and extremely often (GD or higher res.) pink frames which are ruining the clips. RAW Module on the other hand works flawlessly even with 67fps!

While I shoot slow motion most of the time and without sound anyway I don't need MLV, but there is a huge advantage - it can be opened by MLVFS for my so beloved super space-saving workflow. It really makes a difference, if you have a 1 Terabyte project going on and don't have to copy your footage to CDNG's before formatting to ProRes for example. I even tried renaming footage.RAW to footage.MLV and opening it with MLVFS hehehe Would it somehow be possible to do so :)?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 03, 2016, 02:56:29 PM
I won't support the RAW format as is. I don't believe it should be used in its current state for anything other than development purposes. However, MLVFS is open source, so you are free to modify it as you see fit and add support for whatever format yourself.

My goals and intentions for working on this project are probably at lot less pragmatic than most general users (who simply want something that works and gets the job done). My goal is to help drive the project forward: improving the quality of the code, increasing my understanding of ML and the internals of the underlying cameras (and helping others do the same). By not supporting the RAW format, I hope to encourage users not to take the easy way out, and actually try and solve the problem and move the project forward.

Faced with your problem, which of the two following paths to a solution do you think helps to move the project forward?
1. Just use the old raw_rec
2. Investigate and fix the pink frame issue with mlv_rec or propose better, alternate solutions




Since it does seem that there is a real resource constraint when using mlv_rec at high FPS or while using CPU intensive ML overlays, perhaps there is something that can be done without incurring any extra overhead that solves the following issues I have with the old RAW format:
1. The file extension should not be .RAW (this is way too generic, and users think they can open it with something like photoshop), it should be something unique
2. File header information and metadata should be stored at the beginning of the file, not at the end
3. There should be more metadata than is currently provided (at the very least, camera model, ML version, etc.)
4. There should be header information in every file if it is split into chunks

I see two possible solutions to address these issues:
1. Create a whole new file format
2. Create a special extension to the MLV format that allows us to write the extra metadata and whatnot we need at the beginning of the file (using existing defined MLV block structure), and then after that simply dump frames sequentially into the file like the raw format (maybe we need a different file extension if we do this?).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 03, 2016, 04:12:47 PM
Quote2. Create a special extension to the MLV format that allows us to write the extra metadata and whatnot we need at the beginning of the file (using existing defined MLV block structure), and then after that simply dump frames sequentially into the file like the raw format (maybe we need a different file extension if we do this?).

+1
And if this could be used straight in MLVFS it would be great.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 06, 2016, 05:13:09 AM
Exciting news for Windows users: https://bitbucket.org/dmilligan/mlvfs/issues/7/dokany-support

We're also close to having the focus pixel issue solved. New builds soon.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on February 06, 2016, 06:31:07 AM
Hello everyone,

     I have had some issues today with handling my .mlv files with MLVFS. I am running OS X Fuse 2.8.3.

      For some reason, After Effects CC 2015 (Version 13.7 - most recent) is not communicating well with my Macbook Pro, Late 2013, with a Nvidia GeForce GT 750M Graphics Card, and MLVFS.

     I removed OS X Fuse in System Preferences, and reinstalled, but I still had the same issue.

     (A few weeks ago I had downgraded from OS X El Capitan 10.11.3 to OS X Mavericks 10.9.5. The main reasons were because of many crashes, force quits, and Nvidia Cuda wasn't communicating with Premiere Pro CC 2015 when editing. - wasn't showing effects/dissolves, and I was forced to use OpenCL by OS X El Capitan. So I had downgraded to Mavericks since when I bought my Macbook Pro that was the operating system that was installed. Apparently the operating system the computer is initially installed with works the best, from what I've read.

    So after still having After Effects freeze with MLVFS, I called Adobe, and they told me the recent After Effects updates require OS X Yosemite 10.10.5. (This is odd since last week they told me Mavericks 10.9.5 was the minimal required version.)

     This afternoon I did a clone of my internal drive, made a bootable OS X Yosemite drive with Disk Maker X, did a clean install of 10.10.5, and installed my programs and files from the clone I made of my internal drive.

     After all of this, I still had the same exact issue, now with OS X Yosemite 10.10.5.

     So I called Adobe, and they told me I might need to convert the dng files to TIFs, and then import those. This completely disregards the point of MLVFS and an online directory's convenience. I have done this in the past, and it is quite time consuming, and takes up a lot of hard drive space.

    I asked Adobe if a clean install of Adobe Creative Cloud and the programs would help. They said it was worth a try.

    So, I uninstalled all Adobe applications and the Creative Cloud Desktop Manager. I couldn't get the desktop manager to reinstall, so I had to reinstall OS X Yosemite 10.10.5.

    I now have all my Adobe programs working and Creative Cloud Desktop Manager as well. However, I am still having After Effects give the spinning beach ball with MLVFS.

   
    The only other possible reasons I can think of that may have caused MLVFS to act odd are:

    Using OnyX software to clean my computer, possibly deleting important files (prior to this morning).

    The folder's name was changed and moving it back and forth between the internal drive and a G Drive Thunderbolt/USB 3.0, possibly confused files.

    But what's weird is, MLVFS loads the files rather quickly. Google Chrome and Safari are pretty equal in speed, so using one over the other wasn't the issue. But I cannot copy the MLVFS files and put them on another hard drive now, which I could do before (I can but it says it will take 18 hours for 64 gb, which makes no sense).

    I'm using Bad Pixel Fix at Aggressive, Vertical Lines Fix On, and Fix Noise Pattern On.

    Bottom line, something's not right. If anyone has any input on what I can do to make MLVFS and After Effects run smoother I would greatly appreciate it. I apologize if what I typed is a bit confusing, I tried to make it as clear as possible. I just wanted to make sure I included everything I tried today in order to pinpoint what might be causing the problem. Thanks so much.

- Paul Harwood
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on February 06, 2016, 01:49:34 PM
Pattern noise fix is still somewhat experimental and VERY slow. The bad pixel fix will slow things down a little too.

Bad pixel fix and focus pixel fix are two different things. Focus pixel fix is pretty fast and is default to on for cameras that need it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on February 06, 2016, 03:38:33 PM
Hey dmilligan,

     I changed the settings on MLVFS to just Vertical Stripes fix, and everything is so much faster now! Thank you so much, I'm so relieved!

- Paul Harwood
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andy kh on February 06, 2016, 11:28:58 PM
very exciting as a window user
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: hjfilmspeed on February 07, 2016, 01:41:58 AM
I've been using on windows without a hitch. I had some issues with ACR but that's it. I have been mounting the MLVs right on the card, then compressing them with slimraw raw right from the mounted DNGs! So basically I now turn my transfer stage into a convert to DNG, compress, and transfer to hard drive all it one step! Saves soooooo much time!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 07, 2016, 04:38:55 AM
The latest MLVFS is now available for Windows!

Thanks to g3gg0 and the dokany (https://github.com/dokan-dev/dokany) project, MLVFS now works on Windows just like the Mac/Linux FUSE version.

Download: MLVFS_x86.zip (https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS_x86.zip)
You'll need to install Dokany (https://github.com/dokan-dev/dokany/releases/download/v0.8.0/DokanInstall_0.8.0.exe)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andy kh on February 07, 2016, 05:21:03 AM
thank you so much@dmilligan & g3ggo for this wonderful mlvfs. love it
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on February 07, 2016, 08:32:29 AM
Cool!  :D Need to check this out. Hot pixel fix plus pattern noise reduction avaliable for windows now?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 07, 2016, 08:42:13 AM
Am I the first one who really tried?
Dokany installed. I see mounter.exe process launched from Dokany install dir.
Here is my attempts to launch mlvfs:

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\VOLD>D:\_soft\4\MLVFS\MLVFS_x86\mlvfs.exe
MLVFS: no mlv path specified


MLVFS options:
    --mlv-dir=%s
    --port=%s
    --resolve-naming
    --cs2x2
    --cs3x3
    --cs5x5
    --bad-pix
    --really-bad-pix
    --stripes
    --dual-iso-preview
    --dual-iso
    --amaze-edge
    --mean23
    --no-alias-map
    --alias-map
    --prefetch=%d
    --fps=%f
    --deflicker=%d
    --fix-pattern-noise
    --version


C:\Users\VOLD>D:\_soft\4\MLVFS\MLVFS_x86\mlvfs.exe Z:\MLVFS --mlv-dir=Z:\out

C:\Users\VOLD>D:\_soft\4\MLVFS\MLVFS_x86\mlvfs.exe Z:\MLVFS
MLVFS: no mlv path specified


MLVFS options:
    --mlv-dir=%s
    --port=%s
    --resolve-naming
    --cs2x2
    --cs3x3
    --cs5x5
    --bad-pix
    --really-bad-p --dual-iso-preview
    --dual-iso
    --amaze-edge
    --mean23
    --no-alias-map
    --alias-map
    --prefetch=%d
    --fps=%f
    --deflicker=%d
    --fix-pattern-noise
    --version


C:\Users\VOLD>D:\_soft\4\MLVFS\MLVFS_x86\mlvfs.exe Z:\MLVFS --mlv-dir=Z:\out

C:\Users\VOLD>

If I try to start the exe with all requered params it just exits silently:(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on February 07, 2016, 09:00:11 AM
Is the MLVFS folder at the path and environment variables (http://www.computerhope.com/issues/ch000549.htm)?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 07, 2016, 09:14:38 AM
@surami: It makes me wonder, why, but I tried with no difference. Did you get it working this way? If no, please, don't advice just guessing.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: surami on February 07, 2016, 09:58:30 AM
@mothaibaphoto: Sorry, I just guessed. I can't get it working till now, let's wait for a pro...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 07, 2016, 10:07:48 AM
mlvfs <mount point> --mlv_dir=<directory with MLV files>
for windows there is only the possibility to mount as drive, so the path must be a drive letter

e.g.
mlvfs x: --mlv-dir=d:\raw
to mount on X:


edit:
clarified my post, didnt see that you've tried to mount into a dir.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 07, 2016, 10:53:12 AM
Great, it works!!! Thanks to g3gg0, MLVFS for us, windowsers just around the corner !!!
Issues:
1. I can browse with explorer, but Bridge prompts that there is no free space on disc. If enter to folder with Bridge anyway, mlvfs.exe crashes:
Problem signature:
  Problem Event Name: APPCRASH
  Application Name: mlvfs.exe
  Application Version: 0.0.0.0
  Application Timestamp: 56b6ad0b
  Fault Module Name: VCRUNTIME140.dll
  Fault Module Version: 14.0.23026.0
  Fault Module Timestamp: 558ce3d9
  Exception Code: c0000005
  Exception Offset: 0000d960
  OS Version: 6.3.9600.2.0.0.256.4
  Locale ID: 1049
  Additional Information 1: 5861
  Additional Information 2: 5861822e1919d7c014bbb064c64908b2
  Additional Information 3: a10f
  Additional Information 4: a10ff7d2bb2516fdc753f9c34fc3b069

And Resolve crashes mlvfs.exe too, without promps.

2. If I try to open http://localhost:8000/ I get messages in cmd window with started mlvfs:
Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\VOLD>D:\_soft\4\MLVFS\MLVFS_x86\mlvfs.exe X: --mlv-dir=Z:\out
webgui: scanning Z:\out/...
load_resource: fopen error
webgui: scanning Z:\out/favicon.ico...
load_resource: fopen error
webgui: scanning Z:\out/favicon.ico...
load_resource: fopen error

Page in browser stays empty.
I tried AE too: it imports and renders without problems...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 07, 2016, 12:03:55 PM
For windows version Web based GUI works but strangely all fields in "Configuration Options" are empty or not checked. If you try to check any radio button the "Configuration Options" table looses it's last three lines (e.g. "Interpolation" and below, same in firefox, internet explorer and microsoft edge). Refreshing page gives all blank fields again. MLV files info is working, previews are working as well.

So the paratemets can not be changed from the web I guess. Did not try from CLI switches though.

Thanx for the great solution for ml raw editing :)
bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 07, 2016, 12:10:54 PM
@mothaibaphoto:
i am using it on a folder with a dozen of faulty, old, new and weird MLVs without any crash or (unexpected) error.
but i am only using the windows explorer and picture viewer.

can you nail it down to either a specific MLV file that causes trouble or the tool that raises that errors?
maybe its a combination of two things, idk

you can add the option -d to mlvfs to show a debug log, please post/upload one when it's crashing.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 07, 2016, 12:36:37 PM
It's a fresh and correct MLVs shot with last MKIII build, non-crop mode, nothing special. Windows explorer and even Adobe AE works flawlessly for me too. Here is what happens with Bridge:
        Attributes: 0 (0x0)
        Options: 96 (0x60)
CreateFile status = 0 - lastError = 2
###GetFileInfo 13838
GetFileInfo : \M05-1619.MLV\M05-1619_000005.dng
        result =  0
        FileStandardInformation
###Read 13838
ReadFile : \M05-1619.MLV\M05-1619_000005.dng from 4210688 len 2048
###Cleanup 13838
Cleanup: \M05-1619.MLV\M05-1619_000005.dng

###Close 13838
Close: \M05-1619.MLV\M05-1619_000005.dng

###Read 13834
ReadFile : \M05-1619.MLV\M05-1619_000005.dng from 65536 len 1040384
###Read 13834
ReadFile : \M05-1619.MLV\M05-1619_000005.dng from 1105920 len 1040384
###Read 13834
ReadFile : \M05-1619.MLV\M05-1619_000005.dng from 2146304 len 1040384
###Read 13834
ReadFile : \M05-1619.MLV\M05-1619_000005.dng from 3186688 len 1026048
###Create 13839
CreateFile : \M05-1619.MLV\M05-1619_000005.dng
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
CreateFile status = 0 - lastError = 18
###GetFileInfo 13839
GetFileInfo : \M05-1619.MLV\M05-1619_000005.dng
        result =  0
        FileBasicInformation
###Cleanup 13839
Cleanup: \M05-1619.MLV\M05-1619_000005.dng

###Close 13839
Close: \M05-1619.MLV\M05-1619_000005.dng

###Create 13840
CreateFile : \M05-1619.MLV\M05-1619_000005.dng
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
CreateFile status = 0 - lastError = 3
###GetFileInfo 13840
GetFileInfo : \M05-1619.MLV\M05-1619_000005.dng
        result =  0
        FileNetworkOpenInformation
###Cleanup 13840
Cleanup: \M05-1619.MLV\M05-1619_000005.dng

###Close 13840
Close: \M05-1619.MLV\M05-1619_000005.dng

###Create 13841
CreateFile : \M05-1619.MLV\M05-1619_000005.xmp
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
CreateFile status = 1073741772 - lastError = 3
###GetFileInfo 13841
GetFileInfo : \M05-1619.MLV\M05-1619_000005.xmp
        result =  3221225524
###Cleanup 13841
Cleanup: \M05-1619.MLV\M05-1619_000005.xmp

###Close 13841
Close: \M05-1619.MLV\M05-1619_000005.xmp

###Create 13842
CreateFile : \M05-1619.MLV\M05-1619_000005.dng
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
CreateFile status = 0 - lastError = 3
###GetFileInfo 13842
GetFileInfo : \M05-1619.MLV\M05-1619_000005.dng
        result =  0
        FileBasicInformation
###Cleanup 13842
Cleanup: \M05-1619.MLV\M05-1619_000005.dng

###Close 13842
Close: \M05-1619.MLV\M05-1619_000005.dng

###Create 13843
CreateFile : \M05-1619.MLV\M05-1619_000005.dng
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
CreateFile status = 0 - lastError = 2
###GetFileInfo 13843
GetFileInfo : \M05-1619.MLV\M05-1619_000005.dng
        result =  0
        FileBasicInformation
###Cleanup 13843
Cleanup: \M05-1619.MLV\M05-1619_000005.dng

###Close 13843
Close: \M05-1619.MLV\M05-1619_000005.dng

###Create 13844
CreateFile : \
        DesiredAccess: SYNCHRONIZE|FILE_READ_ATTRIBUTES|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 32 (0x20)
CreateFile status = 0 - lastError = 0
###GetFileInfo 13844
GetFileInfo : \
        result =  0
  unknown type:55
###FindFiles 13844
FindFiles :\
index from 0
FileMatch? : . (M05-1619.MLV,0,0)
FileMatch? : .. (M05-1619.MLV,0,0)
FileMatch? : M05-1617.MLV (M05-1619.MLV,0,0)
FileMatch? : M05-1619.MLV (M05-1619.MLV,0,0)
  =>return single entry
index to 1
###Cleanup 13844
Cleanup: \

###Close 13844
Close: \

###Create 13845
CreateFile : \M05-1619.MLV
        DesiredAccess: SYNCHRONIZE|FILE_READ_ATTRIBUTES|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 32 (0x20)
CreateFile status = 0 - lastError = 2
###GetFileInfo 13845
GetFileInfo : \M05-1619.MLV
        result =  0
  unknown type:55
###FindFiles 13845
FindFiles :\M05-1619.MLV
index from 0
FileMatch? : . (M05-1619_000006.dng,0,0)
FileMatch? : .. (M05-1619_000006.dng,0,0)
FileMatch? : M05-1619.wav (M05-1619_000006.dng,0,0)
FileMatch? : M05-1619.log (M05-1619_000006.dng,0,0)
FileMatch? : M05-1619_000000.dng (M05-1619_000006.dng,0,0)
FileMatch? : M05-1619_000001.dng (M05-1619_000006.dng,0,0)
FileMatch? : M05-1619_000002.dng (M05-1619_000006.dng,0,0)
FileMatch? : M05-1619_000003.dng (M05-1619_000006.dng,0,0)
FileMatch? : M05-1619_000004.dng (M05-1619_000006.dng,0,0)
FileMatch? : M05-1619_000005.dng (M05-1619_000006.dng,0,0)
FileMatch? : M05-1619_000006.dng (M05-1619_000006.dng,0,0)
  =>return single entry
index to 1
###Cleanup 13845
Cleanup: \M05-1619.MLV

###Close 13845
Close: \M05-1619.MLV

###Create 13846
CreateFile : \
        DesiredAccess: SYNCHRONIZE|FILE_READ_ATTRIBUTES|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 32 (0x20)
CreateFile status = 0 - lastError = 0
###GetFileInfo 13846
GetFileInfo : \
        result =  0
  unknown type:55
###FindFiles 13846
FindFiles :\
index from 0
FileMatch? : . (M05-1619.MLV,0,0)
FileMatch? : .. (M05-1619.MLV,0,0)
FileMatch? : M05-1617.MLV (M05-1619.MLV,0,0)
FileMatch? : M05-1619.MLV (M05-1619.MLV,0,0)
  =>return single entry
index to 1
###Cleanup 13846
Cleanup: \

###Close 13846
Close: \

###Create 13847
CreateFile : \M05-1619.MLV
        DesiredAccess: SYNCHRONIZE|FILE_READ_ATTRIBUTES|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 32 (0x20)
CreateFile status = 0 - lastError = 2
###GetFileInfo 13847
GetFileInfo : \M05-1619.MLV
        result =  0
  unknown type:55
###FindFiles 13847
FindFiles :\M05-1619.MLV
build_index: fopen error: Too many open files
###Create 13848
CreateFile : \M05-1619.MLV\M05-1619_000005.dng
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
CreateFile status = 0 - lastError = 3
###GetFileInfo 13848
GetFileInfo : \M05-1619.MLV\M05-1619_000005.dng
        result =  0
        FileBasicInformation
###Cleanup 13848
Cleanup: \M05-1619.MLV\M05-1619_000005.dng

###Create 13849
CreateFile : \M05-1619.MLV\M05-1619_000005.dng
###Close 13848
        DesiredAccess: Close: \M05-1619.MLV\M05-1619_000005.dng

FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
CreateFile status = 0 - lastError = 18
###GetFileInfo 13849
GetFileInfo : \M05-1619.MLV\M05-1619_000005.dng
        result =  0
        FileNetworkOpenInformation
###Cleanup 13849
Cleanup: \M05-1619.MLV\M05-1619_000005.dng

###Create 13850
###Close 13849
CreateFile : \M05-1619.MLV\M05-1619_000005.xmp
Close: \M05-1619.MLV\M05-1619_000005.dng

        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
CreateFile status = 1073741772 - lastError = 3
###GetFileInfo 13850
GetFileInfo : \M05-1619.MLV\M05-1619_000005.xmp
        result =  3221225524
###Cleanup 13850
Cleanup: \M05-1619.MLV\M05-1619_000005.xmp

###Close 13850
Close: \M05-1619.MLV\M05-1619_000005.xmp

ReturnedLength 0
ReturnedLength 0
ReturnedLength 0
ReturnedLength 0
ReturnedLength 0
###Cleanup 13834
Cleanup: \M05-1619.MLV\M05-1619_000005.dng

ReturnedLength 0
ReturnedLength 0
Ioctl failed for wait with code 55.
Ioctl failed for wait with code 55.
Ioctl failed for wait with code 55.
Thread will be terminated
Ioctl failed for wait with code 55.
Thread will be terminated
Dokan Error: Ioctl failed with code 55
Ioctl failed for wait with code 55.
Thread will be terminated
Thread will be terminated
Thread will be terminated
Ioctl failed for wait with code 55.
ReturnedLength 0
Ioctl failed for wait with code 55.
Thread will be terminated
Thread will be terminated
Ioctl failed for wait with code 55.
Ioctl failed for wait with code 55.
Thread will be terminated
Thread will be terminated

C:\Users\VOLD>


And with Resolve:

FileMatch? : M05-1617_000572.dng (_PREVIEW.gif,0,0)
FileMatch? : M05-1617_000573.dng (_PREVIEW.gif,0,0)
FileMatch? : M05-1617_000574.dng (_PREVIEW.gif,0,0)
FileMatch? : M05-1617_000575.dng (_PREVIEW.gif,0,0)
FileMatch? : M05-1617_000576.dng (_PREVIEW.gif,0,0)
FileMatch? : M05-1617_000577.dng (_PREVIEW.gif,0,0)
FileMatch? : _PREVIEW.gif (_PREVIEW.gif,0,0)
  =>return single entry
index to 1
###Cleanup 0173
Cleanup: \M05-1617.MLV

###Close 0173
Close: \M05-1617.MLV

###Create 0174
CreateFile : \M05-1617.MLV
        DesiredAccess: SYNCHRONIZE|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 33 (0x21)
CreateFile status = 0 - lastError = 2
###FindFiles 0174
FindFiles :\M05-1617.MLV
index from 0
FileMatch? : . (M05-1617_000000.dng,0,0)
FileMatch? : .. (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617.wav (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617.log (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617_000000.dng (M05-1617_000000.dng,0,0)
  =>return single entry
index to 1
###Cleanup 0174
Cleanup: \M05-1617.MLV

###Close 0174
Close: \M05-1617.MLV

###Create 0175
CreateFile : \M05-1617.MLV\M05-1617_000000.dng
        DesiredAccess: FILE_GENERIC_READ
        ShareAccess: FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 128 (0x80)
        Options: 96 (0x60)
CreateFile status = 0 - lastError = 2
###Read 0175
ReadFile : \M05-1617.MLV\M05-1617_000000.dng from 0 len 4212736
###Cleanup 0175
Cleanup: \M05-1617.MLV\M05-1617_000000.dng

###Close 0175
###Create 0176
CreateFile : \M05-1617.MLV
        DesiredAccess: SYNCHRONIZE|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 33 (0x21)
Close: \M05-1617.MLV\M05-1617_000000.dng

CreateFile status = 0 - lastError = 2
###FindFiles 0176
FindFiles :\M05-1617.MLV
index from 0
FileMatch? : . (M05-1617_000000.dng,0,0)
FileMatch? : .. (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617.wav (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617.log (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617_000000.dng (M05-1617_000000.dng,0,0)
  =>return single entry
index to 1
###Cleanup 0176
Cleanup: \M05-1617.MLV

###Close 0176
Close: \M05-1617.MLV

###Create 0177
CreateFile : \M05-1617.MLV
        DesiredAccess: SYNCHRONIZE|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 33 (0x21)
CreateFile status = 0 - lastError = 2
###FindFiles 0177
FindFiles :\M05-1617.MLV
index from 0
FileMatch? : . (M05-1617_000000.dng,0,0)
FileMatch? : .. (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617.wav (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617.log (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617_000000.dng (M05-1617_000000.dng,0,0)
  =>return single entry
index to 1
###Cleanup 0177
Cleanup: \M05-1617.MLV

###Create 0178
CreateFile : \M05-1617.MLV\M05-1617_000000.dng
        DesiredAccess: FILE_GENERIC_READ
        ShareAccess: FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 128 (0x80)
        Options: 96 (0x60)
###Close 0177
Close: \M05-1617.MLV

CreateFile status = 0 - lastError = 2
###Read 0178
ReadFile : \M05-1617.MLV\M05-1617_000000.dng from 0 len 4212736
###Cleanup 0178
Cleanup: \M05-1617.MLV\M05-1617_000000.dng

###Close 0178
###Create 0179
CreateFile : \M05-1617.MLV
        DesiredAccess: SYNCHRONIZE|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 33 (0x21)
Close: \M05-1617.MLV\M05-1617_000000.dng

CreateFile status = 0 - lastError = 2
###FindFiles 0179
FindFiles :\M05-1617.MLV
index from 0
FileMatch? : . (M05-1617_000000.dng,0,0)
FileMatch? : .. (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617.wav (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617.log (M05-1617_000000.dng,0,0)
FileMatch? : M05-1617_000000.dng (M05-1617_000000.dng,0,0)
  =>return single entry
index to 1
###Cleanup 0179
Cleanup: \M05-1617.MLV

###Close 0179
Close: \M05-1617.MLV

###Create 0180
CreateFile : \M05-1617.MLV\M05-1617.wav
        DesiredAccess: FILE_GENERIC_READ
        ShareAccess: FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 128 (0x80)
        Options: 96 (0x60)
CreateFile status = 0 - lastError = 0
###GetFileInfo 0180
GetFileInfo : \M05-1617.MLV\M05-1617.wav
        result =  0
        FileStandardInformation
###Read 0180
ReadFile : \M05-1617.MLV\M05-1617.wav from 0 len 4096
###Read 0180
ReadFile : \M05-1617.MLV\M05-1617.wav from 630 len 512

C:\Users\VOLD>
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 07, 2016, 02:14:11 PM
@bouncyball

What browser? IE? I noticed this too, there's something wrong with the jquery that reads the initial values, it still works to set the values of options though. The last three options are actually supposed to be hidden unless dual ISO is turned on, since they only apply to dual ISO.

Does it work with other browsers?

@g3gg0
I actually managed to mount into an empty directory (Win7 x86 VM)

@mothaibaphoto
You need to start MLVFS from the directory it's in (cd to mlvfs.exe dir before launching it) so it can find it's support files. That's why the webgui is blank.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 07, 2016, 03:20:05 PM
Quote from: dmilligan on February 07, 2016, 02:14:11 PM
@mothaibaphoto
You need to start MLVFS from the directory it's in (cd to mlvfs.exe dir before launching it) so it can find it's support files. That's why the webgui is blank.
I suspect I did something dumb :)
Ok, now webgui behaves exactly as @bouncyball described with Firefox and MSIE.
Command-line parameters seems works - I successfully tested --stripes on file with vertical stripes.
It's possible to open files one by one.
Attempt to select all files and copy to another disc - crashes mlvfs.exe, such as Bridge and Resolve.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 07, 2016, 03:40:35 PM
@dmilligan:
repeated copying several times, worked properly for a while. selectively copying files is fine.

after maybe about 2k DNGs i got:
build_index: fopen error: Too many open files

so there is something troublesome indeed.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 07, 2016, 03:54:26 PM
ok think i've found the reason.
it doesnt always close chunks. prepared a patch.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 07, 2016, 05:59:19 PM
@dmilligan:
https://bitbucket.org/g3gg0/mlvfs/commits/48aa2f6f7434df7bdf05b33b33d1e6fe5621b8fb
https://bitbucket.org/g3gg0/mlvfs/commits/a6602a4b9cf649cfa58db3bc1db6e86946430e9c

those two address some issues that cause the mlvfs.exe to crash.
e.g. the .gif reading crashed in a reproducable manner when offsets were beyond end and debugging was enabled.
so i also sanitized parameters in .dng reading.

it also adds a lot of checks to malloc'ed areas to play safe.
i hope it will fix the reported problems.


still i am wondering how it will free allocated image buffers.
the memory footprint raises slowly but continuously when copying files.
memory is allocated via get_or_create_image_buffer and never free'd.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 07, 2016, 05:59:59 PM
ah shiiit. VS doesnt take the indentation the project uses by default...
will fix.

thats the fixed version:
https://bitbucket.org/g3gg0/mlvfs/commits/594552cf4727b1f6acc291189565062de1b59b0d
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 07, 2016, 06:24:41 PM
for those who want to try: mlvfs.exe (http://magiclantern.fm/modules/modules/mlvfs.exe/mlvfs.exe)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 07, 2016, 06:26:09 PM
@dmilligan
the problem shows up in every browser: firefox, msie, chrome, microsoft edge.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 07, 2016, 06:31:44 PM
@g3gg0

Image buffers should be getting marked when the file is closed and then disposed when new image buffers are needed. I recently changed some of this behavior so there may be some issues. All this code is rather convoluted b/c we try to be efficient as possible. For example if some program opens a file, reads part of it, closes it and then opens it again, we don't want to have to reprocess the frame, so we try to to be smart about keeping processed frames around for a bit after being closed. MLVFS was much simpler and efficient when we could just process the requested section of the file, but now we have processing options that need to process the whole frame at once (decompression, dual ISO, etc).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 07, 2016, 06:34:27 PM
yeah, makes sense and fits to what ive seen.

proposal:
check if image buffers that are already allocated > x MiB and free the oldest ones to get < y MiB
every buffer has an age field (timestamp of the last access using time()? or some counter?)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 07, 2016, 06:48:57 PM
With last exe browsed some folders with Bridge(!!!), than run Resolve ... and get smiley bsod never seen before...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 07, 2016, 10:03:42 PM
no troubles here with LR :(

but when stumbling through the code, i found another thing that can be troublesome.
can you redownload the mlvfs.exe from the last link and try if its better?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 08, 2016, 12:46:14 AM
Quote from: g3gg0 on February 07, 2016, 06:34:27 PM
proposal:
check if image buffers that are already allocated > x MiB and free the oldest ones to get < y MiB
every buffer has an age field (timestamp of the last access using time()? or some counter?)
I've cleaned up and simplified that code quite a bit. Since its a linked list we just delete from the beginning of the list and add on to the end, so the oldest buffers are always at the beginning. Haven't verified that there aren't any leaks, so there could still be some, or leaks from other places.

Just curious, which version of dokany are you compiling against? I was using 0.8.0 and provided links for that version. Just want to make sure we're on the same page.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 08, 2016, 01:04:56 AM
yeah it's the oldest one, but is it also the LRU one? hehe.

right, i am using 0.8.0
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on February 08, 2016, 01:46:05 AM
Yeah I know, but vast majority of the time programs access frame sequentially. I think KISS applies.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 08, 2016, 01:46:18 AM
ok ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 08, 2016, 05:08:12 AM
new builds uploaded
* fixed the webgui issue on windows (windows file paths contain "\" and json doesn't like that)
* streamlined buffer management (and eliminated 'prefetch' option)
* lots of stability improvements from g3gg0


still todo: embed git version info on the Windows version like the Mac/Linux version
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 08, 2016, 12:54:55 PM
Last mlvfs.exe with Resolve:

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\VOLD>cd /d D:\_soft\4\MLVFS\MLVFS_x86

D:\_soft\4\MLVFS\MLVFS_x86>mlvfs.exe X: --mlv-dir=Z:\out
mlvfs_wrap_read(1426): caught exception 0xC0000005

D:\_soft\4\MLVFS\MLVFS_x86>

Resolve can be downloaded here (https://www.blackmagicdesign.com/products/davinciresolve)
Start with any new project, Media Storage - try to navigate to any MLVFS folder - immediate crash of mlvfs.exe :(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 08, 2016, 01:47:43 PM
Hi Again,

Problem 1
E:\MLVFS>mlvfs E:\DNG --mlv-dir=E:\MLV
mlvfs_wrap_read(1440): caught exception 0xC0000005
mlvfs_wrap_read(1440): caught exception 0xC0000005

no difference if you mount it to drive letter or empty dir.
exceptions mostly caught when accessing WAV file. Wave shows correct size but I can not copy it. However I could copy it with the previous mlvfs version.

Problem 2
It seams that MLVs bigger than 2048mb (also MLVs + M00...) can not be mounted at all?! they are not shown on dir list. Checked all available versions of mlvfs.

OS: Windows 10 Pro x64, Dokan 0.8 (checked Dokan 1.0-beta and it shows the same behavior)

@dmilligan: Webui works. BTW it lists every MLV even the >2gb ones which are not mounted (seen) in file explorer.

regards
bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 08, 2016, 03:10:20 PM
please redownload mlvfs.exe (http://magiclantern.fm/modules/modules/mlvfs.exe/mlvfs.exe) and try.
it will still crash but show an address that might help to find the exact crash position.

meanwhile i am downloading resolve.
guess it's some off-by-one-error or such with the wave decoder that causes the troubles.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 08, 2016, 03:20:18 PM
No output to CMD window, problem details from popup:

Problem signature:
  Problem Event Name: APPCRASH
  Application Name: mlvfs.exe
  Application Version: 0.0.0.0
  Application Timestamp: 56b89cfa
  Fault Module Name: StackHash_8e87
  Fault Module Version: 6.3.9600.17415
  Fault Module Timestamp: 54504b0d
  Exception Code: c0000374
  Exception Offset: PCH_80_FROM_ntdll+0x0003CEEC
  OS Version: 6.3.9600.2.0.0.256.4
  Locale ID: 1049
  Additional Information 1: 8e87
  Additional Information 2: 8e87aeb06e784aea107beb1d064656d5
  Additional Information 3: dc1c
  Additional Information 4: dc1c75fae54762c904db111a92126c62
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 08, 2016, 04:13:33 PM
Just accessing dir or accesing wav:
E:\MLVFS>mlvfs_debug E:\DNG --mlv-dir=E:\MLV
mlvfs_wrap_read(1440): caught exception 0xC0000005 at address 0x739AE21C

Sometimes (very rearly) DokanMounter service stops and needs to be started manually.
E:\MLVFS>mlvfs_debug E:\DNG --mlv-dir=E:\MLV
Unmount

!!! Checked on 3 PCs MLVs >2GB are not mountig at all and can not be seen in file explorer (win10prox64, unfortunatelly have no win7 or 8.1 to test it on) !!!

@mothaibaphoto: What ver of win do you use and do you have the problem with MLVs bigger than 2GB

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 08, 2016, 05:53:38 PM
Quote from: bouncyball on February 08, 2016, 04:13:33 PM
@mothaibaphoto: What ver of win do you use and do you have the problem with MLVs bigger than 2GB
1. win 8.1x64
2. Yes.

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\VOLD>dir X:\
Volume in drive X is DOKAN

Directory of X:\

05.02.2016  16:18    <DIR>          M05-1617.MLV
05.02.2016  16:19    <DIR>          M05-1619.MLV
05.02.2016  16:20    <DIR>          M05-1620.MLV
05.02.2016  16:21    <DIR>          M05-1621.MLV
05.02.2016  16:29    <DIR>          M05-1629.MLV
05.02.2016  16:30    <DIR>          M05-1630.MLV
05.02.2016  16:31    <DIR>          M05-1631.MLV
05.02.2016  16:36    <DIR>          M05-1636.MLV
               0 File(s)              0 bytes
               8 Dir(s)               0 bytes free

C:\Users\VOLD>dir Z:\out
Volume in drive Z is EVO850
Volume Serial Number is F873-BE21

Directory of Z:\out

08.02.2016  00:30    <DIR>          .
08.02.2016  00:30    <DIR>          ..
08.02.2016  00:30            11 836 M05-1617.IDX
05.02.2016  16:18       396 013 620 M05-1617.M00
05.02.2016  16:18     1 714 848 352 M05-1617.MLV
07.02.2016  20:54            31 216 M05-1618.IDX
05.02.2016  16:19     1 137 176 628 M05-1618.M00
05.02.2016  16:19     4 599 571 040 M05-1618.MLV
07.02.2016  17:55             1 684 M05-1619.IDX
05.02.2016  16:19        61 763 636 M05-1619.M00
05.02.2016  16:19       207 090 272 M05-1619.MLV
07.02.2016  17:55            12 088 M05-1620.IDX
05.02.2016  16:20       421 445 684 M05-1620.M00
05.02.2016  16:20     1 729 380 960 M05-1620.MLV
07.02.2016  20:54            14 248 M05-1621.IDX
05.02.2016  16:21       486 842 420 M05-1621.M00
05.02.2016  16:21     2 070 897 248 M05-1621.MLV
07.02.2016  20:54            31 384 M05-1622.IDX
05.02.2016  16:22     1 104 478 260 M05-1622.M00
05.02.2016  16:22     4 588 671 584 M05-1622.MLV
07.02.2016  20:54            23 056 M05-1623.IDX
05.02.2016  16:24       802 926 644 M05-1623.M00
05.02.2016  16:24     3 436 962 400 M05-1623.MLV
07.02.2016  20:54            60 280 M05-1624.IDX
05.02.2016  16:27     2 132 660 276 M05-1624.M00
05.02.2016  16:27     8 875 790 944 M05-1624.MLV
07.02.2016  20:54             3 568 M05-1629.IDX
05.02.2016  16:29       153 260 084 M05-1629.M00
05.02.2016  16:29       672 367 200 M05-1629.MLV
07.02.2016  20:54             2 464 M05-1630.IDX
05.02.2016  16:30        98 877 492 M05-1630.M00
05.02.2016  16:30       449 892 960 M05-1630.MLV
07.02.2016  20:54             2 944 M05-1631.IDX
05.02.2016  16:31       168 091 700 M05-1631.M00
05.02.2016  16:31       726 749 792 M05-1631.MLV
07.02.2016  20:54             2 740 M05-1636.IDX
05.02.2016  16:36        79 102 004 M05-1636.M00
05.02.2016  16:36       543 826 528 M05-1636.MLV
07.02.2016  20:54            15 832 M05-1639.IDX
05.02.2016  16:40       821 092 404 M05-1639.M00
05.02.2016  16:40     2 395 668 480 M05-1639.MLV
              39 File(s) 39 875 661 952 bytes
               2 Dir(s)  24 641 163 264 bytes free

C:\Users\VOLD>
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 08, 2016, 08:22:44 PM
WAV: nailed it down and fixing it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 08, 2016, 08:51:27 PM
please redownload mlvfs.exe (http://magiclantern.fm/modules/modules/mlvfs.exe/mlvfs.exe) and try.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 08, 2016, 09:42:55 PM
4GiB files also fixed: please redownload mlvfs.exe (http://magiclantern.fm/modules/modules/mlvfs.exe/mlvfs.exe) and try.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 09, 2016, 05:52:36 AM
Good news:
Resolve imports and plays great, with sound.
Not so good:
Big files still ignored. The directory list I posted recently looks the same.
Several consecutive refreshing of webGUI with changing parameters causes smiley BSOD. Reproduced several times - comp. restart - webGUI - attempt to save anything - BSOD.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 09, 2016, 01:39:32 PM
@g3gg0: Great! Wav plays nicely :). However, as mothaibaphoto mentioned above MLVs >2GB ignored again. Can you compile the test binary against the Dokan version 1.0-beta1? The new version gets rid of a DokanMounter service and adds windows mount manager support.

Quote from: mothaibaphoto on February 09, 2016, 05:52:36 AM
Several consecutive refreshing of webGUI with changing parameters causes smiley BSOD. Reproduced several times - comp. restart - webGUI - attempt to save anything - BSOD.

Think I nailed exact reason of it. Switching "Naming Scheme" from "Deafult" to "DaVinci Resolve" and after that accessing folders, every time, ends up with exception. Sometimes restarting DokanMounter service helps and you can remount the folder, but sometimes OS freezes and only hard reset is a solution.
Other parameters in webgui are working just fine and only "Fix Pattern Noise" slows the process down a little bit. I think It's a normal behavior.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 09, 2016, 03:53:00 PM
@bouncyball:
tried with a forged 19GiB file and that works well.
will have to do some tests what may cause that.

will try 1.0
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 09, 2016, 05:12:41 PM
Quote from: g3gg0 on February 09, 2016, 03:53:00 PM
tried with a forged 19GiB file and that works well.

hmm... will try to record some huge mlvs on exfat. From ~50 test files of mine worked none which is bigger than 2048mb. They just do not show up in mounted folder.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 09, 2016, 05:21:35 PM
@dmilligan:
What does exacty switching "Naming Scheme" from "Default" to "DaVinci Resolve"? Changes some cdng metadata in favor of resolve? Or something else?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 09, 2016, 06:26:30 PM
It changes how the DNGs that show up in the mount folder are named. If you use a particular naming convention that Resolve likes (one that mimics the naming convention used by BM cameras), then the parent folders of cDNGs show up as video clips in media browser.

It makes sense that it would be the source of issues on Windows since file naming rules are different (case insensitive, different path separator, etc.), and it's adding a layer of complexity on top of the path translations mlvfs is doing (FUSE sends mlvfs a file path, mlvfs has to figure out what data to send back).

Quote from: g3gg0 on February 09, 2016, 03:53:00 PM
forged 19GiB file
Heh, this made me think of using FUSE to generate fake, huge MLV files for testing. It could just take an existing small MLV file and basically just repeat the frame chunks over and over. Then we can have huge testing MLV files that don't waste a lot of hard drive space.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 09, 2016, 06:46:12 PM
Not trying to be the party pooper in here for Windows users ... I'm very excited for all of you and it's been a long wait!

However, I just wanted to share that I downloaded the latest MLVFS for Mac and now the web GUI won't work and how can I help troubleshoot this or at least get a log for you?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on February 09, 2016, 07:01:14 PM
Quote from: dmilligan on February 09, 2016, 06:26:30 PM
...If you use a particular naming convention that Resolve likes (one that mimics the naming convention used by BM cameras)...
Speaking of DaVinci Resolve Naming Scheme and Black Magic camera file names--

(https://farm2.staticflickr.com/1653/24291908904_546bd19854_z.jpg)

I shot this test today and the filename has a date for last month. The date is correct in the browser window.

More on DaVinci Resolve Naming Scheme--I found this information about how the file name is formatted:

QuoteFile Naming Convention
The Blackmagic Cinema Camera uses the following file naming convention when recording video.
[Camera ID]_[Reel Number]_[yyyy-mm--dd]_[hhmm]_C[Clip number].mov
This is an example of how this would appear on the touch screen of the camera:
BMC01_1_2012-08-08_1631_C0002.mov QuickTime Movie Filename
BMC01_1_2012-08-08_1631_C0002.mov Camera ID
BMC01_1_2012-08-08_1631_C0002.mov Reel Number
BMC01_1_2012-08-08_1631_C0002.mov Date (2012 Aug 08)
BMC01_1_2012-08-08_1631_C0002.mov Time (16:31pm - 24hrs)
BMC01_1_2012-08-08_1631_C0002.mov Clip Number
For CinemaDNG files, the folder of the image sequence will also be named the same way.

If you check discussions from people that are using these files in post you'll find a lot of dissatisfaction. Besides the long filename the information that editors need to see, the clip number, is at the end of the name and depending on the editing system it might be cut off.

Still, the date should be correct and it would be an improvement to add the time.

If we take an MLV file as it comes off the camera:
M02-0828.MLV

The Black Magic naming would probably go something like this:
M_02_2012-08-08_1631_C0828

I'm not suggesting such a radical change because it isn't a very practical name for most editing systems and the "M" is way too generic for a camera name but at least the fields in the name have some meaning instead of being padded out with zeros. Just something to think about.

Maybe something like this?
M02-0828_02_2012-08-08_1631_C0828
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 09, 2016, 07:08:43 PM
How is folder/audio embed working with the dng files in the case of shortening the folder name but not the dng files? The longer extension of zeroes are there to give each frame a sequential number.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 09, 2016, 07:10:32 PM
Quote from: dfort on February 09, 2016, 07:01:14 PM
I shot this test today and the filename has a date for last month. The date is correct in the browser window.
Ah yes. You found a bug :) The tm.month field is zero based (0-11), so we need to add 1 before we print it out. This is already corrected in some places, but you found a place it was wrong.

Quote from: DeafEyeJedi on February 09, 2016, 06:46:12 PM
However, I just wanted to share that I downloaded the latest MLVFS for Mac and now the web GUI won't work
I think I know what's wrong: forgot to include some resource files in the Mac build. Will fix.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 09, 2016, 08:08:31 PM
@4Gib:
(https://picload.org/thumbnail/woccpwd/largemlv.jpg) (https://picload.org/view/woccpwd/largemlv.png.html)
for me it works :-S
no clue what the problem really is.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 09, 2016, 09:45:33 PM
Quote from: DeafEyeJedi on February 09, 2016, 06:46:12 PM
However, I just wanted to share that I downloaded the latest MLVFS for Mac and now the web GUI won't work and how can I help troubleshoot this or at least get a log for you?
Quote from: dmilligan on February 09, 2016, 07:10:32 PM
I think I know what's wrong: forgot to include some resource files in the Mac build. Will fix.
Actually, everything seems fine. Web GUI works just fine here.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 09, 2016, 09:52:15 PM
@dmilligan:
get_mlv_filename needs to be reworked.
currently it's not possible to create files in directories within a MLV file.
and we have to use dokan v1 since file creation doesnt work with 0.8.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 10, 2016, 03:23:38 AM
Quote from: g3gg0 on February 09, 2016, 09:52:15 PM
and we have to use dokan v1 since file creation doesnt work with 0.8.
v1 doesn't seem to work on my win7 VM, fuse_main() just returns 1 immediately

Quote from: g3gg0 on February 09, 2016, 09:52:15 PM
get_mlv_filename needs to be reworked.
I think all the path handling should be much better now
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 10, 2016, 09:34:50 AM
oh thats odd. works fine on my win 10.

wow you did a lot of improvements. nice.

the problem with virtual folder is still persistent.
for a requested file:

/path/to/some/file.mlv/with/some/subfolder/leads/to.txt

we have to iterate over every element and check if it is a (existing!) MLV.

/path
/path/to
/path/to/some
/path/to/some/file.mlv


and cut the remaining part
/with/some/subfolder/leads/to.txt

and recombine it again with the MLD
/path/to/some/file.MLD/with/some/subfolder/leads/to.txt


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 10, 2016, 10:11:20 AM
@dmilligan, g3gg0
You really did some magic to the source code :)

In the last uploaded mlvfs (which btw is linked againts debug versions of the vclibs)
works:
1. files >2gb!!! (strange but) - YES!  I have not changed anything in my system, same Dokan v0.8
2. webgui "Naming Scheme" - YES
3. webgui all other params - YES
4. wav - YES

not working:
1. webgui gif previews in "DaVinci Resolve" naming mode
I guess something related to win paths and naming again

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 10, 2016, 10:12:48 AM
oh i am glad to read this.
then we are on a good path ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 10, 2016, 02:06:28 PM
Quote from: bouncyball on February 10, 2016, 10:11:20 AM
@dmilligan, g3gg0
You really did some magic to the source code :)

In the last uploaded mlvfs (which btw is linked againts debug versions of the vclibs)
works:
1. files >2gb!!! (strange but) - YES!  I have not changed anything in my system, same Dokan v0.8
2. webgui "Naming Scheme" - YES
3. webgui all other params - YES
4. wav - YES

not working:
1. webgui gif previews in "DaVinci Resolve" naming mode
I guess something related to win paths and naming again

bb
Redownloaded the exe after reading this and everything the same (:
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 10, 2016, 04:38:56 PM
Quote from: mothaibaphoto on February 10, 2016, 02:06:28 PM
Redownloaded the exe after reading this and everything the same (:

Have you downloaded this file?
https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS_x86.zip (https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS_x86.zip)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 10, 2016, 05:06:40 PM
Did some test copy of 4,918,485,826 bytes (1168 files) from one SSD to second SSD

result (average of 3 runs):
Dokan 1.0 - 58mb/sec, mlvfs.exe consumed ~13% of CPU (Intel Xeon E3 @ 3.4Ghz)
Dokan 0.8 - 52mb/sec, mlvfs.exe consumed ~12% of CPU

A little bit subjective and depends on hardware but v1.0 has some performance boost (maybe new fs driver optimizations?)

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 10, 2016, 05:10:11 PM
No, This one, as posted by g3gg0 (http://magiclantern.fm/modules/modules/mlvfs.exe/mlvfs.exe)
I tried this one too: it asks for vcruntime140d.dll, I find it and put to program dir, than ucrtbased.dll, than...
it just show the message "The application was umable to start correctly (0x000007b)." Click OK to close.
:(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 10, 2016, 05:16:36 PM
yes it's linked against debug version of dlls.

get 2 files below and drop to the saem dir with mlvfs.exe:
http://www.dll-files.com/dllindex/dll-files.shtml?vcruntime140d (http://www.dll-files.com/dllindex/dll-files.shtml?vcruntime140d)
http://www.dll-files.com/dllindex/dll-files.shtml?ucrtbased (http://www.dll-files.com/dllindex/dll-files.shtml?ucrtbased)

oh forgot to mention get 32bit versions
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 10, 2016, 05:19:13 PM
I find these files myself on that site, but it anyway don't run with message from my previous post.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 10, 2016, 05:22:28 PM
If you got 32bit versions and still have an error message then something messed up in your system with ms c++ redistributables.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 10, 2016, 05:29:54 PM
Ups, it seems I mixed versions, now it works and I see all the files !!!
Great thanks !!!
Edit:
I think it's meaningless to measure performance of the debug version: it definitely less. With that version Resolve plays with 24 FPS(real time) with this - 14.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 10, 2016, 06:03:56 PM
Good point. We'll see... happy bright future is almost over here! Not only for MAC users ;)

Just kidding, have 2 old hackintoshes myself. But love doing/testing video/audio things under win or linux.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 10, 2016, 06:33:15 PM
125MiB/s for 19GiB with plain old HDD and my x64 build (http://magiclantern.fm/modules/modules/mlvfs_x64_rc.exe/mlvfs_x64_rc.exe) for dokan v1.0-rc ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 10, 2016, 06:47:16 PM
Wow...
And where did you get that dokan v1.0-rc?
here:
https://github.com/dokan-dev/dokany/releases
only 1.0.0-BETA1
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 10, 2016, 06:49:27 PM
yeah sorry, its the beta1, you are right.
thats the correct one.

here some more optimized builds (http://magiclantern.fm/modules/modules/mlvfs_optimized.zip/mlvfs_optimized.zip)
choose the one that works for your CPU.

AVX2 is the best of course, but not all CPUs support it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 10, 2016, 07:44:06 PM
Congratulations, you did a great job,
Great thanks to everyone involved!!!
Now it would be perfect if some more people try this asap to localize possible issues.
I tried for now only 64 build without avx(as i have old bloomfield) -
Resolve plays 30 fps FHD project in full screen, cpu load 90%(but 75% by Resolve itself), video - 40%,
I can't estimate disk load as Task manager shows 0%(probably due to indirect access).
Edit:
And regarding that Naming Scheme - i can't reproduce anymore with 12.3 version of Resolve that "show but not play" behavior. Maybe they relaxed naming requirements.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 10, 2016, 09:19:06 PM
@bouncyball:
can you do a another comparison of the MLVFS speed with the optimized builds?
as you seem to have rea-world footage and a high end machine i think thats a good basis for comparisons

i have those variants:

x86 vs. x64  [CPU architecture]
nopar vs. par [multicore parallelization]
noSSE, SSE, SSE2 on x86 [vectoring engine]
noAVX, AVX, AVX2 on x64 [vectoring engine]

i am the most curious about the nopar/par performance boost (if any).
so if you want to do a minimum setup, it would be the best you choose x64/avx2 and compare nopar vs. par.
if the tool crashes, your CPU probably doesnt support avx2, try avx then.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on February 11, 2016, 06:05:14 AM
@g3gg0

I have ben watching this thread with interest for some time now and decided to try mlvfs.

My Gear:
---------------------------------------
CPU:            Intel i5-3317U quad core
Ram:           6GB
Graphics:     Intel HD 4000
OS:             w10 Pro 64bit updated with the most recent patches.

I did have Pismo file mount for Windows installed but uninstalled it to avoid any potential conflicts with DOKAN.

The initial installation of DOKAN (v1) displayed the following message;

     http://1drv.ms/1WfsIs3
    (note the screen shot was from v0.800 but apart from the version number in the message, the message was the same).

I clicked "YES" then proceeded to download and install MS Visual C++ - Redistributable (x64). It installed without error.

DOKAN v1 appears to have installed correctly as verified via the cmd line screenshot below.

     http://1drv.ms/1KGNvnA
     Note: version was checked before and after manually starting the Driver and network service. Given the absence of error messages I
              assume that this is working correctly.

When I try to launch MLVFS.exe I initially had errors re: vcruntime140.dll and msvcp140.dll not found yet both of these files were in the C:\Windows\System32 directory. Based on some google research I moved copies of these files into the C:\Windows\SysWOW64 directory.

Subsequent to this the "dll not Found" error messages no longer appears but the following one does.
     http://1drv.ms/1KGO0xG

I acknowledge the error and the application fails to launch. Note that I have tried to launch it at the cmd line level with the same results.

Any thoughts, ideas? Is there an architecture dependency (e.g.: graphics card) that I'm missing?

Edit add;  If I run dokanctl.exe from the C:\Program Files\Dokan\DokanLibrary directory it appears to work correctly. However,
                 If I run it from the C:\Program Files (x86)\Dokan\DokanLibrary directory, it fails with the same error as mlvfs.exe.


Might be a clue to what is going on.


Cheers... Otto
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 11, 2016, 07:55:09 AM
@Ottoga:
1. Docan install always ask to install VC Redist, whether it installed, or not. No problem to reinstall it.
2. The message you got after manually adding dll files in respond to  "dll not Found" meaning you mix versions, ie, run 32 bit exe with 64 bit dll or vice versa.
To fix everything try:
1. Discard all the dll files you copied manually.
2. Install/reinstall Docan & VC redist. No need to run anything manually.
3. Unpack Optimized builds (http://magiclantern.fm/modules/modules/mlvfs_optimized.zip/mlvfs_optimized.zip)
to directory where you previosly unpacked mlvfs.zip.
4. As you on 64 win, run 64-bit exe, mlvfs_x64_rc.exe or even mlvfs_x64_rc_avx2.exe as your processor seems support that.
Make text file with 2 next rows and save it with .bat extension:
cd /d path_to_your_mlvfs_x64_rc.exe
mlvfs_x64_rc X: --mlv-dir=path_to_your_MLV_dir
where X - the Letter for disk not yet present in your computer.
Then just run that bat file and configure MLVFS with your web browser pointing it to http://localhost:8000/.
For example, my working bat file:
cd /d D:\PortableSoft\video\bin\mlvfs_optimized
mlvfs_x64_rc X: --mlv-dir=Z:\out
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 11, 2016, 08:31:17 AM
@Ottoga:
as mothaibaphoto already wrote - MSVC runtime is a bit bitchy. never just download the DLLs.
always search for "vc_redist" packages when you miss the DLLs.
the optimized builds (which are still experimental of course) do not require the debug versions of the DLLs.

@dmilligan:
can you upload the "release" build of the package?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on February 11, 2016, 08:38:55 AM
I'm having some issues with the new Mac build. MLV previews aren't visible in the Resolve media window. Folders are visible where there used to be previews. When I go into the folders most of them look empty. I can't load any MLV files into Resolve at all.  :'(

Also when I try to refresh the browser I get a "can't connect to the server" error message.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on February 11, 2016, 08:40:42 AM
I'm running Yosemite 10.10.3, Davinci Resolve 12.0, Safari 8.06, and Fuse 2.8.2

I went back to the Feb 2015 build: all good. One thing I noticed: my MLV folder takes a few seconds to load (spinning wheel of death). There's no spinning wheel of death showing up when I try to load the MLV folder in Resolve with the new build.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on February 11, 2016, 11:32:08 AM
VC Redist issues, maybe fix it with the pack: http://forums.guru3d.com/showthread.php?t=393405
(right click run as Admin)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 11, 2016, 12:33:21 PM
@g3gg0

Machine: cpu Xeon E3-1245 @ 3.4ghz, mem 16gb, 2x 120 gig kingston ssds <- they are no highend ones but not so bad anyway :)
Folder with MLVs reside on one ssd, mounted to drive Z: -> copied to second ssd.
Copied 19,852,533,609 bytes, 4717 files, 5 folders

Without runing mlvfs:
Result of sequential copy of a large file from 1st to second gives ~220mb/sec.
Result of sequential copy of a dir with a real DNGs of the same size gives ~175mb/sec.

results are below (orange line - mlvfs cpu usage):
(http://nic.caucasus.net/mlvfs/?dir=&download=x64_avx2_nopar.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x64_avx2_par.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x64_avx_nopar.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x64_avx_par.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x64_noavx_nopar.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x64_noavx_par.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x86_nosse_nopar.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x86_nosse_par.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x86_sse2_nopar.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x86_sse2_par.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x86_sse_nopar.jpg)

(http://nic.caucasus.net/mlvfs/?dir=&download=x86_sse_par.jpg)

regards
bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 11, 2016, 12:50:22 PM
@g3gg0

I think every binary performance is about the same. Par versions are just CPU hogs (would be nice to test them in random or parralel DNG file access mode). Anyway it is nice to have native 64bit binary and avoiding WOW64 :)
Forgot to mention, test was performed with mlvfs default parameters.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 11, 2016, 03:48:25 PM
@bouncyball:
wow, nice. thanks a lot!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on February 11, 2016, 03:56:48 PM
@mothaibaphoto  @g3gg0

Thanks for your replies. I've got it working albeit I'm still driving it manually via the cmd line. I'll set-up the batch file and figure out how to point the application to my web browser after I've had some zzz's

One thought/suggestion: I've had to manually run dokanctl to unmount the mapped drive (i.e.: X) which in turn automatically closes the MLVFS application. However, if I exit the MLVFS application via say ctl/c then I'm unable to unmount the mapped drive letter via Dokanctl.

Does MLVFS monitor for a hidden/undocumented "quit" command that would send the unmount command to DOKAN and then gracefully exit MLVFS? or is that part of the web GUI that I've yet to figure out how to set up?

Other than that, fantastic job porting this to windows to all involved. I've successfully tested it with normal and FRSP mlvs. I use Cyberlink photo and video editing software and both happily imported the dngs from the mounted mlv's. one less step in the workflow. Who could complain about that.

Will play some more with the optimized builds tomorrow. mlvfs_x64_rc_avx2.exe crashes but mlvfs_x64_rc_avx.exe seems to work.

Tomorrows another day.  Thanks again gentlemen.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 11, 2016, 04:00:46 PM
ok my conclusion:
parallelization for the 12bit->16bit conversion is pointless due to probably thread start overhead.
too much CPU usage and no visible performance boost.

the vectorization. hmm also not convincing.
does anyone have a "clearly visible" performance boost with some version?
guess only x64 gives a proper speedup.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 11, 2016, 04:03:07 PM
@Ottoga:
you should be able to just run mlvfs and it will automatically mount it at the drive letter (or folder) you specified.
to unmount you simply close/ctrl-c mlvfs.exe and thats it.
30 seconds later the drive letter should disappear.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on February 11, 2016, 04:23:38 PM
@g3gg0

Thanks. Absolutely correct,  I just didn't wait long enough to observe this.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 11, 2016, 04:36:57 PM
@g3gg0: u r welcome

Just did copy same files with microsoft tool "robocopy /MT:8" 8 threads turned on. mlvfs_x64_avx2_nopar vs mlvfs_x64_avx2_par -  2:17min vs 2:14min accordingly.
Result is 3 milisecond difference ;)

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 11, 2016, 05:38:43 PM
i guess seconds? ;)
nah thats not worth it. hoped there is a lightweight multithreading implementation involved, but seems not.

can you do another try with mlvfs_x64_avx2_par.exe ?
i modified it to use OpenMP. perhaps ist a bit better.
just redownload, the file date is 11.02.2016 17:37
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 11, 2016, 07:23:58 PM
Yup, my bad 3 seconds ;)

Windows 10 standard copy:
(http://nic.caucasus.net/mlvfs/?dir=&download=x64_avx2_par_openmp.jpg)

robocopy time was 2:18. In both cases max rate was at 158-162mb/sec and min rate 75-85. Rate rise at the beginning ~30sec I guess is a cache effect then it normalizes with some spikes. There are 5 mlvs (under 1gb, under 2gb, under 3gb, under 4gb and more than 4gb). Disk usage in resource monitor shows that 3 or 4 MLV files are open simultaneously for reading by mlvfs process.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 11, 2016, 08:05:37 PM
okay thanks again.
so no parallelization method helps speeding up RAW->DNG pixel operations.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 12, 2016, 09:00:49 AM
btw. with dokan v0.8 and win10:

if you copy DNGs from the MLVFS and cancel copying (no matter when) does it crash?

or is this only a dokoan v1.0 issue?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 12, 2016, 11:54:50 AM
u r welcome again :)

Returned to Dokan v0.8, did tests with x64_avx2_par and x64_avx2_nopar versions ubraptly terminating copy process with ESC. Par version crashed more often.
Once printed "mlvfs_wrap_read(1497): caught exception 0xC0000005 at address 0x7F64C85B" but still running!
Once OS rebooted?!

Installed Dokan 1.0 again, did same tests (also copying to the not cleaned dest dir) par version crashed several times.
Once printed "smlvfs_wrap_read(1497): caught exception 0xC0000005 at address 0x2627C85B" but still running. 
Nopar never crashed.

Switched to x64_noavx_nopar version. Several times ran 3 copy processes simultaneously from the same source to the same destination (file overwrite on) and terminated with ESC or mouse (X) click. No crashes.

(http://nic.caucasus.net/mlvfs/?dir=&download=x64_noavx_nopar_3xcopy.jpg)

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 12, 2016, 05:51:33 PM
ok another speedup.
reduced the instruction count for RAW->DNG pixel conversion from 18-20 to 14 instructions on x64.
x86 should also benefit, its now 15 instructions.

here the optimized builds (http://magiclantern.fm/modules/modules/mlvfs_optimized.zip/mlvfs_optimized.zip)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 12, 2016, 08:39:15 PM
robocopy no threads

new builds:
2:35 mlvfs_x64.exe
2:36 mlvfs_x64_avx.exe
2:36 mlvfs_x64_avx2.exe

old builds:
2:37 mlvfs_x64_noavx_nopar.exe
2:38 mlvfs_x64_avx_nopar.exe
2:37 mlvfs_x64_avx2_nopar.exe

Don't even know what to say :) I guess 4717 files are not sufficient to measure the good difference and I'm too lazy to test it with 15K ;)
In real usage prog is quite snappy and rearly crashes. Used with resolve and it's fine.

Weekend is coming! :) Will test on my haswell 4790k 4.5ghz in premiere and after effects at home tomorrow.

Thanx g3gg0
bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on February 12, 2016, 10:53:08 PM
Those pesky kids and their Windows builds...


Linux now fails :( 


Somewhat expected: https://bitbucket.org/dmilligan/mlvfs/commits/097ffc1508d3daf7b827d4bfb1b39247debefc9b







gcc -Wall -std=gnu99 -D_FILE_OFFSET_BITS=64 -DVERSION="\"584941c\"" -DBUILD_DATE="\"2016-02-12 21:51:31\"" -DFUSE_USE_VERSION=26 main.c dng.o index.o wav.o stripes.o cs.o amaze_demosaic_RT.o hdr.o histogram.o mongoose/mongoose.o webgui.o resource_manager.o lj92.o gif.o patternnoise.o LZMA/7zAlloc.o LZMA/7zBuf.o LZMA/7zBuf2.o LZMA/7zCrc.o LZMA/7zCrcOpt.o LZMA/7zDec.o LZMA/7zFile.o LZMA/7zIn.o LZMA/7zStream.o LZMA/Alloc.o LZMA/Bcj2.o LZMA/Bra.o LZMA/Bra86.o LZMA/BraIA64.o LZMA/CpuArch.o LZMA/Delta.o LZMA/LzFind.o LZMA/Lzma2Dec.o LZMA/Lzma2Enc.o LZMA/Lzma86Dec.o LZMA/Lzma86Enc.o LZMA/LzmaDec.o LZMA/LzmaEnc.o LZMA/LzmaLib.o LZMA/Ppmd7.o LZMA/Ppmd7Dec.o LZMA/Ppmd7Enc.o LZMA/Sha256.o LZMA/Xz.o LZMA/XzCrc64.o -pthread -lfuse -lm -o mlvfs
main.c: In function 'mlvfs_getattr':
main.c:1017:23: error: storage size of 'mlv_stat' isn't known
         struct stat64 mlv_stat;
                       ^
main.c:1019:26: warning: implicit declaration of function 'stat64' [-Wimplicit-function-declaration]
         int mlv_status = stat64(mlv_filename, &mlv_stat);
                          ^
main.c:1017:23: warning: unused variable 'mlv_stat' [-Wunused-variable]
         struct stat64 mlv_stat;
                       ^
Makefile:35: recipe for target 'mlvfs' failed
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 12, 2016, 11:01:56 PM
well, things develop step by step.

add a
#define stat64 stat
and try again
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on February 12, 2016, 11:09:15 PM
Looks like that was that... 10q


The mlvfs file is now ~40kb smaller ( 466kb vs 504kb ), is that expected?


Should I bother with a PR or ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on February 13, 2016, 01:03:34 AM
A possible bug and some questions please.

Today I have launched MLVFS_x64_avx to a local directory containing a single MLV file mapped as drive X: All is well as confirmed via the localhost:8000 view in my browser.

     http://1drv.ms/1o7Mtqk

While I was randomly viewing individual frames (by double clicking on them) I noticed in the MLVFS cmd screen a number of loadchunk error messages. These related to mlv files tested yesterday that reside in a different (external HD) location. This suggests that MLVFS is storing previously analysed MLV file names/details somewhere. In itself, these error messages are not an issue however from a performance perspective MLVFS probably shouldn't be trying to open MLV files other than those in the mapped directory.

    http://1drv.ms/1o7OHpy

Edit:
  The generated errors messages may have been my fault. I had originally launched:
      mlvfs_x64_avx2.exe X: --mlv-dir=H:\Video\20150825_Banff_Surrouds\MLVs (where the 4  MLVs related to the error messages reside - it crashed),  I then launched

      mlvfs_x64_avx.exe X: --mlv-dir=D:\MLVs (where the single MLV resided)

  As there was no appreciable delay between the crashing of the mlvfs_x64_avx2 instance vs the launching of the mlvfs_x64_avx instance it suggests that maybe I should
  have waited a bit longer to allow the first instance to clean up after itself.


Questions

Where does MLVFS store details of historical MLVs processed? (Maybe irrelevant).
Configuration options set in the localhost view don't appear to be persistent. Is this the expected behaviour?
Having set the configuration options in localhost at what point will they apply e.g: on the fly upon opening, viewing  or extracting the frames?
MLVS caters for a number of parameters:
     - what are the defaults settings?
     - what the purpose of --port and is its sub-parameter a directory name or port number (or both)?

Possibly the above questions could be answered by adding a little more commentary to the options list provided when running MLVFS without parameters.
     
      http://1drv.ms/1TgUrdK

Thanks in advance.... Otto
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 13, 2016, 04:50:41 AM
--port is the port number for the webserver, default is 8000, it's so that you can have multiple instances running at the same time

default values are the ones you see in the webgui initially
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on February 14, 2016, 02:06:09 AM
@dimilligan.... Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: jogodedentro on February 14, 2016, 03:55:37 AM
just reached this forum after having no joy converting mlv files for use with davinci resolve, there seems to be a solution in these threads, but my understanding of coding and command line use isn't great, but please can someone send me a command line sequence that has worked for them in a similar set-up to me, I have windows 10 64bit, I have already downloaded Dokany and mlvfs_86 and the optimized mlvfs zip, I also have oracle virtual box. How do I get from having all this to successfully converting mlv files for use with davinci resolve....thanks to all for working on this.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on February 14, 2016, 04:30:36 AM
@jogodedentro

I had problems with a flaky Visual studio Redistribution 2015 runtime library install that was causing the application to fail.

The problem was fixed by re-installing all of the MS redistributable runtimes via the package advised to me by Licaon_Kter in the following post:
http://www.magiclantern.fm/forum/index.php?topic=13152.msg162133#msg162133

The following post contains guidance by Mothaibaphoto on how to run it.
http://www.magiclantern.fm/forum/index.php?topic=13152.msg162127#msg162127

There is a wealth of information within this topic that will help with your understanding. My suggestion, if you haven't already done so is, read the 1st couple of posts and then as the Windows developments are more recent go to the last post and work your way backwards.

A small tip, If you have already installed dokany and MLVFS open a cmd line window. Then:

for Dokany - cd /d C:\Program Files\Dokan\DokanLibrary
                   dokanctl.exe [enter]
and

for MLVFS - CD to wherever you have unpacked MLVFS to
                  mlvfs.exe [enter]

Both applications will present you with a list of applicable parameters.

Hope this helps.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: jogodedentro on February 14, 2016, 05:52:38 AM
thanks for your reply, yes I can see the options for DokanyLibrary using command prompt, so I assume I need to create a mount point but I can't tell how you do that from the options listed, what option in the DokanyLibrary is used to create a mount point?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: Ottoga on February 14, 2016, 06:50:42 AM
@jogodedentro

2nd link in my original post to you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 14, 2016, 12:29:11 PM
SemiOffTopic info :)

Bad news for windows users using SlySoft's great image mounter "Virtual CloneDrive". Dokan 1.0-BETA driver conflicts and renders useless that mounter. after dokanctl /r d (removing driver) and rebooting things are sorting out. No problem with Dokan 0.8 though (0.8 version has it's own issues, and with latest version of mlvfs I'd recommend 1.0)

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on February 14, 2016, 12:34:19 PM
That's a pity for Windows 7 users! Are other image mounters affected as well?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 14, 2016, 12:39:40 PM
Don't know. Have not tried any for a while.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 14, 2016, 12:44:41 PM
can you report that issue to the dokany devs?

edit: okay you already did :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 14, 2016, 12:48:22 PM
:)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: jogodedentro on February 14, 2016, 12:49:13 PM
@ottoga

Thanks for pointing me in the right direction I created the .bat and have localhost running in the web browser.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 14, 2016, 01:40:57 PM
Oh! ;) meanwhile Dokan 1.0.0-RC1 has arrived ~20 hours ago! (didn't know it, dev have just told me)

No more experiencing problems with CloneDrive! but .... :)

@g3gg0: They changed/fixed lots of things and also related to dokan_fuse and dokanfuse.dll default location. Think you have to take a look at it.

edit: all dll and lib names changed so none of the versions of mlvfs work. Situation definitely needs your attention ;)

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: michaelfilm on February 14, 2016, 08:24:49 PM
Hi everybody,

I´am new here, maybe I have an question that has already been answered...
- I have an Issue with MLVFS using After Effects CC (current version):

After adjusting the DNG in the Camera Raw Window, I get a sequence in After Effects, but the Raw-Adjustments only affects the first frame...

For any help, I would be very grateful.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 14, 2016, 09:20:19 PM
and this doesnt happen when all files are in one directory on your HDD?
any files being created in that directory?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: liryna on February 15, 2016, 01:32:20 PM
Hi everyone,

I am one of the dokan maintainers.

I just wanted to say that I added the MLVFS project to our home page http://dokan-dev.github.io/  ;)
https://github.com/dokan-dev/dokan-dev.github.io/commit/951b0a9d33cab3301566adc5d3222336cae526bf

If you need any help or remark about dokan, feel free to open an issue on github !

(I would specially be interested about your feelings in the dokan_fuse wrapper)

Thank your for using Dokan !
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 15, 2016, 05:20:26 PM
Quote from: g3gg0 on February 14, 2016, 09:20:19 PM
and this doesnt happen when all files are in one directory on your HDD?
Yes. Correct

Quote from: g3gg0 on February 14, 2016, 09:20:19 PM
any files being created in that directory?
Nope

After Effects tries to change the dng file metadata :) but alas...

Z:\M28-1253.MLV>fc /b M28-1253_000000.dng D:\DEST\M28-1253_000000.dng
Comparing files M28-1253_000000.dng and D:\DEST\M28-1253_000000.DNG
00000004: 08 00
00000005: 00 48
00000006: 00 40
FC: D:\DEST\M28-1253_000000.DNG longer than M28-1253_000000.dng

1st real file on HDD changes, others not

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 15, 2016, 09:25:01 PM
hi liryna .

welcome to our board. and thanks for maintaining and pushing dokany!
i really appreciate your work :)

as of now we are very happy with dokan_fuse and only have minor, not yet closer analyzed glitches that ive seen on 1.0-beta.
e.g. when copying from MLVFS with explorer and cancelling that copy operation, dokan will crash somewhere in nowhere or call fuse cbr's with "ddddddddddddddddddddddddddddddddddddddddddddddd....." as filename.

but i have to
a) install the 1.0-rc and
b) use the debug versions plus
c) create logs for it :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 15, 2016, 10:15:57 PM
just tried dokan-1.0-rc debug with MLVFS and it just doesnt work anymore.
didnt change the code but fuse_main returns 1
no matter if i specify -f, -d, -o debug or all together.
no log messages, nothing.

anything that has changed?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 15, 2016, 11:41:12 PM
Quote from: bouncyball on February 15, 2016, 05:20:26 PM
After Effects tries to change the dng file metadata :) but alas...

So MLVFS tells FUSE that the DNGs are writeable so they show up in the mount as writeable, but they aren't really (which would sort of impossible anyway). The reason it's like this is that ACR/AE refuses to open readonly DNGs, even though it doesn't actually need to write to them. On my Mac ACR doesn't actually try to put metadata in the DNGs, instead it creates XMP sidecar files and puts all its metadata there (there might be a setting for this somewhere for you). MLVFS allows a "passthrough" so programs can create and modify sidecar files like this inside the mount directory. It stores the actual files next to the MLV files in the real file system in an .MLD directory. You can put anything you want except .DNG and .WAV files into the mount. At least, this is how it is supposed to work, and how it works on Mac.

So you to get ACR to use sidecars and then make sure that MLVFS's passthrough works (just try to copy some file not ending in DNG or WAV into the mount)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 16, 2016, 08:40:45 AM
@dmilligan

Thanx for your attention. Yes on my hakintosh AE works well. Have to look to the mounted folder.

Quote from: dmilligan on February 15, 2016, 11:41:12 PM
just try to copy some file not ending in DNG or WAV into the mount

That's exactly what I'm trying to do. Passtrough does not work under win. (I think g3gg0 already mentioned this)
In file explorer: copying file - no error but file is not there (no any .MLD). Making new dir or new file results in error message "file/folder name you are creating is too long" and I have not even input the file name yet.

edit: I'm kinda big fan of the "FAR manager" and using it many years and  just checked copying with it. Can not copy/create file, BUT... can create DIR and MLD appears in mounted folder (can not be deleted after that though, mkdir command does the same). Hmmm... strange and unpredictable behavior. (Bugs! Bugs are everywhere ;) )

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: liryna on February 16, 2016, 07:58:02 PM
@g3gg0

Did you copy dokan1.dll in your runtime folder ?
dokan1.dll is no longer in system32 (for compatibility reason for past / next release).

If this doesn't work, could you open an issue on dokan repository with logs ?

Thank is advance
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 16, 2016, 10:08:01 PM
issue already resolved and created a ticket: https://github.com/dokan-dev/dokany/issues/175 :)

BR,
g3gg0
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 17, 2016, 12:53:11 PM
just symlinked:

MLVFS-DIR\dokan.dll -> DOKAN-RC1-DIR\dokan1.dll
MLVFS-DIR\dokanfuse.dll -> DOKAN-RC1-DIR\dokanfuse1.dll

and "mlvfs_x64" works again
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 17, 2016, 05:18:30 PM
@dmilligan

What the log file in the virtual mlv folder does? As I see it's always empty.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 17, 2016, 05:52:20 PM
The MLV file format spec (https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdHJraTVTOEpmNEIwTVlKd0dHVi1ULUE#gid=0) supports embedding plain text blocks. If there is any in the MLV file, it will appear in the log. I've thought about including other diagnostic information in this file too, like a dump of the header info.

I think currently the only thing that will write these text blocks in the MLV file is if you turn on the debug tracing option (I think you have to load trace.mo too).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 17, 2016, 06:52:28 PM
@dmilligan: Thanx for comprehensive explanation :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 17, 2016, 08:48:52 PM
@dmilligan:
whenever mlv_rec gets a keypress signalled, it will create a "MARK" block.
do you think there is a good use case for it? e.g. insert a text file for this frame number.

same for blocks like LENS which will update sporadically and will reflect aperture and focal length, focal distance etc.
this data could be used on a per-frame basis.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 17, 2016, 09:45:07 PM
well, i dont get any crashes anymore when aborting copy operations with dokan 1.0-rc

MLVFS_x86 (http://www.magiclantern.fm/modules/modules/MLVFS_x86.zip/MLVFS_x86.zip) or MLVFS_x64 (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip) to be used with dokan 1.0-rc -> DokanSetup.exe (https://github.com/dokan-dev/dokany/releases)

make sure you uninstall the old version first.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on February 17, 2016, 11:02:00 PM
There might be something in the CDNG spec for markers, I'll have to take a look. Another possibility is an option that breaks them up into separate sequences, so restart the numbering of the DNGs with a different prefix or something like that.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 18, 2016, 09:53:40 AM
Quote from: g3gg0 on February 17, 2016, 09:45:07 PM
MLVFS_x86 (http://www.magiclantern.fm/modules/modules/MLVFS_x86.zip/MLVFS_x86.zip) or MLVFS_x64 (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip) to be used with dokan 1.0-rc -> DokanSetup.exe (https://github.com/dokan-dev/dokany/releases)

@g3gg0: Great, Thanx! :)
Can you look at the creating .MLD folder bug? e.g. make passthrough mode normally work on win.

edit: Exceptions (at 2 addresses) below caught while maniputating DNGs in Premiere Pro, e. g. stoping starting, scrubing back and forward and changing "lumetri color" parameters. MLVFS does not stop entirely just hickups and running while prevew freezes in premiere. It seems MLVFS does not like random file access very much :)

E:\MLVFS_old>mlvfs_x64.exe Z: --mlv-dir=E:\MLV
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xC476FD74
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xC476FD74
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877
mlvfs_wrap_read(1512): caught exception 0xC0000005 at address 0xBED5C877

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 18, 2016, 05:49:19 PM
@g3gg0, dmilligan

This exeption 0xC0000005 - generated as far as I understand on "segmentation fault"/"access violation" (accesssing virtual/physical memory - the state of wich is not allowing that access) - occurs when you are stopping and starting playing dng sequence in premiere or resolve. It is similar to the situation when you are copying files and then suddenly canceling it. As g3gg0 mentioned above this randomly was ending by a crash (after printing this exeption as I've seen).

x86 version gives a little bit different numbers:
mlvfs_wrap_read(1505 <- is it line number or what?) and surely another virtual address.

E:\MLVFS_x86>mlvfs_x86.exe Z: --mlv-dir=E:\MLV
mlvfs_wrap_read(1505): caught exception 0xC0000005 at address 0x6A70D72E

In short mlvfs completely crashes very rearly, but exeption is generated almost on every play/stop while prog goes on runnig-hickup-running.

Unfortunatelly I'm not a c/c++ dev in any way.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 19, 2016, 02:05:19 AM
can you redownload from the same URL and test again?
it also contains debug info that will slow down the tool.

the things not working:

- editing *.dng, *.gif, *.log, *.wav of a virtual MLV folder (subdirs are okay)
- placing *.dng, *.gif, *.log, *.wav into a virtual MLV folder (subdirs are okay)
- copying/moving *.dng, *.gif, *.log, *.wav of a virtual MLV folder into a subdirectory
- deleting folders (https://github.com/dokan-dev/dokany/pull/166)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Lasting Image on February 19, 2016, 07:38:48 AM
Quote from: dmilligan on February 15, 2016, 11:41:12 PM
At least, this is how it is supposed to work, and how it works on Mac.

I'm using a Mac and have the same problem where only the first dng in the After Effects sequence has the Camera Raw corrections. There's a video I saw where Reddeercity uses Camera Raw in After Effects on the first dng and each subsequent dng has the corrections - not just the first. I'm not noticing any difference in what I'm attempting from the settings in reddeercity's video (I think). Any ideas on what I'm doing wrong? In other threads I recall people complaining about later versions of After Effects. Could it be an issue with the latest version of AE CC?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 19, 2016, 08:16:20 AM
@Lasting Image:
Quote from: Lasting Image on February 19, 2016, 07:38:48 AM
I'm using a Mac and have the same problem where only the first dng in the After Effects sequence has the Camera Raw corrections.
This is how AE interprets DNG sequences: It opens ACR window with first file, you store settings in either file itself or sidecar xmp according to your settings.
From this point it converts any other file in sequence disregarding has it his own metadata, or not.
This has some benefits (no flickering, for example), and drawbacks (you can't ramp exposure, or wb, for example), but this is absolute offtopic to MLVFS.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 19, 2016, 10:25:59 AM
Quote from: g3gg0 on February 19, 2016, 02:05:19 AM
can you redownload from the same URL and test again?
it also contains debug info that will slow down the tool.

With previous no debug version of mlvfs binaries:
Under the root dir (e.g. mount point Z:\ or whatever) where MLV virtual folders are - no file/dir operations possible
Under virtual MLV folders - only directory creation is possible (no subdirs), .MLD created in the orig mlv folder

With last posted debug version of mlvfs binaries:
Under the root DIR - possible: creating dir/file, copying dir/file, not possible: delete created dir/file (as expected)
Under any virtual MLV folder - nothing's possible! no file/dir creation/copying/moving/deleting, hence no MLD directories appear in the orig mlv destination

BTW Am I supposed to see the .IDX .M01 .M02 files in the root dir along with the virtual .MLV folders? They are there :-\ in last debug version.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 19, 2016, 02:02:31 PM
Quote from: mothaibaphoto on February 19, 2016, 08:16:20 AM
From this point it converts any other file in sequence disregarding has it his own metadata, or not.
Actually, this is wrong. It does not disregard xmp files or embedded metadata for other images in the sequence if they are there. That's how this script (http://www.magiclantern.fm/forum/index.php?topic=8850.0) is even able to work. I have used it many times with AE.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 19, 2016, 02:53:20 PM
@dmilligan:
I'm a big fan and very power user of your script :)
Take this opportunity to say Great Thanks for it one more time :)
But ... my AE works as I described. And it's generally good for me. Just specially tried to save different settings and open sequence. Maybe some settings I don't know affect this behavior.
After applying your script I always convert right in Bridge - CtrlA->CtrlR
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: Ottoga on February 19, 2016, 03:49:27 PM
@Dmilligan

A big thumbs up from me as well for your Windows version.

I've been giving it a solid try out during the mlv lite testing and love the on the fly the processing.

Took me a while to figure out that that I had to set the parameters before opening the Dokan mounted file. [Facepalm].
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 20, 2016, 10:28:19 AM
@bouncyball:
can you check it?

Quote from: g3gg0 on February 19, 2016, 02:05:19 AM
can you redownload from the same URL and test again?
it also contains debug info that will slow down the tool.

the things not working:

- editing *.dng, *.gif, *.log, *.wav of a virtual MLV folder (subdirs are okay)
- placing *.dng, *.gif, *.log, *.wav into a virtual MLV folder (subdirs are okay)
- copying/moving *.dng, *.gif, *.log, *.wav of a virtual MLV folder into a subdirectory
- deleting folders (https://github.com/dokan-dev/dokany/pull/166)

Quote from: g3gg0 on February 17, 2016, 09:45:07 PM
well, i dont get any crashes anymore when aborting copy operations with dokan 1.0-rc

MLVFS_x86 (http://www.magiclantern.fm/modules/modules/MLVFS_x86.zip/MLVFS_x86.zip) or MLVFS_x64 (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip) to be used with dokan 1.0-rc -> DokanSetup.exe (https://github.com/dokan-dev/dokany/releases)

make sure you uninstall the old version first.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 20, 2016, 01:41:09 PM
@g3gg0

I already did :)

http://www.magiclantern.fm/forum/index.php?topic=13152.msg162572#msg162572 (http://www.magiclantern.fm/forum/index.php?topic=13152.msg162572#msg162572)

Has anything changed since then?

It's not crashing but, as you said, unusable in real situations because of the standard output log slowing it down so much.
Could not manage to stress it so to see some exceptions.

Passthrough related things are in my message above.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 20, 2016, 02:53:11 PM
redownloaded binaries. yeah they dated 02/20/16 :). after checking, can admit, that applies exactly the same what I've been talking about in 2 earlier messages.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 20, 2016, 09:38:08 PM
sorry, too much noise so i didnt see your post.
thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 21, 2016, 08:08:14 PM
@dmilligan

I have one question about bad pixel removal. Can we do this with pixel maps (like focus pixel maps by dfort). Just having an option to remove them by the map or current algorithm and then anyone can make pixel maps for his/her needs: sensor and resolution (crop, non crop etc). It would be lot more faster.

Besides I have just one pixel and that's all ;)

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on February 21, 2016, 11:26:05 PM
Here's an idea. How about if there is a pixel map file with the camera serial number in the MLVFS Contents? That way it will only fix the dead pixels on that specific camera. This would be in addition to the focus pixel map files because these cameras can also have dead pixels.

Of course this would require some effort on the user but it shouldn't be any more difficult than the instructions in dcraw:

FILES
       ./.badpixels, ../.badpixels, ../../.badpixels, ...
              List of your camera's dead pixels, so that dcraw can interpolate
              around them.  Each line specifies the column, row, and UNIX time
              of death for one pixel.  For example:

               962   91 1028350000  # died between August 1 and 4, 2002
              1285 1067 0           # don't know when this pixel died

              These coordinates are before any stretching or rotation, so  use
              dcraw -j -t 0 to locate dead pixels.


I don't see any real need to keep track of the date.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 22, 2016, 12:52:54 AM
that unix time idea is quite funny t.b.h :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on February 22, 2016, 01:35:09 AM
You mean the "UNIX time of death for one pixel" idea?  :P I never use it but dcraw "badpixels" lists require something in that field. I just fill it in with zeros.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on February 22, 2016, 04:39:18 AM
Excellent concept @bouncyball and great suggestion by @dfort!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 22, 2016, 08:38:28 AM
Quote from: dfort on February 21, 2016, 11:26:05 PM
Here's an idea. How about if there is a pixel map file with the camera serial number in the MLVFS Contents? That way it will only fix the dead pixels on that specific camera. This would be in addition to the focus pixel map files because these cameras can also have dead pixels.

Nice idea! Let's wait what master David will say :)

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 22, 2016, 09:05:50 AM
Actually, is there dead pixel = Fn(Camera Serial, ISO, Shutter, Sensor temp, Etc) :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 22, 2016, 09:51:23 AM
QuoteHere's an idea. How about if there is a pixel map file with the camera serial number in the MLVFS Contents? That way it will only fix the dead pixels on that specific camera. This would be in addition to the focus pixel map files because these cameras can also have dead pixels.

Are dead/hot pixels always on the same place? My understanding is that the will vary from time to time. Of course a dead pixel is a dead pixel but a hot pixel could pop up anywhere? Am I wrong here?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 22, 2016, 10:37:23 AM
Ah, my bad :(
Bad pixels always there.
Cold/hot pixels can be relatively stable. So,
Is there dead pixel = Fn(Camera Serial, Time)
Is there Cold/hot pixel = Fn(Camera Serial, ISO, Shutter, Sensor temp, Time, Etc)
Ultimately, it would be great to have sensor profiling tool, similar to dot_tune.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 22, 2016, 01:22:26 PM
Quote from: Danne on February 22, 2016, 09:51:23 AM
Are dead/hot pixels always on the same place? My understanding is that the will vary from time to time. Of course a dead pixel is a dead pixel but a hot pixel could pop up anywhere? Am I wrong here?

I think you are right :). Appearance of that kind of pixels depends on several things:
AFAIK canon software has factory callibration built in and maps pixels itself (however you can kinda alter this remapping with several second manual sensor cleaning procedure. That's how I cleaned up some hot pixels - not bads - on my former 7D) if you got chance read raw buffer before that then you have vanilla information with all off uncorercted pixels etc and I think this is a case when we get croped resolutions of raw data from the sensor (some buffers in memory I guess). Also line skipping/bining affects coordinates and appearence of those pixels. In short, whether they appear in output raw or not, depends on the camera mode and what canon firmware does in that moment and also what we are doing with great help of ML software ;).

But... the pixel correction map is still for a dedicated resolution and mode, hence valid in most cases.

This is my understanding... correct me please if i'm wrong.


bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 22, 2016, 01:28:05 PM
Would have been smooth with a dot picker in MLVFS which creates the map on the fly when clicking your dots on an dng file :P (Wishful thinking)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 22, 2016, 01:43:04 PM
Saying Clicking do you mean automatic removal algo? Sometimes it misses some dots ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 22, 2016, 01:46:40 PM
Since it often happens you only have one or two annoying pixel dots you could find its coordinates on pixel level(zoomed in around 400%). Since Davids web gui works in realtime maybe this could be achieved by grabbing neighbouring pixels like in dcraw and the previewing dng would instantly show the result? As I said, wishful thinking  8)
There,s also opcode lists which could work with the dng format. But how I don,t know.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on February 22, 2016, 01:48:59 PM
If I may use a term that dmilligan used previously, K-I-S-S (https://en.wikipedia.org/wiki/KISS_principle).

Focus pixels and bad/dead pixels are always there. Maybe they are not always visible or pose a problem but the location of these pixels on the sensor for a particular camera never change. These are the pixels that can be relatively easily mapped and eliminated.

Pixels that change according to the temperature of the sensor or other conditions are something else. Astrophotographers go to great lengths to separate celestial objects from this noise. Google astrophotography bias frames and you'll see what I mean.

Then there is fixed pattern noise (FPN) (https://en.wikipedia.org/wiki/Fixed-pattern_noise) which is something that can also change and thus cannot be mapped out.

Maybe I'm wrong but it seems that sensor profiling cannot be done just once and it will be good for the life of that camera. Focus pixels and bad/dead pixels fixing is permanent.

As far as a dot picker--unless MLVFS gets a graphical user interface that can zoom in far enough to work on individual pixels it is easy enough to map out dead pixels with Photoshop or Gimp.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 22, 2016, 08:49:18 PM
@bouncyball:
> Under any virtual MLV folder - nothing's possible! no file/dir creation/copying/moving/deleting, hence no MLD directories appear in the orig mlv destination

can create dirs and files without any issues.
delete/copy not possible. trying to work around that dokan issue (https://github.com/dokan-dev/dokany/issues/178)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: jogodedentro on February 23, 2016, 05:30:19 AM
Hi There

With help from this forum I was happily colour grading MLV files in davinci resolve. But now after a reinstall of windows 10, I have installed Dokany and despite entering the path to DokanyLibrary the dokanctl.exe command brings up a 'not recognized internal or external command...' error message. This command was previously working on the same Windows 10 machine, but now after a re-install of windows I have this issue. I have installed the latest version of Dokany. Any ideas how I can get this to work again, advice will be much appreciated...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: jogodedentro on February 23, 2016, 06:36:59 AM
ok solved problem re-install Dokany but don't re-install microsoft redistritbutable on prompt, dokany installation will then complete.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 23, 2016, 11:01:23 AM
path to dokanktl.exe (Dokan 1.0.0-RC1): C:\Program Files\Dokan\Dokan Library-1.0.0\dokanctl.exe

edit: ah ok. nice to hear that :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 23, 2016, 11:54:39 AM
@g3gg0

> can create dirs and files without any issues.
> delete/copy not possible. trying to work around that dokan issue

Post your latest compiled binaries please and I will check them with pleasure :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 23, 2016, 12:37:01 PM
@g3gg0

with the last debug version that I have

root dir looks like this. mlv folder plus IDX and Mxx files and here I can create dirs and files
------------------------------------------------------------------------
Z:\>dir
Volume in drive Z is DOKAN

Directory of Z:\

02/11/2016  12:40 PM                  5 ,572 M28-1253.IDX
05/28/2015  12:53 AM    <DIR>               M28-1253.MLV
02/11/2016  12:40 PM                   4,204 M28-1255.IDX
05/28/2015  12:56 AM    <DIR>               M28-1255.MLV
02/11/2016  12:39 PM                 10,144 M28-1313.IDX
05/28/2015  01:14 AM    <DIR>               M28-1313.MLV
02/11/2016  12:39 PM                 15,460 M28-1316.IDX
05/28/2015  01:16 AM    <DIR>               M28-1316.MLV
02/11/2016  12:39 PM                 35,512 M28-1322.IDX
05/28/2015  01:24 AM     4,294,385,716 M28-1322.M00
05/28/2015  01:24 AM          94,462,004 M28-1322.M01
05/28/2015  01:23 AM    <DIR>               M28-1322.MLV
               7 File(s)  4,388,918,612 bytes
               5 Dir(s)               0 bytes free
-------------------------------------------------------------------------

in mlv folders:

trying creating dir: no output but in reality folder did not created!
Z:\M28-1253.MLV>mkdir folder1

trying creating file: error message
Z:\M28-1253.MLV>echo 123 > file.txt
The system cannot find the file specified.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 23, 2016, 01:15:20 PM
Can someone please help - i am a noob with this - i have followed the steps in the OP and included the following code in the C:/ but getting a syntax error. What is the 'mount point'?

I am on windows 10.

Need to learn how to use this program to decode MLV Lite files. At the moment MLViewer is showing pitch black screens of video recorded with the MLV Lite module.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 23, 2016, 10:23:01 PM
updated the MLVFS_x64.zip, please retry
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 24, 2016, 12:54:05 AM
Sorry guys but for me I find this hard to believe that no other Mac users is experiencing what I am going through for the past week or so...

https://vimeo.com/156488401

It's rather quite annoying and I've also trying deleting .IDX and .MRX files just to be sure as well as updating/downgrading OS X FUSE to no avail.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on February 24, 2016, 01:27:38 AM
Anything in ~/mlvfs.log? Any crash logs?
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 24, 2016, 02:30:18 AM
I'm afraid I may be doing this incorrectly via Terminal...

(https://farm2.staticflickr.com/1687/25106404052_71d25f21e1_n.jpg) (https://flic.kr/p/EfyTeo)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 24, 2016, 10:22:48 AM
@g3gg0

> updated the MLVFS_x64.zip, please retry

with new one - copy/creation of file/dir is the same (no avail) as described above.
one minor change to root dir - .IDX files disappeared (good), .Mxx files are still there (bad) :)

bb
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: koenkooi on February 24, 2016, 12:10:46 PM

Quote from: DeafEyeJedi on February 24, 2016, 02:30:18 AM
I'm afraid I may be doing this incorrectly via Terminal...

(https://farm2.staticflickr.com/1687/25106404052_71d25f21e1_n.jpg) (https://flic.kr/p/EfyTeo)

Try 'cat ~/mlvfs.log' to show the contents in the terminal window or 'open ~/mlvfs.log' to try opening it with the osx log reader. Or point Finder to your home directory, probably /Users/DeafEyeJedi.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 24, 2016, 12:47:43 PM
@DeafEyeJedi can you please describe textually what problem you are experiencing?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 24, 2016, 10:19:59 PM
Quote from: g3gg0 on February 24, 2016, 12:47:43 PM
@DeafEyeJedi can you please describe textually what problem you are experiencing?
Well I believe this whole 'culprit' of mine started around the time when the Windows version of MLVFS was finally released by @dmilligan (which I am happy about since we want more users to come on board with us) however it is my understanding that this anomaly shouldn't be occurring with different types of MLV files from 5D3, 7D, EOSM or even SL1 which used to work without the infamous web gui error (hence in my video above) which shows some 5D3 wont work and some would along with 7D files working just fine. It's just so random to me.

I then was suggested by @dfort to go ahead and downgrade to an older version of MLVFS (Version: 1bc6f7f 2016-02-04 03:51:18) which seems to be working fine on my end with the same exact files that weren't working earlier with the latest version of MLVFS. Not sure why though and I can't seem to find any crash logs probably because it hasn't crashed yet.

https://vimeo.com/156615377

Quote from: koenkooi on February 24, 2016, 12:10:46 PM
Try 'cat ~/mlvfs.log' to show the contents in the terminal window or 'open ~/mlvfs.log' to try opening it with the osx log reader. Or point Finder to your home directory, probably /Users/DeafEyeJedi.
Tried all these to no avail. Thanks for the courtesy tho and much appreciated. So I would like to help further down by narrowing down this so called 'culprit' since I can now open all the files that I wouldn't be able to run with your latest MLVFS as oppose to the Feb 3rd version.

Any ideas/suggestions is welcome...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 24, 2016, 10:49:13 PM
as i am on a slow network i cannot watch this video and cannot understand what the problem in general is.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 24, 2016, 11:00:21 PM
With all the MLV Files that I am trying to work on my Mac (mostly 5D3 with 7D and few SL1) which doesn't work with latest MLVFS (Version: a853aac 2016-02-10 02:05:13) but then once I downgrade to an older version of MLVFS (Version: 1bc6f7f 2016-02-04 03:51:18) which works fine like normal and can work on them in Post with no problems.

So AFAIK something happened between those two dates somewhere in the source code of some kind.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 24, 2016, 11:06:33 PM
what does "does not work" mean?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 24, 2016, 11:18:01 PM
@bouncyball:
thanks, i found the reason. didnt create the .MLD properly
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 24, 2016, 11:49:43 PM
Quote from: g3gg0 on February 24, 2016, 11:06:33 PM
what does "does not work" mean?

•Not working (Version: a853aac 2016-02-10 02:05:13) = (screenshot below)
(https://farm2.staticflickr.com/1676/25242774385_b21acacedf_n.jpg) (https://flic.kr/p/EsBPo2)

•Working (Version: 1bc6f7f 2016-02-04 03:51:18) = (screenshot below)
(https://farm2.staticflickr.com/1491/25124486682_058863e7ab_n.jpg) (https://flic.kr/p/EhayyS)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on February 25, 2016, 12:08:12 AM
Sorry, there's a dot.


cat ~/.mlvfs.log
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 25, 2016, 12:50:54 AM
Okay so now this brings me to another question ... Do I run this command while running the older version of MLVFS that works or with the latest one that's supposedly causing problems on my end?

I would assume to go with the latest one since that's the one that you guys want to know, correct?

Thanks, D!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on February 25, 2016, 01:55:00 AM
That is just a log file. When you run MLVFS using the Automator workflow, the standard output is written to that file. The cat command simply displays the file, alternatively you can open it with a text editor or whatever.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on February 25, 2016, 02:12:06 AM
I posted about the new Mac build being DOA a while back http://www.magiclantern.fm/forum/index.php?topic=13152.msg162129#msg162129
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on February 25, 2016, 01:38:34 PM
Put MLVFS through its paces again today with a variety of MLV_Lite generated files that included single/multi-chunk mlvs, auto stopped mlvs due to frame skipping and one crash mlv when it tried to switch from 5x to 10 recording. All good, no bad frames that I could find and no error messages in the mlvfs cmd window.

Camera is a 7D.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 25, 2016, 04:52:24 PM
@g3gg0

Quote from: g3gg0 on February 24, 2016, 11:18:01 PM
i found the reason. didnt create the .MLD properly

Hey, great! Can you post new binaries? :)

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 25, 2016, 09:24:59 PM
yep, uploaded
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 26, 2016, 02:59:14 AM
Hey everyone, I must admit that I am completely lost with what to do. Have reached a point (after endless hours of sitting on the computer and a bad back with a few 10's of $'s spent on buying programs that will let me buy .dll files and all) where I can go into the cmd and request the computer to execute the mlfvs executable and all that. But do not understand what to do next ... for example where do the converted DNG's surface. Cannot see these anywhere .. and since there is no error on the cmd screen, I reckon all must be good.

My workflow:

I first download the mlvfs_x64.exe zip and then unzip (naturally  ;))
Then copy the unzipped mlvfs_x64 folder to the my f:/ drive
Mount the f:/ drive to my z:/ drive - so now all that is on f:/ shows up on z:/ (every thing and the mlvfs_x64 folder too)
I also copy the mlv that I want to convert to the f:/ root folder
Then, I just go to cmd where if first bring the computer to f:/ (then) it is f:/mlvfs_x64 (then) f:/mlvfs_x64/mlfvs_x64.exe f:/ --mlv-dir=z:/out (where 'out' is my intended output folder on z:/ where I want my DNG's to go

Not sure what to do with the dokey elements. When I download it, it is not coming on but there is an error in the .exe

I am on Windows 10 at the moment. Will try using these executables (FUSE and all) on the my Macbook pro to see if I come up with different results.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 26, 2016, 03:53:41 AM
Once you start mlvfs like this:
mlvfs_x64.exe Z:\ --mlv-dir=F:\

That is all you have to do.

Now if you copy an MLV file to the root of F:\, look in Z:\, you should see a folder with the same name as the MLV file, inside it are your DNGs.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 26, 2016, 08:21:55 AM
do not mount the drive somewhere with windows methods.

just extract the files to f:\mlvfs_x64\
open a commandline


f:
cd \mlvfs_x64
mlvfs_x64.exe Z:\ --mlv-dir=F:\


if it doesnt work, show us screenshots of what you typed
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 26, 2016, 08:53:12 AM
Quote from: dmilligan on February 25, 2016, 01:55:00 AM
That is just a log file. When you run MLVFS using the Automator workflow, the standard output is written to that file. The cat command simply displays the file, alternatively you can open it with a text editor or whatever.

Thanks anyway I had to take parts of the log out into context since it was too large to paste the entire textfile on this post so here it is...

Last login: Thu Feb 25 23:41:04 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ cat ~/.mlvfs.log
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2141.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2142.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2143.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2147.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2144.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2145.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2151.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2146.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2153.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2152.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon-precomposed.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2156.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2159.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2248.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2246.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2249.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2250.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2251.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2258.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2301.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2257.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon-precomposed.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2302.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/favicon.ico...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2304.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2303.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2308.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2306.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2307.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2305.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2309.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2310.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2316.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2313.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2312.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2314.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2311.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon-precomposed.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2315.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2321.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2319.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2320.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2318.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2317.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2141.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2146.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2145.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2144.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2143.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2142.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon-precomposed.png...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/favicon.ico...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2147.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2151.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2248.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2153.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2152.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2156.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2246.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2250.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2251.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2249.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2159.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2258.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2303.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2302.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2301.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2257.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2304.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2306.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2309.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2308.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2307.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2305.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2310.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2314.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2316.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2313.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2312.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2311.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2315.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2320.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2321.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2319.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2318.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2317.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2141.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2142.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2143.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2147.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2144.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2145.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2151.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2146.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2153.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2152.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2156.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2159.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2248.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2246.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2249.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2250.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2251.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2258.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2301.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2257.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2302.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2303.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2304.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2305.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2306.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2308.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2307.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2310.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2309.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2311.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2314.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2313.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2312.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2315.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2316.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2320.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2319.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2318.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2317.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2321.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 26, 2016, 09:00:39 AM
@g3gg0

Quote from: g3gg0 on February 25, 2016, 09:24:59 PM
yep, uploaded

Thanx! Great improvement over the last one !!!

1. Can create dirs and files everywhere including root dir
2. Can delete files! but not folders (guess related to the dokan issue)
3. One annoying thing: .M00, .M01 ... files are visible in root dir. Definitely we don't need this, because it's possible to delete them and mlvfs crashes its a..s off :) tested.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 26, 2016, 09:06:42 AM
After some pause I finally find time to continue with MLVFS testing.
Setup: Dokan 1.0.0-RC1, MLVFS_x64.zip 25-Feb-2016 20:24, Resolve 12.3.1.001
Issues:
1. Any file with non-MLV and non-IDX extension present in the root of the virtual disk. All the chunks, wavs, and so on. Actually, its not a problem, just wonder why.
2. Performance is greatly redused. I have constant lags in playback. Without any option enbled(everythig "OFF" in "Configuration Options") mlvfs_x64 consumes 52-56% of CPU all the time and this looks as main reason for that.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 26, 2016, 09:29:39 AM
Right. On my system plain file copy shows that, CPU is higher 5-7% vs 10-14%, but speed is about the same 100-125mb/sec.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 26, 2016, 10:01:15 AM
@g3gg0: (OT) what the snd_viz.mo does?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 26, 2016, 05:31:01 PM
computer is looking for dokanfuse1.dll on my windows 10 machine after i press the command on cmd

now downloaded the latest mlvfs_x86 file version 2/11/2016 and when i execute cmd i get the message 'unmount' but no files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 26, 2016, 05:51:06 PM
No success. I finally give up :-(

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on February 26, 2016, 06:52:29 PM
Just installed it for the first time on W7. Dokany 0.8.0. Worked.
May I offer a Teamviewer session?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 26, 2016, 07:05:52 PM
what is a team viewer session?

after giving up on windows 10, i tried the mac but am getting a permission denied for the mlvfs executable file - what do I do from here?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 26, 2016, 08:54:54 PM
@Canon eos m:

win10:
please uninstall dokan from your control panel
install dokan v1.0.0-rc1 (https://github.com/dokan-dev/dokany/releases/download/v1.0.0-RC1/DokanSetup.exe)
unpack mlvfs_x64 (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip) into your folder f:\mlvfs_x64\
open a commandline

f:
cd \mlvfs_x64
mlvfs_x64.exe Z:\ --mlv-dir=F:\


osx:
did you do a chmod +x FILENAME ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 26, 2016, 10:38:04 PM
@g3gg0

you are right too much noise did you read my messages above?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 26, 2016, 11:20:04 PM
yeah, the higher load -> need to examine
Mxx files -> todo

my goal, after breaking some stuff, is to make it usable again and these are nonfunctional requirements.
had to rework a lot of stuff due to the win32 specialities and added new features like proper .MLD support with nesting folders etc.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 26, 2016, 11:42:29 PM
yeah take your time, thanks for doing all this stuff.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 27, 2016, 02:45:40 AM
ok getting closer to the end goal .. on my mac have everything lineup but now there is localhost8000 error: safari cannot open page localhost8000 because safari cannot connect to server 'localhost'

tried to execute chmod + x but get the error the file does not exist (at least now know what chmod does  :-)



success on mac .. finally .. will try on Windows10 later
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on February 27, 2016, 03:20:23 AM
Now that makes the two of us reporting these so called web GUI errors on OS X ... Anyone else experiencing this as well?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 27, 2016, 05:32:52 AM
delete ~/.mlvfs.log and then run it one time where it fails and then send me the log (I want the log of a single failing run)

is there anything mlvfs related in ~/Library/Logs/DiagnosticReports ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 27, 2016, 06:19:37 AM
i finally cracked it. downloaded the latest FUSE version that g3gg0 shared yesterday (see a few postings above the last one). It was an easy ride thereafter:

1. the FUSE automatically stations in the desktop folder
2. then, i created a blank folder in the desktop folder (mine was called MLV) - remember this should be blank
3. transferred, the camera recorded mlv file to a new folder on my desktop (not the blank MLV folder mentioned above - this should stay blank for till the FUSE is run on it)
4. then went to this folder with the mlv recorded file and right clicked. Created an association with the mlvfs format - this is the second last menu 5. 5. option on the drop down that shows up when you right click the mlv recorded folder
6. the moment you click the mlvfs tab it will show a folder for you to look for the place where you want to mount and transfer your extracted mlv file. 7. here i just directed the mac to the previously created blank MLV folder and the rest is history
8. opened the extracted files as a sequence in AE .. all showed up fine

Hope this helps everyone struggling to make this work - i have literally died trying to set this up - cannot still work out how to do this on Windows (that would be better since i have all my recorded mlv's in there) - but the donkey folder shows up some error (shared the error with g3gg0) - lets see

I am still trying to figure out how to covert several mlv files in one go or at different times using the same folder the I first created (at the moment have to create a new one each time). Plus, the FUSE GUI that opens up continues to show the first file i covered and now the new ones

Lastly, i covered the files and imported the sequence in AE. This took me to the ACR GUI. I changed the frame but the changes stay with the first frame - anyone know how i can transfer the effects and changes to other frames in the sequence

Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on February 27, 2016, 06:42:16 AM
"Lastly, i covered the files and imported the sequence in AE. This took me to the ACR GUI. I changed the frame but the changes stay with the first frame - anyone know how i can transfer the effects and changes to other frames in the sequence"

When you import an image sequence in AE, the changes to the first frame should be applied to all frames in the sequence.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 27, 2016, 08:22:29 AM
Hi Frank, they aren't. Have been googling this since morning. when i log out of ACR the xmp sidecar isn't created either. even forcing ACR is not working. So, when I import the first frame is the corrected one but other don't carry the change. still looking will share if i do find what the problem was.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 27, 2016, 09:57:32 AM
@Canon eos m:
did you uninstall all dokan versions from your control panel before?

> but the donkey folder shows up some error (shared the error with g3gg0) - lets see
you mean "dokan". you sent me the *installer* logfile.

for the installer: are you sure you have all windows updates installed on your system?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 27, 2016, 11:04:37 AM
@bouncyball:
if you redownload, it might be a bit faster. i am caching internal structures now and skip checking for existense of the MLV file internally.

getting inconstistent performance measurement results when using resolve.
the most CPU time seems to be used in the locking mechanism.

guess resolve is loading the files in different threads and the locking is a bit coarse.
but i am trying to find that out.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 27, 2016, 03:18:28 PM
@g3gg0, I did uninstall all versions that the I had and checked. Not sure about the latest windows updates - Wil check later tonight and report.

Dokan is clearly the challenge. Have the Mlvfs working on the Mac for the moment and am satisfied with the speed of conversion bit struggle to understand the GUi interface and related. Will work to master in the coming days and report progress. I wanted to work with mlvfs since I want to work with mlv lite. Hope to test that too this week and next.

But for now the more pressing issue is the dng and xmp side car problem I am facing. Need to work out how to transfer modifications made with ACR on a single frame to all others. Mac not creating sidecars but understand that is not possible with dngs anyway since the xmp data is baked into the dng. So, the problem maybe with AE - not sure yet.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on February 27, 2016, 03:27:46 PM
XMP sidecars work fine on Mac
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 27, 2016, 04:19:59 PM
@g3gg0

Quote from: g3gg0 on February 27, 2016, 11:04:37 AM
if you redownload, it might be a bit faster. i am caching internal structures now and skip checking for existense of the MLV file internally.

Nice :D you are on a good direction!

Plain copy mlvfs CPU usage: (tested on my home pc which is more powerful (1ghz diff) hence smaller numbers than in previous posts)
25feb build - average ~7-8%, peak ~18%
27feb build - average ~4-4.5% peak ~15%
Throughput about the same ~115mb/sec
Results are collected from 10 runs each, hence no placebo :)

In resolve 27feb build definitely snappier! almost no lug on play/pause/rewind/fast forward/scrub, however, not very-very-very sure 'bout it, could be a lil'bit placebo effect ;)

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 27, 2016, 05:07:03 PM
Hopefully fixed fuse pass through issue on Mac (XMP sidecars not getting created), new build uploaded.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on February 27, 2016, 05:07:59 PM
Quote from: g3gg0 on February 27, 2016, 11:04:37 AM
if you redownload, it might be a bit faster.

mlvfs_wrap_getattr(1746): caught exception 0xC000001D at address 0xB664A3F3

As soon as i try to open virtual disk :(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 27, 2016, 05:32:04 PM
Quote from: mothaibaphoto on February 27, 2016, 05:07:59 PM
mlvfs_wrap_getattr(1746): caught exception 0xC000001D at address 0xB664A3F3

Strange. None of exceptions here. And this is not the exception (mlvfs_wrap_read: 0xC0000005) I've been getting  in earlier builds, but when it was happening, I'm sure, cause of it was antivirus real time protection. Do you run mlvfs with administrator rights? Happens with all mlvs?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lanternman on February 27, 2016, 10:43:33 PM
Hi guys,

I'm on Windows 10 x64 and using the new port with Dokan and mlvfs x86.

Tried mounting my MLV video folder on a random drive letter (X or Z) and there weren't errors, but the drive did not appear in my windows file explorer. any idea how I can get it to show up on my Premiere Pro CC?
http://imgur.com/aPaKGJo
(http://i.imgur.com/aPaKGJo.png)
(http://i.imgur.com/loDyJq3.png)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on February 27, 2016, 10:49:54 PM
Are you able to access localhost:8000 in your web browser?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 27, 2016, 11:41:30 PM
@dmilligan:
i created a new branch (https://bitbucket.org/dmilligan/mlvfs/branch/path_rework) which is working already well on windows.
it features a proper subdir handling and rename/delete support without errors.

can you test if it does it compile on osx?
thanks :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 27, 2016, 11:59:48 PM
no, O_BINARY is not defined, nor is _unlink, and there are some implicit function declarations

in addition theres a bunch of const warnings for modifying the 'path' parameter, which is declared const, perhaps its best to make a copy
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 28, 2016, 12:04:24 AM
ah it is complaining about the implicit const to non-const conversion in e.g. mlvfs_resolve_path?
how picky clang is :)


thanks, found it. indeed modified const string.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 28, 2016, 12:26:40 AM
seems to be working though

OSX Finder has issues copying files in and such but it always has. Command line cp works fine, except it reports: fchmod failed: Function not implemented (which has also been the case previously)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 28, 2016, 12:34:00 AM
okay fixed these issues in repository hopefully. yay cool, looks good then.

i also added unlink and stafs (which returns fake data), it wasn't implemented before.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lanternman on February 28, 2016, 03:56:43 PM
No I am unable to. localhost = 404 not found.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on February 28, 2016, 04:11:38 PM
Check:
Do you have VC++ Runtime installed? Install both, shouldn't bother.
Which Dokany version installed?
Do you have started CMD as Administrator?
Running
net start|find /I "DokanMounter"
should report
E:\MLVFS_x86\MLVFS_x86>net start|find /I "DokanMounter"
   DokanMounter
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 28, 2016, 04:15:16 PM
404 means that there is something missing. did you run it from the directory where it is installed?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 28, 2016, 05:32:07 PM
g3gg0, checked widows is uptodate but dokan still has installation error 0x80091007 - hash value is not correct
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Canon eos m on February 28, 2016, 05:39:02 PM
g3gg0, if i use the old dokan on the opening page then the error is - the program cant start because dokanfuse.dll is missing from your computer. try installing the program to fix this problem
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 28, 2016, 07:45:33 PM
@g3gg0, dmilligan

compiled path_rework branch, win version of mlvfs in VS
commit e7a8e68 - with lot of warnings but ok
following commits (3 more) - failed with errors

mlvfs.h(137): error C2010: '.': unexpected in macro formal parameter list
mlvfs.h(139): error C2010: '.': unexpected in macro formal parameter list

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 28, 2016, 08:12:32 PM
ah, boo microsoft

variadic macros work fine in gcc and clang
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 28, 2016, 09:06:29 PM
ok, maybe fixed, try again
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 28, 2016, 10:02:07 PM
> Error   C2039   '_unlink': is not a member of 'fuse_operations'   mlvfs   D:\cygwin\home\g3gg0\git\mlvfs\mlvfs\main.c   1808      Build      
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on February 28, 2016, 10:04:21 PM
you can use _unlink on windows too btw
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on February 29, 2016, 11:32:56 AM
Compile latest path_rework branch -> OK! (with lots of warnings as usual)

Tested plain copy average cpu/speed: 3.5-4%/105-110mb/sec
Passtrough is working exept deletion as expected. No nasty exceptions so far.

Nice :)

Congrats!
bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on March 01, 2016, 06:01:40 AM
So, I need to compile myself? I'm not sure if it will work with cygwin i just managed to compile ML itself...
Binaries from mlvfs_optimized.zip are the last one worked fast not furious for me with Resolve.
But now they claims for dokanfuse.dll while last MLVFS has dokanfuse1.dll.
I just renamed dokanfuse1.dll to dokanfuse.dll and happy until developers finally polish this great tool.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on March 01, 2016, 08:02:10 AM
If you wanna test latest bin pm me. Can't just post it here. Would be interesting to see because you got some exception recently.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on March 01, 2016, 08:42:46 AM
Thanks, I appreciate your willing to help but I already converted footage i had with described setup and now have no more records and spare time, so lets things move as it could.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ruber on March 01, 2016, 12:46:07 PM
Hi guys,

I am in windows 10

I actually use Pismo and convert mlv files directly in davinci resolve. Is really smooth.

But Resolve did not recognized audio and video linked so I spent hours relinking audio and video together

Is there a solution? (newer version, or a different software than Pismo)?

Thank you. Cheers.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lanternman on March 01, 2016, 12:48:38 PM
hi guys, MLVFS finally worked in my other desktop.

1 question: How do I quickly add ALL clips in 1 folder to Premiere Pro CC, instead of having to select the 1st DNG of EVERY short clip manually?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 01, 2016, 12:58:19 PM
Quote from: ruber on March 01, 2016, 12:46:07 PM
Is there a solution? (newer version, or a different software than Pismo)?
Yes, start reading here: http://www.magiclantern.fm/forum/index.php?topic=13152.msg161902#msg161902
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ruber on March 01, 2016, 07:36:18 PM
thanks dmilligan

kind as usual  ;)

I'll download it and install
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 01, 2016, 09:40:29 PM
Shot some test MLV_Lite files on 5D3 running 123 today earlier and just tried using the latest MLVFS (Version: 93684e7 2016-02-27) to check out the files.

The WebGUI works like normal but then the Mount folders are empty. Both of them.

Here's part of the log from the beginning since the entire code is too long for this post.

Last login: Mon Feb 29 14:55:41 on console
Apples-Macintosh-10:~ DeafEyeJedi$ cat ~/.mlvfs.log
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2141.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2142.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2143.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2147.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2144.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2145.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2151.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2146.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2153.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2152.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon-precomposed.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2156.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2159.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2248.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2246.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2249.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2250.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2251.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2258.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2301.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2257.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon-precomposed.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2302.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/favicon.ico...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2304.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2303.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2308.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2306.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2307.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2305.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2309.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2310.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2316.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2313.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2312.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2314.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2311.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon-precomposed.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2315.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2321.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2319.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2320.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2318.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2317.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2141.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2146.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2145.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2144.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2143.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2142.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon-precomposed.png...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/favicon.ico...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2147.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2151.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2248.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/apple-touch-icon.png...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2153.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2152.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2156.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2246.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2250.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2251.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2249.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2159.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2258.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2303.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2302.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2301.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2257.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2304.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2306.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2309.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2308.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2307.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2305.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2310.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2314.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2316.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2313.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2312.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2311.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2315.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2320.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2321.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2319.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2318.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2317.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2141.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2142.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2143.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2147.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2144.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2148.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2145.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2151.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2146.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2153.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2152.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2156.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2159.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2248.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2246.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2249.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2250.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2251.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2258.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2301.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2257.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV...
/Volumes/KB_128GB/DCIM/100EOS5D/M16-2256.MLV: Error reading frame headers: vidf block for frame 0 was not found
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2302.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2303.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2304.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2305.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2306.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2308.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2307.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2310.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2309.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2311.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2314.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2313.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2312.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2315.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2316.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2320.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2319.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2318.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2317.MLV...
webgui: analyzing /Volumes/KB_128GB/DCIM/100EOS5D/M16-2321.MLV...
webgui: scanning /Volumes/KB_128GB/DCIM/100EOS5D/...


Should I consider downgrading to previous versions of MLVFS? I'm also running these same files with MLP atm and all seems fine and running like normal though I still have to wait for the final results.

Will report back and Thanks, D!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 01, 2016, 10:14:19 PM
The beginning of the log tells me nothing, it's old (same as the one you posted before). Send the end of the log, upload the whole log somewhere, or preferably: delete the log before running MLVFS, then run MLVFS with the problem files and then send the log that is generated so that it's just the info the run that failed.


mv ~/.mlvfs.log ~/old.mlvfs.log
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 01, 2016, 10:28:54 PM
Okay something odd is going on. I just randomly decided to try AE and they show up fine in there. I can see the files in the Mount folder. However, when clicking on DAvinci Resolve within Web GUI -- thats when the files disappear inside the mount folder and thus I can't see the files in DR12.

Here's log from while in DR mode:

Last login: Tue Mar  1 13:24:29 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ cat ~/.mlvfs.log
webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1246.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1251.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1247.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1252.MLV...
webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1246.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1247.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1251.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1252.MLV...
webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
Apples-Macintosh-10:~ DeafEyeJedi$



Here's the log from while in Default mode:

Last login: Tue Mar  1 13:24:31 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ Last login: Tue Mar  1 13:24:29 on ttys000
Apples-Macintosh-10:~ DeafEyeJedi$ cat ~/.mlvfs.log
webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1246.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1251.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1247.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1252.MLV...
webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1246.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1247.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1251.MLV...
webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1252.MLV...
webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MDeafEyeJedi  ttys000                   Tue Mar  1 13:27   still logged in
DeafEyeJedi  ttys000                   Tue Mar  1 13:24 - 13:26  (00:02)
DeafEyeJedi  ttys000                   Tue Mar  1 13:24 - 13:24  (00:00)
DeafEyeJedi  ttys000                   Tue Mar  1 13:23 - 13:24  (00:00)
DeafEyeJedi  ttys000                   Tue Mar  1 13:22 - 13:23  (00:01)
DeafEyeJedi  ttys000                   Tue Mar  1 12:36 - 12:41  (00:04)
DeafEyeJedi  ttys000                   Sat Feb 27 14:52 - 15:20  (00:28)

wtmp begins Fri Feb 26 19:16
Apples-Macintosh-10:~ DeafEyeJedi$ Apples-Macintosh-10:~ DeafEyeJedi$ cat ~/.mlvfs.log
-bash: Apples-Macintosh-10:~: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1246.MLV...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1251.MLV...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1247.MLV...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1252.MLV...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1246.MLV...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1247.MLV...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1251.MLV...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: analyzing /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/M01-1252.MLV...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ webgui: scanning /Users/appleuser/Desktop/SHOGUN vs MLV_Lite/MLVLite/...
-bash: webgui:: command not found
Apples-Macintosh-10:~ DeafEyeJedi$ Apples-Macintosh-10:~ DeafEyeJedi$
-bash: Apples-Macintosh-10:~: command not found
Apples-Macintosh-10:~ DeafEyeJedi$
Apples-Macintosh-10:~ DeafEyeJedi$


Now I am baffled at why the log is coming out differently even tho it's using the same exact set of MLV Lite files?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 01, 2016, 10:32:23 PM
The naming scheme option changes the file/folder names, so if you are going to do it, then you should do it first, immediately before doing anything else like browsing files. Otherwise you are ending up in folders that don't exist any more (b/c you changed the naming scheme for them).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on March 02, 2016, 12:53:41 AM
It's not a user error issue. It's an issue with the new Mac build. I'm using MLVS and Resolve in the same manner I've been using them for almost a year. Choosing the Resolve naming scheme at launch prior to launching Resolve results in the MLVS folders appearing empty in the Resolve media window. In previous builds a thumbnail image appeared in place of a folder. Leaving the default naming scheme in place at launch results in a single file appearing within the MLVS folders in the Resolve media window (appearing as a thumbnail image). I can't open any of my Resolve MLVS projects because they were all created with the Reslove naming scheme.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 02, 2016, 04:44:42 AM
There's been some rework to the path handling and the resolve naming scheme had not yet been implemented (I didn't realize this). I've just implemented it and uploaded a new build. The actual naming scheme had to change slightly b/c of the new way we are doing things, sorry if this breaks your existing projects.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: newsense on March 02, 2016, 07:43:45 AM
I'm using MLVFS to convert my MLV files and it works great. I want to use a camera profile in ACR but none show up. the stock camera profiles include profiles for the 650D but in ACR my camera is listed as Rebel T4i, would that info be stored in the DNG's produced by MLVFS?
Title: Please, white and black levels in web gui
Post by: mothaibaphoto on March 02, 2016, 08:56:26 AM
Some time ago I realised that MLVP produce dng that looks something different in Resolve.
Difference among other convertors was minor, while with MLVP - huge.
Same time ACR gets confused with predefined camera profiles, only "Embedded" looks good.
Author responded that its due to different white level - MLVP use 16383.
Moreover, it's possible to change this value.
Need to say my camera is always 16383 in video mode(okay, while not tricked with adtg_gui :) ).
MLVFS seems use 15000 hardcoded.
Today I tried to tune MLVFS dng with exiftool.
Need to say my exiftool is very choosy regarding changing these tags, white level works only with -IFD0:
exiftool -IFD0:WhiteLevel=16383 *.dng -overwrite_original
And ... I didn't find the way to change black level(from 2047 to 2044)
so I don't know if it matter or not but changing white level alone makes difference.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: newsense on March 02, 2016, 09:17:18 AM
Quote from: mothaibaphoto on March 02, 2016, 08:56:26 AM
Some time ago I realised that MLVP produce dng that looks something different in Resolve.
Difference among other convertors was minor, while with MLVP - huge.
Same time ACR gets confused with predefined camera profiles, only "Embedded" looks good.
Author responded that its due to different white level - MLVP use 16383.
Moreover, it's possible to change this value.
Need to say my camera is always 16383 in video mode(okay, while not tricked with adtg_gui :) ).
MLVFS seems use 15000 hardcoded.
Today I tried to tune MLVFS dng with exiftool.
Need to say my exiftool is very choosy regarding changing these tags, white level works only with -IFD0:
exiftool -IFD0:WhiteLevel=16383 *.dng -overwrite_original
And ... I didn't find the way to change black level(from 2047 to 2044)
so I don't know if it matter or not but changing white level alone makes difference.
Thanks but I think I'll wait to see if anyone has a proper fix or try another app but I do appreciate the help
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on March 02, 2016, 09:41:33 AM
Quote from: newsense on March 02, 2016, 09:17:18 AM
Thanks but I think I'll wait to see if anyone has a proper fix or try another app but I do appreciate the help
Sorry, my post was not addressed to your problems.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on March 02, 2016, 09:42:13 AM
Compiling under VS fails again:

>main.obj : error LNK2001: unresolved external symbol slre_match
>mlvfs.exe : fatal error LNK1120: 1 unresolved externals
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 02, 2016, 01:09:36 PM
Quote from: bouncyball on March 02, 2016, 09:42:13 AM
Compiling under VS fails again:
yep, I know, see the commit messages

Quote from: mothaibaphoto on March 02, 2016, 08:56:26 AM
MLVFS seems use 15000 hardcoded.
Nope. It uses the value in the MLV file, same with black level. If it's wrong, then it's wrong in the MLV file => therefore you need to go to the source of the MLV file to fix the issue.

Quote from: mothaibaphoto on March 02, 2016, 08:56:26 AM
And ... I didn't find the way to change black level(from 2047 to 2044)
so I don't know if it matter or not but changing white level alone makes difference.
White level and black level are objective values that are either correct or incorrect. There is no "tuning" you should do to these values to try and adjust the "look" of your image.

Quote from: newsense on March 02, 2016, 07:43:45 AM
the stock camera profiles include profiles for the 650D but in ACR my camera is listed as Rebel T4i, would that info be stored in the DNG's produced by MLVFS?
Yep, and again this is something that comes out of the MLV file.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: newsense on March 02, 2016, 02:11:16 PM
Quote from: dmilligan on March 02, 2016, 01:09:36 PM
yep, I know, see the commit messages
Nope. It uses the value in the MLV file, same with black level. If it's wrong, then it's wrong in the MLV file => therefore you need to go to the source of the MLV file to fix the issue.
White level and black level are objective values that are either correct or incorrect. There is no "tuning" you should do to these values to try and adjust the "look" of your image.
Yep, and again this is something that comes out of the MLV file.
Just to make sure I understand the UniqueCameraId is stored in the MLV file so it's ML reporting my camera as rebel t4i instead of 650d so this happens before I even convert to DNG?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on March 02, 2016, 02:26:21 PM
You are double posting http://www.magiclantern.fm/forum/index.php?topic=16738.msg163431#msg163431.
Your camera can either register the t4i or with 650D tag, it,s nothing wrong with either one. If acr plugins or other applications can,t read that tag you can change the tag to whatever you want with exiftool.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on March 02, 2016, 02:50:38 PM
Quote from: dmilligan on March 02, 2016, 04:44:42 AM
There's been some rework to the path handling and the resolve naming scheme had not yet been implemented (I didn't realize this). I've just implemented it and uploaded a new build. The actual naming scheme had to change slightly b/c of the new way we are doing things, sorry if this breaks your existing projects.

Okay, it appears we're back in business. MLVs are loading in Resolve and the thumbnails are visible. No problem re the naming scheme change, I can run an older version for existing projects.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on March 02, 2016, 04:32:20 PM
Tried to reproduce this issue (http://www.magiclantern.fm/forum/index.php?topic=13152.msg159718#msg159718), but couldn't (probably it was fixed meanwhile).

Instead, I've found an issue with webgui on Linux (which used to work): it gives a blank page, and this message in console:

load_resource: fopen error: html_template.html


It doesn't seem to look in the right path - getcwd() returns the directory where I've started mlvfs from.

The old version where the webgui used to work didn't use an external HTML template - it was hardcoded in webgui.c.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on March 02, 2016, 05:07:47 PM
Quote from: dmilligan on March 02, 2016, 01:09:36 PM
yep, I know, see the commit messages

@dmilligan: Ah, my bad. Added slre source and header to the project and it compiled flawlessly :)
testing...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 02, 2016, 06:00:15 PM
Quote from: a1ex on March 02, 2016, 04:32:20 PM
It doesn't seem to look in the right path - getcwd() returns the directory where I've started mlvfs from.
That's odd, it just sends a relative path to fopen. Maybe it doesn't like the mode option "rb"?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mothaibaphoto on March 02, 2016, 06:07:51 PM
Quote from: dmilligan on March 02, 2016, 01:09:36 PM
Nope. It uses the value in the MLV file, same with black level. If it's wrong, then it's wrong in the MLV file => therefore you need to go to the source of the MLV file to fix the issue.
So, it's hardcoded in raw.c:
#define WHITE_LEVEL 15000
Actually, raw.c has routine to define white level - autodetect_white_level, but it's not called,
at least not in video mode: i uncommented debug message inside it and that "White..."
didn't appears on screen. Is that a bug?
Quote from: dmilligan on March 02, 2016, 01:09:36 PM
White level and black level are objective values that are either correct or incorrect. There is no "tuning" you should do to these values to try and adjust the "look" of your image.
It's up to you, whether to implement or not the feature I suggest to your project.
Just let me decide myself what and how I can "tune".
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on March 02, 2016, 11:50:43 PM
Quote from: dmilligan on March 02, 2016, 06:00:15 PM
That's odd, it just sends a relative path to fopen.

That relative path is just "html_template.html", and I guess it should include "data" (maybe also find somehow the directory of the executable, or expect some specific path where it should be installed).

So, I started mlvfs from the data directory (../mlvfs [options]) and the webgui works again.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 03, 2016, 01:55:23 AM
I intended to just have everything in the same directory to avoid cross platform discrepancies (like different path separators and whatnot). I realize it's confusing for those things to be in the "data" directory in the source tree. I'm open to suggestions.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: QuickHitRecord on March 04, 2016, 09:39:08 AM
After updating to the latest nightly build today, I also installed the newest version of MLVFS and decided to do another body cap ISO test (see the video description for more details):

https://vimeo.com/157694455

I'm not convinced that changing the boxes in the web interface is doing anything. The clips with "Bad Pixel Fix" checked don't seem to be any different than those without it. What am I doing wrong? I restarted Resolve after making the changes to the web interface so that they would take.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 04, 2016, 09:56:31 AM
Quote from: QuickHitRecord on March 04, 2016, 09:39:08 AM
I'm not convinced that changing the boxes in the web interface is doing anything. The clips with "Bad Pixel Fix" checked don't seem to be any different than those without it. What am I doing wrong? I restarted Resolve after making the changes to the web interface so that they would take.

Maybe you forgot to 'refresh' the Web interface after making your adjustments checked? I've had this happened to me occasionally whenever I forget. [emoji6]

@dmilligan - I ran a few test MLV_Lite files with latest MLVFS (Thanks for the update) and it works like normal with these files.

However, when I try running a folder with regular MLV files which then shows me the web gui error. Strange?

Link to MLVFS Log ... https://mega.nz/#F!alU0DTJa!9Z-kho7OPZPg7msyltc9-g

*edit*

Ah, Ha ... Looks like I managed to figured it out on my end (not sure why this wasn't an issue in the past) but I decided to rename my Folders that didn't have any (spaces or ' , _ etc) and reran latest MLVFS with all of my old MLV files from varies of cameras and all seems to be working like normal again. Whew!

Sorry for the False alarm even tho I could swear that Ive used Spaces or Punctuation Marks with no problems (according to my memory) or no?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: QuickHitRecord on March 04, 2016, 09:17:28 PM
Quote from: DeafEyeJedi on March 04, 2016, 09:56:31 AM
Maybe you forgot to 'refresh' the Web interface after making your adjustments checked? I've had this happened to me occasionally whenever I forget. [emoji6]

You were right. Thanks for the tip. I re-did the test, and expanded on it:

https://vimeo.com/157694455

Some takeaways from this exercise:
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 04, 2016, 10:27:40 PM
Quote from: DeafEyeJedi on March 04, 2016, 09:56:31 AM
Sorry for the False alarm even tho I could swear that Ive used Spaces or Punctuation Marks with no problems (according to my memory) or no?
Spaces, quotes, and all sorts of manner of strange characters work just fine in the file system. It appears that some characters cause files not to show up in the webgui, this is probably b/c the json/url requests aren't being escaped properly.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 05, 2016, 02:02:52 AM
Think it should be fixed now.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on March 05, 2016, 09:49:24 PM
Hi all,

I love the concept, but I cant get Premiere CC to properly deal with the files. After importing the sequence, scrubbing works nowhere (media bowser/bins/timeline), hence I initialy thought it does not work at all, BUT it plays back with no problem after a pausing a moment. Mind you that I have no such problems with dng extracted using raw2cdng 1.7.4

Alas, the biggest issue remains rendering, where I get "error compiling movie. unknown error" and therefore being unable to export at all! Again, this does not occur with dng's extracted with raw2cdng 1.7.4.. Here's what I get in console after export attempt:

H:\DCIM\100EOS5D/M04-1511.MLV: Error reading frame headers: vidf block for frame 0 was not found
H:\DCIM\100EOS5D/M04-1511.MLV: Error reading frame headers: vidf block for frame 0 was not found
H:\DCIM\100EOS5D/M04-1511.MLV: Error reading frame headers: vidf block for frame 0 was not found
H:\DCIM\100EOS5D/M04-1511.MLV: Error reading frame headers: vidf block for frame 0 was not found
mlvfs_wrap_read(1497): caught exception 0xC0000005 at address 0x7061D72E


I'm running this on Win7 SP1


Context Menu for Win

I would also like to make a small contribution to windows users adding a context menu option "Mount with MLVFS" to your folders. Technically registry entry only should suffice, but I found that webGUI loads blank and console outputs fopen errors without setting command path to mlvfs folder of residence before executing, hence the bat file..
Obviously, you'll have to change paths to suit your requirements.

mlvfs.reg (execute this file to add context menu option for your folders):
Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\MLVFS]
@="Mount with MLVFS"

[HKEY_CLASSES_ROOT\Directory\shell\MLVFS\command]
@="\"C:\\Program Files (x86)\\MLVFS_x86\\mlvfs.bat\" \"%1\""


mlvfs.bat (place this next to mlvfs.exe)
@echo off
c:
cd "c:\Program Files (x86)\MLVFS_x86"
mlvfs.exe x:\ --mlv_dir=%1
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on March 06, 2016, 02:52:20 PM
Thanks. Context menu is nice.

I've tried this method some time ago and mlvfs path was a problem for webgui as you described and I ended up with .bat command file as you did :). Besides I always run mlvfs with admin rights b/c dokan sometimes could not properly unmount the drive. I've tested with mlvfs in compatibility mode as admin (opens 2 console windows :P if executed from bat) and with "runas.exe /user:Administrator ..." command and it sorta worked the way I'd like but it complicated things a bit (enter password etc) and I gave up and just use the shortcut with "run as administrator" checked on or just run it from FarManager console session.

I think the best way to solve this problem for all platforms is to hardcode the html and js resource into mlvfs binary itself (early versions were this way if I'm not mistaken). However, I guess, David had his own reasons for current implementation.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 06, 2016, 07:02:50 PM
Constantly recompiling to fix html, css, and ajax is annoying. Writing all the html as string literals is also quite cumbersome, and I don't know of a good, simple, reliable, cross-platform way of inserting text documents into binaries. Some particular method might work well for gcc/make but not LLVM or VS and vice versa. Much simpler to just load them. With the whole focus pixel thing we have the need for additional resource files as well, so it was starting to get a little out of hand to keep everything in the binary. Plus it means if/when new focus pixel cameras are added, pixel maps can be added without recompiling, and people can make their own maps to fix things like troublesome hot/dead pixels.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on March 06, 2016, 07:53:32 PM
the only proper solution is making some kind of "resource" (i.e. zipping all files) that is either linked (depends on which linker you use) or converted to a .c with some  "char resourceData[] = { 0xaa, 0xbb, ... }".
but thats a bit annoying as you need additional helpers.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on March 07, 2016, 03:37:00 PM
any idea about the unhandheld (haha, sorry/not sorry) exceptions and errors reading frame headers, which I am guessing are the causes of problematic handling by premiere?...

I've described those few posts above..
http://www.magiclantern.fm/forum/index.php?topic=13152.msg163728#msg163728

Please let me know if there is any troubleshooting or testing I could do in helping to improve it!

Thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Roberto Mena on March 08, 2016, 10:34:45 AM
Hi Guys. Thanks for all the amazing work. I downloaded FUSE and MLVFS but I am not getting the "Services" option when I right click a folder containing mlv files, showing at the bottom of the window near the "Tags" section. What am I doing wrong? Thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on March 08, 2016, 01:45:49 PM
Quote from: voblaunsane on March 07, 2016, 03:37:00 PM
any idea about the unhandheld (haha, sorry/not sorry) exceptions and errors reading frame headers, which I am guessing are the causes of problematic handling by premiere?...

I never experienced frame header error with premiere or anything else. You should double check that particular mlv file in other tools.

mlvfs_wrap_read(1497): caught exception 0xC0000005 - was very common (in early windows mlvfs builds) until I disabled antivirus real time protection.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on March 09, 2016, 12:55:14 AM
Quote from: voblaunsane on March 07, 2016, 03:37:00 PM
any idea about the unhandheld (haha, sorry/not sorry) exceptions and errors reading frame headers, which I am guessing are the causes of problematic handling by premiere?...

can you check if the file with vidf errors reads fine using mlv_dump?
can you run mlvfs.exe with parameter -d and upload the whole log? (append "> log.txt" to log into a file)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: USADavid on March 10, 2016, 01:29:38 AM
Hey guys,

long time creep, but never had a reason to post anything. Great work everyone is doing here. Love the simplicity and cross-platform capability of MLVS.
Unfortunately, I've had issues with the sound for a while now, blamed it on me first, but after trying a lot of different things never seem to be able to fix it.

I'm running the latest MLVFS on my Mac (2016-03-04) and all my MLVs which are around > 2GB have an empty wav file. The wav files still show up even with the alleged size, but there's no sound. If I open up the MLVs in MlRawViewer, it plays just fine. I'm on the latest Fuse and even tried the Beta, also the ~./mlvfs.log file doesn't offer any clues on why this occurs.

Any help would be greatly appreciated.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on March 10, 2016, 05:48:46 AM
Quote from: g3gg0 on March 09, 2016, 12:55:14 AM
can you check if the file with vidf errors reads fine using mlv_dump?
can you run mlvfs.exe with parameter -d and upload the whole log? (append "> log.txt" to log into a file)

Thanks!

I tried all combinations of commands I could've possibly ran, alas was unable to reproduce, very strange because I received those all first day of use and then next day it was gone like nothing ever happened! Also deleted the files before reading you post, aww..

Yet main issue being unable to render persists! So I ran mlvfs with -d 2>log.txt mounting a folder with one sequence (3 files mlv/m00/idx), created new premiere project, located sequence in media browser, dropped it straight on the timeline and hit render.  This produced "error compiling movie. unknown error" and 54.6MB logfile here: https://drive.google.com/file/d/0Bzfm11RAqkBzaXgzOTUwQ1dIYUU/view?usp=sharing (https://drive.google.com/file/d/0Bzfm11RAqkBzaXgzOTUwQ1dIYUU/view?usp=sharing). Not knowing what I'm looking for tried to do some detective work and the following seems to be all over the place: CreateFile status = 0 - lastError = 2

This mlv was recorded with no sound, but I see the following in report alot.
CreateFile : \M04-1513.MLV\M04-1513.wav

I left an instance of mlvfs with -d running while typing this post, new log file is nearly 2GB - I'm scared :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on March 10, 2016, 08:19:15 PM
if you extract the files using mlvfs and let premiere work on the extracted files, does that work?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on March 10, 2016, 08:44:05 PM
Found a few things here:

1. Finally I managed to render mlvfs on premiere! I found that 25fps material works flawlessly at a lighting speed, when 23.976fps is impossible to scrub and it doesn't render "error compiling movie. unknown error"... when that same 23.976fps mlv, extracted with raw2cdng works just fine. Any ideas?

I logged the activity with separately mounted differing fps files here: https://drive.google.com/file/d/0Bzfm11RAqkBzN0dnV05kMGpmSms/view?usp=sharing

2. Mounting a full CF results in empty mlv folders. localhost:8000 reports 0 frames in files found. Freeing some space resolves the issue. It could be nice have mlvfs reporting "not enough disk space".. Otherwise it gives the impression that files are damaged..

Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on March 10, 2016, 08:49:47 PM
Quote from: g3gg0 on March 10, 2016, 08:19:15 PM
if you extract the files using mlvfs and let premiere work on the extracted files, does that work?

no. same issues with 23.976. Works fine, same as on the fly with 25fps..

Using mounted 23.976 material renders always fail at a particular stage, until I drag the clip to another sequence, then it gets a new fail point. More often then not fail point is 99%.. The extract from mlvfs, even in new sequence, always fails at the similar point.. but same material extracted again, gets a new fail point .. makes sense?

Also caught the exception, actually all 6 of them, but was running many tests and can not reproduce, here's the log: https://drive.google.com/file/d/0Bzfm11RAqkBzb0xwbHpTT3JqSEE/view?usp=sharing
Title: Dokany trouble
Post by: Markus on March 12, 2016, 07:00:53 PM
I've installed dokan to try it out on win10 64bit but can't get it to function properly. I can mount the files and se the dng-files inside but if i try to open anything i just get: "load_chunks: fopen error: No such file or directory"

Anyone have any idea of what could be the problem?

v. 0.8.0 of dokan and MLVFSx86 from first post link
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on March 13, 2016, 05:35:33 AM
@Markus

First try using the latest supported versions of Dokan and MLVFS.

Links to these can be found here:  http://www.magiclantern.fm/forum/index.php?topic=13152.msg163041#msg163041

Note that you will need to uninstall Dokan first before reinstalling the latest version. The MLVFS download totally replaces your existing install instance.

Also, do the paths to your MLV files contain spaces?  If so, create a path without spaces and try again.

If it still fails after that, report back with some screen shots demonstrating successful Loading of Dokan, File mounts (A localhost one) and the error message when trying to access a specific DNG.

What application are you using to access the DNG's. The issue may be with the application and not MLVFS. what happens if you just double click on a DNG to view the image with windows default viewer.

@DMilligan - Might be worthwhile to update the links in the 1st post to reflect the latest supported versions.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on March 13, 2016, 05:50:19 AM
@Developers

Just a note to advise that Dokan v1.0.0-RC2 is available and, at least on my W10 system is working error free. Tested with single and multi-chunk MLV's.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on March 13, 2016, 12:04:04 PM
@Ottoga: Thanks for pointing it out!

@g3gg0, dmilligan: delete/copy file/dir in passthrough mode is OK! for RC2 :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on March 14, 2016, 12:17:22 PM
@Ottoga Thanks for reply! I'll try it out as soon as possible. Excited of getting this to work instead of pismo. Fix pattern noise seems extremely useful.
Does anyone know if it is possible to process dng - sequences for fixed pattern noise if you haven't saved the mlv's?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on March 14, 2016, 12:30:03 PM
@Markus

QuoteDoes anyone know if it is possible to process dng - sequences for fixed pattern noise if you haven't saved the mlv's?

From memory, you might be able to use mlv_dump to recreate an MLV file from a DNG series. One of the Devs may be able to confirm that.

If this is confirmed, then the answer to your question is likely to be "yes".

Oh, I also uninstalled Pismo and rebooted my PC before installing Dokan. I didn't see any point having two services running in the background that are essentially trying to do the same thing. Just something else to consider.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on March 14, 2016, 09:22:59 PM
Not sure if this a issue use the new dual iso video for 5d2 but i'm have problems with the latest mlvfs for mac.
sometimes when i toggle on dual iso processing with default setting it doesn't always process  the dual iso .
Tried deleting the .idx file re-mounting the file , seems find on a older version i have on my hackintoss .
I'm working off a SSD , i see if there's any log files .
Also Is dual iso processing normally very slow ? I using AMaZe for Interpolation , Alias Map & Fullres Blending enable.
First time working with dual iso , with non dual iso there hardly any delay and can but edited etc.. in realtime in Resolve
but this is not possible with dual iso it seems , i play around with some of the setting and see if i can improve on this .
Just curious , when the web page opens and read the files etc.. shouldn't it say it's "dual iso" ? of give the two iso setting ?
all i see is the lower number e.g. 100 out of 100/800
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on March 14, 2016, 10:16:15 PM
The dual ISO algorithm is just very complex and slow, there's no way around it. There is a fast preview mode, but it is very low quality. So the usefulness of dual ISO in relation to the MLVFS "on the fly" paradigm is rather limited. Personally, I think dual ISO video is just not worth the effort and drawbacks (aliasing, loss in resolution, etc) and most people want to use simply because it sounds cool and somewhat of a placebo effect.

As for the webgui, there is no dual ISO related metadata in the MLV file (IMO there probably should be). So the only way to figure out if something is dual ISO or not is to actually do some complex analysis on a frame. Doing this would bog down the gui for everyone, not just dual ISO users.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on March 14, 2016, 10:58:56 PM
Sound good , i understand now thanks.



Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on March 16, 2016, 07:24:06 PM
There seems to be an issue with the timecode on 23.98fps material.

Using exiftool to look at the timecode from a DNG frame numbered 0 and the timecode is 00:00:00.00 as expected. All is fine until you get to frame 24 and the timecode jumps to 00:00:01.01. This +1-frame is cumulative adding a frame to the timecode every second.

I don't know if this is intentional or if it affects any other framerate.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on March 16, 2016, 07:43:17 PM
Still can't get MLVFS to work. (Win10 64bit)
I installed the new version of mlvfs with x64 bit support and dokan 1RC2 (tryed RC1 as vell) Uninstalled before reinstalling with boot in between.
MLVFS version from this post http://www.magiclantern.fm/forum/index.php?topic=13152.msg163041#msg163041  (http://www.magiclantern.fm/forum/index.php?topic=13152.msg163041#msg163041)
Used the following string to mount: "mlvfs_x64.exe C:\MLV --mlv_dir=F:\DCIM\100EOS5D\"
The MLV-file I tryed to mount was about 4.5 gig in size
This time I can not even enter the mounted catalog and keep getting the following error message when I try to:
mlvfs_wrap_getattr(1746): caught exception 0xC000001D at address 0x5AF8A3F3

even tryed going offline and deactivating avast anti-virus program...
Also uninstalled Pismo mounting.

So now I'm at a loss what more to try.
Anyone got any suggestions?
Any help would be greatly appreciated.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 16, 2016, 09:16:33 PM
Quote from: dfort on March 16, 2016, 07:24:06 PM
There seems to be an issue with the timecode on 23.98fps material.
Feel free to correct this math, it came from chmee's tool. I suspect it doesn't handle rounding correctly.

    //from raw2cdng, credits: chmee
    int hours = (int)floor((double)frame / framerate / 3600);
    frame = frame - (hours * 60 * 60 * (int)framerate);
    int minutes = (int)floor((double)frame / framerate / 60);
    frame = frame - (minutes * 60 * (int)framerate);
    int seconds = (int)floor((double)frame / framerate) % 60;
    frame = frame - (seconds * (int)framerate);
    int frames = frame % MAX(1,(int)round(framerate));
   
    buffer[*data_offset] = to_tc_byte(frames) & 0x3F;
    if(drop_frame) buffer[*data_offset] = buffer[*data_offset] | (1 << 7); //set the drop frame bit
    buffer[*data_offset + 1] = to_tc_byte(seconds) & 0x7F;
    buffer[*data_offset + 2] = to_tc_byte(minutes) & 0x7F;
    buffer[*data_offset + 3] = to_tc_byte(hours) & 0x3F;


Here's the full output for a 23.976 file

$ exiftool -TimeCodes *.dng
======== M14-1233.MLV-C_000000.dng
Time Codes                      : 00:00:00.00
======== M14-1233.MLV-C_000001.dng
Time Codes                      : 00:00:00.01
======== M14-1233.MLV-C_000002.dng
Time Codes                      : 00:00:00.02
======== M14-1233.MLV-C_000003.dng
Time Codes                      : 00:00:00.03
======== M14-1233.MLV-C_000004.dng
Time Codes                      : 00:00:00.04
======== M14-1233.MLV-C_000005.dng
Time Codes                      : 00:00:00.05
======== M14-1233.MLV-C_000006.dng
Time Codes                      : 00:00:00.06
======== M14-1233.MLV-C_000007.dng
Time Codes                      : 00:00:00.07
======== M14-1233.MLV-C_000008.dng
Time Codes                      : 00:00:00.08
======== M14-1233.MLV-C_000009.dng
Time Codes                      : 00:00:00.09
======== M14-1233.MLV-C_000010.dng
Time Codes                      : 00:00:00.10
======== M14-1233.MLV-C_000011.dng
Time Codes                      : 00:00:00.11
======== M14-1233.MLV-C_000012.dng
Time Codes                      : 00:00:00.12
======== M14-1233.MLV-C_000013.dng
Time Codes                      : 00:00:00.13
======== M14-1233.MLV-C_000014.dng
Time Codes                      : 00:00:00.14
======== M14-1233.MLV-C_000015.dng
Time Codes                      : 00:00:00.15
======== M14-1233.MLV-C_000016.dng
Time Codes                      : 00:00:00.16
======== M14-1233.MLV-C_000017.dng
Time Codes                      : 00:00:00.17
======== M14-1233.MLV-C_000018.dng
Time Codes                      : 00:00:00.18
======== M14-1233.MLV-C_000019.dng
Time Codes                      : 00:00:00.19
======== M14-1233.MLV-C_000020.dng
Time Codes                      : 00:00:00.20
======== M14-1233.MLV-C_000021.dng
Time Codes                      : 00:00:00.21
======== M14-1233.MLV-C_000022.dng
Time Codes                      : 00:00:00.22
======== M14-1233.MLV-C_000023.dng
Time Codes                      : 00:00:00.23
======== M14-1233.MLV-C_000024.dng
Time Codes                      : 00:00:01.01
======== M14-1233.MLV-C_000025.dng
Time Codes                      : 00:00:01.02
======== M14-1233.MLV-C_000026.dng
Time Codes                      : 00:00:01.03
======== M14-1233.MLV-C_000027.dng
Time Codes                      : 00:00:01.04
======== M14-1233.MLV-C_000028.dng
Time Codes                      : 00:00:01.05
======== M14-1233.MLV-C_000029.dng
Time Codes                      : 00:00:01.06
======== M14-1233.MLV-C_000030.dng
Time Codes                      : 00:00:01.07
======== M14-1233.MLV-C_000031.dng
Time Codes                      : 00:00:01.08
======== M14-1233.MLV-C_000032.dng
Time Codes                      : 00:00:01.09
======== M14-1233.MLV-C_000033.dng
Time Codes                      : 00:00:01.10
======== M14-1233.MLV-C_000034.dng
Time Codes                      : 00:00:01.11
======== M14-1233.MLV-C_000035.dng
Time Codes                      : 00:00:01.12
======== M14-1233.MLV-C_000036.dng
Time Codes                      : 00:00:01.13
======== M14-1233.MLV-C_000037.dng
Time Codes                      : 00:00:01.14
======== M14-1233.MLV-C_000038.dng
Time Codes                      : 00:00:01.15
======== M14-1233.MLV-C_000039.dng
Time Codes                      : 00:00:01.16
======== M14-1233.MLV-C_000040.dng
Time Codes                      : 00:00:01.17
======== M14-1233.MLV-C_000041.dng
Time Codes                      : 00:00:01.18
======== M14-1233.MLV-C_000042.dng
Time Codes                      : 00:00:01.19
======== M14-1233.MLV-C_000043.dng
Time Codes                      : 00:00:01.20
======== M14-1233.MLV-C_000044.dng
Time Codes                      : 00:00:01.21
======== M14-1233.MLV-C_000045.dng
Time Codes                      : 00:00:01.22
======== M14-1233.MLV-C_000046.dng
Time Codes                      : 00:00:01.23
======== M14-1233.MLV-C_000047.dng
Time Codes                      : 00:00:01.00
======== M14-1233.MLV-C_000048.dng
Time Codes                      : 00:00:02.02
======== M14-1233.MLV-C_000049.dng
Time Codes                      : 00:00:02.03
======== M14-1233.MLV-C_000050.dng
Time Codes                      : 00:00:02.04
======== M14-1233.MLV-C_000051.dng
Time Codes                      : 00:00:02.05
======== M14-1233.MLV-C_000052.dng
Time Codes                      : 00:00:02.06
======== M14-1233.MLV-C_000053.dng
Time Codes                      : 00:00:02.07
======== M14-1233.MLV-C_000054.dng
Time Codes                      : 00:00:02.08
======== M14-1233.MLV-C_000055.dng
Time Codes                      : 00:00:02.09
======== M14-1233.MLV-C_000056.dng
Time Codes                      : 00:00:02.10
======== M14-1233.MLV-C_000057.dng
Time Codes                      : 00:00:02.11
======== M14-1233.MLV-C_000058.dng
Time Codes                      : 00:00:02.12
======== M14-1233.MLV-C_000059.dng
Time Codes                      : 00:00:02.13
======== M14-1233.MLV-C_000060.dng
Time Codes                      : 00:00:02.14
======== M14-1233.MLV-C_000061.dng
Time Codes                      : 00:00:02.15
======== M14-1233.MLV-C_000062.dng
Time Codes                      : 00:00:02.16
======== M14-1233.MLV-C_000063.dng
Time Codes                      : 00:00:02.17
======== M14-1233.MLV-C_000064.dng
Time Codes                      : 00:00:02.18
======== M14-1233.MLV-C_000065.dng
Time Codes                      : 00:00:02.19
======== M14-1233.MLV-C_000066.dng
Time Codes                      : 00:00:02.20
======== M14-1233.MLV-C_000067.dng
Time Codes                      : 00:00:02.21
======== M14-1233.MLV-C_000068.dng
Time Codes                      : 00:00:02.22
======== M14-1233.MLV-C_000069.dng
Time Codes                      : 00:00:02.23
======== M14-1233.MLV-C_000070.dng
Time Codes                      : 00:00:02.00
======== M14-1233.MLV-C_000071.dng
Time Codes                      : 00:00:02.01
======== M14-1233.MLV-C_000072.dng
Time Codes                      : 00:00:03.03
======== M14-1233.MLV-C_000073.dng
Time Codes                      : 00:00:03.04
======== M14-1233.MLV-C_000074.dng
Time Codes                      : 00:00:03.05
======== M14-1233.MLV-C_000075.dng
Time Codes                      : 00:00:03.06
======== M14-1233.MLV-C_000076.dng
Time Codes                      : 00:00:03.07
======== M14-1233.MLV-C_000077.dng
Time Codes                      : 00:00:03.08
======== M14-1233.MLV-C_000078.dng
Time Codes                      : 00:00:03.09
======== M14-1233.MLV-C_000079.dng
Time Codes                      : 00:00:03.10
======== M14-1233.MLV-C_000080.dng
Time Codes                      : 00:00:03.11
======== M14-1233.MLV-C_000081.dng
Time Codes                      : 00:00:03.12
======== M14-1233.MLV-C_000082.dng
Time Codes                      : 00:00:03.13
======== M14-1233.MLV-C_000083.dng
Time Codes                      : 00:00:03.14
======== M14-1233.MLV-C_000084.dng
Time Codes                      : 00:00:03.15
======== M14-1233.MLV-C_000085.dng
Time Codes                      : 00:00:03.16
======== M14-1233.MLV-C_000086.dng
Time Codes                      : 00:00:03.17
======== M14-1233.MLV-C_000087.dng
Time Codes                      : 00:00:03.18
======== M14-1233.MLV-C_000088.dng
Time Codes                      : 00:00:03.19
======== M14-1233.MLV-C_000089.dng
Time Codes                      : 00:00:03.20
======== M14-1233.MLV-C_000090.dng
Time Codes                      : 00:00:03.21
======== M14-1233.MLV-C_000091.dng
Time Codes                      : 00:00:03.22
======== M14-1233.MLV-C_000092.dng
Time Codes                      : 00:00:03.23
======== M14-1233.MLV-C_000093.dng
Time Codes                      : 00:00:03.00
======== M14-1233.MLV-C_000094.dng
Time Codes                      : 00:00:03.01
======== M14-1233.MLV-C_000095.dng
Time Codes                      : 00:00:03.02
======== M14-1233.MLV-C_000096.dng
Time Codes                      : 00:00:04.04
======== M14-1233.MLV-C_000097.dng
Time Codes                      : 00:00:04.05
======== M14-1233.MLV-C_000098.dng
Time Codes                      : 00:00:04.06
======== M14-1233.MLV-C_000099.dng
Time Codes                      : 00:00:04.07
======== M14-1233.MLV-C_000100.dng
Time Codes                      : 00:00:04.08
======== M14-1233.MLV-C_000101.dng
Time Codes                      : 00:00:04.09
======== M14-1233.MLV-C_000102.dng
Time Codes                      : 00:00:04.10
======== M14-1233.MLV-C_000103.dng
Time Codes                      : 00:00:04.11
======== M14-1233.MLV-C_000104.dng
Time Codes                      : 00:00:04.12
======== M14-1233.MLV-C_000105.dng
Time Codes                      : 00:00:04.13
======== M14-1233.MLV-C_000106.dng
Time Codes                      : 00:00:04.14
======== M14-1233.MLV-C_000107.dng
Time Codes                      : 00:00:04.15
======== M14-1233.MLV-C_000108.dng
Time Codes                      : 00:00:04.16
======== M14-1233.MLV-C_000109.dng
Time Codes                      : 00:00:04.17
======== M14-1233.MLV-C_000110.dng
Time Codes                      : 00:00:04.18
======== M14-1233.MLV-C_000111.dng
Time Codes                      : 00:00:04.19
======== M14-1233.MLV-C_000112.dng
Time Codes                      : 00:00:04.20
======== M14-1233.MLV-C_000113.dng
Time Codes                      : 00:00:04.21
======== M14-1233.MLV-C_000114.dng
Time Codes                      : 00:00:04.22
======== M14-1233.MLV-C_000115.dng
Time Codes                      : 00:00:04.23
======== M14-1233.MLV-C_000116.dng
Time Codes                      : 00:00:04.00
======== M14-1233.MLV-C_000117.dng
Time Codes                      : 00:00:04.01
======== M14-1233.MLV-C_000118.dng
Time Codes                      : 00:00:04.02
======== M14-1233.MLV-C_000119.dng
Time Codes                      : 00:00:04.03
======== M14-1233.MLV-C_000120.dng
Time Codes                      : 00:00:05.05
======== M14-1233.MLV-C_000121.dng
Time Codes                      : 00:00:05.06
======== M14-1233.MLV-C_000122.dng
Time Codes                      : 00:00:05.07
======== M14-1233.MLV-C_000123.dng
Time Codes                      : 00:00:05.08
======== M14-1233.MLV-C_000124.dng
Time Codes                      : 00:00:05.09
======== M14-1233.MLV-C_000125.dng
Time Codes                      : 00:00:05.10
======== M14-1233.MLV-C_000126.dng
Time Codes                      : 00:00:05.11
======== M14-1233.MLV-C_000127.dng
Time Codes                      : 00:00:05.12
======== M14-1233.MLV-C_000128.dng
Time Codes                      : 00:00:05.13
======== M14-1233.MLV-C_000129.dng
Time Codes                      : 00:00:05.14
======== M14-1233.MLV-C_000130.dng
Time Codes                      : 00:00:05.15
======== M14-1233.MLV-C_000131.dng
Time Codes                      : 00:00:05.16
======== M14-1233.MLV-C_000132.dng
Time Codes                      : 00:00:05.17
======== M14-1233.MLV-C_000133.dng
Time Codes                      : 00:00:05.18
======== M14-1233.MLV-C_000134.dng
Time Codes                      : 00:00:05.19
======== M14-1233.MLV-C_000135.dng
Time Codes                      : 00:00:05.20
======== M14-1233.MLV-C_000136.dng
Time Codes                      : 00:00:05.21
======== M14-1233.MLV-C_000137.dng
Time Codes                      : 00:00:05.22
======== M14-1233.MLV-C_000138.dng
Time Codes                      : 00:00:05.23
======== M14-1233.MLV-C_000139.dng
Time Codes                      : 00:00:05.00
======== M14-1233.MLV-C_000140.dng
Time Codes                      : 00:00:05.01
======== M14-1233.MLV-C_000141.dng
Time Codes                      : 00:00:05.02
======== M14-1233.MLV-C_000142.dng
Time Codes                      : 00:00:05.03
======== M14-1233.MLV-C_000143.dng
Time Codes                      : 00:00:05.04
======== M14-1233.MLV-C_000144.dng
Time Codes                      : 00:00:06.06
======== M14-1233.MLV-C_000145.dng
Time Codes                      : 00:00:06.07

The seconds are always correct, but the frame numbers are off.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on March 17, 2016, 12:28:31 AM
@dfort -- I wonder if your Genius Brother can help solve this formula for the TimeCode bug?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 17, 2016, 01:55:56 AM
So this gets close:

    double time = frame / framerate;
    int hours = (int)floor(time / 3600);
    int minutes = ((int)floor(time / 60)) % 60;
    int seconds = ((int)floor(time)) % 60;
    int frames = (int)floor((time - floor(time)) * framerate);
   
    buffer[*data_offset] = to_tc_byte(frames) & 0x3F;
    if(drop_frame) buffer[*data_offset] = buffer[*data_offset] | (1 << 7); //set the drop frame bit
    buffer[*data_offset + 1] = to_tc_byte(seconds) & 0x7F;
    buffer[*data_offset + 2] = to_tc_byte(minutes) & 0x7F;
    buffer[*data_offset + 3] = to_tc_byte(hours) & 0x3F;


But it doesn't "drop" frames at the standardized amount for NTSC (30/1.001 fps) which is: two frames every minute except each tenth minute. It just rounds the exact time to the nearest frame.

I don't know if the same convention applies for 23.976.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on March 17, 2016, 02:55:50 AM
I thought it was because of these lines:

    frame = frame - (seconds * (int)framerate);
    int frames = frame % MAX(1,(int)round(framerate));


The first line isn't rounding the framerate so it is changing 23.976 to 23. I tried changing it but ran into compiling problems that I didn't have before.

As for NTSC drop frame timecode, it only applies to 29.97 and 59.94 frame rates. As far as I know there is no drop frame timecode for 23.976 video, at least I've never encountered it. Someone can do a fact check on that. In any case, the only times I had to deal with drop-frame timecode was on NTSC master tapes that had to be delivered for broadcast. All the editing and VFX workflows that I've encountered used non-drop timecode. In addition, the math is so much easier with non-drop frame timecode when converting from frame numbers to timecode and back.

In other words, if you strip out all the drop-frame code nobody should miss it.

Here's that compiling issue I mentioned. The build script finishes and I get a binary but it won't create a mounted volume or webpage.

./build_installer.sh
rm -f mlvfs dng.o index.o wav.o stripes.o cs.o amaze_demosaic_RT.o hdr.o histogram.o mongoose/mongoose.o webgui.o resource_manager.o lj92.o gif.o patternnoise.o LZMA/7zAlloc.o LZMA/7zBuf.o LZMA/7zBuf2.o LZMA/7zCrc.o LZMA/7zCrcOpt.o LZMA/7zDec.o LZMA/7zFile.o LZMA/7zIn.o LZMA/7zStream.o LZMA/Alloc.o LZMA/Bcj2.o LZMA/Bra.o LZMA/Bra86.o LZMA/BraIA64.o LZMA/CpuArch.o LZMA/Delta.o LZMA/LzFind.o LZMA/Lzma2Dec.o LZMA/Lzma2Enc.o LZMA/Lzma86Dec.o LZMA/Lzma86Enc.o LZMA/LzmaDec.o LZMA/LzmaEnc.o LZMA/LzmaLib.o LZMA/Ppmd7.o LZMA/Ppmd7Dec.o LZMA/Ppmd7Enc.o LZMA/Sha256.o LZMA/Xz.o LZMA/XzCrc64.o
gcc -c -Wall -D__DARWIN_64_BIT_INO_T=1 -D__FreeBSD__=10 -D_FILE_OFFSET_BITS=64 -I/usr/local/include/osxfuse -std=gnu99 -DVERSION="\"30aea14\"" -DBUILD_DATE="\"2016-03-17 01:18:13\"" -DFUSE_USE_VERSION=26 dng.c -o dng.o
In file included from dng.h:27:0,
                 from dng.c:28:
mlvfs.h:107:22: error: unknown type name '__always_inline'
#define FORCE_INLINE __always_inline
                      ^
dng.c:793:8: note: in expansion of macro 'FORCE_INLINE'
static FORCE_INLINE size_t dng_get_image_data_inline(struct frame_headers * frame_headers, uint16_t * packed_bits, uint8_t * output_buffer, off_t offset, size_t max_size, int32_t bpp)
        ^
dng.c:793:28: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'dng_get_image_data_inline'
static FORCE_INLINE size_t dng_get_image_data_inline(struct frame_headers * frame_headers, uint16_t * packed_bits, uint8_t * output_buffer, off_t offset, size_t max_size, int32_t bpp)
                            ^
dng.c: In function 'dng_get_image_data':
dng.c:841:20: warning: implicit declaration of function 'dng_get_image_data_inline' [-Wimplicit-function-declaration]
             return dng_get_image_data_inline(frame_headers, packed_bits, output_buffer, offset, max_size, 8);
                    ^
make: *** [dng.o] Error 1
cp: mlvfs/mlvfs: No such file or directory
...
...
created: /Users/rosiefort/mlvfs/pack.temp.dmg
chmod: /Volumes/MLVFS/.Trashes: Permission denied
"disk7" unmounted.
"disk7" ejected.
Preparing imaging engine...
Reading Protective Master Boot Record (MBR : 0)...
   (CRC32 $9A1DC65A: Protective Master Boot Record (MBR : 0))
Reading GPT Header (Primary GPT Header : 1)...
   (CRC32 $9598AAEB: GPT Header (Primary GPT Header : 1))
Reading GPT Partition Data (Primary GPT Table : 2)...
   (CRC32 $7D08CF8A: GPT Partition Data (Primary GPT Table : 2))
Reading  (Apple_Free : 3)...
   (CRC32 $00000000:  (Apple_Free : 3))
Reading disk image (Apple_HFS : 4)...
...
   (CRC32 $C1163E58: disk image (Apple_HFS : 4))
Reading  (Apple_Free : 5)...
...
   (CRC32 $00000000:  (Apple_Free : 5))
Reading GPT Partition Data (Backup GPT Table : 6)...
...
   (CRC32 $7D08CF8A: GPT Partition Data (Backup GPT Table : 6))
Reading GPT Header (Backup GPT Header : 7)...
...
   (CRC32 $B05F628B: GPT Header (Backup GPT Header : 7))
Adding resources...
...
Elapsed Time:  2.070s
File size: 1500593 bytes, Checksum: CRC32 $B1F95B1F
Sectors processed: 16384, 13964 compressed
Speed: 3.3Mbytes/sec
Savings: 82.1%
created: /Users/rosiefort/mlvfs/MLVFS.dmg
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 17, 2016, 03:33:54 AM
try using clang to compile instead of gcc, I'll try and fix the issue soon

how about this:

    //use drop frame if the framerate is close to 30 / 1.001
    int drop_frame = round((1.0 - framerate / ceil(framerate)) * 1000) == 1 && ceil(framerate) == 30;
    int hours, minutes, seconds, frames;
   
    if (drop_frame)
    {
        int d = frame / 17982;
        int m = frame % 17982;
        int f = frame + 18 * d + 2 * (MAX(0, m - 2) / 1798);
       
        hours = ((f / 30) / 60) / 60;
        minutes = ((f / 30) / 60) % 60;
        seconds = (f / 30) % 60;
        frames = f % 30;
    }
    else
    {
        //round the framerate to the next largest integral framerate
        //tc will not match real world time if framerate is not an integer
        double time = framerate == 0 ? 0 : frame / (framerate > 1 ? round(framerate) : framerate);
        hours = (int)floor(time / 3600);
        minutes = ((int)floor(time / 60)) % 60;
        seconds = ((int)floor(time)) % 60;
        frames = framerate > 1 ? (frame % ((int)round(framerate))) : 0;
    }
   
    buffer[*data_offset] = to_tc_byte(frames) & 0x3F;
    if(drop_frame) buffer[*data_offset] = buffer[*data_offset] | (1 << 7); //set the drop frame bit
    buffer[*data_offset + 1] = to_tc_byte(seconds) & 0x7F;
    buffer[*data_offset + 2] = to_tc_byte(minutes) & 0x7F;
    buffer[*data_offset + 3] = to_tc_byte(hours) & 0x3F;
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on March 17, 2016, 03:42:55 AM
@MARKUS

QuoteUsed the following string to mount: "mlvfs_x64.exe C:\MLV --mlv_dir=F:\DCIM\100EOS5D\"

It looks like the format of your MLVFS launch command is incorrect. 'C:\MLV' is a path to a folder not a drive letter. Try the following:

launch a command prompt (in administrator mode) then:

cd /d [path_to_your_mlvfs_x64_rc.exe]

mlvfs_x64.exe X: --mlv-dir=[path_to_your_MLV_dir]

Where X: - the logical drive letter that will be assigned by the system for mounting the MLV file/s. Note that it doesn't have to be X: but it needs to be a drive letter that is not already in use on your computer.

Then type the url http://localhost:8000/ into your default browser that will bring up the GUI and let you configure the MLVFS correction/adjustment parameters.

Also, once you have done the above, if you launch Windows File Explorer you should see your chosen drive letter listed and the individual MLV files listed as sub-directories.

A small tip: set your preferred correction/adjustment setting in the GUI before accessing a mounted mlv folder in your logical drive or the settings won't be applied.

The MLVFS process will automatically build index files for each of the MLV files in your --mlv-dir location.  I see from your mount string that you are going directly to the CF card. under this scenario the index files will be created on your CF card. There might be a performance hit from this approach. Also, this is where your original footage is. If there is a hiccup with your PC that causes the card to corrupt then you have potentially lost everything.

It would be safer and I dare say faster if you copied your MLVs to a directory on your local hard drive.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on March 17, 2016, 04:36:29 AM
Quote from: dmilligan on March 17, 2016, 03:33:54 AM
try using clang to compile instead of gcc, I'll try and fix the issue soon

That fixed the compiling issue--looks like I got to fix that hack in the compiling tutorial I posted a while back. Been procrastinating busy!

Anyway, at first I got this error:

dng.c:545:9: error: redefinition of 'drop_frame'
    int drop_frame = round((1.0 - framerate / ceil(framerate)) * 1000) == 1 && ceil(framerate) == 30;
        ^
dng.c:539:52: note: previous definition is here
static uint32_t add_timecode(double framerate, int drop_frame, uint64_t frame, uint8_t * buffer, uint32_t * data_offset)


and it wouldn't create a dng.o so I did this to your code:

-    int drop_frame = round((1.0 - framerate / ceil(framerate)) * 1000) == 1 && ceil(framerate) == 30;
+    drop_frame = round((1.0 - framerate / ceil(framerate)) * 1000) == 1 && ceil(framerate) == 30;


But as you can see there's still issues.

======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000000.dng
Time Codes                      : 00:00:00.00
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000001.dng
Time Codes                      : 00:00:00.00
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000002.dng
Time Codes                      : 00:00:00.01
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000003.dng
Time Codes                      : 00:00:00.02
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000004.dng
Time Codes                      : 00:00:00.03
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000005.dng
Time Codes                      : 00:00:00.04
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000006.dng
Time Codes                      : 00:00:00.05
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000007.dng
Time Codes                      : 00:00:00.06
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000008.dng
Time Codes                      : 00:00:00.07
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000009.dng
Time Codes                      : 00:00:00.08
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000010.dng
Time Codes                      : 00:00:00.09
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000011.dng
Time Codes                      : 00:00:00.10
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000012.dng
Time Codes                      : 00:00:00.11
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000013.dng
Time Codes                      : 00:00:00.12
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000014.dng
Time Codes                      : 00:00:00.13
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000015.dng
Time Codes                      : 00:00:00.14
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000016.dng
Time Codes                      : 00:00:00.15
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000017.dng
Time Codes                      : 00:00:00.16
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000018.dng
Time Codes                      : 00:00:00.17
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000019.dng
Time Codes                      : 00:00:00.18
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000020.dng
Time Codes                      : 00:00:00.19
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000021.dng
Time Codes                      : 00:00:00.20
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000022.dng
Time Codes                      : 00:00:00.21
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000023.dng
Time Codes                      : 00:00:00.22
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000024.dng
Time Codes                      : 00:00:01.00
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000025.dng
Time Codes                      : 00:00:01.00
======== /Users/rosiefort/Desktop/mount/M27-1302.MLV/M27-1302_000026.dng
Time Codes                      : 00:00:01.01


I'm a rank beginner in C so I'm a bit lost with what's going on here.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 17, 2016, 11:49:48 AM
Sorry, I've made other changes elsewhere. Easiest way to test is to just put that in a function in separate program, call it from a loop, and print out the time code.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on March 17, 2016, 01:32:04 PM
@Ottoga This is as far as I get. Everyting seems fine in browser but I cant access X:\. If I type sub-paths in --mlv_dir command I can't see them in browser.
Had the same problem trying it out on win 7...
(https://lh4.googleusercontent.com/-BYHw0WuenIs/Vuqh_kN6XLI/AAAAAAAACc0/vKkweGplmQEpZsWn8shI83NR2zGLii8rA/w888-h897-no/MLVFS%2BTrouble.jpg)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mindfull on March 17, 2016, 05:43:11 PM
is this normal?

El capitan OSX 10.11

(https://dl.dropboxusercontent.com/u/65174281/Screen%2520Shot%25202016-03-17%2520at%252011.04.04%2520AM.png)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on March 17, 2016, 08:59:45 PM
There's a folder selection dialog waiting for you to select a folder
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on March 17, 2016, 10:03:46 PM
Quote from: dmilligan on March 17, 2016, 11:49:48 AM
Sorry, I've made other changes elsewhere. Easiest way to test is to just put that in a function in separate program, call it from a loop, and print out the time code.

Uh, yeah. That's a bit beyond my current C coding abilities. However, I do have lots of experience in post production (IMDB link (http://www.imdb.com/name/nm0287123/)) and am wondering if there's a reason why so much effort is going into drop frame timecode? Is someone asking for it? I see no point in automatically forcing drop-frame timecode on 29.97fps DNG files. The productions that I have worked on have specified not to use drop-frame timecode on anything with a few exceptions for NTSC videotapes that were delivered to a broadcasting facility.

In any case, something is broken with the timecode and I'd like to help so I guess I need to hit the C programing tutorials.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on March 18, 2016, 01:40:30 AM
@Markus

QuoteYour mlvfs launch command:

c:\mlvfs_x64\mlvfs_x64.exe X:\ --mlv-dir=c:\

The '\' in your X:\ parameter is not required.

The parameter --mlv-dir=c:\  as per your launch command will result in MLVFS (or more likely the background dokan service) passing through, trying to map and interpret all files and sub-directories under the root of C: to drive X:

I'd suggest that under this scenario it is probably hitting a file that it can't deal with elegantly and as such causing the error. It also looks like it is trying to process the same MLV file twice which may be causing a conflict.

the --mlv-dir= parameter should contain the path to the folder where the MLV files that you want to process actually reside.

So, based on your screen shots I suggest that your mlvfs launch command should be:

c:\mlvfs_x64\mlvfs_x64.exe X: --mlv-dir=c:\HEj

MLVFS as a process appears to be working as it has successfully analysed M17-1143.mlv given that it is listed with all details in your localhost view. So hopefully with a corrected launch command you will be successful.

If you are still getting the error after that, then I'm at a loss and will need to hand your issue up to DMilligan. just provide fresh screen shots including some from File manager showing drive X: and any of its sub directories as well as one with a sub-directory contents showing.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on March 18, 2016, 05:51:49 AM
Testing MLVFS for Windows7 , everything when as per instructions for x86 .
Web GUI work without any problems , did have same crashes  with dual ISO from the
5d2 , when I try to copy the dng's from the DOKAN\Z: fold to a real folder it will crash after
about 500 frame out of 1500 , stop responding etc... then lost the mounted folder
tried to remount but had problems , rebooted Windows everything back to normal
but I haven't tried to copy the file . Will try a different mount drive .
besides that works just like the Mac version .
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kgv5 on March 18, 2016, 06:11:25 AM
@reddeercity - could you please make a video tutorial how to get the newest method of MLVFS on PC working? I am still using the old version, it is a little bit hard to follow some new developments in this area though, i really enjoyed your previous tutorials. I bet it would be great for many people here if someone could sum it up to this point. Thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on March 18, 2016, 02:21:53 PM
After trying anything and everything. I got an idea to try the x86 version of mlvfs instead and got it working right away. So it seems x64 version will not run on win 10 (at least not for me). Something goes wrong when it tries to mount the files to X:

I noticed that Pismo is much faster so I'll be sticking with that for a while. I mount my files from the card directly and then use slimraw to copy/compress them to harddrive. In that way I only get compressed footage on the harddrive. With Pismo and Slimraw this is just a little bit slower than copying the mlvs to the harddrive directly.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on March 18, 2016, 10:59:01 PM
Quote from: kgv5 on March 18, 2016, 06:11:25 AM
@reddeercity - could you please make a video tutorial how to get the newest method of MLVFS on PC working? I am still using the old version, it is a little bit hard to follow some new developments in this area though, i really enjoyed your previous tutorials. I bet it would be great for many people here if someone could sum it up to this point. Thanks.

Sure , I was thinking that myself the other day  ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on March 19, 2016, 05:12:18 PM
@kgv5 here you go as requested ,  Video Tutorial for MLVFS for Windows .
I mainly talk about windows7 but the same will hold true for Win8 & Win10

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kgv5 on March 19, 2016, 07:07:53 PM
@reddeercity
Great, thank you very much. Very informative and helpful tutorial, as usual :) I needed such an explanation, it was little bit hard to follow this thread, now everything is clear!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: axelcine on March 19, 2016, 08:13:38 PM
@reddeercity - Thank you very much. This was exactly what I needed.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on March 19, 2016, 11:44:08 PM
Thanks Guys !  Just doing my Part : I like doing ML Tutorial , it let me take a brake from my
paid project as I'm in the really boring part  "grading" indie movie .
It can be exciting sometimes ! when a image come to life though hours & hours of works  ::)
then you sit back , look at it and go " I  Love "F"ing Raw"  :D 

Only 520 clips to go  , that what keep telling myself 519 to go ,  518 to go ,  517 to go .......
Done 500 already   :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: USADavid on March 20, 2016, 02:26:15 AM
Is anyone else having empty .wav files with .MLV files > 2GB on Mac?
Plays fine with MlRawViewer!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ruber on March 21, 2016, 03:11:25 PM
Hi Guys,

thx to your good tips, I am working on a project with mlvfs and Dokan for the first time

I am in windows 10 pro, working with Davinci Resolve right now.

Clips are 1080p 25fps (NO dual iso)
My camera is the 5dmk3

I have succesfully imported all the clips (about 50 files) into davinci but the problem now is that playback is really slow, it stutters and it's not possible to work this way

Surprinsingly, when I deliver (render) one of these clips, rendering speed is quite good (40fps minimum with DNxHD)

EDIT: now playback in timeline is smooth and that's the most important thing. The problem is when I play the clips in the "media" section.

Would you have some tips? :)

Thank you. Cheers.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: markodarko on March 22, 2016, 04:41:08 PM
Firstly, thank you @dmilligan for developing this solution for the community. It really is most excellent. I wonder if it would be possible for you to answer these questions for me as my search on this forum didn't bring up any:

1. What is the difference between "On" and "Aggressive" for bad pixel correction in terms of use and speed differences (if any)?
2. What are the differences in speed between opening the cDNGs using MLVFS or by first copying out the cDNG files to a folder and opening them there?

I ask because rendering 1080p clips from my 5D III in DaVinci Resolve 12 Lite is painfully slow for me. I have a MBP i7 8Gb RAM / 512Mb Nvidia with 512Gb SSD system drive and 2TB SSD media drive installed and rendering a 2 minute clip to ProRes 4444 with "Aggressive" bad pixel correction (as "on" didn't get rid of them) as well as colour grading in Resolve took 6 hours. I'm currently rendering another timeline with various clips and a total duration of 4 minutes and the estimated render time is SIXTEEN hours - it's 11 hours in right now so I'm not going to stop it to do some tests at this point.

I'd just like to get to the bottom of where the bottleneck is (aside from my mediocre Mac system compared to today's standards!) as I've read that others are getting 40fps render times. Is it the "Aggressive" mode or is it opening the cDNG files directly instead of first copying them?

Thanks for your help,

Mark.

EDIT: I just stopped the render as it was ridiculous so I'm going to do some tests. Incidentally, MLVFS is using a lot of RAM...

(http://i1032.photobucket.com/albums/a404/markodarkophoto/Screen%20Shot%202016-03-22%20at%2015.50.22_zpshvezcsaj.png)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 22, 2016, 07:10:42 PM
Quote from: markodarko on March 22, 2016, 04:41:08 PM
Incidentally, MLVFS is using a lot of RAM...
I've fixed some memory leaks recently, maybe I haven't updated the build, I'll check.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on March 22, 2016, 07:31:41 PM
@markodarko
If bad pixel fix slows down your process much you can get rid of the bad pixel with a secondary node in resolve instead. You make super small masks that blur out the stuck pixels. It's quite tiresome and fiddely to make masks for all the bad pixels but once you have done it you can save all the masks as a preset that you can reuse later since the bad/stuck pixels will be in the same places.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: markodarko on March 22, 2016, 07:51:49 PM
Quote from: dmilligan on March 22, 2016, 07:10:42 PM
I've fixed some memory leaks recently, maybe I haven't updated the build, I'll check.

Ah ok. I'm not sure which build I have. Perhaps you could amend the http://localhost:8000 control page to display the current build of MLVFS installed to make checking if we're running the latest version easier?

Quote from: Markus on March 22, 2016, 07:31:41 PM
If bad pixel fix slows down your process much you can get rid of the bad pixel with a secondary node in resolve instead. You make super small masks that blur out the stuck pixels. It's quite tiresome and fiddely to make masks for all the bad pixels but once you have done it you can save all the masks as a preset that you can reuse later since the bad/stuck pixels will be in the same places.

Funny you should mention this as I was going to shoot a black matte at high ISO to make just a power grade with the technique you mentioned! :) Great minds an' all that...

But I think I've discovered the culprit, speed wise. I did some tests with a 60Gb cDNG folder and copying it to various places.

1. Bad Pixel Fix = "Aggressive". Copying from MLVFS > HDD took 130 minutes.
2. Bad Pixel Fix = "Off". Copying from MLVFS > HDD took 11 minutes.
3. Copying the copied folder to another location on my HDD took 7 minutes.

So from my quick test it takes 11 times longer with "Aggressive" but when set to "Off" it's only 1.5 longer than copying from HDD to HDD (only SATA II on this laptop unfortunately) so MLVFS is incredibly fast when set to "Off". Obviously, 130/7 = 18 times quicker when opening the copied DNGs directly from my HDD.

I'm now going to do another test in Resolve whereby I open the copied folders instead of the MLVFS folders to see if there's any significant difference in UI / Playback / Rendering speed over opening from MLVFS directly.

Cheers,

Mark.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 22, 2016, 08:47:37 PM
Quote from: markodarko on March 22, 2016, 07:51:49 PM
Perhaps you could amend the http://localhost:8000 control page to display the current build of MLVFS installed to make checking if we're running the latest version easier?
I already have so if you don't seen version info at the bottom of the page, you must be running an old version.

A good way to measure speed is to cat the DNGs to /dev/null and time that. That way there's no dependency on the write speed of destination drive (in *nix, /dev/null is just an empty place to dump data, it goes nowhere). In the terminal:

time cat <path-to-mlvfs-mount>/<path-to-mlv-dir>/*.dng > /dev/null


Example:

$ time cat ~/MLVFS/M14-2300.MLV/*.dng > /dev/null

real 0m6.936s
user 0m0.016s
sys 0m0.782s
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on March 23, 2016, 12:21:43 AM
Win 7 context menu "Mount with MLVFS" with Admin rights elevation

mlvfs.reg - run this to create registry entry adding "Mount with MLVFS" context menu option to every folder.
Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\MLVFS]
@="Mount with MLVFS"

[HKEY_CLASSES_ROOT\Directory\shell\MLVFS\command]
@="\"C:\\Program Files\\MLVFS_x64\\mlvfs.bat\" \"%1\""


mlvfs.bat - place this file next to mlvsf_x64.exe
@echo Off
Setlocal
:: First check if we are running As Admin/Elevated
FSUTIL dirty query >nul
if %errorlevel% EQU 0 goto START

::Create and run a temporary VBScript to elevate this batch file
set _batchFile=%~f0
set _Args=%*
:: double up any quotes
set _batchFile=""%_batchFile:"=%""
set _Args=%_Args:"=""%

echo Set UAC = CreateObject^("Shell.Application"^) > "%temp%\~ElevateMe.vbs"
echo UAC.ShellExecute "cmd", "/c ""%_batchFile% %_Args%""", "", "runas", 1 >> "%temp%\~ElevateMe.vbs"

cscript "%temp%\~ElevateMe.vbs"
exit /B

:START
:: set the current directory to the batch file location
cd /d %~dp0

:: run webgui
start http://localhost:8000

:: run mlvfsx64
mlvfs_x64.exe x:\ --mlv_dir=%1
:: -d 1>log_std-out-err.txt 2>&1


You might need to modify paths to fit your setup.. and voilà!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: markodarko on March 23, 2016, 02:17:58 AM
Quote from: dmilligan on March 22, 2016, 08:47:37 PM
I already have so if you don't seen version info at the bottom of the page, you must be running an old version.

Ha. No problem. Have just updated. :)

Quote from: dmilligan on March 22, 2016, 08:47:37 PM
A good way to measure speed is to cat the DNGs to /dev/null and time that.

Great tip, thanks! I shall use that in future. For now though as I have some historic time data I re-ran my jobs after re-linking the media to cDNGs that I'd copied from the MLVFS volume with the "Aggressive" flag set. As to be expected it has made a massive difference doing it that way. A job which took 12.5hrs to render now took only 1.5hrs.

There's no real noticeable difference in rendering speed as far as I can see (in my admittedly un-scientific tests) between jobs that are rendered via cDNGs that are opened directly via MLVFS with the Fix Pixel flag set to OFF or copied cDNGs. The only slow-down seems to be when setting the Fix Pixel flag to Aggressive. In that instance it seems much quicker to first copy out the cDNGs from MLVFS to a new folder with the flag set to "aggressive" (took 2hrs) and then relink the media to the copied files rather than opening them directly through MLVFS.

This is on my machine of course and with the clips fully graded in Resolve - i.e., they're not simply "spat out" the other side in delivery from the source clips. I'd imagine that would speed things up dramatically too.

Cheers,

Mark.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 23, 2016, 02:43:43 AM
We can actually make the bad pixel fix considerably faster by only analyzing one frame and then simply applying the correction to those pixels in all frames. This should work in theory since bad pixels are typically fixed. I've updated the Mac download with a build that implements this. Please let me know how it's working. Also fixed is the TC issue @dfort found.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: markodarko on March 23, 2016, 03:43:51 AM
Quote from: dmilligan on March 23, 2016, 02:43:43 AM
We can actually make the bad pixel fix considerably faster by only analyzing one frame and then simply applying the correction to those pixels in all frames. This should work in theory since bad pixels are typically fixed. I've updated the Mac download with a build that implements this. Please let me know how it's working. Also fixed is the TC issue @dfort found.

But does the bad pixel feature not take into account corresponding colours when it "blends"?

May I ask what is the difference between "On" and "Aggressive" in terms of what's happening to the image?

Thanks,

Mark.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on March 23, 2016, 07:13:20 AM
Quote from: dmilligan on March 23, 2016, 02:43:43 AM
Also fixed is the TC issue @dfort found.

Thanks! I just checked it and it looks good on 23.976 material. I'd like to check it out on 29.97 because it looks like you're using drop-frame timecode.

I also checked a 59.94 MLV--I've got some ideas for that but want to think about it first.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 23, 2016, 12:10:11 PM
Quote from: markodarko on March 23, 2016, 03:43:51 AM
But does the bad pixel feature not take into account corresponding colours when it "blends"?
99.999% of the time spent by the bad pixel fix feature is *looking* for bad pixels (you have to check each and every pixel, and there are millions, and analyze all of it's neighbors) and the tiny remainder is the time to *correct* them (there are typically only a few hundred to thousand to correct). We only need to *look* for bad pixels in the first frame, then we store which pixels were bad and correct those pixels in each subsequent frame.

Quote from: markodarko on March 23, 2016, 03:43:51 AM
what is the difference between "On" and "Aggressive" in terms of what's happening to the image?
'Aggressive' has lower thresholds for what is considered a "bad" pixel. Therefore it will include more pixels (and probably more false positives) in the list of pixels it considers to be corrected, but it's more likely to identify bad pixels that don't stand out as much, but are still bad pixels.

@dfort, just FYI: the bad pixel fix algorithm will print to stdout the list of bad pixels it finds, so if you have focus pixel footage where you can really make the focus pixels standout, this might be a good check or starting point for creating focus pixels maps.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on March 23, 2016, 12:47:30 PM
@dmilligan

So does bad pixel equate to hot pixel in ML speak and recovery process?

I would define a bad pixel as one which is say permanently off (dead) or permanently stuck to a specific colour. I can see analysis the first frame only working for these. Whereas a hot pixel may not manifest itself until some time after the camera has been in use, specifically when the camera starts to get hot. if this true then only checking the first frame would risk missing these.

If you have to have a separate process for hot verses bad pixels would defeat the purpose of your proposed bad pixel management. Don't get wrong, I like the idea. Just thinking out loud of a potential pitfall that could just lead whole new slew of support issues.

Apologies if my thoughts are a pile of nonsense.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 23, 2016, 03:14:27 PM
Well, we can always just analyze the last frame, though it's not impossible for a camera to actually cool down, and there are other things besides temperature that influence behavior.

An ideal algorithm would probably pick a handful of frames and use all of them *together* to find stuck pixels, but such a complex algorithm is beyond the amount of free time I have to work on this project. Additionally, Resolve Lite is pretty much the only software that doesn't automatically take care of these anyway (and I don't use Resolve personally).


On a side note, from my understanding there's no real difference in "hot" pixel and "stuck" pixel, they are caused by the same phenomenon, there are just various levels of "hotness" a pixel can have. There does tend to be a non-linear response to the various parameters that effect them (temperature, ISO, exposure time, etc.), which gives the impression that some pixels are always "stuck" and some are only "hot", but really the "hot" ones are just on the more linear side of the non-linear response, and the "stuck" ones are past the "elbow".
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 24, 2016, 03:52:58 AM
Any Mac users experiencing the web GUI error with the latest MLVFS update from today even though it kind of works but in a weird way?

I'm not sure why this is happening on my end now when it just worked fine last night.

Basically right after the web GUI error I went ahead and opened up the Virtual Folder which somehow had all the Files w DNG's in it but no .IDX would spit out UNLESS I manually go through each Folder which then spits out .IDX for each however still I have no controls over web GUI since it doesn't show anything.

Here's the screenflow of what I was able to accomplish despite of the new bug:

https://vimeo.com/160190883
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on March 24, 2016, 09:01:24 PM
Logs?

Try rebooting your computer.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 24, 2016, 09:22:11 PM
Actually for some reason last night after a few restarts ... problem still persisted and gave up but then this morning when I woke up and decided to do a hard reboot and volia you were right on that one. Oops, my bad I guess?  :o

Running Version: 109ac1a Mar 22 201621:26:57 and all is well so I'll be testing out the TC feature and will report back when I can.

Thanks again, David!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: markodarko on March 25, 2016, 04:38:47 PM
Quote from: dmilligan on March 23, 2016, 12:10:11 PM
99.999% of the time spent by the bad pixel fix feature is *looking* for bad pixels (you have to check each and every pixel, and there are millions, and analyze all of it's neighbors)

Ah, of course. Makes sense.

Quote from: dmilligan on March 23, 2016, 03:14:27 PM
An ideal algorithm would probably pick a handful of frames and use all of them *together* to find stuck pixels, but such a complex algorithm is beyond the amount of free time I have to work on this project.

I understand. How about a third option, that is to keep the "Aggressive" flag as it was (i.e., checking every frame) and add another setting to the UI called "Aggressive (first frame check only)"? That way if we experience hot pixels due to camera heat / cooling down etc then we could choose the "every frame" option but otherwise just use the "quick" option... Thoughts?

Incidentally, I have a problem running MLVFS from a certain drive. It results in a crash:


Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000027800000010
Exception Note:        EXC_CORPSE_NOTIFY


I can send you the full crash report if you want.

Basically I moved all of my MLV files to an external Raid 0 drive and tried to mount the MLV folder from there. It goes through the process of showing the "choose your mount point" window but as soon as I do it opens Safari to a blank page saying that it could not open the page.

I've tried having the mount folder locally and on the external drive but to no avail.

I've set the privileges on the drive to "Read & Write" for me, "Staff" and "Everyone" as I assume it's a write error?

This does not happen with all external drives. Works fine with another external SSD drive that I have.

Cheers,

Mark.

EDIT: I've found the problem with the error. It seems that MLVFS doesn't like spaces (or it could be brackets) in the drive name. I renamed the drive to use underscores instead of spaces (and removed the brackets) and all now works as expected. :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 26, 2016, 03:09:34 AM
Quote from: markodarko on March 25, 2016, 04:38:47 PM
EDIT: I've found the problem with the error. It seems that MLVFS doesn't like spaces (or it could be brackets) in the drive name. I renamed the drive to use underscores instead of spaces (and removed the brackets) and all now works as expected. :)
Cannot reproduce. Spaces and brackets work fine here. Upload the whole crash log and also ~/mlvfs.log. Can you reproduce the issue if you change the name back to what it was? What exactly was the drive name and what format?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: markodarko on March 26, 2016, 05:04:20 PM
Quote from: dmilligan on March 26, 2016, 03:09:34 AM
Cannot reproduce. Spaces and brackets work fine here. Upload the whole crash log and also ~/mlvfs.log. Can you reproduce the issue if you change the name back to what it was? What exactly was the drive name and what format?

Can't seem to find mlvfs.log on my system but changing the name back resulted in the same error:

(http://i1032.photobucket.com/albums/a404/markodarkophoto/Screen%20Shot%202016-03-26%20at%2015.55.35_zpsfuqtkg6u.png)


The volume format is Mac OS Extended (Journaled) and the name was "MLV Backup (Samsung 2TB)". Maybe it's the length of the name?

Here's the latest crash report:

Process:               mlvfs [427]
Path:                  /Users/USER/Library/Services/*/mlvfs
Identifier:            mlvfs
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           mlvfs [427]
User ID:               501

Date/Time:             2016-03-26 15:55:21.382 +0000
OS Version:            Mac OS X 10.11.3 (15D21)
Report Version:        11
Anonymous UUID:        2F2B3BBB-6F28-0BF2-82FE-21ED37D41433


Time Awake Since Boot: 75 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000027800000010
Exception Note:        EXC_CORPSE_NOTIFY

VM Regions Near 0x27800000010:
    Process Corpse Info    000000010a53b000-000000010a73b000 [ 2048K] rw-/rwx SM=COW 
-->
    MALLOC_TINY            00007fc382c00000-00007fc382e00000 [ 2048K] rw-/rwx SM=PRV 

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib              0x00007fff93f54152 strlen + 18
1   libsystem_c.dylib              0x00007fff93faeb79 strdup + 18
2   mlvfs                          0x0000000107af5d25 main + 309
3   libdyld.dylib                  0x00007fff8a2395ad start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000001  rbx: 0x0000000000000000  rcx: 0x0000027800000019  rdx: 0x0000027800000019
  rdi: 0x0000027800000010  rsi: 0x00007fff64bea248  rbp: 0x00007fff5810caa0  rsp: 0x00007fff5810caa0
   r8: 0x0000000000000000   r9: 0x00007fc382d00145  r10: 0x00007fff9bcb7f01  r11: 0x00007fff93faeb67
  r12: 0x0000000000000000  r13: 0x0000000000000000  r14: 0x0000027800000019  r15: 0x0000000000000000
  rip: 0x00007fff93f54152  rfl: 0x0000000000010202  cr2: 0x0000027800000010
 
Logical CPU:     2
Error Code:      0x00000004
Trap Number:     14


Binary Images:
       0x107af3000 -        0x107b61ff7 +mlvfs (0) <FE4AA6CB-2C80-3284-BEEC-C38549A7E38A> /Users/USER/Library/Services/*/mlvfs
       0x10a2d8000 -        0x10a2f6fe7 +libosxfuse_i64.2.dylib (10.3) <B953F9DD-EF0F-3FA6-AB08-D8D96131C9CE> /usr/local/lib/libosxfuse_i64.2.dylib
    0x7fff64bea000 -     0x7fff64c21007  dyld (360.19) <9D05FDF4-65CE-3B53-86D4-ABE1A5BF35F3> /usr/lib/dyld
    0x7fff84f05000 -     0x7fff84f21ff7  libsystem_malloc.dylib (67) <9EECAB18-F025-34C4-8E32-7EFFA6720EFC> /usr/lib/system/libsystem_malloc.dylib
    0x7fff8798f000 -     0x7fff87997fff  libcopyfile.dylib (127) <F5133269-0B22-388C-A57C-079667B6291E> /usr/lib/system/libcopyfile.dylib
    0x7fff87f9d000 -     0x7fff87f9dff7  libunc.dylib (29) <1D0F8265-F026-3CBD-93D3-F8DF14FFCE68> /usr/lib/system/libunc.dylib
    0x7fff89117000 -     0x7fff8911aff7  libsystem_sandbox.dylib (460.30.1) <3E0036AF-FC64-3352-8DA4-6B550C2C2562> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff8977a000 -     0x7fff897a9ffb  libsystem_m.dylib (3105) <26655445-CA97-321E-B221-801CB378D1AA> /usr/lib/system/libsystem_m.dylib
    0x7fff899ae000 -     0x7fff89aa0ff7  libiconv.2.dylib (44) <F05A0A5A-92A9-3668-8F20-F27CBDA26BE9> /usr/lib/libiconv.2.dylib
    0x7fff89aa1000 -     0x7fff89b02ff7  libsystem_network.dylib (583.20.10) <865FE79A-A22D-3733-A14F-FC7B37F3AECD> /usr/lib/system/libsystem_network.dylib
    0x7fff89b31000 -     0x7fff89b48ff7  libsystem_asl.dylib (322.30.1) <9B500E4E-E462-321E-828E-5524DC984C1B> /usr/lib/system/libsystem_asl.dylib
    0x7fff89c92000 -     0x7fff89e9fffb  libicucore.A.dylib (551.41) <CFFD7342-A7D6-323A-AC14-B9EECF6EFFED> /usr/lib/libicucore.A.dylib
    0x7fff8a236000 -     0x7fff8a239ffb  libdyld.dylib (360.19) <AA629043-C6F6-32FE-8007-E3478E99ACA7> /usr/lib/system/libdyld.dylib
    0x7fff8b048000 -     0x7fff8b08eff7  libauto.dylib (186) <999E610F-41FC-32A3-ADCA-5EC049B65DFB> /usr/lib/libauto.dylib
    0x7fff8b4ed000 -     0x7fff8b50bfff  libsystem_kernel.dylib (3248.30.4) <9CEB6C3B-1CAF-3C32-A9FD-93BC72CBCEA1> /usr/lib/system/libsystem_kernel.dylib
    0x7fff8b581000 -     0x7fff8b581ff7  liblaunch.dylib (756.20.4) <EDF719D6-D2BB-38DD-8C94-4272BEFDA2CD> /usr/lib/system/liblaunch.dylib
    0x7fff8b5b0000 -     0x7fff8b5b0ff7  libkeymgr.dylib (28) <09397E01-6066-3179-A50C-2CE666FDA929> /usr/lib/system/libkeymgr.dylib
    0x7fff8b5e7000 -     0x7fff8b5f0ff7  libsystem_pthread.dylib (138.10.4) <327CECD0-B881-3153-8FCC-4FD4818B7F16> /usr/lib/system/libsystem_pthread.dylib
    0x7fff8ba5c000 -     0x7fff8ba89fff  libdispatch.dylib (501.20.1) <324C9189-2AF3-3356-847F-6F4CE1C6E901> /usr/lib/system/libdispatch.dylib
    0x7fff8c76a000 -     0x7fff8c76bffb  libSystem.B.dylib (1226.10.1) <5A4257EF-3145-3BB3-87A4-0D2404A9462D> /usr/lib/libSystem.B.dylib
    0x7fff8cbef000 -     0x7fff8d065fff  com.apple.CoreFoundation (6.9 - 1256.14) <768A7FB7-9143-3148-8591-7C6ED3162D35> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
    0x7fff8d06c000 -     0x7fff8d074fe7  libsystem_platform.dylib (74.10.3) <D3A27E10-7F08-3603-ACC8-7A92B2C04BAB> /usr/lib/system/libsystem_platform.dylib
    0x7fff8d58e000 -     0x7fff8d590ff7  libquarantine.dylib (80) <163CF63A-7455-3D1F-AE57-8C4475A9204C> /usr/lib/system/libquarantine.dylib
    0x7fff8d591000 -     0x7fff8d5e4ff7  libc++.1.dylib (120.1) <8FC3D139-8055-3498-9AC5-6467CB7F4D14> /usr/lib/libc++.1.dylib
    0x7fff901b1000 -     0x7fff901b3ff7  libsystem_configuration.dylib (802.20.7) <5FD79070-36CC-3D02-BEA7-BB5D2AE97D5D> /usr/lib/system/libsystem_configuration.dylib
    0x7fff910de000 -     0x7fff910e2fff  libcache.dylib (75) <6B245C0A-F3EA-383B-A542-5B0D0456A41B> /usr/lib/system/libcache.dylib
    0x7fff937bb000 -     0x7fff937e4fff  libsystem_info.dylib (477.20.1) <6513635B-4ADE-3B45-BF63-ED7AC565B0C9> /usr/lib/system/libsystem_info.dylib
    0x7fff93f44000 -     0x7fff93f45fff  libsystem_secinit.dylib (20) <FD6ECF2C-1489-32CA-981B-9045B5EB1FAA> /usr/lib/system/libsystem_secinit.dylib
    0x7fff93f53000 -     0x7fff93fe0fff  libsystem_c.dylib (1082.20.4) <EAB38A6C-8671-3B13-B500-90EC1B912063> /usr/lib/system/libsystem_c.dylib
    0x7fff94718000 -     0x7fff94720fff  libsystem_networkextension.dylib (385.20.6) <DC8A102A-BF02-31A4-8914-65C34DF6B592> /usr/lib/system/libsystem_networkextension.dylib
    0x7fff9478c000 -     0x7fff9478dfff  libDiagnosticMessagesClient.dylib (100) <4243B6B4-21E9-355B-9C5A-95A216233B96> /usr/lib/libDiagnosticMessagesClient.dylib
    0x7fff9526b000 -     0x7fff95273ffb  libsystem_dnssd.dylib (625.20.4) <945B5FB1-DA91-3D45-A961-A8FAD53C1E7E> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff95508000 -     0x7fff95509ffb  libremovefile.dylib (41) <B8D1A5FC-CFD5-3AAB-8A10-14DDC129710A> /usr/lib/system/libremovefile.dylib
    0x7fff955ee000 -     0x7fff95617fff  libc++abi.dylib (125) <DCCC8177-3D09-35BC-9784-2A04FEC4C71B> /usr/lib/libc++abi.dylib
    0x7fff957bc000 -     0x7fff957bdfff  libsystem_blocks.dylib (65) <49D42329-7DE9-3413-92C3-A473A7E9CF35> /usr/lib/system/libsystem_blocks.dylib
    0x7fff95bf7000 -     0x7fff95c00ff3  libsystem_notify.dylib (150.20.3) <243FADE1-255A-3B78-8033-F336CD64B817> /usr/lib/system/libsystem_notify.dylib
    0x7fff9615f000 -     0x7fff96164ff7  libmacho.dylib (875.1) <CB745E1F-4885-3F96-B38B-2093DF488FD5> /usr/lib/system/libmacho.dylib
    0x7fff96629000 -     0x7fff9662bfff  libsystem_coreservices.dylib (19.2) <1B3F5AFC-FFCD-3ECB-8B9A-5538366FB20D> /usr/lib/system/libsystem_coreservices.dylib
    0x7fff96a33000 -     0x7fff96a38ff3  libunwind.dylib (35.3) <124E0F05-2350-3774-A32C-7F5BF38EDE73> /usr/lib/system/libunwind.dylib
    0x7fff96ca0000 -     0x7fff96cabff7  libcommonCrypto.dylib (60075.20.1) <766BC3F5-41F3-3315-BABC-72718A98EA92> /usr/lib/system/libcommonCrypto.dylib
    0x7fff96f8d000 -     0x7fff96f94ff7  libcompiler_rt.dylib (62) <D3C4AB40-23B4-3BC6-8C38-5B8758D14E80> /usr/lib/system/libcompiler_rt.dylib
    0x7fff97028000 -     0x7fff97039ff7  libz.1.dylib (61.20.1) <B3EBB42F-48E3-3287-9F0D-308E04D407AC> /usr/lib/libz.1.dylib
    0x7fff97840000 -     0x7fff97851ff7  libsystem_trace.dylib (201.10.3) <B485369F-E3A1-319E-998C-89AAF606079E> /usr/lib/system/libsystem_trace.dylib
    0x7fff9896f000 -     0x7fff98cda657  libobjc.A.dylib (680) <58CB8CFC-7DBD-3A53-BD72-A42FF799B21E> /usr/lib/libobjc.A.dylib
    0x7fff99bbb000 -     0x7fff99be4fff  libxpc.dylib (756.20.4) <61AB4610-9304-354C-9E9B-D57198AE9866> /usr/lib/system/libxpc.dylib
    0x7fff9ae26000 -     0x7fff9ae3cff7  libsystem_coretls.dylib (83.20.8) <75C97D88-0A63-3093-AE83-DE33EB7405CE> /usr/lib/system/libsystem_coretls.dylib
    0x7fff9aea3000 -     0x7fff9af1afe7  libcorecrypto.dylib (335.20.1) <C6BD205F-4ECE-37EE-BCAB-A76F39CDCFFA> /usr/lib/system/libcorecrypto.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 131
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=106.9M resident=0K(0%) swapped_out_or_unallocated=106.9M(100%)
Writable regions: Total=61.8M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=61.8M(100%)

                                VIRTUAL   REGION
REGION TYPE                        SIZE    COUNT (non-coalesced)
===========                     =======  =======
Activity Tracing                  2048K        2
Kernel Alloc Once                    4K        2
MALLOC                            10.2M        8
MALLOC guard page                   16K        4
Process Corpse Info               2048K        2
STACK GUARD                       56.0M        2
Stack                             8192K        2
VM_ALLOCATE                          4K        2
__DATA                            42.2M       50
__LINKEDIT                        91.3M        5
__TEXT                            15.6M       48
__UNICODE                          552K        2
shared memory                        8K        3
===========                     =======  =======
TOTAL                            227.8M      119

Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on March 26, 2016, 08:24:37 PM
Sorry, I always forget about the dot.


cat ~/.mlvfs.log
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: markodarko on March 29, 2016, 01:48:51 AM
Is there a way to find the White Balance information so that I can input it into Resolve? Currently, "As shot" returns 2770.

Thanks,

Mark.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: newsense on March 29, 2016, 02:11:06 AM
When i copy .dng's from a mounted MLV and import those dng's as a sequence into after effects using ACR the video imports but when I render I only get one frame rendered. I tried just importing the mounted .dng's and every frame rendered. Am I doing something wrong?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on March 29, 2016, 02:37:39 AM
Quote from: newsense on March 29, 2016, 02:11:06 AM
When i copy .dng's from a mounted MLV and import those dng's as a sequence into after effects using ACR the video imports but when I render I only get one frame rendered. I tried just importing the mounted .dng's and every frame rendered. Am I doing something wrong?

Perhaps you haven't tried using SmartImport2 (https://secure.magiclantern.fm/forum/index.php?topic=12422.0SmartImport2) while using MLVFS within AE.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: newsense on March 29, 2016, 02:37:57 AM
Quote from: newsense on March 29, 2016, 02:11:06 AM
When i copy .dng's from a mounted MLV and import those dng's as a sequence into after effects using ACR the video imports but when I render I only get one frame rendered. I tried just importing the mounted .dng's and every frame rendered. Am I doing something wrong?

UPDATE: figured it out, I had to copy the whole folder and not just the .dng's. For some reason that worked.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andy kh on April 02, 2016, 06:02:45 PM
mlvfs for windows works like a charm. i was away from home for so many months so i couldnt experiment or join the forum but today i just tried and it works but pink dots are not removed. i hope it is not implemented for windows
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 02, 2016, 08:28:58 PM
You must launch MLVFS from the directory containing the focus pixel maps, and your camera metadata in the MLV file must be correct (MLVFS must know which camera, and some older versions of ML don't record camera metadata for certain cameras). Does the webgui work? Is the camera model metadata shown in the webgui correct?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andy kh on April 02, 2016, 09:19:22 PM
@dmilligan camera model is not shown. mine is an older version. will test the new version tomorow morning

edit:just downloaded the latest version of ml and done a quick test and this time pink dot has gone.n  camera version is shwon in webgui. this is awesome. wil do more test and report any bug. thank q
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: TheJuice on April 03, 2016, 05:12:59 PM
Hi,
I noticed that if the directory the MLV files are in contains accentuated characters (for instance "é"), it won't work.
I can rename the folders, but I have many, and it would be nice to have UTF-8 support :-)

Cheers
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 04, 2016, 03:06:38 AM
Just tried a folder named: "MLV🍺 " (U+1F37A) and it worked just fine, you might have to be a little more specific about what you tried, version, OS, etc.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on April 04, 2016, 09:26:21 AM
Hey David,

Just checking to see if you have been following this along lately because after shooting a bunch of test footage in slo-mo with the new crop_rec.mo from @a1ex's recent update (http://www.magiclantern.fm/forum/index.php?topic=17021.msg165282#msg165282crop_rec.mo%20from%20@a1ex's%20recent%20update) which doesn't seem work fully 100% with current MLVFS on those new files but some do show up and some don't.

(https://farm2.staticflickr.com/1615/26227026035_1ce0bb2dd5_n.jpg) (https://flic.kr/p/FXAn74)

Do you see similar symptoms on your end?

If not, then here are a few of my sample files with one working and the other not 'showing up'.

https://mega.nz/#F!j4dQQYhT!UW4PnUEo7j9AMRpZ6biEAA

Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on April 04, 2016, 10:37:25 AM
I haven't had any problem with crop_rec files in MLVFS on OS X Yosemite or El Capitan.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 04, 2016, 06:03:48 PM
Quote from: DeafEyeJedi on April 04, 2016, 09:26:21 AM
crop_rec.mo from @a1ex's recent update (http://www.magiclantern.fm/forum/index.php?topic=17021.msg165282#msg165282crop_rec.mo%20from%20@a1ex's%20recent%20update) which doesn't seem work fully 100% with current MLVFS on those new files but some do show up and some don't.
It's probably a coincidence and actually due to something else. crop_rec doesn't affect raw recording in anyway other than the actual image data that is recorded (format, metadata, etc should all be the same).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on April 04, 2016, 07:25:54 PM
It was most likely due to the CF card I had used (w USB 3.0 Kingston Card reader). Even tho it's never a problem to do this workflow with KB128GB 1066x cards but this one is a 5 year old SanDisk Extreme Pro 90MB/s UDMA 32GB. Not my favorite card but while I was testing this out during the weekend away from home and all of my KB's cards were full with MLV's that I wanted to held on to. Saving up to get yet another external drive for archival purposes. Ugh!

(https://farm2.staticflickr.com/1683/25963358710_67a242c6e9_n.jpg) (https://flic.kr/p/FyhZZG)

Because I just dragged all of the files from the card into the system on OS X and ran MLVFS which all seems normal now. One file didn't show up because it was corrupted (M04-1342.MLV -- small size 608 bytes) probably due from me fiddling with the 5D3 too quick in-between takes (pressing play button twice, etc).

I'll try to reproduce this on my end. All of this was used with the recent new update of crop_rec.mo (http://www.magiclantern.fm/forum/index.php?topic=17021.msg165282#msg165282crop_rec.mo) that was released by @a1ex in conjunction with @beauchampy's ML configure settings (http://www.magiclantern.fm/forum/index.php?topic=5473.msg165307#msg165307@beauchampy's%20ML%20configure%20settings) in order to try and troubleshoot down the corruption frames bug during slo-mo's on 5D3's.

Meanwhile, here's that particular corrupted file if anyone's curious ... https://mega.nz/#!3p9QUJCA!xeBR6vQEuPVEAlMO2LidLxGjwAxoWqLuiMoYoJXC43I
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: TheJuice on April 04, 2016, 08:00:00 PM
Hi,

Quote
Just tried a folder named: "MLV🍺 " (U+1F37A) and it worked just fine, you might have to be a little more specific about what you tried, version, OS, etc.

I have an MVL file in this path for example :
I:\_Photos\2014-08 Oléron\Movies\sieste\M19-1650.MLV

When I enter the "I:\_Photos\2014-08 Oléron\" directory :

Explorer screenshot:
(http://snag.gy/qT44x.jpg)

Web Server screenshot:
(http://snag.gy/LvUyB.jpg)

I have Windows 10, and just installed Dokan and MLVFS yesterday. I don't know how to check MLVFS's version, as it displays :
>mlvfs --version
Version: UNKNOWN
Date:    UNKNOWN

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on April 04, 2016, 10:10:10 PM
according to https://github.com/dokan-dev/dokany/blob/master/dokan_fuse/src/fusemain.cpp
the strings must be UTF-8.
utf8_to_wchar_buf(name, find_data.cFileName, MAX_PATH);

converting the filenames properly from CP to UTF-8 resulted in the directory with accent not being listed anymore.
seems like some mlvfs and dokan issue.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andy kh on April 12, 2016, 09:38:55 AM
im not able drag and drop mlv audio on timeline in the latest version of premiere pro cc 2015.2. i can drag and drop only the video

edit: change some settings in audio track and now it works
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: TheJuice on April 16, 2016, 12:26:03 PM
Quoteaccording to https://github.com/dokan-dev/dokany/blob/master/dokan_fuse/src/fusemain.cpp
the strings must be UTF-8.
utf8_to_wchar_buf(name, find_data.cFileName, MAX_PATH);

converting the filenames properly from CP to UTF-8 resulted in the directory with accent not being listed anymore.
seems like some mlvfs and dokan issue.

Hi,
Is it going to be fixed in a future version ?
Thanks :-)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: canneloni on April 28, 2016, 04:34:23 PM
I somehow feel like I am way to stupid to use MLVFS. I use Windows 8.1 x64 and downloaded Dokani and MLVFS from this post:

http://www.magiclantern.fm/forum/index.php?topic=13152.msg163041#msg163041

I unpacked it directly to "C:/" and start the Command Prompt with the Parameters. After that, there is no error message but i don't see a new drive in my Computer. When i try to connect to "localhost:8000" there is no server found.

I want to use MLVFS to shoot RAW with my canon 100D and get rid of the focus pixels. I tried MLRaw Viewer, which is okay, but isn't developed any more. And i tried MLVProducer which does not offer the Focus Pixel Map for the 100D. When i use the full frame one i feel like it loses some of its sharpness.

(http://www.image-share.com/upload/3226/237m.jpg) (http://www.image-share.com/ijpg-3226-237.html)

Edit:

Is there a way to update the first post with all the right download links and stuff? I feel like there have been so many discussions going on that the first post hasen't been updated in a while. Maybe someone will write a complete updated workflow tutorial?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andy kh on April 28, 2016, 05:45:42 PM
#canneloni
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: canneloni on April 29, 2016, 11:17:43 AM
Thanks for your answer. I already whatched that video but i am wondering, since i am on Windows 8.1 do i have to use mlvfs_x86 or mlvfs_x64 ? And do i have to use Dokan 0.8 or 1.0 ? I will try again with x_86 and dokan 0.8 just to see if it works.

Thank you!

Edit: I tried with the download file in the first post, so mlvfs_x86 and Dokan 0.8 but it still won't work :/ Maybe because i also tried to use the quick mount function at one point?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on April 30, 2016, 12:06:45 AM
This has been working for me just fine on a MacBook Pro OS X Yosemite 10.10.5

But with exactly the same files - and following the same install - on a 2013 Mac Pro OS X El Capitan 10.11.4 I get a "localhost:8000" error in Safari and the mounts don't appear

Any ideas how I can fix this?

Many thanks
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on May 02, 2016, 03:12:24 AM
Post the contents of ~/.mlvfs.log
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: markodarko on May 02, 2016, 07:36:54 AM
Quote from: Simonwb on April 30, 2016, 12:06:45 AM
This has been working for me just fine on a MacBook Pro OS X Yosemite 10.10.5

But with exactly the same files - and following the same install - on a 2013 Mac Pro OS X El Capitan 10.11.4 I get a "localhost:8000" error in Safari and the mounts don't appear

The only time I experienced that error was when using an attached drive with characters or length of drive name that MLVFS didn't like. I renamed it to a one-word name and all was well.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 02, 2016, 03:25:30 PM
Quote from: dmilligan on May 02, 2016, 03:12:24 AM
Post the contents of ~/.mlvfs.log

Thanks but I couldn't find such a log - I looked in Console.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 02, 2016, 05:42:06 PM
Quote from: markodarko on May 02, 2016, 07:36:54 AM
The only time I experienced that error was when using an attached drive with characters or length of drive name that MLVFS didn't like. I renamed it to a one-word name and all was well.

Thanks for the tip, but I was using a folder I simply named "dng"
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on May 03, 2016, 01:04:42 AM
It's a hidden file in your home folder.

Open the terminal and type

cat ~/.mlvfs.log


Try a different folder name. The name "dng" could potentially confuse MLFVS (b/c DNG files end in "dng".)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Roberto Mena on May 03, 2016, 03:28:22 AM
How come when I turn off my computer and I open up Premiere Pro the MLV files I converted with MLVFS to DNG go missing on my Premiere Pro project and only the template for the files remain but the actual files disappear?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Licaon_Kter on May 03, 2016, 09:22:38 AM
And you did restart MLVFS right?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 03, 2016, 02:31:29 PM
Quote from: dmilligan on May 03, 2016, 01:04:42 AM
It's a hidden file in your home folder.

Open the terminal and type

cat ~/.mlvfs.log


Try a different folder name. The name "dng" could potentially confuse MLFVS (b/c DNG files end in "dng".)

Hi @dmilligan Thanks for the tip on where to find the log.  Here's the contents:

Last login: Fri Apr 29 21:01:04 on console
Simons-MBP-2:~ simonwilkinson-blake$ cat ~/.mlvfs.log
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
nohup: ./mlvfs: No such file or directory
Simons-MBP-2:~ simonwilkinson-blake$

I did try a different folder name, still the same error.

Thanks for your help with this!

Simon




Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 03, 2016, 03:05:54 PM
download and install again, it looks like the actual mlvfs executable is missing (it should be inside the automator workflow package)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 03, 2016, 03:55:52 PM
Quote from: dmilligan on May 03, 2016, 03:05:54 PM
download and install again, it looks like the actual mlvfs executable is missing (it should be inside the automator workflow package)

@dmilligan thanks.  Reinstalled OSXFUSE and MLVFS - same error as before, unfortunately

However, looking in my Library>Automator folder, I can't see anything that looks obviously MLVFS.  Doing a search on my Mac for 'MLVFS' doesn't bring up anything that looks like an executable file.  But then, neither can I find a MLVFS file on my other Mac, on which MLVFS works fine
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 03, 2016, 04:10:56 PM
the executable should be located at: ~/Library/Services/MLVFS.workflow/Contents/mlvfs

in the terminal type

ls ~/Library/Services/MLVFS.workflow/Contents/


You can also run mlvfs from the terminal, if you do without arguments you should see the command line help:

~/Library/Services/MLVFS.workflow/Contents/mlvfs
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 03, 2016, 04:43:55 PM
Quote from: dmilligan on May 03, 2016, 04:10:56 PM
the executable should be located at: ~/Library/Services/MLVFS.workflow/Contents/mlvfs

in the terminal type

ls ~/Library/Services/MLVFS.workflow/Contents/


You can also run mlvfs from the terminal, if you do without arguments you should see the command line help:

~/Library/Services/MLVFS.workflow/Contents/mlvfs


@dmilligan thanks

On your first code, I get:

Last login: Tue May  3 15:34:24 on ttys000
Simons-MBP-2:~ simonwilkinson-blake$ /Library/Services/MLVFS.workflow/Contents/
-bash: /Library/Services/MLVFS.workflow/Contents/: is a directory
Simons-MBP-2:~ simonwilkinson-blake$

On your second, I get

Last login: Tue May  3 13:27:23 on ttys000
Simons-MBP-2:~ simonwilkinson-blake$ ~/Library/Services/mlvfs
-bash: /Users/simonwilkinson-blake/Library/Services/mlvfs: No such file or directory
Simons-MBP-2:~ simonwilkinson-blake$ ~/Library/Services/MLVFS.workflow/contents/mlvfs
-bash: /Users/simonwilkinson-blake/Library/Services/MLVFS.workflow/contents/mlvfs: No such file or directory
Simons-MBP-2:~ simonwilkinson-blake$ mlvfs
-bash: mlvfs: command not found
Simons-MBP-2:~ simonwilkinson-blake$ /library/services/mlvfs
-bash: /library/services/mlvfs: No such file or directory
Simons-MBP-2:~ simonwilkinson-blake$ /library/services/MLVFS.workflow/Contents/mlvfs
main.c:1995:main(): MLVFS: no mlv path specified

usage: mlvfs mountpoint [options]

general options:
    -o opt,[opt...]        mount options
    -h   --help            print help
    -V   --version         print version

FUSE options:
    -d   -o debug          enable debug output (implies -f)
    -f                     foreground operation
    -s                     disable multi-threaded operation

no mount point

MLVFS options:

- File/folder options:
    --mlv-dir=%s           Directory containing MLV files
    --resolve-naming       DNG file names compatible with DaVinci Resolve

- Processing options:
    --cs2x2                2x2 chroma smoothing
    --cs3x3                3x3 chroma smoothing
    --cs5x5                5x5 chroma smoothing
    --bad-pix              Fix bad pixels (autodetected)
    --really-bad-pix       Aggressive bad pixel fix
    --fix-pattern-noise    Fix row/column noise in shadows (slow)
    --stripes              Vertical stripe correction in highlights (nonuniform column gains)
    --deflicker=%d         Per-frame exposure compensation for flicker-free video
                           (your raw processor must interpret the BaselineExposure DNG tag)

- Dual ISO options:
    --dual-iso-preview     Preview Dual ISO files (fast)
    --dual-iso             Render Dual ISO files (high quality)
    --amaze-edge           Dual ISO: interpolation method (high quality)
    --mean23               Dual ISO: interpolation method (fast)
    --no-alias-map         Dual ISO: disable alias map
    --alias-map            Dual ISO: enable alias map

- Web GUI options:
    --port=%s              Port used for web GUI (default: 8000)
    --fps=%f               FPS used for playback in web GUI

- Diagnostic options:
    --version              Display MLVFS version

Simons-MBP-2:~ simonwilkinson-blake$
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 03, 2016, 05:17:23 PM
in the terminal type

ls ~/Library/Services/MLVFS.workflow/Contents/


@dmilligan sorry, missed out the 'ls' in the code.  When I include it, I get:

Last login: Tue May  3 15:42:25 on ttys000
Simons-MBP-2:~ simonwilkinson-blake$ ls /Library/Services/MLVFS.workflow/Contents/
80000301_1808x1190.fpm   80000326_2592x1108.fpm   80000346_1872x1060.fpm
80000301_1808x727.fpm   80000331_1808x1190.fpm   80000346_2592x1108.fpm
80000301_1872x1060.fpm   80000331_1808x727.fpm   Info.plist
80000301_2592x1108.fpm   80000331_1872x1060.fpm   document.wflow
80000326_1808x1190.fpm   80000331_2592x1108.fpm   html_template.html
80000326_1808x727.fpm   80000346_1808x1190.fpm   jquery-1.12.0.min.js
80000326_1872x1060.fpm   80000346_1808x727.fpm   mlvfs
Simons-MBP-2:~ simonwilkinson-blake$

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 03, 2016, 07:13:50 PM
FYI, file paths are case sensitive.

There doesn't appear to be anything wrong. The error message: "nohup: ./mlvfs: No such file or directory" is a mystery. You can try actually launching MLVFS from the command line (like you did when you saw the help), but actually supply the required parameters:

~/Library/Services/MLVFS.workflow/Contents/mlvfs <mount point> --mlv_dir=<directory with MLV files>


where <mount point> and <directory with MLV files> are actual file paths (hint: drag and drop folders onto the terminal window will insert the file path for that folder at the cursor position)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 03, 2016, 08:40:35 PM
Quote from: dmilligan on May 03, 2016, 07:13:50 PM
FYI, file paths are case sensitive.

There doesn't appear to be anything wrong. The error message: "nohup: ./mlvfs: No such file or directory" is a mystery. You can try actually launching MLVFS from the command line (like you did when you saw the help), but actually supply the required parameters:

~/Library/Services/MLVFS.workflow/Contents/mlvfs <mount point> --mlv_dir=<directory with MLV files>


where <mount point> and <directory with MLV files> are actual file paths (hint: drag and drop folders onto the terminal window will insert the file path for that folder at the cursor position)

@dmilligan I tried and got this:

Last login: Tue May  3 19:29:11 on ttys000
Simons-MBP-2:~ simonwilkinson-blake$ /Library/Services/MLVFS.workflow/Contents/mlvfs </Users/simonwilkinson-blake/Desktop/test> --mlv_dir=</Users/simonwilkinson-blake/Desktop/mlv>
-bash: syntax error near unexpected token `newline'
Simons-MBP-2:~ simonwilkinson-blake$

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 03, 2016, 09:26:55 PM
the tilde ("~") at the beginning is very important, and you don't actually include the "<" and ">"


~/Library/Services/MLVFS.workflow/Contents/mlvfs /Users/simonwilkinson-blake/Desktop/test --mlv_dir=/Users/simonwilkinson-blake/Desktop/mlv
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 03, 2016, 09:44:56 PM
Quote from: dmilligan on May 03, 2016, 09:26:55 PM
the tilde ("~") at the beginning is very important, and you don't actually include the "<" and ">"


~/Library/Services/MLVFS.workflow/Contents/mlvfs /Users/simonwilkinson-blake/Desktop/test --mlv_dir=/Users/simonwilkinson-blake/Desktop/mlv


@dmilligan thanks for the corrections.

When I add that command as you've written it into Terminal, I get back:

Last login: Tue May  3 19:35:37 on ttys000
Simons-MBP-2:~ simonwilkinson-blake$ ~/Library/Services/MLVFS.workflow/Contents/mlvfs /Users/simonwilkinson-blake/Desktop/test --mlv_dir=/Users/simonwilkinson-blake/Desktop/mlv
-bash: /Users/simonwilkinson-blake/Library/Services/MLVFS.workflow/Contents/mlvfs: No such file or directory
Simons-MBP-2:~ simonwilkinson-blake$
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 03, 2016, 10:30:05 PM
this is getting weird, try:

otool -L ~/Library/Services/MLVFS.workflow/Contents/mlvfs
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 03, 2016, 10:53:52 PM
otool -L ~/Library/Services/MLVFS.workflow/Contents/mlvfs

[/quote]

@dmilligan I got:

Simons-MBP-2:~ simonwilkinson-blake$ otool -L ~/Library/Services/MLVFS.workflow/Contents/mlvfs
xcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.
Simons-MBP-2:~ simonwilkinson-blake$

I didn't install any tools. If you want me to, pls let me know which, I wouldn't know where to start!
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on May 04, 2016, 01:58:21 AM
Meh, IDK what is wrong, this is very strange. Maybe something wrong with OSXFuse installation. It's really strange that MLVFS will display the command line help, but you get the no such file error when you actually try to use it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 04, 2016, 12:55:59 PM
Quote from: dmilligan on May 04, 2016, 01:58:21 AM
Meh, IDK what is wrong, this is very strange. Maybe something wrong with OSXFuse installation. It's really strange that MLVFS will display the command line help, but you get the no such file error when you actually try to use it.

Many thanks for taking a look, I really appreciate what you and the other developers here have done with ML.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: canneloni on May 04, 2016, 05:40:48 PM
Isn't there anyone with an idea for what might be the problem in my case? I would really appreciate an answer.

Thank you very much for your work on Magic Lantern and for the tools to make the most out of it! :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on May 05, 2016, 04:52:19 AM
@cannelloni

Re: the m22-1744 in your --mlv-dir path. Is this the actual mlv file or the directory where that mlv resides. The path should be to the directory where the mlv files reside.

Also, have you actually checked that dokan is loaded? See below for how to check (in a cmd window):



C:\Program Files\Dokan\Dokan Library-1.0.0>cd /d C:\Program Files\Dokan\Dokan Library-1.0.0

C:\Program Files\Dokan\Dokan Library-1.0.0>dokanctl
Driver path: 'C:\WINDOWS\system32\drivers\dokan1.sys'
dokanctl /u MountPoint (/f)
dokanctl /m
dokanctl /i [d||n]
dokanctl /r [d||n]
dokanctl /v

Example:
  /u M                : Unmount M: drive
  /u C:\mount\dokan : Unmount mount point C:\mount\dokan
  /u 1                : Unmount mount point 1
  /u M /f             : Force unmount M: drive
  /i d                : Install driver
  /i n                : Install network provider
  /r d                : Remove driver
  /r n                : Remove network provider
  /d [0-9]            : Enable Kernel Debug output
  /v                  : Print Dokan version

C:\Program Files\Dokan\Dokan Library-1.0.0>dokanctl /v
Driver path: 'C:\WINDOWS\system32\drivers\dokan1.sys'
dokanctl : Mar  8 2016 13:07:15
Dokan version : 100
Dokan driver version : 0x190

C:\Program Files\Dokan\Dokan Library-1.0.0>




The above is for the 64bit version. If you don't get any or, the correct version displayed then you probably need to install the dokan driver and network provider.

Do this via the following commands:
dokanctl /i d
dokanctl /i n

You only need to do this once, it will automatically load at boot time from then on.

You might also like to create a batch file to launch MLVFS. below is an example of mine.



cd /d C:\OtherPrograms\MagicLantern\MLVFS_x64
mlvfs_x64.exe X: --mlv-dir=D:\MLVs




You won't see the drive mounted in the command window, check under "This PC" in file manager to see the dokan virtual drive.  Also, you need to keep the command window active the entire time that you are using MLVFS. Just minimise it. If you close the command window, the dokan virtual drive will be unmounted.

Assuming that you can now see your virtual dokan drive. Fire up you browser and enter http://localhost:8000/ You should now see all of the MLVs listed.

A little tip:  Choose your MLVFS configuration options in the browser before accessing the mlv file in the dokan virtual drive. It saves you having to force or wait for a refresh cycle in file manger.

Hope this helps....
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marekk on May 09, 2016, 10:20:25 PM
Hi dmilligan,
Thanks for your hard work with mlvfs.

I'm currently converting mlv footage from my holidays trip to dng sequences and I've found that vertical stripes fix doesn't work. I've tested it with mlv_dump and it works fine. Is it possible for you to implement full vertical stripes fix from original raw2dng code ? I've found your fix code is different than raw2dng..

Thanks in advance
marekk
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on May 09, 2016, 10:55:25 PM
@marekk -- Did you forget to refresh the webpage (local host) after changing your options choices?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marekk on May 10, 2016, 09:20:20 AM
Quote from: DeafEyeJedi on May 09, 2016, 10:55:25 PM
@marekk -- Did you forget to refresh the webpage (local host) after changing your options choices?

I've tried it but it doesn't work correctly. I've found that part of the mlvfs code responsible for vertical stripes fixing is the same but procedure to fix is much shorter.


void stripes_apply_correction(struct frame_headers * frame_headers, struct stripes_correction * correction, uint16_t * image_data, off_t offset, size_t size)
{
    if(correction == NULL || !correction->correction_needed) return;
    if(frame_headers->rawi_hdr.xRes % 8 != 0) return;
   
    uint16_t black = frame_headers->rawi_hdr.raw_info.black_level;
    uint16_t white = frame_headers->rawi_hdr.raw_info.white_level;
    size_t start = offset % 8;
    for(size_t i = 0; i < size; i++)
    {
        double correction_coeffficient = correction->coeffficients[(i + start) % 8];
        if(correction_coeffficient && image_data[i] > black + 64)
        {
            image_data[i] = (uint16_t)MIN(white, (image_data[i] - black) * correction_coeffficient / FIXP_ONE + black);
        }
    }
}




It removes vertical stripes only from some parts of the dng file. Full code from raw2dng removes all vertical stripes. I shot 256GB of mlv footage (6-10 sec), so I've verified it in a lot of files.
Mlv_dump output is nice but in Davinci Resolve 12.5 is presented as a folder with video and audio file. Correct naming scheme doesn't help. I think that Davinci REsolve is looking for specific tags in DNG file metadata, maybe "Reel Name"..
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 10, 2016, 12:56:43 PM
MLVFS caches the correction coefficients so it runs faster. Everything else is the same.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marekk on May 10, 2016, 01:52:02 PM
Quote from: dmilligan on May 10, 2016, 12:56:43 PM
MLVFS caches the correction coefficients so it runs faster. Everything else is the same.

I think that correction coefficients is done in the same way, but "white" is calculated in different way..
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 10, 2016, 03:13:51 PM
White is only used to constrain the output values (to make sure the correction calculation doesn't produce values greater than white), therefore the only place you will see any effect is in areas that have been clipped.
Here's the equivalent code for the correction calculation in the code you posted:

    image_data[i] = (image_data[i] - black) * correction_coeffficient / FIXP_ONE + black;
    if(image_data[i] > white) image_data[i] = white;


Caching the white value as computed by the original raw2dng is a bad idea for our purposes, b/c this white level can actually be quite wrong if applied to a different frame (raw2dng does not compute the true white level, it computes a heuristic that is good "*for this particular purpose*"). If the analyzed frame is quite dark, the white level computed by this method will be much lower than the true white level; if you then take that value and apply it to a brighter frame, you will end up clipping a whole bunch of values that aren't actually clipped. So instead, we rely on the one from metadata because computing the true white level for every frame would severely impact performance.


        /** 
         * inexact white level will result in banding in highlights, especially if some channels are clipped 
         * 
         * so... we'll try to use a better estimation of white level *for this particular purpose* 
         * start with a gross under-estimation, then consider white = max(all pixels) 
         * just in case the exif one is way off 
         * reason: 
         *   - if there are no pixels above the true white level, it shouldn't hurt; 
         *     worst case, the brightest pixel(s) will be underexposed by 0.1 EV or so 
         *   - if there are, we will choose the true white level 
         */ 
        int white = raw_info.white_level * 2 / 3; 

        struct raw_pixblock * row; 
         
        for (size_t i = 0; i < size; i++) 
        { 
            white = MAX(white, image_data[i]); 
        } 





For these reasons, I really don't think it has anything to do with white level. My guess as to why it might not be working in some cases is actually the caching. Caching the coefficients makes two assumptions: the defect is constant over time (for at least for the duration of a clip), and all frames will show the defect (so we can analyze any frame and use it's coefficients to correct other frames). I think the first assumption is pretty safe, but the second one might not be if there's a dramatic change in the scene/lighting/exposure/etc.

You have a couple of options then:
- analyze every frame (slow -> you might loose real-time playback, IDK)
- analyze a random sample of frames, choose the 'best' coefficients (this has the potential of actually working better than the original algorithm, since you might be able to correct a frame that doesn't show the defect very strongly, based on other frames that do)
- analyze every N frames (maybe N is user settable)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marekk on May 10, 2016, 03:19:54 PM
Thanks for explanation.

How can I force it to analyze every frame ? I don't need real-time playback.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 10, 2016, 03:50:13 PM
comment out line 983 in main.c


               //if(correction == NULL)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marekk on May 10, 2016, 05:58:31 PM
I've just compiled it ad now it works like a charm :)
Thanks !
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marekk on May 10, 2016, 06:22:43 PM
I don't know what's wrong but I compiled it also with original code and it works...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: canneloni on May 12, 2016, 02:00:05 PM
Quote from: Ottoga on May 05, 2016, 04:52:19 AM
@cannelloni

Re: the m22-1744 in your --mlv-dir path. Is this the actual mlv file or the directory where that mlv resides. The path should be to the directory where the mlv files reside.

Also, have you actually checked that dokan is loaded? See below for how to check (in a cmd window):
...

@Ottoga
First of all thank you for your response!

I used the path where the mlv file is, so there is no subfolder left.

I tried the to check if dokan is loaded. When i type in the comments, there is is just a very short popup of another command prompt but i dont see what is written there. There is no further information in the prompt where i typed in the command. I tried to install the drivers but there was the same issue.

Would it help if i record my screen?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on May 12, 2016, 02:27:37 PM
@canneloni

Some screen shots or a video clip taken with your smartphone would help.

Also, double check whether you have a 32 or 64 bit version of windows installed and that you are using the correct version for your operating system.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: canneloni on May 12, 2016, 03:53:33 PM
Im using the x64 bit version and i have a x64 Windows as you can see in the video below:

https://youtu.be/Jp3c3TVPrfc
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bengibb on May 12, 2016, 09:49:06 PM
Been a bit since I've used MLVFS but when I select it from the right-click menu I'm just getting a spinning gear with a 0% complete message in the menu bar. This is on Mac. What am I doing wrong?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on May 12, 2016, 11:31:40 PM
Did you choose the mount folder?Sometimes the dialog comes up behind other windows.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bengibb on May 12, 2016, 11:50:38 PM
Wasn't prompted for a mount folder. Checked if it was hidden. That's kind of what's got me scratching my head.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 13, 2016, 12:58:38 AM
That's the very first thing it does, so you probably missed it. Close all open windows and try again.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on May 13, 2016, 04:50:21 AM
@canneloni

Thanks for the video clip. A couple of points:

From what I've seen, I think that the dokan service is installed and running. to convince yourself just try the commands (exactly as per below again in a cmd window with Admin rights). :




C:\Program Files\Dokan\Dokan Library-1.0.0>cd /d C:\Program Files\Dokan\Dokan Library-1.0.0

C:\Program Files\Dokan\Dokan Library-1.0.0>dokanctl

Driver path: 'C:\WINDOWS\system32\drivers\dokan1.sys'
dokanctl /u MountPoint (/f)
dokanctl /m
dokanctl /i [d||n]
dokanctl /r [d||n]
dokanctl /v

Example:
  /u M                : Unmount M: drive
  /u C:\mount\dokan : Unmount mount point C:\mount\dokan
  /u 1                : Unmount mount point 1
  /u M /f             : Force unmount M: drive
  /i d                : Install driver
  /i n                : Install network provider
  /r d                : Remove driver
  /r n                : Remove network provider
  /d [0-9]            : Enable Kernel Debug output
  /v                  : Print Dokan version

C:\Program Files\Dokan\Dokan Library-1.0.0>dokanctl /v
Driver path: 'C:\WINDOWS\system32\drivers\dokan1.sys'
dokanctl : Mar  8 2016 13:07:15
Dokan version : 100
Dokan driver version : 0x190

C:\Program Files\Dokan\Dokan Library-1.0.0>




As I said, I think that the dokan service is up and running on your PC. Can you do a similar clip of what happens when you launch MLVFS. remember it also needs to be in a cmd window with admin rights. Please include the launch process, what you see in file manager and the localhost browser details.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: canneloni on May 13, 2016, 08:46:43 AM
Ah okay, now dokan works/ shows information. But still MLVFS does not work right. I made another video:

https://youtu.be/gTMvMPtRpyY

Thank you so much for your help! :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on May 13, 2016, 09:54:15 AM
@canneloni , MLVFS is very simple to use you are making it more complicated then it really is .
I made a video for PC user's on how to install & execute the program , It is base on Win7 pro
but it still applies to win 10 as it basically has all the some commands .
Video Tutorial--> http://www.magiclantern.fm/forum/index.php?topic=13152.msg164559#msg164559
Now the only difference is win 10 uses the x64 Dokan file version where as win 7 uses x86 that's it.

The Command are the same except navigation to the MLVFSx86 in your case MLVFSx64.
The command stuff starts around the 4:40 mark I do believe .
If you are still having troubles , let me know and I do a quick Win10 Video Tutorial and post it here .

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on May 13, 2016, 02:13:38 PM
@cannelloni

Hmmm,  you appear to be doing everything correctly now. The only thing that I can think of that may be causing it not to work for you is the virtual drive letter that you are using ('X') when launching mlvfs. This virtual drive letter must be unique.

In you clip I noticed that you had three network drives assigned. The first was active but I didn't see what drive letter was assigned to it. The other two were inactive with drive letters "Y' and 'Z' assigned respectively. I am thinking that maybe your first network drive has already been assigned to drive 'X'.

If this is the case, just pick a free (unused) drive letter for use with mlvfs.  e.g.: mlvfs64.exe M: --mlv-dir=c:\mlvs

Failing this, you could try reddeercitys' suggestion to use the 32 bit version of mlvfs.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: canneloni on May 13, 2016, 05:43:48 PM
Okay it works now. I removed everything from Dokan i had installed and cleaned the regestry of every entry with the word "Dokan" in it. I then reinstalled the newest version of Dokan and used mlvfsx64.

Thank you guys so so so much for your work and time! Im so glad i can finally use mlfvs. I have to figure out wich workflow works best for me and my 100D but im pretty sure there is a good way ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: celestyx on May 15, 2016, 11:57:31 PM
Hello,

I installed Dokan and MLVFS_x86 from the first page (running windows 7 64bits).
At first everything seemed to work fine, I can mount the dng files and open them in After Effects. But when I apply settings in ACR inside After Effects, they are only applied to the first frame, not to the whole sequence. It works fine if I put dng files in a regular folder without using MLVFS.
I then realized that I cannot copy a file onto the drive that is mounted, which probably explains that AE can not copy metadata files or write inside the dng files, therefore the settings are applied only to the first frame of the sequence.
I noticed other people here had the same problem, but could not find a clear solution in this thread. Should I use newer versions that are not released yet? If so where do I find them and how do I proceed?

Thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Mefist on May 22, 2016, 04:51:54 PM
@reddeercity

Hello. Why capture DualISO band . When they disappear - the picture becomes darker ? On your video in the second case they were . English I do not know ( ( ( ( . I use a translator .
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: hindra on May 24, 2016, 02:50:24 AM
THANK YOU! I just wanted to make sure you understand how much I appreciate this software.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: fotorebelion on May 25, 2016, 05:06:15 PM
hey.. for some reason.. i dont seem to get git fetch the source code..
can somebody kindly send me the commandline syntax ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 25, 2016, 05:22:17 PM

git clone https://bitbucket.org/dmilligan/mlvfs
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: piloui on May 27, 2016, 08:28:05 PM
Hi there,

After some hours of test and post reading, here is some feedback.

3 different windows test :

- Windows 10 pro upgrade from a windows 7 pro :

- Windows 10 home upgrade from windows 8 home ( laptop ) :
Error 0x80091007: Hash mismatch for path: C:\ProgramData\Package Cache\.unverified\vcredist_2015_x64.exe, expected: 9A19A51D1F40CD5CD5ECB6E4E4F978F18DA8212A, actual: 3155CB0F146B927FCC30647C1A904CD162548C8C
-Windows 7 pro :

I wish i could give more log info, specially for the win 10 pro crashes. Maybe some tips to where i can find dokan/mlvfs logs please ?
Thanks for this great tool.

*edit*
A little batch command with prompt to find your mlvs location. Place the .cmd file within MLVFS_X86 or MLVFS_X64 folder and feel free to place a short cut anywhere you want :
@if (@a==@b) @end /*

:: fchooser2.bat
:: batch portion

@echo off
setlocal

for /f "delims=" %%I in ('cscript /nologo /e:jscript "%~f0"') do (
    echo You chose %%I
    C:\MLVFS_x86\mlvfs.exe z: --mlv-dir=%%I
)

goto :EOF

:: JScript portion */

var shl = new ActiveXObject("Shell.Application");
var folder = shl.BrowseForFolder(0, "Please choose a folder.", 0, 0x00);
WSH.Echo(folder ? folder.self.path : '');
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lostfeliz on May 28, 2016, 05:59:57 AM
Hey all. I've installed FUSE and MLVFS. I see the command file is in the Services folder, but MLVFS doesn't show up as a service when I right click on MLV files. No services show up. I tried re-installing. No help. Any suggestions?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on May 28, 2016, 07:37:45 AM
@lostfeliz - You should be right clicking on a Folder (not the MLV file) and here's how I do it below...

https://vimeo.com/168433918
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lostfeliz on May 29, 2016, 12:32:19 AM
@DeafEyeJedi --- you rock. thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on May 30, 2016, 11:54:36 PM
This might be a noobie question but I use MLVFS and Smart importer 2 in AE. In my mounted folder I have footage with different frame rates and now I wonder what the easiest way to process the footage in AE is, without messing around with the differnet framerates!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Frank7D on May 31, 2016, 05:50:46 AM
@ _OLLE_

So, in AE under edit preferences, under import, you can set the default sequence frame rate. Set that to whatever is your most common frame rate (or change it as needed).

In Smart Import 2, there is a field for the comp frame rate. If you leave it blank, the comp frame rates will be set at the default value you set in AE as mentioned above. If you put a value in there, it should override the default AE value.

Using these two settings, you should be able to deal with different frame rates. However, you might need to run Smart Import 2 script separately for each frame rate.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on May 31, 2016, 11:57:04 AM
Thanks @Frank7D but is there a easy way to divide the footage into lets say one folder with 60fps and one with 30fps? I think it is really timeconsuming to find out the footage framerate with you just have the MLV file
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: aschille84 on May 31, 2016, 12:37:41 PM
This is much easier in resolve, where you can export your log masters in the source framerate/resolution.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: QuickHitRecord on May 31, 2016, 03:03:37 PM
I haven't had this happen with MLVFS before. Every time I try to mount, the web browser (Firefox) gives me this:

QuoteUnable to connect

Firefox can't establish a connection to the server at localhost:8000.

    The site could be temporarily unavailable or too busy. Try again in a few moments.
    If you are unable to load any pages, check your computer's network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

Any idea why this is happening?
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on May 31, 2016, 05:13:44 PM
I get this too whenever I use the wifi connection at work where they exclusively use FireWalls for the sake of the company. I just simply use my iPhone's hotspot to use the Internet in order to bypass this.

Even tho I was told that we can actually use MLVFS without Internet though I'd rather be able to see the local host page.

Do you have anti virus software installed by any chance @QuickHitRecord? Just curious have you also tried on Safari?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Simonwb on May 31, 2016, 06:21:26 PM
Ah similar problem I had, reported here a few pages back. I'll look at my firewall settings. Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 31, 2016, 07:15:29 PM
If you can't see the webgui AND you can't see the MLVFS mount, then MLVFS probably didn't start at all. Check the console output or if on Mac: ~/.mlvfs.log

If you can see the MLVFS mount, but not the webgui, then it means one of several possibilities:

- Your account doesn't have permission to open a TCP port (whichever port you started MLVFS with, the default is 8000).

- That particular TCP port is in use (this could be another instance of MLVFS or something else), try a different one (the Mac Automator service doesn't give you a way to specify the port, so you'll have to use the CLI). There's a slim chance that a previous instance of MLVFS could have crashed but is still holding the port open; to fix this, just restart your computer.

- You are missing the webgui support files, they are in the wrong location, or MLVFS was not launched with the correct working directory and therefore cannot find them (this shouldn't be a problem if using Mac Automator service).


Quote from: DeafEyeJedi on May 31, 2016, 05:13:44 PM
Even tho I was told that we can actually use MLVFS without Internet though I'd rather be able to see the local host page.
You can use MLVFS, even the webgui, without internet connectivity.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on May 31, 2016, 08:07:17 PM
I encountered the "port open" issue after creating a virtual partition with a script. Restart would fix it. Any how, is it possible to do kill -9 <pid> as with any other application process? (no need for restart.)
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on May 31, 2016, 08:28:16 PM
@dmilligan -- Anxious to know how to run MLVFS without internet connectivity (not that much of a priority tho) however I am still encountering that name scheming so called bug? I cannot use punctuation marks while naming the Folder (easier to organize) but still OK with not being able to use it even though I would like to.

Also if I remember correctly I've been able to use them in the past with previous builds without any issues.

https://vimeo.com/168821878

Thanks, David!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 31, 2016, 10:29:22 PM
Quote from: DeafEyeJedi on May 31, 2016, 08:28:16 PM
Anxious to know how to run MLVFS without internet connectivity
1. Disconnect from the internet
2. Use MLVFS as you normally would

I just tried this and it works fine. Tested it with Chrome, Safari, and Firefox. The address http://localhost:8000/ is a local address and it should resolve to your own computer. It does not require any internet connectivity to get to http://localhost:8000/ and MLVFS does not try to use the internet in anyway (that's the main reason jquery is a local resource rather than loading it from say: http://code.jquery.com/jquery-1.11.0.min.js).

Quote from: DeafEyeJedi on May 31, 2016, 08:28:16 PM
however I am still encountering that name scheming so called bug? I cannot use punctuation marks while naming the Folder
The problem is shell expansion. The parenthesis have special meaning and must be escaped. I thought I fixed that one, but I guess not. It's fixed now. New build uploaded.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on June 01, 2016, 06:34:10 AM
Ah, I had thought you'd fixed that bug in previous builds too. Maybe it was just a loophole that I've caused on your end?  :P Just tried running your latest build (cd2d9bb-dirty May 31 201616:12:23) with a bunch of name schemings in which all seems solid and Thanks for the quick fix.

Anyway, in regards to the offline-capabilities which I've learned that once Safari says "You're not connected to internet ... etc" then I copied the http://localhost:8000 (http://localhost:8000) URL link and pasted into a blank web page within Chrome and voila the web gui showed up.

https://vimeo.com/168887077

What can I do to make this work with Safari or better yet to be able to have MLVFS open up with Chrome instead of Safari if there are no other work arounds for this user error on my part?

Thanks again for your time, David!

*edit*

Quote from: Danne on May 31, 2016, 08:07:17 PM
Any how, is it possible to do kill -9 <pid> as with any other application process? (no need for restart.)

+1
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 01, 2016, 05:13:31 PM
Heh.. does OSX interpretes "kill -9"?! not the way other OSes do? :o

If you are root/admin you can kill anything you want exept kernel processes. Just kill mlvfs correctly (no need for -9) that's how it works under linux at least.

If you run mlvfs with some user permission under osx I guess you should not have any problem to kill it with that user too.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 01, 2016, 05:40:42 PM
It's much easier to just tell people to restart their computer than it is to explain to them how to use the CLI to kill a process. It's also foolproof and more effective.

Quote from: DeafEyeJedi on June 01, 2016, 06:34:10 AM
What can I do to make this work with Safari or better yet to be able to have MLVFS open up with Chrome instead of Safari if there are no other work arounds for this user error on my part?
Have you tried manually entering the address in Safari? or refreshing the page? Also try http://127.0.0.1:8000/ and if you're on a proxy (you mentioned something about work), check your proxy settings to make sure localhost and 127.0.0.1 bypass the proxy.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 01, 2016, 06:06:19 PM
Quote from: dmilligan on June 01, 2016, 05:40:42 PM
It's much easier to just tell people to restart their computer than it is to explain to them how to use the CLI to kill a process. It's also foolproof and more effective.

Yeah, very familiar but mostly for windows machines :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: markodarko on June 02, 2016, 02:24:44 AM
MLVFS Version: 6cf1742 Mar 4 201617:23:30

Not sure if this has been reported, @dmilligan, or even if it's a bug at all and just my crummy system (8Gb RAM) but...

Copying out cDNG files from a 7.5Gb MLV takes around a minute, but... copying out cDNG files from a 30Gb MLV (I know, it's crazy) takes over 6 HOURS.

The reason I was copying out cDNG files at all was because that 30Gb MLV file wouldn't load in After Effects. MLVFS would run at 100% and after an hour, still no joy, so I thought I'd copy out the files (as it's quite a large one) to see if that would make a difference and that's when I noticed the same problem - MLVFS runs at 100% and the process took over 6 hours to complete, yet for a file a quarter of the size it takes around a minute. Naturally I was expecting it to take around 4 minutes.

Let me know if you need me to run any specific tests. I think I still have the original MLV file somewhere but I'm not sure. This happened 2 weeks ago and I completely forgot to report it back then I'm afraid.  :-[

Cheers,

Mark.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 02, 2016, 01:23:38 PM
I recently uploaded a new version. Try it. If its still slow and you'd like to help debug the issue and don't mind installing Xcode (free in the app store), then I can walk you through what to do (I need a profiler report from the "Instruments" tool).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: canneloni on June 02, 2016, 04:29:07 PM
will there be a new version for windows too ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: C T Mok on June 07, 2016, 03:28:52 AM
Can someone pls tell me how to remove/uninstall MLVFS from a Mac. Installed on a wrong machine. Thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 08, 2016, 01:30:56 PM
delete ~/Library/Services/MLVFS.workflow
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on June 08, 2016, 11:23:01 PM
Quote from: dmilligan on June 01, 2016, 05:40:42 PM
Have you tried manually entering the address in Safari? or refreshing the page? Also try http://127.0.0.1:8000/ and if you're on a proxy (you mentioned something about work), check your proxy settings to make sure localhost and 127.0.0.1 bypass the proxy.

Yes, I've tried using the http://127.0.0.1:8000/ (http://127.0.0.1:8000/) which works just fine. Even with a firewall wifi at work after manually typing in that URL link. Thanks for the tip!

(https://c6.staticflickr.com/8/7284/27548551805_1874c92592_n.jpg) (https://flic.kr/p/HYnwF8)

However, would you mind explaining on how exactly can I set the localhost 127.0.0.1 to bypass the proxy within Network Settings on the Mac?

(https://c1.staticflickr.com/8/7103/27271862560_c1a9077944_n.jpg) (https://flic.kr/p/HxVqAU)

Thanks again, David!

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: C T Mok on June 09, 2016, 06:46:02 AM
Quote from: dmilligan on June 08, 2016, 01:30:56 PM
delete ~/Library/Services/MLVFS.workflow
Thx dmilligan. You saved my day. However, I would like to add that one must first show the hidden files in OSX in order to see this file path.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: piloui on June 14, 2016, 11:36:21 PM
Hi there,

Does anyone else have huge long time of dng loading from virtual drive inside davinci resolve ? This time seems to increase with number of mlvs, which looks logical. But the navigation inside windows explorer is much more faster than resolve's media pool. With ~ 60 mlvs, it takes an eternity or even never end.
( windows 10, dokan + mlvfs X86 )
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on July 28, 2016, 04:24:36 PM
Hi all,

Having troubles to finding the latest versions for win. mlvfs I list bellow is newer than the one listed in the first page of this thread, but it's from febuary, mac appears to be right?.. would anyone confirm please that these are the latest?

win:
MLVFS x64.zip 27-Feb-2016 10:01: http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/

mac:
MLVFS.dmg 2016-05-31: https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS.dmg


Many thanks,
V
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PabloW on August 05, 2016, 09:26:51 PM
Does anyone used MLVFS on Windows with footage from a T4i / 650D ?

Does this remove pink dots automatically? (focus and sensor dots)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on August 05, 2016, 09:44:47 PM
Yes
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kinkersnick on August 11, 2016, 10:34:31 AM
Hi,

I've been using MLVFS for months without any problems. I've just moved to a new location (and therefore internet connection) and now it's not working. I mention the move, because that's the only thing that's changed.

I'm on OSX 10.11.6.

When I try to run the MLVFS service, I get the 'Site can't be reached' error. If I try to run it in Terminal, it says 'Segmentation fault: 11'.

I am running the latest versions of OSX, Fuse and MLVFS.

I have checked my hosts file is ok, I have made sure I can access localhost (when I go to 127.0.0.1 it says 'It works!').

I cannot figure out how to make this work, nor what I have changed that could have stopped it from working.

Help!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tom2 on August 16, 2016, 10:44:12 AM
Hi,
I love MLVFS because it is so fast. I use MLVFS on my macbook pro.
But can anyone tell me how can I fix the vertical stripes on my 5D mark III footage?
The "Vertical Stripes Fix" option appears on the webpage, but for me it seems to be too late because the "virtual" cndg file has already been mounted... Should I use the "Fix pattern noise" option?...
So far the only app which gives a true "vertical stripes fix" on my 5D Mark III footage is MLRAwViewer. I wish I could do the same with MLVFS
best regards
Tom
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on August 16, 2016, 05:33:13 PM
You still can. Just be sure to 'refresh' the web page after your make your selections (vertical stripe fix, etc) before you start rendering the exports otherwise.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tom2 on August 16, 2016, 09:35:39 PM
thanks a lot!
I did refresh the page, but in the end I couldn't see any difference between the "on" and "off" versions of the  "Vertical Stripes Fix" , on the exported clips of my 5D Mark III
When using MLRawviewer as a DNG converter, the "Vertical Stripes fix" seems to be much more efficient!...
It's strange...
best regards
Tom
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 16, 2016, 10:01:16 PM
Have you tried working with darkframe subtraction. I seen some very good results eliminating banding in shadow areas. Deafeye had some nice results with this.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tom2 on August 16, 2016, 10:10:51 PM
never heard of "darkframe substraction"...
To "develop" my files, I use DaVinciResolve, maybe there is an equivalent in it?
see u
Tom
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 16, 2016, 10:19:26 PM
I think it can be done in resolve but I don,t know how well it works. Mlv_dump provides an averaging setting. You can check more on this implemented here. http://www.magiclantern.fm/forum/index.php?topic=13512.msg164871#msg164871
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on August 17, 2016, 12:52:45 AM
Is the vertical banding present in the highlights or shadows?

If the banding is in the highlights, darkframe subtraction won't help. Banding in the highlights is a sign of a multiplicative defect (the column gains are off slightly, which is fixed by multiplying pixels values by some correction factor).

If the banding is in the shadows, the "vertical stripe fix" won't help. Banding in the shadows is a sign of an additive defect (there is some linear offset, which is fixed by simply subtracting some correction value from the pixel values).
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on August 17, 2016, 04:03:04 AM
Perhaps share some or at least one of the original MLV that shows these issues @tom2?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kinkersnick on August 17, 2016, 09:27:28 PM
Hi,

I've used MLVFS a few times in the past with no problems, but now all of a sudden it has stopped working. Every time I try to use it, it opens up the web browser to http://localhost:8000/ but says:

This site can't be reached
localhost refused to connect.


I've read in some places that this can be a problem with apache, but I've checked in Terminal and it's running. When I go to 127.0.0.1 it says 'It works!' so there's no problem there.

I have tried moving just one MLV file into a folder and it has the same problem.

I'm on a really tight deadline here, so any help will be much appreciated! In the meantime, can anyone recommend an alternate method for batch converting MLV to DNG on Mac?

Here is an example crash report: https://we.tl/CDRuYH9JY3
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on August 20, 2016, 11:05:11 PM
Any strange or unusual characters in the path name of the folder you're trying to mount? Anything in ~/.mlvfs.log ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tom2 on August 21, 2016, 09:44:56 AM
hi,
by request I send 3 pictures of the vertical stripes, which shows that MLRawviewer seems much more efficient than MLVFS and Rawmagic... https://we.tl/xzqMJCw23n
I wish the fast MLVFS could be as efficient as MLRAWviewer for the vertical stripes!
tom
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 21, 2016, 12:31:52 PM
From looking at your examples i think you would get even better results with darkframe averaging. Dfa will also get rid of color artifacts.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on August 21, 2016, 09:16:36 PM
Didn't ask for samples. Asked for originals. Here's an 'example' of what DF avg processed soley in MLP can do (Thanks @Danne & @baldavenger as well as to @g3gg0 for updating mlv_dump) w a footage shot from 5D3 w 16-35L in 24p 1080p wide open @ ISO 3200 (keep in mind I intentionally pushed the exposure up +5.00 in ACR for this test comparison) below.

Pre-DF avg:
(https://c3.staticflickr.com/9/8314/28849933250_49cf5ecc86.jpg) (https://flic.kr/p/KXns3f)                                                                                                 

Post-DF avg:
(https://c2.staticflickr.com/9/8465/29059880121_92c0f55fd9.jpg) (https://flic.kr/p/LgVtVT)

I believe this can also be done while running MLVFS in conjunction with MLP's DF avg processing together as well. Oops, my wrong! (check below)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 21, 2016, 10:20:49 PM
Great example Deafeye. Can,t be done via mlvfs. It,s thanks to g3gg0,s mlv_dump this works. I put together som streamlined workflows though which could be used seemlessly in MLP together with non averaged footage.
Title: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on August 21, 2016, 10:34:50 PM
Quote from: Danne on August 21, 2016, 10:20:49 PM
It,s thanks to g3gg0,s mlv_dump this works.

Thanks for the reminder!

@kinkersnick -- were you able to get it working again with MLVFS on your end?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: hyalinejim on August 23, 2016, 02:32:01 AM
I'd like to quickly and efficiently be able to do this for some high ISO shots before bringing them into ACR. But I'm on a PC.

Are there any GUI based software applications that can perform a dark frame subtraction on Windows? I know that mlv_dump can do it, but I can't figure out how to use it. Even if I could, I get the feeling that it involves a lot of typing and moving files around manually and that there would be a certain amount of referring to a GUI based program to identify the correct ISO, etc. etc.

And I know that MLVProducer can do it as well, but the DNGs it exports are no different with or without the subtraction - it only seems to work when exporting to other formats and I want to stick with ACR and Cinelog. So are there any GUIs that can do dark frame averaging and subtraction?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PabloW on August 23, 2016, 03:01:14 AM
I just started using MLVFS and made a tutorial for myself to remember how to use it, but in Spanish. I'll just paste it here, maybe it can help someone.

QuoteComo convertir MLV a secuencias DNG, listas para editar.

1. Bajar e instalar Dokany y MLVFS_x86 desde el siguiente link (http://magiclantern.fm/forum/index.php?topic=13152.0)
el MLVFS_x86 es una carpeta, conviene ubicarla en el disco C: o similar para facilitar los siguientes pasos.

2. Una vez instalado todo, hay que montar los archivos MLV a un disco virtual, para poder usarlos como secuencias DNG que es lo podemos editar en Premiere, AE, Resolve, etc.

a) Crear una carpeta donde vayamos a ubicar los archivos de video .MLV (idealmente en el disco mas rápido que tengamos). Por ejemplo en el disco E: y la carpeta le ponemos MLVFS de nombre.

b) Iniciar el programa abriendo un cmd y poniendo esto
c:
cd \MLVFS_x86
mlvfs.exe M:\ --mlv-dir=E:\MLVFS

(en el primer paso fuimos al disco C, en el segundo a la carpeta MLVFS_x86, en el tercero abrimos el programa y montamos un disco virtual "M:" tomando la dirección de la carpeta que creamos antes que en este caso la hice en "E:\MLVFS")

Dejar abierto este CMD siempre que usemos los archivos que estemos editando.

3. Ya podemos ver el disco virtual M: creado. Si tiramos los archivos MLV a la carpeta MLVFS aparecerán en el disco M: en carpetas ya convertidos a DNG, listos para usar y sin ocupar espacio extra.

4. Podemos entrar en el navegador a: http://localhost:8000/
Ahi podemos editar ciertas características de nuestros archivos y ver un preview en GIF
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: hyalinejim on August 23, 2016, 03:42:02 AM
Thanks Pablo, but that's not what I was asking. MLVFS cannot do dark frame averaging and subtraction and I'd like to know if there's a GUI frontend for Windows that can accomplish that.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: AWPStar on August 26, 2016, 05:04:26 PM
Hi, hyalinejim!
MLVProducer should export to dng with FPN processed.
Here is example.
https://sourceforge.net/projects/mlvproducer/files/FPN_DNG_test.zip/download (https://sourceforge.net/projects/mlvproducer/files/FPN_DNG_test.zip/download)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: hyalinejim on August 27, 2016, 10:12:11 AM
Thanks AWP Star, I'll continue the discussion in the thread for MLV Producer
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Quentin on August 29, 2016, 02:24:32 PM
Just tried MLVFS on my PC and my Canon 5d mk3.
Am I mistaken or is not possible to take advantage of Adobe's RAW color processing ?
I use AfterFX and although the sequence is loaded fine, the processing I do applies only on first frame.
I assume its because FS is read only.

Please help
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Quentin on September 04, 2016, 06:25:42 AM
After careful observations I realized that the FS is read only, thats why its impossible to use ACR.
By opening all DNGs in Photoshop first and process the colors, its possible to use this adjustment in AFX.
Nothing to do in Premiere. Overall the FS although promising, its not mature enough for real use, in my opinion.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Lars Steenhoff on September 04, 2016, 09:30:40 AM
just copy the the dng files to a folder on your computer
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dmilligan on September 04, 2016, 06:11:43 PM
That defeats the whole purpose.

MLVFS is not read only, but I believe there may be some dokany issues (either with dokany itself or the way MLVFS interacts with it). I don't have Windows machine so I can't really support the Windows version, but feel free to pull the source and try with the latest dokany version.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: waza57 on September 07, 2016, 08:01:44 PM
Hello,
For a gain of speed I try to play with registry in windows .

in
          HKEY_CLASSES_ROOT\Directory\shell\ we can create a key named "mount MLV" (clik right->new key)
then in
          HKEY_CLASSES_ROOT\Directory\shell\mount MLV we must create a key named "command" 
and then:
          we create on right side a string value like this (right click-> new string value) and tape this

cmd /c "start /D <path where mlvs.exe is located> mlvfs.exe x: --mlv_dir="%1" && "C:\Program Files (x86)\Mozilla Firefox\firefox.exe" localhost:8000"

be Careful we must left a space between the path and mlvfs.exe!

Now with a right click on a folder with MLV files we can mount it and start the web gui in the same time.

hope this helps.


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on September 24, 2016, 01:22:55 AM
Hello everyone,

     Has anyone tested MLVFS on Mac OS Sierra? I haven't upgraded yet, and don't want to unless I know it works. Thanks.

- Paul Harwood
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 24, 2016, 08:09:14 PM
Works fine here
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on September 25, 2016, 04:56:55 AM
Alright awesome, thanks dmilligan!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: canneloni on October 10, 2016, 11:43:31 AM
Quote from: waza57 on September 07, 2016, 08:01:44 PM
Hello,
For a gain of speed I try to play with registry in windows .

in
          HKEY_CLASSES_ROOT\Directory\shell\ we can create a key named "mount MLV" (clik right->new key)
then in
          HKEY_CLASSES_ROOT\Directory\shell\mount MLV we must create a key named "command" 
and then:
          we create on right side a string value like this (right click-> new string value) and tape this

cmd /c "start /D <path where mlvs.exe is located> mlvfs.exe x: --mlv_dir="%1" && "C:\Program Files (x86)\Mozilla Firefox\firefox.exe" localhost:8000"

be Careful we must left a space between the path and mlvfs.exe!

Now with a right click on a folder with MLV files we can mount it and start the web gui in the same time.

hope this helps.

Thank you very much! This is a great way to use MLVFS on Windows. But I did have to alter your code a bit. Im not sure whether this is a windows 10 Problem, but Windows didn't want me to have a mlvfs in "C:\Program Files\MLVFS" for some reason, so i had to move my folder directly to "C:\MLVFS". I also changed it to start mlvfs_x64.exe.

This brings me to another question though, which is the newest version of mlvfs for windows and where can I find it?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 11, 2016, 03:12:16 AM
Looks like dokany has released a stable 1.0.0 version (https://github.com/dokan-dev/dokany/releases/tag/v1.0.0). It would probably be worth the time for anyone who can compile on Windows (all you need is Visual Studio) to try it and see if there are any improvements/fixes to some of the current issues/limitations in the Windows version.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: allemyr on October 18, 2016, 05:53:27 PM
Hi,

Since I really like my 5D3 camera I use it a lot and the raw2cdng app has saved me all the time,
and is so easy to use and is a big part of my quite easy workflow with Davinci Resolve and Premiere Pro.

I'am about to upgrade/downgrade to Windows 10 unfortunately. What app could I use that is sort of similar to raw2cdng 1.6.1?

Embedding a short video I made of raw2cdng 1.6.1 that I find really fast and easy to use.

https://vimeo.com/187842751
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: jzedurqn on October 19, 2016, 09:09:13 AM
@allemyr
You're looking at it. MLVFS is cross-platform.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on October 20, 2016, 12:15:34 AM
I'm pretty sure raw2cndg is .NET based. I can't think of any reason it wouldn't already work on Windows 10 as is. MLVFS support on Windows is working but still somewhat immature.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on October 20, 2016, 04:16:46 AM
raw2cdng works fine under w10.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on October 20, 2016, 05:38:44 AM
@ allemyr you know there's a Search tab on the left upper part of the forum web page.
Oh look what this :o Windows 10 + Magic Lantern Raw Video Tools  (http://www.magiclantern.fm/forum/index.php?topic=15654.msg152335#msg152335)   
MLVFS For Wins 7/8/10 - Cdng on the fly convertor for Magic Lantern Raw  (http://www.magiclantern.fm/forum/index.php?topic=13152.msg164559#msg164559)
New PC Tools to replace ACR & A.E. with same or better results (Without Resolve)  (http://www.magiclantern.fm/forum/index.php?topic=16422.msg159671#msg159671)
QuoteRe: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter  (http://www.magiclantern.fm/forum/index.php?topic=13152.msg161902#msg161902)
from dmilligan
The latest MLVFS is now available for Windows!
Thanks to g3gg0 and the dokany project, MLVFS now works on Windows just like the Mac/Linux FUSE version.
Download: MLVFS_x86.zip
You'll need to install Dokany




Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Krumly on October 24, 2016, 09:53:44 PM
Hi all, am I correct in assuming the only way MLVFS works on Windows is to assign it a drive letter? (z:, x:, etc) Is it not possible to mount the virtual file within the same folder on the same drive in which the original MLV file exists?

I've got MLVFS working correctly on my Win 10 machine (MLVFS_x86 and Dokan), but would love to be able to have the virtual file mount exist in the same folder of the original MLV file instead of on an "external" virtual drive.

I can do this via command line with PFM, but I need the vertical stripe fix that MLVFS provides through the browser interface. Alternately, if anyone knows how to implement the vertical stripe fix with PFM, please let me know. Thanks in advance for any advice.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Xenith on October 26, 2016, 03:20:49 PM
I just want to say thank you to the developers for this amazing tool. and to waza57 for the menu entry.
Grading MLV in Resolve 12 without any proxys is amazing :-)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cloudnine on October 27, 2016, 01:45:43 PM
Hi I am still trying with windows 10 64 bit. I receive message missing dokanfuse.exe. i am using dokan and X86 is this correct?
many thanks to all for your time here.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Xenith on October 29, 2016, 05:55:24 PM
Depends on your operating system.

What works nice for me: Dokan 1.0.0.500 (https://github.com/dokan-dev/dokany/releases)
You have to install the 64bit Version of Dokan if you have a 64bit Windows.
To work nicely with Dokan 64bit I use MLVFS_x64 (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/)

Note: On my system, when mountig MLV directry from my Komputerbay CF in a LEXAR USB3.0 Reader I get frame number of 0 for all files.
When mouning from a regular HDD/SSD it works fine. I believe this is a special issue with my device configuration.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cloudnine on October 30, 2016, 11:58:24 AM
Xenith, thankyou so much for your time here and although I haven't tried it yet the links you have supplied look promising. Keen to get it going.
Yes I am Windows 10 64bit, 32 ram SSD etc so should cope well.
thank you again.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: budafilms on October 31, 2016, 05:29:17 AM
Hi,
I installed MLFVS on El Capitan, everything works like the video explained.
But it's too slowly when I check the material in Resolve.

- My mac is one of the newest - ready and working in resolve, Raid disc, discrete Gpu, etc.
- DNG are ten  files of 2-3 gb, are not very big.

I tried to copy the content created by the MLFVS virtual disc, in the hard disc, and it's ridiculous slow.

What am I doing wrong?

Thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 01, 2016, 12:31:03 AM
What processing options did you enable? Some options are very slow.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: budafilms on November 01, 2016, 06:26:54 AM
Bad Pixel ON
Vertical Stripes Fix

Should I try without these options. But I tried for that options.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 02, 2016, 01:20:27 AM
If you disable them, is it faster?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: MitchLally on November 03, 2016, 05:29:58 AM
First of all thank you for this amazing piece of software! It has allowed me to create amazing things. Just wanted to check as I'm having issues with speed too. When I mount a folder of 128GB of .MLV files I get the "(Loading...)" text after the entries in the browser. All boxes left in default (labor intensive settings unchecked). It will usually take up to 30 minutes to load all of the files. I'm on a 2015 15 inch macbook pro (all possible hardware upgrades from apple store). After the files load I import them to resolve I usually get the spinning wheel for around 10-15 minutes before the dialogue box to change the project settings pops up and I can continue processing. I've seen the video tutorials and it seems like no one else experiences this wait time even with large .MLVs. What am I doing wrong? Could my USB 3.0 HDD be the bottleneck? Rendering out of Resolve is generally running at 12-15 fps. Thanks in advance.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 03, 2016, 11:36:30 PM
The initial delay at the beginning is MLVFS indexing the MLV files. As long as you don't delete the .IDX files it creates, this only happens once. This process should only be limited by hard drive speed.

Quote from: MitchLally on November 03, 2016, 05:29:58 AM
Could my USB 3.0 HDD be the bottleneck?
Storage speed is almost always the bottleneck unless you have really intensive processing options turned on. You can use the Activity Monitor (in Applications/Utilities) to get a better idea of what is going on.

The hard drive file system format may have an effect as well. I've noticed some performance issues with exFAT on macOS with SD cards that (dunno if it would also apply to HDD, certainly possible). I would recommend using Apple's builtin file system if you're not already.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ddelreal on November 04, 2016, 04:00:12 PM
How do I uninstall MLVFS from my Mac?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 04, 2016, 06:30:27 PM
delete the file ~/Library/Services/MLVFS.workflow
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ddelreal on November 04, 2016, 07:57:51 PM
Worked, thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: MitchLally on November 05, 2016, 02:56:32 AM
Thanks! I went out and picked up two Samsung T3 external SSD and configured them to raid 0. Probably overkill but that solved my issue for the most part. I guess it is also just the sheer size of the folder I'm trying to mount right now.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lureb74 on November 05, 2016, 09:11:20 PM
Hi Xenith, regarding your post:

Quote from: Xenith on October 29, 2016, 05:55:24 PM
...
What works nice for me: Dokan 1.0.0.500 (https://github.com/dokan-dev/dokany/releases)
You have to install the 64bit Version of Dokan if you have a 64bit Windows.
To work nicely with Dokan 64bit I use MLVFS_x64 (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/)
...

can you please tell how to get it works? After uninstall the dokan080 (first post), I tried to install the x64 version (actually the newer release, version 1.0.1) and tried to run it as the previous method pointing the cmd in the MLVFS_x64 folder. Well, what I see in the cmd window is again the path, no errors and/or any message, simply the virtual path isn't be created. May be there's a step more that I missed?
Thanks, L.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cloudnine on November 06, 2016, 12:17:15 AM
Hi again, and again I thank those for all their work behind the scenes, but like lureb74 I also cant get this to work. The links supplied above take you to a multiple of choices to download - which one do I download and install/extract. Yes i am semi technical but not technical enough to understand the finer detail. I have observed in the MLVFS_x64 folder that appears in my "C" drive there is "dokanfuse1.dll" plus other dokan files but in the MLVFS_x86 folder (tried that also) the dokanfuse files are not there, this appears to be a major discrepancy? HELP!!!

Thanks again to all for your time.
And once I am up and running I will contribute further by fully writting here the method to get this going on Windows 10 professional 64 Bit.
Title: 100D issues
Post by: skrk on November 06, 2016, 01:16:37 AM
MLVFS is amazing, as are most ML-related things -- but I'm having a couple of issues.

I swear I tried to google this and came up empty :-)

I'm using a 100D (Rebel SL1). Latest ML version for it.

- when I use MLVFS the extracted frames are always exactly half the reported resolution -- is this expected due to demosaicing or something? All the tutorials and examples I see do not seem to halve the resolution, so it seems odd. E.g. the resolution set in the camera and shown in MLVFS is 1728x972, but the resulting .dng's are 864x486. Normal?

UPDATE: In MLVProducer, the frames from the same .MLV file are created full-sized. So this seems related to MLVFS somehow?

- I'm on linux; changes in the MLVFS interface are rarely reflected in the files, despite constant reloading of the webpage interface and refreshing in the file browser or exiting/entering via the command line. The normal-vs-davinci filenames change reflects itself immediately (without even a refresh of the page), everything else seems to not take effect no matter what I do. Once I was able to get some of the options to work, but usually I can not. The settings of the form seem to only take effect the first time the .MLV file is entered in the web GUI -- if I restart it, the settings take effect. Perhaps I need to disable some caching or something?

I know I can just set those options on the command line, so it's not a crisis, but it'd be handy for comparisons if the web interface worked.

Just mentioning this in case it might be relevant: I'm on linux -- it compiled fine but I wasn't sure how to set up the execution environment. I ended up running the executable from inside the build directory where it was created, but I had to copy the contents of data/ to the same directory so it could find the html/js files, etc.

Thanks for any ideas!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on November 06, 2016, 06:14:13 AM
Earlier today when testing around with 10-bit, 12-bit files from EOSM, 100D and 5D3 and noticed I couldn't get MLVFS to work and then decided to reinstall and try again to no avail. Then I decided to try with old original MLV files that I've previously used (still has the .IDX files) just to be sure and yet it won't do anything other than giving me 'http://www.localhost.com:8000' can't load message on all my browsers.

So at first I thought it was related to the new stuff but now it's happening with everything else ... seriously what gives?

Assuming everyone else has MLVFS working just fine, right?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on November 06, 2016, 06:41:13 AM
Hey DeafEyeJedi,

     I've had this issue before. What worked was having the files on my internal drive, then mounting MLVFS. I don't have this issue now for some reason, but that was what worked at the time being. Maybe try reinstalling Fuse to a previous version, or a previous version of MLVFS? Maybe it's issues with OS Sierra? I'm on El Capitan.

- Paul Harwood
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on November 06, 2016, 06:58:32 AM
Quote from: PaulHarwood856 on November 06, 2016, 06:41:13 AM
I've had this issue before. What worked was having the files on my internal drive, then mounting MLVFS. I don't have this issue now for some reason, but that was what worked at the time being. Maybe try reinstalling Fuse to a previous version, or a previous version of MLVFS? Maybe it's issues with OS Sierra? I'm on El Capitan.

Actually it was just the FUSE that needed to be updated (guess the recent Sierra update 10.12.1 did something to this) so THANKS for yet another friendly reminder, Paul!  8)

(https://c7.staticflickr.com/6/5752/30506383910_dc52ee1fd3.jpg) (https://flic.kr/p/NtKc7o)

My apologizes @dmilligan for making a mess on your thread. Will continue to test 10/12-bit (http://www.magiclantern.fm/forum/index.php?topic=5601.msg174306#msg17430610/12-bit) and report my findings. Still quite buggy but definitely looks promising!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 06, 2016, 08:52:42 PM
Quote from: skrk on November 06, 2016, 01:16:37 AM
- when I use MLVFS the extracted frames are always exactly half the reported resolution -- is this expected due to demosaicing or something? All the tutorials and examples I see do not seem to halve the resolution, so it seems odd. E.g. the resolution set in the camera and shown in MLVFS is 1728x972, but the resulting .dng's are 864x486. Normal?
How did you determine that the dngs are half resolution? Can you upload a DNG you think is half resolution and/or a very short sample MLV file that exhibits the issue?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on November 07, 2016, 12:29:34 AM
You're welcome DeafEyeJedi, glad to help!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: skrk on November 07, 2016, 09:08:40 AM
QuoteHow did you determine that the dngs are half resolution?

D'oh, sorry -- it's a viewing issue with ufraw and related libraries. If I open a DNG in ufraw and convert to png, it's fine, it's just when opening the .dng with image viewing programs (which use ufraw to view it) that it shows as half the expected resolution. My mistake.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lureb74 on November 07, 2016, 02:14:31 PM
Quote from: lureb74 on November 05, 2016, 09:11:20 PM
Hi Xenith, regarding your post:

can you please tell how to get it works? After uninstall the dokan080 (first post), I tried to install the x64 version (actually the newer release, version 1.0.1) and tried to run it as the previous method pointing the cmd in the MLVFS_x64 folder. Well, what I see in the cmd window is again the path, no errors and/or any message, simply the virtual path isn't be created. May be there's a step more that I missed?
Thanks, L.

Regarding my last post, I forgot to write that I'm trying to run with updated dokany and MLVFS_x64 in order to solve just a Dual Iso processing problem:
When I select Full (20bit) and AMaZE interpolation (leaving all other previous and following parameters as defaults), I can't see any visible improvement (in therms of quality, resolution, details) from Mean32 setting; the only difference merely visible when using AMaZE is that many ugly blemishes appears randomly across all frames:

(https://s12.postimg.org/lyq6irakd/MLVFS_AMa_ZE_blemishes.gif)

NB: since the image is an animated gif, it has some dots due to the file format: only pay attention on differences, specially those black dots/strips!

Don't know if it was already reported in this forum, anyway...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on November 13, 2016, 07:43:46 PM
I've been doing some tests on a new focus pixel map file and thought I'd try MLVFS via the command line but the fpm files aren't working. This is probably an issue with setting the working directory? I also tried it by changing to the ~/Library/Services/MLVFS.workflow/Contents/mlvfs directory and running it from there but I'm still seeing dots.

Here's what I've been doing--on OSX.

mount:
~/Library/Services/MLVFS.workflow/Contents/mlvfs <mount point> --mlv_dir=<directory with MLV files>

The FUSE volume is "OSXFUSE Volume 0 (mlvfs)" on the desktop and doesn't show up on the Devices list like when launching it with Control click and choosing the MLVFS from Services. However, the dng files are there and everything seems to be fine except that the focus pixels are back.

To unmount:
umount <mount point>
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 13, 2016, 07:52:25 PM
What is your working directory when launching MLVFS, and where are the files?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on November 13, 2016, 08:08:30 PM
Files and mount point are on the Desktop. Tried it using the following working directories "pwd":

~/
~/Desktop
~/Library/Services/MLVFS.workflow/Contents/mlvfs
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 13, 2016, 08:26:22 PM
Files need to be in the current working directory mlvfs is launched from. You can always just put them in ~/Library/Services/MLVFS.workflow/Contents/mlvfs and launch MLVFS the normal way.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on November 13, 2016, 08:46:48 PM
That's an unusual place to put the files. Tried it and while it works fine when launching MLVFS in the "normal way" which I assume is by right clicking and choosing the mount point, it doesn't work from the command line. Note that it doesn't seem to matter where I have my files, launching the normal way seems to always work while I have yet to find a way to mount via the command line and not see the focus pixels on the dng files.

By the way, I've got a new fpm file that works with the new crop_rec video mode on the EOSM that @rbrune is working on. It is uses the same full raw buffer size as mv720 and as you pointed out when we were working on focus pixels this could be a problem. By merging the two map files I found that it works with both mv720 and crop_rec modes. Maybe it is not an ideal solution but it works. Sent you a pull request for it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 13, 2016, 10:41:28 PM
Before you launch MLVFS type
ls
Do you see your files? You should, if you don't, you're not in the right working directory.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on November 13, 2016, 11:43:20 PM
That makes sense because the normal way to launch MLVFS is by pointing to the directory with the MLV files in it, a.k.a. mlv_dir (--mlv_dir=<directory with MLV files>) However, that didn't work either. I also tried it moving the folder with the files into the ~/Library/Services/MLVFS.workflow/Contents/mlvfs directory and launch MLVFS from there. The problem with that was that I couldn't find the mount. There was no error message on the terminal and the "mount" folder was gone from the Desktop so I tried:

$ ls ~/Desktop/mount
ls: mount: Device not configured


Re-reading your previous post I might have gotten it wrong by putting the files in a subdirectory so I moved them into the Contents directory and launched from there. Once again, can't find the FUSE mount but this time:

$ ls ~/Desktop/mount/
Applications Volumes opt
Incompatible Software bin private
Library cores sbin
Network dev usr
System home
Users net


Sorry for all the trouble, I thought this would be straight forward. Putting files inside of the MLVFS/Contents is rather impractical if that does turn out to be the solution.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 14, 2016, 12:50:17 AM
Type "ls" and nothing else. Do you see your fpm files listed?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 14, 2016, 01:00:07 AM
You do realize there's a difference between

~/Library/Services/MLVFS.workflow/Contents/mlvfs <mount point> --mlv_dir=<directory with MLV files>


And


cd ~/Library/Services/MLVFS.workflow/Contents/
./mlvfs <mount point> --mlv_dir=<directory with MLV files>


(The latter being the one you need to do)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on November 14, 2016, 01:40:45 AM
Let's see if I'm doing this right. I put the MLV files inside of the MLVFS.workflow/Contents directory then:

RosieFoComputer:Contents rosiefort$ pwd
/Users/rosiefort/Library/Services/MLVFS.workflow/Contents
RosieFoComputer:Contents rosiefort$ ./mlvfs /Users/rosiefort/Desktop/mount --mlv_dir=/Users/rosiefort/Library/Services/MLVFS.workflow/Contents
RosieFoComputer:Contents rosiefort$ ls /Users/rosiefort/Desktop/mount
M13-1148.MLV M13-1149.MLV M13-1150.MLV


Great--volume is mounted and visible on the Desktop but the focus pixels are still showing up on the dng files. This happened every time that I started MLVFS from the command line no matter where the files are located or what working directory I start from.

As you know shooting a few dark frames with an EOSM/650D/700D/100D and bring up the exposure adjustment in ACR (or other raw editing software) will show if the focus pixel removal is working. This is what I'm seeing:

(https://c8.staticflickr.com/2/1680/24685948831_03ef5db8c1.jpg)

And this is with focus pixel removal working on MLVFS:

(https://c8.staticflickr.com/2/1619/24685948231_d9924646dd.jpg)

One of the reasons some users prefer to work with mlv_dump is because it can be run automatically from a script. I'm trying to show that there is a way to do the same only faster and better with MLVFS.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 14, 2016, 03:21:54 AM
This is getting so confusing. Why did you put the MLV files in there? I thought this whole thing was about MLVFS not finding your focus pixel map files. The focus pixel maps need to be in the same directory that you are in when you launch MLVFS. The MLV files can be wherever you like, simply pass their location to the --mlv-dir argument like you did. If you type "ls" and ONLY that, no other arguments, you should see the .fpm files listed.

BTW, when I've been saying "files" in all my previous posts, I'm talking about the fpm files NOT the mlv files. It occurs to me now that you thought I was talking about the MLV files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on November 14, 2016, 06:24:01 AM
Quote from: dmilligan on November 14, 2016, 03:21:54 AM
...It occurs to me now that you thought I was talking about the MLV files.

Yes, that's what I thought you were referring to.

Ok--so I copied all of the fpm files to the same directory as the mlv files, changed into that directory and launched mlvfs from the command line but the focus pixels are still there. Launching by right clicking on the folder using the same mount point and it works fine.

When mounting the "normal" way the mounted volume looks like this and it appears in the Devices list on the left panel of a finder window:

(https://c7.staticflickr.com/6/5498/30856402702_affeda664f_n.jpg)

And when mounting via the command line the mounted volume doesn't appear in the Devices list but it does appear on the Desktop. Here's the folder with the mlv files next to the mounted volume:

(https://c4.staticflickr.com/6/5329/30884624171_b56040a06d_n.jpg)

I can send you short MLV files from and EOSM and 700D that clearly shows the focus pixels if you want--but I thought you mentioned that you have an EOSM?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 14, 2016, 09:11:01 AM
I think your fpm files should be inside mlvfs inside Contents folder. Not together with you MLV files. Launch mlvfs binary from there and most likely path will work from there.
And as dmilligan points out. cd into the contents folder before dragging the binary to terminal window.
By the way. Is mlvfs paths all hardcoded to library folder or could it be placed and used from any other location while using mlvfs via command line?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on November 14, 2016, 02:23:37 PM
Quote from: Danne on November 14, 2016, 09:11:01 AM
I think your fpm files should be inside mlvfs inside Contents folder. Not together with you MLV files. Launch mlvfs binary from there and most likely path will work from there.

Been there, done that. Is focus pixel removal working for you? I keep thinking that I must be doing something wrong.

Quote from: Danne on November 14, 2016, 09:11:01 AM
Is mlvfs paths all hardcoded to library folder or could it be placed and used from any other location while using mlvfs via command line?

I just tried it and it works--with the exception of the focus pixel removal. Put it in a location that's in your path and you can launch it from anywhere. That's great if you want to use mlvfs from the command line and don't need the web browser interface.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 14, 2016, 02:30:49 PM
Didn,t try. At work. Did you erase mlvfs from library folder before testing with mlvfs on the other location?
You also did cd into the Contents folder before running?

*Just tried entering Contents folder and working from command line on a 100D file. Fpm files don,t apply. Either from original library location nor when put somewhere else. Something,s seems broken. Works when launched through services contextual menu.

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on November 14, 2016, 03:44:20 PM
That's what I'm seeing. Works perfectly except focus pixel fixing isn't applied when mlvfs is launched from the command line--no matter where the fpm are located.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on November 14, 2016, 08:20:12 PM
In your home directory, create a folder called MLVFS_TEST
Place the mlvfs executable in this folder.
Place your fpm files in this folder

In your home directory create another folder called MLV_TEST
Place your MLV files in this folder.

In your home directory create another folder called MOUNT_TEST

Type EXACTLY this in the console

cd ~/MLVFS_TEST
./mlvfs ~/MOUNT_TEST --mlv-dir=~/MLV_TEST -f -o local -o volname="TEST" -o allow_other


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on November 14, 2016, 08:52:05 PM
That is working. So -f -o local volname and -o allow other has to be specified in there as well.
QuoteDaniels-MBP:MLVFS_TEST Daniel$ ./mlvfs ~/MOUNT_TEST --mlv-dir=~/MLV_TEST -f -o local -o volname="TEST" -o allow_other
Loading focus pixel map '80000346_1808x1190.fpm'...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on November 14, 2016, 11:20:27 PM
Fantastic--thanks for your patience.

I also see the "Loading focus pixel map '80000331_1808x727.fpm'..." message in the terminal and focus pixel removal is working.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cloudnine on November 16, 2016, 12:46:16 PM
Hi guys, still looking for help to have this running on windows 10 64 bit. Lureb back on page 54 gave two links to download from, but both links have multiple files, can i be advised as to which files to actually use please?
Many thanks to all.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on November 16, 2016, 01:01:16 PM
Read first post of this thread for installation instructions.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: viktor255 on November 20, 2016, 02:38:12 AM
Hello guys, first of all thanks for great work.

Something to contribute. I just spent all day trying to get MLVFS working on Windows 10 64-bit. Old 32-bit version from 1.page worked from the begining, but to start newest MLVFS 64-bit version with newest Dokany wat PITA.

cloudnine - this may be useful for you. I got newest 1.0.1 DokanSetup.exe from https://github.com/dokan-dev/dokany/releases (https://github.com/dokan-dev/dokany/releases) and it was running without problem.
But when I tried start MLVFS http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/ (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/) from CMD(classic comand chain, cd to dir, and mlvfs_x64.exe x:\ --mlv-dir=C:\MLV) nothing happend, MLVFS did not start, exactly like lureb74 described on page 54, no error message nothing, localhost was not working neither. After all day reinstalling everything from Visual C++, dokany, clearing registry I found a solution for me.

I deleted dokan1.dll and dokanfuse1.dll from MLVFS directory and like magic everything works!

I do not understand why this was the problem, maybe somebody can explain further what those files do in MLVFS directory?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lureb74 on November 23, 2016, 05:58:18 PM
Ciao!
For "cloudnine" and "viktor255", it works for me like this: dokan v0.8.0 and MLVFS_x86 (first post), but I put in the MLVFS_x86 folder the optimized x64 exe files (from http://www.magiclantern.fm/modules/modules/mlvfs_optimized.zip/) and then point at them in the cmd:
mlvfs_x64_avx drive-letter:\ --mlv_dir=directory-where-mlv-files-are

In my PC the avx2 version doesn't work but the avx yes!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eNnvi on November 26, 2016, 03:10:47 PM
Hi all guys,
first thanks a lot for your active development, i'm a developer too and i know sometimes how much can be frustrating doing something just for the open source  :-X

anyway i'm using MLVFS on windows and i'm running it as a process using NSSM (https://nssm.cc/)

it seems to be stable so i wanted to share a little guide on how to achive this, so here we are.

How to install MLVFS as service under Windows using NSSM
First i want to explain what this does and how this can speed up your workflow.
Services under windows are software running under a system account so you won't see any black command line window. The power of services is the possibility to launch a process at startup (or login) and kill that application when you turn off your PC.
Now having MLVFS running as service can let you always edit your footage forgetting al those command lines and you will never forget again about opening MLVFS before starting your favourite editing software  ;)

let's go step by step:
1. Install MLVFS as described in the first post and TRY IT! be sure that everything is running fine
2. Now that you know that everything is running fine proceed downloading NSSM from https://nssm.cc/
      - direct link for download -> https://nssm.cc/release/nssm-2.24.zip (but check if there are new versions)
3. Open the zip folder, enter the nssm folder, double click on the windows version you have installed (win32 or win 64) and copy the nssm.exe file
4. Now we have to place this executable somewhere, i suggest you to copy it in your main drive (usually C:) in the program files folder (C:\Program Files\)
5. Now NSSM is "installed"

ok we have installed NSSM, so what's next?
NSSM is an interface to allow any executable to become a service in windows.

let's go on
6. In the program files folder (or in the folder you copied nssm.exe) keep pressed shift+right click in a empty space of the folder, this will open the usual dropdown menu, but you can click "open command window here" this will open the old-fashioned command line tool of windows in the folder you have nssm.exe
7. Type this in your command line:
nssm install MLVFS
8. Administrator window should appear, click yes
9. A window like the follow should appear
(https://nssm.cc/images/install_application.png)

Now let's see the fields you need:
Path: Is the path to your MLVFS.exe file. For example having a second hard drive i installed it in the "D:\Program Files (x86)\MLVFS_x86\" folder.
So my path will be "D:\Program Files (x86)\MLVFS_x86\mlvfs.exe"
Startup directory: Is the directory you want the service to be started from, i placed "D:\Program Files (x86)\MLVFS_x86\" but it can be left blank if you want
Arguments: here is the place where you want to give all the command lines arguments to the mlvfs service. The basic is this:
Z:\ --mlv-dir="C:\dir\to\your\mlv\footage"
if you (like me) own a 700D probably you will want to add the 2x2 chroma smoothing, so argument field will ahve to be filles with:
Z:\ --mlv-dir="C:\dir\to\your\mlv\footage" --cs2x2
and if you also want the naming for davinci
Z:\ --mlv-dir="C:\dir\to\your\mlv\footage" --cs2x2 --resolve-naming
so any command line argument that can be found here: https://bitbucket.org/dmilligan/mlvfs
(of course you can edit this also from the web gui, but every time you restart your pc the settings made via web interface are deleted, adding these lines in the arguments field will make them "permanent").

10. (almost done) now get in the Details tab
(https://nssm.cc/images/install_details.png)
in the Display Name put "MLVFS"
in the description put everyting you want
in the startup type select Automatic

11. click "install service"

yeeeee we're finally done  ;D

now we have to start the service  :P
this is quite simple.


now you're back to the command window, just type
nssm start MLVFS
and it should start  ;D

if this doesn't happen (returns error) do te following:
1. Open control panel->System and Security->Administrative tools
2. New windows will pop up, open Services
3. Find in your list MLFVS, then right click->start

ok in your system now i'll see the fake "Z:\" disk containing all your footage ready to edit  ;)


Workflow
The wokflow stays the same as before, but the folder that will be scaned from mlvfs is one... so my advice is this:
create a new sub-folder for every project in the "scanned" folder you choosed above (the "C:\dir\to\your\mlv\footage" path)
as MLVFS supports the folder organization it can help keeping your footage in a good order  ;) then if you use davinci like me, you can set the main media folder in Z:\



Hope this can be helpful to someone  ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: g3gg0 on November 26, 2016, 05:58:36 PM
> An error occurred during a connection to nssm.cc. Peer's Certificate has been revoked. Error code: SEC_ERROR_REVOKED_CERTIFICATE

scam? but looks legit.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eNnvi on November 26, 2016, 08:53:16 PM
Quote from: g3gg0 on November 26, 2016, 05:58:36 PM
> An error occurred during a connection to nssm.cc. Peer's Certificate has been revoked. Error code: SEC_ERROR_REVOKED_CERTIFICATE

scam? but looks legit.
mmmh not having that certificate problem with both firefox and chrome for me...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: jmanord on December 01, 2016, 01:23:49 AM
I just wanted to say how awesome MLVFS is! I decided to give CompressMLV another try today, and was elated that it worked without incident. For whatever reason, I was unable to get it to work when I initially tried it over a year ago. I have been trying to keep the original MLVs as backups, and cinemadngs compressed with slimRaw as my working media. Unfortunately, this has become too tedious and time consuming. With CompressMLV, I can use the MLV files for both my working media and archive copy while using half the space. Thank you dmilligan for your brilliant work!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: budafilms on December 22, 2016, 09:10:42 AM
@dmilligan

I installed  as the description in the first post.
But I can't get see my files in the empty folder.

I'm using Sierra 10.12.2, and I think could be a problem with this.

I double check last version of FUSE in the site.

I am missing something? Or I should wait for an upgrade?

Thank you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on December 30, 2016, 01:33:00 AM
Hello everyone,

     It's really great 10 bit raw recording on the T3i is able to be used with MLVFS. I do have one problem though, with all raw video recording with the T3i and MLVFS. For some reason, I get an Embedded Profile in Adobe Camera Raw. I cannot use Cinelog-C unless I use a Terminal Command with MLV_Dump. I read on the ACR VisionLog Camera Profile Post on this Forum that this was also an issue for the T4i. Please note, this does not occur on my 7D. Is there any way to get the T3i to not have an Embedded Profile? It would be great, and save a bit more time. Here is a link to the forum post I was referring to: https://www.magiclantern.fm/forum/index.php?topic=16738.50 (https://www.magiclantern.fm/forum/index.php?topic=16738.50) Thanks!

- Paul Harwood
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 30, 2016, 09:01:02 PM
As explained in the thread you linked, there's not an elegant solution on the MLVFS side. The fix should really be on the ML side.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 30, 2016, 09:23:08 PM
I,m curious. If on 'ML side' then both localized and the generic camera model tag must exist when this occurs. Do they? If not, should this name tag switch be done in mlv_rec rather than in the conversion tool itself you mean?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on December 30, 2016, 09:37:24 PM
Isn't the camera model identified from Canon ID? (e.g. 0x80000287 = 60D)

From the linked thread, I guess the model string is only used as fallback (here's why) (https://www.magiclantern.fm/forum/index.php?topic=16738.msg163496#msg163496); if the main method of identifying the camera is from model ID, the proper fix would be in the DNG converters (because MLV already includes the camera model ID).

Can you confirm this hypothesis?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 30, 2016, 10:13:46 PM
The canon ID tag can be multiple names for some cameras depending from where it is bought. For instance 600D.
(from exiftool Canon.pm file)
0x80000286 => 'EOS Rebel T3i / 600D / Kiss X5',
Only valid tag in acr is the Canon EOS 600D tag and should be set to the 'UniqueCameraModel' and the other camera model tag in dng spec.
It seems Magic lantern registers that localized tag probably cause there is no more tags to get? Adobe dng converter rewrites to the generic tag during conversion.

*I was wrong about adobe dng converter rewriting the tag. It actually add the localized tags present in the metadata when converting.

bouncyball put in a void part in ml-dng mlv_dump to fix this issue. Example here.
void set_unique_camera_name(mlv_idnt_hdr_t *idnt_hdr)
{
    switch(idnt_hdr->cameraModel)
    {
        case 0x80000286:
            memcpy(idnt_hdr->cameraName, "Canon EOS 600D", 32);
            break;
    }
}
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eNnvi on December 31, 2016, 12:38:47 AM
Why not Just hardcore using compiler directives for every ml camera?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 31, 2016, 05:35:45 AM
Quote from: eNnvi on December 31, 2016, 12:38:47 AM
Why not Just hardcore using compiler directives for every ml camera?
Why not just find and use the Canon property that reports the non-localized camera name rather than using the one we currently use that reports the localized camera name?

Quote from: a1ex on December 30, 2016, 09:37:24 PM
Isn't the camera model identified from Canon ID? (e.g. 0x80000287 = 60D)

From the linked thread, I guess the model string is only used as fallback (here's why) (https://www.magiclantern.fm/forum/index.php?topic=16738.msg163496#msg163496); if the main method of identifying the camera is from model ID, the proper fix would be in the DNG converters (because MLV already includes the camera model ID).

Can you confirm this hypothesis?
There is nothing for a model ID tag in the DNG spec. There's only UniqueCameraModel and LocalizedCameraModel in the spec.

I don't like the idea of having to create and maintain a lookup table of camera names in MLFVS.

Just thinking about the MLV spec itself, wouldn't it make more sense that "camera name" metadata that the spec defines would be a non-localized name, and that every camera that is the same camera would report the same name?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on December 31, 2016, 10:40:00 AM
Quote from: dmilligan on December 31, 2016, 05:35:45 AM
Why not just find and use the Canon property that reports the non-localized camera name rather than using the one we currently use that reports the localized camera name?

I doubt there is any.


grep --binary-files=text -E "(REBEL|600D)" ROM*.BIN
ROM0.BIN:Canon EOS REBEL T3i
ROM1.BIN:Canon EOS REBEL T3i

grep --binary-files=text -E "(REBEL|1100D)" ROM*.BIN
ROM1.BIN:Canon EOS REBEL T3


How does a CR2 from a localized camera look like?

In this case, overriding the model string property (maybe using a definition in consts.h, or in platform makefiles) should work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 31, 2016, 10:49:53 AM
I can,t find any extra id strings when checking exiftool docs. Here is a 600D CR2 file coming from here(http://www.photographyblog.com/reviews/canon_eos_600d_review/sample_images/) with local tags(Canon Model ID) shown as well as the generic one(check bottom) and with the generic(Camera Model Name) tag.

600D CR2
https://bitbucket.org/Dannephoto/magic-lantern/downloads/canon_eos_600d_01.cr2
100D dng file from mlvfs
https://bitbucket.org/Dannephoto/magic-lantern/downloads/100D_mv1080MLV_1_2016-01-27_0001_C0000_000000.dng

DNG spec docs are a bit vague(http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf). They state under localized tags:
Default  Same as UniqueCameraModel
Which indicates all is coming from one tag?

Checking the 600D with exiftool one finds all tags represented under Canon Model ID tag
Canon Model ID                  : EOS Rebel T3i / 600D / Kiss X5

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on December 31, 2016, 11:10:41 AM
This CR2 contains only the European model string. Are you sure the camera they reviewed reports a localized name in ML?


exiftool -htmlDump canon_eos_600d_01.cr2 > rawexif.html


= "Canon EOS 600D", no traces of REBEL or Kiss. Same with grep.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 31, 2016, 11:16:09 AM
I have no idea. I have a 600D at work. I might be able to check that. Or even better. @Paulharwood856 maye could upload a CR2 from his 600D?

*google and found this webpage. A collection of raw samples. Might be useful.
http://www.rawsamples.ch/index.php/en/canon
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on December 31, 2016, 02:25:43 PM
The 750D sample has "Canon EOS Rebel T6i" in EXIF, but no traces of 750D inside.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 31, 2016, 02:58:46 PM
Yes, and yet the adobe tag comes out like this when baked in adobe dng converter. They must run some template.
Unique Camera Model             : Canon EOS 750D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on December 31, 2016, 03:05:56 PM
Right.

Can you create a list of UniqueCameraModel names, with the exact spelling used by Adobe DNG converter?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 31, 2016, 03:11:56 PM
They are quite consistent. Here is the code snippet from bouncyball and the correct names

void set_unique_camera_name(mlv_idnt_hdr_t *idnt_hdr)
{
    switch(idnt_hdr->cameraModel)
    {
        case 0x80000285:
            memcpy(idnt_hdr->cameraName, "Canon EOS 5D Mark III", 32);
            break;
        case 0x80000218:
            memcpy(idnt_hdr->cameraName, "Canon EOS 5D Mark II", 32);
            break;
        case 0x80000302:
            memcpy(idnt_hdr->cameraName, "Canon EOS 6D", 32);
            break;
        case 0x80000250:
            memcpy(idnt_hdr->cameraName, "Canon EOS 7D", 32);
            break;
        case 0x80000325:
            memcpy(idnt_hdr->cameraName, "Canon EOS 70D", 32);
            break;
        case 0x80000287:
            memcpy(idnt_hdr->cameraName, "Canon EOS 60D", 32);
            break;
        case 0x80000261:
            memcpy(idnt_hdr->cameraName, "Canon EOS 50D", 32);
            break;
        case 0x80000326:
            memcpy(idnt_hdr->cameraName, "Canon EOS 700D", 32);
            break;
        case 0x80000301:
            memcpy(idnt_hdr->cameraName, "Canon EOS 650D", 32);
            break;
        case 0x80000286:
            memcpy(idnt_hdr->cameraName, "Canon EOS 600D", 32);
            break;
        case 0x80000270:
            memcpy(idnt_hdr->cameraName, "Canon EOS 550D", 32);
            break;
        case 0x80000252:
            memcpy(idnt_hdr->cameraName, "Canon EOS 500D", 32);
            break;
        case 0x80000288:
            memcpy(idnt_hdr->cameraName, "Canon EOS 1100D", 32);
            break;
        case 0x80000331:
            memcpy(idnt_hdr->cameraName, "Canon EOS M", 32);
            break;
        case 0x80000346:
            memcpy(idnt_hdr->cameraName, "Canon EOS 100D", 32);
            break;
    }
}


list
Canon EOS 5D Mark III
Canon EOS 5D Mark II
Canon EOS 6D
Canon EOS 7D
Canon EOS 70D
Canon EOS 60D
Canon EOS 50D
Canon EOS 700D
Canon EOS 650D
Canon EOS 600D
Canon EOS 550D
Canon EOS 500D
Canon EOS 1100D
Canon EOS M
Canon EOS 100D

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on December 31, 2016, 07:20:28 PM
Cool trick. Here's one from a localized 700D. There isn't a trace of "700D" in the CR2 file.

exiftool -htmlDump IMG_1695.CR2 > rawexif.html

(https://c6.staticflickr.com/1/408/31890104061_9dc66eb766_c.jpg)

After running Adobe DNG Converter:

(https://c1.staticflickr.com/1/664/31969786096_9682a05a05_c.jpg)[/url]

I had a similar issue on the Assign lens focal length and name for non cpu lenses (http://www.magiclantern.fm/forum/index.php?topic=18083.msg176995#msg176995) (currently broken until a commit message is fixed :-X) when I was discussing how lenses are automatically identified in ACR (https://forums.adobe.com/message/9202969#9202969) with an Adobe staff member. Seems that they use different EXIF tags for Nikon and Canon lenses so it isn't surprising that they use different tags for CR2 and DNG files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on December 31, 2016, 08:05:59 PM
Does everything still work if we use LocalizedCameraModel instead of UniqueCameraModel. If so I can make the switch.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 31, 2016, 08:31:00 PM
How do you mean? Localized tags turned into uniquecameramodel tags won,t work.
The uniquecameramodel tag has to be a global/generic tag according to the list I wrote before.
It,s actually only a few cameras that have those localized tags. Check exiftool docs for which. I can do it later, not home...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on December 31, 2016, 09:17:13 PM
Quote from: dmilligan on December 31, 2016, 08:05:59 PM
Does everything still work if we use LocalizedCameraModel instead of UniqueCameraModel. If so I can make the switch.

I think the issue is that MLVFS puts the LocalizedCameraModel in the UniqueCameraModel EXIF tag:

exiftool -htmlDump M31-1256_000000.dng > rawexif.html

(https://c5.staticflickr.com/1/646/31971755916_3207ee202d_c.jpg)

That should be "Canon EOS 700D". Note that Adobe DNG Converter writes both the LocalizedCameraModel and UniqueCameraMode tags and they are different on this camera.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 31, 2016, 09:31:08 PM
Canon id tag is registered as a localized tag in magic lantern mlv files. Probably the only tag in there. The only tag needed is the unique one(used in ACR). That one can replace any of the other tags. Or not, doesn,t matter.
Isn,t a1ex looking into this anyway?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 01, 2017, 02:13:02 AM
Quote from: dfort on December 31, 2016, 09:17:13 PM
I think the issue is that MLVFS puts the LocalizedCameraModel in the UniqueCameraModel EXIF tag:
Righti know that, but it's the only thing to put there, which is why I asked the question. So I'll ask again, if you craft a DNG with LocalizedCameraModel and no UniqueCameraModel does it work correctly?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 01, 2017, 02:39:56 AM
If you are asking to exclude the unique camera model tag from the DNG spec entirely that will not work since it,s the tag used in ACR. Well, I havn,t tested if ACR will rely on some other name tag if unique tag is excluded. Just erase it and see what happens.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 01, 2017, 03:03:42 AM
I just erased the unique camera model tag in mlv_dump and ACR then reports the dng file as a DIGITAL NEGATIVE
(https://s23.postimg.org/uwxoqyxu3/Screen_Shot_2017_01_01_at_03_01_13.png)

The file still has the camera model name tag with the Camera Model Name               : Canon EOS REBEL SL1 tag but ACR doesn,t read that at all.
I didn,t put in the localized tag but I,m pretty confident it won,t work. I can see if I can put it in with exiftool.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 01, 2017, 03:10:40 AM
Oops, I was wrong. The localized tag actually is read by ACR if the unique camera model tag is removed. Here is proof.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/100D_mv1080_frame_000003.dng
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 01, 2017, 03:43:02 PM
*Clearance
Even if the localized tag is shown in ACR this does not mean we can use the local tag itself with dcp profiles. We still are dependent on the camera-specific constants(list of camera model names). Even if it,s possible to technically switch out the unique tag with the localized tag it,s not the solution without correct camera model names.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andy600 on January 01, 2017, 04:03:04 PM
ACR will usually read a DNG regardless of which model tags are used but will default to an 'embedded' calibration if no tags are included or recognized. This means only the embedded meta is used. The embedded meta might or might not be correct depending on how the calibration was obtained and the type of calibration that was performed. Think of this as tier 3 calibration - i.e. there are typically one or 2 color matrices allowing ACR to find a good auto white balance if needed. Some ML apps also embed forward matrices which offer some degree of color tweaking to the white balanced image.

If a model tag other than UniqueCameraModel is recognized by ACR (i.e. it is from a camera that has a calibration hardcoded into ACR) then ACR will use its internally coded calibration for that specific camera model and embedded color matrices will be overridden. ACR will show something like ACR V9.x for recent cameras and ACR v4.x for older cameras depending on when the last update was for that model. Think of this as tier 2 - it is calibrated (by Adobe) but doesn't allow selection of camera specific color profiles. Forward matrices are used only if embedded in the DNG.

If the UniqueCameraModel tag is filled then ACR will let you select alternate profiles (Adobe Standard, Camera Standard, Camera Neutral etc) where available. Think of this as tier 1 with the most options and typically the best, most up-to-date calibration. Embedded calibration tags are overridden by the profile.

The UniqueCameraModel tag is used by most, if not all, cam manufacturers that have cams that capture to the DNG/CinemaDNG format (BMD,Digital Bolex, Convergent Design, Zenmuse etc). It is the most commonly used identifier and may or may not be important for accessing camera-specific calibration and certain features in some NLE and color apps.

Regional tags are best translated to uniqueCameraModel tags if possible/required. In terms of calibration and color, there are no internal differences between the US, Japanese and Euro versions of a camera. The main difference is usually the front badge.

The only time the UniqueCameraModel tag should be purposely omitted (or better still, changed to something unique for the purpose of identifying a specific camera) is when a specific sensor has been purposefully lab calibrated and where it's images will usually only be developed in ACR i.e. to ensure this very specific calibration is not over ridden in any way by ACR - this is usually only done for industrial or research reasons.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 01, 2017, 04:10:21 PM
Thank you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Andy600 on January 01, 2017, 04:18:26 PM
You can find references to hard-coded calibrations by searching ACR plugin in notepad or other basic reader. Obviously, being code, most of what you see is random characters but the camera models that are supported show up in English. Try looking for a localized tag to see if it's supported.


EDIT:

Interestingly, if you reconvert your de-tagged DNG (or any detagged DNG) with DNG converter it adds a UniqueCameraModel tag (UCM) called 'Digital Negative'. This means the local tag is not sufficient enough to identify the camera to the converter or ACR (as I thought - else it would have added the usual UCM tag) but it likely needs the tag filled for validation (need to recheck DNG specs).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 01, 2017, 05:23:51 PM
Cool findings Andy600(Digital Negative).
I downloaded my own file and renamed the localized tag to Canon EOS 5D mark III to see if it would work to use dcp profiles but only embedded option available. This verifies the need for the uniquecameramodel tag and it has to be set set to adobe dng standard naming.
(https://s29.postimg.org/hfu0r9hcn/Screen_Shot_2017_01_01_at_17_20_57.png)

In following example I added back the UniqueCameraModel tag and named it Canon EOS 5d Mark III and now all dcp profiles and alternate profiles works for a 5D mark III(even if the file has matrices for the eos 100D).
Linear dcp profile added
(https://s28.postimg.org/jiz7lnbh9/Screen_Shot_2017_01_01_at_17_26_22.png)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: PaulHarwood856 on January 01, 2017, 05:54:19 PM
Hello everyone,

     I apologize for the delay, here is a CR2 from my T3i on the latest Nightly Build, December 19th, 2016.

     https://drive.google.com/open?id=0Bz5n0qYxIkVOTno1M0NHZ0pSeXM (https://drive.google.com/open?id=0Bz5n0qYxIkVOTno1M0NHZ0pSeXM)

- Paul Harwood
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 01, 2017, 08:48:03 PM
I recently discovered you can actually alter the metadata tags inside the virtual dng files with http://www.exiv2.org/
I use this tool to rename camera model name via cr2hdr.app and also for auto white balancing. Now what I don,t understand is where the information is being stored when doing this? No MLD folders are created, it just writes, seemingly, straight to the virtual dng file? Some cached memory solution?
I run this command for instance.
find . -iname '*.dng' -print0 | xargs -0 -P 8 -n 1 exiv2 -M"set Exif.Image.UniqueCameraModel Ascii Canon EOS 100D"
Check the movie file. 
https://www.youtube.com/watch?v=-MRuFpj25Y4&feature=youtu.be
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 01, 2017, 11:10:34 PM
That shouldn't be possible and is some crazy black magic. Debugging on the MLVFS side, write() is never called, and when data is requested from MLVFS via read(), MLVFS returns the original file data, yet the requesting process gets the overwritten data. Trying to do the same thing with a hex editor, results in an error when attempting to save changes as expected.




It looks like there's not going to be an easy and satisfactory solution to this issue. Note that this issue also means the cam matrices MLVFS embeds are also broken for "Rebels". I don't really like these being hardcoded either. So I think I'm just going to get rid of all of it, and make a way to have metadata tags defined in data files referenced by camera id. This will probably be a far amount of work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 01, 2017, 11:36:01 PM
Quoteblack magic
My exact words to bouncyball when he tried doing the same on a windows machine and it didn,t work on his end.
I use it for altering default scale tag, AsShotNeutral and UniqueCameraModelTag so it really works for any tag seemingly.
http://www.magiclantern.fm/forum/index.php?topic=15108.msg177259#msg177259
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on January 02, 2017, 08:19:36 AM
Hi there

Really strange. In win, any time when altering dngs with exiv2, "altered" dng files are copied to MLD folder, but the content of files is identical to the original virtual file. Hexedit or any write operation surely ends with error (which is correct). "Definitely" apple did some sort of (black, gray, orange, whatever) magic on read/write operations :P.

bb

P.S. I should try this on linux.
Title: Help tracing a problem with sloooow MLVFS performance...
Post by: andrew_dotdot on January 14, 2017, 09:15:10 AM
Hi,

Up until this summer, I used MLVFS on my mac *all the time* to work with MLV files from my 5D3 in Davinci Resolve, and it worked great. I could play back MLV in real time. This fall I did a project all in camera codec, so a break from raw, but I'm now working with .MLV files again -- and MLVFS is sooo slow!

I've tested my SSDs and un/re-installed MLVFS and updated Resolve and done all of the normal things I can think of to fix this. If I convert the MLV files on my PC, I can play those folders of DNG files back in Resolve in the mac at 40-something FPS, but the same source MLV under MLVFS is, maybe 7fps, and eventually bogs down and stops playing back.

Where can I look (some log file?) to help me get some clues to get my poor MLVFS back on track? I'm willing to work for this, but I don't even know where to start!

Thanks,

//Angus
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 14, 2017, 08:12:47 PM
Start by disabling any extra processing options you may have turned on. If it's still slow, check activity monitor and see what CPU usage looks like. Finally, you can download xcode and run MLVFS in the profiler (this is fairly easy and requires no coding skill).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andrew_dotdot on January 14, 2017, 11:31:05 PM
Thanks for the tips!

All the processing options are off. No stripes or dots or smoothing or anything like that. The processor shows that the mlvfs process stays around 100% while it works, which is just one of the 4 cores going like mad, I guess. Then Resolve's got either 26% or 126% at various times when I'm trying to render to Prores 422 HQ.

Sooo, I guess it's Xcode and profiler...which I've not used, but how hard can it be? We'll soon see..

//A
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ethan on January 15, 2017, 03:36:43 AM
Has this program been compiled into a single downloadable file that I can download and run like any other program?

I have no knowledge of coding or running commands and I am completely lost on how to use this program. I even followed the video tutorial but no luck.

Thanks,
-Ethan
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andrew_dotdot on January 15, 2017, 11:55:46 AM
@Ethan...

Once you have it installed (on Mac anyway), the idea is that you right click the folder where you have your MLV files and set that up as a mounted "drive". You go down the right-click menu to where it says Services, then select MLVFS. It's not an "application" situation, but rather more transparent -- it just starts to show you a new drive in the Finder window's list.

The thing that I always found tricky is how the next dialog that comes up (where you select the empty directory that will act as the "drive") can be hidden under other Finder windows. But MLVFS doesn't work unless you make that selection. So once you've started using MLVFS, you know you MUST do this; but the first time, it can be tricky-tricky.
Title: How do I use Xcode's Profiler to monitor MLVFS?
Post by: andrew_dotdot on January 15, 2017, 12:03:19 PM
If I have Xcode installed, how do I then use it to monitor what MLVFS is doing, as was suggested earlier in the thread? To recap, MLVFS works as expected except that it is stoooopid slow.

Do I need to open MLVFS in Xcode as a project somehow? I was thinking that it might come up in the "Debug...Attach to process...", but there, it doesn't seem to be referring to processes running at large, but rather to processes inside the project that's loaded into Xcode.

I found the MLVFS.workfow, which just refers to "./mlvfs", but I'm not sure where that actually refers to. I guess it's in an environment variable somewhere, but that's at the outer reaches of my knowledge.

I'm not sure what to point Xcode at, and how.

Thx! 

:A)>
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 16, 2017, 03:33:27 PM
You do have to download the MLVFS source code and then open the xcode project in xcode.


git clone https://bitbucket.org/dmilligan/mlvfs.git


or https://bitbucket.org/dmilligan/mlvfs/get/9f8191808407.zip

Then open the mlvfs.xcodeproj in xcode.
Go to Product > Scheme > Edit Scheme to modify the command line parameters for debugging to suit your environment (mainly you just need to change the first argument to some empty mount directory, and the "--mlv_dir=..." argument to somewhere with MLV files). Now you can just hit "Build and Run" button at the top left of the main window. Once it's running you can view some basic information about CPU, memory, etc. or on the CPU info screen and click "Profile in Instruments" for complete and detailed information. Once the "instruments" is recording, do some stuff with MLVFS (play something back in Resolve, etc.). Save the trace and send it to me.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kaco on January 18, 2017, 12:05:32 PM
Hi,

I have a problem with windows port.
When called on a lot of files (1000+), after calling localhost it will process/index only approx first 500. first 345. The rest will start indexing after I open the mounted folder, which crashes the Resolve while waiting for the folder to list.

I think this must be some kind of hardcoded file limit? Is there a chance to higher this number? I get hunders of files from every shooting day. :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kaco on January 18, 2017, 12:30:06 PM
The file limit displayed (which means also indexed) by localhost is 345.

In the source code I have found that you have disabled indexing after files are mounted. What was the reason for this? For me, it would be much better to mount, wait until everything is indexed and then start working with it.

Now it crashes even Windows Explorer, if I try to open big not indexed files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: daniel.t on January 18, 2017, 02:44:06 PM
Hello, guys. I'm just starting my MLVFS experience but I met a huge performance drop compared to normally converted dng files.

I'm using MacBock Pro 2016 and it gives me realtime playback with dng files. But when i try MLVFS playback fps drops down to 2-3, sometimes it just "freeze". Also it takes few minute just to import files into media page. MLV files located on external HDD (exfat file system) don't know is that matters. Have someone same issues?

Just tested files from the internal SSD, smooth as butter. But normal dng files plays from external HDD pretty easily in realtime. Tried to choose directory on SSD and on external HDD, nothing helps.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 18, 2017, 08:18:33 PM
It's probably the fact that it's formatted exFAT. exFAT support in macOS is not very good, and I've seen the issues I think you are probably experiencing (I have SD cards formatted as exFAT). Unfortunately, I couldn't find a viable solution. IIRC, fclose() takes a really long time and eats CPU for seemingly no reason (there's nothing intensive or complicated fclose should be doing since the files were not written to, only read from). I could make it just not close the files, but then you eventually run into the max number of allowed open file handles.

You can try using a different format on the HDD and see if that fixes your issue.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 18, 2017, 08:22:29 PM
Quote from: kaco on January 18, 2017, 12:30:06 PM
In the source code I have found that you have disabled indexing after files are mounted. What was the reason for this?
I have no clue what you are talking about. Could you reference what source file and line number you are referring to?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kaco on January 19, 2017, 12:06:21 PM
Quote from: dmilligan on January 18, 2017, 08:22:29 PM
I have no clue what you are talking about. Could you reference what source file and line number you are referring to?

Comment for this file: https://bitbucket.org/dmilligan/mlvfs/src/9f8191808407bb49112b9ab14c27053ae5022749/mlvfs/index.h?at=master&fileviewer=file-view-default
Windows: IDX files no longer created upon mounting

The problems with 345 files limit is not on macOS, I am working under WIN10 64bit, disk formatted in NTFS.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 19, 2017, 12:46:08 PM
That is a very old and irrelevant comment that only has to do with the old pismo version which doesn't exist anymore.

Are you saying that the webgui will only show you the first 345 files if you have more than that in a single folder? Have you tried a different set of files? Is the limit still 345? Could you be more precise about what exactly is happening and maybe include screen shots? Have you looked at the console output? Have you tried separating the files into sub folders so there's not so many in a single folder?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kaco on January 19, 2017, 02:49:04 PM
Quote from: dmilligan on January 19, 2017, 12:46:08 PM
That is a very old and irrelevant comment that only has to do with the old pismo version which doesn't exist anymore.

Are you saying that the webgui will only show you the first 345 files if you have more than that in a single folder? Have you tried a different set of files? Is the limit still 345? Could you be more precise about what exactly is happening and maybe include screen shots? Have you looked at the console output? Have you tried separating the files into sub folders so there's not so many in a single folder?

Windows 10, 64bit, RAID drive, exfat, folder with more then 345 files (1000+):
1. mount the folder with mlvfs, console shows nothing
(if I will open mounted drive in Resolve or under Windows Explorer, it will hang and then crash, because files are not indexed yet. MLVFS will however start to index those being opened by Resolve/Explorer)
2. when called locahost:8000, it will start to index. However, it will index only first 345, it will also print only the first 345 on the localhost page. Console displays just indexing info, nothing unsual. All files are valid. Same behaviour for different folders.
All files are in the root folder, no subfolders.

So for now, I have to run mlvfs, call localhost, let index first 345, move indexed files to another folder and then refresh localhost to let another 345 files to index.

Let me know, what could I do to help. Record youtube video?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 19, 2017, 03:18:54 PM
There is a maximum size for the HTML generated by the MLVFS webgui. You can make it bigger if you like.  More work, but also possible would be to dynamically allocate that buffer, eliminating the limit.

If you don't put so many files in a single folder you shouldn't have this issue, and you also shouldn't have to force MLVFS to generate index files using the webgui like you are doing. When you browse to a folder in the virtual file system, all the MLV files in the corresponding real directory have to be indexed before that operation can complete. Did windows explorer actually crash or was it just taking a long time and you gave up and killed it? It may take quite a while to index 1000+ MLV files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kaco on January 19, 2017, 03:44:47 PM
Quote from: dmilligan on January 19, 2017, 03:18:54 PM
There is a maximum size for the HTML generated by the MLVFS webgui. You can make it bigger if you like.  More work, but also possible would be to dynamically allocate that buffer, eliminating the limit.

If you don't put so many files in a single folder you shouldn't have this issue, and you also shouldn't have to force MLVFS to generate index files using the webgui like you are doing. When you browse to a folder in the virtual file system, all the MLV files in the corresponding real directory have to be indexed before that operation can complete. Did windows explorer actually crash or was it just taking a long time and you gave up and killed it? It may take quite a while to index 1000+ MLV files.

With Resolve, if the file is big (sometimes single file can be 100Gb+), it takes longer then Resolve can wait and it imports not all of the cdng. With no warning. So that is why I generate index files via webgui.

Also with Windows Explorer, if I copy not indexed files from mounted drive (sometimes I need to copy cdng for postproduction studio), for some very big files it will copy empty folder (probably the windows explorer copy function has some timeout?).

Would you be so kind and help me where to change the size of html?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: kaco on January 19, 2017, 03:46:59 PM
Or would it be to complicated to change the mlvfs so that it will automatically start indexing all of the files after mounting?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 19, 2017, 04:33:10 PM
in webgui.c change the line:

#define HTML_SIZE 65535


Automatically starting to index files when mounting is probably a bad idea. It would significantly hurt performance while indexing is happening, requiring one to basically wait for MLVFS to search and index the entire directory structure before being able playback any files. If anything I would like to make indexing even more "lazy loaded". That might actually fix the issue you are seeing, and if you take my suggestion to use sub folders, I think you will also see improvement.

If you really want to, you should be able to use mlv_dump -x to force generate index files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Teamsleepkid on January 19, 2017, 11:24:52 PM
I've got Focus Pixels with the new eos m 203 experimental build. And 2x2 produces large blue red and white artifacts. I'm using 10 bit and 12 bit. Not sure if it's the build or if it's mlvfs. It used to work in 10 bit and 12 bit. Been hearing a lot of talk about people using mldump. Does mlvfs use this? Definitely not complaining so please don't take it that way:)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 20, 2017, 12:57:17 AM
MLVFS does not use mlv_dump. Theoretically, chroma smoothing shouldn't be necessary, MLVFS uses some bad pixel maps to remove the focus pixels. If you think there's an issue, posting some examples would be a good idea.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on January 20, 2017, 01:10:06 AM
Most likely this issue (http://www.magiclantern.fm/forum/index.php?topic=16608.msg178443#msg178443).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on January 20, 2017, 03:56:20 AM
@Teamsleepkid -- If you are shooting with the crop_rec module it has a different focus pixel pattern than the regular mv720 mode so you need a special focus pixel map file. I created an experimental map file that covers both the mv_720 and crop_rec modes for the EOSM. You need to install this inside the MLVFS application. It replaces the 80000331_1808x727.fpm file.

EOSM_crop_rec_mv720_combo_map_file (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/f7eff7ac533e7d1bcf5c6ec485b2d240919e1564/focus_pixel_map_files/fpm_archived/EOSM_crop_rec_mv720_combo_map_file/?at=default)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Teamsleepkid on January 20, 2017, 04:21:27 AM
sweet. how do i do it? how to locate 80000331_1808x727.fpm file? looked but its not like a normal program.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on January 20, 2017, 04:35:01 AM
If you are on a Mac, go to where MLVFS.workflow is installed. It should be in your home Library in the "Services" directory. Control click -> Show Package Contents -> Contents and you should see all the .fpm files in there. If you are on Windows, I'm not sure but it should be similar.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Teamsleepkid on January 20, 2017, 04:56:49 AM
cool thank you :D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: futografia on February 11, 2017, 08:24:01 PM
Does anybody is running this on sierra?

It doesn't show at right-click/services for me.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 12, 2017, 03:27:13 AM
Works fine here. You're right clicking on a folder not an MLV file right?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: futografia on February 12, 2017, 04:28:18 PM
Quote from: dmilligan on February 12, 2017, 03:27:13 AM
Works fine here. You're right clicking on a folder not an MLV file right?

Yeah, I'm dumb :)

thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ibrahim on February 15, 2017, 01:43:01 PM
Hi,

I have converted MLV file to cDNGs using raw2cdng but got green cast on my footage.
Then I tried MLVFS resulting in no cast, but I wonder if the images are in 16 bit as in the case of my CDNGs generated via raw2cdng?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 15, 2017, 09:28:53 PM
The DNGs are 16 bits per pixel, but it makes no difference, the extra bits are just padding.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ibrahim on February 15, 2017, 09:53:57 PM
That's great  :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: futografia on February 21, 2017, 04:21:17 AM
Somebody knows approximately how long it will take to mount 600gb of footage?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on February 21, 2017, 01:06:53 PM
Mounting doesn't take any time at all. It's indexing MLV files for the first time that might take a while. That should take on the order of the read speed of you storage device (indexing the files should be an I/O bound activity). Once your files are indexed or if they have already been index (e.g. you played them back in camera), you can play them immediately.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 22, 2017, 10:53:17 AM
Color matrix 1 for eosm is not correct. just noticed. It should be the same as for 700D.

        { 6985, 10000, -1611, 10000, -397, 10000, -3596, 10000, 10749, 10000, 3295, 10000, -349, 10000, 1136, 10000, 6512, 10000 },
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on February 25, 2017, 08:52:55 PM
Hi there, I have a issue with the audio sync in DR 12.5.

On the web pop up window it says I have audio but when I draging the cdng folders into the media pool I have no audio.

Plz help. What should I do?

Regards Olle
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on February 27, 2017, 11:42:57 PM
no one that has any advice on how to get the auto audio sync from MLVFS to work in DR working??
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on February 28, 2017, 12:25:29 AM
Did you by any chance refresh the web browser after selecting "DaVinci Resolve" checkbox within MLVFS's web guide yet?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 28, 2017, 06:13:23 AM
It,s working here. If you can,t provide logs or even a sample file there is not much help to give. Testing other converters as well as uploading a sample file will probably confirm what is not working on your end.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on February 28, 2017, 07:54:47 PM
Quote from: DeafEyeJedi on February 28, 2017, 12:25:29 AM
Did you by any chance refresh the web browser after selecting "DaVinci Resolve" checkbox within MLVFS's web guide yet?

I have tried to refresh the web page!

Here are some screen shots and the log file inside the cdng folder. the folder also contains the waw file.
(https://c1.staticflickr.com/4/3911/33017027352_bfd0db5f86_n.jpg) (https://flic.kr/p/SiATWh)Screen Shot 2017-02-28 at 19.41.53 (https://flic.kr/p/SiATWh) by Olle Olsson (https://www.flickr.com/photos/143911581@N04/), on Flickr (https://c1.staticflickr.com/4/3672/33017027182_f39a53f35d_n.jpg) (https://flic.kr/p/SiATTm)Screen Shot 2017-02-28 at 19.44.48 (https://flic.kr/p/SiATTm) by Olle Olsson (https://www.flickr.com/photos/143911581@N04/), on Flickr

Log file: https://drive.google.com/file/d/0B2mDAyOt5Qr0dTI0V0xWWHQ1T0E/view?usp=sharing (https://drive.google.com/file/d/0B2mDAyOt5Qr0dTI0V0xWWHQ1T0E/view?usp=sharing)

I really appreciate your support, thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 28, 2017, 08:11:22 PM
The log file is completely empty. Best if you upload the MLV file. If it,s too long just film a shorter sample and check if that also isn,t working before sharing it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on February 28, 2017, 09:42:46 PM
Quote from: Danne on February 28, 2017, 08:11:22 PM
The log file is completely empty. Best if you upload the MLV file. If it,s too long just film a shorter sample and check if that also isn,t working before sharing it.

looks like all log files are zero bytes!

here is a MLV file: https://drive.google.com/file/d/0B2mDAyOt5Qr0V1pjWDBvbDFuLUE/view?usp=sharing (https://drive.google.com/file/d/0B2mDAyOt5Qr0V1pjWDBvbDFuLUE/view?usp=sharing)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 28, 2017, 09:58:19 PM
There is no audio recorded. There is a zero byte wav file in mlvfs and when processed in mlv_dump there is no audio file at all. Did you set mlv_snd to on? Anyway, that,s probably your problem.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on February 28, 2017, 10:46:20 PM
Quote from: Danne on February 28, 2017, 09:58:19 PM
There is no audio recorded. There is a zero byte wav file in mlvfs and when processed in mlv_dump there is no audio file at all. Did you set mlv_snd to on? Anyway, that,s probably your problem.

yes of course mlv_snd was on! it´s weird that the web page of MLVFS says the it is out recorded tho. Maybe there is a bug with the build I use! thanks anyway.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 28, 2017, 10:54:13 PM
I think mlvfs always keep a wav file whether audio was recorded or not so problem is elsewhere.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on February 28, 2017, 11:59:52 PM
Maybe, I´m definitely not an expert but when mlv_snd is off Mlvfs web page says no audio. Do you think it will work if I update ML with a newer build?
Title: Re: MLVFS (LINUX) PROBLEMS WITH GUI
Post by: arboldeconfianza on March 03, 2017, 07:37:54 PM
Hi!, i can mount MLVFS and converter to DNG, buy i need help for use the GUI in LINUX!!!, if you guys can help me!?,and what is the better way to process LUTS with linux?, i appreciate all the work in this forum!!! thank youu! :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: slowlearner28 on March 12, 2017, 10:56:55 PM
Is it possible to have filenames generated from the "DaVinci Resolve Naming Scheme" setting consistent across Mac and Windows versions?

For example, on the Mac, filenames appear like this: "M13-0137MLV_1_2017-02-13_0001_C0000"

On Windows, the same file will appear like this: "M13-0137_1_2017-02-13_0001_C0000"

Seems like an easy fix to omit the extra "MLV" in the Mac version, but I tried editing the source code in Xcode and for the life of me couldn't figure it out. I'm trying to share Resolve projects between Mac and Windows workstations, and having the naming scheme consistent would make re-linking files much easier. Thank you!!! Love MLVFS and love this forum!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 14, 2017, 11:40:04 PM
Looking at the code, there's no reason they should be different. Unfortunately, I have no way to test or debug on Windows.


static int get_mlv_basename(const char *path, char ** mlv_basename)
{
    if(!(string_ends_with(path, ".MLV") || string_ends_with(path, ".mlv"))) return 0;
    char *temp = copy_string(path);
    const char *start = find_last_separator(temp) ? find_last_separator(temp) + 1 : temp;
    char *dot = strrchr(start, '.');
    if(dot == NULL) { free(temp); return 0; }
    *dot = '\0';
    struct frame_headers frame_headers;
    if(mlvfs.name_scheme == 1 && mlv_get_frame_headers(path, 0, &frame_headers))
    {
        *mlv_basename =  malloc(sizeof(char) * (strlen(start) + 1024));
        sprintf(*mlv_basename, "%s%s_1_%d-%02d-%02d_%04d_C%04d", start, dot + 1, 1900 + frame_headers.rtci_hdr.tm_year, frame_headers.rtci_hdr.tm_mon + 1, frame_headers.rtci_hdr.tm_mday, 1, 0);
    }
    else
    {
        *mlv_basename = copy_string(start);
    }
    free(temp);
    return 1;
}
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on March 14, 2017, 11:47:30 PM
Quoteand for the life of me couldn't figure it out.
+1
Spent some hours on trying omitting that MLV part myself  8)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on March 15, 2017, 12:28:13 AM
I never noticed that before. The issue looks like it is on the Mac and not Windows. It is pretty easy to rename the files so they match the files on Windows:

(https://c1.staticflickr.com/3/2849/33405053286_db697f9cd2_z.jpg)

Maybe fixing this on the Mac won't break it Windows?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 15, 2017, 03:18:51 AM
You can't rename virtual files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on March 15, 2017, 06:20:16 AM
Quote from: dmilligan on March 15, 2017, 03:18:51 AM
You can't rename virtual files.

Right, I was assuming he was copying the DNG files from one system to the other but that might be a false assumption. I tried some other Mac tricks like hiding the extension but the file names always came up with MLV in the name when using the DaVinci Resolve Naming Scheme. The default naming is fine.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: slowlearner28 on March 15, 2017, 07:38:06 AM
I have a much older version of MLVFS.workflow which didn't spit out virtual files with the extra MLV part. I'm going to try opening that in Xcode and find the same section of code that Dan pointed out, and copy and replace into the latest version. Will post back hopefully with a successful result!

And if you're thinking, why don't I just use this older version of MLVFS, unfortunately it has a bug that makes the number of the month one month earlier. So I'm still back to square one with the virtual file names not matching the Windows version.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 15, 2017, 05:35:25 PM
The MLV is supposed to be in the name, the surprising thing is that it reportedly isn't on Windows, unfortunately I can't even verify the claim that that's the behavior on Windows, much less debug it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: johannsebastianbach on March 15, 2017, 10:58:17 PM
Hey guys :)

I wanted to open my MLVFS virtual drive (OSX) in Davinci Resolve, but when I do Resolve "hangs" and by looking on the Activity Monitor MLVFS reads bytes very slowly, it would take ages. It's no problem when I import cDNGs, so I´m guessing it wants to read/playback all the data (200 GB) trough before just open the clips in the media pool. (I think the AE workflow shows the same behavior, but its way faster)

I just wanted to have a space friendly workflow and only open the MLV and render it out in Resolve to ProRes without doubling the size with export to cDNGs!

[EDIT]
Never mind, I tried it again and resolve was stuck for like a minute but afterwards it went on and showed the clips :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on March 17, 2017, 10:31:56 AM
Quote from: dmilligan on March 15, 2017, 05:35:25 PM
The MLV is supposed to be in the name, the surprising thing is that it reportedly isn't on Windows, unfortunately I can't even verify the claim that that's the behavior on Windows, much less debug it.
The windows shows "M05-1725MLV_1_2016-12-05_0001_C0000_000000.dng" as expected. Latest MLVFS with latest dokan.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: slowlearner28 on March 18, 2017, 02:28:29 AM
@bouncyball: Can you tell me how you got the latest version of Dokan to work with MLVFS? I've installed Dokan 1.0.2 but nothing happens when I enter the usual commands in the Command Prompt. Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: togg on March 18, 2017, 03:53:12 PM
A couple of things:

1) It doesn't work while targetting the same drive. Expected behaviour right?

2) How does fixed pattern noise works? What about chroma smoothing?

3) Is the latest release from 1 years ago?

thanks for the nice development :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 18, 2017, 05:20:57 PM
1. I'm not sure what you mean

2. https://bitbucket.org/dmilligan/mlvfs/src/9f8191808407bb49112b9ab14c27053ae5022749/mlvfs/patternnoise.c?at=master
https://bitbucket.org/dmilligan/mlvfs/src/9f8191808407bb49112b9ab14c27053ae5022749/mlvfs/chroma_smooth.c?at=master

3. 2016-5-31
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on March 18, 2017, 06:06:13 PM
Quote1) It doesn't work while targetting the same drive. Expected behaviour right?
I,m pretty sure he refers to when first selecting a folder with MLV files and then select the very same folder which should be the empty one(not MLV file folder again). Causes MLVFS to freeze(loop?). Maybe put in a bash/apple script notification when this mistake takes place and exit automator?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: togg on March 18, 2017, 06:34:24 PM
On the same drive, not even the same folder. Everything starts to go wrong yes.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on March 18, 2017, 06:40:43 PM
Quote from: togg on March 18, 2017, 06:34:24 PM
On the same drive, not even the same folder. Everything starts to go wrong yes.
Assuming you are on windows, mounting MLVFS to existing drive is _wrong_. Choose unused drive letter. Z: is very safe choice.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on March 19, 2017, 12:16:57 AM
On PC you can use 2 systems to mount mlv's , one uses the method of Virtual Drive , that's run on a simple cmd prompt here (http://www.magiclantern.fm/forum/index.php?%20topic=13152.msg127218#msg127218) is the setup for PC with the web gui ,
I made a short video tutorial about how to install and run it's about halfway down the page.
The other way is to "quick mount" with Prismo , It was dropped in favor of the MAC Style Web GUI where you can have a B/W preview and adjust different raw features.
I made a old video tutorial about Quick Mount Prismo , watch it here -->https://youtu.be/hidN3C1B4xk?t=1m44s
With the quick mount Yes you can mount mlv's anywhere on the PC from CF/SD Cards to Local or external drives even will works on Net storage drives e.g. "freenas" boxes etc... .
Great right ! will the only problem no way to access the mlvfs options easy like the web gui , but if options are not needed , Quick Mount is very useful works very well with resolve A.E. etc... .
I should mention thou that if you use a "NAS" for storage of mlv's and are thinking of quick mounting then it is a "must" to have very fast drives, e.g. raid "0" or "5" and not to mention
a least a 1Gb/s (1000Mb/s) Network , which mean about 80-120 MB/s R/W and of course lots of ram(16GB Min. in the nas) .
In my case I have a ZFS configured 3.5 TB with LZ4 compression raid 5 (5-1tb 7200rpm drives) (similar to mac's journaled format)
and I can't tell any different between locally or on the nas . I should also mention that you can map to your Network drive for mlvfs web gui version just like you would locally to a
internally HD or SSD .
Related information to install Pismo quick mount
Download and install PFMAP, and then register the MLV formatter DLL:
pfm register mlvfs.dll
https://bitbucket.org/dmilligan/mlvfs/downloads/mlvfs.dll
http://pismotec.com/download/archive/pfm-171-testing-win.exe

I have Pismo ver.171 installed on my pc/win7pro , you could use newer ones I think never tried thou.

I recommend PC Ver. MLVFS Web GUI , as Pismo does have problems on large volume of mlv's and will lockup.
In my opinion it's great to quickly check a few files here and there but as a main pipeline no way .  :D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: togg on March 19, 2017, 09:44:56 AM
I'm on mac! I'll come back to this when I'll get my hands on a 5D III :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 19, 2017, 06:06:39 PM
There's no problem having the target directory and the mount directory on the same logical or physical drive (I develop MLVFS almost exclusively this way). What could be problematic would be having the the mount directory somewhere inside the target directory.

You'll have to be much more specific than simply "everything starts to go wrong".
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: togg on March 19, 2017, 08:09:55 PM
I will in 3-4 days, promise! Thanks for the help so far. I just wanted to understand globally what new options there are now.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arboldeconfianza on March 21, 2017, 05:20:29 AM
Hi! in this last time i try with differents workflows and scripts in linux, today i running MLVFS and is amazing, but i cant use the webGUI, that is the problem! --->


webgui: scanning /home/tushita/MLV//...
webgui.c:120:load_resource(): fopen error: 'html_template.html': No such file or directory
webgui: scanning /home/tushita/MLV//favicon.ico...
webgui.c:120:load_resource(): fopen error: 'html_template.html': No such file or directory

maybe somebody can  help me!, and other question is, what is the best way to copy this DNG to other folder, i have windows too and can use lightroom or adobe after effects with this DNG, but take to much time in copy!, i try the make my workflow in linux  with MLVFS and MLVCONV, buy i need resolve change the DNG to NEUTRAL for this i have dependecies with LIGHTROOM,thank you for the attention..


PD:sorry for my english!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 21, 2017, 12:35:17 PM
The files in the mlvfs/data folder must be in the current working directory when you launch mlvfs.

make
cd data
../mlvfs <options>


Quote from: arboldeconfianza on March 21, 2017, 05:20:29 AM
but take to much time in copy!
What kind of speed are you getting? How fast is the drive? Are you copying to the same drive the MLVs are on? What processing options turned on? What's CPU usage of MLVFS like?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arboldeconfianza on March 21, 2017, 06:22:53 PM
@dmilligan perfect i can use the webgui now, but i dont see changes with give differents values, the changs is in the metadata?

(linux)
Sincerely my computer is the laptop i5 6gb ram and sata disk, is not the really computer prepared for render, but i try the see what a i can make, but in windows i cant use mlvfs for i cant install dokany to much problems of dependecys, maybe is more easy have mlvfs running in windows and i don need copy the archives, the velocity of copy es 768kb/s and i have only one sata in my computer,  CPU usage is 25% i try put more priority but nothing happen, in my terminal mlvfs open one by one dngs when copy, proccess dual-iso, i wish that information help!, thank you for the attention!


(windows)
mlvfs run good with PFM but i dont know where is the place for changes values and proccess DUAL-ISO!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 21, 2017, 08:29:20 PM
Dual ISO processing is very slow, not much you can do about that, the algorithm is very complex. MLVFS uses basically the same code as cr2hdr and performance is about the same. You can change some of the dual ISO options to make things a little faster, but quality will be reduced. There's a preview mode that is very fast, but quality is very low.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tweak on March 27, 2017, 05:21:49 PM
I tried out MLVFS on my old Macbook 2013 in Resolve but it was too slow so gave it a miss. Now I've come back to it with a brand new 15inch MBP and Resolve and although it plays back fine, in Reslove all the clips have a blue cast to them... Does anyone know what I'm doing wrong?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ZakWestbrook on March 29, 2017, 02:42:24 AM
When I right click and click MLVFS it puts them into "New (MLVFS Mount)" application. It also opens the web browser but nothing comes up. Any thoughts on why they're opening directly in the app and not on the browser window?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 29, 2017, 08:54:36 PM
I don't know what you mean by "them", "they're" or "app". Maybe post a screenshot(s) or video.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mattcraigthomas on March 29, 2017, 09:47:05 PM
Hi dmilligan,

I love MLVFS, been using it with total reliably for the past two years or so. Absolutely fantastic tool, thank you very much for all your hard work.

I'm particularly fond of the automatic (5÷3)x vertical scale of slow motion footage when processed through ACR; that was a brilliant discovery.

I'm as yet undecided about the merits of upscaling to UHD (there are arguments for and against that I won't go into here), but if we were to do it, it would best be done at the debayering stage, just as with the (5÷3)x desqueeze.

Would you please consider adding a 200% scale toggle to the options in the web GUI?

For those of us always shooting 1920 pixels across, this would (probably, maybe) give us the best possible 3840 pixel wide image for when mixing with natively shot higher resolution footage and/or finishing in UHD.

Much appreciated, thank you!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 29, 2017, 11:16:17 PM
Quote from: mattcraigthomas on March 29, 2017, 09:47:05 PM
it would best be done at the debayering stage, just as with the (5÷3)x
The vertical scale thing is just metadata (and not all raw processors even honor it).
As far as scaling goes, there's no reason to do it before debayering. It could be done as part of debayering, but MLVFS does not do debayering, and I have no intention of implementing anything like that. The only way I can even think of to upscale bayered data is to is to debayer, upscale it, then "re-bayer" it, which would be kind of silly.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mattcraigthomas on March 30, 2017, 01:50:56 PM
With respect, I think you've misunderstood me, dmilligan.

All I'm asking you to implement is an option to include the Adobe 200% scale metadata in the virtual CDNGs that MLVFS creates, so that the scaling can be done automatically as part of debayering with ACR, just as you have already done so with the (5÷3)x vertical scale applied to slow motion footage, although obviously that is not optional.

I really don't think what I'm proposing is silly. If you need or want to scale from 1920 wide to 3840 wide, and you debayer with ACR through After Effects, surely this would be the best time of the post processing pipeline to do so, while you still have access to all the raw data?
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on March 30, 2017, 05:18:43 PM
Quote from: mattcraigthomas on March 30, 2017, 01:50:56 PM
...I really don't think what I'm proposing is silly. If you need or want to scale from 1920 wide to 3840 wide, and you debayer with ACR through After Effects, surely this would be the best time of the post processing pipeline to do so, while you still have access to all the raw data?

+1
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on March 30, 2017, 08:01:57 PM
You can't specify an arbitrary scaling in the metadata, it's a pixel aspect ratio. You can only specify a change in the ratio of width to height, not some absolute change in both width and height.

EDIT: there does actually appear to be "BestQualityScale" tag that might work for this purpose. Anyone is welcome to fire up exiftool and try and see if it actually works. I don't plan on implementing this specific request, but rather have been meaning to for a while provide a generic way to override/add any particular DNG metadata. Just haven't had the time (probably won't for a while, if ever).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on March 30, 2017, 10:58:41 PM
Quote...provide a generic way to override/add any particular DNG metadata
Cool.

I tried some but don,t really know if it change anything. A bit short on time but I added and changed the tag with exiv2 and then checked metadata with exiftool. Couldn,t remember how to alter scale numbers with exiftool but exiv2 works.

So if anyone wants to experiment here,s a little howto. Install both exiv2 and exiftool with homebrew . First install homebrew.
In terminal do:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
then do:
brew install exiv2
brew install exiftool

To simply add the "BestQualityScale" tag to a dng file do:
exiv2 -M"set Exif.Image.BestQualityScale Rational 1/1" INPUT.dng
Then maybe try:
exiv2 -M"set Exif.Image.BestQualityScale Rational 5/3" INPUT.dng
or maybe
exiv2 -M"set Exif.Image.BestQualityScale Rational 7/3" INPUT.dng

Read more on the tag here:
http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec.pdf


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: COMMANDES on April 04, 2017, 12:03:36 PM
Hello. MLVFS is a very convenient thing, but for some reason it does not load the correction profiles of phase AF points when opening files written by "new" versions of ML.

To files from 3.07.2016 the profiles are still loaded, but to files written with "new" versions of ML - no.

Camera 650D
OS Windows 8.1
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 04, 2017, 01:30:33 PM
You're going to have to be more precise than just saying "new" and probably provide samples.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: COMMANDES on April 04, 2017, 07:38:53 PM
So, at the opening sequence MLV-file (shot 07.03.2016, the firmware was up-to-date at the time) mounted in MLVFS console appears "Loading focus pixel map '80000301_1872x1060.fpm' ..." and no AF points.

When opening files with firmware from 6.01.2017 (and later) to 10/14 bit, no information is displayed in the console, and the files open in Davinci with AF points.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on April 04, 2017, 08:35:34 PM
There was a July 9 build and a June 13 build. You can find them here:

https://builds.magiclantern.fm/650D-104-all-builds-changes.html

Run a test with these two builds and report back whether or not you can reproduce the issue. (Good luck--I doubt that's the problem.)

It would help if you could post a short MLV file that MLVFS can't remove the focus pixels. You said that it is happening with the later builds so it would be best if you use the newest nightly (unified) build if you want to record a new MLV file. You mentioned 10/14 bit so maybe you've been using an experimental build?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: COMMANDES on April 04, 2017, 11:11:14 PM
My results:

• Build on Jun13:
ok, no AF
• Build on Jul09:
ok, no AF
• Build on Mar30-2017:
bad, AF

By the link: in the build folder: jpeg file + dng file (from MLVFS) and original MLV (https://drive.google.com/drive/folders/0B4kbZldbAjcqaUwycm4xLVJvQW8?usp=sharing)
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dfort on April 05, 2017, 06:47:29 PM
It would really help if you can find the exact build when this first started happening. Start with the build right after July 9.

Is there another 650D user reading this that can reproduce the issue?


Sent from my iPhone using Tapatalk
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 05, 2017, 07:48:18 PM
As I suspected, the problem is the raw buffer size changed (from 1808x1190 to 1808x1189). MLVFS detects the map to use based on the camera id and the raw buffer size. All you need to do is duplicate the file 80000301_1808x1190.fpm and rename it to 80000301_1808x1189.fpm.

With the new RAWC block and crop_rec stuff we can (and will have to) come up with a little better system for getting the correct map.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: COMMANDES on April 05, 2017, 08:08:17 PM
Thank you so much. And what about a crop mode/others resolutions?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 05, 2017, 11:47:36 PM
You tell me. I don't own a 650D.

FYI when I say resolution, I'm not talking about the resolution you seleceted in the menu, rather the full (max) resolution of the current video mode. Since there is not (yet) metadata about what the video mode and crop settings are, we have to guess the video mode based on the full raw buffer resolution (each video mode has a unique raw buffer resolution currently, though that is no longer true with crop_rec and the new "4K" stuff)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: COMMANDES on April 05, 2017, 11:57:07 PM
Based on personal experience I would say that the other profiles are used for crop mode.

But I do not know the buffer-size to rename files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on April 06, 2017, 12:03:49 AM
mlv_dump -v
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: COMMANDES on April 06, 2017, 02:04:42 AM
Thank you, I've fixed everything I needed modes.

Based on this block:
Block: RAWI
  Offset: 0x00000034
    Size: 180
    Time: 209.197000 ms
    Res:  1920x640
    raw_info:
      api_version      0x00000001
      height           1107
      width            2592
      pitch            3240
      frame_size       0x004C9EA8
      bits_per_pixel   10
      black_level      127
      white_level      937
      active_area.y1   28
      active_area.x1   74
      active_area.y2   1101
      active_area.x2   2592
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on April 06, 2017, 02:04:55 AM
Quote from: dmilligan on April 05, 2017, 07:48:18 PM
As I suspected, the problem is the raw buffer size changed (from 1808x1190 to 1808x1189). MLVFS detects the map to use based on the camera id and the raw buffer size. All you need to do is duplicate the file 80000301_1808x1190.fpm and rename it to 80000301_1808x1189.fpm.

With the new RAWC block and crop_rec stuff we can (and will have to) come up with a little better system for getting the correct map.

Right, Danne ran into that too so he did some adjustments that allowed for a little slop in the vertical resolution. Fortunately is isn't shifting the coordinates of the focus pixels so it still works.

  mv1080)
    raw_buffer=1808x11**
  mv720)
    raw_buffer=1808x72*
  mv1080crop)
    raw_buffer=1872x10**
  zoom)
    raw_buffer=2592x110*


Not sure how that would apply in MLVFS but there's also the crop_rec module that requires a different FPM file that's the same size as the mv720 raw buffer. I experimented with combining the map files but it didn't work so we'll have to deal with this at some point. Isn't there new metadata for crop_rec that can be used?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: festr on April 24, 2017, 09:04:37 PM
Hi Dmiligan, do you plan to integrate the new lossless raw compression?

https://www.magiclantern.fm/forum/index.php?topic=18443.msg183653#msg183653
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ludzik3d on May 02, 2017, 03:05:53 AM
Hi
I want to try MLVFS but I get an error message "dokanfuse.dll is missing, please reinstall dokany.
I use Windows 10 64bit.
I tried everything what I found on the forum and in the manuals but it still doesn't work :(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: TrEK on May 02, 2017, 04:20:59 PM
Quote from: dmilligan on April 05, 2017, 11:47:36 PM


Hello.
I was make MLVSF

(http://i.imgur.com/us98922.png)

(http://i.imgur.com/njRe6bI.png)

But when i try put one folder on the timeline Adobe Premiere cc 2015 i receive such result:

(http://i.imgur.com/4tJKVAw.png)

(http://i.imgur.com/FVa6kck.png)

CDNG sequence duplicate such times as how available quanity of dng-pictures....

and i need put first dng-picture on timeline for avoid it:

(http://i.imgur.com/0K48UVG.png)

(http://i.imgur.com/vJSVVyf.png)

in such way its all correct...

what i can do for ability put all folder from BIN to timeline in one time ?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on May 06, 2017, 01:00:12 AM
Merged @bouncyball's lj92 stuff (https://bitbucket.org/dmilligan/mlvfs/pull-requests/11), and updated the Mac build (https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS.dmg).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on May 06, 2017, 07:56:16 AM
Only problem is, at 7.7MB per frame, I can't get real time 3K playback in Resolve with MLVFS (all options off) even on a relatively fast Mac (3.33 GHz 6 Core 48GB ram SSD/raid GTX-980. Tested on Yosemite Resolve 12.5 and El Capitan Resolve 14.

The same MLV file converted with MLV_dump (6.6MB per frame) plays back smoothly in Resolve.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on May 06, 2017, 07:04:39 PM
Quote from: squig on May 06, 2017, 07:56:16 AM
The same MLV file converted with MLV_dump (6.6MB per frame) plays back smoothly in Resolve.
Have you tried playback, with thouse 7.7MB MLVFS DNGs copied to HDD? If playback is OK than definitely (actually I'm quite confident about it) decompression overhead is an issue. If Canon HW compressor would produce multi slice (not the same as multi component) RAW data, decompression could be multi threaded per slice and hence be faster.

Unfortunatelly I'm quite far from this kind of programming (MLVFS actually is multi threaded app in its core) and can't say anything specific about making this portion of code multi threaded.

P.S. What is the CPU load during playback? (is it hackintosh?)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: a1ex on May 06, 2017, 10:01:54 PM
Quote from: bouncyball on May 06, 2017, 07:04:39 PM
If Canon HW compressor would produce multi slice (not the same as multi component) RAW data, decompression could be multi threaded per slice and hence be faster.

It actually does (http://www.magiclantern.fm/forum/index.php?topic=18443.msg176809#msg176809) (2 slices).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on May 07, 2017, 08:54:11 AM
Quote from: bouncyball on May 06, 2017, 07:04:39 PM
Have you tried playback, with thouse 7.7MB MLVFS DNGs copied to HDD? If playback is OK than definitely (actually I'm quite confident about it) decompression overhead is an issue.

P.S. What is the CPU load during playback? (is it hackintosh?)

Mac Pro 2010. CPU load with MLVFS running is around 27%. 14-19% with the copied file. The copied file plays back smoothly.

(http://i.imgur.com/AoqGdSN.png)
(http://i.imgur.com/8oPR2oX.png)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on May 07, 2017, 10:42:14 AM
@squig

Can you uncompress that high resolution MLV (mlv_dump -o uncompressed_dst.mlv compressed_src.mlv) and try to play in MLVFS?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on May 07, 2017, 01:31:36 PM
Quote from: bouncyball on May 07, 2017, 10:42:14 AM
@squig

Can you uncompress that high resolution MLV (mlv_dump -o uncompressed_dst.mlv compressed_src.mlv) and try to play in MLVFS?

Sure, but you're gonna have to give me the exact command. The file name is M01-2316.MLV
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on May 07, 2017, 02:42:19 PM
Try this:
mlv_dump -d -o OUTPUT.MLV M01-2316.MLV
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on May 07, 2017, 03:14:47 PM
Quote from: Danne on May 07, 2017, 02:42:19 PM
Try this:
mlv_dump -d -o OUTPUT.MLV M01-2316.MLV

Same stuttering playback in Resolve.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on May 07, 2017, 03:24:29 PM
@Danne: Thanks for correcting me.

@squig:
Quote from: squig on May 07, 2017, 03:14:47 PM
Same stuttering playback in Resolve.
Hmm... that means decompression is not a bottleneck. At least on your system. When I have time will check this under win.
Thanks for the answer and sorry about forgetting "-d" :).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on May 07, 2017, 07:19:16 PM
Quote from: bouncyball on May 07, 2017, 03:24:29 PM
Thanks for the answer and sorry about forgetting "-d" :).

No problem. It seems more like a disk access issue, but my SSD and raid have 400MB/s+ read speeds. The 3K file is only 185MB/s, and that's not an issue playing DNGs in Resolve at that data-rate.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on May 07, 2017, 07:22:06 PM
Squig. Could you post a short sample file? I´d like to test it over here on my computer and check how slow/fast it is.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on May 08, 2017, 08:05:07 AM
Quote from: Danne on May 07, 2017, 07:22:06 PM
Squig. Could you post a short sample file? I´d like to test it over here on my computer and check how slow/fast it is.

https://drive.google.com/open?id=0BzJ3L6nv6Fn0RG9tUzBmY3phb1k
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on May 08, 2017, 12:21:33 PM
@squig:

Tested your file at work:
CPU Xeon E3-1245 3.4Ghz
GPU (very slow) Nvidia GTX 650
SSD
~18fps (both mlvfs/no mlvfs)  CPU load ~64%
Looks like DR GPU bottleneck here.

Edit:
Tested at home:
CPU - 4790K 4.2Ghz
GPU - Nvidia GTX 1080
SSD
23.976 fps (realtime both mlvfs/no mlvfs) CPU Load ~43%

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: squig on May 11, 2017, 07:36:21 AM
Quote from: bouncyball on May 08, 2017, 12:21:33 PM

Tested at home:
CPU - 4790K 4.2Ghz
GPU - Nvidia GTX 1080
SSD
23.976 fps (realtime both mlvfs/no mlvfs) CPU Load ~43%

bb

Hmm. Mac OS or Windows?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on May 11, 2017, 09:25:50 AM
Quote from: squig on May 11, 2017, 07:36:21 AM
Hmm. Mac OS or Windows?
Win10 x64, DR 14b1
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: erikbaldwinson on May 23, 2017, 09:55:04 PM
I'm having the same problem stuttering problem with 12bitlossless mlvfs in both Windows 7 64 and linux centos7 in Resolve 14 beta2. In windows its awful, in linux it usually sorts itself out in a couple seconds but it makes editing tiring.

I've been running trial and error with nvidia gpus.

GTX 970 = bad
GTX 1060 = not good
GTX 1070 = not bad still not good

I've overclocked my 3770k to 4.4Ghz, turned off intel speedstep and other power saving stuff in bios and it's working at around 40%-60% capacity in Linux.

The stuttering seems like a hard drive issue but I have a samsung m2 drive in the pci port that does well on the crystal mark test.

The GTX 970 was not doing any work at all according to the gpu monitor at around 12-20% and 1GB of memory max used. This is basically the same case with the 1060 and 1070 yet playback performance is quite obviously a noticeable improvement from the 970 as the stuttering is less.

Why are the GPU monitors stating only 20% load and why is Resolve stating 23.98fps playback when it is clearly stuttering and playing back maybe 12fps?

I also turned off v-sync.

I have a feeling there is something else going on and upgrading the gpu is just easing off the tension indirectly helping the problem.

Any ideas? Trying to finish a project.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on May 30, 2017, 01:10:02 AM
Hi all,

It's been a while since I used mlvfs and now that I have fresh windows and reinstalled everything, took me a while to get this beast running..

Anyway, I am experiencing a massive stutter and general performance issue with 23.9 footage to the point where it is hardly possible to edit. I reported this long ago and someone found a timecode problem. I was under impression this was fixed and I think I was using it with no problems at some point..err..or maybe I'm dreaming? hahaha..

Or maybe using a wrong build or something..here how I got it working:

1. Installed Dokan 1.0.3.1000 (x64) from https://github.com/dokan-dev/dokany/releases/tag/v1.0.3

2. Unzipped MLVFS_x64.zip from http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/

3. Unzipped optimized.zip to MLVFS_x64 folder and renamed files adding "_opt" from http://www.magiclantern.fm/modules/modules/mlvfs_optimized.zip/

4. Renamed dokanfuse1.dll (in MLVFS_x64 folder) to dokanfuse.dll and dokan1.dll to dokan.dll

I tried all (mlvfs_x64.exe, mlvfs_x64_avx.exe, mlvfs_x64_opt.exe and mlvfs_x64_avx_opt.exe) and finaly capitalised on using mlvfs_x64_avx_opt.exe which seemed to be the fastest (super subjective/based on feeling)


all the settings on localhost panel are set to off, disabled, default or none (default values)

Same issues occur with footage copied from mlvfs drive onto HDD;
Yet unpacked using raw2cdng 1.7.5 works really well, not using mlvfs that is;

Also 25fps with mlvfs works like a charm too..


Thanks a ton! Good night..
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: erikbaldwinson on May 30, 2017, 10:59:28 AM
Quote from: voblaunsane on May 30, 2017, 01:10:02 AM
Hi all,

It's been a while since I used mlvfs and now that I have fresh windows and reinstalled everything, took me a while to get this beast running..

Anyway, I am experiencing a massive stutter and general performance issue with 23.9 footage to the point where it is hardly possible to edit. I reported this long ago and someone found a timecode problem. I was under impression this was fixed and I think I was using it with no problems at some point..err..or maybe I'm dreaming? hahaha..

Or maybe using a wrong build or something..here how I got it working:

1. Installed Dokan 1.0.3.1000 (x64) from https://github.com/dokan-dev/dokany/releases/tag/v1.0.3

2. Unzipped MLVFS_x64.zip from http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/

3. Unzipped optimized.zip to MLVFS_x64 folder and renamed files adding "_opt" from http://www.magiclantern.fm/modules/modules/mlvfs_optimized.zip/

4. Renamed dokanfuse1.dll (in MLVFS_x64 folder) to dokanfuse.dll and dokan1.dll to dokan.dll

I tried all (mlvfs_x64.exe, mlvfs_x64_avx.exe, mlvfs_x64_opt.exe and mlvfs_x64_avx_opt.exe) and finaly capitalised on using mlvfs_x64_avx_opt.exe which seemed to be the fastest (super subjective/based on feeling)


all the settings on localhost panel are set to off, disabled, default or none (default values)

Same issues occur with footage copied from mlvfs drive onto HDD;
Yet unpacked using raw2cdng 1.7.5 works really well, not using mlvfs that is;

Also 25fps with mlvfs works like a charm too..


Thanks a ton! Good night..

Hey man, thank you for sharing your installation workflow in great detail but unfortunately the optimized versions are outdated and do not support the Canon HW compressed MLVs.

The problem I am experiencing is playback with the 12bit lossless files. Some files playback perfectly and some files, for whatever reason, do not. Large files (ex.40gb+) both lossless and raw do not playback well - maybe your installation suggestions will solve that, I'll check next project.
Also, perhaps some of these optimized mlvfs.exe files could include the lossless playback support to resolve this playback problem.

I think having a latest greatest CPU would just strengthen through this lossless playback problem possibly. Although that being said in Linux my 3770K was not working hard at all (40-60%) and the playback was still stuttering as if the drive was failing or the CPU was maxed. Then suddenly playback was perfect with all files all sizes.. until I hit stop and then play.. and stuttering ensues..

Sometimes with big files on Windows the CPU is maxed.
Title: Problem with 10 bit dual iso
Post by: loknar on May 30, 2017, 09:57:10 PM
Hi,
i tried to use mlvfs on 10-bit mlv dual iso video from EOS M and i thought this is working combination; however, when i try to use dng file from inside mlvfs it opens interlaced and in console is:
ISO pattern     : BBdd RGGB
White levels    : 10000 5000
Noise levels    : 8.00 8.00 8.00 8.00 (14-bit)
Doesn't look like interlaced ISO

Am I doing something wrong?

cr2hdr does the same:
Input file      : /data/mlvfs/M30-1955.MLV/M30-1955_000100.dng
Camera          : Canon EOS M
Full size       : 2512 x 1068
Active area     : 2512 x 1068
Doesn't look like interlaced ISO


EDIT: It does, however, works in preview mode  ???
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on May 31, 2017, 07:34:49 AM
There seems to be reports that some people are having playback problems with MLVFS (PC) and Blackmagic Resolve
on large files . I just ran a few tests to fine any bottle necks or issues in general , here are the details :
Test Machine :
CPU : AMD 8 core 8350 (Overcocked to 4.8 GHz on water cooler)
MB  : Asus Sabertooth 990FX (ver.1.0)
Ram: DDR3 2x4GB kit(8GB total) Crucial Ballistix 1600 MHz Dual Channel
GPU: (2)x NVidia GTX 580 1.5GB Vram ea. (resolve only used 1 GPU for cuda processing  as per the free licence  and 2nd for GUI)
SSD: OS Windows 7 Pro
Media File Storage : On board 8TB Raid0 (Striped)(4x2GB 7200 rpm Seagate Barracuda SATA 6 Gb/s NCQ 64MB Cache drives R/W 500+MB/s @ 50% free)
Test File: Green Screen Talking Head 10bit Full MLV 2144x1076 23.976fps +audio 12,000 frames = about 7:30 (from 5d2 so 8 spanning files = 30GB)
Blackmagic Resolve ver. 12.5.2 (free ver.)
Resolve setting:
2144x1076 23.976 project
32bit floating point
DaVinci YRGB Color science
Cache frames in Uncompressed 10bit
Enable background caching after 5 seconds
------------------------------------------------
Optimized Media:
Resolution (choose automatically)
format-Umcompressed 10-bit
---------------------------------------
Working Folders:
Cache files location (cache clips on D drive 8tb raid0
------------------------------------------------------------
Basic raw adjustment for log output

MLVFS with Dokan + Web GUI 10bit input file  >16bit output Cdng's+Audio (no options enabled) (location on 8TB raid0)
Play file in full res @ 23.976 with 3 frame drops at 2:00 , 5:00 & 7:00+ 1-2 frames audio out of sync. (but a quick pause & start sync is restored)
All 8 CPU working @ about 60% with 6.3GB being used with resolve .

MLVFS with Pismo (File Mount Audit Package build 171) (right click quick mount) No web GUI or options for processing just extracts 16bit Cdng+Audio to virtual folder in the same folder
no CMD prompt to deal with , mlv's can be mounted anywhere on the pc system . Mounted the 30GB file on 8TB raid0.
Only 5 CPU's working @ 30% with the same 6.3GB of ram being used with resolve running.
Play file on timeline @ 23.976 and no frame drops of sync loss .

So with dokan there twice the load with 8 core working @60%
and with Pismo only 5 core working @ 30% load .

So if performance is the issue where you need real-time playback (for editing) and not just transcoding
then used Pismo (https://bitbucket.org/dmilligan/mlvfs) (look at the bottom of page) I use both systems on my PC.



Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: goose on May 31, 2017, 09:59:00 PM
Hi everyone, i followed the installation instructions for MLVFS and it appears as a service to run when i right click the folder. Once i select it though nothing all happens.

Any ideas?

Thanks in advance.
Modify message
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 01, 2017, 03:11:37 PM
There's probably a file selection dialog that came up behind other windows (hide other open windows and you should see it). It's really annoying that Automator does that, but not anything I can do about it.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 01, 2017, 03:23:22 PM
There´s actually an easy way to make the dialog box open as the front window. Add an applescript and print tell me to activate before the upcoming "Ask for finders items" and the promt window will always be placed on top of everything else.

(https://s4.postimg.org/wdxvje2wt/Screen_Shot_2017-06-01_at_15.22.53.png)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 01, 2017, 04:26:14 PM
Cool!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: goose on June 01, 2017, 07:57:21 PM
that was the answer thank you, cant believe that was the answer, feel so thick lol
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on June 03, 2017, 07:08:50 PM
so I've tested bouncyball's version of mlvfs (with dokan 1.0.3) as it seems to be the latest mlvfs compile out there and still have 23.9 fps stuttering issue..

to demonstrate I would like to share Performance Monitor graphs produced while scrubbing ~3 minute long footate in Premiere

25 fps graph looks nice: constant load on hdd, processor cruncing numbers, works lika a charm and the whole take can be previewed in a blink
(http://thumb.ibb.co/ht5oJF/25_scrubbing.gif) (http://ibb.co/ht5oJF)

while 23.9 fps is a real mess: hdd constantly choking and processor is having a nap, visual performance - stuck on one frame mostly...
(http://thumb.ibb.co/cvPhdF/23_9_scrubbing.gif) (http://ibb.co/cvPhdF)


anyone else experienceing this?.. I find this is the case with all versions of mlvfs I found out there...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on June 04, 2017, 03:03:34 AM
Quote from: voblaunsane on June 03, 2017, 07:08:50 PM
so I've tested bouncyball's version of mlvfs

What is this ? Can someone explain bouncyball's  MLVFS !
That's dmilligan's project from my understanding on PC & MAC.
I can't even find his source code , why would bouncyball be modifying mlvfs as it work flawlessly as is .
Can't even find any threads about this to follow .

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on June 04, 2017, 03:40:16 AM
Quote from: reddeercity on June 04, 2017, 03:03:34 AM
Can't even find any threads about this to follow .

mlvfs on steroids (https://www.magiclantern.fm/forum/index.php?topic=18975.0mlvfs%20on%20steroids)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on June 04, 2017, 05:14:49 AM
Thanks @ DeafEyeJedi , OK it's just mlv_dump not mlvfs , I thought I missed something .
really nothing new I can see specially for any digic4 cams looks like digic5 & up thou.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 04, 2017, 09:21:51 AM
@reddeercity

Quote from: reddeercity on June 04, 2017, 03:03:34 AM
I can't even find his source code , why would bouncyball be modifying mlvfs as it work flawlessly as is .
Can't even find any threads about this to follow .
The correct thread link is here (http://www.magiclantern.fm/forum/index.php?topic=19429.0)

Anyway, after some testing I did pull request and David merged it to main MLVFS repo. It's all there now :)

regards,
bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on June 06, 2017, 02:23:21 PM
Quote from: reddeercity on June 04, 2017, 03:03:34 AM
What is this ? Can someone explain bouncyball's  MLVFS !
sorry for not being clear enough.. mlvfs.exe (2017-05-06) https://bitbucket.org/bouncyball/mlv-tools/downloads/

Quote from: reddeercity on June 04, 2017, 03:03:34 AM
<...> as it work flawlessly as is .
do you not experience problems with 23.9fps on PC?

(http://thumb.ibb.co/mcN7fv/VUPH9485_1.gif) (http://ibb.co/mcN7fv)

It appears I might be but a singleton...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eraizgn on June 06, 2017, 07:31:15 PM
guys the out put is DNG or cDNG? I didnt get it whats going on, lots of codes are swimming around.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 06, 2017, 07:58:20 PM
Read the headline.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eraizgn on June 07, 2017, 12:31:07 AM
I download `mlvfs ` which is 289-292KB and edit to, send to for `@echo off
for %%a in (*.MLV) do ( md "%%~na" 2>nul )&(E:\RAW\mlvfs.exe -o %%~na\%%~na_ --dng %%~na.MLV )`

But just create a empty folder which name MLV is..

Code is not true?

mlv_dump is working, but I want to convert cDNG not DGN

What should I do?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 07, 2017, 01:26:02 AM
There are some tutorial videos linked in the first post.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 07, 2017, 09:15:43 AM
Quote from: voblaunsane on June 06, 2017, 02:23:21 PM
do you not experience problems with 23.9fps on PC?
What is your hardware?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eraizgn on June 07, 2017, 01:43:16 PM
Quote from: dmilligan on June 07, 2017, 01:26:02 AM
There are some tutorial videos linked in the first post.

I did what exactly he has done but, check it out

(http://thumb.ibb.co/cM0xFv/46b3b49f174385905d1d50570d2166ae.png) (http://ibb.co/cM0xFv)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on June 08, 2017, 03:44:22 AM
Quote from: eraizgn on June 07, 2017, 01:43:16 PM
I did what exactly he has done but, check it out
Looks like mlvfs is working , you may have a problem with the mlv file . If it a small file upload a link to it so it can be check by others.
Does the mlv file look Ok with mlv_dump ? is this a 4k & compressed (lossless) 14bit raw file ?
If the files is really big you can shorten with  mlvdiag.1.0.1 (http://dl.phreekz.de/raw2cdng/mlvdiag.1.0.1.zip)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eraizgn on June 08, 2017, 11:15:51 AM
Quote from: reddeercity on June 08, 2017, 03:44:22 AM
Looks like mlvfs is working , you may have a problem with the mlv file . If it a small file upload a link to it so it can be check by others.
Does the mlv file look Ok with mlv_dump ? is this a 4k & compressed (lossless) 14bit raw file ?

Yes it is working with mlv_dump. It was recorded 3.5K lossless 14bit raw
the file was about 520MB, with your program 5mb now

https://files.fm/u/4p7q3ygh
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on June 08, 2017, 01:17:11 PM
Probably you are using some old version before lossless compression support was added.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on June 08, 2017, 03:28:17 PM
Quote from: bouncyball on June 07, 2017, 09:15:43 AM
What is your hardware?

My hardware is good enough to handle 25fps footage with absolutely no problems, but 23.9fps is not possible to edit..it's lagging, stuttering, etc..

Before you blame it on hardware please examine following piece of evidence: graphs showing resource usage by mlvfs process

Quote from: voblaunsane on June 03, 2017, 07:08:50 PM

25 fps graph looks nice: constant load on hdd, processor cruncing numbers, works lika a charm and the whole take can be previewed in a blink
(http://thumb.ibb.co/ht5oJF/25_scrubbing.gif) (http://ibb.co/ht5oJF)

while 23.9 fps is a real mess: hdd constantly choking and processor is having a nap, visual performance - stuck on one frame mostly...
(http://thumb.ibb.co/cvPhdF/23_9_scrubbing.gif) (http://ibb.co/cvPhdF)

It looks like working with 23.9, mlvfs is not utilising resources properly...


Now to answer your question:
i7 - 3770K @ 3.50GHz
8GB RAM
NVIDIA GTX 570
7200 rpm hdd
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 08, 2017, 03:51:45 PM
Sample mlv? I run mac but surely others could check your file, metadata etc.
Is nearby fps like 24 fps working? Is it only with 23.9?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eraizgn on June 08, 2017, 04:03:12 PM
Quote from: dmilligan on June 08, 2017, 01:17:11 PM
Probably you are using some old version before lossless compression support was added.

that MLV file works with mlv_dump btw

@dmilligan

DokanInstall_0.8.0
MLVFS_x86

(http://thumb.ibb.co/k1s1av/78f3bfd25a65f335ec2d09875c8b88bb.png) (http://ibb.co/k1s1av)

@voblaunsane

Windows 7 Ultimate
I7-4930K
32GB Ram
NVIDIA GeForce 770X (gonna buy SLI 1080ti )
3x 256 GB SSD ( C:windows / d: ProgramFiles / e: Works on Projects )
8TB HDD ( Storage )
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eraizgn on June 08, 2017, 04:08:49 PM
Quote from: Danne on June 08, 2017, 03:51:45 PM
Sample mlv? I run mac but surely others could check your file, metadata etc.
Is nearby fps like 24 fps working? Is it only with 23.9?

Yes it is sample mlv..
I do not know but yea 24.01fps
(http://thumb.ibb.co/ivvZTF/8afd8358fd4cdc4cfc60c8df8dfe91a8.png) (http://ibb.co/ivvZTF)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 08, 2017, 05:47:03 PM
@voblaunsane
Hardware is ok exept hdd maybe. There is no difference between 24/23.976 and 25 FPS for MLVFS. It does not care about FPS at all. What cares about it is the NLE while processing cinema DNGs (adobe premiere as I understand in this case). I never had this kind of stutter under windows with premiere/davinci resollve even on the less powerful hardware than you've got (note: I used SSD).

@eraizin
Quote from: eraizgn on June 08, 2017, 04:03:12 PM
DokanInstall_0.8.0
MLVFS_x86
Very old stuff. If your Windows is x64 then try links here (http://www.magiclantern.fm/forum/index.php?topic=19429.0) to download last MLVFS x64 and latest Dokany at the bottom of the 1st post.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on June 08, 2017, 06:16:20 PM
Quote from: bouncyball on June 08, 2017, 05:47:03 PM
@voblaunsane
Hardware is ok exept hdd maybe. There is no difference between 24/23.976 and 25 FPS for MLVFS. It does not care about FPS at all. What cares about it is the NLE while processing cinema DNGs (adobe premiere as I understand in this case). I never had this kind of stutter under windows with premiere/davinci resollve even on the less powerful hardware than you've got (note: I used SSD)

I've tried all sorts of things and now can not be 100% certain, but I believe to have tried extracting cdngs with mlvfs and mlv2cdng with only later rendering normal results and that precisely makes me think the problem lies within mlvfs.

I'll be able to do some more tests and confirm this next week. Is there any particular facet of this you would be interested in, @bouncyball??
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eraizgn on June 08, 2017, 09:03:10 PM
Quote from: bouncyball on June 08, 2017, 05:47:03 PM
@voblaunsane
Hardware is ok exept hdd maybe. There is no difference between 24/23.976 and 25 FPS for MLVFS. It does not care about FPS at all. What cares about it is the NLE while processing cinema DNGs (adobe premiere as I understand in this case). I never had this kind of stutter under windows with premiere/davinci resollve even on the less powerful hardware than you've got (note: I used SSD).

@eraizinVery old stuff. If your Windows is x64 then try links here (http://www.magiclantern.fm/forum/index.php?topic=19429.0) to download last MLVFS x64 and latest Dokany at the bottom of the 1st post.

bb

@bouncyball / I downloaded mlvfs.exe ( 2017-05-06 ) and Dokany (DokanSetup.exe  6.51 MB) Now, I need to drag into to C:\MLVFS_x64 ? replace the old one?


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eraizgn on June 09, 2017, 11:56:35 AM
@bouncyball I did it works, thanks for any comment to me for fix it out, One more question

After converting MLV to cDNG, closing cmd but dokan(Z) do not disappear whatever I do, If I just close the PC

is there any command for quit?/eject? etc?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 09, 2017, 02:54:38 PM
Quote from: voblaunsane on June 08, 2017, 06:16:20 PM
I'll be able to do some more tests and confirm this next week. Is there any particular facet of this you would be interested in, @bouncyball??
Nope nothing particular, raw2cdng (there is no mlv2cdng AFAIK) extracts phisical cdng files and writes to disk. MLVFS shows virtual dng files and sure has an overhead on CPU. You can just copy dng files from MLVFS virtual dirs to phisical disk and try opening them in NLE (23,976 ones).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 09, 2017, 03:02:40 PM
Quote from: eraizgn on June 09, 2017, 11:56:35 AM
After converting MLV to cDNG, closing cmd but dokan(Z) do not disappear whatever I do, If I just close the PC
is there any command for quit?/eject? etc?
Look at my post here (http://www.magiclantern.fm/forum/index.php?topic=19429.msg184274#msg184274)

You have 2nd option though: run MLVFS in foreground with -f switch (which is not displayed by windows version of MLVFS with usage help :) but it's there and works). It will open console window and if you just close this window it's gonna unmount the virtual MLVFS drive too.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on June 17, 2017, 12:51:23 AM
Hi, I have a deadline on Thursday, yea I know it´s stupid to use ML on that types of jobs... ::)

anyway, in my Cdng folders the .wav contains zero bytes and in Davinci Resolve there is of course now audio channel.

Please help what to do?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 17, 2017, 10:10:07 AM
@_OLLE_

Did you try extracting with mlv_dump? If yes Is it normal playable wav?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: _OLLE_ on June 17, 2017, 01:01:37 PM
Quote from: bouncyball on June 17, 2017, 10:10:07 AM
@_OLLE_

Did you try extracting with mlv_dump? If yes Is it normal playable wav?

no I use MLVFS. What is mlv_dump?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 17, 2017, 03:04:10 PM
Download latest build (2017-06-15 16:39) of mlv_dump for your operating system here (http://builds.magiclantern.fm/experiments.html).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on June 17, 2017, 04:58:29 PM
Quote from: bouncyball on June 09, 2017, 02:54:38 PM
Nope nothing particular, raw2cdng (there is no mlv2cdng AFAIK) extracts phisical cdng files and writes to disk. MLVFS shows virtual dng files and sure has an overhead on CPU. You can just copy dng files from MLVFS virtual dirs to phisical disk and try opening them in NLE (23,976 ones).

yeah, sorry it's raw2cng, not mlv.. Tried 3 methods to access dngs on that same 23.976 file and can confirm that both mlvfs virtual cdngs and cdngs extracted to a drive are stuttering and are impossible to work with, overal cpu load is *only* 18%.. Now using extracted dngs wit raw2cdng (16bit maximised) procesor load goes up to ~40% and it's playing back normally..

Also just noticed that mlvfs does not offer time code?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Welles on June 18, 2017, 07:32:49 PM
Quote from: voblaunsane on May 30, 2017, 01:10:02 AM
4. Renamed dokanfuse1.dll (in MLVFS_x64 folder) to dokanfuse.dll and dokan1.dll to dokan.dll

Thanks a ton! That did the trick for me! I had the same symtoms as others: mlvsfs was dying without any message.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 20, 2017, 07:35:40 AM
Quote from: voblaunsane on June 17, 2017, 04:58:29 PM
Also just noticed that mlvfs does not offer time code?
Actually it does. Every DNG frame has it's time code in the header.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on June 20, 2017, 12:26:08 PM
Quote from: voblaunsane on June 17, 2017, 04:58:29 PM
yeah, sorry it's raw2cng, not mlv.. Tried 3 methods to access dngs on that same 23.976 file and can confirm that both mlvfs virtual cdngs and cdngs extracted to a drive are stuttering and are impossible to work with, overal cpu load is *only* 18%.. Now using extracted dngs wit raw2cdng (16bit maximised) procesor load goes up to ~40% and it's playing back normally..

@bouncyball, any idea why is this happening??
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 20, 2017, 02:44:11 PM
Quote from: voblaunsane on June 20, 2017, 12:26:08 PM
@bouncyball, any idea why is this happening??
Nope. I have to take a look at DNGs produced by raw2dng.

I don't even know how raw2dng does (as you say) 16 bit maximization - really converts values to 16 bit (black/white will be also affected) or just unpacks them to 16 bit word.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on June 21, 2017, 03:54:17 PM
Quote from: voblaunsane on June 17, 2017, 04:58:29 PM
Also just noticed that mlvfs does not offer time code?

ok, so the reason I thought mlvfs does not offer timecode was because the first frame is reporting 00:00:00.00 when I'm using free runing timecode, hence it's incorrect!..

and first frame extracted with raw2cdng has a timecode of 15:47:12.0 which is a correct timecode!

Quote from: bouncyball on June 20, 2017, 02:44:11 PM
Nope. I have to take a look at DNGs produced by raw2dng.

I don't even know how raw2dng does (as you say) 16 bit maximization - really converts values to 16 bit (black/white will be also affected) or just unpacks them to 16 bit word.

I wish I could understand that code or the theory behind it, but I'm sure you could see how it's done here: https://bitbucket.org/chmee/raw2cdng

mlvfs 1st frame:
https://drive.google.com/open?id=0BzU9K72Ryy5YMEZnN3Z2V3BrNTQ

raw2cdng 1st frame:
https://drive.google.com/open?id=0BzU9K72Ryy5Ya1FIaFNfNGxaQ0k
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 21, 2017, 04:17:37 PM
Timecode starts from zero then counts frames synched with fps. Are you referring to timestamp? Can't check the files atm.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on June 21, 2017, 04:35:08 PM
Quote from: Danne on June 21, 2017, 04:17:37 PM
Timecode starts from zero then counts frames synched with fps. Are you referring to timestamp? Can't check the files atm.

Well on 5d mkIII 1.2.3 I have two timecode options:
rec run, which starts from zero and
free run, which is equal to cam time..

I'm talking about free running, because it's the setting of my choice..
My point is, raw2cdng dng files gets it right and also cam native MOV files gets it right and mlvfs seem to lack this functionality..
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 21, 2017, 07:00:49 PM
@voblaunsane:

Thanks for DNG files. Used exiftool and rawdigger to view the raw data.

raw2cdng (in maximized mode) converts and interpolates 10/12/14 bit values to real 16bit range (0-65535) and sets Black level to 0 and White level to 65535 in DNG header. But I can't say why NLE likes those DNGs more than MLVFS produced ones and why it chokes on your system when fps is 23.976.

And you are right about the time code. MLVFS always starts from 00:00:00.00. It seems raw2cdng gets time from RTCI block and sets timecode start to that time and then increments according to FPS.

bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: arboldeconfianza on June 22, 2017, 08:13:02 PM
Hi guys I wanna begin with MLVFS WORKFLOW in macbook pro, I used after effects and my macbook pro have ati radeon and cudda is not compatible, anybody haved experience with final cut in MLV workflow?, probably in the future i can make a test and share!, thank you!!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 8rnity on July 06, 2017, 05:35:00 PM
hi, is it possible NOT to mount "_PREVIEW.gif" file and "*.log" files?
these two files block AE to import DNG sequence dragging in the relative folder.
I can't find an option in command, and is a trouble as if I import all DNG files as sequence and not as folder when I render in camera raw just first file will be adjusted.
instead, when I import the whole folder, modify original is enough to modify overall sequence in the folder.

hope I explained well. thank you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 8rnity on July 07, 2017, 01:38:09 AM
well, the problem is not the presence of *.log and _PREVIEW.gif but the NAME format of log file mounted.
to fix this it needs to be mounted with _ at the start of filename.
example:

filename.MLV > MOUNT > _PREVIEW.gif, filename.log, filename_000000.dng, filename_000001.dng ... etc
AE analyze filename.log as first file of sequence as its structure is similar to .dng files and returns error.

if filename.log is mounted as _filename.log everything goes well.

please would someone fix it or help me to modify the source and recompile.
thank you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: COMMANDES on July 09, 2017, 08:05:56 AM
Whether it is possible to include AA algorithms as AMaZE or mean32 not only in the mode of Dual-ISO processing, but also for normal RAW video to get rid of aliasing?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: sys73r on July 11, 2017, 04:15:31 AM
Quote from: arboldeconfianza on June 22, 2017, 08:13:02 PM
Hi guys I wanna begin with MLVFS WORKFLOW in macbook pro, I used after effects and my macbook pro have ati radeon and cudda is not compatible, anybody haved experience with final cut in MLV workflow?, probably in the future i can make a test and share!, thank you!!

1) Download it, double click on the file it will install as service.
2) You can either copy the RAW files from your camera to the Mac or keep them in the camera (slower)
3) right click on the directory you have the MLV (RAW) files (on step 2)
4) Create or use an empty directory whenever you want (movies, downloads, anywhere.)
5) In the menu, go to the end SERVICES > MVLFS and it will ask you to SELECT a "mount point" choose the directory you create or use on STEP 4
6) It will open a browser window with options and current status of the process, once completed go to the DIRECTORY on STEP 4, there you have your usable files. you can now open them with "MlRawViewer.app" (https://bitbucket.org/baldand/mlrawviewer/downloads/) you can use any other app out there it's just a suggestion, you can review and export the files for further post-processing.

I wrote the instructions off the top my head. so feel free to check me.

cheers.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: nikki on July 11, 2017, 10:33:00 PM
Quote from: sys73r on July 11, 2017, 04:15:31 AM
1) Download it, double click on the file it will install as service.
2) You can either copy the RAW files from your camera to the Mac or keep them in the camera (slower)
3) right click on the directory you have the MLV (RAW) files (on step 2)
4) Create or use an empty directory whenever you want (movies, downloads, anywhere.)
5) In the menu, go to the end SERVICES > MVLFS and it will ask you to SELECT a "mount point" choose the directory you create or use on STEP 4
6) It will open a browser window with options and current status of the process, once completed go to the DIRECTORY on STEP 4, there you have your usable files. you can now open them with "MlRawViewer.app" (https://bitbucket.org/baldand/mlrawviewer/downloads/) you can use any other app out there it's just a suggestion, you can review and export the files for further post-processing.

I wrote the instructions off the top my head. so feel free to check me.

cheers.




Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Boygetslost on July 16, 2017, 08:26:33 AM
Do people think the MLVFS is that much better then something like Rawmagic?

With MLVFS  once you have mounted the MLV files, is it best to load them directly. Or copy them out of the mounted drive so they are just normal DNG folders?

Thanks

Chris

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: justinbacle on July 16, 2017, 03:09:42 PM
Quote from: Boygetslost on July 16, 2017, 08:26:33 AM
With MLVFS  once you have mounted the MLV files, is it best to load them directly. Or copy them out of the mounted drive so they are just normal DNG folders?

The best thing is that you load them directly so that you don't have to store DNGs and the original MVL footage. As mlvfs allows to make changes on the fly (chroma smoothing, banding removal ans so on) I just keep all the MLV files and fire up MLVFS when needed to work with these files.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: garfield72 on July 17, 2017, 08:05:15 AM
Hello, I installed Dokany and download MLVFS_x86, now I don't know what to do with the MLVFS zip file.
Should it be decompressed? And where to copy it?

How to use
"Start it via the command line like this:
Code: [Select]
cd <mlvfs_exe_dir>
mlvfs.exe <mount point> --mlv_dir=<directory with MLV files>"

Thanks for your help

François
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: the associate on July 17, 2017, 08:58:28 PM
Hi, relative newb to terminal commands in Ubuntu 16.04. Just wondering if someone who's installed MLVFS on Linux successfully can share their step by step command lines? As per the app's installation instructions, I've already installed FUSE, and also have an unzipped MLVFS Master File ready to go in my downloads folder. Just not sure how to compile from the command line using MAKE. I see this example in the Installation ReadMe:

<< mlvfs <mount point> --mlv_dir=<directory with MLV files>  >>

Just wondering if any Linux users out there can cut and paste their actual terminal command lines? Am decent at cutting and pasting my own values over existing templates, but am still not all that well-versed in generating original terminal commands from scratch.

To date, have been able to run most of my ML programs (MLV Mystic, MLRawViewer, etc.) via wine, but alas, no can do with MLVFS. Help?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: justinbacle on July 17, 2017, 09:15:56 PM
Quote from: garfield72 on July 17, 2017, 08:05:15 AM
Hello, I installed Dokany and download MLVFS_x86, now I don't know what to do with the MLVFS zip file.
Should it be decompressed? And where to copy it?

How to use
"Start it via the command line like this:
Code: [Select]
cd <mlvfs_exe_dir>
mlvfs.exe <mount point> --mlv_dir=<directory with MLV files>"

Thanks for your help

François

Here is my cmd file to launch MLVFS :

r:
cd R:\MLRAW\MLVFS_x64
mlvfs_x64.exe V: --mlv-dir=R:\50D
pause


Where R:\MLRAW\MLVFS_x64 is the path where I unzipped the MLVFS zip file
and R:\50D is the path were are stored my MLV footage files
You'll have to install dokany before launching the script to make it work.
Once it is launched, just go http://localhost:8000/ on your favorite web browser to see the settings and preview the files

Hope it helps :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on July 18, 2017, 06:04:13 PM
This should be all you need to do to compile on ubuntu (not tested)

sudo apt-get install libfuse-dev
sudo apt-get install git
cd ~/
git clone https://bitbucket.org/dmilligan/mlvfs
cd mlvfs/mlvfs
make
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: garfield72 on July 18, 2017, 07:12:24 PM
Thanks justinbacle, i create a cmd file with:

c:
cd C:\MLVFS_x64
mlvfs_x64.exe V: --mlv-dir=C:\RAW
pause

When i launch it, i have the message (see cmd.png), after i go to http://localhost:8000/ and i have the message (see brownser.png).


how to do?

(https://thumb.ibb.co/hqTYSv/brownser.png) (https://ibb.co/hqTYSv)

(https://thumb.ibb.co/hLop0F/cmd.png) (https://ibb.co/hLop0F)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: reddeercity on July 19, 2017, 04:49:01 AM
I don't understand what the browser message is  , but I bet that's the default browser Microsoft Edge  right ?
I have nothing but trouble with it , it doesn't even support it's own Silverlight plugin .
Use a different browser e.g. IE 11 (Internet Explorer) or maybe chrome , firefox , etc... or change the security setting
without the web page GUI you can still go to the virtual drive to get the Cdng's you just don't have access to the option
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: garfield72 on July 19, 2017, 08:59:58 AM
Hello, the browser is not edge but Opera and I have the same message with firefox. I will try without the GUI, but or found the virtual drive? Is my cmd file good?

Thanks

François
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on July 19, 2017, 04:24:47 PM
It's very unlikely the browser, the MLVFS webgui only uses super basic and very standard HTML and JS stuff. Probably the problem is either you didn't start MLVFS from the correct directory (the one with the webgui support files) or you need to use the -f switch to force foreground mode (on *nix OSs this is necessary, not sure how dokany/Windows works).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: justinbacle on July 19, 2017, 08:35:10 PM
Quote from: garfield72 on July 18, 2017, 07:12:24 PM
Thanks justinbacle, i create a cmd file with:

c:
cd C:\MLVFS_x64
mlvfs_x64.exe V: --mlv-dir=C:\RAW
pause

When i launch it, i have the message (see cmd.png), after i go to http://localhost:8000/ and i have the message (see brownser.png).

how to do?


There is a problem with your cmd file or with your installation of MLVFS, you should see the "pause" if there is no problem when launched.
Did you get an error message when launching the cmd file ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: garfield72 on July 20, 2017, 06:51:32 PM
Hello, when lauching the cmd file, i have no error (see attached files).
My folder MLVFS_x64 is in the under C:\
(https://thumb.ibb.co/f3pDx5/cmd.png) (https://ibb.co/f3pDx5)

(https://thumb.ibb.co/gG7NqQ/folder.png) (https://ibb.co/gG7NqQ)

image hosting (https://fr.imgbb.com/)
 and the folder which contains MLV footage files (RAW)also.

François
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tonij on August 02, 2017, 04:47:33 PM
Can somebody please explain to me how to use 'Fix Pattern Noise' with MLVFS?
When I turn it 'ON' and refresh the web page, it doesn't seem to make any visual changes to my DNG files. I tried in both Windows 7 and Mac without success.

Using MLV Dump I can successfully remove fixed pattern noise using a dark frame.
Example images:


(https://thumb.ibb.co/iZfUH5/1.jpg) (https://ibb.co/iZfUH5)


(https://thumb.ibb.co/ggzOc5/2.jpg) (https://ibb.co/ggzOc5)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 03, 2017, 01:05:26 AM
Could be some caching issue. Might want to make sure you turn the option on very first thing before doing anything else, or even start MLVFS with it on via the command line (MLVFS does some internal caching, but the FUSE library or the OS might also be doing some caching that MLVFS can't do anything about).
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: DeafEyeJedi on August 03, 2017, 03:13:06 AM
Have you tried updating the FUSE library in OS X yet @tonij?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tonij on August 03, 2017, 07:42:57 AM
@DeafEyeJedi Can you tell me how I go about updating the FUSE Library? I'm not great with Macs

@dmilligan When I include "--fix-pattern-noise" in Windows Command Prompt it's applied successfully.

Is anyone successfully making these changes through http://localhost:8000/ on Windows or Mac?

Also how is MLVFS able to remove fixed pattern noise when I'm not giving it a suitable darkframe? Should I be providing darkframes if so how?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 03, 2017, 07:27:56 PM
Quote from: tonij on August 03, 2017, 07:42:57 AM
Also how is MLVFS able to remove fixed pattern noise when I'm not giving it a suitable darkframe? Should I be providing darkframes if so how?
http://www.magiclantern.fm/forum/index.php?topic=11787.msg158766#msg158766
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on August 06, 2017, 02:31:31 AM
Quote from: tonij on August 03, 2017, 07:42:57 AM
@DeafEyeJedi Can you tell me how I go about updating the FUSE Library? I'm not great with Macs

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: AneMotion on August 12, 2017, 10:53:13 PM
Hi guys. Does anybody know if there's a Windows 10 64 bit version of the MLVFS folder?
This current MLVFS_x86 does not work on Win 10 64.
Or is there another way to do this?


(http://thumb.ibb.co/kSr4Uv/error_1.png) (http://ibb.co/kSr4Uv)


(http://thumb.ibb.co/m4tJpv/error_2.png) (http://ibb.co/m4tJpv)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on August 13, 2017, 08:36:21 AM
@AneMotion:

Look at this (http://www.magiclantern.fm/forum/index.php?topic=19429.0) thread. Download windows binaries.
Also read this (http://www.magiclantern.fm/forum/index.php?topic=19429.msg188254#msg188254) post and follow instructions mentioned there.
Don't forget to uninstall previously installed Dokany before installing newer one.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 12georgiadis on August 19, 2017, 06:04:43 PM
Hello,

I just made some shot this week and when dealing with mlvfs, one file had a blue spot on it (seems to be a light). I tried all settings in MLVFS but cannot get rid of. Any idea ?
(http://thumb.ibb.co/nK88CQ/Capture_d_cran_2017_08_19_21_19_09.png) (http://ibb.co/nK88CQ)

image hosting (http://fr.imgbb.com/)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on August 19, 2017, 07:50:47 PM
Maybe a hot or dead pixel. If nothing else helps it is possible to get rid of it by creating a pixel map file in Switch using imageJ.
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: 12georgiadis on August 19, 2017, 09:59:18 PM
Quote from: Danne on August 19, 2017, 07:50:47 PM
Maybe a hot or dead pixel. If nothing else helps it is possible to get rid of it by creating a pixel map file in Switch using imageJ.
Ok I'll Try it. It's very big in the Picture


Envoyé de mon iPhone en utilisant Tapatalk
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: sigurdneby on August 25, 2017, 07:05:04 PM
Excuse my ignorance, but I could not help to ask after trying countless times and not finding my answer in the nest of the thread. I just might be an idiot. Feel free to point me to page numbers.

I have earlier found MLVFS to be very quick, whether its dumping the DNGs directly into resolve or copying them to a drive - but now I no longer have that experience. The speed is at 0.4mb per second and Resolve is working awfully slow. Have tried updating FUSE, using both the virtual drive as well as the directed folder, I can't see the spinning wheel and I am on El Capitan 10.11.6

Cheers


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 25, 2017, 08:23:48 PM
What options do you have enabled? (some are quite slow)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: sigurdneby on August 25, 2017, 08:30:03 PM
Quote from: dmilligan on August 25, 2017, 08:23:48 PM
What options do you have enabled? (some are quite slow)

Vertical stripes and dead pixels - might not need the latter one. Will try to reinstall MLVFS now, the .MLV's are using quite a while to load in MLVFS, strange thing - always been very reliable!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: sigurdneby on August 25, 2017, 09:18:19 PM
SOLVED - Reinstallation of MLVFS, FUSE, a good old restart and only vertical stripes ticked. Should have done all of this before crying, but I appreciate the attention!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: EuAcelasi on August 28, 2017, 10:37:39 PM
Could anuboduy please explain ho to install on Windows 10
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Ottoga on August 29, 2017, 10:22:03 AM
Just unzip the application to a location of your choice and run the app directly.

Everything that it needs is self contained within its folder. 
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: EuAcelasi on August 29, 2017, 10:24:22 PM
downloaded the 103 release... lunched the DokanSetup... Installation complete... what next? nothing new on the desktop or in the Explorer...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: EuAcelasi on August 30, 2017, 06:05:48 PM
Sorry for asking stupid questions... I`m trying o install the on the fly Converter on Windows 10, but don`t understand exactly how... on x32 system you work in cmd prompt, but here... what are exactly the steps to take... I downloaded the latest version 1.0.3, installed the Visual Studio Comunity 2015 with SDK and WDK. Then lunched the dokanSetup file and it installed. How do I get to mount the virtual Z drive.  Please, may be anybody could explain explain step by step procedure... Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: magnesiumnitrate on August 31, 2017, 04:52:51 AM
Forgive me if this has been asked and answered, I couldn't find anything googling or searching the forum.

Is there any way to enable multithreading in the new MLVFLS that is compatible with 12/14 bit lossless? I'm running fix pattern noise and it's absurdly slow but only using one thread on one core so 100%/2400% in activity monitor. (12c/24t Mac Pro)

I saw some promising info with raw2cng that looked like it was forcing MLVdump to use as many threads as you told it to, but I have no idea how to go about modifying the script for MLVFS to make it work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on August 31, 2017, 05:53:00 PM
MLVFS is multithreaded, but not all algorithms can be parallelized (easily or at all). The fix pattern noise algorithm is an experimental algorithm written by a1ex. The implementation as written by a1ex is not easily parallelizable (or maybe not at all, IDK) per frame. However, multiple frames can be processed at a time, but you have to request them. So for example if you ask for one DNG and then wait til it's done to ask for the next one, you will end up waiting. Most programs that access files won't behave like that, b/c it's extremely uncommon for file access to be CPU bound.

If the real problem is that the algorithm is too slow, there are all sorts of optimization avenues to explore, not just multi-threading (vectorization, gpu/hardware acceleration, etc). But you will have to understand and analyze the code and algorithm to do any of this. Any optimization or other improvements to the implementation are welcome contributions.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on September 01, 2017, 09:28:18 AM
Quote from: dmilligan on August 31, 2017, 05:53:00 PM
Any optimization or other improvements to the implementation are welcome contributions.
+1 :D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: magnesiumnitrate on September 01, 2017, 03:52:57 PM
I'm at best a script kiddie, so actually working the code and optimizing it is way beyond me. However, I will go on record saying I'm happy to do whatever testing anyone else needs, just shoot me a PM!

Quote from: dmilligan on August 31, 2017, 05:53:00 PM
MLVFS is multithreaded, but not all algorithms can be parallelized (easily or at all). The fix pattern noise algorithm is an experimental algorithm written by a1ex. The implementation as written by a1ex is not easily parallelizable (or maybe not at all, IDK) per frame. However, multiple frames can be processed at a time, but you have to request them. So for example if you ask for one DNG and then wait til it's done to ask for the next one, you will end up waiting. Most programs that access files won't behave like that, b/c it's extremely uncommon for file access to be CPU bound.

If the real problem is that the algorithm is too slow, there are all sorts of optimization avenues to explore, not just multi-threading (vectorization, gpu/hardware acceleration, etc). But you will have to understand and analyze the code and algorithm to do any of this. Any optimization or other improvements to the implementation are welcome contributions.
Title: MLVFS for Windows 10 64bit?
Post by: samson33 on September 02, 2017, 07:55:50 PM
Is it possible to use MLVFS for Windows 10 64bit?
The thread "MLVFS for Windows" is deprecated.

Is it still possible and does a tutorial exist?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on September 03, 2017, 01:53:46 PM
For Windows x64 take a look at this post: http://www.magiclantern.fm/forum/index.php?topic=13152.msg188446#msg188446 (http://www.magiclantern.fm/forum/index.php?topic=13152.msg188446#msg188446)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: EuAcelasi on September 05, 2017, 08:57:51 AM
I`ve tried for days to install the MLVFS binary... nothing worked just because the mlvfs.exe file was x86, even in the link to x64 binary... Finally found on a russian site a mlvfs_64.exe... http://www.pr-video.com/i_mlvfs.html
it finally created the new virtual drive. The problem is that it called Removable Disk (Z) and acces denied from Explorer. I can see the drive in DaVinci, but it doesn`t show any file or directory... Anybody knows why???
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: EuAcelasi on September 06, 2017, 08:40:59 PM
Could anybody please point to a link with a mlvfs_64 binary... this one doesn`t work... http://www.magiclantern.fm/forum/index.php?topic=13152.msg188446#msg188446 it `stops working`...
Here http://www.pr-video.com/i_mlvfs.html I found a 64 version and it creates the drive, but I cannot acces it `unavailable`.... I see the drive in Resolve but nothing from the RAW folder...  The only positive thing is that I see it in http://localhost:8000/ Anybody please HELP!!!

EVRIKA!!! It works!!! with http://www.magiclantern.fm/modules/modules/mlvfs_optimized.zip/
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: samson33 on September 10, 2017, 06:41:55 PM
Quote from: EuAcelasi on September 06, 2017, 08:40:59 PM

EVRIKA!!! It works!!! with http://www.magiclantern.fm/modules/modules/mlvfs_optimized.zip/

Hello,
can you just point out - step by step - what you have done to get it working on Windows 10.?
I tried it for days and not working.

Thank you
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: EuAcelasi on September 18, 2017, 08:18:46 AM
Same for me... It seems that the mlvf.exe was not working... I finally found the right one for my system in the link above... mlvfs_optimized... mlvfs_x64.exe
Before downloading the MLVFS zip and running the exe file in Command Prompt I Installed the Visual Studio 2015, the WDK and SDK, then the latest Dokan 1.0.4 with all the .exe files...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cas2336 on September 19, 2017, 07:21:59 AM
Forgive me if this has already been discussed, but for some reason MLVS doesn't load my MLV files on the web-based GUI. It worked once, but I've since been unable to get any files to load.

Any idea?

Signed,
Total Noob

(http://thumb.ibb.co/f32Lyk/Screen_Shot_2017_09_18_at_10_20_18_PM.png) (http://ibb.co/f32Lyk)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on September 21, 2017, 12:12:35 AM
Do the actual files still show in the Finder? Can you upload ~/.mlvfs.log?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: wangtrirat on October 01, 2017, 04:39:31 AM
Is it any improvement in Compressed RAW playback. It's pretty intensive task I can barely play the clip in resolve. I mostly convert to proxy and edit before going back to the raw.

Uncompressed has no problem at all. But I currently most shot compressed raw for disk space reason.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: samson33 on October 02, 2017, 09:46:13 AM
Hi,

still no way to use it with Windows 10?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on October 06, 2017, 07:14:50 PM
@dmilligan

Hi David, I wanted to ask this question long time ago :)

Why when 'Fullres Blending' is off in MLVFS dualiso settings, produced image almost pitch black? Is that some bug derived from a1ex's cr2hdr code?

regards
bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: leojames on October 11, 2017, 07:38:14 PM
Hello I am having an issue with Davinci Resolve 14 and MLVFS once adding in the MLV files that are 14bit lossless Davinci Resolve crashes after i have imported from the MLVFS mount drive, has anybody had the same experience?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: levisfaustus on October 21, 2017, 01:59:43 AM
WINDOWS 10, x64, stable with this files:

DOKAN installed v. 1.0.3.1000

and this folder on drive C:

https://drive.google.com/file/d/0BwToT6nrQcPYNm55WTh2VHdoU1E/view?usp=sharing
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on November 23, 2017, 03:19:01 AM
hello can someone give me a hand?
i did everything and the virtual drive is mounted but when i go to
http://localhost:8000
it just says
HTTP Error 404. The requested resource is not found.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: GutterPump on November 23, 2017, 10:48:22 AM
Quote from: papasapien on November 23, 2017, 03:19:01 AM
hello can someone give me a hand?
i did everything and the virtual drive is mounted but when i go to
http://localhost:8000
it just says
HTTP Error 404. The requested resource is not found.

Do you have these files, especially html_template.html and jquery-1.12.0.min in your MLVFS folder ?

(http://image.noelshack.com/fichiers/2017/47/4/1511430387-mlvfs.jpg)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on November 24, 2017, 09:29:45 AM
yes i have just checked and i have those files in my folder
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: GutterPump on November 24, 2017, 11:10:27 AM
What is your operating system + what is your version of Dokan ?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on November 24, 2017, 10:30:33 PM
my operating system is windows 10
i am running dokan 1.0.3. 1000

.
i have also tried turning off windows firewall but that did not help.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: GutterPump on November 24, 2017, 11:32:41 PM
I also use W10 and i use this (https://github.com/dokan-dev/dokany/releases/download/v1.0.3/DokanSetup_redist.exe) version of Dokan and this (http://www.dropbox.com/s/4ymbt8xxfrwtn1h/MLVFS.zip?dl=0) version of MLVFS without any problems.

Hope this will help you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on November 25, 2017, 01:37:09 AM
sadly it did not :/ . same results. thank you for trying though . I am getting a new hard drive in a couple days and will do a clean install of windows10 . so i will not waste anyones time trying to help me if this is a weird probably having to do with my system. i will come back in a couple days though if the problem is still occuring on my new set up. ( I still dont know what going on on this set up haha but I guess it might not matter soon anyways. )
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on November 26, 2017, 03:36:01 AM
does anyone know how to exit the program? when i close cmd the virtual drive is still there . (also my new hard drive lets it works properly)
windows 10
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: GutterPump on November 26, 2017, 11:15:33 AM
Ctrl + shift + skip, check mlvfs.exe and stop the program.

Maybe there is best solution than this one. 
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on November 29, 2017, 02:26:30 PM
Here (http://www.magiclantern.fm/forum/index.php?topic=19429.msg184274#msg184274) is my old post describing how to correctly unmount the mlvfs virtual drive under windows. Path to 'dokanctl.exe' depends on which version of dokany you use.

Another way is to run mlvfs in foreground with '-f' switch (ironically mlvfs help does not show this switch under windows but it is there and works) and when you close the console mlvfs goes down and dokany mounted drive unmounts.

The command is:
mlvfs -f Z: --mlv-dir=\mlvdir

regards
bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lureb74 on December 09, 2017, 11:46:56 PM
Quote from: papasapien on November 26, 2017, 03:36:01 AM
does anyone know how to exit the program? when i close cmd the virtual drive is still there . (also my new hard drive lets it works properly)
windows 10

A way that I found is: after closed the cmd window, re-open it again and use the same command and path.
This works for me, it unmounts the virtual drive. I'm on Win 7, don't know on other OS...

For me the -f method suggested by @bouncyball doesn't work... but the previous method he suggested yes it works!

SO YOU CAN USE THIS:

@ECHO MLVFS - DRIVE UNMOUNTER
@ECHO OFF
C:
cd C:\Program Files\Dokan\Dokan Library-1.1.0\
pause
dokanctl.exe /u M


Paste it in a notepad, modify it if needed (with correct drive and path where dokanctl.exe is, and assigned virtual drive letter, M in my case) and save it as anyname.bat

Double click on it, hit any key to confirm
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: lureb74 on December 10, 2017, 01:05:54 AM
Hi everybody!

Is there a way to "tell" MLVFS to fix black levels at 2048? As the option --black-fix=2048 on MLV_DUMP does?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on December 10, 2017, 12:02:03 PM
AFAIK, nope :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on December 23, 2017, 11:50:06 PM
hey is there no way to fix focus pixels in MLVFS?
(http://thumb.ibb.co/in4tQR/grab_1_1_1.jpg) (http://ibb.co/in4tQR)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on December 24, 2017, 05:27:55 AM
come to think of it i think i have a problem. i dont think focus pixel maps are working at all. all of my videos have focus pixels in them can someone help?. im using the dokan 1.0.3. and the x64 version of mlvfs. thank you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on December 24, 2017, 03:07:06 PM
You need to provide me with the sample MLV. Then I'll check what is the cause of focus pixels not working in your case.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on December 24, 2017, 09:04:20 PM
i uploaded one to google drive. here is the link.

https://drive.google.com/open?id=1m9wSoVrRrecTuKuL8YHu4Xwpi8SB1H9I

thank you
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on December 25, 2017, 08:55:02 AM
Download this map (http://nic.caucasus.net/ml/?dir=&download=80000326_1808x1189.fpm) and put it to the folder with MLVFS binary.

Then report back :)

good luck
bb
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on December 25, 2017, 10:43:21 AM
Looks good to me. I´ll be using it in Switch as well.

14bit(notice the stinky hot pixels created)
(https://s9.postimg.org/9pcduuilr/Screen_Shot_2017-12-25_at_10.39.07.png)

12bit
(https://s9.postimg.org/65qg51vbj/Screen_Shot_2017-12-25_at_10.39.34.png)

10bit
(https://s9.postimg.org/y5ujpaw73/Screen_Shot_2017-12-25_at_10.39.52.png)

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on December 26, 2017, 04:43:47 AM
woah thank you bb it worked! :0
youre the best!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on December 26, 2017, 08:06:00 AM
Never mind ;)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 12georgiadis on January 02, 2018, 08:13:48 PM
Hi !
Does MLVFS features "the pattern noise" function that exists in MLV app ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tonij on January 03, 2018, 11:30:03 AM
Hi,
It seems I've got a stability issue with MLVFS and I'm after some help. I'm using Windows 7 on my Macbook.
I periodically get popup messages saying MLVFS has stopped working and I have no choice but to close off my project and load up MLVFS again.
I've been living with this issue for a while but trying to work with 1TB worth of footage and having MLVFS always stopping has put me to a halt.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 03, 2018, 11:57:29 AM
Why are you running windows version on a macbook?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tonij on January 03, 2018, 12:26:44 PM
Quote from: Danne on January 03, 2018, 11:57:29 AM
Why are you running windows version on a macbook?

Windows is just what I'm comfortable using
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dmilligan on January 04, 2018, 01:28:40 AM
Upload a crash dump or at the very least the console output
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tonij on January 09, 2018, 01:41:18 AM
Quote from: dmilligan on January 04, 2018, 01:28:40 AM
Upload a crash dump or at the very least the console output

I thought the issue went away after I uninstalled/reinstalled some Microsoft Visual C++ Redistributable things but it still crashes.
Where exactly do I get the crash dump from?

Also it's MLVFS 32-bit I'm using on a 64-bit system, is that what I'm supposed to be using?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: IDA_ML on January 09, 2018, 11:15:54 AM
Tonij,

Here is a brief instruction on how I got MLVFS wokling:

----------------------------------------------------------------

For those MLVFS users on Win7 x64 please note:  Win7 x64 works well with the 32-bit versions of your MLVFS software but crashes with the 64-bit ones.

1) Uninstall the Dokany 0.8.0 library and restart your PC

2) Install the DokanSetup_redist-1.0.5.1000 package

3) Copy the two files from Bouncyball's ZIP file into your MLVFS_x86 directory.  Please download the ZIP file from here in the next 7 days:
   
https://we.tl/i1EveIQ461

4) Rename the old mlvfs.exe to mlvfs_old.exe in case you wish to revert and rename mlvfz.exe to mlvfs.exe

5) http://localhost:8000/

6) Enjoy MLVFS working also with losslessly compressed MLV files !


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tonij on January 09, 2018, 12:18:03 PM
Thanks IDA_ML I'd like to give it a go.
Where can I get the DokanSetup_redist-1.0.5.1000 package from?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: IDA_ML on January 09, 2018, 02:28:55 PM
From their website, of course:

https://github.com/dokan-dev/dokany/releases
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tonij on January 10, 2018, 01:43:37 AM
Thanks IDA_ML,
So I followed your instructions and got MLVFS working, but when I open up an existing project in Premiere I can't get smooth playback in the timeline anymore it freezes after only a few seconds of playback. Wasn't like that before.
I cleared Adobe media cache but it didn't help. Is there something I can clear/reset on the MLVFS side of things?

@dmilligan,
I found what I assume is a MLVFS crash dump file (42mb) https://ufile.io/fe8r3
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: IDA_ML on January 10, 2018, 09:39:19 AM
I am sorry, I cannot help you with Premiere.  I use Resolve and have no problems with the above MLVFS installation.  Works perfectly!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: voblaunsane on January 12, 2018, 08:37:47 PM
Quote from: tonij on January 10, 2018, 01:43:37 AM
but when I open up an existing project in Premiere I can't get smooth playback in the timeline anymore it freezes after only a few seconds of playback. Wasn't like that before.

I haven't used MLVFS for a while now so a few things might have been upgraded, but I encountered this problem you describe with 23.9fps footage and 25fps seemed to be working well...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: tonij on January 13, 2018, 06:46:31 AM
Quote from: voblaunsane on January 12, 2018, 08:37:47 PM
I haven't used MLVFS for a while now so a few things might have been upgraded, but I encountered this problem you describe with 23.9fps footage and 25fps seemed to be working well...

Now that you mention it all my footage is 23.976fps. I'll have to try another frame rate and see...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: chchrlam on January 15, 2018, 08:03:46 PM
I had a problem with focus pixels appearing in the MLVFS dng files:

650D with magiclantern-Nightly.2017Dec07.650D104
24 FPS 1280x720 and 24 FPS 12870x720 600D movie crop mode

I copied 80000301_1808x1190.fpm to 80000301_1808x1189.fpm and the problem went away.
Now MLVFS reports: Loading focus pixel map '80000301_1808x1189.fpm
and all the pink dots are gone

Cheers

Edit: Needed to copy 80000301_1872x1060.fpm  to 80000301_1872x1059.fpm for the 1x1 subsampling 600D crop-mode
Title: Re: MLVFS - a FUSE based, &quot;on the fly&quot; MLV to CDNG converter
Post by: dfort on January 17, 2018, 10:26:09 PM
The latest focus pixel map files are here:

https://bitbucket.org/daniel_fort/ml-focus-pixels/src/8bf385a7e828aa07aa605bc911ea94921b5c3c58/focus_pixel_map_files/?at=default

As you discovered there is a lot of duplication in the map files. Read the focus pixel topic if you're curious why that is.


Sent from my iPhone using Tapatalk
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dcw1979 on January 30, 2018, 11:01:41 AM
Hi.

Is there an option to add a lut? I dont see an option but thought I would ask in case Ive just missed it. Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: BTP253 on February 10, 2018, 09:03:43 AM
Hi there,
Sorry if this is a stupid question, I run MLVFS and choose my empty file however I always receive this message:
(https://thumb.ibb.co/jM2WWS/Screen_Shot_2018_02_09_at_11_59_33_PM.png) (https://ibb.co/jM2WWS)



Im running Mac OS High Sierra 10.13.3
Any help would be much appreciated
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: haeki on February 10, 2018, 10:53:01 AM
Hi, sadly I can't bring MLVFS back to life. I had it working back in 2017, but now it's not working anymore. Updated both, fuse and mlvfs but the dialog to choose the mountingfolder doesn't pop up and the gear keeps on turning staying at 0%. Any suggestions?

I'm on a mac 10.12.6

(http://666kb.com/i/dqv60jz8zshd6r6rm.png)

Kind regards
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: haeki on February 10, 2018, 10:28:50 PM
Quote from: haeki on February 10, 2018, 10:53:01 AM
Hi, sadly I can't bring MLVFS back to life. I had it working back in 2017, but now it's not working anymore. Updated both, fuse and mlvfs but the dialog to choose the mountingfolder doesn't pop up and the gear keeps on turning staying at 0%. Any suggestions?

I'm on a mac 10.12.6

(http://666kb.com/i/dqv60jz8zshd6r6rm.png)

Kind regards

It's all good, the dialogbox was hidden somewhere far away. :-)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on March 23, 2018, 05:55:41 PM
Trying to get mlvfz working for compressed mlvs. I have normal mlvfs working on this computer but when issuing mount command with mlvfz instead of mlvfs.exe nothing happens. Is their some step of the install process I'm missing/ have forgotten?

Mlvfz works fine on my other computer...

Win10

Any help is appreciated.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on March 24, 2018, 02:40:29 PM
Here (https://www.magiclantern.fm/forum/index.php?topic=19429.msg196250#msg196250) is your older post and my answer. It seams you have to reinstall win.

And please stop calling it mlvfZ! It was just typo made by me ;). It is plain 32bit mlvfs with compressed MLV support.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on March 26, 2018, 04:58:34 PM
Ok, thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: clanlee on April 04, 2018, 11:36:34 PM
MLVFS Installation on Windows 10 x64

Install latest version of Dokany:
https://github.com/dokan-dev/dokany/releases (https://github.com/dokan-dev/dokany/releases)

Reboot if necessary.

MLVFS_x64
Download http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip)
Extract MLVFS_x64.zip

Delete the following files from the extracted folder:
dokan1.dll
dokanfuse1.dll

Using MLVFS:
Note the path of the extracted MLVFS_x64.exe
Open Command Prompt

Run program from within directory into the executable:
The web gui interface only works if you call the executable from the same directory
cd mlvfs_x64
mlvfs_x64.exe Z:\ --mlv-dir=C:\Videos\Files

Run from any location by typing full path to executable:
If you run this from a remote directory, the web gui will not work
C:\mlvfs_x64\mlvfs_x64.exe Z:\ --mlv-dir=C:\Videos\Files

If your folder name contains spaces, use quotation marks for path:
"C:\Extracted files\mlvfs_x64\mlvfs_x64.exe" Z:\ --mlv-dir="C:\Videos from Camera\Files"

Or:
C:\mlvfs_x64\mlvfs_x64.exe Z:\ --mlv-dir="C:\Videos from Camera\Files"

Or:
"C:\Extracted files\mlvfs_x64\mlvfs_x64.exe" Z:\ --mlv-dir=C:\Videos\Files

Notes:
The Z:\ option is setting a mount point.  You'll have to use an unused drive letter in your computer.  When you open Windows Explorer, you'll see a list of all the drives in your computer.  Use a drive letter that is not used.


Quote from: g3gg0 on February 26, 2016, 08:54:54 PM
@Canon eos m:

win10:
please uninstall dokan from your control panel
install dokan v1.0.0-rc1 (https://github.com/dokan-dev/dokany/releases/download/v1.0.0-RC1/DokanSetup.exe)
unpack mlvfs_x64 (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip) into your folder f:\mlvfs_x64\
open a commandline

f:
cd \mlvfs_x64
mlvfs_x64.exe Z:\ --mlv-dir=F:\


osx:
did you do a chmod +x FILENAME ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on April 08, 2018, 12:08:35 PM
Thanks clanlee for detailed instructions and links for MLVFS win 10 x64 Install!
Got it running now. I had to go in and manually delete all old dokan entrys in regedit to be able to install new dokan. Even though I had uninstalled previous versions there were so much crap left there so the new dokan thought old versions were still installed and refused to install at first.

Mounting some really large files now, several around 200GB. It seems to take quite awile for dokan/mlvfs to load them. Is this normal? Previous versions i think it was pretty much intant even if the files were very large?

Also, does this version support mlvs with lossless compression? All the files I'm testing to mount now are 12bit lossless.

Switched back to old version with support för lossless. Files mounted instantaneously and it still worked with new version of Dokan.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: clanlee on April 09, 2018, 07:25:10 AM
I've run test with 15-45 seconds of RAW video clips.  With those small clips, MLVFS and Dokan (1.1.0.2) does not seem to have delay.

Which version are you running that allows large files to mount immediately?

Part of the reason that I posted instructions was to help write updated information.  This is the first one that I was successfully able to figure out.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on April 09, 2018, 08:37:38 AM
I said too much to fast, takes some time to load them when the files are large. When I tried to mount 2TB+ files at the same time Mlvfs got stuck on loading forever. I'm going to try to add one big file at a time to the mount directory to see if It has a easier time indexing them one by one.

I'm using a x86 version of mlvfs I got from bouncyball that has support for compressed mlvs. Latest Dokan version.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on April 13, 2018, 04:23:38 PM
Just wanted to share my latest experience dealing with large amounts of data on Windows 10 using bouncyballs x86 compile of MLVFS with support for lossless.

We filmed a live show with a runtime of around 2h with 5 cameras. We filmed the show twice the same day and collected over 3tb of 12bit raw lossless data.
You can film around 93min with sound on one 256GB CF card and it turns out that one Original LP-E6 has just enough power to power the camera for that amount of time.

Had no big problems filming but the postprocessing was extremely slow. The idx-file generation on on these big and long file must have taken 24h or more. I just left my office computer on and checked in on it remotely from time to time.
After all idx files had been generated I started converting all mounted dngs through slimraw to disk. Strange thing here was that mlvfs stopped serving files after converting around 170K frames. So remotely I had to remount and restart Slimraw at 170K intervals to get through the 1.5 million frames (thank god for remote desktop :-)). After that everything went smooth exporting DNXHD files from resolve.

Is the slow experience with MLVFS  consistent with using it on Mac or Linux?

/end rant
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on April 13, 2018, 04:38:15 PM
Wow, that's a lot of dng files :).
It sounds about right that indexing would take this long. I'm even surprised you got all working in the end.
I would also suggest exporting to dng files directly through mlv_dump using the -c option which also compresses the dng lossless. Nothing against slimraw(really cool and fast) but you wouldn't need to go through mlvfs.
I wonder how batch_mlv will work on a humongous mlv file.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: clanlee on April 14, 2018, 01:08:07 AM
I figured out the problem with the web gui not working.  I looked thru a bunch of post and saw this one:

Quote from: bouncyball on April 27, 2017, 10:00:10 AM
Don't know about resolve issue but for correctly display settings web page you absolutly have to change dir (cd) to MLVFS folder when running it from the command line. Otherwise mlvfs can not find needed files to display page corectly. Also it can not find focus pixel maps.

I'll leave the post of just in case someone else needs the information.  You must change directory to the mlvfs.exe or mlvfs_x64.exe directory to get the web gui working.


---Original post starts here---

I realized that my setup works great and I can access the dng files via windows explorer but the web interface isn't working. 

Using:
C:\mlvfs_x64\mlvfs_x64.exe Z:\ --mlv-dir=C:\Videos\Files


I wasn't getting any error messages or able to stop the drive.  Instead, I added -f and I was able to get error messages and by pressing CTRL + C, it would umount the drive:
C:\mlvfs_x64\mlvfs_x64.exe -f Z:\ --mlv-dir=C:\Videos\Files


I'm getting the following message when attempting to access the web gui (localhost:8000):

webgui: scanning C:\Videos\Files/...
load_resource: fopen error
webgui: scanning C:\Videos\Files/favicon.ico...
load_resource: fopen error


Any thoughts as to why web gui stopped working?

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on April 18, 2018, 12:28:14 PM
Quote from: Danne on April 13, 2018, 04:38:15 PM
Wow, that's a lot of dng files :).
It sounds about right that indexing would take this long. I'm even surprised you got all working in the end.
I would also suggest exporting to dng files directly through mlv_dump using the -c option which also compresses the dng lossless. Nothing against slimraw(really cool and fast) but you wouldn't need to go through mlvfs.
I wonder how batch_mlv will work on a humongous mlv file.

After checking all the files from the live event where we filmed ~20h 12bit lossless raw there is one file that's only outputting 50K frames when It should have 80K+ in it. I know this because the filesize and number of frames that I get out of It does not add up. I can also see the missing gap in the timeline I made In premiere with all cameras synced up from the event.
I guess It's a metadata corruption where the file thinks it only have 50K frames in it even though there is much more data there. Does anyone have any experience with this kind of problem? Is it possible to get the missing files out of the mlv or is this a lost cause?

Got the same amount of frames both from MLVFS and mlv dump/mlv batch, both missing 30K frames...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: D_Odell on April 21, 2018, 12:36:00 AM
Hi,

So I tried this evening with a new project, haven't filmed for a while. Now on a new computer, slower, but newer. Running latest MLVFS and Davinci 14.2. When I start Davinci MLVFS stops working, all files goes offline meanwhile Davincis running. When I reload MLVFS it works, then power Davinci and it goes offline.. Anyone experienced this? I can't seem to get around it? Do I need to use another tool than MLVFS?

MacOS Sierra 10.12.6
MLVs is compressed and normal MLVs.

Thanks a bunch!
David
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on April 23, 2018, 01:59:44 PM
@D_Odell
I always convert mounted dngs with slimraw and put them on disk. Working directly with mounted dngs isn't stable with MLVFS on Windows either. It was pretty stable and faster with pismo but no support except standard 14bit raw there and mlvfs has all metadata handling working correctly.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: D_Odell on April 24, 2018, 01:51:33 PM
Quote from: Markus on April 23, 2018, 01:59:44 PM
@D_Odell
I always convert mounted dngs with slimraw and put them on disk. Working directly with mounted dngs isn't stable with MLVFS on Windows either. It was pretty stable and faster with pismo but no support except standard 14bit raw there and mlvfs has all metadata handling working correctly.
Thanks for replying. I see the problem being much less hassle with converted DNGs of course, but since I really love MLVFS and virtual DNGs its a blast for storage and low budgets. So the solution to get it fixed again, it wasn't MLVFS but Davinci 14... Going back to 12.55 it works like a dream again.

Weird..

Best,
David
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ArcziPL on June 08, 2018, 10:55:58 PM
Quote from: clanlee on April 04, 2018, 11:36:34 PM
MLVFS_x64
Download http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip)
Extract MLVFS_x64.zip

Hi, is there a newer windows build available, supporting MLV with lossless JPEG compression? Tried already several builds, out of which this one seems to be the newest one, but only uncompressed MLV are opening properly.


EDIT:
huh, here it is...

https://www.magiclantern.fm/forum/index.php?topic=19429.0

https://bitbucket.org/bouncyball/mlv-tools/downloads/

Thank you, bouncyball!


All in all it's a fantastic tool! But all the docus are totally outdated...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on June 13, 2018, 09:12:15 AM
hey can someone help me?my mlvfs isnt working. i do the correct command lines and it just takes me back to the root folder like nothing ever happened please help. it worked a couple months ago but now im just coming back to it and it doesnt work.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: sys73r on June 14, 2018, 01:08:55 AM
hey guys, I have been using for a long time the stable build recording 1080p with sound for long time and post processing with the latest MLVFS without any issues.

Today I'm testing the last build crop_rec4k.2018Mar10 with mlv_lite and mlv_snd to try  to record 3.5K with sound, It records fine but when I post process the image I see no sound at all. So think it's because the version of MLVFS I have is no longer able to grab the data from the latest files.

is there any new version or patch I can install for OSX to keep using MLVFS?


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: bouncyball on June 14, 2018, 06:12:27 PM
Here (https://www.magiclantern.fm/forum/index.php?topic=19429.0)

However if you use as you said latest MLVFS this build (crop_rec4k.2018Mar10) does something different.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: freely on June 20, 2018, 03:55:00 PM
Hello.

It seems trivial but I found something a little cumbersome for me. If you select Naming Scheme as DaVinci Resolve in MLVFS, "MLV" is inserted in the middle of the file name.

(https://s19.postimg.cc/pu0m7zyjn/Screen_Shot_2018-06-20_at_10.36.56_PM.png) (https://postimages.org/)

(https://s19.postimg.cc/oez1jbuwj/Screen_Shot_2018-06-20_at_10.21.56_PM.png) (https://postimages.org/)

It is not included in other conversion programs (MLV App, Switch).
(https://s19.postimg.cc/9j0ibqr7n/Screen_Shot_2018-06-20_at_10.22.25_PM.png) (https://postimages.org/)


Here is my mlv file.
https://drive.google.com/open?id=1GqMSKjZ-EqFx3KaJG6XzJ2zBs6AMFYS-

I used https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS.dmg

I'm not sure, but I think this is a bug in MLVFS, what's your opinion? If this is a bug, I hope it is fixed.

Thank you.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 20, 2018, 04:49:16 PM
It´s not a bug. Bouncyball once helped me find how to exclude it. Somewhere along these lines. FIddle with it and recompile MLVFS and it should work:

Something like following. Can´t remember exactly:
In main.c changed to this to erase the MLV part from the name to match Resolve naming scheme with other cams for instance

sprintf(*mlv_basename, "%s_1_%d-%02d-%02d_%04d_C%04d", start, 1900 + frame_headers.rtci_hdr.tm_year, frame_headers.rtci_hdr.tm_mon + 1, frame_headers.rtci_hdr.tm_mday, 1, 0);

....

static int get_mlv_name_from_basename(const char *path, char ** mlv_name)
{
    if(mlvfs.name_scheme == 1)
    {
        struct slre_cap caps[2];
        memset(caps, 0, sizeof(caps));
        if(slre_match("(.+)_1_\\d+-\\d+-\\d+_\\d+_[C|c]\\d+", path, (int)strlen(path), caps, 1, 0) >= 0)
        {
            char * result = get_capture(caps[0]);
            *mlv_name = concat_string(result, ".MLV");
            free(result);
            return 1;
        }
        else
        {
            return 0;
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: freely on June 21, 2018, 09:30:25 AM
Thank you.
Thanks to the guide, I was able to get what I wanted. Thank you.

In addition, I would like to tell you what I have learned and what I have thought about in this process.
I did a search on Google with the DaVinci Resolve Naming Scheme. So I found something like the following on the following page.
https://forum.blackmagicdesign.com/viewtopic.php?f=2&t=32836

(https://s19.postimg.cc/v7m2tjgg3/Screen_Shot_2018-06-21_at_1.54.17_PM.png) (https://postimages.org/)

You can see that this is named CAMERA ID / Date / Time / Clip #. After knowing this, there is one question to be asked. For example, it has the name M13-1450_1_2018-06-13_0001_C0000. Why do MLVFS, MLV App, and Switch all display time information in clip name as 0001? Personally, it is more intuitive to display the time information than the meaningless information of 0001, so I think that there will be some profit because I can know the shooting time.

I tried to modify some of the code as a test and changed the name like M13-1450_1_2018-06-13_1450_C0000.

sprintf(*mlv_basename, "%s_1_%d-%02d-%02d_%02d%02d_C%04d", start, 1900 + frame_headers.rtci_hdr.tm_year, frame_headers.rtci_hdr.tm_mon + 1, frame_headers.rtci_hdr.tm_mday, frame_headers.rtci_hdr.tm_hour,  frame_headers.rtci_hdr.tm_min, 0);


(https://s19.postimg.cc/edgvoas2r/Screen_Shot_2018-06-21_at_3.22.47_PM.png) (https://postimages.org/)


I think the naming scheme is more useful for displaying time information. Then I thought that "_1" after the camera "M13-1450" had no meaning, so I thought it seemed unnecessary.

I tried it and found that it was possible.
(https://s19.postimg.cc/4zfjibe4z/Screen_Shot_2018-06-21_at_4.08.13_PM.png) (https://postimages.org/)


For your reference, let me tell you why I want to remove "MLV" from the clip name. My system will drop frames when playing clips loaded with MLVFS. So I have to convert it to dng for smooth editing. When I am done editing, the dngs are deleted, and sometimes I need to make some changes later. At this time, it is too time-consuming and energy-intensive to convert all back to dng, so I want to load it into MLVFS instead. At this time, if the clip names are the same, I can simply continue the work by specifying a new folder.

Thank you for your cooperation.



Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on June 22, 2018, 06:56:51 AM
Or maybe it could simply be:
M13-1450_2018-06-13_C0000_000000
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: papasapien on June 26, 2018, 11:20:12 PM
do i need windows 10 professional Intel virtualization technology to run mlfvs on windows?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: briwil on August 26, 2018, 09:31:44 AM
Quote from: clanlee on April 04, 2018, 11:36:34 PM
MLVFS Installation on Windows 10 x64

Install latest version of Dokany:
https://github.com/dokan-dev/dokany/releases (https://github.com/dokan-dev/dokany/releases)

Reboot if necessary.

MLVFS_x64
Download http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip (http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip)
Extract MLVFS_x64.zip

Delete the following files from the extracted folder:
dokan1.dll
dokanfuse1.dll

Using MLVFS:
Note the path of the extracted MLVFS_x64.exe
Open Command Prompt

Run program from within directory into the executable:
The web gui interface only works if you call the executable from the same directory
cd mlvfs_x64
mlvfs_x64.exe Z:\ --mlv-dir=C:\Videos\Files

Run from any location by typing full path to executable:
If you run this from a remote directory, the web gui will not work
C:\mlvfs_x64\mlvfs_x64.exe Z:\ --mlv-dir=C:\Videos\Files

If your folder name contains spaces, use quotation marks for path:
"C:\Extracted files\mlvfs_x64\mlvfs_x64.exe" Z:\ --mlv-dir="C:\Videos from Camera\Files"

Or:
C:\mlvfs_x64\mlvfs_x64.exe Z:\ --mlv-dir="C:\Videos from Camera\Files"

Or:
"C:\Extracted files\mlvfs_x64\mlvfs_x64.exe" Z:\ --mlv-dir=C:\Videos\Files

Notes:
The Z:\ option is setting a mount point.  You'll have to use an unused drive letter in your computer.  When you open Windows Explorer, you'll see a list of all the drives in your computer.  Use a drive letter that is not used.

I've done all the steps here, and everything seems to work, except all my DNG's end up looking like pink noise, even if I open in Photoshop. What am I missing here?
On Windows 10, btw.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: clanlee on August 27, 2018, 09:40:09 AM
Try the x86 executable.

https://bitbucket.org/bouncyball/mlv-tools/downloads/

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: henricusmaria on January 04, 2019, 09:11:51 AM
I just started using MLVFS. How can I remove the pink focus dots? I work on a mac.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on January 04, 2019, 11:53:21 PM
Quote from: henricusmaria on January 04, 2019, 09:11:51 AM
How can I remove the pink focus dots? I work on a mac.

You shouldn't have to do anything, it is automatic in MLVFS. There are some new resolutions in the experimental branches and there are a few new focus pixel map files that haven't been merged in yet. You can download the latest map files from here:

https://bitbucket.org/daniel_fort/ml-focus-pixels/src/default/focus_pixel_map_files/

Yeah, a little geeky but you need to know your camera ID and put them in the MLVFS application. You can also make your own map files (https://www.magiclantern.fm/forum/index.php?topic=16054.msg207474#msg207474) if you are working on bleeding edge builds.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: DeafEyeJedi on January 07, 2019, 08:07:06 AM
Quote from: Markus on April 18, 2018, 12:28:14 PM
...I guess It's a metadata corruption where the file thinks it only have 50K frames in it even though there is much more data there. Does anyone have any experience with this kind of problem? Is it possible to get the missing files out of the mlv or is this a lost cause?

Got the same amount of frames both from MLVFS and mlv dump/mlv batch, both missing 30K frames...

Unfortunately I had this similar phenomenal happened (though it had nothing to do w MLVFS) during a full day shooting at a wedding last year where I shot 12-bit lossless during the entire event. I decided to give up on trying to back track it and managed to make it work with the rest of footage that came out well except for one humongous MLV file!  :P

The only thing I could think of was by using the external monitor while plugged in via HDMI (5D3 running 123) may have been the culprit in here and was this a similar set-up you had on your end?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: henricusmaria on January 07, 2019, 09:20:36 AM
Quote from: dfort on January 04, 2019, 11:53:21 PM
You shouldn't have to do anything, it is automatic in MLVFS. There are some new resolutions in the experimental branches and there are a few new focus pixel map files that haven't been merged in yet. You can download the latest map files from here:

https://bitbucket.org/daniel_fort/ml-focus-pixels/src/default/focus_pixel_map_files/

Yeah, a little geeky but you need to know your camera ID and put them in the MLVFS application. You can also make your own map files (https://www.magiclantern.fm/forum/index.php?topic=16054.msg207474#msg207474) if you are working on bleeding edge builds.

Thanks, I will look into it. Meanwhile I've exported some mlv clips through MLVapp as prores and encountered another problem.
I see a lot of color artifacts in the footage, especially in the horizontal lines. I've added an example. What can be done about this? I shoot with the following settings: 3x3_mv1080_EOSM, 1080p24, 1736x1120 1:2 14bit lossless with the sd card hack.


(https://i.ibb.co/3cCZKDb/Screen-Shot-2019-01-07-at-16-14-35.png) (https://ibb.co/3cCZKDb)


Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 07, 2019, 09:24:09 AM
Can you upload a sample of the mlv file you can cut it in Mlv App.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: henricusmaria on January 07, 2019, 03:58:51 PM
https://drive.google.com/file/d/1HRp-SPyHb6LZkc6Vdh3tKOnqEXqyhOjI/view?usp=drivesdk
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 07, 2019, 04:08:05 PM
Aliasing. To be expected under certain circumstances.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: henricusmaria on January 08, 2019, 12:46:04 AM
Yes, but it's happening way too much.. Can I change something to the ML settings to prevent or lessen this?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on January 08, 2019, 01:42:31 AM
Quote from: henricusmaria on January 08, 2019, 12:46:04 AM
Can I change something to the ML settings to prevent or lessen this?

First of all this was shot in 5x3 sampling, that will give the most amount of aliasing.

      sampling        5x3 (read 1 line, skip 4, bin 3 columns)


Try shooting in 3x3 or even better in a 1x1 sampling setting. I see you're using the EOSM so try Danne's latest experimental builds to get those resolutions.

Another thing you can do is turn on some chroma smoothing and see if that helps.

(https://farm8.staticflickr.com/7855/46602819302_0da6260dd5.jpg) (https://flic.kr/p/2e18Cyw)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: henricusmaria on January 08, 2019, 02:50:58 AM
Thanks! I was under the impression that when I chose the 3x3_mv1080_EOSM mode I was shooting in 3x3 sampling.
I have tried the new experimental builds before. The mv1080p options still gave me aliasing problems. 1:1 sampling is fine. But I wanted to shoot without the crop factor. This is why I tried an older build that is recommended by this guy on youtube (Zeek). He doesn't have aliasing issues. I guess it also depends on the lens you use. With the Canon 18-55 lens the aliasing artifacts are more prominent at the wide angle.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on January 08, 2019, 03:44:58 AM
Quote from: henricusmaria on January 08, 2019, 02:50:58 AM
I guess it also depends on the lens you use. With the Canon 18-55 lens the aliasing artifacts are more prominent at the wide angle.

Really? That doesn't sound right. Aliasing is more apparent in high contrast boundary areas of diagonal lines, like your roof lines or power lines. So maybe a soft focus, low contrast lens will show less artifacts.

I looked up Zeek on YouTube and he is using an old C-mount "TV" lens on his camera. Probably not very sharp. He's also using a cheap ebay speed booster, again this is probably taking a hit in sharpness and contrast.

(https://farm8.staticflickr.com/7907/31715079387_e70ab7390f.jpg) (https://flic.kr/p/Qjy5az)

Quote from: henricusmaria on January 08, 2019, 02:50:58 AM
...The mv1080p options still gave me aliasing problems....I tried an older build...

Try Danne's latest builds, you should be getting true 3x3 sampling.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: henricusmaria on January 08, 2019, 04:17:02 AM
What doesn't sound right? Maybe the 18-55 lens has less contrast in the tele region, therefore less artifacts. Zeek also uses m42 and ef lenses with apparently no aliasing. But the speed booster could have something to do with it, and of course the fact that he is shooting at longer focal lengths with wide apertures.
I am back on Danne's latest build now. 1:1 looks great, (mv1080p) still has aliasing. I will do some tests with the 2.5 k setting.
Thanks again!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dfort on January 08, 2019, 07:56:01 AM
Took a closer look at your sample, maybe it is 3x3 and reported wrong? Seems like every app I used stretched or squashed it to a different aspect ratio.

In any case, it does look pretty good. If you post it on YouTube it will probably blur out some of that aliasing.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: henricusmaria on January 08, 2019, 02:07:56 PM
Yeah.. i did some tests today with the 2.5k setting in 1920x1080 10bit, so it was possible to shoot long takes. It looks beautiful!
For some reason though after 2 successful exports, the rest of the exported prores clips still have the focus points. These were done under the same settings in mlvapp. Any idea how this can happen?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 08, 2019, 02:39:43 PM
Best is if you upload the problematic file. Shortened.

By the way. YOu can do higher resolutions if running 2.35:1. Here´s a nice setting I use:
(https://i.postimg.cc/ZqpzJsy0/VRAM0-PPM-scaled.png)

(https://i.postimg.cc/YCJwKB0J/VRAM1-PPM-scaled.png)

(https://i.postimg.cc/3xyHcx8X/VRAM2-PPM-scaled.png)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: henricusmaria on January 08, 2019, 03:13:33 PM
Thanks Danne, will try it out tomorrow!

Now all the exported files have the focus points in them. Here is a short mlv file:

https://drive.google.com/file/d/1EeeE05vqRVAGBZK1zh28XbLVFNe_YKbx/view?usp=drivesdk

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 08, 2019, 03:40:56 PM
Seems to be working here. Did you accidentally turn fix focus dots off?
Off
(https://i.postimg.cc/wM2ZwkFj/Screenshot-2019-01-08-at-15-36-57-png-scaled.png)

On
(https://i.postimg.cc/LXzr1S1f/Screenshot-2019-01-08-at-15-37-04-png-scaled.png)

dfort´s maps doing the trick.
You could also add chroma smooth 3x3 which is very effective in eliminating dots.

Example using only cs3x3:
(https://i.postimg.cc/9QCtmhWK/Screenshot-2019-01-08-at-15-42-10-png-scaled.png)

(https://i.postimg.cc/DyqP3ZK1/Screenshot-2019-01-08-at-15-42-16-png-scaled.png)

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: henricusmaria on January 08, 2019, 04:21:10 PM
Fix focus dots is on. Switching from on to off does not make a difference anymore. With the first 2 exports it worked like a charm. Strange..
Luckily the chroma smooth 3x3 works effective indeed! I thought it would degrade the quality, but it seems to be the same. Will use this then for now. Maybe my computer needs a rest ;) Hopefully tomorrow it is working like normal again. Thanks for the help Danne, much appreciated!

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: planetary on January 23, 2019, 08:44:32 PM
I've been trying to get this working the past 14 hours. My mlvfs stopped working last week due to my computer rolling back somehow after a consistent 3-4 years without a problem. I wasn't sure exactly what changed so I reinstalled mlvfs and dokan. I was using 0.8 dokan before.

The problem I have now is any time I preview mlv dng's, it shows pink static (on the WHOLE screen, completely unusable).
When I run mlvfs, the virtual mount is there however the file just shows pink static in resolve.
They appear fine in MLV APP viewer and I've worked with these same mlv's before so I know it's not the files.
I've tried to install and reinstall dokan and mlvfs_x86 but nothing's really helping.

I've tried localhost and its the same thing there. Am I missing a codec? what am I forgetting here?
If I remember correctly, doesn't mlvfs automatically convert mlv's from the virtual drive?
Any help would be greatly appreciated. T_T
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on January 23, 2019, 09:47:37 PM
If you say your PC rolled back and you don't know what things were touched: Why not re-installing Resolve as well?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: planetary on January 25, 2019, 09:12:02 AM
@walter shulz I've reinstalled windows 7 and davinci. The problem is still the same.
I'm guessing it's the dokan version. I'm currently using .8.

I'm having errors where it says it can't find dokanfuse.dll with other versions.

Anyone know if I need any specific drivers or such for davinci resolve that may have gotten rolled back?
Anything like that? OR any codes for dng's or anything? I'm at a complete loss.
I'm not sure what to do and I have a huge headache. :(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: planetary on January 25, 2019, 10:04:51 AM
Also MLV APP can open these files but not MLVFS or dokan?? What am I missing here???
Besides dokan and mlvfs_x86, what other scripts or codecs do you need to view?
The preview from these dngs from the virtual drive are all pink as well so I'm guessing
something is not being applied or rendered during. Any idea what this could be?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on January 25, 2019, 11:28:15 AM
Don't have MLVFS running right now. Have to build a W7 environment (no lack of machines, though). Will take some time.
Anyone with ideas/knowledge able to jump in?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ArcziPL on January 25, 2019, 12:40:28 PM
Can you try with non-compressed MLV? You might have an old MLVFS version, without support for lossless compression. You also don't need to install Dokan or anything else. Just get the right binary of MLVFS with correct DLLs provided.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: planetary on January 25, 2019, 06:27:34 PM
Quote from: ArcziPL on January 25, 2019, 12:40:28 PM
Can you try with non-compressed MLV? You might have an old MLVFS version, without support for lossless compression. You also don't need to install Dokan or anything else. Just get the right binary of MLVFS with correct DLLs provided.

Are there any dlls/insfrastructure that I need to run mlvfs besides the files specified in this thread?
I'm not sure what you mean by non-compressed.
Any idea which dll's I should get and where I should put them?
Sorry I'm not very good with the technicals :(

I've spent the last 48 hours scouring the forum for answers and similar cases.
Can someone please simply detail where/what I'm downloading and where I should put them please?
I'm stumped :(
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ArcziPL on January 25, 2019, 10:54:09 PM
I can post a working version in 2 days, when I'm back home.

What I mean with uncompressed: record a MLV with selected "14-bit" (or 12, 10...) format in Magic Lantern menu for RAW video and NOT "14-bit lossless". It's just for trying it out, to nail the problem down. Normally you can/should use the lossless compression.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: planetary on January 26, 2019, 10:32:54 AM
Quote from: ArcziPL on January 25, 2019, 12:40:28 PM
Can you try with non-compressed MLV? You might have an old MLVFS version, without support for lossless compression. You also don't need to install Dokan or anything else. Just get the right binary of MLVFS with correct DLLs provided.

Oh my gosh. You are a genous Arczi.
I recorded uncompressed and the image came through so it must be an error involved with shooting lossless!!
Thank you so much. I was tearing my hair out and stressing out.
Any idea what I can do to get lossless working again? 
I'm using mlvfs_x64 at the moment.

Which mlvfs supports lossless compression?
Thank you again!!!!!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: planetary on January 26, 2019, 10:58:31 AM
I JUST SOLVED THE LOSSLESS COMPRESSION ISSUE TOO!!!!

I downloaded mlvfs_winx86.zip and used mlvfz.exe and put dokanfuse1.dll into my program/x86 
https://bitbucket.org/bouncyball/mlv-tools/downloads/

Ran it in the cmd under mlvfs_x64 and everything is good now!!!
Thank you guys!!!!!!!! This solved the pink static screen.
Hopefully this helps anyone else with the same issue as this.

Using dokan 1.0.3.1000.

GOOD NIGHT GUYS!! TIME TO REST AND DESTRESS
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ArcziPL on January 30, 2019, 11:30:46 PM
Great that you found the right file. As the question is recurring and documentation of this great tool outdated, I post here a compilation of both 32 and 64-bit version of mlvfs already supporting lossless compression + required DLLs, so one doesn't need to install anything, just use it as is:

https://bitbucket.org/ArcziPL/magic-lantern/downloads/mlvfs_x86_x64.zip

All inside this package is found compiled on some online repositiories of users of this forum. I just put it all together in one place.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: blankester on February 11, 2019, 07:20:48 PM
Hi guys
Amazing converter, really saves time and space. Apologies for the newb question but sometimes the files mount ok on the virtual directory and sometimes they don't. I have tried opening from different drives, and always open to an empty folder on my desktop as this has worked before. My folders contain only .mlv files and .mrx files. Some folders open fine and I can edit in Resolve, others I can't get to open. The safari window pops up and the drive mounts, it's just totally empty. Any ideas would be much appreciated.
Thanks
Barney
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: aulden on April 30, 2019, 11:52:30 AM
Is it possible to alter the "Bad pixel fix" so that it targets ALL the frames and not just the first one?
Title: Mlvfs can it fix Eos m focus dots
Post by: berechiah on May 21, 2019, 08:46:31 PM
Hi I used a 5d and put the mlv files in mlvfs this was the best way for me

now I am using a Eos m
but the the mlv app there is a setting to remove the focus dots
but the focus dots are visible in the mlvfs way

is there a way to update mlvfs to include this?
I really miss this mlvfs way of ding things

can any one help

thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: masc on May 22, 2019, 12:18:57 PM
@berechiah: Here is explained what to do: https://www.magiclantern.fm/forum/index.php?topic=16054.msg215358#msg215358
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: berechiah on May 22, 2019, 10:37:12 PM
Quote from: masc on May 22, 2019, 12:18:57 PM
@berechiah: Here is explained what to do: https://www.magiclantern.fm/forum/index.php?topic=16054.msg215358#msg215358

Thanks for the info
It is perfect! thanks for the reply I means a lot!

I used my 5d for a while and was so upset I could not use the mlvfs due to the focus dots and thaught the only way was mlvapp (mlvapp is fine by the way but mlvfs easy to work with)

Also some of the info is so complex to do but this was good. I am Kida tec savy but this kind of stuff goes right over my head

Thanks masc

much appreciated
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Jip-Hop on May 28, 2019, 05:15:00 PM
I got static noise too. So downloaded the files from https://bitbucket.org/ArcziPL/magic-lantern/downloads/mlvfs_x86_x64.zip. But can't get it to work with the same command which worked before (but resulted in footage with just noise). I tried "mlvfs_x64_lossless.exe mount/path/ --mlv_dir=path/to/mlv/folder" and "mlvfz.exe mount/path/ --mlv_dir=path/to/mlv/folder". But both just return to the command line immediately. Isn't this the correct command any longer?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Jip-Hop on May 29, 2019, 04:51:51 PM
I followed the instructions from clanlee. https://www.magiclantern.fm/forum/index.php?topic=13152.msg199351#msg199351 But this version doesn't work with compressed MLV files: http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/MLVFS_x64.zip So I trashed the mlvfs_x64_avx.exe and mlvfs_x64.exe files and instead put the mlvfs.exe from bouncyball in that folder: https://bitbucket.org/bouncyball/mlv-tools/downloads/mlvfs.exe.

Seems to be working now with the following command:


mlvfs.exe Z:\ --mlv-dir=D:\Files\ML --resolve-naming

I'm using installed Dokan_x64.msi version 1.2.2.1000.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Jip-Hop on June 07, 2019, 06:01:36 PM
I've been using mlvfs like described in my previous post with Davinci Resolve 16 Beta 3 on Windows 10 and it was working fine until today.
Edited and exported a video, all good, but today I wanted to make a small change so I opened the project again but it seems mlvfs is crashing.

So I run mlvfs with the -d option to see debug log output.

Before I open resolve, the previews of the clips load fine in the web gui.
When I open the Resolve project it shows the thumbnails of all the clips on the timeline.
When I start playback, I can get a couple of frames and then it locks up.

Mlvfs doesn't output anything special on the command line.
Last line logged was "ReadFile: \path\to\dng from 4096 len 4096"

In the file explorer the mounted drive becomes unavailable.
I get this message: "Z:\ is not accessible. Insufficient system resources exist to complete the requested service."

localhost:8000 won't load any longer.

Then Resolve says "Media Offline".

I didn't install anything in the mean time...
But I tried installing the new Davinci Resolve 16 Beta 4, which didn't make a difference.

Finally when I terminate mlvfs in the command line it says "send global release for Z:".
So it seems mlvfs doesn't crash completely...

Is it still working for you?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marcelolippi on June 10, 2019, 06:07:45 PM
Hi, mlvfs work or not? Shelved o updated I see many old post here. I need to use eosm and 5dm2 4.5k with focus pixels and 3k of reddercity build and Davinci resolve 15. Thanks for any idea.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Jip-Hop on June 10, 2019, 06:31:17 PM
On which OS do you want to use MLVFS?

For me it's not working on Windows any more. I decided to copy all of the DNG folders to the hard drive, while the MLV files were mounted using MLVFS. Basically converting MLV to DNG with MLVFS. Just so I get the correct directory structure so I can open my Davinci Resolve project again.

But MLVFS stopped working again. It 'converted' 25% of the files and then crashed. This was done without opening Davinci Resolve, so seems like my issues are not related to Davinci Resolve.

(https://i.ibb.co/qYgtFCM/IMG-4753.png) (https://ibb.co/RT6x4Np)
(https://i.ibb.co/hm7kLx6/IMG-4752.jpg) (https://ibb.co/MC2bZyv)
afbeelidingen (https://nl.imgbb.com/)

So now I'm using MLV App to batch convert the MLV files to DNG so I can at least open my project again in Resolve.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: TKez on June 19, 2019, 03:30:46 AM
Quite happy with MLVFS > DaVinci workflow these days, but just seeing a lot of talk here about updates to the Windows version.
There's been a lot of progress lately with bit-depth and lossless on the 70D. Does the current MacOS version have everything needed to handle these formats correctly? I notice no updates since July 2018.

I'm guessing not as they appear as 0 frames with no info.

Is there any kind of working Mac version of this around now, or are we forced to use a different workflow with lossless compression on MacOS?

I took a quick look at the source, but the C stuff is a bit beyond me.
Be awesome to have this workflow rolling again as I don't think we can turn back from lossless now we have it.

thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Jip-Hop on July 01, 2019, 08:37:26 PM
Quote from: freely on June 21, 2018, 09:30:25 AM
Thanks to the guide, I was able to get what I wanted. Thank you.
[...]
For your reference, let me tell you why I want to remove "MLV" from the clip name.
[...]

@Danne & Freely, do any of you mind sharing a compiled version of mlvfs for macOS with the MLV part removed from the name? Would be great if the naming would be compatible with MLV App.

@TequilaKez
I just installed mlvfs on macOS and it works with the 14bit lossless compressed MLV files from my EOS M.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on July 04, 2019, 08:53:39 AM
Could you test this version:
https://bitbucket.org/Dannephoto/mlvfs_minus_mlv/downloads/MLVFS_minus_MLV.dmg

Code:
https://bitbucket.org/Dannephoto/mlvfs_minus_mlv/src/master/
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Jip-Hop on July 04, 2019, 03:36:01 PM
Cool! Thanks. Now I can quickly review footage in DaVinci Resolve with MLVFS and then switch to exported DNG from MLV App later :)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: marcelolippi on July 08, 2019, 09:09:34 AM
 I think will be start with the mlvfs on windows 10-resolve...it work on win10? I read difficulty in lot opinions.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: berechiah on July 22, 2019, 11:56:24 PM
Quote from: masc on May 22, 2019, 12:18:57 PM
@berechiah: Here is explained what to do: https://www.magiclantern.fm/forum/index.php?topic=16054.msg215358#msg215358

Thanks
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: adrjork on August 07, 2019, 09:04:46 AM
Sorry if this question isn't new (I confess I've read 69 pages VERY rapidly, and all seems in progress still now):
Does MLVFS' on-the-fly DNGs slowdown a bit Davinci? Currently, do you actually recommend a deep color grading stable workflow with on-the-fly DNGs? Or is it still better (for quality of the grading and speed) working with rendered DNGs?

Thanks a lot.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: masc on August 07, 2019, 10:25:52 AM
MLVFS vs rendered DNG is identical in terms of quality. For performance you will get different results for different systems, so you only can try it out with your system.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dariSSight on August 17, 2019, 09:16:42 AM
What is the latest Version of MLVFS and How do I know I'm running latest version? I'm using MLVFS in DaVinci Studio 16, sometimes when I open project it changes name of clips and I can't relink files to project. Please let me know what's the most stable workflow with MLVFS.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 2blackbar on September 04, 2019, 12:08:15 AM
mlvfs.exe Z:\ --mlv-dir=H:\DCIM\100CANON\
Finally got it working with lossless, great idea
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: andy kh on October 28, 2019, 01:36:50 PM
i stil prefer to use mlvfs with pismo mount but mlv lite is not supported. i think the problem is with mlvfs dll. can anyone help me out
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: TrEK on October 30, 2019, 08:21:56 PM
Hello all!
Who know what is last version of MLVFS for WINDOWS ?
and where i can download latest correct version ?

(//)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eyeland on July 27, 2020, 02:23:08 PM
Quote from: ArcziPL on January 30, 2019, 11:30:46 PM
Great that you found the right file. As the question is recurring and documentation of this great tool outdated, I post here a compilation of both 32 and 64-bit version of mlvfs already supporting lossless compression + required DLLs, so one doesn't need to install anything, just use it as is:

https://bitbucket.org/ArcziPL/magic-lantern/downloads/mlvfs_x86_x64.zip

All inside this package is found compiled on some online repositiories of users of this forum. I just put it all together in one place.
quick necro, what version of Visual Studio are you using?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: eyeland on July 27, 2020, 03:13:04 PM
not quite sure what I am doing wrong with the windows version.
When using the version of MLVFS.exe linked in the OP, I get the pure noise output as others have experienced (probably from using the lossless 12b it compression)
When I try the fixed described above (to put mlvfz.exe in the mlv folder and run it) I get a "missing dokanfuse1.dll" msg.
When I copy the missing file into the folder and run mlvfz again, CMD returns to prompt and nothing happens.

EDIT:
Using the latest dokany & visual studio & the folders linked below, I have everything working in x64 :) Yay, thanks!

Quote from: ArcziPL on January 30, 2019, 11:30:46 PM
Great that you found the right file. As the question is recurring and documentation of this great tool outdated, I post here a compilation of both 32 and 64-bit version of mlvfs already supporting lossless compression + required DLLs, so one doesn't need to install anything, just use it as is:
https://bitbucket.org/ArcziPL/magic-lantern/downloads/mlvfs_x86_x64.zip
All inside this package is found compiled on some online repositiories of users of this forum. I just put it all together in one place.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: pc_b on October 19, 2020, 11:53:41 AM
Hello everybody,
I'm just trying to download de latest version of MLVFS for Windows, but all the bitbucket.org links doesn't work... Is there any other place where I can download it?

Thank you very much.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on October 19, 2020, 01:36:34 PM
Hell of a string (https://web.archive.org/web/20200621151431/https://bbuseruploads.s3.amazonaws.com/3f5144c4-ac62-40a5-9661-de9d52f6bf3c/downloads/9b3d97e2-58cc-4241-aabc-283ce1f744a4/mlvfs_x86_x64.zip?Signature=eoUlJgU9XryvJitmPBjZpOvdt%2Fg%3D&Expires=1592754271&AWSAccessKeyId=AKIA6KOSE3BNJRRFUUX6&versionId=BOXOMU0sZdnsEseC6PqUa85j55aFy1gJ&response-content-disposition=attachment%3B%20filename%3D%22mlvfs_x86_x64.zip%22)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: juliusE on October 20, 2020, 11:09:38 AM
hi folks
I know it may all be datet,
But I'm using The mlvfs on mac, working perfectly, except theres lots of dots when I open the files in davinci, which are not there when I export a cDNg from mlvapp.
I tried taking the Fmp files from mlvapp contents and putting into mlvfs , but this did not seem to do a thing..
Please help
:)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Boygetslost on January 09, 2021, 02:49:10 PM
I used to use MLVFS a few years ago. I am now trying Magic lantern again. Though currently I am using MLV APP.

I always preferred the simplicity of MLVFS.

With the 14bit raw from the 5d 2 and 3, does the MLV to CDNG conversation have the same results in image quality? Or does one program have the edge on this?

Any thoughts from seasoned users?


Thanks

Chris

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 2blackbar on January 10, 2021, 02:30:12 AM
imo 5d2 frim mlvapp has crap greenyellow tint to the colors and skintones if you do color work in it unless you fix it with hsl .
so unless you calibrwte that i recommend not letting mlvapp debayer dngs.simple white balance  wont fix this.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: masc on January 10, 2021, 11:32:10 AM
Conversion from MLV to CDNG has in both MLVFS and MLVApp the same image quality, because CDNG is still RAW. If possible, I would prefere MLVFS, because you don't need time and diskspace for conversion.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Dmytro_ua on January 10, 2021, 12:05:59 PM
Quote from: Walter Schulz on October 19, 2020, 01:36:34 PM
Hell of a string (https://web.archive.org/web/20200621151431/https://bbuseruploads.s3.amazonaws.com/3f5144c4-ac62-40a5-9661-de9d52f6bf3c/downloads/9b3d97e2-58cc-4241-aabc-283ce1f744a4/mlvfs_x86_x64.zip?Signature=eoUlJgU9XryvJitmPBjZpOvdt%2Fg%3D&Expires=1592754271&AWSAccessKeyId=AKIA6KOSE3BNJRRFUUX6&versionId=BOXOMU0sZdnsEseC6PqUa85j55aFy1gJ&response-content-disposition=attachment%3B%20filename%3D%22mlvfs_x86_x64.zip%22)

On a Windows version in 64 folder I get an error about dokanfuse1.dll. I see dokanfuse.dll file in a folder, but not dokanfuse1.dll

(https://i.ibb.co/Lxtn10s/Screenshot-2021-01-10-125909.jpg) (https://ibb.co/rx64HQX)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on January 10, 2021, 12:22:14 PM
Try 32-bit version.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Dmytro_ua on January 10, 2021, 12:24:15 PM
Quote from: Walter Schulz on January 10, 2021, 12:22:14 PM
Try 32-bit version.

Already did.
Does nothing at all.


And the stupid method: just copy of dokanfuse1.dll from 32 folder to 64 gives me this error (my hopes were not to high while doing this  :D :D):

(https://i.ibb.co/gPbfLxC/Screenshot-2021-01-10-132501.jpg) (https://ibb.co/Gtf84qj)


My system is Win 10 64
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Boygetslost on January 10, 2021, 05:56:54 PM
Thanks for the advice,

I do find my footage has a slight red tint to it, I am having trouble removing it.
How has other people dealt with this issue?


Thanks
Chris
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on January 10, 2021, 06:34:25 PM
DNG is raw. Red tint in whatever app, Mlv App or acr for instance has all to do with color matrices, white balance and so on. Or maybe white level is strange, or the preset you use. Who knows.
If something is odd with the dng files when comparing Mlvfs or Mlv App you could show us your examples here. Also post your example dng files. The more specific and the more visual information you provide, the faster you´ll get correct solutions and answers.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ML700D on April 20, 2021, 06:36:08 PM
I tried MLVFS but has problem with pink dot that appear, how to solve this?

thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ML700D on April 22, 2021, 10:01:03 AM
hi @dmilligan thank you for your MLVFS. it's great!

it makes process easier and no need to convert the file.
btw, how to process dual ISO using MLVFS?

thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: masc on April 22, 2021, 10:40:47 AM
Quote from: ML700D on April 22, 2021, 10:01:03 AM
btw, how to process dual ISO using MLVFS?
What about switching dualiso to "full 20bit" in MLVFS?
(https://timeinpixels.com/public_data/2015-02-22_common_roundtrip_issues/02_mlvfs_web.jpg)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ML700D on April 22, 2021, 10:49:40 AM
Quote from: masc on April 22, 2021, 10:40:47 AM
What about switching dualiso to "full 20bit" in MLVFS?
(https://timeinpixels.com/public_data/2015-02-22_common_roundtrip_issues/02_mlvfs_web.jpg)

ohh.. thanks. I just know it  :D
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ML700D on April 22, 2021, 11:20:27 AM
hi @masc  :D
I've tried it but the footage look like this:
(https://i.ibb.co/6FsnnhD/Screenshot-2021-04-22-161427.png)

do I have mistake here?
(https://i.ibb.co/kDnL0Hj/image-2021-04-22-175905.png)

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: COMMANDES on June 05, 2021, 07:27:29 PM
Hello friends. I am having a problem: white/colored dots in underexposed/dark areas.
(https://i.ibb.co/kSf8PGB/M02-0013-MLV00086406.png) (https://ibb.co/kSf8PGB)

DNG frame https://drive.google.com/file/d/1p-xIot9jHEvbCVPuKYnOrok3tudorMt7/view?usp=sharing (https://drive.google.com/file/d/1p-xIot9jHEvbCVPuKYnOrok3tudorMt7/view?usp=sharing)
MLV sample (280MB) https://drive.google.com/file/d/1ZkOOi5jZwBaHSQirIq5eRmHjinqE2G-N/view?usp=sharing (https://drive.google.com/file/d/1ZkOOi5jZwBaHSQirIq5eRmHjinqE2G-N/view?usp=sharing)


EDIT:
The problem occurs even at low ISOs
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ustik86 on June 20, 2021, 11:43:27 AM
[Mod: Full-Quote removed]

I can't download MLVFS.dmg for macOS because of the error on bitbucket. Can please someone upload the files to another place for download? Thanks!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on June 20, 2021, 11:59:30 AM
https://bitbucket.org/dmilligan/mlvfs/downloads/MLVFS.dmg
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ustik86 on June 20, 2021, 12:17:41 PM
(https://ibb.co/XSN14gF)

All I got is this error. I've tried different networks and computers - can't make it work...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ustik86 on June 20, 2021, 12:18:03 PM
https://ibb.co/XSN14gF
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on June 20, 2021, 12:46:04 PM
No problems here with Edge, Firefox, Opera.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ustik86 on June 20, 2021, 12:51:53 PM
I managed to download the full repository with MLVFS.dmg and installed OSXFUSE 4.12.

Then I clicked MLVFS.workflow and it added it to the context menu.

I select an empty folder, that see how localhost opens in browser, but it displays this: https://ibb.co/w7sYzPh

Something seems to be broken. I've used MLVFS on mac a few years ago but now it can't start correctly...

I have macOS Catalina 10.15.7

Please help me to make it work again!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ustik86 on June 20, 2021, 01:01:59 PM
Quote from: Walter Schulz on June 20, 2021, 12:46:04 PM
No problems here with Edge, Firefox, Opera.

It seems that I can't download MLVFS.dmg correctly from Amazon S3. I'm in Turkey now, maybe it's some kind of network problem.

Could you be so kind to upload this dmg somewhere so I can Install it?

Thanks!

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ustik86 on June 20, 2021, 01:59:01 PM
Problem solved. Just downloaded file with Tor. Works clean first try...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ustik86 on July 09, 2021, 11:06:08 AM
I noticed some weird bug recently - when I render video from Davinci in final output there are some random frames in the middle.

When I preview or playback in Davinci - there is none of that.

I suppose this is some kind of bug with MLVFS.

Did someone experience something similar?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 70MM13 on July 09, 2021, 11:47:50 AM
yes, i had the same experience on a few occasions.  it stopped me from using mlvfs...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ustik86 on July 10, 2021, 10:31:41 AM
Quote from: 70MM13 on July 09, 2021, 11:47:50 AM
yes, i had the same experience on a few occasions.  it stopped me from using mlvfs...

I'm sure there is some kind of solution to handle that.

I remember days when it worked smoothly.

Maybe older versions of Dokan do the job?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: dragonsfire1981 on October 24, 2021, 01:54:17 PM
Okay I'm trying to mount MLVFS for the first time, using the cmd line-

mlvfs.exe --mlv_dir=E:\Canon Temp\East London Castle\Rudes\Iceman.mlv

However I'm getting
MLVFS: no mlv path specified

What's wrong with the formatting? Can someone clarify what <mount point> is supposed to be? Not very clear explanation.

Edit: I managed to find the path, but it's not telling access is denied and that this version of the software cannot run on my PC...
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Walter Schulz on October 24, 2021, 01:56:12 PM
Avoid blanks in path and file names or use " or SFN (Short File Names convention).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ciaranhmc on November 24, 2021, 12:24:12 PM
Hi,

I can't use MLVFS on mac Big Sur. Has anyone had the same issue? Been using it for years, I had just finished a project using it and then decided to update my OS to Big Sur. Now I need to access those files again but localhost:8000 won't connect. Got the "macOS cannot verify the developer of "mlvfs". Are you sure you want to open it?" but when I allow nothing seems to be working.

Any help? Are there other alternatives? Thanks for your help!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ciaranhmc on November 24, 2021, 12:50:35 PM
Ok got it to work. So in case it happens to someone I did a reinstall of everything and it works... refresh the localhost:8000 page at the end.

Cheers
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: julien becker on July 06, 2022, 02:39:15 PM
Quote from: ciaranhmc on November 24, 2021, 12:50:35 PM
Ok got it to work. So in case it happens to someone I did a reinstall of everything and it works... refresh the localhost:8000 page at the end.

Cheers

Hi ciaranhmc,
I was wondering how well MLVFS works for you ? I'm also on Big Sur and I don't know if I'm doing something wrong but once le MLV folders are mounted they are unusable for me in Resolve. I mean when I import in Resolve everything is so slow that it is unusable.
If I export my 5K anamorphic footage as cDNG in the MLV app everything works fine. What's your experience ? Thanks !

Julien
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: omygolly on August 30, 2022, 11:51:24 AM
I got it working fine in Big Sur but unfortunateluy my Mac updated itself to Monteray and now it seems broken. Anyone got any ideas how to fix it?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: woefi on September 02, 2022, 04:47:40 PM
I also was a happy user of mlvfs about a year ago. In the meantime I updated my mac from MacOS 10.14 Mojave to MacOS 12 Monterey and now I cannot connect to my CDNGs anymore... I get a message "http://localhost:8000 could not be opened"

At the time I linked the dynamically created cDNGs directly in a DaVinci project and now I am hoping to successfully open the project again.
please, has anybody fixed this?

btw. I got 200Gigs of footage from that project that I would love to permanently convert to a compressed cDNG so I can finally archive it. Is there a better way of dealing with MLVs in an "RAW" kind of way?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: TG on January 06, 2023, 03:03:02 PM
Hi, can someone share latest version of Docan and  MLVFS_x64 for Win 10 to work with Resolve 17? Thank YOU!:)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: ML700D on January 10, 2023, 04:02:53 PM
Quote from: woefi on September 02, 2022, 04:47:40 PM
I also was a happy user of mlvfs about a year ago. In the meantime I updated my mac from MacOS 10.14 Mojave to MacOS 12 Monterey and now I cannot connect to my CDNGs anymore... I get a message "http://localhost:8000 could not be opened"

At the time I linked the dynamically created cDNGs directly in a DaVinci project and now I am hoping to successfully open the project again.
please, has anybody fixed this?

btw. I got 200Gigs of footage from that project that I would love to permanently convert to a compressed cDNG so I can finally archive it. Is there a better way of dealing with MLVs in an "RAW" kind of way?

try using  MLVapp (https://mlv.app/) and discuss topic here (https://www.magiclantern.fm/forum/index.php?topic=20025.0)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: julien becker on February 02, 2023, 10:14:19 AM
Quote from: woefi on September 02, 2022, 04:47:40 PM
I also was a happy user of mlvfs about a year ago. In the meantime I updated my mac from MacOS 10.14 Mojave to MacOS 12 Monterey and now I cannot connect to my CDNGs anymore... I get a message "http://localhost:8000 could not be opened"

At the time I linked the dynamically created cDNGs directly in a DaVinci project and now I am hoping to successfully open the project again.
please, has anybody fixed this?

btw. I got 200Gigs of footage from that project that I would love to permanently convert to a compressed cDNG so I can finally archive it. Is there a better way of dealing with MLVs in an "RAW" kind of way?

Hey ! Did you got MLVFS working on Monterey ?
MLVApp is great, but working with Resolve and being able to work with the .mlv files "almost" natively is a huge time saver !
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 70MM13 on February 02, 2023, 11:40:03 AM
@TG:
I am using win10x64 with mlvfs and it is rock solid.  i have it packaged in the file i linked in the opening post of this thread:
https://www.magiclantern.fm/forum/index.php?topic=25473.msg232499#msg232499
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on February 02, 2023, 11:41:10 AM
Quote from: julien becker on February 02, 2023, 10:14:19 AM
Hey ! Did you got MLVFS working on Monterey ?
MLVApp is great, but working with Resolve and being able to work with the .mlv files "almost" natively is a huge time saver !
I have it working with Ventura even. I run mlvfs through program Switch. Arm64 fuse install is needed.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on February 18, 2023, 12:40:54 AM
Quote from: 70MM13 on February 02, 2023, 11:40:03 AM
@TG:
I am using win10x64 with mlvfs and it is rock solid.  i have it packaged in the file i linked in the opening post of this thread:
https://www.magiclantern.fm/forum/index.php?topic=25473.msg232499#msg232499

Hi! Seems like you have MLVFS working with a newer version of Dokany? I can´t get my mounting to work again with Dokan 0.8.0 and mlvfz on my new PC (Been a while since I did this!). I downloaded the files you shared, installed a new version of Dokan in the hopes that it might do the trick but no luck. Any suggestions/what might I be missing to get it working?

Win10x64 installed Dokan 2.0.6.1000 (haven´t uninstalled v0.8.0)
used commandline: mlvfs_x64_lossless.exe z:\ --mlv_dir=f:\DCIM\100EOS5D

Any help greatly appreciated

Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 70MM13 on February 18, 2023, 03:01:51 AM
mine is 1.0.3.1000

i hope that fixes it for you... fingers crossed!
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Markus on February 19, 2023, 07:36:05 PM
Got it working again!
Thanks alot =).
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: julien becker on March 14, 2023, 03:13:04 PM
Quote from: Danne on February 02, 2023, 11:41:10 AM
I have it working with Ventura even. I run mlvfs through program Switch. Arm64 fuse install is needed.
Oh so is this a different method than that one : https://github.com/davidmilligan/MLVFS (https://github.com/davidmilligan/MLVFS) ?
Is there a page or ressource somewhere that explains what's needed ? Thanks Danne.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on March 14, 2023, 04:41:18 PM
I have bash integration with Mlvfs through menu navigation in this app/workflow:
https://www.magiclantern.fm/forum/index.php?topic=15108.0

select "ml" from the root menu. Fuse from here:
https://osxfuse.github.io/
Install apple silicon or intel dependeing on what machine you are running.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: julien becker on March 15, 2023, 11:32:46 AM
Thanks Danne, I wasn't aware of this and I will dive into it.
By the way, what you provide to the ML community is incredible.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 70MM13 on March 28, 2023, 04:38:10 AM
To windows users:
I ran into a problem with very high resolution footage using card spanning, with frames being corrupted.
It seems to be solved by updating to dokan v 1.5.1.1000
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mlrocks on March 30, 2023, 03:56:22 AM
"upgrading to the last 1.x version of dokan seems to have solved the issue... v 1.5.1.1000"

what is this dokan for?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: 70MM13 on March 30, 2023, 02:43:30 PM
windows doesn't have fuse.  dokan is the alternative.  look at the first post on this thread for details.

the dokan link provided there is from 2014 and is a much older version.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: mlrocks on March 30, 2023, 05:42:04 PM
thanks a lot for the info.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Norcine on April 01, 2023, 04:59:38 PM
Beginner here who's been using MLVFS recently for editing mlv files from my eos-m in Resolve and it is working just as I hoped. Except for focus pixels / pink dots in the video.
I tried to search the forum for solution, but could not easily find it (forgive me if I missed it). Only simple instructions I found were this:

Quote from: dfort on April 18, 2019, 09:21:50 PM
Realized that there are several newcomers trying to figure out how to use the focus pixel map files so here's a quick tutorial:

The easiest way to get the map files is to download the whole repository from the downloads page:

https://bitbucket.org/daniel_fort/ml-focus-pixels/downloads/

Take all of the .fmp files in the focus_pixel_map_files directory and place them next to the MLVFS binary. I'm on a Mac so it is located in my home directory > Library > Services > MLVFS > Contents (doing Control click on the MLVFS in order to get into the package contents). Never used MLVFS on Windows or Linux so I'm not sure how that works.

Same thing with MLV App, put the .fpm files next to the binary. MLV App is currently the best way to work with 5K Anamorphic files and yes, it is possible to export cdng using that application with fixed focus pixels and adjusted aspect ratio.

Either MLVFS or MLV App is a great way to get your raw footage into editing systems that can handle cdng files like DaVinci Resolve or After Effects. However -- only editing apps that use Adobe Camera Raw (like After Effects) seem to respect the resize and aspect ratio metadata in the cdng files so you'll have to adjust the image size if you want to use these dng files directly in Premiere or Resolve. You can also export to Prores or other several other file formats from MLV App and not worry about the resize/aspect ratio issues.

But the link is not working.
How do I remove these Focus Pixels in MLVFS now?

MLVFS Version: 24ebdf5 May 5 201718:56:15
Resolve 17
macOS Catalina

Example
(https://i.postimg.cc/wTjfdXFD/Focuspixeltest1.png)
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: julien becker on April 22, 2023, 05:58:35 PM
Quote from: Danne on March 14, 2023, 04:41:18 PM
I have bash integration with Mlvfs through menu navigation in this app/workflow:
https://www.magiclantern.fm/forum/index.php?topic=15108.0

select "ml" from the root menu. Fuse from here:
https://osxfuse.github.io/
Install apple silicon or intel dependeing on what machine you are running.

Thanks Danne for pointing to Switch, it's working very good. At least for non-dualiso files.
Perhaps I'm doing something wrong, but when I choose option (10) to render Dual ISO and than A to activate MLVFS, I get following message :
QuotePlease enter your selection number below:
A
ls: /tmp/no_files: No such file or directory
ls: /tmp/SCALETAG: No such file or directory
ls: /tmp/DUALISO/crop_rec: No such file or directory
fuse: unknown option `-n'

and the Finder seems to freeze, I need to restart.
Do you know why this happens ?
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: Danne on April 22, 2023, 08:05:47 PM
Dualiso is not very nice in mlvfs.
Rendering to prores is quite nice.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: julien becker on April 23, 2023, 04:34:21 PM
Ok thanks.
Title: Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
Post by: cmh on December 30, 2023, 08:28:27 PM
MLVFS for Dokany 2.1
https://github.com/cmhamiche/MLVFS/releases/tag/Release

- Install Dokany v2.1.0.1000
https://github.com/dokan-dev/dokany/releases/tag/v2.1.0.1000

- Edit mlvfs.bat by changing the path to you MLV files (or just the letter of your SD card) in the argument --mlv-dir=E:\DCIM\100CANON\ or use it directly in a command prompt


Nvm, it produces white dots on the dark parts of the picture.

If someone need a workflow that works with the free Resolve version, I shared my files there:
https://github.com/cmhamiche/Resolve_DNG_Workflow