MLVFS - a FUSE based, "on the fly" MLV to CDNG converter

[dmilligan]

done, also added the stackoverflow link from Danne about mac security settings

[DavidSh]

beautiful.

[Danne]

Dual iso movie conversion the fast way. Thanks for all efforts. This is massive.

*Issues with dng exiftool samelevel command in lightroom. Still gets converted though but dng tags doesn,t get written. See if Kitchehof finds some useful info on this.

[ayshih]

New version of the Windows/PFM MLVFS DLL: like the other implementations, XMP sidecar files are now preserved in an associated .MLD directory.  Other types of files can be created, but they will be discarded upon unmount.

[Canon eos m]

Tried activating the Pismo, etc. but gave up.
Find the g3gg0 solution better for now at least. There is so much happening on this forum and so fast difficult to keep up 

[Frank7D]

ayshih, the "smart import 2" script didn't seem to have a problem with the latest mlvfs.dll, but I ran into another issue:
1. Mounted the MLVs
2. Created an AE project
3. Imported the sequence(s)
4. Closed and saved the project
5. Unmounted and then mounted the MLVs again
6. Opened the AE project (so far, so good)
7. Did a Ctrl+H on a sequence (replaces footage; this is a useful way to adjust the import settings after the fact)
8. ACR opened again; when I said "OK" to open in AE, I got this message:

[dmilligan]

I've implemented vertical banding correction code from raw2dng. Please test if you have a 5D3 (I don't).

[Danne]

Hi David!
I,m yet to see vertical banding on my 5d mark 3. I think I encountered one occasion with banding that was fixed with mlrawviewer.
Tried to reproduce a mlv file with banding cause the one I got is a raw file.
Just for you to clarify. Do you intend to include raw 1.0 to this workflow or is the philosophy here to stick strictly to mlv conversion? The reason I use raw is purely 3x mode and 2240x1260 24fps hacked preview which gives my several minutes of filming. Mlv is not really getting that speed.

Ok, so my test couldn,t be done cause of errors. It seems the files are getting different information in them with the latest binary. SOme shows with thumbnails, some don,t. I managed to get one dng out on my desktop, only to open up a completely black picture in acr. The mount randomly unmounts, very unstable.
OS X Mavericks macbook pro 15 inch

Here,s the dng sample
https://drive.google.com/file/d/0B4tCJMlOYfirU254U0JxR0l2aU0/edit?usp=sharing

A few error codes when I tried either open a file or drag/drop onto desktop.

I upload some pictures here:
[img=http://s28.postimg.org/l8whed015/Screen_Shot_2014_09_27_at_14_42_23.png]

translation: Photoshop could not follow you request...
[img=http://s28.postimg.org/4b2269g15/Screen_Shot_2014_09_27_at_14_42_55.png]

[img=http://s28.postimg.org/jii1qm7vt/Screen_Shot_2014_09_27_at_14_43_53.png]

[img=http://s28.postimg.org/p9884cfvt/Screen_Shot_2014_09_27_at_14_44_06.png]

[img=http://s28.postimg.org/534q5gk89/Screen_Shot_2014_09_27_at_14_44_15.png] 

On a sidenote. Could view mlvfiles fine in mlrawviewer with the former mlvfs build but exports to mov don, t work. Is this expected behavior?

Thanks for good work!

[swinxx]

hello,
i have tried the latest mlvfs with my mac 10.9.5, i copied the mlv dir on the desktop also the empty dir. then i mounted the dir with the mlv file, it mounted the files to the empty one, then after 10 seconds it suddenly disappears,..
tried different mlv files, its the same with all other files.
tested the same procedure on an external drive, also not working.

thx. sw

[dmilligan]

I think this is a multithreading issue. You can use the -s option to force single threaded mode. I've updated a new dmg that uses the -s option, so try that, there may still be other stability issues.

We're going to need to do some thread synchronization for this to work properly in multithreaded mode, not sure of a good multi-platform, generic way to do this since semaphores and locks are usually OS specific, you got any suggestions @ayshih?

[ayshih]

@dmilligan: I've only encountered a few problems with cross-platform support for POSIX threads, and even those problems were localized to thread attributes and handling (joining, canceling, etc.).  As long as we're not talking about instantiating threads ourselves, I don't believe we'd encounter any weirdness at all with just mutexes.

@Frank7D: I think I may have tracked down the problem with the Window/PFM version.  Stay tuned.

[mageye]

And on my hackintosh ... Mavericks 10.9.5 ... I am not experiencing any problems since the 22/09/2014 update of MLVFS.

(@ Danne) It's interesting you are getting those errors? That is what I was getting pre 22/09/2014 (the unmounting and all error messages if you tried to import to After Effects.)

I must say now I am very happy with it as part of the workflow. I can mount, then open, adjust and export with relative ease and stability.

Oh. One thing I haven't really had a chance to try out yet is how it works with Davinci Resolve. I guess that will be fine though?

Thank you so much for all who have made this possible. It's genius!

[Danne]

New test.

Didn,t notice a different in banding with former versions. Probably because my pics don,t have banding?
With the latest mlvfs I think the vertical banding part is causing some irregular behaviour. It goes like this.
Open a dng within the mount folder and a thin pixelborder appears on top. When closing down and opening again, the border issue is gone.
When dragging a dng to desktop a big pixelated border is present on top. See included pics. These are only a sample from the latest mlvfs. These are crops and overexposed.

Straight from mlvfs mount
[img=http://s2.postimg.org/vg8m71bfp/Screen_Shot_2014_09_27_at_18_00_36.png]
after closing down and opening again
[img=http://s2.postimg.org/7qj6icd2d/Screen_Shot_2014_09_27_at_18_00_52.png]
dragged to desktop
[img=http://s2.postimg.org/vishtvf39/Screen_Shot_2014_09_27_at_18_03_24.png]

[ayshih]

@Frank7D: I've uploaded a new version that fixed any crashing on my side, but I'm not using AE.  Can you try it with your workflow?

[Frank7D]

It works! Another step eliminated; thanks!

[dmilligan]

@Danne, I think I fixed your issues with the bad data at the top of the file, try the latest.

Didn,t notice a different in banding with former versions. Probably because my pics don,t have banding?

Probably. The correction should be identical to the correction provided by raw2dng/mlv_dump, and the algorithm does not do any correction unless the coefficients are above a specific threshold.

[Danne]

Confirmed working! Fast progress. Amazing.

[dmilligan]

okay, I used pthreads_mutex, seems to work pretty well, so the stripe correction now supports multithreading mode. I made a LOCK() and UNLOCK() macro, so just redefine these if you need to use something else on a different platform.

[dmilligan]

Some big improvements! Chroma smoothing is now implemented (I'm not getting real-time playback with it, but it is reasonably fast ~10 fps, perhaps could be realtime on higher end systems), and this has also paved the way for easily implementing dual ISO processing, by implementing a way to process an entire frame all at once. And this has provided a fix for the issue with exFAT on OS X (b/c we avoid constantly opening and closing the MLV file, which was slow on exFAT for some unknown reason).

I haven't posted a binary yet, b/c I'm not quite sure how to provide a way to easily enable/disable the chroma smoothing options. The possible options are to provide two different automator workflows (one with chroma smooth and one without), or to prompt the user to select options (which is some extra mouse clicks). Any preference, or other ideas?

[swinxx]

Whow Great!
What about a choosing option when mounting??
Greets sw

[Danne]

Did I hear dual iso processing  :

Could the setting be stored and than changed back again as desired than I would go for the click option but otherwise different  automator workflows is cool with me.

[ayshih]

The possible options are to provide two different automator workflows (one with chroma smooth and one without), or to prompt the user to select options (which is some extra mouse clicks). Any preference, or other ideas?

I'm limited on the Windows/PFM side because I can't actually supply any parameters for the mounting.  My likely solution is to have a configuration text file (e.g., "config.ini") in the MLV's directory that can be parsed for such parameters on the fly.

Note that with the "config.ini" saved in the associated .MLD directory, this approach would make possible per-MLV settings (e.g., one file needs more chroma smoothing than others) that are persistent.

[dmilligan]

Mac binaries updated, I just made multiple workflows for now, install the main one and any of the cs ones you want to use. If there are more options in the future, we may have to come up with a better way to select them. (some kind of embedded webserver might be cool)

[Danne]

Cool!
Tested on my macbook pro 2.6 ghz 16gb ram 15 inch. The chroma smoothing is giving a not a super smooth playback(tested in mlrawviewer)but I,m not complaining .
Just to make sure. There is still the exiftool temp issue as before converting dual iso files? Converted as before, still the poop temp file
Thanks for nice work David

[dmilligan]

I've implemented a very experimental dual ISO conversion (I really prefer to think of it more as a preview). It's nowhere near the quality of cr2hdr and probably not all that useable for real work yet, but it's realtime

It will try to convert dual ISO if the path name contains "DUAL", so put your dual ISO mlv files in a folder called DUAL or rename them with the word DUAL in them, to try it out.

There are all sorts of issues (like strange color artifacts, banding, bad aliasing, etc.) b/c of all the short cuts I take compared to cr2hdr (and b/c I'm no a1ex when it comes to this computational image processing stuff, I need to read the dual ISO pdf about 10 more times), but I feel like it might at least be somewhat useful for previewing and doing quick edits.

Anyway, I'm putting it out there b/c I don't really have much good test footage, so please do some tests and post some comparisons, and let me know what you think.