Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter  (Read 924831 times)

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #125 on: September 22, 2014, 02:01:34 PM »
done, also added the stackoverflow link from Danne about mac security settings

DavidSh

  • Member
  • ***
  • Posts: 141
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #126 on: September 22, 2014, 04:20:03 PM »
beautiful.
600D | 5D3 | macOS Sierra | http://www.GentleDogMovie.com

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #127 on: September 22, 2014, 08:01:26 PM »
Dual iso movie conversion the fast way. Thanks for all efforts. This is massive.

*Issues with dng exiftool samelevel command in lightroom. Still gets converted though but dng tags doesn,t get written. See if Kitchehof finds some useful info on this.


ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #128 on: September 26, 2014, 12:52:33 AM »
New version of the Windows/PFM MLVFS DLL: like the other implementations, XMP sidecar files are now preserved in an associated .MLD directory.  Other types of files can be created, but they will be discarded upon unmount.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Canon eos m

  • Hero Member
  • *****
  • Posts: 562
  • 5D Mark III, Canon EOS M
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #129 on: September 26, 2014, 02:37:38 AM »
Tried activating the Pismo, etc. but gave up.
Find the g3gg0 solution better for now at least. There is so much happening on this forum and so fast difficult to keep up  :)
Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)

Frank7D

  • Senior
  • ****
  • Posts: 457
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #130 on: September 27, 2014, 12:52:55 AM »
ayshih, the "smart import 2" script didn't seem to have a problem with the latest mlvfs.dll, but I ran into another issue:
1. Mounted the MLVs
2. Created an AE project
3. Imported the sequence(s)
4. Closed and saved the project
5. Unmounted and then mounted the MLVs again
6. Opened the AE project (so far, so good)
7. Did a Ctrl+H on a sequence (replaces footage; this is a useful way to adjust the import settings after the fact)
8. ACR opened again; when I said "OK" to open in AE, I got this message:

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #131 on: September 27, 2014, 04:29:21 AM »
I've implemented vertical banding correction code from raw2dng. Please test if you have a 5D3 (I don't).

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #132 on: September 27, 2014, 02:55:52 PM »
Hi David!
I,m yet to see vertical banding on my 5d mark 3. I think I encountered one occasion with banding that was fixed with mlrawviewer.
Tried to reproduce a mlv file with banding cause the one I got is a raw file.
Just for you to clarify. Do you intend to include raw 1.0 to this workflow or is the philosophy here to stick strictly to mlv conversion? The reason I use raw is purely 3x mode and 2240x1260 24fps hacked preview which gives my several minutes of filming. Mlv is not really getting that speed.

Ok, so my test couldn,t be done cause of errors. It seems the files are getting different information in them with the latest binary. SOme shows with thumbnails, some don,t. I managed to get one dng out on my desktop, only to open up a completely black picture in acr. The mount randomly unmounts, very unstable.
OS X Mavericks macbook pro 15 inch

Here,s the dng sample
https://drive.google.com/file/d/0B4tCJMlOYfirU254U0JxR0l2aU0/edit?usp=sharing

A few error codes when I tried either open a file or drag/drop onto desktop.

I upload some pictures here:
[img=http://s28.postimg.org/l8whed015/Screen_Shot_2014_09_27_at_14_42_23.png]

translation: Photoshop could not follow you request...
[img=http://s28.postimg.org/4b2269g15/Screen_Shot_2014_09_27_at_14_42_55.png]

[img=http://s28.postimg.org/jii1qm7vt/Screen_Shot_2014_09_27_at_14_43_53.png]

[img=http://s28.postimg.org/p9884cfvt/Screen_Shot_2014_09_27_at_14_44_06.png]

[img=http://s28.postimg.org/534q5gk89/Screen_Shot_2014_09_27_at_14_44_15.png] 

On a sidenote. Could view mlvfiles fine in mlrawviewer with the former mlvfs build but exports to mov don, t work. Is this expected behavior?

Thanks for good work!

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #133 on: September 27, 2014, 04:06:56 PM »
hello,
i have tried the latest mlvfs with my mac 10.9.5, i copied the mlv dir on the desktop also the empty dir. then i mounted the dir with the mlv file, it mounted the files to the empty one, then after 10 seconds it suddenly disappears,..
tried different mlv files, its the same with all other files.
tested the same procedure on an external drive, also not working.

thx. sw

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #134 on: September 27, 2014, 05:19:56 PM »
I think this is a multithreading issue. You can use the -s option to force single threaded mode. I've updated a new dmg that uses the -s option, so try that, there may still be other stability issues.

We're going to need to do some thread synchronization for this to work properly in multithreaded mode, not sure of a good multi-platform, generic way to do this since semaphores and locks are usually OS specific, you got any suggestions @ayshih?

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #135 on: September 27, 2014, 05:36:18 PM »
@dmilligan: I've only encountered a few problems with cross-platform support for POSIX threads, and even those problems were localized to thread attributes and handling (joining, canceling, etc.).  As long as we're not talking about instantiating threads ourselves, I don't believe we'd encounter any weirdness at all with just mutexes.

@Frank7D: I think I may have tracked down the problem with the Window/PFM version.  Stay tuned.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

mageye

  • Senior
  • ****
  • Posts: 296
  • Atheist
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #136 on: September 27, 2014, 05:50:44 PM »
And on my hackintosh ... Mavericks 10.9.5 ... I am not experiencing any problems since the 22/09/2014 update of MLVFS.

(@ Danne) It's interesting you are getting those errors? ::) That is what I was getting pre 22/09/2014 (the unmounting and all error messages if you tried to import to After Effects.)

I must say now I am very happy with it as part of the workflow. I can mount, then open, adjust and export with relative ease and stability.

Oh. One thing I haven't really had a chance to try out yet is how it works with Davinci Resolve. I guess that will be fine though?

Thank you so much for all who have made this possible. It's genius! :)
5DMKII | 500D | KOMPUTERBAY 32GB Professional 1000x |Canon EF 50mm f/1.8 II | Samyang 35mm f/1.4 ED AS UMC | Canon EF 75-300mm f/4-5.6 III | Zoom H2 (4CH. audio recorder) | Mac OS X 10.9.2 | Photoshop CC | After Effects CC | Final Cut Pro 7

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #137 on: September 27, 2014, 06:08:25 PM »
New test.

Didn,t notice a different in banding with former versions. Probably because my pics don,t have banding?
With the latest mlvfs I think the vertical banding part is causing some irregular behaviour. It goes like this.
Open a dng within the mount folder and a thin pixelborder appears on top. When closing down and opening again, the border issue is gone.
When dragging a dng to desktop a big pixelated border is present on top. See included pics. These are only a sample from the latest mlvfs. These are crops and overexposed.

Straight from mlvfs mount
[img=http://s2.postimg.org/vg8m71bfp/Screen_Shot_2014_09_27_at_18_00_36.png]
after closing down and opening again
[img=http://s2.postimg.org/7qj6icd2d/Screen_Shot_2014_09_27_at_18_00_52.png]
dragged to desktop
[img=http://s2.postimg.org/vishtvf39/Screen_Shot_2014_09_27_at_18_03_24.png]


ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #138 on: September 27, 2014, 06:43:16 PM »
@Frank7D: I've uploaded a new version that fixed any crashing on my side, but I'm not using AE.  Can you try it with your workflow?
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Frank7D

  • Senior
  • ****
  • Posts: 457
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #139 on: September 27, 2014, 07:05:19 PM »
It works! Another step eliminated; thanks!

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #140 on: September 27, 2014, 07:20:10 PM »
@Danne, I think I fixed your issues with the bad data at the top of the file, try the latest.

Didn,t notice a different in banding with former versions. Probably because my pics don,t have banding?
Probably. The correction should be identical to the correction provided by raw2dng/mlv_dump, and the algorithm does not do any correction unless the coefficients are above a specific threshold.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #141 on: September 27, 2014, 08:19:15 PM »
Confirmed working! Fast progress. Amazing.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #142 on: September 27, 2014, 10:35:29 PM »
okay, I used pthreads_mutex, seems to work pretty well, so the stripe correction now supports multithreading mode. I made a LOCK() and UNLOCK() macro, so just redefine these if you need to use something else on a different platform.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #143 on: September 29, 2014, 12:05:16 AM »
Some big improvements! Chroma smoothing is now implemented (I'm not getting real-time playback with it, but it is reasonably fast ~10 fps, perhaps could be realtime on higher end systems), and this has also paved the way for easily implementing dual ISO processing, by implementing a way to process an entire frame all at once. And this has provided a fix for the issue with exFAT on OS X (b/c we avoid constantly opening and closing the MLV file, which was slow on exFAT for some unknown reason).

I haven't posted a binary yet, b/c I'm not quite sure how to provide a way to easily enable/disable the chroma smoothing options. The possible options are to provide two different automator workflows (one with chroma smooth and one without), or to prompt the user to select options (which is some extra mouse clicks). Any preference, or other ideas?

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #144 on: September 29, 2014, 12:09:44 AM »
Whow Great!
What about a choosing option when mounting??
Greets sw

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #145 on: September 29, 2014, 12:18:48 AM »
Did I hear dual iso processing  :o : :D :D

Could the setting be stored and than changed back again as desired than I would go for the click option but otherwise different  automator workflows is cool with me.


ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #146 on: September 29, 2014, 01:17:13 AM »
The possible options are to provide two different automator workflows (one with chroma smooth and one without), or to prompt the user to select options (which is some extra mouse clicks). Any preference, or other ideas?
I'm limited on the Windows/PFM side because I can't actually supply any parameters for the mounting.  My likely solution is to have a configuration text file (e.g., "config.ini") in the MLV's directory that can be parsed for such parameters on the fly.

Note that with the "config.ini" saved in the associated .MLD directory, this approach would make possible per-MLV settings (e.g., one file needs more chroma smoothing than others) that are persistent.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #147 on: September 29, 2014, 04:15:28 AM »
Mac binaries updated, I just made multiple workflows for now, install the main one and any of the cs ones you want to use. If there are more options in the future, we may have to come up with a better way to select them. (some kind of embedded webserver might be cool)

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #148 on: September 29, 2014, 07:37:34 AM »
Cool!
Tested on my macbook pro 2.6 ghz 16gb ram 15 inch. The chroma smoothing is giving a not a super smooth playback(tested in mlrawviewer)but I,m not complaining :).
Just to make sure. There is still the exiftool temp issue as before converting dual iso files? Converted as before, still the poop temp file :)
Thanks for nice work David

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #149 on: October 02, 2014, 03:15:12 AM »
I've implemented a very experimental dual ISO conversion (I really prefer to think of it more as a preview). It's nowhere near the quality of cr2hdr and probably not all that useable for real work yet, but it's realtime :)

It will try to convert dual ISO if the path name contains "DUAL", so put your dual ISO mlv files in a folder called DUAL or rename them with the word DUAL in them, to try it out.

There are all sorts of issues (like strange color artifacts, banding, bad aliasing, etc.) b/c of all the short cuts I take compared to cr2hdr (and b/c I'm no a1ex when it comes to this computational image processing stuff, I need to read the dual ISO pdf about 10 more times), but I feel like it might at least be somewhat useful for previewing and doing quick edits.

Anyway, I'm putting it out there b/c I don't really have much good test footage, so please do some tests and post some comparisons, and let me know what you think.