Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter  (Read 924928 times)

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #425 on: January 27, 2015, 07:00:32 AM »
hi,

@dmilligan:
mlvfs worked so fine the last couple of month, its a great tool.
today i opened my working folder to remount some of the older mlvfs dirs, but suddenly safari can not link to the webhost, i get the message that the page could not be loaded..?
what happened. the files i try to mount are working normally..?
i run the latest fuse 2.7.4 drivers and the latest mlvfs

update: puh, i dont know why, but i had to "ignore the rights of the owner of the hdd" to make it work again. so if others have the same problems, just check the box of the information window of your drive.

best wishes, sw

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #426 on: January 27, 2015, 07:07:53 AM »
I would probably recommend against using MLVFS for processing full res dual ISO shots since it takes so long. It's better to use a more tradition conversion approach where you permanently process the DNG frames (so you don't have to do it again). The other reason is that the MLVFS mount is going to appear incredibly unresponsive, because it's have to work so hard to process frames.

I second that now... From here on out I will try to refrain from using Dual-ISO's during Timelapse FRSP (unless if I need to) in order to use your app more often when it comes to TL-FRSP within MLV files.

Thanks again for your time on this!


update: puh, i dont know why, but i had to "ignore the rights of the owner of the hdd" to make it work again. so if others have the same problems, just check the box of the information window of your drive.

was this done on a Mac?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #427 on: January 27, 2015, 08:43:21 AM »
I've been prompted to update my Nvidia CUDA and went ahead with it.

Now whenever I try to open Davinci Resolve 11 (with DR enabled within MLVFS) and once I start to scroll through REGULAR MLV's (not Dual-ISO's)... it would then become unresponsive and crash.

I would then get this window...



Not sure if this is related to MLVFS because the local web host is still active and this custom-modified Mac Mini sure seems to be up to standards for this type of stuff... (& Yes AE [w Smartimport2] as well as CC PP 2014 works just fine w MLVFS in case you were wondering)

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #428 on: January 27, 2015, 12:52:38 PM »
Sounds like a problem with Resolve or the graphics driver. MLVFS does not use the GPU in any way.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #429 on: January 27, 2015, 09:01:26 PM »
Okay thanks -- just thought it would because of the CUDA update that I just did.

Anyone else out here have a similar system OS X with the graphic card that I have?

Anyone experiencing in lagging during playback with MLVFS mounted files?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #430 on: January 27, 2015, 09:05:49 PM »
Sorry, that was supposed to say GPU, MLVFS does not use the GPU or CUDA, so it's not anything related to MLVFS.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #431 on: January 27, 2015, 09:57:20 PM »
Okay gotcha -- so would the graphic card be a factor then?

Strange it worked fine before the CUDA update.

Otherwise should I just downgrade DR back to 10 or 9 -- would this be even worth to try? I think not.

Hopefully I'll get to the bottom of this sooner rather than later.

Thanks again @dmilligan for your efforts!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #432 on: January 27, 2015, 11:13:06 PM »
CUDA is a driver for the graphics card that allows programs like Resolve to take advantage of some special types of parallel processing that the GPU is good at. This is typically used for debayering. It sounds like there's some kind of issue between the CUDA driver update you installed and Resolve (google is your friend!). This would have nothing to do with MLVFS.

niklas.seehausen

  • New to the forum
  • *
  • Posts: 6
  • 60D RAW
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #433 on: January 31, 2015, 11:40:18 AM »
The question is pretty dump. But is audio "glued" to the video or do I get 1 audio file and a folder with  cdngs ? :-\

Lars Steenhoff

  • Senior
  • ****
  • Posts: 473
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #434 on: January 31, 2015, 06:18:10 PM »
Is there a way to change the default settings?

for example I would like to have bad pixel correction and stripe correction always on
What would I need to change to make this the default setting when starting MLVFS?

Thanks

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #435 on: January 31, 2015, 08:23:42 PM »
The question is pretty dump. But is audio "glued" to the video or do I get 1 audio file and a folder with  cdngs ? :-\
I'm not sure what you mean by "glued", they are not in the same file, but in when viewed in media browsers for most apps (like Premiere of Resolve) they show up and import as one single clip with audio.

Is there a way to change the default settings?
Yes. You can edit the MLVFS service in Automator. One of the "steps" is a shell script that actually invokes MLVFS. You can specify what ever additional command line options you would like there. MLVFS has its own command line options that control the various settings you talk about (see near the bottom of main.c in the source code for a list), there are also a lot of OSXFUSE built in options that control how the mount works. See the OSXFUSE documentation for more info about those.

Lars Steenhoff

  • Senior
  • ****
  • Posts: 473
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #436 on: January 31, 2015, 08:59:49 PM »
Thanks dmilligan

I found the ones that I wanted:
--stripes
--bad-pix
--resolve-naming


Saves me a few clicks every time  :)

DanHaag

  • Freshman
  • **
  • Posts: 70
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #437 on: February 01, 2015, 11:30:01 AM »
Hi Lars, can you copy&paste the shell script with your adjustments in it so I can see what it looks like?  :) Need those changes, too. But I know nothing about using Automator or writing code.  :-[

Markus

  • Senior
  • ****
  • Posts: 338
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #438 on: February 01, 2015, 12:01:32 PM »
Any work being done on the ported "Pismo" version for Windows? I find it preferable to extracting the DNGs as I did before and it's really easy to use. It's very fast and responsive when you have mounted the mlv's.  However the mounting time takes quite a while for large files. Mounting like 256gb of mlv's from the harddrive must be somewhere  around 30mins or so. As I understand it its almost instant on the original fuse mac version?

Lars Steenhoff

  • Senior
  • ****
  • Posts: 473
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #439 on: February 01, 2015, 05:56:07 PM »
@ DANHAAG


nohup ~/Library/Services/MLVFS.workflow/Contents/mlvfs "$2" --stripes --bad-pix --resolve-naming --mlv_dir="$1" -f -o volname="$(basename $2) (MLVFS Mount)" -o local -o allow_other > /dev/null 2>&1 &



DanHaag

  • Freshman
  • **
  • Posts: 70
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #440 on: February 01, 2015, 07:08:09 PM »
Thank you, Lars! Would never have figured that out myself, helped a lot!  :)

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #441 on: February 01, 2015, 07:14:01 PM »
Any work being done on the ported "Pismo" version for Windows?
There is no one working on it ATM.

However the mounting time takes quite a while for large files.
That's because MLVFS needs to build an index file (.IDX file). Basically it searches through the MLV file and builds a table of where all the data is, that way it can access the data quickly. This may take some time for extremely large files. It does this on Mac too, but only "on demand", so the Mac mount will show up instantly, but you may have to wait for the index files to get built before you can actually browse or use the mount. You only have to build the index files once though. So the next time you mount those files, it should be much faster (as long as you don't delete the .IDX file). If you playback the video in camera, the camera does this too, and it's the same format (actually much the same code), that builds that file, and MLVFS can actually use those index files. So if you playback in camera, make sure to copy the .IDX files, and MLVFS should mount faster.

Having "random access" storage (SSD, SD, CF, etc.) will likely make building the index files a lot faster as well. To build the table we don't have to read very much data, but we do have read a little data from all over then entire file. Spinning disks are not so good at this because of "seek time" required to move the head around to different areas. With solid state, we can read from anywhere in the same amount of time (hence "random access").

Markus

  • Senior
  • ****
  • Posts: 338
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #442 on: February 01, 2015, 07:36:16 PM »
Thanks for your great reply! I guessed it had to do with indexing. I can see I have the Idx files saved next to mlv's I have mounted.  I'll have to test remounting some of them and  see if it's faster.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #443 on: February 02, 2015, 07:03:28 PM »
Once the indexing files (.IDX files) are made for each MLV files -- then the next time you mount them with MLVFS it basically would be on the fly.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

johnwangphoto

  • New to the forum
  • *
  • Posts: 15
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #444 on: February 03, 2015, 12:34:29 AM »
HI,

I got the MLVFS to work on my iMac and imported the DMG to Resolve 11. The problem I have is that all the raw clips turned way too blue and I have to adjust the white balance quite a bit. i don't know why as if I converted the MLV with MIRawViewer the white balance is normal. Could you help?

Thanks!

John

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #445 on: February 03, 2015, 01:26:44 AM »
Well, it's not particularly important, typically you'll want to make adjustments to the WB anyway, that is one of the advantages to shooting raw.

What did you have the WB set to in camera? Does that not match what MLVFS is producing? I think MLRawViewer ignores the WB metadata in the file, so perhaps the default WB MLRawViewer is using just happens to be correct, while you actually had the WB from the camera set incorrectly and that is the WB MLVFS is using.

If you still think it's a problem with MLVFS, can you upload a sample MLV file that exhibits the problem?

johnwangphoto

  • New to the forum
  • *
  • Posts: 15
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #446 on: February 03, 2015, 04:17:44 AM »
Thanks for the reply. I used the Canon 5D Mark III and I set the WB as Auto. The WB should be around 5900K but the file from MLVFS has 3200K when it is opened in Resolve. I can adjust it for sure but just feel something wrong in the setting of the MLVFS. I would love to upload the MLV file for you to check and just tell me how to do so.

John

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #447 on: February 03, 2015, 07:12:03 AM »
@johnwangphoto Dropbox would do?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

silenttapes

  • New to the forum
  • *
  • Posts: 15
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #448 on: February 03, 2015, 01:37:49 PM »
Hey!

Thanks for this app! - it has certainly saved me many hours of conversions and wondering wether to keep originals etc.

Can anyone recommend a work flow for deleting unused clips with in Davinci Resolve? I used to update usage data then delete one by one from my folders, but now they are virtual folders so the files arn't real :D


dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #449 on: February 05, 2015, 11:46:05 PM »
Compressed (LZMA) MLV file support added. You probably won't see realtime playback speeds for compressed MLV files, I would recommend simply lowering the bit depth of the MLV file if you want compression (to 12 or 10 bits), with that method you still get speedy playback. You'll probably see almost no IQ loss converting to 12 bit if you want to save some space with your MLV files. The LZMA compressed MLV files are not much smaller than 12 bit ones, so I'd drop the bit depth before I tried compression.

There is one issue if you use mlv_dump to change the bit depth of the MLV file and that is: it doesn't also update the black and white levels appropriately (if we drop bits off the image data values, we also need to drop bits off of the black and white levels). I have a fix for this, until it gets merged, you can try this mlv_dump.