Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter (Read 939654 times)

DeafEyeJedi · « **Reply #400 on:** January 18, 2015, 02:01:53 AM »

Thanks for the update @dmilligan!

surami · « **Reply #401 on:** January 18, 2015, 09:47:00 AM »

@dmilligan: Thanks for checking the files and for the explanation.

swinxx · « **Reply #402 on:** January 18, 2015, 09:37:09 PM »

great. thx dmilligan.
so we now only need silentpic fullres working for canon 5d mk3 123

))
5 month no progression.. hope alex is back soon..
greets.

Ottoga · « **Reply #403 on:** January 19, 2015, 08:33:36 AM »

Quote

great. thx dmilligan.
so we now only need silentpic fullres working for canon 5d mk3 123 ))
5 month no progression.. hope alex is back soon..

And the 7D.....

Cheers

DeafEyeJedi · « **Reply #404 on:** January 21, 2015, 09:04:41 AM »

@dmilligan -- any idea why the MLV files that are produced from adv_intervalometer during time-lapse in the newly updated FRSP module that I used on 7D 2.0.3 w latest nightly (Jan 15) aren't corresponding with latest MLVFS ... it just keeps crashing on me few mins after I enable '20-bit cr2hdr' and sometimes it would just become unresponsive. (Same even if I don't select any of the options and leave all as default)

Like it would start with a few MLV's (shows DNG's) and then the rest of MLV's (w empty folder -- no dog's) when I thought it may be a long process.... waited for a quite while and still no avail?

Heck sometimes the mount folder would just disappear on its own after a while and then I would have to manually restart my Mac Mini to get the 'Mount Folder' back... so strange!?

I even deleted the IDX files just to be sure.

This only applies to MLV's from time-lapses (both non & Dual-ISO) with FRSP enabled -- regular RAW/MLV work fine atm.

This was done on OS X Yosemite 10.10.1 in case you were wondering.

Thoughts?

Thanks!

poromaa · « **Reply #405 on:** January 21, 2015, 10:10:32 AM »

Hi and thanks for yet another great utility for us film makers!

I guess everybody hate requests, so don't see it as one, but I am filming raw with 5D2 and editing on the fly with my macbook air (!) yes, it works. And MLVFS is really what makes it possible.

Just one thought, would it be possible to have a setting for virtual dngs image resolution? I'm thinking it would work like a proxy and be faster than debayering the whole thing? Or is it not working that way at all?

If possible that would enabling having the real footage in low resolution while editing, and then just change settings before render/grading.

dubzeebass · « **Reply #406 on:** January 21, 2015, 01:49:37 PM »

That's a good idea!

DeafEyeJedi · « **Reply #407 on:** January 21, 2015, 06:49:58 PM »

Good thinking @poromaa!

N/A · « **Reply #408 on:** January 21, 2015, 07:48:52 PM »

Expanding on poromaa's suggestion, maybe have an option to have a full-size embedded jpeg preview, and use it as the proxy? Low rez proxies would work too but might get complicated when we have to edit our project settings for different resolutions.

DeafEyeJedi · « **Reply #409 on:** January 21, 2015, 08:28:26 PM »

Point taken @N/A...

dmilligan · « **Reply #410 on:** January 21, 2015, 09:04:56 PM »

Quote from: poromaa on January 21, 2015, 10:10:32 AM

Just one thought, would it be possible to have a setting for virtual dngs image resolution? I'm thinking it would work like a proxy and be faster than debayering the whole thing? Or is it not working that way at all?

Do you not already get real-time playback?

Premiere, SpeedGrade, and Resolve can all playback this stuff in real-time with GPU acceleration (and MLRawViewer too). The main limiting factor is usually hard drive speed. MLVFS would still have to read the entire image data from the MLV file anyway. And also this is a problem:

Quote from: N/A on January 21, 2015, 07:48:52 PM

might get complicated when we have to edit our project settings for different resolutions.

Quote from: N/A on January 21, 2015, 07:48:52 PM

Expanding on poromaa's suggestion, maybe have an option to have a full-size embedded jpeg preview, and use it as the proxy?

CDNG doesn't allow embedded previews (only 1 main IFD is allowed). Though, you could make it an option where MLVFS just outputs JPEGs or TIFFs, and I've thought about this, but it opens up a whole new can of worms that I really have no desire to get into, since you're no longer dealing with raw data: you've got to debayer (and deal with GPU acceleration needed for that), white balance, colorspace transformations, highlight recovery, etc. There are already professional tools out there that do this stuff, better/faster than I ever could. MLVFS does what it needs to do: provide these tools with the raw data in a format they can understand.

dmilligan · « **Reply #411 on:** January 21, 2015, 11:28:34 PM »

Quote from: DeafEyeJedi on January 21, 2015, 09:04:41 AM

it just keeps crashing on me few mins after I enable '20-bit cr2hdr' and sometimes it would just become unresponsive. (Same even if I don't select any of the options and leave all as default)

Do you see any mlvfs crash logs in ~/Library/Logs/DiagnosticReports

Quote from: DeafEyeJedi on January 21, 2015, 09:04:41 AM

Heck sometimes the mount folder would just disappear on its own after a while and then I would have to manually restart my Mac Mini to get the 'Mount Folder' back... so strange!?

If that happens you can type 'umount path/to/mount/folder' in the terminal (note the lack of an 'n' in umount, that is not a typo)

mufanito · « **Reply #412 on:** January 22, 2015, 12:11:40 PM »

Hi there,

I´m having trouble getting a timeline in Resolve with dual-iso and regular iso clips. I cannot select the dual-iso option for the dual-iso files on the webserver without afecting the non dual-iso files. So, when mixing the 2 kind of files on a timeline i either get a correct image on the dual-iso files and a weird looking image on the non dual, or if i don´t select the dual-iso option on the webserver, the dual-iso files have that interlaced look and the non dual are correct. I´ve tried to put the two files types on separate folders and mount each one, but as soon as i mount the second one the first webserver tab gives place to a new one and it´s settings override the ones i set previously.

So, any tips on how to get dual-iso and non dual-iso files on the same timeline?

Thanks

dmilligan · « **Reply #413 on:** January 22, 2015, 04:10:06 PM »

You can start the webserver for the 2nd instance with a different port number with the --port=[port#] option. Then you can have two separate webguis. If you used for example 8000 and 8001 for port numbers, you'll use http://localhost:8000 and http://localhost:8001 to control the different instances.

However, if you have a non-dual ISO shot even if cr2hdr is turned on, it should detect that is not dual ISO and leave it alone (this check will hurt performance somewhat though). What we really need is dual ISO metadata provided in camera, so we know for sure whether to skip or apply dual ISO processing. I believe the MLV format has a provision for this, it's just not implemented. I'll look into implementing this, it shouldn't be too hard.

mufanito · « **Reply #414 on:** January 22, 2015, 08:31:01 PM »

Thanks!

"However, if you have a non-dual ISO shot even if cr2hdr is turned on, it should detect that is not dual ISO and leave it alone (this check will hurt performance somewhat though)." - i´ve tried this but to no avail.

silenttapes · « **Reply #415 on:** January 22, 2015, 09:33:23 PM »

Ive tried to search this thread for the answer but I can't seem to find it.

MLVFS does not show up in the Davinci Resolve library for me (11 lite). In OSX I can mount the files fine and they appear in the sidebar of finder. I have selected 'naming scheme Davinci resolve'.

silenttapes · « **Reply #416 on:** January 22, 2015, 10:05:26 PM »

Nevermind I figured it out somewhat. It doesn't look like other peoples screen shots but I can access the mounted folders.

dmilligan · « **Reply #417 on:** January 22, 2015, 11:00:55 PM »

Quote from: mufanito on January 22, 2015, 08:31:01 PM

"However, if you have a non-dual ISO shot even if cr2hdr is turned on, it should detect that is not dual ISO and leave it alone (this check will hurt performance somewhat though)." - i´ve tried this but to no avail.

I found the problem. The issue was that the metadata was being written as if the file had been dual ISO processed even if it had been skipped. The image data was correct, but the black level was wrong (cr2hdr processing changes the blacklevel). I'll post a new build shortly.

dmilligan · « **Reply #418 on:** January 23, 2015, 12:11:32 AM »

okay, I've uploaded a build with the fix

DeafEyeJedi · « **Reply #419 on:** January 23, 2015, 12:16:26 AM »

@dmilligan -- sorry I've been on the road and will be back home tonight to check for MLVFS crash logs per your request.

Also would you think the latest update from today would solve my issue?

Ether way I'll still hope to locate the crash logs for you.

Thanks again for the update, David!

mufanito · « **Reply #420 on:** January 23, 2015, 08:12:32 PM »

Thanks dmilligan! Gonna try the new build now.

DeafEyeJedi · « **Reply #421 on:** January 25, 2015, 07:59:10 AM »

Excuse me for the delay...

Here are the recent crash logs per your request from earlier. (I included a brief-test FRSP made MLV file as well just for you to check out)

'For @dmilligan' <-- fixed this link (sorry)

currently updating Davinci Resolve and will let you know how it goes.

Thanks, again!

dmilligan · « **Reply #422 on:** January 25, 2015, 03:08:54 PM »

Quote from: DeafEyeJedi on January 25, 2015, 07:59:10 AM

http://www.dropbox.com/home/Public/For_dmilligan

That's not the right link. You need to get the public link from dropbox (only you can access this link you gave me).

DeafEyeJedi · « **Reply #423 on:** January 25, 2015, 06:58:03 PM »

Sorry dmilligan -- guess I was half asleep last night...

here's the updated link...

'For dmilligan'

dmilligan · « **Reply #424 on:** January 25, 2015, 08:24:51 PM »

The crashes are because you had some MLV files that either didn't have the 'bits per pixel' metadata correct, or MLVFS didn't read it right (my guess would be the former).

The problem you are having with the FRSP dual ISO is actually due to the FUSE library timing out. A FUSE client app (MLVFS) has only so long to respond to a request before the FUSE library will timeout and kill the client and the mount. By default this is set to I think 30 seconds. It's taking the cr2hdr code longer than this to process a single frame, so it times out. You can override the timeout by supplying -o daemon_timeout=0

I would probably recommend against using MLVFS for processing full res dual ISO shots since it takes so long. It's better to use a more tradition conversion approach where you permanently process the DNG frames (so you don't have to do it again). The other reason is that the MLVFS mount is going to appear incredibly unresponsive, because it's have to work so hard to process frames.

News:

Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter (Read 939654 times)