MLVFS - a FUSE based, "on the fly" MLV to CDNG converter

Started by dmilligan, August 31, 2014, 02:01:24 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Danne


Danne

Thanks David. Got audio and 16-bit dual iso DNG working by shortening audio with sox and adding the ixml info from your wav.c using bwfmetaedit. Also added TimeCode 00:00:00.00 with exiftool. Will wrench it together and put it up in cr2hdr-r in a while. Thanks again.

DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dmilligan

Some updates:

I added a "prefetch" option. This will cause MLVFS to preemptively load a certain number of frames to allow utilization of multiple CPU cores even if the program accessing MLVFS isn't loading multiple files at once (this should give a fair amount of speed improvement if you're using processing heavy options with apps that don't already preload a certain number of frames, like a Finder copy). MLVFS was already "multithreaded" but this was only taken advantage of if the accessing program was actually trying to load multiple files at the same time. Now MLVFS will load future frames in expectation that they will be requested at some point soon. Let me know how well this works for you and if you find any bugs/crashing. Probably a good idea not to put a number in there higher than the number of CPU cores you have, or you will just incur extra threading overhead. Some programs like SpeedGrade already do this themselves (and you can set how many frames in the settings), if that's the case, I would recommend using that instead of MLVFS's prefetch.

Added the Adobe calibration matrices (thanks Andy!) for all ML cameras to the DNG metadata. This should help improve color "correctness" in non-Adobe programs. Please test (see the linked thread for more info).

swinxx

@dmilligan:
Thank you very much! Should the new update speed up the appereance of the webgui when mounting a big set of mlv files? Nice.
And should it also improve dualiso conversion?
Would it be possible to add an eject button in the webgui? Had some problems with unmounting it in the viewer..


Big thumbs up and a big Thank You for your improvements, swinxx

kgv5

www.pilotmovies.pl   5D Mark III, 6D, 550D

swinxx

i have now tried out the new mlvfs color matrix.

in the following example i seperated the white of the shirt and set it to the center of the vectorscope in both shots

https://copy.com/LvhkenDVEDPDkyj2

to my suprise the blue color changed dramatically.
left side is the old one, right side the new one.

i would prefere the left side, so perhaps dmilligan: would it be possilbe to put the new matix to the webgui as an optional setting?

thx

dmilligan

Quote from: swinxx on November 28, 2014, 09:37:18 AM
would it be possilbe to put the new matix to the webgui as an optional setting?
It's actually not possible to make this an option, the size of the CDNG IFD is hardcoded. I wouldn't want to make something like this an option anyway, there's probably something else going on that needs to be fixed, and that's the real problem that should be addressed. It might be that the AsShotNeutral is getting computed wrong because it's using routines from dcraw that use the old dcraw matrices. I'll update them and see if it makes any difference.

Quote from: swinxx on November 28, 2014, 06:50:08 AM
Should the new update speed up the appereance of the webgui when mounting a big set of mlv files?
Probably not.
Quote from: swinxx on November 28, 2014, 06:50:08 AM
And should it also improve dualiso conversion?
Yes, especially since you're using the Finder, which only tries to read the files one by one.
Quote from: swinxx on November 28, 2014, 06:50:08 AM
Would it be possible to add an eject button in the webgui?
Probably. You can always type "umount /path/to/mlvfs/mount" in the terminal. (Note there is no "n" in "umount")


Quote from: DeafEyeJedi on November 25, 2014, 08:19:37 AM
Dual ISO MLV would be sweet!
Quote from: kgv5 on November 28, 2014, 08:04:33 AM
Oh yes, that would be great
Dual ISO conversion has been implemented in MLVFS for some time now.

DeafEyeJedi

Thanks @dmilligan and that quote was actually for @Danne -- my fault on the confusion and again thanks for your kind update!

Enjoy your Holidays!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

Cool stuff David. Will test speed when time is on my hands.
@deafeyejedi. Cr2hdr-r has been able to convert mlv dualiso files from the very beginning. Check the first post in the cr2hdr-r thread.

DeafEyeJedi

I knew that I guess what I meant was regarding audio directly.

Happy Thanksgiving to you all and everyone else for never stopping the magic!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

baldavenger

The new matrix has seemingly added a red/magenta bias to my images now.  I'm using a 5D Mark III, bought in March this year.  Would it be possible to download an older build of MLVFS so I can make comparisons between before and after?
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Andy600

Sorry guys,

After looking at the previous matrices I think I might have got the matrix order wrong. I'm gonna check a few shots and if it's what I think I will fork the repo, change the code and submit a pull req.


Edit: Yes, it's my fault  :-[ There is an extra digit in ColorMatrix2 that shouldn't be there (green-green) and the forward matrices are incorrect. I will recheck everything.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

Andy600

@dmilligan - submitted a pull req that should fix 5D mark III footage. I'll submit a separate pull req for the other cams later. Sorry for the hassle  :-[


Edit: pull req submitted for all cams. See note on Bitbucket.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

swinxx


swinxx


dmilligan

Updated the binaries with Andy's changes. Let us know if it's better now.

Danne

Hi David!
Dragged a folder with dual iso files to desktop but can,t view the files in acr.
Here is one of the converted dng:s
https://sprend.com/download.htm?C=4085435b91c746f7aebb0ad4821d3baa

Message from AE



dmilligan

reproduced..., this is what the Adobe DNG SDK has to say about it:

*** Error: ForwardMatrix1 does not map equal camera values to XYZ D50 ***


here are the relevant DNG tags:

Model: "Canon EOS 60D"
ColorMatrix1:
      0.7428  -0.1897  -0.0491
     -0.3505   1.0963   0.2929
     -0.0337   0.1242   0.6413
ColorMatrix2:
      0.6719  -0.0994  -0.0925
     -0.4408   1.2426   0.2211
     -0.0887   0.2129   0.6051
AsShotNeutral: 0.6215 1.0000 0.4568
CalibrationIlluminant1: Standard light A
CalibrationIlluminant2: D65
ForwardMatrix1:
      0.7550   0.0645   0.1448
      0.2138   0.8936   0.1075
     -0.0005  -0.4306   1.2562
ForwardMatrix2:
      0.7286   0.1385   0.0972
      0.2600   0.9468  -0.2068
     -0.0093  -0.2268   1.0426


any thoughts @Andy600?

dmilligan

found the problem, some wrong negative signs, I'll check them for all cameras and then update it

dmilligan

Fixed problems in the 5D3, 5D2, and 60D forward matrices (and verified 550D is correct). Please test (especially for other cameras, which I haven't checked).

Andy600

I'll check the others. Looks like I made some typos on Bitbucket  ::)

edit: checked and double checked all cams - submitted pull req
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

kgv5

I would like to make sure - i am on windows 7 - does updating mlvfs requires to replace mlvfs.dll only or do i need to make something more (like registering in cmd every time for example)? Thanks
www.pilotmovies.pl   5D Mark III, 6D, 550D

DeafEyeJedi

*feedbacks*

I just ran the latest MLFVS w a Dual-ISO MLV file and seems to look better with the thumbnails looking cleaner than ever! Great job on that @dmilligan -- however I ran into this odd pop up window after trying to export through with smartscript2 in AE...





same AE error-code as @danne posted from earlier BUT this MLV file was produced directly from an EOS-M body...

thoughts?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dmilligan

That error means the color matrices in the DNG metadata are wrong/invalid. I'll double check them for EOSM (make absolutely sure you have the latest version).