Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter  (Read 924932 times)

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #325 on: November 24, 2014, 06:55:47 AM »
Thanks, will check on this.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #326 on: November 24, 2014, 05:47:10 PM »
Thanks David. Got audio and 16-bit dual iso DNG working by shortening audio with sox and adding the ixml info from your wav.c using bwfmetaedit. Also added TimeCode 00:00:00.00 with exiftool. Will wrench it together and put it up in cr2hdr-r in a while. Thanks again.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #327 on: November 25, 2014, 08:19:37 AM »
Great news, Danne!

Dual ISO MLV would be sweet!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #328 on: November 28, 2014, 03:16:25 AM »
Some updates:

I added a "prefetch" option. This will cause MLVFS to preemptively load a certain number of frames to allow utilization of multiple CPU cores even if the program accessing MLVFS isn't loading multiple files at once (this should give a fair amount of speed improvement if you're using processing heavy options with apps that don't already preload a certain number of frames, like a Finder copy). MLVFS was already "multithreaded" but this was only taken advantage of if the accessing program was actually trying to load multiple files at the same time. Now MLVFS will load future frames in expectation that they will be requested at some point soon. Let me know how well this works for you and if you find any bugs/crashing. Probably a good idea not to put a number in there higher than the number of CPU cores you have, or you will just incur extra threading overhead. Some programs like SpeedGrade already do this themselves (and you can set how many frames in the settings), if that's the case, I would recommend using that instead of MLVFS's prefetch.

Added the Adobe calibration matrices (thanks Andy!) for all ML cameras to the DNG metadata. This should help improve color "correctness" in non-Adobe programs. Please test (see the linked thread for more info).

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #329 on: November 28, 2014, 06:50:08 AM »
@dmilligan:
Thank you very much! Should the new update speed up the appereance of the webgui when mounting a big set of mlv files? Nice.
And should it also improve dualiso conversion?
Would it be possible to add an eject button in the webgui? Had some problems with unmounting it in the viewer..


Big thumbs up and a big Thank You for your improvements, swinxx

kgv5

  • Hero Member
  • *****
  • Posts: 643
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #330 on: November 28, 2014, 08:04:33 AM »
Dual ISO MLV would be sweet!

Oh yes, that would be great
www.pilotmovies.pl   5D Mark III, 6D, 550D

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #331 on: November 28, 2014, 09:37:18 AM »
i have now tried out the new mlvfs color matrix.

in the following example i seperated the white of the shirt and set it to the center of the vectorscope in both shots

https://copy.com/LvhkenDVEDPDkyj2

to my suprise the blue color changed dramatically.
left side is the old one, right side the new one.

i would prefere the left side, so perhaps dmilligan: would it be possilbe to put the new matix to the webgui as an optional setting?

thx

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #332 on: November 28, 2014, 02:16:27 PM »
would it be possilbe to put the new matix to the webgui as an optional setting?
It's actually not possible to make this an option, the size of the CDNG IFD is hardcoded. I wouldn't want to make something like this an option anyway, there's probably something else going on that needs to be fixed, and that's the real problem that should be addressed. It might be that the AsShotNeutral is getting computed wrong because it's using routines from dcraw that use the old dcraw matrices. I'll update them and see if it makes any difference.

Should the new update speed up the appereance of the webgui when mounting a big set of mlv files?
Probably not.
And should it also improve dualiso conversion?
Yes, especially since you're using the Finder, which only tries to read the files one by one.
Would it be possible to add an eject button in the webgui?
Probably. You can always type "umount /path/to/mlvfs/mount" in the terminal. (Note there is no "n" in "umount")


Dual ISO MLV would be sweet!
Oh yes, that would be great
Dual ISO conversion has been implemented in MLVFS for some time now.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #333 on: November 28, 2014, 04:51:09 PM »
Thanks @dmilligan and that quote was actually for @Danne -- my fault on the confusion and again thanks for your kind update!

Enjoy your Holidays!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #334 on: November 28, 2014, 05:22:27 PM »
Cool stuff David. Will test speed when time is on my hands.
@deafeyejedi. Cr2hdr-r has been able to convert mlv dualiso files from the very beginning. Check the first post in the cr2hdr-r thread.
 

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #335 on: November 28, 2014, 07:30:11 PM »
I knew that I guess what I meant was regarding audio directly.

Happy Thanksgiving to you all and everyone else for never stopping the magic!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

baldavenger

  • Member
  • ***
  • Posts: 177
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #336 on: November 28, 2014, 11:49:03 PM »
The new matrix has seemingly added a red/magenta bias to my images now.  I'm using a 5D Mark III, bought in March this year.  Would it be possible to download an older build of MLVFS so I can make comparisons between before and after?
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Andy600

  • Moderator
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #337 on: November 29, 2014, 02:33:41 AM »
Sorry guys,

After looking at the previous matrices I think I might have got the matrix order wrong. I'm gonna check a few shots and if it's what I think I will fork the repo, change the code and submit a pull req.


Edit: Yes, it's my fault  :-[ There is an extra digit in ColorMatrix2 that shouldn't be there (green-green) and the forward matrices are incorrect. I will recheck everything.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

Andy600

  • Moderator
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #338 on: November 29, 2014, 03:11:32 AM »
@dmilligan - submitted a pull req that should fix 5D mark III footage. I'll submit a separate pull req for the other cams later. Sorry for the hassle  :-[


Edit: pull req submitted for all cams. See note on Bitbucket.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #339 on: November 29, 2014, 07:17:43 AM »
Thx andy, i'll give it a try when it will be updated ;)

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #340 on: November 29, 2014, 08:09:34 AM »
And thx dmilligan for the fast update!! You are great!!

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #341 on: November 29, 2014, 04:19:42 PM »
Updated the binaries with Andy's changes. Let us know if it's better now.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #342 on: November 29, 2014, 04:46:41 PM »
Hi David!
Dragged a folder with dual iso files to desktop but can,t view the files in acr.
Here is one of the converted dng:s
 https://sprend.com/download.htm?C=4085435b91c746f7aebb0ad4821d3baa

Message from AE



dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #343 on: November 29, 2014, 05:16:23 PM »
reproduced..., this is what the Adobe DNG SDK has to say about it:
Code: [Select]
*** Error: ForwardMatrix1 does not map equal camera values to XYZ D50 ***

here are the relevant DNG tags:
Code: [Select]
Model: "Canon EOS 60D"
ColorMatrix1:
      0.7428  -0.1897  -0.0491
     -0.3505   1.0963   0.2929
     -0.0337   0.1242   0.6413
ColorMatrix2:
      0.6719  -0.0994  -0.0925
     -0.4408   1.2426   0.2211
     -0.0887   0.2129   0.6051
AsShotNeutral: 0.6215 1.0000 0.4568
CalibrationIlluminant1: Standard light A
CalibrationIlluminant2: D65
ForwardMatrix1:
      0.7550   0.0645   0.1448
      0.2138   0.8936   0.1075
     -0.0005  -0.4306   1.2562
ForwardMatrix2:
      0.7286   0.1385   0.0972
      0.2600   0.9468  -0.2068
     -0.0093  -0.2268   1.0426

any thoughts @Andy600?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #344 on: November 29, 2014, 06:25:05 PM »
found the problem, some wrong negative signs, I'll check them for all cameras and then update it

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #345 on: November 29, 2014, 06:42:38 PM »
Fixed problems in the 5D3, 5D2, and 60D forward matrices (and verified 550D is correct). Please test (especially for other cameras, which I haven't checked).

Andy600

  • Moderator
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #346 on: November 29, 2014, 11:09:36 PM »
I'll check the others. Looks like I made some typos on Bitbucket  ::)

edit: checked and double checked all cams - submitted pull req
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

kgv5

  • Hero Member
  • *****
  • Posts: 643
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #347 on: November 30, 2014, 10:15:35 AM »
I would like to make sure - i am on windows 7 - does updating mlvfs requires to replace mlvfs.dll only or do i need to make something more (like registering in cmd every time for example)? Thanks
www.pilotmovies.pl   5D Mark III, 6D, 550D

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #348 on: December 06, 2014, 09:17:50 AM »
*feedbacks*

I just ran the latest MLFVS w a Dual-ISO MLV file and seems to look better with the thumbnails looking cleaner than ever! Great job on that @dmilligan -- however I ran into this odd pop up window after trying to export through with smartscript2 in AE...





same AE error-code as @danne posted from earlier BUT this MLV file was produced directly from an EOS-M body...

thoughts?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #349 on: December 06, 2014, 03:49:37 PM »
That error means the color matrices in the DNG metadata are wrong/invalid. I'll double check them for EOSM (make absolutely sure you have the latest version).