Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter  (Read 923252 times)

Frank7D

  • Senior
  • ****
  • Posts: 457
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #75 on: September 14, 2014, 07:45:57 PM »
"I presume you want the filename of the MLV video prepended to the individual DNG files and WAV file?"

That would be perfect (for me at least).

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #76 on: September 14, 2014, 07:56:10 PM »
Is this xmp creation present on mac as well working with ae or is it a windows thing?
For now, just on the Windows side.  Because of the differences between FUSE and PFM and the rapid nature of MLVFS development, I imagine this won't be the last time you'll see features show up on one version before they show up on the other versions.

"I presume you want the filename of the MLV video prepended to the individual DNG files and WAV file?"

That would be perfect (for me at least).
Does anyone else have any thoughts?  I don't want to make this change if there turns out to be a compelling reason to keep it as it currently is (or to change it to some third option).
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7541
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #77 on: September 14, 2014, 08:06:17 PM »
Cool, would it be able to store a new folder as well containing copies of dng;s like storing an xmp? I,m thinking Kitchehof lightroom app workflow in first hand. This would entirely skip the first converting step when working dual iso movie files.

surami

  • Member
  • ***
  • Posts: 115
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #78 on: September 14, 2014, 08:55:35 PM »
Quote from: ayshih
Does anyone else have any thoughts?  I don't want to make this change if there turns out to be a compelling reason to keep it as it currently is (or to change it to some third option).
I prefer this kind of file naming, source MLV: M18-1424.MLV, DNG: M18-1424-00000.DNG ... M18-1424-00250.DNG
550D + nightly ML

Frank7D

  • Senior
  • ****
  • Posts: 457
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #79 on: September 14, 2014, 09:07:45 PM »
99,999 frames at 23.976 fps would give about 69.5 minutes of video, which should be more than enough for one take, so I agree with surami's suggestion.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #80 on: September 15, 2014, 01:01:14 AM »
I chose "_AUDIO.WAV" so that the audio would appear first. I'm fine with adding the MLV filename as a prefix, but we probably need to decide something and then not change it anymore, as it would mess up any projects already created that point to specific filenames. The other option is making it configurable with various options from MLV metadata (such as cam name/serial, text/take tags, date/time, etc). Some kind of string format could be a command line parameter. Ex: "$CAMERA_$TAKE_$TIME_" => 7D_003_07-30_00000000.DNG. The advantage to making it customizable is that we can avoid breaking old projects that depend on specific paths. Of course the disadvantage is this is significantly more complex.

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3184
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #81 on: September 15, 2014, 01:17:50 AM »
ive chosen M26-1335_000000.dng in my windows MLV client.
thought thats a good choice. frame count should be enough and it contains the MLV name as prefix.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #82 on: September 15, 2014, 07:05:13 AM »
I've changed the naming scheme in the FUSE and Windows/PFM versions to match g3gg0's scheme.  (dmilligan: please update the OS X binaries)
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Frank7D

  • Senior
  • ****
  • Posts: 457
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #83 on: September 15, 2014, 07:40:20 AM »
So right now, the audio file is named like "M13-1837_audio.wav"
Could it just be "M13-1837.wav"?
It would work better with the After Effects script I am using ("Smart Import 2.jsx").

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3184
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #84 on: September 15, 2014, 08:10:33 AM »
scripts can be simply changed ;)
main question is, if it makes sense to remove _audio or to keep it.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Frank7D

  • Senior
  • ****
  • Posts: 457
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #85 on: September 15, 2014, 08:25:48 AM »
I agree. Is there a reason to have the "_audio" in the name?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #86 on: September 15, 2014, 01:32:06 PM »
I don't really see a reason to have "_audio" in the name anymore using the prefixes. Before, when not using prefixes, it had to have some kind of name so I just named it what it was, "audio", and the "_" makes it show up first. Now, I see no reason not to just use "M12-3456.WAV"

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #87 on: September 15, 2014, 03:47:17 PM »
Yeah, there really doesn't appear to be any benefit for having "_audio" in the filename when it already has the extension ".wav", so the cleaner approach would be to remove the "_audio".
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3184
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #88 on: September 15, 2014, 03:52:08 PM »
will do it the same as MLVFS does. so you both decide.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #89 on: September 15, 2014, 05:39:40 PM »
With no objections to removing "_audio", I have made that change.  New Windows/PFM DLL uploaded.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #90 on: September 15, 2014, 10:40:49 PM »
mac binary updated

Frank7D

  • Senior
  • ****
  • Posts: 457
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #91 on: September 16, 2014, 06:04:36 AM »
Quick note/tip for pismo method:
If having difficulty importing the (virtual) DNG sequences into your favorite software, try:
1. Making each virtual MLV folder writable.
2. Deleting the IDX files that get created alongside the folders when you mount the MLV.

I had to do both of the above when using the smart import scripts in After Effects.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #92 on: September 18, 2014, 12:02:58 AM »
@g3gg0,

Well, I'm sort of stuck, this math seems to be over my head. I just sort of blindly implemented your C# code, but mathematically, I have no idea what's going on or what's supposed to be happening. I'm pretty sure I've computed xyzKelvinWb matrix correctly, but I don't really know where to go from there to get AsShotNeutral. If you don't mind, take a look: https://bitbucket.org/dmilligan/mlvfs/branch/white_balance

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #93 on: September 18, 2014, 12:14:12 AM »
If it helps, I've extracted the ufraw kelvin routines in a separate C file:
https://bitbucket.org/hudson/magic-lantern/src/cr2hdr-20bit/modules/dual_iso/kelvin.c
https://bitbucket.org/hudson/magic-lantern/src/cr2hdr-20bit/modules/dual_iso/kelvin.h

Usage examples in cr2hdr.c (look for AsShotNeutral and kelvin): https://bitbucket.org/hudson/magic-lantern/src/cr2hdr-20bit/modules/dual_iso/cr2hdr.c
For initialization, call adobe_coeff("Canon", "EOS ...") - I'm doing this in dcraw-bridge.c.

Also, a good reference is this: http://users.soe.ucsc.edu/~rcsumner/rawguide/RAWguide.pdf

If you'll still be stuck, I can sit down and figure it out.

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3184
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #94 on: September 18, 2014, 01:52:45 AM »
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3184
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #95 on: September 18, 2014, 09:19:10 AM »
a good XYZ/xyY/wavelength/kelvin converter is there: https://www.sylvania.com/en-us/tools-and-resources/Pages/led-color-calculator.aspx
it's from osram and meant for LED gamut calculation
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3184
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #96 on: September 18, 2014, 10:08:19 AM »
if you want to debug your code, here the results of my code:

Code: [Select]
            /* get the XYZ --> cone reference whites for requested temperatures */
            Matrix temp = KelvinToXYZ(ColorTemperature);
            /* verify: do all our matrices that came from dcraw really convert raw to D65? */
            Matrix tempD65 = KelvinToXYZ(6500);

            Matrix dst = coneDomain * temp;
            Matrix src = coneDomain * tempD65;

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #97 on: September 18, 2014, 06:25:44 PM »
Thanks for the link a1ex, that's the most helpful thing I've read. I think I've started to make some progress in my understanding. Please correct me where I'm wrong:

Compute XYZKelvinWB matrix using the "chromatic adaptation" techniques in the Bradford cone domain (I'm pretty sure now, that I've got this part right). Then use the XYZKelvinWB matrix to transform "white" (1,1,1) to some different color temperature (I assume this is what the XYZKelvinWB matrix does, but I'm not totally sure). Then convert this new "white" back to the camera raw color space using the inverse of the "ColorMatrix1" (which if I understand correctly this matrix is a Cam->XYZ). Now we have a neutral color in camera raw coordinates right? So then the multipliers (aka AsShotNeutral) are simply {R/G, 1, B/G}. Mathematically this would be:

CamNeutral = [ColorMatrix1]-1 * ([XYZKelvinWB] * [1,1,1])
AsShotNeutral = [ CamNeutral R / CamNeutral G, 1, CamNeutral B / CamNeutral G ]

Is this correct, or am I totally off?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #98 on: September 19, 2014, 10:44:35 PM »
well none of that worked, but I just refactored kelvin.c from c2hdr20bit to work with mlvfs (had to get rid of some global variables) and its working, yay!

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7541
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #99 on: September 20, 2014, 09:18:09 AM »
Nice work.
Hi David. Love this workflow. Any thoughts on including raw format and maybe also some "write" possibilities(direct use in after effects)? Noticed some of this going on in the windows workflow but I have no idea how complicated this will be for mac mlvfs.