Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter  (Read 664056 times)

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3748
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1350 on: November 14, 2016, 03:44:20 PM »
That's what I'm seeing. Works perfectly except focus pixel fixing isn't applied when mlvfs is launched from the command line--no matter where the fpm are located.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1351 on: November 14, 2016, 08:20:12 PM »
In your home directory, create a folder called MLVFS_TEST
Place the mlvfs executable in this folder.
Place your fpm files in this folder

In your home directory create another folder called MLV_TEST
Place your MLV files in this folder.

In your home directory create another folder called MOUNT_TEST

Type EXACTLY this in the console
Code: [Select]
cd ~/MLVFS_TEST
./mlvfs ~/MOUNT_TEST --mlv-dir=~/MLV_TEST -f -o local -o volname="TEST" -o allow_other


Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6461
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1352 on: November 14, 2016, 08:52:05 PM »
That is working. So -f -o local volname and -o allow other has to be specified in there as well.
Quote
Daniels-MBP:MLVFS_TEST Daniel$ ./mlvfs ~/MOUNT_TEST --mlv-dir=~/MLV_TEST -f -o local -o volname="TEST" -o allow_other
Loading focus pixel map '80000346_1808x1190.fpm'...

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3748
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1353 on: November 14, 2016, 11:20:27 PM »
Fantastic--thanks for your patience.

I also see the "Loading focus pixel map '80000331_1808x727.fpm'..." message in the terminal and focus pixel removal is working.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

cloudnine

  • New to the forum
  • *
  • Posts: 4
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1354 on: November 16, 2016, 12:46:16 PM »
Hi guys, still looking for help to have this running on windows 10 64 bit. Lureb back on page 54 gave two links to download from, but both links have multiple files, can i be advised as to which files to actually use please?
Many thanks to all.

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 7230
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1355 on: November 16, 2016, 01:01:16 PM »
Read first post of this thread for installation instructions.
Photogs and videographers: Assist in proof reading upcoming in-camera help!. Your input is wanted and needed!

viktor255

  • New to the forum
  • *
  • Posts: 2
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1356 on: November 20, 2016, 02:38:12 AM »
Hello guys, first of all thanks for great work.

Something to contribute. I just spent all day trying to get MLVFS working on Windows 10 64-bit. Old 32-bit version from 1.page worked from the begining, but to start newest MLVFS 64-bit version with newest Dokany wat PITA.

cloudnine - this may be useful for you. I got newest 1.0.1 DokanSetup.exe from https://github.com/dokan-dev/dokany/releases and it was running without problem.
But when I tried start MLVFS http://www.magiclantern.fm/modules/modules/MLVFS_x64.zip/ from CMD(classic comand chain, cd to dir, and mlvfs_x64.exe x:\ --mlv-dir=C:\MLV) nothing happend, MLVFS did not start, exactly like lureb74 described on page 54, no error message nothing, localhost was not working neither. After all day reinstalling everything from Visual C++, dokany, clearing registry I found a solution for me.

I deleted dokan1.dll and dokanfuse1.dll from MLVFS directory and like magic everything works!

I do not understand why this was the problem, maybe somebody can explain further what those files do in MLVFS directory?

lureb74

  • New to the forum
  • *
  • Posts: 46
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1357 on: November 23, 2016, 05:58:18 PM »
Ciao!
For "cloudnine" and "viktor255", it works for me like this: dokan v0.8.0 and MLVFS_x86 (first post), but I put in the MLVFS_x86 folder the optimized x64 exe files (from http://www.magiclantern.fm/modules/modules/mlvfs_optimized.zip/) and then point at them in the cmd:
mlvfs_x64_avx drive-letter:\ --mlv_dir=directory-where-mlv-files-are

In my PC the avx2 version doesn't work but the avx yes!

eNnvi

  • New to the forum
  • *
  • Posts: 48
  • 700D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1358 on: November 26, 2016, 03:10:47 PM »
Hi all guys,
first thanks a lot for your active development, i'm a developer too and i know sometimes how much can be frustrating doing something just for the open source  :-X

anyway i'm using MLVFS on windows and i'm running it as a process using NSSM

it seems to be stable so i wanted to share a little guide on how to achive this, so here we are.

How to install MLVFS as service under Windows using NSSM
First i want to explain what this does and how this can speed up your workflow.
Services under windows are software running under a system account so you won't see any black command line window. The power of services is the possibility to launch a process at startup (or login) and kill that application when you turn off your PC.
Now having MLVFS running as service can let you always edit your footage forgetting al those command lines and you will never forget again about opening MLVFS before starting your favourite editing software  ;)

let's go step by step:
1. Install MLVFS as described in the first post and TRY IT! be sure that everything is running fine
2. Now that you know that everything is running fine proceed downloading NSSM from https://nssm.cc/
      - direct link for download -> https://nssm.cc/release/nssm-2.24.zip (but check if there are new versions)
3. Open the zip folder, enter the nssm folder, double click on the windows version you have installed (win32 or win 64) and copy the nssm.exe file
4. Now we have to place this executable somewhere, i suggest you to copy it in your main drive (usually C:) in the program files folder (C:\Program Files\)
5. Now NSSM is "installed"

ok we have installed NSSM, so what's next?
NSSM is an interface to allow any executable to become a service in windows.

let's go on
6. In the program files folder (or in the folder you copied nssm.exe) keep pressed shift+right click in a empty space of the folder, this will open the usual dropdown menu, but you can click "open command window here" this will open the old-fashioned command line tool of windows in the folder you have nssm.exe
7. Type this in your command line:
Code: [Select]
nssm install MLVFS8. Administrator window should appear, click yes
9. A window like the follow should appear


Now let's see the fields you need:
Path: Is the path to your MLVFS.exe file. For example having a second hard drive i installed it in the "D:\Program Files (x86)\MLVFS_x86\" folder.
So my path will be "D:\Program Files (x86)\MLVFS_x86\mlvfs.exe"
Startup directory: Is the directory you want the service to be started from, i placed "D:\Program Files (x86)\MLVFS_x86\" but it can be left blank if you want
Arguments: here is the place where you want to give all the command lines arguments to the mlvfs service. The basic is this:
Code: [Select]
Z:\ --mlv-dir="C:\dir\to\your\mlv\footage"if you (like me) own a 700D probably you will want to add the 2x2 chroma smoothing, so argument field will ahve to be filles with:
Code: [Select]
Z:\ --mlv-dir="C:\dir\to\your\mlv\footage" --cs2x2and if you also want the naming for davinci
Code: [Select]
Z:\ --mlv-dir="C:\dir\to\your\mlv\footage" --cs2x2 --resolve-namingso any command line argument that can be found here: https://bitbucket.org/dmilligan/mlvfs
(of course you can edit this also from the web gui, but every time you restart your pc the settings made via web interface are deleted, adding these lines in the arguments field will make them "permanent").

10. (almost done) now get in the Details tab

in the Display Name put "MLVFS"
in the description put everyting you want
in the startup type select Automatic

11. click "install service"

yeeeee we're finally done  ;D

now we have to start the service  :P
this is quite simple.


now you're back to the command window, just type
Code: [Select]
nssm start MLVFSand it should start  ;D

if this doesn't happen (returns error) do te following:
1. Open control panel->System and Security->Administrative tools
2. New windows will pop up, open Services
3. Find in your list MLFVS, then right click->start

ok in your system now i'll see the fake "Z:\" disk containing all your footage ready to edit  ;)


Workflow
The wokflow stays the same as before, but the folder that will be scaned from mlvfs is one... so my advice is this:
create a new sub-folder for every project in the "scanned" folder you choosed above (the "C:\dir\to\your\mlv\footage" path)
as MLVFS supports the folder organization it can help keeping your footage in a good order  ;) then if you use davinci like me, you can set the main media folder in Z:\



Hope this can be helpful to someone  ;)

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3155
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1359 on: November 26, 2016, 05:58:36 PM »
> An error occurred during a connection to nssm.cc. Peer’s Certificate has been revoked. Error code: SEC_ERROR_REVOKED_CERTIFICATE

scam? but looks legit.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

eNnvi

  • New to the forum
  • *
  • Posts: 48
  • 700D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1360 on: November 26, 2016, 08:53:16 PM »
> An error occurred during a connection to nssm.cc. Peer’s Certificate has been revoked. Error code: SEC_ERROR_REVOKED_CERTIFICATE

scam? but looks legit.
mmmh not having that certificate problem with both firefox and chrome for me...

jmanord

  • New to the forum
  • *
  • Posts: 46
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1361 on: December 01, 2016, 01:23:49 AM »
I just wanted to say how awesome MLVFS is! I decided to give CompressMLV another try today, and was elated that it worked without incident. For whatever reason, I was unable to get it to work when I initially tried it over a year ago. I have been trying to keep the original MLVs as backups, and cinemadngs compressed with slimRaw as my working media. Unfortunately, this has become too tedious and time consuming. With CompressMLV, I can use the MLV files for both my working media and archive copy while using half the space. Thank you dmilligan for your brilliant work!

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1362 on: December 22, 2016, 09:10:42 AM »
@dmilligan

I installed  as the description in the first post.
But I can't get see my files in the empty folder.

I'm using Sierra 10.12.2, and I think could be a problem with this.

I double check last version of FUSE in the site.

I am missing something? Or I should wait for an upgrade?

Thank you.

PaulHarwood856

  • Senior
  • ****
  • Posts: 292
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1363 on: December 30, 2016, 01:33:00 AM »
Hello everyone,

     It's really great 10 bit raw recording on the T3i is able to be used with MLVFS. I do have one problem though, with all raw video recording with the T3i and MLVFS. For some reason, I get an Embedded Profile in Adobe Camera Raw. I cannot use Cinelog-C unless I use a Terminal Command with MLV_Dump. I read on the ACR VisionLog Camera Profile Post on this Forum that this was also an issue for the T4i. Please note, this does not occur on my 7D. Is there any way to get the T3i to not have an Embedded Profile? It would be great, and save a bit more time. Here is a link to the forum post I was referring to: https://www.magiclantern.fm/forum/index.php?topic=16738.50 Thanks!

- Paul Harwood

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1364 on: December 30, 2016, 09:01:02 PM »
As explained in the thread you linked, there's not an elegant solution on the MLVFS side. The fix should really be on the ML side.

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6461
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1365 on: December 30, 2016, 09:23:08 PM »
I,m curious. If on 'ML side' then both localized and the generic camera model tag must exist when this occurs. Do they? If not, should this name tag switch be done in mlv_rec rather than in the conversion tool itself you mean?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12390
  • Quarantined
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1366 on: December 30, 2016, 09:37:24 PM »
Isn't the camera model identified from Canon ID? (e.g. 0x80000287 = 60D)

From the linked thread, I guess the model string is only used as fallback (here's why); if the main method of identifying the camera is from model ID, the proper fix would be in the DNG converters (because MLV already includes the camera model ID).

Can you confirm this hypothesis?

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6461
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1367 on: December 30, 2016, 10:13:46 PM »
The canon ID tag can be multiple names for some cameras depending from where it is bought. For instance 600D.
(from exiftool Canon.pm file)
0x80000286 => 'EOS Rebel T3i / 600D / Kiss X5',
Only valid tag in acr is the Canon EOS 600D tag and should be set to the 'UniqueCameraModel' and the other camera model tag in dng spec.
It seems Magic lantern registers that localized tag probably cause there is no more tags to get? Adobe dng converter rewrites to the generic tag during conversion.

*I was wrong about adobe dng converter rewriting the tag. It actually add the localized tags present in the metadata when converting.

bouncyball put in a void part in ml-dng mlv_dump to fix this issue. Example here.
void set_unique_camera_name(mlv_idnt_hdr_t *idnt_hdr)
{
    switch(idnt_hdr->cameraModel)
    {
        case 0x80000286:
            memcpy(idnt_hdr->cameraName, "Canon EOS 600D", 32);
            break;
    }
}

eNnvi

  • New to the forum
  • *
  • Posts: 48
  • 700D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1368 on: December 31, 2016, 12:38:47 AM »
Why not Just hardcore using compiler directives for every ml camera?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1369 on: December 31, 2016, 05:35:45 AM »
Why not Just hardcore using compiler directives for every ml camera?
Why not just find and use the Canon property that reports the non-localized camera name rather than using the one we currently use that reports the localized camera name?

Isn't the camera model identified from Canon ID? (e.g. 0x80000287 = 60D)

From the linked thread, I guess the model string is only used as fallback (here's why); if the main method of identifying the camera is from model ID, the proper fix would be in the DNG converters (because MLV already includes the camera model ID).

Can you confirm this hypothesis?
There is nothing for a model ID tag in the DNG spec. There's only UniqueCameraModel and LocalizedCameraModel in the spec.

I don't like the idea of having to create and maintain a lookup table of camera names in MLFVS.

Just thinking about the MLV spec itself, wouldn't it make more sense that "camera name" metadata that the spec defines would be a non-localized name, and that every camera that is the same camera would report the same name?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12390
  • Quarantined
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1370 on: December 31, 2016, 10:40:00 AM »
Why not just find and use the Canon property that reports the non-localized camera name rather than using the one we currently use that reports the localized camera name?

I doubt there is any.

Code: [Select]
grep --binary-files=text -E "(REBEL|600D)" ROM*.BIN
ROM0.BIN:Canon EOS REBEL T3i
ROM1.BIN:Canon EOS REBEL T3i

grep --binary-files=text -E "(REBEL|1100D)" ROM*.BIN
ROM1.BIN:Canon EOS REBEL T3

How does a CR2 from a localized camera look like?

In this case, overriding the model string property (maybe using a definition in consts.h, or in platform makefiles) should work.

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6461
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1371 on: December 31, 2016, 10:49:53 AM »
I can,t find any extra id strings when checking exiftool docs. Here is a 600D CR2 file coming from here(http://www.photographyblog.com/reviews/canon_eos_600d_review/sample_images/) with local tags(Canon Model ID) shown as well as the generic one(check bottom) and with the generic(Camera Model Name) tag.

600D CR2
https://bitbucket.org/Dannephoto/magic-lantern/downloads/canon_eos_600d_01.cr2
100D dng file from mlvfs
https://bitbucket.org/Dannephoto/magic-lantern/downloads/100D_mv1080MLV_1_2016-01-27_0001_C0000_000000.dng

DNG spec docs are a bit vague(http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf). They state under localized tags:
Default  Same as UniqueCameraModel
Which indicates all is coming from one tag?

Checking the 600D with exiftool one finds all tags represented under Canon Model ID tag
Canon Model ID                  : EOS Rebel T3i / 600D / Kiss X5


a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12390
  • Quarantined
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1372 on: December 31, 2016, 11:10:41 AM »
This CR2 contains only the European model string. Are you sure the camera they reviewed reports a localized name in ML?

Code: [Select]
exiftool -htmlDump canon_eos_600d_01.cr2 > rawexif.html

= "Canon EOS 600D", no traces of REBEL or Kiss. Same with grep.

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6461
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1373 on: December 31, 2016, 11:16:09 AM »
I have no idea. I have a 600D at work. I might be able to check that. Or even better. @Paulharwood856 maye could upload a CR2 from his 600D?

*google and found this webpage. A collection of raw samples. Might be useful.
http://www.rawsamples.ch/index.php/en/canon

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12390
  • Quarantined
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #1374 on: December 31, 2016, 02:25:43 PM »
The 750D sample has "Canon EOS Rebel T6i" in EXIF, but no traces of 750D inside.