MLVFS - a FUSE based, "on the fly" MLV to CDNG converter

[dmilligan]

You can't specify an arbitrary scaling in the metadata, it's a pixel aspect ratio. You can only specify a change in the ratio of width to height, not some absolute change in both width and height.

EDIT: there does actually appear to be "BestQualityScale" tag that might work for this purpose. Anyone is welcome to fire up exiftool and try and see if it actually works. I don't plan on implementing this specific request, but rather have been meaning to for a while provide a generic way to override/add any particular DNG metadata. Just haven't had the time (probably won't for a while, if ever).

[Danne]

...provide a generic way to override/add any particular DNG metadata

Cool.

I tried some but don,t really know if it change anything. A bit short on time but I added and changed the tag with exiv2 and then checked metadata with exiftool. Couldn,t remember how to alter scale numbers with exiftool but exiv2 works.

So if anyone wants to experiment here,s a little howto. Install both exiv2 and exiftool with homebrew . First install homebrew.
In terminal do:

Code: [Select]

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" then do:

Code: [Select]

brew install exiv2

Code: [Select]

brew install exiftool
To simply add the "BestQualityScale" tag to a dng file do:

Code: [Select]

exiv2 -M"set Exif.Image.BestQualityScale Rational 1/1" INPUT.dngThen maybe try:

Code: [Select]

exiv2 -M"set Exif.Image.BestQualityScale Rational 5/3" INPUT.dngor maybe

Code: [Select]

exiv2 -M"set Exif.Image.BestQualityScale Rational 7/3" INPUT.dng
Read more on the tag here:
http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec.pdf

[COMMANDES]

Hello. MLVFS is a very convenient thing, but for some reason it does not load the correction profiles of phase AF points when opening files written by "new" versions of ML.

To files from 3.07.2016 the profiles are still loaded, but to files written with "new" versions of ML - no.

Camera 650D
OS Windows 8.1

[dmilligan]

You're going to have to be more precise than just saying "new" and probably provide samples.

[COMMANDES]

So, at the opening sequence MLV-file (shot 07.03.2016, the firmware was up-to-date at the time) mounted in MLVFS console appears "Loading focus pixel map '80000301_1872x1060.fpm' ..." and no AF points.

When opening files with firmware from 6.01.2017 (and later) to 10/14 bit, no information is displayed in the console, and the files open in Davinci with AF points.

[dfort]

There was a July 9 build and a June 13 build. You can find them here:

https://builds.magiclantern.fm/650D-104-all-builds-changes.html

Run a test with these two builds and report back whether or not you can reproduce the issue. (Good luck--I doubt that's the problem.)

It would help if you could post a short MLV file that MLVFS can't remove the focus pixels. You said that it is happening with the later builds so it would be best if you use the newest nightly (unified) build if you want to record a new MLV file. You mentioned 10/14 bit so maybe you've been using an experimental build?

[COMMANDES]

My results:

• Build on Jun13:
ok, no AF
• Build on Jul09:
ok, no AF
• Build on Mar30-2017:
bad, AF

By the link: in the build folder: jpeg file + dng file (from MLVFS) and original MLV

[dfort]

It would really help if you can find the exact build when this first started happening. Start with the build right after July 9.

Is there another 650D user reading this that can reproduce the issue?


Sent from my iPhone using Tapatalk

[dmilligan]

As I suspected, the problem is the raw buffer size changed (from 1808x1190 to 1808x1189). MLVFS detects the map to use based on the camera id and the raw buffer size. All you need to do is duplicate the file 80000301_1808x1190.fpm and rename it to 80000301_1808x1189.fpm.

With the new RAWC block and crop_rec stuff we can (and will have to) come up with a little better system for getting the correct map.

[COMMANDES]

Thank you so much. And what about a crop mode/others resolutions?

[dmilligan]

You tell me. I don't own a 650D.

FYI when I say resolution, I'm not talking about the resolution you seleceted in the menu, rather the full (max) resolution of the current video mode. Since there is not (yet) metadata about what the video mode and crop settings are, we have to guess the video mode based on the full raw buffer resolution (each video mode has a unique raw buffer resolution currently, though that is no longer true with crop_rec and the new "4K" stuff)

[COMMANDES]

Based on personal experience I would say that the other profiles are used for crop mode.

But I do not know the buffer-size to rename files.

[dmilligan]

mlv_dump -v

[COMMANDES]

Thank you, I've fixed everything I needed modes.

Based on this block:

Code: [Select]

Block: RAWI
  Offset: 0x00000034
    Size: 180
    Time: 209.197000 ms
    Res:  1920x640
    raw_info:
      api_version      0x00000001
      height           1107
      width            2592
      pitch            3240
      frame_size       0x004C9EA8
      bits_per_pixel   10
      black_level      127
      white_level      937
      active_area.y1   28
      active_area.x1   74
      active_area.y2   1101
      active_area.x2   2592
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1

[dfort]

As I suspected, the problem is the raw buffer size changed (from 1808x1190 to 1808x1189). MLVFS detects the map to use based on the camera id and the raw buffer size. All you need to do is duplicate the file 80000301_1808x1190.fpm and rename it to 80000301_1808x1189.fpm.

With the new RAWC block and crop_rec stuff we can (and will have to) come up with a little better system for getting the correct map.

Right, Danne ran into that too so he did some adjustments that allowed for a little slop in the vertical resolution. Fortunately is isn't shifting the coordinates of the focus pixels so it still works.

Code: [Select]

  mv1080)
    raw_buffer=1808x11**
  mv720)
    raw_buffer=1808x72*
  mv1080crop)
    raw_buffer=1872x10**
  zoom)
    raw_buffer=2592x110*

Not sure how that would apply in MLVFS but there's also the crop_rec module that requires a different FPM file that's the same size as the mv720 raw buffer. I experimented with combining the map files but it didn't work so we'll have to deal with this at some point. Isn't there new metadata for crop_rec that can be used?

[festr]

Hi Dmiligan, do you plan to integrate the new lossless raw compression?

https://www.magiclantern.fm/forum/index.php?topic=18443.msg183653#msg183653

[ludzik3d]

Hi
I want to try MLVFS but I get an error message "dokanfuse.dll is missing, please reinstall dokany.
I use Windows 10 64bit.
I tried everything what I found on the forum and in the manuals but it still doesn't work

[TrEK]

Hello.
I was make MLVSF





But when i try put one folder on the timeline Adobe Premiere cc 2015 i receive such result:





CDNG sequence duplicate such times as how available quanity of dng-pictures....

and i need put first dng-picture on timeline for avoid it:





in such way its all correct...

what i can do for ability put all folder from BIN to timeline in one time ?

[dmilligan]

Merged @bouncyball's lj92 stuff, and updated the Mac build.

[squig]

Only problem is, at 7.7MB per frame, I can't get real time 3K playback in Resolve with MLVFS (all options off) even on a relatively fast Mac (3.33 GHz 6 Core 48GB ram SSD/raid GTX-980. Tested on Yosemite Resolve 12.5 and El Capitan Resolve 14.

The same MLV file converted with MLV_dump (6.6MB per frame) plays back smoothly in Resolve.

[bouncyball]

The same MLV file converted with MLV_dump (6.6MB per frame) plays back smoothly in Resolve.

Have you tried playback, with thouse 7.7MB MLVFS DNGs copied to HDD? If playback is OK than definitely (actually I'm quite confident about it) decompression overhead is an issue. If Canon HW compressor would produce multi slice (not the same as multi component) RAW data, decompression could be multi threaded per slice and hence be faster.

Unfortunatelly I'm quite far from this kind of programming (MLVFS actually is multi threaded app in its core) and can't say anything specific about making this portion of code multi threaded.

P.S. What is the CPU load during playback? (is it hackintosh?)

[a1ex]

If Canon HW compressor would produce multi slice (not the same as multi component) RAW data, decompression could be multi threaded per slice and hence be faster.

It actually does (2 slices).

[squig]

Have you tried playback, with thouse 7.7MB MLVFS DNGs copied to HDD? If playback is OK than definitely (actually I'm quite confident about it) decompression overhead is an issue.

P.S. What is the CPU load during playback? (is it hackintosh?)

Mac Pro 2010. CPU load with MLVFS running is around 27%. 14-19% with the copied file. The copied file plays back smoothly.

[bouncyball]

@squig

Can you uncompress that high resolution MLV (mlv_dump -o uncompressed_dst.mlv compressed_src.mlv) and try to play in MLVFS?

[squig]

@squig

Can you uncompress that high resolution MLV (mlv_dump -o uncompressed_dst.mlv compressed_src.mlv) and try to play in MLVFS?

Sure, but you're gonna have to give me the exact command. The file name is M01-2316.MLV