Author Topic: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter  (Read 924805 times)

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #300 on: November 14, 2014, 07:28:09 PM »
For full dual ISO processing it usually takes about 1.5 seconds per frame on my computer. If you turn AMaZE off, it's usually about half that or faster. CDNG doesn't take any extra time. I also haven't done a speed comparison with cr2hdr. The dual ISO preview should be realtime or close to realtime.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #301 on: November 14, 2014, 07:30:43 PM »
Ok, thanks. It seems about right. I,ll try to do some comparison. The issue would be the lagging effect which I,m gonna try to narrow down.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #302 on: November 15, 2014, 04:19:27 PM »
The reason it's "lagging" is because the Finder tries to load the file to show you a preview, which causes MLVFS to do a full render. This takes several seconds if you have dual ISO "Full" on. I would recommend using a Finder view that doesn't show previews, or avoid browsing around in the Finder once you turn the dual ISO on "Full".

I did some investigating on the speed issues as well. On a test image MLVFS is taking about 6.5 seconds to render a single frame, while cr2hdr20bit only takes about 3. Of the 6.5 seconds, 5.2 is spent doing the AMaZE interpolation. If you use mean32 instead of AMaZE, it only takes 1.5 seconds and the mean32 interpolation part only takes 0.2s. I don't really know why MLVFS's implementation is so much slower, could be a compiler optimization thing, who knows.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #303 on: November 15, 2014, 04:32:40 PM »
Thanks for sharing. Maybe implement an cr2hdr20bit export option with possibility to multithread? Might be messy and not in philosophy of the mlvfs workflow but it might speed up things a bit.

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #304 on: November 15, 2014, 10:11:11 PM »
@dmilligan

would it be possible to implement an "EJECT" button in the webserver?
that would be handy, cause in the finder that eject is very laggy and non responsive.

Update:
Sorry Mate, i have not understood the dual iso thing completely.. so my question:
would it be a good idea, to let mlvfs search for a folder called "dualiso" and let only those mlvs be processed like dual iso mlvs? i ask this because dual iso conversion is a slow process and when those shots are placed in a special folder within the mounted folder, mlvfs could speed up conversion process...? no?

thank you.

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #305 on: November 15, 2014, 11:19:23 PM »
Are the SSE2 additions to AMaZE working as expected?

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #306 on: November 15, 2014, 11:59:22 PM »
While AMaZE itself is somewhat slower, the vast majority of the slowdown appears to be coming from cr2hdr code that does stuff with the AMaZE output, not AMaZE itself. IDK if that stuff is being vectorized or can be. Perhaps I should try compiling MLVFS with GCC and see if it makes any difference.

Andrzej

  • New to the forum
  • *
  • Posts: 33
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #307 on: November 17, 2014, 06:05:02 AM »
Hi, i am geting: "safari can't connect to server"
sys wer 10.7.5 security settings changed.
safari 6.1.6
any suggestions?
Canon 5dm3, 550d, Canon 24-105mm 4.0 IS, Canon 18-55mm 3.5-5.6 IS, Sigma 30mm 1.4, Sigma 20mm 1.8, Sigma 70-300mm 4.0-5.6, Samyang 14mm 2.8, Falcon 85mm 1.4, Zeiss 180mm 2.8 and more...

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #308 on: November 17, 2014, 08:02:46 AM »
Perhaps a problem with write permissions of the drive with the mounting point.. I had that problem after updating mac osx to yosemite, just change it that everyone can read and write. Greets

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #309 on: November 17, 2014, 01:43:03 PM »
This is super cool =). I've been trying it out on windows with pismo file mount program. Is it possible to control hotpixel fix and stripe removal in windows?
Not yet, no, but the other demands on my attention have largely subsided, so I should be able to spend time on ML/MLVFS again.  Stay tuned!
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #310 on: November 17, 2014, 01:44:34 PM »
Hi, i am geting: "safari can't connect to server"
Is it just the webserver that doesn't work? (does the mount still show up?) Or does it not work at all?

Andrzej

  • New to the forum
  • *
  • Posts: 33
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #311 on: November 17, 2014, 03:31:25 PM »
Thanks for trying to help :)

Perhaps a problem with write permissions of the drive with the mounting point.. I had that problem after updating mac osx to yosemite, just change it that everyone can read and write. Greets
Changed that, still the same.

Is it just the webserver that doesn't work? (does the mount still show up?) Or does it not work at all?
not work at all. MLVFS service is visible under "services"

Maybe i do something wrong with the installation proces? Is this method correct?
Canon 5dm3, 550d, Canon 24-105mm 4.0 IS, Canon 18-55mm 3.5-5.6 IS, Sigma 30mm 1.4, Sigma 20mm 1.8, Sigma 70-300mm 4.0-5.6, Samyang 14mm 2.8, Falcon 85mm 1.4, Zeiss 180mm 2.8 and more...

silvertonesx24

  • Freshman
  • **
  • Posts: 73
  • 5Dii
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #312 on: November 17, 2014, 04:26:08 PM »
I also get the cannot "connect to server error as well", quite often. The problem seems to be when the operation chokes on certain MLV files. There is no indication of why it chokes on these files, and the only way I can tell is by adding groups of MLV files to a folder to find the one that eventually crashes FUSE.

A crash log is below, let me know if you need any more
http://1drv.ms/1wO7Ny6

Markus

  • Senior
  • ****
  • Posts: 338
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #313 on: November 17, 2014, 07:50:21 PM »
ayshih:
Really looking forward to the possibility to get the webserver options controlable on Windows

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #314 on: November 17, 2014, 10:40:02 PM »
@silvertonesx24
divide by zero error on this line:
Code: [Select]
result = sizeof(struct wav_header) + (uint64_t)mlv_wavi_hdr.bytesPerSecond * (uint64_t)mlv_file_hdr.sourceFpsDenom * (uint64_t)mlv_get_frame_count(path) / (uint64_t)mlv_file_hdr.sourceFpsNom;

Which means that the FPS metadata in the file is either wrong, or being read incorrectly. I can prevent the error, but your FPS is still going to be wrong. It's worth investigating *why* the data is bad rather than just suppressing the error (could be an ML issue). Is there anyway you can send a file that has this issue? or perhaps at least the output of mlv_dump -v

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #315 on: November 18, 2014, 11:18:27 PM »
updated with some bug fixes (hopefully)

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #316 on: November 18, 2014, 11:51:11 PM »
thank you.
will test tomorrow.

greets.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #317 on: November 19, 2014, 02:47:22 AM »
Will test it out tonight as soon as I get home from work. Thanks again @dmilligan!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #318 on: November 19, 2014, 09:28:10 PM »
@dmilligan:
i have just converted some DI Shots with MLVFS, and wanted to inform you that in mac osx 10.10 the finder does not show the image correctly. here is a link to one mlvfs converted di image.
thank you very much, great app.
https://copy.com/aCMp94TfSCCF8jK2

sw

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #319 on: November 19, 2014, 10:20:57 PM »
finder does not show the image correctly
Can you elaborate?
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #320 on: November 19, 2014, 10:26:48 PM »
of course.
when hitting the space bar, finder shows a preview.
this previewed dng is pink/white. there is a screenshot in the upper link.
thx.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #321 on: November 19, 2014, 10:45:37 PM »
Not much to do about that. It's a known bug with the Mac Finder, it uses a hard coded black level of 2048 and ignores the black level in the file (which is different because of dual ISO processing that actually utilizes 16 bit). Tell Apple to fix their software.

swinxx

  • Hero Member
  • *****
  • Posts: 606
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #322 on: November 20, 2014, 02:32:06 PM »
ah. ok.
i just wanted to report, cause all other previewed files work as expected

thx

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7585
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #323 on: November 23, 2014, 09:15:09 PM »
Hi David! I have a question.
I somehow managed to create a davinci resolve naming scheme to cr2hdr-r following your example and naming scheme in MLVFS. Without audio the files appears as it should with no need to enter the folders but when including audio of course you have to enter the folder and audio and movie files are separated.
using exiftool I can put in the TimeCodes: 00:00:00.00 in my DNG:s but that alone don,t seem to do the trick. When I view the RIFF info in your wav file I see TimeReference: 0 amongst other info that I don,t find in my wav file. Trying to changing/adding metadata in RIFF section in my wav file seems pretty hard. Don,t really get exiftool to apply here. Would you know how maybe?
Would be really nice to hear what you have to say about this metadata issue and maybe point me in some direction how to solve this?
Using your audio file coming from MLVFS works together with my converted DNG files so changing metadata in wav somehow should do the trick. But how :)?
Update* Used a tool called BWF Metaedit which lets me change metadata but couldn,t really copy the order of things. Still something missing but can,t find out what, hmm

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: MLVFS - a FUSE based, "on the fly" MLV to CDNG converter
« Reply #324 on: November 24, 2014, 12:13:30 AM »
IDK if you'll be able to do it any way besides actually writing code to modify the WAV file itself directly. For reference on what all you need in the WAV file and how to do it see wav.c in the MLVFS source.