MLVFS - a FUSE based, "on the fly" MLV to CDNG converter

Started by dmilligan, August 31, 2014, 02:01:24 AM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

dmilligan

For full dual ISO processing it usually takes about 1.5 seconds per frame on my computer. If you turn AMaZE off, it's usually about half that or faster. CDNG doesn't take any extra time. I also haven't done a speed comparison with cr2hdr. The dual ISO preview should be realtime or close to realtime.

Danne

Ok, thanks. It seems about right. I,ll try to do some comparison. The issue would be the lagging effect which I,m gonna try to narrow down.

dmilligan

The reason it's "lagging" is because the Finder tries to load the file to show you a preview, which causes MLVFS to do a full render. This takes several seconds if you have dual ISO "Full" on. I would recommend using a Finder view that doesn't show previews, or avoid browsing around in the Finder once you turn the dual ISO on "Full".

I did some investigating on the speed issues as well. On a test image MLVFS is taking about 6.5 seconds to render a single frame, while cr2hdr20bit only takes about 3. Of the 6.5 seconds, 5.2 is spent doing the AMaZE interpolation. If you use mean32 instead of AMaZE, it only takes 1.5 seconds and the mean32 interpolation part only takes 0.2s. I don't really know why MLVFS's implementation is so much slower, could be a compiler optimization thing, who knows.

Danne

Thanks for sharing. Maybe implement an cr2hdr20bit export option with possibility to multithread? Might be messy and not in philosophy of the mlvfs workflow but it might speed up things a bit.

swinxx

@dmilligan

would it be possible to implement an "EJECT" button in the webserver?
that would be handy, cause in the finder that eject is very laggy and non responsive.

Update:
Sorry Mate, i have not understood the dual iso thing completely.. so my question:
would it be a good idea, to let mlvfs search for a folder called "dualiso" and let only those mlvs be processed like dual iso mlvs? i ask this because dual iso conversion is a slow process and when those shots are placed in a special folder within the mounted folder, mlvfs could speed up conversion process...? no?

thank you.

Audionut

Are the SSE2 additions to AMaZE working as expected?

dmilligan

While AMaZE itself is somewhat slower, the vast majority of the slowdown appears to be coming from cr2hdr code that does stuff with the AMaZE output, not AMaZE itself. IDK if that stuff is being vectorized or can be. Perhaps I should try compiling MLVFS with GCC and see if it makes any difference.

Andrzej

Hi, i am geting: "safari can't connect to server"
sys wer 10.7.5 security settings changed.
safari 6.1.6
any suggestions?
Canon 5dm3, 550d, Canon 24-105mm 4.0 IS, Canon 18-55mm 3.5-5.6 IS, Sigma 30mm 1.4, Sigma 20mm 1.8, Sigma 70-300mm 4.0-5.6, Samyang 14mm 2.8, Falcon 85mm 1.4, Zeiss 180mm 2.8 and more...

swinxx

Perhaps a problem with write permissions of the drive with the mounting point.. I had that problem after updating mac osx to yosemite, just change it that everyone can read and write. Greets

ayshih

Quote from: Markus on November 14, 2014, 05:00:31 PM
This is super cool =). I've been trying it out on windows with pismo file mount program. Is it possible to control hotpixel fix and stripe removal in windows?
Not yet, no, but the other demands on my attention have largely subsided, so I should be able to spend time on ML/MLVFS again.  Stay tuned!
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

dmilligan

Quote from: Andrzej on November 17, 2014, 06:05:02 AM
Hi, i am geting: "safari can't connect to server"
Is it just the webserver that doesn't work? (does the mount still show up?) Or does it not work at all?

Andrzej

Thanks for trying to help :)

Quote from: swinxx on November 17, 2014, 08:02:46 AM
Perhaps a problem with write permissions of the drive with the mounting point.. I had that problem after updating mac osx to yosemite, just change it that everyone can read and write. Greets
Changed that, still the same.

Quote from: dmilligan on November 17, 2014, 01:44:34 PM
Is it just the webserver that doesn't work? (does the mount still show up?) Or does it not work at all?
not work at all. MLVFS service is visible under "services"

Maybe i do something wrong with the installation proces? Is this method correct? https://www.youtube.com/watch?v=vNO4ks5Xgn8
Canon 5dm3, 550d, Canon 24-105mm 4.0 IS, Canon 18-55mm 3.5-5.6 IS, Sigma 30mm 1.4, Sigma 20mm 1.8, Sigma 70-300mm 4.0-5.6, Samyang 14mm 2.8, Falcon 85mm 1.4, Zeiss 180mm 2.8 and more...

silvertonesx24

I also get the cannot "connect to server error as well", quite often. The problem seems to be when the operation chokes on certain MLV files. There is no indication of why it chokes on these files, and the only way I can tell is by adding groups of MLV files to a folder to find the one that eventually crashes FUSE.

A crash log is below, let me know if you need any more
http://1drv.ms/1wO7Ny6

Markus

ayshih:
Really looking forward to the possibility to get the webserver options controlable on Windows

dmilligan

@silvertonesx24
divide by zero error on this line:

result = sizeof(struct wav_header) + (uint64_t)mlv_wavi_hdr.bytesPerSecond * (uint64_t)mlv_file_hdr.sourceFpsDenom * (uint64_t)mlv_get_frame_count(path) / (uint64_t)mlv_file_hdr.sourceFpsNom;


Which means that the FPS metadata in the file is either wrong, or being read incorrectly. I can prevent the error, but your FPS is still going to be wrong. It's worth investigating *why* the data is bad rather than just suppressing the error (could be an ML issue). Is there anyway you can send a file that has this issue? or perhaps at least the output of mlv_dump -v

dmilligan


swinxx


DeafEyeJedi

Will test it out tonight as soon as I get home from work. Thanks again @dmilligan!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

swinxx

@dmilligan:
i have just converted some DI Shots with MLVFS, and wanted to inform you that in mac osx 10.10 the finder does not show the image correctly. here is a link to one mlvfs converted di image.
thank you very much, great app.
https://copy.com/aCMp94TfSCCF8jK2

sw

ayshih

Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

swinxx

of course.
when hitting the space bar, finder shows a preview.
this previewed dng is pink/white. there is a screenshot in the upper link.
thx.

dmilligan

Not much to do about that. It's a known bug with the Mac Finder, it uses a hard coded black level of 2048 and ignores the black level in the file (which is different because of dual ISO processing that actually utilizes 16 bit). Tell Apple to fix their software.

swinxx

ah. ok.
i just wanted to report, cause all other previewed files work as expected

thx

Danne

Hi David! I have a question.
I somehow managed to create a davinci resolve naming scheme to cr2hdr-r following your example and naming scheme in MLVFS. Without audio the files appears as it should with no need to enter the folders but when including audio of course you have to enter the folder and audio and movie files are separated.
using exiftool I can put in the TimeCodes: 00:00:00.00 in my DNG:s but that alone don,t seem to do the trick. When I view the RIFF info in your wav file I see TimeReference: 0 amongst other info that I don,t find in my wav file. Trying to changing/adding metadata in RIFF section in my wav file seems pretty hard. Don,t really get exiftool to apply here. Would you know how maybe?
Would be really nice to hear what you have to say about this metadata issue and maybe point me in some direction how to solve this?
Using your audio file coming from MLVFS works together with my converted DNG files so changing metadata in wav somehow should do the trick. But how :)?
Update* Used a tool called BWF Metaedit which lets me change metadata but couldn,t really copy the order of things. Still something missing but can,t find out what, hmm

dmilligan

IDK if you'll be able to do it any way besides actually writing code to modify the WAV file itself directly. For reference on what all you need in the WAV file and how to do it see wav.c in the MLVFS source.