[deprecated] MLVFS windows client

[LRF]

I'll give it a try later today. Thanks.
Everything that might have reduced performance was off, as far as I remember.

[LRF]

Testing new update.
Antivir and other stuff - off
Power plan - High performance

10s MLV clip (almost static scene)
Resolve render cache off:
first play - 1 min 12s (approx 50% of black frames)
2nd - 30s (80% black frames)
3rd - 12s (only few frames visible, the rest is black)

With Smart render cache:
1st play - 24s (approx 10% of frames visible, rest is black)
2nd - 24s (few frames visible)
3rd - 24s (few frames visible)

When cursor is randomly placed on timeline, if the that frame was black during playback, remains black. Doesn't show preview.

So, in my case, too many bottlenecks to review the app properly. GTX 275 with 850MB VRAM and 7200rpm 1TB HD is OK for H264, but not in this case. Let me know if I can help in some way, but I think my results are not representative because of hardware limitations.

[g3gg0]

okay i would say anyway that this solution is not meant for realtime playback etc.
is it really necessary for post processing?

[LRF]

Hmm... I'd say that if it wouldn't be possible to have realtime playback regardless of computer parameters, then it becomes quite problematic.

If one can't see the edit or color corrected clips in realtime, usefulness of such solution is rather limited, in my opinion.

[g3gg0]

update: speeded up code to probably maximum possible speed using network sockets
now a GET request takes 27 msec, and 25 msec are "writing the DNG to the socket"

set prefetch images to 100 or more for your tests.

next optimization step: check why disk reads are somewhat slow

[LRF]

I wish I had good news, but...

Prefetch set to 150.
It took 3 minutes 7 seconds to play the clip. A lot of frozen frames, jumps, and blacks. Practically no playback at all. This time also some pink artifacts like horizontal lines and parts of the frame in stylish "pink and white" color palette 

At the end Microsoft .NET Framework has reported  unhandled exception.

************** Exception Text **************
System.ArgumentOutOfRangeException: Value of '1004' is not valid for 'Value'. 'Value' should be between 'minimum' and 'maximum'.
Parameter name: Value
   at System.Windows.Forms.ProgressBar.set_Value(Int32 value)
   at WebDAVServer.WebDAVServerForm.RefreshBar()
   at WebDAVServer.WebDAVServerForm.UpdateTimer_Tick(Object sender, EventArgs e)
   at System.Windows.Forms.Timer.OnTick(EventArgs e)
   at System.Windows.Forms.Timer.TimerNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

After that Resolve refused to cooperate. Damn 

[g3gg0]

confirmed, that the crash may happen.
fixed it.

can you open your browser, enable request logging and look for the GET ....dng requests, how long they take?

[LRF]

can you (...) enable request logging and look for the GET ....dng requests, how long they take?

If you tell me how to to it, of course.
I tried to figure out what am I supposed to do, but have no clue, sorry.

[g3gg0]

 - browser: http://localhost:8082/log
 - click [Start] at the bottom of the page
 - play some files
 - check WebDavServer window (or browser) for logged GET data

[LRF]

Here are some values from the beginning of the log:
 412,0236 ms
 470,0268 ms
 1531,0876 ms
 1585,0907 ms
 2219,1269 ms
 2263,1295 ms
 451,0258 ms
 2981,1705 ms
 5494,3143 ms
 5596,3201 ms
 2016,1153 ms
 2953,1689 ms
 596,0341 ms
 424,0242 ms
 186,0106 ms
 553,0316 ms
 10284,5883 ms

Prefetch 150, no render cache in Resolve.
Smallest value i noticed is 52,003 ms. Biggest, the last one in log, is11354,6495 ms. The numbers seem quite high. Did I win something? 

[g3gg0]

wow is that slow oO
if you view the files' properties using the explorer, what do the numbers look like?

[LRF]

If you mean MLV file i used for the test, it's 767 MB, 10 seconds 9 frames long. No color correction applied.

[g3gg0]

na, browse the virtual MLV directory where the DNGs are and do a rightclick, properties on a few DNGs one after another.
this should trigger a GET command and you will get again log entries.
tell me how long these GET requests took.

[LRF]

They look like this:

312,0006 ms
62,4001 ms
62,4001 m
62,4001 ms
62,4001 ms
78,0001 ms
78,0001 ms
78,0001 ms
78,0001 ms
15,6 ms
0 ms
31,2 ms
78,0001 ms

The file is in mapped folder on a hard drive. OS and apps are on separate disk in different partitions.

[g3gg0]

oh in this case...
your machine is way slower than mine.
i guess that is the problem

[LRF]

Yep, that's what i suspected. So far, for h264 it was more than enough. But since I started experimenting with RAW recently, looks like it's time for serious upgrade.
Anyways, thanks a lot, g3gg0.

[LRF]

I'm making some more tests, checking if location of mapped folder makes any difference. Just noticed, that times can go as low as 10ms or 20ms, and then for example 80,000 for several times, and then again 10. Will let you know if I reach any conclusions.

[g3gg0]

thanks that will help a lot.
dont have any tools that could play dng

[Canon eos m]

getting this error. what can I do

Edit: don't know how to attach pic so directly typing the error:

SW: Adobe After Effect 2014
Error: After Effects error: File exists but couldn't open for reading "\\127.0.0.1@8082\M12-1507_000022.jpg". (3::0)
File format: .jpg
Error after pressing import in Ae file menu

[Canon eos m]

Error 2: File 'H\5D3\20140912\M12-1558.m00' has an a invalid block at offset 0xB192BF34

[g3gg0]

ok will check this tomorrow.
meanwhile i've added raw FITS support (e.g. DeepSkyStacker reads such files)

[g3gg0]

major bugfixes:
 - reworked file accesses as sometimes it has thrown access exceptions on rapid accesses
 - changed buffering to butter at most 500MiB to prevent memory exceptions
 - tweaked network socket settings to squeeze out the last milisecond

@Canon eos m:
can you try again?

@all:
overall performance should be better now. can you playback now in realtime?

[LRF]

I didn't finish previous tests, so still don't have precise data, but it might be insignificant since I didn't manage to get realtime playback with previous build anyways.

With this build still no luck with realtime, but numbers seem to be lower.

Here's data from "GET" properties in Explorer:
283,0162 ms
321,0184 ms
2,0001 ms
3,0002 ms
3,0001 ms
4,0002 ms
3,0002 ms
3,0002 ms
3,0002 ms
0 ms
0 ms
0 ms
0 ms
0 ms
15,6 ms
0 ms
0 ms
0 ms

Precache set to 150.
Where there is 0 ms, I was still getting preview and properties without problem.

In Resolve it took 36 seconds for first play. But there as a lot of skipping and black frames. Second half of the clip black. Some error and .NET crash at the end, so wasn't able to collect results.

I also benchmarked the storage hard drive, and more or less it keeps manufacturer's parameters: Average Seek Time 8.9ms (10.3 in my case), Average Latency 4.17ms, Interface  SATA 3.0Gb/s, 1TB 7200 RPM, 32MB Cache.
But since it's obvious that the whole system can't cope with such amount of data, my results might be totally misleading.

[g3gg0]

@Canon eos m:
can you also provide the output of a mlv_dump -v for the file causing trouble?

[Canon eos m]

g3gg0 really sorry for not re-testing the mlvfs win version after you worked to refine it. It is already 2:06 am in Singapore and I am still pre-occupied with office work. I should be done by tonight and promise to test the refreshed version of mlvfs first thing tomorrow morning and report back to you soon after.

Apologies again.