[deprecated] MLVFS windows client

Started by g3gg0, September 11, 2014, 03:54:18 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

LRF

I'll give it a try later today. Thanks.
Everything that might have reduced performance was off, as far as I remember.

LRF

Testing new update.
Antivir and other stuff - off
Power plan - High performance

10s MLV clip (almost static scene)
Resolve render cache off:
first play - 1 min 12s (approx 50% of black frames)
2nd - 30s (80% black frames)
3rd - 12s (only few frames visible, the rest is black)

With Smart render cache:
1st play - 24s (approx 10% of frames visible, rest is black)
2nd - 24s (few frames visible)
3rd - 24s (few frames visible)

When cursor is randomly placed on timeline, if the that frame was black during playback, remains black. Doesn't show preview.

So, in my case, too many bottlenecks to review the app properly. GTX 275 with 850MB VRAM and 7200rpm 1TB HD is OK for H264, but not in this case. Let me know if I can help in some way, but I think my results are not representative because of hardware limitations.

g3gg0

okay i would say anyway that this solution is not meant for realtime playback etc.
is it really necessary for post processing?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

LRF

Hmm... I'd say that if it wouldn't be possible to have realtime playback regardless of computer parameters, then it becomes quite problematic.

If one can't see the edit or color corrected clips in realtime, usefulness of such solution is rather limited, in my opinion.

g3gg0

update: speeded up code to probably maximum possible speed using network sockets :)
now a GET request takes 27 msec, and 25 msec are "writing the DNG to the socket"

set prefetch images to 100 or more for your tests.

next optimization step: check why disk reads are somewhat slow
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

LRF

I wish I had good news, but...

Prefetch set to 150.
It took 3 minutes 7 seconds to play the clip. A lot of frozen frames, jumps, and blacks. Practically no playback at all. This time also some pink artifacts like horizontal lines and parts of the frame in stylish "pink and white" color palette  ;)

At the end Microsoft .NET Framework has reported  unhandled exception.

************** Exception Text **************
System.ArgumentOutOfRangeException: Value of '1004' is not valid for 'Value'. 'Value' should be between 'minimum' and 'maximum'.
Parameter name: Value
   at System.Windows.Forms.ProgressBar.set_Value(Int32 value)
   at WebDAVServer.WebDAVServerForm.RefreshBar()
   at WebDAVServer.WebDAVServerForm.UpdateTimer_Tick(Object sender, EventArgs e)
   at System.Windows.Forms.Timer.OnTick(EventArgs e)
   at System.Windows.Forms.Timer.TimerNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

After that Resolve refused to cooperate. Damn  :D

g3gg0

confirmed, that the crash may happen.
fixed it.

can you open your browser, enable request logging and look for the GET ....dng requests, how long they take?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

LRF

Quote from: g3gg0 on September 18, 2014, 11:42:04 PM
can you (...) enable request logging and look for the GET ....dng requests, how long they take?

If you tell me how to to it, of course.
I tried to figure out what am I supposed to do, but have no clue, sorry.

g3gg0

 - browser: http://localhost:8082/log
- click [Start] at the bottom of the page
- play some files
- check WebDavServer window (or browser) for logged GET data

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

LRF

Here are some values from the beginning of the log:
412,0236 ms
470,0268 ms
1531,0876 ms
1585,0907 ms
2219,1269 ms
2263,1295 ms
451,0258 ms
2981,1705 ms
5494,3143 ms
5596,3201 ms
2016,1153 ms
2953,1689 ms
596,0341 ms
424,0242 ms
186,0106 ms
553,0316 ms
10284,5883 ms

Prefetch 150, no render cache in Resolve.
Smallest value i noticed is 52,003 ms. Biggest, the last one in log, is11354,6495 ms. The numbers seem quite high. Did I win something?  ;)

g3gg0

wow is that slow oO
if you view the files' properties using the explorer, what do the numbers look like?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

LRF

If you mean MLV file i used for the test, it's 767 MB, 10 seconds 9 frames long. No color correction applied.

g3gg0

na, browse the virtual MLV directory where the DNGs are and do a rightclick, properties on a few DNGs one after another.
this should trigger a GET command and you will get again log entries.
tell me how long these GET requests took.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

LRF

They look like this:

312,0006 ms
62,4001 ms
62,4001 m
62,4001 ms
62,4001 ms
78,0001 ms
78,0001 ms
78,0001 ms
78,0001 ms
15,6 ms
0 ms
31,2 ms
78,0001 ms

The file is in mapped folder on a hard drive. OS and apps are on separate disk in different partitions.

g3gg0

oh in this case...
your machine is way slower than mine.
i guess that is the problem :(
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

LRF

Yep, that's what i suspected. So far, for h264 it was more than enough. But since I started experimenting with RAW recently, looks like it's time for serious upgrade.
Anyways, thanks a lot, g3gg0.

LRF

I'm making some more tests, checking if location of mapped folder makes any difference. Just noticed, that times can go as low as 10ms or 20ms, and then for example 80,000 for several times, and then again 10. Will let you know if I reach any conclusions.

g3gg0

thanks that will help a lot.
dont have any tools that could play dng :)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Canon eos m



getting this error. what can I do

Edit: don't know how to attach pic so directly typing the error:

SW: Adobe After Effect 2014
Error: After Effects error: File exists but couldn't open for reading "\\127.0.0.1@8082\M12-1507_000022.jpg". (3::0)
File format: .jpg
Error after pressing import in Ae file menu
Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)

Canon eos m

Error 2: File 'H\5D3\20140912\M12-1558.m00' has an a invalid block at offset 0xB192BF34

Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)

g3gg0

ok will check this tomorrow.
meanwhile i've added raw FITS support (e.g. DeepSkyStacker reads such files)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

g3gg0

major bugfixes:
- reworked file accesses as sometimes it has thrown access exceptions on rapid accesses
- changed buffering to butter at most 500MiB to prevent memory exceptions
- tweaked network socket settings to squeeze out the last milisecond ;)

@Canon eos m:
can you try again?

@all:
overall performance should be better now. can you playback now in realtime?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

LRF

I didn't finish previous tests, so still don't have precise data, but it might be insignificant since I didn't manage to get realtime playback with previous build anyways.

With this build still no luck with realtime, but numbers seem to be lower.

Here's data from "GET" properties in Explorer:
283,0162 ms
321,0184 ms
2,0001 ms
3,0002 ms
3,0001 ms
4,0002 ms
3,0002 ms
3,0002 ms
3,0002 ms
0 ms
0 ms
0 ms
0 ms
0 ms
15,6 ms
0 ms
0 ms
0 ms

Precache set to 150.
Where there is 0 ms, I was still getting preview and properties without problem.

In Resolve it took 36 seconds for first play. But there as a lot of skipping and black frames. Second half of the clip black. Some error and .NET crash at the end, so wasn't able to collect results.

I also benchmarked the storage hard drive, and more or less it keeps manufacturer's parameters: Average Seek Time 8.9ms (10.3 in my case), Average Latency 4.17ms, Interface  SATA 3.0Gb/s, 1TB 7200 RPM, 32MB Cache.
But since it's obvious that the whole system can't cope with such amount of data, my results might be totally misleading.

g3gg0

@Canon eos m:
can you also provide the output of a mlv_dump -v for the file causing trouble?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Canon eos m

g3gg0 really sorry for not re-testing the mlvfs win version after you worked to refine it. It is already 2:06 am in Singapore and I am still pre-occupied with office work. I should be done by tonight and promise to test the refreshed version of mlvfs first thing tomorrow morning and report back to you soon after.

Apologies again.
Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)