Messages

@dmilligan: Thanks for checking the files and for the explanation.

I just started to play again with the MLVFS on my freshly reinstalled system.
- OS: Windows 7 Ultimate 64bit
- Pismo File Mount version: 2014.7.11
- mlvfs.dll version: latest from bitbucket (2014-10-03)
- AE CC 2014 version: 13.2.0.49 (2014.2 Release)
- ML: Nightly.2015.Jan15.550D109, mercurial changeset 4fff9f6cb33f (unified) tip

I can't import the DNG sequence into AE. After I adjust the settings in ACR (version 8.7.1) during the import process and hit OK, it gives me an error, saying:
"After Effects error: The file format module could not parse the file 45::35"

This happens only with my new MLV file (recorded today). If I import an older MLV file, which was recorded with a nightly build from previous year (I don't know the date), then everything works fine.

Somebody could help me out with checking the MLV files?

Update #21: AXIOM Beta 4K RAW PC recording option

Spread the word!

I like the result. You have used pretty strong denoising on chroma, right?

Yes, here are the ACR settings, I didn't touch the other tabs. I'm sure that similar settings in UFRaw can be done (I don't know that program), so we could get flat DNGs, but at first the best solution for FPN & pink/purple corrections have to be done somehow. As I see your & other ML developers' & Axiom team's knowledge it will be solved quite fast.

I see some tweaks made in ACR with it: lets say, Exposure +0.7, Contrast -100 and so on. I didn't see any color related tweaks, except Saturation -35(By the way, -100 looks much better ). Did i miss something? What does it mean "graded in AE + added some grain" and where is the result?
Regarding image itself - it's probably captured under harsh lighting condition with wrong exposure. I hope it's not a sensor failure/limitation.

Well I tried to get a flat frame before I import into AE. I'm not an expert colorist, but as I read and experienced, it's the best starting point in the grading process. The best would be, if the Axiom could process eg. something called MagicLOG (flat) frames straight into DNG (MLV) and after that people could edit it as they want (save presets etc.).

So I imported the ACR "tweaked" flat DNG into AE and I tweaked again on the timeline (contrast, curve, color, saturation, grain, etc.) with a preset, so simple. The result is there, the JPG.

Yes I think that too, that the original DNG (read the #92 post in this topic!) wasn't exposed correctly, but a1ex could get some more information back with his algorithm, so I used the last processed one from him.

Feel free to push them and show your results, but don't expect to be as clean as a dual ISO image

If the pink/purple things can be fixed during the building up the DNG file, then this will be wild and it will rock!

last video (in under 1 minute) doesn't impress me. Aggressive sound, flickering, awful skin tones. I'd like to see something like that: http://vimeo.com/87374695

Don't forget, that this is an open developement and I hope the international community behind this will achieve that quality what you expect.

Yes, I want the same blond into the sensor. 

But the reddish-brown with the OPEN decolletage will be as good as the blondes, you can choose later...

I pushed a1ex's "image02-fpnfix-hilite.DNG" in ACR and graded in AE + added some grain:
https://www.dropbox.com/sh/g7lfrm7ccc1g063/AADHVFTBhcOT9MpHwQBNCCwfa?dl=0

I just informed my friend about the project, he is an economic columnist at one of the largest online news portal of Hungary. Let's see what happens.

Your batch scripts are very effective/useful, they speed up the workflow. Thanks!

I recommend it for everyone, it's not hard to setup.

Running ffmpeg command line should tell you what it's receiving ie: yuv420p etc etc

Yes, I thought the same, but ffmpeg gives an error:

Code Select Expand

[avi @ 0000000002ad88e0] Could not find codec parameters for stream 0 (Video: no
ne (DFSC / 0x43534644), 1920x818): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
signpost.avi: could not find codec parameters
Input #0, avi, from 'signpost.avi':
  Duration: 00:00:02.40, start: 0.000000, bitrate: 6 kb/s
    Stream #0:0: Video: none (DFSC / 0x43534644), 1920x818, 25 fps, 25 tbr, 25 t
bn, 25 tbc
No decoder for stream #0:0, filtering impossible
Error opening filters!

and MPC-HC's MediaInfo shows this:

Code Select Expand

General
Format                         : AVI
Format/Info                    : Audio Video Interleave
File size                      : 1.90 KiB
Duration                       : 2s 400ms
Overall bit rate               : 6 493 bps

Video
ID                             : 0
Format                         : VFW
Codec ID                       : DFSC
Codec ID/Info                  : DebugMode FrameServer VFW
Duration                       : 2s 400ms
Bit rate                       : 800 bps
Width                          : 1 920 pixels
Height                         : 818 pixels
Display aspect ratio           : 2.35:1
Frame rate                     : 25.000 fps
Bits/(Pixel*Frame)             : 0.000
Stream size                    : 240 Bytes (12%)

I just took a look the PP CS5.5 SDK site and well it's 8bit, because the AFS is written for CS5.5, but works in CS6 too.
The CC version supports RGB 444 10bit too.

Maybe just invest time in learning the basics of Resolve instead of flogging a dead horse.

But it's a mutant donkey and still alive!!! Let's jump on the pegasus!

But you've shifted your direction now, no longer talking about 10bit output maintaining quality and you're not getting 10bit still.

You have right, I just wrote that it can be used for this kind of monitoring and convert to different formats. I said, I'm not an expert.

But back again to 10 bit. How could I test the AFS's AVI file bit depth? Of course it would be the best if someone could jump into this (to find a way to build up a 16/12/10 bit frame server):
Instructions on how to setup and build the Debugmode Frameserver: https://code.google.com/p/frame-server/wiki/BuildInstructions
Premiere Pro CS6 SDK: http://www.adobe.com/devnet/premiere/sdk/cs6.html
Premier Pro CC SDK: http://www.adobe.com/devnet/premiere/sdk/cc.html (Other changes is important here.)

Anyway, this curve test came to my mind because of this. Earlier I made a test with the forced 16 bit setting. I got a MOV file, but I could rewrap the information from MOV to AVI, without quality lost. In theory the AVI files can contain only 8 bit things not? So how is this again?

Do you have sound with your playback? Are you watching an edit? What resolution and duration are your clips off a 550d? Will it work just as well for 5D MKIII?

I don't have sound, I watched a footage, what was graded by ACR + contrast + saturation in AE, the resolution is 1920x818, 2-3 seconds long and if you cache all the frames, then it should work for longer footages too. I think the best would be for this an SSD, eg.: 64 GB just only for cache.

In Resolve Lite I can't grade the DNG seqeunce yet, the white balance is not ok and if I start to play with the settings everything will be worse. It hardly can be that this comes from my gappy knowledge. Resolve is totally new for me.

Try playing with something like this:  https://dl.dropboxusercontent.com/u/34113196/ML/_MG_4821.DNG

I just downloaded your file and droped this curve on it, the DNG, PNG are equal, the JPG seems a bit "sharper?" or how I say, but there isn't any banding.

Avisynth is 8bit, any colour space conversions like rgb <> 4:4:4 using Avisynth functions will be 8bit unless you look into Dither functions and avs2yuv to get interleaved 16bit yuv or rgb, at the moment you're just putting 8bit into 10bit prores / h264 / h265. The whole monster you've created is a total waste of time, honestly pointless. sorry.

No problem, I'm in learning phase. With this monster you can playback the graded MLV (DNG sequence + sidecar XMP + AE adjustments) as an AVI file (AFS's AVI) in fullscreen mode, realtime in eg. MPC-HC, there is no need to encode.

Why not do yourself a big favour and just use Resolve?, or as you've already got ACR, Photoshop and AE why not use them? Export to a decent intermediate format like DNxHD or image sequences and encode to h264 or h265 from there?

I want to avoid intermediate filest if I can, we have already the best one, it's the MLV. If I can't avoid, yes you have right, but this is a way of experimenting.

PS: Once I fed MeGUI mod with the AVS file and I saw BGR48 somewhere at the encoding. Unfortunately I saw this only once and I forgot the settings. Anyway interesting thing to experiment with this monster.

So you're trying to frameserve from some application, to use whatever codec, and you're wanting to check the quality through the frameserve?

Yes, this is a pipe from MLV (16 bit DNG sequence + sidecar XMP adjusments) to fileformat + codec, what ffmpeg covers. It seems the quality is pixelperfect through this pipe, but it's another question, that what ffmpeg will do during the encoding.

Well bit depth loosely means dynamic range. And here, the lower bit depth is only going to be an issue if you start tone mapping after a reduction in bit depth.  For display purposes, you can probably get away with 6 bits of precision.

Yes, I thought that too and it should be so. I mean to tone map files, what have different bit depth should give us different results (If I understand correctly). So I'm confused again.   I did another comparision in AE. I put a 16 bit DNG frame, a 16 bit PNG (saved from the DNG) and a 8 bit JPG (saved from the DNG) into an AE project (compositions: sRGB, 16 bpc) and I applied again an extreme curve on them, the result are the same. It's ok that, I have an 8bit + 2bit dithered display and I can see only around 10bit, but with curve applied the screenshot results should be different not (16 bit vs 8 bit)?

I'm trying to maintain the quality till the final file, I mean converting to eg. MP4 or MOV.

If the AFS works correctly, then with ffmpeg we can do the converting to MP4 (x264 is better then Main Concept), or to MOV (ProRes 444) and any other video formats what ffmpeg covers.

I'm asking for check this thing, because the extreme curve test on the 16 bit PNG looks the same as the AFS's RGB24 AVI file. In theory the PNG is 16bit and the AVI is 8bit, there should be difference with this extreme curve test between the 2 files not or I miss something somewhere?

I did again an extreme curve comparision (At first I wrote about it on doom9 forum in june) and I would like to ask people to check it, if it's correct or not, because I'm not a video expert. (On that forum nobody checked it.)

The main idea behind this is maintaining the best quality from RAW (MLV) till the end (eg.: MOV, MP4).

To check it, please follow this:
01. Download the Advanced FrameServer and install it.
02. Make a preset in Media Encoder (Format: Advanced FrameServer, Color space: RGB24)
03. Download the AviSynth+ and install it.
04. Make an eg.: "E:\rendering\signpost.avs" file:

Code Select Expand

#24 fps source
#AVISource("E:\rendering\signpost.avi", audio=false).AssumeFPS(24,1)
#25 fps source
AVISource("E:\rendering\signpost.avi", audio=false).AssumeFPS(25,1)
#30 fps source
#AVISource("E:\rendering\signpost.avi", audio=false).AssumeFPS(30,1)

# ConvertToRGB24()

# 4:4:4
#ConvertToYV24()

# 4:2:2
#ConvertToYV16()

# 4:2:0
#ConvertToYV12()

05. Download the VirtualDub and the Gradation Curves Plugin.
06. Download one of the MLVFS Windows clients and setup.
07. Mount a MLV with the MLVFS.
08. Open the first DNG from the sequence with ACR, make color correction, open it in PS and save as 16 bit PNG.
09. Open the PNG in VirtualDub and add the Gradation Curves filter on it, make an extreme curve, something like this. (Export the curve, it will be used later too.) Click on show preview and make a sreenshot, save it on a PS layer.
10. Import the DNG sequence into AE (Project settings: 16 bpc, sRGB), save the project.
11. Open Media Encoder, use the previously mentioned AFS preset (02) on the AE project file, start the frame serving to eg.: E:\rendering. You will get an small AVI (eg. signpost.avi).
12. Open the AVI in VirtualDub and add the Gradation Curves filter on it, import the previously saved curve (09), again show preview, screenshot, save it on PS layer.
13. Don't stop the frame serving in Media Encoder.
14. Open the AVS file (comment # every line except one of the AVISource line with the FPS setting and save it) with VirtualDub and add the Gradation Curves filter on it, import the previously saved curve (09), again show preview, screenshot, save it on PS layer.
15. Open the AVS file and delete the comment sign before (4:4:4 setting) save it, open the file with VirtualDub and add the Gradation Curves filter on it, import the previously saved curve (09), again show preview, screenshot, save it on PS layer.
16. If you made everything correctly, then there will be only a negligible difference between the reference PNG frame and the YV24 (4:4:4) setting. Check and uncheck the layers in PS. The fully commented AVS or RGB24 settings will be the same as the PNG and AVI.

If you want to convert the sequence to MOV, MP4 or HEVC, then download the latest ffmpeg build and use it. Here are some example BATs, what I made:

ProRes 4:4:4 10bit:

Code Select Expand

ffmpeg.exe -i "signpost.avs" -c:v prores_ks -qscale:v 1 -pix_fmt yuv444p10 -filter:v unsharp=5:5:0.5:5:5:0.5 "prores_1080p_444_10bit.mov"
pause

x264 4:2:0 8bit:

Code Select Expand

ffmpeg.exe -r 25 -i "signpost.avs" -s 1920x1080 -c:v libx264 -b:v 25000k -minrate 25000k -maxrate 25000k -bufsize 25000k -x264-params keyint=25:min-keyint=1 -g 25 -pix_fmt yuv420p -filter:v unsharp=5:5:0.5:5:5:0.5 -r 25 "x264_1080p_420_8bit.mp4"
pause

x265 4:2:0 8bit:

Code Select Expand

ffmpeg.exe -i "signpost.avs" -s 1920x1080 -c:v libx265 -x265-params crf=1:vbv-bufsize=12500:vbv-maxrate=12500 -pix_fmt yuv420p -filter:v unsharp=5:5:0.5:5:5:0.5 "x265_1080p_420_8bit.hevc"
pause

If you have time please check this workflow till point 16.

Here is my extreme curve comparision, download the JPGs to check them and put them into a PS file, zoom in 200% or more.

I think there are many ML users out there, who knows nothing about the roots of the ML project.

I follow this magic since the begining of 2011 and this kind of open developement is a very respectful thing. I like it not because of it's free for everyone, I like it because of the open spirit, thinking what's behind of this. It's not about the money, it's about passion, enthusiasm, respect, fun, just simple help out each other, share the knowledge and postpone the boundaries.

What is awful, that their are people who are chewing this open spirit.

Here are the roots guys:
http://trmm.net/Magic_Lantern_firmware
http://en.wikipedia.org/wiki/Magic_Lantern_%28firmware%29
https://vimeo.com/groups/magiclantern/forum/
https://www.flickr.com/groups/magiclanternfirmware/

Think about it a bit if you can. If you aren't an open thinking person, you will never know and feel what moves forward this community.

Much respect for the developers, big up guys!

Does anyone else have any thoughts?  I don't want to make this change if there turns out to be a compelling reason to keep it as it currently is (or to change it to some third option).

I prefer this kind of file naming, source MLV: M18-1424.MLV, DNG: M18-1424-00000.DNG ... M18-1424-00250.DNG

1. Ok, I understand.

2. Well it was because of the cached files, so they were there yet at the playback, thats why it was so smooth.
I write this because now I tested again with random directory swith + random file playback and its not smooth, so it's a cache thing.

redownload and repeat my instructions.
if it suddenly crashes, check for a "WebDAVServer.log" and upload it.

if it doesnt crash, check the log messages for exceptions.

It doesn't crashes now (SSE2 version), there aren't any exceptions at the log tab of WebDAVServer and there isn't any "WebDAVServer.log" file generated, so I think it works well now with Resolve 11 Lite. I didn't made the test in PP CS6 dynamic linked with AE CS6.

Another 3 things what I discovered:
1. I constantly watched the cache folder (C:\Windows\ServiceProfiles\LocalService\AppData\Local\Temp\TfsStore\Tfs_DAV) at playbacks and the stange thing is here:
https://www.dropbox.com/s/18s62rphzej0e3x/mlv_webdavserver_error_02.jpg?dl=0

As you can see, on the left side (in cache directory) the ending numbers of DNGs aren't correct, they are 7 digit long, there is an extra "0" at the end + there are some uncorrectly generated DNGs at the end of the directory. Right side is the mounted server (virtual drive).

2. Previously I mentioned, that In Resolve 11 Lite the playback isn't smooth. But now I discovered, that it plays back smoothly if the actual file in the middle box is played and on the left box an another directory is selected.
https://www.dropbox.com/s/vojt2rqxwc5ek1x/mlv_webdavserver_error_03.jpg?dl=0

I used now the SSE2 optimized version, it's a bit really faster, but this crashes too in Resolve Lite.

why does it crash?
any more information?

I mounted 6 MLVs (2-3 seconds long shots, each around 150 MB) to the virtual drive.
Started Resolve Lite and on the Library tab I started to play them back one after another (the playback isn't realtime, it's choppy).
The first was ok, the second too, but at the third came the window, that WebDAVServer crashed (.

I could log the things till the crash, but after that I can't refresh it (because the server stopped).
So I can't get the information from the log to send for you.

If you can implement such kind of thing, which saves the log in txt file, when the server stops, then I can send something.

Many thanks g3gg0 and of course thanks for dmilligan, ayshih, a1ex, chmee, baldand and other developers too.

I got proper DNGs on the virtual drive with the new build (22:05) + installed MSVCRT runtime '13 + after manually cleaning the cache files, what were generated by the older build.

I can't open the sequence directly with PP CS6, but dynamic linked through AE CS6 it works.
Resolve 11 Lite crashes the WebDAVServer.

Where did you save it to?

I saved it to another HDD (D:\), not on the same where my MLV files are (E:\)

How did you start it?

Just double clicked on the WebDAVServer.exe. (I have administrator rights)

Which windows version?

Windows 7 (64 bit)

Makes the WebDAVServer other files/modifications outside the (eg. D:\WebDAVServer) directory? I mean WebDAVServer settings etc. not the MLV folders, IDXs.
May I delet or clear something somewhere on the system?

I found the cache files, they are at this location: %systemdrive%\windows\ServiceProfiles\LocalService\AppData\Local\Temp\TfsStore (source)

SOLVED, read the reply from g3gg0, #14.

Yes, I downloaded the new build from here and overwrote the files.

SOLVED, read the reply from g3gg0, #14.

With the new build I see only empty MLV directories and the IDX files + at the WebDAVServer Log messages there are much from this kind of lines:
'System.DllNotFoundException' in connection from 127.0.0.1:50279 (HandleContent/HandleRequest)
Connection from 127.0.0.1:50280 (PROPFIND)

Here is the mentioned MLV:
https://www.dropbox.com/s/latxxeebzzufgr4/M18-1424.MLV?dl=0

SOLVED, read the reply from g3gg0, #14.

Surami, can you upload such dng file?

If I copy a DNG from the mounted "drive", its filesize will be 0 byte, so there is nothing inside.
https://www.dropbox.com/s/ol2r5lpmzxxqb02/M18-1424_000000.dng?dl=0

Now I'm uploading the source MLV file.

SOLVED, read the reply from g3gg0, #14.

I just tested as you described at the quickstart, I can't open the DNGs:
https://www.dropbox.com/s/8ddsb0cxdejjcbi/mlv_webdavserver_error_01.jpg?dl=0

What's wrong?

SOLVED, read the reply from g3gg0, #14.