MLVProducer: [v3200]

[kd503_b]

Download is borken. Anyone has the same problem?

[g3gg0]

Hi, g3gg0!

It does support lj92.
-At least for MLV. Does raw_rec support lj92?
-0x20? I noticed that it is always 0x21

Are these used somewhere?

indeed it supports. layer 8 bug.
it was the first post that says "latest version INTEL" which lead me to download a quite old version.
maybe i am not the only one who falls into this trap

the class is kind of a bitmap.
the lowest 4 bits are the format bits where the upper 4 bits are some kind of encoding information.
keep it as you have it, its fine how you did it.

the bits are not used anymore. delta and LZMA were experiments to compress files for archiving.
noone probably ever used them.

[reddeercity]

Had a crash loading 5D2 12bit 1:1 1856x1044 23.976p 30GB file (11299 frames)
Had a " Subscript out of range" tried older builds same thing , found the crash log

Code: [Select]

MLVProducer alpha: build 3203
Threads: 4
command line:
cmdparsing...
cmd:'""'
files:2
D:\10-12bit-for youtube\M11-0006.MLV
opening...
D:\10-12bit-for youtube\M11-0006.MLV
extension:.mlv
D:\10-12bit-for youtube\M11-0006.M00
D:\10-12bit-for youtube\M11-0006.M01
D:\10-12bit-for youtube\M11-0006.M02
D:\10-12bit-for youtube\M11-0006.M03
D:\10-12bit-for youtube\M11-0006.M04
D:\10-12bit-for youtube\M11-0006.M05
D:\10-12bit-for youtube\M11-0006.M06
parsing...
Compression: RAW
MLVI: fc=1475
FPS: 23.976
Levels: 448 - 4050
Size: 1856x1044
v1: 2040
v2: 3060
v3: 1267
v4: 1880
v5: 1249
Bits Per Pixel = 12
Compression: RAW
MLVI: fc=1477
FPS: 23.976
Compression: RAW
MLVI: fc=1477
FPS: 23.976
Compression: RAW
MLVI: fc=1475
FPS: 23.976
Error:
Method: ShowFrame
Line: 17
Number: 9
Description: Subscript out of range
Error:
frmMain: MouseMove
Line: 50
Number: 9
Description: Subscript out of range
Notice this

Code: [Select]

MLVI: fc=1475is that "fc" frame count ? if so it's reporting 1475 where it should be 11299 .
I tried the new MLV App (win version) and had not problem loading that 30GB file and exporting .

[AWPStar]

@reddeercity
Thank you! Useful bug-report.

is that "fc" frame count ? if so it's reporting 1475 where it should be 11299 .

Yes, this it frame count. But for single file.

Try this. https://sourceforge.net/projects/mlvproducer/files/MLVProducer.exe/download

[reddeercity]

Thanks @AWPStar , if finally loaded just exporting to prores 4444  right now as I write this .
But on the down side , on first run It crashed/no responds (Windows Blue Wheel of Death) 
So the second time I cleared all the ".idx" files and the a setting file then loaded it up , take very long time to get it to come up
Normally there's a progress bar that reads the file (somethingMb/s) on my raid or ssd it's 400-500 but here nothing , I notice on the bottom of the view screen
the frame count started to increase slowly , first 1400 them 3000 etc. ... until it hit 11299 frame around 2-3 minutes if not more -- very very slow
so slow I had to open task man to see if it hang up , but it was working slowly .
here the edited log  , also it seems slow exporting , I have the 30GB file on my Raid0 (4x2TB=8GB r/w @ 800MB/s @ 50% capacity (4GB free)
and rendering to the OS drive SSD(sata3) but only at 2.2 frames per second  (cpu-amd 8 core@ 4.8Ghz fx8350+8GB ram win7pro)
Is this normal speed? 

Code: [Select]

MLVProducer alpha: build 3272
Threads: 4
command line:
cmdparsing...
cmd:'""'
files:2
openraw/mlv: "D:\10-12bit-for youtube\M11-0006.MLV"
cmdparsing...
cmd:'"D:\10-12bit-for youtube\M11-0006.MLV"'
files:2
D:\10-12bit-for youtube\M11-0006.MLV
D:\10-12bit-for youtube\M11-0006.MLV
opening...
D:\10-12bit-for youtube\M11-0006.MLV
extension:.mlv
D:\10-12bit-for youtube\M11-0006.M00
D:\10-12bit-for youtube\M11-0006.M01
D:\10-12bit-for youtube\M11-0006.M02
D:\10-12bit-for youtube\M11-0006.M03
D:\10-12bit-for youtube\M11-0006.M04
D:\10-12bit-for youtube\M11-0006.M05
D:\10-12bit-for youtube\M11-0006.M06
parsing...
Compression: RAW 1
MLVI: fc=1475
FPS: 23.976
Mode Size: 2040 , 1267
Crop corrds: 160 , 18
Levels: 448 - 4050
Size: 1856x1044
Bits Per Pixel = 12
Compression: RAW 1
MLVI: fc=1477
FPS: 23.976
Compression: RAW 1
MLVI: fc=1477
FPS: 23.976
Compression: RAW 1
MLVI: fc=1475
FPS: 23.976
Compression: RAW 1
MLVI: fc=1477
FPS: 23.976
Compression: RAW 1
MLVI: fc=1474
FPS: 23.976
Compression: RAW 1
MLVI: fc=1477
FPS: 23.976
Compression: RAW 1
MLVI: fc=998
FPS: 23.976
opened
Total bytes: 2901322240
Total time: 75.8 sec
Parse speed: 36 MB/s
parsed at 75.84
Audio track duration > video
audio:
Sampling Rate:44100
Duration(sec): 471.262930839002
Video Duration: 471.262932895392
init buffer
init cache frame:1856x1044 fc=11299  c=1073741824
init frame:1856x1044
init frame DONE
init sound: 368902176
LoadsoundRET=-1879048190
clear black level, load cslut
parsing complete

FFMPEG CMD
C:\Users\FX8350\Videos\10-12bit test\M11-0006.mov
C:\Users\FX8350\Downloads\mlvp.alpha.build3261\ffmpeg.exe -threads 4 -f rawvideo -pix_fmt rgb48le -s 1856x1044 -r 23.976 -i - -f mov -vframes 11299 -vcodec:v prores_ks -profile:v 4 -qscale:v 0 -vendor ap10 -s 1856x1044 -pix_fmt yuv444p10le -threads 4 -r 23.976 "C:\Users\FX8350\Videos\10-12bit test\M11-0006_tmp.mov"


ffmpeg version N-80892-g2a8dadb Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 48.101 / 57. 48.101
  libavformat    57. 41.100 / 57. 41.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 47.100 /  6. 47.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100

Input #0, rawvideo, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: 2229956 kb/s
    Stream #0:0: Video: rawvideo (RGB0 / 0x30424752), rgb48le, 1856x1044, 2229956 kb/s, 23.98 tbr, 23.98 tbn, 23.98 tbc
[mov @ 0036e120] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, mov, to 'C:\Users\FX8350\Videos\10-12bit test\M11-0006_tmp.mov':
  Metadata:
    encoder         : Lavf57.41.100
    Stream #0:0: Video: prores (prores_ks) (ap4h / 0x68347061), yuv444p10le, 1856x1044, q=2-31, 200 kb/s, 23.98 fps, 11988 tbn, 23.98 tbc
    Metadata:
      encoder         : Lavc57.48.101 prores_ks
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> prores (prores_ks))

frame=    3 fps=2.1 q=-0.0 size=    3947kB time=00:00:00.08 bitrate=387251.1kbits/s speed=0.0583x   
frame=    6 fps=2.9 q=-0.0 size=    7919kB time=00:00:00.20 bitrate=310964.2kbits/s speed= 0.1x   
frame=    8 fps=2.8 q=-0.0 size=   10569kB time=00:00:00.29 bitrate=296468.2kbits/s speed=0.101x   
frame=   10 fps=2.7 q=-0.0 size=   13215kB time=00:00:00.37 bitrate=288333.6kbits/s speed=0.102x   
frame=   12 fps=2.7 q=-0.0 size=   15865kB time=00:00:00.45 bitrate=283220.5kbits/s speed=0.103x   
frame=   13 fps=2.6 q=-0.0 size=   17186kB time=00:00:00.50 bitrate=281254.3kbits/s speed=0.0991x   
frame=   15 fps=2.6 q=-0.0 size=   19835kB time=00:00:00.58 bitrate=278228.4kbits/s speed=0.101x   
frame=   17 fps=2.6 q=-0.0 size=   22483kB time=00:00:00.66 bitrate=275957.8kbits/s speed=0.101x   
frame=   19 fps=2.6 q=-0.0 size=   25130kB time=00:00:00.75 bitrate=274179.4kbits/s speed=0.102x   
frame=   21 fps=2.6 q=-0.0 size=   27776kB time=00:00:00.83 bitrate=272748.5kbits/s speed=0.102x   
frame=   23 fps=2.5 q=-0.0 size=   30425kB time=00:00:00.91 bitrate=271606.1kbits/s speed=0.101x   
frame=   25 fps=2.5 q=-0.0 size=   33073kB time=00:00:01.00 bitrate=270644.1kbits/s speed= 0.1x   
frame=   27 fps=2.5 q=-0.0 size=   35720kB time=00:00:01.08 bitrate=269821.7kbits/s speed=0.101x   
frame=   29 fps=2.5 q=-0.0 size=   38366kB time=00:00:01.16 bitrate=269104.3kbits/s speed=0.101x   
frame=   31 fps=2.5 q=-0.0 size=   41013kB time=00:00:01.25 bitrate=268494.8kbits/s speed=0.101x   
frame=   33 fps=2.5 q=-0.0 size=   43659kB time=00:00:01.33 bitrate=267958.8kbits/s speed= 0.1x
...........   
frame= 2663 fps=2.2 q=-0.0 size= 3529425kB time=00:01:51.02 bitrate=260412.8kbits/s speed=0.0904x   
Notice that it only logs every other frame or is it skipping frames ?
just waiting for the file to fully export .
Edit: It's seems the frame count in the log doesn't add to the total frame count , I get around 9000 unless I missed some numbers

[AWPStar]

but only at 2.2 frames per second

As you have cpu with 8 cores you can use 8 thread for rendering. Check settings->Preformance->Threads
Also, try to use ProRef Fast preset. Other two are pretty slow.

So the second time I cleared all the ".idx" files and the a setting file then loaded it up , take very long time to get it to come up

Is your disk compressed?
Please, check this. https://sourceforge.net/projects/mlvproducer/files/MLVProducer.exe/download
Remove orr rename *.idx file before.


Tell me if it works. I cannot test it because all me files(even huge) are loading at normal speed.

[IDA_ML]

Hello AWPStar,

I have been using v.3261 quite a lot lately and I love it.  Now it renders my favorite ProResFast_4444 format really fast.  However, I noticed a malicious stability bug that, unfortunately, I am unable to reproduce.  Sometimes, after editing a clip and trying to export it using "Render Task", Producer will stop rendering after the second frame and will freeze.  Today, it happened with a 10-bit MLV file shot with the 7D at 1728x972 resolution.  If that happens, the only way to get out of Producer is through the Task manager.  This behavior is very annoying since it may happen after you have spent a lot of time editing several clips.  In that case, all your work is lost.  Happened twice on me already.  The funny thing is that restarting Producer does not fix the issue.  You have to restart the computer to get Producer render properly again. 

I have to say, I have been using v. 3203 for several months before replacing it with v. 3261.  I have never observed the above behavior before, with v. 3203.

Could you please take a look at v. 3261?  I really do not want to revert back to v. 3203 again.

Thank you and keep up your excellent work! 

[AWPStar]

@IDA_ML
Not sure, but try this one. https://sourceforge.net/projects/mlvproducer/files/MLVProducer.exe/download
Also it has many other fixes and improvements.

Can you provide log file?

[IDA_ML]

@IDA_ML
Not sure, but try this one. https://sourceforge.net/projects/mlvproducer/files/MLVProducer.exe/download
Also it has many other fixes and improvements.

Can you provide log file?

I don't know what you mean by "log file" and how to generate it but I will definitely test the above version.  Thanks.

[reddeercity]

@AWPStar , No my raid0 isn't compressed , I'll try the new .exe and report back .

I don't know what you mean by "log file" and how to generate it but I will definitely test the above version.  Thanks.

See blue hi lighted bar

In the root of the program , it's auto generated when MPVProducer starts or at least when a file is loaded

Edit: Also here you can clear it or start it


Edit#2: @AWPStar yes that's faster
build 3354

Code: [Select]

Total time: 80 sec
Parse speed: 392 MB/s
parsed at 80.05with fast preview I get Real Time paly back 24fps , thou audio is slow by a few seconds
build 3272

Code: [Select]

Total time: 120.3 sec
Parse speed: 22 MB/s
parsed at 120.38
I really like the new Fast Color plugin  give a nice creamy cinematic look.
I did notice 12bit to be a more pleasing image then 14bit , really nice roll off's on the hi end and clean shadow
The most Filmic I ever seen from a 5d2 even better then 5d3 from the example I've seen (too video-ish , for my liken)
It's a pleasure to grade it , specially with mlvproducer 
If at all possible I'll be filming in 12bit (not to save disk space or write speed) cause the image is that good !
Finally the diamond is starting to reveal it self , I my never upgrade at this rate. 

[IDA_ML]

Thanks a lot, Reddeercity,

The next time the software hangs or crashes, I will copy the log.txt file and paste it here. 

My first impression of version 3354 is very positive.  Producer keeps improving!  Thanks a lot, AWPStar!

[dariSSight]

@AWPStar , No my raid0 isn't compressed , I'll try the new .exe and report back .
See blue hi lighted bar

In the root of the program , it's auto generated when MPVProducer starts or at least when a file is loaded

Edit: Also here you can clear it or start it


Edit#2: @AWPStar yes that's faster
build 3354

Code: [Select]

Total time: 80 sec
Parse speed: 392 MB/s
parsed at 80.05with fast preview I get Real Time paly back 24fps , thou audio is slow by a few seconds
build 3272

Code: [Select]

Total time: 120.3 sec
Parse speed: 22 MB/s
parsed at 120.38
I really like the new Fast Color plugin  give a nice creamy cinematic look.
I did notice 12bit to be a more pleasing image then 14bit , really nice roll off's on the hi end and clean shadow
The most Filmic I ever seen from a 5d2 even better then 5d3 from the example I've seen (too video-ish , for my liken)
It's a pleasure to grade it , specially with mlvproducer 
If at all possible I'll be filming in 12bit (not to save disk space or write speed) cause the image is that good !
Finally the diamond is starting to reveal it self , I my never upgrade at this rate.

Hello Reddeercity
Which build are you using on 5d2 for 12bit and are you doing a new tutorial soon?

[reddeercity]

Hello Reddeercity
Which build are you using on 5d2 for 12bit

https://www.magiclantern.fm/forum/index.php?topic=5601.msg198349#msg198349

are you doing a new tutorial soon?

Kind of , more like a ml development update for D4's/5D2's
and call for more developer to join up etc. .....

[IDA_ML]

Hello AWPStar,

I have noticed a bug with v. 3354 of Producer.  When rendering MLV files shot at the full resolution of the sensor, such as timelapse sequences consisting of FRSPs taken with the Silent module, Producer exports them fine but fails to record the final file in the destination folder.  I have prepared a 950 MB MLV file shot with the Silent module of the 100D as a FRSP sequence (30 frames) for you to check and to see how Producer behaves at the end of the export.   Download link:

https://we.tl/BD5NQBikma

The link will be active for 7 days.

Also here:

https://www.magiclantern.fm/forum/index.php?topic=20025.925

I have provided some feedback on my comparative observations of Producer vs. MLVApp.  You may want to have a look at that.
 

[AWPStar]

That's weird. When i export to ProRes it creates unreadable file(100D_30frames_Silent_Module_FRSP.mov). With any resolution, any frame rate...
h264 and mpeg2 works fine.

[liyangheimao]

I had the same issue since version after 2722, can't straightly open the MLP
But I can run it with Locate Emulator(Known for running Japanese games on non Japanese language systems) in Japanese or English

I am using Simplified Chinese Windows 10 and I assume your's using non English language system as well.

[lightmeup]

I had the same issue since version after 2722, can't straightly open the MLP
But I can run it with Locate Emulator(Known for running Japanese games on non Japanese language systems) in Japanese or English

I am using Simplified Chinese Windows 10 and I assume your's using non English language system as well.

I am also using a CHS version of MS Windows 10 and I will try that on my machine. :-)

[Sta911]

Hello.
I have a problem with presets made on one computer when opening on another. Tried on different versions for the last 2 months. The pattern found only in the path to the AdobeRGB.cslut file (C:\Users\Administrator\Desktop\COLOR\data\cs\AdobeRGB.cslut). Versions on both computers are the same, but different operating systems (windows 10 and 7). What kind of information do I need to provide so that you can fix this bug?

[AWPStar]

@Sta911
Known issue. Presets are using absolute path.
This will be fixed.  wait for new version.

[50mm1200s]

Hi @AWPStar, do you know what algorithm is MLVPDM? How does it compare with AMaZE and IGV? Do you think these are too complex for video?
Also, just as suggestion too, Bart Goossens have some really interesting publications in this subject. These two, for example:

- An Overview of state-of-the-art denoising and demosaicking techniques: toward a unified framework for handling artifacts during image reconstruction

- "Computationally Efficient Locally Adaptive Demosaicing of Color Filter Array Images Using the Dual-Tree Complex Wavelet Packet Transform"

I think the proposed "DTCWPT" algorithm would be very suitable for MLVProducer. His website seem to be this one.

[keel]

Can someone walk me through using MLVproducer?

Specifically the FPN (focus dots, hot pixels, vertical stripes) procedure.



(tried to follow the video, but there wasn't an option for "NEW" under demosaic algorithm)

edit - using version 3261

[anto]

hello guys, sorry I found out now this MLVproducer. does it work with lossless compression?

[Walter Schulz]

See feature list on first page: lj92 support.

[yoshinoriX]

Hi @AWPStar, do you know what algorithm is MLVPDM? How does it compare with AMaZE and IGV? Do you think these are too complex for video?
Also, just as suggestion too, Bart Goossens have some really interesting publications in this subject. These two, for example:

- An Overview of state-of-the-art denoising and demosaicking techniques: toward a unified framework for handling artifacts during image reconstruction

- "Computationally Efficient Locally Adaptive Demosaicing of Color Filter Array Images Using the Dual-Tree Complex Wavelet Packet Transform"

I think the proposed "DTCWPT" algorithm would be very suitable for MLVProducer. His website seem to be this one.

Seems the thing is patented
no wonder, it looks REALLY GOOD

Anyone good enough in hard math and this mumbojumbo coding could develop a open version of this algorithm (?)

[70MM13]

I also respectfully request some development of the demosaicing algorithm.  For me, it's the only thing really holding back this excellent program!

It tends to alias pretty badly on the edges of strong highlights.

On another note, I find that presets don't remember the green channel settings, such as "lift" (my favourite!)

I'm on windows 7-64, running version 3261alpha

Thanks!