MLV App - All in one MLV Raw Video Processing App [Windows, Mac and Linux]

Started by ilia3101, July 08, 2017, 10:19:19 PM

Previous topic - Next topic

0 Members and 3 Guests are viewing this topic.

vastunghia

Quote from: masc on January 20, 2023, 03:56:56 PM
add the new compiler in "Compilers" in your screenshot frame, and then select in Kits the llvm compiler

Thanks a lot masc! So apparently I can confirm very slightly better performance during uncompressed cDNG export with Qt6 / llvm15 local build on Mac x86:

  • Dual ISO:

    • Official 1.14 release: 1.92 s / frame
    • Qt6-llvm15 build: 1.79 s / frame
  • Single ISO:

    • Official 1.14 release: 9.9 fps
    • Qt6-llvm15 build: 10.5 fps
Frame rate during playback has a higher gain! From 4 to 7 fps for a 14-bit 3.5K 1:1 lossless-compressed single-ISO MLV file (avg MLV data rate @129 MB/s).

Not a game changer, but nice.

Sergio
5D3 for video
70D for photo

vastunghia

A separate post for a feature request: Would it be hard to add an option to export the MLV files attribute table (as reported in the Session Area when in Table Mode) in a very simple and handy format, say csv maybe?
5D3 for video
70D for photo

masc

Quote from: vastunghia on January 21, 2023, 02:12:04 PM
So apparently I can confirm very slightly better performance during uncompressed cDNG export with Qt6 / llvm15 local build on Mac x86:
cDNG export is not a major strength of MLVApp - more a side product for a longer feature list. Lossless de-/compression is single threaded. So don't expect much changes here. MLVFS is in advantage here (even if - i think - they use the same algorithm), if the NLE preloads more than one frame - then you get multithreading.

The table was/is already planned and should be doable. I just need the time to do this.
https://github.com/ilia3101/MLV-App/issues/212
5D3.113 | EOSM.202

Danne

I have reworked Mlv App compiler.app a lot today. It will atm only work with arm64 machines(M1,M2 etc) But might look into including intel processing as well if I find the time. By running the app you start by selecting:
(U)   install dependencies(run it once)
Which will download, configure and compile Qt 6 onto your system. All automated.
Also possible to erase dependencies if you want to start fresh:
(E)   Erase dependencies

Full menu:
(c)   compile MLV App with openmp(arm64)
(b)   check out branches
(D)   install dependencies(run it once)
(E)   erase dependencies
(m)  make clean


My focus has been to get something working which is not too complicated and automated.
https://bitbucket.org/Dannephoto/mlv_app_compiler-git/downloads/

Feel free to let me know how it works if tested.

masc

Quote from: vastunghia on January 21, 2023, 02:16:10 PM
A separate post for a feature request: Would it be hard to add an option to export the MLV files attribute table (as reported in the Session Area when in Table Mode) in a very simple and handy format, say csv maybe?
Done. Please test.

@Danne: nice! FocusPixelMap auto downlod also works now?
5D3.113 | EOSM.202

vastunghia

5D3 for video
70D for photo

Danne

Quote from: masc on January 21, 2023, 06:02:56 PM
@Danne: nice! FocusPixelMap auto downlod also works now?
Hm, that part always worked for me, and also when iaburn tested.
Uploaded a version here. https://bitbucket.org/Dannephoto/mlv_app_compiler-git/downloads/MLV_App_arm64.zip
Maybe works now? Or else we need to check some path issue.

masc

Hm, here it doesn't work with your version. Loading a clip which needs a map doesn't open the known messagebox. Pixels are visible. On my Intel Mac I've seen, Qt did not find libssl, but it downloaded the maps anyway. In Windows version it was needed for downloading. So in my latest commits I added openssl to the linked LIBs. I don't see libssl and libcrypto in your build yet. Could we try again?
5D3.113 | EOSM.202

Danne

Added -lssl like follows:
QMAKE_CC=/opt/homebrew/opt/llvm@15/bin/clang \
QMAKE_CXX=/opt/homebrew/opt/llvm@15/bin/clang++ \
QMAKE_LINK=/opt/homebrew/opt/llvm@15/bin/clang++ \
QMAKE_CFLAGS+=-fopenmp \
QMAKE_CFLAGS+=-ftree-vectorize \
QMAKE_CXXFLAGS+=-fopenmp \
QMAKE_CXXFLAGS+=-std=c++15 \
QMAKE_CXXFLAGS+=-ftree-vectorize \
INCLUDEPATH+=-I/opt/homebrew/opt/llvm@15/include \
LIBS+=-L/opt/homebrew/opt/llvm@15/lib LIBS+=-lomp \
LIBS+=-L/opt/homebrew/opt/openssl/lib LIBS+=-lssl
QMAKE_APPLE_DEVICE_ARCHS=arm64


Mlv App increased a few mb in size so something seemed to happen. Could you test?
https://bitbucket.org/Dannephoto/mlv_app_compiler-git/downloads/MLV_App_arm64.zip

Not sure where to add libcrypto. Part of openssl I noticed. Both libs are included now in Frameworks.

masc

Unfortunately no change. Maybe it is Qt6: AVFoundation sound export had an issue, where Qt6 file, path and call commands did not work at all. With standard C functions: no prob. Map download inherits many many of these Qt lib calls. E.g. a simple QFileInfo("filename").exists() always fails here with Qt6.4, same for QProcess::execute("application").
5D3.113 | EOSM.202

Danne

Is there a Qt 6.5?
Strange that it works on iaburns end, hmmm
I guess if pne compile locally ot could behave better.

masc

Qt6.5 is in beta state. You can load it from online installer. They plan to release end of march.
https://wiki.qt.io/Qt_6.5_Release
5D3.113 | EOSM.202

vastunghia

5D3 for video
70D for photo

vastunghia

Quote from: masc on January 21, 2023, 03:40:39 PM
MLVFS is in advantage here, if the NLE preloads more than one frame - then you get multithreading.

Understood. Too bad MLVFS is broken in Ventura. Or isn't it?
5D3 for video
70D for photo

masc

Quote from: vastunghia on January 21, 2023, 11:37:50 PM
Just sharing MLV App installer (DMG) 'nightly build' (ref. commit 3330563) for OSX x86, built with Qt6 and llvm15:

https://www.icloud.com/iclouddrive/061ffs2OUqHzzwsMapMtkx6gQ#MLVApp.3330563.llvm15.OSX

Sergio
Auto FocusPixelMap install is broken also with this version. Seems not to work on Qt6.4 yet. Many path related Qt6.4 functions fail (e.g. QFileInfo(<fileName>).exists() ). No idea why. No such problem on Qt5.15.
5D3.113 | EOSM.202

Danne


iaburn

Quote from: Danne on January 22, 2023, 12:12:04 PM
Strange. Also not broken on all machines.

It's funny that focus map download worked in your compilation but then it doesn't work when I was testing things and tried on one of my compilations O_O I'll try to find a reason...

Regarding the dual ISO update with the code that you posted from cr2hdr (I think I took the latest from the 2), I could put everything together and it's a total mess now, but it works and it gives much better results with the highlight stripes without having to tweak the tolerances like I was doing.

The problematic video from the 5D mark III looks much better (no horizontal stripes, no shadow noise) but it flickers like crazy. It's very likely that I messed something while trying to make the updated code work. I'll take a look next weekend while cleaning things and fixing all the processing settings that are not working now  :D
The video flicker issue: https://mega.nz/file/l9cgDYRC#S4FmCFDKsaZD9ZNgFiToBz5Dvjzw7Kmmpo56vsGeRRk
Comparison from the old and the new code:


With my test clips, this is how it looks with no corrections at all (very zoomed in to see something...):


A old vs new comparison (with a bit increased highlights) to illustrate the difference, same recipe on both:





Danne

Wow. Impressive. Very close.
The flicker. What are whitelevel numbers if you export to dualiso processed dng files?

vastunghia

Quote from: masc on January 22, 2023, 11:09:48 AM
Auto FocusPixelMap install is broken also with this version.

Mmh strange. I don't own an M, so I downloaded iaburn's test clips and checked. Opening them with my build, it automatically downloaded focus pixel maps and applied them correctly -- I can clearly see focus pixels re-appear if I disable the fix.

Also, I report a massive playback speed improvement with my local build for Mac x86 (Qt 6 + llvm15) vs official 1.14 build. With iaburn's test clip that Danne was referring to (M28-1445), I pass from 5-6 fps to 22.

Sergio
5D3 for video
70D for photo

iaburn

Quote from: Danne on January 22, 2023, 02:11:38 PM
Wow. Impressive. Very close.
The flicker. What are whitelevel numbers if you export to dualiso processed dng files?

Same on both (2 frames with different exposure): white level 64800, dark level 8188.

theBilalFakhouri

@iaburn

Nice work on Dual ISO :) , finally we have new one who can deal with Dual ISO code :D
Could you fix multi-threading issue too? :P

Dual ISO multi threaded vs single threaded

Keep it up!

vastunghia

5D3 for video
70D for photo

Danne

Quote from: vastunghia on January 22, 2023, 03:25:14 PM
Mmh strange. I don't own an M, so I downloaded iaburn's test clips and checked. Opening them with my build, it automatically downloaded focus pixel maps and applied them correctly -- I can clearly see focus pixels re-appear if I disable the fix.

Also, I report a massive playback speed improvement with my local build for Mac x86 (Qt 6 + llvm15) vs official 1.14 build. With iaburn's test clip that Danne was referring to (M28-1445), I pass from 5-6 fps to 22.

Sergio
That's crazy. 22fps on intel processor.

vastunghia

Quote from: Danne on January 22, 2023, 05:42:08 PM
That's crazy. 22fps on intel processor.

Well, I'm still stuck at 7 fps (up from 4) with 1:1 3.5K on 5D3.. ::)
5D3 for video
70D for photo

names_are_hard

Quote from: Danne on January 22, 2023, 05:42:08 PM
That's crazy. 22fps on intel processor.

I get about 25fps on my linux desktop.  With gcc, qt5, 1.13.  AMD 3800.  I think we're learning that old Macs are slow, not new Macs are fast :)