MLV App - All in one MLV Raw Video Processing App [Windows, Mac and Linux]

Started by ilia3101, July 08, 2017, 10:19:19 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

masc

Quote from: theBilalFakhouri on May 29, 2022, 12:17:50 PM
Thanks @bouncyball and @masc for the details.

Nah, MLVApp utilize my Ryzen 3900x up to ~80% during export, in reply #4961 I was talking about CPU utilization during Playback . .
Hm... on M1 I also (can) get 100% on playback. It is below 100%, if final framerate is reached with less CPU power. When using Shadows/Highlights/Clarity/DualISO("on" only) it is always below 100%, because these algorithms are single threaded.
5D3.113 | EOSM.202

DeafEyeJedi

What a fun read that was... dating back from 2020 to now has been nothing short of remarkable. Love you guys and keep up with the fabulous work, as always!  8)

If you don't mind me asking, but is there any chance that we could have some sort of a checkbox to temporarily disable 'Graduation Curves' for viewing 'before/after' purposes?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

masc

Quote from: DeafEyeJedi on June 07, 2022, 08:24:00 PM
What a fun read that was... dating back from 2020 to now has been nothing short of remarkable. Love you guys and keep up with the fabulous work, as always!  8)

If you don't mind me asking, but is there any chance that we could have some sort of a checkbox to temporarily disable 'Graduation Curves' for viewing 'before/after' purposes?
Thank you!
Hm... not exactly what you're asking, but:
- Ctrl+C
- reset curves
- Ctrl+V
will do the job.
5D3.113 | EOSM.202

DeafEyeJedi

Ah, ha. Thanks for the reminder on 'Receipt Mask Setup'.

This is actually much better. Brain fart on my part.

No wonder why no one complained.  :P
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

theBilalFakhouri

Hello!

Is it possible to have bad pixels map files same as focus pixel map?
e.g. let us pick bad pixels in MLVApp (using "Fix Bad Pixels" and "Map" option) then export our pixels selections to bad pixel map file, this way we can create bad pixel maps for different Binning modes (basically same as focus pixel maps).

Or at least let "Fix Bad Pixels" with "Map" option have multiple profiles which we can toggle between them when using "Map" option (also if profiles could be saved to external file), but personally I prefer a dedicated bad pixel maps files . .

Thanks

masc

Hey theBilalFakhouri,
it is exactly the way it works. Focus and Bad Pixel maps are identical, just have another file ending. So rename from .fpm to .bpm and you can see the focus pixel map as bad pixel map. Do your editing, rename back, and you have a new focus pixel map.
5D3.113 | EOSM.202

theBilalFakhouri


BickyNutt

Hi.

I'm pretty new to MLV App and editing RAW video. But I'm having trouble with MLV App. Which I've searched on here for any answers and I can't find anything.

Firstly I'm on Windows and have MLV App v1.13. Win64.static

I'm not too sure if I have to install anything with MLV app. I just extracted what was in the .rar file and it launched when clicking the MLVApp.exe

I'm using a Canon EOS M

So the main problem.

I whenever I export a file as CinemaDNG Uncompressed. It comes up with this error.

Export File Error
Could not save: "blahblah.dng
How do you like to proceed?
(then asks me to pick an option)
Skip Frame - Abort Current Export - Abort Batch Extort.

---

Also When I try to export it any other way. It says it is exporting, it gets to 100%. Tells me "Export is Ready". And then nothing has actually been exported.

I'm wondering if anyone could help me. Is it the settings on my camera that aren't right for the actual MLV files. Or is it something I have to do on my PC.

I'm just pulling my hair out here.

Any help would be appreciated.

theBilalFakhouri

ehm ehm,

MLVApp keep crashing when using "Darkframe Subtraction", the longer MLV clip is the high chance of MLVApp crash. I am on Windows 10, I am using 1.13 MLVApp x64 version.
How to debug the issue?

Thanks!

masc

Compile using QtCreator , Qt 5.x, Debug mode, MinGW64 compiler. Then start application with debugger from QtCreator and wait for crash. With some luck you'll see the line where it crashes.

For macOS it looks like this - on Windows it is similar.
https://youtu.be/pmngCFQzFdo
5D3.113 | EOSM.202

theBilalFakhouri

Okay, I have followed all the steps, compiled MLVApp, and ran MLVApp from Debug icon, and MLVApp did crash, where should I see the crash message?

masc

QtCreator shows it. Did you screenshot it? It automatically opens the source code file and marks the line. Plus you should see the call stack and all variable values.

How big are your files to reproduce this? If I can reproduce, I could try to help.
5D3.113 | EOSM.202

theBilalFakhouri

Here is a screenshot:
QT-Debug" border="0

MLVApp crashed (but still open in BG, frozen).

If I clicked on "Interrupt GDB for MLVApp", I can see these:

GDB" border="0




-MLVApp settings and MLV clip info:

-1736x976 @ 23.976 14-bit lossless, 54 seconds.
-Exposure +3.0, Film profile, Dark-frame is generated by recording 5 seconds MLV (same camera settings as main MLV clip) with cap on lens, then imported to MLVApp and exported as MLV and "Averaged Frame", then loaded into MLVApp (just to check it), and then add it to main MLV clip via "Dark subtraction".

-Export settings: ProRess 444, up scale to 1920 (AR locked), export audio is on, force Amaze, ffmpeg Kostya.
-Few clips are loaded in MLVApp too, also other clips with "Dark subtraction" option, and same .MLV dark frame was re-used.

-Also the longer MLV clip with "Dark subtraction" (like 2 minutes or higher) the higher chance of crash will happen.

Above steps should re-produce the issue, this happens always to me.

theBilalFakhouri

I forgot to mention that QtCreator didn't open anything or mark anything . .

masc

I'll try to do the same when I am back to my EOSM.

What I see in your screenshots:
It seems MLVApp code is not the problem of your crash (otherwise you would see the line in code window and find the class/file in stack widget). The cxx and dll written in the application output are part of the Windows system (as far as I find via google). I do not know any feature in MLVApp using this. So it might be a Qt bug or a Windows bug. --> This will be very hard to fix, because we have no hint at all.
5D3.113 | EOSM.202

theBilalFakhouri

Quote from: masc on June 16, 2022, 03:56:28 PM
... The cxx and dll written in the application output are part of the Windows system (as far as I find via google).

I think these "cxx and dll written in the application output" are being written when opening MLVApp and before the crash, so it's not related to the crash.

I will try to reproduce the issue on another PC.

masc

However... the debugger doesn't point on our code and doesn't list our classes in the stack. So this is difficult to fix. Looks like compiled library code.

I tried to reproducec exactly what ou described with my M, but only have my Macbook M1 here. I exported a 2:10min clip several times with darkframe as you described. No crash. Now searching for a Windows...
5D3.113 | EOSM.202

theBilalFakhouri

Thanks for testing.

I think I got the problem, the debugger should something and a line of code when I compiled MLVApp with MinGW_32bit, here is the messages:

1" border="0

2" border="0

AVIR thing?
I made two tests and the two tests showed the same error after the crash.

I did also the test with the same settings (same everything) but without "Darkframe Subtraction" and there is no crash.

masc

Interesting. Still strange: AVIR is 100% independant from Darkframe subtraction. And I really can't tell how to crash the pointed line... I mean float value = float value. How to fail here?! Hmmmmmm... Maybe the destination is invalid - but this also would happen without Darkframe.

Edit:
I could reproduce the problem (on Windows only) now. I get the same problem also without darkframe subtraction, without avir, and even with any other debayer - at least today, maybe after 30min of export... and I remember a similar bugsearch years ago.
What happens on crash:
The basic C function malloc/calloc (getting memory from the OS) doesn't work sometimes in Windows. This happens on any place in our app. I even can check if these functions succeeded: if the answer is "no" one time, it will be "no" until you restart the app. Microsoft: WTF?! I have GBs left in RAM and we free all memory we ask for! My taskmanager tells about 200-300MB RAM used for MLVApp. Are my free GBs of RAM so fragmeneted, so unusable?
So the only solution for now: use any other OS. It will work everywhere. This seems to be a Windows only bug. I've never seen this on any other OS.

If someone has a solution for that: please help. The question is how to get memory if your RAM is free and the OS says "no".

Example:

"debayerto" got memory, while red1d, green1d and blue1d did not get memory from OS. Before this crash, it worked some thousand times while running. Note: this can happen on any point using malloc/calloc - if it happens, I don't know any way to get memory again.
5D3.113 | EOSM.202

theBilalFakhouri

Thanks for your efforts masc by narrowing down the issue, yeah it seems like only happens on Windows.
It would be nice if someone could help for finding a solution for it.

Funny thing, I downloaded linux version of MLVApp 1.13, and ran it using WSL2 on Windows 10, and it works fine with *native* processing speed :P

names_are_hard

Do you free all memory as you go?  Any chance of memory leaks?

You really should check the return value from malloc / calloc in all cases, before using the memory.  You could do this by wrapping / redefining malloc, if you don't want to change every call site.  You can attempt to retry, but generally you are screwed if malloc fails (and some systems, e.g. default Linux, will have malloc never fail at point of allocation - the failure will occur only when you attempt to use the obtained pointer).  You can at least fail gracefully with more information (at least on platforms that have a malloc that will ever fail).

Because Linux has "optimistic" malloc behaviour, and you only see this behaviour on Windows, I wonder if you are sometimes allocating very large amounts, and not using them?  You could log the peak size allocated (some static global, wrap malloc with a logging call and output the peak value as you go).

You could try building with ASAN, that may give you more information (including memory leaks).  You could measure peak memory usage - Windows probably has some per process memory limit which you could be hitting even though the system has memory left.

2blackbar


masc

Quote from: theBilalFakhouri on June 17, 2022, 02:05:52 PM
Funny thing, I downloaded linux version of MLVApp 1.13, and ran it using WSL2 on Windows 10, and it works fine with *native* processing speed :P
Cool! How does this work?

Quote from: names_are_hard on June 17, 2022, 05:03:00 PM
Do you free all memory as you go?  Any chance of memory leaks?
We do our very best to not have memory leaks. But there is always a very little chance... If there is a leak, it is very small, so you'll not see it e.g. using the task manager. I yesterday reviewed all mallocs/callocs and did not find a single call without beeing free() later.

Quote from: names_are_hard on June 17, 2022, 05:03:00 PM
You really should check the return value from malloc / calloc in all cases, before using the memory.  You could do this by wrapping / redefining malloc, if you don't want to change every call site.  You can attempt to retry, but generally you are screwed if malloc fails (and some systems, e.g. default Linux, will have malloc never fail at point of allocation - the failure will occur only when you attempt to use the obtained pointer).  You can at least fail gracefully with more information (at least on platforms that have a malloc that will ever fail).
This is more or less what I already tried in past. But if malloc/calloc fails once, it will fail until you restart the complete app. So all I could do is telling the user "no more processing now, the app will quit now", instead of crashing. The difference is not big: the user won't get a single additional frame beeing processed. This problem only exists on Windows.

If someone knows how to better search for memory leaks: feel free to search. Each bugfix is welcome!
5D3.113 | EOSM.202

masc

Quote from: 2blackbar on June 17, 2022, 08:48:44 PM
Is there any chance that MLVApp will get VP9 codec export ?
At least one pass should be doable.
https://trac.ffmpeg.org/wiki/Encode/VP9

What mode is required? Lossless? Or constant bitrate? Or...
5D3.113 | EOSM.202

names_are_hard

Okay, probably not a leak using all your allowed process memory...  I'm not great at diagnosing real software on Windows, just the stupid stuff malware does.  I can try and look for problems in the Linux version, they might apply on both.  The behaviour of malloc failing being persistent seems odd to me unless all mem is exhausted.  Did you try an ASAN build?

Is this the right place to get current version?
https://github.com/ilia3101/MLV-App