MLV App 1.14 - All in one MLV Video Post Processing App [Windows, Mac and Linux]

[IDA_ML]

Unfortunately not, but I will keep testing the filters as they are.  I am convinced that useable results are quite feasible!  Great job, Masc!

[70MM13]

rawtherapee has a wavelet denoiser that is quite powerful, and photivo has a very good denoiser also.
they don't use the same principle as neat video, but they are very good and maybe one of them can be added without too much difficulty?
it might be worth a look.
when i think about denoisers it reminds me of those "denoisaicing" algorithms that look so incredible.  i wonder if anyone is using them yet in any open source projects?

[masc]

What demosaicing algorithms do you mean? The most powerful should be still AMaZE and for footage with much moiree or noise IGV.

[70MM13]

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5492375/

[Luther]

What demosaicing algorithms do you mean? The most powerful should be still AMaZE and for footage with much moiree or noise IGV.

There's some already implemented. See:
https://github.com/cchen156/Learning-to-See-in-the-Dark
https://github.com/cig-skoltech/deep_demosaick

Other research:
https://groups.csail.mit.edu/graphics/demosaicnet/data/demosaic.pdf
www.nowozin.net/sebastian/papers/khashabi2014demosaicing.pdf
https://www.researchgate.net/publication/323165044_Joint_Demosaicing_and_Denoising_with_Perceptual_Optimization_on_a_Generative_Adversarial_Network

[bouncyball]

Hmm...

Hey Danne I think you will be interested in deep_demosaick to use it in Switch

[Danne]

I thought both first links was pretty spectacular. Not very straight forward how to get this boat floating but maybe could get something going after some tinkering.

[theBilalFakhouri]

There's some already implemented. See:
https://github.com/cchen156/Learning-to-See-in-the-Dark

Is this research mean: how to turn your camera into sony a7s?

[masc]

Hm... none of them is usable in MLVApp without reinventing the world. Our application is C, C++, Objective C, and code should be at least one of them. Or maybe somelike likes to translate python to C.

I tried out some other debayers yesterday:
- Counter Stencils (https://www.ipol.im/pub/art/2012/g-dwcs/)
- SSD (https://www.ipol.im/pub/art/2011/bcms-ssdd/)
- AICC (https://www.ipol.im/pub/art/2015/145/)

All are working, but are very slow (we count minutes per image).
- Counter Stencils looked comperable to bilinear (whyever not better, it did 250 iterations, 337sec per image)
- SSD looked close to LMMSE + some more artifacts, therefor slower
- AICC looked close to AMaZE, a little different here and there, but not better, @4 threads around 86sec per image
All tests were made at 1856x1044.

[Luther]

Hm... none of them is usable in MLVApp without reinventing the world.

The processing would take hours without CUDA support. Yeah, not really feasible. But neural networks open a new world of possibilities for image processing. First time I read this stuff I couldn't believe it was true. Down below there's some impressive debluring algorithms [1][2][3][4], moire reduction [5] and super-resolution [6][7].

If someone here is looking to upgrade the demosaicing in MLVApp, I think it would be best to look what the high-end industry is using right now.
I'm not an expert, but most people recording with Panavision DXL2/mini or ALEXA LF seem to use Colorfront Transkoder or Codex Production Suite. ARRI recommends their own algorithm, called "ADA-5", that can be tested using their freeware software (sample footage from ALEXA here). This ADA-5 is now a standard called "RDD 31:2014" (couldn't access the paper myself).



[1] http://openaccess.thecvf.com/content_ECCV_2018/papers/Jiangxin_Dong_Learning_Data_Terms_ECCV_2018_paper.pdf
[2] https://github.com/jacquelinelala/GFN
[3] https://github.com/jiangsutx/SRN-Deblur
[4] https://arxiv.org/abs/1808.00605
[5] https://arxiv.org/abs/1805.02996
[6] https://github.com/jiangsutx/SPMC_VideoSR
[7] https://github.com/xinntao/ESRGAN

[masc]

I don't think MLVApp will get any CUDA support - at least as long as I develop for it... I have no such hardware and no idea how to program it.

I don't think ARRI has the same problems we have. E.g. AMaZE is superb, if it gets good RAW data. We still have the problem of line skipping what produces the artifacts we all don't like. At 1x1 recording, our algorithms should be more than okay.

[amp829]

 Can somebody tell me about the export settings to get the best result for a playback video ,  because even though i am choosing some settings , as H265 / mpeg amaze / etc . the result is only a few mb from a GB size mlv file and there is obvious too much loss of quality and information that makes think that i am doing something wrong ...

[ArcziPL]

Can somebody tell me about the export settings to get the best result for a playback video ,  because even though i am choosing some settings , as H265 / mpeg amaze / etc . the result is only a few mb from a GB size mlv file and there is obvious too much loss of quality and information that makes think that i am doing something wrong ...

Use a ProRes format for output.

@masc: is there an option to adjust bitrate for h264/h265 export? Or can it be added? Or maybe a custom command-line option to ffmpeg; shall be more than enough.

Just used v1.6 few days ago for a new video. Amazing work! I like the new features a lot.

BTW: Could you please describe what is the order of applying the operations on the image? I guess it's not top-to-bottom like organized in the right  panel, as this would rather be sub-optimal.

[masc]

In MLVApp v1.6 h264/265 is set to "24". In latest commit it is set to maximum quality, what a "normal PC" is able to decode and playback.

See help / wiki to see which features are realized in which order. Maybe some small newer things are missing... please ask if so.

[amp829]

you are right i used the options proves and the quality is much better

[2blackbar]

Great App, i have a question, i have canon M and app detects raw white level as 3817 but correct one is 3490 ,is there any way to change this so it will be correct on every clip ? So i dont have to do it manually all the time.
I have pink highlights if white level is incorrect.
Or is my thinking wrong and theres some other way to deal with pink highlights?
---
It looks like different clips have different white level anway

[Danne]

What are your settings in cam when getting white level 3490? What eos m build are you using? Could you upload a sample mlv?

[2blackbar]

Im using very latest build  crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Mar29.EOSM202
After more investigation i think i was wrong and just didnt had ticked reconstruct highlights, so all is fine now besides the fact that im a noob in ML RAW.


Or not, i found other clip, reset all settings for correction , just ticked reconstruction and highlights down, theres pink, until i go raw white level to 3490
Ill upload the clip, IMO it has something to do with highlights being clipped, when theyre not clipped then default value of 3817 is fine.
IT would be supercool if app would detect clipped highlights and use 3490 if thats the case.

[2blackbar]

Heres the file
https://drive.google.com/open?id=1TNF4tVdaNMKJoAijhbYi1ftfrOGrh_d7
3817

3490

[Danne]

If using crop rec in this branch  crop_rec_4k_mlv_snd_isogain_1x3_presets always go with bitdepth reduction from with Crop mode sub menu. In RAW video keep it at 14bitlossless. More info here:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg208959#msg208959

EDIT: Seems you might have been running some RAW video setting. Anyway. To get most out of your eosm I recommend enabling cro rec module and run the presets in there.

[masc]

Per session you can enter white level once, then press Ctrl+C, select Whitelevel only, Ctrl+A, Ctrl+V.

[2blackbar]

Thanks a lot !
I also was wondering about blacks and darks noise....
It seems like black areas of the image are noisy even at ISO100, probably becaise constrast is flattened out compared to what we see in camera ( i was able to see in raw what was totally black in neutral pic style).Canon's contrast settings for pic styles hide noise in the dark.
So.. this got me thinking, if its possible to reconstruct highlights, is it also possible to reconstruct darks ? Green channel is lost in highlights and introduces magenta , if my thinking is right.
In darks there is green noise prevalent so maybe similar reconstruction to darks would help to at least smooth out the noise a bit ?
Or am i totally wrong and theres other cause of noise ?

[masc]

We don't reconstruct noise, we reconstruct blown out areas. 2 channels need to be correct for this. I can't remember having seen this for darks.

[2blackbar]

Would having chroma blur only for dark areas help ? Is it currentlyu for entire image , darks,mids and highs ?
Also do we have control over h.264 compression ? I could use more detail when exporting.Theres huge detail gap between mjpeg and h264/265 export
Im using LMMSE  which produces sharpest image but im getting artifacts(red dots ) when theres a lot of bright areas in the middle of image

Are these focus dots leftovers ? Thats what i think , maybe removal isnt working in the sam way when LMMSE is active

[ilia3101]

Would having chroma blur only for dark areas help ? Is it currentlyu for entire image , darks,mids and highs ?

It is currently for the entire image. Are you able to compile the app? as @masc has added really really good denoising that you could try, it can do what chroma blur does without ruining colour at edges.

Also do we have control over h.264 compression ? I could use more detail when exporting.Theres huge detail gap between mjpeg and h264/265 export

I think some progress may have been made with this since last release.