Author Topic: MLV App 1.14 - All in one MLV Video Post Processing App [Windows, Mac and Linux]  (Read 1096014 times)

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 849
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1675 on: July 07, 2018, 09:44:13 AM »
why a use a very chunky AVI codec (1.5Gb/s) ?
It was just a quick workaround to get rid of compatibility issues with NLEs. I would prefer to have cineform codec for avi.

the codec is called  "zoe-lossless-codec" to download codec or source code go to https://github.com/edanvoye/zoe-lossless-codec , supports 32 & 64 bit windows
Interesting. Is it fast enough?

I've been using UT Video under windows with very good success.

timbytheriver

  • Senior
  • ****
  • Posts: 476
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1676 on: July 07, 2018, 09:52:07 AM »
For a 10 sec clip you'll see the 100% for 5-20 minutes until it is done, depending on your hardware. If you stop export before, file is not playable.
Thanks masc. I understand. But the app actually became non-responsive during the 100% message – and locked up. Will try another export and wait longer.

5D3 1.1.3
5D2 2.1.2

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 849
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1677 on: July 07, 2018, 10:06:25 AM »
3) My favorite one is the MLV export.  Setting Cut-in and -out points, cleaning the focus dots and save as compressed MLV is extremely useful.  The resulting files open nicely in Resolve via MLVFS.
I think you did not thoroughly check dots on exported compressed clip. During MLV export (compressed or uncompressed) there is none of raw processing is done, it is intended to preserve original MLV raw data as is without changes.

bakersdozen

  • Freshman
  • **
  • Posts: 89
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1678 on: July 07, 2018, 10:18:17 AM »
it is intended to preserve original MLV raw data as is without changes.

I was also wondering if this was the case. Any idea of my previous question, which is a similar situation?ie. Applying the raw processing just without the curve during export to ProRes or other video codec?
EOS M + 5D3

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 849
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1679 on: July 07, 2018, 11:08:09 AM »
Any idea of my previous question, which is a similar situation?
It is not the same situation. When I'm talking about RAW processing I mean altering the RAW data itself by correcting focus or bad pixels and not debayering process and after that RGB data manipulation by curves etc.

Applying the raw processing just without the curve?
Well, I know nothing about LUTs you mentioned and do not know what data input they require. When you say it applied to DNGs we need to know which color space it initially was after debayer. E.g. linear or most probably black magic's own log color space (BMD Film) which we currently do not support in MLV App.

bakersdozen

  • Freshman
  • **
  • Posts: 89
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1680 on: July 07, 2018, 11:20:02 AM »
Thanks bouncyball. Further research tells me it is BMD film colorspsce and gamma.
EOS M + 5D3

IDA_ML

  • Hero Member
  • *****
  • Posts: 1014
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1681 on: July 07, 2018, 02:46:59 PM »
I think you did not thoroughly check dots on exported compressed clip. During MLV export (compressed or uncompressed) there is none of raw processing is done, it is intended to preserve original MLV raw data as is without changes.

Well, you certainly are right.  I tested just the 2520x1304 resolution and am happy to say that I do not see any focus dots.  Probably, MLVFS killed dem prior to opening in DaVinci Resolve.  Would it be possible to have MLVApp kill the focus dots with compressed MLV export?  No further RAW processing, just the focus dots.

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 849
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1682 on: July 07, 2018, 03:17:54 PM »
Yes it is possible as any other low level raw processing during MLV export. But IMHO it is pointless, especially focus dot fixing, which is so fast that has nearly no impact on performance. With MLVFS you just have to use the appropriate .fpm, and that's all. I'm too lazy to dig into and modify MLVFS code again for automatic focus pixel fixing ;).

IDA_ML

  • Hero Member
  • *****
  • Posts: 1014
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1683 on: July 07, 2018, 04:29:50 PM »
Yes it is possible as any other low level raw processing during MLV export. But IMHO it is pointless, especially focus dot fixing, which is so fast that has nearly no impact on performance. With MLVFS you just have to use the appropriate .fpm, and that's all. I'm too lazy to dig into and modify MLVFS code again for automatic focus pixel fixing ;).

As far as the focus pixels are concerned, you may be  right - we have MLVFS and it cleans them very well.  But, Dual ISO is a totally different story.  MLVFS does not convert Dual ISO files.  If that could be done in MLVApp right on the Dual ISO MLV file, along with the Cut-in and -out operation and subsequent compressed MLV export, that would be extremely useful.

I have been playing a lot with Dual ISO 100/800 at 2520x1304@18fps and processing with MLVApp v.017 and quality is fantastic!  I will upload a sample soon.

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 849
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1684 on: July 07, 2018, 05:01:39 PM »
But, Dual ISO is a totally different story.
I agree it IS ;) in lot of different ways, good and bad...

MLVFS does not convert Dual ISO files.
Ironically ( yeah this is really funny part ;) ), MLV App based on MLVFS dual iso source code! Which is on the other hand based on a1ex's cr2hdr sources with some specific adaptation to MLVFS idea.

Actually MLVFS was the first app to offer on the fly conversion of dual iso MLV to DNG. We just adapted this code for latest low bit uncompressed and lossless dual iso MLVs.

If that could be done in MLVApp right on the Dual ISO MLV file, along with the Cut-in and -out operation and subsequent compressed MLV export, that would be extremely useful.
Why don't you just export processed cDNGs and use them in resolve?

BTW MLV format been invented not for archiving purpose but for acquisition :) and keeping raw data intact IMHO is the most correct approach to save valuable original assets for future, better processing tools.

regards
bb

IDA_ML

  • Hero Member
  • *****
  • Posts: 1014
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1685 on: July 07, 2018, 05:30:06 PM »
... keeping raw data intact IMHO is the most correct approach to save valuable original assets for future, better processing tools.

Absolutely!  This is the reason why I want to keep the MLV files, compressed, clean from focus dots, properly cut and Dual ISO converted.  It would be great if all this could be done in one application that is as good and advanced as MLVApp and does not require the computer to be very powerful.  Tools for processing MLV/RAW files get better and better and it is much wiser to keep the MLVs rather than some other lossy compressed format that requires complex grading.  Moreover, I am not good at grading and to me the RAW workflow in Resolve (or ACR) is the simliest and easiest.


bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 849
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1686 on: July 07, 2018, 05:47:03 PM »
Absolutely!  This is the reason why I want to keep the MLV files, compressed, clean from focus dots, properly cut and Dual ISO converted.
I meant there also can be new, better dual iso processor or dot and stripe remover :) and if you delete vanilla MLVs there won't be the way back...

dfort

  • Guest
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1687 on: July 07, 2018, 05:49:17 PM »
@dfort, @bouncyball
Is it possible to transcode pixel coordinates into a fpm list? Let's say you create your own "dcraw" coordinates and then needs them working with fpm lists in mlvfs etc? Maybe doable through fpm utility?

Lots of activity here -- trying to keep up.

The main differences between a dcraw "badpixels" file and our fpm format is that dcraw includes the "UNIX time of death" and fpm needs to have either a header or the filename must include the hex code for the camera model and either the full raw buffer size or the image size. Note that the dcraw developer has documented the "badpixels" format but fpm is mostly undocumented.

If you add the metadata to the files you uploaded they might work without any further processing. Depending on the app you're using the trailing zeros are ignored. We just aren't interested in "UNIX time of death" for the pixels that we're fixing.

By the way, I'm all for fixing the focus pixels when creating the DNG files but prefer not to mess around with modifying the original MLV files. They should be archived the way they were saved in camera because some day we'll have better MLV processing tools and may want to go back to our camera originals.

dfort

  • Guest
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1688 on: July 07, 2018, 06:39:53 PM »
Is it possible to transcode pixel coordinates into a fpm list?

Just thought of something. The dcraw "badpixels" format files are formatted to fit the final image size while fpm files usually map the full raw buffer. You should be able to write a script that takes the Crop/Pan values along with the raw buffer size that is in the MLV metadata and re-map the coordinates so they line up with the full raw buffer.

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 2011
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1689 on: July 07, 2018, 06:48:39 PM »
Thanks for the clarifications.
currently I'm trying with a video that contains 659 frames
I wait until the message " export is ready" appears. this advice appears after only a few seconds that the bar has reached 100%. the duration of the complete export process is about four minutes for entire video.
maybe a record video could explain better than my bad english:
in this video there is the whole export procedure. I exported only 100 frames of not edited video, to speed up the process.
https://youtu.be/b6iqG3AXDxI
Thx for help! :)
Hm... what is the size of the file? Can any other application play it (VLC for example)? Does it work better without aliasing filter?

why a use a very chunky AVI codec (1.5Gb/s) ?
IMO there's far better open source  AVI codec  out there ,  it's nearly impossible be to work with .
Japp... I think the same. Someone had a wish to have that in MLVApp and it was easy to add... this guy will have a reason to use it. ;)

the codec is called  "zoe-lossless-codec" to download codec or source code go to https://github.com/edanvoye/zoe-lossless-codec , supports 32 & 64 bit windows
Hm... that's coded for Visual Studio. Unfortunately that is not very compatible to MinGW compiler (we use), if I understood right. I think we would have to "translate" some classes to Standard C/C++. Then it would be also cross platform. But that is much work!

Applying the raw processing just without the curve during export to ProRes or other video codec?
We could completely switch off processing... then you'll have a dark greenish picture and you nearly can't see anything (I made some performance tests in this way)... I don't think that is a good idea to export that.
I also would like to add BMD Film color space... but I don't find some useful specs about. I would need a formula from linear to BMD Film.
5D3.113 | EOSM.202

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 849
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1690 on: July 07, 2018, 07:09:49 PM »

I also would like to add BMD Film color space... but I don't find some useful specs about. I would need a formula from linear to BMD Film.
Yup, exactly. Besides there are several flavors of BMD Film for different sensors BM uses for their cameras.

We should ask Andy600 for help ;)

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1691 on: July 07, 2018, 08:27:54 PM »
Someone had a wish to have that in MLVApp and it was easy to add... this guy will have a reason to use it. ;)
Yes, he will... ;)
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 2011
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1692 on: July 07, 2018, 08:33:53 PM »
Yes, he will... ;)
Hehe, for you we had the 8bit rawvideo avi... do you use now 10bit V210 avi as well for your astro pictures?
5D3.113 | EOSM.202

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1693 on: July 07, 2018, 09:42:03 PM »
Hehe, for you we had the 8bit rawvideo avi... do you use now 10bit V210 avi as well for your astro pictures?
Oh, I thought, you talked about 8bit rawvideo. I should read more carefully  ;D
I just finished my project in porting my astrostuff from windows to openSUSE ( https://sternenkarten.com/kstars-libindi-astrometry-net-on-opensuse/ ) and only had a quick look here into the mlvapp-thread :)
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

IDA_ML

  • Hero Member
  • *****
  • Posts: 1014
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1694 on: July 07, 2018, 10:06:05 PM »
As I promised, I have prepared a short 100D video that demonstrates the advantage of filming high-contrast scenes  with Dual ISO.  For this purpose, the new 2,5-crop recording mode on the Rebels and EOS-M at 18fps frame rate works very well since there is no aliasing and moire whatsoever in this mode and also in the 3 and 4K modes.  Feel free to download the video in the next 7 days from here:

https://we.tl/GhLdeiloLw

While evaluating the comparison between the Normal and Dual ISO shots, please pay attention to the noise in the darkest areas of the shots (TV-set), as well as the detail recovery at the brightest ones (the curtains).  I have to say, I am amazed at how well version 017 of MLVApp handles these very difficult shots.  Except for some flashing of a pink ribbon in the window area, (probably a glitch in the highlight recovery function), video is very clean, with no evidence of focus dots or other artefacts and I am totally satisfied with the results. Again, CONGRATULATIONS to all developers involved in this remarkable achievement!

ArcziPL

  • Contributor
  • Member
  • *****
  • Posts: 190
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1695 on: July 07, 2018, 11:31:45 PM »
Hi,

each of my RAW videos recorded with 700D or M comes out in MLVApp very bleak, faint. Interesting fact: they always open with default white balance of 6000K and using the white balance picker on gray or skintone areas changes the setting to the range of 9k of 10k (and still looking crappy), despite recorded in normal dayligt, so there is definitely something wrong. I don't see any simple way to make it look right in MLVApp using the standard sliders.

Some sample MLVs downloaded from this forum do not show such problem but they were from another camera models.

My MLVs opened in Fast Cinema DNG or Adobe Camera RAW (single DNGs extracted using MLVFS) look normal.

I'm recording using mlv_lite from crop_rec_4k_mlv_snd from https://builds.magiclantern.fm/experiments.html using 5x zoom, mostly 12-bit lossless but in full frame mode it was the same. MLVApp tested 0.15, 0.16, 0.17. All are same.

Where does the problem lay?

MLVApp with default settings:


Fast Cinema DNG with default settings:
M50.110 [main cam] | G7X III [pocket cam] | 70D.112 [gathers dust] | M.202 [gathers dust] | waiting for M5II

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 2011
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1696 on: July 07, 2018, 11:39:31 PM »
@ArcziPL: when opening a MLV it is very normal if colors a bleak. WB at 6000K is also normal when you recorded in auto mode. With WB picker I had no problem for all files I got from users, also no problem for EOS M and 700D. So best would be you upload a sample file (you can shorten it before to some frames). In different programs it will look different - sure.

@IDA_ML: thanks! Looks great! Could you provide the clip with the blinking highlight? Maybe my chosen recovery-range is to small...
5D3.113 | EOSM.202

ArcziPL

  • Contributor
  • Member
  • *****
  • Posts: 190
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1697 on: July 08, 2018, 12:27:04 AM »
@ArcziPL: when opening a MLV it is very normal if colors a bleak. WB at 6000K is also normal when you recorded in auto mode. With WB picker I had no problem for all files I got from users, also no problem for EOS M and 700D. So best would be you upload a sample file (you can shorten it before to some frames). In different programs it will look different - sure.
Thanks masc, this helped already. I didn't expect having to touch most of the sliders to make it look similar to what other programs usually default to. But after doing so, indeed, the results are VERY GOOD!
M50.110 [main cam] | G7X III [pocket cam] | 70D.112 [gathers dust] | M.202 [gathers dust] | waiting for M5II

feureau

  • Hero Member
  • *****
  • Posts: 604
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1698 on: July 08, 2018, 04:05:36 AM »
MLV.App.v0.17.alpha.Win64.static keeps crashing when batch exporting with Dark Frame Subtraction enabled.

Steps to reproduce:

Import multiple MLVs into Session, enable DarkFrame Subtraction Ext with pre-prepared MLVs for each file, batch export all MLVs as prores.

First MLV will complete render just fine, then MLVApp will crash on second or third MLV.

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 849
Re: Making a New MLV Processing App! [Windows, Mac and Linux]
« Reply #1699 on: July 08, 2018, 08:41:53 AM »
each of my RAW videos recorded with 700D or M comes out in MLVApp very bleak,
In current MLV App by default saturation is quite low. So crank it up and everything's gonna be fine :)