MLV App - All in one MLV Raw Video Processing App [Windows, Mac and Linux]

Started by ilia3101, July 08, 2017, 10:19:19 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

RhythmicEye

All sensors use color science to record light hitting the sensor. It's the color science of the sensor we are talking about not the RAW DNG file. RAW files regardless of flavour still have color science which is determined by the sensor when recording light/color information.

Camera raw data is usually proprietary to each manufacturer. Which is why I'm asking about ML DNG files seeing as it's using a Canon sensor to capture the .MLV file.

names_are_hard

I'm not quite sure what you mean.  Different sensors will have different physical properties, including different sensitivities to different frequencies of light.  But they don't have a colour space.  I thought you meant colour space by "color science".

I was originally responding to you saying "I'm assuming ML must use some form of Canon Log for the color space data" - ML does not.  ML doesn't alter the colour space of JPG and raw doesn't have colour space.

vastunghia

Quote from: names_are_hard on March 28, 2023, 03:33:49 AM
Are you talking about raw images here?  Those don't really have a colour space.

+1

That was the starting point of my answer above. There is no CST that needs to be handled by DVR as there is no image demosaic-ed already with some CS baked in. DVR needs to do the demosaic itself, so it can pick already the correct working CS.
5D3 for video
70D for photo

RhythmicEye

Quote from: vastunghia on March 28, 2023, 07:36:06 AM
+1

That was the starting point of my answer above. There is no CST that needs to be handled by DVR as there is no image demosaic-ed already with some CS baked in. DVR needs to do the demosaic itself, so it can pick already the correct working CS.

Just to be clear I'm not asking about color space nor CST in DR. I'm asking about how ML handles the sensor data from the Canon sensor when capturing the .MLV file.

ML is obviously not capturing a Canon RAW file but it's clearly using a Canon sensor that's capturing a CinemaDNG file. What is happening in that transformation? That is the nub of my question

There are 9 different varieties of Camera RAW in DR all based off the manufacturer's sensor data
1. ARRI Alexa 2. Blackmagic RAW 3. Canon RAW 4. CinemaDNG 5. Nikon RAW 6. Panasonic Varicam RAW 7. Phantom Cine 8. RED 9. Sony RAW


names_are_hard

CinemaDNG isn't used at any point by ML, or by Canon, as far as I know.  Where are you seeing this?

ML takes the sensor data from camera memory and copies it unchanged into MLV files.  I would expect CR2 files to use the sensor data in the same way; by not changing it.  The file format of CR2 differs, the sensor data should be the same.

vastunghia

+1, once again.

Raw data is just a collection of 14-bit values (think of them as numbers ranging from 0 to 2^14-1) each representing relative amount of light (think of this as a linear count of photons) collected by one of the photodiodes on the sensor.

These photodiodes are arranged in a RGGB (as proposed by Bayer first) square pattern, and each of them can collect light only in R, G or B channel.

Simplifying things a lot, if your sensor has N pixels, it means that it has 4*N photodiodes. Demosaic methods have the task of transforming this 4*N-long 1-dimensional array of data into an N-long 3-dimensional (R,G,B) array. Then, this resulting data is also tone mapped according to some sort of gamma(*), and color-coded according to some CS (still simplifying here). Finally, it can be saved as, say, JPEG (or, say, MPEG if we are talking video).

So if you are working with Raw data, demosaic has not yet been performed and you are presented with bare numbers representing amount of light collected on sensor (precisely "raw data", that is).

Only if you work with non-Raw data then you know that color information has already been elaborated and coded according to some predefined CS. Then you have to take this into account in NLE such as DVR in order to transform CS from input to working CS, all the way to output CS.

Don't be fooled by the fact that there are different formats of Raw files: these are just different formats of the container of the data. Raw data is raw data, it is always the same regardless of how it is stored, i.e. it has always the same meaning and once you crack open the container you are always presented with 14-bit series of photodiodes reads.

HTH

PS: may be worth playing with pure raw data to get acquaintance with it: found this, haven't tried, but seems to work. Personally I worked a bit on raw data in PixInSight, cfr for instance my report here.

(*) Human perception is nonlinear (roughly logarithmic) and such that two perceived light *additive* gains are equivalent only if photon count is *multiplied* by the same amplifying coefficient — e.g. you need to pass from 2 to 4 photons in order to get the same gain that you obtained in passing from 1 to 2 photons.
5D3 for video
70D for photo

RhythmicEye

Quote from: names_are_hard on March 28, 2023, 08:48:06 AM
CinemaDNG isn't used at any point by ML, or by Canon, as far as I know.  Where are you seeing this?

ML takes the sensor data from camera memory and copies it unchanged into MLV files.  I would expect CR2 files to use the sensor data in the same way; by not changing it.  The file format of CR2 differs, the sensor data should be the same.

Again my bad, I meant DNG not CinemaDNG. An .MLV file is just a container for a sequence of DNG files. I use MLVFS to unpack them real time so I can grab still frames and short DNG image sequences easily. MLV App is a much better workflow though.

https://www.magiclantern.fm/forum/index.php?topic=13152.0

Anyhow you just answered my question which gives me the information I was looking for.

Thanks for your patience.

names_are_hard

I don't think MLV contains DNG files, these are different formats.  MLV is a container for raw sensor data, as is DNG.  I think MLFS converts MLV to DNG.  Could be wrong, haven't checked the code.

Skinny

If you want to know how to color correct mlv raw files to match other cameras, you may look at what photoshop does with it's camera profiles..
For example, if you convert mlv to dng and open it in photoshop, it will look almost the same (or the same) as usual cr2 file from the camera.

So it is obviously the same raw data. I don't know what adobe software does with color, but they have profiles for all cameras somewhere in folders, you can find there your canon camera profile. Maybe this profile can be converted to some LUT or something somehow...

masc

Quote from: RhythmicEye on March 28, 2023, 10:31:53 AM
An .MLV file is just a container for a sequence of DNG files.
That's wrong. MLV and DNG are different formats, which hold the same RAW image data, but in a very different internal structure.
For color calibration MLVApp uses the camera matrices. No idea what DR does here.
5D3.113 | EOSM.202

RhythmicEye

Quote from: masc on March 28, 2023, 03:58:59 PM
That's wrong. MLV and DNG are different formats, which hold the same RAW image data, but in a very different internal structure.
For color calibration MLVApp uses the camera matrices. No idea what DR does here.

Thanks for the explanation. MLVFS converts so quickly it seems to be more like a wrapper removal than a transcode.

Always happy to be wrong in the process of learning.

RhythmicEye

Quote from: Skinny on March 28, 2023, 01:23:03 PM
If you want to know how to color correct mlv raw files to match other cameras, you may look at what photoshop does with it's camera profiles..
For example, if you convert mlv to dng and open it in photoshop, it will look almost the same (or the same) as usual cr2 file from the camera.

So it is obviously the same raw data. I don't know what adobe software does with color, but they have profiles for all cameras somewhere in folders, you can find there your canon camera profile. Maybe this profile can be converted to some LUT or something somehow...

Good suggestion but for Davinci Resolve I just needed to confirm the flavour is Canon RAW sensor data. That's what I assumed but was never 100% due the .MLV to DNG file conversion.

Now for some lighting and color chart testing to see just how accurate the color reproduction is for each of the various RAW flavours within DR.

Cheers RE

togg

To get the same look in Resolve you can use rec709, no need to make it more complicated. It's just a curve that MLV app does, unless I am wrong. I played a bit with the raw panel contrast, gain, and a curve. Almost perfectly match.



masc

I think you are wrong. All the detail comes out very different and shadows look just dead with this setting. It is way more than just a curve... If you look for the MLVApp look in another application, the closest should be ACR. Using Resolve for MLVApp look this is a really hard task.





Your setting without wheel and curve:
5D3.113 | EOSM.202

vastunghia

You cannot think that one simple preset with some gain and contrast will be enough to match MLV App rendering with defaults settings in DVR for *any* scene.

That being said, of course, matching MLV App in DVR *is* possible, for any scene -- it just takes some time to tweak settings.
5D3 for video
70D for photo

Skinny

Quote from: masc on April 12, 2023, 01:27:29 PM


By the way if you compare these two images you will notice that MLV App version has some kind of yellow tint over the image, at certain areas it is more visible.. While LR does not have this issue. It is always present in the mlv raw footage opened with MLV App. So I usually start tweaking white balance to add blue but that will break other colors and the whole balance.. I can fix it to some degree with LUTs but sometimes it is just so hard to do it, especially if you want warm image but without this unpleasant color..

So guys (and Ilia) if you will have some free time, could you look at MLV App color science and maybe tweak some things here and there? This is probably the number one issue I always have with MLV App, this yellow cast.. ::) :)

elenhil

Guys, I've a question about highlight recovery and bit depth. I just reread https://www.magiclantern.fm/forum/index.php?topic=24933.0 and was mightily surprised by the findings, because in my 70D experience with MLVApp highlight recovery for 10-bit material is vastly inferior to 14-bit (to the point that I stopped shooting in anything other than 14-bit).

Is ACR's highlight recovery so much more advanced (or MLVApp's so much more... the other thing) that the latter can recover almost nothing from blown highlights in 10-bit (I just get pink areas), while the former can?

Отправлено с моего SM-G960F через Tapatalk


masc

@elenhil: Blown out highlights means: absolutely no information on 2 (GG) of 4 (RGGB) pixels. And yes, ACR is very powerful here, creating information like magic. The algorithm is Adobe's secret. We can't just copy it (same for everything else from Adobe). On the other side, MLVApp is always able to fix the pink itself: adjust RAW white level as written in the manual/wiki. Bitdepth don't cares for that. If one likes the result is another question. Best bet is to not overexpose.
5D3.113 | EOSM.202

elenhil

Thanks for the explanation, masc! I am a sucker for ETTR, so overexposed highlights is something of a job hazard.

I am curious about 11-bit vs 14-bit highlight recovery in MLVApp, though. Why is 14-bit material so much more accommodating in this regard (at least out of the box, without tweaking RAW white level first)?

masc

White level is hard coded, not measured. But real white level might vary. No idea how good the hard coded value is for all the possible settings and scenaries. If it is to high, highlight recovery (MLVApp) can't work. If it is to low, you lose dynamic range. For most cases, 2nd is not good.
5D3.113 | EOSM.202

Deadcode

Could you please improve multithreading support using some help from ChatGPT?

i just tried to do a benchmark with a 10 sec long dual ISO 1736x2214 clip from my 700D, and i have got interesing results

Converting to Lossless CDNG with Dual ISO turned ON, everything else is default:

M1 Macbook with the Arm64 version of MLVAPP: about 5 minutes render time
AMD Epyc based vitrual machine in tensorcore cloud computing service with 6 cores with Win64 Dynamic version of MLVAPP: about 16 minutes
AMD Epyc based vitrual machine in tensorcore cloud computing service with 44 cores with Win64 Dynamic version of MLVAPP: about 15 minutes

Converting to Lossless CDNG with Dual ISO turned OFF, everyting else is default:

Macbook M1: 30 sec
AMD Epyc 6 core: 40 sec
AMD Epyc 44 core: 40 sec

theBilalFakhouri

Quote from: Deadcode on April 14, 2023, 10:07:05 AM
Converting to Lossless CDNG with Dual ISO turned ON, everything else is default:

Currently Dual ISO code in official MLVApp reop doesn't support mutlithreading. There is a version made by @iaburn has multithreading support for Dual ISO, you find compiled Windows version here:
https://www.magiclantern.fm/forum/index.php?topic=20025.msg241996#msg241996

Could you run the same test on AMD Epyc (at 6 and 44 cores) with this version and Dual ISO setting enabled?

iaburn

There is an improved version (fixed exposure consistency across frames) thanks to @cedricp, best is to download and compile directly: https://github.com/anibarro/MLV-App

I found issues with weird colors while exporting to cDNG, and also this version defaults to option 3 for focus pixels removal, which I realised is not always the best for non-dual ISO.
I was to lazy lately to take a look  :-[

masc

Quote from: Deadcode on April 14, 2023, 10:07:05 AM
Could you please improve multithreading support using some help from ChatGPT?

...

Converting to Lossless CDNG with Dual ISO turned OFF, everyting else is default:

Macbook M1: 30 sec
AMD Epyc 6 core: 40 sec
AMD Epyc 44 core: 40 sec

If someone has a working multithreaded lj92 version for lossless encoding&decoding, please let me know. (As far as I know this is technically impossible, so runs always single threaded, for monochrome image data.)
5D3.113 | EOSM.202

Deadcode

Quote from: theBilalFakhouri on April 14, 2023, 10:42:47 AM
Currently Dual ISO code in official MLVApp reop doesn't support mutlithreading. There is a version made by @iaburn has multithreading support for Dual ISO, you find compiled Windows version here:
https://www.magiclantern.fm/forum/index.php?topic=20025.msg241996#msg241996

Could you run the same test on AMD Epyc (at 6 and 44 cores) with this version and Dual ISO setting enabled?

I tried iaburn's 1.4 build with the same footage
4 threads: 5min31sec
32 threads: 5min04sec

masc: dumb question: it's not possible to force the threads to work on different image frames?