Messages

Hi Goran, cool to see this in action!
FYI, I was the one providing the log formulas
https://bitbucket.org/baldand/mlrawviewer/commits/406eaabb69134fd2a9189971187c7063f0a67825

Just my 2c: I'd recommend to stick with S-Log2, as this is the most accurate formula that was available (via Sony whitepaper). S-Log2 is S-Log with a different exposure (mid gray at different value), but the same formula. There are also interesting 1D technical LUTs available (S-Log2 to r709 for example).

Cheers and rock on!

@Tos_Maverick. ah, ok, thats nothing especial. for this "media"-behaviour you just need kind of filenaming-convention.

folder: blabla_C0000
dng-file: blabla_C0000_00001.dng
(for example)

thats the reason i built the filenamegenerator into raw2cdng. interesting would be the part with embedded audio, does this work? look here and here.

Ha, you are right, it works now!
Now it would be great if MLRV does that out of the box

Can't help you with audio, my clips are .RAW (v1) without audio...

TOS_Maverick could you give us so me info (folder/filenaming) and at least one dng and a WAV?

Here you go:
http://www.michaelfischer.at/files/DNGDiff.zip

The RAWMagic variant (C0...) works fine, even only with this one DNG.

Thank you for your efforts Andrew!
cDNG compression works great!

1) I can confirm though, that the exposure is a little different than the RawMagic conversion. Something about 0.4 stops. My RM export needs 1.25 stops adjustment, your DNG export needs 1.65 stops. After that, the result seems to be the same (Color etc.)
2) Somehow RawMagic's cDNG is recognized by Resole on the spot, with MLRV's export I need to enter the directory to access the clip. Any idea why that could be? Some header missing in the dngs?

Let me know if you need further information!

Can't find how to remove imported LUTs on Mac? I'm imported too many, and now try to remove some of them.

You have to go to the hidden ".mlrawviewer" folder in your home folder and delete the lut3d file. This will purge all 3D LUTs.
There is no way at the moment to remove individual ones, just the whole bunch!

Also, I just had a look at the bits you are working on (Slog/Slog2 etc). We did a lot of work to find an idealized log gamma for MLV for output to 10bit ProRes/DNxHD using the math from Arri, Sony, Canon etc and anything above Cineon log (~13.5 f-Stops) tends to degrade the image when linearized. It's not apparent in every shot so it might not be obvious without shooting a lot of test charts, a backlit DSC Xyla DR chart etc.

For MLV, Slog and Log-C are basically wasting a lot of space. Also, they tend work best when paired with their associated gamut (i.e. Log-C with Alexa Wide Gamut RGB, S-Log with S-Gamut, Canon C-Log Wide Gamut) and always require a transform lut unless the colorist really knows what they are doing. BMD Film 4k (gamma) is close to ideal for MLV and you should be able to derive the transfer function from the BMD luts that come with Resolve.

Hi Andy, thanks for chiming in!

We are aware that S-Log and Log-C cover a huge DR spectrum. S-Log2, with the DR from a 5D for example only covers 75 % of the resulting 10 bit range.
That's because of the huge range above middle gray. When the 5D3 clips, there's still 2-3 stops of additional space in S-Log2.
I think it's still useful for compatibility.

@Tos_Maverick - BMD Film is (afaik) a proprietary, non-linear colorspace with no published information on how to get to and from it. Until Blackmagic release this info (they have no plans to do so) you will never get 100% accurate color, although it does still look nice (film-like) to a lot of users. The new Resolve Color Match tool requires you to know 100% what colorspace and gamma the source footage is and, even when this is known, it will only get you closer to photometric color and TBH this is nearly always best done manually with scopes. It currently falls apart with anything when trying to match the DSC One Shot chart (even Alexa, Red, F55 etc) and never gives uniform results with MacBeth charts. It's a nice idea but it's not quite there yet - use with extreme caution.

Coming back to what's the optimal DR for our cameras or the MLV format. The log curve that hit's middle gray perfectly and uses the space most efficiently is IMHO the 8 stop variant.
I don't know what BlackMagic is doing, but they seem to have come to a similar conclusion.

Look at what Color Match has to say, when you feed it with our Log8 ProRes input:



That's not an exact science for sure, but it seems to work and you can transform it to various other gammas as well.

so i tried different workflows and they are all not ideal at the moment, cause of the wrong primary color profiling of ml raw material.. (this will be a problem until some of the major companies start to support the Canon Raw Sensor Output of those cameras (eg. 5dMK3 - which could be  seen as best option for ml raw filmers at the moment)
when using resolve we are forced to use profiles like bmd4k or bmd, rec709, ect. and they are not working very well with all the different luts cause the colors don´t look right..

Hi swinxx,

do you happen to have a ColorChecker chard available?
If you import cDNGs in Resolve, use BMDFilm and then use the "Color Match" function, you will get correct colors!
https://www.blackmagicdesign.com/products/davinciresolve/color

This also works with Log8 footage. BMDFilm seems to be really close to Log8, therefore this works with ProRes footage as well. I tested this and got 99 % identical results!

BTW, there's something going on behind-the-scenes as well
https://bitbucket.org/baldand/mlrawviewer/commits/406eaabb69134fd2a9189971187c7063f0a67825

LUT Import Test with VisionColor Osiris

I'm really puzzled that this looks like that
How does it look like if you set it to Linear => Linear to Log 8 => Vision Color LUT?
You then can control the contrast by switching to a lower log curve and adjusting exposure.

Going sRGB => Log 5 = > VC might cause a strange gamma curve...

Just my 2c

The built in Log Luts are amazing! Having Log5-16 like that is great! Would it be possible to add 1-4 though? Could be useful when combining with other Luts.

Just FYI, the number indicates roughly the amount of DR/stops in the picture. The 8 stop variant (Log 8 ) seems to be the sweet spot for capturing most of the Canon sensor's dynamic range vs the noise and also seems to work well with ProRes. If you use a more contrasty log curve, you can get away with more compression (ProRes 422 for example).

Hi Kurt, ProRes 422 can be configured easily. If you need a different picture size, you can add a resize filter to the ffmpeg string.

Have a look at page 31, everything should be there!

Also the "log" doesn't look right. It's all pink and too bright. It has to be changed to represent something like the BMCC's log which is more accurate for color correction and appropriate for this kind of sensor.

Hey Guys, nice to see you here!

I don't have any color-cast issues on my Hackintosh at all (after setting WB, but I assume you did that).
Specs are i5 3570k, GTX570 TI, 16 GB RAM

Regarding the log curve contrast I have the following proposal:


or in Derive notation:
y = LOG((x + 2^(14 - z))/2^(14 - z), 2)*(1024/z)

"z" describes the amount of DR you want to map logarythmically into the picture. Don't know if that is a proper sentence, but I hope you know what I mean

Currently it's at 10, which brings 0 EV / 18 % grey in at 60 %. 7,8 should bring it to 50 % and 6,2 to 40.
From my PoV 8 stops of DR should be sufficient, since the Canon cameras are quite noisy in the shadows anyway.

If you would need more DR, the HDR curve is quite nice as well and, if exposure is adjusted, looks similar to Adobe Camera RAW's gamma.
You need to expand the highlights a bit, but that's a quick fix...

Finally I've tried DNxHD and was able to get "real" 8 bit data, since this is explicitly "supported" (ProRes defaults to 10 bit no matter what).

It looks like this:



The 10 bit DNxHD is comparable to PR422 but at a higher file size.

If you want to try for yourself, here are the settings:
DNxHD
-f mov -vf vflip -vcodec dnxhd -b:v 185M -pix_fmt yuv422p10le %s.MOV

DNxHD 8 bit
-f mov -vf vflip -vcodec dnxhd -b:v 185M -pix_fmt yuv422p %s.MOV

ProRes 422
-f mov -vf vflip -vcodec prores_ks -profile:v 2 -pix_fmt yuv422p10le %s.MOV

ProRes 422 HQ
-f mov -vf vflip -vcodec prores_ks -profile:v 3 -pix_fmt yuv422p10le %s.MOV

ProRes 4444
-f mov -vf vflip -vcodec prores_ks -profile:v 4 %s.MOV

You can also try, e.g. "-q:v 2" or "-q:v 0". All those settings should affect the final bitrate (and hence quality)

(The main reason to use the fixed qscale is that it encodes much faster, but if it turns out to be causing poor quality that will have to change!)

Thank you for the suggestions, that did work!

Q0 was clean, Q2 was bigger than Q0 and showed macro blocking. Q3 is the same size and shows blocking as well. Seems the bitrate distribution is different right?



I think we should be safe using this:
-f mov -vf vflip -vcodec prores_ks -profile:v 4 %s.MOV

Hi Andrew,

thank you for making the FFMPEG settings configurable!
I was playing around with it and it seems I've opened pandora's box.

I was curious if I could see a difference between ProRes4444 and plain old H264 in the form of X264 @ CRF 15, which should be transparent. I noticed no difference with the avg bitrate being around 25 mbit. Then I stumbled upon this thread:
http://forums.creativecow.net/thread/291/1137

It seems that FFMPEG converts RGB48 to YUV in 8 bit, which would be really bad. I fired up Davinci Resolve and had a look for myself.

I exported a clip with MlRawViewer in Log and Resolve with the BMDFilm gamma curve and matched the gamma curve between the two. Then I crushed the blacks severely to see if banding occurs.

Here are the results:




Conclusion:
FFMPEG seems to indeed process in 8 bit and ProRes4444 in Resolve is worth the extra bandwidth/bitrate.

I don't know if it's possible to feed FFMPEG directly with yuv444p12le to avoid the conversion but I think so. The conversion code for MlRawViewer has to be written though...

Hi. There is no option for low res proxies at the moment.

I saw in the code that FFMPEG is executed with "-profile:4". No need for another symbol in the UI, but it would be really cool to switch between profiles or even the constant quantizer mode (maybe a config file?). I think ProRes 422 would sufficient for most cases...
http://transcoding.wordpress.com/2012/01/29/prores-ffmpeg/

- Fix for default brightness of log output (it was 16 times too dark)

I've also changed my build environments for both Windows and Mac. It's possible this will cause some problems for some people, but I hope it won't.

Please let me know here if you have tried this version out, and if it worked ok, or if you saw some problems.

Thanks!

Hi baldand,
thank you for the fixes!

The brightness of the log output is correct now indeed! I'm mainly using OSX but was trying MlRawViewer on Linux as well and noticed the brightness difference in 1.1.6. Seems it was working correctly in Linux!
Does that only apply for the Win/Mac build?

Didn't test much else but it felt quite solid!
Was the WB/Exposure saving feature (G and H keys) persistent between sessions in the last build as well? Just noticed yesterday while playing around

BR
Michael

Hi Michael,

My goal with the log curve was simply to have a reversible mapping function which prevents some of the detail in the darker areas being lost during the reduction from 14bits to 10bits that happens when encoding to ProRes. In theory, the data could be linearised afterwards with a matching inverse function. However, I haven't made such an inverse function (e.g. in a LUT file) to do this, so it's still just a theory.

It would be possible to change the existing log implementation, or add new ones, or add LUT support so that this could be made part of a tested workflow. But I would probably need some help with in terms of requirements, code contribution and testing.

Hi Andrew,

thank you for your kind reply!

Could you get back to me regarding the default exposure compensation in log mode?
It would be really helpful to have it preset so that 18 % grey is at 40 IRE without adjusting

At the moment I have to press 13 times the UP key, which should be a factor of 3,45 if my math is correct (1 press = 1.1 times increase, right?).

The appropriate curve would look like that:

Has anyone tested how the log converted prores files react when converted to VISIONLOG\ Cinelog? Im curious if we are still able to use that workflow for grading if we convert toREC709 or LOG Prores in the viewer (as that would save LOTs of space), then convert to standardize VISIONlog, and hand grade or add OSIRIS\Impulz LUTs as desired?

Anyone had success with that?

We will be shooting a short movie where we will be going RAW => ProRes w/log curve => Edit => Color grade.
I'd like to keep things simple and still retain most of the quality. So far this workflow super easy once you are in ProRes and the quality due to the log space is amazing.
Also the color matrix seems to be really good! A liberation compared to the builtin H264 video

Why would you go from Andrew's log implementation to Cinelog? Is it really that different?

Hi baldand,

first of all, thank you for this great tool!

I have a few questions regarding the default exposure of the log curve.
I'm usually very careful about my exposures and just checked, with RawDigger, where 18 % grey falls in the linear space. It seems Canon places it in the area around 1024, which is at -4 stops.

The sRGB curve that you use for example, seems to use 2048 for it's grey value.

I've checked the code as you suggested, fired up good old Derive and plotted a chart, normalized to 10 bit output (0-1023).
This is what I got:


According to this, the 18 % grey, exposed with a light meter, should fall at 60 % or 614.
I tried checking the code for any default exposure correction value, but could not find it so far.

Could you maybe explain why the log curve is a bit darker?
Or did I overlook something?
Arri is putting grey at 40 % IRE for example, Sony with S-Log around 38 %:
http://blog.abelcine.com/wp-content/uploads/2009/12/SRW_ITG_S-Log_001_IO_EN.pdf

My goal is to expose correctly, set WB, switch to the log curve and batch convert to ProRes.
In my NLE I would then apply a log2r709 curve, with an S to avoid clipping.

Just my thoughts/2c,
BR,
Michael