Magic Lantern Forum

Using Magic Lantern => Raw Video => Raw Video Postprocessing => Topic started by: baldavenger on September 01, 2015, 11:41:51 PM

Title: DaVinci Resolve and ML Raw
Post by: baldavenger on September 01, 2015, 11:41:51 PM
The latest release of DaVinci Resolve (free version) makes working with ML Raw video a doddle, and thanks to developments such as MLVFS the overall post-production process has come a long way since the early days.  Anyway, this thread is for anyone who wants to post advice and tips with regards to using ML Raw and Resolve 12.


The introduction of Resolve Color Management is the big game changer.  My preferred method is working in Alexa LogC, and this is now I go about it:

In the Master Project Settings, choose DaVinci YRGB Color Managed (as opposed to the default DaVinci YRGB)

In the Color Management section of Project Settings there is Input Colorspace, Timeline Colorspace, and Output Colorspace.  The way RCM works is that it converts all footage to Scene Linear at the initial stage and from then on it converts to whatever colorspace is selected for the timeline, and from there onto the output transform.  For non raw footage, it's necessary to correctly input the footage colorspace otherwise the process won't work properly.  With raw footage that's not necessary, as RCM can directly debayer into your chosen Timeline colorspace.  The same goes for Output Colorspace, but I select Bypass in the options as I prefer to see what's going on under the hood.

Selecting Alexa LogC for Timeline Colorspace, and Rec.709 Gamma 2.4 for Output Colorspace (it doesn't make any difference what is entered in the Input Colorspace field with raw footage), the image you see on your monitor will be the same as the one you see when you're in normal DaVinci YRGB and you picked Rec.709 for Color Space and Gamma in the Camera Raw panel.  By selecting Bypass as the Output Colorspace option, you will see your image in LogC Gamma and Alexa Wide Gamut.  The Arri LogC LUTs that come with Resolve don't expand the signal out to Gamma 2.4, which is annoying, but there is a solution to this.  A free website that produces and coverts LUTs.  It's awesome.

http://cameramanben.github.io/LUTCalc/LUTCalc/index.html

Here's a screenshot with the settings I used to generate a LogC Alexa Wide Gamut to Rec.709 Gamma 2.4 LUT.

(http://i.imgur.com/ucOIgzp.png)


The website can be used to generate Gamma or Gamut only LUTs, so you can isolate both parts of the process.  You can also import a LUT and modify it for your needs.  Very handy if say you like the colour transform of a LUT but the contrast is too much and you'd rather replace the Gamma function with something more appropriate, or even remove that part altogether and do the tone curve yourself.

That should do for now.  More updates will follow  :)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 02, 2015, 10:58:05 PM
Right.  Now that a basic workflow has been established (Alexa LogC), it's time to look at a common issue that arises when dealing with footage in a log space.  It's really hard to pull keys when the footage is flat.  This is how I get around it.

Using the website from the previous post we were able to get an adequate LogC Alexa Wide Gamut to Rec709 Gamma 2.4 LUT, so that goes on the last node and we grade under it to take advantage of the log space.  A basic set-up would be this:

(http://i.imgur.com/cZPwvIH.png)

On the first node I made a layer mode, then added a serial node to the underlying (hidden) node.  The last node has the LogC to Rec709 LUT applied.  The same LUT is applied to the first (underlying node).  Nothing that happens to nodes 1 and 4 above have any effect on the main stream as node 3 has a full alpha channel by default so blocks their input.  You can pull a key on node 4 from a full Rec709 image and pump the alpha channel directly to node 5.  Pretty straightforward, and in no way demanding on your computers resources.

To pull multiple keys, you simply incorporate parallel nodes after node 1:

(http://i.imgur.com/l5a3XR9.png)


All this will be handy to anyone who uses DaVinci.  In upcoming posts I plan to look at working with separate LUTS for Gamma and Gamut functions, replacing Gamma LUTs with powergrades, and eventually finding a way to replace Gamut LUTs with powergrades too, therefore creating a LUT free environment.  I'll be looking at some popular creative look LUTs and seeing if they can be modified to suit Magic Lantern Raw video better, ultimately in the form of accessible powergrades.

Good times ahead folks :)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on September 03, 2015, 07:41:32 AM
Very useful, if I can get it to work :).  Tried a while ago to alter a lut. Thanks for sharing.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on September 03, 2015, 07:58:21 AM
Great read and thanks for sharing once again @baldavenger -- now I need a new Mac to run DR!
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Andy600 on September 03, 2015, 01:56:22 PM
@baldavenger - Good tips ;) I would add a couple of things. The Log-C to Rec709 transfer function will clip highlights at output as there is no s-curve. Try adding soft-clipping to the lut in the Lut Panel and export a new one. This will clamp the output but allow you to smooth highlight rolloff (it's not the best way to do it but highlights should look better and not affect midtones/shadows).

Also, if you are explicitly stating a legal levels output you need to check your output codec and settings as you may get a gamma mismatch, especially in other apps or on Vimeo, Youtube etc. Legal output is for deliverables. Resolve can automatically select the correct levels according to the codec if you are using the full color management pipeline (input/workspace/output) but because you are overriding the last stage with a lut you should probably test to see what happens should someone want to render something other than 8-bit (H.264 etc) and set levels accordingly in the main settings.

Also, the last time I checked, the native Log-C to Rec709 lut in Resolve wasn't correctly transforming the colorspace. It does something weird to the gamut and I'm surprised no one has ever mentioned it (caveat: haven't checked the v12 version). The online ARRI lut generator is also worth a mention here, especially for this workflow http://www.arri.com/camera/alexa/tools/lut_generator/ (http://www.arri.com/camera/alexa/tools/lut_generator/).

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 03, 2015, 03:39:35 PM
Thanks Andy600, I'll be getting round to all those issues soon enough.  The Data to Legal settings featured in the original post are based on the ARRI LUT Generator, as it always stipulates that when producing a Log to Video LUT.  Anyway, all other options will be explored in due time.  The clipping issue you speak of is somewhat offset by mapping reflected to 95% instead of 100%, though I am looking into a better approach.  Have a play with Resolve 12 and try out the approach with a grey ramp and ML Raw DNGs.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 03, 2015, 08:04:50 PM
Right.  Time to look at separating the Gamma and Gamut functions from the LUT.  Although convenient to have both operations performed by the same LUT, in the long run I feel it's better to have them in separate nodes.  In LUTCalc you can generate a LogC Alexa to Rec709 2.4 Alexa (for just Gamma conversion) and also LogC Alexa to LogC Rec709 (for Gamut).  Gamut transforms can be quite complex, involving a 3x3 matrix, so for now we'll stick with the LUT.  They tend to be accurate and not destructive, so as LUTs go they're not so bad.  Let's see if we can do something with the Gamma LUT.

Import a grey ramp into your Timeline in Resolve.  Here's a good quality Tiff you can use:

https://www.sendspace.com/file/ncno7k

In your waveform there should be a direct diagonal line from 0 to 100.

(http://i.imgur.com/9Fp9nba.png)


Enable RCM in the Project settings, and select Rec709 Gamma 2.4 in the Input Colorspace, Alexa LogC in the Timeline Colorspace, and Bypass in the Output Colorspace.  The waveform should now resemble this:

(http://i.imgur.com/L7mZOdm.png)


Open the curves section and perform something similar to this, with similar results:

(http://i.imgur.com/mzRedv2.png)

It's by no means perfect, but it's still quick and easy to do.  That's a Gamma only LogC to Rec709 2.4 transform.  Right-click the main image and select Grab a Still.  Give it a label and put it in your  powergrades and you're good to go.  It's non-destructive, doesn't break Resolve's 32bit colorspace, and can be easily adjusted.  I'm working on a more accurate alternative, but for now it'll do.  Combine the powergrade with the Gamut LUT (in serarate nodes).  All good.

Next entry will be about the Gamut LUT.  I mentioned the 3x3 matrix before.  Have you noticed that in the RGB Mixer that there are 3 sections with 3 inputs each?  They may be of some use... 
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on September 03, 2015, 08:11:25 PM
Incredibly sweet stuff, guys!
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 04, 2015, 07:31:37 PM
So basically, if you can get hold of the matrix values for a Gamut conversion you can enter them as value in the RGB Mixer.  It's not a perfect approach because you can't enter the exact values, but you can get pretty close and with some minor tinkering while using your scopes you can narrow the gap.  Because we're working in Alexa Wide Gamut, and Arri are a vey cool company, we have access to those matrices.

You can find them in this document:

http://www.arri.com/?eID=registration&file_uid=8026

And there's further explanation of their colour science in this white paper:

http://www.arri.com/?eID=registration&file_uid=7347

The important ones to note are the tone mapped Alexa Wide Gamut to Rec709 Gamut:

(http://i.imgur.com/sI1qnHQ.png)

Plus the Film Matrix one:

(http://i.imgur.com/q8AIs6h.png)

You can get the Alexa Wide Gamut to Film Matrix in a LUT as well, from their LUT Generator, as an alternative or just a comparison.

In the RGB Mixer you enter the numbers as follows:  (1st Line) Red Output R G B, (2nd Line) Green Output R G B, (3rd Line) Blue Output R G B

Also worth noting is the order of operations in Resolve, and in particular the nodes:

(http://i.imgur.com/9psYnCG.png)

A good practicing test chart would be something like this:

(http://i.imgur.com/EIA0Wa7h.jpg)

You can download a 1920x1080 version here:

https://www.sendspace.com/file/zl5z2a

So that's a few more options covered.  Next I'll be looking at an alternative Gamma curve to better suit ML Raw footage (it's always darker that how it was shot), reverse engineering creative looks, and maybe even a wee surprise in the form of a free present :)
Title: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on September 04, 2015, 09:11:06 PM
Incredible read once again and especially on the Film Matrix stuff. [emoji1]

One question though... If I were to download and print the test chart won't it display proper colors because every printer uses different inks, etc?

Thanks for sharing anyway and I'll try printing one out in large 16x9 paper just for the heck of it at work later! [emoji106]
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 04, 2015, 09:11:59 PM
I just installed the latest Beta of Resolve (number 4) and it would appear that they've tinkered with the colour science again.  The LogC to Rec709 Gamma 2.4 LUTS now make the image too bright.  To get the image you get in non RCM, you go from Arri LogC to Rec709, and not Rec709 Gamma 2.4.  Anyway, I had a quick tinker myself, and with the grey ramp I selected Input Colorspace Rec709, Timeline Colorspace Arri LogC, and Bypass for the Output Colorspace.  With the curves I was able to do a simple reverse engineer, and saved it as a still.  I popped that on a ML Raw clip in Arri LogC, and in a node after that I added the Alexa Wide Gamut to Rec709 Matrix values to its RGB Mixer (via powergrade), and the image matched the non RCM version.  All good.

PM me if you have any queries and would rather not post in the thread.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 04, 2015, 09:20:26 PM
@DeafEyeJedi

The image is for viewing on a monitor, to use with scopes etc.  It's an 8bit HD image, and you'd really want to be looking at something with a higher bit depth and much higher resolution for printing.  Standard printing expects usually 300 pixels per inch, so the chart would be very small.  If you want to use a real life chart I suggest X-Rite Color Passport, or better still something from DSC Labs.

PM me if you have any more queries, and I'll post them on the thread if they're relevant to the main topic.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 06, 2015, 01:59:05 AM
I still have to get round to to exploring reverse engineering techniques, plus adding a few other handy tips, but for the time being you can play around with these creative look LUTs that I put together.  I played around with some of the popular and freely available Print Emulation LUTs (Juan Malera, Resolve Film Looks, etc) in LUTCalc, and put together a nice little package especially for Magic Lantern Raw in Resolve.  I'm calling them the Magic Lantern Fudak LUTs Version 1.

There are 5 looks, with 2 versions of each.  The LogC Alexa LUTs are to be applied to footage in LogC Gamma Alexa Gamut, and the Rec709 Alexa LUTs are to be applied to footage that's already been Gamma corrected but still in Alexa Wide Gamut.  Pretty straight forward.  They're a bit punchy, but they're Version 1 so I figured I'd be a bit indulgent this time round.  More refined versions will probably follow.

https://www.sendspace.com/file/6o4e3i

Bon Appetit  :)



Title: Re: DaVinci Resolve 12 and ML Raw
Post by: reddeercity on September 06, 2015, 05:49:31 AM
Great Stuff @baldavenger  thanks, keep this up you are almost convincing my to try out Resolve for ML,
Actually This is very good information , I'm Editing & Grading a film shot on a BMCC (waiting for the Hard Drives) and I originally was going
to make ProRes4444 XQ and use FCPX etc.... but now I think I will either do a round trip or do the whole film in Reslove.
 :)
 
Title: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on September 06, 2015, 07:50:53 AM
... ProRes4444 XQ and use FCPX etc.... but now I think I will either do a round trip or do the whole film in Reslove.
 :)

Likewise, Mate...

A very well job done to @baldavenger for your continued contributions & please keep 'em coming!

Will test out your LUTS and report back.

[emoji106]
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 06, 2015, 09:40:43 PM
Here's something I came up with just now.  It's a bonus grain tip.

(http://i.imgur.com/PHXWbUT.png)


You can make compound nodes now in Resolve.  Very handy.  In the above example there's a grain scan imported as a matte, added via overlay blend in a layer node, and selectively restricted to preferred areas of the image i.e. shadows and mids rather than highlights, via a luminosity mask pulled in node 1 and piped into node 7.  Select all.  Compound node.  Rename.  Powergrade.  Bosh.  You can go back in for a tinker about as well, if needs be.

Grain is cool.  Don't ever let anyone say otherwise.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DanHaag on September 06, 2015, 09:57:40 PM
I love grain on my master files but have never been pleased with how it turned out after uploading to Vimeo or Youtube. Tried the dedicated x264 settings as well as all kinds of ProRes versions or whatever else was suggested online for web compression. It just never worked for me as soon as I put it on an online platform but in it's original state in my uncompressed master files I wouldn't wanna live without it.  8)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 06, 2015, 11:06:15 PM
Yeah, streaming sites tend to butcher grain and leave awful artifacts.  It increases bitrate requirements considerably, so compression algorithms target it first.  The more aggressive the compression algorithm, the more blotchy and warped the grain becomes.  Best to add it at the end, or as a track node, that way you can easily disable it and render out both a grained and grain free version, as requirements dictate.

When I started tinkering about with VFX I saw grain as the enemy, and valued nothing more than a 'clean' de-grained image.  Not anymore.  Good grain enhances everything.  Perceived sharpness, plus no authentic film look is complete without it.  I was convinced the last season of Mad Men was shot on film, until I was informed it was in fact shot on the Alexa.  More than anything else, it was the skilful addition of grain that fooled me.  Criterion's recent release of remastered films from the last 70's/early 80's such as Thief and Blow Out painstakingly preserve the original grain structure, and they look amazing.  Especially on a big screen where you can really observe it in action.  Long may it continue to be so.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: mothaibaphoto on September 07, 2015, 06:09:13 AM
Just curious: why all these complexities? Just to mimic 12bit logarithmic RAW workflow with 14 bit linear RAW? Yes, no doubt, it's possible :) But I don't see the point.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 07, 2015, 11:18:27 AM
@mothaibaphoto

Everything posted here is optional, with regards to how one might utilise Resolve.  If you prefer to debayer straight into Rec709 Gamma and Gamut then grand, that works too of course.  Some people though prefer the option to debayer into a log gamma and wider gamut, and with Arri LogC you get the advantage of being able to utilise Alexa colour science, etc.  Also, being able to incorporate ML Raw footage into a common colorspace with footage from other cameras could be deemed useful.  As for the 'complexities', although you may have no interest in them, the node structures and instructions on analysing and modifying LUTs may be of interest to some people.

Please PM me if you have any specific queries.  I would prefer to keep validation related discussions from the thread as they are not in keeping with its intended purpose.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: mothaibaphoto on September 07, 2015, 12:12:49 PM
Nowhere in your thread mentioned before that
Everything posted here is optional ...
Any newbie looking info about working with ML RAW in DR will get false impression than all these absolutely mandatory.

...I would prefer to keep validation related discussions from the thread as they are not in keeping with its intended purpose.
- Ok, than you should change your topic theme from broad "DaVinci Resolve 12 and ML Raw" to "DaVinci Resolve 12 and ML Raw for LUT nerds" or "DaVinci Resolve 12 and ML Raw for anyone willing to fool himself he shoot Arri"
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 07, 2015, 12:35:36 PM
You are clearly a wonderfully joyous human being.  Thank you for your marvellous contribution.

Have a good day sir.  Be a decent chap and please refrain from any further posts on this thread.  Perhaps your powers of philanthropic computational assessment and cultural sensitivity may be better suited elsewhere.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: mothaibaphoto on September 07, 2015, 06:33:21 PM
I'm so sorry you have nothing to say on topic, rather than speculating on my humble personality.
But I have:
Would you please be so kind to illustrate this proposition:
Next I'll be looking at an alternative Gamma curve to better suit ML Raw footage (it's always darker that how it was shot)
For me ML Raw footage looks exactly as was shot in DR and the same, say in ACR.
Thank you.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 07, 2015, 07:36:59 PM
I've already asked you twice to stop posting on this thread.  I might have been willing to help, but you were being rude and obnoxious.  I don't know what your problem is Dmitry, but if you are having some sort of break down then I suggest you have it somewhere else.  Let Olga take care of you instead.

PS Seriously dude, no more posts here, and don't bother PMing me either.  I have no interest in any more dealings with you.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 07, 2015, 08:34:00 PM
So yeah, pressing on, time for an update with regards to using an appropriate LogC to Rec709 LUT.  Resolve 12 Beta launched with a Rec709 Gamma 2.4 option only, as opposed to having just Rec709 as well.  Anyway, they corrected that omission in Beta 4.  The following two images display the settings required for a Gamma and Gamut conversion LUT, and a Gamma only one.

(http://i.imgur.com/FBLEYCS.png)


(http://i.imgur.com/wdJj9ea.png)



Try them out on ML Raw footage in Arri LogC colorspace in Resolve.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 07, 2015, 09:28:42 PM
Back in the 7th post I gave an example of how to reverse engineer the LogC to Rec709 tone curve in Resolve.  However, reverse engineering the tone curve of a LUT is not so straight forward, especially if it's a log to lin/rec709 transfer.  There is a way around it though.

Come out of DaVinci YRGB Color Managed, and select the grey ramp.  There should be a straight diagonal in the waveform scope, as to be expected.  Use two nodes in the node graph, and in the second node place the LogC to Rec709 1D LUT.  The waveform will now resemble this:

(http://i.imgur.com/L9UU6HV.png)

Trying to reverse engineer the LUT in a node after it would not work, as LUTs clip information above 1.0 (in most cases) and that information cannot be retrieved.  So instead we do the reversal before.  However, the resulting curve in the node represents a Rec709 to LogC transfer, which is the reverse of what we're after.  The trick is to bring all the values in the custom panel from 100 to 0.  100 to 50 cancels the effect of the curve, and 50 to 0 reverses it.  So we get our LogC to Rec709 after all.

(http://i.imgur.com/7jJDHHI.png)



I included the Lum Mix figure in the bottom left, not because it's necessary for this process, but because it is involved when attempting to reverse engineer colour information.


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 08, 2015, 01:57:40 AM
Here's another bonus node tree I put together not that long ago.  Nothing particularly special, but it might be of use in some situations.


(http://i.imgur.com/3jFSvIl.png)


Ignore the splitter combiner in the middle for the time being, and pay attention to nodes 02 and 04, which are the two components of a layer node.  Node 02 has saturation down to 0 so it's just a b/w image, and in node 04 in the Primary Bars section the Y bar of Gain is turned all the way down (leaving just colour information).  In the actual Layer node Add is selected as the composite mode.  This process separates the chroma and luma into different streams before uniting them again.  Many handy uses for this.  One such use is adding a look LUT to a node before node 04, to see the colour without disturbing the gamma.  You can apply sharpen to node 02 and denoise/blur to node 04, but you can also do that in a single node anyway by selecting LAB space, and deactivating the channels as required.

The splitter combiner was added because someone requested the ability to adjust the luma of individual channels without affecting the overall colour, and this set-up did the trick.

Try it out if you feel so inclined, or not as the case may be.  It's a free world, or at least it is in this thread where the software, LUTs, and suggestions cost nothing.  Keep the spirit of Magic Lantern alive and try to be positive and supportive of one another.  We're all trying to find ways to make more beautiful images, so if we work together and help each other out then surely we will all benefit in the long run and hopefully become better artists.  I like to think so anyway.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: andy kh on September 08, 2015, 02:27:36 PM
thanks for sharing
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 09, 2015, 12:52:46 AM
We'll have a quick look at using curves to reverse engineer colour.  This is quite the most complex element of colour grading, and serious pros take detailed measures (combined with years of experience) to achieve noteworthy results, often utilising a number of specialist charts, lit with a range of different exposures, on calibrated monitors and external scopes.  But we all have to start somewhere, and these following examples might give you an idea of where to begin.


This first image features an untouched set of curves and a waveform scope representing the luma of the three channels.  As you can see just from the scopes the image is very saturated, with a dominant green element.

(http://i.imgur.com/beg1Q8s.png)


Notice how it says 100.00 in the Lum Mix in the lower left hand corner.  That's the luma is fully incorporated into the curves, so if an adjustment is made to one of the colour curves, the other two react as a kind of counter balance.  Turning the value down to 0.00 that interaction is removed, so now you can alter the individual curves independently.


(http://i.imgur.com/xUNm2vF.png)


In the image above, the curves have been adjusted to try to remove colour and saturation.  Not the greatest effort by any means, but you see now by bringing the colour elements together ultimately colour is removed.  To reverse the procedure, drag the Edit values all the way from 100 to 0, and the effect is reversed.  The main image will now be very saturated (as you have in effect added doubled over in colour), but save the node on its own as a still and you can apply and adjust the colour on another image (preferably neutralised first). 

There are other ways to remove and then add colour from an image, such as via the RGB Mixer, though that would take a very experienced individual to make it work.  There's also a mathematical equation also, whereby an image with colour is divided by a desaturated version of itself, and the result is multiplied by the desaturated version of the image you wish to add the colour to.  Anyway, there may be in amongst all that something of possible interest of even use to you.  Proceed with caution  :)


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 09, 2015, 04:04:22 AM
Ok, this one should prove useful for practically anyone who uses DaVinci, or any other colour grading application out there.  I hope those of you who have availed of the ML Fudak LUTs are finding them useful.  This is how I made them.

"Give a man a fish, and you feed him for a day; show him how to catch fish, and you feed him for a lifetime."

(http://i.imgur.com/MH5BQCih.png)

Open up LUTCalc.  There's actually an app you can install, instead of using the website.  Especially handy when there isn't internet access for whatever reason.

https://cameramanben.github.io/LUTCalc/

Click the preview button to reveal the picture above, and above it click the buttons to activate the Waveform and Vectorscope.  Input the values as shown on the left hand side.  It's pretty much the same set-up as when making the optimum ML LogC Alexa to Rec709 Rec709 LUT.  You can adjust the different parameters and see an immediate effect on the image and scopes.

There's an option to import your own image, so we'll try that.  In Resolve, select an image form a clip in the timeline, grab a still, and right click the still in the gallery and choose export image. I would choose tiff for maximum accuracy, but jpeg or png will do also.  You can select the image in Rec709 or LogC, it doesn't matter.  I choose an image that was in LogC, and when importing the image you're asked to pick the Image Gamma, Colorspace, and Legal range.  For a LogC Alexa image I chose the above options.


(http://i.imgur.com/li3h8vwh.png)


Pretty straight forward.  The image does not appear in LogC as it has been converted to Rec709.  The process is the same here as it is nor non-RAW footage in Resolve Colour Management.  It converts the image from the input colorspace to the recorded colorspace and on to the output colorspace.  If I had put in the wrong information when importing the preview image, then it would not correspond correctly to the chosen output setting, just like in RCM.

We're now going to import a look LUT so that we can combine and optionally modify its colour information with the our preferred Gamma.  I find the best LUTs to use for this are standard Rec709, as Log LUTs seem to lack a bit of punch no matter what how I adjust the settings.  I'm still learning though so maybe there's something I'm not quite doing right.  Anyway, click on the white bar and the loading options appear.


(http://i.imgur.com/zCBOysCh.png)


Input the appropriate settings to correspond with the LUT.  As this one was a Rec709 Gamma and Gamut LUT I picked those settings, and they do indeed work best.  Click the analyse button, and a second or two later it will be processed and you will now have the option to select the LUT's Gamma and Gamut separately.  Select the new option in the Output Gamut drop-down, and also for the Highlight Gamut.  The preview image will update and you should see the new colour combined with the original Rec709 Gamma.  You can then make whatever modifications with the available tools (I tend to use the main saturation, and targeted saturation controls as above), with almost instant updates in the preview image and scopes.  Very handy.  When finished simply generate the new LUT as usual.  Those of you who find themselves jumping to and fro from Resolve to test new LUTS all the time will especially find this approach practical.  I certainly do.

So yeah, that's how I made them.  Now you can too.  Big Up once again to Mr Ben Turley for producing such an amazingly useful piece of software.  Well done sir.


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 10, 2015, 05:43:13 AM
Here's a clip featuring some basic approaches that have been explored in previous posts.  Both shots featured only had white balance adjustment performed, so no other camera raw tweaks or colour grading was applied.  The first section features 3 states:

1. The image in regular Davinci YRGB and Rec709
2. The image in Davinci YRGB Color Management and Arri LogC colorspace
3. The image after having a powergrade curve adjustment and RGB Mixer Gamut input to convert from LogC Alexa to Rec709 Gamma and Gamut.  No LUT.

The second section features LogC footage with a powergrade curve adjustment, and a Gamut only Print Emulation Look LUT, in 5 varieties.

Though far from perfect, they represent a much more solid starting point for further grading, thanks to customisation and the more active involvement of powergrades.





Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 14, 2015, 12:33:52 AM
"Pursuing a film aesthetic through a digital medium."

The final missing piece in the roll your own jigsaw is the Look LUT to powergrade conversion.  As stated before on this thread this is by no means an easy task, but after running a few tests I came about something not a million miles away.  The key here is to consider why the LUT does what it does, and what it's trying to achieve.  In the case of the Print Emulation LUTs you might think the answer is obvious, but then why is there still such mystery surrounding them and how they work?  A better understanding might come form paying more attention to the actual nature of celluloid film, and it's relation with light levels and saturation.

“…the most saturated colors that film can reproduce are dark cyans, magentas and yellows, each produced by a maximum density of its respective image dye, but resulting in low luminance levels. In contrast, the most saturated colors on a digital display are the bright red, green and blue primaries, each produced by the maximum output of its additive color channel, and therefore resulting in the maximum luminance for that channel.”

— Kennel, Glenn. Color and Mastering for Digital Cinema. Focal Press, 2007. Page 22, “Unstable Primaries.”

Glenn is now the CEO at ARRI, so you can see how this logic is influencing their cameras.  A very clever chap called Art Adams looked at the luminance and saturation relationship with regards a comparison between ARRI and Sony, and his findings are relevant to Canon too.



http://www.dvinfo.net/article/post/making-the-sony-f55-look-filmic-with-resolve-9.html



He devised a Luma to Saturation curve that helps mimic the Alexa response.  Worth having a look at, even just for curiosity's sake.

(http://i.imgur.com/fYMIecy.png)


So with these things in mind lets have a look at two images.  One regular Rec709, and the other with a 2383 Print LUT applied.

(http://i.imgur.com/8wmkV7Q.png)

(http://i.imgur.com/9viP5wb.png)

Notice not just the hue changes, but also the luma values.  Saturation is invariably maintained but at the expense of brightness, as is the case with actual print film.  It helps to understand the reasoning behind this so as to in effect better emulate an emulation.

The main crux of the reverse engineering can be done in two nodes.  The first node uses the custom curves (with Luma Mix down to 0) to neutralise the main body of the LUT as follows:

(http://i.imgur.com/4qjKedq.png)

(http://i.imgur.com/cQpILzU.png)


It's not exactly pretty, but it's a start, and in the second node the Hue vs Hue and Hue vs Sat curves can be used to tweak the overall effect.  In an optional third node a high saturation only key can be pulled to use the Hue vs Luma curve to darken very saturated parts of the image.  I've found this technique to be quite effective.  I recommend testing on an image such as this one so as better to monitor subtle changes (also via the scopes):

(http://i.imgur.com/dBDDs30h.png)

Once satisfied with the results, simply select the nodes, convert to compound to node, then save as a powergrade.  You should now have the option to use powergrades and/or LUTs however you please.

Although I refer to the ARRI Alexa a lot, don't be mistaken into believing that I am in any way proposing that a Magic Lantern camera (even the 5D Mark III) can somehow compete with the industry standard digital film camera of choice.  All I'm purporting is that by paying attention to how they operate and the logic behind their colour science we can improve our end results, and if some of use move on to better things then at least the transition won't be so daunting.



This brings to an end the main body of what I set out to do, so from here on in feel free to post any questions, or observations, criticisms, etc.  If you have any tips of your own and feel inclined to share then by all means please do.  Let's keep the thread running, and let's try to keep a positive and supportive tone throughout.

Thank you.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 15, 2015, 12:53:45 AM
So I was watching episode 3 of Fear The Walking Dead, and I couldn't help notice now Arri Alexa it looked.  I mean, proper greeny yellowy tinged Film Matrix look.  Very distinctive, and its absence was noticeable when they used a different camera for the crane shots and other shots where they deemed a smaller and more practical camera was needed.  I decided to review the ML Fudak LUTs, and put together a Version 2 pack.  These ones are just for footage already converted to Rec709 Gamma and Gamut, with the option of placing a node with the Film Matrix values before it.  They will appear less punchy (saturated) than the previous version, but simply adding saturation on the same node as the LUT will bring it back to preferable levels and no funny errors should occur in the highlights.  Ideally you would reverse engineer them into powergrades as per previous instructions, but as LUTs go they are pretty safe, provided you put them as the last node.  If the Film matrix workflow results in too much saturation then simply select the node and in the Node Key panel reduce the Key Output until you are satisfied with the results.

Here's the link:

https://www.dropbox.com/s/3isuydphexyc77u/ML%20Fudak%20Version%202.zip?dl=0


Here are some examples of regular Rec709 powergrade, LUT with saturation boost, and Fim Matrix and LUT.  All images had white balance in camera raw, but no other colour grading was performed.

(http://i.imgur.com/rNsrRJml.png)

(http://i.imgur.com/3u7eoQll.jpg)

(http://i.imgur.com/0tO5XMfl.jpg)

(http://i.imgur.com/SNUnEgnl.png)

(http://i.imgur.com/URZkYTnl.jpg)

(http://i.imgur.com/OQzUlBTl.jpg)


(http://i.imgur.com/imdLJPLl.png)

(http://i.imgur.com/yPExGwtl.jpg)

(http://i.imgur.com/lNwQbgjl.jpg)

(http://i.imgur.com/2bf92wol.jpg)

(http://i.imgur.com/EfuUfxWl.jpg)



So yeah, have a go with those and see how you get on.  I believe that they are a far more viable option with regards to a controlled workflow than the previous bunch.  Feel free to post your results and observations.  It would be interesting to hear how people are getting on with them, and the workflow in general.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DanHaag on September 16, 2015, 12:49:07 AM
Thanks so far, your amazing work helps me a lot. I don't really use your whole workflow or the LUTs right now, as I'm going for different looks with my current project. I already use a whole lot of information and individual techniques shown in this thread, though, so it's incredibly useful regardless if you work with LUTs or not.  8)   

BTW: I love the idea of trying to come up with individual LUTs and Power Grades to mimic certain film's or series' looks with ML raw footage! Very interesting, I hope there's more to come and others join the party!
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 16, 2015, 01:52:30 AM
Most professional colourists try to avoid using LUTs due to obvious restrictions to the grading process.  There is, however, one particular occasion when a LUT has to be used, and that is during a conventional DIT process where the footage is graded under a PFE before being sent to a lab to be actually printed to film.  After the original film negative is processed it is scanned to DPX files in Cineon log, and that is what is graded by the colourist, with the PFE at the end.  Once complete the PFE is removed and the footage is exported to the lab, and the final print should match the image in Resolve.

So basically those Kodak and Fuji print LUTs that are already in Resolve (along with various other proper print LUTs like the Juan Malera collection), are designed to be used on scanned film frames that have been converted to Cineon log DPX files.  Cineon log is only a transfer curve, so the process doesn't convert the colour information into another Gamut, i.e. if you were to apply a Cineon to Rec709 LUT to a scanned file the you'd get the original image (more or less).

Let's have a look at some images:

(http://i.imgur.com/NQ3rxAvl.jpg)

(http://i.imgur.com/5XHzY6Dl.jpg)

(http://i.imgur.com/y5oHc8Pl.jpg)

As you can see, the first image is an actual scanned film frame in a Cineon log DPX, the second image has had a PFE applied, and the third is just a regular Alexa Rec709 frame.  We'll try to convert the Rec709 frame so that it matches the film scan.

Although there is the choice to debayer straight into Cineon colorspace in Resolve, in order for the correct process to be applied the footage has to be first in Alexa Wide Gamut, so we stay with the usual Arri LogC colorspace option.  Converting LogC to Cineon is easy.  Use the grey ramp from before, select Cineon as Input Colorspace, Arri LogC as Timeline Colorspace, and Bypass as Output Colorspace.  Then use a simple curve adjustment to straighten the line in the waveform, grab a still, and that's your LogC to Cineon transform.

(http://i.imgur.com/NxIwp8Nl.png)

(http://i.imgur.com/0BpI9C5l.png)

Arri's colour science comes from their experience and knowledge of scanning film.  LogC is an adaption of Cineon, but designed to contain more lattitude, and their Film Matrix is specifically designed to convert LogC Alexa Wide Gamut so that footage matches the colour of film scans.  In theory, once the Film Matrix has been applied to LogC footage, it can be graded along side actual film scans with a PFE output LUT.  Evidence suggests that this system does indeed work.

(http://i.imgur.com/Gwrism2l.png)

(http://i.imgur.com/TrbYh0sl.jpg)

(http://i.imgur.com/zHWoQGll.jpg)

(http://i.imgur.com/nd5As6ul.jpg)

The first image above shows the node graph.  The original image being in Arri LogC Colorspace, the first node converts LogC to Cineon, the second applies the Film Matrix, the third is an optional darkening of the more saturated colours (pull a saturation only qualifier and use Offset to darken), and the last is a PFE, in this case the Kodak Rec709 2383 D65 LUT in the film looks folder in Resolve.

The second image image has had the LogC to Cineon conversion performed, and the Film Matrix applied.  The third image has had the saturation darkening applied (digital images tend to have bright saturation, so this is an adjustment in line with more conventional film response).  The final image has had the PFE applied in the last node, and the results aren't too shabby.  I've tried this method with various images, and the various official PFEs (Juan Malera ones are pretty good too) all work in a pleasantly filmic manner.

I've only just come up with these findings today, so if they seem a tad sketchy then please forgive my clumsiness.  It's pretty exciting stuff nonetheless.  There's a lot bluff out there with regards film emulation, and expensively marketed packages that promise the moon on a stick, but a sneak peek into how the actual pros go about their business will often reveal far more practical information.  So yeah, have a go yourselves, test it out, give us some feedback, get involved  :)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 18, 2015, 09:02:31 PM
I've looked back over what I posted in this thread, and there are some useful things, but overall it's still a bit messy.  One thing that was pointed out on more than one occasion was when I said ML Raw Footage was always darker than how it was shot.  That's wrong, and I apologise for that and also for not accounting for it sooner.  The idea stemmed from my previous (pre Resolve 12) workflow with involved conversion LUTs, and I always had to boost exposure.  This apparently was common practice, and referenced in these posts:

http://www.magiclantern.fm/forum/index.php?topic=10151.msg151811#msg151811

http://www.magiclantern.fm/forum/index.php?topic=10151.msg138372;topicseen#msg138372

http://www.magiclantern.fm/forum/index.php?topic=10151.msg122090;topicseen#msg122090

However, it was an issue with converting from BMD Film to Cineon Alexa Wide Gamut by a LUT, and not directly because of ML Raw.  Apologies once again.

There are other inaccuracies in the thread, such as the suggestions in the opening posts with regards to what LUTs to use, as the release of Beta 4 rendered them no longer optimum, but I did address that in later posts.  I am very much learning as I go along, so please bear with me.

I will, however, stick to my assertion that the RGB Mixer is a 3x3 Matrix, albeit one with limited input controls (I'm hoping Blackmagic Design will address eventually).  Here's a description of the Matrix node in Nuke, which effectively describes what the RGB Mixer does:

(http://i.imgur.com/bG5ssouh.png)

The free version of Nuke can reveal a good number of useful conversion matrices.  A handy discovery.

(http://i.imgur.com/kMP6q9Mh.png)

These matrices are dependant on the footage being in Linear space, but that's how RCM works anyway.  1D shaper LUTs (or ideally a mathematical equation inputted into a node curve) could be used to get to and fro Linear space.

I'm still trying to envisage a LUT free workflow, and I believe it is feasibly within grasp.  Perhaps with better integration with Fusion this will be possible.

Also, in case anyone was actually countenancing the thought that I might at any stage try selling a product based on LUTs (or whatever), then rest assured that this will not be the case.  Not now.  Not ever.  This thread is very much all about the free.

I'll be quiet with regards big entries on this thread for the next fortnight or so as I need to finish off my showreel, but please free free to post any questions or contributions in the way of tips and advice.  Thank you kindly.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 21, 2015, 07:55:32 PM
I posted this on another site just now, so it's basically a copy and paste job, but useful stuff nonetheless.



Before I present it I'd like to have a quick look at what a Matrix actually does.

(http://i.imgur.com/bG5ssouh.png)

So it is just like the RGB Mixer (with preserve luminance unticked). If the first line of the 3X3 Matrix was (x1 y1 z1), the second line (x2 y2 z2), and the bottom line (x3 y3 z3), then to get the new RGB values from a Matrix transform you would perform the following:

(R*x1)+(G*y1)+(B*z1)=R

(R*x2)+(G*y2)+(B*z2)=G

(R*x3)+(G*y3)+(B*z3)=B

So because it is presently not possible to input specific values into the RGB Mixer, then in order to replicate a transform we will need to separate the channels, multiply them by the appropriate Matrix values, and add them back together in the appropriate stream.

Starting with the final (compound node) stage, and working backwards from there we get these:

(http://i.imgur.com/O4p0xm3h.png)

(http://i.imgur.com/U3rqroQh.png)

(http://i.imgur.com/3sOh6fXh.png)

(http://i.imgur.com/N168p1Dh.png)

(http://i.imgur.com/lfnknLsh.png)

(http://i.imgur.com/3yKWpzTh.png)

By limiting node functions to simple addition, subtraction, and multiplication (and Channel Swap in RGB Mixer), and removing the need to engage with Resolves GUI controls, a more mathematically precise approach can be realised.

It can be a hog on resources though, but the compound node can be cached. The Matrix values are entered in the Key Output section of the nodes labelled above. This limits the multiplication factor to between 0 and 1.0, but if a value greater than 1 is required then simply multiply the node by a factor and then divide the Matrix value by that same factor, and the net result will comply with requirements. You can type a very specific value into the Key Output, but it will round that to 3 decimal places. Whether that's an actual rounding up, or the input is preserved and the display is rounded up for appearance sake, I do not know. Perhaps someone at Blackmagic Design can clarify.

To put things into perspective, the ability to perform specific Matrix transformations within Resolve means it has just as many options as Baselight with regards to Colorspaces. Resolve Colour Management is basically a fixed set of transforms, involving a conversion to Scene Linear, a 3X3 Matrix conversion for the Primaries, and a conversion from Scene Linear to which ever tone curve is stipulated. There was a recent case when someone wanted to incorporate RedLog DragonColor2 footage into RCM, but that wasn't an option. DragonColor2 to XYZ Matrix values are available, so using the system above the footage can be converted to whichever Colorspace you prefer (via two Matrix conversions, but these can be concatenated into one Matrix).

http://colour-science.org/api/0.3.5/html/colour.models.dataset.red.html

There is a great source of Matrices to be found in the free version Nuke, covering both a number of Primaries and Illuminants.

(http://i.imgur.com/fOCDtlsh.png)

(http://i.imgur.com/hlXZxcIh.png)

(http://i.imgur.com/NCbROGfh.png)

Here's a link to folder that contains the following:

3X3 Matrix Template Compound.drx
Add Alexa FM.drx
Alexa Wide Gamut(Tone Mapped) to Rec709.drx
Remove Alexa FM.drx

LogC to Linear 1D Shaper LUT
Linear to LogC 1D Shaper LUT
LogC to Rec709 1D ShaperLUT
LogC Alexa Wide Gamut to Rec709 Rec709 3D LUT


https://www.sendspace.com/file/89ktr7


The LUTs were generated with Ben Turley's excellent LUTCalc:

http://cameramanben.github.io/LUTCalc/LUTCalc/index.html

You can download the app for Mac as well.


If someone had a word with Blackmagic Design and persuaded them to allow for more accessible RGB Mixer and Custom Curves Inputs, then all the node havoc above would no longer be necessary. Even just an option in Preferences to activate custom Input would do the trick. It would make a world of difference, and wouldn't require any extra fancy coding.

Please feel free to test and give feedback. Thank you.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 28, 2015, 07:13:46 PM
This is an update on the previous post, again mostly taken from stuff I posted on another site.

It would appear there was a problem with some of the LUTs I had posted, in particular the ones concerning Log to Lin and Lin to Log operations.  It was pointed out that a good deal of information was being clipped following the Lin to Log LUT.  This makes sense, as the LUT specified Min 0 and 1 values.  Performing a LogC to Linear transformation means expanding the compressed transfer curve to its full extent, and this actually ranges from just below zero to over 55 (Cineon goes up to 13.5).  That's a lot of additional stops that would be clipped unless the LUT specified a higher Max Input Value.

(http://i.imgur.com/aCA5DqYh.png)

Thankfully a solution was found in LUTCalc by entering the appropriate values when generating the Lin to LogC 1D LUT.  I have included the new LUTs in the download below.

The last attempt at emulating a 3x3 Matrix in Resolve was highly impractical (though still moderately accurate).  I devised a newer, more realistic model that although may seem a bit weird at first glance actually works quite well.  I like to think so anyway :)

A lot fewer nodes this time.  This is the template:

(http://i.imgur.com/hfH6VYFh.png)

I used the RGB Mixer to swap channels in 3 nodes, so that they contain only Red, Green, or Blue.

(http://i.imgur.com/ReDZGYWh.png)

I then imported 3 EXR(16bit half float) files as External Mattes and multiplied the specific nodes by the appropriate EXR.  The EXRs are constants generated in a VFX application, and allow for plus or minus values in 3 channels up to 6 decimal places, which is the exact level of accuracy stipulated in the Alexa White Papers.  The three nodes are then added together and the result is a perfectly accurate Matrix transformation.  Each EXR has values in the RGB channels that match those in the columns of the Matrix, so the Red node is multiplied by an EXR comprising of values taken from the 1st column, the Green node is multiplied by an EXR comprising of values taken from the 2nd column, and the Blue node is multiplied by an EXR comprising of values taken from the 3rd column.

In the download you'll find a PowerGrade for the above set-up, along with a few folders containing EXRs.  I recommend you put the main Matrix EXR folder on your main drive and select it as a favourite in the Media page.  That way you can easily access the folders within it and import the EXRs into your media pool as External Mattes.  When you right click on the Mattes in the PowerGrade you'll then have the option to allocate the appropriate one.


https://www.sendspace.com/file/d2lwva


Have a go and see what you think.  Feel free to post results and observations.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 30, 2015, 02:38:11 AM
Back in Reply no.15 on the first page I showed how to create a grain node.  I came across some scanned grain footage that used to be freely available from a VFX compositing application's website, but now is no longer listed. I converted it into Prores 422HQ files, which can be imported as External Mattes.

Here's a list of what's in the download package:

(http://i.imgur.com/PzQ1VCZh.png)


https://mega.nz/#!kQJmHC7I!khxcVhnPxU4RFYdqwmTn9_ge-9e6AZzZpIWmeEK8oSI


I do like a bit of grain, so I do.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on October 01, 2015, 08:00:37 PM
 :)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on October 02, 2015, 02:19:02 AM
 :)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: budafilms on November 05, 2015, 07:52:53 AM
Incredible work.
I don't have skills to apply all these information.

I used to work with Resolve, starting to put BMD on DNG files. After I correct the exposition, blacks and whites to 0 and 1000 in the scopes. And before the color, white balance.

But, if understood correct, here are a lot of better workflows.

Can someone explain an easy way to apply the work of baldavenger? Or is not ready for a guide?

Thanks!
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: hjfilmspeed on November 05, 2015, 04:56:55 PM
Oh my word this is such a great thread cant wait to read all of this. Amazing work yall. I did know you could overlay grain in resolve. Do you think cine grain will also work this way in resolve?

I also agree, even though it is raw footage, there are proper ways to get a more accurate starting point and better color as mentioned in the workflows in this thread. This is essential to a raw workflow.
I was doing this:
Default Color options set in resolve menus. Then I choose BMD color and gamma in the clip raw tab. Then I made a LUT which combines BMD film to linear and linear to cineon (both LUTs come stock in resolve). I would then take my new combo LUT and add it as a Input lut in the color menu of resolve.
This gives you a good flat cineon log and it grade okay but i will say i am having some issues with skintones ect.

@baldavenger Is there anyway I could help you develope this color workflow? Or at least help simplify it? Maybe make a tutorial?
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on November 05, 2015, 06:06:53 PM
Having a look back at the grain post, it would appear that the luminance mask method isn't as effective as first thought. It would be better to add a corrector node, link input to source, and output to the key input of the node that features the image combined with grain.

However, I started looking into how to better integrate the grain into the image so there is a more integral relationship, and not just layered on top. I read about the concept of adding grain in Log mode, so looked into it and though a series of improvised attempts came about something that works ok and doesn't involve loads of nodes.

Basically, you add scaled down grain to an image in log space, then convert the image to whichever display referred space it's destined for. The grain has its contrast restored, but more importantly it is distributed in a filmic manner i.e. more intense in the mids to highs, and rolled off in the shadows and upper highlights. All very organic. The contrast control in Resolve is not linear when increasing contrast (values over 1), but is linear when reducing contrast (values between 1 and 0), so that's ideal for scaling down the grain in the grain plate (pivot at 0.5). The same contrast control can be used to increase or reduce grain levels (non-destructively), while the filmic distribution is maintained throughout.

When transforming the image and grain combo from Log to Lin (Gamma 2.2, or whatever target gamma you prefer), make sure it doesn't involve a colour space transform too as that it will adversely affect the hue and saturation of the grain, i.e. only use a Log to Lin 1D LUT.

(http://i.imgur.com/NM0Ot36h.png)


I'm balls deep in calibration research at the moment so unfortunately haven't the time right now to make a tutorial, but by all means if anyone wishes to add in the thread in any way then please do. There's plenty of resources online for people to learn Resolve. Add links if you like, plus any interpretations you might have.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: hjfilmspeed on November 05, 2015, 06:28:59 PM
^^^^ This is brilliant! The grain would already be there if you were grading actual film so this makes way more sense. this is awesome.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: hjfilmspeed on November 06, 2015, 08:01:10 AM
I was able to get this grain method to work with cine grain. Over laying grain to your log footie, then grading is the way to go. Thank you for this tip.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: QuickHitRecord on November 08, 2015, 07:44:42 AM
I'm just looking for a good starting point for the grade that conforms to a common LOG color space and I think I'm getting better results with the Resolve 12 workflow in baldavenger's first post. There seems to be more color information in the skintones to work with than previously available options.

Oddly, I've never gotten FilmConvert's Log C settings to give me anything usable in that regard. The gamma always seems way off and correcting it introduces an unusable amount of noise. FilmConvert kind of presents itself as a library of LUTs, but maybe it's not meant to be used that way. But then why include LOG profiles at all? Anyone have experience with this?

Take a look:

(http://i1243.photobucket.com/albums/gg541/QuickHitRecord/01_ARRI_LOGC_zpsikfmnzeq.png)
ARRI Log C (Davinci YRGB Color Managed)

(http://i1243.photobucket.com/albums/gg541/QuickHitRecord/02_LOGC_TO_709_LUTCALC_zpspaauxjcg.png)
ARRI Log C (Davinci YRGB Color Managed) to Rec 709 (LutCalc settings from post #1)

(http://i1243.photobucket.com/albums/gg541/QuickHitRecord/03_LOGC_TO_709_BMD_zpsapxwwzoy.png)
ARRI Log C (Davinci YRGB Color Managed) to Rec 709 (BMD-supplied LUT)

(http://i1243.photobucket.com/albums/gg541/QuickHitRecord/04_LOGC_TO_FC_zpsx9ycdusw.png)
ARRI Log C (Davinci YRGB Color Managed) to FilmConvert (Log C to KD 5207 Vis3)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 01, 2015, 12:55:59 AM
I did a revision to the grain package I posted before. This time it's a direct conversion from the source, plus by flip flopping the plates there is now 4 original grain images for every 1 source image. They work great with the composite technique I posted recently.

I also managed to build 14bit 1D shaper LUTs. The VFX LUTs in Resolve are 12bit (4096) and normally that's plenty to prevent quantisation errors, but conversions involving Cineon Log and in particular LogC require a bit more precision in certain circumstances. I'll post something about Resolve LUTs (1D and 3D) soon, covering how to read them and how they work. It's technical stuff but worth knowing nonetheless.

https://www.dropbox.com/s/lpo6lnes2uzq13h/Grain%20Scans%20HD%20Flip%20Flopped.zip?dl=0

https://www.dropbox.com/s/a85ijc0bj46gh5o/14bit%20LUTs.zip?dl=0



Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on December 01, 2015, 03:57:08 AM
Excellent progress so far as usual, @baldavenger!

Just because I've been silent (and quite busy) but never could refrain myself from reading your posts in this incredible thread!

Because now I can run DR12 without any problems on my new 2012 MBP with a decent graphic card and massive spaces.

Keep 'em coming and once my work schedule has settled down ... will finally get in touch with you about this DR12 workflow.

Thanks again, B!
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: hjfilmspeed on December 01, 2015, 01:36:25 PM
Downloading LUT! I will try asap !
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 02, 2015, 03:30:36 AM
I tweaked the method of compiling 1D LUTs. This is how you do it.

The free/learning version of Nuke does the job, so get hold of any version from the last few years. Only one node is required, so you don't have to get bogged down with Nuke's intimidating infrastructure. Once launched, select a ColorLookup node.

(http://i.imgur.com/u96lVABh.png)

Select Red, Green, and Blue from the list on the left.

(http://i.imgur.com/mJh97byh.png)

Right-click in the middle, and select Edit > Edit Expression

(http://i.imgur.com/juZYOfWh.png)

The following little table pops up.

(http://i.imgur.com/mtJcQCKh.png)

Paste an expression into each channel space. Here I pasted the Cineon Log to Linear expression, the result on the right representing the output value when the input value is 1.

(http://i.imgur.com/MzG56dzh.png)

This is the resulting curve.

(http://i.imgur.com/ooscnKCh.png)

Next, right-click the middle and select File > Export Ascii

(http://i.imgur.com/kdyjwTJh.png)

Another little table pops up. The default increment will be 1/255, but that's only 8bit, so to get 14bit enter 16383 (because 0 is also counted as a value, so 16384 is still achieved)

(http://i.imgur.com/w0FaxF1h.png)

When exporting values, the very last one is omitted for some reason, so change End at: to 1.00001 and the last value gets included.

(http://i.imgur.com/NwaT0NGh.png)

This is the first set of values of the exported list.  Note the first value.

(http://i.imgur.com/hIy178eh.png)

This is the last set of values.  Note the final value, and also note that the list goes right up to 16384. Full 14bit.

(http://i.imgur.com/jeh29Quh.png)

Press the node's reset button, then repeat the initial procedure, this time entering the inverse expression (Linear to Cineon Log).

(http://i.imgur.com/fNyNpnqh.png)

A little extra work is required for the next step. The Start at and End at values correspond with the first and last values from the previous exported list. The increment value of 1 is replaced by the full extent of the previous export expansion. The 16383 value from before is repeated.

(http://i.imgur.com/a5PgcvTh.png)

As was the case before, the End at value needs to be extended by a token amount to include the final value. Here I rounded up to 4 decimal places.

(http://i.imgur.com/TVrfW3ph.png)

Here's the first set of values from the exported list. Note how the 1st value isn't quite 0, but the difference is minuscule and that value can be changed to 0.0000000000 in a text editor and saved as such.

(http://i.imgur.com/3brxM9Sh.png)

Here are the last few values. Pretty spot on, without any need to clamp values.

(http://i.imgur.com/moODEwbh.png)

To convert the lists into 1D LUTs that Resolve will read, simply copy and paste the headers from the appropriate 1D LUT in their VFX IO folder. The only things changed (when required) are the Size and Range values.

(http://i.imgur.com/SgZ9sPK.png)

Here (for the Cineon Log to Linear LUT) the size was changed to 16384, while the range values remain the same (0,1).

(http://i.imgur.com/mvdanTq.png)

Save the LUT as a .cube file.

(http://i.imgur.com/BJkSvYp.png)

For the Linear to Cineon Log LUT, the Size becomes 16384 as before, but the Range becomes the first and last values from the preceding LUT.

(http://i.imgur.com/4bzBKZt.png)

Save as a .cube file, and you have a matching pair of 14bit Cineon LUTs. The process can be repeated with any of the expressions for the other transfer functions. You can find the expressions for the various Log to Linear transfers by pressing S in the Properties pane, selecting one of the gammas in the LUT section, right-clicking, and selecting Edit>Edit Expression.

(http://i.imgur.com/Zw6wlvX.png)

Here's a list of the more popular expressions.


Cineon Log to Linear:

(pow(10,(1023*x-685)/300)-.0108)/(1-.0108)

Linear to Cineon Log:

(((log10(x*(1-.0108)+.0108))*300)+685)/1023

Arri LogC to Linear:

x > 0.1496582 ? (pow(10.0, (x - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (x - 0.092809) / 5.367655

Linear to Arri LogC:

x > 0.010591 ? 0.247190 * log10(5.555556 * x + 0.052272) + 0.385537 : 5.367655 * x + 0.092809

sRGB to Linear:

x < .04045 ? x/12.92 : pow((x+.055)/1.055, 2.4)

Linear to sRGB:

x < .0031308 ? x*12.92 : (1.055*pow(x,1/2.4)) -0.055

rec709 to Linear:

x < .081 ? x/4.5 : pow((x+.099)/1.099, 1/.45)

Linear to rec709:

x <= 0.018 ? (x * 4.5) : 1.099 * pow( x, (0.45) ) - 0.099

Gamma 2.4 to Linear:

pow(x,2.4)

Linear to Gamma 2.4:

pow(x,1/2.4)



I'll follow this up with a look at how to read both 1D and 3D LUTs, so you can tell what they're about just by looking at them in a text editor.


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 04, 2015, 05:11:20 AM
I put together an Excel spreadsheet that compiles 14bit Cineon and Arri LogC LUTs. It can be opened in Google Sheets, and also adapted for other transfer curves and bit depths.

https://www.dropbox.com/s/yw69icg9wunj625/Log2Lin2Log%2014bit.zip?dl=0

It was a useful exercise to learn how to compile a LUT at the most basic level.  Its utility can only be further enhanced by demystifying what it is and what it can do. It's a look up table at the end of the day, with a list of input values and corresponding output values. No magic shazam voodoo. Very useful when properly used though.

Since this is a DaVinci Resolve related thread we'll have a look at the 1D and 3D LUTs that are specific to that application, but in doing so the basics of all variety of LUT will be covered. I spent a good part of this year trying to get my head around how to properly interpret the values presented when viewing the LUT in a text editor. I scoured the internet, but the necessary specifics were hard to come by. Anyway, eventually things started to click, first with 1D LUT structure, and then finally 3D.

Starting with 1D LUT structure, open up a null/unity 1D Resolve LUT in a text editor that has an accompanying numerical list.

(http://i.imgur.com/L0yzcMYh.png)

This LUT has output values identical to its input values.  Some LUTs display the input values, but Resolve LUTs do not. It calculates the input values from the header. The Range (by default 0 to 1 in floating point values) is divided equally by the Size number of points (in this case 1024), and the three columns in the LUT represent the corresponding value for the Red, Green, and Blue channel. Corresponding points can be identified by using the numerical list.

The default Range of 0 to 1 is standard for all LUTs, and is the reason for clipping when it occurs. A LUT will only interpret values within that range (anything below 0 becomes 0 and anything above 1 becomes 1), but has no limit to the range of what it can output. If a Log to Lin transform is applied to a signal, the compressed Log (with values contained within the range of 0 and 1) can be expanded to values as high as 13.5 (Cineon) or even 55.1 (Arri LogC), and a LUT applied to afterwards will clip all values over 1. Everything over 1 will be superwhites, but it's still relevant information.  Resolve LUTs can stipulate the Input Range, so the extra information can be brought back into the image stream. No more clipping if you know what kind of signal to expect. An issue that can arise from this though is quantisation. An Arri LogC to Linear LUT expands the compressed log signal from 0 to 1 range to -0.017 to 55.1, and the reverse LUT divides around 55 by its Size as opposed to just 1. The number of points allocated to remapping 0 to 90% scene reflectance (100 IRE) is only about 74 even with a 12bit (4096) LUT. Because Resolve has to interpolate values in between the points, the fewer there are the more likely there will be inaccuracies incurred. The 1D Shaper LUTs in Resolve only go up to 12bit, but the 14bit LUTs here work and can make a big difference. 4 times as many points, which is especially useful in the LogC scenario.

3D LUTS took a while longer to figure out, but the same approach to identifying the input values apply.  Open a 3D null/unity LUT in the test editor.

(http://i.imgur.com/uPGKSfTh.png)

Whereas 1D LUTs can only affect channels separately from one another, 3D LUTs are more convoluted. The maximum number of output values a 1D LUT can produce is the LUT size e.g. 1048 (10bit) times the number of channels, so typically 3 X the number so 3144 in this case. 3D LUTS can produce far more values, as the LUT size is multiplied by itself the number of channels there are, so typically it is cubed.  10bit cubed is 1048 X 1048 X 1048, so 1,151,022,592 possible output values. In order to make the LUT smaller and more functional, the LUT size is greatly reduced, but even a bare minimum 17 X 17 X 17 LUT produces 4913 output values. The null LUT reveals how the reduced size is managed, and how the input values are ordered. They are ordered in cycles of 17 points, with the left column (red channel) consistently ordered by 17 increments from 0 to 1, with the other two columns (green and blue) holding the same incremental value per cycle, until every 3 value permutation is accounted for. There're not as easy (i.e. linear) to read as 1D LUTs, but having a null LUT to compare helps.

With regards to Magic Lantern Raw files, the two main log considerations are Cineon and Arri LogC. Log can definitely be useful sometimes. It's purpose is to effectively squeeze a higher dynamic range signal into a 0 to 1 range. The 5D MkIII can achieve up to 11 stops DN without Dual-ISO, so Cineon is best suited to squeeze the signal into a display referred scene.  You can of course use Raw controls to pull the signal into place, but what you're doing there is basically imitating the log encode process, but with more contrast.  Arri LogC involves even more signal compression, and is probably overkill for regular footage, but actually works great with Dual-ISO especially when pushing for up to 14 stops of DN.  I did some tests in Brazil earlier in the year where I pushed the camera to see what I could get. I shot some sun rises by the sea, so had 100/1600 ISO going. Then the footage was processed and brought into Resolve, and only LogC was able to contain the signal without pulling down highlights in Raw controls.  I was able to see the whole signal on the monitor and the scopes straight away, so I knew what I had to work with. Anyway, it's all about the right tool for the job.

I had to do some YUV related research recently, and besides attaining a body of knowledge that perfectly equips me to get a job as a TV engineer in 1986, I learned a few new possibly handy creative grading tricks. I'll post something related to that in the next few days.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: hjfilmspeed on December 05, 2015, 02:08:46 AM
Love this thread!
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 14, 2015, 03:09:48 AM
I stuck with the spreadsheet learning thing a bit longer than planned, but it's proven to be worthwhile. I built a new one (posted below) that deals specifically with Log C, providing the means to build 14bit 1D and 65x65x65 3D LUTS. The main issue with Log C is that when it's linearised the signal has a huge range, and that doesn't work well with LUTs. I've tried various methods of managing the signal without incurring errors such as normalising the linear signal, but that effectively scales 18% reflectance to 0.2 divided by @55.1, which presents another challenge altogether.

Anyway, one of the better methods I discovered was splitting the signal. It involves two layer nodes after Log C to Lin is performed, with a 3D LUT in each.  One LUT deals with values up to 1 and clips all above that, while the other deals with values greater 1 and clips all below it. The two separate signals can then be added back together.

(http://i.imgur.com/i4X0V2Ah.png)

This method has a lot of potential, and works for any high dynamic range signal and not just Arri.

https://www.dropbox.com/s/1tiiue3n1pbzj73/Arri%20LUTs%20Spreadsheet.xlsx?dl=0


I've been meaning to post something about dealing with FilmConvert and Koji, with respects to using their Arri Log C functions. I was working on converting them earlier in the summer, but without much joy. I believe it's an issue with latitude. If you try grading some actual Arri footage in Resolve you'll find that you can push it around quite vigorously without it falling apart. It means that in order to affect contrast you need quite a strong S curve, and that's what the Koji LUTs and FilmConvert do. You can still achieve the same results though, just not via the direct route. It's very tempting to always go for the Arri named option, as it seems classy and you'd assume it's top quality, but it's not always the most sensible choice. It's a mistake I've made a lot.

You can try for yourself with some footage from here:

https://www.arri.com/EN/camera/alexa/learn/alexa_sample_footage/


If anyone has any questions or would like to comment then please do. I still haven't gotten round to looking at the YUV option, but hope to soon.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on December 14, 2015, 09:52:14 PM
Hi!
Trying some of this split signal workflow and I get somethin like this.

(http://s15.postimg.org/x5dk94ki3/Screen_Shot_2015_12_14_at_21_49_12.png)

(http://s15.postimg.org/9sfiqm4ej/Screen_Shot_2015_12_14_at_21_49_48.png)

According to instructions
Quote
selecting Arri Log C for Timeline and Bypass for Output in project settings so that you get a LogC signal and Alexa Wide Gamut colour space.
Not sure how to bypass for Output project. I apply Arri_Lin_to_LogC_1D_14bit.cube in the timeline.

Quote
In the first node apply the LogC_to_Arri_Lin_1D_14bit LUT, which decodes the LogC signal into its full range of -0.017 to 55.08. Add a Serial node, then add a Layer node.
Done

Quote
In the top Layer (underlying) node add the Arri_Lin_Split_Normal_AWG_to_Rec709_3D_65 LUT. This LUT converts the signal colour space to Rec709, and clips all values over 1.

done

Quote
In the bottom Layer (overlying) node add the Arri_Lin_Split_Superwhites_AWG_to_Rec709_3D_65 LUT.  This also converts the signal colour space to Rec709, but clips all values under 1 and offsets every output value by minus 1. In the composite node select Add, and the signal is complete again.

After this all turns very bright, hmm, I,m not getting this right.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 14, 2015, 10:45:51 PM
Are you using resolve colour management in project settings?

Here are the LUTs if others want a go:

https://www.dropbox.com/s/1vpyltthhc18h60/Arri%20LUTs.zip?dl=0
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on December 14, 2015, 10:47:01 PM
aha, that most be the culprit. I,m still on DR version 11. Time to update stuff.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 14, 2015, 10:52:18 PM
The signal splitting can still be achieved, but I do recommend the update. The layer mixer nodes should be applied to a linear signal with full superblacks and superwhites.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on December 15, 2015, 08:17:16 AM
Definitely worth the update to DR12, @Danne. Heck you even got me into it, right? You know you did, Mate! lol

Thanks for sharing @baldavenger as always and will continue to follow this thread even tho I'm still kind stranded with work.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on December 16, 2015, 12:07:05 AM
Here is a file (dual_iso) that I used in my previous post.
https://drive.google.com/file/d/0B4tCJMlOYfirYlhyTFBCMnBCMjQ/view?usp=sharing

Havn,t upgraded to DR 12 yet. I tried out using Baldands Arri-luts in MLP by converting them to 3D luts in DR 11. Looks really good in MLP but I,m sure there are hickups using 1D logs as 3D luts. Nevertheless.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 16, 2015, 02:55:28 AM
That Dual-ISO image is a great example of both the ability of ML Raw and the difficulty of fully availing of all the extra information on offer. I couldn't find the actual ISO values from exiftool, but I assume given the circumstances it was probably 100/1600. I brought it into Resolve 12 and picked Arri LogC for the Timeline colour space. The image appeared dark as it visually more represented the 100 ISO setting, so in Camera Raw settings I increased exposure to 4.0 (thereby scaling the signal by 4 stops, i.e. to 1600 ISO).

I left dealing with the Alexa Wide Gamut to Rec709 colour space transform until later, as first it was important to sort out the transfer curve. I built a LogC to sRGB 1D LUT with a softclip on both the superblacks and superwhites, but even though the superwhite softclip mapped the 55.08 to around 6 via a 2.4 gamma, it still proved difficult to pull the superwhites into range (Resolve's softclip function had little effect).

So I built a new LUT with a gamma 10 for the superwhites instead, resulting in a much stronger curve and a peak at around 1.5.  This allowed for Resolve's softclip function to pull the extreme highlights into range.

This is the expression I applied to the post LogC to Lin values in order to build the LUT:

=IF(D1<0,POWER(ABS(D1),2.4)*-1,IF(D1>1,POWER(D1,1/10),IF(D1<0.0031308,D1*12.92,(1.055*POWER(D1,1/2.4))-0.055)))

I've included the original sRGB LUT and the Gamma 10 version, plus two gamut 3D LUTs. Although not ideal as they're for log footage and not linear, they should be applied to a node before applying the LogC to sRGB 1D LUT. One is Rec709, while the other is LC709 which is a more muted version and believed to be more akin to the Alexa saturation response.

https://www.dropbox.com/s/mosgqwhos4rvp1h/Kitchen%20Dual-ISO%20LUTs.zip?dl=0

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 17, 2015, 01:23:09 AM
About a month ago and with the help of Andy600 I compiled a set of dark frames for my 5D MkIII. Hopefully a more expansive set will be compiled in the not too distant future that includes most, if not all, ML capable cameras.

https://www.dropbox.com/s/998ck7qpbs9fmi4/5D3%201920x1080%2024FPS%20Dark%20Frames.zip?dl=0

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on December 18, 2015, 01:01:36 PM
I,ve been working on getting some good looks and luts with MLP found here
http://www.magiclantern.fm/forum/index.php?topic=13512.msg130562#msg130562

Thanks to this thread I started experimenting with Ben Turleys online lut calculator, with various results. I tried to create a log and a REC709 to go with that. I am not completetly satisfied with the results. It is rather hard to obtain luts that will bring out most information but also be correct in brightness etc. footage sometimes gets noisy or too dark for example.
In my testing frenzy I noticed I could get really nice files by adding the Linear_to_Rec.709.cube form VFX folder in Davinci resolve straight to the footage output in MLP.
It would be useful with feedback from the standpoint of MLP. MLP works by letting dcraw pipe dng files through FFmpeg which creates prores4444 10-bit files. My conclusion is that dcraw is to be set to -H 2(highlight mode) and gamma to 2.4 0 which brings the output towards linear and brings out the shadows. Through FFmpeg I can add 3D luts to create looks(unfortunately 1D looks can,t be used).
My intent is to be able to create log and rec709 and to understand what kind of color space to use. In this case they have to be 3D luts. I think the online lut calculator could do the trick? My thinking is that I use the png files created with the setting dcraw -H2 -gamma_2.4_0(2nd picture on the test charts) and bring that loggish picture to the online lut calculator and go from there. Or is it better to leave the gamma and work with the dcraw -H2 png files instead which would be more REC709/linear kind of output? Could use some feedback.
Below are examples of what can be done in MLP. The three pictures has the following settings:

1 - Regular gamma and dcraw highlight mode set to -H 2(lowest setting but lowers brightness considerably)
2 - Gamma pushed to 2.4 0 dcraw highlight mode -H 2. This is the base output I intend to start with in the lut calculator
3 - Regular gamma, dcraw -H 2 and a Linear_to_Rec.709.cube coming from the VFX IO folder in DaVinci resolve converted to a 3D lut.

I,d really like to get some tip here to build some optimized log luts and some REC709 and whatevere else could be used here.

All files and original dng files can be downloaded here
https://drive.google.com/file/d/0B4tCJMlOYfirdTFDQllYdkpqc1U/view?usp=sharing


(http://s10.postimg.org/gln1mjd0p/Screen_Shot_2015_12_18_at_12_15_54.png)

(http://s10.postimg.org/t1jrga6cp/Screen_Shot_2015_12_18_at_12_17_35.png)

(http://s10.postimg.org/3lfw3ufu1/Screen_Shot_2015_12_18_at_12_18_40.png)

(http://s10.postimg.org/hzeohuca1/Screen_Shot_2015_12_18_at_12_21_39.png)

(http://s10.postimg.org/mp97k6so9/Screen_Shot_2015_12_18_at_12_24_15.png)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 18, 2015, 08:06:51 PM
When you say regular gamma, do you mean gamma 1.0 (linear)? And what does dcraw -H2 actually do to the signal? Does it scale it down in a linear way, or apply a softclip to pull the highlights down without affecting the midtones and shadows?

Any gamma that isn't linear (1.0) is technically log, so if dcraw can apply a 2.4 gamma expression, i.e. pow(x,1/2.4), to the linear signal then it should be able to apply a cineon or logC expression. Any values outside the range 0.0 to 1.0 are probably clipped when a LUT is applied, so best to have the whole signal contained before that happens.  A 2.4 gamma can give the impression of a flat, log image, but it doesn't remap superblacks and superwhites.

The VFX IO Linear to Rec709 1D LUT is meant for a limited signal (0.0 to 1.0) with gamma 1.0, and the output values are contained to the same limited signal. It doesn't account for superblacks or superwhites. Pulling the highlights down to squeeze them into the limited signal prevents clipping, but the LUT expects the value 1.0 to be 90% reflectance, so although the image might look ok but it is mathematically unbalanced.

If you can devise a pipeline where you have initial access to full scene linear data (16384 14bit?), you can encode the values so they fit into the 0.0 to 1.0 range, and then apply a 3D LUT for final output.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on December 18, 2015, 08:35:35 PM
Actually dcraw pipes out 16-bit files to FFmpeg.

Here are two more comparisons. Settings, only Linear in dcraw and the other Linear and H2(rebuilds highlights and darken the overall picture)

I really like the rebuild highlight setting so could this be kept?

Full png files here
https://drive.google.com/file/d/0B4tCJMlOYfirWmZCa011VU43c1E/view?usp=sharing

(http://s27.postimg.org/ssw0nf0tf/Screen_Shot_2015_12_18_at_20_24_48.png)

(http://s27.postimg.org/eaytff9ib/Screen_Shot_2015_12_18_at_20_26_26.png)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 18, 2015, 08:46:05 PM
I'm doing a quick read over dcraw commands to better understand things. The H2 command seemingly softclips superwhites, but in this case I would recommend it. Any Linear to sRGB/Rec709/etc. is easy to achieve then.  Convert the VXO IO LUTs to 3D LUTs, or build them with a spreadsheet.

It might be possible to devise a log command (looking into it now), along with a custom colorspace such as Alexa Wide Gamut.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on December 18, 2015, 09:06:27 PM
Sounds great. Exciting stuff. I actually converted the vfx io already (tip from Andy600) and when I use linear dcraw setting they are way dark but without linear and with H2 results looks good. I have a hard time to find sweet spot.
I ll check back in a couple of hours. Busy atm.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 18, 2015, 10:27:07 PM
Though still possible, perhaps encoding to a log signal and wider gamut is overkill for what you want. If what's required is a ready to go output and an interim format, then it might be best to combine both into one practical format.

You can use dcraw to get sRGB and Rec709 gamma without using a LUT, and that would be more precise than using a 3D LUT.
I found this while searching the net:

g power toe_slope

Set the gamma curve, by default BT.709 (-g 2.222 4.5). If you prefer sRGB gamma, use -g 2.4 12.92. For a simple power curve, set the toe slope to zero.

With the -H 2 command you can softclip superwhites, so that provides for a decent output format. It also makes for a good interim format too, as any 3D LUT applied doesn't have to deal with gamma transform and can focus just on colour.  This is where LUTCalc comes in very handy. Use sRGB/Rec709 for Input Gamut and Gamma and whatever Output you're after, including Film Emulation Looks. How to put together such LUTs is dealt with earlier in the thread.

Linear will always look dark, because it is how the camera sees light and how light is in the real world. Gamma correction such as sRGB, Rec709, 2.2,2.4, etc., is a means of making a signal more 'realistic' to human perception, though what appears to be real to us is in fact somewhat of an illusion.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on December 18, 2015, 11:56:43 PM
Thanks for info. It would be interesting to have the knowledge to do both.
A while ago I came to the conclusion to use the sRGB gamma -g 2.4 0 and go from there. I then created a log Alexa log lut
Code: [Select]
TITLE "Custom LUT"
LUT_3D_SIZE 33
# Rec709 - γ1.90 (exp2.22)/Alexa Wide Gamut -> LogC (Sup 2.x)/Alexa Wide Gamut, CineEI Shift 0.00, Black Level 8.03% IRE, Data Input -> Data Output - Created with LUTCalc v2.3.4 by Ben Turley November 2015


Canon log lut
Code: [Select]
TITLE "Custom LUT"
LUT_3D_SIZE 33
# Rec709 - γ1.90 (exp2.22)/Canon Cinema Gamut -> Canon C-Log2/Canon Cinema Gamut, CineEI Shift 0.00, Black Level 3.54% IRE, Data Input -> Data Output - Created with LUTCalc v2.3.4 by Ben Turley November 2015

Now I had to push the slope cause it came out to dark. I then created a complimentary REC709 to these logs. Now the question is if it,s better to skip the log and go right to the REC709?

Would Linear to REC709 be something to work with in lut calculator?
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 19, 2015, 02:17:26 AM
I'd better post this and get it out of the way, and then I'll return to the MLP related stuff.

I did some tests with FilmConvert and Koji to see what might be the best approach with regards ML Raw. As was mentioned on here and in other threads, the Arri options are too aggressive with ML Raw footage, even when converted to LogC.

This is the FilmConvert Arri LogC curve:

(http://i.imgur.com/xbiff3B.png)

And the Koji one:

(http://i.imgur.com/T6onhJd.png)

The download below includes this list:

(http://i.imgur.com/5hlzYijh.png)

Of course, you can still get great images even if the LUT or setting doesn't exactly match the preferred parameters, but I find that if you can get the gamma aspect close from the get go then you're likely to have more control over the creative process. With FilmConvert I found that Resolve ML RAW BMD Film works best with log footage, whether its Cineon, LogC, or BMDFilm. It would appear that it expects a Rec709 gamut, and not the wide gamut of BMDFilm, but I got that by applying it to Alexa Wide Gamut footage and the image remained slightly desaturated. When I converted the footage to Rec709 gamut the results were more in line with what was to be expected. ML RAW BMD Film M1 is the same except for 1 stop exposure adjustment, i.e. if your footage was 1 stop underexposed you would add this instead. If you're caught between the two, pick the first one and use the Exposure control in Camera Raw to adjust. ML RAW BMD Film with Exposure turned to 1.0 = ML RAW BMD Film M1. In fact, unless the footage was very underexposed and needed all the help it could get, I would always go for regular ML RAW BMD Film and use only Exposure and White Balance to set the base correction in Camera Raw, and anything else would be done in the nodes.

With Koji LUTs, after unsuccessfully trying to convert the Arri ones to a Cineon equivalent in the past, I reckon only the Cineon to Cineon, or Rec709 to Rec709 are of any practical use. But that's just my opinion. When it comes to LUTs, it is better to err on the side of too little contrast, as that can be added afterwards, whereas as too much initial contrast presents a battle which is almost impossible to win. Think of them like shoes. Ideally you want one that fits perfectly straight away, but if it's too small then you're killing your foot trying to squeeze into it, whereas if the shoe is too big then at least you can wear more socks. Capiche?

Some other observations:

BMD Film gamut is different in regular DaVinci YRGB than it is in RCM. In regular DaVinci YRGB skin tones are richer and there's generally more saturation.

FilmConvert outputs a legal signal, that is to say a full range image with values from 0 IRE to 100 IRE will shrink somewhat after the plugin is applied.

Lift Gamma Gain controls operate AFTER the effect of the plugin.

If working with LogC Alexa Wide Gamut footage (or Cineon Alexa Wide Gamut footage) it is better to convert the gamut with a LUT before applying the plugin. Standard would be Rec709, but try LC709 instead as it results is less saturated highlights. Both LUTs are available in a previous post.

I'm sure there's plenty of diverse views on this subject matter, so please feel free to post what you think.

https://www.dropbox.com/s/2y6hqef4umhxbl1/Koji_FilmConvert_Curves.zip?dl=0

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 19, 2015, 02:36:30 AM
Anyway, back to the MLP post. If you want to export footage in log by applying a 3D LUT at the last stage, then it would be better if it is applied to a linear signal. 33 points is not much and converting Rec709 2.2 to LogC involves a lot, and in theory would have to go via linear anyway. Better to give it linear and reduce the chances of error. It all depends what you want to export.

Both examples are a bit off as you're applying a Rec709 2.2 to LogC LUT to a Gamma 2.4 signal, so you won't technically get LogC. Bringing up the black level also means it's not technically LogC. LogC appears darker than other logs because it squeezes more a signal into a range of 0.0 to 1.0, so 90% reflectance is mapped lower, as is 18% reflectance, to fit all the signal in.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 25, 2015, 12:21:42 AM
The joy of YUV. This is very much an experimental/creative approach, but no harm in giving it a try. To keep things simpler we'll work in Cineon and Rec709 gamut. The examples are an sRGB chart and a ML Raw DNG, so in settings pick sRGB for Input Colorspace, Cineon Log for Timeline Colorspace, and Bypass for Output Colorspace. I've included 14bit Cineon LUTs (sRgb,Rec709,2.4) in the download, and used the Cineon to sRGB LUT in the examples. Here are the two images with just the LUT applied.

(http://i.imgur.com/m1AllAph.png)

(http://i.imgur.com/qptzVELh.png)

The first YUV approach involves changing the colorspace of the node (right-click it) to YUV and deselect the 2nd and 3rd channel. The image will retain its luminance but will appear desaturated. To restore saturation add a node and use the saturation control. The colour returns, though in a somewhat different manner.

(http://i.imgur.com/kkcXSsNh.png)

(http://i.imgur.com/3KfULJ9h.png)

Final approach is a repeat of the last one, only this time in the second node saturation is adjusted using YUV controls. Change the node colorspace to YUV and deselect the 1st channel. Then use the main curves interface to add saturation back. The nature of the colour is similar to before, but the curves allow more diverse control. More intuitive. LAB controls can be used to adjust saturation, but require very particular adjustments, whereas the YUV controls are more flexible. Also, the response tends to bias the mid-tones which is a more filmic approach.

(http://i.imgur.com/YIRIYXjh.png)

(http://i.imgur.com/VOL1LMnh.png)


https://www.dropbox.com/s/st3kaadh68wwl1i/YUV%20shenanigans.zip?dl=0


More options. Cool beans. Happy Festivus  :D

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 27, 2015, 07:56:29 PM
RCM is great and all, but what if you could build a custom emulation with 1D LUTs? That would mean more control, plus there would then be the option of recreating RCM in previous versions of Resolve. I asked the senior colourist from Interstellar and The Hateful Eight if it is possible and he confirmed that it is, which is good enough for me.

It's an area I explored extensively earlier in the thread, but with my newly found ability to build custom LUTs via spreadsheet and expressions I can now build a speedier and more precise process. As a bonus development, the ability to re-map highlights while observing the process in the viewer and scopes. Really useful for when working with Dual-ISO footage.

(http://i.imgur.com/gkKdbK6h.png)

The example above is from LogC footage (available from the Arri website). I applied a 1D 14bit LogC to Linear LUT in the first node (to convert the signal to scene linear), then split the signal in three. Each node uses the RGB Mixer to swap channels (with preserve luminance disabled) so that it's all Red channels in the top, Green in the second, and Blue in the third. Hence why the nodes are B/W. Using Layer nodes I added a custom 1D 8bit LUTs, and multiplied. The 1D 8bit LUTS convert the nodes to constants, representing column values from a 3x3 matrix. The resulting nodes are added back together, with the colour space converted from Alexa Wide Gamut to Rec709.

(http://i.imgur.com/OLPXDIMh.png)


The final stage involves converting the scene linear signal to a display referred signal (such as sRGB, Rec709, or gamma 2.4). The problem with a scene linear signal from a high dynamic range camera is that the range is vast (up to @55.1 for Alexa) so even a 14bit amount of points is spread thinly. Splitting the signal again with two LUTS allows for much greater precision. The bonus is that by using the Highlight option in Resolve you can view and shape the highlights independent of the rest of the signal.

(http://i.imgur.com/kPr9W6dh.png)


(http://i.imgur.com/LeSOYCVh.png)


(http://i.imgur.com/CNFvUmWh.png)


The following download includes a set of Arri Lin Split LUTs, plus the Alexa Wide Gamut to Rec709 Matrix LUTs.


https://www.dropbox.com/s/rbxd7xq8nyu3av7/Arri_Lin_Split_LUTs.zip?dl=0


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: hjfilmspeed on December 27, 2015, 08:20:55 PM
Man this is so much info. All these different processes, which one to use! I'm getting a little lost ha ha. It would be awesome to make a simplified version of this whole thread.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 28, 2015, 08:00:19 PM
I'd say in terms of which process to use, choose the most suitable one for your needs. These aren't one size fits all type methods, they're more likely for very particular workflows plus this thread is an ever evolving experimental trial and error endeavour so a lot of the earlier entries have been rendered obsolete by newer and better approaches (or changes in the actual software).

If somebody wants to start a Resolve related thread and focus on a more beginner orientated approach then I think that's a great idea, and I'll be happy to contribute help and advice when requested plus everything here is freely available to reference or republish.

I'd prefer to continue to focus on developing new approaches though, so it's better if someone else got the new thread up and running. I highly recommend it to anyone considering doing so, as the whole process is a great way to learn new things and consolidate what you already know.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: SpcCb on December 30, 2015, 02:28:02 AM
(...)
I'd prefer to continue to focus on developing new approaches though, (...)
+1

This thread is massively interesting.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on December 31, 2015, 05:20:30 AM
I shot a lot of difficult footage during the summer as a means of testing my camera, ML Raw, and my post production process. One of the most difficult scenarios is when shooting a scene with a high dynamic range and lots of glare, especially if you're pushing the camera by way of higher fps (at the expense of vertical resolution) or Dual-ISO. Sometimes the aliasing and moire (and other weird artifacts) are too pronounced to overcome, therefore rendering the image useless. However, in cases where the outcome is borderline it is definitely worth exploring possible redemptive options.

I was reasonably happy with the composition of some 60fps footage I shot on a beach, even though I was having problems with my polariser, UV, and ND filters (bad skills on my part). I compounded the already bad situation by closing down the aperture to f18, and along with shooting at a resolution of 1728 x 458 (to get 60fps without skipping) I was asking for trouble.

I used MLVFS to convert the MLV file to DNGs. I found that the Bad Pixel and Chroma Smoothing actually made the artifacts worse, so I got better results by not selecting those options.

(http://i.imgur.com/kTW23KLh.png)

I found a noticeable difference when I opened one of the DNGs in ACR, compared to Resolve. ACR still has a better debayer system and provides a cleaner starting point, but overall Resolve offers more control with regards to video.

(http://i.imgur.com/Kt84sL1h.png)

(http://i.imgur.com/6gkid0kh.png)

You can already see how washed out and almost colourless the image is, and how all the micro details and highlights can prove difficult to deal with.

(http://i.imgur.com/D79RcoZh.png)

Selecting Highlight Recovery in Camera Raw didn't help much

(http://i.imgur.com/NOwrsAth.png)

I tried different ways of rescuing the highlights, but pulling keys and blurring channels proved a bit ineffective. I tried splitting the signal, but no joy, and even when there was progress there was detrimental effects elsewhere.
What finally worked came from a chance discovery. I tried out a blur plugin (free from Red Giant Universe) and turned vertical blur down to zero, creating the streaks that covered the artifacts without messing up the smaller waves. It provided the base for the rest of the grade. I fine-tuned and settled on a preferable wash, then added grain in a post-group clip. Although I tried originally in sRGB, I found it was overall easier to deal with both the good and bad while in Cineon.

(http://i.imgur.com/WRPp6H4h.png)

(http://i.imgur.com/xPNaBAWh.png)


Here are some downloads. The first is the original MLV file, the second is a ProRes 422HQ final pass, and the third is a compressed H264 version.


https://www.dropbox.com/s/ztf8zh0bjqw9ks3/Test%20MLV%20Folder.zip?dl=0

https://www.dropbox.com/s/dbetn0s74j2f1zu/Brazil_Beach_Test_Pass_2.mov?dl=0

https://www.dropbox.com/s/hp5hq2ulo9jrsgo/Brazil_Beach_Test_Pass_2_H264.mov?dl=0


And a token butchered version on Youtube

 


So there you go. Solutions can be found to even the most difficult conundrums, albeit with a bit of luck now and again.



Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on January 09, 2016, 12:55:26 AM
There is now an OFX plugin that allows for the input of math expressions in Resolve, and it's open source. It's called ChannelMath, and was primarily compiled for Natron but functions just fine in Resolve. Simply download from Github and place in Library/OFX/Plugins.

https://github.com/nicholascarroll/openfx-misc/releases

I tested with a LogC to Linear expression, an Alexa Wide Gamut to Rec709 3x3 Matrix expression, and a Linear to Rec709 transfer function expression.

(http://i.imgur.com/aScFxRBh.png)

(http://i.imgur.com/xPgyqaQh.png)

(http://i.imgur.com/fc5OwT3h.png)


It's slow (especially with a modest set-up) but caching is your friend, and even if you prefer to stick with speedier LUTs there will always be the option of 100% verification.

Well done Mr Nicholas Carroll, whoever you are  :D
Title: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on January 09, 2016, 03:15:34 AM
Seriously how'd you managed to find that plugin @baldavenger?!?!

Looks useful and definitely worth a try. Will dig into this some more and Thanks for sharing.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on January 09, 2016, 05:29:55 AM
It was very much a chance occurrence. I installed Natron a while back so I could get a better understanding of the basics of OFX plugins, and in the process I signed up for monthly email updates (or something like that). None of the plugins I managed to install were of any use, so I didn't pay that much attention whenever updates occasionally came along, but I half noticed the word math when I quickly glanced at the email that came today so figured what the harm in checking it out. Even when I installed it and booted up Resolve I either expected it not to resister at all in the OFX library, or to be just another dud.

What a lovely surprise it was to be proven wrong. Really unexpected, and really quite wonderful. A few minor tweaks and it could prove to be a major contribution to workflow in Resolve.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Andy600 on January 09, 2016, 09:30:42 AM
Nice find! Apparently all the Natron plugins 'can' work in Resolve. (see: https://github.com/devernay/openfx-misc/issues/36 (https://github.com/devernay/openfx-misc/issues/36))

They just need a small tweak to the manifest when compiling and some linked DLL's to be included in the distro (plus you might also need FFMpeg installed for some to work). The plugins are in bundles but each can be compiled separately. You'll need a C++ compiler to do it - I've got VS so I'll have a go at a Windows build over the weekend and share if I get anywhere.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on January 09, 2016, 03:59:36 PM
Thanks Andy  :) I managed to get a few of the Natron plugins working in OSX by some basic compiling, but the more practical stuff never used to work. Something to do with parametric parameters. I know very little by way of coding, so I was more stumbling in the dark and hoping to get lucky.

If the plugin could be modified to have more expression inputs (like a similar node in Nuke or Fusion) and incorporate proper GPU acceleration, then it really would be a very useful tool. I'm sure there's a way to make that so.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on January 13, 2016, 02:37:33 AM
I ran a few more tests with the ChannelMath plugin. The parametric parameters issue that plagues the other OFX plugins is still present, but it's just a minor inconvenience really. I had hoped to use param2 to turn the plugin into an extended waveform viewer, but that can still be done only without the added ease of a slider.

I had a go at nesting expressions into expressions, and the results were good. That is to say, instead of having separate plugins to perform specific tasks, I wrote everything into one long expression to see if concatenation was viable.

I tried combining a LogC to Linear, Alexa Wide Gamut to Rec709, and Linear to LogC.

LogC to Lin

r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655

g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655

b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655

Alexa Wide Gamut to Rec709

(r * 1.617523) + (g * -0.537287) + (b * -0.080237)

(r * -0.070573) + (g * 1.334613) + (b * -0.26404)

(r * -0.021102) + (g * -0.226954) + (b * 1.248056)

Lin to LogC

r > 0.010591 ? 0.247190 * log10(5.555556 * r + 0.052272) + 0.385537 : 5.367655 * r + 0.092809

g > 0.010591 ? 0.247190 * log10(5.555556 * g + 0.052272) + 0.385537 : 5.367655 * g + 0.092809

b > 0.010591 ? 0.247190 * log10(5.555556 * b + 0.052272) + 0.385537 : 5.367655 * b + 0.092809

Nesting the LogC to Linear expression into the Alexa Wide Gamut to Rec709 expression, and then nesting that into the Linear to LogC expression, results in these expressions (for R, G, and B respectfully):

(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655) * 1.617523) + ((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655) * -0.537287) + ((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655) * -0.080237)) > 0.010591 ? 0.247190 * log10(5.555556 * (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655) * 1.617523) + ((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655) * -0.537287) + ((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655) * -0.080237)) + 0.052272) + 0.385537 : 5.367655 * (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655) * 1.617523) + ((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655) * -0.537287) + ((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655) * -0.080237)) + 0.092809


(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655) * -0.070573) + ((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655) * 1.334613) + ((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655) * -0.26404)) > 0.010591 ? 0.247190 * log10(5.555556 * (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655) * -0.070573) + ((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655) * 1.334613) + ((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655) * -0.26404)) + 0.052272) + 0.385537 : 5.367655 * (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655) * -0.070573) + ((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655) * 1.334613) + ((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655) * -0.26404)) + 0.092809



(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655) * -0.021102) + ((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655) * -0.226954) + ((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655) * 1.248056)) > 0.010591 ? 0.247190 * log10(5.555556 * (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655) * -0.021102) + ((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655) * -0.226954) + ((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655) * 1.248056)) + 0.052272) + 0.385537 : 5.367655 * (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655) * -0.021102) + ((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655) * -0.226954) + ((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655) * 1.248056)) + 0.092809


(http://i.imgur.com/WoE9XO8h.png)


There's a serious performance hit with such long expressions, but it's good to know that it works and that it is an option.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on February 05, 2016, 05:13:33 AM
There was something odd about the Cineon Film Log option in RCM with the 12.2 update in Resolve. Turns out it was only affecting Raw files (well, at least CinemaDNG), and not regular footage. The same was also happening when choosing Linear. I exported some dpx files and brought them into Nuke, and after testing with different varieties of illuminants and primaries it turns out it was ACES colour space (AP0 primaries to be exact). Weird. It would have been cool if Blackmagic Design had actually let people know if was going to adopt this approach, but anyway...

As there isn't an official ACES pipeline established for ML footage I haven't paid a lot of attention to it, but a little extra research has resulted in enough usable information (for now).

You'll find relevant matrix values on this page:

https://github.com/ampas/aces-dev/blob/master/transforms/ctl/README-MATRIX.md

And additional relevant information here:

http://www.reduser.net/forum/showthread.php?134789-ACES-workflow-tested-and-verified&p=1533260&viewfull=1#post1533260

The CIEXYZ option in RCM is basically Linear XYZ, so that's the reference gamma and gamut and effectively the core starting point for all transformations. More useful matrices can be found in the first link.


Ben Turley's LUTCalc is also a handy source. Always select Data In and Data Out when exporting a LUT (Resolve operates in full Data, and handles any legal conversions so stick to Data only with LUTs). I wasn't sure of this before, but I am now.

I am of the opinion now that any gamma transform performed by a LUT should be done separately and only in a 1D LUT, and 3D LUTs should only deal with the colour element. 3D LUTs used for calibrating a monitor can incorporate subtle non-linear gamma adjustments, but they are the exception. From now on I prefer to use a 3D LUT in only a Log to Log manner, such as LogC Alexa Wide Gamut to LogC LC709 (which can't be achieved with a matrix so only a 3D LUT will work).

Please feel free to try out some of the suggestions here and post feedback :)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on February 16, 2016, 08:19:15 PM
I listened again to an old podcast where the colourist Peter Doyle spoke about some of the grading techniques he used on the final Harry potter movie. When I first listened to it about 2 years ago I understood very little of what he was talking about, but things are a bit clearer now. 2 techniques in particular are worth looking at inside of Resolve.


https://www.fxguide.com/fxpodcasts/the-art-of-grading-harry-potter-peter-doyle/


When he mentions 'matrices' around the 15 minute mark it turns out that he actually means expressions.

He stated the intention of the expression was to prevent cyan being greater than the sum of the other two channels (red and green), so given that green would be a common factor it suggests limiting the blue channel value so that it never exceeds the value of the red channel. He mentions the same technique later in the podcast when discussing the scene in the river.

Using the ChannelMath OFX plugin, leave red and green empty, and enter

b > r ? r : b


I believe ChannelMath processes are done in full 32bit float with no clipping/range restrictions, but it can hog the system.

An alternative would be to export as a 3D LUT straight from Resolve, though it's limited to a 33x33x33 cube (whereas Resolve can actually process a 65x65x65 cube)

A 65x65x65 cube can be instead be built in Nuke (using an Expression Node) or Fusion (using a Custom Tool 'tool'). Fusion only offers ALUT3, ITX, and 3DL as LUT options, but the ITX version is identical to that which Resolve exports, you just need to change the extension from '.itx' to '.cube' and Resolve will recognise it.

When I started trying to figure out the expression I came up with some slightly more complex yet nonetheless interesting versions. One limits the channel so that it is never the highest value (despill), the other so that it is never more than the third highest value.

g > b ? (r > g ? g : r) : (r > b ? b : r)

g > b ? (r > b ? b : r) : (r > g ? g : r)

r > b ? (g > r ? r : g) : (g > b ? b : g)

r > b ? (g > b ? b : g) : (g > r ? r : g)

r > g ? (b > g ? g : b) : (b > r ? r : b)

r > g ? (b > r ? r : b) : (b > g ? g : b)

A specific Luma range can be incorporated into the expressions (and LUTs), but it makes more sense to leave that to a more flexible luma key in the application.

The other technique he mentions is splitting the colour from the luma. This is pretty common place now, though perhaps not in such a specific way as he describes.

(http://i.imgur.com/CsxfLlzh.png)

The Splitter-Combiner node reveals which channels are cleaner (usually green) or dirtier (usually blue), so some early repair can be done. Then the signal gets split, using RGB Mixer to select Monotone (Preserve Luma unticked) and then rebuilding the luma channel by substituting blue for green (usually). The bottom layer node has YUV selected and using RGB Mixer (Preserve Luma unticked) to put a zero value in the red channel, and the composite mode is Add. This method doesn't work with LAB. Separate sharpening is done on the luma, and blur on the chroma. Pretty much as he describes it in the podcast.

In the immortal words of Harry Potter, "Shazam!"


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on February 22, 2016, 12:44:29 AM
I looked a bit further into the idea of rebuilding highlights in Resolve, and I've put together a selection of clips that I hope people will have a go at. I'm interested in hearing feedback as to what approaches work better than others.

There are four clips, with two shot regular, and two in Dual-ISO. The regulars are in the form of 100 frame MLV files, and the Dual-ISO are folders with 100 already converted DNG files. All available for download, along with 4 signal splitter 1D LUTs and the ChannelMath plugin (OS X only).

If using the ChannelMath plugin, the following images show the expressions for splitting the signal:

(http://i.imgur.com/srHv8G2h.png)

(http://i.imgur.com/ZHZyDKbh.png)


The clips (in Rec709 Gamma and Gamut) are presented in this order: Viewer + Node Graph, Highlight Recovery Off and Waveform, Highlight Recovery On and Waveform.

(http://i.imgur.com/ZWktVqHh.png)
(http://i.imgur.com/vbUtW06h.png)
(http://i.imgur.com/iuSqRgIh.png)
(http://i.imgur.com/qxfz8DOh.png)
(http://i.imgur.com/KVF2MQhh.png)

(http://i.imgur.com/QKjlXU9h.png)
(http://i.imgur.com/mN6tW8Ch.png)
(http://i.imgur.com/d97bSDph.png)
(http://i.imgur.com/btONKPwh.png)
(http://i.imgur.com/LQCbya1h.png)

(http://i.imgur.com/V4N8af4h.png)
(http://i.imgur.com/v2pWEDJh.png)
(http://i.imgur.com/5o3r3lbh.png)
(http://i.imgur.com/mxUCpWsh.png)
(http://i.imgur.com/nVhBdZbh.png)

(http://i.imgur.com/6YYqjf2h.png)
(http://i.imgur.com/u6LetmXh.png)
(http://i.imgur.com/pJ3WMCch.png)
(http://i.imgur.com/Pr8Iu01h.png)
(http://i.imgur.com/ctcmmBsh.png)


The Node Graph set-up allows for viewing the highlights, along with the option to grade, rebuild, and blur if so required. Rebuilding should be done using the RGB Mixer, with Preserve Luminance unticked.

(http://i.imgur.com/BFiJue9h.png)

The last operation should be the use of Soft Clip (High Soft) in the Custom Curves section:

(http://i.imgur.com/bvHAMNXh.png)

The Soft Clip is a handy way to bring your highlights into display values, though it is limited to how much it can grab so try to have your highlights scaled to no higher than the 384 mark on the waveform once you're finished working on them.

There is great work being done right now by the ML community, so it is only right that equal effort should be put into making the most out of the information contained in the files. Onward and upwards.

Here are the downloads:

https://www.dropbox.com/s/bsf6zp4zp1dn3xm/Signal%20Splitters.zip?dl=0

https://www.dropbox.com/s/gj3mvwcrouum94k/indoors.mlv.zip?dl=0

https://www.dropbox.com/s/ub8iuuysrunf6zy/balcony.mlv.zip?dl=0

https://www.dropbox.com/s/f25khctemium4kh/Beach_Dual_ISO.zip?dl=0

https://www.dropbox.com/s/1jm9x7ywxr60mrq/Sunrise_Dual_ISO.zip?dl=0


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on February 23, 2016, 04:29:27 PM
I converted both the MLVs into folders containing the DNG files (for those who prefer the quicker and easier option).


https://www.dropbox.com/s/7p8e5y8qvc2ddtf/indoors_1_2014-07-25_0001_C0000.zip?dl=0

https://www.dropbox.com/s/8nl3i0yltrmu4n5/balcony_1_2014-10-11_0001_C0000.zip?dl=0



One of the issues with dealing with a split signal is not being able to view the individual streams and make adjustments to earlier nodes (with the Highlight function limiting adjustments to the selected node). This can be somewhat addressed by adding a node to the stream you want to 'switch off' and using the RGB Mixer controls to enter 0 for every channel. This node can then be enabled/disabled as per when required.


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on February 26, 2016, 07:20:24 PM
I recently reviewed my DIT workflow and decided to add a few changes. I focused more on removing the gamma function from the Print LUT in order to have a 'cleaner' signal path.

As a starting point I decided to go with Linear, Linear, Bypass in RCM, which puts the footage into Linear gamma and ACES (AP0) primaries. In the first node I hacked a 3x3 matrix into the RGB Mixer, converting ACES (AP0) to Alexa Film Matrix. The second node has a 14bit 1D LUT applied, converting Linear to Cineon Log Gamma and therefore completing the conversion to a something resembling a scanned negative.

The Film Print LUT stage is now broken into two parts. The first is a Cineon to Cineon 3D Film LUT that deals with chromaticity, followed by a 14bit 1D LUT that converts Cineon to sRGB (or whichever display gamma is required). The Film LUTs that are supplied come in two varieties, with Inter (interim) and Out (more punch) the options. They still add some s-curve, but I modified how much they push the mid-tones and shadows to avoid baked in compression.

(http://i.imgur.com/z3EazCyh.png)

The only adjustments I made to the above image were pulling a small bit of red from the highlights in Colour Wheels, and pushing more green into the foliage with Hue vs Hue.

Also included are Arri to Cineon LUTs, which I wrote some expressions for in order to prevent clipping while still preserving the main part of the signal (0%, 18%, and 90% reflectance).

Print Film LUTs are primarily built for Cineon scanned negative, and given that scanned negative is still the most robust medium for grading it means that the LUTs can be quite brutal on weaker footage (most forms of digital). Also, it expects a high signal midpoint so any footage that is a bit lower than this really gets squashed. Even LogC gets over pushed, which is why I made the Arri to Cineon converters. LogC has a lower midpoint because it actually squeezes in an extra two stops of latitude at both ends of the signal, but it still needs to be conformed in order to get the cleanest possible results.

Anyway, have a go and leave some feedback if you get a chance.


https://www.dropbox.com/s/zckc4t2y8ux43yc/DIT_Workflow_Resolve.zip?dl=0


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: cmccullum on February 27, 2016, 12:18:10 AM
Loving all the info here. I don't really get it, but it's a fascinating thread. That being said, will there ever be a streamlined or "for dummies" guide to all this?

I did some tests using RCM and then the included Alexa log to rec709 lut, and after a little exposure tweaking got what I think is a damn good looking image. I don't really know how to use DR at all so can you break down the benefits of learning such a (seemingly) complicated workflow as opposed to just doing what I did?
(RCM input and timeline Alexa log. bypass output. First node> Alexa log to rec709. next node> adjust exposure/color)
Also, am I even doing that right?
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 01, 2016, 07:11:45 PM
Here's a list of the formulas that are involved in converting LogC Alexa Wide Gamut to Cineon Log Alexa Film Matrix. The larger ones might seem daunting at first sight, but they are merely concatenations of several smaller components. They can be applied directly via the ChannelMath plugin, or exported as LUTs.


LogC to Cineon functions:


log(13.5216945686, 55.0800869419) = 0.649646535749151

0.6496465358

log(abs(-0.0056508194), abs(-0.0172904190)) = 1.275618891720147

1.2756188918

R = r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655

G = g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655

B = b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655

R > 1 ? pow(R, 0.6496465358) : (R < 0 ? pow(abs(R), 1.2756188918) * -1 : R)

G > 1 ? pow(G, 0.6496465358) : (G < 0 ? pow(abs(G), 1.2756188918) * -1 : G)

B > 1 ? pow(B, 0.6496465358) : (B < 0 ? pow(abs(B), 1.2756188918) * -1 : B)


r < 0 ? pow(abs(r),0.7839332002)*-1 : (r > 1 ? 1 : r)

(r > 1 ? pow(r, 0.6496465358) : 1) - 1

AWG to Film Matrix

(r*1.271103)+(g*-0.284279)+(b*0.013176)
(r*-0.127165)+(g*1.436429)+(b*-0.309264)
(r*-0.129927)+(g*-0.510286)+(b*1.640214)

LogC AWG to Arri Linear Film Matrix

((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)


((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)


((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)


LogC AWG to Cineon Linear Film Matrix


(R) > 1 ? pow((R), 0.6496465358) : ((R) < 0 ? pow(abs(R), 1.2756188918) * -1 : (R))

(G) > 1 ? pow((G), 0.6496465358) : ((G) < 0 ? pow(abs(G), 1.2756188918) * -1 : (G))

(B) > 1 ? pow((B), 0.6496465358) : ((B) < 0 ? pow(abs(B), 1.2756188918) * -1 : (B))



(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)) > 1 ? pow((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)), 0.6496465358) : ((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)) < 0 ? pow(abs(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)), 1.2756188918) * -1 : (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)))



(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)) > 1 ? pow((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)), 0.6496465358) : ((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)) < 0 ? pow(abs(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)), 1.2756188918) * -1 : (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)))



(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)) > 1 ? pow((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)), 0.6496465358) : ((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)) < 0 ? pow(abs(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)), 1.2756188918) * -1 : (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)))



LogC AWG to Cineon Film Matrix


(((log10((R)*(1-.0108)+.0108))*300)+685)/1023


(((log10((G)*(1-.0108)+.0108))*300)+685)/1023


(((log10((B)*(1-.0108)+.0108))*300)+685)/1023




(((log10(((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)) > 1 ? pow((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)), 0.6496465358) : ((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)) < 0 ? pow(abs(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176)), 1.2756188918) * -1 : (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*1.271103)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.284279)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*0.013176))))*(1-.0108)+.0108))*300)+685)/1023


(((log10(((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)) > 1 ? pow((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)), 0.6496465358) : ((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)) < 0 ? pow(abs(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264)), 1.2756188918) * -1 : (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.127165)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*1.436429)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*-0.309264))))*(1-.0108)+.0108))*300)+685)/1023


(((log10(((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)) > 1 ? pow((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)), 0.6496465358) : ((((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)) < 0 ? pow(abs(((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214)), 1.2756188918) * -1 : (((r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655)*-0.129927)+((g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655)*-0.510286)+((b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655)*1.640214))))*(1-.0108)+.0108))*300)+685)/1023

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 03, 2016, 04:27:20 PM
The actual order of application with regards to the 3x3 Film Matrix is not exactly clear in the Arri white papers. You would assume that it is meant for a signal in linear light space (especially given that it is referenced in connection with a VFX pipeline), but I'm not so sure. It seems odd to countenance the idea of applying a 3x3 matrix to a log signal, but that might actually be Arri's intention. Anyway, here's a list of formulas that incorporate the log option.


LogC AWG to Arri Linear Film Matrix


((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655


((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655


((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655


LogC AWG to Cineon(clipped) Film Matrix


(((log10((((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655)*(1-.0108)+.0108))*300)+685)/1023


(((log10((((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655)*(1-.0108)+.0108))*300)+685)/1023


(((log10((((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655)*(1-.0108)+.0108))*300)+685)/1023


LogC AWG to Linear(Cineon scaled) Film Matrix


(((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655) > 1 ? pow((((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655), 0.6496465358) : ((((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655) < 0 ? pow(abs(((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655), 1.2756188918) * -1 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655))


(((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655) > 1 ? pow((((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655), 0.6496465358) : ((((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655) < 0 ? pow(abs(((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655), 1.2756188918) * -1 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655))


(((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655) > 1 ? pow((((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655), 0.6496465358) : ((((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655) < 0 ? pow(abs(((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655), 1.2756188918) * -1 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655))


LogC AWG to Cineon(scaled) Film Matrix


(((log10(((((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655) > 1 ? pow((((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655), 0.6496465358) : ((((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655) < 0 ? pow(abs(((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655), 1.2756188918) * -1 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) > 0.1496582 ? (pow(10.0, (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*1.271103)+(g*-0.284279)+(b*0.013176)) - 0.092809) / 5.367655)))*(1-.0108)+.0108))*300)+685)/1023


(((log10(((((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655) > 1 ? pow((((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655), 0.6496465358) : ((((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655) < 0 ? pow(abs(((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655), 1.2756188918) * -1 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) > 0.1496582 ? (pow(10.0, (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.127165)+(g*1.436429)+(b*-0.309264)) - 0.092809) / 5.367655)))*(1-.0108)+.0108))*300)+685)/1023


(((log10(((((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655) > 1 ? pow((((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655), 0.6496465358) : ((((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655) < 0 ? pow(abs(((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655), 1.2756188918) * -1 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) > 0.1496582 ? (pow(10.0, (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (((r*-0.129927)+(g*-0.510286)+(b*1.640214)) - 0.092809) / 5.367655)))*(1-.0108)+.0108))*300)+685)/1023


It's pretty much a return to the original LogC approach from earlier in the thread, albeit with more informed precision. So much for the short-lived dalliance with ACES primaries.

It's probably not a bad time to start putting the various different techniques and processes together into one main pipeline.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Andy600 on March 04, 2016, 10:14:54 AM
re: the Film Matrix - It's the only matrix I know of that is/was intended to be applied in logspace. Log-C with Alexa wide gamut primaries to be precise. It's function is merely to emulate the saturation and hue shift you get when scanning negative stock (most likely 5219) on the ARRIScan. If you apply it as you would most matrices i.e. linear, it will cause over-saturation, particularly in the shadows and highlights. With actual film there is color crosstalk so you will never get an exact match from just using a trivial matrix. ACES and Filmlight are, so far, the only procedural mathematical efforts to try to emulate films' color crosstalk and saturation characteristics but it's a constantly moving target. Print luts are still a very viable alternative for specific stocks and the FM is mainly intended for pipelines using them.

Try to visualize in 3 dimensions what happens and you can see that applying the matrix in logspace has some advantages and disadvantages. The advantage being that you have a more even distribution of saturation (because fundamentally, logspace has/should have an approximate equal number of code values for each stop) and less relative saturation the highlights (a more film like response). The disadvantage is that once you introduce non-linearity (i.e. log, gamma or an s-curve) you can get all kinds of weirdness, hue shifts, hue rotations or over/under-saturated primaries depending on the RGB weighting factor of the saturation values being used - basically the FM is a glorified saturation matrix that is roughly the same in terms of saturation as Rec709 but with some twists. The FM looks pleasing because it shifts skin tones to warmer hues but at the same time it can turn deep reds to orange and deep blues to cyan but, importantly, it keeps grey values neutral and does not affect gamma. 
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 04, 2016, 04:39:52 PM
Thank you Andy, that helps clear up a lot of things :) I'll have to revise some of the earlier posted processes.

With regards to the distribution of saturation, won't this be an issue when applying it to ML Raw footage encoded in LogC ? When encoding, it would appear that peak white (1.0) is mapped to the same point as 90% reflectance is with Arri footage, therefore the saturation fall-off that occurs with Arri footage does not apply (to the same extent) to LogC encoded ML Raw footage as the signal was originally scaled to fit into a display referred space.

Apologies for dragging on with this particular line of enquiry (and over multiple threads too), but there's still something I'm not quite getting. Hopefully the penny will drop sooner or later :)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Andy600 on March 04, 2016, 05:32:40 PM
Yes and no. With Cineon, the sat rolloff is different to the Alexa but not by a great deal once you add contrast. I opted for Cineon instead of Log-C but with Cinelog the film matrix is still applied to Log-C because it was developed for Log-C and looks different when applied to other gammas i.e. it's an extended transform from cineon AWG to Log-C (1D) > add fm > Log-C to Cineon (1D). It would be relatively easy to scale the matrix if it were for linear application but not trivial to do so in logspace and besides, the extended transform is not very CPU intensive so there is little point.

The important thing here is that using matrices to transform a gamut is far easier to invert/undo than non-linear adjustments or complex luts. There are potentially as many 'film matrices' as there are film stocks, as there are scanners etc but you need to shoot charts on film to define a set of target values then find the closest matrix match. Applying a saturation matrix in logspace helps maintain a psuedo-film aesthetic (illusion) but you need to go to good print luts for the crosstalk characteristic and to get closer to film.

Try playing with saturation matrices in logspace. It's quite interesting :)

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 04, 2016, 05:55:41 PM
I see the transform from Cineon to LogC, but doesn't that still preserve the mapping of peak white whereas that should represent reference white instead? That is to say, the saturation roll-off isn't the same as original Arri LogC because the signal is scaled/compressed down. Contrast will increase the signal (and therefore better resemble the preferred distribution in the higher end) but also deepen the lower end, which is probably not desirable. It seems a bit 'loose' to me, at least on a technical level (though I may well be missing something).

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 20, 2016, 08:50:53 PM
Back in Reply #61 there was a brief mention of Dark Frames, along with a link for footage shot on a 5DMkIII. Before proceeding to build a workflow consisting of the 'greatest hits' of the thread, I'll go through an optional process of removing some noise from your footage, especially if shot at a higher ISO.

The Dark Frame has to match the footage it is intended for. That is to say, if you shot a particular resolution (e.g. 1920 by 1080) at a particular frame rate (e.g 24FPS), with a particular ISO setting (e.g. 1600) on a particular camera sensor (e.g. 5D MkIII) then your Dark Frame would have to have been originally compiled using the same camera type and with those settings. Otherwise you will not achieve optimum results.

So for the example above, put the lens cap on the camera (with the correct settings) and record a few seconds. If you were to view the frames of the MLV you would see almost black frames (except for a magenta looking noise). A Dark Frame is a single frame that is to be subtracted from every frame of the destined (actual footage) MLV, so an average frame is made from the few seconds that were shot with the lens cap on.

In Terminal (in OSX) type 'cd' (without exclamation marks) and drag the folder with mlv_dump.osx onto it and press enter. In the same folder have your pre averaged MLV file.

Type

mlv_dump -o Output.mlv -a Input.mlv

With Input.mlv being your pre averaged MLV file.

The result leaves a new Output.mlv in the folder, that is the size of one DNG. Change to the name to something more appropriate, e.g. Darkframe1600.mlv

Bring the MLV you want to perform the subtraction into the folder.

Type

mlv_dump -o footageDframed.mlv -s Darkframe1600.mlv footage.mlv

The result leaves a new footageDframed.mlv in the folder, that is the same size as the original footage.mlv. Use MLVFS to extract a DNG from both MLVs (before and after Dark Frame removal) and compare.

Here are some examples:



(http://i.imgur.com/q7NmpYVh.png)

(http://i.imgur.com/FCxgvJsh.png)


(http://i.imgur.com/pQeP9fJh.png)

(http://i.imgur.com/HKExNsqh.png)



Hopefully this process can be included in one or more of the current post-production options. In the link below you'll find some examples to work on.

Ideally what might happen is that Dark Frames are compiled for every ML ready camera for a variety of resolution, frame rate, and ISO combinations, so that a more automated process can be developed.



https://drive.google.com/file/d/0B0WuqGJ11_Egcm9wVjMwbWJPNDA/view?usp=sharing


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on March 20, 2016, 09:18:55 PM
Thanks for this info Baldavenger. I see how I can insert this in MLP and I will eventually. Many of the steps could be automated. For instance. One can leave a darkframe MLV in A_lut_hold folder which would activate a whole range of processing ending up with complete set of new mlv files. Cool stuff.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on March 21, 2016, 12:18:30 AM
This is actually incredible and gnarly work done by @baldavenger as per usual! [emoji108]
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 22, 2016, 05:09:10 PM
Although really only suitable for viewing purposes, it's possible to use 1D LUTs to turn nodes into constants and therefore perform specific offset operations. Basically, you can view parts of the signal greater than 1.0 in both the viewer and waveform. I built a set of 8bit 1D LUTs for this purpose.


(http://i.imgur.com/3VYgSGKh.png)


In the above example the Layer Node is set to Add, and in Node 02 is placed the ResolveConstant_-1.cube, revealing in the viewer and waveform the original part of the signal with values ranging from 1.0 to 2.0. Disabling Node 02 only disables the effects of the Node, so the result would actually be the original signal added to itself. Instead, replace the LUT with ResolveConstant_Zero.cube, which turns the node into a constant with 0 for every value, therefore effectively disabling the operation.

The LUTs can be combined for more specific offsets. The next example features a standard chart featuring values ranging from 0.0 to 1.0. Applying a 1D LogC to Linear LUT to it expands the signal values as far as 55.08007431

(http://i.imgur.com/UuOts64h.png)

An offset of -55 can be performed by placing ResolveConstant_10.cube and ResolveConstant_-5.cube in Nodes 02 and 04, multiplying them, then place ResolveConstant_-5.cube in Node 08 and set the final Layer node to Add. In the viewer and waveform the values originally ranging from 55 to 55.08007431 can be seen. An exercise in theory of course, but it also shows that the signal doesn't always have to be compressed into a limited space in order to be viewed.


There are other uses for constants, such as a 0.5 constant as part of high-pass sharpening. Here's a fairly simple example:

(http://i.imgur.com/iD48iuUh.png)

The first Layer Node is set to Subtract, with Blur performed in Node 04, and ResolveConstant_-0.5.cube placed in Node 05. The final Layer Node can be set to Overlay, Softlight, or Hardlight (depending on taste).



https://www.dropbox.com/s/os7bms2p8legctb/Resolve%20Constants.zip?dl=0


Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 23, 2016, 06:05:28 PM
For those interested in viewing the whole signal (including sub zero values) there's an easy method using a simple constant.

(http://i.imgur.com/3VYgSGKh.png)

In the above example when ResolveConstant_-1.cube has been placed in Node 02, by selecting Multiply in the Layer Node the signal gets inverted/rotated 180 degrees around the x axis. In the case of a full logC signal linearised, values can go as low as -0.017290419, and this can now be viewed without pulling and pushing the rest of the image. Perhaps not of any great use, but a nice option nonetheless. I've included a Null LUT in the previous post download that clips above 1.0 and below 0 for signal isolation purposes.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 24, 2016, 02:19:01 AM
I looked into the effects of extending the White Level value in the DNGs metadata. With 5D MkIII footage it is 15000, which is mapped to the value 1.0 in Resolve. The reasons for White Level being lower than the full 14bit limit (16383) are well documented - Peak Saturation, non-linearity past that point, magenta cast - but when Highlight Recovery is enabled these extra values come into play anyway.

I ran some tests with DNGs whose White Level I altered using Exiftool. There were three varieties: Normal 15000, Full 16583, and Bright 10000

In Adobe Camera Raw they appeared as expected, with Full 16383 slightly darker than Normal 15000 (due to the values being mapped lower to accommodate the additional 1383 values) and Bright 10000 much brighter in keeping with the fact that the lower value is mapped to peak white thereby raising all values accordingly.

Using the Exposure and Highlight controls it was very easy to bring back the highlight values that got clipped or heavily compressed, so in this case the White Level is really just a convenient guideline for Adobe Camera Raw to map Peak White, and no actual raw data is compromised by altering the metadata.

In Resolve it is a somewhat different matter. When the three DNGs are initially brought in the appearance is more or less the same as in ACR, but then using controls like Exposure and Gain to bring back highlight detail it is revealed that this information has been clipped, with Bright 10000 clearly showing the missing information. Only by enabling Highlight Recovery is that signal information brought back into play, but only though whichever algorithms Resolve uses to rebuild signals.

Now, even with Highlight Recovery on Full 16383 Resolve tries to build more highlight detail, even though there is no reserve signal to call upon. I prefer to use the signal splitting techniques mentioned earlier in the thread to do highlight rebuilding, but that can be combined with Highlight Recovery to increase dynamic range if done judiciously. The less of a signal Highlight Recovery has to anchor to initially, the more control you'll have overall, which is why I now recommend (for those using Resolve and willing to put in the extra work in order to achieve superior results) that you change the White Level of your DNGs to their max, which is 16383 for regular and 65535 for Dual-ISO. This process will involve the Exposure control to scale the signal such that the exposure value of a white diffusing reflector is 1.0

This is a deliberate attempt to take the DNG process away from its originally intended display referred/stills approach, and adapt it towards a more scene referred/film approach like that of Cineon and LogC.

With the sterling aid of the two Daniels (@Danne and @dfort) I've been able to convert with relative ease the White Level of both DNGs and MLVs.

Code: [Select]
$ exiftool -IFD0:WhiteLevel=16383 * -overwrite_original -r.
Is a quick and dirty way to convert every DNG in the folder (and subfolders)

Code: [Select]
echo "0000068: FF 3F" | xxd -r - Input.MLV
Converts the White Level in the header of a MLV. Very handy if you're using MLVFS in combination with Resolve.
@dfort figured out the code and the HEX values needed.

15000 = 98 3A
16383 = FF 3F
10000 = 10 27
60000 = 60 EA
65535 = FF FF
30000 = 30 75

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 24, 2016, 10:38:52 PM
Bonus Tip relating to the first example from Reply #98

Create a scrolling Waveform window by just adding a Node and adjusting Gain.

(http://i.imgur.com/fRkAIj3h.png)

In Node 05 you simply adjust the Gain control in LGG to create the scrolling effect. Gain has values from 0.0 to 16.0, so that covers quite a stretch of signal. To disable the process either replace ResolveConstant_-1.cube with ResolveConstant_Zero.cube, or change the Layer Node option from Add back to Normal.

Even better is if you have a control surface and use the Gain dial to scroll. The displayed Gain value will also identify which part of the signal is currently being viewed in the waveform 'window'.
Title: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on March 27, 2016, 12:24:56 AM
Hey I have question regarding DarkFraming average processing workflow in which I've been doing a lot of rough testing with MLP and was curious if it would matter how long I record an MLV (3 secs vs 10 sec) to make the average before the render begins onto the footage for optimum results?

I also find the results to be more noticeable (improvements) if I work with DNG's directly than with ProRes4444 that was spitted out from ffmpeg. So instead I would take the DNG into AE and export as ProRes4444XQ for best quality as flat log unless I do all the dirty work (CC, LUTS, etc) which then can be exported as H264 for online viewing purposes.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 27, 2016, 12:35:45 AM
With regards recording time, it is subject to the laws of diminishing returns, so anything more than a few seconds makes little or no difference.

As for your other enquiry, what are the exact workflows that you are using?
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on March 27, 2016, 04:55:48 AM
Exactly what I thought. I figured 3 seconds would be plenty for the DF avg to be made and Thanks for the clarification!

I'm currently testing out 'Scenario #1' for DF avg processing within MLP which seems fairly straightforward and works like wonders!

Re: workflows ... I find it more effectively if use the DNG's (DF avg process already applied) and export them as H264 or ProRes. I also played with some Cinelog-C together with these DNG's which was rather interesting to experiment on my end.

Admittedly, I am in the middle of this mess that I made on MLP's thread from last night so please excuse me for a moment as I'll post up my better test results comparisons and will share them with you shortly.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 28, 2016, 02:15:36 PM
Here's a useful post on another forum:

https://forum.blackmagicdesign.com/viewtopic.php?f=2&t=45985#p266945

Interesting and relevant thread too.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on March 29, 2016, 12:44:21 AM
So here's a basic workflow that involves some of the methods that have featured so far in this thread.

ML Raw -> RCM -> LogC AWG -> Linear -> Signal Split -> Cineon -> Print LUT (optional) -> Grain Overlay -> Display Gamma (sRGB/Rec.709)

An alternative workflow involves a Film Matrix conversion.

I recommend setting MLV White Level to its max value (though optional) and raising Exposure by up to 1 stop in Camera Raw controls to compensate for DNG display referred scaling.

(http://i.imgur.com/DWvGZ0mh.png)


Here's the workflow that involves Film Matrix:

(http://i.imgur.com/mlQo87Wh.png)


If you want to replace the 1D LUTs with a hand-made powergrade, then simply take the reverse LUT (i.e. the sRGB to Cineon LUT if trying to replace the Cineon to sRGB LUT) and apply it to the Grey Ramp and manipulate the controls until it's straight again (as done previously in the thread).

Included in the ML Resolve Workflow Pack are a set of 1D LUTs, a pair of 3x3 Matrices, a pack of 3D Print LUTs, Signal Splitters, and a few other useful elements along with the two main powergrades that feature above. Import the powergrades by right-clicking in the Gallery tab and choosing Import.


https://www.dropbox.com/s/w46zz4g2swtjgbe/ML%20Resolve%20Workflow.zip?dl=0


Here's the link to the Grain samples that featured previously in the thread:


https://www.dropbox.com/s/lpo6lnes2uzq13h/Grain%20Scans%20HD%20Flip%20Flopped.zip?dl=0


Have fun grading and feel free to post results.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on April 19, 2016, 01:09:43 AM
Big Resolve update released today (12.5). Lots of new and improved features that that will be familiar to anyone who has been following this thread.

Download the free version here (and also download the free version of Fusion as they can now both be linked):

https://www.blackmagicdesign.com/uk/support

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on June 25, 2016, 02:34:59 AM
Quick PFE LUT tip.

I tried improving on the Print LUTs that were previously posted by reducing the photometric highlight compression and slight shadow lifting that was still prevalent, but it proved persistently problematic. Instead I settled for a more practical compromise, which basically involves using an alpha channel on the node that is receiving the LUT. As the signal going into the node is Cineon log it is better to pull the key in a previous node and pipe it down stream.

Original sRGB image:

(http://i.imgur.com/EIA0Wa7l.jpg)

With LUT applied but key disabled:

(http://i.imgur.com/uBGEFbe.png)

With LUT and key applied:

(http://i.imgur.com/sn0Sulw.png)


Now, Resolve's Luminance Keyer isn't the greatest (and something I'm looking into improving on with an OFX plugin), but the system still works and produces better results than with just the LUT on its own. I reckon I've gone as far as you can go with regards to cleaning up the 'muckiness' of Print LUTs. Always a battle, but sometimes it's worth it.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on June 25, 2016, 02:45:03 AM
All the previously posted formulas can be applied via DCTL, and without a great deal of effort. Simply replace r,g, and b with p_R, p_G, and p_B, and add f to the end of every numerical value.


For example, the original AWG to Rec709 matrix conversion


(r * 1.617523) + (g * -0.537287) + (b * -0.080237)
(r * -0.070573) + (g * 1.334613) + (b * -0.26404)
(r * -0.021102) + (g * -0.226954) + (b * 1.248056)


can be applied as follows:


Code: [Select]
__DEVICE__ float3 transform(int p_Width, int p_Height, int p_X, int p_Y, float p_R, float p_G, float p_B)
{
const float r = (p_R * 1.617523f) + (p_G * -0.537287f) + (p_B * -0.080237f);
const float g = (p_R * -0.070573f) + (p_G * 1.334613f) + (p_B * -0.26404f);
const float b = (p_R * -0.021102f) + (p_G * -0.226954f) + (p_B * 1.248056f);

return make_float3(r, g, b);
}


Likewise the LogC to Linear transform


r > 0.1496582 ? (pow(10.0, (r - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (r - 0.092809) / 5.367655
g > 0.1496582 ? (pow(10.0, (g - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (g - 0.092809) / 5.367655
b > 0.1496582 ? (pow(10.0, (b - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (b - 0.092809) / 5.367655


becomes:


Code: [Select]
__DEVICE__ float3 transform(int p_Width, int p_Height, int p_X, int p_Y, float p_R, float p_G, float p_B)
{
const float r = p_R > 0.1496582f ? (_powf(10.0f, (p_R - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_R - 0.092809f) / 5.367655f;
const float g = p_G > 0.1496582f ? (_powf(10.0f, (p_G - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_G - 0.092809f) / 5.367655f;
const float b = p_B > 0.1496582f ? (_powf(10.0f, (p_B - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_B - 0.092809f) / 5.367655f;

return make_float3(r, g, b);
}


Now of course the necessity for these particular transforms via DCTL is no longer relevant because of the Color Space Transform OFX plugin, but it at least shows that options outside of what is already available in Resolve are easily accessible. The real challenge now is to build custom OFX plugins that in effect remove all limitations to what you can do in Resolve. Learning C++ is proving to be a slow process, but seeing as Blackmagic Design have been good enough to provide the access and resources to really elevate Resolve, then it would be a shame not to at least have a go at making it work.

Note: DCTL option is only available on Resolve Studio version, but the formulas can still be applied via previous methods such as LUTs or the ChannelMath plugin.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on July 07, 2016, 12:21:30 AM
Here are some more potentially useful DCTLs

Add Arri Film Matrix

Code: [Select]
// Arri Film Matrix addition (to be applied to LogC AWG)


__DEVICE__ float3 transform(int p_Width, int p_Height, int p_X, int p_Y, float p_R, float p_G, float p_B)
{
    const float r = (p_R * 1.271103f) + (p_G * -0.284279f) + (p_B * 0.013176f);
    const float g = (p_R * -0.127165f) + (p_G * 1.436429f) + (p_B * -0.309264f);
    const float b = (p_R * -0.129927f) + (p_G * -0.510286f) + (p_B * 1.640214f);

    return make_float3(r, g, b);
}


Preparing Arri Linear for conversion to Cineon

Code: [Select]
// Arri Linear to Cineon Linear Prep

__DEVICE__ float3 transform(int p_Width, int p_Height, int p_X, int p_Y, float p_R, float p_G, float p_B)
{
    const float r = p_R > 1.0f ? _powf(p_R, 0.6496465358f) : (p_R < 0.0f ? _powf(_fabs(p_R), 1.2756188918f) * -1.0f : p_R);

    const float g = p_G > 1.0f ? _powf(p_G, 0.6496465358f) : (p_G < 0.0f ? _powf(_fabs(p_G), 1.2756188918f) * -1.0f : p_G);

    const float b = p_B > 1.0f ? _powf(p_B, 0.6496465358f) : (p_B < 0.0f ? _powf(_fabs(p_B), 1.2756188918f) * -1.0f : p_B);

    return make_float3(r, g, b);

}


Full LogC to Cineon transform with roll-off

Code: [Select]
// LogC to Cineon (with highlight roll-off to preserve dynamic range)

__DEVICE__ float3 transform(int p_Width, int p_Height, int p_X, int p_Y, float p_R, float p_G, float p_B)
{
    const float r = (((_log10f(((p_R > 0.1496582f ? (_powf(10.0f, (p_R - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_R - 0.092809f) / 5.367655f) > 1.0f ? _powf((p_R > 0.1496582f ? (_powf(10.0f, (p_R - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_R - 0.092809f) / 5.367655f), 0.6496465358f) : ((p_R > 0.1496582f ? (_powf(10.0f, (p_R - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_R - 0.092809f) / 5.367655f) < 0.0f ? _powf(_fabs((p_R > 0.1496582f ? (_powf(10.0f, (p_R - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_R - 0.092809f) / 5.367655f)), 1.2756188918f) * -1.0f : (p_R > 0.1496582f ? (_powf(10.0f, (p_R - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_R - 0.092809f) / 5.367655f))) * (1 -.0108) + .0108)) * 300) + 685) / 1023;

    const float g = (((_log10f(((p_G > 0.1496582f ? (_powf(10.0f, (p_G - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_G - 0.092809f) / 5.367655f) > 1.0f ? _powf((p_G > 0.1496582f ? (_powf(10.0f, (p_G - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_G - 0.092809f) / 5.367655f), 0.6496465358f) : ((p_G > 0.1496582f ? (_powf(10.0f, (p_G - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_G - 0.092809f) / 5.367655f) < 0.0f ? _powf(_fabs((p_G > 0.1496582f ? (_powf(10.0f, (p_G - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_G - 0.092809f) / 5.367655f)), 1.2756188918f) * -1.0f : (p_G > 0.1496582f ? (_powf(10.0f, (p_G - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_G - 0.092809f) / 5.367655f))) * (1 -.0108) + .0108)) * 300) + 685) / 1023;

    const float b = (((_log10f(((p_B > 0.1496582f ? (_powf(10.0f, (p_B - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_B - 0.092809f) / 5.367655f) > 1.0f ? _powf((p_B > 0.1496582f ? (_powf(10.0f, (p_B - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_B - 0.092809f) / 5.367655f), 0.6496465358f) : ((p_B > 0.1496582f ? (_powf(10.0f, (p_B - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_B - 0.092809f) / 5.367655f) < 0.0f ? _powf(_fabs((p_B > 0.1496582f ? (_powf(10.0f, (p_B - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_B - 0.092809f) / 5.367655f)), 1.2756188918f) * -1.0f : (p_B > 0.1496582f ? (_powf(10.0f, (p_B - 0.385537f) / 0.2471896f) - 0.052272f) / 5.555556f : (p_B - 0.092809f) / 5.367655f))) * (1 -.0108) + .0108)) * 300) + 685) / 1023;

    return make_float3(r, g, b);
}


Perhaps the best way to utilise Arri Film Matrix with Canon footage is to adjust the saturation response so that it more resembles that of an actual Alexa. The Luma Vs Saturation control can do this, as can applying an LC709A 3D LUT (gamut only). One possible route might be ML Raw -> LogC AWG -> LogC LC709A -> LogC AWG (LogC Rec709 to LogC AWG transform) -> Arri Film Matrix -> LogC to Cineon -> Print LUT

The LC709A LUT can be generated here:

https://cameramanben.github.io/LUTCalc/ (https://cameramanben.github.io/LUTCalc/)

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on July 07, 2016, 01:28:08 AM
Looks like you are getting into programming @baldavenger. When will you put this in dcraw  ;D :P
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on July 07, 2016, 01:30:26 AM
Seriously @baldavenger you rock and eventually it'll be implemented into dcraw which would be a dream come true for all of us one day.

Thanks and keep them coming!
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on July 08, 2016, 10:44:15 PM
Here are some more film emulation type LUTs I put together recently. I was aiming towards a more film-scan/alexa type response, and the LUTs are smaller (17^3) than usual. They expect a Cineon image with Rec709 primaries, and work well in Resolve as well as Photoshop (use the cinelog V3 profile in camera raw if you have it).

There are 6 in total. 2 PFE output, 2 film-scan + PFE output, and 2 film-scan log output (1 for PFE, 1 for Rec709/sRGB)


Original Chart

(http://i.imgur.com/ZG1wQych.jpg)


Cineon Rec709 to Kodak

(http://i.imgur.com/C75pt7Rh.jpg)


Cineon Rec709 to Fuji

(http://i.imgur.com/6Eg7soxh.jpg)


Cineon Rec709 to Film-Scan + Kodak

(http://i.imgur.com/QA9a9U9h.jpg)


Cineon Rec709 to Film-Scan + Fuji

(http://i.imgur.com/hdTKU5Uh.jpg)


Cineon Rec709 to Film-Scan (for PFE)

(http://i.imgur.com/Rwjgir7h.jpg)


Cineon Rec709 to Film-Scan (for Rec709/sRGB)

(http://i.imgur.com/3ibl4tbh.jpg)



https://drive.google.com/open?id=0B0WuqGJ11_EgU1plSkktaGpPNmc (https://drive.google.com/open?id=0B0WuqGJ11_EgU1plSkktaGpPNmc)


Nice and uncomplicated (for a change). Have fun and post some feedback if something springs to mind.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on July 30, 2016, 02:56:12 AM
I've been working on some OFX plugins for Resolve 12.5, and they can be found (along with some LUTs and DCTLs) on my GitHub page:

https://github.com/baldavenger?tab=repositories (https://github.com/baldavenger?tab=repositories)

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: DeafEyeJedi on August 02, 2016, 12:34:53 AM
Thanks for sharing the OFX plugins @baldavenger! I'll definitely take a dive with them and report back when I can.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on September 02, 2016, 07:48:51 PM
I did some spring cleaning on the OFX plugins. I removed as much unnecessary code as I deemed probable, plus I employed a better naming structure (as well as a wee icon for the top left corner). Hopefully it'll be easier now for those wanting to compile for Win or Linux.

https://github.com/baldavenger?tab=repositories
 (https://github.com/baldavenger?tab=repositories)

I took a break from coding endeavours for a month or so, but I hope to add some new ones soon (plus do some more fine-tuning on the ones already there).



Title: Re: DaVinci Resolve 12 and ML Raw
Post by: kfprod on November 22, 2016, 05:54:56 PM
Hi,

There's an amazing amount of knowledge here but very hard to get an overview of the work you've done.

I work a lot with mixed footage from ML, Red and Alexa and I'm looking for a simple way to get my Magic Lantern footage to the Alexa Log-C colourspace (as it is now the industry standard).

The best way I've found of doing this so far is with the Cinelog DCP transformation Luts from http://www.cinelogdcp.com/ but it does seem a bit odd the transformations that happen there and I feel there must be an easier (better?) way...

Also your OpenFX plugins look amazing but I don't really know where to start, is there a site where you show how to use them and what they're for?

Best regards,
Karl
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: 66dellwood on November 22, 2016, 11:29:31 PM
This is amazing work!  Great job and Thanks much.
1.  Need a LUT recommendation for Resolve.  For cdng footage shot on a 5d2.  I set my raw settings to BMD Film and use Hunters LUT for quick basic color correction (not for style).  Hunters LUT does a nice job but is getting old and displays some artifacts.  Other LUTs are not working well. Most make white balance correction very difficult.

2.  Speaking of white balance, how does the Balance/white balance plugin work?  WB is often my most time consuming fix for basic color correction.  Wish it was as quick and simple as WB fix for raw in LightRoom
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: kfprod on November 23, 2016, 12:32:08 PM
66dellwood: The EOS HD lut is much better than the Hunter LUT: http://www.eoshd.com/2013/11/introducing-eoshd-film-lut-5d-mark-iii-raw-resolve-10/

I'm also interested in the white balance plugin, it's so weird that Resolve still doesn't have a color picker!?
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: Danne on November 23, 2016, 12:50:01 PM
Quote
it's so weird that Resolve still doesn't have a color picker!?
+1
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: 66dellwood on November 23, 2016, 08:13:29 PM
I have been using the EOS HD LUT for a while. On much of my footage I have the hardest time neutralizing the white balance with the EOS LUT. No such problem with the Hunter LUT but it has other issues.  The EOS LUT is probably optimized for the 5d3.  Perhaps 5d2 footage is creating some color balance issues with it.
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on January 20, 2017, 03:42:13 AM
The OFX plugins have been refined and improved, and Win compatible versions are also available.

Mac versions

https://github.com/baldavenger/Plugin-Downloads (https://github.com/baldavenger/Plugin-Downloads)

Win versions

https://github.com/baldavenger/Win-OFX (https://github.com/baldavenger/Win-OFX)

Both format versions work fine with CUDA, but no OpenCL yet on Win (and only limited functionality on Mac).


One possible use of the plugins is as an alternative to the Dark Frame process of removing magenta cast in high ISO image shadows. Play around with the ChannelBoxV2 plugin to see what results you might get.

Title: Re: DaVinci Resolve 12 and ML Raw
Post by: baldavenger on April 02, 2019, 06:50:46 PM
If you're working in ACES in Resolve, then this might be worth a look.





There's more info, and download links, here:

https://acescentral.com/t/davinci-resolve-dctl-and-ofx-plugins/1385/89?u=paul_dore (https://acescentral.com/t/davinci-resolve-dctl-and-ofx-plugins/1385/89?u=paul_dore)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: yokashin on April 02, 2019, 08:12:04 PM
I'm also interested in the white balance plugin, it's so weird that Resolve still doesn't have a color picker!?

(https://cdn.fstoppers.com/styles/full/s3/media/2017/08/05/how-to-set-white-balance-in-davinci-resolve.jpg)
Title: Re: DaVinci Resolve and ML Raw
Post by: baldavenger on April 19, 2019, 02:19:00 AM
I put together a small set of LMTs for editing photos in Resolve

The ACES TOOLS set consists of the following:

IDT_INV_REC709
IDT_INV_SRGB
LMT_DNG_OFX
LMT_COLOR_BALANCE_OFX
LMT_HSL_OFX
LMT_HUE_VS_LUMA_OFX
LMT_PFE_OFX
LMT_PFE_OFX_NEUTRAL

(https://i.imgur.com/rqegQPTl.jpg)

LMT_DNG_OFX helps with images that are a little compressed in the shadows.


(https://i.imgur.com/hUF93EEl.jpg)

(https://i.imgur.com/FvKkRCfl.jpg)

(https://i.imgur.com/28FFnFVl.jpg)



LMT_COLOR_BALANCE_OFX, LMT_HSL_OFX, and LMT_HUE_VS_LUMA_OFX emulate some of the controls familiar to users of Photoshop and Camera Raw. They can be used separately, or in conjunction with other LMTs (including LMT PFE)


https://github.com/baldavenger/DCTLs/tree/master/ACES%20TOOLS (https://github.com/baldavenger/DCTLs/tree/master/ACES%20TOOLS)

The tools work with either CUDA or METAL in Resolve 15 and upwards.
Title: Re: DaVinci Resolve and ML Raw
Post by: baldavenger on April 20, 2019, 12:32:25 AM
This video covers some of the LMT PFE OFX DCTL code.






https://github.com/baldavenger/DCTLs/blob/master/ACES%20TOOLS/LMT_PFE_OFX.dctl (https://github.com/baldavenger/DCTLs/blob/master/ACES%20TOOLS/LMT_PFE_OFX.dctl)
Title: Re: DaVinci Resolve 12 and ML Raw
Post by: allemyr on April 24, 2019, 06:35:34 PM
I looked into the effects of extending the White Level value in the DNGs metadata. With 5D MkIII footage it is 15000, which is mapped to the value 1.0 in Resolve. The reasons for White Level being lower than the full 14bit limit (16383) are well documented - Peak Saturation, non-linearity past that point, magenta cast - but when Highlight Recovery is enabled these extra values come into play anyway.

I ran some tests with DNGs whose White Level I altered using Exiftool. There were three varieties: Normal 15000, Full 16583, and Bright 10000

In Adobe Camera Raw they appeared as expected, with Full 16383 slightly darker than Normal 15000 (due to the values being mapped lower to accommodate the additional 1383 values) and Bright 10000 much brighter in keeping with the fact that the lower value is mapped to peak white thereby raising all values accordingly.

Using the Exposure and Highlight controls it was very easy to bring back the highlight values that got clipped or heavily compressed, so in this case the White Level is really just a convenient guideline for Adobe Camera Raw to map Peak White, and no actual raw data is compromised by altering the metadata.

In Resolve it is a somewhat different matter. When the three DNGs are initially brought in the appearance is more or less the same as in ACR, but then using controls like Exposure and Gain to bring back highlight detail it is revealed that this information has been clipped, with Bright 10000 clearly showing the missing information. Only by enabling Highlight Recovery is that signal information brought back into play, but only though whichever algorithms Resolve uses to rebuild signals.

Now, even with Highlight Recovery on Full 16383 Resolve tries to build more highlight detail, even though there is no reserve signal to call upon. I prefer to use the signal splitting techniques mentioned earlier in the thread to do highlight rebuilding, but that can be combined with Highlight Recovery to increase dynamic range if done judiciously. The less of a signal Highlight Recovery has to anchor to initially, the more control you'll have overall, which is why I now recommend (for those using Resolve and willing to put in the extra work in order to achieve superior results) that you change the White Level of your DNGs to their max, which is 16383 for regular and 65535 for Dual-ISO. This process will involve the Exposure control to scale the signal such that the exposure value of a white diffusing reflector is 1.0

This is a deliberate attempt to take the DNG process away from its originally intended display referred/stills approach, and adapt it towards a more scene referred/film approach like that of Cineon and LogC.

With the sterling aid of the two Daniels (@Danne and @dfort) I've been able to convert with relative ease the White Level of both DNGs and MLVs.

Code: [Select]
$ exiftool -IFD0:WhiteLevel=16383 * -overwrite_original -r.
Is a quick and dirty way to convert every DNG in the folder (and subfolders)

Code: [Select]
echo "0000068: FF 3F" | xxd -r - Input.MLV
Converts the White Level in the header of a MLV. Very handy if you're using MLVFS in combination with Resolve.
@dfort figured out the code and the HEX values needed.

15000 = 98 3A
16383 = FF 3F
10000 = 10 27
60000 = 60 EA
65535 = FF FF
30000 = 30 75

I finally have read thru the thread once and twice, haven't understand everything yet. Thank you very much for this tip! It looks better in Resolve with the 16000 ish white level. What do you think about the 2048 black level on the 5D3? I use pre tone curve still, the image looks overall better now, thx!
Title: Re: DaVinci Resolve and ML Raw
Post by: baldavenger on April 26, 2019, 04:13:52 PM
Quote
What do you think about the 2048 black level on the 5D3?

I wouldn't mess with the black level value. It will screw up the white balance, and more than likely you'll just be introducing more noise into the shadows. However, you can experiment with the raw white and black levels in MLV App to see for yourself the effect.
Title: Re: DaVinci Resolve and ML Raw
Post by: baldavenger on May 01, 2019, 10:03:34 PM
Demonstration of some of the LMTs in ACES TOOLS, as well as an approach to grading CinemaDNGs in ACES





https://github.com/baldavenger/DCTLs/tree/master/ACES%20TOOLS
 (https://github.com/baldavenger/DCTLs/tree/master/ACES%20TOOLS)

https://github.com/baldavenger/BaldavengerPlugins/blob/master/Frequency.ofx.bundle.zip (https://github.com/baldavenger/BaldavengerPlugins/blob/master/Frequency.ofx.bundle.zip)
Title: Re: DaVinci Resolve and ML Raw
Post by: Danne on May 02, 2019, 06:45:01 AM
This is highly interesting. You have done some serious work on this. Thanks for sharing.
Title: Re: DaVinci Resolve and ML Raw
Post by: Kharak on July 15, 2019, 11:26:46 AM
Very nice tools baldavenger.

I've been using your Frequency separator to deal with moire, very easy to remove the color cast of moire with YUV or LAB, but do you know a way to deal with the spatial frequency of moire? Maybe someway to trick Resolve in to Baselights texture management?

I feel like there could be something better than masking and blurring it away, as it just looks blurred. Any ideas?
Title: Re: DaVinci Resolve and ML Raw
Post by: baldavenger on July 20, 2019, 03:58:37 PM
Very nice tools baldavenger.

I've been using your Frequency separator to deal with moire, very easy to remove the color cast of moire with YUV or LAB, but do you know a way to deal with the spatial frequency of moire? Maybe someway to trick Resolve in to Baselights texture management?

I feel like there could be something better than masking and blurring it away, as it just looks blurred. Any ideas?

It's a tricky one, because the spatial frequency you speak of i.e. high frequency pattern, is often burnt into the image by the time you get to grade it. You can be quite aggressive with the colour component of an image without fundamentally altering the perceptual structure, but even the slightest adjustment to the luma component stands out a mile.

The Frequency Separation OFX Plugin only has the option of sharpening high frequency data. It's literally applying gain to the isolated image data, and because the values lay either side of 0 this is, in effect, a pure linear contrast. If the parameter (which returns a value between 0 and 3) would also have the option of returning a negative value, then this would allow for reducing sharpening (contrast) of the high frequency data, therefore blending it back into the low frequency domain. However, it may not produce the effect you're looking for and, to be honest, it's never going to be a simple task. If it's hardcoded part of the luma signal then it's a VFX removal job (and all that entails).
Title: Re: DaVinci Resolve and ML Raw
Post by: Danne on September 30, 2019, 02:57:14 PM
I have an issue where cdng sequences will not embed teh folder/dng files into one movie file like was the case before. I´m on resolve 15.3.1. I search the web and there´s mention of ditching cdng format in favour of bmraw, resolve own raw format but will this include davinci resolve not even reading cdng files as before too?
Tested cdng sequences produced in mlv app and in Switch. Only opens single files. Or I can open all files in the folder and it´s treated as a sequence but the smooth folder embed workflow seems gone?
Title: Re: DaVinci Resolve and ML Raw
Post by: Kharak on September 30, 2019, 05:14:20 PM
i think of two posibilities.

Its either the naming of your folders/files, because with your windows batch converter, which has resolve naming, it works. Or you checked the import setting that makes images being imported individually and not sequenced, i dont remember where that setting is. I believe its in the media window somewhere, you click the (...)

Ps, you should stay on 15.3.1 it is way faster and stabile compared to ver 16 with cdng. Also the layer mixer when applied to a timeline node does not work properly in the versions after 15.3.1 and reverting back to 15.3.1 does not fix it, once you brake it you lose it.
Title: Re: DaVinci Resolve and ML Raw
Post by: Danne on September 30, 2019, 05:28:03 PM
i think of two posibilities.

Its either the naming of your folders/files, because with your windows batch converter, which has resolve naming, it works. Or you checked the import setting that makes images being imported individually and not sequenced, i dont remember where that setting is. I believe its in the media window somewhere, you click the (...)

Ps, you should stay on 15.3.1 it is way faster and stabile compared to ver 16 with cdng. Also the layer mixer when applied to a timeline node does not work properly in the versions after 15.3.1 and reverting back to 15.3.1 does not fix it, once you brake it you lose it.
Shoot. I just upgraded...
I can't seem to find that particular import setting. I try to import by choosing 'import file'. Other ways to import?
Dang. Noob...

EDIT:
(https://i.postimg.cc/KYzbXkwY/Screenshot-2019-09-30-at-17-38-22-png-800px.png)

(https://i.postimg.cc/PrWT1k1d/Screenshot-2019-09-30-at-17-38-34-png-800px.png)


EDIT2: OOOps! HAd to go through media storage. Simply importing file not working the same. SOLVED!
(https://i.postimg.cc/Jh2GSWT9/Screenshot-2019-09-30-at-17-41-56-png-800px.png)
Title: Re: DaVinci Resolve and ML Raw
Post by: Dmytro_ua on September 30, 2019, 07:00:08 PM
you should stay on 15.3.1 it is way faster and stabile compared to ver 16 with cdng.

I don't see the speed difference. The only one is 15's version uses GPU by default (a dedicated Graphic Card) and 16's ver it may choose an internal Intel GPU.
You should change it in preferences.
Title: Re: DaVinci Resolve and ML Raw
Post by: DeafEyeJedi on October 15, 2019, 06:31:27 PM
Has this been tested on V.16 (while running on Catalina) or am I better off leaving it in V.15 for DR re: on OS X?