DaVinci Resolve and ML Raw

Started by baldavenger, September 01, 2015, 11:41:51 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

baldavenger

The latest release of DaVinci Resolve (free version) makes working with ML Raw video a doddle, and thanks to developments such as MLVFS the overall post-production process has come a long way since the early days.  Anyway, this thread is for anyone who wants to post advice and tips with regards to using ML Raw and Resolve 12.


The introduction of Resolve Color Management is the big game changer.  My preferred method is working in Alexa LogC, and this is now I go about it:

In the Master Project Settings, choose DaVinci YRGB Color Managed (as opposed to the default DaVinci YRGB)

In the Color Management section of Project Settings there is Input Colorspace, Timeline Colorspace, and Output Colorspace.  The way RCM works is that it converts all footage to Scene Linear at the initial stage and from then on it converts to whatever colorspace is selected for the timeline, and from there onto the output transform.  For non raw footage, it's necessary to correctly input the footage colorspace otherwise the process won't work properly.  With raw footage that's not necessary, as RCM can directly debayer into your chosen Timeline colorspace.  The same goes for Output Colorspace, but I select Bypass in the options as I prefer to see what's going on under the hood.

Selecting Alexa LogC for Timeline Colorspace, and Rec.709 Gamma 2.4 for Output Colorspace (it doesn't make any difference what is entered in the Input Colorspace field with raw footage), the image you see on your monitor will be the same as the one you see when you're in normal DaVinci YRGB and you picked Rec.709 for Color Space and Gamma in the Camera Raw panel.  By selecting Bypass as the Output Colorspace option, you will see your image in LogC Gamma and Alexa Wide Gamut.  The Arri LogC LUTs that come with Resolve don't expand the signal out to Gamma 2.4, which is annoying, but there is a solution to this.  A free website that produces and coverts LUTs.  It's awesome.

http://cameramanben.github.io/LUTCalc/LUTCalc/index.html

Here's a screenshot with the settings I used to generate a LogC Alexa Wide Gamut to Rec.709 Gamma 2.4 LUT.




The website can be used to generate Gamma or Gamut only LUTs, so you can isolate both parts of the process.  You can also import a LUT and modify it for your needs.  Very handy if say you like the colour transform of a LUT but the contrast is too much and you'd rather replace the Gamma function with something more appropriate, or even remove that part altogether and do the tone curve yourself.

That should do for now.  More updates will follow  :)
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

Right.  Now that a basic workflow has been established (Alexa LogC), it's time to look at a common issue that arises when dealing with footage in a log space.  It's really hard to pull keys when the footage is flat.  This is how I get around it.

Using the website from the previous post we were able to get an adequate LogC Alexa Wide Gamut to Rec709 Gamma 2.4 LUT, so that goes on the last node and we grade under it to take advantage of the log space.  A basic set-up would be this:



On the first node I made a layer mode, then added a serial node to the underlying (hidden) node.  The last node has the LogC to Rec709 LUT applied.  The same LUT is applied to the first (underlying node).  Nothing that happens to nodes 1 and 4 above have any effect on the main stream as node 3 has a full alpha channel by default so blocks their input.  You can pull a key on node 4 from a full Rec709 image and pump the alpha channel directly to node 5.  Pretty straightforward, and in no way demanding on your computers resources.

To pull multiple keys, you simply incorporate parallel nodes after node 1:




All this will be handy to anyone who uses DaVinci.  In upcoming posts I plan to look at working with separate LUTS for Gamma and Gamut functions, replacing Gamma LUTs with powergrades, and eventually finding a way to replace Gamut LUTs with powergrades too, therefore creating a LUT free environment.  I'll be looking at some popular creative look LUTs and seeing if they can be modified to suit Magic Lantern Raw video better, ultimately in the form of accessible powergrades.

Good times ahead folks :)
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Danne

Very useful, if I can get it to work :).  Tried a while ago to alter a lut. Thanks for sharing.

DeafEyeJedi

Great read and thanks for sharing once again @baldavenger -- now I need a new Mac to run DR!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Andy600

@baldavenger - Good tips ;) I would add a couple of things. The Log-C to Rec709 transfer function will clip highlights at output as there is no s-curve. Try adding soft-clipping to the lut in the Lut Panel and export a new one. This will clamp the output but allow you to smooth highlight rolloff (it's not the best way to do it but highlights should look better and not affect midtones/shadows).

Also, if you are explicitly stating a legal levels output you need to check your output codec and settings as you may get a gamma mismatch, especially in other apps or on Vimeo, Youtube etc. Legal output is for deliverables. Resolve can automatically select the correct levels according to the codec if you are using the full color management pipeline (input/workspace/output) but because you are overriding the last stage with a lut you should probably test to see what happens should someone want to render something other than 8-bit (H.264 etc) and set levels accordingly in the main settings.

Also, the last time I checked, the native Log-C to Rec709 lut in Resolve wasn't correctly transforming the colorspace. It does something weird to the gamut and I'm surprised no one has ever mentioned it (caveat: haven't checked the v12 version). The online ARRI lut generator is also worth a mention here, especially for this workflow http://www.arri.com/camera/alexa/tools/lut_generator/.

Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

baldavenger

Thanks Andy600, I'll be getting round to all those issues soon enough.  The Data to Legal settings featured in the original post are based on the ARRI LUT Generator, as it always stipulates that when producing a Log to Video LUT.  Anyway, all other options will be explored in due time.  The clipping issue you speak of is somewhat offset by mapping reflected to 95% instead of 100%, though I am looking into a better approach.  Have a play with Resolve 12 and try out the approach with a grey ramp and ML Raw DNGs.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

Right.  Time to look at separating the Gamma and Gamut functions from the LUT.  Although convenient to have both operations performed by the same LUT, in the long run I feel it's better to have them in separate nodes.  In LUTCalc you can generate a LogC Alexa to Rec709 2.4 Alexa (for just Gamma conversion) and also LogC Alexa to LogC Rec709 (for Gamut).  Gamut transforms can be quite complex, involving a 3x3 matrix, so for now we'll stick with the LUT.  They tend to be accurate and not destructive, so as LUTs go they're not so bad.  Let's see if we can do something with the Gamma LUT.

Import a grey ramp into your Timeline in Resolve.  Here's a good quality Tiff you can use:

https://www.sendspace.com/file/ncno7k

In your waveform there should be a direct diagonal line from 0 to 100.




Enable RCM in the Project settings, and select Rec709 Gamma 2.4 in the Input Colorspace, Alexa LogC in the Timeline Colorspace, and Bypass in the Output Colorspace.  The waveform should now resemble this:




Open the curves section and perform something similar to this, with similar results:



It's by no means perfect, but it's still quick and easy to do.  That's a Gamma only LogC to Rec709 2.4 transform.  Right-click the main image and select Grab a Still.  Give it a label and put it in your  powergrades and you're good to go.  It's non-destructive, doesn't break Resolve's 32bit colorspace, and can be easily adjusted.  I'm working on a more accurate alternative, but for now it'll do.  Combine the powergrade with the Gamut LUT (in serarate nodes).  All good.

Next entry will be about the Gamut LUT.  I mentioned the 3x3 matrix before.  Have you noticed that in the RGB Mixer that there are 3 sections with 3 inputs each?  They may be of some use... 
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

baldavenger

So basically, if you can get hold of the matrix values for a Gamut conversion you can enter them as value in the RGB Mixer.  It's not a perfect approach because you can't enter the exact values, but you can get pretty close and with some minor tinkering while using your scopes you can narrow the gap.  Because we're working in Alexa Wide Gamut, and Arri are a vey cool company, we have access to those matrices.

You can find them in this document:

http://www.arri.com/?eID=registration&file_uid=8026

And there's further explanation of their colour science in this white paper:

http://www.arri.com/?eID=registration&file_uid=7347

The important ones to note are the tone mapped Alexa Wide Gamut to Rec709 Gamut:



Plus the Film Matrix one:



You can get the Alexa Wide Gamut to Film Matrix in a LUT as well, from their LUT Generator, as an alternative or just a comparison.

In the RGB Mixer you enter the numbers as follows:  (1st Line) Red Output R G B, (2nd Line) Green Output R G B, (3rd Line) Blue Output R G B

Also worth noting is the order of operations in Resolve, and in particular the nodes:



A good practicing test chart would be something like this:



You can download a 1920x1080 version here:

https://www.sendspace.com/file/zl5z2a

So that's a few more options covered.  Next I'll be looking at an alternative Gamma curve to better suit ML Raw footage (it's always darker that how it was shot), reverse engineering creative looks, and maybe even a wee surprise in the form of a free present :)
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

DeafEyeJedi

Incredible read once again and especially on the Film Matrix stuff. [emoji1]

One question though... If I were to download and print the test chart won't it display proper colors because every printer uses different inks, etc?

Thanks for sharing anyway and I'll try printing one out in large 16x9 paper just for the heck of it at work later! [emoji106]
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

baldavenger

I just installed the latest Beta of Resolve (number 4) and it would appear that they've tinkered with the colour science again.  The LogC to Rec709 Gamma 2.4 LUTS now make the image too bright.  To get the image you get in non RCM, you go from Arri LogC to Rec709, and not Rec709 Gamma 2.4.  Anyway, I had a quick tinker myself, and with the grey ramp I selected Input Colorspace Rec709, Timeline Colorspace Arri LogC, and Bypass for the Output Colorspace.  With the curves I was able to do a simple reverse engineer, and saved it as a still.  I popped that on a ML Raw clip in Arri LogC, and in a node after that I added the Alexa Wide Gamut to Rec709 Matrix values to its RGB Mixer (via powergrade), and the image matched the non RCM version.  All good.

PM me if you have any queries and would rather not post in the thread.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

@DeafEyeJedi

The image is for viewing on a monitor, to use with scopes etc.  It's an 8bit HD image, and you'd really want to be looking at something with a higher bit depth and much higher resolution for printing.  Standard printing expects usually 300 pixels per inch, so the chart would be very small.  If you want to use a real life chart I suggest X-Rite Color Passport, or better still something from DSC Labs.

PM me if you have any more queries, and I'll post them on the thread if they're relevant to the main topic.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

I still have to get round to to exploring reverse engineering techniques, plus adding a few other handy tips, but for the time being you can play around with these creative look LUTs that I put together.  I played around with some of the popular and freely available Print Emulation LUTs (Juan Malera, Resolve Film Looks, etc) in LUTCalc, and put together a nice little package especially for Magic Lantern Raw in Resolve.  I'm calling them the Magic Lantern Fudak LUTs Version 1.

There are 5 looks, with 2 versions of each.  The LogC Alexa LUTs are to be applied to footage in LogC Gamma Alexa Gamut, and the Rec709 Alexa LUTs are to be applied to footage that's already been Gamma corrected but still in Alexa Wide Gamut.  Pretty straight forward.  They're a bit punchy, but they're Version 1 so I figured I'd be a bit indulgent this time round.  More refined versions will probably follow.

https://www.sendspace.com/file/6o4e3i

Bon Appetit  :)



EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

reddeercity

Great Stuff @baldavenger  thanks, keep this up you are almost convincing my to try out Resolve for ML,
Actually This is very good information , I'm Editing & Grading a film shot on a BMCC (waiting for the Hard Drives) and I originally was going
to make ProRes4444 XQ and use FCPX etc.... but now I think I will either do a round trip or do the whole film in Reslove.
:)

DeafEyeJedi

Quote from: reddeercity on September 06, 2015, 05:49:31 AM
... ProRes4444 XQ and use FCPX etc.... but now I think I will either do a round trip or do the whole film in Reslove.
:)

Likewise, Mate...

A very well job done to @baldavenger for your continued contributions & please keep 'em coming!

Will test out your LUTS and report back.

[emoji106]
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

baldavenger

Here's something I came up with just now.  It's a bonus grain tip.




You can make compound nodes now in Resolve.  Very handy.  In the above example there's a grain scan imported as a matte, added via overlay blend in a layer node, and selectively restricted to preferred areas of the image i.e. shadows and mids rather than highlights, via a luminosity mask pulled in node 1 and piped into node 7.  Select all.  Compound node.  Rename.  Powergrade.  Bosh.  You can go back in for a tinker about as well, if needs be.

Grain is cool.  Don't ever let anyone say otherwise.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

DanHaag

I love grain on my master files but have never been pleased with how it turned out after uploading to Vimeo or Youtube. Tried the dedicated x264 settings as well as all kinds of ProRes versions or whatever else was suggested online for web compression. It just never worked for me as soon as I put it on an online platform but in it's original state in my uncompressed master files I wouldn't wanna live without it.  8)

baldavenger

Yeah, streaming sites tend to butcher grain and leave awful artifacts.  It increases bitrate requirements considerably, so compression algorithms target it first.  The more aggressive the compression algorithm, the more blotchy and warped the grain becomes.  Best to add it at the end, or as a track node, that way you can easily disable it and render out both a grained and grain free version, as requirements dictate.

When I started tinkering about with VFX I saw grain as the enemy, and valued nothing more than a 'clean' de-grained image.  Not anymore.  Good grain enhances everything.  Perceived sharpness, plus no authentic film look is complete without it.  I was convinced the last season of Mad Men was shot on film, until I was informed it was in fact shot on the Alexa.  More than anything else, it was the skilful addition of grain that fooled me.  Criterion's recent release of remastered films from the last 70's/early 80's such as Thief and Blow Out painstakingly preserve the original grain structure, and they look amazing.  Especially on a big screen where you can really observe it in action.  Long may it continue to be so.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

mothaibaphoto

Just curious: why all these complexities? Just to mimic 12bit logarithmic RAW workflow with 14 bit linear RAW? Yes, no doubt, it's possible :) But I don't see the point.

baldavenger

@mothaibaphoto

Everything posted here is optional, with regards to how one might utilise Resolve.  If you prefer to debayer straight into Rec709 Gamma and Gamut then grand, that works too of course.  Some people though prefer the option to debayer into a log gamma and wider gamut, and with Arri LogC you get the advantage of being able to utilise Alexa colour science, etc.  Also, being able to incorporate ML Raw footage into a common colorspace with footage from other cameras could be deemed useful.  As for the 'complexities', although you may have no interest in them, the node structures and instructions on analysing and modifying LUTs may be of interest to some people.

Please PM me if you have any specific queries.  I would prefer to keep validation related discussions from the thread as they are not in keeping with its intended purpose.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

mothaibaphoto

Nowhere in your thread mentioned before that
Quote from: baldavenger on September 07, 2015, 11:18:27 AM
Everything posted here is optional ...
Any newbie looking info about working with ML RAW in DR will get false impression than all these absolutely mandatory.

Quote from: baldavenger on September 07, 2015, 11:18:27 AM
...I would prefer to keep validation related discussions from the thread as they are not in keeping with its intended purpose.
- Ok, than you should change your topic theme from broad "DaVinci Resolve 12 and ML Raw" to "DaVinci Resolve 12 and ML Raw for LUT nerds" or "DaVinci Resolve 12 and ML Raw for anyone willing to fool himself he shoot Arri"

baldavenger

You are clearly a wonderfully joyous human being.  Thank you for your marvellous contribution.

Have a good day sir.  Be a decent chap and please refrain from any further posts on this thread.  Perhaps your powers of philanthropic computational assessment and cultural sensitivity may be better suited elsewhere.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

mothaibaphoto

I'm so sorry you have nothing to say on topic, rather than speculating on my humble personality.
But I have:
Would you please be so kind to illustrate this proposition:
Quote from: baldavenger on September 04, 2015, 07:31:37 PM
Next I'll be looking at an alternative Gamma curve to better suit ML Raw footage (it's always darker that how it was shot)
For me ML Raw footage looks exactly as was shot in DR and the same, say in ACR.
Thank you.

baldavenger

I've already asked you twice to stop posting on this thread.  I might have been willing to help, but you were being rude and obnoxious.  I don't know what your problem is Dmitry, but if you are having some sort of break down then I suggest you have it somewhere else.  Let Olga take care of you instead.

PS Seriously dude, no more posts here, and don't bother PMing me either.  I have no interest in any more dealings with you.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

So yeah, pressing on, time for an update with regards to using an appropriate LogC to Rec709 LUT.  Resolve 12 Beta launched with a Rec709 Gamma 2.4 option only, as opposed to having just Rec709 as well.  Anyway, they corrected that omission in Beta 4.  The following two images display the settings required for a Gamma and Gamut conversion LUT, and a Gamma only one.








Try them out on ML Raw footage in Arri LogC colorspace in Resolve.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5