Author Topic: DaVinci Resolve and ML Raw  (Read 80943 times)

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #50 on: December 02, 2015, 03:30:36 AM »
I tweaked the method of compiling 1D LUTs. This is how you do it.

The free/learning version of Nuke does the job, so get hold of any version from the last few years. Only one node is required, so you don't have to get bogged down with Nuke's intimidating infrastructure. Once launched, select a ColorLookup node.



Select Red, Green, and Blue from the list on the left.



Right-click in the middle, and select Edit > Edit Expression



The following little table pops up.



Paste an expression into each channel space. Here I pasted the Cineon Log to Linear expression, the result on the right representing the output value when the input value is 1.



This is the resulting curve.



Next, right-click the middle and select File > Export Ascii



Another little table pops up. The default increment will be 1/255, but that's only 8bit, so to get 14bit enter 16383 (because 0 is also counted as a value, so 16384 is still achieved)



When exporting values, the very last one is omitted for some reason, so change End at: to 1.00001 and the last value gets included.



This is the first set of values of the exported list.  Note the first value.



This is the last set of values.  Note the final value, and also note that the list goes right up to 16384. Full 14bit.



Press the node's reset button, then repeat the initial procedure, this time entering the inverse expression (Linear to Cineon Log).



A little extra work is required for the next step. The Start at and End at values correspond with the first and last values from the previous exported list. The increment value of 1 is replaced by the full extent of the previous export expansion. The 16383 value from before is repeated.



As was the case before, the End at value needs to be extended by a token amount to include the final value. Here I rounded up to 4 decimal places.



Here's the first set of values from the exported list. Note how the 1st value isn't quite 0, but the difference is minuscule and that value can be changed to 0.0000000000 in a text editor and saved as such.



Here are the last few values. Pretty spot on, without any need to clamp values.



To convert the lists into 1D LUTs that Resolve will read, simply copy and paste the headers from the appropriate 1D LUT in their VFX IO folder. The only things changed (when required) are the Size and Range values.



Here (for the Cineon Log to Linear LUT) the size was changed to 16384, while the range values remain the same (0,1).



Save the LUT as a .cube file.



For the Linear to Cineon Log LUT, the Size becomes 16384 as before, but the Range becomes the first and last values from the preceding LUT.



Save as a .cube file, and you have a matching pair of 14bit Cineon LUTs. The process can be repeated with any of the expressions for the other transfer functions. You can find the expressions for the various Log to Linear transfers by pressing S in the Properties pane, selecting one of the gammas in the LUT section, right-clicking, and selecting Edit>Edit Expression.



Here's a list of the more popular expressions.


Cineon Log to Linear:

(pow(10,(1023*x-685)/300)-.0108)/(1-.0108)

Linear to Cineon Log:

(((log10(x*(1-.0108)+.0108))*300)+685)/1023

Arri LogC to Linear:

x > 0.1496582 ? (pow(10.0, (x - 0.385537) / 0.2471896) - 0.052272) / 5.555556 : (x - 0.092809) / 5.367655

Linear to Arri LogC:

x > 0.010591 ? 0.247190 * log10(5.555556 * x + 0.052272) + 0.385537 : 5.367655 * x + 0.092809

sRGB to Linear:

x < .04045 ? x/12.92 : pow((x+.055)/1.055, 2.4)

Linear to sRGB:

x < .0031308 ? x*12.92 : (1.055*pow(x,1/2.4)) -0.055

rec709 to Linear:

x < .081 ? x/4.5 : pow((x+.099)/1.099, 1/.45)

Linear to rec709:

x <= 0.018 ? (x * 4.5) : 1.099 * pow( x, (0.45) ) - 0.099

Gamma 2.4 to Linear:

pow(x,2.4)

Linear to Gamma 2.4:

pow(x,1/2.4)



I'll follow this up with a look at how to read both 1D and 3D LUTs, so you can tell what they're about just by looking at them in a text editor.


EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #51 on: December 04, 2015, 05:11:20 AM »
I put together an Excel spreadsheet that compiles 14bit Cineon and Arri LogC LUTs. It can be opened in Google Sheets, and also adapted for other transfer curves and bit depths.

https://www.dropbox.com/s/yw69icg9wunj625/Log2Lin2Log%2014bit.zip?dl=0

It was a useful exercise to learn how to compile a LUT at the most basic level.  Its utility can only be further enhanced by demystifying what it is and what it can do. It's a look up table at the end of the day, with a list of input values and corresponding output values. No magic shazam voodoo. Very useful when properly used though.

Since this is a DaVinci Resolve related thread we'll have a look at the 1D and 3D LUTs that are specific to that application, but in doing so the basics of all variety of LUT will be covered. I spent a good part of this year trying to get my head around how to properly interpret the values presented when viewing the LUT in a text editor. I scoured the internet, but the necessary specifics were hard to come by. Anyway, eventually things started to click, first with 1D LUT structure, and then finally 3D.

Starting with 1D LUT structure, open up a null/unity 1D Resolve LUT in a text editor that has an accompanying numerical list.



This LUT has output values identical to its input values.  Some LUTs display the input values, but Resolve LUTs do not. It calculates the input values from the header. The Range (by default 0 to 1 in floating point values) is divided equally by the Size number of points (in this case 1024), and the three columns in the LUT represent the corresponding value for the Red, Green, and Blue channel. Corresponding points can be identified by using the numerical list.

The default Range of 0 to 1 is standard for all LUTs, and is the reason for clipping when it occurs. A LUT will only interpret values within that range (anything below 0 becomes 0 and anything above 1 becomes 1), but has no limit to the range of what it can output. If a Log to Lin transform is applied to a signal, the compressed Log (with values contained within the range of 0 and 1) can be expanded to values as high as 13.5 (Cineon) or even 55.1 (Arri LogC), and a LUT applied to afterwards will clip all values over 1. Everything over 1 will be superwhites, but it's still relevant information.  Resolve LUTs can stipulate the Input Range, so the extra information can be brought back into the image stream. No more clipping if you know what kind of signal to expect. An issue that can arise from this though is quantisation. An Arri LogC to Linear LUT expands the compressed log signal from 0 to 1 range to -0.017 to 55.1, and the reverse LUT divides around 55 by its Size as opposed to just 1. The number of points allocated to remapping 0 to 90% scene reflectance (100 IRE) is only about 74 even with a 12bit (4096) LUT. Because Resolve has to interpolate values in between the points, the fewer there are the more likely there will be inaccuracies incurred. The 1D Shaper LUTs in Resolve only go up to 12bit, but the 14bit LUTs here work and can make a big difference. 4 times as many points, which is especially useful in the LogC scenario.

3D LUTS took a while longer to figure out, but the same approach to identifying the input values apply.  Open a 3D null/unity LUT in the test editor.



Whereas 1D LUTs can only affect channels separately from one another, 3D LUTs are more convoluted. The maximum number of output values a 1D LUT can produce is the LUT size e.g. 1048 (10bit) times the number of channels, so typically 3 X the number so 3144 in this case. 3D LUTS can produce far more values, as the LUT size is multiplied by itself the number of channels there are, so typically it is cubed.  10bit cubed is 1048 X 1048 X 1048, so 1,151,022,592 possible output values. In order to make the LUT smaller and more functional, the LUT size is greatly reduced, but even a bare minimum 17 X 17 X 17 LUT produces 4913 output values. The null LUT reveals how the reduced size is managed, and how the input values are ordered. They are ordered in cycles of 17 points, with the left column (red channel) consistently ordered by 17 increments from 0 to 1, with the other two columns (green and blue) holding the same incremental value per cycle, until every 3 value permutation is accounted for. There're not as easy (i.e. linear) to read as 1D LUTs, but having a null LUT to compare helps.

With regards to Magic Lantern Raw files, the two main log considerations are Cineon and Arri LogC. Log can definitely be useful sometimes. It's purpose is to effectively squeeze a higher dynamic range signal into a 0 to 1 range. The 5D MkIII can achieve up to 11 stops DN without Dual-ISO, so Cineon is best suited to squeeze the signal into a display referred scene.  You can of course use Raw controls to pull the signal into place, but what you're doing there is basically imitating the log encode process, but with more contrast.  Arri LogC involves even more signal compression, and is probably overkill for regular footage, but actually works great with Dual-ISO especially when pushing for up to 14 stops of DN.  I did some tests in Brazil earlier in the year where I pushed the camera to see what I could get. I shot some sun rises by the sea, so had 100/1600 ISO going. Then the footage was processed and brought into Resolve, and only LogC was able to contain the signal without pulling down highlights in Raw controls.  I was able to see the whole signal on the monitor and the scopes straight away, so I knew what I had to work with. Anyway, it's all about the right tool for the job.

I had to do some YUV related research recently, and besides attaining a body of knowledge that perfectly equips me to get a job as a TV engineer in 1986, I learned a few new possibly handy creative grading tricks. I'll post something related to that in the next few days.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: DaVinci Resolve 12 and ML Raw
« Reply #52 on: December 05, 2015, 02:08:46 AM »
Love this thread!

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #53 on: December 14, 2015, 03:09:48 AM »
I stuck with the spreadsheet learning thing a bit longer than planned, but it's proven to be worthwhile. I built a new one (posted below) that deals specifically with Log C, providing the means to build 14bit 1D and 65x65x65 3D LUTS. The main issue with Log C is that when it's linearised the signal has a huge range, and that doesn't work well with LUTs. I've tried various methods of managing the signal without incurring errors such as normalising the linear signal, but that effectively scales 18% reflectance to 0.2 divided by @55.1, which presents another challenge altogether.

Anyway, one of the better methods I discovered was splitting the signal. It involves two layer nodes after Log C to Lin is performed, with a 3D LUT in each.  One LUT deals with values up to 1 and clips all above that, while the other deals with values greater 1 and clips all below it. The two separate signals can then be added back together.



This method has a lot of potential, and works for any high dynamic range signal and not just Arri.

https://www.dropbox.com/s/1tiiue3n1pbzj73/Arri%20LUTs%20Spreadsheet.xlsx?dl=0


I've been meaning to post something about dealing with FilmConvert and Koji, with respects to using their Arri Log C functions. I was working on converting them earlier in the summer, but without much joy. I believe it's an issue with latitude. If you try grading some actual Arri footage in Resolve you'll find that you can push it around quite vigorously without it falling apart. It means that in order to affect contrast you need quite a strong S curve, and that's what the Koji LUTs and FilmConvert do. You can still achieve the same results though, just not via the direct route. It's very tempting to always go for the Arri named option, as it seems classy and you'd assume it's top quality, but it's not always the most sensible choice. It's a mistake I've made a lot.

You can try for yourself with some footage from here:

https://www.arri.com/EN/camera/alexa/learn/alexa_sample_footage/


If anyone has any questions or would like to comment then please do. I still haven't gotten round to looking at the YUV option, but hope to soon.

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5710
Re: DaVinci Resolve 12 and ML Raw
« Reply #54 on: December 14, 2015, 09:52:14 PM »
Hi!
Trying some of this split signal workflow and I get somethin like this.





According to instructions
Quote
selecting Arri Log C for Timeline and Bypass for Output in project settings so that you get a LogC signal and Alexa Wide Gamut colour space.
Not sure how to bypass for Output project. I apply Arri_Lin_to_LogC_1D_14bit.cube in the timeline.

Quote
In the first node apply the LogC_to_Arri_Lin_1D_14bit LUT, which decodes the LogC signal into its full range of -0.017 to 55.08. Add a Serial node, then add a Layer node.
Done

Quote
In the top Layer (underlying) node add the Arri_Lin_Split_Normal_AWG_to_Rec709_3D_65 LUT. This LUT converts the signal colour space to Rec709, and clips all values over 1.

done

Quote
In the bottom Layer (overlying) node add the Arri_Lin_Split_Superwhites_AWG_to_Rec709_3D_65 LUT.  This also converts the signal colour space to Rec709, but clips all values under 1 and offsets every output value by minus 1. In the composite node select Add, and the signal is complete again.

After this all turns very bright, hmm, I,m not getting this right.

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #55 on: December 14, 2015, 10:45:51 PM »
Are you using resolve colour management in project settings?

Here are the LUTs if others want a go:

https://www.dropbox.com/s/1vpyltthhc18h60/Arri%20LUTs.zip?dl=0
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5710
Re: DaVinci Resolve 12 and ML Raw
« Reply #56 on: December 14, 2015, 10:47:01 PM »
aha, that most be the culprit. I,m still on DR version 11. Time to update stuff.

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #57 on: December 14, 2015, 10:52:18 PM »
The signal splitting can still be achieved, but I do recommend the update. The layer mixer nodes should be applied to a linear signal with full superblacks and superwhites.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3200
  • 5D3 / M1 / 7D / 70D / SL1 / M2
Re: DaVinci Resolve 12 and ML Raw
« Reply #58 on: December 15, 2015, 08:17:16 AM »
Definitely worth the update to DR12, @Danne. Heck you even got me into it, right? You know you did, Mate! lol

Thanks for sharing @baldavenger as always and will continue to follow this thread even tho I'm still kind stranded with work.
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101 • EOSM2.*

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5710
Re: DaVinci Resolve 12 and ML Raw
« Reply #59 on: December 16, 2015, 12:07:05 AM »
Here is a file (dual_iso) that I used in my previous post.
https://drive.google.com/file/d/0B4tCJMlOYfirYlhyTFBCMnBCMjQ/view?usp=sharing

Havn,t upgraded to DR 12 yet. I tried out using Baldands Arri-luts in MLP by converting them to 3D luts in DR 11. Looks really good in MLP but I,m sure there are hickups using 1D logs as 3D luts. Nevertheless.

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #60 on: December 16, 2015, 02:55:28 AM »
That Dual-ISO image is a great example of both the ability of ML Raw and the difficulty of fully availing of all the extra information on offer. I couldn't find the actual ISO values from exiftool, but I assume given the circumstances it was probably 100/1600. I brought it into Resolve 12 and picked Arri LogC for the Timeline colour space. The image appeared dark as it visually more represented the 100 ISO setting, so in Camera Raw settings I increased exposure to 4.0 (thereby scaling the signal by 4 stops, i.e. to 1600 ISO).

I left dealing with the Alexa Wide Gamut to Rec709 colour space transform until later, as first it was important to sort out the transfer curve. I built a LogC to sRGB 1D LUT with a softclip on both the superblacks and superwhites, but even though the superwhite softclip mapped the 55.08 to around 6 via a 2.4 gamma, it still proved difficult to pull the superwhites into range (Resolve's softclip function had little effect).

So I built a new LUT with a gamma 10 for the superwhites instead, resulting in a much stronger curve and a peak at around 1.5.  This allowed for Resolve's softclip function to pull the extreme highlights into range.

This is the expression I applied to the post LogC to Lin values in order to build the LUT:

=IF(D1<0,POWER(ABS(D1),2.4)*-1,IF(D1>1,POWER(D1,1/10),IF(D1<0.0031308,D1*12.92,(1.055*POWER(D1,1/2.4))-0.055)))

I've included the original sRGB LUT and the Gamma 10 version, plus two gamut 3D LUTs. Although not ideal as they're for log footage and not linear, they should be applied to a node before applying the LogC to sRGB 1D LUT. One is Rec709, while the other is LC709 which is a more muted version and believed to be more akin to the Alexa saturation response.

https://www.dropbox.com/s/mosgqwhos4rvp1h/Kitchen%20Dual-ISO%20LUTs.zip?dl=0

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #61 on: December 17, 2015, 01:23:09 AM »
About a month ago and with the help of Andy600 I compiled a set of dark frames for my 5D MkIII. Hopefully a more expansive set will be compiled in the not too distant future that includes most, if not all, ML capable cameras.

https://www.dropbox.com/s/998ck7qpbs9fmi4/5D3%201920x1080%2024FPS%20Dark%20Frames.zip?dl=0

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5710
Re: DaVinci Resolve 12 and ML Raw
« Reply #62 on: December 18, 2015, 01:01:36 PM »
I,ve been working on getting some good looks and luts with MLP found here
http://www.magiclantern.fm/forum/index.php?topic=13512.msg130562#msg130562

Thanks to this thread I started experimenting with Ben Turleys online lut calculator, with various results. I tried to create a log and a REC709 to go with that. I am not completetly satisfied with the results. It is rather hard to obtain luts that will bring out most information but also be correct in brightness etc. footage sometimes gets noisy or too dark for example.
In my testing frenzy I noticed I could get really nice files by adding the Linear_to_Rec.709.cube form VFX folder in Davinci resolve straight to the footage output in MLP.
It would be useful with feedback from the standpoint of MLP. MLP works by letting dcraw pipe dng files through FFmpeg which creates prores4444 10-bit files. My conclusion is that dcraw is to be set to -H 2(highlight mode) and gamma to 2.4 0 which brings the output towards linear and brings out the shadows. Through FFmpeg I can add 3D luts to create looks(unfortunately 1D looks can,t be used).
My intent is to be able to create log and rec709 and to understand what kind of color space to use. In this case they have to be 3D luts. I think the online lut calculator could do the trick? My thinking is that I use the png files created with the setting dcraw -H2 -gamma_2.4_0(2nd picture on the test charts) and bring that loggish picture to the online lut calculator and go from there. Or is it better to leave the gamma and work with the dcraw -H2 png files instead which would be more REC709/linear kind of output? Could use some feedback.
Below are examples of what can be done in MLP. The three pictures has the following settings:

1 - Regular gamma and dcraw highlight mode set to -H 2(lowest setting but lowers brightness considerably)
2 - Gamma pushed to 2.4 0 dcraw highlight mode -H 2. This is the base output I intend to start with in the lut calculator
3 - Regular gamma, dcraw -H 2 and a Linear_to_Rec.709.cube coming from the VFX IO folder in DaVinci resolve converted to a 3D lut.

I,d really like to get some tip here to build some optimized log luts and some REC709 and whatevere else could be used here.

All files and original dng files can be downloaded here
https://drive.google.com/file/d/0B4tCJMlOYfirdTFDQllYdkpqc1U/view?usp=sharing











baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #63 on: December 18, 2015, 08:06:51 PM »
When you say regular gamma, do you mean gamma 1.0 (linear)? And what does dcraw -H2 actually do to the signal? Does it scale it down in a linear way, or apply a softclip to pull the highlights down without affecting the midtones and shadows?

Any gamma that isn't linear (1.0) is technically log, so if dcraw can apply a 2.4 gamma expression, i.e. pow(x,1/2.4), to the linear signal then it should be able to apply a cineon or logC expression. Any values outside the range 0.0 to 1.0 are probably clipped when a LUT is applied, so best to have the whole signal contained before that happens.  A 2.4 gamma can give the impression of a flat, log image, but it doesn't remap superblacks and superwhites.

The VFX IO Linear to Rec709 1D LUT is meant for a limited signal (0.0 to 1.0) with gamma 1.0, and the output values are contained to the same limited signal. It doesn't account for superblacks or superwhites. Pulling the highlights down to squeeze them into the limited signal prevents clipping, but the LUT expects the value 1.0 to be 90% reflectance, so although the image might look ok but it is mathematically unbalanced.

If you can devise a pipeline where you have initial access to full scene linear data (16384 14bit?), you can encode the values so they fit into the 0.0 to 1.0 range, and then apply a 3D LUT for final output.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5710
Re: DaVinci Resolve 12 and ML Raw
« Reply #64 on: December 18, 2015, 08:35:35 PM »
Actually dcraw pipes out 16-bit files to FFmpeg.

Here are two more comparisons. Settings, only Linear in dcraw and the other Linear and H2(rebuilds highlights and darken the overall picture)

I really like the rebuild highlight setting so could this be kept?

Full png files here
https://drive.google.com/file/d/0B4tCJMlOYfirWmZCa011VU43c1E/view?usp=sharing




baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #65 on: December 18, 2015, 08:46:05 PM »
I'm doing a quick read over dcraw commands to better understand things. The H2 command seemingly softclips superwhites, but in this case I would recommend it. Any Linear to sRGB/Rec709/etc. is easy to achieve then.  Convert the VXO IO LUTs to 3D LUTs, or build them with a spreadsheet.

It might be possible to devise a log command (looking into it now), along with a custom colorspace such as Alexa Wide Gamut.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5710
Re: DaVinci Resolve 12 and ML Raw
« Reply #66 on: December 18, 2015, 09:06:27 PM »
Sounds great. Exciting stuff. I actually converted the vfx io already (tip from Andy600) and when I use linear dcraw setting they are way dark but without linear and with H2 results looks good. I have a hard time to find sweet spot.
I ll check back in a couple of hours. Busy atm.

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #67 on: December 18, 2015, 10:27:07 PM »
Though still possible, perhaps encoding to a log signal and wider gamut is overkill for what you want. If what's required is a ready to go output and an interim format, then it might be best to combine both into one practical format.

You can use dcraw to get sRGB and Rec709 gamma without using a LUT, and that would be more precise than using a 3D LUT.
I found this while searching the net:

g power toe_slope

Set the gamma curve, by default BT.709 (-g 2.222 4.5). If you prefer sRGB gamma, use -g 2.4 12.92. For a simple power curve, set the toe slope to zero.

With the -H 2 command you can softclip superwhites, so that provides for a decent output format. It also makes for a good interim format too, as any 3D LUT applied doesn't have to deal with gamma transform and can focus just on colour.  This is where LUTCalc comes in very handy. Use sRGB/Rec709 for Input Gamut and Gamma and whatever Output you're after, including Film Emulation Looks. How to put together such LUTs is dealt with earlier in the thread.

Linear will always look dark, because it is how the camera sees light and how light is in the real world. Gamma correction such as sRGB, Rec709, 2.2,2.4, etc., is a means of making a signal more 'realistic' to human perception, though what appears to be real to us is in fact somewhat of an illusion.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5710
Re: DaVinci Resolve 12 and ML Raw
« Reply #68 on: December 18, 2015, 11:56:43 PM »
Thanks for info. It would be interesting to have the knowledge to do both.
A while ago I came to the conclusion to use the sRGB gamma -g 2.4 0 and go from there. I then created a log Alexa log lut
Code: [Select]
TITLE "Custom LUT"
LUT_3D_SIZE 33
# Rec709 - γ1.90 (exp2.22)/Alexa Wide Gamut -> LogC (Sup 2.x)/Alexa Wide Gamut, CineEI Shift 0.00, Black Level 8.03% IRE, Data Input -> Data Output - Created with LUTCalc v2.3.4 by Ben Turley November 2015


Canon log lut
Code: [Select]
TITLE "Custom LUT"
LUT_3D_SIZE 33
# Rec709 - γ1.90 (exp2.22)/Canon Cinema Gamut -> Canon C-Log2/Canon Cinema Gamut, CineEI Shift 0.00, Black Level 3.54% IRE, Data Input -> Data Output - Created with LUTCalc v2.3.4 by Ben Turley November 2015

Now I had to push the slope cause it came out to dark. I then created a complimentary REC709 to these logs. Now the question is if it,s better to skip the log and go right to the REC709?

Would Linear to REC709 be something to work with in lut calculator?

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #69 on: December 19, 2015, 02:17:26 AM »
I'd better post this and get it out of the way, and then I'll return to the MLP related stuff.

I did some tests with FilmConvert and Koji to see what might be the best approach with regards ML Raw. As was mentioned on here and in other threads, the Arri options are too aggressive with ML Raw footage, even when converted to LogC.

This is the FilmConvert Arri LogC curve:



And the Koji one:



The download below includes this list:



Of course, you can still get great images even if the LUT or setting doesn't exactly match the preferred parameters, but I find that if you can get the gamma aspect close from the get go then you're likely to have more control over the creative process. With FilmConvert I found that Resolve ML RAW BMD Film works best with log footage, whether its Cineon, LogC, or BMDFilm. It would appear that it expects a Rec709 gamut, and not the wide gamut of BMDFilm, but I got that by applying it to Alexa Wide Gamut footage and the image remained slightly desaturated. When I converted the footage to Rec709 gamut the results were more in line with what was to be expected. ML RAW BMD Film M1 is the same except for 1 stop exposure adjustment, i.e. if your footage was 1 stop underexposed you would add this instead. If you're caught between the two, pick the first one and use the Exposure control in Camera Raw to adjust. ML RAW BMD Film with Exposure turned to 1.0 = ML RAW BMD Film M1. In fact, unless the footage was very underexposed and needed all the help it could get, I would always go for regular ML RAW BMD Film and use only Exposure and White Balance to set the base correction in Camera Raw, and anything else would be done in the nodes.

With Koji LUTs, after unsuccessfully trying to convert the Arri ones to a Cineon equivalent in the past, I reckon only the Cineon to Cineon, or Rec709 to Rec709 are of any practical use. But that's just my opinion. When it comes to LUTs, it is better to err on the side of too little contrast, as that can be added afterwards, whereas as too much initial contrast presents a battle which is almost impossible to win. Think of them like shoes. Ideally you want one that fits perfectly straight away, but if it's too small then you're killing your foot trying to squeeze into it, whereas if the shoe is too big then at least you can wear more socks. Capiche?

Some other observations:

BMD Film gamut is different in regular DaVinci YRGB than it is in RCM. In regular DaVinci YRGB skin tones are richer and there's generally more saturation.

FilmConvert outputs a legal signal, that is to say a full range image with values from 0 IRE to 100 IRE will shrink somewhat after the plugin is applied.

Lift Gamma Gain controls operate AFTER the effect of the plugin.

If working with LogC Alexa Wide Gamut footage (or Cineon Alexa Wide Gamut footage) it is better to convert the gamut with a LUT before applying the plugin. Standard would be Rec709, but try LC709 instead as it results is less saturated highlights. Both LUTs are available in a previous post.

I'm sure there's plenty of diverse views on this subject matter, so please feel free to post what you think.

https://www.dropbox.com/s/2y6hqef4umhxbl1/Koji_FilmConvert_Curves.zip?dl=0

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #70 on: December 19, 2015, 02:36:30 AM »
Anyway, back to the MLP post. If you want to export footage in log by applying a 3D LUT at the last stage, then it would be better if it is applied to a linear signal. 33 points is not much and converting Rec709 2.2 to LogC involves a lot, and in theory would have to go via linear anyway. Better to give it linear and reduce the chances of error. It all depends what you want to export.

Both examples are a bit off as you're applying a Rec709 2.2 to LogC LUT to a Gamma 2.4 signal, so you won't technically get LogC. Bringing up the black level also means it's not technically LogC. LogC appears darker than other logs because it squeezes more a signal into a range of 0.0 to 1.0, so 90% reflectance is mapped lower, as is 18% reflectance, to fit all the signal in.

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #71 on: December 25, 2015, 12:21:42 AM »
The joy of YUV. This is very much an experimental/creative approach, but no harm in giving it a try. To keep things simpler we'll work in Cineon and Rec709 gamut. The examples are an sRGB chart and a ML Raw DNG, so in settings pick sRGB for Input Colorspace, Cineon Log for Timeline Colorspace, and Bypass for Output Colorspace. I've included 14bit Cineon LUTs (sRgb,Rec709,2.4) in the download, and used the Cineon to sRGB LUT in the examples. Here are the two images with just the LUT applied.





The first YUV approach involves changing the colorspace of the node (right-click it) to YUV and deselect the 2nd and 3rd channel. The image will retain its luminance but will appear desaturated. To restore saturation add a node and use the saturation control. The colour returns, though in a somewhat different manner.





Final approach is a repeat of the last one, only this time in the second node saturation is adjusted using YUV controls. Change the node colorspace to YUV and deselect the 1st channel. Then use the main curves interface to add saturation back. The nature of the colour is similar to before, but the curves allow more diverse control. More intuitive. LAB controls can be used to adjust saturation, but require very particular adjustments, whereas the YUV controls are more flexible. Also, the response tends to bias the mid-tones which is a more filmic approach.






https://www.dropbox.com/s/st3kaadh68wwl1i/YUV%20shenanigans.zip?dl=0


More options. Cool beans. Happy Festivus  :D

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #72 on: December 27, 2015, 07:56:29 PM »
RCM is great and all, but what if you could build a custom emulation with 1D LUTs? That would mean more control, plus there would then be the option of recreating RCM in previous versions of Resolve. I asked the senior colourist from Interstellar and The Hateful Eight if it is possible and he confirmed that it is, which is good enough for me.

It's an area I explored extensively earlier in the thread, but with my newly found ability to build custom LUTs via spreadsheet and expressions I can now build a speedier and more precise process. As a bonus development, the ability to re-map highlights while observing the process in the viewer and scopes. Really useful for when working with Dual-ISO footage.



The example above is from LogC footage (available from the Arri website). I applied a 1D 14bit LogC to Linear LUT in the first node (to convert the signal to scene linear), then split the signal in three. Each node uses the RGB Mixer to swap channels (with preserve luminance disabled) so that it's all Red channels in the top, Green in the second, and Blue in the third. Hence why the nodes are B/W. Using Layer nodes I added a custom 1D 8bit LUTs, and multiplied. The 1D 8bit LUTS convert the nodes to constants, representing column values from a 3x3 matrix. The resulting nodes are added back together, with the colour space converted from Alexa Wide Gamut to Rec709.




The final stage involves converting the scene linear signal to a display referred signal (such as sRGB, Rec709, or gamma 2.4). The problem with a scene linear signal from a high dynamic range camera is that the range is vast (up to @55.1 for Alexa) so even a 14bit amount of points is spread thinly. Splitting the signal again with two LUTS allows for much greater precision. The bonus is that by using the Highlight option in Resolve you can view and shape the highlights independent of the rest of the signal.










The following download includes a set of Arri Lin Split LUTs, plus the Alexa Wide Gamut to Rec709 Matrix LUTs.


https://www.dropbox.com/s/rbxd7xq8nyu3av7/Arri_Lin_Split_LUTs.zip?dl=0


EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: DaVinci Resolve 12 and ML Raw
« Reply #73 on: December 27, 2015, 08:20:55 PM »
Man this is so much info. All these different processes, which one to use! I'm getting a little lost ha ha. It would be awesome to make a simplified version of this whole thread.

baldavenger

  • Member
  • ***
  • Posts: 177
Re: DaVinci Resolve 12 and ML Raw
« Reply #74 on: December 28, 2015, 08:00:19 PM »
I'd say in terms of which process to use, choose the most suitable one for your needs. These aren't one size fits all type methods, they're more likely for very particular workflows plus this thread is an ever evolving experimental trial and error endeavour so a lot of the earlier entries have been rendered obsolete by newer and better approaches (or changes in the actual software).

If somebody wants to start a Resolve related thread and focus on a more beginner orientated approach then I think that's a great idea, and I'll be happy to contribute help and advice when requested plus everything here is freely available to reference or republish.

I'd prefer to continue to focus on developing new approaches though, so it's better if someone else got the new thread up and running. I highly recommend it to anyone considering doing so, as the whole process is a great way to learn new things and consolidate what you already know.

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5