Reverse Engineering Picture Styles

[DeafEyeJedi]

Note that Cinestyle was released in 2011 and there was no further development since. Meanwhile technology in this area has progressed by leaps and bounds. Well, you know what I mean--we should be able to create a better log profile for our cameras. Thing is, Cinestyle is closed source, locked up and somebody seems to have swallowed the key.

Well said, Dan and yes I also agree w the fact that we should all come up w some kind of an original ML Log for our Cameras and perhaps as well our own set of ML Color LUT's to apply them on top together in post ... Man imagine what that would be like for us?

[eyeland]

@Dfort
I might have gotten slightly confused as it was late night when I posted
I understand the difference between "flatting" the image in-camera vs ACR (or similar).
I still use h264 with cinestyle on occasion as I have been doing some super-low-budget stuff for friends recently.
To word my question differently, is there a better S-curve lut for cinestyle than the low resolution one mentioned by Andy?
Or perhaps also a better flat picture style all together? (I know that most of the effort concerning h264 died out after ML raw appeared, but I suspect that many people still use canons h264 regularly)

[dfort]

@eyeland

I would not say that we need a better flat Picture Style though we could use an open source log gamma curve Picture Style. Although a log gamma Picture Style will look flat, a flat Picture Style doesn't necessarily mean that it has a log gamma curve.

Although there has been some debate about this it does seem that Cinestyle was designed as a log Picture Style. Sure, there are users that prefer other picture styles because they are easier to grade. This also happens in professional environments where camera operators for news and sports programs are told to use a "Wide Dynamic Range Gamma" setting instead of a log setting because it is quicker to grade--if it gets graded at all before going to air.

As far as a better S-curve lut for Cinestyle, I believe most people aren't even using the lut originally provided by Technicolor and instead are creating their own custom settings. Andy600 also adds some noise in his setting and I agree that it helps smooth out the notorious 8-bit banding issue.

For the most part the latest DSLR's still don't feature recording video to raw, H.265 or even ProRes. H.264 is still very much alive. (Slightly off topic the newly announced Nikon D5 records up to 3 minutes of 8-bit H.264 video--wow. )

[eyeland]

@dfort Yea, that's more or less what I have been doing, guess I'll stick to tweaking curves as needed until I can convince my clients to go the extra mile with ML Raw

[markanini]

Is it possible to convert ICC to PF2?

[dfort]

Is it possible to convert ICC to PF2?

I can't find a published document on the Canon Picture Style file format so for now I'd say that isn't possible. I'd love to be proved wrong on this.

[dfort]

Updated first post with release announcement of log picture styles.

[Danne]

In addition to the posted picture styles above I would like to post a few comparison examples.

logNeutral


cinestyle


logStandard


logNeutral


cinestyle


logStandard


logNeutral


cinestyle

[ddelreal]

I will most definitely try this out on my 7D2. I record using the HDMI out to the Atomos Ninja 2 so this should be fun to experiment with. Thanks guys!

[dfort]

Danne and I have been doing a lot of testing with these and we think they are very close--almost perfect. If anyone can do better, please share.

CineStyle

logNeutral_v1.1

There is lots of information available from Canon, Sony, Panasonic, Arri, (any others?) about shooting log but the documentation for the original Technicolor CineStyle is very sparse.

Based on Technicolor’s usage and testing of its CineStyle™ Profile, we
recommend the following camera settings to optimize the image quality of your
Canon EOS camera:
Sharpness: 0
Contrast: -4
Saturation: -2
Color Tone: 0
ISO: a multiple of the camera’s native ISO
(i.e. a multiple of 100 or 160 depending on the camera)

CineStyle loads up with the recommended Sharpness, Contrast and Color Tone as defaults but the Saturation is set to 0. Since our first goal was to make as close a copy as possible these picture styles also load up the same way as the original. According to some users who have lots more experience that I do about these things turning down the Saturation may not be the best recommendation.

In addition, the shooting on multiples of ISO settings recommendation is rather vague. According to some users who tested their cameras there is a multiple that is so called "native" another that is pushed or gain up while another is pulled or gain down. Pushing increases noise, pulling reduces noise while the "native" setting has the most dynamic range. It seems that if you're using a log profile you're probably more interested in dynamic range.

Some professional cameras limit white balance choices when shooting log and technical specifications lists a limited range of recommended ISO settings. Take for example this chart from Arri to illustrate why setting the ISO to 800 on the Alexa is recommended:



Does this apply to our Canon DSLR's? I've got more questions than answers after starting this project.

[a1ex]

Take for example this chart from Arri to illustrate why setting the ISO to 800 on the Alexa is recommended:
[...]
Does this apply to our Canon DSLR's?

This chart appears to show an ISO-less sensor (not sure how to read it, but the increments at both top and bottom ends match the ISO increments, +/- 0.1 stops). Usually, with such a sensor you can just shoot at the lowest ISO and increase the brightness in post as you need - there is little reason to use a higher ISO, as the shadow details will not get significantly better (they will get a little better, but not much).

On high-end cameras, I remember reading ISO is just metadata; if this is the case, it really doesn't matter which one you use. Or maybe it affects the exposure indicators and that's it (just a guess, as I have zero experience with these cameras).

Canon sensor is different - if you look at a dynamic range chart, you will see that, until about ISO 400, the DR does not drop much. Remember ISO is defined from the clipping point. Therefore, increasing ISO helps capturing more shadow detail. However, at higher ISOs (around 1600), Canon sensor becomes nearly ISO-less - there's little to gain by pushing ISO beyond that.

So, on an ISO-less sensor, you can just lower the ISO to protect your highlights without worrying much about the shadows, while on Canon sensor (especially on full-frame), you need to find a balance between how many highlights you clip (when increasing the ISO) and how many shadow details you capture.

Side note: with dual ISO, Canon sensor behaves a lot like an ISO-less sensor (so, when the light changes a lot, I just set it to 100/1600 and forget about it).

[reddeercity]

I found the FAQ for Technicolor CineStyle cinestyle-faq
So as far as ISO is concerned it's recommended to use ISO of multiple of 160,  so that would be a push @ 100  but at 360 is pulled down 400 .
So digital ISO's have more D.R. then Analog ISO's  e.g. 100,200 etc..... . in the Compressed internal pipe line ?
But in Raw Video this is not the case as we know that the Analog ISO 100,200,400 etc....  have more D.R. & cleaner a much lower noise floor.
We know the Video stream for HDMI & H264 comes from the Jpeg chip , which I understand is YUV full range 0-255 .
The reason I mention it because I notice with the log picture style  in the PSE (Picture Style Editor) I see the range is clipped at 16-235 ,
not sure if that will cause any limited D.R. Just a thought.

I also in may search for info on this thread topic I came across this alternative Cinestyle Look up tables    (January 03, 2014) on the ML forum ,
 look like you're not the only one @dfort that's looking for a Cinestyle workflow for compressed video.
I downloaded the lut pack , the image below shows all the formats (ICC , Cube , etc... )

says he or she took Cinestyle .mga and converted to Cube , ICC etc.. also say "Please keep in mind, that I take no responsibility for 100% correct Look up tables".
Hope there some useful information in there to work with .

[Danne]

Creating a simple 3D lut upon the S-curve isn,t hard. Is there an inverse somewhere in cube format? Would it even work. People complaining the S-curve being too simple.

[reddeercity]

From my understanding the .mga file is from Technicolor and they use a s-curve to transform back to Rec709.
So it may be that easy , don't know unless you try.
I'm still trying to found more information , on all this thou.

[Danne]

Could you inverse the S-curve numbers for me Reddeercity? My mathbrain is not working.

[ph2007]

is there anyone tried the clog someone posted here not long ago?

just got my ass out yesterday to test the clog on 6D,
its very close to cinestyle, just slightly boost on saturation and slightly underexpose then cinestyle.
clog:

cinestyle:


(grade just apply default premiere Alexa_Default_LogC2Rec709 lut)
clog graded:

cinestyle graded::


clog:

cinestyle:


(grade just apply default premiere Alexa_Default_LogC2Rec709 lut)
clog graded:

cinestyle graded::

[Andy600]

@Danne - here you go: https://s3-us-west-2.amazonaws.com/ofxpublic/Cinestyle_S-curve_1D_fwd_and_rev.zip

I've got the s-curve function somewhere that can reproduce the curve exactly but I need to find it (on an old drive).

Cinestyle is not the inverse of the S-curve though the inverse could be used to create a flat PS by multiplying the float values by 256 - though you only get 10 control points in the PSE.

Great work with your PS. It looks bang-on in DPP on my monitor. I'll have a look on a step wedge to see if the shadows are holding up - that's where things can get tricky but it looks very good.

BTW I don't think .pf3 is compatible with older cameras (50D, 5D2 etc)

[Danne]

Thanks Andy600. It,s been quite the challenge to obtain closest match especially in shadows as you mention. Dfort had test charts to verify a lot of the ground work. For subtle luma/color changes I had to create some extreme lighting conditions, bright lamps etc. I think it turned out alright. Of course with a 10 point curve tool moving only one step will often alter the overall look so getting the last subtle matching points was a lot of work. I welcome any closer inspections and refinements possible on the subject. Thanks for the 1D luts. Will check those later.
I bet dfort will put up pf2 versions soon . Meanhwile it,s possible to open up the pf3 and resave them to pf2 in picture style editor.

[dfort]

Wow, this topic came back to life. Ironically we introduced the new picture styles on the three year anniversary when the Technicolor CineStyle Facebook page was last updated.

One of the reasons why I got back to this project was the announcement of the 5D Mark IV. Although some people are underwhelmed by the specs it is still destined to become a popular camera. If or when Magic Lantern will be ported to it is something we won't even discuss at this point but having a log picture style that can be tweaked for 4K mjpeg video is certainly within our reach. Note that early reviewers have reported that C-Log is not available for the 5D4. Slightly off topic the 1.74x crop factor when shooting 4K is close enough to the Super 35 sensor used in the C100, C300, C500 that 5D4 users might consider doing hardware hacks on EF-S lenses like the popular Canon EF-S 17-55mm f/2.8 IS USM Lens.

Ok, back on topic. I'll use .pf2 for the next release so we don't leave out owners of some older camera models. The files are much smaller and we aren't using any of the advanced .pf3 features anyway. First I want to revisit the highlights. The 1.1 release improved on the shadows but it looks like 1.0 might have been a slightly better match to CineStyle in the highlights. These are minute details that can't be seen by eye but show up on a waveform monitor.

@ph2007 - We looked at that new C-Log picture style that @Chungdha posted. We would certainly like to know more about it. It is locked for editing and he hasn't answered questions that users posted so we don't know if he wants to play with us.

@a1ex - Thanks for the technical insights. You gave us a lot to think about.

@reddeercity @ddelreal - It will be interesting to see if we can eventually make a log picture style that is optimized for external recorders off the HDMI output.

And most of all--thanks @Danne, you did all the hard work on this.

[baldavenger]

This chart appears to show an ISO-less sensor (not sure how to read it, but the increments at both top and bottom ends match the ISO increments, +/- 0.1 stops). Usually, with such a sensor you can just shoot at the lowest ISO and increase the brightness in post as you need - there is little reason to use a higher ISO, as the shadow details will not get significantly better (they will get a little better, but not much).

On high-end cameras, I remember reading ISO is just metadata; if this is the case, it really doesn't matter which one you use. Or maybe it affects the exposure indicators and that's it (just a guess, as I have zero experience with these cameras).

Canon sensor is different - if you look at a dynamic range chart, you will see that, until about ISO 400, the DR does not drop much. Remember ISO is defined from the clipping point. Therefore, increasing ISO helps capturing more shadow detail. However, at higher ISOs (around 1600), Canon sensor becomes nearly ISO-less - there's little to gain by pushing ISO beyond that.

So, on an ISO-less sensor, you can just lower the ISO to protect your highlights without worrying much about the shadows, while on Canon sensor (especially on full-frame), you need to find a balance between how many highlights you clip (when increasing the ISO) and how many shadow details you capture.

Side note: with dual ISO, Canon sensor behaves a lot like an ISO-less sensor (so, when the light changes a lot, I just set it to 100/1600 and forget about it).

I believe the link between high ISO and extra highlight latitude is based on choice of exposure for a given scene. It's certainly not intuitive (especially from a stills perspective) but there is a logic there. In the previously shown chart for Alexa, the premise is that the scene exposure level is the same for every ISO rating, and every change in ISO is offset by changing the amount of light that hits the sensor (via aperture, shutter speed, shutter angle, ND filters, external light levels, etc.). Therefore, to match the correct exposure level with a low ISO you need a lot of light entering the camera which is great for shadows but also means the sensor hits saturation sooner, therefore highlight latitude is reduced. On the other hand, exposing with a high ISO means less actual light entering the camera, so less latitude in the shadows but more in the highlights. This is also based on a Raw workflow, where the ISO is indeed just metadata that can be adhered to, altered, or removed in post. 800 ISO is the base for Alexa for practical reasons.

Dual-ISO is in deed quite similar. Shooting 100/1600 and exposing for 1600 (to gain an extra few stops highlight latitude in post) is much like a DP choosing an ISO of 1600 (with a 1 stop ND filter attached to the lens) in order to gain an extra stop of highlight latitude. 

[dfort]

I bet dfort will put up pf2 versions soon . Meanhwile it,s possible to open up the pf3 and resave them to pf2 in picture style editor.

So saving the pf3 as a pf2 will show this warning:



And poof! There goes our tone curve. I'm going to move the nodes over to the tone curve that's on the "Specific Colors" tab in order to save a pf2 version. Let's see if that doesn't mess up all the work we've done on our curve.

[Andy600]

Moving to the pf2 curve will likely produce a different result than you get with the pf3 tone curve. The pf2 curve is applied after white balance and will probably upset the color balance. Canon could easily enable pf3 support in older cameras if they wanted to 


Edit: Just tried a pf3 in a 50D and the tone curve works - so it should work in the 5D2 too

[dfort]

Edit: Just tried a pf3 in a 50D and the tone curve works - so it should work in the 5D2 too

Great because copying the nodes from the Tone curve (RGB) didn't work. Using the pf2 curve was like starting over, only more difficult.

@Andy600 -- A while back you made a comment about the -2 saturation setting recommended by Technicolor:

I'm personally a little skeptical of the recommended saturation setting as desaturating before encoding could actually be losing much color information. If the -2 setting is to emulate the appearance of a wider gamut it's not really a good idea because pixels with RGB Values that already have close to no color will become grey and this cannot be recovered.

I compared saturation settings 0, -1 and -2 and applied the Technicolor CineStyle lut that ships with DaVinci Resolve. Looking at the RGB Parade it seems to me that the -2 saturation is a much closer match to a reference shot with the Neutral picture style. Perhaps the -2 setting accounts for the increased saturation when the gamma is adjusted in post?

I was thinking of saving the default -2 saturation setting on the next version.

The reason I'm bringing this up is because I've often loaded up CineStyle on a new camera and have forgotten to set the saturation to -2. The CineStyle picture style defaults to 0. Strange, because you would think that the recommended settings would be saved as the default. When I throw on a lut for a rough idea of what I shot it often seems a bit too saturated on footage shot on the 0 setting.

[Edit] New release, v1.2. Managed to combine the highlights from v1.0 with the shadows of v1.1 to make an even better match. Saturation now defaults to -2 so users won't have to remember to change that setting if they want to use Technicolor's recommendations. Note that Sharpness 0, Contrast -4, Saturation -2 and Color tone 0 is also the way to adjust the Neutral picture style to Prolost Flat, a style that has been around about as long as CineStyle.

[reddeercity]

This was posted 11 hours ago on Instagram by Phillip Bloom very interesting James Miller C-logV1.2
Here's James Miller's Instagram . Can't find it anywhere yet , I wonder if that C-log Pic Style
is from him? I'm trying to get my hands on it .

[Danne]

Great work on this one dfort. Not sure if I would dial down the saturation though since technicolor doesn,t even if they recommend it. It,s 8bit color. We cannot afford it. Better to loose it in post?
I still have one CR2 which shows a slight, slight shift so I guess we have one more version to try and achieve. But it,s the closest I,ve seen.

Comprison done with saturation put back to original state

cinestyle


v.1.2


v.1.1