Linear raw to percentage to IRE in resolve

Started by baldavenger, January 08, 2016, 09:18:40 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

baldavenger

Apologies for what is probably a very noobish enquiry, but what are the values associated with 18% and 90% scene reflectance? For instance, when viewing a dng in Resolve in linear space and 18% scene reflectance is 20 IRE on the waveform scope (and 90% is 100 IRE), which original 14 bit values are they derived from? If white level is 15000, how many stops above 90% reflectance is that?

Many thanks in advance to anyone who can help clarify this for me.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Audionut

There's a couple of different measurements you are trying to scale to there.  Linear raw count, % scene reflectance and IRE.

Log2(value/value) describes the EV difference between the two values.
Log2(100/18) (or simply Log2(18%)) = 2.47 EV.  In other words, 18% is 2.47 stops below camera saturation.
If we were using linear raw values, we could do something like log2(15000/2707) = 2.47 EV.

Linear count changes by two to the power of the EV change.
So 15000 * 2 to the power of -2.47(EV) = 2707.
In other words, a linear raw count of 2707 is 2.47 stops below a linear raw count of 15000.

baldavenger

So, if 15000 is the white level then 18% reflectance registers 2707 in linear raw value, therefore 2707 x 5 is registered by 90% reflectance (13,535).

Between 13,535 and 15,000 are superwhites, between 90% and 100% reflectance, and between 100 and 109 IRE.

In Resolve, in scene linear, 90% reflectance is at 100 IRE, and 1.0 in 32 bit float, so does that mean the highest value that can be registered from the raw values is 1.09? (Not including Dual-ISO) If I wanted to build a LUT that encodes the scene linear signal to log (or display referred) then the max input value would be no more than 1.09?

Thank you for the feedback. I'm still trying to get my head around some of the basics, so any advice and considered direction is greatly appreciated.
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

Audionut

IRE has legal limits of -20 IRE through 120 IRE, with the video signal constrained to 7.5 IRE through 100 IRE.
If you convert to an 8 bit digital signal using video levels (16-235), this allows signal slightly below 7.5 IRE and slightly above 100 IRE to fall into BTB and WTW.  But if you convert to an 8 bit digital signal using PC level (0-255), all values below 7.5 IRE and above 100 IRE should be scrubbed.

109, or 1.09 or whatever are not raw values.  If you software is showing results in IRE from a digital signal, it has already applied a conversion of the linear raw values to IRE.  This question needs to be taken to Resolve support or someone who knows how the signal is being converted.

In either case, I think you would have significantly more success in finding a solid brick wall and banging your head against it.  But best of luck.

baldavenger

Resolve support have a tendency to be quite selective about what information they reveal to those making enquiries, so I did some research of my own. I ran some tests on dngs that feature superwhites, and exported OpenEXRs so that I could view the channel values properly in Nuke. The viewer in Resolve is limited to 8bit integer (0-255), but by splitting the signal and offsetting by 1.0 you can view the superwhites (from 1.0 to 2.0) in the waveform scope. All dngs were debayered into scene linear, so in theory scene reflectance of 18% should register as 0.2 in scene linear and 20 IRE on the waveform scope (Resolve workspace is PC levels as far as I know), and 90% reflectance is 1.0 and 100 IRE.



This first image has highlight recovery off.



This shows the superwhites when highlight recovery is off.



Superwhites with highlight recovery on. They don't appear to exceed 2.0 but it's still a consideravbe difference.







In this case superwhites exceed 2.0 when highlight recovery is on.







Another example of a big difference.







And again.

















The examples above deal with a Dual-ISO image (courtesy of Danne). The first four have no exposure adjustment, and the remaining four have Exposure set to 4.0 in Camera Raw to bring the image from ISO 100 to 1600.

In Nuke I applied a Curve Tool node to ascertain the max luma values, and they generally corresponded with what the waveform in Resolve was suggesting. Without highlight recovery, superwhites rarely exceeded values of 1.3, but with it on some values exceeded 3.0. Whichever white balance was selected in Resolve also has an effect on the levels of both superwhies and superblacks. In the case of the Dual-ISO, the exposure adjusted image registered values up to 30.0 with As Shot selected as White Balance, but up to 42.0 with Tungsten.



So, my hope that definitive values can be derived for specific Min Input and Max Output for log encoding seems a bit unrealistic now. Highlight recovery has to be taken into account, but it seems unpredictable so best to play it safe with a conservative value of about 3.5 for standard dynamic and might as well stick with LogC for Dual-ISO as it covers a range up to @55.

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

baldavenger

I think my assumption that 90% scene reflectance is automatically mapped to the value 1.0 (32bit float) in Resolve is mistaken. Peak white (15000) would appear to be mapped instead, and more than likely represents values that exceed 90% scene reflectance. This also suggests that the superwhite values (above 1.0 in 32bit float) registered (without Highlight Recovery) are in fact the values between 15000 and 16383.

When a camera like the Alexa records to log it allocates values so that when LogC is linearised in software such as Resolve (or Nuke, Fusion, etc.) 18% scene reflectance is mapped to 0.2 and 90% scene reflectance is mapped to 1.0, and values above 90% are superwhites (up to @55.1 in float values). CinemaDNG seems to bypass this strict conformity to linearity, but would it be possible to somehow replicate it in Resolve?

In other words, is it possible to know what exact values are registered by the camera sensor when registering 18% reflectance, and 90% reflectance? Say, in an ideal set-up, an environment is light by 6500K bulbs, there is an 18% middle grey card, and a reference white card (90% reflectance), and the camera settings are adjusted (with the exception of ISO, which is set at 100) so that when the scene is metered (and observing the LCD viewer) those values are returned. When viewing that CR2 file in RawDigger (or other suitable app), what would be the values returned for those specific areas of the image?

Any help with this matter would be greatly appreciated.

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

DeafEyeJedi

Excellent questions,@baldavenger as I wonder the same since I've gotten myself into Resolve a lot recently and it's rather difficult to try and duplicate the values that you see in LCD viewer from cameras which seems to be easier to achieve in AE from my perspective.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

baldavenger

@DeafEyeJedi

Why do you think it's easier to achieve in AE? I'm not disputing that (still better algorithms in ACR than Resolve), I'm just wondering what AE does differently (so that it can be replicated in Resolve).
EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5

DeafEyeJedi

To be honest ... I'm still not even sure why I happen to think (or at least believe so) that AE does a better job in speeding up my workflow with spitting out dailies or even final products (quick turn arounds) despite the longer rendering times. Still a little faster in AE CC 2015 than AE 12.2 with ProRes4444XQ exports.

Maybe it's just cuz I'm more comfortable and well familiar with Adobe's products?

However, for super quick turnarounds, I have been getting somewhat "lucky" with Resolve as of late (even tho I'm not liking the grading process) in DR12's color grading with Logs/LUTS and whatnot. I use Cinelog-C for all of my non Dual-ISO/HDR work ... Also still using those Customized Koji LUTS in combination along w Cinelog-C which seems to work well for quick turnarounds as well.

Recently I've forced myself to try @reddeercity's suggestion to shoot in Landscape Picture Profile (w sharpening all the way down to 0) for best duplication the values of "Rec709" on LCD LiveView while shooting RAW/MLV without hesitation. Works very well for the most part.

If I happen to be unsure or in dismay of the highlights/shadows then I'll quickly switch over to CineStyle Picture Profile to read the histogram and then back prior to shooting. There isn't really any standard way of exposing but this tip from him definitely adds the icing on the Cake for everyday usage.

Back to the topic: I am still grasping on everything that you have been reporting on your other thread about color grading in DR12 with MLV/RAW so please excuse for my lack of learning curve on your never ending tutorials. Please keep them coming despite of my inability to understand parts of it.

It's literally like I'll have to reread the entire thread again just to understand a few of the most recent post. Not your fault though ... It's part of the trade off for free educational purposes within this community, right?

Thanks for all that you do @baldavenger!

*edit*

In other words I'm starting to realized that I'm in need to get more comfortable and familiarize myself some more in DR12 first in order to follow you along with ease if that makes sense.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Andy600

I'm trying to get my head around what you're trying to work out but I think your talking about scaling.

To begin with, the white level tag in a DNG is 100%. It's the maximum saturation point for RGB and assumed to be white. Values above this are clipped in Resolve, Nuke, ACR etc. The black level tag is 0% and values below this also clipped by the reader. The image remains linear until you assign or view the image in a colorspace (this happens automatically in most apps). If you alter either of these tags values you will likely get a colorcast as there will be missing channel information one or more channels. This is where ACR has the edge because, as the levels are pulled down below 1.0, values are 'borrowed' from other channels to fill in any missing information until the point where the maximum information of the most populated channel is reached (i.e. if red has the most information then green and blue will borrow from it but only by as much as red has actual information) - it doesn't create new information above 1.0 and cannot be replicated without a similar algorithm.

When it comes to Log and Scene Linear float values they can (in most cases - Sony SLog and Canon log are pre-scaled which complicates things a bit) be derived by inverting the log curve and scaling RGB so that recorded white values of a prefect white lambertian diffusor equals 1.0f (or depending on the bitdepth and metering equaling: 255 or 1023, 4095, 16383 etc etc) and assuming you exposed accurately. Values will extend above 1.0 but you need to use image analysis to find how much and then clip values to reduce quantization problems. Resolve's metering isn't really up to the job in this respect as it is display/output characterized.

re: IRE @Audionut has it right in his first couple of replies. I assume Resolve scopes are calibrated to YCbCr Rec709 as they match up to external scopes well. I'm not sure what would happen if you applied a rec709 to linear lut to the Resolve scopes but I doubt it would provide any more accuracy for the numbers you are trying to work out.

It might be easier to think of linear in terms of contrast ratio and read up on foot-lamberts, nits. You can then correlate this with how gamma, log encoding or other colorspaces affects image encoding and how, in turn, this is represented on Resolve's scopes. Ultimately it's probably a futile exercise when it comes to linear raw data because it's somewhat malleable. A similar exercise had me chasing my own tail for months.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

baldavenger

@DeafEyeJedi

Thank you as always for the feedback. Apologies for the messiness of the thread (and any excessive convolution, contradiction, and opaqueness), but I hope to start tidying it up soon so that it resembles something more practical for everyone (including myself). There are still elements of AE that would much improve Resolve (such as OCIO and ACR highlight algorithms), and I still use it myself when I'm on the road and need a more immediate and practical solution.

@Andy600

Scaling is probably the issue here, but I'm still not quite getting it. With regards clipping or peak white (15000), the display clips this (at 8bit value 255), but in 32bit float there would appear to be values beyond this (i.e.over 1.0). When I did the signal splitting test this was revealed, along with a visual representation of the effects of selecting Highlight Recovery. I attempted to replicate ACR's highlight rebuilding approach by manually adjusting the channels with RGB Mixer and using the Soft Clip function to bring the rebuilt signal into display range. A formula could probably be written and turned into an expression that performs a similar task as the ACR one, but perhaps the manual approach would allow for greater precision?

Anyway, I'd still like to settle on a better understanding of the scaling issue. Would there be a standardised way to scale the linearised signal (from a ML DNG) so that 90% scene reflectance registers at a float value of 1.0 (as is the case with LogC and other original log encodings).

Thank you again for your valuable advice.

EOS 5D Mark III | EOS 600D | Canon 24-105mm f4L | Canon 70-200mm f2.8L IS II | Canon 50mm f1.4 | Samyang 14mm T3.1 | Opteka 8mm f3.5