.DNG/.CR2 to 16-bit linear .EXR workflow (w/ overbrights!)

Started by agentirons, August 14, 2015, 12:42:30 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

agentirons

Hello everyone,

I have been developing a VFX pipeline for a stop-motion production (essentially the same requirements as RAW timelapse) and am struggling to find an answer to something that seems very basic.

We are using Canon 60D cameras to capture 5K RAW .cr2 files, and our VFX team uses After Effects for compositing. However, our computers are not really up to the challenge of working directly with .cr2 image sequences using the Adobe Camera Raw importer, at least not when we need to crank out dozens of shots a week.

What I would like to do is convert the .cr2 sequences into 2.5K .exr sequences (linear/scene-referred). In doing so, I would love to keep ALL of the dynamic range that is present in the original .cr2. This seems like a pretty standard workflow for large productions - take raw camera footage, convert to 16-bit linear .exr for grading and VFX, then output to whatever. It's even the standard process for the new Academy ACES workflow.

However, I cannot find a method for transcoding that actually keeps all of the dynamic range (essentially the overbrights), and in the process of my research found that not even Adobe Camera Raw seems capable of doing so within After Effects.

Here is an example of what I mean, using After Effects CC 2015 and Adobe Camera Raw 9.1.1:



I feel like I've tried every format and program under the sun (including Resolve) and still always come back to this same issue - Somewhere there is information being clipped.

Am I simply misunderstanding how RAW processing works? From everything I've read, it really sounds like all major productions have somehow solved this issue. I know that I can adjust the exposure in ACR manually to recover the highlight information I want, but for our workflow with thousands of sequences, manual adjustment simply isn't an option.

Thanks for anyone who can help shed some light on this!


Andy600

@mothaibaphoto - Use Davinci Resolve 12 beta, set it to use YRGB color managed (setting panel). Set input colorspace to linear, set workspace to ACES and output colorspace to linear (also in the main setting panel) then export half float .exr. This is a common professional compositing practice. If you want to work in another colorspace just change the workspace colorspace to something else i.e. sRGB/Rec709. The .exr file will retain the DR.

Alternatively, transcode to a 444 log intermediate format in ProRes or DNxHR for smaller files or convert the cr2 files to DNG then buy a copy of http://www.slimraw.com to losslessly compress them and use a DNG workflow but .EXR and DNG pipelines are slower and more processor intensive than using a log intermediate with no significant gain in quality providing you white balance and correct exposure (if required) before committing to render.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

agentirons

Thank you, Andy! That simple description is what I've been looking for all week.

mothaibaphoto, sorry it was a poor choice of image host. Fixed now!

Yesterday several pieces finally came together that helped me understand the issue. Regarding Adobe Camera Raw, I found this Adobe support thread: https://forums.adobe.com/message/4519452 with this very important post from an Adobe staff member:

QuoteHi Joe,

Our design of ACR/LR is to render images for display or print.  When starting with scene-referred images (e.g., raw), this means tone and color mapping the input linear light data (e.g., whether 12-bit integer or 32-bit float) to values suitable for reproduction (output-referred).  Since the resulting is always intended to be output-referred, the resulting images are always stored using 8-bit or 16-bit values.  We do not see any need to represent output-referred data using 32-bit values.

(ACR/LR is not intended to be a "pass-through" system that can, say, take 32-bit image data as input and also emit 32-bit image data at the end.  It can be made to do so, though clumsily.)

There is a ton of misinformation out there about ACR keeping all of the raw data, and it's simply not true, although whether that matters for your workflow is another issue. I wish I had known that awhile ago.

For some reason that simple string of directions that Andy gave has been very hard to come by, even though it's standard practice at other studios. So thank you again for helping spread that knowledge around!

Andy600

That's not quite true about ACR binning raw information. Normally ACR displays an image that is display referred with a tone curve, gamma and HSV color corrections but we built log camera profiles to enable data to be moved from ACR into After Effects under the inherent ICC profile restrictions without touching any of the ACR controls. Once the image is in AE in a 32bit floating point workspace it can be returned to scene referred linear values using our OCIO config and then exported as .EXR.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

agentirons

Ah, that is very interesting. Are you referring to Cinelog?

agentirons

So I've been playing around with Cinelog at work, it's a great tool for the toolbox! But now I'm trying to settle a debate with my coworkers:

If the Canon 60D has a 14-bit sensor (I'm assuming that means 14-bit per channel) and yet going through ACR with default settings and one of the Canon camera profiles such as Camera Neutral into a 16-bit per channel project somehow does not allow for retaining the full dynamic range, then what is the bit depth of the ACR importer? Is it less than 14-bit?

I have read that these sensors record 14-bit integer data to their .cr2 files, and After Effects can easily do 16-bit integer data, so unless the camera is unsigned and AE is signed, we're having a very hard time understanding why something like Cinelog is even necessary (even though clearly it is for what we want to do). Why isn't the brightest pixel in the 14-bit .cr2 file still the brightest pixel in the 16-bit integer project?

Perhaps 14-bits of un-debayered data is equivalent to much more than 16-bits of integer data?

chris_overseas

Yes the 14 bits is per channel. The thing to bear in mind is that a RAW file is not really an image in the same sense as a JPG or TIFF file that has RGB pixel values that you can view directly on screen. The RAW file contains Bayer patterned data straight from the sensor where the intensity of each point corresponds to what was read from each photosensor. The sensor has very different characteristics to the human eye (linear vs logarithmic being one of them), so it's not feasible to map directly from the 14 bits RAW to 16 bits TIFF and expect results that will look sane.

To translate the data in a RAW file into something that you can view on screen a lot of processing has to be done. Choice of white and black points, demosaicing, gamma correction, white balance, colourspace conversion, noise reduction, sharpening, tone curve adjustments and so on. Virtually all of these operations lose some information along the way (eg clipping shadows or highlights), and/or suffer from rounding errors due to the conversion from one bit depth to another. Of course once the information is lost there's no getting it back, hence why the choice of profile can make such a big difference to the amount of data retained in the initial RGB image data you get after the conversion.
EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II

agentirons

Ok, most of what you're saying makes sense, at least compared to what my eyes tell me, but I'm still struggling with the math.

If a Bayer patterned pixel array goes to a 14-bit analog-to-digital converter in the camera, then presumably for each pixel there's a certain amount of light that will result in a maximum value (ie 16383). My understanding of demosaicing is that at its most basic you are taking, say, a green sensor element with a certain value, and averaging it with the neighboring green elements, and then averaging the nearby red elements and the nearby blue elements to come up with an RGB value for that pixel in the resulting image file.

So, if there's a hard limit in the sensor to the amount of light it can record at one element, and that resulting value even when averaged with nearby values can't possibly go above the maximum value of a 14-bit integer, then shouldn't we be able to convert raw 14-bit data into a 16-bit image with nothing done to the values? If it's a linear sensor, then wouldn't it just give me a linear 16-bit image? I know that doesn't look 'sane' on a monitor, but it's trivial at this point to use a LUT to convert from linear values to something else for viewing or editing.

Have I just answered my own question? Is it simply that ACR is not built to go from linear to linear, and that the conversion from linear to vid (by which I mean display-referred values) is capable of pushing the max linear value of 16383 to a vid value of something much higher, resulting in clipping even in a 16-bit space?

Andy600

@agetirons

QuoteIs it simply that ACR is not built to go from linear to linear, and that the conversion from linear to vid (by which I mean display-referred values) is capable of pushing the max linear value of 16383 to a vid value of something much higher, resulting in clipping even in a 16-bit space?

Yes and no. ACR output is display-referred so even with a linear tone curve it is effectively outputting sRGB when used specifically in conjunction with After Effects. This is because the ACR to AE pipe only works in sRGB colorspace (using an ICC profile). If you use the inverse of the sRGB transfer function as the tone curve you get clipping, even if an exposure offset has been specified in the profile metadata or the exposure slider is used to full down the highlights (I believe this is related to the 2012 process which has 'some' non-linear aspects to it). There was an often-used workaround described by Stu Maschwitz which suggested nulling certain parameters in AE to produce a linear image (but this used the 2003 process which is inferior to the 2012 process) and even then, when comparing the ACR debayered image (exported as a 16bit tiff) to a true linear rendering of the image using DCRaw the RGB values still don't match.

If the ICC profiles in ACR were selectable in the ACR2AE pipeline (the same way you can change them when using Photoshop) it could theoretically deliver a true linear debayered image to AE but unfortunately it is internally locked to sRGB and I doubt Adobe will ever change this.

The Cinelog ACR profiles basically compress the RGB values enough to ensure image data can be passed into AE (with nulled ACR settings) without any clipping and then be delogged in AE in 32bit floating point (which will usually produce some negative values - hence the requirement for the additional conversion to Alexa Wide Gamut RGB using OCIO when transcoding intermediates). Linear in OCIO is scene referred with operations always happening in 32bit floating point but even then there is no way to reproduce exact linear RGB values (i.e. that would match DCRaw, Resolve, Nuke, Fusion etc etc) because of the nature of the 2012 ACR process which always attempts some highlight reconstruction of clipped values based on unclipped channels. In reality this difference is negligible.

We get asked by some users why the image looks very dark and clipped when their DNG images are transformed to Linear and this is simply due to viewing it under a display-referred profile (i.e. Rec709) without an s-curve and it's why any linear output rendering always needs to be a minimum of 16bit Tiff or DPX and preferably float .EXR.

We have built several different delogging curves (to better match DCRaw linear, Resolve linear etc) but didn't add them to the OCIO config as it would be too confusing for most users and would mean output referred transforms (rec709, look luts etc) are incorrect (lower contrast) but I will add a subset of these linear matching functions to an additional profile with the coming update.   
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

agentirons

Yeah, thank you again for your information Andy, I finally feel like I'm getting a solid grasp of all this!

And for anyone else reading this, I can definitely recommend the Cinelog package, very much worth it, especially for timelapse/stop-motion.

DeafEyeJedi

@Andy600:

Quotewhich will usually produce some negative values - hence the requirement for the additional conversion to Alexa Wide Gamut RGB using OCIO when transcoding intermediates

So does this mean I'll have to do this process after finishing the normal routine that I do with Cinelog-C in AE within OCIO as a requirement? I didn't even know about this and am starting to feel like an imbecile unless this was intended directly for DR's workflow?

Quoteit's why any linear output rendering always needs to be a minimum of 16bit Tiff or DPX and preferably float .EXR.

This also doesn't mean that these options are better if not similar to PR4444XQ for Mac users?

If so, then is this where I can get the codecs from? --> http://www.fnordware.com/ProEXR/ or this --> http://www.openexr.com

Thanks!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Andy600

@DeafEyeJedi

You don't need to do anything different. You are already using the OCIO plugin which handles everything for you :)

re: .EXR - It's useful to have individual frames when compositing or doing animation/stop motion etc as you can re-render only selected parts when you make changes i.e. cut down on processing time plus an .EXR can include more channels but for general video work you are better off with a video codec and ProRes XQ is as probably as good as you will ever need. Both .EXR and .DPX are heavy file sizes. I'm not sure about ProEXR for Mac TBH but I don't think you need it anyway. Resolve has full EXR support.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

agentirons

ProEXR is for giving EXR functionality to Adobe products, and in After Effects' case some extra functionality above what's already there. AE's built-in EXR plugin is a free version of ProEXR, minus the ability to work with multiple channels.

DeafEyeJedi

*whew* Thanks, @Andy600!

Slightly off-topic:

So it would also mean somewhat pointless for me to continue doing experiments between exporting as PR4444XQ versus Photoshop Sequence?

I then would import the psd files as image sequence into FCPX and drag the audio over from MLVFS. worked like a charm.

According to my eyes it seems like you save more of the details/colors/DR if I were to export as h264 with these psd files versus the intermediate file in ProRes XQ or should this not even been a discussion to begin with?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

tajino

Quote from: Andy600 on August 14, 2015, 03:33:52 PM
@mothaibaphoto - Use Davinci Resolve 12 beta, set it to use YRGB color managed (setting panel). Set input colorspace to linear, set workspace to ACES and output colorspace to linear (also in the main setting panel) then export half float .exr. This is a common professional compositing practice. If you want to work in another colorspace just change the workspace colorspace to something else i.e. sRGB/Rec709. The .exr file will retain the DR.


Tried your suggested settings, the resulting .exr doesn't retain the DR, input is a .CR2 file. Are you referring to Project Settings > Master Project Settings > Color science > Davinci YRGB Color Managed? Then in Color Management > Input Color Space > Linear , Color Management > Output Color Space > Linear?

DR is clipped in the output .exr.