Magic Lantern Forum

Using Magic Lantern => Raw Video => Raw Video Postprocessing => Topic started by: evoxio on May 26, 2013, 07:34:17 AM

Title: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: evoxio on May 26, 2013, 07:34:17 AM
When processing DNGs from ML RAW video using Adobe Camera RAW (in either AE, Photoshop, or LR) the color/luma of the resulting footage shifts when the objects in the scene change. I'm a little surprised I haven't come across any discussion of this yet given how much of an issue it's been for me and how much I've noticed it in others' footage.

This can be seen subtly at 1:20 here (the luma of the water changes when the dark subjects cross the foreground):

Here's a shot where it's especially noticeable (watch the luma of the building in the background): https://docs.google.com/file/d/0B1keq-qyTGDNT21mR0hmWVFFc2s/edit?usp=sharing

This issue only occurs if you change the Camera RAW settings sliders (Clarity, Highlights, Vibrance, etc.) from their default/neutral states. It therefore seems like Camera RAW uses the color/luma values of the ENTIRE image to determine how to apply the demosaicing algorithm to any given pixel. In other words - the settings sliders aren't absolute controls, but rather they adapt based on the average color/luma of the frame. This phenomenon/method obviously wouldn't be an issue when processing still photos because the average color/luma of the frame doesn't change. But with video, it does.

Has anyone found a method to overcome this?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on May 26, 2013, 07:58:26 AM
I'm seeing sudden exposure shifts in pans in AE but it's not happening in Photoshop with the same footage. I think people haven't noticed it because they haven't been shooting a lot of movement. I haven't found any way around it in AE, I thought it was just a bug in the AE renderer.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: evoxio on May 26, 2013, 09:41:24 PM
Hmmm... I just ran my DNG sequences through photoshop instead of after effects and the same phenomenon is occurring. I'm surprised that you're getting different results when using AE vs Photoshop, given that they both use the exact same Camera RAW algorithm.

Anyone else noticing this problem?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: haysuess on May 26, 2013, 10:35:22 PM
I've seen this too. I have been using Adobe Camera Raw through Photoshop to process the DNGs. I've yet to try After Effects instead, but I'm thinking it'll be the same thing. This definitely makes a lot of footage unusable, so I hope there's a fix!
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: mvejerslev on May 26, 2013, 10:49:45 PM
I've seen this as well, a bit of exposure flickering visible in the portrait in
from 2:39. I'm not convinced its a Camera Raw problem. I know that the internal timer in cameras are not always 100% accurate, so exposures can shift (evident from years of panorama making in M mode). There must be a camera function in the h.264 encoding that prevents this exposure flickering happening.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on May 27, 2013, 12:16:06 AM
I'll run some more Photoshop tests today. Can everyone state which version of Adobe camera raw they're running, their OS, and their GPU.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: g3gg0 on May 27, 2013, 12:32:51 AM
This issue only occurs if you change the Camera RAW settings sliders (Clarity, Highlights, Vibrance, etc.) from their default/neutral states.
this proves that its not from the camera itself
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on May 27, 2013, 12:37:31 AM
It's definitely not the camera.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: N/A on May 27, 2013, 01:30:17 AM
This wouldn't have anything to do with using a linearized working color space, would it?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on May 27, 2013, 06:57:56 AM
Ok I've done some more photoshop + ACR tests and the exposure flickering is happening on any shots with movement/changing light where the highlight, contrast, etc sliders are used. I must have left the settings at default in my earlier test.

So essentially there's no way to make highlight and shadow detail adjustments to the DNGs with the currently available tools. Until somebody makes a tool like ACR that can properly process DNGs with movement we're screwed. We can't even create log style profiles. :'(
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Levinson on May 27, 2013, 01:48:37 PM
wow! I have shot a ton of raw on the 5d3 now, with plenty of pans, subject movement, exposure variations (shooting from 90 degrees to sun then panning into sun etc) and have yet to see this. I'm going to do some testing myself now.

I can certainly see it in the sample links provided.

Untill yesterday, I'd been using the 4gig limited Raw2dng converter and After Effects' ACR front end for all of the work.

I'm almost always reducing highlights, lifting shadows, adjusting white point and exposure in AE ACR- while occasionally adjusting contrast.

I always use ACR's "Auto" for white balance, or tweak slightly myself.

Using May 22nd 5d3 RAW video build.

Most footage I shoot is with full manual lenses like Samyang and Fujinon.

Sandisk 32gig Extreme CF (800x ??)

Shooting at 1600 x 900

It's possibly a sheer fluke that my combination of shooting environment and ACR tweaks don't reveal the issues.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Andy600 on May 27, 2013, 02:00:37 PM
Does anyone know the exact RGB offset values for correcting the chroma shift that's happening? I can make a LUT that will give us a correct starting point for grading and then look at making the same with a LOG curve.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on May 27, 2013, 02:23:01 PM
wow! I have shot a ton of raw on the 5d3 now, with plenty of pans, subject movement, exposure variations (shooting from 90 degrees to sun then panning into sun etc) and have yet to see this.

I can certainly see it in the sample links provided.

If you're only making minor adjustments it may not be visible. Big adjustments like -80 on the highlights should break it.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Levinson on May 27, 2013, 02:25:56 PM
If you're only making minor adjustments it may not be visible. Big adjustments like -80 on the highlights should break it.

I understand what you are saying. However, I often almost always pull the highlights all the way to the extreme left combined with 80+ lift in shadows.

I use spot meter to check the % value of the brightest part of the shoot (including pans etc.) and adjust exposure to either 99 or 100% at that point. Spot metering has always been dead centre of Sensor.

Is it the the 5d3 you guys are using? The bulk of my recent work footage is on the 5D3. I can't say I've noticed it on 5D2 either, but have not shot as much with them recently (and haven't kept any RAW footage myself).

Sorry if the above seems fairly trivial. I thought that the more info I give the easier it is to isolate issues. I'll try changing various parameters in camera and see if I can reproduce it, even though we suspect it's RAW software related.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on May 27, 2013, 03:04:47 PM
MK3, but I'm 100% sure it's not the camera. It could however be a computer/GPU/ACR version issue. Can you fill me in on those details.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Levinson on May 27, 2013, 03:07:53 PM
yep will do as soon as I get to work.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on May 27, 2013, 03:08:16 PM
Does anyone know the exact RGB offset values for correcting the chroma shift that's happening? I can make a LUT that will give us a correct starting point for grading and then look at making the same with a LOG curve.

There's not much point making LUTs until this ACR issue is resolved. Have you tried rawtherapee.com It's a bit buggy but I played around with it a bit and you can apply LUTs and batch process.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Levinson on May 27, 2013, 03:23:57 PM
okay, quick yest here at home.

The clarity slider used above above 25 % starts to show symptoms of exposure changes(flutter or flicker). I rarely use it for video which might explain why I haven't seen it - but it's not as dramatic as what I see in the links.

Looks like shooting sun reflecting on rippling water might be a good test. Still with the settings used in previous post.

Dell L502x Laptop, i7 2670QM, Nvidia gt 540M, camera RAW 7.1, CS6, AE 11.0.1

Will keep checking
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on May 27, 2013, 05:09:33 PM
Well we can rule out OS, GPU, and ACR builds. I think it's just that ACR just wasn't built with video in mind.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Andy600 on May 27, 2013, 05:32:10 PM
@Squig - I haven't got Dynamic Link working with AE and ACR so AE just imports the DNGs into a wrapper and I can use colorista etc to pull back highlights just as much as ACR. It's just altering the shoulder for rolloff. If I know the chroma correction values I can make a LUT that simply corrects the RGB values and there would be no need to use ACR. I get no performance improvements using TIFF seqs over DNG seqs anyway.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: 1% on May 27, 2013, 05:40:28 PM
600D/6D and Rawanizer or ACR both showed expo shifts with fasts panning.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: eoshq on May 27, 2013, 06:27:01 PM
I made a post about this back on may 19th, I don't think anyone noticed so I will say it again. http://www.magiclantern.fm/forum/index.php?topic=5404.msg37031#msg37031 (http://www.magiclantern.fm/forum/index.php?topic=5404.msg37031#msg37031)

As I mentioned before, beginning with ACR 7 and Lightroom 4, Adobe introduced what they call "Process 2012" which has automatic highlight recovery that it applies without user action. Merely loading a DNG, CR2, NEF, or other type of raw file will invoke this automatic highlight recovery that you have no control over, and which I believe has the potential to cause changes in rendering from one from frame to another because it was not designed for video. There is some discussion about process 2012 by still photographers way back in early 2012 when Lightroom 4 was in beta here. http://forums.adobe.com/thread/958989?start=0&tstart=0 (http://forums.adobe.com/thread/958989?start=0&tstart=0)

I think you can avoid this by using "Process 2010" if anyone would like to test it out. I can't do it myself because I cannot run ML on my 5D3 until it supports firmware 1.21.

P.S. Please don't take this the wrong way but something I have noticed since the beginning of Magic Lantern is that there seems to be a huge disconnect between still shooters and video shooters/ML devs. Things that are common knowledge to raw still shooters, such as process versions in ACR, seem to be unknown in the video and Magic Lantern universe. I am sure there are other examples but I just find this divide interesting.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on May 27, 2013, 06:56:02 PM
I just tried Process 2010, it worked!  :) ;) :D ;D 8) :-[  :-* Danke.

Now we can make LUTs.

Ok so we're a bit slow, but we're working at 24fps, sometimes it's hard to keep up.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: 1% on May 27, 2013, 07:40:13 PM
Why is dcraw doing it too?


Not a problem for stills since they are shot 1x1.. or only a few at a time.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: haysuess on May 27, 2013, 09:28:57 PM
I just tried Process 2010, it worked!  :) ;) :D ;D 8) :-[  :-* Danke.

Now we can make LUTs.

Ok so we're a bit slow, but we're working at 24fps, sometimes it's hard to keep up.

Thanks for the update squig! It's a shame because LR 4 clarity and highlight/shadows are so much better, but I guess it's a small price to pay.

Anyone know if it's possible to use the 2010 process in ACR via Photoshop, or can it only be done in Lightroom?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Hazer on May 27, 2013, 10:42:11 PM
Fyi I've been using RawTherapee, which is open source / free.  Similar controls/layout to ACR, and has produced nice results so far, and can also be batched via command line for the technically inclined.  "Auto Levels" is a setting right in the first pane, and it's a simple click to disable this feature to avoid frame to frame exposure changes.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: danidm on May 28, 2013, 12:57:10 AM
I had the same problem with the BMCC footage... Capture One 7 gave me the best results in DR without any flickering (maybe can appear halos if you push the DR to the limit). But at this moment i think that its imposible open the ML dngs into Capure One 7.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: noealz on May 28, 2013, 02:00:25 AM
Not sure if this helps, but I use LRtimelapse which creates smooth transitions between frames. If you set the end points to the same exposure, then the flickr could be gone?

Just my two cents.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Shield on May 28, 2013, 06:05:53 PM
Thanks for the update squig! It's a shame because LR 4 clarity and highlight/shadows are so much better, but I guess it's a small price to pay.

Anyone know if it's possible to use the 2010 process in ACR via Photoshop, or can it only be done in Lightroom?

You can toggle the process version in Photoshop, but the older versions don't support Raw 7 (aka 2012 process).

I'm really torn here guys/gals - I just can't get my image to look how I want with 2010 process version - in 2012 it just seems so much better.  Hell I could argue that I'd rather just shoot with the non-hacked mode in lieu of raw if I have to use 2010.  No matter how I adjust the sliders and curves, I just can't get it the way I want.

I will stick to raw + 2012 and live with the occasional flicker I guess.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: mvejerslev on May 28, 2013, 06:16:54 PM
As norealz points out, there are de-flickering apps out there, intended for use with timelapse, which also displays this problem.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: eoshq on May 28, 2013, 06:28:31 PM
FYI Lightroom and xmp:

Lightroom does not by default store settings in .xmp files, but it can do it just like ACR.

Develop module: Photo menu: Save Metadata to file or Ctrl+S

Library module: Metadata menu: Save Metadata to file or Ctrl+S

To make Lightroom always save .xmp

Edit menu: Catalog Settings: metadata tab: Automatically write changes to xmp (be warned, this will slow Lightroom down)
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Shield on May 28, 2013, 06:30:38 PM
The 5d3 is a remarkable camera.  I normally shoot full manual with Auto-ISO enabled, but will toggle AEL lock before recording.  I'm wondering if any of the light changes could be from someone shooting in a non-100% manual mode like Auto-ISO or A priority where something in the exposure trinity is changing (exposure / F-stop / ISO)?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Shield on May 28, 2013, 06:34:03 PM
FYI Lightroom and xmp:

Lightroom does not by default store settings in .xmp files, but it can do it just like ACR.

Develop module: Photo menu: Save Metadata to file or Ctrl+S

Library module: Metadata menu: Save Metadata to file or Ctrl+S

To make Lightroom always save .xmp

Edit menu: Catalog Settings: metadata tab: Automatically write changes to xmp (be warned, this will slow Lightroom down)

Right, but in Lightroom 4 you have to export the dng file back to a dng first, and the xmp isn't an external "sidecar" file; it's embedded in the dng file itself.  Meaning I can't just grade a dng in Lightroom and export the xmp file and import it back into Adobe ACR (i.e. the first step when using the AE workflow).  One would have to replace the original 000000.dng and overwrite the first one in your dng sequence.  I'm not sure what this accomplishes; the problem appears as you stated earlier with the 2012 camera process itself, not Lightroom.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: dirtcastle on May 31, 2013, 10:56:10 AM
I made a post about this back on may 19th, I don't think anyone noticed so I will say it again. http://www.magiclantern.fm/forum/index.php?topic=5404.msg37031#msg37031 (http://www.magiclantern.fm/forum/index.php?topic=5404.msg37031#msg37031)

As I mentioned before, beginning with ACR 7 and Lightroom 4, Adobe introduced what they call "Process 2012" which has automatic highlight recovery that it applies without user action. Merely loading a DNG, CR2, NEF, or other type of raw file will invoke this automatic highlight recovery that you have no control over, and which I believe has the potential to cause changes in rendering from one from frame to another because it was not designed for video. There is some discussion about process 2012 by still photographers way back in early 2012 when Lightroom 4 was in beta here. http://forums.adobe.com/thread/958989?start=0&tstart=0 (http://forums.adobe.com/thread/958989?start=0&tstart=0)

I think you can avoid this by using "Process 2010" if anyone would like to test it out. I can't do it myself because I cannot run ML on my 5D3 until it supports firmware 1.21.

P.S. Please don't take this the wrong way but something I have noticed since the beginning of Magic Lantern is that there seems to be a huge disconnect between still shooters and video shooters/ML devs. Things that are common knowledge to raw still shooters, such as process versions in ACR, seem to be unknown in the video and Magic Lantern universe. I am sure there are other examples but I just find this divide interesting.

This is really good to know. Thank you!!

It sounds like switching back and forth between Process 2010/2012 will be the way to handle this.

For those who are wondering where the "Process" options in Lightroom are: in the Develop tab it's under Settings.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Levinson on June 04, 2013, 09:00:19 AM

Does switching Processes to 2010 with ACR (After Effects front end) achieve the same results as in Lightroom?

I'm seeing similar exposure variations using the 2010 and 2003 processes in ACR. They appear to be an improvement over 2012, but there is still flicker present under some conditions.

Has anyone made any breakthroughs here discovery wise?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: dandeliondandy on June 04, 2013, 03:56:59 PM
Hmm... I've been shooting with the 5dmkII and editing my shots in photoshop, then importing the sequence into after effects. Shooting indoors with pans from dark walls to windows, or direct sun, etc, and I haven't seen any flickering so far. I'm using photoshop cs6. I'm assuming that would be process 2012? What could be causing that issue for you guys?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: mvejerslev on June 04, 2013, 05:54:51 PM
I havent seen exposure flicker since a very early build of raw_rec. Could it  perhaps be a camera setting doing this? Perhaps something like auto-lighting optimizer or similar in the canon menus?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on June 05, 2013, 01:55:16 PM
We've already covered this, it's ACR and Lightroom, it's very easy to reproduce, just go extreme with the highlight slider.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Levinson on June 05, 2013, 02:28:43 PM
Considering how effective and fast ACR tweaks are it is unfortunate this is happening. I find I can't quite get the latitude from Resolve.

There's some clever processing going on in ACR that the others apps can't seem to match.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Levinson on June 09, 2013, 11:09:58 AM
I'm getting quite good results using the 2012 parametric curves option in ACR, but it's still not as effective as the shadow/highlights controls which manage to retain colour in areas that even an HDR workflow can't keep. Will be great if Adobe cat get this sorted.

Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: s---70 on June 12, 2013, 02:38:34 AM
I tried every process (2003, 2010, 2012) and each of them shows flickering when applying shadow / highlight recovery, but they do it in a quite different way and in different "situations". In the shot I used for testing there is very dull lighting in the beginning and then an overexposed window gets into the view. The 2010 process produces a completely different image as soon as the window is slightly visible ad then stays exactly the same. So it seems that it's based on the brightest point in the image. Actually I made an example video showing each process but when rendering somehow ACR applied the 2012 setting to eacht of the clips. As I don't want to wait out 10min. rendering again + 40 min vimeo conversion right now I upload it again tomorrow.

Also I thought that the shadow / highlight recovery somehow uses blackpoint / whitepoint metadata (that might vary frame to frame ??) to determine its paramters and that we could somehow override those, but this seems unlikely as the flickering also appears in cinema DNG converted sequences which have an explicitly set blackpoint (e.g. 2037, 16000 for 5dm3)
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on June 12, 2013, 03:09:59 AM
It's happening with the exposure, highlight, and fill light sliders on all 3 processes and yeah not to the same extent in each. The black and shadow sliders in the second (curves) window are ok for me so far with process 2010.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: s---70 on June 12, 2013, 01:27:10 PM
yeah but the curves don't really "recover" anything, you could as well apply them outside of ACR. It's a pity that resolves demosaicing is so much worse, because you can kind of mimic the ACR highlight recovery in it. On my 5dm2 it introduces really bad aliasing on narrow lines which doesn't appear in ACR at all (except for brick walls and those things of course).
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: rockfallfilms on June 12, 2013, 06:26:48 PM
I've just experienced this, it's pretty annoying. Has anyone tried the camera raw 8 update?

Just tried it, no change unfortunately.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: s---70 on June 13, 2013, 04:52:45 PM
I just downloaded the Capture One 7 trial and with that it doesn't flicker. Shadows are nice, highlight recovery is a bit worse though. It looks a bit like the "Shadows/Highlights" effect in AE. Also it's an awkward workflow. But on my 5dm2 it does the best demosaicing of all, even better than ACR!
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: xNiNELiVES on June 13, 2013, 07:42:22 PM
So you can do your editing in capture one 7, without color/Luma shift, and then export?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: s---70 on June 14, 2013, 06:14:05 PM
If you mean colour editing, yes. But the shadow and highlight recovery isn't really good unfortunately, it produces some halos.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Kabuto1138 on June 17, 2013, 09:35:58 AM
Hey guys,

Please help!!  So is there a way to prevent this flickering when processing in ACR?  That flicker is driving me nuts.

thanks
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Shield on June 17, 2013, 01:13:45 PM
I still get flickers before the files even hit ACR.  Just shoot a scene in normal lighting and pan into a bright background, or even walk in front of a lens - it will flicker, even with ISO and all settings on manual.

When this problem happens I have to import these dng files in Lightroom and go to the file where the "flicker" happens, adjust exposure +/-, and either sync the remaining files or until the exposure comes back to where it was.  Often times it's a full stop flicker.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Ryan Lightbourn on June 22, 2013, 04:44:53 PM
When this problem happens I have to import these dng files in Lightroom and go to the file where the "flicker" happens, adjust exposure +/-, and either sync the remaining files or until the exposure comes back to where it was.  Often times it's a full stop flicker.

Shield, do you see a change in the metadata when inspecting the 'shifted' stills, or are they the same as the others?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Fauxto on June 23, 2013, 08:58:03 PM
It's a problem for me as well. I've seen it happen big with a shot of kids playing with water under full sun.

I'm using Camera Raw 6.7 in AE CS.5.5. If I don't use recovery or fill light it doesn't happen at all.

I was planning on shooting a short film in a couple of weeks but now I'm having second thoughts.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: senzazn12 on July 04, 2013, 06:26:15 PM
The flicker is driving me nuts as well. : (
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: hjfilmspeed on July 05, 2013, 05:09:58 AM
I have seen it too. From what ive seen it seems real subtle. Kind reminds me of film and how it would flicker a little.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: jmanord on July 05, 2013, 06:17:34 AM
I have been trying to find ways to avoid the flicker as well. I haven't done extensive testing, but I'm wondering if the cause is more related to the same flicker that occurs in time lapse video from the aperture blades not closing, or staying closed, to the exact same diameter. I shot several videos today with a Rokinon lens and haven't noticed any flicker among the videos I've processed so far. Since the aperture blades are controlled manually, it should alleviate the problem. I am going to test with my autofocus lenses tomorrow by shooting them stopped down and wide open to see if this has any effect.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: s---70 on July 05, 2013, 05:22:29 PM
It doesn't have anything to do with the aperture blades. when taking photos they move into the right diameter for every frame and it does that with slight variations. during livewiev the aperture is obviously locked so that can't be the problem. We already know the why it flickers, it's just the algorithm of camera raw that was not designed to be used with video. Just don't use ACR highlight or shadow recovery and contrast in the 2012 process in the shots where the flickering occurs.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: senzazn12 on July 05, 2013, 06:49:42 PM
It doesn't have anything to do with the aperture blades. when taking photos they move into the right diameter for every frame and it does that with slight variations. during livewiev the aperture is obviously locked so that can't be the problem. We already know the why it flickers, it's just the algorithm of camera raw that was not designed to be used with video. Just don't use ACR highlight or shadow recovery and contrast in the 2012 process in the shots where the flickering occurs.

I am using process 2010 in AE ACR 6.7 and the same flickering happens when using fill light and recovery.  Is there a workflow other than Cineform, Ginger HDR or Resolve that can process these files withoug flicker?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: 1% on July 05, 2013, 07:05:10 PM
Don't use those features and it won't flicker.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: senzazn12 on July 05, 2013, 07:38:26 PM
Don't use those features and it won't flicker.

I see. But then how are we suppose to take advantage of the dynamic range if we can't use those features?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: s---70 on July 06, 2013, 01:15:25 AM
you can reover highlights by lowering exposure and then adjust curves to get the brightness back
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: senzazn12 on July 06, 2013, 01:55:50 AM
you can reover highlights by lowering exposure and then adjust curves to get the brightness back

Thanks s---70. You are right. I forgot about doing it that way. I've been tackling this with such a one dimensional mind set using fill light and shadows sliders that I forgot about playing with curves. :D
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on July 06, 2013, 05:56:38 AM
Exposure adjustments cause the flicker too.  :'(
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: senzazn12 on July 06, 2013, 02:38:22 PM
Exposure adjustments cause the flicker too.  :'(

I just tested using the curves to bring back highlights and shadows and while it does not flicker as bad, the only thing is that bringing back the highlights and shadows using the curves is a little bit inferior compared to using the sliders. Man, any more enlightenment on this issue would be helpful.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: iunknown on July 06, 2013, 06:34:42 PM
Why not take it into resolve, an app that was designed for motion?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: 1% on July 06, 2013, 07:04:40 PM
Maybe need a list of what process and what ACR features cause the flicker. I.e. will auto WB do it? etc
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on July 07, 2013, 05:22:56 AM
WB is ok, chroma controls appear to be ok. Contrast is ok too. Shadow and black level sliders are ok.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: senzazn12 on July 07, 2013, 04:04:54 PM
WB is ok, chroma controls appear to be ok. Contrast is ok too. Shadow and black level sliders are ok.

I can confirm with Squig. It seems like fill light, highlight recovery and recovery do cause the flicker. So far adjusting through curves is okay. However, you can't recover much through the curves method alone. It sucks that the exposure slider does cause the flicker as well. I was thinking we can try to save the highlights as much as we can through curves in ACR and then adjust the exposure in our NLE after exporting from After Effects or just nail our exposure in the ball park when filming. Any more suggestions? Even though we can't recover much to avoid the flickering, we still get the added benefit of increased resolution and finer noise I guess. It would be sweet if ACR can fix the issue with the flickering though. I can't use Resolve cause my GPU is not good enough. Maybe I will be upgrade it soon. However, I heard even the demosaic processing and highlight recovery in Resolve isn't nearly as good as ACR.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: mikepa on July 08, 2013, 11:08:33 AM
Saw this in the RAWMagic thread. Looks like it could be a solution, if it could be implemented in one of the raw converter tools

"I have a suggestion, which may or may not be a good idea!

It concerns a possible fix for putting the CDNG's through ACR, which sometimes causes exposure variances, even with using Process 2010 in ACR. It seems that ACR needs the lightest and darkest points in the frame to be constant, otherwise any highlight or shadow recovery causes the exposure shifts.

If RawMagic could add another 2 lines to the image, say a pure white one at the top and a pure black one at the bottom, this would fool ACR into treating the high and low point of all the frames as the same (because they would be the same!)"
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on July 08, 2013, 01:36:32 PM
Sounds like a good solution.  ;D if it works.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Danne on July 08, 2013, 05:10:22 PM
Saw this in the RAWMagic thread. Looks like it could be a solution, if it could be implemented in one of the raw converter tools

"I have a suggestion, which may or may not be a good idea!

It concerns a possible fix for putting the CDNG's through ACR, which sometimes causes exposure variances, even with using Process 2010 in ACR. It seems that ACR needs the lightest and darkest points in the frame to be constant, otherwise any highlight or shadow recovery causes the exposure shifts.

If RawMagic could add another 2 lines to the image, say a pure white one at the top and a pure black one at the bottom, this would fool ACR into treating the high and low point of all the frames as the same (because they would be the same!)"

Could you do some testing on this? Do a short batch with light and dark points as suggested and compare with the regular acr workflow?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: mvejerslev on July 09, 2013, 03:28:08 PM
I have no good footage displaying this flickering so far. But for those that do: Does it make a difference if you are careful not to clip either end of the histogram?

Also, I've created an entry at the Photoshop feature request site here: http://feedback.photoshop.com/photoshop_family/topics/camera_raw_flickering_and_raw_video_magic_lantern?rfm=1 - feel free to contribute.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: iunknown on July 09, 2013, 06:35:35 PM
Here are my test.  This was shot in manual modes with no auto settings or ettr settings on.  The exposure shift is happening without any shadow/highlight or any post effects applied.  Rawanizer is causing the exposure shifts.

-If I export using raw2cdng, it does not cause the exposure shifts.  But aliasing is more prevalant than Alex's updated raw2dng.  And I suck at bringing back highlights using the qualifier controls in resolve to bring back highlights compared to lightroom/acr.

-Exporting using latest raw2dng and creating a DNxHD in rawanizer causes exposure shifts, image looks great otherwise.

-If I bring in those same dng's into after effects and export, it looks fine.

First clip is video file out of rawanizer, second is out of after effects cs6.



I guess if you want to apply heavy shadow/highlight effects use AE CS6, or render out to prores 444 and apply the effects to the exported file.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: senzazn12 on July 09, 2013, 07:03:46 PM
Does turning on exposure override, fps override or gradual exposure affect the luma shift too? Just wondering.
Title: DXO OPTICS
Post by: romeus on July 11, 2013, 03:05:01 AM
Did anyone test the DxO Smart lighting option ? it's like a shadow recovery
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: squig on July 11, 2013, 03:19:37 AM
Does turning on exposure override, fps override or gradual exposure affect the luma shift too? Just wondering.

It's a post issue.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Faringam on September 03, 2013, 12:34:35 AM
it was resolved the problem or is there yet??
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: RenatoPhoto on September 03, 2013, 02:35:00 AM
it was resolved the problem or is there yet??
Resolved

UPDATE:
In ACR 7.1 do not abuse the highlight, shadow recovery sliders because this might bring flickering from frame to frame.
In ACR 6.7 do not move the fill light and recovery sliders.
If you experience flickering then start backing off on ACR adjustment sliders.  I have successfully used the sliders in ACR 7.1 Tone Curve for highlight and shadow control without experiencing any flickering.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Faringam on September 03, 2013, 11:34:16 AM
Resolved

UPDATE:
In ACR 7.1 do not abuse the highlight, shadow recovery sliders because this might bring flickering from frame to frame.
In ACR 6.7 do not move the fill light and recovery sliders.
If you experience flickering then start backing off on ACR adjustment sliders.  I have successfully used the sliders in ACR 7.1 Tone Curve for highlight and shadow control without experiencing any flickering.

Renato sei italiano? in sostanza mi stai dicendo di non usare highlits e shadow? quindi niente hdr? ti ho mandato un pm mi pare per sentirci in privato :)
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Faringam on September 03, 2013, 12:32:33 PM
But I have ACR 8.1

is the same or better?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: RenatoPhoto on September 03, 2013, 07:25:10 PM
I tested ACR 8.2 beta and it did not help with the flicker.  Now I want to roll back and dont know how?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: kh3naz on September 04, 2013, 10:14:54 AM
I was doing some test and on one shot i was getting really bad flickering in the sky , so after some messing around in ACR 8.1 i found out that the only setting that was causing it was contrast , pushed highlights and shadows to the max , no flicker , put everything back to default with contrast to -20% , lots of flickering

Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Faringam on September 04, 2013, 11:29:36 AM
I was doing some test and on one shot i was getting really bad flickering in the sky , so after some messing around in ACR 8.1 i found out that the only setting that was causing it was contrast , pushed highlights and shadows to the max , no flicker , put everything back to default with contrast to -20% , lots of flickering

If I switch to 2010 process, there isn't flickering for me!
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: kh3naz on September 04, 2013, 11:44:35 AM
If I switch to 2010 process, there isn't flickering for me!

Interesting , but i open my DNGs in after effects with ACR , is ther ea way to change to 2010 there ?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Rewind on September 04, 2013, 11:47:55 AM
Interesting , but i open my DNGs in after effects with ACR , is ther ea way to change to 2010 there ?
Of course, ACR is the same in After Effects )
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fipic.su%2Fimg%2Fimg7%2Ffs%2Fkiss_25kb.1378287952.jpg&hash=ae2b242ed052e0dbe61cc2e02733db65)
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: kh3naz on September 04, 2013, 12:27:53 PM
Of course, ACR is the same in After Effects )
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fipic.su%2Fimg%2Fimg7%2Ffs%2Fkiss_25kb.1378287952.jpg&hash=ae2b242ed052e0dbe61cc2e02733db65)

awesome thanks
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: RenatoPhoto on September 04, 2013, 01:03:59 PM
I tried the 2010 with ACR8.2RC and flicker is still there.  :'(
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: kh3naz on September 04, 2013, 07:03:34 PM
I tried the 2010 with ACR8.2RC and flicker is still there.  :'(

would you mind sharing a short sequence (4-5 DNG frames) where you have flickering so we can do some testing, people seem to get different results, i got rid of flickering on all my shots yesterday
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: RenatoPhoto on September 05, 2013, 02:17:47 AM
Will upload some by next week when I go to a place where I can get some bandwidth!
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: feureau on September 15, 2013, 07:35:25 PM
Of course, ACR is the same in After Effects )
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fipic.su%2Fimg%2Fimg7%2Ffs%2Fkiss_25kb.1378287952.jpg&hash=ae2b242ed052e0dbe61cc2e02733db65)

Wait, which raw rec module/raw2dng converter do you use? I tried rawanizer, eyeframe but when I open the DNG, it wouldn't let me choose the camera profile in the ACR.  And only shows embedded: http://i.imgur.com/euuS0LP.png

Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: dawsmart on September 23, 2013, 06:35:46 AM
As I mentioned before, beginning with ACR 7 and Lightroom 4, Adobe introduced what they call "Process 2012" which has automatic highlight recovery that it applies without user action. Merely loading a DNG, CR2, NEF, or other type of raw file will invoke this automatic highlight recovery that you have no control over, and which I believe has the potential to cause changes in rendering from one from frame to another because it was not designed for video. There is some discussion about process 2012 by still photographers way back in early 2012 when Lightroom 4 was in beta here. http://forums.adobe.com/thread/958989?start=0&tstart=0 (http://forums.adobe.com/thread/958989?start=0&tstart=0)

I think you can avoid this by using "Process 2010" if anyone would like to test it out. I can't do it myself because I cannot run ML on my 5D3 until it supports firmware 1.21.

Many thanks!  I'm using ACR and switching to 2010 fixed it for me.  :)   I thought a big part of shooting in RAW is more control - strange they would put something automatic in there you can't disable....
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: RenatoPhoto on September 24, 2013, 05:08:54 AM
would you mind sharing a short sequence (4-5 DNG frames) where you have flickering so we can do some testing, people seem to get different results, i got rid of flickering on all my shots yesterday
Ok finally uploaded the trouble DNGs here

https://docs.google.com/file/d/0B9z8Y0rg-pu8VjZncnN0WU5jZEE/edit?usp=sharing
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: bnvm on September 24, 2013, 06:19:57 PM
Ok finally uploaded the trouble DNGs here

https://docs.google.com/file/d/0B9z8Y0rg-pu8VjZncnN0WU5jZEE/edit?usp=sharing

I tried out you sequence and it flickers for me as well unless I only change the WB settings. I don't think there is much that can be done until adobe addresses this issue. For now you have 2 options.

1.) Don't use ACR for anything other than WB just make sure to use 16 or 32 bit comps and do all of you color correction in AE rather than ACR.

2.) If you really want to use ACR then I may have one option that will work, the catch is that requires a plugin that costs $100, although this plugin would be useful and a big time saver for other tasks. Basically you import your footage in to AE with no correction done in ACR, next import the same sequence again and do you corrections in ACR. Next use the plugin RE:Match, which is a automatic color matching plugin, to automatically adjust the un-process sequence to the processed one. Done.

RE:Match http://www.revisionfx.com/products/rematch/ (http://www.revisionfx.com/products/rematch/) was able to perfectly match the grade when I tested the demo version and it allows you to store the adjustment at 1 frame and then reuse for the rest so there will be no flickering when done. Also this plugin would make matching grades from 1 shot to the next much faster and may be worth it for that alone.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: RenatoPhoto on September 25, 2013, 01:26:18 AM
1.) Don't use ACR for anything other than WB just make sure to use 16 or 32 bit comps and do all of you color correction in AE rather than ACR.

Thanks for your analysis.  I am able to do very good editing with ACR using the Tone Curve where I can independently control
Highlihgts, Lights, Darks, and Shadows

I think kh3naz wanted to try switching to 2010 process but I had no luck and it flickered for me with 2010. 

BTW I am very happy using ACR with Tone Curve.  Excellent results IMHO!
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Shield on September 25, 2013, 10:23:13 AM
For what it's worth, do any of you see the flicker in the un-edited DNG files?
Also, do you shoot in manual with Auto-ISO?  I've found I rarely get any flickers if I hard set the ISO.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: bnvm on September 25, 2013, 05:15:54 PM
For what it's worth, do any of you see the flicker in the un-edited DNG files?
Also, do you shoot in manual with Auto-ISO?  I've found I rarely get any flickers if I hard set the ISO.

The footage is fine if I make no changes or only change the white balance. From what I understand ACR works on 1 image at a time and the adjustment sliders are based of the current image they are not global adjustments rather relative to the current image. I suspect that the shaking bird exposing and hiding its white feathers in this sequence is causing enough value change to trigger the flickering. Just a theory though.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: bnvm on September 25, 2013, 05:19:05 PM
Thanks for your analysis.  I am able to do very good editing with ACR using the Tone Curve where I can independently control
Highlihgts, Lights, Darks, and Shadows

I think kh3naz wanted to try switching to 2010 process but I had no luck and it flickered for me with 2010. 

BTW I am very happy using ACR with Tone Curve.  Excellent results IMHO!

You may want to consider option 2 then. It was able to perfectly match the adjustments I made in ACR automatically with zero flickering. I does complicate the workflow somewhat but its not too bad. BTW I did my testing with CC so adobe hasn't made any recent improvements in this area.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Jonit on October 15, 2013, 02:15:58 PM
I have the same problem but luma is shifting even before it opens up in ACR. Here i uploaded 5 DNGs straight from the raw2dng (also same problem when processing in raw2cdng). Due to this almost all of my 50 shots are completely unusable  :( So in my case the problem must be either the camera or raw2dng.

https://mega.co.nz/#!2NE1wTAT!Uiek1yOYeWeusNlzIihTvxRVGpo5JoN03dgLh1S94do
 (https://mega.co.nz/#!2NE1wTAT!Uiek1yOYeWeusNlzIihTvxRVGpo5JoN03dgLh1S94do)
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: glubber on October 16, 2013, 01:03:22 PM
I have the same problem but luma is shifting even before it opens up in ACR. Here i uploaded 5 DNGs straight from the raw2dng (also same problem when processing in raw2cdng). Due to this almost all of my 50 shots are completely unusable  :( So in my case the problem must be either the camera or raw2dng.

https://mega.co.nz/#!2NE1wTAT!Uiek1yOYeWeusNlzIihTvxRVGpo5JoN03dgLh1S94do
 (https://mega.co.nz/#!2NE1wTAT!Uiek1yOYeWeusNlzIihTvxRVGpo5JoN03dgLh1S94do)

Just a quick guess:
You are using auto-exposure in ACR.
When opening your DNGs in ACR 3 or Rawtherapee i can't see any lumashifting. (Unless my eyes aren't able to see it  :P)

The upper ACR-screenshot is using manual settings, the lower is with "Auto" turned on. In the last case i see a shifting between pic 37 and 38.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fimg228.imagevenue.com%2Floc254%2Fth_920378277_ACR_user_vs_auto_122_254lo.jpg&hash=0a9a852ae0b6605a481ed2a2d063a490) (http://img228.imagevenue.com/img.php?image=920378277_ACR_user_vs_auto_122_254lo.jpg)

I hope this the solution to your problem.
     
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Jonit on October 17, 2013, 10:08:17 PM
Just a quick guess:
You are using auto-exposure in ACR.
Thanks glubber,
no, that wasn't the case. I found out that the problem was with my photo viewer. I am using Picassa to view DNG files and it seems like it applies some corrections to individual frames. I rendered the frames as a video clip and there was no luma shifting  :)
But i really appreciate your help, thank you  :)

John.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Haliburton on December 16, 2013, 07:21:56 AM
Saw this in the RAWMagic thread. Looks like it could be a solution, if it could be implemented in one of the raw converter tools

"I have a suggestion, which may or may not be a good idea!

It concerns a possible fix for putting the CDNG's through ACR, which sometimes causes exposure variances, even with using Process 2010 in ACR. It seems that ACR needs the lightest and darkest points in the frame to be constant, otherwise any highlight or shadow recovery causes the exposure shifts.

If RawMagic could add another 2 lines to the image, say a pure white one at the top and a pure black one at the bottom, this would fool ACR into treating the high and low point of all the frames as the same (because they would be the same!)"


^^ This sounds promising - has anyone tried this? ^^

.

To the matter of ACR : Is it possible that there may be some settings stored within the DNG that cause or exacerbate the flickering problem?

I haven’t experienced the problem yet; most of my scenes are pretty consistently lit.

The one scene I recall having panned across widely differing lighting, I made extensive ACR settings on significant frames , then hand-tweened the .xmp files between them, as “developing” transitions.

For what it’s worth, I bulk-lock the newly-minted DNG files in the Mac Finder Get Info when they come out of raw2dng, so that development settings (process 2012) are stored only in the XMP sidecar files.

Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Rafael Duarte on December 18, 2013, 01:18:54 PM
Hello, everyone. I recently bought a Blackmagic Cinema Camera and I have this same problem. Apart from this discussion, I wasn't able to get much information anywhere. You can check it out here:


The flickering is pretty intense, but I figure it is because I really cranked up the ACR sliders. This is one of the reasons I prefer using it instead of Davinci. The amount of recovery and tweeking you get in Camera Raw is not matched by Davinci, even though it is a really complete program in terms of color grading.
Personally, I really like the way ACR process raw. I have been using it with my stills and love it. It appears - at least to my eyes - that it really gets a cinematic quality out of the picture. If we can't freely use the sliders, than there's not really a point in using it. It is where the power is, for me. Using the 2010 process helps, but it also takes a lot of the power away.

I hope we can find an alternative. It seems that ACR is essential for determined picture styles.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: stevefal on April 27, 2014, 06:06:46 PM
I posted a feature request/bug at Adobe and started a discussion on their forum. Chime in for visibility:  http://forums.adobe.com/message/6334477#6334477 (http://forums.adobe.com/message/6334477#6334477)

~~~~~~

Discussed at http://www.magiclantern.fm/forum/index.php?topic=5710.0

People using ACR to import video image sequences into After Effects have discovered that ACR's highlight recovery and other exposure controls produce varied exposures depending on the content of each frame.

The result is "flickering" in the rendered video when the scene changes such as when light or dark objects come into view.

Users have experimented with Process 2010 and Process 2003, but they all exhibit the same problem to some degree.

The problem makes ACR unusable for some video shots where it would otherwise be a great solution due to its superior highlight recovery.

I discovered this issue when reviewing my footage rendered from After Effects. Any with significantly bright or dark objects coming into frame changed the exposure of the rest of the frame, irreparably.

SUGGESTION
The suggestion is to offer an “Exposure Lock” feature in Adobe Camera RAW, which would lock the absolute exposure values determined via the prototype frame (typically the first frame), and use those for every frame of video rendered. This means that exposure computations performed based on the content of each frame would be disabled during rendering. The computations would be performed only when the user interacts with the exposure controls via the ACR UI.

The benefit of this feature is that ACR could be legitimately used to initially process high quality RAW video. A side benefit is any situation, including still photography, where the user needs to match absolute exposure settings between shots.

I hope Adobe will consider adding this feature or something equivalent that provides, for video, ACR’s unsurpassed ability to tame highlights.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: mannfilm on May 30, 2014, 08:14:19 PM
Fixed with current  ADOBE after effects CC (mac)

If you import DNG's into photoshop CC, the ACR 7.0 opens up as the "photoshop shelled" of ACR,  and you still get the flicker.

BUT if you open After Effects CC, and import the DNG's as a image sequence, a very slightly DIFFERENT version of ACR 7.0 pops up. You can make your changes in this "After Effects shelled" ACR, and it does not flicker.

I assume that the Photoshop version of ACR works as a still editor adapting to each single frame, while the after effects version of ACR works as a video editor and "locks" the effect across all frames.

Confirming this, the exact same ACR control setting produces  different results (in frames other then the "master,") in the photoshop or after effects exported movie.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: stevefal on June 02, 2014, 05:58:36 PM
I just tested this and AE/ACR CC flickers exactly the same way as CS6. I see no evidence that settings are locked.

I'm beginning to think that the ACR highlight control performs microcontrast adjustments, which, if true, may make flickering inevitable, and the idea of "locking" a fallacy.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: domisol on June 06, 2014, 07:51:55 AM
Fixed with current  ADOBE after effects CC (mac)

If you import DNG's into photoshop CC, the ACR 7.0 opens up as the "photoshop shelled" of ACR,  and you still get the flicker.

BUT if you open After Effects CC, and import the DNG's as a image sequence, a very slightly DIFFERENT version of ACR 7.0 pops up. You can make your changes in this "After Effects shelled" ACR, and it does not flicker.

I assume that the Photoshop version of ACR works as a still editor adapting to each single frame, while the after effects version of ACR works as a video editor and "locks" the effect across all frames.

Confirming this, the exact same ACR control setting produces  different results (in frames other then the "master,") in the photoshop or after effects exported movie.

Thanks for this information.
Can you source it or does that come from your own experience ? How is it that you use ACR 7 ? Do you mean that this works only with 7 and not 8 ?
This doesn't work for me, using last version AE CC on Os X, with ACR 8.4.1 : 2012 process gives me slightly exposure differences in highlights betweens frames.

Jean-David
 
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Rafael Duarte on June 25, 2014, 05:28:39 PM
Hello, everyone. I have also tried AE CC on Mavericks and the flickering is also there, albeit less.
I have downloaded Davinci Resolve 11 beta yesterday and they have added some new options on the camera raw tab that are very derivative of camera raw. Besides exposure, temperature and tint, there is now highlights and shadows, color boost (which seems relative to vibrance) and midtone detail (which seems like clarity). I gave just a brief look but it seems nice. Maybe Blackmagic has been hearing us after all. :)
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: seb_ on August 05, 2014, 04:41:55 PM
Saw this in the RAWMagic thread. Looks like it could be a solution, if it could be implemented in one of the raw converter tools

"I have a suggestion, which may or may not be a good idea!

It concerns a possible fix for putting the CDNG's through ACR, which sometimes causes exposure variances, even with using Process 2010 in ACR. It seems that ACR needs the lightest and darkest points in the frame to be constant, otherwise any highlight or shadow recovery causes the exposure shifts.

If RawMagic could add another 2 lines to the image, say a pure white one at the top and a pure black one at the bottom, this would fool ACR into treating the high and low point of all the frames as the same (because they would be the same!)"

Yes, I tried this and it solved the flickering issue for me!

I have to add that I don't use Magic Lantern and RAW video, though. I just grade my regular H.264 videos in ACR and AE. But since it seems to be an issue with ACR, this should work with RAW footage as well.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: stevefal on August 05, 2014, 09:32:22 PM
I don't think this works reliably. I believe that ACR's algorithms behave based on the number of white/black or highlight/shadow pixels in the image, not just the existence of some. For instance, an image that is mostly black with a little light, will be changed differently than one which is mostly white with a little dark.

The "two lines" approach is the first thing I tested when measuring ACR's behavior, and it did not improve the flickering problem. But trust me, I'd love to be wrong.

Can you demonstrate your technique working with aggressive ACR settings?

Also, what you mean by "grade my H.264 videos in ACR?" How is that possible?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: seb_ on August 05, 2014, 11:08:59 PM
I've only discovered this today, so I haven't had the chance to test it extensively. But I made a short video demonstration. It's not pretty, but it gets the point across.


Recovery and Fill Light sliders were bumped to 100 in both videos. I use 2010 process.

Also, what you mean by "grade my H.264 videos in ACR?" How is that possible?

ACR doesn't only import RAWs, but also other image formats. So I convert my video clip into a TIFF sequence which I then import into AE and grade it with ACR, like I'd do with a RAW sequence (see this video for more
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: stevefal on August 06, 2014, 12:57:54 AM
Well clearly that had a huge impact. I'll have to look at my test data and see if I did the two-line test with Process 2010.

Your unaltered footage seems to clearly show extreme microcontrast, which I assume is from the highlight recovery. I mean, the white clouds are darker than the blue sky, which is weird.

Anyway, cool result.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: stevefal on August 06, 2014, 01:07:30 AM
I looked at my data and yes, I confirmed that Process 2010 recovery and fill are tricked with black lines. It does not work with Process 2003 or 2012, as those algorithms consider the number of white/black pixels.

I did not try the white line case.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Danne on August 06, 2014, 01:25:22 AM
Nice results with 2010 process. Could you try to process with a later process lifting shadows and highlight recovery with the gradient tool and see if that works better as well on the sky clip? My example is from lightroom but works just the same in acr coming from photoshop. Don,t have a good clip to test with.

Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: stevefal on August 06, 2014, 01:33:10 AM
Here is my test data and results. Process 2012 is a disaster for this issue:

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2Fgao7HVJ.png&hash=da0d75570300d0d4254e592df69ff46d)

gradgrey_black
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FrrDpzx4.png&hash=d2adeaf735d6c89358f92368f072e390)

gradgrey_white
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FSVpo40V.png&hash=0f3e663b6abfd2c14473ea37362a1e86)

gradgrey_white_sliverblack
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FJyogdWK.png&hash=fd7544dcb92bad68637ea4ee7d19b3e3)
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Danne on August 06, 2014, 01:48:22 AM
Hi Stevefal. Are your numbers from the gradient tool experiment?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: stevefal on August 06, 2014, 02:28:51 AM
My numbers are from testing I did months ago. I wanted to determine how dynamic range of an image impacted the behavior of various exposure controls, and for each of the Processes.

In each test, I tracked the value of an individual pixel before and after making ACR exposure changes, and compared across images that had the same value in that pixel location (150,150), but different overall image dynamic range. A red value means that there was a difference, which essentially amounts to the flicker issue.

The last test image corresponds to seb_'s experiment, in which a small amount of black added to the gradgray_white image negates the differences otherwise introduced. This is only true for Process 2010.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: johannsebastianbach on July 27, 2016, 06:00:02 AM
It has been two years since the last post in this thread. Things may have changed. I googled this issue with no new solutions so better ask here  :)

Did anybody solve the problem, or found another way for flicker-free recover highlights in the awesomely way ACR does?
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: jmanord on July 27, 2016, 10:13:14 PM
I am curious as well, and hope to do some simple objective tests in the next couple of days. I have several clips I plan to test, but I don't expect any improvements in results for the controls as outlined by stevefal. However, unless I have incorrect project settings in Resolve, the debayered results from Lightroom's Process2012, before using the highlight or shadow sliders, appears to do a better job preserving dynamic range than the results from using Resolve's highlight recovery slider.

After originally deciding the results from Resolve were good enough, based on how much easier it is to skip Lightroom as part of the workflow, I'm finding that following the ETTR methodology is resulting in too many instances of distracting blown highlights when using Resolve alone. Avoiding the controls that cause issues when using Lightroom's Process2010 profile avoids the flickering problem, but doesn't give results as nice as using Resolves highlight recovery tools. Using Resolve's highlight recovery tools works well, but often introduces bizarre artifacts too the image. I know there are several other raw editors, but Lightroom's stack and auto-stacking features makes it extremely easy to manage clips and apply develop settings to the dngs.

If anything new and helpful is revealed I'll post it here.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Kharak on July 28, 2016, 09:50:16 AM
Can not speak for resolve, but to pull back the highlights without introducing flicker, Cinelog-C DCP 2016 will do just that for you and more :) but you're heading in to log workflow which might seem daunting at first, but there are so many threads and posts on this issue on the forum, that you'll get it no time
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: jmanord on July 30, 2016, 02:05:26 AM
@Kharak - I took your advice and purchased Cinelog-C DCP 2016. After swearing off trying to grade Log footage, I still have a7s slog-2 nightmares, I'm determined to find a quick and effective workflow between Lightroom and Resolve.

My objective test quickly turned into subjective tests. I edited 10 clips shot earlier this month using Lightroom's controls, including those known to introduce flicker, to obtain the aesthetic results I was looking for. After exporting the files to 8-bit tiffs and loading the sequence into Resolve, only 2 clips exhibited visual flickering. Rather than doing individual slider tests to verify which controls introduced the flickering, due to time constraints, I reset all the controls that are known to introduce flicker and verified that the flickering was gone, which it was. I'm optimistic that introducing Cinelog-C DCP 2016 into my workflow will allow me to recover the highlight information I know is in the raw files without introducing additional artifacts.

Before deciding to purchase Cinelog, I wanted to give grading in Lightroom and Resolve one more try and compare my results. After selecting 4 clips and spending a few hours doing my best to achieve a similar look in both programs, I committed the easily avoidable sin of forgetting to save my project in Resolve. It crashed during rendering. Instead of trying to get the clips to match again, I edited them together to demonstrate the differences I experienced in editing the same footage in both. This is the result:


I mostly wanted to show the pink cast and artifacts that often occur in Resolve when trying to recover highlights. The moving center strip is the output from Lightroom. I was fairly happy with the results I obtained within Lightroom while avoiding the Highlights, Shadows, Contrast, Whites, and Clarity sliders. But that was only the case for footage which was exposed to the right by a stop. The footage which was exposed properly or slightly under-exposed, e.g. the third clip in the video above, was easier to grade in Resolve since highlight recovery was not necessary. My fingers are crossed that Cinelog-C tiffs exported from Lightroom will allow me to more easily grade both types of footage in Resolve.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: jmanord on July 30, 2016, 10:34:55 PM
I wanted to follow up with a video showing an altered Lightroom clip that flickers and a similarly altered Resolve version using Cinelog-C DCP 2016. It may have been better to leave the mask static, but I wanted to be able to scrub between the different areas to compare the outputs. The flickering is more subtle after uploading it to youtube, but if you keep your eyes on the shadow on the left dune you can see it. The Resolve version is the sliding center strip. Sorry for the shaky footage.

Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Andy600 on July 30, 2016, 11:21:55 PM
@jmanord - Did you alter any of the raw controls before exporting the tiffs from Lightroom? The profile can't induce flicker if the controls are nulled. It may be that the DNGs themselves have some sort of issue (could be shutter related) or possibly, the output codec. What app did you use to unpack to DNG? Did you use a DNG compressor such as Slimraw or are these straight from the .mlv/.raw file?

If you want to send me that short DNG image sequence I can test it over multiple workflows in AE, Photoshop, Lightroom and Resolve to see where the modulation is occurring.

BTW, I noticed you exported 8bit TIFFs previously!? - 8bits is not enough (10bit min). If you export Cinelog-C to TIFF it should be 16bit.

Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: jmanord on July 30, 2016, 11:47:10 PM
@Andy600 - Sorry, my description was not very concise.  :( I was trying to show that Lightroom's Highlight, Shadow, etc sliders still introduce the flickering. Cinelog DCP solved that problem by exporting the dngs as 16bit tiffs using the Cinelog-C DCP profile in Lightroom and adjusting them in Resolve. I wanted to show how I was able to achieve a very similar look in Resolve from the Cinelog-C tiffs compared to the Lightroom version that had the flickering. Cinelog is awesome, thank you again for your help!
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: Andy600 on July 30, 2016, 11:58:09 PM
No problem :)

I'm still working on the 'Applying Cinelog luts in Lightroom and Photoshop' workflows. It all works but I'll probably need to build a special ICC output profile for Lightroom to get around some of the limitations.
Title: Re: Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
Post by: johannsebastianbach on August 01, 2016, 03:28:00 AM
Just found this gem. This software is still in the concept phase - they are looking for developers!

http://apertus.org/opencine