ETTR + deflicker producing extremely over exposed images

[ColinKennedy]

I'm using ETTR + deflicker for timelapse photography. The ETTR feature works great during daytime shots and deflicker also works but as soon as it transitions from day to night, there comes a point when the exposure gradually gets brighter and brighter until my camera reaches "expo limit". I examined the .XMP sidecar files it produced and the suggested exposure change was 7+, which ACR is not able to do (not that it would be good to add more EV to an already over exposed image). Lowering/Disabling my SNR limits has not fixed this and I'm not sure what I'm doing wrong. I followed the flicker free timelapse thread post here to get the results that I'm having now. I've also snooped around on the forums here for solutions - a couple of which have said that ETTR is not supported with t3i but I'm skeptical because the posts are old and there are people that claim that it works for them. Also ETTR works fine in daylight so I believe it's something in my settings.

Can anyone please shed light on why my ETTR+deflick setup is so messed up in low light?



Hardware/Software:
Canon T3i
Bower 8mm fish eye lens
Nightly.2015Apr19.600D102 (latest Nightly build as of this post)

Canon Settings:
Manual mode
Aperture - manual ring (on lens)
ISO - not on auto
Image Review - 4 seconds
Custom Functions: Noise reduc. on long exposure: On, High ISO noise reduc.: strong

ML Settings:
[Shoot]
Auto ETTR - always on, slowest shutter: 32", Highlight Ignore: 0.1%, Midtone/Shadow SNR Limit 6/2 or 8/4 (both don't work), Link to Canon Shutter: Off, Link to Dual ISO: off Allow Beeps: Off

[Overlay]
Global Draw - On, all modes
Spotmeter - RAW, AFBox
Histogram - RAW RGB, Log, dots, Use RAW histogram: full histogram

[Shoot]
Intervalometer - Take pic every 1m15s, start after 3 sec
Post Deflicker - Sidecar file type: XMP, Deflicker percentile: 50%, Deflicker target level: -4 EV

[garry23]

@ColinKennedy

Others will add to this I'm sure.

Many get confused with ETTR and sunset or sunrise transitions.

ETTR will always seek out a solution 'to the right', ie the brightest exposure you can get short of clipping.

You create the required timelapse transition (day to night or night to day) by adjusting the exposure in post.

Deflickering is a separate process.

Have a look at some of the LRTimelapse tutorials.

Garry

[Kharak]

Try deactivating Noise reduction all together in Canon menu.

And how long are your exposures?

[Walter Schulz]

High-ISO noise reduction will only be used in post-processing by tools actually reading Canon-specific MakerNotes section in EXIF. Put Canon DPP aside and this setting is just ignored.
Noise reduction on long exposure however will record a darkframe for each pic meeting requirement.

[ColinKennedy]

Walter, thanks for the tip! I don't use software that make use of that data so I'll definitely disable high-iso reduction. That'll let me speed up my intervalometer a bit and hopefully produce better ETTR results.
Kharak&garry23 - My exposure times go to its very limit, 32" ISO 3200. ETTR suggested this exposure at 8:30PM in the evening. It was still light out (I live in FL so the sun sets late), the photo was practically completely white. I understand that ETTR seeks right but what I'm seeing is completely unrecoverable-white data. Also, when I examined the XMPs for those photos, the EV values were +7, +9, etc. so the deflicker process was clearly not working properly. I'm not sure if it's a fault of ETTR necessarily but it was something to note.

I'll try another shoot tonight with my ETTR on "Press Set" instead of "Always On" (an idea I got from this forum http://www.magiclantern.fm/forum/index.php?topic=12846.0) and disabling all of Canon's custom functions. Otherwise that though are my settings OK?

Thanks for your replies, I'll look into LRTimelapse as well

[Walter Schulz]

That'll let me speed up my intervalometer a bit and hopefully produce better ETTR results.

I might be genuinely surprised if there will be a difference at all, apart from an EXIF value.

[garry23]

I think as has been said, you need to zero out the S/N settings, as these override the clipping.

This way ETTR ensure your scene is never overexposed.

Experiment with the settings in photo mode to better understand the impact of the S/N options. Rather than when doing your timelapse.

I think for a timelapse you will need ETTR always on, ie not SET, which is really for photo mode. Unless I've got it wrong :-)

[ColinKennedy]

@garry23

Just to be clear when you say S/N settings, do you mean the Midtone/Shadow SNR? Or something else?

Edit: Ignore what I just asked. S/N = signal/noise (SNR).

I'm starting to get the idea of the two settings now that I understand it as a ratio. I'm no longer using 8/4 because I believe that was causing my overexposure. Also, using SET behaves the same as Always On in intervalometer mode, just as an observation.

[garry23]

Yes. If you zer these out, only the highlight clipping will be used to set the ETTR exposure.

[garry23]

Also, be sure to experiment with % ignore, as you are doing sunset or sunrise.

If you have % ignore too low, eg 0, thus will ensure the sun is exposed correctly {:-)}, but you will find the rest of the scene underexposed.

As I said early, play around in photo mode to explore ETTR.

[a1ex]

Can you upload a CR2 after the point where exposure gets brighter and brighter?

If you can reproduce the issue, a DNG created with raw_diag may be even more helpful, as it contains exactly the same raw data used by ETTR. In a CR2, there may be further postprocessing applied to the image (such as hot pixel removal).

You should be able to reproduce the issue just by taking photos in the dark, with ETTR enabled, and without LiveView.