Flicker Free ETTR Timelapse: - -Beginners Guide & Basic Post Processing --

[DJHaze596]

You are missing the "\" before "*.ufraw"
ufraw-batch C:\Users\Johnnie\Desktop\Timelapse\ufrawconvert\*.ufraw --out-type=jpeg

Still doesn't work.  i open the batch and it flashes for half a second and does nothing.

[a1ex]

http://www.cs.princeton.edu/courses/archive/spr05/cos126/cmd-prompt.html

[mSpider]

No questions....

[Morghus]

Hi! I tried this feature last weekend and it worked well - it kept ETTRing all night and it resulted in this:

post processed using LRTimelapse, After Effects and GBDeflicker but as you can see I was not fully able to remove flickering, during the transitions there's still some flicker left and yes, I adjusted the frames that have different exposure settings so there's no flickering but here's my problem: two frames with exactly the same exposure have a different histogram. I'll attach those two raw files so you can see:

https://www.dropbox.com/s/q2ns6wv7vbc2pis/IMG_6296.CR2
https://www.dropbox.com/s/6kxr5msgzls3pf9/IMG_6295.CR2

As you can see shutter speed was slow so it should not be caused by shutter flicker neither should it be aperture flicker because I unscrewed the lens while holding down the DOF button. Of course I can fix these two frames manually but there are many like these and they are hard to find, and neither GBDeflicker nor LRTimelapse were able to detect and remove it automatically. It's driving me crazy - is there some setting I forgot?

I was using a 650D with a EF 10-22mm lens.

[a1ex]

Looks quite nice.

Did you use ML post deflicker? do you still have the sidecar files for these two shots?

[Morghus]

Thanks, yes I did use ML post deflicker but I couldn't get it to underexpose during the night - it was trying to brighten up the images considerably resulting in a lot of noise so I started using LRTimelapse which does luminance analysis and exposure compensation as well, or did I miss something?

Here are the sidecar files:
https://www.dropbox.com/s/m2pzmchfl1en38q/IMG_6295.XMP
https://www.dropbox.com/s/m3zmhhvr1svufu5/IMG_6296.XMP

In any case, the EXIF shows that these two RAW files should have a similar histogram because they are exposed exactly the same. Could it be that some in camera post processing like High ISO speed NR, Auto Light Optimizer or something is to blame? I think I turned everything off but there might be something I missed.

Is it possible to modify the ETTR module so it only adjusts in full stops and not in thirds? Or even better, make it delay exposure changes a frame or two so there are no unnecessary changes in the exposure settings, that would make it considerably easier to post process manually. If there's a way to make this work perfectly it'd be fire and forget for holy grail timelapses.

[TheJuice]

Hi,
There is something that's still not clear for me about ETTR and Timelapses :
Before I explain myself, I guess that you all use ETTR with the "Always On" setting, as I don't see the other settings as useable in a Timelapse configuration...

As I understand ETTR, it uses the picture that has just been taken to decide new settings to be set for the next picture. But that means that the next photo will be taken with the correct settings... that were decided from the previous photo ! Am I correct ?
What I mean is that every photo taken may have wrong settings : with a one-photo-settings shift.

Maybe I a missing some point here, but if I am correct the ETTR would work only for short interval timelapses, without too much change between each picture. If there is too much change in lighting, the ETTR module will try and keep-up, but will always be left behind by one photo...
Another way of ensuring correct exposition of all the pictures would be to set ETTR in "auto snap" mode, and then get rid of all badly exposed photos in post... Without knowing if it would be necessary to delete 2, 3 photos, or none at all if the photo was well exposed to begin with...

[a1ex]

Is a 0.1 EV difference that important?!

Or are you taking one picture per hour?

[TheJuice]

I'm just trying to understand :-)
If the difference is so low as you say, like 0.1EV, then it's no bother indeed.

I'm not taking one photo per hour... but I could :-p
Even one every ten minutes **could** be too much... I don't know, as I said I am trying to understand exactly how it works and how to use it at its best.

Would it be possible to implement yet another mode : pre-calculation ?
It would take a "phantom" picture to compute the histogram, then take the real picture based on the new calculated ETTR settings ? Like an auto-snap, but resulting only in the final image...

[a1ex]

You have the full source code, feel free to modify it. I don't see the point for this.

[andyshon]

I've had quite a few tries with ETTR timelapse now and I'm very, very impressed. It's the best ramping solution I've tried, and I've tried a fair few. Most of what I've shot has ended up in the bin due to my crappy photography, or the rebellious British weather, but the two shots bellow have worked out ok. Besides the fly on the lens in the first one, and the motion control going tits up towards the end. But ETTR has performed admirably in all.

These have been shot essentially with default ETTR settings, bar increasing the max exposure time and setting the sidecar to XMP. Processing is limited to static white balance and curves, plus ML deflicker. Ramping over a 13 stop range in the first shot!

I've got a few questions/requests if you don't mind.

Is it possible to process these same files through dcraw, using the deflicker in the xmp files? I'd like to compare results but I'm a bit wet behind the ears when it comes to the command line.

Am I right in thinking that if I tweak the deflicker target down a stop, this will offset the post gain applied by the same value? Would it be possible for this to be a slightly more intuitive scale? Not knowing how the algorithm works setting -4EV to get essentially unity makes no sense to me. And could it be adjustable in finer increments, half or third stops?

A utility that allowed you to offset exposure gain in post, whilst maintaining deflicker, would be very handy. Does anyone know of a way to do this?

I've had a few dropped exposures. Not many but enough to be a concern. Am I pushing it to do a shot every 12 secs with a max exposure of 4 secs on a 5D2? They seem to happen when the exposure is on the long side, but that's an impression, not carefully tested fact.

The resultant RAW files sometimes seem to lack lens correction data, but not always. Easily rectified but I thought worth pointing out.

Is there any way that starting the intervalometer could automatically turn the LCD brightness right down? Be a really handy power saving feature.

My biggest request is the obvious one, speed. The amount of time it takes to process is a limiting factor in these day-to-night type situations. Any little optimisations here would make a big difference.

Once again, I tip my hat to you chaps. And I think Canon ought to send you a very nice christmas card, with a very big cheque inside!

[glubber]

Is it possible to process these same files through dcraw, using the deflicker in the xmp files? I'd like to compare results but I'm a bit wet behind the ears when it comes to the command line.

dcraw can't read the xmp files. Ufraw would read the ufr-sidecars..But I don't know a way to interchange between xmp<-> ufr.

A utility that allowed you to offset exposure gain in post, whilst maintaining deflicker, would be very handy. Does anyone know of a way to do this?

I would be using LRTimelapse for manipulating the xmp files. Adjust some keyframes in LR an LRTimlapse will interpolate beetween them.


Edit:  Btw.... great work! :-)

[andyshon]

I would be using LRTimelapse for manipulating the xmp files. Adjust some keyframes in LR an LRTimlapse will interpolate beetween them.

I don't know of a way to adjust exposure in LR or LRTimelapse without loosing ML deflicker data, and from tests so far I'd say the ML deflicker works better in this situation than LRTimelapses, not to mention the time saved... Perhaps if we all peck him on his forums, he'll add ML-XMP import to version 3. 
http://forum.lrtimelapse.com/Thread-must-xmp-be-initialized

[mvejerslev]

Is it just me, or does anyone else see a certain choppyness to the motion in the timelapses from ML ETTR timelapse? I see it in my own tests and in the other videos here. A certain constant rythm to the motion. Perhaps from varying exposure time?

[brapodam]

Not knowing how the algorithm works setting -4EV to get essentially unity makes no sense to me. And could it be adjustable in finer increments, half or third stops?

Your photography isn't crappy

K, back on topic. I do agree with you on the finer increments. Sometimes the difference between 1 stop can be very significant.

A utility that allowed you to offset exposure gain in post, whilst maintaining deflicker, would be very handy. Does anyone know of a way to do this?

Yes we do indeed need a utility to offset exposure gain in post. I asked that a while back and the devs said you could write a script to offset the expo comp values in the sidecar files, but I don't know how to do that. I did also ask for help on writing a script to do that but got no reply.

I don't know if it will work for you but you could try the script in http://www.magiclantern.fm/forum/index.php?topic=7022.0
Do not use it with post deflicker. You also have to change the shadow_level (to 1) and highlight_level (to something very high like 500000) to disable any recovery.

[glubber]

I don't know of a way to adjust exposure in LR or LRTimelapse without loosing ML deflicker data, and from tests so far I'd say the ML deflicker works better in this situation than LRTimelapses, not to mention the time saved... Perhaps if we all peck him on his forums, he'll add ML-XMP import to version 3. 
http://forum.lrtimelapse.com/Thread-must-xmp-be-initialized

Ok... i must admit i only did this once  and i did it the most cumbersome way:

- developing CR2 + ETTR-XMP in LR (flicker free but too dark)
- exporting the developed files to DNG (to maintain the raw-data)
- going into LRTimelapse, doing the usual workflow including the exposurelift in LR but w/o LRT-deflickering

It's a hard way, but i'm still able to tweak my photos in Lightroom postshooting.

[andyshon]

@mvejerslev
I think I know what you mean, these sequences do have a look don't they. They're always going to have a relatively small effective shutter angle, which I'm sure is part of it. But perhaps there's more than that, can't quite put my finger on it, but...

@glubber
After some searching it seems this is simple to do with exiftool. A command of:
exiftool -Exposure2012+=-0.5 /path/to/ettr_sequence_folder
will take 0.5 EV off all the exposure values.

[Morghus]

After messing with Lightroom and LRTimelapse a long time (export to DNG with corected WB, then normal LRTimelapse workflow) and then manually removing some remaining flicker, I ended up with this:

18 EV ramp down and up. There were a few unnecessary adjustments, especially when adjusting ISO along with shutter speed, but nothing you can't correct.

I'll try again this weekend, weather permitting, and use the sidecar files before converting to DNG.

[glubber]

@glubber
After some searching it seems this is simple to do with exiftool. A command of:
exiftool -Exposure2012+=-0.5 /path/to/ettr_sequence_folder
will take 0.5 EV off all the exposure values.

Thank you andyshon, i will give this a try!

[discocalculi]

Awesome stuff!

[andyshon]

Another ETTR example, this time performing admirably in some very changeable conditions.

I've shot with default settings and then used exiftool to scale exposures without loosing the deflicker. Processed through LR+AE. Not tested the dcraw workflow yet, having too much fun shooting. Shots that were next to impossible are now easy!

Is there a way to limit the max shutter speed as well as the minimum? Be great to have a bit more control over shutter angle.

Being able to damp the exposure compensation, so it let the shot get a little darker/lighter as ambient brightness changed, would also be great but I guess quite tricky.

[a1ex]

What command did you use for exiftool?

You can limit the slowest shutter speed, but not the fastest one (for obvious reasons).

[andyshon]

exiftool -Exposure2012+=-0.5 /path/to/ettr_sequence_folder will knock half a stop off all the exposures.

I was thinking for a shot like this you could set the shutter to between half a second and two seconds. Start around one second at 400 ISO and you've got three stops either way but without letting the shutter get too quick, which would help capture a bit of motion blur, and thus smoother video. But I guess changing ISO that much is undesirable?

[a1ex]

By default, ETTR slows down the shutter as much as it can, so adding an artificial limit doesn't make sense (it will only prevent it from exposing properly).

[andyshon]

Point taken.

Another thought. Is there any way the intervelometer could be set to take the next shot as soon as the ettr module has finished processing? Be sure you were shooting as fast as possible then, and the interval could also adapt to longer exposures as light levels dropped.