Flicker Free ETTR Timelapse: - -Beginners Guide & Basic Post Processing --

[dmilligan]

I was shooting a time lapse during daytime so maybe ETTR isn't needed in this situation?

Right. If you're scene lighting isn't changing then you don't need this. Just set the exposure in M mode and shoot your timelapse.

Is it best for changing light scenes (ie. sunrise/sunset)?

Exactly. Shooting a sunrise or sunset is much more challenging (or impossible) without this workflow.

[zeinikuzeiniku]

Thanks. That is what I thought when I was looking into all of this.

I noticed when I was doing some tests that the ETTR was jumping up and down from one shot to the next. How do you get it to even out?

Will this be mitigated in post delficker even if ETTR is "hunting" around for the correct exposure?

[dmilligan]

Will this be mitigated in post delficker even if ETTR is "hunting" around for the correct exposure?

Yep. The deflicker script takes care of it.

[zeinikuzeiniku]

Well this is some pretty powerful stuff then and will be so useful! In fact, this is the main reason I haven't tried time lapse because of the flicker issue. This will help out a lot without extra equipment or software for the time being. I'm very grateful for this script.

Also, when people talk about ramping exposure via the script what does this exactly mean? Manually adjusting certain groups of frames for correct exposure?

[dmilligan]

Gradually/smoothly adjusting some parameter (could be exposure, could be WB, contrast, or anything else) over time from one value to another value (like key framing). I use the ramping a lot with WB in day to night timelapses since the WB changes drastically over this time period. In a sunset it typically goes from daylight around 5500 up to 6500-7000 during twilight and then way down to around 3500 once it's fully dark. I use the ramping to smoothly change the WB like this so that there isn't just a sudden noticeable change in WB. You can do the same for any other ACR setting.

There's a special ramping mode for exposure that is additive. This allows you to preserve the deflicker but change the overall exposure (either ramp it or just increase/decrease it by some constant value for all frames). I prefer to just set exposure first then run the deflicker, but you can do it either way.

[smier]

Just read the entire thread, one question. Are the post processing XMP and UFRaw guides in the OP two different methods that produce similar results or are they both required? If the former, why wouldn't everyone just use XMP since it's less than half the steps of UFRaw? There must be some sort of advantage. Maybe I misinterpreted the OP.

[dmilligan]

If the former, why wouldn't everyone just use XMP since it's less than half the steps of UFRaw?

Not everyone owns (expensive) Adobe software. UFRaw is free software.

[smier]

Not everyone owns (expensive) Adobe software. UFRaw is free software.

That makes sense, thanks for clarifying.

I just did my first test yesterday evening. It worked out pretty good but still had some flickering. I looked at the XMP values and several times they jump quite large distances. I had a Highlight Ignore of 5%, 5 Midtone SNR, 2 Shadow SNR. Any advice? This is on a 6D

[a1ex]

To troubleshoot this, you can download raw_diag.mo from here, and select the "Dump RAW buffer" option. This will save a DNG copy of each picture (it's the raw buffer used for ETTR, among others). If there are obvious differences between the CR2 and the DNG, upload them.

[smier]

Thanks Alex, I'll try this tonight after work!

[smier]

So I just had another go, seems to be a lot less flickering when I increased the highlight ignore to 8% from 5%, but it still occurs.

Here's the video

around the 30 second mark there's several flickers. I found the frames that occur at one of these flickers. Between 8564 and 8565 the camera exposure settings are the same and there is no significant change in the scene lighting but the deflicker exposure value changes significantly. On 8565 the shutter speed changes but the deflicker exposure value correctly compensates it.

CR2, XMP and diagnostic DNG files can be found here

So I guess the camera just didn't calculate the correct post deflicker exposure value?

[a1ex]

If you open the two DNGs (8564 and 65) and apply the exposure compensations from the XMPs (2.14 and 1.81) you will get two identical exposures. This is confirmed on the histograms of the developed JPEGs (in gimp for example, it prints the median value, which is where metering is done at default settings).

So, the deflicker algorithm worked well.

However, if you apply the same exposure compensations on the CR2's, you will not get identical exposure.

BUT

If you extract the raw data (dcraw -4 -E *.CR2 *.dng) and you compare it pixel by pixel, they are 100% identical. There is a vertical offset of 14 pixels, if you want to try. Of course, between 8564 and 65 there is a visible difference in the clouds (surprisingly enough, the tree leaves did not move at all).

So we have two sets of images, with 100% identical raw data, which get rendered with different exposures. What could be the difference?

I'll let you guess the answer. Hint: bug confirmed, will fix. Great catch.

[smier]

Very interesting! Glad I could help troubleshoot some stuff. Let me know if there's anything more I can test.

[a1ex]

Here's a quick fix: deflick.mo, let me know if it works.

(I'll push the code tomorrow, since it contains the answer to the above riddle, and I don't want to spoil it right now )

To fix existing footage, I recommend dmilligan's deflicker script: http://www.magiclantern.fm/forum/index.php?topic=8850

[smier]

I'll test the module when I get home. Remoting to my home computer and trying out the script you linked, procrastinating at work

[smier]

The module seems to be fixed! Zero flickers in today's time lapse. Had to cut it short cause it started to rain, you can see the leaves start to droop at the end.

1080p

4k glory!


An observation — to save transfer speed time I tried lowering my RAW size to medium and small but both had "RAW ERROR" displayed on the screen when it tried to calculate the ETTR. Looking at this thread it seems to be a known bug though.


Thanks the quick fix on the module. The script you linked also worked to deflicker the previous time lapse as well, thanks for that too.

[a1ex]

Working with sRAW/mRAW is an enhancement, not a bug. These modes have different raw buffer configuration than plain RAW, at least on one camera (the one I tried back then )

And after reading this ( http://photography-on-the.net/forum/showthread.php?t=730030 ), I think it's better not to support these modes. Also, all my current and upcoming raw processing tools will rely on plain RAW quality setting.

edit: the main image in the POTN thread is down right now; here's a mirror:

[smier]

I've noticed during a night time lapse that my 6d only uses full stops of ISO (100, 200, 400, .., 6400) and not any of the intermediary ISO levels between the full stops. Really makes night scenes less flexible to minor changes compared to day scenes. Is there any plan to modify this in a future nightly?

[a1ex]

No, because you will not get any improvement by using intermediate ISOs. The deflicker module is there for a reason.

[smier]

Is this because shooting different ISOs and modifying exposure in post result in the same image? For example, if I take a photo at 5",  ISO6400 with the exposure lowered equal to ISO4000, and take another photo at 5" ISO4000 they will result in the exact same image?

Does it work in reverse too? An image shot with ISO4000 with it's exposure increased in post equal to ISO6400 will look like a photo shot at ISO6400?

[Audionut]

Because intermediate ISOs, are by and large, simply digital manipulation of the analog ISOs.

Analog ISOs are useful, digital ISOs are not.
http://www.magiclantern.fm/forum/index.php?topic=10111.0

[smier]

Great thread, I had no idea.

[CreativeEndeavor]

It is a rare, sunny, blue day here in Seattle without a cloud in the sky.  I am headed up to Kerry park to capture an ETTR sunset timelapse of the Seattle Skyline with Mt. Rainier in the background between 8 to 10 P.M.  Sunset is at 9:11 P.M.

I have a Quick Question about step 17.  Take two or three pictures until the RAW histogram ETTR hint is less than 0.2

What should I see after taking a few initial test shots?  Does ETTR automatically adjust the ISO & shutter speed to get the .2 hint setting?  Do I need to change the Highlight/Midtone/Shadow settings in AUTO ETTR to get this .2?  I am a bit confused by what I should see and do at this point in the process.

Here are the settings I will be using tonight.  Any suggestions are greatly appreciated.

AUTo ETTR

• Slowest Shutter = 32 seconds
• Highlight Ignore = 2% (For downtown City lights.  No Sun or Moon will be in Frame.
• Midtone SNR = 5 EV
• Shadow SNR = 3 EV

Intervalometer

• Interval is set to 1 minute starting after 5 seconds from leaving the menu for 120 shots which will yield 5 seconds of footage at 24FPS

Post Deflicker

Adobe XMP, 50% Deflicker %, at -4EV target level[/li]

Ok thats I all I have for now.  Wish me luck.  I will post my results/questions soon as I process it.  Thanks a bunch!

[Audionut]

You just need to take some shots first for the algorithm to settle on the correct exposure settings.  Correct exposure settings may result in something other then 0.2 hint, depending on the settings you have chosen.
To speed up the process, I recommend shooting an underexposed image first, as AETTR will settle on the correct exposure settings faster (should only need 1 shot for correct settings).

Highlight ignore of 2% is probably a little high for city lights.  The issue here, is that 2% is very large for the twilight period, and will probably generate to much unwanted overexposure.  Depending on how many lights there are, and more importantly, how much of the frame they occupy, you probably want to try a smaller percentage.

[CreativeEndeavor]

Thank-you Audionut for the response,

After shooting a timelapse last night, I am understanding the process a bit better.

Just have a couple more questions:

• Is the small number on the Global Overlay Histogram Panel the ETTR Hint?
• A couple times while I was taking a long exposure, Magic Lantern stopped taking pictures because someone passed in front of the frame.  Did ETTR/Intervalometer/DeFlick stop because their was a dramatic shift in the exposure due to a large  figure blocking out the shot for a few seconds?

Thanks again for the input.