(Auto) ETTR (Exposure to the Right): -- History & Beginners Guide

[Audionut]

But when you increase the ISO, noise increases and the tonality quality decreases.

This is actually completely incorrect.  You're describing noise and bit depth here, I'll tackle bit depth first because it's the shortest explanation.

http://www.cambridgeincolour.com/tutorials/bit-depth.htm

14bit cameras are capable of producing 16,384 distinct shades of grey (colors).  Unfortunately, this is only when the histogram is pushed to the right and it nears overexposure  So as the histogram moves further to the left, there are less bits available to encode the data.  So extreme shadows may only have 4 bits (16 shades of grey) or less.  Since the reduction of luminance also reduces the possible number of colors, 4 bits may be enough.

The issue arises where there is wanted detail in these extreme shadows, and post processing is used to increase the luminance.  As the luminance increases, more colors are capable of being reproduced, however since the original data may have only contained 16 colors, trying to generate (for instance) 120 distinct colors from 16 colors is a lesson in frustration.

Bit depth is only a concern to the user if he decides to post process.


Noise is a little more complicated since there are various sources in digital photography.  There are two main sources of noise in everyday photography, shot noise and read noise.
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html

The sun, a light bulb, or any other light source emits light.  This light is made up of particles called photons.  These photons bounce all over the place, with things like mirrors reflecting lots of the photons, and a black hole reflecting none.


The first thing a photo/videographer should understand is that ISO does not control exposure.  Period!
Exposure is the capture of light (photons) from a scene, and this can only be controlled with shutter and aperture.  Aperture determining the size of the opening though which the photons travel through the lens, and shutter controlling the length of time that the sensor is exposed to the photons.  Open the aperture and you create a bigger hole for the photons to flow through.  Increase the shutter time and you create a longer time period for the photons to flow.

Your exposure settings control the shot noise.  Shot noise is the square root of the number of photons.  So shot noise actually increases with increasing light, but luckily for us, the number of photons increases faster then the level of noise.  More light is better, always!  So in terms of noise, a photographer should always attempt to capture as many photons as possible, only being limited by motion (shutter) and depth of field (aperture).

ISO is used to control the sensitivity of the sensor to light (read noise). 

http://www.clarkvision.com/articles/iso/index.html

Code: [Select]

                                Bright                              Dark
Total dynamic range of scene   [****************************************]
ISO 100                        [*************************]
ISO 200                            [************************]
ISO 400                                [***********************]
ISO 800                                    [**********************]
ISO 1600                                       [*********************]
So ISO 100 for example may be able to capture up to 80,000 photons of light, but since it is rather noisy on Canon cameras, once the photon count drops below 40, there isn't enough light to outweigh the noise produced by the camera.  Whereas ISO 1600 for example may only be able to capture up to 5,000 photons, however, it can capture photon counts as low as 4 before the noise produced by the camera is greater then the number of photons captured.  These are example numbers, but I'm sure you get the point.

So one thing that should becoming clear is that higher ISOs do not increase noise.  In fact, higher ISOs are less noisy.  Higher ISOs can capture a smaller number of photons, before that number of photons is less then the noise produced by the camera.  Higher ISOs cannot capture the same amount of photons (light) as lower ISOs, but a photographer should only be using higher ISOs when the number of photons (the amount of light) is lower then the maximum for that ISO.

Where the lines get blurred is due to the shot noise.  So we have a dark scene, we crank the ISO up, but the image appears to have more noise then an image taken at ISO 100, with the same rendered brightness.  That's simply because the light itself that has been captured, contains more noise.  Darker scene = less light = less photons = more (shot) noise. 

It is very easy to see that higher ISOs have less noise then lower ISOs.  Using ETTR principles, take a shot with the camera set to ISO 1600, then without changing any other settings, take another shot at ISO 100.  In post processing increase the ISO 100 shot by 4EV and observe the results.


TLDR:  Set the shutter speed to capture the motion.  Use ISO as necessary to ETTR.

[Danieloncarevic]

Hi there.

I'm quite new with timelapses and never heard of ETTR before I came across this thread.

From the little I've read, the ETTR is the solution for timelapses where the exposure is changing a lot like sunrises/sunsets.

Did I get the basic part right? I hope so.

Anyways, I tried to follow the steps in this thread and some other instructions I found online in order to get a proper exposure during my timelapses, especially during sunset ones.

So far, I got bad results. Really bad ones.

The problem I've noticed is that the ETTR is over-compensating too much. So instead of the scene to go gradually darker it gets gradually lighter which is not natural and looks quite bad.

And I'm talking about very short timelapses here, not all day long. Like 5 seconds long. (120 frames, 24fps)

So far I didn't have time to play too much with it but first I used it on the default setting and then I tried tweaking it a bit hoping for better results which failed even more.

So the first frames are good (because it's the exposure I set). The contrast is good, the colors are good. Everything is fine. As the frames are passing, the scene gets lighter, the contrast is lowered and the colors become dull.

I'd post an example but I was so frustrated with the results that I deleted all those failed attempts.

I know this is not a thread about timelapses and that there are million of instructions online but I was just wondering if someone had some suggestions for me.

I have few timelapses shot in manual mode where the light didn't change and that's fine. No special skills are need there anyways.

But when the light changed a lot I tried using manual mode which resulted in the scene getting too dark very fast. I tried aperture priority where I got the best results but I wasn't able to correct the flickering in post production even using LRTimelapse software. And with ML and ETTR I got the worst results which I already mentioned.

Any advices would be much appreciated.

Thanks.

Daniel

[garry23]

@Danieloncareric

Go and watch the videos on the LRTimelapse site. For example.

You need to do some post processing.

[Danieloncarevic]

I did watch the LRTimelapse tutorials from the original creator but I still couldn't get anything useful.

And I haven't found any video where he mentions the ETTR which is my main trouble and the essence of this thread.

[l_d_allan]

> TLDR:  Set the shutter speed to capture the motion.  Use ISO as necessary to ETTR.

Thanks for the thorough explanation. As I expected, my "rule of thumb" was less-than-informed.

BTW: This issue is being written about on the DPR web-site:
"What's that noise?"
http://www.dpreview.com/articles/8189925268/what-s-that-noise-shedding-some-light-on-the-sources-of-noise

[garry23]

But don't blindly use ISO with ETTR.

Up go about 1600 on a 5DII, http://www.clarkvision.com/articles/iso/

[dmilligan]

The problem I've noticed is that the ETTR is over-compensating too much. So instead of the scene to go gradually darker it gets gradually lighter which is not natural and looks quite bad.

If ETTR is overexposing (non-specular highlights are getting clipped to white), then check your SNR limits, they're probably too high. If it just looks "too bright", but nothing is overexposed, well that's the point of ETTR. You will simply turn down the exposure in post to make it "look right", the result of this will be identical to using lower exposure in camera, except with less noise. This is the whole point of ETTR.

For deflickering you can try my bridge script, or the deflick.mo module (deflicker in camera).

[yannick_c]

Hi,

I've done my first AETTR night to day timelapse yesterday with my Canon 550D and ML v2.3 with an interval of 25sec and slowest shutter at 20sec. Everything went well except a little problem. The shutter speeds written in the EXIF data into the .CR2 files seems wrong for some pictures.

As an exemple here as some exemple of 2 consecutive pictures with different luminosity even the exif file show the same shutter speed value (that's not apertuer flickering cause I used the lens twist technique and the weather didn't change between these 2 pictures).



Maybe AETTR use different shutter speeds than the ones from the 550D camera (with shorter interval than 1/3 EV) and that's the ones from the camera that are written in the EXIF data but it's just a supposition.

Is there a way to have the proper value in my CR2 files except using the deflicker module? (I don't want to apply deflicker in the camera cause I should increase my interval to let it calculate deflicker. Also I'm using LRTimelapse to postprocess my timelapses and I need to keep the exposure initial value at 0 to let me change it as I want on postprod). Or maybe there is a way to let the deflicker module only create the xmp files (with the correct shutter speed value) but without applying deflicker.

These bumps in luminosity without the right shutter speed value make LRtimelapse unable to find the right HolyGrail keyframes and induce big bumps after applying the HolyGrail compensation.
Here is what it looks like after the HolyGrail compensation. (The pink curve shouldn't have these crazy bumps with right shutter speed value in the CR2 files)



Any suggestion to sort that out will be great.

Thanks

Yannick

[Audionut]

25 seconds is a long time.  My guess is your scene luminance changed.

AETTR is not designed to produce the same overall exposure between shots.  It's designed to ETTR.

[dmilligan]

The EXIF values in the CR2 are correct. There's really no way they could be wrong. ML does not interfere with this process in any way.

I noticed that in your screenshot it says f4, yet you say you used the lens twist trick. If that was true then the camera can't communicate to the lens and there shouldn't be any aperture EXIF information (normally you get something like f00). So how did this information get there?

[yannick_c]

Thanks for the answers.

I had to use 25 sec interval in order to shot the stars for the night part of the timelapse.

If the scene have changed (for exemple with a cloud going in front of the sun), the luminosity would only have change on the ground and not in the sky. Here, that's the overall luminosity that have changed and the sun was not even out at that time.

I've well understand the purpose of ETTR to change the overall exposure between shots but when the ETTR make changes, my shutter speed should have changed in the EXIF. Here, the values are the same. There are 24 consecutive pictures with a shutter speed of 1/200 sec and there's just a big bump after the 13 first ones. The same weird bump happen after 7 of the 19 pictures at 1/100 sec.

The next exact simlar bumps only happen when the ETTR change the shutter speed to 1/100, 1/125, 1/160, 1/200 or 1/250 (I only talk here of the small part of the all sequence)

That's why I was assuming ETTR may take pictures with shutter speed different from the Canon Camera like 1/115 or 1/225 and writting the wrong value in the EXIF. I should look at the code to understand it better.

But thanks for you suggestion even I don't think the problem comes from the scene that have changed.

Dmillihan,for the f0, you're right, I've change it afterwards in LRTimelapse and added it on the capture to show the lens original value.

(how can I active auto notification on the forum? I've activated it in the options, but don't see where I can switch it on a post, it doesn't appear in the "Attachement and other options")

[dmilligan]

It is only possible to use Canon shutter speeds (however the values Canon displays aren't always rounded correctly, e.g. what Canon calls 30" is actually 32")

The reason I asked the question I did was that I assumed you must have changed the metadata yourself, and so is there a possibility you also accidentally modified the shutter metadata as well?

If you really think that metadata is truly wrong from the camera, then find a way to reproduce the issue on a static scene and upload some untouched example CR2s.

(Email notifications don't work on this forum, AFAIK it's disabled due to bandwidth constraints on the mail server, with the exception of a few threads and PMs. You can, however, use the tapatalk app to get notifications.)

[Audionut]

If the scene have changed (for exemple with a cloud going in front of the sun), the luminosity would only have change on the ground and not in the sky. Here, that's the overall luminosity that have changed and the sun was not even out at that time.

25 seconds is an extremely long time for photons to hit the sensor.  Your eyes won't notice the exposure difference you see in the images above, because you are there and the exposure change is gradual (your eyes/brain taking constant exposures).  The camera on the other hand, only sees 25 second blocks of luminance, and it doesn't capture gradual changes to scene luminance during the exposure.  It simply captures the total number of photons hitting the sensor during that time period.

During sunrise/sunset, scene luminance constantly changes.  In the brighter image, more photons hit the camera sensor.

[yannick_c]

I think I have spotted the problem. It comes from the file shoot.c

Looking at the source code, the shutter speed values are rounded like that :

Code: [Select]

int expo_value_rounding_ok(int raw, int is_aperture)
{
    if (is_aperture)
        if (raw == lens_info.raw_aperture_min || raw == lens_info.raw_aperture_max) return 1;
   
    int r = ABS(raw) % 8;
    if (r != 0 && r != 4 && r != 3 && r != 5)
        return 0;
    return 1;
}

int round_shutter(int tv, int slowest_shutter)
{
    int tvr;
    tv = MIN(tv, FASTEST_SHUTTER_SPEED_RAW);
    tvr = MAX(tv    , slowest_shutter); if (expo_value_rounding_ok(tvr, 0)) return tvr;
    tvr = MAX(tv - 1, slowest_shutter); if (expo_value_rounding_ok(tvr, 0)) return tvr;
    tvr = MAX(tv + 1, slowest_shutter); if (expo_value_rounding_ok(tvr, 0)) return tvr;
    tvr = MAX(tv - 2, slowest_shutter); if (expo_value_rounding_ok(tvr, 0)) return tvr;
    tvr = MAX(tv + 2, slowest_shutter); if (expo_value_rounding_ok(tvr, 0)) return tvr;
    tvr = MAX(tv + 3, slowest_shutter); if (expo_value_rounding_ok(tvr, 0)) return tvr;
    tvr = MAX(tv + 4, slowest_shutter); if (expo_value_rounding_ok(tvr, 0)) return tvr;
    return 0;
}
It seems to do that in order to have shutter speed values that respect 1/3 or 1/2 EV increments (there is the same thing in lens.c but only in movie mode) so it let the choice between 4 shutter speed values for each 1EV step

Code: [Select]

if (r != 0 && r != 4 && r != 3 && r != 5)
On my Canon 550D (maybe it's not the same on other cameras) for each 1EV step, there are only 3 values possible : for exemple 1/100, 1/125 and 1/160

If I look at all my pictures, there are 4 bumps for each 1EV step (which is normal looking at the code) but there are only 3 values possible for the shutter speed in the EXIF data so there is one bump that is not repercuted in the EXIF cause the shutter speed value doesn't exist in the camera even it used by the ETTR. These non repercuted bumps happen at each stop with values like 1/50, 1/100, 1/200, 1/400, ...

Maybe the cause is the lens twist trick that let ETTR change the shutter speed value without being able to see if it's ok or not (I've seen some waiting for changes confirmation in lens.c)

But even if it's the lens trick that causes it, it would be great to let the camera only choose between the 3 values available for each step and not for 4.

Yannick

[dmilligan]

On my Canon 550D (maybe it's not the same on other cameras) for each 1EV step, there are only 3 values possible : for exemple 1/100, 1/125 and 1/160

Change your Canon settings to 1/2 EV step instead of 1/3 EV step and you will see another possible value.

cause the shutter speed value doesn't exist in the camera even it used by the ETTR.

This is not possible, ETTR cannot use a shutter speed that "doesn't exist in the camera". ETTR simply asks the Canon firmware to change the shutter speed (in essence, ETTR simply clicks some buttons for you). Sending the Canon firmware an invalid shutter speed can brick your camera, that's why values are rounded in ML code in the first place.

[yannick_c]

Thanks for the infos dmilligan, it's more and more clear in my head now. I've seen the 1/2 values and it seems that there are these values that are not repercuted in the EXIF data.

In the picture below, you can see the different bumps in luminance and the shutter speed values written in the EXIF. As you can see, there are bumps that are not repercuted. These bumps are corresponding to the 1/2 values. As you said, these values exist in the camera, ETTR use them, but the Camera seems unable to write the proper value in the EXIF data (and that's what I don't understand here). The Camera writes the next 1/3 value instead of the 1/2 one.



Yannick

edit : Preview clearer

[Audionut]

I simulated a sunset timelapse with camera on tripod, and a constant light source that was moved away in small increments. 
Results were as expected (6D).  That is, exposure dropped as light source was moved away, until limit reached and next shutter value used.  No half stops with camera set to 1/3rd.

For clarity yannick_c, those half stop bumps you marked in the image above should be at the marked exif boundaries.  1/160 > 1/180 > 1/200, but it looks like you mark 1/160 > 1/200 > 1/180 > 1/250.

Can you upload some unmolested CR2s, say from 1/160 through 1/320?

[yannick_c]

I've changed the preview for better comprehension. I'm uploading the CR2 files (1.4 Go, should be ok in 4 hours ) I have the original JPG also that are less big, but I think you will prefer the CR2.

[yannick_c]

I've uploaded 6 CR2 files (the first 3 that are obviously taken at shutter 1/180 (but tagged 1/200) and 3 after the bump that are tagged correctly are 1/200).

http://lets-fall.com/timelapse/Timelapse_shutter_180_200_bump.zip

I'll upload the 1.4 Go zip file with more CR2 tonight (it's 4am here).

Thanks for your help.

PS : it's a sunrise timelapse

[yannick_c]

And here are all the CR2 from shutter 1/160 to 1/320 : http://lets-fall.com/timelapse/Timelapse_cr2_shutter_160_to_320.zip (1.4Go)

With your test reproducing indoor sunset, did you use the lens twist trick or not? Maybe it's the fact that the lens is disconnected that lets the ETTR module choose a shutter speed with 1/2 stop (even if the camera is set to 1/3rd). I don't remember hearing 2 beeps (just 1 beep) when ETTR changed the shutter speed value during my timelapse, it could be due to a non confirmation of the new shutter value by the lens.

[dmilligan]

The lens has nothing to do with the shutter and the lens does not "confirm" shutter speeds, so this explanation is unlikely.

One beep means that ETTR settled on a solution, two beeps means that another iteration is probably needed, three beeps means exposure limits reached.

[yannick_c]

Hi,

I tried to see what I can find in the EXIF Data (from Jeffrey's Exif Viewer online here : http://regex.info/exif.cgi)
I've uploaded the 2 jpg pictures where a bump happens between "assumed" shutter value of 1/180 and the next at 1/200 (the 2 pictures from the preview)

First, I didn't think that there were so much informations stored, there's a lot of interesting things.

Here what I've found in it (1st value for picture assumed taken at 1/180, 2nd value for picture after the bump to 1/200)

EXIF

- Shutter Speed Value : 1/197 - 1/197
- Exposure Time : 1/200 - 1/200

MakerNotes

- Target Exposure Time : 1/203 - 1/203
- Exposure Time   : 1/181 - 1/203
- Exposure Time : 1/197 - 1/197

Composite
This block of data is computed based upon other items. Some of it may be wildly incorrect, especially if the image has been resized.

- Shutter Speed : 1/200 - 1/200


I don't know what these MakerNotes are and why there are 2 lines with Exposure Time but you can clearly see that the first is taken at around 1/180 and the second at around 1/200 even the two are tagged 1/200 in the EXIF Data.

I've tested with 3 pictures before and 3 after the bumps. The 3 before have that line "Exposure Time" = 1/181 and the 3 after have "Exposure Time" = 1/ 203 in the MakerNotes

Here are a zip with the jpg pictures (3 before the bump, 3 after) like the 6 CR2 I've uploaded before : http://lets-fall.com/timelapse/Timelapse_shutter_180_200_bump_original_jpg.zip

If there's nothing that can be done to get the correct value from camera, maybe an easy script could replace the wrong EXIF data with that correct stored value and write it in the xmp file.

Yannick

EDIT : the correct Shutter Speed Value is easily found with exiftool with the tag -Makernotes:ExposureTime

[yannick_c]

I have found a way to correct all my .CR2 EXIF Data with the correct Shutter Speed Value with a simple line of code :

exiftool "-EXIF:ExposureTime<Makernotes:ExposureTime" CR2_Directory

where CR2_Directory is the folder with all my CR2 files.

Maybe it's not the best to directly change the original CR2 source file like that ?

[yannick_c]

Here is the LRTimelapse preview for luminance (pink curve) with both non corrected and corrected EXIF shutter speed info.




LRTimelapse HolyGrail correction still have some problem when ETTR changes values every 1,2 or 3 pictures (the big bumps at the center) but if you look at the second part of the curve, it's more smooth now and the weird bumps with the 1/2 EV shutter values have now disapeared.

LRTimelapse will then do the job deflickering the all sequence (even with the non corrected version), but it's still better to start from right exif values (even if I don't know if the 1/181 and 1/203 are really the "real" exposition time but it's more accurate than 1/200 which I had before for both).

[Paul Maier]

Hi everyone!
I'm pretty new to timelapses, but i have been using auto ettr successfully in the last weeks. My Question is: Is it possible to set a minimum aperture for the ettr tool? I'd like to get the maximum sharpness with my standard objective and thats just not possible at the widest open aperture.
camera: canon eos 600D
Any Tips would be great.
Paul