New AETTR Update Improves Usability

Started by RenatoPhoto, September 02, 2013, 02:27:19 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

RenatoPhoto

The new AETTR module now has two new options which allow for better control of the highlight exposure function.  I have tested these options and obtained very impressive results..

In the past I had used the highlight ignore function to change the amount of overexposure in the highlights and this worked very well but it was time consuming.  In this update now it is possible to obtain some impressive results by setting the amount of noise that we are willing to accept in the shadows and midtones.  At first this does not lead to a quick understanding of the function and its application but by testing I have a better understanding on how to use it.

The midtones SNR limit varies from 1EV to 8EV.  I think of it 1EV = lots of noise and 8EV = no noise.  So if you want strict control of the noise in midtones then set it to 8 and the module will raise the highlight ignore until the midtones are perfectly exposed to the right.  If the scene has lots of dynamic rang then the highlights will be blown out (ignored) to maintain a proper exposure to the right in the midtones.

Here you can find a more details on this subject and excellent  samples by Audionut  on the midtones exposure: http://www.magiclantern.fm/forum/index.php?topic=5200.msg70150#msg70150
The shadows SNR limit varies from 1EV to 4EV.  I think of it 1EV = lots of noise and 4EV = no noise in the shadows.   So if you want strict control of the noise in shadows then set it to 4 and the module will raise the highlight ignore until the shadows are perfectly exposed to the right.

The most important feature of this options is that you can independently control these two limits, one is how much noise is in the shadows, and the other is how much noise in the midtones.  If we use both functions then the resulting image will depend on which of the two limits will be hit first.  By having two options measured at once, there is better control of the highlight ignore function in a much quicker way than setting the h.i. from  0 to 50%.

The module is enabled as default midtones at 6EV and shadows at 3EV.  This will ignore a great amount of the highlights in return of properly exposed midtones and shadows.   This is the equivalent of setting highlight ignore of 2 to 3%.  But if the scene has lots of shadows then the shadow limit  will kick in and raise the highlight ignore to say 10%.  This means that the module is automatically adjusting the highlight ignore to meet my shadow requirements.

By using these two settings now we have a much more usable module which adjusts the highlight ignore to meet our requirements.

From my testing:
Indoor photography ignoring highlights from windows set midtones = 6, shadows =3
For outside photography in bright lights only ignoring extreme highlights m=4,  s=2
I have also used it for night-to-day Timelapse and found that m=4 and s=2 gives some nice results.  Usually the shadows are very dark when the sun starts to come up, with this setting I get enough light in the shadows to bring them up in post for a good final compromise.  If I use m=6 and s=3 the night will be overexposed but the shadows in the transition will be better.  I don't like this option because it forces the camera to use very high ISO at night.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

Stedda

Thanks for the testing and info... I hadn't realized anything changed.
5D Mark III -- 7D   SOLD -- EOS M 22mm 18-55mm STM -- Fuji X-T1 18-55 F2.8-F4 & 35 F1.4
Canon Glass   100L F2.8 IS -- 70-200L F4 -- 135L F2 -- 85 F1.8 -- 17-40L --  40 F2.8 -- 35 F2 IS  Sigma Glass  120-300 F2.8 OS -- 50 F1.4 -- 85 F1.4  Tamron Glass   24-70 2.8 VC   600EX-RT X3

ouuzi

So this is if you want to clip the highlights.What about if you want no clipped ones?Do you set Midtones and Shadows to off,and stick to 0.2 %?

RenatoPhoto

Quote from: ouuzi on September 07, 2013, 09:11:05 PM
What about if you want no clipped ones?Do you set Midtones and Shadows to off,and stick to 0.2 %?
Yep highlight ignore = 0, Midtones and Shadows to off
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

ouuzi

Thanks a lot.I tried to shoot today and I had the default Values for mid tones and shadows on.And I was getting clipped highlights because of the hight dynamic range scene.Thanks again for the information.

ouuzi

Can you explain what exactly exposure target level does. I am using Lightroom and I save recover quite a bit in some shots,this is directly relevant with the target but if its possible an explanation would be awesome.

RenatoPhoto

Quote from: ouuzi on September 07, 2013, 10:45:39 PM
Can you explain what exactly exposure target level does. I am using Lightroom and I save recover quite a bit in some shots,this is directly relevant with the target but if its possible an explanation would be awesome.
I guess is needed to make the algorithm function properly starting from below the correct exposure.  The lower the ev the darker the image, so -2 would take picture -2ev from the correct ettr.  I dont use it, if you find a use for it please report back.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

a1ex

From ML help:
Quote
                .name = "Highlight ignore",
                .help  = "How many bright pixels are allowed above the target level.",

At highlight ignore = 1%, the image is metered at 99th percentile: that brightness value is adjusted to match the target level. That means, 99% of pixels will be darker than target, the other 1% will be brighter.  See http://en.wikipedia.org/wiki/Percentile

For deflicker purposes, you would meter at 50th percentile (median), so exactly half pixels will be darker than the exposure target, and the other half will be brighter.

You may say why not compute the average brightness (simple or weighted) and use that for metering? Answer: median and percentiles are statistically robust (they ignore outliers => less flicker). Average (mean) is not.

RenatoPhoto

I guess the question is about target level?  Not about highlight ignore

Quote from: ouuzi on September 07, 2013, 10:45:39 PM
Can you explain what exactly exposure target level does.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

a1ex

Good luck explaining target level without highlight ignore.

Audionut

Quote from: ouuzi on September 07, 2013, 10:45:39 PM
Can you explain what exactly exposure target level does. I am using Lightroom and I save recover quite a bit in some shots,this is directly relevant with the target but if its possible an explanation would be awesome.

Can you describe further what it is you are saving?  I assume it's highlights.

AutoETTR is first and foremost a white point selection, the white point being the brightest pixels that retain detail (not clipped).
If you have AETTR set to no clipping (0% highlight ignore, clipping mode set to no clipping, midtone and shadow limits off), there will be no clipped pixels.  Here, the exposure level target defines where to set that white point. An exposure level target of -0.5EV, tries to place the brightest pixels in the image as far to the right of the histogram as possible without pushing any pixels to clipping.

This might appear as if you have pushed to much detail to white.  It has, but you need to be aware that none of that detail has actually been clipped to white.  There is a defined difference.
Have a read of this page.  I have linked directly to the zone system on that page. 
You can see that there are 3 zones above the highlights that still retain detail.  These 3 zones are the areas where the camera is performing it's best.  This is where AETTR is very useful, as it's designed to make it easy to push as much of the detail into these 3 zones as possible.

When you take one of these images into lightroom, you might see an extreme amount of very bright pixels, this is a good thing.  Remember earlier that AETTR was set to ensure that no pixels have actually been clipped to white.  To fix such an image in LR, you simply have to lower the exposure.  Take the exposure slider in LR and move it to the left.

If you were to set the exposure target level to -2EV, you are telling AETTR to to make the brightest pixels in the image -2EV below clipped white.  The image might look like the correct exposure when taken directly into LR, but you have wasted 2 full stops of the sensors best performance.

Now, if we decide we want to allow some clipping (specular highlights, or just a simple exposure decision), the exposure target now becomes the point where pixels at this set clipping point are mapped.  So if we decide to allow 3% of pixels to be clipped and we set the exposure target to -2EV, those 3% of pixels will be allowed to be above the -2EV point and all other pixels will be below this point.

Now, if we decide to allow no pixels to be clipped to white (as per the first example above), but we have the midtone SNR limit set to 6EV, we are now saying, I don't want any pixels clipped to white, but if the midtone point of the exposure has a Signal to Noise Ratio lower then the point set in the midtone SNR limit settings, let's allow some pixels to be clipped to white to ensure that midtone level has the request Signal to Noise Ratio.

http://www.magiclantern.fm/forum/index.php?topic=5200.msg70150#msg70150

Audionut

I've just discovered something I love. 

I can use Autoexpo and set AETTR to autosnap.  So I get my correct exposure with autoexpo, and then I immediately get an exposure with the settings I have set in AETTR.
That's pretty awesome.

Dual-ISO is pretty much to the point where it can be used exclusively also.  Especially @ sane settings (100/400) :)

ouuzi

Ī¤hanks a lot for the detailed answer.
So to utilise the highlight recovery of Lightroom the optimal settings are to select clip any or clip green, so you will still have at least one channel not clipped ( that means that detail is there.Correct?).
And -0.5 EV is in a way a safety setting to go close to pure white but not 255 white.
Correct me if I understood all these details in a wrong way.

BTW Guys you are the best.ML rocks

kgv5

Sorry for asking that but where could i find.mo file with the new ETTR features (if it exists)? my ML ver is 22 Aug. The link with the last commit doesn't seem to have it ready for download, am i right?
cheers
www.pilotmovies.pl   5D Mark III, 6D, 550D

RenatoPhoto

Quote from: kgv5 on September 08, 2013, 10:44:06 AM
Sorry for asking that but where could i find.mo file with the new ETTR features (if it exists)? my ML ver is 22 Aug. The link with the last commit doesn't seem to have it ready for download, am i right?
cheers
For 5D3 and many others try the nightly builds here: http://builds.magiclantern.fm/#/
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

Audionut

Quote from: ouuzi on September 08, 2013, 08:00:21 AM
So to utilise the highlight recovery of Lightroom the optimal settings are to select clip any or clip green, so you will still have at least one channel not clipped ( that means that detail is there.Correct?).

I don't mind clipping some color channels when it's a cloud or something.  But for a wedding for instance, I wouldn't take my chance blowing a color channel in the white dress and hoping that I can recover it in LR.  I would expose to ensure the white dress is not clipped in any color channel (full detail).
LR/ACR are pretty darn good at recovering detail even if it's been clipped to white (255) in one of the color channels though.  Raw therape does a good job at recovering details also.

Quote from: ouuzi on September 08, 2013, 08:00:21 AM
And -0.5 EV is in a way a safety setting to go close to pure white but not 255 white.

That's correct.  And if AETTR is setting the exposure to -0.5EV, you can enable a little amount of highlight ignore to push the histogram even further to the right.

kgv5

Quote from: RenatoPhoto on September 08, 2013, 01:56:34 PM
For 5D3 and many others try the nightly builds here: http://builds.magiclantern.fm/#/

Damn, how could i overlook this  :o Thank you RenatoPhoto, i have searched couple of threads and didnt see new layout DOWNLOAD tab.
www.pilotmovies.pl   5D Mark III, 6D, 550D

Canon eos m

Just a very dumb question. I went for a light show today and was trying ETTR but kept getting the "ETTR failure" message. Any particular reason for that. Am on the 26th March nightly.
Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)


Canon eos m

Quote from: a1ex on March 28, 2014, 05:14:10 PM
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

Sorry :-[ forgot about that.

I cannot recall the settings on the camera, but the ETTR settings were:

Half dbl click
Slowest shutter: 1/44
ISO 1600 and it was really dark

Rest settings on ETTR were all default

I know this is not too much to go by. Will try and replicate this issue again tonight. All I recall is that it kept giving the ETTR failure message.

Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)

a1ex

... what camera? were you in photo mode or LiveView?

Canon eos m

Canon 5d mark 3. Photo mode with live on.
Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)

a1ex

In this case, the possible paths that would show this message are:

- triple half-shutter click
- LiveView stopped in the middle of metering
- precondition checks (unlikely, but who knows)

Canon eos m

You are right the triple click is causing the failure message.
Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)