Auto ETTR based on RAW histogram (ettr.mo)

[1%]

The date on the zip is what you have. ETTR should be 1:1 with ML.

[Audionut]

I moved the development discussion into it's own thread.

http://www.magiclantern.fm/forum/index.php?topic=10808.0

Cheers.

[PaulJBis]

Hello. I've been trying AutoETTR for a sunset timelapse this weekend, and I have some questions about it.

First of all, until now I was shooting using the old bulb ramping feature. When I shoot sunset timelapses, my opinion is that it doesn't matter if the latter shots (the ones at night) have a lower exposure value than the earlier ones; after all, it's supposed to be at night, and if you try to overexpose them so that they match the daylight shots, you'll risk blowing out the lights in the shot. Thus, until now I was using bulb ramping to ramp down the EV value.

I understand that this is not possible with AutoETTR, since its philosphy is to expose always to the right in order to decrease noise, and if you want to ramp down EV, you can always do it later in post. Which is fine... except that sometimes, when it's really dark at night, AutoETTR simply can't increase the exposure anymore. The only way it can do so is by increasing the ISO to ridiculous levels (3200, 6400), which kind of defeats the purpose of avoiding noise. (It happened to me this weekend).

Thus, I'm wondering if there's any way to "cap" the ISO levels, or the EV values that AutoETTR aims for. One way I'm thinking of doing it is by activating the advance intervalometer and adding keyframes in it for ISO, and then using the "slowest shutter" option. Is there any other way? Or, in more general terms: is there a way to ramp up/down the EV target of AutoETTR + adv_int.mo for situations like these?

[a1ex]

The only way it can do so is by increasing the ISO to ridiculous levels (3200, 6400), which kind of defeats the purpose of avoiding noise.

I have yet to see a higher ISO noisier than a lower one (in electrons). If you want to reduce the ISO (all other variables being equal), it's not to get less noise, but to capture more highlights.

FYI:
http://www.clarkvision.com/articles/evaluation-canon-5diii/index.html
http://www.clarkvision.com/articles/evaluation-canon-7d/index.html
and so on

on-topic: change auto ISO limit in Canon menu.

[PaulJBis]

on-topic: change auto ISO limit in Canon menu.

Quick question: but I thought that AutoETTR didn't work if you set the ISO in auto mode?

(I'm reading the links right now. Will post later with questions if I have doubts :-) ).

[garry23]

Alex

At the risk of complicating this post I would like to try an idea on you, that relates to a feature request I posted the other day.

The idea is to bring an ML 1 deg spot together with AETTR.

At the moment my understanding is that the AETTR algorithm scans the entire scene and works out the shift in exposure to ensure the user's criteria is met.

I use manual (ie non ML) ETTR a lot, but with a spot meter. My work flow is simple, scan the scene with my 1 deg spot meter and select the 'hottest' part of the image that I wish to see shifted to the right, ie the highlights that I want to see in zone 7 or 8, rather than the camera's mid tone zone 5.

Through calibration I know on my 5DMkII the DR is about 7.2Ev and the highlight edge is at 3.2Ev. So I can safely shift by up to 3Ev.

I use an external meter as this is lens independent.  BTW on my 5DMkIII I would need to use about a 350mm lens to achieve a 1 deg spot - see here: http://www.spotmetering.com/1deg.htm

If ML could emulate a 1 deg spot, ie by noting the lens FL and the sensor size and returning an exposure restricted to the ML 1 deg spot, this would mean ML is a real alternative to, say, the Sekonic meter, which I use. If the lens doesn't return its FL, then ML would use a default FL that the user could change, ie to account  for a manual lens.

Once there is an ML 1 deg ML spot meter built in, the user would than be able scan the scene in LV, select the part of the scene to AETTR and let ML do the rest, ie shift the exposure or at least indicate what that shift would be.

The two killer features here are a 1 deg spot (or a selection of angles?) coupled to the user selecting the area of the scene to shift from.

[a1ex]

Well, the spotmeter algorithm is quite a lot simpler than ETTR.

I could add a window size option for ETTR, which is the simplest way to implement a spotmeter without reinventing the wheel. However, there's a important difference:

- In the current configuration (full-screen), the SNR limits and highlight ignore percentage are very important to get a good balance between noise and highlight detail, given the limited DR in Canon cameras.

- With a small spotmeter, these settings will have no effect and will become confusing (a spotmeter can simply do a plain average and that's it).

So I'm not sure if - for a UI perspective - spotmeter really makes sense as a sub-mode in ETTR, or as a completely separate tool.

[garry23]

Alex

I understand and would humbly suggest that it would be well worth doing.

I still believe, but I'm not a coder, that by first getting a 1deg spot in ML, the ability to survey the scene in LV mode, as now but a 1deg, would be great.

If ML could then return the exposure at that 1deg point in the scene and maybe give an ETTR hint, that would be good.

'All' I'm suggesting is to create an inbuilt 1deg spot meter that is independent of FL, ie lens/sensor coupled.

If this can be linked to an ETTR shift great, but honestly that is a bonus, as if I know the 1deg spot exposure reading, I can manually shift to zone 7 or 8.

[PaulJBis]

I have yet to see a higher ISO noisier than a lower one (in electrons).

Okay, I just did a few tests. You were right!!    I guess I was still thinking about ISO in terms of film stocks...

However, my larger point stands: I just finished another sunset timelapse using AutoETTR, and at the end (when it was dark), it blew my highlights to the point that now I can't recover them in post. Thus, I'd like to know if there's a way to ramp EV so that, at the end of the timelapse, the software aims for a different value than at the beginning.

[a1ex]

You probably had SNR limits set too high.

[PaulJBis]

I had them on their default values, 6 and 2. So if I lower them, I'll get a picture less "to the right" at the cost of more potential noise, right? I'll experiment with other values then.

[PaulJBis]

Okay, I've been rereading this thread, paying special attention to the posts where this new feature (SNR limits) was introduced. However, there's one thing I don't understand:

So, when I say the midtone SNR is 5 EV, this means half of the image has a SNR less than 5 EV, and the other half has a SNR higher than 5 EV. Pretty easy and statistically robust.

Why are you talking about SNR in terms of EV units instead of, say, dB?

(Please note: I'm no professional developer nor physicist, just a humble amateur photographer. Pointers to basic notions are welcome. After googling I found this explanation:

http://www.imatest.com/docs/noise/

But I don't know if it's relevant to the subject).

[a1ex]

Yes, I prefer to use EV units, since that's how you dial the exposure.

So, with these SNR limits, you ask "don't underexpose more than that", and ETTR will do that religiously (even if that means it will burn up to 95% of your image - worst case).

Note that ETTR approximates SNR to signal / optical black noise (that is, it assumes a constant sensor noise model, as in the column A from your link). On 5D3 ISO 100, this is true until around 4 EV above the noise floor. So, ETTR assumes the SNR follows the black line from this graph (that is, it assumes 1 EV of slower shutter == 1 EV of less noise, which is only true in shadows):



(so you can ask for 8 EV in midtones, but in practice you'll only get 6)

I could change it to be more exact, but I need to know some more data about each ISO. Right now, the black line is completely determined by dynamic range figures, which I copied from DxO. If I also consider the max SNR in highlights, I could get a better approximation.

[a1ex]

Some slides I've prepared a while ago: comparing Canon's exposure meter with ETTR. Hope it helps understanding the difference between the two, and the motivation behind ETTR.



So, ETTR analyzes everything from your scene, but doesn't care where they are placed (the exposure decision is based only on the histogram), while Canon metering is much too center-weighted (or focus-point-weighted) for my taste.

[Audionut]

When iso_research is completed, I would like to add get_iso_research_dynamic_range.
Can you use the SNR curves from raw_diag to help with accuracy?

The biggest problem with Canon metering, IMO, is that it is dumb.  It doesn't care about highlights, shadows, SNR, DR.  Simply, find the average, and place it at -4 EV.

[barepixels]

Someone asked on page 3 how come AETTR can't work with SRAW1 or SRAW2.  Can someone explain why the option is not available.  For doing time-lapse I find SRAW1 is half the file size, Thus I can shoot twice as long without changing memory card.

[dmilligan]

Can someone explain why the option is not available.

The camera behaves differently when in these modes and so they would need to be reverse engineered (like everything else in ML).

[garry23]

Because I usually use SET to call ETTR I have only just noticed a conflict with back button focus.

When I move ETTR to double press, it works as it should off the exposure button, but it also triggers off a double press of the back button.

Is there any way to decouple ETTR being triggered off the back button when ETTR is used by a double press?

[tetsusaiga]

PaulJBis:  You said in your post above that you "just finished another sunset timelapse using AutoETTR, and at the end (when it was dark), it blew my highlights to the point that now I can't recover them in post. Thus, I'd like to know if there's a way to ramp EV so that, at the end of the timelapse, the software aims for a different value than at the beginning."

When you say "ramp EV exposure," are you talking about the "Exposure target" parameter in AutoETTR? How have your sunset/sunrise timelapses come along so far with AutoETTR? I'm trying to learn all that I can so any help or tips would be appreciated.

[dmilligan]

PaulJBis problem was likely too high of "SNR Limits", the SNR limits will override the clipping limit (things will get overexposed) to maintain the requested SNR. Solution is to set the SNR limits lower.

[garry23]

Re my shutter half press question, does any one know how to just get this on the shutter button, ie and not the back button.

I would have thought no one would want the double press working on the back button.

Or am I wishing for unobtanable things?

[PaulJBis]

PaulJBis:  You said in your post above that you "just finished another sunset timelapse using AutoETTR, and at the end (when it was dark), it blew my highlights to the point that now I can't recover them in post. Thus, I'd like to know if there's a way to ramp EV so that, at the end of the timelapse, the software aims for a different value than at the beginning."

When you say "ramp EV exposure," are you talking about the "Exposure target" parameter in AutoETTR? How have your sunset/sunrise timelapses come along so far with AutoETTR? I'm trying to learn all that I can so any help or tips would be appreciated.

In the context of that post, I didn't mean the "Exposure target" parameter that you can set in AutoETTR. I meant the internal EV target that AutoETTR sets when it performs its calculations.

BTW; I've been out of the loop, shooting timelapses (among many other things). Will try to catch up tonight and see if there's been anything interesting.

[jackmoro]

Hello.

Is it possible (miracle  ) to make ETTR calculate exposore only for working area in RAW recording? Like if we use 2:20 aspect, some area will be not be used in real footage, but it might (often) has some bright parts.

Alex.

[moodlover]

When exposing a scene, should we always go for ettr? Without ML, in photo mode, I did a test where I'd expose a shot twice, once at the metered exposure for 0 and once at +2 stops over. In lightroom when I applied -2ev to the over one to match the exposures, the over one had extremely clean shadows (no noise) compared to the normally metered one.

Can/does this apply when recording raw video? As in, a) will ettr always ensure we capture a good exposure and b) will going over ettr allow us to reduce exposure in post and give a better image?

https://luminous-landscape.com/the-optimum-digital-exposure/

[dubzeebass]

Can/does this apply when recording raw video?

a) will ettr always ensure we capture a good exposure and
b) will going over ettr allow us to reduce exposure in post and give a better image?

Yes it applies as long as you don't blow out any parts of the image

a) it may overexpose and require pulling it back. Overexposure != good exposure but you've got the gist of it.
b) see first sentence of this reply

Optimally for more efficient workflow you'd use a light meter to match the exposures.