AETTR + DUAL ISO: The Ultimate Automated Perfect Image Exposure-Beginners'Guide

[RenatoPhoto]

Update: Sept 17
Note: This is brand new and may not work on all cameras.  It works on 5D3 from my testing.  Please let me know if you have tested this modules with other cameras to update this post

If you know nothing about ETTR then go here:
(Auto) ETTR (Exposure to the Right): -- History & Beginners Guide
http://www.magiclantern.fm/forum/index.php?topic=5693.0

If you know nothing about DUAL ISO then go here:
Dual ISO - massive dynamic range improvement
http://www.magiclantern.fm/forum/index.php?topic=7139.0

For a comparison of ETTR and DUAL ISO go here:
Dual ISO vs Auto ETTR
http://www.magiclantern.fm/forum/index.php?topic=7835.0

What is new?

The Auto Exposure To The Right module has now been expanded to squeeze the last bit of information of the camera sensor by the automatic use of DUAL ISO.  This is best used in situations where the high dynamic range of the scene require DUAL ISO to better capture the details of the image.  The latest implementation analyzes the image and automatically decides when to use DUAL ISO.

For the latest ETTR and DUAL ISO Modules go here:
Nightly Builds - try the very latest stuff here
http://www.magiclantern.fm/forum/index.php?topic=3072.0

PLEASE BE ADVISED THAT THESE MODULES ONLY WORK IF YOU HAVE ENABLED THE BOOTFLAG IN YOUR CAMERA.

If you want to enable the bootflag in your camera (550D, 600D, 50D, 60D, 5D2)  then go here:
Magic Lantern v2.3 - Install Guide
http://wiki.magiclantern.fm/install

For other cameras go here for a starting point.
RAW video & ML -- Beginners Guide, FAQ & Useful Links -- READ FIRST
http://www.magiclantern.fm/forum/index.php?topic=5520.0

HOW TO USE IT!

1. The modules are located in the EXPO tab:  Auto ETTR and Dual ISO


2. Auto ETTR has a simple mode:



Click on the Advanced mode to display all of the settings.

3. ... and an Advanced mode:



4. Trigger mode allows you to select three different methods for triggering the use of the AETTR.

    Press SET means that every time press the SET button the current scene is displayed in the LCD monitor with an analysis of the image.
    HalfS DblClick  means that every time double press the Half Shutter the current scene is displayed in the LCD monitor with an analysis of the image.
    Always on means that every time you take a picture the image is displayed in the LCD monitor with an analysis of the image.
    Auto Snap means that every time you take a picture the image is displayed in the LCD monitor with an analysis of the image, and that if the results are not adequate it will automatically take another image.

5. Slowest shutter: Set the slowest shutter that will allow for the current scene.

6. Exposure Target: A general exposure target for the scene,  cannot be set to zero but it could be set lower to expose darker.  This is not really used very much, usually left at -0.5

7.  Highlight ignore:  Percentage of (highlight) pixels to ignore when calculating ETTR.  This is used if you want to allow overexposure in the highlights, for example room lights.  Typical setting is 0.2% and means ignore almost nothing.   If you want to ignore more highlights try 1, 3, 5 or even 10%.

8.  Allow Clipping:  No clipping  This selection allows to select Clip Green channerl, and Clip ANY channel.   Only used if you want to allow clipping of higlights for any channel or only Green Channel specifically.  Some people will allow for some green channel clipping since there is more detail information on the other channels.  This would allow for the maximum exposure to the right and yet maintaining the highlight details.

9. Midtone SNR limit: Is defined in EV levels of noise in the Midtones form 1 to 8 EV.  The higher the SNR the less noise.  Typical settings from 4 to 8 EV.

10. Shadow SNR limit: Is defined in EV levels of noise form 1 to 4 EV.  The higher the SNR the less noise.  Typical settings from 2 to 4 EV.

11. Link to Canon shutter:  This links the Slowest shutter to the shutter value you set via small scroll wheel.  This is very useful for quickly changing the Slowest shutter without having to go into the ML menu.  Just set the shutter speed and on the next photo Slowest shutter = new shutter.

12.  Link to DUAL ISO: ON or OFF.  In ON position the camera will analyze the scene via Trigger mode (4) and decide if DUAL ISO is required or not.  Requires the DUAL ISO module to be enabled.
Note: If Link to DUAL ISO is set to OFF, the module will not calculate the required DUAL ISO and the image will only be set to meet the ETTR requirements.

13. Show debug info: ON OFF.  Will show some of the important calculation results of this module.

Convergence to results:

a. This module with require at least two images (sometimes three or four) to come up with the proper exposure.  Alternatively you can set the Trigger mode to Set in which case the camera will go to live view, analyze the scene, and return to photo mode with the correct settings.

b. When the algorithm cannot find an adequate exposure it will beep twice and give you a message in the LCD screen. ETTR: expo limits reached, indicating the ISO and speed limits used.

c. When the algorithm finds an adequate exposure it will beep once and give you a message in the LCD screen. ETTR settled at ISO xxx and the shutter speed.
Note: the image review time in the LCD is set via Canon menu in Image Review.  You may want to put 8 seconds so you can see the results in the LCD before it turns off.

Raw histogram use:

It is best to enable RAW Histograms via Global draw.  This will put a little histogram on the LCD during image review.  This histogram can show an ETTR hint (how many EVs are required to reach ETTR), or the Dyanmic Range of the image.  It will also show the clipped channels via colored numbered circles.

If you also enabled RAW zebras via Global draw, you will also see, if any, areas of the image are over-exposed.  Black will be completely overexposed, then you can see red, green, and blue channels clipped areas shown with their respective colors.  White will be dark areas completely under-exposed.

More about SNR Levels:

Here you can find some samples at different SNR levels:
http://www.magiclantern.fm/forum/index.php?topic=5200.msg70150#msg70150

Some test I did here:
http://www.magiclantern.fm/forum/index.php?topic=7835.0

[chris_overseas]

Thank you for this, it's fantastic. I picked up quite a few useful snippets of information and I'm sure I'll be referring back to this regularly while I figure out all the various options in detail.

One small typo: "You may want to put 8 or 8 second so you can see the results in the ..."  You haven't finished the sentence.

Would it be possible for you to expand a little on the SNR limits, in particular the pros/cons of the different values and when you might prefer one over the other?

Finally, am I right in thinking the main use case for Auto ETTR + Dual ISO be a scene where there is high dynamic range but there is also some movement in the scene (people, etc) that makes it unsuitable for bracketing + HDR/blending/Exposure Fusion etc instead? Are there other cases where it might be useful too?

[RenatoPhoto]

Thank you for this, it's fantastic. I picked up quite a few useful snippets of information and I'm sure I'll be referring back to this regularly while I figure out all the various options in detail.

One small typo: "You may want to put 8 or 8 second so you can see the results in the ..."  You haven't finished the sentence.

Would it be possible for you to expand a little on the SNR limits, in particular the pros/cons of the different values and when you might prefer one over the other?

Finally, am I right in thinking the main use case for Auto ETTR + Dual ISO be a scene where there is high dynamic range but there is also some movement in the scene (people, etc) that makes it unsuitable for bracketing + HDR/blending/Exposure Fusion etc instead? Are there other cases where it might be useful too?

You are welcome, every user of ML should contribute if possible.
I fixed the typo, thank you.
I have expanded a little near the bottom.
Yes, high dynamic range can only be captured in ONE picture only by the use of DUAL ISO.
Yes, perfect application for movement in high dynamic range scene.

[Audionut]

Finally, am I right in thinking the main use case for Auto ETTR + Dual ISO be a scene where there is high dynamic range but there is also some movement in the scene (people, etc) that makes it unsuitable for bracketing + HDR/blending/Exposure Fusion etc instead? Are there other cases where it might be useful too?

I use it for tone mapping in camera.  As I am predominantly a portrait shooter, skin tones are very important for me, but I also like to respect the highlights in the scene (not blow them to white).

With AETTR, I have very fine white point control.
Dual ISO allows lifting of the tones below the white point.
SNR limits allow me to accomplish this without thinking about it on location.

There are many times where the scene you are capturing might only have 9EV or so of dynamic range, but in this scene, the skin tones might be getting metered incorrectly (Canon likes to save highlight detail and will underexpose midtones to accomplish this), pale skin could end up at -2EV-0EV (from a midtone standpoint), or even worse, dark skin tones could be right down at -4EV.  This is still within the dynamic range capabilities of all cameras within the last few years (it's still within the dynamic range limits of what most people would call acceptable noise levels, let alone scientific SNR measurements where SNR is measured from 0), but most portrait shooters will know that the most important detail (the skin tone) has been pushed to far towards the shadows.
You can lift the skin tones in post, but the tonal range has been destroyed from being captured to far towards the shadows, and fine detail is also lost (to noise).  The other (better) option is to place more priority on the skin tones when making your exposure decision and suffer the highlight loss to white.

Note:  Like everything in photography, tonal range also works in stops.  A 14bit raw file has 16,384 available levels of tone.  1 stop below overexposure and this gets reduced to 8192 available levels of tone.  In other words, the very top stop of exposure gets reproduced with 8192 levels of tone, and the other combined 10 stops of exposure only get the remaining 8192 levels of tone.  In the example above where the pale skin tones have been captured at -2EV (-6EV from white), we are only left with around 256 levels of tone.  To make matters worse, as the skin tones have been underexposed by 3 stops, we need to lift the exposure in post to compensate, and in doing so, we only have 256 levels of tone to map to this new brightness level.

With AETTR (SNR limits)/dual ISO, we gain a number of advantages.

First, we have very fine white point control with AETTR.  We can decide to keep all highlights in the scene, or we can allow a limited amount of highlight overexposure (with the possibility of regaining some detail in post).

We have SNR limits that we can use for automatic exposure of the midtones (less thinking about exposure and more thinking about composition).

And then we have dual ISO for lifting the midtones.  In the example above where skin tones have been underexposed by 3 stops, we can use +3EV for the recovery ISO.  Of course, we have the option of using less separation of ISOs for better highlight control (where needed), or more separation for even better skin tone exposure.  And we can control this automatically with SNR limits.

We then have both ISOs mapped into 16bit DNG (16bit image files have 65,536 levels of tone) with it's greater precision.


Further reading:
http://www.normankoren.com/digital_tonality.html
http://dpanswers.com/content/tech_zonesystem.php
http://www.fredparker.com/ultexp1.htm

[chris_overseas]

I use it for tone mapping in camera.  As I am predominantly a portrait shooter, skin tones are very important for me, but I also like to respect the highlights in the scene (not blow them to white).

Interesting, thanks for explaining that, makes sense. I don't do much in the way of portrait shooting so didn't appreciate the issue with underexposed skin tones but I can see how the same reasoning would apply to similar situations. Basically whenever the subject is at the lower end of the scene's dynamic range, even if the entire DR is within the sensor's capabilities since more signal (via higher ISO) can give better S/N - as per standard Auto ETTR without dual ISO.

We then have both ISOs mapped into 16bit DNG (16bit image files have 65,536 levels of tone) with it's greater precision.

This I'm not so convinced about. Data is only captured with 14 bit precision at best. There's still noise which probably shaves off a couple of meaningful bits from the low ISO data, plus the high ISO data ends up underexposed by a few stops, each stop shaving off a bit (which was probably also noise anyway). cr2hdr interpolates the data into a 16 bit DNG but that doesn't mean you end up with 16 bits of accuracy in the resultant DNG, just precision.

[Audionut]

Interesting, thanks for explaining that, makes sense. I don't do much in the way of portrait shooting so didn't appreciate the issue with underexposed skin tones but I can see how the same reasoning would apply to similar situations. Basically whenever the subject is at the lower end of the scene's dynamic range, even if the entire DR is within the sensor's capabilities since more signal (via higher ISO) can give better S/N - as per standard Auto ETTR without dual ISO.

Yes exactly.  And we can ETTR the midtones/shadows of the image while still respecting the white point.

This I'm not so convinced about. Data is only captured with 14 bit precision at best..................  cr2hdr interpolates the data into a 16 bit DNG but that doesn't mean you end up with 16 bits of accuracy in the resultant DNG, just precision.

Each ISO is captured with 14bits at best.  It's my understanding that bitdepth (all but) = DR.
But we have 2 captures, that are by design increasing the DR (bitdepth).  We are not just mapping 11bits (9 useable) of signal into a 16bit container with it's greater precision, we are actually mapping 14.5 (or so) bits into a 16bit container.  Here, the greater precision is helping with the accuracy.  Because our shadows which would otherwise be mapped into 1-4bits of tonal precision (in a 14bit container), are now being mapped into a higher bitdepth.

Higher bitdepth is higher accuracy.  And we are mapping actual detail into a higher bitdepth.  Start tone mapping that data (especially the shadow data), and the increased precision will allow much greater freedom.

There's still noise which probably shaves off a couple of meaningful bits from the low ISO data, plus the high ISO data ends up underexposed by a few stops, each stop shaving off a bit (which was probably also noise anyway).

I don't understand how the high ISO data is being underexposed?

[Danne]

i must say this aettr module combined with dual-iso is one of the greatest functions in my camera right now. The accuracy of this modules are simply astonishing. Huge thanks Alex.
Also thanks for this tutorial. I learned a thing or two. Question. What camera are you using @Renatophoto? You mention "auto snap" but I don,t seem to find this option on my 5d mark 3?

[Audionut]

4. Trigger mode allows you to select three different methods for triggering the use of the AETTR.

    Press SET means that every time press the SET button the current scene is displayed in the LCD monitor with an analysis of the image.
    HalfS DblClick  means that every time double press the Half Shutter the current scene is displayed in the LCD monitor with an analysis of the image.
    Always on means that every time you take a picture the image is displayed in the LCD monitor with an analysis of the image.
    Auto Snap means that every time you take a picture the image is displayed in the LCD monitor with an analysis of the image, and that if the results are not adequate it will automatically take another image.

It's there.

[Danne]

Thanks, will try it out when I get home

[chris_overseas]

Each ISO is captured with 14bits at best.  It's my understanding that bitdepth (all but) = DR.
But we have 2 captures, that are by design increasing the DR (bitdepth).  We are not just mapping 11bits (9 useable) of signal into a 16bit container with it's greater precision, we are actually mapping 14.5 (or so) bits into a 16bit container.  Here, the greater precision is helping with the accuracy.  Because our shadows which would otherwise be mapped into 1-4bits of tonal precision (in a 14bit container), are now being mapped into a higher bitdepth.

Higher bitdepth is higher accuracy.  And we are mapping actual detail into a higher bitdepth.  Start tone mapping that data (especially the shadow data), and the increased precision will allow much greater freedom.

Well... yes and no. I haven't worked through all the math (and don't intend to) but it's more complex than what you're stating above. For example, mapping 14.5 bits into a 16 bit container makes the accuracy worse. It's similar to the problems faced with representing a decimal number like 0.2 in binary. You end up with 0.00110011... recurring, so have to truncate it somewhere. That introduces roundoff error and you lose some accuracy in the process (yet you gain precision). Of course in some cases that might mitigate the additional quantisation error introduced during the ADC process, in some cases it may make it worse. With regards to bit depth, yes it is somewhat related to dynamic range but again it's not a straightforward relationship since noise plays a very important role too. There's a couple of good articles on the topic here:

http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html
http://francoismalan.com/2011/10/raw-12bit-or-14bit-lossy-or-lossless/

Personally I'm not too fussed about trying to understand exactly what gains and losses there are in terms of accuracy, precision, noise, dynamic range or whatever. If anyone wants to debate it or even prove it mathematically then be my guest but I'm staying out of it. The basic theory is sound and the results of dual ISO speak for themselves which is good enough for me!

I don't understand how the high ISO data is being underexposed?

Sorry, my fault for being sloppy with my terminology. By "underexposed" I meant the process of cr2hdr pulling down the high ISO data by a few stops (and discarding the low order bits in the process), such that it is now exposed correctly.

[akumiszcza]

AETTR+Dual ISO works great on 50D, too. I've tested it in churches  — before these functions I had to bracket and combine the photos as hdr. Not working for now: auto snap, always on (these work in tragic lantern builds, but no aettr+dual iso combination there yet) and beeps for good/bad ettr and no message if ettr is settled.

[britom]

@RenatoPhoto and everybody Thank you very much for all the info in this thread!

[pedrosuarez]

My apologies if this has been covered, I haven't been able locate enough information about it. I intend to try out the practices in question, I'm just curious if anyone has had previous success.

I've been trying to utilize AETTR + Dual ISO for daytime / astro time lapse and am having a few problems. My goal is to be able to complete a 24+ hour time lapse incorporating a 5 axis slider while maintaining proper exposure. Ideally, I would like to get around 2900 dual ISO frames in a 24 hour period, and end up with 1450 merged files. I'm having trouble with synchronization and exposure, though, so I've only been patch together 6-9 hour segments at a time.

First, I can't seem to expose the scene properly at night. The auto exposure immediately jumps to ISO 12800 in dark scenes, which usually overexposes my shot by more than 3 stops and blows out elements of fine detail (stars, etc). Is it possible limit the max ISO with AETTR and/or Dual ISO? I'd like to take advantage of faster glass and longer exposures to pull out as much detail as possible, but at 30" / f2.8 ISO 12800, the night sky is overexposed beyond recovery.

Second, I'd like to time the moves on my slider to be at consistent intervals between shots. I haven't found an issue when the unit stationary; AETTR and ML's intervalometer can be set to cap at a 32" exposure and, say, a ten second interval. The camera will continuously calculate exposure and take pictures. However, I'm not sure how to time the motors on my slider to move during each interval. For instance, if AUTOSNAP is engaged, the camera might need to take three or four pictures to find exposure. If I'm taking a 32" picture, the delay has to be set to over 2 minutes between shots so the slider doesn't move while the shutter is released. For a 24 hour sequence, that results in 720 pictures, or 30 seconds of film. The problem at night, again, is that my shots end up too overexposed, so only about 15-20 seconds of that film is usable.

Most Recent Settings:

Trigger mode. Auto Snap
Slowest shutter: 32"
Highlight ignore: 0.1%
Allow clipping: OFF
Midtone SNR limit: 6EV
Shadow SNR limit: 2EV
Link to Canon Shutter: OFF
Link to Dual ISO: ON
Shadow metered areas: ON
Show debug info: ON

Dual ISO (enabled)

Intervalometer / Slider: Exposure: 32", Interval - 1:40m (to allow for AUTOSNAP)

Post Deflicker:
Sidecar file type: Adobe XMP
Deflicker percentile: 50%
Deflicker target level : -4EV

Does anyone have advice as to better improve this workflow? I will continue to try out different settings, but I would love it if I could shorten the move interval to 1 minute and double my number of exposures. Not sure if that's possible with AUTOSNAP engaged, and I'm not sure if I need it engaged, so let me know if that's the case. Like I said, the main goal is 2900 dual ISO frames with 1 minute delay between moves (every other frame). It's okay if the frame pairs are ISO 100/100, etc. before merging, I'm just trying to capture greater dynamic range when possible.

Thanks in advance, and thanks to all the devs for your hard work.

[Audionut]

AETTR should not be creating overexposure above the limits you have set.  Can you try with just AETTR and then with AETTR+Dual ISO with no ISO speed limits.
Having said that, if your only highlights are the stars in your night shots, it's possible that 0.1% highlight ignore might be enough for AETTR to completely ignore the stars, in which case if the next brightest object is significantly darker then the stars the exposure will be much higher creating the overexposure.
You could try 0.0% highlight ignore, and allow some clipping in the green channel. 
Once you have your highlights under control, tweak the midtone/shadow settings.

Dual ISO can be limited with the "auto ISO" setting in Canon menu.
AETTR can be limited with the "ISO speed range" setting in Canon menu.

[ShootingStars]

Any image samples?

[RenatoPhoto]

For instance, if AUTOSNAP is engaged, the camera might need to take three or four pictures to find exposure.

I never used Autosnap for timelapse, I only use Always ON.  XMP will carry the exposure adjustment required to defllicker.  Also I never set my intervalometer above 1 min because the motion becomes too jerky.  I have also noted if intervalometer is higher than 1 min, and the exposure change is very fast, ETTR cannot properly deflicker so I keep intervalomenter at 1 min or less.

Also I am not sure what you mean by merged files...  It would seem like you are reducing the number of shots but I am not familiar with this process.  In any way if you are merging shots it is probably required to do it after deflickering is applied, otherwise exposure will not match from shot to shot.

I dont know if this helps but these are some tips I can provide with my limited experience.

[akumiszcza]

I've tried aettr+dualiso on latest nightly on 50D. The messages after each shot (in autosnap) are slower now, which makes it harder to be sure the exposure is settled (no beeps here). Besides, turning raw zebras on shows underexposed areas and draws it quite slowly in image review + making it hard to see the photo – screenshot: . Turning it off doesn't help in message speed, though.

[akumiszcza]

Nvm my prev post. I think the speed of showing histogram and ettr message was slower, because I used slower CF card.

[djronbxs]

when taking hdr photos is it better to switch off ETT and dual iso ? Can you tell me the best workflow for taking hdr please ?

thanks
ron

[mranch]

Good Morning,
I have ML loaded on my 600D/T3i. I then loaded the Dual ISO Module.  Now the ETTR is gone from the menu. Is it possible to  use both ETTR and Dual ISO on the 600D/T3i? I've search and could not find any info on the subject. Thanks for any enlightenment.

[martareis]

Hi,

I can't precise the build I was using on my 60D but I had AETTR feature until I decide to uprage to the latest nightly build  (2013-10-29 23:15:09 +0000).

I presume it was removed due to some problems. Can anyone confirm that information?

In which cameras is that feature (AETTR) working properly?

Can any one provide a list of features for 60D latest nightly build?

Thx

[a1ex]

According to the compile log, it's there: http://builds.magiclantern.fm/ and http://nanomad.magiclantern.fm/jenkins/features.html

[Critical Point]

I have Tragic Lantern v1 installed because I need the H.264 GOP and bitrate controlls, can I install also this module with dual iso ?

[Lupo_wolfi]

Problem,
I have horizontal lines recording on Dual ISO and ETTR. Got the last ML release of Dec 28.

I followed your post and read all of the ETTR menu meanings.

Too bad that the final result got this

Thanks for any help!

[RenatoPhoto]

Problem,
I have horizontal lines recording on Dual ISO and ETTR. Got the last ML release of Dec 28.

That means your images are DUAL ISO and you need to do further processing to merge the ISO...  follow this thread

Dual ISO - massive dynamic range improvement
http://www.magiclantern.fm/forum/index.php?topic=7139.0