Magic Lantern Forum

Developing Magic Lantern => Feature Requests => Topic started by: RenatoPhoto on September 29, 2013, 09:23:21 PM

Title: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on September 29, 2013, 09:23:21 PM
Dear Alex:

The functionality of ETTR outside M mode is not good.

In wildlife photography I use TV, C1, C2, C3 for very quick mode change to instantly adapt to the shooting requirements.
For example I program C3 for bird in flight, with shutter at 2000,  AI servo, High speed continuous shooting, 61 pt AF, Auto Iso, +3 EC.
C2 for bird against light, with shutter at 320, AI servo, High speed continuous shooting, AF Area Sorround, Auto Iso, +2 EC.
C1 for bird on the ground, with shutter at 250, AI servo, High speed continuous shooting, AF Spot, Auto Iso, +1 EC (kind-of ETTR).

I use these setting for quickly getting the photo without delays (Auto-power OFF=Disable).

When I have a little more time (the bird is posing for me) then I switch to M and optimize the light situation with ETTR and take a few more pictures.  Then I switch to RAW Video and get the most amazing 5X Cropped video of the birds.

Since ETTR cannot work quickly (instantly) in Tv mode (I use Tv exclusively for bird photography) it forces me to enter ML menu and disable ETTR, therefore slowing the process down, and sometimes I forget if ETTR is enable for when I really wanted ie. in M mode.

My request is to disable ETTR outside M mode as an option (in case that someone wants it back) so I can make use the instant change mode for quick photos between C1, C2, C3.  Also when I switch to M mode my ETTR is sure to be ready for me without having to go into ML menu.

I hope this is possible since it will benefit many users who want that very quick instant reaction of changing per-programmed modes C1, C2, C3

As Always: Thanks for your amazing work.

P.S.  In case it is not clear, I want to leave ETTR ON and not have to turn it OFF (the ETTR module should not work in these modes) when I am in C1, C2, C3, Tv and possible others like Av.
Title: Re: Auto Dissable ETTR outside M mode
Post by: Marsu42 on September 29, 2013, 09:38:49 PM
Quote from: RenatoPhoto on September 29, 2013, 09:23:21 PM
I hope this is possible since it will benefit many users who want that very quick instant reaction of changing per-programmed modes C1, C2, C3

An alternative approach would be what I've done in the auto_iso module: Add an option to disable the module in C modes (i.e. in the source use shooting_mode instead of shooting_mode_custom), no matter if they are Tv, Av or M.
Title: Re: Auto Dissable ETTR outside M mode
Post by: Audionut on September 30, 2013, 12:11:07 AM
What's the matter with setting the trigger mode to set or half shutter double click?
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on September 30, 2013, 02:30:17 AM
Quote from: Audionut on September 30, 2013, 12:11:07 AM
What's the matter with setting the trigger mode to set or half shutter double click?
I find both methods very disrupting since they block view via viewfinder of my subject. 
When I do wildlife photography I use the viewfinder to follow the bird, check focus, check speed, ISO. 

I also ask the same type question to you.

What's the matter with disabling something that does not work?
Title: Re: Auto Dissable ETTR outside M mode
Post by: dmilligan on September 30, 2013, 02:35:03 AM
+1

Quote from: Audionut on September 30, 2013, 12:11:07 AM
What's the matter with setting the trigger mode to set or half shutter double click?

b/c that goes into LV, which is slow and kind of annoying, and you have to do that process every time you want to re-meter. I'd rather have it just keep metering off of the pics I've taken, but not in other modes. Perhaps it would be better to have it "Always On" in a specific C mode, so that C mode is basically "AutoETTR" mode and leave M as actually full manual, which IMO it should be. AutoETTR is great, but not perfect - nor will any computer algorithm ever be lest all photographers be out of a job - sometimes I just need to make all of the exposure decisions manually. (This would already be possible if config presets didn't require a restart)
Title: Re: Auto Dissable ETTR outside M mode
Post by: Marsu42 on October 01, 2013, 12:02:33 PM
Quote from: dmilligan on September 30, 2013, 02:35:03 AM(This would already be possible if config presets didn't require a restart)

This is the "tie ml features to dial settings" request, and I'm working on a module this ... it's just that currently I'm more busy shooting than hacking ml, but as I want this for myself I'll do it eventually, at least for dual_iso, bracketing and ettr.

Problem is that the code looks horrible since modules don't work with CONFIG_ARRAY, so I explicitly have to add a variable for each dial setting :-\
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 02, 2013, 03:29:52 PM
I tested today ETTR in Tv mode and there is still some influence of ETTR in the exposure.  In some cases is ok but in some situations the exposure is way off and images are unusable. 
So please can ETTR be automatically disabled outside M mode?
Title: Re: Auto Dissable ETTR outside M mode
Post by: Audionut on October 03, 2013, 04:10:39 AM
IIRC, AETTR got implemented in Tv/Av from a request.  It would be better to fix (any problems with) the current implementation rather then removing it IMO.

I had a quick play with it and it seems to work as expected imo.  Just like ETTR in M, where there is a large amount of overexposure it can have problems determining the correct adjustment.
With AETTR in always on mode, it adjusted EC as needed.  This EC adjustment only changes with scene luminance (in AETTR), but the user is free to adjust EC before the next shot.

Either you want AETTR (have it enabled), or you don't (have it disabled).  A little forward thinking is all that is required IMHO.  If you're playing with AETTR and you know you need the ability to switch to a semi-auto mode in a moments notice without the effects of AETTR, perhaps it might be better to use the histogram/zebras for ETTR rather then some auto mode (AETTR).

M exposure settings don't change when you move to Tv/Av and back to M.

edit:  With a button manger, you could assign a button to toggle features.  Then those who want AETTR in Tv/Av get it, and those that only want AETTR in M can toggle it with a button rather then entering menus.
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 03, 2013, 03:08:21 PM
Quote from: Audionut on October 03, 2013, 04:10:39 AM
Either you want AETTR (have it enabled), or you don't (have it disabled).  A little forward thinking is all that is required IMHO.

Not!  No predictable way to know when a bird will fly above your head, or suddenly show up somewhere else.  I spent time writing the OP to explain why this is important!

Quote from: Audionut on October 03, 2013, 04:10:39 AM
edit:  With a button manger, you could assign a button to toggle features.  Then those who want AETTR in Tv/Av get it, and those that only want AETTR in M can toggle it with a button rather then entering menus.

Yes, that could help but it still would require the accurate press of a button.  If you miss it you could miss the shot and frustration increases.
Title: Re: Auto Dissable ETTR outside M mode
Post by: Audionut on October 04, 2013, 05:06:17 AM
Quote from: RenatoPhoto on October 03, 2013, 03:08:21 PM
Not!  No predictable way to know when a bird will fly above your head, or suddenly show up somewhere else.  I spent time writing the OP to explain why this is important!

Yes, but you are only using AETTR in M where you have the time.  Regardless of this feature request, in the meantime, why not use the other features of ML to ETTR where you have the time. 
You can practice your exposure without the worry of AETTR remaining enabled and ruining your shots needed in quick time.
Title: Re: Auto Dissable ETTR outside M mode
Post by: Marsu42 on October 04, 2013, 08:40:45 AM
Quote from: RenatoPhoto on October 03, 2013, 03:08:21 PM
Not!  No predictable way to know when a bird will fly above your head, or suddenly show up somewhere else.  I spent time writing the OP to explain why this is important!

I've spend a lot of time trying to make similar points in other threads, I guess one has to have some experience with these type of conditions to appreciate the requirements they generate.

Until this is done by someone else, why not do it on your own? Compiling ML is easy, and modifying ettr.c is also easy: Just remove SHOOTING_MODE_AV and SHOOTING_MODE_TV from the function auto_ettr_step() in line 683, done.
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 04, 2013, 02:29:58 PM
Thanks Marsu!!!!!!!!!!!!!! 

I found four instances of SHOOTMODE_AV AND SHOOTMODE_TV. 

Maybe all four should be removed or just the 689..

The line 619 of the if (expo_override_active()
The line 689 of the auto_ettr_step()
The line 707 auto_ettr_check_pre_lv()
The line 1193 MENU_UPDATE_FUNC(auto_ettr_update)

Also I want to clarify that my wildlife shooting mode requires AUTO ISO for fast shooting.  Now if I am in M mode and put AUTO ISO, then ETTR does not work.   But if I am in Tv mode, with AUTO ISO, the ETTR is enabled and the results are not satisfactory.   Basically I see this option as a excellent way of fixing this problems, what I mean is to disable the ETTR module in Tv, Av, and P modes as long as AUTO ISO is used.  There seems to be some faulty logic in this functioning of ETTR, ie disable ETTR in M mode if AUTO ISO is on but enable ETTR in TV, AV, P mode if AUTO ISO is on??
Title: Re: Auto Dissable ETTR outside M mode
Post by: Marsu42 on October 04, 2013, 09:18:41 PM
Quote from: RenatoPhoto on October 04, 2013, 02:29:58 PM
Thanks Marsu!!!!!!!!!!!!!! 

:-)

Quote from: RenatoPhoto on October 04, 2013, 02:29:58 PM
I found four instances of SHOOTMODE_AV AND SHOOTMODE_TV.  Maybe all four should be removed or just the 689..

When in doubt, remove :-p

Quote from: RenatoPhoto on October 04, 2013, 02:29:58 PM
Basically I see this option as a excellent way of fixing this problems, what I mean is to disable the ETTR module in Tv, Av, and P modes as long as AUTO ISO is used.

Afaik ettr + auto iso should work in theory, but if you say it's dodgy I believe you. I just created a pull request for ml auto iso as a module, I could add the option to disable ettr when this is enabled - problem is I still don't know how to disable one module (ettr) from another module (auto_iso).

Btw. you really have my sympathies, I'm also shooting wildlife (with fill flash) and I know ml isn't there yet since most people around here do either video or other stills in constant lighting that don't move :-p
Title: Re: Auto Dissable ETTR outside M mode
Post by: a1ex on October 04, 2013, 10:20:38 PM
Quote from: RenatoPhoto on October 02, 2013, 03:29:52 PM
I tested today ETTR in Tv mode and there is still some influence of ETTR in the exposure.  In some cases is ok but in some situations the exposure is way off and images are unusable. 

This is effectively ETTR + Canon meter (which is why I didn't implement it for M mode). I think I'll add an option to enable ETTR in auto modes if you really want it (but disabled by default).
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 05, 2013, 02:18:58 AM
Yes, yes!! Disable it by default!  Leave and option for others!
Thanks Alex.

BTW I deleted the SHOOTMODE_AV AND SHOOTMODE_TV and worked beautifully!  Thanks Marsu!
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 06, 2013, 05:10:57 PM
Ok so now I am able to shoot in Tv mode without the influence of ETTR.  This is very helpful!

I also have another nuisance that I would like to fix.


If I am taking photos in M mode with Always ON trigger mode and switch to video.  I want the ETTR to automatically switch from Always ON to Pess SET trigger mode.  This way I do not get those abrupt changes in exposure if the light changes a bit.

Yes I know that this can be done by entering into the ML menu and switching the trigger mode, but since smooth ISO changes in video mode are not possible then the Always ON trigger mode becomes useless (IMHO) in video mode and should be replaced by Press SET.

Again this is one of those things that will improve my technique to capture those precious wildlife moments without wasting time.

Title: Re: Auto Dissable ETTR outside M mode
Post by: a1ex on October 06, 2013, 05:25:07 PM
This one sounds like a different setting for photo/video, so I'd say presets is the way to implement this. The only problem is that you need to reboot the camera.
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 06, 2013, 05:33:08 PM
Quote from: a1ex on October 06, 2013, 05:25:07 PM
This one sounds like a different setting for photo/video, so I'd say presets is the way to implement this. The only problem is that you need to reboot the camera.
Yes, I thought about presets but again it takes time away for the subject.

Is there a way to change the code so (in my case) I can always revert to Press SET in Movie mode?  Of course when I switch back to photo mode I have to go back to Always ON.
Title: Re: Auto Dissable ETTR outside M mode
Post by: a1ex on October 06, 2013, 05:39:09 PM
Right now I can't imagine a clean solution. A hackish one is easy, but if somebody will want a similar thing (say he wants dual ISO 100/400 in movie mode and 100/1600 in photo), and we start hardcoding these things in every single feature, the code will become hard to maintain.
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 06, 2013, 05:53:26 PM
Maybe a Preffered trigger for video feature?  The functinality of Video Mode in Always ON is just not optimum, so when you want to use in Photo, you should not be forced to used it in video.
Title: Re: Auto Dissable ETTR outside M mode
Post by: a1ex on October 06, 2013, 07:09:12 PM
See reply #18.
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 06, 2013, 08:10:20 PM
Quote from: a1ex on October 06, 2013, 05:39:09 PM
A hackish one is easy.

I would love to add that piece of hack to my ettr code so I can use my 5D3 more effectively.    The problem is that I cant figure out (I dont have good programming skills)  how the code works so I am not able to develop an effective hack that would do something simple like:

If  SHOOTMODE VIDEO THEN AUTO_ETTR_TRIGGER_BY_SET
IF SHOOTMODE M (photo) THEN AUTO_ETTR_TRIGGER_ALWAYS_ON
Title: Re: Auto Dissable ETTR outside M mode
Post by: Marsu42 on October 06, 2013, 08:33:07 PM
Quote from: RenatoPhoto on October 06, 2013, 08:10:20 PM
If  SHOOTMODE VIDEO THEN AUTO_ETTR_TRIGGER_BY_SET
IF SHOOTMODE M (photo) THEN AUTO_ETTR_TRIGGER_ALWAYS_ON

Welcome to the world of ml hacking - that's how I started to spend a lot of time wading through the code, trying to figure out what is what.

To modify some things you don't need great skills but some time to burn... on occasion just ask for specifics here, but it's worth it because on you *know* how to modify things you get more ideas on how to make the camera do what *you* want.

I don't think you'll get lucky though asking for somebody else to do specifics you'd like (see "help vampire" :-)).
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 06, 2013, 09:01:37 PM
Quote from: Marsu42 on October 06, 2013, 08:33:07 PM
I don't think you'll get lucky though asking for somebody else to do specifics you'd like (see "help vampire" :-)).

I have written good programs in basic, fortran, and industrial ladder logic but this is just a language that I never used.  I can read it but there is a lot of stuff that does not register in my head.

If this was something that was just for my benefit then I would not dream of asking for it, but this are basic operational problems that linger in the ML code and IMHO should be fixed or removed from the code.

1.  If ETTR does not work in TV AV mode then simply remove it.  Why should ML interfere with Canons FW and produce BAD results.  Ok this is doable and I have an ETTR module thanks to your easy hack that that fixes this issue.  By the way, now I have to figure out how to disable ETTR in video mode while the camera is in TV AV modes also.

2.  Now the issue about ETTR in video is the same thing.  If you are recording a Video with ETTR in Always ON then the video is unusable.  So why have it???  Maybe if your Auto ISO module can provide access to smooth iso changes then it would be much better.
Title: Re: Auto Dissable ETTR outside M mode
Post by: Marsu42 on October 06, 2013, 09:09:01 PM
Quote from: RenatoPhoto on October 06, 2013, 09:01:37 PM
If this was something that was just for my benefit then I would not dream of asking for it, but this are basic operational problems that linger in the ML code and INHO should be fixed or removed from the code.

Sorry, I really didn't want to imply you're being egoistic - the thing is: I also have a very long list of things that can be improved about ml usability & workflow... most prominently about ettr & dual_iso behavior and interaction, but also about button management and feature-button allocation.

I also did some feature requests on things that bug me the most - but at the current state, optimizing it might be too early, it's still in the "omg, it's really working!" stage. So personally, I have postponed a lot of these interaction & workflow requests, tbh also because discussing my a** off why I would really want that and if it cannot be done any other way (which I know it cannot) is tiresome. But don't let me stop you :-)
Title: Re: Auto Dissable ETTR outside M mode
Post by: a1ex on October 06, 2013, 09:12:20 PM
1) I've tried it and it works.

2) Did you try to press record with ETTR enabled? I bet you didn't.
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 06, 2013, 09:21:37 PM
I am very happy to be able to use and help with ML. 
I have done what I can to make it better and that makes me happy! 
A good clean discussion (like this one) I enjoy and is very constructive and makes me happy.
;D ;D ;D
I will keep looking at the code when time allows, and will share any findings.
Thanks for your insight and help.  Hope your new module moves forward.
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 06, 2013, 10:09:29 PM
Quote from: a1ex on October 06, 2013, 09:12:20 PM
1) I've tried it and it works.

2) Did you try to press record with ETTR enabled? I bet you didn't.
Thanks for your test.
Regarding issue 1)
In Tv mode, there are some situation that the ETTR will change the exposure and produce bad results.
I found a test where I get the same results showing the problem:
Set ETTR on with Always ON. Camera in Tv mode
Find a window with bright exposure outside and dark inside (daytime)
Aim the camera outside
Take a shot.  In this case my exposure is OK, and the second exposure is OK too.  Very little influence from ETTR.
Now simply aim at the inside part (darker) and still have some of the bright window in the frame (40% of the frame has the bright window).
First shot is a bit overexposed of the inside and outside is overexposed.  not too bad...
Second shot has the outside properly exposed but the inside is underexposed.  Without ETTR this shot would have the outside would be overexposed and the inside properly exposed. 
If you shoot again then the inside becomes properly exposed and the outside overexposed.
If you shoot again then the opposite.
Each time I get the message ETTR: next ISO 0 1/320 (was iso 0 320)
Sometimes it just stays in the underexposed shot even if I move the camera to aim only inside so there is no bright window in view.

Regarding the issue 2)
You are right.   Thanks, I did not press the record button because the exposure was fluctuating.  Once press it stops metering.
But...
But if you are in TV or AV then the video gets recorded with fluctuation in the ISO, so I am looking to disable ETTR in video mode when I am in Tv or AV mode.

Title: Re: Auto Dissable ETTR outside M mode
Post by: a1ex on October 06, 2013, 10:51:38 PM
1) yes, this is because Canon meter is center-weighted and ETTR is not.

2) Right, that was a bug, will fix. But once I'll move post deflicker to a PC tool, it will also work on raw videos, so ETTR will make sense for video too.
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 06, 2013, 11:11:51 PM
1.  I am glad it is not just me making some stupid mistake.  ETTR and TV can produce bad results.

2. For now can it be set to stop changing ISO once the rec button is pressed in TV and AV modes?  Or is there an easy hack I can add or delete a line in my ETTR module to accomplish this?
Title: Re: Auto Dissable ETTR outside M mode
Post by: a1ex on October 06, 2013, 11:15:27 PM
Yes, it's very easy.
Title: Re: Auto Dissable ETTR outside M mode
Post by: RenatoPhoto on October 07, 2013, 01:09:01 AM
 :o
Title: Re: Auto Dissable ETTR outside M mode
Post by: Audionut on October 07, 2013, 02:53:32 AM
Quote from: RenatoPhoto on October 06, 2013, 11:11:51 PM
1.  I am glad it is not just me making some stupid mistake.  ETTR and TV can produce bad results.

With the Canon exposure meter being center (focus point in evaluative (http://media.digitalcameraworld.com/files/2012/06/Metering_mode_photography_cheat_sheet.jpg)) weighted, any slight changes to framing can cause the exposure settings to change (in Auto/Semi-Auto modes).
Move the frame (focus point in evaluative) to outside and the exposure calculation will apply more weight to that outside scene in it's exposure decision.  The same applies if you then move the frame/focus point to the inside, where the exposure will now have weight on the inside.

AETTR in semi-auto modes adjusts Exposure Compensation. (http://en.wikipedia.org/wiki/Exposure_compensation)

You must give AETTR enough time to settle on the correct settings.  If you expose for the outside where it's probably going to use +-0EC or maybe even a positive EC if enough of the outside is in the frame, when you move to the inside, this same positive EC value will overexpose the outside part of the frame (in your inside shot) by a large margin.
As we all should know with overexposure, AETTR does not know exactly how far the scene has been overexposed (1 stop, 3 stops, 14 stops, it doesn't know it only guesses, see lines 289-295 (https://bitbucket.org/hudson/magic-lantern/src/9316f87066ad6f1d2d802af4ab7496dfa7654b93/modules/ettr/ettr.c?at=unified)*code box below), so it will probably take 2 or 3 tries before it gets it right.

            /* scene changed? measurements from previous shot not confirmed or vary too much?
             *
             * we'll use a heuristic: for 1% of blown out image, go back 1EV, for 100% go back 13EV */
            float overexposed = raw_hist_get_overexposure_percentage(GRAY_PROJECTION_AVERAGE_RGB | GRAY_PROJECTION_DARK_ONLY) / 100.0;
            //~ bmp_printf(FONT_MED, 0, 80, "overexposure area: %d/100%%\n", (int)(overexposed * 100));
            //~ bmp_printf(FONT_MED, 0, 120, "fail info: (%d %d %d %d) (%d %d %d)", raw_values[0], raw_values[1], raw_values[2], raw_values[3], (int)(diff_from_lower_percentiles[0] * 100), (int)(diff_from_lower_percentiles[1] * 100), (int)(diff_from_lower_percentiles[2] * 100));
            float corr = - log2f(1 + overexposed*overexposed);


Where AETTR has settled on the inside shot with the outside window in frame, there will now be a large negative EC value.  If you need to ask why this is the case, I suggest you read up on the Canon exposure modes and how EC effects exposure.

A large negative EC value is going to be incorrect on almost all scenes expect those with massive DR, and where you are exposing for the shadows and compensating for the highlights.

Move the frame back to the outside and Canon is going to meter for it, but, with the large negative EC value still applied it's going to take a shot for AETTR to realize.  The benefit being that it's going to be underexposed so it should only take AETTR one capture to settle itself.

In summary, AETTR+Av/TV works as expected.
Title: Re: Auto Dissable ETTR outside M mode
Post by: Audionut on October 07, 2013, 03:27:03 AM
Observe these 3 shots in Av mode + AETTR.

ISO 100 - f/3.5 - 1/1600s - EC -2EV - Focus point center
(https://dl.dropboxusercontent.com/u/34113196/ML/ETTR/_46A1482.jpg)

Moving the focus point to the bottom right
ISO 100 - f/3.5 - 1/200s - EC -2EV
(https://dl.dropboxusercontent.com/u/34113196/ML/ETTR/_46A1483.jpg)

AETTR corrected
ISO 100 - f/3.5 - 1/1600s - EC -5EV
(https://dl.dropboxusercontent.com/u/34113196/ML/ETTR/_46A1484.jpg)

The second shot has blown highlights because the Canon meter is now metering (weighting) the shadows and this is the first shot with new exposure settings (changed by the Canon metering in semi-auto) so AETTR doesn't know that something has changed.

The first and last shots have the same end result exposure settings (highlight correct), with different EC setting.
The last shot required -5EV of Exposure Compensation because the Canon meter was metering (weighting) the shadows, and AETTR applied this -5EV of EC to a different set of base exposure settings (compared to the focus point center shot) based on those Canon meter readings.

Metadata has been saved with the above shots