Lua camera.shutter.apex vs camera.shutter.ms

[garry23]

David

I think I've now confirmed there is something strange happening with camera.shutter.apex. I've reported a 'bug'.

The code below works with shutter.ms but not with shutter.apex

Cheers

Garry

Code: [Select]

--[[
Manual Landscape bracketing helper script
Version 0.1.0 for EOS-M
Garry George March 2016
http://photography.grayheron.net/
Workflow helper script for focus and exposure bracketing for the EOS-M
Toggle through the workflow using the top of the main dial.
Yellow means move more towards the infinity end, Red means move more towards macro end,
Green means at the sweet spot. If Green not seen, move to just being in the Yellow from being in the Red.
--]]
factor = 0.2
b1 = lens.dof_far
fp = lens.focal_distance
c1 = b1 - (b1 - fp*10)*factor
inf = 1000000
started = false
current_tv = 0
play = 0

event.keypress = function(key)
if keymenu.submenu["Turn On/Off"].value == "Off" then
started = false
return true
elseif key == 12
then
play = play + 1
if play == 5 then play = 0 end
return false
elseif key == KEY.FULLSHUTTER
then
b1 = lens.dof_far -- in mm
fp = lens.focal_distance -- in cm
c1 = b1 - (b1 - fp*10)*factor
started = true
return true
end
end

lv.info
{
    name = "Info 1",
    value = "",
    priority = 100,
    update = function(this)
    this.value = ""
    if keymenu.submenu["Turn On/Off"].value == "On" and started and fp ~= 0
    then
this.foreground = COLOR.BLACK
a2 = lens.dof_near
if a2 > b1 then
this.background = COLOR.RED
elseif a2 < c1 then
this.background = COLOR.YELLOW
else
this.background = COLOR.GREEN1
end
if lens.dof_far >= inf then
this.value = "INF"
else
this.value = "   "
end
else
this.value = ""
end
end
}

lv.info
{
    name = "Info 2",
    priority = 100,
    value = "",
    update = function(this)
    if keymenu.submenu["Turn On/Off"].value == "On" then
    i = menu.set("Shoot","Advanced Bracket",0)
i = menu.set("Expo","Auto ETTR",0)
this.foreground = COLOR.YELLOW
this.background = COLOR.BLUE
if keymenu.submenu["Turn On/Off"].value == "On" then
if play == 0 then
current_tv = camera.shutter.ms
this.value = " Off"
elseif play == 1 then
this.value = "-BE "
camera.shutter.ms = 1
i = menu.set("Shoot","Advanced Bracket",0)
elseif play == 2 then
this.value = "BE+ "
camera.shutter.ms = current_tv
elseif play == 3 then
this.value = "ETTR"
i = menu.set("Expo","Auto ETTR",1)
i = menu.set("Auto ETTR","Trigger mode",3)
elseif play == 4 then
this.value = "Auto"
i = menu.set("Shoot","Advanced Bracket",1)
else
this.value = ""
end
end
end
end
}

keymenu = menu.new
{
parent = "Shoot",
name = "Landscape Helper",
help = "Toggle through workflow using top of main dial",
help2 = "Grab exposure or take picture as required",
submenu =
{
{
name = "Turn On/Off",
help = "Switches the script on/off",
help2 = "Simply enables additional button functionality",
choices = {"On","Off"},
},
{
name = "Bracket to bracket overlap",
help = "Amount to overlap each bracket",
help2 = "% of FP-2-DoF(far) distance",
choices = {"20","10", "5"},
update = function(this)
factor = tonumber(keymenu.submenu["Bracket to bracket overlap"].value)/100
end,
}
}
}

[dmilligan]

http://catb.org/~esr/faqs/smart-questions.html#beprecise
http://catb.org/~esr/faqs/smart-questions.html#symptoms

The most effective way to be precise about a code problem is to provide a minimal bug-demonstrating test case. What's a minimal test case? It's an illustration of the problem; just enough code to exhibit the undesirable behavior and no more. How do you make a minimal test case? If you know what line or section of code is producing the problematic behavior, make a copy of it and add just enough supporting code to produce a complete example

[garry23]

My humble apologies as I thought I had done that on the bug report.

The above was to confirm, through my testing, the bug appears real.

This example piece of code from my script illustrates:

Code: [Select]

if keymenu.submenu["Turn On/Off"].value == "On" then
if play == 0 then
current_tv = camera.shutter.ms
this.value = " Off"
elseif play == 1 then
this.value = "-BE "
camera.shutter.ms = 1
i = menu.set("Shoot","Advanced Bracket",0)
elseif play == 2 then
this.value = "BE+ "
camera.shutter.ms = current_tv
elseif
In the above extract from my script, if I replace camera.shutter.ms with camera.shutter.apex the code does not work as expected, ie the current_tv is not returned as expected as play is indexed.

To my humble eyes, the camera.shutter.apex appears to have a 'bug' somewhere.

Sorry I can't be more helpful.

[dmilligan]

From quick inspection, there doesn't appear to be anything wrong with camera.shutter.apex, so please post a MCVE, not just an excerpt from your script.

I'm asking you to put forth a little more effort to solve this problem. I was hoping you would get the nudge, but it seems not. So I'll be more explicit: Have you printed out the values to see what they were? What value does camera.shutter.apex return for various shutter speeds? Is this what you expected? What about compared to camera.shutter.ms? What value are you setting it to? When you set it to some hardcoded value does it set to the value you expect (try lots of different values)? If not, what value did you set it to, what was the expected shutter speed and what was the actual shutter speed? (and so forth)

These are all time consuming questions I would have to answer myself, but you are perfectly capable of answering, and then encapsulating in a simple MCVE.

[garry23]

Hint taken 

[a1ex]

This should help:

Code: [Select]

function printf(s,...)
    print(string.format(s,...))
end

function api_tests()
    menu.close()
    console.clear()
    console.show()
    printf("Testing module 'camera'...")
    printf("Camera    : %s (%s) %s", camera.model, camera.model_short, camera.firmware)
    printf("Shoot mode: %s", camera.mode)
    printf("Shutter   : %s (raw %s, %ss, %sms, apex %s)", camera.shutter, camera.shutter.raw, camera.shutter.value, camera.shutter.ms, camera.shutter.apex)
    printf("Aperture  : %s (raw %s, 1/%s, apex %s)", camera.aperture, camera.aperture.raw, camera.aperture.value, camera.aperture.apex)
    printf("ISO       : %s (raw %s, %s, apex %s)", camera.iso, camera.iso.raw, camera.iso.value, camera.iso.apex)
    printf("EC        : %s (raw %s, %s EV)", camera.ec, camera.ec.raw, camera.ec.value)
    printf("Flash EC  : %s (raw %s, %s EV)", camera.flash_ec, camera.flash_ec.raw, camera.flash_ec.value)
    printf("Kelvin    : %s", camera.kelvin)
    key.wait()
    console.hide()
end

keymenu = menu.new
{
    name = "Script API tests",
    select = function(this) task.create(api_tests) end,
}

I used it to check the lua_fix branch, and also wanted to change some values before giving the green light and including it in the nightly.

Note this script is not exactly compatible with the nightly (I did some tweaks to the API), so just compile the lua_fix branch before trying it.

Extensions to this script are welcome.

[garry23]

David/A1ex

Just to confirm there appears to be a bug in the camera.shutter area of Lua. To confirm this, on my EOS-M, I have used this very simple test script:

Code: [Select]

a,b,c = 0,0,0

menu.new
{
  name = "Run",
  select = function() task.create(run) end,
}

run = function()
display.clear()
a = camera.shutter.ms
camera.shutter.ms = 10
b = camera.shutter.ms
camera.shutter.ms = a
c = camera.shutter.ms
i = a..","..b..","..c
display.notify_box (i,4000)
end
If, for example I set an ML (bottom bar) reported shutter time of 20 sec, the above reports 20749,10,20749. The key reporting being the first and last ms shutter speeds are the same.

I now use this script, which only replaces camera.shutter.ms with camera.shutter.apex:

Code: [Select]

a,b,c = 0,0,0

menu.new
{
  name = "Run",
  select = function() task.create(run) end
}

run = function()
display.clear()
a = camera.shutter.apex
camera.shutter.apex = 100
b = camera.shutter.apex
camera.shutter.apex = a
c = camera.shutter.apex
i = a..","..b..","..c
display.notify_box (i,5000)
end
Now the reporting is: -43,100,-40. The key thing being that the first and last shutter times are different. This never occurs with the .ms version.

I hope this helps developers

[garry23]

David

Was the last post of any help?  Is there a bug in the APEX part of camera.shutter?

Cheers

Garry

[a1ex]

Yes, it's a bug.

Traced back to 2013, changeset 0cb3891954fa. I should probably try things more thoroughly after others merge them, rather than trusting their decision blindly...

[dmilligan]

There's also the matter of how apex values are rounded for 1/3 stop increments when they are computed. Currently you get the value rounded to the nearest 10th (i.e. -4.3), but since it's floating point now perhaps we can be more precise (i.e. -4.333333). Although, since the raw value is in 1/8 EV, that begs the question of what the true value is. Is this just Canon's internal implementation and the true values are actually 1/3 stop, or are we actually getting 3/8 and 5/8 instead (i.e. -4.375)? I don't really know, and I imagine that 1/24 EV difference would be hard to measure.

[a1ex]

Solved, so going back and forth should give the same value now.

As for the true value, one way to measure them would be to check the shutter speed reported by ML in movie mode.

On 5D3, this is read from ADTG registers, so it is as low-level as it can be. Values for 1080p, covering all shutter speeds available in movie mode without expo override:

Code: [Select]

video mode | shutter setting | shutter blanking | frame duration (FPS timer B) | computed shutter (us, s)
1080p25        96, 1/30             500                   2000                     30000, 1/33.333
1080p25        99, 1/40             796                   2000                     24080, 1/41.528
1080p25       100, 1/45             896                   2000                     22080, 1/45.290
1080p25       101, 1/50            1000                   2000                     20000, 1/50.000
1080p25       104, 1/60            1219                   2000                     15620, 1/64.020
1080p25       107, 1/80            1398                   2000                     12040, 1/83.056
1080p25       108, 1/90            1448                   2000                     11040, 1/90.580
1080p25       109, 1/100           1500                   2000                     10000, 1/100.000
1080p25       112, 1/125           1610                   2000                      7800, 1/128.205
1080p25       115, 1/160           1699                   2000                      6020, 1/166.113
1080p25       116, 1/180           1724                   2000                      5520, 1/181.159
1080p25       117, 1/200           1747                   2000                      5060, 1/197.628
1080p25       120, 1/250           1805                   2000                      3900, 1/256.410
1080p25       123, 1/320           1850                   2000                      3000, 1/333.333
1080p25       124, 1/350           1862                   2000                      2760, 1/362.319
1080p25       125, 1/400           1874                   2000                      2520, 1/396.825
1080p25       128, 1/500           1903                   2000                      1940, 1/515.464
1080p25       131, 1/640           1925                   2000                      1500, 1/666.667
1080p25       132, 1/750           1931                   2000                      1380, 1/724.638
1080p25       133, 1/800           1937                   2000                      1260, 1/793.651
1080p25       136, 1/1000          1952                   2000                       960, 1/1041.667
1080p25       139, 1/1250          1963                   2000                       740, 1/1351.351
1080p25       140, 1/1500          1966                   2000                       680, 1/1470.588
1080p25       141, 1/1600          1969                   2000                       620, 1/1612.903
1080p25       144, 1/2000          1976                   2000                       480, 1/2083.333
1080p25       147, 1/2500          1982                   2000                       360, 1/2777.778
1080p25       148, 1/3000          1983                   2000                       340, 1/2941.177
1080p25       149, 1/3200          1985                   2000                       300, 1/3333.334
1080p25       152, 1/4000          1988                   2000                       240, 1/4166.667

1080p30        96, 1/30             184                   1820                     29993, 1/33.341
1080p30        99, 1/40             506                   1820                     24090, 1/41.511
1080p30       100, 1/45             615                   1820                     22092, 1/45.266
1080p30       101, 1/50             730                   1820                     19983, 1/50.042
1080p30       104, 1/60             968                   1820                     15620, 1/64.020
1080p30       107, 1/80            1163                   1820                     12045, 1/83.022
1080p30       108, 1/90            1218                   1820                     11037, 1/90.607
1080p30       109, 1/100           1275                   1820                      9992, 1/100.083
1080p30       112, 1/125           1394                   1820                      7810, 1/128.041
1080p30       115, 1/160           1492                   1820                      6013, 1/166.297
1080p30       116, 1/180           1519                   1820                      5518, 1/181.214
1080p30       117, 1/200           1544                   1820                      5060, 1/197.628
1080p30       120, 1/250           1607                   1820                      3905, 1/256.082
1080p30       123, 1/320           1656                   1820                      3007, 1/332.594
1080p30       124, 1/350           1670                   1820                      2750, 1/363.636
1080p30       125, 1/400           1682                   1820                      2530, 1/395.257
1080p30       128, 1/500           1714                   1820                      1943, 1/514.579
1080p30       131, 1/640           1738                   1820                      1503, 1/665.188
1080p30       132, 1/750           1745                   1820                      1375, 1/727.272
1080p30       133, 1/800           1751                   1820                      1265, 1/790.513
1080p30       136, 1/1000          1767                   1820                       972, 1/1029.159
1080p30       139, 1/1250          1779                   1820                       752, 1/1330.376
1080p30       140, 1/1500          1783                   1820                       678, 1/1474.200
1080p30       141, 1/1600          1786                   1820                       623, 1/1604.277
1080p30       144, 1/2000          1794                   1820                       477, 1/2097.900
1080p30       147, 1/2500          1800                   1820                       367, 1/2727.270
1080p30       148, 1/3000          1802                   1820                       330, 1/3030.300
1080p30       149, 1/3200          1803                   1820                       312, 1/3208.553
1080p30       152, 1/4000          1807                   1820                       238, 1/4195.800

1080p24        96, 1/30             639                   2275                     29993, 1/33.341
1080p24        99, 1/40             961                   2275                     24090, 1/41.511
1080p24       100, 1/45            1070                   2275                     22092, 1/45.266
1080p24       101, 1/50            1185                   2275                     19983, 1/50.042
1080p24       104, 1/60            1423                   2275                     15620, 1/64.020
1080p24       107, 1/80            1618                   2275                     12045, 1/83.022
1080p24       108, 1/90            1673                   2275                     11037, 1/90.607
1080p24       109, 1/100           1730                   2275                      9992, 1/100.083
1080p24       112, 1/125           1849                   2275                      7810, 1/128.041
1080p24       115, 1/160           1947                   2275                      6013, 1/166.297
1080p24       116, 1/180           1974                   2275                      5518, 1/181.214
1080p24       117, 1/200           1999                   2275                      5060, 1/197.628
1080p24       120, 1/250           2062                   2275                      3905, 1/256.082
1080p24       123, 1/320           2111                   2275                      3007, 1/332.594
1080p24       124, 1/350           2125                   2275                      2750, 1/363.636
1080p24       125, 1/400           2137                   2275                      2530, 1/395.257
1080p24       128, 1/500           2169                   2275                      1943, 1/514.579
1080p24       131, 1/640           2193                   2275                      1503, 1/665.188
1080p24       132, 1/750           2200                   2275                      1375, 1/727.272
1080p24       133, 1/800           2206                   2275                      1265, 1/790.513
1080p24       136, 1/1000          2222                   2275                       972, 1/1029.159
1080p24       139, 1/1250          2234                   2275                       752, 1/1330.376
1080p24       140, 1/1500          2238                   2275                       678, 1/1474.200
1080p24       141, 1/1600          2241                   2275                       623, 1/1604.277
1080p24       144, 1/2000          2249                   2275                       477, 1/2097.900
1080p24       147, 1/2500          2255                   2275                       367, 1/2727.270
1080p24       148, 1/3000          2257                   2275                       330, 1/3030.300
1080p24       149, 1/3200          2258                   2275                       312, 1/3208.553
1080p24       152, 1/4000          2262                   2275                       238, 1/4195.800


Code: [Select]

/* to be called from get_current_shutter_reciprocal_x1000 */
static void log_shutter(int blanking, float shutter)
{
    /* log only shutter changes */
    static int old_blanking = 0;
    if (old_blanking == blanking) return;
    old_blanking = blanking;
   
    int shutter_us = (int)roundf(shutter * 1000000);
    int shutter_rec_x1000 = (int)roundf(1.0 / shutter * 1000);

    FILE* f = FIO_CreateFileOrAppend("shutter.log");
    my_fprintf(f, "%4dp%d       %3d, %6s          %6d                 %6d                  %8d, 1/%s%d.%03d\n",
        video_mode_resolution ? 720 : 1080, video_mode_fps,
        lens_info.raw_shutter, lens_format_shutter(lens_info.raw_shutter),
        blanking, fps_timer_b, shutter_us, FMT_FIXEDPOINT3(shutter_rec_x1000)
    );
    FIO_CloseFile(f);
}

Plotting these numbers:


So, the answer to this:

Is this just Canon's internal implementation and the true values are actually 1/3 stop, or are we actually getting 3/8 and 5/8 instead (i.e. -4.375)?

seems to be that we are actually getting 3/8 and 5/8 values. However, at least in movie mode, there seems to be some additional tweaking applied, in particular at 1/50 and 1/100 (probably to avoid flicker in PAL land). The differences at high shutter speeds are probably roundoff errors (adding 5 microseconds to my computed values corrects a large part from the "tail" at the right).

To reproduce the above graph, here's the octave script: plot_shutter.m and the input file: SHUTTER.LOG.

[a1ex]

I've written some pretty harsh tests for ISO and shutter API, checking all the possible methods for setting them (APEX, linear value, raw value). They all pass now on 5D3

https://bitbucket.org/hudson/magic-lantern/src/lua_fix/scripts/api_test.lua