[DONE - lens.focus updated] Lua Request/Idea

[garry23]

@BBA

My all in one bracketing script is now working in two focus modes: from the FP to infinity and from the FP to the macro end.

I got it running through 'hard linking' in the macro minimum FP.

BTW here is the full script: https://gist.github.com/pigeonhill/b67462aec370495fdef8de74f303c734

Cheers

Garry

[BBA]

@garry23

Quite impressed by the way your work... I thank you very much for sharing this.

I will need some time to study it and will report any feedback. From my part, I do prefer something that works and is correct (even if I have to be wrong) don't you?

I would like to know what you mean by "registering the lens".

BTW, is it possible to attach a spreadsheet file to a forum post ? Above the "post reply window" there are insert icons (insert vimeo video url, insert image, insert hyperlink, insert mail, insert FTP link, insert table, insert code, insert quote)...

[garry23]

@BBA

Sorry, I don't know how to attach a spreadsheet to a post 

I tend to write up my work and photography on my blog at http://photography.grayheron.net/

The 'registering' idea was my simple way of getting round not being able to solve the soft limit error.

All you need to do, when you go into the focus bracketing submenu is to rotate the lens to the macro end and tell the script this is the macro stop, ie all you do is press SET. This macro stop remains good whilst the camera is on.

I still haven't worked out how to store my own variables, ie when the camera switches off: next job 

[dmilligan]

Still have problems with lens.focus? Does the API test work correctly?

Code: [Select]

        printf("Focus distance: %s\n",  lens.focus_distance)

        -- note: focus direction is not consistent
        -- some lenses will focus to infinity, others to macro
        printf("Focusing backward...\n")
        while lens.focus(-1,3,true) do end
        printf("Focus distance: %s\n",  lens.focus_distance)

        msleep(500)
       
        for i,step in pairs{3,2,1} do
            for j,wait in pairs{true,false} do
                printf("Focusing forward with step size %d, wait=%s...\n", step, wait)
                local steps_front = 0
                while lens.focus(1,step,true) do
                    printf(".")
                    steps_front = steps_front + 1
                end
                printf("\n")
                printf("Focus distance: %s\n",  lens.focus_distance)
               
                msleep(500)
               
                printf("Focusing backward with step size %d, wait=%s...\n", step, wait)
                local steps_back = 0
                while lens.focus(-1,step,true) do
                    printf(".")
                    steps_back = steps_back + 1
                end
                printf("\n")
                printf("Focus distance: %s\n",  lens.focus_distance)

                msleep(500)

                printf("Focus range: %s steps forward, %s steps backward. \n",  steps_front, steps_back)
            end
        end
        printf("Focus test completed.\n")

If not, does it work correctly on some camera/lenses and not on others? Which ones?

(It works fine here on 60D)

[garry23]

Just ran the API test several times and it failed each time on a 5D3 + 24-105 F4/L in manual  mode.

Code: [Select]

===============================================================================
ML/SCRIPTS/TEST.LUA - 2016-6-27 22:18:55
===============================================================================

Strict mode tests...
Strict mode tests passed.

Generic tests...
camera = table:
  shutter = table:
    raw = 115
    apex = 7.375
    ms = 6
    value = 0.006024
  aperture = table:
    raw = 67
    apex = 7.375
    value = 12.8
    min = table:
      raw = 40
      apex = 4
      value = 4
    max = table:
      raw = 80
      apex = 9
      value = 22.6
  iso = table:
    raw = 72
    apex = 5
    value = 100
  ec = table:
    raw = 0
    value = 0
  flash_ec = table:
    raw = -5
    value = -0.624999
  kelvin = 2500
  mode = 3
  metering_mode = 3
  drive_mode = 0
  model = "Canon EOS 5D Mark III"
  model_short = "5D3"
  firmware = "1.1.3"
  temperature = 151
  state = 0
  reboot = function: p
  shoot = function: p
  bulb = function: p
event = table:
  pre_shoot = nil
  post_shoot = nil
  shoot_task = nil
  seconds_clock = nil
  keypress = nil
  custom_picture_taking = nil
  intervalometer = nil
  config_save = nil
console = table:
  write = function: p
  show = function: p
  clear = function: p
  hide = function: p
lv = table:
  enabled = true
  paused = false
  running = true
  zoom = 1
  start = function: p
  resume = function: p
  info = function: p
  stop = function: p
  pause = function: p
  wait = function: p
lens = table:
  name = "EF24-105mm f/4L IS USM"
  focal_length = 24
  focus_distance = 1820
  hyperfocal = 5649
  dof_near = 1383
  dof_far = 2659
  af = true
  af_mode = 0
  focus = function: p
display = table:
  idle = true
  height = 480
  width = 720
  screenshot = function: p
  rect = function: p
  pixel = function: p
  on = function: p
  load = function: p
  line = function: p
  off = function: p
  notify_box = function: p
  print = function: p
  draw = function: p
  clear = function: p
  circle = function: p
key = table:
  last = 0
  press = function: p
  wait = function: p
menu = table:
  visible = false
  close = function: p
  get = function: p
  block = function: p
  set = function: p
  new = function: p
  open = function: p
testmenu = userdata:
  value = 0
  name = "Script API tests"
  help = "Various tests for the Lua scripting API."
  help2 = "When adding new Lua APIs, tests for them should go here."
  advanced = 0
  depends_on = 0
  edit_mode = 0
  hidden = false
  icon_type = 5
  jhidden = false
  max = 0
  min = 0
  selected = true
  shidden = false
  starred = false
  submenu_height = 0
  submenu_width = 0
  unit = 0
  works_best_in = 0
  run_in_separate_task = 0
  select = function: p
  update = nil
  info = nil
  rinfo = nil
  warning = nil
movie = table:
  recording = false
  start = function: p
  stop = function: p
dryos = table:
  clock = 29
  ms_clock = 29777
  prefix = "2P6A"
  dcim_dir = table:
    exists = true
    create = function: p
    children = function: p
    files = function: p
    parent = table:
      exists = true
      create = function: p
      children = function: p
      files = function: p
      parent = table:
        exists = true
        create = function: p
        children = function: p
        files = function: p
        parent = nil
        path = "B:/"
      path = "B:/DCIM/"
    path = "B:/DCIM/100EOS5D/"
  config_dir = table:
    exists = true
    create = function: p
    children = function: p
    files = function: p
    parent = table:
      exists = true
      create = function: p
      children = function: p
      files = function: p
      parent = table:
        exists = true
        create = function: p
        children = function: p
        files = function: p
        parent = nil
        path = "A:/"
      path = "ML/"
    path = "ML/SETTINGS/"
  ml_card = table:
    cluster_size = 32768
    drive_letter = "A"
    file_number = 2459
    folder_number = 100
    free_space = 27909408
    type = "CF"
    path = "A:/"
    _card_ptr = userdata
  shooting_card = table:
    cluster_size = 131072
    drive_letter = "B"
    file_number = 2459
    folder_number = 100
    free_space = 51559424
    type = "SD"
    path = "B:/"
    _card_ptr = userdata
  date = table:
    month = 6
    min = 18
    yday = 179
    isdst = false
    sec = 57
    year = 2016
    day = 27
    hour = 22
    wday = 2
  directory = function: p
  call = function: p
  remove = function: p
interval = table:
  time = 0
  count = 0
  running = 0
  stop = function: p
battery = table:
  level = 51
  id = 3
  performance = 3
  time = 3519
  drain_rate = 52
task = table:
  create = function: p
  yield = function: p
property = table:
Generic tests completed.

Module tests...
Copy test: autoexec.bin -> tmp.bin
Copy test OK
Append test: tmp.txt
Append test OK
Testing exposure settings, module 'camera'...
Camera    : Canon EOS 5D Mark III (5D3) 1.1.3
Lens      : EF24-105mm f/4L IS USM
Shoot mode: 3
Shutter   : ‚160 (raw 115, 0.006024s, 6ms, apex 7.375)
Aperture  : 12 (raw 67, f/12.8, apex 7.375)
Av range  : 4.0..22 (raw 40..80, f/4..f/22.6, apex 4..9)
ISO       : 100 (raw 72, 100, apex 5)
EC        : 0.0 (raw 0, 0 EV)
Flash EC  : -0.6 (raw -5, -0.624999 EV)
Setting shutter to random values...

===============================================================================
ML/SCRIPTS/TEST.LUA - 2016-6-27 22:22:03
===============================================================================

Strict mode tests...
Strict mode tests passed.

Generic tests...
camera = table:
  shutter = table:
    raw = 125
    apex = 8.624999
    ms = 3
    value = 0.002532
  aperture = table:
    raw = 67
    apex = 7.375
    value = 12.8
    min = table:
      raw = 40
      apex = 4
      value = 4
    max = table:
      raw = 80
      apex = 9
      value = 22.6
  iso = table:
    raw = 72
    apex = 5
    value = 100
  ec = table:
    raw = 0
    value = 0
  flash_ec = table:
    raw = -5
    value = -0.624999
  kelvin = 2500
  mode = 3
  metering_mode = 3
  drive_mode = 0
  model = "Canon EOS 5D Mark III"
  model_short = "5D3"
  firmware = "1.1.3"
  temperature = 152
  state = 0
  shoot = function: p
  bulb = function: p
  reboot = function: p
event = table:
  pre_shoot = nil
  post_shoot = nil
  shoot_task = nil
  seconds_clock = nil
  keypress = nil
  custom_picture_taking = nil
  intervalometer = nil
  config_save = nil
console = table:
  write = function: p
  clear = function: p
  hide = function: p
  show = function: p
lv = table:
  enabled = true
  paused = false
  running = true
  zoom = 1
  stop = function: p
  resume = function: p
  pause = function: p
  start = function: p
  info = function: p
  wait = function: p
lens = table:
  name = "EF24-105mm f/4L IS USM"
  focal_length = 24
  focus_distance = 1060
  hyperfocal = 5649
  dof_near = 895
  dof_far = 1298
  af = true
  af_mode = 0
  focus = function: p
display = table:
  idle = true
  height = 480
  width = 720
  rect = function: p
  notify_box = function: p
  on = function: p
  draw = function: p
  off = function: p
  pixel = function: p
  load = function: p
  print = function: p
  circle = function: p
  screenshot = function: p
  line = function: p
  clear = function: p
key = table:
  last = 0
  press = function: p
  wait = function: p
menu = table:
  visible = false
  new = function: p
  get = function: p
  open = function: p
  set = function: p
  block = function: p
  close = function: p
testmenu = userdata:
  value = 0
  name = "Script API tests"
  help = "Various tests for the Lua scripting API."
  help2 = "When adding new Lua APIs, tests for them should go here."
  advanced = 0
  depends_on = 0
  edit_mode = 0
  hidden = false
  icon_type = 5
  jhidden = false
  max = 0
  min = 0
  selected = true
  shidden = false
  starred = false
  submenu_height = 0
  submenu_width = 0
  unit = 0
  works_best_in = 0
  run_in_separate_task = 0
  select = function: p
  update = nil
  info = nil
  rinfo = nil
  warning = nil
movie = table:
  recording = false
  start = function: p
  stop = function: p
dryos = table:
  clock = 23
  ms_clock = 23938
  prefix = "2P6A"
  dcim_dir = table:
    exists = true
    create = function: p
    children = function: p
    files = function: p
    parent = table:
      exists = true
      create = function: p
      children = function: p
      files = function: p
      parent = table:
        exists = true
        create = function: p
        children = function: p
        files = function: p
        parent = nil
        path = "B:/"
      path = "B:/DCIM/"
    path = "B:/DCIM/100EOS5D/"
  config_dir = table:
    exists = true
    create = function: p
    children = function: p
    files = function: p
    parent = table:
      exists = true
      create = function: p
      children = function: p
      files = function: p
      parent = table:
        exists = true
        create = function: p
        children = function: p
        files = function: p
        parent = nil
        path = "A:/"
      path = "ML/"
    path = "ML/SETTINGS/"
  ml_card = table:
    cluster_size = 32768
    drive_letter = "A"
    file_number = 2459
    folder_number = 100
    free_space = 27909408
    type = "CF"
    _card_ptr = userdata
    path = "A:/"
  shooting_card = table:
    cluster_size = 131072
    drive_letter = "B"
    file_number = 2459
    folder_number = 100
    free_space = 51559424
    type = "SD"
    _card_ptr = userdata
    path = "B:/"
  date = table:
    wday = 2
    yday = 179
    isdst = false
    min = 22
    sec = 4
    month = 6
    hour = 22
    day = 27
    year = 2016
  remove = function: p
  call = function: p
  directory = function: p
interval = table:
  time = 0
  count = 0
  running = 0
  stop = function: p
battery = table:
  level = 50
  id = 3
  performance = 3
  time = 3450
  drain_rate = 52
task = table:
  create = function: p
  yield = function: p
property = table:
Generic tests completed.

Module tests...
Copy test: autoexec.bin -> tmp.bin
Copy test OK
Append test: tmp.txt
Append test OK
Testing exposure settings, module 'camera'...
Camera    : Canon EOS 5D Mark III (5D3) 1.1.3
Lens      : EF24-105mm f/4L IS USM
Shoot mode: 3
Shutter   : ‚400 (raw 125, 0.002532s, 3ms, apex 8.624999)
Aperture  : 12 (raw 67, f/12.8, apex 7.375)
Av range  : 4.0..22 (raw 40..80, f/4..f/22.6, apex 4..9)
ISO       : 100 (raw 72, 100, apex 5)
EC        : 0.0 (raw 0, 0 EV)
Flash EC  : -0.6 (raw -5, -0.624999 EV)
Setting shutter to random values...


[a1ex]

The test log is incomplete => auto power off was enabled?

[garry23]

Script fails at Lua 213 "shutter"

BTW Auto Power is OFF

[garry23]

OK worked it out: Expo Override was ON.

All tests worked.

BTW Your lens.focus works, ie doesn't generate a soft focus error: but mine does 

I use lens.focus(1,1,false) as I thought setting false meant the lens wouldn't wait for focus: is that correct?

[a1ex]

Script fails at Lua 213 "shutter"

That means, you found a bug in the API test script: it doesn't log errors.

I use lens.focus(1,1,false) as I thought setting false meant the lens wouldn't wait for focus: is that correct?

Disabling the wait flag should give smoother motion (as the focus commands are queued), but I'm not sure how well error handling works in this case. If in doubt, enable this flag (so it focuses one step at a time and waits for confirmation).

[garry23]

A1ex

Post crossed. API script runs ok if Expo Override OFF.

Here is the relevant piece of my script that is not 'working'.

Code: [Select]

soft_err = true
repeat
soft_err = lens.focus(1,1,true)
until soft_err == false

It stops straight away, implying lens.focus is returning false.

I can't work it out

[a1ex]

From api_test.lua:

Code: [Select]

    while lens.focus(-1,3,true) do end

Does this one work?

[garry23]

I'll do some checking tonight and report back.

Cheers

[BBA]

focus test posted by dmilligan works fine for me (with 5DmIII and 35mm f2 IS USM) (except, as already noted, the lens move direction which is, in this case, the opposite one)

[garry23]

@BBA (A1ex/David)

I have convinced myself my original script is corrupting things, but I don't know why.

The following test script works as expected, ie detects the soft limit reached:

Code: [Select]

soft_err = true
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance


function move(direction)
local a1 = 0
local b1 = 0
if direction == -1 then -- focus stack ti infinity soft stop
repeat
a1 = a2
b1 = b2
repeat
soft_err = lens.focus(-1,1,false)
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance
until a2> b1 or soft_err == false
msleep(1000)
until soft_err == false
else  -- focus stack to macro soft stop
repeat
a1 = a2
b1 = b2
repeat
soft_err = lens.focus(1,1,false)
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance
until b2 < a1 or soft_err == false
msleep(1000)
until soft_err == false
end
end

function start() -- test out focus stacking works through the soft stop error return
move(-1)
move(1)
move(-1)
end

scrnshot_menu = menu.new
{
    name = "Test",
select = function(this)
task.create(start)
end
}


The only 'problem' I see is that if lens.focus generates another error, my script will detect that but assume the soft limit has been reached. That is I can't differentiate lens.focus error code returns.

For this reason I had to use false in lens.focus, as it could generate a focus error if I use true.

I think I will redo my focus & exposure bracketing script, starting from the above test script.

Cheers

Garry

[BBA]

@garry23

I am glad you found some working solution. As A1ex said, try to simplify as much as possible.

I don't know why you have to use lens.focus with "no wait" (false) option. I wondered the other day, if you didn't use special Canon focus settings in your 5dmIII as you are a specialist (birds photographs?). In my case, I only use default settings as the focus settings may be quite complicated for me: I haven't had time to study them : I didn't find out as lens.focus is a low level command and the very sophisticated Canon settings seem to work at a higher level.

I don't know if lens.focus can return false in other circumstances then when the software limits are reached. It is a good question! I am not used enough to best lua programming practices to know if it would be a good practice (Imho, I would differentiate the returns...). I would love to know how to best deal with errors in lua.

From my experiments, the focal point returned by lens.focus is a step function https://en.wikipedia.org/wiki/Step_function (which I can understand) and not a continuous one https://en.wikipedia.org/wiki/Continuous_function. We have to know that because it changes the behavior of your algorithm.

Your algorithm works perfectly with a continuous function. With a step function, it detects the discontinuities (steps) which is very good and effective overall  (there are enough steps), but can lead, in theory, to focus overlapping problems (dof, diffraction,...) (to be more studied).
Don't worry : if needed, I think this can be solved by interpolating between steps and forcing more pictures when approaching infinity (I can help if needed).

[garry23]

@BBA

BTW in the test code I left out handling focus overlaps, eg in the 'real' code I do something like this:

Code: [Select]

if direction == -1 then -- focus stack ti infinity soft stop
repeat
a1 = a2
b1 = b2
repeat
soft_err = lens.focus(-1,1,false)
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance
until a2> b1 or soft_err == false
                                lens.focus(1,1,false) -- move back to ensure focus overlap
msleep(1000)
until soft_err == false

[BBA]

You are right and this is very logical (with a continuous function). In our case, this is done at each step boundary which translates all the pictures by one step to macro end.

Also, logically, if you decrease the coc, the number of pictures to be taken should increase (automatically). Inversely if you dramatically increase the coc, at the limit, only one picture could be sufficient, couldn't it ? Here, imho, the number of pictures taken should not change as much.
But, there isn't to worry very much as there are many steps (range of 22 steps for the EF 24mm f/1.4 II USM  + 1 at both ends if we use the software limit).

[garry23]

I hope David or A1ex read this as I've worked out what is causing my script to fail. Here is the test script:

Code: [Select]

soft_err = true
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance


function move(direction)
local a1 = 0
local b1 = 0
if direction == -1 then -- focus stack ti infinity soft stop
repeat
a1 = a2
b1 = b2
repeat
soft_err = lens.focus(-1,1,false)
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance
until a2> b1 or soft_err == false
msleep(1000)
until soft_err == false
else  -- focus stack to macro soft stop
repeat
a1 = a2
b1 = b2
repeat
soft_err = lens.focus(1,1,false)
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance
until b2 < a1 or soft_err == false
camera.shoot(64, false)
until soft_err == false
end
end

function start() -- test out focus stacking works through the soft stop error return
move(-1)
move(1)
move(-1)
end

scrnshot_menu = menu.new
{
    name = "Test",
select = function(this)
task.create(start)
end
}
If I don't use camera.shoot(64, false), ie dummy it out with a msleep(1000), say, as in the block of code above the camera.shoot(64, false) call, all is well.

As soon as camera.shoot(64, false) is called in the above, the script stops, ie camera.shoot(64, false) seems to cause lens.focus to return a false.

Does anyone have any insight as to the logic here?

[BBA]

@garry23

Lest's suppose you replace 64 (as wait time) in shoot with 1000 as in msleep.
I wonder: when you call lens.focus with false, the focus commands are queued, let's say 30 times lens.focus per step. Then the camera.shoot is called with a wait time of 64 (ms?).
Maybe if you give a little more time to camera.shoot so the focus commands can be unqueued ?

[a1ex]

When calling lens.focus, are you in LiveView?

[garry23]

A1ex

Yes. The script runs in LV.

Cheers

Garry

[garry23]

@BBA

I think I understand what you are saying, however, the need is to find a way forward without using AF.

In my focus stacking script I do not want the lens AF to do any work: for two reasons.

First, I'm focusing at the 'correct' FP.

Second, in a low contrast scene the focus may not be found.

Bottom line: I need to find a way for lens.focus to work with camera.shoot when both have AF as false.

[a1ex]

garry23: to make sure you are doing each lens.focus call in LiveView, try this:

Code: [Select]

assert(lv)
lens.focus(...)


[garry23]

@A1ex

Tried the assert but same result.

If camera.shoot is called, it then appears to trigger lens.focus to return a false.

BTW here is the test script with the assert added in:

Code: [Select]

soft_err = true
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance


function move(direction)
local a1 = 0
local b1 = 0
if direction == -1 then -- focus stack ti infinity soft stop
repeat
a1 = a2
b1 = b2
repeat
assert(lv)
soft_err = lens.focus(-1,1,false)
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance
until a2> b1 or soft_err == false
msleep(1000)
until soft_err == false
else  -- focus stack to macro soft stop
repeat
a1 = a2
b1 = b2
repeat
assert(lv)
soft_err = lens.focus(1,1,false)
b2 = lens.dof_far
a2 = lens.dof_near
fp = lens.focus_distance
until b2 < a1 or soft_err == false
camera.shoot(64, false)
until soft_err == false
end
end

function start() -- test out focus stacking works through the soft stop error return
move(-1)
move(1)
move(-1)
end

scrnshot_menu = menu.new
{
    name = "Test",
select = function(this)
task.create(start)
end
}

[a1ex]

That means, the assert doesn't get triggered, correct?

After taking a picture, you should wait for it to finish and return to LiveView before sending more focus commands. The assert above checks just for that, but since it didn't trigger, it looks like the camera already returned to LiveView.

You could add a large delay after taking the picture, maybe it helps. I'm not able to try it right now, but I'll add an API test for this scenario as well.