Assign lens focal length and name for non cpu lenses

[zLOST]

btw: even during those bright moments when i had the xmp file written correctly, the exif data in cr2 and jpg (when i've switched to shooting to both formats) were showing the original unaltered data.

If you'd need any help with testing, let me know - i have 650D+6D and couple of non-chipped and chipped adapters to play with.



I've also split the lens name to brand+model, so the lens name is shorter (MOG would eat 20 bytes off the name )

Code: [Select]

lenses = -- {{{
{
    { make = "Carl Zeiss Jena DDR",     name = "Tessar 50mm f/2.8",             focal_length = 50,      manual_aperture = 2.8,  serial = "8333263" },
    { make = "E. Ludwig",               name = "Meritar 50mm f/2.9",            focal_length = 50,      manual_aperture = 2.9,  serial = "1610554" },
    { make = "Helios",                  name = "44-2 58mm f/2",                 focal_length = 58,      manual_aperture = 2,    serial = "7765411" },
    { make = "Industar",                name = "50-2 50mm f/3.5",               focal_length = 50,      manual_aperture = 3.5,  serial = "7149595" },
    { make = "Industar",                name = "61 L/Z (MC) 50mm f/2.8",        focal_length = 50,      manual_aperture = 2.8,  serial = "8707286" },
    { make = "Meyer-Optik Gorlitz",     name = "Telemegor 180mm f/5.5",         focal_length = 180,     manual_aperture = 5.5,  serial = "1728275" },
    { make = "Meyer-Optik Gorlitz",     name = "Trioplan 50mm f/2.9",           focal_length = 50,      manual_aperture = 2.8,  serial = "2224344" },
    { make = "Pentacon",                name = "auto 29mm f/2.8",               focal_length = 29,      manual_aperture = 2.8,  serial = "6308110" },
    { make = "Pentacon",                name = "Prakticar 50mm f/1.8 MC",       focal_length = 50,      manual_aperture = 1.8,  serial = "7710425" },
    { make = "Samyang",                 name = "85mm f/1.4 AS IF UMC",          focal_length = 85,      manual_aperture = 1.4} --       serial = "A217D0264" }
--    { make = "", name = "", focal_length = , manual_aperture = , serial = "" },
} -- }}}

I also had an issue where the lens.lua had crashed when i've selected the last lens in the list - 85mm Samyang. With something like "invalid parameter #3 for __nextval(", i can't remember now and since i'm procrastinating at work, i don't have my camera with me now to replicate it at the moment

and one more thing - the serial number was written as "0" even when it was set cirrectly in xmp.properties.. no idea why..

[aprofiti]

Some of the recent changes in lua library had changed contents of dryos variables and the dryos.dcim_dir.path (used in get_sidecar_filename() @ xmp.lib) does not  work anymore.
Being on linux and using daktable+gimp i have to use IMG_1234.[CR2|JPG].xmp name to have it recognized and used by DT, so i've modified the get_sidecar to:

Code: [Select]

function xmp.get_sidecar_filename() -- {{{
    return dryos.shooting_card:image_path(1) .. ".xmp"
end -- }}}

This refactor is what is causing the issue. Adapting this branch to the new API should do the trick,
Was considering to fix in the PR after a1ex had a look again to the memory management backend.

Maybe try with "dryos.shooting_card:dcim_dir"

I made some changes in the XMP file format (to use the same as exiftool) and tried to save the xmp file line by line to lower the memory consumption..
Unfortunately now it crashes immediately after taking a picture (for a split of a second i can see an error with wiritng to the xmp file (which was working for me for a while, but apparently i broke something a bit later)) and all i end up with is a picture and an assert log

Right, I need to look into that. Maybe backing out b0a2f95 (i.e. re-applying 065ceae) could help.

Should be related to what a1ex said, I noticed the assert for first time in this new build, when switching lens selection; I can't remember if happened also to me under same circumstance now, but probably yes.

Btw: is there a way i could test these scripts in qemu on my own? restarting the camera, mounting the sdcard, copying scripts, unmounting sdcard, starting camera is pretty exhausting and annoying especially when all i find is yet another bug in my own code

I really wanted this was possible at current stage, will make much less stressful and time consuming, but we need to wait a little more

btw: even during those bright moments when i had the xmp file written correctly, the exif data in cr2 and jpg (when i've switched to shooting to both formats) were showing the original unaltered data.

It's minded to work like this: Write metadata to .xml sidecart file (and/or in .mlv if full-res picture or raw video) instead of playing with .cr2 as is considered to be dangerous from what I've read in old discussion in this thread.

Maybe a safe way to inject into .cr2 can be found?

I've also split the lens name to brand+model, so the lens name is shorter (MOG would eat 20 bytes off the name )

Code: [Select]

lenses = -- {{{
{
    { make = "Carl Zeiss Jena DDR",     name = "Tessar 50mm f/2.8",             focal_length = 50,      manual_aperture = 2.8,  serial = "8333263" },
    { make = "E. Ludwig",               name = "Meritar 50mm f/2.9",            focal_length = 50,      manual_aperture = 2.9,  serial = "1610554" },
    { make = "Meyer-Optik Gorlitz",     name = "Telemegor 180mm f/5.5",         focal_length = 180,     manual_aperture = 5.5,  serial = "1728275" },
    { make = "Samyang",                 name = "85mm f/1.4 AS IF UMC",          focal_length = 85,      manual_aperture = 1.4} --       serial = "A217D0264" }
--    { make = "", name = "", focal_length = , manual_aperture = , serial = "" },
} -- }}}


Shouldn't make really a difference in space usage... as Photoshop and similar, need to read the whole name to find correct lens correction profile.

Instead could be useful when you got really long lens name, as I can see from your pictures when selecting a lens from menu;

This could be implemented in the PR to have it in next build if you and others find useful; just need to concatenate "make" and "name" when writing ELNS block and XMP metadata.

I also had an issue where the lens.lua had crashed when i've selected the last lens in the list - 85mm Samyang. With something like "invalid parameter #3 for __nextval(", i can't remember now and since i'm procrastinating at work, i don't have my camera with me now to replicate it at the moment

and one more thing - the serial number was written as "0" even when it was set cirrectly in xmp.properties.. no idea why..

Can you tried to reproduce this? I'm also interested to know if this is happening just in the latest build or not.
Note: the script iterate through attribute in lens table, It may be possible that is not finding the correct values after introducing "make" attribute.

Regarding serial number: save a .mlv and watch for metadata with the modified mlv_dump (compile from manual_leins_info branch). Just to understand if it's only related to XMP or if ML is't saving datas in memory.

Note: mlv_dump was modified in the pending PR to match metadata format, so I don't remember if was working good in old version (current build on ML site).

If you'd need any help with testing, let me know - i have 650D+6D and couple of non-chipped and chipped adapters to play with.

Yes, absolutely! I have just a couple of cheap adapter on which I based when writing the lens recognition of the script.

They show as "1-65535mm" for name and focal length.
Already refactored to a separate function just in case to make it easy to read and adapt.

Code: [Select]

function is_manual_lens()
  -- Adapter with no AF Chip -> ID = 0
  -- Adapter with AF confirm Chip -> name and focal length "1-65535mm"
  if (lens.id == 0 or lens.name == "1-65535mm" or lens.name == "(no lens)") then
    return true
  else
    return false
  end
end

Can you check if also your adapter works like this? I imagine should be the same even if different makers, except for Dandelion adapters.

Also need more feedbacks about Raw video recording stability if you are daily video user.

[zLOST]

My point of splitting lens name to brand+type was to keep the lens.name shorter than 32 chars. We can always concatenate these two fields in UI and when exporting the xmp, where it should not matter and should be used by LR/DT/anything to detect the used lens.

Plus i'd like to add few more exif fields to the xmp output (i wrote a simple bash script to "label" my pics - https://www.zlej.net/tmp/exif.txt and it would be great to see this done automatically by ML as i tend to forget what i had attached two weeks ago when shooting some pics..)

Is there any way to scroll within the debug console? Scripts may complain about something usefull, but when the screen gets wiped out due to the assert crash, i'm lost

I thought, that when we "fake" the lens.[name|aperture|whatever], the canon FW will use those values when saving the exif data in cr2 or jpeg, but apparently it does not bother. Plus they use Exif.Photo and Exif.Canon* sections for similar data

[aprofiti]

My point of splitting lens name to brand+type was to keep the lens.name shorter than 32 chars. We can always concatenate these two fields in UI and when exporting the xmp, where it should not matter and should be used by LR/DT/anything to detect the used lens.

As used to work on code related to ELNS block, I initially believed you were referring to the actual file size saved on card... then understood after posting that you were talking abound lens name length showed in UI.

So yes it could be done. Slightly better could be to check if a lens name is to big to be showed in lens selection menu and truncate just the maker name, instead just print the whole name (make + model).

Noticed you modified the XMP lib's tag related to selected lens from "aux:" to "exif:".
Made some research and didn't fully come to a conclusion on which is better to use, as Canon stick with old tag system and doesn't come to troubles with tradition editing software.

Also noticed that "aux:Lens" just put the focal length range with my lens....
What are the advantage if we replace with the "exif:lensMake"/"exifEX:lensMake" and "exif:LensModel"/"exifEX:LensModel" attributes?

Plus i'd like to add few more exif fields to the xmp output (i wrote a simple bash script to "label" my pics - https://www.zlej.net/tmp/exif.txt and it would be great to see this done automatically by ML as i tend to forget what i had attached two weeks ago when shooting some pics..)

I looked at ExifTool's XMP tag list and XMP specification and found under tags used by Photoshop the string: "xmp:Label" which looks like it could be used for your purpose.

Reading an article from FastRawViewer his content doesn't looks forced to a predefined set; instead as being a simply string I tried to manual modify a .xmp file and get retrieved by Adobe Bridge as a new type of label.

Alternatively we could add a new custom tag to the xmp library (ex. under ML namespace), but should't be necessary.

To add a new xmp attribute to the script, follow these steps:
1. Add a new predefined properties to xmp.lua library (if you want the code to looks a bit cleaner, otherwise you could skip this step and adapt the second one):
"xmp.label = { name = "xmp:Label" }"

Code: [Select]

-- predefined xmp properties
xmp.lens_name = { name = "aux:Lens" }
xmp.lens_serial = { name = "aux:SerialNumber" }
xmp.focal_length = { name = "exif:FocalLength", format = "%d/1"}
xmp.aperture = { name = "exif:FNumber", format = "%d/10" }

2. Tell to the lens.lua script to save the new tag when taking a picture:
"xmp:add_property(xmp.label, function() return lenses[selector_instance.index].label end)"

Code: [Select]

-- Property to be written in .xmp file
xmp:add_property(xmp.lens_name, function() return lens.name end)
xmp:add_property(xmp.focal_length, function() return lens.focal_length end)
xmp:add_property(xmp.aperture, function() return (lens.manual_aperture * 10) end)
xmp:add_property(xmp.lens_serial, function() return lens.serial end)
Note: Here I'm retrieving value from a local variable of lens.lua script, instead of using something like "lens.label" as it is an API not available at the moment.
You need to add getter and setter functions in modules/lua/lua_lens.c to make API available to use and save value somewhere (ex. ML lens_info structure in /src/lens.h), then recompile ML.

Also I'm not sure if value get updated when switching lens by using this method and if you get an error for missing field.
If one of these happen try to add check for "label" inside the for cycle:

Code: [Select]

function update_lens()
    -- Reset lens_info structure to get correct values in Lens Info Menu and Metadata
    reset_lens_values()
    -- Update attribute from selected lens
    for k,v in pairs(lenses[selector_instance.index]) do
        -- avoid not found exception with the key "label"
        if k ~= "label" then
         lens[k] = v
        else
         -- update label value   
         xmp:add_property(xmp.label, function() return lenses[selector_instance.index].label end) -- may not be necessary
       end
    end
    -- Allow to write sidecar
    xmp:start()
    -- Update flag
    lensSelected = true
    -- Allow to write values in Lens Info Menu
    lens.exists = true
end

3. Add the attribute "label" to the corresponding lenses:

Code: [Select]

-- From your lenses Table version

lenses = -- {{{
{
    { make = "Carl Zeiss Jena DDR",     name = "Tessar 50mm f/2.8",             focal_length = 50,      manual_aperture = 2.8,  serial = "8333263",
 label = "tessar50" },
    { make = "Industar",                name = "61 L/Z (MC) 50mm f/2.8",        focal_length = 50,      manual_aperture = 2.8,  serial = "8707286",
 label = "industar61" },
    { make = "Meyer-Optik Gorlitz",     name = "Telemegor 180mm f/5.5",         focal_length = 180,     manual_aperture = 5.5,  serial = "1728275",
 label = "telemegor180" },

....

} -- }}}

Looks like you already dive deep in the script structure, so you should not have particular problems to add the new tag

[zLOST]

So, i wrote a simple dumper script and tested all the adapters i found with it:

Code: [Select]

-- Lens properties dump

require("config")
require("logger")

local log = logger("adapter.log")

for k,v in pairs(lens) do
        if k ~= "focus" then
                if type(v) == "string" then
                        log:write("lens."..k.."="..v)
                elseif type(v) == "boolean" or type(v) == "number" then
                        log:write("lens."..k.."()="..tostring(v))
                elseif type(v) == "function" then
                        r = v()
                        if type(r) == "string" then
                                log:write("lens."..k.."()="..r)
                        elseif type(r) == "boolean" or type(r) == "number" then
                                log:write("lens."..k.."()="..tostring(r))
                        end
                elseif type(v) == "table" then
                        for a,b in pairs(v) do
                                log:write("lens. "..k.."."..a.."="..b)
                        end
                end
        end
end

There was a little hope in finding some way to differentiate between each one of them (some should be "programmable" and of different versions as i bought each separately in past few years).
Unfortunately with no luck:

Code: [Select]

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:51:36 ----> no chip at all (Samyang 85/1.4)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.name=
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.focal_length()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.hyperfocal()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.dof_near()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.dof_far()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:52:55 ----> with AF chip (M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:56 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:53:16 ----> with AF chip (M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:53:42 ----> Sigma 50/2.8 DG Macro (AF turned on)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.name=EF50mm f/2.5 Compact Macro
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.hyperfocal()=30888
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.dof_near()=-340372800
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.af()=true
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.af_mode()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:45 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:54:29 ----> with AF chip (praktica B -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.focal_length()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.hyperfocal()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.dof_near()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.dof_far()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:54:53 ----> with AF chip (another M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:53 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:53 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.autofocus()=false

lens.focus was crashing and i was too lazy to search for a way to dump it correctly, so it's omitted from the output

@aprofiti] my "deep dive" is more of an endless circle of "hack->save->start the camera->facepalm->hack..."

So far i've "tweaked" a bit the xmp.lua, logger.lua and lens.lua.

------

Selecting a lens from the list results in this to be logged:

Code: [Select]

===============================================================================
ML/SCRIPTS/LENS.LUA - 2019-1-19 00:20:22
===============================================================================

ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - selector_instance:8
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: focal_length: 29
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: manual_aperture: 2.8
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: name: auto 29mm f/2.8
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: serial: 6308110
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: make: Pentacon

Lens.* dumper shows:

Code: [Select]

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-19 00:33:20
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.name=auto 29mm f/2.8
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.focal_length()=29
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.hyperfocal()=30888
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.dof_near()=-340372800
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.autofocus()=false

And xmp.lua with write() modified to only log what it should be writing to xmp file logs:

Code: [Select]

===============================================================================
ML/SCRIPTS/LIB/xmp.lua - 2019-1-19 00:33:06
===============================================================================

ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - Writing xmp file...
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:LensSpecification=0/10 0/10 0/1 0/1
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:LensSerialNumber=0
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:FNumber=14/10
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:MaxApertureValue=F1.4
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:MinApertureValue=Fnil
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:38 - key: exif:LensModel=Pentacon auto 29mm f/2.8
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:38 - key: exif:LensMake=Pentacon
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:38 - key: exif:FocalLength=29/1

The modified write() looks like this:

Code: [Select]

--[[---------------------------------------------------------------------------
Writes the xmp metdata to file
@param filename optional - the filename to write to, leave blank to use the sidecar filename for the most recently taken photo
@function write
]]
function xmp:write(filename) -- {{{
--      local f = assert(io.open(filename or self.get_sidecar_filename(), "w+"))
--      assert(f:write(self.header, "\n"))

        xmplog:write("Writing xmp file...")
        for k,v in pairs(self.properties) do
                if type(v) == "function" then
                        xmplog:write("key: "..k.."="..v())
--                      assert(f:write(string.format(self.property_format, k, v(), k), "\n"))
                elseif type(v) == "boolean" or type(v) == "number" then
                        xmplog:write("key2: "..k.."="..tostring(v))
--                      assert(f:write(string.format(self.property_format, k, tostring(v), k), "\n"))
                else
                        xmplog:write("key2: "..k.."="..v)
                end
        end
        --f:write(string.format(self.template, str))
--      assert(f:write(self.footer))
--      assert(f:close())
end --- }}}

Unfortunately xmp.write() crashes when i try to let it write the xmp file and the error console is immediately wiped out by subsequent assert() crash..

[zLOST]

I've just found what was wrong when setting params for my Samyang - lens.serial is expected to be integer, but this lens has string..




(two hours and numerous edits of lens.lua+xmp.lua later)
I'm finally able to write the xmp file. Unfortunately only to the root of SD card - for some reason it still crashes when i try to use the dryos path, even when it looks fine.
Could someone with LR/PS/anything else verify that this format is acceptable by software other than darktable&gimp?

Code: [Select]

<?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="MagicLantern">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230">
   <exif:LensModel>Samyang 85mm f/1.4 AS IF UMC</exif:LensModel>
   <exif:FocalLength>85/1</exif:FocalLength>
   <exif:MaxApertureValue>F1.4</exif:MaxApertureValue>
   <exif:FNumber>14/10</exif:FNumber>
   <exif:LensSpecification>10/10 655350/10 0/1 0/1</exif:LensSpecification>
   <exif:LensSerialNumber>A217D0264</exif:LensSerialNumber>
   <exif:LensMake>Samyang</exif:LensMake>
   <exif:MinApertureValue>F22</exif:MinApertureValue>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

LensSpecification is still showing wrong string (it should be "min_focal max_focal Fmax@min_focal Fmax@max_focal"), but i'll polish that one out, plus it's absolutely unimportant for anyone but me.

As a workaround for the serial number i've renamed "serial" in lenses{} to serialN, which is handled as string() by default, so i'm able to access it in xmp() and save. I suppose, that the serial==number() limitation is hardcoded somewhere in ML sources, right?

I've added four more params to lenses{}:
1] max_aperture and min_aperture, which are used to trim down the list of available Fnumbers plus they'll end up in the <exif:M[in|ax]ApertureValue>F22</exif:M[in|ax]ApertureValue> elements

2] FminL, FmaxL - apertures on wide/long ends of zoom lenses if they differ. Used in LensSpecification field.

Btw: i've checked exif data of my pics and the Exif.Photo.LensModel is really just a model without the lens brand (pics taken with trioplan were labeled by my old bash script):

Code: [Select]

exiv2 -g Exif.Photo.LensModel *.CR2|sed -e "s/^.*Ascii *.*  //"|sort|uniq -c|sed -e "s/^ *//" -e "s/ /x /"
2x EF17-40mm f/4L USM
435x EF-S10-18mm f/4.5-5.6 IS STM
102x Meyer-Optik Görlitz Trioplan 50mm f/2.9
7x MP-E65mm f/2.8 1-5x Macro Photo


[Lars Steenhoff]

Interesting finds!

I can test in adobe camera raw if you can share a sample of the footage with the lens data you captured.

[zLOST]

cr2+corresponding xml can be downloaded at https://www.zlej.net/sample.zip

It still seems to me, that during/after the exposure some lens{} parameters are being overwritten with defaults either from camera or the chipped adapter..

Interesting finds!

I can test in adobe camera raw if you can share a sample of the footage with the lens data you captured.

[Lars Steenhoff]

This is what I get in adobe bridge, the zoom lens is one I have never seen

[zLOST]

Those are the default values from the adapter/camera, which are stored in exif. So the xmp file was not processed at all.

Can you rename the xmp file to test.xmp and try it again, please?

[Lars Steenhoff]

I just did that and no difference in adobe

when I open the file on apple preview and press info I can get Canon EF 35-80 f4-5.6

but this also is shown without the side car file present.

[aprofiti]

So, i wrote a simple dumper script and tested all the adapters i found with it:

Code: [Select]

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:51:36 ----> no chip at all (Samyang 85/1.4)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.name=
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.focal_length()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.hyperfocal()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.dof_near()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.dof_far()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:52:55 ----> with AF chip (M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:56 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:53:16 ----> with AF chip (M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:53:42 ----> Sigma 50/2.8 DG Macro (AF turned on)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.name=EF50mm f/2.5 Compact Macro
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.hyperfocal()=30888
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.dof_near()=-340372800
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.af()=true
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.af_mode()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:45 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:54:29 ----> with AF chip (praktica B -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.focal_length()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.hyperfocal()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.dof_near()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.dof_far()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:54:53 ----> with AF chip (another M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:53 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:53 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.autofocus()=false


Thank You for testing more adapters. Here is one of mine (second one has a broken contact so no chip doesn't work anymore, anyway basically info):

Code: [Select]

===============================================================================
ML/SCRIPTS/LENSDUMP.LUA - 2019-1-19 20:44:31  => PK -> EOS  and similiar to C/Y -> EOS
===============================================================================

lens.name=1-65535mm
lens.focal_length()=50
lens.focus_distance()=0
lens.hyperfocal()=94084
lens.dof_near()=-1735480800
lens.dof_far()=1000000lens.af()=false
lens.af_mode()=771
lens.autofocus()=false
It's the same with mine, so the method used until now is ok

I've just found what was wrong when setting params for my Samyang - lens.serial is expected to be integer, but this lens has string..

As a workaround for the serial number i've renamed "serial" in lenses{} to serialN, which is handled as string() by default, so i'm able to access it in xmp() and save. I suppose, that the serial==number() limitation is hardcoded somewhere in ML sources, right?

That's something @g3gg0 may review for us, as he worked here to adapt serial number conversion.
Don't know what is best to do here. Better to to wait for him to read this post.

I'm finally able to write the xmp file. Unfortunately only to the root of SD card - for some reason it still crashes when i try to use the dryos path, even when it looks fine.

This is how I modified xml.lua to solve the error due to the refactor:

Code: [Select]

function xmp.get_sidecar_filename()
    return dryos.shooting_card.dcim_dir.path ..  dryos.image_prefix .. string.format("%04d", dryos.shooting_card.file_number) .. ".XMP"
end


I'm finally able to write the xmp file. Unfortunately only to the root of SD card - for some reason it still crashes when i try to use the dryos path, even when it looks fine.
Could someone with LR/PS/anything else verify that this format is acceptable by software other than darktable&gimp?

Adobe Bridge/Photoshop will show correct info for lens make,model and focal length
Serial number as 630312148, max aperture value as f/1.0

I've added four more params to lenses{}:
1] max_aperture and min_aperture, which are used to trim down the list of available Fnumbers plus they'll end up in the <exif:M[in|ax]ApertureValue>F22</exif:M[in|ax]ApertureValue> elements

2] FminL, FmaxL - apertures on wide/long ends of zoom lenses if they differ. Used in LensSpecification field.

Btw: i've checked exif data of my pics and the Exif.Photo.LensModel is really just a model without the lens brand (pics taken with trioplan were labeled by my old bash script)

Thinked before about that, but then added the "f_values" attribute to lenses, which enable the possibility to list only correct aperture for the selected lens, so I decided to don't introduce a redundant attributes.

Was thinking about to retrieve first and last values from the Manual Lens menu Aperture choice (which is updated from f_values if available otherwise use Fnumbers predefined list), to use as aperture_min and aperture_max.

I haven't exactly understood what FminL, FmaxL are for.

You remembered me to update xmp file to have all the values stored in ELNS block. So currently i have:

Code: [Select]

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FNumber="4/10"
    exif:MinApertureValue="22/10"
    aux:Lens="My Lens"
    xmp:Label="test"
    exif:FocalLength="100/1"
    aux:SerialNumber="0"
    exif:MaxApertureValue="1.4/10">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
Mainly missing Focal Length range which is already handled by the lens.lua script, so I have only to add property to xmp

I'm trying to implement an idea to the ui.lua, which could allow to read all the lens name even if too long.
Not completely sure about splitting lens name into "make" and "model".

@a1ex When I try to backing out b0a2f95 using Sourcetree, it end up in duplicating the branch... so tried to do it manually and verified "Small-malloc test" and raw recording (this is not working good in last build due to the missing commit; camera freezes) with are working good (not sure about the counter next to the coloured camera icon while recording, it print some garbage to me instead of a recording timer).

I have in local all the changes. Maybe I can try upload a build in next days if I have time to finish experimenting with it

[zLOST]

I did couple more facepalms in past few hours and it seems, that io.open() crashes when there is a CR2 as a part of the filename. I have no idea why - the debug console gets wiped out by assert crash before i can notice anything

So, with your code, i am able to write the xmp file to dcim...

Code: [Select]

function xmp.get_sidecar_filename() -- {{{
        local path = dryos.shooting_card.dcim_dir.path ..  dryos.image_prefix .. string.format("%04d", dryos.shooting_card.file_number) .. ".XMP"
        xmplog:write("Path:"..path)
        return path
end -- }}}

I'm using this page as a reference for exif data and the LensSerialNumber should be just a plain ascii string:
http://www.exiv2.org/tags.html

My sidecar now looks like this:

Code: [Select]

<?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="MagicLantern">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
   xmlns:exif="http://ns.adobe.com/exif/1.0/"
   xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
   exif:ExifVersion="0230"
   aux:Lens="Helios 44-2 58mm f/2"
   exif:LensSerialNumber="7765411"
   exif:LensSpecification="1/1 65535/1 0/1 0/1"
   exif:LensMake="Helios"
   exif:MinApertureValue="97/1"
   exif:LensModel="Helios 44-2 58mm f/2"
   exif:FNumber="14/10"
   exif:MaxApertureValue="97/1"
   exif:FocalLength="58/1">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Unfortunately darktable completely ignores the provided lens name and shows some default Canon set lenses.

[Lars Steenhoff]

Maybe try with a more common lens name to see if its picket up. 

This one is normally pocket up by adobe and auto sets the lens corrections.

lens.name = "Zeiss Distagon T* 2/25 ZF.2"
lens.focal_length = 25
lens.aperture = 20

[Lars Steenhoff]

@dfort maybe you know something about this? you know most things nowadays

[zLOST]

It seems, that guys grom darktable insist on using originalfilename.suffix.xmp naming for xmp sidecar files (there is a bug opened eight years ago about this - https://redmine.darktable.org/issues/8403)

And canon/ml/lua refuses to write a file with two suffixes (at least in my case).

Code: [Select]

-- test

f = io.open("test.XMP", "a")
f:write("bleble")
f:close()
The above works fine, but when i try io.open("test.CR2.XMP") to make darktable happy, the io operation fails and subsequent f:write() crashes with "attempt to index a nil value (global 'f')".

and dryos.rename("TEST.XMP", "TEST.CR2.XMP") does not work either.

looks like i gotta slam my head against the wall a little more

[Lars Steenhoff]

you can always batch rename them on the computer right ?
As long as the fie has the info you need

[zLOST]

you can always batch rename them on the computer right ?
As long as the fie has the info you need

Of course But the more stuff can be done automatically by the camera, the better for me.. I'll give it a shot with rawtherapee, but i haven't used it before, so i may fail (again).

[zLOST]

Of course But the more stuff can be done automatically by the camera, the better for me.. I'll give it a shot with rawtherapee, but i haven't used it before, so i may fail (again).

Well, rawtherapee seems to ignore the xmp file altogether And the saga continues..

Btw: is there any chance to get rid of those memory-related crashes after each shot?

[aprofiti]

Btw: is there any chance to get rid of those memory-related crashes after each shot?

Maybe backing out b0a2f95 (i.e. re-applying 065ceae) could help.

You can find the build I have in local at this download page (for 5D3,5D2,7D,6D,50D)
Should also avoid camera freezes while recording raw videos.

I was trying to make lens serial number as a string in lua, but I get a crash log with the modified lua_lens.c:

Code: [Select]

/***
 Lens functions

 @author Magic Lantern Team
 @copyright 2014
 @license GPL
 @module lens
 */

#include <dryos.h>
#include <string.h>
#include <lens.h>
#include <focus.h>
#include <module.h>
#include "lua_common.h"

static int luaCB_lens_index(lua_State * L)
{
    LUA_PARAM_STRING_OPTIONAL(key, 2, "");
    /// Get/Set the name of the lens (reported by the lens)
    // @tfield string name
    if(!strcmp(key, "name")) lua_pushstring(L, lens_info.name);
    /// Get the lens id
    // @tfield int id readonly
    else if(!strcmp(key, "id")) lua_pushinteger(L, lens_info.lens_id);
    /// Get the serial number of the lens
    // @tfield string serial
    else if(!strcmp(key, "serial")) lua_pushstring(L, lens_info.lens_serial);
    /// Get/Set the focal length of the lens (in mm)
    // @tfield int focal_length
    else if(!strcmp(key, "focal_length")) lua_pushinteger(L, lens_info.focal_len);
    /// Get/Set the minimum focal length of the lens (in mm)
    // @tfield int focal_min
    else if(!strcmp(key, "focal_min")) lua_pushinteger(L, lens_info.lens_focal_min);
    /// Get/Set the maximum focal length of the lens (in mm)
    // @tfield int focal_max
    else if(!strcmp(key, "focal_max")) lua_pushinteger(L, lens_info.lens_focal_max);
    /// Get the current focus distance (in mm). Only updated in LiveView.
    // @tfield int focus_distance readonly
    else if(!strcmp(key, "focus_distance")) lua_pushinteger(L, lens_info.focus_dist * 10);
    /// Get the raw relative focus motor position, in steps.
    /// This counter is 0 at camera startup, its range depend on the lens,
    /// and is updated only when the focus motor moves. It will lose track
    /// of the lens position during manual focus, unless you use a focus-by-wire lens.
    /// Details: [www.magiclantern.fm/forum/index.php?topic=4997](http://www.magiclantern.fm/forum/index.php?topic=4997).
    // @tfield int focus_pos readonly
    else if(!strcmp(key, "focus_pos")) lua_pushinteger(L, lens_info.focus_pos);
    /// Get the hyperfocal distance of the lens (in mm). Only updated in LiveView.
    ///
    /// Computed from focal length, focus distance and aperture, see Focus -> DOF Settings menu for options.
    // @tfield int hyperfocal readonly
    else if(!strcmp(key, "hyperfocal")) lua_pushinteger(L, lens_info.hyperfocal);
    /// Get the distance to the DOF near (in mm). Only updated in LiveView.
    ///
    /// Computed from focal length, focus distance and aperture, see Focus -> DOF Settings menu for options.
    // @tfield int dof_near readonly
    else if(!strcmp(key, "dof_near")) lua_pushinteger(L, lens_info.dof_near);
    /// Get the distance to the DOF far (in mm). Only updated in LiveView.
    ///
    /// Computed from focal length, focus distance and aperture, see Focus -> DOF Settings menu for options.
    // @tfield int dof_far readonly
    else if(!strcmp(key, "dof_far")) lua_pushinteger(L, lens_info.dof_far);
    /// Get whether or not auto focus is enabled.
    // @tfield bool af readonly
    else if(!strcmp(key, "af")) lua_pushboolean(L, !is_manual_focus());
    /// Get the current auto focus mode (may be model-specific, see PROP\_AF\_MODE in property.h).
    // @tfield int af_mode readonly
    else if(!strcmp(key, "af_mode")) lua_pushinteger(L, af_mode);
    /// Get whether the lens is currently autofocusing.
    ///
    /// This does not include manual lens movements from lens.focus or ML follow focus -
    /// only movements from Canon autofocus triggered by half-shutter / AF-ON / * button.
    /// It is updated several ms (sometimes hundreds of ms) after the half-shutter event.
    ///
    /// On cameras with continuous autofocus, the return value is unknown - please report.
    ///
    /// Known not to work on EOS M.
    // @tfield bool autofocusing readonly
    else if(!strcmp(key, "autofocusing")) lua_pushboolean(L, lv_focus_status == 3);
    /// Use to manually set the lens aperture value for non-chipped lenses (for metadata purposes)
    // @tfield bool manual_aperture
    else if(!strcmp(key, "manual_aperture")) lua_pushnumber(L, lens_info.aperture / 10.0);
    /// Use to manually set the len's minumum aperture value for non-chipped lenses (for metadata purposes)
    // @tfield bool aperture_min
    else if(!strcmp(key, "aperture_min")) lua_pushnumber(L, RAW2VALUE(aperture, lens_info.raw_aperture_min) / 10.0);
    /// Use to manually set the len's maximum aperture value for non-chipped lenses (for metadata purposes)
    // @tfield bool aperture_max
    else if(!strcmp(key, "aperture_max")) lua_pushnumber(L, RAW2VALUE(aperture, lens_info.raw_aperture_max) / 10.0);
    /// Get if the lens chipped
    // @tfield bool is_chipped readonly
    else if(!strcmp(key, "is_chipped")) lua_pushboolean(L, lens_info.lens_exists);
    else lua_rawget(L, 1);
    return 1;
}

static int luaCB_lens_newindex(lua_State * L)
{
    LUA_PARAM_STRING_OPTIONAL(key, 2, "");
    if(!strcmp(key, "name"))
    {
        LUA_PARAM_STRING(value, 3);
        strncpy(lens_info.name, value, sizeof(lens_info.name)-1);
    }
    else if(!strcmp(key, "exists"))
    {
        LUA_PARAM_BOOL(value, 3);
        lens_info.lens_exists = value;
    }
    else if(!strcmp(key, "focal_length"))
    {
        LUA_PARAM_INT(value, 3);
        lens_info.focal_len = value;
    }
    else if(!strcmp(key, "focal_min"))
    {
        LUA_PARAM_INT(value, 3);
        lens_info.lens_focal_min = value;
    }
    else if(!strcmp(key, "focal_max"))
    {
        LUA_PARAM_INT(value, 3);
        lens_info.lens_focal_max = value;
    }
    else if(!strcmp(key, "manual_aperture"))
    {
        LUA_PARAM_NUMBER(value, 3);
        lens_info.aperture = (int)(value * 10);
        lens_info.raw_aperture = VALUE2RAW(aperture, lens_info.aperture);
    }
    else if(!strcmp(key, "aperture_min"))
    {
        LUA_PARAM_NUMBER(value, 3);
        int tmp = (int)(value * 10);
        lens_info.raw_aperture_min = VALUE2RAW(aperture, tmp);
    }
    else if(!strcmp(key, "aperture_max"))
    {
        LUA_PARAM_NUMBER(value, 3);
        int tmp = (int)(value * 10);
        lens_info.raw_aperture_max = VALUE2RAW(aperture, tmp);
    }
    else if(!strcmp(key, "serial"))
    {
        LUA_PARAM_STRING(value, 3);
        strncpy((uint64_t)lens_info.lens_serial, value, sizeof(lens_info.lens_serial));
    }
    else if(!strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af") || !strcmp(key, "is_chipped"))
    {
        return luaL_error(L, "'%s' is readonly!", key);
    }
    else
    {
        lua_rawset(L, 1);
    }
    return 0;
}

/***
 Moves the focus motor a specified number of steps.

 Only works in LiveView.
 @tparam int num_steps How many steps to move the focus motor (signed).
 @tparam[opt=2] int step_size Allowed values: 1, 2 or 3.

 Step 1 may give finer movements, but may get stuck or may be very slow on some lenses.
 @tparam[opt=true] bool wait Wait until each focus command finishes, before queueing others.

 wait=false may give smoother movements, but may no longer return accurate status for each command,
 and is known to **crash** on some cameras. The exact behavior may be camera- or lens-dependent.

 __Do not disable it without a good reason!__

 @tparam[opt] int delay Delay between focus commands (ms)

 With wait=true, the delay is after each focus command is executed (as reported by Canon firmware).

 With wait=false, the delay is after each focus command is started (without waiting for it to finish).

 (_default_ 0 if wait=true, 30ms if wait=false)


 @treturn bool whether the operation was successful or not.
 @function focus
 */
static int luaCB_lens_focus(lua_State * L)
{
    LUA_PARAM_INT(num_steps, 1);
    LUA_PARAM_INT_OPTIONAL(step_size, 2, 2);
    LUA_PARAM_BOOL_OPTIONAL(wait, 3, true);
    LUA_PARAM_INT_OPTIONAL(delay, 4, wait ? 0 : 30);

    if (!lv) return luaL_error(L, "lens.focus() only works in LiveView.");
    if (is_manual_focus()) return luaL_error(L, "lens.focus() requires autofocus enabled.");
    if (is_continuous_af()) return luaL_error(L, "lens.focus() requires %s AF disabled.", is_movie_mode() ? "movie servo" : "continuous");

    lua_pushboolean(L, lens_focus(num_steps, step_size, wait, delay));

    return 1;
}

static int wait_focus_status(int timeout, int value1, int value2)
{
    int t0 = get_ms_clock();

    while (get_ms_clock() - t0 < timeout)
    {
        msleep(10);

        if (lv_focus_status == value1 || lv_focus_status == value2)
        {
            return 1;
        }
    }
    return 0;
}

/***
 Performs autofocus, similar to half-shutter press.

 Works in both LiveView and in plain photo mode.
 @treturn bool whether the operation was successful or not.
 @function autofocus
 */
static int luaCB_lens_autofocus(lua_State * L)
{
    int focus_command_sent = 0;

    if (is_manual_focus())
    {
        goto error;
    }

    /* these models won't AF with half-shutter in LiveView */
    int back_btn_af_lv = lv && (
        is_camera("5D2", "*") ||
        is_camera("50D", "*") ||
        is_camera("500D", "*")
    );

    if (back_btn_af_lv)
    {
        /* FIXME: this method fails on 60D, why? */
        int af_request = 1;
        prop_request_change(PROP_REMOTE_AFSTART_BUTTON, &af_request, 4);
    }
    else
    {
        lens_setup_af(AF_ENABLE);
        module_send_keypress(MODULE_KEY_PRESS_HALFSHUTTER);
    }

    focus_command_sent = 1;

    if (!lv)
    {
        for (int i = 0; i < 20; i++)
        {
            msleep(100);

            /* FIXME: this may fail on recent models where trap focus is not working */
            if (get_focus_confirmation())
            {
                goto success;
            }
        }

        goto error;
    }

    /* 3 = focusing, 1 = idle (most models), 2 = idle (100D) */
    if (wait_focus_status(1000, 3, 3))
    {
        if (wait_focus_status(5000, 1, 2))
        {
            goto success;
        }
        else
        {
            /* timeout */
            printf("[%s] focus status: %d (expected 1 or 2)\n", lua_get_script_filename(L), lv_focus_status);
            goto error;
        }
    }
    printf("[%s] focus status: %d (expected 3)\n", lua_get_script_filename(L), lv_focus_status);

error:
    lua_pushboolean(L, false);
    goto cleanup;

success:
    lua_pushboolean(L, true);
    goto cleanup;

cleanup:
    if (focus_command_sent)
    {
        if (back_btn_af_lv)
        {
            int af_request = 0;
            prop_request_change(PROP_REMOTE_AFSTART_BUTTON, &af_request, 4);
        }
        else
        {
            module_send_keypress(MODULE_KEY_UNPRESS_HALFSHUTTER);
            lens_cleanup_af();
        }
    }
    return 1;
}

static const char * lua_lens_fields[] =
{
    "name",
    "focal_length",
    "focus_distance",
    "hyperfocal",
    "dof_near",
    "dof_far",
    "af",
    "af_mode",
    NULL
};

static const luaL_Reg lenslib[] =
{
    { "focus",      luaCB_lens_focus },
    { "autofocus",  luaCB_lens_autofocus },
    { NULL, NULL }
};

LUA_LIB(lens)
Lens.lua

Code: [Select]

lenses =
{
...

    { name = "My Zoom Lens", focal_length = 105, manual_aperture = 4, focal_min = 70, focal_max = 200, serial = "123456789" },

...
}
function reset_lens_values()
...

  lens.serial = ""

...
end

The log I get:

Code: [Select]

[43909204] lua_script_task: NULL PTR (34333231,e1a00000)
pc=     180 lr= a21fe80 stack=1ae5b0+0x10000
entry=bc5ff0(984464)
e1a00000 38373635 e59ff014 e59ff014
e59ff014 e1a00000 e59ff010 e59ff010

Magic Lantern version : Nightly.2019Jan21.50D109
Mercurial changeset   : b1818aa3ae43+ (manual_lens_info) tip
Built on 2019-01-21 12:37:46 UTC by alex@PCAlessandro.
Free Memory  : 227K + 2617K

@a1ex how can I interpret these crash logs to get more infos?

[zLOST]

You can find the build I have in local at this download page (for 5D3,5D2,7D,6D,50D)
Should also avoid camera freezes while recording raw videos.

Awesome, thank you. I'll give it a shot when i'm back home.

In the meantime i've moved a bit forward with getting the lens model accepted by darktable and found some strange behavior of other exif metadata in dt->gimp process.

[a1ex]

@a1ex how can I interpret these crash logs to get more infos?

The above shows a null pointer (i.e. lua_script_task wrote something to address 0). ML catches this issue a relatively long time after it happens (i.e. at the next task context switch). The mem_prot module attempts to pinpoint it, but... in this case, you are trying to debug dynamically allocated code (i.e. module code), so even if you know the exact PC, it won't help much. This kind of debugging is hard; I'd rather stick to printf.

For the general question, look here. It is possible to trace crashes that happen in Canon code, or in core ML code, but modules are not loaded at the same address every time. It may be possible to dump the linker information right before unloading TCC, but I didn't explore this route yet.

[zLOST]

Thanks for the non-crashing build, Alessandro. It's perfect!




Unfortunately the focal length and aperture values come from exif data embeded in cr2, so it shows the defaul values from the adapter. i haven't tried the non-chipped version yet.

At the moment i'm using this xmp output (the exifEX:LensModel is the right field for DarkTable...):

Code: [Select]

<?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="MagicLantern">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
   xmlns:exif="http://ns.adobe.com/exif/1.0/"
   xmlns:exifEX="http://cipa.jp/exif/1.0/"
   xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
   exif:ExifVersion="0230"
   exif:FNumber="20/10"
   exif:ApertureValue="20/10"
   exif:LensModel="44-2 58mm f/2"
   exif:Lens="Helios 44-2 58mm f/2"
   exif:MaxApertureValue="2./1"
   exif:LensSerialNumber="7765411"
   aux:Lens="Helios 44-2 58mm f/2"
   exif:LensSpecification="1/1 65535/1 2./1 2./1"
   exif:FocalLength="580/10"
   exif:LensMake="Helios"
   exif:MinApertureValue="16/1"
   exifEX:LensModel="Helios 44-2 58mm f/2">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Funny thing is, that when i use gimp to open cr2, it calls darktable, which then passes the processed photo back to gimp and save it as jpg, this jpeg has correct focal length and aperture in Exif.Photo. Unfortunately the Exif.Canon* fields contain 50/1.4 as reported by the adapter.

Code: [Select]

[zlost@nb-zimmermann2 ~]$ exiv2 -g Exif /media/zlost/EOS_6D/DCIM/101CANON/test.jpg |grep -i -e focal -e FNumber -e aperture|sort
Exif.CanonCs.DisplayAperture                 Short       1  0
Exif.CanonCs.MaxAperture                     Short       1  F1.4
Exif.CanonCs.MinAperture                     Short       1  F1.4
Exif.Canon.FocalLength                       Short       4  50.0 mm
Exif.CanonSi.ApertureValue                   Short       1  F1.4
Exif.CanonSi.TargetAperture                  Short       1  F1.4
Exif.Photo.ApertureValue                     Rational    1  F2
Exif.Photo.FNumber                           Rational    1  F2
Exif.Photo.FocalLength                       Rational    1  58.0 mm
Exif.Photo.FocalPlaneResolutionUnit          Short       1  inch
Exif.Photo.FocalPlaneXResolution             Rational    1  3810.58
Exif.Photo.FocalPlaneYResolution             Rational    1  3815.9
[zlost@nb-zimmermann2 ~]$

To ease my struggle with finding the right field to use for lens name, which will be accepted by darktable, i've modded xmp.lua, so the properties name can be a string and table/array as well to generate three lines with a single call to add_property() (feel free to call me lazy )

Code: [Select]

xmp.lens_make           = { name = "exif:LensMake",             format = "%s" } -- Exif.Photo.LensMake  Ascii  This tag records the lens manufactor as an ASCII string.
xmp.lens_name           = { name = "exif:LensModel",            format = "%s" } -- Exif.Photo.LensModel  Ascii  This tag records the lens's model name and model number as an ASCII string
xmp.lens                = { name = {"aux:Lens","exif:Lens","exifEX:LensModel"},         format = "%s" }
xmp.lens_serial         = { name = "exif:LensSerialNumber",     format = "%s" } -- Exif.Photo.LensSerialNumber  Ascii  This tag records the serial number of the interchangeable lens that was used in photography as an ASCII string.
xmp.focal_length        = { name = "exif:FocalLength",          format = "%d/10" }
xmp.FNumber             = { name = "exif:FNumber",              format = "%d/10" }
xmp.apertureValue       = { name = "exif:ApertureValue",        format = "%d/10" } -- Exif.Photo.ApertureValue  Rational  The lens aperture. The unit is the APEX value.
xmp.MinApertureValue    = { name = "exif:MinApertureValue",     format = "%s/1" }
xmp.MaxApertureValue    = { name = "exif:MaxApertureValue",     format = "%s/1" } -- Exif.Photo.MaxApertureValue  Rational  The smallest F number of the lens. The unit is the APEX value. Ordinarily it is given in the range of 00.00 to 99.99, but it is not limited to this range.
xmp.lens_specification  = { name = "exif:LensSpecification",    format = "%s" } -- Exif.Photo.LensSpecification  Rational  This tag notes minimum focal length, maximum focal length, minimum F number in the minimum focal length, and minimum F number in the maximum focal length, which are specification information for the lens that was used in photography. When the minimum F number is unknown, the notation is 0/0

and the way of generating these got a bit more complicated as well..

Code: [Select]

function add_properties() -- {{{
        xmp:add_property(xmp.lens_make,                 function() return (lens.make or "-- unknown --") end)
        xmp:add_property(xmp.lens_name,                 function() return lens.name end)
        xmp:add_property(xmp.lens,                      function()
                        if lens.make ~= nil then
                                return lens.make.." "..lens.name
                        else
                                return lens.name
                        end
                end)
        xmp:add_property(xmp.focal_length,              function() return (lens.focal_length*10) end)
        xmp:add_property(xmp.apertureValue,             function() return (lens.manual_aperture*10) end)
        xmp:add_property(xmp.FNumber,                   function() return (lens.manual_aperture*10) end)
        xmp:add_property(xmp.MaxApertureValue,          function()
                        local max_aperture      = (lens.max_aperture or lens.manual_aperture)
                        if lens.f_values ~= nil then
                                local Fns       = lens.f_values
                                max_aperture    = Fns[1]
                        end
                        return max_aperture
                end)
        xmp:add_property(xmp.MinApertureValue,          function()
                        local min_aperture      = (lens.min_aperture or lens.manual_aperture)
                        if lens.f_values ~= nil then
                                local Fns       = lens.f_values
                                min_aperture    = Fns[#Fns]
                        end
                        return min_aperture
                end)
        xmp:add_property(xmp.lens_serial,               function() return lens.serialN end)
        xmp:add_property(xmp.lens_specification,        function()
                        local focal_min = (lens.focal_min or lens.focal_length)
                        local focal_max = (lens.focal_max or lens.focal_length)
                        local FminL     = (lens.FminL or lens.manual_aperture)
                        local FmaxL     = (lens.FmaxL or lens.manual_aperture)
                        return string.format("%s/1 %s/1 %s/1 %s/1", tostring(focal_min), tostring(focal_max), tostring(FminL), tostring(FmaxL))
                end)
end -- }}}

add_properties()

The unfortunate fact is, that the min/max aperture values and lens_specs don't work the way i want them to. But technically those are not important at all for normal people..

[Walter Schulz]

Well, rawtherapee seems to ignore the xmp file altogether

Have you contacted rawtherapee dev(s) about this? In the past we had some friendly contact with user heckflosse to solve some issues.

[zLOST]

Have you contacted rawtherapee dev(s) about this? In the past we had some friendly contact with user heckflosse to solve some issues.

Not yet. This was like the second or third time i've run it so far. But it might be a nice feature to have the output format configurable for individual editors (including the filename, but in DT case it would require some troubleshooting why the heck io.* crashes when working on files with two suffixes).

Seems like we'd need completely different format.. But it might be doable as well I'll play with it a bit tomorrow..
(this is a part of pp3 file used as meta storage by RT, which may be of some use for us)

Code: [Select]

[LensProfile]
LcMode=lfauto
LCPFile=
UseDistortion=true
UseVignette=true
UseCA=true
LFCameraMake=
LFCameraModel=
LFLens=

[MetaData]
Mode=1

[Exif]
Exif.UserComment=user comment. blabla

[IPTC]
Caption=iptc description...;
CaptionWriter=;
Category=;
City=;
Copyright=;
Country=;
Creator=;
CreatorJobTitle=;
Credit=;
DateCreated=;
Headline=;
Instructions=;
Province=;
Source=;
Title=;
TransReference=;