Author Topic: Assign lens focal length and name for non cpu lenses  (Read 66154 times)

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 177
Re: Assign lens focal length and name for non cpu lenses
« Reply #450 on: February 23, 2019, 07:23:49 PM »
my version of {lens|xmp|ui|logger}.lua is at https://www.zlej.net/manual_lens.tar.gz (you'll just need to delete settings/LENS.LCF, otherwise it will keep on crashing when loading/saving the configuration due to added "Format" select to the lens configuration menu),

Thank you zLost! Had a look to understand why my version of xmp.lua with added support for array of property wasn't working and found i wasn't converting to table before adding them.

It's not currently present in the updated PR at the moment to keep it simple, but I was thinking that is good to add and leave the possibility to use this features in the future.

With my "exiv2" template the output is not a XMP sidecarfile for each photo, but one bash script (with one line being added for each pic), which i'll run once i'm done with shooting and it updates the exif data of the existing CR2/JPG files, so no messing with metadata during saving the picture. So this should be safe way to get the data there ;)

An example of such line is:
Code: [Select]
exiv2 -k -M"set Exif.CanonCs.LensType 65535" -M "set Exif.CanonCs.MinAperture 288" -M "set Xmp.aux.Lens Meyer-Optik Gorl
itz Telemegor 180mm f/5.5" -M "set Exif.CanonCs.Lens 180 180 1" -M "set Exif.Photo.LensSpecification 180/1 180/1 56/10 5
6/10" -M "set Exif.Photo.MaxApertureValue 325770/65536" -M "set Exif.Photo.FNumber 55/10" -M "set Exif.Photo.FocalLength
 180/1" -M "set Exif.Photo.LensMake Meyer-Optik Gorlitz" -M "set Exif.CanonSi.TargetAperture 157" -M "set Exif.CanonSi.A
pertureValue 157" -M "set Exif.CanonCs.MaxAperture 159" -M "set Exif.Photo.ApertureValue 322363/65536" -M "set Exif.Cano
n.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensSerialNumb
er 1728275" -M "set Exif.Image.LensInfo 180/1 180/1 56/10 56/10" IMG_6929.CR2
This looks interesting :)

I would add a new menu (like for Autoload option) to "Lens Pref" instead of inside "Lens" submenu, just to have all the options organised and easy accessible.

"Config.lua" library already support multiple menus and config tables in the same script in this branch

There are changes all over the place, so it may be better to discuss this by email, skype, icq or irc.
We could add a couple of commits with your features in the PR if we can manage to keep script simple (both usage and code complexity).

Take a look on how mercurial (Sourcetree or equivalent gui) works and how to contribute :)

What about an external DB of lenses and user will only say, which are of some interest for him/her? ;)
Currently the script already have it in a certain way: It's the table "Lenses" with the commented entry (Carl Zeiss, Nikon, Samyang...).

At the moment on each script update, the user need to insert again his lenses to the table...

It's possible to create a config from that table and add it automatically to the generated configuration file (lens.lcf), but it's a bit ugly:
Code: [Select]
return {
       [1] = {
        ["name"] = "My Lens",
        ["focal_length"] = 50,
        ["manual_aperture"] = "1.8"
       },
       [2] = {
        ["name"] = "My Other Lens",
        ["focal_length"] = 25,
        ["manual_aperture"] = "2.8",
        ["f_values"] = {
          [1] = "2.8",
          [2] = "4",
          [3] = "5.6",
          [4] = "8"}
        },
["Autoload Lens"] = "ON",
["Manual Lens"] = {
["Lens"] = 1,
["Aperture"] = "1.8",
["Focal Length"] = 50,
},
}
Maybe it's better to put it inside a "Lenses" block so it's a bit easier to understand what is it by looking at the config.

I was trying to add this directly to the library by adding and additional parameter to the function responsible for config creation, but got some problems and was taking too much time.

It can also be done in the script by doing something like this:
Code: [Select]
ExampleTable = {Lenses = {...}}
lenses_cfg = config.create(ExampleTable)

Attribute added locally:
-Fmin,Fmax (from zLOST)
-lens specification (from zLOST)
Would like to calculate aperture min and max from FNumber automatically instead of adding 2 more attribute to lenses.
I've added a function set_lens_defaults(), which generates certain attributes. Including the Fmax Fmin from f_values (if available)...
Made a in a similiar way. First was added to restore_lens_values, then refactored a bit and added update_aperture_focal_range(), because it was using common code needed when switching lens and autoloading the script.

The Samyang lens didn't do as well in ACR.
The profile should be included.
I'm still a little bit temped to not separate Lens name in Make + Model if possible... but these cases is what is making things hard regarding automatic profile selection...

From what i have understood by reading your posts, is that each editor may look for different attribute when selecting profile (Lens, Lens Make, Lens Model....)
Update the image's attributes by modifying exif like zLost did with exiv2 is something to keep in mind.

Updated the PR with latest tweaks (except for lensSpecification and support for multiple name for the same attribute in xmp.lua), so now it's possible to compile a non crashing build for all cameras to experiment with it.

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 177
Re: Assign lens focal length and name for non cpu lenses
« Reply #451 on: February 27, 2019, 10:46:33 PM »
@zLost I'm trying to understand what attribute is used on each editing software and what metadata need to be rewritten to have correct lens name embedded in the picture.

I selected "Samyang 8mm f/3.5 UMC Fish-Eye CS II" from the script and taken a CR2 picture without a lens installed, so no lens-related metadata were included.

XMP sidecar looks like this: (let's keep it simple for now, no exif:lensModel or others... is exif:lensModel to be preferred to aux:lens instead?)
Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FocalLength="8/1"
    exif:FNumber="35/10"
    aux:SerialNumber="0"
    aux:Lens="Samyang 8mm f/3.5 UMC Fish-Eye CS II"
    exif:MaxApertureValue="35/10"
    exif:MinApertureValue="226/10">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Lens correction profile is applied automatically with Photoshop and probably also with Lightroom, but other software appear to skip the sidecar and try to recognise lens using metadata...

EXIF looks like this with exiv2 and exiftool:
Code: [Select]
exiv2 -g Lens -g Aperture -Pknyt IMG_2321.CR2
Exif.Photo.ApertureValue                     ApertureValue               Rational   Finf
Exif.CanonCs.LensType                        LensType                    Short      (0)
Exif.CanonCs.Lens                            Lens                        Short      0.0 mm
Exif.CanonCs.MaxAperture                     MaxAperture                 Short      65504
Exif.CanonCs.MinAperture                     MinAperture                 Short      65504
Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
Exif.CanonSi.TargetAperture                  TargetAperture              Short      65504
Exif.CanonSi.ApertureValue                   ApertureValue               Short      65504
Exif.Canon.LensModel                         LensModel                   Ascii
Code: [Select]
Lens Type                       : Unknown (0)
Lens Model                      :
Lens                            : 0.0 mm
Lens ID                         : Unknown (0)
Lens                            : 0.0 mm (35 mm equivalent: 0.0 mm)

Now let's rewrite metadata with this command line (from one of your posts):
Code: [Select]
exiv2 -k -M"set Exif.CanonCs.LensType 65535" -M "set Exif.CanonCs.MinAperture 285" -M "set Exif.Photo.ApertureValue 322363/65536" -M "set Exif.CanonCs.Lens 180 180 1" -M "set Exif.Photo.LensSpecification 180/1 180/1 56/10 56/10" -M "set Exif.Photo.MaxApertureValue 322363/65536" -M "set Exif.Photo.LensSerialNumber 1728275" -M "set Exif.CanonSi.TargetAperture 157" -M "set Exif.Canon.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.FocalLength 180/1" -M "set Exif.CanonSi.ApertureValue 157" -M "set Exif.CanonCs.MaxAperture 157" -M "set Exif.Photo.FNumber 55/10" -M "set Xmp.aux.Lens Meyer-Optik Gorlitz Telemegor 180mm f/5.5" -M "set Exif.Image.LensInfo 180/1 180/1 56/10 56/10" -M "set Exif.Photo.LensMake Meyer-Optik Gorlitz" IMG_2321_test.CR2
Code: [Select]
Exif.Photo.ApertureValue                     ApertureValue               Rational   F5.5
Exif.Photo.MaxApertureValue                  MaxApertureValue            Rational   F5.5
Exif.CanonCs.LensType                        LensType                    Short      n/a
Exif.CanonCs.Lens                            Lens                        Short      180.0 mm
Exif.CanonCs.MaxAperture                     MaxAperture                 Short      F5.5
Exif.CanonCs.MinAperture                     MinAperture                 Short      F22
Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
Exif.CanonSi.TargetAperture                  TargetAperture              Short      F5.5
Exif.CanonSi.ApertureValue                   ApertureValue               Short      F5.5
Exif.Canon.LensModel                         LensModel                   Ascii      Telemegor 180mm f/5.5
Exif.Photo.LensSpecification                 LensSpecification           Rational   180/1 180/1 56/10 56/10
Exif.Photo.LensMake                          LensMake                    Ascii      Meyer-Optik Gorlitz
Exif.Photo.LensModel                         LensModel                   Ascii      Telemegor 180mm f/5.5
Exif.Photo.LensSerialNumber                  LensSerialNumber            Ascii      1728275
Exif.Image.LensInfo                          LensInfo                    Rational   180/1 180/1 56/10 56/10
Xmp.aux.Lens                                 Lens                        XmpText    Meyer-Optik Gorlitz Telemegor 180mm f/5.5

Lens is showed differently on each software:

Adobe Photoshop                          -> Telemegor 180mm f/5.5
DxO Optic Pro 10                          -> 180mm
Darktable                                     -> n/a                                            ---> is the same for you zLost?
RawTherapee / Preview(MacOS)     ->  Canon Telemegor 180mm f/5.5   ---> what is causing the prefix?

Let's try compare with a picture taken with an automatic lens: (Tamron SP AF 17-50mm F/2.8 XR Di II LD Aspherical [IF])
Code: [Select]
exiv2 -g Lens -g Aperture -Pknyt IMG_1874.CR2
Exif.Photo.ApertureValue                     ApertureValue               Rational   F3.5
Exif.CanonCs.LensType                        LensType                    Short      Tamron AF 17-50mm f/2.8 Di-II LD Aspherical
Exif.CanonCs.Lens                            Lens                        Short      17.0 - 50.0 mm
Exif.CanonCs.MaxAperture                     MaxAperture                 Short      F2.8
Exif.CanonCs.MinAperture                     MinAperture                 Short      F32
Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
Exif.CanonSi.TargetAperture                  TargetAperture              Short      F3.6
Exif.CanonSi.ApertureValue                   ApertureValue               Short      F3.6
Exif.Canon.LensModel                         LensModel                   Ascii      17-50mm
Code: [Select]
Lens Type                       : Canon EF 28-70mm f/2.8L USM or Sigma or Tamron Lens
Lens Model                      : 17-50mm
Lens                            : 17.0 - 50.0 mm
Lens ID                         : Tamron AF 17-50mm f/2.8 Di-II LD Aspherical
Lens                            : 17.0 - 50.0 mm (35 mm equivalent: 26.9 - 79.2 mm)

This is what I have observed:

Adobe Photoshop             -> 17-50mm                                                                     ----> is using "Exif.Canon.LensModel" (NO Automatic lens corretion!)
DxO Optic Pro 10             -> Tamron SP AF17-50mm f/2.8 XR DI ii LD Aspherical (IF)
                                           looks like is comparing "Exif.CanonCs.LensType" with an internal lenses database (prompt for downloading correct module)
Darktable                        -> Tamron AF 17-50mm f/2.8 Di-II LD Aspherical                ----> is using "Exif.CanonCs.LensType" straight off exiv2
RawTherapee                   -> Canon 17-50mm                                                          ----> looks like is using "Exif.Canon.LensModel" with "Canon" as prefix (still Bad...)
Preview(MacOS)              -> Tamron AF 17-50mm f/2.8 Di-II LD Aspherical                ----> looks better now... is "Exif.CanonCs.LensType"

So, it's important to override Exif.Canon.LensModel with correct lens name (maker + model), find a way to handle "Exif.CanonCs.LensType" (Lens ID database) plus additional infos about focal length and aperture to make software heuristic happy?

Can someone try to replicate this experiment with your lenses (both Canon and Third Party if possible) and editing software?

zLOST

  • New to the forum
  • *
  • Posts: 31
Re: Assign lens focal length and name for non cpu lenses
« Reply #452 on: March 29, 2019, 11:42:35 PM »
Sorry for being a bit quiet. In the meantime i've played with the script, moved the format submenu to the one with general script settings (autoload_menu) and made up this page, where i've summarized current state:

http://ml.zlej.net/

There is a LensName issue with Samyang 85/1.4, but otherwise we should be pretty much set ;)

Lars Steenhoff

  • Senior
  • ****
  • Posts: 453
Re: Assign lens focal length and name for non cpu lenses
« Reply #453 on: March 29, 2019, 11:44:01 PM »
Nice overview!  thanks for sharing

zLOST

  • New to the forum
  • *
  • Posts: 31
Re: Assign lens focal length and name for non cpu lenses
« Reply #454 on: March 29, 2019, 11:49:24 PM »
From what i have understood by reading your posts, is that each editor may look for different attribute when selecting profile (Lens, Lens Make, Lens Model....)
Update the image's attributes by modifying exif like zLost did with exiv2 is something to keep in mind.

LensFun library, which is used by DarkTable, Gimp and RawTherapee, is trying to do some "intelligent" guessing and matching of exif data with existing lens profiles. They may use the LensType ID as well https://sno.phy.queensu.ca/~phil/exiftool/TagNames/Canon.html#LensType because what for example my Sigma 150-500 is reporting as LensModel (just "150-500mm") is pretty much useless. I haven't checked lensfun sources, but they seem to be combining LensMake (when available) with LensModel and some variations of LensModel ("EF 70-200" versus "EF70-200" and so on).

zLOST

  • New to the forum
  • *
  • Posts: 31
Re: Assign lens focal length and name for non cpu lenses
« Reply #455 on: March 29, 2019, 11:50:49 PM »
Nice overview!  thanks for sharing

With pleasure, Lars ;)

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 177
Re: Assign lens focal length and name for non cpu lenses
« Reply #456 on: March 30, 2019, 12:32:32 AM »
Sorry for being a bit quiet. In the meantime i've played with the script, moved the format submenu to the one with general script settings (autoload_menu) and made up this page, where i've summarized current state:

http://ml.zlej.net/
Thank you for the sample image, I'll take a look at metadata as soon as possibile.

Was working on a refactor of your addition for format selection last couple of days.
I'm trying to find a way to don't mess code too much and avoid the current limitation to allow user to use custom tags also for exiv2,

There is a LensName issue with Samyang 85/1.4, but otherwise we should be pretty much set ;)
Can you try with the scripts version from the pending PR?

LensFun library, which is used by DarkTable, Gimp and RawTherapee, is trying to do some "intelligent" guessing and matching of exif data with existing lens profiles. They may use the LensType ID as well https://sno.phy.queensu.ca/~phil/exiftool/TagNames/Canon.html#LensType because what for example my Sigma 150-500 is reporting as LensModel (just "150-500mm") is pretty much useless. I haven't checked lensfun sources, but they seem to be combining LensMake (when available) with LensModel and some variations of LensModel ("EF 70-200" versus "EF70-200" and so on).
I also think each software is guessing using some heuristics from focal length, lens ID an maybe others... camparing with an internal database to choose from.

Maybe just override metadatas in Exif and Canon Makernotes, may be sufficient. Avoiding inserting redundant info like Photo.LensMake (which will also avoid necessity to handle and save other field in script)