Author Topic: Assign lens focal length and name for non cpu lenses  (Read 81318 times)

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 190
Re: Assign lens focal length and name for non cpu lenses
« Reply #450 on: February 23, 2019, 07:23:49 PM »
my version of {lens|xmp|ui|logger}.lua is at https://www.zlej.net/manual_lens.tar.gz (you'll just need to delete settings/LENS.LCF, otherwise it will keep on crashing when loading/saving the configuration due to added "Format" select to the lens configuration menu),

Thank you zLost! Had a look to understand why my version of xmp.lua with added support for array of property wasn't working and found i wasn't converting to table before adding them.

It's not currently present in the updated PR at the moment to keep it simple, but I was thinking that is good to add and leave the possibility to use this features in the future.

With my "exiv2" template the output is not a XMP sidecarfile for each photo, but one bash script (with one line being added for each pic), which i'll run once i'm done with shooting and it updates the exif data of the existing CR2/JPG files, so no messing with metadata during saving the picture. So this should be safe way to get the data there ;)

An example of such line is:
Code: [Select]
exiv2 -k -M"set Exif.CanonCs.LensType 65535" -M "set Exif.CanonCs.MinAperture 288" -M "set Xmp.aux.Lens Meyer-Optik Gorl
itz Telemegor 180mm f/5.5" -M "set Exif.CanonCs.Lens 180 180 1" -M "set Exif.Photo.LensSpecification 180/1 180/1 56/10 5
6/10" -M "set Exif.Photo.MaxApertureValue 325770/65536" -M "set Exif.Photo.FNumber 55/10" -M "set Exif.Photo.FocalLength
 180/1" -M "set Exif.Photo.LensMake Meyer-Optik Gorlitz" -M "set Exif.CanonSi.TargetAperture 157" -M "set Exif.CanonSi.A
pertureValue 157" -M "set Exif.CanonCs.MaxAperture 159" -M "set Exif.Photo.ApertureValue 322363/65536" -M "set Exif.Cano
n.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensSerialNumb
er 1728275" -M "set Exif.Image.LensInfo 180/1 180/1 56/10 56/10" IMG_6929.CR2
This looks interesting :)

I would add a new menu (like for Autoload option) to "Lens Pref" instead of inside "Lens" submenu, just to have all the options organised and easy accessible.

"Config.lua" library already support multiple menus and config tables in the same script in this branch

There are changes all over the place, so it may be better to discuss this by email, skype, icq or irc.
We could add a couple of commits with your features in the PR if we can manage to keep script simple (both usage and code complexity).

Take a look on how mercurial (Sourcetree or equivalent gui) works and how to contribute :)

What about an external DB of lenses and user will only say, which are of some interest for him/her? ;)
Currently the script already have it in a certain way: It's the table "Lenses" with the commented entry (Carl Zeiss, Nikon, Samyang...).

At the moment on each script update, the user need to insert again his lenses to the table...

It's possible to create a config from that table and add it automatically to the generated configuration file (lens.lcf), but it's a bit ugly:
Code: [Select]
return {
       [1] = {
        ["name"] = "My Lens",
        ["focal_length"] = 50,
        ["manual_aperture"] = "1.8"
       },
       [2] = {
        ["name"] = "My Other Lens",
        ["focal_length"] = 25,
        ["manual_aperture"] = "2.8",
        ["f_values"] = {
          [1] = "2.8",
          [2] = "4",
          [3] = "5.6",
          [4] = "8"}
        },
["Autoload Lens"] = "ON",
["Manual Lens"] = {
["Lens"] = 1,
["Aperture"] = "1.8",
["Focal Length"] = 50,
},
}
Maybe it's better to put it inside a "Lenses" block so it's a bit easier to understand what is it by looking at the config.

I was trying to add this directly to the library by adding and additional parameter to the function responsible for config creation, but got some problems and was taking too much time.

It can also be done in the script by doing something like this:
Code: [Select]
ExampleTable = {Lenses = {...}}
lenses_cfg = config.create(ExampleTable)

Attribute added locally:
-Fmin,Fmax (from zLOST)
-lens specification (from zLOST)
Would like to calculate aperture min and max from FNumber automatically instead of adding 2 more attribute to lenses.
I've added a function set_lens_defaults(), which generates certain attributes. Including the Fmax Fmin from f_values (if available)...
Made a in a similiar way. First was added to restore_lens_values, then refactored a bit and added update_aperture_focal_range(), because it was using common code needed when switching lens and autoloading the script.

The Samyang lens didn't do as well in ACR.
The profile should be included.
I'm still a little bit temped to not separate Lens name in Make + Model if possible... but these cases is what is making things hard regarding automatic profile selection...

From what i have understood by reading your posts, is that each editor may look for different attribute when selecting profile (Lens, Lens Make, Lens Model....)
Update the image's attributes by modifying exif like zLost did with exiv2 is something to keep in mind.

Updated the PR with latest tweaks (except for lensSpecification and support for multiple name for the same attribute in xmp.lua), so now it's possible to compile a non crashing build for all cameras to experiment with it.

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 190
Re: Assign lens focal length and name for non cpu lenses
« Reply #451 on: February 27, 2019, 10:46:33 PM »
@zLost I'm trying to understand what attribute is used on each editing software and what metadata need to be rewritten to have correct lens name embedded in the picture.

I selected "Samyang 8mm f/3.5 UMC Fish-Eye CS II" from the script and taken a CR2 picture without a lens installed, so no lens-related metadata were included.

XMP sidecar looks like this: (let's keep it simple for now, no exif:lensModel or others... is exif:lensModel to be preferred to aux:lens instead?)
Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FocalLength="8/1"
    exif:FNumber="35/10"
    aux:SerialNumber="0"
    aux:Lens="Samyang 8mm f/3.5 UMC Fish-Eye CS II"
    exif:MaxApertureValue="35/10"
    exif:MinApertureValue="226/10">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Lens correction profile is applied automatically with Photoshop and probably also with Lightroom, but other software appear to skip the sidecar and try to recognise lens using metadata...

EXIF looks like this with exiv2 and exiftool:
Code: [Select]
exiv2 -g Lens -g Aperture -Pknyt IMG_2321.CR2
Exif.Photo.ApertureValue                     ApertureValue               Rational   Finf
Exif.CanonCs.LensType                        LensType                    Short      (0)
Exif.CanonCs.Lens                            Lens                        Short      0.0 mm
Exif.CanonCs.MaxAperture                     MaxAperture                 Short      65504
Exif.CanonCs.MinAperture                     MinAperture                 Short      65504
Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
Exif.CanonSi.TargetAperture                  TargetAperture              Short      65504
Exif.CanonSi.ApertureValue                   ApertureValue               Short      65504
Exif.Canon.LensModel                         LensModel                   Ascii
Code: [Select]
Lens Type                       : Unknown (0)
Lens Model                      :
Lens                            : 0.0 mm
Lens ID                         : Unknown (0)
Lens                            : 0.0 mm (35 mm equivalent: 0.0 mm)

Now let's rewrite metadata with this command line (from one of your posts):
Code: [Select]
exiv2 -k -M"set Exif.CanonCs.LensType 65535" -M "set Exif.CanonCs.MinAperture 285" -M "set Exif.Photo.ApertureValue 322363/65536" -M "set Exif.CanonCs.Lens 180 180 1" -M "set Exif.Photo.LensSpecification 180/1 180/1 56/10 56/10" -M "set Exif.Photo.MaxApertureValue 322363/65536" -M "set Exif.Photo.LensSerialNumber 1728275" -M "set Exif.CanonSi.TargetAperture 157" -M "set Exif.Canon.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.FocalLength 180/1" -M "set Exif.CanonSi.ApertureValue 157" -M "set Exif.CanonCs.MaxAperture 157" -M "set Exif.Photo.FNumber 55/10" -M "set Xmp.aux.Lens Meyer-Optik Gorlitz Telemegor 180mm f/5.5" -M "set Exif.Image.LensInfo 180/1 180/1 56/10 56/10" -M "set Exif.Photo.LensMake Meyer-Optik Gorlitz" IMG_2321_test.CR2
Code: [Select]
Exif.Photo.ApertureValue                     ApertureValue               Rational   F5.5
Exif.Photo.MaxApertureValue                  MaxApertureValue            Rational   F5.5
Exif.CanonCs.LensType                        LensType                    Short      n/a
Exif.CanonCs.Lens                            Lens                        Short      180.0 mm
Exif.CanonCs.MaxAperture                     MaxAperture                 Short      F5.5
Exif.CanonCs.MinAperture                     MinAperture                 Short      F22
Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
Exif.CanonSi.TargetAperture                  TargetAperture              Short      F5.5
Exif.CanonSi.ApertureValue                   ApertureValue               Short      F5.5
Exif.Canon.LensModel                         LensModel                   Ascii      Telemegor 180mm f/5.5
Exif.Photo.LensSpecification                 LensSpecification           Rational   180/1 180/1 56/10 56/10
Exif.Photo.LensMake                          LensMake                    Ascii      Meyer-Optik Gorlitz
Exif.Photo.LensModel                         LensModel                   Ascii      Telemegor 180mm f/5.5
Exif.Photo.LensSerialNumber                  LensSerialNumber            Ascii      1728275
Exif.Image.LensInfo                          LensInfo                    Rational   180/1 180/1 56/10 56/10
Xmp.aux.Lens                                 Lens                        XmpText    Meyer-Optik Gorlitz Telemegor 180mm f/5.5

Lens is showed differently on each software:

Adobe Photoshop                          -> Telemegor 180mm f/5.5
DxO Optic Pro 10                          -> 180mm
Darktable                                     -> n/a                                            ---> is the same for you zLost?
RawTherapee / Preview(MacOS)     ->  Canon Telemegor 180mm f/5.5   ---> what is causing the prefix?

Let's try compare with a picture taken with an automatic lens: (Tamron SP AF 17-50mm F/2.8 XR Di II LD Aspherical [IF])
Code: [Select]
exiv2 -g Lens -g Aperture -Pknyt IMG_1874.CR2
Exif.Photo.ApertureValue                     ApertureValue               Rational   F3.5
Exif.CanonCs.LensType                        LensType                    Short      Tamron AF 17-50mm f/2.8 Di-II LD Aspherical
Exif.CanonCs.Lens                            Lens                        Short      17.0 - 50.0 mm
Exif.CanonCs.MaxAperture                     MaxAperture                 Short      F2.8
Exif.CanonCs.MinAperture                     MinAperture                 Short      F32
Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
Exif.CanonSi.TargetAperture                  TargetAperture              Short      F3.6
Exif.CanonSi.ApertureValue                   ApertureValue               Short      F3.6
Exif.Canon.LensModel                         LensModel                   Ascii      17-50mm
Code: [Select]
Lens Type                       : Canon EF 28-70mm f/2.8L USM or Sigma or Tamron Lens
Lens Model                      : 17-50mm
Lens                            : 17.0 - 50.0 mm
Lens ID                         : Tamron AF 17-50mm f/2.8 Di-II LD Aspherical
Lens                            : 17.0 - 50.0 mm (35 mm equivalent: 26.9 - 79.2 mm)

This is what I have observed:

Adobe Photoshop             -> 17-50mm                                                                     ----> is using "Exif.Canon.LensModel" (NO Automatic lens corretion!)
DxO Optic Pro 10             -> Tamron SP AF17-50mm f/2.8 XR DI ii LD Aspherical (IF)
                                           looks like is comparing "Exif.CanonCs.LensType" with an internal lenses database (prompt for downloading correct module)
Darktable                        -> Tamron AF 17-50mm f/2.8 Di-II LD Aspherical                ----> is using "Exif.CanonCs.LensType" straight off exiv2
RawTherapee                   -> Canon 17-50mm                                                          ----> looks like is using "Exif.Canon.LensModel" with "Canon" as prefix (still Bad...)
Preview(MacOS)              -> Tamron AF 17-50mm f/2.8 Di-II LD Aspherical                ----> looks better now... is "Exif.CanonCs.LensType"

So, it's important to override Exif.Canon.LensModel with correct lens name (maker + model), find a way to handle "Exif.CanonCs.LensType" (Lens ID database) plus additional infos about focal length and aperture to make software heuristic happy?

Can someone try to replicate this experiment with your lenses (both Canon and Third Party if possible) and editing software?

zLOST

  • New to the forum
  • *
  • Posts: 36
Re: Assign lens focal length and name for non cpu lenses
« Reply #452 on: March 29, 2019, 11:42:35 PM »
Sorry for being a bit quiet. In the meantime i've played with the script, moved the format submenu to the one with general script settings (autoload_menu) and made up this page, where i've summarized current state:

http://ml.zlej.net/

There is a LensName issue with Samyang 85/1.4, but otherwise we should be pretty much set ;)

Lars Steenhoff

  • Senior
  • ****
  • Posts: 460
Re: Assign lens focal length and name for non cpu lenses
« Reply #453 on: March 29, 2019, 11:44:01 PM »
Nice overview!  thanks for sharing

zLOST

  • New to the forum
  • *
  • Posts: 36
Re: Assign lens focal length and name for non cpu lenses
« Reply #454 on: March 29, 2019, 11:49:24 PM »
From what i have understood by reading your posts, is that each editor may look for different attribute when selecting profile (Lens, Lens Make, Lens Model....)
Update the image's attributes by modifying exif like zLost did with exiv2 is something to keep in mind.

LensFun library, which is used by DarkTable, Gimp and RawTherapee, is trying to do some "intelligent" guessing and matching of exif data with existing lens profiles. They may use the LensType ID as well https://sno.phy.queensu.ca/~phil/exiftool/TagNames/Canon.html#LensType because what for example my Sigma 150-500 is reporting as LensModel (just "150-500mm") is pretty much useless. I haven't checked lensfun sources, but they seem to be combining LensMake (when available) with LensModel and some variations of LensModel ("EF 70-200" versus "EF70-200" and so on).

zLOST

  • New to the forum
  • *
  • Posts: 36
Re: Assign lens focal length and name for non cpu lenses
« Reply #455 on: March 29, 2019, 11:50:49 PM »
Nice overview!  thanks for sharing

With pleasure, Lars ;)

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 190
Re: Assign lens focal length and name for non cpu lenses
« Reply #456 on: March 30, 2019, 12:32:32 AM »
Sorry for being a bit quiet. In the meantime i've played with the script, moved the format submenu to the one with general script settings (autoload_menu) and made up this page, where i've summarized current state:

http://ml.zlej.net/
Thank you for the sample image, I'll take a look at metadata as soon as possibile.

Was working on a refactor of your addition for format selection last couple of days.
I'm trying to find a way to don't mess code too much and avoid the current limitation to allow user to use custom tags also for exiv2,

There is a LensName issue with Samyang 85/1.4, but otherwise we should be pretty much set ;)
Can you try with the scripts version from the pending PR?

LensFun library, which is used by DarkTable, Gimp and RawTherapee, is trying to do some "intelligent" guessing and matching of exif data with existing lens profiles. They may use the LensType ID as well https://sno.phy.queensu.ca/~phil/exiftool/TagNames/Canon.html#LensType because what for example my Sigma 150-500 is reporting as LensModel (just "150-500mm") is pretty much useless. I haven't checked lensfun sources, but they seem to be combining LensMake (when available) with LensModel and some variations of LensModel ("EF 70-200" versus "EF70-200" and so on).
I also think each software is guessing using some heuristics from focal length, lens ID an maybe others... camparing with an internal database to choose from.

Maybe just override metadatas in Exif and Canon Makernotes, may be sufficient. Avoiding inserting redundant info like Photo.LensMake (which will also avoid necessity to handle and save other field in script)

zLOST

  • New to the forum
  • *
  • Posts: 36
Re: Assign lens focal length and name for non cpu lenses
« Reply #457 on: April 18, 2020, 12:19:45 AM »
Damned, it's been over a year already..

In the meantime i did couple changes in my code, fixed some stuff, implemented other and changed one exif field.

I've also replaced the dummy sample images at http://ml.zlej.net with real ones i took few weeks ago to compare all my manual lenses. All those jpegs with apricot tree have exif data updated by my script, so feel free to grab them and test in your favorite editor. So far the only lenses with lensfun profiles in my case are zenitar 16/2.8, samyang 85/1.4 and industar-50-2 50/3.5..

What i've found out is, that darktable is using Exif.CanonCs.LensType to search for the correction profile. I've been setting it to 65536 == "not available", which made DT to ignore all Lens* fields. So in current version of the script i'm completely dropping this field and voila - detection works :)

cheers,

zLOST

Thank you for the sample image, I'll take a look at metadata as soon as possibile.

Was working on a refactor of your addition for format selection last couple of days.
I'm trying to find a way to don't mess code too much and avoid the current limitation to allow user to use custom tags also for exiv2,
Can you try with the scripts version from the pending PR?
I also think each software is guessing using some heuristics from focal length, lens ID an maybe others... camparing with an internal database to choose from.

Maybe just override metadatas in Exif and Canon Makernotes, may be sufficient. Avoiding inserting redundant info like Photo.LensMake (which will also avoid necessity to handle and save other field in script)

IBIRRU

  • New to the forum
  • *
  • Posts: 38
Re: Assign lens focal length and name for non cpu lenses
« Reply #458 on: April 23, 2020, 11:54:12 AM »
zLOST Thanks for your update! :)
I try with a EOS 700D and as stated in a previous post the experimental build do not works.
I download the Aprofiti build and after add your last lua.zip it works!

I prefer the exiv2 mode that build the manual.sh script

I download also the exiv2 binary for windows and after unzip and add to the PATH i'm able to run it directly from Win10 command prompt (I open the manual.sh in Notepad++ and save as manual.bat)

EOS 700D 1.1.5, EOS 500D

zLOST

  • New to the forum
  • *
  • Posts: 36
Re: Assign lens focal length and name for non cpu lenses
« Reply #459 on: April 23, 2020, 06:25:11 PM »
You're welcome.
You can easily change the filename in lib/xmp.lua at line 54:
Code: [Select]
        exiv2           = {
                filepath        = function() return dryos.shooting_card.dcim_dir.path .. "manual.sh" end,

Just replace manual.sh with manual.bat to save you a bit of work.. I haven't tried it with windows, so you are probably my first tester :)

zLOST Thanks for your update! :)
I try with a EOS 700D and as stated in a previous post the experimental build do not works.
I download the Aprofiti build and after add your last lua.zip it works!

I prefer the exiv2 mode that build the manual.sh script

I download also the exiv2 binary for windows and after unzip and add to the PATH i'm able to run it directly from Win10 command prompt (I open the manual.sh in Notepad++ and save as manual.bat)

IBIRRU

  • New to the forum
  • *
  • Posts: 38
Re: Assign lens focal length and name for non cpu lenses
« Reply #460 on: April 27, 2020, 02:09:23 PM »
zLOST, thanks again.
Just replaced manual.sh to manual.bat.

Shots some pictures and verified.

Double click on manual.bat and it run flawlessly. No needs to open the Command prompt window.
 :)
EOS 700D 1.1.5, EOS 500D

zLOST

  • New to the forum
  • *
  • Posts: 36
Re: Assign lens focal length and name for non cpu lenses
« Reply #461 on: April 27, 2020, 08:47:04 PM »
And photoshop or whichever editor/viewer you use is showing the right exif values now?

I'm really glad to hear that.

zLOST, thanks again.
Just replaced manual.sh to manual.bat.

Shots some pictures and verified.

Double click on manual.bat and it run flawlessly. No needs to open the Command prompt window.
 :)

IBIRRU

  • New to the forum
  • *
  • Posts: 38
Re: Assign lens focal length and name for non cpu lenses
« Reply #462 on: April 28, 2020, 01:39:23 PM »
And photoshop or whichever editor/viewer you use is showing the right exif values now?

I'm really glad to hear that.

Yes PS2020 and Rawtherapee show the correct value. I need to try with Samyang 14 mm if they detect the right lens profile.

I'll update soon about this

EOS 700D 1.1.5, EOS 500D

zLOST

  • New to the forum
  • *
  • Posts: 36
Re: Assign lens focal length and name for non cpu lenses
« Reply #463 on: April 28, 2020, 03:17:57 PM »
Lensfun expects the lens model to be "Samyang 14mm f/2.8 AE ED AS IF UMC", so this line should do the trick:
Code: [Select]
{ make = "Samyang", name = "14mm f/2.8 AE ED AS IF UMC",  focal_length = 14, manual_aperture = 2.8,  serialN = "123456",  aperture_max = 2.8, aperture_min = 22, f_values = {"2.8","4","5.6","8","11","16","22"} },

Yes PS2020 and Rawtherapee show the correct value. I need to try with Samyang 14 mm if they detect the right lens profile.

I'll update soon about this

Milk and Coffee

  • Freshman
  • **
  • Posts: 85
Re: Assign lens focal length and name for non cpu lenses
« Reply #464 on: June 22, 2020, 07:40:30 PM »
Can this module be used to program EMF/Focus Confirm chips on adapted manual lenses?
Gear: Canon 5D Mark II

zLOST

  • New to the forum
  • *
  • Posts: 36
Re: Assign lens focal length and name for non cpu lenses
« Reply #465 on: June 26, 2020, 11:26:17 AM »
no idea, to be honest. after couple failed attempts to find one with chip, which actually worked with my digic and/or how to program that damned thing, i've abandoned these "smarter" adapters and am using only the "dumb" ones.
i assume, that the chip would have to be much much smarter to fully fake the modern lens behavior.

Can this module be used to program EMF/Focus Confirm chips on adapted manual lenses?