Assign lens focal length and name for non cpu lenses

[IBIRRU]

@garry23
yes I have the xmp sidecar file generated from lua script into the SD card toghether the CR2 file

[garry23]

@dfort

No sidecar file on my sic) EOSM

[dfort]

Strange. Try resetting your Canon settings and deleting the ML settings and see if that does it. You'll have to switch to manual mode to get to the menus that enable shooting w/o a lens to use your Rokinon lens.

Though one more thing that might be getting in the way--is your Rokinon lens chipped? Try shooting a frame without a lens mounted and see if that creates the .xmp file.

[garry23]

Reset Canon settings and see .xmp file

I carry on trying to get things up and running a report back.

Plus strange colours all gone !

Thanks for your support

[dmilligan]

You probably had QR turned off.

Config saving is working as expected on my end. Pushed a couple of other fixes: menu named had been changed (that's why it wasn't showing up under "Lens Info/Prefs", actually was showing up as it's own menu top right), and selection dialog disappearing immediately on subsequent lens changes.

[garry23]

David

Sorry, why do you need image review on for this to work?

[dmilligan]

Code: [Select]

function xmp:start()
    local this = self
    function property.GUI_STATE:handler(value)
        if value == GUISTATE_QR then
            this:write()
        end
    end
end


[garry23]

David

Sorry I didn't spot that in the script.

But, unless I'm mistaken, this only works if image review is on, and that seems too restrictive. What about a timelapse with no image review on?

Could an alternative be to always test if a manual lens is flagged via event.post_shoot: which would always create an xmp after an image has been taken?

Just a thought.

Cheers

Garry

[JohanJ]

@garry23

Could an alternative be to always test if a manual lens is flagged via event.post_shoot: which would always create an xmp after an image has been taken?

Sounds like a good idea! I am about to test magiclantern-manual_lens_info.2017Jan08.60D111 . When shooting a burst of several pics only the last picture taken produces an xmp sidecar file. One xmp for each picture would be preferable!

[dfort]

Config saving is working as expected on my end. Pushed a couple of other fixes...

Working on my end now too. Last lens used is highlighted when camera starts. The thing is that you have to press set. Doing a shutter half-press will not erase the lens selection menu and gets you into a sort of weird state and it is sometimes easier to restart the camera than to keep punching buttons to get out of it.

Pulled in the fixes and Lens Info/Prefs shows the manual lens but other fields that maybe should be filled in show up as "(no lens)".



The lens name is too long for the "Manual Lens" field but that's how the lens profiles are named by Adobe. Here are the manual lenses that I own that have Adobe lens profiles:



Actually, I don't have any "Samyang" branded lenses, they are all Bower or Rokinon (whatever was on sale), and they are all cine lenses so I guess if I'm hacking profiles I could also shorten the names. However, Zeiss ZF.2 lenses are working without having to modify the profiles so we'll just have to deal with long names.

I've been using LiveView so much that I almost missed that the lens appears on the regular screen. All lens information is missing from the ML LiveView screen when a manual lens is mounted.

...When shooting a burst of several pics only the last picture taken produces an xmp sidecar file...

Reproduced that issue here.



Also tried turning off Image Review off and no XMP file was generated. I usually keep Image Review off so that might be why I didn't realize that XMP was working a couple of months ago. It would be great if another trigger for saving the XMP could be used.



Finally, I spent way too much time today tinkering with a feature that's already working but could be just a little better. The formatting of the XMP file:

Code: [Select]

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
        exif:FNumber="14/10"
    exif:FocalLength="50/1"
    aux:Lens="Zeiss Planar T* 1.4/50 ZF.2"
>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Notice the extra indent of the "exif:FNumber" field and the widowed ">" tag element. I'd like it to look like this:

Code: [Select]

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FNumber="14/10"
    exif:FocalLength="50/1"
    aux:Lens="Zeiss Planar T* 1.4/50 ZF.2">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

The indent issue was easy to solve by removing the extra indentation in scripts/lib/xmp.lua but couldn't figure out how to remove the last newline. In any case, no big deal because it is working. Just spent too many years as a film editor obsessing over every minor detail. How it looks is everything in the film industry.

[dmilligan]

Could an alternative be to always test if a manual lens is flagged via event.post_shoot: which would always create an xmp after an image has been taken?

event.post_shoot doesn't actually work. The module CBR it uses isn't actually implemented in the ML core. Implementing it correctly is actually probably a fair amount of work. There are lots of tricky situations to deal with. Burst mode will probably be a challenge.


To me, the xml format issue is not worth the hassle and extra code necessary to fix.

[garry23]

@dmilligan

I must say, I and I don't wish to appear negative, at the moment I will most probably drop using the lens info script.

For timelapse, where I tend to use manual lenses, the ISO and shutter time are recorded, so holy grail shooting should be ok. There are also other ways to fix the missing FL and apperture EXIF that should work in LR, eg lens tagger: but I haven't used that approach for a while.

I will keep an eye on developments and might come back to the script approach later.

Cheers

Garry

[dfort]

@garry23

At least the manual_lens_info builds will give you the latest lua fixes. Of course there's nothing stopping you from writing your own variation of the script, you are pretty good at that!

I was thinking about timelapse work and how some photographers untwist their lenses to prevent flickering. The information for an EF lens can be put into the script so that ACR can automatically detect the lens even if that lens wasn't communicating with the body.

To me, the xml format issue is not worth the hassle and extra code necessary to fix.

How about this? It looks good and works with ACR:

Code: [Select]

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FocalLength="50/1"
    aux:Lens="Samyang 50mm f/1.4 AS UMC"
    exif:FNumber="14/10"
    >
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

To get this was really quite simple:

scripts/lib/xmp.lua

Code: [Select]

-xmp.property_format = [[
-    %s="%s"
-]]
+xmp.property_format = [[%s="%s"
+    ]]

Of course you can decide if you want pretty code or pretty xmp files.

The version of the lens.lua script that I'm using for testing includes all the Zeiss ZF.2 and Samyang lenses with lens profiles that are included in ACR. It also covers Rokinon and Bowers and the Cine variations. I was wondering if you would consider adding it so it will appear in the experimentals downloads. The lenses are all commented out so the user simply needs to comment out or remove "My Lens" and "My Other Lens" and uncomment the lenses to include. There's some hacking necessary for the Samyang lens profiles to auto detect in ACR which I have documented but the Zeiss ZF.2 lenses are working fine.

Should I submit pull requests for these changes?

[garry23]

Dan

Yes, I had already decided to use manual_lens_info to access the 'latest' Lua fixes

I have done the lens twist trick myself, so a good point.

As I said, I'll most probably relook at LR Lens Tagger, ie can change EXIF from within LR.

Cheers

Garry

[garry23]

@all

In case you hadn't picked up on LensTagger, it adds in FL and aperture, and other EXIF data, from within LR.

It works for me

http://www.essl.de/wp/software/lenstagger-lightroom-plugin/

[JohanJ]

Tested the latest experimental magiclantern-manual_lens_info.2017Jan09.60D111 and added the following lens definition to lens.lua (taken from dforts script, but altered manual_aperture from 3.5 to 5.6).

    { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS",    focal_length =   8, manual_aperture = 5.6 },

Corresponding xmp (part):

Code: [Select]

<rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
        aux:Lens="Samyang 8mm f/3.5 UMC Fish-Eye "
    exif:FocalLength="8/1"
    exif:FNumber="49/10"
>

Some findings:

1: In XMP aux:Lens seems to be cut after a certain number of chars.

Code: [Select]

aux:Lens="Samyang 8mm f/3.5 UMC Fish-Eye ""CS" is missing at the end of the string.

I understood from dforts investigations in Adobe's forum that there is a general issue in automatically identifying Samyang lens profiles. But could it be of help to have the full lens name in the string as Adobes profile definition is working with the following parameters:

Code: [Select]

       stCamera:Make="Canon"
       stCamera:CameraRawProfile="True"
       stCamera:Lens="8mm f/3.5"
       stCamera:CameraPrettyName="Canon"
       stCamera:LensPrettyName="Samyang 8mm f/3.5 UMC Fish-Eye CS"
       stCamera:ProfileName="Adobe (Samyang 8mm f/3.5 UMC Fish-Eye CS, Canon)"
2: I chose a manual aperture 5.6 and was expecting "exif:Fnumber = 56/10" but in XMP it is "49/10".  Any hint why?
When setting manual_aperture = 3.5  exif:Fnumber is "35/10".

3: Import to Lr6 brings another issue: Focal lenght is 50mm instead of 8mm, whereas aperture is shown as f/4.9
In xmp I altered exif:FocalLength manually to different values != 50mm, for test reasons. But Lr did not except anything and set it back to standard value 50mm.  The  lens profile was not recognized. Does this cause a dummy focal length?

[dfort]

@JohanJ -- Excellent reporting.

1:

I can confirm that long lens names are being cut off in the XMP file. All of the Zeiss ZF.2 lenses should be automatically detected by ACR but I found one that wasn't working. It looked good in the lua script:

Code: [Select]

{ name = "Zeiss Makro-Planar T* 2/100 ZF.2",     focal_length = 100, manual_aperture = 2   },
It shows the whole name in the Lens Info/Prefs menu:



But it is getting cut off in the XMP file:

Code: [Select]

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FocalLength="100/1"
    exif:FNumber="20/10"
    aux:Lens="Zeiss Makro-Planar T* 2/100 ZF.">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Note that the "2" is cut off the end of the lens name. Also note how pretty the XMP formatting looks with the latest changes.

My understanding is that Adobe lens profiles have several fields that can be used to automatically match the lens with the profile. The software can probably work around cut off lens names by matching up a unique lens ID numbers. You can look up these numbers in the exiftool source code. However, some manual lenses don't have unique lens ID numbers so we're probably stuck with using the lens name.

2:

I couldn't reproduce the manual aperture issue.

lens.lua

Code: [Select]

{ name = "Samyang 8mm f/3.5 UMC Fish-Eye CS",    focal_length =   8, manual_aperture = 5.6 },
XMP

Code: [Select]

    exif:FocalLength="8/1"
    exif:FNumber="56/10"
    aux:Lens="Samyang 8mm f/3.5 UMC Fish-Eye ">

For me it showed up as 56/10 and not 49/10 as in your example.

3:

Can't reproduce that one either. I'm not sure how you got it into Lightroom but it takes some lens profile hacking on the Samyang lenses so this is what I did to work around your first issue.

Instead of using the entire name, shorten it:

Code: [Select]

{ name = "Samyang 8mm f/3.5 Fish-Eye",           focal_length =   8, manual_aperture = 5.6 },
Use that exact same name in the lens profile--note that you don't need to rename the lens profile's file name, only the stCamera:Lens field:

Code: [Select]

stCamera:Lens="Samyang 8mm f/3.5 Fish-Eye"
This is what I get in Lightroom:



Sifting through the EXIF information in the CR2 file I found this:

Code: [Select]

F Number                        : 0
Aperture Value                  : inf
Focal Length                    : 0.0 mm
Lens Info                       : 0mm f/0
Lens Model                      :
Lens Serial Number              : 0000000000
Lens                            : 0.0 mm
Focal Length                    : 0.0 mm (35 mm equivalent: 0.0 mm)
Hyperfocal Distance             : inf m
Lens                            : 0.0 mm (35 mm equivalent: 0.0 mm)

Lightroom is probably not reading your XMP file and it might be defaulting to a 50mm lens perhaps to avoid any divide by zero operations.

[dmilligan]

In ML core, the lens name has a max length of 32 characters (statically allocated). We can make it larger, but it will still get cut off in MLV files (and that can't be fixed without a breaking change to the spec).

[dfort]

Ah, but isn't "Zeiss Makro-Planar T* 2/100 ZF.2" exactly 32 characters? We're getting shorted one character in the XMP while it shows up properly in the Lens Info/Prefs menu.

The long Samyang names we can deal with because we need to hack the Adobe lens profiles anyway but it would be good if we can work with the Zeiss lenses so those profiles don't need to be hacked.

Code: [Select]

Zeiss Makro-Planar T* 2/100 ZF.2
12345678901234567890123456789012
         1         2         3

[dmilligan]

The last character of the string has to be the null terminator (0). So I slightly misspoke, the lens name is limited to 32 bytes (31 ascii characters). The menu is correct b/c the menu's value is set directly by from Lua (where the full string is defined). The xmp retrieves the value from ML core where the limitation exists.

[a1ex]

The PROP_LENS_NAME property (0x80030021) has a length of 70 characters.

The 32-char limit was chosen in 2009 (changeset b6dca6a7e55a), probably arbitrarily, and looks like nobody questioned it until now.

Changing the MLV spec would only break converters that use mlv_lens_hdr_t.lensSerial (the field after lensName), which is currently filled with spaces. The potential breakage is minor (showing part of the lens name instead of serial number). So I'd say, let's change it.

[g3gg0]

possibilities to choose from:

a) redesign LENS to use e.g. name[128] and serial[32]
b) modify LENS to keep the basic format and resize name[48] and serial[16]
c) modify LENS to remove serial and just have lens[64]
d) keep the old LENS for compatibility reasons (truncated name) and add a new optional block e.g. ELNS which has additions like the longer name, serial and other things you could need

solution a) requires a new major version for the MLV format and all tools will have to get updated

b) and c) could work well with some converters, would fail with others but won't probably need a major number change as it will:
-> work with capable readers
-> return weird results with old readers but only the lens serial would have weird values
while c) would be odd for people who manually have lens serials (e.g. defined by LUA) or the feature somewhen really can be used in ML

solution d) - which to be honest is my favorited way to go - will
-> work with capable readers
-> work with old ones as before, just having an unknown block 'ELNS' to them (which every old reader should just ignore silently)

so what to pick?

-> added ELNS proposal https://docs.google.com/spreadsheets/d/1ItNuLj34JlK6bZgiAJLrmG7p3iyCmCju4XYyiwjqJOM at the bottom

[dfort]

Is there a length limit on the lens field specified by some of EXIF and/or Adobe XMP document? I guess I could run some tests and see how long I could make the name in the XMP and lens profile before it breaks.

@g3gg0 - Back in Reply #88 I think you concluded that the lens serial number is not included in the property. I just ran a test with a build that has the latest lens_info_fix and it is still not showing up.

Code: [Select]

Block: LENS
  Offset: 0x0000014c
    Size: 96
    Time: 0.003000 ms
     Name:        'EF-S17-55mm f/2.8 IS USM'
     Serial:      ''
     Focal Len:   17 mm
     Focus Dist:  65535 mm
     Aperture:    f/2.80
     IS Mode:     14
     AF Mode:     0
     Lens ID:     0x000000F0
     Flags:       0x00000000

I agree with your preference for solution d). It seems to be open-ended and would cause the least amount of disruption.

[EDIT] @dmilligan -- as an alternative to using the lens name it should be possible to use the lens ID. These are listed in the exiftool source code and in many of the Adobe lens profiles. I'll need to run some tests to see how this works in ACR.

[dfort]

Some finding on limits to the lens name. Basically, there is no limit on lens profiles and XMP files. I checked the XMP documentation and there is a limit on the size of the file (64kb for JPEG files) but there isn't a limit on each field.

I tested this with the XML "aux:Lens" and lens profile "stCamera:Lens" fields. These are the fields that ACR uses to automatically detect the lens. I tried it with up to 1,000 characters and it still worked. Note that ACR displays the "stCamera:LensPrettyName" and "stCamera:ProfileName" but not the "stCamera:Lens" so it can be anything at all--as long as it doesn't start with a number, found that out by trial and error.

Another test was trying to get auto detection to work in ACR using the lens ID number, but no success so far.

I also looked up EXIF and IPTC documentation and I could not find any limit, though exiftool will cut certain fields off at around 250 characters.

http://www.sno.phy.queensu.ca/~phil/exiftool/ancient_history.html

[JohanJ]

Now working with magiclantern-manual_lens_info.2017Jan11.60D111

@dfort:
I followed your advise and tweaked a user own lens profile by setting "stCamera:Lens" to the exact same value as "aux:Lens" comes with in xmp. And it works fine! Lr chooses my own profile automatically! Thanks! (I tweaked also "stCamera:ProfileName" to "...BS" instead of "CS" just to see which profile was picked).

https://www.flickr.com/photos/147593195@N03/shares/8815Cp

Also the aperture value is shown correctly. I can't reproduce the original artifact which is strange. Consider it solved.

Unfortunately the focal length is still the wrong one! Lr switches to 50mm, no matter what!
https://www.flickr.com/photos/147593195@N03/shares/r9nXi7

I will continue investigating from here.