Magic Lantern Forum

Developing Magic Lantern => Feature Requests => Topic started by: Lars Steenhoff on October 29, 2016, 12:04:45 PM

Title: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 29, 2016, 12:04:45 PM
If we can assign lens focal length and name for non cpu lenses, ( like using nikon lenses on a canon with an adapter) then I can use this data in post processing to identify what lens was used and which lens profile I should apply for distortion correction.

Nikon has something like this and It saves me to manually write down all the lenses used per shot.



Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 29, 2016, 01:58:11 PM
Interesting. I used Nikons when I was making a living in photography (too many years ago) and never had to replace my old lenses, just get them modified to AI for metering on newer bodies.

That Nikon feature you are pointing out only works with Nikon bodies and F-mount lenses and is designed to allow metering through the camera while keeping the aperture wide open until the moment of exposure. Canon EF doesn't use any mechanical coupling so if you adapt a Nikon lens to a Canon body the lens stops down as you close the aperture. This is something desirable in time-lapse and stop motion animation because it eliminates mechanically induced flicker.

However, what you want is for the lens information to be saved in the metadata. It doesn't seem like this Nikon feature specifies the lens model, only the focal length and maximum aperture. Don't you need more information for distortion control? Do the programs that provide lens profile distortion correction even allow looking up Nikon lenses on Canon bodies? Maybe some experimenting modifying the metadata with exiftool is in order before going any further.

Note that maybe this can be done as a lua script that includes a list of just the lenses that you own.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 29, 2016, 02:46:33 PM
Yes thats correct I would not need the aperture information so much
My mean use would be for distortion profile for the wide angle lenses that are manual focus. ( and nikon mount )

If I just could write the focal length and a description text (like ZEISS 50) or (Nikon 50) in the meta data then I know what to select in ACR.  Ideally ACR would figure out which lens I used automatically. but that would need some investigation into which how the EXIF data is formatted.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 29, 2016, 02:52:18 PM
As seen in this video about the ACR lens profiles

There is the automatic selection if it detects the lens meta data, and I can select here the right lens.
Just after shooting on trip with different zeiss lenses after some days I forget which lens was used in my shots. so this would help a lot for that.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 29, 2016, 04:29:02 PM
Of course with your feature request you still have to remember to change the lens information in the menu every time you switch lenses but it would be nice to have the metadata recorded properly in camera instead of having to write it down or try to remember when you're editing the photos.

I just checked in Lightroom and it is possible to manually select any lens no matter what camera body you used, even combinations that are impossible like a COOLPIX lens on a Canon body.

Ok, the thing is will your editing program recognize the lens from the metadata? This is what I'm getting from a CR2 shot on an EOSM:
Code: [Select]
Lens Info                       : 11-22mm f/0
Lens Model                      : EF-M11-22mm f/4-5.6 IS STM
Lens Serial Number              : 0000004904

Hum, f/0? Anyway, it looks like the Lens Model tag is what we're interested in. (Off topic: I can't understand why people blank out serial numbers on posts unless they are using stolen equipment.) According to the exiftool instructions you can write to that tag. Let's say you want to put my lens on your file named "myfile.CR2"
Code: [Select]
exiftool -LensModel="EF-M11-22mm f/4-5.6 IS STM" myfile.CR2
I haven't tried this myself and maybe you don't want to modify the original file but there's also a way to write a sidecar XMP file with exiftool. Come to think of it writing an XMP file in camera might be doable and it looks like your feature request would a good candidate for a lua script.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 29, 2016, 04:49:52 PM
I just did test with the exif info you provided and Adobe camera raw recognises the lens profile from that info :)

I used this app to edit the exif info, but im sure you can edit the same info also with the exif tool
https://itunes.apple.com/nl/app/exif-editor/id477514607?l=en&mt=12

(http://www.larssteenhoff.com/forum_pics/exif.png)
Title: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on October 29, 2016, 07:06:57 PM
Excellent progress guys and Thanks for starting this thread @Lars Steenhoff -- I also am intrigued by this exiftool app (Exif Editor) from the iTunes App store. Seems like a nice gem but for $10.99 I'm not so sure yet. Care to try and convince me?  :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 29, 2016, 11:31:48 PM
We know that it is possible in post with various exif editing apps.

There seems to be two ways to do this in camera. The easy way would be to write an XMP sidecar file every time you shoot a picture. The harder way would be to re-assign the lens information so that it writes the metadata directly into the raw file. Either way should work with Adobe Camera Raw to do automatic lens correction.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 29, 2016, 11:53:45 PM
Right now my main interest is to get the metadata into an mlv file.
then when the dngs are generated in post this metadata will be put inside the dng files.

My knowledge of how the metadata is written in mlv video is limited.

Anyone know how its written in the mlv?

for stills it would be nice to have it in the cr2 but this might be more difficult. then the xmp route might be best for stills.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 29, 2016, 11:57:13 PM
@DeafEyeJedi. I just bought the tool because i was impatient to use the terminal and just bought it as it was the first thing that i found that looked simple, it is very simple to use 😎
but there are many free tools that can do the same.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 30, 2016, 02:08:32 AM
My knowledge of how the metadata is written in mlv video is limited.

Same here but we do know that the lens information does shows in the DOF display:

(https://c4.staticflickr.com/6/5441/30649992635_53f170edef.jpg)

And it is in the MLV file in the Lens Block:

Code: [Select]
Block: LENS
  Offset: 0x0000014c
    Size: 96
    Time: 0.003000 ms
     Name:        'EF-M11-22mm f/4-5.6 IS STM'
     Serial:      ''
     Focal Len:   21 mm
     Focus Dist:  22 mm
     Aperture:    f/9.10
     IS Mode:     0
     AF Mode:     0
     Lens ID:     0x00000033
     Flags:       0x00000000

So it looks like your feature request is probably doable if we can get the attention of someone who can point this discussion in the right direction. Maybe there's a way to change the values for lens_info.name and maybe also lens_info.lens_id? Look in src/lens.h and src/lens.c. Don't know if there are lua hooks that can reassign property values. If you look in modules/lua/lua_property.c you'll find this:

Code: [Select]
// !!!DANGER WILL ROBINSON!!!
//#define LUA_PROP_REQUEST_CHANGE
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on October 30, 2016, 02:34:16 AM
Maybe there's a way to change the values for lens_info.name and maybe also lens_info.lens_id? Look in src/lens.h and src/lens.c. Don't know if there are lua hooks that can reassign property values. If you look in modules/lua/lua_property.c you'll find this:

Code: [Select]
// !!!DANGER WILL ROBINSON!!!
//#define LUA_PROP_REQUEST_CHANGE
Not the way to do it. You don't need to mess with setting Canon properties. lens_info struct is in ML's control. All you need to do is set it's fields to the values you want. This is not currently exposed to lua (the lens properties are readonly), but it wouldn't be hard to change the API so you could write to them from lua.

You also need to check when PROP_LENS_NAME prop handler runs (I don't know when/how often it runs). It's going to overwrite any changes you might have made. You could also maybe use a prop handler for that one as a trigger to prompt the user to enter the lens name (when there's no value from Canon).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 30, 2016, 04:41:15 PM
As usual I was heading in the wrong direction.

@dmilligan -- as always thanks for the tips.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 30, 2016, 09:32:26 PM
Yes great info,  nice to hear that its onder ML control !

I'm just making some photoshop mockups

I think it will be useful to have a predefined list that can be loaded on the memory card that has the custom the exif names, so that in the camera a user only has to select the right lens.
A favorites section of most used lenses can be handy too.

I will do some more thinkering

(http://www.larssteenhoff.com/forum_pics/lens.png)


Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 31, 2016, 12:07:19 AM
Nice, though the choices should probably go into a submenu. If you use that feature a lot you can always put it into a custom menu.

The Focus menu is rather bare on the EOSM but there's lots going on in other cameras--maybe some of this will eventually get into the EOSM?

(https://c5.staticflickr.com/6/5689/30042413444_36bbb11c8c.jpg)

I was thinking of reading from a lens database stored in the ML directory, in flat ascii text format to make it easy. You can put in all your non cpu lenses in there. Of course it would have to be exactly the same as the names in the ACR database.

Another idea is to turn off that menu item when a cpu lens is attached in case the user forgets to switch it off in the menu.
Title: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on October 31, 2016, 12:19:12 AM
Nice, though the choices should probably go into a submenu. If you use that feature a lot you can always put it into a custom menu.

True. Nice mockup @Lars Steenhoff! :)

I was thinking of reading from a lens database stored in the ML directory, in flat ascii text format to make it easy. You can put in all your non cpu lenses in there. Of course it would have to be exactly the same as the names in the ACR database.

Another idea is to turn off that menu item when a cpu lens is attached in case the user forgets to switch it off in the menu.

Great ideas -- especially the menu item shut off w cpu lens attached.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on October 31, 2016, 01:23:15 AM
I was thinking of reading from a lens database stored in the ML directory, in flat ascii text format to make it easy. You can put in all your non cpu lenses in there. Of course it would have to be exactly the same as the names in the ACR database.

The nice thing about Lua is that it's also a great "configuration" language for storing this sort of information. To load the file from a Lua script simply use dofile() (this is how the config.lua module works, your script's configuration information is stored itself as a Lua script). Example:
Code: [Select]
return
{
    "Lens Name #1",
    "Lens Name #2",
    "Lens Name #3",
    "etc..."
}

and to load it:
Code: [Select]
local lens_list = dofile('lens_list.cfg")


This might actually be something you wouldn't put in a menu at all. If you can reliably detect when a non-CPU lens gets attached (should be possible from Lua I think), then simply popup a prompt that asks the user to select the lens they just attached. This also avoids the situation of the user forgetting to make the correct selection.

Also, to implement this for stills, you can probably just write the lens information in an xmp sidecar (already completely possible from Lua).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 31, 2016, 06:46:35 AM
...to implement this for stills, you can probably just write the lens information in an xmp sidecar...

That was my first thought too. That should be pretty easy.

Like you said earlier,

...lens_info struct is in ML's control. All you need to do is set it's fields to the values you want. This is not currently exposed to lua (the lens properties are readonly), but it wouldn't be hard to change the API so you could write to them from lua.

I see that lens_info.name is assigned in src/lens.c
Code: [Select]
PROP_HANDLER( PROP_LENS_NAME )
{
    if( len > sizeof(lens_info.name) )
        len = sizeof(lens_info.name);
    memcpy( (char*)lens_info.name, buf, len );
}

Wouldn't it just be a matter of adding an else statement in order to assign a custom name (up to 32 characters) to lens_info.name? Of course a new variable needs to be declared so does that variable need to be defined in a special way so it is available to lua? Looks like I've got some studying to do.

It shouldn't matter how often the PROP_LENS_NAME handler runs because the length of the string will always be zero if the lens doesn't communicate with the body. If the user changes lenses with the camera on it will also zero out when communications between the lens and body is broken. Am I on the right track with this?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 31, 2016, 01:56:31 PM
Maybe a good choice for the custom lens name variable could be user_lens_info.name:

I also realized a problem after writing that previous post. lens_info.name should be reset before checking it against PROP_LENS_NAME.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 10, 2016, 08:38:02 AM
Thought I'd revisit this.

Looks like the missing link here is to be have a lua variable that can substitute a user supplied lens name in lens_info.name. This is really only necessary for MLV because as it turns out silent still DNG's don't save the lens name. Maybe it should? Otherwise automatic lens distortion correction doesn't work on picture editing applications.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 10, 2016, 01:24:45 PM
Yes we are closing in on theoretical solution, in practice there a couple of steps needed now.

- Not currently exposed to lua (the lens properties are readonly)  (dmilligan)
- Who is able to expose this value, dfort do you know were in bitbucket this value is stored?

- If we have this value we make a list of lenses that are supported in adobe camera raw and make this user customisable
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 10, 2016, 06:33:59 PM
What might work is to modify the backends so that if lens_info.name is empty it will use a lua writeable user_lens_info.name in its place. This shouldn't be too difficult but it is beyond my coding abilities. At least the part about adding something to the lua API.

@dmilligan -- are we on the right track here?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on November 11, 2016, 03:15:05 AM
I don't think that's necessary. Just make it so you can write to lens_info.name from lua.

Not tested:

Code: [Select]
diff --git a/modules/lua/lua_lens.c b/modules/lua/lua_lens.c
--- a/modules/lua/lua_lens.c
+++ b/modules/lua/lua_lens.c
@@ -48,7 +48,12 @@
 static int luaCB_lens_newindex(lua_State * L)
 {
     LUA_PARAM_STRING_OPTIONAL(key, 2, "");
-    if(!strcmp(key, "name") || !strcmp(key, "focal_length") || !strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af"))
+    if(!strcmp(key, "name"))
+    {
+        LUA_PARAM_STRING(value, 3);
+        strncpy(lens_info.name, value, 31);
+    }
+    else if(!strcmp(key, "focal_length") || !strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af"))
     {
         return luaL_error(L, "'%s' is readonly!", key);
     }

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 11, 2016, 08:14:07 AM
Cool -- Thanks!

Looks like we've something to chew on for a while.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 16, 2016, 01:07:19 AM
Baby steps.

I applied the patch to the lua_fix branch, seems like a good one for this.

Next I wrote this very simple script:

lens.lua
Code: [Select]
-- Non CPU Lens
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Rokinon Cine Prime"
end
print("lens: "..lens.name)
print "Press any key to exit."
key.wait()
console.hide()

Ran the script once with a lens attached then removed the lens and ran it again. It works!

Out of curiosity I also ran this on a build that didn't have the patch that dmilligan provided and got an error stating that "name is read only" when there was no lens attached.

Next test, will the metadata stick? Here's the Lens block data from mlv_dump -v:

Code: [Select]
Block: LENS
  Offset: 0x0000014c
    Size: 96
    Time: 0.003000 ms
     Name:        'Rokinon Cine Prime'
     Serial:      ''
     Focal Len:   50 mm
     Focus Dist:  0 mm
     Aperture:    f/0.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

Looks like everything is working. Now how to write a lua script to make this practical? @garry23 are you following this?

Thanks again @dmilligan ! Should we (I or you) submit a PR to the lua_fix branch for this?

[EDIT] Should the Serial and Focal Len also be writable? Is there a way to look up the Lens ID? It might be useful for let's say using Nikon CPU lenses with a type G adapter.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 16, 2016, 01:34:29 AM
Great progress !   :)

Focal length I think yes
Perhaps if the name has the focal length included then it may not be nessesary.

Did you open the dng file in Adobe Camera Raw?   And did it detect the profile?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 16, 2016, 04:17:17 AM
I just got it working and haven't done much testing yet.  Let me know what camera(s) you have and I can send you a patched ML build with this script.

I saw on another topic that you're thinking of setting up a development system. What's cool about lua scripting is that you don't need to set up anything and can start coding with just a text editor and a little help from your ML forum friends.


Sent from my iPhone using Tapatalk
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 16, 2016, 10:45:02 AM
Cool,  I have the canon 5d mark 3, and a couple of nikon mount zeiss and nikon lenses

Zeiss 25 f2 zf.2
Zeiss 35 f2 zf.2
Zeiss 50 f2 zf.2 mp
Zeiss 85 f1.4 zf.2

I will try the build and see if i can get then reconised.
About compiling,  i still have to read up a bit on how it works, will give it a try soon.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 16, 2016, 06:50:46 PM
@Lars_Steenhoff

Here is a build for you to try out. It is from the lua_fix branch with dmilligan's patch and has the lens.lua script in it:
https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-custom_lens_name.2016Nov16.5D3113.zip

The lua script I posted earlier had a mistake in it. You can't do:

Code: [Select]
lens_info.name = "Zeiss 25 f2 zf.2"
To assign a value to lens_info.name from lua you need to:

Code: [Select]
lens.name = "Zeiss 25 f2 zf.2"
I copied a previous version when I was writing the post and corrected it later. I also edited the post.

Here are the EXIF lens tags from the dng files extracted from the MLV file:

Quote
F Number                        : 0
Subject Distance              : 0 m
Focal Length                    : 50.0 mm
Lens Model                      : Rokinon Cine Prime

Note that some of the information from the LENS Block of the MLV file didn't get saved in the dng metadata. I also checked if the Lens Model is saved when using the Silent (stills) module and it does when using MLV but it doesn't with the DNG option. As expected CR2 also didn't save Lens Model.

We should probably test this some more before making any other lens properties writable to lua. I can see the value of having control over the Focal Length and Lens Serial Number information in controlled situations. For example when doing stop motion animation with manual lenses it would help to have this in the metadata if someone needs to do a reshoot and the lens wasn't noted on the shot log. Of course if they didn't take the time to fill out the shot log they probably didn't run the lua script either. In any case, this project may appeal to just a few users but it does seem worthwhile fleshing it out.

If you want to check out the code without having to apply the patch it is in my custom_lens_name branch.

https://bitbucket.org/daniel_fort/magic-lantern/branch/custom_lens_name
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 17, 2016, 11:20:13 AM
Thanks dfort,

I just tested the script, and it works fine!

I changed the name in the lua script to match the name in ACR, in this case it looked like this

Code: [Select]
-- Non CPU Lens
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Distagon T* 2/25 ZF.2"
end
print("lens: "..lens.name)
print "Press any key to exit."
key.wait()
console.hide()


One more property I would love to write to the file is the focal length, and if possible also the aperture.  Then we have complete lens info for distortion corrections as wel as for cataloging.
Now I see in ACR 50 mm while its a 25mm

Here on the left is the auto detection result   :)   
(http://www.larssteenhoff.com/forum_pics/lenseacr1.png)   (http://www.larssteenhoff.com/forum_pics/lensacr2.png)

And here on the right is the list of zeiss lenses how they are written in ACR
When using these exact names ACR selects the profile automatically, so thats a great result!  Thanks
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 17, 2016, 03:28:07 PM
and it shows up fine in MLVplayer too  :)

(http://www.larssteenhoff.com/forum_pics/cap.png)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 17, 2016, 06:46:59 PM
One more property I would love to write to the file is the focal length, and if possible also the aperture.

That should be doable. On cpu lenses the aperture tracks the F-stop and focal length changes dynamically with zoom lenses but with non cpu lenses these are set to the default values of f0.0 and 50mm. Lens names are probably updated only when the camera senses a change in the lens but we're not sure how often the aperture and focal length properties are polled so there's a chance that it might not work.

Another thing that should be doable is to put these tags into dng files when using the silent module. CR2 files will most likely need the xmp sidecar treatment. Since you are mainly interested in working with MLV files let's concentrate on that first.

If you have some coding ability feel free to jump in, otherwise you'll have to wait for me to figure out how to expand on dmilligan's patch.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on November 17, 2016, 06:52:21 PM
Wonderful stuff @Lars Steenhoff and Thanks for sharing these useful links as you obviously got @dfort's attention!  :P

and it shows up fine in MLVplayer too  :)

That's pretty cool actually. Useful details on the previewer!

That should be doable. On cpu lenses the aperture tracks the F-stop and focal length changes dynamically with zoom lenses but with non cpu lenses these are set to the default values of f0.0 and 50mm. Lens names are probably updated only when the camera senses a change in the lens but we're not sure how often the aperture and focal length properties are polled so there's a chance that it might not work.

Ah, that's true. Point taken. I still think it may be worth to try?

Another thing that should be doable is to put these tags into dng files when using the silent module. CR2 files will most likely need the xmp sidecar treatment. Since you are mainly interested in working with MLV files let's concentrate on that first.

If you have some coding ability feel free to jump in, otherwise you'll have to wait for me to figure out how to expand on dmilligan's patch.

Ouch that's a tough call... Wish I could have time to get my hands dirty with coding. Anyone else care to jump onboard with this project?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 17, 2016, 09:39:02 PM
https://helpx.adobe.com/camera-raw/kb/supported-lenses.html (https://helpx.adobe.com/camera-raw/kb/supported-lenses.html)

Here is the whole list of lenses that are supported in adobe, I'm thinking that it would be cool to make a selector in the menu that has this list already.

But first the basics, I will have a look at the source code.
I can understand some coding, and I'm able to change some scripts, real coding however is not really my skill.  so I may get stuck when I have to write a proper function :)


@DeafEyeJedi  great feedback
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 17, 2016, 10:24:52 PM
Looking at that list it seems that some lens names are longer than the 32 characters assigned to that EXIF tag. Have you checked that ACR recognizes your lens automatically? It might instead be looking at Lens ID and going through a lookup table. If that's case we also need to have the MLV processing apps pass along that tag to the DNG files for automatic lens distortion correction to work properly.

That's the goal, isn't it? The Nikon feature that you pointed out is so that metering would work with non-cpu lenses. That's something that probably isn't possible on Canon cameras because there is no mechanical connection between the camera and lens aperture control.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 17, 2016, 10:39:11 PM
Yes ACR recognised the lens automatically with only the name in the EXIF field.

The name on that list is the same as the one I tested, which was "Zeiss Distagon T* 2/25 ZF.2"

I do see some that are longer too, like Nikon AF-S DX VR Zoom-NIKKOR 55-200mm f/4-5.6G IF-ED.
I can do some further testing

Yes metering is not whats needed just the lens info, (name, focal lengt, f stop) so in post I can use this info to set the right profile, or to sort shots by lens.
The nikon video was more to illustrate the concept of setting lens info via a menu into the file.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 17, 2016, 11:06:10 PM
I'm trying to make a list so that I can select a lens in lua, this is how far I got:

Code: [Select]
lenslist = {"Zeiss Makro-Planar T* 2/50 ZF.2", "Zeiss Planar T* 1,4/50 ZF.2", "Zeiss Planar T* 1,4/85 ZF.2", "Zeiss Distagon T* 2/35 ZF.2",
            "Zeiss Distagon T* 2,8/25 ZF.2", "Nikon AF DC-NIKKOR 105mm f/2D", "Nikon AF DC-NIKKOR 135mm f/2D"}

Now I need to write a function to let me select one from this list,

...   Not sure how to proceed from here
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 18, 2016, 02:37:59 AM
...   Not sure how to proceed from here

Find a script that has a menu similar to what you want and copy it. There are several examples in the Scripting Corner (http://www.magiclantern.fm/forum/index.php?board=53.0) of the forum. You can also ask someone that has written some ML lua scripts like @garry23

Your menu is about to become more complicated. I figured out how to write to the focal_length variable. Well, almost--keep reading.

Code: [Select]
    if(!strcmp(key, "focal_length"))
    {
        LUA_PARAM_INT(value, 3);
        lens_info.focal_len = value;
    }
    else if(!strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af"))

This is an integer so it needed to be handled differently from the name. I also took off focal_length from the warning list that it cannot be written to. I'm not sure what to do about the aperture because it isn't a part of lua_lens_fields.

So here's a new lua script that works with this latest change:

Code: [Select]
-- Non CPU Lens
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Distagon T* 2/25 ZF.2"
    lens.focal_length = 25
end
print("lens: "..lens.name)
print("focal_length: "..lens.focal_length)
print "Press any key to exit."
key.wait()
console.hide()

There is a problem though. The focal length value doesn't stick. The script runs fine the first time you run it but not the second time so reassigning the lua lens.focal_length doesn't seem to be changing the lens_info.focal_len property value variable and the MLV file shows the default 50mm lens instead of the script's 25mm.

In any case, this is a work in progress so here's the latest 5D3 build if you want to play with it:

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-custom_lens_name.2016Nov17.5D3113.zip
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 18, 2016, 03:32:05 AM
Thanks!,
Yes I just tested with the same results, the focal length was not saved.

I did make a gif preview to show what the effect of the lens profile is in the case of the zeiss 25.
It corrects for vignetting and for distortion. this is not the best example, in an architecture shot it would be more obvious that lines are straight now.
And a quick tip if you import at 200% in ACR you get a nice 4K upscale from the RAW debayer.

(http://www.larssteenhoff.com/forum_pics/correction.gif)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 18, 2016, 07:22:32 PM
Just tried it and ACR recognized the lens information and applied the correction automatically. Nice!

I take it that this only works properly with the 5D3 using mv1080 (full frame non-crop mode) because the vignetting and distortion would be different with a cropped frame.

I've been searching through the code trying to figure out why assigning the lens focal length isn't working. If that isn't possible I thought that maybe changing the default lens focal length from 50 to 0 might avoid some confusion but I can't find where that's taking place. I have a suspicion that lens_info.focal_len is assigned a value because there are some calculations done with it and a zero value might cause problems. Still, I'd like to give it a try and see what happens.

@dmilligan -- Mind if I put in a pull request of your patch to the lua_fix branch?

In the meantime let's move forward with the lua script. Are you putting your code on bitbucket or someplace where others can collaborate with you?  If your script is small enough you can post it here.

I'm not very familiar with lua and can't answer your question on how to set up a menu but this should help:

http://davidmilligan.github.io/ml-lua/modules/menu.html
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 20, 2016, 11:31:37 PM
@Lars_Steenhoff -- we're so close to being able to assign a value to the focal length:

https://www.magiclantern.fm/forum/index.php?topic=18279.msg175214#msg175214

Basically, I've got a build that works perfectly with manual lenses. No more defaulting to 50mm with non cpu lenses. The problem is that it also needs to work with cpu lenses.

A question for you and @garry23 - is assigning the aperture really going to gain us anything? It seems to me that setting the f-stop or maximum lens aperture will create more problems because there is no way that I can see to set it automatically, setting it manually adds complexity  during the shoot and it doesn't give us anything useful when processing the images other than the old, "I shot it at F5.6 at 250th of a second" information that seemed to be significant only in photography magazines. I mean we can try to put it in but I'm not sure how difficult it will be.

BTW -- has anyone explored programmable chips with Nikon to Canon adapters? Seems like there are some chipped adapters that work with focus confirmation. I heard there are programmable chips but I have no experience with them.

https://www.fotodioxpro.com/products/nikf-eos-dc
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on November 21, 2016, 12:53:04 AM
@dfort

From my perspective, as a stills photographer, not having 'any' meta data is not much of a problem.

The real value, I believe, will be for the timelapsers.

Cheers

Garry
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 21, 2016, 12:55:53 AM
@dfort great to see the focal length close to working, if the cpu lenses have a problem with this then perhaps a way to only enable the focal length setting when a non cpu lens is selected in the menu.

As for the aperture, yes it can be handy to have,  There could be away to set it automatically, The wide open value from the lens name would be used, so for example the "Zeiss Distagon T* 2/25 ZF.2"  would set the aperture to "f2"
This could then be manually changed into a more closed value.   To add the least complexity just taking the wide open value would be a nice start.


I have used a fotodiox pro adapter before, and it gave a beep and a red focus point blink when the shot was in focus.  I don't have one right now but I might get one for trap focus.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 21, 2016, 01:49:20 AM
Ok--let's take this a step at a time. The lens name is working so writing a lua script for that is now possible. Well, it is possible with dmilligan's patch or my branch of lua_fix.

https://bitbucket.org/daniel_fort/magic-lantern/branch/custom_lens_name

I already committed the change needed for lua to have write access to the focal length. Once we work out an issue with getting this working with both manual and cpu lenses we can move on to the aperture.

This can probably benefit the "timelapsers" by saving the lens settings when the lens is twisted off slightly to reduce flickering. Basically, if you hold the depth of field preview button while twisting off the lens just enough to break contact it will lock the aperture so it won't open and close every time the shutter fires and possibly give inconstant exposures from frame to frame.

A little more searching turned up that lens chips may be more trouble than they are worth and years ago users were looking for a solution like the one we're working on.

https://www.magiclantern.fm/forum/index.php?topic=3312.0
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 21, 2016, 05:02:29 PM
Lens name and focal length now seem to be working properly. This time I made a test suite that includes all platforms along with the simple lua script so any ML user should be able to jump in and start scripting.

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-custom_lens_name.2016Nov21.zip

Next challenge, being able to change the metadata for aperture.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 22, 2016, 08:14:12 AM
I thought this would take longer but:

Code: [Select]
Block: LENS
  Offset: 0x0000014c
    Size: 96
    Time: 0.003000 ms
     Name:        'Zeiss Distagon T* 2/25 ZF.2'
     Serial:      ''
     Focal Len:   25 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

Writing to aperture is working. Here's the modified lua script:

Code: [Select]
-- Non CPU Lens
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Distagon T* 2/25 ZF.2"
    lens.focal_length = 25
    lens.aperture = 20
end
print("lens: "..lens.name)
print("focal_length: "..lens.focal_length)
print("aperture: "..lens.aperture/10)
print "Press any key to exit."
key.wait()
console.hide()

Note that the aperture needs to be divided by 10. There's a whole bunch of fancy stuff going on to display the aperture properly on the screen but it is written to the MLV header as an integer. This shouldn't be a problem when saving a value for aperture, simply have the script multiply the f-stop by 10. In other words f5.6 would be saved as 56. There doesn't seem to be a consistent way that the lens aperture appears in the lens name so it would be easier to make a separate field for it than to try and parse it out of the name.

A few interesting things came up. Perhaps mlv_dump should format the f-stop the same way as it is displayed in the live view display. Another thing is that the Serial number isn't being saved.

In any case, this is what the latest commit looks like (https://bitbucket.org/daniel_fort/magic-lantern/commits/bde3925e6ac28faf4fed0c322aaa05f1edda5a75). A block of code was indented to pretty up the formatting on a conditional statement so the diff shows it as a bigger change than it actually is. I tried some things that didn't work and am not 100% sure why somethings are working but hey--it passed my tests.

It is a little late so I'll post a test suite in the morning. I've also got some cleaning up to do before making a pull request to lua_fix.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 22, 2016, 01:57:47 PM
Great work!   8)

I will happily test it out when you have a new build,


I don't know if could you possibly include the new crop-mode module in the build?

I just had the idea to also add a field in the metadata if the shot was recorded with the crop mode active. ( or to just multiply the focal length with the crop, which is not really happening, so that may not be the best )

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on November 22, 2016, 07:51:39 PM
From an API standpoint, it's a bit confusing that there's now a lens.aperture, when there was already a camera.aperture
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on November 22, 2016, 08:16:21 PM
Well, aperture is a lens parameter, while shutter and ISO are camera (or sensor) parameters. On the other hand, all 3 are related (the exposure triangle).

They could probably be moved to expo.aperture, expo.shutter and expo.ISO. Also, expo.ec and expo.flash_ec (or maybe flash.ec, if we plan to expose some more flash functionality) could work.

Exposure transformations might also be useful (e.g. expo.raw2apex).

Just a suggestion.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 22, 2016, 08:44:55 PM
From an API standpoint, it's a bit confusing that there's now a lens.aperture, when there was already a camera.aperture

I totally missed that. Looks like I'm not quite done with this or is a1ex's comments putting the ball in dmilligan's court? As far as I'm concerned writing the aperture of a manual lens isn't as useful as lens name and focal length. In fact it looks like focal length isn't even necessary for automatic distortion correction in ACR. On the other hand if we've gotten this far why not make lens ID and serial number lua writable too?

Great work!   8)

I will happily test it out when you have a new build,


I don't know if could you possibly include the new crop-mode module in the build?

I just had the idea to also add a field in the metadata if the shot was recorded with the crop mode active. ( or to just multiply the focal length with the crop, which is not really happening, so that may not be the best )

Thanks.

I'll be even happier when I see a lua script from you. You can start with the lens name and focal length and add the aperture later.

Adding metadata means changing the MLV specification. I'd check with g3gg0 on that first.

I'm not sure what effect cropping in on the image will have when ACR does automatic lens distortion correction. It is working now with 1920x1080 full frame 5D3 MLV files because it is using the entire sensor area.

In other words, even if we do add everything you're asking for it might not work. Try shooting some cropped and un-cropped footage of a test chart using the same lens making sure you line up the images so they match and run it through ACR's automatic lens distortion correction. Let's see what happens.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 22, 2016, 09:00:10 PM
@dfort yes your exactly right about the crop mode, the correction should not be applied.
Therefore it would be good to know if a shot was filmed in crop mode or normal. so that the lens correction is only used when it was a full frame shot. (I could manually disable the corrections for crop mode shots )

I will have some free time in the weekend and will have a look at getting up to speed with lua.  :)
Title: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on November 22, 2016, 09:47:53 PM
@dfort yes your exactly right about the crop mode, the correction should not be applied.
Therefore it would be good to know if a shot was filmed in crop mode or normal. so that the lens correction is only used when it was a full frame shot.

+1 -- because in the past I've accidentally overstretched some of the footage in post due to being shot crop-mode.

I will have some free time in the weekend and will have a look at getting up to speed with lua.  :)

Sweet. Looking forward to it! :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 22, 2016, 10:33:23 PM
Code: [Select]
lens.name = "X3 Zeiss Distagon T* 2/25 ZF.2"
There could also be a way to write the crop info in the lens name, so no api change is needed, and the lens profile would not be automatically applied.
This does feel a bit of a hack, and there is the character limit of the lens name exif field you mentioned before.

Alternatively the focal length could represent the crop

Code: [Select]
lens.name = "Zeiss Distagon T* 2/25 ZF.2
lens.focal_length = 75

But the fist question @dmilligan would be: is the crop mode setting exposed to lua ?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 23, 2016, 05:48:02 AM
Code: [Select]
lens.name = "X3 Zeiss Distagon T* 2/25 ZF.2"

Does this work with ACR? If you need to enter the lens name manually to correct lens distortion doesn't it kind of defeat the purpose of what we're doing?

@a1ex hinted at adding some new metadata to MLV but how would that work with ACR?

If you would be satisfied with being able to look up a library of flat frames to fix vignetting and some other issues I think that would be possible to implement that without having to add any extra metadata with the cooperation from the developers of applications tailored specifically for MLV files. Applications like MLVFS, MLV Producer and MLP. Though we're talking about a lot of work. I believe @Danne is probably the furthest along with the flat frame thing in MLP. I'm drifting a bit off topic now because this should work for any lens, not just manual lenses.

@dmilligan -- I can rework lens.aperture to camera.aperture or expo.aperature or whatever. What we have going on here is being able to write the lens name, focal length and aperture only for manual lenses via lua. If a lens that communicates with the camera is detected these become read only. This seemed like a worthwhile thing to do but I don't want to put up a pull request that clashes with your vision of the API.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 23, 2016, 07:18:38 PM
@Lars_Steenhoff
@garry23
@DeafEyeJedi

New test builds. This time I went on a merge fest so these have crop_rec, 10bit-12-bit and a few other enhancements along with the lua non-cpu lens thing we're working on here. These are only for the 5D3.113 and EOSM.202 since that's what we seem to be testing this on and are the only platforms that seem to work with the crop_rec mode that was requested for this next round of testing.

Good luck!

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-x-perimental.2016Nov23.zip
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 23, 2016, 07:39:24 PM
Yes this is great,  :) I was just installing the build tools on the mac and already downloaded the xperimental branch, after seeing all the merges going in there  8)

Im still figuring out how to make the build, so thanks for making it available as a download.  Will try it out on the 5d3 now.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 23, 2016, 10:41:15 PM
So far great results! @dfort

I did a few tests by just making copies of the script and adding the lens names to the script.

This works well if I restart the camera for before each lens change, if I just run a different script after each other only the first one will stay active.
The 25 mm and the 35 mm and 50 mm work great

ZEISS25f2.LUA

Code: [Select]
-- ZEISS 25 f2
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Distagon T* 2/25 ZF.2"
    lens.focal_length = 25
    lens.aperture = 20
end
print("lens: "..lens.name)
print("focal_length: "..lens.focal_length)
print("aperture: "..lens.aperture/10)
print "Press any key to exit."
key.wait()
console.hide()


ZEISS35f2.LUA
Code: [Select]
-- ZEISS 35 f2
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Distagon T* 2/35 ZF.2"
    lens.focal_length = 35
    lens.aperture = 20
end
print("lens: "..lens.name)
print("focal_length: "..lens.focal_length)
print("aperture: "..lens.aperture/10)
print "Press any key to exit."
key.wait()
console.hide()

(http://www.larssteenhoff.com/forum_pics/35.png)

ZEISS50MP.LUA

Code: [Select]
-- ZEISS 50MP f2
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Makro-Planar T* 2/50 ZF.2"
    lens.focal_length = 50
    lens.aperture = 20
end
print("lens: "..lens.name)
print("focal_length: "..lens.focal_length)
print("aperture: "..lens.aperture/10)
print "Press any key to exit."
key.wait()
console.hide()


(http://www.larssteenhoff.com/forum_pics/50good.png)

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 23, 2016, 11:31:30 PM
The Zeiss Planar T* 1,4/85 ZF.2 does not show up in ACR for some reason.
And if I compare a shot from the lens on a Nikon camera the exif data does not have the lens name, but ACR does recognise the lens for profile. so I'm not sure were ACR reads this name from.

Canon dng file processed with MLVFS
http://www.larssteenhoff.com/forum_pics/M23-2308MLV_1_2016-11-23_0001_C0000_000010.dng (http://www.larssteenhoff.com/forum_pics/M23-2308MLV_1_2016-11-23_0001_C0000_000010.dng)

Nikon raw file from the nikon df with the zeiss lens.
http://www.larssteenhoff.com/forum_pics/DSC_7499.NEF (http://www.larssteenhoff.com/forum_pics/DSC_7499.NEF)

ZEISS85.LUA
Code: [Select]
-- ZEISS 85 f1.4
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Planar T* 1,4/85 ZF.2"
    lens.focal_length = 85
    lens.aperture = 14
end
print("lens: "..lens.name)
print("focal_length: "..lens.focal_length)
print("aperture: "..lens.aperture/10)
print "Press any key to exit."
key.wait()
console.hide()



(http://www.larssteenhoff.com/forum_pics/85_zeiss_canon.png)
Canon dng file processed with MLVFS
http://www.larssteenhoff.com/forum_pics/M23-2308MLV_1_2016-11-23_0001_C0000_000010.dng (http://www.larssteenhoff.com/forum_pics/M23-2308MLV_1_2016-11-23_0001_C0000_000010.dng)

(http://www.larssteenhoff.com/forum_pics/85_zeiss_on_nikon_df.png)

Nikon raw file from the nikon df with the zeiss lens.
http://www.larssteenhoff.com/forum_pics/DSC_7499.NEF (http://www.larssteenhoff.com/forum_pics/DSC_7499.NEF)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 24, 2016, 12:00:31 AM
ACR can't find "Zeiss Planar T* 1,4/85 ZF.2" but your other lenses are fine? Just checked that file and reproduced the issue. That's very strange. In the Nikon NEF file I see this:

Code: [Select]
Lens ID                         : Carl Zeiss Planar T* 1.4/85 ZF.2
So I edited the "Lens Model" your file with exiftool replacing the "1,4" with "1.4" and it worked:

(https://c8.staticflickr.com/6/5594/31089355831_c9c9f90b1a_n.jpg)

Go figure.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 24, 2016, 12:31:47 AM
Great you found it!   so the comma should be a dot in lens name.

Now all my lenses work!  Thanks for figuring that out


ZEISS85.LUA
Code: [Select]
-- ZEISS 85 f1.4
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Planar T* 1.4/85 ZF.2"
    lens.focal_length = 85
    lens.aperture = 14
end
print("lens: "..lens.name)
print("focal_length: "..lens.focal_length)
print("aperture: "..lens.aperture/10)
print "Press any key to exit."
key.wait()
console.hide()

ZEISS50.LUA
Code: [Select]
-- ZEISS 50 f1.4
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Planar T* 1.4/50 ZF.2"
    lens.focal_length = 50
    lens.aperture = 14
end
print("lens: "..lens.name)
print("focal_length: "..lens.focal_length)
print("aperture: "..lens.aperture/10)
print "Press any key to exit."
key.wait()
console.hide()
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 24, 2016, 12:40:59 AM
The way I tested it was just to rename the scripts and load them all, just to test if they work.
Actually its quite an easy way to select different lenses  :)  just have to restart the camera after a lens change but need to do that anyway.

I guess there is no way to store a lens setting so when I turn the camera on it will auto load the last setting?


(http://www.larssteenhoff.com/forum_pics/shot1.JPG)
(http://www.larssteenhoff.com/forum_pics/shot2.JPG)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 24, 2016, 01:05:56 AM
Found it!

If I turn on autorun the lens is loaded at startup, this is perfect.

Now the only thing not in user menu is the aperture setting, that should be relatively easy to code, I will have a look at some examples
( or you can use the text editor in camera  :D )

(http://www.larssteenhoff.com/forum_pics/autorun1.jpg)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 24, 2016, 01:37:37 AM
All these Zeiss lens names are recognised in ACR
Code: [Select]
Zeiss Distagon T* 2.8/15 ZF.2
Zeiss Distagon T* 3.5/18 ZF.2
Zeiss Distagon T* 2.8/21 ZF.2
Zeiss Distagon T* 2.8/25 ZF.2
Zeiss Distagon T* 2/25 ZF.2
Zeiss Distagon T* 2/28 ZF.2
Zeiss Distagon T* 2/35 ZF.2
Zeiss Distagon T* 1.4/35 ZF.2
Zeiss Makro-Planar T* 2/50 ZF.2
Zeiss Planar T* 1.4/50 ZF.2
Zeiss Planar T* 1.4/85 ZF.2
Zeiss Makro-Planar T* 2/100 ZF.2
Zeiss Apo Sonnar T* 2/135 ZF.2



Now I'm trying to get a Nikon lens to be recognised in ACR but it does not reconise it.

Quote
Nikon AF NIKKOR 35mm f/2D

There are no commas in the name, perhaps the name is wrong or ACR is looking in another exif field for Nikon lenses.
I took those names from the same list on the adobe website.

https://helpx.adobe.com/camera-raw/kb/supported-lenses.html#Nikon (https://helpx.adobe.com/camera-raw/kb/supported-lenses.html#Nikon)

Quote
Nikon AF Fisheye-Nikkor 16mm f/2.8D
Nikon AF NIKKOR 50mm f/1.4D
Nikon AF NIKKOR 35mm f/2D
Nikon AF NIKKOR 24mm f/2.8D

None of these are recognised
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 24, 2016, 01:41:32 AM
Look what I found in the exiftool source code:

lib/Image/ExifTool/Nikon.pm
Code: [Select]
# nikon lens ID numbers (ref 8/11)
Code: [Select]
    '00 40 2D 2D 2C 2C 00 00' => 'Carl Zeiss Distagon T* 3.5/18 ZF.2',
    '00 48 32 32 24 24 00 00' => 'Carl Zeiss Distagon T* 2.8/21 ZF.2',
    '00 54 3C 3C 18 18 00 00' => 'Carl Zeiss Distagon T* 2/28 ZF.2',
    '00 54 44 44 0C 0C 00 00' => 'Carl Zeiss Distagon T* 1.4/35 ZF.2',
    '00 54 44 44 18 18 00 00' => 'Carl Zeiss Distagon T* 2/35 ZF.2',
    '00 54 50 50 0C 0C 00 00' => 'Carl Zeiss Planar T* 1.4/50 ZF.2',
    '00 54 50 50 18 18 00 00' => 'Carl Zeiss Makro-Planar T* 2/50 ZF.2',
    '00 54 62 62 0C 0C 00 00' => 'Carl Zeiss Planar T* 1.4/85 ZF.2',
    '00 54 68 68 18 18 00 00' => 'Carl Zeiss Makro-Planar T* 2/100 ZF.2',
    '00 54 72 72 18 18 00 00' => 'Carl Zeiss Apo Sonnar T* 2/135 ZF.2',

So if exiftool has to look up the lens via the Lens ID ACR can probably use either the Lens ID or Lens Model EXIF tags to identify the lens.

In any case, looks like things are working. Now we need a real lua script. It should be able to run Autorun so that when a non-cpu lens is detected it will look up a configure file for the last lens used and have a menu where you can choose from a list of lenses. We might consider putting all the possible lens combinations that are in the Adobe list that can be adapted to a Canon body in a text file so users don't need to figure out that the way a lens is displayed in ACR might not be exactly the way it is saved in the EXIF tag.

@Lars_Steenhoff - The lens you are looking for is probably:
Code: [Select]
AF Nikkor 35mm f/2Got that from the exiftool code.

@garry23 - It would be great if you can you jump start this when you get back from vacation.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 24, 2016, 01:51:50 AM
http://www.larssteenhoff.com/forum_pics/DSC_7500.NEF (http://www.larssteenhoff.com/forum_pics/DSC_7500.NEF)

Here is a file shot on Nikon with the AF Nikkor 35mm f/2,   I tried this name but it still does not recognise it.
Perhaps you can look in this file to see how and were it stores the lens data.

If I open this file in an exif editor I don't see anything in the lens model field.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 24, 2016, 02:02:26 AM
http://u88.n24.queensu.ca/exiftool/forum/index.php?topic=2754.0 (http://u88.n24.queensu.ca/exiftool/forum/index.php?topic=2754.0)

Here is some talk about encrypted exif data. I do remember that Nikon encrypts its raw files

Quote
Use the exiftool -v3 to see both the encrypted and decrypted LensData in hex.

https://forums.adobe.com/thread/350943 (https://forums.adobe.com/thread/350943)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 24, 2016, 04:53:29 AM
I'm stumped on the Nikon AF Nikkor 35mm f/2 lens. It may be because this is an automatic lens and Nikon uses the LensID tag which is an integer instead of LensModel which is a string. In addition, LensID numbers seem to be dependent on the camera. I could change the LensID tag to show various Canon lenses but when I used the LensID for that lens (66) it came up:

Quote
Lens ID                         : Unknown (66)

So we might be out of luck when it comes to mounting Nikon AF lenses on your Canon and using them as manual lenses. Note that the Zeiss ZF.2 lenses are chipped but they are manual lenses so those seem to work fine.

[EDIT] I posted a question in the Adobe Camera Raw forum (https://forums.adobe.com/message/9158301#9158301). Let's see if we can get an answer on this.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 26, 2016, 08:48:44 AM
Some interesting documentation that shows Lens ID information that was posted on another topic (in French):

http://www.magiclantern.fm/forum/index.php?topic=18308.msg175438#msg175438

There's a long list of lenses along with their Lens ID numbers. Interesting that the EF 70-200mm f/2.8L IS II USM lens has different ID numbers depending on whether or not a tele extender is being used.

Code: [Select]
Lens Id Hex Marque Modèle

251     FB  Canon  EF 70-200mm f/2.8L IS II USM
252     FC  Canon  EF 70-200mm f/2.8L IS II USM + 1.4x
253     FD  Canon  EF 70-200mm f/2.8L IS II USM + 2x

It also lists Zeiss ZE lenses but not ZF.2 lenses which are optically identical and we know are showing up properly in ACR.

Code: [Select]
Lens Id Hex Marque Modèle
33   4  21  Zeiss  Distagon T* 15mm f/2.8 ZE
33   5  21  Zeiss  Distagon T* 18mm f/3.5 ZE
33   6  21  Zeiss  Distagon T* 21mm f/2.8 ZE
33   7  21  Zeiss  Distagon T* 25mm f/2 ZE
33   8  21  Zeiss  Distagon T* 28mm f/2 ZE
33   9  21  Zeiss  Distagon T* 35mm f/2 ZE
33  10  21  Zeiss  Distagon T* 35mm f/1.4 ZE
33  11  21  Zeiss  Planar T* 50mm f/1.4 ZE
33  12  21  Zeiss  Makro-Planar T* 50mm f/2 ZE
33  13  21  Zeiss  Makro-Planar T* 100mm f/2 ZE
33  14  21  Zeiss  Apo-Sonnar T* 135mm f/2 ZE

I have no idea how this works because Lens ID only accepts a single integer value and all the Zeiss ZE lenses use the same integer.

Still no answers on the Adobe forum about the Nikon AF lens issue.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on November 26, 2016, 12:55:19 PM
Yes I think on the adobe forum there won't be much help,
I had some other idea to make new profiles based on the current Nikon profiles and with a name that is known to be recognised.

There is the profile creator from adobe, but I have not yet found a way to open excisting profiles.

https://www.adobe.com/support/downloads/detail.jsp?ftpID=5489 (https://www.adobe.com/support/downloads/detail.jsp?ftpID=5489)

I can understand why the lenses with an extender have a different name, as the profile should also be different as only the center of the lens is used.  this is the same effect of the magic lantern in camera crop module, so it is interesting to see that there is also provision for crop in the profiles.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on November 26, 2016, 09:29:29 PM
I started working on this, progress so far: https://bitbucket.org/hudson/magic-lantern/branch/manual_lens_info
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 27, 2016, 04:40:27 AM
Fantastic!

Ok if I ask for a feature request on this feature request? How about adding lens information tags to dng files so this will work for silent.mo too. We don't need a sidecar xmp for dng files, do we? Would this be in the scope of the manual_lens_info branch?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on November 27, 2016, 09:48:42 AM
just noticed this thread.
it is only about having a proper string for the current lens on MLV side?
or would it require additional data that must be embedded into DNG?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 27, 2016, 05:30:10 PM
@g3gg0

For this to work with MLV, DNG and CR2 --

CR2 will require writing an xmp sidecar file.

DNG needs to have the lens data to be embedded.

MLV does have the required lens data -- though I found a minor issue if we decide to someday add the lens serial number as a user defined field because it isn't being saved. This is from a lens that should have something in the lens serial number field:

Code: [Select]
     Name:        'EF-S17-55mm f/2.8 IS USM'
     Serial:      ''
     Focal Len:   55 mm
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Ant123 on November 27, 2016, 06:05:45 PM
Here you can find parameters taken from 3 lenses:
Code: [Select]
MIME-Version: 1.0
Content-Type: application/octet-stream; name="lensinfo.zip"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="lensinfo.zip"

UEsDBBQAAgAIAIq4PknKhn0C2gsAAPo2AAATAAAATGVucyBpbmZvIEVGLU0tMjItMu2b3Y7cthWA
r62noO/si2Qlzc/ODpICHImakVcSVZGaycYwhCCetgvETrDeFG5v+wTtI/W6D1X+SSIpckYJigJF
MxfGUvp0+HMOD885kn/+7un56csffv5uCz49s79BCxsKns7vH5/O3z8//vgxyJ4eATn/BBYhiONt
HG1XaxCH0dokAT1/egavfubinj89c4mvAb4PgpcvQYMIoiDBZV0gisDLl0EQvE2TAkSfnt+Br6/+
QMCe7RJc0QYXEUho0R0QTFGzBf0v/BzGFsUacZWOiKRCk0ohhR08wryAuwJJmDYtMqkan1DT4apr
0O97gRksiIUhTDp+QaOkMIk9JAUCGbvZxcvDt8bAdGkatpqFLcJZ2HIzC1vNk7aeYOOyjdRtPEvY
7byJ3s0b2t16FhaFtjg3Fs/CULUrHOsRZLtuD2vzjvETNhm0VU57QwIkr+7tJ6Q0sZO4jZWwSkGG
k5a4ZmpieZNblBuzITfmGJzEArlL+kmIwXUlprgxt0xQoIp0WVslXY2aMickxxWos60lz4ll9Sws
0TQmVs5J6QtzgSKWrCAnHaGQtgQcUcLmx5Q/eo9xZCNGTsx5TG0AVzpDmTtlDVy/Iqfohpzi19ve
PDTq0FLKVpnUCGm+jVF3YlwlThGocFPCwmFzskPBpA8VLPPEOagggSVqYN/nvsnrrsYkp3k/C+lI
DYqisu6OsMjHYamNYmC8c6tTQvOisLAGTYamrIcrDbC12rMjJavwdAJ8bLdLwcllAuOfW4vTKfFv
xzfihNKwLq+6bzEuLaGWsIEyhWrCxr3OjI52aZMfh3UZhfWU3FrcOLaO8fcUbCnukgITZFFBckDJ
fUfa0ueIws+7ZdAfy/HH97/4WI59x3JoUTOO5dh5LCv9n/CuS9iSsfXIaof9glAgpzylB+9sBVLi
I1KiHFYUSAc2MZsp0i+/0GFnGLiYmUnVeUWFlV+k8hLuEWmbDCZIzVNqkpk9b3c0L/Nq75r+KEwO
yEnONguOKbPIPz7OCNeYVTgMwZYaCX0fUUNyl2c0KNLu/KSayLX4yhWcTMMrFzWNrlzUrw6uXML+
72OrQB6GFzQquSm2moXZOvVgtlI9mK1UD2ZrVeh0Qtla9QizterBbK36sPUszNaqD4tdWH/A52nk
dQtwM1ILv/MIR2rppTaLoUvlPN6G72xaF9ZTkZsaAhTU5LDoKjwRZwRFA2WLc1PxLGoxi1q6KJ6I
wPRNS1iswQ5X98HBQoB9g9s6sgI5/Se1KbB4HraYhy3nYat52HoedjsP21zFAhHF8vM7a7kZJXnD
NoRuI2x9bz1UpFMpUxYLTXJ0RRj0UIawmaf83Vqe8kXyK2syBU3+yzUZESF12MhFvZWbLNNIpTCR
cst0g0kjXYETM7GdcrEzNRfzVFlyL8+RmgvMpByZuYPiiYQrfLNkTdN3qX4dc05Ajr9PMUT6JePH
hsWNzK9URKTwYnENSuY7/X1NBQZFKKpxSw1QLq2jS6NeMC0YKFYvFozcBEucocoEs1fOg9krpwaH
mgY3LN/LsEvpA6dhDk05salxjL2SU06TgwCdqu/PZhPrJtm3A4OZt1cmgTPj5gSw2rcFHOoRQ+HD
wmTtt5LXeyzYPzS4e4Anb3UslEjNR+c76Pl+F+UJryQN8Unqs/IGsSPUWVWQu2m5llyak/oipzJz
lnn5MENeCb+5xLGIqq9BSFs0ygW6QIMS++s6dYDkuiyrrKBR0c0GwCNyWM1ARXfBG8L8ALlQCGWy
1szXsu6YwXTOQYEhlsEJ0xMzqL0nx49j8OFDwB2nVbjySGuHeWaQl/NSxyYxsApNOQcmM3AmcxIO
OzAuc+SEIzUoMR2Oaum8QxhsGMKWRxz7W01Y4K6XaL+7xSa8XmARlFiAEjb73Je+B9kVoi/DsD3i
rdbE0fJ2uVmsl7cB5y5UY0yW7SfB+nINFxtfZ9m+p7BKXJYpf+vVarEC339QKvkWVz42WgV49wYl
lK3RvsqzBwfHjuVjvq8QpUzlobdMEmqUZ8IiuB8pz1QZtdMoTwrGqIzHC5dtSUayNV5dxsLPaMGo
yDO7kWLhwoVjQFJfhuIX8FDjokCDjC6+ntHJeDa5mE1G3rB80vtscjGLhDsW0vjdNgDLDWcqbx1P
MItZVcS7Ib+YV1y2astz0gtvbTm6XltWb2mvTySLhon8Vg71U7+VQ/+j5dD/8dKTiPz9RThl2wY1
LcIJamNS9tCtLhuUYHa0Ule34edFNKWmKxb1wio2V59fQNkXZRx/+AD+cMMybFqqosHgZBz1AGHC
NjVJ+ZzUpB4grWQizP3S38YqO55yY5O6gXNoOx586o5Or6EMXAwoKkTh6ri90GkMWFg4BytRcoBA
q9+4scYOjxxTkJQRHvmFLa5jMgURJRi98mF87+E0kXFsJmWaiI8yTUQbmiWMeGZgYq40x4GZJuIb
mmkivuWwTcTTqW0iXsw0ES9mmshFLJ6HLa5jpo2oFZFVO3YqeKpKcn1Hyl0tsilHKXEcmibMWaGa
YBPX4cZcNVMHNvEdOhaQtq5xQ/vKJBrebSPTkkpEDzjVSk8sxSz099+Sm2BZQrodTO4LSESOIRbO
SamS8pEPVcaNqnw2Yu6KogMzC3IezC7IeTAxtomnuYB1Ar2O7fH2orQiL4dvZnxYWyV6EUYpa6rM
Tn4P03OhA7EMJJzx3dNQlrz86ZP2hZGoyPVVvo77Ig1XFbmhwneBC2HokMed1hx5Dm6pyfN93SRK
aRY1/bpJBF1DnxdkLW3KKauvUAoHJwrY/HMlpjmCi1bpXr5p0Kt39hQHadHaAq21NUDhQflcfcU+
Do6lPk56KjNCYjbd0spL36MH092olLM7QfOzMV3gWlARoRcpGMDMD4yy5mSpt2yVg692DxT9Lhje
wohzTShquBRPLy2ml5bTSyv7knhvsVV28NUup7zjCEDUKdcbijYqU3VBtpOi/zBStneowCcy3odv
zPt1Zraz2mwXw5eWYfDF9MeYWKtkh6LNx9Q/w9tZqstkii/Ndpbqc4qZezbbbHckuXBDsl3CqpVe
yTemBTsJUnVmhKrN13Wrt9W7jb6damPi7T60kW1kzGEBxGsU4e5l++n86fz05/P7rXdMS1AfMMVi
am5GbibA97uY81AK4VVdceYCnjj1J0zG/AZTrrgGtv3XL6Vs85+61hLjmnyYn7YFrI2HmT93dd5f
NZ5XLp+tk3Yx4IGJS8R4XdAJbYogeCv65o5tzutx7RcU54+f8lTboF+LvFLln0AmlnYKCV6Rx7+e
tyC6Bbu/PJ9fB6fH92eQ/fj9dz8weX98/hOTIt9y0PMP7hvB2yxvSqCS5V8y6uBtBDYgfAeYDDby
XzFpJoM/Gr0LQIrbN/mLFy8eEAnYEc22G2tUOBAeFWp/F+rvEiYNVn8TnNH+OjnkGR2uU1wi1chI
wjSYigbv9Zb1eiB8cRVQoVMtRIoxwAbBXa1u5TpX82EMvbUHVKkGO4cStkN7ATsiGqLFO9ywDtE3
7NhL+z5kyeDH8YFuERv3yGkcGtZERQsmi+4pf1vJBynml3Z866jH5Rk9tutT0xUl7dLE6Bvy5Rkf
4JGTat9Hd+GuZnYopd9H67uxKcawZGNQRZ0VP0UkSHnsK/elepTPSwRmstlWBTuZKkyoGjhbE8QO
x5Rp7R/13//1irY79Fp2sXonB3YDsxu2K5OKMaG8tR5WM7rtFxAiOKxfRoj4hkn2IVzb2OSTl7Ei
V57k2SFx3z/MP7+NbpYg+9s/1fTjyfS5/bCTK2kSqoykUa+0X7yIbljAAeSXN7SBFeFZyagI7qhU
T3frQ8fO/EI1+5eRSqVsZ3Ztremd25D6SEPrNFo3qICUzUeuM78hw8NCKeWovjivsOoorWs11uP5
6cv443sp/47JT/IKlbBL2InH/7+KHIkM8tPmqOye4gaNzft4czdslpSddX2DC43DQVULzZy7eDSa
vNzzF1xdctiPGilEiFvAB3FpmGzGUmJ8En0P10h/RVuTeDNeeTTGw/3NriUPMpfgyujE9lYTZelF
nfZ9/htQSwMEFAACAAgAQKx1SRo0cnVVCwAA9TYAABcAAABMZW5zIGluZm8gRUYtUy0xOC01NS1J
U+2bX4/cthHAn6NPQb/ZD7ZX2tvz3SIpwJWoXfkkURWp25wNQwjia2sg9gXnc9D2tZ+g/Uh97ofq
8J9ESuSeHAQFCmQTBEfqpyE5HA45I+bLD/cP9y9++vLDFn1+gL9Rh1uO7m/ff7i//fHhw92nqLr7
hOq7X1ASw7/bdbxN1ihZxecuifjt5wf09IsQ9/D5QUh8huhVFEVvs7RE8eeHd+i7R38oSmnVp7Tm
LS1jlPKyPxCckXaLzG/119VqQkEhqbMR8VIZ5rjH17go8a4kCs5xyYiLNfRI2p7WfUv+aCR6MEJZ
LyosSmOKu0lLgnJ42idnhzdO12xxFrZZhK1Xi7Czi0XYZpm082XYq2QZtmykl8savTxfhMWrqTg/
lizCSL0rfY1G+a7f48Z95PykXUZdXXBjS4gV9dX0DS2uJYwIsKpwnaGcph3zNetiRVtMKD82hfyY
p3e6c2qlmFHIzvUV5bR1l01Ukpr1eVenfUPaqmCsoDVq8u1EnhfLm0VY+mYRZmvmFMam0qKC9Yxj
3jF0TVIYIljA6EVGbsTYEXzI3A5ontsQBw8KBdo8Zcf4JTsmz7bGRizq0HEOmmYNIZaTU5ZUCJ1n
BNW0rXDpMTzdooSymxpXRervVpTiirTYtLpvi6ZvKCt4YQaifKpDcVI1/TUui7FjesE4mGh90irj
RVlOsJbM+mb0DzOHQFt7wlFe0/kIRN+E1lqhN6EoNP65nXA2Jf/bi/U4oyysL+r+DaXVROhE2EC5
Qi1h45IHw+N91hbXg15GYYZSC0yYx9bTf0PhjtM+LSkjU9tIDyS96llXhfyRwvQOnXx6/9U7dLJo
h04W7dDJiR06OtJdn4LKQB954zFgtJLIscj4IThaiVT0mmhRHiuKlBubmc0cMeqXc9g7Bi5H5lJN
UXNp5SeposJ7wro2xynR41RTBGYvyj0vqqLe+4Y/ClMd8pK/xiyKTx8WnNzAKjyGMJcq5vuatKzw
OUeHYt0uTOqB/H7QOo39Hx60IrUnnphWxc2xzSJsOq0BbDqtAWw6rQFsOq0BbDqtIWzZSKfTGsLO
F2HTaQ1hiQ8z23yRxSedg6HWi6izU5TBtAt5u3o3pW1hhor91HBMIW2By76mM3HO0WigpuL8VLKI
Wi+iznyUiEpw9rpjcOKALTa0fUT7lnZNPDnO2T81mxJLlmHrZdjZMmyzDDtfhr1ahl08ikXyLCt2
8bwTZpQWLSwI20akfv1UPKEiOKAU5FFhfmom7Cv2+jL9lUmakqf/6ySNPCj19GruuHy5nDy3UD1l
MgJXYQeIY31JUzfMnXOJN1JXClQxs5HnCdQl5lKeON1DiYDiccoTzM875h2AwkyoIcMwdY5s4fwI
nqVmMqBX6nAwFfgYwNaagzFOGtpxh/RgnvzBPIGgWTt5MHIzLPWeV2bYVHcBbKo73TnStrSFyC+n
vmkfOAvzzJUXm5vH2Co7Fjw9SNA7+WZ/drF+Fod7MJwHWwUJghkXKML1vivxkJsYsiATDFZlIxUq
6g0W7W9a2t/gYzBdtlJII3oX2uzFipeJiqAkCwlJMvF5S2Ab9eYX3AxDVrBmCQcxWAhzOfz9Y5zO
MyhbdBIHNuhQcn09Th0we1zWJMFgUfHLC4SvicdqbOo1Az/ATmRGJdXV0BwYTO/tFBrOMzSFeQKD
2gei/RX6+DESnnOSwQoI64Zh5lik9jLPGnGwmsw5D6ZCcZA5OxF7MCFzewqT4xGsFdh7MNwCAuqR
W//WxiJ/6sTWyJJEixx8hdt9EYrho/wRwuRiYHkEUzZwxoHHJzIxEoGFI5FgYDEiSRCB5cxxnfoM
TiPox49ay29oHeJWEd29JimHoe/rIr/xO5rrYl8TzmESVydSICMVGNmECgxuQgUCK3MGOG0cCmvo
5hEbUlQcGJ1DnXDsinqxkr9IHB5OCnTI+OQXGJtMFpPrxWQcPGzPWl9MrheReAeHlLAjFuYHSB1M
0Snk62KGZWnjSdZ4ScjwG2SNv24kv2c6T2K/Zzp/40zn/3lSSZ7nw+k1jTnUPL3mo6ZdnzTZkpTC
9sp9zY7CbMqrMYXVMNaQc9DB/+BaPHG9mskpNovc/Ngssg9I83/Mn2L19JTkx2YZAD+2E6dIMjuP
TrgEcVLKLNT19oS4BMEhbwlWkfSAkZWK8WPt9FQUxpJl2PpxTEUTMp3i7DH2VQ6vmVjiXMw1kyDm
mklYWsgwXcwXs3gw10yCmGsmIY1MzSQgbmomQcw1kyDmmslJLFmGrR/HXDNxrvHAZhBIETkXbwTm
z/3MME9u0CvNm3CaYTMP4sd8SVAPNvMgNhaxrmloy02qkQwfrYlrSxXhB5pZmSQIG0v7w7biZlie
sn6H06sSs8P2NKaTxNeir252beT8KUIP5mbYAtg0wxbAZOdmDucE1kv0cWxPtyellUU1XIcJYV2d
2mkVk2CYTWevrroYbuVBJiay5FLTkGc8fa3JujwkU2wmbdcLf2Th05TdIs6WJxzXEnmnOJOm8Fxc
GrNxIzW/uGRRJnFyWtZABWQpTDo5mZEWN5Fg5hgtOz3383TcdIj2OF1wolsHlF5UjDWUvrMkdkor
gbyMAacWrD31FblxHY6OOfsjdm+EOQIlFTP+CIXzMGBpeWmU+u3uhpM/RMNnFbm36S+LuipRVbFV
tdYUGavO5i9uplXyQ8RWN/3truCi4Rhh0mvnu5JlUmW6QpXT0lx7VOUdKemRjc/xa/d5k7vlvHHL
5XCPchU9n/+ASazU9EqWRZ/MO6KcZ7ZMmPjKLedmCCiWZZg2a4yJWB1pId2QKle47pRXCvVpDTtB
pveMlS4LvW51G7KsP1aYcqb7ZMrmeKPeJ84Y1kh+F5HuXpXvbz/f3v9y+34b7NMZag6UUzk0P6MW
ExLrXY55SIWIXK3cdZGImcwOk4PfgMmVdWjYdnilKsTPVHbMUyk23BI35nXedvIDk695U6vfl6j2
+aAoS2YkziY+CWO9pGvwFVH0VjYtPNuSj97WLypvP30uMmuFficXylrFnkgFlSR/zuKL55vNx4/o
Ty/XLzbPNy/ORX+fsg9/v92iZIN2f3u4fRYdP7y/Rfndjz/8BHL//PAXkBZfiA8Y/Pan6YPNRjyI
3uZFWyEdMH9N76O3KwT/vIPg9i2M4FcMHmSIV+N3Ecpo97r45ptvbgiLQKew7qBQ00i6Vmz9XRqo
wmlL9QNGc16avw9Fzod6Titi3shZCnOZqZJo+BU0fGBCz5qvybGRQiWPW4J3jSkVBpSlRvRlaLE7
kNo8gW0phQWrHuEdk38PTV5Ak+R72AQzw6vcwZ1pc8f6daIFy0fsOPaNWpLiNYjiey6+RYpeCgbc
j1hHqqD266HYHNu+rHifpU7DWOhHDVDw4hCly1fx5WrXgEUO5fNLqyz7cAZ90MmdjdhSVFNcnITV
EkWDHtRlUlXs6hK2qZoyrjsOKiGwU2Ywcf9q/vmfp7zbkWeqic071bWXOH8JFp/WwKzUo/NBmfEr
3Q4m2GgvZ0zeUVIl6eXGohi8OjaKmZM4bBdX+lVxxTZ+eYbyf/xbVVwlw9ihKBsXxgNbWNqm3NK3
WAWiCF5DhSUkA1OVF2x4i2smqobpEH5LvXt5fujhDFCOdi40qDsG67PvmrFlYUP6FobVcnzekhJz
GJJStHigDotaKr/WV8trqiqypjH9vXtA7MvPP9/dP9y+f6FauYRW0qImFe5T2AXF/6aiuqMO/ll7
ra2f05aMxavk4lIuGtkC7H+mIIQmq2HG1pZV98loO0W1F9+8+vSwH6amlKfeEt8MNXLEOUTK9Cib
dux97IxSTHIx1nxwuiM8z65jNyq8ENPRy0WuxwkRR5OZNqMnTxDE5zVJub6DwAl68iT6L1BLAwQU
AAIACAAAuj5JBJp5OCYLAADCNgAAFgAAAExlbnMgaW5mbyBTaWdtYSAxOC0yNTDtm1+P3LYRwJ+j
T8FH+yHxSnt3vlskBbgStSufJKoidZuzYQhBvGgPiO3gfA7avvYTtB+pz/1QHf6TSInclYOgQICs
geBI/TQkh8MhZ8R8/uHx6fGbnz7/sEGfnuBv1OGWo8fju4fH449PDx8/RPnjA2LHn9F6hZL1Jr7Y
rGOUrOIrl0T8+OkJPfssxD19ehISnyN6G0XRmywtUfzp6S367uwPRSmt+pTWvKVljFJe9nuCM9Ju
kPmt/rZaTSgoJHU2Il4qwxz3+A4XJd6WRME5LhlxsYYeSNvTum/Jn41ED0Yo60WFRWlMcfdpSVAO
T/vkYv/a6ZotzsIuF2Hr1SLs4noRdrlM2tUy7GWyDFs20ptljd5cLcLi1VScH0sWYaTelr5Go3zb
73DjPnJ+0i6jri64sSXEivp2+oYW1xJGBFhVuM5QTtOO+Zp1saItJpQfm0J+zNM73Tm1UswoZOf6
inLaussmKknN+ryr074hbVUwVtAaNflmIs+L5c0iLH29CLM1cwpjU2lRwXrGMe8YuiMpDBEsYPQi
Izdi7AA+ZG4HNM9tiIMHhQJtnrFD/IIdkucbYyMWte84B02zhhDLySlLKoTOM4Jq2la49BieblFC
2X2NqyL1dytKcUVabFrdtUXTN5QVvDADUT7VoTipmv4Ol8XYMb1gHEy0PmmV8aIsJ1hLZn0z+oeZ
Q6CtHeEor+l8BKJvQmut0JtQFBr/3Ew4m5L/7cV6nFEW1hd1/5rSaiJ0ImygXKGWsHHJg+HxPmuL
u0EvozBDqQUmzGPj6b+hcMdpn5aUkaltpHuS3vasq0L+SGF6h04+vPviHTpZtEMni3bo5MQOHR3o
tk9BZaCPvPEYMFpJ5FBkfB8crUQqeke0KI8VRcqNzcxmjhj1yznsHQOXI3Oppqi5tPKTVFHhHWFd
m+OU6HGqKQKzF+WeF1VR73zDH4WpDnnJX2MWxYeHBSc3sAqPIcylivm+Iy0rfM7RoVi3DZN6IH8c
tE5jv8ODVqT2xBPTqrg5drkIm05rAJtOawCbTmsAm05rAJtOawhbNtLptIawq0XYdFpDWOLDzDZf
ZPFJ52Co9SLq4hRlMO1C3qzeTmlbmKFiPzUcU0hb4LKv6UycczQaqKk4P5UsotaLqAsfJaISnL3q
GJw4YIsNbR/RrqVdE0+Oc/ZPzabEkmXYehl2sQy7XIZdLcNeLsOuz2KRPMuKXTzvhBmlRQsLwrYR
qV8/FU+oCA4oBTkrzE/NhH3BXl+mvzJJU/L0/52kkQelnt7OHZcvl5PnFqqnTEbgKuwAcawvaeqG
uXMu8UbqSoEqZjbyPIG6xFzKE6d7KBFQnKc8wfy8Y94BKMyEGjIMU+fIFs6P4FlqJgN6pQ4HU4GP
AWytORjjpKEdd0gP5skfzBMImrWTByM3w1LveWWGTXUXwKa6050jbUtbiPxy6pv2gbMwz1x5sbl5
jK2yQ8HTvQS9k2/2ZxfrZ3G4B8N5sFWQIJhxgSJc77oSD7mJIQsywWBVNlKhot5g0e6+pf09PgTT
ZSuFNKJ3oc1erHiZqAhKspCQJBOftwS2UW9+wc0wZAVrlnAQg4Uwl8Pfn+N0nkHZopM4sEGHkuvr
PLXH7LysSYLBouIX1wjfEY/V2NQrBn6AnciMSqqroTkwmN7bKTScZ2gK8wQGtQtE+yv0/n0kPOck
gxUQ1g3DzLFI7WWeNeJgNZlzHkyF4iBzdiL2YELm5hQmxyNYK7D3YLgFBNQjt/6NjUX+1ImtkSWJ
Fjn4Cre7IhTDR/kZwuRiYHkEUzZwxoHHJzIxEoGFI5FgYDEiSRCB5cxxnfoMTiPox/day69pHeJW
Ed2+IimHoe/qIr/3O5q7YlcTzmESVydSICMVGNmECgxuQgUCK3MGOG0cCmvo5RkbUlQcGJ1DnXDs
ivpmJX+RODycFOiQ8ckvMDaZLCbXi8k4eNietb6YXC8i8RYOKWFHLMwPkDqYolPIl8UMy9LGk6zx
kpDhN8gaf9lI/sh0nsT+yHT+xpnO33lSSZ7nw+k1jTnUPL3mo6ZdnzTZkpTC9sp9zY7CbMqrMYXV
MNaQc9DB/+BaPHG9mskpNovc/Ngssg9I83/Mn2L19JTkx2YZAD+2FadIMjuPTrgEcVLKLNTd5oS4
BMEhbwlWkXSPkZWK8WPt9FQUxpJl2Po8pqIJmU5x9hj7KofXTCxxLuaaSRBzzSQsLWSYLuaLWTyY
ayZBzDWTkEamZhIQNzWTIOaaSRBzzeQklizD1ucx10ycazywGQRSRM7FG4H5cz8zzJMb9ErzJpxm
2MyD+DFfEtSDzTyIjUWsaxracpNqJMNHa+LaUkX4nmZWJgnCxtL+sK24GZanrN/i9LbEbL85jekk
8Z3oq5tdGzl/itCDuRm2ADbNsAUw2bmZwzmB9RI9j+3o5qS0sqiG6zAhrKtTO61iEgyz6ezVVRfD
rTzIxESWXGoa8oynrzVZl4dkis2k7Xrhjyx8mrJbxNnyhONaIu8UZ9IUnotLYzZupOYXlyzKJE5O
yxqogCyFSScnM9LiJhLMHKNlp+d+no6bDtEepwtOdOuA0ouKsYbSd5bETmklkJcx4NSCtae+Jfeu
w9ExZ3/A7o0wR6CkYsbPUDgPA5aWl0ap327vOflTNHxWkXub/rKoq5J51VpXkbHqYk5dTqvkh4iN
bvrbbcFFwzHCpNfOdyXLpMp0hSqnpbn2qMpbUtIDG5/jV+7zJnfLeeOWy+Ee5Sr6ev4DJrFS0ytZ
Fn0y74hyntkyYeIrt5xn9pgScM9uGVZHWkg3pMoVrjvllUJ9WsNOkOk9Y6XLQq+iIjZl/bHClDPd
J1M2xxv1PnHGsEbyu4h096r8ePx0fPzl+G4T7NMFavaUUzk0P6MWExLrXY55SIWIXK3cdZGImcwO
k4PfgMmVdWjYdnilKsTPVHbMqeRtRyKx35a4MW/LOnDovtZNrf26dvmgJ6udSBxNfBLGeknX4Cqi
6I1sWji2Jd+8rV9UHj98KjJrgX4n18n1jYwpkYop4+uvk8vV+/foGXv4x3GDbtD270/H59Hh4d0R
5R9//OEnEPOXp7/Cy/G1+FzBjz9NH4AA8SR6kxdthXR8/CW9jd6sEPx7C7HsG+jxrxgsyBCvxm8j
lNHuVfHVV1/dExaBDmGZQaGmkfSk2Pq7NFCF05bqB4zmvDR/74ucD/WcVsS8kbMU5i5TJdHwS2h4
z4ReNV+TQzMIxS3B20YXCoNJSY3oydBetye1eQJ7UAqrU0vYMvn30OA1NEi+hx3PjE7lCT4OeL9O
7CfsMPZLd0sqbA1y+I6Lr46ihwIBRyNWjO6t3JmHYnNo+7LifZbarWKhmYEWhyVVvI1vVtsGLM8U
r27Gomz+AprXGZxLsW8okIvjrlqHaByQPIqpYleXsBfVlHHdZ1AFge0wg+n6d/Ov/z7j3ZY8V01c
vlX9eoHzF7AO0xqYlXp0NSgxfqnbwQQbveWMyYtIqiRd2VgUA1dnQzFjEoc94Va/Ku7Rxi8uUP7P
/+ihJ7OhC5OBfSptU24pT9i+KIJvULEHycBA5S0a3uKaiaphJoR3Uu/eXO172OjL0bqFBnXHYFX2
XTO2LGxHX7WwWo6vWlJiDkNSihYP1IlQS+V3+v54TVVF1jSmvx+fEPv8888fH5+O775RrdxAK2lR
kwr3KWx14v9FUd1Rp/usvdNWz2lLxuJtcn0zrJUMNjlTEEKT1TBja8ue+2S0naLaiQ9bfbrfDVNT
yqNtie+HGjniHMJhepBNO6Y+dkYpJrkeax6c7gh/s+3YvYohxHT0cnHrcUJY0WSmzf8BUEsBAhQA
FAACAAgAirg+ScqGfQLaCwAA+jYAABMAAAAAAAAAAAAgAAAAAAAAAExlbnMgaW5mbyBFRi1NLTIy
LTJQSwECFAAUAAIACABArHVJGjRydVULAAD1NgAAFwAAAAAAAAAAACAAAAALDAAATGVucyBpbmZv
IEVGLVMtMTgtNTUtSVNQSwECFAAUAAIACAAAuj5JBJp5OCYLAADCNgAAFgAAAAAAAAAAACAAAACV
FwAATGVucyBpbmZvIFNpZ21hIDE4LTI1MFBLBQYAAAAAAwADAMoAAADvIgAAAAA=


But attachments in this forum still don't work for me.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 27, 2016, 07:49:53 PM
I have no idea how you produced this or if it has anything to do with this topic for non cpu lenses but it is interesting.

Here you can find parameters taken from 3 lenses:
...
But attachments in this forum still don't work for me.

You don't need to attach text files but there is a 20,000 character limit per post and Tapatalk users will probably hate this:

EF-M22mm f/2 STM
Code: [Select]
uartr.lua: start UART redirection
Fri Sep 30 22:21:56 2016 UART redirect Test (uartrtst.lua) OK

!! RESET COMPLETE !!


[DCL 1st] ==================================
COM_CONTROL1 CTL_HEADER:         0x02
COM_CONTROL1 COM_2ND:            0x00
COM_CONTROL1 DATA_AVAILABLE:     TRUE
COM_CONTROL1 POWER_ON_REQ:       FALSE
COM_CONTROL1 EOS_COM_REQ:        TRUE

COM_CYCLE FREQ_24HZ:             FALSE
COM_CYCLE FREQ_25HZ:             FALSE
COM_CYCLE FREQ_30HZ:             FALSE
COM_CYCLE FREQ_48HZ:             FALSE
COM_CYCLE FREQ_50HZ:             FALSE
COM_CYCLE FREQ_60HZ:             TRUE
COM_CYCLE FREQ_72HZ:             FALSE
COM_CYCLE FREQ_75HZ:             FALSE
COM_CYCLE FREQ_90HZ:             FALSE
COM_CYCLE FREQ_96HZ:             FALSE
COM_CYCLE FREQ_100HZ:            FALSE
COM_CYCLE FREQ_120HZ:            FALSE
COM_CYCLE FREQ_ENBL:             TRUE

FB_GAP:                          0x00

UNIT_CONTROL SINK:               TRUE

RESET_COMMAND FOCUS:             FALSE
RESET_COMMAND IRIS:              FALSE
RESET_COMMAND IS:                FALSE
RESET_COMMAND SINK:              FALSE

POWER_CONTROL FOCUS_MOTOR:       FALSE

LENS_FUNC_PERMISSION PF:         FALSE
LENS_FUNC_PERMISSION FP:         FALSE
LENS_FUNC_PERMISSION CZ:         TRUE
LENS_FUNC_PERMISSION IRIS:       TRUE
LENS_FUNC_PERMISSION IS:         TRUE

IS_STATUS VECTOR_ENABLE:         FALSE
IS_STATUS SW_ON:                 ON
IS_STATUS START_STOP(SW1/SW2):   0x00
IS_STATUS SHUTTER_SPEED:         0x09

IS_MODE NORMAL:                  ON
IS_MODE DYNAMIC:                 ON

CAMERA_STATUS GRIP_POSITION:     0x00
CAMERA_STATUS TEMP_VALID:        FALSE
CAMERA_STATUS MODE:              STILL
CAMERA_STATUS REC:               FALSE

IRIS TARGET FNO:                 0x0074
IRIS_SPEED IRIS_SPEED:           0x00
IRIS_SPEED SPEED_UNIT:           0x00

IRIS_SPEED_IN_ZOOM IRIS_SPEED:   0x00
IRIS_SPEED_IN_ZOOM SPEED_UNIT:   0x00

IRIS_CONTROL SILENT_DRIVE:       0x00
IRIS_CONTROL POWER_STOP:         0x00
IRIS_CONTROL AUTO_CLOSE:         0x00

CHECK_SUM:                       0xB4



[DCL 2nd] ==================================
COM_CONTROL2 CTL_HEADER:         0x00
COM_CONTROL2 COM_2ND:            0x00
COM_CONTROL2 DATA_AVAILABLE:     FALSE

WOB_CENTER_FPC:                  0
WOB_WIDTH:                       0
WOB_MOVE_CENTER:                 0
FOCUS_SPEED:                     0
FOCUS_CONTROL DRIVE_MODE:        0x00
FOCUS_CONTROL PINT_STATE:        0x00
FOCUS_CONTROL IMAGESURFACE_FPC:  0x00

GET_FPC_TIMING:                  0x00
FOCUS_DRIVE_TIMING:              0x00

CHECK_SUM:                       0x00



[DCL Init] ================================
CTL_HEADER:                      0x01
COM_VERSION:                     0x01
COM_SUB_VERSION:                 0x00

COM_CYCLE FREQ_24HZ:             TRUE
COM_CYCLE FREQ_25HZ:             TRUE
COM_CYCLE FREQ_30HZ:             TRUE
COM_CYCLE FREQ_48HZ:             FALSE
COM_CYCLE FREQ_50HZ:             TRUE
COM_CYCLE FREQ_60HZ:             TRUE
COM_CYCLE FREQ_72HZ:             FALSE
COM_CYCLE FREQ_75HZ:             FALSE
COM_CYCLE FREQ_90HZ:             FALSE
COM_CYCLE FREQ_96HZ:             FALSE
COM_CYCLE FREQ_100HZ:            FALSE
COM_CYCLE FREQ_120HZ:            FALSE

START_CYCLE FREQ_24HZ:           FALSE
START_CYCLE FREQ_25HZ:           FALSE
START_CYCLE FREQ_30HZ:           FALSE
START_CYCLE FREQ_48HZ:           FALSE
START_CYCLE FREQ_50HZ:           FALSE
START_CYCLE FREQ_60HZ:           TRUE
START_CYCLE FREQ_72HZ:           FALSE
START_CYCLE FREQ_75HZ:           FALSE
START_CYCLE FREQ_90HZ:           FALSE
START_CYCLE FREQ_96HZ:           FALSE
START_CYCLE FREQ_100HZ:          FALSE
START_CYCLE FREQ_120HZ:          FALSE

CAMERA_ID1:                      0xA8
CAMERA_ID3:                      0x00
CAMERA_ID4:                      0x83

CAMERA_VERSION[0]:               0x00
CAMERA_VERSION[1]:               0x00

CAMERA_SERIAL_NO[0]:             0x00
CAMERA_SERIAL_NO[1]:             0x00
CAMERA_SERIAL_NO[2]:             0x00
CAMERA_SERIAL_NO[3]:             0x00
CAMERA_SERIAL_NO[4]:             0x00

FB_ADJUST_DATA:                  0x04

GROUP1_VALID:                    FALSE
GROUP2_VALID:                    FALSE
GROUP3_VALID:                    FALSE
GROUP4_VALID:                    FALSE
GROUP5_VALID:                    FALSE
GROUP6_VALID:                    FALSE
GROUP7_VALID:                    FALSE
GROUP8_VALID:                    FALSE

STILL_CONFUSIONCIRCLE[0]:        0x07
STILL_CONFUSIONCIRCLE[1]:        0xD0

MOVIE_CONFUSIONCIRCLE[0]:        0x0A
MOVIE_CONFUSIONCIRCLE[1]:        0x00

CHECK_SUM:                       0x96



[DLC 1st] ==================================
COM_CONTROL1 LTC_HEADER:         0x02
COM_CONTROL1 COM_2ND:            0x00
COM_CONTROL1 DATA_AVAILABLE:     TRUE
COM_CONTROL1 IMAGE_OK:           TRUE
COM_CONTROL1 POWER_OFF_OK:       FALSE

UNIT_STATUS1 IS_LOCK:            FALSE

UNIT_STATUS2 SINK:               0x02

RESET_STATUS1 FOCUS:             0x02
RESET_STATUS1 IRIS:              0x02
RESET_STATUS1 ZOOM:              0x00
RESET_STATUS1 IS:                0x00

RESET_STATUS2 SINK:              0x02

POWER_STATUS FOCUS_RING_SENSOR:  TRUE
POWER_STATUS IRIS_SENSOR:        TRUE
POWER_STATUS STEPOUT_SENSOR:     FALSE
POWER_STATUS FOCUS_MOTOR:        FALSE

LENS_FUNC_STATUS PF:             FALSE
LENS_FUNC_STATUS CZ:             FALSE
LENS_FUNC_STATUS IS:             FALSE
LENS_FUNC_STATUS SINK:           FALSE

ERROR_INFO FOCUS:                FALSE
ERROR_INFO IS:                   FALSE
ERROR_INFO IRIS:                 FALSE

SWITCH_INFO1 IS:                 FALSE
SWITCH_INFO1 IS_MODE:            FALSE
SWITCH_INFO1 AF:                 FALSE

IS_INFO_AVAILABLE ANGULAR_SPEED: FALSE
IS_INFO_AVAILABLE COMPLENS_INFO: FALSE

GYRO_YAW:                        0
GYRO_PITCH:                      0
COMP_POS_YAW:                    0
COMP_POS_PITCH:                  0

IRIS_REAL_FNO:                   0x0046
IRIS_DISP_FNO:                   0x0040
IRIS_MIN_FNO:                    0x0046
IRIS_MAX_FNO:                    0x0120

IRIS_STATUS DRIVE:               0x00
IRIS_STATUS POWER:               0x00
IRIS_STATUS PHASE:               0x00
IRIS_STATUS CLOSE:               0x00
1/8 AVEF:                        0x19
JSSTESP:                         0x06
UNSTABLE_PHASE:                  0x00

FOCAL_LENGTH:                    22 mm
ZOOM_POSITION:                   0x00

FOCUS_STATUS FAR_END:            FALSE
FOCUS_STATUS NEAR_END:           FALSE
FOCUS_STATUS DRIVE_FAR:          FALSE
FOCUS_STATUS DRIVE_NEAR:         TRUE
FOCUS_STATUS ZOOM_DRIVING:       FALSE
FOCUS_STATUS ARRIVAL_CHECK:      TRUE

FPC:                             9380
WOB_CENTER_FPC:                  9380
NEAR_MARGIN:                     0
FAR_MARGIN:                      0
WOB_MIN_WIDTH:                   2147483647
MIN_SPEED:                       2147483647
MAX_SPEED1:                      2147483647
MAX_SPEED2:                      2147483647
DISTANCE:                        65535 cm
FOCUS_ZONE:                      15
OBJECT_MAGNIFY:                  2

VIGNETTING0:                     0x00
VIGNETTING1:                     0x04
VIGNETTING2:                     0x0B
VIGNETTING3:                     0x0F

PO:                              0xD0
PO5:                             0xE3
PO10:                            0xEE
PITCH:                           0.000000
SENS0:                           0.000000
SENS1P:                          0.000000
SENS2P:                          0.000000
SENS3P:                          0.000000
SENS1M:                          0.000000
SENS2M:                          0.000000
SENS3M:                          0.000000
ABSINF:                          48
ABSNER:                          43
CHECK_SUM:                       0x09



[DLC 2nd] =================================
COM_CONTROL2 LTC_HEADER:         0x02
COM_CONTROL2 COM_2ND:            0x01
COM_CONTROL2 DATA_AVAILABLE:     TRUE

CHECK_SUM:                       0xF1



[DLC Init] ================================
CTL_HEADER:                      0x01
COM_VERSION:                     0x01
COM_SUB_VERSION:                 0x00

COM_CYCLE FREQ_24HZ:             TRUE
COM_CYCLE FREQ_25HZ:             TRUE
COM_CYCLE FREQ_30HZ:             TRUE
COM_CYCLE FREQ_48HZ:             FALSE
COM_CYCLE FREQ_50HZ:             TRUE
COM_CYCLE FREQ_60HZ:             TRUE
COM_CYCLE FREQ_72HZ:             FALSE
COM_CYCLE FREQ_75HZ:             FALSE
COM_CYCLE FREQ_90HZ:             FALSE
COM_CYCLE FREQ_96HZ:             FALSE
COM_CYCLE FREQ_100HZ:            FALSE
COM_CYCLE FREQ_120HZ:            FALSE

CAMERA_SERIAL_NO[0]:             0x00
CAMERA_SERIAL_NO[1]:             0x00
CAMERA_SERIAL_NO[2]:             0x00
CAMERA_SERIAL_NO[3]:             0x00
CAMERA_SERIAL_NO[4]:             0x00

LENS_VERSION[0]:                 0x01
LENS_VERSION[1]:                 0x08
LENS_VERSION[2]:                 0x00

LENS_RECOGNITION[0]:             0x31
LENS_RECOGNITION[1]:             0x10

LENS_NAME:                       EF-M22mm f/2 STM

UNIT_AVAILABLE1 FOCUS:           TRUE
UNIT_AVAILABLE1 IRIS:            TRUE
UNIT_AVAILABLE1 ZOOM:            FALSE
UNIT_AVAILABLE1 IS:              FALSE
UNIT_AVAILABLE1 ND:              FALSE
UNIT_AVAILABLE1 SINK:            TRUE
UNIT_AVAILABLE1 BARRIER:         FALSE

UNIT_AVAILABLE2 TELE_CONV:       FALSE
UNIT_AVAILABLE2 WIDE_CONV:       FALSE
UNIT_AVAILABLE2 MECHA LOCK:      FALSE
UNIT_AVAILABLE2 RING1:           TRUE
UNIT_AVAILABLE2 RING2:           FALSE
UNIT_AVAILABLE2 RING3:           FALSE
UNIT_AVAILABLE2 ZOOM_PRESET:     FALSE

UNIT_CONTROLLABLE1 FOCUS:        TRUE
UNIT_CONTROLLABLE1 IRIS:         TRUE
UNIT_CONTROLLABLE1 ZOOM:         FALSE
UNIT_CONTROLLABLE1 IS:           FALSE
UNIT_CONTROLLABLE1 ND:           FALSE
UNIT_CONTROLLABLE1 SINK:         TRUE
UNIT_CONTROLLABLE1 BARRIER:      FALSE

UNIT_CONTROLLABLE2 TELE_CONV:    FALSE
UNIT_CONTROLLABLE2 WIDE_CONV:    FALSE
UNIT_CONTROLLABLE2 MECHA LOCK:   FALSE
UNIT_CONTROLLABLE2 RING1:        FALSE
UNIT_CONTROLLABLE2 RING2:        FALSE
UNIT_CONTROLLABLE2 RING3:        FALSE
UNIT_CONTROLLABLE2 ZOOM_PRESET:  FALSE

RESET_REQ FOCUS:                 TRUE
RESET_REQ IRIS:                  TRUE
RESET_REQ ZOOM:                  FALSE
RESET_REQ IS:                    FALSE
RESET_REQ ND:                    FALSE
RESET_REQ SINK:                  FALSE
RESET_REQ BARRIER:               FALSE

SUPPORT FOCUS_REDRIVE_TIME:      FALSE

METHOD_AVAILABLE WOBLING:        FALSE
METHOD_AVAILABLE FCS_BACKLASH:   TRUE
METHOD_AVAILABLE FCS_POWER_OVER: TRUE

SWITCH_AVAILABLE IS:             FALSE
SWITCH_AVAILABLE IS_MODE:        FALSE
SWITCH_AVAILABLE AF:             FALSE
SWITCH_AVAILABLE FCS_PRESET:     FALSE
SWITCH_AVAILABLE FCS_PRESET_SET: FALSE
SWITCH_AVAILABLE FCS_PRESET_GO:  FALSE
SWITCH_AVAILABLE FCS_LIMIT:      FALSE
SWITCH_AVAILABLE FUNCTION:       FALSE

FOCUS_REDRIVE_TIME_SILENT:       0
FOCUS_REDRIVE_TIME:              0

IS_MODE NORMAL:                  FALSE
IS_MODE DYNAMIC:                 FALSE

IRIS_MAX_DISP_FNO_TELE:          0x0120
IRIS_MIN_DISP_FNO_TELE:          0x00A0
IRIS_MAX_DISP_FNO_WIDE:          0x0120
IRIS_MIN_DISP_FNO_WIDE:          0x0140
IRIS_MAX_SPEED IRIS_SPEED:       0x06
IRIS_MAX_SPEED SPEED_UNIT:       0x01
IRIS_MIN_SPEED IRIS_SPEED:       0x04
IRIS_MIN_SPEED SPEED_UNIT:       0x00

IRIS_UNIT_INFO STOP_RESOLUTION:  0x02

FOCAL_LENGTH_WIDE:               0x0016

FOCAL_LENGTH_TELE:               0x0016

ZOOM_MAX_POSITION:               0x0000

FOCUS_MAX_ZONE:                  0x001F

SWITCH_AVAILABLE2 ZOOM_KEY:      FALSE

COM_2ND_WAIT:                    0x06
COM_1ST_WAIT:                    0x0A
AF_WAIT:                         0x06

CHECK_SUM:                       0x72


<BYTE>
STATUS ST1:   0x00
STATUS ST2:   0x00
STATUS ST3:   0x00
STATUS ST4:   0x00
STATUS ST5:   0x00
STATUS ERROR: 0x00

<BIT>
ST1 AE_NG:    0
ST1 EMD_NG:   0
ST1 CL_ON:    0
ST1 BELOWS:   0
ST1 AJ_ON:    0
ST1 PF_ON:    0
ST1 FP_ON:    0
ST1 LSW_ON:   0
-------------------
ST2 CLOSE:    0
ST2 EMD_ON:   0
ST2 FD_ON:    0
ST2 ZM_ON:    0
ST2 FD_NG:    0
ST2 AF_NG:    0
ST2 SPECIAL:  0
ST2 MANUAL:   0
-------------------
ST3 ISDIS:    0
ST3 ISSTAT:   0
ST3 ISSW:     0
ST3 ISDON:    0
ST3 ISLOCK:   0
ST3 ED_ON:    0
ST3 GYRO_SET: 0
ST3 reserved: 0
-------------------
ST4 PHOTO_NG: 0
-------------------

FOCUS UNIT SPEC ========
WOBBLING LENS: FALSE
FD SLOW LENS : TRUE
STM LENS     : TRUE
USM LENS     : FALSE
OVERLAP LENS : TRUE

IS UNIT SPEC ========
IS UNIT      : FALSE
DYNAMIC IS   : FALSE

SINK UNIT SPEC ========
SINK UNIT    : CTRL

[LENS INFO] =============================================
LensID           = 0x1031
LENS NAME: EF-M22mm f/2 STM (Size: 17 Byte)
Wide FocalLength = 22 mm
Tele FocalLength = 22 mm

[FIRM VERSION] ==========================================
[1 8 0]

[ID INFO] ===============================================

[ID1]
 DOUJI YES
 NOEMD NO
 ZOOMA NO
 ZOOML NO
 MACRO NO
 SOFTL NO
 SHIFT NO
 SOTOME NO
 FSC SPD NO

[ID7]
 HSLENS NO
 NEWPRO YES
 AREABP NO
 ISLENS NO
 PZOOM NO
 SHUHEN NO
 CALCDIS YES
 ABSDIS YES

[ID8]
 EXT_ID YES
 LENS_No YES
 ABS_32 YES
 LENS_SW NO
 NEWPO YES

[ID13]
 TGTPOS_BP NO
 FD_SLOW YES
 IRIS_SLOW YES
 PWR_LMT_DC YES
 LENS_NAME YES
 IRIS_TIME YES
 K190BP    NO
 K169BP    NO

[ID14]
 SERIAL_5BYTE NO
 TS_POS      NO
 ABS_DRIVE NO
 UNLCK_NOSTP NO
 EXTEND_DT íPëÃ(TUBE)

[ID15]
 IRIS/AF/IS CNT 0

[ID16]
 EXT_ID17 YES
 AAEA2 YES
 FSS_DATA NO
 GYRO_DATA NO
 LENSDRIVE_EN YES
 FP_OK YES
 PINT1/4 Fɬ NO
 K269BP    NO

[ID17]
 AE_CRCT NO
 IRIS_INFO 1/16
 MOVIE_TRANSPORT YES
 PWR_OVER YES
 96H_ST_L YES
 ZOOM_POS NO
 FIRM_UP YES

[ID18]
 STATUS2 NO
 IRIS_16RELATIVEDRIVE NO
 DISP_FL NO
 TV_SPEED_NO YES
 DPP_INFO Ver.2nd

[ID19]
 CINEMA_CMD_ENBL NO
 FRESET_DRV NO
 STORE_DRV NO
 K289BP NO
 DAF_BP NO

[ID20]
 EXT_ID3 NO
 NEWPO_2BYTE NO
 IMG_MAG_CHG_EN YES
 L_DISPLAY_EN NO
 IRIS_FOLLOW_DRV NO
 IRIS_SLOW_DRV NO
 IRIS_128_DRV NO
 iBP NO

[ID21]
 BUSY_TIME_OVER_LENS NO
 FREE_SPD_EN NO

Tip for Mac users, save Ant123's code as a plain text file and open it with Stuffit Expander.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Ant123 on November 27, 2016, 08:24:11 PM
You don't need to attach text files
I can't attach any file. There is no "attach" button.

Quote
it has anything to do with this topic for non cpu lenses
maybe it could be helpfull for EF lens exploring...
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 27, 2016, 08:35:15 PM
I can't attach any file. There is no "attach" button.

You have to link to a download URL on this forum, like with videos and photos.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Ant123 on November 27, 2016, 08:43:35 PM
You have to link to a download URL on this forum, like with videos and photos.
It's not an attachment. Why this feature (http://wiki.simplemachines.org/smf/SMF2.0:Attachments_and_Avatars) is not enabled on ML forum?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 27, 2016, 09:29:51 PM
Probably because it would quickly fill up the server with some large files. You can ask g3gg0.

In any case, let's get back on topic. We might want to use the Lens ID code instead of the name. According to your file:

Code: [Select]
LensID           = 0x1031
LENS NAME: EF-M22mm f/2 STM (Size: 17 Byte)

exiftool -- lib/Image/ExifTool/Canon.pm
Code: [Select]
    4145 => 'Canon EF-M 22mm f/2 STM', #34

mlv_dump -v
Code: [Select]
     Name:        'EF-M22mm f/2 STM'
     Serial:      ''
     Focal Len:   22 mm
     Focus Dist:  43 mm
     Aperture:    f/12.80
     IS Mode:     0
     AF Mode:     0
     Lens ID:     0x00000031

@g3gg0
Looks like there might be a problem with the MLV Lens ID. Shouldn't it be 0x00001031 (decimal 4145)?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on November 28, 2016, 12:00:00 AM
Code: [Select]
a) print_msg(MSG_INFO, "     Lens ID:     0x%08X\n", lens_info.lensID);
b) hdr->lensID = lens_info.lens_id;
c) uint32_t    lensID;    /* hexadecimal lens ID (delivered by properties?) */
d) uint16_t                lens_id;
e) lens_info.lens_id = info[4] | (info[5] << 8);

hmm so then it must be the property being interpreted wrong or it doesnt have that id?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on November 28, 2016, 01:16:55 AM
right, at e) seems to be the bug. trying to fix it...
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on November 28, 2016, 02:11:40 AM
can you please check this PR?
https://bitbucket.org/hudson/magic-lantern/pull-requests/775/lens-info-fix/diff
thanks
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 28, 2016, 06:54:12 PM
Lens ID works on EOSM and 700D with that PR. Still no Serial number though.

Code: [Select]
     Name:        'EF-M22mm f/2 STM'
     Serial:      ''
     Lens ID:     0x00001031

Nice additions to the Lens Info/Prefs menu.

(https://c2.staticflickr.com/6/5594/31185535441_44fcb72512.jpg)

Thanks!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on November 28, 2016, 07:51:26 PM
guess that S/N thingy i found on 600D was some false alarm.

do you know the S/N of your lens? maybe you could dump the PROP_LENS content to check if the S/N is in there.
maybe at offset 0x16/17/18 is some information?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 28, 2016, 08:17:49 PM
I know the serial number of my lens but not sure how to dump the PROP_LENS content.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on November 28, 2016, 09:32:29 PM
Code: [Select]
@@ -1104,5 +1114,15 @@
     #endif
     
+    FILE * f = NULL;
+
+    f = FIO_CreateFile("LENS.BIN");
+    if (f)
+    {
+        bmp_printf(FONT_LARGE, 0, 60, "Writing LENS");
+        FIO_WriteFile(f, (void*) info, 0x1E);
+        FIO_CloseFile(f);
+    }
+   
     if (lens_info.raw_aperture < lens_info.raw_aperture_min || lens_info.raw_aperture > lens_info.raw_aperture_max)
     {
         int raw = COERCE(lens_info.raw_aperture, lens_info.raw_aperture_min, lens_info.raw_aperture_max);
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on November 28, 2016, 09:44:58 PM
oh eh, add this to lens.c in function

PROP_HANDLER(PROP_LENS)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on November 28, 2016, 10:51:23 PM
Ok--this is the entire contents of LENS.BIN viewed through a hex editor it looks like this:

Code: [Select]
01185010 31001600 1680439A 7C00FF00 00000000 00004118 33010800 3D19
The serial number of the lens is:

Code: [Select]
950202019971
In hex that would be:

Code: [Select]
DD3C743083

If that the serial number is saved as ASCII it should look like this:

Code: [Select]
39 35 30 32 30 32 30 31 39 39 37 31 0a
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on November 28, 2016, 11:20:38 PM
thanks, so no serial in the property. makes sense as it would be really long
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 05, 2016, 06:40:43 AM
@dmilligan

Is your manual_lens_info (https://bitbucket.org/hudson/magic-lantern/branch/manual_lens_info) branch at a point where we can start trying out some scripts? Seems like lens.name is working which is the one that is needed for the ACR automatic lens distortion correction to work. Are you planning on adding lens.focal_length and have you decided what to do with lens.aperture or camera.aperture? It would be great to have these available to lua along with the Lens ID and serial number but we should be able to get something functional with just lens.name.


Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on December 05, 2016, 07:55:15 PM
Yes, you can try it. All three properties should be working. I implemented aperture override as lens.manual_aperture. In lens.lua, simply add your lens definitions like the ones already there, and you can add a "manual_aperture" field to the lens definition (the existing examples just have name and focal_length).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 06, 2016, 05:42:58 AM
@Lars_Steenhoff -- here's something for you to test:

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-manual_lens_info.2016Dec05.5D3113.zip

This part should be familiar to you by now.
(https://c1.staticflickr.com/6/5584/31414306456_e671f7ce0d.jpg)

When you start up with a non-chipped lens attached or if you remove a chipped lens while the camera is on you'll get this lens selector:
(https://c4.staticflickr.com/6/5666/31451185555_f562851f47.jpg)

These are your Zeiss ZF.2 manual lenses. Pick the one you have mounted and the metadata will be embedded in your MLV files.

Code: [Select]
Block: LENS
  Offset: 0x0000014c
    Size: 96
    Time: 0.003000 ms
     Name:        'Zeiss Planar T* 1.4/85 ZF.2'
     Serial:      ''
     Focal Len:   85 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

@dmilligan -- note that the Aperture isn't showing up. This is what I did in your lua script:
[EDIT] My bad, seems like I was looking at a file before adding the manual_exposure field.

Code: [Select]
lenses =
{
--  Zeiss ZF.2 manual lenses
--    { name = "Zeiss Distagon T* 2.8/15 ZF.2", focal_length = 15, manual_aperture = 2.8 },
--    { name = "Zeiss Distagon T* 3.5/18 ZF.2", focal_length = 18, manual_aperture = 3.5 },
--    { name = "Zeiss Distagon T* 2.8/21 ZF.2", focal_length = 21, manual_aperture = 2.8 },
--    { name = "Zeiss Distagon T* 2.8/25 ZF.2", focal_length = 25, manual_aperture = 2.8 },
    { name = "Zeiss Distagon T* 2/25 ZF.2", focal_length = 25, manual_aperture = 2 },
--    { name = "Zeiss Distagon T* 2/28 ZF.2", focal_length = 28, manual_aperture = 2 },
    { name = "Zeiss Distagon T* 2/35 ZF.2", focal_length = 35, manual_aperture = 2 },
--    { name = "Zeiss Distagon T* 1.4/35 ZF.2", focal_length = 35, manual_aperture = 1.4 },
    { name = "Zeiss Makro-Planar T* 2/50 ZF.2", focal_length = 50, manual_aperture = 2 },
--    { name = "Zeiss Planar T* 1.4/50 ZF.2", focal_length = 50, manual_aperture = 1.4 },
    { name = "Zeiss Planar T* 1.4/85 ZF.2", focal_length = 85, manual_aperture = 1.4 },
--    { name = "Zeiss Makro-Planar T* 2/100 ZF.2", focal_length = 100, manual_aperture = 2 },
--    { name = "Zeiss Apo Sonnar T* 2/135 ZF.2", focal_length = 135, manual_aperture = 2 },
}

These are all the Zeiss ZF.2 lenses. The ones that aren't in Lars Steenhoff's arsenal are commented out to reduce clutter. I'm planning on adding the Samyang (Rokinon, Bower) Cine lenses to the list because they are also popular manual lenses. Other lenses can be added by the user but as we learned, not all lenses are automatically recognized by Adobe Camera Raw. Fortunately the Zeiss ZF.2 lenses are:

(https://c8.staticflickr.com/6/5765/31451263535_c4a33f056f.jpg)

When you switch manual lenses or if you want to check and see which lens is currently selected in the camera go to the wrench icon and pick Lens Info Prefs. You can bring up the lens selector by selecting the lens name.

(https://c1.staticflickr.com/6/5767/31414306496_7c43b249be.jpg)

Some things on my wish list are:
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on December 06, 2016, 10:36:27 PM
Cool stuff @dfort!

Im preparing for a trip, this will come in handy.

And when we get lens ID working i have some hope that nikon lenses will be reconised by id.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Danne on December 06, 2016, 11:19:07 PM
So cool!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 07, 2016, 02:21:01 AM
Yeah, cool if you are using Zeiss ZF.2 lenses.

It is looking pretty solid. I had to go back and edit some of those issues I pointed out earlier because manual aperture and the scrolling display are working.

I just tried it with some Samyang lens profiles and it isn't working in Adobe Camera Raw. I also went back to that Nikon AF NIKKOR 35mm f/2D lens to see if I could get that working but it still doesn't work with ACR, at least not automatically like it should.

Code: [Select]
Block: LENS
  Offset: 0x0000014c
    Size: 96
    Time: 0.003000 ms
     Name:        'Nikon AF NIKKOR 35mm f/2D'
     Serial:      ''
     Focal Len:   35 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

I got an answer from the Adobe forum but the answer came back with an example for a completely different lens and I don't have a deep enough understanding of ACR to decipher what it means.

https://forums.adobe.com/message/9158301
Quote
Hi Daniel,
 
The list of supported lenses generally matches the lens display names and not the specific EXIF value for Lens. Also, for many Zeiss SLR lenses, a generic EXIF name may be substituted inline with a specific product name. The code that handles this doesn't take into account mounting lenses meant for one camera system to another.
 
As far as getting the auto match to work, there are a couple of options.
 
1. Set a new Lens Profile Default in the Lens Corrections panel after manually locating the correct lens profile.
 
OR
 
2. Add a copy of the lens profile(s) to the user profile area and add an alternate lens ID (see below or example) for each applicable section that contains a LensID element.
 
.....
<rdf:Description
       stCamera:Author="Adobe (www.adobe.com)"
       stCamera:Make="Canon"
       stCamera:CameraRawProfile="True"
       stCamera:LensID="137"
       stCamera:Lens="17.0-50.0 mm"
       stCamera:LensInfo="17/1 50/1 0/0 0/0"
       stCamera:CameraPrettyName="Canon"
       stCamera:LensPrettyName="SIGMA 17-50mm F2.8 EX DC OS HSM"
       stCamera:ProfileName="Adobe (SIGMA 17-50mm F2.8 EX DC OS HSM, Canon)"
       stCamera:SensorFormatFactor="1.5"
       stCamera:FocalLength="17"
       stCamera:FocusDistance="999999995904"
       stCamera:ApertureValue="2.970854">
      <stCamera:AlternateLensIDs>
       <rdf:Seq>
        <rdf:li>240</rdf:li>
       </rdf:Seq>
      </stCamera:AlternateLensIDs>
......
 
Regards,
 
- Chris
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 07, 2016, 04:47:53 AM
Here's an idea for the lua script. Added some of the more popular manual lenses. Only problem right now is that the Zeiss lenses are being automatically detected in Adobe Camera Raw but the Samyang lenses and most other lenses I tried, are not. Let's see if someone at the Adobe forum comes up with a solution for this.

Code: [Select]
lenses =
{
--  The following is for testing purposes. Comment out the following lines then either uncomment the lenses
--  from the list or add your own lenses.

    { name = "My Lens", focal_length = 50 },
    { name = "My Other Lens", focal_length = 25 },

--  Zeiss ZF.2 manual lenses Nikon mount
--  { name = "Zeiss Distagon T* 2.8/15 ZF.2",        focal_length =  15, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 3.5/18 ZF.2",        focal_length =  18, manual_aperture = 3.5 },
--  { name = "Zeiss Distagon T* 2.8/21 ZF.2",        focal_length =  21, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 2.8/25 ZF.2",        focal_length =  25, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 2/25 ZF.2",          focal_length =  25, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 2/28 ZF.2",          focal_length =  28, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 2/35 ZF.2",          focal_length =  35, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 1.4/35 ZF.2",        focal_length =  35, manual_aperture = 1.4 },
--  { name = "Zeiss Makro-Planar T* 2/50 ZF.2",      focal_length =  50, manual_aperture = 2   },
--  { name = "Zeiss Planar T* 1.4/50 ZF.2",          focal_length =  50, manual_aperture = 1.4 },
--  { name = "Zeiss Planar T* 1.4/85 ZF.2",          focal_length =  85, manual_aperture = 1.4 },
--  { name = "Zeiss Makro-Planar T* 2/100 ZF.2",     focal_length = 100, manual_aperture = 2   },
--  { name = "Zeiss Apo Sonnar T* 2/135 ZF.2",       focal_length = 135, manual_aperture = 2   },

--  Samyang manual lenses - also branded as Rokinon and Bower. Cine versions use the same lens profile.
--  { name = "Samyang 8mm f/2.8 UMC Fisheye",        focal_length =   8, manual_aperture = 2.8 },
--  { name = "Samyang 8mm f/2.8 UMC Fisheye II",     focal_length =   8, manual_aperture = 2.8 }, --   8mm T3.1 Cine
--  { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS",    focal_length =   8, manual_aperture = 3.5 },
--  { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS II", focal_length =   8, manual_aperture = 3.5 }, --   8mm T3.8 Cine
--  { name = "Samyang 10mm f/2.8 ED AS NCS CS",      focal_length =  10, manual_aperture = 2.8 }, --  10mm T3.1 Cine
--  { name = "Samyang 12mm f/2 NCS CS",              focal_length =  12, manual_aperture = 2   }, --  12mm T2.2 Cine
--  { name = "Samyang 12mm f/2.8 ED AS NCS Fisheye", focal_length =  12, manual_aperture = 2.8 }, --  12mm T3.1 Cine
--  { name = "Samyang 14mm f/2.8 ED AS IF UMC",      focal_length =  14, manual_aperture = 2.8 }, --  14mm T3.1 Cine
--  { name = "Samyang 16mm f/2 ED AS UMC CS",        focal_length =  16, manual_aperture = 2   }, --  16mm T2.2 Cine
--  { name = "Samyang 21mm f/1.4 ED AS UMC CS",      focal_length =  21, manual_aperture = 1.4 }, --  21mm T1.5 Cine
--  { name = "Samyang 24mm f/1.4 ED AS IF UMC",      focal_length =  24, manual_aperture = 1.4 }, --  24mm T1.5 Cine
--  { name = "Samyang 35mm f/1.4 AS IF UMC",         focal_length =  35, manual_aperture = 1.4 }, --  35mm T1.5 Cine
--  { name = "Samyang 50mm f/1.2 AS UMC CS",         focal_length =  50, manual_aperture = 1.2 },
--  { name = "Samyang 50mm f/1.4 AS UMC",            focal_length =  50, manual_aperture = 1.4 }, --  50mm T1.5 Cine
--  { name = "Samyang 85mm f/1.4 AS IF UMC",         focal_length =  85, manual_aperture = 1.4 }, --  85mm T1.5 Cine
--  { name = "Samyang 100mm f/2.8 ED UMC MACRO",     focal_length = 100, manual_aperture = 2.8 }, -- 100mm T3.1 Cine
--  { name = "Samyang 135mm f/2 ED UMC",             focal_length = 135, manual_aperture = 2   }, -- 135mm T2.2 Cine
--  { name = "Samyang 300mm f/6.3 ED UMC CS",        focal_length = 300, manual_aperture = 6.3 },
}
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 10, 2016, 04:37:24 AM
Been having quite a conversation on the Adobe Camera Raw forum (https://forums.adobe.com/message/9192028) with an Adobe staff member. I did some hacking on a profile for a lens that I own and came up with something that works. Basically, the stCamera:Lens tag in the lens profile seems to be the key.

Here's the line to add to our lua script:

Code: [Select]
{ name = "Bower 16mm T2.2 Cine", focal_length = 16, manual_aperture = 2.2 },
This lens has the same formula for a lens that has a profile in ACR, the Canon M (Bower 16mm f2 ED AS UMC CS) - RAW.lcp. Changed it to this:

Canon(Bower 16mm T2.2 Cine) - RAW.lcp
Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"
    xmlns:stCamera="http://ns.adobe.com/photoshop/1.0/camera-profile">
   <photoshop:CameraProfiles>
    <rdf:Seq>
     <rdf:li>
      <rdf:Description
       stCamera:Author="Adobe (www.adobe.com)"
       stCamera:Make="Canon"
       stCamera:CameraRawProfile="True"
       stCamera:CameraPrettyName="Canon"
       stCamera:Lens="Bower 16mm T2.2 Cine"
       stCamera:LensPrettyName="Bower 16mm T2.2 Cine"
       stCamera:ProfileName="Adobe (Bower 16mm f/2 ED AS UMC CS, Canon)"
       stCamera:SensorFormatFactor="1.6"
       stCamera:ImageWidth="4770"
       stCamera:ImageLength="3177"
       stCamera:FocusDistance="1.200143">
      <stCamera:PerspectiveModel
       stCamera:Version="2"
       stCamera:FocalLengthX="0.69907"
       stCamera:FocalLengthY="0.69907"
       stCamera:ImageXCenter="0.508043"
       stCamera:ImageYCenter="0.513054"
       stCamera:ResidualMeanError="0.00024"
       stCamera:ResidualStandardDeviation="0.000368"
       stCamera:RadialDistortParam1="-0.088214"
       stCamera:RadialDistortParam2="0.073774"
       stCamera:RadialDistortParam3="-0.030652"/>
      </rdf:Description>
     </rdf:li>
     <rdf:li>
      <rdf:Description
       stCamera:Author="Adobe (www.adobe.com)"
       stCamera:Make="Canon"
       stCamera:CameraRawProfile="True"
       stCamera:CameraPrettyName="Canon"
       stCamera:Lens="Bower 16mm T2.2 Cine"
       stCamera:LensPrettyName="Bower 16mm T2.2 Cine"
       stCamera:ProfileName="Adobe (Bower 16mm f/2 ED AS UMC CS, Canon)"
       stCamera:SensorFormatFactor="1.6"
       stCamera:ImageWidth="4770"
       stCamera:ImageLength="3177"
       stCamera:FocusDistance="3"
       stCamera:ApertureValue="6">
      <stCamera:PerspectiveModel>
       <rdf:Description
        stCamera:Version="2"
        stCamera:FocalLengthX="0.69907"
        stCamera:FocalLengthY="0.69907"
        stCamera:ImageXCenter="0.508043"
        stCamera:ImageYCenter="0.513054"
        stCamera:RadialDistortParam1="-0.088214"
        stCamera:RadialDistortParam2="0.073774"
        stCamera:RadialDistortParam3="-0.030652">
       <stCamera:VignetteModel
        stCamera:FocalLengthX="0.69907"
        stCamera:FocalLengthY="0.69907"
        stCamera:VignetteModelParam1="-0.302536"
        stCamera:VignetteModelParam2="0.05392"
        stCamera:VignetteModelParam3="-0.051277"/>
       </rdf:Description>
      </stCamera:PerspectiveModel>
      </rdf:Description>
     </rdf:li>
    </rdf:Seq>
   </photoshop:CameraProfiles>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

It works!

(https://c6.staticflickr.com/1/318/31426461061_00cec8aab7_z.jpg)

I kept the same profile name so I know where it came from.

Ok, so I believe that there's a problem at Adobe's end because these lenses should be automatically recognized but the good news is that it is rather simple to modify an existing lens profile to work.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on December 10, 2016, 05:14:46 AM
That's one great hack @dfort!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 14, 2016, 03:50:52 AM
@Lars_Steenhoff

Had a breakthrough today. To get your Nikon AF NIKKOR 35mm f2D lens working with the lua script simply copy the profile from the ACR installation. I'm not sure what the Windows location is but on the Mac it is in:

Code: [Select]
/Library/Application Support/Adobe/CameraRaw/LensProfiles/1.0/Nikon/NIKON D3X (Nikon AF NIKKOR 35mm f2D) - RAW.lcp
and put the copy in your user area, again on the Mac it is in:

Code: [Select]
/Users/<you>/Library/Application Support/Adobe/CameraRaw/LensProfiles/1.0/
Then edit the stCamera:Lens from this:

Code: [Select]
<stCamera:Lens>35.0 mm f/2.0</stCamera:Lens>
To this:

Code: [Select]
<stCamera:Lens>Nikon AF NIKKOR 35mm f2D</stCamera:Lens>
Do a copy/replace because there are multiple instances of that tag.

Finally put that lens name in your lua script and that's it.

For more details, here's my latest post on the Adobe Camera Raw forum:

https://forums.adobe.com/message/9200782#9200782
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on December 14, 2016, 08:36:34 AM
thats very nice to see @dfort  :)

so a few extra steps but once its setup it all automatic.
I read about your idea of submitting custom profiles to the adobe database, is this still something worth doing ? 
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 14, 2016, 09:26:35 AM
All the Samyang/Rokinon/Bower cine lenses are missing from the official Adobe library but they are the same formula as the still lenses so that would be an easy place to start. Setting up the test chart and doing a real lens profile takes some effort but that is probably the best thing to do to insure that it is working for the image area that you are actually using. Things "seem" to be working now for the Zeiss ZF.2 lenses but those also have the most complicated profiles when compared to some other prime lenses.

In any case, it seems we have a good starting place. There is more that can be done with the script, especially the part about getting xmp sidecar files working but the modifications dmilligan did on the code seems like it is ready for some serious lua scripting.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on December 17, 2016, 04:58:25 PM
Dear all,
I'm an absolute beginner in lua scripting.
You are doing a great job!

I use some manual lens for astrophotography timelapse and I'll love the possibilities to set manually the exif data to avoid to do it in post.

I'm trying to use lua.mo module. All default scripts work without problems.

Then I try to load into my 700D ML card the "lens" lua scripting (adding also ui, config, xmp into library directory) but it doesn't work properly.
I cannot set the lens (I add also the name of my lens copying from this tread Samyang parts)

I find the "Lens Info Prefs" in the Prefs Menu, then "Manual lens" Option, but if I press "set" nothing appear :-\

I have tried a nightly 2016Nov30 " Genuine" and also a self compiled 

Do you know if this script work in EOS 700D?

Thanks
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 17, 2016, 07:45:50 PM
Hi @IBIRRU

Great to hear you're interested in this.

I find the "Lens Info Prefs" in the Prefs Menu, then "Manual lens" Option, but if I press "set" nothing appear :-\

I have tried a nightly 2016Nov30 " Genuine" and also a self compiled 

Do you know if this script work in EOS 700D?

I hope you are compiling from either of these branches:

https://bitbucket.org/hudson/magic-lantern/branch/manual_lens_info
https://bitbucket.org/daniel_fort/magic-lantern/branch/manual_lens_lua_script

I've just been working on adding some lenses and am having productive conversations with a staff member at Adobe to get more lens profiles for manual lenses working with this.

When you set the lens you won't "see" anything until you work with the MLV files. Right now that's the only format that this works with. Still will require some more work to get the xmp "sidecar" files working and DNG needs some back end modifications.

It should work on all platforms including the 700D. I've got one of those and just tested it to make sure.

I can see that you are able to compile--great! If you need any help, have a feature request or want a special build, just post it here.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on December 18, 2016, 04:18:20 PM
Thanks DFort :)

I compiled both branches you suggest and try both them.
I add Samyang 14mm and an old Sigma 28mm to the list just begin to try.

They both work for MLV (silent) and CR2 still pictures (with xmp sidecar file). I will wait for DNG files with correct exif data. :)

ui.lua: I found 2 little error at row 25 ("utton" instead of "button") and row 50 ("b" instead of "self"). Otherwise the lua editor doesn't work.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 18, 2016, 05:11:13 PM
They both work for MLV (silent) and CR2 still pictures (with xmp sidecar file)...

ui.lua: I found 2 little error at row 25 ("utton" instead of "button") and row 50 ("b" instead of "self"). Otherwise the lua editor doesn't work.

You got xmp working? How? I haven't even ventured there yet. Please feel free to make pull requests for any fixes you're making. Let me know if I can help with that.

So how do you plan to use this feature? Lars Steenhoff wanted it for doing automatic lens distortion correction in Adobe Camera Raw. That lead to some interesting discussions with an Adobe staff member that might make future profiles for manual lenses a bit more user friendly. Right now the profiles for the lenses that you are using need to be hacked. More information here:

https://forums.adobe.com/message/9200782#9200782
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on December 22, 2016, 07:24:00 PM
I take hundreds of shots using manual lens with time exposure > 10 s up to 60s for nightly timelapse and load them to LRTimelapse or other software.
My goal would be take "dng Silent pictures" to save shutter wearing
Unfortunately  "dng" silent pictures do not save aperture or time or ISO into exif data.

I try xmp sidecar with CR2 and it work because when I load to LRTimelapse the aperture is correctly loaded

I try to use MLV format (but it is risky) when I deflate into dng files I could find the right lens desciption string and using ExifToolGui I can batch change into all dng files.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 22, 2016, 07:58:44 PM
You lost me:

I try xmp sidecar with CR2 and it work because when I load to LRTimelapse the aperture is correctly loaded

xmp with CR2 isn't working yet, or at least it isn't in the current lens lua script. I'm not sure how to script it yet.

I try to use MLV format (but it is risky) when I deflate into dng files I could find the right lens desciption string and using ExifToolGui I can batch change into all dng files.

If you're using the script it should save the metadata into the MLV file and all the DNG files should have the EXIF tags. How are you "deflating" the DNG's? Since you're using the 700D I'd recommend MLVFS (http://www.magiclantern.fm/forum/index.php?topic=13152.0). That should take care of any focus pixels from that camera. Are you are using full resolution silent picture? (Search for FRSP) I haven't tried that yet with this script.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on December 22, 2016, 08:50:31 PM
I tried with Bridge and LRTimelapse.
The xmp sidecar generated from script is correctly loaded.
All lens data are present and if I change manually the data, into the xmp, Bridge load the new data without problem.

I tried MLV with MLVFS and it's ok.

Thanks DFort  :D
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on December 23, 2016, 08:35:43 AM
I still don't follow you @IBIRRU -- care to share screenshots of some sort?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on December 23, 2016, 10:29:11 AM
I'll try to resume:
0) I choose a manual lens data from script
1) CR2: works OK with XMP sidecar generated by script.
That means I can record the exposure exif data of manual lens and when load the pictures on Adobe Bridge or ACR I have all exposure data, time, aperture and ISO.
2) FULL SILENT PICTURE
a) Mlv format works OK  with Mlvfs. That means: the exif data embedded into dng single shots are correctly passed from script to mlv to dng.
b) DNG format: the single DNG picture doesn't have the right exif data.

I don't use the exif data for geometric correction.

I try for Samyang 14 mm a lcp profile created for  EOS 650. Edit it for EOS 700. ACR is able to use it but I don't know if is right.

Inviato dal mio GT-I9300 utilizzando Tapatalk

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on December 23, 2016, 02:44:19 PM
Merged the lens info stuff from g3gg0 and fixed the typos that IBIRRU found: https://bitbucket.org/hudson/magic-lantern/branch/manual_lens_info

Haven't really tried it, there might be some weirdness with the menus now that there's a bunch of lens info in that menu.

And yes, XMP should be working.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on December 23, 2016, 06:23:45 PM
Nice find @IBIRRU and Thanks for the quick fix @dmilligan!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 24, 2016, 09:33:29 AM
I'm getting these compile errors on several modules that seemed to have been introduced with the lens_info_fix merge.

Code: [Select]
Building module lua...
Updated HGVERSION
[ README   ]   module_strings.h
Traceback (most recent call last):
  File "../readme2modulestrings.py", line 150, in <module>
    size = extent_func(commit_msg)[0]
  File "/Users/rosiefort/magic-lantern/modules/rbf_read.py", line 64, in extent_func
    w += font.wTable[ord(c) - font.charFirst]
IndexError: list index out of range
make[4]: *** [module_strings.h] Error 1

********************************************************
WARNING: module lua failed to build, deleting
********************************************************
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on December 24, 2016, 09:52:50 AM
Non-ASCII characters in commit message.

This looks good, but adds yet another dependency to the build process: https://www.tablix.org/~avian/blog/archives/2009/01/unicode_transliteration_in_python/

Code: [Select]
>>> from unidecode import unidecode
>>> print unidecode(u"Merge ‘lens_info_fix’ into ‘manual_lens_info’")
Merge 'lens_info_fix' into 'manual_lens_info'

A solution without additional dependencies, but non-ASCII characters are removed:
Code: [Select]
>>> msg = "Merge ‘lens_info_fix’ into ‘manual_lens_info’"
>>> print msg.decode('ascii', 'ignore')
Merge lens_info_fix into manual_lens_info
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on December 24, 2016, 03:36:40 PM
Yeah I couldn't figure this one out either. I had ended up going back and making a new branch and cherry picking everything on to it. Not sure how the non-ASCII single quotes got in there. Must have been SourceTree.

I'm in favor of the no dependency option.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 24, 2016, 06:39:06 PM
So how do you edit a commit message? I assume it has to be done by a developer who can access the hudson repository. Last time I submitted a pull request that backed out a changeset (https://bitbucket.org/hudson/magic-lantern/pull-requests/763/backout-changeset/diff) it caused all sorts of issues on my repository so I'll just go sit in a corner and wait.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 04, 2017, 01:59:36 AM
Ok, had enough sitting in a corner and decided to work around the commit issue. It wasn't as straightforward as I had hoped but it worked.

Now the problem is that the lua_fix branch doesn't draw the information on the EOSM screen properly so of course neither does the manual_lens_info branch after the merge.

(https://c3.staticflickr.com/1/735/32049369666_7a747b4c51.jpg)

Note that I have only experienced this on the EOSM. It looks fine on the 700D.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on January 04, 2017, 09:56:21 AM
I can try to troubleshoot it in QEMU, but I need some things from you:

- your autoexec.bin (the one that shows incorrect display)
- a startup log from dm-spy-experiments branch, with mpu_send/mpu_recv stubs (0x1dd2c, 0x36A0); background info here (http://www.magiclantern.fm/forum/index.php?topic=2864.msg166938;topicseen#msg166938) and here (http://www.magiclantern.fm/forum/index.php?topic=2388.msg113154#msg113154).
- same log from 700D (stubs already there)
- a serial flash dump (here (http://www.magiclantern.fm/forum/index.php?topic=16040.msg155949#msg155949); will commit a more polished version soon)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 04, 2017, 06:10:46 PM
Thanks a1ex, I'm working on getting what you asked for.

I made a regression test and found that commit 56464bc is where the issue first appeared.

Quote
bmp_printf: allow transparent text on transparent background
(note: this breaks binary compatibility with modules)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on January 04, 2017, 06:44:37 PM
Just in case: did you recompile everything with make clean?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 04, 2017, 07:04:11 PM
Yes, I'm running make clean on every compile run.

I got the autoexec.bin files from the lua_fix branch for both the EOSM (incorrect display) and the 700D (correct display)

Now I'm stuck trying to figure out how to get a startup log with the dm-spy-experiments branch.
[EDIT] Dumb mistake -- logging is working.

I saw the Serial flash dumper (https://bitbucket.org/hudson/magic-lantern/pull-requests/793/serial-flash-dumper/diff#comment-None) pull request. It has a note with my name on it and thought, uh oh -- what did I get myself into? I'm doing my best to get this together. Note, I've never used QEMU though that's on my to learn list.

In the meantime I'm adding the files you're requesting in this dropbox folder:
https://www.dropbox.com/sh/m17imm8wqavd1q3/AADOhSg95KRemu6WGc9zRqgaa?dl=0
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on January 04, 2017, 07:10:57 PM
Just CONFIG_DEBUG_INTERCEPT_STARTUP = y should be enough for 700D. The log file is created at startup, a few seconds after starting the camera.

These stubs are in dm-spy-extra.c (easy to find with grep, or with a text editor that can search in all files from a directory).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 04, 2017, 07:38:59 PM
Got it. Startup logs are in the dropbox folder. Next challenge, serial flash dump.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 04, 2017, 08:12:57 PM
Ok serial flash dumps are uploaded to the dropbox folder.

https://www.dropbox.com/sh/m17imm8wqavd1q3/AADOhSg95KRemu6WGc9zRqgaa?dl=0

I made the dump with only the sf_dump module enabled and saw some 404 reports on the console for dual_iso and other modules that were not enabled. I played around with it a little more and found that depending on which modules are loaded it might be saving different information in the dump. I'll run another dump with these modules loaded:

Code: [Select]
dual_iso
ettr
mlv_snd (mlv_rec)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 04, 2017, 08:24:07 PM
@a1ex -- You should have everything you asked for now. I'm also doing a pull request for the dm-spy-experiments branch to add the mpu_send/mpu_recv stubs for the EOSM.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on January 05, 2017, 05:26:16 AM
Seriously what an excellent way to start off the new year on a great note @dfort and thanks for making this happen!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on January 06, 2017, 05:55:17 PM
In the Latest Experimentals Build (2017-01-06 13:40) for 700D "Non-CPU lens info" is missing the lua.mo module ::)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on January 06, 2017, 06:11:50 PM
The same if try to compile :-[

Building module lua...
Updated HGVERSION
[ README   ]   module_strings.h
Traceback (most recent call last):
  File "../readme2modulestrings.py", line 150, in <module>
    size = extent_func(commit_msg)[0]
  File "/home/fabio/magic-lantern/manual_lens_lua_script/modules/rbf_read.py", line 64, in extent_func
    w += font.wTable[ord(c) - font.charFirst]
IndexError: list index out of range
make[4]: *** [module_strings.h] Errore 1

********************************************************
WARNING: module lua failed to build, deleting
********************************************************
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on January 06, 2017, 06:20:12 PM
Fixed.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on January 06, 2017, 06:29:08 PM
Magic @a1ex !  8)

it compiles now.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 06, 2017, 08:40:37 PM
It compiles and I also found a solution to the issue reported in Reply #116. Simply clear the Canon settings. 4th wrench, Clear settings: Clear all camera settings. Of course now I have to re-enter my preferred settings and hope that the issue doesn't return.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 06, 2017, 08:44:14 PM
In the Latest Experimentals Build (2017-01-06 13:40) for 700D "Non-CPU lens info" is missing the lua.mo module ::)

The name of the file is lens.lua and the name of the module as it appears on the lua menu is Manual Lens Information.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on January 06, 2017, 08:46:14 PM
Just to clarify... exactly which branch are you guys compiling from?

I'd like to give this a try even if its already implemented into the Nightlies.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 06, 2017, 08:50:25 PM
It is the manual_lens_info branch:

https://bitbucket.org/hudson/magic-lantern/branch/manual_lens_info

Note to self:  Reset camera to default settings before making bug reports.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 07, 2017, 01:43:11 AM
Great news. You don't need to learn how to compile or beg to get a build.

https://builds.magiclantern.fm/experiments.html

Scroll down to the Non-CPU lens info section. I'm still hosting builds for the 5D3.123. Sorry 1100D/T3 users, there are still some issues that need to be resolved before you can use this.

I've added some popular manual lenses along with instructions on a lens.lua script that I'm testing. Here it is if you want to try it out. The script is self documented.

lens.lua
Code: [Select]
-- Manual Lens Information
-- Supplies lens information for manual lenses
-- Whenever there is a non-chipped lens detected, we prompt the user to select the attached manual lens from a list

require("ui")
require("config")
require("xmp")

lenses =
{
--  The following is for testing purposes. Comment out the following lines then either uncomment only the lenses
--  that you want to use from the list or add your own lenses. Tip: Put your most used lenses at the top of the list.

    { name = "My Lens", focal_length = 50 },
    { name = "My Other Lens", focal_length = 25 },

--  Zeiss ZF.2 manual lenses Nikon mount - these work with the lens profiles that ship with Adobe Camera Raw
--  { name = "Zeiss Distagon T* 2.8/15 ZF.2",        focal_length =  15, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 3.5/18 ZF.2",        focal_length =  18, manual_aperture = 3.5 },
--  { name = "Zeiss Distagon T* 2.8/21 ZF.2",        focal_length =  21, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 2.8/25 ZF.2",        focal_length =  25, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 2/25 ZF.2",          focal_length =  25, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 2/28 ZF.2",          focal_length =  28, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 2/35 ZF.2",          focal_length =  35, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 1.4/35 ZF.2",        focal_length =  35, manual_aperture = 1.4 },
--  { name = "Zeiss Makro-Planar T* 2/50 ZF.2",      focal_length =  50, manual_aperture = 2   },
--  { name = "Zeiss Planar T* 1.4/50 ZF.2",          focal_length =  50, manual_aperture = 1.4 },
--  { name = "Zeiss Planar T* 1.4/85 ZF.2",          focal_length =  85, manual_aperture = 1.4 },
--  { name = "Zeiss Makro-Planar T* 2/100 ZF.2",     focal_length = 100, manual_aperture = 2   },
--  { name = "Zeiss Apo Sonnar T* 2/135 ZF.2",       focal_length = 135, manual_aperture = 2   },

--  Samyang manual lenses - also branded as Rokinon and Bower. Cine versions use the same lens profile.
--  The lens profiles for Samyang manual lenses that ship with Adobe Camera raw must be modified in order
--  for automatic lens detection to work.
--  More information here: http://www.magiclantern.fm/forum/index.php?topic=18083.msg176261#msg176261
--  { name = "Samyang 8mm f/2.8 UMC Fisheye",        focal_length =   8, manual_aperture = 2.8 },
--  { name = "Samyang 8mm f/2.8 UMC Fisheye II",     focal_length =   8, manual_aperture = 2.8 }, --   8mm T3.1 Cine
--  { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS",    focal_length =   8, manual_aperture = 3.5 },
--  { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS II", focal_length =   8, manual_aperture = 3.5 }, --   8mm T3.8 Cine
--  { name = "Samyang 10mm f/2.8 ED AS NCS CS",      focal_length =  10, manual_aperture = 2.8 }, --  10mm T3.1 Cine
--  { name = "Samyang 12mm f/2 NCS CS",              focal_length =  12, manual_aperture = 2   }, --  12mm T2.2 Cine
--  { name = "Samyang 12mm f/2.8 ED AS NCS Fisheye", focal_length =  12, manual_aperture = 2.8 }, --  12mm T3.1 Cine
--  { name = "Samyang 14mm f/2.8 ED AS IF UMC",      focal_length =  14, manual_aperture = 2.8 }, --  14mm T3.1 Cine
--  { name = "Samyang 16mm f/2 ED AS UMC CS",        focal_length =  16, manual_aperture = 2   }, --  16mm T2.2 Cine
--  { name = "Samyang 21mm f/1.4 ED AS UMC CS",      focal_length =  21, manual_aperture = 1.4 }, --  21mm T1.5 Cine
--  { name = "Samyang 24mm f/1.4 ED AS IF UMC",      focal_length =  24, manual_aperture = 1.4 }, --  24mm T1.5 Cine
--  { name = "Samyang 35mm f/1.4 AS IF UMC",         focal_length =  35, manual_aperture = 1.4 }, --  35mm T1.5 Cine
--  { name = "Samyang 50mm f/1.2 AS UMC CS",         focal_length =  50, manual_aperture = 1.2 },
--  { name = "Samyang 50mm f/1.4 AS UMC",            focal_length =  50, manual_aperture = 1.4 }, --  50mm T1.5 Cine
--  { name = "Samyang 85mm f/1.4 AS IF UMC",         focal_length =  85, manual_aperture = 1.4 }, --  85mm T1.5 Cine
--  { name = "Samyang 100mm f/2.8 ED UMC MACRO",     focal_length = 100, manual_aperture = 2.8 }, -- 100mm T3.1 Cine
--  { name = "Samyang 135mm f/2 ED UMC",             focal_length = 135, manual_aperture = 2   }, -- 135mm T2.2 Cine
--  { name = "Samyang 300mm f/6.3 ED UMC CS",        focal_length = 300, manual_aperture = 6.3 },
}

selector_instance = selector.create("Select Manual Lens", lenses, function(l) return l.name end, 600)

xmp:add_property(xmp.lens_name, function() return lens.name end)
xmp:add_property(xmp.focal_length, function() return lens.focal_length end)
xmp:add_property(xmp.aperture, function() return math.floor(camera.aperture.value * 10) end)

function property.LENS_NAME:handler(value)
    if lens.is_chipped == false then
        task.create(select_lens)
    else
        if selector_instance ~= nil then
            selector_instance.cancel = true
        end
        xmp:stop()
    end
end

function select_lens()
    if #lenses > 1 then
        local menu_already_open = menu.visible
        if not menu_already_open then
            menu.open()
            display.rect(0, 0, display.width, display.height, COLOR.BLACK, COLOR.BLACK)
        end
        if selector_instance:select() then
            update_lens()
        end
        if not menu_already_open then
            menu.close()
        end
    elseif #lenses == 1 then
        update_lens()
    end
end

function update_lens()
    for k,v in pairs(lenses[selector_instance.index]) do
        lens[k] = v
    end
    xmp:start()
end

lens_menu = menu.new
{
    parent = "Lens Info Prefs",
    name = "Manual Lens",
    help = "Info to use for attached non-chipped lens",
    icon_type = ICON_TYPE.ACTION,
    select = function()
        if lens.is_chipped == false then
            task.create(select_lens)
        end
    end,
    rinfo = function()
        return lens.name
    end,
    warning = function()
        if lens.is_chipped then
            return "Chipped lens is attached"
        end
    end
}

if lens.is_chipped == false then
    task.create(select_lens)
end
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 07, 2017, 10:28:23 AM
@anyone

I've downloaded the lens info experimental build for the EOSM.

I've tested if it moves my 24-105 and it does: great!

But, when I try running one of my image taking scripts, that run fine on my 5D3, things go bad :(

Could someone tell me if any of the Lua calls, eg camera.shoot, have changed their calling syntax.

Cheers

Garry
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on January 07, 2017, 02:16:00 PM
This branch has the latest lua_fix merged in.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 07, 2017, 03:46:47 PM
Sorry, David, where can I find out the changes, i.e. in function calls etc.

Sorry to be a pain.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on January 07, 2017, 04:01:38 PM
https://bitbucket.org/hudson/magic-lantern/commits/branch/manual_lens_info
https://bitbucket.org/hudson/magic-lantern/src/0f1e71c6529d/modules/lua/lua_camera.c?at=manual_lens_info&fileviewer=file-view-default#lua_camera.c-265
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 07, 2017, 04:22:18 PM
@dmilligan

Great, thank you: got the full exposure/focus bracketing code running on the EOSM now :)

As usual, thanks for all you support.

All I need to do now is speed things up, ie using single steps is too slow. I need a more intelligent move_to algorithm.

Cheers

Garry
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 07, 2017, 04:41:45 PM
@David

In the Lua fix, has the control of an EF lens changed. It looks like a lens.focus(1,1,false) call, say, seems to drive the lens differently than previously, ie finer steps.

Cheers

Garry
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 08, 2017, 07:02:33 AM
Saw that the latest commits include, "lens.lua: store selected lens in config" -- I just tested it and it doesn't seem to work, at least not as I would have expected a configuration file would work which is to highlight the last used lens when the camera is started.

Ok, that's a new feature so maybe it isn't ready yet or I'm not using it properly. Let's go over what's working and what's not.

The .xmp "sidecar" file that was implimented back in November is working great for CR2 and JPEG files. We know that the Zeiss ZF.2 lenses are being automatically detected by Adobe Camera Raw so I pretended that my cheap Samyang lens was an expensive Zeiss.

(https://c7.staticflickr.com/1/616/31801902910_8e18046708_z.jpg)

Nice! Here's the .xmp file:

Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
        aux:Lens="Zeiss Planar T* 1.4/50 ZF.2"
    exif:FNumber="14/10"
    exif:FocalLength="50/1"
>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Time to get nit picky. Shouldn't that ">" that is sitting all the way to the left right after the FocalLength field be on the end of the previous line? It doesn't affect functionality but since everything is so neatly formatted it looks out of place.

XMP files are not created for "silent" DNG files. Although exiftool detects some lens tags in the files, it doesn't matter if a manual lens or a Canon EF lens is attached, the data is always the same:

Code: [Select]
F Number                        : 0
Max Aperture Value              : 1.0
Focal Length                    : 0.0 mm
Focal Length In 35mm Format     : 0 mm
Lens Model                      :
Focal Length                    : 0.0 mm

Of course since ML code is producing the DNG files maybe we can look into writing the lens metadata directly into the file?

Another file format that isn't benefitting from the manual lens lua script is H.264. Chipped lenses (electronic chips not glass  ;) ) are saved in the metadata but not manual lenses.

Code: [Select]
Canon Image Type                : MVI:Canon EOS REBEL T5i
Lens                            : 0.0 mm
Scale Factor To 35 mm Equivalent: 1.6
Focal Length                    : 0.0 mm (35 mm equivalent: 0.0 mm)
Hyperfocal Distance             : inf m
Lens                            : 0.0 mm (35 mm equivalent: 0.0 mm)

The first feature that was working was saving the manual lens metadata to MLV files and that's still working great. Note that the "Serial" field isn't implimented in the lua script but this also seems to be an issue with chipped lenses that save the lens serial number in CR2 and H.264 files but not to MLV files.

Code: [Select]
Block: LENS
  Offset: 0x0000014c
    Size: 96
    Time: 0.003000 ms
     Name:        'Samyang 50mm f/1.4 AS UMC'
     Serial:      ''
     Focal Len:   50 mm
     Focus Dist:  0 mm
     Aperture:    f/1.40
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

When the "lens_info_fix" branch was merged into "manual_lens_info" we gained several new fields but for now they are blank. A problem with this is that it is no longer possible to switch lenses from the "Lens Info/Prefs" menu.

(https://c7.staticflickr.com/1/740/31335484614_b5533cca15.jpg)

Before the merge it looked like this:

(https://c5.staticflickr.com/1/737/31335484684_34779ea44b.jpg)

Finally, I brought up an issue with the EOSM on Reply #116 (http://www.magiclantern.fm/forum/index.php?topic=18083.msg177532#msg177532) and it happened again with the 700D. I was able to clear it up on the EOSM by clearing all Canon camera setting but this time it was intermittent and wouldn't show up when loading a new build that doesn't have a "magic.cfg" or any other configuration file saved yet. I'm still hunting down what condition causes this but it started after lua_fix was merged on December 10.

(https://c7.staticflickr.com/1/527/31335483054_245e60c212.jpg)

Please check out the latest builds on the experimental downloads page (https://builds.magiclantern.fm/experiments.html), try this on other cameras and report back. Thanks in advance to the testers and a big thanks to @dmilligan for all the work he put into this.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 08, 2017, 09:15:15 AM
@dfort

Just got up!

Just loaded the latest experimental EOSM build and the good news is my scripts still work  ;)

The mystery for me at the moment is to work out what to do with the new lens script, as I haven't been following this development.

I'm testing with a 14mm Rokinon.

Any hints as to how to use the new functionality?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 08, 2017, 11:17:46 AM
OK having looked at the script, I think I worked it out...but

The pop up comes up and I select my lens, eg:

Code: [Select]
{ name = "Rokinon 14mm", focal_length = 14, manual_aperture = 2.8},
I take a .cr2 image.

I don't see a .xmp file on the card  ???

I import in Lightroom and 'only' see the exposure it captured.

Also, unless I'm mistaken, I haven't see a the shutter button freeze for a while. Does this build fix it?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 08, 2017, 11:53:06 AM
Further EOSM testing.

The screen randomly flashes colours, orange or magenta, when you take a picture. Have never seen that before.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 08, 2017, 04:58:50 PM
@garry23 -- Are you using the lens.lua script I posted in Reply #134 (http://Reply #134)? For the Rokinon 14mm all you need to do on the script is comment out the lines with "My Lens" and "My Other Lens" (those are just for testing purposes) and uncomment the "Samyang 14mm f/2.8 ED AS IF UMC" (the Rokinon brand is manufactured by Samyang). You can make your own entry but it has to be named exactly the way the way it is in the lens profile file for it to work automatically in Lightroom.

I'm surprised that it isn't working for you. I just tried the version from the experimentals download (https://builds.magiclantern.fm/experiments.html) page on the EOSM and it is working for me. In fact I used it on a camera that had all the Canon settings set to default values so in order to get it working with a manual lens I had to go through the drill of switching to Manual mode to get into the Custom Functions(C.Fn) and setting C.FnIV:Operation/Others Release shutter w/o lens to 1:ENABLE. Once lua and the lens.lua script were activated it created the .xmp file for CR2 shots, in fact it also works with JPEG so I added that note to my previous post.

Ok so let's go back to the lens.lua script that is in the experimentals download and putting in your lens exactly the way you entered it here:

Code: [Select]
{ name = "Rokinon 14mm", focal_length = 14, manual_aperture = 2.8},
To get this working with ACR and Lightroom find the profile for that lens. On my Mac it is located here:
Code: [Select]
/Library/Application\ Support/Adobe/CameraRaw/LensProfiles/1.0/Rokinon/Canon
Now put a copy of that file into your user library. Of course your user name is different than mine but on my system (ok, my wife's system) it is:
Code: [Select]
/Users/rosiefort/Library/Application\ Support/Adobe/CameraRaw/LensProfiles/1.0
Note that you can create directories within the 1.0 folder to keep things organized or if you just have a few manual lenses that isn't really necessary.

Now for the hack to get this lens profile working automatically. Add this line to each section of the lens profile. In the Rokinon 14mm lens profile there are two sections, each starting with <rdf:Description. I put it this line just before the LensPrettyName tag but I don't think that it really matters where you put it:

Code: [Select]
       stCamera:Lens="Rokinon 14mm"       

I just went through all these steps and tested it on the EOSM and Lightroom was able to automatically detect the lens using the .xmp sidecar file.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 08, 2017, 05:34:13 PM
Daniel

Thanks for find the time to come back to me so quickly.

I did what you said, ie added that profile to every occurrence, eg under Canon the pscache.dat file entry for canon looks like this (note I added the <>):

Code: [Select]
<rdf:li rdf:parseType="Resource">
                  <stCamera:DocumentID>392EC5D0752F1230840E46CB1142785C</stCamera:DocumentID>
                  <stCamera:Make>Canon</stCamera:Make>
                  <stCamera:Model>Canon EOS-1Ds Mark III</stCamera:Model>
   <stCamera:Lens="Rokinon 14mm">
                  <stCamera:LensPrettyName>Rokinon 14mm f/2.8 ED AS IF UMC</stCamera:LensPrettyName>
                  <stCamera:CameraPrettyName>Canon EOS-1Ds Mark III</stCamera:CameraPrettyName>
                  <stCamera:CameraRawProfile>True</stCamera:CameraRawProfile>
                  <stCamera:SensorFormatFactor>1.000000</stCamera:SensorFormatFactor>
                  <stCamera:ModifyDate>131240269000000000</stCamera:ModifyDate>
                  <stCamera:FishEyeLens>False</stCamera:FishEyeLens>

Still no update in LR.

BTW the new lens script, will I see xmp files on the card? I don't at the moment.

Also, this build does interact with the EOSM 'LV' when taking a .CR2 image, screen flashes strange colours.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on January 08, 2017, 06:18:49 PM
This is my experience with the  Experiments release of "Non-CPU lens info" 07-jan-2017

In my 700D the xmp sidecar file for CR2 works OK

In the lcp profile If I add the tag <stCamera:Lens>Samyang 14mm f/2.8 ED AS IF UMC</stCamera:Lens>  before <stCamera:LensPrettyName>Samyang 14mm f/2.8 ED AS IF UMC</stCamera:LensPrettyName> then ACR found automatically the lens profile, otherwise I need to choose it manually from list.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 08, 2017, 06:22:13 PM
@IBIRRU

Just to help me.

You actually see a .xmp file on the card?

I don't on the EOSM.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 08, 2017, 06:28:23 PM
I did what you said, ie added that profile to every occurrence, eg under Canon the pscache.dat file entry for canon looks like this (note I added the <>):

The pscache.dat file? I never touched that. Seems like it is some sort of automatically generated database file so don't edit it!

Let's start with getting that .xmp file working. Without that, there's no point importing the CR2 (or JPEG) into Lightroom.

Try this, from the magiclantern-manual_lens_info.2017Jan08.EOSM202 (https://builds.magiclantern.fm/jenkins/job/manual_lens_info/6/artifact/platform/EOSM.202/magiclantern-manual_lens_info.2017Jan08.EOSM202.zip) you downloaded, replace the lens.lua script with this one:

lens.lua
Code: [Select]
-- Manual Lens Information
-- Supplies lens information for manual lenses
-- Whenever there is a non-chipped lens detected, we prompt the user to select the attached manual lens from a list

require("ui")
require("config")
require("xmp")

lenses =
{
    { name = "Rokinon 14mm", focal_length = 14, manual_aperture = 2.8},
}

selector_instance = selector.create("Select Manual Lens", lenses, function(l) return l.name end, 600)

lens_config = config.create({})

if lens_config.data ~= nil and lens_config.data.name ~= nil then
    for i,v in ipairs(lenses) do
        if v.name == lens_config.data.name then
            selector_instance.index = i
            break
        end
    end
end

xmp:add_property(xmp.lens_name, function() return lens.name end)
xmp:add_property(xmp.focal_length, function() return lens.focal_length end)
xmp:add_property(xmp.aperture, function() return math.floor(camera.aperture.value * 10) end)

function property.LENS_NAME:handler(value)
    if lens.is_chipped == false then
        task.create(select_lens)
    else
        if selector_instance ~= nil then
            selector_instance.cancel = true
        end
        xmp:stop()
    end
end

function select_lens()
    if #lenses > 1 then
        local menu_already_open = menu.visible
        if not menu_already_open then
            menu.open()
            display.rect(0, 0, display.width, display.height, COLOR.BLACK, COLOR.BLACK)
        end
        if selector_instance:select() then
            update_lens()
        end
        if not menu_already_open then
            menu.close()
        end
    elseif #lenses == 1 then
        update_lens()
    end
end

function update_lens()
    for k,v in pairs(lenses[selector_instance.index]) do
        lens[k] = v
    end
    lens_config.data = { name = lenses[selector_instance.index].name }
    xmp:start()
end

lens_menu = menu.new
{
    parent = "Lens Info Prefs",
    name = "Manual Lens",
    help = "Info to use for attached non-chipped lens",
    icon_type = ICON_TYPE.ACTION,
    select = function()
        if lens.is_chipped == false then
            task.create(select_lens)
        end
    end,
    rinfo = function()
        return lens.name
    end,
    warning = function()
        if lens.is_chipped then
            return "Chipped lens is attached"
        end
    end
}

if lens.is_chipped == false then
    task.create(select_lens)
end

There is only one choice so the lens selection menu doesn't even come up--kind of a neat feature if you're only using that one manual lens.

Did you get an .xmp "sidecar" file? Yes? Good, now in your user Camera Raw Lens Profile 1.0 library, don't touch pscache.dat--put in this lens profile:

Canon (Rokinon 14mm f2.8 ED AS IF UMC) - RAW.lcp
Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"
    xmlns:stCamera="http://ns.adobe.com/photoshop/1.0/camera-profile">
   <photoshop:CameraProfiles>
    <rdf:Seq>
     <rdf:li>
      <rdf:Description
       stCamera:Author="Adobe (www.adobe.com)"
       stCamera:Make="Canon"
       stCamera:Model="Canon EOS-1Ds Mark III"
       stCamera:UniqueCameraModel="Canon EOS-1Ds Mark III"
       stCamera:CameraRawProfile="True"
       stCamera:CameraPrettyName="Canon EOS-1Ds Mark III"
       stCamera:Lens="Rokinon 14mm"       
       stCamera:LensPrettyName="Rokinon 14mm f/2.8 ED AS IF UMC"
       stCamera:ProfileName="Adobe (Rokinon 14mm f/2.8 ED AS IF UMC, Canon)"
       stCamera:SensorFormatFactor="1"
       stCamera:ImageWidth="5640"
       stCamera:ImageLength="3752"
       stCamera:FocusDistance="1.082921">
      <stCamera:PerspectiveModel
       stCamera:Version="2"
       stCamera:FocalLengthX="0.399824"
       stCamera:FocalLengthY="0.399824"
       stCamera:ImageXCenter="0.495736"
       stCamera:ImageYCenter="0.495205"
       stCamera:ResidualMeanError="0.001928"
       stCamera:ResidualStandardDeviation="0.004525"
       stCamera:RadialDistortParam1="-0.095549"
       stCamera:RadialDistortParam2="0.041922"
       stCamera:RadialDistortParam3="-0.005911"/>
      </rdf:Description>
     </rdf:li>
     <rdf:li>
      <rdf:Description
       stCamera:Author="Adobe (www.adobe.com)"
       stCamera:Make="Canon"
       stCamera:Model="Canon EOS-1Ds Mark III"
       stCamera:UniqueCameraModel="Canon EOS-1Ds Mark III"
       stCamera:CameraRawProfile="True"
       stCamera:CameraPrettyName="Canon EOS-1Ds Mark III"
       stCamera:Lens="Rokinon 14mm"
       stCamera:LensPrettyName="Rokinon 14mm f/2.8 ED AS IF UMC"
       stCamera:ProfileName="Adobe (Rokinon 14mm f/2.8 ED AS IF UMC, Canon)"
       stCamera:SensorFormatFactor="1"
       stCamera:ImageWidth="5640"
       stCamera:ImageLength="3752"
       stCamera:FocusDistance="3"
       stCamera:ApertureValue="6">
      <stCamera:PerspectiveModel>
       <rdf:Description
        stCamera:Version="2"
        stCamera:FocalLengthX="0.399824"
        stCamera:FocalLengthY="0.399824"
        stCamera:ImageXCenter="0.495736"
        stCamera:ImageYCenter="0.495205"
        stCamera:RadialDistortParam1="-0.095549"
        stCamera:RadialDistortParam2="0.041922"
        stCamera:RadialDistortParam3="-0.005911">
       <stCamera:VignetteModel
        stCamera:FocalLengthX="0.399824"
        stCamera:FocalLengthY="0.399824"
        stCamera:VignetteModelParam1="-0.387264"
        stCamera:VignetteModelParam2="0.150392"
        stCamera:VignetteModelParam3="-0.042729"/>
       </rdf:Description>
      </stCamera:PerspectiveModel>
      </rdf:Description>
     </rdf:li>
    </rdf:Seq>
   </photoshop:CameraProfiles>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

If you're using Windows you'll have to search for the location where this is stored -- please report back.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: IBIRRU on January 08, 2017, 06:35:38 PM
@garry23
yes I have the xmp sidecar file generated from lua script into the SD card toghether the CR2 file
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 08, 2017, 06:39:00 PM
@dfort

No sidecar file :( on my sic) EOSM
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 08, 2017, 07:28:59 PM
Strange. Try resetting your Canon settings and deleting the ML settings and see if that does it. You'll have to switch to manual mode to get to the menus that enable shooting w/o a lens to use your Rokinon lens.

Though one more thing that might be getting in the way--is your Rokinon lens chipped? Try shooting a frame without a lens mounted and see if that creates the .xmp file.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 08, 2017, 07:41:33 PM
:)

Reset Canon settings and see .xmp file :)

I carry on trying to get things up and running a report back.

Plus strange colours all gone !

Thanks for your support :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on January 08, 2017, 08:18:18 PM
You probably had QR turned off.

Config saving is working as expected on my end. Pushed a couple of other fixes: menu named had been changed (that's why it wasn't showing up under "Lens Info/Prefs", actually was showing up as it's own menu top right), and selection dialog disappearing immediately on subsequent lens changes.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 08, 2017, 08:51:29 PM
David

Sorry, why do you need image review on for this to work?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on January 08, 2017, 09:23:57 PM
Code: [Select]
function xmp:start()
    local this = self
    function property.GUI_STATE:handler(value)
        if value == GUISTATE_QR then
            this:write()
        end
    end
end
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 08, 2017, 09:39:03 PM
David

Sorry I didn't spot that in the script.

But, unless I'm mistaken, this only works if image review is on, and that seems too restrictive. What about a timelapse with no image review on?

Could an alternative be to always test if a manual lens is flagged via event.post_shoot: which would always create an xmp after an image has been taken?

Just a thought.

Cheers

Garry
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on January 08, 2017, 10:44:20 PM
@garry23

Quote
Could an alternative be to always test if a manual lens is flagged via event.post_shoot: which would always create an xmp after an image has been taken?

Sounds like a good idea! I am about to test magiclantern-manual_lens_info.2017Jan08.60D111 . When shooting a burst of several pics only the last picture taken produces an xmp sidecar file. One xmp for each picture would be preferable!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 09, 2017, 02:16:01 AM
Config saving is working as expected on my end. Pushed a couple of other fixes...

Working on my end now too. Last lens used is highlighted when camera starts. The thing is that you have to press set. Doing a shutter half-press will not erase the lens selection menu and gets you into a sort of weird state and it is sometimes easier to restart the camera than to keep punching buttons to get out of it.

Pulled in the fixes and Lens Info/Prefs shows the manual lens but other fields that maybe should be filled in show up as "(no lens)".

(https://c6.staticflickr.com/1/650/31382238293_81e840a1a3.jpg)

The lens name is too long for the "Manual Lens" field but that's how the lens profiles are named by Adobe. Here are the manual lenses that I own that have Adobe lens profiles:

(https://c2.staticflickr.com/1/771/31382238153_d36f8801e3.jpg)

Actually, I don't have any "Samyang" branded lenses, they are all Bower or Rokinon (whatever was on sale), and they are all cine lenses so I guess if I'm hacking profiles I could also shorten the names. However, Zeiss ZF.2 lenses are working without having to modify the profiles so we'll just have to deal with long names.

I've been using LiveView so much that I almost missed that the lens appears on the regular screen. All lens information is missing from the ML LiveView screen when a manual lens is mounted.

(https://c6.staticflickr.com/1/502/31386929653_4ae39bd1e3.jpg)

...When shooting a burst of several pics only the last picture taken produces an xmp sidecar file...

Reproduced that issue here.

(https://c3.staticflickr.com/1/599/32048606962_bc98aa7a1c.jpg)

Also tried turning off Image Review off and no XMP file was generated. I usually keep Image Review off so that might be why I didn't realize that XMP was working a couple of months ago. It would be great if another trigger for saving the XMP could be used.

(https://c3.staticflickr.com/1/572/32159382146_e62049e00e.jpg)

Finally, I spent way too much time today tinkering with a feature that's already working but could be just a little better. The formatting of the XMP file:

Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
        exif:FNumber="14/10"
    exif:FocalLength="50/1"
    aux:Lens="Zeiss Planar T* 1.4/50 ZF.2"
>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Notice the extra indent of the "exif:FNumber" field and the widowed ">" tag element. I'd like it to look like this:

Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FNumber="14/10"
    exif:FocalLength="50/1"
    aux:Lens="Zeiss Planar T* 1.4/50 ZF.2">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

The indent issue was easy to solve by removing the extra indentation in scripts/lib/xmp.lua but couldn't figure out how to remove the last newline. In any case, no big deal because it is working. Just spent too many years as a film editor obsessing over every minor detail. How it looks is everything in the film industry.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on January 09, 2017, 04:07:32 AM
Could an alternative be to always test if a manual lens is flagged via event.post_shoot: which would always create an xmp after an image has been taken?
event.post_shoot doesn't actually work. The module CBR it uses isn't actually implemented in the ML core. Implementing it correctly is actually probably a fair amount of work. There are lots of tricky situations to deal with. Burst mode will probably be a challenge.


To me, the xml format issue is not worth the hassle and extra code necessary to fix.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 09, 2017, 08:09:11 AM
@dmilligan

I must say, I and I don't wish to appear negative, at the moment I will most probably drop using the lens info script.

For timelapse, where I tend to use manual lenses, the ISO and shutter time are recorded, so holy grail shooting should be ok. There are also other ways to fix the missing FL and apperture EXIF that should work in LR, eg lens tagger: but I haven't used that approach for a while.

I will keep an eye on developments and might come back to the script approach later.

Cheers

Garry
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 09, 2017, 08:51:31 AM
@garry23

At least the manual_lens_info builds will give you the latest lua fixes. Of course there's nothing stopping you from writing your own variation of the script, you are pretty good at that!

I was thinking about timelapse work and how some photographers untwist their lenses to prevent flickering. The information for an EF lens can be put into the script so that ACR can automatically detect the lens even if that lens wasn't communicating with the body.

To me, the xml format issue is not worth the hassle and extra code necessary to fix.

How about this? It looks good and works with ACR:

Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FocalLength="50/1"
    aux:Lens="Samyang 50mm f/1.4 AS UMC"
    exif:FNumber="14/10"
    >
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

To get this was really quite simple:

scripts/lib/xmp.lua
Code: [Select]
-xmp.property_format = [[
-    %s="%s"
-]]
+xmp.property_format = [[%s="%s"
+    ]]

Of course you can decide if you want pretty code or pretty xmp files.

The version of the lens.lua script that I'm using for testing includes all the Zeiss ZF.2 and Samyang lenses with lens profiles that are included in ACR. It also covers Rokinon and Bowers and the Cine variations. I was wondering if you would consider adding it so it will appear in the experimentals downloads (https://builds.magiclantern.fm/experiments.html). The lenses are all commented out so the user simply needs to comment out or remove "My Lens" and "My Other Lens" and uncomment the lenses to include. There's some hacking necessary for the Samyang lens profiles to auto detect in ACR which I have documented but the Zeiss ZF.2 lenses are working fine.

Should I submit pull requests for these changes?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 09, 2017, 02:33:00 PM
Dan

Yes, I had already decided to use manual_lens_info to access the 'latest' Lua fixes :)

I have done the lens twist trick myself, so a good point.

As I said, I'll most probably relook at LR Lens Tagger, ie can change EXIF from within LR.

Cheers

Garry
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 09, 2017, 10:53:23 PM
@all

In case you hadn't picked up on LensTagger, it adds in FL and aperture, and other EXIF data, from within LR.

It works for me :)

http://www.essl.de/wp/software/lenstagger-lightroom-plugin/ (http://www.essl.de/wp/software/lenstagger-lightroom-plugin/)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on January 10, 2017, 12:55:25 AM
Tested the latest experimental magiclantern-manual_lens_info.2017Jan09.60D111 and added the following lens definition to lens.lua (taken from dforts script, but altered manual_aperture from 3.5 to 5.6).

Quote
    { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS",    focal_length =   8, manual_aperture = 5.6 },

Corresponding xmp (part):
Code: [Select]
<rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
        aux:Lens="Samyang 8mm f/3.5 UMC Fish-Eye "
    exif:FocalLength="8/1"
    exif:FNumber="49/10"
>

Some findings:

1: In XMP aux:Lens seems to be cut after a certain number of chars.
Code: [Select]
aux:Lens="Samyang 8mm f/3.5 UMC Fish-Eye ""CS" is missing at the end of the string.

I understood from dforts investigations in Adobe's forum that there is a general issue in automatically identifying Samyang lens profiles. But could it be of help to have the full lens name in the string as Adobes profile definition is working with the following parameters:
Code: [Select]
       stCamera:Make="Canon"
       stCamera:CameraRawProfile="True"
       stCamera:Lens="8mm f/3.5"
       stCamera:CameraPrettyName="Canon"
       stCamera:LensPrettyName="Samyang 8mm f/3.5 UMC Fish-Eye CS"
       stCamera:ProfileName="Adobe (Samyang 8mm f/3.5 UMC Fish-Eye CS, Canon)"

2: I chose a manual aperture 5.6 and was expecting "exif:Fnumber = 56/10" but in XMP it is "49/10".  Any hint why?
When setting manual_aperture = 3.5  exif:Fnumber is "35/10".

3: Import to Lr6 brings another issue: Focal lenght is 50mm instead of 8mm, whereas aperture is shown as f/4.9
In xmp I altered exif:FocalLength manually to different values != 50mm, for test reasons. But Lr did not except anything and set it back to standard value 50mm.  The  lens profile was not recognized. Does this cause a dummy focal length?

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 11, 2017, 03:13:20 AM
@JohanJ -- Excellent reporting.

1:

I can confirm that long lens names are being cut off in the XMP file. All of the Zeiss ZF.2 lenses should be automatically detected by ACR but I found one that wasn't working. It looked good in the lua script:

Code: [Select]
{ name = "Zeiss Makro-Planar T* 2/100 ZF.2",     focal_length = 100, manual_aperture = 2   },
It shows the whole name in the Lens Info/Prefs menu:

(https://c4.staticflickr.com/1/341/32238298915_32edf6d412.jpg)

But it is getting cut off in the XMP file:

Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FocalLength="100/1"
    exif:FNumber="20/10"
    aux:Lens="Zeiss Makro-Planar T* 2/100 ZF.">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Note that the "2" is cut off the end of the lens name. Also note how pretty the XMP formatting looks with the latest changes (https://bitbucket.org/hudson/magic-lantern/pull-requests/802/slight-adjustment-of-xmp-indentation/diff).

My understanding is that Adobe lens profiles have several fields that can be used to automatically match the lens with the profile. The software can probably work around cut off lens names by matching up a unique lens ID numbers. You can look up these numbers in the exiftool source code. However, some manual lenses don't have unique lens ID numbers so we're probably stuck with using the lens name.

2:

I couldn't reproduce the manual aperture issue.

lens.lua
Code: [Select]
{ name = "Samyang 8mm f/3.5 UMC Fish-Eye CS",    focal_length =   8, manual_aperture = 5.6 },
XMP
Code: [Select]
    exif:FocalLength="8/1"
    exif:FNumber="56/10"
    aux:Lens="Samyang 8mm f/3.5 UMC Fish-Eye ">

For me it showed up as 56/10 and not 49/10 as in your example.

3:

Can't reproduce that one either. I'm not sure how you got it into Lightroom but it takes some lens profile hacking on the Samyang lenses so this is what I did to work around your first issue.

Instead of using the entire name, shorten it:

Code: [Select]
{ name = "Samyang 8mm f/3.5 Fish-Eye",           focal_length =   8, manual_aperture = 5.6 },
Use that exact same name in the lens profile--note that you don't need to rename the lens profile's file name, only the stCamera:Lens field:

Code: [Select]
stCamera:Lens="Samyang 8mm f/3.5 Fish-Eye"
This is what I get in Lightroom:

(https://c7.staticflickr.com/1/765/32089912662_6def7af03b.jpg)

Sifting through the EXIF information in the CR2 file I found this:

Code: [Select]
F Number                        : 0
Aperture Value                  : inf
Focal Length                    : 0.0 mm
Lens Info                       : 0mm f/0
Lens Model                      :
Lens Serial Number              : 0000000000
Lens                            : 0.0 mm
Focal Length                    : 0.0 mm (35 mm equivalent: 0.0 mm)
Hyperfocal Distance             : inf m
Lens                            : 0.0 mm (35 mm equivalent: 0.0 mm)

Lightroom is probably not reading your XMP file and it might be defaulting to a 50mm lens perhaps to avoid any divide by zero operations.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on January 11, 2017, 03:27:00 AM
In ML core, the lens name has a max length of 32 characters (statically allocated). We can make it larger, but it will still get cut off in MLV files (and that can't be fixed without a breaking change to the spec).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 11, 2017, 08:02:28 AM
Ah, but isn't "Zeiss Makro-Planar T* 2/100 ZF.2" exactly 32 characters? We're getting shorted one character in the XMP while it shows up properly in the Lens Info/Prefs menu.

The long Samyang names we can deal with because we need to hack the Adobe lens profiles anyway but it would be good if we can work with the Zeiss lenses so those profiles don't need to be hacked.

Code: [Select]
Zeiss Makro-Planar T* 2/100 ZF.2
12345678901234567890123456789012
         1         2         3
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on January 11, 2017, 12:51:00 PM
The last character of the string has to be the null terminator (0). So I slightly misspoke, the lens name is limited to 32 bytes (31 ascii characters). The menu is correct b/c the menu's value is set directly by from Lua (where the full string is defined). The xmp retrieves the value from ML core where the limitation exists.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on January 11, 2017, 01:56:27 PM
The PROP_LENS_NAME property (0x80030021) has a length of 70 characters.

The 32-char limit was chosen in 2009 (changeset b6dca6a7e55a (https://bitbucket.org/hudson/magic-lantern/commits/b6dca6a7e55a)), probably arbitrarily, and looks like nobody questioned it until now.

Changing the MLV spec would only break converters that use mlv_lens_hdr_t.lensSerial (the field after lensName), which is currently filled with spaces. The potential breakage is minor (showing part of the lens name instead of serial number). So I'd say, let's change it.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on January 11, 2017, 03:28:26 PM
possibilities to choose from:

a) redesign LENS to use e.g. name[128] and serial[32]
b) modify LENS to keep the basic format and resize name[48] and serial[16]
c) modify LENS to remove serial and just have lens[64]
d) keep the old LENS for compatibility reasons (truncated name) and add a new optional block e.g. ELNS which has additions like the longer name, serial and other things you could need

solution a) requires a new major version for the MLV format and all tools will have to get updated

b) and c) could work well with some converters, would fail with others but won't probably need a major number change as it will:
-> work with capable readers
-> return weird results with old readers but only the lens serial would have weird values
while c) would be odd for people who manually have lens serials (e.g. defined by LUA) or the feature somewhen really can be used in ML

solution d) - which to be honest is my favorited way to go - will
-> work with capable readers
-> work with old ones as before, just having an unknown block 'ELNS' to them (which every old reader should just ignore silently)

so what to pick?

-> added ELNS proposal https://docs.google.com/spreadsheets/d/1ItNuLj34JlK6bZgiAJLrmG7p3iyCmCju4XYyiwjqJOM at the bottom
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 11, 2017, 05:02:16 PM
Is there a length limit on the lens field specified by some of EXIF and/or Adobe XMP document? I guess I could run some tests and see how long I could make the name in the XMP and lens profile before it breaks.

@g3gg0 - Back in Reply #88 (https://www.magiclantern.fm/forum/index.php?topic=18083.msg175652#msg175652) I think you concluded that the lens serial number is not included in the property. I just ran a test with a build that has the latest lens_info_fix and it is still not showing up.

Code: [Select]
Block: LENS
  Offset: 0x0000014c
    Size: 96
    Time: 0.003000 ms
     Name:        'EF-S17-55mm f/2.8 IS USM'
     Serial:      ''
     Focal Len:   17 mm
     Focus Dist:  65535 mm
     Aperture:    f/2.80
     IS Mode:     14
     AF Mode:     0
     Lens ID:     0x000000F0
     Flags:       0x00000000

I agree with your preference for solution d). It seems to be open-ended and would cause the least amount of disruption.

[EDIT] @dmilligan -- as an alternative to using the lens name it should be possible to use the lens ID. These are listed in the exiftool source code and in many of the Adobe lens profiles. I'll need to run some tests to see how this works in ACR.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 12, 2017, 12:49:03 AM
Some finding on limits to the lens name. Basically, there is no limit on lens profiles and XMP files. I checked the XMP documentation and there is a limit on the size of the file (64kb for JPEG files) but there isn't a limit on each field.

I tested this with the XML "aux:Lens" and lens profile "stCamera:Lens" fields. These are the fields that ACR uses to automatically detect the lens. I tried it with up to 1,000 characters and it still worked. Note that ACR displays the "stCamera:LensPrettyName" and "stCamera:ProfileName" but not the "stCamera:Lens" so it can be anything at all--as long as it doesn't start with a number, found that out by trial and error.

Another test was trying to get auto detection to work in ACR using the lens ID number, but no success so far.

I also looked up EXIF and IPTC documentation and I could not find any limit, though exiftool will cut certain fields off at around 250 characters.

http://www.sno.phy.queensu.ca/~phil/exiftool/ancient_history.html
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on January 12, 2017, 01:31:50 AM
Now working with magiclantern-manual_lens_info.2017Jan11.60D111

@dfort:
I followed your advise and tweaked a user own lens profile by setting "stCamera:Lens" to the exact same value as "aux:Lens" comes with in xmp. And it works fine! Lr chooses my own profile automatically! Thanks! (I tweaked also "stCamera:ProfileName" to "...BS" instead of "CS" just to see which profile was picked).

https://www.flickr.com/photos/147593195@N03/shares/8815Cp

Also the aperture value is shown correctly. I can't reproduce the original artifact which is strange. Consider it solved.

Unfortunately the focal length is still the wrong one! Lr switches to 50mm, no matter what!
https://www.flickr.com/photos/147593195@N03/shares/r9nXi7

I will continue investigating from here.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 12, 2017, 02:10:15 AM
I tweaked also "stCamera:ProfileName" to "...BS" instead of "CS"
;D

If you use the img tags your screenshot will appear on the forum:
Code: [Select]
[img]https://www.flickr.com/photos/147593195@N03/shares/8815Cp[/img]Though this doesn't seem to work unless you make your file public.

Strange thing you got going on with the 8mm fisheye showing up as a 50mm. It is working fine over here. I'm on a Mac using Adobe Camera Raw 9.8 and Lightroom 2015.8 (hum, all the other CC apps are on 2017). What are you running? Upload your XML and hacked lens profile somewhere and post a link so I can see if it happens on my system.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 16, 2017, 06:32:22 PM
@dmilligan David, if you catch this: a question regarding a previous reply you made http://www.magiclantern.fm/forum/index.php?topic=18083.msg177889#msg177889

Where do I find more about the constants such as GUISTATE_QR, that you use in the .xmp module.

I can see guistate in the api reference, but not qr.

I'm trying to better understand/exploit Lua.

Cheers

Garry
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 16, 2017, 06:39:09 PM
@garry23 - @dmilligan is on tapatalk so if you use the @ sign before his user name like I just did tapatalk will send him an alert. Quoting his post will also send an alert.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 16, 2017, 06:44:09 PM
Dan thanks, I truly didn't know that  :o

Maybe I should get on taptalk
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on January 16, 2017, 08:32:52 PM
gui_common.h (https://bitbucket.org/hudson/magic-lantern/src/1d8c97a6b11a065ca02e808ff32f7eff2859c8a3/src/gui-common.h?at=unified)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: garry23 on January 16, 2017, 08:45:16 PM
@dmilligan

Thank you.

Is this a general principal, that you can tap into other property state constants, assuming they exist and you know where to look?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on January 29, 2017, 08:46:25 PM
@dfort
Quote
Strange thing you got going on with the 8mm fisheye showing up as a 50mm.

Today I found the reason for it: the 60D writes 50mm into EXIF:FocalLength embedded in the CR2 file

EXIF info from a sample picture using magiclantern-manual_lens_info.2017Jan26.60D111:
Code: [Select]
exiftool - FocalLength IMG_6397.CR2 ======== IMG_6397.CR2  Focal Length                    : 50.0 mm


I also tested the latest magiclantern-manual_lens_info.2017Jan26.100D101 using a 100D instead of the 60D.
Code: [Select]
exiftool - FocalLength IMG_3743.CR2 ======== IMG_3743.CR2  Focal Length                    : 0.0 mm

For the 100D everything just worked as expected: XMP file contained all necessary information of the manual lens, import to Lr (6.8 on Win10) had no problem at all, and my proper lens profile for this lens was found immediately. And even  FocalLength 8 mm was shown and used correctly in the processed xmp!

Now the remaining question is how to make Lr use FocalLength from the xmp file created by ML instead of existing (but dummy) information embedded in the CR2 file?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 30, 2017, 06:27:35 AM
Today I found the reason for it: the 60D writes 50mm into EXIF:FocalLength embedded in the CR2 file
...
Now the remaining question is how to make Lr use FocalLength from the xmp file created by ML instead of existing (but dummy) information embedded in the CR2 file?

Interesting finding. I would have thought that the xmp file would take precedence over the EXIF tags in the CR2 file. Just in case that modifying a CR2 with exiftool doesn't give the same results could you please upload a sample CR2 with the manual lens xmp from the 60D so we can take a look at it?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on February 04, 2017, 07:17:01 PM
I would have thought that the xmp file would take precedence over the EXIF tags in the CR2 file. Just in case that modifying a CR2 with exiftool doesn't give the same results could you please upload a sample CR2 with the manual lens xmp from the 60D so we can take a look at it?

I can confirm that EXIF tag FocalLength = 50 set by 60D is the reason for the wrong focal length in Lr. Modifying a sample CR2 using exiftool -FocalLength=0 did the trick. Import to Lr showed the correct value from the xmp file.

Conclusions
 1. Lr chooses EXIF tags from CR2 files over xmp values if not initial (=0)
 2. There are different default values for different models. For 60D it is 50mm for 100D it is initial. I checked even som older pics taken with a 350D and there it was initial too.

Sample shots (w/ and w/o modified EXIF) for 60D and 100D, as well as a modified Lr lens profile for Samyang 8 mm can be found here (see File_info.txt for details):
https://drive.google.com/open?id=0B7N2Wbxd7m2gS293V1Z3a2hqYVk
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on March 05, 2017, 10:40:39 PM
(https://c3.staticflickr.com/1/735/32049369666_7a747b4c51.jpg)

Just got this screen on 5D3.

Setup: thread-safety branch (which includes lua_fix), 5D3 1.2.3, mlv_lite loaded, movie mode. Starting from photo mode and then switching to movie is OK. As you noticed, backing out 56464bc solves it, but why?

The mystery goes on...

Edit: solved (https://bitbucket.org/hudson/magic-lantern/commits/55b1c8a22f6ce383faabac608712a77a7c949f98) - the info bars layout was first done with FONT_MED_LARGE and then replaced with a centered version of the same font. The linked commit did not cause the bug - it simply prevented a layout refresh, so the bug became visible. Classical code duplication bug...
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on April 06, 2017, 12:30:42 PM
Thanks Alex for updating the experimental a lens info branch
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on April 06, 2017, 03:57:52 PM
100D is missing. Build seems to be broken since merging w lua_fix.

Sent from my SM-T719 using Tapatalk

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on April 06, 2017, 06:29:30 PM
@JohanJ -- Where did you get a Non-CPU lens info build for the 100D? The reason it is missing from the Experiments download page is because the 100D hasn't been merged into unified yet. My understanding is that the 100D is next in line and when that happens we'll see it show up with the other experimental builds.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on April 06, 2017, 09:58:33 PM
A1ex had included 100D on the experiments page for branches lua_fix and manual_lens_info. The latter 25th of jan 2017. See my posts for the focal lens info problem on 60D xmp files (50mm fixed in Lr) compared to 100D (initial focal length and therefore working fine in Lr)

Sent from my SM-T719 using Tapatalk
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on April 07, 2017, 08:58:22 PM
Right -- so the 60D puts 50mm in the EXIF lens field when a non-cpu lens (or no lens) is attached and LR uses that instead of the information in the xmp file, is that right? Sorry for the delay but I'm trying to catch up and just checked your files.

Don't know what to do about it because the whole point of writing the XMP file is to avoid having to figure out how to write EXIF data to the CR2 file.

I believe that a while back there was some code in ML that defaulted to 50mm, wonder if that is still the case and if that's causing the issue. I did a quick search of the code and found an instance of 50mm in raw2dng.c which shouldn't be related to this issue.

A few things to look into:
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on April 09, 2017, 12:50:37 AM
Here is a screenshot to illustrate the 60D issue:

(https://c1.staticflickr.com/4/3931/33790318451_7ae5fce8f7_z.jpg)

Focal length is 8mm in the xmp and 50mm in the CR2 EXIF. If it is 0mm in the CR2 EXIF then LR takes the focal length from the xmp file.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: DeafEyeJedi on April 09, 2017, 01:32:40 AM
Maybe I'm a bit of a doofus but still haven't wrapped my head on how to use this with my manual lenses yet? Any recommendations on a start-up guide or some sort of a 'owners manual' upon experimenting these builds?

Been eyeing on this project of yours (that @Lars Steenhoff started for good reasons) for quite awhile... It needs some vintage love!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on April 09, 2017, 01:58:50 AM
Load the lua.mo module, restart, run the lens script (probably want to put it on auto run). Attach manual lens.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on April 09, 2017, 07:36:41 PM
Right -- though the lens.lua script that is packaged with the experimental build is only good for a very basic test:

Code: [Select]
lenses =
{
    { name = "My Lens", focal_length = 50 },
    { name = "My Other Lens", focal_length = 25 },
}

I published a script that includes several popular Zeiss and Samyang manual lenses back in Reply #134 (http://www.magiclantern.fm/forum/index.php?topic=18083.msg177735#msg177735).

This project was started in order to do automatic lens corrections in Lightroom. Right now the only lenses that are working properly without having to hack into the lens profiles are the Zeiss ZF.2 primes. Well, almost. One of the Zeiss lenses has a name that exceeds the number of characters allocated by ML for the lens name.

I've been meaning to get back to this project once we figure out how to deal with that lens name issue. It requires some work on the ML back ends and also a change to the MLV specifications. There are other issues if you want true automatic lens correction. One of the corrections that Lightroom can do is to eliminate lens vignetting. However, the vignetting varies depending on the lens aperture used. Obviously it isn't all that practical to have to go into the menu to manually change the lens setting every time you adjust the f/stop.

There is another issue saving xmp files for every single frame if you shoot outside of LiveView or if you shoot bursts of CR2. In addition, @JohanJ discovered that the 60D (and possibly other camera) will default to 50mm in the EXIF focal length and Lightroom uses this information instead of what the script is writing to the xmp file.

If you are shooting silent DNG frames, forget it. This script doesn't write to the EXIF or create an xmp for DNG files.

So yeah, there's more to do here but if you're just looking for a way to keep track of which manual lens you used when shooting MLV, it works pretty good.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: histor on April 24, 2017, 12:26:08 PM
It sounds quite on the contrary to the existing solution but…

We know how programmable EMF chips work – they just return several predefined values (f-number, FL, focus prompt compensation). Instead, let's imagine one could hack this values directly in-camera. We'll get immediately working focus confirmation (+ focus correction) and enough EXIF data to identify the lens (you can add the proper lens names with Exiftool later).

As for the AF-MF switch - it’s nothing but a software switch. It’s easily overridden (with API commands, for example). Newer EMF chips simulate AF mode now.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 24, 2017, 03:28:16 PM
@dfort are you able to make a build that has some of the newer features enabled?

10/12/14 bit compressed
3x crop 1080p
Lens info
MLV sound

I'm looking to use the Zeiss lenses with it.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 24, 2017, 08:23:59 PM
I see that you got part of your question answered, MLV sound isn't working with 10/12/14 bit compressed. At least not yet.

Have you tried the Lens info lua script with the latest crop_rec_4k? I'm at work right now so I can't try it but it looks like the lua fixes needed for that script might have already been merged into that branch.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 24, 2017, 11:29:46 PM
I'm working on it,  I managed to set up builds for my Mac with Sierra

Now I'm trying to make a build with crop rec that has sound + the Lua fixes.


Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 25, 2017, 12:00:13 AM
I dont know how to solve the merge conflicts, so I will have to wait.

I tried to import the lua_fix branch into the crop_rec branch.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 25, 2017, 11:06:09 AM
https://bitbucket.org/hudson/magic-lantern/pull-requests/822/manual-lens-info-lua/diff (https://bitbucket.org/hudson/magic-lantern/pull-requests/822/manual-lens-info-lua/diff)

Do you mean these fixes?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 25, 2017, 02:24:06 PM
That looks right. The lua_fix branch was already merged into the crop_rec_4k branch so all you needed to do was to merge manual_lens_info into crop_rec_4k--there were no merge conflicts when I tried it. If you want to keep things in order I'd recommend first making a branch of crop_rec_4k (I named my branch crop_rec_4k_manual_lens_info (https://bitbucket.org/daniel_fort/magic-lantern/branch/crop_rec_4k_manual_lens_info)). You can pull it from my bitbucket repository and build it yourself or for a limited time only I'm making it available on my bitbucket download page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/) for the 5D3.113/123.

On my builds I also added the Zeiss ZF.2 and Samyang (Rokinon/Bower/etc.) manual lenses that are in the Adobe Camera Raw database. You'll have to edit ML/scripts/lens.lua to comment out the test lines and uncomment the lines for your lenses.

I've been messing around all over the place in ML and not really concentrating on one project--like the manual lens info script. I think where we left off there was a problem with a length limit on the lens name in the MLV header. All of the Zeiss lenses were working except the 100mm Macro because the lens name is one character too long. To get that lens working required hacking the name in the ACR lens profile. For the Samyang lenses it takes some more work to get it working with ACR. If your goal is just to get the lens metadata into your MLV files, that part is working fine as long as you keep the lens name to less than 32 characters.

I did some more research into lens profiles and to do it right you should take into consideration the aperture used for each shot in order to properly compensate for vignetting and distortion. Of course in most situations it would be rather impractical to reset your camera every time you adjust the aperture.

In any case, please report your findings to that pull request you pointed out. Feedback is important in order to get new features merged into the main code base.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 25, 2017, 02:59:10 PM
Thanks for the branch with the Lens info.

for me it fine without the 100 Zeiss for now as I don't have that one.
The main use for me is to have a name that is descriptive and the focal length,   so I can apply distortion correction for the manual lenses.

The one I tried to merge with the lens info was the crop_rec branch.
https://bitbucket.org/hudson/magic-lantern/branch/crop_rec

that one has sound but its a bit older I know that.

For me sound + crop 3x + lens info are the most important.

Compression and higher resolutions are nice but not essential.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Danne on August 25, 2017, 03:05:16 PM
Create a branch of regular crop_crec and merge manual lens info to that not working?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 25, 2017, 03:11:39 PM
@Danne,
I tried that and have 3 merge conflicts in these files

scr/debug.c
scr/module.c
src/module.h

I can try to see if I can use the files from crop_rec to resolve the merge conflicts.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Danne on August 25, 2017, 03:14:32 PM
I see. You can try and resolve them in atlassian(source tree). Not sure which way 'mine' or 'theirs' but some testing should reveal what's going on :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 25, 2017, 03:35:24 PM
I made a build with resolve using mine  (crop_rec)

and when I run it on the cam I get

duplicate menu error module  ( 32 )

I saw some fixes in the crop4k for duplicate menu errors, maybe I have to cherry pic some thing from there.



Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Danne on August 25, 2017, 03:49:36 PM
And using 'theirs' is giving?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 25, 2017, 03:53:42 PM
Thats what source tree uses as a name, ( resolve conflict using mine, is keep current file from crop_rec and ignore file from manual_lens_info )

Does it make sense what I mean?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Danne on August 25, 2017, 04:02:35 PM
I'm no expert but I usually succeed resolving either using mine or theirs. Pretty shallow advice but if nothing works you seem to be able to get yourself into the code :).

Your description above is nothing I know about. Did you create a branch that you're working from?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 25, 2017, 07:19:30 PM
For me sound + crop 3x + lens info are the most important.

g3gg0 has been working on a branch to get mlv_snd working with crop_rec_4k (https://bitbucket.org/hudson/magic-lantern/branch/crop_rec_4k_mlv_snd). Not sure if that is ready for testing yet.

Have you tried H.264 proxy? It is under the Advanced settings. That will get you audio along with a .mov file for on set playback. You'll have to work a bit to sync it in post but the workflow isn't all that bad.

I haven't tackled those merge conflicts yet but I'd suggest trying out the crop_rec_4k_manual_lens_info build before spending too much time on it.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 25, 2017, 07:45:17 PM
Yes I tried the crop_rec_4k_manual_lens_info build on firmware 1.1.3 jus now and it works fine.  :)

The lens info is embedded and photoshop recognises the Zeiss profile and can apply the corrections.
The other benefit is organisational, I can search the footage by lens.

I did try to use the h264 but that was not stable. also there is the offset in sound that is not nice to deal with for each shot to match them up.

I also did the same merge you did, just I don't know why the mlv rec and mlv sound don't compile from this branch.

If you can help with the merge conflicts, I can make builds now, but resolving conflicts is bit sticky for me as I don't know much of what the code does.  8)

https://bitbucket.org/hudson/magic-lantern/branch/crop_rec_4k_mlv_snd (https://bitbucket.org/hudson/magic-lantern/branch/crop_rec_4k_mlv_snd)
This branch does not compile the Lua module, so I'm out of luck there for now
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 25, 2017, 08:49:09 PM
The only thing that does not work is the auto run on startup of the Lua script
https://bitbucket.org/daniel_fort/magic-lantern/branch/crop_rec_4k_manual_lens_info (https://bitbucket.org/daniel_fort/magic-lantern/branch/crop_rec_4k_manual_lens_info)

the script shows up but when touching a button live view turns black and the lens selection menu disappears.

It works fine on its own without the autorun
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 25, 2017, 09:34:57 PM
I have managed to merge manual_lens_info branch into a copy of the branch for crop_mlv, by resolving some conflicts by more or less guessing.

https://www.dropbox.com/s/88vc19s41o5ga0c/magiclantern_crop_rec_manual_lens_info.2017Aug25.5D3113.zip?dl=0

this build has
sound,
mlvrec,
crop 3x 1080,
and lens info.

the same problem is here that is does not autorun the Lua script on start, because the live view takes over
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 26, 2017, 12:11:24 AM
Are you sure it isn't running? There have been some changes to it so it pretty much runs only when needed. Try putting on a CPU lens then remove it. The manual lens selection menu should come up.

I spent some time messing around with crop_rec_4k_mlv_snd, even merged it with manual_lens_info but it isn't quite ready to release into the wild.

I got a build with all of the items from your wish list. For some reason the silent module keeps failing. I'm looking into it and will post something to my download page soon, maybe without the silent module since it isn't on your list.

https://bitbucket.org/daniel_fort/magic-lantern/branch/raw_video_10bit_12bit_manual_lens_info

[EDIT] Tried your build and it is working better than mine so you should go with it.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 26, 2017, 12:48:20 AM
Cool, I will also make a build from your branch

My build has problems with mlv_play.
I get a stack overflow free=0 used=16384 error in red when trying to playback.

As for the lens without cpu, When I turn on the camera with the Lua script enabled it shows up for a moment, but I can't interact with it.
It works well when the camera is already on and I activate from the menu.

I'm looking forward to see the results of crop_rec_4k_mlv_snd. This seems the most future proof way to go. I'm not sure how far of it is before it can be used well.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 26, 2017, 01:43:32 AM
My build has problems with mlv_play.

You might have to compile raw_twk for mlv_play to work. That dependency has been eliminated in the crop_rec_4k branch.

Yeah, it is a wild ride when you're merging development branches. I'm also looking forward to mlv_snd working with 4k raw lossless compression. With most of this stuff I can only cheer on the developers.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 27, 2017, 12:55:08 PM
I just made a new build from the latest crop_rec4k and merged manual_lens_info into it.
Also merged the aspect ration fix that allow mlv_rec to build.

When I try to autorun the script it gives the screen for a split second and then it is gone.


www.youtube.com/watch?v=TiT4QAGaOec (http://www.youtube.com/watch?v=TiT4QAGaOec)

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on August 27, 2017, 03:03:12 PM
hi guys. thanks for this. how do i add my manual lens to the list? any tips would be welcome.
Also had 3 crashes on my camera when using this. where should i upload the logs, if it matters. thanks in advance
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 27, 2017, 03:09:58 PM
hi guys. thanks for this. how do i add my manual lens to the list? any tips would be welcome.
Also had 3 crashes on my camera when using this. where should i upload the logs, if it matters. thanks in advance

You open the script lens.lua in a text editor in the scripts folder of Magic Lantern, and add new lenses there from line 11.  the -- is to disable the ones you don't need.

lens.lua
Code: [Select]
-- Manual Lens Information
-- Supplies lens information for manual lenses
-- Whenever there is a non-chipped lens detected, we prompt the user to select the attached manual lens from a list

require("ui")
require("config")
require("xmp")

lenses =
{
--  Zeiss ZF.2 manual lenses
--    { name = "Zeiss Distagon T* 2.8/15 ZF.2", focal_length = 15, manual_aperture = 2.8 },
--    { name = "Zeiss Distagon T* 3.5/18 ZF.2", focal_length = 18, manual_aperture = 3.5 },
--    { name = "Zeiss Distagon T* 2.8/21 ZF.2", focal_length = 21, manual_aperture = 2.8 },
--    { name = "Zeiss Distagon T* 2.8/25 ZF.2", focal_length = 25, manual_aperture = 2.8 },
    { name = "Zeiss Distagon T* 2/25 ZF.2", focal_length = 25, manual_aperture = 2 },
--    { name = "Zeiss Distagon T* 2/28 ZF.2", focal_length = 28, manual_aperture = 2 },
    { name = "Zeiss Distagon T* 2/35 ZF.2", focal_length = 35, manual_aperture = 2 },
--    { name = "Zeiss Distagon T* 1.4/35 ZF.2", focal_length = 35, manual_aperture = 1.4 },
    { name = "Zeiss Makro-Planar T* 2/50 ZF.2", focal_length = 50, manual_aperture = 2 },
    { name = "Zeiss Planar T* 1.4/50 ZF.2", focal_length = 50, manual_aperture = 1.4 },
    { name = "Zeiss Planar T* 1.4/85 ZF.2", focal_length = 85, manual_aperture = 1.4 },
--    { name = "Zeiss Makro-Planar T* 2/100 ZF.2", focal_length = 100, manual_aperture = 2 },
--    { name = "Zeiss Apo Sonnar T* 2/135 ZF.2", focal_length = 135, manual_aperture = 2 },
}

selector_instance = selector.create("Select Manual Lens", lenses, function(l) return l.name end, 600)

xmp:add_property(xmp.lens_name, function() return lens.name end)
xmp:add_property(xmp.focal_length, function() return lens.focal_length end)
xmp:add_property(xmp.aperture, function() return math.floor(camera.aperture.value * 10) end)

function property.LENS_NAME:handler(value)
    if lens.is_chipped == false then
        task.create(select_lens)
    else
        if selector_instance ~= nil then
            selector_instance.cancel = true
        end
        xmp:stop()
    end
end

function select_lens()
    if #lenses > 1 then
        local menu_already_open = menu.visible
        if not menu_already_open then
            menu.open()
            display.rect(0, 0, display.width, display.height, COLOR.BLACK, COLOR.BLACK)
        end
        if selector_instance:select() then
            update_lens()
        end
        if not menu_already_open then
            menu.close()
        end
    elseif #lenses == 1 then
        update_lens()
    end
end

function update_lens()
    for k,v in pairs(lenses[selector_instance.index]) do
        lens[k] = v
    end
    xmp:start()
end

lens_menu = menu.new
{
    parent = "Lens Info Prefs",
    name = "Manual Lens",
    help = "Info to use for attached non-chipped lens",
    icon_type = ICON_TYPE.ACTION,
    select = function()
        if lens.is_chipped == false then
            task.create(select_lens)
        end
    end,
    rinfo = function()
        return lens.name
    end,
    warning = function()
        if lens.is_chipped then
            return "Chipped lens is attached"
        end
    end
}

if lens.is_chipped == false then
    task.create(select_lens)
end
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on August 27, 2017, 03:33:38 PM
thanks for your quick reply.
Are we sorting some sort of list of lenses for the future to act like a "db"? Guess it would be easier to delete than to add manually.
Anyway heres my listing regarding Lensbaby lenses:

Code: [Select]
{
        { name = "LB Sweet 35", focal_length = 35 },
{ name = "LB Sweet 50", focal_length = 50 },
{ name = "LB Twist 60", focal_length = 60 },
{ name = "LB Edge 50", focal_length = 50 },
{ name = "LB Edge 80", focal_length = 80 },
{ name = "LB Circular Fisheye", focal_length = 5.8 },
{ name = "LB Soft Focus Optic", focal_length = 50 },
{ name = "LB Double Glass Optic", focal_length = 50 },
{ name = "LB Single Glass Optic", focal_length = 50 },
{ name = "LB Plastic Optic", focal_length = 50 },
{ name = "LB Pinhole Optic", focal_length = 50 },
{ name = "LB Fisheye Optic", focal_length = 12 },
{ name = "LB Velvet 56", focal_length = 56 },
        { name = "LB Velvet 85", focal_length = 85 },
{ name = "LB Creative Aperture", focal_length = 50 },
}
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 27, 2017, 04:12:13 PM
It will be good to have a doc with all the names somewhere.
you can edit this one

https://docs.google.com/spreadsheets/d/1OE4pbVJ7xzUZQ-X4dqu_NRoYguyIiAuSpGslqcnXzpQ/edit?usp=sharing

I made this one, at some point will be added to the magic lantern bitbucket.


Code: [Select]

--  Zeiss ZF.2 manual lenses Nikon mount - these work with the lens profiles that ship with Adobe Camera Raw

--  { name = "Zeiss Distagon T* 2.8/15 ZF.2",        focal_length =  15, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 3.5/18 ZF.2",        focal_length =  18, manual_aperture = 3.5 },
--  { name = "Zeiss Distagon T* 2.8/21 ZF.2",        focal_length =  21, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 2.8/25 ZF.2",        focal_length =  25, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 2/25 ZF.2",          focal_length =  25, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 2/28 ZF.2",          focal_length =  28, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 2/35 ZF.2",          focal_length =  35, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 1.4/35 ZF.2",        focal_length =  35, manual_aperture = 1.4 },
--  { name = "Zeiss Makro-Planar T* 2/50 ZF.2",      focal_length =  50, manual_aperture = 2   },
--  { name = "Zeiss Planar T* 1.4/50 ZF.2",          focal_length =  50, manual_aperture = 1.4 },
--  { name = "Zeiss Planar T* 1.4/85 ZF.2",          focal_length =  85, manual_aperture = 1.4 },
--  { name = "Zeiss Makro-Planar T* 2/100 ZF.2",     focal_length = 100, manual_aperture = 2   },
--  { name = "Zeiss Apo Sonnar T* 2/135 ZF.2",       focal_length = 135, manual_aperture = 2   },

--  Nikon lenses
--  { name = "Nikon Zoom Ais ED 50-300", focal_length = 300, manual_aperture = 4.5 },
--  { name = "Nikon AF NIKKOR 28mm f/1.4D", focal_length = 28, manual_aperture = 1.4 },


--  Samyang manual lenses - also branded as Rokinon and Bower. Cine versions use the same lens profile.
--  The lens profiles for Samyang manual lenses that ship with Adobe Camera raw must be modified in order
--  for automatic lens detection to work.
--  More information here: http://www.magiclantern.fm/forum/index.php?topic=18083.msg176261#msg176261

--  { name = "Samyang 8mm f/2.8 UMC Fisheye",        focal_length =   8, manual_aperture = 2.8 },
--  { name = "Samyang 8mm f/2.8 UMC Fisheye II",     focal_length =   8, manual_aperture = 2.8 }, --   8mm T3.1 Cine
--  { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS",    focal_length =   8, manual_aperture = 3.5 },
--  { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS II", focal_length =   8, manual_aperture = 3.5 }, --   8mm T3.8 Cine
--  { name = "Samyang 10mm f/2.8 ED AS NCS CS",      focal_length =  10, manual_aperture = 2.8 }, --  10mm T3.1 Cine
--  { name = "Samyang 12mm f/2 NCS CS",              focal_length =  12, manual_aperture = 2   }, --  12mm T2.2 Cine
--  { name = "Samyang 12mm f/2.8 ED AS NCS Fisheye", focal_length =  12, manual_aperture = 2.8 }, --  12mm T3.1 Cine
--  { name = "Samyang 14mm f/2.8 ED AS IF UMC",      focal_length =  14, manual_aperture = 2.8 }, --  14mm T3.1 Cine
--  { name = "Samyang 16mm f/2 ED AS UMC CS",        focal_length =  16, manual_aperture = 2   }, --  16mm T2.2 Cine
--  { name = "Samyang 21mm f/1.4 ED AS UMC CS",      focal_length =  21, manual_aperture = 1.4 }, --  21mm T1.5 Cine
--  { name = "Samyang 24mm f/1.4 ED AS IF UMC",      focal_length =  24, manual_aperture = 1.4 }, --  24mm T1.5 Cine
--  { name = "Samyang 35mm f/1.4 AS IF UMC",         focal_length =  35, manual_aperture = 1.4 }, --  35mm T1.5 Cine
--  { name = "Samyang 50mm f/1.2 AS UMC CS",         focal_length =  50, manual_aperture = 1.2 },
--  { name = "Samyang 50mm f/1.4 AS UMC",            focal_length =  50, manual_aperture = 1.4 }, --  50mm T1.5 Cine
--  { name = "Samyang 85mm f/1.4 AS IF UMC",         focal_length =  85, manual_aperture = 1.4 }, --  85mm T1.5 Cine
--  { name = "Samyang 100mm f/2.8 ED UMC MACRO",     focal_length = 100, manual_aperture = 2.8 }, -- 100mm T3.1 Cine
--  { name = "Samyang 135mm f/2 ED UMC",             focal_length = 135, manual_aperture = 2   }, -- 135mm T2.2 Cine
--  { name = "Samyang 300mm f/6.3 ED UMC CS",        focal_length = 300, manual_aperture = 6.3 },


--  Lensbaby lenses

--  { name = "LB Sweet 35", focal_length = 35 },
--  { name = "LB Sweet 50", focal_length = 50 },
--  { name = "LB Twist 60", focal_length = 60 },
--  { name = "LB Edge 50", focal_length = 50 },
--  { name = "LB Edge 80", focal_length = 80 },
--  { name = "LB Circular Fisheye", focal_length = 5.8 },
--  { name = "LB Soft Focus Optic", focal_length = 50 },
--  { name = "LB Double Glass Optic", focal_length = 50 },
--  { name = "LB Single Glass Optic", focal_length = 50 },
--  { name = "LB Plastic Optic", focal_length = 50 },
--  { name = "LB Pinhole Optic", focal_length = 50 },
--  { name = "LB Fisheye Optic", focal_length = 12 },
--  { name = "LB Velvet 56", focal_length = 56 },
--  { name = "LB Velvet 85", focal_length = 85 },
--  { name = "LB Creative Aperture", focal_length = 50 },
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on September 01, 2017, 10:54:33 PM
I'd like to add these to the repository, but the Lensbaby entries are missing the manual_aperture field. Looking up each one of them would take ages from here (slow network), so I'm going to ask for a little help. Also, OK to replace LB with Lensbaby in all entries?

Are the names fixed so the lens profiles can be recognized in Adobe or some other software? If so, is there a place to verify these names, e.g. to make sure they don't contain typos?

BTW, the manual_aperture field appears unused; maybe it's meant to be max_aperture or widest_aperture, and used as a limit for manual aperture override?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on September 01, 2017, 11:15:25 PM
hi a1ex.

Regarding the Lensbaby:
- I will organize the aperture field. i already done it for most of the optics at my website http://www.lensbabylovers.info/2013/03/compatability-chart.html Just to confirm, you want each unique f value possible with each optic or just the min and max f? I have them all, can confirm, just let me know if you want the min-max or all the possible f values.
- Yes you can replace LB with Lensbaby (i was just lazy);
- The lens names dont have typos. You can verify lensbaby lens names here https://lensbaby.com/optic-compatibility/

Technically, lensbaby lenses (besides Circular Fisheye, Velvet 56 and Velvet 85) are called optics, because they can be used with one of these optic bodies (Composer, Composer Pro, Composer Pro II, Spark, Muse). But in my point of view, theres not really a point of having to list the lens body, when what matters is the optic.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on September 01, 2017, 11:20:35 PM
The Manual aperure field states the maximum aperture opening, (wide open)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on September 01, 2017, 11:36:56 PM
cool.

heres the values:
Code: [Select]
--  Lensbaby lenses

--  { name = "Lensbaby Sweet 35", focal_length = 35, manual_aperture =  2.5 },
--  { name = "Lensbaby Sweet 50", focal_length = 50, manual_aperture =  2.5 },
--  { name = "Lensbaby Twist 60", focal_length = 60, manual_aperture =  2.5 },
--  { name = "Lensbaby Edge 50", focal_length = 50, manual_aperture = 3.2 },
--  { name = "Lensbaby Edge 80", focal_length = 80, manual_aperture =  2.8 },
--  { name = "Lensbaby Circular Fisheye", focal_length = 5.8, manual_aperture =  3.5 },
--  { name = "Lensbaby Soft Focus Optic", focal_length = 50, manual_aperture =  2 },
--  { name = "Lensbaby Double Glass Optic", focal_length = 50, manual_aperture =  2 },
--  { name = "Lensbaby Single Glass Optic", focal_length = 50, manual_aperture =  2 },
--  { name = "Lensbaby Plastic Optic", focal_length = 50, manual_aperture =  2 },
--  { name = "Lensbaby Pinhole Optic", focal_length = 50, manual_aperture =  19 },
--  { name = "Lensbaby Fisheye Optic", focal_length = 12, manual_aperture =  4 },
--  { name = "Lensbaby Velvet 56", focal_length = 56, manual_aperture =  1.6 },
--  { name = "Lensbaby Velvet 85", focal_length = 85, manual_aperture =  1.8 },
--  { name = "Lensbaby Creative Aperture", focal_length = 50, manual_aperture =  2 },
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on September 02, 2017, 03:16:02 AM
A while back we discussed the possibility of adding an optional block that could take more than 32 characters (http://www.magiclantern.fm/forum/index.php?topic=18083.msg178163#msg178163), actually 31 characters terminated by a null (0).

Note the truncated name (...ZF.2)

Code: [Select]
Block: LENS
  Offset: 0x00000164
    Size: 96
    Time: 0.806000 ms
     Name:        'Zeiss Makro-Planar T* 2/100 ZF.'
     Serial:      ''
     Focal Len:   100 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

Any progress on that yet?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on September 10, 2017, 11:24:22 PM
What's the best way to make a build that combines the crop_rec_4k experiment with the manual lens info branch?

Import the manual lens into the crop_rec_4k branch?

and can this perhaps be done already for the main crop_rec_4k branch.  So that the new experimental build also include the option to set the lens info.

I don't want to make a pull request yet because I don't know what implication this has for the crop_rec_4k branch.
I would like to see a newer build for people to experiment with so we get more feedback on its use.

I can make one but I don't know how to upload it to the experiments page.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on September 11, 2017, 01:54:59 AM
What's the best way to make a build that combines the crop_rec_4k experiment with the manual lens info branch?

Merge

I did that a while ago and it seems to work fine. Seeing your message I thought I'd update it. There were some conflicts that I'm pretty sure I resolved properly. The branch is here:

https://bitbucket.org/daniel_fort/magic-lantern/branch/crop_rec_4k_manual_lens_info

It would be nice to add manual_lens_info to crop_rec_4k sort of like how raw_video_10bit_12bit was merged into crop_rec_4k. Only a few cameras are working with crop_rec_4k so those other branches are not going away any time soon.

As far as posting to the experiments page, I believe only a developer can do that. Of course if you can compile ML you can post your own test builds. Hint (https://bitbucket.org/daniel_fort/magic-lantern/downloads/).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on September 11, 2017, 12:55:49 PM
Thanks
Yea I tested your build too, I still can't get it to auto run on startup on the canon 5d mk3 1.2.3
I guess you have the same?

And for the mlv play module to load mlv_rec has to be disabled.
But I think thats a known issue.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on September 11, 2017, 01:55:13 PM
In case you did not notice yet. @a1ex placed a pull request to merge lens_info into lua_fix which is probably the easiest and fastest way to get this function into nightly and also different experimental branches. Might be  good to comment this PR.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on September 11, 2017, 02:47:35 PM
@JohanJ. Good tip, thanks
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on September 14, 2017, 12:19:22 PM
I added a few more Nikon lenses.

Code: [Select]
--  Nikon lenses
    { name = "Nikon AF Fisheye-NIKKOR 16mm f/2.8D",          focal_length =  16, manual_aperture = 2.8 },
    { name = "Nikon AF NIKKOR 20mm f/2.8D",          focal_length =  20, manual_aperture = 2.8 },
    { name = "Nikon AF NIKKOR 28mm f/1.4D",          focal_length =  28, manual_aperture = 1.4 },
    { name = "Nikon AF NIKKOR 35mm f/2D",            focal_length =  35, manual_aperture = 2   },
    { name = "Nikon AF NIKKOR 50mm f/1.4D",          focal_length =  50, manual_aperture = 1.4 },
    { name = "Nikon AFS-NIKKOR 50mm f/1.8G",         focal_length =  50, manual_aperture = 1.8 },
    { name = "Nikon AF Micro-NIKKOR 60mm f/2.8D",    focal_length =  60, manual_aperture = 2.8 },
    { name = "Nikon AF NIKKOR 85mm f/1.4D IF",       focal_length =  85, manual_aperture = 1.4 },
    { name = "Nikon AF DC-NIKKOR 105mm f/2D",        focal_length = 105, manual_aperture = 2   },
    { name = "Nikon AF DC-NIKKOR 135mm f/2D",        focal_length = 135, manual_aperture = 2   },
    { name = "Nikon AF NIKKOR 28-105 f3.5-4.5D",     focal_length =  28, manual_aperture = 3.5 },

--  Nikon Manual focus lenses
    { name = "Nikon Zoom Ais ED 50-300",             focal_length = 300, manual_aperture = 4.5 },
    { name = "Nikon Ais NIKKOR 24mm f/2.8",          focal_length =  24, manual_aperture = 2.8 },
    { name = "Nikon Ais NIKKOR 35mm f/1.4",          focal_length =  35, manual_aperture = 1.4 },
    { name = "Nikon Ais NIKKOR 55mm f/1.2",          focal_length =  55, manual_aperture = 1.2 },
    { name = "Nikon Ais NIKKOR 105mm f/1.8",         focal_length = 105, manual_aperture = 1.8 },
    { name = "Nikon Ais NIKKOR 500mm f/8",           focal_length = 500, manual_aperture = 8   },
    { name = "Kiron Macro 100mm f/2.8",              focal_length = 105, manual_aperture = 2.8 },

--  Anamorphic Lenses
    { name = "Iscorama Anamorphic 50 f/2.8mm ",      focal_length =  50, manual_aperture = 2.8 },

I know the list is not complete but there are so many manual lenses
http://www.photosynthesis.co.nz/nikon/lenses.html

Full manual lenses from Nikon won't be recognised in ACR because adobe does not have profile for them.
There is a way to make profiles with the adobe profile creator.
http://supportdownloads.adobe.com/detail.jsp?ftpID=5493.

Here an article on the editor
http://blogs.adobe.com/jkost/2013/09/create-your-own-lens-profiles.html

The best way to auto recognise them is what fort described here.
@Lars_Steenhoff

Had a breakthrough today. To get your Nikon AF NIKKOR 35mm f2D lens working with the lua script simply copy the profile from the ACR installation. I'm not sure what the Windows location is but on the Mac it is in:

Code: [Select]
/Library/Application Support/Adobe/CameraRaw/LensProfiles/1.0/Nikon/NIKON D3X (Nikon AF NIKKOR 35mm f2D) - RAW.lcp
and put the copy in your user area, again on the Mac it is in:

Code: [Select]
/Users/<you>/Library/Application Support/Adobe/CameraRaw/LensProfiles/1.0/
Then edit the stCamera:Lens from this:

Code: [Select]
<stCamera:Lens>35.0 mm f/2.0</stCamera:Lens>
To this:

Code: [Select]
<stCamera:Lens>Nikon AF NIKKOR 35mm f2D</stCamera:Lens>
Do a copy/replace because there are multiple instances of that tag.

Finally put that lens name in your lua script and that's it.

For more details, here's my latest post on the Adobe Camera Raw forum:

https://forums.adobe.com/message/9200782#9200782
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on September 23, 2017, 06:14:32 PM
update with two new lenses:
Code: [Select]
{ name = "Lensbaby Sweet 80", focal_length = 80, manual_aperture =  2.8 },
{ name = "Lensbaby Creative Bokeh", focal_length = 50, manual_aperture =  2.5 },
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on September 23, 2017, 08:54:15 PM
Any progress on that yet?

nope.
did i miss a field there? maybe something you want/have to add?
as soon it is "live" there is no way back :)

Code: [Select]
typedef struct {
    uint8_t     blockType[4];    /* ELNS - Extended LENS block with longer lens name and optional fields, depending on camera */
    uint32_t    blockSize;
    uint64_t    timestamp;
    uint16_t    focalLengthMin;  /* shortest focal length in mm                       */
    uint16_t    focalLengthMax;  /* longest focal length in mm                        */
    uint16_t    apertureMin;     /* lowest f-number * 100                             */
    uint16_t    apertureMax;     /* highest f-number * 100                            */
    uint32_t    version;         /* lens internal version number, if available        */
    uint8_t     extenderInfo;    /* extender information, if provided by camera       */
    uint8_t     capabilities;    /* capability information, if provided by camera     */
    uint8_t     chipped;         /* when not zero, lens is communicating with camera  */
    /*uint8_t     lensName[variable]*/     /* full lens string, null terminated                 */
}  mlv_elns_hdr_t;
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on September 23, 2017, 09:38:28 PM
as alex pointed out, the lens firmware version could be added.
anyone knows about its maximum length?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on September 24, 2017, 06:56:12 AM
Maybe this from exiftool - Canon.pm will help?

Code: [Select]
    0x93 => { %ciMinFocal }, #15
    0x95 => { %ciMaxFocal }, #15
    0x97 => { #15
        Name => 'LensType',
        Format => 'int16uRev', # value is big-endian
        SeparateTable => 1,
        ValueConvInv => 'int($val)', # (must truncate decimal part)
        PrintConv => \%canonLensTypes,
    },
    0xa4 => { #PH
        Name => 'FirmwareRevision',
        Format => 'string[8]',
    },
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on September 24, 2017, 07:36:25 AM
I was adding the lenses ML users have added and this one threw me:

Code: [Select]
--  Nikon lenses
 ...
    { name = "Kiron Macro 100mm f/2.8",              focal_length = 105, manual_aperture = 2.8 },

Really? Well OK. I'm making a pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/859/added-and-organized-lenses/diff) to add more lenses to the script. Note that all lenses are commented out so you need to uncomment the lenses you plan to use.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on September 24, 2017, 10:28:55 AM
Thanks for organising them, the old macro lens from kiron is also there, Its not a nikon lens, buts its a nikon mount lens.

It should be in its own category.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on September 24, 2017, 06:44:12 PM
Right, I added a Miscellaneous category at the end of the list. Read up on Kiron lenses and it turns out that the U.S. distributor was located just a few miles from where I live yet I never heard of them. The lenses were manufactured in Tokyo by Kino Precision Industries Limited and they made lenses for companies like Vivitar. Theirs is a similar story as the current Samyang lenses.

In any case, there's no way to include every possible lens but if ML users want to contribute specs for the lenses they own, as long as the list doesn't get too long!

(Note that I added my 55mm 2.8 Micro Nikkor.)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on October 03, 2017, 09:46:35 PM
any idea when will we have a build with this? last one was in April
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 04, 2017, 07:22:14 AM
I've got a pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/859/added-and-organized-lenses/diff) open for additional lenses but other than the branch hasn't changed much. You could comment on the pull request or just take the script from bitbucket (https://bitbucket.org/daniel_fort/magic-lantern/raw/f2874dd366ca4908402996659f7e52afe130afc9/scripts/lens.lua) and replace the one in your ML/scripts directory.

By the way, there's another pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/822/manual-lens-info-lua/diff) for merging manual_lens_info into lua_fix. That would be a very good thing for this project because at some point lua_fix will be merged into unified and it will be available on the nightly builds.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 04, 2017, 12:19:05 PM
Yes the merge with Lua and ultimately with croprec4k is nice to get more feedback, because yes your right the last build is a bit old
I imagine you mostly want to use it with the latest experiments and with the latest nightly builds.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on October 07, 2017, 04:33:20 PM
I've got a pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/859/added-and-organized-lenses/diff) open for additional lenses but other than the branch hasn't changed much. You could comment on the pull request or just take the script from bitbucket (https://bitbucket.org/daniel_fort/magic-lantern/raw/f2874dd366ca4908402996659f7e52afe130afc9/scripts/lens.lua) and replace the one in your ML/scripts directory.

By the way, there's another pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/822/manual-lens-info-lua/diff) for merging manual_lens_info into lua_fix. That would be a very good thing for this project because at some point lua_fix will be merged into unified and it will be available on the nightly builds.

tired that, with no success. the script isnt shown on the SCRIPTS menu. when i get to DEBUG menu theres the Memory Error in red.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 07, 2017, 05:51:05 PM
@NunoCaldeira

What exactly are you doing? Updating a lua script shouldn't be that hard.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 07, 2017, 06:24:04 PM
I think you copied the script to a build from 4k crop without the capabilty of manual lens scripts?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on October 07, 2017, 11:27:04 PM
I think you copied the script to a build from 4k crop without the capabilty of manual lens scripts?

@NunoCaldeira

What exactly are you doing? Updating a lua script shouldn't be that hard.

6D Latest Build: 2017-10-03
copy the lua script to script folder
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 07, 2017, 11:38:24 PM
6D Latest Build: 2017-10-03

That's the problem. You need to use the one from the experiments page (https://builds.magiclantern.fm/experiments.html): magiclantern-manual_lens_info.2017Apr04.6D116.zip
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 09, 2017, 12:37:44 AM
EDIT build is now old
@NunoCaldeira you can try this build from today where I merged the manual lens info branch on to the 4kcrop.

Please report if it works well for you, or if you have any issues.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on October 09, 2017, 01:06:39 AM
https://www.dropbox.com/s/jwuambvb6it1xmz/magiclantern-Nightly.manual_lens_info.2017Oct09.5D3123.zip?dl=0

@NunoCaldeira you can try this build from today where I merged the manual lens info branch on to the 4kcrop.

Please report if it works well for you, or if you have any issues.
thanks, but have a 6D. not 5D MKIII :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 09, 2017, 01:30:51 AM
Ok I just tried building for the 6D but I got some errors.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 10, 2017, 12:22:17 AM
The 6D isn't ready for crop_rec_4k. Grab the Non-CPU lens info or the Latest Lua updates + fixes for the 6D (experiments download page (http://builds.magiclantern.fm/experiments.html)) and put the new lens.lua file in the scripts directory. Make sure you read the script, comment out the test lines and uncomment (or add) the lenses your are using.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on October 15, 2017, 03:25:43 PM
Ok I just tried building for the 6D but I got some errors.
no worries Lars. was just trying to get it working with the latest nightly. cheers
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 15, 2017, 05:49:59 PM
and did you get it to work with any version you downloaded at all?
perhaps the lua fixes one as thats more recent
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: NunoCaldeira on October 15, 2017, 07:56:01 PM
and did you get it to work with any version you downloaded at all?
perhaps the lua fixes one as thats more recent
didnt manage it to work,only with the April one
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on October 16, 2017, 01:07:27 AM
@NunoCaldeira

The lens.lua script doesn't work on the unified branch (Nightly builds) but maybe we can get this working on the latest experimental build?

http://www.magiclantern.fm/forum/index.php?topic=15088.msg191691;topicseen#msg191691

Need some help from 6D users.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 16, 2017, 04:06:17 PM
I managed to make the script works with manual lens with AF confirm Chip.
Also possibly found a way to avoid selecting lens everytime shoot mode is changed (this also avoid getting errors when changing more mode at a time without selecting lens from menu) but it need more testing.

Right now my main interest is to get the metadata into an mlv file.
then when the dngs are generated in post this metadata will be put inside the dng files.

My knowledge of how the metadata is written in mlv video is limited.

Anyone know how its written in the mlv?

for stills it would be nice to have it in the cr2 but this might be more difficult. then the xmp route might be best for stills.

I have found a way to get Lens Info to work with Manual Lenses and consequently Lens Name and Focal Length (maybe also Aperture value) get written in Mlv file generated by Silent Pictures. Don't know if it also works for raw video files.

I can push changes to the repository (if someone find this interesting) after testing a cleaner way to get values and finished to optimize the script
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on December 16, 2017, 09:12:36 PM
Yes very intersting!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 16, 2017, 10:32:51 PM
Nice!

It would be great if you can make a pull request for this.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 17, 2017, 06:17:22 AM
Nice!

It would be great if you can make a pull request for this.

Here is the PR (https://bitbucket.org/hudson/magic-lantern/pull-requests/887), take a look please.

I had to remove the check for switching shoot mode as sometimes it was responsible for breaking lens name value in Lens Info, but I didn't checked if values were saved correctly in MLV metadata.
I noticed a strange behavior in the handler's execution... For some reason it reached the end of the function only after a second call occured, maybe it can be the problem or I am missing some information on how lua property works.

Here is the script i was using if you want to try:
Code: [Select]
-- Manual Lens Information V2
-- Supplies lens information for manual lenses
-- Whenever there is a non-chipped lens detected, we prompt the user to select the attached manual lens from a list

require("ui")
require("config")
require("xmp")

lenses =
{
--  The following is for testing purposes. Comment out the following lines then either uncomment only the lenses
--  that you want to use from the list or add your own lenses. Tip: Put your most used lenses at the top of the list.

    { name = "My Lens", focal_length = 50 },
    { name = "My Other Lens", focal_length = 25 },
    { name = "Yashica ML 50mm f1.9", focal_length = 50 },
    { name = "Pentax SMC 80-200mm f4,5", focal_length = 200, manual_aperture = 4.5 },

--  Zeiss ZF.2 manual lenses Nikon mount - these work with the lens profiles that ship with Adobe Camera Raw

--  { name = "Zeiss Distagon T* 2.8/15 ZF.2",        focal_length =  15, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 3.5/18 ZF.2",        focal_length =  18, manual_aperture = 3.5 },
--  { name = "Zeiss Distagon T* 2.8/21 ZF.2",        focal_length =  21, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 2.8/25 ZF.2",        focal_length =  25, manual_aperture = 2.8 },
--  { name = "Zeiss Distagon T* 2/25 ZF.2",          focal_length =  25, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 2/28 ZF.2",          focal_length =  28, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 2/35 ZF.2",          focal_length =  35, manual_aperture = 2   },
--  { name = "Zeiss Distagon T* 1.4/35 ZF.2",        focal_length =  35, manual_aperture = 1.4 },
--  { name = "Zeiss Makro-Planar T* 2/50 ZF.2",      focal_length =  50, manual_aperture = 2   },
--  { name = "Zeiss Planar T* 1.4/50 ZF.2",          focal_length =  50, manual_aperture = 1.4 },
--  { name = "Zeiss Planar T* 1.4/85 ZF.2",          focal_length =  85, manual_aperture = 1.4 },
--  { name = "Zeiss Makro-Planar T* 2/100 ZF.2",     focal_length = 100, manual_aperture = 2   },
--  { name = "Zeiss Apo Sonnar T* 2/135 ZF.2",       focal_length = 135, manual_aperture = 2   },

--  Nikon lenses
--  { name = "Nikon Zoom Ais ED 50-300",             focal_length = 300, manual_aperture = 4.5 },
--  { name = "Nikon AF NIKKOR 28mm f/1.4D",          focal_length =  28, manual_aperture = 1.4 },


--  Samyang manual lenses - also branded as Rokinon and Bower. Cine versions use the same lens profile.
--  The lens profiles for Samyang manual lenses that ship with Adobe Camera raw must be modified in order
--  for automatic lens detection to work.
--  More information here: http://www.magiclantern.fm/forum/index.php?topic=18083.msg176261#msg176261

--  { name = "Samyang 8mm f/2.8 UMC Fisheye",        focal_length =   8, manual_aperture = 2.8 },
--  { name = "Samyang 8mm f/2.8 UMC Fisheye II",     focal_length =   8, manual_aperture = 2.8 }, --   8mm T3.1 Cine
--  { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS",    focal_length =   8, manual_aperture = 3.5 },
--  { name = "Samyang 8mm f/3.5 UMC Fish-Eye CS II", focal_length =   8, manual_aperture = 3.5 }, --   8mm T3.8 Cine
--  { name = "Samyang 10mm f/2.8 ED AS NCS CS",      focal_length =  10, manual_aperture = 2.8 }, --  10mm T3.1 Cine
--  { name = "Samyang 12mm f/2 NCS CS",              focal_length =  12, manual_aperture = 2   }, --  12mm T2.2 Cine
--  { name = "Samyang 12mm f/2.8 ED AS NCS Fisheye", focal_length =  12, manual_aperture = 2.8 }, --  12mm T3.1 Cine
--  { name = "Samyang 14mm f/2.8 ED AS IF UMC",      focal_length =  14, manual_aperture = 2.8 }, --  14mm T3.1 Cine
--  { name = "Samyang 16mm f/2 ED AS UMC CS",        focal_length =  16, manual_aperture = 2   }, --  16mm T2.2 Cine
--  { name = "Samyang 21mm f/1.4 ED AS UMC CS",      focal_length =  21, manual_aperture = 1.4 }, --  21mm T1.5 Cine
--  { name = "Samyang 24mm f/1.4 ED AS IF UMC",      focal_length =  24, manual_aperture = 1.4 }, --  24mm T1.5 Cine
--  { name = "Samyang 35mm f/1.4 AS IF UMC",         focal_length =  35, manual_aperture = 1.4 }, --  35mm T1.5 Cine
--  { name = "Samyang 50mm f/1.2 AS UMC CS",         focal_length =  50, manual_aperture = 1.2 },
--  { name = "Samyang 50mm f/1.4 AS UMC",            focal_length =  50, manual_aperture = 1.4 }, --  50mm T1.5 Cine
--  { name = "Samyang 85mm f/1.4 AS IF UMC",         focal_length =  85, manual_aperture = 1.4 }, --  85mm T1.5 Cine
--  { name = "Samyang 100mm f/2.8 ED UMC MACRO",     focal_length = 100, manual_aperture = 2.8 }, -- 100mm T3.1 Cine
--  { name = "Samyang 135mm f/2 ED UMC",             focal_length = 135, manual_aperture = 2   }, -- 135mm T2.2 Cine
--  { name = "Samyang 300mm f/6.3 ED UMC CS",        focal_length = 300, manual_aperture = 6.3 },


--  Lensbaby lenses

--  { name = "Lensbaby Sweet 35",                    focal_length =  35, manual_aperture = 2.5 },
--  { name = "Lensbaby Sweet 50",                    focal_length =  50, manual_aperture = 2.5 },
--  { name = "Lensbaby Twist 60",                    focal_length =  60, manual_aperture = 2.5 },
--  { name = "Lensbaby Edge 50",                     focal_length =  50, manual_aperture = 3.2 },
--  { name = "Lensbaby Edge 80",                     focal_length =  80, manual_aperture = 2.8 },
--  { name = "Lensbaby Circular Fisheye",            focal_length = 5.8, manual_aperture = 3.5 },
--  { name = "Lensbaby Soft Focus Optic",            focal_length =  50, manual_aperture = 2   },
--  { name = "Lensbaby Double Glass Optic",          focal_length =  50, manual_aperture = 2   },
--  { name = "Lensbaby Single Glass Optic",          focal_length =  50, manual_aperture = 2   },
--  { name = "Lensbaby Plastic Optic",               focal_length =  50, manual_aperture = 2   },
--  { name = "Lensbaby Pinhole Optic",               focal_length =  50, manual_aperture = 19  },
--  { name = "Lensbaby Fisheye Optic",               focal_length =  12, manual_aperture = 4   },
--  { name = "Lensbaby Velvet 56",                   focal_length =  56, manual_aperture = 1.6 },
--  { name = "Lensbaby Velvet 85",                   focal_length =  85, manual_aperture = 1.8 },
--  { name = "Lensbaby Creative Aperture",           focal_length =  50, manual_aperture = 2   },

}

selector_instance = selector.create("Select Manual Lens", lenses, function(l) return l.name end, 600)

lens_config = config.create({})

if lens_config.data ~= nil and lens_config.data.name ~= nil then
    for i,v in ipairs(lenses) do
        if v.name == lens_config.data.name then
            selector_instance.index = i
            break
        end
    end
end

-- Property to be written in .xmp file
xmp:add_property(xmp.lens_name, function() return lens.name end)
xmp:add_property(xmp.focal_length, function() return lens.focal_length end)
xmp:add_property(xmp.aperture, function() return math.floor(camera.aperture.value * 10) end)

-- Flag variable to avoid selecting lens when switching shott mode
lensSelected = 0

function is_manual_focus()
  console.write("In is_manual_focus()\n")
  console.write(lens.lens_id)
  console.write(lens.lens_exists)

  if (lens.lens_id == 0 or lens.lens_id == "(no lens)" or
      lens.name == "1-65535mm" or lens.focal_length == "1-65535mm") then
    console.write("In is_manual_focus() - TRUE\n")
    return true
  else
    console.write("In is_manual_focus() - FALSE\n")
    return false
  end
end

--  Handler for lens_name property
--  Get Called when:
--  Switching lens
--  Switching shoot mode
function property.LENS_NAME:handler(value)
    console.write("In Handler - LensSelected:\n")
    console.write(lensSelected)

    -- Check if lens has already been selected by menu
    if lensSelected == 0 then
      if is_manual_focus() then
          task.create(select_lens)
      else
        -- Not a manual Lens, no need to write sidecart file
        if selector_instance ~= nil then
            selector_instance.cancel = true
            console.write("Selector_instance is NULL")
        end
        -- Clear flag variable for next run
        lensSelected = 0
        console.write("XMP STOP")

        xmp:stop()
      end
    end

    console.write("Exiting from Handler\n")
end

function select_lens()
    if #lenses > 1 then
        local menu_already_open = menu.visible
        if not menu_already_open then
            menu.open()
            display.rect(0, 0, display.width, display.height, COLOR.BLACK, COLOR.BLACK)
        end
        if selector_instance:select() then
            update_lens()
        end
        if not menu_already_open then
            menu.close()
        end
    elseif #lenses == 1 then
        update_lens()
    end
end

-- Copy lens attribute from
function update_lens()
    console.write("In update_lens()\n")

    for k,v in pairs(lenses[selector_instance.index]) do
        lens[k] = v
    end
    lens_config.data = { name = lenses[selector_instance.index].name }
    xmp:start()

    -- Avoid to select lens again when switching shoot mode
    lensSelected = 1
    -- Enable to read values for Lens Info and MLV file
    lens.lens_exists = true
    console.write("Setting lens_exists to TRUE\n")
end

lens_menu = menu.new
{
    parent = "Lens Info Prefs",
    name = "Manual Lens V2",
    help = "Info to use for attached non-chipped lens",
    icon_type = ICON_TYPE.ACTION,
    select = function()
        if is_manual_focus() then
            task.create(select_lens)
        end
    end,
    rinfo = function()
        return lens.name
    end,
    warning = function()
        flag = is_manual_focus()
        if flag == false then
            return "Chipped lens is attached"
        end
    end
}

-- Check lens on start
if is_manual_focus() then
    task.create(select_lens)
end

Note: I get a compilation error for silent.o from manual_lens_info even without my changes, it need some investigation but I copied a compiled version to run the test
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 19, 2017, 05:44:59 PM
I have pushed more changes to PR. Currently I am trying to save Serial Number in MLV File

This is what i get from MLV metadata:
Code: [Select]
Block: LENS
  Offset: 0x000001f0
    Size: 96
    Time: 9.048000 ms
     Name:        'Pentax SMC 80-200mm f4,5'
     Serial:      '0'
     Focal Len:   200 mm
     Focus Dist:  0 mm
     Aperture:    f/4.50
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x0000001B
     Flags:       0x00000000


(https://thumb.ibb.co/c2jFZm/VRAM8.jpg) (https://ibb.co/c2jFZm)


This is what I modified in modules/mlv_rec/mlv.c:
Code: [Select]
 
void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp)
{
...
 
    strncpy((char *)hdr->lensName, lens_info.name, 32);
    //Changed this one with snprintf()
    //strncpy((char *)hdr->lensSerial, "", 32);
    snprintf((char *)hdr->lensSerial, 32, "%d", lens_info.lens_serial);

    //strncpy((char *)hdr->lensName, lens_info.lens_serial, 32); Doesn't works well, It print a weird string
   
...
}

Is this a valid way to copy value from lens_info struct?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on December 19, 2017, 05:56:51 PM
I prefer snprintf also in the first case (which is actually scnprintf (https://bitbucket.org/hudson/magic-lantern/branch/vsnprintf), btw); it guarantees a null-terminated string and won't write more than 32 characters. Since g3gg0 used strncpy, I assume the MLV spec does not require a null-terminated string (can't find it in the spec), but having one is desirable IMO (fewer surprises for C-based MLV converters).

Here, strncpy won't work because lens_serial is not a string; iirc it should be printed as hex though (%X).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on December 19, 2017, 10:57:30 PM

Is this a valid way to copy value from lens_info struct?

can you try the latest patches, please?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 19, 2017, 11:39:02 PM
can you try the latest patches, please?

It works! I have manually copied changes regarding Serial Number from your commits

Code: [Select]
Block: LENS
  Offset: 0x000001f0
    Size: 96
    Time: 8.662000 ms
     Name:        'Pentax SMC 80-200mm f4,5'
     Serial:      '0075BCD15'
     Focal Len:   200 mm
     Focus Dist:  0 mm
     Aperture:    f/4.50
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

Serial: 123456789 is saved as in Lens Info Menu format
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on December 20, 2017, 01:17:06 AM
can you also update mlv_dump? it should show you the serial as plain number.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 20, 2017, 12:46:36 PM
can you also update mlv_dump? it should show you the serial as plain number.
How do I do to recompile it? I can’t find it in any ml folder
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 20, 2017, 03:35:58 PM
@aprofiti -- mlv_dump is in modules/mlv_rec
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 20, 2017, 10:44:03 PM
@aprofiti -- mlv_dump is in modules/mlv_rec

I get this: (OSX)

Code: [Select]
PCAlessandro:mlv_rec alex$ make mlv_dump
[ HOST_CC  ]   mlv_dump.host.o
clang: error: unknown argument: '-mno-ms-bitfields'
make: *** [mlv_dump.host.o] Error 1
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Danne on December 21, 2017, 12:38:04 AM
Try follow these steps
http://www.magiclantern.fm/forum/index.php?topic=16012.0

Your issue is related to clang and gcc.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 21, 2017, 03:31:06 AM
@aprofiti -- Cut to the chase (for Mac OSX):

...make a new file called Makefile.user and put in just these lines:

Code: [Select]
#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)

Warning: DON'T EDIT Makefile.user.default -- Put the changes you want to make to Makefile.user.default in your Makefile.user file. Think of this as a way to customize the Magic Lantern build environment without modifying the code.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 21, 2017, 05:01:45 AM
Thank you Danne and dfort!

Here it is:
Code: [Select]
Block: LENS
  Offset: 0x000001f0
  Number: 7
    Size: 96
    Time: 8.662000 ms
     Name:        'Pentax SMC 80-200mm f4,5'
     Serial:      '0075BCD15' (123456789)
     Focal Len:   200 mm
     Focus Dist:  0 mm
     Aperture:    f/4.50
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

With no valid serial in mlv file:
Code: [Select]
Block: LENS
  Offset: 0x000001f0
  Number: 7
    Size: 96
    Time: 9.048000 ms
     Name:        'Pentax SMC 80-200mm f4,5'
     Serial:      '' (no valid S/N)
     Focal Len:   200 mm
     Focus Dist:  0 mm
     Aperture:    f/4.50
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x0000001B
     Flags:       0x00000000
Good work g3gg0!


Little sneak peek: I'm trying to add submenus to be able to change aperture and focal length when using the lens; at the moment it works but can't retrive value from lens.focal_len to set starting value in menu and bounds for min and max. Hope I have time to finish in next days
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 22, 2017, 03:00:57 PM
Added possibility to change value while using lens:

(https://thumb.ibb.co/ek64Em/VRAM9.jpg) (https://ibb.co/ek64Em) (https://thumb.ibb.co/iikLn6/VRAM10.jpg) (https://ibb.co/iikLn6) (https://thumb.ibb.co/fft6S6/VRAM11.jpg) (https://ibb.co/fft6S6) (https://thumb.ibb.co/csfWum/VRAM12.jpg) (https://ibb.co/csfWum) (https://thumb.ibb.co/gxSKEm/VRAM13.jpg) (https://ibb.co/gxSKEm)

Could be nice to have direct access to submenu on keypress for a fast change of aperture and focal
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on December 22, 2017, 04:28:06 PM
Yea easy access button would be good,
one way you could already do it is to add these manual lens scripts to "my menu"
 and let "my menu" be the fist menu that comes up when pressing menu.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 23, 2017, 02:19:09 AM
I'm checking last things to be able to close PR and I have found a problem when using f-number with a dot (ex. 2.8, 4.5 ...)
Value is written correctly in Lens Info, MLV and .xmp sidecar but not in Manual Lens submenu (but it works when setting value in submenu itself)... It set automatically aperture to 1.0 when menu is opened

After reading again ML Lua Documentation and digging deep in lua's source code, I have now understood how value for a submenu with attribute "choices" is set.

I believe the problem here is in function update_menu() when assigning aperture... It expect a String but when assigning from lenses[].manual_aperture it return a Number

Code: [Select]
lenses = {
     ....
    { name = "Portrait 85mm", focal_length = 85, manual_aperture = 2.8 },
    { name = "Portrait 105mm", focal_length = 105, focal_min = 105, focal_max = 105, manual_aperture = 4.5 },
}

-- f-number values in 1/2 Stop
Fnumbers = {"1.0","1.2","1.4","1.7","2","2.4","2.8","3.3","4","4.8","5.6","6.7","8","9.5","11","13","16","19","22","27","32"}

lens_menu = menu.new
{
    ...
    submenu =
    {
        ...
        {
            name    = "Aperture",
            help    = "Set Aperture to metadata",
            choices = Fnumbers,
            -- Update Aperture with selected value from submenu
            update = function(this)
                      if lensSelected == true then
                        lens.manual_aperture = tonumber(this.value)
                        update_xmp()
                      else
                        -- Reset menu value to the corrected one
                        this.value = lens.manual_aperture
                      end end,
            warning = function()
                        if lensSelected == false then
                          return "this value is not supported for non-manual lens"
                      end end,
        }
    },
    ...
}

-- Update the menu with values for Focal Length and Aperture from selected Lens
-- To be called when switching manual lens
function update_menu()
  ....
  lens_menu.submenu["Aperture"].value = lens.manual_aperture
end

I tried converting to string by using:
Code: [Select]
lens_menu.submenu["Aperture"].value = tostring(lens.manual_aperture)
Doesn't works... instead this works when aperture value chosen is in Fnumbers{}:
Code: [Select]
lens_menu.submenu["Aperture"].value = "2.8"
Someone can explain to me why can't get it working? I have also tried to merge into lua_fix to get latest fixes, but nothing usefull happened.

Question: Should I include also 1/3 stop values to Fnumbers? Currently if an aperture value of the lens isn't found in Fnumbers when opening Manual Lens submenu, it get set to first index (ex. aperture 4.5 not found -> set by script to 1.0)

[EDIT]
It's a problem of floating points rapresentation in lua:
Code: [Select]
1.4 -> 1.399999
2.8 -> 2.799999
4.5 -> 4.5 (ok)
5.6 -> 5.59999
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 29, 2017, 04:34:15 AM
I have pushed pushed more changes to PR:
1. Values are automatically copied from lenses array when changing shooting mode (Need more testing. WIP: persist previusly selected aperture and focal length)
2. Added custom aperture value to choose from and auto lower bound for aperture
3. Fixed Focal length and aperture in LV

Know Issues:
1. [Solved] Lua's behavior has side effect on some decimal values, they can't be assigned in menu aperture field when script select lens. Values are saved correctly in metadata but they are overwritten when opening Manual Lens menu (see https://bitbucket.org/hudson/magic-lantern/issues/2824/wrong-rapresentation-of-decimal-numbes-in (https://bitbucket.org/hudson/magic-lantern/issues/2824/wrong-rapresentation-of-decimal-numbes-in))
2. Apertute is changed to adapter's default value after pressing Shutter button in canon GUI when using lens with AF Chip

Can someone review it or better to test it on field? I have only one working AF adapter, so I need to know if others adapters return lens Name and focal length as script checks
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on December 30, 2017, 12:47:54 AM
Whats the AF confirm adapter you have?

I have to buy one, right now I have only passive adapters from novoflex.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dmilligan on December 30, 2017, 02:12:17 PM
Someone can explain to me why can't get it working? I have also tried to merge into lua_fix to get latest fixes, but nothing usefull happened.

Question: Should I include also 1/3 stop values to Fnumbers? Currently if an aperture value of the lens isn't found in Fnumbers when opening Manual Lens submenu, it get set to first index (ex. aperture 4.5 not found -> set by script to 1.0)

[EDIT]
It's a problem of floating points rapresentation in lua:
Code: [Select]
1.4 -> 1.399999
2.8 -> 2.799999
4.5 -> 4.5 (ok)
5.6 -> 5.59999

Check the ftoa function in ml_lua_shim.c, it could probably be improved (maybe there's a usable implementation in dietlibc that's better). The sprintf in Canon firmware doesn't support floats at all, and we don't have a complete, standards compliant libc implementation available in ML. So for now there are hacky implementations here and there of stuff we need (space in the compiled binary is at a premium).

As a workaround you could store lens.manual_aperture as string and then when you need to use it as a number, convert it to number with tonumber().
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on December 30, 2017, 11:13:36 PM
Knowing where to look makes things really easy - fix committed (https://bitbucket.org/hudson/magic-lantern/commits/aa9c7f1b64b2ea4f4252b3470d80a03329894fe0).

Sorry, didn't test further yet.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 31, 2017, 06:01:34 AM
Lots of nice additions here but we still have a rather basic issue with the 32 character limit for the lens name. It would be great if this could be addressed.

Using @aprofiti pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/887/manual-lens-info/diff) version and the latest mlv_dump from the crop_rec_4k branch.

Code: [Select]
{ name = "Zeiss Makro-Planar T* 2/100 ZF.2",     focal_length = 100, manual_aperture = 2   },
Code: [Select]
Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.956000 ms
     Name:        'Zeiss Makro-Planar T* 2/100 ZF.'
     Serial:      '' (no valid S/N)
     Focal Len:   100 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

I also tried another lens with a long name and this time the Aperture isn't listed properly (float issue?): [EDIT] Merged in lua_fix (https://bitbucket.org/hudson/magic-lantern/pull-requests/720/lua-fixes-part-2/diff) with the "round to our precision (https://bitbucket.org/hudson/magic-lantern/commits/aa9c7f1b64b2)" fix and the Aperture is now being displayed properly. Thanks a1ex!

Code: [Select]
{ name = "Samyang 12mm f/2.8 ED AS NCS Fisheye", focal_length =  12, manual_aperture = 2.8 }, --  12mm T3.1 Cine
Code: [Select]
Block: LENS
  Offset: 0x0000016c
  Number: 6
    Size: 96
    Time: 0.003000 ms
     Name:        'Samyang 12mm f/2.8 ED AS NCS Fi'
     Serial:      '' (no valid S/N)
     Focal Len:   12 mm
     Focus Dist:  0 mm
     Aperture:    f/2.80
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 31, 2017, 05:15:39 PM
Knowing where to look makes things really easy - fix committed (https://bitbucket.org/hudson/magic-lantern/commits/aa9c7f1b64b2ea4f4252b3470d80a03329894fe0).

Sorry, didn't test further yet.
Really thank you a1ex! At the moment I am unable to test it, I'll try in next days.

I also tried another lens with a long name and this time the Aperture isn't listed properly (float issue?):

Code: [Select]
{ name = "Samyang 12mm f/2.8 ED AS NCS Fisheye", focal_length =  12, manual_aperture = 2.8 }, --  12mm T3.1 Cine
Code: [Select]
Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.881000 ms
     Name:        'Samyang 12mm f/2.8 ED AS NCS Fi'
     Serial:      '' (no valid S/N)
     Focal Len:   12 mm
     Focus Dist:  0 mm
     Aperture:    f/1.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000
Yes, It's from floating point issue. Value 1.0 is selected by Ml (first index in Fnumbers) because Aperture wasn't found in submenu choices.

Can you try with a1ex commit or by merging PR into lua_fix (use lastest lual_lens.c changes to resolve conflict)?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 31, 2017, 05:45:42 PM
Can you try with a1ex commit or by merging PR into lua_fix (use lastest lual_lens.c changes to resolve conflict)?

Just did and it is working fine now. Edited Reply #277 (http://www.magiclantern.fm/forum/index.php?topic=18083.msg195410#msg195410).

I'm sounding like a broken record asking for more characters in the lens name (http://www.magiclantern.fm/forum/index.php?topic=18083.225) but the main developers agreed that it should be fixed (http://www.magiclantern.fm/forum/index.php?topic=18083.msg178163#msg178163). Not that I'm complaining--just bringing @aprofiti up to speed on this. Maybe this could be incorporated into all the great changes he's making to this branch. It is after all an "Experimental" branch.  :D
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 03, 2018, 01:56:24 AM
Updated PR with autoload functionality and made some tests to solve aperture with AF Chip.

Here is some ideas to solve issue no.2:
1. Make HALF_SHUTTER handler to works outside live view and update aperture from here (appear to be the easiest, any Hints?)
2. Change camera aperture by make it works from camera.aperture API (At the moment camera will freeze)
3. "Emulating" MPU or EF protocol messages

I'm sounding like a broken record asking for more characters in the lens name (http://www.magiclantern.fm/forum/index.php?topic=18083.225) but the main developers agreed that it should be fixed (http://www.magiclantern.fm/forum/index.php?topic=18083.msg178163#msg178163). Not that I'm complaining--just bringing @aprofiti up to speed on this. Maybe this could be incorporated into all the great changes he's making to this branch. It is after all an "Experimental" branch.  :D
How many characters in lens_info.name? I think a minimum of 48-64 should be addressed to cover most lenses.
Some lenses use a very long name, especially Carl Zeiss (ex. Carl Zeiss Vario-Sonnar T* 100-300 mm f/ 4.5-5.6 C/Y -> 52 characters).

I tried with 64 byte in lens.h and after updating lua_lens.c wrapper to avoid weird aperture values, I have got full name in .xmp file but Silent Pictures wasn't working anymore (error: "Manual exposure is required."). I wasn't able to test .mlv
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on January 03, 2018, 05:09:58 AM
How many characters in lens_info.name? I think a minimum of 48-64 should be addressed to cover most lenses.

I ran some tests a while back to see how characters are allowed in Adobe Camera Raw and I never reached a limit. The 32 character limit in MLV is too short for a lens that should work fine in ACR: "Zeiss Makro-Planar T* 2/100 ZF.2" If it could be doubled to 64 characters that should cover 99.99% of the lens names.

I tried with 64 byte in lens.h and after updating lua_lens.c wrapper to avoid weird aperture values, I have got full name in .xmp file but Silent Pictures wasn't working anymore (error: "Manual exposure is required."). I wasn't able to test .mlv

That's a common user error. Silent Pictures only works in manual exposure mode and you need to setup back button focus.

The problem with extending the lens name in MLV is that it will break some (most?) of the MLV processing apps.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 03, 2018, 11:55:07 AM
That's a common user error. Silent Pictures only works in manual exposure mode and you need to setup back button focus.

The problem with extending the lens name in MLV is that it will break some (most?) of the MLV processing apps.
I was in M Mode and checked Iso and Shutter time, but wasn't able to take pictures. It's first time I see this error, maybe is related to lens_info.name length? What about back button focus?

As name is internal to ML my general idea was to change his length, then check MLV should return lens name as before in LENS Block (because it copies only the first 32 characters) and then experimenting by adding ELNS Block to get a working export of full name's length. Could it works?

I'll be a little busy in next days. I don't know if I can work on it  :-\
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 05, 2018, 07:00:18 PM
Got it!

Was a problem with missing '\0' in mlv.c when strncpy() of legacy lensName.
I made some tests with mlv_rec and mlv_dump and I can find "ELNS" Block and extended name using a Hex Editor, but i need to figure out why mlv_dump prints an empty name in ELNS Block.

Note: I get Black screen in LV when entering again after a raw recording. Need to test if it's a bug from those changes or not.

Also found what was the problem with Silent Pictures: I was using .mo compiled from lua_fix because in manual_lens_info branch it won't compile. I updated src and compiled it from scratch and now it's working with 64 byte lensName.

Should I commit to this branch or maybe it's better to merge PR first and then experiment with longer Lens Name?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on January 08, 2018, 11:55:53 AM
experimenting sounds like a good way to go forward
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 10, 2018, 11:34:40 PM
experimenting sounds like a good way to go forward

I still can't figure out why I get a black LV and no Ml menu when switching in and out from LV after a raw video...
LV works good when using Silent Pictures.

@a1ex or @g3gg0 Can you review this Patch (https://pastebin.com/mvfGhXVd) to see if I missed something or there is something wrong with my camera? Problem will occur also when changing only the length of lens_info.name without implementing ELNS Block, this let me think there is something related to update.

Also I get wrong prefix for .xmp files like "NIL0185.xmp". I tried to reset Canon's config but It didn't solved the issue
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on January 11, 2018, 12:11:35 AM
One caveat: after changing a structure in ML core, you have to recompile all modules; otherwise, they will no longer be compatible, binary-wise. I should include this change in lua_fix before merging into unified, as that branch has the module API version already bumped (a mechanism used to prevent undefined behavior from situations like this).

Sorry, unable to test more right now; maybe over the weekend. Overall I like where this is heading.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 12, 2018, 12:14:14 AM
Sorry, unable to test more right now; maybe over the weekend. Overall I like where this is heading.

While testing again I noticed that when LV is black, pressing info button will let you switch to Canon overlay and pressing again will let you see ML overlay (it's frozen), but again on both screens no image from sensor... Like if LV wasn't correctly enabled again.

This happens also with mlv_lite which is still unmodified. Tracing lens_info.name usage in code will not help finding more functions to be updated to make they work with extended length.

Hope those infos are useful for debugging.. Let me know if you had time to check
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on February 02, 2018, 03:43:21 PM
Figured out the "black screen" issue (https://bitbucket.org/hudson/magic-lantern/pull-requests/887/manual-lens-info); building an experimental build with the latest changes from aprofiti.

edit: done, also included crop_rec_4k changes on a few models.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 02, 2018, 06:51:00 PM
Figured out the "black screen" issue (https://bitbucket.org/hudson/magic-lantern/pull-requests/887/manual-lens-info); building an experimental build with the latest changes from aprofiti.
Good! This one doesn’t has the experimental patch as it’s still in local because was expected to be committed after solving config’s lib refactor.

I have a question to you @a1ex: Do you prefer a new PR containing only the changeset related to lens_info.name, ELNS block and mlv_dump so it can be merged relatively fast in crop-rec-4k (and can be tested by users) or in manual_lens_info (as now it should have an updated mlv_dump and mlv_rec), or a PR with the continuation of manual_lens_info?

Work in progress:
- refactor of config.lua (causing a large dalay because a recursive implementation to allow traversing submenu is causing thrubles regarding scoping an passing tables as argument to allow copy by reference...)
- add apertureMin/apertureMax to be saved from lua script (already implemented, just needs to be committed), so can be saved in “ELSN” block and maybe implement in lens.lua a full lower and upper bound for aperure when no custom f-values specified (currently only lower bound is computed automatically)
- maybe add lens name to Flex Info to camera doesn’t already have (also add support for 50D, doesn’t seems easy as expected) to allow a quick check on which lens is selected without opening menu

Desidered:
- find a way to show current aperture in Photo info display. Can we use PatchManager to modify canon’s aperture value displayed? Or better... Use Qemu’s MPU Emulation to send a message to cpu regarding aperture?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on February 02, 2018, 08:48:36 PM
Let's continue with manual_lens_info (i.e. stay on topic). I've already included the crop_rec_4k stuff on the models that can be built from that branch, and I can include it anywhere later if it works well. For now, I still don't quite trust the change I did to lua_property (gut feeling says it will fail as soon as loading some more scripts that will eat the entire UMM memory buffer), but I'm not sure what to do to fix it either (I'll probably have to rework to the main memory backend).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: ToniX on February 09, 2018, 12:03:51 AM
I have installed the build "magiclantern-manual_lens_info.2018Feb02.600D102" on my 600D.

I've found there the crop_rec. mod, which  I was very curious to test but couldn't find anywhere else.
 
Unfortunatly, enabling the Crop-rec. module result in error and any other active module is being disabled as well.

( Instead, Lua script seem to work fine here, but I didn't tested extensively)

Is it this a known issue of the 600D or  is it there  some specific procedure to follow in order to make it work?



Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on February 09, 2018, 02:35:56 AM
crop_rec and lossless compression aren't yet available on the 600D. It might be possible though there is very little memory on that camera to work with. Back on topic--how is the manual lens lua script working on the 600D?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on February 09, 2018, 11:58:12 AM
@a1ex
Ran my first test with magiclantern-manual_lens_info.2018Feb02.60D111 in photomode Av using the included lens.lua script together with my Samyang 8mm Fishey. Since 60D is not on 4K I thought it should not be a problem but unfurtunately I got the following assert after taking a picture.

Code: [Select]
ML ASSERT:
0
at ../../src/mem.c:844 (__mem_free), task PropMgr
lv:0 mode:2

PropMgr stack: 110c68 [110f68-10ff68]
0x00BA1C9C @ bacfb8:110d00
0xUNKNOWN  @ ba1d80:110ce0
0xUNKNOWN  @ bb13d0:110cd0
0x00BC8754 @ bb14c4:110cc8
0x00B9D0FC @ bc87c0:110cb8
0x1FE00CB8 @ 1fe036ac:110c98
0x1FE00378 @ 1fe00d14:110c68

Magic Lantern version : manual_lens_info.2018Feb02.60D111
Mercurial changeset   : e7ca98e11cd4 (manual_lens_info)
Built on 2018-02-02 19:26:24 UTC by jenkins@nightly.
Free Memory  : 313K + 1320K

Still both .CR2 and .xmp files got written to the card. 
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: ToniX on February 09, 2018, 12:45:19 PM
@dfort   
crop_rec and lossless compression aren't yet available on the 600D. It might be possible though there is very little memory on that camera to work with. Back on topic--how is the manual lens lua script working on the 600D?

Thank you for the confirmation. ::)

 As for the script I can only tell you what happens in the camera:

1) After activating the module, go to the> Script screen.
2) select the Manual Lens Information item in a sub menu Run> Run
3) I follow the instructions and go to the menu> Prefs> Lens Info Prefs
4) In the subMenu -> Manual lens there are 4 items:

Lens: shows the type of lens (EF-S18-55 .... ISII)
Focal length: 0
Openings: 5.6
Autoload lens: on

At the bottom of the screen, an orange text says "Chipped Lens Detected" (in fact, this lens has AF that does not focus well).
 
The camera need 2 trials before the module is deactivated
The module does not automatically reload, it must be reactivated each time after turning off the camera.
I noted that enabling LUA it use an amount  of memory: from 254K+1001k to 242K +720K

I hope I was helpful. In case you need anything else, please ask.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on February 09, 2018, 01:07:36 PM
@JohanJ:

The error happens from a module (they are dynamically loaded, so can't really interpret (https://www.magiclantern.fm/forum/index.php?topic=19933) the stack trace). Loading lua.mo and lens.lua, selecting some dummy manual lens, and taking a picture, doesn't trigger an assert.

However, XMPs are named nil0004.XMP, rather than IMG_0004.CR2. Does this happen for you?

Settings (ML/SETTINGS directory) and steps to reproduce your issue?

@ToniX: it's  a build system issue, the crop_rec module shouldn't have been included in the first place.

Cannot reproduce the "need 2 trials before the module is deactivated" part (not even sure what exactly you mean by that).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: JohanJ on February 09, 2018, 01:54:44 PM
Quote
However, XMPs are named nil0004.XMP, rather than IMG_0004.CR2. Does this happen for you?
Yes. IMG_6764.CR2 got a sidecar file named nil6764.XMP

ML/SETTINGS can be found here  https://drive.google.com/open?id=1uW_69RLXEnGiBBSmZIaSUbGOvb-Z5GDv (https://drive.google.com/open?id=1uW_69RLXEnGiBBSmZIaSUbGOvb-Z5GDv):
I added an nil...xmp as well as the latet ASSERT.LOG to this folder.

Did the following steps (nothing fancy)
1. Activated lua.mo (no other module was active in parallel!)
2. restarted cam
3. Started lens.lua manually
4. Chose one of the 3 lenses suggested
5. Took a picture --> no assert
6. Changed script to autorun
7. restarted cam
8. took a picture --> ASSERT
9. Changed script setting from autrun back to manual
10. restarted cam
11. stared script manually
12. took a picture --> ASSERT

My first impression was that the assert was caused by a the autorun setting. But that was proven wrong as the assert also triggert when changing back to manual start. There was no logic explanation, I fear
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 09, 2018, 02:19:02 PM
@JohanJ:
However, XMPs are named nil0004.XMP, rather than IMG_0004.CR2. Does this happen for you?

It happens also to me, it's a problem I noticed before and asked for feedback some posts ago.
If I remember correcly, it showed up in the middle of developing and wasn't sure if was related to canon's settings.

I tried now with 04April build and .xmp file are named correctly.

EDIT: reproduced ASSERT. It showed up on first picture after enabling script. It's the first time I see it.
@a1ex maybe it is related to commit 396a8a9 (https://bitbucket.org/hudson/magic-lantern/commits/b0b332a7ea8355c2a97d28e38e34b5b191963c4f?at=manual_lens_info) or b0b332a (https://bitbucket.org/hudson/magic-lantern/commits/02e0430e2d4c090fcae31a9f6cd0f267538b63f1?at=manual_lens_info)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: ToniX on February 09, 2018, 02:31:42 PM
@ a1ex

@ToniX: it's  a build system issue, the crop_rec module shouldn't have been included in the first place.

Cannot reproduce the "need 2 trials before the module is deactivated" part (not even sure what exactly you mean by that).

While testing a lot of different builds I have noted that in some of them (couldn't tell exactly which ones) it requires more than one camera on/off cycle for the modules to be activated/deactivated.

To be explicit: when enabling a module and restarting the camera, the module is not being loaded indeed, and the "ON-Will load" text is still there.

 Two,  sometime three camera restarts, are needed to have it enbled. Same at reverse.

This is a confusing and time consuming issue, but I thougtht it was marginal afterall, given all the ML benefits.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Walter Schulz on February 09, 2018, 02:43:29 PM
If you don't give ML enough time to shutdown properly you will see this happen.
Treat it right (= Let some seconds pass *after* power down) and modules will load/unload as intended.

More serious issue: After opening card slot (after!) cam will access card and if you do not wait for write operations to end you endanger your filesystem integrity. Or short: You can kill your card if you're impatient.
This access will happen with power off, too!

I think it's time reading some lines documented in Top of page -> User Guide -> Install Guide
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on February 09, 2018, 04:35:19 PM
It happens also to me, it's a problem I noticed before and asked for feedback some posts ago.

Sorry, didn't notice it before. Can you try this change? (based on api_test.lua)
Code: [Select]
diff -r e7ca98e11cd4 scripts/lib/xmp.lua
--- a/scripts/lib/xmp.lua
+++ b/scripts/lib/xmp.lua
@@ -37,3 +37,3 @@
 function xmp.get_sidecar_filename()
-    return string.format("DCIM/%03dCANON/%s%04d.XMP", dryos.shooting_card.folder_number, dryos.prefix, dryos.shooting_card.file_number)
+    return dryos.dcim_dir.path ..  dryos.image_prefix .. string.format("%04d", dryos.shooting_card.file_number) .. ".XMP"
 end

This should also cover camera models that do not use 100CANON, but 100EOS5D or whatever. There are some unresolved issues with certain settings, such as _MG_1234 or custom image prefix settings on 5D3, maybe also other models (issues revealed by api_test.lua).

Quote
@a1ex maybe it is related to commit 396a8a9 (https://bitbucket.org/hudson/magic-lantern/commits/b0b332a7ea8355c2a97d28e38e34b5b191963c4f?at=manual_lens_info) or b0b332a (https://bitbucket.org/hudson/magic-lantern/commits/02e0430e2d4c090fcae31a9f6cd0f267538b63f1?at=manual_lens_info)

Yes, clearly. I'm afraid the fix must be done in the memory backend.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: ToniX on February 09, 2018, 11:09:24 PM
@Walter Schulz    Thanks. I will add few second more, over the 5s as recommended by the UG. :D
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on February 10, 2018, 10:30:51 PM
I tested the last manual
lens info build from
experiments, and I had some customs scripts i made that failed to load because the filename was too long, did the max filename got reduced recently?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Walter Schulz on February 11, 2018, 12:25:25 AM
Cam does SFN (8.3).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 12, 2018, 01:49:15 AM
This should also cover camera models that do not use 100CANON, but 100EOS5D or whatever. There are some unresolved issues with certain settings, such as _MG_1234 or custom image prefix settings on 5D3, maybe also other models (issues revealed by api_test.lua).

Yes, I made a quick test on a custom build and It appears to work. Thank You a1ex!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 16, 2018, 04:38:32 AM
I made a new PR (https://bitbucket.org/hudson/magic-lantern/pull-requests/904/manual-lens-info-pt2/diff) with experimental 64byte lens name and config.lua rework.

@dfort Can you try it and check if extended name is saved correctly? I tested mostly with silent pictures and some raw video with mlv_rec, but no tests with mlv_lite.

New version of config's lib should be backward compatible with script which use previous version, but it needs to be tested further.
Maybe is better to post a description of what's changed in lua's discussion after a review.

EDIT: There is an issue with the refactor of config.lua code, It work with previous version but not with the one in the PR. I need to figure out where is the problem... I'll try tomorrow if I have some time
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on February 16, 2018, 07:22:02 AM
Ok, let me know when it is ready to test. The easiest way to check it with the Zeiss Makro-Planar 100mm ZF.2 lens because the Zeiss ZF.2 lenses work out of the box with Adobe Camera Raw. That's how I first found out about the lens name length issue.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 16, 2018, 01:45:54 PM
Ok, let me know when it is ready to test. The easiest way to check it with the Zeiss Makro-Planar 100mm ZF.2 lens because the Zeiss ZF.2 lenses work out of the box with Adobe Camera Raw. That's how I first found out about the lens name length issue.

Fixed Typos introduced right before committing, I'll refactor later when I'm not in a hurry.

Particular attention should be paid to "ELNS" Block metadata. Please check validity with FRSP and Raw Rec (mlv_rec, mlv_lite).
Let me know  ;)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on February 16, 2018, 05:16:50 PM
First test, using just the unmodified lua script, simply remove a CPU lens and reattach in photo mode (not in LV):

Code: [Select]
ML ASSERT:
0
at ../../src/mem.c:781 (__mem_malloc), task PropMgr
lv:0 mode:3

PropMgr stack: 14cc00 [14cf68-14bf68]
0x00AB01DC @ abb4f8:14cce0
0xUNKNOWN  @ ab02c0:14ccc0
0x00081A94 @ ad1170:14cc80
0x0007F5E4 @ 81ad0:14cc30
0x0007EF78 @ 7f640:14cc00

Magic Lantern version : Nightly.2018Feb16.700D115
Mercurial changeset   : c274347c2d19 (manual_lens_info_64byte) tip
Built on 2018-02-16 15:04:22 UTC by rosiefort@RosieFoComputer.
Free Memory  : 128K + 2782K

Next test, activate a few lenses with long names, and again in photo mode shoot a CR2.

Here's the XMP sidecar file:

Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FocalLength="100/1"
    aux:SerialNumber="0"
    exif:FNumber="20/10"
    aux:Lens="Zeiss Makro-Planar T* 2/100 ZF.2">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

But it also saved an ASSERT log.

Code: [Select]
ML ASSERT:
0
at ../../src/mem.c:844 (__mem_free), task PropMgr
lv:0 mode:3

PropMgr stack: 14cc68 [14cf68-14bf68]
0x00AB047C @ abb798:14cd00
0xUNKNOWN  @ ab0560:14cce0
0xUNKNOWN  @ abfbb0:14ccd0
0x00AD7094 @ abfca4:14ccc8
0x00AAB8DC @ ad7100:14ccb8
0x0007F5E4 @ 81fcc:14cc98
0x0007EF78 @ 7f640:14cc68

Magic Lantern version : Nightly.2018Feb16.700D115
Mercurial changeset   : c274347c2d19 (manual_lens_info_64byte) tip
Built on 2018-02-16 15:04:22 UTC by rosiefort@RosieFoComputer.
Free Memory  : 128K + 2774K

FRSP DNG, the XMP file looks fine but the ASSERT log is slightly different:

Code: [Select]
ML ASSERT:
0
at ../../src/mem.c:844 (__mem_free), task PropMgr
lv:0 mode:3

PropMgr stack: 14cc68 [14cf68-14bf68]
0x00AB354C @ abe868:14cd00
0xUNKNOWN  @ ab3630:14cce0
0xUNKNOWN  @ ac2c80:14ccd0
0x00ADA164 @ ac2d74:14ccc8
0x00AAE9AC @ ada1d0:14ccb8
0x0007F5E4 @ 81fcc:14cc98
0x0007EF78 @ 7f640:14cc68

Magic Lantern version : Nightly.2018Feb16.700D115
Mercurial changeset   : c274347c2d19 (manual_lens_info_64byte) tip
Built on 2018-02-16 15:04:22 UTC by rosiefort@RosieFoComputer.
Free Memory  : 128K + 2737K

FRSP MLV using mlv_lite and mlv_dump from your manual_lens_info_64byte branch (current mlv_dump skips the ELNS block):

Code: [Select]
Block: LENS
  Offset: 0x000001f0
    Size: 96
    Time: 1094.548000 ms
     Name:        'Zeiss Makro-Planar T* 2/100 ZF.'
     Serial:      '000000000'
     Focal Len:   100 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000
Block: ELNS
  Offset: 0x00000250
    Size: 95
    Time: 1094.573000 ms
     Name:                'Zeiss Makro-Planar T* 2/100 ZF.2'
     Focal Length Min:    0 mm
     Focal Length Max:    0 mm
     Aperture Min:        f/0.00
     Aperture Max:        f/0.00
     Version:             0
     Extender Info:       0x00
     Capabilities:        0x00
     Chipped:             0x01

Yay! However, it also saved an XMP sidecar file. The XMP file looks fine but I don't see the point in saving an XMP file when shooting MLV. In my test the MLV file is named 28160000.MLV and the XMP file is named IMG_2816.XMP.

There was also an ASSERT log:

Code: [Select]
ML ASSERT:
0
at ../../src/mem.c:844 (__mem_free), task PropMgr
lv:0 mode:3

PropMgr stack: 14cc68 [14cf68-14bf68]
0x00AB738C @ ac26a8:14cd00
0xUNKNOWN  @ ab7470:14cce0
0xUNKNOWN  @ ac6ac0:14ccd0
0x00ADDFA4 @ ac6bb4:14ccc8
0x00AB27EC @ ade010:14ccb8
0x0007F5E4 @ 81fcc:14cc98
0x0007EF78 @ 7f640:14cc68

Magic Lantern version : Nightly.2018Feb16.700D115
Mercurial changeset   : c274347c2d19 (manual_lens_info_64byte) tip
Built on 2018-02-16 15:04:22 UTC by rosiefort@RosieFoComputer.
Free Memory  : 128K + 2700K

Next I shot several FRSP in MLV format in succession to see what happens if several shots are saved to a single MLV. It wasn't very practical because the console kept popping up with the ASSERT message and it saved a log for each exposure. It saved a single XMP file and in the MLV file each VIDF block was followed by an RTCI, EXPO, LENS and ELNS block. I believe this is by design because when taking still photos you can change lenses while continuing to save to the same MLV file.

Moving on to video tests -- and some good news.

MLV video shot in mlv_lite worked perfectly! This time as expected there was only a single LENS and ELNS block and none after each VIDF block like in the still photo test. The LENS and ELNS blocks have the same information as the still test so I'll save some space on this post and not duplicate it.

MLV shot with mlv_rec and mlv_snd also worked perfectly.

So it looks like you got the movie modes working and only need to get the ASSERT messages in still photo modes figured out. Saving XMP when shooting silent MLV files should also be addressed.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 16, 2018, 05:39:40 PM
Thank you dfort for the extensive testing :D
So it looks like you got the movie modes working and only need to get the ASSERT messages in still photo modes figured out. Saving XMP when shooting silent MLV files should also be addressed.

The ASSERT messages where introduced after a1ex's attempt to fix issues related to calling lua's property.
He said that a rework of lua's memory backend is necessary to address this problem, so we need to wait news from him.

Regarding sidecar generation when using MLV format, I don't remember if it was happening from before or can be caused by the fix regarding sidecar's filename.

I'll test and look if is an easy task... disabling xmp engine when mlv format is selected should fix this but I have no idea where this can be checked correctly.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on February 18, 2018, 04:49:56 PM
This one happens when trying to start a task from the property handler. Still, something I have to fix in the backend somehow.

Another issue: I had to delete the old config file (lens.cfg), otherwise got some errors in config.lua. I can try to reproduce if needed.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 18, 2018, 06:52:15 PM
Another issue: I had to delete the old config file (lens.cfg), otherwise got some errors in config.lua. I can try to reproduce if needed.

It's because menu are serialized in a different format now (tables), instead simple "config" are key -> value pairs to be backward compatible (is it ok, or i preferable to follow new format?)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on February 18, 2018, 08:00:18 PM
It's fine by me, but I'd change the config file extension.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on February 19, 2018, 12:04:17 AM
The ASSERT messages where introduced after a1ex's attempt to fix issues related to calling lua's property.
He said that a rework of lua's memory backend is necessary to address this problem, so we need to wait news from him.

Hopefully solved. Wasn't easy (tried the same approach when merging the first PR, but could not pass the tests from selftest.mo, so I ended up committing a workaround - that turned out not to be very good); however, today I've found the issue: yet another quirk in Canon's large memory allocators (the reason why it was crashing after ~1000 blocks or so (https://bitbucket.org/hudson/magic-lantern/commits/c1c17862229a788f58879bbca54f91b4c69dd683)). Seems to work in my quick tests, but I'd still like to sleep on it before including in lua_fix and crop_rec_4k.

I've extracted a patch for crop_rec_4k with only the changes related to ELNS and 64-byte lens name (mlv_dump, mlv*, silent and lens.h), but I'm afraid I have no more energy left to run any tests today (so I'm not committing it yet). If you try it, please report back.

Code: [Select]
# HG changeset patch
# User alex@thinkpad
# Date 1518995384 -3600
#      Mon Feb 19 00:09:44 2018 +0100
# Branch crop_rec_4k
# Node ID 804086b3e47771bd72ed123bdd797c608738bf4f
# Parent  5138c0050854c3c44eb2de21cf63c445baa0b245
* Experimental: 64Byte Extended Lens Name
* mlv_dump.c: Fix duplicate ELNS Block processing
* mlv_rec.c: Fix duplicate ELNS Block processing

diff -r 5138c0050854 -r 804086b3e477 modules/lua/lua_lens.c
--- a/modules/lua/lua_lens.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/lua/lua_lens.c Mon Feb 19 00:09:44 2018 +0100
@@ -72,7 +72,55 @@
 static int luaCB_lens_newindex(lua_State * L)
 {
     LUA_PARAM_STRING_OPTIONAL(key, 2, "");
-    if(!strcmp(key, "name") || !strcmp(key, "focal_length") || !strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af"))
+    if(!strcmp(key, "name"))
+    {
+        LUA_PARAM_STRING(value, 3);
+        strncpy(lens_info.name, value, sizeof(lens_info.name)-1);
+    }
+    else if(!strcmp(key, "exists"))
+    {
+        LUA_PARAM_BOOL(value, 3);
+        lens_info.lens_exists = value;
+    }
+    else if(!strcmp(key, "focal_length"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.focal_len = value;
+    }
+    else if(!strcmp(key, "focal_min"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_min = value;
+    }
+    else if(!strcmp(key, "focal_max"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_max = value;
+    }
+    else if(!strcmp(key, "manual_aperture"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        lens_info.aperture = (int)(value * 10);
+        lens_info.raw_aperture = VALUE2RAW(aperture, lens_info.aperture);
+    }
+    else if(!strcmp(key, "aperture_min"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_min = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "aperture_max"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_max = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "serial"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_serial = value;
+    }
+    else if(!strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af") || !strcmp(key, "is_chipped"))
     {
         return luaL_error(L, "'%s' is readonly!", key);
     }
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_lite/mlv_lite.c
--- a/modules/mlv_lite/mlv_lite.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_lite/mlv_lite.c Mon Feb 19 00:09:44 2018 +0100
@@ -350,7 +350,8 @@
 static GUARDED_BY(RawRecTask)   mlv_rawc_hdr_t rawc_hdr;
 static GUARDED_BY(RawRecTask)   mlv_idnt_hdr_t idnt_hdr;
 static GUARDED_BY(RawRecTask)   mlv_expo_hdr_t expo_hdr;
-static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
+static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
+static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t elns_hdr;
 static GUARDED_BY(RawRecTask)   mlv_rtci_hdr_t rtci_hdr;
 static GUARDED_BY(RawRecTask)   mlv_wbal_hdr_t wbal_hdr;
 static GUARDED_BY(LiveViewTask) mlv_vidf_hdr_t vidf_hdr;
@@ -2807,7 +2808,8 @@
 
     mlv_fill_idnt(&idnt_hdr, mlv_start_timestamp);
     mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
-    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
     mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
     mlv_fill_wbal(&wbal_hdr, mlv_start_timestamp);
 
@@ -2827,6 +2829,7 @@
     if (FIO_WriteFile(f, &idnt_hdr, idnt_hdr.blockSize) != (int)idnt_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) return 0;
+    if (FIO_WriteFile(f, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &wbal_hdr, wbal_hdr.blockSize) != (int)wbal_hdr.blockSize) return 0;
     if (mlv_write_vers_blocks(f, mlv_start_timestamp)) return 0;
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv.c
--- a/modules/mlv_rec/mlv.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv.c Mon Feb 19 00:09:44 2018 +0100
@@ -55,13 +55,35 @@
     hdr->autofocusMode = af_mode;
     hdr->flags = 0;
 
+    char name[33];
     char buf[33];
+    snprintf(name, 32, "%s", lens_info.name);
     snprintf(buf, sizeof(buf), "%X%08X", (uint32_t) (lens_info.lens_serial >> 32), (uint32_t)(lens_info.lens_serial & 0xFFFFFFFF));
-   
-    strncpy((char *)hdr->lensName, lens_info.name, 32);
+
+    strncpy((char *)hdr->lensName, name, 32);
     strncpy((char *)hdr->lensSerial, buf, 32);
 }
 
+void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp){
+    /* prepare header */
+    mlv_set_type((mlv_hdr_t *)hdr, "ELNS");
+    mlv_set_timestamp((mlv_hdr_t *)hdr, start_timestamp);
+    hdr->blockSize = sizeof(mlv_elns_hdr_t);
+
+    char bufName[65];
+    snprintf(bufName, 64, "%s", lens_info.name);
+
+    strncpy((char *)hdr->lensName, bufName, 64);
+    hdr->focalLengthMin = lens_info.lens_focal_min;
+    hdr->focalLengthMax = lens_info.lens_focal_max;
+    hdr->apertureMin = RAW2VALUE(aperture, lens_info.raw_aperture_min) / 10.0;
+    hdr->apertureMax = RAW2VALUE(aperture, lens_info.raw_aperture_max) / 10.0;
+    hdr->version = lens_info.lens_version;
+    hdr->extenderInfo = lens_info.lens_extender;
+    hdr->capabilities = lens_info.lens_capabilities;
+    hdr->chipped = lens_info.lens_exists;
+}
+
 void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp)
 {
     /* prepare header */
@@ -151,7 +173,7 @@
     mlv_set_type((mlv_hdr_t *)hdr, "IDNT");
     mlv_set_timestamp((mlv_hdr_t *)hdr, start_timestamp);
     hdr->blockSize = sizeof(mlv_idnt_hdr_t);
-   
+
     hdr->cameraModel = camera_model_id;
     memcpy(hdr->cameraName, camera_model, 32);
     memcpy(hdr->cameraSerial, camera_serial, 32);
@@ -163,16 +185,16 @@
 {
     int block_length = (strlen(version_string) + sizeof(mlv_vers_hdr_t) + 1 + 3) & ~3;
     mlv_vers_hdr_t *header = malloc(block_length);
-   
+
     /* prepare header */
     mlv_set_type((mlv_hdr_t *)header, "VERS");
     mlv_set_timestamp((mlv_hdr_t *)header, start_timestamp);
     header->blockSize = block_length;
     header->length = strlen(version_string);
-   
+
     char *vers_hdr_payload = (char *)&header[1];
     strcpy(vers_hdr_payload, version_string);
-   
+
     *hdr = header;
 }
 
@@ -239,12 +261,12 @@
 {
     int mod = -1;
     int error = 0;
-   
+
     do
     {
         /* get next loaded module id */
         mod = module_get_next_loaded(mod);
-       
+
         /* make sure thats a valid one */
         if(mod >= 0)
         {
@@ -252,7 +274,7 @@
             const char *mod_name = module_get_name(mod);
             const char *mod_build_date = module_get_string(mod, "Build date");
             const char *mod_last_update = module_get_string(mod, "Last update");
-           
+
             if(mod_name != NULL)
             {
                 /* just in case that ever happens */
@@ -264,15 +286,15 @@
                 {
                     mod_last_update = "(no version)";
                 }
-               
+
                 /* separating the format string allows us to measure its length for malloc */
                 const char *fmt_string = "%s built %s; commit %s";
                 int buf_length = strlen(fmt_string) + strlen(mod_name) + strlen(mod_build_date) + strlen(mod_last_update) + 1;
                 char *version_string = malloc(buf_length);
-               
+
                 /* now build the string */
                 snprintf(version_string, buf_length, fmt_string, mod_name, mod_build_date, mod_last_update);
-               
+
                 /* and finally remove any newlines, they are annoying */
                 for(unsigned int pos = 0; pos < strlen(version_string); pos++)
                 {
@@ -281,23 +303,23 @@
                         version_string[pos] = ' ';
                     }
                 }
-               
+
                 /* let the mlv helpers build the block for us */
                 mlv_vers_hdr_t *hdr = NULL;
                 mlv_build_vers(&hdr, mlv_start_timestamp, version_string);
-               
+
                 /* try to write to output file */
                 if(FIO_WriteFile(f, hdr, hdr->blockSize) != (int)hdr->blockSize)
                 {
                     error = 1;
                 }
-               
+
                 /* free both temporary string and allocated mlv block */
                 free(version_string);
                 free(hdr);
             }
         }
     } while(mod >= 0 && !error);
-   
+
     return error;
 }
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv.h
--- a/modules/mlv_rec/mlv.h Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv.h Mon Feb 19 00:09:44 2018 +0100
@@ -165,6 +165,22 @@
 }  mlv_lens_hdr_t;
 
 typedef struct {
+    uint8_t     blockType[4];    /* ELNS - Extended LENS block with longer lens name and optional fields, depending on camera */
+    uint32_t    blockSize;
+    uint64_t    timestamp;
+    uint16_t    focalLengthMin;  /* shortest focal length in mm                       */
+    uint16_t    focalLengthMax;  /* longest focal length in mm                        */
+    uint16_t    apertureMin;     /* lowest f-number * 100                             */
+    uint16_t    apertureMax;     /* highest f-number * 100                            */
+    uint32_t    version;         /* lens internal version number, if available        */
+    uint8_t     extenderInfo;    /* extender information, if provided by camera       */
+    uint8_t     capabilities;    /* capability information, if provided by camera     */
+    uint8_t     chipped;         /* when not zero, lens is communicating with camera  */
+    uint8_t     lensName[64];     /* full lens string, null terminated                 */
+    // TODO: Review Specs
+}  mlv_elns_hdr_t;
+
+typedef struct {
     uint8_t     blockType[4];       /* RTCI: real-time clock metadata */
     uint32_t    blockSize;          /* total frame size */
     uint64_t    timestamp;          /* hardware counter timestamp for this frame (relative to recording start) */
@@ -287,6 +303,7 @@
 void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
+void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv_dump.c Mon Feb 19 00:09:44 2018 +0100
@@ -1894,6 +1894,7 @@
     lv_rec_file_footer_t lv_rec_footer;
     mlv_file_hdr_t main_header;
     mlv_lens_hdr_t lens_info;
+    mlv_elns_hdr_t elns_info;
     mlv_expo_hdr_t expo_info;
     mlv_idnt_hdr_t idnt_info;
     mlv_wbal_hdr_t wbal_info;
@@ -1904,6 +1905,7 @@
     /* initialize stuff */
     memset(&lv_rec_footer, 0x00, sizeof(lv_rec_file_footer_t));
     memset(&lens_info, 0x00, sizeof(mlv_lens_hdr_t));
+    memset(&elns_info, 0x00, sizeof(mlv_elns_hdr_t));
     memset(&expo_info, 0x00, sizeof(mlv_expo_hdr_t));
     memset(&idnt_info, 0x00, sizeof(mlv_idnt_hdr_t));
     memset(&wbal_info, 0x00, sizeof(mlv_wbal_hdr_t));
@@ -3652,6 +3654,46 @@
                     print_msg(MSG_INFO, "     Flags:       0x%08X\n", lens_info.flags);
                 }
             }
+            else if(!memcmp(buf.blockType, "ELNS", 4))
+            {
+                uint32_t hdr_size = MIN(sizeof(mlv_elns_hdr_t), buf.blockSize);
+
+                if(fread(&elns_info, hdr_size, 1, in_file) != 1)
+                {
+                    print_msg(MSG_ERROR, "File ends in the middle of a block\n");
+                    goto abort;
+                  }
+
+                /* skip remaining data, if there is any */
+                file_set_pos(in_file, position + elns_info.blockSize, SEEK_SET);
+
+                lua_handle_hdr(lua_state, buf.blockType, &elns_info, sizeof(elns_info));
+
+                if(verbose)
+                {
+                    //TODO: Check Aperture Min/Max and if other fields are needed
+                    print_msg(MSG_INFO, "     Name:                '%s'\n", elns_info.lensName);
+                    print_msg(MSG_INFO, "     Focal Length Min:    %d mm\n", elns_info.focalLengthMin);
+                    print_msg(MSG_INFO, "     Focal Length Max:    %d mm\n", elns_info.focalLengthMax);
+                    print_msg(MSG_INFO, "     Aperture Min:        f/%.2f\n", (double)elns_info.apertureMin);
+                    print_msg(MSG_INFO, "     Aperture Max:        f/%.2f\n", (double)elns_info.apertureMax);
+                    print_msg(MSG_INFO, "     Version:             %d\n", elns_info.version);
+                    print_msg(MSG_INFO, "     Extender Info:       0x%02X\n", elns_info.extenderInfo);
+                    print_msg(MSG_INFO, "     Capabilities:        0x%02X\n", elns_info.capabilities);
+                    print_msg(MSG_INFO, "     Chipped:             0x%02X\n", elns_info.chipped);
+                }
+
+                if(mlv_output && !no_metadata_mode && (!extract_block || !strncasecmp(extract_block, (char*)elns_info.blockType, 4)))
+                {
+                    /* correct header size if needed */
+                    elns_info.blockSize = sizeof(mlv_elns_hdr_t);
+                    if(fwrite(&elns_info, elns_info.blockSize, 1, out_file) != 1)
+                    {
+                        print_msg(MSG_ERROR, "Failed writing into .MLV file\n");
+                        goto abort;
+                    }
+                }
+            }
             else if(!memcmp(mlv_block->blockType, "INFO", 4))
             {
                 mlv_info_hdr_t block_hdr = *(mlv_info_hdr_t *)mlv_block;
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv_rec.c
--- a/modules/mlv_rec/mlv_rec.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv_rec.c Mon Feb 19 00:09:44 2018 +0100
@@ -195,6 +195,7 @@
 
 static mlv_expo_hdr_t last_expo_hdr;
 static mlv_lens_hdr_t last_lens_hdr;
+static mlv_elns_hdr_t last_elns_hdr;
 static mlv_wbal_hdr_t last_wbal_hdr;
 static mlv_styl_hdr_t last_styl_hdr;
 
@@ -1449,14 +1450,17 @@
             trace_write(raw_rec_trace_ctx, "[polling_cbr] queueing INFO blocks");
             mlv_expo_hdr_t *expo_hdr = malloc(sizeof(mlv_expo_hdr_t));
             mlv_lens_hdr_t *lens_hdr = malloc(sizeof(mlv_lens_hdr_t));
+            mlv_elns_hdr_t *elns_hdr = malloc(sizeof(mlv_elns_hdr_t));
             mlv_wbal_hdr_t *wbal_hdr = malloc(sizeof(mlv_wbal_hdr_t));
 
             mlv_fill_expo(expo_hdr, mlv_start_timestamp);
             mlv_fill_lens(lens_hdr, mlv_start_timestamp);
+            mlv_fill_elns(elns_hdr, mlv_start_timestamp);
             mlv_fill_wbal(wbal_hdr, mlv_start_timestamp);
 
             msg_queue_post(mlv_block_queue, (uint32_t) expo_hdr);
             msg_queue_post(mlv_block_queue, (uint32_t) lens_hdr);
+            msg_queue_post(mlv_block_queue, (uint32_t) elns_hdr);
             msg_queue_post(mlv_block_queue, (uint32_t) wbal_hdr);
         }
 
@@ -2567,6 +2571,7 @@
         mlv_rtci_hdr_t rtci_hdr;
         mlv_expo_hdr_t expo_hdr;
         mlv_lens_hdr_t lens_hdr;
+        mlv_elns_hdr_t elns_hdr;
         mlv_idnt_hdr_t idnt_hdr;
         mlv_wbal_hdr_t wbal_hdr;
         mlv_styl_hdr_t styl_hdr;
@@ -2574,6 +2579,7 @@
         mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
         mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
         mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+        mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
         mlv_fill_idnt(&idnt_hdr, mlv_start_timestamp);
         mlv_fill_wbal(&wbal_hdr, mlv_start_timestamp);
         mlv_fill_styl(&styl_hdr, mlv_start_timestamp);
@@ -2585,10 +2591,12 @@
         idnt_hdr.timestamp = 4;
         wbal_hdr.timestamp = 5;
         styl_hdr.timestamp = 6;
+        elns_hdr.timestamp = 7;
 
         mlv_write_hdr(f, (mlv_hdr_t *)&rtci_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&expo_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&lens_hdr);
+        mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&idnt_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&wbal_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&styl_hdr);
@@ -3100,13 +3108,16 @@
 
         mlv_expo_hdr_t old_expo = last_expo_hdr;
         mlv_lens_hdr_t old_lens = last_lens_hdr;
+        mlv_elns_hdr_t old_elns = last_elns_hdr;
 
         mlv_fill_expo(&last_expo_hdr, mlv_start_timestamp);
         mlv_fill_lens(&last_lens_hdr, mlv_start_timestamp);
+        mlv_fill_elns(&last_elns_hdr, mlv_start_timestamp);
 
         /* update timestamp for comparing content changes */
         old_expo.timestamp = last_expo_hdr.timestamp;
         old_lens.timestamp = last_lens_hdr.timestamp;
+        old_elns.timestamp = last_elns_hdr.timestamp;
 
         /* write new state if something changed */
         if(memcmp(&last_expo_hdr, &old_expo, sizeof(mlv_expo_hdr_t)))
@@ -3123,6 +3134,13 @@
             memcpy(hdr, &last_lens_hdr, sizeof(mlv_lens_hdr_t));
             msg_queue_post(mlv_block_queue, (uint32_t) hdr);
         }
+        /* write new state if something changed */
+        if(memcmp(&last_elns_hdr, &old_elns, sizeof(mlv_elns_hdr_t)))
+        {
+            mlv_hdr_t *hdr = malloc(sizeof(mlv_elns_hdr_t));
+            memcpy(hdr, &last_elns_hdr, sizeof(mlv_elns_hdr_t));
+            msg_queue_post(mlv_block_queue, (uint32_t) hdr);
+        }
     }
 
     if(mlv_update_styl && (mlv_metadata & MLV_METADATA_SPORADIC))
diff -r 5138c0050854 -r 804086b3e477 modules/silent/silent.c
--- a/modules/silent/silent.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/silent/silent.c Mon Feb 19 00:09:44 2018 +0100
@@ -28,6 +28,7 @@
 extern WEAK_FUNC(ret_0) void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
+extern WEAK_FUNC(ret_0) void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
@@ -365,6 +366,7 @@
     mlv_rtci_hdr_t rtci_hdr;
     mlv_expo_hdr_t expo_hdr;
     mlv_lens_hdr_t lens_hdr;
+    mlv_elns_hdr_t elns_hdr;
     mlv_idnt_hdr_t idnt_hdr;
     mlv_wbal_hdr_t wbal_hdr;
     mlv_styl_hdr_t styl_hdr;
@@ -478,6 +480,7 @@
     mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
     mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
     mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
 
     expo_hdr.isoValue = metadata.iso;
     expo_hdr.shutterValue = 1000000000 / metadata.tvr;
@@ -486,7 +489,8 @@
     if (FIO_WriteFile(save_file, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) goto write_error;
-   
+    if (FIO_WriteFile(save_file, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) goto write_error;
+
     memset(&vidf_hdr, 0, sizeof(mlv_vidf_hdr_t));
     mlv_set_type((mlv_hdr_t *)&vidf_hdr, "VIDF");
     mlv_set_timestamp((mlv_hdr_t *)&vidf_hdr, mlv_start_timestamp);
diff -r 5138c0050854 -r 804086b3e477 src/lens.h
--- a/src/lens.h Sat Feb 17 22:37:51 2018 +0100
+++ b/src/lens.h Mon Feb 19 00:09:44 2018 +0100
@@ -37,7 +37,7 @@
 struct lens_info
 {
         void *                  token;
-        char                    name[ 32 ];
+        char                    name[ 64 ];
         unsigned                focal_len; // in mm
         unsigned                focus_dist; // in cm
         unsigned                IS; // PROP_LV_LENS_STABILIZE
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 19, 2018, 02:05:44 AM
I've extracted a patch for crop_rec_4k with only the changes related to ELNS and 64-byte lens name (mlv_dump, mlv*, silent and lens.h), but I'm afraid I have no more energy left to run any tests today (so I'm not committing it yet). If you try it, please report back.

I can't apply patch to crop_rec_4k branch: (updated my local repository by pulling latest commits from remote)
Code: [Select]
patch -f -p0 --input=/Users/alex/Desktop/pullML/tmp/a1exPatch64.diff
can't find file to patch at input line 15
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|# HG changeset patch
|# User alex@thinkpad
|# Date 1518995384 -3600
|#      Mon Feb 19 00:09:44 2018 +0100
|# Branch crop_rec_4k
|# Node ID 804086b3e47771bd72ed123bdd797c608738bf4f
|# Parent  5138c0050854c3c44eb2de21cf63c445baa0b245
|* Experimental: 64Byte Extended Lens Name
|* mlv_dump.c: Fix duplicate ELNS Block processing
|* mlv_rec.c: Fix duplicate ELNS Block processing
|
|diff -r 5138c0050854 -r 804086b3e477 modules/lua/lua_lens.c
|--- a/modules/lua/lua_lens.c Sat Feb 17 22:37:51 2018 +0100
|+++ b/modules/lua/lua_lens.c Mon Feb 19 00:09:44 2018 +0100
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
can't find file to patch at input line 75
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff -r 5138c0050854 -r 804086b3e477 modules/mlv_lite/mlv_lite.c
|--- a/modules/mlv_lite/mlv_lite.c Sat Feb 17 22:37:51 2018 +0100
|+++ b/modules/mlv_lite/mlv_lite.c Mon Feb 19 00:09:44 2018 +0100
--------------------------
No file to patch.  Skipping patch.
3 out of 3 hunks ignored
can't find file to patch at input line 106
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv.c
|--- a/modules/mlv_rec/mlv.c Sat Feb 17 22:37:51 2018 +0100
|+++ b/modules/mlv_rec/mlv.c Mon Feb 19 00:09:44 2018 +0100
--------------------------
No file to patch.  Skipping patch.
7 out of 7 hunks ignored
can't find file to patch at input line 247
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv.h
|--- a/modules/mlv_rec/mlv.h Sat Feb 17 22:37:51 2018 +0100
|+++ b/modules/mlv_rec/mlv.h Mon Feb 19 00:09:44 2018 +0100
--------------------------
No file to patch.  Skipping patch.
2 out of 2 hunks ignored
can't find file to patch at input line 281
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv_dump.c
|--- a/modules/mlv_rec/mlv_dump.c Sat Feb 17 22:37:51 2018 +0100
|+++ b/modules/mlv_rec/mlv_dump.c Mon Feb 19 00:09:44 2018 +0100
--------------------------
No file to patch.  Skipping patch.
3 out of 3 hunks ignored
can't find file to patch at input line 347
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv_rec.c
|--- a/modules/mlv_rec/mlv_rec.c Sat Feb 17 22:37:51 2018 +0100
|+++ b/modules/mlv_rec/mlv_rec.c Mon Feb 19 00:09:44 2018 +0100
--------------------------
No file to patch.  Skipping patch.
7 out of 7 hunks ignored
can't find file to patch at input line 436
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff -r 5138c0050854 -r 804086b3e477 modules/silent/silent.c
|--- a/modules/silent/silent.c Sat Feb 17 22:37:51 2018 +0100
|+++ b/modules/silent/silent.c Mon Feb 19 00:09:44 2018 +0100
--------------------------
No file to patch.  Skipping patch.
4 out of 4 hunks ignored
can't find file to patch at input line 473
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff -r 5138c0050854 -r 804086b3e477 src/lens.h
|--- a/src/lens.h Sat Feb 17 22:37:51 2018 +0100
|+++ b/src/lens.h Mon Feb 19 00:09:44 2018 +0100
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
Completed with errors, see above

Note: focus.c is also modified to avoid display problems when using lens with longer name
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on February 19, 2018, 03:16:05 PM
Try "patch -p1" or "hg import".

This was meant only as a temporary patch to test the changes to MLV tools on the more recent codebase (e.g. for mlv_dump), until merging.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 19, 2018, 05:09:56 PM
Try "patch -p1" or "hg import".
Nothing to do, recloned remote repo but It's partially applied lo lens.h, lua_len.c and silent.c

I'm trying to apply patch after this commit:

Merged in bouncyball/magic-lantern-merged/crop_rec_4k (pull request #901)
Revision: 17069
Changeset: e1794eb89f68170728b40b0153bb693a2ab9ed4c [e1794eb89f68]

Maybe your local repo is a bit different from remote and so can't find references?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: nikfreak on April 01, 2018, 11:19:04 AM
Does this already allow to "overwrite / assign" an attached cpu lens to be one of the list of supported non-cpu lenses??

example: fake an EF-S 24 STM to be a Nikkor/Zeiss lens...

I am thinking about reviving old vintage glas I own.
There are chips for glueing available as well as adpaters (e.g. M42 / Nikon) which already include a chip for focus confirmation beeps in photo mode. I don't own one of these yet but the adapters are stated to work in M/Av modes and report a fixed lens info and apterture e.g, 50mm f1.4. Some adapters seem to be programmable but let's skip that...
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on April 01, 2018, 01:21:31 PM
Does this already allow to "overwrite / assign" an attached cpu lens to be one of the list of supported non-cpu lenses??
Yes, It was extended to support adapters with AF confirm Chip.
There is a check which detect when adapter is used, it's based primarily on lens's name/focal length returned by chip (observed "1-65535mm" the one I have).

You have to disable this check if you want to override EF-S lenses, but shouldn't be the case you are talking about.

Saving XMP when shooting silent MLV files should also be addressed.
I was thinking to add a check inside xmp's lib to see which file format is selected.
Initially I was unsure if this could interfere with ML normal functionality like deflickr.mo because it need to save a .xml, but as this issues is present when silent.mo is used and it already have a function to check which file format is selected.

Is it possible to link code between modules? From silent.c to lua_camera.c? (Aldo I don't remember if silent.h exists)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: nikfreak on April 01, 2018, 01:29:42 PM
Great info. Thanks for clarification.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on April 03, 2018, 01:39:21 PM
(https://thumb.ibb.co/kcTUKx/VRAM0.jpg) (https://ibb.co/kcTUKx)

This show up when lua.mo is loaded with mlv_rec.mo and a lot of modules loaded.

Enabling only lua.mo and mlv_rec.mo will not shows this error but Raw recording is not working in both cases (Camera freezes after recording is stopped).
It works only without Lua loaded.

Recording works fine with mlv_lite but I get this even without Lua loaded if a lot of modules are enabled:

(https://thumb.ibb.co/fDn3CH/VRAM1_1.jpg) (https://ibb.co/fDn3CH)


Tested with manual_lens_info.2018Feb19 build and without enabling any script.
Note: I can see crop_rec.mo into modules list. Is this correct or should appears only in 4k branch?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: starbase64 on April 03, 2018, 02:30:50 PM
Hi,

i have downloaded magiclantern-manual_lens_info.2018Feb19.1100D105.zip for my 1100d, but camera can't start with this version.

SD LED always blinking...

regards

Maik
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on April 03, 2018, 04:08:00 PM
i have downloaded magiclantern-manual_lens_info.2018Feb19.1100D105.zip for my 1100d, but camera can't start with this version.

Maybe is related to memory constraint resource of 1100D.

I tried to merge latest changes from lua_fix to see if my previous reported issue could be solved with newer memory backend but camera won't show any signs apart the MPU handlings buttons and lens...

EDIT: I was using a workspace with CONFIG_QEMU defined from a previous work and didn't noticed, so this is why it wasn't running on camera.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Ron100 on April 07, 2018, 01:30:07 PM
is it possible to make the 6D think that the Canon EF 50 f1, 2 lens is installed when installing the manual lens? if Yes the tell me, please, as this do?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on April 10, 2018, 12:22:15 PM
I tested again this time with lua_fix.2018Mar31.50D and camera crashes also with this branch. So I think there is something wrong with memory management.

(https://thumb.ibb.co/g6q4Ex/VRAM2.jpg) (https://ibb.co/g6q4Ex)(https://thumb.ibb.co/f3L4Ex/VRAM3_1.jpg) (https://ibb.co/f3L4Ex)(https://thumb.ibb.co/nvSvMc/VRAM4.jpg) (https://ibb.co/nvSvMc)(https://thumb.ibb.co/doOrux/VRAM7.jpg) (https://ibb.co/doOrux)

How to reproduce:
1. enable a lot of modules and mlv_rec.mo with or without lua.mo enabled
2. start a raw recording
3. camera freezes with black screen (sometimes red led on) after stopping recording
if lua.mo is enabled recording it crash immediately. Also noticed some strange behaviour of the text next to the green camera icon, it is suppose to give you current recording time? or an estimated remaining time? Because it seems to print random numbers...

Using mlv_lite.mo instead  (with or without lua.mo loaded) it can record multiple clip in a row but debug menu will show a overflow error

@a1ex Can you take a look please? Any others have this issue with 50D or other camera?

EDIT: Merging lua_fix into manual_lens appears to solve camera crash. I need to retest this branch without the merge a d dig a little more
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 11, 2018, 09:12:01 PM
I like to try the latest experimental 4k branch Build (2018-07-09 21:47) with the lens info inside.
Do I need to compile from source, or would copying the scripts folder from the Non-CPU lens info to the 4k build be enough?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 12, 2018, 12:44:51 PM
Ok so I decided to compile a new build myself, but merging the manual_lens_info branch into crop_rec_4k_mlv_snd results in merge conflicts and I don't know yet how to fix them.  anyone who can help with this?



Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 12, 2018, 01:34:29 PM
I was thinking to do this on a temporary branch after pushing some little improvements, but I have no time to do this at the moment.

Some time ago a1ex posted a patch for solving merge conflicts and asked to test, but it wasn't recognised by patch tool the I tried.
Hopefully solved. Wasn't easy (tried the same approach when merging the first PR, but could not pass the tests from selftest.mo, so I ended up committing a workaround - that turned out not to be very good); however, today I've found the issue: yet another quirk in Canon's large memory allocators (the reason why it was crashing after ~1000 blocks or so (https://bitbucket.org/hudson/magic-lantern/commits/c1c17862229a788f58879bbca54f91b4c69dd683)). Seems to work in my quick tests, but I'd still like to sleep on it before including in lua_fix and crop_rec_4k.

I've extracted a patch for crop_rec_4k with only the changes related to ELNS and 64-byte lens name (mlv_dump, mlv*, silent and lens.h), but I'm afraid I have no more energy left to run any tests today (so I'm not committing it yet). If you try it, please report back.

Code: [Select]
# HG changeset patch
# User alex@thinkpad
# Date 1518995384 -3600
#      Mon Feb 19 00:09:44 2018 +0100
# Branch crop_rec_4k
# Node ID 804086b3e47771bd72ed123bdd797c608738bf4f
# Parent  5138c0050854c3c44eb2de21cf63c445baa0b245
* Experimental: 64Byte Extended Lens Name
* mlv_dump.c: Fix duplicate ELNS Block processing
* mlv_rec.c: Fix duplicate ELNS Block processing

diff -r 5138c0050854 -r 804086b3e477 modules/lua/lua_lens.c
--- a/modules/lua/lua_lens.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/lua/lua_lens.c Mon Feb 19 00:09:44 2018 +0100
@@ -72,7 +72,55 @@
 static int luaCB_lens_newindex(lua_State * L)
 {
     LUA_PARAM_STRING_OPTIONAL(key, 2, "");
-    if(!strcmp(key, "name") || !strcmp(key, "focal_length") || !strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af"))
+    if(!strcmp(key, "name"))
+    {
+        LUA_PARAM_STRING(value, 3);
+        strncpy(lens_info.name, value, sizeof(lens_info.name)-1);
+    }
+    else if(!strcmp(key, "exists"))
+    {
+        LUA_PARAM_BOOL(value, 3);
+        lens_info.lens_exists = value;
+    }
+    else if(!strcmp(key, "focal_length"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.focal_len = value;
+    }
+    else if(!strcmp(key, "focal_min"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_min = value;
+    }
+    else if(!strcmp(key, "focal_max"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_max = value;
+    }
+    else if(!strcmp(key, "manual_aperture"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        lens_info.aperture = (int)(value * 10);
+        lens_info.raw_aperture = VALUE2RAW(aperture, lens_info.aperture);
+    }
+    else if(!strcmp(key, "aperture_min"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_min = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "aperture_max"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_max = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "serial"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_serial = value;
+    }
+    else if(!strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af") || !strcmp(key, "is_chipped"))
     {
         return luaL_error(L, "'%s' is readonly!", key);
     }
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_lite/mlv_lite.c
--- a/modules/mlv_lite/mlv_lite.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_lite/mlv_lite.c Mon Feb 19 00:09:44 2018 +0100
@@ -350,7 +350,8 @@
 static GUARDED_BY(RawRecTask)   mlv_rawc_hdr_t rawc_hdr;
 static GUARDED_BY(RawRecTask)   mlv_idnt_hdr_t idnt_hdr;
 static GUARDED_BY(RawRecTask)   mlv_expo_hdr_t expo_hdr;
-static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
+static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
+static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t elns_hdr;
 static GUARDED_BY(RawRecTask)   mlv_rtci_hdr_t rtci_hdr;
 static GUARDED_BY(RawRecTask)   mlv_wbal_hdr_t wbal_hdr;
 static GUARDED_BY(LiveViewTask) mlv_vidf_hdr_t vidf_hdr;
@@ -2807,7 +2808,8 @@
 
     mlv_fill_idnt(&idnt_hdr, mlv_start_timestamp);
     mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
-    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
     mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
     mlv_fill_wbal(&wbal_hdr, mlv_start_timestamp);
 
@@ -2827,6 +2829,7 @@
     if (FIO_WriteFile(f, &idnt_hdr, idnt_hdr.blockSize) != (int)idnt_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) return 0;
+    if (FIO_WriteFile(f, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &wbal_hdr, wbal_hdr.blockSize) != (int)wbal_hdr.blockSize) return 0;
     if (mlv_write_vers_blocks(f, mlv_start_timestamp)) return 0;
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv.c
--- a/modules/mlv_rec/mlv.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv.c Mon Feb 19 00:09:44 2018 +0100
@@ -55,13 +55,35 @@
     hdr->autofocusMode = af_mode;
     hdr->flags = 0;
 
+    char name[33];
     char buf[33];
+    snprintf(name, 32, "%s", lens_info.name);
     snprintf(buf, sizeof(buf), "%X%08X", (uint32_t) (lens_info.lens_serial >> 32), (uint32_t)(lens_info.lens_serial & 0xFFFFFFFF));
-   
-    strncpy((char *)hdr->lensName, lens_info.name, 32);
+
+    strncpy((char *)hdr->lensName, name, 32);
     strncpy((char *)hdr->lensSerial, buf, 32);
 }
 
+void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp){
+    /* prepare header */
+    mlv_set_type((mlv_hdr_t *)hdr, "ELNS");
+    mlv_set_timestamp((mlv_hdr_t *)hdr, start_timestamp);
+    hdr->blockSize = sizeof(mlv_elns_hdr_t);
+
+    char bufName[65];
+    snprintf(bufName, 64, "%s", lens_info.name);
+
+    strncpy((char *)hdr->lensName, bufName, 64);
+    hdr->focalLengthMin = lens_info.lens_focal_min;
+    hdr->focalLengthMax = lens_info.lens_focal_max;
+    hdr->apertureMin = RAW2VALUE(aperture, lens_info.raw_aperture_min) / 10.0;
+    hdr->apertureMax = RAW2VALUE(aperture, lens_info.raw_aperture_max) / 10.0;
+    hdr->version = lens_info.lens_version;
+    hdr->extenderInfo = lens_info.lens_extender;
+    hdr->capabilities = lens_info.lens_capabilities;
+    hdr->chipped = lens_info.lens_exists;
+}
+
 void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp)
 {
     /* prepare header */
@@ -151,7 +173,7 @@
     mlv_set_type((mlv_hdr_t *)hdr, "IDNT");
     mlv_set_timestamp((mlv_hdr_t *)hdr, start_timestamp);
     hdr->blockSize = sizeof(mlv_idnt_hdr_t);
-   
+
     hdr->cameraModel = camera_model_id;
     memcpy(hdr->cameraName, camera_model, 32);
     memcpy(hdr->cameraSerial, camera_serial, 32);
@@ -163,16 +185,16 @@
 {
     int block_length = (strlen(version_string) + sizeof(mlv_vers_hdr_t) + 1 + 3) & ~3;
     mlv_vers_hdr_t *header = malloc(block_length);
-   
+
     /* prepare header */
     mlv_set_type((mlv_hdr_t *)header, "VERS");
     mlv_set_timestamp((mlv_hdr_t *)header, start_timestamp);
     header->blockSize = block_length;
     header->length = strlen(version_string);
-   
+
     char *vers_hdr_payload = (char *)&header[1];
     strcpy(vers_hdr_payload, version_string);
-   
+
     *hdr = header;
 }
 
@@ -239,12 +261,12 @@
 {
     int mod = -1;
     int error = 0;
-   
+
     do
     {
         /* get next loaded module id */
         mod = module_get_next_loaded(mod);
-       
+
         /* make sure thats a valid one */
         if(mod >= 0)
         {
@@ -252,7 +274,7 @@
             const char *mod_name = module_get_name(mod);
             const char *mod_build_date = module_get_string(mod, "Build date");
             const char *mod_last_update = module_get_string(mod, "Last update");
-           
+
             if(mod_name != NULL)
             {
                 /* just in case that ever happens */
@@ -264,15 +286,15 @@
                 {
                     mod_last_update = "(no version)";
                 }
-               
+
                 /* separating the format string allows us to measure its length for malloc */
                 const char *fmt_string = "%s built %s; commit %s";
                 int buf_length = strlen(fmt_string) + strlen(mod_name) + strlen(mod_build_date) + strlen(mod_last_update) + 1;
                 char *version_string = malloc(buf_length);
-               
+
                 /* now build the string */
                 snprintf(version_string, buf_length, fmt_string, mod_name, mod_build_date, mod_last_update);
-               
+
                 /* and finally remove any newlines, they are annoying */
                 for(unsigned int pos = 0; pos < strlen(version_string); pos++)
                 {
@@ -281,23 +303,23 @@
                         version_string[pos] = ' ';
                     }
                 }
-               
+
                 /* let the mlv helpers build the block for us */
                 mlv_vers_hdr_t *hdr = NULL;
                 mlv_build_vers(&hdr, mlv_start_timestamp, version_string);
-               
+
                 /* try to write to output file */
                 if(FIO_WriteFile(f, hdr, hdr->blockSize) != (int)hdr->blockSize)
                 {
                     error = 1;
                 }
-               
+
                 /* free both temporary string and allocated mlv block */
                 free(version_string);
                 free(hdr);
             }
         }
     } while(mod >= 0 && !error);
-   
+
     return error;
 }
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv.h
--- a/modules/mlv_rec/mlv.h Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv.h Mon Feb 19 00:09:44 2018 +0100
@@ -165,6 +165,22 @@
 }  mlv_lens_hdr_t;
 
 typedef struct {
+    uint8_t     blockType[4];    /* ELNS - Extended LENS block with longer lens name and optional fields, depending on camera */
+    uint32_t    blockSize;
+    uint64_t    timestamp;
+    uint16_t    focalLengthMin;  /* shortest focal length in mm                       */
+    uint16_t    focalLengthMax;  /* longest focal length in mm                        */
+    uint16_t    apertureMin;     /* lowest f-number * 100                             */
+    uint16_t    apertureMax;     /* highest f-number * 100                            */
+    uint32_t    version;         /* lens internal version number, if available        */
+    uint8_t     extenderInfo;    /* extender information, if provided by camera       */
+    uint8_t     capabilities;    /* capability information, if provided by camera     */
+    uint8_t     chipped;         /* when not zero, lens is communicating with camera  */
+    uint8_t     lensName[64];     /* full lens string, null terminated                 */
+    // TODO: Review Specs
+}  mlv_elns_hdr_t;
+
+typedef struct {
     uint8_t     blockType[4];       /* RTCI: real-time clock metadata */
     uint32_t    blockSize;          /* total frame size */
     uint64_t    timestamp;          /* hardware counter timestamp for this frame (relative to recording start) */
@@ -287,6 +303,7 @@
 void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
+void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv_dump.c Mon Feb 19 00:09:44 2018 +0100
@@ -1894,6 +1894,7 @@
     lv_rec_file_footer_t lv_rec_footer;
     mlv_file_hdr_t main_header;
     mlv_lens_hdr_t lens_info;
+    mlv_elns_hdr_t elns_info;
     mlv_expo_hdr_t expo_info;
     mlv_idnt_hdr_t idnt_info;
     mlv_wbal_hdr_t wbal_info;
@@ -1904,6 +1905,7 @@
     /* initialize stuff */
     memset(&lv_rec_footer, 0x00, sizeof(lv_rec_file_footer_t));
     memset(&lens_info, 0x00, sizeof(mlv_lens_hdr_t));
+    memset(&elns_info, 0x00, sizeof(mlv_elns_hdr_t));
     memset(&expo_info, 0x00, sizeof(mlv_expo_hdr_t));
     memset(&idnt_info, 0x00, sizeof(mlv_idnt_hdr_t));
     memset(&wbal_info, 0x00, sizeof(mlv_wbal_hdr_t));
@@ -3652,6 +3654,46 @@
                     print_msg(MSG_INFO, "     Flags:       0x%08X\n", lens_info.flags);
                 }
             }
+            else if(!memcmp(buf.blockType, "ELNS", 4))
+            {
+                uint32_t hdr_size = MIN(sizeof(mlv_elns_hdr_t), buf.blockSize);
+
+                if(fread(&elns_info, hdr_size, 1, in_file) != 1)
+                {
+                    print_msg(MSG_ERROR, "File ends in the middle of a block\n");
+                    goto abort;
+                  }
+
+                /* skip remaining data, if there is any */
+                file_set_pos(in_file, position + elns_info.blockSize, SEEK_SET);
+
+                lua_handle_hdr(lua_state, buf.blockType, &elns_info, sizeof(elns_info));
+
+                if(verbose)
+                {
+                    //TODO: Check Aperture Min/Max and if other fields are needed
+                    print_msg(MSG_INFO, "     Name:                '%s'\n", elns_info.lensName);
+                    print_msg(MSG_INFO, "     Focal Length Min:    %d mm\n", elns_info.focalLengthMin);
+                    print_msg(MSG_INFO, "     Focal Length Max:    %d mm\n", elns_info.focalLengthMax);
+                    print_msg(MSG_INFO, "     Aperture Min:        f/%.2f\n", (double)elns_info.apertureMin);
+                    print_msg(MSG_INFO, "     Aperture Max:        f/%.2f\n", (double)elns_info.apertureMax);
+                    print_msg(MSG_INFO, "     Version:             %d\n", elns_info.version);
+                    print_msg(MSG_INFO, "     Extender Info:       0x%02X\n", elns_info.extenderInfo);
+                    print_msg(MSG_INFO, "     Capabilities:        0x%02X\n", elns_info.capabilities);
+                    print_msg(MSG_INFO, "     Chipped:             0x%02X\n", elns_info.chipped);
+                }
+
+                if(mlv_output && !no_metadata_mode && (!extract_block || !strncasecmp(extract_block, (char*)elns_info.blockType, 4)))
+                {
+                    /* correct header size if needed */
+                    elns_info.blockSize = sizeof(mlv_elns_hdr_t);
+                    if(fwrite(&elns_info, elns_info.blockSize, 1, out_file) != 1)
+                    {
+                        print_msg(MSG_ERROR, "Failed writing into .MLV file\n");
+                        goto abort;
+                    }
+                }
+            }
             else if(!memcmp(mlv_block->blockType, "INFO", 4))
             {
                 mlv_info_hdr_t block_hdr = *(mlv_info_hdr_t *)mlv_block;
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv_rec.c
--- a/modules/mlv_rec/mlv_rec.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv_rec.c Mon Feb 19 00:09:44 2018 +0100
@@ -195,6 +195,7 @@
 
 static mlv_expo_hdr_t last_expo_hdr;
 static mlv_lens_hdr_t last_lens_hdr;
+static mlv_elns_hdr_t last_elns_hdr;
 static mlv_wbal_hdr_t last_wbal_hdr;
 static mlv_styl_hdr_t last_styl_hdr;
 
@@ -1449,14 +1450,17 @@
             trace_write(raw_rec_trace_ctx, "[polling_cbr] queueing INFO blocks");
             mlv_expo_hdr_t *expo_hdr = malloc(sizeof(mlv_expo_hdr_t));
             mlv_lens_hdr_t *lens_hdr = malloc(sizeof(mlv_lens_hdr_t));
+            mlv_elns_hdr_t *elns_hdr = malloc(sizeof(mlv_elns_hdr_t));
             mlv_wbal_hdr_t *wbal_hdr = malloc(sizeof(mlv_wbal_hdr_t));
 
             mlv_fill_expo(expo_hdr, mlv_start_timestamp);
             mlv_fill_lens(lens_hdr, mlv_start_timestamp);
+            mlv_fill_elns(elns_hdr, mlv_start_timestamp);
             mlv_fill_wbal(wbal_hdr, mlv_start_timestamp);
 
             msg_queue_post(mlv_block_queue, (uint32_t) expo_hdr);
             msg_queue_post(mlv_block_queue, (uint32_t) lens_hdr);
+            msg_queue_post(mlv_block_queue, (uint32_t) elns_hdr);
             msg_queue_post(mlv_block_queue, (uint32_t) wbal_hdr);
         }
 
@@ -2567,6 +2571,7 @@
         mlv_rtci_hdr_t rtci_hdr;
         mlv_expo_hdr_t expo_hdr;
         mlv_lens_hdr_t lens_hdr;
+        mlv_elns_hdr_t elns_hdr;
         mlv_idnt_hdr_t idnt_hdr;
         mlv_wbal_hdr_t wbal_hdr;
         mlv_styl_hdr_t styl_hdr;
@@ -2574,6 +2579,7 @@
         mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
         mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
         mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+        mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
         mlv_fill_idnt(&idnt_hdr, mlv_start_timestamp);
         mlv_fill_wbal(&wbal_hdr, mlv_start_timestamp);
         mlv_fill_styl(&styl_hdr, mlv_start_timestamp);
@@ -2585,10 +2591,12 @@
         idnt_hdr.timestamp = 4;
         wbal_hdr.timestamp = 5;
         styl_hdr.timestamp = 6;
+        elns_hdr.timestamp = 7;
 
         mlv_write_hdr(f, (mlv_hdr_t *)&rtci_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&expo_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&lens_hdr);
+        mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&idnt_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&wbal_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&styl_hdr);
@@ -3100,13 +3108,16 @@
 
         mlv_expo_hdr_t old_expo = last_expo_hdr;
         mlv_lens_hdr_t old_lens = last_lens_hdr;
+        mlv_elns_hdr_t old_elns = last_elns_hdr;
 
         mlv_fill_expo(&last_expo_hdr, mlv_start_timestamp);
         mlv_fill_lens(&last_lens_hdr, mlv_start_timestamp);
+        mlv_fill_elns(&last_elns_hdr, mlv_start_timestamp);
 
         /* update timestamp for comparing content changes */
         old_expo.timestamp = last_expo_hdr.timestamp;
         old_lens.timestamp = last_lens_hdr.timestamp;
+        old_elns.timestamp = last_elns_hdr.timestamp;
 
         /* write new state if something changed */
         if(memcmp(&last_expo_hdr, &old_expo, sizeof(mlv_expo_hdr_t)))
@@ -3123,6 +3134,13 @@
             memcpy(hdr, &last_lens_hdr, sizeof(mlv_lens_hdr_t));
             msg_queue_post(mlv_block_queue, (uint32_t) hdr);
         }
+        /* write new state if something changed */
+        if(memcmp(&last_elns_hdr, &old_elns, sizeof(mlv_elns_hdr_t)))
+        {
+            mlv_hdr_t *hdr = malloc(sizeof(mlv_elns_hdr_t));
+            memcpy(hdr, &last_elns_hdr, sizeof(mlv_elns_hdr_t));
+            msg_queue_post(mlv_block_queue, (uint32_t) hdr);
+        }
     }
 
     if(mlv_update_styl && (mlv_metadata & MLV_METADATA_SPORADIC))
diff -r 5138c0050854 -r 804086b3e477 modules/silent/silent.c
--- a/modules/silent/silent.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/silent/silent.c Mon Feb 19 00:09:44 2018 +0100
@@ -28,6 +28,7 @@
 extern WEAK_FUNC(ret_0) void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
+extern WEAK_FUNC(ret_0) void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
@@ -365,6 +366,7 @@
     mlv_rtci_hdr_t rtci_hdr;
     mlv_expo_hdr_t expo_hdr;
     mlv_lens_hdr_t lens_hdr;
+    mlv_elns_hdr_t elns_hdr;
     mlv_idnt_hdr_t idnt_hdr;
     mlv_wbal_hdr_t wbal_hdr;
     mlv_styl_hdr_t styl_hdr;
@@ -478,6 +480,7 @@
     mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
     mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
     mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
 
     expo_hdr.isoValue = metadata.iso;
     expo_hdr.shutterValue = 1000000000 / metadata.tvr;
@@ -486,7 +489,8 @@
     if (FIO_WriteFile(save_file, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) goto write_error;
-   
+    if (FIO_WriteFile(save_file, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) goto write_error;
+
     memset(&vidf_hdr, 0, sizeof(mlv_vidf_hdr_t));
     mlv_set_type((mlv_hdr_t *)&vidf_hdr, "VIDF");
     mlv_set_timestamp((mlv_hdr_t *)&vidf_hdr, mlv_start_timestamp);
diff -r 5138c0050854 -r 804086b3e477 src/lens.h
--- a/src/lens.h Sat Feb 17 22:37:51 2018 +0100
+++ b/src/lens.h Mon Feb 19 00:09:44 2018 +0100
@@ -37,7 +37,7 @@
 struct lens_info
 {
         void *                  token;
-        char                    name[ 32 ];
+        char                    name[ 64 ];
         unsigned                focal_len; // in mm
         unsigned                focus_dist; // in cm
         unsigned                IS; // PROP_LV_LENS_STABILIZE
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 12, 2018, 03:26:35 PM
Would be great if you can do this temporary branch, no need to be right this moment, just when you manage to have the time  :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 12, 2018, 06:10:45 PM
Tried applying that patch but it didn't apply cleanly for me so I had to do some manual fixing. Ran out of time to do any significant testing but you can pull it from my ML fork, crop_rec_4k_ELNS branch (https://bitbucket.org/daniel_fort/magic-lantern/src/crop_rec_4k_ELNS/).

Made a fake pull request to check what has changed:

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/22/elns-for-crop_rec_4k/diff

There's some experimental tuff in the current crop_rec_4k branch, especially for the 700D and EOSM, so we should probably wait until that is sorted out before adding ELNS.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 12, 2018, 09:09:47 PM
It should be ok, but it needs "script/lib" folder from manual_lens in order to run lens.lua

Lars, do you need also changes from crop_rec_4k_mlv_snd or building on top of crop_rec_4k is ok for now?

At the moment the experimental manual_lens branch was tested also by a1ex and dfort.
There could be some issues when recording raw video, probably inherited from lua_fix and I would like to know if are present also on top of crop_rec_4k.

Also I would like to know how it behave with different adapter from the cheap one I have, so further testing and feedback from other users is really appreciated :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 12, 2018, 09:50:42 PM
Im trying a make a bleeding edge build with mlv compressed + snd.

I did make a build from dforts branch just now and will test it out in a bit.

Im using a 5dmk3 and would like to use compressed raw + snd with lens info to save space.

I normally use an older build with lens info,  but no sound with compressed.
I will share what scripts i use a bit later.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 12, 2018, 09:54:37 PM
It should be ok, but it needs "script/lib" folder from manual_lens in order to run lens.lua

Thanks. Added that but it isn't working yet. Need to dig in a little deeper when I've got some time.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 12, 2018, 10:02:47 PM
Thanks. Added that but it isn't working yet. Need to dig in a little deeper when I've got some time.

Checked again the branch and noticed some missing diff in lua which are responsible to update lens_info structure
Copy these files from manual_lens_info:
- modules/lua/lua_lens.c
- modules/lua/lua_menu.c
- modules/lua/lua_property.c

Refer to this (https://bitbucket.org/hudson/magic-lantern/pull-requests/904/manual-lens-info-pt2/diff) diff and this (https://bitbucket.org/hudson/magic-lantern/pull-requests/887/manual-lens-info/diff) to see which files has been modified and also remember of a1ex's commits (https://bitbucket.org/hudson/magic-lantern/pull-requests/822/manual-lens-info-lua/commits) regarding memory backend (crop_rec_4k and lua_fix appears to have the same commits for mem.c and self_test.c, so you can see diff directly from manual_lens_info pull request)

My testing is limited because 50D won't compile in crop_rec_4k and Qemu simulates a chipped lens attached
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 13, 2018, 12:57:04 AM
Ha ha --

Quote from: g3gg0
the only thing that is bugging me - it will cause conflicts when merging the crop_rec branch.

Yeah, this is getting complicated.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 13, 2018, 01:53:10 AM
Dfort, while I was trying to compile from crop_rec_4k, I noticed that lua module wasn't going fine due to a missing reference to "Value2Raw", this isn't also into a1ex's patch

I saw your latest commit and it's missing some diff; let's add the patch below.
Code: [Select]
diff --git a/modules/lua/lua_lens.c b/modules/lua/lua_lens.c
--- a/modules/lua/lua_lens.c
+++ b/modules/lua/lua_lens.c
@@ -17,12 +17,24 @@
 static int luaCB_lens_index(lua_State * L)
 {
     LUA_PARAM_STRING_OPTIONAL(key, 2, "");
-    /// Get the name of the lens (reported by the lens).
-    // @tfield string name readonly
+    /// Get/Set the name of the lens (reported by the lens)
+    // @tfield string name
     if(!strcmp(key, "name")) lua_pushstring(L, lens_info.name);
-    /// Get the focal length of the lens (in mm). Only updated in LiveView.
-    // @tfield int focal_length readonly
+    /// Get the lens id
+    // @tfield int id readonly
+    else if(!strcmp(key, "id")) lua_pushinteger(L, lens_info.lens_id);
+    /// Get the serial number of the lens
+    // @tfield int serial
+    else if(!strcmp(key, "serial")) lua_pushinteger(L, lens_info.lens_serial);
+    /// Get/Set the focal length of the lens (in mm)
+    // @tfield int focal_length
     else if(!strcmp(key, "focal_length")) lua_pushinteger(L, lens_info.focal_len);
+    /// Get/Set the minimum focal length of the lens (in mm)
+    // @tfield int focal_min
+    else if(!strcmp(key, "focal_min")) lua_pushinteger(L, lens_info.lens_focal_min);
+    /// Get/Set the maximum focal length of the lens (in mm)
+    // @tfield int focal_max
+    else if(!strcmp(key, "focal_max")) lua_pushinteger(L, lens_info.lens_focal_max);
     /// Get the current focus distance (in mm). Only updated in LiveView.
     // @tfield int focus_distance readonly
     else if(!strcmp(key, "focus_distance")) lua_pushinteger(L, lens_info.focus_dist * 10);
@@ -84,7 +96,55 @@
 static int luaCB_lens_newindex(lua_State * L)
 {
     LUA_PARAM_STRING_OPTIONAL(key, 2, "");
-    if(!strcmp(key, "name") || !strcmp(key, "focal_length") || !strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af") || !strcmp(key, "is_chipped"))
+    if(!strcmp(key, "name"))
+    {
+        LUA_PARAM_STRING(value, 3);
+        strncpy(lens_info.name, value, sizeof(lens_info.name)-1);
+    }
+    else if(!strcmp(key, "exists"))
+    {
+        LUA_PARAM_BOOL(value, 3);
+        lens_info.lens_exists = value;
+    }
+    else if(!strcmp(key, "focal_length"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.focal_len = value;
+    }
+    else if(!strcmp(key, "focal_min"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_min = value;
+    }
+    else if(!strcmp(key, "focal_max"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_max = value;
+    }
+    else if(!strcmp(key, "manual_aperture"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        lens_info.aperture = (int)(value * 10);
+        lens_info.raw_aperture = VALUE2RAW(aperture, lens_info.aperture);
+    }
+    else if(!strcmp(key, "aperture_min"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_min = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "aperture_max"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_max = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "serial"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_serial = value;
+    }
+    else if(!strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af") || !strcmp(key, "is_chipped"))
     {
         return luaL_error(L, "'%s' is readonly!", key);
     }
diff --git a/modules/lua/lua_property.c b/modules/lua/lua_property.c
--- a/modules/lua/lua_property.c
+++ b/modules/lua/lua_property.c
@@ -21,8 +21,6 @@
 // !!!DANGER WILL ROBINSON!!!
 //#define LUA_PROP_REQUEST_CHANGE
 
-struct msg_queue * lua_prop_queue = NULL;
-
 struct lua_prop
 {
     struct lua_prop * next;
@@ -32,100 +30,69 @@
     int prop_handler_ref;
 };
 
-struct lua_prop_msg
-{
-    unsigned property;
-    void * value;
-    unsigned len;
-};
-
-int lua_prop_task_running = 0;
 static struct lua_prop * prop_handlers = NULL;
 
-//TODO: create a new task per script so that scripts don't block each other's prop handlers, necessary?
-static void lua_prop_task(int unused)
+static void lua_prophandler(unsigned property, void * priv, void * buf, unsigned len)
 {
-    lua_prop_queue = msg_queue_create("lua_prop_queue", 1);
-    TASK_LOOP
+    struct lua_prop * lua_prop = 0;
+    int found = 0;
+    for (lua_prop = prop_handlers; lua_prop; lua_prop = lua_prop->next)
     {
-        struct lua_prop_msg * msg = NULL;
-        int err = msg_queue_receive(lua_prop_queue, &msg, 0);
-       
-        if(err || !msg) continue;
-       
-        struct lua_prop * lua_prop = NULL;
-        int found = 0;
-        for(lua_prop = prop_handlers; lua_prop; lua_prop = lua_prop->next)
+        if (lua_prop->prop_id == property)
         {
-            if(lua_prop->prop_id == msg->property)
+            found = 1;
+            break;
+        }
+    }
+   
+    if (found)
+    {
+        lua_State * L = lua_prop->L;
+        struct semaphore * sem = NULL;
+        if (lua_take_semaphore(L, 100, &sem) == 0)
+        {
+            ASSERT(sem);
+            if (lua_rawgeti(L, LUA_REGISTRYINDEX, lua_prop->prop_handler_ref) == LUA_TFUNCTION)
             {
-                found = 1;
-                break;
+                lua_rawgeti(L, LUA_REGISTRYINDEX, lua_prop->self_ref);
+                if (len > 4)
+                {
+                    /* fixme: pass proper data types (requires knowledge about each property) */
+                    //long, probably a string
+                    //((char*)(msg->value))[msg->len] = 0x0;
+                    lua_pushstring(L, (char*)buf);
+                }
+                else
+                {
+                    lua_pushinteger(L, *(uint32_t*)buf & (0xFFFFFFFF >> ((4-len) * 8)) );
+                }
+
+                int t0 = get_us_clock();
+
+                if (docall(L, 2, 0))
+                {
+                    fprintf(stderr, "[Lua] prop handler failed:\n %s\n", lua_tostring(L, -1));
+                    lua_save_last_error(L);
+                }
+
+                int t1 = get_us_clock();
+
+                if (t1 - t0 > 10000)
+                {
+                    printf("[%s] slow property handler %X (%d us)\n", lua_get_script_filename(L), lua_prop->prop_id, t1 - t0);
+                }
             }
+            give_semaphore(sem);
         }
-       
-        if(found && msg->value)
+        else
         {
-            lua_State * L = lua_prop->L;
-            struct semaphore * sem = NULL;
-            if (lua_take_semaphore(L, 1000, &sem) == 0)
-            {
-                ASSERT(sem);
-                if(lua_rawgeti(L, LUA_REGISTRYINDEX, lua_prop->prop_handler_ref) == LUA_TFUNCTION)
-                {
-                    lua_rawgeti(L, LUA_REGISTRYINDEX, lua_prop->self_ref);
-                    if(msg->len > 4)
-                    {
-                        //long, probably a string
-                        ((char*)(msg->value))[msg->len] = 0x0;
-                        lua_pushstring(L, (char*)(msg->value));
-                    }
-                    else if(msg->len == 4) lua_pushinteger(L, *((uint32_t*)(msg->value)));
-                    else if(msg->len >= 2) lua_pushinteger(L, *((uint16_t*)(msg->value)));
-                    else lua_pushinteger(L, *((uint8_t*)(msg->value)));
-                    if(docall(L, 2, 0))
-                    {
-                        fprintf(stderr, "[Lua] prop handler failed:\n %s\n", lua_tostring(L, -1));
-                        lua_save_last_error(L);
-                    }
-                }
-                give_semaphore(sem);
-            }
-            else
-            {
-                printf("[Lua] semaphore timeout: prop handler %d (%dms)\n", lua_prop->prop_id, 1000);
-            }
+            printf("[%s] semaphore timeout: prop handler %X (%dms)\n", lua_get_script_filename(L), lua_prop->prop_id, 100);
         }
-        free(msg->value);
-        free(msg);
-    }
-}
-
-static void lua_prophandler(unsigned property, void * priv, void * addr, unsigned len)
-{
-    if (!lua_prop_queue) return;
-    struct lua_prop_msg * msg = malloc(sizeof(struct lua_prop_msg));
-    msg->property = property;
-    msg->len = MIN(len,255);
-    msg->value = malloc(msg->len + 1);
-    if(msg->value)
-    {
-        memcpy(msg->value, addr, MIN(len,255));
-        msg_queue_post(lua_prop_queue, (uint32_t)msg);
-    }
-    else
-    {
-        fprintf(stderr, "[Lua] lua_prophandler: malloc error");
     }
 }
 
 static void lua_register_prop_handler(unsigned prop_id)
 {
-    if(!lua_prop_task_running)
-    {
-        lua_prop_task_running = 1;
-        task_create("lua_prop_task", 0x1c, 0x10000, lua_prop_task, 0);
-    }
     //check for existing prop handler
     struct lua_prop * current;
     for(current = prop_handlers; current; current = current->next)
diff --git a/src/lens.c b/src/lens.c
--- a/src/lens.c
+++ b/src/lens.c
@@ -1941,7 +1941,11 @@
         factor = 1/2.54;
     }
     
-    if(lens_info.lens_focal_min == lens_info.lens_focal_max)
+    if(!lens_info.lens_focal_min || !lens_info.lens_focal_max)
+    {
+        MENU_SET_VALUE("%d %s", (int)(lens_info.focal_len * factor), unit);
+    }
+    else if(lens_info.lens_focal_min == lens_info.lens_focal_max)
     {
         MENU_SET_VALUE("%d %s", (int)(lens_info.lens_focal_min * factor), unit);
     }
@@ -1955,6 +1959,7 @@
    {
         .name = "Lens Info Prefs",
         .select   = menu_open_submenu,
+ .submenu_width = 700,
         .children =  (struct menu_entry[]) {
             #ifndef CONFIG_FULLFRAME
             {
diff --git a/src/lens.h b/src/lens.h
--- a/src/lens.h
+++ b/src/lens.h
@@ -307,7 +307,12 @@
 //~ static const int codes_aperture[] =  {0,13,14,16,19,21,24,27,29,32,35,37,40,44,45,48,51,52,53,56,59,60, 61, 64, 68, 69, 72, 75, 76, 77, 80, 83, 84, 85, 88, 91, 92, 93, 96};
 
 #define RAW2VALUE(param,rawvalue) ((int)values_##param[raw2index_##param(rawvalue)])
-#define VALUE2RAW(param,value) ((int)val2raw_##param(value))
+#define VALUE2RAW(param,value) val2raw_##param(value)
+
+/* prototypes for VALUE2RAW */
+int VALUE2RAW(iso, int);
+int VALUE2RAW(shutter, int);
+int VALUE2RAW(aperture, int);
 
 // UNIT_1_8_EV
 #define APEX_TV(raw) ((int)(raw) - 56)
diff --git a/src/mem.c b/src/mem.c
--- a/src/mem.c
+++ b/src/mem.c
@@ -582,7 +582,8 @@
 
     memcheck_add(ptr, file, line);
     
-    /* keep track of allocated memory and update history */
+    /* keep track of allocated memory and update history */
+    unsigned int state = cli();
     allocators[allocator_index].num_blocks++;
     allocators[allocator_index].mem_used += len + 2 * MEM_SEC_ZONE;
     alloc_total += len;
@@ -590,7 +591,8 @@
     alloc_total_peak_with_memcheck = MAX(alloc_total_peak_with_memcheck, alloc_total_with_memcheck);
     history[history_index] = MIN(alloc_total_with_memcheck / 1024, USHRT_MAX);
     history_index = MOD(history_index + 1, HISTORY_ENTRIES);
-   
+    sei(state);
+
     return (void*)(ptr + MEM_SEC_ZONE);
 }
 
@@ -784,12 +786,64 @@
     /* if we arrive here, you should probably solder some memory chips on the mainboard */
     return -1;
 }
+
+static void * check_and_adjust_ptr(void * ptr, size_t size, unsigned int flags, const char* file, unsigned int line, int allocator_index)
+{
+    if (!ptr)
+    {
+        /* didn't work? */
+        snprintf(last_error_msg_short, sizeof(last_error_msg_short), "%s(%s,%x)", allocators[allocator_index].name, format_memory_size_and_flags(size, flags));
+        snprintf(last_error_msg, sizeof(last_error_msg), "%s(%s) failed at %s:%d, %s.", allocators[allocator_index].name, format_memory_size_and_flags(size, flags), file, line, get_current_task_name());
+        dbg_printf("alloc fail, took %s%d.%03d s\n", FMT_FIXEDPOINT3(t1-t0));
+    }
+    else
+    {
+        /* force the cacheable pointer to be the way user requested it */
+        /* note: internally, this library must use the vanilla pointer (non-mangled) */
+        ptr = (flags & MEM_DMA) ? UNCACHEABLE(ptr) : CACHEABLE(ptr);
+
+        dbg_printf("alloc ok, took %s%d.%03d s\n", FMT_FIXEDPOINT3(t1-t0));
+    }
+
+    return ptr;
+}
+
+/* used for property handler / vsync hooks / GUI handlers / other tricky places */
+/* not to be called directly, but exported to modules for test purposes (seltest.mo) */
+void * __fast_malloc(size_t size, unsigned int flags, const char* file, unsigned int line)
+{
+    /* for small sizes only */
+    ASSERT(size < 65536);
+
+    /* Canon code uses AllocateMemory for these, so... let's try */
+    int allocator_index = 1;
+
+    /* is this thread-safe? */
+    void * ptr = memcheck_malloc(size, file, line, allocator_index, flags);
+
+    return check_and_adjust_ptr(ptr, size, flags, file, line, allocator_index);
+}
 
 /* these two will replace all malloc calls */
 
 /* returns 0 if it couldn't allocate */
 void* __mem_malloc(size_t size, unsigned int flags, const char* file, unsigned int line)
-{
+{
+    /* running from PROP_HANDLER's will cause trouble
+     * slow allocators (in particular, srm_malloc) will keep the semaphore busy
+     * this would delay Canon's property handlers, possibly overflowing the queue
+     * similar issues may appear with GUI handlers, LiveView "vsync" hooks etc
+     * todo: generic way to figure out whether we are running from such hooks or from regular tasks
+     */
+    const char * current_task_name = get_current_task_name();
+    if (streq(current_task_name, "PropMgr") ||
+        streq(current_task_name, "GuiMainTask") ||
+        streq(current_task_name, "Evf"))
+    {
+        dbg_printf("fast_malloc(%s) from %s:%d task %s\n", format_memory_size_and_flags(size, flags), file, line, get_current_task_name());
+        return __fast_malloc(size, flags, file, line);
+    }
+
     ASSERT(mem_sem);
     take_semaphore(mem_sem, 0);


At this point we should have a changeset which extends lens name as a1ex's patch and a possible lua implementation, but I don't expect to run lens.lua script fine because It misses a lot of old commits from manual_lens_info, which I don't know exactly how to manage as my work is based on top of old version of lens.lua

Anyway it compiles for 5D3 but I can't test it. Can you try it?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 13, 2018, 07:01:37 AM
The patch didn't apply cleanly on mem.c so I had to do some manual labor but it is working!

Of course it could use some more testing but the manual lens lua script menu does come up if you start the camera without a lens attached.

Turned into a monster changeset.

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/22/elns-for-crop_rec_4k/diff
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 13, 2018, 11:32:14 AM
Great work!
I compiled your branch dfort and when I set the script to run at startup it shows the script but after it disappears.
( before I can select something )

See this video:

This is on 5dmk3 with 1.2.3. firmware.

I always had this problem with the lens info script.
so what I did previously is make a separate script for every lens I have that does not require user input and set that to auto run.
The nice thing about this was that I could just set the lens one time and on each startup it remembers my setting.

for example I had this one for a 25 mm Zeiss. and called the script ZEISS 25 f2.lua
[/s]

Code: [Select]
-- ZEISS 25 f2
-- Prints lens_info.name on the console

menu.close()
console.show()
if lens.name == "" then
    print("non-cpu lens")
    lens.name = "Zeiss Distagon T* 2/25 ZF.2"
    lens.focal_length = 25
    lens.aperture = 20
end
print("lens: "..lens.name)
print("focal_length: "..lens.focal_length)
print("aperture: "..lens.aperture/10)
-- print "Press any key to exit."
-- key.wait()
console.hide()

I need to look into making a minimal script again with the new options for each lens I have.

The above is no longer needed, It was user error, I assumed I needed to load the script only, but I see the menu where I can change the lens info now, and it works!
Thanks!

And finally if you can try to merge this branch with the crop_rec_4k_mlv_snd?, I did that and only mlv.lite has some merge conflict.

My goal is to have sound with mlv compressed. Thank you for this monster merge :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 13, 2018, 12:56:21 PM
After testing I see one strange thing. 
I did enable raw video ( mlv rec )
and crop rec 1:1 1080

and then when I restart the cam while still in crop rec, I see vertical lines in black and the live view does not show up.
live view works again when I toggle the crop rec off. or when I press canon menu and go back to live view again
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 13, 2018, 12:57:20 PM
I compiled your branch dfort and when I set the script to run at startup it shows the script but after it disappears.
( before I can select something )

This should be caused from memory malloc as you can see from comments inside mem.c. Maybe a review could solve this issue.
This modification to memory management was needed because an issues which was crashing the camera was observed, It was managed by a1ex as can be found into comments (https://bitbucket.org/hudson/magic-lantern/pull-requests/887/manual-lens-info/diff#comment-54485854) of the merged pull request and solved temporary with a tmp patch (https://bitbucket.org/hudson/magic-lantern/pull-requests/887/manual-lens-info/diff#comment-54555703) which wasn't committed, but revised later with a new commit (https://bitbucket.org/hudson/magic-lantern/commits/065ceae1247d909e9952a2bc9eb35f3924267719?at=lua_fix#chg-src/mem.c) after merging PR

It doesn't happens on my camera as I can wait as long as I want into lens selection menu.
I used the script last month for a shooting session and if I remember correctly I could see some warning in console regarding timeout during the shoot but the script was running fine.

Can you select a lens from the menu if you quickly press the set button? Also it's strange that your choice is preserved with your script after a camera shutdown, autoload functionality was added to manage this
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 13, 2018, 01:05:40 PM
Just to be clear I can use the lens selection menu when I go to the menu: lens info prefs.
I enabled auto load lens : on
and it works fine.

It loads the scripts at startup and the lens I selected before is loaded every time I start the camera.
before I thought I needed to interact with the script at startup but this is not needed as I can just set it in the menu.

and to answer you question, no I can't select at startup when I press the SET Botton.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 13, 2018, 01:36:08 PM
Now when I try to open one of the mlv's in MLVapp this error comes up.
Probably normal, as they app need to add support for the new lens block.


(https://thumb.ibb.co/kEPT7T/Screen_Shot_2018_07_13_at_13_28_44.png) (https://ibb.co/kEPT7T)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 13, 2018, 01:48:48 PM
Tryi with MLVFS or incamera with mlv_play.

You can check metadata with mlv_dump if you compile from this branch or from manual_lens_info
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 13, 2018, 01:53:47 PM
Just tested with MlRawViewer 1.5 and it shows the meta data of the lens.
Good results!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 14, 2018, 12:49:15 AM
And finally if you can try to merge this branch with the crop_rec_4k_mlv_snd?, I did that and only mlv.lite has some merge conflict.

My goal is to have sound with mlv compressed. Thank you for this monster merge :)

This seems to be easy to fix. It's because of a refactor from the branch you was merging

You can merge crop_rec_4k_mlv_snd, resolve merge conflicts using the new code and place this line in-between the block that was causing the conflict (for example at line number 3848):
Code: [Select]
fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
But this method it's not recommended at this stage, because merging will also remove some lines used to check and process "elns" block...
Code: [Select]
static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t elns_hdr;
mlv_fill_elns(&elns_hdr, mlv_start_timestamp);

Use instead this patch I made from current status of dfort's branch:
Code: [Select]
diff --git a/modules/dual_iso/dual_iso.c b/modules/dual_iso/dual_iso.c
--- a/modules/dual_iso/dual_iso.c
+++ b/modules/dual_iso/dual_iso.c
@@ -69,6 +69,8 @@
 #include <fileprefix.h>
 #include <raw.h>
 #include <patch.h>
+#include "../mlv_rec/mlv.h"
+#include "../mlv_rec/mlv_rec_interface.h"
 
 static CONFIG_INT("isoless.hdr", isoless_hdr, 0);
 static CONFIG_INT("isoless.iso", isoless_recovery_iso, 3);
@@ -667,6 +669,24 @@
     },
 };
 
+/* callback routine for mlv_rec to add a custom DISO block after recording started (which already was specified in mlv.h in definition phase) */
+static void isoless_mlv_rec_cbr (uint32_t event, void *ctx, mlv_hdr_t *hdr)
+{
+    /* construct a free-able pointer to later pass it to mlv_rec_queue_block */
+    mlv_diso_hdr_t *dual_iso_block = malloc(sizeof(mlv_diso_hdr_t));
+   
+    /* set the correct type and size */
+    mlv_set_type((mlv_hdr_t *)dual_iso_block, "DISO");
+    dual_iso_block->blockSize = sizeof(mlv_diso_hdr_t);
+   
+    /* and fill with data */
+    dual_iso_block->dualMode = dual_iso_is_active();
+    dual_iso_block->isoValue = isoless_recovery_iso;
+   
+    /* finally pass it to mlv_rec which will free the block when it has been processed */
+    mlv_rec_queue_block((mlv_hdr_t *)dual_iso_block);
+}
+
 static unsigned int isoless_init()
 {
     if (is_camera("5D3", "1.1.3") || is_camera("5D3", "1.2.3"))
@@ -985,6 +1005,9 @@
         isoless_hdr = 0;
         return 1;
     }
+   
+    mlv_rec_register_cbr(MLV_REC_EVENT_PREPARING, &isoless_mlv_rec_cbr, NULL);
+   
     return 0;
 }
 
diff --git a/modules/mlv_lite/mlv_lite.c b/modules/mlv_lite/mlv_lite.c
--- a/modules/mlv_lite/mlv_lite.c
+++ b/modules/mlv_lite/mlv_lite.c
@@ -42,6 +42,8 @@
  * Boston, MA  02110-1301, USA.
  */
 
+#define __MLV_LITE_C__
+
 #include <module.h>
 #include <dryos.h>
 #include <property.h>
@@ -64,11 +66,13 @@
 #include "focus.h"
 #include "fps.h"
 #include "../mlv_rec/mlv.h"
+#include "../mlv_rec/mlv_rec_interface.h"
 #include "../trace/trace.h"
 #include "powersave.h"
 #include "shoot.h"
 #include "fileprefix.h"
 #include "timer.h"
+#include "ml-cbr.h"
 #include "../silent/lossless.h"
 #include "ml-cbr.h"
 
@@ -100,7 +104,6 @@
 static int cam_5d3 = 0;
 static int cam_5d3_113 = 0;
 static int cam_5d3_123 = 0;
-
 /**
  * resolution (in pixels) should be multiple of 16 horizontally (see http://www.magiclantern.fm/forum/index.php?topic=5839.0)
  * furthermore, resolution (in bytes) should be multiple of 8 in order to use the fastest EDMAC flags ( http://magiclantern.wikia.com/wiki/Register_Map#EDMAC ),
@@ -152,6 +155,7 @@
 static CONFIG_INT("raw.small.hacks", small_hacks, 1);
 
 static CONFIG_INT("raw.h264.proxy", h264_proxy_menu, 0);
+static CONFIG_INT("raw.sync_beep", sync_beep, 1);
 
 static CONFIG_INT("raw.output_format", output_format, 3);
 #define OUTPUT_14BIT_NATIVE 0
@@ -305,10 +309,12 @@
                            max_frame_size for uncompressed data, lower for compressed */
     int payload_size;   /* size effectively used by image data */
     int frame_number;   /* from 0 to n */
+    int is_meta;        /* when used by some other module and does not contain VIDF, disables consistency checks used for video frame slots */
     enum {
         SLOT_FREE,          /* available for image capture */
         SLOT_RESERVED,      /* it may become available when resizing the previous slots */
         SLOT_CAPTURING,     /* in progress */
+        SLOT_LOCKED,        /* locked by some other module */
         SLOT_FULL,          /* contains fully captured image data */
         SLOT_WRITING        /* it's being saved to card */
     } status;
@@ -326,6 +332,7 @@
 static GUARDED_BY(LiveViewTask) int capture_slot = -1;              /* in what slot are we capturing now (index) */
 static volatile                 int force_new_buffer = 0;           /* if some other task decides it's better to search for a new buffer */
 
+
 static GUARDED_BY(LiveViewTask) int writing_queue[COUNT(slots)+1];  /* queue of completed frames (slot indices) waiting to be saved */
 static GUARDED_BY(LiveViewTask) int writing_queue_tail = 0;         /* place captured frames here */
 static GUARDED_BY(RawRecTask)   int writing_queue_head = 0;         /* extract frames to be written from here */
@@ -341,6 +348,7 @@
 static GUARDED_BY(RawRecTask)   int writing_time = 0;               /* time spent by raw_video_rec_task in FIO_WriteFile calls */
 static GUARDED_BY(RawRecTask)   int idle_time = 0;                  /* time spent by raw_video_rec_task doing something else */
 static volatile                 uint32_t edmac_active = 0;
+static volatile                 uint32_t skip_frames = 0;
 
 /* for compress_task */
 static struct msg_queue * compress_mq = 0;
@@ -350,7 +358,7 @@
 static GUARDED_BY(RawRecTask)   mlv_rawc_hdr_t rawc_hdr;
 static GUARDED_BY(RawRecTask)   mlv_idnt_hdr_t idnt_hdr;
 static GUARDED_BY(RawRecTask)   mlv_expo_hdr_t expo_hdr;
-static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
+static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
 static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t elns_hdr;
 static GUARDED_BY(RawRecTask)   mlv_rtci_hdr_t rtci_hdr;
 static GUARDED_BY(RawRecTask)   mlv_wbal_hdr_t wbal_hdr;
@@ -358,12 +366,180 @@
 static GUARDED_BY(RawRecTask)   uint64_t mlv_start_timestamp = 0;
        GUARDED_BY(RawRecTask)   uint32_t raw_rec_trace_ctx = TRACE_ERROR;
 
-/* interface to other modules: these are called when recording starts or stops  */
-extern WEAK_FUNC(ret_0) unsigned int raw_rec_cbr_starting();
-extern WEAK_FUNC(ret_0) unsigned int raw_rec_cbr_stopping();
-
 static int raw_rec_should_preview(void);
 
+/* old mlv_rec interface stuff here */
+struct msg_queue *mlv_block_queue = NULL;
+/* registry of all other modules CBRs */
+static cbr_entry_t registered_cbrs[32];
+
+
+/* allow modules to set how many frames should be skipped */
+void mlv_rec_skip_frames(uint32_t count)
+{
+    skip_frames = count;
+}
+
+/* register a callback function that is called when one of the events specified happens.
+   event can be a OR'ed list of the events specified in mlv_rec_interface.h
+ */
+uint32_t mlv_rec_register_cbr(uint32_t event, event_cbr_t cbr, void *ctx)
+{
+    if(RAW_IS_RECORDING)
+    {
+        return 0;
+    }
+   
+    uint32_t ret = 0;
+    uint32_t old_int = cli();
+    for(int pos = 0; pos < COUNT(registered_cbrs); pos++)
+    {
+        if(registered_cbrs[pos].cbr == NULL)
+        {
+            registered_cbrs[pos].event = event;
+            registered_cbrs[pos].cbr = cbr;
+            registered_cbrs[pos].ctx = ctx;
+            ret = 1;
+            break;
+        }
+    }
+    sei(old_int);
+   
+    return ret;
+}
+
+/* unregister the specified CBR from all registered events */
+uint32_t mlv_rec_unregister_cbr(event_cbr_t cbr)
+{
+    if(RAW_IS_RECORDING)
+    {
+        return 0;
+    }
+   
+    uint32_t ret = 0;
+    uint32_t old_int = cli();
+    for(int pos = 0; (registered_cbrs[pos].cbr != NULL) && (pos < COUNT(registered_cbrs)); pos++)
+    {
+        /* is this the callback routine to be unregistered? */
+        if(registered_cbrs[pos].cbr == cbr)
+        {
+            /* if so, just shift all entries below one entry up. this keeps a void-less list with all CBRs to improve performance */
+            int32_t remaining = COUNT(registered_cbrs) - pos - 1;
+           
+            registered_cbrs[pos].cbr = NULL;
+           
+            if(remaining > 0)
+            {
+                memcpy(&registered_cbrs[pos], &registered_cbrs[pos + 1], remaining * sizeof(cbr_entry_t));
+                registered_cbrs[COUNT(registered_cbrs)-1].ctx = NULL;
+                ret = 1;
+                break;
+            }
+        }
+    }
+    sei(old_int);
+   
+    return ret;
+}
+
+/* call registered callbacks for the events specified */
+static void mlv_rec_call_cbr(uint32_t event, mlv_hdr_t *hdr)
+{
+    for(int pos = 0; (registered_cbrs[pos].cbr != NULL) && (pos < COUNT(registered_cbrs)); pos++)
+    {
+        /* there is still a possible race condition - if a module unregisters it's CBR during this function being called.
+           while this is unlikely to ever happen (all current modules register their CBRs upon init and not within a CBR,
+           this is still something that should be hardened. locks might be a bit too expensive.
+           copying every entry to stack also a bit costly, but will cause other side effects.
+        */
+        if(registered_cbrs[pos].event & event)
+        {
+            registered_cbrs[pos].cbr(event, registered_cbrs[pos].ctx, hdr);
+        }
+    }
+}
+
+/* helper to write a MLV block into a FILE* and return if successful */
+static int32_t mlv_write_hdr(FILE* f, mlv_hdr_t *hdr)
+{
+    mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, hdr);
+
+    uint32_t written = FIO_WriteFile(f, hdr, hdr->blockSize);
+
+    return written == hdr->blockSize;
+}
+
+/* return details about allocated slot */
+void mlv_rec_get_slot_info(int32_t slot, uint32_t *size, void **address)
+{
+    if(slot < 0 || slot >= total_slot_count)
+    {
+        *address = NULL;
+        *size = 0;
+        return;
+    }
+   
+    *address = slots[slot].ptr;
+    *size = slots[slot].size;
+}
+
+/* this can be called from anywhere to get a free memory slot. must be submitted using mlv_rec_release_slot() */
+int32_t mlv_rec_get_free_slot()
+{
+    int32_t ret = -1;
+   
+    for (int i = 0; (i < total_slot_count) && (ret == -1); i++)
+    {
+        uint32_t old_int = cli();
+        if (slots[i].status == SLOT_FREE)
+        {
+            slots[i].status = SLOT_LOCKED;
+            ret = i;
+        }
+        sei(old_int);
+    }
+   
+    return ret;
+}
+
+/* mark a previously with mlv_rec_get_free_slot() allocated slot for being reused or written into the file */
+void mlv_rec_release_slot(int32_t slot, uint32_t write)
+{
+    if(slot < 0 || slot >= total_slot_count)
+    {
+        return;
+    }
+
+    if(write)
+    {
+        uint32_t old_int = cli();
+        slots[slot].status = SLOT_FULL;
+        slots[slot].is_meta = 1;
+        writing_queue[writing_queue_tail] = slot;
+        INC_MOD(writing_queue_tail, COUNT(writing_queue));
+        sei(old_int);
+    }
+    else
+    {
+        slots[slot].status = SLOT_FREE;
+    }
+}
+
+/* set the timestamp relative to recording start */
+void mlv_rec_set_rel_timestamp(mlv_hdr_t *hdr, uint64_t timestamp)
+{
+    hdr->timestamp = timestamp - mlv_start_timestamp;
+}
+
+/* queuing of blocks from other modules */
+uint32_t mlv_rec_queue_block(mlv_hdr_t *hdr)
+{
+    mlv_set_timestamp(hdr, mlv_start_timestamp);
+    msg_queue_post(mlv_block_queue, (uint32_t) hdr);
+   
+    return 1;
+}
+
 static inline int pre_recording_buffer_full()
 {
     /* fixme: not very accurate with variable frame sizes */
@@ -1460,9 +1636,11 @@
             }
 
             int color = slots[i].status == SLOT_FREE      ? COLOR_GRAY(10) :
+                        slots[i].is_meta                  ? COLOR_BLUE :
                         slots[i].status == SLOT_WRITING   ? COLOR_GREEN1 :
                         slots[i].status == SLOT_FULL      ? COLOR_LIGHT_BLUE :
                         slots[i].status == SLOT_RESERVED  ? COLOR_GRAY(50) :
+                        slots[i].status == SLOT_LOCKED    ? COLOR_YELLOW :
                                                             COLOR_RED ;
 
             uint32_t x1 = (uint32_t) slots[i].ptr - chunk_start;
@@ -2578,6 +2756,19 @@
         frame_count++;
         return;
     }
+
+    /* some modules may do some specific stuff right when we started recording */
+    if (frame_count == 1)
+    {
+        mlv_rec_call_cbr(MLV_REC_EVENT_STARTED, NULL);
+    }
+
+    /* other modules can ask for some frames to skip, e.g. for syncing audio */
+    if (skip_frames > 0)
+    {
+        skip_frames--;
+        return;
+    }
     
     if (edmac_active)
     {
@@ -2670,7 +2861,7 @@
     if (!RAW_IS_RECORDING) return 0;
     if (!raw_lv_settings_still_valid()) { raw_recording_state = RAW_FINISHING; return 0; }
     if (buffer_full) return 0;
-
+   
     /* double-buffering */
     raw_lv_redirect_edmac(fullsize_buffers[fullsize_buffer_pos % 2]);
 
@@ -2828,7 +3019,7 @@
 
     mlv_fill_idnt(&idnt_hdr, mlv_start_timestamp);
     mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
-    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
     mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
     mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
     mlv_fill_wbal(&wbal_hdr, mlv_start_timestamp);
@@ -2841,18 +3032,49 @@
 }
 
 static REQUIRES(RawRecTask)
-int write_mlv_chunk_headers(FILE* f)
+int write_mlv_chunk_headers(FILE* f, int chunk)
 {
-    if (FIO_WriteFile(f, &file_hdr, file_hdr.blockSize) != (int)file_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &rawi_hdr, rawi_hdr.blockSize) != (int)rawi_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &rawc_hdr, rawc_hdr.blockSize) != (int)rawc_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &idnt_hdr, idnt_hdr.blockSize) != (int)idnt_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &wbal_hdr, wbal_hdr.blockSize) != (int)wbal_hdr.blockSize) return 0;
-    if (mlv_write_vers_blocks(f, mlv_start_timestamp)) return 0;
+    /* looks a bit cleaner not to have several return points */
+    int fail = 0;
+   
+    /* all chunks contain the MLVI header */
+    fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&file_hdr);
+   
+    /* only the first chunk contains this information if nothing changes */
+    if(chunk == 0)
+    {
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&rawi_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&rawc_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&idnt_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&expo_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&lens_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&rtci_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&wbal_hdr);
+        fail |= mlv_write_vers_blocks(f, mlv_start_timestamp);
+    }
+   
+    /* write all queued blocks, if any */
+    uint32_t msg_count = 0;
+    msg_queue_count(mlv_block_queue, &msg_count);
+   
+    for(uint32_t msg = 0; msg < msg_count; msg++)
+    {
+        mlv_hdr_t *block = NULL;
+
+        /* there is a block in the queue, try to get that block */
+        if(!msg_queue_receive(mlv_block_queue, &block, 0))
+        {
+            fail |= !mlv_write_hdr(f, block);
+            free(block);
+        }
+    }
+   
+    /* any of the above writes failed, exit */
+    if(fail)
+    {
+        return 0;
+    }
     
     int hdr_size = FIO_SeekSkipFile(f, 0, SEEK_CUR);
     
@@ -2874,13 +3096,19 @@
 void finish_chunk(FILE* f)
 {
     file_hdr.videoFrameCount = chunk_frame_count;
+   
+    /* call the CBRs which may update fields */
+    mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, (mlv_hdr_t *)&file_hdr);
+   
     FIO_SeekSkipFile(f, 0, SEEK_SET);
     FIO_WriteFile(f, &file_hdr, file_hdr.blockSize);
     FIO_CloseFile(f);
     chunk_frame_count = 0;
 }
 
-/* This saves a group of frames, also taking care of file splitting if required */
+/* This saves a group of frames, also taking care of file splitting if required.
+   Parameter num_frames is meant for counting the VIDF blocks for updating MLVI header.
+ */
 static REQUIRES(RawRecTask)
 int write_frames(FILE** pf, void* ptr, int group_size, int num_frames)
 {
@@ -2897,7 +3125,7 @@
         if (!g) return 0;
         
         file_hdr.fileNum = mlv_chunk;
-        written_chunk = write_mlv_chunk_headers(g);
+        written_chunk = write_mlv_chunk_headers(g, mlv_chunk);
         written_total += written_chunk;
         
         if (written_chunk)
@@ -2954,7 +3182,7 @@
         if (!g) return 0;
         
         file_hdr.fileNum = mlv_chunk;
-        written_chunk = write_mlv_chunk_headers(g);
+        written_chunk = write_mlv_chunk_headers(g, mlv_chunk);
         written_total += written_chunk;
         
         int r2 = written_chunk ? FIO_WriteFile(g, ptr, group_size) : 0;
@@ -3012,6 +3240,7 @@
     raw_recording_state = RAW_PREPARING;
     give_semaphore(settings_sem);
 
+    mlv_rec_call_cbr(MLV_REC_EVENT_PREPARING, NULL);
     /* locals */
     FILE* f = 0;
     int last_block_size = 0; /* for detecting early stops */
@@ -3087,25 +3316,25 @@
         goto cleanup;
     }
 
+    /* Need to start the recording of audio before the init of the mlv chunk */
+    mlv_rec_call_cbr(MLV_REC_EVENT_STARTING, NULL);
+
     init_mlv_chunk_headers(&raw_info);
-    written_total = written_chunk = write_mlv_chunk_headers(f);
+    written_total = written_chunk = write_mlv_chunk_headers(f, mlv_chunk);
     if (!written_chunk)
     {
         NotifyBox(5000, "Card Full");
         goto cleanup;
     }
-
+   
     hack_liveview(0);
     liveview_hacked = 1;
 
-    /* this will enable the vsync CBR and the other task(s) */
-    raw_recording_state = pre_record ? RAW_PRE_RECORDING : RAW_RECORDING;
-
     /* try a sync beep (not very precise, but better than nothing) */
-    beep();
-
-    /* signal that we are starting */
-    raw_rec_cbr_starting();
+    if(sync_beep)
+    {
+        beep();
+    }
 
     /* signal start of recording to the compression task */
     msg_queue_post(compress_mq, INT_MAX);
@@ -3116,6 +3345,9 @@
     int fps = fps_get_current_x1000();
     
     int last_processed_frame = 0;
+
+    /* this will enable the vsync CBR and the other task(s) */
+    raw_recording_state = pre_record ? RAW_PRE_RECORDING : RAW_RECORDING;
     
     /* main recording loop */
     while (RAW_IS_RECORDING && lv)
@@ -3173,6 +3405,7 @@
         int last_grouped = w_head;
         
         int group_size = 0;
+        int meta_slots = 0;
         for (int i = w_head; i != w_tail; INC_MOD(i, COUNT(writing_queue)))
         {
             int slot_index = writing_queue[i];
@@ -3184,13 +3417,21 @@
                 break;
             }
 
-            /* consistency checks */
-            ASSERT(((mlv_vidf_hdr_t*)slots[slot_index].ptr)->blockSize == (uint32_t) slots[slot_index].size);
-            ASSERT(((mlv_vidf_hdr_t*)slots[slot_index].ptr)->frameNumber == (uint32_t) slots[slot_index].frame_number - 1);
-
-            if (OUTPUT_COMPRESSION)
+            /* consistency checks for VIDF slots */
+            if (!slots[slot_index].is_meta)
             {
-                ASSERT(slots[slot_index].size < max_frame_size);
+                ASSERT(((mlv_vidf_hdr_t*)slots[slot_index].ptr)->blockSize == (uint32_t) slots[slot_index].size);
+                ASSERT(((mlv_vidf_hdr_t*)slots[slot_index].ptr)->frameNumber == (uint32_t) slots[slot_index].frame_number - 1);
+               
+                if (OUTPUT_COMPRESSION)
+                {
+                    ASSERT(slots[slot_index].size < max_frame_size);
+                }
+            }
+            else
+            {
+                /* count the number of slots being non-VIDF */
+                meta_slots++;
             }
 
             /* TBH, I don't care if these are part of the same group or not,
@@ -3243,7 +3484,7 @@
         {
             int slot_index = writing_queue[i];
 
-            if (OUTPUT_COMPRESSION)
+            if (OUTPUT_COMPRESSION && !slots[slot_index].is_meta)
             {
                 ASSERT(slots[slot_index].size < max_frame_size);
             }
@@ -3263,7 +3504,7 @@
         idle_time += t0 - last_write_timestamp;
 
         /* save a group of frames and measure execution time */
-        if (!write_frames(&f, ptr, group_size, num_frames))
+        if (!write_frames(&f, ptr, group_size, num_frames - meta_slots))
         {
             goto abort;
         }
@@ -3287,23 +3528,30 @@
             
             int slot_index = writing_queue[i];
 
-            if (frame_check_saved(slot_index) != 1)
+            if (!slots[slot_index].is_meta)
             {
-                bmp_printf( FONT_MED, 30, 110,
-                    "Data corruption at slot %d, frame %d ", slot_index, slots[slot_index].frame_number
-                );
-                beep();
+                if (frame_check_saved(slot_index) != 1)
+                {
+                    bmp_printf( FONT_MED, 30, 110,
+                        "Data corruption at slot %d, frame %d ", slot_index, slots[slot_index].frame_number
+                    );
+                    beep();
+                }
+               
+                if (slots[slot_index].frame_number != last_processed_frame + 1)
+                {
+                    bmp_printf( FONT_MED, 30, 110,
+                        "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number, last_processed_frame + 1
+                    );
+                    beep();
+                }
+                last_processed_frame++;
+            }
+            else
+            {
+                slots[slot_index].is_meta = 0;
             }
             
-            if (slots[slot_index].frame_number != last_processed_frame + 1)
-            {
-                bmp_printf( FONT_MED, 30, 110,
-                    "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number, last_processed_frame + 1
-                );
-                beep();
-            }
-            last_processed_frame++;
-
             free_slot(slot_index);
         }
         
@@ -3346,18 +3594,16 @@
     /* make sure the user doesn't rush to turn off the camera or something */
     gui_uilock(UILOCK_EVERYTHING);
     
-    /* signal that we are stopping */
-    raw_rec_cbr_stopping();
-   
     /* done, this will stop the vsync CBR and the copying task */
     raw_recording_state = RAW_FINISHING;
+    mlv_rec_call_cbr(MLV_REC_EVENT_STOPPING, NULL);
 
     /* wait until the other tasks calm down */
     wait_lv_frames(2);
 
     /* signal end of recording to the compression task */
     msg_queue_post(compress_mq, INT_MIN);
-
+   
     set_recording_custom(CUSTOM_RECORDING_NOT_RECORDING);
 
     if (!RECORDING_H264)
@@ -3368,6 +3614,28 @@
         /* PauseLiveView breaks UI locks - why? */
         gui_uilock(UILOCK_EVERYTHING);
     }
+   
+    /* write all queued blocks, if any */
+    uint32_t msg_count = 0;
+    msg_queue_count(mlv_block_queue, &msg_count);
+   
+    for(uint32_t msg = 0; msg < msg_count; msg++)
+    {
+        mlv_hdr_t *block = NULL;
+
+        /* there is a block in the queue, try to get that block */
+        if(!msg_queue_receive(mlv_block_queue, &block, 0))
+        {
+            /* when this block will get written, call the CBR */
+            mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, block);
+           
+            /* use the write func to write the block */
+            write_frames(&f, block, block->blockSize, 0);
+           
+            /* free the block */
+            free(block);
+        }
+    }
 
     /* write remaining frames */
     /* H.264: we will be recording black frames during this time,
@@ -3388,30 +3656,37 @@
             beep();
         }
 
-        if (frame_check_saved(slot_index) != 1)
+        /* video frame consistency checks only for VIDF */
+        if(!slots[slot_index].is_meta)
         {
-            bmp_printf( FONT_MED, 30, 110,
-                "Data corruption at slot %d, frame %d ", slot_index, slots[slot_index].frame_number
-            );
-            beep();
+            if (frame_check_saved(slot_index) != 1)
+            {
+                bmp_printf( FONT_MED, 30, 110,
+                    "Data corruption at slot %d, frame %d ", slot_index, slots[slot_index].frame_number
+                );
+                beep();
+            }
+
+            if (slots[slot_index].frame_number != last_processed_frame + 1)
+            {
+                bmp_printf( FONT_MED, 30, 110,
+                    "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number, last_processed_frame + 1
+                );
+                beep();
+            }
+            last_processed_frame++;
+           
+            /* if it's a VIDF, then it should be smaller than the max frame size when compression is enabled */
+            if (OUTPUT_COMPRESSION)
+            {
+                ASSERT(slots[slot_index].size < max_frame_size);
+            }
         }
-
-        if (slots[slot_index].frame_number != last_processed_frame + 1)
-        {
-            bmp_printf( FONT_MED, 30, 110,
-                "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number, last_processed_frame + 1
-            );
-            beep();
-        }
-        last_processed_frame++;
-
+       
         slots[slot_index].status = SLOT_WRITING;
-        if (OUTPUT_COMPRESSION)
-        {
-            ASSERT(slots[slot_index].size < max_frame_size);
-        }
+       
         if (indicator_display == INDICATOR_RAW_BUFFER) show_buffer_status();
-        if (!write_frames(&f, slots[slot_index].ptr, slots[slot_index].size, 1))
+        if (!write_frames(&f, slots[slot_index].ptr, slots[slot_index].size, slots[slot_index].is_meta ? 0 : 1))
         {
             NotifyBox(5000, "Card Full");
             beep();
@@ -3466,6 +3741,7 @@
     ResumeLiveView();
     redraw();
     raw_recording_state = RAW_IDLE;
+    mlv_rec_call_cbr(MLV_REC_EVENT_STOPPED, NULL);
 }
 
 static REQUIRES(GuiMainTask)
@@ -3639,6 +3915,13 @@
                 .advanced = 1,
             },
             {
+                .name = "Sync beep",
+                .priv = &sync_beep,
+                .max    = 1,
+                .help = "Beeps on recording start for better sync.",
+                .advanced = 1,
+            },
+            {
                 .name   = "Show EDMAC",
                 .priv   = &show_edmac,
                 .max    = 1,
@@ -4026,6 +4309,9 @@
         }
         NotifyBoxHide();
     }
+   
+    /* allocate queue that other modules will fill with blocks to write to the current file */
+    mlv_block_queue = (struct msg_queue *) msg_queue_create("mlv_block_queue", 100);
 
     lossless_init();
 
@@ -4068,6 +4354,7 @@
     MODULE_CONFIG(use_srm_memory)
     MODULE_CONFIG(small_hacks)
     MODULE_CONFIG(warm_up)
+    MODULE_CONFIG(sync_beep)
     MODULE_CONFIG(output_format)
     MODULE_CONFIG(h264_proxy_menu)
 MODULE_CONFIGS_END()
diff --git a/modules/mlv_rec/mlv.h b/modules/mlv_rec/mlv.h
--- a/modules/mlv_rec/mlv.h
+++ b/modules/mlv_rec/mlv.h
@@ -307,6 +307,7 @@
 void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
+void mlv_fill_wavi(mlv_wavi_hdr_t *hdr, uint64_t start_timestamp);  /* provided by mlv_snd */
 void mlv_build_vers(mlv_vers_hdr_t **hdr, uint64_t start_timestamp, const char *version_string);
 
 /* randomize the 64 bits passed in parameter using LFSR */
diff --git a/modules/mlv_rec/mlv_dump.c b/modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c
+++ b/modules/mlv_rec/mlv_dump.c
@@ -1917,6 +1917,7 @@
     /* start processing */
     lv_rec_file_footer_t lv_rec_footer;
     mlv_file_hdr_t main_header;
+    mlv_diso_hdr_t diso_info;
     mlv_lens_hdr_t lens_info;
     mlv_elns_hdr_t elns_info;
     mlv_expo_hdr_t expo_info;
@@ -1929,6 +1930,7 @@
 
     /* initialize stuff */
     memset(&lv_rec_footer, 0x00, sizeof(lv_rec_file_footer_t));
+    memset(&diso_info, 0x00, sizeof(mlv_diso_hdr_t));
     memset(&lens_info, 0x00, sizeof(mlv_lens_hdr_t));
     memset(&elns_info, 0x00, sizeof(mlv_elns_hdr_t));
     memset(&expo_info, 0x00, sizeof(mlv_expo_hdr_t));
@@ -4093,12 +4095,22 @@
                         print_msg(MSG_ERROR, "Failed writing into .WAV file\n");
                         goto abort;
                     }
-
+                   
                     /* init WAV data size, will be grow later block by block (AUDF) */
                     wav_data_size = 0;
                     wav_header_size = file_get_pos(out_file_wav); /* same as sizeof(struct wav_header) */
                 }
             }
+            else if(!memcmp(mlv_block->blockType, "DISO", 4))
+            {
+                diso_info = *(mlv_diso_hdr_t *)mlv_block;
+
+                if(verbose)
+                {
+                    print_msg(MSG_INFO, "     Mode:        %d\n", diso_info.dualMode);
+                    print_msg(MSG_INFO, "     ISO Value:   %d\n", diso_info.isoValue);
+                }
+            }
             else if(!memcmp(mlv_block->blockType, "NULL", 4))
             {
                 /* those are just placeholders. ignore them. */
diff --git a/modules/mlv_rec/mlv_rec.c b/modules/mlv_rec/mlv_rec.c
--- a/modules/mlv_rec/mlv_rec.c
+++ b/modules/mlv_rec/mlv_rec.c
@@ -53,6 +53,8 @@
 //#define CONFIG_CONSOLE
 //#define TRACE_DISABLED
 
+#define __MLV_REC_C__
+
 #include <module.h>
 #include <dryos.h>
 #include <property.h>
@@ -79,6 +81,7 @@
 #include "../trace/trace.h"
 
 #include "mlv.h"
+#include "mlv_rec_interface.h"
 #include "mlv_rec.h"
 
 /* an alternative tracing method that embeds the logs into the MLV file itself */
@@ -123,6 +126,8 @@
 static uint32_t mlv_writer_threads = 2;
 static uint32_t mlv_max_filesize = 0xFFFFFFFF;
 static uint32_t abort_test = 0;
+static uint32_t skip_frames = 0;
+
 
 uint32_t raw_rec_trace_ctx = TRACE_ERROR;
 
@@ -253,6 +258,83 @@
 
 static volatile int32_t frame_countdown = 0;          /* for waiting X frames */
 
+/* registry of all other modules CBRs */
+static cbr_entry_t registered_cbrs[32];
+
+uint32_t mlv_rec_register_cbr(uint32_t event, event_cbr_t cbr, void *ctx)
+{
+    if(RAW_IS_RECORDING)
+    {
+        return 0;
+    }
+   
+    uint32_t ret = 0;
+    uint32_t old_int = cli();
+    for(int pos = 0; pos < COUNT(registered_cbrs); pos++)
+    {
+        if(registered_cbrs[pos].cbr == NULL)
+        {
+            registered_cbrs[pos].event = event;
+            registered_cbrs[pos].cbr = cbr;
+            registered_cbrs[pos].ctx = ctx;
+            ret = 1;
+            break;
+        }
+    }
+    sei(old_int);
+   
+    return ret;
+}
+
+uint32_t mlv_rec_unregister_cbr(event_cbr_t cbr)
+{
+    if(RAW_IS_RECORDING)
+    {
+        return 0;
+    }
+   
+    uint32_t ret = 0;
+    uint32_t old_int = cli();
+    for(int pos = 0; (registered_cbrs[pos].cbr != NULL) && (pos < COUNT(registered_cbrs)); pos++)
+    {
+        if(registered_cbrs[pos].cbr == cbr)
+        {
+            int32_t remaining = COUNT(registered_cbrs) - pos - 1;
+           
+            registered_cbrs[pos].cbr = NULL;
+           
+            if(remaining > 0)
+            {
+                memcpy(&registered_cbrs[pos], &registered_cbrs[pos + 1], remaining * sizeof(cbr_entry_t));
+                registered_cbrs[COUNT(registered_cbrs)-1].ctx = NULL;
+                ret = 1;
+                break;
+            }
+        }
+    }
+    sei(old_int);
+   
+    return ret;
+}
+
+static void mlv_rec_call_cbr(uint32_t event, mlv_hdr_t *hdr)
+{
+    for(int pos = 0; (registered_cbrs[pos].cbr != NULL) && (pos < COUNT(registered_cbrs)); pos++)
+    {
+        if(registered_cbrs[pos].event & event)
+        {
+            registered_cbrs[pos].cbr(event, registered_cbrs[pos].ctx, hdr);
+        }
+    }
+}
+
+/* allow modules to set how many frames should be skipped */
+void mlv_rec_skip_frames(uint32_t count)
+{
+    skip_frames = count;
+}
+
+
 #if defined(EMBEDDED_LOGGING)
 /* START: helper code for logging into MLV files */
 static uint8_t *mlv_debg_buffer = NULL;
@@ -921,6 +1003,7 @@
             mlv_hdr_t *write_align_hdr = (mlv_hdr_t *)((uint32_t)vidf_hdr + vidf_hdr->blockSize);
             memset(write_align_hdr, 0xA5, write_size_align);
             mlv_set_type(write_align_hdr, "NULL");
+            write_align_hdr->timestamp = 0;
             write_align_hdr->blockSize = write_size_align;
         }
 
@@ -1443,6 +1526,8 @@
             mlv_rtci_hdr_t *rtci_hdr = malloc(sizeof(mlv_rtci_hdr_t));
             mlv_fill_rtci(rtci_hdr, mlv_start_timestamp);
             msg_queue_post(mlv_block_queue, (uint32_t) rtci_hdr);
+           
+            mlv_rec_call_cbr(MLV_REC_EVENT_CYCLIC, NULL);
         }
 
         if(should_run_polling_action(MLV_INFO_BLOCK_INTERVAL, &block_queueing) && (mlv_metadata & MLV_METADATA_CYCLIC))
@@ -1761,9 +1846,12 @@
     }
 }
 
-void mlv_rec_queue_block(mlv_hdr_t *hdr)
+uint32_t mlv_rec_queue_block(mlv_hdr_t *hdr)
 {
+    mlv_set_timestamp(hdr, mlv_start_timestamp);
     msg_queue_post(mlv_block_queue, (uint32_t) hdr);
+   
+    return 1;
 }
 
 void mlv_rec_set_rel_timestamp(mlv_hdr_t *hdr, uint64_t timestamp)
@@ -1851,6 +1939,7 @@
 
     /* set old header to a skipped header format */
     mlv_set_type((mlv_hdr_t *)vidf, "NULL");
+    vidf->timestamp = 0;
 
     /* backup old size into free space */
     ((uint32_t*) vidf)[sizeof(mlv_vidf_hdr_t)/4] = vidf->blockSize;
@@ -2127,10 +2216,14 @@
 
 static void mlv_rec_dma_cbr_r(void *ctx)
 {
+   
 }
 
 static void mlv_rec_dma_cbr_w(void *ctx)
 {
+    /* call the VIDF CBRs */
+    mlv_rec_call_cbr(MLV_REC_EVENT_VIDF, slots[capture_slot].ptr);
+
     mlv_rec_dma_active = 0;
     
     mlv_rec_dma_end = get_us_clock();
@@ -2147,6 +2240,11 @@
         frame_count++;
         return 0;
     }
+   
+    if(frame_count == 1)
+    {
+        mlv_rec_call_cbr(MLV_REC_EVENT_STARTED, NULL);
+    }
 
     /* where to save the next frame? */
     capture_slot = choose_next_capture_slot(capture_slot);
@@ -2217,6 +2315,13 @@
         return 0;
     }
     
+ /* other modules can ask for some frames to skip, e.g. for syncing audio */
+    if(skip_frames > 0)
+    {
+        skip_frames--;
+        return 0;
+    }
+   
     /* if previous DMA isn't finished yet, skip frame */
     if(mlv_rec_dma_active)
     {
@@ -2229,7 +2334,6 @@
         {
             edmac_timeouts = 0;
             raw_recording_state = RAW_FINISHING;
-            raw_rec_cbr_stopping();
         }
         return 0;
     }
@@ -2248,7 +2352,6 @@
     if(!raw_lv_settings_still_valid())
     {
         raw_recording_state = RAW_FINISHING;
-        raw_rec_cbr_stopping();
         return 0;
     }
     
@@ -2337,7 +2440,7 @@
 
 static int32_t mlv_write_hdr(FILE* f, mlv_hdr_t *hdr)
 {
-    raw_rec_cbr_mlv_block(hdr);
+    mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, hdr);
 
     uint32_t written = FIO_WriteFile(f, hdr, hdr->blockSize);
 
@@ -2903,7 +3006,6 @@
         {
 abort:
             raw_recording_state = RAW_FINISHING;
-            raw_rec_cbr_stopping();
             NotifyBox(5000, "Recording stopped:\n '%s'", error_message);
             /* this is error beep, not audio sync beep */
             beep_times(2);
@@ -2975,7 +3077,8 @@
             }
             else
             {
-                raw_rec_cbr_mlv_block(block);
+                /* when this block will get prepended, call the CBR */
+                mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, block);
 
                 /* prepend the given block if possible or requeue it in case of error */
                 int32_t ret = mlv_prepend_block(slot, block);
@@ -3198,6 +3301,8 @@
     /* init stuff */
     raw_recording_state = RAW_PREPARING;
 
+    mlv_rec_call_cbr(MLV_REC_EVENT_PREPARING, NULL);
+
     if(DISPLAY_REC_INFO_DEBUG)
     {
         bmp_printf(FONT_MED, 30, 50, "Prepare recording...");
@@ -3226,10 +3331,7 @@
     
     /* disable Canon's powersaving (30 min in LiveView) */
     powersave_prohibit();
-
-    /* signal that we are starting, call this before any memory allocation to give CBR the chance to allocate memory */
-    raw_rec_cbr_starting();
-
+   
     /* allocate memory */
     if(!setup_buffers())
     {
@@ -3239,6 +3341,9 @@
         goto cleanup;
     }
 
+    /* signal that we are starting */
+    mlv_rec_call_cbr(MLV_REC_EVENT_STARTING, NULL);
+
     msleep(start_delay * 1000);
 
     hack_liveview(0);
@@ -3339,9 +3444,6 @@
         /* this will enable the vsync CBR and the other task(s) */
         raw_recording_state = RAW_RECORDING;
 
-        /* some modules may do some specific stuff right when we started recording */
-        raw_rec_cbr_started();
-
         while((raw_recording_state == RAW_RECORDING) || (used_slots > 0))
         {
             /* on shutdown or writers that aborted, abort even if there are unwritten slots */
@@ -3374,7 +3476,6 @@
                 NotifyBox(5000, "Frame skipped. Stopping");
                 trace_write(raw_rec_trace_ctx, "<-- stopped recording, frame was skipped");
                 raw_recording_state = RAW_FINISHING;
-                raw_rec_cbr_stopping();
             }
 
             /* how fast are we writing? does this speed match our benchmarks? */
@@ -3492,7 +3593,6 @@
                             /* try to free up some space and exit */
                             mlv_rec_release_dummies();
                             raw_recording_state = RAW_FINISHING;
-                            raw_rec_cbr_stopping();
                         }
                         raw_prepare_chunk(handle->file_handle, &handle->file_header);
                     }
@@ -3545,6 +3645,8 @@
             }
         }
         
+        mlv_rec_call_cbr(MLV_REC_EVENT_STOPPING, NULL);
+       
         /* now close all queued files */
         while(1)
         {
@@ -3609,7 +3711,6 @@
 
         /* done, this will stop the vsync CBR and the copying task */
         raw_recording_state = RAW_FINISHING;
-        raw_rec_cbr_stopping();
 
         /* queue two aborts to cancel tasks */
         msg_queue_receive(mlv_job_alloc_queue, &write_job, 0);
@@ -3646,7 +3747,7 @@
 
 cleanup:
     /* signal that we are stopping */
-    raw_rec_cbr_stopped();
+    mlv_rec_call_cbr(MLV_REC_EVENT_STOPPED, NULL);
 
     /*
     if(DISPLAY_REC_INFO_DEBUG)
@@ -3686,7 +3787,6 @@
     {
         abort_test = 1;
         raw_recording_state = RAW_FINISHING;
-        raw_rec_cbr_stopping();
     }
     else
     {
diff --git a/modules/mlv_rec/mlv_rec.h b/modules/mlv_rec/mlv_rec.h
--- a/modules/mlv_rec/mlv_rec.h
+++ b/modules/mlv_rec/mlv_rec.h
@@ -64,7 +64,6 @@
 #define MLV_METADATA_CYCLIC   4
 #define MLV_METADATA_ALL      0xFF
 
-
 /* one video frame */
 struct frame_slot
 {
@@ -144,25 +143,8 @@
  *      This function is called on every single raw frame that is received from sensor with a pointer to frame data as parameter.
  *      If the return value is zero, the frame will get save into the saving buffers, else it is skipped
  *      Default: Do not skip frame (0)
- *
- *    uint32_t raw_rec_save_buffer(uint32_t used, uint32_t buffer_count)
- *      This function is called whenever the writing loop is checking if it has data to save to card.
- *      The parameters are the number of used buffers and the total buffer count
- *      Default: Save buffer (1)
- *
- *    uint32_t raw_rec_skip_buffer(uint32_t buffer_index, uint32_t buffer_count);
- *      Whenever the buffers are full, this function is called with the buffer index that is subject to being dropped, the number of frames in this buffer and the total buffer count.
- *      If it returns zero, this buffer will not get thrown away, but the next frame will get dropped.
- *      Default: Do not throw away buffer, but throw away incoming frame (0)
  */
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_starting();
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_started();
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_stopping();
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_stopped();
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_mlv_block(mlv_hdr_t *hdr);
 extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_skip_frame(unsigned char *frame_data);
-extern WEAK_FUNC(ret_1) uint32_t raw_rec_cbr_save_buffer(uint32_t used, uint32_t buffer_index, uint32_t frame_count, uint32_t buffer_count);
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_skip_buffer(uint32_t buffer_index, uint32_t frame_count, uint32_t buffer_count);
 
 
 /* helpers for reserving disc space */
@@ -204,7 +186,6 @@
 static void hack_liveview_vsync();
 static void unhack_liveview_vsync(int32_t unused);
 static void hack_liveview(int32_t unhack);
-void mlv_rec_queue_block(mlv_hdr_t *hdr);
 void mlv_rec_set_rel_timestamp(mlv_hdr_t *hdr, uint64_t timestamp);
 int32_t mlv_rec_get_free_slot();
 void mlv_rec_get_slot_info(int32_t slot, uint32_t *size, void **address);
diff --git a/modules/mlv_rec/mlv_rec_interface.h b/modules/mlv_rec/mlv_rec_interface.h
new file mode 100644
--- /dev/null
+++ b/modules/mlv_rec/mlv_rec_interface.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright (C) 2013 Magic Lantern Team
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the
+ * Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor,
+ * Boston, MA  02110-1301, USA.
+ */
+
+#ifndef __MLV_REC_INTERFACE_H__
+#define __MLV_REC_INTERFACE_H__
+
+
+/* interface to register callbacks for various stuff */
+typedef void (*event_cbr_t) (uint32_t event, void *ctx, mlv_hdr_t *hdr);
+
+/* event types, can be combined */
+#define MLV_REC_EVENT_STARTING    (1U<<0) /* gets called when recording is going to be started */
+#define MLV_REC_EVENT_STARTED     (1U<<1) /* gets called when recording was started and blocks are being written */
+#define MLV_REC_EVENT_STOPPING    (1U<<2) /* gets called when going to stop recoding for whatever reason. good point to finally push some data before files are closed */
+#define MLV_REC_EVENT_STOPPED     (1U<<3) /* gets called when recording has stopped */
+#define MLV_REC_EVENT_CYCLIC      (1U<<4) /* gets called whenever the cyclic RTCI block is written (usually 2 second interval) */
+#define MLV_REC_EVENT_BLOCK       (1U<<5) /* gets called for every block before it is being written to the file. 'hdr' parameter will contain a pointer to the the block. might get called multiple times per block! */
+#define MLV_REC_EVENT_VIDF        (1U<<6) /* gets called for every VIDF being queued for write. called from EDMAC CBR, so avoid using too much CPU time. */
+#define MLV_REC_EVENT_PREPARING   (1U<<7) /* gets called before any buffer allocation or LV manipulation */
+
+
+#if !defined(__MLV_REC_C__) && !defined(__MLV_LITE_C__)
+
+/* register function as callback routine for all events ORed into 'event' and call that CBR with given 'ctx' */
+extern WEAK_FUNC(ret_0) uint32_t mlv_rec_register_cbr(uint32_t event, event_cbr_t cbr, void *ctx);
+
+/* unregister a previously registered CBR for any event */
+extern WEAK_FUNC(ret_0) uint32_t mlv_rec_unregister_cbr(event_cbr_t cbr);
+
+/* queue a MLV block for writing. timestamp will get set automatically as it requires knowledge of the absolute record starting time */
+extern WEAK_FUNC(ret_0) uint32_t mlv_rec_queue_block(mlv_hdr_t *hdr);
+
+#else
+   
+/* structure entry for registered CBR routines */
+typedef struct
+{
+    uint32_t event;
+    void *ctx;
+    event_cbr_t cbr;
+} cbr_entry_t;
+
+#endif
+
+#endif
diff --git a/modules/mlv_snd/mlv_snd.c b/modules/mlv_snd/mlv_snd.c
--- a/modules/mlv_snd/mlv_snd.c
+++ b/modules/mlv_snd/mlv_snd.c
@@ -30,20 +30,25 @@
 #include <beep.h>
 #include <propvalues.h>
 #include <raw.h>
-#include "ml-cbr.h"
+#include <ml-cbr.h>
 
 #include "../trace/trace.h"
 #include "../mlv_rec/mlv.h"
+#include "../mlv_rec/mlv_rec_interface.h"
 
-#define MLV_SND_BUFFERS 4
+/* allocate that many frame slots to be used for WAVI blocks. two is the minimum to maintain operation */
+#define MLV_SND_SLOTS              2
+/* maximum number of WAVI blocks per slot. the larger, the longer queues will get. should we use more? */
+#define MLV_SND_BLOCKS_PER_SLOT  256
 
 static uint32_t trace_ctx = TRACE_ERROR;
 
-static CONFIG_INT("mlv.snd.enabled", mlv_snd_enabled, 0);
+static CONFIG_INT("mlv.snd.enabled", mlv_snd_enabled, 1);
 static CONFIG_INT("mlv.snd.mlv_snd_enable_tracing", mlv_snd_enable_tracing, 0);
 static CONFIG_INT("mlv.snd.bit.depth", mlv_snd_in_bits_per_sample, 16);
 static CONFIG_INT("mlv.snd.sample.rate", mlv_snd_in_sample_rate, 48000);
 static CONFIG_INT("mlv.snd.sample.rate.selection", mlv_snd_rate_sel, 0);
+static CONFIG_INT("mlv.snd.vsync_delay", mlv_snd_vsync_delay, 1);
 
 extern int StartASIFDMAADC(void *, uint32_t, void *, uint32_t, void (*)(), uint32_t);
 extern int SetNextASIFADCBuffer(void *, uint32_t);
@@ -60,15 +65,22 @@
 extern int32_t mlv_rec_get_free_slot();
 extern void mlv_rec_release_slot(int32_t slot, uint32_t write);
 extern void mlv_rec_set_rel_timestamp(mlv_hdr_t *hdr, uint64_t timestamp);
-extern void mlv_rec_queue_block(mlv_hdr_t *hdr);
+extern void mlv_rec_skip_frames(uint32_t count);
 
-static volatile int32_t mlv_snd_rec_active = 0;
 static struct msg_queue * volatile mlv_snd_buffers_empty = NULL;
 static struct msg_queue * volatile mlv_snd_buffers_done = NULL;
 static volatile uint32_t mlv_snd_in_buffers = 64;
 static volatile uint32_t mlv_snd_frame_number = 0;
 static volatile uint32_t mlv_snd_in_buffer_size = 0;
 
+/* for tracking chunks */
+static volatile uint16_t mlv_snd_file_num = UINT16_MAX;
+/* frames issued to mlv_lite for writing */
+static volatile uint32_t mlv_snd_frames_queued = 0;
+/* frames written to previous chunks */
+static volatile uint32_t mlv_snd_frames_saved = 0;
+
+
 static uint32_t mlv_snd_rates[] = { 48000, 44100, 22050, 11025, 8000 };
 #define MLV_SND_RATE_TEXT "48kHz", "44.1kHz", "22kHz", "11kHz", "8kHz"
 
@@ -91,13 +103,12 @@
 audio_data_t *mlv_snd_next_buffer = NULL;
 
 #define MLV_SND_STATE_IDLE                   0  /* waiting for action, set by writer task upon exit */
-#define MLV_SND_STATE_PREPARE                1  /* recording was started, set by mlv_snd_start() */
-#define MLV_SND_STATE_READY                  2  /* buffers etc are set up, set by mlv_snd_alloc_buffers() */
-#define MLV_SND_STATE_SOUND_RUNNING          3  /* ASIF sound recording was started, set by mlv_snd_vsync() */
-#define MLV_SND_STATE_SOUND_STOPPING         4  /* stop audio recording, set by mlv_snd_stop() */
-#define MLV_SND_STATE_SOUND_STOP_ASIF        5  /* waiting for ASIF to process its last buffer, set by mlv_snd_asif_in_cbr() */
-#define MLV_SND_STATE_SOUND_STOP_TASK        6  /* waiting for thread to stop, set by mlv_snd_asif_in_cbr() */
-#define MLV_SND_STATE_SOUND_STOPPED          7  /* all threads and stuff is stopped, finish cleanup, set by task */
+#define MLV_SND_STATE_READY                  1  /* buffers etc are set up, set by mlv_snd_cbr_starting() */
+#define MLV_SND_STATE_SOUND_RUNNING          2  /* ASIF sound recording was started, set by mlv_snd_vsync() */
+#define MLV_SND_STATE_SOUND_STOPPING         3  /* stop audio recording, set by mlv_snd_stop() */
+#define MLV_SND_STATE_SOUND_STOP_ASIF        4  /* waiting for ASIF to process its last buffer, set by mlv_snd_asif_in_cbr() */
+#define MLV_SND_STATE_SOUND_STOP_TASK        5  /* waiting for thread to stop, set by mlv_snd_asif_in_cbr() */
+#define MLV_SND_STATE_SOUND_STOPPED          6  /* all threads and stuff is stopped, finish cleanup, set by task */
 
 static uint32_t mlv_snd_state = MLV_SND_STATE_IDLE;
 
@@ -192,29 +203,32 @@
         audio_data_t *entry = NULL;
         if(msg_queue_receive(queue, &entry, 10))
         {
-            trace_write(trace_ctx, "mlv_snd_flush_entries: msg_queue_receive(queue, ) failed");
+            trace_write(trace_ctx, "mlv_snd_flush_entries: msg_queue_receive() failed");
             return;
         }
     
-        trace_write(trace_ctx, "mlv_snd_flush_entries: entry is MLV slot");
         mlv_audf_hdr_t *hdr = (mlv_audf_hdr_t *)entry->mlv_slot_buffer;
         
         if(clear)
         {
-            trace_write(trace_ctx, "mlv_snd_flush_entries: NULL slot %d entry", entry->mlv_slot_id);
+            trace_write(trace_ctx, "mlv_snd_flush_entries:   NULL slot %d entry", entry->mlv_slot_id);
             mlv_set_type((mlv_hdr_t *)hdr, "NULL");
+            hdr->timestamp = 0;
         }
         else
         {
-            trace_write(trace_ctx, "mlv_snd_flush_entries: data %d entry for frame #%d", entry->mlv_slot_id, entry->frameNumber);
+            trace_write(trace_ctx, "mlv_snd_flush_entries:   data %d entry for frame #%d", entry->mlv_slot_id, entry->frameNumber);
             mlv_set_type((mlv_hdr_t *)hdr, "AUDF");
             hdr->frameNumber = entry->frameNumber;
             mlv_rec_set_rel_timestamp((mlv_hdr_t*)hdr, entry->timestamp);
+           
+            /* set the highest frame number for updating header later */
+            mlv_snd_frames_queued = MAX(mlv_snd_frames_queued, entry->frameNumber + 1);
         }
         
         if(entry->mlv_slot_end)
         {
-            trace_write(trace_ctx, "mlv_snd_flush_entries: entry is MLV slot %d (last buffer, so release)", entry->mlv_slot_id);
+            trace_write(trace_ctx, "mlv_snd_flush_entries:   last entry of slot %d -> release for writing", entry->mlv_slot_id);
             mlv_rec_release_slot(entry->mlv_slot_id, 1);
         }
         free(entry);
@@ -254,8 +268,6 @@
     mlv_snd_flush_entries(mlv_snd_buffers_done, 0);
     trace_write(trace_ctx, "mlv_snd_stop: flush mlv_snd_buffers_empty");
     mlv_snd_flush_entries(mlv_snd_buffers_empty, 1);
-
-    mlv_snd_state = MLV_SND_STATE_IDLE;
 }
 
 static void mlv_snd_queue_slot()
@@ -280,7 +292,7 @@
     }
     
     /* make sure that there is still place for a NULL block */
-    while((used + block_size + sizeof(mlv_hdr_t) < size) && (queued < 128))
+    while((used + block_size + sizeof(mlv_hdr_t) < size) && (queued < MLV_SND_BLOCKS_PER_SLOT))
     {
         /* setup AUDF header for that block */
         mlv_audf_hdr_t *hdr = (mlv_audf_hdr_t *)((uint32_t)address + used);
@@ -290,9 +302,9 @@
         
         mlv_set_type((mlv_hdr_t *)hdr, "NULL");
         hdr->blockSize = block_size;
-        hdr->frameNumber = 0xFFFFFFFF;
+        hdr->frameNumber = 0;
         hdr->frameSpace = hdr_size - sizeof(mlv_audf_hdr_t);
-        hdr->timestamp = 0xFFFFFFFFFFFFFFFF;
+        hdr->timestamp = 0;
         
         /* store information about the buffer in the according queue entry */
         audio_data_t *entry = malloc(sizeof(audio_data_t));
@@ -308,7 +320,7 @@
         entry->mlv_slot_end = 0;
         
         /* check if this was the last frame and set end flag if so */
-        if((used + block_size + sizeof(mlv_hdr_t) >= size) || (queued >= 128))
+        if((used + block_size + sizeof(mlv_hdr_t) >= size) || (queued >= (MLV_SND_BLOCKS_PER_SLOT - 1)))
         {
             /* this tells the writer task that the buffer is filled with that entry being done and can be committed */
             entry->mlv_slot_end = 1;
@@ -323,13 +335,12 @@
     
     mlv_set_type((mlv_hdr_t *)hdr, "NULL");
     hdr->blockSize = size - used;
-    hdr->timestamp = 0xFFFFFFFFFFFFFFFF;
+    hdr->timestamp = 0;
 }
 
 
 static void mlv_snd_prepare_audio()
 {
-    mlv_snd_frame_number = 0;
     mlv_snd_in_sample_rate = mlv_snd_rates[mlv_snd_rate_sel];
 
     /* some models may need this */
@@ -350,11 +361,10 @@
     mlv_snd_in_buffer_size = (mlv_snd_in_sample_rate * (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels) / fps;
     trace_write(trace_ctx, "mlv_snd_alloc_buffers: mlv_snd_in_buffer_size = %d", mlv_snd_in_buffer_size);
     
-    mlv_snd_queue_slot();
-    mlv_snd_queue_slot();
-
-    /* now everything is ready to fire - real output activation happens as soon mlv_snd_running is set to 1 and mlv_snd_vsync() gets called */
-    mlv_snd_state = MLV_SND_STATE_READY;
+    for(int slot = 0; slot < MLV_SND_SLOTS; slot++)
+    {
+        mlv_snd_queue_slot();
+    }
 }
 
 static void mlv_snd_writer(int unused)
@@ -410,6 +420,7 @@
                 {
                     trace_write(trace_ctx, "   --> WRITER: entry is MLV slot %d (last buffer, so release)", buffer->mlv_slot_id);
                     mlv_rec_release_slot(buffer->mlv_slot_id, 1);
+                    mlv_snd_frames_queued = hdr->frameNumber + 1;
                     mlv_snd_queue_slot();
                 }
                 free(buffer);
@@ -430,15 +441,35 @@
     {
         char filename[] = "mlv_snd.txt";
         trace_ctx = trace_start("mlv_snd", filename);
-        trace_format(trace_ctx, TRACE_FMT_TIME_REL | TRACE_FMT_COMMENT, ' ');
+        trace_format(trace_ctx, TRACE_FMT_TIME_REL | TRACE_FMT_COMMENT | TRACE_FMT_TASK_ID | TRACE_FMT_TASK_NAME, ' ');
     }
 
     trace_write(trace_ctx, "mlv_snd_start: starting");
     
     mlv_snd_prepare_audio();
     task_create("mlv_snd", 0x16, 0x1000, mlv_snd_writer, NULL);
+}
+
+void mlv_fill_wavi(mlv_wavi_hdr_t *hdr, uint64_t start_timestamp)
+{
+    mlv_set_type((mlv_hdr_t*)hdr, "WAVI");
+    hdr->blockSize = sizeof(mlv_wavi_hdr_t);
+    mlv_set_timestamp((mlv_hdr_t *)hdr, start_timestamp);
+
+    if(!mlv_snd_enabled)
+    {
+        /* not recording sound, don't trick MLV decoders :) */
+        mlv_set_type((mlv_hdr_t*)hdr, "NULL");
+        return;
+    }
     
-    mlv_snd_state = MLV_SND_STATE_PREPARE;
+    /* this part is compatible to RIFF WAVE/fmt header */
+    hdr->format = 1;
+    hdr->channels = mlv_snd_in_channels;
+    hdr->samplingRate = mlv_snd_in_sample_rate;
+    hdr->bytesPerSecond = mlv_snd_in_sample_rate * (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels;
+    hdr->blockAlign = (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels;
+    hdr->bitsPerSample = mlv_snd_in_bits_per_sample;
 }
 
 static void mlv_snd_queue_wavi()
@@ -448,72 +479,159 @@
     /* queue an WAVI block that contains information about the audio format */
     mlv_wavi_hdr_t *hdr = malloc(sizeof(mlv_wavi_hdr_t));
     
-    mlv_set_type((mlv_hdr_t*)hdr, "WAVI");
-    hdr->blockSize = sizeof(mlv_wavi_hdr_t);
-    mlv_rec_set_rel_timestamp((mlv_hdr_t*)hdr, get_us_clock());
-   
-    /* this part is compatible to RIFF WAVE/fmt header */
-    hdr->format = 1;
-    hdr->channels = mlv_snd_in_channels;
-    hdr->samplingRate = mlv_snd_in_sample_rate;
-    hdr->bytesPerSecond = mlv_snd_in_sample_rate * (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels;
-    hdr->blockAlign = (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels;
-    hdr->bitsPerSample = mlv_snd_in_bits_per_sample;
+    mlv_fill_wavi(hdr, get_us_clock());
     
     mlv_rec_queue_block((mlv_hdr_t *)hdr);
 }
 
-/* public functions for raw_rec */
-uint32_t raw_rec_cbr_starting()
+static void mlv_snd_cbr_starting(uint32_t event, void *ctx, mlv_hdr_t *hdr)
 {
     if(!mlv_snd_enabled)
     {
-        return 0;
+        return;
     }
     
+    if(mlv_snd_state != MLV_SND_STATE_IDLE)
+    {
+        return;
+    }
+   
+    /* recording is about to start, everything was set up there, now it is our turn */
+    trace_write(trace_ctx, "mlv_snd_cbr_starting: starting mlv_snd");
+    mlv_snd_start();
+    mlv_snd_queue_wavi();
+    mlv_snd_alloc_buffers();
+   
+    /* reset all variables first */
+    mlv_snd_file_num = UINT16_MAX;
+    mlv_snd_frame_number = 0;
+    mlv_snd_frames_queued = 0;
+    mlv_snd_frames_saved = 0;
+   
+
+    mlv_snd_state = MLV_SND_STATE_READY;
+}
+
+/* will get called from mlv_lite's vsync hook */
+static void mlv_snd_cbr_started(uint32_t event, void *ctx, mlv_hdr_t *hdr)
+{
+    /* the first time we get called from vsync, start recording */
+    if(mlv_snd_state != MLV_SND_STATE_READY)
+    {
+        return;
+    }
+   
+    /* "delaying audio" in the video timeline means to skip video frames */
+    mlv_rec_skip_frames(mlv_snd_vsync_delay);
+   
+    /* fetch buffers to start recording */
+    uint32_t msgs = 0;
+    msg_queue_count(mlv_snd_buffers_empty, &msgs);
+   
+    if(msgs < 2)
+    {
+        trace_write(trace_ctx, "mlv_snd_cbr_started: fatal error, no buffers");
+        bmp_printf(FONT(FONT_MED, COLOR_RED, COLOR_BLACK), 10, 130, "fatal: no buffers");
+        beep();
+        return;
+    }
+   
+    trace_write(trace_ctx, "mlv_snd_cbr_started: starting audio");
+   
+    /* get two buffers and queue them to ASIF */
+    mlv_snd_current_buffer = NULL;
+    mlv_snd_next_buffer = NULL;
+   
+    msg_queue_receive(mlv_snd_buffers_empty, &mlv_snd_current_buffer, 10);
+    msg_queue_receive(mlv_snd_buffers_empty, &mlv_snd_next_buffer, 10);
+   
+    if(!mlv_snd_current_buffer || !mlv_snd_next_buffer)
+    {
+        trace_write(trace_ctx, "mlv_snd_cbr_started: fatal error, no buffers");
+        bmp_printf(FONT(FONT_MED, COLOR_RED, COLOR_BLACK), 10, 130, "fatal: no buffers");
+        beep();
+        return;
+    }
+
+    audio_configure(1);
+    StartASIFDMAADC(mlv_snd_current_buffer->data, mlv_snd_current_buffer->length, mlv_snd_next_buffer->data, mlv_snd_next_buffer->length, mlv_snd_asif_in_cbr, 0);
+   
+    /* the current one will get filled right now */
+    mlv_snd_current_buffer->timestamp = get_us_clock();
+    trace_write(trace_ctx, "mlv_snd_cbr_started: starting audio DONE");
+   
+    mlv_snd_state = MLV_SND_STATE_SOUND_RUNNING;
+}
+
+static void mlv_snd_cbr_stopping(uint32_t event, void *ctx, mlv_hdr_t *hdr)
+{
+    if(mlv_snd_state != MLV_SND_STATE_SOUND_RUNNING)
+    {
+        return;
+    }
+   
+    trace_write(trace_ctx, "mlv_snd_cbr_stopping: stopping");
+    mlv_snd_stop();
+
+    mlv_snd_state = MLV_SND_STATE_IDLE;
+}
+
+static void mlv_snd_cbr_stopped(uint32_t event, void *ctx, mlv_hdr_t *hdr)
+{
     if(mlv_snd_state == MLV_SND_STATE_IDLE)
     {
-        trace_write(trace_ctx, "raw_rec_cbr_starting: starting mlv_snd");
-        mlv_snd_rec_active = 1;
-        mlv_snd_start();
+        return;
     }
     
-    return 0;
+    trace_write(trace_ctx, "mlv_snd_cbr_stopped: seems recording aborted during setup");
+    mlv_snd_stop();
+    mlv_snd_state = MLV_SND_STATE_IDLE;
 }
 
-uint32_t raw_rec_cbr_started()
+
+static void mlv_snd_cbr_mlv_block(uint32_t event, void *ctx, mlv_hdr_t *hdr)
 {
-    if(mlv_snd_state == MLV_SND_STATE_PREPARE)
-    {
-        trace_write(trace_ctx, "raw_rec_cbr_started: allocating buffers");
-        mlv_snd_alloc_buffers();
-        mlv_snd_queue_wavi();
-    }
-    return 0;
-}
-
-uint32_t raw_rec_cbr_stopping()
-{
-    if(mlv_snd_state != MLV_SND_STATE_IDLE)
-    {
-        trace_write(trace_ctx, "raw_rec_cbr_stopping: stopping");
-        mlv_snd_stop();
-        mlv_snd_rec_active = 0;
-    }
-    return 0;
-}
-
-uint32_t raw_rec_cbr_mlv_block(mlv_hdr_t *hdr)
-{
-    if(!memcmp(hdr->blockType, "MLVI", 4))
+    if(hdr && !memcmp(hdr->blockType, "MLVI", 4))
     {
         mlv_file_hdr_t *file_hdr = (mlv_file_hdr_t *)hdr;
         
-        /* this block is filled on recording start and when the block gets updates on recording end */
-        file_hdr->audioClass = 1; /* 0=none, 1=WAV */
-        file_hdr->audioFrameCount = mlv_snd_frame_number;
+        uint16_t file_num = file_hdr->fileNum;
+       
+        trace_write(trace_ctx, "mlv_snd_cbr_mlv_block: called for file %d", file_num);
+       
+        /* the MLV block is filled on recording start and when the block gets updates on recording end */
+        if(mlv_snd_state == MLV_SND_STATE_READY)
+        {
+            trace_write(trace_ctx, "mlv_snd_cbr_mlv_block: first chunk");
+            /* first chunk's header is being written, save the number of frames in previous chunks */
+            file_hdr->audioClass = 1; /* 0=none, 1=WAV */
+            file_hdr->audioFrameCount = 0;
+        }
+        else if(file_num == mlv_snd_file_num)
+        {
+            /* block gets updated, capture AUDF count */
+            uint32_t queued = mlv_snd_frames_queued;
+           
+            file_hdr->audioFrameCount = queued - mlv_snd_frames_saved;
+           
+            trace_write(trace_ctx, "mlv_snd_cbr_mlv_block: update: q:%d, s:%d", queued, mlv_snd_frames_saved);
+           
+            mlv_snd_frames_saved = queued;
+        }
+        else if(file_num > mlv_snd_file_num)
+        {
+            trace_write(trace_ctx, "mlv_snd_cbr_mlv_block: next");
+            /* next chunk's header is being written */
+            file_hdr->audioClass = 1; /* 0=none, 1=WAV */
+            file_hdr->audioFrameCount = 0;
+        }
+        else
+        {
+            /* suddenly file_num goes back? unexpected, do not tamper with */
+        }
+       
+        mlv_snd_file_num = file_num;
     }
-    return 0;
 }
 
 static void mlv_snd_trace_buf(char *caption, uint8_t *buffer, uint32_t length)
@@ -531,77 +649,42 @@
 }
 
 
-static unsigned int mlv_snd_vsync(unsigned int unused)
-{
-    if(!mlv_snd_enabled)
-    {
-        return 0;
-    }
-   
-    if(mlv_snd_state != MLV_SND_STATE_READY)
-    {
-        return 0;
-    }
-   
-    /* in running mode, start audio recording here */
-    uint32_t msgs = 0;
-    msg_queue_count(mlv_snd_buffers_empty, &msgs);
-   
-    if(msgs >= 2)
-    {
-        trace_write(trace_ctx, "mlv_snd_vsync: starting audio");
-       
-        /* get two buffers and queue them to ASIF */
-        mlv_snd_current_buffer = NULL;
-        mlv_snd_next_buffer = NULL;
-       
-        msg_queue_receive(mlv_snd_buffers_empty, &mlv_snd_current_buffer, 10);
-        msg_queue_receive(mlv_snd_buffers_empty, &mlv_snd_next_buffer, 10);
-       
-        if(mlv_snd_current_buffer && mlv_snd_next_buffer)
-        {
-            mlv_snd_state = MLV_SND_STATE_SOUND_RUNNING;
-       
-            audio_configure(1);
-            StartASIFDMAADC(mlv_snd_current_buffer->data, mlv_snd_current_buffer->length, mlv_snd_next_buffer->data, mlv_snd_next_buffer->length, mlv_snd_asif_in_cbr, 0);
-           
-            /* the current one will get filled right now */
-            mlv_snd_current_buffer->timestamp = get_us_clock();
-            trace_write(trace_ctx, "mlv_snd_vsync: starting audio DONE");
-        }
-        else
-        {
-            trace_write(trace_ctx, "mlv_snd_vsync: msg_queue_receive(mlv_snd_buffers_empty, ...) failed, retry next time");
-        }
-    }
-   
-    return 0;
-}
-
 static struct menu_entry mlv_snd_menu[] =
 {
     {
-        .name = "MLV Sound",
-        .priv = &mlv_snd_enabled,
-        .max = 1,
-        .help = "Enable sound recording for MLV.",
-        .submenu_width = 710,
-        .children = (struct menu_entry[])
+        .name       = "Sound recording",
+        .select     = menu_open_submenu,
+        .priv       = &mlv_snd_enabled,
+        .help       = "Sound recording options provided by mlv_snd.",
+        .children   = (struct menu_entry[])
         {
             {
-                .name = "Sampling rate",
-                .priv = &mlv_snd_rate_sel,
-                .min = 0,
-                .max = COUNT(mlv_snd_rates)-1,
-                .choices = CHOICES(MLV_SND_RATE_TEXT),
-                .help = "Select your sampling rate.",
+                .name       = "Enable sound",
+                .priv       = &mlv_snd_enabled,
+                .max        = 1,
+                .help       = "[mlv_snd] Enable sound recording for MLV.",
             },
             {
-                .name = "Trace output",
-                .priv = &mlv_snd_enable_tracing,
+                .name       = "Sampling rate",
+                .priv       = &mlv_snd_rate_sel,
+                .min        = 0,
+                .max        = COUNT(mlv_snd_rates)-1,
+                .choices    = CHOICES(MLV_SND_RATE_TEXT),
+                .help       = "[mlv_snd] Select your sampling rate.",
+            },
+            {
+                .name = "Audio delay",
+                .priv = &mlv_snd_vsync_delay,
                 .min = 0,
-                .max = 1,
-                .help = "Enable log file tracing. Needs camera restart.",
+                .max = 32,
+                .help = "Delay the audio that many frames. (experimental)",
+            },
+            {
+                .name       = "Trace output",
+                .priv       = &mlv_snd_enable_tracing,
+                .min        = 0,
+                .max        = 1,
+                .help       = "[mlv_snd] Enable log file tracing. Needs camera restart.",
             },
             MENU_EOL,
         },
@@ -619,15 +702,32 @@
     //}
     
     trace_write(trace_ctx, "mlv_snd_init: init queues");
-    mlv_snd_buffers_empty = (struct msg_queue *) msg_queue_create("mlv_snd_buffers_empty", 300);
-    mlv_snd_buffers_done = (struct msg_queue *) msg_queue_create("mlv_snd_buffers_done", 300);
+    mlv_snd_buffers_empty = (struct msg_queue *) msg_queue_create("mlv_snd_buffers_empty", MLV_SND_BLOCKS_PER_SLOT * MLV_SND_SLOTS);
+    mlv_snd_buffers_done = (struct msg_queue *) msg_queue_create("mlv_snd_buffers_done", MLV_SND_BLOCKS_PER_SLOT * MLV_SND_SLOTS);
+
+    /* will the same menu work in both submenus? probably not */
+    if (menu_get_value_from_script("Movie", "RAW video") != INT_MIN)
+    {
+        menu_add("RAW video", mlv_snd_menu, COUNT(mlv_snd_menu));
+    }
+    else if (menu_get_value_from_script("Movie", "RAW video (MLV)") != INT_MIN)
+    {
+        menu_add("RAW video (MLV)", mlv_snd_menu, COUNT(mlv_snd_menu));
+    }
+
+    trace_write(trace_ctx, "mlv_snd_init: done");
     
-    menu_add("Audio", mlv_snd_menu, COUNT(mlv_snd_menu));
-    trace_write(trace_ctx, "mlv_snd_init: done");
+    /* register callbacks */
+    mlv_rec_register_cbr(MLV_REC_EVENT_STARTING, &mlv_snd_cbr_starting, NULL);
+    mlv_rec_register_cbr(MLV_REC_EVENT_STARTED, &mlv_snd_cbr_started, NULL);
+    mlv_rec_register_cbr(MLV_REC_EVENT_STOPPING, &mlv_snd_cbr_stopping, NULL);
+    mlv_rec_register_cbr(MLV_REC_EVENT_STOPPED, &mlv_snd_cbr_stopped, NULL);
+    mlv_rec_register_cbr(MLV_REC_EVENT_BLOCK, &mlv_snd_cbr_mlv_block, NULL);
     
     return 0;
 }
 
+
 static unsigned int mlv_snd_deinit()
 {
     if(trace_ctx != TRACE_ERROR)
@@ -644,7 +744,6 @@
 MODULE_INFO_END()
 
 MODULE_CBRS_START()
-    MODULE_CBR(CBR_VSYNC, mlv_snd_vsync, 0)
     MODULE_NAMED_CBR("snd_rec_enabled", mlv_snd_snd_rec_cbr)
 MODULE_CBRS_END()
 
@@ -654,4 +753,5 @@
     MODULE_CONFIG(mlv_snd_in_bits_per_sample)
     MODULE_CONFIG(mlv_snd_rate_sel)
     MODULE_CONFIG(mlv_snd_in_sample_rate)
+    MODULE_CONFIG(mlv_snd_vsync_delay)
 MODULE_CONFIGS_END()
diff --git a/modules/trace/trace.c b/modules/trace/trace.c
--- a/modules/trace/trace.c
+++ b/modules/trace/trace.c
@@ -366,6 +366,27 @@
     {
         trace_write_timestamp(ctx, TRACE_FMT_TIME_DATE, tsc, linebuffer, &linebuffer_pos);
     }
+    if(ctx->format & TRACE_FMT_TASK_ID)
+    {
+        char tmp[32];
+       
+        linebuffer[linebuffer_pos++] = ' ';
+        snprintf(tmp, sizeof(tmp), "%d", (current_task->taskId & 0xFF));
+        memcpy(&linebuffer[linebuffer_pos], tmp, strlen(tmp));
+        linebuffer_pos += strlen(tmp);
+        linebuffer[linebuffer_pos++] = ' ';
+        linebuffer[linebuffer_pos] = '\000';
+    }
+    if(ctx->format & TRACE_FMT_TASK_NAME)
+    {
+        char *name = get_current_task_name();
+       
+        linebuffer[linebuffer_pos++] = ' ';
+        memcpy(&linebuffer[linebuffer_pos], name, strlen(name));
+        linebuffer_pos += strlen(name);
+        linebuffer[linebuffer_pos++] = ' ';
+        linebuffer[linebuffer_pos] = '\000';
+    }
     if(ctx->format & TRACE_FMT_COMMENT)
     {
         linebuffer[linebuffer_pos++] = ' ';
diff --git a/modules/trace/trace.h b/modules/trace/trace.h
--- a/modules/trace/trace.h
+++ b/modules/trace/trace.h
@@ -19,6 +19,8 @@
 #define TRACE_FMT_TIME_REL        0x0010 /* write the time since start as hh:mm:ss.msec*/
 #define TRACE_FMT_TIME_DELTA      0x0020 /* write the relative time as hh:mm:ss.msec since last entry*/
 #define TRACE_FMT_TIME_DATE       0x0040 /* write the time of day */
+#define TRACE_FMT_TASK_ID         0x0080 /* write the task id */
+#define TRACE_FMT_TASK_NAME       0x0100 /* write the task name */
 #define TRACE_FMT_COMMENT         0x1000 /* headers are C like comments */
 
 #define TRACE_FMT_META            0x0100 /* on start and stop write some metadata (e.g. day, time, ...) */

Hope this is what you was looking for :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 14, 2018, 01:50:52 AM
Different time zone here and I was working today.

@aprofiti - You have a good understanding of this. I only got the crop_rec_4k merge barely limping along. It would be great if you could create a branch on your fork and point us to it. Of course eventually make a pull request on one of the experimental branches in the main repository.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 14, 2018, 07:02:36 PM
@aprofiti - You have a good understanding of this. I only got the crop_rec_4k merge barely limping along. It would be great if you could create a branch on your fork and point us to it. Of course eventually make a pull request on one of the experimental branches in the main repository.
https://bitbucket.org/aprofiti/magic-lantern/pull-requests/1/crop-rec-4k-mlv-snd-elns/diff (https://bitbucket.org/aprofiti/magic-lantern/pull-requests/1/crop-rec-4k-mlv-snd-elns/diff)

Initially I tried to merge lua_fix into manual_lens_info to get fix for memory and latest updates, then merged into crop_rec_4k_mlv_snd but I was getting a lot of conflicts due to lua_fix not being merged into crop_rec_4k from some times...

So I started again from scratch and merged manual_lens_info directly into crop_rec and started to solve the conflicts we already managed in Dfort branch; also I have to check manually a lot diff introduces by merge tool, as it was considering manual_lens_info as most updated between two and picking automatically a lot of diff from it...

I noticed new conflicts from those files:
src/raw.c
src/property.c
src/menu.c
src/focus.c
and probably others I don't remember

I need someone who understand better than me (never worked on it) what was introduced into crop_rec_4k branch and review the PR.

Hoping g3ggo have time to dig into it and report.
@a1ex can you have a look?

Waiting for you guys, as I can't test this PR because my camera is not supported
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 14, 2018, 10:53:31 PM
Waiting for you guys, as I can't test this PR because my camera is not supported

Tried your new branch but it doesn't bring up the manual lens info menu when starting the camera without a lens mounted. It also shows this message when starting up the camera with the lens.lua script on Autorun mode.

(https://farm2.staticflickr.com/1782/42505616175_bdb77e3b88.jpg) (https://flic.kr/p/27L5oPi)

My attempt does bring up the menu but mlv_lite isn't working.  :-[
[EDIT] However, silent MLV using mlv_lite does work as does xml with CR2  :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 15, 2018, 09:08:50 PM
Wow, this is getting messy but some good points are being raised.

Much of the conversation is going on a "fake" pull request I made to see if we could merge the changes needed to get the manual lens lua script working on the crop_rec_4k branch:

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/22/elns-for-crop_rec_4k/diff

aprofiti did a similar pull request on his repository against the crop_rec_4k_mlv_snd branch:

https://bitbucket.org/aprofiti/magic-lantern/pull-requests/1/crop-rec-4k-mlv-snd-elns/diff

However, the conversation is continuing on my repository so I merged my changes so far while trying to follow the conversation and created a new pull request to see the changes against the crop_rec_4k_mlv_snd branch:

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/23/add-elns-to-crop_rec_4k_mlv_snd-branch/diff

Still very much a work in progress.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 16, 2018, 02:11:09 AM
Will check it out soon, good progress!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 16, 2018, 03:04:18 PM
Noticed that simple silent pictures (MLV option) is broken with my manual_lens_info -> crop_rec_4k_mlv_snd merge attempt.

This is with only the silent module with either mlv_lite or mlv_rec active -- the lua module is not active in this case just to isolate the problem:

(https://farm1.staticflickr.com/926/43397382232_7a7597bf7c.jpg) (https://flic.kr/p/297SVN9)

FRSP seems to be fine.

Resolving this issue will probably get us a step closer to getting ELNS (extended lens metadata?) working on the crop_rec_4k branch.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 16, 2018, 05:51:35 PM
I just tested the branch from dfort and the results are good for video.

MLVLite compressed + sound + lens metadata works
crop recording also works.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 16, 2018, 07:11:04 PM
Hum--maybe it is because I'm testing it on the EOSM?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 16, 2018, 07:17:12 PM
I don't normally use silent pictures, so I did not test for that.
Video mlvlite functions seems to work normal.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 16, 2018, 07:39:45 PM
Resolving this issue will probably get us a step closer to getting ELNS (extended lens metadata?) working on the crop_rec_4k branch.
Using the revised version of ELNS block, I can't get silent pictures works, not simple neither FRSP.
Mlv video files are created and can be viewed from mlv_play when produced with mlv_rec.
No ELNS metadata is written to file....
 
Silent pictures code prints that error when block size doesn't match or can't write MLVI header. As example It get called from those line:
Code: [Select]
if (FIO_WriteFile(save_file, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) goto write_error;
or
if (!silent_write_mlv_chunk_headers(save_file, raw_info, 0)) goto write_error;

Yesterday I got an assert triggered with mlv_rec:
Code: [Select]
ASSERT((current_hdr->blockSize > 0) && (current_hdr->blockSize < 0x20000000));

I suspect that is an issues when calculating or checking block/header size... I remember that some time block size was printed as a negative number for some unknown reason...
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 16, 2018, 07:51:14 PM
Tested MLV rec, and this does not work.

The screen displays :
FAILED 'cr'. queued : failed
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 16, 2018, 08:51:30 PM
if I comment this line of codee:
Code: [Select]
if (FIO_WriteFile(save_file, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) goto write_error;     //silent.c
if (FIO_WriteFile(f, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) return 0;                     //mlv_lite.c
I get silent and mlv_lite to works, I can review mlv from pc and no ELNS block inside

Don't know why mlv_rec is not working anymore...

There should be something wrong with this code block:
Code: [Select]
void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp){
    /* Calculate total block length: header + fixed data + variable lensName string */
    int string_length = strlen(lens_info.name);
    int block_length = (string_length + sizeof(mlv_elns_hdr_t) + 3) & ~3;                    //Maybe this is causing troubles

    mlv_elns_hdr_t *header = malloc(block_length);

    /* prepare header */
    mlv_set_type((mlv_hdr_t *)header, "ELNS");
    mlv_set_timestamp((mlv_hdr_t *)header, start_timestamp);
    header->blockSize = block_length;
    header->length = string_length;

    header->focalLengthMin = lens_info.lens_focal_min;
    header->focalLengthMax = lens_info.lens_focal_max;
    header->apertureMin = RAW2VALUE(aperture, lens_info.raw_aperture_min) / 10.0;
    header->apertureMax = RAW2VALUE(aperture, lens_info.raw_aperture_max) / 10.0;
    header->version = lens_info.lens_version;
    header->extenderInfo = lens_info.lens_extender;
    header->capabilities = lens_info.lens_capabilities;
    header->chipped = lens_info.lens_exists;

    /* Store lensName string at the end of mlv_elns_hdr_t */
    char *lens_hdr_payload = (char *)&header[1];                  //Is this ok? Doesn't return a pointer right after header ending?
    strcpy(lens_hdr_payload, lens_info.name);                     //Will overwrite wrong memory region if above pointer is wrong

    /* update block with new values */
    hdr = header;
}
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 17, 2018, 02:42:04 AM
@aprofiti - confirmed your findings but am at a loss how to fix it.

lens_info.name is used for both mlv_fill_lens and mlv_fill_elns. Isn't there some sort of confusion there?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 23, 2018, 12:40:40 PM
I made a 5dMk3 firmware 1.2.3 build for testing:

https://bitbucket.org/larssteenhoff/magic-lantern-build/downloads/magiclantern-Nightly.2018Jul23.5D3123.zip

It has mlv lite + snd + lens info

(I removed the older mlv rec module from the zip so it can't be used because it's broken )

Build from dfort branch
https://bitbucket.org/daniel_fort/magic-lantern/branch/crop_rec_4k_ELNS

I changed one timing in src/shoot.c to do a quicker 10x zoom on half shutter.  Because I don't use an auto focus lens I prefer to use half shutter for focus check.

Code: [Select]
             #ifdef CONFIG_ZOOM_HALFSHUTTER_UILOCK
-            msleep(500);
+            msleep(100);

14 bit compressed with sound works and it gives the lens info to photoshop for automatic lens corrections.

(Except the Zeiss Makro-Planar T* 2/100 ZF.2)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 23, 2018, 06:07:21 PM
14 bit compressed with sound works and it gives the lens info to photoshop for automatic lens corrections.

(Except the Zeiss Makro-Planar T* 2/100 ZF.2)

The only thing that doesn't seem to be working is the ELNS metadata which will take care of that. To find out why we need ELNS go back to Reply #166 (https://www.magiclantern.fm/forum/index.php?topic=18083.msg178132#msg178132) and start reading from there. The Zeiss Makro-Planar T* 2/100 ZF.2 lens name shows properly on an XMP file (when shooting CR2 still frames) but it isn't getting into the MLV metadata. ELNS does work on the manual_lens_info branch but there are considerable difference between these branches.

My tests on the EOSM are showing either no LENS metadata (when the lens is changed with the camera turned on):

Code: [Select]
Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.839000 ms
     Name:        ''
     Serial:      '000000000' (no valid S/N)
     Focal Len:   50 mm
     Focus Dist:  0 mm
     Aperture:    f/0.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

or LENS name with some padded zeros (when starting the camera with a non-chipped lens mounted):

Code: [Select]
Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.817000 ms
     Name:        'Zeiss Makro-Planar T* 2/100 ZF.2000000000'
     Serial:      '000000000' (no valid S/N)
     Focal Len:   100 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

in all cases, there is no ELNS block in the MLV files.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on July 23, 2018, 06:24:02 PM
Yes I also go the padded zero's

Still I'm very happy how far we got,  :)

I can now at least film compressed and save half the storage and still have sound and info for the lenses that I have ( don't have the 100 makro, but wanted to see if the script works )

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on July 23, 2018, 07:04:00 PM
This is how it is supposed to work--using the manual_lens_info branch.

Code: [Select]
Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.818000 ms
     Name:        'Zeiss Makro-Planar T* 2/100 ZF.'
     Serial:      '000000000' (no valid S/N)
     Focal Len:   100 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000
Block: ELNS
  Offset: 0x000001e4
  Number: 6
    Size: 95
    Time: 0.838000 ms
     Name:                'Zeiss Makro-Planar T* 2/100 ZF.2'
     Focal Length Min:    0 mm
     Focal Length Max:    0 mm
     Aperture Min:        f/0.00
     Aperture Max:        f/0.00
     Version:             0
     Extender Info:       0x00
     Capabilities:        0x00
     Chipped:             0x00

Uh oh, just discovered a problem. Extracting DNG files using "mlv_dump --dng" and looking at the DNG metadata with exiftool:

Code: [Select]
Lens Model                      : Zeiss Makro-Planar T* 2/100 ZF.

If there is an ELNS block mlv_dump should be using it instead of the LENS block. Also tried it with MLVFS and found the same issue.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on July 23, 2018, 09:10:40 PM
ive just seen that there is some again fixed-length implementaition in manual_lens_info_64byte branch and manual_lens_info.

is there any tool yet that processes the ELNS block?
i still want the name to be dynamic - just like some other blocks are.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on July 23, 2018, 11:50:24 PM
ive just seen that there is some again fixed-length implementaition in manual_lens_info_64byte branch and manual_lens_info

We are experimenting dynamic length version on top of fake PR.
manual_lens_info_64byte was merged into manual_lens_info

is there any tool yet that processes the ELNS block?
i still want the name to be dynamic - just like some other blocks are.

Yes, mlv_dump from fake PR was modified to manage dynamic length with help from "length" field (which need to be removed as you said) to know how many chars to read and print.

Still need some help with the pointer (https://www.magiclantern.fm/forum/index.php?topic=18083.msg204239#msg204239) of extended lens name (in mlv.c)... I think it is causing troubles like writing over wrong memory location when string copy is used
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on August 03, 2018, 06:52:39 PM
Good news Guys! Managed to discover what was preventing to save .mlv files some days ago and found a fix! :)

Currently elns struct is passed by value, so mlv_fill_elns will leaves struct initialised in memory, preventing FIOWriteFile to write correct datas...

Passing elns's struct argument as a double pointer and adapting related code will makes thing works:
Code: [Select]
void mlv_fill_elns(mlv_elns_hdr_t **hdr, uint64_t start_timestamp);

Tested on silent and mlv_lite; I need to see how to adapt mlv_rec and mlv_dump in next days.
@dfort Ill'try to send you a patch to test on crop_rec_4k

Uh oh, just discovered a problem. Extracting DNG files using "mlv_dump --dng" and looking at the DNG metadata with exiftool:

Code: [Select]
Lens Model                      : Zeiss Makro-Planar T* 2/100 ZF.

If there is an ELNS block mlv_dump should be using it instead of the LENS block. Also tried it with MLVFS and found the same issue.

Found where this is handled in dng.c: added in dng_fill_header() function
It's easy to adapt it to use elns instead if present, but it needs to be able to see datas by adding in dng.h:
Code: [Select]
struct frame_info
{
     ....

    /* block headers */
    ....

    mlv_elns_hdr_t elns_hdr;

    ....
};
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 03, 2018, 07:28:46 PM
Great news!

I added the ELNS block header to dng.h and look forward to the patch.

This is needed in silent.c, mlv.h and mlv.c, right?

Code: [Select]
void mlv_fill_elns(mlv_elns_hdr_t **hdr, uint64_t start_timestamp);
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on August 05, 2018, 09:08:03 AM
Nice find!
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on August 07, 2018, 10:38:51 PM
I post the patch also here just in case someone want to try this on top of dfort's repository (https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/22/elns-for-crop_rec_4k/diff):
Code: [Select]
diff --git a/modules/mlv_lite/mlv_lite.c b/modules/mlv_lite/mlv_lite.c
--- a/modules/mlv_lite/mlv_lite.c
+++ b/modules/mlv_lite/mlv_lite.c
@@ -359,10 +359,10 @@
 static GUARDED_BY(RawRecTask)   mlv_idnt_hdr_t idnt_hdr;
 static GUARDED_BY(RawRecTask)   mlv_expo_hdr_t expo_hdr;
 static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
-static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t elns_hdr;
 static GUARDED_BY(RawRecTask)   mlv_rtci_hdr_t rtci_hdr;
 static GUARDED_BY(RawRecTask)   mlv_wbal_hdr_t wbal_hdr;
-static GUARDED_BY(LiveViewTask) mlv_vidf_hdr_t vidf_hdr;
+static GUARDED_BY(LiveViewTask) mlv_vidf_hdr_t vidf_hdr;
+static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t *elns_hdr;
 static GUARDED_BY(RawRecTask)   uint64_t mlv_start_timestamp = 0;
        GUARDED_BY(RawRecTask)   uint32_t raw_rec_trace_ctx = TRACE_ERROR;
 
@@ -3048,7 +3048,7 @@
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&idnt_hdr);
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&expo_hdr);
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&lens_hdr);
-        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)elns_hdr);
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&rtci_hdr);
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&wbal_hdr);
         fail |= mlv_write_vers_blocks(f, mlv_start_timestamp);
diff --git a/modules/mlv_rec/dng/dng.c b/modules/mlv_rec/dng/dng.c
--- a/modules/mlv_rec/dng/dng.c
+++ b/modules/mlv_rec/dng/dng.c
@@ -669,7 +669,12 @@
             {tcBaselineExposureOffset,      ttSRational,RATIONAL_ENTRY2(0, 1, header, &data_offset)},
             {tcImageDescription,            ttAscii,    STRING_ENTRY(frame_info->info_str, header, &data_offset)},
         };
-       
+
+        /* Use Lens Model from ELNS Block if present */
+        char* lens = (frame_info->elns_str != NULL)
+              && (strlen(frame_info->elns_str) > strlen(frame_info->lens_hdr.lensName))
+              ? frame_info->elns_str : frame_info->lens_hdr.lensName;
+
         struct directory_entry EXIF_IFD[EXIF_IFD_COUNT] =
         {
             {tcExposureTime,                ttRational, RATIONAL_ENTRY2((int32_t)frame_info->expo_hdr.shutterValue/1000, 1000, header, &data_offset)},
@@ -682,7 +687,7 @@
             {tcFocalPlaneXResolutionExif,   ttRational, RATIONAL_ENTRY(focal_resolution_x, header, &data_offset, 2)},
             {tcFocalPlaneYResolutionExif,   ttRational, RATIONAL_ENTRY(focal_resolution_y, header, &data_offset, 2)},
             {tcFocalPlaneResolutionUnitExif,ttShort,    1,      camera_id[current_cam].focal_unit}, //inches
-            {tcLensModelExif,               ttAscii,    STRING_ENTRY((char*)frame_info->lens_hdr.lensName, header, &data_offset)},
+            {tcLensModelExif,               ttAscii,    STRING_ENTRY(lens, header, &data_offset)},
         };
         
         /* do not put image description, if the length is zero */
diff --git a/modules/mlv_rec/dng/dng.h b/modules/mlv_rec/dng/dng.h
--- a/modules/mlv_rec/dng/dng.h
+++ b/modules/mlv_rec/dng/dng.h
@@ -76,8 +76,8 @@
     mlv_rawc_hdr_t rawc_hdr;
     mlv_expo_hdr_t expo_hdr;
     mlv_lens_hdr_t lens_hdr;
-    mlv_elns_hdr_t elns_hdr;
     mlv_wbal_hdr_t wbal_hdr;
+    char *elns_str;
     char *info_str;
 };
 
diff --git a/modules/mlv_rec/mlv.c b/modules/mlv_rec/mlv.c
--- a/modules/mlv_rec/mlv.c
+++ b/modules/mlv_rec/mlv.c
@@ -62,7 +62,7 @@
     strncpy((char *)hdr->lensSerial, buf, 32);
 }
 
-void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp)
+void mlv_fill_elns(mlv_elns_hdr_t **hdr, uint64_t start_timestamp)
 {
     /* Calculate total block length: header + fixed data + variable lensName string */
     int string_length = strlen(lens_info.name);
@@ -73,6 +73,7 @@
     mlv_set_type((mlv_hdr_t *)header, "ELNS");
     mlv_set_timestamp((mlv_hdr_t *)header, start_timestamp);
     header->blockSize = block_length;
+    /* Fill ELNS data */
     header->focalLengthMin = lens_info.lens_focal_min;
     header->focalLengthMax = lens_info.lens_focal_max;
     header->apertureMin = RAW2VALUE(aperture, lens_info.raw_aperture_min) / 10.0;
@@ -84,10 +85,10 @@
 
     /* Store lensName string at the end of mlv_elns_hdr_t */
     char *lens_hdr_payload = (char *)&header[1];
-    strcpy(lens_hdr_payload, lens_info.name);
+    snprintf(lens_hdr_payload, string_length + 1, "%s", lens_info.name);
 
     /* update block with new values */
-    hdr = header;
+    *hdr = header;
 }
 
 void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp)
diff --git a/modules/mlv_rec/mlv.h b/modules/mlv_rec/mlv.h
--- a/modules/mlv_rec/mlv.h
+++ b/modules/mlv_rec/mlv.h
@@ -176,9 +176,7 @@
     uint8_t     extenderInfo;       /* extender information, if provided by camera       */
     uint8_t     capabilities;       /* capability information, if provided by camera     */
     uint8_t     chipped;            /* when not zero, lens is communicating with camera  */
-    uint8_t     length;             /* to allow lens with name longer than 32byte        */
  /* uint8_t     lensName[variable];    full lens string, null terminated                 */
-    // TODO: Review Specs
 }  mlv_elns_hdr_t;
 
 typedef struct {
@@ -304,7 +302,7 @@
 void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
-void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
+void mlv_fill_elns(mlv_elns_hdr_t **hdr, uint64_t start_timestamp);
 void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
 void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
diff --git a/modules/mlv_rec/mlv_dump.c b/modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c
+++ b/modules/mlv_rec/mlv_dump.c
@@ -1941,7 +1941,8 @@
     memset(&rtci_info, 0x00, sizeof(mlv_rtci_hdr_t));
     memset(&rawc_info, 0x00, sizeof(mlv_rawc_hdr_t));
     memset(&main_header, 0x00, sizeof(mlv_file_hdr_t));
-
+
+    char elns_string[1024] = "";
     char info_string[1024] = "";
 
     /* this table contains the XREF chunk read from idx file, if existing */
@@ -3468,7 +3469,8 @@
                             frame_info.expo_hdr             = expo_info;
                             frame_info.lens_hdr             = lens_info;
                             frame_info.wbal_hdr             = wbal_info;
-                            frame_info.rawc_hdr             = rawc_info;
+                            frame_info.rawc_hdr             = rawc_info;
+                            frame_info.elns_str             = elns_string;
                             frame_info.info_str             = info_string;
                             frame_info.rawi_hdr.xRes        = lv_rec_footer.xRes;
                             frame_info.rawi_hdr.yRes        = lv_rec_footer.yRes;
@@ -3780,30 +3782,24 @@
             {
                 mlv_elns_hdr_t block_hdr = *(mlv_elns_hdr_t *)mlv_block;
 
-                /* get the string length and malloc a buffer for that string */
-                int str_length = block_hdr.length;
-
-                if(str_length)
+                void *payload = BYTE_OFFSET(mlv_block, sizeof(mlv_elns_hdr_t));
+                int str_length = MIN(block_hdr.blockSize - sizeof(block_hdr), sizeof(elns_string) - 1);
+
+                /* Fill lens model data for DNG processing */
+                strncpy(elns_string, payload, str_length);
+                elns_string[str_length] = '\000';
+
+                if(verbose)
                 {
-                    void *payload = BYTE_OFFSET(mlv_block, sizeof(mlv_elns_hdr_t) - 1);
-                    char *buf = malloc(str_length + 1);
-
-                    strncpy(buf, payload, str_length);
-                    buf[str_length] = '\000';
-
-                    if(verbose)
-                    {
-                        print_msg(MSG_INFO, "     Name:                '%s'\n", buf);
-                        print_msg(MSG_INFO, "     Focal Length Min:    %d mm\n", elns_info.focalLengthMin);
-                        print_msg(MSG_INFO, "     Focal Length Max:    %d mm\n", elns_info.focalLengthMax);
-                        print_msg(MSG_INFO, "     Aperture Min:        f/%.2f\n", (double)elns_info.apertureMin);
-                        print_msg(MSG_INFO, "     Aperture Max:        f/%.2f\n", (double)elns_info.apertureMax);
-                        print_msg(MSG_INFO, "     Version:             %d\n", elns_info.version);
-                        print_msg(MSG_INFO, "     Extender Info:       0x%02X\n", elns_info.extenderInfo);
-                        print_msg(MSG_INFO, "     Capabilities:        0x%02X\n", elns_info.capabilities);
-                        print_msg(MSG_INFO, "     Chipped:             0x%02X\n", elns_info.chipped);
-                    }
-                    free(buf);
+                    print_msg(MSG_INFO, "     Name:                '%s'\n", payload);
+                    print_msg(MSG_INFO, "     Focal Length Min:    %d mm\n", block_hdr.focalLengthMin);
+                    print_msg(MSG_INFO, "     Focal Length Max:    %d mm\n", block_hdr.focalLengthMax);
+                    print_msg(MSG_INFO, "     Aperture Min:        f/%.2f\n", (double)block_hdr.apertureMin);
+                    print_msg(MSG_INFO, "     Aperture Max:        f/%.2f\n", (double)block_hdr.apertureMax);
+                    print_msg(MSG_INFO, "     Version:             %d\n", block_hdr.version);
+                    print_msg(MSG_INFO, "     Extender Info:       0x%02X\n", block_hdr.extenderInfo);
+                    print_msg(MSG_INFO, "     Capabilities:        0x%02X\n", block_hdr.capabilities);
+                    print_msg(MSG_INFO, "     Chipped:             0x%02X\n", block_hdr.chipped);
                 }
             }
             else if(!memcmp(mlv_block->blockType, "ELVL", 4))
diff --git a/modules/mlv_rec/mlv_rec.c b/modules/mlv_rec/mlv_rec.c
--- a/modules/mlv_rec/mlv_rec.c
+++ b/modules/mlv_rec/mlv_rec.c
@@ -200,9 +200,9 @@
 
 static mlv_expo_hdr_t last_expo_hdr;
 static mlv_lens_hdr_t last_lens_hdr;
-static mlv_elns_hdr_t last_elns_hdr;
 static mlv_wbal_hdr_t last_wbal_hdr;
-static mlv_styl_hdr_t last_styl_hdr;
+static mlv_styl_hdr_t last_styl_hdr;
+static mlv_elns_hdr_t *last_elns_hdr;
 
 
 /* for debugging */
@@ -1535,13 +1535,13 @@
             trace_write(raw_rec_trace_ctx, "[polling_cbr] queueing INFO blocks");
             mlv_expo_hdr_t *expo_hdr = malloc(sizeof(mlv_expo_hdr_t));
             mlv_lens_hdr_t *lens_hdr = malloc(sizeof(mlv_lens_hdr_t));
+            mlv_wbal_hdr_t *wbal_hdr = malloc(sizeof(mlv_wbal_hdr_t));
             mlv_elns_hdr_t *elns_hdr = malloc(sizeof(mlv_elns_hdr_t));
-            mlv_wbal_hdr_t *wbal_hdr = malloc(sizeof(mlv_wbal_hdr_t));
 
             mlv_fill_expo(expo_hdr, mlv_start_timestamp);
             mlv_fill_lens(lens_hdr, mlv_start_timestamp);
-            mlv_fill_elns(elns_hdr, mlv_start_timestamp);
-            mlv_fill_wbal(wbal_hdr, mlv_start_timestamp);
+            mlv_fill_wbal(wbal_hdr, mlv_start_timestamp);
+            mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
 
             msg_queue_post(mlv_block_queue, (uint32_t) expo_hdr);
             msg_queue_post(mlv_block_queue, (uint32_t) lens_hdr);
@@ -2674,10 +2674,10 @@
         mlv_rtci_hdr_t rtci_hdr;
         mlv_expo_hdr_t expo_hdr;
         mlv_lens_hdr_t lens_hdr;
-        mlv_elns_hdr_t elns_hdr;
         mlv_idnt_hdr_t idnt_hdr;
         mlv_wbal_hdr_t wbal_hdr;
         mlv_styl_hdr_t styl_hdr;
+        mlv_elns_hdr_t *elns_hdr;
 
         mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
         mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
@@ -2694,12 +2694,12 @@
         idnt_hdr.timestamp = 4;
         wbal_hdr.timestamp = 5;
         styl_hdr.timestamp = 6;
-        elns_hdr.timestamp = 7;
+        elns_hdr->timestamp = 7;
 
         mlv_write_hdr(f, (mlv_hdr_t *)&rtci_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&expo_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&lens_hdr);
-        mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
+        mlv_write_hdr(f, (mlv_hdr_t *)elns_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&idnt_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&wbal_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&styl_hdr);
@@ -3211,7 +3211,7 @@
 
         mlv_expo_hdr_t old_expo = last_expo_hdr;
         mlv_lens_hdr_t old_lens = last_lens_hdr;
-        mlv_elns_hdr_t old_elns = last_elns_hdr;
+        mlv_elns_hdr_t *old_elns = last_elns_hdr;
 
         mlv_fill_expo(&last_expo_hdr, mlv_start_timestamp);
         mlv_fill_lens(&last_lens_hdr, mlv_start_timestamp);
@@ -3220,7 +3220,7 @@
         /* update timestamp for comparing content changes */
         old_expo.timestamp = last_expo_hdr.timestamp;
         old_lens.timestamp = last_lens_hdr.timestamp;
-        old_elns.timestamp = last_elns_hdr.timestamp;
+        old_elns->timestamp = last_elns_hdr->timestamp;
 
         /* write new state if something changed */
         if(memcmp(&last_expo_hdr, &old_expo, sizeof(mlv_expo_hdr_t)))
@@ -3240,8 +3240,8 @@
         /* write new state if something changed */
         if(memcmp(&last_elns_hdr, &old_elns, sizeof(mlv_elns_hdr_t)))
         {
-            mlv_hdr_t *hdr = malloc(sizeof(mlv_elns_hdr_t));
-            memcpy(hdr, &last_elns_hdr, sizeof(mlv_elns_hdr_t));
+            mlv_hdr_t *hdr = malloc(last_elns_hdr->blockSize);
+            memcpy(hdr, last_elns_hdr, last_elns_hdr->blockSize);
             msg_queue_post(mlv_block_queue, (uint32_t) hdr);
         }
     }
diff --git a/modules/silent/silent.c b/modules/silent/silent.c
--- a/modules/silent/silent.c
+++ b/modules/silent/silent.c
@@ -28,10 +28,10 @@
 extern WEAK_FUNC(ret_0) void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
-extern WEAK_FUNC(ret_0) void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0) void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
+extern WEAK_FUNC(ret_0) void mlv_fill_elns(mlv_elns_hdr_t **hdr, uint64_t start_timestamp);
 extern WEAK_FUNC(ret_0_long) uint64_t mlv_generate_guid();
 extern WEAK_FUNC(ret_0) void mlv_init_fileheader(mlv_file_hdr_t *hdr);
 extern WEAK_FUNC(ret_0) void mlv_set_type(mlv_hdr_t *hdr, char *type);
@@ -366,11 +366,11 @@
     mlv_rtci_hdr_t rtci_hdr;
     mlv_expo_hdr_t expo_hdr;
     mlv_lens_hdr_t lens_hdr;
-    mlv_elns_hdr_t elns_hdr;
     mlv_idnt_hdr_t idnt_hdr;
     mlv_wbal_hdr_t wbal_hdr;
     mlv_styl_hdr_t styl_hdr;
     mlv_vidf_hdr_t vidf_hdr;
+    mlv_elns_hdr_t *elns_hdr;
     FILE* save_file = NULL;   
     
     /* default case: use last filename */
@@ -489,7 +489,7 @@
     if (FIO_WriteFile(save_file, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) goto write_error;
-    if (FIO_WriteFile(save_file, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) goto write_error;
+    if (FIO_WriteFile(save_file, elns_hdr, elns_hdr->blockSize) != (int)elns_hdr->blockSize) goto write_error;
 
     memset(&vidf_hdr, 0, sizeof(mlv_vidf_hdr_t));
     mlv_set_type((mlv_hdr_t *)&vidf_hdr, "VIDF");
diff --git a/scripts/lib/config.lua b/scripts/lib/config.lua
--- a/scripts/lib/config.lua
+++ b/scripts/lib/config.lua
@@ -79,8 +79,26 @@
 
 end
 
+-- Load config from file if exists, otherwise create a new table in memory
 local create_internal = function(default,thisfile)
+  local filename = string.format("%s%s.lcf", dryos.config_dir.path,thisfile)
+  local cfg = config.findConfig(filename)
 
+  if cfg == nil then
+    -- Create a config from scratch
+    cfg = {}
+    cfg.filename = filename
+    cfg.default = default -- TODO: Replicate .data's structure
+    cfg.data = {}
+    -- check for existing .cfg to load
+    setmetatable(cfg,config)
+    -- load previus config from file if available
+    cfg.data = cfg:load()
+    -- add to data structure
+    table.insert(config.configs,cfg)
+  end
+
+  return cfg
 end
 
 local function recursiveLoad(m,cfg)
@@ -108,33 +126,17 @@
 ]]
 function config.create(default)
   local short_name = string.match(debug.getinfo(2,"S").short_src,"/([^/%.]+)%.[^/%.]+$")
-  local filename = string.format("%s%s.lcf", dryos.config_dir.path,short_name)
+  local cfg = create_internal(default,short_name)
 
-  local cfg = config.findConfig(filename)
-  if cfg ~= nil then
-    -- Append config data
-    for k,v in pairs(default) do
-      cfg.data[k] = v
-    end
-  else
-    -- Create a config from scratch
-    cfg = {}
-    cfg.filename = filename
-    cfg.default = default -- TODO: Replicate .data's structure
-    cfg.data = {}
-    -- check for existing .cfg to load
-    setmetatable(cfg,config)
-    cfg.data = cfg:load()
-    if cfg.data == nil then
-      -- Create a config from scratch
-      for k,v in pairs(default) do
-        cfg.data[k] = v
-      end
-    end
-    table.insert(config.configs,cfg)
-   end
+  -- Append config data
+  for k,v in pairs(default) do
+    cfg.default[k] = v
+    cfg.data[k] = v
+  end
 
-   return cfg
+  --  cfg.default = default -- TODO: Replicate .data's structure
+
+  return cfg
 end
 
 --[[---------------------------------------------------------------------------
@@ -158,37 +160,18 @@
     insertMenu(default,m)
 
     local short_name = string.match(debug.getinfo(2,"S").short_src,"/([^/%.]+)%.[^/%.]+$")
-    local filename = string.format("%s%s.lcf", dryos.config_dir.path,short_name)
+    local cfg = create_internal(default,short_name)
 
-    local cfg = config.findConfig(filename)
-    if cfg ~= nil then
-      -- Already present in config.configs, append menu
-      if cfg.data[m.name] ~= nil then
-        -- Avoid overwriting values when loading config form .cfg
-        cfg.data[m.name].menu = m
-        recursiveLoad(m,cfg.data[m.name])
-      else
-        insertMenu(cfg.data,m)
-      end
+    if cfg.data[m.name] == nil then
+      -- Create a config for menu from scratch
+      insertMenu(cfg.data,m)
     else
-      -- Create a config from scratch
-      cfg = {}
-      cfg.filename = filename
-      cfg.default = default -- TODO: Replicate .data's structure
-      cfg.data = {}
-      -- check for existing .cfg to load
-      setmetatable(cfg,config)
-      cfg.data = cfg:load()
-      if cfg.data == nil then
-        -- Create a config from scratch
-        insertMenu(cfg.data,m)
-      else
-        -- load values to menu
-        cfg.data[m.name].menu = m
-        recursiveLoad(m,cfg.data[m.name])
-      end
-      table.insert(config.configs,cfg)
-     end
+      -- Already present in config.configs, load values to menu from config
+      cfg.data[m.name].menu = m
+      recursiveLoad(m,cfg.data[m.name])
+    end
+
+    --    cfg.default = default -- TODO: Replicate .data's structure
 
     return cfg.data[m.name]
 end
@@ -213,9 +196,6 @@
 @function saving
 ]]
 function config:saving()
-local short_name = string.match(debug.getinfo(2,"S").short_src,"/([^/%.]+)%.[^/%.]+$")
-local filename = string.format("%s%s.lcf", dryos.config_dir.path,short_name)
-
   -- Copy values of each menu
   for k,v in pairs(self.data) do
     -- k -> A table representing a menu entry or a single entry of a simple config
@@ -238,14 +218,15 @@
 ]]
 function config:save()
     local f = io.open(self.filename,"w")
+    assert(f ~= nil, "Could not save config: "..self.filename)
+    -- Serialize data into a loadable format
     f:write("return ")
-    assert(f ~= nil, "Could not save config: "..self.filename)
-    config.serialize(f,self.data)
+    config.serialize(f,self.data,1)
     f:close()
 end
 
 --private
-function config.serialize(f,o)
+function config.serialize(f,o,lvl)
     if type(o) == "number" or type(o) == "boolean" then
         f:write(tostring(o))
     elseif type(o) == "string" then
@@ -254,13 +235,17 @@
         f:write("{\n")
         for k,v in pairs(o) do
           if k ~= "menu" then
-            f:write("\t[")
-            config.serialize(f,k)
+            -- Indent starting line
+            f:write(string.rep("\t", lvl))
+            f:write("[")
+            config.serialize(f,k,lvl+1)
             f:write("] = ")
-            config.serialize(f,v)
+            config.serialize(f,v,lvl+1)
             f:write(",\n")
           end
         end
+        -- Indent closing bracket
+        f:write(string.rep("\t", lvl-1))
         f:write("}")
     else
         --something we don't know how to serialize, just skip it

Tested those changes in manual_lens_info and adapted for crop_rec_4k.
Any Feedback is welcome :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 08, 2018, 01:50:49 AM
Applied the patch and it is looking much better. The ELNS Block is working but the trailing zeros in the LENS Block looks like it is still an issue.

Code: [Select]
Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.803000 ms
     Name:        'Samyang 8mm f/3.5 UMC Fish-Eye C000000000'
     Serial:      '000000000' (no valid S/N)
     Focal Len:   8 mm
     Focus Dist:  0 mm
     Aperture:    f/3.50
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000
Block: ELNS
  Offset: 0x000001e4
  Number: 6
    Size: 68
    Time: 0.988000 ms
     Name:                'Samyang 8mm f/3.5 UMC Fish-Eye CS II'
     Focal Length Min:    0 mm
     Focal Length Max:    0 mm
     Aperture Min:        f/0.00
     Aperture Max:        f/0.00
     Version:             0
     Extender Info:       0x00
     Capabilities:        0x00
     Chipped:             0x01

When a CPU lens is attached it is fine:

Code: [Select]
Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 15.291000 ms
     Name:        'EF-M11-22mm f/4-5.6 IS STM'
     Serial:      '000004904' (18692)
     Focal Len:   16 mm
     Focus Dist:  65535 mm
     Aperture:    f/8.00
     IS Mode:     0
     AF Mode:     0
     Lens ID:     0x00001033
     Flags:       0x00000000
Block: ELNS
  Offset: 0x000001e4
  Number: 6
    Size: 60
    Time: 15.476000 ms
     Name:                'EF-M11-22mm f/4-5.6 IS STM'
     Focal Length Min:    11 mm
     Focal Length Max:    22 mm
     Aperture Min:        f/4.00
     Aperture Max:        f/26.00
     Version:             65536
     Extender Info:       0xFF
     Capabilities:        0x37
     Chipped:             0x01
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on August 08, 2018, 03:32:58 AM
I think because it's printing serial number at the end of lens name, due to a missing termination character of the string... (present after first 32 char)

Two option here:
1) hardening string with a termination character in mlv.c:
Code: [Select]
diff --git a/modules/mlv_rec/mlv.c b/modules/mlv_rec/mlv.c
--- a/modules/mlv_rec/mlv.c
+++ b/modules/mlv_rec/mlv.c
@@ -58,7 +58,7 @@
     char buf[33];
     snprintf(buf, sizeof(buf), "%X%08X", (uint32_t) (lens_info.lens_serial >> 32), (uint32_t)(lens_info.lens_serial & 0xFFFFFFFF));
     
-    strncpy((char *)hdr->lensName, lens_info.name, 32);
+    snprintf((char *)hdr->lensName, 32, "%s", lens_info.name);
     strncpy((char *)hdr->lensSerial, buf, 32);
 }
As a drawback we will loose a char of the string, making space for a 31 char string without using ELNS block

2) modify mlv_dump to copy first 32 character of the string in a "buffer" and then print it instead of lens_info.lensName
Code: [Select]
diff --git a/modules/mlv_rec/mlv_dump.c b/modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c
+++ b/modules/mlv_rec/mlv_dump.c
@@ -3670,18 +3670,22 @@
 
                 if(verbose)
                 {
+                    /* Ensure lens name is null trminated to avoid printing serial number */
+                    char name[33];
+                    snprintf(name, sizeof(name), "%s", lens_info.lensName);
+
                     uint64_t serial = 0;
                     char serial_str[64];
                     char *end;
-                   
+
                     strcpy(serial_str, "no valid S/N");
                     serial = strtoull((char *)lens_info.lensSerial, &end, 16);
                     if (serial && !*end)
                     {
                         sprintf(serial_str, "%"PRIu64, serial);
                     }
-                   
-                    print_msg(MSG_INFO, "     Name:        '%s'\n", lens_info.lensName);
+
+                    print_msg(MSG_INFO, "     Name:        '%s'\n", name);
                     print_msg(MSG_INFO, "     Serial:      '%s' (%s)\n", lens_info.lensSerial, serial_str);
                     print_msg(MSG_INFO, "     Focal Len:   %d mm\n", lens_info.focalLength);
                     print_msg(MSG_INFO, "     Focus Dist:  %d mm\n", lens_info.focalDist);

Can cause similar issue when processing with other tools if they don't take care of it
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 08, 2018, 07:23:47 AM
As a drawback we will loose a char of the string, making space for a 31 char string without using ELNS block

Isn't that what we already have? Check out Reply #167 (https://www.magiclantern.fm/forum/index.php?topic=18083.msg178133#msg178133) from @dmilligan and the following couple of posts.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on August 08, 2018, 11:05:19 AM
LENS block processing is still untouched from some times (both mlv.c and mlv_dump.c); the only changes was about serial number fix made by g3gg0.

I think it was unoticed  until now or maybe it showed up after increasing size of lens_info.name to 64 byte.

In mlv 2.0 specifications, lens name is a optional null terminated string, so maybe option number 2 is to be preferred to avoid interference with other processing tools?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: g3gg0 on August 08, 2018, 11:26:43 AM
In mlv 2.0 specifications, lens name is a optional null terminated string, so maybe option number 2 is to be preferred to avoid interference with other processing tools?

absolutely. this is my recommendation.
in MLV strings dont *have to* be null terminated as you pointed out correctly.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 11, 2018, 04:37:43 PM
Applied the second option, patched mlv_dump:

Code: [Select]
Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.803000 ms
     Name:        'Samyang 8mm f/3.5 UMC Fish-Eye C'
     Serial:      '000000000' (no valid S/N)
     Focal Len:   8 mm
     Focus Dist:  0 mm
     Aperture:    f/3.50
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000
Block: ELNS
  Offset: 0x000001e4
  Number: 6
    Size: 68
    Time: 0.988000 ms
     Name:                'Samyang 8mm f/3.5 UMC Fish-Eye CS II'
     Focal Length Min:    0 mm
     Focal Length Max:    0 mm
     Aperture Min:        f/0.00
     Aperture Max:        f/0.00
     Version:             0
     Extender Info:       0x00
     Capabilities:        0x00
     Chipped:             0x01

In addition, we gained one character in the LENS block so the lens that started this whole extended lens name issue is showing up fine now.

Code: [Select]
Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.812000 ms
     Name:        'Zeiss Makro-Planar T* 2/100 ZF.2'
     Serial:      '000000000' (no valid S/N)
     Focal Len:   100 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000
Block: ELNS
  Offset: 0x000001e4
  Number: 6
    Size: 64
    Time: 0.997000 ms
     Name:                'Zeiss Makro-Planar T* 2/100 ZF.2'
     Focal Length Min:    0 mm
     Focal Length Max:    0 mm
     Aperture Min:        f/0.00
     Aperture Max:        f/0.00
     Version:             0
     Extender Info:       0x00
     Capabilities:        0x00
     Chipped:             0x01

Why? Because the LENS name is no longer null terminated.

The last character of the string has to be the null terminator (0). So I slightly misspoke, the lens name is limited to 32 bytes (31 ascii characters).

in MLV strings dont *have to* be null terminated as you pointed out correctly.

@aprofiti -- our repositories seem to be out of sync because your patch didn't apply cleanly. Do you want to do the pull request or do you want me to do it? Note that we'll need to break it down into smaller commits.

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/22/elns-for-crop_rec_4k/diff
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on August 11, 2018, 06:10:47 PM
Do you want to do the pull request or do you want me to do it?

If everything is working fine I would like to commit the "dynamic" version first on manual_lens_info (to have the original branch up to date) and then make a new PR to crop_rec_4k_mlv_snd (is this the currently bleeding edge branch? I'll need your help to review this PR). What do you think about?

Note that we'll need to break it down into smaller commits.
Regarding breaking down into smaller commits, I can do this in manual_lens_inf PR; do it needs to be done also when merging to crop_rec?

our repositories seem to be out of sync because your patch didn't apply cleanly.
My repo was based on top of crop_rec_4k but then we moved to the one with sound support.
I had to solve more conflicts when merged mlv_snd but due to some problems determining which diff were the most up to data, I screw it up and moved working on top of manual_lens_info adapting changes to works with you repo.

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/22/elns-for-crop_rec_4k/diff
Having your repo as a reference could be useful to avoid missing something when merging to crop_rec_4k*, because differences from crop_rec_4k and crop_rec_4k_mlv_snd are not exactly clear to me.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 12, 2018, 02:13:43 AM
Ok--you can do the pull request on manual_lens_info but the development branch DuJour is crop_rec_4k_mlv_snd. That's where g3gg0 did the latest fixes on mlv_dump.

The crop_rec_4k branch has some recent commits that aren't working yet--EOSM and 700D enabled FRAME_SHUTTER_BLANKING_READ/WRITE. Besides, the goal is to eliminate mlv_rec with mlv_lite and the crop_rec_4k_mlv_snd branch looks like it is ready for that step.

My crop_rec_4k_ELNS branch has the crop_rec_4k_mlv_snd merged in and seems to be working nicely so go ahead and grab that branch from my repository when you do the pull request on the crop_rec_4k_mlv_snd branch.

You can try merging manual_lens_info into crop_rec_4k_mlv_snd and resolve the conflicts using my crop_rec_4k_ELNS branch but that will create a huge changeset. Better break it down into manageable hunks.

Oh--and don't worry about who gets credit for which line of code. You did all the heavy lifting on this. I was just the enabler--you know, the good definition of enabler.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on August 13, 2018, 02:03:02 PM
Found a bug after taking a silent picture in a commit (https://bitbucket.org/hudson/magic-lantern/commits/a39b95f04ccec982fa9070d7157353604281b340) from lua_fix.

Doens't happens with mlv_rec, mlv_lite and if I comment "force_liveview()" line.
Maybe it get queued?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on August 14, 2018, 02:56:33 AM
I'm trying to merge manual_lens_info to crop_rec_4k_mlv_snd but I'm a bit confused on which changes use to solve conflicts from commits made in lua_fix and the experimental stuffs from the other branch.

Is it possible to merge lua_fix into crop_rec_4k_mlv_snd? This should make managing conflicts from manual_lens_info much easier, and should also make changeset much lighter... Also should leaves diffs from manual_lens_info code when pushing the PR, making room for commits names suggested by g3gg0
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 14, 2018, 07:02:59 AM
Is it possible to merge lua_fix into crop_rec_4k_mlv_snd?

Done.

Merged cleanly and tested out fine so I went ahead and committed it. The lua_fix branch has been merged into crop_rec_4k a few time and then crop_rec_4k merged into crop_rec_4k_mlv_snd but it looks like these branches are a bit behind the latest lua_fix changes. Since there is some work in progress on the crop_rec_4k branch that affects the EOSM and 700D I didn't want to mess around with merging lua_fix into it.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on August 14, 2018, 11:16:41 AM
Merged cleanly and tested out fine so I went ahead and committed it. The lua_fix branch has been merged into crop_rec_4k a few time and then crop_rec_4k merged into crop_rec_4k_mlv_snd but it looks like these branches are a bit behind the latest lua_fix changes. Since there is some work in progress on the crop_rec_4k branch that affects the EOSM and 700D I didn't want to mess around with merging lua_fix into it.

Thank you dfort, It's looking much better and now i have only to figure out a conflict with raw.c (It touching 70D) apart the ones on the modules (easy to solve). Will look into it later or tomorrow.

Are these the commits name to follow?
Code: [Select]
“advanced lens information in ML core”
“added LUA interface and functions to set manual lens data”
“mlv_rec: add ELNS support”
“mlv_lite: add ELNS support”
“silent: handle advanced lens info”
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on August 15, 2018, 07:20:02 AM
Are these the commits name to follow?
Code: [Select]
“advanced lens information in ML core”
“added LUA interface and functions to set manual lens data”
“mlv_rec: add ELNS support”
“mlv_lite: add ELNS support”
“silent: handle advanced lens info”

That's the list g3gg0 gave us (https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/22/elns-for-crop_rec_4k/diff#comment-69963883) so I would say yes.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on August 15, 2018, 10:09:01 PM
Submitted a new pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/940/manual-lens-info-support-to-crop_rec_4k/diff) with changes for crop_rec_4k and another one (https://bitbucket.org/hudson/magic-lantern/pull-requests/939) to update manual_lens_info branch (also here lua_fix can be merged into the official repo's branch, to make it easier to see what's changed).

To make the first one I started with a merge from manual_lens_info and then removed all manual_lens_info diffs (except the one not included in lua_fix like property.c, modules.c... they are in first commit) from files by reverting changes to crop_rec_4k_mlv_snd revision.
After that I started to reapply diffs to implement ELNS processing and committed into separate changesets.

Is this an acceptable way to guarantee the maintainability that @g3gg0 was looking for?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on September 15, 2018, 05:07:52 PM
Having troubles again with selftest module.

With the fix for fast_malloc return type it can be compiled and run on camera, but the test for "Small-block malloc test" will not works (ui freezes and ERR99 in viewfinder) when I merge lua_fix into it...
Tried also the inverse (manual_lens_info -> lua_fix) but same problem occurs, only works as current state of manual_lens_info branch.

@a1ex Can you merge lua_fix into manual_lens_info an solve conflicts for me?

Also I have a doubt about the script:
It set lens_info.exists to true to enable all the extra info (lens info menu, aperture/focal length in lv etc...) to works with a manual lens.

Noticed that this commit (https://bitbucket.org/aprofiti/magic-lantern/commits/a2bea9f026db9aa7cf5b5eed8afb7842386829c4?at=crop_rec_4k_mlv_snd) by dmilligan where it was originally adding a field is_Chipped, now replaced with d0e55b3 (https://bitbucket.org/hudson/magic-lantern/commits/d0e55b332fa3152ef0c00dbf940a42caff28bb58?at=lua_fix) (use field lens_info.exists).
Here setting field exists will have as consequence some menu text to be displayed, enable dot_tune for non chipped adapter and the one is questioning me more: ETTR exposure bug with manual_lens

Is better to add another field like is_mManual or is_Chipped to be set by lens.lua and replace adapt those checks?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on September 22, 2018, 06:56:14 PM
Wish I could help
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on September 24, 2018, 02:21:47 PM
Wish I could help
Having additional feedback is welcome.

You can have a look about this list:
1. Overall camera stability
2. Raw recording and silent pictures stability, especially mlv_rec which was the most problematic to make it works
3. General feedback about the script and metadata correctness
4. Eventually other bugs that can pop out

Those should be checked on both PR (take the branches from my repo); if you are a main crop_rec_4k branch user, having that on a daily usage is a nice addition to extensively testing it.

Waiting for you feedback. After all, you starter the thread :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on September 24, 2018, 09:24:58 PM
Can you merge lua_fix into manual_lens_info an solve conflicts for me?

I looked into this and the merge conflicts shouldn't be difficult to resolve. I resolved it by taking the mem.c changes from the lua_fix branch because it has the newer memory backend updates. However, a problem after merging is that the selftest module will no longer compile. I believe that the latest changes to that module was done in the lua_fix branch and simply copying selftest.c from lua_fix it compiles and seems to work properly.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on September 26, 2018, 10:08:43 PM
@aprofiti.

with one is the best for testing
Latest Build (2018-09-13 23:06)
https://builds.magiclantern.fm/experiments.html
this one form experiments or one from your branch?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on September 27, 2018, 01:33:37 AM
this one form experiments or one from your branch?
Take the ones from my branch (manual_lens_info and crop_rec_4k_mlv_snd_elns).

The build from experimental page is an older one, not with the "dynamic" field length, it only include fix for compilation of selftest module but can also be less stable due to missing merge of latest lua_fix used to avoid camera crash when recording raw.

I looked into this and the merge conflicts shouldn't be difficult to resolve. I resolved it by taking the mem.c changes from the lua_fix branch because it has the newer memory backend updates. However, a problem after merging is that the selftest module will no longer compile. I believe that the latest changes to that module was done in the lua_fix branch and simply copying selftest.c from lua_fix it compiles and seems to work properly.
yes, they aren't difficult to resolve, just include the changes regarding allocator to fix conflicts, but there is something which will mess selftest's module functionality.

Just taking selftest from lua_fix will be missing "Small-block malloc" test which is failing after merging; same thing if just copying mem.c (some improvement in memory management was done for manual lens); maybe there are some changes in mem.c from lua_fix which will interfere with the version in manual_lens_info
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on September 29, 2018, 02:38:27 AM
I'm trying to see where it fails using Qemu.

First run saved a couple of log in CF and "Small-malloc test" appears prints debug info in console.
Observed it was allocating 1600 block (which 1000 of these should be from fast_malloc).

Here is the crash log of second run:
Code: [Select]
ASSERT: pvAddr
at Memory\Memory.c:170, RscMgr:ff867b7c
lv:0 mode:0

RscMgr stack: 13ed20 [13ef98-13df98]
0xUNKNOWN  @ ff86f9b0:13ef90
0xUNKNOWN  @ ff99058c:13ef68
0xFF98FFE8 @ ff82aa6c:13ef40
0xUNKNOWN  @ ff990018:13ef30
0xUNKNOWN  @ ff9900a0:13ef10
0xFF8AD684 @ ff8ae2a8:13eef8
0xFF8AA0D4 @ ff8ad73c:13eec8
0xFF869068 @ ff8aa4c0:13ee58
0xFF868868 @ ff8690e4:13ee40
0xFF86874C @ ff86890c:13ee28
0xFF86AFB0 @ ff868788:13ee10
0xFF867B54 @ ff86afc8:13ed60
0xFF814AC0 @ ff867b78:13ed58
0x0004B378 @ 4b84c:13ed20

Magic Lantern version : Nightly.2018Sep29.50D109
Mercurial changeset   : cc361edea9e6+0040e6ccea8d+ (manual_lens_info)
Built on 2018-09-29 00:21:17 UTC by alex@MacBook-Pro-di-Alessandro-8.local.
Free Memory  : 64K + 56K

Qemu console of second run:
Code: [Select]
ASSERT : Memory\Memory.c, Task = RscMgr, Line 170

 AllocateMemory 40
 TASK:[RscMgr]
  20:0x8000
  19:0x8000
  18:0x4
  17:0xff86afc4
  16:0x8b2900
  15:0x28
  14:0x8000
  13:0x63eaac
  12:0x19980218
  11:0x13ed60
  10:0x13eddc
   9:0x8000
   8:0xff816904
   7:0xac0078
   6:0
   5:0x63eaac
   4:0x8000
   3:0x8000
   2:0xff98c694
   1:0x387a4  1290: 12935.424 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = RscMgr
  1291: 12935.424 [STARTUP] ERROR ASSERT : Line 170
  1292: 12935.424 [STARTUP] ERROR ASSERT : pvAddr
  1293: 12937.216 [STARTUP] startupErrorRequestChangeCBR (0x1d)
  1294: 12937.216 [STARTUP] startupErrorRequestChangeCBR : ErrorSend (101, ABORT)
[MPU] Received: 08 06 03 03 65 01 00 00  (unknown - unnamed)
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 546
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 546
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 546
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
  1295: 12959.232 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1296: 12959.232 [STARTUP] ERROR ASSERT : Line 170
  1297: 12959.232 [STARTUP] ERROR ASSERT : pvAddr
  1298: 12959.232 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1299: 12959.488 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1300: 12959.488 [STARTUP] ERROR ASSERT : Line 170
  1301: 12959.488 [STARTUP] ERROR ASSERT : pvAddr
  1302: 12959.488 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1303: 12960.000 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1304: 12960.000 [STARTUP] ERROR ASSERT : Line 170
  1305: 12960.000 [STARTUP] ERROR ASSERT : pvAddr
  1306: 12960.000 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1307: 12960.256 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1308: 12960.256 [STARTUP] ERROR ASSERT : Line 170
  1309: 12960.256 [STARTUP] ERROR ASSERT : pvAddr
  1310: 12960.256 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1311: 12960.512 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1312: 12960.512 [STARTUP] ERROR ASSERT : Line 170
  1313: 12960.512 [STARTUP] ERROR ASSERT : pvAddr
  1314: 12960.512 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1315: 12961.024 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1316: 12961.024 [STARTUP] ERROR ASSERT : Line 170
  1317: 12961.024 [STARTUP] ERROR ASSERT : pvAddr
  1318: 12961.024 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1319: 12961.024 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1320: 12961.024 [STARTUP] ERROR ASSERT : Line 170
  1321: 12961.024 [STARTUP] ERROR ASSERT : pvAddr
  1322: 12961.280 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1323: 12961.536 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1324: 12961.536 [STARTUP] ERROR ASSERT : Line 170
  1325: 12961.536 [STARTUP] ERROR ASSERT : pvAddr
  1326: 12961.536 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1327: 12961.792 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1328: 12961.792 [STARTUP] ERROR ASSERT : Line 170
  1329: 12961.792 [STARTUP] ERROR ASSERT : pvAddr
  1330: 12961.792 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1331: 12962.048 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1332: 12962.048 [STARTUP] ERROR ASSERT : Line 170
  1333: 12962.048 [STARTUP] ERROR ASSERT : pvAddr
  1334: 12962.048 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1335: 12962.304 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1336: 12962.304 [STARTUP] ERROR ASSERT : Line 170
  1337: 12962.304 [STARTUP] ERROR ASSERT : pvAddr
  1338: 12962.304 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1339: 12962.816 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1340: 12962.816 [STARTUP] ERROR ASSERT : Line 170
  1341: 12962.816 [STARTUP] ERROR ASSERT : pvAddr
  1342: 12962.816 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1343: 12963.072 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1344: 12963.072 [STARTUP] ERROR ASSERT : Line 170
  1345: 12963.072 [STARTUP] ERROR ASSERT : pvAddr
  1346: 12963.072 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1347: 12963.328 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1348: 12963.328 [STARTUP] ERROR ASSERT : Line 170
  1349: 12963.328 [STARTUP] ERROR ASSERT : pvAddr
  1350: 12963.328 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1351: 12963.584 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1352: 12963.584 [STARTUP] ERROR ASSERT : Line 170
  1353: 12963.584 [STARTUP] ERROR ASSERT : pvAddr
  1354: 12963.584 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1355: 12963.840 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1356: 12963.840 [STARTUP] ERROR ASSERT : Line 170
  1357: 12963.840 [STARTUP] ERROR ASSERT : pvAddr
  1358: 12963.840 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1359: 12964.352 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1360: 12964.352 [STARTUP] ERROR ASSERT : Line 170
  1361: 12964.352 [STARTUP] ERROR ASSERT : pvAddr
  1362: 12964.352 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1363: 12964.608 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1364: 12964.608 [STARTUP] ERROR ASSERT : Line 170
  1365: 12964.608 [STARTUP] ERROR ASSERT : pvAddr
  1366: 12964.608 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1367: 12965.120 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1368: 12965.120 [STARTUP] ERROR ASSERT : Line 170
  1369: 12965.120 [STARTUP] ERROR ASSERT : pvAddr
  1370: 12965.120 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1371: 12965.376 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1372: 12965.376 [STARTUP] ERROR ASSERT : Line 170
  1373: 12965.376 [STARTUP] ERROR ASSERT : pvAddr
  1374: 12965.376 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1375: 12965.632 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1376: 12965.632 [STARTUP] ERROR ASSERT : Line 170
  1377: 12965.632 [STARTUP] ERROR ASSERT : pvAddr
  1378: 12965.632 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1379: 12966.144 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1380: 12966.144 [STARTUP] ERROR ASSERT : Line 170
  1381: 12966.144 [STARTUP] ERROR ASSERT : pvAddr
  1382: 12966.144 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1383: 12966.400 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1384: 12966.400 [STARTUP] ERROR ASSERT : Line 170
  1385: 12966.400 [STARTUP] ERROR ASSERT : pvAddr
  1386: 12966.400 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1387: 12966.656 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1388: 12966.656 [STARTUP] ERROR ASSERT : Line 170
  1389: 12966.656 [STARTUP] ERROR ASSERT : pvAddr
  1390: 12966.656 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1391: 12966.912 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1392: 12966.912 [STARTUP] ERROR ASSERT : Line 170
  1393: 12966.912 [STARTUP] ERROR ASSERT : pvAddr
  1394: 12967.168 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1395: 12967.424 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1396: 12967.424 [STARTUP] ERROR ASSERT : Line 170
  1397: 12967.424 [STARTUP] ERROR ASSERT : pvAddr
  1398: 12967.424 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1399: 12967.680 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1400: 12967.680 [STARTUP] ERROR ASSERT : Line 170
  1401: 12967.680 [STARTUP] ERROR ASSERT : pvAddr
  1402: 12967.680 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1403: 12967.936 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1404: 12967.936 [STARTUP] ERROR ASSERT : Line 546
  1405: 12967.936 [STARTUP] ERROR ASSERT : 0
  1406: 12967.936 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1407: 12968.192 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1408: 12968.192 [STARTUP] ERROR ASSERT : Line 170
  1409: 12968.192 [STARTUP] ERROR ASSERT : pvAddr
  1410: 12968.192 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1411: 12968.448 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1412: 12968.448 [STARTUP] ERROR ASSERT : Line 170
  1413: 12968.448 [STARTUP] ERROR ASSERT : pvAddr
  1414: 12968.448 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1415: 12968.704 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1416: 12968.704 [STARTUP] ERROR ASSERT : Line 546
  1417: 12968.704 [STARTUP] ERROR ASSERT : 0
  1418: 12968.704 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1419: 12968.960 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1420: 12968.960 [STARTUP] ERROR ASSERT : Line 170
  1421: 12968.960 [STARTUP] ERROR ASSERT : pvAddr
  1422: 12968.960 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1423: 12969.216 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1424: 12969.216 [STARTUP] ERROR ASSERT : Line 170
  1425: 12969.216 [STARTUP] ERROR ASSERT : pvAddr
  1426: 12969.216 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1427: 12969.472 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1428: 12969.472 [STARTUP] ERROR ASSERT : Line 546
  1429: 12969.472 [STARTUP] ERROR ASSERT : 0
  1430: 12969.472 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1431: 12969.728 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1432: 12969.728 [STARTUP] ERROR ASSERT : Line 170
  1433: 12969.728 [STARTUP] ERROR ASSERT : pvAddr
  1434: 12969.728 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1435: 12969.984 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = CtrlSrv
  1436: 12969.984 [STARTUP] ERROR ASSERT : Line 170
  1437: 12969.984 [STARTUP] ERROR ASSERT : pvAddr
  1438: 12969.984 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
ASSERT : ShootMemory.c, Task = RscMgr, Line 1112
ASSERT : ShootMemory.c, Task = RscMgr, Line 1114
ASSERT : PackMemory\PackMem.c, Task = run_test, Line 479
ASSERT : Memory\Memory.c, Task = RscMgr, Line 170

 AllocateMemory 40
 TASK:[RscMgr]
  20:0x8000
  19:0x8000
  18:0x4
  17:0xff86afc4
  16:0x8caa88
  15:0x28
  14:0x8000
  13:0x63eaac
  12:0x19980218
  11:0x13ed60
  10:0x13eddc
   9:0x8000
   8:0xff816904
   7:0xac0078
   6:0
   5:0x63eaac
   4:0x8000
   3:0x8000
   2:0xff98c694
   1:0x30798  1439: 13002.752 [STARTUP] ERROR ASSERT : ShootMemory.c, Task = RscMgr
  1440: 13002.752 [STARTUP] ERROR ASSERT : Line 1112
  1441: 13003.008 [STARTUP] ERROR ASSERT : Error == SUCCESS
  1442: 13003.008 [STARTUP] ERROR ASSERT : ShootMemory.c, Task = RscMgr
  1443: 13003.008 [STARTUP] ERROR ASSERT : Line 1114
  1444: 13003.008 [STARTUP] ERROR ASSERT : Error == SUCCESS
  1445: 13003.008 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1446: 13003.008 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1447: 13003.264 [STARTUP] ERROR ASSERT : PackMemory\PackMem.c, Task = run_test
  1448: 13003.264 [STARTUP] ERROR ASSERT : Line 479
  1449: 13003.264 [STARTUP] ERROR ASSERT : IsChunkSignature( hChunk )
  1450: 13003.264 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
  1451: 13003.520 [STARTUP] ERROR ASSERT : Memory\Memory.c, Task = RscMgr
  1452: 13003.520 [STARTUP] ERROR ASSERT : Line 170
  1453: 13003.520 [STARTUP] ERROR ASSERT : pvAddr
  1454: 13004.800 [STARTUP] startupErrorRequestChangeCBR : OverWrite (0x1d => 0x1d)
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 546
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 546
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 546
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : Memory\Memory.c, Task = CtrlSrv, Line 170
ASSERT : ShootMemory.c, Task = RscMgr, Line 1112
ASSERT : ShootMemory.c, Task = RscMgr, Line 1114

Maybe there is something with the changes from lua_fix in allocator selection or dimension?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on October 18, 2018, 02:54:14 AM
Figured out what is causing the crash when running "Small-block malloc" test.

These are the parameters of the allocators on both branch:
Code: [Select]
Values from lua_fix merge:

        //Malloc Allocator
        .preferred_free_space = 384 * 1024,
        .minimum_free_space = 256 * 1024,

        //Shoot malloc Allocator:
        .try_next_allocator = 1,

        .minimum_alloc_size = 64 * 1024,
        .minimum_free_space = 256 * 1024,

Values from manual_lens_info:

        //Malloc
        .preferred_free_space = 128 * 1024,
        .minimum_free_space = 64 * 1024,

        //Shoot Malloc
        .minimum_alloc_size = 5 * 1024,

Test will run fine in QEMU If I revert using the old parameters.

@a1ex They are from commit 0e56fe7 (https://bitbucket.org/hudson/magic-lantern/commits/0e56fe7ef5c6683c6e2133a81becf6b1faf6ce3e), What parameter I have to use?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on October 19, 2018, 12:41:34 AM
Good find!

I'm running a build without much problems for some time now, mosty 14 bit lossless with h264 proxy.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on December 23, 2018, 12:07:48 PM
Narrowed it down - the 50D and 5D2 use AllocateMemory for their RscMgr data structures; most other models use malloc. When allocating memory from shoot_malloc, I need to be sure Canon firmware is going to have enough space in both AllocateMemory and malloc pools. Otherwise, the firmware may run into a situation with plenty of memory in shoot_malloc, that cannot be allocated because the general-purpose allocators are full (so, RscMgr is unable to allocate its data structures for keeping track of shoot_malloc buffers).

Back then, I've assumed these two are just like 500D, which is from the same generation. Turns out, 500D uses malloc and doesn't crash on this test.

Fix coming soon.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 23, 2018, 02:18:36 PM
Glad to hear this :)

Is the same also with 7D? If I remember correctly it was crashing also on this model when dfort tested it
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on December 23, 2018, 06:33:04 PM
Fix coming soon.

Yay!

Would love to get this feature into some of the other branches. We put a lot of work into this:

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/22/elns-for-crop_rec_4k/diff
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on December 23, 2018, 11:52:13 PM
Will check, but I've got other priorities for this holiday season (https://www.magiclantern.fm/forum/index.php?topic=23084.msg208821#msg208821). After I'll finish them, with pleasure.

Meanwhile I've published an updated build that passes those memory allocation tests in QEMU. I'm not sure it's completely fixed; these property handlers in Lua (in particular, memory allocations performed by them) are pretty demanding for the backend.

Just for fun, here's the 5D2 failing the test in QEMU (memory corruption breaking Canon UI):

(https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-nightly-tests/510/artifact/qemu-eos/magiclantern-Nightly.2018Jul03.5D2212/14-small-malloc-fini.png)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on December 24, 2018, 10:40:55 AM
Happy Holiday! 
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on December 24, 2018, 11:12:28 AM
Quick tested the new build:
Small-malloc test runs good, tried to run more times in a row without any problem :)
Api_test.lua failed at lv_test, will look again later.

Unfortunally it appears to be less stable when recording raw videos, like in previous builds without mem backend improvements for property handler...
Maybe fast_malloc is still needed?

Test I was running back to check raw rec:
Enter liveview and start a recording and then stop, if camera doesn't freezes exit liveview and repeat the same procedure (may fails when entering lives or after stopping rec); mlv_rec is usually less forgivable compared to mlv_lite.

In this build it fails on second run with mlv_rec and I got a camera freeze with mlv_lite when I was navigating ML menu in liveview mode after a recording

Lot of prop_request timeout in console and triggered an assert after selecting lens from menu:
Code: [Select]
ML ASSERT:
0
at ../../src/mem.c:799 (__mem_malloc), task PropMgr
lv:0 mode:3

PropMgr stack: 139c60 [139f70-138f70]
0x009D8CC4 @ 9e3fe0:139d10
0xUNKNOWN  @ 9d8da8:139cf0
0x0004E3FC @ 9fa3e4:139cd0
0x0004BB38 @ 4e438:139c90
0x0004B378 @ 4bb94:139c60

Magic Lantern version : manual_lens_info.2018Dec23.50D109
Mercurial changeset   : 0de7b671c52c (manual_lens_info) tip
Built on 2018-12-23 22:33:00 UTC by jenkins@nightly.
Free Memory  : 232K + 3150K
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on December 24, 2018, 11:59:23 AM
OK, this assert is exactly what I was looking for. I wasn't able to trigger it in my tests on 5D2 after removing fast_malloc, but I'll try again.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 18, 2019, 10:20:51 AM
Hi,

For couple of evenings I'm playing with the latest nightly build on my 6D (damned i need some sleep) and noticed some issues.

Some of the recent changes in lua library had changed contents of dryos variables and the dryos.dcim_dir.path (used in get_sidecar_filename() @ xmp.lib) does not  work anymore.
Being on linux and using daktable+gimp i have to use IMG_1234.[CR2|JPG].xmp name to have it recognized and used by DT, so i've modified the get_sidecar to:

Code: [Select]
function xmp.get_sidecar_filename() -- {{{
    return dryos.shooting_card:image_path(1) .. ".xmp"
end -- }}}

I made some changes in the XMP file format (to use the same as exiftool) and tried to save the xmp file line by line to lower the memory consumption..
Unfortunately now it crashes immediately after taking a picture (for a split of a second i can see an error with wiritng to the xmp file (which was working for me for a while, but apparently i broke something a bit later)) and all i end up with is a picture and an assert log

Code: [Select]
ML ASSERT:
0
at ../../src/mem.c:799 (__mem_malloc), task PropMgr
lv:0 mode:3

PropMgr stack: 170dc0 [170f68-16ff68]
0xUNKNOWN  @ 468a88:170e70
0x00BFD44C @ bf8b8c:170e50
0x00BFCC88 @ bfd484:170e40
0x0044F5E0 @ bfcca8:170e30
0x0044C9F4 @ 44f61c:170df0
0x0044C478 @ 44ca50:170dc0

Magic Lantern version : manual_lens_info.2018Dec23.6D116
Mercurial changeset   : 0de7b671c52c (manual_lens_info) tip
Built on 2018-12-23 22:39:23 UTC by jenkins@nightly.
Free Memory  : 384K + 1935K

At the moment my xmp stuff looks like this:

Code: [Select]
xmp.header= [[<?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="MagicLantern">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230">]]

xmp.footer      = [[  </rdf:Description>
 </rdf:RDF>     
</x:xmpmeta>
]]

xmp.property_format = [[   <%s>%s</%s>]]

-- predefined xmp properties
xmp.lens_make           = { name = "exif:LensMake",             format = "%s" }
xmp.lens_name           = { name = "exif:LensModel",            format = "%s" }
xmp.lens_serial         = { name = "exif:LensSerialNumber",     format = "%s" }
xmp.focal_length        = { name = "exif:FocalLength",          format = "%d/1"}
xmp.aperture            = { name = "exif:FNumber",              format = "%d/10" }
xmp.apertureMin         = { name = "exif:MinApertureValue",     format = "F%s" }
xmp.apertureMax         = { name = "exif:MaxApertureValue",     format = "F%s" }

function xmp.get_sidecar_filename() -- {{{
    return dryos.shooting_card:image_path(1) .. ".xmp"
end -- }}}

....

function xmp:write(filename) -- {{{
        local f = assert(io.open(filename or self.get_sidecar_filename(), "w+"))
        assert(f:write(self.header, "\n"))

        for k,v in pairs(self.properties) do
                if type(v) == "function" then
                        self.log:write("klic:"..k.."="..v())
                        assert(f:write(string.format(self.property_format, k, v(), k), "\n"))
                else   
                        self.log:write("klic:"..k.."="..v)
                        assert(f:write(string.format(self.property_format, k, v, k), "\n"))
                end     
        end     
        --f:write(string.format(self.template, str))
        assert(f:write(self.footer))
        assert(f:close())
end --- }}}

Btw: is there a way i could test these scripts in qemu on my own? restarting the camera, mounting the sdcard, copying scripts, unmounting sdcard, starting camera is pretty exhausting and annoying especially when all i find is yet another bug in my own code :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on January 18, 2019, 12:00:10 PM
Right, I need to look into that. Maybe backing out b0a2f95 (i.e. re-applying 065ceae) could help.

QEMU... doesn't go as far as completing a CR2 photo capture, so it won't trigger an XMP. It is able to emulate a full-res silent picture, and that one might trigger the XMP (possibly after some fiddling). However, silent pictures don't share the same naming/numbering as regular photos, so XMPs are not going to match there. LiveView emulation is not exactly working either (it shows Canon's GUI elements, but that's all). I have both of these halfway (or rather, 10-20%) working, but they are not straightforward.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 18, 2019, 12:26:12 PM
btw: even during those bright moments when i had the xmp file written correctly, the exif data in cr2 and jpg (when i've switched to shooting to both formats) were showing the original unaltered data.

If you'd need any help with testing, let me know - i have 650D+6D and couple of non-chipped and chipped adapters to play with.

(https://i.ibb.co/b3LZkgm/IMG-20190117-222920.jpg) (https://ibb.co/b3LZkgm) (https://i.ibb.co/PGdvv8j/received-536418370194244.jpg) (https://ibb.co/PGdvv8j)

I've also split the lens name to brand+model, so the lens name is shorter (MOG would eat 20 bytes off the name :) )
Code: [Select]
lenses = -- {{{
{
    { make = "Carl Zeiss Jena DDR",     name = "Tessar 50mm f/2.8",             focal_length = 50,      manual_aperture = 2.8,  serial = "8333263" },
    { make = "E. Ludwig",               name = "Meritar 50mm f/2.9",            focal_length = 50,      manual_aperture = 2.9,  serial = "1610554" },
    { make = "Helios",                  name = "44-2 58mm f/2",                 focal_length = 58,      manual_aperture = 2,    serial = "7765411" },
    { make = "Industar",                name = "50-2 50mm f/3.5",               focal_length = 50,      manual_aperture = 3.5,  serial = "7149595" },
    { make = "Industar",                name = "61 L/Z (MC) 50mm f/2.8",        focal_length = 50,      manual_aperture = 2.8,  serial = "8707286" },
    { make = "Meyer-Optik Gorlitz",     name = "Telemegor 180mm f/5.5",         focal_length = 180,     manual_aperture = 5.5,  serial = "1728275" },
    { make = "Meyer-Optik Gorlitz",     name = "Trioplan 50mm f/2.9",           focal_length = 50,      manual_aperture = 2.8,  serial = "2224344" },
    { make = "Pentacon",                name = "auto 29mm f/2.8",               focal_length = 29,      manual_aperture = 2.8,  serial = "6308110" },
    { make = "Pentacon",                name = "Prakticar 50mm f/1.8 MC",       focal_length = 50,      manual_aperture = 1.8,  serial = "7710425" },
    { make = "Samyang",                 name = "85mm f/1.4 AS IF UMC",          focal_length = 85,      manual_aperture = 1.4} --       serial = "A217D0264" }
--    { make = "", name = "", focal_length = , manual_aperture = , serial = "" },
} -- }}}

I also had an issue where the lens.lua had crashed when i've selected the last lens in the list - 85mm Samyang. With something like "invalid parameter #3 for __nextval(", i can't remember now and since i'm procrastinating at work, i don't have my camera with me now to replicate it at the moment ;)

and one more thing - the serial number was written as "0" even when it was set cirrectly in xmp.properties.. no idea why..
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 18, 2019, 01:45:29 PM
Some of the recent changes in lua library had changed contents of dryos variables and the dryos.dcim_dir.path (used in get_sidecar_filename() @ xmp.lib) does not  work anymore.
Being on linux and using daktable+gimp i have to use IMG_1234.[CR2|JPG].xmp name to have it recognized and used by DT, so i've modified the get_sidecar to:

Code: [Select]
function xmp.get_sidecar_filename() -- {{{
    return dryos.shooting_card:image_path(1) .. ".xmp"
end -- }}}
This refactor (https://bitbucket.org/hudson/magic-lantern/commits/8b34a1ae6fd472dfab0d62dea9b9802d77aa8f48?at=manual_lens_info) is what is causing the issue. Adapting this branch to the new API should do the trick,
Was considering to fix in the PR (https://bitbucket.org/hudson/magic-lantern/pull-requests/939/dynamic-length-elns-block/diff) after a1ex had a look again to the memory management backend.

Maybe try with "dryos.shooting_card:dcim_dir"

I made some changes in the XMP file format (to use the same as exiftool) and tried to save the xmp file line by line to lower the memory consumption..
Unfortunately now it crashes immediately after taking a picture (for a split of a second i can see an error with wiritng to the xmp file (which was working for me for a while, but apparently i broke something a bit later)) and all i end up with is a picture and an assert log
Right, I need to look into that. Maybe backing out b0a2f95 (i.e. re-applying 065ceae) could help.
Should be related to what a1ex said, I noticed the assert for first time in this new build, when switching lens selection; I can't remember if happened also to me under same circumstance now, but probably yes.

Btw: is there a way i could test these scripts in qemu on my own? restarting the camera, mounting the sdcard, copying scripts, unmounting sdcard, starting camera is pretty exhausting and annoying especially when all i find is yet another bug in my own code :)
I really wanted this was possible at current stage, will make much less stressful and time consuming, but we need to wait a little more :)

btw: even during those bright moments when i had the xmp file written correctly, the exif data in cr2 and jpg (when i've switched to shooting to both formats) were showing the original unaltered data.
It's minded to work like this: Write metadata to .xml sidecart file (and/or in .mlv if full-res picture or raw video) instead of playing with .cr2 as is considered to be dangerous from what I've read in old discussion in this thread.

Maybe a safe way to inject into .cr2 can be found?

I've also split the lens name to brand+model, so the lens name is shorter (MOG would eat 20 bytes off the name :) )
Code: [Select]
lenses = -- {{{
{
    { make = "Carl Zeiss Jena DDR",     name = "Tessar 50mm f/2.8",             focal_length = 50,      manual_aperture = 2.8,  serial = "8333263" },
    { make = "E. Ludwig",               name = "Meritar 50mm f/2.9",            focal_length = 50,      manual_aperture = 2.9,  serial = "1610554" },
    { make = "Meyer-Optik Gorlitz",     name = "Telemegor 180mm f/5.5",         focal_length = 180,     manual_aperture = 5.5,  serial = "1728275" },
    { make = "Samyang",                 name = "85mm f/1.4 AS IF UMC",          focal_length = 85,      manual_aperture = 1.4} --       serial = "A217D0264" }
--    { make = "", name = "", focal_length = , manual_aperture = , serial = "" },
} -- }}}
Shouldn't make really a difference in space usage... as Photoshop and similar, need to read the whole name to find correct lens correction profile.

Instead could be useful when you got really long lens name, as I can see from your pictures when selecting a lens from menu;

This could be implemented in the PR to have it in next build if you and others find useful; just need to concatenate "make" and "name" when writing ELNS block and XMP metadata.

I also had an issue where the lens.lua had crashed when i've selected the last lens in the list - 85mm Samyang. With something like "invalid parameter #3 for __nextval(", i can't remember now and since i'm procrastinating at work, i don't have my camera with me now to replicate it at the moment ;)

and one more thing - the serial number was written as "0" even when it was set cirrectly in xmp.properties.. no idea why..
Can you tried to reproduce this? I'm also interested to know if this is happening just in the latest build or not.
Note: the script iterate through attribute in lens table, It may be possible that is not finding the correct values after introducing "make" attribute.

Regarding serial number: save a .mlv and watch for metadata with the modified mlv_dump (compile from manual_leins_info branch). Just to understand if it's only related to XMP or if ML is't saving datas in memory.

Note: mlv_dump was modified in the pending PR to match metadata format, so I don't remember if was working good in old version (current build on ML site).

If you'd need any help with testing, let me know - i have 650D+6D and couple of non-chipped and chipped adapters to play with.
Yes, absolutely! I have just a couple of cheap adapter on which I based when writing the lens recognition of the script.

They show as "1-65535mm" for name and focal length.
Already refactored to a separate function just in case to make it easy to read and adapt.
Code: [Select]
function is_manual_lens()
  -- Adapter with no AF Chip -> ID = 0
  -- Adapter with AF confirm Chip -> name and focal length "1-65535mm"
  if (lens.id == 0 or lens.name == "1-65535mm" or lens.name == "(no lens)") then
    return true
  else
    return false
  end
end

Can you check if also your adapter works like this? I imagine should be the same even if different makers, except for Dandelion adapters.

Also need more feedbacks about Raw video recording stability if you are daily video user.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 18, 2019, 02:26:20 PM
My point of splitting lens name to brand+type was to keep the lens.name shorter than 32 chars. We can always concatenate these two fields in UI and when exporting the xmp, where it should not matter and should be used by LR/DT/anything to detect the used lens.

Plus i'd like to add few more exif fields to the xmp output (i wrote a simple bash script to "label" my pics - https://www.zlej.net/tmp/exif.txt and it would be great to see this done automatically by ML as i tend to forget what i had attached two weeks ago when shooting some pics..)

Is there any way to scroll within the debug console? Scripts may complain about something usefull, but when the screen gets wiped out due to the assert crash, i'm lost :(

I thought, that when we "fake" the lens.[name|aperture|whatever], the canon FW will use those values when saving the exif data in cr2 or jpeg, but apparently it does not bother. Plus they use Exif.Photo and Exif.Canon* sections for similar data
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 18, 2019, 07:05:14 PM
My point of splitting lens name to brand+type was to keep the lens.name shorter than 32 chars. We can always concatenate these two fields in UI and when exporting the xmp, where it should not matter and should be used by LR/DT/anything to detect the used lens.
As used to work on code related to ELNS block, I initially believed you were referring to the actual file size saved on card... then understood after posting that you were talking abound lens name length showed in UI.

So yes it could be done. Slightly better could be to check if a lens name is to big to be showed in lens selection menu and truncate just the maker name, instead just print the whole name (make + model).

Noticed you modified the XMP lib's tag related to selected lens from "aux:" to "exif:".
Made some research and didn't fully come to a conclusion on which is better to use, as Canon stick with old tag system and doesn't come to troubles with tradition editing software.

Also noticed that "aux:Lens" just put the focal length range with my lens....
What are the advantage if we replace with the "exif:lensMake"/"exifEX:lensMake" and "exif:LensModel"/"exifEX:LensModel" attributes?

Plus i'd like to add few more exif fields to the xmp output (i wrote a simple bash script to "label" my pics - https://www.zlej.net/tmp/exif.txt and it would be great to see this done automatically by ML as i tend to forget what i had attached two weeks ago when shooting some pics..)
I looked at ExifTool's XMP tag (https://sno.phy.queensu.ca/~phil/exiftool/TagNames/XMP.html) list and XMP specification and found under tags used by Photoshop the string: "xmp:Label" which looks like it could be used for your purpose.

Reading an article (https://www.fastrawviewer.com/usermanual14/xmp-metadata) from FastRawViewer his content doesn't looks forced to a predefined set; instead as being a simply string I tried to manual modify a .xmp file and get retrieved by Adobe Bridge as a new type of label.

Alternatively we could add a new custom tag to the xmp library (ex. under ML namespace), but should't be necessary.

To add a new xmp attribute to the script, follow these steps:
1. Add a new predefined properties to xmp.lua library (if you want the code to looks a bit cleaner, otherwise you could skip this step and adapt the second one):
"xmp.label = { name = "xmp:Label" }"
Code: [Select]
-- predefined xmp properties
xmp.lens_name = { name = "aux:Lens" }
xmp.lens_serial = { name = "aux:SerialNumber" }
xmp.focal_length = { name = "exif:FocalLength", format = "%d/1"}
xmp.aperture = { name = "exif:FNumber", format = "%d/10" }

2. Tell to the lens.lua script to save the new tag when taking a picture:
"xmp:add_property(xmp.label, function() return lenses[selector_instance.index].label end)"
Code: [Select]
-- Property to be written in .xmp file
xmp:add_property(xmp.lens_name, function() return lens.name end)
xmp:add_property(xmp.focal_length, function() return lens.focal_length end)
xmp:add_property(xmp.aperture, function() return (lens.manual_aperture * 10) end)
xmp:add_property(xmp.lens_serial, function() return lens.serial end)
Note: Here I'm retrieving value from a local variable of lens.lua script, instead of using something like "lens.label" as it is an API not available  (https://bitbucket.org/hudson/magic-lantern/src/0de7b671c52c9bee7ac291bcc9ff8a88f3de9f38/modules/lua/lua_lens.c?at=manual_lens_info&fileviewer=file-view-default)at the moment.
You need to add getter (https://bitbucket.org/hudson/magic-lantern/src/0de7b671c52c9bee7ac291bcc9ff8a88f3de9f38/modules/lua/lua_lens.c?at=manual_lens_info&fileviewer=file-view-default#lua_lens.c-17) and setter (https://bitbucket.org/hudson/magic-lantern/src/0de7b671c52c9bee7ac291bcc9ff8a88f3de9f38/modules/lua/lua_lens.c?at=manual_lens_info&fileviewer=file-view-default#lua_lens.c-96) functions in modules/lua/lua_lens.c to make API available to use and save value somewhere (ex. ML lens_info structure in /src/lens.h), then recompile ML (https://www.magiclantern.fm/forum/index.php?topic=991.0).

Also I'm not sure if value get updated when switching lens by using this method and if you get an error for missing field.
If one of these happen try to add check for "label" inside the for cycle:
Code: [Select]
function update_lens()
    -- Reset lens_info structure to get correct values in Lens Info Menu and Metadata
    reset_lens_values()
    -- Update attribute from selected lens
    for k,v in pairs(lenses[selector_instance.index]) do
        -- avoid not found exception with the key "label"
        if k ~= "label" then
         lens[k] = v
        else
         -- update label value   
         xmp:add_property(xmp.label, function() return lenses[selector_instance.index].label end) -- may not be necessary
       end
    end
    -- Allow to write sidecar
    xmp:start()
    -- Update flag
    lensSelected = true
    -- Allow to write values in Lens Info Menu
    lens.exists = true
end

3. Add the attribute "label" to the corresponding lenses:
Code: [Select]
-- From your lenses Table version

lenses = -- {{{
{
    { make = "Carl Zeiss Jena DDR",     name = "Tessar 50mm f/2.8",             focal_length = 50,      manual_aperture = 2.8,  serial = "8333263",
 label = "tessar50" },
    { make = "Industar",                name = "61 L/Z (MC) 50mm f/2.8",        focal_length = 50,      manual_aperture = 2.8,  serial = "8707286",
 label = "industar61" },
    { make = "Meyer-Optik Gorlitz",     name = "Telemegor 180mm f/5.5",         focal_length = 180,     manual_aperture = 5.5,  serial = "1728275",
 label = "telemegor180" },

....

} -- }}}

Looks like you already dive deep in the script structure (https://www.magiclantern.fm/forum/index.php?topic=18083.msg210691#msg210691), so you should not have particular problems to add the new tag
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 19, 2019, 12:52:38 AM
So, i wrote a simple dumper script and tested all the adapters i found with it:

Code: [Select]
-- Lens properties dump

require("config")
require("logger")

local log = logger("adapter.log")

for k,v in pairs(lens) do
        if k ~= "focus" then
                if type(v) == "string" then
                        log:write("lens."..k.."="..v)
                elseif type(v) == "boolean" or type(v) == "number" then
                        log:write("lens."..k.."()="..tostring(v))
                elseif type(v) == "function" then
                        r = v()
                        if type(r) == "string" then
                                log:write("lens."..k.."()="..r)
                        elseif type(r) == "boolean" or type(r) == "number" then
                                log:write("lens."..k.."()="..tostring(r))
                        end
                elseif type(v) == "table" then
                        for a,b in pairs(v) do
                                log:write("lens. "..k.."."..a.."="..b)
                        end
                end
        end
end

There was a little hope in finding some way to differentiate between each one of them (some should be "programmable" and of different versions as i bought each separately in past few years).
Unfortunately with no luck:

Code: [Select]
===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:51:36 ----> no chip at all (Samyang 85/1.4)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.name=
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.focal_length()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.hyperfocal()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.dof_near()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.dof_far()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:52:55 ----> with AF chip (M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:56 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:53:16 ----> with AF chip (M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:53:42 ----> Sigma 50/2.8 DG Macro (AF turned on)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.name=EF50mm f/2.5 Compact Macro
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.hyperfocal()=30888
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.dof_near()=-340372800
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.af()=true
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.af_mode()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:45 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:54:29 ----> with AF chip (praktica B -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.focal_length()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.hyperfocal()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.dof_near()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.dof_far()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:54:53 ----> with AF chip (another M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:53 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:53 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.autofocus()=false

lens.focus was crashing and i was too lazy to search for a way to dump it correctly, so it's omitted from the output :)

@aprofiti] my "deep dive" is more of an endless circle of "hack->save->start the camera->facepalm->hack..." :)

So far i've "tweaked" a bit the xmp.lua, logger.lua and lens.lua.

------

Selecting a lens from the list results in this to be logged:
Code: [Select]
===============================================================================
ML/SCRIPTS/LENS.LUA - 2019-1-19 00:20:22
===============================================================================

ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - selector_instance:8
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: focal_length: 29
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: manual_aperture: 2.8
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: name: auto 29mm f/2.8
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: serial: 6308110
ML/SCRIPTS/LENS.LUA: 2019-01-19 00:20:30 - Update_lens: make: Pentacon

Lens.* dumper shows:
Code: [Select]
===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-19 00:33:20
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.name=auto 29mm f/2.8
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.focal_length()=29
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.hyperfocal()=30888
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.dof_near()=-340372800
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-19 00:33:20 - lens.autofocus()=false

And xmp.lua with write() modified to only log what it should be writing to xmp file logs:
Code: [Select]
===============================================================================
ML/SCRIPTS/LIB/xmp.lua - 2019-1-19 00:33:06
===============================================================================

ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - Writing xmp file...
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:LensSpecification=0/10 0/10 0/1 0/1
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:LensSerialNumber=0
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:FNumber=14/10
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:MaxApertureValue=F1.4
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:37 - key: exif:MinApertureValue=Fnil
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:38 - key: exif:LensModel=Pentacon auto 29mm f/2.8
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:38 - key: exif:LensMake=Pentacon
ML/SCRIPTS/LIB/xmp.lua: 2019-01-19 00:33:38 - key: exif:FocalLength=29/1

The modified write() looks like this:
Code: [Select]
--[[---------------------------------------------------------------------------
Writes the xmp metdata to file
@param filename optional - the filename to write to, leave blank to use the sidecar filename for the most recently taken photo
@function write
]]
function xmp:write(filename) -- {{{
--      local f = assert(io.open(filename or self.get_sidecar_filename(), "w+"))
--      assert(f:write(self.header, "\n"))

        xmplog:write("Writing xmp file...")
        for k,v in pairs(self.properties) do
                if type(v) == "function" then
                        xmplog:write("key: "..k.."="..v())
--                      assert(f:write(string.format(self.property_format, k, v(), k), "\n"))
                elseif type(v) == "boolean" or type(v) == "number" then
                        xmplog:write("key2: "..k.."="..tostring(v))
--                      assert(f:write(string.format(self.property_format, k, tostring(v), k), "\n"))
                else
                        xmplog:write("key2: "..k.."="..v)
                end
        end
        --f:write(string.format(self.template, str))
--      assert(f:write(self.footer))
--      assert(f:close())
end --- }}}

Unfortunately xmp.write() crashes when i try to let it write the xmp file and the error console is immediately wiped out by subsequent assert() crash..


(https://i.ibb.co/SKb4GrS/lenses.jpg) (https://ibb.co/SKb4GrS) (https://i.ibb.co/2Sb21vQ/select-lens.jpg) (https://ibb.co/2Sb21vQ)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 19, 2019, 01:41:46 PM
I've just found what was wrong when setting params for my Samyang - lens.serial is expected to be integer, but this lens has string..


(https://i.ibb.co/wYF6tWr/serial-number.jpg) (https://ibb.co/wYF6tWr)

(two hours and numerous edits of lens.lua+xmp.lua later)
I'm finally able to write the xmp file. Unfortunately only to the root of SD card - for some reason it still crashes when i try to use the dryos path, even when it looks fine.
Could someone with LR/PS/anything else verify that this format is acceptable by software other than darktable&gimp? ;)
Code: [Select]
<?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="MagicLantern">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230">
   <exif:LensModel>Samyang 85mm f/1.4 AS IF UMC</exif:LensModel>
   <exif:FocalLength>85/1</exif:FocalLength>
   <exif:MaxApertureValue>F1.4</exif:MaxApertureValue>
   <exif:FNumber>14/10</exif:FNumber>
   <exif:LensSpecification>10/10 655350/10 0/1 0/1</exif:LensSpecification>
   <exif:LensSerialNumber>A217D0264</exif:LensSerialNumber>
   <exif:LensMake>Samyang</exif:LensMake>
   <exif:MinApertureValue>F22</exif:MinApertureValue>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

LensSpecification is still showing wrong string (it should be "min_focal max_focal Fmax@min_focal Fmax@max_focal"), but i'll polish that one out, plus it's absolutely unimportant for anyone but me.

As a workaround for the serial number i've renamed "serial" in lenses{} to serialN, which is handled as string() by default, so i'm able to access it in xmp() and save. I suppose, that the serial==number() limitation is hardcoded somewhere in ML sources, right?

I've added four more params to lenses{}:
1] max_aperture and min_aperture, which are used to trim down the list of available Fnumbers plus they'll end up in the <exif:M[in|ax]ApertureValue>F22</exif:M[in|ax]ApertureValue> elements

2] FminL, FmaxL - apertures on wide/long ends of zoom lenses if they differ. Used in LensSpecification field.

Btw: i've checked exif data of my pics and the Exif.Photo.LensModel is really just a model without the lens brand (pics taken with trioplan were labeled by my old bash script):
Code: [Select]
exiv2 -g Exif.Photo.LensModel *.CR2|sed -e "s/^.*Ascii *.*  //"|sort|uniq -c|sed -e "s/^ *//" -e "s/ /x /"
2x EF17-40mm f/4L USM
435x EF-S10-18mm f/4.5-5.6 IS STM
102x Meyer-Optik Görlitz Trioplan 50mm f/2.9
7x MP-E65mm f/2.8 1-5x Macro Photo
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on January 19, 2019, 01:48:25 PM
Interesting finds!

I can test in adobe camera raw if you can share a sample of the footage with the lens data you captured.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 19, 2019, 02:48:43 PM
cr2+corresponding xml can be downloaded at https://www.zlej.net/sample.zip

It still seems to me, that during/after the exposure some lens{} parameters are being overwritten with defaults either from camera or the chipped adapter..

Interesting finds!

I can test in adobe camera raw if you can share a sample of the footage with the lens data you captured.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on January 19, 2019, 04:27:50 PM
This is what I get in adobe bridge, the zoom lens is one I have never seen :)

(http://www.larssteenhoff.com/forum_pics/shota.png)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 19, 2019, 08:10:30 PM
Those are the default values from the adapter/camera, which are stored in exif. So the xmp file was not processed at all.

Can you rename the xmp file to test.xmp and try it again, please?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on January 19, 2019, 09:30:28 PM
I just did that and no difference in adobe

when I open the file on apple preview and press info I can get Canon EF 35-80 f4-5.6

but this also is shown without the side car file present.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 19, 2019, 09:49:34 PM
So, i wrote a simple dumper script and tested all the adapters i found with it:

Code: [Select]
===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:51:36 ----> no chip at all (Samyang 85/1.4)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.name=
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.focal_length()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:37 - lens.hyperfocal()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.dof_near()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.dof_far()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:51:38 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:52:55 ----> with AF chip (M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:55 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:52:56 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:53:16 ----> with AF chip (M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:16 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:53:42 ----> Sigma 50/2.8 DG Macro (AF turned on)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.name=EF50mm f/2.5 Compact Macro
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.hyperfocal()=30888
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.dof_near()=-340372800
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.af()=true
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:42 - lens.af_mode()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:53:45 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:54:29 ----> with AF chip (praktica B -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.focal_length()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.hyperfocal()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.dof_near()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.dof_far()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:29 - lens.autofocus()=false

===============================================================================
ML/SCRIPTS/ADAPTE~1.LUA - 2019-1-18 23:54:53 ----> with AF chip (another M42 -> EOS)
===============================================================================

ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:53 - lens.name=1-65535mm
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:53 - lens.focal_length()=50
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.focus_distance()=0
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.hyperfocal()=61676
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.dof_near()=561148697
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.dof_far()=1000000
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.af()=false
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.af_mode()=3
ML/SCRIPTS/ADAPTE~1.LUA: 2019-01-18 23:54:54 - lens.autofocus()=false
Thank You for testing more adapters. Here is one of mine (second one has a broken contact so no chip doesn't work anymore, anyway basically info):
Code: [Select]
===============================================================================
ML/SCRIPTS/LENSDUMP.LUA - 2019-1-19 20:44:31  => PK -> EOS  and similiar to C/Y -> EOS
===============================================================================

lens.name=1-65535mm
lens.focal_length()=50
lens.focus_distance()=0
lens.hyperfocal()=94084
lens.dof_near()=-1735480800
lens.dof_far()=1000000lens.af()=false
lens.af_mode()=771
lens.autofocus()=false
It's the same with mine, so the method used until now is ok :)

I've just found what was wrong when setting params for my Samyang - lens.serial is expected to be integer, but this lens has string..

As a workaround for the serial number i've renamed "serial" in lenses{} to serialN, which is handled as string() by default, so i'm able to access it in xmp() and save. I suppose, that the serial==number() limitation is hardcoded somewhere in ML sources, right?
That's something @g3gg0 may review for us, as he worked here (https://bitbucket.org/hudson/magic-lantern/commits/396a8a9aa39cfc43e2736ada15d8a1e405f69d47?at=lua_fix) to adapt serial number conversion.
Don't know what is best to do here. Better to to wait for him to read this post.

I'm finally able to write the xmp file. Unfortunately only to the root of SD card - for some reason it still crashes when i try to use the dryos path, even when it looks fine.
This is how I modified xml.lua to solve the error due to the refactor:
Code: [Select]
function xmp.get_sidecar_filename()
    return dryos.shooting_card.dcim_dir.path ..  dryos.image_prefix .. string.format("%04d", dryos.shooting_card.file_number) .. ".XMP"
end
I'm finally able to write the xmp file. Unfortunately only to the root of SD card - for some reason it still crashes when i try to use the dryos path, even when it looks fine.
Could someone with LR/PS/anything else verify that this format is acceptable by software other than darktable&gimp? ;)
Adobe Bridge/Photoshop will show correct info for lens make,model and focal length
Serial number as 630312148, max aperture value as f/1.0

I've added four more params to lenses{}:
1] max_aperture and min_aperture, which are used to trim down the list of available Fnumbers plus they'll end up in the <exif:M[in|ax]ApertureValue>F22</exif:M[in|ax]ApertureValue> elements

2] FminL, FmaxL - apertures on wide/long ends of zoom lenses if they differ. Used in LensSpecification field.

Btw: i've checked exif data of my pics and the Exif.Photo.LensModel is really just a model without the lens brand (pics taken with trioplan were labeled by my old bash script)
Thinked before about that, but then added the "f_values" attribute to lenses, which enable the possibility to list only correct aperture for the selected lens, so I decided to don't introduce a redundant attributes.

Was thinking about to retrieve first and last values from the Manual Lens menu Aperture choice (which is updated from f_values if available otherwise use Fnumbers predefined list), to use as aperture_min and aperture_max.

I haven't exactly understood what FminL, FmaxL are for.

You remembered me to update xmp file to have all the values stored in ELNS block. So currently i have:
Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FNumber="4/10"
    exif:MinApertureValue="22/10"
    aux:Lens="My Lens"
    xmp:Label="test"
    exif:FocalLength="100/1"
    aux:SerialNumber="0"
    exif:MaxApertureValue="1.4/10">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
Mainly missing Focal Length range which is already handled by the lens.lua script, so I have only to add property to xmp

I'm trying to implement an idea to the ui.lua, which could allow to read all the lens name even if too long.
Not completely sure about splitting lens name into "make" and "model".

@a1ex When I try to backing out b0a2f95 using Sourcetree, it end up in duplicating the branch... so tried to do it manually and verified "Small-malloc test" and raw recording (this is not working good in last build due to the missing commit; camera freezes) with are working good (not sure about the counter next to the coloured camera icon while recording, it print some garbage to me instead of a recording timer).

I have in local all the changes. Maybe I can try upload a build in next days if I have time to finish experimenting with it
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 20, 2019, 12:26:50 AM
I did couple more facepalms in past few hours and it seems, that io.open() crashes when there is a CR2 as a part of the filename. I have no idea why - the debug console gets wiped out by assert crash before i can notice anything :(

So, with your code, i am able to write the xmp file to dcim...
Code: [Select]
function xmp.get_sidecar_filename() -- {{{
        local path = dryos.shooting_card.dcim_dir.path ..  dryos.image_prefix .. string.format("%04d", dryos.shooting_card.file_number) .. ".XMP"
        xmplog:write("Path:"..path)
        return path
end -- }}}

I'm using this page as a reference for exif data and the LensSerialNumber should be just a plain ascii string:
http://www.exiv2.org/tags.html

My sidecar now looks like this:
Code: [Select]
<?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="MagicLantern">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
   xmlns:exif="http://ns.adobe.com/exif/1.0/"
   xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
   exif:ExifVersion="0230"
   aux:Lens="Helios 44-2 58mm f/2"
   exif:LensSerialNumber="7765411"
   exif:LensSpecification="1/1 65535/1 0/1 0/1"
   exif:LensMake="Helios"
   exif:MinApertureValue="97/1"
   exif:LensModel="Helios 44-2 58mm f/2"
   exif:FNumber="14/10"
   exif:MaxApertureValue="97/1"
   exif:FocalLength="58/1">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Unfortunately darktable completely ignores the provided lens name and shows some default Canon set lenses.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on January 20, 2019, 12:38:09 AM
Maybe try with a more common lens name to see if its picket up. 

This one is normally pocket up by adobe and auto sets the lens corrections.

lens.name = "Zeiss Distagon T* 2/25 ZF.2"
lens.focal_length = 25
lens.aperture = 20

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on January 20, 2019, 12:40:19 AM
@dfort maybe you know something about this? you know most things nowadays :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 20, 2019, 10:52:19 PM
It seems, that guys grom darktable insist on using originalfilename.suffix.xmp naming for xmp sidecar files :( (there is a bug opened eight years ago about this - https://redmine.darktable.org/issues/8403)

And canon/ml/lua refuses to write a file with two suffixes (at least in my case).
Code: [Select]
-- test

f = io.open("test.XMP", "a")
f:write("bleble")
f:close()
The above works fine, but when i try io.open("test.CR2.XMP") to make darktable happy, the io operation fails and subsequent f:write() crashes with "attempt to index a nil value (global 'f')".

and dryos.rename("TEST.XMP", "TEST.CR2.XMP") does not work either.

looks like i gotta slam my head against the wall a little more :)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on January 20, 2019, 10:54:34 PM
you can always batch rename them on the computer right ?
As long as the fie has the info you need
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 20, 2019, 11:15:06 PM
you can always batch rename them on the computer right ?
As long as the fie has the info you need

Of course :) But the more stuff can be done automatically by the camera, the better for me.. I'll give it a shot with rawtherapee, but i haven't used it before, so i may fail (again).
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 21, 2019, 10:22:42 AM
Of course :) But the more stuff can be done automatically by the camera, the better for me.. I'll give it a shot with rawtherapee, but i haven't used it before, so i may fail (again).

Well, rawtherapee seems to ignore the xmp file altogether :) And the saga continues..

Btw: is there any chance to get rid of those memory-related crashes after each shot?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 21, 2019, 02:38:38 PM
Btw: is there any chance to get rid of those memory-related crashes after each shot?
Maybe backing out b0a2f95 (i.e. re-applying 065ceae) could help.
You can find the build I have in local at this download page (https://bitbucket.org/aprofiti/magic-lantern/downloads/) (for 5D3,5D2,7D,6D,50D)
Should also avoid camera freezes while recording raw videos.

I was trying to make lens serial number as a string in lua, but I get a crash log with the modified lua_lens.c:
Code: [Select]
/***
 Lens functions

 @author Magic Lantern Team
 @copyright 2014
 @license GPL
 @module lens
 */

#include <dryos.h>
#include <string.h>
#include <lens.h>
#include <focus.h>
#include <module.h>
#include "lua_common.h"

static int luaCB_lens_index(lua_State * L)
{
    LUA_PARAM_STRING_OPTIONAL(key, 2, "");
    /// Get/Set the name of the lens (reported by the lens)
    // @tfield string name
    if(!strcmp(key, "name")) lua_pushstring(L, lens_info.name);
    /// Get the lens id
    // @tfield int id readonly
    else if(!strcmp(key, "id")) lua_pushinteger(L, lens_info.lens_id);
    /// Get the serial number of the lens
    // @tfield string serial
    else if(!strcmp(key, "serial")) lua_pushstring(L, lens_info.lens_serial);
    /// Get/Set the focal length of the lens (in mm)
    // @tfield int focal_length
    else if(!strcmp(key, "focal_length")) lua_pushinteger(L, lens_info.focal_len);
    /// Get/Set the minimum focal length of the lens (in mm)
    // @tfield int focal_min
    else if(!strcmp(key, "focal_min")) lua_pushinteger(L, lens_info.lens_focal_min);
    /// Get/Set the maximum focal length of the lens (in mm)
    // @tfield int focal_max
    else if(!strcmp(key, "focal_max")) lua_pushinteger(L, lens_info.lens_focal_max);
    /// Get the current focus distance (in mm). Only updated in LiveView.
    // @tfield int focus_distance readonly
    else if(!strcmp(key, "focus_distance")) lua_pushinteger(L, lens_info.focus_dist * 10);
    /// Get the raw relative focus motor position, in steps.
    /// This counter is 0 at camera startup, its range depend on the lens,
    /// and is updated only when the focus motor moves. It will lose track
    /// of the lens position during manual focus, unless you use a focus-by-wire lens.
    /// Details: [www.magiclantern.fm/forum/index.php?topic=4997](http://www.magiclantern.fm/forum/index.php?topic=4997).
    // @tfield int focus_pos readonly
    else if(!strcmp(key, "focus_pos")) lua_pushinteger(L, lens_info.focus_pos);
    /// Get the hyperfocal distance of the lens (in mm). Only updated in LiveView.
    ///
    /// Computed from focal length, focus distance and aperture, see Focus -> DOF Settings menu for options.
    // @tfield int hyperfocal readonly
    else if(!strcmp(key, "hyperfocal")) lua_pushinteger(L, lens_info.hyperfocal);
    /// Get the distance to the DOF near (in mm). Only updated in LiveView.
    ///
    /// Computed from focal length, focus distance and aperture, see Focus -> DOF Settings menu for options.
    // @tfield int dof_near readonly
    else if(!strcmp(key, "dof_near")) lua_pushinteger(L, lens_info.dof_near);
    /// Get the distance to the DOF far (in mm). Only updated in LiveView.
    ///
    /// Computed from focal length, focus distance and aperture, see Focus -> DOF Settings menu for options.
    // @tfield int dof_far readonly
    else if(!strcmp(key, "dof_far")) lua_pushinteger(L, lens_info.dof_far);
    /// Get whether or not auto focus is enabled.
    // @tfield bool af readonly
    else if(!strcmp(key, "af")) lua_pushboolean(L, !is_manual_focus());
    /// Get the current auto focus mode (may be model-specific, see PROP\_AF\_MODE in property.h).
    // @tfield int af_mode readonly
    else if(!strcmp(key, "af_mode")) lua_pushinteger(L, af_mode);
    /// Get whether the lens is currently autofocusing.
    ///
    /// This does not include manual lens movements from lens.focus or ML follow focus -
    /// only movements from Canon autofocus triggered by half-shutter / AF-ON / * button.
    /// It is updated several ms (sometimes hundreds of ms) after the half-shutter event.
    ///
    /// On cameras with continuous autofocus, the return value is unknown - please report.
    ///
    /// Known not to work on EOS M.
    // @tfield bool autofocusing readonly
    else if(!strcmp(key, "autofocusing")) lua_pushboolean(L, lv_focus_status == 3);
    /// Use to manually set the lens aperture value for non-chipped lenses (for metadata purposes)
    // @tfield bool manual_aperture
    else if(!strcmp(key, "manual_aperture")) lua_pushnumber(L, lens_info.aperture / 10.0);
    /// Use to manually set the len's minumum aperture value for non-chipped lenses (for metadata purposes)
    // @tfield bool aperture_min
    else if(!strcmp(key, "aperture_min")) lua_pushnumber(L, RAW2VALUE(aperture, lens_info.raw_aperture_min) / 10.0);
    /// Use to manually set the len's maximum aperture value for non-chipped lenses (for metadata purposes)
    // @tfield bool aperture_max
    else if(!strcmp(key, "aperture_max")) lua_pushnumber(L, RAW2VALUE(aperture, lens_info.raw_aperture_max) / 10.0);
    /// Get if the lens chipped
    // @tfield bool is_chipped readonly
    else if(!strcmp(key, "is_chipped")) lua_pushboolean(L, lens_info.lens_exists);
    else lua_rawget(L, 1);
    return 1;
}

static int luaCB_lens_newindex(lua_State * L)
{
    LUA_PARAM_STRING_OPTIONAL(key, 2, "");
    if(!strcmp(key, "name"))
    {
        LUA_PARAM_STRING(value, 3);
        strncpy(lens_info.name, value, sizeof(lens_info.name)-1);
    }
    else if(!strcmp(key, "exists"))
    {
        LUA_PARAM_BOOL(value, 3);
        lens_info.lens_exists = value;
    }
    else if(!strcmp(key, "focal_length"))
    {
        LUA_PARAM_INT(value, 3);
        lens_info.focal_len = value;
    }
    else if(!strcmp(key, "focal_min"))
    {
        LUA_PARAM_INT(value, 3);
        lens_info.lens_focal_min = value;
    }
    else if(!strcmp(key, "focal_max"))
    {
        LUA_PARAM_INT(value, 3);
        lens_info.lens_focal_max = value;
    }
    else if(!strcmp(key, "manual_aperture"))
    {
        LUA_PARAM_NUMBER(value, 3);
        lens_info.aperture = (int)(value * 10);
        lens_info.raw_aperture = VALUE2RAW(aperture, lens_info.aperture);
    }
    else if(!strcmp(key, "aperture_min"))
    {
        LUA_PARAM_NUMBER(value, 3);
        int tmp = (int)(value * 10);
        lens_info.raw_aperture_min = VALUE2RAW(aperture, tmp);
    }
    else if(!strcmp(key, "aperture_max"))
    {
        LUA_PARAM_NUMBER(value, 3);
        int tmp = (int)(value * 10);
        lens_info.raw_aperture_max = VALUE2RAW(aperture, tmp);
    }
    else if(!strcmp(key, "serial"))
    {
        LUA_PARAM_STRING(value, 3);
        strncpy((uint64_t)lens_info.lens_serial, value, sizeof(lens_info.lens_serial));
    }
    else if(!strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af") || !strcmp(key, "is_chipped"))
    {
        return luaL_error(L, "'%s' is readonly!", key);
    }
    else
    {
        lua_rawset(L, 1);
    }
    return 0;
}

/***
 Moves the focus motor a specified number of steps.

 Only works in LiveView.
 @tparam int num_steps How many steps to move the focus motor (signed).
 @tparam[opt=2] int step_size Allowed values: 1, 2 or 3.

 Step 1 may give finer movements, but may get stuck or may be very slow on some lenses.
 @tparam[opt=true] bool wait Wait until each focus command finishes, before queueing others.

 wait=false may give smoother movements, but may no longer return accurate status for each command,
 and is known to **crash** on some cameras. The exact behavior may be camera- or lens-dependent.

 __Do not disable it without a good reason!__

 @tparam[opt] int delay Delay between focus commands (ms)

 With wait=true, the delay is after each focus command is executed (as reported by Canon firmware).

 With wait=false, the delay is after each focus command is started (without waiting for it to finish).

 (_default_ 0 if wait=true, 30ms if wait=false)


 @treturn bool whether the operation was successful or not.
 @function focus
 */
static int luaCB_lens_focus(lua_State * L)
{
    LUA_PARAM_INT(num_steps, 1);
    LUA_PARAM_INT_OPTIONAL(step_size, 2, 2);
    LUA_PARAM_BOOL_OPTIONAL(wait, 3, true);
    LUA_PARAM_INT_OPTIONAL(delay, 4, wait ? 0 : 30);

    if (!lv) return luaL_error(L, "lens.focus() only works in LiveView.");
    if (is_manual_focus()) return luaL_error(L, "lens.focus() requires autofocus enabled.");
    if (is_continuous_af()) return luaL_error(L, "lens.focus() requires %s AF disabled.", is_movie_mode() ? "movie servo" : "continuous");

    lua_pushboolean(L, lens_focus(num_steps, step_size, wait, delay));

    return 1;
}

static int wait_focus_status(int timeout, int value1, int value2)
{
    int t0 = get_ms_clock();

    while (get_ms_clock() - t0 < timeout)
    {
        msleep(10);

        if (lv_focus_status == value1 || lv_focus_status == value2)
        {
            return 1;
        }
    }
    return 0;
}

/***
 Performs autofocus, similar to half-shutter press.

 Works in both LiveView and in plain photo mode.
 @treturn bool whether the operation was successful or not.
 @function autofocus
 */
static int luaCB_lens_autofocus(lua_State * L)
{
    int focus_command_sent = 0;

    if (is_manual_focus())
    {
        goto error;
    }

    /* these models won't AF with half-shutter in LiveView */
    int back_btn_af_lv = lv && (
        is_camera("5D2", "*") ||
        is_camera("50D", "*") ||
        is_camera("500D", "*")
    );

    if (back_btn_af_lv)
    {
        /* FIXME: this method fails on 60D, why? */
        int af_request = 1;
        prop_request_change(PROP_REMOTE_AFSTART_BUTTON, &af_request, 4);
    }
    else
    {
        lens_setup_af(AF_ENABLE);
        module_send_keypress(MODULE_KEY_PRESS_HALFSHUTTER);
    }

    focus_command_sent = 1;

    if (!lv)
    {
        for (int i = 0; i < 20; i++)
        {
            msleep(100);

            /* FIXME: this may fail on recent models where trap focus is not working */
            if (get_focus_confirmation())
            {
                goto success;
            }
        }

        goto error;
    }

    /* 3 = focusing, 1 = idle (most models), 2 = idle (100D) */
    if (wait_focus_status(1000, 3, 3))
    {
        if (wait_focus_status(5000, 1, 2))
        {
            goto success;
        }
        else
        {
            /* timeout */
            printf("[%s] focus status: %d (expected 1 or 2)\n", lua_get_script_filename(L), lv_focus_status);
            goto error;
        }
    }
    printf("[%s] focus status: %d (expected 3)\n", lua_get_script_filename(L), lv_focus_status);

error:
    lua_pushboolean(L, false);
    goto cleanup;

success:
    lua_pushboolean(L, true);
    goto cleanup;

cleanup:
    if (focus_command_sent)
    {
        if (back_btn_af_lv)
        {
            int af_request = 0;
            prop_request_change(PROP_REMOTE_AFSTART_BUTTON, &af_request, 4);
        }
        else
        {
            module_send_keypress(MODULE_KEY_UNPRESS_HALFSHUTTER);
            lens_cleanup_af();
        }
    }
    return 1;
}

static const char * lua_lens_fields[] =
{
    "name",
    "focal_length",
    "focus_distance",
    "hyperfocal",
    "dof_near",
    "dof_far",
    "af",
    "af_mode",
    NULL
};

static const luaL_Reg lenslib[] =
{
    { "focus",      luaCB_lens_focus },
    { "autofocus",  luaCB_lens_autofocus },
    { NULL, NULL }
};

LUA_LIB(lens)
Lens.lua
Code: [Select]
lenses =
{
...

    { name = "My Zoom Lens", focal_length = 105, manual_aperture = 4, focal_min = 70, focal_max = 200, serial = "123456789" },

...
}
function reset_lens_values()
...

  lens.serial = ""

...
end

The log I get:
Code: [Select]
[43909204] lua_script_task: NULL PTR (34333231,e1a00000)
pc=     180 lr= a21fe80 stack=1ae5b0+0x10000
entry=bc5ff0(984464)
e1a00000 38373635 e59ff014 e59ff014
e59ff014 e1a00000 e59ff010 e59ff010

Magic Lantern version : Nightly.2019Jan21.50D109
Mercurial changeset   : b1818aa3ae43+ (manual_lens_info) tip
Built on 2019-01-21 12:37:46 UTC by alex@PCAlessandro.
Free Memory  : 227K + 2617K

@a1ex how can I interpret these crash logs to get more infos?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 21, 2019, 03:24:53 PM
You can find the build I have in local at this download page (https://bitbucket.org/aprofiti/magic-lantern/downloads/) (for 5D3,5D2,7D,6D,50D)
Should also avoid camera freezes while recording raw videos.

Awesome, thank you. I'll give it a shot when i'm back home.

In the meantime i've moved a bit forward with getting the lens model accepted by darktable and found some strange behavior of other exif metadata in dt->gimp process.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on January 21, 2019, 05:06:55 PM
@a1ex how can I interpret these crash logs to get more infos?

The above shows a null pointer (i.e. lua_script_task wrote something to address 0). ML catches this issue a relatively long time after it happens (i.e. at the next task context switch). The mem_prot module attempts to pinpoint it, but... in this case, you are trying to debug dynamically allocated code (i.e. module code), so even if you know the exact PC, it won't help much. This kind of debugging is hard; I'd rather stick to printf.

For the general question, look here (http://www.magiclantern.fm/forum/index.php?topic=19933). It is possible to trace crashes that happen in Canon code, or in core ML code, but modules are not loaded at the same address every time. It may be possible to dump the linker information right before unloading TCC, but I didn't explore this route yet.

Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 21, 2019, 10:45:30 PM
Thanks for the non-crashing build, Alessandro. It's perfect!

(https://i.ibb.co/7vSWjfb/lucerna.jpg) (https://ibb.co/7vSWjfb) (https://i.ibb.co/DkmmsSH/lucerna.png) (https://ibb.co/DkmmsSH)


Unfortunately the focal length and aperture values come from exif data embeded in cr2, so it shows the defaul values from the adapter. i haven't tried the non-chipped version yet.

At the moment i'm using this xmp output (the exifEX:LensModel is the right field for DarkTable...):
Code: [Select]
<?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="MagicLantern">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
   xmlns:exif="http://ns.adobe.com/exif/1.0/"
   xmlns:exifEX="http://cipa.jp/exif/1.0/"
   xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
   exif:ExifVersion="0230"
   exif:FNumber="20/10"
   exif:ApertureValue="20/10"
   exif:LensModel="44-2 58mm f/2"
   exif:Lens="Helios 44-2 58mm f/2"
   exif:MaxApertureValue="2./1"
   exif:LensSerialNumber="7765411"
   aux:Lens="Helios 44-2 58mm f/2"
   exif:LensSpecification="1/1 65535/1 2./1 2./1"
   exif:FocalLength="580/10"
   exif:LensMake="Helios"
   exif:MinApertureValue="16/1"
   exifEX:LensModel="Helios 44-2 58mm f/2">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>

Funny thing is, that when i use gimp to open cr2, it calls darktable, which then passes the processed photo back to gimp and save it as jpg, this jpeg has correct focal length and aperture in Exif.Photo. Unfortunately the Exif.Canon* fields contain 50/1.4 as reported by the adapter.

Code: [Select]
[zlost@nb-zimmermann2 ~]$ exiv2 -g Exif /media/zlost/EOS_6D/DCIM/101CANON/test.jpg |grep -i -e focal -e FNumber -e aperture|sort
Exif.CanonCs.DisplayAperture                 Short       1  0
Exif.CanonCs.MaxAperture                     Short       1  F1.4
Exif.CanonCs.MinAperture                     Short       1  F1.4
Exif.Canon.FocalLength                       Short       4  50.0 mm
Exif.CanonSi.ApertureValue                   Short       1  F1.4
Exif.CanonSi.TargetAperture                  Short       1  F1.4
Exif.Photo.ApertureValue                     Rational    1  F2
Exif.Photo.FNumber                           Rational    1  F2
Exif.Photo.FocalLength                       Rational    1  58.0 mm
Exif.Photo.FocalPlaneResolutionUnit          Short       1  inch
Exif.Photo.FocalPlaneXResolution             Rational    1  3810.58
Exif.Photo.FocalPlaneYResolution             Rational    1  3815.9
[zlost@nb-zimmermann2 ~]$

To ease my struggle with finding the right field to use for lens name, which will be accepted by darktable, i've modded xmp.lua, so the properties name can be a string and table/array as well to generate three lines with a single call to add_property() (feel free to call me lazy ;) )

Code: [Select]
xmp.lens_make           = { name = "exif:LensMake",             format = "%s" } -- Exif.Photo.LensMake  Ascii  This tag records the lens manufactor as an ASCII string.
xmp.lens_name           = { name = "exif:LensModel",            format = "%s" } -- Exif.Photo.LensModel  Ascii  This tag records the lens's model name and model number as an ASCII string
xmp.lens                = { name = {"aux:Lens","exif:Lens","exifEX:LensModel"},         format = "%s" }
xmp.lens_serial         = { name = "exif:LensSerialNumber",     format = "%s" } -- Exif.Photo.LensSerialNumber  Ascii  This tag records the serial number of the interchangeable lens that was used in photography as an ASCII string.
xmp.focal_length        = { name = "exif:FocalLength",          format = "%d/10" }
xmp.FNumber             = { name = "exif:FNumber",              format = "%d/10" }
xmp.apertureValue       = { name = "exif:ApertureValue",        format = "%d/10" } -- Exif.Photo.ApertureValue  Rational  The lens aperture. The unit is the APEX value.
xmp.MinApertureValue    = { name = "exif:MinApertureValue",     format = "%s/1" }
xmp.MaxApertureValue    = { name = "exif:MaxApertureValue",     format = "%s/1" } -- Exif.Photo.MaxApertureValue  Rational  The smallest F number of the lens. The unit is the APEX value. Ordinarily it is given in the range of 00.00 to 99.99, but it is not limited to this range.
xmp.lens_specification  = { name = "exif:LensSpecification",    format = "%s" } -- Exif.Photo.LensSpecification  Rational  This tag notes minimum focal length, maximum focal length, minimum F number in the minimum focal length, and minimum F number in the maximum focal length, which are specification information for the lens that was used in photography. When the minimum F number is unknown, the notation is 0/0

and the way of generating these got a bit more complicated as well..
Code: [Select]
function add_properties() -- {{{
        xmp:add_property(xmp.lens_make,                 function() return (lens.make or "-- unknown --") end)
        xmp:add_property(xmp.lens_name,                 function() return lens.name end)
        xmp:add_property(xmp.lens,                      function()
                        if lens.make ~= nil then
                                return lens.make.." "..lens.name
                        else
                                return lens.name
                        end
                end)
        xmp:add_property(xmp.focal_length,              function() return (lens.focal_length*10) end)
        xmp:add_property(xmp.apertureValue,             function() return (lens.manual_aperture*10) end)
        xmp:add_property(xmp.FNumber,                   function() return (lens.manual_aperture*10) end)
        xmp:add_property(xmp.MaxApertureValue,          function()
                        local max_aperture      = (lens.max_aperture or lens.manual_aperture)
                        if lens.f_values ~= nil then
                                local Fns       = lens.f_values
                                max_aperture    = Fns[1]
                        end
                        return max_aperture
                end)
        xmp:add_property(xmp.MinApertureValue,          function()
                        local min_aperture      = (lens.min_aperture or lens.manual_aperture)
                        if lens.f_values ~= nil then
                                local Fns       = lens.f_values
                                min_aperture    = Fns[#Fns]
                        end
                        return min_aperture
                end)
        xmp:add_property(xmp.lens_serial,               function() return lens.serialN end)
        xmp:add_property(xmp.lens_specification,        function()
                        local focal_min = (lens.focal_min or lens.focal_length)
                        local focal_max = (lens.focal_max or lens.focal_length)
                        local FminL     = (lens.FminL or lens.manual_aperture)
                        local FmaxL     = (lens.FmaxL or lens.manual_aperture)
                        return string.format("%s/1 %s/1 %s/1 %s/1", tostring(focal_min), tostring(focal_max), tostring(FminL), tostring(FmaxL))
                end)
end -- }}}

add_properties()

The unfortunate fact is, that the min/max aperture values and lens_specs don't work the way i want them to. But technically those are not important at all for normal people..
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Walter Schulz on January 21, 2019, 10:56:17 PM
Well, rawtherapee seems to ignore the xmp file altogether :)

Have you contacted rawtherapee dev(s) about this? In the past we had some friendly contact with user heckflosse to solve some issues.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 21, 2019, 11:02:21 PM
Have you contacted rawtherapee dev(s) about this? In the past we had some friendly contact with user heckflosse to solve some issues.

Not yet. This was like the second or third time i've run it so far. But it might be a nice feature to have the output format configurable for individual editors (including the filename, but in DT case it would require some troubleshooting why the heck io.* crashes when working on files with two suffixes).

Seems like we'd need completely different format.. But it might be doable as well ;) I'll play with it a bit tomorrow..
(this is a part of pp3 file used as meta storage by RT, which may be of some use for us)
Code: [Select]
[LensProfile]
LcMode=lfauto
LCPFile=
UseDistortion=true
UseVignette=true
UseCA=true
LFCameraMake=
LFCameraModel=
LFLens=

[MetaData]
Mode=1

[Exif]
Exif.UserComment=user comment. blabla

[IPTC]
Caption=iptc description...;
CaptionWriter=;
Category=;
City=;
Copyright=;
Country=;
Creator=;
CreatorJobTitle=;
Credit=;
DateCreated=;
Headline=;
Instructions=;
Province=;
Source=;
Title=;
TransReference=;
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 22, 2019, 10:04:35 AM
Not yet. This was like the second or third time i've run it so far. But it might be a nice feature to have the output format configurable for individual editors (including the filename, but in DT case it would require some troubleshooting why the heck io.* crashes when working on files with two suffixes).

Seems like we'd need completely different format.. But it might be doable as well ;) I'll play with it a bit tomorrow..
(this is a part of pp3 file used as meta storage by RT, which may be of some use for us)
Code: [Select]
[LensProfile]
LcMode=lfauto
LCPFile=
UseDistortion=true
UseVignette=true
UseCA=true
LFCameraMake=
LFCameraModel=
LFLens=

I've just tested it and the LCCamera and LFLens fields are used to store the data related to chosen correction profile, but i assume, that these are used after the profile is selected by user or when there is a profile matching Lens.Name from Exif.

So i'm afraid, that as long as we don't have the "faked" data in exif, this is a not going to help.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 22, 2019, 08:00:34 PM
Thanks for the non-crashing build, Alessandro. It's perfect!
Happy to hear this!
Please report any strange behaviour if you plan to use it in the next days.

To ease my struggle with finding the right field to use for lens name, which will be accepted by darktable, i've modded xmp.lua, so the properties name can be a string and table/array as well to generate three lines with a single call to add_property()
Nice idea! May be useful to make code a little bit shorter if we add the new tags. Will look into it :)

At the moment i'm using this xmp output (the exifEX:LensModel is the right field for DarkTable...)
Would like to know if inserting the same information but with different namespace can make any editing software angry or not, as what I understood from your post, each software looks at different tags in the xmp.
Is possible to make a short recap on which tags each software use?

We could fill .xmp using different namespace (aux, exif, exifEx ) for the same information (name, aperture...) If there aren't any problems.

Code: [Select]
xmp.lens_specification  = { name = "exif:LensSpecification",    format = "%s" } -- Exif.Photo.LensSpecification  Rational  This tag notes minimum focal length, maximum focal length, minimum F number in the minimum focal length, and minimum F number in the maximum focal length, which are specification information for the lens that was used in photography. When the minimum F number is unknown, the notation is 0/0
Code: [Select]
xmp:add_property(xmp.lens_specification,        function()
                        local focal_min = (lens.focal_min or lens.focal_length)
                        local focal_max = (lens.focal_max or lens.focal_length)
                        local FminL     = (lens.FminL or lens.manual_aperture)
                        local FmaxL     = (lens.FmaxL or lens.manual_aperture)
                        return string.format("%s/1 %s/1 %s/1 %s/1", tostring(focal_min), tostring(focal_max), tostring(FminL), tostring(FmaxL))
                end)
Can't find this in exiftool's tag list (https://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/EXIF.html). It may be this instead:
Code: [Select]
0xa432 LensInfo rational64u[4] ExifIFD (4 rational values giving focal and aperture ranges, called LensSpecification by the EXIF spec.)
Found other example in other forum which use "exifEx:"... Which one is to use?

The unfortunate fact is, that the min/max aperture values and lens_specs don't work the way i want them to. But technically those are not important at all for normal people..
What do you need exactly?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 24, 2019, 02:56:54 PM
Happy to hear this!
Please report any strange behaviour if you plan to use it in the next days.

Well, i'm using my camera only for shooting - no high-level magic, so i'm afraid i'm of no use when it comes to testing..

Nice idea! May be useful to make code a little bit shorter if we add the new tags. Will look into it :)
Would like to know if inserting the same information but with different namespace can make any editing software angry or not, as what I understood from your post, each software looks at different tags in the xmp.
Is possible to make a short recap on which tags each software use?
We could fill .xmp using different namespace (aux, exif, exifEx ) for the same information (name, aperture...) If there aren't any problems.
Can't find this in exiftool's tag list (https://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/EXIF.html). It may be this instead:
Code: [Select]
0xa432 LensInfo rational64u[4] ExifIFD (4 rational values giving focal and aperture ranges, called LensSpecification by the EXIF spec.)
Found other example in other forum which use "exifEx:"... Which one is to use?

I'm using http://www.exiv2.org/tags.html and http://www.exiv2.org/tags-xmp-exifEX.html as a reference.. And i've only noticed the exifEx:LensModel in darktable forums, hence i do not know which other keys we could use for other editors.

What do you need exactly?

My aim is/was to "fake" as much data as possible and if there is a way we could make the camera believe, that these values came from the lenses directly instead, so they get saved with another exif data in the image header, then go for it. That would make them instantly available for all image editors at once..
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 25, 2019, 10:34:47 AM
At the moment i've got these:
Code: [Select]
-- predefined xmp properties
xmp.lens_make           = { name = {"exif:LensMake","exifEx:LensMake"},                         format = "%s" }         -- Exif.Photo.LensMake  Ascii  This tag records the lens manufactor as an ASCII string.
xmp.lens_name           = { name = {"exif:LensModel","exifEx:LensModel"},                       format = "%s" }         -- Exif.Photo.LensModel  Ascii  This tag records the lens's model name and model number as an ASCII string
xmp.lens                = { name = {"aux:Lens","exif:Lens"},                                    format = "%s" }
xmp.lens_serial         = { name = {"exif:LensSerialNumber","exifEx:LensSerialNumber"},         format = "%s" }         -- Exif.Photo.LensSerialNumber  Ascii  This tag records the serial number of the interchangeable lens that was used in photography as an ASCII string.
xmp.focal_length        = { name = "exif:FocalLength",                                          format = "%d/1" }       -- Exif.Image.FocalLength  Rational  The actual focal length of the lens, in mm.
xmp.FNumber             = { name = "exif:FNumber",                                              format = "%d/10" }      -- Exif.Image.FNumber  Rational  The F number.
xmp.apertureValue       = { name = "exif:ApertureValue",                                        format = "%d/10" }      -- Exif.Photo.ApertureValue  Rational  The lens aperture. The unit is the APEX value.
xmp.MinApertureValue    = { name = "exif:MinApertureValue",                                     format = "%s/10" }      -- Exif.Photo.MinApertureValue  Rational  The highest F number of the lens. The unit is the APEX value. Ordinarily it is given in the range of 00.00 to 99.99, but it is not limited to this range.
xmp.MaxApertureValue    = { name = "exif:MaxApertureValue",                                     format = "%s/10" }      -- Exif.Photo.MaxApertureValue  Rational  The smallest F number of the lens. The unit is the APEX value. Ordinarily it is given in the range of 00.00 to 99.99, but it is not limited to this range.
xmp.lens_specification  = { name = {"exif:LensInfo","exif:LensSpecification","exifEx:LensSpecification"},       format = "%s" }         -- Exif.Photo.LensSpecification  Rational  This tag notes minimum focal length, maximum focal length, minimum F number in the minimum focal length, and minimum F number in the maximum focal length, which are specification information for the lens that was used in photography. When the minimum F number is unknown, the notation is 0/0

And they are initialized with this:
Code: [Select]
function add_properties() -- {{{
        xmp:add_property(xmp.lens_make,                 function() return (lens.make or "-- unknown --") end)
        xmp:add_property(xmp.lens_name,                 function() return lens.name end)
        xmp:add_property(xmp.lens,                      function()
                        if lens.make ~= nil then
                                return lens.make.." "..lens.name
                        else
                                return lens.name
                        end
                end)
        xmp:add_property(xmp.focal_length,              function() return (lens.focal_length) end)
        xmp:add_property(xmp.apertureValue,             function() return (apex(lens.manual_aperture)*10) end)
        xmp:add_property(xmp.FNumber,                   function() return (lens.manual_aperture*10) end)
        xmp:add_property(xmp.MaxApertureValue,          function()
                        local max_aperture      = (lens.max_aperture or lens.manual_aperture)
                        if lens.f_values ~= nil then
                                local Fns       = lens.f_values
                                max_aperture    = Fns[1]
                        end
                        return (apex(max_aperture)*10)
                end)
        xmp:add_property(xmp.MinApertureValue,          function()
                        local min_aperture      = (lens.min_aperture or lens.manual_aperture)
                        if lens.f_values ~= nil then
                                local Fns       = lens.f_values
                                min_aperture    = Fns[#Fns]
                        end
                        return (apex(min_aperture)*10)
                end)
        xmp:add_property(xmp.lens_serial,               function() return lens.serialN end)
        xmp:add_property(xmp.lens_specification,        function()
                        local focal_min = (lens.focal_min or lens.focal_length)
                        local focal_max = (lens.focal_max or lens.focal_length)
                        local FminL     = (lens.FminL or lens.manual_aperture)
                        local FmaxL     = (lens.FmaxL or lens.manual_aperture)
                        return string.format("%s/1 %s/1 %s/1 %s/1", tostring(focal_min), tostring(focal_max), tostring(FminL), tostring(FmaxL))
                end)
end -- }}}
               
add_properties()

If we could generate more data in order to make the camera FW happy, i'll gladly do it :)

And as a bit of a motivation, here goes my beloved 50mm Trioplan...
(https://i.ibb.co/xzkq07D/img-0001.jpg) (https://ibb.co/xzkq07D)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 25, 2019, 06:57:17 PM
My aim is/was to "fake" as much data as possible and if there is a way we could make the camera believe, that these values came from the lenses directly instead, so they get saved with another exif data in the image header, then go for it. That would make them instantly available for all image editors at once..
According to this old post (https://www.magiclantern.fm/forum/index.php?topic=18083.msg173979#msg173979), writing to Canon data structure is considered dangerous .

Maybe @a1ex discovered some other possible methods to inject exif data to .cr2?
I imagine MPU is most responsible for this and it usually doesn't "collaborate"; so not much expectation from this at the moment.

By the way, I would like to have aperture overridden in Photo menu... it would be useful to remember and check what value selected from the script.

I've just found what was wrong when setting params for my Samyang - lens.serial is expected to be integer, but this lens has string.
Regarding managing serial number as string... I made some tests and got a half working version:
Code: [Select]
/* lua/lua_lens.c */

#include <bmp.h>
#include <inttypes.h>

// Retrive value
static int luaCB_lens_index(lua_State * L)
{

...

    /// Get the serial number of the lens
    // @tfield string serial
    //else if(!strcmp(key, "serial")) lua_pushstring(L, lens_info.lens_serial);
    else if(!strcmp(key, "serial"))
    {
      char buf[20];

      //  snprintf(buf, 17, "%#" PRIx64, (uint64_t)lens_info.lens_serial);
      snprintf(buf, 17, "%llu", lens_info.lens_serial);
      //  snprintf(buf, 17, "%#" PRIu64 , lens_info.lens_serial);

      bmp_printf( FONT(FONT_LARGE,COLOR_RED,COLOR_BG), 30, 250,"lens.serial 2: %llu",lens_info.lens_serial);
      bmp_printf( FONT(FONT_LARGE,COLOR_RED,COLOR_BG), 30, 300,"buff: %s",buf);

      lua_pushstring(L, buf);
    }

....

// Assign new value
static int luaCB_lens_newindex(lua_State * L)
{
  else if(!strcmp(key, "serial"))
    {
        //LUA_PARAM_INT(value, 3);
        //lens_info.lens_serial = value;

        LUA_PARAM_STRING(value, 3);
        lens_info.lens_serial = strtoul(value, NULL, 0);

        bmp_printf( FONT(FONT_LARGE,COLOR_RED,COLOR_BG), 30, 100,"value: %s",value);
        bmp_printf( FONT(FONT_LARGE,COLOR_RED,COLOR_BG), 30, 150,"lens.serial 1: %d", lens_info.lens_serial );
    }
This save correctly the number as unsigned long integer in ML lens_info data structs (can be seen in Lens Info debut menu), but then when I try to get it as a string from the lua script, it give me nothing usefull... like a empty string with the format specifier as prefix.

Code: [Select]
    { make = "Samyang", name = "85mm f/1.4 AS IF UMC", focal_length = 85, manual_aperture = 1.4}
      --  serial = "A217D0264" }
Also in your case it is a alphanumerics value and add support to this isn't straightforward... I can change "strtoul" arguments to parse number in hexadecimal base (0-9 , A-F) and try to adapt script, but would not cove all possible case... as example old canon lenses (http://alanwatsonforster.org/writing/canon-fd-lens-date-codes.html) use all of the letter from alphabet and other manufactures may do the same.

Given that, it may not be worth to manage serial in lua as a string; it's converted from number to string when saving .xmp, but actually it works only for decimal numbers.

@a1ex @dmilligan Any suggestion on what to do?
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: a1ex on January 25, 2019, 07:10:15 PM
If the question is about how to modify EXIF data in a CR2, then it all happens on the main CPU. Where exactly... I'm afraid I don't know.

It doesn't work in QEMU yet, but that's "just" a matter of implementing the LJ92/JPG encoders and other image processing paths used while creating the CR2. Once that works, you will be able to experiment with modified EXIF info.

Some of the metadata comes, indeed, from the MPU, but I'm pretty sure that can be faked somehow.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on January 25, 2019, 10:46:59 PM
I don't want to mess with images, but prefer to give the lens details to the camera itself and let it do its job (if possible, of course), which should be the safe way, i suppose. Theoreticaly the point is to emulate chipped lenses with lua in a way, that there is no difference in those for the original camera firmware.

The other way is to write a script, which will use exiv2/exiftool to process existing xmp files and store/update corresponding exif keys of matching files.

But a si wrote before - having the camera do this automatically is preferable way for a lazy ass like me :)

By the way, I would like to have aperture overridden in Photo menu... it would be useful to remember and check what value selected from the script.
You mean this one?

(https://i.ibb.co/G5LP574/lucerna-clona.jpg) (https://ibb.co/G5LP574)


Prob is, that each adapter may behave in a different way depending on presence and version of the chip. And we'd have to override this behavior by giving the camera a list of available apertures to choose from. Which may technically be the same as setting the lens name and focal length.

For the serial number - i had no idea it can get this complicated :) And in this area i'm of sub-zero use as i don't have experience with C..
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on January 26, 2019, 02:25:29 PM
If the question is about how to modify EXIF data in a CR2, then it all happens on the main CPU. Where exactly... I'm afraid I don't know.

It doesn't work in QEMU yet, but that's "just" a matter of implementing the LJ92/JPG encoders and other image processing paths used while creating the CR2. Once that works, you will be able to experiment with modified EXIF info..
Good to know! Thank you a1ex, maybe one day we will be able to experiment with it :)

You mean this one?

(https://i.ibb.co/G5LP574/lucerna-clona.jpg) (https://ibb.co/G5LP574)


Prob is, that each adapter may behave in a different way depending on presence and version of the chip. And we'd have to override this behavior by giving the camera a list of available apertures to choose from. Which may technically be the same as setting the lens name and focal length.
Yes, exactly. It’ the same problem as with lens data, overridding from cpu memory/code will possibly do the trick.

A temporary solution would be to use flexinfo (https://bitbucket.org/hudson/magic-lantern/src/7a3b5fa3f4c6f085e5bb9d58ab4da0002c01e4a3/src/flexinfo.c?at=unified&fileviewer=file-view-default#flexinfo.c-160) to print the aperture value on top of Canon’s text. Not very clean but doable if needed....
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 02, 2019, 11:59:40 PM
Hi, it's me again ;)

I've played with lens.lua + xmp.lua a bit and added one more submenu option to select desired output format of the XMP file. My aim was to generate exiv2 command line, which would set all the desired exif data for me.

So, when the lens is selected:
Code: [Select]
===============================================================================
ML/SCRIPTS/LENS.LUA - 2019-02-02 23:24:34
===============================================================================

ML/SCRIPTS/LENS.LUA: 2019-02-02 23:24:40 - Update_lens: name: 50-2 50mm f/3.5
ML/SCRIPTS/LENS.LUA: 2019-02-02 23:24:40 - Update_lens: serialN: 7149595
ML/SCRIPTS/LENS.LUA: 2019-02-02 23:24:40 - Update_lens: manual_aperture: 3.5
ML/SCRIPTS/LENS.LUA: 2019-02-02 23:24:40 - Update_lens: make: Industar
ML/SCRIPTS/LENS.LUA: 2019-02-02 23:24:40 - Update_lens: focal_length: 50

Then you can use this dropdown to choose the format. For exiv2 the "exiv2 ....magic... IMAGE.CR2" is appended to img_folder/MANUAL.SH, so you'd run only one script and all matching pics will be updated at once..
(https://i.ibb.co/4SWWwDq/magic-exiv2.jpg) (https://ibb.co/4SWWwDq)


And the result looks like this (i've just added a bit of logging to xmp.lua, so what is written to xmp file, get's also logged..):
Code: [Select]
===============================================================================
ML/SCRIPTS/LIB/xmp.lua - 2019-02-02 23:24:34
===============================================================================

ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:00 - Writing the xmp file...
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:00 - Path:B:/DCIM/101CANON/IMG_6864.XMP
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:02 - Writing XMP file B:/DCIM/101CANON/IMG_6864.XMP in 'default' format for IMG_6864.CR2
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:02 - <?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="MagicLantern">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
   xmlns:exif="http://ns.adobe.com/exif/1.0/"
   xmlns:exifEX="http://cipa.jp/exif/1.0/"
   xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
   exif:ExifVersion="0231"
   exif:MaxApertureValue="970/10"
   aux:Lens="Industar 50-2 50mm f/3.5"
   exif:FocalLength="50/1"
   exifEx:LensModel="50-2 50mm f/3.5"
   exifEx:LensSpecification="10/10 655350/10 14/10 14/10"
   exif:LensModel="50-2 50mm f/3.5"
   exif:LensMake="Industar"
   exif:FNumber="14/10"
   exif:ApertureValue="970/10"
   exif:LensSpecification="10/10 655350/10 14/10 14/10"
   exif:LensInfo="10/10 655350/10 14/10 14/10"
   exifEx:LensMake="Industar"
   exif:LensSerialNumber="7149595"
   exif:Lens="Industar 50-2 50mm f/3.5"
   exifEx:LensSerialNumber="7149595">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:02 - XMP file written.
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:20 - Writing the xmp file...
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:20 - Path:B:/DCIM/101CANON/IMG_6865.XMP
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:23 - Writing XMP file B:/DCIM/101CANON/IMG_6865.XMP in 'darktable' format for IMG_6865.CR2
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:23 - <?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="MagicLantern">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
   xmlns:exif="http://ns.adobe.com/exif/1.0/"
   xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
   xmlns:exifEX="http://cipa.jp/exif/1.0/"
   exif:ExifVersion="0231">
   <exif:MaxApertureValue>970/10</exif:MaxApertureValue>
   <aux:Lens>Industar 50-2 50mm f/3.5</aux:Lens>
   <exif:FocalLength>50/1</exif:FocalLength>
   <exifEx:LensModel>50-2 50mm f/3.5</exifEx:LensModel>
   <exifEx:LensSpecification>10/10 655350/10 14/10 14/10</exifEx:LensSpecification>
   <exif:LensModel>50-2 50mm f/3.5</exif:LensModel>
   <exif:LensMake>Industar</exif:LensMake>
   <exif:FNumber>14/10</exif:FNumber>
   <exif:ApertureValue>970/10</exif:ApertureValue>
   <exif:LensSpecification>10/10 655350/10 14/10 14/10</exif:LensSpecification>
   <exif:LensInfo>10/10 655350/10 14/10 14/10</exif:LensInfo>
   <exifEx:LensMake>Industar</exifEx:LensMake>
   <exif:LensSerialNumber>7149595</exif:LensSerialNumber>
   <exif:Lens>Industar 50-2 50mm f/3.5</exif:Lens>
   <exifEx:LensSerialNumber>7149595</exifEx:LensSerialNumber> 
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:25:23 - XMP file written.
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:27:40 - Writing the xmp file...
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:27:44 - Writing XMP file B:/DCIM/101CANON/manual.sh in 'exiv2' format for IMG_6866.CR2
ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:27:44 - exiv2 -M "set Exif.Photo.MaxApertureValue 970/10" -M "set Xmp.aux.Lens Industar 50-2 50mm f/3.5" -M "set Exif.Photo.FocalLength 50/1" -M "set Exif.Photo.LensModel 50-2 50mm f/3.5" -M "set Exif.Photo.LensSpecification 10/10 655350/10 14/10 14/10" -M "set Exif.Photo.LensModel 50-2 50mm f/3.5" -M "set Exif.Photo.LensMake Industar" -M "set Exif.Photo.FNumber 14/10" -M "set Exif.Photo.ApertureValue 970/10" -M "set Exif.Photo.LensSpecification 10/10 655350/10 14/10 14/10" -M "set Exif.Image.LensInfo 10/10 655350/10 14/10 14/10" -M "set Exif.Photo.LensMake Industar" -M "set Exif.Photo.LensSerialNumber 7149595" -M "set Xmp.aux.Lens Industar 50-2 50mm f/3.5" -M "set Exif.Photo.LensSerialNumber 7149595" IMG_6866.CR2


ML/SCRIPTS/LIB/xmp.lua: 2019-02-02 23:27:44 - XMP file written.

I've noticed couple things:

So at the moment i'm able to "fix" Exif.Image and Exif.Photo keys, but it seems, that i could also alter the Exif.Canon* stuff, so in the end the CR2/JPEG could contain completely valid set of metadata even for manual lenses ;)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 05, 2019, 03:51:55 PM
Looks like i'm finally there ;)

Exif data can be batch-updated with the exiv2 format of what used to be the XMP file:
Code: [Select]
cat MANUAL.SH
exiv2 -k -M"set Exif.CanonCs.LensType 65535" -M "set Exif.CanonCs.MinAperture 285" -M "set Exif.Photo.ApertureValue 322363/65536" -M "set Exif.CanonCs.Lens 180 180 1" -M "set Exif.Photo.LensSpecification 180/1 180/1 56/10 56/10" -M "set Exif.Photo.MaxApertureValue 322363/65536" -M "set Exif.Photo.LensSerialNumber 1728275" -M "set Exif.CanonSi.TargetAperture 157" -M "set Exif.Canon.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.FocalLength 180/1" -M "set Exif.CanonSi.ApertureValue 157" -M "set Exif.CanonCs.MaxAperture 157" -M "set Exif.Photo.FNumber 55/10" -M "set Xmp.aux.Lens Meyer-Optik Gorlitz Telemegor 180mm f/5.5" -M "set Exif.Image.LensInfo 180/1 180/1 56/10 56/10" -M "set Exif.Photo.LensMake Meyer-Optik Gorlitz" IMG_6899.CR2

exiv2 -k -M"set Exif.CanonCs.LensType 65535" -M "set Exif.CanonCs.MinAperture 285" -M "set Exif.Photo.ApertureValue 63626/65536" -M "set Exif.CanonCs.Lens 85 85 1" -M "set Exif.Photo.LensSpecification 85/1 85/1 14/10 14/10" -M "set Exif.Photo.MaxApertureValue 63626/65536" -M "set Exif.Photo.LensSerialNumber A217D0264" -M "set Exif.CanonSi.TargetAperture 31" -M "set Exif.Canon.LensModel 85mm f/1.4 AS IF UMC" -M "set Exif.Photo.LensModel 85mm f/1.4 AS IF UMC" -M "set Exif.Photo.FocalLength 50/1" -M "set Exif.CanonSi.ApertureValue 31" -M "set Exif.CanonCs.MaxAperture 31" -M "set Exif.Photo.FNumber 14/10" -M "set Xmp.aux.Lens Samyang 85mm f/1.4 AS IF UMC" -M "set Exif.Image.LensInfo 85/1 85/1 14/10 14/10" -M "set Exif.Photo.LensMake Samyang" IMG_6900.CR2

exiv2 -k -M"set Exif.CanonCs.LensType 65535" -M "set Exif.CanonCs.MinAperture 256" -M "set Exif.Photo.ApertureValue 131072/65536" -M "set Exif.CanonCs.Lens 58 58 1" -M "set Exif.Photo.LensSpecification 58/1 58/1 20/10 20/10" -M "set Exif.Photo.MaxApertureValue 131072/65536" -M "set Exif.Photo.LensSerialNumber 7765411" -M "set Exif.CanonSi.TargetAperture 64" -M "set Exif.Canon.LensModel 44-2 58mm f/2" -M "set Exif.Photo.LensModel 44-2 58mm f/2" -M "set Exif.Photo.FocalLength 58/1" -M "set Exif.CanonSi.ApertureValue 64" -M "set Exif.CanonCs.MaxAperture 64" -M "set Exif.Photo.FNumber 20/10" -M "set Xmp.aux.Lens Helios 44-2 58mm f/2" -M "set Exif.Image.LensInfo 58/1 58/1 20/10 20/10" -M "set Exif.Photo.LensMake Helios" IMG_6901.CR2

The result looks fine to me (and should be fully accepted by PS, Gimp, DarkTable or whichever jpg/raw editor (i hope))

Code: [Select]
exiv2 -g Lens -g Aperture -Pknyt IMG_6899.CR2 IMG_6900.CR2 IMG_6901.CR2
IMG_6899.CR2          Exif.Photo.ApertureValue                     ApertureValue               Rational   F5.5
IMG_6899.CR2          Exif.Photo.MaxApertureValue                  MaxApertureValue            Rational   F5.5
IMG_6899.CR2          Exif.CanonCs.LensType                        LensType                    Short      (65535)
IMG_6899.CR2          Exif.CanonCs.Lens                            Lens                        Short      180.0 mm
IMG_6899.CR2          Exif.CanonCs.MaxAperture                     MaxAperture                 Short      F5.5
IMG_6899.CR2          Exif.CanonCs.MinAperture                     MinAperture                 Short      F22
IMG_6899.CR2          Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
IMG_6899.CR2          Exif.CanonSi.TargetAperture                  TargetAperture              Short      F5.5
IMG_6899.CR2          Exif.CanonSi.ApertureValue                   ApertureValue               Short      F5.5
IMG_6899.CR2          Exif.Canon.LensModel                         LensModel                   Ascii      Telemegor 180mm f/5.5
IMG_6899.CR2          Exif.Photo.LensSpecification                 LensSpecification           Rational   180/1 180/1 56/10 56/10
IMG_6899.CR2          Exif.Photo.LensMake                          LensMake                    Ascii      Meyer-Optik Gorlitz
IMG_6899.CR2          Exif.Photo.LensModel                         LensModel                   Ascii      Telemegor 180mm f/5.5
IMG_6899.CR2          Exif.Photo.LensSerialNumber                  LensSerialNumber            Ascii      1728275
IMG_6899.CR2          Exif.Image.LensInfo                          LensInfo                    Rational   180/1 180/1 56/10 56/10
IMG_6899.CR2          Xmp.aux.Lens                                 Lens                        XmpText    Meyer-Optik Gorlitz Telemegor 180mm f/5.5

Code: [Select]
IMG_6900.CR2          Exif.Photo.ApertureValue                     ApertureValue               Rational   F1.4
IMG_6900.CR2          Exif.Photo.MaxApertureValue                  MaxApertureValue            Rational   F1.4
IMG_6900.CR2          Exif.CanonCs.LensType                        LensType                    Short      (65535)
IMG_6900.CR2          Exif.CanonCs.Lens                            Lens                        Short      85.0 mm
IMG_6900.CR2          Exif.CanonCs.MaxAperture                     MaxAperture                 Short      F1.4
IMG_6900.CR2          Exif.CanonCs.MinAperture                     MinAperture                 Short      F22
IMG_6900.CR2          Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
IMG_6900.CR2          Exif.CanonSi.TargetAperture                  TargetAperture              Short      F1.4
IMG_6900.CR2          Exif.CanonSi.ApertureValue                   ApertureValue               Short      F1.4
IMG_6900.CR2          Exif.Canon.LensModel                         LensModel                   Ascii      85mm f/1.4 AS IF UMC
IMG_6900.CR2          Exif.Photo.LensSpecification                 LensSpecification           Rational   85/1 85/1 14/10 14/10
IMG_6900.CR2          Exif.Photo.LensMake                          LensMake                    Ascii      Samyang
IMG_6900.CR2          Exif.Photo.LensModel                         LensModel                   Ascii      85mm f/1.4 AS IF UMC
IMG_6900.CR2          Exif.Photo.LensSerialNumber                  LensSerialNumber            Ascii      A217D0264
IMG_6900.CR2          Exif.Image.LensInfo                          LensInfo                    Rational   85/1 85/1 14/10 14/10
IMG_6900.CR2          Xmp.aux.Lens                                 Lens                        XmpText    Samyang 85mm f/1.4 AS IF UMC

Code: [Select]
IMG_6901.CR2          Exif.Photo.ApertureValue                     ApertureValue               Rational   F2
IMG_6901.CR2          Exif.Photo.MaxApertureValue                  MaxApertureValue            Rational   F2
IMG_6901.CR2          Exif.CanonCs.LensType                        LensType                    Short      (65535)
IMG_6901.CR2          Exif.CanonCs.Lens                            Lens                        Short      58.0 mm
IMG_6901.CR2          Exif.CanonCs.MaxAperture                     MaxAperture                 Short      F2
IMG_6901.CR2          Exif.CanonCs.MinAperture                     MinAperture                 Short      F16
IMG_6901.CR2          Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
IMG_6901.CR2          Exif.CanonSi.TargetAperture                  TargetAperture              Short      F2
IMG_6901.CR2          Exif.CanonSi.ApertureValue                   ApertureValue               Short      F2
IMG_6901.CR2          Exif.Canon.LensModel                         LensModel                   Ascii      44-2 58mm f/2
IMG_6901.CR2          Exif.Photo.LensSpecification                 LensSpecification           Rational   58/1 58/1 20/10 20/10
IMG_6901.CR2          Exif.Photo.LensMake                          LensMake                    Ascii      Helios
IMG_6901.CR2          Exif.Photo.LensModel                         LensModel                   Ascii      44-2 58mm f/2
IMG_6901.CR2          Exif.Photo.LensSerialNumber                  LensSerialNumber            Ascii      7765411
IMG_6901.CR2          Exif.Image.LensInfo                          LensInfo                    Rational   58/1 58/1 20/10 20/10
IMG_6901.CR2          Xmp.aux.Lens                                 Lens                        XmpText    Helios 44-2 58mm f/2

(https://i.ibb.co/hYkhDYB/IMG-1260.jpg) (https://ibb.co/hYkhDYB) (https://i.ibb.co/dBCKxcq/samyang85.jpg) (https://ibb.co/dBCKxcq) (https://i.ibb.co/VM2BCJS/helios44-2.jpg) (https://ibb.co/VM2BCJS)

If you'd want to test how the updated CR2 behaves in your favorite editor, you can grab the last one at https://www.zlej.net/IMG_6901.CR2
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on February 05, 2019, 06:02:46 PM
Excellent!

exiftool
Code: [Select]
...
Lens Info                       : 58mm f/2
Lens Make                       : Helios
Lens Model                      : 44-2 58mm f/2
Lens Serial Number              : 7765411
...

Adobe Camera Raw shows that it is a 58mm f/2 lens but there is no lens profile for that Helios lens.

Could you upload one with the Samyang 85mm? The xml is working with ACR on that lens so it would be great to see if it works the same as your embedded EXIF.

Are you pushing your code to a repository? It would be great to be able to try it out.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 05, 2019, 07:55:21 PM
Excellent!

exiftool
Code: [Select]
...
Lens Info                       : 58mm f/2
Lens Make                       : Helios
Lens Model                      : 44-2 58mm f/2
Lens Serial Number              : 7765411
...

Adobe Camera Raw shows that it is a 58mm f/2 lens but there is no lens profile for that Helios lens.

Could you upload one with the Samyang 85mm? The xml is working with ACR on that lens so it would be great to see if it works the same as your embedded EXIF.

Are you pushing your code to a repository? It would be great to be able to try it out.

I haven't pushed anything yet as there are still few features i'd like to implement and test ;)

CR2 with samyang data is at https://www.zlej.net/IMG_6900.CR2

exiv2 seems to have some windows builds, so the batch script i'm creating might be used on Windows as well, but if you'd know of any other way to batch-update a set of images with defined metadata, let me know and i can add another template..
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on February 05, 2019, 08:33:21 PM
Can you try with lensdata that arc has a profile for?

for example:

this should be autodetected with a profile

name = "Zeiss Distagon T* 2.8/25 ZF.2",        focal_length =  25, manual_aperture = 2.8 },
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 05, 2019, 11:31:57 PM
Can you try with lensdata that arc has a profile for?

for example:

this should be autodetected with a profile

name = "Zeiss Distagon T* 2.8/25 ZF.2",        focal_length =  25, manual_aperture = 2.8 },

The raw with fixed exif is at https://www.zlej.net/IMG_6910.CR2
But the problem is, that the profile for Distagon does not exactly follow the schema used by normal "chipped" lenses, where stCamera:Lens is exactly what's being returned  by the lens:
Code: [Select]
<rdf:Description
       stCamera:Author="Adobe (www.adobe.com)"
       stCamera:Make="Canon"
       stCamera:CameraRawProfile="True"
       stCamera:LensID="174"
       stCamera:Lens="150-500mm"
       stCamera:LensInfo="150/1 500/1 0/0 0/0"
       stCamera:CameraPrettyName="Canon"
       stCamera:LensPrettyName="SIGMA APO 150-500mm F5-6.3 DG OS HSM"
       stCamera:ProfileName="Adobe (SIGMA APO 150-500mm F5-6.3 DG OS HSM, Canon)"
       stCamera:SensorFormatFactor="1"
       stCamera:FocalLength="150"
       stCamera:FocusDistance="999999995904"
       stCamera:ApertureValue="6.643856">
Code: [Select]
    <rdf:li rdf:parseType="Resource">
      <stCamera:Author>Adobe Systems, Inc.</stCamera:Author>
      <stCamera:Make>Canon</stCamera:Make>
      <stCamera:Model>Canon EOS-1Ds Mark III</stCamera:Model>
      <stCamera:UniqueCameraModel>Canon EOS-1Ds Mark III</stCamera:UniqueCameraModel>
      <stCamera:CameraRawProfile>True</stCamera:CameraRawProfile>
      <stCamera:LensID>237</stCamera:LensID>
      <stCamera:Lens>EF24-105mm f/4L IS USM</stCamera:Lens>
      <stCamera:LensInfo>24/1 105/1 0/0 0/0</stCamera:LensInfo>
      <stCamera:CameraPrettyName>Canon EOS-1Ds Mark III</stCamera:CameraPrettyName>
      <stCamera:LensPrettyName>Canon EF 24-105mm f/4 L IS USM</stCamera:LensPrettyName>
      <stCamera:ProfileName>Adobe (Canon EF 24-105mm f/4 L IS USM)</stCamera:ProfileName>

While the profile for Distagon has also the brand included in stCamera:Lens:
Code: [Select]
     <rdf:Description
       stCamera:Author="Adobe (www.adobe.com)"
       stCamera:Make="Canon"
       stCamera:CameraRawProfile="True"
       stCamera:LensID="33"
       stCamera:Lens="Zeiss Distagon T* 2/25 ZE"
       stCamera:LensInfo="25/1 25/1 0/0 0/0"
       stCamera:CameraPrettyName="Canon"
       stCamera:LensPrettyName="Zeiss Distagon T* 2/25 ZE"
       stCamera:ProfileName="Adobe (Zeiss Distagon T* 2/25 ZE)"

Let's hope that it will match with the Xmp.aux.Lens field:
Code: [Select]
IMG_6910.CR2          Exif.CanonCs.DisplayAperture                 DisplayAperture             Short      0
IMG_6910.CR2          Exif.CanonCs.Lens                            Lens                        Short      25.0 mm
IMG_6910.CR2          Exif.CanonCs.LensType                        LensType                    Short      (65535)
IMG_6910.CR2          Exif.CanonCs.MaxAperture                     MaxAperture                 Short      F2.8
IMG_6910.CR2          Exif.CanonCs.MinAperture                     MinAperture                 Short      F23
IMG_6910.CR2          Exif.Canon.LensModel                         LensModel                   Ascii      Distagon T* 2.8/25 ZF.2
IMG_6910.CR2          Exif.CanonSi.ApertureValue                   ApertureValue               Short      F2.8
IMG_6910.CR2          Exif.CanonSi.TargetAperture                  TargetAperture              Short      F2.8
IMG_6910.CR2          Exif.Image.LensInfo                          LensInfo                    Rational   25/1 25/1 28/10 28/10
IMG_6910.CR2          Exif.Photo.ApertureValue                     ApertureValue               Rational   F2.8
IMG_6910.CR2          Exif.Photo.LensMake                          LensMake                    Ascii      Zeiss
IMG_6910.CR2          Exif.Photo.LensModel                         LensModel                   Ascii      Distagon T* 2.8/25 ZF.2
IMG_6910.CR2          Exif.Photo.LensSerialNumber                  LensSerialNumber            Ascii     
IMG_6910.CR2          Exif.Photo.LensSpecification                 LensSpecification           Rational   25/1 25/1 28/10 28/10
IMG_6910.CR2          Exif.Photo.MaxApertureValue                  MaxApertureValue            Rational   F2.8
IMG_6910.CR2          Xmp.aux.Lens                                 Lens                        XmpText    Zeiss Distagon T* 2.8/25 ZF.2
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on February 06, 2019, 12:58:29 AM
Good news, with the IMG_6910.CR2 it loads the profile automatically !

It found the right lens and made the corrections for it. screenshots for proof :)

(http://www.larssteenhoff.com/forum_pics/acr1.gif)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on February 06, 2019, 05:50:35 AM
The Samyang lens didn't do as well in ACR.

(https://farm8.staticflickr.com/7864/46949954592_95010226b9_n.jpg) (https://flic.kr/p/2ewNMMJ)

The profile should be included.

(https://farm5.staticflickr.com/4860/47001814321_5d5e207f5a.jpg) (https://flic.kr/p/2eBozSZ)

I can pick it manually but that defeats what we're trying to do here:

(https://farm8.staticflickr.com/7851/47001861201_6cffcc39f5.jpg) (https://flic.kr/p/2eBoPPg)

Downloaded from the experiments page, activated the Samyang 85mm and did a test shot to see if the xml works:

Code: [Select]
ML ASSERT:
0
at ../../src/mem.c:799 (__mem_malloc), task PropMgr
lv:0 mode:3

PropMgr stack: 14cdc0 [14cf68-14bf68]
0xUNKNOWN  @ 9c188:14ce70
0x00B25EAC @ b215ec:14ce50
0x00B256E8 @ b25ee4:14ce40
0x00081F80 @ b25708:14ce30
0x0007F688 @ 81fbc:14cdf0
0x0007EF78 @ 7f6e4:14cdc0

Magic Lantern version : Nightly.2019Feb05.700D115
Mercurial changeset   : 0de7b671c52c (manual_lens_info)
Built on 2019-02-06 04:44:22 UTC by rosiefort@Rosie-Forts-Computer.local.
Free Memory  : 148K + 2732K

Not my day--and I'm leaving tomorrow for a week so no time to play around with this right now.

[EDIT] ok--tried one more thing. Used aprofiti's crop_rec_4k_mlv_snd_elns branch and got what looks like a good xml but it still doesn't work automatically in ACR.

Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FocalLength="85/1"
    exif:FNumber="14/10"
    aux:SerialNumber="0"
    aux:Lens="Samyang 85mm f/1.4 AS IF UMC">
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 06, 2019, 09:58:37 AM
The crashes of experimental manual_lens branch were caused by one of recent commits, so for my testing i've used build from Alessandro - https://bitbucket.org/aprofiti/magic-lantern/downloads/

My versions of lens.lua (+ xmp.lua, logger.lua and ui.lua) are not available anywhere except my notebook and camera.

Apparently the Sam 85/1.4 should be supported by ACR, but the question is, what they expect to be in exif data. Would it be possible to get some archive with lens profiles from ACR? I got one from Lightroom, but Samyang is missing there :(

My version of lens.lua works in a way, that you:

The point is to find the right lens parameters for the editor to correctly detect the lens and match it with lens profile it supports..

So far i have these in my list (and with this make+name the distagon worked correctly as you've seen:
Code: [Select]
lenses = -- {{{
{
--  The following is for testing purposes. Comment out the following lines then either uncomment only the lenses
--  that you want to use from the list or add your own lenses. Tip: Put your most used lenses at the top of the list.
--  Attributes to use:
--  name                -> Name to be showed in Lens Info and in metadata
--  focal_length        -> Current focal length to be saved in metadata (Zoom Lenses: Default value to be used after selecting lens)
--  manual_aperture     -> Selected Aperture of the Lens
--  aperture_min        -> Min available aperture for a lens (limits the available FNumbers to the min-max range)
--  aperture_max        -> Max available aperture for a lens
--  focal_min           -> min Focal Length (optional for prime lenses)
--  focal_max           -> max Focal Length (optional for prime lenses)
--  serial              -> Lens Serial Number (optional)
--  f_values            -> Available f-stop for selected lens (optional) f_values = {"2.8","4","5.6","8"}
--  FminL               -> Max aperture on short end
--  FmaxL               -> Max aperture on long end
        { make = "Carl Zeiss Jena DDR", name = "Tessar 50mm f/2.8",             focal_length = 50,      manual_aperture = 2.8,  serialN = "8333263",    aperture_max = 2.8, aperture_min = 22 },
        { make = "E. Ludwig",           name = "Meritar 50mm f/2.9",            focal_length = 50,      manual_aperture = 2.9,  serialN = "1610554",    aperture_max = 2.9, aperture_min = 16 },
        { make = "Helios",              name = "44-2 58mm f/2",                 focal_length = 58,      manual_aperture = 2,    serialN = "7765411",    aperture_max = 2, aperture_min = 16, f_values = {"2","2.8","4","5.6","8","11","16"} },
        { make = "Industar",            name = "50-2 50mm f/3.5",               focal_length = 50,      manual_aperture = 3.5,  serialN = "7149595",    aperture_max = 3.5, aperture_min = 16 },
        { make = "Industar",            name = "61 L/Z (MC) 50mm f/2.8",        focal_length = 50,      manual_aperture = 2.8,  serialN = "8707286",    aperture_max = 2.8, aperture_min = 16 },
        { make = "Meyer-Optik Gorlitz", name = "Telemegor 180mm f/5.5",         focal_length = 180,     manual_aperture = 5.5,  serialN = "1728275",    aperture_max = 5.5, aperture_min = 22 },
        { make = "Meyer-Optik Gorlitz", name = "Trioplan 50mm f/2.9",           focal_length = 50,      manual_aperture = 2.9,  serialN = "2224344",    aperture_max = 2.9, aperture_min = 22, f_values = {"2.9","4","5.6","8","11","16","22"} },
        { make = "Pentacon",            name = "auto 29mm f/2.8",               focal_length = 29,      manual_aperture = 2.8,  serialN = "6308110",    aperture_max = 2.8, aperture_min = 22 },
        { make = "Pentacon",            name = "Prakticar 50mm f/1.8 MC",       focal_length = 50,      manual_aperture = 1.8,  serialN = "7710425",    aperture_max = 1.8, aperture_min = 16 },
        { make = "Samyang",             name = "85mm f/1.4 AS IF UMC",          focal_length = 85,      manual_aperture = 1.4,  serialN = "A217D0264",  aperture_max = 1.4, aperture_min = 22 },
        { make = "Zeiss",               name = "Distagon T* 2.8/25 ZF.2",       focal_length = 25,      manual_aperture = 2.8, aperture_max = 2.8, aperture_min = 22 },
--    { make = "", name = "", focal_length = , manual_aperture = , serial = "" },
} -- }}}

There is still a problem with chipped adapters, which (from camera's point of view) behave as regular lens - with focal_min = 1, focal_max = 65536, F_numbers = { "1.4" }. Which means, that our attempts to set the focal length and aperture are stubbornly ignored by the camera when the metadata are written. With dumb non-chipped adapters it's all fine and the camera accepts values we fake with lua script.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: Lars Steenhoff on February 06, 2019, 11:45:25 AM
Here is a list of lenses with built in profile support:

https://helpx.adobe.com/camera-raw/kb/supported-lenses.html (https://helpx.adobe.com/camera-raw/kb/supported-lenses.html)

And I found this name on the list

Samyang 85mm f/1.4 AS IF UMC
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 06, 2019, 12:13:55 PM
Here is a list of lenses with built in profile support:

https://helpx.adobe.com/camera-raw/kb/supported-lenses.html (https://helpx.adobe.com/camera-raw/kb/supported-lenses.html)

And I found this name on the list

Samyang 85mm f/1.4 AS IF UMC

That's what i've found as well, but apparently the exif values we have in the CR2 don't fully match. What would help are lens profile definition files directly from ACR, so i could rip them off a bit ;)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: dfort on February 06, 2019, 06:23:19 PM
Hey Walter, your posts were flagged as "Hall of Shame" but you do have a good point:

"Samyang 14 mm f/2.8 ED AS IF UMC?

I'm a little late for the party and haven't browsed through all replies.
Maybe I missed the point:
Why messing up with lens info for a lens with electronic coupling? AFAIK this particular lens is reporting its ID.

Turns out these lenses are available with or without a chip.

(https://farm8.staticflickr.com/7853/47007506121_c9a77c2cb8.jpg) (https://flic.kr/p/2eBTKRt)

We were discussing the 85mm but the same applies to the 14mm and other Samyang/Rokinon manual lenses. I've got the cine version of these lenses but the photo version lens profiles should work fine on them.

You highlighted ED but that means ED: contains extra-low dispersion element(s) (https://photo.stackexchange.com/questions/496/what-do-all-those-cryptic-number-and-letter-codes-in-a-lens-name-mean/76191#76191) and has nothing to do with whether it is chipped. The chipped and unchipped lenses use the same name and thus the same lens profile as far as I can tell.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 06, 2019, 10:33:55 PM
@dfort - technically you can turn any lens/adapter into chipped one by sticking an EMF chip onto it (see https://www.ebay.com/itm/EMF-AF-Confirm-Chip-for-M42-PK-Nikon-CY-LR-OM-Canon-EOS-EF-Adapter-programmable/132534235628)

Problem is, that when there is a chip, which tells the camera anything about focal length and aperture, the camera will use these values instead of what we'll specify with the script. Or at least it seems to work like this from what i've seen so far.
What we'd need is to override certain parameters (camera.aperture.max, camera.aperture.min and something related to min/max focal length). Having customized camera.aperture.max/min would also give us a possibility to set camera.aperture.value and i assume, that the firmware would then use these values when writing the exif data to jpeg/cr2..

@walter - the point of all this is to add a certain metadata to images taken with old/non-chipped lenses, so the image editors can apply some lens-correction filters. and some people (me included) may find it useful for catalogizing pictures by used lenses.

Code: [Select]
find ./ -type f -name '*.CR2' -exec exiv2 -g Exif.Photo.LensModel -Pt {} 2>/dev/null \; |sort|uniq -c |sed -e "s/^ *//" -e "s/ /x /"
101x
5x 10-20mm
92x 150-500mm
37x 1-65535mm
758x EF17-40mm f/4L USM
293x EF24-105mm f/4L IS USM
18x EF50mm f/1.8 STM
61x EF50mm f/2.5 Compact Macro
12x EF70-300mm f/4-5.6 IS USM
496x EF-S10-18mm f/4.5-5.6 IS STM
102x Meyer-Optik Görlitz Trioplan 50mm f/2.9
238x MP-E65mm f/2.8 1-5x Macro Photo
26x Samyang 85mm f/1.4 Aspherical IF
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 08, 2019, 01:21:41 PM
Actually, having the correct exif data embedded in the image makes generating of lens-correction profiles a lot easier ;)
https://wilson.bronger.org/lens_calibration_tutorial/

So far i've only played with vignetting data, but preparing pics for chromatic aberration and distortion is a piece of cake..

Code: [Select]
<lensdatabase>

    <lens>
        <maker>Carl Zeiss Jena</maker>
        <model>Tessar 50mm f/2.8</model>
        <mount>M42</mount>
        <cropfactor>1</cropfactor>
        <calibration>
            <vignetting model="pa" focal="50" aperture="2.8" distance="10" k1="-1.1957" k2="0.8042" k3="-0.2577"/>
            <vignetting model="pa" focal="50" aperture="2.8" distance="1000" k1="-1.1957" k2="0.8042" k3="-0.2577"/>
        </calibration>
    </lens>

    <lens>
        <maker>E. Ludwig</maker>
        <model>Meritar 50mm f/2.9</model>
        <mount>EXA</mount>
        <cropfactor>1</cropfactor>
    </lens>

    <lens>
        <maker>Helios</maker>
        <model>44-2 58mm f/2</model>
        <mount>M42</mount>
        <cropfactor>1</cropfactor>
        <calibration>
            <vignetting model="pa" focal="58" aperture="1.4" distance="10" k1="-0.2865" k2="-0.3428" k3="0.1544"/>
            <vignetting model="pa" focal="58" aperture="1.4" distance="1000" k1="-0.2865" k2="-0.3428" k3="0.1544"/>
        </calibration>
    </lens>

Small issue is, that hellios had a chipped adapter, so the camera used the provided aperture of F1.4..
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 08, 2019, 03:18:16 PM
Small issue is, that hellios had a chipped adapter, so the camera used the provided aperture of F1.4..
Are you referring to the embedded exif data of the image or the generated .xmp sidecar?

I'm sorry, regarding embedded exif we still need to figure out how to inject in the image processing pipeline as a1ex said..

If you are referring to the .xmp didn't noticed this issue with my local build, it should take the data in the ML lens_info struct

The crashes of experimental manual_lens branch were caused by one of recent commits, so for my testing i've used build from Alessandro - https://bitbucket.org/aprofiti/magic-lantern/downloads/
Currently I'm working on:
- Refactor config lib to make code more readable (recursiveCopy is a completely mess... not proud of It...)
- add more attribute to lenses looking from zLOST ..xmp
- add support for multiple name for the same attribute to .xmp lib (ex. Lens name using aux: exif: and exifEX:)

Still thinking about the possibility to have user's lenses in a separate config than embedded in the script.
This will be useful in case of an update, as user doesn't need to insert lenses each time

Attribute added locally:
-Fmin,Fmax (from zLOST)
-lens specification (from zLOST)

Would like to calculate aperture min and max from FNumber automatically instead of adding 2 more attribute to lenses.
Will look into it.

@zLOST can you show your .xmp.lua?
I'm having problems in adding support for multiple name for the same attribute. It doesn't print some of the other ones

Also can't figure out why lua isn't saving my data with the modified recusiveCopy I'm working on.. they are table, so passing is by reference...

Also noticed a strange behaviour with Lightroom, it looks like doesn't read the .xmp when opening the image... so no correction applied.

Does this also appen to you guys?

I will update my remote repo and the PR after solving those problems
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 08, 2019, 04:52:15 PM
Hello Alessandro,

my version of {lens|xmp|ui|logger}.lua is at https://www.zlej.net/manual_lens.tar.gz (you'll just need to delete settings/LENS.LCF, otherwise it will keep on crashing when loading/saving the configuration due to added "Format" select to the lens configuration menu),
There are changes all over the place, so it may be better to discuss this by email, skype, icq or irc.
I may have screwed something up, but dumb adapters/lenses work fine - the focal length+aperture values are saved correctly, but with chipped adapter (one, which tells the camera some focal length/aperture values, these are used instead of the ones chosen by me.

With my "exiv2" template the output is not a XMP sidecarfile for each photo, but one bash script (with one line being added for each pic), which i'll run once i'm done with shooting and it updates the exif data of the existing CR2/JPG files, so no messing with metadata during saving the picture. So this should be safe way to get the data there ;)

An example of such line is:
Code: [Select]
exiv2 -k -M"set Exif.CanonCs.LensType 65535" -M "set Exif.CanonCs.MinAperture 288" -M "set Xmp.aux.Lens Meyer-Optik Gorl
itz Telemegor 180mm f/5.5" -M "set Exif.CanonCs.Lens 180 180 1" -M "set Exif.Photo.LensSpecification 180/1 180/1 56/10 5
6/10" -M "set Exif.Photo.MaxApertureValue 325770/65536" -M "set Exif.Photo.FNumber 55/10" -M "set Exif.Photo.FocalLength
 180/1" -M "set Exif.Photo.LensMake Meyer-Optik Gorlitz" -M "set Exif.CanonSi.TargetAperture 157" -M "set Exif.CanonSi.A
pertureValue 157" -M "set Exif.CanonCs.MaxAperture 159" -M "set Exif.Photo.ApertureValue 322363/65536" -M "set Exif.Cano
n.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensSerialNumb
er 1728275" -M "set Exif.Image.LensInfo 180/1 180/1 56/10 56/10" IMG_6929.CR2
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: zLOST on February 08, 2019, 05:07:25 PM
- add more attribute to lenses looking from zLOST ..xmp
- add support for multiple name for the same attribute to .xmp lib (ex. Lens name using aux: exif: and exifEX:)
Technically if we could insert the correct exif data with exiv2, we won't have to rely on editors to accept&parse the xmp sidecar at all as the picture would look like shot with modern lens ;)

Still thinking about the possibility to have user's lenses in a separate config than embedded in the script.
This will be useful in case of an update, as user doesn't need to insert lenses each time
What about an external DB of lenses and user will only say, which are of some interest for him/her? ;)

Attribute added locally:
-Fmin,Fmax (from zLOST)
-lens specification (from zLOST)

Would like to calculate aperture min and max from FNumber automatically instead of adding 2 more attribute to lenses.
Will look into it.
I've added a function set_lens_defaults(), which generates certain attributes. Including the Fmax Fmin from f_values (if available)...

@zLOST can you show your .xmp.lua?
I'm having problems in adding support for multiple name for the same attribute. It doesn't print some of the other ones
Link to archive with everything i've mutilated is in the post above...

Also can't figure out why lua isn't saving my data with the modified recusiveCopy I'm working on.. they are table, so passing is by reference...

Also noticed a strange behaviour with Lightroom, it looks like doesn't read the .xmp when opening the image... so no correction applied.

Does this also appen to you guys?

I will update my remote repo and the PR after solving those problems

Sorry, i'm darktable+gimp@debian only ;)
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 23, 2019, 07:23:49 PM
my version of {lens|xmp|ui|logger}.lua is at https://www.zlej.net/manual_lens.tar.gz (you'll just need to delete settings/LENS.LCF, otherwise it will keep on crashing when loading/saving the configuration due to added "Format" select to the lens configuration menu),

Thank you zLost! Had a look to understand why my version of xmp.lua with added support for array of property wasn't working and found i wasn't converting to table before adding them.

It's not currently present in the updated PR at the moment to keep it simple, but I was thinking that is good to add and leave the possibility to use this features in the future.

With my "exiv2" template the output is not a XMP sidecarfile for each photo, but one bash script (with one line being added for each pic), which i'll run once i'm done with shooting and it updates the exif data of the existing CR2/JPG files, so no messing with metadata during saving the picture. So this should be safe way to get the data there ;)

An example of such line is:
Code: [Select]
exiv2 -k -M"set Exif.CanonCs.LensType 65535" -M "set Exif.CanonCs.MinAperture 288" -M "set Xmp.aux.Lens Meyer-Optik Gorl
itz Telemegor 180mm f/5.5" -M "set Exif.CanonCs.Lens 180 180 1" -M "set Exif.Photo.LensSpecification 180/1 180/1 56/10 5
6/10" -M "set Exif.Photo.MaxApertureValue 325770/65536" -M "set Exif.Photo.FNumber 55/10" -M "set Exif.Photo.FocalLength
 180/1" -M "set Exif.Photo.LensMake Meyer-Optik Gorlitz" -M "set Exif.CanonSi.TargetAperture 157" -M "set Exif.CanonSi.A
pertureValue 157" -M "set Exif.CanonCs.MaxAperture 159" -M "set Exif.Photo.ApertureValue 322363/65536" -M "set Exif.Cano
n.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensModel Telemegor 180mm f/5.5" -M "set Exif.Photo.LensSerialNumb
er 1728275" -M "set Exif.Image.LensInfo 180/1 180/1 56/10 56/10" IMG_6929.CR2
This looks interesting :)

I would add a new menu (like for Autoload option) to "Lens Pref" instead of inside "Lens" submenu, just to have all the options organised and easy accessible.

"Config.lua" library already support multiple menus and config tables in the same script in this branch

There are changes all over the place, so it may be better to discuss this by email, skype, icq or irc.
We could add a couple of commits with your features in the PR if we can manage to keep script simple (both usage and code complexity).

Take a look on how mercurial (Sourcetree or equivalent gui) works and how to contribute :)

What about an external DB of lenses and user will only say, which are of some interest for him/her? ;)
Currently the script already have it in a certain way: It's the table "Lenses" with the commented entry (Carl Zeiss, Nikon, Samyang...).

At the moment on each script update, the user need to insert again his lenses to the table...

It's possible to create a config from that table and add it automatically to the generated configuration file (lens.lcf), but it's a bit ugly:
Code: [Select]
return {
       [1] = {
        ["name"] = "My Lens",
        ["focal_length"] = 50,
        ["manual_aperture"] = "1.8"
       },
       [2] = {
        ["name"] = "My Other Lens",
        ["focal_length"] = 25,
        ["manual_aperture"] = "2.8",
        ["f_values"] = {
          [1] = "2.8",
          [2] = "4",
          [3] = "5.6",
          [4] = "8"}
        },
["Autoload Lens"] = "ON",
["Manual Lens"] = {
["Lens"] = 1,
["Aperture"] = "1.8",
["Focal Length"] = 50,
},
}
Maybe it's better to put it inside a "Lenses" block so it's a bit easier to understand what is it by looking at the config.

I was trying to add this directly to the library by adding and additional parameter to the function responsible for config creation, but got some problems and was taking too much time.

It can also be done in the script by doing something like this:
Code: [Select]
ExampleTable = {Lenses = {...}}
lenses_cfg = config.create(ExampleTable)

Attribute added locally:
-Fmin,Fmax (from zLOST)
-lens specification (from zLOST)
Would like to calculate aperture min and max from FNumber automatically instead of adding 2 more attribute to lenses.
I've added a function set_lens_defaults(), which generates certain attributes. Including the Fmax Fmin from f_values (if available)...
Made a in a similiar way. First was added to restore_lens_values, then refactored a bit and added update_aperture_focal_range(), because it was using common code needed when switching lens and autoloading the script.

The Samyang lens didn't do as well in ACR.
The profile should be included.
I'm still a little bit temped to not separate Lens name in Make + Model if possible... but these cases is what is making things hard regarding automatic profile selection...

From what i have understood by reading your posts, is that each editor may look for different attribute when selecting profile (Lens, Lens Make, Lens Model....)
Update the image's attributes by modifying exif like zLost did with exiv2 is something to keep in mind.

Updated the PR with latest tweaks (https://bitbucket.org/hudson/magic-lantern/pull-requests/939/dynamic-length-elns-block/commits) (except for lensSpecification and support for multiple name for the same attribute in xmp.lua), so now it's possible to compile a non crashing build for all cameras to experiment with it.
Title: Re: Assign lens focal length and name for non cpu lenses
Post by: aprofiti on February 27, 2019, 10:46:33 PM
@zLost I'm trying to understand what attribute is used on each editing software and what metadata need to be rewritten to have correct lens name embedded in the picture.

I selected "Samyang 8mm f/3.5 UMC Fish-Eye CS II" from the script and taken a CR2 picture without a lens installed, so no lens-related metadata were included.

XMP sidecar looks like this: (let's keep it simple for now, no exif:lensModel or others... is exif:lensModel to be preferred to aux:lens instead?)
Code: [Select]
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37        ">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
    exif:ExifVersion="0230"
    exif:FocalLength="8/1"
    exif:FNumber="35/10"
    aux:SerialNumber="0"
    aux:Lens="Samyang 8mm f/3.5 UMC Fish-Eye CS II"
    exif:MaxApertureValue="35/10"
    exi