Assign lens focal length and name for non cpu lenses

Started by Lars Steenhoff, October 29, 2016, 12:04:45 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

dmilligan

Quote from: aprofiti on December 23, 2017, 02:19:09 AM
Someone can explain to me why can't get it working? I have also tried to merge into lua_fix to get latest fixes, but nothing usefull happened.

Question: Should I include also 1/3 stop values to Fnumbers? Currently if an aperture value of the lens isn't found in Fnumbers when opening Manual Lens submenu, it get set to first index (ex. aperture 4.5 not found -> set by script to 1.0)

[EDIT]
It's a problem of floating points rapresentation in lua:

1.4 -> 1.399999
2.8 -> 2.799999
4.5 -> 4.5 (ok)
5.6 -> 5.59999


Check the ftoa function in ml_lua_shim.c, it could probably be improved (maybe there's a usable implementation in dietlibc that's better). The sprintf in Canon firmware doesn't support floats at all, and we don't have a complete, standards compliant libc implementation available in ML. So for now there are hacky implementations here and there of stuff we need (space in the compiled binary is at a premium).

As a workaround you could store lens.manual_aperture as string and then when you need to use it as a number, convert it to number with tonumber().

a1ex

Knowing where to look makes things really easy - fix committed.

Sorry, didn't test further yet.

dfort

Lots of nice additions here but we still have a rather basic issue with the 32 character limit for the lens name. It would be great if this could be addressed.

Using @aprofiti pull request version and the latest mlv_dump from the crop_rec_4k branch.

{ name = "Zeiss Makro-Planar T* 2/100 ZF.2",     focal_length = 100, manual_aperture = 2   },

Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.956000 ms
     Name:        'Zeiss Makro-Planar T* 2/100 ZF.'
     Serial:      '' (no valid S/N)
     Focal Len:   100 mm
     Focus Dist:  0 mm
     Aperture:    f/2.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000


I also tried another lens with a long name and this time the Aperture isn't listed properly (float issue?): [EDIT] Merged in lua_fix with the "round to our precision" fix and the Aperture is now being displayed properly. Thanks a1ex!

{ name = "Samyang 12mm f/2.8 ED AS NCS Fisheye", focal_length =  12, manual_aperture = 2.8 }, --  12mm T3.1 Cine

Block: LENS
  Offset: 0x0000016c
  Number: 6
    Size: 96
    Time: 0.003000 ms
     Name:        'Samyang 12mm f/2.8 ED AS NCS Fi'
     Serial:      '' (no valid S/N)
     Focal Len:   12 mm
     Focus Dist:  0 mm
     Aperture:    f/2.80
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

aprofiti

Quote from: a1ex on December 30, 2017, 11:13:36 PM
Knowing where to look makes things really easy - fix committed.

Sorry, didn't test further yet.
Really thank you a1ex! At the moment I am unable to test it, I'll try in next days.

Quote from: dfort on December 31, 2017, 06:01:34 AM
I also tried another lens with a long name and this time the Aperture isn't listed properly (float issue?):

{ name = "Samyang 12mm f/2.8 ED AS NCS Fisheye", focal_length =  12, manual_aperture = 2.8 }, --  12mm T3.1 Cine

Block: LENS
  Offset: 0x00000184
  Number: 5
    Size: 96
    Time: 0.881000 ms
     Name:        'Samyang 12mm f/2.8 ED AS NCS Fi'
     Serial:      '' (no valid S/N)
     Focal Len:   12 mm
     Focus Dist:  0 mm
     Aperture:    f/1.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x00000000
     Flags:       0x00000000

Yes, It's from floating point issue. Value 1.0 is selected by Ml (first index in Fnumbers) because Aperture wasn't found in submenu choices.

Can you try with a1ex commit or by merging PR into lua_fix (use lastest lual_lens.c changes to resolve conflict)?

dfort

Quote from: aprofiti on December 31, 2017, 05:15:39 PM
Can you try with a1ex commit or by merging PR into lua_fix (use lastest lual_lens.c changes to resolve conflict)?

Just did and it is working fine now. Edited Reply #277.

I'm sounding like a broken record asking for more characters in the lens name but the main developers agreed that it should be fixed. Not that I'm complaining--just bringing @aprofiti up to speed on this. Maybe this could be incorporated into all the great changes he's making to this branch. It is after all an "Experimental" branch.  :D

aprofiti

Updated PR with autoload functionality and made some tests to solve aperture with AF Chip.

Here is some ideas to solve issue no.2:
1. Make HALF_SHUTTER handler to works outside live view and update aperture from here (appear to be the easiest, any Hints?)
2. Change camera aperture by make it works from camera.aperture API (At the moment camera will freeze)
3. "Emulating" MPU or EF protocol messages

Quote from: dfort on December 31, 2017, 05:45:42 PM
I'm sounding like a broken record asking for more characters in the lens name but the main developers agreed that it should be fixed. Not that I'm complaining--just bringing @aprofiti up to speed on this. Maybe this could be incorporated into all the great changes he's making to this branch. It is after all an "Experimental" branch.  :D
How many characters in lens_info.name? I think a minimum of 48-64 should be addressed to cover most lenses.
Some lenses use a very long name, especially Carl Zeiss (ex. Carl Zeiss Vario-Sonnar T* 100-300 mm f/ 4.5-5.6 C/Y -> 52 characters).

I tried with 64 byte in lens.h and after updating lua_lens.c wrapper to avoid weird aperture values, I have got full name in .xmp file but Silent Pictures wasn't working anymore (error: "Manual exposure is required."). I wasn't able to test .mlv

dfort

Quote from: aprofiti on January 03, 2018, 01:56:24 AM
How many characters in lens_info.name? I think a minimum of 48-64 should be addressed to cover most lenses.

I ran some tests a while back to see how characters are allowed in Adobe Camera Raw and I never reached a limit. The 32 character limit in MLV is too short for a lens that should work fine in ACR: "Zeiss Makro-Planar T* 2/100 ZF.2" If it could be doubled to 64 characters that should cover 99.99% of the lens names.

Quote from: aprofiti on January 03, 2018, 01:56:24 AM
I tried with 64 byte in lens.h and after updating lua_lens.c wrapper to avoid weird aperture values, I have got full name in .xmp file but Silent Pictures wasn't working anymore (error: "Manual exposure is required."). I wasn't able to test .mlv

That's a common user error. Silent Pictures only works in manual exposure mode and you need to setup back button focus.

The problem with extending the lens name in MLV is that it will break some (most?) of the MLV processing apps.

aprofiti

Quote from: dfort on January 03, 2018, 05:09:58 AM
That's a common user error. Silent Pictures only works in manual exposure mode and you need to setup back button focus.

The problem with extending the lens name in MLV is that it will break some (most?) of the MLV processing apps.
I was in M Mode and checked Iso and Shutter time, but wasn't able to take pictures. It's first time I see this error, maybe is related to lens_info.name length? What about back button focus?

As name is internal to ML my general idea was to change his length, then check MLV should return lens name as before in LENS Block (because it copies only the first 32 characters) and then experimenting by adding ELNS Block to get a working export of full name's length. Could it works?

I'll be a little busy in next days. I don't know if I can work on it  :-\

aprofiti

Got it!

Was a problem with missing '\0' in mlv.c when strncpy() of legacy lensName.
I made some tests with mlv_rec and mlv_dump and I can find "ELNS" Block and extended name using a Hex Editor, but i need to figure out why mlv_dump prints an empty name in ELNS Block.

Note: I get Black screen in LV when entering again after a raw recording. Need to test if it's a bug from those changes or not.

Also found what was the problem with Silent Pictures: I was using .mo compiled from lua_fix because in manual_lens_info branch it won't compile. I updated src and compiled it from scratch and now it's working with 64 byte lensName.

Should I commit to this branch or maybe it's better to merge PR first and then experiment with longer Lens Name?

Lars Steenhoff

experimenting sounds like a good way to go forward

aprofiti

Quote from: Lars Steenhoff on January 08, 2018, 11:55:53 AM
experimenting sounds like a good way to go forward

I still can't figure out why I get a black LV and no Ml menu when switching in and out from LV after a raw video...
LV works good when using Silent Pictures.

@a1ex or @g3gg0 Can you review this Patch to see if I missed something or there is something wrong with my camera? Problem will occur also when changing only the length of lens_info.name without implementing ELNS Block, this let me think there is something related to update.

Also I get wrong prefix for .xmp files like "NIL0185.xmp". I tried to reset Canon's config but It didn't solved the issue

a1ex

One caveat: after changing a structure in ML core, you have to recompile all modules; otherwise, they will no longer be compatible, binary-wise. I should include this change in lua_fix before merging into unified, as that branch has the module API version already bumped (a mechanism used to prevent undefined behavior from situations like this).

Sorry, unable to test more right now; maybe over the weekend. Overall I like where this is heading.

aprofiti

Quote from: a1ex on January 11, 2018, 12:11:35 AM
Sorry, unable to test more right now; maybe over the weekend. Overall I like where this is heading.

While testing again I noticed that when LV is black, pressing info button will let you switch to Canon overlay and pressing again will let you see ML overlay (it's frozen), but again on both screens no image from sensor... Like if LV wasn't correctly enabled again.

This happens also with mlv_lite which is still unmodified. Tracing lens_info.name usage in code will not help finding more functions to be updated to make they work with extended length.

Hope those infos are useful for debugging.. Let me know if you had time to check

a1ex

Figured out the "black screen" issue; building an experimental build with the latest changes from aprofiti.

edit: done, also included crop_rec_4k changes on a few models.

aprofiti

Quote from: a1ex on February 02, 2018, 03:43:21 PM
Figured out the "black screen" issue; building an experimental build with the latest changes from aprofiti.
Good! This one doesn't has the experimental patch as it's still in local because was expected to be committed after solving config's lib refactor.

I have a question to you @a1ex: Do you prefer a new PR containing only the changeset related to lens_info.name, ELNS block and mlv_dump so it can be merged relatively fast in crop-rec-4k (and can be tested by users) or in manual_lens_info (as now it should have an updated mlv_dump and mlv_rec), or a PR with the continuation of manual_lens_info?

Work in progress:
- refactor of config.lua (causing a large dalay because a recursive implementation to allow traversing submenu is causing thrubles regarding scoping an passing tables as argument to allow copy by reference...)
- add apertureMin/apertureMax to be saved from lua script (already implemented, just needs to be committed), so can be saved in "ELSN" block and maybe implement in lens.lua a full lower and upper bound for aperure when no custom f-values specified (currently only lower bound is computed automatically)
- maybe add lens name to Flex Info to camera doesn't already have (also add support for 50D, doesn't seems easy as expected) to allow a quick check on which lens is selected without opening menu

Desidered:
- find a way to show current aperture in Photo info display. Can we use PatchManager to modify canon's aperture value displayed? Or better... Use Qemu's MPU Emulation to send a message to cpu regarding aperture?

a1ex

Let's continue with manual_lens_info (i.e. stay on topic). I've already included the crop_rec_4k stuff on the models that can be built from that branch, and I can include it anywhere later if it works well. For now, I still don't quite trust the change I did to lua_property (gut feeling says it will fail as soon as loading some more scripts that will eat the entire UMM memory buffer), but I'm not sure what to do to fix it either (I'll probably have to rework to the main memory backend).

ToniX

I have installed the build "magiclantern-manual_lens_info.2018Feb02.600D102" on my 600D.

I've found there the crop_rec. mod, which  I was very curious to test but couldn't find anywhere else.

Unfortunatly, enabling the Crop-rec. module result in error and any other active module is being disabled as well.

( Instead, Lua script seem to work fine here, but I didn't tested extensively)

Is it this a known issue of the 600D or  is it there  some specific procedure to follow in order to make it work?



600D - EFs18-55 ISII

dfort

crop_rec and lossless compression aren't yet available on the 600D. It might be possible though there is very little memory on that camera to work with. Back on topic--how is the manual lens lua script working on the 600D?

JohanJ

@a1ex
Ran my first test with magiclantern-manual_lens_info.2018Feb02.60D111 in photomode Av using the included lens.lua script together with my Samyang 8mm Fishey. Since 60D is not on 4K I thought it should not be a problem but unfurtunately I got the following assert after taking a picture.


ML ASSERT:
0
at ../../src/mem.c:844 (__mem_free), task PropMgr
lv:0 mode:2

PropMgr stack: 110c68 [110f68-10ff68]
0x00BA1C9C @ bacfb8:110d00
0xUNKNOWN  @ ba1d80:110ce0
0xUNKNOWN  @ bb13d0:110cd0
0x00BC8754 @ bb14c4:110cc8
0x00B9D0FC @ bc87c0:110cb8
0x1FE00CB8 @ 1fe036ac:110c98
0x1FE00378 @ 1fe00d14:110c68

Magic Lantern version : manual_lens_info.2018Feb02.60D111
Mercurial changeset   : e7ca98e11cd4 (manual_lens_info)
Built on 2018-02-02 19:26:24 UTC by jenkins@nightly.
Free Memory  : 313K + 1320K


Still both .CR2 and .xmp files got written to the card. 
60D.111 / 100D.101 / M2.103

ToniX

@dfort   
Quote from: dfort on February 09, 2018, 02:35:56 AM
crop_rec and lossless compression aren't yet available on the 600D. It might be possible though there is very little memory on that camera to work with. Back on topic--how is the manual lens lua script working on the 600D?

Thank you for the confirmation. ::)

As for the script I can only tell you what happens in the camera:

1) After activating the module, go to the> Script screen.
2) select the Manual Lens Information item in a sub menu Run> Run
3) I follow the instructions and go to the menu> Prefs> Lens Info Prefs
4) In the subMenu -> Manual lens there are 4 items:

Lens: shows the type of lens (EF-S18-55 .... ISII)
Focal length: 0
Openings: 5.6
Autoload lens: on

At the bottom of the screen, an orange text says "Chipped Lens Detected" (in fact, this lens has AF that does not focus well).

The camera need 2 trials before the module is deactivated
The module does not automatically reload, it must be reactivated each time after turning off the camera.
I noted that enabling LUA it use an amount  of memory: from 254K+1001k to 242K +720K

I hope I was helpful. In case you need anything else, please ask.
600D - EFs18-55 ISII

a1ex

@JohanJ:

The error happens from a module (they are dynamically loaded, so can't really interpret the stack trace). Loading lua.mo and lens.lua, selecting some dummy manual lens, and taking a picture, doesn't trigger an assert.

However, XMPs are named nil0004.XMP, rather than IMG_0004.CR2. Does this happen for you?

Settings (ML/SETTINGS directory) and steps to reproduce your issue?

@ToniX: it's  a build system issue, the crop_rec module shouldn't have been included in the first place.

Cannot reproduce the "need 2 trials before the module is deactivated" part (not even sure what exactly you mean by that).

JohanJ

QuoteHowever, XMPs are named nil0004.XMP, rather than IMG_0004.CR2. Does this happen for you?
Yes. IMG_6764.CR2 got a sidecar file named nil6764.XMP

ML/SETTINGS can be found here https://drive.google.com/open?id=1uW_69RLXEnGiBBSmZIaSUbGOvb-Z5GDv:
I added an nil...xmp as well as the latet ASSERT.LOG to this folder.

Did the following steps (nothing fancy)
1. Activated lua.mo (no other module was active in parallel!)
2. restarted cam
3. Started lens.lua manually
4. Chose one of the 3 lenses suggested
5. Took a picture --> no assert
6. Changed script to autorun
7. restarted cam
8. took a picture --> ASSERT
9. Changed script setting from autrun back to manual
10. restarted cam
11. stared script manually
12. took a picture --> ASSERT

My first impression was that the assert was caused by a the autorun setting. But that was proven wrong as the assert also triggert when changing back to manual start. There was no logic explanation, I fear
60D.111 / 100D.101 / M2.103

aprofiti

Quote from: a1ex on February 09, 2018, 01:07:36 PM
@JohanJ:
However, XMPs are named nil0004.XMP, rather than IMG_0004.CR2. Does this happen for you?

It happens also to me, it's a problem I noticed before and asked for feedback some posts ago.
If I remember correcly, it showed up in the middle of developing and wasn't sure if was related to canon's settings.

I tried now with 04April build and .xmp file are named correctly.

EDIT: reproduced ASSERT. It showed up on first picture after enabling script. It's the first time I see it.
@a1ex maybe it is related to commit 396a8a9 or b0b332a

ToniX

@ a1ex
Quote from: a1ex on February 09, 2018, 01:07:36 PM

@ToniX: it's  a build system issue, the crop_rec module shouldn't have been included in the first place.

Cannot reproduce the "need 2 trials before the module is deactivated" part (not even sure what exactly you mean by that).

While testing a lot of different builds I have noted that in some of them (couldn't tell exactly which ones) it requires more than one camera on/off cycle for the modules to be activated/deactivated.

To be explicit: when enabling a module and restarting the camera, the module is not being loaded indeed, and the "ON-Will load" text is still there.

Two,  sometime three camera restarts, are needed to have it enbled. Same at reverse.

This is a confusing and time consuming issue, but I thougtht it was marginal afterall, given all the ML benefits.
600D - EFs18-55 ISII

Walter Schulz

If you don't give ML enough time to shutdown properly you will see this happen.
Treat it right (= Let some seconds pass *after* power down) and modules will load/unload as intended.

More serious issue: After opening card slot (after!) cam will access card and if you do not wait for write operations to end you endanger your filesystem integrity. Or short: You can kill your card if you're impatient.
This access will happen with power off, too!

I think it's time reading some lines documented in Top of page -> User Guide -> Install Guide