Lua Scripting (lua.mo)

Started by dmilligan, March 29, 2015, 04:44:07 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

garry23


a1ex

Can you try the Jan26 lua_focus_pos build again?

Hint: they are identical.

garry23

@a1ex

That build, ie this one, doesn't flash.




a1ex

Tested in the same conditions as the previous builds, or just swapped an older card, possibly with different settings?

Please check the Help screen of the lua_fix_dbg #10 and compare the changeset IDs.

garry23

@a1ex

All I did was replace the .fir, .bin and the ML folder.

All camera settings unchanged.


garry23


JohanJ

Quote from: a1ex on June 24, 2017, 11:07:09 PM
That's yet another null pointer issue (likely in one of these modules, during initialization).

I tested all modules booting them separately or in a combination they are required to function (like mlv_snd needs mlv_rec to be activated too) and found the following candidates creating asserts:

dual_iso.mo
ML ASSERT:
a
at ../../src/stdio.c:44 (streq), task module_task
lv:0 mode:2


deflick.mo:

ML ASSERT:
a
at ../../src/stdio.c:44 (streq), task module_task
lv:0 mode:2


raw_rec.mo (2 different ones)

ML ASSERT:
new_entry->name
at ../../src/menu.c:1224 (menu_add), task module_task
lv:0 mode:2


ML ASSERT:
entry->name
at ../../src/menu.c:6430 (check_duplicate_entries), task menu_redraw_task
lv:0 mode:2
60D.111 / 100D.101 / M2.103

a1ex

Fixed the first two locally (the asserts caught yet another menu bug).

For the last two, there's no raw_rec module in this branch, and mlv_lite doesn't give any errors. The raw_rec I already had on the card did not trigger any error (it refused to load, with status OldAPI).

JohanJ

Quote from: a1ex on June 29, 2017, 12:23:12 PM
For the last two, there's no raw_rec module in this branch, and mlv_lite doesn't give any errors. The raw_rec I already had on the card did not trigger any error (it refused to load, with status OldAPI).

Strange! I will check the SD card when returning home tonight. Normally I am deleting ML folders, auto...bin and .fir before loading a new version. No idea how this module came in there otherwise.

[EDIT] @a1ex: downloaded lua_fix for 100D and 60D from experiments page again: raw_rec.mo is definitively included in the modules folder for these builds. Probably a left-ower in makefile? When starting the raw_rec module on 100D I get exactly the same error messages as reported here:  http://www.magiclantern.fm/forum/index.php?topic=16040.msg186543#msg186543
Might be obsolete though when emliminating raw_rec.mo from these builds.

60D.111 / 100D.101 / M2.103

a1ex

Good catch - the build system was not smart enough to remove it when running "make clean"...

Pushed the fix and rebuilding now.

In the mean time, here's what I'm working on (as the exact location of th issue is often not obvious from the crash logs).

garry23

@a1ex

Of no great importance to me, but that orange flash still occurs with the latest experimental nightly.

Cheers

Garry

a1ex

OK, I should probably create a build binary identical to Jan26 lua_focus_pos. Currently, the only difference between that (which doesn't flash) and #299 (which flashes) is the version number, build date and similar strings (but the strings have slightly different lengths, which resulted in different offsets in the binary). There should be no functional difference at all.

dmilligan

Quote from: a1ex on July 01, 2017, 04:12:48 PM
Good catch - the build system was not smart enough to remove it when running "make clean"...

make clean for modules pretty much never works because of this (in modules/Makefile):

MODULES_CLEAN_ALL_TARGETS = $(call targets_rules, $(shell ls -d */), clean)


The rule tries to run clean on every directory it finds in the modules folder, regardless of whether or not that folder actually has a makefile in it. As soon as you get to a folder without a makefile, the whole things errors out and nothing after that folder gets cleaned. This is a huge problem when switching branches with different modules in them, since the version control will delete the makefile when switching branches, but none of the build files or the directory.

A long time ago, I made a patch that prevented this rule from failing, by only trying clean on directories that had makefiles inside them, but this still doesn't fix the issue of switching branches leaving behind build files, but no makefile that tells how to delete them.

So for a complete solution, I would recommend changing the module build system to work something like this: have all modules create all their build files in a subdirectory (e.g. bin/). Then have the modules/Makefile, clean by simply deleting all modules/*/bin/ directories it finds, rather than relying on calling the module's makefile (which might not be present). Of course this would be a rather sweeping change to module build system and require touch a lot of files.

Putting build files in a subdirectory would have the added benefit of avoiding all the clutter in the modules' source directories which can make it hard to find the actual source files when browsing to them.

Doubt I'll have time to do it myself, but it would be a good easy coding task.

Danne

QuoteThis is a huge problem when switching branches with different modules in them, since the version control will delete the makefile when switching branches, but none of the build files or the directory.
Oh yes.

grnbrg




how to post pictures online for free
Isolated the crash I was having last week, related to LiveView and shutter speed.  Running this against todays (Jul01) lua_fix build.


-- 5DmkII LiveView crash test
-- 5DmkII + LV + fast shutter speeds causes a crash.

menu.close()
console.show()

print ("This will work.")

key.wait()

task.yield(1000) -- wait for the key to be released

camera.shutter.value = (1/8000)

print ("Shutter value: ", camera.shutter.value)

lv.start()

print ("This will work.  (Sort of.  It also starts a recording?)")

key.wait()

task.yield(1000)

camera.shutter.value=(1/4000)

print ("Shutter value: ", camera.shutter.value)

print ("This will crash.")

key.wait()

task.yield(1000)

camera.shutter.value = (1/8000)

print ("Shutter value: ", camera.shutter.value)

key.wait()

print("Done.")


Interestingly, this only crashes on my 5DmkII.  My 70D runs it flawlessly.

Essentially, (on the 5DmkII) once LiveView has been activated, trying to set a shutter speed faster than 1/4000th of a second crashes the script with a complaint that "camera.shutter.value" (or whichever shutter field) can't be set.  Odd.


(Brian) grnbrg.

a1ex

Quote
Essentially, (on the 5DmkII) once LiveView has been activated, trying to set a shutter speed faster than 1/4000th of a second crashes the script with a complaint that "camera.shutter.value" (or whichever shutter field) can't be set.

Same does the 5D3 when running this script in Movie mode (Canon limitation).

If you need faster shutter speeds, look into shutter fine-tuning (Movie menu). You should be able to push it to 1/50000 on 5D2 (5D3 only goes to 1/25000).

Quote
(Sort of.  It also starts a recording?)")

Did you press SET when the script was waiting for a key? key.wait() doesn't keep the event from reaching the Canon firmware (maybe I should fix that, as it's not expected behavior, but it's a bit tricky to handle in a cross-platform way, as there are some other gui events that are similar to buttons, but should be passed back, and they are model-specific)

bastien42

Quote from: nikfreak on July 05, 2017, 12:02:08 PM
bastien42, the whole overlay (inlcuding waveform, histo, scope etc..) will be visible in playback mode like on all other cameras which have "BTN_ZEBRAS_FOR_PLAYBACK" defined. Guess it will be included into the experimentals once a1ex finds it worth to include. I could update ofc the 1st post with a "nightly" but that wouldn't be the same "experimental" build that a1ex provides you. I personally find we are already overdue with becoming part of the nightlies and i decided to not merge the "experimental" stuff myself as it would just interfere.
So I'm passing by to just ask when and if a1ex saw the new update that nikfreak did if no I shall just ask for it can't wait to have a hand on it and when do you find you could implement it into the experimental cheers :)

so you know the link to the conversation for background ^^ : http://www.magiclantern.fm/forum/index.php?topic=16040.600
100D ,EF-S 18-55mm f/3,5-5,6 IS STM, EF-S 55-250mm f/4-5.6 IS STM, EF 50mm f/1.8 STM, Yongnuo YN-568EX II ,Manfrotto Compact + 3D head

a1ex

Quote(Sort of.  It also starts a recording?)")

Hopefully fixed. That means, key.wait() should now block the key(s) it's waiting for, and pass all other events to Canon.

For example, in hello.lua:

print "Press SET to exit."
key.wait(KEY.SET)
console.hide()


That means, you can press any key other than SET, and it will be passed to Canon code (or to other ML modules). Or, you can press SET to close the console; in this case, SET should not perform other actions (such as starting to record on 5D2).

That's a very simplistic way of handling keys; for more general cases, you can use event.keypress (where you can choose what keys to block and what keys to allow, useful for interactive scripts), or keys.lua (it blocks all keys, so it's suitable only for full-screen apps like editor and calc).

grnbrg

Quote from: a1ex on July 02, 2017, 01:08:47 AM
Quote from: grnbrg on July 02, 2017, 12:30:01 AM

Essentially, (on the 5DmkII) once LiveView has been activated, trying to set a shutter speed faster than 1/4000th of a second crashes the script with a complaint that "camera.shutter.value" (or whichever shutter field) can't be set.  Odd.

Same does the 5D3 when running this script in Movie mode (Canon limitation).


I found a solution!  At least for my needs...  The shutter speed limitation is only imposed in movie mode.  This (on the 5DmkII, at least) is controlled by a menu option:



If I set this to "Stills only", I get the full range of shutter speeds back.  I suspect the reason I had no issues with the 70D is that body selects movie mode via a hardware switch.

Thanks for your time and work!



Brian.

BBA

I need your advice to help me choose a good environment for debugging Lua scripts before testing and using them in camera.

One way or another, the question has already been asked but I would like to add some requirements (if possible).

The on-camera Lua uses modules libraries to help (thanks very much for that) Lua scripting developers.
I have tried to emulate some of those modules (sometimes in a dirty way I am afraid, but nevertheless it still provides some help).
For instance, the menu emulation is not a priority...

I would really like to emulate a minimum set of the display module functions (to display lines, rectangles, text) and debug the scripts on a desktop computer (I am on a mac).
It would really help a lot.

I have tried to use the "turtle.lua" module in the ZeroBrane studio environment which, in turn, makes use of a wxWidgets module (require "wx") .
This environment seems interesting as some static checks can be made on the code (I am a newbie).
The problem is that the pointers are difficult to follow as only the "reference" address to user data is displayed while debugging : those references can change over time.

What do you think ?
Should I continue in that direction or do you know a better environment to use ?
Is there something I should better use in "Visual Studio Code" ?

Many thanks in advance for your help !



garry23

@BBA

I wasn't going to post on your last comments, but here goes.

Like you, I would love to have a proper Lua development environment running.

I currently use Visual Studio Code, but have failed to even get it running in debug mode: I'm sure the problem is me, as I also still have issues compiling ML. Bluntly, I get 'confused' with all the references to forks etc.

So, I can code, ie in Lua and even a little C++, but my issues start when I try and debug or compile.

So my Lua workflow is very simple and inefficient. Use VSC as a 'text editor' and try and debug via ML in-camera errors: Ugh!!!!!

chris_overseas

It might be worth looking at IntelliJ IDEA Community edition (https://www.jetbrains.com/idea/download/) combined with the Lua plugin (https://plugins.jetbrains.com/plugin/5055-lua - don't download this, just install it through IDEA by choosing File->Settings->Plugins->Browse Repositories and search for "Lua"). Note that I haven't tried the Lua plugin myself (and given it is written by the 3rd party, it's quality is uncertain), but it does appear to have some debugging support and a host of other useful features. IDEA itself is superb for the other languages I use (Java, Kotlin, Python) and I can't imagine using any other IDE for those. I also dabble in a bit of C/C++ for Arduino using CLion which is a C++ IDE based on the same codebase as IDEA, so the Lua plugin should work with CLion too.

If anyone does try the above, I'd be interested to hear how it goes.
EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II

BBA

@garry23

Never mind, thank you for you post : you are welcome !
I agree with you.
Honestly, I am impressed by the results you can achieve with (y)our inefficient workflow... like with the « focus.lua » script.
I have tried the focus bar at home (not last version though) : impressive to see the sharpness that can be achieved when « correctly » focusing : thanks very much for that !!!

One thing is that when « drawing » man machine interfaces, it is important to see the end result : interactive debugging is a necessity because it may be necessary to restart from scratch at any time to better display/interact with things.
Little story : I have coded in Fortran 77+ thousands of lines of code for a « plotting graphics package »: I could  just print, on a listing, the values of the variables to find the errors (between compiling and linking sessions) : It's like blind work but somehow, I liked it because it was somehow like a detective story. I was younger and there was no other way to do that. At the end, I could still find an error a day (plotter pens are very precise). I think It makes you careful at avoiding errors at the time of writing ( when you know the language, which needs tests and errors ).

There are more effective development tools now : with internet, everything is exploding (in french : explosion combinatoire ). It becomes important to be able to follow a correct path in that jungle and avoid the foreseeable dead ends (bad english ?).
I have managed to partially emulate the display module with wxLua in the ZeroBraneStudio environment though I only know a very small part of the wxWidgets capabilities. As it is, it is dirty and wx would need to be studied further because there are lots of useful features there.
That's why I decided to post here : I think it is it a « correct » direction to go but I have no experience at all at a higher level.
I don't want to make a perfect/complete emulation (always a work in progress), only the necessary functions we use.
Maybe I am wrong but the Qemu environment has been said to be usable to test lua scripts.

I use ZeroBrane Studio because it is the Lua environment I have been able to easily push the further (furthest?) for ML lua scripting :
- i can test snippets of code and debug them : it is useful to learn lua which needs to be learnt carefully and deeply;
- it has a « use as scratchpad » project mode where you can  « live » test small scripts, changing values and options in real time;
- the code can be statically analyzed (till a certain point : first use of (undeclared) variables, unused declared variables, unused function parameters,....);
- I can use wxLua with it to draw : I hope to be able to use event driven code to emulate part of the key module and, maybe, a dirty lens module ; I am less interested in menu bars, radio buttons, ....

@chris_overseas

I had not seen your post before.
Thank you; I will take a look

garry23

@chris_overseas

Downloaded and tried the IDEA Lua: as usual I failed :-(

Too many errors in just getting it going and too complicated.

I'll stick with my inefficient, but for me workable, workflow.

Cheers

Garry

garry23

@BBA

Pleased you've tried out the focus bar. I must say it is the script I'm most proud of.

BTW the latest version provides a nice feature that allows you to optimize the depth of field at infinity and in the near field, using different blur criterion. Giving you the best chance of nailing the optimum capture for your needs.

Then, of course, if one image is not sufficient, you can use the focus stacking feature.

Of course the above is my humble view. Based on feedback, i.e. not much, you and I may be the only ones using it  ;)

Cheers

Garry