Proposal: user docs embedded in ML source code?

[a1ex]

The small one-line help strings are pretty much always up to date. Would be nice if we could say the same thing about the larger help pages (they are not always in the same order as the menus, it's hard to tell which camera has which features, or sometimes the functionality just changed).

So, a proposal somewhat inspired from the literate programming philosophy:

Code Select Expand


#define DOC(x,...)

struct menu_entry zebra_menus[] = {
    {
        .name = "Global Draw",
        .priv       = &global_draw,
        .max = 3,
        .select_Q   = toggle_disp_mode_menu,
        .display    = global_draw_display,
        .icon_type = IT_BOOL,
        .help = "Enable/disable ML overlay graphics (zebra, cropmarks...)",
        DOC(
            Choose when to enable Magic Lantern overlay graphics: (zebra, cropmarks, histograms, audio meters, ML shooting info...):

            * in **LiveView** only;
            * in **QuickReview** (photo mode outside LiveView) only;
            * **Both modes** (default);
            * **Disabled**.

            If you use `display presets`_, press Q on this item to switch the presets while you are in ML menu.
        )
    },
    ...


So, the RST code is embedded in ML source code, is discarded when compiling, and probably can be extracted without much effort. One can also preprocess the source code (mainly the camera-specific ifdef tags) and see what features are enabled on what model. This also ensures consistent naming of menu entries and help pages (since help pages are identified by the name of the menu entry).

Advantage: the help text is in the middle of the data structure that describes the menu (so it's probably a lot easier to notice when it becomes outdated, and just update it). It will probably be also helpful for new ports (one can see what feature X does without having to open a separate window for the manual).

Current user guide takes under 100K, so there's also a possibility to render the help as plain text (not as bitmaps). It will increase ML binary size and may not look as nice as the pre-rendered BMPs.

Thoughts?

[ilguercio]

Poll? It's ok for me, i build my own autoexec for the 50D only so the bin file doesn't get too big. I can live with a bigger autoexec and not have BMPs for the help.

[a1ex]

Here's a more complete example. This time I've tried some special comment syntax: /** doc text here **/, so the doc contents are not included in the binary file, but they can be extracted with a simple parser and used for building the user guide.

http://pastebin.com/z0SUDAEF

Aside for the obvious syntax, there are a few special cases: some features might have multiple label names (e.g. magic zoom - zoom while recording), or some features might share the same doc page (e.g. histogram and waveform). These are optional, the docs can be rewritten to avoid these things and keep the syntax clean.

[jplxpto]

Alex,

I understand your arguments, but this time I do not agree with you. Sorry.
This your idea reminds me of the java docs.
The documentation is written in the middle of the source code. I confess that despite giving solution seem simple I never liked her much.
I think, the source code must be as clean as possible comments.
In my view, the documentation mixed in code only makes it unreadable.
This solution also has a major limitation makes it difficult to have the documentation in several different languages​​.

[jplxpto]

I think the great difficulty in keeping up to date documentation is in the structure of the code itself.
Maybe it was a good idea to put the structures that define menus in separate files (menu-expo.c, menu-overlays.c, movie.c-menu, menu-debug.c, etc.)

I'd like to see a # define for each feature. This made it easy to port of new cameras. The features that were not yet functioning simply would not be defined for the new port. These # defines also allowed to control the items that appear on menus and text that was written in the pages of the help menu.

[a1ex]

Now that we have nice proportional fonts, I'm tempted to revisit this and get rid of the old BMP help pages (so we'll have a plain text user guide available in the camera).

The main point of this approach is to have docs in the same place as menu entries, in order to avoid the docs becoming outdated.

A similar approach was used in the Lua API by dmilligan.
Example code with docs as special comments: https://bitbucket.org/hudson/magic-lantern/commits/90b08db80895
Autogenerated docs: http://davidmilligan.github.io/ml-lua/

[dmilligan]

I was thinking of writing an in camera text editor in lua (I have just about all the necessary APIs done for such a thing). Drawing routines from this could also be used to display help information. Lua makes a really good configuration language (because of the block quotes and how easily lua code can serialize itself into valid lua code). So the help files themselves could actually be simple, auto-generated lua code that knows how/when to draw itself.

Quick Example:

Comments in C code:

Code Select Expand


/***
@title Intervalometer
@summary Take pictures or movies at fixed intervals
@description You can stop the intervalometer by rotating the mode dial, by pressing MENU or PLAY, or by turning off the camera.
To avoid flicker, shoot in manual mode, use manual white balance, avoid short exposure times and use a manual lens (if you use an
EF lens, lock the aperture and unscrew it).

To make a timelapse without increasing shutter count, do not use the intervalometer; instead, set FPS override to a very low value (for
example, 3fps) and start recording. When using the intervalometer in LiveView with noisy mode, your shutter will wear twice as fast
than outside LiveView.

If the intervalometer can't be stopped (it may happen in crazy mode), turn the camera off or open the card door. Adjust your auto-off
setting to longer than your timelapse interval the camera will turn off before the second shot. When not in LiveView, press DISP or
INFO to turn the display off. In LiveView, ML will turn the display and the sensor off during idle times if you enable this option from
Powersave menu. While the intervalometer is running, the card led will blink once per second to let you know it's alive and kicking.]]
*/


becomes =>

Code Select Expand


title = [[Intervalometer]]
summary = [[Take pictures or movies at fixed intervals. ]]
description = [[You can stop the intervalometer by rotating the mode dial, by pressing MENU or PLAY, or by turning off the camera.
To avoid flicker, shoot in manual mode, use manual white balance, avoid short exposure times and use a manual lens (if you use an
EF lens, lock the aperture and unscrew it).

To make a timelapse without increasing shutter count, do not use the intervalometer; instead, set FPS override to a very low value (for
example, 3fps) and start recording. When using the intervalometer in LiveView with noisy mode, your shutter will wear twice as fast
than outside LiveView.

If the intervalometer can't be stopped (it may happen in crazy mode), turn the camera off or open the card door. Adjust your auto-off
setting to longer than your timelapse interval the camera will turn off before the second shot. When not in LiveView, press DISP or
INFO to turn the display off. In LiveView, ML will turn the display and the sensor off during idle times if you enable this option from
Powersave menu. While the intervalometer is running, the card led will blink once per second to let you know it's alive and kicking.]]



In lua you just dofile() this file to show the help.

Just some thoughts.

[Erik Krause]

I like this proposal. I guess it would be the best possibility to have up to date help files once. I'd like to do some work to make this happen. Unfortunately I'm not familiar with the ML development ecosystem at all.

I read through the forum a bit and I think the text from the User Guide thread is the best we have currently. I wouldn't like to wait until they are up to date, since this probably won't happen. Could this be copied automatically to the source files or to an intermediate place like the structure proposed by kitchetof and already implemented in https://bitbucket.org/kichetof/magic-lantern/src/2208edf04a73b86a839fefd799329521cea9b6f0/help/en/?at=help_files ?

I understand the argument by jplxpto about the difficulty to have different language texts  in the source files. However, the english version could be considered as master with other languages held in indexed files, such that they are used if present. It would be relatively easy for users to provide such files.

My guess about what needs to be done:

• The developers should agree on a syntax and format, be it comments or conditional defines.
• Current text from the User Guide thread should be copied to either the source files or an intermediate place for peer review and update.
• A mechanism should be established to provide the help in-camera (if not the old bitmap format is used)

And all that, without increasing the workload for the developers too much.

Is this somehow correct or am I totally wrong? What would it be I can do immediately? I have some experience in wiki editing (I'm the maintainer and one of the main writers of wiki.panotools.org, I dabble in PHP, HTML, CSS and javascript, I can read C and python a bit).

[a1ex]

I'm the maintainer and one of the main writers of wiki.panotools.org

Sounds great; big fan of enfuse here

There were voices against the old BMP format since we've got proportional fonts, but currently there's nothing to replace it either. The shortest path is probably to fix the LaTeX build process for the BMPs and just stick to that.

What you can do right away? Since you mentioned wiki, maybe a good idea would be to transfer the newest user guide from the Redrocks forum post to the wiki. The content should have been there (on the wiki) from the beginning, if you ask me, but looks like there were some login issues (not everybody could edit the wiki back then). I hope these are solved by now; if they are not, please report back.

Once the content is on the wiki, I think it's best to have a way to update the content both ways:
- from source code (comments or macros or whatever) to wiki (likely straightforward, RST to wiki markup)
- from wiki back into source (user edits turned into source code changes, possibly with semi-automated pull requests)

That should solve the problem of docs not staying up to date, and make it easy for community members to edit them.

[Erik Krause]

Wiki login works with the same credentials as for the forum. However, apparently I don't have permissions to create a page, not even in the playground. And I didn't find any hint on who is admin or moderator of the wiki or otherwise responsible.

[Erik Krause]

Just to be clear: wiki means https://wiki.magiclantern.fm/ not the one on wikia. Correct?

[a1ex]

Right; forwarded to g3gg0. You should be able to edit existing pages meanwhile.

[g3gg0]

does it work for you?

[a1ex]

I can edit existing pages, but cannot find a button to create new pages either.

[Floont]

I'm not sure what wiki software you guys are using, but it seems to work like Dokuwiki - just type in the URL of the page you want to create (e.g. https://wiki.magiclantern.fm/testing123), then you'll get a "This topic doesn't exist yet" page with a "Create this page" button at the bottom.  Once it's created, you can link to it from other pages.  I'm not sure if that's the best way to create a page, but it works...

(Edit: duh, it is Dokuwiki...)

[Erik Krause]

Tried that, but I didn't get a Create button.

[Floont]

That's odd... here's what I see when I click the link in my previous post:





...and clicking the "Create this page" button does seem to create a new page (although I haven't actually saved one, since I don't have anything useful to add.)
Perhaps there's some interference from an ad blocker or privacy plug-in?

[Erik Krause]

It works now. Hope it stays that way ;-)  Sorry for the hassle...

[Erik Krause]

A first version of the User Guide thread https://www.magiclantern.fm/forum/index.php?topic=11269.0 is copied to the wiki. It's a dump copy, I didn't any updating so far. I've split it in two pages to have the part intended for in-camera help separate. Currently in playground namespace: https://wiki.magiclantern.fm/playground:camera_help and https://wiki.magiclantern.fm/playground:general_help

dokuwiki doesn't allow for links in the headlines, hence the URL's are below the respective headlines where  necessary.

What do you think? Shall I transfer it to the main namespace? Is the name ("Camera Help") ok? What would be the right forum section to publish this and ask for updating help?

[a1ex]

That way, we'll have two different sets of documentation to maintain: one for camera and another for offline reading. A large part is going to be redundant.

I'd prefer one single manual, but use some markers (possibly color) for paragraphs (or formatting codes) that should only appear in various versions of the manual.

Also, many entries simply enumerate the available submenu options and repeat the help currently displayed at the bottom of the screen:

Automatically expose to the right.

Submenu options:

Trigger mode Always ON, Auto Snap, Press SET & HalfS DblClick. Method for executing Auto ETTR.

Slowest shutter 32" to 1/4000. Set your lowest speed.

Exposure target -0.5, -1, -2, -3 & -4EV. -0.5 or -1 recommended.

Highlight ignore 0 to 50%. Percentage of bright pixels allowed above target level. Use this to allow spec(ta)cular highlights to be clipped.

Allow clipping OFF, Green channel & Any channel. Choose which channels to be clipped.

Midtone SNR limit 0 to 8EV. Stop underexposing when at least 50% of the image gets noisier than selected SNR.

Shadow SNR limit 0 to 6EV. Stop underexposing when at least 5% of the image gets noisier than selected SNR.

Link to Canon shutter Hack to adjust slowest shutter from main dial.

Link to Dual ISO Let ETTR change Dual ISO settings so you get the SNR values in mids & shadows. It will disable dual ISO if not needed.

Show metered areas Show where the white point and SNR levels are metered (what exactly is considered highlight, midtone and shadow).

Show debug info For nerds.

What's the point of duplicating the information? You can already get the same info by scrolling through the menu and looking at the bottom of the screen. Many users are not reading that info though; maybe the font needs to be enlarged or the color changed?

For each option, I'd prefer one page of background info, without repeating the available options and their help.

[Erik Krause]

Ok. So that seems not what you expected... I see your poínt of course, but my current goal was to transfer the content of the User Guide thread to where it can be easier edited.

That way, we'll have two different sets of documentation to maintain: one for camera and another for offline reading. A large part is going to be redundant.

I don't think so. Wiki syntax can be translated to markdown or RST and vice versa. Or at least that'S what anyone says.

I'd prefer one single manual, but use some markers (possibly color) for paragraphs (or formatting codes) that should only appear in various versions of the manual.

That would be better indeed. However, I guess the casual user will more likely edit a wiki paragraph than the source code. So the next step should be to motivate users to enhance the wiki and in parallel investigate how to transform to and from RST which can be used in the source code. There are several projects on github that can be used as a starting point.

Also, many entries simply enumerate the available submenu options and repeat the help currently displayed at the bottom of the screen

Yes, apparently it's intended as a framework to be filled by the users.

In time all this information will be transferred to the official User Guide and you will have it on your camera, but we have to create that text first and this is one way non developers can help the community.

That was four years ago and there was little progress apparently. My hope is, that now that the text is on the wiki it will go a bit faster. I leave it in the playground for now, perhaps the threshold is lower this way for people to help.

As far as I researched there are few ways to convert dokuwiki to RST and vice versa. Unfortunately Pandoc doesn't support dokuwiki directly, it first has to be converted to markdown (mediawiki would be easier in this regard). Or are there scripts already that do the conversion?

Dokuwiki features a lot of plugins, there's even one, which interprets RST, but I fear the nice WYSIWSG editor won't work with this. Any dokuwiki specialists here?