Canon EOS M

[Danne]

Here is a version with filled preview screen in mv1080p. Didn´t test so please report back.
Will mainly work with mv1080p mode selected or x5 modes. If crop_rec disabled expect corrupted files...
https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2018Aug07.EOSM202_mv1080p.zip

@IDA_ML nothing new for 100D coming from here yet...

[ricardopt]

Hi Danne, thanks for the build...one quick question...do we need to follow the following steps to use mv1080p:

https://www.magiclantern.fm/forum/index.php?topic=9741.msg204144#msg204144

i activated crop_rec and mlv lite and it works...do i need to activate the sd uhs hack?

[Danne]

You don't need the hack but if used you should be able to record continuous 24 fps with a fast sd card.

[ricardopt]

just noticed that, only able to record 6 seconds, i only got into this whole "raw/magic lantern/eos-m boat" recently... i apologize for perhaps asking the same questions.

1- to get 1736x1120 i need to setup the aspect ratio to 1:1 or 1:2, thats the only way i can select that resolution that i'm aware.

2- Is there a difference between mv720p and mv1080p in a "x" resolution? For example 1728x736 is the highest i can select using an aspect ratio of 2:35 on both mv720 and mv1080.

Thanks

[Danne]

1: yes
2: check files, not able to compare output. If you manage to get 1120 working just stick to that if it's working.

[Pacerx999]

Also this helps for general mv1080 setup this guide helps:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg204144#msg204144

I find that with 1736x1120 I need a write speed of 44.6 MB/s which requires the SD UHS hack to  get 40+ MB/s speeds

The Framing preview is now un-stretched which is great, but obviously still not in realtime (which would be grand).

I've had a couple of issues where pressing half shutter to go realtime means that the camera doesn't go back to more accurate preview.

There also appears to be a difference in the brightness of the accurate preview vs realtime preview.

The realtime preview appears to have accurate top/left/right framing, just the bottom say 15%. I'll try and shoot some charts to see if I can get an accurate read on it.

In terms of the actual shooting, the files are coming out without corruption. I've shot for approx 1min continuous, but will see what the max limit it.

MLV app takes the footage just find and reads the metadata correctly so isn't applying any transformation.

More tests to come, but it's looking pretty solid. The icing on the cake would be getting a more realtime, accurate preview for monitoring etc.

But overall, thumbs firmly up

More to come.

[darthvim]

I am wondering, if it would be possible to get 2.5k recording with proper framing on the screen, so without the 5x zoom. Do you think this could be possible ?

[Danne]

Set preview to framing in raw video settings. Should get you slowish full screen preview.

[darthvim]

so it has proper framing? good to know. thank you . Do you think one day it can be faster?

[jandy123]

First of all, thank you all devs. for the hard work you guys put in the ML development.

===

I'm a new user of ML on an EOS M and very much impressed by the features offered by ML on this camera... I also have a feature I miss, that is focus stacking, which according to the feature matrix is not supported on EOS M. I was wondering, what is the missing info that is required to get this feature working ? I was studying the ML code and I already have some prime suspect, but I'd like to ask advice from devs.

Also, what are the main tools for figuring out possible stubs/property cases, etc. ? (Btw., I know a bit about how CHDK works, but I'm new to ML.)

Thanks !   

EDIT: Well, in the meantime I went ahead and recompiled ML sources with enabled FEATURE_FOLLOW_FOCUS, FEATURE_RACK_FOCUS and  FEATURE_FOCUS_STACKING, and (at least) focus stacking seems to work . Am I missing something ?

[a1ex]

Welcome, other folks from CHDK are probably still around.

For me, the main reverse engineering tools are currently QEMU and io_trace. Unfortunately, the former can't handle LiveView yet. The latter can get some very detailed low-level logs, but it's not exactly focused on properties.

For properties, here is a global property handler in debug.c that prints the raw data with tiny fonts and highlights the changes. I haven't touched this code in years, so it may not compile out of the box. Edit: this seems to fix the compilation:

Code: [Select]

diff -r d05eaebda31d src/debug.c
--- a/src/debug.c
+++ b/src/debug.c
@@ -26,2 +26,4 @@
 
+#define CONFIG_DEBUGMSG 1
+
 #ifdef CONFIG_DEBUG_INTERCEPT
@@ -1127,3 +1129,4 @@
 
-ack:
+ack:;
+    extern void* _prop_cleanup(void* token, int property);
     return (void*)_prop_cleanup( debug_token, property );

You could also add prop_request_change_log to a gdb script and run the firmware in QEMU; this will cover most UI properties, but not autofocus, since that's controlled by the main CPU only in LiveView, which is not emulated well.

For focus functionality, please try the lua_fix experimental branch, as it has some significant backend changes in this area. It looks like some newer models still have some problems with it though, as api_test.lua doesn't complete. For EOSM, dfort looked into this here and here; also had some detailed feedback from garry23. Still need to refresh my memory about the outcome of their experiments...

[jandy123]

@a1ex: Thanks for the info. I'll start reading the posts in the links you gave me.

But, maybe you haven't noticed my Edit in the initial message, but focus stacking definitely works here . One issue is that sometimes after running the stack the focus gets stuck, i.e., it doesn't respond to half-shutter presses. Any idea how this can be cured ?

Thanks ! 

[a1ex]

Other than finding a way to reproduce and inserting printf's in the source code to see where it fails, no idea. Is the entire UI locked up? or just half-shutter? or only some of the actions triggered by half-shutter, such as autofocus?


Some unrelated issue I've noticed last night. Looks like the EOS M might be affected by this bug, but I'm unable to see any weird things in QEMU (likely because LiveView is not emulated well). I did notice 90 is not a valid value for GUIMODE_ML_MENU and ran some tests on a 700D (borrowed one for a few days). Based on these tests, I'm expecting the following:

- if you enable Canon histogram with the INFO key, then open ML menu in LiveView, some parts of our menu might be erased by Canon firmware;

- if you enable face detection AF, then yoou open ML menu and then you point the camera at some face, the face detection box might pop up over ML menu;

- if the menu gets erased by Canon firmware in the above scenarios, or other similar ones (maybe audio meters or other animated overlays), I'd expect this change to fix it:

Code: [Select]

diff -r 6764356de075 platform/EOSM.202/consts.h
--- a/platform/EOSM.202/consts.h
+++ b/platform/EOSM.202/consts.h
@@ -106,3 +106,3 @@
 
-#define GUIMODE_ML_MENU ( RECORDING_H264 ?  99 : 90 ) // any from 90...102 ?!
+#define GUIMODE_ML_MENU ( RECORDING_H264 ?  99 : 91 ) // any from 90...102 ?!

Maybe either 99 or 91 could work for both standby and recording; IIRC the values were tweaked in order to allow scrollwheel operation in ML menu.

Can you or anyone else confirm the issue?

[Danne]

Can confirm the issue below and the provided fix takes care of the issue. ML menu stays clean, not erased when fix provided.

- if you enable face detection AF, then yoou open ML menu and then you point the camera at some face, the face detection box might pop up over ML menu;

Only way I know how to enable histogram is by opening up an image or MOV file and then hit info button. Exiting this previewing mode is getting me into liveview, no problems. Was this to test for? Couldn´t find any issues before or after.

- if you enable Canon histogram with the INFO key, then open ML menu in LiveView, some parts of our menu might be erased by Canon firmware;

[jandy123]

Other than finding a way to reproduce and inserting printf's in the source code to see where it fails, no idea. Is the entire UI locked up? or just half-shutter? or only some of the actions triggered by half-shutter, such as autofocus?

It's only the autofocus system not reacting to the half shutter. The GUI behaves normally, well sort of... When the issue happens, if I press the half-shutter the Canon bottom bar appears (as usual), but (re)focusing doesn't work (not even the focus window appears). Then, if I depress the shutter and try a second time,  the bottom Canon bar appears but shows that exposure is locked. IF I also press the exposure lock button, and keep shutter pressed half-way, then focus seems to kick in. If I depress the buttons focus becomes inactive and it all repeats... Btw. I can shoot a picture at any time, but it will be out of focus. So, to me this sounds more of a messed-up state, right ?

 

[a1ex]

Right, sounds like something's messed up on the MPU side (the secondary CPU that actually turns the focus motor). On older models, I had to throttle the AF commands to avoid a camera crash. Maybe the EOS M has similar quirks.

There are some low-level focus commands that can give a very smooth movement, but they also leave the MPU in a messed up state on other models. Tried a couple of times to get some half-decent operation with these, unfortunately without success.

The MPU architecture on DIGIC 5 and newer is not exactly something I'm familiar with. In contrast, the architecture of the MPU used on DIGIC 4 (TX19A) is well documented and emulation is probably doable, given enough time to sit down (very unlikely during the next year, from my side). Understanding what the MPU code does might be helpful.

One low-hanging fruit is recording Canon's debug messages during AF in LiveView, hoping the focus commands are initiated on the main CPU. The startup-log-mpu builds might be a good starting point. Then, one may try to replay the MPU messages, hoping some of them may be low-level focus commands.

@Danne: on other models, pressing INFO while in LiveView activates Canon overlays, and one of them has a histogram. I'd be surprised if the EOS M doesn't have one.

[Danne]

Got it. Histogram correctly eats through ML menu and the fix is the same as for face detection. Confirmed working.

[a1ex]

Cool. While you are at it, can you check whether 99 is required for browsing ML menu while recording H.264?

This value was introduced back in 2013, commit 5043182; possibly unwanted change, as that commit included some unrelated stuff, besides what was described in the commit message. Just wanted to know what's up with it.

From QEMU experiments, I'd expect both 91 and 99 to work fine for both standby and recording. Maybe one of them blocks scrollwheel events or has some other quirks, no idea.

[jandy123]

@a1ex: I'll have to look into the throttling commands. Thanks for pointing me to your commit.

One more little annoyance is the bottom Canon bar overwriting the ML one, when half-shutter pressed. I've came across a commit dealing with this -- forgot exactly which one -- but that would merely freeze the ML bar so that the Canon one takes cleanly over. Not exactly the right way, I'd say....

What I have in mind is a follows: By default, show the ML bottom bar, as is. If half-shutter pressed, restore background (part of image in LV), hide the ML bar from the bottom-most position and have it rendered on top of the (bottom) Canon bar. All additional stuff like DOF (near/far points) can be translated on top of the ML bar AND be updated in 'realtime'. At this instance (half-shutter still pressed), some items from the ML bar can also be hidden, since they are redundant (they appear anyhow in the Canon bar). This in my opinion would be nice... 

For now, I simply moved the ML bar upwards just enough such that the Canon bar never overwrites it. It's quite fine, though what I've tried to describe above would be nicer, I think. Is this doable ? May main question is now -- can we quickly 'erase' the ML bar ? I.e., restore image pixels underneath ?

[a1ex]

I'd probably try to block Canon bar completely; unfortunately, that's not straightforward (read: Canon code is too convoluted for me to understand what to patch). So, the next best thing was to block our bar and allow theirs. On 700D it works reasonably well, but there are some parts of Canon overlays conflicting with ML top bar this time. These parts are not present on older models.

Maybe you have looked at this PR ?

Caveat: this tiny issue may turn into a big rabbit hole: we have already tried many workarounds (though none of them attempted to save and restore the bitmap overlay contents). And I don't currently have a way to test this workaround on cameras I don't own (need to update the emulator to handle LiveView, and that's not exactly an easy task).

[Danne]

Tried:

Code: [Select]

#define GUIMODE_ML_MENU ( RECORDING_H264 ?  91 : 91 ) // any from 90...102 ?!While recording h264 and entering ML menu will work but almost immediately goes out of ML menu and some buggy black screen will appear when stopping recording. Pressing menu button and again let´s me start using the camera again.

Retesting again with:

Code: [Select]

#define GUIMODE_ML_MENU ( RECORDING_H264 ?  99 : 91 ) // any from 90...102 ?!and everything works as expected. No bugs and going into ML menu works while recording and so on.

[a1ex]

Alright, what about defining both as 99?

[Danne]

Code: [Select]

#define GUIMODE_ML_MENU ( RECORDING_H264 ?  99 : 99 ) // any from 90...102 ?!Also works fine.

[jandy123]

Maybe you have looked at this PR ?

Indeed. Problems with this are: i) freezes the ML bar (no distance update near/far, etc. info), and ii) Canon's bar only partially covers the ML one, so it looks ugly.

Caveat: this tiny issue may turn into a big rabbit hole: we have already tried many workarounds (though none of them attempted to save and restore the bitmap overlay contents). And I don't currently have a way to test this workaround on cameras I don't own (need to update the emulator to handle LiveView, and that's not exactly an easy task).

What would it take to restore image info underneath ? On the other hand, it's not a big issue, as you've already pointed out. Also, I'm fine with my current solution (moved the ML bar slightly upwards so that it's never under the Canon bar, and re-enable real-time updates).

One more thing . I've played with magic zoom. I have it kick in on half-shutter + zoom ring (Default, I think). However, it only works every other time I use the half shutter. So on first half-press, magic zoom pops up as expected. If I depress half-shutter, it's still on screen... Only if I push half-shutter a second time it goes away, then it all repeats. Is this the normal behaviour ? I suppose not. I'd expect, once I release the half-shutter, it goes away and then the whole sequence repeats.

What could I do to attempt fixing this ? Where should I start ?

[a1ex]

Canon code uses double-buffering; the back buffer ("idle" buffer in ML source) might be good enough for a backup copy.

At first, Magic Zoom used the Zoom In button for toggling; that would require one click to enable the zoomed window and another click to disable it. However, on some models, the Zoom In button did not work while recording (the MPU just does not send these events); in this case, half-shutter was used instead, but the overall logic wasn't changed. I assume you want it to be displayed only while holding half-shutter, right?

The toggling logic is in zebra.c, handle_zoom_overlay.

Similar tweaks were requested for the DIGIC peaking feature and for Canon's x10 zoom shortcut via half-shutter. The former should be easy, the latter conflicts with Canon's AF, so it's only enabled in MF.