Author Topic: ML on EOS-M2  (Read 252697 times)

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7240
Re: ML on EOS-M2
« Reply #225 on: January 23, 2018, 04:54:56 PM »
dfort, what an achevement, bravo, and happy birthday as well. And thanks to a1ex who makes all of this possible.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #226 on: January 23, 2018, 06:42:06 PM »
Thanks @Danne - Not much is working and it has a long way to go but it feels like we reached a milestone.

@a1ex - The dm-spy-experiments merge broke a few things so I think I'll make that a separate branch and keep the wip branch in its current somewhat working state so I can cherry pick the pieces to put into a new clean branch.

I was able to create a startup log with mpu events that might help with QEMU. The camera locked up on startup and I had to do a battery pull but here's the log it created. The lock up problem also happened on the EOSM so it isn't just an EOSM2 issue, it worked fine on the 700D.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

bouncyball

  • Contributor
  • Hero Member
  • *****
  • Posts: 840
Re: ML on EOS-M2
« Reply #227 on: January 23, 2018, 07:10:05 PM »
@dfort: Congratulations! And Happy Birthday! :D

vagaboundedu

  • New to the forum
  • *
  • Posts: 10
Re: ML on EOS-M2
« Reply #228 on: January 23, 2018, 11:05:56 PM »
Congratulations on this huge milestone! I'm very impressed! It's all hidden in mystery for me. :)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: ML on EOS-M2
« Reply #229 on: January 23, 2018, 11:57:27 PM »
The lock up problem also happened on the EOSM [...]

Is it still an issue with current startup-log builds?

(I thought I've fixed that one back then, when you reported it, but maybe there are still rough edges)

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #230 on: January 24, 2018, 12:56:18 AM »
I thought we fixed it too but that EOSM build you pointed to is doing the same thing, locking up the camera on startup. The LED stays lit solid--no read/write activity and requires a battery pull. Startup logs do get saved though.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: ML on EOS-M2
« Reply #231 on: January 25, 2018, 11:51:26 PM »
If I remember well, the issue was caused by MPU_DELAY_SEND in LiveView; that's why I've redefined it to enable it only outside LV. That works fine on 5D3, even when starting the camera at 60 FPS.

Does it help if you undefine MPU_DELAY_SEND?

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #232 on: January 26, 2018, 02:21:53 AM »
Actually, now that I think of it on the EOSM what worked was to start up using the playback button. I made sure to have an image on the card so it won't go into LiveView, pressed playback and everything seems to work without a hitch. Here is the link to the startup log.

[EDIT] I also tried undefining MPU_DELAY_SEND and it worked fine starting up the camera normally into LiveView. This time it created two log files on startup.

DM-0000.LOG
DM-0001.LOG
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #233 on: January 27, 2018, 05:13:10 PM »
I have been going through the stubs and constants with a fine toothed comb and made some corrections on the wip branch before getting the pull request started.

One thing that is a problem is that the LiveView keeps freezing up. When shooting a still image it shows, "Raw error, falling back to YUV overlays" so I took a closer look at the YUV addresses and found something that I don't understand at all:

src/mem.c
Code: [Select]
static struct { uint32_t addr; char* name; } common_addresses[] = {
    { RESTARTSTART,         "RST"},
    { YUV422_HD_BUFFER_1,   "HD1"},
    { YUV422_HD_BUFFER_1,   "HD2"},
    { YUV422_LV_BUFFER_1,   "LV1"},
    { YUV422_LV_BUFFER_2,   "LV2"},
    { YUV422_LV_BUFFER_3,   "LV3"},
};

Shouldn't the "HD2" line look like this?

Code: [Select]
    { YUV422_HD_BUFFER_2,   "HD2"},

Back to the addresses I'm using, I copied this from the EOSM but it is likely different on the EOSM2:

platform/EOSM2.103/consts.h
Code: [Select]
// http://magiclantern.wikia.com/wiki/ASM_Zedbra
// skipped for now, will come up with a way to autodetect these values
// http://www.magiclantern.fm/forum/index.php?topic=15895.msg186493#msg186493
#define YUV422_HD_BUFFER_1 0x44000080
#define YUV422_HD_BUFFER_2 0x46000080

I read the links but can't quite figure it out. Back on Reply #133 I posted this QEMU output:

Code: [Select]
   270:  3187.200 [RSC] --- Indev Mode ----
   271:  3187.456 [RSC] [INDVMGR]               0x0
   272:  3187.456 [RSC] YUV                     0x4AEEA000 0x0222C000 35831808
   273:  3187.456 [RSC] YUV_OUT                 0x42000000 0x0222C000 35831808
   274:  3187.712 [RSC] INDV_WORK               0x0 0x00000000 0

Maybe that's a clue to finding the YUV422_HD_BUFFER addresses?

One of the three YUV422_LV_BUFFER addresses was found and mentioned in Reply #87 but what about the other two? I found out that the addresses on other cameras are offset by 0x410000 so taking a guess at which one of the three addresses was published I came up with this:

platform/EOSM2.103/consts.h
Code: [Select]
// Started out by using the one found value (0x4f3d7800)for all three as a workaround
// then adding 0x410000 to determine the other two.
// http://www.magiclantern.fm/forum/index.php?topic=15895.msg186493#msg186493
#define YUV422_LV_BUFFER_1 0x4F3D7800
#define YUV422_LV_BUFFER_2 0x4F7E7800
#define YUV422_LV_BUFFER_3 0x4FBF7800
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: ML on EOS-M2
« Reply #234 on: January 27, 2018, 05:23:16 PM »
These addresses all visible from the EDMAC channels screenshot (edmac.mo); can be guessed from their size.

When there are more image buffers used, all of them will end up, sooner or later, in the same EDMAC screenshot; keep the down arrow pressed so they will cycle faster.

Back then, we had all sorts of methods to guess them - diff'ing RAM dumps, FFT analysis to find image size, trying to make sense of various display structures... now it's much easier. We may still have to redo some of this low-level stuff for D6, but I don't expect any surprises on D5.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #235 on: January 27, 2018, 05:54:24 PM »
These addresses all visible from the EDMAC channels screenshot (edmac.mo); can be guessed from their size.

Where do we start guessing?




5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: ML on EOS-M2
« Reply #236 on: January 27, 2018, 07:31:22 PM »
Here: http://magiclantern.wikia.com/wiki/VRAM/Geometry
or here: http://www.magiclantern.fm/forum/index.php?topic=15233

The LV buffer is 720x480 in most D4/5 cameras (the only exception being 1100D); the HD buffer is usually a bit larger.

(I should get rid of all these hardcoded buffers, as there is at least one camera where their address is not fixed)

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #237 on: January 28, 2018, 07:50:47 AM »
The LV buffer is 720x480..

I don't see anything that size on the EDMAC screen. I also checked on the original EOSM and it looks almost identical--no 720x480 sized buffer on the EDMAC list.

Oh wait--1440 is half of 720 so this must be it.



Searching the wiki I came across VRAM ADDR from code which looks very promising. I modified it so it should work with the EOSM2. The only change needed was to find SetBitmapVramAddress which was quite easy:

Code: [Select]
int i;
unsigned int *bmp_ram_addr = bmp_vram_info;
for (i=0; i<2; i++)
  DebugMsg( DM_MAGIC, 3, "bmp_vram[]=0x%08x, 0x%08x, 0x%08x",
    bmp_ram_addr[3*i+0],  bmp_ram_addr[3*i+1], bmp_ram_addr[3*i+2] );
unsigned int *vram_info_addr = vram_info;
for (i=0; i<3; i++)
  DebugMsg( DM_MAGIC, 3, "vram_info[]=0x%08x, w=%4d, h=%4d, p=%4d, n=%4d",
      vram_info_addr[5*i+0],  vram_info_addr[5*i+1],
      vram_info_addr[5*i+2], vram_info_addr[5*i+3], vram_info_addr[5*i+4] );
unsigned int *stateobj_disp = 0x90494+0x128; // see FF137ACC SetBitmapVramAddress
    DebugMsg( DM_MAGIC, 3, "stateobj_disp+0xb0[]=0x%08x,0x%08x,0x%08x,",
    stateobj_disp[0], stateobj_disp[1], stateobj_disp[2]);

The problem is, there's no instructions on where this code goes. At first I tried it in run_test() in debug.c but that didn't work. It seems that it should go here:

vram.c:
Code: [Select]
void vram_init()
{
    menu_add("VRAM", vram_menus, COUNT(vram_menus));
    old_buffer_pos = YUV422_LV_BUFFER_1;
}

// code goes in here

INIT_FUNC(__FILE__, vram_init);
#endif

However, I couldn't get it to display on the console or write to a log. Too bad, the output should provide some of the missing information. This example is from the 550D.109:

Code: [Select]
2109:  9760.005 [MAGIC] bmp_vram[]=0xc0f140d0, 0x00000000, 0x02087100
2110:  9760.045 [MAGIC] bmp_vram[]=0xc0f140d4, 0x00000000, 0x02087100
2111:  9760.082 [MAGIC] vram_info[]=0x40d07800, w= 720, h= 720, p= 480, n=   2
2112:  9760.116 [MAGIC] vram_info[]=0x4c233800, w= 720, h= 720, p= 480, n=   0
2113:  9760.149 [MAGIC] vram_info[]=0x4f11d800, w= 720, h= 720, p= 480, n=   0
2114:  9760.194 [MAGIC] stateobj_disp+0xb0[]=0x42080008,0x40d07800,0x40958c3c,

The more I look into this, the more it seems that this section is correct. In fact checking the EOSM2 side by side with an EOSM it looks like much of this YUV/VRAM stuff is identical on the two cameras.

Whatever is causing the instability issues is probably elsewhere. Checking Free Memory will reboot or even crash the camera when it gets to SRM job total. There are other obvious problems:



[EDIT] Had a screenshot of a lens issue here but I was able to resolve it so I removed the image. Found out that in EOSM2 uses the same prop_lv_lens structure as the 6D and 5D3.123. Didn't expect that, thought it would be the same as the EOSM.

Sorry EOSM2 enthusiasts, this needs some more work before releasing a test build.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #238 on: January 28, 2018, 07:45:26 PM »
I got lucky finding such a simple fix for the lens data issue. Sometimes this camera is more like the 6D than the original EOSM.

Today I thought I'd tackle the shutter issue hoping for a simple solution and that it also resolve some of the other problems--maybe something is using some memory that's messing things up? A while back I put some stuff on the back burner and now that ML is running on the camera it is time to revisit these items.

Code: [Select]
// Get back to this after figuring out how parse_bin.py works
// not used on 700D and 100D so maybe it can wait until adtg_gui is working on EOSM2
//#define FRAME_SHUTTER_BLANKING_ZOOM   (*(uint16_t*)0x40481B20) // ADTG register 805f
//#define FRAME_SHUTTER_BLANKING_NOZOOM (*(uint16_t*)0x40481B24) // ADTG register 8061
//#define FRAME_SHUTTER_BLANKING_READ   (lv_dispsize > 1 ? FRAME_SHUTTER_BLANKING_NOZOOM : FRAME_SHUTTER_BLANKING_ZOOM) /* when reading, use the other mode, as it contains the original value (not overriden) */
//~ #define FRAME_SHUTTER_BLANKING_WRITE  (lv_dispsize > 1 ? &FRAME_SHUTTER_BLANKING_ZOOM : &FRAME_SHUTTER_BLANKING_NOZOOM)
#define LV_DISP_MODE (MEM(0xDF1BC + 0x7C) != 3)

parse_bin.py lives in modules/adtg_log and there is very little documentation on it other than this from commit 8855c9a on Bitbucket:
Quote
adtg_log: python script to parse binary log file. "parse_bin.py -f TEST.BIN > out.txt"

Which .BIN file are we talking about? I suppose it is ROM1.BIN but when I ran parse_bin.py on it all I got was gibberish that didn't match anything that I saw in the ADTG and CMOS registers topic. Lots of talk about the adtg_log module, but I tried several times to compile that module. One small issue seems to be that it is missing this line:

Code: [Select]
#include <beep.h>
However, I'm still getting compile errors even after turning on these options:

Code: [Select]
CONFIG_GDB      = y
CONFIG_GDBSTUB  = y

I also compiled the trace module, looked up where *"[REG] @@@@@@@@@@@@ Start ADTG[CS:%lx:%lx]" is at in the disassembly, ffd60614, but couldn't figure out what to do with that. Ugh, another dead end.

So how about adtg_gui? It works!



Plugged in the numbers:

Code: [Select]
#define FRAME_SHUTTER_BLANKING_ZOOM   (*(uint16_t*)0x416D7B7C) // ADTG register 805f
#define FRAME_SHUTTER_BLANKING_NOZOOM (*(uint16_t*)0x416D7B80) // ADTG register 8061
/* when reading, use the other mode, as it contains the original value (not overriden) */
#define FRAME_SHUTTER_BLANKING_READ   (lv_dispsize > 1 ? FRAME_SHUTTER_BLANKING_NOZOOM : FRAME_SHUTTER_BLANKING_ZOOM)
#define FRAME_SHUTTER_BLANKING_WRITE  (lv_dispsize > 1 ? &FRAME_SHUTTER_BLANKING_ZOOM : &FRAME_SHUTTER_BLANKING_NOZOOM)
#define LV_DISP_MODE (MEM(0xDF1BC + 0x7C) != 3)

It works in still photo mode - Yay!



But in movie mode--the shutter angle looks right but the shutter speed is alternating between 0.7" and 1.7" which is obviously wrong.





The camera is still quite unstable but at least the issues are getting resolved one baby step at a time.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

vagaboundedu

  • New to the forum
  • *
  • Posts: 10
Re: ML on EOS-M2
« Reply #239 on: January 29, 2018, 10:31:24 PM »
Sorry EOSM2 enthusiasts, this needs some more work before releasing a test build.

Hey, we're just thrilled to see the progress. Very cool!

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #240 on: January 30, 2018, 02:27:11 AM »
I've been running the Stubs API test just like Danne is doing on the 1100D. Both the EOSM2 and EOSM are failing on the same tests as the 1100D. Of course that needs to be worked on but I wanted to see which tests are failing and if there was something that might be easier to start with so I tried running one test at a time.

These are the tests that are failing:

Code: [Select]
//        stub_test_malloc_n_allocmem();      stub_test_save_log(); // fails and doesn't save log
//        stub_test_exmem();                  stub_test_save_log(); // fails and doesn't save log

Something that I thought would be easier to debug was the failures in the stub_test_gui:

EOSM2
Code: [Select]
[Pass] is_play_mode() => 0x1
       SetGUIRequestMode(1); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x1
       SetGUIRequestMode(2); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x2
       SetGUIRequestMode(0); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x0
[Pass] display_idle() => 0x1
       GUI_Control(BGMT_PLAY, 0, 0, 0); msleep(1000);
[Pass] PLAY_MODE => 0x1
[Pass] MENU_MODE => 0x0
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(1000);
[FAIL] MENU_MODE => 0x0
[FAIL] PLAY_MODE => 0x1
[Pass] dialog->type => 'DIALOG'
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(500);
[Pass] MENU_MODE => 0x0
[FAIL] PLAY_MODE => 0x1
       SW1(1,100)
[Pass] HALFSHUTTER_PRESSED => 0x1
       SW1(0,100)
[Pass] HALFSHUTTER_PRESSED => 0x0
[Pass] is_play_mode() => 0x1
[FAIL] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x1
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[FAIL] is_pure_play_movie_mode() => 0x1

The EOSM is also failing on some of these:
Code: [Select]
[Pass] is_play_mode() => 0x1
       SetGUIRequestMode(1); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x1
       SetGUIRequestMode(2); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x2
       SetGUIRequestMode(0); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x0
[Pass] display_idle() => 0x1
       GUI_Control(BGMT_PLAY, 0, 0, 0); msleep(1000);
[Pass] PLAY_MODE => 0x1
[Pass] MENU_MODE => 0x0
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(1000);
[Pass] MENU_MODE => 0x1
[Pass] PLAY_MODE => 0x0
[Pass] dialog->type => 'DIALOG'
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(500);
[Pass] MENU_MODE => 0x0
[Pass] PLAY_MODE => 0x0
       SW1(1,100)
[Pass] HALFSHUTTER_PRESSED => 0x1
       SW1(0,100)
[Pass] HALFSHUTTER_PRESSED => 0x0
[Pass] is_play_mode() => 0x1
[FAIL] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x1
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[FAIL] is_pure_play_movie_mode() => 0x1

The 700D passes all of the tests in camera but when I tried the stub_test_gui in QEMU it had similar failures:
Code: [Select]
[Pass] is_play_mode() => 0x1
       SetGUIRequestMode(1); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x1
       SetGUIRequestMode(2); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x2
       SetGUIRequestMode(0); msleep(1000);
[Pass] CURRENT_GUI_MODE => 0x0
[Pass] display_idle() => 0x1
       GUI_Control(BGMT_PLAY, 0, 0, 0); msleep(1000);
[FAIL] PLAY_MODE => 0x0
[Pass] MENU_MODE => 0x0
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(1000);
[FAIL] MENU_MODE => 0x0
[Pass] PLAY_MODE => 0x0
[FAIL] dialog->type => ''
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(500);
[Pass] MENU_MODE => 0x0
[Pass] PLAY_MODE => 0x0
       SW1(1,100)
[FAIL] HALFSHUTTER_PRESSED => 0x0
       SW1(0,100)
[Pass] HALFSHUTTER_PRESSED => 0x0
[Pass] is_play_mode() => 0x1
[FAIL] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[FAIL] is_pure_play_movie_mode() => 0x0
[Pass] is_play_mode() => 0x1
[Pass] is_pure_play_photo_mode() => 0x0
[Pass] is_pure_play_movie_mode() => 0x0

I checked gui.h on various cameras it looks like most cameras use pretty much the same addresses. I thought that maybe something was different because the EOSM2 (top black camera) has an extra photo mode setting that the EOSM (bottom red camera) doesn't have.



However, running the extract_button_codes.py script in QEMU it turns out that both cameras have identical codes.

So is this anything to be concerned about at this point? Seems like I have bigger fish to fry.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #241 on: January 30, 2018, 05:06:50 AM »
Looks like I was a bit impatient - stub_test_malloc_n_allocmem does save a log file but it takes a very long time. The fail is just like the EOSM. I thought that maybe this commit for the 1100D would help but it didn't.

Here's where things go bad on the EOSM2--same place as the EOSM:

Code: [Select]
...
[Pass] ABS(MOD(t1-t0, 1048576)/1000 - 250) => 0x4
       LoadCalendarFromRTC( &now )
       s0 = now.tm_sec => 0x26
       Date/time: 2018/01/29 08:36:38
       msleep(1500)
       LoadCalendarFromRTC( &now )
       s1 = now.tm_sec => 0x28
[Pass] MOD(s1-s0, 60) => 0x2
[Pass] MOD(s1-s0, 60) => 0x2
       m0 = MALLOC_FREE_MEMORY => 0x516c0
[Pass] p = (void*)_malloc(50*1024) => 0x12ecf8
[Pass] CACHEABLE(p) => 0x12ecf8
       m1 = MALLOC_FREE_MEMORY => 0x44eb0
       _free(p)
       m2 = MALLOC_FREE_MEMORY => 0x516c0
[Pass] ABS((m0-m1) - 50*1024) => 0x10
[Pass] ABS(m0-m2) => 0x0
       m0 = GetFreeMemForAllocateMemory() => 0x200014
[Pass] p = (void*)_AllocateMemory(256*1024) => 0xaa72d8
[Pass] CACHEABLE(p) => 0xaa72d8
       m1 = GetFreeMemForAllocateMemory() => 0x1c0008
       _FreeMemory(p)
       m2 = GetFreeMemForAllocateMemory() => 0x200014
[Pass] ABS((m0-m1) - 256*1024) => 0xc
[Pass] ABS(m0-m2) => 0x0
       m01 = MALLOC_FREE_MEMORY => 0x516c0
       m02 = GetFreeMemForAllocateMemory() => 0x200014
[Pass] p = (void*)_alloc_dma_memory(256*1024) => 0x40aa7318
[Pass] UNCACHEABLE(p) => 0x40aa7318
[Pass] CACHEABLE(p) => 0xaa7318
[Pass] UNCACHEABLE(CACHEABLE(p)) => 0x40aa7318
       _free_dma_memory(p)
[FAIL] p = (void*)_shoot_malloc(24*1024*1024) => 0x0
[FAIL] UNCACHEABLE(p) => 0x40000000
       _shoot_free(p)
       m11 = MALLOC_FREE_MEMORY => 0x516a0
       m12 = GetFreeMemForAllocateMemory() => 0x1fffbc
[Pass] ABS(m01-m11) => 0x20
[Pass] ABS(m02-m12) => 0x58
[FAIL] p = (void*)_shoot_malloc(24*1024*1024) => 0x0
[FAIL] UNCACHEABLE(p) => 0x40000000
[FAIL] p = (void*)_shoot_malloc(24*1024*1024) => 0x0
[FAIL] UNCACHEABLE(p) => 0x40000000
...
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: ML on EOS-M2
« Reply #242 on: January 30, 2018, 01:53:02 PM »
In QEMU, the Free Memory dialog crashes at srm_malloc_suite - wrong stub?

Code: [Select]
shoot buffer: 4dd16064 ... 4f3ce063
shoot buffer: 4ae00064 ... 4aee8063
shoot buffer: 45f240e4 ... 45ffc0e3
shoot buffer: 44000064 ... 45e88063
srm_malloc_suite(0)...
[MPU] Received: 06 05 04 01 01 00  (PROP_ICU_UILOCK - spell #56)
[MPU] Sending : 06 05 04 01 01 00  (PROP_ICU_UILOCK)
UILock: 00000000 -> 41000001 => 41000001
< Error Exception>
TYPE        : 4
ISR         : 0
TASK IDSR   : 5111821
TASK Name   : RscMgr

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #243 on: January 30, 2018, 07:41:07 PM »
How in the world are you running this in QEMU? I can't get to the ML menus at all on the EOSM2, EOSM or 1100D for that matter. Are you getting the same error on the EOSM? I'm getting the same errors on both EOSM and EOSM2. I don't think that I've been ever able to successfully run the Stubs API test from the selftest module on the EOSM so what are the chances of it working on the EOSM2?

Checked ptpPropSetUILock and all of the Memory allocation and ExMem stubs once again and they seem to check out fine. Maybe there's something quirky in one of the stubs? I remember a while back finding something on the 700D that didn't match what was being used on other cameras. Searched for it but couldn't find the discussion. Was it "AbortEDmac" ?
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: ML on EOS-M2
« Reply #244 on: January 30, 2018, 07:50:04 PM »
On 1100D it works out of the box, but uses a different button - look at this picture.

On M2, patch handle_ml_menu_erase so it doesn't check for GUISTATE_IDLE, and define BGMT_TRASH as 0xD (probably a button not physically present on the camera). That was the easy coding task mentioned earlier.

What's interesting is that BGMT_TRASH is defined as 04 01 on the MPU side (the raw button code that comes from the MPU) on every single camera model that uses a dedicated button for this function (and most other buttons also have consistent codes). Also, since EOS firmwares appear to be derived from a common codebase, they happen to interpret other button codes correctly as well.

Checked on EOSM - in the Free Memory dialog, shoot_total is only 23MB. Do you get different value on the camera? Even 1100D has 39MB total; the test tries to allocate 35MB (it used to ask for 64MB before the 1100D changes).

On M2, I believe SRM_AllocateMemoryResourceFor1stJob is at FF0E9660 (unless I'm looking at the wrong firmware version :D )

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #245 on: January 30, 2018, 09:11:55 PM »
Checked on EOSM - in the Free Memory dialog, shoot_total is only 23MB. Do you get different value on the camera?

That's what I get on the EOSM with the unified build.

EOSM


On M2, I believe SRM_AllocateMemoryResourceFor1stJob is at FF0E9660 (unless I'm looking at the wrong firmware version :D )

Must have gone over that stub 10 times but now that you point it out... Couldn't get to the Free Memory before that change. Looks pretty close to the original:

EOSM2


So is this why the M1 and M2 keep failing the Stubs API test?

On M2, patch handle_ml_menu_erase so it doesn't check for GUISTATE_IDLE, and define BGMT_TRASH as 0xD (probably a button not physically present on the camera). That was the easy coding task mentioned earlier.

You make juggling chainsaws look easy. Ok, you dropped several hints but I still don't quite get how you did it. So it doesn't go into the abyss when you get into LiveView in QEMU? I can never get back to the Canon menu after closing it with the "M" key in QEMU.

Speaking of buttons, so it was like pressing the button for wrong floor on the elevator all this time with the 1100D. BTW, shouldn't commit 4ea94f5 from unified be applied to the allocate-raw-lv-buffer branch?
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: ML on EOS-M2
« Reply #246 on: January 30, 2018, 09:25:12 PM »
Yeah, that's what I've got in QEMU as well.

Quote
So it doesn't go into the abyss when you get into LiveView in QEMU?

Of course it does - that's the entire point of this patch, to allow you to open ML menu before entering LiveView (for example, while you are on Canon menu). Will commit it, as I don't expect LiveView emulation anytime soon.

platerytter

  • New to the forum
  • *
  • Posts: 3
Re: ML on EOS-M2
« Reply #247 on: February 02, 2018, 10:15:59 AM »
Late to the party bus but just wanted to chime in with congrats on both birthday and reaching this milestone!

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: ML on EOS-M2
« Reply #248 on: February 02, 2018, 11:47:52 AM »
Finally got the exmem tests working hopefully on all models (tested 1100D, EOSM, EOSM2, 5D4, 6D, 500D, 550D, 700D, all in QEMU). Had to change them a bit to account for the significant variations in available memory. Committed the menu hack, too.

Incredible how stuff working well on some models fails miserably on others. Then, after fixing it for these others, it fails on the models where it used to work. Rinse, repeat.

Still got stuff to fix, for example the memory benchmarks aren't starting on all models (problem in memory backend - it doesn't know how much free space is in the shoot memory buffer before attempting to allocate from it). Many models are able to allocate 2x16MB from there, but not all.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: ML on EOS-M2
« Reply #249 on: February 05, 2018, 07:19:40 AM »
Just wanted to document where things are at with the EOSM2.

I wanted to surprise everyone by posting some EOSM2 screenshots on the 12-bit (and 10-bit) RAW video development discussion as per Reply #1579. At first I tried merging the EOSM2 a bit at a time into the allocate-raw-lv-buffer branch just to see what it would be like to cherry pick from the wip branch but ran into some problems and the camera wouldn't start up so I ended up merging the branches and it worked. Sort of. The important part of the test is switching video modes and it obviously isn't working yet. Maybe it needs a different edmac channel or something like that?



There's definitely something fishy going on because when leaving the ML menu sometimes LiveView is stuck on a still frame and/or the ML overlays won't come back. There is also some noise every once in a while on the ML menus. This is something I've seen before on a very early 100D build. What I haven't seen before is the shutter speed flicking to different values in movie mode. This is something I pointed out before.



[EDIT] This is probably related - the FPS override flips between a couple of values even though it isn't active:





I think that I should resolve these issues before moving on. I was able to do this which might help:

You can include dm-spy-experiments in your current wip branch and use it for reverse engineering (understanding how things work). This branch breaks things on some cameras...so let's not include this one in the PR yet.


So I currently have three branches, allocate-raw-lv-buffer, dm-spy-experiments and of course the main work in progress branch. I think the issues noted above should be fixed before moving on to the pull request branch. Paraphrasing the instructions for the PR build:

new-dryos-task-hooks..., qemu, lua_fix... I think it's best to start a clean branch starting from unified, with the first 3 branches merged.

You could also consider including 100D, which is very similar to EOSM2 and also waits for the first 3 backends to reach the mainline (otherwise it's pretty much ready)...

I'm kind of not sure what to tweak in order to get the camera to do something useful. I mean ML is running on the camera but I can't take any pictures or record video with it. I could post a build for testers--call it the tits on a bull build? Just kidding of course but I could use a hint or two.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102