Author Topic: ML on EOS-M2  (Read 45892 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10545
  • 5D Mark Free
Re: ML on EOS-M2
« Reply #50 on: June 03, 2017, 08:44:44 AM »
I'm not sure how you came up with 0xFFDD60CB from 0xFFDD60C8 to make it conform to a 32-bit operation. I tried looking it up but don't quite get it.

A 32-bit read from 0xFFDD60C8 actually means 4 bytes from that address: 0xFFDD60C8, 0xFFDD60C9, 0xFFDD60CA, 0xFFDD60CB.

Quote
The size of the copied block is 0xB709F but it needs to be rounded up to be evenly divisible by 32 (0x20) right? So if I followed the lesson plan properly it is also 0xB70A0 just like the 1.0.2 firmware.

Nothing is rounded up; this is the exact size. Copying 4 bytes at a time cannot give you an odd number of bytes ;)

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #51 on: June 05, 2017, 03:45:14 PM »
Alright, so let's start from where we've left off.

I'm going through this slowly and trying to make sure my results are matching up. Some things aren't quite the same which is to be expected because I'm using a different firmware version (1.0.3) and another reason may be because I'm QEMU on a Mac which might not be set up quite right just yet.

For the debugmsg.gdb file, we'll need some basic stubs: DebugMsg and task_create (easy to find - one is used for most debug messages, the other can be easily identified from things that look like task names) and two addresses: CURRENT_TASK and CURRENT_ISR (interrupt service routine). We can find the latter in the interrupt handler - quote from debugmsg.gdb:
If you can find that by examining disassembled code, great. If not, QEMU to the rescue:
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d io,int,nochain -singlestep
...
Taking exception 5 [IRQ]
[INT]      at 0x00000510:0000050C [0xC0201004] -> 0x28      : Requested int reason 28 (INT 0Ah)
[INT]      at 0x00000594:00000568 [0xC0201010] <- 0xA       : Enabled interrupt 0Ah

(note: -d nochain -singlestep gives more precise locations in I/O logs - otherwise, QEMU groups the ARM instructions in blocks for faster execution, so the reported location will be approximate)

First of all I went through the disassembly and as best as I can determine the stubs that we're using haven't changed between the firmware versions. Great! However, I am seeing different results.
Code: [Select]
(gdb) disas 0x4B0,0x6B0[EOS] trigger int 0x0A (delayed)
Taking exception 5 [IRQ]
[INT]      at 0x00000510:0000050C [0xC0201004] -> 0x28      : Requested int reason 28 (INT 0Ah)
[TIMER]    at 0xFF350400:000072A4 [0xC0242014] -> 0x6E900   : DIGIC clock
[INT]      at 0x00000594:FF0C1578 [0xC0201010] <- 0xA       : Enabled interrupt 0Ah

The clock is running continuously so I suppose that can be disregarded but the difference between 0x00000568 and 0xFF0C1578 isn't exactly "approximate."

OK, so the register we are looking for is read at 0x510. Let's disassemble with GDB:
Code: [Select]
(gdb) disas 0x4B0,0x6B0
...
   0x0000050c: ldr r0, [pc, #368] ; 0x684
   0x00000510: ldr r4, [r0]
   0x00000514: str r4, [pc, #304] ; 0x64c
...
(gdb) x 0x684
0x684: 0xc0201004
(gdb) x 0x64c
0x64c: 0x00000028
(gdb) print 0x28>>2
$1 = 0xa

Hit a wall here because I'm not seeing anything at all like that.

I've been updating the EOSM2 preliminary setup pull request as I work my way through this.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10545
  • 5D Mark Free
Re: ML on EOS-M2
« Reply #52 on: June 05, 2017, 05:06:05 PM »
You need to type the gdb commands at the gdb prompt. You'll need to start qemu like this:
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb

Of course, you can mix gdb with other options freely. In the above invocation, gdb will be the foreground process (the one you interact with on the console); if you swap their order, QEMU will be the foreground process (which doesn't do anything interesting by default, unless you redirect its monitor to console, with -monitor stdio).

The DIGIC clock line appeared after me fine-tuning the verbosity (so it's OK). Interrupts are not deterministic (since they use the host's wall clock as a reference, so the results depend a lot on the emulation speed, and you will not get the same code sequence on two different runs. The same happens on real hardware (run the startup-log twice and compare the logs).

Besides, the execution trace within a timer interrupt also depend on whatever DryOS decides to do in that interrupt (such as switching tasks or running some timer event). To see this particular case more clearly, try:
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d io,int,nochain -singlestep |& grep "Enabled interrupt 0Ah"

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #53 on: June 07, 2017, 08:54:50 PM »
Hope I'm not wearing out my QEMU welcome -- still stuck here:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S & arm-none-eabi-gdb -x debug-logging.gdb
(note: we will need to create a GDB file for EOS M2, named EOSM2/debugmsg.gdb to match other models, but since we don't have one yet, we can just use the generic version - which we are going to include in the platform-specific file later)

Let it run for a while (continue), then hit CTRL-C and examine the exception vector (located at address 0):

Code: [Select]
(gdb) disas 0,0x40
Dump of assembler code from 0x0 to 0x40:
   0x00000000: nop ; (mov r0, r0)
   0x00000004: ldr pc, [pc, #20] ; 0x20
   0x00000008: ldr pc, [pc, #20] ; 0x24
...

The line in red is what is tripping me up, specifically the (continue). How am I supposed to do that?

Code: [Select]
[EOS] loading './EOSM2/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './EOSM2/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[EOS] loading './EOSM2/SFDATA.BIN' as serial flash, size=0x1000000
[MPU] FIXME: no MPU spells for EOSM2.
[MPU] FIXME: no MPU button codes for EOSM2.
Setting BOOTDISK flag to 0
continue
-bash: continue: only meaningful in a `for', `while', or `until' loop

So that's not right. QEMU on Mac has a "Machine" menu item where you can "Pause" and "Resume" and that shows some more action in the terminal but CTRL-C always gets me back to the bash prompt--shouldn't I be getting the (gdb) prompt? Launching gdb and continuing your instructions always results in:

Code: [Select]
(gdb) disas 0,0x40
Dump of assembler code from 0x0 to 0x40:
   0x00000000: Cannot access memory at address 0x0

This might be a Macintosh issue but more likely a problem with a clueless user.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10545
  • 5D Mark Free
Re: ML on EOS-M2
« Reply #54 on: June 07, 2017, 09:05:23 PM »
After typing that long command, I get the (gdb) prompt. Type "continue" there.

Selecting "Pause" from QEMU menu also gets me to the (gdb) prompt.

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #55 on: June 08, 2017, 12:57:32 AM »
I don't get that at all, that long command launches QEMU. I do see an error message flash by but terminal isn't show the full history. I redirected the error messages like this
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S & arm-none-eabi-gdb -x debug-logging.gdb 2> error.txt
and found this:
Code: [Select]
-bash: arm-none-eabi-gdb: command not found
Seems like I've got a configuration issue. It appears to be in the right place.
Code: [Select]
/Users/rosiefort/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gdb
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #56 on: June 08, 2017, 08:29:47 AM »
If I make that long command even longer, it works:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S & ~/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gdb -x debug-logging.gdb
Is there supposed to be a symlink in ~/qemu for arm-none-eabi-gdb?
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10545
  • 5D Mark Free
Re: ML on EOS-M2
« Reply #57 on: June 08, 2017, 10:30:49 AM »

vagaboundedu

  • New to the forum
  • *
  • Posts: 6
Re: ML on EOS-M2
« Reply #58 on: June 14, 2017, 02:18:37 AM »
Just wanted you all to know that I created an account on this site specifically to say I'm really looking forward to being able to install ML on my M2! Can't wait! Thank you so much for all your work.

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #59 on: June 18, 2017, 11:49:29 PM »
Thought I'd make a little noise on this topic just to let people know that I'm still working my way through this albeit at a very slow pace.

QEMU is under active development so before posting anything I pull in the latest commits and merge my EOSM2 and Mac specific build tweaks.

Ok, so here's where I left off:

Alright, so let's start from where we've left off.
...
This 0x648 looks like a counter that tells how many nested interrupts we are handling (yes, they can be nested - unfortunately, as this makes it a lot harder to understand, debug, emulate and so on). Let's confirm its functionality with RAM tracing:
Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d io,int,ram,nochain -singlestep
...
[tcm_code] at 0x000004D4:19980218 [0x00000648] -> 0x0       
[tcm_code] at 0x000004E0:19980218 [0x00000648] <- 0x1       : was 0x0;
...
[tcm_code] at 0x0000050C:0000050C [0x00000684] -> 0xC0201004
[INT]      at 0x00000510:0000050C [0xC0201004] -> 0x28      : Requested int reason 28 (INT 0Ah)
[tcm_code] at 0x00000514:0000050C [0x0000064C] <- 0x28      : was 0x0;
...
[INT]      at 0x00000594:00000568 [0xC0201010] <- 0xA       : Enabled interrupt 0Ah
[tcm_code] at 0x00000598:00000568 [0x00000648] -> 0x1       
[tcm_code] at 0x000005A0:00000568 [0x00000648] <- 0x0       : was 0x1;

Looks right!

I'm seeing pretty much the same only with some additional information probably as a result of some of the enhancements a1ex has been adding to QEMU.

Code: [Select]
...
[tcm_code] at TaskM:000004D4:19980218 [0x00000648] -> 0x0       
[tcm_code] at TaskM:000004E0:19980218 [0x00000648] <- 0x1       : was 0x0;
...
[tcm_code]  at init:0000050C:0000050C [0x00000684] -> 0xC0201004
[INT]       at init:00000510:0000050C [0xC0201004] -> 0x28      : Requested int reason 28 (INT 0Ah)
[tcm_code]  at init:00000514:0000050C [0x0000064C] <- 0x28      : was 0x0;
...
[INT]       at init:00000594:00000568 [0xC0201010] <- 0xA       : Enabled interrupt 0Ah
[tcm_code]  at init:00000598:00000568 [0x00000648] -> 0x1       
[tcm_code]  at init:000005A0:00000568 [0x00000648] <- 0x0       : was 0x1;

In some cases I'm getting errors that shouldn't be happening. For example:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d callstack,ramw |& grep -B 20 "not restored"
-bash: syntax error near unexpected token `&'

The '|&' shorthand was added to bash 4 as an abbreviation for '2>&1 |' and I'm running this version of bash:

Code: [Select]
GNU bash, version 4.4.12(1)-release (x86_64-apple-darwin16.3.0)
Go figure -- must be a Mac thing. Changing '|&' to '2>&1 |' still didn't give me the same results a1ex posted.

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d callstack,ramw 2>&1 | grep -B 20 "not restored"
[tcm_data] at 0x0010DA10:00000040 [0x40000F60] <- 0x4       : was 0x4C;
[tcm_data] at 0x0010DA10:00000040 [0x40000F64] <- 0x20      : was 0x4C;
[tcm_data] at 0x0010DA10:00000040 [0x40000F68] <- 0x8       : was 0x4C;
[tcm_data] at 0x0010DA10:00000040 [0x40000F6C] <- 0x40      : was 0x1F;
[tcm_data] at 0x0010DA08:00000100 [0x40000F70] <- 0x10      : was 0x4C;
[tcm_data] at 0x0010DA08:00000100 [0x40000F74] <- 0x80      : was 0x4C;
[tcm_data] at 0x0010DA08:00000100 [0x40000F78] <- 0x20      : was 0x40;
[tcm_data] at 0x0010DA08:00000100 [0x40000F7C] <- 0x100     : was 0x40;
[tcm_data] at 0x0010DA10:00000C00 [0x40000F80] <- 0x40      : was 0x1F;
[tcm_data] at 0x0010DA10:00000C00 [0x40000F84] <- 0x200     : was 0x1;
[tcm_data] at 0x0010DA10:00000C00 [0x40000F88] <- 0x80      : was 0x0;
[tcm_data] at 0x0010DA10:00000C00 [0x40000F8C] <- 0xC00     : was 0x0;
[tcm_data] at 0x0010DA2C:00000C00 [0x40000F90] <- 0xFF      : was 0x7;
[tcm_data] at 0x0010DA2C:00000C00 [0x40000F94] <- 0xFFF     : was 0x1;
[tcm_data] at 0x0010329C:001000C8 [0x40000F9C] <- 0x1       : was 0x0;
[tcm_data] at 0x0010329C:001000C8 [0x40000FA0] <- 0x1000C8  : was 0x10007C;
[tcm_data] at 0x0010329C:001000F8 [0x40000F9C] <- 0x1       : was 0x1;
[tcm_data] at 0x0010329C:001000F8 [0x40000FA0] <- 0x1000F8  : was 0x1000C8;
[tcm_data] at 0x001022C0:0010013C [0x40000F9C] <- 0x1       : was 0x1;
[tcm_data] at 0x001022C0:0010013C [0x40000FA0] <- 0x10013C  : was 0x1000F8;
Warning: R10 not restored (0xa -> 0x1)                                           at [ffff101c:ffff09cc]

Moving on or I'll never get through this. Here we can finally see some indication of the different firmware versions that we're looking at, a1ex on EOSM2 1.0.2 while I'm on 1.0.3.


Code: [Select]
./run_canon_fw.sh EOSM2 -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb
...
[      init:ff352264 ] task_create(PowerMgr, prio=20, stack=400, entry=ff352088, arg=0)
[      init:ff1470d4 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[      init:0003671c ] task_create(DbgMgr, prio=1f, stack=0, entry=36628, arg=46e584)
[      init:ff0c3334 ] (8b:16)
K355 ICU Firmware Version 1.0.2 ( 6.0.5 )
[      init:ff0c3348 ] (8b:05)
ICU Release DateTime 2013.12.02 09:28:54
[      init:ff0f5ea0 ] (00:03) [SEQ] CreateSequencer (Startup, Num = 6)
[      init:ff0f5f28 ] task_create(Startup, prio=19, stack=2800, entry=ff0f5d6c, arg=46e880)
[      init:ff0f60f4 ] (00:02) [SEQ] NotifyComplete (Startup, Flag = 0x10000)
[      init:ff0f6158 ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x10000, Flag = 0x10000)
[      init:ff0c33d4 ] task_create(TaskMain, prio=1d, stack=0, entry=ff0c28d8, arg=0)
[   Startup:ff0f5db4 ] (00:05) [SEQ] seqEventDispatch (Startup, 0)
[   Startup:ff0c373c ] (8b:05) startupEntry
[   Startup:ff0c375c ] task_create(Startup2, prio=11, stack=400, entry=ff0c3604, arg=0)
[  Startup2:ff1310c4 ] (02:16) PROPAD_CreateFROMPropertyHandle DRAMAddr 0x416d5b00
[  Startup2:ff14a204 ] (00:01) [SF] IsAddressSerialFlash 0x2d0000
...

And my results running that same command.

Code: [Select]
./run_canon_fw.sh EOSM2 -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb
...
[      init:ff352260 ] task_create(PowerMgr, prio=20, stack=400, entry=ff352084, arg=0)
[      init:ff1470d0 ] (00:01) [PM] DisablePowerSave (Counter = 1)
[      init:0003671c ] task_create(DbgMgr, prio=1f, stack=0, entry=36628, arg=46e584)
[      init:ff0c3334 ] (8b:16)
K355 ICU Firmware Version 1.0.3 ( 6.0.6 )
[      init:ff0c3348 ] (8b:05)
ICU Release DateTime 2016.08.09 13:28:42
[      init:ff0f5e9c ] (00:03) [SEQ] CreateSequencer (Startup, Num = 6)
[      init:ff0f5f24 ] task_create(Startup, prio=19, stack=2800, entry=ff0f5d68, arg=46e880)
[      init:ff0f60f0 ] (00:02) [SEQ] NotifyComplete (Startup, Flag = 0x10000)
[      init:ff0f6154 ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x10000, Flag = 0x10000)
[      init:ff0c33d4 ] task_create(TaskMain, prio=1d, stack=0, entry=ff0c28d8, arg=0)
[   Startup:ff0f5db0 ] (00:05) [SEQ] seqEventDispatch (Startup, 0)
[   Startup:ff0c373c ] (8b:05) startupEntry
[   Startup:ff0c375c ] task_create(Startup2, prio=11, stack=400, entry=ff0c3604, arg=0)
[  Startup2:ff1310c0 ] (02:16) PROPAD_CreateFROMPropertyHandle DRAMAddr 0x416d5b00
[  Startup2:ff14a200 ] (00:01) [SF] IsAddressSerialFlash 0x2d0000
...

Crawling along to a1ex's next post I ran into the '|&' issue again.

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d romcpy |& grep ROMCPY
[ROMCPY] 0xFFFF0000 -> 0x0        size 0x40       at 0xFFFF0980
[ROMCPY] 0xFFFE0000 -> 0x100000   size 0xFF2C     at 0xFFFF0FCC
[ROMCPY] 0xFFD1F02C -> 0x1900     size 0xB70A0    at 0xFF0C000C
[ROMCPY] 0xFF0C0E04 -> 0x4B0      size 0x1E8      at 0xFF0C0D70
[ROMCPY] 0xFFA12904 -> 0x4E0E98   size 0xC7C      at 0x8645C   

Only this time substituting '2>&1 |' worked, sort of. Seems it ended a little earlier.

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d romcpy 2>&1 | grep ROMCPY
[ROMCPY] 0xFFFF0000 -> 0x0        size 0x40       at 0xFFFF0980
[ROMCPY] 0xFFFE0000 -> 0x100000   size 0xFF2C     at 0xFFFF0FCC
[ROMCPY] 0xFFD1F0E4 -> 0x1900     size 0xB70A0    at 0xFF0C000C
[ROMCPY] 0xFF0C0E04 -> 0x4B0      size 0x1E8      at 0xFF0C0D70
Binary file (standard input) matches
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10545
  • 5D Mark Free
Re: ML on EOS-M2
« Reply #60 on: June 19, 2017, 12:10:34 AM »
Quote
Changing '|&' to '2>&1 |' still didn't give me the same results a1ex posted.

These warnings should only appear if .current_task_addr is not defined. There are lots of them, on all models.

Quote
Binary file (standard input) matches

Maybe grep gets confused by ANSI color codes. Adding ansi2txt to the pipeline, or using grep --text, may help.

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #61 on: June 19, 2017, 02:05:14 AM »
As I suspected the bash issue was a Mac thing.

Code: [Select]
echo $BASH_VERSION
3.2.57(1)-release

Hold on there, what happened to bash 4 that we installed via Homebrew a while back? That works with the shell scrips but the default shell was still the old BSD version. To make GNU bash the default needs an adjustment of the terminal preferences.



The default login was still going to /bin/bash but the new shell is in /usr/local/bin. I've got "Shell opens with: Default login shell" picked and it is now going to the new bash.

Code: [Select]
echo $BASH_VERSION
4.4.12(1)-release

And of course the '|&' shortcut is now working.

I also got the full gcc-arm-none-eabi toolset working by doing a:

Code: [Select]
ln -s ~/gcc-arm-none-eabi-4_8-2013q4/bin/* /usr/local/bin/
Lots of gotchas running QEMU on the Mac. Someday I should put them all together into a tutorial.


5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

vagaboundedu

  • New to the forum
  • *
  • Posts: 6
Re: ML on EOS-M2
« Reply #62 on: June 19, 2017, 03:00:19 AM »
Been checking back everyday and was excited to see the updates today. Thanks so much for your continued work on this!

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #63 on: June 19, 2017, 06:52:38 AM »
I was getting excited because I was on the post where QEMU was showing the camera's UI but I got nothing yet -- probably because I neglected to understand this part:

Anyway, let's continue figuring out the emulation.
Code: [Select]
        .serial_flash_size      = 0x1000000,Oops, I was wrong here.

I wasn't really sure what to do here. We are using the SFDATA.BIN from a 100D and added ".serial_flash_size      = 0x1000000," to the new EOSM2 section in "qemu/qemu-2.5.0/hw/eos/model_list.c" -- right? So maybe this isn't right and I need to change the size to something else? Do I also need to somehow "fix" the SFDATA.BIN?

Taking a hint I did a QEMU run with "-d sflash" like this:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -d sflash
...
K355 READY
128K Sector FROM From BL 0xffff
[SF] InstallSerialFlash 6 0xc022c0d4 0x0 0x1000000 1

[SF] Bufcon Base 0xc022c0d4
[EEPROM] CS = 1
SerialFlash Initialize
[EEPROM] CS = 0
[EEPROM] Verbose: Got READ (03h)
[EEPROM] Verbose: address is now: 0x2D0000
...
     0:     3.072 [STARTUP]
K355 ICU Firmware Version 1.0.3 ( 6.0.6 )
     4:     3.328 [PROPAD] PROPAD_CreateFROMPropertyHandle DRAMAddr 0x416d5b00
     5:     3.840 [PROPAD] SerialFlash Packages!! 0x2d0000
[SFIO] sfio_trigger_int_DMA
[SFIO] eos_handle_sfio (copying now)
[EEPROM-DMA]! [0x2D0000] -> [0x416D5B00] (0x1DC400 bytes)
...
     6:     9.216 [PROPAD] SerialFlash Packages!! 0x7
     7:     9.216 [PROPAD] PROPAD_CreateFROMPropertyHandle DRAMAddr 0x41435b00
     8:     9.728 [PROPAD] SerialFlash Packages!! 0x10000
[SFIO] sfio_trigger_int_DMA
[SFIO] eos_handle_sfio (copying now)

At first I skipped over this and continued to set up QEMU for the EOSM2 but when it didn't work out as I hoped I returned to the part I didn't quite understand. I also committed the changes I made to my QEMU EOSM2 pull request even though it isn't working yet. Maybe someone could check if I'm on the right track?  ;D

I also committed some more changes to the build tweaks I've been doing to get QEMU on Mac running a little smoother.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10545
  • 5D Mark Free
Re: ML on EOS-M2
« Reply #64 on: June 19, 2017, 09:43:34 AM »
The serial flash activity looks alright. The issue is at MPU messages.

Simply adding files to the repository will not change anything - they must be included somehow in the build process. For example, if it's a .c file, generally you'll need to include the corresponding object file into the executable (example). If it's a .h file, you'll need to #include it somewhere. And once you have done that, you usually have to reference the new stuff somehow (call some functions or use some data structures from the new files), unless these files are used to replace or override some older definitions. That's generally how it works in a C program.

QEMU does something interesting here - any function or variable must be used somehow (unless it's a global API - in this case it requires a prototype - or an inline function), otherwise you get a compile error. So, if you define button_codes_EOSM2[], but you don't reference it from anywhere, you should get this:
Code: [Select]
error: ‘button_codes_EOSM2’ defined but not used [-Werror=unused-variable]
 static int button_codes_EOSM2[] = {
            ^

In this case, you don't even have to create new files or new definitions. Simply look in mpu.c to see how it's done on other models.

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #65 on: June 19, 2017, 04:17:45 PM »
Simply look in mpu.c to see how it's done on other models.

I actually did that but forgot to commit mpu.c. Fixed that on the pull request. I'm juggling three branches, the Mac specific build tweaks, the EOSM2 setup and a working branch where these two branches are merged. I'm also merging in changes from the main qemu branch to keep everything up to date. When I want to save a change I made in the working branch I commit the change in either build tweaks or EOSM2 so the pull requests get updated then merge back into the working branch, rinse and repeat.

Whatever problem I'm experiencing probably isn't because I'm on a Mac because QEMU seems to be running properly. I saved the terminal outputs for a fresh QEMU build and run in PASTEBIN in order to take a closer look at the warnings and error messages.

install.sh
configure_eos.sh
make -j8
run_canon_fw.sh EOSM2 (The terminal buffer resets so some of the history is lost. Don't know if I'm using the right terminology--no pun intended.)

I would think that running "run_canon_fw.sh EOSM2" would show the minimal autoexc.bin output like on the EOSM but maybe not? I've been able to follow most of the walk through except for the part where QEMU displays the camera's UI. If I can get past that hump I would think that the next step would be to compile a "hello world" autoexec.bin to run in QEMU then figure out how to turn on the camera boot flag to run "hello world" in the camera--or not, I may be jumping an important step here.

5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10545
  • 5D Mark Free
Re: ML on EOS-M2
« Reply #66 on: June 19, 2017, 05:11:16 PM »
All OK so far - just keep following the walkthrough.

The terminal reset is intended, as it allows you to scroll to the beginning of the log. Would be nice to print the command line after clearing the screen.

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #67 on: June 20, 2017, 01:52:04 AM »
Seems oh so close.

EOSM2/debugmsg.gdb
Code: [Select]
# patch localI2C_Write to always return 1 (success)
set *(int*)0xFF356DE0 = 0xe3a00001
set *(int*)0xFF356DE8 = 0xe12fff1e

Not 0xe12fff1e, right? Having a tough time with these black magic numbers. I can sort of see how 0xe3a00001 returns 1 but nothing jumps out at me on that second value.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #68 on: June 20, 2017, 07:43:49 AM »
Thanks for the hints a1ex. Fortunately I saved a dump of the old 1.0.2 firmware so I could check my addresses against yours. I was close which would have counted for something in horseshoes but not here. Haven't managed to find my way around the menus but I did change the time date and place to document this historic event.  8)



5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #69 on: June 20, 2017, 03:49:55 PM »
Ok, so I got menus:



Should the next goal be to boot into ML like the EOSM?



Quote
Exercises:
- (easy) patch the loop that times out (probably when initializing the touch IC) and the localI2C_Read loop;
- (easy) print Hello World with the minimal ML target;
- (moderate) print Hello World with the regular ML source code;
- (moderate) implement touch screen emulation;
- (moderate) bring ML menu in QEMU;
- (harder) full ML port, with most features working;
- (extreme) LiveView emulation.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10545
  • 5D Mark Free
Re: ML on EOS-M2
« Reply #70 on: June 20, 2017, 04:25:13 PM »
The second screen should be as easy as specifying "boot=1".

Reason: the ROM dumped from EOS M2 does not have the boot flag enabled, while the ROM dumped from EOS M probably has it.

At this point, we have everything we need to port ML and debug it in the emulator. At least, things like Hello World or ML menu navigation will work without requiring additional changes to QEMU.

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #71 on: June 21, 2017, 06:49:39 AM »
The second screen should be as easy as specifying "boot=1".

Yes indeed. Strange thing is that this is an EOSM2 running firmware version 1.0.3 but it is showing unknown with 1.0.2.



Working on a virtual camera. This will be interesting.
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10545
  • 5D Mark Free
Re: ML on EOS-M2
« Reply #72 on: June 21, 2017, 10:06:33 AM »
That comes from the way properties are stored in memory: there are many setting blocks, and only one of them is active. The property parser from the recovery branch is was not very smart (it doesn't look at whether a block is active or not - it just parses everything). So, it takes the values from the last block in memory that has them (not necessarily the active one).

It probably takes a couple of restarts after the firmware upgrade to replace all the old blocks with 1.0.3.

Updated prop_diag to only parse the active blocks (recovery branch). You can build the display test from that branch, from platform/portable.000 (comment out the ROM dumping routine and cache disabling tricks and that's the plain display test).

Tip: compiling the same branch from some camera directory will give additional capabilities (such as LED blinking), but the binary will no longer be portable.

dfort

  • Hero Member
  • *****
  • Posts: 2189
Re: ML on EOS-M2
« Reply #73 on: June 21, 2017, 04:05:07 PM »
Updated prop_diag to only parse the active blocks (recovery branch). You can build the display test from that branch, from platform/portable.000...

Looks good with the latest recovery autoexec.bin.

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=1"

It also created a fresh ROM dump in QEMU:


However, the MD5 values from the "original" ROM1 and the new QEMU dump don't match. Is this expected?

Code: [Select]
MD5 (/Users/rosiefort/qemu/EOSM2/ROM0.BIN) = f1712be00d98f8cea522f32da216cf96
MD5        (/Volumes/EOS_DIGITAL/ROM0.BIN) = f1712be00d98f8cea522f32da216cf96

MD5 (/Users/rosiefort/qemu/EOSM2/ROM1.BIN) = 3788c332c4a8b2a6b3f521cc92c508ce
MD5        (/Volumes/EOS_DIGITAL/ROM1.BIN) = 5a78ec64480d11275dd064da2cedd136
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103

vagaboundedu

  • New to the forum
  • *
  • Posts: 6
Re: ML on EOS-M2
« Reply #74 on: June 21, 2017, 07:47:08 PM »
Wow--big progress! Did you already try it on your M2?