Author Topic: ML on EOS-M2  (Read 109185 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #100 on: July 03, 2017, 07:24:38 PM »
So confused -- why doesn't find_fnt.py just spit out the right numbers?

It does - try this:
Code: [Select]
python find_fnt.py ROM0.BIN 0xF0000000

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #101 on: July 03, 2017, 09:12:39 PM »
python find_fnt.py ROM0.BIN 0xF0000000
Code: [Select]
Find bitmap fonts in Canon DSLR firmwares
Arm.Indy. based on work by Pel, Trammel Hudson and A1ex

0xf0124838: FNT
0xf012483c: (+0x04) 0xffd8
0xf012483e: (+0x06) font_width = 40
0xf0124840: (+0x08) charmap_offset = 0x24
0xf0124844: (+0x0c) charmap_size = 0x31b4
0xf0124848: (+0x10) bitmap_size = 0x8ab3e
0xf012484c: (+0x14) font name = 'HCanonGothic///'
0xf012485c: (+0x24) char_codes[]. 3181 chars
0xf0127a10: (+0x31d8) offsets[]. Last offset value = 0x8ab08
0xf012abc4: (+0x638c) bitmaps[]
  0xf01b56cc: (+0x90e94) last bitmap
  +0x00: bitmap width = 28
  +0x02: bitmap height = 28
  +0x04: char width = 36
  +0x06: X offset = 4
  +0x08: Y offset = 16
    bitmap size = 0x70

0xf01b5704: FNT
0xf01b5708: (+0x04) 0xffd8
0xf01b570a: (+0x06) font_width = 40
0xf01b570c: (+0x08) charmap_offset = 0x24
0xf01b5710: (+0x0c) charmap_size = 0x188
0xf01b5714: (+0x10) bitmap_size = 0x31c4
0xf01b5718: (+0x14) font name = 'CanonMonospace'
0xf01b5728: (+0x24) char_codes[]. 98 chars
0xf01b58b0: (+0x1ac) offsets[]. Last offset value = 0x3142
0xf01b5a38: (+0x334) bitmaps[]
  0xf01b8b7a: (+0x3476) last bitmap
  +0x00: bitmap width = 22
  +0x02: bitmap height = 22
  +0x04: char width = 22
  +0x06: X offset = 0
  +0x08: Y offset = 0
    bitmap size = 0x42

So this should work?

Code: [Select]
// http://magiclantern.wikia.com/wiki/Fonts
#define BFNT_CHAR_CODES    0xf012485c
#define BFNT_BITMAP_OFFSET 0xf0127a10
#define BFNT_BITMAP_DATA   0xf012abc4

In addition 0xF[0-7] can be used so 0xf712485c, etc. will make it look more like the 5D3 and 5D2 addresses used in consts.h.

Wow, talk about overcomplicating things. So I guess my mistake was using the wrong base address--but isn't the base address for the EOSM2 0xFF0C0000?

Just for fun (yeah, this is fun) I tried a DIGIC 4 camera that has the fonts in ROM0 -- the 5D2. We want to get to this:

Code: [Select]
#define BFNT_CHAR_CODES    0xf7c5E9C0
Running "python find_fnt.py ROM0.BIN" returns 0xff06e9c0 which wouldn't work but "python find_fnt.py ROM0.BIN 0xF0c00000" returns 0xf0c5e9c0 which is equivalent to 0xf7c5E9C0. (There's mixed upper/lower case in the 5D2 code but that shouldn't matter.)

So what's up with this?

Code: [Select]
if (len(sys.argv)>2):
  base = int(sys.argv[2], 16)
else:
  base = 0xff010000

I thought that was supposed to work with DIGIC 4 cameras.

P.S. Here's an easy coding task - print the name of the buttons in the logs.

You mean in the mem_spy module interpret the button codes to print out the names of the buttons?
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #102 on: July 03, 2017, 09:50:39 PM »
... that script assumes the ROM dump loads at 0xff010000 (this was the main firmware start address for DIGIC 4 models, so in the early days, ROM files were saved from that address)

Meaning: the script expects the address where the ROM file was dumped from; it doesn't care where the main firmware starts inside that ROM.

Quote
You mean in the mem_spy module interpret the button codes to print out the names of the buttons?

No, that's complicated. What I meant was about QEMU logs (the user interface side) - instead of "Key event: 2a -> 0e0e002d", print something human-readable.

Mapping button codes to names is doable as well; it's done in ARM-console to some extent (bgmt command), but should probably be rewritten with unicorn instead (similar to extract_button_codes.py). That requires some low-level knowledge though.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #103 on: July 03, 2017, 10:36:36 PM »
Got it...I think.

boot-hack.c
Code: [Select]
static void backup_rom_task()
{
    backup_region("ML/LOGS/ROM1.BIN", 0xF8000000, 0x01000000);
    backup_region("ML/LOGS/ROM0.BIN", 0xF0000000, 0x01000000);
}

Shouldn't all of the new ROM dumps be saved at these addresses? If that's the case find_fnt.py should probably be updated to:

Code: [Select]
if (len(sys.argv)>2):
  base = int(sys.argv[2], 16)
else:
  base = 0xf0000000

I suppose that the ROM dumpers also use these addresses?

In any case...moving on.

...QEMU logs...instead of "Key event: 2a -> 0e0e002d", print something human-readable.

You mean like "Key event: 2a -> 0e0e002d (half-shutter press)", "Key event: aa -> 0e0e002e (half-shutter release)" or unpress -- if I got that right.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #104 on: July 04, 2017, 08:20:31 PM »
Finished all I could on stubs.S and consts.h. If anyone wants to follow along I'm working on a work-in-progress branch which is sort of a scratch pad to take notes as I go along. Before making the pull request (provided I can get that far) I'll make a new branch so the pull request will be as clean as possible.

Following some of the hints and references in the code led me to some helpful scripts like find_fnt.py that could use some updating to make it easier on the next guy that uses it.

https://bitbucket.org/hudson/magic-lantern/pull-requests/844/find_fntpy-update/diff

Another interesting script is in magic-lantern/modules/adtg_log and is called parse_bin.py. I'm not sure how to use it (no documentation I could find)

[EDIT] Well I did find this:

the bin can be parsed with the python script in the module directory ;)

In any case, running this:

Code: [Select]
python parse_bin.py -f ROM1.BIN
created a large amount of information. I was looking for FRAME_SHUTTER_BLANKING_ZOOM and FRAME_SHUTTER_BLANKING_NOZOOM which required looking up some ADTG registers. I used the adtg_gui before but never figured out adtg_log. In any case, there was this discussion on the forum which looked promising but I never could find what I was looking for using parse_bin.py.

[EDIT] To be more specific, I was trying to find this:

Code: [Select]
#define FRAME_SHUTTER_BLANKING_ZOOM   (*(uint16_t*)0x40481B20) // ADTG register 805f
#define FRAME_SHUTTER_BLANKING_NOZOOM (*(uint16_t*)0x40481B24) // ADTG register 8061

There are a few more addresses to look up then time to get back to QEMU to try and get "Hello world" working and hopefully find the firmware signature.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #105 on: July 06, 2017, 02:29:32 AM »
Thought I'd try something easy:

Exercises:
- (easy) patch the loop that times out (probably when initializing the touch IC) and the localI2C_Read loop;
- (easy) print Hello World with the minimal ML target;
- (moderate) print Hello World with the regular ML source code;
- (moderate) implement touch screen emulation;
- (moderate) bring ML menu in QEMU;
- (harder) full ML port, with most features working;
- (extreme) LiveView emulation.

So I went into magic-lantern/minimal/qemu-frsp and tried this:

Code: [Select]
MODEL=EOSM2 make
...
[ LD       ]   magiclantern
stdio.o: In function `my_fprintf':
/Users/rosiefort/magic-lantern/minimal/qemu-frsp/../../src/stdio.c:32: undefined reference to `FIO_WriteFile'
make: *** [magiclantern] Error 1

Am I missing something in EOSM2? Maybe but I can't build any model. Next I went into magic-lantern/minimal/qemu-hptimer and:

Code: [Select]
MODEL=EOSM2 make
...
magiclantern.bin: 3052 bytes
[ CC       ]   reboot.o
[ CC       ]   disp_direct.o
[ CC       ]   footer.o
[ LD       ]   autoexec
[ OBJCOPY  ]   autoexec.bin
[ XOR_CHK  ]   autoexec.bin

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x000060 0x0009e1e0 0x0009e1e0 0x00bec 0x10c20 RWE 0x10

Yay!

Running autoexec.bin in QEMU:



Boo!

Is this because I haven't gotten the firmware signature yet? Keep thinking that maybe that is the next challenge.

[EDIT] Doh! Figured out why that message is coming up--I was using the EOSM.202 firmware signature as a filler for the still undiscovered EOSM2.103 firmware signature. Commenting out the firmware signature and the minimal autoexec.bin will just hang. So I'm back in that Catch 22 situation. I did try some of the debugging tricks with a regular ML Hello World build but that's a moderately difficult exercise to complete.

Here is how that went:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=1" -s -S -d autoexec,ramw,calls,io,v & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb

Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - DFFFFFFF: eos.iomem
[EOS] enabling code execution logging.
[EOS] enabling memory access logging.
[EOS] enabling singlestep.
[EOS] disabling verbose logging until autoexec.bin starts...
[EOS] loading './EOSM2/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './EOSM2/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[EOS] loading './EOSM2/SFDATA.BIN' as serial flash, size=0x1000000
[MPU] warning: non-empty spell #31 (08 06 03 1f) has duplicate(s): #40
[MPU] warning: non-empty spell #41 (0a 08 01 3b) has duplicate(s): #37 #38
[MPU] warning: non-empty spell #45 (26 24 01 4e) has duplicate(s): #46

[MPU] Available keys:
- Arrow keys   : Navigation
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess
- M            : MENU
- P            : PLAY
- I            : INFO/DISP
- Q            : guess
- L            : LiveView
- Shift        : Half-shutter
- B            : Open battery cover
- F1           : show this help

Setting BOOTDISK flag to FFFFFFFF
0xffff0000 in ?? ()
+macro define CURRENT_TASK   ((int)0xFFFFFFFF)
+macro define CURRENT_ISR    ((int)0xFFFFFFFF)
+macro define RTC_VALID_FLAG ((int)0xFFFFFFFF)
+macro define NUM_CORES      1
+macro define PRINT_CALLSTACK 0
+set pagination off
+set output-radix 16
+define hook-quit
+define KRED
+define KCYN
+define KBLU
+define KGRN
+define KYLW
+define KRESET
+macro define CURRENT_TASK_NAME (((int*)CURRENT_TASK)[0] ? ((char***)CURRENT_TASK)[0][9] : CURRENT_TASK)
+define print_callstack
+define print_current_location
+define print_current_location_with_callstack
+define print_formatted_string
+define DebugMsg_log
+define DebugMsg1_log
+define printf_log
+define task_create_log
+define task_switch_log
+define msleep_log
+define assert_log
+define assert0_log
+define create_semaphore_log
+define delete_semaphore_log
+define print_sem_name
+define take_semaphore_log
+define give_semaphore_log
+define create_msg_queue_log
+define print_mq_name
+define post_msg_queue_log
+define try_receive_msg_queue_log
+define receive_msg_queue_log
+define register_interrupt_log
+define register_func_log
+define mpu_decode
+define mpu_send_log
+define mpu_recv_log
+define try_expand_ram_struct
+define try_post_event_log
+define delayed_call_print_name
+define delayed_call_log
+define SetTimerAfter_log
+define SetHPTimerAfterNow_log
+define SetHPTimerNextTick_log
+define CancelTimer_log
+define engine_resource_description
+define engine_resources_list
+define CreateResLockEntry_log
+define LockEngineResources_log
+define AsyncLockEngineResources_log
+define UnLockEngineResources_log
+define StartEDmac_log
+define SetEDmac_log
+define print_date_time
+define set_date_time
+define load_default_date_time_log
+define log_result
Breakpoint 1 at 0x4398
Breakpoint 2 at 0x7360
Breakpoint 3 at 0xff0c5144
FFFF096C: MCR p15,0,Rd,cr9,cr1,0: XSCALE_LOCK_ICACHE_LINE <- 0x40000006 (40000000 - 40000FFF, 0x1000)
FFFF0970: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x2078
FFFF0978: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0x12078   
FFFF3F54: MCR p15,0,Rd,cr6,cr0,0:  946_PRBS0 <- 0x3F       (00000000 - FFFFFFFF, 0x100000000)
FFFF3F5C: MCR p15,0,Rd,cr6,cr1,0:  946_PRBS1 <- 0x3D       (00000000 - 7FFFFFFF, 0x80000000)
FFFF3F64: MCR p15,0,Rd,cr6,cr2,0:  946_PRBS2 <- 0xE0000039 (E0000000 - FFFFFFFF, 0x20000000)
FFFF3F6C: MCR p15,0,Rd,cr6,cr3,0:  946_PRBS3 <- 0xC0000039 (C0000000 - DFFFFFFF, 0x20000000)
FFFF3F74: MCR p15,0,Rd,cr6,cr4,0:  946_PRBS4 <- 0xFF00002F (FF000000 - FFFFFFFF, 0x1000000)
FFFF3F7C: MCR p15,0,Rd,cr6,cr5,0:  946_PRBS5 <- 0x37       (00000000 - 0FFFFFFF, 0x10000000)
FFFF3F84: MCR p15,0,Rd,cr6,cr6,0:  946_PRBS6 <- 0xF700002F (F7000000 - F7FFFFFF, 0x1000000)
FFFF3F8C: MCR p15,0,Rd,cr2,cr0,0: DCACHE_CFG <- 0x70       
FFFF3F90: MCR p15,0,Rd,cr3,cr0,0:       DACR <- 0x70       
FFFF3F94: MCR p15,0,Rd,cr2,cr0,1: ICACHE_CFG <- 0x70       
FFFF3F9C: MCR p15,0,Rd,cr5,cr0,0:    DATA_AP <- 0x3FFF     
FFFF3FA0: MCR p15,0,Rd,cr5,cr0,1:    INSN_AP <- 0x3FFF     
FFFF3FA4: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x12078
FFFF3FC8: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC001307D
FFFF0984: MCR p15,0,Rd,cr9,cr1,1: XSCALE_UNLOCK_ICACHE <- 0x6        (00000000 - 00000FFF, 0x1000)
FFFF0988: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC001307D
FFFF0990: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005307D
FFFF09B8: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005307D
FFFF09C0: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
SD LOAD OK.
Open file for read : AUTOEXEC.BIN
SD: CMD12 in a wrong state
[SDIO] Error
SD: CMD12 in a wrong state
[SDIO] Error
File size : 0x671C0
Now jump to AUTOEXEC.BIN!!
0010E5F8: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010E600: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
[EOS] enabling verbose logging for autoexec.bin...
      PC jump? 0x800000 lr=10e65c                                                at [10e674:10e65c]
      0x0010e674:  e1a0f001      mov pc, r1
[EOS] enabling verbose logging for autoexec.bin...
[BOOT] changing user_mem_start from 0x17ec74 (1567860) to 0x117940 (1145152)
[BOOT] changing init_task from 0xff0c57a0 (-15968352) to 0x9e1f0 (647664)
DRYOS PANIC: Module Code = 1, Panic Code = 2
^C
Program received signal SIGINT, Interrupt.
0xff0c19f4 in ?? ()
(gdb)
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #106 on: July 06, 2017, 05:24:47 PM »
Took a closer look at the firmware signature - it's affected by our QEMU patches. So, to find it, either debug it without enabling other ROM patches (those with SET), or find it out outside GDB (e.g. print it on the screen), or just keep it commented out.

Next:
Code: [Select]
[BOOT] changing user_mem_start from 0x17ec74 (1567860) to 0x117940 (1145152)

This one looks fishy to me. Why? Because we "push" this address to the right in order to make room from our application. Let's take a look at the memory map (printed by 0xFF0C6E70).

Start the firmware under GDB, then, once it's running, hit CTRL-C and call this function. The simplest (and most hackish) way to do this, without loading guest code (autoexec.bin), is:

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S -d debugmsg & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb
(wait until it boots)
CTRL-C
set $pc = 0xFF0C6E70
continue

This will break the execution flow (after returning from that function, there will be dragons), but at least we'll get what we are looking for. The function probably has no arguments (R0 and R1 are first written, and then read), therefore simply jumping to it should be fine.

Result:
Code: [Select]
...
sys_mem_start   0x0018ac00
sys_mem_max        1179648
user_mem_start  0x00100d80
user_mem_max        515828
...

Here, user_mem is malloc, sys_mem is AllocateMemory the place where DryOS allocats stack space for its tasks.

That means, user_mem_start is not 0x17EC74, but 0x100d80. You'll find that in cstart as well. Set RESTARTSTART to that, or a little bit higher - e.g. 0x100E00). Now, the boot process will "push" user_mem_start to start right where our code ends.

BTW - most models have "RESTARTSTART is selected to be just above the end of the bss". Coincidentally or not, Canon's BSS (the end of what they zero out before cstart) is also 0x100D80; however, it is my understanding that we are not modifying Canon's BSS, but user_mem_start (therefore shrinking their malloc memory pool).

Hence, the 7D2 has an updated comment:
Code: [Select]
# DryOS memory map
# RESTARTSTART is selected to be at user_mem_start
# (aka heap start / DRY_HEAP_START / malloc memory pool)
#
RESTARTSTART  = 0x001CC400

There, Canon's BSS ends at 0x745E8, so the previous comments about BSS were incorrect. Time to fix them.

Having fixed RESTARTSTART and the (misnamed) HIJACK_INSTR_BSS_END, let's check the memory map again, this time loading some minimal ML:

Code: [Select]
sys_mem_start   0x0018ac00
sys_mem_max        1179648
user_mem_start  0x00104580
user_mem_max        501492

Notice the user_mem (malloc) pool is now smaller and shifted to the right. The size needed by our binary (MemSiz when compiling) is 0x03780 (ymmv), so it appears to match.

How much free space do we have? The 7D2 source contains 3 important hints: malloc_info, sysmem_info and smemShowFix (all easily found in the firmware). Let's call the first one (from GDB, to avoid recompiling):

Without ML:
Code: [Select]
Malloc Information (onetime type)
  Start Address       = 0x00100d88
  End Address         = 0x0017ea60
  Total Size          = 0x0007dcd8 (   515288)
  Allocated Size      = 0x00015ab0 (    88752)
  Allocated Peak      = 0x00015ab0 (    88752)
  Allocated Count     = 0x0000005a (       90)
  Free Size           = 0x00068228 (   426536)
  Free Block Max Size = 0x000681f0 (   426480)
  Free Block Count    = 0x00000002 (        2)

With minimal ML:
Code: [Select]
Malloc Information (onetime type)
  Start Address       = 0x00104588
  End Address         = 0x0017ea60
  Total Size          = 0x0007a4d8 (   500952)
  Allocated Size      = 0x00015ab0 (    88752)
  Allocated Peak      = 0x00015ab0 (    88752)
  Allocated Count     = 0x0000005a (       90)
  Free Size           = 0x00064a28 (   412200)
  Free Block Max Size = 0x000649f0 (   412144)
  Free Block Count    = 0x00000002 (        2)


OK, so we don't have enough space to load the entire ML (maybe only if you compile it without any features). We'll have move somewhere else.

This was the "classic" boot process, originally used by 5D2. Not all models have enough free space here to accommodate the full autoexec.bin.

One option is to try the AllocateMemory (like 550D or 1100D), and another is to hunt for possibly unused memory areas (smemShowFix - FF222B68) - see http://www.magiclantern.fm/forum/index.php?topic=5071.0.

Note: I wouldn't call random stuff on a real camera without triple-checking the stub, the number of arguments, any prerequisites that might be and so on. Here we are on a virtual machine, so there's nothing to break - feel free to experiment away.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #107 on: July 07, 2017, 05:18:22 PM »
(after returning from that function, there will be dragons)

I've been meticulously going through all your notes and when I don't understand something I look it up. If this is a reference to the John Ringo book:

Quote
The world is a paradise-and then, in a moment, it ends.

Yep, that's pretty much what happened.

All that debugging you did was due to a very basic mistake on my part. I started out by copying over all the EOSM.202 settings and then went through every line and changed the values to what I could find in the EOSM2.103 disassembly. Well, almost every line. Somehow I skipped over Makefile.platform.default. Oops. The hints were there and I might have been able to figure it out, except for the "a little bit higher" part. From my understanding of your last post:

magic-lantern/platform/EOSM2.103/Makefile.platform.default
Code: [Select]
#Makefile.setup.platform for EOS M2

ifeq ($(FW_VERSION),103)
CANON_NAME_FIR=EOSM2103.FIR

ROMBASEADDR = 0xFF0C0000

# DryOS memory map
# RESTARTSTART is selected to be at user_mem_start
# (aka heap start / DRY_HEAP_START / malloc memory pool)
# (actually 0x00100d80 but make it a little bit higher)
#
RESTARTSTART = 0x100E00
endif

Having fixed RESTARTSTART and the (misnamed) HIJACK_INSTR_BSS_END...

Not sure about the HIJACK_INSTR_BSS_END fix. There is a comment that should be changed:

magic-lantern/platform/EOSM2.103/consts.h
Code: [Select]
#define HIJACK_INSTR_BSS_END 0xFF0C1C94  //BSS_END is 0x17EC74
in order to match the disassembly:
Code: [Select]
; ff0c1c90: (00100d80)
; ff0c1c94: (0017ec74)

...let's check the memory map again, this time loading some minimal ML:

The hackish way to do this doesn't work when loading the QEMU minimal autoexec.bin so this probably requires doing it via debugmsg.gbd, right? Uh, how?

Speaking of debugmsg.gbd - looking at other similar cameras (EOSM and 700D), shouldn't this work on the EOSM2?

Code: [Select]
b *0x1900
assert_log

That might be useful.

The 7D2 source contains 3 important hints: malloc_info, sysmem_info and smemShowFix (all easily found in the firmware).

The closest I could find in the EOSM2 source is: "Malloc Information (%s type)", "ShowWinsysMemory" and "..smemShowFix"

Let's call the first one (from GDB, to avoid recompiling):

I'm lost. How did you do that?

Ok--getting back to the "easy" stuff:

Exercises:
- (easy) patch the loop that times out (probably when initializing the touch IC) and the localI2C_Read loop;
- (easy) print Hello World with the minimal ML target;

The loop that times out is annoying.

Code: [Select]
...
[  TouchMgr:ff346008 ] (c1:16) Touch IC Ver : 0x0000
   154:   993.280 [TCH]Touch IC Ver : 0x0000
[  TouchMgr:ff345160 ] (c1:03) ExecuteSIO32 err
...
[  TouchMgr:ff344f58 ] (c1:03) ExecuteSIO8 err
...
[  TouchMgr:ff34602c ] (c1:06)  TouchPanelIC Ack Error
[   Startup:0003671c ] task_create(DisplayMgr, prio=12, stack=0, entry=36628, arg=8ce0e8)

Am I on the right track here?

Code: [Select]
# patch TouchMgr to avoid err messages and loop timeout
# ExecuteSIO8
set *(int*)0xff344f?? = 0xe???????
# ExecuteSIO32
set *(int*)0xff3451?? = 0xe???????

I'm still at a loss at how to print Hello World with the minimal ML target. For now I've got the firmware signature commented out. However, it seems that you keep hinting that it can be found with QEMU.

Took a closer look at the firmware signature - it's affected by our QEMU patches. So, to find it, either debug it without enabling other ROM patches (those with SET), or find it out outside GDB (e.g. print it on the screen), or just keep it commented out.

The code to find it doesn't seem too complex:

Code: [Select]
static int compute_signature(int* start, int num)
{
    int c = 0;
    int* p;
    for (p = start; p < start + num; p++)
    {
        c += *p;
    }
    return c;
}

start = SIG_START which on Digic V cameras like the EOSM2 is 0xFF0C0000
num = 0x10000

So shouldn't it be possible to write a rather simple script to find the firmware signature from ROM1.BIN or would that be short circuiting the porting process?
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #108 on: July 07, 2017, 06:36:17 PM »
Quote
The world is a paradise-and then, in a moment, it ends.

What happens here is because of how function calls are done on ARM processors. When a BL executes, the next address (where the code should return after the function ends) is placed into LR, then PC is updated to the first address of the called function.

What we did in GDB was to interrupt the firmware somewhere (we don't know where) and change the PC to some function (that takes no arguments). That function will run just fine (we can use its results), but it won't know where to return - and that's where the dragon appears.

In QEMU it's fine - it will just return somewhere. It may lock up, it may go back to the caller function without finishing the function we interrupted... or it may somehow jump to EraseSectorOfRom or other similar address. That's why you must never try this on the real firmware.

Quote
...let's check the memory map again, this time loading some minimal ML:
...
The hackish way to do this doesn't work when loading the QEMU minimal autoexec.bin so this probably requires doing it via debugmsg.gbd, right? Uh, how?

First you need to compile ML from some minimal target (you already did). Then place it on the card image, then start it with boot=1.

To call the function, you may now use either GDB (ctrl-c + jumping to the function), or insert a call to that function somewhere in the minimal ML. The second method will not break the execution flow, as the function will know where to return.

Quote
The closest I could find in the EOSM2 source is: "Malloc Information (%s type)", "ShowWinsysMemory" and "..smemShowFix"

Yes, these are. For example, malloc_info is FF0C750C (the function that references the first string). It takes no arguments, so you can just jump to it.

Quote
shouldn't this work on the EOSM2?
Code: [Select]
b *0x1900
assert_log

It does. Not all models have assert at 0x1900 (1300D doesn't).

Quote
So shouldn't it be possible to write a rather simple script to find the firmware signature from ROM1.BIN or would that be short circuiting the porting process?

It is. But once you patch something via GDB (such as "set *(int*)0xFF356E28 = 0xe3a00001"), the value will change.

Quote
I'm still at a loss at how to print Hello World with the minimal ML target.

Look in src/minimal.c.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #109 on: July 08, 2017, 03:04:49 AM »
Exercises:
- (easy) patch the loop that times out (probably when initializing the touch IC) and the localI2C_Read loop;

One small step for a1ex, one giant leap for dfort:

Code: [Select]
# patch TouchMgr to avoid loop timeout
# ExecuteSIO8
set *(int*)0xFF344F40 = 0xe1a0000b
# ExecuteSIO32
set *(int*)0xFF345148 = 0xe1a0000b

At least it doesn't slow down to a crawl when it hits the TouchMgr section.

Not sure what the issue is with the localI2C_Read loop. I'm not noticing any slowdown or delays though this is printed in red to the console:

Code: [Select]
   154:   824.832 [TCH]ERROR  TouchPanelIC Ack Error
   155:   760.576 WARN [I2C] localI2C_Read : 378 (Task : Startup)
   156:   760.832 WARN [I2C] localI2C_Read : 378 (Task : Startup)
   157:   761.088 WARN [I2C] localI2C_Read : 378 (Task : Startup)
   158:   761.344 WARN [I2C] localI2C_Read : 378 (Task : Startup)
   159:   761.600 WARN [I2C] localI2C_Read : 378 (Task : Startup)
   160:   761.856 WARN [I2C] localI2C_Read : 378 (Task : Startup)
   161:   761.856 WARN [I2C] localI2C_Read : 378 (Task : Startup)
   162:   762.112 WARN [I2C] localI2C_Read : 378 (Task : Startup)
   163:   762.368 WARN [I2C] localI2C_Read : 378 (Task : Startup)
   164:   833.536 [IMPP] H264E InitializeH264EncodeFor1080pDZoom
   165:   833.536 [IMPP] H264E InitializeH264EncodeFor1080p25fpsDZoom
   172:   836.608 [STARTUP] startupInitializeComplete
   173:   841.216 [MR] mvrChangeAckCBR : Video - Mode=0, Type=2, Rate=30, GOP=15
   174:   850.176 [MR] mvrChangeAckCBR : Video - Mode=0, Type=2, Rate=30, GOP=15
   180:   863.232 [CEC]CEC OFF
   182:   882.176 [RTC] !! RTC CHECK ERROR !!

Also had a breakthrough on making a minimal ML. I simply copied what was done in magic-lantern/minimal/600D.102 (and other cameras). Continuing from where I got lost last time:

Code: [Select]
^C
Program received signal SIGINT, Interrupt.
0xff352318 in ?? ()
(gdb) set $pc = 0xFF0C6E70
(gdb) continue
Continuing.
...
sys_mem_start   0x0018ac00
sys_mem_max        1179648
user_mem_start  0x00100d80
user_mem_max         14336
...

With that same minimal ML:

Code: [Select]
^C
Program received signal SIGINT, Interrupt.
0xff352318 in ?? ()
(gdb) set $pc = 0xFF0C750C
(gdb) continue
Continuing.
Malloc Information (onetime type)
  Start Address       = 0x00100d88
  End Address         = 0x00104370
  Total Size          = 0x000035e8 (    13800)
  Allocated Size      = 0x000025c0 (     9664)
  Allocated Peak      = 0x000025c0 (     9664)
  Allocated Count     = 0x00000011 (       17)
  Free Size           = 0x00001028 (     4136)
  Free Block Max Size = 0x00001028 (     4136)
  Free Block Count    = 0x00000001 (        1)

To be continued (still no Hello World) but first - to quote kichetof - take a beer and enjoy it!
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #110 on: July 08, 2017, 07:32:58 AM »
At least it doesn't slow down to a crawl when it hits the TouchMgr section.

... it no longer boots the GUI ...

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #111 on: July 08, 2017, 09:03:31 AM »
... it no longer boots the GUI ...

That's strange -- I can see the GUI here. You are talking about the Canon menus, right? I still don't have a minimal ML that displays "Hello World" much less any ML GUI with or without that TouchMgr "fix."

This is the entire debugmsg.gdb:

Code: [Select]
# ./run_canon_fw.sh EOSM2 -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb

source -v debug-logging.gdb

macro define CURRENT_TASK 0x8FBCC
macro define CURRENT_ISR  (*(int*)0x648 ? (*(int*)0x64C) >> 2 : 0)

b *0x4398
DebugMsg_log

b *0x1900
assert_log

b *0x7360
task_create_log

# patch DL to avoid DL ERROR messages
set *(int*)0xFF156348 = 0xe3a00015

# patch localI2C_Write to always return 1 (success)
set *(int*)0xFF356E24 = 0xe3a00001
set *(int*)0xFF356E28 = 0xe12fff1e

# patch TouchMgr to avoid loop timeout
# ExecuteSIO8
set *(int*)0xFF344F40 = 0xe1a0000b
# ExecuteSIO32
set *(int*)0xFF345148 = 0xe1a0000b

# skip SerialFlash version check
set *(int*)0xFF0C4278 = 0xe3a00000

# break infinite loop at Wait LeoLens Complete
b *0xFF0C5144
commands
  printf "Patching LeoLens (infinite loop)\n"
  set *(int*)($r4 + 0x28) = 1
  c
end

continue

Think I found ShowWinsysMemory

No ML:
Code: [Select]
   185:  2214.144 [WINSYS] allocated=44140 max_region=412876

Minimal ML:
Code: [Select]
[  PowerMgr:ff13432c ] (04:16) allocated=0 max_region=1066808

Couldn't find smemShowFix ("..smemShowFix") but this looks interesting:

No ML:
Code: [Select]
(gdb) set $pc = 0xff133b80
(gdb) c
Continuing.
[SVG] Free/Total:3229904/8118272
[SVG] SVG Alloc:0
[SVG] Max Region:2763608
[SVG] number of allocated block:4863
[SVG] dslr_malloc ResetAllocatePointer!!!!!!
[SVG] Memory Allocation Error at :429392408
[SVG] svg_malloc(size:0, pFileName, lineNo)
MaxRegion = 2763608
[SVG] Free/Total:3229904/8118272
[SVG] Max Region:2763608
[SVG] number of allocated block:4863

Minimal ML:
Code: [Select]
[SVG] Free/Total:6431420/8118272
[SVG] SVG Alloc:0
[SVG] Max Region:6427636
[SVG] number of allocated block:1537
[SVG] dslr_malloc ResetAllocatePointer!!!!!!
[SVG] Memory Allocation Error at :429392408
[SVG] svg_malloc(size:0, pFileName, lineNo)
MaxRegion = 6427636
[SVG] Free/Total:6431420/8118272
[SVG] Max Region:6427636
[SVG] number of allocated block:1537
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #112 on: July 08, 2017, 09:38:43 AM »
Hm, it's working now. Previously it got stuck in a loop and stayed like that for some 15 minutes...

Tip: comment out DebugMsg_log and replace it with -d debugmsg (much faster)

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #113 on: July 08, 2017, 06:02:29 PM »
Yes, much faster.

Am I doing it right?

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S -d debugmsg & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb
Following your link shows this neat trick:

Code: [Select]
env QEMU_EOS_DEBUGMSG=0x5b90 ./run_canon_fw.sh 5D3,firmware="boot=0" -d debugmsg,callstack,v
but that 0x5b90 address is specific to the 5D3, right?

Booting into the Canon menus "boot=0" seems pretty solid though there are some error messages logged. Making the minimal autoexec.bin to print "Hello World" is another story. I thought I'd try it with the 5D3.113 but--

Code: [Select]
cd minimal/5D3.113/
RosieFoComputer:5D3.113 rosiefort$ make
../../platform/Makefile.platform.base:46: *** ROMBASEADDR is not defined.  Stop.

What the?
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #114 on: July 08, 2017, 09:10:41 PM »
Yep, looks fine. Of course, each model has a different DebugMsg address; will try to automate it.

5D3 has 2 firmware versions, so the build system doesn't know which one to pick. Adding FW_VERSION=113 fixes it.

This was probably broken when merging 5D3 1.2.3 (and since the minimal target is neither widely used, nor covered by self-tests, it wasn't noticed).

Back to the topic - I've got minimal hello world working on M2 after fixing RESTARTSTART and related consts as discussed above, then compiling your source. That was before the timeout patches, but it's still working afterwards.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #115 on: July 09, 2017, 06:35:35 AM »
I've got minimal hello world working on M2...

Nice!

I've got a minimal "Hello World" working on a real 5D3.113 but not on QEMU. Wanna trade?   :D

The 5D3.113 minimal ML was built in the minimal/5D3.113 directory and the firmware signature was commented out so it should work like an early port. This is what it looks like on the camera. (Thanks for the FW_VERSION=113 tip):



That same minimal ML on QEMU:



The point of this exercise was to see if the problem is with the minimal ML or the QEMU setup or maybe both. Looks like that at least for the 5D3 it is QEMU. That is if my theory holds up which is that the same minimal ML autoexec.bin should be able to run on both the camera and QEMU. This is also assuming that ML on 5D3.113 in QEMU is working.

Note that I also tried the qemu-hptimer version with this result: (Note the "Hello from task init" that wasn't on the previous test.)

Code: [Select]
File size : 0x1940
Now jump to AUTOEXEC.BIN!!
0010DCCC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010DCCC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
Hello from task init
K285 READY
[      init:ff3014ac ] create_semaphore('EventProcedure', 1)
Temporary breakpoint 10 at 0xff3014b0

Program received signal SIGTRAP, Trace/breakpoint trap.
0xff3014b0 in ?? ()
(gdb)

qemu-frsp isn't compiling over here.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #116 on: July 09, 2017, 09:10:21 AM »
For 5D3, try running it with "patches.gdb" instead - that's the only one where I've tried the GUI.

The "Hello world" works without GDB script at all, although that way I get blank screen and no menus (not sure why).

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #117 on: July 09, 2017, 06:38:58 PM »
What finally worked with the 5D3.113 for me was simply:

Code: [Select]
./run_canon_fw.sh 5D3,firmware="boot=1"


That is working with and without the "#define SIG_5D3_113  0x2e2f65f5" in fw-signature.h so in theory this should also work with the EOSM2 minimal ML, right?

Here is the output from ./run_canon_fw.sh EOSM2,firmware="boot=1"
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #118 on: July 09, 2017, 07:54:37 PM »
Some models are able to boot the GUI without any GDB patches (e.g. 60D, 500D). They will show the date/time dialog, but you can close it cleanly - then you get the usual Canon GUI. If you patch date/time, it boots directly to the GUI. The patches.gdb are there, but they are no longer necessary (they used to be before figuring out menu navigation).

Other models (700D, EOSM2) require some sort of patching. They won't boot the GUI without these patches.

5D3 is in-between. Without any patches, it shows a blank screen (no menus), but other than that, it appears to boot almost completely. For example, it initializes the bitmap VRAM - that's why you were able to print Hello World. Patching date/time is enough to enable the GUI, menu navigation and info screens (which is a mystery to me - why is the behavior different from e.g. 60D?)

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #119 on: July 10, 2017, 03:16:27 AM »
So I created and ran minimal ML versions for the EOSM and 700D. No GUI so I was messing around trying to find patches but then realized--if it doesn't initializes the bitmap VRAM then "Hello World" will never print, will it?

Maybe I already have a working EOSM2 minimal ML and don't know it?

That long printout I posted on pastebin might be the print to screen code or maybe calculating the firmware signature? It ends with:

Code: [Select]
[DM] FROM Write Complete!!!
That doesn't look like an error message and the navigation keys seem to be working--so is that it?

I built in the minimal directory and from the platform/EOSM.103 directory with the firmware signature commented out and CONFIG_HELLO_WORLD defined -- both worked. I keep thinking that the next step is to get the firmware signature. Looks like we pretty much there on June 25 but then:

Took a closer look at the firmware signature - it's affected by our QEMU patches. So, to find it, either debug it without enabling other ROM patches (those with SET), or find it out outside GDB (e.g. print it on the screen), or just keep it commented out.

I kept it commented out but it looks like it is time to figure it out in order to get to the next step.

Exercises:
- (easy) patch the loop that times out (probably when initializing the touch IC) and the localI2C_Read loop; Check?
- (easy) print Hello World with the minimal ML target; Maybe check this off too?
- (moderate) print Hello World with the regular ML source code; Next exercise?
- (moderate) implement touch screen emulation;
- (moderate) bring ML menu in QEMU;
- (harder) full ML port, with most features working;
- (extreme) LiveView emulation.

Then there is this:

OK, so we don't have enough space to load the entire ML (maybe only if you compile it without any features). We'll have move somewhere else.
...
Note: I wouldn't call random stuff on a real camera without triple-checking the stub, the number of arguments, any prerequisites that might be and so on. Here we are on a virtual machine, so there's nothing to break - feel free to experiment away.

So at what point can we run a minimal ML on the camera?

5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: ML on EOS-M2
« Reply #120 on: July 10, 2017, 09:26:03 AM »
Quote
[DM] FROM Write Complete!!!

This message usually follows some sort of error (assert). The log has many DL messages, which means it was not run under GDB.

Quote
Maybe I already have a working EOSM2 minimal ML and don't know it?

Very likely :D

Quote
CONFIG_HELLO_WORLD

That won't work, since it will compile with the full feature set (thus reserving as much RAM as a full binary would). However, the minimal hello world requires very little RAM, so it should work (under GDB, with firmware signature commented out).

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #121 on: July 10, 2017, 05:08:03 PM »
...the minimal hello world requires very little RAM, so it should work (under GDB, with firmware signature commented out).

Ok then, it's working. At least QEMU buttons seem to be working--no GUI.

Code: [Select]
./run_canon_fw.sh EOSM2,firmware="boot=0" -s -S -d debugmsg & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb > log_file.txt 2> err_file.txt

Empty error file. That's a good thing, right? However, the log file shows that maybe things ended badly:

Code: [Select]

This GDB was configured as "--host=x86_64-apple-darwin10 --target=arm-none-eabi".

Breakpoint 1 at 0x408660d4: file ../../src/reboot.c, line 211.
Breakpoint 2 at 0x1900
Breakpoint 3 at 0x7360
Breakpoint 4 at 0xff0c5144
[      init:ff352260 ] task_create(PowerMgr, prio=20, stack=400, entry=ff352084, arg=0)
[      init:0003671c ] task_create(DbgMgr, prio=1f, stack=0, entry=36628, arg=46e584)
[      init:ff0f5f24 ] task_create(Startup, prio=19, stack=2800, entry=ff0f5d68, arg=46e880)
[      init:ff0c33d4 ] task_create(TaskMain, prio=1d, stack=0, entry=ff0c28d8, arg=0)
[   Startup:ff0c375c ] task_create(Startup2, prio=11, stack=400, entry=ff0c3604, arg=0)
[   Startup:0003671c ] task_create(PropMgr, prio=14, stack=0, entry=36628, arg=4d7ca8)
[   Startup:ff0d261c ] task_create(DL, prio=f, stack=400, entry=ff0d23fc, arg=0)
[   Startup:0003671c ] task_create(EventMgr, prio=f, stack=0, entry=36628, arg=4e3898)
[   Startup:0003671c ] task_create(FileCache, prio=14, stack=0, entry=36628, arg=603d08)
[   Startup:0003671c ] task_create(RscMgr, prio=12, stack=0, entry=36628, arg=604110)
[   Startup:ff129dbc ] [ASSERT] this at ./FileMgr/FileMgr.c:2279, ff129dc0

Looks like it stopped on "FM_RegisterSpaceNotifyCallback" -- more patching to do?

5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #122 on: July 10, 2017, 09:17:47 PM »
Looks to me like something is going on in the GUI code. Does [DM] stand for Display Manager?

Tried to figure out what was going on here:

Code: [Select]
FF0C11BC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
FF0C11BC: MCR p15, ...          : CACHEMAINT x2456 (omitted)
FF0C11BC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC0051079
FF0C11F0: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC0051079
FF0C11DC: MCR p15, ...          : CACHEMAINT x1 (omitted)
FF0C11F0: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC0050079
FF0C113C: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC0050079
...
FF0C1194: MCR p15, ...          : CACHEMAINT x257 (omitted)
FF0C1194: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC0051079
FF0C11F0: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC0051079
FF0C1180: MCR p15, ...          : CACHEMAINT x1 (omitted)
FF0C11F0: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC0050079
FF0C113C: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC0050079
FF0C1120: MCR p15, ...          : CACHEMAINT x1 (omitted)
FF0C113C: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005007D
FF0C1168: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005007D
FF0C1120: MCR p15, ...          : CACHEMAINT x1 (omitted)
FF0C1168: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D


[DM] FROM Write Complete!!!

Checked stubs.S and it seems to me that these might GUI calls so I doubled checked all the GUI stubs and they look right.

This message usually follows some sort of error (assert). The log has many DL messages, which means it was not run under GDB.

Looks like this might be the assert:

Code: [Select]
[   Startup:ff129dbc ] [ASSERT] this at ./FileMgr/FileMgr.c:2279, ff129dc0
Here is a log with GDB.

I was trying to find a patch to get around this and came up with something rather interesting.

Code: [Select]
# FM_RegisterSpaceNotifyCallback
set *(int*)0xff129d94 = 0xe3a00001

What this does is to create the same issue when running EOSM2,firmware="boot=0" as it does with "boot=1" without this "patch". In other words, I was able to break it in the same way as the minimal ML. Running "boot=0" without the "patch" will boot all the way to the Canon GUI.

Feel like a kid sticking a fork into an electrical outlet to see what happens. As much as I'd like to see something running on the camera I'm glad we've got QEMU to work out these issues.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: ML on EOS-M2
« Reply #123 on: July 11, 2017, 06:45:41 AM »
Did a little more QEMU'ing with the 550D. Why? Because it seems to be the best supported in QEMU and this:

OK, so we don't have enough space to load the entire ML (maybe only if you compile it without any features). We'll have move somewhere else.
...
One option is to try the AllocateMemory (like 550D or 1100D)...

So the minimal ML looks like this: [Edit] Check out that little dot in the lower right corner--I think it represents the camera's LED because it flashes on and off just like on the camera.



That "Hello World" in the retro graphical font shows up everywhere reminding you that ML is running. Kind of cool and annoying at the same time. Ok, we aren't going to do anything with a minimal ML so how about the full ML only without a firmware signature defined and "Hello World" configured?



What is cool about this is that the 550D dump that I'm using doesn't have the boot flag enabled so it is pretty much like the EOSM2 ROMs that I'm using. Is is displaying the correct firmware signature--nice! That's what I thought we were going for with these QEMU exercises.

By the way, these were run with a simple:

Code: [Select]
./run_canon_fw.sh 550D,firmware="boot=1"
Ok so how about setting up AllocateMemory on the EOSM2?

I'm pretty sure that these are correct:

~/magic-lantern/platform/EOSM2.103/consts.h
Code: [Select]
// Used in boot-hack.c with CONFIG_ALLOCATE_MEMORY_POOL
#define ROM_ITASK_START 0xff0c57a0
#define ROM_ITASK_END  0xffc45984 // kinda iffy
#define ROM_CREATETASK_MAIN_START 0xff0c3064
#define ROM_CREATETASK_MAIN_END 0xff0c33dc
#define ROM_ALLOCMEM_END 0xff0c3074 // kinda iffy
#define ROM_ALLOCMEM_INIT 0xff0c307c // kinda iffy
#define ROM_B_CREATETASK_MAIN 0xff0c5814

Then this needs to be turned on:

~/magic-lantern/platform/EOSM2.103/internals.h
Code: [Select]
/** This camera loads ML into the AllocateMemory pool **/
#define CONFIG_ALLOCATE_MEMORY_POOL

Now there's some other adjustments that need to be made like maybe:

now you have to patch the DATA for the LDR. this is the address where the LDR     R3, =0x1F300000 gets the 1F3000000 from.
should be right below the function.
> #define RSCMGR_MEMORY_PATCH_END          0xF80.....   (yes, 0xF800!!! also on your cameras)

interesting that the 650D has 0x1F2C4000 bytes only.... hmm

after you defined that struct, make sure you set RESTARTSTART to 0x1FE00100  (original value minus 2MiB)

Ok--gotta ask a noob question, what is 2MiB? I looked it up but am not sure I'm getting it. Is it 0x800?

Anyway, I don't quite have all the pieces yet because it doesn't build:

Code: [Select]
...
[ LD       ]   magiclantern
boot-hack.o: In function `init_task_patched':
/Users/rosiefort/magic-lantern/platform/EOSM2.103/../../src/boot-hack.c:642: undefined reference to `reloc'
/Users/rosiefort/magic-lantern/platform/EOSM2.103/../../src/boot-hack.c:650: undefined reference to `reloc'
make: *** [magiclantern] Error 1

So reading further along that topic I ran into this:

EOS M2, QEMU:



Code: [Select]
  1062: 48619.520 [RSC] --- Common Top ----
  1063: 48619.520 [RSC] [TOP1]                  0x40C2A000
  1064: 48619.776 [RSC] REPLACE_IMAGE_VRAM      0x40C2A000 0x00032000 204800
  1065: 48623.104 [RSC] SSS_DEVELOP_WORK        0x40C5C000 0x00038000 229376
  1066: 48626.432 [RSC] SDS_DEVELOP_WORK        0x40C94000 0x00038000 229376
  1067: 48629.760 [RSC] DARKCUR_COMP_WORK       0x40CCC000 0x00020000 131072
  1069: 48637.440 [RSC] FENCING_WORK            0x40CEC000 0x00010000 65536
  1070: 48640.768 [RSC] DCFNO                   0x40CFC000 0x00004000 16384
  1071: 48644.096 [RSC] LVMARGE_P_DEF_DATA_1    0x40D00000 0x0000A000 40960
  1072: 48647.424 [RSC] LVMARGE_P_DEF_DATA_2    0x40D0A000 0x0000A000 40960
  1074: 48654.848 [RSC] LVMARGE_P_DEF_DATA_3    0x41FF0000 0x0000A000 40960
  1075: 48657.920 [RSC] LVMARGE_P_DEF_DATA_ZOOM 0x40D14000 0x0000C000 49152
  1076: 48661.504 [RSC] FILE_HEADER             0x40D20000 0x00240000 2359296
  1077: 48664.320 [RSC] SAF WORK                0x40FA0000 0x00200000 2097152
  1078: 48667.136 [RSC] BMPVRAM1                0x411A0000 0x00080000 524288
  1080: 48676.864 [RSC] BMPVRAM2                0x41220000 0x00080000 524288
  1081: 48680.704 [RSC] ENGINE_MIRROR           0x412A0000 0x00044000 278528
  1082: 48688.640 [RSC] VSHADING_COMP_WORK      0x412E4000 0x000DC000 901120
  1084: 48692.992 [RSC] STILL SCAR              0x413C0000 0x00075B00 482048
  1085: 48692.992 [RSC] TUNEDATA                0x41435B00 0x00140000 1310720
  1086: 48692.992 [RSC] TUNEDATA2               0x41575B00 0x00160000 1441792
  1087: 48693.504 [RSC] FIXDATA                 0x416D5B00 0x0021E500 2221312
  1088: 48693.504 [RSC] LVMARGE_P_DEF_DATA_CROP 0x418F4000 0x0000C000 49152
  1089: 48693.760 [RSC] WIRELESS_WORK2          0x41900000 0x00300000 3145728
  1090: 48699.648 [RSC] WIRELESS_WORK1   *      0x41C00000 0x00200000 2097152
  1091: 48702.464 [RSC] ADAPTER_TRANSFER *      0x0 0x00000000 0
  1092: 48702.464 [RSC] EEKO                    0x41E00000 0x001E0000 1966080
  1093: 48702.720 [RSC] SHOOTING_CREATIVEFILTER 0x41FE0000 0x00010000 65536
  1094: 48704.512 [RSC] EXMEM3_AREA_4           0x0 0x00000000 0
  1095: 48706.304 [RSC] --- Usually Mode ----
  1096: 48708.352 [RSC] MEMORY_MGR              0x42000000 0x04000000 67108864
  1098: 48715.520 [RSC] ONLY MEM1 1             0x46000000 0x02000000 33554432
  1099: 48717.568 [RSC] ONLY MEM1 2             0x48000000 0x02000000 33554432
  1100: 48721.152 [RSC] IMGPLAY_WORK            0x4A000000 0x00A00000 10485760
  1101: 48724.736 [RSC] IMGPLAY_WORK2           0x4AA00000 0x00400000 4194304
  1102: 48727.808 [RSC] SS_DEVELOP1             0x4A000000 0x00E00000 14680064
  1104: 48734.976 [RSC] EXMEM3_AREA_2           0x4AE00000 0x000EA000 958464
  1105: 48738.816 [RSC] AVERAGE_WORK_TOP        0x4AEEA000 0x01116000 17915904
  1106: 48742.144 [RSC] AVERAGE_WORK_BOTTOM     0x4C000000 0x01116000 17915904
  1107: 48745.472 [RSC] SS_DEVELOP_OTHER_WORK   0x4D116000 0x00400000 4194304
  1109: 48752.384 [RSC] SLIDE_SHOW_WORK         0x4D516000 0x00800000 8388608
  1110: 48753.920 [RSC] CAPTURE_WORK1           0x4D600000 0x00240000 2359296
  1111: 48760.576 [RSC] EXMEM3_AREA_1           0x4DD16000 0x016BA000 23830528
  1112: 48764.416 [RSC] IMGVRAM1                0x4F3D0000 0x00410000 4259840
  1113: 48764.928 [RSC] IMGVRAM2                0x4F7E0000 0x00410000 4259840
  1114: 48765.184 [RSC] IMGVRAM3                0x4FBF0000 0x00410000 4259840
  1116: 48771.584 [RSC] ---   GIS Mode   ----
  1117: 48772.352 [RSC] TEMPMEM1                0x48000000 0x02000000 33554432
  1118: 48775.680 [RSC] WORK                    0x4A000000 0x08600000 140509184
  1119: 48775.680 [RSC] IMGPLAY_WORK            0x4A000000 0x00A00000 10485760
  1120: 48775.680 [RSC] IMGPLAY_WORK2           0x4AA00000 0x00400000 4194304
  1121: 48779.520 [RSC] MOVIE_REC_WORK          0x4AE00000 0x00FA0000 16384000
  1122: 48779.520 [RSC] MOVIE_PLAY_WORK         0x4BE00000 0x00E00000 14680064
  1123: 48782.848 [RSC] SLIDE_SHOW_WORK         0x4D516000 0x00800000 8388608
  1124: 48787.200 [RSC] MOVIE_STREAM            0x4DD16000 0x01400000 20971520
  1126: 48790.528 [RSC] IMGVRAM1                0x4F3D0000 0x00410000 4259840
  1127: 48796.160 [RSC] IMGVRAM2                0x4F7E0000 0x00410000 4259840
  1128: 48796.160 [RSC] IMGVRAM3                0x4FBF0000 0x00410000 4259840
  1129: 48797.696 [RSC] EXMEM3_1_AREA           0x42000000 0x06000000 100663296
  1130: 48800.256 [RSC] EXMEM3_2_AREA           0x4CC40000 0x004D6000 5070848
  1131: 48802.304 [RSC] ---   HDR Mode   ----
  1133: 48806.912 [RSC] TEMPMEM1                0x48000000 0x02000000 33554432
  1134: 48808.192 [RSC] WORK                    0x4800000 0x0A600000 174063616
  1135: 48813.312 [RSC] IMGPLAY_WORK            0x4A000000 0x00A00000 10485760
  1136: 48815.360 [RSC] IMGPLAY_WORK2           0x4AA00000 0x00400000 4194304
  1137: 48815.360 [RSC] MOVIE_REC_WORK          0x4AE00000 0x00FA0000 16384000
  1138: 48815.360 [RSC] MOVIE_PLAY_WORK         0x4BE00000 0x00E00000 14680064
  1139: 48815.872 [RSC] SLIDE_SHOW_WORK         0x4D516000 0x00800000 8388608
  1140: 48816.896 [RSC] MOVIE_STREAM            0x4DD16000 0x01400000 20971520
  1141: 48817.920 [RSC] IMGVRAM1                0x4F3D0000 0x00410000 4259840
  1142: 48817.920 [RSC] IMGVRAM2                0x4F7E0000 0x00410000 4259840
  1143: 48817.920 [RSC] IMGVRAM3                0x4FBF0000 0x00410000 4259840
  1144: 48820.224 [RSC] EXMEM3_1_AREA           0x42000000 0x06000000 100663296
  1145: 48823.808 [RSC] EXMEM3_2_AREA           0x4CC40000 0x004D6000 5070848
  1146: 48824.320 [RSC] ---    NR Mode   ----
  1148: 48829.952 [RSC] NR_MEMORY_MGR           0x42000000 0x08000000 134217728
  1149: 48833.024 [RSC] COMPOSITION_WORK_TOP    0x0 0x00000000 0
  1150: 48836.096 [RSC] COMPOSITION_WORK_BOTTOM 0x0 0x00000000 0
  1151: 48839.424 [RSC] ---    DP Mode   ----
  1152: 48841.216 [RSC] DP_SINGLE               0x42000000 0x05E00000 98566144
  1153: 48841.216 [RSC] DP_MULTI                0x47E00000 0x04D94000 81346560
  1154: 48843.520 [RSC] DP_CAPTURE_WORK1        0x4CB94000 0x00040000 262144
  1155: 48844.288 [RSC] DP_AVERAGE_TOP          0x4DBD4000 0x01116000 17915904
  1157: 48848.128 [RSC] DP_AVERAGE_BOTTOM       0x4ECEA000 0x01116000 17915904
  1158: 48849.408 [RSC] --- Indev Mode ----
  1159: 48851.200 [RSC] [INDVMGR]               0x0
  1160: 48852.480 [RSC] YUV                     0x4AEEA000 0x0222C000 35831808
  1161: 48852.480 [RSC] YUV_OUT                 0x42000000 0x0222C000 35831808
  1162: 48853.760 [RSC] INDV_WORK               0x0 0x00000000 0

256MB. Note the WORK region has invalid size (it would overflow), so I've patched it to get a proper graph.

No unused areas for us.

No unused areas for us?

Oh please say it ain't so!
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: ML on EOS-M2
« Reply #124 on: July 12, 2017, 01:49:44 AM »