Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Topics - srsa

Pages: [1]
1
General Development / MMU, memory mapping (DIGIC 7 and later)
« on: October 08, 2020, 12:54:10 AM »
For tech details, see ARM Architecture Reference Manual ARMv7-A and ARMv7-R edition. Understanding and using this does require knowledge of ARM assembly.

It's now known that the main ARM cores in DIGIC 7, 8, X are ARM Cortex A9.

There have been occasional comments about using the MMU, but I don't remember seeing any actual research or code.
So, here's this as a start.

The Cortex A9 can address 4GB memory. About the half of that can be RAM.
All DIGIC 7 and later cameras seem to have a very similar memory layout, see this post from a1ex for example.
Most of physical memory is mapped to the same virtual address - except for the first 4kB.

When reviewing the memory mapping related routines on my D7 and D8 cameras, I found that the main firmware (EOS and PowerShot) uses the ROM translation tables when it starts.
The ROM tables are located at the start of ROM (address 0xe0000000), their complete size is 0x4900 bytes.
They consist of the following:
- 0x4000 bytes of L1 table for both cores, describing logical addresses from 32MB upwards
- 0x400 bytes of L2 table describing logical address range 0...1024kB, core0
- 0x400 bytes of L2 table describing logical address range 0...1024kB, core1
- 0x80 bytes of L1 table describing logical address range 0...32MB, core0
- 0x80 bytes of L1 table describing logical address range 0...32MB, core1

It turned out recently that DIGIC X models use a RAM copy of the tables. I have not done any review on a DIGIC X firmware, so it could be that my D7 and D8 findings do not apply there.

So, below is for DIGIC 7 and 8.

There is a RAM copy of translation tables. On PowerShot firmware models, their location is at 0xdffc0000, EOS firmware uses an _uncached_ dynamically allocated RAM copy.
The firmware switches to the RAM copy whenever it is about to write into ROM (and switches back when it's done). I _think_ that mostly happens just before a clean shutdown.
If it turns out that it can happen during normal camera operation, one will have to patch the routines responsible for switching the translation tables.
For my current experiment on my D7 camera, I don't do that.

I have to note that there is a tradeoff between fine-grained memory mapping and CPU performance. The less L2 tables, the better performance.
The Canon tables are pretty efficient, supersections (16MB granularity) are used mostly.

In my current implementation, I settled for 64kB granularity when doing ROM replacement.



Replacing parts of ROM with modified RAM copy.


Since the firmware doesn't change mapping while it runs, it is possible to make a modified static mapping at boot time (when autoexec.bin executes).
Making the modified tables is a job for one CPU core, but both cores need to be set to the new tables.
It is also important to make sure that both cores have their caches in sync before they are set to use the new tables.

In order to be able to replace larger amounts of ROM, one needs RAM that is not used by any part of the camera.
- RAM for the translation tables (0x4900 bytes for copying the Canon tables and 0x400 bytes for each L2 table).
The Canon tables need to be copied to a 16kB-aligned address, the alignment of L2 tables is 0x400 bytes.
- RAM for the replaced ROM pages, 64kB (aligned) for each page.

To find that RAM, one can start with reviewing the output of smemShowFix, looking for holes, preferably in the common section. The hole needs to be verified by reviewing the disassembly/disassemblies, looking for addresses that fall into the hole's range.
In some cases, the firmware of other (ARM, Xtensa) cores needs to be reviewed too.
The other way of making sure a memory area is unused is dumping the suspected hole's content to a file before/after doing all sorts of camera operations, and then comparing the dumps.

When one or more suitable holes are found, one can start doing what I did in my EOS M100 port.
I wrote a few words about that here.
The (currently) latest source can be found attached to this post (the 7z archive).
Files of interest are
platform/m100/sub/mmuutils.c
platform/m100/sub/100a/boot.c (usage example)
platform/m100/sub/100a/replaced.c (usage example)
platform/m100/sub/100a/stubs_entry_2.S (related stubs)
See also comments in those files.

Some other notes:
Changes made to the ROM content are only visible to the main ARM cores. For example, it is not possible to modify ROM data that is later transferred from ROM by DMA.


I'm sure there is a lot I did not mention, I'll try to answer questions. Corrections and different approaches are also welcome.

2
General Development / Canon Basic scripting (DIGIC 8, DIGIC X models)
« on: September 03, 2020, 01:11:27 AM »
I'm moving this to a separate thread as it turned out that more models have support for scripting.

So far confirmed to exist on:
PowerShot SX740
EOS M50
EOS R, RP, R5, R6
EOS 250D

About the language. Card setup.
Note that some people have problems making a usable script card. If that happens, try again. If in doubt, a card prepared for scripting can be verified on any older PowerShot from ~2005 on. The card probably needs to be formatted FAT16/32.

Any script error will crash the camera and require a battery pull.

The language used on DIGIC 8 models is the same as described on the above links, but CHDK-related scripts will not work. The reason for this is that the so-called event procedures (named firmware functions) are mostly different.
Unlike on older PowerShots, most of the available event procedures are pre-registered - can be used without running registration procedures.
Also unlike on older cameras, it is possible to try executing an eventproc that may not exist. This can be achieved by using ExecuteEventProcedure.

Code: [Select]
ExecuteEventProcedure("Peek32",0x1000)is equivalent to
Code: [Select]
Peek32(0x1000)and will return the word read from address 0x1000
Code: [Select]
ExecuteEventProcedure("arbitrary_invalid_function")will return -1

Native code can be executed by using ExportToEventProcedure
Code: [Select]
ExportToEventProcedure("my_func",0xe1234567)will make the firmware function at 0xe1234566 (with thumb bit) available for script as eventproc my_func

All variables of the script are 32-bit signed integers, interpreted as integer, pointer, pointer to string, as required.
It's worth to note that operator precedence is different from other languages. Negative numbers (that includes pointers to ROM) can have funny effects in expressions involving more than two numbers.

File handling capabilities are quite limited when using only available event procedures - native code must be found and used for reading anything from file, writing a binary file, erasing a file, etc.
String constants are limited to 127 bytes. sprintf() only allows up to 8 substitutions (probably a language limit).

The available event procedures can be harvested from ROM, by looking up the registration functions in the disassembly.

Some example scripts
ROM dumper (main image from 0xe0040000, without bootloader)
Code: [Select]
private sub Initialize()
    SaveBootableToFile()
end sub
ROM dumper, not working on M50 due to missing eventproc
Code: [Select]
private sub Initialize()
    SaveAllRomImageToFile()
end sub
Enable bootflag
Code: [Select]
private sub Initialize()
EnableBootDisk()
end sub
Disable bootflag
Code: [Select]
private sub Initialize()
DisableBootDisk()
end sub
Dumping camera log (current session)
Code: [Select]
private sub Initialize()
dumpf()
end sub

I posted a slow dumper script in this older post.

Finally, the usual warning: do not post your ROM dumps publicly.

3
General Development / Unused (unnecessary) firmware tasks
« on: January 04, 2017, 04:01:07 PM »
I read that some ports (EOS M) experience a shortage of tasks. If this is related to the hardcoded limit of DryOS tasks, the following might help. Quoting an old post of mine:
Quote
Looking at the output of 'extask' (extended task information for DryOS), I noticed that a task named EvShel (event shell) has unusually large stack space (0x8000 bytes aka 32kB). This task is only used for debugging over the UART, but it's sitting idle in all cameras used by ordinary people - waste of RAM.
As an experiment, I have prevented this task from starting (on the a470) - it's usually started from the last BL in the 'CommonDrivers' task. Side effect is that this prevents two other tasks from being started (ConsoleSvr, LowConsole) - these would be started by EvShel itself.
That was a PowerShot cam. If cameras on the EOS codebase also survive if those 3 tasks are missing, ML could start 3 tasks more.
I only verified that the above task names are present in the 550d dump. Somebody will have to check if Evshel task's creation can indeed be removed from the boot process, without penalties.

Just an idea.

Pages: [1]