Register Map - We need your support!

[g3gg0]

Hi there.

i recently decided to make a "clean" database of the register map from our wiki.
i defined the data format and made some (win/.net) tool that prints a pretty representation of the registers, like it is in datasheets.

example:

Code: [Select]


        <EngineDescription Name="SDCON">
            <Registers>
                <Register Offset="0x000" Name="" Text="Unknown" Description="Set to 0x00 on init"/>
                <Register Offset="0x004" Name="" Text="Unknown" Description="Set to 0x01 on init"/>
                <Register Offset="0x008" Name="" Text="Unknown" Description="Set to 0x00 on init, 0x01/0xF1 before read/write, not used for status block. means: use DMA?"/>
                <Register Offset="0x00C" Name="" Text="Unknown" Description="Set to 0x14/0x13/0x12/0x11/0x02 on command, after writing regs +0x024, +0x020 and +0x010, with 0x11, registers +0x028/+0x02C is ignored probably"/>
                <Register Offset="0x010" Name="" Text="Status Register" Description="">
                    <RegisterFields>
                        <RegisterField xsi:type="Bit" Pos="0" Name="" Text="Transfer finished" Description="" />
                        <RegisterField xsi:type="Bit" Pos="1" Name="" Text="Error during transfer" Description="" />
                        <RegisterField xsi:type="Bit" Pos="20" Name="" Text="DAT transfer data available in reg +0x06C?" Description="" />
                        <RegisterField xsi:type="Bit" Pos="21" Name="" Text="DAT transfer finished?" Description="" />
                    </RegisterFields>
                </Register>
                <Register Offset="0x014" Name="" Text="Unknown" Description="Set to 0x03 before transfer start, 0x00 on ISR"/>
                <Register Offset="0x018" Name="" Text="Unknown" Description="Set to 0x08 on init"/>
                <Register Offset="0x020" Name="" Text="Command frame lower 32 bits" Description="needs 0x0001 being set (end bit)"/>
                <Register Offset="0x024" Name="" Text="Command frame upper 16 bits" Description="needs 0x4000 being set (transmission bit)"/>
                <Register Offset="0x028" Name="" Text="Unknown" Description="Written with 0x88/0x30/0x30 before CMD"/>
                <Register Offset="0x02C" Name="" Text="Unknown" Description="Written with 0x7F08/0x2701/0x80000000 before CMD"/>
               
                <Register Offset="0x034" Name="" Text="Data received lower 32 bits" Description=""/>
                <Register Offset="0x038" Name="" Text="Data received upper 16 bits" Description=""/>
               
                <Register Offset="0x058" Name="" Text="SD bus width" Description="">
                    <RegisterFields>
                        <RegisterField xsi:type="Bits" Start="0" End="4" Name="" Text="0---0  1 bit" Description="" />
                        <RegisterField xsi:type="Bits" Start="0" End="4" Name="" Text="0---1  4 bit" Description="" />
                        <RegisterField xsi:type="Bits" Start="0" End="4" Name="" Text="1---0  8 bit" Description="" />
                    </RegisterFields>
                </Register>
                <Register Offset="0x05C" Name="" Text="Write transfer block size" Description=""/>
                <Register Offset="0x064" Name="" Text="SD bus width" Description="">
                    <RegisterFields>
                        <RegisterField xsi:type="Bits" Start="0" End="4" Name="" Text="0---0  1 bit" Description="" />
                        <RegisterField xsi:type="Bits" Start="0" End="4" Name="" Text="0---1  4 bit" Description="" />
                        <RegisterField xsi:type="Bits" Start="0" End="4" Name="" Text="1---0  8 bit" Description="" />
                        <RegisterField xsi:type="Bits" Start="20" End="27" Name="" Text="01100000  1 bit" Description="" />
                        <RegisterField xsi:type="Bits" Start="20" End="27" Name="" Text="01100000  4 bit" Description="" />
                        <RegisterField xsi:type="Bits" Start="20" End="27" Name="" Text="01110000  8 bit" Description="" />
                    </RegisterFields>
                </Register>
                <Register Offset="0x068" Name="" Text="Read transfer block size" Description=""/>
                <Register Offset="0x070" Name="" Text="Some flags" Description="set to 0x39 before transfer">
                    <RegisterFields>
                        <RegisterField xsi:type="Bit" Pos="0" Name="Transfer running" Text="" Description="" />
                    </RegisterFields>
                </Register>
                <Register Offset="0x07C" Name="" Text="Read/Write transfer block count" Description=""/>
                <Register Offset="0x080" Name="" Text="Transferred blocks" Description=""/>
                <Register Offset="0x084" Name="SDREP" Text="Status register/error codes" Description=""/>
                <Register Offset="0x088" Name="SDBUFCTR" Text="Buffer counter?" Description="Set to 0x03 before reading/writing"/>
            </Registers>
        </EngineDescription>
       

and the output will be

Code: [Select]

SDCON Engines
-------------------
  0xC0C10000    SDCON0
  0xC0C20000    SDCON1
  0xC0C30000    SDCON2
  0xC0C40000    SDCON3
    +0x0000    Unknown
                Set to 0x00 on init
    +0x0004    Unknown
                Set to 0x01 on init
    +0x0008    Unknown
                Set to 0x00 on init, 0x01/0xF1 before read/write, not used for status block. means: use DMA?
    +0x000C    Unknown
                Set to 0x14/0x13/0x12/0x11/0x02 on command, after writing regs +0x024, +0x020 and +0x010, with 0x11, registers +0x028/+0x02C is ignored probably
    +0x0010    Status Register
      -------- -------- -------- -------X     Transfer finished
      -------- -------- -------- ------X-     Error during transfer
      -------- ---X---- -------- --------     DAT transfer data available in reg +0x06C?
      -------- --X----- -------- --------     DAT transfer finished?

    +0x0014    Unknown
                Set to 0x03 before transfer start, 0x00 on ISR
    +0x0018    Unknown
                Set to 0x08 on init
    +0x0020    Command frame lower 32 bits
                needs 0x0001 being set (end bit)
    +0x0024    Command frame upper 16 bits
                needs 0x4000 being set (transmission bit)
    +0x0028    Unknown
                Written with 0x88/0x30/0x30 before CMD
    +0x002C    Unknown
                Written with 0x7F08/0x2701/0x80000000 before CMD
...


now my question is, will there be some helpers wo try to transfer the information from our wiki and from alex' adtg_gui into a single XML file?

if you want to help, then
 * pick the example registermap.xml
 * and the pretty-printer for win32.
 * go to the wiki/adtg_gui
 * and add a missing section to the XML file
 * check how it looks
 * post it here in the forum

its enough if you just post the

<EngineDescription Name="SIO">
    .....     
</EngineDescription>

and the corresponding group

<Group Name="SIO Engines" Engine="SIO" Device="Digic">
    <Engines>
        <Engine Address="0xC0820000" Name="SIO0"/>
        <Engine Address="0xC0820100" Name="SIO1"/>
        <Engine Address="0xC0820200" Name="SIO2"/>
        <Engine Address="0xC0820300" Name="SIO3"/>
    </Engines>
</Group>

everyone can then merge the new ones into their XML.
its not too complicated and you can get some insight into what is happening in those registers.
maybe some of you have some findings that will improve the register map?

thanks

[budafilms]

Wich are the advantages to do this?

Maybe some people could be motivated knowing this!

Thanks!

[ItsMeLenny]

it is g3gg0 that is asking, who cares what the advantage is

Yes I will do some. Were you going to divide it up into sections so that people aren’t doing the same thing over?


The exe seems to run under Wine in Gnu/Linux, but I can't seem to find the file. Could you please make it order things by file name and also add a filter by *.txt? (as there are no icons for file types in Wine)

[g3gg0]

why this is useful:
as of now i collected all information about the registers in the wiki which is "information with inconsistent markup".
every block looks a bit different, sometimes with a few spaces more or less etc.
in case we need it in machine-readable format we cannot simply parse it from the wiki.

where we need it in machine readable format:
 - tools like adtg_gui that shows logs with description (as of now handcoded)
 - ML firmware where we sometimes access registers in a ugly, hardcoded way. e.g. MEM(0xC0F0ABCD) = 1;  better would be SIO0.TXDATA = 1
 - this requires us to generate header files that define the structs and enumf and base addresses. which is an simple 1-afternoon work if we had the register map in machine readable format.
 - for our emulators we use to test some new code or optimized assembly, everything related to registers is still handcraftet and error prone.


it is also part of householding. when you make your notes, you first start writing it onto a scratchpad.
later on, if the information is valuable, you try to sort, preserve and archive it so it is valuable for a longer period.
(i dont trust data that is stored in someone else's wiki)

and i am asking the community because there are so many people willing to help, but cannot because e.g. arm opcode tinkering is indeed a bit complex.
those who just want to spend 1-2 hours and do good things (and potentially learning about the backgrounds) will find this task an ideal solution.

@ItsMeLenny:
sure, will do tonight when i am at home again. did you try it with 'mono'? should work much better.

i would say, if someone is going to start a section, just post it here. (=locking)
as soon it is finished, just edit your post (=unlock)

e.g.
__
I am going to transfer "SIO"

edit: done, here the XML:
....
__

[g3gg0]

updated the .exe

[ItsMeLenny]

Going to start on "SIO Registers (serial IO)"

Update

(actual picture of me)

Code: [Select]

<?xml version="1.0" encoding="utf-16"?>
<RegisterMap xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

    <EngineDescriptions>

        <!-- these registers are accessed with 32 bit accesses, but most of them contain 16 bit data only -->
        <EngineDescription Name="SIO0 (ADTG on 5D3)">
            <Registers>
                <Register Offset="0xC0820004" Name="" Text="SIO0 flags" Description="">
                    <RegisterFields>
                        <RegisterField xsi:type="Bit" Pos="0" Name="" Text="Busy flag" Description="Set this bit when data to write should be transmitted. SIO controller will clear this bit if its done" />
                    </RegisterFields>
                </Register>
                <Register Offset="0xC082000C" Name="" Text="SIO0 setup register (maybe leading clocks, polarity, phase etc)" Description="The lowest bits are the bit count (8 LSB?)" />
                <Register Offset="0xC0820014" Name="" Text="SIO0 setup register (maybe clock speed?)" Description="Set to 0x00000111 for 'ADTG'" />
                <Register Offset="0xC0820018" Name="" Text="SIO0 TX-Data" Description="Set to 0x00 on init" />
                <Register Offset="0xC082001C" Name="" Text="SIO0 RX-Data" Description="Set to 0x00 on init" />
            </Registers>
        </EngineDescription>

        <EngineDescription Name="SIO1 (sound chip, RTC)">
            <Registers>
                <Register Offset="0xC0820104" Name="" Text="SIO1 flags" Description="">
                    <RegisterFields>
                        <RegisterField xsi:type="Bit" Pos="0" Name="" Text="Busy flag" Description="Set this bit when data to write should be transmitted. SIO controller will clear this bit if its done" />
                    </RegisterFields>
                </Register>
                <Register Offset="0xC082010C" Name="" Text="SIO1 setup register (maybe leading clocks, polarity, phase etc)" Description="Set to 0x800A0300 before reading 'SND' (may have no effect, overwritten later). Set to 0x800A0308 before reading 'SND'. Set to 0x800A0310 before writing 'SND'. Set to 0x800E0610 for 'DISPSTATE' (brightness etc?)" />
                <Register Offset="0xC0820114" Name="" Text="SIO1 setup register (maybe clock speed?)" Description="Set to 0x00000000 after writing 'SND'. Set to 0x00000111 for 'DISPSTATE' (brightness etc?)" />
                <Register Offset="0xC0820118" Name="" Text="SIO1 TX-Data" Description="" />
                <Register Offset="0xC082011C" Name="" Text="SIO1 RX-Data" Description="" />
            </Registers>
        </EngineDescription>

        <EngineDescription Name="SIO2 (CMOS, ADTG, EEPROM)">
            <Registers>
                <Register Offset="0xC0820204" Name="" Text="SIO2 flags" Description="">
                    <RegisterFields>
                        <RegisterField xsi:type="Bit" Pos="0" Name="" Text="Busy flag" Description="Set this bit when data to write should be transmitted. SIO controller will clear this bit if its done" />
                    </RegisterFields>
                </Register>
                <Register Offset="0xC082020C" Name="" Text="SIO2 setup register (maybe leading clocks, polarity, phase etc) default: 0x10000000" Description="The lowest bits are the bit count (8 LSB?). Set to 0x00020210 before writing 'CMOS' (16 bit writes). Set to 0x000A0220 before writing 'ADTG' (32 bit writes). Set to 0x1002061B before writing'EEPROM' (may have no effect, overwritten later). Set to 0x1002060B before writing 'EEPROM' (upper 11 bits). Set to 0x10000610 before writing 'EEPROM' (lower 16 bits)" />
                <Register Offset="0xC0820210" Name="" Text="SIO2 unknown" Description="">
                    <RegisterFields>
                        <RegisterField xsi:type="Bit" Pos="0" Name="" Text="Bit cleared during init" Description="" />
                    </RegisterFields>
                </Register>
                <Register Offset="0xC0820214" Name="" Text="SIO2 setup register (maybe clock speed?)" Description="Set to 0x00000115 before writing 'CMOS'. Set to 0x00000111 before writing 'ADTG'. Set to 0x00000115 before read/write 'EEPROM'" />
                <Register Offset="0xC0820218" Name="" Text="SIO2 TX-Data" Description="CMOS/ADTG/EEPROM data to write (16 bit data length)" />
                <Register Offset="0xC082021C" Name="" Text="SIO2 RX-Data" Description="EEPROM data that was read (16 bit data length)" />
            </Registers>
        </EngineDescription>

        <EngineDescription Name="SIO3 ? (600d 1.0.1 at sub_FF1DB44C)">
            <Registers>
                <Register Offset="0xC0820304" Name="" Text="SIO3 flags" Description="">
                    <RegisterFields>
                        <RegisterField xsi:type="Bit" Pos="0" Name="" Text="Busy flag" Description="Set this bit when data to write should be transmitted. SIO controller will clear this bit if its done" />
                    </RegisterFields>
                </Register>
                <Register Offset="0xC0820308" Name="" Text="SIO3 unknown" Description="Written with 0x01 before interrupt setup for comm. Maybe IE flag" />
                <Register Offset="0xC082030C" Name="" Text="SIO3 setup register (maybe leading clocks, polarity, phase etc)" Description="The lowest bits are the bit count (8 LSB?). Set to 0x13020010 before writing 'LENS'" />
                <Register Offset="0xC0820318" Name="" Text="SIO3 TX-Data" Description="" />
                <Register Offset="0xC082031C" Name="" Text="SIO3 RX-Data" Description="" />
            </Registers>
        </EngineDescription>

        <EngineDescription Name="SIO3 ? (600d 1.0.1 at sub_FF1DB44C)">
            <Registers>
                <Register Offset="0xC02000AC" Name="" Text="" Description="(see FF1DB5C8)" />
                <Register Offset="0xC0A00024" Name="" Text="" Description="(7D IPC Tx)" />
                <Register Offset="0xC0820310" Name="" Text="" Description="(see sub_FF1DB3B4)" />
            </Registers>
        </EngineDescription>

    </EngineDescriptions>
</RegisterMap>


[budafilms]

@g3gg0

Thanks for the explanation, very interesting!
Things that I can't imagine!

[g3gg0]

@ItsMeLenny:
thanks for contributing!

one thing - you only have to define *one* SIO engine with *register*offsets*, and not the direct address.
with those you then instantiate the SIOx engines:
<Engine Address="0xC0820000" Name="SIO0"/>
<Engine Address="0xC0820100" Name="SIO1"/>

(see the example file where SIO is already defined IIRC)

[ItsMeLenny]

g3gg0 I think I did lots of things wrong. I couldn't get it to display in the program either.

[g3gg0]

hehe no problem, take a look into how the example XML works and try changing small things.

[Audionut]

Bump!

I should have some time next week to look at this.

[Audionut]

This looks right to me, but the exe will not load it.

Code: [Select]

<RegisterMap xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <EngineDescriptions>
        <EngineDescription Name="CMOS">
            <Registers>
                <Register Offset="0x0" Name="" Text="Analog ISO" Description="Analog ISO adjustment on most cameras"/>
                <Register Offset="0x1" Name="" Text="Vertical offset" Description=""/>
                <Register Offset="0x2" Name="" Text="Horizontal offset / Column Skipping" Description=""/>
                <Register Offset="0x3" Name="" Text="Analog ISO on 6D" Description="Analog ISO adjustment on 6D"/>
                <Register Offset="0x4" Name="" Text="ISO related?" Description="Looks to be related to ISO"/>
                <Register Offset="0x5" Name="" Text="Fine vertical offset" Description="Maybe related to optical black area"/>
                <Register Offset="0x6" Name="" Text="ISO 50 or timing related" Description="Digital ISO 50: FFF => darker image"/>
                <Register Offset="0x7" Name="" Text="Looks like CMOS is dying" Description=""/>
            </Registers>
        </EngineDescription>
       
        <EngineDescription Name="ADTG">
            <Registers>
                <Register Offset="0x8880" Name="" Text="Black level" Description="Reference value for feedback loop?"/>
                <Register Offset="0x8882" Name="" Text="ISO ADTG gain" Description="Per column, mod 4 or mod 8"/>
                <Register Offset="0x8884" Name="" Text="ISO ADTG gain" Description="Per column, mod 4 or mod 8"/>
                <Register Offset="0x8886" Name="" Text="ISO ADTG gain" Description="Per column, mod 4 or mod 8"/>
                <Register Offset="0x8888" Name="" Text="ISO ADTG gain" Description="Per column, mod 4 or mod 8"/>
            </Registers>
        </EngineDescription>
    </EngineDescriptions>
</RegisterMap>
If I remove <(/)RegisterMap> I get an exception.

System.InvalidOperationException: There is an error in XML document (2, 6). ---> System.InvalidOperationException: <EngineDescriptions xmlns=''> was not expected.

[ItsMeLenny]

Yeah, the exe has problems with mine also, and mine looked right (but I don't think it is).
I'm going to get back into this, but I'll probably do the smaller ones.

[g3gg0]

hehe, compare it with the XML i posted (http://pastebin.com/DeEtEXvz)
XML has to start with
 <?xml version="1.0" encoding="utf-16"?>

then between
 <RegisterMap xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
and
 </RegisterMap>

there can be two things:

EngineDescriptions
    <EngineDescriptions>
            <Registers>
                <Register Offset="0x04" Name="Flags">
                    <RegisterFields>
                        <RegisterField xsi:type="Bit" Pos="0" Name="BUSY" Text="" Description="Set this bit when data to write should be transmitted. SIO controller will clear this bit if its done" />
                    </RegisterFields>
                </Register>
                <Register Offset="0x0C" Name="" Text="Setup register" Description="The lowest bits are the bit count (8 LSB?). Maybe leading clocks, polarity, phase etc."/>
                <Register Offset="0x14" Name="" Text="Setup register" Description="maybe clock speed?"/>
                <Register Offset="0x18" Name="TXDAT" Text="TX-Data" Description=""/>
                <Register Offset="0x1C" Name="RXDAT" Text="RX-Data" Description=""/>
            </Registers>
    </EngineDescriptions>

in this block you describe an engine, which has registers relative to its base.
e.g. you describe a SIO engine there (above block) or any other engine that can appear.

Implementations
    <Implementations>
        <Group Name="SIO Engines" Engine="SIO" Device="Digic">
            <Engines>
                <Engine Address="0xC0820000" Name="SIO0"/>
                <Engine Address="0xC0820100" Name="SIO1"/>
                <Engine Address="0xC0820200" Name="SIO2"/>
                <Engine Address="0xC0820300" Name="SIO3"/>
            </Engines>
        </Group>
    </Implementations>

there you describe where the SIO engines are. we have four engines. there you set also the base address for the engines.

[Audionut]

Apologies g3gg0, didn't mention that having that extra detail makes no difference.

Double-click on the file to be loaded in the open file dialog box, the open file box disappears, and nothing else happens.

Code: [Select]

<?xml version="1.0" encoding="utf-16"?>
<RegisterMap xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <EngineDescriptions>
        <EngineDescription Name="CMOS">
            <Registers>
                <Register Offset="0x0" Name="" Text="Analog ISO" Description="Analog ISO adjustment on most cameras"/>
                <Register Offset="0x1" Name="" Text="Vertical offset" Description=""/>
                <Register Offset="0x2" Name="" Text="Horizontal offset / Column Skipping" Description=""/>
                <Register Offset="0x3" Name="" Text="Analog ISO on 6D" Description="Analog ISO adjustment on 6D"/>
                <Register Offset="0x4" Name="" Text="ISO related?" Description="Looks to be related to ISO"/>
                <Register Offset="0x5" Name="" Text="Fine vertical offset" Description="Maybe related to optical black area"/>
                <Register Offset="0x6" Name="" Text="ISO 50 or timing related" Description="Digital ISO 50: FFF => darker image"/>
                <Register Offset="0x7" Name="" Text="Looks like CMOS is dying" Description=""/>
            </Registers>
        </EngineDescription>
       
        <EngineDescription Name="ADTG">
            <Registers>
                <Register Offset="0x8880" Name="" Text="Black level" Description="Reference value for feedback loop?"/>
                <Register Offset="0x8882" Name="" Text="ISO ADTG gain" Description="Per column, mod 4 or mod 8"/>
                <Register Offset="0x8884" Name="" Text="ISO ADTG gain" Description="Per column, mod 4 or mod 8"/>
                <Register Offset="0x8886" Name="" Text="ISO ADTG gain" Description="Per column, mod 4 or mod 8"/>
                <Register Offset="0x8888" Name="" Text="ISO ADTG gain" Description="Per column, mod 4 or mod 8"/>
            </Registers>
        </EngineDescription>
    </EngineDescriptions>
</RegisterMap>

[g3gg0]

missing in that file:

Implementations
    <Implementations>
        <Group Name="SIO Engines" Engine="SIO" Device="Digic">
            <Engines>
                <Engine Address="0xC0820000" Name="SIO0"/>
                <Engine Address="0xC0820100" Name="SIO1"/>
                <Engine Address="0xC0820200" Name="SIO2"/>
                <Engine Address="0xC0820300" Name="SIO3"/>
            </Engines>
        </Group>
    </Implementations>


for CMOS you could use a virtual address 0 for the CMOS implementation

[a1ex]

Related:

Currently, register descriptions are scattered in many places (wiki: Register Map, ADTG, adtg_gui.c, qemu/eos.c, some forum posts), as you can see on previous messages.

Help is welcome to make sure all these registers are described on the wiki (if not, you may simply copy the description from the other sources, or flag any obvious differences).

The preferred way is to format these descriptions as XML, as asked by g3gg0, but if you find this too difficult, I'd be happy if you would simply help keeping the wiki up to date.

Writing scripts to sync these things could be very helpful. Here's a script that parses adtg_gui source, for example: annotate_log.py - you could modify it to output wiki text, XML or IDA scripts. The XML should include enough information to re-create the data structure from adtg_gui (so, eventually, the XML should become the main source of information about these registers, and all other formats should be auto-generated from it).

In the same way, qemu/eos.c could be parsed to extract the register addresses (although a bit more difficult to handle all the cases, and even more difficult to generate the emulation code from an XML). Autogenerating comments with register descriptions should be doable, though.

Useful commits to watch: adtg_gui.c and qemu branch.

Tip: while doing this, you might understand interesting things, such as how to change the LiveView resolution, how to move the recorded area around, how to adjust the speed of your SD card or how to tweak picture style parameters (just some examples).

[g3gg0]

yeah absolutely.
the fact information is scattered over mutliple knowledge bases will prevent to understand the big picture.
but having it in multiple places like tools or emulators is necessary for the process of understanding too.

before we could do some more "magic" we need a good documentation that consolidates all information.