Canon EOS 1300D / Rebel T6

[a1ex]

Modelling the ROM as IO has just gone over my head complete, so assuming I cant figure that out (yet, im learning) I might continue trying to figure out what might be being mishandled to result in the Assert call.

Yes, that was pretty difficult, as this one requires detailed knowledge of how ROM is configured, and how to do that in QEMU. I think I've figured it out last night, but had to change the ROM layout on all models. This approach appears to break other functionality (e.g. 60D no longer boots), but also gives interesting insights on how ROM reflashing is done (and allows one to implement its emulation, since the ROM addresses appear to behave like I/O during this process):

qemu-1300D.patch

A simpler approach would have been to patch the ROM manually (hardcoding the flash model ID at those addresses where the firmware expects it). Unfortunately, that appears to lock up the bootloader.

A third approach would be to patch the affected function in GDB (see e.g. 700D - patches.gdb) or in ROM (see DIGIC 6 models, but that would remove the ability to run unmodified autoexec.bin's later, since they do a checksum of the ROM at startup to ensure correct firmware version).

Anyway - currently it starts a few tasks, so you can apply the patch and start identifying stubs. Some of them are useful for debugmsg.gdb as well (e.g. DebugMsg, task_create). The current state is also enough for testing the boot process (to see whether ML is able to run code alongside the main Canon firmware, reserve memory for itself, start a task and so on). It won't display any GUI yet, but it shouldn't be hard to reach the Hello World stage without this functionality.

[adamnock]

I am actually floored by how quickly you got this done. I get you know what you are doing, but damn you are committed for an open source project.

RIGHT, brown-nosing over.

Finding the stubs seems to be an accomplishable goal I can do, i found the relevant forum threads and I mostly get the asm command set now, at least typologically, so ill get stuck into that.
I might use the 60D as a base reference for the stubs as it seems to be the most related hardware wise.

This is fun!

[adamnock]

Just a question on starting a new platform definition in the ML source.
Is there a base set of source for the platform that can be used that has a minimal feature/module set enabled?

Ive tried copying the 60D set, and stripping down to minimal components, but im getting feature define compile time failures such as CAM_COLORMATRIX1 and RAW_ZEBRA_ENABLE. I could go through one at a time and fix them, but it seems im working backwards.

Basically I want to start a 1300D platform definition which can really just execute the hello world example, which I believe is what you meant as well?
That and its the only certain way to confirm the found stubs i believe?

[nikfreak]

basically, what I did when starting to port was taking a copy of another camera and rename it.
For e.g. you take 60D or 600D for Digic IV.

Afterwards you rename it and do it step by step as you already guessed.
First you grep for "CONFIG_60D" and then "60D" and then "60d" and you should find almost everything needed. In internals.h you undefine this:

Code: [Select]

#define CONFIG_PROP_REQUEST_CHANGE
and define CONFIG_HELLO_WORLD.

let me know if you get stuck or need help at finding a stub.

[a1ex]

Besides nikfreak's advice, here are some useful tips:

Example for compiling without features: 60D-dm-spy.patch

(I know, they are not isolated very well, as we don't turn them off very often...)

You can also use the minimal target, but that one is really minimal (useful for a lower-level version of Hello World). It uses the platform-specific files (stubs, consts) from the platform directory, a single source file for experiments (minimal.c) and a tiny graphics library (font_direct.c) - besides the loader code in reboot.c. Therefore, it's a good playground environment that does not touch the larger codebase (and does not require a lot of stubs/consts to get started).

The digic6-dumper branch also makes use of the minimal target, but a different way (using a platform-specific minimal.c - because the current boot process has to be changed significantly for newer models). I hope it's not needed for 1300D.

[adamnock]

Hi nikfreak, thanks for the answer, and thanks a1ex again.

Thats what ive been trying, but ive flushed my work and started fresh in case I royally stuffed something.

Basically ive copied to 60D definition, replaced all the specifics, undefined CONFIG_PROP_REQUEST_CHANGE as recommended, and added the 1300D (firmware 110) to the main Makefile and Makefile.platform.map

However with this minimal set, im seeing compile errors eg
../../src/focus.c:1024:33: error: 'GUIMODE_FOCUS_MODE' undeclared (first use in this function)

Whats confusing me is this isnt defined for the 60D either, but that compiles fine.
A walk through the source shows its typically defined in the platform consts

magiclantern@magiclantern-VirtualBox:~/magic-lantern$ grep -rnw './' -e "GUIMODE_FOCUS_MODE"
./platform/1100D.105/consts.h:56:#define GUIMODE_FOCUS_MODE 9
./platform/600D.102/consts.h:112: #define GUIMODE_FOCUS_MODE 9
./platform/550D.109/consts.h:121:#define GUIMODE_FOCUS_MODE 9
./platform/700D.114/consts.h:89:    #define GUIMODE_FOCUS_MODE 0x123456
./platform/7D.203/consts.h:123:#define GUIMODE_FOCUS_MODE 9
./platform/6D.116/consts.h:128:#define GUIMODE_FOCUS_MODE 0x123456
./platform/5D3.123/consts.h:115:#define GUIMODE_FOCUS_MODE 0x123456
./platform/EOSM.202/consts.h:89:#define GUIMODE_FOCUS_MODE 0x123456
./platform/5D3.113/consts.h:100:#define GUIMODE_FOCUS_MODE 0x123456
./platform/unmaintained/40D.111/consts.h:71:#define GUIMODE_FOCUS_MODE 1234
./platform/unmaintained/5DC.111/consts.h:79:#define GUIMODE_FOCUS_MODE 12345
./platform/650D.104/consts.h:89:    #define GUIMODE_FOCUS_MODE 0x123456
./platform/5D2.212/consts.h:87:#define GUIMODE_FOCUS_MODE 9
./platform/500D.111/consts.h:91:#define GUIMODE_FOCUS_MODE 0x27
./platform/50D.109/consts.h:117:#define GUIMODE_FOCUS_MODE 9

Am I missing something obvious here? Should I just define these constants simply for testing compile, or what?
Ill keep trying to figure out what im missing

[adamnock]

AH, I get it.

Right, so there's some platform specific tweaks laying around in the primary project code.

OK, thats fine, ill start digging there.

[a1ex]

Right, 60D has an exception exactly for this constant

Other cameras have dummy definitions (those 0x123456), so anything that checks if the current GUI mode is GUIMODE_FOCUS_MODE will be false.

In general, if you have doubts about a constant, grep the source code to see how it's used. Some of them are used as memory locations where things are written - these need additional care, as the camera bricking does happen (should be recoverable in most cases, but it's best not to get there). This should help understanding why this happens - although the only 100% sure way to prevent bricking is... executing it only in QEMU.

[adamnock]

right well I got ML built using a new platform definition, mostly bogus options of course, but where possible the right setup.

Copied autoexec.bin and 1300D_110.sym to the relevant QEMU 1300D folder, ran with the boot flag and...nothing.
Logout doesnt show autoexec.bin being loaded at all.

Its getting late and its a dawn ANZAC service tomorrow so im calling it a night.
Thanks for the help so far! Im understanding 'some' of what's being achieved, but taking notes. Interesting processes.

[nikfreak]

1300D_110.sym

that one needs to be 8.3. You need to rename and therefore shorten as well as redefine it in your platform dir's makefile.platform.default. Example from 1100D:

Code: [Select]

#Makefile.setup.platform for 1100D

# Definitions for version 105
ML_MODULES_SYM_NAME=t3_$(FW_VERSION).sym
...
So for 1300D you name it t6

[adamnock]

Gotcha.

Ill give that a run in the morrow.

Ta

[adamnock]

(supposed to be sleeping, going to be wrecked)

I wondered about the adapted code and how it was affecting bootflags as memory is being touched in the same area.

Im not sure why, its beyond my understanding of how Qemu is working, but the patch a1ex provided seems to be colliding with the bootflag setter

Code: [Select]

if (strcmp(s->model->name, "1300D") == 0)
    {
        switch (address)
        {
            case 0xF8000000:
            case 0xF8000001:
            case 0xF8000002:
            {
                /* fixme: a bit hackish */
                unsigned int lr = CURRENT_CPU->env.regs[14];
                if (size == 1 && lr == 0x1D4D4)
               {
                    msg = "Flash model ID?";
                    const int model_id[] = { 0xC2, 0x25, 0x39 };
                    ret = model_id[address & 3];
                    break;
                }
            }
        }
    }
If I add a boot flag in there, setting 0xF8000000 to -1, we drop to the FROMUTILITY loader (on the plus side, the Firmware recovery GUI comes up perfectly on the Qemu display). But removing the touches of those two memory locations (0xF8000000 and 0xF8000001) brings us back to the Assert System.c issue.

Could there be a different value expected here for the boot flag? Or does the particular use case for the 1300D need adjustment in the way Qemu is setting that flag?

[a1ex]

Yes, it does (it's a side effect of my modification - one of the reasons I'm not going to commit it in this state). When setting the boot flag, MEM_WRITE_ROM writes to the first copy of the ROM (the one modelled as I/O), and the write is currently ignored.

As a workaround, try writing the bootflag in another copy of the ROM (there are a bunch of mirrored ones - any of them will update all the others). For example, .bootflags_addr = 0xFA000000 (not tested). What I've tested was changing MEM_WRITE_ROM to write at addr+ROM0_SIZE, but that's way too hackish.

Probably it's best to handle it in the ROM write handler.

[adamnock]

Bam, shifting to an alternate copy region works a charm.
Thanks for that.

[adamnock]

Just so this isnt a sudden stop thread, taking a short hiatus because having 2 VM's running for build/testing and searching a 400mb+ file to identify stubs is making my laptop glow red.

Fully intend to resume work in ~2 weeks when I have access to my normal development machine (reason: working remote currently)

Thanks for the help (read: doing 99.99995% of the work) A1ex and Nikfreak. And thanks for the solid explainations. Only a few days in and this has already been my most positive experience with OSS projects to date.

[a1ex]

Thanks for the help (read: doing 99.99995% of the work)

Well, that was because the first assert was not something I'd expect new contributors to be able to figure out (as it was not present on any other model, and requires a very good understanding of the ROM layout - which I don't have yet). This doesn't usually happen with things already documented or mentioned elsewhere.

And I also happened to have a few days off

Edit: looks like the proper way to implement a ROM in QEMU is by using memory_region_init_rom_device. However, that one appears to handle only writes with callbacks. Go figure...

Edit2: looks like memory_region_rom_device_set_romd might do exactly what we are looking for

[adamnock]

Right so identifying the startup stubs.

Thanks to your earlier information cstart's already found, but im confused from that point how to identify bzero32 and create_init_task.
I get both should be being called from cstart, and indeed there are two function calls in the cstart function, but the addresses they reference appear to be outside the ROM space.
Are these functions being called out of RAM instead of ROM? If so, how would one go about dumping RAM in order to identify the functions and hopefully correlate them to their original ROM positions? Or do we not bother and simply reference them in RAM as well?

Unfortunately from what I can gather from forum information, most of the stub locating others have done have been as a result of effectively using manual signature techniques, or at least similarities to other model ROM's and their stubs, but of course we dont have that luxury here.

[a1ex]

Some parts of the ROM are copied to RAM - see reply #14.

You can get the RAM contents either with dd (after identifying what is copied where), or from either QEMU or GDB (they both have commands for dumping the RAM). Or, you can disassemble directly from GDB or from the QEMU monitor console.

You'll need a RAM_OFFSET in the stubs file, similar to DIGIC 5 models. It's explained in the tutorial for finding stubs.

These are helpful:
https://sourceware.org/gdb/onlinedocs/gdb/Machine-Code.html
https://en.wikibooks.org/wiki/QEMU/Monitor

[adamnock]

Hi A1ex

Thanks for that.
I also found some detail in the CHDK wiki which covered about the same as the stubs tut but I think it clicked better.

I get what the job is now, thanks for the response

[a1ex]

I had no luck with memory_region_rom_device_set_romd (the read callback is not called, only the write one), but I think I've found a cleaner workaround:

Code: [Select]

    void * rom_ops_arg = (void *)((uintptr_t) s | rom_id);
    memory_region_init_rom_device(rom, NULL, &rom_ops, rom_ops_arg, name, rom_size, &error_abort);

Then, just hardcode our magic numbers (model ID or whatever that is) in the first 3 bytes of ROM1. That appears to do the trick.

It doesn't log ROM reads though (which is something I wanted on all models, regardless of how the 1300D port will turn out).

Will update the patch later.

[adamnock]

Alright so I created a watchpoint which waited for 0x29898 to change, which is where the first of the RAM referenced functions is located, considering you noted this was occuring before cstart.

This identified the following copy into that location:
f80c00a4:    34812004    strcc   r2, [r1], #4

Following back further, we can identify r1 as being populated with a initial location point of....
f80c0094:    e59f1044    ldr   r1, [pc, #68]   ; f80c00e0: (00001900)

Suggesting the RAM_OFFSET is 0x1900, which is the same as on the DIGIC-V, which helps corroborate my logic hopefully!

(not asking anything, just documenting my process so hopefully some interested person might catch a mistake and go 'WAIT YOU DOLT')

[adamnock]

Wait thats not right....hmm, i think I know what I did wrong there

[adamnock]

Right, so I understand now. Its not a case of identifying where in RAM the ROM portion is copied TO, is where it came FROM.
Also 0x1900 is the start point of the RAM copy of the ROM portion

Leading to the RAM OFFSET value being the location in ROM where the copy is done from, so that the STUB addres would be
the location in ROM, not RAM, which would be

RAM OFFSET address + (RAM function address - RAM start address)

So for the function at 0x29898, the STUB address would be
RAM_OFFSET + (0x29898 - 0x1900).

So now to figure out where its being copied from. That should be trivial I think.....

[adamnock]

Right so assuming my previous was in any way right

Code: [Select]

f80c0090: e59f0044 ldr r0, [pc, #68] ; f80c00dc: (fea87718)
f80c0094: e59f1044 ldr r1, [pc, #68] ; f80c00e0: (00001900)
f80c0098: e59f3044 ldr r3, [pc, #68] ; f80c00e4: (0004f4ac)
loc_f80c009c:
f80c009c:    e1510003 cmp r1, r3
f80c00a0: 34902004 ldrcc r2, [r0], #4
f80c00a4: 34812004 strcc r2, [r1], #4
f80c00a8: 3afffffb bcc loc_f80c009c
f80c00ac:         e59f1034 ldr r1, [pc, #52] ; f80c00e8: (00084d7c)


Code: [Select]

f80c00a0: 34902004 ldrcc r2, [r0], #4

Is loading the relevant ROM data to be copied from the address at r0, with an offset of 4 into r2

Code: [Select]

f80c00a4: 34812004 strcc r2, [r1], #4

is then storing that ROM data into the address in r1, again with an offset of 4, which it gets from r2

Code: [Select]

f80c0094: e59f1044 ldr r1, [pc, #68] ; f80c00e0: (00001900)

Is the RAM start address, which leaves

Code: [Select]

f80c0090: e59f0044 ldr r0, [pc, #68] ; f80c00dc: (fea87718)

Which is the copy FROM location, which is pc + 68, which thanks to the helpful disassembly output, we know is 0xF80C00DC

Hence, the RAM_OFFSET is 0xF80C00DC

Note:

This could be additional verified by the fact that the code at F80C00DC is

Code: [Select]

f80c00dc: fea87718 mcr2 7, 5, r7, cr8, cr8, {0}

Which definitely looks like the type of function you would want in RAM as its intended for a coproc (sic?) and hence would want to be accessible from said RAM as it may reference other local functions the coproc needs to execute/

[a1ex]

[...] the code at F80C00DC is [...]

... is data, not code

edit: committed the initial QEMU code for 1300D (no more need to monkey-patch the ROM with model ID) and also added an option that may help solving your puzzle (see these examples)

(actually I want the memory tracing for other purposes, such as catching non-obvious, but potentially dangerous bugs; here it just happened to be helpful)