Canon EOS 1300D / Rebel T6

[DeinGott]

The ROM0.MD5 is different from the actual MD5 sum. is there a problem? or can i ignore this? should i try to dump, since i get the same md5?

[a1ex]

Yes, that's why the MD5 is there. The file I/O routines from bootloader are not very robust - totally repeatable in QEMU, but not exactly deterministic on real hardware.

[DeinGott]

ok .. i totaly ignored the rom0 since i thought that is not connected .. i will try to get a correct dump .. thx for the help .. hope it works after that..

[DeinGott]

i tried it some more, but there is no way, to get it correct. Is there a "best" way to do it? I partitioned my 8gb cards to 256mb. Now the dumper do not finish at all. What are the problems, why the camera get wrong checksums?

UPDATE: I formated the SD Card fat16 and shrinked it a bit more (240mib). but still wrong md5:

Code Select Expand

user@morbo: /Volumes/Untitled% md5 ROM0.BIN && cat ROM0.MD5
MD5 (ROM0.BIN) = e913c61b9717324b2aa16f366586e081
b7bd14aa3245c539d5327434be9e0e4b  ROM0.BIN

[a1ex]

Best guess: caching issues (more details in the 80D thread).

What worked so far: dd the SD image that comes with QEMU.

You may also try emulating with the bad ROM - with some luck, it may work. Don't forget #7.

[DeinGott]

ok .. i found the problem, why the dump did not run in qemu .. after reading the forum again. i found this post (http://www.magiclantern.fm/forum/index.php?topic=17969.msg172893#msg172893)

- I've assumed there is some sort of mapping from FFFF0000 to F8010000. To run the ROM in QEMU, you will need to patch the dump like this:

Code Select Expand

dd if=ROM1.BIN of=BOOT.BIN bs=64K skip=1 count=1
dd if=BOOT.BIN of=ROM1.BIN bs=64K seek=511

After this, running in QEMU is more or less straightforward, with a small reverse engineering puzzle to solve.

now i get the gui in qemu. Thx for the help. Now we can work with that.

[DeinGott]

So i started finding stubs in the ROM. A1ex can you have a look if this offsets make sense.

Code Select Expand

/** Startup **/
NSTUB( ROMBASEADDR, firmware_entry )                        // 0xF8010000
NSTUB(0xFE0C3A24,  cstart)                               
NSTUB(0x00029898,  bzero32)                               
NSTUB(0xFE0C3AF8,  create_init_task)                       
NSTUB(0xFE1296C8,  init_task)                               
NSTUB(   0x61123,  additional_version)
NSTUB(0xFE11F394,  DryosDebugMsg)     
NSTUB(    0x38FC,  task_create) 

/** File I/O **/
NSTUB(0xFE2A43FC,  FIO_CloseFile)
NSTUB(0xFE2A53D0,  FIO_FindClose)
NSTUB(0xFE2A52F0,  FIO_FindNextEx)                     
NSTUB(0xFE2A41AC, _FIO_ReadFile)                         
NSTUB(0xFE2A425C,  FIO_SeekSkipFile)                   
NSTUB(0xFE2A434C, _FIO_WriteFile)                       
NSTUB(0xFE2A4C3C, _FIO_CreateDirectory)                   
NSTUB(0xFE2A4058, _FIO_CreateFile)                         
NSTUB(0xFE2A51FC, _FIO_FindFirstEx)                     
NSTUB(0xFE2A4578, _FIO_GetFileSize)                       
NSTUB(0xFE2A3F9C, _FIO_OpenFile)                         
NSTUB(0xFE2A4104, _FIO_RemoveFile)                       
NSTUB(0xFE2A4A74, _FIO_RenameFile)

What is the minimum stubs i need to find (and which) so i can test, if i can run ml in qemu?

I copied the 1100D folder in ml/platforms to a new 1300D and poked around in some files. I can run the code, but some stub is not correct. How can i enable an hello world only ml build? is there a tutorial, i did not find?

[a1ex]

Yes, they make sense. Some of the functions will have to be called from RAM (like task_create); if they are in the block copied at 0x1900 (-d romcpy), try that address first.

There are two kinds of hello world: the minimal target (without any ML features, but you can call any Canon code and test your stubs/consts), and CONFIG_HELLO_WORLD on the full ML codebase. You can also compile without features, follow compiler errors to see what stubs you need, then enable them one by one.

Tutorials linked earlier in this thread; QEMU docs (and QEMU itself) are actively updated.

[DeinGott]

so i managed to get code runing. i hooked up the restart.c file, and get my own code runing. But the copy of the ml code and the restart does not work. it always crashes. How do i find the BSS and therefor the RESTARTSTART value? i used the same the 600D does (which is 0x00082000/0x00C80100). is it guessing for empty space, or can i find a structure there, where i can read the address from?

[a1ex]

Have you already looked here?

Next steps: dfort's porting tutorial and the EOS M2 walkthrough.

Things changed a bit since writing the walkthrough (old methods still work, but now there's an easier way): you can now get the same info from the serial console in QEMU ("meminfo -m" at Dry-shell console). Just like M2, you can use the "classic" boot process only for minimal target and maybe for the "full" hello world; we'll have to find some other place to load ML (CONFIG_ALLOCATE_MEMORY_POOL or something else). We still need the "classic" boot process for the installer, and the minimal hello world will certainly work with it, so don't give up yet.

There is a possibly unused (not tested) free memory block here.

[DeinGott]

ok .. i am a bit further .. i can start code. but when ml tries to start the init_task i get an error:

Code Select Expand

DRYOS PANIC: Module Code = 1, Panic Code = 4

do you have a list, what the panic codes mean? i traced it down to the function sub_FEA8A450  which looks like it copies some basic structure and then checks if some checksum is correct. and then returns -1.

In the ml code it is right after the

Code Select Expand

void (*ram_cstart)(void) = (void*) &INSTR( cstart );
ram_cstart();

i verified, everything seams correctly patched ..

it is to late now, to look any further.. will continue tomorrow.

[a1ex]

More likely, you have overwritten some of the DryOS data structures. No idea what the DryOS panic codes are, but you shouldn't need them - that points to memory corruption or invalid code executed somehow.

If you have the source code online somewhere, I can check it.

[adamnock]

Digging up my notes (which I still cant find) whet my appetite. 1300D Firmware booted in QEMU.
Time to remember stuff.

[adamnock]

OK so it seems to compile minimal for the 1300D we need to find the stubs

bmp_vram_info
msleep

*dig dig dig*

[adamnock]

Thought id found them but running the minimal hello world in QEMU's just looping

PrefetchAbort
0007F158

So either ive stuffed what I found or one of the other stubs is wrong (sic)

// Note / Question
Just to be sure, im supposed to be copying the compiled autoexec.bin file to sd.img and running with boot=1 correct?
I am doing a checkpoint each test by setting boot=0 to ensure booting to GUI works as intended, just in case I stuff something else.

More:
OK im confident know there's something wrong other than incorrect msleep or bmp_vram_info stubs.
Removing them from the mix and running a copy of minimal which I would expect to simply die faults at the same point.
Meaning one of the main stubs is probably wrong. Or being called from RAM and we dont have an offset (tempted to go into work and get my laptop with my last efforts where I think I had something on that).

Taking a break and creating a public fork so nothing goes missing again.

Public Fork:
https://bitbucket.org/maugriman/magic-lantern-1300d/

Currently just the initial folder setup and currently identified stubs, with the framework copied from the 1100D

[adamnock]

*sigh* reading through the EOSM2 walkthrough a1ex linked hinted I was having the same issue with GDB not properly loading with QEMU as dfort did.

Fixed, added a breakpoint for firmware start, going to add more for the non-FIO stubs so I can try and pinpoint where is going off target.
No idea what im doing, but im having fun doing it 

[a1ex]

To debug, I recommend analyzing a memory trace, to see exactly what the binary does.

Just fixed some inconsistencies in QEMU when handling memory logging options, so make sure you upgrade QEMU to the latest commit to try the stuff below.

Recommended invocation:

Code Select Expand


. ./export_ml_syms.sh minimal/1300D
./run_canon_fw.sh 1300D,firmware="boot=1" -d calls,io,int,romr,ramw,autoexec


The logs are huge, but they let you identify all the actions of ML startup code (internal XOR check, copying blocks of code around in memory, zeroing out memory, patching Canon's startup code). I can publish a detailed analysis later if needed (with more details than in the EOS M2 post linked above). Just one trick that may be useful to narrow down such huge logs:

Code Select Expand


./run_canon_fw.sh 1300D,firmware="boot=1" -d calls,io,int,romr,ramw,autoexec |& grep -C 10 copy_and_restart


What issue did you have with gdb? 64-bit crashes, or something else? What operating system?

[DeinGott]

Hey, i am so far, that i get the ml bootup code running. but the CONFIG_ALLOCATE_MEMORY_POOL has the problem, that it copies the init code from the rom to the ram, but it is to far apart from the rom, that a normal BL does not work, to jump back to subs, it needs. I am trying to preconvent that by copying that code as well, but this is still broken.

i need to cleanup my code, to publish it ..will do that tonight.

For now my codeflow is this: (copy_and_restart() -> ram_cstart() -> my_init_task() -> init_task_patched() -> new_init_task()) this is where the problem starts.

The copy of the init task, which is patched in init_task_patched() has the wrong offsets, so it cannot jump back to rom. (but only on some functions. A thing i noticed, is if the offset is > 0x800000 it will jump to rom, if it is smaller, it will jump to the offset itself. Therefor there is a gab we cannot jump to :/ any ideas?) next step will be create a jump table next to the init function and try to jump via register jumps

[DeinGott]

btw

i found this functions stub

NSTUB(0xFE0180A8,  print_serial)
extern int print_serial(const char* s, ...);

which does print to serial.

Bitbucket is down at the moment.. so i cannot upload my code there ..

[DeinGott]

alex do you have any suggestions to this error: it comes from the relocate script, which copies init_task and createInitTask.

Code Select Expand

Fixing from FE1296C8 to FE1298AC
FE1296D0: EBFE5CDE BL FFFE5CDE => FE0C0A50
FE1296D0: !!!! can not fixup jump from 0010232C to FE0C0A50 (offset -00810639)
FE1296F4: EB00006D BL 0000006D => FE1298B0
FE129704: EBFE692E BL FFFE692E => FE0C3BC4
FE129704: !!!! can not fixup jump from 00102360 to FE0C3BC4 (offset -0080F9E9)
FE129718: EBFE6960 BL FFFE6960 => FE0C3CA0
FE129718: !!!! can not fixup jump from 00102374 to FE0C3CA0 (offset -0080F9B7)
FE12972C: EBFE6B8C BL FFFE6B8C => FE0C4564
FE12972C: !!!! can not fixup jump from 00102388 to FE0C4564 (offset -0080F78B)
FE12973C: EB0673CD BL 000673CD => FE2C6678
FE12974C: EBFE5E80 BL FFFE5E80 => FE0C1154
FE12974C: !!!! can not fixup jump from 001023A8 to FE0C1154 (offset -00810497)
FE129760: EAFE60FE B  FFFE60FE => FE0C1B60
FE129760: !!!! can not fixup jump from 001023BC to FE0C1B60 (offset -00810219)
FE129770: EB7B63AD BL 007B63AD => 0000262C
FE129780: EAFE5DF0 B  FFFE5DF0 => FE0C0F48
FE129780: !!!! can not fixup jump from 001023DC to FE0C0F48 (offset -00810527)
FE12979C: 7A697320 B  00697320 => FFB86424
FE129814: 745F7469 LD 7, 15, ' => FE1293B3: 745F7164 356 data=812014E5
FE129830: EB066FA7 BL 00066FA7 => FE2C56D4
FE129844: EB066F8F BL 00066F8F => FE2C5688
FE129854: E51F6050 LD 6, 15, 80 => FE1298AC: E51F61A0 416 data=FE884A48
FE12987C: EB066FF3 BL 00066FF3 => FE2C5850
Fixups=10231C entry=102324 free_space=8
Fixing from FE0C1B60 to FE0C1EB8
FE0C1B6C: EBFFFDB5 BL FFFFFDB5 => FE0C1248
FE0C1B6C: !!!! can not fixup jump from 00102554 to FE0C1248 (offset -008104C5)
FE0C1B70: EB015F55 BL 00015F55 => FE1198CC
FE0C1B7C: EB01961C BL 0001961C => FE1273F4
FE0C1B80: EB7D0641 BL 007D0641 => 0000348C
FE0C1B8C: EB7D090E BL 007D090E => 00003FCC
FE0C1B9C: EB7D0667 BL 007D0667 => 00003540
FE0C1BA0: EB01795A BL 0001795A => FE120110
FE0C1BBC: EB7D0300 BL 007D0300 => 000027C4
FE0C1BE4: EB7D0353 BL 007D0353 => 00002938
FE0C1C0C: EB7D03BE BL 007D03BE => 00002B0C
FE0C1C30: EB7D042B BL 007D042B => 00002CE4
FE0C1C44: EB7D081C BL 007D081C => 00003CBC
FE0C1C48: EB017DD3 BL 00017DD3 => FE12139C
FE0C1C50: EB019C21 BL 00019C21 => FE128CDC
FE0C1C60: EB017AE3 BL 00017AE3 => FE1207F4
FE0C1C64: EB01885A BL 0001885A => FE123DD4
FE0C1C6C: EB018094 BL 00018094 => FE121EC4
FE0C1C70: EB017CDB BL 00017CDB => FE120FE4
FE0C1C74: EB01897A BL 0001897A => FE124264
FE0C1C78: EB0189C2 BL 000189C2 => FE124388
FE0C1C84: EB0187D7 BL 000187D7 => FE123BE8
FE0C1C88: EB0187EA BL 000187EA => FE123C38
FE0C1C94: EB01807C BL 0001807C => FE121E8C
FE0C1CA0: EB018079 BL 00018079 => FE121E8C
FE0C1CAC: EB018076 BL 00018076 => FE121E8C
FE0C1CB8: EB018073 BL 00018073 => FE121E8C
FE0C1CC4: EB018070 BL 00018070 => FE121E8C
FE0C1CD0: EB01806D BL 0001806D => FE121E8C
FE0C1CDC: EB01806A BL 0001806A => FE121E8C
FE0C1CFC: EB01750D BL 0001750D => FE11F138
FE0C1D08: EB01767B BL 0001767B => FE11F6FC
FE0C1D10: EB7D07F7 BL 007D07F7 => 00003CF4
FE0C1D18: EBFFFC67 BL FFFFFC67 => FE0C0EBC
FE0C1D18: !!!! can not fixup jump from 00102700 to FE0C0EBC (offset -00810613)
FE0C1D34: EB017596 BL 00017596 => FE11F394
FE0C1D48: EB017591 BL 00017591 => FE11F394
FE0C1D50: EB013313 BL 00013313 => FE10E9A4
FE0C1D70: EB0047FE BL 000047FE => FE0D3D70
FE0C1D70: !!!! can not fixup jump from 00102758 to FE0D3D70 (offset -0080BA7C)
FE0C1D78: E51F4848 LD 4, 15, 0 => FE0C1538: E51F4230 560 data=000310AC
FE0C1D90: 1B01757F BL 0001757F => FE11F394
FE0C1D9C: EB00488A BL 0000488A => FE0D3FCC
FE0C1D9C: !!!! can not fixup jump from 00102784 to FE0D3FCC (offset -0080B9F0)
FE0C1DA0: EB0177FA BL 000177FA => FE11FD90
FE0C1DA4: EB017494 BL 00017494 => FE11EFFC
FE0C1DA8: EB0190C9 BL 000190C9 => FE1260D4
FE0C1DAC: EB001112 BL 00001112 => FE0C61FC
FE0C1DAC: !!!! can not fixup jump from 00102794 to FE0C61FC (offset -0080F168)
FE0C1DB0: EB019E9B BL 00019E9B => FE129824
FE0C1DCC: EB7D06CA BL 007D06CA => 000038FC
FE0C1EA4: 6B736154 BL 00736154 => FFD9A3FC
FE0C1EB0: E51F1980 LD 1, 15, . => FE0C1538: E51F1364 868 data=000310AC
Fixups=102540 entry=102548 free_space=8


I added the checker, if we can reach the RAM, which does not trigger any error.

Code Select Expand

/* relative jumps in ARM mode are +/- 32 MB */
         /* make sure we can reach anything in the ROM (some code, e.g. patchmgr, depend on this) */
         uint32_t jump_limit = (uint32_t) &_bss_end - 32 * 1024 * 1024;
         if (jump_limit > 0xFF000000 || jump_limit < 0xFC000000)
         {
             print_serial("[BOOT] warning: cannot use relative jumps to anywhere in the ROM (limit=%x)\n", jump_limit);


i will check there any further..

[a1ex]

That's ugly. You can work around simple jumps, by replacing B 0xFE001234 with LDR PC, =0xFE001234 and save that constant in the "fixups" area, but I'm not sure how to do a long call with a single instruction.

That checker assumes up to 16MB ROM, like other DIGIC 4/5 models; 1300D has a 32MB ROM.

Luckily, on DIGIC <= 5 you can patch arbitrary stuff in ROM without relocating: try the HIJACK_CACHE_HACK boot method, similar to 600D. That camera also loads ML in the AllocateMemory pool, but does not use the "classic" way to relocate the code.

(side note: for DIGIC 6 I have to do something about the patchmgr code, as it has similar issues, so that checker should probably go away soon)

[DeinGott]

i already tried that .. it starts my code but breaks with an exception in the filemgr ..

Code Select Expand

< Error Exception>
TYPE        : 4
ISR         : 0
TASK IDSR   : 1318396
TASK Name   : FileMgr
R 0         : 6cfe0c08
R 1         : 84fe0c08
R 2         : b0fe0c08
R 3         : cc000004
R 4         : 34fe0c08
R 5         : 4c0010b0
R 6         : 10b0
R 7         : 0
R 8         : 0
R 9         : 0
R10         : 0
R11         : 0
R12         : 0
R13         : 4f4ac
R14         : 0
PC          : 0
CPSR        : c8100008


qemu: fatal: Trying to execute code outside RAM or ROM at 0xe59ff010

i am investigating that .. lets see, where this path breaks..

[DeinGott]

so fixed that backup code thingie ..

[DeinGott]

all i had to do, was fix the offsets. no additional patching required .. here is my ml/platform/1300D.110 folder.

https://www.ultrachaos.de/share/1300D.110/

I basicaly copied the 600 to the 1300. in the stubs.S file i indented every old offset by one space. so i can see the old offsets when i search for new ones (i have the code for the 600 as well, so it is easyer to spott the stubs)

will work there further to find all the stubs .. and fix some internals.h and consts.h. This code should run as is in qemu did not try on an actual camera as of now, because mine does not have the bootflag set yet. Will look into that later.. Try to get the hello world running.

[adamnock]

Mein Gott DeinGott (sorry I had to), thats a leap forward.

Ive copied your work into the repo and also made some adjustments to get CONFIG_HELLO_WORLD to compile (fps-engio and raw had some platform-specific requirements)
Hello World now builds, and autoexec.bin is loaded as you had, but Hello World does not execute.

I grabbed the QEMU output for a run and popped it up here
<REMOVED DUE TO MERGE WITH MAIN REPO>
if anyone wants to have a sticky. File is qemu-bootlog-hw.txt