Magic Lantern Forum

Developing Magic Lantern => General Development Discussion => Topic started by: the12354 on October 03, 2016, 11:51:34 AM

Title: Canon EOS 1300D / Rebel T6
Post by: the12354 on October 03, 2016, 11:51:34 AM
Hi,
i'm a coder/immediate re who just bought a EOS 1300D and would like to port magic lantern to it.
I've read around the forum and the first step for porting is dumping the firmware. I've tried the portable rom dumper but unfortunately nothing happens(black screen, camera needs to be reset using the battery).
Another way i've seen is using specifically crafted .fir files.
What do i need to provide to get a .fir dumper for this camera from you?
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on October 03, 2016, 01:59:13 PM
Try this one (not a ROM dumper, but should print some info on the screen):

http://www.magiclantern.fm/forum/index.php?topic=17714

What file did you run on 1300D? I don't remember publishing a ROM dumper for this camera yet...
Title: Re: Canon EOS 1300D / Rebel T6
Post by: the12354 on October 03, 2016, 03:48:48 PM
Code: [Select]
CHDK CPU info for 0x0 ERROR
-----------------------
ID 0x41059461
Revision 0x1 1
Part 0x946 2374
ARM Arch 0x5 5
Variant 0x0 0
Implementor 0x41 65

Cache type 0x0F112112
Icache words/line 0x2 2 [8]
Icache absent 0x0 0
Icache assoc 0x2 2
Icache size 0x4 4 [8K]
Reserved0_2 0x0 0
Dcache words/line 0x2 2 [8]
Dcache absent 0x0 0
Dcache assoc 0x2 2
Dcache size 0x4 4 [8K]
Reserved1_2 0x0 0
Harvard/unified 0x1 1
Cache type 0x7 7
Reserved2_3 0x0 0
TCM type 0x000C00C0
Reserved0_2 0x0 0
ITCM absent 0x0 0
Reserved1_3 0x0 0
ITCM size 0x3 3 [4K]
Reserved2_4 0x0 0
DTCM absent 0x0 0
Reserved3_2 0x0 0
DTCM size 0x3 3 [4K]
Reserved4_10 0x0 0
Control 0x0005107D
Protect enable 0x1 1
Reserved0_1 0x0 0
Dcache enable 0x1 1
Reserved1_4 0xF 15
Big endian 0x0 0
Reserved2_4 0x0 0
Icache enable 0x1 1
Alt vector 0x0 0
Cache RRR 0x0 0
Disble loadTBIT 0x0 0
DTCM enable 0x1 1
DTCM mode 0x0 0
ITCM enable 0x1 1
ITCM mode 0x0 0
Reserved3_12 0x0 0
Protection Region 0 0x0000003F
Enable 0x1 1
Size 0x1F 31 [4G]
Undef0_7 0x0 0
Base 0x0 0 [0x00000000]
Protection Region 1 0x0000003D
Enable 0x1 1
Size 0x1E 30 [2G]
Undef0_7 0x0 0
Base 0x0 0 [0x00000000]
Protection Region 2 0x00000037
Enable 0x1 1
Size 0x1B 27 [256M]
Undef0_7 0x0 0
Base 0x0 0 [0x000000000]
Protection Region 3 0xC0000039
Enable 0x1 1
Size 0x1C 28 [512M]
Undef0_7 0x0 0
Base 0x60000 393216 [0xC0000000]
Protection Region 4 0xF8000031
Enable 0x1 1
Size 0x18 24 [32M]
Undef0_8 0x0 0
Base 0x7C000 507904 [0xF8000000]
Protection Region 5 0xFE000031
Enable 0x1 1
Size 0x18 24 [32M]
Undef0_7 0x0 0
Base 0x7F000 520192 [0xFE000000]
Protection Region 6 0x00000000
Enable 0x0 0
Size 0x0 0 [invalid]
Undef0_7 0x0 0
Base 0x0 0 [00000000]
Protection Region 7 0x00000000
Enable 0x0 0
Size 0x0 0 [invalid]
Undef0_7 0x0 0
Base 0x0 0 [00000000]
Region data perms 0x00333333
Region 0 0x3 3 [P:RW U:RW]
Region 1 0x3 3 [P:RW U:RW]
Region 2 0x3 3 [P:RW U:RW]
Region 3 0x3 3 [P:RW U:RW]
Region 4 0x3 3 [P:RW U:RW]
Region 5 0x3 3 [P:RW U:RW]
Region 6 0x0 0 [P:-- U:--]
Region 7 0x0 0 [P:-- U:--]
Region inst perms 0x00333333
Region 0 0x3 3 [P:RW U:RW]
Region 1 0x3 3 [P:RW U:RW]
Region 2 0x3 3 [P:RW U:RW]
Region 3 0x3 3 [P:RW U:RW]
Region 4 0x3 3 [P:RW U:RW]
Region 5 0x3 3 [P:RW U:RW]
Region 6 0x0 0 [P:-- U:--]
Region 7 0x0 0 [P:-- U:--]
DCache cfg 0x00000024
Region 0 0x0 0
Region 1 0x0 0
Region 2 0x1 1
Region 3 0x0 0
Region 4 0x0 0
Region 5 0x1 1
Region 6 0x0 0
Region 7 0x0 0
ICache cfg 0x00000024
Region 0 0x0 0
Region 1 0x0 0
Region 2 0x1 1
Region 3 0x0 0
Region 4 0x0 0
Region 5 0x1 1
Region 6 0x0 0
Region 7 0x0 0
Write buffer 0x00000024
Region 0 0x0 0
Region 1 0x0 0
Region 2 0x1 1
Region 3 0x0 0
Region 4 0x0 0
Region 5 0x1 1
Region 6 0x0 0
Region 7 0x0 0
DTCM cfg 0x40000006
Reserved0_1 0x0 0
Size 0x3 3 [4K]
Undef0_7 0x0 0
Base 0x20000 131072 [0x40000000]
ITCM cfg 0x00000006
Reserved0_1 0x0 0
Size 0x3 3 [4K]
Undef0_7 0x0 0
Base 0x0 0 [0x00000000]

Here are the images i took(with postprocessing for readability) for reference:
http://imgur.com/a/OIqck

I've used this one (http://www.magiclantern.fm/forum/index.php?topic=16534.0) but i guess it's only for cameras where ML is already installed?
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on October 03, 2016, 05:24:45 PM
You mean, autoexec.bin? How did you manage to lock up the camera without enabling the boot flag first?!

Anyway, here's the portable ROM dumper: DUMP1300.FIR (http://a1ex.magiclantern.fm/debug/portable-rom-dumper/DUMP1300.FIR)

If successful, please send me the ROM by PM.

The info looks fairly similar to digic 4; the two 32MB ROMs are a bit unusual. RAM seems to be 256M.

Your first task is to run your ROM under QEMU (same for anyone else interested). Without seeing the firmware, I expect:
- loading autoexec from SD card should work with little or no tweaking (it may lock up at some GPIO registers, easy to fix)
- the portable display test should also run with minimal effort
- if you run it under GDB, you should also see a few tasks starting
- if you are lucky, you might even see Canon GUI (but don't get your hopes too high on this one).
Title: Re: Canon EOS 1300D / Rebel T6
Post by: the12354 on October 03, 2016, 06:55:53 PM
Thanks for the dumper.
Unfortunately it does not seem to dump anything. Nothing changed on the SD Card.
It looks like it freezes after saying "Dumping ROM0..." (i reset the camera after 1 hour).


This is the full log i get:
Code: [Select]
Magic Lantern Rescue
--------------------------
- Model ID: 0x0 ERROR
- Camera model: ???
- Firmware version: ??? / ???
- IMG naming: 100?????/????0000.JPG
- Artist: ???
- Copyright: ???
- Boot flags: FIR=0 BOOT=0 RAM=-1 UPD=-1
- Init SD... (101F64)
- Dumping ROM0...
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on October 03, 2016, 07:29:56 PM
You may have better luck with a smaller card, or maybe even with a card formatted at a smaller capacity. For me, this tool works best on an old 256 MB card.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: the12354 on October 03, 2016, 08:21:30 PM
Thanks, resizing the sd card to 256MB worked.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on October 03, 2016, 11:51:56 PM
ROM layout is a little unusual:

- The two ROMs at F8000000 and FE000000 are identical, so it probably has a ROM chip at F8000000, mirrored as usual until FFFFFFFF (4 copies x 32MB). We call this one ROM1.
- There seems to be another 32MB ROM chip at F0000000 (ROM0).
- Bootloader appears to be at F8010000, but the first instruction jumps to FFFF0040. Code at F8010040 looks valid. The ARM946 can start from either 0 (unlikely, that's the RAM) or FFFF0000 (HIVECS configuration). However, the ROM dump after FFFF0000 is... empty!
- I've assumed there is some sort of mapping from FFFF0000 to F8010000. To run the ROM in QEMU, you will need to patch the dump like this:

Code: [Select]
dd if=ROM1.BIN of=BOOT.BIN bs=64K skip=1 count=1
dd if=BOOT.BIN of=ROM1.BIN bs=64K seek=511

After this, running in QEMU is more or less straightforward, with a small reverse engineering puzzle to solve.

Have fun!
Title: Re: Canon EOS 1300D / Rebel T6
Post by: Rongronggg9 on January 14, 2017, 07:11:30 PM
(http://ww4.sinaimg.cn/mw600/d46786adjw1fbqoy32a49j21kw0w0nfn.jpg)
256M SD Card, FAT format
It took 10min to dump.


But without other compatible files, I can't find any differences...
With 1100D files, there's still no difference...
(Maybe I've said something useless..)
_(:зゝ∠)_


I am a high school student from China, so...
There's something I can't understand very well.

How to patch the dump?

I've managed to search for it but I can't find anything useful.
Maybe I am too stupid...
(>д<)
(I apologize for not being word-perfect in English...)
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 21, 2017, 08:48:25 AM
Right im resurrecting this only slightly cooled off thread because its right what I need

Dumped ROM - Success. Did it twice and compared, good dumps I assume as they were identical.
Patched ROM as per above instructions.

Compiled QEMU and added Machine Rego in eos.c for the 1300D.
HOWEVER. I dont actually have any clue what the register address in the source is supposed to be targeting. I set it to FF801000 which is noted above as being the bootloader position, and got some minor output suggesting some code was executed, but it stalled after a few shifts, so im thinking im in the wrong boot position. But honestly, I only have a small idea of what im doing here, just an honest interest in figuring it out.

Any suggestions from the almightly userbase?

Possible Progress?
I tried to figure out the offset from the ROM, and came up with 0xF8008000 based on the above patch to ROM1.
Lo an behold there was some execution and what looks like now idle output on the console. No picture though.

[EOS] loading 'ROM-1300D.BIN' to 0xF0000000-0xF1FFFFFF
[EOS] loading 'ROM-1300D.BIN' to 0xF8000000-0xF9FFFFFF
[???] [0xC7C287C0] -> [0xC7C287C0] PC: 0xF80277A0
[???] [0x00000000] -> [0xC7C287C4] PC: 0xF80277A0
[???] [0xC7C287C0] -> [0xC7C287C0] PC: 0xF80277A0
[???] [0x00000000] -> [0xC7C287C4] PC: 0xF80277A0
[???] [0xCFC00FD8] -> [0xCFC00FD8] PC: 0xF80277A0
[???] [0x00000000] -> [0xCFC00FDC] PC: 0xF80277A0
[???] [0xCFC00FD8] -> [0xCFC00FD8] PC: 0xF80277A0
[???] [0x00000000] -> [0xCFC00FDC] PC: 0xF80277A0
[???] [0xCFC00FD8] -> [0xCFC00FD8] PC: 0xF80277A0
[???] [0x00000000] -> [0xCFC00FDC] PC: 0xF80277A0
[???] [0xCFC00FD8] -> [0xCFC00FD8] PC: 0xF80277A0
[???] [0x00000000] -> [0xCFC00FDC] PC: 0xF80277A0
[???] [0xCFC00FD8] -> [0xCFC00FD8] PC: 0xF80277A0
[???] [0x00000000] -> [0xCFC00FDC] PC: 0xF80277A0
[???] [0xCFC00FD8] -> [0xCFC00FD8] PC: 0xF80277A0
[???] [0x00000000] -> [0xCFC00FDC] PC: 0xF80277A0
[???] [0xCFC00FD8] -> [0xCFC00FD8] PC: 0xF80277A0
[???] [0x00000000] -> [0xCFC00FDC] PC: 0xF80277A0
[???] [0xCFC00FD8] -> [0xCFC00FD8] PC: 0xF8027800
[???] [0x00000000] -> [0xCFC00FDC] PC: 0xF8027800
[???] [0xC7B18BA0] -> [0xC7B18BA0] PC: 0xF80325D0
[???] [0x00000000] -> [0xC7B18BA4] PC: 0xF80325D0
[???] [0xC7B18BA0] -> [0xC7B18BA0] PC: 0xF80325D0
[???] [0x00000000] -> [0xC7B18BA4] PC: 0xF80325D0
[???] [0xC7B18BA0] -> [0xC7B18BA0] PC: 0xF80325D0
[???] [0x00000000] -> [0xC7B18BA4] PC: 0xF80325D0
[???] [0xC7B18BA0] -> [0xC7B18BA0] PC: 0xF80325D0
[???] [0x00000000] -> [0xC7B18BA4] PC: 0xF80325D0
[???] [0xC7B18BA0] -> [0xC7B18BA0] PC: 0xF80325D0
[???] [0x00000000] -> [0xC7B18BA4] PC: 0xF80325D0
[???] [0xCFAE6594] -> [0xCFAE6594] PC: 0xF80325D0
[???] [0x00000000] -> [0xCFAE6598] PC: 0xF80325D0
[???] [0xCFAE6594] -> [0xCFAE6594] PC: 0xF80325D0
[???] [0x00000000] -> [0xCFAE6598] PC: 0xF80325D0
[???] [0xCFAE6594] -> [0xCFAE6594] PC: 0xF80325D0
[???] [0x00000000] -> [0xCFAE6598] PC: 0xF80325D0
[???] [0xC7EAFD58] -> [0xC7EAFD58] PC: 0xF8032878
[???] [0x00000000] -> [0xC7EAFD5C] PC: 0xF8032878
[???] [0xC7EAFD58] -> [0xC7EAFD58] PC: 0xF8032878
[???] [0x00000000] -> [0xC7EAFD5C] PC: 0xF8032878
[???] [0xC7EAFD58] -> [0xC7EAFD58] PC: 0xF8032878
[???] [0x00000000] -> [0xC7EAFD5C] PC: 0xF8032878
[???] [0xC7EAFD58] -> [0xC7EAFD58] PC: 0xF8032878
[???] [0x00000000] -> [0xC7EAFD5C] PC: 0xF8032878
[???] [0xCFE7D4C0] -> [0xCFE7D4C0] PC: 0xF8032878
[???] [0x00000000] -> [0xCFE7D4C4] PC: 0xF8032878
[???] [0x00000040] -> [0xCCFFF340] PC: 0x07C008F4
[???] [0x00000040] -> [0xCCFFF340] PC: 0x07C008F4
[???] [0x00000040] -> [0xCCFFF340] PC: 0x07C008F4
[???] [0x00000040] -> [0xCCFFF340] PC: 0x07C008F4
[???] [0x00000040] -> [0xCCFFF340] PC: 0x07C008F4
[???] [0x00000040] -> [0xCCFFF340] PC: 0x07C008F4

next step I guess is to attach gdb and try and figure out whats actually going on?
Also to figure out what the hell im doing.

EDIT2: OK So it helps if im running qemu using the patches in the current branch, not some old stuff. Whoops.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 22, 2017, 05:31:08 AM
OK, having checked out the correct branch for the current QEMU build, ive created a machine profile for the 1300D using values provided above for RAM size, ROM size and locations etc.

Starting to see some possible results:
FIXME: no MPU spells for 1300D.
FIXME: no MPU button codes for 1300D.
FFFF0AE0: MCR p15,0,Rd,cr6,cr0,0:  946_PRBS0 <- 0x3F       (00000000 - FFFFFFFF, 0x100000000)
FFFF0AE8: MCR p15,0,Rd,cr6,cr1,0:  946_PRBS1 <- 0x3D       (00000000 - 7FFFFFFF, 0x80000000)
FFFF0AF0: MCR p15,0,Rd,cr6,cr2,0:  946_PRBS2 <- 0x37       (00000000 - 0FFFFFFF, 0x10000000)
FFFF0AF8: MCR p15,0,Rd,cr6,cr3,0:  946_PRBS3 <- 0xC0000039 (C0000000 - DFFFFFFF, 0x20000000)
FFFF0B00: MCR p15,0,Rd,cr6,cr4,0:  946_PRBS4 <- 0xF8000031 (F8000000 - F9FFFFFF, 0x2000000)
FFFF0B08: MCR p15,0,Rd,cr6,cr5,0:  946_PRBS5 <- 0xFE000031 (FE000000 - FFFFFFFF, 0x2000000)
FFFF0B10: MCR p15,0,Rd,cr2,cr0,0: DCACHE_CFG <- 0x24       
FFFF0B18: MCR p15,0,Rd,cr3,cr0,0:       DACR <- 0x24       
FFFF0B1C: MCR p15,0,Rd,cr2,cr0,1: ICACHE_CFG <- 0x24       
FFFF0B20: MCR p15,0,Rd,cr5,cr0,0:    DATA_AP <- 0xFFF     
FFFF0B28: MCR p15,0,Rd,cr5,cr0,1:    INSN_AP <- 0xFFF     
FFFF0B2C: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x2078
FFFF0B2C: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC000307D
FFFF00C4: MCR p15,0,Rd,cr9,cr1,1: XSCALE_UNLOCK_ICACHE <- 0x6        (00000000 - 00000FFF, 0x1000)
FFFF00C4: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC000307D
FFFF00C4: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC004307D
FFFF00C4: MCR p15,0,Rd,cr9,cr1,0: XSCALE_LOCK_ICACHE_LINE <- 0x40000006 (40000000 - 40000FFF, 0x1000)
FFFF00C4: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC004307D
FFFF00C4: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005307D
FFFF0108: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005307D
FFFF0108: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
System & Display Check & Adjustment program has started.

And then it hangs

Ill happily admit though that at this point im fairly lost, but ill do some reading to try and keep moving.

In the meantime if anyone can offer some suggestions, ive uploaded exec,int output and a function trace, plus the profile details to
https://drive.google.com/drive/folders/0B6Jkvpb0IV-zRkk0YWxLTXpuc0U?usp=sharing

Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on April 22, 2017, 09:11:22 AM
Looks good. Next step is to prevent the adjustment menu from coming up (and launch the main firmware instead).

You can also use -d io (or -d exec,int,io) to get some more info about what happens, and you may find it helpful following the code branches in IDA (e.g. press space in the disassembly tab). Additionally, -singlestep is useful for getting correct program counters in the io logs (otherwise, you'll often get the start of a small function, rather than the exact address where the MMIO access happens). You'll have to configure the emulator in a way that "forces" the boot code to pick the FROMUTILITY path, instead of the System adjustment menu.

The place where the code path should be changed is not the same place where it locks up (that's a bit tricky). However, all the functionality of the "guest" program (here, the firmware) can be changed from MMIO registers and/or triggering an interrupt (you only need the former method here).

MMIO registers and hardware interrupts are the only external interfaces of this CPU to other devices, as far as I could tell. MMIO registers cover GPIOs, interrupt controller(s?), DMA controllers, communication with other CPU cores, image processing modules, I2C, SPI, UART and so on. In our implementation, all of them are covered in the eos_handle_* functions (which looks a bit different from other QEMU code, as they were ported from another emulator (https://www.magiclantern.fm/forum/index.php?topic=2882.0), back in the old days).

An interrupt can be triggered whenever an external device does something interesting (here's an example (http://www.magiclantern.fm/forum/index.php?topic=2388.msg183168#msg183168)).
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 22, 2017, 09:59:24 AM
Thanks a1ex! Nice to meet you btw.

That makes sense. Ill have a look through the code at how such a boot shunt (term?) is achieved to skip Adjustment on another device so I can see what im looking at. Nitty gritty time.

:)
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 22, 2017, 10:57:52 AM
Right, well from looking at the io logging (thanks for the tip) there was only 2 unique GPIO reads occuring, suggesting I could do this by trial and error. The first caused no boot execution, going to assume that was the wrong one or wrong value :P

The second, which the GPIO handler has annotated as maybe being SD Detect for the 70D and 6D, proved more valuable.
Replacing the output of that overwrite with a 0 value skipped the SDAC and moved ahead. Whoopie!

The process seems to now move a lot further ahead, and positively is now halting with an Assert.

So! Its time for me to setup IDA i think.

Note: Ive uploaded new IO, EXEC and Calls outputs to the Google drive share
https://drive.google.com/drive/folders/0B6Jkvpb0IV-zRkk0YWxLTXpuc0U?usp=sharing

Plus updated the notes document to include the modified eos.c code.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on April 22, 2017, 11:45:41 AM
The second, which the GPIO handler has annotated as maybe being SD Detect for the 70D and 6D, proved more valuable.
Replacing the output of that overwrite with a 0 value skipped the SDAC and moved ahead. Whoopie!

Yep, that's the one.

Not sure what's causing the assert (didn't look much into it yet, other than noticing it depends on the output values given by sub_27C4, which is copied to RAM right before cstart - a process done on other DIGIC 5 and 6 cameras). The 1300D appears to have a few bits from the newer codebases backported on DIGIC 4.

Debugging in IDA may help:

Code: [Select]
./run_canon_fw.sh 1300D -d io -singlestep -s -S

followed by F9 in IDA.

Some useful functions:
Code: [Select]
FE0C0000 main firmware start
FE0C3A28 cstart
FE1279E8 init_task
FE0C1B60 AJ_massive_kernel_init

It also helps extracting the memory blocks copied to RAM and loading them in IDA at the copied location as additional binary files. The functions copied there will be executed from RAM.

Not sure how much it helps, but last night I've added other QEMU startup logs (from other camera models) here:
http://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-dm-spy/
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 22, 2017, 12:24:22 PM
Thanks again!

Was there a good forum post or wiki article on setting up IDA? I havent had the opportunity to use it before, and so far ive not found anything on setup, or which version to use etc etc.

Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 22, 2017, 12:39:30 PM
Ah ok, the eval version of IDA Pro doesnt support GDB remote connections.
I get it now
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 23, 2017, 06:18:51 AM
Not having much luck with IDA, very new to me, but learnings 90% of the fun.

On the flip side, I did try running the 1300D using the boot flag and got the portable display test and recovery screen. From above it seems thats expected and a good sign, so plus 1 I guess :).
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on April 23, 2017, 10:36:53 AM
Sounds like you are on the right track. The assert puzzle appears harder than I've expected - I've tried to debug it yesterday and here's what I found:

The tricky subroutine is called like this:
Code: [Select]
sub_27C4(0xF8000000, &out1, &out2, &out3);

0xF8000000 is the address used for boot flags (http://magiclantern.wikia.com/wiki/Bootflags). This suggests the 3 output values are probably read from there.

The next checks (before the assert) appear to accept the following values for out1-3: C2, 25, 39 or 20 BB 19 or 01 02 19 (hex). Each of these sequences is handled with a different subroutine: 2938 / 2B0C / 2CE4; they all allocate memory, fill in some round values and call FE2B486C (which is complicated).

Inside 27C4, there are a couple of functions that call others indirectly (BX R1); these are easy to get by running the debugger and finding the value of R1 (or PC after the call); understanding where this code takes these values from is a lot harder. Here's how to debug with GDB:

Code: [Select]
./run_canon_fw.sh 1300D -d exec,io -singlestep -s -S

then you need a debugmsg.gdb file:
Code: [Select]
source -v debug-logging.gdb

b *0xFE0C1B60
b *0x27C4

continue

then, in another terminal:
Code: [Select]
arm-none-eabi-gdb -x 1300D/debugmsg.gdb

then:
Code: [Select]
(gdb) layout asm
(gdb) layout regs
(gdb) si

If you want to jump over a function, gdb may complain ("Cannot find bounds of current function"); here's a workaround:
Code: [Select]
B+ │0x27ec  mov    r0, r6
   │0x27f0  bl     0x6e50
   │0x27f4  mov    r3, r8
Code: [Select]
(gdb) tbreak *0x27f4
(gdb) c

Now, our function returns 06 00 00 (instead of one of the accepted sequences). Where does that come from?!
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 23, 2017, 01:02:33 PM
Brilliant, I actually understood most of that.
You wouldnt teach Comp-Sci would you? :)

Ill start looking around at that point. Maybe attack it the rock and hammer method and throw values at it and see what happens.

To horse!
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 23, 2017, 02:10:24 PM
right, I see what you mean about 0xF8000000 changing unexpectedly.
And I figured out how to poke registers so thats another step down.

Eyes are crossing now, more tomorrow!
Thanks and have a good week.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on April 23, 2017, 08:01:59 PM
My hypothesis is that it might be trying to get some sort of manufacturer ID of the flash ROM chip.

See for example the K8P2815UQB (http://www.bdtic.com/DataSheet/SAMSUNG/K8P2815UQB.pdf) datasheet (used in 7D, according to this page (http://magiclantern.wikia.com/wiki/Datasheets)). Here's the I/O and ROM activity for the 7D, when trying to change the boot flag from the FROMUTILITY menu (Serial Console in QEMU window):

Code: [Select]
Is flg written(Y=ON(0xFFFFFFFF)/N=OFF(0x00000000))? :y
[FlashIF]  at 0x00102164:001021B8 [0xC0000000] -> 0x0       : ???
[FlashIF]  at 0x0010216C:001021B8 [0xC0000000] <- 0x1000000 : ???
[FlashIF]  at 0x00102178:001021B8 [0xC0000010] <- 0xD9C50000: 'Write enable' enabled
ROM(0xf8000aaa) = 0xaa (ignored)
ROM(0xf8000554) = 0x55 (ignored)
ROM(0xf8000aaa) = 0x80 (ignored)
ROM(0xf8000aaa) = 0xaa (ignored)
ROM(0xf8000554) = 0x55 (ignored)
ROM(0xf8000000) = 0x30 (ignored)
ROM(0xf8000000) => 0x0
ROM(0xf8000000) => 0x0
... (infinite loop)

This looks similar (but not identical) to the Block Erase sequence. Probably the chip is some related model (not exactly the one listed on the wiki page).

On 1300D, the following ROM accesses are made since "K404 READY":
Code: [Select]
K404 READY
ROM(0xf9000000) => 0x0
ROM(0xf8000000) => 0x0
ROM(0xf8000000) <= 0x6 (ignored)
ROM(0xf9000000) => 0x0
ROM(0xf8000000) => 0x0
ROM(0xf9000000) => 0x0
ROM(0xf8000000) => 0x0
ROM(0xf9000001) => 0x0
ROM(0xf8000001) => 0x0
ROM(0xf9000002) => 0x0
ROM(0xf8000002) => 0x0
ROM(0xf9000000) => 0x0
ROM(0xf8000000) => 0x0
ROM(0xf9000000) => 0x0
ROM(0xf8000000) => 0x0
ROM(0xf9000000) => 0x0
ROM(0xf8000000) => 0x0

Assert: File ./Startup/Startup.c Line 220

So, my best guess is that we should model this copy of the ROM as I/O memory and fake the data somehow.

Note: in QEMU, it's generally not possible to log every single memory access, unless that memory block is configured as I/O. However, memory implemented as I/O cannot contain executable code (so we have to choose one).

Side note: I'm currently looking at Panda (a fork of QEMU), which promises the ability to log any memory access, and a lot more useful analyses (look at plugins in their manual, for example).

https://github.com/moyix/panda/blob/master/docs/manual.md
http://moyix.blogspot.com/2013/09/announcing-panda-platform-for.html
https://gist.github.com/bridgeythegeek/d7a6c449287c6e32187be2639a7920bf
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 04:36:21 AM
So then assuming following your hypothesis, we should see a somewhat related compare between the K404 Ready and the Asset.

Modelling the ROM as IO has just gone over my head complete, so assuming I cant figure that out (yet, im learning) I might continue trying to figure out what might be being mishandled to result in the Assert call.

Also I might try and find some high-res scans of the 1300D motherboard or similar to identify the exact Flash used.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 08:06:33 AM
Hmm, all I can confirm from images (found some OKish on ebay) is:

CPU = Toshiba TMP19A43?DXBG (? appears to be an E. Wiki for 50D has same but with an F, trying to find datasheet to confirm significance of this part string letter)
RAM = 2x ELPIDA E1116A(5/8)E-P, 64MBx16 1GB DDR2-(667/800) (667 for a 5, 800 for a 8)

Other IC's =  Princeton PT6590 LED Matrix Encoder (suppose driving the in-viewfinder data display)
Image Processor = DIGIC4+ (from documentation of course)

All the other IC's are either too small to read on the medium res photo, or covered by a shield, sadly the Flash appears it might be included, but that MIGHT be the LCD Processor given its placement.

Going to dig up the datasheet on the CPU to find out a bit more.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 08:09:38 AM
E part for CPU doesnt exist, so its a

http://datasheet.octopart.com/TMP19A43FDXBG-Toshiba-datasheet-13724305.pdf

Same as the 70D (at least)
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on April 24, 2017, 09:42:23 AM
Modelling the ROM as IO has just gone over my head complete, so assuming I cant figure that out (yet, im learning) I might continue trying to figure out what might be being mishandled to result in the Assert call.

Yes, that was pretty difficult, as this one requires detailed knowledge of how ROM is configured, and how to do that in QEMU. I think I've figured it out last night, but had to change the ROM layout on all models. This approach appears to break other functionality (e.g. 60D no longer boots), but also gives interesting insights on how ROM reflashing is done (and allows one to implement its emulation, since the ROM addresses appear to behave like I/O during this process):

qemu-1300D.patch (http://a1ex.magiclantern.fm/bleeding-edge/1300D/qemu-1300D.patch)

A simpler approach would have been to patch the ROM manually (hardcoding the flash model ID at those addresses where the firmware expects it). Unfortunately, that appears to lock up the bootloader.

A third approach would be to patch the affected function in GDB (see e.g. 700D - patches.gdb) or in ROM (see DIGIC 6 models, but that would remove the ability to run unmodified autoexec.bin's later, since they do a checksum of the ROM at startup to ensure correct firmware version).

Anyway - currently it starts a few tasks, so you can apply the patch and start identifying stubs. Some of them are useful for debugmsg.gdb as well (e.g. DebugMsg, task_create). The current state is also enough for testing the boot process (to see whether ML is able to run code alongside the main Canon firmware, reserve memory for itself, start a task and so on). It won't display any GUI yet, but it shouldn't be hard to reach the Hello World stage without this functionality.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 11:26:12 AM
I am actually floored by how quickly you got this done. I get you know what you are doing, but damn you are committed for an open source project.

RIGHT, brown-nosing over.

Finding the stubs seems to be an accomplishable goal I can do, i found the relevant forum threads and I mostly get the asm command set now, at least typologically, so ill get stuck into that.
I might use the 60D as a base reference for the stubs as it seems to be the most related hardware wise.

This is fun!
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 02:15:17 PM
Just a question on starting a new platform definition in the ML source.
Is there a base set of source for the platform that can be used that has a minimal feature/module set enabled?

Ive tried copying the 60D set, and stripping down to minimal components, but im getting feature define compile time failures such as CAM_COLORMATRIX1 and RAW_ZEBRA_ENABLE. I could go through one at a time and fix them, but it seems im working backwards.

Basically I want to start a 1300D platform definition which can really just execute the hello world example, which I believe is what you meant as well?
That and its the only certain way to confirm the found stubs i believe?
Title: Re: Canon EOS 1300D / Rebel T6
Post by: nikfreak on April 24, 2017, 02:28:39 PM
basically, what I did when starting to port was taking a copy of another camera and rename it.
For e.g. you take 60D or 600D for Digic IV.

Afterwards you rename it and do it step by step as you already guessed.
First you grep for "CONFIG_60D" and then "60D" and then "60d" and you should find almost everything needed. In internals.h you undefine this:

Code: [Select]
#define CONFIG_PROP_REQUEST_CHANGE
and define CONFIG_HELLO_WORLD.

let me know if you get stuck or need help at finding a stub.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on April 24, 2017, 02:37:34 PM
Besides nikfreak's advice, here are some useful tips:

Example for compiling without features: 60D-dm-spy.patch (https://bitbucket.org/hudson/magic-lantern/src/8b7912bd493317c9bd47f6ec659c05e661017dc4/platform/60D.111/60D-dm-spy.patch?fileviewer=file-view-default)

(I know, they are not isolated very well, as we don't turn them off very often...)

You can also use the minimal target, but that one is really minimal (useful for a lower-level version of Hello World). It uses the platform-specific files (stubs, consts) from the platform directory, a single source file for experiments (minimal.c) and a tiny graphics library (font_direct.c) - besides the loader code in reboot.c. Therefore, it's a good playground environment that does not touch the larger codebase (and does not require a lot of stubs/consts to get started).

The digic6-dumper branch also makes use of the minimal target, but a different way (using a platform-specific minimal.c - because the current boot process has to be changed significantly for newer models). I hope it's not needed for 1300D.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 02:59:23 PM
Hi nikfreak, thanks for the answer, and thanks a1ex again.

Thats what ive been trying, but ive flushed my work and started fresh in case I royally stuffed something.

Basically ive copied to 60D definition, replaced all the specifics, undefined CONFIG_PROP_REQUEST_CHANGE as recommended, and added the 1300D (firmware 110) to the main Makefile and Makefile.platform.map

However with this minimal set, im seeing compile errors eg
../../src/focus.c:1024:33: error: 'GUIMODE_FOCUS_MODE' undeclared (first use in this function)

Whats confusing me is this isnt defined for the 60D either, but that compiles fine.
A walk through the source shows its typically defined in the platform consts

[email protected]:~/magic-lantern$ grep -rnw './' -e "GUIMODE_FOCUS_MODE"
./platform/1100D.105/consts.h:56:#define GUIMODE_FOCUS_MODE 9
./platform/600D.102/consts.h:112: #define GUIMODE_FOCUS_MODE 9
./platform/550D.109/consts.h:121:#define GUIMODE_FOCUS_MODE 9
./platform/700D.114/consts.h:89:    #define GUIMODE_FOCUS_MODE 0x123456
./platform/7D.203/consts.h:123:#define GUIMODE_FOCUS_MODE 9
./platform/6D.116/consts.h:128:#define GUIMODE_FOCUS_MODE 0x123456
./platform/5D3.123/consts.h:115:#define GUIMODE_FOCUS_MODE 0x123456
./platform/EOSM.202/consts.h:89:#define GUIMODE_FOCUS_MODE 0x123456
./platform/5D3.113/consts.h:100:#define GUIMODE_FOCUS_MODE 0x123456
./platform/unmaintained/40D.111/consts.h:71:#define GUIMODE_FOCUS_MODE 1234
./platform/unmaintained/5DC.111/consts.h:79:#define GUIMODE_FOCUS_MODE 12345
./platform/650D.104/consts.h:89:    #define GUIMODE_FOCUS_MODE 0x123456
./platform/5D2.212/consts.h:87:#define GUIMODE_FOCUS_MODE 9
./platform/500D.111/consts.h:91:#define GUIMODE_FOCUS_MODE 0x27
./platform/50D.109/consts.h:117:#define GUIMODE_FOCUS_MODE 9

Am I missing something obvious here? Should I just define these constants simply for testing compile, or what?
Ill keep trying to figure out what im missing :)
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 03:03:02 PM
AH, I get it.

Right, so there's some platform specific tweaks laying around in the primary project code.

OK, thats fine, ill start digging there.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on April 24, 2017, 03:12:36 PM
Right, 60D has an exception exactly for this constant :D

Other cameras have dummy definitions (those 0x123456), so anything that checks if the current GUI mode is GUIMODE_FOCUS_MODE will be false.

In general, if you have doubts about a constant, grep the source code to see how it's used. Some of them are used as memory locations where things are written - these need additional care, as the camera bricking does happen (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182570#msg182570) (should be recoverable in most cases, but it's best not to get there). This should help understanding why this happens (https://bitbucket.org/hudson/magic-lantern/pull-requests/825/prevent-canon-settings-from-being-saved/diff) - although the only 100% sure way to prevent bricking is... executing it only in QEMU.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 03:55:50 PM
right well I got ML built using a new platform definition, mostly bogus options of course, but where possible the right setup.

Copied autoexec.bin and 1300D_110.sym to the relevant QEMU 1300D folder, ran with the boot flag and...nothing.
Logout doesnt show autoexec.bin being loaded at all.

Its getting late and its a dawn ANZAC service tomorrow so im calling it a night.
Thanks for the help so far! Im understanding 'some' of what's being achieved, but taking notes. Interesting processes.

Title: Re: Canon EOS 1300D / Rebel T6
Post by: nikfreak on April 24, 2017, 04:06:33 PM
1300D_110.sym

that one needs to be 8.3. You need to rename and therefore shorten as well as redefine it in your platform dir's makefile.platform.default. Example from 1100D:
Code: [Select]
#Makefile.setup.platform for 1100D

# Definitions for version 105
ML_MODULES_SYM_NAME=t3_$(FW_VERSION).sym
...

So for 1300D you name it t6 (https://en.wikipedia.org/wiki/Canon_EOS_1300D)
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 04:17:34 PM
Gotcha.

Ill give that a run in the morrow.

Ta :)
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 24, 2017, 05:19:27 PM
(supposed to be sleeping, going to be wrecked)

I wondered about the adapted code and how it was affecting bootflags as memory is being touched in the same area.

Im not sure why, its beyond my understanding of how Qemu is working, but the patch a1ex provided seems to be colliding with the bootflag setter

Code: [Select]
if (strcmp(s->model->name, "1300D") == 0)
    {
        switch (address)
        {
            case 0xF8000000:
            case 0xF8000001:
            case 0xF8000002:
            {
                /* fixme: a bit hackish */
                unsigned int lr = CURRENT_CPU->env.regs[14];
                if (size == 1 && lr == 0x1D4D4)
               {
                    msg = "Flash model ID?";
                    const int model_id[] = { 0xC2, 0x25, 0x39 };
                    ret = model_id[address & 3];
                    break;
                }
            }
        }
    }

If I add a boot flag in there, setting 0xF8000000 to -1, we drop to the FROMUTILITY loader (on the plus side, the Firmware recovery GUI comes up perfectly on the Qemu display). But removing the touches of those two memory locations (0xF8000000 and 0xF8000001) brings us back to the Assert System.c issue.

Could there be a different value expected here for the boot flag? Or does the particular use case for the 1300D need adjustment in the way Qemu is setting that flag?
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on April 24, 2017, 06:20:27 PM
Yes, it does (it's a side effect of my modification - one of the reasons I'm not going to commit it in this state). When setting the boot flag, MEM_WRITE_ROM writes to the first copy of the ROM (the one modelled as I/O), and the write is currently ignored.

As a workaround, try writing the bootflag in another copy of the ROM (there are a bunch of mirrored ones - any of them will update all the others). For example, .bootflags_addr = 0xFA000000 (not tested). What I've tested was changing MEM_WRITE_ROM to write at addr+ROM0_SIZE, but that's way too hackish.

Probably it's best to handle it in the ROM write handler.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 25, 2017, 12:29:46 AM
Bam, shifting to an alternate copy region works a charm.
Thanks for that.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: adamnock on April 25, 2017, 12:11:48 PM
Just so this isnt a sudden stop thread, taking a short hiatus because having 2 VM's running for build/testing and searching a 400mb+ file to identify stubs is making my laptop glow red.

Fully intend to resume work in ~2 weeks when I have access to my normal development machine (reason: working remote currently)

Thanks for the help (read: doing 99.99995% of the work) A1ex and Nikfreak. And thanks for the solid explainations. Only a few days in and this has already been my most positive experience with OSS projects to date.
Title: Re: Canon EOS 1300D / Rebel T6
Post by: a1ex on April 25, 2017, 12:43:38 PM
Thanks for the help (read: doing 99.99995% of the work)

Well, that was because the first assert was not something I'd expect new contributors to be able to figure out (as it was not present on any other model, and requires a very good understanding of the ROM layout - which I don't have yet). This doesn't usually happen with things already documented or mentioned elsewhere.

And I also happened to have a few days off :D

Edit: looks like the proper way to implement a ROM in QEMU is by using memory_region_init_rom_device (https://github.com/qemu/qemu/blob/master/docs/memory.txt). However, that one appears to handle only writes with callbacks. Go figure...

Edit2: looks like memory_region_rom_device_set_romd (https://github.com/qemu/qemu/blob/master/include/exec/memory.h#L1006) might do exactly what we are looking for :D