Messages

Oh some easter eggs from a1ex. They have a filling too  . So I need to read the man page for patch(1) on my old days...
On the graphics buffer I had been tinkering with opencv on the dumps and making a kind of more universal tool for it, but nothing of value has yet been created.
Also on the electronics route I've got a cheap battery grip lying here waiting for autopsy on the pins.
I've also noted there is some software on the market claiming to be able to talk to the factory settings and dump eeprom for MainPCB change etc., all over USB... The new versions of wireshark can now log USB traffic if you do "sudo ifconfig XHC20 up" before and then choose XHC20 device.

Hmm,
from what I find about this thing it is quite nice, providing OpenGL and vector graphics. I found a japanese PDF https://www.khronos.org/assets/uploads/developers/library/2010-tokyo-press-update/Takumi-GShark_Tokyo-Dec10.pdf stating that the core is implemented using the AXI Interface from ARM https://en.wikipedia.org/wiki/Advanced_Microcontroller_Bus_Architecture#Advanced_eXtensible_Interface_(AXI). There seems to be an Android driver, haven't found it yet as OpenSource might be helpful.

So after some googling I found another user by the Name Ant on the CHDK Forum talking about this device
And I now understand the sw command that doesn't work on the emulation.
So to make it work we would need to implement at last two more processors into the qemu emulation ( a xtensa and the gshark) to get it to work... maybe start again looking for real serial io on the camera itself.

Hey space928,
what do you mean by "abort at boot"?
Do you stay at the bootloader shell or the dryos shell (the one where you enter akashimorino)?
The latter should be possible. We don't have a GUI at the moment as, as far as I understand it, as the memory address of the frame buffer is unknown.  Look here on 80D thread:

found in RAM dump

See also what alex wrote as a reply.
I'm tight on time at the moment and only looking up the forum from time to time to see what is happening.

Hello world,
back from where there is no internets  (but cool photo spots  ).
@space928 : did you manage to get the SFDATA?
I see more then 100 change sets I think I need to catch up ...

Code Select Expand

**** SROM(SIO2) Menu ****
0.Exit from SROM Menu
1.Erase Chip   0x00800000
2.Erase Block  0x00010000
3.Erase Sector 0x00001000
4.Write Data
5.Write from Card
6.SROM Dump(SIO Read)
7.SROM Dump(QUAD Read)
8.Get Info
>>8
0x80000393

To be honest I've cheated when looking for the address and took the same offset as a1ex found on the 80D (0x5c from the address shown in SIO2 line) and then found it when "unaligning" in radare2. Mac has only 64bit GDB which really seems to have some issues (i.e. crashes).
I pushed the dumper and even did some progress counter while the user waits for the dump.

@a1ex do I get my ".serial_flash_size      = 0x800000," line in model_list.c now 
maybe also the ram size?

There seems to be a lot of momentum and I'm trying to catch up with you guys and get the 80D stuff to 750D as my time permits.

Hi,
question:
When running qemu with the -d calls option is it possible to suppress certain calls? If I start a run into FROM Utility the poll serial io call fill up a log file with hundreds of MB quite fast.

Yeah in the meantime I did even try to dump some bytes to the screen (like the original function did) to make sure it is not the file writing that is wrong but to no avail.
So next thing I did was to modify the rom dumper and look what is in memory (at the moment only the copied section 0xX0100000 to ..e500. So far the following discoveries (using radiff2 from radare2):
- adresses 0x0000e470, 0x0000e480 are different every time. No other difference if I take a photo in the meantime (so no "shutter count"). This is true on qemu and cam.
- on the cam 0x0000e495 differs between starts and on qemu 0x0000e496 also some more differences in that area between the two "architectures".

Will include a1ex wrapper and make a push later.

BTW: Wouldn't it be better to move to the RE section and doing a "Joint digic6" thread?

Yep no success...

@sombree:
I've changed the code so that it will initialise the canon routines if you only compile with the CONFIG_BOOT_SROM_DUMPER=y option, but also with the CONFIG_BOOT_DUMPER=y so both routines or just one can be selected for the image.

@all:
- the code is prepared for more models, and as described above modularised
- still todo:
   - the byte vs char stuff to get it correct
   - make it work on my camera   qemu works now fine for me but the camera its still unwilling to put something else than 0x00 into the file  .

Based on the ideas of a1ex I've started a dumper tool look here https://www.magiclantern.fm/forum/index.php?topic=17627.msg195393#msg195393 happy hacking.

Great explanation. Espescially the bit on removing autoexec.bin to get to FROM Utility. That makes me really thinking on ordering some pogo pins a Bus pirate http://dangerousprototypes.com/docs/Bus_Pirate and either a cheap grip, a cheap fake battery and dismantle or make an adapter with the 3D Printer...

-I've enabled the SROM in qemu and made a pull request.

-I don't have any luck with this https://bitbucket.org/t3r4n/magic-lantern/commits/0676ba7fab9259607393a821a150f88e0f60cec3 my C seems a bit rusty but as a start. The qemu and camera survived but all empty. Its prepared for 80D as well.

-Playing around with the SROM utility and dry shell I noticed that they are using some opcodes that the emulator doesn't understand right now.

-If I interpret some of the boot code right they have code for different FLASH manufacturers in place (not sure yet).

-I have definitely seen a reference to a Marconix MX29GL256F NOR Flash driver (look into Strings but could bet the coprocessor section, anyway seems to be a beast with password protection and and so on) . 

Yes there are more routines .

The Bootloader copies 0xe500 bytes of code into RAM at location 0x40100000.
In there I found a routine at 0x40106200 that dumps a few bytes of the serial flash.
I put a simple jump to this in the rebbot.c code. If you switch qemu to serial console you can see it waiting for input. If you enter e.g. 0 and press enter it will show the first few bytes from the sflash. After that it returns to the main SIO Menu where there is another function (enter 7) which will let you choose the range of bytes to dump (don"t have that exact entry point as I messed up). I expect them to have maybe a function underneath to access the sflash. I don't know whether I will have some more time this year  I put a messy (but working) bit up in bitbucket. Compile it

Code Select Expand

make clean ;make install_qemu ML_MODULES_DYNAMIC= CONFIG_BOOT_DUMPER=y
in the portable dir and open in qemu with an attached debugger set breakpoint at said dress and happy tracing.
https://bitbucket.org/t3r4n/magic-lantern/branch/sf_dump_trial

Hmm,
so I thought I had found the entry of SF_CreateSerial and SF_readSerialFlash.
With that I thought I can take the digic_dumper branch and the sf_dump.c from unified branch and do something like this in reboot.c:

Code Select Expand


...
#define BUF_SIZE    1024
static void __attribute__((long_call)) (*SF_CreateSerial)() = NULL;
static void __attribute__((long_call)) (*SF_readSerialFlash)(int src, void* dest, int size) = NULL;
//static int (*SF_Destroy)() = NULL;

/* optional; dumping more will just repeat the contents */
static int SF_flash_size = 0x1000000;

static void sf_dump(int drive)
{
  SF_CreateSerial     = (void*) 0xfe32a226;
  SF_readSerialFlash  = (void*) 0xfe32a1ea;
  //SF_Destroy          = (void*) 0xFF13AF5C;
  // TODO correct size
  SF_flash_size       = 0x800000;
  // This is where the magic happens
  printf("Opening serial flash...\n");
  /* todo: check return values */
  SF_CreateSerial();
  char buffer[BUF_SIZE];
  int filecount =0;
  char sffile[10];
  printf("Reading serial flash...\n");
  /* TODO testing, only 10 files */
  for (int i=0;(i<SF_flash_size)&&(filecount<10);i+=BUF_SIZE) {
    SF_readSerialFlash(i, &buffer, BUF_SIZE);
    snprintf(sffile, sizeof(sffile), "SF%03d.MD5\0", filecount);
    save_file(drive, sffile, (void *)buffer, sizeof(buffer));
    printf("\b\b\b\b%3d%%", (i + BUF_SIZE) * 100 / SF_flash_size);
    filecount++;
  }
}
...

I didn't see any malloc being used hence the idea of using static blocks of char.
But sadly when being run this will fail as the functions are being reached (confirmed by hitting the breakpoint) but some other stuff isn't properly aligned at this stage and they wind up in a loop in memory locations I don"t see when run under firmware conditions.
Maybe I should be looking into starting a thread first and then dump ... guess more debugging

Hey,
I've done two minor changes to the qemu install script to make installation of an updated version a bit easier. That was the small step  .

I hope I did everything right by forking the repository, creating a new branch, uploading the commit and creating the pull request on the emu branch (the old sticky thread here has lost all images so I was guessing from the remaining text).

@a1ex asked about this one:

Noticed a message that I don't remember seeing on previous versions:

Code Select Expand

49:53: execution error: The variable qemu is not defined. (-2753)

Doesn't seem to hurt anything, just wondering if others are seeing it.

Don't remember seeing it; when/where does it appear, and in what color?

it happens on the 750D around this time in the boot, I tried to recreate the colours as they were (normal colour is qemu and red is camera?):
(sorry the code tag and colours don't mix)

[SF] InstallSerialFlash 2 0xd20b0d8c 0x0 0x800000 1

[ROM-DMA0] Copy [0xFD13A000] -> [0x407FFFA0], length [0x00001363], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA0] Copy [0xFDCC0000] -> [0x40D6C000], length [0x0015B6AF], flags [0x11100003]
[XDMAC0] OK
    0:     3.328 [STARTUP]
K393 ICU Firmware Version 1.0.0 ( 8.7.2 )
     5:     4.864 [PROPAD] ERROR Not Exist Valid ComboPackages!! 0x20000
49:53: execution error: Die Variable ,,qemu" ist nicht definiert. (-2753)
[ROM-DMA0] Copy [0xFD200000] -> [0x408020C0], length [0x000205E3], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA0] Copy [0xFD320000] -> [0x40842160], length [0x0001AEDB], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA0] Copy [0xFD360000] -> [0x40862200], length [0x0001DCA7], flags [0x11100003]
[XDMAC0] OK
[ROM-DMA1] Copy [0xFE744B88] -> [0xDFF00000], length [0x00002E77], flags [0x11100003]
[XDMAC1] OK
[MPU] Received: 06 04 02 00 00 00  (Init - spell #1)
[MPU] Sending : 2c 2a 02 00 03 03 03 04 03 00 00 48 00 00 00 14 50 00 00 00 00 81 06 00 00 04 06 00 00 04 06 00 00 04 01 01 00 00 00 00 4d 4b 01 00  (Init)
    9:    64.512 [PROPAD] ERROR SearchPropertyPackage DataType (0) = 0x01000000(L:3294)

So I couldn't wait like a child before Christmas  and started with the SFDATA I got from an 700D without wasting time on patching and after the magic word and and "i" I get this :

Code Select Expand


Name            ID   State Pri         Wait(ID)      Stack  % StackTop StackEnd       SP
EventMgr   0037000c    WAIT  13  RCVMQ(00360006)  01a0/1000 10 002f0560 002f1560 002f14a8
EFLensComT 0042000d    WAIT  15    SEM(003f0025)  0098/0400 14 002e8520 002e8920 002e88b8
MainCtrl   007c0016    WAIT  16  RCVMQ(007b001a)  01d0/1000 11 002f7598 002f8598 002f8510
RTCMgr     00520010    WAIT  17  RCVMQ(00510012)  0288/0400 63 002ecd38 002ed138 002ed080
RscMgr     005a0012    WAIT  17  RCVMQ(00590014)  02f0/1000 18 002f3578 002f4578 002f44c0
OmarInit   00280009    WAIT  18  RCVMQ(00270003)  02f0/0400 73 002e8118 002e8518 002e8460
PropMgr    0031000a    WAIT  20  RCVMQ(00300004)  0378/1000 21 002ee550 002ef550 002ef498
MainSubTas 0047000e    WAIT  20  RCVMQ(0043000e)  00b0/0400 17 002e8928 002e8d28 002e8ca8
FileCache  00570011    WAIT  20  RCVMQ(00560013)  00e8/1000 05 002f2570 002f3570 002f34b8
GuiLockTas 00730013    WAIT  23  RCVMQ(00720017)  00b8/1000 04 002f4580 002f5580 002f54f8
EvShel     008f001c RUNNING  24         -------   0320/8000 02 002fc5c0 003045c0 --------
ConsoleSvr 0097001e    WAIT  24  RCVMQ(00920020)  01e8/0800 23 00304dd0 003055d0 00305540
Startup    00220005    WAIT  25  RCVMQ(00210002)  03a8/2800 09 002ea530 002ecd30 002ecca8
FileMgr    004b000f    WAIT  25  SLEEP(004b000f)  02a8/1000 16 002f1568 002f2568 002f23e0
TOMgr      00750014    WAIT  25  RCVMQ(00740018)  0140/1000 07 002f5588 002f6588 002f64d0
Fstorage   007a0015    WAIT  25  RCVMQ(00790019)  00e8/1000 05 002f6590 002f7590 002f74d8
HDRMgr     00800018    WAIT  25  RCVMQ(007f001b)  00e8/1000 05 002f85a0 002f95a0 002f94e8
HDRStage   00820019    WAIT  25  RCVMQ(0081001c)  00e8/1000 05 002f95a8 002fa5a8 002fa4f0
GISMgr     0086001a    WAIT  25  RCVMQ(0085001d)  00e8/1000 05 002fa5b0 002fb5b0 002fb4f8
GISStage   0088001b    WAIT  25  RCVMQ(0087001e)  00e8/1000 05 002fb5b8 002fc5b8 002fc500
LowConsole 0096001d SUSPEND  25         -------   00d4/0800 10 003045c8 00304dc8 00304d58
NFCMgr     0033000b    WAIT  26  RCVMQ(00320005)  01c0/1000 10 002ef558 002f0558 002f04a0
AEmodeJudg 007e0017    WAIT  26    SEM(007d0045)  0090/0400 14 002ed140 002ed540 002ed4e0
DbgMgr     001e0004    WAIT  31  RCVMQ(001d0001)  0288/1000 15 002e9528 002ea528 002ea470
PowerMgr   001c0003   READY  32         -------   0060/0400 09 002e9120 002e9520 002e9500
idle       00010001   READY  33         -------   006c/0100 42 002e8010 002e8110 002e80d8

or sysConfig

Code Select Expand

K393[1]>sysConfig
vers_dry                 DRYOS version 2.3, release #0055+p4
vers_mk                  2.63
use_smp                  0
act_spi_sem              1
act_spi_event            1
act_spi_mq               1
act_spi_mutex            1
act_spi_cond             1
act_spi_timer            1
act_spi_clock            1
act_spi_isr              0
act_spi_objlist          1
act_spi_objinfo          1
act_spi_objsetname       1
act_timeout              1
act_objname              1
dbg_stack_check          1
dbg_error_check          1
dbg_logging              0
pu_max                   1
sys_mem_start   0x002e8000
sys_mem_max         933888
user_mem_start  0x001cc400
user_mem_max       1133044
sys_objs_start  0x002e0df4
sys_objs_end    0x002e8000
priority_max            34
task_max               119
semaphore_max          317
event_max               80
message_q_max          150
mutex_max              100
condition_max            0
timer_max                0
vector_max               0
it4_mbx_max              0
it4_mpf_max              0
it4_mpl_max              0
level_low                0
level_timer            128
level_kern             128
prio_default            16
stack_default         4096
stack_idle             256
stack_init            4096
stack_addr_idle 0x00000000
stack_addr_init 0x00000000
use_barrier              0
barrier_max              0

sysConfig returned 1(0x1)

endless fun 

Code Select Expand


K393[1]>memMap
00000000 : DRY_EXCEP_VECTOR
00000000 : ATCM_START_ADDR
00004000 : ATCM_END_ADDR
80000000 : BTCM_START_ADDR
80010000 : BTCM_END_ADDR
001cc400 : heap start
           0x001149f4(1133044)
002e0df4 : heap end
002e0df4 : DRY_SYS_OBJS_START
           0x0000720c(29196)
002e8000 : DRY_SYS_MEM_START
           0x000e4000(933888)
001cc000 : DRY_ERREX_STACK_START
           0x00000400(1024)
001cc400 : DRY_ERREX_STACK
80000000 : DRY_EXCEP_STACK_START
           0x00000800(2048)
80000800 : DRY_EXCEP_STACK

memMap returned 0(0x0)


... and those "year end holidays" around the corner to play  

Development is a high word. Let's call it discovery.
When time permits I'm stepping through with qemu and start reading the arm docs with every new discovery. Understanding the minimal.c in the recovery branch and changing bits and pieces (@a1ex haven't found a ROM0 either all dumps in the 0xFxxxxxxx below 0xfc000000 returned nothing)

Question: does anybody know why the interrupt vector table gets build in the boot loader and then again once the "Bootloader END" is on stderr?

Is this picture https://www.aliexpress.com/item-img/The-original-mother-board-big-board-PCB-for-Canon-EOS-750D-Rebel-T6i-DS126571SLR-camera/32693901124.html all that is available? I'm not really willing to open mine. Someone must have dropped one so we can do an autopsy.

Hey a1ex,
thanks for the quick reply.
Got the portable dumper working (on cam and qemu), someone posted two updates on the recovery branch yesterday  .
About the rom0 I read about it in the qemu/hw/eos/mode_list.c under default digic6.
I will do some poking.

With the hardware in the battery compartment hmm looks doable. I'm missing the information on Voltage levels but other than that, by looking at the strings output of the rom, the factory menu behind akashimorino seems to offer some more "poking" options.

Hi,
@matteopd : is europe precise enough

@all :
I would like to summarise  the information I researched from various sources and garnish them with some questions:
- so my 750D has a processor called digic6 which is in fact 2(3?) processors in one chip the main one being an arm7 cortex
- being an arm7 cortex it understands two dialects of opcodes: arm and thumb2. To switch between the two a ldx or blx instruction is used. Our disassemblers have a problem in deciding when to use which dialect (still true or old information?)
- there is a co processor handling much of the io stuff the main processor talks to it via e.g. cdp mcr
- there is a tool enabling the boot flag on the camera. is there a source for that (could be that I'm missing the obvious)?
- qemu enters the firmware at 0x7c000008 why this odd (even though it is even ) address and not 0x7e02000 which is the firmware start? The first eight bytes at 0x7c000000 are no legal opcodes I understand.
- in the qemu code it says under digic6 .rom0 not yet dumped. is that something we are missing? haven't found anything in the forum on the topic
- I have been casually stepping through the startup code and noticed several loop doing nothing else but counting down r0 from e.g.. 0x20a or 0xf there are other loop copying stuff (e.g. from rom at 0xfe020000..40 to 0x0 in tcm.code) I assume these are to sync stuff and wait for some things to settle or is there any "magic" arm stuff I'm missing out?

At the moment I'm experimenting with radare2 as a replacement for gdb. It has some nice analysis features (if I had the money for IDA I would have bought a different camera instead ) it is even possible to dry simulate an arm system without qemu attached.
The camera itself seems to have an interface in the battery compartment (JTAG? serial? ) Does anyone have information on that? I could 3D print a battery dummy an put connectors as they're used on testbeds for PCBs on it to get an interface but I'm a  bit reluctant to do this to my camera.

Well so far, more questions as they arise.
 

Hi,
a few days back a1ex asked me in the 750D thread to proofread his new README for the qemu branch.
Well as time is a premium I was able to do some testing only the other day. So here are some remarks, in what is hopefully the right thread to post them.

1. I was first testing with my 750D ROM files, which are not booting fully at the moment as we need to find some more places to patch away the endless waiting loops but with the instructions given in the readme and a looooot of hitting step on gdb that will be a nice task for cold winter evenings .

2. I wanted to understand better what is going on and managed to get my hands on an EOS 700D, compiled and installed ML and copied the ROM0 and ROM1 as described to the qemu/700D115 directory. Tried to start but it wasn't booting  as it was missing a SFDATA.BIN file.
So there is a missing section in the  README telling you to also compile the sf_dump Module and putting it on the ML Card and activate it and then Reboot the camera and use the module from the Debug Menu of ML. (only found it through full text search on the whole ML directory)

3. In the README under DEBUGGING you also write to use "make CONFIG_qemu=y" and the "make install_qemu" which wouldn't compile in the unified branch for the 700D. I found out that the qemu (or no dm_spy_experiments) branch is needed to use these options, maybe stress that out a bit more in the section.

These are my comments so far. Hope it helps.

Hi a1ex,
thanks for the reply regarding booting I was able to solve that. On my way to the qemu directory I saw the the makeboot script and inside there I saw my fault of not giving the right name to the card ... 

CPU Info from here http://www.magiclantern.fm/forum/index.php?topic=17714.25 works as FIR not the autexec.bin Version (stalls after the second line). I will try to post some good images in the relevant thread later (during blanking the phone focusses out and when it has gained focus again the data already scrolled away  and the camera with manual focus is not able to shoot its own back 
The display Test http://www.magiclantern.fm/forum/index.php?topic=14732.0 works but will show only one solid colour as background.
Model ID: 0x393 750D, Camera Model: EOS K393, Firmware 1.0.0 / 8.7

Regarding qemu after some trouble with python pip not working with the compiler I had enabled in my environment I was able to compile qemu and boot it with the default ML and my ROM in the end I got the same result as above.

Hi,
thanks to a1ex for his work.
So OK I'm willing to step in and help. My skillset includes programming micro controllers in assembler (back in the days it was the only "free" option to program atmel AVR chips before we got gcc and for the tiny ones still the best way). I can do a fair bit in C even though I'm not using it on a daily basis. Atlassian is not git but seems to be quite similar (no war regarding this statement please ;-) ).
   
So far I've been able to:
a) dump my ROM using the DMDP750.FIR
b) set the BOOT FLAG with BOOT750F.FIR

Questions
a) I've been reading here and in the CHDK wikis and found several Versions of the disassemble.pl script anyone with an idea which one to use best?  Does anyone use the CAPDIS and finish2 tools from CHDK? Is ARM-Console still usable with the digic6 and thumb2 processor?

b) with the bootflag enabled I've not been able to run any of the tools from a1ex post from 22nd of September. The card I'm using is a very old 256 MB card and according to fdisk on my Mac the partition is marked bootable. If I understand it right put the autoexec.bin on the card, put the camera dial on M and insert battery and turn on.