Magic Lantern Forum

Developing Magic Lantern => Reverse Engineering => Topic started by: g3gg0 on September 24, 2012, 12:08:57 AM

Title: ARM + EOS Emulator
Post by: g3gg0 on September 24, 2012, 12:08:57 AM
well, it think i can make it public.

1. ready-to-run package

Quote
i added GDB stubs to my emulator.
what this means?
you can use IDA Pro to connect to the emulator and step through code using breakpoints, dumping memory etc etc.
check contents of main routine at line 938 and make fit them to your firmware.

1. start TriX
2. select your firmware image as input file
3. click on "Scripts" tab ("General", "Scripts", "Editor")
4. click the lens at the bottom, right of "Script" and the textbox
5. choose "armulate_shell_eos.trx"
6. click "Add" button at bottom
7. click "Start" in the top toolbar
8. a few register/disassembly windows pop up
9. arrange them that you see every window
10. in the main dialog again where it asks you "Your choice", below is a text box. enter the number "16" and press enter


then in IDA just connect to localhost, port 23946 using gdb as debugger interface.

before connecting: in "Debugger Setup", "Set specific options" you should set "Max packet size" to 512
and in same window under "Memory map" you have to insert (rightclick into the list) a new memory segment which
starts at 0x000000 and ends at 0xFFFFFFFE, base 0, 32 bit, read only. delete the old one, if one is defined.

enjoy :)

http://upload.g3gg0.de/pub_files/0e7cc977a512c2168003a4ceb0e82932/TriX_EOS.7z

2. do-it-all-yourself repository

1. get a SVN client (e.g. TortoiseSVN)
2. checkout http://svn.g3gg0.de/svn/default/trunk/nokia/TriX/  (user: trix, pass: trix)
3. get Visual Studio 2008 (v9.0)
4. get Qt SDK (e.g. i have v4.5.1) and build/install *
5. set environment variable QTDIR to your Qt-Dir (that contains bin, lib, include, tools, ...) *
6. open \platform\msvc\TriX.sln
7. rebuild all

* = if you cannot get the project "TriX" compiling because of Qt issues, but the plugins TriX_DisARM, TriX_ARMulate, TriX_HWemuEOS are building fine, then it is also okay.
the most important stuff for emulating canon firmware is in HWemuEOS anyway.
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 24, 2012, 02:31:13 AM
Quote from: g3gg0 on September 24, 2012, 12:08:57 AM
well, it think i can make it public.

Thank you ... I open your emulator and found it has a good presentation. Once I have time I will test it.

My antivirus says that an application can be dangerous! What features has its application for him to say that?

Thank you.
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 24, 2012, 12:41:20 PM
uuh, well a lot :)
it is a patcher plattform that is scriptable and has plugins.
so there are some techniques that are e.g. loading DLLs into RAM by injecting in EXE etc.
that is for loading encrypted plugins. but not used here in this example.

i (ab)use my TriX to load a ARM emulator and a ARM disassembler plugin and control that with a script (.trx).
these scripts are C-code that gets compiled dynamically.

its open source too and also contains a plugin named HWemu_EOS which emulates the DIGiC behavior.
e.g. serial port, DMA timers and even flash chip emulation with reflashing etc.
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 24, 2012, 12:48:31 PM
and if you dont start the GDB stub with '17', bu you start execution normally ('8' (run until) then '0' (or any oher address))
you will see the boot rom output in the terminal window.

it is waiting for input there. you can enter commands in this terminal window like testing audio ;)
but of course many things wont work. but it is helpful for testing how things work.
e.g. setting boot flags or PAL/NTSC parameters should work in this version as flash emulation is integrated.

pressing ctrl-c in terminal will break execution
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 24, 2012, 02:49:44 PM
Quote from: g3gg0 on September 24, 2012, 12:48:31 PM
and if you dont start the GDB stub with '17', bu you start execution normally ('8' (run until) then '0' (or any oher address))
you will see the boot rom output in the terminal window.

it is waiting for input there. you can enter commands in this terminal window like testing audio ;)
but of course many things wont work. but it is helpful for testing how things work.
e.g. setting boot flags or PAL/NTSC parameters should work in this version as flash emulation is integrated.

pressing ctrl-c in terminal will break execution


I did a basic test and it worked :) I will test your emulator with GDB.
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 24, 2012, 03:39:53 PM
makes most sense with IDA pro :)
Title: Re: ARM + EOS Emulator
Post by: nanomad on September 24, 2012, 04:46:26 PM
Well, after trying the emulator I have to admit it's quite powerful. One question, how far in the firmware execution can you go?
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 24, 2012, 05:09:23 PM
Quote from: g3gg0 on September 24, 2012, 03:39:53 PM
makes most sense with IDA pro :)

Yes it's true! I agree with you but is too expensive for many of us :)
There are less expensive solutions like Eclipse CDT, DDD, Insight that should be sufficient for many of our tests.
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 24, 2012, 06:37:47 PM
Quote from: nanomad on September 24, 2012, 04:46:26 PM
Well, after trying the emulator I have to admit it's quite powerful. One question, how far in the firmware execution can you go?

thanks ;)
well, after patching one data word in flash (no idea why this was neccessary) the os booted and started tasks.
task switching worked, as i emulated the timer interrupt too.
not sure if this is still in the current version.

the patch i made is below.
it will not work anymore, since the flash data cannot simply be written with a SetWord due to the flash ic emulation.
best is to do this in original flash image file :)
Quote
    // some timeout? set to 1 else debugmsg will fail :(
    SetWord(0xF85A1630, 1);

i got a lot debug messages during boot.
cant find where ive stored the logs atm :)

Title: Re: ARM + EOS Emulator
Post by: nanomad on September 24, 2012, 06:47:00 PM
Uhm, weird. I'm loading the 600D bin you provided and it looks like it boots into some kind of factory mode for K286. It prints the first message then hangs
The last routine called is at 0xFFFF18F8
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 24, 2012, 08:01:02 PM
Quote from: nanomad on September 24, 2012, 06:47:00 PM
Uhm, weird. I'm loading the 600D bin you provided and it looks like it boots into some kind of factory mode for K286. It prints the first message then hangs
The last routine called is at 0xFFFF18F8

I think its very interesting tool, but this conversation starts to get out of the initial scope.
I suggest that you create another thread to discuss this issue. I'll join that conversation ..
Title: Re: ARM + EOS Emulator
Post by: nanomad on September 24, 2012, 08:13:21 PM
Done
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 24, 2012, 08:51:10 PM
Quote from: nanomad on September 24, 2012, 08:13:21 PM
Done

Thank you...
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 24, 2012, 09:20:47 PM
Quote from: nanomad on September 24, 2012, 06:47:00 PM
Uhm, weird. I'm loading the 600D bin you provided and it looks like it boots into some kind of factory mode for K286. It prints the first message then hangs
The last routine called is at 0xFFFF18F8

press enter in the "Terminal" window ;)

you are booting the bootloader.
to boot normal firmware, modify the .trx file or set the PC (register 15) manually to 0xf8010000
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 25, 2012, 12:18:56 AM
I liked knowing that my original post have given rise to this thread. I think this topic is very interesting and will give much to talk about.
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 25, 2012, 01:35:55 AM

'Script is not signed correctly.'

This message appears. What does that mean?
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 25, 2012, 02:04:53 AM
g3gg0 you're my hero!

Now, I could see the service menu of the 600D. My congratulations!
You work very well! The port of the 7D is a good example! ... The IDx debugger is beautiful;)

K286 System & Display Check & Adjustment program has started.

********** FACTORY ADJUSTMENT MENU VER 0.01 **********
0. Exit from Factory Adjustment
1. Leak Check
2. SDRAM Check
3. ROM Check
4. HDMI Implementation Check
5. Video Adjustment
6. ICU Version Check
7. Audio Implementation Check
8. Adjustment Data Display and Change
9. Check Flag Display and Initialization
D. Debug Command
A. ALL Check
B. ALL Check without Unique
Y. Input Unique
Z. Input_Device_Unique
CheckPro> 2
--- Start SDRAM access check ---
Data Bus Test.
Write Address : 0x04000000
Write Address : 0x04000000
NG
--- Finish ---
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 25, 2012, 02:51:28 AM
I do not know why but it does not work in the dump 40D v1.1.1. I start to debug step by step and from one moment to another memory seems to be all '0 'and' arm 'throws an exception ... invalid instruction.

The problem occurs in:

FF8100C0  LDR PC, =rom_cstart_FF8100DC
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 25, 2012, 07:57:28 PM
are you sure about all the addresses?
sure that at the address is a valid instruction (try setting PC manually there)

can you upload the firmware file (http://upload.g3gg0.de/) and tell me the start address?
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 25, 2012, 08:15:33 PM
Yes I made a mistake. I edited the file .trx of eos and then got in debugging the 40D .fir

I did not understand very well some things in the file .trx but I could see some things working.

I will greatly appreciate your help to redirect messages to the console of your emulator.

Together we will surely get ... This simple but powerful tool can help a lot in the port of 40D camera.

In a few hours I'll send you some files :)


Thank you
Title: Re: ARM + EOS Emulator
Post by: 1% on September 25, 2012, 09:19:25 PM
Input device unique? Is that the serial.
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 25, 2012, 11:00:38 PM
Quote from: 1% on September 25, 2012, 09:19:25 PM
Input device unique? Is that the serial.

Also available is the door USB.Excelente would redirect the debug logs, and serial console for your emulator.
I think you did it with the 600D camera.
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 25, 2012, 11:06:44 PM
Quote from: g3gg0 on September 25, 2012, 07:57:28 PM
are you sure about all the addresses?
sure that at the address is a valid instruction (try setting PC manually there)

can you upload the firmware file (http://upload.g3gg0.de/) and tell me the start address?


I've sent you some files ...
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 25, 2012, 11:10:11 PM
Thank you nanomad!
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 27, 2012, 02:26:06 PM
using this ARMulate plugin:
http://upload.g3gg0.de/pub_files/94598913edfe93f9c9b556a1f02bf925/TriX_ARMulate.txp

and this HWEmuEOS plugin:
http://upload.g3gg0.de/pub_files/2770e0e87a7c7a8ac36e441e072cfdda/TriX_HWemuEOS.txp

you should get a bit further.
it might lock somewhere in CF code.
if you tell me some 0xC0... registers, how they should behave, i can implement this.

but you still should be able to boot the ROMUTIL.
simply execute until 0xFFFF0320 (using '8', 'FFFF0320') then skip that one BL instruction that
executes FROMUTIL (this causes emulator to hang with some output on terminal)

simply continue execution at FFFF0324 and you get the ROMUTIL asking you what to do ;)
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 27, 2012, 02:27:38 PM
Quote from: 1% on September 25, 2012, 09:19:25 PM
Input device unique? Is that the serial.

not sure, it might be the signature for signing "authentic" camera photos?
afaik this feature was disabled because it was already hacked.
Title: Re: ARM + EOS Emulator
Post by: 1% on September 27, 2012, 05:51:32 PM
This feature: http://techcrunch.com/2011/04/28/both-nikon-and-canons-image-authentication-systems-busted/

Used for court....
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 28, 2012, 03:58:59 AM
Thanks for your explanation ... either way I had managed to follow these steps. These files are new? had some correction?
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 28, 2012, 08:34:02 PM
Quote from: jplxpto on September 28, 2012, 03:58:59 AM
Thanks for your explanation ... either way I had managed to follow these steps. These files are new? had some correction?

yes i added some new features like MRC/MCR handling of basis CP15 functions.
this is needed e.g. for setting up cache etc. and some HWemu function.
its not complete yet.

are you interested in the source code of TriX/ARMulate/HWemuEOS?
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 29, 2012, 01:55:48 AM
i updated the description in the first post.
you can checkout TriX and the emulator plugins from my SVN.
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 30, 2012, 03:38:09 PM
Quote from: g3gg0 on September 29, 2012, 01:55:48 AM
i updated the description in the first post.
you can checkout TriX and the emulator plugins from my SVN.


How can I have access to your SVN?
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on September 30, 2012, 09:14:34 PM
Quote from: jplxpto on September 30, 2012, 03:38:09 PM

How can I have access to your SVN?

eeerh, thats in the first post ;)
Title: Re: ARM + EOS Emulator
Post by: jplxpto on September 30, 2012, 09:38:21 PM
Quote from: g3gg0 on September 30, 2012, 09:14:34 PM
eeerh, thats in the first post ;)

sorry
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on October 01, 2012, 02:09:34 AM
i updated the main post.
the package is now available here (http://upload.g3gg0.de/pub_files/0e7cc977a512c2168003a4ceb0e82932/TriX_EOS.7z) and gdbstub is avaible via menu id 16

i found a severe bug that i have fixed now (one kind of STRH wrote a whole word...)
but i am sure, there are still some bugs :)
Title: Re: ARM + EOS Emulator
Post by: ilguercio on October 01, 2012, 02:29:43 AM
I guess non-developers can make no use of this, right?
Title: Re: ARM + EOS Emulator
Post by: nanomad on October 01, 2012, 10:12:47 AM
Not really... the section is called "Reverse Engineering and Firmware Design" after all
Title: Re: ARM + EOS Emulator
Post by: coderat on December 09, 2013, 10:37:23 PM
My question is: how fat you can emulate ? Pushing shutter ?
And what about TX19A chip, how do you emulate this one ?

Best regards,
coderat
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on December 09, 2013, 10:48:11 PM
nah, we do not emulate the Tx19a, maybe it would make sense to add intercom via TCP/IP or some other kind of sockets to your emulator.
but we barely reverse engineered that MIPS firmware. it doesnt make any fun :)
Title: Re: ARM + EOS Emulator
Post by: coderat on December 09, 2013, 11:03:35 PM
Quote from: g3gg0 on December 09, 2013, 10:48:11 PM
nah, we do not emulate the Tx19a, maybe it would make sense to add intercom via TCP/IP or some other kind of sockets to your emulator.
but we barely reverse engineered that MIPS firmware. it doesnt make any fun :)
Ok, how about this one: get an ARM CPU emulator into our existing NikonEmulator. You get in this way complete system with all frameworks: disassembler, debugger, logger, ports simulation, etc+ TX19A emulation, so can build a complete system may be ?

TCPIP is perhaps not so good because of performance issues?

We are also interested in ARM, because new cameras like D7100 are ARM Thumb-2.

Best regards,
coderat
Title: Re: ARM + EOS Emulator
Post by: g3gg0 on December 09, 2013, 11:32:47 PM
when checking the "market" about 7 years ago, there was no suitable ARM emulator.
suitable = simple C, not bloated, easy to understand API

there were two major candidates - qemu and GBAemu.
qemu failed because the API was not structured well and it was not easy to integrate in the debugger/analyzer a friend and i coded (nok5rev)
so we chose to use GBAemu, but it had a few bugs that i fixed step by step.
in the end i learned a lot C during that time ;)
meanwhile i merged that code as plugin into my scriptable patching system (TriX) which this thread is about.

http://trix:[email protected]/svn/default/trunk/nokia/TriX/plugins/TriX_ARMulate/armulate.c
http://trix:[email protected]/svn/default/trunk/nokia/TriX/plugins/TriX_ARMulate/opcodes.h


what i want to say: i am not sure if it is the best choice.
Title: Re: ARM + EOS Emulator
Post by: coderat on December 10, 2013, 01:08:05 AM
Quote from: g3gg0 on December 09, 2013, 11:32:47 PM
when checking the "market" about 7 years ago, there was no suitable ARM emulator.

suitable = simple C, not bloated, easy to understand API
I used several in fact (even Visual Studio Windows Phone Emulator :) ). But agree no one was simple and easy configurable.
Quotethere were two major candidates - qemu and GBAemu.
Qemu fails always due to the same reason ;)
Quotemeanwhile i merged that code as plugin into my scriptable patching system (TriX) which this thread is about.

http://trix:[email protected]/svn/default/trunk/nokia/TriX/plugins/TriX_ARMulate/armulate.c
http://trix:[email protected]/svn/default/trunk/nokia/TriX/plugins/TriX_ARMulate/opcodes.h


what i want to say: i am not sure if it is the best choice.
What about license for this source in TriX ? If I see correct, it is Thumb code ?

Well, our Emulator is in Java :) The point is that we had 2 Chips configuration from begining, so we got a lot of experience with it. You must have the same clock (we say MasterClock) for both chips and all other parts, otherwise you can't get system runing stable. The MasterClock guarantees that everything runs at same tact ratio relative to each other: if one CPU emulation needs more time, another one will be slowed down automatically because of design. After we got this, a box of Pandora opened :)

So asynchronous solutions with TCPIP connector will be pretty difficult to implement, because of clock synchronisation. Or ?

Next question will be Interrupt Controller and Cache Controller AND eventually MMU unit if your target have one ? I suppose almost each ARM core has one MMU.

P.S. I posted in your forum how to use our TX19A disassembler, for the case you want to poke on MIPS part.

Best regards,
coderat
Title: Re: ARM + EOS Emulator
Post by: bootrom on March 03, 2015, 11:07:55 PM
Quotehttp://trix:[email protected]/svn/default/trunk/nokia/TriX/plugins/TriX_ARMulate/armulate.c
http://trix:[email protected]/svn/default/trunk/nokia/TriX/plugins/TriX_ARMulate/opcodes.h

what i want to say: i am not sure if it is the best choice.
Checked this to max possible, but failed: Niccon using ARMv6, Thumb and Thumb2. You are lucky man that only need usual ARM.