Porting ML to XSi (450D)

[rwl408]

ML has been ported to 5D and 40D but there seems to be lack of activity for XSi, which I believed Canon should have sold quite a lot. Yep, you are right, I have one. But I am looking at this porting thing not as an ML user but as an ML developer. For the past couple of weeks I have roamed this website and CHDK in an effort to understand what this "hacking" business is all about and to size up the development work. Well, it is big, at least to me. As someone who doesn't do video, I am wondering if it is worth the effort for me to jump in. From the information I came across it seems that firmware dump, v1.0.9 has been available (http://chdk.setepontos.com/index.php?topic=3402.0) but a later thread asking for help to dump the same firmware (http://chdk.setepontos.com/index.php?topic=4973.0) while using the same file I/O function pointers. Intrigued by this and as an IDA learning exercise (new tool to me), I set out to find what is going on. Well, I found the correct pointer values for those file I/O functions in Canon's flasher code and they are the same for both v1.0.9 and v1.1.0. Hmm, I wonder how one could get the firmware from apparently wrong function pointers. No, I have not dumped anything yet (need to set up the build environment in Ubuntu first, not to mention the build scripts) but I am sure mine are correct because I have cross-checked with those known working pointers in 40D firmware dump code in this thread (http://chdk.setepontos.com/index.php?topic=1651.45 to make sure they executes the same flasher file I/O code and they do. As a matter of fact, XS (1000D) has the exact same flasher file I/O code. They are just in different places (pointer values). I have not decided whether to continue further and certainly will not in the next two/three months but if anyone is interested in porting, I have no problem published the correct pointer values. Just want to get this porting thing move forward, albeit tortoise-wise.

- Rick

[rwl408]

I got two extra weeks before my planned two month's away from home to do some more exploration. The file I/O pointer values that I found in the flasher code are correct. I have used them to dump Canon's firmware v1.1.0 successfully (from 0xFF810000 to 0xFFFFFFFF). It is not as straight as I thought. There were some more details I had to know about to actually dump the firmware without the risk of killing my XSi.

[g3gg0]

so you were able to dump the rom content?
i recommend to dump starting from 0xF8000000 btw

[rwl408]

That (rom dump) is affirmative. I started by dumping 0xFF800000 - 0xFFFFFFFF but then discovered that IDA  isn't happy about 0xFFFFFFFF being used (used by IDA?) and that 0xFF800000 - 0xFF8100000 just contains 0xFF's. I ended up just dumping 0xFF810000 - 0xFFFFFFFC so that I don't have to change file size when loading it into IDA. I am too lazy even though one only has to do it once - the first time loading.

0xF8000000 - 0xFFFFFFFF is a huge chunk (128MB). Can IDA handle that? Or would it be more advantageous to dump it in separate smaller chunks. I know 0xF8000000 - 0xF800000F is used as boot_flags and I guess it is probably in non-volatile memory. How big it may be? Also what other regions that presents interesting stuff for porting. I have taken a snapshot of the source code and am able to compile it. But I have not looked into the code yet.

By the way I think the next thing to do is to find the function pointers in the firmware to enable/disable boot-capability. I have followed the lead in a posting by Coutts (http://www.magiclantern.fm/forum/index.php?topic=1452.0) and found something close to the code (for 5D) in the posting but not quite the same. In 1000D (v1.0.5) I also found the same (XSi) code (in different places but very close by). Both of them are above 0xFFFF0000. I believe I have found them but until I learn ARM enough to be able to follow the code, I am not 100% sure.

After this exercise I have a better grasp of this porting business. It is not very challenged per se if one has the right knowledge (ARM assembly language for me) but requires a ton of labor to find where those equivalent pointers are in XSi, for starter. There should be more things I don't know yet.

[g3gg0]

ROM0: 0xF0000000 - 0xF7FFFFFF (when camera has a ROM0, then its 8 or 16M, so its 0xF0000000-0xF0FFFFFF)
ROM1: 0xF8000000 - 0xFFFFFFFF (most cameras have 16M, so its 0xF8000000-0xF8FFFFFF)

use these ranges, they are the the real base address and repeat over and over until 0xFFFFFFFF.
the processor uses the image at various addresses.
on boot, 0xFFFF0000 is executed, which is at 0xF7FF0000. later it may jump to 0xFF810000 which really is 0xFF010000 etc.

but when dumping, you best use the ranges i quoted above.
relocating in memory can be done later in IDA without any problem

for IDA: just use addresses until 0xFFFFFFFC

[rwl408]

Thanks for the information about ROM0 and ROM1. I read about them but have not put much thought into them yet. A couple of things to clarify though.

About the memory ranges to dump. You mean to dump just the lower 16M (0xF0000000-0xF0FFFFFF for ROM0 and 0xF8000000-0xF8FFFFFF). Right?

About the address mapping. I got lost here. How can 0xFFFF0000 (ROM1 range) be mapped from 0xF7FF0000 (ROM0 range)?

[g3gg0]

haha ok you really understood what i wrote
yeah, i had a typo.

of course 0xFFFF0000 does map to 0xF8FF0000

[rwl408]

Or 0xF8FF0000 maps to 0xFFFF0000.

Anyway I went ahead to dump 0xF0000000-0xF0FFFFFF (16M) and 0xF8000000-0xF8FFFFFF (16M)ranges. The data from the 1st range (ROM0) are all 0's. I guess there isn't ROM0 in XSi (a blank ROM should be all 0xFF's). Data in the 2nd range seems to be two copies of the same code/data with an address offset of 8M. I got the impression when searching for a couple of specific patterns that matched once in my previous dump (0xFF810000-0xFFFFFFFF) and got two matches now. That probably means the size of ROM1 is 8M for XSi. Hmm, interesting.

[1%]

IDA  isn't happy about 0xFFFFFFFF

You can also cut a little bit off the end with a hex editor.. Its usually garbage like F's or 0s at the end.

I'm also wondering about 400D.. would be nice to have dual ISO on that.

[rwl408]

By the way, the first 2K bytes (0xF8000000 and 0xF8800000 too) start with 8 0x00's and then all 0xFF's. So 0xF8000000, 0xF8000004, 0xF8000008 and 0xF800000C have value of 0, 0, -1 and -1. Seems to make sense.

[g3gg0]

yep, the first sector contains startup flags

[rwl408]

If you can write to it, it must be some kind of non-volatile memory to keep the flags across boot/power loss.

[g3gg0]

its a flash chip. commands to erase/write are simple and standardized.

[rwl408]

Ah, good old flash chip. Must be serial type - pin count and size matter. I had dealt with them (both serial and parallel) before. I plan to convert my XSi into IR someday and will look for it when I open it up. Back to the porting thing, which model to pick as the base for XSi in your opinion?

[g3gg0]

its a parallel one, as it is mapped into memory space

[rwl408]

its a parallel one, as it is mapped into memory space

You are so right.

I poked around the ML source code trying to find some details about the boot_flag thing. And I found the bootDisk  flag is @0xF8000004, not what I original thought @0xF8000000! Compare these sections of firmware code, I think I found the equivalent 500D enable/diable_bookDisk code in XSi.

500D Firmware v1.1.1?:

Code Select Expand

ROM:FF066748 ; =============== S U B R O U T I N E =======================================
ROM:FF066748
ROM:FF066748
ROM:FF066748 sub_FF066748                            ; CODE XREF: sub_FF013210+154p
ROM:FF066748                 STMFD   SP!, {R4,LR}
ROM:FF06674C                 ADR     R2, unk_FF066718
ROM:FF066750                 MOV     R1, #0
ROM:FF066754                 ADR     R0, aEnablefirmware ; "EnableFirmware"
ROM:FF066758                 BL      sub_FF1A5B58
ROM:FF06675C                 ADR     R2, sub_FF066724
ROM:FF066760                 MOV     R1, #0
ROM:FF066764                 ADR     R0, aDisablefirmwar ; "DisableFirmware"
ROM:FF066768                 BL      sub_FF1A5B58
ROM:FF06676C                 ADR     R2, sub_FF066700
ROM:FF066770                 MOV     R1, #0
ROM:FF066774                 ADR     R0, aEnablebootdisk ; "EnableBootDisk"
ROM:FF066778                 BL      sub_FF1A5B58
ROM:FF06677C                 ADR     R2, sub_FF06670C
ROM:FF066780                 MOV     R1, #0
ROM:FF066784                 ADR     R0, aDisablebootdis ; "DisableBootDisk"
ROM:FF066788                 BL      sub_FF1A5B58
ROM:FF06678C                 ADR     R2, sub_FF066730
ROM:FF066790                 MOV     R1, #0
ROM:FF066794                 ADR     R0, aEnablemainfirm ; "EnableMainFirm"
ROM:FF066798                 BL      sub_FF1A5B58
ROM:FF06679C                 LDMFD   SP!, {R4,LR}
ROM:FF0667A0                 ADR     R2, sub_FF06673C
ROM:FF0667A4                 MOV     R1, #0
ROM:FF0667A8                 ADR     R0, aDisablemainfir ; "DisableMainFirm"
ROM:FF0667AC                 B       sub_FF1A5B58
ROM:FF0667AC ; End of function sub_FF066748
ROM:FF0667AC
ROM:FF0667AC ;
ROM:FF0667B0 dword_FF0667B0  DCD 0xF8000004          ; DATA XREF: sub_FF066700r
ROM:FF0667B0                                         ; sub_FF06670Cr
ROM:FF0667B4 dword_FF0667B4  DCD 0xF800000C          ; DATA XREF: sub_FF066724r
ROM:FF0667B8 aEnablefirmware DCB "EnableFirmware",0  ; DATA XREF: sub_FF066748+Co
ROM:FF0667C7                 DCB    0
ROM:FF0667C8 aDisablefirmwar DCB "DisableFirmware",0 ; DATA XREF: sub_FF066748+1Co
ROM:FF0667D8 aEnablebootdisk DCB "EnableBootDisk",0  ; DATA XREF: sub_FF066748+2Co
ROM:FF0667E7                 DCB    0
ROM:FF0667E8 aDisablebootdis DCB "DisableBootDisk",0 ; DATA XREF: sub_FF066748+3Co
ROM:FF0667F8 aEnablemainfirm DCB "EnableMainFirm",0  ; DATA XREF: sub_FF066748+4Co
ROM:FF066807                 DCB    0
ROM:FF066808 aDisablemainfir DCB "DisableMainFirm",0 ; DATA XREF: sub_FF066748+60o

                           ===== bootdisk_enable =====
ROM:FF066700 ; =============== S U B R O U T I N E ================
ROM:FF066700
ROM:FF066700
ROM:FF066700 sub_FF066700                            ; DATA XREF: sub_FF066748+24o
ROM:FF066700                 LDR     R0, =0xF8000004
ROM:FF066704                 MOV     R1, 0xFFFFFFFF
ROM:FF066708                 B       sub_FF06664C
ROM:FF066708 ; End of function sub_FF066700

                           ===== bootdisk_disable =====
ROM:FF06670C ; =============== S U B R O U T I N E ================
ROM:FF06670C
ROM:FF06670C
ROM:FF06670C sub_FF06670C                            ; DATA XREF: sub_FF066748+34o
ROM:FF06670C                 LDR     R0, =0xF8000004
ROM:FF066710                 MOV     R1, #0
ROM:FF066714                 B       sub_FF06664C
ROM:FF066714 ; End of function sub_FF06670C

XSi Firmware v1.1.0

Code Select Expand


ROM:FFD184A0 ; =============== S U B R O U T I N E =======================================
ROM:FFD184A0
ROM:FFD184A0
ROM:FFD184A0 sub_FFD184A0                            ; CODE XREF: sub_FF811340+164p
ROM:FFD184A0
ROM:FFD184A0 var_4           = -4
ROM:FFD184A0
ROM:FFD184A0                 LDR     R0, =aEnablefirmware ; "EnableFirmware"
ROM:FFD184A4                 MOV     R1, #0
ROM:FFD184A8                 LDR     R2, =sub_FFD18158
ROM:FFD184AC                 STR     LR, [SP,#var_4]!
ROM:FFD184B0                 BL      sub_FFD0A128
ROM:FFD184B4                 LDR     R0, =aDisablefirmwar ; "DisableFirmware"
ROM:FFD184B8                 MOV     R1, #0
ROM:FFD184BC                 LDR     R2, =sub_FFD1822C
ROM:FFD184C0                 BL      sub_FFD0A128
ROM:FFD184C4                 LDR     R0, =aEnablebootdisk ; "EnableBootDisk"
ROM:FFD184C8                 MOV     R1, #0
ROM:FFD184CC                 LDR     R2, =unk_FFD18538
ROM:FFD184D0                 BL      sub_FFD0A128
ROM:FFD184D4                 LDR     R0, =aDisablebootdis ; "DisableBootDisk"
ROM:FFD184D8                 MOV     R1, #0
ROM:FFD184DC                 LDR     R2, =unk_FFD18550
ROM:FFD184E0                 BL      sub_FFD0A128
ROM:FFD184E4                 LDR     R0, =aEnablemainfirm ; "EnableMainFirm"
ROM:FFD184E8                 MOV     R1, #0
ROM:FFD184EC                 LDR     R2, =sub_FFD18300
ROM:FFD184F0                 BL      sub_FFD0A128
ROM:FFD184F4                 LDR     R0, =aDisablemainfir ; "DisableMainFirm"
ROM:FFD184F8                 MOV     R1, #0
ROM:FFD184FC                 LDR     R2, =sub_FFD183D0
ROM:FFD18500                 LDR     LR, [SP+4+var_4],#4
ROM:FFD18504                 B       sub_FFD0A128
ROM:FFD18504 ; End of function sub_FFD184A0
ROM:FFD18504
ROM:FFD18504 ; ---------------------------------------------------------------------------
ROM:FFD18508 off_FFD18508    DCD aEnablefirmware     ; DATA XREF: sub_FFD184A0r
ROM:FFD18508                                         ; "EnableFirmware"
ROM:FFD1850C off_FFD1850C    DCD sub_FFD18158        ; DATA XREF: sub_FFD184A0+8r
ROM:FFD18510 off_FFD18510    DCD aDisablefirmwar     ; DATA XREF: sub_FFD184A0+14r
ROM:FFD18510                                         ; "DisableFirmware"
ROM:FFD18514 off_FFD18514    DCD sub_FFD1822C        ; DATA XREF: sub_FFD184A0+1Cr
ROM:FFD18518 off_FFD18518    DCD aEnablebootdisk     ; DATA XREF: sub_FFD184A0+24r
ROM:FFD18518                                         ; "EnableBootDisk"
ROM:FFD1851C off_FFD1851C    DCD unk_FFD18538        ; DATA XREF: sub_FFD184A0+2Cr
ROM:FFD18520 off_FFD18520    DCD aDisablebootdis     ; DATA XREF: sub_FFD184A0+34r
ROM:FFD18520                                         ; "DisableBootDisk"
ROM:FFD18524 off_FFD18524    DCD unk_FFD18550        ; DATA XREF: sub_FFD184A0+3Cr
ROM:FFD18528 off_FFD18528    DCD aEnablemainfirm     ; DATA XREF: sub_FFD184A0+44r
ROM:FFD18528                                         ; "EnableMainFirm"
ROM:FFD1852C off_FFD1852C    DCD sub_FFD18300        ; DATA XREF: sub_FFD184A0+4Cr
ROM:FFD18530 off_FFD18530    DCD aDisablemainfir     ; DATA XREF: sub_FFD184A0+54r
ROM:FFD18530                                         ; "DisableMainFirm"
ROM:FFD18534 off_FFD18534    DCD sub_FFD183D0        ; DATA XREF: sub_FFD184A0+5Cr
ROM:FFD18538 unk_FFD18538    DCB 0x4F ; O            ; DATA XREF: sub_FFD184A0+2Co
ROM:FFD18538                                         ; ROM:off_FFD1851Co
ROM:FFD18538 ;

---------------------------- bootdisk_enable? ---------------
ROM:FFD18538
ROM:FFD18538 loc_FFD18538                            ; DATA XREF: sub_FFD184A0+2Co
ROM:FFD18538                                         ; ROM:off_FFD1851Co
ROM:FFD18538                 MOV     R0, 0xF8000004
ROM:FFD18540                 MOV     R1, 0xFFFFFFFF
ROM:FFD18544                 STR     LR, [SP,#-4]!
ROM:FFD18548                 LDR     LR, [SP],#4
ROM:FFD1854C                 B       loc_FFD18078
ROM:FFD18550 ;


---------------------------- bootdisk_disable? ---------------
ROM:FFD18550
ROM:FFD18550 loc_FFD18550                            ; DATA XREF: sub_FFD184A0+3Co
ROM:FFD18550                                         ; ROM:off_FFD18524o
ROM:FFD18550                 MOV     R0, 0xF8000004
ROM:FFD18558                 MOV     R1, #0
ROM:FFD1855C                 STR     LR, [SP,#-4]!
ROM:FFD18560                 LDR     LR, [SP],#4
ROM:FFD18564                 B       loc_FFD18078

This similarity makes me think if 500D is a good candidate of reference for XSi porting even though these two models  have different OS. Anyway, now is the time for ARM assembly language and architecture, and perhaps VxWorks. Until then, I don't think I can make any meaningful progress.

[astroxsi]

Hi all,

Any news on this topic?

I'm a 450d user (I use it mainly for astrophotography stuff) and ML would be a nice addon, I really wish to see it one day . I haven't moved to a newer canon model because this 450d is heavily modified (IR filter removed, peltier cooling), so I would like to stick with that camera at least for a while.

I'm afraid I cannot help with the code because I don't know the internals of Canon firmware, and I don't have experience hacking the ML code, but I can offer myself to try to test any code you generate.

[Walter Schulz]

I think it would be wise not to wait for a port but to use some advanced remote control device.

[Ant123]

Some progress with converting 40D port to 450D:

[Marsu42]

Some progress with converting 40D port to 450D

Oh my, it isn't photoshopped? If not: good to see some people are not going along with planned obsolescence and try to make use of the good ol' gear!

Having said that, I personally wouldn't go below the long-running 18mp sensor for general photography nowadays, there are so many cams around with it and ML support so they should be more than affordable... I just bought a used 7d1. That's b/c I had a 450d-class cam (Nikon 3100) for some weeks on loan and wouldn't want to touch that usability and sensor iq again even with a 10-foot pole.

[juvain]

Some progress with converting 40D port to 450D:

Can you post a link to this test build?

[Ant123]

It's not finished yet. There is no normal installer. Shooting functions not implemented. Only intervalometer works at the moment...

[juvain]

It's not finished yet. There is no normal installer. Shooting functions not implemented. Only intervalometer works at the moment...

Can I help you somehow?

[Ant123]

I compiled a modern version of ML with the changes made by rufustfirefly.
But I've got it working only when I used GCC 4.7.3
With GCC 4.8 or 4.9 I'm getting camera crash during font loading.
Is it necessarily to use GCC 4.8.3 ?

Garbage at the rightmost column was removed when I changed BMP_W_PLUS from 720 to 721 in bmp.h:


Is this trick correct?

Source code: https://bitbucket.org/Ant123/magic-lantern/src
Installer for 450D firmware 1.1.0. : https://bitbucket.org/Ant123/magic-lantern-40d/downloads/ML450D110.fir
It works the same as 40D/5DC installer.
ML test build for 450D firmware 1.1.0. : https://bitbucket.org/Ant123/magic-lantern-40d/downloads/magiclantern-Nightly.2016Oct26.450D110.zip
Only intervalometer can be useful at the moment...

[a1ex]

Added the 450D and 40D changes to the vxworks branch in the main repo. Did some minor cleanups (compiler warnings), had some attempts at emulating it in QEMU, but didn't really check any deeper yet.

For reverse engineering, I've adapted the dm-spy-experiments branch as vxworks-dm-spy (tested in qemu).