Author Topic: Porting ML to 1000D (AKA Compiling for 1000D/XS)  (Read 11113 times)

Levas

  • Hero Member
  • *****
  • Posts: 873
  • 6d - Nightly build user
Re: Compiling for 1000D/XS
« Reply #25 on: December 04, 2016, 12:34:07 PM »
@Alex

Look, both leds are blinking, even made them alternate blinking, just like you asked for  ;D

http://drive.google.com/open?id=0B1BxGc3dfMDaRUZweUJ5NWZUTTQ

What's next ?

Levas

  • Hero Member
  • *****
  • Posts: 873
  • 6d - Nightly build user
Re: Compiling for 1000D/XS
« Reply #26 on: December 04, 2016, 03:18:27 PM »
@shmadul and @Syscall,

Meanwhile I checked the topic of the 450d progress.
http://www.magiclantern.fm/forum/index.php?topic=8119.msg174078#msg174078

Seems to be a vxworks branch for this type of camera's
https://bitbucket.org/hudson/magic-lantern/src/18ac6b0f992918c7ba6dd282c3e74ca42574561c/?at=vxworks

In the platform folder, you can find the 450d folder and it's source files.


shmadul

  • New to the forum
  • *
  • Posts: 28
Re: Compiling for 1000D/XS
« Reply #27 on: December 05, 2016, 06:39:29 PM »
@levas, Nice Work  :D, Ive Heard the 450D is almost the same as the 1000D so maybe we could either see if we can apply the code to our port or see if we could get one of the guys working on the 450D port to help us get ML running, We already have the FW exploited and booting custom code, we just need to port the ML GUI and stuff

Direct Link to VXWorks Repo
https://bitbucket.org/hudson/magic-lantern/get/vxworks.zip

SysCall

  • New to the forum
  • *
  • Posts: 33
Re: Compiling for 1000D/XS
« Reply #28 on: December 05, 2016, 07:48:52 PM »
-----------------------------
Documentation
-----------------------------

Before we can port magic lantern to this camera, we have to understand what magic lantern is doing in the first place.

I will try to explain what my understanding of how magic lantern works so far is.

If some information are wrong or missing, please let me know.


A. Boot-up process
---------------------

Cases:

1. Turn camera on (normal):
The camera will boot up the firmware, normal camera operation.

2. Update the camera (normal):
With the original canon.fir on the SD card.

In the camera menu -> firmware update execution.

The new firmware will written into the internal flash.

This is the typical process for any firmware updates, not only on cameras.

After the firmware update finished, the camera reboots and the camera operates as usual.


3. Update with custom generated .fir file

With the custom .fir and AUTOEXEC.BIN on the SD card.

In the camera menu -> firmware update execution.

The custom firmware .fir on the SD card sets a boot-flag, which allows us to load the AUTOEXEC.BIN from the SD card.

The camera restart, now the camera is in normal operation + custom magic lantern code running in parallel.

Since the boot-flag is set, the camera will load the AUTOEXEC.BIN automatically each time the camera is started,

when turn on and off with camera switchI guess.


Comments on case 3:

This seems the normal operating process of magic lantern on most of the implementation so far.

Coutts explained here a little bit deeper:
https://www.magiclantern.fm/forum/index.php?topic=1452.0

And here in the wiki:
http://magiclantern.wikia.com/wiki/Packing_FIR_Files


B. Current development
---------------------------

Now that is interesting, because in my example "led_blinking.zip" I don't remember setting any bootflags.

So why does it work anyway?

The code for the modification of the boot process is usually defined in:

entry.S


Well looking into the entry.S shows:

Code: [Select]

.text
.org 0
.globl _start, start

start:
_start:
sysInit:
B main

.align 2
fin:



And the there is no entry_subs.S file so far in "led_blinking.zip" release.

So the question would be, does it also work without the AUTOEXEC.BIN on the SD card?

After a quick test with only the 1000d.fir on the SD card.

The answer is, yes it does. But why?


Well, thinking about it, what is the 1000d.fir?

It is base on the AUTOEXEC.BIN, which is again base on our code in the main.c.

Compilation order:  brain -> main.c -> AUTOEXEC.BIN -> 1000d.fir

Simple speaking, the 1000d.fir is like an .exe in windows with our code.

Each time we execute the 1000d.fir through the firmware update menu, our code will be executed.


That means:

1. The .fir file and AUTOEXEC.BIN, both can have the same code.

2. The .fir file can be executed standalone without the AUTOEXEC.BIN.

3. Each time we turn the camera off and on, we have to start the firmware update process again to load our code.

4. The AUTOEXEC.BIN can only be executed if the bootflag is set.

   And setting the bootflag can be done with the .fir file, since we know how to sign it and the camera can read and executed it.
 

Now, I just need some confirmation to back up my theory.

After thinking a while, I remember somebody gave a talk about magic lantern.

A quick google search and here we go:

Presentation:
https://events.ccc.de/congress/2013/Fahrplan/system/attachments/2246/original/Magic_Lantern_30C3.pdf

Video:
https://media.ccc.de/v/30C3_-_5554_-_en_-_saal_6_-_201312281400_-_magic_lantern_-_michael_zoller


From slide 23 of the presentation:

8.2 How is ML being executed?

Method 1: Firmware update
- We forge a firmware update file (.fir) with Magic Lantern as payload
   - Contains no flash data
   - Flash loaders in .fir are replaced with our code
   - Checksums and hashes need recalculation

- User runs firmware update from original firmware menu

- Firmware restarts into boot loader

- Bootloader loads .fir into RAM and executes it

- Magic Lantern takes over control of the system and boots original firmware


Method 2: Autoboot

- Using .fir we set a flag in flash called "BOOTDISK"

- Bootloader checks for "AUTOEXEC.BIN" on CF/SD

- If found, it gets loaded and executed without checks

- Magic Lantern takes over control of the system and boots original firmware

----------------------------

Now, looking at Coutts latest release, we do have functions to set and de-assert the bootflag.

The functions to enable and disable of the boot-flag is defined in:
entry_subs.S

NSTUB(EnableBootDisk, 0xFFD21248)
NSTUB(DisableBootDisk, 0xFFD21260)


C. Development outlook
----------------------------

Highest priority:

- Currently the screen is black during the code execution (led_blinking.zip).

- We have to find out, how to load the original firmware to get into the normal operation mode where we can see the camera menu.

- After that we can try to get the magic lantern menu showing up on the screen and porting all functions to get magic lantern works correctly.


Low priority: Highest priority:

Figured out how to set the bootflag correctly to auto load the AUTOEXEC.BIN.

If it was that simple, why Coutts comment the code section out with the functions call in his latest release?

Translating his comments of the latest release, he had troubles to get it working correctly.

SysCall

  • New to the forum
  • *
  • Posts: 33
Re: Compiling for 1000D/XS
« Reply #29 on: December 05, 2016, 08:12:06 PM »
@shmadul, @levas

Thanks guys, but I have read a lot so far and the canon 1000d is often mentioned to be similar to this and that camera.

Sometime it is similar to 400 plus and sometimes you should use the 5Dc for the port and then maybe the 450D.


Some example, by Coutts himself:

https://chdk.setepontos.com/index.php?topic=6238.msg86614;topicseen#msg86614

https://www.magiclantern.fm/forum/index.php?topic=2054.0


We have to clarify first, which one makes sense and worth the effort.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10180
  • 5D Mark Free
Re: Compiling for 1000D/XS
« Reply #30 on: December 05, 2016, 08:20:21 PM »
A little more stuff:

There are two parts of the firmware: the bootloader and the main firmware.

The bootloader can:
- perform some low-level configuration (e.g. the MPU*) registers, which can give hints about the memory map)
- load FIR files
- load AUTOEXEC.BIN (if the boot flag is enabled)
- show a small diagnostic menu (aka FROMUTILITY)
- jump to main firmware

*) MPU = memory protection unit (see the ARM docs). Not to be confused with what Canon calls MPU (a microcontroller).

In DIGIC 5 and earlier, the bootloader starts at 0xFFFF0000 (aka HIVECS or high exception vectors in ARM docs).

The two parts are pretty much independent: once in bootloader context, your program can call other bootloader routines, and once in main firmware, you can call main firmware routines. If you mix them, it won't work. Exception: some very simple routines that don't use any state (such as strlen).

ML runs alongside main firmware. In order to do that, it has to modify the startup routine, reserve some memory (so Canon firmware won't overwrite it) and launch one user task. Once you did that, our code is running as a regular task on Canon's operating system (be it DryOS or VxWorks). From here, it can launch other tasks, call file I/O routines or whatever.

For trying out things, you can compile the "recovery" branch from ML, from platform/portable.000. This will give you a portable autoexec.bin that runs on all cameras (and also in QEMU), and has display access (from the bootloader context). Of course, this autoexec can be signed as FIR and loaded without requiring a boot flag.

The boot flag can be enabled from both bootloader context (as done in the old 5D) or from main firmware. EnableBootDisk only works from main firmware, but usually there is an equivalent bootloader routine, found in the FROMUTIL menu.

Ant123

  • Freshman
  • **
  • Posts: 68
Re: Compiling for 1000D/XS
« Reply #31 on: December 05, 2016, 08:20:34 PM »
C. Development outlook
----------------------------

Highes priority:

- Currently the screen is black during the code execution (led_blinking.zip).

- We have to find out, how to load the original firmware to get into the normal operation mode where we can see the camera menu.

- After that we can try to get the magic lantern menu showing up on the screen and porting all functions to get magic lantern works correctly.


Low priority:

Figured out how to set the bootflag correctly to auto load the AUTOEXEC.BIN.

If it was that simple, why Coutts comment the code section out with the functions call in his latest release?

Translating his comments of the latest release, he had troubles to get it working correctly.


If you want to convert 450D port, the first thing you should do is set bootflag and check it with help of AUTOEXEC.BIN from "display test" topic. It's because all VxWorks ports use AUTOEXEC.BIN startup method.

Then you should find addresses of functions for 1000D in your firmware dump and change them in "\platform\450D.110\stubs.S". It can take many days or weeks.

After this you should edit cache related stuff in "\platform\450D.110\init.c", and edit another files in "\platform\450D.110\" and in "\src"


Levas

  • Hero Member
  • *****
  • Posts: 873
  • 6d - Nightly build user
Re: Compiling for 1000D/XS
« Reply #32 on: December 05, 2016, 08:51:13 PM »
Great to see Alex arrive in this post  :D

Indeed the 1000d and 450d have a lot in common, but I'm not sure if that helps a lot with porting ML.
Since we know that even different firmwares need different ports, like in the 5d3.

It seems that most of the work is in finding the right addresses in the firmware.
But for that we need to have a firmware dump.

Which, as far as I know, isn't available for the 1000d ?
So is the first thing to do now, get a firmware dump ?
And how, can we write a program to make the 1000d do a firmware dump on the SD card.
Or can we just use the FIR file from the 1000d 1.0.7 update from the Canon site ?

And Alex, are there advantages to make it run in QEMU, or can we just try our stuff out on our real alive and kicking 1000d's, how big is the risk of bricking them ?

Levas

  • Hero Member
  • *****
  • Posts: 873
  • 6d - Nightly build user
Re: Compiling for 1000D/XS
« Reply #33 on: December 05, 2016, 08:56:26 PM »
Just wondering, is there a moderator who can change the title of this topic?
Since it's no longer about compiling, but more about, porting ML to 1000d ?
Or at least, small steps in trying to port ML to the 1000d  :P

Ant123

  • Freshman
  • **
  • Posts: 68
Re: Compiling for 1000D/XS
« Reply #34 on: December 05, 2016, 09:06:58 PM »
Which, as far as I know, isn't available for the 1000d ?
So is the first thing to do now, get a firmware dump ?
And how, can we write a program to make the 1000d do a firmware dump on the SD card.

Look it on CHDK forum. There is 1000D dump in their collection.

SysCall

  • New to the forum
  • *
  • Posts: 33
Re: Compiling for 1000D/XS
« Reply #35 on: December 05, 2016, 09:22:08 PM »
@ a1ex and Ant123

Thank you very much.

I will look into it.

shmadul

  • New to the forum
  • *
  • Posts: 28
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #36 on: December 05, 2016, 09:34:40 PM »
@Levas, Changed the Title as you suggested :)

shmadul

  • New to the forum
  • *
  • Posts: 28
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #37 on: December 05, 2016, 09:58:52 PM »
Where does the autoexec.bin get its code from ? I think it already boots to autoexec.bin but cant tell because the autoexec.bin is encrypted .....

SysCall

  • New to the forum
  • *
  • Posts: 33
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #38 on: December 06, 2016, 06:55:50 PM »
@ a1ex

Quote

The boot flag can be enabled from both bootloader context (as done in the old 5D) or from main firmware. EnableBootDisk only works from main firmware, but usually there is an equivalent bootloader routine, found in the FROMUTIL menu.

Is there any differences, if the boot flag was set with boot loader or from the main firmware routines?

Ant123

  • Freshman
  • **
  • Posts: 68
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #39 on: December 06, 2016, 07:24:04 PM »
on 450D this code works well.

SysCall

  • New to the forum
  • *
  • Posts: 33
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #40 on: December 06, 2016, 07:36:44 PM »
@ Ant123

I'm sorry Ant123, I don't understand.


Btw, it looks like I got autoboot working too, at least when I change the AUTOEXEC.BIN I see that it is automatically executed.

Also after switching the camera off and on I see my code running, I also get into main Canon menu. So I guess the firmware just boot normally and the code runs in parallel.

I want to confirm hat as you said by using the display test.

Quote
the first thing you should do is set bootflag and check it with help of AUTOEXEC.BIN from "display test" topic.

And also

Quote
For trying out things, you can compile the "recovery" branch from ML, from platform/portable.000. This will give you a portable autoexec.bin that runs on all cameras (and also in QEMU), and has display access (from the bootloader context). Of course, this autoexec can be signed as FIR and loaded without requiring a boot flag

Unfortunately I get noises as image.

a1ex mentioned, he changed a value for the canon 1000d

Quote
(I'm placing the image buffer at 0x4000000 = 64MB, so this may explain the issue)

http://www.magiclantern.fm/forum/index.php?topic=14732.msg172723#msg172723

Ant123

  • Freshman
  • **
  • Posts: 68
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #41 on: December 06, 2016, 07:49:44 PM »
Btw, it looks like I got autoboot working too, at least when I change the AUTOEXEC.BIN I see that it is automatically executed.

Also after switching the camera off and on I see my code running, I also get into main Canon menu. So I guess the firmware just boot normally and the code runs in parallel.

I want to confirm hat as you said by using the display test.

I.e. you are sure that BOOTFLAG was successfully set and AUTOEXEC.BIN from there works?

SysCall

  • New to the forum
  • *
  • Posts: 33
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #42 on: December 06, 2016, 07:51:13 PM »
Ok, fixed it myself.

In Trammell_Hudson_ML/src/disp_direct.c

Replace this

Code: [Select]
static uint8_t *disp_framebuf = (uint8_t *)0x04000000;

with this

Code: [Select]
static uint8_t *disp_framebuf = (uint8_t *)0x00400000;

Ant123

  • Freshman
  • **
  • Posts: 68
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #43 on: December 06, 2016, 07:58:34 PM »
Do you get picture like this using autoexec.bin, not *.FIR?

SysCall

  • New to the forum
  • *
  • Posts: 33
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #44 on: December 06, 2016, 08:08:38 PM »
@ Ant123

Well looking at the image, I don't know what it is all about to be honest, especially what the BOOT value means, guess BOOT=-1 is wrong?

https://1drv.ms/f/s!AsC1K_kH7N9pbYhpDPUbSuC8Iss

But I can confirm, I can change the AUTOEXEC.BIN only and it will executed after each restart:
- removed battery and put it back
- switch the camera off and on

Maybe I should make a video.

Ant123

  • Freshman
  • **
  • Posts: 68
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #45 on: December 06, 2016, 08:21:33 PM »
Well looking at the image, I don't know what it is all about to be honest, especially what the BOOT value means, guess BOOT=-1 is wrong?
Read that topic from the begining to the end again.

If BOOTFLAG is set you will see "BOOT=-1" ( -1 == 0xFFFFFFFF)

SysCall

  • New to the forum
  • *
  • Posts: 33
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #46 on: December 06, 2016, 08:24:33 PM »
Video is online, same link as previous post.

Btw, why is your flag set?

You got it already running?



Quote
If BOOTFLAG is set you will see "BOOT=-1" ( -1 == 0xFFFFFFFF)


Guess my boot flag is set then, look at my link.

Ant123

  • Freshman
  • **
  • Posts: 68
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #47 on: December 06, 2016, 08:34:56 PM »
Video is online, same link as previous post.
Now I see. But why you are playing with LEDs and don't modifying VxWorks branch. What is your final objective?

Quote
Btw, why is your flag set?
You got it already running?
My picture of 450D display test was made before BOOTFLAG was set.

SysCall

  • New to the forum
  • *
  • Posts: 33
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #48 on: December 06, 2016, 08:46:53 PM »
Quote
Now I see. But why you are playing with LEDs and don't modifying VxWorks branch

Start simple :)


What I did was, in Coutts latest release, I just comment out the following code section.

Code: [Select]
// EnableBootDisk() 
Funktion f = 0xFFD21248;
f();

After that, I made the SD card bootable by executing in terminal:

Code: [Select]
sudo ./make_bootable.sh

After executing .fir the first time, I don't need it anymore and the camera loads the AUTOEXEC.BIN automatically.

Ant123

  • Freshman
  • **
  • Posts: 68
Re: Porting ML to 1000D (AKA Compiling for 1000D/XS)
« Reply #49 on: December 06, 2016, 08:56:47 PM »
Start simple :)

What I did was, in Coutts latest release

I don't know what is "Coutts latest release" for.
I was working only with VxWorks ML ports (40D and 5DC ) to get ML on 450D with minimum functionality .