Author Topic: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)  (Read 78181 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12141
  • Out for a while, taking a short break.
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #75 on: September 08, 2018, 12:21:00 PM »
Nice.

The next step is to enable the boot flag (it will toggle a flag in the ROM so the firmware will load and execute code from the SD card). That appears to be a diagnostic feature in Canon's bootloader, present since DIGIC 2 or maybe earlier. It's the first time we are trying this procedure on DIGIC 7; the same code was tested on 5DS first, then on 80D, 750D, 760D, and 7D2 without surprises.

Very old versions of the boot flag enabling code caused some issues in the past (years ago): when they were run from LiveView, the "erase" step of the ROM flashing procedure succeeded, but the "write" step failed; as a result, the entire boot flag area was set to FFFFFFFF. Luckily, this was still a valid configuration with a ML card inserted, but the camera refused to boot plain Canon firmware and asked for a firmware update. Placing Canon's firmware update on the card fixed the issue.

Now we are running this code directly from bootloader. Source code is on the recovery branch (i.e. bootloader experiments, with portable code that runs on all EOS cameras from DIGIC 2 to DIGIC 8 ), compiled from platform/portable.000 with CONFIG_BOOT_BOOTFLAG = y. You may review this code by running it in QEMU if you prefer.

If you (or anyone else) are OK with the risks, just send me a PM and I'll give you the FIR file.

In the emulator, if you run with firmware="boot=1", this option enables the boot flag, so you can see what exactly the firmware does differently. It will check the card contents at every startup, so there might be a small delay after this change. Other than that, with a regular (non-bootable) card is inserted, the camera will run plain Canon firmware.

edoardoc

  • New to the forum
  • *
  • Posts: 8
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #76 on: September 08, 2018, 01:29:34 PM »
In the emulator ( " ./run_canon_fw.sh 60D,firmware="boot=1" " ) I got this:



Is it ok?

Before compiling ML ( " make -C ../magic-lantern 60D_install_qemu " )  I got the same screen of ML Rescue that I got directly from my camera.
Canon EOS 200D/SL2

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12141
  • Out for a while, taking a short break.
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #77 on: September 08, 2018, 04:42:22 PM »
The emulation doesn't go very far; it doesn't show the GUI yet. It covers the bootloader, SD card I/O and roughly half of the startup process. At some point it times out waiting for the second CPU, as the communication between them is not well understood.

To try the boot flag enabler:
Code: [Select]
cd magic-lantern
hg up recovery -C
cd platform/portable.000
make clean && make install_qemu ML_MODULES= CONFIG_BOOT_BOOTFLAG=y # optionally CONFIG_QEMU=y for more debugging info
( cd ../../../qemu-eos && ./run_canon_fw.sh 200D,firmware="boot=1" )

In this simulation, the boot flag would be already enabled, so the above won't show anything interesting (Canon code will attempt to flash the ROM only if the flag is actually changed). You could try to disable the boot flag instead (like this), as the emulation runs with boot flag enabled. In this case, you will see some low-level activity and the emulator will lock up. That's OK. It will print the address of the function it's going to call from Canon's bootloader, and you can look it up in the disassembly to confirm it's indeed the right one and has the correct syntax ( I already did that, but you should probably not trust me blindly :D ).

edoardoc

  • New to the forum
  • *
  • Posts: 8
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #78 on: September 08, 2018, 08:03:46 PM »
Which file should I edit to make this changes?
Canon EOS 200D/SL2

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12141
  • Out for a while, taking a short break.
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #79 on: September 08, 2018, 09:48:32 PM »
First two lines from the patch are telling you the file name:
Code: [Select]
--- a/src/reboot.c
+++ b/src/reboot.c

You should also be on the "recovery" branch.

edoardoc

  • New to the forum
  • *
  • Posts: 8
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #80 on: September 09, 2018, 12:33:13 PM »
Thanks.

I disabled the boot flag from reboot.c and I ran the emulation with firmware="boot=1" , here's the result:




Reading reboot.c I noticed that ( line 842 ) :
Code: [Select]
/* DIGIC 7: the main ROM is at 0xE0000000 and there is another one at 0xF0000000, so it should be OK */
from my screenshot:
Code: [Select]
ROMBASEADDR: 0xE0040000
Why my ROM starts after the addr in the previous comment?

And... what's the next step? Loading the FIR with the bootflag enabled from a real camera?
Canon EOS 200D/SL2

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12141
  • Out for a while, taking a short break.
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #81 on: September 09, 2018, 04:33:13 PM »
0xE0000000 is the start address of the first ROM chip (we call it ROM0).

0xE0040000 is the address of Canon's main firmware. Below that address, you can find Canon's bootloader; they are both in the same physical ROM chip.

Yes, the next step is running the FIR on real hardware and praying things won't go wrong.

edoardoc

  • New to the forum
  • *
  • Posts: 8
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #82 on: September 09, 2018, 11:44:19 PM »
Very old versions of the boot flag enabling code caused some issues in the past (years ago): when they were run from LiveView, the "erase" step of the ROM flashing procedure succeeded, but the "write" step failed; as a result, the entire boot flag area was set to FFFFFFFF. Luckily, this was still a valid configuration with a ML card inserted, but the camera refused to boot plain Canon firmware and asked for a firmware update. Placing Canon's firmware update on the card fixed the issue.
  • Is it the only problem?
  • What should I expect installing this FIR?
  • Should I copy both the ML's FIR and the Canon's firmware update on the same SD card to be safe?

Sorry for the questions but I just want to make sure what I'm doing.
Canon EOS 200D/SL2

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12141
  • Out for a while, taking a short break.
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #83 on: September 09, 2018, 11:54:09 PM »
That's the only surprise we had so far with this procedure, but anecdote != data. Anything can happen.

Expected outcome - see e.g. for 80D.

If you copy both FIR files, Canon's bootloader will just pick one. So, you should copy only one FIR.

edoardoc

  • New to the forum
  • *
  • Posts: 8
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #84 on: September 11, 2018, 03:07:49 PM »
Loading the FIR that enable the bootflag will avoid the warranty?

Can I reset the bootflag in the future?




Inviato dal mio iPhone utilizzando Tapatalk
Canon EOS 200D/SL2

ids1024

  • New to the forum
  • *
  • Posts: 3
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #85 on: September 16, 2018, 04:29:25 PM »
Loading the FIR that enable the bootflag will avoid the warranty?

There is an FAQ entry for that: https://wiki.magiclantern.fm/faq#does_it_void_my_warranty

Can I reset the bootflag in the future?

Yes.


Dennis_K

  • New to the forum
  • *
  • Posts: 1
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #86 on: September 30, 2018, 02:24:06 PM »
I own a 77D and I would love to help the development. Is there anything I can do? Unfortunately, I have zero expereience with coding.

aprofiti

  • Member
  • ***
  • Posts: 162
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #87 on: October 01, 2018, 09:07:12 PM »
I'm trying to find stubs for 77D, using 200D as a reference.

This is what I have at the moment:
Code: [Select]
/** Startup **/
NSTUB( ROMBASEADDR, firmware_entry )
NSTUB(0xE00400EC,  cstart)                 /* calls bzero32 and create_init_task(..., init_task, ...) */
NSTUB(0xDF00D285,  bzero32)                /* zeros out a data structure. From sub_E0428334
      LDR             PC, =(loc_DF00D284+1) */

NSTUB(0xDF006515,  create_init_task)       /* low-level DryOS initialization. From sub_E0427890
      LDR             PC, =(sub_DF006514+1) */
NSTUB(0xE0040215,  init_task)              /* USER_MEM size checking, dmSetup, termDriverInit, stdlibSetup etc */
NSTUB(0xe065e278,  dcache_clean)           /* loop with MCR p15 c7,c10,1; DSB */
NSTUB(0xe065e34c,  icache_invalidate)      /* loop with MCR p15 c7,c5,1; c7,c1,6; c7,c1,0; ISB */

/** Tasks **/
NSTUB(0xDF008F7E,  task_create)            /* used to start TaskMain, GuiMainTask etc */
NSTUB(0xDF0087FE,  msleep)                 /* argument is always multiple of 10 */
//NSTUB(    0x????,  current_task)           /* from task_create; pointer to the current task structure */
//NSTUB(    0x????,  current_interrupt)      /* from interrupt handler (VBAR + 0x18); where the interrupt ID is stored */

/** Dumper **/
NSTUB(0xe007fc46,  dump_file)              /* tries to save a file to either "A:/%s" or "B:/%s"; calls FIO_RemoveFile/CreateFile/WriteFile/CloseFile/Flush */

/** Memory info **/
NSTUB(0xe02640b4,  malloc_info)            /* Malloc Information */
NSTUB(0xe026414c,  sysmem_info)            /* System Memory Information */
NSTUB(0xe01eaf80,  memmap_info)            /* Exception vector, DRYOS system memory etc */
NSTUB(0xe0164ca6,  smemShowFix)            /* Common Lower, Common Upper etc */

/** Memory allocation **/
NSTUB(0xDF0079D3,  GetMemoryInformation)   /* called from AllocateMemory */

/** Debug messages **/
NSTUB(0xDF006E6D,  DryosDebugMsg)          /* lots of debug messages; format string is third argument */

/** Eventprocs (call by name) **/
NSTUB(0xe04d8aee,  call)                   /* many functions called by name (lv_start, lv_stop etc) */

/** GUI timers **/
NSTUB(0xe04d499a,  CancelTimer)            /* from error message */
NSTUB(0xe05aad76,  SetHPTimerAfterNow)     /* from error message */
NSTUB(0xe05aadca,  SetHPTimerNextTick)     /* same "worker" function as SetHPTimerAfterNow */
NSTUB(0xe04d48e4,  SetTimerAfter)          /* from error message */

/** Interrupts **/
//NSTUB(    0x????,  pre_isr_hook)
//NSTUB(    0x????,  post_isr_hook)
//NSTUB(   0x?????,  isr_table_handler)
//NSTUB(   0x?????,  isr_table_param)

/** MPU communication **/
NSTUB(0xE01E781F,  mpu_send)                  // "dwSize < TXBD_DATA_SIZE"
NSTUB(0xE058866B,  mpu_recv)                  // passed as last argument by InitializeIntercom and eventually stored into mpu_recv_cbr
NSTUB(    0x7CF4,  mpu_recv_cbr)              // mpu_recv is called indirectly through this function pointer
NSTUB(   0x887D4,  mpu_send_ring_buffer)      // ring buffer used in mpu_send
NSTUB(    0x7CD8,  mpu_send_ring_buffer_tail) // ring buffer index incremented in mpu_send
NSTUB(   0x88694,  mpu_recv_ring_buffer)      // ring buffer used in SIO3_ISR, subroutine that processes two chars at a time
NSTUB(    0x7CD0,  mpu_recv_ring_buffer_tail) // ring buffer index incremented in the above subroutine

/** Misc **/
NSTUB(0xe11f93d4,  vsnprintf)              /* called near dmstart; references "01234567", "0123456789", "0123456789abcdef" and "0123456789ABCDEF"; second arg is size; the one called by DebugMsg only knows %s */
Still missing a couple of references to data structures.
After figuring out it should be able to run the same minimal code as current state of 200D.

If someone want to double check and find the remaining stubs, please write down here your finding.

EDIT: More address found. Still missing last isr related stubs
Code: [Select]
NSTUB(    0x1020,  current_task)           /* from task_create; pointer to the current task structure */
NSTUB(    0x1008,  current_interrupt)      /* from interrupt handler (VBAR + 0x18); where the interrupt ID is stored */

/** Interrupts **/
//NSTUB(    0x????,  pre_isr_hook)
//NSTUB(    0x????,  post_isr_hook)
NSTUB(   0x6D0C0,  isr_table_handler)
//NSTUB(   0x?????,  isr_table_param)

jack001214

  • New to the forum
  • *
  • Posts: 13
  • EOS 200D
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #88 on: October 07, 2018, 11:23:29 AM »
does ML even work for 200D?
add me on minecraft

Mikerofilm

  • New to the forum
  • *
  • Posts: 2
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #89 on: October 07, 2018, 04:45:00 PM »
Through rom dumpers are we able to expand the range of the video codec on the 6d mk2?

Walter Schulz

  • Hero Member
  • *****
  • Posts: 6642
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #90 on: October 07, 2018, 06:33:32 PM »
Misconception: ROM dumpers read ROM data from cam. That's a very basic step in ML development. Don't hold your breath waiting for ML files loaded from card and ML code running in the cam. Devs just taking the very first steps in a very, very long journey.

Don't know what you expect in "codec range expanding" but you may take a look into what ML enabled cams are actually able to do. You may be disappointed, though.
Photogs and videographers: Assist in proof reading upcoming in-camera help!. Your input is wanted and needed!

jack001214

  • New to the forum
  • *
  • Posts: 13
  • EOS 200D
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #91 on: October 10, 2018, 06:23:52 PM »
What if i want to run some simple code on my 200D? Like a simple calculation and store it in the RAM or some accessible space. What can i do too access the full CPU instructions in assembler? I am not looking for fancy Screen gui's and touch or anything above those categories.
add me on minecraft

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12141
  • Out for a while, taking a short break.
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #92 on: October 10, 2018, 07:23:07 PM »
You can already do that in the emulator - see e.g. reply #83. For example, a simple intervalometer coded in C would be very easy to write.

After enabling the boot flag, you will be able to do the same on real hardware. Proof of concept code was already tested and confirmed to work. I'm looking for a volunteer willing to be the first one who enables the boot flag on DIGIC 7 (see previous page); nobody took the risk yet. The FIR file for enabling the boot flag for 200D is ready, just drop me a PM if (or when) you are prepared to run it.

jack001214

  • New to the forum
  • *
  • Posts: 13
  • EOS 200D
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #93 on: October 11, 2018, 09:24:06 AM »
I'm ready
add me on minecraft

juarez13

  • New to the forum
  • *
  • Posts: 2
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #94 on: October 29, 2018, 11:11:35 PM »
I didnt undertand how to install the magic lantern on 77d, someone can help?

Jonn3y

  • New to the forum
  • *
  • Posts: 2
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #95 on: October 30, 2018, 06:18:09 PM »
I didnt undertand how to install the magic lantern on 77d, someone can help?

There is no tested and working ML for 77D yet :)

calle2010

  • New to the forum
  • *
  • Posts: 42
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #96 on: November 02, 2018, 03:43:50 PM »
Hi,

I've dumped the ROMs of my 77D and created a build environment that brings me up to this point:





I also did the test with changing reboot.c to disable the boot flag. Is the output of this still relevant for anybody?

I have a few questions now:
- I unterstand the next step is to collect the stubs. Is there already a more complete list than aprofiti posted on October 1st?
- Has anybody already done the steps described here? https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/HACKING.rst?fileviewer=file-view-default#rst-header-adding-support-for-a-new-camera-model
- In branch "digic6-dumper" I see directory "platform/77D.100". Shouldn't this be "77D.102" since the current firmware version is 1.0.2?
- Is there a repository where people work together on porting to the 77D?

Thanks you so far for the good documentation. I hope I can help to get some steps further to a working port of ML for the 77D.

Cheers,
Christian.

PS: If you are interested on how I created my environment have a look at https://github.com/calle2010/magic-lantern-77d-vagrant. I use Vagrant and VirtualBox hosted on MacOs.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12141
  • Out for a while, taking a short break.
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #97 on: November 02, 2018, 07:38:20 PM »
PS: If you are interested on how I created my environment have a look at https://github.com/calle2010/magic-lantern-77d-vagrant. I use Vagrant and VirtualBox hosted on MacOs.

Very nice! Not familiar with vagrant, but if it makes easier to setup a build environment, it might be an interesting option.

I've tried Docker some time ago, but the experience wasn't straightforward. I couldn't install it on my main machine (OpenSUSE Tumbleweed), so I had to try it in a virtual machine.

Moving qemu.monitor into /tmp sounds interesting.

Quote
- I unterstand the next step is to collect the stubs. Is there already a more complete list than aprofiti posted on October 1st?

That's the most complete one. I didn't double-check them yet, only noticed the Thumb bit was not set in most of the stubs (and it should be; refer to 200D for details).

Quote
- Has anybody already done the steps described here? https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/HACKING.rst?fileviewer=file-view-default#rst-header-adding-support-for-a-new-camera-model

Emulation is at the same level as 200D and all other DIGIC 7 ports. It gets stuck as soon as the two cores expect to talk to each other.

I hope this experiment is going to capture the info needed to understand these interactions, but it has to be adapted to the ARMv7 architecture. The two cores are talking via MMIO registers, interrupts, and they both access the same memory (flat mapping, except for a private 4K page for each core, just like EOS M5).

Quote
- In branch "digic6-dumper" I see directory "platform/77D.100". Shouldn't this be "77D.102" since the current firmware version is 1.0.2?

That's right. Initial experiments on 77D were done before Canon published a firmware update.

Quote
- Is there a repository where people work together on porting to the 77D?

The digic6-dumper branch currently covers the DIGIC 6 and newer models, so... that's pretty much it. RE notes and other findings are shared on the forum.

Some bootloader experiments can be found in the "recovery" branch. That's portable code, running on all models since DIGIC 2.

BTW, we could successfully enable the boot flag on 200D; will post the FIR files soon.

calle2010

  • New to the forum
  • *
  • Posts: 42
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #98 on: November 03, 2018, 12:22:29 PM »
Very nice! Not familiar with vagrant, but if it makes easier to setup a build environment, it might be an interesting option.

I do only my first steps with Vagrant. I want to automate all the manual steps and setting up the build environment on my MacOS created too much clutter for my taste.

Moving qemu.monitor into /tmp sounds interesting.

This was just the first place that came to my mind. The working directory in this setup is on the VirtualBox filesystem mounted with nodev, so creation of the socket fails with "no permission" error message.

I didn't double-check them yet, only noticed the Thumb bit was not set in most of the stubs (and it should be; refer to 200D for details).

I will check the 200D code and see if I can find the same stubs for 77D. My assembler experience is very limited, though. Also I do not yet quite understand how to test the stubs without the GUI emulation.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12141
  • Out for a while, taking a short break.
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #99 on: November 04, 2018, 09:09:10 AM »
Also I do not yet quite understand how to test the stubs without the GUI emulation.

The emulation goes far enough to start a couple of Canon tasks; it even initializes the virtual SD card and is able to save logs. That's pretty much what can be tested at this stage.

Canon firmware even creates a DCIM directory when started from an empty card. From QEMU test results:
Code: [Select]
Testing file I/O (DCIM directory)...
     [...]
    77D: OK
   200D: OK
    6D2: OK
   800D: OK

Once the startup process works, you'll be able to get logs directly from the camera and start experimenting.