Canon EOS R / RP

Started by SpenceM, September 05, 2018, 03:09:27 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

garry23

@srsa

Thanks for the insight.

BTW I looked over on the CHDK site and now understand the limitations of Canon Basic.

Cheers

Garry

Nigel95

Quote from: coon on October 06, 2020, 12:30:25 AM
I don't get the colors right yet, but I can draw proper shapes on the screen. We have debug output now. :)


Great to see progress. As a noob I am wondering. In terms of any basic first version that requires testing. Is the current stage at something like 5% progress, 20%?
Is it possible at all to tell?

In no way I'm trying to push just curious how to interpret some posts over here as a noob. I will just think like there will never be a ML on this cams :) Thanks for the hard work.

names_are_hard

Early, 1% perhaps.  There is no real way to tell.  This is a puzzle game, you don't know how many levels there are, and you don't know if one of the levels will be a puzzle that is too hard for you to solve.

coon

All what we have done so far (and are still doing) is preparatory work so we will have a fundament for beginning the actual ML port.

What happened until now:
- FIR encryption on newer models has changed so we had no easy option anymore (only UART) to set the boot flag, which is required for ML to boot. Luckily Canon Basic support has been added which allowed it again, so we used it.

- Portable dumper was bootable after finding some required routines in the original firmware and after a little fix in the display code. This showed that we can execute arbitrary code in general. However, portable dumper starts before the firmware has booted and all hardware is initialized. Most of the cameras functions cannot be used at this stage. Therefore our next goal was to boot digic6-dumper, which is a minimal version of ML.

- After finding some more routines in the firmware we were able to boot digic6-dumper and let the SD card access LED blink. We can now execute arbitrary code while hardware is initialized. We can also use the LED to give us some feedback of what our code does.

- We found out where the frame buffers are and what their image format is. We are now able to draw on the screen and can use it for a very convenient debug output.

What happens next:
- Find out how the buttons work, so we can interact with our code and don't have to compile a new firmware every time if we want to test something. Maybe implementing some interactive RAM dumper where target address and dump size can be set by user for example. We could also use that to write to specific RAM addresses, to test if we found correct hardware addresses.
- Get stock firmware of RP run in QEMU.


Quote from: Nigel95 on October 06, 2020, 11:38:08 PM
Great to see progress. As a noob I am wondering. In terms of any basic first version that requires testing. Is the current stage at something like 5% progress, 20%?
Is it possible at all to tell?

In no way I'm trying to push just curious how to interpret some posts over here as a noob. I will just think like there will never be a ML on this cams :) Thanks for the hard work.

I don't expect anything usable before mid 2021.
EOS RP

Walter Schulz

What you wrote:

Quote from: coon on October 07, 2020, 11:14:18 AM
I don't expect anything usable before mid 2021.

What average user reads
Quote
You have the right to go ballistic if there isn't a full working ML at 01.07.2021 00:00:01.

coon

And then what? I am doing this in my free time. I am doing this to make my fancy camera even more fancy. Other people may profit from that as well. Nobody pays me so I don't owe someone something.

What I have meant: An unstable ML with minimal features, which I wouldn't recommend for daily use, could be available at the earliest mid 2021 for alpha testing, if we are very very lucky.
EOS RP

theBilalFakhouri

Quote from: coon on October 07, 2020, 11:59:03 AM
I don't owe someone something.

Of course, some users might build their dreams in that date you gave, like We will have 4K RAW capability on EOS RP in mid 2021! or something like that, they would think you gave a promise at that date, be prepared for some aggressive users :P

Otherwise the most users understand what porting ML is, and they understand what you mean.

Nice work by the way! :D

Nigel95

Good to clarify this up indeed. Work on the project whenever you want and try to enjoy it as much as possible. If you and others ever succeed to put ML on this camera that would be amazing! And if it doesn't get to that stage. Well at least you had fun in it if you work on it when you want and you also learned something along the way.

BTW is there a reason why ML is free? Has this something to do with Canon to avoid any law suits?

If it takes so much work for developers I think it would be fair to compensate them for firmware that works properly. (if it's stable and sort of reliable). Maybe people don't like to donate in advance in the hope there ever will be ML on a cam. Although for 'stable firmware' I think people would like to pay some money? This would be nice for developers to earn a bit of money by doing their hobby. It's probably not enough for all the hours that go into it. But it's always nice to gain a little bit of money by doing what you love.

Walter Schulz

Quote from: Nigel95 on October 07, 2020, 01:40:38 PM
If it takes so much work for developers I think it would be fair to compensate them for firmware that works properly. (if it's stable and sort of reliable). Maybe people don't like to donate in advance in the hope there ever will be ML on a cam. Although for 'stable firmware' I think people would like to pay some money?

Lookup your fallacy here. Poll result: https://mobile.twitter.com/autoexec_bin/status/1182941915337150464

Nigel95

Quote from: Walter Schulz on October 07, 2020, 01:49:43 PM
Lookup your fallacy here. Poll result: https://mobile.twitter.com/autoexec_bin/status/1182941915337150464

To lazy to read it all. Based on what I know about fallacies. I would assume you mean faulty generalization. Good one, it was my own opinion and shouldn't sell it as the opinion of a larger sample :)

Your comments make me laugh sometimes on the forum here. To the point although a bit cocky. Maybe it's the result of all the noob questions here. Maybe it's the way you are as a person. Or both?

coon

Finding button states in RAM is not an easy task (at least for me) so after hours of searching I have given up that route for now and did some research on the MPU instead.
I have used this post from a1ex as a reference: https://magiclantern.fm/forum/index.php?topic=2864.msg169517#msg169517

By reading mpu_recv_ring_buffer and using the LCD to display all received mpu messages I have retreived almost all button codes of RP:


Power Switch:               0x00 => params: on: 0x01, off: 0x00
half shutter:               0x05
shutter:                    0x06
* AE-Lock/FE-Lock           0x07
AF-ON:                      0x08
AF Point/Index/Magnify:     0x0B
M-Fn:                       0x0C
Q/set:                      0x0D
menu:                       0x0E
Playback:                   0x0F
info:                       0x10
Trashcan:                   0x11
movie shooting:             0x14
up:                         0x15
down:                       0x16
right:                      0x17
left:                       0x18
Main dial:                  0x19 => params: clockwise: 0x01, anti clockwise: 0xFF
Quick control dial:         0x1A => params: clockwise: 0xFF, anti clockwise: 0x01
multi-function lock switch: 0x1B
Mode dial:                  0x23 => params:
  0x01: C3
  0x02: Tv
  0x05: Movie
  0x06: P
  0x07: SCN
  0x08: Manual
  0x09: C2
  0x0A: Av
  0x0C: Bulb
  0x0D: C1
  0x0E: Fv
  0x0F: Auto


Reading mpu ring buffer via polling should make simple helper menus for convenient developing possible now.
EOS RP

71m363nd3r


coon

As already mentioned in Battery grip pins / UART Thread, I've managed to get access to physical UART on RP without the need of disassembling the camera. I have found out the following pinout so far:


1: (Unknown traffic on boot) (1v8)
2: Maybe RXICU (1v8)
3: TXICU (1v8)
4: GND
5: Maybe RXMPU (3v3)
6: TXMPU (3v3)
7: Unknown (3v3)
8: Unknown (3v3)


Here is some UART traffic I have received:

MPU output when switching on camera:

RSTFLG : 0010
E0ON
E1ON
openBattery

MON>>>sw task Initialize Complete
/USB_DET undetected
batt task Initialize Complete
adapter task Initialize Complete
temp_task.c:356 I2C Comm Error ret=1
disp task Initialize Complete
acc task Initialize Complete
setCCSnkRst
temp task Initialize Complete
Main Task receive from ICU ptr(0x20003940) size=0x04 cmmand=0x01 dataId=0x00
================SD send recode 0 0
Main Task receive from ICU ptr(0x20003940) size=0x0A cmmand=0x02 dataId=0x10
temp_task.c:421 I2C Comm Error ret=1
Main Task receive from ICU ptr(0x20003940) size=0x0B cmmand=0x0C dataId=0x00
Main Task receive from ICU ptr(0x20003940) size=0x0B cmmand=0x0C dataId=0x00
temp_task.c:421 I2C Comm Error ret=1
temp_task.c:421 I2C Comm Error ret=1
temp_task.c:421 I2C Comm Error ret=1
temp_task.c:421 I2C Comm Error ret=1
temp_task.c:421 I2C Comm Error ret=1
temp_task.c:421 I2C Comm Error ret=1


I don't know why there are so many I2C errors. Maybe because I didn't had any lens attached.

ICU output when switching on camera:

BootLoade[00]K433 READY
       31:      8.828 [STARTUP] K433 ICU Firmware Version 1.5.0 ( 3.8.7 )
      169:    116.237 [BATCHK] ERROR PROP_ACTIVE_SWEEP_STATUS : 0
      178:    116.696 [BATCHK] ERROR PROP_ACTIVE_SWEEP_STATUS : 0
      570:    155.915 [PA] WARN EventDispatch Current=1,dwEventID=3,dwParam=0x0
      980:    250.283 [PROPAD] ERROR GetPropertyData ID (0) = 0x0E090000
      981:    250.289 [PRP] ERROR NotFoundID:0x0E090000,Func:0xE0447AA4
     2002:    497.515 [GUI] ERROR m_hThisBDialog NULL !!
     2003:    497.645 [GUI] ERROR RefreshAfFrameSelectDialog NULL !!
     2083:    634.894 [FM] EV_INSERTION_COMPLETE : ID = 1(0), stat = 8192
     2337:    853.225 ------------------------------
     2338:    853.233 Camera Flag Current Setting
     2339:    853.238 ------------------------------
     2340:    853.242       14 Mode ON
     2341:    853.266 ------------------------------
     2427:    900.955 [GUI] ERROR RefreshAfFrameSelectDialog NULL !!
     2514:    948.596 ERROR [MEM] Over Permanet Memory cache 48 18100


Now this is a bit disappointing. I was expecting a more detailed output here. When using QEMU with gdb attached I can see some NotifyComplete messages. These would be interesting to see on real HW. Is there a way to enable those?
EOS RP

deviousfusion

Quote from: coon on September 18, 2020, 07:21:02 PM
Interesting. Shutter count also broken here or has some encoding scheme.
Thanks for providing this!

Could you edit your post and wrap the XML output into code blocks like this?

[code]
<xml code here>
[/code]

It's easier to read like this.

It meight be interesting getting a dump of a fresh RP with 0 shots taken to find out the origin value. But it will be very hard to find such I guess...

Not sure if this is still applicable or needed, but I just bought a RP and have taken a single shot. Here is my XML:

<?xml version="1.0"?>
  <Canon>
    <CameraInfo>
      <Serial>XXXXX</Serial>
      <FirmwareVer>
        <Internal>0.3.8.7</Internal>
        <Major>1.5.0</Major>
      </FirmwareVer>
      <ErrorList></ErrorList>
      <TotalShoot>1</TotalShoot>
      <TotalShutter>1086947310</TotalShutter>
      <PowerOnCount>16</PowerOnCount>
      <TotalRunningTime>1829</TotalRunningTime>
    </CameraInfo>
  </Canon>


Based on this the origin value appears to be 1086947309

coon

Canon has released Firmware Update 1.6.0 for RP recently which I have applied today. Canon Basic does still work. I have taken a dump already.

I also got some E70 errors in the error log, which I have triggered in the past by my code accidentally:


<ErrorList>
  <Kind>
  <ID>E70</ID>
  <Count>4</Count>
  <ErrorLog>
    <BatTemperature>max:0 min:0</BatTemperature>
    <FirstOccurTime>2020.10.03 00:28:07</FirstOccurTime>
    <LastOccurTime>2020.11.20 21:05:14</LastOccurTime>
    <Log>
      <DateTime>2020.11.20 21:05:14</DateTime>
      <Reason>DS-EID:101</Reason>
      <BatTemperature>0</BatTemperature>
      <LensID>00000000</LensID>
      <ReleaseCount>1844</ReleaseCount>
    </Log>
    <Log>
      <DateTime>2020.10.24 00:47:12</DateTime>
      <Reason>DS-EID:101</Reason>
      <BatTemperature>0</BatTemperature>
      <LensID>00000000</LensID>
      <ReleaseCount>1833</ReleaseCount>
    </Log>
    <Log>
      <DateTime>2020.10.24 00:46:04</DateTime>
      <Reason>DS-EID:101</Reason>
      <BatTemperature>0</BatTemperature>
      <LensID>00000000</LensID>
      <ReleaseCount>1833</ReleaseCount>
    </Log>
    <Log>
      <DateTime>2020.10.03 00:28:07</DateTime>
      <Reason>DS-EID:101</Reason>
      <BatTemperature>0</BatTemperature>
      <LensID>00000000</LensID>
      <ReleaseCount>1830</ReleaseCount>
    </Log>
  </ErrorLog>
  </Kind>
</ErrorList>


I have triggered them when trying to read from memory >= 0x8000000 via DMA access. I think these memory regions may be read via buffered reads.

Downgrade back to 1.5.0 was possible out of the box. No battery door trick or similiar was needed.
EOS RP

coon

Quote from: deviousfusion on October 30, 2020, 12:41:13 PM
Not sure if this is still applicable or needed, but I just bought a RP and have taken a single shot. Here is my XML:

<?xml version="1.0"?>
  <Canon>
    <CameraInfo>
      <Serial>XXXXX</Serial>
      <FirmwareVer>
        <Internal>0.3.8.7</Internal>
        <Major>1.5.0</Major>
      </FirmwareVer>
      <ErrorList></ErrorList>
      <TotalShoot>1</TotalShoot>
      <TotalShutter>1086947310</TotalShutter>
      <PowerOnCount>16</PowerOnCount>
      <TotalRunningTime>1829</TotalRunningTime>
    </CameraInfo>
  </Canon>


Based on this the origin value appears to be 1086947309

Sorry, I have overlooked your answer until now. Thanks for that interesting information! Yes, it seems the shutter count starts at 1086947309 for some reason.
EOS RP

kitor

Too many Canon cameras.
If you have a dead R or RP mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

DeafEyeJedi

Damn @coon, Great work and Keep it up!  ;D
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

coon

All information in this post do refer to EOS RP with firmware 1.6.0.

I have done some further investigations on VRAM handling and it seems that Canon has changed it on newer models. On my initial investigations I have done a full RAM dump and searched for the MARV signatures manually to find the VRAM buffers. This worked fine as a first entry point to draw simple things on the LCD but drawing in EVF or HDMI output did not work properly with this method. Therefore I have analysed the SaveVRAM event procedure to find out some more information about the buffer handling.

In the past the structure of struct bmp_vram_info was like this:


struct bmp_vram_info
{   
    struct MARV * vram1;        /* one of the two bitmap buffers - statically allocated? */
    struct MARV * vram2;        /* the other bitmap buffer */
    struct MARV * back_vram;    /* we need to write to the other one */
};


vram1 and vram2 are two buffers which are written in an alternating way to prevent flicker.
back_vram points to the back buffer.

On RP Canon has changed it to this:


struct bmp_vram_info
{   
    struct MARV * vram1;        /* one of the two bitmap buffers - statically allocated? */
    struct MARV * vram2;        /* the other bitmap buffer */
};


Plus now there is a global buffer index at 0xFF6C with the index of the current back buffer. It is either 0 or 1.

There are 3 YUV (UYVYAA) buffer pairs in total. One back and one front buffer each. Only one of the three buffer pairs seems to used:


vramBufferPairNo = 0: Final image displayed on LCD or EVF without sensor image
vramBufferPairNo = 1: Nullptr (Unknown)
vramBufferPairNo = 2: Nullptr (Unknown)


I don't know if buffer pairs 1 and 2 are ever used.

Formular for getting YUV buffers:


DAT_e0255238 + vramBufferPairNo * 0x28 + (1 - vram_current_buffer) * 4 - 0x5c


DAT_e0255238 is the dereferenced value on address 0xE0255238 which is 0x722B0.
vramBufferPairNo selects the buffer pair. Valid values are 0 to 2. It seems that buffer 1 and 2 are unused.
vram_current_buffer is the dereferenced value of 0xFF6C and is either 0 or 1.

There are 6 BGRA buffers in total. They are not double buffered and only two them are used in practice:


vramBufferNo = 0: Menu and Live View Overlays
vramBufferNo = 1: Focus point overlay
vramBufferNo = 2: Nullptr (Unknown)
vramBufferNo = 3: Nullptr (Unknown)
vramBufferNo = 4: Nullptr (Unknown)
vramBufferNo = 5: Nullptr (Unknown)


Formular for getting BGRA buffers:

DAT_e0331888 + vramBufferNo * 0xc



With this information I am now able to draw proper images on LCD, EVF and HDMI output. However, due to the pixel wise "software" rendering via CPU the current rendering performance with about 3 fps is quite bad.
I assume that actual rendering by Canon code is done in the BGRA buffers and are then composited and converted to YUV by some coprocessor. I don't know how to trigger that conversion in Canon firmware. Once I have found that out rendering speed should increase significantly. Until full ML does not run on RP, speed is not in focus so I will keep it like that for now.

Bad news are, that the new VRAM handling does require some adjustment in ML code. Good news are that R5 and R6 do also seem to use this new rendering method so they will profit from these changes as well. EOS R does still use the old method.
EOS RP

sombree

This Canon Basic script enables silent shutter on RP:
private sub Initialize()
    ExecuteEventProcedure("sht_EnableManualSilent")
end sub

Works in M, Av, Tv and P both in single and continuous modes. Works fine with interval timer. Fails with ERR70 in bulb.

c_joerg

Would be interesting to know whether you could also switch off the crop mode for EFS lenses.
EOS R

coon

Quote from: sombree on December 27, 2020, 07:32:03 PM
This Canon Basic script enables silent shutter on RP:
private sub Initialize()
    ExecuteEventProcedure("sht_EnableManualSilent")
end sub

Works in M, Av, Tv and P both in single and continuous modes. Works fine with interval timer. Fails with ERR70 in bulb.

Nice finding! It seems we have a first feature for RP :). Usually silent shutter is only possible in SCN mode or when performing a focus bracketing shoot but not in any other modes.
When do you get ERR70 in bulb? Directly after shutter press or when it tries to close the shutter?
EOS RP

coon

I have found a firmware version independent and read only method to find LED GPIO addresses of newer camera models and created a little Canon Basic script for that.
Sample usage for RP:


'LED GPIO dumper for EOS RP
dim gpio_base      = 0xD0130000
dim led_obj_ptr    = 0x4E28
dim led_obj_offset = 0x28
dim led_count_max  = 15

private sub save_led_gpio_addresses(fileName)
  RemoveFile(fileName)

  f = OpenFileCREAT(fileName)
  CloseFile(f)

  f = OpenFileWR(fileName)
  pLedObjectBase = *led_obj_ptr

  ledKind = 0

  do while ledKind < led_count_max
    pLedObject = pLedObjectBase + ledKind * led_obj_offset
    ledGpioOffset = pLedObject + 8

    WriteFileString(f, "[0x%08X] LED %d: 0x%08X\n", pLedObject, ledKind, gpio_base + *ledGpioOffset)
    ledKind = ledKind + 1
  loop

  CloseFile(f)
end sub

private sub Initialize()
  save_led_gpio_addresses("B:/LED_GPIOS.TXT")
end sub


The script saves the following content:


[0x00D2DE8C] LED 0: 0xD01300D4
[0x00D2DEB4] LED 1: 0xD0130000
[0x00D2DEDC] LED 2: 0xD01300D8
[0x00D2DF04] LED 3: 0xD0130000
[0x00D2DF2C] LED 4: 0xD0130000
[0x00D2DF54] LED 5: 0xD0130000
[0x00D2DF7C] LED 6: 0xD01300DC
[0x00D2DFA4] LED 7: 0xD0130000
[0x00D2DFCC] LED 8: 0xD0130000
[0x00D2DFF4] LED 9: 0xD0130000
[0x00D2E01C] LED 10: 0xD0130000
[0x00D2E044] LED 11: 0xD0130000
[0x00D2E06C] LED 12: 0xD0130000
[0x00D2E094] LED 13: 0xD0130000
[0x00D2E0BC] LED 14: 0xD0130000


The output suggests that there are three LEDs on RP (four LEDs if you also count the addresses ending with 0).


LED 0: 0xD01300D4 - Card LED
LED 2: 0xD01300D8 - Auto Focus Assist LED
LED 6: 0xD01300DC - Unknown / Nothing happens
Other LEDs: 0xD0130000 - Unknown / Nothing happens


Writing to these addresses will toggle the LEDs.
Values to toggle GPIOs on RP are:


LED_ON_BITMASK:  0x4D0002
LED_OFF_BITMASK: 0x4C0003


The script can also be run on other newer models. One has to find and set the following parameters for their camera model:


gpio_base
led_obj_ptr
led_obj_offset
led_count_max


The parameters can be found by inspecting led.set.directly function which can be found at 0xE0697DAA on RP 1.6.0 firmware. The function may also be named LEDDrv_SetLEDDirectly on other models.

Function signature is:


int led_set_directly(uint ledKind, int controlId)


Parameters for EOS R are:


'LED GPIO dumper for EOS R
dim gpio_base      = 0xD0130000
dim led_obj_ptr    = 0x4DDC
dim led_obj_offset = 0x24
dim led_count_max  = 13


Script from above with these values produces:


[0x00D7D7B4] LED 0: 0xD01300D4
[0x00D7D7D8] LED 1: 0xD0130000
[0x00D7D7FC] LED 2: 0xD01300D8
[0x00D7D820] LED 3: 0xD0130000
[0x00D7D844] LED 4: 0xD0130000
[0x00D7D868] LED 5: 0xD0130000
[0x00D7D88C] LED 6: 0xD0130000
[0x00D7D8B0] LED 7: 0xD0130000
[0x00D7D8D4] LED 8: 0xD0130000
[0x00D7D8F8] LED 9: 0xD0130000
[0x00D7D91C] LED 10: 0xD0130000
[0x00D7D940] LED 11: 0xD0130000
[0x00D7D964] LED 12: 0xD0130000


GPIO addresses are similiar to RP but LED 6 does also end with 0 here.

However, it's not trivial to get the values for R6 since the value for gpio_base is stored in RAM.
Could someone with an R6 execute this and tell me the result?


'Get R6 LED base

private sub save_led_base(fileName)
  RemoveFile(fileName)

  f = OpenFileCREAT(fileName)
  CloseFile(f)

  f = OpenFileWR(fileName)

  pLedBase = 0x4F54
  WriteFileString(f, "LED base 1: [0x%08X]: 0x%08X\n", pLedBase, *pLedBase)

  pLedBase = 0x4F5C
  WriteFileString(f, "LED base 2: [0x%08X]: 0x%08X\n", pLedBase, *pLedBase)

  CloseFile(f)
end sub

private sub Initialize()
  save_led_base("B:/LED_BASE.TXT")
end sub


EOS RP

sombree

Quote from: coon on December 28, 2020, 06:56:59 PM
Nice finding! It seems we have a first feature for RP :). Usually silent shutter is only possible in SCN mode or when performing a focus bracketing shoot but not in any other modes.
When do you get ERR70 in bulb? Directly after shutter press or when it tries to close the shutter?

Directly after shutter button press although it seems that camera starts taking picture - I can hear lens iris moving and card LED lights up (but no image saved on the card).

coon

I have found a function which is used to set some GPIO pins at 0xE03B4FDC.

Signature is:


void gpioSet(int gpioOffset, int onOffBitmask)


gpioOffset is added to the fixed address 0xD0130000.
onOffBitmask is mostly 0x4D0002 to set the GPIO and 0x4C0003 to reset the GPIO. Bitmasks are different on some addresses.

The function is used to toggle GPIO pins, and some chip enable pins. It is also used to toggle some GPIO pin to low at the end of send_mpu which meight be an interrupt pin for MPU. This function may be used to identify several GPIOs.
EOS RP