Canon EOS R / RP

[c_joerg]

"I don't now how to do all the things on the EOS R without ML (f. e. Intervalometer, Motion detection, the modules). "

At least simple things like Intervalometer, would work with Canon Basic.
https://chdk.setepontos.com/index.php?topic=13522.0

[kitor]

At least simple things like Intervalometer, would work with Canon Basic.

We have "factory" evprocs that ML calls for this reason. They work, unless they didn't and leave camera in weird state.

They have a bug in sw1/sw2 handling - callbacks are installed for that and if eg focusing fails, callback is not removed. It fires on next (even real) shot attempt, with null pointer (iirc on recursive lock) - crashing the camera.
If it even won't crash (if I reset CBRs myself after failed attempt), it leaves camera in a state where it thinks photo is developing but it is not.

If there's another possibility on EOS firmwares, I'll be happy to take it as like I mentioned on previous page I spent too much time on those functions that we used previously

[sast]

Ist hier noch etwas in der Entwicklung?

[sast]

Is there anything else in development here?

[Dmytro_ua]

Is there anything else in development here?

No news here = no news in development.
How hard to understand this? 

[coon]

Only boring internal stuff, but nothing usable for the end user. Research and development is still on going.

[stan101]

No news here = no news in development.
How hard to understand this? 

It's hard enough that it couldn't be understood in two languages, apparently.

[vincesd]

Only boring internal stuff, but nothing usable for the end user. Research and development is still on going.

is it possible for an early release? the shutter curtain close mod that you showed already has value for end user.

[kitor]

Some fun tests.

Benchmarks with the cheapest UHS-II card I was able to get (Lexar LSD64GCB1667)



In desktop UHS-II reader:



What about overexposure warning in LV?

Yes, ML has this feature on older cameras. But DIGIC registers doesn't work the same way as before.
I was able to alter threshold and highlight style by poking registers, but I'm not able to activate the function (yet?). For now it works like warning you can enable in Play mode.

[Walter Schulz]

Faster card here:

[kitor]

Two different firmware 1.8.0.
https://gdlp01.c-wss.com/gds/8/0400006288/01/eosr-v180-win.zip
https://gdlp01.c-wss.com/gds/8/0400006288/02/eosr-v180-win.zip

Confusion confirmed. There are at least two versions of R 1.8.0 firmware

One that all development is based on:

Code: [Select]

K424 ICU Firmware Version 1.8.0 ( 7.3.9 )
ICU Release DateTime Jul 27 2020 18:48:59
First file has modification date of 21.10.2020. It holds the same firmware as above, as 1.8.0 was publicly released on 19.11.2020.

One that we just found about (this one matches modification date of 2nd file):

Code: [Select]

K424 ICU Firmware Version 1.8.0 ( 7.4.0 )
ICU Release DateTime Feb 4 2021 17:01:29
They differ enough (everything is slightly shifted, differently aligned) that it will not work on newer version.

You can verify which one you have with Canon Basic script, mentioned on previous pages:

Code: [Select]

private sub Initialize()
    CamInfo_Debug(1)
end sub

It contains some info, including:

Code: [Select]

<FirmwareVer>
    <Internal>0.7.3.9</Internal>
    <Major>1.8.0</Major>
</FirmwareVer>


[c_joerg]

You can verify which one you have with Canon Basic script, mentioned on previous pages:

It contains some info, including:

Code: [Select]

<FirmwareVer>
    <Internal>0.7.3.9</Internal>
    <Major>1.8.0</Major>
</FirmwareVer>


I have a new EOS R, one month old that shows

Code: [Select]

<FirmwareVer>
    <Internal>0.7.4.0</Internal>
    <Major>1.8.0</Major>
</FirmwareVer>


[coon]

Currently we are doing some research on PTP. There is a tool called ptpcam` in https://github.com/coon42/magiclantern_simplified/tree/ptpcam_add_more_canon_codes/contrib/ptpcam which allows reading some camera info via PTP. Every camera exposes all PTP commands it supports which can be queried via PTP. I've added alot of missing PTP command ids to ptpcam and got the following list of command IDs by executing:

Code: [Select]

./ptpcam --list-operations

Code: [Select]

  0x1001: GetDeviceInfo
  0x1002: OpenSession
  0x1003: CloseSession
  0x1004: GetStorageIDs
  0x1005: GetStorageInfo
  0x1006: GetNumObjects
  0x1007: GetObjectHandles
  0x1008: GetObjectInfo
  0x1009: GetObject
  0x100a: GetThumb
  0x100b: DeleteObject
  0x100c: SendObjectInfo
  0x100d: SendObject
  0x100f: FormatStore
  0x1014: GetDevicePropDesc
  0x1016: SetDevicePropValue
  0x101b: GetPartialObject
  0x902f: UNKNOWN
  0x9033: UNKNOWN
  0x9050: PTP_OC_CANON_InitiateEventProc0
  0x9051: PTP_OC_CANON_TerminateEventProc_051
  0x905c: PTP_OC_CANON_InitiateEventProc1
  0x905d: PTP_OC_CANON_TerminateEventProc_05D
  0x9060: PTP_OC_CANON_IsNeoKabotanProcMode
  0x9068: PTP_OC_CANON_GetWebServiceSpec
  0x9069: PTP_OC_CANON_GetWebServiceData
  0x906a: PTP_OC_CANON_SetWebServiceData
  0x906b: PTP_OC_CANON_DeleteWebServiceData
  0x906c: PTP_OC_CANON_GetRootCertificateSpec
  0x906d: PTP_OC_CANON_GetRootCertificateData
  0x906e: PTP_OC_CANON_SetRootCertificateData
  0x906f: PTP_OC_CANON_DeleteRootCertificateData
  0x9077: PTP_OC_CANON_GetTranscodeApproxSize
  0x9078: PTP_OC_CANON_RequestTranscodeStart
  0x9079: PTP_OC_CANON_RequestTranscodeCancel
  0x9101: PTP_OC_CANON_GetStorageIDS
  0x9102: PTP_OC_CANON_GetStorageInfo
  0x9103: PTP_OC_CANON_GetObjectInfo
  0x9104: PTP_OC_CANON_GetObject
  0x9105: PTP_OC_CANON_DeleteObject
  0x9106: PTP_OC_CANON_FormatStore
  0x9107: PTP_OC_CANON_GetPartialObject
  0x9108: PTP_OC_CANON_GetDeviceInfoEX
  0x9109: PTP_OC_CANON_GetObjectInfoEX
  0x910a: PTP_OC_CANON_GetThumbEX
  0x910c: PTP_OC_CANON_SetObjectAttributes
  0x910f: PTP_OC_CANON_Remote_Release
  0x9110: PTP_OC_CANON_SetDevicePropvalueEX
  0x9114: PTP_OC_CANON_SetRemoteMode
  0x9115: PTP_OC_CANON_SetEventMode
  0x9116: PTP_OC_CANON_GetEvent
  0x9117: PTP_OC_CANON_TransferComplete
  0x9118: PTP_OC_CANON_CancelTransfer
  0x911a: PTP_OC_CANON_PCHDDCapacity
  0x911b: PTP_OC_CANON_SetUILock
  0x911c: PTP_OC_CANON_ResetUILock
  0x911d: PTP_OC_CANON_KeepDeviceON
  0x911e: PTP_OC_CANON_SetNullPacketMode
  0x911f: PTP_OC_CANON_UpdateFirmware
  0x9122: PTP_OC_CANON_SetWFTPROFILE
  0x9123: PTP_OC_CANON_GetWFTPROFILE
  0x9124: PTP_OC_CANON_SetPROFILETOWFT
  0x9127: PTP_OC_CANON_RequestDevicePropvalue
  0x9128: PTP_OC_CANON_RemoteReleaseON
  0x9129: PTP_OC_CANON_RemoteReleaseOFF
  0x912b: PTP_OC_CANON_ChangePhotoStudioMode
  0x912c: PTP_OC_CANON_GetPartialObjectEX
  0x912d: PTP_OC_CANON_ReSizeImageData
  0x912e: PTP_OC_CANON_GetReSizeData
  0x912f: PTP_OC_CANON_ReleaseReSizeData
  0x9130: PTP_OC_CANON_ResetMirrorLockupState
  0x9131: PTP_OC_CANON_PopupBuiltinFlash
  0x9132: PTP_OC_CANON_EndGetPartialObjectEX
  0x9133: PTP_OC_CANON_MovieSelectSWOn
  0x9134: PTP_OC_CANON_MovieSelectSWOff
  0x9135: PTP_OC_CANON_GetCtgInfo
  0x9136: PTP_OC_CANON_GetLensAdjust
  0x9137: PTP_OC_CANON_SetLensAdjust
  0x9138: PTP_OC_CANON_GetMusicInfo
  0x9139: PTP_OC_CANON_CreateHandle
  0x913a: PTP_OC_CANON_SendPartialObjectEx
  0x913b: PTP_OC_CANON_EndSendPartialObjectEx
  0x913c: PTP_OC_CANON_SetCtgInfo
  0x913d: PTP_OC_CANON_SetRequestOlcInfoGroup
  0x913e: PTP_OC_CANON_SetRequestRollingPitching
  0x913f: PTP_OC_CANON_GetCameraSupport
  0x9140: PTP_OC_CANON_SetRating
  0x9141: PTP_OC_CANON_RequestDevelopStart
  0x9143: PTP_OC_CANON_RequestDevelopEnd
  0x9144: PTP_OC_CANON_GetGpsLoggingData
  0x9145: PTP_OC_CANON_GetGpsLogCurrentHandle
  0x9146: PTP_OC_CANON_SetImageRecoveryData
  0x9147: PTP_OC_CANON_GetImageRecoveryList
  0x9148: PTP_OC_CANON_FormatRecoveryData
  0x9149: PTP_OC_CANON_GetPresetLensAdjustParam
  0x914a: PTP_OC_CANON_GetRawDispImage
  0x914b: PTP_OC_CANON_SaveImageRecoveryData
  0x914c: PTP_OC_CANON_BLERequest
  0x914d: PTP_OC_CANON_DrivePowerZoom
  0x914e: PTP_OC_CANON_SendTimeSyncMessage
  0x914f: PTP_OC_CANON_GetIptcData
  0x9150: PTP_OC_CANON_SetIptcData
  0x9153: PTP_OC_CANON_GetViewfinderData
  0x9154: PTP_OC_CANON_DoAF
  0x9155: PTP_OC_CANON_DriveLens
  0x9157: PTP_OC_CANON_ClickWB
  0x9158: PTP_OC_CANON_Zoom
  0x9159: PTP_OC_CANON_ZoomPosition
  0x915a: PTP_OC_CANON_SetLiveAFFrame
  0x915b: PTP_OC_CANON_TouchAfPosition
  0x915c: PTP_OC_CANON_SetLvPcFlavoreditMode
  0x915d: PTP_OC_CANON_SetLvPcFlavoreditParam
  0x9160: PTP_OC_CANON_AFCancel
  0x916b: PTP_OC_CANON_SetImageRecoveryDataEx
  0x916c: PTP_OC_CANON_GetImageRecoveryListEx
  0x916d: PTP_OC_CANON_CompleteAutoSendImages
  0x916e: PTP_OC_CANON_NotifyAutoTransferStatus
  0x916f: PTP_OC_CANON_GetReducedObject
  0x9170: PTP_OC_CANON_GetObjectInfo64
  0x9171: PTP_OC_CANON_GetObject64
  0x9172: PTP_OC_CANON_GetPartialObject64
  0x9173: PTP_OC_CANON_GetObjectInfoEx64
  0x9174: PTP_OC_CANON_GetPartialObjectEx64
  0x9177: PTP_OC_CANON_NotifySaveComplete
  0x9178: PTP_OC_CANON_GetTranscodedBlock
  0x9179: PTP_OC_CANON_TransferCompleteTranscodedBlock
  0x9180: UNKNOWN
  0x9181: UNKNOWN
  0x9182: PTP_OC_CANON_NotifyEstimateNumberofImport
  0x9183: PTP_OC_CANON_NotifyNumberofImported
  0x9184: PTP_OC_CANON_NotifySizeOfPartialDataTransfer
  0x9185: PTP_OC_CANON_NotifyFinish
  0x91ae: UNKNOWN
  0x91af: UNKNOWN
  0x91b9: PTP_OC_CANON_SetFELock
  0x91d3: UNKNOWN
  0x91d4: PTP_OC_CANON_SendCertData
  0x91d5: UNKNOWN
  0x91d7: PTP_OC_CANON_DistinctionRTC
  0x91d8: PTP_OC_CANON_NotifyGpsTimeSyncStatus
  0x91d9: UNKNOWN
  0x91da: UNKNOWN
  0x91db: UNKNOWN
  0x91dc: UNKNOWN
  0x91dd: UNKNOWN
  0x91de: UNKNOWN
  0x91df: PTP_OC_CANON_GetAdapterFirmData
  0x91e1: UNKNOWN
  0x91e2: UNKNOWN
  0x91e3: PTP_OC_CANON_ceresSEndScanningResult
  0x91e4: PTP_OC_CANON_ceresSEndHostInfo
  0x91e6: PTP_OC_CANON_NotifyAdapterStatus
  0x91e7: UNKNOWN
  0x91e8: PTP_OC_CANON_ceresNotifyNetworkError
  0x91e9: PTP_OC_CANON_AdapterTransferProgress
  0x91ea: PTP_OC_CANON_ceresRequestAdapterProperty
  0x91eb: UNKNOWN
  0x91ec: PTP_OC_CANON_ceresSEndWpsPinCode
  0x91ed: PTP_OC_CANON_ceresSEndWizardInfo
  0x91ee: UNKNOWN
  0x91ef: PTP_OC_CANON_ceresSEndBtSearchResult
  0x91f0: PTP_OC_CANON_TransferComplete2
  0x91f1: PTP_OC_CANON_CancelTransfer2
  0x91f2: PTP_OC_CANON_ceresGetUpdateFileData
  0x91f3: PTP_OC_CANON_NotifyUpdateProgress
  0x91f4: UNKNOWN
  0x91f5: PTP_OC_CANON_ceresSEndFactoryProperty
  0x91f6: UNKNOWN
  0x91f8: PTP_OC_CANON_ceresSEndBtPairingResult
  0x91f9: PTP_OC_CANON_ceresNotifyBtStatus
  0x91fb: UNKNOWN
  0x91fc: PTP_OC_CANON_SendTimeSyncInfo
  0x91fd: PTP_OC_CANON_SetAdapterBatteryReport
  0x91fe: PTP_OC_CANON_fapiMessageTX
  0x91ff: PTP_OC_CANON_fapiMessageRX

We may also execute this on other cameras while being connected via USB for getting an overview what is possible with PTP for each model.

[coon]

RP can run Doom btw:

In this hack ML is loaded into 4K clean HDMI buffer 0 and Doom into 4k clean HDMI buffer 1, as Doom needs a plenty amount of code and work ram size. Both buffers offer 4MB of memory each. Thus one shouldn't enable 4K clean HDMI while playing Doom or Canon firmware will overwrite ML and Doom code and camera will crash horribly. I may port this into a generic ML module later, so it can be run on any camera.

[vvzvlad]

Good afternoon, I have an RP camera, and I really suffer from the fact that with my favorite lens, the cropped frame is forced to turn on.
I would like to fix the firmware and make this feature not work. A very simple fix, no menus, no testing on many lenses, no syncing to the main ML code base. I think it would be on the level of the fix above, where the shutter closes when the camera is turned off, simple enough. I have decades of experience developing for microcontrollers, and some experience reverse-engineering firmware. I really want to try this, and if I succeed, I'll post the result.

However, I have a distinct lack of understanding of reverse engineering the firmware for EOS/ML.

1)How do I get the firmware from my camera? Can I take the file from the website, modify it and flash it as usual, or are there mechanisms out there that don't allow me to flash the modified file?
2)If I can't do that, do I have to download the firmware from the camera itself? I've seen it done by people in this thread, but it was sketchy and I never figured out exactly how to do it. Can it be done by downloading the code to the card or does it require access to the insides of the camera?
3)What are the risks in reading and writing firmware? Can I brick the camera in a way that requires something non-trivial steps to repair?
4)Should I fix the firmware code directly or can I write a small program which will run at startup and patch the camera's memory, to reduce the risk of the camera turning into a brick?
5)What is the best way to disassemble the firmware? Programs, tools?

Please feel free to RTFM and poke me in the documentation. I need directions, not a teacher's help at every step.

[kitor]

What about overexposure warning in LV?

Yes, ML has this feature on older cameras. But DIGIC registers doesn't work the same way as before.
I was able to alter threshold and highlight style by poking registers, but I'm not able to activate the function (yet?). For now it works like warning you can enable in Play mode.

Not valid anymore. I spent some time and found out that now this is a set of three registers, not a single one.

And both overexposure and underexposure warning can be enabled independently at the same time with different visual settings.

And while running in Clean HDMI mode, one can configure HDMI and LCD to display different things, with independent thresholds and styles

Complete registers documentation is on wiki.
This should apply to all Digic 8 models (+/- clean hdmi limitations), I need to explore code on Digic 6 and 7.

[e] Digic 6 and 7 use different register addresses and a bit different values, but capabilities are generally the same. I added those in Digic 6 section of Wiki.

PoC will follow soon.

[paradox87]

hi guys,
great to see that there's development on this build.

one question: how likely is it to have a canon R build at some point in the near future, shooting 14bit raw video in 4K, while keeping the AF methods available (eye tracking in particular).
I'm thinking whether to get a 5d mk III and enjoy ML, or wait for an R release.

Thanks.

[Walter Schulz]

one question: how likely is it to have a canon R build at some point in the near future, shooting 14bit raw video in 4K, while keeping the AF methods available (eye tracking in particular).

Not at all likely to come true this year.

[zLOST]

Hi,
and what about stripped-down ML with only LUA scripting and https://www.magiclantern.fm/forum/index.php?topic=18083 ?
That'd make my day/week/month/everything

cheers

[Walter Schulz]

LUA? Don't hold your breath.

[zLOST]

LUA? Don't hold your breath.

well, the fact is, that i've been using ML for nothing but this script on 6D + 650D and it's a pitty, that i can't do the same on R. but i haven't contributed to ML at all, nor do i have any skills to do that, so all i can do now is just silently wait

[Walter Schulz]

If there is no ML for your cam now act like there will be no ML for your cam ever.

[kitor]

There's no such thing as "only lua scripting" anyway.
Lua can only expose what is already supported by the build. If you strip something from build, you strip the lua ability to use that.

Not to mention that Lua is probably completely broken on Digic 6+, and will be not trivial to fix.

Existing code crashes camera with seemingly random reasons. We don't release anything for your own safety. Do not request any builds as those requests will be ignored.

Not at all likely to come true this year.

I see post was deleted? (or not approved by mod). Anyway,  there will be no RAW recording support, unless announced otherwise. Hardware is different.

[kitor]

However, I have a distinct lack of understanding of reverse engineering the firmware for EOS/ML.

Hi,
looks that I missed that post - since newcomers are moderated and may show up with delay.

1)How do I get the firmware from my camera? Can I take the file from the website, modify it and flash it as usual, or are there mechanisms out there that don't allow me to flash the modified file?

Easiest way to grab the firmware is via Canon Basic script - see General Development section, pinned topic.
We don't modify / flash original firmware - so I can't answer that question. Magic Lantern is altering DryOS bootloader in RAM, and then just runs as regular task(s) in DryOS.

2)If I can't do that, do I have to download the firmware from the camera itself?
I've seen it done by people in this thread, but it was sketchy and I never figured out exactly how to do it.
Can it be done by downloading the code to the card or does it require access to the insides of the camera?

Yes, because we do not redistribute firmware dumps. Those contain copyrighted Canon code.
You can access UART (Reverse engineering section, pinned topic) and do some things directly from bootloader, but that is not needed.

3)What are the risks in reading and writing firmware? Can I brick the camera in a way that requires something non-trivial steps to repair?

Reading - no risks. Writing - again, can't answer. Main firmware runs from 25xx flash on board so it is theoretically possible to desolder those and program externally.
If you break MPU firmware (hardware "hypervisor", controls buttons, some peripherlials comm and most important - power to main CPU (ICU)) it will get tricky, as this one stores firmware in internal flash - and @coon probably found Jtag there just last week. We don't touch MPU code though.

4)Should I fix the firmware code directly or can I write a small program which will run at startup and patch the camera's memory, to reduce the risk of the camera turning into a brick?

See 1a - we do exactly that. Patch in RAM (what we can) and just run our own tasks that alter camera state where possible.
On older models it was possible to use so called Cache hacks to patch ROM code on runtime - but this feature is missing from new generation CPUs.
In theory you can use MMU to remap whole pages of ROM with patched code - however we don't do that (yet). CHDK uses that functionality, just so far we didn't have to patch ROM directly (again - yet), thus it wasn't investigated.

5)What is the best way to disassemble the firmware? Programs, tools?

The best tool is the tool you know to use. Nowadays we mostly use Ghidra,

[s0v3r1gn]

I just got a Canon RP and I have some experience with Ghidra and reverse engineering some though nothing this complex.

I’d be interested in helping the project out if anyone on the development team could help to bring me up to speed.