Canon EOS R / RP

Started by SpenceM, September 05, 2018, 03:09:27 PM

Previous topic - Next topic

0 Members and 3 Guests are viewing this topic.

coon

I have found a JSON library Canon calls JSONPG or cJson and here are the stubs to all of its functions for EOS RP 1.6.0:


0xe0185d14: cJsonArrayList_Create
0xe0185f24: cJsonArrayList_DisplaceBack
0xe01862e8: cJsonArrayList_DisplaceForward
0xe0185e06: cJsonArrayList_ExtendBufferIfNecessary
0xe0185c28: cJsonArrayList_Finalize
0xe0185c60: cJsonArrayList_Free
0xe0185dae: cJsonArrayList_Get
0xe0185d7e: cJsonArrayList_GetCount
0xe0185c9c: cJsonArrayList_Initialize
0xe018638a: cJsonArrayList_Remove
0xe0186260: cJsonArrayList_Replace
0xe018bb4c: cJsonBufferInput_Create
0xe018bc8e: cJsonBufferInput_Finalize
0xe018bbca: cJsonBufferInput_Free
0xe018badc: cJsonBufferInput_Initialize
0xe018bc06: cJsonBufferInput_Read
0xe018657e: cJsonBufferOutput_Create
0xe01865ea: cJsonBufferOutput_ExtendBufferIfNecessary
0xe018648c: cJsonBufferOutput_Finalize
0xe01864c4: cJsonBufferOutput_Free
0xe0186500: cJsonBufferOutput_Initialize
0xe0186672: cJsonBufferOutput_Write
0xe018c010: cJsonCharProviderUtf8_Create
0xe018bebc: cJsonCharProviderUtf8_Finalize
0xe018bef8: cJsonCharProviderUtf8_Free
0xe018c1d0: cJsonCharProviderUtf8_Get
0xe018c22c: cJsonCharProviderUtf8_GetPosition
0xe018bf34: cJsonCharProviderUtf8_Initialize
0xe018c148: cJsonCharProviderUtf8_Peek
0xe018c0ba: cJsonCharProviderUtf8_ReadIfBufferEmpty
0xe0116254: cJsonDocument_CreateNodeFromTokenType
0xe0117778: cJsonDocument_FreeBuffer
0xe0116e0a: cJsonDocument_GenerateArray
0xe0117154: cJsonDocument_GenerateObject
0xe01172cc: cJsonDocument_GenerateRoot
0xe0117452: cJsonDocument_GenerateToBuffer
0xe0117626: cJsonDocument_GenerateToStream
0xe0116310: cJsonDocument_ParseArray
0xe01169ca: cJsonDocument_ParseFromBuffer
0xe0116af6: cJsonDocument_ParseFromStream
0xe01164a2: cJsonDocument_ParseObject
0xe0116806: cJsonDocument_ParseRoot
0xe0116f7e: cJsonDocument_WriteFieldByNodeType
0xe0116d1a: cJsonDocument_WriteValueByNodeType
0xe0496b78: cJsonGenerator_BackupState
0xe0494b48: cJsonGenerator_BuildString
0xe0494c0a: cJsonGenerator_Close
0xe0495144: cJsonGenerator_Create
0xe049531c: cJsonGenerator_CreateForStream
0xe0494c74: cJsonGenerator_Finalize
0xe0496ad0: cJsonGenerator_Flush
0xe0494cfc: cJsonGenerator_Free
0xe0496a52: cJsonGenerator_GetBuffer
0xe04969d4: cJsonGenerator_GetBufferLength
0xe0494e5a: cJsonGenerator_Initialize
0xe0494de2: cJsonGenerator_InitializeForBuffer
0xe0495234: cJsonGenerator_InitializeForStream
0xe0494d5e: cJsonGenerator_InitializeTextWriter
0xe049566a: cJsonGenerator_PrepareWrite
0xe049545e: cJsonGenerator_RestoreState
0xe0496964: cJsonGenerator_VerifyCompleted
0xe0495f78: cJsonGenerator_VerifyField
0xe04955ac: cJsonGenerator_VerifyToken
0xe0495850: cJsonGenerator_WriteBeginArray
0xe0496200: cJsonGenerator_WriteBeginArrayField
0xe049593a: cJsonGenerator_WriteBeginObject
0xe049630a: cJsonGenerator_WriteBeginObjectField
0xe049668a: cJsonGenerator_WriteBoolField
0xe0495d0e: cJsonGenerator_WriteBoolValue
0xe04958c4: cJsonGenerator_WriteEndArray
0xe0495a1c: cJsonGenerator_WriteEndObject
0xe049609e: cJsonGenerator_WriteFieldTemplate
0xe04964be: cJsonGenerator_WriteIntegerField
0xe0495b6e: cJsonGenerator_WriteIntegerValue
0xe049678e: cJsonGenerator_WriteNullField
0xe0495e80: cJsonGenerator_WriteRawValue
0xe049684e: cJsonGenerator_WriteRawValueField
0xe04965a4: cJsonGenerator_WriteRealField
0xe0495c3e: cJsonGenerator_WriteRealValue
0xe04963c8: cJsonGenerator_WriteStringField
0xe0495a8e: cJsonGenerator_WriteStringValue
0xe04956e8: cJsonGenerator_WriteValueTemplate
0xe0186f94: cJsonHashTable_AddToBucket
0xe0186a74: cJsonHashTable_Clear
0xe0186be0: cJsonHashTable_Create
0xe01869a6: cJsonHashTable_CreateItem
0xe0186abe: cJsonHashTable_Finalize
0xe0186c4a: cJsonHashTable_FindItem
0xe0186b02: cJsonHashTable_Free
0xe0186968: cJsonHashTable_FreeItem
0xe018720e: cJsonHashTable_GetIterator
0xe018692c: cJsonHashTable_Hash
0xe0186b40: cJsonHashTable_Initialize
0xe0187038: cJsonHashTable_Rehash
0xe01873ca: cJsonHashTable_Remove
0xe01872c8: cJsonHashTable_RemoveFromBucket
0xe01870f2: cJsonHashTable_Set
0xe01b6894: cJsonLexer_AppendCharToToken
0xe01b869a: cJsonLexer_ClearBuffer
0xe01b854a: cJsonLexer_Create
0xe01b7bec: cJsonLexer_DecodeUnicode
0xe01b67dc: cJsonLexer_Finalize
0xe01b6820: cJsonLexer_Free
0xe01b6fae: cJsonLexer_GetChar
0xe01b6938: cJsonLexer_GetCharIfTokenContinue
0xe01b685c: cJsonLexer_GetTokenString
0xe01b8364: cJsonLexer_Initialize
0xe01b86d2: cJsonLexer_IsValueStart
0xe01b862c: cJsonLexer_Next
0xe01b830a: cJsonLexer_OnError
0xe01b6ab4: cJsonLexer_OnKeyword
0xe01b8182: cJsonLexer_OnNeutral
0xe01b6e52: cJsonLexer_OnNumExp
0xe01b70ac: cJsonLexer_OnNumExpSign
0xe01b733c: cJsonLexer_OnNumExpStart
0xe01b748a: cJsonLexer_OnNumFrac
0xe01b761a: cJsonLexer_OnNumFracStart
0xe01b7752: cJsonLexer_OnNumInt
0xe01b7908: cJsonLexer_OnNumIntZero
0xe01b7a96: cJsonLexer_OnNumSign
0xe01b7fdc: cJsonLexer_OnStr
0xe01b7dfa: cJsonLexer_OnStrEscape
0xe01b7c9e: cJsonLexer_OnStrEscapeUnicode
0xe0145b0c: cJsonNode_AddToArray
0xe0144ae0: cJsonNode_ClearArray
0xe0144ae0: cJsonNode_ClearArray
0xe0144a38: cJsonNode_ClearObject
0xe0146480: cJsonNode_Clone
0xe0145fe6: cJsonNode_ContainsKeyInObject
0xe0144d44: cJsonNode_CreateObject
0xe0145e7c: cJsonNode_DetachFromArray
0xe01462b0: cJsonNode_DetachFromObject
0xe0146632: cJsonNode_Finalize
0xe0144b8c: cJsonNode_Free
0xe01459b2: cJsonNode_GetArrayCount
0xe014586c: cJsonNode_GetBigNumber
0xe0145ad0: cJsonNode_GetFromArray
0xe0145fae: cJsonNode_GetFromObject
0xe014575c: cJsonNode_GetInteger
0xe0145674: cJsonNode_GetNodeType
0xe0146356: cJsonNode_GetObjectIterator
0xe01457e4: cJsonNode_GetReal
0xe01456da: cJsonNode_GetString
0xe0144bec: cJsonNode_InitializeArray
0xe014527e: cJsonNode_InitializeBigNumber
0xe01466a6: cJsonNode_InitializeInteger
0xe01454ea: cJsonNode_InitializeKeyword
0xe0144cda: cJsonNode_InitializeObject
0xe01466d8: cJsonNode_InitializeReal
0xe0144ff2: cJsonNode_InitializeString
0xe0145bd2: cJsonNode_InsertToArray
0xe0145da0: cJsonNode_RemoveFromArray
0xe01461c4: cJsonNode_RemoveFromObject
0xe014608a: cJsonNode_SetToObject
0xe01463e0: cJsonNode_ShallowCopy
0xe014b432: cJsonObjectIterator_GetKey
0xe014b486: cJsonObjectIterator_GetMutableValue
0xe014b4b8: cJsonObjectIterator_GetValue
0xe014b3c0: cJsonObjectIterator_Next
0xe01b8864: cJsonParserStateStack_CopyContent
0xe01b89d6: cJsonParserStateStack_Create
0xe01b88ce: cJsonParserStateStack_Finalize
0xe01b8908: cJsonParserStateStack_Free
0xe01b8944: cJsonParserStateStack_Initialize
0xe01b8ad2: cJsonParserStateStack_IsEmpty
0xe01b8b08: cJsonParserStateStack_Pop
0xe01b8a6a: cJsonParserStateStack_Push
0xe014b5b8: cJsonParser_Close
0xe014b856: cJsonParser_CreateFromBuffer
0xe014bc2c: cJsonParser_CreateFromStream
0xe014bd48: cJsonParser_DecodeNumber
0xe014b626: cJsonParser_Finalize
0xe014b69a: cJsonParser_Free
0xe014c9bc: cJsonParser_GetEncoding
0xe014c5c2: cJsonParser_GetFieldName
0xe014c6fe: cJsonParser_GetInteger
0xe014ca1a: cJsonParser_GetPosition
0xe014c80a: cJsonParser_GetRawValue
0xe014c784: cJsonParser_GetReal
0xe014c660: cJsonParser_GetString
0xe014c4d8: cJsonParser_GetTokenType
0xe014b7ba: cJsonParser_Initialize
0xe014c0fa: cJsonParser_InitializeCharProvider
0xe014b6fa: cJsonParser_InitializeFromBuffer
0xe014bb4c: cJsonParser_InitializeFromStream
0xe014c1e2: cJsonParser_InitializeLexer
0xe014c53e: cJsonParser_IsBigNumber
0xe014be18: cJsonParser_PreRead
0xe014c2c2: cJsonParser_Read
0xe014bee4: cJsonParser_ResolveEncoding
0xe01b2d0c: cJsonString_AppendChar
0xe01b2ebc: cJsonString_AppendString
0xe01b2de0: cJsonString_AppendStringN
0xe01b2f4a: cJsonString_Clear
0xe01b28aa: cJsonString_Create
0xe01b2fb6: cJsonString_Equals
0xe01b29d4: cJsonString_ExtendBufferIfNecessary
0xe01b2790: cJsonString_Finalize
0xe01b27ca: cJsonString_Free
0xe01b2934: cJsonString_Get
0xe01b2828: cJsonString_Initialize
0xe01b2984: cJsonString_Length
0xe01b2a9a: cJsonString_Set
0xe0187622: cJsonSyntaxVerifier_Create
0xe0187510: cJsonSyntaxVerifier_Finalize
0xe018754a: cJsonSyntaxVerifier_Free
0xe0187588: cJsonSyntaxVerifier_Initialize
0xe01876b8: cJsonSyntaxVerifier_TransitState
0xe018777c: cJsonSyntaxVerifier_Verify
0xe01879fa: cJsonSyntaxVerifier_VerifyEof
0xe0187ac2: cJsonTextWriterUtf8_Create
0xe0187c08: cJsonTextWriterUtf8_Finalize
0xe0187b2c: cJsonTextWriterUtf8_Free
0xe0187a78: cJsonTextWriterUtf8_Initialize
0xe0187b68: cJsonTextWriterUtf8_WriteString
0xe0188d8a: cJsonUtil_DiscriminateEncoding
0xe0188406: cJsonUtil_DoubleToString
0xe0187ff4: cJsonUtil_EscapeAndAppendString
0xe0188788: cJsonUtil_GetForVerify
0xe018814a: cJsonUtil_IntToString
0xe0188728: cJsonUtil_IsInteger
0xe01889ec: cJsonUtil_IsNumberString
0xe0188c2e: cJsonUtil_IsStartWithBom
0xe0188a62: cJsonUtil_IsString
0xe0188890: cJsonUtil_PeekForVerify
0xe0188658: cJsonUtil_StringToDouble
0xe018852a: cJsonUtil_StringToInt
0xe0187e28: cJsonUtil_UnicodeEscape
0xe0188ad2: cJsonUtil_Utf16CharToUtf8


Not all of these functions are needed to use the library. Most of them are used by internal mechanisms. I'll open another thread in the Reverse Engineering section with more info on how to use the library.
EOS RP

coon

As already mentioned here https://www.magiclantern.fm/forum/index.php?topic=25662.msg235562#msg235562 I have found a way to close and open the shutter manually on RP.
To do so, FA_MechaShutterClose and FA_MechaShutterClose can be called.

On the EOS R the shutter will close when the camera is turned off do reduce the risk of getting dust onto the sensor. Unfortunately this feature is not available on RP. By using this functions it should be possible to also enable that on RP.
EOS RP

kitor

R doesn't close shutter on lens detached so it may be a nice feature to implement too.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor



Funny stuff - when your camera has 2GB of ram, and you have a problem to find just about 3MB for yourself.

Our current progress uses a lot of ram considering "regular" Digic 4/5 cameras:

- ~500kb already used by Zebra for double buffering
- ~500kb for indexed RGB buffer that emulates old style of drawing (so we don't need to rewrite a lot of GUI code)
- ~500kb for bitmap font (gone from D6+ ROMs, at the moment I loaded one extracted from my 50D)
- ~2MB for RGBA layer (see thread on Compositor)

This is already more than regular memory available on R ???

While 3rd and 4th are optional:
- Bitmap font was already mostly replaced, and we will either prepare a much smaller BMP font that we can distribute, or get rid of it at all
- Own compositor layer can be avoided - at the moment it has no big benefits over drawing directly to GUI layer like 200D, M

it is still less than 3MB of memory available  >:(

For the last two days with names_are_hard I was looking at memory management functions. In general there are two main allocators (malloc/free and AllocateMemory/FreeMemory) that can be "freely" used.

What I noticed is that 2nd one (AllocateMemory/FreeMemory) just wraps calls to more general functions that take memory pool address as first param. Those functions are also used by some other components. It turns out that Canon has nice "class" for managing memory area - called... you guessed it, MemoryManager.

There's just a single function to initialize - takes just start memory address and memory length. After using it over arbitrary block of memory (of course assuming it is not used by anyone else) one can just call any of those general functions (e.g. void* ptr = AllocateMemory_impl(pMemMgr, size) or FreeMemory_impl(pMemMgr, ptr).

As I found potentially unused 4MB memory block, I was able to just initialize MemoryManager and substitute functions stubs with ones that wrap into that block. Code is fairly clean and simple and you can see it in this commit in my repo.

While my code just replaces usage of usual Canon pool, it can be integrated as additional space into our memory management system. I just replaced it as this way I was able to keep my changes in platform dir for now, and this way code is forced to use it - so I'll probably see some crashes is something else uses the mentioned memory area. Hopefully it won't happen ;)
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

Moar stuff!  8)

Turns out that there were a lot of property changes on Digic 8 generation (unconfirmed on DSLR, but applicable at least for R* and M50).



I started today's journey with fixing shutter counter. Guess what, new property is used...

At least this one was easy. Although I didn't find "total count" (the one that includes LV), but this was expected as R is LV all the time. Requires verification on D8 DSLR.
I also found another counter - which includes shots made in silent (electronic) shutter mode. I decided to add it to the menus as a new, currently R only feature  8)

Fixing Lens info was a more complex task. Turns out that multiple properties were replaced with two, called nicely LENS_STATIC_DATA and LENS_DYNAMIC_DATA

I had to work out new structures and make a new handlers for those. As you can see, they work. Confirmed on RF and EF lenses (latter both on Canon and Tamron lenses).
Changes for now are on in my repo on kitor_dev branch
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

coon

I have catched up with R / M50 / 200D progress and finally got Magic Lantern menus running on RP:



Running ML was not easily possible for a while since from RP on Canon decided to use a different way to render graphics then on previous models like R. This new method is also used on R6, R5 etc.
However, due to kitors research on the compositor this is not an issue anymore.

Navigating through the menu is possible but no feature works currently. Camera tends to crash when running menu entries and behaves in a undefined manner so this is nothing what can be released to the public for now. Next step is getting menus stable and enable features :).
EOS RP

Corallaro

Wow that's great!!!! :)

My RP is very excited! !

coon

And here is the first Magic Lantern feature for RP: Close shutter when turning the camera off. For some reason this is only available on R, R6 and R5 but not for RP... At least until today :).

EOS RP

Walter Schulz


kitor

Yup, on @coon's SD card ;)

This requires a proper integration with ML codebase first. Right now it is just yolo-attached to ML shutdown code  :)

====

On the other hand - I spent almost whole week trying to implement intervalometer. Not only factory functions looks to be implemented differently on this generation (we used some of those to make shoots), but after roughly understanding how they work I hit a wall - they have a bug. If photo is not taken (e.g. AF fails) camera ends up in a state where next photo will crash it [1].
I'm looking for other methods of making shoots from code.

Inb4 "bbbbbut R has intervalometer built in!". No, it doesn't! It has movie timelapse mode only, something completely different but often mistaken by people who never had R in their hands.

[1] Technical TL;DR: Post-shoot callback is set up which releases a semaphore, that is already removed on failure. Kernel fails on null pointer...
I tried cleaning all memory locations that hold CBR address, but no luck - it is still executed. After a week I needed a break.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

Walter Schulz

"Use Autofocus" on any recent ML build will break intervalometer, audio remote shot and focus trap (motion detection): https://foss.heptapod.net/magic-lantern/magic-lantern/-/issues/2923

kitor

Quote7D's reaction: Shoot like crazy, burst mode (7D.206 running dfort's build)
Confirmed by user critix running 500D (latest nightly).

Quote650D behaves different. Screen will be frozen Canon's "Q"-view. Pics will be taken by full press, but no reaction on half-shutter, set, info, menu buttons.

Both are possible outcomes on R, depending on function used. 3rd, more common one is just hard crash due to kernel assertion on mentioned null pointer.
Thanks for pointing this out. So it is not R being crazy, but we just need a better method to make DryOS do photos.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

Corallaro

Quote from: coon on June 16, 2021, 06:03:04 PM
And here is the first Magic Lantern feature for RP: Close shutter when turning the camera off. For some reason this is only available on R, R6 and R5 but not for RP... At least until today :).

A this point the second feature can be silent shutter in all modes......... ;)


Walter Schulz

CRC32:
178F25F7
EDD1561F

Dammit, Canon! Why?

ArcziPL

Nice, hehe. Just FYI: first one comes from canon-europe.com, the second one from asia.canon.
M50.110 [main cam] | G7X III [pocket cam] | 70D.112 [gathers dust] | M.202 [gathers dust] | waiting for M5II

Walter Schulz

Any insights about the differences?

names_are_hard

The FIR files aren't really meaningful to compare, they're mostly an encrypted blob so they appear entirely different apart from the short header (which has some small differences).  The files are different sizes, so there's probably some real changes.  Needs dumps to compare sensibly.

I don't think this is the first time this has happened, seem to recall it in source somewhere.

Walter Schulz

I remember 70D where cams got delivered (first firmware) with 4 different firmware versions but same version string. Unaware of actual firmware files in the wild showing differences.

kitor

Ouch  :o

If I don't forget about it, I'll try flashing and dumping both next week.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

sast

Is there any progress yet?

kitor

All recent discussions were a huge milestones, some even for CHDK project and not only Magic Lantern.

But like we told you already a few times, no posts = no progress.

Quote from: sast on May 18, 2019, 08:52:46 PMI don't now how to do all the things on the EOS R without ML (f. e. Intervalometer, Motion detection, the modules).

If you count on getting any of those features in near future, stop right now. Assume they won't happen unless they are announced.

In fact I already wrote about intervalometer just a few posts above - and since then I spent too much time to understand how shooting works, without any success.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

sast

ok, Thanks for answering

IDA_ML

"I don't now how to do all the things on the EOS R without ML (f. e. Intervalometer, Motion detection, the modules). "

Just sell the EOS-R and get yourself the 5DMkIII for the money.  You won't regret !!!

nikfreak

Quote from: Walter Schulz on June 20, 2021, 03:20:25 AM
I remember 70D where cams got delivered (first firmware) with 4 different firmware versions but same version string. Unaware of actual firmware files in the wild showing differences.

true pain in my a$$  ;D
[size=8pt]70D.112 & 100D.101[/size]