Messages

I suggest you to start reading posts from Digic 7 development thread, it has very useful information on how progress was made by other users to make digic6-dumper branch to work on D7 cameras and will help you to have an idea on what to do to start porting ML for EOS RP.

Starting from here there are some of my previous posts of me understanding how to port code to 77D using only emulator as I don't have a real camera, also looks for posts from calle2010 who kept progressing the work.

After that you will like to print an Hello World on EOS RP and share a picture!

I think I am now at a point where I have to dive deeper into how the initialization process works and how to find out the proper device addresses / RAM size etc. to get even further on the boot process.

Does the emulation looks like it keep running or instead it hangs up?

You can also try to improve emulation for Digic X camera to have Canon GUI to render and navigate from QEMU, here you would certainly like some hints from a1ex to address you into the right direction

This should generate a valid link, wait for redirect:
https://web.archive.org/web/20200723124921/http://bitbucket.org/hudson/magic-lantern/downloads/magiclantern-5Dc.111.Beta4.zip

Found this one with the original archive from Beta4 (2012):
https://bbuseruploads.s3.amazonaws.com/hudson/magic-lantern/downloads/magiclantern-5Dc.111.Beta4.zip?Signature=fw4tWNNPQCMr83nG2ylPP6xrVsI%3D&Expires=1595510361&AWSAccessKeyId=AKIA6KOSE3BNJRRFUUX6&versionId=null&response-content-disposition=attachment%3B%20filename%3D%22magiclantern-5Dc.111.Beta4.zip%22

Which build were you using?

It may be an HW problem.
I bought a spare 5Dc (ML has never been loaded on it) with the same issues you have, it has a damaged D-PCB (main board) that overheat near processor/ram and also fried the fuse of the DC power board. Top lcd and MCU are working as normal and blue/red leds stay on

More repository with useful work not identified by the script:

https://bitbucket.org/coutts/1000d_dev/
https://bitbucket.org/coutts/5dplus/
https://github.com/400plus/400plus/ (ML Related)

2) All other interesting Mercurial repositories from Bitbucket, not related to ML, in particular, unmaintained projects (these will be lost forever, unless archive.org already crawled them).

Tried some times ago using Wayback machine to save PR's source code, but unfortunately was failing due to JS based views  and also noticed even no screenshoots in many years for pages in main repo...

I imported all branches and history into Github a while back:
https://github.com/reticulatedpines/magiclantern

Looks like it's update up to 3 Jul 2018, I can't find most relevant branches (crop_rec_*, manual_lens_info, vxworks...) into your repo unfortunally.
@names_are_hard Can you check please? May be usefull as backup.

Still wishing we can get more good PR merged into codebase as possible before sunsetting to avoid loosing some works from cummunity...

Tried to solve some issues these days and finally got the module subsystem to load

I'm still trying to figure out how to solve module loading (same problem with config reset/load and all other src files which are using ATTR_DIRECTORY code-block) and why Histogram is not being drawn.

I tried to enable Screenshoot but did't saved anything to card yet... with White Balance feature I get property change request errors.
Didn't tried to enable others because I was first figuring how to fix previous features with the recent codebase.

Double checked FIO stubs and found there was an error in the 5DC codebase:

Code Select Expand


//NSTUB(0xFFB4720C,  FIO_FindNextEx)    wrong address
NSTUB(0xFFAE6018,  FIO_FindNextEx)


I got module subsistem loading but it's listing wrong module's name and so fail to load them due to missing file on card.
This is how it's showing them:

edmac.mo -> c.mo
mpu_dump -> dump.mo
mlv_lite -> lite.mo
file_name.mo -> name.mo
arcanoid.mo-> noid.mo
pic_view.mo -> view.mo

Looks like it is using only 4 characters from position 4 to 7.

Figured it out, on 5DC fio_file struct is a bit different and name is at different offset, don't know about the others fields:

Code Select Expand


/** Directory entry returned by FIO_FindFirstEx() */
struct fio_file {
        //! 0x10 == directory, 0x22
        uint32_t                mode;           // off_0x00;
        uint32_t                size;
        uint32_t                timestamp;      // off_0x08;

        // 5DC has name 4 byte before DryOS, not sure about other VxWorks camera
        #ifndef CONFIG_5DC   
        uint32_t                off_0x0c;
        #endif

        char                    name[ FIO_MAX_PATH_LENGTH ];
        uint32_t                a;
        uint32_t                b;
        uint32_t                c;
        uint32_t                d;
};




Using a recent codebase compared to beta4 was breaking overlay drawing in playback mode, found it was missing functions in zebra-5dc.c due to more recent changes in main codebase, still need to fix Histogram which is empty at the moment; I tried also to use zebra.c and after some time got all working except zebras which makes camera freezes and Histogram which get draw as a green rectangle.

Wish list: custom af points and lua scripting.

At the moment, I'm tring to check if features of beta4 are working with current codebase from vxworks branch and also when merging unified into it; need to found how to make Property Changes working, make screenshoot works and check for dependant features from it, find a way to makes benchmark and other debugging functions to work, add missing features from features.h and refactor some code blocks.

Will try to commit changes to my repository soon, so everyone can check and hope to have a more featured build to share also.

Didn't noticed a fast battery drain with my camera.
Which build where you using?

I found how to figure out shutter counts for 5dc, it works on my camera but still not doubled checked on a second body.
Don't ask for posting more explanations about, no chances at the moment, at least until it's sure won't hurt ML community....

Thanks heder, I solved using a different approach:
I observed texts are rendered good if using even values for line position.

So I changed only the code block related to vxworks camera, without impacting other ML code.



As a side effect first left line Is sligthly touching text of the task screen name.

Here is the commit showing what I changed.

I would like to know if this approach is working also for 40D and 450D, or only for 5DC.

1) I was following the "stubs tutorial"...
....
Do you write own, custom scripts that parse entire disassembly file and look for the occurrence of a label in line N and push command in line N+1 or are there some established scripts that help with stub searching?

Most of the time looking for strings is a good way to have a looks in the right direction (or code segment), usually can be done by searching inside decompiled files if using dissasemble.pl script or by looking for xrefs in IDA.

check-stubs.py can also be useful when porting firmware update.

that Memory functions, but they are easy, still curious how did you identify that "create" function

Also using QEMU for some tricks (.idc file of function called during emulation and code blocks copied from ROM to RAM), helped me a lot to have more functions marked inside the disassembly or to retrieve the RAM address instead of the ROM address for a specific stub.

IDA will replace and display calls to RAM version of functions automatically if code segments are loaded into the project; the exact function can be found inside the disassembly without using them, but having RAM address instead come in handy when we need to easly patch them.

2) I assume that stub searching is easier if you have an older camera with stubs identified: I assume that large % the low-level API stays unchanged from version to version, and one could look e.g for a similar sequence of commands or sth like that.  Am I making sense here?

Definitely useful to have a reference camera which use same DIGIC generation, it makes looking for stubs much easier because much of the code is shared and usually unchanged between them.

But because most of the code is shared or evolved during years, also comparing disassembly from older cameras can reveal the same or similar code structure to compare with: If I remember correctly I also double checked using a disassembly from 50D (Digic IV) when working on 77D and others (Digic VII).

Unfortunately comparing too much older cameras with a more recent one, ie. 5DC with 50D or even 40D (only 1 generation newer) won't help much... while disassembling 5DC, a lot of stubs looks like 50% smaller or completely different from newer cameras... also error messages (ie. looking for error code pattern from "finding stubs") looks like are handled in a different way, a lot of strings missing... making things harder...

3) the stubs.S files for other cameras have around 50-100 stubs that are identified. I am surprised that this number is actually so low. There way more functions mentioned in the string files for the ROM i downloaded.

Some older camera have unused stubs which were commented because they were replaced by standard library instead of canon's version, or because some stubs required by some specific feature are not necessary anymore due to some refactor of ML code.

Obviously having lesser stubs to find makes porting much easier for us.

The ML code is not able to operate merely on these 50-100 functions right? How is that so?  does this: "THUMB_FN(0xE04E706A,  call)                          /* many functions called by name (lv_start, lv_stop etc) */" open the door to call everything else?

A much better answer to last question need to be provided by main devs or someone which saw how ML code evolved during years.

Found stubs for 6d2 1.0.4 some times ago.

They need to be double checked by a1ex before
a bootflag enabler will be released to public, then hello world should be straightforward.

I'm perfectly fine with the a1ex's review and is also nice to know from the poll results where others wish some improvements.
There will be so much work to do and support needed from all of us.

Hope to find a solution for legal side soon so this new chapter of ML development can start.

Considering the slowdown to the development of ML probably we all already noticed about, there are more than one discussion on the forum of a possible approach to this topic...

I would like to ask you if have ever really considered donating some money to help the community or would like to start.
Personally would like to give some support to main devs even without a guarantee of a short time return in developing effort.

What do you think about? Would you make a recurring donation? Share your opinion so maybe we can make something possible

It's normal if the pink clip is on the 2nd or 3rd frame (link to old post for reference) it's from the 10-12bit experimental method for D4 cameras

@reddeercity is this with your current repo codebase?
Do you think the problem you where having with 5d2 will pretty much likely happen also with 50D?

Was thinking to have a look to understand crop_rec_4k code using 50D for testing, but I'm a bit busy these days...

Would like just try to use as much of the main codebases as possible compared to current waza changes, but due to not being familiar with those changes I would like to know if are they all necessary?

[Shutter Counter]

I'm trying to find Shutter Counter by dumping MPU memory region and trying to understand EEPROM content with help from a1ex.

This is after taking a second dump after releasing the shutter:

You can see some areas with seems to be mirrored like in camera firmware and a byte incremented by one digit.

Using this as starting point, after taking a bounce of pictures the same byte at right is incremented, but something strange happened when taking more to make counter rolls over:


Assuming a Big-Endian representation we was expecting to see these FF near the last byte to change, but apparently first byte from left is incremented (looking like to be counter or a index or some other meaning) and next digit start to increment at the next byte from the previous one.



If someone want to help you can download "magiclantern-BETA.2019Sep24.5DC111_mpu_dump.zip" from this download page and report.

How to run: go to Debug Tab -> Don't click me
Then you will find MPU dumps from ROM,ROM2 and EEPROM. (each test will overwrite previous file)

Camera will write immediately to EEPROM after releasing the shutter, so take a couple, diff them and compare to mine.

it will be much valuable to understand how it works If you are aware of approximate shutter counts (reported by camera service).

Didn't understood well where is the problem and what you are trying to achieve, but the highlighted line of code is never taken with the 5d2, because the condition for true statement as it is written now is: "go inside the true branch if is not a 5d2 and current register is 0xC0F06804".
Or are you working in local with a different code at that line?

Maybe try to manipulate the register 0xC0F06804 inside the false branch like waza57 did (or remove !is_5d2), if you want to follow his route.
If I mislead something just ignore, didn't digged inside crop_rec code yet

Thanks for checking. Need to figure out somehow to solve the issue, I'll try to have a look again

@heder Can you load modules?
Make sure module_task is launched from ml_big_init_task() inside custom init.c

I get wrong behaviour with 5DC, they are listed with wrong filename (it show only last 4 characters of the module's name) so can't be found on card.
Would like to know if it's the same with 40D.

Canon 5DC to compare with other vxworks cameras:

Code Select Expand


  Magic Lantern Rescue
----------------------------
- Model ID: 0x0 ERROR
- Camera model: ???
- Firmware version: ??? / ???
- IMG naming: 100?????/????0000.JPG
- User PS: ??? ??? ???
- Boot flags: FIR=0 BOOT=-1 RAM=-1 UPD=-1
- ROMBASEADDR: 0xFF810000

CHDK CPU info for 0x0 ERROR
------------------------------
ID         0x41059461
  Revision             0x1 1
  Part                 0x946 2374
  ARM Arch             0x5 5
  Variant              0x0 0
  Implementor          0x41 65
Cache type 0x0F112112
  Icache words/line    0x2 2 [8]
  Icache absent        0x0 0
  Icache assoc         0x2 2
  Icache size          0x4 4 [8K]
  Reserved0_2          0x0 0
  Dcache words/line    0x2 2 [8]
  Dcache absent        0x0 0
  Dcache assoc         0x2 2
  Dcache size          0x4 4 [8K]
  Reserved1_2          0x0 0
  Harvard/unified      0x1 1
  Cache type           0x7 7
  Reserved2_3          0x0 0
TCM type   0x000C00C0
  Reserved0_2          0x0 0
  ITCM absent          0x0 0
  Reserved1_3          0x0 0
  ITCM size            0x3 3 [4K]
  Reserved2_4          0x0 0
  DTCM absent          0x0 0
  Reserved3_2          0x0 0
  DTCM size            0x3 3 [4K]
  Reserved4_10         0x0 0
Control    0x0005107D
  Protect enable       0x1 1
  Reserved0_1          0x0 0
  Dcache enable        0x1 1
  Reserved1_4          0xF 15
  Big endian           0x0 0
  Reserved2_4          0x0 0
  Icache enable        0x1 1
  Alt vector           0x0 0
  Cache RRR            0x0 0
  Disble load TBIT     0x0 0
  DTCM enable          0x1 1
  DTCM mode            0x0 0
  ITCM enable          0x1 1
  ITCM mode            0x0 0
  Reserved3_12         0x0 0
Protection Region 0 0x0000003F
  Enable               0x1 1
  Size                 0x1F 31 [4G]
  Undef0_7             0x0 0
  Base                 0x0 0 [0x00000000]
Protection Region 1 0x0000003B
  Enable               0x1 1
  Size                 0x1D 29 [1G]
  Undef0_7             0x0 0
  Base                 0x0 0 [0x00000000]
Protection Region 2 0xF8000035
  Enable               0x1 1
  Size                 0x1A 26 [128M]
  Undef0_7             0x0 0
  Base                 0x7C000 507904 [0xF8000000]
Protection Region 3 0xC0000039
  Enable               0x1 1
  Size                 0x1C 28 [512M]
  Undef0_7             0x0 0
  Base                 0x60000 393216 [0xC0000000]
Protection Region 4 0xFF80002D
  Enable               0x1 1
  Size                 0x16 22 [8M]
  Undef0_7             0x0 0
  Base                 0x7FC00 523264 [0xFF800000]
Protection Region 5 0x00000035
  Enable               0x1 1
  Size                 0x1A 26 [128M]
  Undef0_7             0x0 0
  Base                 0x0 0 [0x00000000]
Protection Region 6 0x00000000
  Enable               0x0 0
  Size                 0x0 0 [invalid]
  Undef0_7             0x0 0
  Base                 0x0 0 [0x00000000]
Protection Region 7 0x00000000
  Enable               0x0 0
  Size                 0x0 0 [invalid]
  Undef0_7             0x0 0
  Base                 0x0 0 [0x00000000]
Region data perms 0x00333333
  Region 0             0x3 3 [P:RW U:RW]
  Region 1             0x3 3 [P:RW U:RW]
  Region 2             0x3 3 [P:RW U:RW]
  Region 3             0x3 3 [P:RW U:RW]
  Region 4             0x3 3 [P:RW U:RW]
  Region 5             0x3 3 [P:RW U:RW]
  Region 6             0x0 0 [P:-- U:--]
  Region 7             0x0 0 [P:-- U:--]
Region inst perms 0x00333333
  Region 0             0x3 3 [P:RW U:RW]
  Region 1             0x3 3 [P:RW U:RW]
  Region 2             0x3 3 [P:RW U:RW]
  Region 3             0x3 3 [P:RW U:RW]
  Region 4             0x3 3 [P:RW U:RW]
  Region 5             0x3 3 [P:RW U:RW]
  Region 6             0x0 0 [P:-- U:--]
  Region 7             0x0 0 [P:-- U:--]
DCache cfg 0x00000030
  Region 0             0x0 0
  Region 1             0x0 0
  Region 2             0x0 0
  Region 3             0x0 0
  Region 4             0x1 1
  Region 5             0x1 1
  Region 6             0x0 0
  Region 7             0x0 0
ICache cfg 0x00000030
  Region 0             0x0 0
  Region 1             0x0 0
  Region 2             0x0 0
  Region 3             0x0 0
  Region 4             0x1 1
  Region 5             0x1 1
  Region 6             0x0 0
  Region 7             0x0 0
Write buffer 0x00000030
  Region 0             0x0 0
  Region 1             0x0 0
  Region 2             0x0 0
  Region 3             0x0 0
  Region 4             0x1 1
  Region 5             0x1 1
  Region 6             0x0 0
  Region 7             0x0 0
DTCM cfg   0x40000006
  Reserved0_1          0x0 0
  Size                 0x3 3 [4K]
  Undef0_7             0x0 0
  Base                 0x20000 131072 [0x40000000]
ITCM cfg   0x00000006
  Reserved0_1          0x0 0
  Size                 0x3 3 [4K]
  Undef0_7             0x0 0
  Base                 0x0 0 [0x00000000]

- card_bootflags 101c14
- boot_read/write_sector 107d78 107d80
- 102424: cf_dir (cfata_init error)\n
- 1020a8: cf_read_dma (cfata_init error)\n
- 107C74 Card init => 0
- Saving RESCUE.LOG ...


What do you get if you try "unified" helloworld with digic6 branch?
Maybe there are some changes from a1ex useful to make it stable enough.

The Canon UI is unresponsive, often the camera locks up.  Sometimes it doesn't and half press will give you exposure info.  Sometimes the off switch does a clean shutdown, generally not.

I had experience of GUI lockup when updating 5DC port to more recent codebase; If I was quick enough I could as example enter the canon menu before ml override the task responsabile for buttons handling.

Don't remember exactly how I solved but probably was some missing initializzation in the custom ml_init for vxworks cameras. I had to doublecheck with the "unified" code to notice what was missing to load gui.

Also was missing initializzation of memory allocator...

ML is getting far enough that it prints a warning: "Camera was not shutdown cleanly. Skipping module loading" - pretty good.

Maybe here it is hanging on the first startup of Ml (after copy files to card, wiping previous settings) locking the camera and skipping module initializzation on the next round because of the undeleted module lock file in the card...

On vxworks it try to scan the directory to list all modules but doesn't iterate thought, so  camera appears as freezed...

Try to notice what is happening with fresh a install and see if the lock file is still present after shutdown.
If you want to disable modules, just comment module_task creation in ml_big_init

[Only use public methods/returned values]

config.create_from_menu works great for me.
But how can I get and set values in the config without using a menu?
My autorun script needs to store state between reboots, and using a hidden menu (in addition to actual menu items) just to store state is not great

Thats basically because you are not using the "pointer" returned by "config.create()" or "config.create_from_menu()".

I tried using "config" instead, but no dice.

Only use public methods/returned values and don't access/modify global variables or private functions from config.lua (ex. "config" table or config.serialize() ), because if the current implementation will change in the future, your script will not works or will not behave as you may expecting.

After loading the config from a file, the library initialize and load all values inside his internal data structure.
This is a simplified overview on how it's memorized in memory:

Code Select Expand


config = {} -> a table, basically the script it's memorizing here what it needs during his usage
|- ...
|- configs -> in this table there is one entry for each script
|- ...

Each script's entry is following this structure:
|- filename
|- default -> a table or simple entry, initial value for each entry of data
|- data    -> this is a table containing the actual values
   Example for one menu:
   |- "MenuName" -> key for the corresponding menu table
      |- ... -> one <key,value> pairs for each menu entry (value is a table if it's a submenu)
   Example of a table:
   |- {"key" = value}


An example of usage with a configuration saved as a table:

Code Select Expand


require("config")

configuration = {["Apples"] = 1,["Eggs"] = 2}
my_config = config.create(configuration);

print(my_config.data["Apples"])

If during the script you change values in my_config somehow, they should get saved to file and loaded at the next run.



Very Important: There is a bug in the current implementation of Config's Lib (lua_fix and derived) when calling more than one config.create() in the same script.

Take a look at this example (it's untested, it's from what I remember):

Code Select Expand


FirstTable = {1,2,3, name = "alex"}
SecondTable = {"a","b", text = "Hello World!"}

config.create(FirstTable);
config.create(SecondTable);


This is a representation of what is memorized:

Code Select Expand


{
  [1] = "a",
  [2] = "b",
  [3] = 3,
  ["name"] = "alex",
  ["text"] = "Hello World!"
}

Basically the second config is overwriting the first two entries because they doesn't have a specified key so they are mapped to integers.

I have found Lua API documentation mostly helpful, but some entries are rather minimal.

Yes, it needs some improvements

https://bitbucket.org/hudson/magic-lantern/src/manual_lens_info/scripts/lens.lua (example for menu with custom state)

This script is using a modified version of the library, so keep it as reference if you need a complex usage but don't forget to use correct config.lua from manual_lens_info otherwise will not works with multiple menu's config in the same script as in lens.lua.

Bottom line: I am happy to support, ie I share my humble scripting attempts to help others, but I truly believe the Lua API documentation needs someone with 'insider knowledge' to add (more) value. That is 'right first time'.

@garry23 I added support for nested submenus and allowed to have separate menu entries (to avoid packing all into a single big chunk and have separate menus organized by functionality); Let me know if there is something wrong in the case you or someone else test it.

Don't know if this is a bug or not (I'm not a regular 50D user) but ISO does not change from the
magic lantern menu , it only changes setting by the top dial on camera , in the little LCD screen on top
has the correct reading but ML report a different thing e.g. 100ISO . Using mlv_rec .

Can any other 50D user confirm this or not or is this how it is ?

Quick check using lua_fix (both mlv_lite and mlv_rec) by changing ISO in ML menu I get correct values in top lcd and photo mode, but I get the value doubled in ML info bar while using LV.

Same happens if I use top buttons to change ISO.

Figured out what wasn't working with my patch.
Continuous blinking led was due to dump_with_buffer() in init.c and no boot was due to a wrong stub (double-checked Stubs.s with the one from scrax)

@scrax I committed current changes to my repository.
If anyone wants to try the current changes (+ PLAY mode saturation re-enabled but not committed yet), you can find a compiled build here

I'm still trying to figure out how to solve module loading (same problem with config reset/load and all other src files which are using ATTR_DIRECTORY code-block) and why Histogram is not being drawn.

I tried to enable Screenshoot but did't saved anything to card yet... with White Balance feature I get property change request errors.
Didn't tried to enable others because I was first figuring how to fix previous features with the recent codebase.

DebugMsgs are working, need to find how to intercept messages from MPU.

Regarding text in "Show Task" submenu... I played a bit with the code and got it printed good, it appears to be a problem with x position/spacing between each line; also Canon tasks are so much and need an additional windows to lists them all.

Found how to re-enable overlay in play mode, but I need to figure out why Histogram is not rendered (can see only borders) and Vectorscope/Waveform/Spotmeter get away when zebra and Focus Peek are enabled because it clean the framebuffer.

Regarding Intervalometer... it worked sometimes but I need to check again in which cases picture is taken.

Modules loading still crashes without enabling anything (only available arkanoid, file_man, bench, pic_view from compilation), but this time error message is different.

Finally I found where module loading get stuck, but did't found a solution yet...

Code Select Expand


do
    {
        if (file.mode & ATTR_DIRECTORY){
            printf("Is a directory\n");             
           continue; // is a directory            <---- HERE
         }
        if (module_valid_filename(file.name))     <---- NEVER TAKEN
        {
            printf("NOT a directory\n");
            char module_name[MODULE_FILENAME_LENGTH];

            .....

    } while( FIO_FindNextEx( dirent, &file ) == 0);
    FIO_FindClose(dirent);

Is looping in the folder condition indefinitely....

@a1ex What do you think? Is code not compatible with vxworks or maybe there is a problem with the FIO_FindNextEx stub?

I checked if unified got some fix but there is nothing related and don't know if it's a problem also for other vxworks cameras like 450D (Can't check in Qemu because I think the rom dump I have is not ok)