Current state of Magic Lantern project (updated February 2024)

Started by kitor, April 01, 2023, 09:12:56 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

kitor

A very RAW update, February 2024!
Current state of Magic Lantern project (updated February 2024)

New update, October 2023!
Another mini-update.
Current state of Magic Lantern project (updated October 2023)

New update, July 2023!
This time just a quarterly mini update, so requires you to read April update for the context.
Current state of Magic Lantern project (updated July 2023)

Hello!

This is a relatively short summary about what is going on with the project in last years. This thread started as an update on development for "new era" models - this includes cameras based on DIGIC 6, 6+, 7, 8 and DIGIC X processors. But with recent exciting news for older models we decided we owe you a full update  :)

We will try to keep this post written in layman's words - providing just links to more in-depth readings where suitable.

Credits to @theBilalFakhouri, @names_are_hard and @Walter Schulz for their help in preparing this write up.

For your reading convenience, thread is split into chapters, each of which is a separate post:

Chapter 1: What is Magic Lantern? Who is responsible for it?
Chapter 2: Developments in already supported generations (DIGIC 3, 4, 4+)
Chapter 3: DIGIC 5 and 5+: Exciting updates: SD overclocking, Full resolution LV preview
Chapter 4: What is going on with new models? All supported cameras are 10 years old!
Chapter 5: So what is the status of newer models? (DIGIC 6, 6+, 7, 8, X)
Chapter 6: I noticed PowerShots on the list? Do you want to steal CHDK user base?!
Chapter 7: What can I expect from you? How can I help?

Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

Chapter 1: What is Magic Lantern? Who is responsible for it?

What is Magic Lantern?



Magic Lantern is a firmware add-on. It runs alongside your original camera firmware. It doesn't replace Canon firmware and it won't install inside camera.
We know this may be very confusing - to use Magic Lantern we ask you to use "Firmware update" menu which runs "Magic Lantern Install" program.

When you use "Magic Lantern Install", it will just enable a hidden option of your camera firmware. This hidden option tells the camera:
Hey! When you turn on, check if memory card is prepared in a special way and there's a file called autoexec.bin on storage card. If it is - please execute it!

This means - if you simply format your card (or use a card without Magic Lantern files), Magic Lantern is gone.
This hidden option (known as bootflag) has a small bug - if you ever delete autoexec.bin from the card, but don't format it - you camera will appear dead.
However fear not! Just take out both the battery and the card. Then insert only the battery to confirm it is fine. If so - just format your card on PC.

Who is responsible for it?

Magic lantern is an open source project, written by enthusiasts. It is not affiliated with Canon in any way.
People contribute a lot of their spare time and, very often, in-depth specialized knowledge in areas like programming, reverse engineering, embedded hardware - all of that for free for you to use.
With over 10 years in the making, since Tramell Hudson started the project we have seen people come and go. Many of us spent literally years contributing in one way or another.

Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

Chapter 2: Developments in already supported generations (DIGIC 3, 4, 4+)

DIGIC 3: Heder's 40D port

Did you expect RAW video on 16 years old camera? Heder made that possible:

Quote from: heder on August 24, 2020, 10:37:56 AM
OPen beta updated to 1.0.5

https://www.magiclantern.fm/forum/index.php?topic=1452.msg224594#msg224594


What are the limits of 40D? What works, what doesn't? Check the linked post in the quote above to learn more :)


DIGIC 4 and 4+: 1300D port

While there's not much going on in DIGIC 4 models, all recent low ends (1300D, 2000D, 4000D) are based on a slightly upgraded version of DIGIC 4 known as "4+"

Critix created a 1300D port. It works on older firmware (1.1.0), petabyte recently created an updated version for firmware 1.2.0.
Not everything is working yet as DIGIC 4+, while very similar to older DIGIC 4, requires some amount of reverse engineering too.

Quote from: critix on February 17, 2021, 01:00:27 PM
First boot with ML:

... and the first self-test...

Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

Chapter 3: DIGIC 5 and 5+: Exciting updates: SD overclocking, Full resolution LV preview

For the past few years the development was focused on RAW video area, beside the custom crop_rec builds which offer various custom RAW video modes, there were two major RAW video improvements:

-The write speed increase:

A. back in 2018 a1ex proved that increasing SD write speed a.k.a SD Overclocking was possible on DIGIC 5 models, but it was limited to 160 MHz frequency, here is example shows Stock write speed (at 96 MHz) vs Overclocked one (at 160 MHz) from 700D:

Quote from: theBilalFakhouri on February 12, 2021, 10:32:15 PM



Stock 700D160MHz

Then after two years from that, @Bilal took a closer look into SD Overclocking process and was trying to get higher (stable) SD frequencies like 192 MHz and 240 MHz across all DIGIC 5 models (5D3 / 6D / 70D / 650D / 700D / EOS M / EOS M2 / 100D), and that did happen just last year in 2022, benchmarks shows write speed at 192 MHz and 240 MHz:

Quote from: theBilalFakhouri on February 12, 2021, 10:32:15 PM



192MHz240MHz

B. Unfortunately above benchmarks don't represent actual write speed when recording RAW video, they were done outside LiveView, in order to maximize effective write speed during RAW video recording, we will need to lower CPU usage and save it to RAW recording, and yet again @Bilal take a look into it and was able to disable some unnecessary tasks during RAW video recording which increased write speed performance significantly:

Quote from: theBilalFakhouri on April 07, 2022, 06:20:22 AM
Hi, is this April 1st?
                                                                                                                                                                                                                                                         
700-D-Live-View-Write" border="0 April-1st" border="0

By combining these two things, the final write speed was increased:

-Up to 125% in 6D / 70D                             (from  ~40 MB/s  to  ~90 MB/s)
-Up to 100% in entry level models like 700D (from  ~40 MB/s  to  ~80 MB/s)
-Up to   45% on 5D3                                   (from ~100 MB/s to ~145 MB/s)

Which greatly exteneded RAW video recording times in all RAW video modes.

-Full Real-Time preview:

ML offers many custom RAW video modes which can present higher resolutions than native Canon video modes, offering better image quality and more shooting capabilities.
One problem with these custom modes is preview during RAW video recording, it can be either cropped or stretched (part of frame is missing), here are some examples:

2560x1440p 1:1 mode on 700D:

-Here we are comparing frame size among Real-Time Canon Cropped preview (what's being displayed on screen) vs Actual frame size, ML Framing preview (what's being recorded):
[gifv]https://i.imgur.com/KATkDai.mp4[/gifv]


-So why not to just use ML Framing preview? good question, because it has very low refresh rate (around ~2-3 FPS), here is a refresh rate comparison among Real-Time Canon Cropped preview and ML Framing preview:
[gifv]https://i.imgur.com/Ktzidb6.mp4[/gifv]


Framing preview becomes even more laggy when recording, beside it has negative impact on write speed too.
At this stage you may ask us to use that card, unfortunately that magic doesn't work here . . but Magic Lantern is full of Magic, let's explore it then :) :

Back in 2020 @Bilal started looking into how Preview works, and was able to control preview successfully to some point with some limitations.

It's 2023 now, so what's up? Well, it truned out that our DIGIC 5 models which are ~10 years old are fully capable of previewing RAW data at high resolutions:
Here is an showcase ML Framing preview vs Full Real-Time preview in 2560x1440p 1:1 custom mode:

[gifv]https://i.imgur.com/SzN87Ct.mp4[/gifv]



Another comparison:

1736x2214 1x3 mode, Stretched Canon preview vs ML Framing vs New Full Real-Time preview:

[gifv]https://i.imgur.com/G0uTba0.mp4[/gifv]



Yet another comparison, shows how preview got evolved:
(2.5K 1:1 preset as example)

[gifv]https://i.imgur.com/ti7NDyX.mp4[/gifv]


-It just a perfect match to Framing preview but in real-time
-If DIGIC 5 was that capable, what about DIGIC 6/7/8/X :)
-A new era for MLV RAW video
-This is applicable to 650D / 700D / EOS M / EOS M2 / 100D family
-Epic video about the new preview, also none epic one

The announcement for the new preview work came out yesterday by Bilal (on April 1st 2023),

Quote from: theBilalFakhouri on April 01, 2023, 06:37:50 PM

For more info, details and availability, follow this topic which will be updated frequently:
Crop mood - WYSIWYG Edition

And for users who are wondering, the contents of the announcement video, also the topic . . were 100% real :)
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

Chapter 4: What is going on with new models? All supported cameras are 10 years old!

Most of the supported cameras run Canon DIGIC 4 or 5 processor. This is because project started with EOS 5D mk2 and then was ported to EOS 550D - both DIGIC 4 based models.
Later it was found DIGIC 5 was a very similar processor internally with much more capabilities. And firmware was quite similar. It was relatively easy to maintain a single project covering both generations.

Problems started with DIGIC 6. This processor internally uses a different architecture than 4 and 5. Not very different, but enough that it didn't have some crucial features required by Magic Lantern to achieve many of its possibilities.
Canon also changed a few details in hardware inner-workings, which added to the difficulty of porting.

The slowdown.

Unfortunately at the same time project slowed down due to lack of... people. It is simply impossible to support more than a few models by a single person, and this is what had happened.
A1ex, the project lead - tried to attract new developers, but this didn't work out. His last effort was EOS M50 "fishy" build as a proof it is possible to achieve.

At some moment @names_are_hard joined the scene with his 200D. After that @coon with EOS RP and me with EOS R.

Quote from: Greg on April 01, 2019, 01:43:25 PM


A light in the tunel.

After a lot of work with great help of other forums and Discord server members, in early 2020 we made an achievement - for the first time since "fishy" build we were able to boot new models into Magic Lantern menus. This included mentioned EOS 200D, R and RP.

Quote from: names_are_hard on March 29, 2021, 02:13:37 AM
Don't get excited, it's just the menus, nothing works...  but:

[gifv]https://i.imgur.com/UP8gqR0.mp4[/gifv]


Teamwork with Kitor, thanks!

Quote from: coon on June 14, 2021, 10:36:09 AM
I have catched up with R / M50 / 200D progress and finally got Magic Lantern menus running on RP:



We started to learn a lot about reverse engineering and also Canon cameras - remember that we "just joined" the project at that moment, without any knowledge how earlier models work.
We quickly found that while similar on the surface - almost everything had changed in minor or major way -this includes both hardware and firmware. Unfortunately this means double the work - we had to understand both how old models work and how that relates to the ones we work on.

This is (unfortunately) the ongoing theme for last 2 years.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

Chapter 5: So what is the status of newer models? (DIGIC 6, 6+, 7, 8, X)

In general: it is a very early work.

We were able to bring up multiple Digic 6, 7, 8 and X models into Magic Lantern menus.
There are some features that work, but nothing really exciting. What works depends mostly on camera generation.

Except EOS 200D there are no public builds available. This may change in future!

DIGIC 6: EOS 750D, EOS 7D Mark II
DIGIC 6 is the "big unknown". It lacks some features needed to modify firmware behavior (cache hacks).

On 750D we were able to get into ML menu, LiveView overlays, attached lens info details and disable 30 minute liveview shutdown timer aka "webcam mode".
@names_are_hard was able to get his 7D Mark II into menus last month. For now it has some groundbreaking issues that we are not sure how to fix.

We also know how to activate over and underexposure zebras in LV!
This needs a major rewrite in Magic Lantern as new hardware can do much more (e.g. both under and over exposure at the same time with a custom thresholds and styles)

Quote from: names_are_hard on March 06, 2023, 12:14:12 AM
https://i.imgur.com/apyBOfQ.mp4

Barebones port, no ML features yet.

DIGIC 7: EOS 200D
DIGIC 7 introduces new features that replace those missing in previous generation (MPU).
We have some issues with memory management that will be needed later to use bigger parts of camera RAM.

EOS 200D is probably the best supported model. Like 750D, ML menu, LiveView overlays, attached lens info details and disable 30 minute liveview shutdown timer aka "webcam mode".
With many other "under the hood" things that are not yet important for end users.

Quote from: names_are_hard on October 27, 2022, 12:06:48 AM
Don't get too excited - very few features work.  But, it's ML, and it's on 200D.

(...)

https://i.imgur.com/yvZX1V1.mp4

DIGIC 8: EOS M50, R, RP, 850D, 250D, PowerShot SX70 HS and SX740 HS

What is the general status?

DIGIC 8 is similar-ish to DIGIC 7. Unfortunately on this generation we have a long standing bug.
It leads the camera to a crash in a few minutes of operation. We roughly know where it crashes, we don't understand why.
Additionally the memory management mentioned in DIGIC 7 is completely broken now - we can't use it at all.



With the firmware on DIGIC 8 models Canon changed some internal representations of data like aperture, shutter speed and ISO.
Magic Lantern code needs a major adaptation to fit in those new values into existing functions. This is a major blocker for the future.

With models starting with EOS R, firmware encryption has changed. This mean we can't create "ML install" program anymore.
Fortunately we found another way to enable the boot flag. Less user friendly, but exists.

What models we work on?

Currently all listed models: M50, R, RP, 850D and 250D have similar functionality. ML Menu, LV overlays, attached lens info, and ability to disable 30 minute  liveview shutdown timer aka "webcam mode".

On PowerShot SX740 HS we were able to activate CR3 RAW shooting, even though this camera allows only for JPEGs.

Recently I started working on PowerShot SX70 HS. It already reaches ML menus, but I likely have a stub or two wrong, as it locks the camera if some features are enabled.

Quote from: kitor on March 26, 2022, 09:52:02 PM
CR3 raw from SX740  8)

Extra points: on EOS R in "clean HDMI mode" we can enable zebras both on internal screen and HDMI - each screen with its own thresholds and styles!



Quote from: kitor on January 03, 2022, 06:48:57 PM


DIGIC X: EOS R5

DIGIC X inherits all problems listed on DIGIC 8 and unfortunately adds an extra one:
This time memory management doesn't work because Canon replaced old solution with a brand new one.
We know nothing about how exactly it works (to simply explain an idea - EOS R5 have more than 4GB of RAM, but can access only ~1.5GB at a time).
A lot (!) of Magic Lantern code is dependent on old solution right now.

Additionally, only EOS R5 and R6 work with the method of enabling Magic Lantern mentioned in DIGIC 8 generation.
Starting from EOS R7/R10 (technically from R3 but it is not ML target anyway) this method got unfortunately locked.

EOS R5: goes into Magic Lantern menus and displays Lens information.

Quote from: kitor on June 06, 2022, 01:08:05 PM

https://twitter.com/_kitor/status/1533767862531268608
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

Chapter 6: I noticed PowerShots on the list? Do you want to steal CHDK user base?!

Different firmware variants
Historically there were three "variants" of Camera firmware on Canon devices. We usually named them by their origins:
- "PowerShot" - running on point-and-shoot cameras
- "EOS" - DSLRs and mirrorless
- "Cinema EOS" - video cameras

Those three variants are very different. Think about your smartphone, internet modem and satellite box. There's a high likelihood they are all based on Linux - but each runs a different set of applications on top of it.
In "our" world the concept is very similar - each of those three areas runs operating system known as DryOS. Everything else (how camera operation is handled - majority of the software) is different.

Project targets
This is why CHDK project works only on devices that run "PowerShot" variant and only there.
Magic Lantern works on DSLRs and mirror less devices.
Cinema EOS is uncharted territory for now. (edit: for update on those, see July 2023 update)

It is very hard if not impossible to port CHDK over "EOS" firmware and Magic Lantern over "PowerShot" variant.

But life is confusing
If you own EOS M3, M5, M6, M10 or M100 (DIGIC 6 and 7 models of M series), you may already know they are target of CHDK project.
Canon decided to use "PowerShot" firmware variants on those models. Why? Our guess is as good as yours.

With DIGIC 8 generation - Canon tried to simplify things internally, and they got rid of "PowerShot" variant entirely - or at least this is what we think has happened.
EOS M50, M50 Mk II, M6 Mk II, M200 - they all run "EOS" variant again and thus are target of Magic Lantern.

But this also means that DIGIC 8 based PowerShot SX70 HS, SX740 HS, G5 X Mark II, G7 X Mark III, as well as the weird PowerShot Zoom - they run "EOS" firmare.
And by extension - those can be supported by Magic Lantern, even though this may be counter-intuitive at first glance.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

Chapter 7: What can I expect from you? How can I help?

First, a gentle reminder that ML project is not a company or commercial institution.
This means that while we are happy to listen to your suggestions and feedback, the project doesn't have any timelines, obligations and we are not bound to fulfill any of those.
If there's a build available for your camera - this is all we can offer. It includes all the features and also bugs you may encounter. Everything else should be considered as our best effort to support the community.

What is the status for X? When you will implement Y? Why not Z?
We realise people ask these questions because they're enthusiastic about ML - people want it on their cam and they want it to work well!  This is understandable.
When you have a hundred people asking the same questions, it can come across as nagging, or demanding.
Please try to remember that we will always announce significant progress on the forums, and we will always ask for testers if we need them.

For people who have technical knowledge and want to contribute - there's source code, QEMU emulation, Wiki, development Discord and many more.
We are aware our build system is very outdated and documentation is very lacking. Please consider joining Discord server - we will be more than happy to help you get started.

What about ChatGPT?
Slightly out of context, but since this question is asked very often in every single open source project on the planet.
ChatGPT will not help Magic Lantern. The majority of the work is in reverse engineering (and in general research). AI is trained on already available data - it can help you with problems that were already solved elsewhere.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.


pmiramon

Thank you for all this information, it is very valuable to understand where things come from, in such a well structured way! Documentation and information like this helps projects like this more than it seems.

Quote from: kitor on April 01, 2023, 09:13:24 PM


By the way, what does this graph represent? The activity in the forum or in firmware development? It is very enlightening

Thanks for your efforts

Dardo

I love the work you guys do. Thanks for everything!  :D

kitor

Quote from: pmiramon on April 02, 2023, 04:10:04 PM
By the way, what does this graph represent? The activity in the forum or in firmware development? It is very enlightening

This is commits graph in the Magic Lantern repository. To be exact, until 2021 it is commits from official Bitbucket (now Heptapod), starting from 2021 it is magiclantern_simplified Github fork where we work on Digic 6/7/8/X support.

There will be another status update in coming days from @names_are_hard - this time technical one.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

niamyagnik

Is there still active work ongoing on the DIGIC 8? or has the bug stopped work.
Thanks

kitor

The bug never stopped the work. Just consider there are exactly two of us working on all DIGIC 6, 7, 8 and X models. There's no point spending much effort on DIGIC 8/X when it inherits all the 6/7 issues anyway.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

niamyagnik

so you are fixing the 6/7 issues and not changing the digc 8/X?
Is there a version of ML available to the public for the M50 or is it still in testing?
thanks

names_are_hard

We are working on all D678X cameras that we feel like working on.  There is no ML for M50 at this time.

kitor

I mean, this was stated very clearly:

QuoteExcept EOS 200D there are no public builds available.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

Walter Schulz

There will always be a fair amount of users still asking what has been answered.
All you can do is hoping to lower the amount by having a good write-up. And you have!

Until we have a better way to communicate progress (or lack of): What about a quarterly update (can be much shorter)?

scrizz

Quote from: Walter Schulz on April 19, 2023, 09:54:32 AM
What about a quarterly update (can be much shorter)?

That's beginning to sound like work.  ;)
Doing quarterly updates for the execs to justify your existence and expenses.  ::)

We have to remember that these are volunteers. If it starts feeling like a job/chore, they might lose interest and will no longer want to do it.

just my 2c  :)

kitor

QuoteWe have to remember that these are volunteers.

You forgot that you quoted actual mod and very experienced community member.
In fact we discussed that internally some time ago and we will try to do that. Just because it actually reduces noise, since we can redirect any of those recurring questions to the status thread.

Next update will be posted somewhere early June. Depends when I'll have time to write it and others to review.
Since quarterly updates are not 5-years-of-stuff-to-describe, it will be likely much shorter one, but a bit more detailed. We want to keep the form of writing "in layman's words", providing links to additional (more detailed) resources where possible.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.


kitor

Please note that this update contains things that happen since April 2023, when we posted a huge "status" update summarizing last 5 years of development.

If you are not familiar with the previous update, or want to refresh your memory a bit, here it is (just scroll this thread to the first post :) ):
Current state of Magic Lantern project (updated April 2023)



Chapter 1: Updates on Digic 6 / 7 / 8 / X support

Except existing "not yet dead edition" build for EOS 200D, there are no new builds available.
Everything listed here is available as source code only, and that's for a good reason.

Since April, we have initial support for a few extra models:
DIGIC 6: EOS 80D (on par with 750D), 5D Mark IV (goes to menus, very unstable)
DIGIC 7: EOS 77D (on par with 200D)
DIGIC 8: SX70 is finally merged, but still unstable.

BTW: @names_are_hard also merged 70D support to magiclantern_simplified (D6+ repo) as we both have one as our test subject.

Quote from: names_are_hard on April 24, 2023, 10:07:20 AM
I now have a 5D4.

I also have this:



Quote from: kitor on May 21, 2023, 03:45:18 PM


77D.

Quote from: kitor on April 27, 2023, 08:16:41 AM


After a quick test I have a bad stub or two (tasks view doesn't display names properly[1], in LV our menu closes almost instantly[2]), other than that seems on par with 750D.
That was a quick (< 3 hour) job after i learned how to use Ghidra Version Tracking, thanks to @reyalp.

Ok, it took a week prior to that as I took the opportunity to create a proper loader script for Ghidra that should improve ghidra project quality a lot. More on that later.

Side note: Canon XF605 / Cinema EOS discoveries

Recently we've been contacted by a group of people trying to reverse engineer XF605 camera.
This is Cinema EOS line, which is a different firmware "base" than our models. For details refer to Chapter 6: I noticed PowerShots on the list? Do you want to steal CHDK user base?! of previous status update.

What they found is very interesting (and promising!). Digic DV7 seems very similar to DIGIC X as we know it. But not only that - firmware seems to be very related to our "EOS" variant, there's in fact a ton of similarities. Not tested yet, but we think ML "hello world" should be portable over this platform, maybe even our GUI system.

Some details are available in @ponguinforum threads:
Canon XF605: Teardown and Datasheets
Canon XF605: Digic DV 7 == Digic X rebranded for camcorders?
D214 / Diva / drysh UART on XF605

How this relate to Magic Lantern? Is support of C100/C300/etc. possible? We don't know.


Chapter 2: Code level improvements

I won't go into much details here, for more info check out Names's recent technical status update.
Let me just take some highlights:

Some time ago we started documenting relation between DryOS versions and data structures. This was started by @names_are_hard find that some structures in ML code doesn't seem to match any known model.
This resulted in a big cleanup of task-related structures that in the long run should reduce complexity and avoid some hard to debug bugs.

I cleaned up Digic 6+ boot code to match older models. Now it is much easier to start a port of new model / rom version for those cameras.

Names is working on crop_rec_4k_mlv_snd branch merge into magiclantern_simplified repo. This is a challenging task due to how branches were previously handled.

Chapter 3: Experiments

I experimented a bit around Zico (GPU) core on Digic 6 and up. Turns out we have some OpenVG and possibly OpenGL capabilities via EGL. Unfortunately a lot of code is missing, so utilizing them may be a huge task.
However, what I was able to achieve is GPU-accelerated ML GUI. For now it exists only as an experiment as we still didn't solve some caveats (refresh rate on D6/D7, instability on D8).
More on that in Compositors, layers, contexts in RGB and YUV - How Digic 7(6?)+ draw GUI.

Quote from: kitor on June 14, 2023, 09:56:58 PM
A few days ago I was able to achieve this. Full double-buffered ML menu with hardware indexed rgb -> RGBA rendering on Zico.

https://www.youtube.com/watch?v=q5aps3qHU_w

While old method felt "laggy" and was prone to rendering glitches, this is so damn fast. Feels exactly like using 5D3.

This is all thanks to a single Zico function - GrypDsCoreDrawImageToVramForEqualPhase.

Chapter 4: What can I expect from you? How can I help?

First, a gentle reminder that ML project is not a company or commercial institution.
This means that while we are happy to listen to your suggestions and feedback, the project doesn't have any timelines, obligations and we are not bound to fulfill any of those.
If there's a build available for your camera - this is all we can offer. It includes all the features and also bugs you may encounter. Everything else should be considered as our best effort to support the community.

What is the status for X? When you will implement Y? Why not Z?
We realise people ask these questions because they're enthusiastic about ML - people want it on their cam and they want it to work well!  This is understandable.
When you have a hundred people asking the same questions, it can come across as nagging, or demanding.
Please try to remember that we will always announce significant progress on the forums, and we will always ask for testers if we need them.

For people who have technical knowledge and want to contribute - there's source code, QEMU emulation, Wiki, development Discord and many more.
We are aware our build system is very outdated and documentation is very lacking. Please consider joining Discord server - we will be more than happy to help you get started.

What about ChatGPT?
Slightly out of context, but since this question is asked very often in every single open source project on the planet.
ChatGPT will not help Magic Lantern. The majority of the work is in reverse engineering (and in general research). AI is trained on already available data - it can help you with problems that were already solved elsewhere.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

October 2023 (delayed) update

There was not much going on in D6+ areas, as we got busy IRL. However we have a three notable mentions coming from @a.sintes:

Ultrafast framed preview (5D3)

A new, ultra-fast preview mode for 5D3 and possibly other models not supported by "crop mood" builds right now. This is a mid-step between old preview and hardware accelerated one.

Learn more here

Quote from: a.sintes on August 23, 2023, 05:10:58 PM
After some time spent tweaking Danne's crop_rec codebase for the 5D3, I finally got some interesting results around the framed preview, with visible performance increases, leading to an ultrafast framed preview feature.


Latest feature overview video:


SD/CF dual slots free space display (5D3)

Pretty self explanatory :) Details: LINK

Quote from: a.sintes on October 19, 2023, 04:50:25 PM
Hello,

Following a request on our Discord channel, I've implemented a very small feature to replace the legacy "free space" display in the ML top bar (top-right corner) with a new one including:
- a regular refresh (every 2 seconds min.) of the top bar (no need to do some specific operation in the menus to potentially trig a random update)
- an indication of the card councerned by the display (CF or SD prefix)
- a potential display of the two card slots in case of SD+CF usage (e.g.: 5D3)

It look like this:


A new focus sequencing module

Description, downloads: Focus sequencing (5D3)

Quote from: a.sintes on October 25, 2023, 05:26:31 PM
Hello,

Just released a new focus sequencing feature allowing us to create and manage a list of focus point created via autofocus with specific transition durations, then easily replay the sequence step-by-step while recording a video, using a single button push.

Demonstration video:


Technical updates

@names_are_hard prepared a technical update covering last months. You can read it here.

And the (obligatory) reminder :)
What can I expect from you? How can I help?

First, a gentle reminder that ML project is not a company or commercial institution.
This means that while we are happy to listen to your suggestions and feedback, the project doesn't have any timelines, obligations and we are not bound to fulfill any of those.
If there's a build available for your camera - this is all we can offer. It includes all the features and also bugs you may encounter. Everything else should be considered as our best effort to support the community.

What is the status for X? When you will implement Y? Why not Z?
We realise people ask these questions because they're enthusiastic about ML - people want it on their cam and they want it to work well!  This is understandable.
When you have a hundred people asking the same questions, it can come across as nagging, or demanding.
Please try to remember that we will always announce significant progress on the forums, and we will always ask for testers if we need them.

For people who have technical knowledge and want to contribute - there's source code, QEMU emulation, Wiki, development Discord and many more.
We are aware our build system is very outdated and documentation is very lacking. Please consider joining Discord server - we will be more than happy to help you get started.

What about ChatGPT?
Slightly out of context, but since this question is asked very often in every single open source project on the planet.
ChatGPT will not help Magic Lantern. The majority of the work is in reverse engineering (and in general research). AI is trained on already available data - it can help you with problems that were already solved elsewhere.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.

kitor

A very RAW update, February 2024

Another small, a bit delayed (for a reason), but this time a very exciting update!

To put this in context: Draft for Q4 update was ready for over a month now. But the situation was so dynamic we decided to push it for a bit later.
As you will see, this was definitely worth the wait :)

Chapter 1: RAW video on 7D2 (Digic 6)!

To the point: @heder recorded a first clip of RAW video on 7D2!

No builds, no code - don't expect them soon. It was a very dirty PoC. Proves it is possible though :)

Quote from: heder on January 17, 2024, 10:53:04 PM
I have RAW video up and running. (...) But going there without ML RAW video is bad karma, so I had to investigate every possible way of getting the camera to do as I wish.
(...)
Im was recording the entire RAW buffer including black border, 1984x1068x14bits,  around 85 MiB/s for 24 fps. I will try to record a large file using a fast CF card ( DMA-6)  in the following days to see if I get frame drops. The following video was recording using 25 fps, and recoded to H.264





Chapter 2: RAW video / mlv_lite on 200D (Digic 7)!!

@names_are_hard had a breakthrough in understanding new EDMAC API on Digic 7 models.
With that, after a bit of hacking now 200D records RAW

Quote from: names_are_hard on February 09, 2024, 04:08:05 AM
Several years of learning ML code, trying to understand fragmentary docs, and in some cases writing the docs myself, 200D has raw video:



At the moment this is only 14 bit, no lossless mode, with 40MB/s limit of SD card interface it is enough for continuous 720P recording.

There are no builds available for now, only dirty source code. For more details visit the quoted thread.



Chapter 3: RAW video / mlv_lite on EOS R (Digic 8 )!!!

Remember how I always written "don't count on D8 models soon?" This was because D8 models changed EDMAC once again, only this time also hardware has changed.
And we found no patents to put those new things into any perspective.

Yet, after @names_are_hard got 200D to work he pointed me to some bits of R code that may be useful. And they were.
We don't understand new controller very well, but it was enough to make it work for mlv_lite. Which after a few days of hard work turned into this:

Quote from: kitor on February 11, 2024, 05:50:20 PM


Yes, that's EOS R. Yes, it is Digic 8. Yes, we said don't count on it as EDMAC is completely different.
Yet here we are  8)

Again, no builds yet. This one is ever more hacked on top of what Names did for 200D. At this moment whatever resolution and frame rate I choose, R is able to get me around 30-60 frames of RAW video.
This is not a limitation of SD interface (it is SDXC, my card easily benchmarks over 100MB/s write), but rather some bug in a code I didn't trace yet.

In any way, we made a significant process in understanding how EDMAC works in "new world", you can expect technical documentation for those coming somewhere in a near future :)

Update:
Quote from: kitor on February 17, 2024, 10:25:57 PM


Turns out I just forgot I have two nearly identical cards: Lexar 64GB x1000 and x1666. And I used the slower one.
With x1666 (~100MB/s write) I get continuous 1080p30 14 bit RAW.

Chapter 4: What can I expect from you? How can I help?

As always, a gentle reminder that ML project is not a company or commercial institution.
This means that while we are happy to listen to your suggestions and feedback, the project doesn't have any timelines, obligations and we are not bound to fulfill any of those.
If there's a build available for your camera - this is all we can offer. It includes all the features and also bugs you may encounter. Everything else should be considered as our best effort to support the community.

What is the status for X? When you will implement Y? Why not Z?
We realise people ask these questions because they're enthusiastic about ML - people want it on their cam and they want it to work well!  This is understandable.
When you have a hundred people asking the same questions, it can come across as nagging, or demanding.
Please try to remember that we will always announce significant progress on the forums, and we will always ask for testers if we need them.

For people who have technical knowledge and want to contribute - there's source code, QEMU emulation, Wiki, development Discord and many more.
We are aware our build system is very outdated and documentation is very lacking. Please consider joining Discord server - we will be more than happy to help you get started.

What about ChatGPT?
Slightly out of context, but since this question is asked very often in every single open source project on the planet.
ChatGPT will not help Magic Lantern. The majority of the work is in reverse engineering (and in general research). AI is trained on already available data - it can help you with problems that were already solved elsewhere.
Too many Canon cameras.
If you have a dead R, RP, 250D mainboard (e.g. after camera repair) and want to donate for experiments, I'll cover shipping costs.