Messages

I have no evidence it is faster or slower, and wasn't trying to explain that.

Did you try Levas's advice in the linked thread?  You should do that and report results.  In that thread, not this one

1.2.3 quad buffers output, rather than triple buffers.  This is probably related to changes in output capability.  I don't know for sure if this is why things are "slower" (or even if 1.2.3 is slower!  There's certainly no "extra processor" allocated).  You would expect this to increase latency by about a frame, which is unlikely to be perceptible.

ML enables clean HDMI in the same way it works on every other cam, by stopping display of Canon decorations.  Not quite sure what you're asking here.

As always, if you're concerned about changes in performance: you have to measure it.  People are terrible about estimating performance and often think a change makes things worse (or better!) when really it just feels different.

Technically we agree on point 4 - MLVFS converts MLV files, in a way which is generally not noticeable   I called this "yes" because you have to install this extra thing in order to use files from ML with other software.

1 - probably not
4 - yes
5 - yes (ML lives on the card, not the cam)

ever thing i have is in python 2 so i guess i'll stay there as its what i know

Python 2 is dead.  It's been dead since 2020.  You'll have to update at some point.
https://www.python.org/doc/sunset-python-2/

The good news is I already updated everything in my repo to work with modern linux and modern python.

Are you saying that from the hardware perspective any 100D will work perfectly and support all video recording modes specified in the crop mood topic if i have a compatible memory card and a correctly configured ML?

Nah, I was just saying you're mistaken to think the 100D has special problems.  You can find reports of problems for all cams.  You're checking the wrong thing.  Can you find reports of 100D *doing what you want it to do*?  The answer is yes.  There are still other reasonable questions in that area, including "can *I* make it do that reliably enough for my needs?"

And specifically re "will work perfectly" - that is not what Magic Lantern has ever been   It is glitchy, confusing, and will never be 100% reliable.  We try to improve it over time.

You did, but you maybe didn't read as far forwards as you needed, and you don't have enough understanding of the context.

The reported problems were (probably, in both cases, provably in one case) due to the user not understanding how to setup ML to work properly on their cam.

Many people have reported 100D working fine.  Some have reported problems.  The conclusion to draw here is that ML can be awkward to configure correctly, and that variances in card performance can make very hi-res recordings unreliable for *some* people.

You can also find reports of people unable to get ML working how they want on M, 650D, 5D3 - in fact, every camera.

Found a quite interesting shell command, EngMirrorDump.  This takes an integer, and returns info on a shamem region.  This includes the name of the subsystem (engine), and address offset in shamem.  It also dumps content to card.  Results from 200D.  The full list is long, but these at least are interesting and show it's useful to us:

Code Select Expand

EngMirrorDump 3   (CHANNEL) Addr:4000,Size:1000,Flag:0
EngMirrorDump 29  (CHANNEL) Addr:26000,Size:1000,Flag:0
EngMirrorDump 37  (CHANNEL) Addr:30000,Size:1000,Flag:0
EngMirrorDump 65  (CHANNEL) Addr:57000,Size:1000,Flag:0
EngMirrorDump 66  (CHANNEL) Addr:58000,Size:1000,Flag:0

These are the EDMAC channel ranges.

Full list:

Code Select Expand

EngMirrorDump 0   (PWRCNT/SETTER) Addr:1000,Size:1000,Flag:0
EngMirrorDump 1   (KAISER) Addr:2000,Size:1000,Flag:0
EngMirrorDump 2   (SHAREMEM) Addr:3000,Size:1000,Flag:0
EngMirrorDump 3   (CHANNEL) Addr:4000,Size:1000,Flag:0
EngMirrorDump 4   (CHNSW) Addr:5000,Size:1000,Flag:0
EngMirrorDump 5   (HEAD) Addr:6000,Size:1000,Flag:0
EngMirrorDump 6   (HEAD) Addr:7000,Size:1000,Flag:0
EngMirrorDump 7   (PEPPER) Addr:8000,Size:1000,Flag:0
EngMirrorDump 8   (FAUST) Addr:9000,Size:1000,Flag:0
EngMirrorDump 9   (WOMBAT) Addr:a000,Size:1000,Flag:0
EngMirrorDump 10  (WOMBAT) Addr:b000,Size:1000,Flag:0
EngMirrorDump 11  (WOMBAT) Addr:c000,Size:1000,Flag:0
EngMirrorDump 12  (AFFINE) Addr:10000,Size:1000,Flag:0
EngMirrorDump 13  (FIXER) Addr:11000,Size:1000,Flag:0
EngMirrorDump 14  (TAIWAN) Addr:12000,Size:1000,Flag:0
EngMirrorDump 15  (RABBIT) Addr:13000,Size:1000,Flag:0
EngMirrorDump 16  (POSTER/SARIDON) Addr:15000,Size:1000,Flag:0
EngMirrorDump 17  (HISTORY) Addr:16000,Size:1000,Flag:0
EngMirrorDump 18  (PCFG) Addr:18000,Size:1000,Flag:0
EngMirrorDump 19  (WOMBAT) Addr:19000,Size:1000,Flag:0
EngMirrorDump 20  (LUCKY) Addr:1a000,Size:1000,Flag:0
EngMirrorDump 21  (DANCING) Addr:1b000,Size:1000,Flag:0
EngMirrorDump 22  (DANCING) Addr:1c000,Size:1000,Flag:0
EngMirrorDump 23  (PONY) Addr:1d000,Size:1000,Flag:0
EngMirrorDump 24  (LOTUS) Addr:1e000,Size:1000,Flag:0
EngMirrorDump 25  (LUCKY) Addr:20000,Size:1000,Flag:0
EngMirrorDump 26  (LUCKY) Addr:22000,Size:1000,Flag:0
EngMirrorDump 27  (CPUIF) Addr:24000,Size:1000,Flag:0
EngMirrorDump 28  (LUCKY) Addr:25000,Size:1000,Flag:0
EngMirrorDump 29  (CHANNEL) Addr:26000,Size:1000,Flag:0
EngMirrorDump 30  (DISTER) Addr:28000,Size:1000,Flag:0
EngMirrorDump 31  (SARIDON2) Addr:29000,Size:1000,Flag:0
EngMirrorDump 32  (QUARK) Addr:2a000,Size:1000,Flag:0
EngMirrorDump 33  (SUMMA) Addr:2b000,Size:1000,Flag:0
EngMirrorDump 34  (SMAP) Addr:2d000,Size:1000,Flag:0
EngMirrorDump 35  (OHYEAR) Addr:2e000,Size:1000,Flag:0
EngMirrorDump 36  (MICROU) Addr:2f000,Size:1000,Flag:0
EngMirrorDump 37  (CHANNEL) Addr:30000,Size:1000,Flag:0
EngMirrorDump 38  (MOSSY) Addr:34000,Size:1000,Flag:0
EngMirrorDump 39  (PEPPER) Addr:37000,Size:1000,Flag:0
EngMirrorDump 40  (SHREK) Addr:38000,Size:1000,Flag:0
EngMirrorDump 41  (SUSAN_A) Addr:39000,Size:1000,Flag:0
EngMirrorDump 42  (SUSAN_B) Addr:3a000,Size:1000,Flag:0
EngMirrorDump 43  (LUCKY) Addr:3c000,Size:1000,Flag:0
EngMirrorDump 44  (LUCKY) Addr:3e000,Size:1000,Flag:0
EngMirrorDump 45  (LUCKY) Addr:40000,Size:1000,Flag:0
EngMirrorDump 46  (LTKIDS) Addr:42000,Size:1000,Flag:0
EngMirrorDump 47  (SUSAN_A) Addr:43000,Size:1000,Flag:0
EngMirrorDump 48  (PEPPER) Addr:45000,Size:1000,Flag:0
EngMirrorDump 49  (BAUST) Addr:46000,Size:1000,Flag:0
EngMirrorDump 50  (HAIZEN) Addr:47000,Size:1000,Flag:0
EngMirrorDump 51  (HISTORY2) Addr:48000,Size:1000,Flag:0
EngMirrorDump 52  (HISTORY2) Addr:49000,Size:1000,Flag:0
EngMirrorDump 53  (HISTORY2) Addr:4a000,Size:1000,Flag:0
EngMirrorDump 54  (HISTORY2) Addr:4b000,Size:1000,Flag:0
EngMirrorDump 55  (CUMULO) Addr:4c000,Size:1000,Flag:0
EngMirrorDump 56  (CAPTAIN) Addr:4d000,Size:1000,Flag:0
EngMirrorDump 57  (CDM) Addr:4e000,Size:1000,Flag:0
EngMirrorDump 58  (WEAVER) Addr:4f000,Size:1000,Flag:0
EngMirrorDump 59  (WEAVER) Addr:50000,Size:1000,Flag:0
EngMirrorDump 60  (VERSARCH) Addr:51000,Size:1000,Flag:0
EngMirrorDump 61  (OPTIMUS) Addr:52000,Size:1000,Flag:0
EngMirrorDump 62  (ELISION) Addr:53000,Size:1000,Flag:0
EngMirrorDump 63  (COMPASS) Addr:54000,Size:1000,Flag:0
EngMirrorDump 64  (PURE) Addr:55000,Size:1000,Flag:0
EngMirrorDump 65  (CHANNEL) Addr:57000,Size:1000,Flag:0
EngMirrorDump 66  (CHANNEL) Addr:58000,Size:1000,Flag:0
EngMirrorDump 67  (ECHIZEN) Addr:59000,Size:1000,Flag:0
EngMirrorDump 68  (ECHIZEN) Addr:5a000,Size:1000,Flag:0
EngMirrorDump 69  (ECHIZEN) Addr:5b000,Size:1000,Flag:0
EngMirrorDump 70  (ECHIGO) Addr:5c000,Size:1000,Flag:0
EngMirrorDump 71  (ECHIGO) Addr:5d000,Size:1000,Flag:0
EngMirrorDump 72  (ECHIGO) Addr:5e000,Size:1000,Flag:0
EngMirrorDump 73  (POSTER) Addr:5f000,Size:1000,Flag:0
EngMirrorDump 74  (BIZEN) Addr:60000,Size:1000,Flag:0
EngMirrorDump 75  (BIZEN) Addr:61000,Size:1000,Flag:0
EngMirrorDump 76  (BIZEN) Addr:62000,Size:1000,Flag:0
EngMirrorDump 77  (BINGO) Addr:63000,Size:1000,Flag:0
EngMirrorDump 78  (BINGO) Addr:64000,Size:1000,Flag:0
EngMirrorDump 79  (BINGO) Addr:65000,Size:1000,Flag:0
EngMirrorDump 80  (COBALT) Addr:66000,Size:1000,Flag:0
EngMirrorDump 81  (COBALT_2) Addr:67000,Size:1000,Flag:0
EngMirrorDump 82  (CUMULO_2) Addr:68000,Size:1000,Flag:0
EngMirrorDump 83  (DAFIGARO) Addr:69000,Size:1000,Flag:0
EngMirrorDump 84  (ELISION_2) Addr:6a000,Size:1000,Flag:0
EngMirrorDump 85  (LUCKY) Addr:6b000,Size:1000,Flag:0
EngMirrorDump 86  (LUCKY) Addr:6c000,Size:1000,Flag:0
EngMirrorDump 87  (PEPPER) Addr:6d000,Size:1000,Flag:0
EngMirrorDump 88  (SADIRS) Addr:6e000,Size:1000,Flag:0
EngMirrorDump 89  (SIBORE) Addr:6f000,Size:1000,Flag:0
EngMirrorDump 90  (POSTER) Addr:70000,Size:1000,Flag:0
EngMirrorDump 91  (XRESON) Addr:71000,Size:1000,Flag:0
EngMirrorDump 92  (OPTIMUS_2) Addr:72000,Size:1000,Flag:0
EngMirrorDump 93  (SUB_CTL_A_0) Addr:73000,Size:1000,Flag:0
EngMirrorDump 94  (SUB_CTL_A_1) Addr:74000,Size:1000,Flag:0
EngMirrorDump 95  (SUB_CTL_A_2) Addr:75000,Size:1000,Flag:0
EngMirrorDump 96  (SUB_CTL_B_12) Addr:76000,Size:1000,Flag:0
EngMirrorDump 97  (SUB_CTL_B_3) Addr:77000,Size:1000,Flag:0
EngMirrorDump 98  (SUB_CTL_C) Addr:78000,Size:1000,Flag:0
EngMirrorDump 99  (SUB_CTL_D_13) Addr:79000,Size:1000,Flag:0
EngMirrorDump 100 (SUB_CTL_D_2) Addr:7a000,Size:1000,Flag:0
EngMirrorDump 100 (SUB_CTL_D_2) Addr:7a000,Size:1000,Flag:0
EngMirrorDump 101 (SUB_CTL_D_4) Addr:7b000,Size:1000,Flag:0
EngMirrorDump 102 (SUB_CTL_E_1) Addr:7c000,Size:1000,Flag:0
EngMirrorDump 103 (SUB_CTL_E_2) Addr:7d000,Size:1000,Flag:0
EngMirrorDump 104 (SUB_CTL_F) Addr:7e000,Size:1000,Flag:0
EngMirrorDump 105 (SUB_CTL_G) Addr:7f000,Size:1000,Flag:0
EngMirrorDump 106 (SUB_CTL_H) Addr:80000,Size:1000,Flag:0
EngMirrorDump 107 (SUB_CTL_K) Addr:81000,Size:1000,Flag:0
EngMirrorDump 108 (SUB_CTL_M) Addr:82000,Size:1000,Flag:0
EngMirrorDump 109 (ELISION_MULTI) Addr:83000,Size:1000,Flag:0
EngMirrorDump 110 (CUMULO_MULTI) Addr:84000,Size:1000,Flag:0
EngMirrorDump 111 (OPTIMUS_MULTI) Addr:85000,Size:1000,Flag:0
EngMirrorDump 112 (COBALT_MULTI) Addr:86000,Size:1000,Flag:0
EngMirrorDump 113 (ECHIZEN2_MULTI) Addr:87000,Size:1000,Flag:0
EngMirrorDump 114 (ECHIZEN3_MULTI) Addr:88000,Size:1000,Flag:0
EngMirrorDump 115 (ECHIGO2_MULTI) Addr:89000,Size:1000,Flag:0
EngMirrorDump 116 (ECHIGO3_MULTI) Addr:8a000,Size:1000,Flag:0
EngMirrorDump 116 (ECHIGO3_MULTI) Addr:8a000,Size:1000,Flag:0
EngMirrorDump 117 (LUCKY1_MULTI) Addr:8b000,Size:1000,Flag:0
EngMirrorDump 118 (LUCKY2_MULTI) Addr:8c000,Size:1000,Flag:0
EngMirrorDump 119 (LUCKY3_MULTI) Addr:8d000,Size:1000,Flag:0

If you know the name of an Engine, and want to monitor MMIO usage, this lets you find where it occurs.

Sorry, it's not that easy

Progress does not happen at a steady speed

I'm still documenting DMA and EDMAC stuff, got distracted by not having working tools for benchmarks.

I know you got DMA working directly, but did you ever have dma_memcpy() work on 7D2?  Or related functions, there's an async one for example.

Do you mean add features to the build?  You'll need to know C, and have time to learn the code, and Ghidra, to understand the rom.  What do you know so far?

The 6D2 does not have a CF slot.

Okay, good to learn about another major bug.  Well, you know, "good".

Obviously this is something that should be fixed.  Especially since on modern cams, trying to allocate all the SRM buffers triggers an OS assert and therefore crashes.

Looking here in exmem.c, this looks like a badly designed API by us:

Code Select Expand

283 struct memSuite *_shoot_malloc_suite(size_t size)
284 {
285     //qprintf("_shoot_malloc_suite(%x)\n", size);
286
287     if(size)
288     {
289         /* allocate exact memory size */
290         return shoot_malloc_suite_int(size);
291     }
292     else
293     {
294         /* allocate as much as we can */
295         size_t max_size = shoot_malloc_autodetect();
296         struct memSuite * hSuite = shoot_malloc_suite_int(max_size);
297         shoot_full_suite = hSuite;
298         return hSuite;
299     }
300 }

The max_size guessing code should probably be split out into its own function.  Then if you want to allocate a very large amount, you call that first, and pass the size to shoot_malloc_suite().  But now you can subtract some margin first, and avoid exhausting the pool.  I imagine it won't be that simple, but the current design doesn't even let you try!

This is an interesting topic, but it's also off topic for this thread.  Please create a new one if more discussion is useful.

Yes! But on the EOS M, the screenshot fumction only works to capture the ML Menus. Raw video needs to be turned off, so False colors and other overlays are not recorded.

Are you only checking using screenshots from benchmark?  Have you tried screenshots from Debug menu?

Debug menu does this:

Code Select Expand

take_screenshot(SCREENSHOT_FILENAME_AUTO, SCREENSHOT_BMP | SCREENSHOT_YUV);

Bench module code does this:

Code Select Expand

take_screenshot("bench%d.bmp", SCREENSHOT_BMP);

The screenshot code can capture both.  Benchmark code is asking it to not copy LV data.

Not sure what you mean.  Like this?

[Updates for 2023 Oct - 2024 Mar]

Updates for 2023 Oct - 2024 Mar
Last commit considered: https://github.com/reticulatedpines/magiclantern_simplified/commit/8ce0e1903594b0c8cd4ff9a7712f5066a9394b3c

A long time since my last update!  There was significant Real Life, but also, it took me quite a lot of time to get raw video working on 200D.

Headline changes
More in details below, but here's the bigger or funner pieces that made it in.

- Raw video for 200D, 7D2, R (in branches, no, you can't use this yet, but probably next month)
- Screenshots, including from benchmarks, are now .bmp, not .ppm.  Much more convenient
- Benchmarks are now much faster on slow cards / cams

Details!
1100D build was broken due to the low memory limits on this cam.  Some features removed to make it work again.

7D2 had a hard to track down bug due to being very close to the limit of allowed tasks.  Increased this limit during early boot process.

EDMAC stubs found for 7D2 and 200D.  Kitor has found these for R as well (maybe RP too?  I haven't been keeping track of external branches).  This is extremely important for new cams   In dev branch, there's code for EDMAC mem -> mem copies on 200D.  We have raw video working on a few cams, but this code is in branches.  Support is not clean yet, we need to integrate the different approaches on different Digic gens, and make the code more general, before it can be nicely merged into dev.

Significant bugs in ML were found and fixed during this work.  Lots of null pointer bugs in the old code, since the old cams don't crash if you read (or write!) through a null pointer.  MMU cams are not so forgiving.

Related to EDMAC, I refactored some FPS_REGISTER code to make it easier for D678X cams.  These cams use a different address range for this MMIO region, and in some cases different ways of accessing.  We'll need to do more of this, since the old ML code often hard-codes MMIO addresses, in files in src/, when in reality these are per cam constants - and different from D678X cams.

Created a tool for automatically determining image format and displaying it with correct proportions, given a buffer of data.  Works pretty well, can handle RGB, YUV and Bayer in the formats commonly found in cam memory.  See "src/tools/image/image_buffer_guesser/display_buf.py -h"

Moved all of contrib to tools.  Contrib had become a bad name.  Removed some unused code, too, including qemu, this has had its own repo for a long time now: https://github.com/reticulatedpines/qemu-eos/tree/qemu-eos-v4.2.1/magiclantern

More Dev Guide work.  This is slow work, but I think it will eventually be very helpful.  This update adds an overview about hardware: https://github.com/reticulatedpines/magiclantern_simplified/blob/eb84a464e5994a036f22dfa61fa95f885b684bf7/developer_guide/02_00_hardware.md

6D2 support finally tested on a physical cam.  I added this years ago and nobody volunteered.  Now we have a new dev / tester who has updated this cam to newer firmware, and fixed some bugs in my blind port.  Thanks!

Some improvements to benchmarks.  These are now time boxed, that is, they don't write a fixed amount of data, but will stop early if enough time has passed.  On fast cams / cards, this is the same behaviour as before.  On slow cams / cards, this makes the tests much faster to complete, with (so far) no observed problems with the results.

Screenshots: these now save in BMP format, not PPM.  This means...  you can view them just by opening the card.

6D2 1.1.1 now has limited support.  Our one dev / tester (unsure if they have an account here, CrypticKlippo on Discord), suggests it's stable.  It's the standard "has ML GUI, has no features" early build.  A good place to work from to add things.

https://github.com/reticulatedpines/magiclantern_simplified/commit/d16c293288737193bc785f09e8f8cd86a888feb6

Great, let me know if you have any problems.  The docs are very incomplete.  The docs generally for ML are quite awkward, sadly.  They're split over many different sources, quality varies.  Finding what you need is often difficult.

Hence why I'm trying to make a coherent dev guide, but I'm expecting this to take a long time.

ML probably logged the errors to the card, that would also be useful.

And of course a pic of the errors shown on screen.

Thanks, that I can try and copy   Won't be for a while though, got a bunch of tasks queued up.

It's expected to work.

If you explain why it doesn't work for you, perhaps we can help.

Thanks, but that doesn't really mean anything to me, and doesn't explain how I could test what difference dual ISO makes.

I did it to compare the maximum dynamic range achievable with magic lantern, because 200D is basically ISO invariant and wouldn't benefit from dual ISO, but the older cams do

I have a 200D.  What's the evidence that it's "ISO invariant"?  How would I measure this, and how would I measure dual ISO performance?

I have dual ISO working on 200D but don't know that much about photography, being more of a dev, so don't know how to test the results.

Yes, programming, reverse engineering, I have a background in embedded, firmware, virtualization development.

Good timing, we have someone who is currently updating the partially-supported ROM version from 1.0.5 to 1.1.1.  I am checking the changes currently.

I think it's likely that 6D2 can quickly be brought up to the same level of support as 200D, it's the same digic gen.  That would mean getting raw video on 6D2 is quite plausible.  On 200D, dual ISO works, too, as well as network support.

Discord is probably the easiest way to get you quickly setup.  But I'm happy to help here.  You will want this repo: https://github.com/reticulatedpines/magiclantern_simplified/commits

It should build easily on modern linux, I use Debian Testing.  We don't have a good checker for pre-reqs, missing rst2html is a common problem, for that you want python3-docutils.

Currently, it's fw 1.0.5.  1.1.1 is the new target, I expect my checks will be completed and 1.1.1 support merged within a week.

Micro SD vs SD card, both are the same.  Real shooting experience.

There is no way for us to replicate what you are seeing because you don't give any information, so it's not useful to anyone.  If you list what cards you use, in what conditions, people could check.