Topics

I've just created an About page for ML: http://www.magiclantern.fm/about

Any suggestions?

Hi-res pics (thanks Roger)

http://www.pbase.com/rcicala/6d

[Device safety]

I think it's good to know what not to do and what things require special care when digging into ML code - so here's a quick draft. Suggestions or relevant links are welcome.

Device safety

Theory

- Safe Hacking - or how to practice Safe Hex (must read!)

Persistent modifications

- Be very careful when changing properties (camera settings and events) - they are saved in NVRAM. A bad setting can cause permanent malfunction.
- Disable prop_request_change calls for early ports! Only enable them after checking ALL prop_request_change calls used in ML.

Memory usage

- For new ports: ML should reserve some memory for itself, and mark it as used (so Canon code will never overwrite it). Currently it's done by shrinking the memory pools used for malloc or AllocateMemory.
- Memory usage: malloc pool should have at least 100K free, AllocateMemory pool should have at least 1MB free. If not, remove some features or try to use shoot_malloc.
- Malloc checking: http://www.magiclantern.fm/forum/index.php?topic=3506.0 (work in progress)
- Stack checking: look in Debug->Show Tasks.
- Don't create too many tasks / semaphores / message queues / property handlers / other objects.

CPU usage

- Look in Debug->Show CPU usage.
- Use msleep, semaphores and message queues - these will allow the other tasks to run.
- Don't use tasks with very high priority (lower values = higher priority). These may block Canon code.

Don't slow down or block Canon code

- Don't do complicated things from property handlers (don't make them wait - you will slow down Canon code). Setting variables and changing other properties is OK. Taking pictures is not (call schedule_remote_shot instead).

- Don't use (big) delays in gui_main_task (this includes all event handling functions). If your code takes too much time to run, consider running it on a separate task.

- When you hook from state objects, make sure your code finishes really fast (especially in LiveView hoks). Otherwise you will slow down the frame rate and cause jitter (not funny at all for video users).

Error handling

- Be very careful disabling Canon's error handling (side effects are unknown, but can be bad).
- Check for errors on: malloc, AllocateMemory, file I/O, things that return a pointer...
- Use the ASSERT macro (it will save small log files whenever assertions are broken).

At shutdown

- DryOS tasks will shutdown at:
    - next msleep call
    - next take_semaphore call
    - next msg_queue_receive call, but ONLY if the message queue is empty

- Don't pump events faster than the process from the other end can process them (otherwise the camera will not turn off).
- Use if (ml_shutdown_requested) to know when to stop (in most cases you won't need it).

If you screw it up

- Ask before trying any of the following!
- http://magiclantern.wikia.com/wiki/Unbricking
- Diagnostic tools




Legal safety

- Check your local law about reverse engineering, copyright and digital locks!
- Read EFF's FAQ on reverse engineering (also here)
- Do not copy Canon code! (it's copyrighted)
- Do not publish things like encryption keys or copyrighted code! You can only publish what's needed to develop an interoperable program (e.g. document Canon's API, which is not copyrightable - see here).
- Do not agree to EULAs that prohibit reverse engineering! (tip: the firmware updates provided by Canon Europe do not contain anti-reverse engineering clauses at the time of writing this).
- Be careful not to infringe any patents (e.g. don't implement a MP3 player feature).
- Be careful not to mass-brick cameras! (this will cost Canon a lot; hint: Why Panasonic want to block hacked firmware)


Happy Hacking!

The small one-line help strings are pretty much always up to date. Would be nice if we could say the same thing about the larger help pages (they are not always in the same order as the menus, it's hard to tell which camera has which features, or sometimes the functionality just changed).

So, a proposal somewhat inspired from the literate programming philosophy:

Code Select Expand


#define DOC(x,...)

struct menu_entry zebra_menus[] = {
    {
        .name = "Global Draw",
        .priv       = &global_draw,
        .max = 3,
        .select_Q   = toggle_disp_mode_menu,
        .display    = global_draw_display,
        .icon_type = IT_BOOL,
        .help = "Enable/disable ML overlay graphics (zebra, cropmarks...)",
        DOC(
            Choose when to enable Magic Lantern overlay graphics: (zebra, cropmarks, histograms, audio meters, ML shooting info...):

            * in **LiveView** only;
            * in **QuickReview** (photo mode outside LiveView) only;
            * **Both modes** (default);
            * **Disabled**.

            If you use `display presets`_, press Q on this item to switch the presets while you are in ML menu.
        )
    },
    ...


So, the RST code is embedded in ML source code, is discarded when compiling, and probably can be extracted without much effort. One can also preprocess the source code (mainly the camera-specific ifdef tags) and see what features are enabled on what model. This also ensures consistent naming of menu entries and help pages (since help pages are identified by the name of the menu entry).

Advantage: the help text is in the middle of the data structure that describes the menu (so it's probably a lot easier to notice when it becomes outdated, and just update it). It will probably be also helpful for new ports (one can see what feature X does without having to open a separate window for the manual).

Current user guide takes under 100K, so there's also a possibility to render the help as plain text (not as bitmaps). It will increase ML binary size and may not look as nice as the pre-rendered BMPs.

Thoughts?

Last night I've tried to use the roll/pitch sensors from 5D3 in order to detect when the camera is steady. It worked... sort of, but with serious stability issues (e.g. just pressing shutter halfway was interferring with the sensor).

Then I've tried the same trick in LiveView, with the motion detection engine, but reversed. This time, if the motion level is under the threshold for the last 5 frames, I assume you are holding the camera steady and a picture can be taken. You need to hold the shutter pressed half-way (or the star button).

Code: http://bitbucket.org/hudson/magic-lantern/changeset/bab21e8a4b5c

Idea: http://www.magiclantern.fm/forum/index.php?topic=3487.msg17859#msg17859

(and I've also forgot picture quality to JPEG after clearing the settings)

So, I've added a framework where ML can warn you if you change some settings by mistake. I've started with what I'd use:
- Mode: other than M
- Picture quality: other than RAW

Code: https://bitbucket.org/hudson/magic-lantern/changeset/7c3541430760

What other settings can benefit from such warnings?

Today I've found a wrong stub on 5D3 (strcmp). Most of the time it was correct - it gave a few false positives though. A few weeks ago we also spent a lot of time to find another wrong stub (msg_queue_post). Also, new ports tend to have some wrong constants that are not widely used, so the side effect is subtle or hard to notice (e.g. DLG_SIGNATURE, used only for redraws).

So... here's a series of automatic tests that should catch these mistakes easier: https://bitbucket.org/hudson/magic-lantern/changeset/a899850d2a39

The idea is to test each stub and constant (if possible), so it should cover pretty much all the low-level Canon API used by ML.

I've only tested it on 5D3. Try it on your cameras, and if anything fails, post your log file.

Feel free to add more tests or to improve the test engine.

Code Select Expand


[Pass] strlen("abc") => 0x3
[Pass] strlen("qwertyuiop") => 0xa
[Pass] strlen("") => 0x0
[Pass] strcpy(msg, "hi there") => 0x1a7560
[Pass] msg => 'hi there'
[Pass] snprintf(a, sizeof(a), "foo") => 0x3
[Pass] snprintf(b, sizeof(b), "foo") => 0x3
[Pass] strcmp(a, b) => 0x0
[Pass] snprintf(a, sizeof(a), "bar") => 0x3
[Pass] snprintf(b, sizeof(b), "baz") => 0x3
[Pass] strcmp(a, b) => 0xfffffff8
[Pass] snprintf(a, sizeof(a), "Display") => 0x7
[Pass] snprintf(b, sizeof(b), "Defishing") => 0x9
[Pass] strcmp(a, b) => 0x4
[Pass] memcpy(foo, bar, 6) => 0xb9158
[Pass] foo => 'asdfghuiop'
[Pass] memset(bar, '*', 5) => 0xb9164
[Pass] bar => '*****hjkl;'
       bzero32(bar + 5, 5)
[Pass] bar => '****'
       t0 = *(uint32_t*)0xC0242014 => 0x7273b
       msleep(250)
       t1 = *(uint32_t*)0xC0242014 => 0xb14ca
[Pass] ABS(mod(t1-t0, 1048576)/1000 - 250) => 0x7
       LoadCalendarFromRTC( &now )
       s0 = now.tm_sec => 0x14
       Date/time: 2012/10/29 23:44:20
       msleep(1500)
       LoadCalendarFromRTC( &now )
       s1 = now.tm_sec => 0x16
[Pass] mod(s1-s0, 60) => 0x2
[Pass] mod(s1-s0, 60) => 0x2
       m0 = MALLOC_FREE_MEMORY => 0x5bc28
       p = malloc(50*1024) => 0xe3c00
       m1 = MALLOC_FREE_MEMORY => 0x4f418
       free(p)
       m2 = MALLOC_FREE_MEMORY => 0x5bc28
[Pass] ABS((m0-m1) - 50*1024) => 0x10
[Pass] ABS(m0-m2) => 0x0
       m0 = GetFreeMemForAllocateMemory() => 0x3eb0d4
       p = AllocateMemory(256*1024) => 0x79f9d4
       m1 = GetFreeMemForAllocateMemory() => 0x3ab0c8
       FreeMemory(p)
       m2 = GetFreeMemForAllocateMemory() => 0x3eb0d4
[Pass] ABS((m0-m1) - 256*1024) => 0xc
[Pass] ABS(m0-m2) => 0x0
       m01 = MALLOC_FREE_MEMORY => 0x5bc28
       m02 = GetFreeMemForAllocateMemory() => 0x3eb0d4
       p = alloc_dma_memory(256*1024) => 0x4079fa14
       free_dma_memory(p)
[Pass] UNCACHEABLE(p) => 0x4079fa14
[Pass] CACHEABLE(p) => 0x79fa14
[Pass] UNCACHEABLE(CACHEABLE(p)) => 0x4079fa14
       p = (void*)shoot_malloc(24*1024*1024) => 0x5be60074
       shoot_free(p)
       m11 = MALLOC_FREE_MEMORY => 0x5bc28
       m12 = GetFreeMemForAllocateMemory() => 0x3eb0d4
[Pass] ABS(m01-m11) => 0x0
[Pass] ABS(m02-m12) => 0x0
       EngDrvOut(0xC0F14400, 0x1234)
[Pass] shamem_read(0xC0F14400) => 0x1234
       call("TurnOffDisplay")
[Pass] DISPLAY_IS_ON => 0x0
       call("TurnOnDisplay")
[Pass] DISPLAY_IS_ON => 0x1
       SetGUIRequestMode(1); msleep(500);
[Pass] CURRENT_DIALOG_MAYBE => 0x1
       SetGUIRequestMode(2); msleep(500);
[Pass] CURRENT_DIALOG_MAYBE => 0x2
       SetGUIRequestMode(0); msleep(500);
[Pass] CURRENT_DIALOG_MAYBE => 0x0
       GUI_Control(BGMT_PLAY, 0, 0, 0); msleep(500);
[Pass] PLAY_MODE => 0x1
[Pass] MENU_MODE => 0x0
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(500);
[Pass] MENU_MODE => 0x1
[Pass] PLAY_MODE => 0x0
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(500);
[Pass] MENU_MODE => 0x0
[Pass] PLAY_MODE => 0x0
       task_create("test", 0x1c, 0x1000, test_task, 0) => 0x8c800c8
[Pass] test_task_created => 0x1
[Pass] get_task_name_from_id(get_current_task()) => 'run_test'
[Pass] mq = (void*)msg_queue_create("test", 5) => 0x8ca00a6
[Pass] msg_queue_post(mq, 0x1234567) => 0x0
[Pass] msg_queue_receive(mq, (struct event **) &m, 500) => 0x0
[Pass] m => 0x1234567
[Pass] msg_queue_receive(mq, (struct event **) &m, 500) => 0x9
[Pass] sem = create_named_semaphore("test", 1) => 0x8cc02e2
[Pass] take_semaphore(sem, 500) => 0x0
[Pass] take_semaphore(sem, 500) => 0x9
[Pass] give_semaphore(sem) => 0x0
[Pass] take_semaphore(sem, 500) => 0x0
[Pass] rlock = CreateRecursiveLock(0) => 0x8ce00c6
[Pass] AcquireRecursiveLock(rlock, 500) => 0x0
[Pass] AcquireRecursiveLock(rlock, 500) => 0x0
[Pass] ReleaseRecursiveLock(rlock) => 0x0
[Pass] ReleaseRecursiveLock(rlock) => 0x0
[Pass] ReleaseRecursiveLock(rlock) => 0xf
[Pass] f = FIO_CreateFileEx(CARD_DRIVE"test.dat") => 0x4
[Pass] FIO_WriteFile(f, (void*)ROMBASEADDR, 0x10000) => 0x10000
[Pass] FIO_WriteFile(f, (void*)ROMBASEADDR, 0x10000) => 0x10000
       FIO_CloseFile(f)
[Pass] FIO_GetFileSize(CARD_DRIVE"test.dat", &size) => 0x0
[Pass] size => 0x20000
[Pass] p = alloc_dma_memory(0x20000) => 0x4079fa14
[Pass] f = FIO_Open(CARD_DRIVE"test.dat", O_RDONLY | O_SYNC) => 0x4
[Pass] FIO_ReadFile(f, p, 0x20000) => 0x20000
       FIO_CloseFile(f)
       free_dma_memory(p)
[Pass] FIO_RemoveFile(CARD_DRIVE"test.dat") => 0x0
       SW1(1,100)
[Pass] HALFSHUTTER_PRESSED => 0x1
       SW1(0,100)
[Pass] HALFSHUTTER_PRESSED => 0x0
[Pass] DLG_SIGNATURE => 0x6e4944


What if, instead of "if defined(CONFIG_50D) || defined(CONFIG_60D)" etc, we would use some macros to describe device capabilities?

e.g.

CAP_ROLLING_PITCHING // 60D, 7D, 5D3
CAP_DISPLAY_BUFFER_OVERRIDE // 5D2, 550D and newer
CAP_LCD_SENSOR // 550D, 500D
CAP_ASIF_DMA // all except 50D and VxWorks
CAP_EXMEM
CAP_PTP
...

Idea: http://www.stockholmviews.com/mup/mirror-up.html ("This is especially obvious at shutter speed 1/80s to  1/15s.")
Code: https://bitbucket.org/hudson/magic-lantern/changeset/866be16e840a
Try in tomorrow's nightly build.

Will it work? Worth polishing it?

Looks cool, anyone interested?

http://forums.dpreview.com/forums/post/41758598

I'd like someone with good writing skills to check these pages, and maybe suggest better text.

Limitations: fixed-width font, max 58 characters per line.

For the "undocumented feature", you can also suggest a page with the proportional fonts normally used for in-camera help, but that one would require the user to copy new help files for it to work.

Code Select Expand


void
draw_404_page()
{
    bmp_fill(COLOR_BLACK, 0, 0, 720, 480);

    bfnt_puts("404 Undocumented Feature", 10, 20, COLOR_WHITE, COLOR_BLACK);
   
    bmp_printf(FONT_MED, 10, 100, "This feature is probably not yet documented.");
    bmp_printf(FONT_MED, 10, 120, "After all, we are programmers, not tech writers.");

    bmp_printf(FONT_MED, 10, 180, "But... you can simply try it and see what it does.");

    bmp_printf(FONT_MED, 10, 240, "Then, write a short paragraph to describe it,");
    bmp_printf(FONT_MED, 10, 260, "and we will include it in the user guide.");

    bmp_printf(FONT_MED, 10, 320, "Thanks!");

}


Code Select Expand


void
draw_help_not_installed_page()
{
    bmp_fill(COLOR_BLACK, 0, 0, 720, 480);

    bfnt_puts("Help files not found", 10, 20, COLOR_WHITE, COLOR_BLACK);
   
    bmp_printf(FONT_MED, 10, 150, "Magic Lantern help files seem to have disappeared. Whoops!");

    bmp_printf(FONT_MED, 10, 250, "To browse help, please copy the DOC directory on your card.");

    bmp_printf(FONT_MED, 10, 350, "Or... just go out and take some pictures :)");
}


Thanks!

Nice series of short tutorials from our friend Enrique Rodben (in Spanish).

Tip #0 - Magic Zoom http://vimeo.com/29053985
Tip #1 - Image Effects http://vimeo.com/47638064
Tip #2 - Histograma http://vimeo.com/48671237
Tip #3 - Intervalometro & Bulb Ramping http://vimeo.com/49885260
Tip #4 - Monitorear Audio http://vimeo.com/51164054

[What are nightly builds?]

What are nightly builds?
- Nightly builds are binaries compiled daily from the latest source code, by an automatic program.
- Download: http://builds.magiclantern.fm/#/
- Feature comparison matrix
- Changeset log on Bitbucket

Update by RenatoPhoto (Oct 1, 2013):

Delete ALL old files from card i.e. autoexec.bin and the entire ML directory and replace by new files.  Do not mix old with new.

Who should use nightly builds?
- Anyone who likes to live on the bleeding edge, wants to help ML development and doesn't care about some features being broken.

Why should I use a nightly build?
- To try the latest features (e.g. you have requested a feature and it was implemented).
- To check if a bug was fixed.
- To help us with testing (please report any bugs you find).

What are the disadvantages of nightly builds?
- You are running untested code. The source code is updated daily, and development sometimes progresses very quickly. We can't test every little change on all cameras, so a few things may not work as expected.
- No documentation - the only help for new features is a small line of text, and ML source code.
- No guarantees - if it breaks, you get to keep both pieces. Well, the same is true for stable releases - but at least, the chances of things going wrong should be much lower.

What's new?
- See the feature comparison matrix.
- Browse the changeset log on Bitbucket.
- Look for feature requests marked as "DONE". You might find some cool stuff there.
- The list of changes is huge - if you want a nice summary, you need to wait for the next stable release.

Will it work on my camera?
- Hopefully yes.
- Make sure you download the correct version for your camera.
- Some cameras are no longer maintained by anyone, so a lot of things are probably broken. If you find broken stuff, and you have some basic C skills, some spare time, and you like the idea of programming your own camera, give it a try. Patches welcome.

Anything else?
- If you run bleeding edge code, we kindly ask you to give something back - bug reports. Tell us what is not working, so we can fix it.

Enjoy!

[What to download?]

Most features from this section were implemented after the 2.3 stable release. But... maybe you want to try something new from here (e.g. you have requested a feature and it was done). What to download?

Well... if you don't care about running bleeding edge code, you have two options:

1) Compile the latest source code (recommended - you have full control over the process, you can review the source code, adjust it to fit your needs and share your improvements).

2) Download a nightly build. They are compiled daily from the latest source code, by an automatic program - the build bot. However, you will be probably the first one who will try new (untested) stuff on your camera model.

If you run bleeding edge code, we kindly ask you to give something back - bug reports. Tell us what is not working, so we can fix it.

If you have trouble, just go back to the stable release.

To see what's new, watch the changeset on the source code repository. The list of changes is huge - if you want a nice summary, wait for the next stable release.

Thanks!

[hyperlinked]

Adam Sneddon just showed me today a wifi card similar to Eye-Fi, that might be compatible with ML and runs Linux.

I can't post the link to their website, because it would break their terms of use [ ]; instead, I'll link to some reviews.

http://arktronic.com/weblog/2011-10-01/flucard-pro-review
http://lesterchan.net/blog/2012/07/27/flucard-pro-review/

2.2 Copyright
The copyright in this website is owned by Trek or its licensors. No part or parts hereof may be reproduced, distributed, adapted, modified, republished, displayed, broadcast, hyperlinked, framed or retransmitted in any manner or by any means or stored in an information retrieval system without the prior written permission from Trek. However, you may download and print the Materials on this website for personal, non-commercial use only provided you do not modify the Materials and that you retain all copyright and other proprietary notices contained in the Materials. Without the permission from Trek, you may not insert a hyperlink to this website on any other website or reproduce any Material contained on this website on any other server.

The terms of use appeared when I tried to buy this card. And, obviously, it can't be found in any major stores and they don't ship to me.

Not related to ML, but this is must-watch.

We are very happy to announce the first public test version for the 5D mark III. Since a complete port may take some time and we realize that a lot of people are waiting for it, we have decided to release early a minimal version that should be useful and - we hope - with very little risk.

This first alpha version is based on v2.3 core and only has a small feature set, like zebras and focus peaking (see screenshot). It will not attempt to change any Canon settings or do any kind of permanent firmware modification. This also means the autoboot feature does not work yet - you will have to load Magic Lantern every time by running Firmware Update from Canon menu.

Since the 5D mark III has two card slots, we recommend running Magic Lantern from a small SD card that stays in the camera. All recording can take place on CF cards. The CF cards don't have to be bootable or prepared in any way, and you can freely swap and format them without problems.

What's next?Our goal is to have a rock solid and feature-complete port of Magic Lantern for 5D Mark III in the upcoming version 2.4. We are starting today with a minimal feature set, and once you, the testers, convince us that it's very stable, we can gradually add more features. The 5D mark III version will remain in alpha status until all important features can be fully ported.

 

Download ML for 5D3


Click here to read this article!

"""
With the Canon Rebel T2i, you can mod with Magic Lantern to turn 2k resolution to 4k resolution recording in 1080p @ 30fps.
"""

Source: a guy trying to sell a T2i

[Download ML for 5D Mark III - Alpha 3]

Firmware revision 1.1.3 now has an updated firmware installer:  http://www.magiclantern.fm/forum/index.php?topic=14704.0

Download ML for 5D Mark III - Alpha 3.

Usage:
1) Update camera firmware to 1.1.3.
2) Format the card from the camera.
3) Copy ML files on the card and run Firmware Update.

NOTES
THIS VERSION OF MAGICLANTERN HAS AUTOBOOT DISABLED
Which means you'll have to re-run the firmware update every time you power off the camera.

It's a safety measure: this version will not attempt to change any camera settings. Try it if you are not yet confortable with the nightly builds.

Risks:
- This is one of the safest versions as of right now, but its feature set is quite limited.
- Nothing is written into ROM, and no camera settings are changed by this version, so risks should be minimal.
- If anything goes wrong, we don't pay for repairs. Use Magic Lantern at your own risk!

Features:

- Card test at startup
- Histogram and waveform moved to bottom
- New experimental display for focus peaking (extreme sharpness)
- 16:9 bars for anamorphic preview
- Focus box moves faster (not yet customizable)
- After taking a picture and pressing Zoom In right away, you can scroll through pictures
- Saturation boost when adjusting white balance
- Warnings for bad settings (e.g. if you set picture quality to JPEG instead of RAW by mistake)
- Dim the red LED while recording (make it less distracting)

From alpha 2:
- Magic Zoom (zoom while recording), experimental focus peaking modes, ghost image, display presets
- Movie indicators, movie logging, rec/standby notification, force LiveView for manual lenses
- Gradual exposure in movie mode
- HDR video
- Brightness, contrast, saturation, display gain, color schemes, UniWB correction, upside-down mode
- Clean HDMI with pillarboxes
- Anamorphic and fisheye correction
- Image review tweaks (exposure adjust, remember zoom position...)
- Task and CPU usage info

From alpha 1:
- zebras, focus peaking, cropmarks, spotmeter, histogram, waveform, vectorscope, audio meters.
- card benchmark, debug info, stability tests.

Tips:
- Disable Auto Power Off.
- Recommended usage: copy ML on a small SD card, keep it in the camera, and use CF cards for shooting.
- You can use any card combination, just don't put ML on both cards.
- EyeFi cards are working!!! (thanks kikouyou)

Known issues:
- Some users reported a few random lock-ups with Alpha 1, but I could not reproduce any.
- The experimental focus peaking modes are a bit too slow (will slowdown LiveView frame rate).




FOR DEVELOPERS ONLY

Want to help with the porting proces? Use this special updater file to enable the bootflag and dump the firmware. With this, you can compile your own autoexec.bin and run it on your camera.

(BE AWARE: This file will set a boot flag which cannot be undone (yet).  This will increase your camera boot time slightly and also increase the wake up time from 0.2 sec to 2 sec.) added by Renato 11-6-2013

You will be able to try the entire ML feature set, but the risks of permanent damage are a lot higher than with the public alphas. Use it at your own risk.

[Post Deflicker]

This script is obsolete; I've implemented the same algorithm directly in the camera (Post Deflicker).

If you are interested in the internals of this algorithm, you are welcome to take a look.

Bulb ramping is not 100% flicker-free and it will never be, since it measures the exposure from previous shots and it won't account for natural flicker. At fast shutter speeds, the flicker comes from the discrete shutter speeds enforced by Canon firmware.

So, I'm trying to get a script which should do perfect deflicker, based on robust statistics (median) and RAW image data. It should work even if you simply shoot in Auto or Manual, without any ramping.

Version 0.2 - tested on 800 pictures (video below) and added a real-time deflicker graph.
Version 0.1 - only tested on 10 pictures, with very promising results.

You need Python 2.x, numpy/scipy/matplotlib, dcraw, ufraw, imagemagick... and hopefully that's all.

Usage: create a folder named "raw" and another one named "jpg", paste your CR2 files under "raw" and run the script.

Code Select Expand


# RAW deflickering script
# Copyright (2012) a1ex. License: GPL.

from __future__ import division
import os, sys, re, time, datetime, subprocess, shlex
from math import *
from pylab import *

def progress(x, interval=1):
    global _progress_first_time, _progress_last_time, _progress_message, _progress_interval
   
    try:
        p = float(x)
        init = False
    except:
        init = True
       
    if init:
        _progress_message = x
        _progress_last_time = time.time()
        _progress_first_time = time.time()
        _progress_interval = interval
    elif x:
        if time.time() - _progress_last_time > _progress_interval:
            print >> sys.stderr, "%s [%d%% done, ETA %s]..." % (_progress_message, int(100*p), datetime.timedelta(seconds = round((1-p)/p*(time.time()-_progress_first_time))))
            _progress_last_time = time.time()


def change_ext(file, newext):
    if newext and (not newext.startswith(".")):
        newext = "." + newext
    return os.path.splitext(file)[0] + newext

def get_median(file):
    cmd1 = "dcraw -c -D -4 -o 0 '%s'" % file
    cmd2 = "convert - -type Grayscale -scale 500x500 -format %c histogram:info:-"
    #~ print cmd1, "|", cmd2
    p1 = subprocess.Popen(shlex.split(cmd1), stdout=subprocess.PIPE)
    p2 = subprocess.Popen(shlex.split(cmd2), stdin=p1.stdout, stdout=subprocess.PIPE)
    lines = p2.communicate()[0].split("\n")
    X = []
    for l in lines[1:]:
        p1 = l.find("(")
        if p1 > 0:
            p2 = l.find(",", p1)
            level = int(l[p1+1:p2])
            count = int(l[:p1-2])
            X += [level]*count
    m = median(X)
    return m

ion()

progress("Analyzing RAW exposures...");
files = sorted(os.listdir("raw"))
i = 0;
M = [];
for k,f in enumerate(files):
    m = get_median(os.path.join('raw', f))
    M.append(m);

    E = [-log2(m/M[0]) for m in M]
    E = detrend(array(E))
    cla(); stem(range(1,len(E)+1), E);
    xlabel('Image number')
    ylabel('Exposure correction (EV)')
    title(f)
    draw();
    progress(k / len(files))

progress("Developing JPG images...");
i = 0;
for k,f in enumerate(files):
    ec = 2 + E[k];
    cmd = "ufraw-batch --out-type=jpg --overwrite --clip=film --saturation=2 --exposure=%s '%s' --output='jpg/%s'" % (ec, os.path.join("raw", f), change_ext(f, ".jpg"))
    os.system(cmd)
    progress(k / len(files))
[/size]

Here's some code which causes ERR02 - Card1 cannot be accessed.

Code Select Expand


FILE* f = FIO_CreateFileEx("A:/test.log");
for (int i = 0; i < 1000; i++)
    FIO_WriteFile(f, "Will it blend?\n", 15);
FIO_CloseFile(f);


On the other hand, this code runs fine:

Code Select Expand


for (int i = 0; i < 1000; i++)
{
    FILE* f = FIO_CreateFileEx("A:/test.log");
    FIO_WriteFile(f, "Will it blend?\n", 15);
    FIO_CloseFile(f);
}


What could be wrong?

If you like fast zebras, it may be possible to get focus peaking with a similar method - with DIGIC registers.

We have 4 scenarios for focus peaking:

- movie recording: we can only use C0F14xxx (display device), since these registers do not affect recording.
- photo mode (1x) or movie mode standby: we can also use registers that adjust image sharpness.
- zoom mode (5x, 10x): same as photo mode, but some registers may be different.
- playback mode: C0F14xxx (display device) or any custom image processing algorithm (not real-time).

The method of using sharpness registers looks very promising, at least for photo mode, so here are my findings on this:

- c0f2116c controls picture style sharpness: 0 = none, ffff0000 = max. It affects most of the other sharpness controls, so set a positive sharpness in picture style menu or in this register.
- c0f0f280 looks like a high-pass filter, but amplifies noise a lot. c0f2135c does some noise reduction and can be used here.
- c0f23164 enhances edges from bright areas, and looks great for peaking as long as the image is exposed to the right. If this could be configured to enhance edges from midtones, it's probably what we are looking for.
- c0f0f29c (cartoon look) enhances most edges, but it's too strong and adds a lot of noise. Doesn't work in zoom mode.
- other registers that seem to enhance edges or alter sharpness: c0f0f17c, c0f23140, c0f2314c, c0f2313c, c0f23134, c0f23128, c0f23124, c0f2311c, c0f23118, c0f230bc.

You can already try these things with some older binaries, or by compiling digic poke yourself, see  http://magiclantern.wikia.com/wiki/Register_Map/Brute_Force

If you notice other registers that can be used for focus peaking, please let me know.

Problem: Canon logs get trimmed (because they are too big), or they don't log all messages.

Is there a way to log all DebugMsg calls?

YES

https://bitbucket.org/hudson/magic-lantern/src/tip/src/dm-spy.c

Credits goto g3gg0 for the ultimate patching tool

Code Select Expand


        cache_fake((uint32_t)&DryosDebugMsg, B_INSTR((uint32_t)&DryosDebugMsg, my_DebugMsg), TYPE_ICACHE);


Like title says.

Eddie Svard (discocalculi) just found the register that controls picture style sharpness. By setting this register to 0, I've noticed that sharpness becomes slightly lower than Canon's 0 level (as in Neutral 0,-4,-4,0). The effect is very slight, noticeable at 10x zoom on some fine texture, but it's definitely there.

Code: https://bitbucket.org/hudson/magic-lantern/changeset/e1ee35bfef54 - you can set the sharpness to -1 from Picture Style submenu.

Would this be helpful in any ways? Maybe masking moire or bitrate artifacts?

http://www.kickstarter.com/projects/phillipstearns/glitch-textiles

Watch the video and don't try it at home

... for researching how to brick and unbrick it.

Before killing it completely (which could happen if the unbricking research fails), I think it's a good idea to fix some 500D quirks first.

What I've noticed at first sight:
- Focus limit warnings when autofocusing in LiveView (minor).
- Canon tells you to use the * button to focus (minor, should be easy to hide).
- ML bottom bar may draw over Canon bottom bar (this bug was in 5D2 too, but it's fixed in 2.3).

Other known ML bugs or missing features in 500D:
- HDR video not working (it was working for a few frames when holding AF button).
- Scrollwheel not working in ML menu in movie mode.
- Stability test fails in some tricky situations (like focusing back and forth quickly while running the test).
- Some small FPS override quirks (didn't try this).
- No focus confirmation in LiveView.

Reported, but could not reproduce:
- ISO reverts from 160 to 200 in photo mode.

Other quirks noticed?

[1) Portable display test]

1) Portable display test

If your camera is not booting, this is the first diagnostic tool you can try, to make sure the hardware is alive.

This tool attempts to print some messages on the display without calling any routine from the main firmware. It only uses the bootloader routines, and works on any ML-enabled camera, any firmware version. More info about this test here.

autoexec.bin


2) LED blink test

If the display test did not work, here's a simpler test that only blinks a LED.

Works on: 500D, 550D, 600D, 50D, 60D, 5D Mark II, 1100D, all firmware versions.
autoexec.bin

This test blinks the card LED (C0220134 - SD LED and C02200BC - CF LED) from bootloader.

If it blinks:
- Camera is alive (obvious)
- Bootloader is OK (it loads autoexec.bin)
- We can run diagnostic code on it (for example, computing a checksum of the ROM or examining the boot process).

If it doesn't blink:
- Card is not bootable OR
- Bootflag is not enabled in the camera (ML not installed) OR
- Hardware failure (main board doesn't power up) OR
- Bootloader corrupted (very unlikely) OR
- Unsupported camera model (the LED address is different)

More LED addresses: http://magiclantern.wikia.com/wiki/Led_addresses

3) ROM dumper

From main firmware: at startup, ML will save the ROM contents to ML/LOGS/ROM*.BIN. To get a fresh ROM dump, simply delete those files, start the camera from a ML card, and wait until the card activity LED stops.

From bootloader (use if the main firmware cannot start):
Works on: 550D, 60D, 600D, 5D3 (all firmware versions). Can be updated for any other camera, on request.
autoexec.bin

4) Startup log

You can get a very detailed log about Canon boot process using the dm-spy-experiments branch.

http://builds.magiclantern.fm/jenkins/job/startup-log/

You can get even more details if you add some stubs (tracing points) and compile from source:

http://www.magiclantern.fm/forum/index.php?topic=2388.msg113154#msg113154

5) Startup log blinker

If the camera is unable to save a log file, we may want to blink Canon's startup messages using the card LED. You may use a second ML-enabled camera, or an Arduino board with a photocell, to decode the messages.

http://magiclantern.fm/forum/index.php?topic=13070

http://www.nycresistor.com/2012/08/21/ghosts-in-the-rom/

[5D2 only!]

Like title says. What parts of Canon code are running when you take a pic?

Background:

The tasks which handle picture taking, including RAW/JPEG compression, vignette correction and dark frame subtraction are designed as state machines.

http://magiclantern.wikia.com/wiki/StateObjects
http://a1ex.bitbucket.io/ML/states/

Some examples from 5D2:

SCSState, take a pic:

Code Select Expand


(1) -- 17 scsReleaseOn        -->(1)
(1) --  1 scsReleaseStart     -->(2)
(2) --  2 scsReleaseData      -->(4)
(4) --  3 scsDummyCapEnd      -->(5)
(5) --  4 scsDummyReadoutDone -->(6)
(6) --  5 scsCapReady         -->(7)
(7) --  6 scsCapEnd           -->(8)
(8) -- 10 scsReleaseEnd       -->(8)
(8) -- 18 scsReleaseOff       -->(8)
(8) --  7 scsFinalReadoutDone -->(1)


SCSState, bulb pic:

Code Select Expand


(1)  --  1 scsReleaseStart    -->(2)  // tg start, lock engine resources...
(2)  --  2 scsReleaseData     -->(4)  // lots of initializations, iso, tv, av, tp, po, register head1 interrupt to dummy cap...
(4)  --  3 scsDummyCapEnd     -->(5)  // some engio; register head1 interrupt to dummy readout done
(5)  --  4 scsDummyReadoutDone-->(6)  // engio, decides between normal capture, srcapture or eshutcapture
(6)  --  5 scsCapReady        -->(10) // setup image pass, engio, head1 interrupt to CapEnd, register vd interrupt
(10) --  8 scsVdInterrupt     -->(10)
(10) --  8 scsVdInterrupt     -->(10) // calls GetImageBuffer and seems to re-setup the sensor (also setting the FPS override registers)
....                                  // so maybe a long exposure is done by adding a few short ones?!
(10) --  8 scsVdInterrupt     -->(10)
(10) -- 11 scsBulbEnd         -->(10) // scs_struct.bulb_end = 1 => this causes scsVdInterrupt to "stop accumulation" (whatever that means)
(10) --  8 scsVdInterrupt     -->(10) // stop accumulation => NotifyLastVdTimerCBR => post event 9 to something
(10) -- 10 scsReleaseEnd      -->(10) // scs_struct.release_end = 1
(10) --  9 scsCapReady        -->(7)
(7)  --  6 scsCapEnd          -->(8)  // register head1 to readout done, RealtimeDefectsGrade -> 0xC0F0814C, post event 0xB to something else
(8)  --  7 scsFinalReadoutDone-->(1)  // huge, probably does raw conversion, noise reduction...


2.5s pic with dark frame NR:

Code Select Expand


FSS  :(0)  --  7 fssSW1On            -->(0)
SBS  :(1)  -- 13 sbsIgnoreRelease    -->(1)
SCS  :(1)  --  1 scsReleaseStart     -->(2)
SCS  :(2)  --  2 scsReleaseData      -->(4)
SCS  :(4)  --  3 scsDummyCapEnd      -->(5)
FSS  :(0)  --  8 fssSW2On            -->(0)
SCS  :(5)  --  4 scsDummyReadoutDone -->(6)
SCS  :(6)  --  5 scsCapReady         -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(10) --  8 scsVdInterrupt      -->(10)
SCS  :(7)  -- 10 scsReleaseEnd       -->(7)
FSS  :(0)  --  9 fssSW2Off           -->(0)
SCS  :(10) --  9 scsCapReady         -->(7)
SCS  :(7)  --  6 scsCapEnd           -->(8)
SCS  :(8)  --  7 scsFinalReadoutDone -->(1)
SBS  :(1)  --  1 sbsPostJob?         -->(2)
SBS  :(2)  --  2 sbsPrepareDark      -->(3)
SBS  :(3)  --  3 sbsDummyCapEnd      -->(4)
SBS  :(4)  --  4 sbsDummyReadoutDone -->(5)
SBS  :(5)  --  5 sbsDark             -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  6 sbsVdInterrupt      -->(6)
SBS  :(6)  --  7 sbsLongSecondTimer  -->(7)
SBS  :(7)  --  8 sbsCapEnd           -->(8)
SBS  :(8)  --  9 sbsReadoutDone      -->(9)
SBS  :(9)  -- 10 sbsDarkSub          -->(10)
SBS  :(10) -- 11 sbsWbDetect         -->(11)
SBS  :(11) -- 12 sbsWbDetectDone     -->(1)
SPS  :(2)  --  2 spsPostJob          -->(3)
SPS  :(3)  --  3 spsStartProcess     -->(3)
SDSf1:(2)  --  2 sdsPostJob          -->(2)
SDSr :(2)  --  2 sdfGetDevelopDevType?->(5)
SPS  :(3)  --  4 spsStartProcess_main-->(2)
SDSf3:(0)  --  3 sdsMem1toRAWcompressg-->(1)
SDSf3:(1)  --  3 sdsMem1toJpegDevelop-->(1)
SDSr :(5)  --  5 sdsYuvToDcfCompression>(5)
SDSr :(5)  --  5                     -->(2)
FCS  :(1)  --  1 fcsPostJob          -->(1)
FCS  :(1)  --  2 fcsExecJob          -->(1)
FSS  :(0)  --  2 fssNotifyJob        -->(0)
FSS  :(0)  --  3 fssPostJob          -->(0)
FSS  :(0)  --  4 fssAvailable        -->(0)
FSS  :(0)  --  6 fssCompleteJob      -->(0)
FSS  :(0)  --  7 fssSW1On            -->(0)


Code for getting these logs (5D2 only!):

Code Select Expand


/**
* Experiments on state objects
*
* http://magiclantern.wikia.com/wiki/StateObjects
*
**/

#include "dryos.h"
#include "bmp.h"
#include "state-object.h"
#include "property.h"

// 550D:
//~ #define SCS_STATE (*(struct state_object **)0x31cc)
//~ #define SCSES_STATE (*(struct state_object **)0x31D0)
//~ #define SCSSR_STATE (*(struct state_object **)0x31D4)

// 5D2:
#define SCS_STATE (*(struct state_object **)0x3168)
#define SCSES_STATE (*(struct state_object **)0x316c)
#define SCSSR_STATE (*(struct state_object **)0x3170)

#define SBS_STATE (*(struct state_object **)0x31C4)
#define SDS_REAR_STATE (*(struct state_object **)0x363C)
#define SDS_FRONT1_STATE (*(struct state_object **)0x36B0)
#define SDS_FRONT2_STATE (*(struct state_object **)0x36B4)
#define SDS_FRONT3_STATE (*(struct state_object **)0x36B8)
#define SDS_FRONT4_STATE (*(struct state_object **)0x36BC)

#define SPS_STATE (*(struct state_object **)0x320C)
#define FSS_STATE (*(struct state_object **)0x3c24)
#define FCS_STATE (*(struct state_object **)0x3c34)

#define LOG_SIZE 10000
static char log[LOG_SIZE] = "";

static int (*StateTransition)(void*,int,int,int,int) = 0;
static int stateobj_spy(struct state_object * self, int x, int input, int z, int t)
{
    int old_state = self->current_state;
    int ans = StateTransition(self, x, input, z, t);
    int new_state = self->current_state;
   
    if (self == SBS_STATE) { STR_APPEND(log, "SBS  :"); }
    else if (self == SCS_STATE) { STR_APPEND(log, "SCS  :"); }
    else if (self == SCSES_STATE) { STR_APPEND(log, "SCSes:"); }
    else if (self == SCSSR_STATE) { STR_APPEND(log, "SCSsr:"); }
    else if (self == SDS_REAR_STATE) { STR_APPEND(log, "SDSr :"); }
    else if (self == SDS_FRONT1_STATE) { STR_APPEND(log, "SDSf1:"); }
    else if (self == SDS_FRONT2_STATE) { STR_APPEND(log, "SDSf2:"); }
    else if (self == SDS_FRONT3_STATE) { STR_APPEND(log, "SDSf3:"); }
    else if (self == SDS_FRONT4_STATE) { STR_APPEND(log, "SDSf4:"); }
    else if (self == SPS_STATE) { STR_APPEND(log, "SPS  :"); }
    else if (self == FSS_STATE) { STR_APPEND(log, "FSS  :"); }
    else if (self == FCS_STATE) { STR_APPEND(log, "FCS  :"); }
   
    STR_APPEND(log, "(%d) -- %2d -->(%d)\n", old_state, input, new_state);
   
    return ans;
}

static int stateobj_start_spy(struct state_object * stateobj)
{
    if (!StateTransition)
        StateTransition = (void *)stateobj->StateTransition_maybe;
   
    else if ((void*)StateTransition != (void*)stateobj->StateTransition_maybe) // make sure all states use the same transition function
    {
        beep();
        return;
    }
  stateobj->StateTransition_maybe = (void *)stateobj_spy;
  return 0; //not used currently
}

static void shootspy_init(void* unused)
{
    log[0] = 0;
    stateobj_start_spy(SCS_STATE);
    stateobj_start_spy(SCSES_STATE);
    stateobj_start_spy(SCSSR_STATE);
    stateobj_start_spy(SBS_STATE);
    stateobj_start_spy(SDS_REAR_STATE);
    stateobj_start_spy(SDS_FRONT1_STATE);
    stateobj_start_spy(SDS_FRONT2_STATE);
    stateobj_start_spy(SDS_FRONT3_STATE);
    stateobj_start_spy(SDS_FRONT4_STATE);
    stateobj_start_spy(SPS_STATE);
    stateobj_start_spy(FSS_STATE);
    stateobj_start_spy(FCS_STATE);
}

void save_log()
{
    NotifyBox(1000, "%d ", strlen(log));
    FILE* f = FIO_CreateFileEx(CARD_DRIVE"shoot.log");
    FIO_WriteFile(f, UNCACHEABLE(log), strlen(log));
    FIO_CloseFile(f);
    beep();
}

INIT_FUNC("shootspy_init", shootspy_init);


What does it mean, in plain English?

* Bulb exposure seems to be done in small steps (smaller exposures added digitally). Not yet sure, but if it's true, it can be used to implement a lot of things from feature requests area.
* Dark frame duration can probably be controlled by user. That would mean, 5-minute exposure and 10-second dark frame. Could this be useful?
* Maybe understanding how to take pics without moving the shutter mechanism? This would be huge!

I'll be away during August, most probably with very limited internet access. Hopefully nothing will go wrong during this period.

For administrative issues, you can contact the other moderators, as it's very likely that I won't be able to reply.

Thanks for understanding, and enjoy Magic Lantern!

StartPlayWaveData - plays a test WAV file stored in ROM.

550D: WAV data is at ff45715c - contains RIFF and WAVE strings.

If memory resource is already allocated, the code does this:

Code Select Expand


if audio_struct.continuous_mem_buf /*off_0x2C, 0x26F8*/ != 0 /*NE*/:
    AJ_guess_sortof_a_copy__FROM.R1__TO.R0__LEN.R2(audio_struct.continuous_mem_buf /*off_0x2C, 0x26F8*/, 0xff45715c: pointer to 0x46464952, 0x2ee24)
    SamplingRate_bit_channel(0xbb80, 0x10, HALFWORD(*0xFF457172), 0x2ee00)


0xff064eb8 SamplingRate_bit_channel:

Code Select Expand


con_puts('\nSamplingRate = %d, bit = %d, channel = %d\n', arg0, arg1, arg2) => ret_con_puts_FF064ED8
audio_struct.off_0x4 /*0x26D0*/ = BYTE(arg2)
audio_struct.off_0x60 /*0x272C*/ = arg3
audio_struct.continuous_mem_buf /*off_0x2C, 0x26F8*/ = 44 + audio_struct.continuous_mem_buf /*off_0x2C, 0x26F8*/
SetSamplingRate_FSamplingRate(arg0, 0x1)
src:ASIF\ASIF.c()
AJ_PowerAudioOutput()
Audio\AudioIC.c_hHPTimer(0x3)
*(-32 + sp0) = @called_by:SamplingRate_bit_channel
StartASIFDMADAC(44 + audio_struct.continuous_mem_buf /*off_0x2C, 0x26F8*/, arg3, 44 + audio_struct.continuous_mem_buf /*off_0x2C, 0x26F8*/, arg3)


StartASIFDMADAC uses the following DIGIC registers:
0xC092020C, 0xC0920220, 0xC05000xx.

[Even if the camera is barely alive and just starts]

Do you have a damaged camera body lying around and picking up dust? We can give it a second life and get some use out of it.

Developers of Magic Lantern require physical camera bodies for research and experiments. We perform numerous trial and error tests done on cameras before getting something useful and stable. However, the camera doesn't have to be in perfect order to be useful. Depending on the nature and degree of the damage, most cameras still can be used for development or research purposes.

Here are some examples of suitable bodies:

• If you have a rental company and bodies have too high shutter counts or are simply too worn out to be still used out in the field: we can experiment with it until it dies.
• If the LCD screen or sensor has many dead pixels, or the screen is cracked. No problem for us, it's perfectly usable.
• Damaged lens mounts and contacts. Also very useful except for AF functions.
• If the camera has an electronic flaw that causes the battery to drain quickly, but overall function is OK: no problem for us.
• Even if the camera is barely alive and just starts, we need those cameras for researching how to revive them if anything goes wrong - for this, we need to try potentially dangerous code, without the fear of causing permanent damage. We can't afford running these tests on good cameras.Completely dead bodies can be used for studying the circuit board; also, once in a while bloggers can make sample pictures with other models than the ones they own.
• We are also looking for high-resolution pictures of circuit boards, to identify the chips used and look up their datasheets.

We are interested in the following models:

Canon 5D Mark I,II,III, 7D, EOS M, 500D/T1i, 550D/T2i, 600D/T3i, 650D/T4i, 1100D/T3, 450D, 400D, 40D, 50D, 60D.

With your help, we can overcome the need to share a single camera body over some continents (like we are doing now with the 7D). If there are more developers having the same model, they will be able to help each other out and make progress faster. So if you have a camera that you can no longer use, we will give it a second life!

If you'd like to help, please contact us at [email protected], or by replying to this forum thread.

The entire Magic Lantern community will be thankful if you'll just spread the word - either on your blog or on social networks.

Thank you!

Click here to read this article!

There were quite a few reports saying that Magic Lantern empties the battery "much faster" than Canon firmware. Obviously, no one said how much faster, so I've repeated my measurements.

Back in 2011, I've estimated it at roughly 5% more power draw, with zebras and peaking active, using the estimation displayed in Debug menu in 60D. That estimation measures the time needed for battery percentage to get lower by 1%. To measure power draw of Canon firmware, I simply stopped all ML tasks and left only the battery check - that was back then.

These days I've ran another test: I've started with a fully charged battery and left the 550D in LiveView until the battery fully discharged. First with Canon firmware, then did the same with ML. I've pressed the shutter halfway every 20 minutes to prevent the LiveView from turning off automatically.

Discharge times:
With Canon firmware (movie mode, 25p, backlight at max): 1h49m (109 minutes)
With Magic Lantern (same settings in Canon firmware, also with fast zebra, peaking (d2xy) and histogram active - starting from default config): 1h50m (110 minutes).

Take it with a grain of salt - I only did one discharge cycle for each firmware: first with Canon firmware (yesterday) and then, today with ML. I did not check the repeatability of the measurements in any way. Also, my 550D battery is quite old.

I hope this clears up all those urban legends, and I invite you to do your own tests.

Full review: http://www.dslrnewsshooter.com/2012/07/19/the-new-magic-lantern-2-hack-for-canon-dslrs-marcus-waterloo-tests-if-it-ready-for-pro-use/

[when removing or swapping CF cards]

Some users still experience shutdown problems on 5D Mark II when removing or swapping CF cards (they have to remove the battery to start the camera again). Problem fully understood, see fix below.

Short answer: after opening CF card door, wait for at least 5 seconds before removing the card.


I've never had this problems with the RC versions, and thought this to be fully solved on May 7.

If you have a 5D Mark II with firmware 2.1.2, please try this binary:
http://a1ex.magiclantern.fm/debug/5D2/dummy/autoexec.bin

It does nothing else other than starting Canon firmware. If you don't have the shutdown problem with this binary, but you have it with ML in a consistent way, please let me know via PM.

Since I can't reproduce this on my camera, the only way to fix it is with the help of a tester who has this problem.

[Read the user guide]

0. Read the user guide and try the nightly builds - what you need is probably already done.

1. Check if the feature was already requested. Do not open duplicate threads.

2. Remember that Magic Lantern is only a software mod, so don't request things like 1000 FPS, 4K or clean ISO 128000.

3. Do not bump your own feature requests!

4. Do not flood the forum with countless feature requests.

Breaking any of these rules may result in temporary ban. If you continue breaking them, the ban will be permanent.

Also, we'd like you to open a thread for each feature. Try not to hijack an existing thread, and try not to add more feature requests in the same thread. This lets us track the requests and mark them as DONE or WONTFIX.

As you know, in the PDFs that come in the stable releases we also mention the people who have contributed. But the task for mentioning everyone's contributions properly is not as easy as it seems - so I need a bit of help. The toughest problem that I want to avoid is to forget an important contribution.

Here's my draft for the in-camera version:

http://a1ex.magiclantern.fm/bleeding-edge/credits.png

http://twitter.com/snapsort/status/220264367210508289

My quick list of candidates:

- Auto ISO limits in movie mode
- Picture style controls (same controls as with Canon menu)
- 60D level indicator (Canon has one too)
- Defishing (very rough in LiveView, although good in playback mode)
- Image effects (MOVIE menu) - they don't do any harm (they are simply DIGIC controls), but I don't think anyone will use them for normal shooting.
- Flash / NoFlash (I've used it on compact cameras and missed it on dslr, but now I no longer use it)
- Silent pics - advanced modes (I intend to keep only simple, burst and maybe matrix modes)
- Mirror lockup sync'ed with self timer (anyone using it?)
- Follow focus with LCD sensor (I don't remember anyone saying he used this)
- From Display menu: image position, orientation, upside-down mode (although they can be useful when you need them)
- Sticky DOF preview / HalfShutter (I never used those, except sticky halfshutter when running through DIGIC register). Trap focus no longer needs this.
- Some things from Debug menu (like tasks, CPU usage...) - but keep indicators with free memory and so on.

From that list, only silent pics is known not to work in matrix mode on some cameras, and I'll just disable that feature where it doesn't work. All the other things are working fine, but aren't that useful IMO.

You can come up with other suggestions too.

Removed:
- auto ISO limits (unstable, interferes with manual ISOs)
- movie remap (sorry for that, I know it was very handy, but causes stability problems)
- record with SET button (100% useless)
- ML auto ISO algorithm (was present in december version, but was too slow for real use)
- zoom in face detection mode (found to be unreliable, and couldn't find any use for it)

I'm having some issues with movie mode remap (and ISO+LV). There were also a few bug reports on it (500D: black screen, 60D: not remembering settings).

Today I've got ERR70 on 60D by doing this:
1) started camera in M mode;
2) enabled LiveView;
3) pressed ISO+LV to go to movie mode;
4) pressed REC => instant ERR70.

I've tried with a minimal code base (the only extra code was switching to movie mode after 10 seconds). Still ERR70. After clearing Canon settings the problem disappeared, but it clearly shows that there's something fishy with this feature.

I'm seriously thinking to disable movie mode remap completely. Suggestions?

The RSS feed for the forum only contains 5 items. Can this be increased to around 20?

[Release Candidate]

Canon 1100D / T3
Current state: Release Candidate Danger, Will Robinson! Danger!

The main issues that are holding ML are:
Blindly maintained (development happens without a physical camera in our hands)
Lack of physical buttons (for menu, for example)
Lack of RAM (only 128MB; most other Rebel models have 256MB)
Low resolution sensor: 4290x2858 => 1430x952 theoretical max in LiveView
Slow SD interface (20MB/s, not enough for raw video)
Low display resolution (1-pixel wide items may be incorrectly displayed or aliased)

Short-term Todo List:
Enable raw video and silent photo capture (almost there)
Enable Lua scripting (feedback needed -- what works and what not?)
Make sure the stability tests are passing (feedback needed)
Write down which features work and which ones not (menu walkthrough, for testers)
Update the user guide (task common for all other camera models)
Done: Port 550D button hack to 1100D AE_COMP to open menu but retain the original long-press functionality.
Done: Start working on the fonts and display routines

Random bonus stuff:
Done: In-camera audio meters.

I will be posting updates here as I make progress -- Nanomad

The current version compiles and runs on 1100D.
RC 3: http://nanomad.magiclantern.fm/1100D/magiclantern-v2.3.1100D.RC3.zip Use development builds instead!!
Use the included .fir (for 1100D 1.0.5) to enable the bootflag.

Source code: http://bitbucket.org/hudson/magic-lantern/src/tip/platform/1100D.105

Main builds: http://builds.magiclantern.fm/ (some features not working)
Development builds: http://builds.magiclantern.fm/experiments.html (feedback needed)

ML will work on 5D Mark III. Right now it's in very early stage - the early testers reported it as almost unusable.

First try of ML bulb ramping:

Focus ramping:

[Step 1. Get the source code]

Step 1. Get the source code
hg clone -u unified https://bitbucket.org/hudson/magic-lantern


Step 2. Get a compiler


2a -> pre-built toolchain from gcc-arm-embedded (preferred 5_4-2016q3, but any other version should work)

Manual toolchain setup should be straightforward on Linux, Mac and Windows Subsystem for Linux.

General instructions:

Either unzip the downloaded toolchain in your HOME directory (for example, you may get ~/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gcc), or install a package that provides arm-none-eabi-gcc and make sure it's in your executable PATH. That way, the compiler will be picked up by our Makefiles without additional tweaking.

Ubuntu: package gcc-arm-none-eabi (older) or gcc-arm-embedded from ppa:team-gcc-arm-embedded/ppa (latest). Or just unzip the downloaded toolchain into your HOME directory if you prefer to avoid a system-wide installation.

Mac: the easiest way is probably this tutorial from Daniel Fort, but manual setup should work just as well.

Windows 10 WSL is known to require some tweaking (such as installing an X server for QEMU), but generally works well.

Windows (native, without WSL): you may install Cygwin (but make sure there are no spaces in your username). For Windows XP you will have to use an older version of Cygwin, but other than that, it should work without much trouble.




2b -> QEMU installation script will download the recommended toolchain and install it for you.

This method is recommended for Ubuntu, Mac and Win10 WSL; other Linux distros may require additional tweaking.

Besides setting up and compiling QEMU, this script also installs the ARM compiler and debugger, and will provide an environment to run your camera firmware and Magic Lantern in the emulator (very useful for debugging and reverse engineering - check out the guide).

Videos: for Ubuntu and Mac (todo: WSL and native Windows).



To run QEMU install script:

Code Select Expand


hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg up qemu -C
cd contrib/qemu
./install.sh




2c -> Linaro bare-metal toolchain should also work (use ARM_ABI=eabi in Makefile.user).



2d -> pre-built VM if you feel lazy (Kudos to Anton2707) (User/Root password is 123456)

Also, nikfreak has a VM available here (with ARM-console preinstalled).

Note: these VMs are old and do not have the latest bells and whistles (QEMU etc), but these should be easy to install.



2e -> Docker (experimental, feedback welcome if you decide to try it)



2f -> Build the compiler yourself.



Step 3. Start hacking

Code Select Expand


hg up unified -C
cd platform/550D.109
make clean && make zip
make install
make install_qemu


Useful branches

- unified (mainline)
- Experiments section on the download page (WIP stuff)
- Pull requests on Bitbucket (same)
- All branches (lots of unfinished work and experiments)
- crop_rec_4k (4K recording and lossless compression, very bleeding edge)
- dm-spy-experiments (logging tools for reverse engineering)
- iso-research (CMOS/ADTG ISO experiments, adtg_gui, raw_diag)
- qemu (camera firmware emulation)
- new-sound-system (experimental audio controls for DIGIC 5)
- 1200D, 1300D, 100D_merge_fw101, 70D_merge_fw112, 5Ds_experiments etc (ports in progress)
- vxworks (450D, 40D, other oldies are welcome); vxworks-dm-spy (logging)
- digic6-dumper (80D, 750D, 7D2 etc)
- recovery (experiments running from bootloader context, mostly diagnostics or ROM dumping)

Useful stuff

- Writing modules tutorial #1: Hello, World!
- How can I run Magic Lantern in QEMU? (aka QEMU guide)
- EOS firmware in QEMU - development and reverse engineering guide (must read for understanding how the firmware works)
- Notes on old wiki: http://www.magiclantern.wikia.com/wiki/For_Developers
- Using WiFi SD (FlashAir and Transcend) to avoid swapping the card back and forth
- Speed up compilation:
   export MAKEFLAGS="-j8"  # parallel build
   make install ML_MODULES="lua ettr"  # only compile the modules you are interested in
- Examine ASM code: tag a function with DUMP_ASM, then run: make dump_asm
- Stack trace in crash logs
- Submitting a pull request

* Lua scripting ( http://groups.google.com/group/ml-devel/browse_thread/thread/6f708e124d700712 ).
* PicoC scripting ( http://www.magiclantern.fm/forum/index.php?topic=3769.0 )
* Android USB controller ( http://groups.google.com/group/ml-devel/browse_thread/thread/4b0d01d64a459485 )
* Follow focus / remote control with a simple TV remote ( https://groups.google.com/group/ml-devel/browse_thread/thread/6ec6723d7da4119e )
* Custom file prefix (for example, 5D2_1234.CR2, 02JA1234.CR2 etc). Maybe also absolute file numbering from shutter count.  Done: https://bitbucket.org/hudson/magic-lantern/commits/49682329d18213d88f74a9fe516c202091f14d22
* Exposure simulation for extremely dark scenes [DONE]
* Templates for HDR and focus stacking scripts
* Very short uncompressed 422 video clips (1-2 seconds, maybe more on 5D3) Done: http://www.magiclantern.fm/forum/index.php?board=49.0

You can find some more features still to be implemented in this google doc.

[Features copied from another Canon camera]

Features copied from another Canon camera
Copying Canon code or functionality may carry legal risk for us. We do respect the Canon company and love their products and we are strict about staying on the right side of the law.

1080p 60fps, 2K, 4K, RAW video...
The best we could do was 1080p 35fps on 60D and 600D. Update: 4K works, but has major limitations.

Custom codecs
Codecs are not implemented on the general-purpose ARM processor. We can only use what Canon has already included in hardware (H.264, JPEG, LJ92) and fine-tuned their parameters (such as the H.264 bit rate).

The lossless compression used for raw video is the same "codec" Canon uses for CR2. The same processing path (codenamed JPCORE) might be able to handle (M)JPEG. However, we cannot implement additional codecs (such as H.265, JPEG2000 or ProRes). Even if these might be able to run on Canon's image processing hardware, we simply don't know where to start.

Things that can be done in post
Why spending development time on things like in-camera HDR? Magic Lantern is not a replacement for Photoshop

Previewing is OK (e.g. HDR preview, anamorphic preview, fisheye correction preview).

Real-time video processing (e.g. stabilization, sharpness algorithm)
We can't program the image processor. These things can only be done if the functionality is already in Canon firmware (i.e. some parameters that can be tweaked - like in the Image Effects menu).

AF microadjustment
Not possible to control AF outside LiveView. Update: dot_tune works on cameras where AFMA is present in Canon menu.
Not possible on other cameras, with our current knowledge.

Image on both LCD and external monitor at the same time
Not possible (unless proven otherwise by DIGIC investigation).

AF confirmation without chipped adapters
Not possible (camera refused any attempts to fake lens info).

Timecode
Very difficult (see http://www.magiclantern.wikia.com/wiki/Timecode ). The 5D Mark III has it.

Continuous AF in movie mode
Very difficult to do it right (we couldn't).

Scrollwheel controls
It's not possible to remap them while recording. In standby, ML menu uses a trick: it opens some Canon dialog in background to steal wheel events from it, but this trick doesn't work while recording.

1D support
These cameras are way outside our reach. Even if we could buy them, very few 1D users would benefit from ML.
There are also legal concerns regarding Canons Pro line of cameras.



Sure, at some point, some of these might become possible, but chances are extremely small. Spending time on those is effectively searching for the needle in the haystack.



A detailed explanation by dmilligan, on why Magic Lantern cannot increase the FPS of cameras. 

Your question really boils down to this:
"Why can't I capture more information, by throwing away information?"

Now from a more practical standpoint:
Compression (what you refer to as "lowering the bitrate") is a difficult, computationally intensive task (it's also impossible). It is not a magical process where you throw some data in and it comes out smaller. The only way to get enough of an effective compression ratio for the incredibly huge size of a video data stream, is to just throw away some of it. The goal here being to throw out the least important information, but we are throwing away information nonetheless. The better an algorithm is at throwing away data (i.e. the better it is at figuring out what data is unimportant), typically the more complex it is. There are very easy ways to throw away data, such as reducing the resolution and line skipping, and there are very hard ways of throwing away data such as DCT

Lets now consider (a very oversimplified) pipeline that a video stream goes through in the camera:
Sensor -> Raw Data -> Image Processing (demosaic, wb, pic style, curves, etc.) -> H.264 Encoder -> Storage

When you talk of "bitrate" you are only talking about the bitrate at the very last step of this pipeline, the bitrate out of the encoder to the storage media. There are many other steps prior to this to consider. If you want a 1080p stream out of the encoder, you also need that 1080p stream to make it's way through the rest of that pipeline (at 60fps). That's where the limitation is, in fact there are probably many, I'll just go over some of the possible ones:
1. The H.264 encoder, can't handle 1080p of video data coming into it at 60 fps (remember it has to do something very complex and computationally intensive with the data and then spit out that result very quickly)
2. The image processing electronics can't handle 1080p of raw data at 60 fps
3. The internal buses that move the raw data from the sensor to the image processors can't handle that much data (1920*1080*14bit*60fps = 1.7 Gigabits per second)
4. The sensor itself isn't fast enough to sample 1080 lines at 60 fps (it takes some finite amount of time to read out each line, and they are read one by one)

I'm not saying that all of those are true, but at least one or more of them are, and that's why 60p mode is a lower resolution. Overcoming any of these obstacles is possible, but it would require more transistors (i.e. faster, more complicated electronics), which would make the camera more expensive. So without more expensive internal electronics, the only way to get enough "compression" to be able to even get our video data to the encoder, is to "compress" the data starting at the sensor itself, and what's the only way to do that? line skipping and reducing the resolution -> basically don't read in as many pixels.