Topics

https://bitbucket.org/hudson/magic-lantern/commits/e01856b86df6

Confirming the other comments on the commit from 'Gregory O'Connor', can't compile ML on OS X anymore due to missing `truncate` command.

Anybody figure out a work-around? I went macports last time I re-installed OS X (not brew) and it's kind of a big project to install brew (more work than practical just to install one thing).

At first glance it appears it will be possible. So, it appears even vxworks will do it

Today I shipped my M to 1%, he will be taking over from here. I decided I just don't have the time right now to figure out some of the problems.

This looks promising:

- new pocket camera that shoots 1080p raw uncompressed 12-bit and costs <$1000
- new production camera updated with a new sensor and 4K recording ($4000)

http://gizmodo.com/5994043/blackmagic-pocket-cinema-camera-raw-video-in-the-palm-of-your-hand-for-1000

This changes the game for Canon and Nikon, they're playing catch-up now

Great youtube video that explains some basic ideas that can be applied for all videographers:

I think I'm probably going to need to find a new toolchain, but maybe someone has some input.

After commit ab5a4c0 (https://bitbucket.org/hudson/magic-lantern/commits/ab5a4c0b8f1d047a1c36681a559304ad1ca8f955) ML now uses uint64_t, which Yagarto has some issue with. Now compiling produces these errors:

/Users/coutts/yagarto/yagarto-4.7.1/bin/../lib/gcc/arm-none-eabi/4.7.1/libgcc.a(unwind-arm.o): In function `get_eit_entry':
/Users/mfischer/Projects/yagarto/gcc-build/arm-none-eabi/libgcc/../../../gcc-4.7.1/libgcc/unwind-arm-common.inc:221: undefined reference to `__exidx_start'
/Users/mfischer/Projects/yagarto/gcc-build/arm-none-eabi/libgcc/../../../gcc-4.7.1/libgcc/unwind-arm-common.inc:221: undefined reference to `__exidx_end'

Anyone have some input? I'd like to continue using this toolchain because it's easy / fast to setup, but if required I will begin using a new one if someone has a better option.

I've often wondered what (who) makes state machines change their state. Last year while porting the 5dc I mapped some complex structures in VxWorks that all tied into state machines. I've finally done the same for DryOS while investigating EVF (live view) state.

Here's a comment from my commit here which explains how state machines change states:


/**
*  EVF Manager - 6D v1.1.2
*
*  Managers are a high-level data structure in DryOS and VxWorks. For structure definitions, look in state-object.h
*
*  Before, state-objects were the highest level data structure we understood in DryOS, now I understand what actually
*  triggers the state changes, so we can track down the functions that do these changes. For this demonstration I'll
*  examine the EVF Manager in the 6D v1.1.2 firmware.
*
*  The Taskclass structure is pretty important, as this is where events are queued up before processed. Events are
*  posted to the taskclass queue by taskclass_post_message (0x39F04). Arg2 to this function is the event number to
*  post. A taskclass has a generic task, which takes events posted to the taskclass message queue, and calls the
*  respective EventDispatch function, in our case it's evfEventDispatch (0xFF0DD22C). This function is called by the
*  taskclass task via a pointer stored in the Manager struct. It's because of this that we can hijack the EventDispatch
*  handler just like state objects, to catch taskclass events as they're processed.
*
*  All we need to look for to find who posts the events are calls to taskclass_post_message, specifically the ones that
*  reference the EVF Manager's struct. I hijacked the EVF Manager's event dispatch here, and got the same results as when
*  I hijacked the state machine, so it works! For EVF state, i observed 3 events happening in each frame: 5, 3, and 4.
*  Look at my youtube video here of live view slowed down to 2fps, the Tick message happens once every second (on another task).
*      --> www.youtube.com/watch?v=B4n1eh8YUtE
*
*  The debug log after running this looked like:
*
*  [MAGIC] name|arg1|arg2|arg3: Evf | 0x5 | 0x0 | 0x0
*  [MAGIC] name|arg1|arg2|arg3: Evf | 0x3 | 0x0 | 0x0
*  [MAGIC] name|arg1|arg2|arg3: Evf | 0x4 | 0x0 | 0x0
*      (repeated)
*
**/



//~ Structures for DryOS, derived from research on VxWorks.
struct Manager
{
    const char *                    name;                   //~ off_0x00    name of manager. ie: Evf
    int                             off_0x04;               //~ off_0x04    unknown
    struct TaskClass *              taskclass_ptr;          //~ off_0x08    pointer to taskclass struct
    const struct state_object *     stateobj_ptr;           //~ off_0x0C    pointer to stateobject struct
};

struct TaskClass    //~ size=0x18
{
    const char *                identifier;             //~ off_0x00    "TaskClass"
    const char *                name;                   //~ off_0x04    task class name. ie: PropMgr
    int                         off_0x08;               //~ unknown     initialized to 1 in CreateTaskClass
    const struct task *         task_struct_ptr;        //~ off_0x0c    ret_CreateTask (ptr to task struct) called from CreateTaskClass
    const struct msg_queue *    msg_queue_ptr_maybe;    //~ off_0x10    some kind of message queue pointer (very low level functions at work)
    void *                      eventdispatch_func_ptr; //~ off_0x14    event dispatch pointer. ie: propmgrEventDispatch
};




The taskclass reminds me a lot of generics from object-oriented languages like Java. There's one "generic" task for every taskclass, they just rely on the Manage struct passed to them to keep track of the name and state machine associated with the events.


Summary:
Functions post events to the taskclass queue using taskclass_post_message. Each taskclass has a task running at all times, constantly checking a message queue for new "events". If there's an event to process, the taskclass calls the EventDispatch function associated with the manage struct (passed as arg0 to taskclass_post_message), passing the event number as one of the arguments. Through hijacking this EventDispatch function, I've figured out that this is what triggers the state machine to change states. There's a function called by the EventDispatch; a generic "change state" function, which uses the event number to find the offset in the state matrix, to know which state change to call. The state machine changes states, and remains in that state until a new event tells it to change to a different state.

Now, what calls the functions that post to the taskclass queue? From what I've seen, they appear to be callback routine (CBR) functions as far as EVF goes. I'll keep digging, but we're getting close to understanding what copies the LV frame data from the sensor to the LV and HD buffers.

This may be more-so geared to g3gg0, but some of you guys have been using IDA longer than us, and might have a good color scheme better than the eye-killing white background stock profile.

Anybody have a color scheme they'd want to share? You can import/export them from Options -> Colors

Recently I've tried to use make_bootable.sh but I seem to be having some weird errors now, anybody have the old version? I know it was modified to add ExFat support but it must have broken something.

Pretty neat to hear his experience. He complained about the audio, so maybe he didn't know about magic lantern 

http://www.usatoday.com/videos/tech/2012/12/18/1778293/

For some who don't know the name, he's made some pretty famous movies:
- Saving Private Ryan
- Man on a Ledge
- Newlyweds

[blindly maintained]

Current state: blindly maintained (the developers who maintain it don't have a 6D, and those who do are refusing to maintain it or are too busy with real life). Anybody with coding skills and lots of free time (and of course a 6d) is welcome to maintain this port.

Nightly builds: http://builds.magiclantern.fm/

Installation:
1) Format the card from the camera.
2) Make sure you are running Canon firmware 1.1.3.
3) Copy ML files on the card and run Firmware Update.

Uninstallation:
1) Run Firmware Update from your ML card.
2) Follow the instructions.



-- a1ex




FOR DEVELOPERS ONLY
Want to help with the porting proces? Use this special updater file to dump the firmware, set your bootflag, and dump the debug log. This is NOT an alpha release of Magic Lantern, it only sets the bootflag (that's it). I removed the check for autoexec.bin and made it so it doesn't make your card bootable. This is only useful for dumping the firmware and setting your bootflag to get started developing.


15-Dec-2012:
- firmware dumped, working on mapping functions
- bootflag enabled, able to boot autoexec.bin files now
- Canon firmware booted by ML - so first step to "hello world"

16-Dec-2012:
- Hello World!

18-Dec-2012:
- released custom FIR updater FOR DEVELOPERS ONLY that sets the bootflag, dumps the rom, and dumps the debug log.


January 2013 - February 2014:
- Tragic Lantern took over the 6D development and refused to contribute anything back to the main tree, leaving the 6D port unmaintained.

February 15, 2014:
- Marsu42 confirmed the unmaintained port is still working, and submitted a pull request to get it back into the nightly builds. We, the Magic Lantern developers, decided to do a blind backporting of Tragic Lantern code (without a 6D in our hands) and brought the port up to date.

March 20, 2014:
- Simplified installation process.
- Tragic Lantern usage and discussion is no longer accepted on this forum.

=> current state: blindly maintained, we need your help to get it back on track.

[Update 11/25/2012:]

The EOS M port starts now! g3gg0 successfully dumped the firmware and set the bootflag, so we're all set to begin porting


--> More updates coming soon

Update 11/25/2012:







Update 12/10/2012:
Alpha1 release! Let me know what bugs you notice. Almost everything that is going to work on the M is enabled now, minus audio stuff (need more work on that). Some features won't ever work though. ie: Trap Focus - because the M doesn't send focus confirmation when in manual focus.

Update 12/10/2012 #2:
- Fixed installer (thanks Alex), no more unsafe operations (ie: try to set bootflag while in live view).



Install Procedure
Refer to Unified install page:
http://magiclantern.wikia.com/wiki/Unified/Install


Warning: I'm not responsible for anything that might happen. This is an EXPERIMENTAL build. Please report bugs in this thread.
Download Alpha 1: Here

[View all pictures from my disassembly here:]

Well sadly this morning while trying some cache_hack code, my 5dc appeared to be bricked. Basically it acted like there wasn't a battery in it despite the fact the battery was charged. So, I began disassembling to see if anythings noticeably wrong. Here's some of my findings.

View all pictures from my disassembly here:


(Full-resolution 5mp photos: Here )
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
--> Note, this is a very good PDF to have if you are going to be disassembling a 5d classic:
          https://dl.dropbox.com/u/33161628/ML/Datasheets/EOS_5D_Parts_Catalog.pdf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

TX19A (MPU) Chip



Toshiba TMP1942CZU
CH4-6311-000
0707AA JAPAN

Datasheets:
BCE0023_catalog.pdf
TMP1942CZU.pdf




K213 Microcontroller



Toshiba T6H82CG-104
JAPAN 0548HAL
021725

Datasheets:
** Refer to BCE0023_catalog.pdf linked above **




Digic / SDRAM



Canon DIGIC 2
4x 512Mib SDRAM (256mb total)
021725

Datasheets:
IC-ON-LINE.CN_edd5116adta-5cli-e_4529134.pdf


[ ..more coming soon ]

http://www.canonwatch.com/canon-eos-m-shipping-in-the-usa-delayed-to-end-of-october/

So looks like I'll be waiting until then to get my hands on this camera..

Any developers have this camera? With the 5dc and 40d being worked on, it should be easy to port the 1000d to ML as well now. The 1000d has live view, so it should be possible to capture video with it. 1000d shares the same low quality screen as the 5dc so I bet all of the vram stuff is the same.

Initial porting is already done by me. Look at this project here to get started, it boots the main firmware and sets the bootflag. Look at the 5dc entry.S / init.c to get an idea of where to start for the 1000d and ML.

https://bitbucket.org/coutts/1000d_dev

The entry.S of the 5dc can be seen here. As you can see it's very similar to the 1000d one I already wrote above:
https://bitbucket.org/hudson/magic-lantern/src/bbaee3c6d8d0/platform/5DC.111/entry.S

Only thing that will need fixed up is the call to where it sets up one of the main heaps. We need to shrink the heap and make room for ML to be copied there, and then link to this spot. It is not safe to link to 0x800000 from what everybody says (though I don't yet understand entirely why).

I have not worked on it in a while (no longer have access to a 1000d), but it is all set for someone to take over. Maybe someone has a 1000d to send to me?

[flush_caches__start_firmware]

I was tracing back the 5dc bootloader when I thought I should start making some notes of how it boots things.

The code that actually starts the firmware can be found at FFFF36EC

The flush_caches__start_firmware function is at  FFFFE358

So, working backwards.. jump_to_firmware is called in 3 different spots. Arg0 is the name of the file that the bootloader has loaded/ is going to jump into, so it could either be the FIR filename or it will be AUTOEXEC.BIN. Arg1 is the actual address it is going to jump to. This address is determined by load_file_to_execute (FFFF4388), which is the function that I believe copies the file into RAM at 0x800000.

The 3 spots that jump_to_firmware are called give somewhat of a clue as to what it is booting:
- one passes arg0 as AUTOEXEC.BIN
- one passes arg0 as an inputted filename (probably used for debug purposes by canon)
- one passes arg0 as the return of sub_FFFF46B8. Initial look at this indicates maybe this is an auto-calculated FIR filename, used with the flasher during a normal firmware update procedure.



So, it appears it goes like this:
- load_file_to_execute is called with arg0 as the filename to load into memory. arg1 to this function is some variable (stack variable) to hold a return value. This return value is the link address of the file being loaded into memory. It has 2 options: it will either be 0x800000 or 0x10800000. Not sure yet what this caching bit of 0x10000000 means.
- jump_to_firmware is called with arg0 as the filename being executed and arg1 as the load address of that file.

Has anyone figured out what this does / is used for yet?

Looking at some PropMgr states, I found some valuable info, names of different ranged properties. From VxWorks in 5dc v1.1.1:

0x1000000 range = TuneData
0x2000000 range = RingData
0x3000000 range = CustomData
0x4000000 range = PcData
0x5000000 range = WftData
0x80000000 range = FixData

important functions:
check_property_list__maybe: 0xFFAB9FA8 --> seems to check if a property passed as arg0 is valid (in one of the "prop lists").
PROPPAD_GetPropertyData: 0xFFAB6D5C --> arg0 is a property, it checks if it fits in one of the lists too maybe? this is where I found the names above.

[[see post below for updated one]]

I spent about 6 hours today mapping these structures in VxWorks from the 5dc v1.1.1 firmware. They should be similar if not identical in DryOS as well. These all have to do with the event system, which is ultimately what I'm trying to understand / hijack. My end goal is the ability to inject whatever event(s) needed to trigger bulb exposures on the 5dc (since it lacks PROP_REMOTE_SW2).

I'm not really too sure how well I did with the data types, but I tried to put as many notes in as I could. I'll work on this more tomorrow and slowly refine what does what. Now I see things like TaskClass and StageClass in the firmware and I'm not so lost/scared looking at it

Hijacking a StageClass or TaskClass will be almost identical to how we hijack state objects, so won't be hard to do at all.

[see post below for updated one]

Feel free to contribute / discuss.

I just realized I've never made an intro on here, so here it is! My name is Andrew, I'm a 21 year old computer science major. I first discovered ML just over 1 year ago, and began speaking to A1ex. He has since taught me everything I know about how ML works / how to develop / port it. Before that I had no knowledge / understanding of embedded systems at all. I hadn't even heard of assembly language or any of that. Needless to say, I owe him (and the other developers like Indy) a lot. I began working on the 500d port last year, and eventually brought it into the unified tree (it was a dead port with no active developer at the time, so I had to save it!).

Now I have sold my 500d and recently (a few months ago) bought a 5dc. I've been working to port ML to it and making good progress (check the New Ports section for the log).

I'm really happy that there is finally a central forum for all ML related things, so now the community can begin to grow. Looking forward to seeing how ML progresses over the coming months, have a nice day!


- P.S. am I still the only american involved here besides Trammel?

I've hit a bit of a wall again on the 5dc, it appears there is no gui event for halfshutter press, there's only an unpress event. so, anybody have a creative idea for detecting shutter half press?

Intervalometer is working! Very stable too so far (I didn't have any issues with it last night). Unfortunately the clouds rolled in after an hour, so I only have an hours worth of footage to produce this 6 second long video.

Maybe 5DC will be included in the next major release if I can finish it in time


Canon 5D Classic
Sigma 24mm f/1.8
Intervalometer set to every 30 seconds.
25s exp @ f/4.0
ISO1000
Processed with VirtualDub.

(p.s. not sure what's with that thumbnail on the video)


5DC port thread:
http://www.magiclantern.fm/forum/index.php?topic=1010

Sometimes the stack still confuses me, so I need to look back at how it works. In all canon cameras, the stack is a Full Descending one. It grows to lower addresses in memory. I found a short word document on google which explains this pretty well. To view the document itself, just look here: www.cems.uwe.ac.uk/~cduffy/es/ARMstacks.doc

[Open Beta]

5dc Port Progress
Current state: Open Beta



I started this project months ago, progress has been slow. The reason is that the 5dc is a bit unique. It runs VxWorks, but has a lot of similarities to modern DryOS cameras. As a result, simply porting Magic Lantern or 400plus was not an option, I had to start fresh.
EDIT The 5dc will work on the unified tree! Big thanks to A1ex for the help in making this happen.

Short-term Todo List:
Blink LEDs from user FIR file boot.
Enable bootflag for autoexec.bin booting.
Hijack Canon's GUI task.
Port BMP routines from ML for screen printing.
Port menu system from ML.
Fix fonts to be sized correctly on screen.
Fix up properties and property handlers from ML.
Begin porting all non-lv features from ML (intervalometer, audio triggering, trap focus, etc).
. . .
? ? ?

Install Instructions (READ FIRST!!!)
*** Remember, I am not responsible for any damage to your camera. This is BETA software, and I can't predict what could happen. If you are not comfortable with this, then please do not download this firmware. ***
1.) Make sure you are running v1.1.1 firmware (if not, upgrade to it!)
2.) Double check that you are running v1.1.1 firmware.
3.) One more time make sure that you are running v1.1.1 firmware!
4.) Charge battery to 100% and remove any battery grip for the install procedure.
5.) Download the zip file from the link below, unzip all of the files to your CF card.
7.) Launch the firmware update process. If you see any red led, then something went wrong. A solid blue led means that the bootdisk is now enabled. A blinking blue led means the bootdisk is now disabled (if you want to completely uninstall ML). Each time you run the firmware update it just toggles the bootdisk to the opposite of whatever it is currently. The installer FIR will make your CF card bootable for you, no further work required here.
8.) Turn the power switch off and pull your battery to turn the camera off. Just turn the camera on (make sure autoexec.bin and ML directory are on the card!) and you're all set. Press the Trash button to bring up the ML menu.

You only need to run the firmware update to enable the bootflag ONCE. Each time after that will only disable, re-enable, disable, re-enable, etc. the bootflag.

If you have any questions, please read the ML documentation to see if it can be answered before posting here:
http://www.magiclantern.fm/documentation


Download (click here!)

[MOD: Link no longer works. Use this instead.]


Changelog / Release Notes

Beta 4 - 10/13/2012 (see also the announcement)

* All stability tests passed

* New stuff:
   - DOF bracketing ( http://www.magiclantern.fm/forum/index.php?topic=3113 )
   - Flash bracketing (not tested, I don't have a flash)
   - press DirectPrint to toggle MLU
   - RGB zebras and histogram
   - Display saturation (including grayscale preview - if you use UniWB, for example)
   - Exposure adjustment in playback mode (rough approximation)

* Complete feature list:
   - zebras, focus peaking, spotmeter, histogram, waveform, vectorscope (in playback mode)
   - exposure bracketing: shutter, ISO, flash; 2x0.5 ... 9x5EV; auto mode (covers the entire dynamic range).
   - DOF bracketing (changes aperture and shutter while keeping exposure constant)
   - Intervalometer
   - Mirror lockup (shortcut key, or link to self-timer)
   - Flash exposure compensation (-10...+3EV) - not tested
   - Flash / no flash (like in fuji 6500) - not tested
   - Display saturation (grayscale/normal/oversaturated)
   - Fast zoom (one click goes to 100%, like on 5D3/1Dx)
   - Quick erase (like on 5D3)
   - "Ken Rockwell" zoom (press Zoom In after you took a picture, without pressing PLAY)
   - Exposure adjustment in playback mode (very rough)
   - Timelapse playback - http://vimeo.com/48482815
   - In-camera help (press INFO)
   - Benchmark (check read/write speed of the card)
   - Display operating system internals (tasks, free memory)

Basically, this is what we can achieve on 5Dc with our current knowledge. For more, one needs to dig deeper in the firmware and reverse engineer it. The camera is about 7 years old if I'm not mistaken - but simply the fact that it runs a large subset of the same ML code designed for DIGIC 4 cameras, is amazing!

Beta 3 - 10/2/2012
Big thanks to Alex for this release! He found the missing VRAM buffers which enabled a lot of features in playback mode.

Working:
- auto bracketing, intervalometer
- zebra, focus peaking (huge help for this tiny screen!), spotmeter, histogram, waveform
- fast zoom in playback mode, quick erase (well... not quite perfect, but almost there)
- card and focus peaking benchmarks (5Dc's ARM processor is twice as slow as 5D3's)
- custom color palette for ML

Not working:
- RGB tools, bulb, trap focus, AF patterns (very hard or impossible)
- AF options, MLU (easy to fix, just takes time)


Beta 2 - 9/25/2012
- Changed boot code from how it's done in the ROM to how it's done in the firmware update flasher code. My camera boots, hope it works for all other 5dcs too.
- Updated to latest source tree, nothing changed for 5dc though.
- Now included is a FIR file that will only make your card bootable - just run the firmware update and pull your battery when you see the blue LED, your card is now bootable after that.


Beta 1 (initial release)
- Potentially very buggy. Try it out, let me know how it works. Many features are disabled because they don't work yet / will never work (obviously all LiveView features).
What Works:
- Intervalometer
- HDR Bracketing
- Flash Tweaks