Linux on your Canon DSLR? Why not?

Started by a1ex, April 01, 2015, 08:00:24 AM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

g3gg0

@rbrune:
no, i tried to compile them manually. will give it a try, thanks!
had already to patch it to make it accept FLAT options. nto a good start though :)

@sooda:
we need a framebuffer driver that can do 4bpp :)
i started to build a skeleton, but it didnt really work.
maybe i just didnt understand how things work at all.
(all this was a result of the last few days t.b.h, targetet for april 1st fool)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

a1ex

@nikfreak: were you able to compile the Linux demo from source?

If yes, can you run a "hg bisect" to see where it stopped working? The Linux demo was started from the display test demo, which worked fine on 6D and 70D.

sooda

@g3gg0: could you spare some details? I can see some fb stuff in the patch, does it do anything yet? I grepped for the memory address (0xC0F140D0) and looks like disp_direct.c puts the actual memory buffer address there, and the kernel code just passes the address to the fb driver and I'd guess it would use that directly for writing the pixel data, so a pointer indirection seems to be missing.

I'll try to apply the patch on a kernel source and build things from scratch. Buildroot is a good idea too, I've used that before successfully for a whole tiny "distro".

nikfreak

@a1ex: i didn't try to compile myself but will try to find out when it stopped working by compiling myself
[size=8pt]70D.112 & 100D.101[/size]

g3gg0

@sooda:
the last time i tried, display stopped updating.

the bootloader sets some frame buffer address by writing it into 0xC0F140D0.
the kernel reads from that register back, but we could also pass it via commandline.

cool, good to see helpful hands :)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

sooda

Ah, I misread the assignment. Now it makes sense. I'll try to dig in but first these toolchains need to be set up...

thearczoro

Camera: Canon 500D
FW: 1.1.1
ML: Nightly (4 October 2017)

sooda

ok, i can has tools set up. verified that everything gets built by modifying the kernel's console writer into a rainbow, and added my name in the userspace tool that prints hello:



my phone's camera doesn't like blue though.

N/A

Since we can access the bootloader now and control what boots up on the camera, can we control WHERE it boots from? Say if I have a CF with 2 partitions and would like to be able to select which partition I boot from, is this possible now? Also, will we be able to boot from and use ex-fat formatted cards in cameras that don't originally support them (7d, 50d)?
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

Fernando

That is awesome news! I have been wanting to use Linux on my camera for at least 3 years... I believe once a proper setup is made and people are able to develop and compile tools for this, we are going to enter a real golden age of DSLR's...

g3gg0

@sooda:

very nice, so you can confirm that all patches/manuals to build linux and userspace apps are correct?

now there are some things to resolve:
- build libc-based binaries using the elf2flt linker
- build busybox
- improve kernel drivers
- get rid of bootloder code after kernel booted
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

kaanklky

Hey!
I want to help development of linux for Canon DSLRs but I couldn't find the source at bitbucket.


sooda

@g3gg0: yes, that's right. I just used a slightly different place to store the binaries and elf2flt to test things around. And the little readme was missing the instructions for the initrd image, but the usual loop device trick worked. I also didn't need to compile binutils separately since Gentoo's crossdev did that.

I was wondering what's the strange jump-straight-to-loadaddr change in load_flat_binary() (I believe it won't help/work in the long run), and took it away to see what happens. With some debug prints in schedule(), it seems that the process starts up (or is at least put somewhere to be scheduled properly), but its instruction pointer never increments from the start address between the calls to schedule(). Not sure if there's something wrong in some low-level assembly stuff that boots processes up?

Also, I'm slightly confused about some memory addresses (and the kernel is too). Backtraces (from WARN_ON debugging, and the panic seen by all) show correct numerical addresses that can be tracked back to file/line/function name as long as CONFIG_DEBUG_INFO is set to y in .config. However, the function names in the traces are completely bogus. I believe this is because the kernel is loaded as executable-in-place (why's that? it's meant for loading from rom, afaik) into a position (0x00800000) that is smaller than the ram start (0x01000000). How is that possible or is that just a different ram mapping? And why is it done this way in the bootloader? Anyway, to help debugging, I deleted the check "if (s->addr < kernel_start_addr) return 0;" in scripts/kallsyms.c:symbol_valid() as a workaround and got the backtraces working. That sanity check is to disable loading some unnecessary symbol names to the kernel.

About buildroot, I couldn't find support for this particular cpu in its menus and also it wants to use elf binaries even if mmu support is turned off. We could try if yocto is any better out-of-the-box.

g3gg0

yeah, how to create an initrd was not explained because i thought: the ones who get that far, will know it :D

yeah that strange jump. i dont understand where and how the context is changing.
the registers in the struct get set up, but where now exactly those take effect, i dont know.
guess this is somewhere in scheduler, but didnt find it yet.

that the kernel is XIP was just a simplicity issue.
else the kernel had to load itself to its target position and that didnt work well.
when using zImage it showed a really weird behavior - i couldnt emulate it in my emu and on camera it simply hung.
its also not clear if zImage works for NOMMU kernels. at least i couldnt confirm that it does.
might be a different bahavior now, as maaany things in memory map changed since i tried that.

memory setup:
0x00000000 - 0x00000040    reset vectors (first canon, updated by bootloader, then linux')
0x00000100 - 0x00008000    ATAGS (parameters for kernel)
0x00008000 - 0x00008008    bootloader interface (function pointers for emulating text console, as linux has no fb driver yet)
    0x00008000 void (*print_line)(uint32_t color, uint32_t scale, char *txt, uint32_t count);
    0x00008004 void (*print_char)(void); /* pritns the character stored in the word at 0x00008008 */
    0x00008008 uint32_t char_to_print
0x00008000 - 0x00698658    relocated bootloader (first 12 bytes get overwritten with function pointers)
0x0072D100 - 0x00800000    YUV/bitmap buffers
    0x0072D100 disp_yuvbuf
    0x007D5D00 disp_bmpbuf
0x00800000 - 0x01000000    linux kernel
0x01000000 - 0x10000000    via ATAGs reported RAM

at 0x01000000 - (sizeof(initrd)) the bootloader places the initrd image and reports that via ATAGs
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

pc_bel

Being readed all the post, I have no doubt about the importance of this (Linux on Canon hardware), but at the same time, with cero skills in coding and all the rest involved in the process, I have some questions (I'm sure I'm not the only one):
In practical terms, what does it mean?, Which kind of software can we have in the future? Wich kind of funcionalities can we have in our cameras?

Sorry for my ignorance... :-[

and congratulations to the developers, of course!!!! :)

sooda

Thanks for the memory map, that confirms what I've read so far. (edit: oh, and any pointers for qemu? fiddling with the memory card is slowish)

The timer/irq stuff also confuses me. What do you mean by "sometimes it doesnt get the timer interrupt, or looses it and it wont fire again"? I'd like to get a stable led blinker and a proper clock at least for the kernel log... About the irq lookup failure comment in the code, I'm not sure if you first need to register irq numbers to specific handlers and stuff in order to work with that. I've only worked with slightly higher-level irqs before, so not sure about that before actually looking around the code.

About the general questions on what is possible: At first, this is obviously just playing around for fun to see how far we can go. There is this joke that you can easily ping localhost (i.e. have working network stack just inside the box) but not actually do anything meaningful (such as take pics, or talk with the outer world) on stuff that you install Linux on. I believe it would be easier to write modules as just userspace processes that would use drivers written into the kernel, with proper multiprocessing, maybe even a windowing system. How easy would that be for the end-user without a mouse is another question. With the original firmware missing, we obviously have full control of the hardware, but we'd still need to know how exactly to control it. With "normal" ML, we can jump into the original firmware to take photos and all that, but the firmware probably assumes that its own operating system is running, so any of this cannot be done from Linux.

I also wonder how not having an mmu (memory management unit, providing virtual addresses) affects the userspace. At least any kind of memory protection is not there, as all pointers represent physical memory, and all processes can poke around other processes' memory even if they just get pointers wrong. Less controlled application crashing, more memory corruption. And forking (duplicating processes) is done in some strange way. Linux was not really designed for mmu-less processors, but nowadays it still works kind of well, with certain restrictions.

One of the best things is that compiling stuff for Linux is kind of easy whatever the hardware is, and all the interfaces are well documented. But as the hw is different, you can't just take gimp built for a desktop pc and copy it to the memory card and expect it to work at all. (ML's module system is good already, though.)

Python or some other scripting language would also be interesting to get working asap as a proof-of-concept, as it wouldn't need cross-compiling (binary libs for it are a different thing). Of course, it's possible to run that also without Linux directly, but anyway.

g3gg0

i didnt try it in qemu. couldnt get it compile on windows and had to run the emulator in a virtual machine.
so ive re-used my own ARM emulator and tried to getting it running there.

btw, i updated the memory map, as the autoexec.bin gets relocated to 0x8000 during startup.
wanted to have it in cacheable RAM and in a place where the linux kernel has no access.

about the other points, you are absolutely right.
linux was not designed for MMU-less systems. its more or less dirty, as linux doesnt even use
the MPU to protect binaries against each other to catch the situation you described.

yeah, its a proof of concept.
but i really think about post processing stuff happening on the camera.
just to test how e.g. dual iso pictures come out etc.

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

luke1100

I am speechless. You guys are gods among men.  Thank you, I'll be going to paypal to donate soon.  I hope other people do the same, it's the only way developers can measure support.

My question that I didn't see posted anywhere.

Is this a piggy-back like ML is now, or is it loading without original Canon firmware? 

Also, will we see custom scripts that could gain more accesses, harnessing more power from the hardware?

Thanks again.

cryptworks

Something to consider is if you get this over to the crazy android developers that hang out at XDA they may help build an android build for your canon. that would be something, as there is a great deal of fascinating things you can do with the camera from there.


nikfreak

Quote from: a1ex
@nikfreak: were you able to compile the Linux demo from source?

If yes, can you run a "hg bisect" to see where it stopped working? The Linux demo was started from the display test demo, which worked fine on 6D and 70D.

Just compiled the recovery branch which created a 90kb autoexec for me. That behaves the same on 70D as the one from display test but showing me additionally "boot file i/o stubs incorrect". Retried again the big autoexec from 1st post but i can't boot it. lcd stays black. Just a 1 sec led light up after closing sdcard door.
[size=8pt]70D.112 & 100D.101[/size]

g3gg0

@buildroot:
http://buildroot-busybox.2317881.n4.nabble.com/Best-way-of-building-for-unknown-architecture-td30768.html

getting exact that error after patching some stuff that enables NOMMU / FLAT support.
trying to figurie out where it comes from... :(

edit:
wow, seems i got a bit further in building the toolchains.
just a few hours, handfull of patches and modified default configs later, it builds all the standard tools for the initrd...
..still building, so no idea if the output is usable at all :)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

g3gg0

oh and if someone wants to support, still missing:
- bzImage
- compressed initrd

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

ItsMeLenny

I was going to suggest netBSD maybe being more appropriate, but it doesn't run MMUless either.
And for the record, plan9 doesn't seem to either :P