Canon 80D

[OlRivrRat]

I do not "qemu-eos" so guessing I never got all the way through Via Compiler before getting Stymied

[OlRivrRat]

So now I'm here >

WDeans-Mac-Pro:qemu ORRz$ cd /Users/ORRz/qemu
WDeans-Mac-Pro:qemu ORRz$ ./run_canon_fw.sh 100D,firmware=boot=1
diskimages-helper 10084 ORRz    5u   REG   1,17 259522560 2604389 sd.img

Error: please unmount the SD image.
WDeans-Mac-Pro:qemu ORRz$

[OlRivrRat]

& now with SD Gone >

WDeans-Mac-Pro:qemu ORRz$  ./run_canon_fw.sh 100D,firmware=boot=1
Please call configure before running make!
make: *** [config-host.mak] Error 1
WDeans-Mac-Pro:qemu ORRz$

[OlRivrRat]

Another attempt @ Installing QEMU Via Compiler gets Me to >


Would you like to install QEMU?
[y/n]
y

installing QEMU
0 files updated, 0 files merged, 0 files removed, 0 files unresolved

This will setup QEMU for emulating Magic Lantern.
Thou shalt not be afraid of compiling stuff on Linux
Continue? [y/n] y

*** Installing dependencies for Mac...

./install.sh: line 106: lsb_release: command not found
*** WARNING: 64-bit GDB is known not to work.

*** WARNING: a valid arm-none-eabi-gdb could not be found.
*** Downloading a toolchain and installing it without the package manager.
*** Will be installed in your home directory (Makefile.user.default expects it there).


*** Will download gcc-arm-none-eabi-5_4-2016q3 from:
    https://developer.arm.com/open-source/gnu-toolchain/gnu-rm

*** Toolchain already installed in:
    ~/gcc-arm-none-eabi-5_4-2016q3

*** Please add gcc binaries to your executable PATH.
*** Run this command, or paste it into your .profile and reopen the terminal:
    export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH

*** WARNING: 64-bit GDB is known not to work.
Continue anyway? [y/N]

         Thinkin I'm just Wastin Your Time ~ Might be good name for a song ~

[kotik]

@OlRivrRat
I'm wondering which MacOSX version you're using.
In message #351 your screen dump still has the aqua graphical user interface which Apple flattened in 2014 (MacOS Yosemite)!

[a1ex]

@a1ex
Trying some older stuff:

Code Select Expand

(
>   sleep 3; echo "akashimorino";
>   sleep 1; echo "dumpf";
> ) | (
>   ./run_canon_fw.sh 80D -serial stdio -s -S &
>    arm-none-eabi-gdb -x 80D/patches.gdb
> )
-bash: arm-none-eabi-gdb: command not found
iMac-27:qemu-eos $ qemu-system-arm 25643   12u   REG   1,10 259522560 8133999 sd.img

Error: please unmount the SD image.


There are two error messages. The first one requires this step, which you already did during installation, but its effect vanished after opening a new terminal:

Code Select Expand


    export PATH=~/gcc-arm-none-eabi-whatever/bin:$PATH


though, for Mac, it's best to compile the latest gdb from source (todo: update the install script to do that).

The second one tells you the SD image is in use by another QEMU process.

[Danne]

export path issues probably due to installation part in Compiler.app:

Code Select Expand

    cd ~
    echo 'export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH' >> .profile
    source .profile
    cd magic-lantern
    hg update qemu -C
    cd contrib/qemu/
    ./install.sh

dfort made a .profile which sends export path there. Not sure if that is being used in terminal. Will check. It also appends the path so every time installing qemu it gets added to a new line. Will look over that one too.

brew install gdb is at version 8.1 atm.
/usr/local/cellar/gdb might not be where we want it though?

Native gdb mac OS sierra is already on v8:

Code Select Expand

Last login: Thu May 31 10:14:30 on ttys001
dans-MacBook-Pro:~ dan$ arm-none-eabi-gdb
GNU gdb (GNU Tools for Arm Embedded Processors 7-2017-q4-major) 8.0.50.20171128-git

Ok, the issue is that .bash.profile will run first and leave .profile alone after this. By adding the export path to .bash_profile instead the export path was noe working. Will change the install routine in Compiler.app.

Of course this has to change again if compiling version 8 

Fixes added:

Code Select Expand

    cd ~
    if ! grep 'export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH' <<< $(cat .bash_profile)
    then
    echo 'export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH' >> .bash_profile
    fi
    source .bash_profile
    cd magic-lantern
    hg update qemu -C
    cd contrib/qemu/
    ./install.sh

[kotik]

Still learning, trying examples etc.
After this command line and (partially?) output the Terminal locks up.

Code Select Expand

./run_canon_fw.sh 80D,firmware="boot=0" -d debugmsg 2>&1 | grep -E --text Notify.*Cur
[        init:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2018000, Flag = 0x10000)
[      SFRead:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2008000, Flag = 0x8000)
[     RomRead:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 0, 0x2000000, Flag = 0x2000000)
[    PowerMgr:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 1, 0x2, Flag = 0x2)
[     Startup:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x20420010, Flag = 0x20000000)
[     FileMgr:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x420010, Flag = 0x10)
[     FileMgr:fe0dc20d ] (00:03) [SEQ] NotifyComplete (Cur = 2, 0x420000, Flag = 0x400000)

Learned from Danne that '|&' should be altered into the older '2>&1 |'. Otherwise you get an error.

Code Select Expand

-bash: syntax error near unexpected token `&'

[Danne]

Bash 3 is default, |& is from bash 4.


Nice(100D):


Not knowing too much here but is it valid to run gdb on native mac version? I mean why install it if already version 8?

Code Select Expand

gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.38)
Target: x86_64-apple-darwin16.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Edit: added check for native gcc 8:

Code Select Expand

    cd ~
#check for native gcc version(if less than version 8 use ongoing version)
   gcc=$(echo $(gcc --version | tail -n +1 | head -1 | cut -d ' ' -f4 | cut -d '.' -f1))
   if (( $gcc > 8))
    then
     if ! grep 'export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH' <<< $(cat .bash_profile)
     then
     echo 'export PATH=~/gcc-arm-none-eabi-5_4-2016q3/bin:$PATH' >> .bash_profile
     source .bash_profile
     fi
   fi
    cd magic-lantern
    hg update qemu -C
    cd contrib/qemu/
    ./install.sh

[kotik]

@Danne
On my iMac: version 9.1.0

Code Select Expand

gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.1.0 (clang-902.0.39.2)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Installed bash 4

Code Select Expand

$ bash --version
3.2.57(1)-release
brew install bash
sudo chsh -s /usr/local/bin/bash

bash --version
GNU bash, versie 4.4.19(1)-release (x86_64-apple-darwin17.3.0)
Copyright (C) 2016 Free Software Foundation, Inc.

Still getting:

Code Select Expand

-bash: syntax error near unexpected token `&'

Danne suggested to skip the '&' altogether and now I do get a complete list of debug messages.

Code Select Expand

./run_canon_fw.sh 80D,firmware="boot=0" -d debugmsg | grep -E --text Notify.*Cur
...
[DM] FROM Write Complete!!!
[      DbgMgr:fe5336b5 ] (00:01) [PM] Enable (ID = 18, cnt = 0/2)

But the terminal locks up again!

[Danne]

If running the qemu installer on that machine the script will skip .bash_profile as it looks now. Should work. At least with dbg...

[a1ex]

@Danne: you don't need to run native GDB (that's used to debug host programs; in your case, Mac programs). You need to run arm-none-eabi-gdb. Version 8.0.50.20171008-git is known not to work, version 8.1 is known to work. I did not test intermediate versions.

There is a reason why the user has to type that PATH command. A subshell cannot change environment variables of the parent shell. Don't hide it.

No idea what to do about terminal lock-ups (aka problems from 1998). Run Linux in a VM?

[OlRivrRat]

                     @Kotic

           I spend probably 99% of my Computer Time in 10.6.8 as there is some functionality that I have grown

accustomed to that has gone away in newer OSs. When I need to use a newer OS I use 10.11.6.

[kotik]

@a1ex

I suspect the problem lies within Qemu.
If I press C to "open" the card door => also clean shutdown within Magic Lantern Rescue
the terminal locks up.

system_powerdown --> No response!

@OlRivrRat

Maybe someone here can confirm that your problems with ML could be related with the older MacOSX El Capitan.

[Danne]

@Danne: you don't need to run native GDB (that's used to debug host programs; in your case, Mac programs). You need to run arm-none-eabi-gdb. Version 8.0.50.20171008-git is known not to work, version 8.1 is known to work. I did not test intermediate versions.

There is a reason why the user has to type that PATH command. A subshell cannot change environment variables of the parent shell. Don't hide it.

No idea what to do about terminal lock-ups (aka problems from 1998). Run Linux in a VM?

Ok, thanks.
Back to the drawing board...

[OlRivrRat]

                     @Kotik

           Back when Switch came on scene I asked Danne if it would run in 10.11.6 & was assured that it would &

it does, Don't recall if I asked Him about Compiler but it does seem to work OK when I do things Correctly.

           I'm just pretty much a GUI kind of user so stumble a bunch in Terminal >

Might say I have a Terminal Illness, I'm > Terminally Not Savvy!

[kotik]

@a1ex

Did the full install of the Mac sticky tweet.
Got the same result as with the Compiler.app.
If I press 'C' to "open" the card door => also clean shutdown within Magic Lantern Rescue,
or type 'system_powerdown' there is no response at all.

When I close the Qemu window:

Code Select Expand

iMac-27:qemu-eos $ c
-bash: c: command not found
And I'm able to use the same Terminal window as usual.

[a1ex]

Sounds about right. The keys are not yet handled on DIGIC 6, as the emulation doesn't reach the GUI. This shutdown method applies to models where the GUI works; it depends on DryOS and the GUI tasks being alive to handle the shutdown events.

[emklap]

Great to see some progress :-)

I found some time and have created new dumps on my camera  again, just for  fun. (80D / 1/.02) .Any interest in the files ROM1.BIN and SFDATA.BIN?

By the way, I performed 2 attempts on two SD cards. The MD5's of SFDATA is matching, the ones for ROM1 are not.

[a1ex]

Looks good

Please find the updated ROM dumper, with serial flash support:

DMPF_80D.FIR

Outputs ROM1.BIN (bootloader + main firmware) and SFDATA.BIN (serial flash contents).
Codebase: portable ROM dumper (check for background info and requirements / limitations).
To run FIR files, you do not need the boot flag enabled; just copy the FIR to a formatted card (not bootable) and run Firmware Update.

Next steps:
- you may use this ROM dump for emulation, firmware analysis, finding stubs etc
- to run your code on the camera (autoexec.bin files, e.g. if you compile from source), you need to:
  - enable the boot flag in the camera
  - make your card bootable (EosCard, MacBoot or make_bootable.sh/py).

Please check whether it works and produces valid files (just in case; it's the same code as above).

[emklap]

Tested this new dumper, manually executed from 2GB card.  The files looks fine to me, 32 MB & 8 MB with what looks to me read data.
Both BIN files contain data; SFDATA md5 matches previous ones, ROM1 md5 not.





2nd attempt on 128MB card + autoexecute.bin.Now background is normal (black) again.
New MD5 for ROM1.bin again, md5 SFDATA unchanged.

[a1ex]

That means, the caching issues are not fully solved yet. Feel free to look into it - source code is in the recovery branch. Refer to 5DS experiments as well. Keep in mind the issue is not deterministic - first step would be to find out how many start-up attempts are required to get at least one bad screen (rough guess: 10? 50?)

I'm unable to make further progress on this issue, as it requires trial and error with a DIGIC 6 camera in one's hands, and a large number of test runs to confirm the issue was, indeed, solved.

[OlRivrRat]

Attempting to Run this

https://a1ex.magiclantern.fm/bleeding-edge/80D/DMPF_80D.FIR

on 258MB SD on My 80D causes Soft Brick.

[a1ex]

Nothing printed on the screen? Repeatable? I find it unlikely to work only for some users, but not for others. Keep in mind the color palette issue is not deterministic (expect anything, not just red and black).



Updated the logging experiments to capture the entire MPU communication (previous logs were incomplete). I'm interested in the following logs:

- with latest changeset (1985d10): plain startup logs in regular photo mode, without LiveView (a few different logs, as the startup process is not deterministic)
- with previous changeset (22476f9): one plain startup log in regular photo mode
- if the above won't work, narrow down by trying older changesets

I've tested the logging code in QEMU, but that doesn't guarantee it will work on actual hardware.

Caveat: the latest logging code will create files named DEBUGMSG.LOG (unnumbered). You will need to either change their name or place them into folders.

Tip: you can debug this code in the emulator by compiling with:

Code Select Expand


make clean && make install_qemu ML_MODULES= CONFIG_QEMU=y


You'll get the "qprintf" messages on the QEMU console. However, the binary compiled that way won't work on the camera; use this for testing on real hardware:

Code Select Expand


make clean && make install ML_MODULES=


[sombree]

With 1985d10 as last changeset:
- camera boots up
- DEBUGMSG.LOG is created
- logs - 1 to 3 were taken without lens; 4 to 6 - with lens attached

With 22476f9 as last changeset:
- camera boots up with Err70 but doesn't lock up - I can turn it off with a switch

With 4a8d74a as last changeset:
- camera boots up with Err70 but doesn't lock up - I can turn it off with a switch

With b35a216 as last changeset:
- camera boots up
- DEBUGMSG.LOG is created
- log