Author Topic: How to run Magic Lantern into QEMU?!...  (Read 154512 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12317
  • Emergencies only
Re: How to run Magic Lantern into QEMU?!...
« Reply #325 on: June 16, 2018, 09:00:55 AM »
Pretty sure the log doesn't start with "fssRegister"; there must be some more stuff on the terminal. Best guess, from the line with "BrightnessControl from GUI 0": you had automatic LCD brightness enabled in Canon menu; this part of the code is not emulated well. All my working ROMs have LCD brightness on Manual. I've got one ROM with automatic LCD brightness set, but that one does open Canon menu. I've got another non-working ROM with Control over HDMI enabled, but that one loops in some I2C-related loop.

If it doesn't work with LCD brightness set to Manual, please send me a copy of your ROMs (there might be some other Canon setting that's not emulated well).

You can use any Ubuntu version you wish, including the older one. Other Linux versions should also work.

t3r4n

  • Contributor
  • New to the forum
  • *****
  • Posts: 48
Re: How to run Magic Lantern into QEMU?!...
« Reply #326 on: June 17, 2018, 03:13:55 PM »
Hi guys,
didn't have much time lately. But nonetheless I got some PNs here regarding the use of radare2 for debugging. I thought I might share some of the things I use and some ideas I have for it on the qemu side and maybe someone joins in on it.
So first of all why radare2 its not the standard used by the other guys here. Well frankly IDA pro with ARM debugging cost more than a Camera which has features of MagicLantern available and on gdb I was never able to get a decent gui running and its missing some features I like its free and there are lots of articles on reversing with it. It can do ARM and ARM Thumb and has scripting (python, java ruby ...) interface. It has even an build in emulation which is quite capable of running emulating stuff without qemu.
So lets start.
First you can connect radare2 to QEMU like you would with gdb (-s -S or in the nc qemu.monitor and gdbserver)
The start r2 with the following:
Code: [Select]
r2 -i load_db.r2 -d gdb://localhost:1234 so what is load_db.r2? It is a file giving lots of commands for a good setup here are some of the things I put in it:
Code: [Select]
## Pretty stuff
# Solarized theme
eco solarized
# Use UTF-8 to show cool arrows
e scr.utf8 = true
e scr.utf8.curvy=true
# Show comments at right of disassembly if they fit in screen
e asm.cmtright=true
## Processor stuff
# set arch and cpu type
e io.va = true
e asm.arch = arm
e asm.bits = 16
e asm.cpu=cortex
# anal.armthumb (aae computes arm/thumb changes (lot of false positives ahead))
e anal.armthumb=true
# Shows pseudocode in disassembly. Eg mov eax, str.ok = > eax = str.ok
e asm.pseudo = true
# (Show ESIL instead of mnemonic)
# e asm.esil = true
# Selected: asm.describe (Show opcode description)
e asm.describe = false
#asm.emu (Run ESIL emulation analysis on disasm)
e asm.emu = true
e asm.section.sub = true
e io.va=true
that was quite generic and the comments should tell ya whats happening. The following is camera specific it sets up memory regions and gives names to these regions.
Code: [Select]
S 0x00000000 0x00000000 0x00003fff tcmcode mrwx #00000000 - 00003FFF: eos.tcm_code
S 0x00004000 0x00004000 0x1FFFC000 eosram mrw- #00004000 - 1FFFFFFF: eos.ram
S 0x40000000 0x40000000 0x00004000 eosramuncached0 mrw- #40000000 - 40003FFF: eos.ram_uncached0
S 0x40004000 0x40004000 0x1FFFC000 eosramuncached mrw- #40004000 - 5FFFFFFF: eos.ram_uncached
S 0x80000000 0x80000000 0x00010000 tcmram mrw- #80000000 - 8000FFFF: eos.tcm_data
S 0xBFE00000 0xBFE00000 0x00200000 eosramextra mrw- #BFE00000 - BFFFFFFF: eos.ram_extra
S 0xc0000000 0xc0000000 0x20000000 eosiomem mrw- #C0000000 - DFFFFFFF: eos.iomem
S 0xfc000000 0xfc000000 0x20000000 eosrom1 mr-x #FC000000 - FDFFFFFF: eos.rom1
S 0xfe000000 0xfe000000 0x20000000 eosrom1m mr-x#FE000000 - FFFFFFFF: eos.rom1_mirror
the next lines will setup analysis and define some flags in memory taken from debugmsgs.gdb. r2 uses flags for everything and if I understand documentation right functions follow a fcn.<name> scheme. I have so far not been able to use the afn command to create functions but more later. It would be possible to define these flags as breakpoints and put a modified version of a1ex script for indentifiying functions here. Speaking of which.
I used the script of a1ex to create as described above in this thread an .idc file.
Here I modified the header as follows:
Code: [Select]
#include "stubshelper.h"

int  main(void)
{
  MakeAutoNamedFunc(0xFE0FD5C9, "LoadScript");

and another file stubhelper.h 
Code: [Select]
#include <stdio.h>


void MakeAutoNamedFunc(unsigned int ea ,char name[])
{
  printf("f %s = 0x%0X\n",name,ea);
}

void NSTUB(unsigned int ea ,char name[])
{
  printf("f %s = 0x%0X\n",name,ea);
}
compile and pipe the output to your load_db.r2
inside radare2 you can now use
Code: [Select]
af @@@f
s fcn.<name>
Vpp
to inspect a function.

Ideas:
- Radare provides a scripting interface. Use python script to search through memory for e.g. Frambuffer
- define Names for IO areas to have them marked in the assembly
- is it possible to use the signaturez function of radare to help speedup new firmware ports or new ports.
- ...

Questions:
- anyone got a better idea on how to define functions

Further reading:
- i found this Video of a talk very helpful where the inventor of radare describes how to use it to reverse an ARM based radio: http://radare.org/r/talks.html the talk in 2017. I have not yet looked into the possibility of emulation of io devices via the scripted breakpoints ...

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12317
  • Emergencies only
Re: How to run Magic Lantern into QEMU?!...
« Reply #327 on: June 17, 2018, 04:07:42 PM »
Very cool, maybe you should consider a sticky topic in the Reverse Engineering area. The memory map (and possibly an initial r2 script) should be autogenerated from QEMU, in a way similar to the IDCs.

One quick note:
(aae computes arm/thumb changes (lot of false positives ahead))

Same with IDA. That's why the autogenerated .idc script (the one from -d idc, without function names) specifies whether a function is ARM or Thumb, based on how it was actually executed during emulation.

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #328 on: June 19, 2018, 06:27:10 AM »
Something weird is going on with QEMU , tried a new saved rom dump (5D2) (i made sure the lcd screen was set to manual)
Code: [Select]
./run_canon_fw.sh 5D2

 
says in the top left connor "QEMU(5d2)" but try and read it as 5d3.113  ???

Code: [Select]
./run_canon_fw.sh 5D2
DebugMsg=0xFF86AF64 (from GDB script)
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
E8000000 - E8052FFF: eos.ram_extra
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - CFFFFFFF: eos.mmio
[EOS] loading './5D2/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] mirrored data; unique 0x400000 bytes repeated 0x4 times
[EOS] loading './5D2/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[EOS] mirrored data; unique 0x800000 bytes repeated 0x2 times
[MPU] warning: non-empty spell #5 (PROP_CARD3_STATUS) has duplicate(s): #16
[MPU] warning: non-empty spell #17 (PROP_CARD1_STATUS) has duplicate(s): #48
[MPU] warning: non-empty spell #31 (PROP 8003001A) has duplicate(s): #36
[MPU] warning: non-empty spell #37 (PROP_VIDEO_MODE) has duplicate(s): #38

[MPU] Available keys:
- Arrow keys   : Navigation
- Numpad keys  : Joystick (8 directions)
- Numpad 5     : Joystick center
- PgUp, PgDn   : Sub dial (rear scrollwheel)
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess (press only)
- M            : MENU (press only)
- P            : PLAY (press only)
- I            : INFO/DISP (press only)
- L            : LiveView (press only)
- W            : Pic.Style (press only)
- Z/X          : Zoom in/out
- Shift        : Half-shutter
- 0/9          : Mode dial (press only)
- V            : Movie mode (press only)
- B            : Open battery door
- C            : Open card door
- F10          : Power down switch
- F1           : show this help

[DMA3] Copy [0xF8760000] -> [0xE8000000], length [0x00053000], flags [0x00000001]
[DMA3] OK
FFFF2368: MCR p15,0,Rd,cr6,cr0,0:  946_PRBS0 <- 0x3F       (00000000 - FFFFFFFF, 0x100000000)
FFFF2370: MCR p15,0,Rd,cr6,cr1,0:  946_PRBS1 <- 0x3D       (00000000 - 7FFFFFFF, 0x80000000)
FFFF2378: MCR p15,0,Rd,cr6,cr2,0:  946_PRBS2 <- 0xE0000039 (E0000000 - FFFFFFFF, 0x20000000)
FFFF2380: MCR p15,0,Rd,cr6,cr3,0:  946_PRBS3 <- 0xC0000039 (C0000000 - DFFFFFFF, 0x20000000)
FFFF2388: MCR p15,0,Rd,cr6,cr4,0:  946_PRBS4 <- 0xFF80002D (FF800000 - FFFFFFFF, 0x800000)
FFFF2390: MCR p15,0,Rd,cr6,cr5,0:  946_PRBS5 <- 0x39       (00000000 - 1FFFFFFF, 0x20000000)
FFFF2398: MCR p15,0,Rd,cr6,cr6,0:  946_PRBS6 <- 0xF780002D (F7800000 - F7FFFFFF, 0x800000)
FFFF23A0: MCR p15,0,Rd,cr2,cr0,0: DCACHE_CFG <- 0x70       
FFFF23A8: MCR p15,0,Rd,cr3,cr0,0:       DACR <- 0x70       
FFFF23AC: MCR p15,0,Rd,cr2,cr0,1: ICACHE_CFG <- 0x70       
FFFF23B0: MCR p15,0,Rd,cr5,cr0,0:    DATA_AP <- 0x3FFF     
FFFF23B8: MCR p15,0,Rd,cr5,cr0,1:    INSN_AP <- 0x3FFF     
FFFF23BC: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0x2078
FFFF23BC: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC000307D
FFFF05F8: MCR p15,0,Rd,cr9,cr1,1: XSCALE_UNLOCK_ICACHE <- 0x6        (00000000 - 00000FFF, 0x1000)
FFFF05F8: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC000307D
FFFF05F8: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC004307D
FFFF0634: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC004307D
FFFF0634: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC004107D
FFFF0634: MCR p15,0,Rd,cr9,cr1,0: XSCALE_LOCK_ICACHE_LINE <- 0x40000006 (40000000 - 40000FFF, 0x1000)
FFFF0634: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC004107D
FFFF0634: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D
CF LOAD OK.
Open file for read : AUTOEXEC.BIN
Total_size=6D300
Now jump to AUTOEXEC.BIN!!
0010C08C: MRC p15,0,Rd,cr1,cr0,0:      SCTLR -> 0xC005107D
0010C08C: MCR p15,0,Rd,cr1,cr0,0:      SCTLR <- 0xC005107D

I tried other roms from other cams (eosm , 6D, 5d3.123 , 100D
same thing .

I guess i could try re-installing QEMU and see if that corrects the issue.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12317
  • Emergencies only
Re: How to run Magic Lantern into QEMU?!...
« Reply #329 on: June 25, 2018, 01:12:16 PM »
Hey a1ex,
its a bit rough for start but it works. Seems like gdb 8 resolves the issue with the temporary breakpoints.

[...]

Can someone with a mac verify.

Included this in the install script. Please test, in particular on Mac, WSL, 32-bit Ubuntu and 64-bit Ubuntu.

I'm going to test it as well on some fresh VMs.



Edit - some test results from non-fresh systems (QEMU upgraded from previous installations):

OpenSuse 64-bit: the script installed gcc-arm-none-eabi-5_4-2016q3 and was happy with the gdb found there (32-bit, known to work well).
Mac VM: the script installed gcc 7-2014-q4, then compiled gdb 8.1 from source even though a (buggy) 64-bit gdb was already available from the gcc package. Looks like it's working!
Win10 WSL: seems to be working, still at "configure" after half an hour :P
Xenial 32-bit: OK, script happy with previous gcc/gdb.



Fresh Xenial 32-bit:
Code: [Select]
sudo apt install mercurial
hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg up qemu
cd contrib/qemu
./install.sh
y
1
y
y

Option 1 is gcc/gdb from package manager. Also tested with 2 (launchpad), 3 (ppa) and 4 (gcc from package manager, gdb 8.1 from source).

Fresh Artful 64-bit: tested options 1 and 4. Had to install makeinfo in order to compile gdb. (solved)

Fresh Bionic 64-bit (latest Ubuntu): tried options 1 (it no longer has the 32-bit GDB package; it has gdb-multiarch instead) and 4 (worked).

Nanomad VM: too old, it doesn't have arm-none-eabi-gcc prepackaged, pip not working, qemu compiles but doesn't run... not worth the hassle

Nikfreak VM: disk full, maybe later.

t3r4n

  • Contributor
  • New to the forum
  • *****
  • Posts: 48
Re: How to run Magic Lantern into QEMU?!...
« Reply #330 on: June 25, 2018, 07:53:36 PM »
Included this in the install script. Please test, in particular on Mac, WSL, 32-bit Ubuntu and 64-bit Ubuntu.


I tested it on Mac without the PATH to gdb set the first time it complained about the gdb being there and told me how to export the PATH. I did that.
Second run it complained about gdb being V7 it did install V8 then and told me to export PATH again.
Third time round it did the normal routine of compiling qemu.
So thumbs up from me.

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #331 on: July 17, 2018, 04:28:55 AM »
A new install of Ubuntu 18.04 made a new VDI in VirturailBox
then ran this
Code: [Select]
sudo apt install mercurial
hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg up qemu
cd contrib/qemu
./install.sh
y
4
y
y
Got this at the end .
Code: [Select]
*** Please add GDB binaries to your executable PATH, then run this script again.
*** Run this command, or paste it into your .profile and reopen the terminal:
    export PATH=/home/rdtv/gdb-arm-none-eabi-8_1/bin/:$PATH

Unfortunately this all very strange to me , being a windows 7 guy
can someone help out please ?

.profile ?
How do I add this ?  ???
here my install log/text file
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/install.txt

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12317
  • Emergencies only
Re: How to run Magic Lantern into QEMU?!...
« Reply #332 on: July 17, 2018, 05:37:07 AM »
First results from some search engines:

http://www.theunixschool.com/2011/07/what-is-profile-file.html
https://www.stefaanlippens.net/bashrc_and_others/
https://unix.stackexchange.com/questions/40708/what-is-the-difference-between-profile-bashrc-bash-profile-gnomer

TLDR: generally, settings in *nix systems are changed by editing text files. To change the executable PATH (you have that one in Windows 7, too), you add commands like the above to your .profile file with a text editor. You could just paste that command the terminal, but the effect would be temporary (i.e. you'd have to do this again when opening a new terminal).

Here's some code to automate this step (didn't try yet): https://github.com/mitsuhiko/pipsi/pull/148/files

Option 1 (the recommended choice) didn't work?

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #333 on: July 17, 2018, 06:19:58 AM »
Thanks a1ex , I thought I need to use option #4 , I'll re-run with #1

Edit: Option#1 same to work Ok , it's compiling qemu as I write this
Didn't Compile , Not too sure why I may have to remake the vdi and start fresh
Code: [Select]
collect2: error: ld returned 1 exit status
Makefile:193: recipe for target 'qemu-system-arm' failed
make[1]: *** [qemu-system-arm] Error 1
Makefile:184: recipe for target 'subdir-arm-softmmu' failed
make: *** [subdir-arm-softmmu] Error 2

*** Compilation failed.
*** Please check what went wrong, try to fix it and report back.
rdtv@reddeercity:~/magic-lantern/contrib/qemu$
Log/txt file from the fail
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/failed%20qemu.txt

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #334 on: July 17, 2018, 07:10:19 AM »
Fails on these lines from the errors i posted
line 193
Code: [Select]
(cd $(SRC_PATH)/pixman; autoreconf -v --install)line 184
Code: [Select]
[$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" all,)Is this still the path problem i had ?
Maybe best to start with  a fresh vm

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3565
  • Blunt and to the point
Re: How to run Magic Lantern into QEMU?!...
« Reply #335 on: July 17, 2018, 11:07:09 AM »
To change the executable PATH (you have that one in Windows 7, too), you add commands like the above to your .profile file with a text editor. You could just paste that command the terminal, but the effect would be temporary (i.e. you'd have to do this again when opening a new terminal).

In windows GUI


The top path variable is for the current user only, the bottom path variable is system wide.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12317
  • Emergencies only
Re: How to run Magic Lantern into QEMU?!...
« Reply #336 on: July 17, 2018, 02:10:03 PM »
Didn't Compile , Not too sure why I may have to remake the vdi and start fresh

Tried again on my fresh Ubuntu Bionic VM, with option 1. No surprises.

No idea what's going on; expecting to see some sort of error message in the log. Some of the errors mention "make[1]: flex: Command not found", but I don't have this command either on the VM (and I also get these messages). Try running "make V=1" from qemu-eos/qemu-2.5.0:
Code: [Select]
cd ~/qemu-eos/qemu-2.5.0
make V=1

That should be a little more verbose, but normally compile errors should show up without any tweaking...

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #337 on: July 18, 2018, 02:29:49 AM »
Code: [Select]
cd ~/qemu-eos/qemu-2.5.0
make V=1
Yea that worked , thanks a1ex , There still some errors but work to compile QEMU ,
here the log/txt from that --> make_V=1log/txt , then to test I run
Code: [Select]
./run_canon_fw.sh 5D2the log/txt file is here , was successful .
Then tried
Code: [Select]
./run_canon_fw.sh 5D2,firmware="boot=0"Log/txt file here , was not successful got the same grey screen like before . I'll re-make my VM and start from fresh , there same problem there maybe from the all the updates .
I checked the version ,  Ubuntu Bionic Beaver 18.04. So I'll try again and report back .

Edit: Got this in the beginning for error , I noticed
Code: [Select]
Gtk-Message: 18:03:50.078: Failed to load module "canberra-gtk-module"means something but not to me ,

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3739
Re: How to run Magic Lantern into QEMU?!...
« Reply #338 on: July 18, 2018, 04:12:56 AM »
...got the same grey screen like before...

The 5D2 launches into a grey screen on my system too. Pressing the "M" key brings up the Canon menu. However, that error message looks like there might be a problem with your GTK (GIMP Tool Kit) installation. Try reinstalling libgtk2.0-dev. If that doesn't work, maybe try libsdl1.2-dev.

contrib/qemu/install.sh
Code: [Select]
if [  -n "$(lsb_release -i 2>/dev/null | grep Ubuntu)" ]; then
    # Ubuntu-based system? (including WSL)
    # install these packages, if not already
    # only request sudo if any of them is missing
    # instead of GTK (libgtk2.0-dev), you may prefer SDL (libsdl1.2-dev)
    packages="
        build-essential mercurial pkg-config libtool
        git libglib2.0-dev libpixman-1-dev zlib1g-dev
        libgtk2.0-dev xz-utils mtools netcat-openbsd
        python python-pip python-docutils"
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #339 on: July 18, 2018, 06:00:50 AM »
Thanks @dfort , I got it to work now so far   :)) . I Re-made my VM (new install) with Ubuntu Bionic Beaver 18.04 , ran the update and got the latest updates for Ubuntu .
Then ran the install script
Code: [Select]
sudo apt install mercurial
hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg up qemu
cd contrib/qemu
./install.sh
y
1
y
y
Here is the Log/txt file of the process . Yes I see , I pressed "m" and it's there  8)



I'll try some other ROM's also to see if it's fully functional e.g. digic5 , if I'm not mistaken Liveview is not working for 5d2/digic4 right ?
I need it to figure out compressed raw & full res MJpeg at the moment plus other thing down the road .

Edit: Some screen shots of 5D2 with Liveview  ,most of the navigation works so far just need to understand how to use qemu commands .





reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #340 on: July 19, 2018, 07:48:51 AM »
Got it running on my AMD Desktop(FX8350) , followed the same process as my Intel laptop
ran this
Code: [Select]
./run_canon_fw.sh 5D2 -d io,intto get MMIO activity (registers) and interrupts , some very interesting information . Here is the MMIO activity Log/txt file for the 5D2, too many enters to post .
I did see the same error as my laptop
Code: [Select]
Gtk-Message: 23:15:38.959: Failed to load module "canberra-gtk-module"
However, that error message looks like there might be a problem with your GTK (GIMP Tool Kit) installation. Try reinstalling libgtk2.0-dev. If that doesn't work, maybe try libsdl1.2-dev.

contrib/qemu/install.sh
Code: [Select]
if [  -n "$(lsb_release -i 2>/dev/null | grep Ubuntu)" ]; then
    # Ubuntu-based system? (including WSL)
    # install these packages, if not already
    # only request sudo if any of them is missing
    # instead of GTK (libgtk2.0-dev), you may prefer SDL (libsdl1.2-dev)
    packages="
        build-essential mercurial pkg-config libtool
        git libglib2.0-dev libpixman-1-dev zlib1g-dev
        libgtk2.0-dev xz-utils mtools netcat-openbsd
        python python-pip python-docutils"
I'll have to try the fix @dfort suggests , but at this point I'm looking for logs files to help with reverse engineering  on digic iv

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12317
  • Emergencies only
Re: How to run Magic Lantern into QEMU?!...
« Reply #341 on: July 19, 2018, 10:05:18 AM »
Nice, that's how far the emulation goes here as well.

In LiveView, you can look at the MMIO activity; it will show some initial configuration (ADTG, CMOS etc), and at some point it will expect some HEAD timer interrupts. These are not implemented, but I had a few attempts; will keep trying. For the emulation on the home page, I've loaded the image buffers manually.

That error about canberra is probably harmless: it appears to be a sound library, and I didn't try to implement anything audio-related yet.

https://stackoverflow.com/questions/20518346/gtk-message-failed-to-load-module-canberra-gtk-module

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #342 on: July 20, 2018, 08:11:07 AM »
Thanks, I'm now try to do a "FACapture Test" i found this
Code: [Select]
./run_canon_fw.sh 5D2,firmware=boot=1 -display none -monitor stdio -d debugmsg,io,int -serial file:5D2.212-frsp-uart.log -s -S &
arm-none-eabi-gdb -x 5D2/debugmsg.gdb &
Didn't really work , thou it did make the "5D2.212-frsp-uart.log" but with "0" data , It complained about  arm-none-eabi-gdb that I don't have i guess , so i drop that & the -x 5D2/debugmsg.gdb & still no go .
From the HACKING.rst --  I then tried to run the Test suite
Code: [Select]
- Bootloader code (to make sure AUTOEXEC.BIN is loaded from the card)
- Portable display test (all EOS models)
- Portable ROM dumper (EOS models with bootloader file write routines)
- Menu navigation (on supported models) — depends on user settings from the ROM
- Card formatting (and restoring ML)
- Call/return trace until booting the GUI (a rigid test that may have to be updated frequently)
- Call/return trace on bootloader (likely independent of firmware version and user settings)
- Callstack consistency with call/return trace (at every DebugMsg call)
- File I/O (whether the firmware creates a DCIM directory on startup)
- FA_CaptureTestImage (basic image capture process, without compression or CR2 output)
- HPTimer (difficult to get right)
- DryOS task information (current_task, current_interrupt)
- GDB scripts (just a few basics)
- DryOS shell (UART)
- PowerShot models (limited tests)
- Secondary DryOS cores (limited tests)
with
Code: [Select]
./run_tests.sh 5D2 got errors
Code: [Select]
dmiazga@reddeercity:~/qemu-eos/tests$ ./run_tests.sh 5D2
Using netcat: nc -N
Compiling...
/sbin/losetup
/sbin/losetup
/dev/loop12p1 /media/dmiazga/EOS_DIGITAL vfat ro,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro 0 0

Error: please unmount the CF image.

and tried
Code: [Select]
./run_tests.sh 5D2 menu calls-main dryshgot this
Code: [Select]
dmiazga@reddeercity:~/qemu-eos/tests$ ./run_tests.sh 5D2 menu calls-main drysh
Using netcat: nc -N
Compiling...
/sbin/losetup
/sbin/losetup
/dev/loop12p1 /media/dmiazga/EOS_DIGITAL vfat ro,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro 0 0

Error: please unmount the CF image.
"please unmount the CF image"
So how do I unmount ?
I'm just trying to run the FA_CaptureTestImage , is there something I'm missing ? Do i have to have a full set of Image Vram dumps ?  Can that test be done with compression ? CR2 so i can get the info for compressed raw.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12317
  • Emergencies only
Re: How to run Magic Lantern into QEMU?!...
« Reply #343 on: July 20, 2018, 08:28:11 AM »
In your install log, the short guide printed on the console should have told you to use gdb-multiarch instead of arm-none-eabi-gdb. They are interchangeable.

To unmount: click on the Eject icon in the file manager.

Will test these steps later in the VM, maybe there are some other quirks.

For FA_CaptureTestImage, you only need a reference DNG image (you'll see it in the error messages).

critix

  • Contributor
  • Member
  • *****
  • Posts: 113
Re: How to run Magic Lantern into QEMU?!...
« Reply #344 on: July 20, 2018, 03:08:27 PM »
- debugging your code like a PC program, by running it step by step (not just with printf's)
How can debugging step by step?
Thanks
Canon 1300D, 500D, EOS M, EOS M2


reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #346 on: July 22, 2018, 02:42:51 AM »
This looks to be the correct command for cr2 compression image test
To capture a full-res image  using a CR2
Code: [Select]
make -C ../magic-lantern/minimal/qemu-frsp MODEL=5D2 CONFIG_QEMU=y clean install_qemu
env QEMU_EOS_VRAM_PH_QR_RAW='/path/to/IMG_1234.CR2' ./run_canon_fw.sh 5D2,firmware="boot=1"
This was for 5d3  , I changed it to 5d2 . I haven't tried it yet , I guess the big differences from the "Test" in qemu I tried before  is this uses ML running with FRSP .
I also see a "envorment" path , I'll give it a go and see what happens .

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #347 on: July 22, 2018, 03:44:52 AM »
Back to
Code: [Select]
~/qemu-eos/tests$ ./run_tests.sh 5D2
Using netcat: nc -N
Compiling...
/sbin/losetup
/sbin/losetup
/dev/loop13p1 /media/dmiazga/EOS_DIGITAL1 vfat ro,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro 0 0

Error: please unmount the CF image.
got further when I unmounted the CF card just as a1ex suggested in a post above ,
at first i couldn't find it . It ended up in the "Other Location" in Ubuntu  , It had 2 virtual cf cards I unmounted them both , run the
Code: [Select]
./run_tests.sh 5D2 again stopped at "Testing Canon menu ... " could not find the "sponge" command
Code: [Select]
dmiazga@reddeercity:~/qemu-eos/tests$ ./run_tests.sh 5D2
Using netcat: nc -N
Compiling...

Setting up temporary SD/CF card images...
'../magic-lantern/contrib/qemu/sd.img.xz' -> './sd.img.xz'

Testing Canon menu...
./run_tests.sh: line 398: sponge: command not found
Code: [Select]
    ( printf "%7s: " $CAM && test_$1 ) 2>&1 | spongeWhat's the "sponge" command ?
is there some workaround here  , or ?

Edit: Ok got "sponge" to work i had to to install a tools set ubuntu/bionic/moreutils
Code: [Select]
/usr/bin/chronic
.......
/usr/bin/sponge
/usr/bin/ts
/usr/bin/vidir
/usr/bin/vipe
/usr/bin/zrun
/usr/share/doc/moreutils/README
.....

Now it stopped at
Code: [Select]
~/qemu-eos/tests$ ./run_tests.sh 5D2
Using netcat: nc -N
Compiling...

Setting up temporary SD/CF card images...
'../magic-lantern/contrib/qemu/sd.img.xz' -> './sd.img.xz'

Testing Canon menu...
    5D2: ¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿...¿.¿.¿tests/check_grep.sh: line 2: ansi2txt: command not found
FAILED!
so i guess i need to find "ansi2txt" now

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2096
Re: How to run Magic Lantern into QEMU?!...
« Reply #348 on: July 22, 2018, 06:05:08 AM »
However , after checking the "Test"directory in the 5D2 folder i see there is 30 PNG images and a Menu.Log  :))
the LOG makes me so Excited ! I see very valuable info , it's like a startup Log but so much more , even has some "Resources" for compressed raw , not to sure how complete it is but i never seen this level of information .
I'll post the LOG file link shortly .
 

critix

  • Contributor
  • Member
  • *****
  • Posts: 113
Re: How to run Magic Lantern into QEMU?!...
« Reply #349 on: July 22, 2018, 06:13:31 AM »
How long does the script for a camera run? I ran for 1300D, but after 2 hours, I had to stop the script.
Canon 1300D, 500D, EOS M, EOS M2