Compiling Magic Lantern on Windows 10 (using its Linux subsystem)

[jml2017]

Hello ML/Linux Gurus,

I am trying to install QEMU and the procedure described in the first post works for me until the ./install.sh step.

At the very beginning of the script execution it exits with the following messages





I added the command to the profile and checked that the path is now correct with "echo $PATH" command:
The path variable contains
/home/myusername/gcc-arm-none-eabi-5_4-2016q3/bin

I also checked that this folder exists and is populated with files.
However, the script still exits at the same point... I restarted the bash with no success.

It may be obvious for practitionners, but my experience of linux is quite limited. 

I am running Windows 10 with Ubuntu installed as per first post.

Thanks for your help.

[a1ex]

Incidentally I was currently testing QEMU installation on WSL and noticed the same issue - try "hg pull; hg update qemu -C" to get the fix.

No guarantees the next steps are actually working - will try them later after a short break.

edit: seems to work; still want to re-test on a fresh installation.

[jml2017]

Ok, with this command, the script goes further. But now it complains that I dot have the right GDB




I guess the best choice is option 5. However I am not sure how to install this from https://launchpad.net/gcc-arm-embedded. Which command line should I use?

Thanks for the help.

[a1ex]

For trying out current stuff, I hope option 4 should work fine (that's also because most of the GDB patches that were causing trouble are no longer needed).

For 5, just unzip a Linux 64-bit build in your home directory and add it to your PATH.

I've ran some parts of the test suite with gcc-arm-none-eabi-6-2017-q2-update, gcc-arm-none-eabi-6-2017-q2-update and gdb 8.0.50.20171008-git (all 64-bit) and worked fine. However, if I get the 5D3/patches.gdb back from changeset 93e57ce57802 and intentionally break the RTC emulation - so the patch will execute something - all these 3 64-bit versions will fail. The 32-bit gdb from gcc-arm-none-eabi-5_4-2016q3 (and any other 32-bit versions I've tried) will work just fine.

Recently I've managed to crash the 32-bit gdb on 5D classic code, and the 64-bit gdb 8.0.50.20171008-git worked to some extent (so, the 32-bit GDB isn't bug-free either).

My conclusion so far: for serious GDB use on DIGIC 4 and 5 firmwares, you are more likely to get in trouble with a 64-bit GDB (but you should still be able to run the examples from the QEMU guide).

[jml2017]

I selected 4, and now it complains again about the PATH even though the PATH is correct...

[a1ex]

Hm, that's a bug; it should have installed arm-none-eabi-gcc/gdb from the package manager, so they should be already in PATH and recognized as valid...

Do you get any output if you type arm-none-eabi-gdb at the bash prompt?

(will look into it, but need to reinstall the VM first)

[jml2017]

That is what I get:

-bash: /home/jml2017/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gdb: cannot execute binary file: Exec format error

[a1ex]

Hm, it (or maybe you) somehow downloaded the 32-bit toolchain (which WSL can't run), and you have added it to PATH. If you ran the command suggested by the script, close the bash window and open a new one, as that change is temporary.

[a1ex]

Made some WSL changes to QEMU install script, but for some reason I'm unable to bring the Win10 VM back to life (and I'm not currently on a fast internet connection either). Would appreciate some help with testing latest changes (both from scratch and on top on some existing installation).

Please include an installation log, if possible, or some screenshots from the process (for example, the prompts you may get during installation). Thanks.

[jml2017]

Hello,
Reinstalled Ubuntu from scratch and ran the script install.sh for QEMU with no issues until I get...

---------------------------------------------------------------------------------------------------------------
Initialized empty Git repository in /home/username/qemu/qemu-2.5.0/.git/

*** Please tell me who you are.

Run

  git config --global user.email "[email protected]"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: empty ident name (for <[email protected]>) not allowed

-----------------------------------------------------------------------------------------------------

What should I do?

Thanks.

[a1ex]

Okay, that must be a newer git than what I've tested; will check.

If this patch works for you, I'll commit it:

Code Select Expand


--- a/contrib/qemu/install.sh
+++ b/contrib/qemu/install.sh
@@ -389,6 +389,7 @@
   git init
   # git requires a valid email; if not setup, add one for this directory only
   git config user.email || git config user.email [email protected]
+  git config user.name || git config user.name qemu-eos
   git add . && git commit -q -m "$QEMU_NAME vanilla"
fi
cd ..


[jml2017]

Okay QEMU compiles with no extra errors after adding your extra line to the script.

I am now at the step of running ./run canon_fw.sh 6D. However it errors because it cannot open /6D/SFDATA.BIN. If I understand well, I should be able to retrieve this file by running some module sf_dump on my 6D. However, I do not see this module on the ML camera menus. Where is it?

Thanks

[a1ex]

It's covered under Installation (no binary available yet, but I should provide one).

[jml2017]

I followed steps from https://www.magiclantern.fm/forum/index.php?topic=19232.0

I compiled the module with
~/magic-lantern/modules/sf_dump$ make

Then I had to mount the SD card  so that Ubuntu subsystem of windows 10 sees it:

sudo mkdir /mnt/SD
sudo mount -t drvfs F: /mnt/SD (F: is the SD drive under windows 10)

Then  ran :
make install CF_CARD=/mnt/SD

Finally, when starting my 6D with the SD card, I could load the sf_dump module and run it to get the SFDATA.BIN.

So I ran again ./run_canon_fw.sh 6D and got the following output:

Code Select Expand


/sbin/losetup
/sbin/losetup
make: Entering directory '/home/username/qemu/qemu-2.5.0'
CHK version_gen.h
LEX convert-dtsv0-lexer.lex.c
make[1]: flex: Command not found
BISON dtc-parser.tab.c
make[1]: bison: Command not found
LEX dtc-lexer.lex.c
make[1]: flex: Command not found
make: Leaving directory '/home/username/qemu/qemu-2.5.0'
./run_canon_fw.sh 6D &

DebugMsg=0x67c8 (from GDB script)
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - DFFFFFFF: eos.iomem
[EOS] loading './6D/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './6D/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[EOS] loading './6D/SFDATA.BIN' as serial flash, size=0x800000
[MPU] warning: non-empty spell #2 (Complete WaitID = 0x80000001 Mode group) has duplicate(s): #6
[MPU] warning: non-empty spell #52 (PROP_VIDEO_MODE) has duplicate(s): #53

[MPU] Available keys:
- Arrow keys   : Navigation
- PgUp, PgDn   : Sub dial (rear scrollwheel)
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess (press only)
- M            : MENU (press only)
- P            : PLAY (press only)
- I            : INFO/DISP (press only)
- Q            : guess (press only)
- L            : LiveView (press only)
- Shift        : Half-shutter
- 0/9          : Mode dial (press only)
- B            : Open battery door
- C            : Open card door
- F10          : Power down switch
- F1           : show this help

gtk initialization failed
[MPU] WARNING: forced shutdown.

For clean shutdown, please use 'Machine -> Power Down'
(or 'system_powerdown' in QEMU monitor.)

What did I do wrong?

[a1ex]

You need to install an X server (see earlier in this thread) and export the DISPLAY variable. This step is specific to WSL.

Make sure you can run other GUI programs (such as gitk).

[jml2017]

Ok I installed VcXsrv.

Now I get this error:

[a1ex]

Press M and start navigating the menu. For some reason, the 6D starts with main display off - was your camera configured that way before dumping the ROM? Or press I(nfo) to turn it on.

The red messages are printed by Canon firmware to the serial console.

[jml2017]

Ok it works now 




I'd like to generate the zebras and histogram I get with ML on the real camera.

I tried this command with some image
env QEMU_EOS_VRAM_PH_QR_RAW='/path/to/IMG_1234.CR2' ./run_canon_fw.sh 6D,firmware="boot=1", but I do not get anything.

What is the procedure to emulate the capture from an image and the image review that we get right after?

Thanks again.

[clanlee]

I need some assistance.  I followed the instructions and got to the point where I can call up the QEMU window but I can't do anything from there.  It looks like Magic Lantern is not loaded so I have nothing but the QEMU window.  I tried pressing M for menu and all other combinations of buttons but nothing happens.

How do I load Magic Lantern within the Ubuntu environment?  Do I have to copy the contents of the nightly build somewhere?  Thank you.

[dfort]

Do I have to copy the contents of the nightly build somewhere?

Yes, it needs to be in the sd.img or cf.img depending on which (virtual) camera you are using. I had some problems on the EOSM and it turned out there was a problem with the SFDATA.BIN file I was using.

Did you follow these instructions?

https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst

If you have a problem with the user guide or suggestions, a1ex is looking for feedback -- post it on the How to run Magic Lantern into QEMU?!... topic.

[clanlee]

Yes, it needs to be in the sd.img or cf.img depending on which (virtual) camera you are using. I had some problems on the EOSM and it turned out there was a problem with the SFDATA.BIN file I was using.

Did you follow these instructions?

https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst

If you have a problem with the user guide or suggestions, a1ex is looking for feedback -- post it on the How to run Magic Lantern into QEMU?!... topic.

Yes, I was following those instructions but as a linux newbie, I ran into a road block when trying to mount the sd.img file.  I'm trying to setup the environment for the 5D3.123.  I wasn't sure where to post the question since this is specific to the Windows 10 running Ubuntu so I thought this might be a good place.

This is where I ran into the problem.  According to the instructions:

mount the card image (sd.img or cf.img) as /whatever/EOS_DIGITAL, then run make install from your platform directory:

# from the magic-lantern directory
cd platform/60D.111
make clean; make
# make sure your virtual card is mounted (this step is operating system specific)
make install
# make sure your virtual card is no longer mounted

I ran make clean; make command in the platform/5D3.123 folder.

When I attempted to mount the sd.img file, it's failing due to my lack of knowledge in linux commands.  I tried the following command:

sudo mount -t ~/qemu-eos/sd.img ~/magic-lantern/platform/5D3.123/EOS_DIGITAL -o loop

But received a message that it can't find the path in /etc/fstab.  When I listed the contents of fstab, it's nothing like the suggestions I found when searching for answer.  It's probably due to the fact that it's a virtual environment in Windows. Any help would be appreciated.

[a1ex]

Reproduced; looks like mounting the images in WSL is different; however, it's not really needed. You should be able to use "make install_qemu".

Sorry for the delay when it comes to WSL issues; the Windows VM is painfully slow

It took about 1 hour to run the install script, then 15 minutes just to execute the above command (compared to 1 min + 25 seconds on the local machine). It wasn't a fresh install, just hg pull, hg update, ./install.sh and make install_qemu (so all the dependencies were already there). Hoping a physical Win10 installation is faster than that...

[ewinemiller]

Hey folks,

I'm giving this a whirl and running into python problems when it tries to compile the modules. The error I get is

/bin/sh: 1: python: not found

I found a reference in one thread where adding

PYTHON=python2.7

to the Makefile.user helped with this problem. For me not so much. Also tried python2. I know it's using the value because if I change it to python3, it yells at me about a syntax error in the script.

  File "../readme2modulestrings.py", line 40
    print "static char __module_string_%s_name [] MODULE_STRINGS_SECTION = %s;" % (a, c_repr(name))
          ^
SyntaxError: invalid syntax

If I type python2 or python2.7 in bash, I do get the 2.7 python shell so it does seem to be installed correctly.

Any suggestions? I did pick Ubuntu 20.04LTS for my Linux version and I'm trying to compile Danne's branch (though I had the same problem with unified).

Thanks.

[ewinemiller]

Switched to Unbuntu 18.04LTS and it works fine.

I did pick Ubuntu 20.04LTS for my Linux version

[theBilalFakhouri]

Followed g3gg0 guide from the first post, didn't have problems with packages, and I was able to compile ML and QEMU successfully, very fast compiling speed compared to Cygwin

However I am getting this error after trying to run QEMU:

Code Select Expand

gtk initialization failed

Already installed all required packages and doubled checked that, I installed and tried many X servers (VcXsrv, cygwin x, XMING, and I didn't forget to launch it) then followed these commands:

Code Select Expand

export DISPLAY=:0
./run_canon_fw.sh 700D,firmware="boot=0"

then

Code Select Expand

DISPLAY=:0 ./run_canon_fw.sh 700D,firmware="boot=0"

without a luck, same result:

Code Select Expand

./run_canon_fw.sh 700D,firmware=boot=0

DebugMsg=0x395C (from GDB script)
Lockdown read 4
Lockdown read 4
Lockdown read 5
Lockdown read 5
Lockdown read 0
Lockdown read 0
Lockdown read 1
Lockdown read 1
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 0FFFFFFF: eos.ram
40001000 - 4FFFFFFF: eos.ram_uncached
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - DFFFFFFF: eos.mmio
[EOS] loading './700D/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[EOS] loading './700D/SFDATA.BIN' as serial flash, size=0x800000
[MPU] warning: non-empty spell #36 (PROP_VIDEO_MODE) has duplicate(s): #37

[MPU] Available keys:
- Arrow keys   : Navigation
- [ and ]      : Main dial (top scrollwheel)
- SPACE        : SET
- DELETE       : guess (press only)
- M            : MENU (press only)
- P            : PLAY (press only)
- I            : INFO/DISP (press only)
- Q            : guess (press only)
- L            : LiveView (press only)
- A            : Av
- Z/X          : Zoom in/out
- Shift        : Half-shutter
- 0/9          : Mode dial (press only)
- V            : Movie mode (press only)
- B            : Open battery door
- C            : Open card door
- F10          : Power down switch
- F1           : show this help

Setting BOOTDISK flag to 0
gtk initialization failed
[MPU] WARNING: forced shutdown.

For clean shutdown, please use 'Machine -> Power Down'
(or 'system_powerdown' in QEMU monitor.)

Tried to run the following just for testing:

Code Select Expand

gitk

Result:

Code Select Expand

application-specific initialization failed: couldn't connect to display ":0"
Error in startup script: couldn't connect to display ":0"
    while executing
"load /usr/lib/x86_64-linux-gnu/libtk8.6.so Tk"
    ("package ifneeded Tk 8.6.10" script)
    invoked from within
"package require Tk"
    (file "/usr/bin/gitk" line 10)

No idea what's the problem, I am using ubuntu distribution, WSL2 (it could be WSL version problem and how it handle DISPLAY?, I am not sure if your tests are done using WSL1 or WSL2).
Tested on Windows 10 x64 21H2 (19044.1645).

Also tried Windows CMD with bash command and Ubuntu cmd, same result. tried rebooting PC after I installed all packages and X server same thing.