Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: petabyte on July 18, 2021, 01:14:50 AM

Title: MLinstall - USB Installer tool for ML
Post by: petabyte on July 18, 2021, 01:14:50 AM
MLinstall is a Windows/Linux utility that makes it easier to install Magic Lantern.

Read more: https://danielc.dev/mlinstall (https://danielc.dev/mlinstall)
Download: https://github.com/petabyt/mlinstall/releases (https://github.com/petabyt/mlinstall/releases)

Old post:
I've been working on a utility that should help with installing Magic Lantern.
Source code: https://github.com/petabyt/mlinstall (https://github.com/petabyt/mlinstall)

It currently can:

- Run Canon event procedures:
Based on some research in https://github.com/petabyt/sequoia-ptpy, I've
gotten Canon's `0x9052` PTP command to work properly. This allows for
commands like "EnableBootDisk" and "DisableBootDisk" to be run via PTP.
I think this command is standard on almost all Canon cameras, so it
could possibly replace custom firmware files. (?)

- Card Tools:
I've also added a card flag enabler (bootdisk, canon basic), and an
option to disable them without reformatting card.
(just writes an underscore on the first character of each)
Title: Re: Magic Lantern USB Installation Tools
Post by: Walter Schulz on July 18, 2021, 04:14:32 AM
Thanks!
Avoiding messing with custom firmware files and thus avoiding legal issues (without the need to access via UART) will have a big impact IMO!

Any plans to make it work on macOS?
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on July 18, 2021, 04:20:32 AM
I think it should be pretty easy to get it to work. But the only Mac I have is an old iMac G4, which is probably too old.

The old python version (https://github.com/petabyt/mlinstall/tree/python-stable) should work on it.
Title: Re: Magic Lantern USB Installation Tools
Post by: Walter Schulz on July 18, 2021, 02:18:35 PM
No hurry!

Another one: EOS M is not supported by gphoto2, Tornado EOS, EOS Webcam Utility. Has anyone tested your tools with EOS M?
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on July 18, 2021, 06:38:30 PM
Not that I know of. I guess the safe way to check would be looking for the string "ExecuteEventProc" in a ram dump.
Title: Re: Magic Lantern USB Installation Tools
Post by: Ant123 on July 19, 2021, 01:29:58 PM
Does this mean that ptp command 0x9052 (36946) is not supported?
Quotepython3 ./print_device_info.py
Container:
    StandardVersion = 100
    VendorExtensionID = Microsoft
    VendorExtensionVersion = 100
    VendorExtensionDesc =
    FunctionalMode = 0
    OperationsSupported = ['GetDevicePropDesc', 'GetDevicePropValue', 'SetDevicePropValue', 'ResetDevicePropValue', 'GetDeviceInfo', 'OpenSession', 'CloseSession', 'CheckEvent', 36956, 36957, 'ChangeUSBProtocol', 'GetStorageIDs', 'GetStorageInfo', 'GetNumObjects', 'GetObjectHandles', 'GetObjectInfo', 'GetObject', 'GetThumb', 'GetPartialObject', 'SendObjectInfo', 'SendObject', 'DeleteObject', 'FormatStore', 'SetObjectProtection', 'GetObjectSize', 'GetObjectInfoEx', 'GetPartialObjectEx', 'GetObjectAttributes', 'SendPartialObject', 'GetObjectHandleByName', 'SetObjectTime', 'SetObjectArchive', 36940, 'SendObjectInfoByPath', 'SendObjectByPath', 36920, 36921, 36922, 36923, 36939, 36960, 36962, 38913, 38914, 38915, 38916, 38917, 'EOSGetEvent', 'EOSGetStorageIDs', 'EOSGetStorageInfo', 'EOSGetObjectInfo', 'EOSDeleteObject', 'EOSFormatStore', 'EOSGetPartialObject', 'EOSGetObjectInfoEx', 'EOSGetThumbEx', 'EOSSetObjectAttributes', 'EOSTransferComplete', 'EOSCancelTransfer', 37164, 37170, 37173, 37184, 37185, 37187, 'EOSPCHDDCapacity', 37183, 'EOSSetEventMode', 'EOSSetUILock', 'EOSResetUILock', 'EOSKeepDeviceOn', 37181, 37174, 37175, 'EOSSetRemoteMode', 'EOSGetViewFinderImage', 'EOSRemoteReleaseOn', 'EOSRemoteReleaseOff', 'EOSZoom', 'EOSZoomPosition', 'EOSDoAf', 'EOSAfCancel', 'EOSDriveLens', 37211, 36911, 'EOSSetDevicePropValueEx', 'EOSRequestDevicePropValue', 37250, 37251, 37252, 37253, 36944, 36945]
    EventsSupported = ['CancelTransaction', 'ObjectAdded', 'ObjectRemoved', 'StoreAdded', 'StoreRemoved', 'DevicePropChanged', 'ObjectInfoChanged', 'DeviceInfoChanged', 'RequestObjectTransfer', 'StoreFull', 'DeviceReset', 'StorageInfoChanged', 'UnreportedStatus', 49153, 49157, 49162, 49409]
    DevicePropertiesSupported = [53317, 53322, 53294, 53295, 'BatteryLevel', 53250, 'ViewfinderMode', 'UnixTime', 53319, 53318, 53296, 53321, 'CameraModel', 'CameraOwner', 'FlashMemory', 53328, 53329, 53330, 53331, 53332, 53335, 54274, 54278, 54279, 54019]
    CaptureFormats = ['EXIF_JPEG']
    ImageFormats = ['Association', 'Script', 'DPOF', 'WAV', 'EXIF_JPEG', 'UndefinedImage', 'CRW3', 47490, 45317, 48897]
    Manufacturer = Canon Inc.
    Model = Canon EOS M3
    DeviceVersion = 3-14.0.1.0
    SerialNumber = *
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on July 19, 2021, 07:21:42 PM
I guess not, assuming Canon didin't change the command ID or anything.
Title: Re: Magic Lantern USB Installation Tools
Post by: Ant123 on July 19, 2021, 07:58:08 PM
I already found that the command 0x9052 becomes available after calling the command 0x9050 three times.
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on July 19, 2021, 07:59:45 PM
Interesting, do you know why?
Title: Re: Magic Lantern USB Installation Tools
Post by: Ant123 on July 19, 2021, 08:15:18 PM
Just because of Canon's firmware:

uint handle_PTP_OC_0x9050
               (undefined4 param_1,undefined4 *param_2,undefined4 param_3,undefined4 param_4,
               undefined4 param_5)

{
  uint uVar1;
  uint uVar2;
  uint uVar3;
  uint uVar4;
  uint uVar5;
  uint uVar6;
  uint uVar7;
  undefined2 local_38 [2];
  undefined4 local_34;
  undefined4 uStack48;
  undefined4 local_2c;
 
  uVar7 = 0;
  j_bzero(local_38,0x24);
  uStack48 = param_5;
  local_2c = 0;
  DAT_0000ef68 += 1;
  local_34 = param_4;
  if (DAT_0000ef68 == 3) {
    uVar1 = add_ptp_handler(&DAT_00009052,handle_PTP_OC_0x9052 + 1,0);
    uVar2 = add_ptp_handler(&DAT_00009053,handle_PTP_OC_0x9053 + 1,0);
    uVar3 = add_ptp_handler(&DAT_00009057,handle_PTP_OC_0x9057 + 1,0);
    uVar4 = add_ptp_handler(&DAT_00009058,handle_PTP_OC_0x9058 + 1,0);
    uVar5 = add_ptp_handler(&DAT_00009059,handle_PTP_OC_0x9059 + 1,0);
    uVar6 = add_ptp_handler(&DAT_0000905a,handle_PTP_OC_0x905a + 1,0);
    uVar7 = add_ptp_handler(&DAT_0000905b,handle_PTP_OC_0x905b + 1,0);
    uVar7 |= uVar6 | uVar5 | uVar4 | uVar3 | uVar2 | uVar1;
    DAT_0000ef64 = 1;
  }
  if ((uVar7 & 1) == 0) {
    local_38[0] = 0x2001;
    (*(code *)param_2[3])(*param_2,local_38);
  }
  else {
    local_38[0] = 0x201f;
    (*(code *)param_2[3])(*param_2,local_38);
    uVar7 = 1;
  }
  return uVar7;
}

Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on July 19, 2021, 08:24:19 PM
Quote from: Ant123 on July 19, 2021, 07:58:08 PM
I already found that the command 0x9052 becomes available after calling the command 0x9050 three times.

Have you tried calling 0x9052 after doing that?
Title: Re: Magic Lantern USB Installation Tools
Post by: Ant123 on July 19, 2021, 08:28:09 PM
Yes. It works.
Title: Re: Magic Lantern USB Installation Tools
Post by: Ant123 on July 19, 2021, 09:21:11 PM
Is there the way to run event procedures with arguments?
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on July 19, 2021, 09:36:06 PM
Yes, I think you can send integer arguments simply via the standard PTP arguments.

I've set up https://github.com/petabyt/sequoia-ptpy to work like this:
`camera.eos_run_command("EnableBootDisk", [1, 2, 3])`
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on August 04, 2021, 07:30:25 PM
I've added a few more features to the app:
- Redo UI with a GTK "notebook"
- Add "Make card scriptable"
- Add "Make card un-scriptable"
- Add credits and license
- Add button hints
- Better error messages

https://github.com/petabyt/mlinstall/releases/tag/0.1.3 (https://github.com/petabyt/mlinstall/releases/tag/0.1.3)
Title: Re: Magic Lantern USB Installation Tools
Post by: shadowlab on August 08, 2021, 02:25:10 AM
Big thanks for your help. It saves the lazy hardware guys (me) from having to crack open their cameras to get the full features of their devices.

Windows 10 wouldn't work correctly, as it doesn't install a standard PTP driver for my 1600D, so I booted up into a LiveUSB version of Ubuntu 21 and ran your linux version succesfully.
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on August 08, 2021, 09:45:16 PM
I was able to reproduce it on Windows 10. Looking for a solution...
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on August 09, 2021, 03:44:41 AM
Apparently I forgot that I had installed libusb in my Windows 7 testing VM, so
I wasn't aware of this issue.

A quick solution is to download Zadig (https://zadig.akeo.ie/) and replace WinUSB with
libusb-win32. Here's a GIF animation showing exactly what to do:
https://petabyt.github.io/mlinstall/MANUAL#no-ptpusb-device-found

If this doesn't work, please let me know.
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on August 26, 2021, 04:41:56 PM
Experiment for a "one click install": https://diode.zone/w/5BDpMTeBhgZ4mEVXu6QRQj
Plug in camera + SD card, and click a button to fully install Magic Lantern.

PTP is used to detect model/firmware version, and the correct Magic Lantern build
is unzipped into the SD card. The bootdisk flag and the card flags are written as well.

Also see the Github issue: https://github.com/petabyt/mlinstall/issues/2

This is just a proof-of-concept to see if it would work. It won't be released any time soon, if
it is released at all. Feedback/ideas for this are welcome.
Title: Re: Magic Lantern USB Installation Tools
Post by: Stewey on September 14, 2021, 06:37:01 PM
I like this one click idea!
Title: Re: Magic Lantern USB Installation Tools
Post by: theBilalFakhouri on September 15, 2021, 06:05:14 AM
Thanks petabyte, Enabling/Disabling Boot Disk works fine on 700D/5D3, also tested custom event procedure and it works fine on both cameras

I had "No PTP/USB device found" issue on Windows 10, and doing the following fixes the problem:

Quote from: petabyte on August 09, 2021, 03:44:41 AM
A quick solution is to download Zadig (https://zadig.akeo.ie/) and replace WinUSB with
libusb-win32. Here's a GIF animation showing exactly what to do:
https://petabyt.github.io/mlinstall/MANUAL#no-ptpusb-device-found

Please add this as note for Windows users in OP :D

Great work!
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on September 15, 2021, 04:34:27 PM
Sure. Although I would think that the MANUAL.txt in the zip file and the note in the release description would be enough...

Thanks for testing!
Title: Re: Magic Lantern USB Installation Tools
Post by: petabyte on September 23, 2021, 04:33:18 PM
Version 0.7 released.

https://github.com/petabyt/mlinstall/releases/tag/0.7

- Added an untested activator for 0x9052 as per Ant123's findings (https://www.magiclantern.fm/forum/index.php?topic=26162.msg236146#msg236146)
Still not sure what models require it, so it's just a button for now.

- Added a "module store"
I thought this would be a cool idea. You can plug the SD card into your computer and install modules, just like an app store on a smartphone. Here's a demonstration: https://diode.zone/w/4u1GYt88LP5h9t53C1fWNE
Title: Re: Magic Lantern USB Installation Tools
Post by: srsa on February 07, 2022, 12:02:35 AM
Quote from: Ant123 on July 19, 2021, 09:21:11 PM
Is there the way to run event procedures with arguments?
Yes. (https://chdk.setepontos.com/index.php?topic=4338.msg147738#msg147738) Thanks to petabyte for publishing the first usage details.
Title: Re: Magic Lantern USB Installation Tool
Post by: petabyte on February 08, 2022, 04:22:31 AM
Version 0.8.1 Released.

https://github.com/petabyt/mlinstall/releases/tag/0.8.1

I've implemented srsa_4c's findings in the mlinstall EvProc parser.
See https://chdk.setepontos.com/index.php?topic=4338.msg147738#msg147738

It includes a basic command parser. It accepts plain text (commands), 
strings (between quotes), and numbers (base 10 and hex).

Example:
FooBar "Hello, World"
foo_bar 123 0x123

Use at your own risk.
Title: Re: Magic Lantern USB Installation Tool
Post by: petabyte on June 12, 2022, 09:22:05 PM
Pre-release for 0.9:
Implemented ExFAT card flag writing (for Linux only)
Any testing and feedback is appreciated.

https://github.com/petabyt/mlinstall/releases/tag/0.9 (https://github.com/petabyt/mlinstall/releases/tag/0.9)
Title: Re: Magic Lantern USB Installation Tool
Post by: petabyte on July 09, 2022, 04:26:41 AM
Version 0.9.1 has been released.
https://github.com/petabyt/mlinstall/releases/tag/0.9.1
- Improved error messages and handling
- Added bound checks wherever possible
- Added ExFAT support
- Made drive code a lot better

The next release will be the final (and hopefully stable) 0.1.0.
Title: Re: Magic Lantern USB Installation Tool
Post by: Pelican on July 23, 2022, 12:40:50 PM
Why not 1.0.0?
Title: Re: Magic Lantern USB Installation Tool
Post by: petabyte on July 23, 2022, 09:36:30 PM
I wanted to wait a while to see if anybody reports any bugs before I call it "officially stable".
In software 1.0.0 is generally considered to be a stable release.
Title: Re: Magic Lantern USB Installation Tool
Post by: petabyte on August 22, 2022, 04:40:19 PM
For Mac users, or people who want a simpler version of mlinstall:
Here's a standalone python script to enable bootdisk flag:
https://github.com/petabyt/mlinstall/blob/master/boot.py
Untested.
In order to run:
pip3 install ptpy
python3 boot.py EnableBootDisk
Title: Re: Magic Lantern USB Installation Tool
Post by: petabyte on December 13, 2022, 10:34:03 PM
Release 0.9.3
Now recommend win32 libusb filter installer over Zadig
https://github.com/petabyt/mlinstall/releases/tag/0.9.3

I spent a while working on a Windows driver for PTP, and finished it, but it's pretty much useless. I wrote it
with Wia (Windows image acquisition), and the win32 API seems to build a tree of all files on the SD card
when the device is opened, which takes around 5 minutes if your card if filled up. Other than that it works perfectly.
Title: Re: Magic Lantern USB Installation Tool
Post by: petabyte on January 29, 2023, 03:33:51 AM
2023 Update:
I'm currently slowly working on a DLL that will help me interface with PTP devices on Windows natively.
I spent a lot of time writing code that will interface with the ancient WIA driver, but I had to scrap it because it was too slow.
This time, I'm interfacing with WPD (Windows Portable Devices). Like Wia, the driver is completely different compared to something like libusb.
WPD requires sending the command opcode, parameters, and data separately, letting Windows handle transaction IDs, and determining whether the response will have a data phase or not before sending the command.
Title: Re: Magic Lantern USB Installation Tool
Post by: petabyte on February 27, 2023, 04:17:40 AM
Finished porting mlinstall to camlib/WPD. It's a very early test, still needs some work. It only reads the first device found, so it might try and open a phone or USB drive.
https://github.com/petabyt/mlinstall/releases/tag/0.9.4
This means that eventually there should be no more libusb/zadig/filter nonsense on Windows. Everything should work out of the box. On Linux, it should function the same.

For the 1.0.0 release, hoping to implement shutter count reading, and the ability to update to any firmware version over USB.
Title: Re: ML USB Installation Tool
Post by: petabyte on April 17, 2023, 11:54:44 PM
mlinstall 1.0.0 is finally released.
Github release: https://github.com/petabyt/mlinstall/releases/tag/1.0.0

I knew from the start how painful all the libusb nonsense was, but there was no other option at the time. This release deletes all the PTP/USB backend and replaces it with a library I wrote from scratch.
Native Windows support is now available without any modification to the system drivers. This means you won't install Zadig or libusb if you're on Windows.
Title: Re: ML USB Installation Tool
Post by: iaburn on April 18, 2023, 06:50:49 AM
Sounds great! I'll give it a try later  :D
Title: Re: ML USB Installation Tool
Post by: iaburn on May 18, 2023, 05:37:22 PM
I don't need it very often, but it comes in very handy when I need it  :D Great tool  8)
Title: Re: ML USB Installation Tool
Post by: petabyte on May 18, 2023, 09:20:59 PM
Quote from: iaburn on May 18, 2023, 05:37:22 PMI don't need it very often, but it comes in very handy when I need it  :D Great tool  8)
Thanks. I think for now the entire tool is stable and won't get any more major changes. Next updates will probably be a MacOS port, and maybe localization.
MacOS should be pretty easy, camlib is fairly stable and has been tested on Macs, so it should just be a matter of some makefile tweaks.
Title: Re: MLinstall - USB Installer tool for ML
Post by: petabyte on January 16, 2024, 05:07:54 AM
MLinstall v1.1.0 released: https://github.com/petabyt/mlinstall/releases/tag/1.1.0

Main improvements:
- MacOS port
- Added shutter counter
- Shows the camera's firmware build version
- Switched to native UI (Winforms/GTK3/Cocoa)
- Wording improvements
- misc bug fixes over the past year
Title: Re: MLinstall - USB Installer tool for ML
Post by: Danne on January 16, 2024, 07:54:03 AM
Didn't check this tool yet.
Regarding sandboxing issues on Mac I found quarantine routine working better for most cases.
In terminal:
xattr -cr drag/app/to/terminal
Press enter.