OsX: Automate most of ML post task

[scrax]

[DRAFT]

I've added to my fork MLTools
a bash script that is tring to automate most of ML specific task, from setup to conversion of file (HDR, 422 focus stacks and so on).
For now I've just reorganized what was in prepare.Lion.sh and added the option to convert focus stack seq or 422 files.

At first start it will check for magiclantern sources in ~/magic-lantern then if not found it will look for homebrew and PIP and install them if needed, then it will install base system for converting 422 and merging files from ML.

If ~/magic-lantern exist it will search for a mounted card with ML to use for converting what there is, (TODO: if there is an EOS_DIGITAL card mounted but without ML it will install it and make bootable. Ask for confirmation before proceding)

If there is no card with ML it will ask for a folder containing some sort of workable files to process.
That folder can be passed as an option, for example:

Code Select Expand

MLTools /folder/with/422/files
it works also with files:

Code Select Expand

MLTools /your/file/to/convert.422

To install all things needed to compile use:

Code Select Expand

MLTools dev
"d" works as options too.
This will install the base system (if not yet installed), all what needed for compiling ML from source and clone the unified branch to your Home folder if not yet there.

Other options are:
remove (or r)
to uninstall what the install script installed with brew and pip (it will not uninstall all what was installed)
help (or h)
to show a little help page

If something goes wrong try also with

Code Select Expand

MLTools i
to force install of base system.

Working:
Installation base or developer;
422 conversion
HDR merge (with raw conversion)
Focus stack merge (with raw conversion, testing it)
Install ML to card if not yet done
prepare card (make bootable)

Options To be added:
Timelapse to mp4
HDRmovie
cropmark RLE compression

Other thing:
check for log and send them (where?)
split audio from movie?
erase all ml generated stuff on exit

LINK: MLTools

---------------------------------------------
old info:
---------------------------------------------
List of task to do to complete ML feat:
conversion:

• RAWtoJPG/TIFF conversion (when raw pic are used)
  
• 422toJPG conversion ADDED
• BMPtoRLE conversion (for compressing cropmarks)
  

merge:

• HDR photo merge ADDED
• Focus stack merge ADDED

video:

• HDR video rendering
  
• Timelapse video rendering
  

Other:
make card bootable

What we have from source working on all system (Win, Linux, OsX):
make-bootable.sh
422-JPG.py
convertrle.rb
enfuse_all.sh
focus and hdr scripts from camera

For HDR video workflow what we have from vimeo by b4rt for (Win, Linux) using Virtualdub/Avisynth:
Avisynth solution
and for Linux and OsX by Ro-Man using enfuse (or imagemagick) and mplayer/mcoder:
vid2hdr.sh

We need:

• a setup script to prepare the system like for the arm-toolchain
• raw conversion (for now tested using ufraw and imagemagick)
• timelapse rendering (with mplayer, using code from vid2hdr.sh)
• a way to find the correct file from the card and process them to a user selectable folder (not yet tried).
• a way to detect and start the script when a ML installed card is mounted (like checking for /Volumes/EOS_DIGITAL/ML/ or =/Volumes/EOS_DIGITAL/autoexec.bin)

[bart]

Hi Scrax,

I'm not very pleased with my whole avisynth workflow. It's not very easy to setup and the results are not very usable. Only 720@50p to 720@25p looks somewhat ok. Though resolution is a big problem. The fusion2 plugin does work but has a harsh cut between both exposures and crushed blacks.

For Mac OSX users I'd recommend the free Final Cut Pro X plugin from alex4d.
http://alex4d.wordpress.com/2012/01/03/ml-u-hdr-compose-fcpx-plug-in/

Results are just way better.

Same goes for Ginger HDR with so much control over the endresult. And I haven't tested the free merger yet.
http://www.gingerhdr.com/

[scrax]

I've made a quick test with the merger but not sure if it worked as expected, because i had no viedo only a wav and a lot of image files (hdr from the test.mov)
Will try blender before another try with this merger.

[scrax]

Updated first post with link to the script. Please report back issue, still working on it.

[scrax]

Update:
added HDR  conversion from JPG and RAW (that's maybe not a bug but ML will make a script for merging JPG, so if the HDR is done with RAW set or silent pic we have to first batch convert the raw images and then merge them)


Not tested with silent pic

[bart]

Looks nice and I'd like to try it out. But I don't have a mac.

[scrax]

Looks nice and I'd like to try it out. But I don't have a mac.

I think it can be easily ported to linux using something like apt-get for dependencies, for win no idea.

[scrax]

Update: I've cleaned up a bit the script and also added an option to install summon-arm from unified source code.

Now what does is:
If there is no option set, it will scan a EOS DIGITAL mounted card with autoexec.bin on it (if present) to find if there are magiclantern scripts, 422 (and RAW files in /Volumes/EOS_DIGITAL/DCIM/100CANON (default folder).
For each one found it will convert 422 to jpg or run the scripts accordingly to the file type.

If there is an option set it will perform that option (and then exit.)
Options are:
i or install : To install base tools for converting and merging ML files
d or dev : To install base tools and dependencies for compiling Magic Lantern on OsX
sa or summonarm : To install base tools, dev tools the arm toolchain for compiling ML
r or remove : To remove packets installed with Homebrew and PIP
h or help : To display the guide
Converting of raw files is disabled when scanning folder, but it will occur with raw in scripts before launching it.
Before starting 'remove' options it will ask to confirm.

NOTE: I'm learning how to embed scripts in an app bundle. So searching for the best way to make it functional but simple.
For now i've tried with this script and it search the card for ml files (if found) when double clicked;perform the search in a dropped folder or just convert a dropped file.
More check are needed before making it an app, to avoid errors.

[scrax]

Updated to support new align HDR scripts

[scrax]

I'm fixing it and adding basic support for linux, I've made a fresh osx install and summon.arm is running now to prepare the toolchain.
After downloading command_line_tools_for_xcode_june_2012.dmg (154,6MB) from Apple here (is free but needs registration first) instead of 1.83GB for XCode I'm doing all with:

Code Select Expand

MLTools sa

The script installed Homebrew and PIP,
then installed Base dependencies and Dev dependencies:
BREW:
binutils, gmp, mpfr, mpc, libmpc, mercurial, wget, imagemagick, ufraw, mplayer, poppler"
PIP:
numpy, PIL, matplotlib, docutils"
then it will start summon-arm from ml source directory moving patches to your home folder until the toolchain is ready (to fix the address in the script)
once finished it will remove all the downloaded stuff and you are ready to start compiling ML on mac.

on linux since the only option of now is to install dev and doc dependencies (like with prepare.ubuntu.sh) it works without options too (I hope not tested!).

NOTE: If you already have Xcode on Lion you can download the command line tools from Xcode preferences, but if you want to remove Xcode and use only the command line tools you can use this command before installing the tools:

Code Select Expand

sudo /Developer/Library/uninstall-devtools --mode=all

[scrax]

I've just added all python stuff needed for the arm-console, but not tested.
Also added hugin to the things installed cause is the quickest way to have enblend and align_image_stack i've found so far, and it's cross platform.

this is what PIP installs now:
numpy
PIL
matplotlib
docutils
wxPython
scipy
ipython==0.10
pygraphviz
sympy
pydot
yappi
Cheetah
easygui
profilestats
ahocorasick
pyprof2calltree
pyparsing

[bart]

Hi Scrax,

I made a wiki for the 3rd party software where you can present and maintain your software.
http://wiki.magiclantern.fm/3rd_party_software

[scrax]

Thank's b4rt I'll need to make more test and decision on how to package this, for now it's in ml source like prepare.ubuntu.sh and prepare.Lion.sh.
Probably will leave prepare.Lion.sh (but needs to be updated) there and make an app for MLTools (without all the preparation options).

The app will works with drag-n-drop and can be started when a ML_card is inserted for converting files from ML cards, or can be added to the file explorer and then from any folder drag n drop file, scripts, or folder to convert.

I'm not yet sure if it will support HDR video, it will be so slow and crapy that other workflows are surely better then a default script...
Already RAW pic conversion is not so good, but i use it for making automatic preview HDR.

[scrax]

Little update I've made the first MLTools.app but it has dependencies that if not installed will make it not working.
I want to integrate into it also prepare.Lion.sh to install what needed at first start before releasing

[scrax]

I think I have something to test if someone interested please post me a pm or here

[Marsu42]

LINK: MLTools

Link is dead?

[scrax]

yep, i'm packaging a new version. Just making some test still..

[scrax]

Open a new more appropriate topic here: http://www.magiclantern.fm/forum/index.php?topic=3150.0
any help in working on this is really appreciated