Thought I'd share with you a small utility for Windows I've written a few months ago in C# in case it's useful to you guys :)
This is a simple light-weight app which allows you to offload footage from your memory card to your PC. I've found it quite useful as a backup for on-location shooting, and carry it on a small netbook.
Here's a break-down of the features:
- Copy files from memory card to a primary & secondary (optional) location of your choice
- Filter file types to copy (currently MOV, THM, JPG, CR2, WAV, MP3)
- Renaming video files to creation date & time (as a timecode reference)
- Verification of data after copying (by comparing exact file sizes)
- Storing up to 3 profiles (video, audio & photo profiles)
- Quick profile switching using CTRL+1/2/3 keyboard shortcuts
- Copying all Magic Lantern files, only the configuration file (magic.cfg) or the config and hidden.cfg (introduced in ML 2.3) from a pre-defined location
- Generating a shot list based on data obtained from LOG files generated by Magic Lantern, which includes the date & time, lens used, shutter speed, aperture, ISO, focal length, white balance, picture style, FPS, HDR and bitrate
- Exporting shot list to Excel spreadsheet
- Transcoding of HDR videos captured through ML
- Argument passing for running through scripts or batch files
Screenshot:
(http://malcolmdebono.com/apps/hdslross.png)
Download latest version (see posts below for changelog):
http://malcolmdebono.com/apps/HDSLRoffloader_0.6.1.zip (http://malcolmdebono.com/apps/HDSLRoffloader_0.6.1.zip)
Let me know if you have any ideas for new features or encounter any bugs.
Hi.
I just found your tool while strolling through the forum, can't test it now because I'm on work.
It sounds definitely useful, especially all the options to treat the files.
I already have a feature request (you asked for it): Do you know the tool "QTChange"? I use it to get real TC into the files. Is there a possibility to have that functionality in your tool?
Greetings,
Schloime
Hey Schloime,
I will definitely look into it. Apparently QtChange doesn't support command-line interface, so for now I don't think I can integrate it. I will check if there are any other libraries which can do the same thing. So far the closest I found is a modified version of ffmpeg (with added broadcasting features), and according to its documentation, it can write quicktime files containing timecode tracks.
I'll see if I can work something out in the coming days :)
Malcolm
maybe this one is cool for your timecode
http://mediainfo.sourceforge.net/en
MediaInfo only shows you timecode and other video properties. It cannot modify the TC track like QTchange can.
I've done some tests with FFmpeg and it can add a TC to the video file itself, but only by converting the video. What I really like in QTchange is that it saves the TC track in the MOV file without any conversions, so I'm still looking for a better solution.
Wow, cool tool, very useful!
Feature pester request: Any chance of adding a transcode video HDR files (at transfer time) into seperate light/dark files? If using ffmpeg would be nice to choose between frame sequences or duped media at frame rate = half.
looks very usefull. TnX alot!
some questions:
is verification time\size based or CRC?
are both targets verified?
Quote from: 3pointedit on July 04, 2012, 02:58:05 AM
Feature pester request: Any chance of adding a transcode video HDR files (at transfer time) into seperate light/dark files? If using ffmpeg would be nice to choose between frame sequences or duped media at frame rate = half.
I think I can do that with ffmpeg (convert video to image sequence, and split images based on even/odd frames).
Do HDR videos have a different name (in comparison with normal videos)? This can be used to determine whether a video is HDR or not.Just checked and HDR videos are named the same format, so I don't know how I can check if a video is HDR or not automatically. The only other option I can think of is for the user to select which videos are HDR before copying starts (so that transcoding starts straight afterwards without any user intervention).
Update: Another (better) option is to use movie logging, but that can only be used if movie logging was enabled (obviously). The HDR settings used are logged when an HDR video is captured, otherwise this field is missing. I simply have to search for HDR in this file to determine if the video is HDR or not :)
Quote from: GershberG on July 27, 2012, 09:56:20 PM
some questions:
is verification time\size based or CRC?
are both targets verified?
Verification works by checking that the byte size of the copied file matches that of the original file.
Version 0.4
HDRSLR Offloader now supports transcoding HDR videos while importing :)
The following steps can be done automatically:
- Extract the audio track from file (done through ffmpeg - included)
- Extract and sort even and odd frames from HDR videos (also done through ffmpeg)
- Process HDR frames with Enfuse (included) or with Photomatix (user must specify location of PhotomatixCL.exe from Options window)
- Save processed frames back to video (framerate and size can also be customized; done through ffmpeg)
Note: HDR videos are detected through the log file generated by ML, so please enable this option from ML menu before capturing
(http://malcolmdebono.com/apps/hdslroffloader_hdr_features.jpg)
Download link:
http://malcolmdebono.com/apps/HDSLRoffloader_0.4.zip (http://malcolmdebono.com/apps/HDSLRoffloader_0.4.zip)
Now thats a nice new feature. Will try this tomorrow. One feature I was just thinking about would be a command-line option so that one could auto-run this whenever a card is inserted.
Nice application but movie log isn't available for the 50D.
Can you add an option for logless files? :)
Wow Malcom, this is amazing, i will give it immediately a try!! great work indeed
Quote from: Michael Zöller on July 30, 2012, 11:18:32 PM
Now thats a nice new feature. Will try this tomorrow. One feature I was just thinking about would be a command-line option so that one could auto-run this whenever a card is inserted.
That's a cool feature! I'll implement argument passing so that all the required info can be passed through cmd.
Quote from: ilguercio on July 30, 2012, 11:41:42 PM
Nice application but movie log isn't available for the 50D.
Can you add an option for logless files? :)
I'll try to find some other way :) Only other option I can think of is opening a file chooser dialogue so that you can select which videos are HDR manually. This can also be used to process HDR videos without the importing part.
Quote from: screamer on July 31, 2012, 12:07:15 AM
Wow Malcom, this is amazing, i will give it immediately a try!! great work indeed
Thanks :)
Version 0.5
- Manual selection of HDR videos (can be used both during importing stage, or from the Tools menu)
- Argument passing
Arguments supported are:
-p1 / -p2 / -p3 Specify which profile will be loaded automatically when the app starts
-start Automatically starts copying when app starts
Example:
"HDSLR Offloader" -p1 -start
This command starts the app, loads the Video profile, and starts the copying process automatically
To use the profiles, you will first need to have them saved by starting the app, specify which options will be used and the paths, and save the profile to one of the 3 slots through the File menu. This only needs to be done once, as you can then load the profile either manually through the File menu or the shortcut keys (CTRL + 1/2/3), or through the arguments.
Download link:
http://malcolmdebono.com/apps/HDSLRoffloader_0.5.zip
Enjoy :)
Thanks a lot Malcolm ;)
I really appreciate your effort and contribution to the community.
You're welcome! Thanks for your kind words :)
Thanks for sharing your work and even implementing ideas from the community (I think the ml community is really maturing :p)!
A healthy community is all about sharing & helping. I wanted to build this app mainly for personal use, but if it can help others too, why not? :)
Quote from: Malcolm Debono on July 31, 2012, 03:55:16 PM
A healthy community is all about sharing & helping. I wanted to build this app mainly for personal use, but if it can help others too, why not? :)
Looks great tool :) i'm always impressed by peoples with knowledge of C and other languages - ehh.. :)
can i hope for Mac OS X application - or this stays only for Windows? ??? :)
Quote from: cosmo.lv on August 01, 2012, 02:50:17 AM
Looks great tool :) i'm always impressed by peoples with knowledge of C and other languages - ehh.. :)
can i hope for Mac OS X application - or this stays only for Windows? ??? :)
Thanks!
This is written in C#, which is a Windows language. There might be some solutions to compile this for Mac using tools like Mono. I'll see if I can manage to try it out when I find some free time :)
Quote from: Malcolm Debono on August 01, 2012, 12:41:28 PM
Thanks!
This is written in C#, which is a Windows language. There might be some solutions to compile this for Mac using tools like Mono. I'll see if I can manage to try it out when I find some free time :)
That's will be great! :)
Unfortunately it is next to impossible to port it for Mac (would be better to just write it from ground up in another language). The way HDR transcoding works is by passing parameters to external apps such as ffmpeg, enfuse and photomatix through command-line, which would need to be changed completely to work on Macs. There are also some other dependencies such as Microsoft Office, which is used for exporting a spreadsheet file of all the data in the LOG files.
One more thing I wanted to add is that the ML features within the app (mainly copying config file or all ML folder to card) currently do not support ML 2.3 (since the file structure was changed). An updated version will be released soon which offers support for both ML 2.3 and previous version (2.2).
Version 0.6
- Support for ML 2.3
- Fixed shot list generator (now fully compatible with log files generated by ML 2.3)
- Minor bug fixes
It is now easier than ever to keep your cards synchronised. You can now keep a copy of all your ML files (including the standard config and the hidden items config) on your PC, and copy either all the ML files or only the configs to your memory card with the click of a button - useful when you have many different memory cards and need to have the same configs on all cards!
The shot list generator has also been updated and is fully compatible with log files generated by ML 2.3, which now includes HDR and bitrate information. The list can also be exported as a spreadsheet.
Download link:
http://malcolmdebono.com/apps/HDSLRoffloader_0.6.zip
Enjoy :)
I really appreciate your development efforts and support.
Do you ever get any sleep?
I think I need to tr this tool. Looks very interesting.
Quote from: hd1roam on August 11, 2012, 06:20:55 PM
I really appreciate your development efforts and support.
Do you ever get any sleep?
Rest assured that I do :D
This app isn't something that I spend a lot of work on. I only see what needs to be done (both stuff that I require, and user requests that can be implemented) and try to get it done in my free time (which I currently have plenty of since I'm on holidays!).
Quote from: Malcolm Debono on August 02, 2012, 03:27:49 PM
Unfortunately it is next to impossible to port it for Mac (would be better to just write it from ground up in another language). The way HDR transcoding works is by passing parameters to external apps such as ffmpeg, enfuse and photomatix through command-line, which would need to be changed completely to work on Macs.
all those three works on mac too :)
Osx user, MLTools is for mac but it is still just a script because I had no feedback so far, so i've stopped adding things since for me it works and starts automatically when card is inserted.
If feedback are positive i'll bundle it in an app, but without testers i'll not make something that maybe works only on my particular configuration.
@ Malcolm if you want we can try to work together on an osx app
@ scrax
I'll be more than happy to help you out, but I don't have any knowledge in mac programming (as far as I know it uses objective c). Had I known that this app would become 'popular', I would have based it on a more universal language (maybe java), however as it is it requires a complete re-work for it to work on Mac.
If you need to have a look at the code, let me know (it's in C#).
For now i'm just interested in the command line code you use for you conversion, so to try it on mac and compare result with what i have.
So far I can have only an automated process without a lot of user interaction to test the basic functions, but i'll add more options when those thing are tested working ok.
In those day I'm really busy so don't worry if have ont time right now.
This needs g3gg0's >4gb movie repairer...
Hey guys, I'll get back to you next week as I'm currently abroad!
This is awesome! I'm currently working on a similar utility in python to provide a nice front end to some of the timelapse scripts I've seen around here for deflickr and also to add a GUI to set keyframes for some simple ken burns style motion during timelapses. It too uses ffmpeg, imagemagick and other tools that should be very portable.
I think It would be extremely beneficial to develop a GPL'd magic lantern companion software that would support multiple OS, and allow users to simply log footage, create timelapses, simple HDR video and provide extensisibility to take advantage of new ML features as they come out. I'd be more than willing to help support a project like that in any way.
Quote from: jordancolburn on September 13, 2012, 06:14:07 PM
This is awesome! I'm currently working on a similar utility in python to provide a nice front end to some of the timelapse scripts I've seen around here for deflickr and also to add a GUI to set keyframes for some simple ken burns style motion during timelapses. It too uses ffmpeg, imagemagick and other tools that should be very portable.
I think It would be extremely beneficial to develop a GPL'd magic lantern companion software that would support multiple OS, and allow users to simply log footage, create timelapses, simple HDR video and provide extensisibility to take advantage of new ML features as they come out. I'd be more than willing to help support a project like that in any way.
+1 for this, at least all the basic stuff thata ML user should have on his PC.
Quote from: jordancolburn on September 13, 2012, 06:14:07 PM
This is awesome! I'm currently working on a similar utility in python to provide a nice front end to some of the timelapse scripts I've seen around here for deflickr and also to add a GUI to set keyframes for some simple ken burns style motion during timelapses. It too uses ffmpeg, imagemagick and other tools that should be very portable.
I think It would be extremely beneficial to develop a GPL'd magic lantern companion software that would support multiple OS, and allow users to simply log footage, create timelapses, simple HDR video and provide extensisibility to take advantage of new ML features as they come out. I'd be more than willing to help support a project like that in any way.
Such a tool would definitely be useful! Unfortunately I'm not quite familiar with any cross-platform language (except for Java), which is why I started this app (which was very basic at first, and was only intended as a simple personal project!) in C#. If it can be useful, I'd be more than willing to share any code from this app if anyone wants to work on such a tool :)
Quote from: Malcolm Debono on September 13, 2012, 06:36:03 PM
Such a tool would definitely be useful! Unfortunately I'm not quite familiar with any cross-platform language (except for Java), which is why I started this app (which was very basic at first, and was only intended as a simple personal project!) in C#. If it can be useful, I'd be more than willing to share any code from this app if anyone wants to work on such a tool :)
That would be great! I'm an electrical engineer by degree but working as a newbie software developer so I'm still learning a lot.
AFAIK Python tends to be pretty portable (AKA, I can run my scripts on any linux/mac/win computer with all the library dependencies), I'm just not sure how to go about including all dependencies and packing it up really nicely for the different platforms.
It seems like most of the real complicated functions in your utility are calling other programs and the c# just provides the GUI and setting up batches/file operations, right?
Yeah, I'm only using it for the GUI interface and batch operations, which should be easier to convert to another language. However we still need to make sure that all the libraries used are supported by multiple OSs.
Cool, well, I didn't mean to completely derail your original thread because the current app really useful! so should we start a separate thread to gauge potential interest and talk about how best to get started?
No worries! Having a cross-platform app is the way to go since many users use Macs too. Feel free to open a new topic since it's cleaner that way.
@Malcolm
Thankyou for creating this software.
I just test it, and I want to set encoding options.
Because, default output.mov is very heavy when I edit it.
So I want to choose mpeg2-hd or some low compression encoding format for low PC CPU usage.
I think,
Add one more edit box in HDR option tab , then describe full system() argument .
Of course , add new event handler in another edit box , and the new one is realtime update by added event handler.
Finally, user is able to customize new editbox.
It's better to make a full GUI encoding options support, but I know it's a really hard work. So it's a landing point, I think.
Please consider it.
P.S. I'm only using a VisualC++ for my work. I never use C# .......
Thanks miyake! I'll see what I can do in the coming days! Shouldn't be too much work.
Version 0.6.1
- Added support for customizing ffmpeg parameters for output video, so you can use different output formats such as MP4 or MPG
The PDF includes a usage example and an external link with more info on ffmpeg commands!
Download link:
http://malcolmdebono.com/apps/HDSLRoffloader_0.6.1.zip
Thanks Malcolm.
It's very good for me.
my Extra param is
-q:v 1 -vcodec mpeg2video
And multiplexing container is
m2v
Mpeg2 HD and ALL-I is really right wait for CPU.
Welcome! Glad it works for you :)
Malcolm I've worked on MLtools those days and I've managed to add some GUI interaction to the script.
It does most of the thing HDSLR does but in sort of wizard way. So far I have problem to generate FST script with align_image_stack but maybe is the ML script that don't works, more test needed.
I'm still curious to take a look at the script that you use, maybe i can copy something ;)
thanks :)
Sent you a PM :)
If you like to add timecode and reelname to the files then ffmbc can do that with command line switches:
-timecode <Timecode> -metadata reel_name=<ReelName>
The <Timecode> is in this format 00:00:00:00 and the <ReelName> is letters and numbers. I've just added this functionality to EyeFrame Converter. I believe that it is possible to just add the metadata without an actual transcoding.
Very nice program, but it proved to be (almost) catastrohic for me today in a certain situation. Maybe this feedback could improve it?
We work with two camera's. When we have the program rewrite the filename, it is possible that movies that were made on the same moment overwrite eachother (same minute and second - it's possible, and becomes worse with the addition of extra cameras. A random hash after the filename would solve it. Or take a combination of time, serial number of card and folder name (and hash).
Secondly, one of the camera's had a wrong timestamp (the time was 3 hours off), so the the filesorting was completelt thrown off. I'd rather see the timestamp based on the moment we are offloading. This solves the double camera overwriting also.
Is there a possibility to have this situation resolved? I have to admit - of all (commercial) programs I have tried, this freebie was the best - but proved unsufficient for production work at the moment.
I'd even consider donating money to improve it, so it could become the swiss offloading knife for windows user. Let me know if this would be an incentive.
Quote from: Malcolm Debono on July 30, 2012, 04:54:37 PM
Version 0.4
HDRSLR Offloader now supports transcoding HDR videos while importing :)
The following steps can be done automatically:
- Extract the audio track from file (done through ffmpeg - included)
- Extract and sort even and odd frames from HDR videos (also done through ffmpeg)
- Process HDR frames with Enfuse (included) or with Photomatix (user must specify location of PhotomatixCL.exe from Options window)
- Save processed frames back to video (framerate and size can also be customized; done through ffmpeg)
Note: HDR videos are detected through the log file generated by ML, so please enable this option from ML menu before capturing
(http://malcolmdebono.com/apps/hdslroffloader_hdr_features.jpg)
Download link:
http://malcolmdebono.com/apps/HDSLRoffloader_0.4.zip (http://malcolmdebono.com/apps/HDSLRoffloader_0.4.zip)
I'd like to keep this thread alive as it's turned out to be a good solution for any of you Real Estate video shooters out there who have multiple HDR inside window shots. This program (ver 0.6.1) does the job nicely. I found the enfuse option quality too poor, but the Photomatix works well. It's about 3 times faster than the interframe script option plus it batches unlimited number of files. Sadly it's not available for the mac, pc only.
My latest discoveries about this. It takes 12 minutes to process 1 minute of footage.
http://www.magiclantern.fm/forum/index.php?topic=13430.msg129867#msg129867
Hi There,
Hi There I made a test movie and noticed that ...
I get a Charlie Chaplin effect .....although frame rate is correct at 25
The sound isn't embedded in the output file.
I'm probably doing something wrong
Can somebody help me...
Guy