HDSLR Offloader (Windows utility to backup footage & transcode HDR videos)

Started by Malcolm Debono, June 24, 2012, 11:53:53 PM

Previous topic - Next topic

0 Members and 3 Guests are viewing this topic.

Malcolm Debono

Thought I'd share with you a small utility for Windows I've written a few months ago in C# in case it's useful to you guys  :)

This is a simple light-weight app which allows you to offload footage from your memory card to your PC. I've found it quite useful as a backup for on-location shooting, and carry it on a small netbook.

Here's a break-down of the features:
- Copy files from memory card to a primary & secondary (optional) location of your choice
- Filter file types to copy (currently MOV, THM, JPG, CR2, WAV, MP3)
- Renaming video files to creation date & time (as a timecode reference)
- Verification of data after copying (by comparing exact file sizes)
- Storing up to 3 profiles (video, audio & photo profiles)
- Quick profile switching using CTRL+1/2/3 keyboard shortcuts
- Copying all Magic Lantern files, only the configuration file (magic.cfg) or the config and hidden.cfg (introduced in ML 2.3) from a pre-defined location
- Generating a shot list based on data obtained from LOG files generated by Magic Lantern, which includes the date & time, lens used, shutter speed, aperture, ISO, focal length, white balance, picture style, FPS, HDR and bitrate
- Exporting shot list to Excel spreadsheet
- Transcoding of HDR videos captured through ML
- Argument passing for running through scripts or batch files

Screenshot:


Download latest version (see posts below for changelog):
http://malcolmdebono.com/apps/HDSLRoffloader_0.6.1.zip

Let me know if you have any ideas for new features or encounter any bugs.
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

Schloime

Hi.

I just found your tool while strolling through the forum, can't test it now because I'm on work.
It sounds definitely useful, especially all the options to treat the files.
I already have a feature request (you asked for it): Do you know the tool "QTChange"? I use it to get real TC into the files. Is there a possibility to have that functionality in your tool?

Greetings,
  Schloime
600D, EF-S 17-85, EF-S 55-200, SNS-HDR, Photoshop7, Lightroom, Lightwave, Lightworks, Light everywhere

Malcolm Debono

Hey Schloime,

I will definitely look into it. Apparently QtChange doesn't support command-line interface, so for now I don't think I can integrate it. I will check if there are any other libraries which can do the same thing. So far the closest I found is a modified version of ffmpeg (with added broadcasting features), and according to its documentation, it can write quicktime files containing timecode tracks.

I'll see if I can work something out in the coming days  :)

Malcolm

Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

ecek2

EOS 550D 18-55mm IS - 50mm F/1.8

Malcolm Debono

MediaInfo only shows you timecode and other video properties. It cannot modify the TC track like QTchange can.

I've done some tests with FFmpeg and it can add a TC to the video file itself, but only by converting the video. What I really like in QTchange is that it saves the TC track in the MOV file without any conversions, so I'm still looking for a better solution.
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

3pointedit

Wow, cool tool, very useful!

Feature pester request: Any chance of adding a transcode video HDR files (at transfer time) into seperate light/dark files? If using ffmpeg would be nice to choose between frame sequences or duped media at frame rate = half.
550D on ML-roids

GershberG

looks very usefull. TnX alot!

some questions:
is verification time\size based or CRC?
are both targets verified?
7D[ML], 60D[ML], 5Dc[ML]
8-15L, 16-35L II, 24-70L II, 70-200L, 50L f:1.2

Malcolm Debono

Quote from: 3pointedit on July 04, 2012, 02:58:05 AM
Feature pester request: Any chance of adding a transcode video HDR files (at transfer time) into seperate light/dark files? If using ffmpeg would be nice to choose between frame sequences or duped media at frame rate = half.

I think I can do that with ffmpeg (convert video to image sequence, and split images based on even/odd frames). Do HDR videos have a different name (in comparison with normal videos)? This can be used to determine whether a video is HDR or not.

Just checked and HDR videos are named the same format, so I don't know how I can check if a video is HDR or not automatically. The only other option I can think of is for the user to select which videos are HDR before copying starts (so that transcoding starts straight afterwards without any user intervention).

Update: Another (better) option is to use movie logging, but that can only be used if movie logging was enabled (obviously). The HDR settings used are logged when an HDR video is captured, otherwise this field is missing. I simply have to search for HDR in this file to determine if the video is HDR or not  :)

Quote from: GershberG on July 27, 2012, 09:56:20 PM
some questions:
is verification time\size based or CRC?
are both targets verified?

Verification works by checking that the byte size of the copied file matches that of the original file.
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

Malcolm Debono

Version 0.4
HDRSLR Offloader now supports transcoding HDR videos while importing  :)

The following steps can be done automatically:
- Extract the audio track from file (done through ffmpeg - included)
- Extract and sort even and odd frames from HDR videos (also done through ffmpeg)
- Process HDR frames with Enfuse (included) or with Photomatix (user must specify location of PhotomatixCL.exe from Options window)
- Save processed frames back to video (framerate and size can also be customized; done through ffmpeg)

Note: HDR videos are detected through the log file generated by ML, so please enable this option from ML menu before capturing




Download link:
http://malcolmdebono.com/apps/HDSLRoffloader_0.4.zip
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

Michael Zöller

Now thats a nice new feature. Will try this tomorrow. One feature I was just thinking about would be a command-line option so that one could auto-run this whenever a card is inserted.
neoluxx.de
EOS 5D Mark II | EOS 600D | EF 24-70mm f/2.8 | Tascam DR-40

ilguercio

Nice application but movie log isn't available for the 50D.
Can you add an option for logless files? :)
Canon EOS 6D, 60D, 50D.
Sigma 70-200 EX OS HSM, Sigma 70-200 Apo EX HSM, Samyang 14 2.8, Samyang 35 1.4, Samyang 85 1.4.
Proud supporter of Magic Lantern.

screamer

Wow Malcom, this is amazing, i will give it immediately a try!! great work indeed
always trying to use the 100% of magic lantern..
Gear:
Canon 60D, all the samyang lenses, Canon 50 mm 1.4, Canon 60mm macro, Canon 70-300 usm, Sigma 4.5mm fisheye, Sigma 17-70 2.8, Canon 40mm f2.8 pancake, all the Lensbabies and a lot of other pieces, Flash metz 58 af2

Malcolm Debono

Quote from: Michael Zöller on July 30, 2012, 11:18:32 PM
Now thats a nice new feature. Will try this tomorrow. One feature I was just thinking about would be a command-line option so that one could auto-run this whenever a card is inserted.

That's a cool feature! I'll implement argument passing so that all the required info can be passed through cmd.

Quote from: ilguercio on July 30, 2012, 11:41:42 PM
Nice application but movie log isn't available for the 50D.
Can you add an option for logless files? :)

I'll try to find some other way  :) Only other option I can think of is opening a file chooser dialogue so that you can select which videos are HDR manually. This can also be used to process HDR videos without the importing part.

Quote from: screamer on July 31, 2012, 12:07:15 AM
Wow Malcom, this is amazing, i will give it immediately a try!! great work indeed

Thanks  :)
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

Malcolm Debono

Version 0.5
- Manual selection of HDR videos (can be used both during importing stage, or from the Tools menu)
- Argument passing

Arguments supported are:
-p1 / -p2 / -p3            Specify which profile will be loaded automatically when the app starts
-start                         Automatically starts copying when app starts

Example:
"HDSLR Offloader" -p1 -start
This command starts the app, loads the Video profile, and starts the copying process automatically

To use the profiles, you will first need to have them saved by starting the app, specify which options will be used and the paths, and save the profile to one of the 3 slots through the File menu. This only needs to be done once, as you can then load the profile either manually through the File menu or the shortcut keys (CTRL + 1/2/3), or through the arguments.

Download link:
http://malcolmdebono.com/apps/HDSLRoffloader_0.5.zip

Enjoy  :)
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

ilguercio

Thanks a lot Malcolm ;)
I really appreciate your effort and contribution to the community.
Canon EOS 6D, 60D, 50D.
Sigma 70-200 EX OS HSM, Sigma 70-200 Apo EX HSM, Samyang 14 2.8, Samyang 35 1.4, Samyang 85 1.4.
Proud supporter of Magic Lantern.

Malcolm Debono

Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

Michael Zöller

Thanks for sharing your work and even implementing ideas from the community (I think the ml community is really maturing :p)!
neoluxx.de
EOS 5D Mark II | EOS 600D | EF 24-70mm f/2.8 | Tascam DR-40

Malcolm Debono

A healthy community is all about sharing & helping. I wanted to build this app mainly for personal use, but if it can help others too, why not? :)
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

cosmo.lv

Quote from: Malcolm Debono on July 31, 2012, 03:55:16 PM
A healthy community is all about sharing & helping. I wanted to build this app mainly for personal use, but if it can help others too, why not? :)

Looks great tool :) i'm always impressed by peoples with knowledge of C and other languages - ehh.. :)
can i hope for Mac OS X application - or this stays only for Windows?  ??? :)
5D Mark II

Malcolm Debono

Quote from: cosmo.lv on August 01, 2012, 02:50:17 AM
Looks great tool :) i'm always impressed by peoples with knowledge of C and other languages - ehh.. :)
can i hope for Mac OS X application - or this stays only for Windows?  ??? :)

Thanks!

This is written in C#, which is a Windows language. There might be some solutions to compile this for Mac using tools like Mono. I'll see if I can manage to try it out when I find some free time  :)
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

cosmo.lv

Quote from: Malcolm Debono on August 01, 2012, 12:41:28 PM
Thanks!

This is written in C#, which is a Windows language. There might be some solutions to compile this for Mac using tools like Mono. I'll see if I can manage to try it out when I find some free time  :)

That's will be great! :)
5D Mark II

Malcolm Debono

Unfortunately it is next to impossible to port it for Mac (would be better to just write it from ground up in another language). The way HDR transcoding works is by passing parameters to external apps such as ffmpeg, enfuse and photomatix through command-line, which would need to be changed completely to work on Macs. There are also some other dependencies such as Microsoft Office, which is used for exporting a spreadsheet file of all the data in the LOG files.

One more thing I wanted to add is that the ML features within the app (mainly copying config file or all ML folder to card) currently do not support ML 2.3 (since the file structure was changed). An updated version will be released soon which offers support for both ML 2.3 and previous version (2.2).
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

Malcolm Debono

Version 0.6
- Support for ML 2.3
- Fixed shot list generator (now fully compatible with log files generated by ML 2.3)
- Minor bug fixes

It is now easier than ever to keep your cards synchronised. You can now keep a copy of all your ML files (including the standard config and the hidden items config) on your PC, and copy either all the ML files or only the configs to your memory card with the click of a button - useful when you have many different memory cards and need to have the same configs on all cards!

The shot list generator has also been updated and is fully compatible with log files generated by ML 2.3, which now includes HDR and bitrate information. The list can also be exported as a spreadsheet.


Download link:
http://malcolmdebono.com/apps/HDSLRoffloader_0.6.zip

Enjoy  :)
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!

hd1roam

I really appreciate your development efforts and support.
Do you ever get any sleep?

I think I need to tr this tool. Looks very interesting.
ML ver. 2.3: mostly still photos, HDR, pano, and some video. Targets of choice; Art, landscape, wildlife, collector vehicles, events. Equipment; EOS 550D/T2i | EF 100-400mm L USM - EF-S 18-55mm f4-5.6 IS - EF-S 55-250mm IS - Win7Ult., Lightroom, Photomatix, Topaz Labs, Hugin, GIMP, Smugmug

Malcolm Debono

Quote from: hd1roam on August 11, 2012, 06:20:55 PM
I really appreciate your development efforts and support.
Do you ever get any sleep?

Rest assured that I do  :D

This app isn't something that I spend a lot of work on. I only see what needs to be done (both stuff that I require, and user requests that can be implemented) and try to get it done in my free time (which I currently have plenty of since I'm on holidays!).
Wedding & event cinematographer
C100 & 6D shooter
New here?  Check out the FAQs here!