Son of Batch is an app I made which uses raw2dng,dcraw and ffmpeg to batch process Magic Lantern .RAW files. It does dngs, backups and proxies. It shows a preview frame of the RAW files for easier selection. No file spanning yet. It's not perfect and early alpha, but I hope it's useful. Mac OS X (10.6.8 + up) only unfortunately.
I'd like to hear your feedback and bug reports.
Huge thanks to the Magic Lantern team.
Big Update: You can preview RAW files and DNG sequences and set In+Out points!
Spanning will be added, RAW2DNG seems to have problems with it right now.
Please give me some feedback!
UPDATE, many bugs fixed!
Usage: drag one or more files from the source list to the output queue or doubleclick files in the source. Then set in an out points.
You can select a file in the output queue and click on the right half of the info. This lets you change white balance and fps (fps can only be changed for DNG sequences, not RAW files).
Have fun!
derkiki
New Version 7/30:
http://tshit.de/Son_of_Batch_OSX-68.zip
New Link:
https://www.dropbox.com/s/kluyerg59f2c01f/Son%20of%20Batch.zip
--check out our t-shirts on http://tshit.de
Cool thx. Mac dev has been lagging a bit.
Can you update it incorporating A1ex's dual ISO hack-
http://www.magiclantern.fm/forum/index.php?topic=7139.0
http://acoutts.com/a1ex/raw2dng.exe
Great name. Too bad I'm on windows.
Quote from: squig on July 23, 2013, 05:37:20 AM
Cool thx. Mac dev has been lagging a bit.
Can you update it incorporating A1ex's dual ISO hack-
http://www.magiclantern.fm/forum/index.php?topic=7139.0
http://acoutts.com/a1ex/raw2dng.exe
I'd like to update it. Can you point me to a compiled binary of the newest raw2dng for mac? I'm still fighting with my compiler setup.
Thx
When this raw video thing first came out, the videos had vertical black stripes. I believe the raw extracting programs had to implement something to fix this.
The raw files that are extracted with you program have the vertical black bars? Or any issues as such?
Wow nice App!
I like it to have a Preview of the raw file before converting. Even if its only a still image. Can you implement the ability to scroll through the file to see not only the first Pic? That would be great!
Thanks a lot!
Quote from: daSilva on July 23, 2013, 07:30:04 PM
Wow nice App!
I like it to have a Preview of the raw file before converting. Even if its only a still image. Can you implement the ability to scroll through the file to see not only the first Pic? That would be great!
Thanks a lot!
I'll try!
Concerning black stripes: I'm updating to a new raw2dng, will post when done.
Also a start frame and end frame? (IN/OUT point)
(Please?) ;)
Nice initiative!
Quote from: derkiki on July 23, 2013, 04:30:31 PM
I'd like to update it. Can you point me to a compiled binary of the newest raw2dng for mac? I'm still fighting with my compiler setup.
Thx
You could send a pm to scrax and Thomas Worth, they're doing all the mac development.
thanks, i finallly got it to compile.
funny t-shirts! they made me crack ;D
gonna try the app, thank you!
It doesn't let me do anything, this message appears:
@ derkiki
Works just fine. the only problem i found is the fact i can´t change the frame rate on the proxies, to match the frame rate on the raw files.
Big thanks
Quote from: simulacro on July 28, 2013, 04:58:44 PM
It doesn't let me do anything, this message appears:
I have exactly the same problem :(. I want to try the new features on the updated release:
http://www.magiclantern.fm/forum/index.php?topic=7361.msg62363#msg62363 (http://www.magiclantern.fm/forum/index.php?topic=7361.msg62363#msg62363)
Dual ISO processing doesn't seem to be working :'(
Hey Squig, gotta install virtual box. Slow conversion but works on a mac and win 7 :)
Quote from: PlayIt on July 28, 2013, 06:45:24 PM
@ derkiki
Works just fine. the only problem i found is the fact i can´t change the frame rate on the proxies, to match the frame rate on the raw files.
Big thanks
The proxies for the RAW files are set to the frame rate of the RAW files by default.
Quote from: simulacro on July 28, 2013, 04:58:44 PM
It doesn't let me do anything, this message appears:
Sorry about the ugly error, it only works on 64 bit systems, what version of MacOsX are you on?
Quote from: squig on July 28, 2013, 07:50:56 PM
Dual ISO processing doesn't seem to be working :'(
I can't test it (no 5d3), there seems to be some problem with the latest raw2dng because file joining doesn't work either at the moment. When they update I#ll update.
SQUIG, can we ad ¨Follow this topic¨to the topics? It´s an advantage for people like me looking for upgrades for specific things. thx
Quote from: derkiki on July 29, 2013, 02:50:32 AM
Sorry about the ugly error, it only works on 64 bit systems, what version of MacOsX are you on?
10.7.5
I have to say it's a 2008 white macbook 3gb RAM
I'm working without battery (it swollen so i had to take it off) I heard that the efficiency of the computer reduces drastically but i don't know if that is related to the app's error, because other person has the same issue.
OSX 10.7.5 is 64 bit as is 0SX 10.8.4.
If anybody is in any doubt about if your system runs 64 bit mode or not just check your Activity Monitor (/Applications/Utilities/Activity Monitor) and you will see processes under 'kind' listed as 'Intel 64 bit'
I'm on Mountain Lion 64 Bit (10.8.4).
I get these errors when starting the app:
span error - I click OK
then..
An exception of class NilObjectException was not handled. The application must shut down.
I never even get in to the GUI.
Quote from: FBongcam on July 29, 2013, 03:34:49 PM
I'm on Mountain Lion 64 Bit (10.8.4).
I get these errors when starting the app:
span error - I click OK
then..
An exception of class NilObjectException was not handled. The application must shut down.
I never even get in to the GUI.
That's useful, thanks. I'll try and fix it when I have time.
The interface gives me only the chance of looking trough the icon folder structure on the left side to find the .raw files. If I go to: menu/open... the app says this:
An exception of class NilObjectException was not handled. The application must shut down
...and it shuts down
I found out something right now:
You have to double click the .raw file once you find it in the app's browser. Then the file appears on the output queue, now you have to choose an output folder, click OK and...
it's working!
Quote from: simulacro on July 29, 2013, 03:59:59 PM
The interface gives me only the chance of looking trough the icon folder structure on the left side to find the .raw files. If I go to: menu/open... the app says this:
An exception of class NilObjectException was not handled. The application must shut down
...and it shuts down
A common problem indeed :(
Quote from: derkiki on July 29, 2013, 02:48:23 AM
The proxies for the RAW files are set to the frame rate of the RAW files by default.
I check the raw files, with rawmagic and raw2dng, both display 25 FPS, but the proxies created by Son of Batch are 24 FPS.
If u need my camera settings, they are:
Canon Menu:
1920x1080 25fps
ML menu:
Desired FPS 25 (from 25)
Exact FPS
Actual FPS 25.000
I Hope this Helps.
You can select multiple files and drag them to the output queue or doubleclick.
I'll try and fix the bugs soon.
@ PllayIt
Thanks for the info, should be easy to fix. I'll post when the next version's up.
Quote from: derkiki on July 29, 2013, 04:30:01 PM
I'll try and fix the bugs soon.
Thanks it would be much appreciated :)
Quote from: derkiki on July 29, 2013, 03:45:51 PM
That's useful, thanks. I'll try and fix it when I have time.
Great, looking forward to it!
Please try the new version, link in the first post.
Cheers
Quote from: derkiki on August 03, 2013, 01:13:01 PM
Please try the new version, link in the first post.
Cheers
Thank you. It appears to work. I will be trying it out in my future workflow.
Very good :)
Nice! Works with dual-iso files as well. Cool stuff.
When trying to open File, ´open´This occurs
(http://postimg.org/image/rhimtwf0l/)
WHen digging to deep and fast into folders i got this
(http://postimg.org/image/7bpdtfz3x/)
or maybe It was the other way around. However. I moved my file to the root of the external disk and everuthings worked fine.
Good work. I,ll check your t-shirt :)
Great app!
Is there any chance we could get multithread/multicore support? I see dcraw is only using about 85% of one core... this could really scream if it converted multiple frames on separate instances of dcraw.
Will try the dual-ISO support right away and report back.
In my case:
Not working for dual ISO, yet.
I worked with 2 files from about 500 mb each one.
WHat I have to expect? A frame darkness and other one brightness?
It's not stable. If you add other file when the program is converting a preview task , collapsed (I have a Mac Pro 2012). When you press in and out appear to work very slow.
This my result of course, maybe it's a minor problem.
Thanks, is a good idea!!!! go foward!
It works pretty good actually. Only converter for mac that handles dualiso.
You get dng converted from the lower iso with higher iso baked in. When doing recovery in acr you have tons of information getting clean shadows and retained highlights. Think you have to have the isos in reverse order to make it work though. Instead of 100-1600 you set the dualiso 1600-100.
Quote from: Danne on August 04, 2013, 11:46:23 AM
It works pretty good actually. Only converter for mac that handles dualiso.
You get dng converted from the lower iso with higher iso baked in. When doing recovery in acr you have tons of information getting clean shadows and retained highlights. Think you have to have the isos in reverse order to make it work though. Instead of 100-1600 you set the dualiso 1600-100.
I think I am doing a mistake.
PLease, check my task:
- OPen raw file and applied the aplication to a folder.
- I take all the DNG file and open in ACR.
- I save the tiff files into other folder and I use compressor to create pro Res hq.
What I am doing wrong?
Apply highlightand shadow recovery and whitebalance in acr before export
Quote from: Danne on August 04, 2013, 09:29:08 PM
Apply highlightand shadow recovery and whitebalance in acr before export
I understand you but it's imposible for me adjust the darker and the brighter frames to get 24 frames with out any difference in the mid tones . That's why I think that I am doing something wrong.
Is there a numeric way to balance the iso in the frames? I mean, the iso 100 to 800 and the 1600 to 800?
I thought that the way to work with this were the same as HDR, putting a double timeline in Final or Premiere...
Any advice?
you apply the same settings to all dng,s by selecting all of them. Simply apply every information to one picture and apply the exact same to all the pics?
Quote from: Danne on August 04, 2013, 09:29:08 PM
Apply highlightand shadow recovery and whitebalance in acr before export
Highlight recovery and exposure adjustment in ACR causes flicker.
Hi Squig. Not always. You could also try applying a black spot and a white spot created with the lightroom brush on one dng and then copy the black and whitepoint to all pictures. Not sure this helps but had the idea when somebody talked about acr adjusted according to darkest and lightest areas. Could be sailing here but worth a try. After corrections are done you simply synchronize all dng removing the brushadjustment only as the last step.
Quote from: Oedipax on August 03, 2013, 04:51:54 PM
Great app!
Is there any chance we could get multithread/multicore support? I see dcraw is only using about 85% of one core... this could really scream if it converted multiple frames on separate instances of dcraw.
Will try the dual-ISO support right away and report back.
I'm not really sure how to do that. Will look into it.
Quote from: budafilms on August 04, 2013, 11:24:34 AM
In my case:
Not working for dual ISO, yet.
I worked with 2 files from about 500 mb each one.
WHat I have to expect? A frame darkness and other one brightness?
It's not stable. If you add other file when the program is converting a preview task , collapsed (I have a Mac Pro 2012). When you press in and out appear to work very slow.
This my result of course, maybe it's a minor problem.
Thanks, is a good idea!!!! go foward!
Thanks for the feedback. I'll try to fix it. Damn bugs!
Quote from: Danne on August 03, 2013, 04:20:34 PM
Nice! Works with dual-iso files as well. Cool stuff.
When trying to open File, ´open´This occurs
(http://postimg.org/image/rhimtwf0l/)
WHen digging to deep and fast into folders i got this
(http://postimg.org/image/7bpdtfz3x/)
or maybe It was the other way around. However. I moved my file to the root of the external disk and everuthings worked fine.
Good work. I,ll check your t-shirt :)
Thanks Danne. For now, don't use File>open. Doubleclick source files or drag them to the output queue.
No problem! Really good work. A lot of macdudes that are gonna be happy to be able to use dual iso. Question. Alex also provides a cr2hdr converter for converting cr2-files. Would it be hard to get that to work for mac? You seem to have the skills.
Nice t-shirts by the way
//D
Yes. Great app thx you
Quote from: Danne on August 05, 2013, 11:09:33 AM
No problem! Really good work. A lot of macdudes that are gonna be happy to be able to use dual iso. Question. Alex also provides a cr2hdr converter for converting cr2-files. Would it be hard to get that to work for mac? You seem to have the skills.
Nice t-shirts by the way
//D
When downloading a batch of files from one particular folder it pauses during the conversion of each and throws up this error:
http://grab.by/p980
When I click ok it returns to the conversion and then throws the same error up for each .RAW from that file.
Does anyone know what this error means? Is it corrupting/skipping frames on those .RAWs?
Quote from: jonny2graham on August 07, 2013, 03:18:26 PM
When downloading a batch of files from one particular folder it pauses during the conversion of each and throws up this error:
http://grab.by/p980
When I click ok it returns to the conversion and then throws the same error up for each .RAW from that file.
Does anyone know what this error means? Is it corrupting/skipping frames on those .RAWs?
No, it's a display error only. The conversion should be ok.
really want to try this new version.. sounds amazing!!
but.. the link http://tshit.de/Son_of_Batch_OSX-68.zip is broken as the site appears to be down? too much bandwidth from the downloads?
also do you think doing some of the conversions in parallel would speed up the conversion? when running raw2dng i run multiple copies and it seems to speed up the process ( possibly due to the multi core, or possibly because one is reading files with low cpu overhead whilst another is generating prores with low disk activity and high cpu )
keep up the great work!! J
Thanks for your interest! Please try this link. I'll look into the multithread when I have time.
https://www.dropbox.com/s/kluyerg59f2c01f/Son%20of%20Batch.zip
Hi derkiki
This app looks amazing so thanks, il try that new link now!
Ok great .. Parallel processing would be amazing! James
Quote from: derkiki on August 09, 2013, 05:33:38 PM
Thanks for your interest! Please try this link. I'll look into the multithread when I have time.
https://www.dropbox.com/s/kluyerg59f2c01f/Son%20of%20Batch.zip
Quote from: derkiki on August 09, 2013, 05:33:38 PM
Thanks for your interest! Please try this link. I'll look into the multithread when I have time.
https://www.dropbox.com/s/kluyerg59f2c01f/Son%20of%20Batch.zip
Hi Derkiki, thanks for that link!
ok the app is quite good! defiantly showing promise!
a few errors:
1) If not logged in as administrator it says something about NillError exception, the previous one did not do this for non-admin accounts!
2) sometimes when you click a raw it says "Picture Problem" .. you have to click the same file a few times to see a preview
3) its amazing that it accepts a DNG folder, not just raw files, but when i tried one and set it to 25fps it seemed to be as if i had chosen 5 fps.. it played back at the normal rate but it was soo sllooowww .. this this a bug?
& a few suggestions:
1) the ability to see the frames is amazing, could it have a play button that would drop say 4 out of 5 frames and let you see a playback? that would be amazing
2) it took me a while to work out how to go "up a directory" once i had entered one in the file browser ( clicking the path above incase someone else wondered ) could this be given the usual ".." at the top to go up one? also i would love it if you could drag a folder into the app and it would show that one for imports!
3) any sort of basic grade options would be amazing.. even if it was a "brighten up one or two stops" or "darken up one or two stops" option..
and yes, multi processing would be amazing!! for some would make conversions up to 8 times faster! ( at the moment i am running 8 version of the same app to get the speed up with 8 queue's -- one per app obv )
amazing app .. can see this being the default one quite soon! James
Great suggestions and thanks for your praise. I haven't got any time to work on it right now but will look into it.
The dng sequences don't play in realtime right now but you can scrub them pretty quickly.
ok that sounds amazing! the additions so far should save me time in my workflow already! also any chance it would be put-up on github etc..? if so i could contribute a few changes myself :-) James
Quote from: jc on August 12, 2013, 06:51:08 PM
ok that sounds amazing! the additions so far should save me time in my workflow already! also any chance it would be put-up on github etc..? if so i could contribute a few changes myself :-) James
i reckon that the app that goes Open for converting magic lantern RAW / MLV files will become the most popular and developed! :-) J
Hi jc,
I use realbasic and modify raw2dng etc., so, not sure if you can work with that. I'd also have to clean up my messy code a little.
Hi derkiki
haha ok in that case.. :-) i have almost no experience of realbasic & don't own a copy either! can program in most environments tho ( JS, Erlang, Objective C, some C etc.. )
Quote from: derkiki on August 16, 2013, 11:23:45 AM
Hi jc,
I use realbasic and modify raw2dng etc., so, not sure if you can work with that.
ha always the case when open sourcing code :-)
Quote from: derkiki on August 16, 2013, 11:23:45 AM
I'd also have to clean up my messy code a little.
Hey Derkiki. You might know this. Is it possible to recover the RAW files that raw2dmg deletes? I should've used Son of Batch. I was going to. But I figured what's the point, when raw2dng works just fine. Well, guess what, sometimes it works, and other times it deletes raw files without creating dmgs first. Derkiki please let me know if there is any way to recover raw files.
Did you try to look inside the app. Right click and go into contents?
Any work being done on the new .MLV format? :)
Quote from: alexp on August 20, 2013, 06:53:00 AM
Hey Derkiki. You might know this. Is it possible to recover the RAW files that raw2dmg deletes? I should've used Son of Batch. I was going to. But I figured what's the point, when raw2dng works just fine. Well, guess what, sometimes it works, and other times it deletes raw files without creating dmgs first. Derkiki please let me know if there is any way to recover raw files.
Sorry, I don't know how you could recover those. That's why "Son of Batch" doesn't delete the originals.
Good luck
Quote from: derkiki on August 20, 2013, 09:38:19 AM
Sorry, I don't know how you could recover those. That's why "Son of Batch" doesn't delete the originals.
Good luck
I recovered them. Switching to Son Of Batch anyway!
It works great, the only problem i had was that SON OF BATCH creates a proxy movie file, the dng's, and a back up of the PS files. I found that the proxy movie they develope isnt at quality at all of what it should be. In result i turn off the proxy export and the PS file export and just rely on the DNG files....then correct in Photoshop and export via quiktime to a movie sequence. The plus is that it can show a preview of a PS file easily, which does save you a lot of time knowing if the shot is usable or not. Anywyas thanks for the great work everyone, im loving this revolution.
Blainesuque: They are proxy's, designed to be easy to edit with which you then link up with your DNG's later.
To the developers of this: Incredible! I would love a back button in the GUI in case you go to the wrong folder (I need to literally quit the ap and then go back in) I was wondering: Would it be possible for this app to automatically de-squeeze something if it is shot in 720P mode? (like 1280x388?)
Thanks. I'll look into the back button thing. What do you mean by desqueeze. Does it show the wrong aspect ratio now? Please post a screen shot.
If you want to go up one folder just press escape or click on the path above the source list.
Quote from: David J. Fulde on August 26, 2013, 01:08:20 PM
To the developers of this: Incredible! I would love a back button in the GUI in case you go to the wrong folder
Hi David.. yes i was looking for the same thing.. the trick is.. click on the file path at the top, just to the left of where it says "output que" ... this will take you up one folder level!
my second trick is this.. make say 4 folders named 1 to 4 ( or 1 to 8 if you have an 8 core machine etc.. ) and place a copy of "Son of Batch" in each one.. then run all 4 / 8 at the same time.. this is a hack way to make it use all cores of your machine! it will be much faster!! ( until multi processing comes where it can be downloading all the time and then have 2,4,8,12 parallel processes all doing the raw to prores conversions! )
also any updates on the development.. at the moment this is the most promising version! will be even more exciting when the version 2 format becomes stable with all the meta data ( and sound!!) .. hopefully thats very soon!!
Quote from: derkiki on August 27, 2013, 09:35:20 AM
Thanks. I'll look into the back button thing. What do you mean by desqueeze. Does it show the wrong aspect ratio now? Please post a screen shot.
If you want to go up one folder just press escape or click on the path above the source list.
When shooting in 720P mode is actually shoots in 1280x388 and then you need to desqueeze by 1.40x it would be great if we could set SoaB to do this automatically. Here is what it looks like: http://imgur.com/6cUt76U
It's kind of a way to do video compression without doing video compression (Yes, this looks horrendous but that is beside the point right now :P )
It
s a pain to need to do this manually for every clip. Maybe there is some sort of meta data you could use as a flag to desqueeze it?
WHat does error lb2 mean?
@derkiki, I get this error every 5 minutes. What does it mean?
hi derkiki.
i have tried your app and compared it to magilcantern raw, and rawmagic..
the dngs created with your app looks best..
you can have a look to the testfiles..
https://www.dropbox.com/sh/wtzx0dgimdqqpca/RJtAYZsa22
thx so much for your time and effort.
sw
Hi derkiki, any chance for spanned 4gb files?? Raw Magic seems to have had it for a while now, so you could ask those guys how they did it if its problematic? Best regards and keep up the good work. James
Quote from: jc on September 05, 2013, 12:44:52 AM
Hi derkiki, any chance for spanned 4gb files?? Raw Magic seems to have had it for a while now, so you could ask those guys how they did it if its problematic? Best regards and keep up the good work. James
The first post in this thread by Derkiki mentions:
"Spanning will be added, RAW2DNG seems to have problems with it right now."
Raw Magic can do it ... So it is possible ..
Tho maybe as it goes to cine dng not normal dng that makes a difference ...
Hi,
I've some problems to convert the format mlv with MLVConverter in OSX.
will "Son of Batch" be implemented with this function?
Thanks
Quote from: sebamax on October 11, 2013, 09:10:44 AM
Hi,
I've some problems to convert the format mlv with MLVConverter in OSX.
will "Son of Batch" be implemented with this function?
Thanks
Eventually, yes. But it'll take some time, I am busy with other things for a while. Thanks for your interest.
Quote from: derkiki on October 11, 2013, 10:33:27 AM
Eventually, yes. But it'll take some time, I am busy with other things for a while. Thanks for your interest.
thank you for your kindly answer
Any chance of getting this updated with the latest raw2dng code? RawMagic is great but doesn't fix my dead pixels, and the Raw2Dng app has no .raw preview.
what happened???
This program was abandoned?
everybody is using .mlv now?
Because I used to use Son of Batch for converting many .raw files into .dng and with the preview, that helped me choosing the best scenes to put on the video. But I stopped using raw for a while and now my program is not working. Is there any possibilities of some one help me or correct this bug?
Because the Son of Batch is opening ok, but when I double click the raw file or drag it to the second column or choose the output folder and then click GO, nothing happens.
Quote from: derkiki on October 11, 2013, 10:33:27 AM
Eventually, yes. But it'll take some time, I am busy with other things for a while. Thanks for your interest.
Derkiki, your software it's the best for ML RAW. If you can add MLV could be perfect.
I hope you can do that!
Thanks for your talent!
Quote from: budafilms on May 19, 2014, 08:17:47 PM
If you can add MLV could be perfect.
I hope you can do that!
He did it
http://www.magiclantern.fm/forum/index.php?topic=11345.0