Magic Lantern Forum

Using Magic Lantern => Raw Video => Topic started by: Francis Frenkel on June 04, 2013, 11:57:02 AM

Title: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Francis Frenkel on June 04, 2013, 11:57:02 AM
Hello from France,

When I record more than 4gb i got 3 files on my CF card :

The 3 files haves the same name.

How can I merge those files to process them in ACR ?
What is the process to have one only file of my scene ?

What tool do I have to use ?

Help !

Francis
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: RenatoPhoto on June 04, 2013, 03:10:50 PM
Here:

RAW Video: PostProcessing -- Beginners Guide --

http://www.magiclantern.fm/forum/index.php?topic=5654.0

Other RAW manuals here:
http://www.magiclantern.fm/forum/index.php?board=49.0

and here:
http://www.magiclantern.fm/forum/index.php?board=15.0
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Francis Frenkel on June 04, 2013, 04:03:36 PM
Thank's but I dont understand :

1. Rename the files as  file1.raw  file2.raw .. etc > OK I understand but witch one is the first, second third ?
> Why do I have 2 WinRAR files and another one not wich one is the first one ?

Use command line > OK but where finding this window to type this command ?

> copy/b file1.raw+file2.raw+.... filenew.raw = copy where, from where ??

Sorry it's not clear for me...

Francis
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Yoshiyuki Blade on June 04, 2013, 04:49:52 PM
If you're using Windows, you can use the command line to merge the files without having to rename anything really. For Vista/7/8, hit the windows key to open up the search bar, type "cmd" (without quotes) and hit Enter to open up the command prompt.

Type the following to merge the files WITH quotes:

copy /b "C:\...\blah.RAW"+"C:\...\blah.R00"+"C:\...\blah.R01"+...+"C:\...\blah.RXX" "C:\...\blah-merged.RAW"

A couple slightly more advanced tips:

1) Open up a notepad file to type out the entire command first and then paste it into the command prompt. Press the windows key, type "notepad" and hit enter to open it up. Once you have everything written, hit Ctrl-A to select all the text, Ctrl-C to copy the text, then paste it into the command prompt window (AFAIK, you can only paste it by right-clicking the window and clicking "Paste").

2) You can quickly copy the path of each file by holding Shift and right-clicking on the file. A new option called "Copy as path" will appear. Click it and paste the path in notepad with Ctrl-V. It'll have quotes too.

edit: Oh, I forgot that some file extensions may be hidden by default and you may not see them unless you show them. The WinRAR files have the extension .R00, .R01, .R02, etc. The first file in the sequence is .RAW followed by .R00, .R01 etc.
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Redrocks on June 04, 2013, 09:20:24 PM
Thanks for the info Yoshiyuki, unfortunately I seem to be missing something. I'm working with these files:

"D:\raw\DNG\M02-0250.RAW"
"D:\raw\DNG\M02-0250.R00"
"D:\raw\DNG\M02-0250.R01"
"D:\raw\DNG\M02-0250.R02"

and have merged them using the line:

copy /b "D:\raw\DNG\M02-0250.RAW"+"D:\raw\DNG\M02-0250.R00"+"D:\raw\DNG\M02-0250.R01"+"D:\raw\DNG\M02-0250.R02" "D:\raw\DNG\M02-0250-merged.RAW"

But when I go to extract the DNG's from the resulting M02-0250-merged.RAW (using Batchelor) I get an empty folder. When I try to drag and drop to Raw2dng, the Cmd window opens and closes rapidly and nothing happens.
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: RenatoPhoto on June 05, 2013, 03:06:26 AM
Quote from: Redrocks on June 04, 2013, 09:20:24 PM
When I try to drag and drop to Raw2dng, the Cmd window opens and closes rapidly and nothing happens.

Make sure you have the latest Raw2dng.exe  The older ones do not open large files.

1.1 Please download the RAw2dng.exe  application for windows.
HERE: https://bitbucket.org/hudson/magic-lantern/downloads

If that is not it then you may have the order of the files wrong?  Check on the sizes and the times. 

Also you can use the simpler:

Download file joiner splitter such as FFSJ v 3.3
http://www.softpedia.com/get/File-managers/File-Splitter-and-Joiner.shtml

All of this information found here:
RAW Video: PostProcessing -- Beginners Guide --
http://www.magiclantern.fm/forum/index.php?topic=5654.0
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Yoshiyuki Blade on June 05, 2013, 04:14:04 AM
Yeah, it may be a problem with the version of Raw2Dng. The command line you posted looks perfectly fine.
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Francis Frenkel on June 05, 2013, 10:33:40 AM
same for me :
When I type the command line it creates the merge file but when i drop the merge file in the RAW2DNG file it doesnt open files.

And :
the merge file his only 2Gb wich is not normal cause 2 files are 4Gb each and the last one 1/2Gb....

The RAW2dng is from the 5th june the en recent one...

I download the File Spliter and joiner, but I cant select the files to be joined : not recognise....

Please, could you propose a simple way to do it....

I want to shoot a short movie, and lots of the takes will be more than 4Gb.... if I have to do this work each time, post-production will take month... I'm ready to work hard, not to get creazy ! ha ha ha ah !

Please, hep me !

Here is the teaser of the short movie :
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Redrocks on June 05, 2013, 12:03:19 PM
My Raw2dng was upto date as of yesterday, I downloaded the latest version anyway and still no joy. Although my merged file was the same size as its parts.

Thanks for the link to the file joiner. I tried it out and couldn't fathom the naming structure you have to work with, but I'll have another crack at it later and will find a way.

Good luck with the film Francis, looks very interesting. Hopefully the final piece will have subtitles!
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Yoshiyuki Blade on June 05, 2013, 03:01:37 PM
Hmmm, yeah I'm kinda stumped. Just updated to the latest build by a.d. (94190a95bbaf), recorded 6104 frames, merged them via the command line and now I'm currently extracting them. Both the raw2dng from the main ML bitbucket and the build from a.d. works. The a.d. build is more recent though and seems to do more with the vertical stripes correction (lots of the corrections are something other than 1.000).
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: RenatoPhoto on June 05, 2013, 03:05:49 PM
Quote from: Francis Frenkel on June 05, 2013, 10:33:40 AM
I download the File Spliter and joiner, but I cant select the files to be joined : not recognise....

Stop and read below:

1a. Rename files as file.001, file.002
Download file joiner splitter such as FFSJ v 3.3
http://www.softpedia.com/get/File-managers/File-Splitter-and-Joiner.shtml

your file joiner is not working because it requires for you to rename the files as specified above i.e.

file.001 the next one file.003, etc
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Francis Frenkel on June 05, 2013, 03:59:04 PM
Yes, this is what I'm doing...
I rename the .raw file (wich is the first one) as : file.001
I rename de WinRar (.R00) file (the second one) as : file.002

None of them are recognise when I select "Joining" in the Spliter & joinger tool and try to select the 2 files in the directory...

------------
The first file is a . RAW file isn't-it ?
The second one is an WinRar (.R00) file ? yes or no ?

What do I do wrong ?

Francis

Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: RenatoPhoto on June 05, 2013, 04:18:45 PM
Quote from: Francis Frenkel on June 05, 2013, 03:59:04 PM
Yes, this is what I'm doing...
I rename the .raw file (wich is the first one) as : file.001
I rename de WinRar (.R00) file (the second one) as : file.002

None of them are recognise when I select "Joining" in the Spliter & joinger tool and try to select the 2 files in the directory...

------------
The first file is a . RAW file isn't-it ?
The second one is an WinRar (.R00) file ? yes or no ?

What do I do wrong ?

Francis

The joiner will only see the xxx.001 file.  If you selected it will join the two files xx.001 and .002

You will have to rename the joint file to file.raw for it to be processed by Raw2dng
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: RenatoPhoto on June 05, 2013, 04:20:27 PM
Also are you using "File Splitter & Joiner (FFSJ v3.3) ?
Windows O.S.
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Francis Frenkel on June 05, 2013, 05:09:59 PM
Yessss !!! I'm so happy !

My error was :
> I was "only" renaming the file, but not chaning the extention of the file, wich is different !!!
The extension file is not visible by default (Windows 7) !

Thank you so much renatoPhoto and Yoshiyuki Blade for your patience !

Francis
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: neopixel on June 05, 2013, 11:59:55 PM
Hello guys!

After reading your posts I still have the same issue as Francis here.
I recorded 3 different files and have this:
file.RAW, file.R01, file.R02
I renamed them file.001, file.002, file.003 and open file "spliter and joiner v3.3" to merge them but the software message me an error after 4% of loading.

An other thing, if I try to convert my .RAW file of 4gb directly from raw2dng, it also opens and closes very quickly!

I tried different things and read all over ML forum but can't find anyone with a detailed answer!

Your help would be much appreciated!

Tahnks a lot!
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: neopixel on June 06, 2013, 12:20:59 AM
new try, this time I only renamed file.r00, file.r01, file.r02 into file.001, file.002, file.003.

Open file splitter & joiner v3.3; it found only file.001. I opened it, set the file directory on desktop, hit the join button.
After 5% of loading an error appear: "can't write to a file stream". I have no clue what I've done wrong!

I am on:
windows 7
latest raw2dng
build 29th may
file splitter & joiner 3.3
5d2
sandisk 64gb, 90mb/s

thanks again!
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: RenatoPhoto on June 06, 2013, 02:44:43 AM
Quote from: neopixel on June 06, 2013, 12:20:59 AM
Open file splitter & joiner v3.3; it found only file.001. I opened it, set the file directory on desktop, hit the join button.
After 5% of loading an error appear: "can't write to a file stream". I have no clue what I've done wrong!

Never had that error.  Probably corrupt RAW files??  Maybe a prob with 5D2, search in the 5D2 for other people having the same prob...

But you should try the newest compiles always since lots of stuff is changing very quickly.
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: neopixel on June 06, 2013, 03:09:28 AM
just tried it! same issue and impossible to convert with raw2dng which still open and close very quickly without converting anything.

I've read all the posts about it, didn't find anything yet. Hopefully a solution will come out! :)
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: rawmania on June 06, 2013, 09:29:03 AM
Quote from: RenatoPhoto on June 05, 2013, 04:18:45 PM
The joiner will only see the xxx.001 file.  If you selected it will join the two files xx.001 and .002

You will have to rename the joint file to file.raw for it to be processed by Raw2dng

Hi
I've followed Renato steps and worked.
However the dng files from the final big raw file that came from .002 original file seems to be corrupt. files from .001
original file is ok.
(If i'm  trying to preview with adobe dng viewer the color is perverted to false colors)
Any help
regards
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: a1ex on June 06, 2013, 09:33:11 AM
Is this with current nightly or with something ancient from last week or so?
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: rawmania on June 06, 2013, 10:24:25 AM
https://bitbucket.org/hudson/magic-lantern/downloads

is this version, downloaded today
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: guentergunter on June 17, 2013, 08:34:18 AM
For MAC I highly suggest RAWMagic (http://www.magiclantern.fm/forum/index.php?topic=6218.0), since it handles the spanned files on the fly and directly converts to CinemaDNGs.
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: fatpig on June 17, 2013, 01:36:55 PM
I just wrote an automatic join app for my utility BATCHelor, it is not quite ready yet- is this still an issue?
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Jagannatha on June 18, 2013, 04:28:18 PM
Quote from: neopixel on June 06, 2013, 03:09:28 AM
just tried it! same issue and impossible to convert with raw2dng which still open and close very quickly without converting anything.

I've read all the posts about it, didn't find anything yet. Hopefully a solution will come out! :)

Hi!
In "file splitter & joiner 3.3" select another destination folder for output file (not your photo memory card!!!). Your card is FAT32 and can not write a file larger than 4Gb and you are already joining 4Gb + X Gb :)

UPDATE:
It joins the files with no problem. But! For me it was like - after jining 2 exmaples one joined file was good and RAWanizer extracted all the DNGs from it and another joined file (joined out of 4 + 0,2 Gbs) was like empty but at the same time of 4,2Gb....
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: lapoic on July 31, 2013, 10:22:25 PM
It took me awhile but I found out how to merge the files in mac OSX...

Download MacHacka
Rename the files you want to combine as file.001, file.002, etc...
In MacHacka File>Alphajoin... and then just select the first one.
It will create one file, which then you add .RAW at the end once it's finished joining the segments.
Now you can successfully convert using RAWMagic and probably raw2dng.
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: Untitledphoto on August 25, 2013, 08:12:00 PM
Thank you very much lapoic!

you've save my day.
But i have a question... how do you know what's the first file of all ?
Title: Re: 4gb limit Raw recording 5d2 (how to merge 2 files ?)
Post by: ifleeter on November 07, 2013, 04:52:58 AM
@lapoic Why didn't you just use terminal to merge the files on your mac using the cat command?