Magic Lantern Forum

Hello everyone, I understand we may have accidentally recorded some RAW video to the SD card instead of the CF card.
We may also have done Load Modules wrong.

My big question is this:
"Can we salvage this footage?"

We have a few dozen RAW files ranging from 300MB to 1.5GB's that are 1920x1080, 29.97.
Losing this footage would be epic. I have Hex Fiend installed for Mac and have changed the footer as instructed.
All I get is 1 frame that's good and the rest are Pink crazy dng's.

Can anyone please help?

thanks
Dan

Let me add this to the "help" list.. I will gladly PayPal someone who can help me salvage these files tonight.
I will pay for your time.

+1 on this..

I have a 42 gig file that ran off the end of my card, so the footer is missing. Stitched together the .raw + .r00, etc. in Hex Fiend, applied the 64 gig footer, (thanks to Shield: http://www.magiclantern.fm/forum/index.php?topic=5732.0 ). I have tried to run this new file through both Raw2DNG and Son of a Batch (running on a mac), but am having the same problem as dangerh33, where every DNG is a pink mess, aside from the first frame.

As I understand it, this issue is only occurring on Mac hardware. Correct? Has anyone been able to process files like these? I believe the issue of 2gig+ files has been resolved, so could it be the footer or the way that they're stitched? I won't pretend to understand Hex code, so pardon my ignorance. I just assume that since RAW2DNG and Son of a Batch work fine with smaller files, the error is likely due to something on my end.

At this point, I'm saving the .RAW + .R00s to an external drive until there's a solution, but I would also be happy to PayPal someone who can help with this situation.

I would be happy to go into further detail if it would be of any help. Many thanks to all of the Devs and those assisting, your work is greatly appreciated.

From the other thread it looks like your aligment is off... ie the first frame is pushed down and the next one is pushed down some more and by the end you just end up with pink noise.

If you have something like 100MB I can try to fix it but 42gb :(

@ 1% ...

thanks for the reply, man. That makes sense, because the pink files are often split in the preview frame for Son of a Batch, or in ACR.  I'm not sure if this is applicable, but when I'm stitching the frames in hex fiend, I noticed that broken files are consistently two digits "short" from filling up the coding line completely (and are obviously missing a footer):

i.e. {52 83 08 0D / 20 3B 83 FC / 0C 72 3D 88 / F8 20 B2 84
       1E 21 3B 85 / B4 1C 42 35 / 48 BC 21 B2 / 84 08 1A __ }

Would it be possible to cut a 50MB sample from the 4 gig .RAW file in HF for you to examine? I can send it through WeTransfer, or whichever method you prefer. And, of course, compensate you for your time.

If you have the link for the thread you mentioned, that would be great as well. I can't seem to find it.

Yea, cut 50MB of it and send me the link. Lets see what happens.

Also can you record a small file like 20 frames with the exact same settings you used?

@1%

Greatly appreciate your help with this. I PM'd you a dropbox link as well as some sample code with a footer from a similar, working sequence. Let me know if there's anything else that would help you.

BTW, get

https://www.macupdate.com/app/mac/22750/0xed

It looks like it shows the ascii on the side so you can see the rawM text at the start of the header.

Just wanted to update this workflow with the fix, thanks to @1%...

Please correct me if you spot an error.

This is a Mac workflow, and is intended to fix the problem with the pink DNGs caused by a corrupted file sequence. Corrupted files often occur when the video sequence takes up more room on the CF card than is available or when there is a problem saving the sequence to the card. In order to recover the salvageable data, you need to apply the correct footer in the hex code for processing in Raw2DNG, Son of a Batch, etc. This also covers how to stitch together 4+ gig files, how to calculate your total frame count, and how to covert that number to hexa decimal code.

- import the desired .raw +.r00, etc. files from your cf card to your drive (I prefer a new folder on my desktop that I can relocate upon completion)

- open the .raw file first using 0xED to view the hex code .. available @ https://www.macupdate.com/app/mac/22750/0xed

- open the sequential .r00 +.r01, etc. files in 0xED and copy and paste the hex code (located on the left side) below the hex code for the .raw file in order to create a sequential batch

- apply the appropriate footer to the end of the batched hex code. To determine this you will need your total frame count and the corresponding hex code for that value.

       - to determine frame count, multiply the gb of the total file by 1024 and divide by your DNG file size in MB (i.e. 40.25 gb * 1024 / 3.7 MB = 11139.45 -> 11140 frames)

       - to determine the corresponding hex code, use a decimal to hexa decimal converter: http://easycalculation.com/decimal-converter.php

       - the hex code is applied backwards in the footer where "XX XX" is: 11140 = 2B84 or "84 2B"

52 41 57 4D 80 07 38 04 00 60 37 00 XX XX 00 00 01 00 00 00 A8 5D 00 00 48 48 04 00 00 00 00 00 01 00 00 00 00 20 15 0B 26 05 00 00 20 08 00 00 38 0E 00 00 50 34 49 00 0E 00 00 00 FF 07 00 00 98 3A 00 00 00 00 00 00 00 00 00 00 88 07 00 00 08 05 00 00 1E 00 00 00 92 00 00 00 26 05 00 00 1A 08 00 00 00 00 00 00 00 00 00 00 00 01 01 02 01 00 00 00 42 1A 00 00 10 27 00 00 85 FD FF FF 10 27 00 00 3D FC FF FF 10 27 00 00 41 EF FF FF 10 27 00 00 AC 30 00 00 10 27 00 00 EC 07 00 00 10 27 00 00 74 FC FF FF 10 27 00 00 72 08 00 00 10 27 00 00 24 16 00 00 10 27 00 00 EB 03 00 00

---------------------------
completed footer for 40.25 gig file:

52 41 57 4D 80 07 38 04 00 60 37 00 84 2b 00 00 01 00 00 00 A8 5D 00 00 48 48 04 00 00 00 00 00 01 00 00 00 00 20 15 0B 26 05 00 00 20 08 00 00 38 0E 00 00 50 34 49 00 0E 00 00 00 FF 07 00 00 98 3A 00 00 00 00 00 00 00 00 00 00 88 07 00 00 08 05 00 00 1E 00 00 00 92 00 00 00 26 05 00 00 1A 08 00 00 00 00 00 00 00 00 00 00 00 01 01 02 01 00 00 00 42 1A 00 00 10 27 00 00 85 FD FF FF 10 27 00 00 3D FC FF FF 10 27 00 00 41 EF FF FF 10 27 00 00 AC 30 00 00 10 27 00 00 EC 07 00 00 10 27 00 00 74 FC FF FF 10 27 00 00 72 08 00 00 10 27 00 00 24 16 00 00 10 27 00 00 EB 03 00 00

- save this new, completed batch from 0xED to your desktop with a new component in the name that you'll recognize.. i.e. M09-0858.RAW -->  M09-0858full0xED.RAW

       - (if you need space on your drive at this point, you can move the original files to an external drive. deletion is possible, but it is recommended that you retain the originals in some form.)

- open this file with your preferred batch processor, i.e. Son of a Batch ( http://www.magiclantern.fm/forum/index.php?topic=7266.50 ), or Raw2DNG ( http://www.magiclantern.fm/forum/index.php?topic=5508.0 ), and convert the .RAW files to .DNGs ... 000000.dng -> 011139.dng

- save these DNGs to a folder on your desktop

- once completed, select the folder containing all of your applicable DNGs and, with all of the files highlighted, double click them to open in Adobe Camera Raw (Photoshop)

- adjust the settings of your file in ACR, click "Select All > Synchronize > Save Images". Save these new DNGs to a subfolder with a new component in their name: 000000.dng > 000000ACR.dng

- once completed, import the DNG image sequence to Adobe After Effects, make sure all of your settings are applicable/as desired (i.e. frame rate, color correction, lossless format, etc.), and export the files as a video sequence for further processing in Premiere Pro, Final Cut Pro, etc.

(There are other methods that can be used for processing, such as GingerHDR and DaVinci Resolve, but I don't have much experience with these programs.)

Hopefully this workflow will be of assistance to you when trying to recover corrupted data off of your CF card. I was able to successfully pull all but the last twenty frames from my 11140 frame sequence. Again, please let me know if I missed anything or if more detail is needed.

Many thanks to all of the Developers. Your work is greatly appreciated.

I followed this workflow.

I could not salvage my frames. Can someone please help? My best shot was on that file. All the frames come out pink except the first.

I have two files, a .RAW (4.29 GB) and a .R00 (2.73 GB). Both are spitting out pink frames through RawMagic.

I used Hex Fiend as for some reason with 0xED I am unable to copy paste 2.73 GB of data onto the clipboard. I'm using a MAC.

This thread was very, very helpful to me. Thank you all!

I recently just shot four 12hr days with the 5d mark 3 ML hack. We used it to shoot a student short film. Going into it I really had no idea how to work everything with the hack, so it was a major learning process for the crew and I. Many of which have never even shot a short narrative. We had an amazing DP who does a lot of professional work, helping us out at a lower than normal charge.

Well the first day we shot everything and it went great. Day two came around and while we were prepping for it I came across this thread. I was reading about the combining of the .R00 files. Immediately I'm like F**K! I told the DIT that those extra .R00 files weren't needed. Day one being an intense dialogue driven day, shitty for us losing all that extra footage. Now who knows how much of the stuff was cut. And all those files that stop at 4gb are corrupt as well. They won't open in any Raw2Dng converter Mac or Windows.

So, I read the thread about replacing the file footer, seems easy, I do it and yes! Now the files are converting to Dng's. Once they finished I opened them in Photoshop and they are all pink crazy dng's. FML. All that footage ruined. I come back to this thread and follow the instructions and do the math and create my own footer.

52 41 57 4D 80 07 38 04 00 60 37 00 01 45 00 00 01 00 00 00 A8 5D 00 00 48 48 04 00 00 00 00 00 01 00 00 00 00 20 15 0B 26 05 00 00 20 08 00 00 38 0E 00 00 50 34 49 00 0E 00 00 00 FF 07 00 00 98 3A 00 00 00 00 00 00 00 00 00 00 88 07 00 00 08 05 00 00 1E 00 00 00 92 00 00 00 26 05 00 00 1A 08 00 00 00 00 00 00 00 00 00 00 00 01 01 02 01 00 00 00 42 1A 00 00 10 27 00 00 85 FD FF FF 10 27 00 00 3D FC FF FF 10 27 00 00 41 EF FF FF 10 27 00 00 AC 30 00 00 10 27 00 00 EC 07 00 00 10 27 00 00 74 FC FF FF 10 27 00 00 72 08 00 00 10 27 00 00 24 16 00 00 10 27 00 00 EB 03 00 00

I save the .RAW file as a copy and run this copy with replaced footer through the RAW2DNG converter. It says it is going to convert over 17,000 frames for 4GB of footage. FML. I know that's not right. So I give a rough estimate and stop it at 1200 frames. When I open the DNG's the pink stuff is all gone and there is my footage! I was beyond happy. I did many more converts like this and kept cutting it off at around 1200 frames. I started to notice that the Pink noise comes back in at exactly frame number 1182 every time.

I'm not good at any of this hex code stuff so using this workflow is fine with me, I just have to cut every corrupt file off at around 1200 frames using the above footer. When using the exact 4gb corrupt file footer from the "How to replace file footer thread" it gives me pink frames but when I use the footer above it creates perfect frames. I'm not sure why this is but I know that footer above works if anyone is having trouble or deleted the .R00 files on accident and still want to use the 3.99Gb file.

Here are some stills from the student film we shot. Thanks to Magic Lantern and everyone working on this! It has been awesome to get to learn and use this Hack.

RAW No CC straight from camera, Film Grain Added.

(http://farm3.staticflickr.com/2878/10009996905_bf65695a6f_b.jpg)

(http://farm4.staticflickr.com/3711/10010981473_c15e29e7bf_b.jpg)

(http://farm4.staticflickr.com/3785/10011108686_3a3bfe2a9c_b.jpg)

Quote from: adityakapur on September 26, 2013, 09:36:19 PM
I followed this workflow.

I could not salvage my frames. Can someone please help? My best shot was on that file. All the frames come out pink except the first.

I have two files, a .RAW (4.29 GB) and a .R00 (2.73 GB). Both are spitting out pink frames through RawMagic.

I used Hex Fiend as for some reason with 0xED I am unable to copy paste 2.73 GB of data onto the clipboard. I'm using a MAC.

@adityakapur

Use the program called terminal. It is the equivalent to command prompt for windows. Use terminal to merge the .RAW file with the .R00 file using this code "cat Blah.RAW Blah.R00 > Blah-Merge.RAW" Terminal should already be installed on your Mac. Replace "Blah" with whatever the file names are. Also, don't use the " in the code. This should work like a charm and merge your files. Once they are merged it should run through any RAW to DNG converter. If not replace the file footer using @pedrosuarez method by creating your own footer.

I've tried the workflow suggested by pedrosuarez a number of times with no success.

First I tried with already merged .RAW files.  I merged these using the Terminal method as outlined here: http://www.youtube.com/watch?v=KcpAmII7hMs

I then tried using the original .RAW, .R00, .R01, .R02 files as described by pedrosuarez. 

All I get is crazy pink DNGs!!!

Any help would be deeply appreciated, I would also be willing to Paypal someone to work this through with me, very very important clips.

I should also mention that I never had the "Error: This ain't a lv_rec RAW file" for my clips. I drag them to raw2dng (on MAC) and they create a folder for the DNGs but nothing is created within there, no ProRes either.
The files were cut short when the camera automatically shut off at 4mins.  They are 15.64GB in size, I worked this out to be 6160 frames.
Thanks

One final thought is that maybe I'm replacing the wrong code with the footer?
I'm using HexFiend and replacing the exact number of lines that the footer pedrosuarez mentioned has.  So I paste the footer from this thread over the last few lines of the my file's hex.  Should I maybe be just be pasting pedrosuarez's footer after the existing hex, not replacing it?

Don't replace the Hex. I did that at first too, it was confusing because it does say "replace file footer" But adding it to the end of the Hex is what you do.

Hope this works for you.

EDIT* I would use this method on the merged file you created in Terminal. Just create your own code for your file size and paste it at the end of the HEX. Don't replace it.

Thanks ifleeter.
I tried this but alas, no difference!

The bottom of my hex in some cases seems to be very damaged, with row after row looking like this:
FF FF FF FF FF FF DF FF FD FF FF FF FF FF FF FF FF FD FF FF FF FF FF FF FF FF 7F FF FF FD FE FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF DF FF FF FF FF FF FF FF FF F7 FF FF FF FF FF FF F7 FF FF FF FF FF FD FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FD FF FF FF FF FF FF FF FF FF FF FF 7F 7F FF FF FF FF FF FF FF FF FF FF FF FF FF FF FD FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 7F FF FF FF FF FF FF FD

The last part with a variety of hex (before the above part) looks like this:
C1 71 C7 CD 1C 17 71 D0 CA 61 12 C7 80 1C 81 70 87 C7 1C 0E 71 58 CB 81 09 47 E0 1C 31 71 07 CE 1C 1D 71 DC CB 61 13 47 54 1D 41 73 07 E4 1E 3A 75 7C E7 11 54 C7 B0 1E 11 75 47 E4 1E 44 74 38 DF 21 3C 87 C4 1D F1 72 07 E3 1D 33 72 98 DB 91 29 C7 B0 1D 61 72 47 E0 1D 31 72 80 D5 81 27 87 90 1D 01 73 07 D5 1D 16 71 78 D1 F1 27 87 84 1D 61 71 C7 D7 1D 1B 71 40 D3 A1 1A 07 2C 1D E1 71 47 D4 1D 19 72 48 D8 21 1F 47 B0 1D 11 72 47 D9 1D 23 71 80 DD B1 2C 47 C8 1D 61 73 C7 D9 1E 42 73 08 E1 31 44 07 1C 1E A1 73 47 E7 1E 50 73 48 DE B1 30 C7 FC 1D 31 73 07 DE 1E 28 73 84 E9 41 2B C7 A0 1E 01 73 47 E6 1D 28 71 D4 D9 D1 1E 47 A5 A5 A5 A5

The code I'm pasting in for a 15.64GB file (6160 frames, hex 1810) is this:
52 41 57 4D 80 07 38 04 00 60 37 00 10 18 00 00 01 00 00 00 A8 5D 00 00 48 48 04 00 00 00 00 00 01 00 00 00 00 20 15 0B 26 05 00 00 20 08 00 00 38 0E 00 00 50 34 49 00 0E 00 00 00 FF 07 00 00 98 3A 00 00 00 00 00 00 00 00 00 00 88 07 00 00 08 05 00 00 1E 00 00 00 92 00 00 00 26 05 00 00 1A 08 00 00 00 00 00 00 00 00 00 00 00 01 01 02 01 00 00 00 42 1A 00 00 10 27 00 00 85 FD FF FF 10 27 00 00 3D FC FF FF 10 27 00 00 41 EF FF FF 10 27 00 00 AC 30 00 00 10 27 00 00 EC 07 00 00 10 27 00 00 74 FC FF FF 10 27 00 00 72 08 00 00 10 27 00 00 24 16 00 00 10 27 00 00 EB 03 00 00

Any more pointers?  These pink DNGs are driving me crazy!!!! Thanks again.

Oh, also I'm shooting on 5D2 in 2.35:1 so my DNGs seem to all be 2.6MB.  Although the pink DNGs are 3.7MB.

Spring Pictures

Is there anyway you could send me your original .RAW file so I could work on it? just give me the 4GB one to work on for now, if you have gmail use googledrive and share it with me, my email is [email protected]

can you paste a code for long 1920x1038 (1.85) files? it would be really nice! thanks a lot.

Shot a bunch of Magic Lantern Raw the other night and am getting the Pink DNG's after the first raw file in all the "spanned" files.  So the first 4GB is fine and then the rest I have the pink dots.  I have tried to edit the footers on my PC and couldn't find a program that would copy all the footers as the memory was not large enough and the crappy Mac I had froze up when I used the program above. 

Questions:

1.  Do I just need a more powerful mac to be able to combine the HEX files?  Why would it freeze up 0Xed?

2.  Is there a PC program that would let me copy over all the files without saying I don't have enough memory to copy over?

3.  Is this an issue that was created by me shooting in the cold (20 degrees F) or is Magic Lantern just not correctly installed?  Have shot using Raw before with no issues...

I was able to correct the hex files as listed above, but the image is still not fixed, it did get SLIGHTLY better, but not to anywhere near the extent needed.  Anyone have any idea how to fix the pink DNG files if the above method doesn't work?  The first 4GB of each file is perfect so I have to think it has something to do with the spanning....

@zach915m Instead of using 0xED(MAC) or HXD (PC) to combine .RAW with .R00 etc.. use command prompt (PC) or Terminal (MAC)

Command Prompt Code: copy /b "F:M18-0340.RAW"+"F:M18-0340.R00" "F:M18-0340-merged.RAW"

Terminal Code: cat M18-0340.RAW M18-0340.R00 > M18-0340-merged.RAW

***(Change code to match your information.)***
Hope this help.

-Ryan

Thank you for this thread.

I have the same problem, and I'm trying to use the Terminal solution which seems to work.

https://www.youtube.com/watch?v=KcpAmII7hMs

Okay, so here's my problem.
I shot a videoclip on my 5D mark III in 1920p 2.35 with 48fps (FPS override in 720p60 setting)
The card went corrupt (camera was showing the card was not compatible with the camera or cannot be used) and when i connected it to my iMac it said the card had to be formatted or ejected.
So with the help of a few recovery programs i got some files of the card but they were .SWF files and with another program i got one big .gz file.
Now i tried using 0xED to add the footer but i'm just doing something i have 0 knowledge of.
I tried it and renamed the file to end with .RAW and while RAWmagic now converts the file i only get the pink crazy DNG files.
Can anybody explain to me what it is that i really need to replace or add to the file?
I'm using one .SWF file tot try it out on and that file is 354,1 MB.
If anybody could help me i would be really happy!

I have a python script that attempts to extract usable .RAW files from this kind of corrupted/recovered partial RAW files.

You can get it here, but you'll need a python runtime to run it (built in on mac, get from http://www.python.org/ for windows).

Script: https://bitbucket.org/baldand/mlrawviewer/raw/master/tools/rawcover.py (https://bitbucket.org/baldand/mlrawviewer/raw/master/tools/rawcover.py)

Do:

python rawcover.py <dir_containing_raw_files_or_one_raw_file> [<dir_for_recovered_raw_file>]

It will recover everything it can find, including sometimes partial frames that have been chopped in half.

Good luck!

This script is VERY useful for my corrupted card.
It saved all my data, except for 2 of 52 videos, which came out pink.

Great script! :)

I'm having the same problem on a 50D i recorded some videos and three of them came out with pink crazyness. I can join the files and extract the dngs but some of the dngs are pink noise with recognizable shapes. I have tried the footer hex fix in several different ways and nothing. Always the same dngs came out corrupted. Im going to try the python script and see the results....

Yes  8) your script worked perfectly. It made some files and inside them the good files. Some of them came out with bandin but at least i could recover most of them. Thanks a lot!

Quote from: baldand on February 20, 2014, 05:08:22 PM
I have a python script that attempts to extract usable .RAW files from this kind of corrupted/recovered partial RAW files.

You can get it here, but you'll need a python runtime to run it (built in on mac, get from http://www.python.org/ for windows).

Script: https://bitbucket.org/baldand/mlrawviewer/raw/master/tools/rawcover.py (https://bitbucket.org/baldand/mlrawviewer/raw/master/tools/rawcover.py)

Do:

python rawcover.py <dir_containing_raw_files_or_one_raw_file> [<dir_for_recovered_raw_file>]

It will recover everything it can find, including sometimes partial frames that have been chopped in half.

Good luck!

I successfully recovered a RAW file consisting of multiple parts using the above script. The .RAW file and the .R01 file recovered fine. However, the .R00 file threw an error. This is the output in the command line window:

e:\Users\Me>C:\Python27\python.exe e:\Users\Me\Desktop\rawcover.py E:\Users\Me\Videos\ZG\100CANON\M14-1911.R00
Source: E:\Users\Me\Videos\ZG\100CANON\M14-1911.R00 Target: E:\Users\Me\Videos\ZG\100CANON
Attempting to recover: E:\Users\Me\Videos\ZG\100CANON\M14-1911.R00
File length: 4294967295
Scanning file for Magic Lantern RAW headers.
0%.. 0%.. 0%.. 1%.. 1%.. 1%.. 2%.. 2%.. 3%.. 3%.. 3%.. 4%.. 4%.. 5%.. 5%.. 5%..
6%.. 6%.. 7%.. 7%.. 7%.. 8%.. 8%.. 8%.. 9%.. 9%.. 10%.. 10%.. 10%.. 11%.. 11%..
12%.. 12%.. 12%.. 13%.. 13%.. 14%.. 14%.. 14%.. 15%.. 15%.. 16%.. 16%.. 16%.. 17
%.. 17%.. 17%.. 18%.. 18%.. 19%.. 19%.. 19%.. 20%.. 20%.. 21%.. 21%.. 21%.. 22%.
. 22%.. 23%.. 23%.. 23%.. 24%.. 24%.. 25%.. 25%.. 25%.. 26%.. 26%.. 26%.. 27%..
27%.. 28%.. 28%.. 28%.. 29%.. 29%.. 30%.. 30%.. 30%.. 31%.. 31%.. 32%.. 32%.. 32
%.. 33%.. 33%.. 33%.. 34%.. 34%.. 35%.. 35%.. 35%.. 36%.. 36%.. 37%.. 37%.. 37%.
. 38%.. 38%.. 39%.. 39%.. 39%.. 40%.. 40%.. 41%.. 41%.. 41%.. 42%.. 42%.. 42%..
43%.. 43%.. 44%.. 44%.. 44%.. 45%.. 45%.. 46%.. 46%.. 46%.. 47%.. 47%.. 48%.. 48
%.. 48%.. 49%.. 49%.. 50%.. 50%.. 50%.. 51%.. 51%.. 51%.. 52%.. 52%.. 53%.. 53%.
. 53%.. 54%.. 54%.. 55%.. 55%.. 55%.. 56%.. 56%.. 57%.. 57%.. 57%.. 58%.. 58%..
58%.. 59%.. 59%.. 60%.. 60%.. 60%.. 61%.. 61%.. 62%.. 62%.. 62%.. 63%.. 63%.. 64
%.. 64%.. 64%.. 65%.. 65%.. 66%.. 66%.. 66%.. 67%.. 67%.. 67%.. 68%.. 68%.. 69%.
. 69%.. 69%.. 70%.. 70%.. 71%.. 71%.. 71%.. 72%.. 72%.. 73%.. 73%.. 73%.. 74%..
74%.. 75%.. 75%.. 75%.. 76%.. 76%.. 76%.. 77%.. 77%.. 78%.. 78%.. 78%.. 79%.. 79
%.. 80%.. 80%.. 80%.. 81%.. 81%.. 82%.. 82%.. 82%.. 83%.. 83%.. 83%.. 84%.. 84%.
. 85%.. 85%.. 85%.. 86%.. 86%.. 87%.. 87%.. 87%.. 88%.. 88%.. 89%.. 89%.. 89%..
90%.. 90%.. 91%.. 91%.. 91%.. 92%.. 92%.. 92%.. 93%.. 93%.. 94%.. 94%.. 94%.. 95
%.. 95%.. 96%.. 96%.. 96%.. 97%.. 97%.. 98%.. 98%.. 98%.. 99%.. 99%..
No Magic Lantern RAW headers found.
Unknown section from 0 to 4294967295
start 0
Found possible bayer frame
Traceback (most recent call last):
  File "e:\Users\Me\Desktop\rawcover.py", line 489, in <module>
    sys.exit(main())
  File "e:\Users\Me\Desktop\rawcover.py", line 486, in main
    recover_file(source,target)
  File "e:\Users\Me\Desktop\rawcover.py", line 458, in recover_file
    rescue_norawm(f,us,ue,fn,rn)
  File "e:\Users\Me\Desktop\rawcover.py", line 307, in rescue_norawm
    f.seek(check)
IOError: [Errno 22] Invalid argument

Does anyone know what this error means? It seems like a program crash.. however the error is consistent with the .R00 file and the script worked with the other files so I have little reason to doubt the code of the script.

@Jip-Hop try merging the files together and rescuing the merged file.

Quote from: casuso on May 16, 2014, 11:46:34 PM
@Jip-Hop try merging the files together and rescuing the merged file.

Thanks for the great tip! I merged the files with the following command in a command line window:

copy /b "C:\...\blah.RAW"+"C:\...\blah.R00"+"C:\...\blah.R01"+...+"C:\...\blah.RXX" "C:\...\blah-merged.RAW"

And after that I was able to recover the raw file with rawcover.py :)

I have several files capped at 4.29 GB, and I've tried merging them and then using the Python script to rescue them, but only some of them work.  When I run the python script the files that don't work have an incorrect estimated frame height (less than 1080).  Should I run the hex code on these files before merging them? Any help would be greatly appreciated!!!

I recently finished shooting a short film with a 5d mark iii only to realize all shots (around 30) that ran until the camera stops, or to 3.99 gbs were unable to convert to cdng. At the time we filmed this, I didn't realize shooting in exFat would take care of the stoppage problem. Raw2Cdng, which I'm running through a windows OS, recognized the frame rate as an outrageously large number and would quit working every time I tried to convert it. I then grabbed a .R00 file from a previous shoot in which used a mark ii and attached it to the .RAW file and sure enough Raw2Cdng recognized it as a 23.976 frame rate. I was then able to convert them and thought I solved my problem. Turns out the all of the frames were crazy pink dngs. Using hexfiend and realizing that our hex code was 2 digits short as "pedrosuarez" had mentioned, we tried the workflow suggested, by calculating a footer and inserting it into hex fiend.

We calculated the footer for only the 3.99 gb RAW file. 3.99 * 1024 = 4,085.76/2.82 mb frames = 1,448.85106 or 1,449. After putting it into the demical to hex converter we were given 5a9 or 09 5A in place of XX XX.

Using the code suggested by "pedrosuarez" we used this as our footer:

52 41 57 4D 80 07 38 04 00 60 37 00 09 5A  00 00 01 00 00 00 A8 5D 00 00 48 48 04 00 00 00 00 00 01 00 00 00 00 20 15 0B 26 05 00 00 20 08 00 00 38 0E 00 00 50 34 49 00 0E 00 00 00 FF 07 00 00 98 3A 00 00 00 00 00 00 00 00 00 00 88 07 00 00 08 05 00 00 1E 00 00 00 92 00 00 00 26 05 00 00 1A 08 00 00 00 00 00 00 00 00 00 00 00 01 01 02 01 00 00 00 42 1A 00 00 10 27 00 00 85 FD FF FF 10 27 00 00 3D FC FF FF 10 27 00 00 41 EF FF FF 10 27 00 00 AC 30 00 00 10 27 00 00 EC 07 00 00 10 27 00 00 74 FC FF FF 10 27 00 00 72 08 00 00 10 27 00 00 24 16 00 00 10 27 00 00 EB 03 00 00

Raw2Cdng recognized the file now as the correct frame rate and converted it without having to attach a .R00 file as we did previously. Raw2Cdng said that it was around 27,000 frames, which obviously wasn't right, but since we noticed ifleeter had this problem we were just going to cut it off around the correct frame amount as he did.

Our problem, is the files still remained pink and fuzzy. Some frames were a little more decipherable than they were previously, but not where they needed to be.

This makes me think I am on the right track, but somewhere I went wrong in the conversion or calculating process.

I was hoping someone could shed light on my problem or suggest any solutions. I could also share the 3.99 gb with you via google drive if that would make things easier. I know nothing of code and have tried figuring it out thru forums and my own skills, but have consistently failed.

Anything would be greatly appreciated.

Also shooting a film and had the same pink problem, that´s what I did to solve it, it´s quite simple:

You need:
- Any non-corrupt file shot with the same settings
-  A calculator


1) First, open the non-corrupt file with the hex editor and copy the footer. At least in my case, it was about the same length of the one posted here. I recognized it because, as the example posted here, it started with: "52 41 57 4D"

2) Second, extract the DNGs from the non-corrupt file to know how big that files are.

3) Then you have to calculate how many frames (approx.) you want extracted from the clip.
I did the following math:

( (4.29 * number of split-files in GB) + size of the corrupt file in GB ) * 1024 * 1.1463 / size of a single DNG file

Explanation:
- (4.29 * number of split-files in GB) = if you have your files split because you shot in a FAT card, you ´ll end having FILE.R00, FILE.R01, FILE.RAW etc. This is just to avoid the step of merging the files, as some people suggest.

- Size of the corrupt file in GB:   The .RAW is the first file, and the highest .R0 is the last (so, that´s the corrupt file and the one you need).

- * 1024 (from GB to MB duh!)

- * 1.1463 . This is how much bigger my files are after conversion from RAW to DNG. I get about 14% bigger files after the extraction, I would advise you do your own calculation with the non-corrupt file, as it may vary if you use different settings (I have no idea)

- size of a single DNG file: the size of a single frame.

4) Convert the number of frames you got for that file size to Hexadecimal
http://easycalculation.com/decimal-converter.php

5) Replace the position in the footer with the correct hex number you now have and paste the footer in the end of your corrupt file.
The previous posts explain the right place to put it, the 13th position (there must be a better term for this) in the footer.
Remember it´s Little Endian, so you have to invert the position of the numbers in the footer!

For example:
You got A3 you write A3 00
you got FD4A you write 4A FD
you got 6C2 you write C2 06

No idea what to do if you have more than FFFF frames (45 min at 24 fps). I guess you could test writing it after or before.

6) Done, extract with your favorite raw to dng converter.
If you feel like you are missing some frames because the math isn´t exact, you can always add frames to the footer (haven´t tested yet, but I guess it will extract some garbage after the good frames finish.



Thanks everybody for the previous posts, I wouldn´t have no idea what to do without them! :)

totally saved my footage on a rollout. this totally works. thanks pedrosuarez.

I feel like I'm close to fixing my clips but am getting hung up on a couple things....

Quote from: Dareios on July 31, 2014, 11:13:32 PM
1) First, open the non-corrupt file with the hex editor and copy the footer. At least in my case, it was about the same length of the one posted here. I recognized it because, as the example posted here, it started with: "52 41 57 4D"

How do I find the footer if it doesn't start with "52 41 57 4D"?

Quote from: Dareios on July 31, 2014, 11:13:32 PM
5) Replace the position in the footer with the correct hex number you now have and paste the footer in the end of your corrupt file.
The previous posts explain the right place to put it, the 13th position (there must be a better term for this) in the footer.

Am I alway replacing the 13th position with the Hexadecimal determined by Step 3 and 4 in Dareios post?

Greetings people ...
I have a problem with multiple MLV files

using the build (BUILD = magiclantern-Nightly.2014Aug07.5D3123)

I'm using MLV Converter 1.9.1, 1.4.2 and Mlrawviewer

there any chance of recovering those files?
Could someone help me, PLEASE?

screenshoot:
(http://i61.tinypic.com/2uge4c6.jpg)

DNG: https://www.dropbox.com/s/4kavri9vaxhl87d/MLV_A001_101233_C00477.000000.dng?dl=0
MLV FILE: https://www.dropbox.com/s/zalvkps6va5dskt/M10-1234.MLV?dl=0

Hi guys.

First of all I'm not sure if I have the same issue as the rest of you, but too seeing crazy pink corrupted frames.
I recorded this on the latest Nightly (magiclantern-Nightly.2015May03.7D203) on my Canon 7D on a Transcend 1000x 64GB.
For more details on settings check the images. The RAW files with corrupted frames are mostly in the beginning, and the later recordings the same shoot are all free from pink frames.
To recover the files I've tried the rawcover.py script, holding my thumbs, however the file came out as corrupted as the input.
I can see that most of the information seem to be in the image, although offsetted and pink, I can imagine it would be possible to restore frame-by-frame....
Any ideas what can be done to restore? And further to avoid the same issue from happning again (i didnt see any warning about dropped or corrupt frames while recording).

http://i.imgur.com/34PwBYB.jpg
http://i.imgur.com/MhgOS1w.jpg
http://i.imgur.com/CM3EeGy.jpg
http://i.imgur.com/BEs9CLr.jpg

Thankful for any input here!

Cheers,

Any update on this?