Taking out SD Card with Magic Lantern, can ruin RAW photos?

Started by JADURCA, December 06, 2019, 01:05:16 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

JADURCA

Used Magic Lantern version: crop_rec_4k.2018Jul22.5D3123

I was ready for a love story photoshoot, then I took out SD Card were Magic Lantern was installed with camera off, because I would only use it for pictures and no video. Then we completed the photoshooting and for our surprise like 6 RAW files were corrupted. This have something to do because I did not uninstalled Magic Lantern before taking out SD card? Or it was something else?

I uploaded 6 RAW files in case someone want to take a look.

https://drive.google.com/drive/folders/1qBz0e46nyW0swLVZta09akEUP0lYemqL?usp=sharing

Walter Schulz

Uninstalling ML before card removal is full-scale voodoo.
 

JADURCA

Maybe I'm not expressing myself correctly. My main language is Spanish, so there may be errors or nonsense sometimes when I try to communicate.

So I must wait the 60 seconds for Magic Lantern to uninstall to make a healthy camera and operation? Or I can take pictures just taking out SD card with ML installed without problems?

Is there a way to bring those RAW files from the upside down?

a1ex

Short answer: if you remove the ML card and place a non-ML card, the camera will start with vanilla firmware.

There might be a rare exception, where DryOS cannot shutdown completely, so there's a tiny chance the previous ML session was still running (although I've never seen it happening on a card swap). Something similar might have happened here. However, should that have happened, I'm pretty sure you would have noticed it (ML overlays on the screen, or camera stuck), so I'm 99% sure this didn't happen. Taking the battery out after the card swap would completely exclude this exception.

Details about the startup process: https://www.magiclantern.fm/forum/index.php?topic=23641.msg213338#msg213338

I'd rather say this is a card failure. In any case, these defects don't seem to be something that ML could have caused.




Something to try: if you copy the problematic CR2 files from the card, once again, do you get the errors in exactly the same place? If in doubt, upload a second copy of these files, after copying them once again from the affected card. No miracles expected, but who knows.

[ I've got an USB stick performing random bit flips after about 1 year of use; every time I read the files from there (either from the file browser, or with dd), I get a different result, with isolated bits being different. The median value of these affected bits does not converge to the "ground truth"; rather, most of them tend to become "1" as I try to read them over and over. ]

From these files, the embedded JPEG previews are still good, so you can extract them with:

dcraw -e *.CR2


Recovering the raw data is more difficult, but I can imagine some things to try. Unless there's already some kind of LJ92 recovery tool out there, that I'm not aware of :)

Kharak

Might be worth looking in to some Data Recovery software, might even be your Card Manufacturer has such a software.

A long time ago, though not exactly the same scenario, I had a Lexar CF card get completely corrupted for some reason, unable to access it at all. I used the Lexar Recovery Software and was able to get all .cr2 files back. The card contained MLV files on it which the software obviously could not recover, because the software does not know that format.
once you go raw you never go back

a1ex

Your problem was a corrupted filesystem, and your data recovery software figured out where each CR2 file is; that is, your data recovery software re-created the file allocation table based on the contents of the data area (because CR2 files are easy to recognize, both where they start and how long they should be). If you want to perform a similar recovery for MLV files, try PhotoRec.

Not the case here. OP was able to copy the CR2 files from the card, but their content is "slightly" different from what it should be. The difference might be as small as one or two bits flipped, or maybe some more bits, but it's hard to tell exactly which bits or how many. The file size matches the CR2 metadata, so I'd say the storage media (or - less likely - the data transfer to PC) caused these issues. First error in 6E2A5649.CR2 appears to be near 0xbb03ff. In any case, it's not a matter of a corrupted (or incorrectly recovered) file allocation table.

IIRC, the 5D3 is able to save the images on both cards (CF and SD). This feature would have been useful here.

Offtopic: does the CR2 have some kind of checksum? I'm not aware of any. I could think of a possible ML feature, to extend the CR2 file format with ECC codes or other kinds of redundancy, that might help recovering such images in case of similar (bit-flip) failures.

Kharak

Ah okey, thank you for clarifying.

Quote from: a1ex on December 06, 2019, 11:59:15 AM
IIRC, the 5D3 is able to save the images on both cards (CF and SD). This feature would have been useful here.

This, together with ML Card spanning is what caused my Lexar card to get corrupted. To clarify, I set in Canon menu to save to CF and SD and then I set ML Card Spanning too. Then recorded a clip where the camera instantly crashed and the card could not be used. It was me being stupid, I had forgotten how the Card Spanning worked, and thought I had to do it also in Canon menu. The card works today, but has a slightly lower speed and when there is about 5 GB (128 gb) left on it, it have very unstable speeds and the camera will stop recordings.
once you go raw you never go back

a1ex

These two options are independent; this setting - from Canon menu - does not have any effect over MLV recording with card spanning. The latter happens on both cards, no matter what is selected in Canon menu. And even if that wasn't the case, I'd be surprised if writing some files on both cards would cause hardware malfunction.

With regular (non-spanning) MLV recording, on some versions, the Canon menu setting only sets the preference between CF and SD, if both of them are inserted. That's all it does. I can't see any possibility of conflicts here. When you enable card spanning, main file goes on CF, secondary file goes on SD, no matter what you have selected in Canon menu.

Flash storage media does wear out though, but that happens with normal usage. The total amount of data written to that card, during its life time, matters here, not some particular setting in Canon menu, or some random ML crash.

Anecdote: one of my SD cards (an old Lexar) failed while copying ML files on it. The process was slower than usual, then the card could not be accessed, I had to format it, and since then, its capacity was reduced to half (the card resized itself somehow from 512MB to 256MB). It still works today, as a 256MB card.


Kharak

I can't tell exactly what happened or how. This happened years ago and I only reported the issue give or take 2 years later when something similar came up on the forum. But the fact of the matter is, that the only thing out of the ordinary that I did at the time, from having this card working 100% for 2 years or more, was to set Canon recording to both cards and ML Card Spanning on, the moment I hit record the camera crashed and I had a corrupted filesystem, this I do remember. What order I did it in, I don't remember, maybe it was that build containing some weird bug or/and it was my way of operation causing it. Just like I have Error 70 and memory errors on my two 5D3's that no one else has, even though I am running same build that thousands of others are using, somehow my way of operation is causing this to happen it seems... I might add that I don't feel I do anything special during shooting, I shoot a lot but very rarely ultra long takes like filling 128gb cards with one or two clips.
once you go raw you never go back

lorenzo353


JADURCA

Being more specific than the first time. I always use 2 cards, SD with ML and a CompactFlash. When using camera just for pictures, I just take out SD with ML, and then use CompactFlash. When going for video, just insert SD with ML again and then go for MLV RAW.

After those corrupted pictures, I almost took like 1000 pictures the other day without any issue. Maybe because I formatted the CompactFlash it was resolved.

Thanks for all suggestions!