SD cards losing format in EOS M

[scottfrey]

I have been having an ongoing intermittent issue shooting long amounts of video on my EOS Ms

We run Ms to capture video of science lectures. Cameras are set up on AC power and run unattended with video restart enabled.

Every once in a while, maybe  out of four times a card will become unreadable after shooting 90 minutes of video. Everything appears fine in camera. Camera's are powered down (and wait for the lights to stop blinking before disconnecting power and opening the doors). This has happened across 3 different EOS Ms, four different cards, three different power adapters (both Canon and generic) and multiple versions of Magic Lantern (all the way back to Tragic Lantern). I have been unable duplicate the problem with a short amount of video. The cameras are always shut down cleanly, and wait at lease 5-10 seconds before opening the door

The problem is complicated by the fact that the only way to repair the card is in Mac OS X Diskutility. Simple repair and we have been putting up with it for a long time as it's trivial to repair. Until the most recent version of OS X (10.11) and it's new Disk Utility. Which will not repair it at all, and I have tried all sorts of ways. I have found nothing else that will repair the volume on these.

I found the following in the FAQ:

▪After opening the card door, always wait for LED confirmation (or for 5 seconds) before removing the card, even if your camera is turned off!!!
Right after opening the card door, Canon firmware accesses the card without turning on the LED (yes, with the main switch turned off). If you remove the card too early, the camera will freeze and will drain the battery, or even cause permanent damage! You will be running random code (remember you are loading executable code from the card), and we can't do anything about it without reflashing Canon firmware with our own code.

However, on the EOS M, the AC power plug goes through the door. You literally cannot open the door without removing power first.

Since it was a few seconds to fix, I hadn't been too concerned about it. However, now that it has become un-repairable, it's a bigger concern.

Any help would be appreciated.

[a1ex]

I assume you are recording H.264, without any bitrate modification; correct? Can you also upload a screenshot of the Modified settings menu?

It's a little time-consuming to test: the error doesn't happen when running plain Canon firmware, right?

If it doesn't, I can prepare a test build, just with the movie restart functionality, to see if it helps. If you have a clear way to reproduce (maybe by leaving the cameras record overnight on the AC power?), we can narrow it down - it will take a long time, but this looks like bug that results in data loss - so I think it's worth the effort.

I can even automate the testing process a bit - for example, record until card gets full, check if the files are readable, format the card and loop forever.

[scottfrey]

I assume you are recording H.264, without any bitrate modification; correct? Can you also upload a screenshot of the Modified settings menu?

H.264, bit rate set to 1.4 (our compromise between quality and size). crop mode on or off (depending on lens used, it corrupted on one of each this last time), white balance to 3200K. I think those are all the movie settings (can't actually check right now)

It's a little time-consuming to test: the error doesn't happen when running plain Canon firmware, right?

Indeed, it is, as it only happens about 25% of the time, and only after recording at least an hour of video.

It does not happen with the same card on an M3 (one of our M's was down and I used an M3 to cover for a while, PITA, no AC adapter and needed to restart manually)

If it doesn't, I can prepare a test build, just with the movie restart functionality, to see if it helps. If you have a clear way to reproduce (maybe by leaving the cameras record overnight on the AC power?), we can narrow it down - it will take a long time, but this looks like bug that results in data loss - so I think it's worth the effort.

Let me test without bitrate mod and see if that has any effect. I am a bit delayed as I am waiting for response from Apple Engineering on why Disk Utility can no longer repair and I have a currently unreadable card. After that clears I'll see if I can get it to trip with Bit Rate at default.

I can even automate the testing process a bit - for example, record until card gets full, check if the files are readable, format the card and loop forever.

I doubt that would catch it. I have checked the files in play mode seen they are good, powered down and removed card and inserted it into a reader to find it corrupt. It really seems the act of powering down or removing the card is what triggers it.

Can you shed any insight on the FAQ point on ( http://wiki.magiclantern.fm/install ):

After opening the card door, always wait for LED confirmation (or for 5 seconds) before removing the card, even if your camera is turned off!!!

Which of course, with AC power, power is removed before opening the door. SO if there is any cleanup to be done, it won't be done as there is no power. Could whatever cleanup needs to be done be done with a manual process in ML? Is there a way to reboot back into Canon firmware without formatting or removing the card first?

[a1ex]

In this case, I'd say the main suspects are the 1.4 bitrate AND the 3x crop mode.

Regarding the shutdown warning, see http://www.magiclantern.fm/forum/index.php?topic=1455.0
If the camera is powered off, then it doesn't apply here.

If you think it has anything to do with shutdown, the suspects would be:
- ML config saving at shutdown (can be disabled)
- module unloading, or the module backend (to exclude both, check Debug->Modules debug->Disable all modules; simply not loading any modules is not enough)

So, my advice would be to try disabling all these 4 items, and let me know if you still have any corrupted files.

[scottfrey]

OK, Diagnostics away to Apple Engineering.

Since the files seem fine and are viewable before shutdown, I am going to aim at your two shutdown suspects first as more likely.

If you think it has anything to do with shutdown, the suspects would be:
- ML config saving at shutdown (can be disabled)
- module unloading, or the module backend (to exclude both, check Debug->Modules debug->Disable all modules; simply not loading any modules is not enough)

Interesting about the modules, I don't use any, but had not disabled. I have now disabled modules under debug. I have disabled Config Files -> Config Autosave (assuming that is what you meant) and will run a series of 90 minute video tests and see if I can trip the corruption that way. At 90 minuted per test, it takes a long time to accumulate enough results to be confident of success at (roughly) 4 to 1 odds

[scottfrey]

To clarify:

In this case, I'd say the main suspects are the 1.4 bitrate AND the 3x crop mode.

Do you mean running the combination of the two is most likely (as in either by themselves would be fine) or that either is EQUALLY likely to be the culprit?

[a1ex]

Either of them equally likely. I'd say you could try disabling all 4 first - if it works fine, it's one of them; if not, we need to look for something else.

[scottfrey]

OK, disabling all four of those and still corrupting cards.

Is it possible that formatting the card (keep ML enabled) is somehow a cause? We habitually do that. I am going to run some sets without formatting

[a1ex]

Possible, though I use it all the time (on other cameras), and so far it didn't seem to cause any filesystem issues.

I always reboot the camera after formatting, though.

[scottfrey]

I always reboot the camera after formatting, though.

and I never have (to be fair, i've never had any problems like this on any other cameras, or EOS M running on battery power either)

e.t.a.: It's entirely possible that the times that it has seemed to "randomly work" have been when the card is formatted at the end of the night (and the camera shut down) vs. when someone forgot and we format right before we start shooting. which intuition tells me is about the same percentage as having format problems. We could be on to something here.

1st test without reboot passed.

[scottfrey]

Well, it has done 3 tests with erasing images rather than formatting and 2 with formatting and rebooting, all without fail. I also tried to make it fail on a 16GB fard (so not as long a vidoe) and have not been able to trigger the error, so it is either size of card or amount of video files related as well

[a1ex]

So, after all, is the issue related to formatting the cards in the camera without rebooting it before recording?

If yes, here's a possible workaround: https://bitbucket.org/hudson/magic-lantern/pull-requests/736/restart-the-camera-after-formatting-the/diff