Flash memory and storing data for the long term

2013-12-30

in Geek stuff, Internet matters

I didn’t know this about flash memory:

Flash memory is really cheap. So cheap, in fact, that it’s too good to be true. In reality, all flash memory is riddled with defects — without exception. The illusion of a contiguous, reliable storage media is crafted through sophisticated error correction and bad block management functions. This is the result of a constant arms race between the engineers and mother nature; with every fabrication process shrink, memory becomes cheaper but more unreliable. Likewise, with every generation, the engineers come up with more sophisticated and complicated algorithms to compensate for mother nature’s propensity for entropy and randomness at the atomic scale.

These algorithms are too complicated and too device-specific to be run at the application or OS level, and so it turns out that every flash memory disk ships with a reasonably powerful microcontroller to run a custom set of disk abstraction algorithms. Even the diminutive microSD card contains not one, but at least two chips — a controller, and at least one flash chip (high density cards will stack multiple flash die).

It reinforces the point that we really have no technology for long-term data storage. Hard drives fail, burned CDs and DVDs likewise. Paper is enduring.

Even backup systems like Apple’s Time Machine have problems. If a file gets corrupted on your hard drive, Time Machine will start backing up corrupted copies, eventually over-writing the good ones. What’s really needed is a system that makes a hash of the files to be backed up and stores distinct copies of all modified versions. Of course, that could require a lot more storage space – especially if the files in question are something like videos being edited.

Report a typo or inaccuracy

{ 1 comment… read it below or add one }

. December 31, 2013 at 3:43 pm

“From the security perspective, our findings indicate that even though memory cards look inert, they run a body of code that can be modified to perform a class of MITM attacks that could be difficult to detect; there is no standard protocol or method to inspect and attest to the contents of the code running on the memory card’s microcontroller. Those in high-risk, high-sensitivity situations should assume that a “secure-erase” of a card is insufficient to guarantee the complete erasure of sensitive data. Therefore, it’s recommended to dispose of memory cards through total physical destruction (e.g., grind it up with a mortar and pestle). “

Leave a Comment

You can use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Previous post:

Next post: