Long-term power-off data retention

[Quailane]

Hi. IÂ’ve been doing some research into this but there doesnÂ’t seem to be much definitive info. IÂ’m looking for the best HDD or SSD for long-term data retention. I know that SLC is better than MLC by an order of magnitude at least. Manufacturers stopped publishing data retention longevity a while ago as they moved to smaller processes and MLC TLC and QLC. The less cycles the NAND has gone through the better. Micron P300 SLC seems perfect for me because they state 10 years of data retention when new, and 1 year when write-cycle limit is reached. Unfortunately there are no longer any new ones and there is a trick to re-writing their SMART data so you canÂ’t trust them. I bought a used Samsung 860 pro before I learned that write cycles are important to this. I also have a Samsung 830 that booted Windows 7 and everything was there and functional just like the day it was previously powered on 5 years prior. IÂ’ve also read that smaller flash manufacturing processes do worse than older larger manufacturing processes. If anyone could help me out I would be grateful.

 

[David]

If you're looking for data retention on the order of tens of years I think that any SSD would carry more risks that either some sort of cloud storage provider or tape backup. Depending on your exact use case it might be worth looking for a tape drive rather than an SSD.

 

[Quailane]

I use iCloud which is fine, but I want to be self-reliant with backups and not depend on me paying the monthly fee forever. IÂ’m not looking for decades, but 10 years would be great! What IÂ’m trying to figure out is if newer SSDs have worse data retention performance than older SSDs. My Samsung 830 seemed to last 5 years at least. Would I be better off buying the least used Samsung 830 drive I can find or buying a new Samsung 860 pro? Or even a few Intel X25-E or a couple of Micron P300? I just want to know if an SSD is better than a magnetic hard drive and what SSD I should be looking for since the manufacturers donÂ’t list this sort of stuff.

 

[Janus67]

https://www.reddit.com/r/DataHoarder/comments/p0pxxk

Unsure if that is also your reddit account, but seems right up your alley for a thread.

 

[EarthDog]

At this point in time, for cold storage/backups, I would only look towards a HDD.

 

[Quailane]

At this point in time, for cold storage/backups, I would only look towards a HDD.

So I've been doing a lot of research lately because this has been bugging me a lot. Trying to search for anything about this topic is a bit tough. Most of the information out there is at least 5 years old. With cloud storage being the norm for long term reliability for consumers, and data centers favoring cheap storage space, power-off data retention is kind of ignored these days. This is rarely an issue or selling point for people nowadays, and it seemed to stem mostly from past concerns over SSD reliability which people don't care about much anymore.

However, I have been able to reach some conclusions that are quite surprising. JEDEC standards, some articles by independent authors, and data sheets released in the past by some manufacturers have allowed me to piece together a good-enough idea about data retention and how to maximize it. It's all about how quickly electrons leak from NAND cell and how difficult it is to read the correct voltage in NAND cells. A new MLC drive with zero write cycles should hold the data for at least 10 years under normal conditions. Write cycles, bits per cell, storage temperature, and reading temperature are all very important factors in how long the data will last, and all under your control. So, if you buy a new, unused MLC drive, write data to it once, keep it at 25 degrees Celsius (or below?), and then warm it up to at least 40 degrees Celsius to read it, it will be pretty much guaranteed to last 10+ years without power. I wouldn't rely on that, but it is good from a fail-safe point of view.

There are three variables I can't account for however. Those are how good the controller is at reading NAND cells, which should be expected to increase over time, how many electrons are stored in each NAND cell, which goes down over time as manufacturing processes get smaller, and how the design and manufacturing quality of NAND cells, which should hopefully increase over time.

Because it's solid state, as long as there aren't any manufacturing defects or it gets physically damaged, there should be a greater chance of the drive working again if it gets connected to power than say a mechanical drive.

One interesting fact is that newer flash memory has 100 or less electrons per cell. In QLC you can imagine it doesn't take many electrons to leak from a worn memory cell for the bits there to be permanently corrupted. The controllers are better than ever, but they need to be kept powered on in order for them to work their magic in retaining the data on the drive.

https://www.reddit.com/r/DataHoarder/comments/p0pxxk

Unsure if that is also your reddit account, but seems right up your alley for a thread.

It's not. That's just a coincidence we asked a similar question at the same time. I do know HTWingut though. Used to argue with him a lot many years ago.

 

[EarthDog]

Thanks for the info! That's good stuff!

Still wondering why you want to use 'fast' (read: $$$) storage compared to a HDD that's a lot cheaper/GB and doesn't have this worry. HDD, cloud, tape.

 

[Quailane]

Thanks for the info! That's good stuff!

Still wondering why you want to use 'fast' (read: $$$) storage compared to a HDD that's a lot cheaper/GB and doesn't have this worry. HDD, cloud, tape.

There are a couple of reasons.

First is that I'm looking to see which, in the most favorable situation, can retain data for the longest amount of time regardless of anything else. From what I've been able to deduce, I think it's definitely that the best SSDs in the best circumstances can retain data for longer than the best mechanical drives in the best circumstances. Take for example the Micron P300 SLC drive. It is rated at 10 years of data retention in normal operation at zero to low PE cycles and under the very conservative enterprise ratings. That is with storing it at higher than lower storage temperatures which can theoretically boost data retention time by 10x if the equations JEDEC uses are correct. The JEDEC graphs relating to storage temperature don't even go past 10 years because nobody really expects the drives to remain in service so long. The same equations and tests that show SLC enterprise drives retaining their data for only a few weeks in the worst conditions point to them being able to retain the data for all practical purposes, indefinitely under ideal conditions. Obviously that isn't testable, but assuming at least 10 years for fresh SLC and MLC under ideal conditions seems to be pretty safe and is even more conservative than what JEDEC says. TLC can't do that and QLC for sure can't do that though. I think this really makes it safe to assume that an SSD is capable of outlasting a mechanical drive in keeping the data. My failure rate for old mechanical drives that sit around for years is pretty high. I luckily haven't had an SSD fail on me yet, although an old TLC Samsung PM851 msata I have is almost worthless because of how slow it is. I also have experience with data corruption on mechanical drives that sit around too long. It happens very slowly, but I wouldn't want to go more than 2 years without re-writing the data to the drive, to be on the safe side.

The second reason is that if you do want to keep the data on the drive for a long time, the upkeep for the SSD is a lot less demanding. All you need to do is to connect it to power and it should be able to figure out it needs to refresh the NAND cells all by itself. A mechanical drive can't do that and it must be connected to a computer or at least a drive cloner to manually re-write everything onto the drive. I'm thinking that if something happens and I can't access the drives or whatever, someone would be able to just find the long-lost drive sitting in my cool basement and would be able to plug it in.

There is also M-Disc which I've thought of and might as well look into as well, but I really like just plugging in a drive an having everything easily accessible, quickly readable, and modifiable. Who knows, maybe this is the option I should go with though. A 128GB MLC drive isn't too pricey. It's enough to save my most cherished family photos, including very old digitized photographs, some compressed family films, all the school work I've ever done, important documents, etc. I'm thinking that I can buy a couple of cheap small 2.5" RAID 1 cases and put hard drives in one, and SSDs in the other. I can do the M-Disc too and throw them all in a box in to keep in my mother's basement. That should be enough peace of mind to assume that at least one of them will survive. And yeah, part of it is just for fun because I'm a nerd and it shouldn't be too expensive overall.

 

[EarthDog]

can retain data for the longest amount of time regardless of anything else.

Is that so? I don't recall hearing that in the past.

As far as storage temps go, they're in your house, right? Not a garage, attic, storage shed, etc? So does storage temps really matter when it's going to be well within the range of both?

The second reason is that if you do want to keep the data on the drive for a long time, the upkeep for the SSD is a lot less demanding. All you need to do is

Do you need to plug a HDD in? If so, why is it any different than an SSD? I'm not sure why you're talking about refreshing and cloning, etc. It's a HDD and doesn't need that. I've got a HDD sitting in a closet for cold storage. I break it out annually and plug it into a system and add to it as needed (I have cold-ish storage active on my PC).

That said, your cold storage is TINY. If what you want to keep fits on a 128GB drive, then it makes more sense to go SSD. But cold storage is the job of HDD, Cloud and Tape in most cases.

 

[Quailane]

Is that so? I don't recall hearing that in the past.

It's true. But whereas the mechanical drive is extremely predictable and doesn't change unless you play with a lot of magnets around it, the SSD's data retention is highly variable based on many factors. In almost all cases the HDD is better at data retention.

As far as storage temps go, they're in your house, right? Not a garage, attic, storage shed, etc? So does storage temps really matter when it's going to be well within the range of both?

The lower the temperature, the better. SSD data retention is usually measured at warm temperatures because that is the normal operating environment. For example, the Samsung 870 EVO in my Mac Mini never goes below 40 degrees. Being TLC at those temperatures, I'm guessing that the controller must be re-writing the data at least once a month in order to retain the data. Going from 25 to 30 degree storage temperature cuts the length of powered off data retention in half. JEDEC only talks about 25-55 degree use for consumer and enterprise users because those are what is reasonable to expect.

Do you need to plug a HDD in? If so, why is it any different than an SSD? I'm not sure why you're talking about refreshing and cloning, etc. It's a HDD and doesn't need that. I've got a HDD sitting in a closet for cold storage. I break it out annually and plug it into a system and add to it as needed (I have cold-ish storage active on my PC).

The data on a HDD slowly goes bad just like on an SSD. Sure, it is slow, but it happens, and accelerates with time. It mostly depends on your tolerance for bad bits. Enterprise SSD use has a much lower tolerance for bit errors so their data retention is rated at 1/4 that of consumer drives which have an order of magnitude greater tolerance for bit errors yet are still very low tolerance. The data retention times are also based on when the drive has gotten through most or all of its P/E cycles which can reduce the data retention length by a factor of 10. If you power on the SSD the controller automatically refreshes any data on the drive that needs it and resets the clock for data retention. A HDD can't do that automatically. The whole reason I was looking into an SSD for this is because I have run into some data retention issues on hard drives kept in a room along with my old laptop that had the old Samsung 830 MLC drive in it. After almost 5 years, there were a few files corrupted on the magnetic hard drives, but everything on the admittedly much smaller SSD all worked fine. I was very surprised by this.

That said, your cold storage is TINY. If what you want to keep fits on a 128GB drive, then it makes more sense to go SSD. But cold storage is the job of HDD, Cloud and Tape in most cases.

I'm just thinking of the most important things to fit into 128GB. At that level the cost difference is negligible. If you need to store a lot of stuff, the cost of the SSD storage will be an order of magnitude greater. Cost is an issue with SSD storage, especially because you need at least a brand new MLC drive and even better an SLC drive.

 

[Mr Alpha]

The second reason is that if you do want to keep the data on the drive for a long time, the upkeep for the SSD is a lot less demanding. All you need to do is to connect it to power and it should be able to figure out it needs to refresh the NAND cells all by itself. A mechanical drive can't do that and it must be connected to a computer or at least a drive cloner to manually re-write everything onto the drive. I'm thinking that if something happens and I can't access the drives or whatever, someone would be able to just find the long-lost drive sitting in my cool basement and would be able to plug it in.

Are you sure about this? Because I am a bit skeptical. I've seen references to background refresh routines in SSDs to refresh the data, but it has more been in relation to read disturb than for passive leakage.

Even assuming it stores a timestamp for last refresh and has an onboard clock to keep track of it, the clock won't be ticking while the drive is powered down, so when you give it power it wont know it has been sitting on a shelf for years. At the very least you would need to connect it to a computer to synchronize the clock.

It may also not be full on timestamp based but rather some simple counter that is ticking away while the SSD is running. In normal usage most of the errors would be introduced while the SSD is running due to read disturb and higher operating temperatures, and that the manufacturer is simply ignoring offline time for the purposes of refresh on the assumption that people won't be using the SSD for archival storage.

Another possibility I would worry is that simply powering the SSD on may not be enough kick off whatever background operations it has. Depending on implementation details of the SSD controller it may require more of a poke to actually get it going.

I would not trust an SSD for long term storage without doing a full drive read every once in a while. That would force the drive to notice any page with a high bit error rate and make it refresh it.