Failure Trends in a Large Disk Drive Population

This detailed measurement study looks at more than 100,000 disk drives at Google that were monitored throughout their lifetime to quantify failure rates and attempt to identify warning signs for failure. The most interesting aspect of this study, which also sets it apart from similar studies, is methodology: Google built a lightweight health monitoring system at least 5 years prior to this paper, collected monitoring data into a large-scale data store (BigTable), and analyzed it using large-scale analysis tools (MapReduce/Sawzall). For a data center operator, one lesson from this paper is that it may help to monitor not only software but also hardware you purchase in order to guide future purchasing and design decisions.

The findings are fairly detailed, but the most surprising ones are:

• Heat seems to have little adverse effect on drives, and in fact higher temperatures seem to help reduce failures up to a point.
• Utilization also has little adverse effect.
• While SMART parameters are correlated with failures, they are not enough to predict failures with reasonable reliability.

It is also interesting how many weird types of spurious data points were found, such as drives reporting higher temperatures than the surface of the sun. Any large-scale statistical analysis needs to watch out for such errors in the data/monitoring.

One potential long-term impact of this work is that it encourages more cloud vendors to collect health databases about various computer components. With companies the size of Google collecting such data, users get a better picture of real-world reliability of the hardware than from manufacturer testing or manufacturer problem databases.