The continued value of HDDs in data centers

Solid-state drives (SSDs) have been much praised in recent years, with SSD manufacturers promoting their accelerated operational speeds and greater robustness compared to hard disk drives (HDDs). SSDs have, as a result supplanted HDDs in many application areas, particularly within the consumer space. When it comes to data center implementations, their value is still very much apparent though.

People may think that HDDs can no longer compete, but they need to look at all aspects when making a proper assessment. If you are putting an SSD with as much as 2500 Gbps of bandwidth and 100kIOPS operational performance up against the fastest HDD (which will be capable of delivering 250Mbps and 300IOPS), the benefits may seem clear… but wait a second. In data centers, we are not simply talking about systems comprising one single SSD versus one single HDD, like we would be in a laptop. So this comparison is invalid. For modern storage dominated applications, there can be hundreds of these units involved – potentially representing petabytes of storage capacity.

Drive on
For large-scale high-capacity storage installations, the price/Gigabyte is the key parameter that needs to be considered when making an investment – and here HDDs are still proving they have superiority. For the past decade, there has been an order of magnitude difference between the price/Gigabyte of enterprise HDD and the enterprise SSD figure. Although reduced SSD pricing has now allowed it to catch up a little with HDDs, there is still a difference of around 8x (when comparing same application- and quality class). The pricing curves are likely to remain more or less parallel for the foreseeable future – and the crossover point, if there ever is one, will not happen until sometime deep into the 2030s. The reason for this is, though SSD prices for a given capacity are dropping, innovation in HDD technology is still keeping it out of reach of SSDs in terms of cost effectiveness. Ever increasing HDD capacities have meant the price/Gigabyte ratio they can achieve continues to make it a highly attractive option.

Even if the price/Gigabyte for HDDs was suddenly at parity with SSDs this time tomorrow, there just would not be enough flash memory production capacity to replace all the HDD resources with solid-state equivalents. In 2019, analyst firm Gartner reported HDD shipments equating to 890Exabyte, while total SSD capacity came to 153Exabyte – so only 16 percent of what HDD achieved. Interestingly, it should be noted that given the 8x better price/Gigabyte ratio, the total cost of all this HDD resource was almost the same as the far smaller SSD quantity.

For SSD to completely replace HDD, SSD production output would need to increase six-fold. Only doubling that output would cost hundreds of billions of dollars in bringing new fabs online, while a 6x increase would require thousands of billions to undertake. Furthermore, that is just to get us to a point where the data storage capacity installed in 2019 would be covered. That still wouldn’t be anywhere near enough to meet the thousands of Exabyte that are projected to be required by 2023 or beyond.

It is clear that reliance solely on a flash memory-based approach is not practical. With the increasing cloud service activity we have seen since the Covid outbreak, the credentials of HDDs in a data center context has been underlined still further. A report recently published by Statista backs this up – showing that high-capacity enterprise HDD shipments are set to keep growing in the years ahead.

It was mentioned earlier that a single HDD is more than 10x slower than an equivalent SSD. But, things aren’t that simple when we are talking about ultra-large data storage systems. Through implementation of smart architectures, many HDDs can be brought together to match the performance of a price-comparable system comprising a lesser number of SSDs. In the Toshiba Lab, we evaluated architectures with 24 to 60 HDDs in both RAID 10 and software-defined configurations. These were capable of delivering performance parameters greater than 10kIOPS and 5Gbps. So, for larger capacity storage systems, a solution featuring many HDDs (at a much lower unit cost) will result in a better price per capacity than fewer SSDs (which each have a much higher unit cost).
The current perception is that HDD-based storage solutions consume more power. In actual implementations with real-life workloads the power budget required by the spinning disks is often overestimated. Power consumption is less than what is often expected, especially if modern high-capacity low-power helium-filled drives are employed. Another argument that is often put forward by SSD manufacturers is that their drives are more reliable than HDDs, as they don’t have any moving parts. This point is again exaggerated. Modern enterprise HDDs, with MTTF figures of 2.5 million hours are directly comparable with any SSD in relation to their working lifespan. Furthermore, HDDs do not have strict limitations on the amount of writeable data they can cope with over their lifetime, whereas SSDs only have relatively restricted endurance. With constantly changing workloads across shared data center resources becoming more commonplace, this is undoubtedly an advantage.

In conclusion, based on current price points and future price projections, we can say that HDDs are still the most commercially viable way to store data on a large scale. The amount of data that our society is generating is rising at an exponential rate, with IDC predicting that by 2025 data generation will have surpassed 175 Zetabyte annually. Consequently, it is clear that HDDs still have a vital role to play. The price/capacity of SSD and the lack of adequate production output means that it will not be able to absorb the data explosion that is currently on the horizon. HDD technology is much better positioned to do so.

The article is authored by Rainer W. Kaese, Toshiba Electronics Europe and published in Data Center Dynamics