Thursday, June 21, 2012

Why high capacity tape storage will be essential to managing the coming archive data tsunami

The continued growth of digital archive data challenges the ability of enterprises to effectively control costs while meeting service level objectives and compliance requirements. Fortunately, the storage industry recognizes these challenges and is looking to help customers address the effective management of the archive information tsunami. A good example is the recent report  published by the Information Storage Industry Consortium entitled , “INSIC’s 2012-2022 International Magnetic Tape Storage Roadmap” (available at 

The INSIC report describes the requirements and technical goals for the tape storage industry for the next decade and the use of tape storage for archive information is recognized as the most significant application driver of the technical roadmap. Participants in this effort spanned the tape storage industry and included representatives from Hewlett Packard, IBM, Imation, Oracle, Quantum, Spectra Logic, and Symantec.  One of the outputs of the report is a set of technical goals for tape capacities over the next decade.  It’s also worth note that the technical roadmap is not a product roadmap but rather a technology roadmap that might represent the average of possible products, so there may be no specific products shipping or planned at these exact numbers. Needless to say, there are a lot of assumptions required to develop a ten year outlook for any technology, including tape storage, and a lot of technical work to be done to realize the goals. For the sake of this discussion, let’s “assume” that the goals are achieved and a product that achieves the projects goals enters the market in 2022. 

The first thing that strikes the reader is the 128 TB tape cartridge! 128 TB is over 25 times larger than the current largest generally available commercial cartridge, a 5 TB tape cartridge from Oracle. It begs the question: Who needs such a large tape cartridge? However, having worked in the storage industry in the late 90’s and early 2000’s, a similar question was raised about the first LTO tape media. The question then was who needs a 100 GB tape cartridge?  The largest commercial cartridges were in the 30-40 GB capacity range and mainframe tape cartridges provided 10-20 GB of capacity. Now, these capacities seem very small, one has to suspect similarly, that ten years from now a 128 TB cartridge will seem quite reasonable.

Let’s explore why a very large tape cartridge is not only reasonable, but essential. Archive data storage requirements are going to be immense. Citing the INSIC report, the rate of growth of archive data is estimated to be 45% CAGR over the coming decade. Now, let’s assume that an organization has 1 Petabyte (PB) of archive data today, and that they experience the 45% annual growth rate. Let’s also assume that they are very effective in managing the tape resource, and have an 80% tape cartridge utilization rate and the data would not benefit from tape compression. This means that today, they would need 1.25 PB of tape storage, and, in 10 years, at a 45% CAGR, they would need slightly more than 51 PB, 41 times today’s storage!

Now, let’s put this into practical terms. The 1.25 PB of archive storage needed today, requires 250 of the highest capacity (5TB) cartridge available today. This certainly seems like a very manageable number. However, the 51 PB of storage in ten years would require 10,272 of the 5 TB cartridges! A much, much more significant management challenge. But, with the planned 128 TB cartridge, only 402 cartridges are needed, a dramatic reduction. 

It’s clear that the INSIC objective of a 128 TB cartridge, while it seems like an impossibly large cartridge today, is an ideal tool to help cost effectively manage the mountain of archive data likely to be stored in the future. Maybe the INSIC 128 TB cartridge goal isn’t achieved, but it’s clear that the industry needs a very large capacity tape storage solution and the higher the capacity, the better!