Friday, August 29, 2014

How to lower long term data retention costs

There is an abundance of innovation taking place in the storage industry today; high capacity disk drives, flash, cloud, software defined storage,  tape and integrated systems that combine elements of each. However, when crafting a solution for the long term storage of digital data, an essential aspect of the solution has to be cost effectiveness. But as important as cost may be, it can be a challenge to sort through all the different technologies and cost components to develop a reliable estimate of the cost of storing digital information over an extended period of time; there are so many variables, and different technology life cycle considerations.  However, by establishing some initial cost estimates, and making a set of projections, it is possible to develop a reasonable assessment of the long term costs of storing file data.

For example, two storage technologies that are frequently considered for long term data storage today are a high capacity, low cost disk system that presents a NAS/CIFS interface to the user, and integrated NAS/tape solutions that present NAS/CIFS interface, but differ from traditional disk systems by transparently placing the file data on LTFS tape storage. To develop a cost projection over 10 years, a few assumptions and estimates have to be made; the amount of data to be stored, whether the data is reducible, the initial solution components and associated costs, the feasible life span of the initial solution and estimates of future support and energy costs.

With these considerations in mind then, let’s get started; we need 1 Petabyte of data stored for 10 years and that it’s not reducible. The tape NAS solution will include an appliance, the Crossroads Systems Strongbox, tape library and LTO Gen 6 tape drives and media. This solution is used for the entire 10 year period.  The disk solution is a composite average of four different low cost, high capacity NAS or object storage solutions that had publicly available pricing and configuration information.  This solution is refreshed in year 6 with a solution based on 12 TB HDD’s effectively reducing the support and energy costs by 2/3 for the remainder of the period.  For all cases, support and energy prices are projected to increase at 3% annually. 

With this set of projections, what is the 10 year estimated cost of storing 1 PB of file data on the two solutions? For the tape solution, the total cost is $ 234,525, for the disk solution, the total cost is $519,904. The total cost of the tape solution is 55% less than the high capacity disk solution.  In addition, the energy consumption of the tape solution is over 90% less than the disk solution. While there are a several estimates and forecasts in this analysis, they are certainly reasonable based on current technology trends. It is clear that for long term data storage, a solution that incorporates tape is likely to substantially reduce the overall cost of storing important file data. Link to report