Big Data has burst onto the Information Technology scene. The confluence of advances in servers, analytic techniques and software has changed the way enterprises deal with computing infrastructures. The variety, volume and velocity of Big Data are also accelerating. Diverse applications such as simulation, visualization, modeling, seismic, video surveillance, and analytics are creating and processing unprecedented amounts of unstructured information. While these applications provide exciting new insights for business, they also place increasing requirements on the storage infrastructure and challenge storage management in a multitude of ways. Users of Big Data require new, innovative storage solutions to cost effectively manage this information while meeting demanding service levels and compliance needs. The StrongBox Big Data solution is a powerful new tool that integrates IBM tape libraries to simplify management of storage repositories and lower associated costs.
For more information on StrongBox see StrongBox . For the complete white paper go to Taming Big Data with StrongBox
Monday, June 3, 2013
Tuesday, April 23, 2013
One of the new buzzwords in both marketing and technology is “Big Data Analytics”. To better understand the new technologies and marketing techniques I attended the first American Marketing Associations conference on Big Data Analytics, “Analytics with Purpose; The Human Edge of Big Data” held March 4th and 5th in San Diego. Reflecting on the conference, it’s clear that technology is providing the marketing function with a set of capabilities that is transforming the marketing role. Technology has done this in other industries and functional areas over the last 50 years: Airline reservation systems changed the way the airline industry managed travel reservations and made it possible for airlines to process millions of reservations in a cost effective way. Computer aided design systems have dramatically increased the productivity of engineering design while improving overall product quality. Material Requirements planning systems changed the way manufacturing companies planned, procured and produced products. Now it’s marketing’s turn , and early adopters are utilizing the power of analytics to improve linkages with customers, identify new opportunities and increase sales. However, the new analytic technologies are complex and require a great deal of expertise of effectively implement and manage. But, they will undoubtedly become easier to use. For example, analytics in the cloud is already a reality. For the full report on the conference, check out my blog post at the Tucson AMA Blog at http://tucsonama.com/blog/. For marketers, and IT professionals, hang on, it’s going to be an exciting and interesting ride.
Thursday, October 4, 2012
IBM had a large storage announcement yesterday that included a wide variety of interesting and exciting news. The overview can be found at: http://www-03.ibm.com/systems/storage/?lnk=mprST-1. Among them was the first announcement by one of the LTO consortium of the next generation LTO Ultrium Generation 6 tape drive. It is officially named the IBM System Storage TS1060 Tape Drive and is supported in the IBM TS3500 tape library. It is a full height fibre channel tape drive that sports an 8 Gbps dual port connection. The native tape capacity cartridge capacity is 2.5 TB. This compares favorably to the native capacity of 1.5 TB for the comparable LTO Generation 5 tape drive. In addition, a number of other improvements have been made; a faster data rate of 160 MB/second versus 140 MB/second and improved energy efficiency. The new tape drive includes a more efficient compression engine, providing 2.5:1 compression versus 2:1 with LTO Generation 5. It also supports important LTO features from previous generations; such as tape encryption and the Linear Tape File System. The list price for the new tape drive is $25,855 versus $23,940 for the Generation 5 equivalent, roughly an 8% price increase. Net, the new tape drive provides 67% more native capacity, a 14% greater data rate at an 8% purchase price increase. First shipment is scheduled for November 9th. The detailed announcement can be found at this link: http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=AN&subtype=CA&htmlfid=897/ENUS112-162&appname=USN
Putting this announcement is some perspective; let’s examine what it means to a customer with one petabyte of archive data that needs to be cost-effectively retained. Assuming the data is compressible at the typical rates, with an LTO Ultrium Generation 5 tape drive, it would require 334 tapes (2:1 compression). With LTO Generation 6, the number of tapes drops to 160 (2.5:1 compression). The combination of higher native capacities and improved compression rates reduces the very large amount data to a very manageable number of tape cartridges. In fact, a medium sized tape library could easily accommodate this amount of data in part of a single rack. The new tape drive will be well suited to meet the needs of large and medium sized enterprises wrestling with cost effective storage of large amounts of archive data.What now? This is likely the tip of the LTO Generation 6 iceberg, and it’s reasonable to expect a plethora of additional tape drive and media announcements over the coming months, as IBM and other LTO tape drive and media suppliers’ role out their offering across a variety of tape libraries at different price, performance and capacity points.
Thursday, June 21, 2012
The continued growth of digital archive data challenges the ability of enterprises to effectively control costs while meeting service level objectives and compliance requirements. Fortunately, the storage industry recognizes these challenges and is looking to help customers address the effective management of the archive information tsunami. A good example is the recent report published by the Information Storage Industry Consortium entitled , “INSIC’s 2012-2022 International Magnetic Tape Storage Roadmap” (available at http://www.insic.org/news/2012Roadmap/news_12roadmap.html).
The INSIC report describes the requirements and technical goals for the tape storage industry for the next decade and the use of tape storage for archive information is recognized as the most significant application driver of the technical roadmap. Participants in this effort spanned the tape storage industry and included representatives from Hewlett Packard, IBM, Imation, Oracle, Quantum, Spectra Logic, and Symantec. One of the outputs of the report is a set of technical goals for tape capacities over the next decade. It’s also worth note that the technical roadmap is not a product roadmap but rather a technology roadmap that might represent the average of possible products, so there may be no specific products shipping or planned at these exact numbers. Needless to say, there are a lot of assumptions required to develop a ten year outlook for any technology, including tape storage, and a lot of technical work to be done to realize the goals. For the sake of this discussion, let’s “assume” that the goals are achieved and a product that achieves the projects goals enters the market in 2022.
The first thing that strikes the reader is the 128 TB tape cartridge! 128 TB is over 25 times larger than the current largest generally available commercial cartridge, a 5 TB tape cartridge from Oracle. It begs the question: Who needs such a large tape cartridge? However, having worked in the storage industry in the late 90’s and early 2000’s, a similar question was raised about the first LTO tape media. The question then was who needs a 100 GB tape cartridge? The largest commercial cartridges were in the 30-40 GB capacity range and mainframe tape cartridges provided 10-20 GB of capacity. Now, these capacities seem very small, one has to suspect similarly, that ten years from now a 128 TB cartridge will seem quite reasonable.
Let’s explore why a very large tape cartridge is not only reasonable, but essential. Archive data storage requirements are going to be immense. Citing the INSIC report, the rate of growth of archive data is estimated to be 45% CAGR over the coming decade. Now, let’s assume that an organization has 1 Petabyte (PB) of archive data today, and that they experience the 45% annual growth rate. Let’s also assume that they are very effective in managing the tape resource, and have an 80% tape cartridge utilization rate and the data would not benefit from tape compression. This means that today, they would need 1.25 PB of tape storage, and, in 10 years, at a 45% CAGR, they would need slightly more than 51 PB, 41 times today’s storage!
Now, let’s put this into practical terms. The 1.25 PB of archive storage needed today, requires 250 of the highest capacity (5TB) cartridge available today. This certainly seems like a very manageable number. However, the 51 PB of storage in ten years would require 10,272 of the 5 TB cartridges! A much, much more significant management challenge. But, with the planned 128 TB cartridge, only 402 cartridges are needed, a dramatic reduction.
It’s clear that the INSIC objective of a 128 TB cartridge, while it seems like an impossibly large cartridge today, is an ideal tool to help cost effectively manage the mountain of archive data likely to be stored in the future. Maybe the INSIC 128 TB cartridge goal isn’t achieved, but it’s clear that the industry needs a very large capacity tape storage solution and the higher the capacity, the better!
Thursday, May 24, 2012
This is my inaugural post and it seems fitting to discuss tape technology since IBM celebrated the 60th anniversary of IBM tape storage last week. IBM 60th Tape Anniversary. The role of tape storage has certainly changed over the years; some of us remember when customer master files were often stored on tape and processed monthly by batch processing applications. Tape was also a favorite for the protection of data, and elaborate software applications and processes were developed to periodically copy important information from disk to tape and restore it in case the original data was lost or damaged. Now, many new disk based backup solutions exist and tape has seen its use for backup transition to less time sensitive backup and archive data.
Archive data represents a significant challenge for many IT environments. Industry consultants and storage suppliers estimate that the amount of digital archive information will continue to grow unabated at an annual rate of 50% or more for the foreseeable future. However, Information Technology budgets are not growing anywhere near as fast; even with impressive improvements in the $/GB cost of storage, it’s very challenging to define and execute an archive data strategy that can successfully support the security and retention policies of an enterprise, while remaining within the available operational and capital budgets.
To make life even more interesting, archive data comes in many different flavors; for example, some is highly structured; such as extracts of relational data base files; other data may be completely unstructured, for example, video or test data. The structured data may be highly reducible, using either compression or deduplication technology. On the other hand, video or test data is likely not reducible, due to either the nature of the data, or due to the fact it has already been reduced. Ironically, files not likely to benefit from data reduction techniques tend to be very large (think Gigabytes or Terabytes) and are also rapidly proliferating.
Tape storage becomes very interesting for these non-reducible files. It is very cost effective and its power requirements are very low (zero unless being actively accessed). It is also very space efficient with native cartridge capacities ranging from 1.5 TB for LTO Gen 5 , to 4 TB (IBM TS1140) or 5 TB (OracleT10000C). Given the attractive economics, energy and space efficiency, why is tape not more widely used for archive? One of the inhibitors to the widespread use of tape storage has been the unique access requirements. Tape is often accessed via applications that incorporate metadata and proprietary formats on top of the end user data when writing to tape. Unfortunately, for long term archive data, this then creates a long term dependency on the tape application for access. This dependency can have many different side effects including the cost of software maintenance and support over years or even decades.
Tape usage has also been inhibited by the time required to access the information on the tape cartridge. Unlike disk drives, before information is read, the cartridge must be moved to a tape drive, the tape loaded, and then positioned to the correct file. Even in a modern tape library, this sequence of activities can take 30 seconds to a minute or more. For active archive information, where the end user may need relatively rapid access to the information, these delays may offset the financial benefits of using tape storage.
A new storage appliance offering from Crossroads Systems called StrongBox®Crossroads StrongBox addresses these inhibitors by using the new LTO® Long Term File System ™ (LTFS) for the writing of data to tape, using the CIFS/NFS file interfaces for data access and using StrongBox intelligent file management to help mask the access delays of tape storage. By leveraging standard interfaces for both ingest and storage, Crossroad Systems effectively liberates the archive data from the dependence on proprietary tape applications. In addition, the intelligent management software included with StrongBox uses file disk buffers to initially respond to access requests, helping mask the delays associated with mounting and loading a tape cartridge.
Just how much might an IT department save by deploying StrongBox in conjunction with tape storage versus a traditional NAS storage system? That was the issue addressed in my white paper “A New Approach to Lowering the Cost of Storing File Archive Information” dated April 16th A new approach. It turns out that a 2 PB archive data requirement; for non-reducible data, that the StrongBox with an IBM LTO tape library solution could save a customer up to $6.5 million over ten years. It’s worth noting that a large number of assumptions are required, to generate a 10 year TCO, and the caveat “your mileage will vary” is certainly applicable. On the other hand the potential savings are so large that it certainly appears that the StrongBox solution may provide significant financial relief to organizations wrestling with the challenge of managing very large amounts of active archive data.
The advent of clever technologies like StrongBox, that leverage the attractive economics of tape, while growing the available use cases may remove some of the inhibitors to tape usage, and substantially increase its usage for long term archive . The next 60 years should be interesting.