email article 
print
page
Digital Archiving Circa 2008: Addressing Long-Term Data Storage and Preservation Requirements
The Digital Content Explosion: A Huge Business Problem in the Making
By Heidi Biggar, Analyst, Enterprise Strategy Group
 |
Consider the healthcare organization that is unable to locate an MRI during a life-critical procedure, a digital media outfit re-mastering film into huge digital files or a government agency forced to comply with federally-mandated lengthy retention periods. Organizations today share an underlying business problem: 1) storing huge volumes of fixed-content, or persistent, data and 2) preserving this data for the long haul. Current archive practices that are based on tape and which think of backup and archive as being one and the same or, worse yet, intermix archive data with mission-critical data on primary storage systems, are inherently flawed. Organizations that don?t have sound strategies in place for storing fixed content today face serious IT and business consequences over time, including high IT costs (both CAPEX and OPEX), potentially hefty regulatory penalties and lost revenue as a result of downtime or missed business opportunity.
To put this in some perspective, according to recent ESG Research,1 total worldwide digital archive capacity is on track to surpass 27,000 petabytes?or 27 exabytes? in 2010 (see Figure 1). That?s approximately triple today?s archive capacity levels and more than 10 times 2005 levels. That?s a whole lot of fixed content that needs to be stored, managed, secured, protected and preserved for the long haul.
 |
| Source: Enterprise Strategy Group, 2007 |
Increasingly, ESG Research also shows that it is not uncommon for organizations today to have to retain data for 10 years or more (see Figure 2). Thirty-seven percent of survey respondents said the average length of time they retained archived file-based data was 6 to 10 years, while 9% of respondents reported an average retention period of 50 years or more. Forty-six percent of survey respondents also reported maximum retention periods in excess of 10 years, with 11% citing maximum retention periods of greater than 50 years.
Just think about the number of generation and format changes that are likely to take place over the course of a single decade, let alone 50 years! What happens when retention periods extend beyond the lifetime of the media or the service life of the storage systems themselves? How is data readability assured? The key lies with automated migration. Manual migration is a short-term fix and can be extremely costly over the long-term from a pure management perspective. As data volumes grow, manual migration policies become downright impractical.
Digital Archives: Circa 2008
Building a 21st century data archive doesn?t have to involve a lot of hefty heavy lifting (i.e., an overhaul of the IT infrastructure), but it does require organizations to adopt a new mindset with regard to storing and preserving digital content. First and foremost, this means:
- Understanding that backup and archive are not synonymous. Backup is a temporary copy of a record or data set for the purpose of data protection or disaster recovery. An archive is a long-term or permanent copy of a record or data set for the purpose of satisfying records management, IT cost control, regulatory compliance or litigation support requirements. These are two distinct processes with two distinct purposes; however, they are often used interchangeably today. In fact, at many organizations, backup copies have long-served double-duty?acting as both the DR and archive copy. This type of practice is risky at best and costly at worst.
Beyond this, organizations need to think a little "out-of-the-box" from both a technology and a process perspective. They need to:
- Recognize that ?backing up to tape? does not an archive make. Tape is not a particularly effective digital archive medium, especially for business compliance data. It simply takes too long for users to find and retrieve data from tape. This time equates to dollars lost?in man-hours consumed to find the data, application downtime, fines, etc. Additionally, there are other ―soft costs to consider, which include the business costs of not being able to leverage the information that these archives may contain.
- Look at archiving as an ?active? process that can involve multiple tiers or technology types. Data is moved among tiers according to established policies for access, security and performance (i.e., the age of the data, its value to the organization, regulatory/security considerations, etc.). The idea is to keep fixed content on the ―lowest cost‖ storage tier possible that provides the level of service that is needed.
- Think about primary storage, too. A lot of the data sitting on primary storage systems today is non-transactional or post-transactional, meaning that it is fixed or static (this type of data is also referred to as persistent data). While keeping data on primary disk doesn?t present a "data access" or "availability" situation like tape does, it does create other problems. Notably, it can drive primary storage costs unnecessarily skyward and boost associated backup costs (see Sidebar: "Benefits of Archiving Tiers").
- Factor in future retention periods. It?s a fact: Organizations are being asked to keep digital content for longer and longer periods of time. Again, 11% of the organizations we polled as part of our 2007 Digital Archiving Research2 said that they kept some of their data for 50 years or more. This has huge implications from both a capacity and technology obsolescence standpoint.
- Consider the likelihood of technology obsolescence. As retention periods lengthen, technology obsolescence becomes a very real concern. Organizations may be unable to read data at all if the hardware/media the data was written on is obsolete. Moving archive data from platform to platform may be the only way for organizations to ensure that data is available and readable years down the road. Organizations need to look at solutions that make this process as transparent as possible. Solutions that do the migration manually, or require other "heavy lifting," should be avoided.
Related Newsletters: IBN - IT Weekly Newsletter
