Dawning Digital Dark Age

The accelerating pace of technology continues to change the way we communicate and do business. Systems, software, and file formats are in a constant state of flux, rapidly altering how we create, use, and store our data and documents. But the emphasis today is on innovation and productivity, often without consideration for how information will be retained and preserved for use in the future. Some experts have even speculated at the possibility that, as a consequence of the rapid technological development, in the future we may inadvertently render huge quantities of legacy data unreadable, obscuring the past and clouding the efforts of historians and record keepers. The effects may already be being felt, as archivists and records managers are now beginning to take steps to protect records with common industry retention periods of as little as 10 years from the risk of loss of readability or fidelity.

Traditional storage solutions are premised on keeping large quantities of data redundantly secured in multiple locations to protect against destruction or loss. But without taking steps to maintain the accessibility of this data so that the “ones and zeroes” can be actually interpreted, much of it may eventually be lost because it simply can no longer be read or used.

Early internet pioneer and Google Vice President, Dr. Vinton Cerf has recently cautioned that an imminent “digital dark age” where files and documents are retained but the ability to read them is lost, either due to hardware obsolescence or software incompatibility, may begin having significant implications for existing records much sooner than predicted. According to Dr. Cerf, “old formats of documents that we’ve created or presentations may not be readable by the latest version of the software because backwards compatibility is not always guaranteed… even if we accumulate vast archives of digital content, we may not actually know what it is.”^[1] With technological change occurring on a timescale of months, not years, this creates an obstacle to preservation far different and more urgent than the physical degradation of tangible records media.

Already, vast quantities of corporate records and data have vanished because they have been retained in data formats and in early versions of word processor files that are no longer accessible, or because they are stored on electronic media  that can no longer be easily read (e.g., 5” floppy disks, punch cards). The source codes underlying some of the earliest software versions of popular programs have now been lost even to the companies that own them, rendering the documents created by the software lost forever. Devices from companies like Palm and Blackberry enjoyed widespread public and business use before being discarded in favor of new products often incompatible with older technology.

Social media and other content platforms like Facebook and YouTube can today appear monolithic and permanent, but they could easily pass out of use in the near future. Just as once-ubiquitous products like Wordperfect and Lotus 1-2-3 have now fallen into obscurity, so too could many of the most popular and widely used software and services of the present. New advances in data storage, such as a newly invented glass-based digital storage technology that can store over 360 terabytes per disk with a shelf life of over 13 billion years,^[2] could be thwarted by constantly changing formats which render such technology all but useless on the timescales for which it was designed.

The loss of compatibility may have significant practical and legal implications for business record keeping. The inability to produce legible copies of digital documents at the request of regulators can lead to loss of company value, significant fines, or even criminal penalties. An emerging consensus among records management professionals is that for records with a 10-year or greater retention, proactive steps should be taken immediately to safeguard the contents against loss or incompatibility.^[3]

Fortunately, records managers can implement numerous strategies today to reduce or eliminate the possibility of data going dark while it is still needed. The longer such efforts are delayed, the greater the chance that remediation becomes infeasible or exorbitantly expensive. While corporate resources for archiving and preservation are often available only to the extent the information is presently considered valuable, in reality a modest investment today can prevent the need for costly data recovery projects in the future. Hence, it is essential that the long term costs and risks are communicated to decision makers and stakeholders at all levels of the enterprise when deciding on a proportional strategy.

Keeping abreast of digital assets by periodically mapping data and records to identify data at risk of obsolescence can help resolve incompatibilities long before the information actually becomes lost. Records managers can replicate records essential to organizational operations or history not just in multiple locations, but also in multiple formats, including old style hard copies when practical. They can also maintain a digital toolbox of software and emulation systems capable of reading older file formats and running old programs in order to extend the life of aging data, or future-proof records by pinpointing at-risk formats and migrating into more modern formats.

By adopting a forward-looking approach today, record managers can greatly reduce exposure to creeping obsolescence tomorrow.

 

Disclaimer: The purpose of this post is to provide general education on Information Governance topics. The statements are informational only and do not constitute legal advice. If you have specific questions regarding the application of the law to your business activities, you should seek the advice of your legal counsel.

 

^[1] http://www.bbc.com/news/science-environment-31450389

^[2] https://idm.net.au/article/0010882-new-laser-storage-can-store-data-138-billion-years

^[3] http://www.cbronline.com/blogs/cbr-rolling-blog/digital-preservation-the-10-year-tipping-point