A Proposal for Archiving in Government

There are two fundamental problems with electronic content archiving in the US Government:

  1. Government employees, particularly senior staff members, are largely in charge of what gets archived and what doesn’t, and what archived content is retained or deleted.
  2. Many government employees use their personal accounts (or, in some cases, their own servers) to conduct government business.

Here’s a simple proposal to address these problems:

  • Every bit of information in emails, text messages, social media posts, files and all other content sources generated by government-owned devices, servers, cloud services and other platforms should be archived by an independent government entity, such as the National Archives and Record Administration (NARA) or the US Government Accountability Office (GAO). This means that every government server automatically archives everything, without exception, and without advice from the employees whose content is archived.
  • Every government employee should be required to agree to one of the following: a) all of their personal emails, text messages, social media posts, files and all other content they generate on personal devices (or personal servers) will be securely archived by an independent government entity; or b) if they opt not to submit to having personal content archived and are later found to have been using a personal device or personally managed platform to transact government business, they will pay a fine equal to the past five years of their gross income and will relinquish any government pension for which they might have been eligible.
  • The independent entity that archives content will determine, at its sole discretion, what can safely be deleted from the archive. Things like spam, phishing emails, content that has no value as a record, and so forth, can be deleted based on policy established by NARA, the GAO or some other independent entity. However, the government employees whose information is archived cannot provide input or be consulted about the content that is retained or deleted. They should be able to access these records, but not provide input about what is retained or not.
  • All content that is retained must be kept for a minimum of 30 years unless NARA or a court determines that a longer retention period is warranted.

Information Overload is a Myth

A search for the term “information overload” in Google returns 3.68 million results, the second of which is a good definition of the problem: “exposure to or provision of too much information or data.” Wikipedia expands on the issue by defining it as “…a term used to describe the difficulty of understanding an issue and effectively making decisions when one has too much information about that issue. Generally, the term is associated with the excessive quantity of daily information.”

While the definitions are accurate, the fundamental issue with information overload is not really a problem with having too much information. Instead, it’s that we don’t have information curated in such a way as to present a limited set of the right information. For example, when I type “who starred in the movie grand prix” into Google, the first thing that shows up are photos of the cast. Google also provided many pages of additional search results, but curated a limited set of options that were most relevant to my inquiry, and it was the first one that satisfied that query. So, if Google had returned 300,000 other links and images, I would not have been overloaded with information because I could disregard everything but the right answer presented to me at the top of the list.

Similarly, if I need to find an email I sent to a prospect three days ago, does it matter if I have 36,745 emails in my inbox if my search returns just the email I was seeking? Not really.

So, what we’re really talking about with information overload is a lack of good search and good curation, which often begins with inadequate archiving of the right information. In the workplace, that lack of good search, curation and archiving manifests itself in a number of ways, most notably in the amount of time that employees spend searching for information. For example, a Software Advice survey found that some employees spend at least six hours per week searching for paper documents. A McKinsey report discovered that employees spend an average of 9.3 hours per week searching and gathering information. When it comes to information that is even more difficult to find, such as the job and client experience of my fellow employees that I might bring to bear on solving a problem, it may take even longer to find this information, if I can find it at all. Add to this the problem of information held in various silos across the enterprise and the situation becomes untenable, leading to regulatory, legal and employee productivity problems of various types.

Consequently, information overload really is not a thing — but inadequate search, curation and archiving definitely is.

How Long Should You Retain Records?

We have been asked many times how long businesses should retain their records, whether in email, files or other venues. The simple answer to the question is that there isn’t “an” answer. Instead, there are a number of issues to consider in determining how long you should retain your records:

  • What does your legal counsel advise?
  • What have court decisions in your industry revealed?
  • What is your organization’s tolerance for risk?
  • What are the consequences of disposing of records too quickly versus keeping them for too long?
  • What do government and industry regulations require as minimum retention periods?

To address the last question, we are assembling a database of regulations focused on data retention. We published the first edition in December with 421 regulations, but will be publishing the next edition in March with approximately 1,000.

Here’s a sample of the types of data retention regulations that exist today:

  • Manufacturers and importers of chemicals must retain documents related to notification of risk, contact information about entities to whom chemicals are distributed, production volumes and other information for three to five years (40 CFR 82.13).
  • Entities that operate as swap data depositories must retain records related to swaps or related cash or forward transactions for a period of five years, the first two years in an easily accessible place, but records of oral communications may be kept for only one year (17 CFR 1.31).
  • Underground mine operators must retain certifications for safety equipment for one year (30 CFR 57.4201).
  • Anyone who imports nonroad and stationary engines must retain documents supporting the information required in EPA Declaration Form 3520-21 for five years (19 CFR 12.74).
  • Entities that operate air curtain incinerators that burn yard waste must retain records about all opacity tests for five years (40 CFR 60.1455).
  • Manufacturers of heavy-duty vehicles and engines must retain records estimating how their fleets will comply with GHG emissions standards; estimated vehicle configuration, test group and fleet production volumes; expected emissions and fuel consumption test group results and fleet average performance; and other information (49 CFR 535.8).
  • The Canada Revenue Agency (CRA) requires entities subject to various sections of the Income Tax Act, the Employment Insurance Act and the Canada Pension Plan to retain for two to 10 years any books and records that will permit the CRA to determine taxation, the qualification of registered charities, permit the verification of various types of donations, etc. (CRA Information Circular IC78-10R5).

There are two key takeaways from this:

  1. There is no such thing as an “unregulated” industry or company in the context of data retention: every business in every industry must retain records for some length of time.
  2. Data retention is not easy, particularly in the context of being able to find archived records, disposing of them properly, and migrating them to new archives and other information platforms. The technology used to archive, search for and migrate records is critical.

For more information on our Data Retention Requirements Guide, click here.

You Should Not Archive Your Email and Texts

This is not a political post, I promise!

There are some lessons to be learned from the FBI no longer having access to five months worth of text messages between two staff members who were investigating former Secretary of State Hillary Clinton’s use of a private email server to conduct government business and the issue of Russian intervention in the 2016 presidential election, and Mrs. Clinton’s use of that private email server for sending classified and non-classified information. The one lesson I will discuss here is a simple one: you should not archive your email and texts.

More accurately, you, as an employee of your company, government agency or non-profit organization, should not archive your own email and texts.

Archiving should be based on pre-established and evolving corporate policy, not your choice of what content to save and what to discard. If your emails, texts, social media posts, files and other electronic content contain business records or any other content that is relevant to retain, it should be retained and archived automatically based on a set of corporate policies that have been established and approved by senior management, legal counsel, compliance, finance and any other stakeholders that are focused on the best interests of the enterprise. You, as an employee, should be involved in that process, but only as a voice among many in determining what to retain — you should not be the one who makes the final decision about what gets archived and what is discarded.

The reason for this is a simple one: there may be incriminating evidence, like mistakes or downright malicious activity in an email or text, that an individual might want to hide from the view of others. Someone responding to an email might mistakenly delete an important business record buried deep in the thread of an email that he or she did not see. Someone might fire off a text message or social media post in anger that reflects poorly on a client or colleague. In short, there is a temptation to delete information that violates corporate policy and we, as employees, should not have the ability to delete information in an attempt to cover that violation. While it might benefit us in the short term, it harms the organization in the long term.

In short, any good archiving process should prevent employees from being the key arbiter on what gets archived and what doesn’t.

You Need to Archive Mobile Text Messages

Osterman Research has found that roughly one-third of the typical information worker’s day is spent working on a mobile device, and an even greater proportion of work-related content is accessed using mobile devices. The impetus for the growing use of mobile devices is driven by a number of factors, although the use of personally owned devices is a key factor in their adoption in the workplace. As shown in the following figure, the use of company-owned and personally-owned smartphones is on the increase.

Untitled.jpeg
Source: Osterman Research, Inc.

The use of messaging applications on mobile devices, such as email and SMS/text messaging, are among the most common applications of mobile devices in the workplace. The vast majority of users who employ a smartphone for work-related uses employ some type of messaging-related application on a regular basis.

There are a number of difficulties associated with the archival of text messaging content. For example:

  • Text messages sent using telecom carriers are often retained only for brief periods, and so these providers cannot be relied upon a source of archived text messages for long periods.
  • Since some companies operate in multiple countries using carriers that often do not provide any sort of text messaging archival service, enterprises often employ different methods to archive text messages, such as doing a physical backup of a device.
  • Further complicating the archival of text messages is the lack of commonality for archiving content depending on the device in use. Some solutions pull content directly from the server (e.g., with the BlackBerry Enterprise Server), while others install an app on the mobile device that transmits text messages to the archive. Other tools, such as SMS Backup+ for Android devices, will move text messages into a user’s Gmail account where they can be backed up or archived indirectly.

The bottom line is that organizations using various and inconsistent methods for archival of text messages makes the process inefficient, expensive and prone to error. The result can be incomplete archives of text messages and the consequences that go along with this level of inconsistency. Therefore, it’s essential to choose the right vendor that can provide a consistent and unified method for text message archival.

We have recently published a white paper on text messaging archiving that you can download here.