There are two fundamental problems with electronic content archiving in the US Government:
Government employees, particularly senior staff members, are largely in charge of what gets archived and what doesn’t, and what archived content is retained or deleted.
Many government employees use their personal accounts (or, in some cases, their own servers) to conduct government business.
Here’s a simple proposal to address these problems:
Every bit of information in emails, text messages, social media posts, files and all other content sources generated by government-owned devices, servers, cloud services and other platforms should be archived by an independent government entity, such as the National Archives and Record Administration (NARA) or the US Government Accountability Office (GAO). This means that every government server automatically archives everything, without exception, and without advice from the employees whose content is archived.
Every government employee should be required to agree to one of the following: a) all of their personal emails, text messages, social media posts, files and all other content they generate on personal devices (or personal servers) will be securely archived by an independent government entity; or b) if they opt not to submit to having personal content archived and are later found to have been using a personal device or personally managed platform to transact government business, they will pay a fine equal to the past five years of their gross income and will relinquish any government pension for which they might have been eligible.
The independent entity that archives content will determine, at its sole discretion, what can safely be deleted from the archive. Things like spam, phishing emails, content that has no value as a record, and so forth, can be deleted based on policy established by NARA, the GAO or some other independent entity. However, the government employees whose information is archived cannot provide input or be consulted about the content that is retained or deleted. They should be able to access these records, but not provide input about what is retained or not.
All content that is retained must be kept for a minimum of 30 years unless NARA or a court determines that a longer retention period is warranted.
A search for the term “information overload” in Google returns 3.68 million results, the second of which is a good definition of the problem: “exposure to or provision of too much information or data.” Wikipedia expands on the issue by defining it as “…a term used to describe the difficulty of understanding an issue and effectively making decisions when one has too much information about that issue. Generally, the term is associated with the excessive quantity of daily information.”
While the definitions are accurate, the fundamental issue with information overload is not really a problem with having too much information. Instead, it’s that we don’t have information curated in such a way as to present a limited set of the right information. For example, when I type “who starred in the movie grand prix” into Google, the first thing that shows up are photos of the cast. Google also provided many pages of additional search results, but curated a limited set of options that were most relevant to my inquiry, and it was the first one that satisfied that query. So, if Google had returned 300,000 other links and images, I would not have been overloaded with information because I could disregard everything but the right answer presented to me at the top of the list.
Similarly, if I need to find an email I sent to a prospect three days ago, does it matter if I have 36,745 emails in my inbox if my search returns just the email I was seeking? Not really.
So, what we’re really talking about with information overload is a lack of good search and good curation, which often begins with inadequate archiving of the right information. In the workplace, that lack of good search, curation and archiving manifests itself in a number of ways, most notably in the amount of time that employees spend searching for information. For example, a Software Advicesurvey found that some employees spend at least six hours per week searching for paper documents. A McKinsey report discovered that employees spend an average of 9.3 hours per week searching and gathering information. When it comes to information that is even more difficult to find, such as the job and client experience of my fellow employees that I might bring to bear on solving a problem, it may take even longer to find this information, if I can find it at all. Add to this the problem of information held in various silos across the enterprise and the situation becomes untenable, leading to regulatory, legal and employee productivity problems of various types.
Consequently, information overload really is not a thing — but inadequate search, curation and archiving definitely is.
The European Union (EU) will put the General Data Protection Directive (GDPR) into effect on May 25th, and with it some potentially difficult and onerous requirements. Here are a few potential issues with which companies worldwide will have to contend:
Article 7(1) of the GDPR states, “Where processing is based on consent, the controller shall be able to demonstrate that the data subject has consented to processing of his or her personal data.” That means that anyone who signs up for a mailing list, a webinar, an email newsletter or any other type of communication from you will need to be fully informed of the “processing” that their data will undergo, and you will need to keep an accurate record of each instance of consent that has been granted. For example, someone who signs up to be on your corporate emailing list is granting consent for their information to be used strictly for the purpose of receiving email from you – you need to maintain a record of that consent. If they sign up for a webinar that you have announced to them in an email, they are granting consent to be contacted with regard to that specific webinar – you need to maintain a record of that, as well.
Our recommendation: excellent and up-to-date recordkeeping is going to be of paramount importance in order to remain compliant with the GDPR. That means good archiving of data subjects’ information, including the ability to search for and retrieve this information quickly and completely, and the ability to defensibly delete this information when needed.
Article 22(1) requires that a “data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling…” and that includes their “location or movements” (Recital 71). What that likely means is that there is a prohibition on determining whether or not someone is an EU “data subject” based on things like their IP address when completing a form on your web site, for example. So, if someone who lives in the United States is on your corporate mailing list, where their information is not subject to GDPR compliance, but later moves to an EU country, where their data is now subject to the GDPR, is the onus on you to know they’ve moved? According to a strict interpretation of Recital 71, you’re not allowed to collect their IP address when they interact with you, and so you may not be able to determine that they have moved.
Our recommendation: act as if everyone is subject to compliance with the GDPR and process information accordingly.
Articles 12 through 23 of the GDPR are the “Rights of the data subject”, which include things like their right to access and have corrected any information that a data processor or controller has on them, and their right to have that information deleted – their “right to be forgotten” – albeit with certain limitations. There are some serious implications for data controllers and processors in these requirements:
You need to know where all of your data is located. Data subjects’ information that might be stored on a departmental file share to which IT or legal does not have ready access, information stored in employees’ personal Dropbox accounts, or information stored on ex-employees’ personal devices could make it difficult or impossible to respond adequately to a data subject’s request for information or their right to have this data corrected or expunged.
Even with access to all of your data, an organization with malicious intent could organize a group of a few thousand people to request their data simultaneously. Given that the GDPR gives data processors and controllers only one month to comply with these requests (up to three months in some situations), an organization with inadequate content management systems in place could easily run afoul of the GDPR.
Our recommendation: conduct a thorough data inventory to determine where all of your data is located, give IT access to it, and implement a robust and scalable archiving capability that will enable all corporate data to be searched and produced quickly and with a minimum of effort.
Many thanks to Anne P. Mitchell, an Internet law and policy attorney and legislative consultant, for her input to this post. Her firm is offering consulting on the legal aspects of the GDPR – you can contact her here.
For more information on the GDPR, you can download our most recent white paper here.
We have been asked many times how long businesses should retain their records, whether in email, files or other venues. The simple answer to the question is that there isn’t “an” answer. Instead, there are a number of issues to consider in determining how long you should retain your records:
What does your legal counsel advise?
What have court decisions in your industry revealed?
What is your organization’s tolerance for risk?
What are the consequences of disposing of records too quickly versus keeping them for too long?
What do government and industry regulations require as minimum retention periods?
To address the last question, we are assembling a database of regulations focused on data retention. We published the first edition in December with 421 regulations, but will be publishing the next edition in March with approximately 1,000.
Here’s a sample of the types of data retention regulations that exist today:
Manufacturers and importers of chemicals must retain documents related to notification of risk, contact information about entities to whom chemicals are distributed, production volumes and other information for three to five years (40 CFR 82.13).
Entities that operate as swap data depositories must retain records related to swaps or related cash or forward transactions for a period of five years, the first two years in an easily accessible place, but records of oral communications may be kept for only one year (17 CFR 1.31).
Underground mine operators must retain certifications for safety equipment for one year (30 CFR 57.4201).
Anyone who imports nonroad and stationary engines must retain documents supporting the information required in EPA Declaration Form 3520-21 for five years (19 CFR 12.74).
Entities that operate air curtain incinerators that burn yard waste must retain records about all opacity tests for five years (40 CFR 60.1455).
Manufacturers of heavy-duty vehicles and engines must retain records estimating how their fleets will comply with GHG emissions standards; estimated vehicle configuration, test group and fleet production volumes; expected emissions and fuel consumption test group results and fleet average performance; and other information (49 CFR 535.8).
The Canada Revenue Agency (CRA) requires entities subject to various sections of the Income Tax Act, the Employment Insurance Act and the Canada Pension Plan to retain for two to 10 years any books and records that will permit the CRA to determine taxation, the qualification of registered charities, permit the verification of various types of donations, etc. (CRA Information Circular IC78-10R5).
There are two key takeaways from this:
There is no such thing as an “unregulated” industry or company in the context of data retention: every business in every industry must retain records for some length of time.
Data retention is not easy, particularly in the context of being able to find archived records, disposing of them properly, and migrating them to new archives and other information platforms. The technology used to archive, search for and migrate records is critical.
For more information on our Data Retention Requirements Guide, click here.
There are some lessons to be learned from the FBI no longer having access to five months worth of text messages between two staff members who were investigating former Secretary of State Hillary Clinton’s use of a private email server to conduct government business and the issue of Russian intervention in the 2016 presidential election, and Mrs. Clinton’s use of that private email server for sending classified and non-classified information. The one lesson I will discuss here is a simple one: you should not archive your email and texts.
More accurately, you, as an employee of your company, government agency or non-profit organization, should not archive your own email and texts.
Archiving should be based on pre-established and evolving corporate policy, not your choice of what content to save and what to discard. If your emails, texts, social media posts, files and other electronic content contain business records or any other content that is relevant to retain, it should be retained and archived automatically based on a set of corporate policies that have been established and approved by senior management, legal counsel, compliance, finance and any other stakeholders that are focused on the best interests of the enterprise. You, as an employee, should be involved in that process, but only as a voice among many in determining what to retain — you should not be the one who makes the final decision about what gets archived and what is discarded.
The reason for this is a simple one: there may be incriminating evidence, like mistakes or downright malicious activity in an email or text, that an individual might want to hide from the view of others. Someone responding to an email might mistakenly delete an important business record buried deep in the thread of an email that he or she did not see. Someone might fire off a text message or social media post in anger that reflects poorly on a client or colleague. In short, there is a temptation to delete information that violates corporate policy and we, as employees, should not have the ability to delete information in an attempt to cover that violation. While it might benefit us in the short term, it harms the organization in the long term.
In short, any good archiving process should prevent employees from being the key arbiter on what gets archived and what doesn’t.
We have recently completed a survey of IT decision makers that are knowledgeable about security issues in their organizations, and we found something surprising: the concern about “shadow IT” — employee use of unauthorized cloud apps or services — is significantly lower in this year’s survey than it was just over a year ago. While there can be variability between surveys because of sampling and other issues, the difference we found is not explained by sampling variability, but instead represents a significant shift of concern away from the problem of shadow IT and BYOD/C/A (Bring Your Own Devices/Cloud/Applications).
First, we have not seen big, headline-grabbing data breaches result from the use of personally owned smartphones, tablets, laptops and other employee-owned and managed devices, cloud applications and mobile applications. While these breaches occur and clearly are a problem, the horror stories that were anticipated from the use of these devices have been few and far between.
Second, senior management — both in IT and in lines of business — have seemingly acquiesced to the notion of employees using their own devices. They realize that stopping employees from using their own devices to access work-related resources is a bit like controlling ocean surf with a broom.
Third, there are some advantages that businesses can realize from employees using their own devices. While lower business costs are an important advantage because IT doesn’t have to purchase devices for some employees, another important benefit is that IT doesn’t have to manage them either. For example, when an employee leaves a company and company-supplied devices need to be deactivated, some organizations aren’t exactly sure who’s responsible for doing so — IT, the employee’s manager, HR or someone else. A survey we conducted some time back asked, “when an employee who had a company-supplied mobile phone leaves your employment, how confident are you that you are not still paying for their mobile service?” We found that only 43 percent of respondents were “completely confident” that the mobile service was deactivated, and 11 percent either were “not really sure” or just didn’t know. Employees using their own devices and plans gets around this problem nicely.
To be sure, unfettered and unmanaged use of employee devices in the workplace is not a good idea. It can lead to a number of problems, such as the inability for IT to know where all of a company’s data is stored, the inability to properly archive that data, the inability to produce all of it during an eDiscovery effort or a regulatory audit, lots of duplicate data, a failure to establish an authoritative record for corporate data, a greater likelihood of data breaches if a device is lost, and the potential for not being able to satisfy regulatory obligations.
That last point is particularly important, especially in the context of the European Union’s General Data Protection Regulation (GDPR). A key element of the GDPR is a data subject’s “right to be forgotten”, which translates to a data holder’s obligation to find and expunge all data it has on a data subject. If an organization cannot first determine all of the data it holds on a data subject and then cannot find all of that data, it runs the risk of violating the GDPR and can pay an enormous penalty as a result.
In short, BYOD/C/A offers a number of important advantages, but it carries with it some serious risks and should be addressed as a high priority issue in any organization.
Osterman Research has found that roughly one-third of the typical information worker’s day is spent working on a mobile device, and an even greater proportion of work-related content is accessed using mobile devices. The impetus for the growing use of mobile devices is driven by a number of factors, although the use of personally owned devices is a key factor in their adoption in the workplace. As shown in the following figure, the use of company-owned and personally-owned smartphones is on the increase.
The use of messaging applications on mobile devices, such as email and SMS/text messaging, are among the most common applications of mobile devices in the workplace. The vast majority of users who employ a smartphone for work-related uses employ some type of messaging-related application on a regular basis.
There are a number of difficulties associated with the archival of text messaging content. For example:
Text messages sent using telecom carriers are often retained only for brief periods, and so these providers cannot be relied upon a source of archived text messages for long periods.
Since some companies operate in multiple countries using carriers that often do not provide any sort of text messaging archival service, enterprises often employ different methods to archive text messages, such as doing a physical backup of a device.
Further complicating the archival of text messages is the lack of commonality for archiving content depending on the device in use. Some solutions pull content directly from the server (e.g., with the BlackBerry Enterprise Server), while others install an app on the mobile device that transmits text messages to the archive. Other tools, such as SMS Backup+ for Android devices, will move text messages into a user’s Gmail account where they can be backed up or archived indirectly.
The bottom line is that organizations using various and inconsistent methods for archival of text messages makes the process inefficient, expensive and prone to error. The result can be incomplete archives of text messages and the consequences that go along with this level of inconsistency. Therefore, it’s essential to choose the right vendor that can provide a consistent and unified method for text message archival.
We have recently published a white paper on text messaging archiving that you can download here.
In the case of Green v. Blitz USA, Inc.– a wrongful death case in which the plaintiff’s husband was killed by an exploding gas can produced by the defendant – the jury ruled unanimously in favor of the defendant. Because of a high-low agreement into which the parties had entered during jury deliberations, the plaintiff received a relatively small payment from the defendant. However, a year after this case was settled, the plaintiff determined that poor data collection practices by the defendant led to non-production of key documents that should have been presented during eDiscovery. Although the statute of limitations under the Federal Rules of Civil Procedure (FRCP) prevented a new trial in this case, the court ordered that:
The defendant must pay $250,000 in civil contempt sanctions to the plaintiff.
The defendant had 30 days to provide a copy of the court’s ruling about its poor collection practices to every plaintiff that had a case against the company during the past two years.
The defendant was ordered to pay a sanction of an additional $500,000 until the court’s orders in this case had been carried out. If Blitz complied with the court’s order, this particular sanction would be terminated.
For the next five years, the defendant was required to provide a copy of the court’s order as part of its initial pleading or filing to every party in every lawsuit in every court in which it might be involved.
Clearly, improper data collection can result in potentially severe sanctions.
Think about the process of sending a single email to one individual:
You create and send an email and a copy of that email is placed into your Sent Items folder (copy 1).
The recipient receives your email (copy 2).
Your email admin makes a nightly backup of your email inbox (copy 3).
The recipient’s admin does likewise (copy 4).
Your company’s archiving system places a copy of your email into archival storage (copy 5).
Ditto for the recipient’s company’s email archiving system (copy 6).
The email you sent to recipient A gets forwarded to someone else (copy 7).
That copy gets placed into a backup and archive (copies 8 and 9).
You, your original recipient and the recipient of the forwarded copy access corporate email on a smartphone and a tablet (copies 10, 11, 12, 13, 14 and 15).
Now, let’s say you decide that you want to delete all of your old email because you’re afraid of incriminating evidence that might turn up in a lawsuit, a regulatory audit, or because you’re running for political office (ahem). Good luck with that. At best, you might be able to delete copy 1 and, if the recipient is nice, copy 2. Copies 3, 4 and 8 might disappear as admins reuse backup tapes over time or as the various mobile devices on which your email is stored deletes older content. But that means that of the 15 or so copies of your email that exist, only about one-third to one-half will ever really disappear.
What should you do? First of all, disabuse yourself of the notion that you can ever completely delete your email. You can’t – it exists and may exist forever in some cases. Second, realize that email will stick around despite your best efforts to purge it, and so plan on it reappearing at some point. That means that if you have incriminating emails floating around your company, it’s best to archive them reliably and prevent their alteration so that at least you have the same evidence that the other side will almost certainly have in a lawsuit or a regulatory audit. While the ideal state is never to have incriminating emails, if you have more than zero employees in your company that’s unlikely to happen.
All of this sounds quite basic, but our work has demonstrated that some are still under the false impression that the process of deleting email actually deletes email. In reality, it does delete email, but only your copies of them – most are still out there somewhere out of your control. The best you can do is ensure that you have copies of your email that you can reliably assume others will also have.
Archiving as a defensive tool is well-trod ground: it’s an important best practice for eDiscovery, litigation hold, regulatory compliance, storage management, and end-user access to content. Every organization should archive their employees’ data to ensure they can meet these defensive uses of archiving – what we call Archiving 1.0.
But what about Archiving 2.0, or a more proactive use of archived data? Here are some things you can do with your archives:
Investigations: The ability to extract intelligence from the content within email archives can significantly reduce the amount of time spent on investigations, such as early case assessments in advance of an anticipated legal action, an investigation about inappropriate employee activity, or an investigation about why a key customer account was lost.
Sales support: Communications with customers independent of a CRM system can be used to determine how sales, support and other staff members’ emails correlate with customer retention and follow-on sales. Similarly, the speed and quality of responses to customer inquiries can be correlated to sales in order to determine how best to respond to inquiries in the future.
Risk mitigation: Archived data can be used to mitigate risks from data breaches, employee fraud and related types of threats. Senior managers can look for employees who are more likely to commit fraud by looking for managers who are treating their employees badly, they can find employees who are communicating with an organization’s competitors, transferring sensitive files to a personal email address, or running a personal business on company time.
Customer service: Archived data can be useful in determining who in an organization is talking with specific customers, to whom in the customer organization they are speaking, the content of their conversations, and other relevant information.
Supply chain management: Another application is analyzing messaging and relationship intelligence to visualize employee communication with unauthorized parties.
Litigation management: Legal can use messaging and relationship intelligence to zero in on individuals or domains to understand communication trends and which individual(s) or domain(s) needs to be investigated further, enabling useful pre-trial or pre-litigation discovery information.
IT support: Help desks can become more proactive by conducting ongoing investigations into what employees are saying about particular applications, the goal of which is to address problems as early as possible.
Human capital management: An archive can be used to determine when employees are going to leave an organization and thereby minimize the impact of an employee departure.
We have written a white paper that focuses on Archiving 2.0 – please feel free to download it here.