1. C O N T E N T
Electronicrecordvolumesareincreasingasenterpriseandbusinesssystemsarmedwithaggregatedcomputing
processingpowercreatehundredsofthousands—evenmillions—ofrecordswithinasinglebatchprocess.This
whitepaperoutlinesissuesthatmakemanagingtransactionalelectronicrecordschallenging,demonstrateswhy
organizationsneedtotamethehigh-volumetransactionalrecordsbeast,andlastly,describeswhattolookforina
transactionalelectronicrecordsmanagementsolution.
MANAGING TRANSACTIONAL ELECTRONIC RECORDS IN HIGH-VOLUME ENVIRONMENTS
RECORD MANAGEMENT’S BREAKING
POINT
2. RECORDS MANAGEMENT’S BREAKING POINT:
MANAGING TRANSACTIONAL ELECTRONIC RECORDS IN
HIGH-VOLUME ENVIRONMENTS
C O N T E N T
ABOUT AIIM
AIIM (WWW.AIIM.
ORG) HAS BEEN
AN ADVOCATE
AND SUPPORTER
OF INFORMATION
PROFESSIONALS FOR
NEARLY 70 YEARS.
THE ASSOCIATION
MISSION IS TO ENSURE
THAT INFORMATION
PROFESSIONALS
UNDERSTAND THE
CURRENT AND
FUTURE CHALLENGES
OF MANAGING
INFORMATION ASSETS
IN AN ERA OF SOCIAL,
MOBILE, CLOUD, AND
BIG DATA. FOUNDED IN
1943, AIIM BUILDS ON
A STRONG HERITAGE
OF RESEARCH AND
MEMBER SERVICE.
TODAY, AIIM IS A
GLOBAL, NON-PROFIT
ORGANIZATION
THAT PROVIDES
INDEPENDENT
RESEARCH, EDUCATION,
AND CERTIFICATION
PROGRAMS TO
INFORMATION
PROFESSIONALS. AIIM
REPRESENTS THE
ENTIRE INFORMATION
MANAGEMENT
COMMUNITY, WITH
PROGRAMS AND
CONTENT FOR
PRACTITIONERS,
TECHNOLOGY
SUPPLIERS,
INTEGRATORS, AND
CONSULTANTS.
EXECUTIVE SUMMARY
What you don’t see can sometimes be more dangerous than what you can.Visit the office of any large organization
and you’ll likely find file cabinets filled with documents and desktop computers runningWord, Excel, email, and other
personal productivity applications. All of this information—from mortgage loan applications and email exchanges
between employees to Excel customer invoices and training videos posted on the company website—is content that
every organization needs to manage.
For most organizations, properly classifying, retaining, and then retrieving these records across multiple offices
and business units is challenging and time-consuming, especially in light of ever-changing regulations and the
omnipresent risk of litigation. However, records created manually and semi-automatically represent just the tip of the
records management iceberg. Consider what’s under the surface: millions of records automatically created by high-
volume, transactional enterprise systems.These electronic records include customer statements, bills, checks, invoices,
and high-volume reports, representing the submerged, potentially dangerous portion of the iceberg. Like theTitanic,
organizations are often able to navigate around the part of the iceberg they can see, but it’s what’s under the surface
that can cause havoc.
Electronic record volumes are increasing as enterprise and business systems armed with aggregated computing
processing power create hundreds of thousands—even millions—of records within a single batch process.
Simultaneous input from hundreds, if not thousands, of users adds to the data volumes.
The good news is that organizations recognize that their electronic records management processes are not up to par,
reporting that electronic records are more than twice as likely as paper records to be“unmanaged.”1
The bad news
is that many lack a cost-effective, consistent, and efficient method of managing these records across the enterprise.
Not only are transactional electronic records out of control, but often organizations can’t find them.The Association
for Information and Image Management (AIIM) discovered that 41 percent of organizations are not confident that
electronic information (excluding emails) is“accurate, accessible, and trustworthy.”2
It’s ironic that although electronic records are the most voluminous records in any organization, they are the most
difficult to manage with current records management solutions—if they are managed at all. In an electronic
records environment, traditional records management solutions that rely on manual processes fall short. A records
management system adequate for retrieving, processing, and archiving several thousand records per day becomes
costly and unwieldy when applied to several million transactional records per day.
Like the volume of content itself, the task of getting these records under control may seem overwhelming. However,
a new method of managing electronic records does exist.This white paper outlines some of the issues that make
electronic records management so challenging, highlights why organizations need to tame the high-volume
transactional records beast, and lastly, describes what to look for in a transactional records management solution.
3. TAMING THE TRANSACTIONAL RECORDS BEAST
Electronic records are easy to create but difficult to manage. In a nightly batch job, a billing or payroll system
can generate thousands of records that need to be captured, classified, retained, and scheduled for disposal.
Organizations also need to integrate the management of these electronic records with other record types
created across the organization, including paper documents, XML, HTML, PCL, AFP, DJDE/Metacode,
PDF, PostScript, rich media formats, Word documents, Excel spreadsheets, emails, and Outlook calendar
entries. Organizations that attempt to apply manual and human-centric records management processes find
that these traditional methods wind up being extremely expensive and impractical for several reasons, as
described below.
OUT OF SIGHT, OUT OF MIND
Unlike paper documents that clutter up a file cabinet or storage room, electronic records reside in mainframe
or distributed systems—or both—out of sight and often out of mind. Like the late, US-based comic Rodney
Dangerfield, electronic records simply “get no respect.” According to AIIM, “Electronic records are still taken
less seriously than paper records.”3
Once applications create these records, organizations then need to store them either in the application itself
or, more typically, in external archiving or storage systems. But even this movement tends to be haphazard:
More than 70 percent of organizations report that they do not have plans or provisioning in place for the long-
term storage of electronic records.4
EXPLODING DATA VOLUMES
There should be little wonder that managing electronic records is so challenging, since data volumes are
exploding and show no signs of slowing down. For example, 18.76 billion payments crossed the Automated
Clearing House (ACH) Network in 2009, an increase of 2.6 percent over 20085
, and electronic single-stock
trading systems now create 37 percent of U.S. equity trading volume.6
Seventy percent of organizations report
that the electronic record volume is “increasing rapidly,” while none report a decrease.7
ACCESS BY ALL
It’s not just unwieldy volumes. The pervasive nature of electronic records is what makes them difficult to
manage. Unlike paper, these records can’t be confined to a locked file cabinet. Employees, customers,
suppliers—even hackers—can view, copy, disseminate, and destroy thousands of records in an instant. Those
organizations without an electronic records management strategy are most at risk, with 60 percent of these
organizations reporting that they are not confident that their electronic records have not been changed,
deleted, or inappropriately accessed.8
SILOED ENVIRONMENT
A technology environment in which siloed mainframe and distributed systems are the norm compounds the
challenge of managing electronic records. Almost a third of organizations have 20 or more siloed content
repositories and more than half have 10 or more.9
Yet the vast majority of records management solutions are
standalone systems that address the records retention needs of a single department or application rather
than integration with an enterprise-wide records retention plan.
TRANSACTIONAL
ELECTRONIC RECORDS
ARE EASY TO CREATE
BUT DIFFICULT TO
MANAGE. APPLYING
MANUAL AND HUMAN-
CENTRIC RECORDS
MANAGEMENT
PROCESSES WINDS
UP BEING EXTREMELY
EXPENSIVE AND
IMPRACTICAL.
4. RECORDS MANAGEMENT’S BREAKING POINT:
MANAGING TRANSACTIONAL ELECTRONIC RECORDS IN
HIGH-VOLUME ENVIRONMENTS
C O N T E N T
While organizations create these electronic records in siloed application systems, regulations and laws
require that organizations have the ability to treat these records as if they were created by or reside in a
single application. For example, a financial institution is court-ordered to place a legal hold on all records of
transactions involving an individual customer account. However, those records may include ATM withdrawals,
wire transfers, customer statements, and loan documentation housed in different application systems. If
the financial institution is global, these records could be contained in different applications throughout the
world, all with different retention and storage taxonomies—and under the governance of various disparate
regulations. The inability to place a hold on an account within the legal time limit may expose the organization
to incurring a multimillion dollar fine.
OUTDATED PROCESSES AND STANDARDS
In theory, the traditional principles of records management apply to any type of record. In practice, the high
volume of these electronic records makes paper-based processes for declaration, classification, and retention
tracking of electronic records costly and impractical. For example, while the U.S. Department of Defense
Directive 5015.2 and the Model Requirements for Electronic Records Management (MoReq) in Europe provide
guidelines that organizations can apply to electronic records management, these standards do not fully
address issues specific to high-volume, application-created electronic records. Rather, organizations need to
implement standards that consider the speed at which electronic records are created, stored, and disposed.
Many organizations use a manually-maintained file plan to manage their paper documents. When a
government agency passes a new record retention regulation, the records manager manually enters the
new retention criteria into the records management system for each impacted record type. This process may
work fine for a limited number of records but becomes exponentially expensive and unwieldy when manually
applied to billions of electronic records accumulated over several years. Organizations, especially those that
are international and must comply with a wide variety of country-specific jurisdictions, would require an
army of records managers to keep pace with changing regulations—as well as the overwhelming numbers of
records requiring oversight.
RETENTION COMPLICATIONS
The IT department typically is responsible for enterprise systems and often—by default—becomes
responsible for the classification and retention of the records within those systems. Rather than considering
risk, legal, or regulatory ramifications or whether the corporate “memory” is properly protected, the IT
department tends to view retention as a storage management issue.
Since the IT department is typically not well-versed in records management issues and may be unsure of how
to apply appropriate retention schedules, it may err on the side of archiving all electronic records. With storage
costs decreasing as a result of new technologies and options such as cloud computing, moving all electronic
records to low-cost or alternative storage may seem like an answer. However, without an enterprise-wide
electronic records management solution, organizations still cannot properly classify, retain, retrieve, and then
delete records stored in siloed systems and databases according to central retention policies.
A strategy of “keeping everything” can also result in penalties. In Germany, courts concluded that
telecommunications companies must delete telephone and Internet data, overturning the 2006 EU Directive
on Data Retention requiring companies to keep such data for six months. Telecommunications companies
failing to delete the data now face fines.10
THE U.S.
DEPARTMENT
OF DEFENSE
DIRECTIVE 5015.2,
AND THE MODEL
REQUIREMENTS
FOR ELECTRONIC
RECORDS
MANAGEMENT
(MOREQ) IN EUROPE
PROVIDE GUIDELINES
THAT ORGANIZATIONS
CAN APPLY TO
ELECTRONIC
RECORDS
MANAGEMENT.
THESE STANDARDS
DO NOT FULLY
ADDRESS ISSUES
SPECIFIC TO
HIGH-VOLUME,
APPLICATION
CREATED
ELECTRONIC
RECORDS.
5. ALTHOUGH
APPLICATIONS
CREATE HUNDREDS
OF THOUSANDS OF
RECORDS AS PART
OF A SINGLE REPORT,
THE LAW REQUIRES
ORGANIZATIONS TO
TREAT EACH AS AN
INDIVIDUAL RECORD.
IN SEVERAL HIGH-
PROFILE CASES,
COURTS HAVE HEAVILY
FINED ORGANIZATIONS
UNABLE TO
PROVIDE ACCESS TO
ELECTRONIC RECORDS
IN A TIMELY MANNER.
WHY HIGH-VOLUME TRANSACTIONAL RECORDS MANAGEMENT MATTERS
Is electronic records management—particularly that associated with high-volume, application-created
records—worth the effort? In a word: Yes. Those organizations not able to fully manage transactional
electronic records risk regulatory noncompliance, legal action, and unauthorized data access that can
negatively impact their business.
REGULATORY RISK
The Bernie Madoff $50 billion Ponzi scheme (2008), the Enron debacle (2001), and the accounting fraud
incident at Indian outsourcing provider Satyam Computer Services (2009) are just a few of the highly publicized
criminal events that have regulators considering passing a slew of new policies designed to improve
information transparency. Although not focused on records management per se, regulations including the
Markets in Financial Instruments Directive (MiFID), the Health Insurance Portability and Accountability Act
(HIPAA), the USA PATRIOT Act, the Sarbanes-Oxley (SOX) Act, the U.S. Food and Drug Administration’s 21 CFR
Part 11, the Consumer Product Safety Improvement Act (CPSIA), and the EU Data Protection Directive include
provisions directly attributable to electronic records management.
For example, while the major objective of MiFID is to harmonize Europe’s financial markets, MiFID Article
51(3) addresses records retention, specifying a five-year retention period for transactional data. HIPAA, which
governs electronic transfers of health information, mandates that organizations retain all medical records,
signed consent forms, authorization forms, and other HIPAA-related documentation for six years. It also
requires organizations to keep records for two years after a patient’s death. The Anti-Money Laundering
Directive 2005/60/ EC requires organizations to keep records for at least five years. Lastly, Directive 2003/88/
EC directs organizations to keep records of employee work hours and provide those records on demand.
In several high-profile cases, courts have heavily fined organizations unable to provide access to electronic
records in a timely manner. Unfortunately, good intentions and attempting to comply with regulations can
still result in hefty fines. For example, the Financial Services Authority (FSA) stated that Goldman Sachs
International did not “set out to hide anything,” but still fined the firm £17.5 million for failure to ensure it could
comply with UK regulatory reporting rules. Margaret Cole, managing director of enforcement and financial
crime for the FSA, stated, “This penalty should send a message—particularly to the senior management of
large institutions—of the need to have their firm’s UK reporting obligations at the forefront of their minds.”11
LEGAL RISK
Organizations need access to records to not only successfully defend a case, but also for proactive legal
activities. In the U.S., the 2006 amendments to the Federal Rules of Civil Procedure (FRCP) incorporated
specific e-discovery guidelines for Electronically Stored Information (ESI) used as evidence in a civil or criminal
case, including text, images, calendar files, databases, spreadsheets, audio files, animation, websites, and
computer programs. Other countries have passed similar rules governing e-discovery.
Although applications create hundreds of thousands of records as part of a single report, the law requires
organizations to treat each as an individual record. Searching for a single record within a batch file containing
thousands of records—or more—is akin to searching for a needle in a haystack. Organizations unable
to retrieve records in a timely manner face multimillion dollar lawsuits, not to mention damage to their
reputation and brands.
6. RECORDS MANAGEMENT’S BREAKING POINT:
MANAGING TRANSACTIONAL ELECTRONIC RECORDS IN
HIGH-VOLUME ENVIRONMENTS
C O N T E N T
ELECTRONIC
RECORDS ARE
CONSIDERED A
“TECHNOLOGY
ISSUE” AND
THEREFORE BECOME
THE DE FACTO
RESPONSIBILITY OF
THE IT DEPARTMENT.
IT DEPARTMENTS
CAN ADDRESS HOLDS
AND OTHER LEGAL
REQUESTS SINCE
THEY “OWN” THE
RECORDS, BUT THEY
MAY NOT POSSESS
THE EXPERTISE
OF LEGAL STAFF
WELL-VERSED
IN LITIGATION
PROCESSES.
AN ELECTRONIC
RECORDS
MANAGEMENT
SOLUTION SHOULD
LINK MAINFRAME
AND DISTRIBUTED
PROCESSING
TRANSACTIONAL
INFORMATION AND
STORE RECORDS
IN THEIR NATIVE
ENVIRONMENT. IT
SHOULD ALSO BE
HIGHLY SCALABLE.
Even those organizations that successfully fulfill e-discovery requests may ultimately spend millions of dollars
in man-hours and lost productivity to comply with legal imperatives. Further, company executives may also
become personally liable in certain situations and face individual fines or jail time.
Even in an increasingly litigious environment, more than half of organizations lack e-discovery procedures.
A recent AIIM survey revealed that while 71 percent of organizations address the legal hold of paper records
in the event of litigation, only 57 percent have a process in place for electronic records.12
Perhaps part of the
reason for a lack of ESI discovery process is that electronic records are considered a “technology issue” and
therefore become the de facto responsibility of the IT department. The organization may ask the IT department
to address holds and other legal requests since they “own” the records, but the IT department may not
possess the expertise of legal staff well-versed in litigation processes.
SECURITY RISK
Electronic records often contain sensitive information, including employee Social Security numbers and
customer account information, making these transactional records attractive to hackers. High-profile data
breaches, such as the 2007 theft from TJX Companies, Inc. of transactional record data that compromised
the security of 40 million credit and debit cardholders, highlight the need for organizations to integrate
data storage and security with electronic records management processes to help protect such sensitive
information.
CRITICAL COMPONENTS OF AN ELECTRONIC RECORDS
MANAGEMENT SOLUTION
An electronic records management retention strategy must enable organizations to mitigate regulatory, legal,
and security risks while addressing the unique challenges of electronic records. Rather than requiring a rip-
and-replace of legacy systems, the solution should manage records across multiple environments yet provide
a single, consolidated view; automate records management tasks, including classification and retention;
enable individual record management; and provide records managers with easy-to-use administrative tools.
WORK WITHIN A HYBRID ENVIRONMENT
Most large organizations operate in a heterogeneous technology environment of both legacy mainframe
applications and distributed processing systems, and will continue to run this environment for the foreseeable
future. However, many records management solutions ignore the reality of this technology environment and
require organizations to sunset legacy systems in order to implement an electronic records management
practice. For the majority of organizations, a wholesale replacement of legacy systems is cost-prohibitive and
overly disruptive to their operations.
Rather than forcing a migration to a different platform, an electronic records management solution should link
mainframe and distributed transactional information and store records in their native environment. It should
also be highly scalable.
7. MANAGING THE
ENORMOUS
VOLUMES OF
TRANSACTIONAL
ELECTRONIC
RECORDS
REQUIRES A
SOLUTION THAT
AUTOMATES ALL
PROCESSES.
SINGLE VIEW INTO MULTIPLE REPOSITORIES
Even though the ideal solution can administer electronic records in different repositories, it should contain
an interface that provides a single view into records stored in many different locations throughout the
organization, regardless of the type or age of the record.
AUTOMATE TRANSACTIONAL RECORDS MANAGEMENT
Managing the enormous volumes of transactional electronic records requires a solution that automates all
processes. First, the solution should ingest application-generated content from mainframe and distributed
systems, whether from enterprise-wide systems or line-of-business applications.
Next, it should apply predefined business rules that determine who has access to the record, where to keep
it, and how long to keep it. The solution should automatically extract metadata for classification and assign
a disposition rule based on company requirements. Finally, the solution should automate retention. If the
corporate retention policy changes, whether due to regulatory updates or corporate mandates, records
managers need only define the new retention criteria once and the solution subsequently applies those
criteria across all existing and new transactional records.
PROVIDE GRANULARITY
Even though applications produce transactional output containing multiple records, legal and compliance
requirements force organizations to identify these records individually. Therefore, the solution should present
records inside of bulk reports as individual records in taxonomy and enterprise file plans, and enable access
controls at the record level. Organizations should be able to hold an individual record in a large report without
retaining the complete report—and then be able to destroy individual records based on business events
without affecting other records in the same archive.
RECORDS MANAGER INTERFACE
To be most efficient and effective, a records manager needs an automated, easy-to-use front end to
administer the solution. A Web-based user interface based on an open architecture is ideal for providing
records managers with a full-range of administrative functions. Records managers can then define
hierarchical taxonomies for automatic classification, automatically define and subscribe to events from line-
of-business applications, and define a corporate retention schedule with multiple policies and rules, applying
those rules to specific record types.
CONCLUSION
Just as the Titanic was sunk by what was under the surface, your organization can also be “sunk” by what you
don’t see: millions of electronic records. Without an automated, enterprise-wide solution specifically designed
for high-volume electronic records management, your organization risks regulatory action, litigation, and
security breaches.
