Audit trails are the single largest cost component

1. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 1
Comments on the Utility vs.
Burden of Audit Trails
“Audit trails are the single largest cost
component of 21 CFR 11 compliance.”
John Doe, presenting at CHPA / FDA 1999

2. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 2
A Word From Our Sponsor
Subpart B—Electronic Records
§ 11.10 Controls for closed systems.
…Such procedures and controls shall include the following:
(e) Use of secure, computer-generated, time-stamped audit trails
to independently record the date and time of operator entries and
actions that create, modify, or delete electronic records. Record
changes shall not obscure previously recorded information. Such
audit trail documentation shall be retained for a period at least as
long as that required for the subject electronic records and shall
be available for agency review and copying.

3. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 3
Part 11’s Literal Meaning
• The only transactions that need audit trails
are ones performed by “operators”
• The only data that is required to be in the
audit trail itself is the date and time
– This means we do not have to replicate data from
the transaction in the audit trail
– Technically, we do not we do not even need to
record the operator’s ID
• There are some very good reasons to take a
minimalist approach to audit trails

4. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 4
Audit Trails - Current
Pharmaceutical Model
• Audit trails are usually replications of a
subset of a transaction record
– “Source record” >>> “Audit record”
• Audit records are usually stored in a similar
(if not the same) data structure
• Ubiquitously, audit records have the same or
lower security level as source records
• Hollis refers to this scheme as
“Data-level Audit Records”

5. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 5
Data-level Audit Records
(Creating a New Record)
SOURCE DATABASE AUDIT DATABASE

6. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 6
Data-Level Audit Records
(Correcting a Typographic Error)
SOURCE DATABASE AUDIT DATABASE

7. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 7
Audit Trails – Current
Financial Model
• The term “audit trails” is misleading; these
are actually “audited transactions”
– System A proposes transaction
– System B proposes agreement
– System X (the security system) examines
• The data labelling
• A’s and B’s privileges
• The structure of the transaction
– System X grants permission for the transaction
• And keeps a log
– All in real-time

8. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 8
System-Level Audit Records
(Any Type of Transaction)
SOURCE DATABASE JOURNAL FILE
Read:Cust_Rec:tquinn2270;
*.*||
Writ>:Xact_prop:tquinn2270;
Cur_Bal;310.65||
Read:ACF_2_Rcpt:Auth_cod:
<result>||
Writ:tquinn2270:Cur_Bal;
310.65:Auth_cod;<result>||

9. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 9
Comparing the Two
• Data-level audit trails:
– Are much easier to program and run
– Tend to produce larger record sets
– Keep the audit and source data in the format
– A MUCH easier to compromise
• System-level audit trails:
– Are much more difficult to include in designs
– Tend to produce smaller record sets
– Keep the audit and source records separate
– Are MUCH more difficult to compromise

10. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 10
Risk Analysis
• Data-level audit records and source data are
(about) equally vulnerable to insider threats
– Insiders are the most common threat
• Replicating data-level audit records provides
outsider adversaries with two attack vectors
– It’s more effective to invest in other defenses
• System-level audit records are only useful in
prevention if they are used in real-time
– In order to assist with detection, they must be
periodically and meticulously reviewed

11. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 11
Recommendations
• Do NOT change the audit trail wording of 21
CFR § 11.10 (e) to require more information in
the audit trail
• Perform a Regulatory Flexibility Analysis to
justify the requirement for audit trails, and
include details of:
– Financial burden of audit trails, particularly upon
small and disadvantaged businesses
– Raw and normalized statistics of when audit trails
have been useful in protecting public health

12. TM
Subject:
The Hollis
Group, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.
Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.QuinnSlide # 12
Questions?
Thomas Quinn, President
The Hollis Group, Inc
37 North Valley Rd. #105
Station Square II
Paoli, PA 19301
tquinn@hollisgroup.com
www.hollisgroup.com
v: 610.889.7350 f: 610.296.2339