The document discusses the evolution of NASA's Aviation Safety Reporting System (ASRS) and Patient Safety Reporting System (PSRS) from a primarily paper-based report processing system to a fully electronic system. Key points: - ASRS and PSRS previously received paper reports by mail and used paper forms to code reports, but were moving to accept more electronic reports. - Three tools were developed - Electronic Report Submission, an Analyst Workbench, and an online Database - to transition to a fully electronic processing system. - The usage-centered design approach was used to identify requirements and ensure the new electronic systems supported the work analysts and users needed to accomplish. - The new systems allow

1. NASA ASRS:
Evolution Toward Full
Electronic Processing
NASA Aviation Safety Reporting System (ASRS)
and Patient Safety Reporting System (PSRS)
PM Challenge 2010
Used with Permission
Linda J. Connell Lester Gong
Director, NASA ASRS & PSRS IT Manager, Booz Allen Hamilton
Elisa Taube Jessica Arias
Research Manager, Booz Allen Hamilton PSRS Deputy PM, Booz Allen Hamilton

2. Agenda
• Background: ASRS / PSRS & Report Processing
• The Evolution Towards Full Electronic Processing
• Solutions for the Evolution of Report Processing
• Approach Methodology
• Before & After: The Evolution of Report Processing
• Current State of the Evolution
• Next Steps / Future Enhancements

3. ASRS and PSRS

4. What are ASRS and PSRS?
• Two separate incident reporting systems:
 ASRS = Aviation Safety Reporting System
 PSRS = Patient Safety Reporting System
• Managed and operated at NASA Ames Research
Center, Moffett Field, California
• Based on three guiding principles:
Voluntary
Confidential
Non-punitive

5. The Aviation Safety Reporting System (ASRS)
• Established in 1976 through interagency
collaboration between NASA and the Federal
Aviation Administration (FAA):
 An independent system for aviation safety
 One of the first lines of defense in identifying
safety issues
 NASA chosen as “Honest Broker”
• ASRS database is a national asset of U.S.
aviation safety data

6. The Aviation Safety Reporting System (ASRS)
• Receives, processes and analyzes voluntarily
submitted incident reports from
 Pilots
 Air Traffic Controllers
 Flight Attendants
 Maintenance Technicians, and others
• Reports submitted to ASRS may describe both
unsafe occurrences and hazardous situations
• ASRS's particular concern is the quality of
human performance in the aviation system

7. The Patient Safety Reporting System (PSRS)
• Initiated in May 2000 via an Interagency Agreement
with the Department of Veterans Affairs (VA)
• Modeled after the ASRS
• A venue for frontline hospital staff to report patient
safety issues externally
• Accepts reports from:
 Physicians
 Nurses
 Ancillary staff (Pharmacy, Physical / Respiratory
Therapy)
 Other

8. The Patient Safety Reporting System (PSRS)
• Collects and analyzes voluntarily submitted
patient safety reports including:
 Close Calls
 Events
 Suggestions
• Strengthens the culture of safety
• Supports the foundation of medical human
factors safety research

9. Report Processing

10. What is Report Processing?
• Report Processing: the methodology for
codification of incident reports to facilitate
extraction from the database for learning
and safety information
• Both programs follow a standard report processing
model of rapid screening, multiple report matching,
alert identification and codification
• Begins with the receipt of the report and ends with
the final coded report entered in either the ASRS or
PSRS Database

11. Report Processing Flow
Each report is read by two expert safety
analysts within three working days

12. The Evolution Towards Full
Electronic Processing

13. Why Evolve?…Paper Based for Decades
Report Processing
 Users sent 100% of reports by U.S. Mail
 All incoming reports were routed in hardcopy
 Analysts used a 12 page paper coding form to select relevant
information for database insertion
 Capitalize on data that is generally more available in
electronic form
 In 2002 ASRS began to receive airline ASAP reports sent
electronically that required in-house printing to enter the
processing flow
 Database “keyers” typed in all coded reports and narrative text
Public Accessibility to Data
 Our public ASRS database was not searchable online
 Access to data was by request only

14. ASRS Reporting Volume
January 1981 – December 2008
 ASRS receives an
5,000
average of 4,200
Smoothed / Forecast
4,500
Actual Intake
report/month - 193 4,000
per working day 3,500
3,000
 Total Report intake 2,500
for 2008 was 50,405
2,000
1,500
1,000
500
0
'81 '82 '83 '84 '85 '86 '87 '88 '89 '90 '91 '92 '93 '94 '95 '96 '97 '98 '99 '00 '01 '02 '03 '04 '05 '06 '07

15. Why Evolve?…
• Increasing pressure for:
 Users to submit reports electronically
 Decreasing processing time / cost
Paper process has limited throughput
Report Volume continues to increase
 A Method that leverages electronically submitted
reporting form data and reduces data input costs
 Searchable databases to access relevant reports for
research, rulemaking, and safety purposes

16. Why Evolve?…
• Identified Challenges:
 Need a well planned and smooth transition
Immediate cutover with no effect on high volume report
production
No opportunity to cease or pause production for the
evolution
 Innovative solutions needed to be developed
Internal staff must shift paradigm for report handling
including special cases such as alert level reports

17. Solutions for The Evolution
of Report Processing

18. Solutions for Evolution
Launched development of 3 Essential Tools:
Electronic Report Submission (ERS) – A External,
method for ASRS and PSRS Reporters to submit on website
reports electronically
Analyst Workbench – A browser based internal Internal for
tool used by ASRS or PSRS Analysts to process Processing
reports through 25 stages
Database Online (DBOL) – Internet accessible, External,
browser based self-serve application for users on website
to search the databases

19. Approach Methodology

20. Approach Methodology
• Used Standard Project Management Techniques1:
 Identified requirements
 Established clear and achievable objectives
 Balanced quality, scope, time, and cost concerns
 Accommodated concerns of key stakeholders
• In order to identify requirements and accommodate
stakeholders, a Usage-Centered Approach2 was used
1 PMBOK Guide, p. 9, 10
2 Constantine, L., & Lockwood, L. (1999). Software for use.
Addison-Wesley-Longman, Inc., NY, NY., p. 23-25

21. Approach Methodology
• Usage-Centered Approach1 – focuses on the
work that users are trying to accomplish and on
what the software will need to supply via the
user interface to help them accomplish it
 User centered design represents a shift of focus
from technology to people, from user interfaces
to users. To design dramatically more usable
tools, however, it is not just users who must be
understood, but usage
1 Constantine, L., & Lockwood, L. (1999). Software for use.
Addison-Wesley-Longman, Inc., NY, NY., p. 23

22. Approach Methodology
• Utilized Basic Elements of Usage-Centered
Approach 1
 Pragmatic Design Guidelines
 Model-driven design process (task models or use
cases, content models, role models)
 Organized development activities
 Iterative improvement
 Measures of quality
1 Constantine, L., & Lockwood, L. (1999). Software for use.
Addison-Wesley-Longman, Inc., NY, NY., p. 24-68

23. Before & After: The Evolution
of Report Processing

24. Solutions for Evolution
Launched development of 3 Essential Tools:
Electronic Report Submission (ERS) – A method External,
for ASRS and PSRS Reporters to submit reports on website
electronically
From Reporting Form to Electronic Reporting
Analyst Workbench – A browser based internal Internal for
tool used by ASRS or PSRS analysts to process Processing
reports through 25 stages
From Coding Form to Online Data Coding
Database Online (DBOL) – Internet accessible, External,
on website
browser based self-serve application for users
to search the databases
From Formal Requests to Direct Access

25. Electronic Report Submission (ERS)
• General Requirements
 Internet accessible, web browser based to accommodate
diverse geographic locations and computer workstations
 Mimics paper form; Intuitive and easy to use
 Ability to print form online for paper submission
 No caching or saving on user computer to protect
confidentiality
 Secure Transmission of data and “on the fly” encryption
 Autopopulates to new Workbench Tool
 Ability to reconstruct original report for Analysts in
electronic form for analysis

26. BEFORE -
Paper Reporting Forms sent by U.S. Mail

27. AFTER -
Reporting Forms Securely Submitted Online
http://asrs.arc.nasa.gov/

28. AFTER -
Reporting Forms Securely Submitted Online

29. Analyst Workbench
• General Requirements
 Browser based & cross platform
 Customized interface based on role
 Single electronic application for completing
and tracking 25 steps in report processing
 Leverages staff “mental models” for analyzing
paper reports
 Reduce errors and keying labor hours by pre-
populating electronic data

30. Before & After Report Processing
BEFORE
AFTER

31. BEFORE -
Coding Form – 12 Pages in Hardcopy

32. BEFORE -
Javascript Data Entry GUI

33. AFTER -
Dual Monitors, Synchronized Displays

34. AFTER -
Analyst Workbench Main Menu

35. AFTER -
Analyst Workbench Full Form Codification Screen

36. Database Online (DBOL)
• General Requirements:
 Internet accessible, web browser based self-serve
query application
 Enable stakeholder communities to construct complex
SQL queries utilizing a simple interface
 Export to variety of formats (Excel, CSV, Word) for
ease of data manipulation
 Rapid topical searches of over 150,000 records and
10 million rows of data
Simple approach (just text) or more advanced queries
are possible

37. BEFORE – Database Online (DBOL)
Internal tool - complex with a high requirement for data structure knowledge

38. AFTER – Database Online (DBOL)
Public tool – Intuitive, easy to use. Serves casual user and domain expert.

39. Current State of the Evolution

40. Current state of
Electronic Report Submission (ERS)
• On Day 1, October 16, 2006:
 First electronic report from ASRS website came within minutes
of launch
 ASRS received 53 ERS reports in the first week
• ASRS received 50,405 total reports in CY 2008. Breakdown in
table below*:
Totals Total %
Electronic 37,076 73.6%
Paper 13,329 26.4%
Totals 50,405 100.0%
*Data thru October 2009

41. Current State of
Database Online (DBOL)
• Database Online Metrics
 In 30 years, 7,100 Search Requests (SRs) have been directly
provided by ASRS Research Staff to various aviation
organizations and agencies
 Since July 2006, 58,857 data queries have been accomplished
by external users since the launch of ASRS Database Online
(DBOL)1
1Thru October 2009

42. Current State of
Analyst Workbench
 May 5, 2009 – smooth transition from paper to
electronic processing was accomplished
 Report analysis staff have been active in quickly
identifying issues and proposing innovative solutions
 Productivity was improved. Average time to process
a report initially has shown increased efficiency
 Need for data keying and printing reports eliminated
 Report tracking requirements have dramatically
improved
Ease of tracking location of each report in process

43. Next Steps / Future Enhancements

44. Next Steps
• Continue to improve usability of the tools
• Electronic Report Submission:
 Shift to html versions to avoid technical difficulties due
to advancing .pdf technology
• Analyst Workbench:
 Enhance tool to allow for adaptability to new domains
 Incorporate workflow editing
• DBOL:
 Use of javascript libraries to simplify handling of client
side code
 Leverage AJAX technology to enhance user
experience