This document provides an overview of registry participation and collecting patient-reported outcome measures through a registry. It discusses the University of Wisconsin's process for collecting PROs in their orthopedic clinics in two phases: a pilot phase and a full implementation phase. The pilot involved collecting PROs in 6 clinics using Epic and tablet computers. Lessons learned included that an integrated tablet/portal solution and coordinated project management were important. The full implementation will expand PRO collection to all orthopedic locations and improve reporting automation.

1. Registry Participation 101:
A Simple Step-by-Step Guide to Understanding What You Really Need
to Know from AJRR
Joe Greene
University of Wisconsin—Department of Orthopedics and Rehabilitation
AJRR Staff:
Phil Dwyer, Program Coordinator
Caryn Etkin, PhD, MPH, Director of Analytics
September Cahue, MPH, Senior Registry Analyst

2. Topics and Agenda
Agenda:
• Introduction to clinical data registries Phil Dwyer
• University of Wisconsin experience Joe Greene
• Patient-Reported Outcome Measures September Cahue
• Quality programs Caryn Etkin
Topics:
• What is a clinical data registry
• Who is using registry information and why
• Value of registry data
• Additional information, resources

3. Origin of Registries –
Sweden

4. What is a Registry?
• A clinical data registry records information about the
health status of patients and the health care they receive
over varying periods of time
• Designed to capture clinical data of importance to health
care practitioners, providers, and patient. Generally,
registries focus on clinical data for specific populations
based on:
– A disease/condition (e.g. cystic fibrosis, spina bifida)
– Procedure (e.g. coronary artery bypass graft)
– Device or drug (e.g. total joint replacement)
• More frequently are able to capture data from multiple
EHRs
• Designed for quality improvement efforts, payment,
research, etc.

5. Who Manages Registries?
• Medical Specialty Societies
• Health Plans
• Patient Advocacy Organizations
• Federal and State Governments
• For-profit Companies and Non-profit
Organizations

6. Who is Interested in Registries?
• Hospitals, health systems
• Private practice groups
• Health plans and other payers
• Researchers
• Patient and consumer organizations
• Federal, state, and local government

7. Look at Some of the National
Options!

8. How Does a Registry Collect Data?

9. How Does a Registry Collect Data?
• Push
• Pull
• Certification
model
• Manual
• Linking with
different
databases
• Departmental
systems
• Direct from
patients
• From devices
Electronic
Health Records
Other IT
systems
Patients

10. Clinical Registry Information
Flow
Provider
documents
patient care
EHR
Registry
Provider
performance report
Provider reviews
report
Patients can
enter their own
data into some
registries
How can I
improve my
performance?

11. More Patients, Less Burden
Clinical
registries
Collecting
structured data
On more patient
populations
Data pull
from EHRs
Is becoming less
dependent on
chart abstraction
Lowering
barriers
Source: https://download.ama-assn.org/resources/doc/cqi/x-pub/pcpi-richardson-102915.pdf

12. AJRR is a multi-stakeholder, independent, not-for-profit with diverse
national constituents led by top orthopaedists
We optimize patient outcomes through collection of data on all
primary and revision total joint replacement procedures in the US
Our mission is to enhance patient safety, improve quality of care, and
reduce the cost of care
Our goal is to capture 90% of all total joint replacement
procedures in the U.S.
AJRR Mission and Vision

13. AJRR is supported by hospitals, health insurers, and
the following groups:
Supporters

14. 20 26
56
79
122
158
177
219
242
269
292
388
417
467
522
612
677
0
50
100
150
200
250
300
350
400
450
500
550
600
650
700
Jan July Jan July Jan July Jan Jul Currently
Total # of
hospitals*
20152012 2013
NumberofHospitals
2014
N=24,971
procedures from
63 hospital ls
N= 43,823
procedures from
123 hospitals
AJRR 2012-2016 Progress
N= 211,721
procedures from
252 hospitals
N= 350,154
procedures from
464 hospitals

15. Where Are We Now?
September 2014
The National Registry:
All 50 States

16. Participating Institution Types
Participating Institution Types
• Hospitals
• Academic Medical Centers
• Multiple Health Systems/Networks
• Ambulatory Surgery Centers
• Private Practice Groups
• Solo Practitioners
• Hospital bed count
– Small = 1-99 beds
– Medium = 100-399 beds
– Large = 400+ beds
Teaching
25%
Rural
9%
Univ/
Research
11% Major
Teaching
(21.0%)
Minor
Teaching
(35.6%)
Non-
Teaching
(43.3%)
Small
(21.1%)
Medium
(48.0%)
Large
(30.9%)

17. Demand Reporting &
Electronic Dashboard System
TrainingAccessing and Comparing
Your Data

18. AJRR Dashboard

19. AJRR Dashboard – Reports

20. AJRR Dashboard

21. AJRR Dashboard

22. AJRR Dashboard

23. AJRR Dashboard

25. Joe Greene
University of Wisconsin
Department of Orthopedics
and Rehabilitation

26. Registry Participation 101
A Simple Step-by-Step Guide from AJRR
The Acquisition and Reporting of Patient
Reported Outcomes
Joe Greene
University of Wisconsin
Department of Orthopedics and
Rehabilitation

27. Objectives
 Outline the UW Health System approach to PRO
questionnaire deployment
 Define current AJRR reporting status and goals
 Describe Phase 1: The pilot collection of PRO
data in orthopedics
 Outline Phase 2: Full implementation of PRO
data collection and reporting
 Discuss lessons learned throughout

28. UW Health Orthopedics and
Rehabilitation
 UW Health Orthopedic Service Line
• 600+ employees
• Highly profitable service line with UW Health
• Highly comprehensive
 Department of Orthopedics and Rehabilitation
• 29 orthopedic surgeons and 16 rehabilitation providers
• 20+ PA’s and APNP’s
• 5 primary care sports medicine providers
 Research and Education
• 25 residents and 4 fellows
• 4 PhD Level Scientists and multiple post-doc’s
• Robust stem cell and regenerative medicine research
• Strong clinical research program

29. AJRR Data Elements
Level 1 Data
• Patient, Surgeon, and Procedure Data
Level 2 Data
• Risk Factors and Co-Morbidities
Level 3 Data
• Patient Reported Outcomes
Level 4 Data
• Radiographic Images

30. Current Status: AJRR Reporting
Level 1 Reporting
• Automated since June, 2014
Level 2 Reporting
• Goal of automating by monthly Epic extract by
June, 2016
Level 3 Reporting
• Automate by monthly extract of PRO data from
Epic by July, 2016
Level 4 Reporting
• Automate when AJRR is ready to accept

31. UW Health System Considerations
Patient Assessment Tools Workgroup
Epic vs Third Party Solution
Getting data in/out of Epic
IT and operational workflows implications
Tablet vs Kiosk vs Portal
Real time access to results
Copyright and cost implications

33. Discrete data storage within EMR
Reporting customization
Multiple data entry mechanisms
Meaningful Use benefits
No interface required
Leveraged Epic investment
Epic will improve over time
Decision: Develop Epic Solution

34. Streamlined collection processes
Eloquent and attractive user interfaces
Better than EMR functionality at present
My Recommendations
Strongly consider reporting capability
Interfacing ability is paramount
Consider depth of their questionnaire build
Third Party Solutions

35. Phase 1
Pilot Collection of PRO Data
6 Clinics at 2 different locations and approximately 100 users including
Staff affected : Physicians, AT’s, PT’s, PA’s, RN’s, MA’s, Schedulers

36. Phase 1
Project Overview Resources
UW Epic teams – 1024.75 hours
• EpicCare Ambulatory
• Cadence
• MyChart
• Prelude/Welcome
• Training Team
• Project Management
• Security
• Clinical Center for Knowledge
Management
• Server and Desktop
• Programming
Consultants (10 months)
• MyChart
• Programmer
• Reporting

37. Phase 1
Questionnaires
 ACL Reconstruction (Clinics: Sports Med, Sports Rehab)
• VR-12 Physical & Psychological Health Form
• IKDC Knee Evaluation Form
• ACL-RSI Return to Sport After Injury Scale
• MARX Activity Rating Scale
• SANE - Single Assessment Numeric Evaluation
 Total Hip Replacement (Clinic: Ortho)
• VR-12 Physical & Psychological Health Form
• HOOS Hip Disability & Osteoarthritis Score Survey
• UCLA Activity-Level Rating Form
• Modified Harris Hip Form
 Low Back Pain (Clinics: Spine Ortho, Neuro, Spine PT)
• VR-12 Physical & Psychological Health Form
• Oswestry Low Back Pain Form

38. Phase 1
Project Overview Workflows
 Assign Questionnaire Series
• Option 1: Scheduling question; Reporting
Workbench used by Scheduler
• Option 2: Order placed by clinician
 Assign a Questionnaire
• Patient Entered Questionnaire Activity
Available on DAR and Multi-provider schedule
 Patient entered questionnaires
• MyChart
• Welcome (Tablet Solution)

39.  Initially sought a Windows 7 tablet
• Located a product and began testing
• Lack of device quantity made the decision to use Windows 8
 Windows 8 models were tested (need about 75)
• Dell Model: Cost ~ $600.00 (Now about $800.00)
• Storage Units: Cost ~ $800.00 (Now about $800.00)
 Extensive testing completed
• 40 hours testing
 Every questionnaire
 Saved responses
 Response filing
 Ease of answering
 Single item on each page
 Auto advance
Tablet Selection and Testing

40. Questionnaire Volume
Pilot Go-Live
October 16th

41. Questionnaire Volume
Pilot Go-Live
October 16th

42. % QNRs completed in MyChart

43. % MyChart Active Patients per #
of Encounters

44. Phase 2
Full Implementation
 Build and validate all remaining orthopedic
questionnaires
 Deploy at all orthopedic locations
 Identify and implement all reporting Needs
 Consider research and HS-IRB implications

45. Phase 2 Considerations
Quality and Outcomes
Research
Clinical Reporting
Registry Reporting
Government and Reporting
Third Party Payors and Bundling
Marketing

46. Phase 2
New Questionnaire Build
 FAAM
 FAAM Sport
 NDI
 SRS 22
 UWRI
 SILS (CJR)
Ped’s QL 4.0
• 5-7, 8-12, 13-18
quickDASH
KOOS
Knee Society Score
PSFS (Flowsheet)

47. Phase 2
Reporting Automation
 Clinical and Marketing Reporting
• Automate Quarterly
 Registry Reporting
• AJRR Level 3
VR-12, HOOS, KOOS
(Why? : WOMAC, Historical Database, CJR)
Pre-op and 1 year
 Payor and Government Reporting
• CJR
VR-12, HOOS, KOOS Pre-op and 1 year
SILS and Oswestry
Total Painful Joint Count

48. Lessons Learned
 Integrated tablet and portal solution was critical
 Coordinated IT and Orthopedic project
management is imperative
 An Epic solution was right for the UW, but may
not be the best, easiest or cost effective in many
cases
 Consider reporting needs early
 Physician and Management Champions are
essential!

49. Questions
Joe Greene (JGreene@uwhealth.org)
Orthopedics Project Manager
Ryley O’Brien (RO’Brien@uwhealth.org)
Patient Assessment Tools Workgroup Facilitator
Sara Balster and Johnna Bledsoe
(Sbalster@uwhealth.org)
(Jbledsoe@uwhealth.org)
IS Project Managers
Leandra Frank (Leandra.Frank@uwmf.wisc.edu)
IS Analyst, Hardware
Bret Wagner (bwagner@huronconsultinggroup.com)
Huron Consulting, Director of Portals and Population Health

50. AJRR Patient Reported Outcome
Measures (Level III) Platform

51. • Electronic submission of PRO scores (already captured at
your hospital/clinic)
• AJRR will provide a data specification file – you submit data to us
(similar to Level I submission)
• Selected EHR vendors can submit to AJRR on your behalf
• Collection direct from patients (or facilitated by clinician) via
AJRR’s Level III platform
How AJRR Can Accept Data
Information can be
found in our PRO
Guide on
www.ajrr.net

52. How to Start a PRO
Program
1. Define your PRO team
2. Define your PRO goals
3. Determine the PRO measure/s you
would like to use
4. Define the frequency for data collection
5. Develop workflow for data capture

53. • Most likely, a core group of colleagues have decided that
including PROs in your practice of care for TJA may provide
important information to improve patient outcomes. Beyond
this core group, it will be critical to involve stakeholders at
each level of patient care
• Some groups to be considered:
• Orthopaedic Department
• Orthopaedic practice groups and clinics
• Quality Department
• Information Technology
• Orthopaedic Service Line
• Research
• Rehabilitation
• Hospital Administration
• Patient Advocate/Patient Representative
• Institutional Review Board (guidance/review)
Define Your PRO team

54. Define Your PRO goals
• For those interested in a PRO program, you and your
colleagues will need to determine the reasons you would
like to launch a PRO program
• You may consider asking yourselves questions such as:
• Are we launching a research initiative with specific aims?
• Are we seeking comparative benchmarks to our peers?
• Are we wanting to quantify our outcomes from our patients’
perspective?
• Do we want to measure if patients have improved function or
reduced pain?
• Do we want to measure whether our patients’ overall health has
improved?
• What are the requirements of the payer-specific program for which
we are hoping to qualify?

55. Determine the PRO Measure/s
• Your reasons for implementing PRO program will direct your
team to a certain instrument(s) that will allow you to meet
your objectives.
• For example, if your institution has decided to collect PROMs
for research initiatives, you may choose an instrument that
measures multiple domains.
• However, if you are focused on measuring patients’ outcomes
as part of your clinical care, a shorter instrument will suffice as
long as it is able to detect change.
Consider:
1. What does the measure assess – quality of life or
function?
2. What is the cost (licensing fee) to use the form?
3. What is the associated patient burden?
a. What is the length of form?
b. How easy/hard is the form to complete?
4. What is the associated staff burden (FTE time)?

56. Define Frequency for PRO
Data Collection
• PROMs guidelines from the International Consortium for
Health Outcome Measurement (ICHOM) have
recommended pre-op (baseline) and one-year follow-up
as appropriate time points for data collection to provide
meaningful data for comparing outcomes across
providers
• The CJR Final Rule also advises that post-operative surveys be
collected between 270 and 365 days post-surgery
• AJRR’s platform will allow for other time points (e.g.,
three-month, six-months, etc.) to be submitted and
stored in AJRR’s database. However, national benchmarks
will only be reported for pre-op and one-year outcomes.
• Each time point will have a two-month window for data
collection

57. Develop Workflow for Data
Capture
Consider the following:
1. Points of patient contact
2. Staff responsibilities
3. Electronic completion versus paper capture of forms
4. Collection at clinic or allow patients to complete on
their own
5. Will surgeon be using the measure responses during
the patient encounter?

60. Other PROM resources
• Patient-Reported Outcomes in Performance Measurement
RTI document
• International Consortium for Health Outcomes
Measurement - Hip & Knee Osteoarthritis Data Collection
Reference Guide
ICHOM document
• National Quality Forum – Patient Reported Outcomes
(PROs) in Performance Measurement
NQF document
• International Society for Quality of Life Research -
User’s Guide to Implementing Patient-Reported
Outcomes Assessment in Clinical Practice
ISOQOL document

61. Health-Related Quality of Life Measures
• VR-12: VR-12
• PROMIS-10 Global: PROMIS-10 Global
• SF-12: SF-12
• SF-36: SF-36
• EQ-5D: EQ-5D
Joint-Specific Measures
• HOOS and KOOS: HOOS and KOOS
• HOOS, JR and KOOS JR.: HOOS JR and KOOS JR
• Oxford Hip: Oxford Hip
• Oxford Knee: Oxford Knee
• KSS: Knee Society Knee Scoring System
• Harris Hip Score: Harris Hip Score
• WOMAC: WOMAC
Other PROM resources

62. AJRR and Quality Programs
• Comprehensive Care for Joint Replacement (CJR)
– CJR Final Rule
– AJRR's efforts related to CJR
• AJRR is also a Qualified Clinical Data Registry
(QCDR), which means Eligible Professionals can
use our platform for the Physician Quality
Reporting System (PQRS)
– AJRR's QCDR Platform
– QCDR Reporting
• Meaningful Use

63. Additional Registry Resources
• National Quality Registry Network NQRN Home Page
– Registry inventory, “What is a registry” handout, registry vendor
assessment
• CMS Qualified Clinical Data Registries QCDR Home Page
• AHRQ Registry of Patient Registries (RoPR)
• Numerous other efforts:

64. Questions
American Joint Replacement Registry
info@ajrr.net
847.292.0530
http://www.ajrr.net

65. 65
Managing Orthopedic Episodes of Care
www.wellbe.me
Wellbe is an authorized vendor to submit
your PRO data to AJRR!