This document proposes a quantitative methodology for evaluating the size and comprehensiveness of an integrated data repository (IDR). It describes developing measures across different levels, from raw data sources up to aggregated views, to assess aspects like clinical scope, temporal coverage, and included data types. Examples are provided of IDRs at different medical organizations. The goal is to facilitate monitoring improvements over time or comparisons between IDRs in a way that is intuitive and does not weight some data types over others. The results would provide snapshots of IDRs that characterize their capabilities for clinical and research users.

1. IDR Snapshot:
Quantitative Assessment Methodology
Evaluating Size and Comprehensiveness of
an Integrated Data Repository
Vojtech Huser, MD PhD
James J Cimino, MD
Laboratory for Informatics Development
NIH Clinical Center
NLM

2. Research question
• How can you evaluate an IDR?
• What makes a good IDR ? (for research)?
• IDR A(in 2007) vs. IDR A (in 2012)
• IDR A vs. IDR B
Vojtech Huser http://code.google.com/p/idrsnapshot/ 2

3. Vojtech Huser http://code.google.com/p/idrsnapshot/ 3

4. Motivation/Assumptions
• Improving IDR
– while acknowledging the a-priori limitations
• Ideal IDR for a researcher
– versus existing data
• General measure on whole-IDR level
– Not research project specific
• Build on CTSA IDR surveys from 2007, 2008
and 2010
Vojtech Huser http://code.google.com/p/idrsnapshot/ 4

5. Examples
• Academic medical center
• with limited number of outpatient clinics
– University of Utah
• Integrated delivery network
• Outpatient and inpatient records
– Partners Healthcare
• HMO
• Health Plan component
– Kaiser Permanente South California
Vojtech Huser http://code.google.com/p/idrsnapshot/ 5

6. Target level (researcher-facing schema)
• Level 1: Epic Clarity (6000+ tables)
• Level 2: Clinical Data Repository (add data from
GE Centricity (outpatient) (plus other sources)
• Level 3: subset of CDR for health plan members
only
• Level 4: i2b2, or VDW, or OMOP, or XYZ
(Deduce, Further, BTRIS)
Vojtech Huser http://code.google.com/p/idrsnapshot/ 6

7. Beyond core data sources
– Diagnoses, Procedures, Labs
– ADT data (admission, ICU)
– Visit data
• Less-common data sources
– Insurance history data
– Over the counter drug data
– Death certificate data
– Link to other sources
– Pharmacy dispensing data
– Out of network claims data
• Health Plan data
» Heath Assessment questionnaire data
» Out of network pharmacy refills data
Vojtech Huser http://code.google.com/p/idrsnapshot/ 7

8. Measures
• Glasgow comma scale of 10 vs. 5
• Apgar score of 7 vs. 5
•
• Good measure:
– Intuitive to interpret (count of patients)
– facilitates monitoring and improvement
– does not place any arbitrary value on individual
measure components
– (e.g., value of 10 years of medication history vs. 10 years of
weight/height history).
Vojtech Huser http://code.google.com/p/idrsnapshot/ 8

9. “Consortia made easy(ier)”
Vojtech Huser http://code.google.com/p/idrsnapshot/ 9

10. Results (site A, site B)
Vojtech Huser, IDR Snapshot
10
http://code.google.com/p/idrsnapshot

11. Vojtech Huser http://code.google.com/p/idrsnapshot/ 11