The 10th Annual Utah Health Services Research Conference: Evaluating information quality in the detection of pediatric asthma encounters. By: Andrew J. Knighton, PhD, CPA - Institute for Healthcare Delivery Research Intermountain Healthcare Patient Centered Research Methods Core, University of Utah, CCTS

1. Evaluating information quality in the
detection of pediatric asthma encounters
ANDREW J KNIGHTON, PHD, CPA
INSTITUTE FOR HEALTH CARE DELIVERY RESEARCH
INTERMOUNTAIN HEALTHCARE

2. Problem
• Quality is an attribute of data
• Meaningful variation in health data quality
• Variation in type and scope
• Eliminating errors entirely is not feasible
• Impact of data quality on decision quality varies

3. Defining terms
Data:
4.21 liters
Information:
Mr. X has an FVC of
4.21 liters on
1/1/2015
Knowledge:
Mr. X’s falling FVC
levels indicate a
problem
Increasing interactions and interrelationships
Increasingcomplexity
Blum (1986); Nelson and Joos (1989)

4. The study
• A retrospective cohort design was used to study patterns of asthma
readmission at a large urban pediatric hospital using administrative
data (Knighton, 2013)
• Variation in defining asthma encounters in the literature and in practice
• Given the pairing of asthma with certain respiratory conditions, we
developed an asthma encounter detection algorithm using two criteria
(Knighton, 2014):
• Approach: We compared the results of the algorithm against a “gold
standard” criteria for asthma encounter detection through medical
record review (n=110)

5. What we found
• Improved detection
accuracy from 0.55
[95% CI: 0.46-.064]
using primary dx only
to 0.83 [0.76-0.90]
• Increased asthma
encounters 55%
• Increased bed days
64%
• Increased estimate of
the economic impact
of asthma 71% from
$8.6M to $14.8MKnighton AJ, et al, Population Health Management, 2014
Figure 1. Receiver operating characteristic (ROC) curve by asthma ICD-9
diagnostic code position (1-5) for a sample of select respiratory encounters
(n=110), including 95% CI for both sensitivity and false positive rate.

6. Implications
Knighton AJ, et al, Population Health Management, 2014
Figure 1. Receiver operating characteristic (ROC) curve by asthma ICD-9
diagnostic code position (1-5) for a sample of select respiratory encounters
(n=110), including 95% CI for both sensitivity and false positive rate.
A
Specificity Sensitivity
B C
Case Detection Methods

7. Particular use
Required evidence
quality
Selecting a case detection method
Case detection method

8. Linking particular use with case detection
Particular Use Example
Nature of
Decision
Required Evidence Quality
Case Detection
MethodSensitivity
False Positive
Rate
Accuracy
Clinical trial
research
Identify
intervention
cases/controls
Patient-level Low to High None Low to Medium A
Observation/
evaluation
studies
Retrospective
health services
research study
Population-level Medium to High Low to Medium Medium to High A,B
Quality
improvement
analyses
Improve health
services process
for asthma care
Process-level Medium to High Low to Medium Medium to High B,C
Cost-
effectiveness
analyses
Impact of
community
intervention
Population-level High Low to Medium Medium to High B,C

9. Conclusions
• Multiple documented methods exist today for detecting asthma encounters
• Using ROC analysis to define evidence quality, we can reasonably link particular
use categories with appropriate case detection methods
• A terminology familiar to producers and consumers of health data
• Categorizing particular use could allow for improved standardization of
measurement
• Potential for broad application to other health conditions and particular uses

10. References
Blum B. Clinical Information Systems – A Review. West J Med. 1986; 145(12): 791-797.
Knighton AJ, Flood A, Speedie SM, Harmon B, Smith P, Crosby C, et al. Does initial
length of stay impact 30-day readmission risk in pediatric asthma patients? J Asthma.
2013; (6): 1–21.
Knighton AJ, Flood A, Harmon B, Smith P, Crosby C, Payne NR. A novel method for
detecting inpatient pediatric encounters using administrative data. Population Health
Management. August 2014, 17(4): 239-246.
Nelson R, Joos I. On language in nursing: from data to wisdom. PLN Vision. 1989, p6.
Wang R, Strong D. Beyond accuracy: what data quality means to data consumers.
Journal of Management Information Systems. 1996; 12(4); 5-34.