2. To this point…
Weeks 1 & 2:
What are PBRNs?
What do PBRNs do?
Why are PBRNs important?
What are the key components of PBRNs?
How do PBRNs function?
Last week:
What are the steps in developing a PBRN?
How generalizable are PBRN research findings
to non-PBRN practices?
3. Tonight
• Louise Acheson, MD, MS
– Professor, Case Department of Family Medicine
• Topic:
– Internet-based data collection and assessment tools
in PBRN research
– Acceptability of these research tools to PBRN
practices and patients
– Potential interfaces with EMRs
– The evidence gaps in family history research
5. Jim Werner, PhD
CTSC PBRN Shared Resource
Case Department of Family Medicine
PBRN Data Collection
Methods
6. Data Collection
• In PBRNs, data are often collected
by:
– Research Assistants, Research Nurses
– Clinicians
– Practice staff
• Data collection methods
– Paper & Pencil
– Surveys
– Chart abstraction
– “Weekly return card”
– Electronic
– Dedicated electronic data collection systems
– EHR-based
– Audio and video recording (qualitative research)
7. The Weekly Return Card
• Clinicians systematically record
observations about consecutive patients
that meet enrollment criteria
• Pocket-sized card
• Forced choice or short completion
• Usually requires 1-2 minutes for a single
patient’s report
• Unobtrusive, constantly available, requires
only a pencil or pen to complete
Green LA. The weekly return card as a practical instrument for data collection in office-
based research: A report from ASPN. Family Medicine 1988(20)3;182-184.
8. Advantages of Weekly Return
Card Method
• Involves clinicians in the research process with
only a modest time requirement
• Always available -- improves adherence to
consecutive sample, reduces selection bias
• Produces useful, publishable pilot data
• Rapid turnaround and feedback to practices
• An established method: More than 40 ‘card
studies’ conducted in the ASPN PBRN alone;
nearly all published
9. Disadvantages of Weekly
Return Card Method
• Descriptive, cross-sectional studies
• Small number of variables
• Limited depth of information
• Missing data can be difficult to obtain
• Requires limited time by clinicians
• IRBs have made card studies more challenging
to conduct
– IRB application process
– CREC certification for clinicians
– Informed consent/HIPAA authorization by patients
10. SNPSA Study
• For safety net patients with type II diabetes in
Cleveland, clinicians wanted to know:
– Demographics
– Health insurance & prescription drug coverage
– Level of glycemic control
– Adherence to low-dose aspirin
– Frequency of self-management goal setting
– Patients' reported barriers to controlling type II
diabetes
– Clinicians' impressions of patients’ barriers
11. Data Collection Card
SNPSA
Safet y Net Providers’ St rategic Alliance
STUDY OF TYPE II DIABETES IN SAFETY NET PATIENTS
Clinician: _ ____ ___ ____ ____ ___ ____ __ __ Practice: __ ____ ____ ____ ___ ____ ___ __ __
Race
(check all that apply) Insurance GlycoHgb
If no aspirin, why not? What self-management
goal was set at this visit?
Date
Patient
Age
White
Black/AfricanAmerican
Asian
NativeHawaiian/Other
PacificIslander
AmericanIndian/Alaska
Native
HispanicorLatino(Y/N)
Homeless(Y/N)
Medicare
Medicaid
Commercial
Uninsured
Rxcoverage(Y/N)
Mostrecentvalue
Month
Day
Prev.Rxforlow-doseaspirin?
(Y/N)
CurrentlytakingASA?(Y/N)
Contraindicated
Other(specify)
Nogoalset
Parkcaratendoflot
Walkduringcommercials
Usestairsvs.elevator
Workingarden__x/week
Walk/swim/bike__x/week
Other:___________________
1
2
3
4
5
6
7
8
9
1
0
Pleasecompleteoppositesideofcard.
Instructions: Record data for 10 consecutive patients with Type II diabetes. Please record additional information on the opposite side for each of the 10
visits.
12. Reverse Side of Card
Patient* Please ask each patient the following questions, and record the information below. Also record your impression of the patient’s barriers.
a) “What makes it difficult for you to stay on top of your diabetes?” b) “What helps you stay on top of your diabetes?”
a) Difficulties (patient’s words):
b) What helps (patient’s words):1
Clinician’s impression of barriers:
a) Difficulties (patient’s words):
b) What helps (patient’s words):2
Clinician’s impression of barriers:
a) Difficulties (patient’s words):
b) What helps (patient’s words):3
Clinician’s impression of barriers:
a) Difficulties (patient’s words):
b) What helps (patient’s words):4
Clinician’s impression of barriers:
a) Difficulties (patient’s words):
b) What helps (patient’s words):5
Clinician’s impression of barriers:
a) Difficulties (patient’s words):
b) What helps (patient’s words):
6
Clinician’s impression of barriers:
a) Difficulties (patient’s words):
b) What helps (patient’s words):
7
Clinician’s impression of barriers:
a) Difficulties (patient’s words):
b) What helps (patient’s words):
8
Clinician’s impression of barriers:
a) Difficulties (patient’s words):
b) What helps (patient’s words):9
Clinician’s impression of barriers:
a) Difficulties (patient’s words):
b) What helps (patient’s words):10
Clinician’s impression of barriers:
*Patient numbers must correspond to patient numbers on opposite side of card.
13. Findings
• 19 clinicians collected data from 181 patient visits
in a 14 day period
• Patient-perceived barriers: adherence (40%),
financial/insurance (23%), psychosocial (13%)
• Clinician-perceived barriers: financial/insurance
(32%), cultural/psychosocial (29%), adherence
(29%)
• Patients’ helpful factors: dietary/medical
adherence (37%), social support (17%)
• Patients were less likely than clinicians to identify
systemic and contextual factors contributing to
poor diabetes care
Reichsman et. al. Opportunities for Improved Diabetes Care Among Patients of
Safety Net Practices: A Safety Net Providers' Strategic Alliance (SNPSA) Study.
J Nat Med Assn, 2008, under review.
15. Effect of parental expectations on
treatment of children with cough
• Previous study showed that half of children
diagnosed with bronchitis did not have
sputum production or rales
• Led to speculation that diagnostic label may
sometimes follow the decision to treat
• Hypothesized that parental expectations may
affect diagnosis
• Developed a study to assess the extent to
which physicians incorporate parental
expectations into medical decision making
Vinson D, Lutz LJ. The effect of parental expectations on treatment of children with a
cough: A report from ASPN. J Fam Pract 1993; 37:23-7.
16. Methods
• Card Study method was used
• Age (newborns to 14), sex, duration of illness,
history of fever, sputum, smoker in household,
enrollment in daycare, allergies, rales, wheezes,
chest radiograph, follow-up plans (none -
hospitalized)
• “Indicate whether you sense an expectation by
the patient’s parent or guardian to prescribe an
antibiotic.”
Vinson D, Lutz LJ. The effect of parental expectations on treatment of children with a
cough: A report from ASPN. J Fam Pract 1993; 37:23-7.
17. Results
• 1398 patients entered into study by
clinicians in 44 practices
• Most were not seriously ill; 63% were not
scheduled for a follow-up visit
• Diagnosis of viral URTI in 35%; bronchitis
in 33%, OM in 27%, asthma in 9%
• Physician sensed parental expectations
for antibiotics in 15.4% of cases
Vinson D, Lutz LJ. The effect of parental expectations on treatment of children with a
cough: A report from ASPN. J Fam Pract 1993; 37:23-7.
18. Key Findings
• Parental expectation was second only to rales
in strength of association with diagnosis of
bronchitis
• When controlling for other variables, parental
expectations were more strongly correlated
with diagnosis of bronchitis than either fever
or sputum production
• If physician perceived that parent expected
an antibiotic prescription, the likelihood that
diagnosis of bronchitis would be made
doubled
Vinson D, Lutz LJ. The effect of parental expectations on treatment of children with a
cough: A report from ASPN. J Fam Pract 1993; 37:23-7.
19. Electronic Data Collection
• Most PBRNs use some form of electronic data
collection methods
• Primary: web forms, tablet PCs, PDAs
• Secondary: EHR, capture of billing data
• Paper-based methods still prevail -- simple and
reliable
• Computer technologies are increasingly more
reliable and cost-effective
20. Benefits of Electronic Data
Collection
• Rapid distribution of data collection forms
• Automated patient identification, patient
registries
• Eliminates paper shuffle on both ends:
opening, sorting, completing, checking,
copying, folding, labeling, mailing, etc.
• Rapid and secure transfer of collected data
21. Benefits (cont.)
• Eliminates need for manual data entry
• Can result in improved data quality
• Enables rapid feedback for clinicians
• Can reduce time from study launch to
publication
22. Electronic Data Collection
Tools
• PC-based web-form data entry
– Eliminates need for separate data entry step
– Simple implementation
– Inexpensive
– Low portability
– Well-suited for physician surveys and patient
surveys from home
– Not suited for POC applications unless exam room
terminals
25. Electronic Data Collection
Tools
• Handheld/Tablet Computer data entry
– Eliminates need for separate data entry step
– Portability for collection at point of care
– Broad range of POC applications
– More complex implementation than paper &
pencil
– More expensive than paper & pencil
26. Patient Reactions to Tablet PCs
• Research assistants required 2-4 minutes to train
patients to complete a survey
• Survey required free-text entry, so a voice recording
option offered
• More than 70% indicated that tablet was easy to use
• 30% reported difficulty, almost entirely with the voice
recording technology
• Elderly patients had the most difficulty
• 2.5% elected to change to paper & pencil
• Other studies show up to 96% patient satisfaction with
touch screen computers for survey completion
Main et al., Exploring patient reactions to pen-tablet computers: A report
from CareNet. Annals of Family Medicine 2004(2)5: 421-424.
27. Internet
- Practice -
University-based
Research Office
Tablet PCs
Data
Instrum
ents
Instrum
ents
Secure Transmission &
Data Storage
Secure
FTP
Secure
FTP
HIPAA-Compliant
Server
Data
E-mail
Research Nurses
28. Challenges in using
Computer Technology
• Capital investment in point of care systems
– Software, hardware
– IT staff
– Trainers
• Integration with EHRs can be complex
• Clinician’s time for training
• Troubleshooting
• Assessing technologies as they rapidly evolve
29. Suggestions
• Carefully estimate the time needed for training &
troubleshooting
• Offer paper-based or web-based back-up
for POC technologies
• Assess technology performance in terms of
implementation time, cost, troubleshooting,
burden on network
• Data security is essential
30. Next Week
• Sampling, measurement, and analysis of
nested data in PBRN research
Stephen Zyzanski, PhD
Professor, Case Department of Family Medicine,
Epidemiology & Biostatistics
31. Audio and Powerpoint
Presentations
Practice-based Research Networks
Seminar Series Podcast
Audio podcasts and the accompanying PowerPoint
slides of the Practice-based Research Networks Seminar
Series are available online at
http://blog.case.edu/jjw17/.
Listen and learn online.
To listen to the podcast in your Web browser, follow
the link to the .mp3 file for that week's entry. The file
will then play in QuickTime or your preferred audio
player.
To view the accompanying slides just follow the link to
the .pdf file to either view the slides on your computer
or to print them out.
I would like to share with you the findings of one card study that was conducted in ASPN, and later published in the Journal of Family Practice in 1993.
This study was inspired by a previous study that showed that half of children diagnosed with bronchitis did not have sputum production or rales. This led to speculation that diagnostic label of bronchitis may sometimes follow the decision to treat. PBRN investigators hypothesized that factors affecting the decision to treat, such as parental expectations, may affect the diagnosis. They then designed a simple study to assess the extent to which physicians incorporate parental expectations into their medical decision making.
The methods employed standard data collection card-style, similar to the card that I have shown. Children aged from newborn to 14 with a chief complaint of cough lasting up to one month were included, and data elements included the patient’s unique identifier, sex, and duration of illness. Dichotomous variables included history of fever, sputum production, smoker in household, enrollment in daycare, allergies, rales, wheezes, chest radiograph. Physicians recorded diagnosis and treatment by checking one or more boxes on the form, and indicated follow-up plans which ranged from none to hospitalized.
The clinicians were also instructed to “indicate on the data card whether you sense an expectation by the patient’s parent or guardian to prescribe an antibiotic.” Participating clinicians were not informed about the study hypotheses.
1398 patients entered into study by clinicians in 44 practices. Most of the children were not seriously ill and 63% were not scheduled for a follow-up visit. A diagnosis of viral URTI was made in 35% of the cases, bronchitis in 33% of cases, otitis media in 27%, and asthma in 9%. Physician indicated that they sensed parental expectations for antibiotics in 15.4% of cases.
The key findings are the following:
If a physician perceived that parent expected an antibiotic prescription, the likelihood that diagnosis of bronchitis would be made doubled.
Only rales was more strongly associated with diagnosis of bronchitis than parental expectation for antibiotics.
When controlling for other variables, parental expectation were more strongly correlated with diagnosis than either fever or sputum production.
Interestingly, parental expectations for antibiotic were not associated with diagnosis of otitis media, sinusitis, or pneumonia.
Thus, there appears to be strong association between parental expectations and the diagnosis of bronchitis.
So that brings us up to where many PBRNs are to date. Another electronic frontier for us is using computers for electronic data collection and management.
Acute Otitis media study
Technologies now more useful, more reliable, more friendly, more cost-effective
Let’s take a look at the benefits of using electronic tools for data collection and management
With computers, you can distribute data collection forms very rapidly by using the internet as a conduit
Essentially eliminates paper forms being filled out by doctors or MAs
Eliminates the paper shuffle on both endsStudy management reports can be automatically generated either at the practice or at the PBRN central office and then e-mailed to the practice. This is especially important for complex studies in which a large number of patients are enrolled over a long period of time.
(continued)
Fast and reliable transfer of collected data; more frequent transfers
Improved data quality
A large part of finishing any study is the process of improving data quality – pursuing missing pieces of data, and clarifying the meaning of out of range values. By specifying required fields, the problem of missing data can be largely eliminated. Data ranges can also be set on the data collection device, greatly reducing the occurrence of out of range values.
Rapid feedback for clinicians
Less time from study roll-out to publication
Iterative studies can be implemented more easily
Really not a dichotomy, but a continuum from PDAs through networked thin client devices to PCs.
