Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Olola ph ddefenseuou10052009_final3



PhD defense presentation

PhD defense presentation



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Olola ph ddefenseuou10052009_final3 Olola ph ddefenseuou10052009_final3 Presentation Transcript

  • Enhancing Continuity of Care Using Emergency Medical Card and Continuity of Care Report
    Christopher H O Olola MSc
    Department of Biomedical Informatics
    University of Utah
    R Scott Evans PhD (Chair)
    Mollie PoyntonAPRN,PhD
    Scott Narus PhD
    Jonathan Nebeker MD
    Joseph Hales PhD
  • Presentation Outline
    Project implementation
    Project evaluation
  • Presentation Outline
    Project implementation
    Project evaluation
  • Current Healthcare Systems
    Healthcare systems are complex and fragmented1,2
    Partly due to slow EMR implementation and adoption3
    USA: 17%-20% Outpatient clinics have comprehensive EMRs
    Results in ineffective patient care
    Patients see multiple healthcare providers (HCPs)
    Mostly because of changes in insurance plans, care quality, care costs, loss of trust etc
    Potentially results in discontinuity of care
    ≈ 44,000 – 98,000 Americans die annually
    These deaths occur because of (mostly avoidable) medicalerrors and ADEs as a result of poor quality information4,5
    1Geissbuhler et al. 2004, 2Bates et al. 2003, 3HIMSS 2006, 4Kohn et al. 2000, 5Moore et al. 2003
  • Continuity of Care (CoC)
    What is Continuity of Care?
    A process by which the patient and the physician are cooperatively involved in ongoing health care management toward the goal of high quality, cost-effective medical care.
    American Academy of Family Physicians – AAFP (2003)
  • Types of Continuity of Care*
    Interpersonal or provider continuity
    Where a patient has a single HCP with whom s/he has developed trust, respect and understanding
    Longitudinal or site continuity
    Where a patient has groups of preferred HCPs that s/he sees for medical care i.e., Team-based or “medical home”
    Informational or referral continuity
    Where there exists organized and shareable patient core information among HCPs
    This is the focus of my project…
    *Freeman et al. 2003, Roos et al. 1980, Saultz et al. 2003 HCP: Healthcare Provider
  • Techniques for promotion of CoC*
    Use of a single primary HCP
    Directly observed treatment (DOT)
    Bar codes
    Radio frequency identification (RFID)
    Regional Health Information Organizations (RHIOs)
    Pocket-sized smart card
    Emergency medical cards (EMCs)
    Continuity of care record (CCR)
    This is the focus of my project…
    *Fry et al. 2005, Haung et al. 2005, Orlova et al. 2005, Auber 2001
  • Presentation Outline
    Project implementation
    Project evaluation
  • Institutional review boards (IRBs)
    All the studies in this project were approved by the University of Utah IRBand the Intermountain Healthcare IRB
  • Problems/Significance of Project
    Patients see multiple HCPsoutside of Intermountain Healthcare network
    Dearth of immediately available patient healthcare information at the point of care
    Lack of standardized patient care information to share across health systems
    Poor quality (accuracy, completeness) patient data
    Healthcare inefficiencies
    In most instances CoCis rarely considered*
    *Mills et al. 2006, Post et al. 2005 HCPs: Healthcare Providers CoC: Continuity of Care
  • Project Objectives
    Developan automatedapplication (“CCR application”) compliant with the CCR standard (E2365-05)*
    Use simulation to assess the use andusefulness of the EMC and CoC report in enhancing CoC
    Evaluate the impact of patient-entered data on the quality (accuracy and completeness) of HCP-entered data in the EMR
    Assess patient satisfaction with usefulness of the EMC and CoC report in enhancing CoC
    *ASTM 2006 CoC: Continuity of Care CCR: Continuity of Care Record
  • Continuity of Care Record (CCR) Standard
    Developers: ASTM & other organizations
    XML-based standard
    Outgrowth of Massachusetts DPH
    patient care referral form (PCRF)
    Core datasetof the most relevantand timely facts on patient’s health care
    Originally designed to be prepared by a practitioner at conclusion of encounter
    To enable next practitionerto readily access pertinent patient information
    CCR may be prepared, displayed and transmitted on paper or electronically
    CCR is notEHR, Progress note, Discharge summary
    XML: eXtensible Mark-up Language DPH: Department of Public Health
    ASTM 2006
  • Presentation Outline
    Project implementation
    Project evaluation
  • Implementation of Emergency Medical Card and Continuity of Care Report for Continuity of Care
    Olola CHO, Rowan B, Narus S, Smith M, Hastings T, Poynton M, Nebeker J, Hales J, Evans RS
    Methods of Information in Medicine
    (Accepted for publication: May 2009)
  • Study Objectives
    To describe the procedures used to design, develop, and implement the CCR application, EMC and CoC report using the CCR standard
    To outline the evaluation studies planned and the majorlessons learned
    EMC: Emergency Medical Card CoC: Continuity of Care CCR: Continuity of Care Record
  • Methods
    Setting and Users
    Develop and integrate the CCR application in the MyHealthpatient portal at Intermountain Healthcare
    The application is used by approximately 30,000 patientsenrolled at 25 outpatient clinicsthat offer My Health web services
    The sites have over 120Internal Medicine, Family Practice, OB/Gyn, diabetes and Endocrinology clinicians
    The project took 2.5 years (December 2006 – October 2008) to implement, evaluation studies 9 months
  • Methods
    The Data Management Architecture
    Phase 1 (CCR application implementation)
    1a/b: Data entry and display
    2: EMC data transformed into XML & stored
    3: CCR XML data are extracted from CDR
    4: pdf of EMC/CoC report generated
    5a/b: EMCs /CoC reports printed or saved
    6: EMCs/CoC reports used with primary HCPs
    7: EMR updated with patient-entered data
    Phase 2 (for future implementation)
    step 8: HCPs extract patient-entered data
    step 9: HCPs manages data discrepancies
    step 10: HCPs verifies data and updates EMR
  • Methods
    The Main Data Management Screen
    CCR application access link
  • The Emergency Medical Card
    8-faced foldable card
    Designed using ISO 7810 ID-1 standard (ATM card size)
    Contains current patient-entered data & EMR data
    (not differentiated)
    Used mainly during medical emergency
    Additional Info
    Insurance & financial
    Emergency contacts
    Current health concerns/problems
    Current medications
    Current health concerns/problems
    Current medications
  • The Continuity of Care Report
    Contains patient-entered data and HCP-entered data
    • All active problems, allergies, medications etc are included
    • 3 months back (plus inactive) problems, meds, labs etc included
    Used mainly during non-emergency clinic visits or taken to patient primary provider to update missing or erroneous data in EMR
  • Results and Discussions
    An application complaint with the CCR standard requirements was designed, developed and integrated with Intermountain’s electronic PHR, My Health
    Patients use online credentials to access their PHRsto
    View, add or modify their PHRs
    Create and print paper-based EMC & CoCreport using both patient-entered & HCP-entered EMR data
    Monitor records, identify possible errors and (using the CoC report) communicate to HCPs for review and EMR update
    Two evaluation studies were designed to assess & report on the application using simulation, reviews and comparisons of EMC/CoC report and EMR data, and patient-satisfaction surveys
  • Lessons learned & study limitations
    EMR data update done only by primary HCPs at Intermountain is limiting – delays EMR updates & information availability at point of care
    Patient proxy are needed e.g., if patient is incapacitated
    Currently, HCPs have no direct access to patient-entered data in the CCR application database
    Keeping the EMC data current even if by using the unverified patient-entered data is vital
    PHC: Personal Health Console
  • Conclusions
    Demonstrated that it is possible to use CCR standard to implement an application that enables patients, not only to view their PHRs but to add or modify records, & to create and print EMCs and CoC reports
    EMCs/CoC reports can be created using the HCP-verified EMR data & not by using patient-entered data only as is currently prevalent in healthcare systems
    Functionalities that enable patients to monitor their records,identify possible errors & communicate to HCPs for prompt EMR updates, are crucial
  • Presentation Outline
    Project implementation
    Project evaluation
  • Use of Simulation to Evaluate the Use and Usefulness of the Emergency Medical Card and Continuity of Care Report in Enhancing Continuity of Care
    Olola CHO, Narus S, Nebeker J, Poynton M, Hales J, Rowan B,
    LeSieur H, Zumbrennen H, Edwards AA, Crawford R, Amundsen S, Kabir Y, Atkin J, Newberry C, Young J, Hanifi T, Risenmay B, SorensenT, Evans RS
    Methods of Information in Medicine
    (Submitted: September 2009)
  • Study Objectives
    To use simulation to evaluate the use of the EMC and the CoC report in enhancing CoC
    To assess the usefulness of the EMC and the CoCreport in enhancing CoC
    EMC: Emergency Medical Card CoC: Continuity of Care
  • Methods
    Setting and participants (“Reviewers”)
    3 Medical Doctors (MDs) & 2 Physician Assistants (PAs) from outpatient clinics at Intermountain Healthcare
    Clinic Managers at the Intermountain clinics made contacts
    7 Fourth-year medical students from the University of Utah School of Medicine
    PI made contacts
  • methods
    2 Create
    1 Random selection
    De-identified study cases
    Cases of patients who entered new data using CCR application
    4 Review
    3 Review
    CoC report
    5 Complete an online survey (with Likert Scale responses)
    • Legibility, easy to use & understand
    • Encounter time, overall HCP knowledge
    • Medical decision making
  • Results and Discussions
    Gender, job designation, specialty and experience had no significant influence in the way the reviewers evaluated the usefulness of the EMC and CoC report
    Ratings significantly varied by reviewer’s age
    RD(95%CI): Risk Difference (95% Confidence Interval) ) #Tool rated as not useful
  • Results and discussions
    The reviewers provided useful comments on how the EMC/CoC report could be improved to avail adequate and appropriate information for effective medical decision making at the point of care
    Reviewers preferredcondensed/summarized and abbreviated information
  • Results and discussions
    Inter-rater agreement
    Overall moderate reviewer agreement (50%-60%) on rating of EMC & CoC report
    Higher agreements observed in individual measures, especially for CoC report
    *Kappa statistics (κ)
    < 0 Poor or None 0.00—0.20 Slight 0.21—0.40 Fair 0.41—0.60 Moderate
    0.61—0.80 Substantial 0.81—1.00 Almost perfect.
  • Results and discussions
    *Discharge home, admission, referral or transfer to another HCP
    Overall, EMC and CoC were found to be highly useful (81.2% vs. 84.5%, resp.)
    100% usefulness in shortening encounter time & in increasing HCPs’ knowledge
    Both were rated highly for legibility and ease of understanding (88.7% vs. 90.3%)
  • Study limitations
    Few MDs and PAs participated (no NPs)
    Decliners were busy, in other projects or not interested
    Non-Intermountain HCPs were excluded
    Policies prohibited analysis of data outside of Intermountain network (non-covered entities)
    Marketing/recruitment policy restrictions
    Intermountain policies permitted only clinical managers at the clinics offering My Health services to recruit study reviewers –direct contact PI may be could have improved recruitment rates
  • Conclusions
    The EMC and CoC report are useful vehicles for transporting patient healthcare information across the healthcare continuum and they can substantially enhance CoC. This was specifically demonstrated in
    Shortening patient-HCP encounter time
    Increasing the HCP overall knowledge
    Decision on prescriptions and on ordering new or repeating laboratory tests
    The reviewers’ perception of the usefulness of the EMC and CoC report in enhancing CoC was associated with age – further (larger) validation studies are needed
  • Assessing Patient Satisfaction with the Continuity of Care and Effect of Patient-Entered Data on the Quality of Healthcare Provider-maintained EMR Data
    Olola CHO, Poynton M, Hales J, Narus S, Nebeker J, Rowan B, Smith M, Evans RS
    The International Journal for Quality in Health Care
    (Prepared for submission)
  • Study Objectives
    To evaluate patient satisfaction with the usefulness of the emergency medical card (EMC) and CoC report in enhancing CoC
    Compared patient-entered data in the CoC report with the HCP-entered data in the EMR data to assess EMR data quality (i.e., accuracy and completeness)
  • methods
    Patient satisfaction survey
    Promotional emails, meetings, “Teaser”, fliers, posters
    Users of the CCR application to create EMC/CoC report
    Excluded – never used EMC/CoC report
    Complete online survey (with Likert Scale)
    (n=101; 76%)
    • Legibility, easy to use & understand
    • Encounter time, overall patient’s knowledge
    • Correct errors & complete missing data in the EMR
    • Overall quality of care
  • methods
    Evaluation of accuracy of HCP-entered data in EMR
    Instances of patient-entered data values
    Excluded – had no corresponding data fields in the EMR
    Study sample
    (n=1,505, 75.5%)
    Compare records using patient ID, name, sex, DoB, date of entry (before date of EMR update)
    Repeating records
    (e.g., labs, vitals)
    Non-repeating records (e.g., address)
    Compare records using patient ID & date of entry (before date of EMR update)
    EMR data
    Summary of instances of accurate data values
  • methods
    Evaluation of completeness of HCP-entered data in EMR
    Instances of all data values (8 months before use of CCR application)
    Instances of all data values (8 months after use of CCR application)
    Compared only data fields that existed before and after use of CCR application
    Instances of data values with non-blank/ missing values
    Instances of data values with non-blank/ missing values
    Summary of instances of complete data values
  • Results and Discussions
    Cumulative CCR application’s page views & actual use
    Promotional techniques significantly increased number of the application’s page views (n=22,024) and actual use (n=133)
    Strong correlation between page views and actual use (γ = 0.994, p=0.0005)
    Email messages to HCPs was best, then “Teaser” in My Health & lastly Fliers to HCPs
  • Results and Discussions
    Profile of responses
    *Data was collected in an Exit survey after patients created the documents
    Documents’ usefulness rating was significantly associated with patient’s age
    Majority created EMC for use in emergencies (56%) or for personal use (26%)
    Only about 15% of patients created CoC report for EMR updates
  • Results and Discussions
    Patients’ rating of the usefulness of EMC & CoC report
    Overall, 63.1% of the patients found EMC and CoC report to be useful
    More patients for CoCthan EMC (64% vs. 63%) , but no significant difference
    Agreement on the usefulness of EMC and CoC report was demonstratedineach CoC measure , but no significant difference in each measure’s ratings
    Patients rated documents highly for usefulness in shortening encounter time
  • Results and Discussions
    Patients-entered data vs. accuracy of EMR data
    69% (70/101) of the patients entered new data values (n=1,994; 1,505 compared)
    44% of newdata was used for EMRupdate, but no significant difference (p=0.109)
    The majority of the EMRupdates were for address, biodata, insurance, Primary HCP, labs
    Significantly lowEMRupdates observed in Biodataand primary HCP data
    NoEMRupdates for allergies & problems data
    Q1/Q3: 25th & 75th Quartiles
  • Results and Discussions
    Patient-entered data vs. completeness of EMR data
    Inconsistency in associations shown between use of CCR application and data completeness
    Significant association between use of CCR application and improved data completeness was observed only in address and appointments data attributes
    Overall, 98.7% complete data before & 76.9% after (OR(95%CI: 0.780(0.779, 0.780), P<0.001)
    Some patient data were not extracted from EMR
    These were annotated by patients because the data was in free text format or not integrated in CDR
    Some sub-group analyses on EMR data quality did not yield results due to sample size issues
    E.g., Insurance, allergies and medical problems – but these may be because they accurately entered in EMR by HCPs
    Reason was not collected for data not updated in EMR
    Opportunity was missed of asking patients reasons why some entered data was not used to update EMR information
  • Conclusions
    Patients demonstrated that the EMC and CoC report were useful tools in enhancing CoC
    Patients also showed that the two documents were useful to identify missing patient information especially in the demographic and appointment categories than other categories
    The patient is an important source of quality control for their record in HCP-maintained EMR
  • Presentation Outline
    Project implementation
    Project evaluation
  • Research Conclusions
    Contribution to the field of Biomedical Informatics
    Showed that the CCR standard can be used to ensure that patientinformationthat is shared among HCPs is standardized
    Demonstrated that HCP-verified EMR data can be used to createEMCs (and CoC reports), in addition to the prevalent use of patient-entered data
    Showed that patientsare able to access their PHRs, monitor and control the quality of their information in the HCP-entered data in EMR
    Showed that timelyand accuratepatientinformationcan be availed to foster enhanced efficiency and effectiveness in the EMRupdates and medicaldecisionmaking by HCPs
  • Research Conclusions
    Lessons Learned and Future Direction
    Free-text, non-integrated data sources and lack of full interoperability of EMRS still pose data extraction difficulties
    Patient is an important source of quality control for their records in HCP-entered data in the EMR
    The EMC and CoC report should be kept up to date since both HCPs and patients have shown the usefulness of the two documents in enhancing CoC – more patients printed EMC
    Use of a variety of promotional techniques are useful for improved (wide scope) marketing and recruitment
    Further validation studies are recommended
  • acknowledgments
    Institutional Review Boards (IRBs)
    University of Utah IRB
    Intermountain Healthcare IRB
    NIH/NLM/Michigan State University
    Julia Royall
    Terrie Taylor
    Reed Gardner, Joyce Mitchell, Lynn Ford, John Hurdle
    Matt Samore, Adi, Lisa-Canon Albright
    Kathy Stoker, Linda Galbreath, JoAnn Thompson
    Faculty, staff and fellow students
    ITS/EDW team
    Jim Livingston, Cheri Hunter, Ming Tu and Vick Deshmukh
    National academy of emergency dispatch
    Jeff Clawson, Alan Fletcher, Pam Stewart
    Intermountain healthcare
    PHC team:
    Belle Rowan (Chair), Matt Smith, Traci Hastings, Carol Askew, Chris Nuccitelli
    Len Bowes, Stan Huff
    ISSA, EDW, SelectHealth…
    UoU/DBMI committee
    R Scott Evans (Chair)
    Mollie Poynton
    Scott Narus
    Jonathan Nebeker
    Joe Hales
    My family
    Mom, Dad, and my Wife and Children