• Like
  • Save


Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Olola ph ddefenseuou10052009_final3



PhD defense presentation

PhD defense presentation



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Olola ph ddefenseuou10052009_final3 Olola ph ddefenseuou10052009_final3 Presentation Transcript

    • Enhancing Continuity of Care Using Emergency Medical Card and Continuity of Care Report
      Christopher H O Olola MSc
      Department of Biomedical Informatics
      University of Utah
      R Scott Evans PhD (Chair)
      Mollie PoyntonAPRN,PhD
      Scott Narus PhD
      Jonathan Nebeker MD
      Joseph Hales PhD
    • Presentation Outline
      Project implementation
      Project evaluation
    • Presentation Outline
      Project implementation
      Project evaluation
    • Current Healthcare Systems
      Healthcare systems are complex and fragmented1,2
      Partly due to slow EMR implementation and adoption3
      USA: 17%-20% Outpatient clinics have comprehensive EMRs
      Results in ineffective patient care
      Patients see multiple healthcare providers (HCPs)
      Mostly because of changes in insurance plans, care quality, care costs, loss of trust etc
      Potentially results in discontinuity of care
      ≈ 44,000 – 98,000 Americans die annually
      These deaths occur because of (mostly avoidable) medicalerrors and ADEs as a result of poor quality information4,5
      1Geissbuhler et al. 2004, 2Bates et al. 2003, 3HIMSS 2006, 4Kohn et al. 2000, 5Moore et al. 2003
    • Continuity of Care (CoC)
      What is Continuity of Care?
      A process by which the patient and the physician are cooperatively involved in ongoing health care management toward the goal of high quality, cost-effective medical care.
      American Academy of Family Physicians – AAFP (2003)
    • Types of Continuity of Care*
      Interpersonal or provider continuity
      Where a patient has a single HCP with whom s/he has developed trust, respect and understanding
      Longitudinal or site continuity
      Where a patient has groups of preferred HCPs that s/he sees for medical care i.e., Team-based or “medical home”
      Informational or referral continuity
      Where there exists organized and shareable patient core information among HCPs
      This is the focus of my project…
      *Freeman et al. 2003, Roos et al. 1980, Saultz et al. 2003 HCP: Healthcare Provider
    • Techniques for promotion of CoC*
      Use of a single primary HCP
      Directly observed treatment (DOT)
      Bar codes
      Radio frequency identification (RFID)
      Regional Health Information Organizations (RHIOs)
      Pocket-sized smart card
      Emergency medical cards (EMCs)
      Continuity of care record (CCR)
      This is the focus of my project…
      *Fry et al. 2005, Haung et al. 2005, Orlova et al. 2005, Auber 2001
    • Presentation Outline
      Project implementation
      Project evaluation
    • Institutional review boards (IRBs)
      All the studies in this project were approved by the University of Utah IRBand the Intermountain Healthcare IRB
    • Problems/Significance of Project
      Patients see multiple HCPsoutside of Intermountain Healthcare network
      Dearth of immediately available patient healthcare information at the point of care
      Lack of standardized patient care information to share across health systems
      Poor quality (accuracy, completeness) patient data
      Healthcare inefficiencies
      In most instances CoCis rarely considered*
      *Mills et al. 2006, Post et al. 2005 HCPs: Healthcare Providers CoC: Continuity of Care
    • Project Objectives
      Developan automatedapplication (“CCR application”) compliant with the CCR standard (E2365-05)*
      Use simulation to assess the use andusefulness of the EMC and CoC report in enhancing CoC
      Evaluate the impact of patient-entered data on the quality (accuracy and completeness) of HCP-entered data in the EMR
      Assess patient satisfaction with usefulness of the EMC and CoC report in enhancing CoC
      *ASTM 2006 CoC: Continuity of Care CCR: Continuity of Care Record
    • Continuity of Care Record (CCR) Standard
      Developers: ASTM & other organizations
      XML-based standard
      Outgrowth of Massachusetts DPH
      patient care referral form (PCRF)
      Core datasetof the most relevantand timely facts on patient’s health care
      Originally designed to be prepared by a practitioner at conclusion of encounter
      To enable next practitionerto readily access pertinent patient information
      CCR may be prepared, displayed and transmitted on paper or electronically
      CCR is notEHR, Progress note, Discharge summary
      XML: eXtensible Mark-up Language DPH: Department of Public Health
      ASTM 2006
    • Presentation Outline
      Project implementation
      Project evaluation
    • Implementation of Emergency Medical Card and Continuity of Care Report for Continuity of Care
      Olola CHO, Rowan B, Narus S, Smith M, Hastings T, Poynton M, Nebeker J, Hales J, Evans RS
      Methods of Information in Medicine
      (Accepted for publication: May 2009)
    • Study Objectives
      To describe the procedures used to design, develop, and implement the CCR application, EMC and CoC report using the CCR standard
      To outline the evaluation studies planned and the majorlessons learned
      EMC: Emergency Medical Card CoC: Continuity of Care CCR: Continuity of Care Record
    • Methods
      Setting and Users
      Develop and integrate the CCR application in the MyHealthpatient portal at Intermountain Healthcare
      The application is used by approximately 30,000 patientsenrolled at 25 outpatient clinicsthat offer My Health web services
      The sites have over 120Internal Medicine, Family Practice, OB/Gyn, diabetes and Endocrinology clinicians
      The project took 2.5 years (December 2006 – October 2008) to implement, evaluation studies 9 months
    • Methods
      The Data Management Architecture
      Phase 1 (CCR application implementation)
      1a/b: Data entry and display
      2: EMC data transformed into XML & stored
      3: CCR XML data are extracted from CDR
      4: pdf of EMC/CoC report generated
      5a/b: EMCs /CoC reports printed or saved
      6: EMCs/CoC reports used with primary HCPs
      7: EMR updated with patient-entered data
      Phase 2 (for future implementation)
      step 8: HCPs extract patient-entered data
      step 9: HCPs manages data discrepancies
      step 10: HCPs verifies data and updates EMR
    • Methods
      The Main Data Management Screen
      CCR application access link
    • The Emergency Medical Card
      8-faced foldable card
      Designed using ISO 7810 ID-1 standard (ATM card size)
      Contains current patient-entered data & EMR data
      (not differentiated)
      Used mainly during medical emergency
      Additional Info
      Insurance & financial
      Emergency contacts
      Current health concerns/problems
      Current medications
      Current health concerns/problems
      Current medications
    • The Continuity of Care Report
      Contains patient-entered data and HCP-entered data
      • All active problems, allergies, medications etc are included
      • 3 months back (plus inactive) problems, meds, labs etc included
      Used mainly during non-emergency clinic visits or taken to patient primary provider to update missing or erroneous data in EMR
    • Results and Discussions
      An application complaint with the CCR standard requirements was designed, developed and integrated with Intermountain’s electronic PHR, My Health
      Patients use online credentials to access their PHRsto
      View, add or modify their PHRs
      Create and print paper-based EMC & CoCreport using both patient-entered & HCP-entered EMR data
      Monitor records, identify possible errors and (using the CoC report) communicate to HCPs for review and EMR update
      Two evaluation studies were designed to assess & report on the application using simulation, reviews and comparisons of EMC/CoC report and EMR data, and patient-satisfaction surveys
    • Lessons learned & study limitations
      EMR data update done only by primary HCPs at Intermountain is limiting – delays EMR updates & information availability at point of care
      Patient proxy are needed e.g., if patient is incapacitated
      Currently, HCPs have no direct access to patient-entered data in the CCR application database
      Keeping the EMC data current even if by using the unverified patient-entered data is vital
      PHC: Personal Health Console
    • Conclusions
      Demonstrated that it is possible to use CCR standard to implement an application that enables patients, not only to view their PHRs but to add or modify records, & to create and print EMCs and CoC reports
      EMCs/CoC reports can be created using the HCP-verified EMR data & not by using patient-entered data only as is currently prevalent in healthcare systems
      Functionalities that enable patients to monitor their records,identify possible errors & communicate to HCPs for prompt EMR updates, are crucial
    • Presentation Outline
      Project implementation
      Project evaluation
    • Use of Simulation to Evaluate the Use and Usefulness of the Emergency Medical Card and Continuity of Care Report in Enhancing Continuity of Care
      Olola CHO, Narus S, Nebeker J, Poynton M, Hales J, Rowan B,
      LeSieur H, Zumbrennen H, Edwards AA, Crawford R, Amundsen S, Kabir Y, Atkin J, Newberry C, Young J, Hanifi T, Risenmay B, SorensenT, Evans RS
      Methods of Information in Medicine
      (Submitted: September 2009)
    • Study Objectives
      To use simulation to evaluate the use of the EMC and the CoC report in enhancing CoC
      To assess the usefulness of the EMC and the CoCreport in enhancing CoC
      EMC: Emergency Medical Card CoC: Continuity of Care
    • Methods
      Setting and participants (“Reviewers”)
      3 Medical Doctors (MDs) & 2 Physician Assistants (PAs) from outpatient clinics at Intermountain Healthcare
      Clinic Managers at the Intermountain clinics made contacts
      7 Fourth-year medical students from the University of Utah School of Medicine
      PI made contacts
    • methods
      2 Create
      1 Random selection
      De-identified study cases
      Cases of patients who entered new data using CCR application
      4 Review
      3 Review
      CoC report
      5 Complete an online survey (with Likert Scale responses)
      • Legibility, easy to use & understand
      • Encounter time, overall HCP knowledge
      • Medical decision making
    • Results and Discussions
      Gender, job designation, specialty and experience had no significant influence in the way the reviewers evaluated the usefulness of the EMC and CoC report
      Ratings significantly varied by reviewer’s age
      RD(95%CI): Risk Difference (95% Confidence Interval) ) #Tool rated as not useful
    • Results and discussions
      The reviewers provided useful comments on how the EMC/CoC report could be improved to avail adequate and appropriate information for effective medical decision making at the point of care
      Reviewers preferredcondensed/summarized and abbreviated information
    • Results and discussions
      Inter-rater agreement
      Overall moderate reviewer agreement (50%-60%) on rating of EMC & CoC report
      Higher agreements observed in individual measures, especially for CoC report
      *Kappa statistics (κ)
      < 0 Poor or None 0.00—0.20 Slight 0.21—0.40 Fair 0.41—0.60 Moderate
      0.61—0.80 Substantial 0.81—1.00 Almost perfect.
    • Results and discussions
      *Discharge home, admission, referral or transfer to another HCP
      Overall, EMC and CoC were found to be highly useful (81.2% vs. 84.5%, resp.)
      100% usefulness in shortening encounter time & in increasing HCPs’ knowledge
      Both were rated highly for legibility and ease of understanding (88.7% vs. 90.3%)
    • Study limitations
      Few MDs and PAs participated (no NPs)
      Decliners were busy, in other projects or not interested
      Non-Intermountain HCPs were excluded
      Policies prohibited analysis of data outside of Intermountain network (non-covered entities)
      Marketing/recruitment policy restrictions
      Intermountain policies permitted only clinical managers at the clinics offering My Health services to recruit study reviewers –direct contact PI may be could have improved recruitment rates
    • Conclusions
      The EMC and CoC report are useful vehicles for transporting patient healthcare information across the healthcare continuum and they can substantially enhance CoC. This was specifically demonstrated in
      Shortening patient-HCP encounter time
      Increasing the HCP overall knowledge
      Decision on prescriptions and on ordering new or repeating laboratory tests
      The reviewers’ perception of the usefulness of the EMC and CoC report in enhancing CoC was associated with age – further (larger) validation studies are needed
    • Assessing Patient Satisfaction with the Continuity of Care and Effect of Patient-Entered Data on the Quality of Healthcare Provider-maintained EMR Data
      Olola CHO, Poynton M, Hales J, Narus S, Nebeker J, Rowan B, Smith M, Evans RS
      The International Journal for Quality in Health Care
      (Prepared for submission)
    • Study Objectives
      To evaluate patient satisfaction with the usefulness of the emergency medical card (EMC) and CoC report in enhancing CoC
      Compared patient-entered data in the CoC report with the HCP-entered data in the EMR data to assess EMR data quality (i.e., accuracy and completeness)
    • methods
      Patient satisfaction survey
      Promotional emails, meetings, “Teaser”, fliers, posters
      Users of the CCR application to create EMC/CoC report
      Excluded – never used EMC/CoC report
      Complete online survey (with Likert Scale)
      (n=101; 76%)
      • Legibility, easy to use & understand
      • Encounter time, overall patient’s knowledge
      • Correct errors & complete missing data in the EMR
      • Overall quality of care
    • methods
      Evaluation of accuracy of HCP-entered data in EMR
      Instances of patient-entered data values
      Excluded – had no corresponding data fields in the EMR
      Study sample
      (n=1,505, 75.5%)
      Compare records using patient ID, name, sex, DoB, date of entry (before date of EMR update)
      Repeating records
      (e.g., labs, vitals)
      Non-repeating records (e.g., address)
      Compare records using patient ID & date of entry (before date of EMR update)
      EMR data
      Summary of instances of accurate data values
    • methods
      Evaluation of completeness of HCP-entered data in EMR
      Instances of all data values (8 months before use of CCR application)
      Instances of all data values (8 months after use of CCR application)
      Compared only data fields that existed before and after use of CCR application
      Instances of data values with non-blank/ missing values
      Instances of data values with non-blank/ missing values
      Summary of instances of complete data values
    • Results and Discussions
      Cumulative CCR application’s page views & actual use
      Promotional techniques significantly increased number of the application’s page views (n=22,024) and actual use (n=133)
      Strong correlation between page views and actual use (γ = 0.994, p=0.0005)
      Email messages to HCPs was best, then “Teaser” in My Health & lastly Fliers to HCPs
    • Results and Discussions
      Profile of responses
      *Data was collected in an Exit survey after patients created the documents
      Documents’ usefulness rating was significantly associated with patient’s age
      Majority created EMC for use in emergencies (56%) or for personal use (26%)
      Only about 15% of patients created CoC report for EMR updates
    • Results and Discussions
      Patients’ rating of the usefulness of EMC & CoC report
      Overall, 63.1% of the patients found EMC and CoC report to be useful
      More patients for CoCthan EMC (64% vs. 63%) , but no significant difference
      Agreement on the usefulness of EMC and CoC report was demonstratedineach CoC measure , but no significant difference in each measure’s ratings
      Patients rated documents highly for usefulness in shortening encounter time
    • Results and Discussions
      Patients-entered data vs. accuracy of EMR data
      69% (70/101) of the patients entered new data values (n=1,994; 1,505 compared)
      44% of newdata was used for EMRupdate, but no significant difference (p=0.109)
      The majority of the EMRupdates were for address, biodata, insurance, Primary HCP, labs
      Significantly lowEMRupdates observed in Biodataand primary HCP data
      NoEMRupdates for allergies & problems data
      Q1/Q3: 25th & 75th Quartiles
    • Results and Discussions
      Patient-entered data vs. completeness of EMR data
      Inconsistency in associations shown between use of CCR application and data completeness
      Significant association between use of CCR application and improved data completeness was observed only in address and appointments data attributes
      Overall, 98.7% complete data before & 76.9% after (OR(95%CI: 0.780(0.779, 0.780), P<0.001)
      Some patient data were not extracted from EMR
      These were annotated by patients because the data was in free text format or not integrated in CDR
      Some sub-group analyses on EMR data quality did not yield results due to sample size issues
      E.g., Insurance, allergies and medical problems – but these may be because they accurately entered in EMR by HCPs
      Reason was not collected for data not updated in EMR
      Opportunity was missed of asking patients reasons why some entered data was not used to update EMR information
    • Conclusions
      Patients demonstrated that the EMC and CoC report were useful tools in enhancing CoC
      Patients also showed that the two documents were useful to identify missing patient information especially in the demographic and appointment categories than other categories
      The patient is an important source of quality control for their record in HCP-maintained EMR
    • Presentation Outline
      Project implementation
      Project evaluation
    • Research Conclusions
      Contribution to the field of Biomedical Informatics
      Showed that the CCR standard can be used to ensure that patientinformationthat is shared among HCPs is standardized
      Demonstrated that HCP-verified EMR data can be used to createEMCs (and CoC reports), in addition to the prevalent use of patient-entered data
      Showed that patientsare able to access their PHRs, monitor and control the quality of their information in the HCP-entered data in EMR
      Showed that timelyand accuratepatientinformationcan be availed to foster enhanced efficiency and effectiveness in the EMRupdates and medicaldecisionmaking by HCPs
    • Research Conclusions
      Lessons Learned and Future Direction
      Free-text, non-integrated data sources and lack of full interoperability of EMRS still pose data extraction difficulties
      Patient is an important source of quality control for their records in HCP-entered data in the EMR
      The EMC and CoC report should be kept up to date since both HCPs and patients have shown the usefulness of the two documents in enhancing CoC – more patients printed EMC
      Use of a variety of promotional techniques are useful for improved (wide scope) marketing and recruitment
      Further validation studies are recommended
    • acknowledgments
      Institutional Review Boards (IRBs)
      University of Utah IRB
      Intermountain Healthcare IRB
      NIH/NLM/Michigan State University
      Julia Royall
      Terrie Taylor
      Reed Gardner, Joyce Mitchell, Lynn Ford, John Hurdle
      Matt Samore, Adi, Lisa-Canon Albright
      Kathy Stoker, Linda Galbreath, JoAnn Thompson
      Faculty, staff and fellow students
      ITS/EDW team
      Jim Livingston, Cheri Hunter, Ming Tu and Vick Deshmukh
      National academy of emergency dispatch
      Jeff Clawson, Alan Fletcher, Pam Stewart
      Intermountain healthcare
      PHC team:
      Belle Rowan (Chair), Matt Smith, Traci Hastings, Carol Askew, Chris Nuccitelli
      Len Bowes, Stan Huff
      ISSA, EDW, SelectHealth…
      UoU/DBMI committee
      R Scott Evans (Chair)
      Mollie Poynton
      Scott Narus
      Jonathan Nebeker
      Joe Hales
      My family
      Mom, Dad, and my Wife and Children