Jason Saleem Slides from AHRQ Kick-Off


Published on

Development and Usability Testing of CDS Alternatives

Published in: Technology, Health & Medicine
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Jason Saleem Slides from AHRQ Kick-Off

  1. 1. Development and Usability Testing of CDS Alternatives January 8 th , 2008 Jason J. Saleem, PhD & Laura Militello, MA LEADERS SYMPOSIUM “ Strategic Planning to Inform a Funded Project on how to Achieve Workflow Integration in Developing and Implementing CDS for CRC Screening”
  2. 2. Study Aims <ul><li>Specific Aim 1: Identify key approaches to CDS development for colorectal cancer screening at two VAMC sites and two nationally recognized non-VA sites, for effective CDS integration into clinical workflow. </li></ul><ul><li>Specific Aim 2: Develop and test CDS design alternatives for improved integration into clinical workflow through a controlled simulation study and subsequent implementation. </li></ul>
  3. 3. Key Informant Interviews of site-specific best-practices for integration of colorectal cancer screening CDS into workflow Direct Observation of colorectal cancer screening CDS for barriers and facilitators to workflow integration Implementation in primary care clinic after simulation study Rapid Prototyping of CDS design alternatives based on Phase 1 findings Simulation Study to test impact of CDS design alternatives on efficiency, usability, and workload Evaluation in primary care clinic after simulation study Figure. Project Overview Phase 1 Phase 2 Phase 3
  4. 4. Human-Computer Interaction Information Technology Laboratory (HCI / IT LAB) D-5014 <ul><li>VA Laptop </li></ul><ul><li>Morae </li></ul><ul><li>Snag-it </li></ul><ul><li>VA Desktops </li></ul><ul><li>Visual Basic </li></ul><ul><li>IU Laptop </li></ul><ul><li>Morae </li></ul><ul><li>Snag-it </li></ul><ul><li>IU Desktops </li></ul><ul><li>Visual Basic </li></ul><ul><li>Azure X 1PC </li></ul>- VA LAN OUTLET - IU LAN OUTLET 47 inch monitor 17” monitor & player for researcher’s office Portable DigiCam w/WA adapter
  5. 5. Rapid Prototyping <ul><li>Prototype design alternatives to address barriers to effective CDS workflow integration identified during the field observations and key informant interviews. </li></ul><ul><li>Rapid – demonstration of designs with no back end programming; low-fidelity mock-ups with prototyping software. </li></ul><ul><li>Screen captures of the current design will be used as a visual base and then redesigned. </li></ul><ul><li>Links and buttons will be made interactive to mimic the actual function of how the electronic medical record and CDS would work if fully programmed. </li></ul>
  6. 6. Simulation Study <ul><li>Usability experiment with simulated outpatient clinic exam room </li></ul><ul><li>Comparison study of the current VHA colorectal cancer CDS and a redesigned prototype. </li></ul><ul><li>Demonstrate improved integration into workflow through relevant surrogate measures of performance, usability, and workload </li></ul><ul><ul><li>Improved efficiency using the CDS, or time to complete tasks, suggests the CDS is better aligned with the providers’ clinical workflow. </li></ul></ul><ul><ul><li>Through improved workflow integration we also expect to see an increase in user satisfaction (usability) and decrease in workload. </li></ul></ul>
  7. 7. Simulation Study <ul><li>At least 20 resident physicians </li></ul><ul><li>Within-subject comparison </li></ul><ul><li>Morae usability software to drive video and audio data collection, and provide direct link between the participant’s clinical workstation and the experimenter’s station </li></ul><ul><ul><li>Record audio and two video sources, one of the computer screen interaction and one of each of the participants’ faces </li></ul></ul><ul><ul><li>Opportunity to record qualitative data on usability critical incidents </li></ul></ul><ul><li>Representative scenarios to construct patient encounters </li></ul><ul><ul><li>Research team member playing the part of the patients; </li></ul></ul><ul><ul><li>Or – use a Standardized Patient (Linder et al., JBI, 2006) </li></ul></ul><ul><li>Debrief session </li></ul>
  8. 8. Dependent Measures Attribute Measure or Tool Efficiency Time to complete scenarios Perceived workload NASA Task Load Index (TLX) User satisfaction Usability questionnaire
  9. 9. NASA Task Load Index (TLX)
  10. 10. Usability Questionnaire <ul><li>Likert-type scale (1=strongly disagree; 7=strongly agree) </li></ul><ul><li>Sample questions: </li></ul><ul><ul><li>The organization of the information on the systems screens is clear. </li></ul></ul><ul><ul><li>The display layouts simplify tasks. </li></ul></ul><ul><ul><li>The sequence of displays is confusing. </li></ul></ul>
  11. 11. Findings <ul><li>Validate and refine prototype through pre-implementation usability testing </li></ul><ul><li>Improved colorectal cancer screening CDS full development and implementation in live clinic for pilot test evaluation (Phase 3) </li></ul><ul><li>Phase 1 (naturalistic observation and key informant interviews) and Phase 2 (rapid prototyping and simulation study) methods based on successful previous work (Saleem et al., JAMIA, 2005; Saleem et al., JAMIA, 2007). </li></ul><ul><ul><li>But fell short of direct impact to current VA system </li></ul></ul><ul><ul><li>Phase 3: Implementation & Dissemination </li></ul></ul>