Your SlideShare is downloading. ×
  • Like
Jason Saleem Slides from AHRQ Kick-Off
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Jason Saleem Slides from AHRQ Kick-Off


Development and Usability Testing of CDS Alternatives

Development and Usability Testing of CDS Alternatives

Published in Technology , Health & Medicine
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Development and Usability Testing of CDS Alternatives January 8 th , 2008 Jason J. Saleem, PhD & Laura Militello, MA LEADERS SYMPOSIUM “ Strategic Planning to Inform a Funded Project on how to Achieve Workflow Integration in Developing and Implementing CDS for CRC Screening”
  • 2. Study Aims
    • Specific Aim 1: Identify key approaches to CDS development for colorectal cancer screening at two VAMC sites and two nationally recognized non-VA sites, for effective CDS integration into clinical workflow.
    • Specific Aim 2: Develop and test CDS design alternatives for improved integration into clinical workflow through a controlled simulation study and subsequent implementation.
  • 3. Key Informant Interviews of site-specific best-practices for integration of colorectal cancer screening CDS into workflow Direct Observation of colorectal cancer screening CDS for barriers and facilitators to workflow integration Implementation in primary care clinic after simulation study Rapid Prototyping of CDS design alternatives based on Phase 1 findings Simulation Study to test impact of CDS design alternatives on efficiency, usability, and workload Evaluation in primary care clinic after simulation study Figure. Project Overview Phase 1 Phase 2 Phase 3
  • 4. Human-Computer Interaction Information Technology Laboratory (HCI / IT LAB) D-5014
    • VA Laptop
    • Morae
    • Snag-it
    • VA Desktops
    • Visual Basic
    • IU Laptop
    • Morae
    • Snag-it
    • IU Desktops
    • Visual Basic
    • Azure X 1PC
    - VA LAN OUTLET - IU LAN OUTLET 47 inch monitor 17” monitor & player for researcher’s office Portable DigiCam w/WA adapter
  • 5. Rapid Prototyping
    • Prototype design alternatives to address barriers to effective CDS workflow integration identified during the field observations and key informant interviews.
    • Rapid – demonstration of designs with no back end programming; low-fidelity mock-ups with prototyping software.
    • Screen captures of the current design will be used as a visual base and then redesigned.
    • Links and buttons will be made interactive to mimic the actual function of how the electronic medical record and CDS would work if fully programmed.
  • 6. Simulation Study
    • Usability experiment with simulated outpatient clinic exam room
    • Comparison study of the current VHA colorectal cancer CDS and a redesigned prototype.
    • Demonstrate improved integration into workflow through relevant surrogate measures of performance, usability, and workload
      • Improved efficiency using the CDS, or time to complete tasks, suggests the CDS is better aligned with the providers’ clinical workflow.
      • Through improved workflow integration we also expect to see an increase in user satisfaction (usability) and decrease in workload.
  • 7. Simulation Study
    • At least 20 resident physicians
    • Within-subject comparison
    • Morae usability software to drive video and audio data collection, and provide direct link between the participant’s clinical workstation and the experimenter’s station
      • Record audio and two video sources, one of the computer screen interaction and one of each of the participants’ faces
      • Opportunity to record qualitative data on usability critical incidents
    • Representative scenarios to construct patient encounters
      • Research team member playing the part of the patients;
      • Or – use a Standardized Patient (Linder et al., JBI, 2006)
    • Debrief session
  • 8. Dependent Measures Attribute Measure or Tool Efficiency Time to complete scenarios Perceived workload NASA Task Load Index (TLX) User satisfaction Usability questionnaire
  • 9. NASA Task Load Index (TLX)
  • 10. Usability Questionnaire
    • Likert-type scale (1=strongly disagree; 7=strongly agree)
    • Sample questions:
      • The organization of the information on the systems screens is clear.
      • The display layouts simplify tasks.
      • The sequence of displays is confusing.
  • 11. Findings
    • Validate and refine prototype through pre-implementation usability testing
    • Improved colorectal cancer screening CDS full development and implementation in live clinic for pilot test evaluation (Phase 3)
    • Phase 1 (naturalistic observation and key informant interviews) and Phase 2 (rapid prototyping and simulation study) methods based on successful previous work (Saleem et al., JAMIA, 2005; Saleem et al., JAMIA, 2007).
      • But fell short of direct impact to current VA system
      • Phase 3: Implementation & Dissemination