• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis [5 Cr2 1100 Landislewis]
 

Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis [5 Cr2 1100 Landislewis]

on

  • 2,322 views

 

Statistics

Views

Total Views
2,322
Views on SlideShare
2,286
Embed Views
36

Actions

Likes
0
Downloads
11
Comments
0

2 Embeds 36

http://www.medicine20congress.com 35
http://www.slideshare.net 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis [5 Cr2 1100 Landislewis] Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis [5 Cr2 1100 Landislewis] Presentation Transcript

  • Landis Lewis, Z. et al.: Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis
    • This slideshow, presented at Medicine 2.0’08 , Sept 4/5 th , 2008, in Toronto, was uploaded on behalf of the presenter by the Medicine 2.0 team
    • Do not miss the next Medicine 2.0 congress on 17/18th Sept 2009 ( www.medicine20congress.com )
    • Order Audio Recordings (mp3) of Medicine 2.0’08 presentations at http://www.medicine20congress.com/mp3.php
  • Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis Zach Landis Lewis, MLIS Gerald Douglas, MSIS Valerie Monaco, PhD, MHCI University of Pittsburgh Department of Biomedical Informatics
  •  
  • Background: Malawi 12.4 million 119,282 km 2 Pennsylvania 13.6 million Population (2007 est.) 118,484 km 2 Area Malawi 43.5 77.8 Life expectancy at birth in years 900,000 (14.2%) 18,000 (0.15%) Number of people living with HIV/AIDS
  •  
  • Background: Baobab Anti-Retroviral Therapy system (BART)
  •     
    •          
  • Objective
    • Our research objective is to determine how efficiently novice users complete tasks using the touchscreen interface of the EMR.
  • Methods
      • Predict skilled task performance
        • Select tasks
        • Use CogTool software application to generate prediction
    •   2. Measure novice task performance
        • Collect timestamp data from user interface events (e.g. pressing a button )
        • Repeat each task three times
    •   3. Compare prediction with results of novice performance
  • Methods: CogTool
    • validated and used in the field of Human Computer Interaction
    • over 100 papers validating or using human performance modeling for evaluation or design of interfaces
    Q: What is predictive human performance modeling? A: A method for predicting how long a skilled user will take to complete a task
      • Examples of real-world applications:
      • - Web pages and browsers
      • - Telephone operator workstations
      • - Space operations database system
      • - Television control system
      • - Intelligent tutoring system
      • - IRS office automation system
      • - Police in vehicle systems
      • - Firefox tab feature
  •     
    •          
  • Methods: CogTool
  • Methods: CogTool
  • Methods: CogTool The five clusters of colored bars represent all the button presses required to perform this task, separated by thinking time. “ 5” “ 8” “ .” “ 3” “ Next”
  • Methods: CogTool This is the final hand movement operator for pressing the button labeled “5”. This pane shows a close-up view of a sequence of cognitive resources being used. Here we see the activities for pressing the button labeled “5”
  • Methods: CogTool This is a “trace” of production rules fired by the ACT-R production rule system during the task performance The highlighted production rules correspond with cognitive activities occurring while a user is pressing the button labeled “5”.
  • Results: CogTool
      • Selected 31 routinely performed tasks in BART
      • Used CogTool to predict skilled task performance
      • Predicted performance times in seconds for each task
  • Results: Novice Performance
      • Rate of errors:
      • Errors are any deviation from the optimal sequence of steps required to complete a task
      • 77% (286) of task performances were error-free and were compared with CogTool predictions
      • 4 of the 31 tasks were performed without error by all subjects on all repetitions
  • Results: Comparison of CogTool Prediction with Novice Performance
  • Discussion
    • 1. CogTool allowed us to rapidly generate predictions of skilled performance
    • 2. Novice subjects demonstrated a low error rate
    •   3. Novices performed faster than CogTool predictions on average :
      • Tasks were modeled independently, but users interleaved some tasks
      • CogTool's assumptions for inserting "Think" events may not be applicable for wizard format interfaces
  • Discussion, continued
    •   4. Unexpected findings:
      • Pittsburgh subjects occasionally used more than one hand to manipulate the interface – (but we haven’t observed that in Malawi… yet)
      • Communication time varied greatly between tasks, sometimes resulting in prolonged dialog rather than a single question and answer
  • Future Work
    • 1. Update the CogTool model to reflect current, more sophisticated understanding of tasks and user actions - We are working with the CogTool team to be able to adjust the models and CogTool itself to fit the assumptions to our tasks and users
    • 2. Characterize the use of the system in a real-world setting
      • Collect anonymized user interface event data in Malawi from a representative group of users
      • Measure system use by novices and skilled users
  • Acknowledgements
      • The National Institutes of Health and the National Library of Medicine, USA
      • - Grant # 5T15LM007059-22 for funding this research
      • Bonnie John, PhD
      • The CogTool Project - http://www.cs.cmu.edu/~bej/cogtool/
      • Greg Cooper, MD, PhD
      • Mike McKay
      • Joe Rauch, DDS
      • Yolanda DiBucci
      • Margaret Henry