Landis Lewis, Z. et al.: Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Perform...
Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool...
 
Background: Malawi 12.4 million 119,282 km 2   Pennsylvania 13.6 million Population  (2007 est.) 118,484 km 2 Area Malawi ...
 
Background: Baobab Anti-Retroviral Therapy system (BART)
      <ul><li>           </li></ul>
Objective <ul><li>Our research objective is to determine how efficiently novice users complete tasks using the touchscreen...
Methods <ul><ul><li>Predict skilled task performance </li></ul></ul><ul><ul><ul><li>Select tasks </li></ul></ul></ul><ul><...
Methods: CogTool <ul><li>validated and used in the field of Human Computer Interaction  </li></ul><ul><li>over 100 papers ...
      <ul><li>           </li></ul>
Methods: CogTool
Methods: CogTool
Methods: CogTool The five clusters of colored bars represent all  the button presses required to perform this task, separa...
Methods: CogTool This is the final hand movement operator for pressing the button labeled “5”.  This pane shows a close-up...
Methods: CogTool This is a “trace” of production rules fired by the ACT-R production rule system during the task performan...
Results: CogTool <ul><ul><li>Selected 31 routinely performed tasks in BART </li></ul></ul><ul><ul><li>Used CogTool to pred...
Results: Novice Performance <ul><ul><li>Rate of errors: </li></ul></ul><ul><ul><li>Errors are any deviation from the optim...
Results: Comparison of CogTool Prediction with Novice Performance
Discussion <ul><li>1. CogTool allowed us to rapidly generate predictions of  skilled performance </li></ul><ul><li>2. Novi...
Discussion, continued <ul><li>  4. Unexpected findings: </li></ul><ul><ul><li>Pittsburgh subjects occasionally used more t...
Future Work <ul><li>1.  Update the CogTool model to reflect current, more  sophisticated understanding of tasks and user  ...
Acknowledgements <ul><ul><li>The National Institutes of Health and the National Library of Medicine, USA </li></ul></ul><u...
Upcoming SlideShare
Loading in …5
×

Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis [5 Cr2 1100 Landislewis]

1,669 views
1,508 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,669
On SlideShare
0
From Embeds
0
Number of Embeds
47
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis [5 Cr2 1100 Landislewis]

    1. 1. Landis Lewis, Z. et al.: Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis <ul><li>This slideshow, presented at Medicine 2.0’08 , Sept 4/5 th , 2008, in Toronto, was uploaded on behalf of the presenter by the Medicine 2.0 team </li></ul><ul><li>Do not miss the next Medicine 2.0 congress on 17/18th Sept 2009 ( www.medicine20congress.com ) </li></ul><ul><li>Order Audio Recordings (mp3) of Medicine 2.0’08 presentations at http://www.medicine20congress.com/mp3.php </li></ul>
    2. 2. Measuring Efficiency of Use in a Web-based EMR Developed for Malawi: Lessons Learned From Performing a Preliminary CogTool Analysis Zach Landis Lewis, MLIS Gerald Douglas, MSIS Valerie Monaco, PhD, MHCI University of Pittsburgh Department of Biomedical Informatics
    3. 4. Background: Malawi 12.4 million 119,282 km 2 Pennsylvania 13.6 million Population (2007 est.) 118,484 km 2 Area Malawi 43.5 77.8 Life expectancy at birth in years 900,000 (14.2%) 18,000 (0.15%) Number of people living with HIV/AIDS
    4. 6. Background: Baobab Anti-Retroviral Therapy system (BART)
    5. 7.      <ul><li>          </li></ul>
    6. 8. Objective <ul><li>Our research objective is to determine how efficiently novice users complete tasks using the touchscreen interface of the EMR. </li></ul>
    7. 9. Methods <ul><ul><li>Predict skilled task performance </li></ul></ul><ul><ul><ul><li>Select tasks </li></ul></ul></ul><ul><ul><ul><li>Use CogTool software application to generate prediction </li></ul></ul></ul><ul><li>  2. Measure novice task performance </li></ul><ul><ul><ul><li>Collect timestamp data from user interface events (e.g. pressing a button ) </li></ul></ul></ul><ul><ul><ul><li>Repeat each task three times </li></ul></ul></ul><ul><li>  3. Compare prediction with results of novice performance </li></ul>
    8. 10. Methods: CogTool <ul><li>validated and used in the field of Human Computer Interaction </li></ul><ul><li>over 100 papers validating or using human performance modeling for evaluation or design of interfaces </li></ul>Q: What is predictive human performance modeling? A: A method for predicting how long a skilled user will take to complete a task <ul><ul><li>Examples of real-world applications: </li></ul></ul><ul><ul><li>- Web pages and browsers </li></ul></ul><ul><ul><li>- Telephone operator workstations </li></ul></ul><ul><ul><li>- Space operations database system </li></ul></ul><ul><ul><li>- Television control system </li></ul></ul><ul><ul><li>- Intelligent tutoring system </li></ul></ul><ul><ul><li>- IRS office automation system </li></ul></ul><ul><ul><li>- Police in vehicle systems </li></ul></ul><ul><ul><li>- Firefox tab feature </li></ul></ul>
    9. 11.      <ul><li>          </li></ul>
    10. 12. Methods: CogTool
    11. 13. Methods: CogTool
    12. 14. Methods: CogTool The five clusters of colored bars represent all the button presses required to perform this task, separated by thinking time. “ 5” “ 8” “ .” “ 3” “ Next”
    13. 15. Methods: CogTool This is the final hand movement operator for pressing the button labeled “5”. This pane shows a close-up view of a sequence of cognitive resources being used. Here we see the activities for pressing the button labeled “5”
    14. 16. Methods: CogTool This is a “trace” of production rules fired by the ACT-R production rule system during the task performance The highlighted production rules correspond with cognitive activities occurring while a user is pressing the button labeled “5”.
    15. 17. Results: CogTool <ul><ul><li>Selected 31 routinely performed tasks in BART </li></ul></ul><ul><ul><li>Used CogTool to predict skilled task performance </li></ul></ul><ul><ul><li>Predicted performance times in seconds for each task </li></ul></ul>
    16. 18. Results: Novice Performance <ul><ul><li>Rate of errors: </li></ul></ul><ul><ul><li>Errors are any deviation from the optimal sequence of steps required to complete a task </li></ul></ul><ul><ul><li>77% (286) of task performances were error-free and were compared with CogTool predictions </li></ul></ul><ul><ul><li>4 of the 31 tasks were performed without error by all subjects on all repetitions </li></ul></ul>
    17. 19. Results: Comparison of CogTool Prediction with Novice Performance
    18. 20. Discussion <ul><li>1. CogTool allowed us to rapidly generate predictions of skilled performance </li></ul><ul><li>2. Novice subjects demonstrated a low error rate </li></ul><ul><li>  3. Novices performed faster than CogTool predictions on average : </li></ul><ul><ul><li>Tasks were modeled independently, but users interleaved some tasks </li></ul></ul><ul><ul><li>CogTool's assumptions for inserting &quot;Think&quot; events may not be applicable for wizard format interfaces </li></ul></ul>
    19. 21. Discussion, continued <ul><li>  4. Unexpected findings: </li></ul><ul><ul><li>Pittsburgh subjects occasionally used more than one hand to manipulate the interface – (but we haven’t observed that in Malawi… yet) </li></ul></ul><ul><ul><li>Communication time varied greatly between tasks, sometimes resulting in prolonged dialog rather than a single question and answer </li></ul></ul>
    20. 22. Future Work <ul><li>1. Update the CogTool model to reflect current, more sophisticated understanding of tasks and user actions - We are working with the CogTool team to be able to adjust the models and CogTool itself to fit the assumptions to our tasks and users </li></ul><ul><li>2. Characterize the use of the system in a real-world setting </li></ul><ul><ul><li>Collect anonymized user interface event data in Malawi from a representative group of users </li></ul></ul><ul><ul><li>Measure system use by novices and skilled users </li></ul></ul>
    21. 23. Acknowledgements <ul><ul><li>The National Institutes of Health and the National Library of Medicine, USA </li></ul></ul><ul><ul><li> - Grant # 5T15LM007059-22 for funding this research </li></ul></ul><ul><ul><li>Bonnie John, PhD </li></ul></ul><ul><ul><li>The CogTool Project - http://www.cs.cmu.edu/~bej/cogtool/ </li></ul></ul><ul><ul><li>Greg Cooper, MD, PhD </li></ul></ul><ul><ul><li>Mike McKay </li></ul></ul><ul><ul><li>Joe Rauch, DDS </li></ul></ul><ul><ul><li>Yolanda DiBucci </li></ul></ul><ul><ul><li>Margaret Henry </li></ul></ul>

    ×