• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Applying Usability and UCD Methodologies to Achieve Meaningful Use
 

Applying Usability and UCD Methodologies to Achieve Meaningful Use

on

  • 2,792 views

Overview of the role of usability and user-centered design methodologies in the context of ONC meaningful use certification criteria.

Overview of the role of usability and user-centered design methodologies in the context of ONC meaningful use certification criteria.

Statistics

Views

Total Views
2,792
Views on SlideShare
2,792
Embed Views
0

Actions

Likes
2
Downloads
100
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Applying Usability and UCD Methodologies to Achieve Meaningful Use Applying Usability and UCD Methodologies to Achieve Meaningful Use Presentation Transcript

    • Applying Usability and User-Centered Design Methodologies to Achieve Meaningful Use
 June 9, 2010, HIMSS Virtual Conference
 Lisa Battle, Jasmin Phua & Duane Degler
 Design for Context

    • Conflict of Interest Disclosure Lisa Battle User-Centered Design Lead Jasmin Phua User Experience Researcher & Designer HIMSS EHR Usability Taskforce member Duane Degler User-Centered Design Strategist & Semantic Web Expert Consult for and have no real or apparent conflicts of interest to report. Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 2
    • Agenda •  Meaningful Use Overview •  Role of Usability in EHR Meaningful Use •  Build-in Usability through User-Centered Processes •  What is user-centered design •  Evaluating product usability •  Designing for usability Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 3
    • Objectives This education session aims to help you: •  Understand the role of usability in helping achieve meaningful use •  Identify methods of evaluating usability •  Apply user-centered design methodologies to incorporate end-user feedback Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 4
    • “It’s not about technology, it’s about transforming healthcare.” Joshua Seidman, Acting Director, Meaningful Use Office of Provider Adoption Support, ONC Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 5
    • What is all this Meaningful Use buzz? Through the provisions of the American Recovery and Reinvestment Act (ARRA; Recovery Act) of 2009, the Centers for Medicare & Medicaid Services (CMS) will provide incentive payments for the meaningful use of certified electronic health record (EHR) technology. Source: Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Proposed Rule. January 13, 2010 Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 6
    • Who qualifies? Eligible professionals (EPs), eligible hospitals, critical access hospitals (CAHs) for: •  Efforts to adopt, implement, or upgrade certified EHR technology, or •  Meaningful use in first year of participation and for demonstrating meaningful use during each of 5 subsequent years. Source: Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Proposed Rule. January 13, 2010 Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 7
    • How is Meaningful Use measured? Stage 1 Stage 2 Stage 3 incremental process 2011 2016 •  Stage 1 has been defined; meaningful use objectives and certification criteria provided by CMS and ONC. •  Stages 2 & 3 have not been define yet but policy priorities have been painted in broad strokes. •  Different measures for eligible professionals (EPs) vs. hospitals. Sources: Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Proposed Rule. January 13, 2010. Health Information Technology: Initial Set of Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology; Interim final Rule. January 12, 2010. Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 8
    • Meaningful Use: Stage One Stage One (2011-2012) focuses on: •  Electronically capturing health information in a coded format, •  Using that information to track key clinical conditions, •  Communicating that information for care coordination purposes, •  Implementing clinical decision support tools to facilitate disease and medication management, •  Reporting clinical quality measures and public health information. Defined measures for: •  Eligible professionals (EPs): 25 measures •  Eligible hospitals: 23 measures Source: Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Proposed Rule. January 13, 2010 Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 9
    • Why Meaningful Use? Anticipated health policy outcomes for meaningful use of EHR technology are: •  Improvements in quality, safety, efficiency, and reductions in health disparities, •  Engagement of patients and families in their health care, •  Improvements in care coordination, •  Improvements in population and public health, •  Adequate privacy and security protections for personal health information. Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 10
    • Example: Meaningful Use Criteria Health outcomes Improving quality, safety, efficiency, and reducing policy priority health disparities. Use evidence-based order sets and CPOE. Care goals Apply clinical decision support at the point of care. EHR mea ningful u se Implement drug-drug, drug-allergy, drug-formulary Objectives checks. (same for eligible professionals and hospitals) Eligible professional/hospital has enabled this Measures functionality. Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 11
    • Role of Usability in Meaningful Use Achieving meaningful use requires successful implementation of certified EHR technology. Efficiency People get things done The technology does not quickly and productively get in the way They get the info they need, Effectiveness complete work accurately They don’t make mistakes and achieve their goals They feel confident and Satisfaction They are not frustrated pleased Usability Quality attribute defined in ISO 9241, Part 11 Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 12
    • Why does usability matter? Good usability can help... increase reduce Patient and staff safety Training time Productivity and accuracy Error rates Staff morale Staff turnover Customer loyalty Product liability Competitive advantage Customer support Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 13
    • 14 Why does usability matter? Risks of poor usability •  Endangers patients •  Increases adverse events •  Information overload leading to erroneous decisions •  High costs of training, errors, rework •  Increased product & practice liability and last but not least... Barrier to EHR implementation and adoption Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 14
    • Poor Usability = Adverse Events Over a 2 year period, voluntary reporting of adverse events resulting from health IT malfunctions to FDA found: 260 reports with potential for patient harm 44 reports of injuries 6 deaths Usability-related adverse events: The user documented A sleep lab’s workstation BEST EMR SYSTEM activities in the task list for Task List software had confusing John Saint one patient and used the John Smith 1. _______ user interface, which led to “previous” or “next” arrows Janet Smith 2. _______ the overwriting and 3. _______ to select another patient L. Smite replacement of one Paul Smote < prev chart, the patient’s task list next > patient’s data with another displayed for second patient’s study. patient. How many are unreported or caught before they become serious problems? Source: Jeffrey Shuren, Director of CDRH, FDA. Testimony to ONC Health IT Policy Committee, February 25, 2010. Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 15
    • Poor Usability=EHR Adoption Barrier Administrators, clinicians, CIOs, CMOs, and policymakers listed Top 10 barriers to EHR implementation: # 10 Usability Hard to use and not well engineered for clinician workflow. # 9 Politics/naysayers Every organization has a powerful clinician or administrator who is convinced that EHRs will cause harm, disruption, and budget disasters. Solvable # 8 Fear of lost productivity with good Concerned they will lose 25% productivity for 3 months after implementation. usability # 7 Computer illiteracy/training Many clinicians are not comfortable with technology; often reluctant to attend training sessions. # 6 Interoperability Applications do not seamlessly exchange data for coordination of care, performance reporting, and public health. Source: John Halamka, CIO, Beth Israel Deaconess Medical Center during “Leadership Strategies for Information Technology in Health Care” class at Harvard. February 1, 2010. http://geekdoctor.blogspot.com/2010/02/top-10-barriers-to-ehr- implementation.html Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 16
    • Poor Usability=EHR Adoption Barrier Administrators, clinicians, CIOs, CMOs, and policymakers listed Top 10 barriers to EHR implementation: # 5 Privacy Significant local variation in privacy policy and consent management strategies. # 4 Infrastructure/IT reliability Many IT departments cannot provide reliable computing and storage support, leading to EHR downtime. Barriers # 3 Vendor product selection/ suitability mitigated Hard to know what product to choose, particularly for specialists who have unique workflow needs. with user- # 2 Cost the stimulus money does not flow until meaningful use is achieved. Who centered will pay in the meantime? processes #1 People Hard to get sponsorship from senior leaders, find clinician champions, and hire the trained workers to get the EHR rollout done. Source: John Halamka, CIO, Beth Israel Deaconess Medical Center during “Leadership Strategies for Information Technology in Health Care” class at Harvard. February 1, 2010. http://geekdoctor.blogspot.com/2010/02/top-10-barriers-to-ehr- implementation.html Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 17
    • Biggest EHR Usability Problem HIMSS EHR Usability Pain Points Workgroup conducted a survey on the specific factors that resulted in poor usability (catch their presentation on Thursday, June 10). Workflow is the overwhelming problem in almost all facets “Must view many areas to capture the entire patient’s story” “Too much info in too many different places, getting lost and overwhelming” “Doesn't match clinician thought process” Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 18
    • “We failed to focus on the most important part of the decision—the human/computer interface....we didn’t listen to our guts on the design of the computer screen that we would have to look at for hours on end. It is about functionality and workflow.” -Joseph G. Cramer, MD “Bought Wrong EMR” in Medical Economics Magazine February 5, 2010, pp 28-30 Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 19
    • Good Usability = a Good “Fit” •  “Fit” between the object and … •  Its purpose •  Human mental and physical capabilities •  Target audience •  Environment in which it will be used •  Tasks it will be used for Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 20
    • Usability by Design •  Usability does not happen by chance •  It can’t be “painted on” at the end •  It can’t be achieved through testing alone... but usability testing gives great insights as to improvements needed! Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 21
    • How to build-in usability? Usability Test Common problem This isn’t Timeline what I expected… Release Date Brainstorm design ideas Usability Observe the Create Revise designs Test work designs Timeline Timeline Fantastic! And Get user Usability test we don’t even feedback need any Interview training Release users Date Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 22
    • What is User-Centered Design? Industry best practice for creating usable products from the beginning ISO 13407 Perspective Discipline based on human factors affordances working memory mental models predictability feedback Fitts’ law Toolkit of methods & guidelines Risk mitigation strategy task analysis usability testing ethnographic studies walkthroughs usage tracking evidence-based design Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 23
    • How does UCD help with Meaningful Use? Participatory process that involves true end-users and other key stakeholders. When done right... •  Meaningful use requirements are met in a way that fits with how all staff members truly work. •  Your EHR has features that your staff actually need, rather than unnecessary “cool ideas”. •  Finding clinician champions and gaining buy-in from key stakeholders is easier because it is an inclusive process. (helps solve the “people” barrier!) Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 24
    • How does UCD help with Meaningful Use? Practice predicated on getting into users’ heads to EHR understand specific: •  Characteristics •  Tasks and goals •  Context in which they do work When done right... •  Realistic, productive workflows for meaningful use requirements are identified. •  Scenarios where meaningful use criteria are applicable and can be met are comprehensively identified. •  The impact of “meaningful use” implementation on staff duties and responsibilities is anticipated. Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 25
    • User-Centered Design Process Understanding Designing a the needs solution that works Who are the users? Brainstorm Design What are their tasks and ITERATION goals? Test What situations bring with users them here? Best practices for What are their usable design expectations? Progressive refinement Multidisciplinary collaboration Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 26
    • Always start with goals •  Business goals •  Stakeholder goals •  User goals •  Usability goals   Business goals include care goals and meaningful use objectives as defined in the proposed rules. Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 27
    • UCD: Observe & Analyze Observe & Analyze Evaluate Envision & & Refine Design Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 28
    • Conduct user studies Observe & Analyze Envision & Design Evaluate & Refine A variety of activities that gather information about •  Users •  Tasks •  Context of use Users are the actual people who use the product. Users are not: •  The CEO •  Their organizations and managers •  You and the development team •  Your public affairs or marketing department Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 29
    • User research methods Observe & Analyze Envision & Design Evaluate & Refine •  Interviewing •  Analysis of emails, requests, or issues •  Site visits/contextual inquiry •  Conferences, training, user •  Usability testing group meetings •  Surveys •  Usage logs •  Focus groups •  Search logs Not all at once—choose the techniques that fit best with your project Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 30
    • Different user groups have different needs Observe & Analyze Envision & Design Evaluate & Refine General practice Emergency room Patient Insurance physician unit and billing When researching user needs, gather requirements from: •  Direct users •  Indirect users, e.g. billing •  Others who have contact with users, e.g. caregivers Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 31
    • UCD in Meaningful Use: User Needs Meaningful use objective: Implement drug-drug, drug-allergy, drug-formulary checks. Certification criteria example: Automatically and electronically generate and indicate in real-time, alerts at the point of care for drug-drug and drug-allergy contraindications based on medication list, medication allergy list, age, and CPOE. Who will use these drug-drug and drug-allergy checks? Do these user groups have the same needs? Which user group can override alerts? Should they? Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 32
    • Document user needs in Personas Observe & Analyze Envision & Design Evaluate & Refine Personas are: •  Personal, composite sketches of real users •  Discovered through user research •  Representative of typical users, not edge cases Used to: •  Debunk false assumptions •  Help envision users and design what is best for them •  Keep a focus on the user throughout the project Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 33
    • Analyze user tasks Observe & Analyze Envision & Design Evaluate & Refine Tasks are specific activities that people need to accomplish. •  Examples: •  Complete all prescription refill requests •  Discuss the MRI results with the neurologist •  Call Ms. Nelson to discuss test results •  Many tasks include both information and action Task Information Action Discuss MRI Read over patient record, Schedule discussion results with the concentrating on problem time with neurologist neurologist list. Consider diagnosis possibilities. Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 34
    • How to analyze tasks Observe & Analyze Envision & Design Evaluate & Refine •  List them High •  Match them to user groups Medium •  Prioritize them based on Frequency Low •  Frequency •  Criticality of failure •  Break them down into Low Medium High Criticality •  their component parts, and/or •  the sequence of steps involved •  Write stories that illustrate them (scenarios) Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 35
    • Documenting tasks in scenario form Observe & Analyze Envision & Design Evaluate & Refine Scenarios are realistic narrative descriptions of activities that users engage in, detailed enough that design implications can be inferred —Based on J.M. Carroll, 1995 How it works: •  Write the story of the work from the user’s perspective •  Share the story with team to help them visualize how tasks occur in the actual work environment •  Conduct walkthroughs of the proposed designs using these scenarios Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 36
    • Example: UCD in Physician Environment Observe & Analyze Envision & Design Evaluate & Refine Type of Total No. per Physician No. Would want to understand: Service (day) Telephone call 21, 796 23.7 •  Typical amount of time spent Laboratory report 17,794 19.5 during each service interaction Visit 16,640 18.1 •  Most beneficial patient E-mail message 15,499 16.8 interaction Consultation report 12,822 13.9 •  Other categories of services e.g. Prescription refill 11,145 12.1 health plan correspondence, Imaging report 10,229 11.1 FMLA forms 4 FTE physicians, each working 50-60 hrs/wk, 230 days/yr. •  Types of interruptions and Frequent Infrequent frequency (e.g. daily, weekly) (e.g. quarterly, annually) •  Tasks that are queued up to be One thing at a time Several at a time addressed all at once vs. (single) (multiple) piecemeal Source: What’s Keeping Us So Busy in Primary Care? A Snapshot from One Practice by Richard J. Baron, M.D., New England Journal of Medicine, 362; 17, April 29, 2010S Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 37
    • Example: UCD in ER or ICU Observe & Analyze Envision & Design Evaluate & Refine Clinician is developing Patient care at individual individual and shared How would you automate an ER clinical and unit levels mental model of patient whiteboard? Pt 1 needs CT scan, had blood work-up this morning Pt 1 scheduled for Pt 2 doesn’t look CTC at 1500. Needs good. May need line changed to be intubated. beforehand Adapted from Representing Reality: The Human Factors of Health Care Information, C P. Nemeth,M. O'Connor, M. Nunnally, and R I. Cook Chapter 28, Handbook of Human Factors and Ergonomics in Health Care and Patient Safety Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 38
    • UCD in Meaningful Use: Task & Context Meaningful use objective: Implement drug-drug, drug-allergy, drug-formulary checks. Certification criteria example: Automatically and electronically generate and indicate in real-time, alerts at the point of care for drug-drug and drug-allergy contraindications based on medication list, medication allergy list, age, and CPOE. In what situations will these alerts come up? •  Acute episodes / hospitalizations •  •  Chronic conditions Primary care encounters Do not use vancomycin X Drug alert! •  Preventative care In what context will these alerts come up? Emergency room, general practice, pharmacy. ICU System Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 39
    • UCD: Envision & Design Observe & Analyze Evaluate Envision & & Refine Design Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 40
    • Bridging the gap from analysis to design Observe & Analyze Envision & Design Evaluate & Refine MI Humans calculate B are good at some …machines things… are good at other things. about rem ind me ions drug interact lookup drug a llergie s To optimize the system, let each focus on what they are good at. Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 41
    • EHR “It is all about design, which we see every day, but mostly ignore....Design of the computer screen and the underlying program is how our brains see the whole picture of the patient.” -Joseph G. Cramer, MD “Bought Wrong EMR” in Medical Economics Magazine February 5, 2010, pp 28-30 Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 42
    • Design is Hard Observe & Analyze Envision & Design Evaluate & Refine •  You don’t get it right the first time •  There are always trade-offs •  Keys to success: •  Start with a deep knowledge of your users and their tasks •  Generate multiple ideas up front (e.g. through parallel design and brainstorming) •  Iteration – walkthroughs and user feedback •  Progressive refinement •  Following guidelines and patterns for usable design Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 43
    • “Computers are strong medicine. Done well, they are wonderful: done poorly they can kill people” - Justin Starren MD Marshfield Clinic, Wisconsin Source: As Doctors Shift to Electronic Health Systems, Signs of Harm Emerge, by Fred Schulte and Emma Schwartz, Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 44 April 20, 2010, Huffington Post
    • UCD in Meaningful Use: Design Meaningful use objective: Implement drug-drug, drug-allergy, drug-formulary checks. Certification criteria example: Automatically and electronically generate and indicate in real-time, alerts at the point of care for drug-drug and drug-allergy contraindications based on medication list, medication allergy list, age, and CPOE. I talked to my users and stakeholders, now magic happens! It can still can go very wrong. For example, CPOE systems often flood doctors with warning alerts, leading physicians to ignore them, which is a human factor principle known as “alert fatigue” or “pop-up fatigue”. Source: Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors, Ross Koppel; Joshua P. Metlay; Abigail Cohen; et al. JAMA. 2005;293(10):1197-1203 Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 45
    • Magic happens or not... Dangerous Drug Interaction Warfarin x Sulfa Potential for bleeding details Patient currently on: Coumadin, since 03/09/2001 Consider alternatives: MyCPOE System Patient: Jane Smith Cephalexin Change order Dangerous Drug Interaction: Warfarin x Sulfa Nitrofurantoin Patient Summary Warfarin x Sulfa interaction: Potential for bleeding Don’t fill Clinical Notes Fill order Pt currently on: Problem List Coumadin (warfarin) Medication ----------------------- Consults Lab Results Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 46
    • Visioning Observe & Analyze Envision & Design Evaluate & Refine Example: Designing a new house What are we trying to build? What would be best for our users? What will help them accomplish their tasks? How can we meet their expectations? What is the best way to meet our goals? Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 47
    • Visioning Observe & Analyze Envision & Design Evaluate & Refine •  In user-centered design, this involves: •  “Blue sky” brainstorming sessions •  Fast, informal generation of ideas •  Flip charts •  Whiteboard drawings Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 48
    • Parallel Design Observe & Analyze Envision & Design Evaluate & Refine Sketch #B “What I really like about this Alert is helpful and lets idea is…?” me change the order if I need to. I don’t need to click through 5 screens again to do that. Sketch #A Serious alerts are easy Sketch #C to spot It lets me get to Like being able to see more details so I the rest of the patient’s don’t need to go record back out to look it up Generates a wide range of design possibilities quickly The full group discusses pros and cons of each Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 49
    • Structural Level: Organizing Observe & Analyze Envision & Design Evaluate & Refine Example: Designing a new house How should it be organized? What are its main sections? What will people do in each area? What will people expect each area to be called? How can we streamline the path from one section to another? Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 50
    • Structural Level: Organizing Observe & Analyze Envision & Design Evaluate & Refine •  In user-centered design, this involves: •  Abstract prototypes •  Sitemaps •  Flow charts •  Card sorts Abstract prototype Sitemap Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 51
    • Abstract Prototype Observe & Analyze Envision & Design Evaluate & Refine Used in discussion with stakeholders to clarify content and organization Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 52
    • Representing and Refining Observe & Analyze Envision & Design Evaluate & Refine Example: Designing a new house Which things should be near each other because they go together? Is this the right style of interaction? Does this layout support the flow of the task? Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 53
    • Representing and Refining Observe & Analyze Envision & Design Evaluate & Refine •  In user-centered design, this involves: •  Sketches •  Low-fidelity prototypes or mockups •  Wireframes Paper prototypes Wireframes Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 54
    • Types of Prototypes Observe & Analyze Envision & Design Evaluate & Refine Low Fidelity High Fidelity Good for testing: Good for testing: Good for testing: - Concepts -  Terminology -  Visual appeal - Organization -  Headings -  Interactions - Screen flow -  Navigation -  Accessibility - User tasks - Main ideas Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 55
    • Iteration Observe & Analyze Envision & Design Evaluate & Refine •  Design in repeated cycles •  Results of each cycle feed into the next cycle Brainstorm Design ITERATION Test with users (or walkthrough with specialists) •  Get input and feedback early and often •  Prototypes don’t need to be working yet •  Less “finished” looking, more options, elicit more feedback Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 56
    • UCD: Evaluate & Refine Observe & Analyze Evaluate Envision & & Refine Design Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 57
    • Many Ways to Get User Feedback Observe & Analyze Envision & Design Evaluate & Refine •  In addition to usability testing, you can use: •  Informal, scenario-based walkthroughs •  Surveys •  Web metrics and usage tracking •  Management information on transactions •  Help desk log •  Emailed feedback Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 58
    • User Feedback Sessions Observe & Analyze Envision & Design Evaluate & Refine •  Set expectations •  Ask the user to “think aloud” and interpret what they see •  Ask the user about realistic tasks •  Ask the user to compare alternatives Refer to handout for details Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 59
    • Testing for Usability Observe & Analyze Envision & Design Evaluate & Refine Get real users Ask them to perform realistic tasks using your system, prototype, or web site Observe, take notes, and see what works and what doesn’t work If they have problems, fix them before the release! Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 60
    • Types of Usability Tests Observe & Analyze Envision & Design Evaluate & Refine •  Formal vs informal •  Formal: In a lab with two-way mirrors, logging software, video cameras, observers •  Informal: In a cafeteria, senior center, at home, or wherever, with nothing but your prototype •  Formative vs summative •  Formative: Identify problems and opportunities for improvement •  Summative: Determine whether performance measures were met, or to set a baseline •  In person vs remote Refer to handout for details Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 61
    • Typical Measures for Usability Observe & Analyze Envision & Design Evaluate & Refine •  Efficiency •  Time to complete task •  Number of clicks •  Number of days/hours training reduced •  Effectiveness •  Success rate (or completion rate) •  Number or % of errors •  Number of attempts before successful completion •  Cost savings from reduced errors •  Satisfaction •  Number of positive and negative statements or feedback messages received from users •  Satisfaction scores on a survey (e.g. SUS, QUIS, ACSI) •  Number of users who rate the system as good or excellent •  Actual usage (number of people, % increase) Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 62
    • Design Walkthroughs Observe & Analyze Envision & Design Evaluate & Refine Walkthrough Abstract Low-Fi High-Fi prototype Prototype Prototype 1) Scenario-Based Walkthrough    2) Persona-Based Walkthrough   3) Requirements Walkthrough    4) Subject-Matter Expert Walkthrough    5) Database Walkthrough    6) Information Needs Walkthrough   7) Usable Design Principles Walkthrough   8) Accessibility Principles Walkthrough   Refer to handout for details Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 63
    • Is there a magic number? Observe & Analyze Envision & Design Evaluate & Refine We have often been asked... No, “Is there a magic number to ther figure out if my product e isn’ passes/fails usability?” t. Why? Usability is measured by: Efficiency Effectiveness Satisfaction and is about balancing user needs. It is not a threshold measure. Is there a magic number for human clinical trials? Why? Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 64
    • When user requirements go awry Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 65
    • Takeaway: Do it early Planning Requirements Design Development Validation Roll-out f C ost o ons ti ifica mod Identify usability and accessibility needs as early as possible to reduce costs Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 66
    • Takeaway: Build it in from the beginning Involve real users early and often Observe their actual work Work collaboratively with a multi-disciplinary team Follow human factors & usable design guidelines Design the user interface deliberately Iterate the design with user feedback Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 67
    • Takeaway: Start now! Start now! You have the means... •  Look at your paper forms and their contents: •  What you are collecting and why? •  How do you intend to use it when it is digital? •  Analyze patterns of work e.g. patient requests, repetitive fixed events, tasks everyone can do •  Look at your current workflow. What’s optimal? •  Scrutinize problem logs. Make sure you don’t automate the problem source! Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 68
    • Better design, better healthcare Transform healthcare through better design reduce increase errors ease of use fatigue productivity repetitive work success rate stress & frustration human comfort loss of time user acceptance training needs satisfaction Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 69
    • Questions? Lisa Battle: lisa@designforcontext.com Jasmin Phua: jasmin@designforcontext.com Duane Degler: duane@designforcontext.com