Usability methods to improve EMRs

June 9, 2010
Usability Methodsto Improve EMRs
Janey Barnes, PhDUser-View, Inc.
Raleigh, NC
Jeff Belden, MD Family &Community MedicineUniversity of Missouri
1

Conflict of Interest Disclosure
Janey Barnes, PhD
Contracted Research:
Allscripts
Patagonia Health
Duke Health Systems
2

Conflict of Interest Disclosure
Jeff Belden, MD
Dr. Belden is on faculty at University of Missouri - Columbia, and an EMR usability consultant with the Tiger Institute, a technology collaborative between the University of Missouri and Cerner Corporation.
Dr. Belden has no financial interest.
3

Session Objectives
Describe stages of software development in which usability methods can be most effective.
List and describe usability methods applicable to each stage of EMR development.
Analyze the cost/benefit of implementing usability methods within a user-centered design process.
4

Software Development Process
5

Waterfall modelof software development
6

7
Waterfall

“Agile” modelof software development
8

9
Agile

Usability methods work for both
10

don’t forget the buyer’s sideof usability
11

12
Agile
Waterfall
Selection
Implementation

Want to be a change-agent?
13

User Centered Design Process
15

16
Incorporating User Centered Designin Software Development Process
Prototyping
Card sorting
Concept exploration
Usability Testing
and testing
Keep testing
Requirements
User stories
Summative testing
Affinity Diagramming
Personas
Use case scenarios
User acceptance testing
Incorporating feedback
Maintain design intent

a few words about usability when selecting & implementing
17

Software Selection Activities
Usability Methods of Interest
Site visit for:
User surveys of current users like you
Performance testing
Observe one of their experienced (but not an ace) users
Do clinical scenario test with real data
Define target measures
Set goals
18

Implementation Activities
Usability Methods of Interest
Task analysis & user observation
Card sorting (picking categories, nomenclature)
Design guidelines & Style guides
Paper prototyping
Have clinician create documents for several patients
19

DiscoveryActivities
Contextual inquiry
Affinity diagramming
Personas
Use case scenarios
For each method we present
Goalof the method
Details for planning& executingeach method
Best timing the software development schedule
How to deliver impactto the product
21

Contextual Inquiry
Persona
22

Goals
understand the user’s:
tasks
workflows
environment
description of a specific person (a target user). Details make them real.
23
Contextual Inquiry
Persona

Contextual Inquiry
Persona
Planning
Execution
Observe users onsite
Be a fly on the wall
Before, during & after
Half day…or longer!
Take photos of environment.
Watch. Observe. Learn.
Make notes.
Stand, sit, to side, in middle.
Stay out of way
Collect any artifacts.

Contextual Inquiry
Persona
Timing
Early
Requirements gathering
Before sprint planning
Before “iteration 1”
Having Impact
By watching, observing, learning

26
Contextual Inquiry
Use Case Scenarios

27
Contextual Inquiry
Use Case Scenarios
Goals
Understand the user’s:
tasks
workflows
environment
Specify how users carry out their tasks in a specified context.

28
Contextual Inquiry
Use Case Scenarios
Planning
Execution
Observe users onsite
Be a fly on the wall
Before, during & after
Half day…or longer!
Take photos of environment.
Watch. Observe. Learn.
Make notes.
Stand, sit, to side, in middle.
Stay out of way
Collect any artifacts.

29
Contextual Inquiry
Use Case Scenarios
Timing
Early
Before sprint planning
Before “iteration 1”
Having Impact
By watching, observing, learning

30
Contextual Inquiry
How long should this take?

31
Contextual Inquiry
How long should we plan?
Plan
Teach team to observe
Not sell, or teach, or fix
Have tools ready

32
Contextual Inquiry
How long should we stay to observe?
It depends
How varied are user groups?
How familiar are you with territory?
Stay until you see patterns emerge

33
Contextual Inquiry
How long to compile findings?
Compile into
Personas
Use case scenarios
Work flows
Task flows

Definition Activities
Requirements
User Stories
Goal
Planning
Execution
Timing
Impact
35

Goal
Identify usability requirements that can be tested later
For each chosen task and user type estimate:
acceptable task time & optimum target
how to score effectiveness - agreeing what errors user might make
the effectiveness target
the satisfaction target.
36

Planning
Execution
Arrange workshop with users, developers.
Review tasks
For each task & user type estimate
Task time & optimum goal
How to score effectiveness
Effectiveness target
Satisfaction Target
Decide on usability requirements among:
Understandability
Learnability
Operability
Attractiveness
Recruiting physicians is challenging.
37

Timing
Early
Impact
Shows importance of usability early on
Provides concrete objectives & testable criteria for usability
38

User Stories
39

User Stories
Goal
More approachable than a formalized use case
Slim, give high-level requirements
Just a sentence or three.
“A surgeon needs latest evolving lab results while moving through the hospital, without stopping to log-in at a PC”.
40

User Stories
Planning
Execution
Arrange short meeting with customer
Bring 3x5 cards for each story
User writes user stories
3x5 note card
Rewrite until clear
Developer may use questions to get user going
41

User Stories
Timing
Early
Impact
Short, quick to generate
Low maintenance
Keeps contact with user
Helps estimate the development work effort
42

Design & User Feedback Activities
Information Design
Prototyping
Card sorting
Interaction Design
Prototyping
Visual Design
Concept exploration
44

Prototyping
45

Prototyping
Goal
clarify requirements
create draft of interaction designs & screen designs to be rapidly simulated & tested
46

Prototyping
Planning
Arrange workshop with
Users
Developers
Facilitator & scribe
Four stages
Concept design
Interaction design
Screen design
Screen testing
47

Prototyping
Execution
Concept design
Sketch possible approaches
See if they meet agree objectives
Interaction design
Use Post-It notes, writing each suggested screen or activity
Group them, name clusters, arrange sequences
48

Prototyping
Execution
Screen design
Brainstorm screen designs with users
Ask user to carry out a realistic task, pointing among screen sketches
Screen testing
Mock-up rough designs.
Walk thru steps, explaining what happens, or get fancier with paper menus, dialog boxes, etc.
49

Prototyping
Timing
Early
Before code
Impact
Find usability problems early before code is written
Communication between users & designers is promoted
Paper is cheap & quick
50

Card sorting
51

Card Sorting
Goal
Aids information design
Discover latent structure in an unsorted list of ideas
Example: “Group these smart-phone functions into ones that you need, and ones that would be optional. Then rank the needed ones in order of importance.”
52

Card Sorting
Planning
Make separate cards for each statement
Number cards on back
Find at least 6 users
Execution
Shuffle the deck each time.
Have users group cards
Two styles of card sort
Closed (you supply category)
Open (user names groups)
Note results. Include names user gave groups, proximity info or comments user made.
53

Card Sorting
Timing
Design phase
Before coding
Impact
Helps discover users’ mental model
Shows how ideas or concepts should be presented
54

Speaker handoff

Visual Design
Concept Exploration
Goal
Communicating information organization
Communicating information priority
Engaging brand
56

Visual Design
It’s not just pretty colors…
57

Visual Design
Good visual design communicates:
Info organization
Info priority
and reduces cognitive load
59

Visual Design
Concept Exploration
Planning: Iterative Process
Execution: Research methods to quantify effectiveness of Visual Design
Timing: In parallel with information & interaction design
Impact: Critical! Visual Design is not just “make it pretty”
60

Prototyping
Low-resolution
High-resolution

Usability Tests
64

Usability Tests
65

Usability Tests - one-on-one sessions where participant user performs key/at risk tasks while researcher gathers data related to performance
Goals
66
Formative
Design decisions
Identify usability issues
Summative
Evaluate usability goals
Validate design

Usability Tests
Formative and Summative
Planning
Execution
Test Objectives
Test Design
Test Plan
including data analysis plan
Test Participants
Test Environment
Test Report
Pilot Test
Logistics
Carry out plan
including test and data analysis
Share with team & IMPACT design
67

Usability Tests
Timing
Formative
Earlier
As soon as you have use cases in a prototype
Summative
Later
Enough
Stability
Early enough to make IMPACT
Fit to sprints
UX a sprint ahead
UX doing formative activities for next sprint and summative activities for current sprint
68

Usability Tests
Having Impact
Actionable & Prioritized Recommendations
Here are 3 examples…

Usability Tests
Actionable & prioritized recommendations
High Priority
Users (patients) cannot find Clinician’s name in the list of physicians (appointment task). Put names in alphabetical order
High Priority
Users (physicians) cannot find “New Prescription” action button. Make “New Prescription” action button visually apparent.
High Priority
Users (physicians) cannot find patient’s name on open chart quickly. Put the patient’s name in a visually distinct location on the screen and be consistent on all screens.

Development Activities
User Acceptance Testing (Software)
Incorporating user feedback
Maintaining the intent of the design
71
Keep the door open between
UX and Development!

User Acceptance Testing (Software)
Goal: Validate that the application works as intended
Planning: Begins during Define Phase
Execution: Automatic versus Manual
Timing: Nearing the end
Impact: Too late to IMPACT the application?
72

Incorporating user feedback
Maintaining the intent of the design
Goal: SUPPORT development team
Planning: Little planning time – rely on process
Execution: SUPPORT
Timing: Nearing the end
Impact: CRITICAL time
73

Beta Testing Activities
74
Usability Activities
during Beta Testing
Goal
From Usability Perspective:
Live usability test to inform current application and next release
System-wide usability test
Preparation for implementations

75
during Beta Testing
Planning
Execution
Test Objectives
Test Design
Test Plan
Test Participants
Test Environment
Test Report
Pilot Test
Logistics
Carry out plan
Share with team & IMPACT design, implementation

during Beta Testing
76
Timing: We are nearing the end
Impact: Not too late to impact the application

Launch Activities
77
Continued Testing
Goal
From Usability Perspective:
Live usability test to inform current application and next release
System-wide usability test

Launch Activities
Continued Testing
78
Planning
Execution
Test Objectives
Test Design
Test Plan
Test Participants
Test Environment
Test Report
Pilot Test
Logistics
Carry out plan
Share with team & IMPACT design, implementation

Launch Activities
79
Continued Testing
Timing: We are nearing the end
Impact: Not too late to impact the application

80
Post-Launch
Post-Launch
Post Launch Activities
Learnings Discover Activities
Implementation
Discover
Define
Launch
Selection
Design
Beta
Develop
Implementation

81
Cost / Benefit

Cost/Benefit
Key to cost-effective product usability is to
Plan & Manage
the usability activities within the development process.
82

Cost/Benefit
83
Summative
Formative
Formative
Prototyping
Card sorting
Concept exploration
Usability Testing
and testing
Keep testing
Requirements
User stories
Summative testing
User acceptance testing
Incorporating feedback
Maintain design intent
Personas
Use case scenarios
specific activity designed to gather data to address specific research question

http://bit.ly/UXmethods
84

85
Costs
Including Usability
in Development Cycle
Design Alternatives
Cost of Changes
Planning
Design
Development
Release

CostsUsability in the Development Cycle 1 of 2
Discount Usability vs Premium Usability
Time (labor)
Employee vs Contractor (expendable)
Design vs Evaluation
Test system setup
Participant recruitment and compensation
86

CostsUsability in the Development Cycle 2 of 2
Lab equipment & usability lab facility
Travel expenses
Planning Time !!
Managers must make the time to plan the integration of usability into the development process.
Sometimes too busy to stop and determine how to integrate usability during development.
87

Measurable Benefitsof Usability Integration
Improved productivity
Shorter time to complete tasks
Fewer errors during tasks
Improved user satisfaction
Lower training costs
Reduced tech support & documentation costs
Increased patient safety
Litigation deterrence
User preferences
Brand loyalty
Marketing voice
88

Tradeoffsbetween Usability Costs & Benefits
Measurable benefits not well documented
“Post” data is often not collected due to budget and lack of vision
Proprietary data used competitively and not available to the public
Usability community has made efforts to quantify benefit of Usability interventions
89

Tradeoffsbetween Usability Costs & Benefits
Case scenarios presented as examples that hopefully demonstrate the cost-benefit tradeoffs across several types of HF activities.
Data for case scenarios taken from:
Good Ergonomics Is Good Economics (1996). Hal Hendrick. Proceedings of the HFES 40th.
Usability Professionals’ Association (UPA) http://www.upassoc.org
90

Best Bang for the BuckKey Usability Activities
Contextual Inquiry
Usability Testing (1 on 1) with prototype or early code
Enough functionality to test
Early enough to make changes
91

Session Objectives
Describe stages of software development in which usability methods can be most effective.
Requirements & Design
List and describe usability methods applicable to each stage of EMR development.
Contextual Inquiry & Usability Testing
Analyze the cost/benefit of implementing usability methods within a user-centered design process.
Early usability methods are most cost effective in terms of planning, carrying out, and incorporating findings in software.
92

Links
Usability Methods Table | Usability.net
HIMSS EHR Usability Links | Online resources

Questions?
For further info, contact:Janey Barnes PhD | jbarnes@user-view.comJeff Belden MD | jeffbelden@gmail.com

Usability methods to improve EMRs

by Jeff Belden MD

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 13

Statistics

Usability methods to improve EMRs Presentation Transcript