Your SlideShare is downloading. ×
Usability methods to improve EMRs
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Usability methods to improve EMRs

3,845
views

Published on

Jeff Belden MD and Janey Barnes PhD co-presented at HIMSS Virtual Conference June 2010. You can hear the audio recording if you are a HIMSS member, available online.

Jeff Belden MD and Janey Barnes PhD co-presented at HIMSS Virtual Conference June 2010. You can hear the audio recording if you are a HIMSS member, available online.


1 Comment
10 Likes
Statistics
Notes
  • Dear uploaded,

    I need to download this presentation. It's really a good presentation. I really appreciate your effort. I'll be thankful to you if you let me download this presentation. Thanks a million.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
3,845
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
186
Comments
1
Likes
10
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Note to Janey - if you conduct the wrong activity for the specific research question = trouble
  • Transcript

    • 1. June 9, 2010
      Usability Methodsto Improve EMRs
      Janey Barnes, PhDUser-View, Inc.
      Raleigh, NC
      Jeff Belden, MD Family &Community MedicineUniversity of Missouri
      1
    • 2. Conflict of Interest Disclosure
      Janey Barnes, PhD
      Contracted Research:
      Allscripts
      Patagonia Health
      Duke Health Systems
      2
    • 3. Conflict of Interest Disclosure
      Jeff Belden, MD
      Dr. Belden is on faculty at University of Missouri - Columbia, and an EMR usability consultant with the Tiger Institute, a technology collaborative between the University of Missouri and Cerner Corporation.
      Dr. Belden has no financial interest.
      3
    • 4. Session Objectives
      Describe stages of software development in which usability methods can be most effective.
      List and describe usability methods applicable to each stage of EMR development.
      Analyze the cost/benefit of implementing usability methods within a user-centered design process.
      4
    • 5. Software Development Process
      5
    • 6. Waterfall modelof software development
      6
    • 7. 7
      Waterfall
    • 8. “Agile” modelof software development
      8
    • 9. 9
      Agile
    • 10. Usability methods work for both
      10
    • 11. don’t forget the buyer’s sideof usability
      11
    • 12. 12
      Agile
      Waterfall
      Selection
      Implementation
    • 13. Want to be a change-agent?
      13
    • 14.
    • 15. User Centered Design Process
      15
    • 16. 16
      Incorporating User Centered Designin Software Development Process
    • a few words about usability when selecting & implementing
      17
    • 31. Software Selection Activities
      Usability Methods of Interest
      Site visit for:
      • User surveys of current users like you
      • 32. Performance testing
      • 33. Observe one of their experienced (but not an ace) users
      • 34. Do clinical scenario test with real data
      • 35. Define target measures
      • 36. Set goals
      18
    • 37. Implementation Activities
      Usability Methods of Interest
      Task analysis & user observation
      Card sorting (picking categories, nomenclature)
      Design guidelines & Style guides
      Paper prototyping
      Have clinician create documents for several patients
      19
    • 38.
    • 39. DiscoveryActivities
      Contextual inquiry
      Affinity diagramming
      Personas
      Use case scenarios
      For each method we present
      • Goalof the method
      • 40. Details for planning& executingeach method
      • 41. Best timing the software development schedule
      • 42. How to deliver impactto the product
      21
    • 43. Contextual Inquiry
      Persona
      22
    • 44. Goals
      • understand the user’s:
      • 45. tasks
      • 46. workflows
      • 47. environment
      • 48. description of a specific person (a target user). Details make them real.
      23
      Contextual Inquiry
      Persona
    • 49. Contextual Inquiry
      Persona
      Planning
      Execution
      Observe users onsite
      Be a fly on the wall
      Before, during & after
      Half day…or longer!
      Take photos of environment.
      • Watch. Observe. Learn.
      • 50. Make notes.
      • 51. Stand, sit, to side, in middle.
      • 52. Stay out of way
      • 53. Collect any artifacts.
    • Contextual Inquiry
      Persona
      Timing
      • Early
      • 54. Requirements gathering
      • 55. Before sprint planning
      • 56. Before “iteration 1”
      Having Impact
      • By watching, observing, learning
    • 26
      Contextual Inquiry
      Use Case Scenarios
    • 57. 27
      Contextual Inquiry
      Use Case Scenarios
      Goals
      • Understand the user’s:
      • 58. tasks
      • 59. workflows
      • 60. environment
      • 61. Specify how users carry out their tasks in a specified context.
    • 28
      Contextual Inquiry
      Use Case Scenarios
      Planning
      Execution
      Observe users onsite
      Be a fly on the wall
      Before, during & after
      Half day…or longer!
      Take photos of environment.
      • Watch. Observe. Learn.
      • 62. Make notes.
      • 63. Stand, sit, to side, in middle.
      • 64. Stay out of way
      • 65. Collect any artifacts.
    • 29
      Contextual Inquiry
      Use Case Scenarios
      Timing
      • Early
      • 66. Requirements gathering
      • 67. Before sprint planning
      • 68. Before “iteration 1”
      Having Impact
      • By watching, observing, learning
    • 30
      Contextual Inquiry
      How long should this take?
    • 69. 31
      Contextual Inquiry
      How long should we plan?
      Plan
      • Teach team to observe
      • 70. Not sell, or teach, or fix
      • 71. Have tools ready
    • 32
      Contextual Inquiry
      How long should we stay to observe?
      It depends
      How varied are user groups?
      How familiar are you with territory?
      Stay until you see patterns emerge
    • 72. 33
      Contextual Inquiry
      How long to compile findings?
      Compile into
      Personas
      Use case scenarios
      Work flows
      Task flows
    • 73.
    • 74. Definition Activities
      Requirements
      User Stories
      35
    • 79. Requirements gathering
      Goal
      • Identify usability requirements that can be tested later
      For each chosen task and user type estimate:
      • acceptable task time & optimum target
      • 80. how to score effectiveness - agreeing what errors user might make
      • 81. the effectiveness target
      • 82. the satisfaction target.
      36
    • 83. Requirements gathering
      Planning
      Execution
      Arrange workshop with users, developers.
      Review tasks
      For each task & user type estimate
      • Task time & optimum goal
      • 84. How to score effectiveness
      • 85. Effectiveness target
      • 86. Satisfaction Target
      Decide on usability requirements among:
      Recruiting physicians is challenging.
      37
    • 90. Requirements gathering
      Timing
      • Early
      Impact
      • Shows importance of usability early on
      • 91. Provides concrete objectives & testable criteria for usability
      38
    • 92. User Stories
      39
    • 93. User Stories
      Goal
      • More approachable than a formalized use case
      • 94. Slim, give high-level requirements
      • 95. Just a sentence or three.
      • 96. “A surgeon needs latest evolving lab results while moving through the hospital, without stopping to log-in at a PC”.
      40
    • 97. User Stories
      Planning
      Execution
      Arrange short meeting with customer
      Bring 3x5 cards for each story
      User writes user stories
      3x5 note card
      Rewrite until clear
      Developer may use questions to get user going
      41
    • 98. User Stories
      Timing
      • Early
      Impact
      • Short, quick to generate
      • 99. Low maintenance
      • 100. Keeps contact with user
      • 101. Helps estimate the development work effort
      42
    • 102.
    • 103. Design & User Feedback Activities
      Information Design
      Interaction Design
      • Prototyping
      Visual Design
      • Concept exploration
      44
    • 105. Design & User Feedback Activities
      Prototyping
      45
    • 106. Prototyping
      Goal
      clarify requirements
      create draft of interaction designs & screen designs to be rapidly simulated & tested
      46
    • 107. Prototyping
      Planning
      Arrange workshop with
      47
    • 115. Prototyping
      Execution
      • Concept design
      • 116. Sketch possible approaches
      • 117. See if they meet agree objectives
      • 118. Interaction design
      • 119. Use Post-It notes, writing each suggested screen or activity
      • 120. Group them, name clusters, arrange sequences
      48
    • 121. Prototyping
      Execution
      • Screen design
      • 122. Brainstorm screen designs with users
      • 123. Ask user to carry out a realistic task, pointing among screen sketches
      • 124. Screen testing
      • 125. Mock-up rough designs.
      • 126. Walk thru steps, explaining what happens, or get fancier with paper menus, dialog boxes, etc.
      49
    • 127. Prototyping
      Timing
      Early
      Before code
      Impact
      Find usability problems early before code is written
      Communication between users & designers is promoted
      Paper is cheap & quick
      50
    • 128. Design & User Feedback Activities
      Card sorting
      51
    • 129. Card Sorting
      Goal
      Aids information design
      Discover latent structure in an unsorted list of ideas
      • Example: “Group these smart-phone functions into ones that you need, and ones that would be optional. Then rank the needed ones in order of importance.”
      52
    • 130. Card Sorting
      Planning
      Make separate cards for each statement
      Number cards on back
      Find at least 6 users
      Execution
      Shuffle the deck each time.
      Have users group cards
      Two styles of card sort
      • Closed (you supply category)
      • 131. Open (user names groups)
      • 132. Note results. Include names user gave groups, proximity info or comments user made.
      53
    • 133. Card Sorting
      Timing
      Design phase
      Before coding
      Impact
      Helps discover users’ mental model
      Shows how ideas or concepts should be presented
      54
    • 134. Speaker handoff
    • 135. Design & User Feedback Activities
      Visual Design
      Concept Exploration
      Goal
      • Communicating information organization
      • 136. Communicating information priority
      • 137. Engaging brand
      56
    • 138. Visual Design
      It’s not just pretty colors…
      57
    • 139. Visual Design
      Contrast | Repetition |Alignment |Proximity
      Contrast | Repetition | Alignment | Proximity
      Contrast | Repetition | Alignment | Proximity
      58
    • 140. Visual Design
      Good visual design communicates:
      Info organization
      Info priority
      and reduces cognitive load
      59
    • 141. Design & User Feedback Activities
      Visual Design
      Concept Exploration
      Planning: Iterative Process
      Execution: Research methods to quantify effectiveness of Visual Design
      Timing: In parallel with information & interaction design
      Impact: Critical! Visual Design is not just “make it pretty”
      60
    • 142.
    • 143. Affinity Diagramming
    • 144. Prototyping
      Low-resolution
      High-resolution
    • 145. Design & User Feedback Activities
      Usability Tests
      64
    • 146. Usability Tests
      65
    • 147. Design & User Feedback Activities
      Usability Tests - one-on-one sessions where participant user performs key/at risk tasks while researcher gathers data related to performance
      Goals
      66
    • Usability Tests
      Formative and Summative
      Planning
      Execution
      including data analysis plan
      including test and data analysis
      • Share with team & IMPACT design
      67
    • 160. Usability Tests
      Formative and Summative
      Timing
      Formative
      • Earlier
      • 161. As soon as you have use cases in a prototype
      Summative
      Fit to sprints
      • UX a sprint ahead
      • 165. UX doing formative activities for next sprint and summative activities for current sprint
      68
    • 166. Usability Tests
      Formative and Summative
      • Having Impact
      • 167. Actionable & Prioritized Recommendations
      • 168. Here are 3 examples…
    • Usability Tests
      Actionable & prioritized recommendations
      High Priority
      Users (patients) cannot find Clinician’s name in the list of physicians (appointment task). Put names in alphabetical order
      High Priority
      Users (physicians) cannot find “New Prescription” action button. Make “New Prescription” action button visually apparent.
      High Priority
      Users (physicians) cannot find patient’s name on open chart quickly. Put the patient’s name in a visually distinct location on the screen and be consistent on all screens.
    • 169. Development Activities
      User Acceptance Testing (Software)
      Incorporating user feedback
      Maintaining the intent of the design
      71
      Keep the door open between
      UX and Development!
    • 170. Development Activities
      User Acceptance Testing (Software)
      • Goal: Validate that the application works as intended
      • 171. Planning: Begins during Define Phase
      • 172. Execution: Automatic versus Manual
      • 173. Timing: Nearing the end
      • 174. Impact: Too late to IMPACT the application?
      72
    • 175. Development Activities
      Incorporating user feedback
      Maintaining the intent of the design
      • Goal: SUPPORT development team
      • 176. Planning: Little planning time – rely on process
      • 177. Execution: SUPPORT
      • 178. Timing: Nearing the end
      • 179. Impact: CRITICAL time
      73
    • 180. Beta Testing Activities
      74
      Usability Activities
      during Beta Testing
      Goal
      From Usability Perspective:
      • Live usability test to inform current application and next release
      • 181. System-wide usability test
      • 182. Preparation for implementations
    • Beta Testing Activities
      75
      Usability Activities
      during Beta Testing
      Planning
      Execution
      including data analysis plan
      including test and data analysis
      • Share with team & IMPACT design, implementation
    • Beta Testing Activities
      Usability Activities
      during Beta Testing
      76
      Timing: We are nearing the end
      Impact: Not too late to impact the application
    • 190. Launch Activities
      77
      Continued Testing
      Goal
      • From Usability Perspective:
      • 191. Live usability test to inform current application and next release
      • 192. System-wide usability test
    • Launch Activities
      Continued Testing
      78
      Planning
      Execution
      including data analysis plan
      including test and data analysis
      • Share with team & IMPACT design, implementation
    • Launch Activities
      79
      Continued Testing
      Timing: We are nearing the end
      Impact: Not too late to impact the application
    • 200. 80
      Post-Launch
      Post-Launch
      Post Launch Activities
      Learnings Discover Activities
      Implementation
      Discover
      Define
      Launch
      Selection
      Design
      Beta
      Develop
      Implementation
    • 201. 81
      Cost / Benefit
    • 202. Cost/Benefit
      Key to cost-effective product usability is to
      Plan & Manage
      the usability activities within the development process.
      82
    • 203. Cost/Benefit
      83
      Summative
      Formative
      Formative
      specific activity designed to gather data to address specific research question
    • 218. http://bit.ly/UXmethods
      84
    • 219. 85
      Costs
      Including Usability
      in Development Cycle
      Design Alternatives
      Cost of Changes
      Planning
      Design
      Development
      Release
    • 220. CostsUsability in the Development Cycle 1 of 2
      Discount Usability vs Premium Usability
      Time (labor)
      Employee vs Contractor (expendable)
      Design vs Evaluation
      Test system setup
      Participant recruitment and compensation
      86
    • 221. CostsUsability in the Development Cycle 2 of 2
      Lab equipment & usability lab facility
      Travel expenses
      Planning Time !!
      • Managers must make the time to plan the integration of usability into the development process.
      • 222. Sometimes too busy to stop and determine how to integrate usability during development.
      87
    • 223. Measurable Benefitsof Usability Integration
      Improved productivity
      • Shorter time to complete tasks
      • 224. Fewer errors during tasks
      • 225. Improved user satisfaction
      Lower training costs
      Reduced tech support & documentation costs
      Increased patient safety
      Litigation deterrence
      User preferences
      • Brand loyalty
      • 226. Marketing voice
      88
    • 227. Tradeoffsbetween Usability Costs & Benefits
      Measurable benefits not well documented
      • “Post” data is often not collected due to budget and lack of vision
      • 228. Proprietary data used competitively and not available to the public
      Usability community has made efforts to quantify benefit of Usability interventions
      89
    • 229. Tradeoffsbetween Usability Costs & Benefits
      Case scenarios presented as examples that hopefully demonstrate the cost-benefit tradeoffs across several types of HF activities.
      Data for case scenarios taken from:
      Good Ergonomics Is Good Economics (1996). Hal Hendrick. Proceedings of the HFES 40th.
      Usability Professionals’ Association (UPA) http://www.upassoc.org
      90
    • 230. Best Bang for the BuckKey Usability Activities
      Contextual Inquiry
      Usability Testing (1 on 1) with prototype or early code
      • Enough functionality to test
      • 231. Early enough to make changes
      91
    • 232. Session Objectives
      Describe stages of software development in which usability methods can be most effective.
      • Requirements & Design
      List and describe usability methods applicable to each stage of EMR development.
      • Contextual Inquiry & Usability Testing
      Analyze the cost/benefit of implementing usability methods within a user-centered design process.
      • Early usability methods are most cost effective in terms of planning, carrying out, and incorporating findings in software.
      92
    • 233. Links
      Usability Methods Table | Usability.net
      HIMSS EHR Usability Links | Online resources
    • 234. Questions?
      For further info, contact:Janey Barnes PhD | jbarnes@user-view.comJeff Belden MD | jeffbelden@gmail.com