Usability methods to improve EMRs

4,363 views

Published on

Jeff Belden MD and Janey Barnes PhD co-presented at HIMSS Virtual Conference June 2010. You can hear the audio recording if you are a HIMSS member, available online.

1 Comment
10 Likes
Statistics
Notes
  • Dear uploaded,

    I need to download this presentation. It's really a good presentation. I really appreciate your effort. I'll be thankful to you if you let me download this presentation. Thanks a million.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,363
On SlideShare
0
From Embeds
0
Number of Embeds
18
Actions
Shares
0
Downloads
194
Comments
1
Likes
10
Embeds 0
No embeds

No notes for slide
  • Note to Janey - if you conduct the wrong activity for the specific research question = trouble
  • Usability methods to improve EMRs

    1. 1. June 9, 2010<br />Usability Methodsto Improve EMRs<br />Janey Barnes, PhDUser-View, Inc.<br />Raleigh, NC<br />Jeff Belden, MD Family &Community MedicineUniversity of Missouri<br />1<br />
    2. 2. Conflict of Interest Disclosure<br />Janey Barnes, PhD<br />Contracted Research: <br />Allscripts<br />Patagonia Health<br />Duke Health Systems<br />2<br />
    3. 3. Conflict of Interest Disclosure<br />Jeff Belden, MD <br />Dr. Belden is on faculty at University of Missouri - Columbia, and an EMR usability consultant with the Tiger Institute, a technology collaborative between the University of Missouri and Cerner Corporation.<br />Dr. Belden has no financial interest.<br />3<br />
    4. 4. Session Objectives<br />Describe stages of software development in which usability methods can be most effective.<br />List and describe usability methods applicable to each stage of EMR development.<br />Analyze the cost/benefit of implementing usability methods within a user-centered design process.<br />4<br />
    5. 5. Software Development Process<br />5<br />
    6. 6. Waterfall modelof software development<br />6<br />
    7. 7. 7<br />Waterfall<br />
    8. 8. “Agile” modelof software development<br />8<br />
    9. 9. 9<br />Agile<br />
    10. 10. Usability methods work for both<br />10<br />
    11. 11. don’t forget the buyer’s sideof usability<br />11<br />
    12. 12. 12<br />Agile<br />Waterfall<br />Selection<br />Implementation<br />
    13. 13. Want to be a change-agent?<br />13<br />
    14. 14.
    15. 15. User Centered Design Process<br />15<br />
    16. 16. 16<br />Incorporating User Centered Designin Software Development Process<br /><ul><li>Prototyping
    17. 17. Card sorting
    18. 18. Concept exploration
    19. 19. Usability Testing
    20. 20. and testing
    21. 21. Keep testing
    22. 22. Requirements
    23. 23. User stories
    24. 24. Summative testing
    25. 25. Affinity Diagramming
    26. 26. Personas
    27. 27. Use case scenarios
    28. 28. User acceptance testing
    29. 29. Incorporating feedback
    30. 30. Maintain design intent</li></li></ul><li>a few words about usability when selecting & implementing<br />17<br />
    31. 31. Software Selection Activities<br />Usability Methods of Interest<br />Site visit for:<br /><ul><li>User surveys of current users like you
    32. 32. Performance testing
    33. 33. Observe one of their experienced (but not an ace) users
    34. 34. Do clinical scenario test with real data
    35. 35. Define target measures
    36. 36. Set goals</li></ul>18<br />
    37. 37. Implementation Activities<br />Usability Methods of Interest<br />Task analysis & user observation<br />Card sorting (picking categories, nomenclature)<br />Design guidelines & Style guides<br />Paper prototyping<br />Have clinician create documents for several patients<br />19<br />
    38. 38.
    39. 39. DiscoveryActivities<br />Contextual inquiry<br />Affinity diagramming<br />Personas<br />Use case scenarios<br />For each method we present<br /><ul><li>Goalof the method
    40. 40. Details for planning& executingeach method
    41. 41. Best timing the software development schedule
    42. 42. How to deliver impactto the product</li></ul>21<br />
    43. 43. Contextual Inquiry<br />Persona<br />22<br />
    44. 44. Goals<br /><ul><li>understand the user’s:
    45. 45. tasks
    46. 46. workflows
    47. 47. environment
    48. 48. description of a specific person (a target user). Details make them real.</li></ul>23<br />Contextual Inquiry<br />Persona<br />
    49. 49. Contextual Inquiry<br />Persona<br />Planning <br />Execution<br />Observe users onsite<br />Be a fly on the wall<br />Before, during & after<br />Half day…or longer!<br />Take photos of environment. <br /><ul><li>Watch. Observe. Learn.
    50. 50. Make notes.
    51. 51. Stand, sit, to side, in middle.
    52. 52. Stay out of way
    53. 53. Collect any artifacts. </li></li></ul><li>Contextual Inquiry<br />Persona<br />Timing<br /><ul><li>Early
    54. 54. Requirements gathering
    55. 55. Before sprint planning
    56. 56. Before “iteration 1”</li></ul>Having Impact<br /><ul><li>By watching, observing, learning</li></li></ul><li>26<br />Contextual Inquiry<br />Use Case Scenarios<br />
    57. 57. 27<br />Contextual Inquiry<br />Use Case Scenarios<br />Goals<br /><ul><li>Understand the user’s:
    58. 58. tasks
    59. 59. workflows
    60. 60. environment
    61. 61. Specify how users carry out their tasks in a specified context.</li></li></ul><li>28<br />Contextual Inquiry<br />Use Case Scenarios<br />Planning <br />Execution<br />Observe users onsite<br />Be a fly on the wall<br />Before, during & after<br />Half day…or longer!<br />Take photos of environment. <br /><ul><li>Watch. Observe. Learn.
    62. 62. Make notes.
    63. 63. Stand, sit, to side, in middle.
    64. 64. Stay out of way
    65. 65. Collect any artifacts. </li></li></ul><li>29<br />Contextual Inquiry<br />Use Case Scenarios<br />Timing<br /><ul><li>Early
    66. 66. Requirements gathering
    67. 67. Before sprint planning
    68. 68. Before “iteration 1”</li></ul>Having Impact<br /><ul><li>By watching, observing, learning</li></li></ul><li>30<br />Contextual Inquiry<br />How long should this take?<br />
    69. 69. 31<br />Contextual Inquiry<br />How long should we plan?<br />Plan<br /><ul><li>Teach team to observe
    70. 70. Not sell, or teach, or fix
    71. 71. Have tools ready</li></li></ul><li>32<br />Contextual Inquiry<br />How long should we stay to observe?<br />It depends<br />How varied are user groups?<br />How familiar are you with territory?<br />Stay until you see patterns emerge<br />
    72. 72. 33<br />Contextual Inquiry<br />How long to compile findings?<br />Compile into<br />Personas<br />Use case scenarios<br />Work flows<br />Task flows<br />
    73. 73.
    74. 74. Definition Activities<br />Requirements <br />User Stories<br /><ul><li>Goal
    75. 75. Planning
    76. 76. Execution
    77. 77. Timing
    78. 78. Impact</li></ul>35<br />
    79. 79. Requirements gathering<br />Goal<br /><ul><li>Identify usability requirements that can be tested later</li></ul>For each chosen task and user type estimate:<br /><ul><li>acceptable task time & optimum target
    80. 80. how to score effectiveness - agreeing what errors user might make
    81. 81. the effectiveness target
    82. 82. the satisfaction target.</li></ul>36<br />
    83. 83. Requirements gathering<br />Planning<br />Execution<br />Arrange workshop with users, developers.<br />Review tasks<br />For each task & user type estimate<br /><ul><li>Task time & optimum goal
    84. 84. How to score effectiveness
    85. 85. Effectiveness target
    86. 86. Satisfaction Target</li></ul>Decide on usability requirements among:<br /><ul><li>Understandability
    87. 87. Learnability
    88. 88. Operability
    89. 89. Attractiveness</li></ul>Recruiting physicians is challenging.<br />37<br />
    90. 90. Requirements gathering<br />Timing<br /><ul><li>Early</li></ul>Impact<br /><ul><li>Shows importance of usability early on
    91. 91. Provides concrete objectives & testable criteria for usability</li></ul>38<br />
    92. 92. User Stories<br />39<br />
    93. 93. User Stories<br />Goal<br /><ul><li>More approachable than a formalized use case
    94. 94. Slim, give high-level requirements
    95. 95. Just a sentence or three.
    96. 96. “A surgeon needs latest evolving lab results while moving through the hospital, without stopping to log-in at a PC”.</li></ul>40<br />
    97. 97. User Stories<br />Planning<br />Execution<br />Arrange short meeting with customer<br />Bring 3x5 cards for each story<br />User writes user stories<br />3x5 note card<br />Rewrite until clear<br />Developer may use questions to get user going<br />41<br />
    98. 98. User Stories<br />Timing<br /><ul><li>Early</li></ul>Impact<br /><ul><li>Short, quick to generate
    99. 99. Low maintenance
    100. 100. Keeps contact with user
    101. 101. Helps estimate the development work effort</li></ul>42<br />
    102. 102.
    103. 103. Design & User Feedback Activities<br />Information Design<br /><ul><li>Prototyping
    104. 104. Card sorting</li></ul>Interaction Design<br /><ul><li>Prototyping</li></ul>Visual Design<br /><ul><li>Concept exploration</li></ul>44<br />
    105. 105. Design & User Feedback Activities<br />Prototyping<br />45<br />
    106. 106. Prototyping<br />Goal<br />clarify requirements<br />create draft of interaction designs & screen designs to be rapidly simulated & tested<br />46<br />
    107. 107. Prototyping<br />Planning<br />Arrange workshop with<br /><ul><li>Users
    108. 108. Developers
    109. 109. Facilitator & scribe
    110. 110. Four stages
    111. 111. Concept design
    112. 112. Interaction design
    113. 113. Screen design
    114. 114. Screen testing</li></ul>47<br />
    115. 115. Prototyping<br />Execution<br /><ul><li>Concept design
    116. 116. Sketch possible approaches
    117. 117. See if they meet agree objectives
    118. 118. Interaction design
    119. 119. Use Post-It notes, writing each suggested screen or activity
    120. 120. Group them, name clusters, arrange sequences</li></ul>48<br />
    121. 121. Prototyping<br />Execution<br /><ul><li>Screen design
    122. 122. Brainstorm screen designs with users
    123. 123. Ask user to carry out a realistic task, pointing among screen sketches
    124. 124. Screen testing
    125. 125. Mock-up rough designs.
    126. 126. Walk thru steps, explaining what happens, or get fancier with paper menus, dialog boxes, etc.</li></ul>49<br />
    127. 127. Prototyping<br />Timing<br />Early<br />Before code<br />Impact<br />Find usability problems early before code is written<br />Communication between users & designers is promoted<br />Paper is cheap & quick<br />50<br />
    128. 128. Design & User Feedback Activities<br />Card sorting<br />51<br />
    129. 129. Card Sorting<br />Goal<br />Aids information design<br />Discover latent structure in an unsorted list of ideas<br /><ul><li>Example: “Group these smart-phone functions into ones that you need, and ones that would be optional. Then rank the needed ones in order of importance.”</li></ul>52<br />
    130. 130. Card Sorting<br />Planning<br />Make separate cards for each statement<br />Number cards on back<br />Find at least 6 users<br />Execution<br />Shuffle the deck each time.<br />Have users group cards<br />Two styles of card sort<br /><ul><li>Closed (you supply category)
    131. 131. Open (user names groups)
    132. 132. Note results. Include names user gave groups, proximity info or comments user made.</li></ul>53<br />
    133. 133. Card Sorting<br />Timing<br />Design phase<br />Before coding<br />Impact<br />Helps discover users’ mental model<br />Shows how ideas or concepts should be presented<br />54<br />
    134. 134. Speaker handoff<br />
    135. 135. Design & User Feedback Activities<br />Visual Design<br />Concept Exploration <br />Goal<br /><ul><li>Communicating information organization
    136. 136. Communicating information priority
    137. 137. Engaging brand</li></ul>56<br />
    138. 138. Visual Design<br />It’s not just pretty colors…<br />57<br />
    139. 139. Visual Design<br />Contrast | Repetition |Alignment |Proximity<br />Contrast | Repetition | Alignment | Proximity<br />Contrast | Repetition | Alignment | Proximity<br />58<br />
    140. 140. Visual Design<br />Good visual design communicates: <br />Info organization<br />Info priority<br />and reduces cognitive load<br />59<br />
    141. 141. Design & User Feedback Activities<br />Visual Design<br />Concept Exploration <br />Planning: Iterative Process<br />Execution: Research methods to quantify effectiveness of Visual Design <br />Timing: In parallel with information & interaction design<br />Impact: Critical! Visual Design is not just “make it pretty”<br />60<br />
    142. 142.
    143. 143. Affinity Diagramming<br />
    144. 144. Prototyping<br />Low-resolution<br />High-resolution<br />
    145. 145. Design & User Feedback Activities<br />Usability Tests<br />64<br />
    146. 146. Usability Tests<br />65<br />
    147. 147. Design & User Feedback Activities<br />Usability Tests - one-on-one sessions where participant user performs key/at risk tasks while researcher gathers data related to performance <br />Goals<br />66<br /><ul><li> Formative
    148. 148. Design decisions
    149. 149. Identify usability issues
    150. 150. Summative
    151. 151. Evaluate usability goals
    152. 152. Validate design</li></li></ul><li>Usability Tests<br />Formative and Summative<br />Planning <br />Execution<br /><ul><li>Test Objectives
    153. 153. Test Design
    154. 154. Test Plan </li></ul>including data analysis plan<br /><ul><li> Test Participants
    155. 155. Test Environment
    156. 156. Test Report
    157. 157. Pilot Test
    158. 158. Logistics
    159. 159. Carry out plan</li></ul> including test and data analysis<br /><ul><li>Share with team & IMPACT design</li></ul>67<br />
    160. 160. Usability Tests<br />Formative and Summative<br />Timing<br />Formative<br /><ul><li> Earlier
    161. 161. As soon as you have use cases in a prototype</li></ul>Summative<br /><ul><li> Later
    162. 162. Enough
    163. 163. Stability
    164. 164. Early enough to make IMPACT</li></ul>Fit to sprints<br /><ul><li> UX a sprint ahead
    165. 165. UX doing formative activities for next sprint and summative activities for current sprint </li></ul>68<br />
    166. 166. Usability Tests<br />Formative and Summative<br /><ul><li>Having Impact
    167. 167. Actionable & Prioritized Recommendations
    168. 168. Here are 3 examples…</li></li></ul><li>Usability Tests<br />Actionable & prioritized recommendations<br />High Priority<br />Users (patients) cannot find Clinician’s name in the list of physicians (appointment task). Put names in alphabetical order<br />High Priority<br />Users (physicians) cannot find “New Prescription” action button. Make “New Prescription” action button visually apparent.<br />High Priority<br />Users (physicians) cannot find patient’s name on open chart quickly. Put the patient’s name in a visually distinct location on the screen and be consistent on all screens.<br />
    169. 169. Development Activities<br />User Acceptance Testing (Software)<br />Incorporating user feedback<br />Maintaining the intent of the design<br />71<br />Keep the door open between<br />UX and Development!<br />
    170. 170. Development Activities<br />User Acceptance Testing (Software)<br /><ul><li>Goal: Validate that the application works as intended
    171. 171. Planning: Begins during Define Phase
    172. 172. Execution: Automatic versus Manual
    173. 173. Timing: Nearing the end
    174. 174. Impact: Too late to IMPACT the application?</li></ul>72<br />
    175. 175. Development Activities<br />Incorporating user feedback<br />Maintaining the intent of the design<br /><ul><li>Goal: SUPPORT development team
    176. 176. Planning: Little planning time – rely on process
    177. 177. Execution: SUPPORT
    178. 178. Timing: Nearing the end
    179. 179. Impact: CRITICAL time</li></ul>73<br />
    180. 180. Beta Testing Activities<br />74<br />Usability Activities<br />during Beta Testing<br />Goal<br />From Usability Perspective:<br /><ul><li>Live usability test to inform current application and next release
    181. 181. System-wide usability test
    182. 182. Preparation for implementations</li></li></ul><li>Beta Testing Activities<br />75<br />Usability Activities<br />during Beta Testing<br />Planning <br />Execution<br /><ul><li>Test Objectives
    183. 183. Test Design
    184. 184. Test Plan </li></ul>including data analysis plan<br /><ul><li> Test Participants
    185. 185. Test Environment
    186. 186. Test Report
    187. 187. Pilot Test
    188. 188. Logistics
    189. 189. Carry out plan</li></ul>including test and data analysis<br /><ul><li>Share with team & IMPACT design, implementation</li></li></ul><li>Beta Testing Activities<br />Usability Activities<br />during Beta Testing<br />76<br />Timing: We are nearing the end<br />Impact: Not too late to impact the application<br />
    190. 190. Launch Activities<br />77<br />Continued Testing<br />Goal<br /><ul><li>From Usability Perspective:
    191. 191. Live usability test to inform current application and next release
    192. 192. System-wide usability test </li></li></ul><li>Launch Activities<br />Continued Testing<br />78<br />Planning <br />Execution<br /><ul><li>Test Objectives
    193. 193. Test Design
    194. 194. Test Plan </li></ul>including data analysis plan<br /><ul><li> Test Participants
    195. 195. Test Environment
    196. 196. Test Report
    197. 197. Pilot Test
    198. 198. Logistics
    199. 199. Carry out plan</li></ul>including test and data analysis<br /><ul><li>Share with team & IMPACT design, implementation</li></li></ul><li>Launch Activities<br />79<br />Continued Testing<br />Timing: We are nearing the end<br />Impact: Not too late to impact the application<br />
    200. 200. 80<br />Post-Launch<br />Post-Launch<br />Post Launch Activities<br />Learnings Discover Activities<br />Implementation<br />Discover<br />Define<br />Launch<br />Selection<br />Design<br />Beta<br />Develop<br />Implementation<br />
    201. 201. 81<br />Cost / Benefit<br />
    202. 202. Cost/Benefit<br />Key to cost-effective product usability is to<br />Plan & Manage<br />the usability activities within the development process. <br />82<br />
    203. 203. Cost/Benefit<br />83<br />Summative<br />Formative <br />Formative <br /><ul><li>Prototyping
    204. 204. Card sorting
    205. 205. Concept exploration
    206. 206. Usability Testing
    207. 207. and testing
    208. 208. Keep testing
    209. 209. Requirements
    210. 210. User stories
    211. 211. Summative testing
    212. 212. User acceptance testing
    213. 213. Incorporating feedback
    214. 214. Maintain design intent
    215. 215. Affinity Diagramming
    216. 216. Personas
    217. 217. Use case scenarios</li></ul>specific activity designed to gather data to address specific research question<br />
    218. 218. http://bit.ly/UXmethods<br />84<br />
    219. 219. 85<br />Costs<br />Including Usability<br />in Development Cycle<br />Design Alternatives<br />Cost of Changes<br />Planning<br />Design<br />Development<br />Release<br />
    220. 220. CostsUsability in the Development Cycle 1 of 2<br />Discount Usability vs Premium Usability<br />Time (labor)<br />Employee vs Contractor (expendable)<br />Design vs Evaluation<br />Test system setup<br />Participant recruitment and compensation<br />86<br />
    221. 221. CostsUsability in the Development Cycle 2 of 2<br />Lab equipment & usability lab facility<br />Travel expenses<br />Planning Time !!<br /><ul><li>Managers must make the time to plan the integration of usability into the development process.
    222. 222. Sometimes too busy to stop and determine how to integrate usability during development.</li></ul>87<br />
    223. 223. Measurable Benefitsof Usability Integration<br />Improved productivity<br /><ul><li>Shorter time to complete tasks
    224. 224. Fewer errors during tasks
    225. 225. Improved user satisfaction</li></ul>Lower training costs<br />Reduced tech support & documentation costs<br />Increased patient safety<br />Litigation deterrence<br />User preferences<br /><ul><li>Brand loyalty
    226. 226. Marketing voice</li></ul>88<br />
    227. 227. Tradeoffsbetween Usability Costs & Benefits<br />Measurable benefits not well documented<br /><ul><li>“Post” data is often not collected due to budget and lack of vision
    228. 228. Proprietary data used competitively and not available to the public</li></ul>Usability community has made efforts to quantify benefit of Usability interventions<br />89<br />
    229. 229. Tradeoffsbetween Usability Costs & Benefits<br />Case scenarios presented as examples that hopefully demonstrate the cost-benefit tradeoffs across several types of HF activities.<br />Data for case scenarios taken from:<br />Good Ergonomics Is Good Economics (1996). Hal Hendrick. Proceedings of the HFES 40th.<br />Usability Professionals’ Association (UPA) http://www.upassoc.org<br />90<br />
    230. 230. Best Bang for the BuckKey Usability Activities<br />Contextual Inquiry<br />Usability Testing (1 on 1) with prototype or early code<br /><ul><li>Enough functionality to test
    231. 231. Early enough to make changes</li></ul>91<br />
    232. 232. Session Objectives<br />Describe stages of software development in which usability methods can be most effective.<br /><ul><li>Requirements & Design</li></ul>List and describe usability methods applicable to each stage of EMR development.<br /><ul><li>Contextual Inquiry & Usability Testing</li></ul>Analyze the cost/benefit of implementing usability methods within a user-centered design process.<br /><ul><li>Early usability methods are most cost effective in terms of planning, carrying out, and incorporating findings in software.</li></ul>92<br />
    233. 233. Links<br />Usability Methods Table | Usability.net<br />HIMSS EHR Usability Links | Online resources<br />
    234. 234. Questions?<br />For further info, contact:Janey Barnes PhD | jbarnes@user-view.comJeff Belden MD | jeffbelden@gmail.com<br />

    ×