Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Usability methods to improve EMRs

4,506 views

Published on

Jeff Belden MD and Janey Barnes PhD co-presented at HIMSS Virtual Conference June 2010. You can hear the audio recording if you are a HIMSS member, available online.

  • Dear uploaded,

    I need to download this presentation. It's really a good presentation. I really appreciate your effort. I'll be thankful to you if you let me download this presentation. Thanks a million.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Usability methods to improve EMRs

  1. 1. June 9, 2010<br />Usability Methodsto Improve EMRs<br />Janey Barnes, PhDUser-View, Inc.<br />Raleigh, NC<br />Jeff Belden, MD Family &Community MedicineUniversity of Missouri<br />1<br />
  2. 2. Conflict of Interest Disclosure<br />Janey Barnes, PhD<br />Contracted Research: <br />Allscripts<br />Patagonia Health<br />Duke Health Systems<br />2<br />
  3. 3. Conflict of Interest Disclosure<br />Jeff Belden, MD <br />Dr. Belden is on faculty at University of Missouri - Columbia, and an EMR usability consultant with the Tiger Institute, a technology collaborative between the University of Missouri and Cerner Corporation.<br />Dr. Belden has no financial interest.<br />3<br />
  4. 4. Session Objectives<br />Describe stages of software development in which usability methods can be most effective.<br />List and describe usability methods applicable to each stage of EMR development.<br />Analyze the cost/benefit of implementing usability methods within a user-centered design process.<br />4<br />
  5. 5. Software Development Process<br />5<br />
  6. 6. Waterfall modelof software development<br />6<br />
  7. 7. 7<br />Waterfall<br />
  8. 8. “Agile” modelof software development<br />8<br />
  9. 9. 9<br />Agile<br />
  10. 10. Usability methods work for both<br />10<br />
  11. 11. don’t forget the buyer’s sideof usability<br />11<br />
  12. 12. 12<br />Agile<br />Waterfall<br />Selection<br />Implementation<br />
  13. 13. Want to be a change-agent?<br />13<br />
  14. 14.
  15. 15. User Centered Design Process<br />15<br />
  16. 16. 16<br />Incorporating User Centered Designin Software Development Process<br /><ul><li>Prototyping
  17. 17. Card sorting
  18. 18. Concept exploration
  19. 19. Usability Testing
  20. 20. and testing
  21. 21. Keep testing
  22. 22. Requirements
  23. 23. User stories
  24. 24. Summative testing
  25. 25. Affinity Diagramming
  26. 26. Personas
  27. 27. Use case scenarios
  28. 28. User acceptance testing
  29. 29. Incorporating feedback
  30. 30. Maintain design intent</li></li></ul><li>a few words about usability when selecting & implementing<br />17<br />
  31. 31. Software Selection Activities<br />Usability Methods of Interest<br />Site visit for:<br /><ul><li>User surveys of current users like you
  32. 32. Performance testing
  33. 33. Observe one of their experienced (but not an ace) users
  34. 34. Do clinical scenario test with real data
  35. 35. Define target measures
  36. 36. Set goals</li></ul>18<br />
  37. 37. Implementation Activities<br />Usability Methods of Interest<br />Task analysis & user observation<br />Card sorting (picking categories, nomenclature)<br />Design guidelines & Style guides<br />Paper prototyping<br />Have clinician create documents for several patients<br />19<br />
  38. 38.
  39. 39. DiscoveryActivities<br />Contextual inquiry<br />Affinity diagramming<br />Personas<br />Use case scenarios<br />For each method we present<br /><ul><li>Goalof the method
  40. 40. Details for planning& executingeach method
  41. 41. Best timing the software development schedule
  42. 42. How to deliver impactto the product</li></ul>21<br />
  43. 43. Contextual Inquiry<br />Persona<br />22<br />
  44. 44. Goals<br /><ul><li>understand the user’s:
  45. 45. tasks
  46. 46. workflows
  47. 47. environment
  48. 48. description of a specific person (a target user). Details make them real.</li></ul>23<br />Contextual Inquiry<br />Persona<br />
  49. 49. Contextual Inquiry<br />Persona<br />Planning <br />Execution<br />Observe users onsite<br />Be a fly on the wall<br />Before, during & after<br />Half day…or longer!<br />Take photos of environment. <br /><ul><li>Watch. Observe. Learn.
  50. 50. Make notes.
  51. 51. Stand, sit, to side, in middle.
  52. 52. Stay out of way
  53. 53. Collect any artifacts. </li></li></ul><li>Contextual Inquiry<br />Persona<br />Timing<br /><ul><li>Early
  54. 54. Requirements gathering
  55. 55. Before sprint planning
  56. 56. Before “iteration 1”</li></ul>Having Impact<br /><ul><li>By watching, observing, learning</li></li></ul><li>26<br />Contextual Inquiry<br />Use Case Scenarios<br />
  57. 57. 27<br />Contextual Inquiry<br />Use Case Scenarios<br />Goals<br /><ul><li>Understand the user’s:
  58. 58. tasks
  59. 59. workflows
  60. 60. environment
  61. 61. Specify how users carry out their tasks in a specified context.</li></li></ul><li>28<br />Contextual Inquiry<br />Use Case Scenarios<br />Planning <br />Execution<br />Observe users onsite<br />Be a fly on the wall<br />Before, during & after<br />Half day…or longer!<br />Take photos of environment. <br /><ul><li>Watch. Observe. Learn.
  62. 62. Make notes.
  63. 63. Stand, sit, to side, in middle.
  64. 64. Stay out of way
  65. 65. Collect any artifacts. </li></li></ul><li>29<br />Contextual Inquiry<br />Use Case Scenarios<br />Timing<br /><ul><li>Early
  66. 66. Requirements gathering
  67. 67. Before sprint planning
  68. 68. Before “iteration 1”</li></ul>Having Impact<br /><ul><li>By watching, observing, learning</li></li></ul><li>30<br />Contextual Inquiry<br />How long should this take?<br />
  69. 69. 31<br />Contextual Inquiry<br />How long should we plan?<br />Plan<br /><ul><li>Teach team to observe
  70. 70. Not sell, or teach, or fix
  71. 71. Have tools ready</li></li></ul><li>32<br />Contextual Inquiry<br />How long should we stay to observe?<br />It depends<br />How varied are user groups?<br />How familiar are you with territory?<br />Stay until you see patterns emerge<br />
  72. 72. 33<br />Contextual Inquiry<br />How long to compile findings?<br />Compile into<br />Personas<br />Use case scenarios<br />Work flows<br />Task flows<br />
  73. 73.
  74. 74. Definition Activities<br />Requirements <br />User Stories<br /><ul><li>Goal
  75. 75. Planning
  76. 76. Execution
  77. 77. Timing
  78. 78. Impact</li></ul>35<br />
  79. 79. Requirements gathering<br />Goal<br /><ul><li>Identify usability requirements that can be tested later</li></ul>For each chosen task and user type estimate:<br /><ul><li>acceptable task time & optimum target
  80. 80. how to score effectiveness - agreeing what errors user might make
  81. 81. the effectiveness target
  82. 82. the satisfaction target.</li></ul>36<br />
  83. 83. Requirements gathering<br />Planning<br />Execution<br />Arrange workshop with users, developers.<br />Review tasks<br />For each task & user type estimate<br /><ul><li>Task time & optimum goal
  84. 84. How to score effectiveness
  85. 85. Effectiveness target
  86. 86. Satisfaction Target</li></ul>Decide on usability requirements among:<br /><ul><li>Understandability
  87. 87. Learnability
  88. 88. Operability
  89. 89. Attractiveness</li></ul>Recruiting physicians is challenging.<br />37<br />
  90. 90. Requirements gathering<br />Timing<br /><ul><li>Early</li></ul>Impact<br /><ul><li>Shows importance of usability early on
  91. 91. Provides concrete objectives & testable criteria for usability</li></ul>38<br />
  92. 92. User Stories<br />39<br />
  93. 93. User Stories<br />Goal<br /><ul><li>More approachable than a formalized use case
  94. 94. Slim, give high-level requirements
  95. 95. Just a sentence or three.
  96. 96. “A surgeon needs latest evolving lab results while moving through the hospital, without stopping to log-in at a PC”.</li></ul>40<br />
  97. 97. User Stories<br />Planning<br />Execution<br />Arrange short meeting with customer<br />Bring 3x5 cards for each story<br />User writes user stories<br />3x5 note card<br />Rewrite until clear<br />Developer may use questions to get user going<br />41<br />
  98. 98. User Stories<br />Timing<br /><ul><li>Early</li></ul>Impact<br /><ul><li>Short, quick to generate
  99. 99. Low maintenance
  100. 100. Keeps contact with user
  101. 101. Helps estimate the development work effort</li></ul>42<br />
  102. 102.
  103. 103. Design & User Feedback Activities<br />Information Design<br /><ul><li>Prototyping
  104. 104. Card sorting</li></ul>Interaction Design<br /><ul><li>Prototyping</li></ul>Visual Design<br /><ul><li>Concept exploration</li></ul>44<br />
  105. 105. Design & User Feedback Activities<br />Prototyping<br />45<br />
  106. 106. Prototyping<br />Goal<br />clarify requirements<br />create draft of interaction designs & screen designs to be rapidly simulated & tested<br />46<br />
  107. 107. Prototyping<br />Planning<br />Arrange workshop with<br /><ul><li>Users
  108. 108. Developers
  109. 109. Facilitator & scribe
  110. 110. Four stages
  111. 111. Concept design
  112. 112. Interaction design
  113. 113. Screen design
  114. 114. Screen testing</li></ul>47<br />
  115. 115. Prototyping<br />Execution<br /><ul><li>Concept design
  116. 116. Sketch possible approaches
  117. 117. See if they meet agree objectives
  118. 118. Interaction design
  119. 119. Use Post-It notes, writing each suggested screen or activity
  120. 120. Group them, name clusters, arrange sequences</li></ul>48<br />
  121. 121. Prototyping<br />Execution<br /><ul><li>Screen design
  122. 122. Brainstorm screen designs with users
  123. 123. Ask user to carry out a realistic task, pointing among screen sketches
  124. 124. Screen testing
  125. 125. Mock-up rough designs.
  126. 126. Walk thru steps, explaining what happens, or get fancier with paper menus, dialog boxes, etc.</li></ul>49<br />
  127. 127. Prototyping<br />Timing<br />Early<br />Before code<br />Impact<br />Find usability problems early before code is written<br />Communication between users & designers is promoted<br />Paper is cheap & quick<br />50<br />
  128. 128. Design & User Feedback Activities<br />Card sorting<br />51<br />
  129. 129. Card Sorting<br />Goal<br />Aids information design<br />Discover latent structure in an unsorted list of ideas<br /><ul><li>Example: “Group these smart-phone functions into ones that you need, and ones that would be optional. Then rank the needed ones in order of importance.”</li></ul>52<br />
  130. 130. Card Sorting<br />Planning<br />Make separate cards for each statement<br />Number cards on back<br />Find at least 6 users<br />Execution<br />Shuffle the deck each time.<br />Have users group cards<br />Two styles of card sort<br /><ul><li>Closed (you supply category)
  131. 131. Open (user names groups)
  132. 132. Note results. Include names user gave groups, proximity info or comments user made.</li></ul>53<br />
  133. 133. Card Sorting<br />Timing<br />Design phase<br />Before coding<br />Impact<br />Helps discover users’ mental model<br />Shows how ideas or concepts should be presented<br />54<br />
  134. 134. Speaker handoff<br />
  135. 135. Design & User Feedback Activities<br />Visual Design<br />Concept Exploration <br />Goal<br /><ul><li>Communicating information organization
  136. 136. Communicating information priority
  137. 137. Engaging brand</li></ul>56<br />
  138. 138. Visual Design<br />It’s not just pretty colors…<br />57<br />
  139. 139. Visual Design<br />Contrast | Repetition |Alignment |Proximity<br />Contrast | Repetition | Alignment | Proximity<br />Contrast | Repetition | Alignment | Proximity<br />58<br />
  140. 140. Visual Design<br />Good visual design communicates: <br />Info organization<br />Info priority<br />and reduces cognitive load<br />59<br />
  141. 141. Design & User Feedback Activities<br />Visual Design<br />Concept Exploration <br />Planning: Iterative Process<br />Execution: Research methods to quantify effectiveness of Visual Design <br />Timing: In parallel with information & interaction design<br />Impact: Critical! Visual Design is not just “make it pretty”<br />60<br />
  142. 142.
  143. 143. Affinity Diagramming<br />
  144. 144. Prototyping<br />Low-resolution<br />High-resolution<br />
  145. 145. Design & User Feedback Activities<br />Usability Tests<br />64<br />
  146. 146. Usability Tests<br />65<br />
  147. 147. Design & User Feedback Activities<br />Usability Tests - one-on-one sessions where participant user performs key/at risk tasks while researcher gathers data related to performance <br />Goals<br />66<br /><ul><li> Formative
  148. 148. Design decisions
  149. 149. Identify usability issues
  150. 150. Summative
  151. 151. Evaluate usability goals
  152. 152. Validate design</li></li></ul><li>Usability Tests<br />Formative and Summative<br />Planning <br />Execution<br /><ul><li>Test Objectives
  153. 153. Test Design
  154. 154. Test Plan </li></ul>including data analysis plan<br /><ul><li> Test Participants
  155. 155. Test Environment
  156. 156. Test Report
  157. 157. Pilot Test
  158. 158. Logistics
  159. 159. Carry out plan</li></ul> including test and data analysis<br /><ul><li>Share with team & IMPACT design</li></ul>67<br />
  160. 160. Usability Tests<br />Formative and Summative<br />Timing<br />Formative<br /><ul><li> Earlier
  161. 161. As soon as you have use cases in a prototype</li></ul>Summative<br /><ul><li> Later
  162. 162. Enough
  163. 163. Stability
  164. 164. Early enough to make IMPACT</li></ul>Fit to sprints<br /><ul><li> UX a sprint ahead
  165. 165. UX doing formative activities for next sprint and summative activities for current sprint </li></ul>68<br />
  166. 166. Usability Tests<br />Formative and Summative<br /><ul><li>Having Impact
  167. 167. Actionable & Prioritized Recommendations
  168. 168. Here are 3 examples…</li></li></ul><li>Usability Tests<br />Actionable & prioritized recommendations<br />High Priority<br />Users (patients) cannot find Clinician’s name in the list of physicians (appointment task). Put names in alphabetical order<br />High Priority<br />Users (physicians) cannot find “New Prescription” action button. Make “New Prescription” action button visually apparent.<br />High Priority<br />Users (physicians) cannot find patient’s name on open chart quickly. Put the patient’s name in a visually distinct location on the screen and be consistent on all screens.<br />
  169. 169. Development Activities<br />User Acceptance Testing (Software)<br />Incorporating user feedback<br />Maintaining the intent of the design<br />71<br />Keep the door open between<br />UX and Development!<br />
  170. 170. Development Activities<br />User Acceptance Testing (Software)<br /><ul><li>Goal: Validate that the application works as intended
  171. 171. Planning: Begins during Define Phase
  172. 172. Execution: Automatic versus Manual
  173. 173. Timing: Nearing the end
  174. 174. Impact: Too late to IMPACT the application?</li></ul>72<br />
  175. 175. Development Activities<br />Incorporating user feedback<br />Maintaining the intent of the design<br /><ul><li>Goal: SUPPORT development team
  176. 176. Planning: Little planning time – rely on process
  177. 177. Execution: SUPPORT
  178. 178. Timing: Nearing the end
  179. 179. Impact: CRITICAL time</li></ul>73<br />
  180. 180. Beta Testing Activities<br />74<br />Usability Activities<br />during Beta Testing<br />Goal<br />From Usability Perspective:<br /><ul><li>Live usability test to inform current application and next release
  181. 181. System-wide usability test
  182. 182. Preparation for implementations</li></li></ul><li>Beta Testing Activities<br />75<br />Usability Activities<br />during Beta Testing<br />Planning <br />Execution<br /><ul><li>Test Objectives
  183. 183. Test Design
  184. 184. Test Plan </li></ul>including data analysis plan<br /><ul><li> Test Participants
  185. 185. Test Environment
  186. 186. Test Report
  187. 187. Pilot Test
  188. 188. Logistics
  189. 189. Carry out plan</li></ul>including test and data analysis<br /><ul><li>Share with team & IMPACT design, implementation</li></li></ul><li>Beta Testing Activities<br />Usability Activities<br />during Beta Testing<br />76<br />Timing: We are nearing the end<br />Impact: Not too late to impact the application<br />
  190. 190. Launch Activities<br />77<br />Continued Testing<br />Goal<br /><ul><li>From Usability Perspective:
  191. 191. Live usability test to inform current application and next release
  192. 192. System-wide usability test </li></li></ul><li>Launch Activities<br />Continued Testing<br />78<br />Planning <br />Execution<br /><ul><li>Test Objectives
  193. 193. Test Design
  194. 194. Test Plan </li></ul>including data analysis plan<br /><ul><li> Test Participants
  195. 195. Test Environment
  196. 196. Test Report
  197. 197. Pilot Test
  198. 198. Logistics
  199. 199. Carry out plan</li></ul>including test and data analysis<br /><ul><li>Share with team & IMPACT design, implementation</li></li></ul><li>Launch Activities<br />79<br />Continued Testing<br />Timing: We are nearing the end<br />Impact: Not too late to impact the application<br />
  200. 200. 80<br />Post-Launch<br />Post-Launch<br />Post Launch Activities<br />Learnings Discover Activities<br />Implementation<br />Discover<br />Define<br />Launch<br />Selection<br />Design<br />Beta<br />Develop<br />Implementation<br />
  201. 201. 81<br />Cost / Benefit<br />
  202. 202. Cost/Benefit<br />Key to cost-effective product usability is to<br />Plan & Manage<br />the usability activities within the development process. <br />82<br />
  203. 203. Cost/Benefit<br />83<br />Summative<br />Formative <br />Formative <br /><ul><li>Prototyping
  204. 204. Card sorting
  205. 205. Concept exploration
  206. 206. Usability Testing
  207. 207. and testing
  208. 208. Keep testing
  209. 209. Requirements
  210. 210. User stories
  211. 211. Summative testing
  212. 212. User acceptance testing
  213. 213. Incorporating feedback
  214. 214. Maintain design intent
  215. 215. Affinity Diagramming
  216. 216. Personas
  217. 217. Use case scenarios</li></ul>specific activity designed to gather data to address specific research question<br />
  218. 218. http://bit.ly/UXmethods<br />84<br />
  219. 219. 85<br />Costs<br />Including Usability<br />in Development Cycle<br />Design Alternatives<br />Cost of Changes<br />Planning<br />Design<br />Development<br />Release<br />
  220. 220. CostsUsability in the Development Cycle 1 of 2<br />Discount Usability vs Premium Usability<br />Time (labor)<br />Employee vs Contractor (expendable)<br />Design vs Evaluation<br />Test system setup<br />Participant recruitment and compensation<br />86<br />
  221. 221. CostsUsability in the Development Cycle 2 of 2<br />Lab equipment & usability lab facility<br />Travel expenses<br />Planning Time !!<br /><ul><li>Managers must make the time to plan the integration of usability into the development process.
  222. 222. Sometimes too busy to stop and determine how to integrate usability during development.</li></ul>87<br />
  223. 223. Measurable Benefitsof Usability Integration<br />Improved productivity<br /><ul><li>Shorter time to complete tasks
  224. 224. Fewer errors during tasks
  225. 225. Improved user satisfaction</li></ul>Lower training costs<br />Reduced tech support & documentation costs<br />Increased patient safety<br />Litigation deterrence<br />User preferences<br /><ul><li>Brand loyalty
  226. 226. Marketing voice</li></ul>88<br />
  227. 227. Tradeoffsbetween Usability Costs & Benefits<br />Measurable benefits not well documented<br /><ul><li>“Post” data is often not collected due to budget and lack of vision
  228. 228. Proprietary data used competitively and not available to the public</li></ul>Usability community has made efforts to quantify benefit of Usability interventions<br />89<br />
  229. 229. Tradeoffsbetween Usability Costs & Benefits<br />Case scenarios presented as examples that hopefully demonstrate the cost-benefit tradeoffs across several types of HF activities.<br />Data for case scenarios taken from:<br />Good Ergonomics Is Good Economics (1996). Hal Hendrick. Proceedings of the HFES 40th.<br />Usability Professionals’ Association (UPA) http://www.upassoc.org<br />90<br />
  230. 230. Best Bang for the BuckKey Usability Activities<br />Contextual Inquiry<br />Usability Testing (1 on 1) with prototype or early code<br /><ul><li>Enough functionality to test
  231. 231. Early enough to make changes</li></ul>91<br />
  232. 232. Session Objectives<br />Describe stages of software development in which usability methods can be most effective.<br /><ul><li>Requirements & Design</li></ul>List and describe usability methods applicable to each stage of EMR development.<br /><ul><li>Contextual Inquiry & Usability Testing</li></ul>Analyze the cost/benefit of implementing usability methods within a user-centered design process.<br /><ul><li>Early usability methods are most cost effective in terms of planning, carrying out, and incorporating findings in software.</li></ul>92<br />
  233. 233. Links<br />Usability Methods Table | Usability.net<br />HIMSS EHR Usability Links | Online resources<br />
  234. 234. Questions?<br />For further info, contact:Janey Barnes PhD | jbarnes@user-view.comJeff Belden MD | jeffbelden@gmail.com<br />

×