Successfully reported this slideshow.

F.A.I.L.U.R.E.

2,476 views

Published on

A mnemonic for testing error messages.

Published in: Technology, Health & Medicine
  • Be the first to comment

F.A.I.L.U.R.E.

  1. 3. Now that I have your attention, prepare to read fast v1.2 FAILURE.QualityFrog.com
  2. 4. You’re testing… And an error message appears v1.2 FAILURE.QualityFrog.com
  3. 5. v1.2 FAILURE.QualityFrog.com
  4. 6. Did you expect it? Or are you surprised? v1.2 FAILURE.QualityFrog.com
  5. 7. How does it make you feel? v1.2 FAILURE.QualityFrog.com
  6. 8. What do you do? v1.2 FAILURE.QualityFrog.com
  7. 9. v1.2 FAILURE.QualityFrog.com Not ready reading drive A Abort, Retry, Ignore, Fail?
  8. 10. Continue to the next test step… v1.2 FAILURE.QualityFrog.com
  9. 11. v1.2 FAILURE.QualityFrog.com
  10. 12. Report a bug… v1.2 FAILURE.QualityFrog.com
  11. 13. v1.2 FAILURE.QualityFrog.com
  12. 14. Resolve unknown conflict… v1.2 FAILURE.QualityFrog.com
  13. 15. v1.2 FAILURE.QualityFrog.com
  14. 16. Find a file… v1.2 FAILURE.QualityFrog.com
  15. 17. v1.2 FAILURE.QualityFrog.com
  16. 18. Eliminate something… v1.2 FAILURE.QualityFrog.com
  17. 19. v1.2 FAILURE.QualityFrog.com
  18. 20. Request permission… v1.2 FAILURE.QualityFrog.com
  19. 21. v1.2 FAILURE.QualityFrog.com
  20. 22. Try again… v1.2 FAILURE.QualityFrog.com
  21. 23. v1.2 FAILURE.QualityFrog.com
  22. 24. Try something new… v1.2 FAILURE.QualityFrog.com
  23. 25. v1.2 FAILURE.QualityFrog.com
  24. 26. Seek help… v1.2 FAILURE.QualityFrog.com
  25. 27. v1.2 FAILURE.QualityFrog.com
  26. 28. Set a reference to an instance… v1.2 FAILURE.QualityFrog.com
  27. 29. v1.2 FAILURE.QualityFrog.com
  28. 30. lick OK… v1.2 FAILURE.QualityFrog.com C
  29. 31. lick OK… v1.2 FAILURE.QualityFrog.com
  30. 32. v1.2 FAILURE.QualityFrog.com
  31. 33. Save… v1.2 FAILURE.QualityFrog.com
  32. 34. v1.2 FAILURE.QualityFrog.com
  33. 35. Enter a multiple of |ECL0… v1.2 FAILURE.QualityFrog.com
  34. 36. v1.2 FAILURE.QualityFrog.com
  35. 37. Exit… v1.2 FAILURE.QualityFrog.com
  36. 38. v1.2 FAILURE.QualityFrog.com
  37. 39. Reconnect… v1.2 FAILURE.QualityFrog.com
  38. 40. v1.2 FAILURE.QualityFrog.com
  39. 41. Declare success… v1.2 FAILURE.QualityFrog.com
  40. 42. v1.2 FAILURE.QualityFrog.com
  41. 43. Assign blame… v1.2 FAILURE.QualityFrog.com
  42. 44. v1.2 FAILURE.QualityFrog.com
  43. 45. Laugh… v1.2 FAILURE.QualityFrog.com
  44. 46. v1.2 FAILURE.QualityFrog.com
  45. 47. Restart… v1.2 FAILURE.QualityFrog.com
  46. 48. v1.2 FAILURE.QualityFrog.com
  47. 49. Guess… v1.2 FAILURE.QualityFrog.com
  48. 50. v1.2 FAILURE.QualityFrog.com
  49. 51. Gasp… v1.2 FAILURE.QualityFrog.com
  50. 52. v1.2 FAILURE.QualityFrog.com
  51. 53. Request a raise… v1.2 FAILURE.QualityFrog.com
  52. 54. v1.2 FAILURE.QualityFrog.com
  53. 55. Swear… v1.2 FAILURE.QualityFrog.com
  54. 56. v1.2 FAILURE.QualityFrog.com
  55. 57. So… v1.2 FAILURE.QualityFrog.com
  56. 58. What do you do when you encounter an error? v1.2 FAILURE.QualityFrog.com
  57. 59. While you think about that, let me tell you what I do. v1.2 FAILURE.QualityFrog.com
  58. 60. Oh, who am I? v1.2 FAILURE.QualityFrog.com
  59. 61. I am Ben Simo [email_address] v1.2 FAILURE.QualityFrog.com
  60. 62. I use software [email_address] v1.2 FAILURE.QualityFrog.com
  61. 63. I test software [email_address] v1.2 FAILURE.QualityFrog.com
  62. 64. I question software [email_address] www.QuestioningSoftware.com v1.2 FAILURE.QualityFrog.com
  63. 65. I code software [email_address] www.QuestioningSoftware.com v1.2 FAILURE.QualityFrog.com
  64. 66. I code software that helps test software [email_address] www.QuestioningSoftware.com v1.2 FAILURE.QualityFrog.com
  65. 67. I talk about software [email_address] www.QuestioningSoftware.com v1.2 FAILURE.QualityFrog.com
  66. 68. I write about software [email_address] www.QuestioningSoftware.com v1.2 FAILURE.QualityFrog.com
  67. 69. I think about software [email_address] www.QuestioningSoftware.com v1.2 FAILURE.QualityFrog.com
  68. 70. I like software v1.2 FAILURE.QualityFrog.com
  69. 71. I like software that serves people v1.2 FAILURE.QualityFrog.com
  70. 72. I like software that helps people v1.2 FAILURE.QualityFrog.com
  71. 73. I dislike bad software v1.2 FAILURE.QualityFrog.com
  72. 74. I dislike bad software that frustrates people v1.2 FAILURE.QualityFrog.com
  73. 75. I dislike bad software that confuses people v1.2 FAILURE.QualityFrog.com
  74. 76. I dislike bad software that angers people v1.2 FAILURE.QualityFrog.com
  75. 77. I dislike bad software that discombobulates people v1.2 FAILURE.QualityFrog.com
  76. 78. I sometimes like big words that discombobulate people v1.2 FAILURE.QualityFrog.com
  77. 79. But only for a moment v1.2 FAILURE.QualityFrog.com
  78. 80. Discombobulate means to emotionally confuse v1.2 FAILURE.QualityFrog.com
  79. 81. Discombobulate means to frustrate v1.2 FAILURE.QualityFrog.com
  80. 82. Discombobulate means to bewilder v1.2 FAILURE.QualityFrog.com
  81. 83. Discombobulate means to confound v1.2 FAILURE.QualityFrog.com
  82. 84. Discombobulate means to befuddle v1.2 FAILURE.QualityFrog.com
  83. 85. Discombobulate means to perplex v1.2 FAILURE.QualityFrog.com
  84. 86. Too many error messages discombobulate people v1.2 FAILURE.QualityFrog.com
  85. 87. Error messages should help people, not confuse them v1.2 FAILURE.QualityFrog.com
  86. 88. You see, It’s not about the software v1.2 FAILURE.QualityFrog.com
  87. 89. It’s about the people! v1.2 FAILURE.QualityFrog.com
  88. 90. Testers often fail to test the error messages v1.2 FAILURE.QualityFrog.com
  89. 91. They fail to think about how they affect people v1.2 FAILURE.QualityFrog.com
  90. 92. So what do I do when I encounter error messages? v1.2 FAILURE.QualityFrog.com
  91. 93. I test them v1.2 FAILURE.QualityFrog.com
  92. 94. I test them using the FAILURE mnemonic v1.2 FAILURE.QualityFrog.com
  93. 95. A mnemonic is a device that aids memory recall v1.2 FAILURE.QualityFrog.com
  94. 96. The FAILURE mnemonic is an acronym that helps me recall guideword heuristics v1.2 FAILURE.QualityFrog.com
  95. 97. Guideword heuristics are words that guide thinking through a problem v1.2 FAILURE.QualityFrog.com
  96. 98. The FAILURE mnemonic’s guideword heuristics help me think about testing error messages v1.2 FAILURE.QualityFrog.com
  97. 99. The F reminds me to think about Functional aspects of the error F AILURE v1.2 FAILURE.QualityFrog.com
  98. 100. I ask Functional questions like Does the error detection function as expected? v1.2 FAILURE.QualityFrog.com
  99. 101. I ask Functional questions like Does the error handling function as expected? v1.2 FAILURE.QualityFrog.com
  100. 102. I ask Functional questions like Does the error reporting function as expected? v1.2 FAILURE.QualityFrog.com
  101. 103. I ask Functional questions like Does the error dialog, window, or page function as expected? v1.2 FAILURE.QualityFrog.com
  102. 104. I ask Functional questions like Does the software recover from the error? v1.2 FAILURE.QualityFrog.com
  103. 105. I ask Functional questions like Was there really an error? v1.2 FAILURE.QualityFrog.com
  104. 106. I ask Functional questions like Are other errors like this reported? v1.2 FAILURE.QualityFrog.com
  105. 107. I ask Functional questions like Do dialog buttons work? v1.2 FAILURE.QualityFrog.com
  106. 108. I ask Functional questions like Is error message information accurate? v1.2 FAILURE.QualityFrog.com
  107. 109. I ask Functional questions like Is it functional? v1.2 FAILURE.QualityFrog.com
  108. 110. Only I don’t stop at Functional v1.2 FAILURE.QualityFrog.com
  109. 111. A Functional error message is not necessarily an Appropriate error message v1.2 FAILURE.QualityFrog.com
  110. 112. The A reminds me to ask if the error reporting is Appropriate F A ILURE v1.2 FAILURE.QualityFrog.com
  111. 113. I ask Appropriate questions like Is the error reporting timely? v1.2 FAILURE.QualityFrog.com
  112. 114. I ask Appropriate questions like Is the error reporting accurate? v1.2 FAILURE.QualityFrog.com
  113. 115. I ask Appropriate questions like Are UI messages free of inappropriate technical detail? v1.2 FAILURE.QualityFrog.com
  114. 116. I ask Appropriate questions like Is there a better way to handle or report the error? v1.2 FAILURE.QualityFrog.com
  115. 117. I ask Appropriate questions like Is the error user friendly? v1.2 FAILURE.QualityFrog.com
  116. 118. I ask Appropriate questions like Are errors detected as early in the user workflow as possible? v1.2 FAILURE.QualityFrog.com
  117. 119. I ask Appropriate questions like Are system resource problems detected while there is still time to remedy the problem? v1.2 FAILURE.QualityFrog.com
  118. 120. I ask Appropriate questions like Is the situation communicated appropriately for the intended audience? v1.2 FAILURE.QualityFrog.com
  119. 121. I ask Appropriate questions like Is a user allowed to waste time and effort only to be told that their work cannot be applied? v1.2 FAILURE.QualityFrog.com
  120. 122. The I reminds me to ask about the Impact of the error FA I LURE v1.2 FAILURE.QualityFrog.com
  121. 123. I ask Impact questions like <ul><li>Is the impact communicated? </li></ul>v1.2 FAILURE.QualityFrog.com
  122. 124. I ask Impact questions like <ul><li>Is the impact communicated </li></ul><ul><li>to the user? </li></ul>v1.2 FAILURE.QualityFrog.com
  123. 125. I ask Impact questions like <ul><li>Is the impact communicated </li></ul><ul><li>to others? </li></ul>v1.2 FAILURE.QualityFrog.com
  124. 126. I ask Impact questions like Will the user understand the impact? v1.2 FAILURE.QualityFrog.com
  125. 127. I ask Impact questions like <ul><li>Is any impact to the user’s data communicated? </li></ul>v1.2 FAILURE.QualityFrog.com
  126. 128. I ask Impact questions like Does the message contain too much information? v1.2 FAILURE.QualityFrog.com
  127. 129. I ask Impact questions like <ul><li>Was a transaction only partially completed? </li></ul>v1.2 FAILURE.QualityFrog.com
  128. 130. The L reminds me to ask about the error Log FAI L URE v1.2 FAILURE.QualityFrog.com
  129. 131. I ask Log questions like Should the error be logged for support? v1.2 FAILURE.QualityFrog.com
  130. 132. I ask Log questions like Should the error be logged for system administrators? v1.2 FAILURE.QualityFrog.com
  131. 133. I ask Log questions like Should the error be logged for developers? v1.2 FAILURE.QualityFrog.com
  132. 134. I ask Log questions like Should the error be logged for technical investigation? v1.2 FAILURE.QualityFrog.com
  133. 135. I ask Log questions like Will log information be available if the user waits to report a problem? v1.2 FAILURE.QualityFrog.com
  134. 136. I ask Log questions like Are log messages standardized to support automated information mining? v1.2 FAILURE.QualityFrog.com
  135. 137. I ask Log questions like Can I grep this error? v1.2 FAILURE.QualityFrog.com
  136. 138. I ask Log questions like Do logs contain enough information? v1.2 FAILURE.QualityFrog.com
  137. 139. I ask Log questions like Do logs contain useless information? v1.2 FAILURE.QualityFrog.com
  138. 140. I ask Log questions like <ul><li>Is this an error that should not be logged? </li></ul>v1.2 FAILURE.QualityFrog.com
  139. 141. I ask Log questions like <ul><li>Is there too much logging? </li></ul>v1.2 FAILURE.QualityFrog.com
  140. 142. I ask Log questions like <ul><li>Does excessive logging impact performance? </li></ul>v1.2 FAILURE.QualityFrog.com
  141. 143. I ask Log questions like <ul><li>Does excessive logging impact disk space? </li></ul>v1.2 FAILURE.QualityFrog.com
  142. 144. I ask Log questions like <ul><li>Does the logging help or complicate error investigation? </li></ul>v1.2 FAILURE.QualityFrog.com
  143. 145. The U reminds me to think about the User Interface FAIL U RE v1.2 FAILURE.QualityFrog.com
  144. 146. I ask UI questions like Is the error reported to the user? v1.2 FAILURE.QualityFrog.com
  145. 147. I ask UI questions like Does this error need to be reported to the user? v1.2 FAILURE.QualityFrog.com
  146. 148. I ask UI questions like What user activity caused the failure? v1.2 FAILURE.QualityFrog.com
  147. 149. I ask UI questions like What user activity caused the failure? Is this communicated to the user? v1.2 FAILURE.QualityFrog.com
  148. 150. I ask UI questions like Who is the user? v1.2 FAILURE.QualityFrog.com
  149. 151. I ask UI questions like Are UI messages worded for the intended audience? v1.2 FAILURE.QualityFrog.com
  150. 152. I ask UI questions like Is this message consistent with the desired look and feel? v1.2 FAILURE.QualityFrog.com
  151. 153. I ask UI questions like Is the UI reporting consistent with reporting for other errors? v1.2 FAILURE.QualityFrog.com
  152. 154. I ask UI questions like Is the UI reporting efficient? v1.2 FAILURE.QualityFrog.com
  153. 155. I ask UI questions like Does the user need to click away excessive dialogs? v1.2 FAILURE.QualityFrog.com
  154. 156. I ask UI questions like Is this the best way to communicate the error? v1.2 FAILURE.QualityFrog.com
  155. 157. I ask UI questions like Could this error be handled without involving the user? v1.2 FAILURE.QualityFrog.com
  156. 158. The R helps me think about error Recovery FAILU R E v1.2 FAILURE.QualityFrog.com
  157. 159. I ask Recovery questions like Is the user told how to recover from the impact of the error? v1.2 FAILURE.QualityFrog.com
  158. 160. I ask Recovery questions like Does the user need to contact support? v1.2 FAILURE.QualityFrog.com
  159. 161. I ask Recovery questions like Is support contact information provided? v1.2 FAILURE.QualityFrog.com
  160. 162. I ask Recovery questions like Does support know how to fix the problem? v1.2 FAILURE.QualityFrog.com
  161. 163. I ask Recovery questions like Does the software guide the user through the recovery process? v1.2 FAILURE.QualityFrog.com
  162. 164. I ask Recovery questions like Or is the user left to figure it out on their own? v1.2 FAILURE.QualityFrog.com
  163. 165. The E leads me to think about the Emotions of users FAILUR E v1.2 FAILURE.QualityFrog.com
  164. 166. I ask Emotions questions like What emotions are likely to be raised by the error message? v1.2 FAILURE.QualityFrog.com
  165. 167. I ask Emotions questions like Does the error reporting evoke the desired user emotions? v1.2 FAILURE.QualityFrog.com
  166. 168. I ask Emotions questions like Does this message enhance user frustration? v1.2 FAILURE.QualityFrog.com
  167. 169. I ask Emotions questions like Does this message quiet user frustration? v1.2 FAILURE.QualityFrog.com
  168. 170. I ask Emotions questions like Does this message encourage users to use some other product? v1.2 FAILURE.QualityFrog.com
  169. 171. I ask Emotions questions like Does this message cause confusion? v1.2 FAILURE.QualityFrog.com
  170. 172. Asking these types of questions can help turn FAILURE into success v1.2 FAILURE.QualityFrog.com
  171. 173. The next time you encounter an error message v1.2 FAILURE.QualityFrog.com
  172. 174. The next time you encounter an error message Test it! v1.2 FAILURE.QualityFrog.com
  173. 175. The next time you encounter an error message Question it! v1.2 FAILURE.QualityFrog.com
  174. 176. Try the FAILURE mnemonic <ul><li>F unctional </li></ul><ul><li>A ppropriate </li></ul><ul><li>I mpact </li></ul><ul><li>L og </li></ul><ul><li>U I </li></ul><ul><li>R ecovery </li></ul><ul><li>E motions </li></ul>v1.2 FAILURE.QualityFrog.com
  175. 177. And let me know how it works for you v1.2 FAILURE.QualityFrog.com F unctional A ppropriate I mpact L og U I R ecovery E motions
  176. 178. Ben Simo [email_address] v1.2 Copyright © 2007 Quality Frog http://FAILURE.QualityFrog.com F unctional A ppropriate I mpact L og U I R ecovery E motions

×