Specification by example and agile acceptance testing


Published on

Specification by example and agile acceptance testing, presentation given to HSBC developers on 21/09/09 for more info see http://specificationbyexample.com

Published in: Technology
  • Specification by Example 책을 정리한 것이라고 합니다.
    Are you sure you want to  Yes  No
    Your message goes here
  • the book that this presentation was based on is now available in preview - see http://specificationbyexample.com
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Specification by example and agile acceptance testing

  1. 1. Specification by Example and Agile Acceptance Testing Bridging the communication gap in software projects Gojko Adzic [email_address] @gojkoadzic http://gojko.net
  2. 2. Why should you care (PM/BA)? <ul><li>Developers will actually read the specifications </li></ul><ul><li>They will understood the stuff correctly </li></ul><ul><li>They will not skip parts of the specs </li></ul><ul><li>You can track the development progress </li></ul><ul><li>Save time on acceptance/smoke testing </li></ul>
  3. 3. Why should you care (dev)? <ul><li>Requirements will be unambiguous and without functional gaps </li></ul><ul><li>Business analysts will really understand those special cases you mentioned </li></ul><ul><li>You will have automated tests to guide development </li></ul><ul><li>It will be easier to take-over and hand-over code </li></ul>
  4. 4. Why should you care (QA)‏ <ul><li>Finally stop those guys from making the same mistakes over and over </li></ul><ul><li>Avoid doing the same stuff all the time </li></ul><ul><li>Build quality in from the start </li></ul><ul><li>Verify business rules by a click on a button </li></ul>
  5. 5. http://www.flickr.com/photos/lambdachialpha/157986473/
  6. 6. http://www.flickr.com/photos/mulesafpilot/3513588967
  7. 7. http://www.defenseimagery.mil/assetDetails.action?guid=68cf92e35ce13e6c7c9c066f0b48b6daaa9bf8d8
  8. 8. An experiment with four active battalions in US Army <ul><li>“ Commander expectations matched actions in only 34% of the cases” </li></ul>L.G.Shattuck, 2000 http://www.au.af.mil/au/awc/awcgate/milreview/shattuck.pdf
  9. 9. The process is very much like a telephone game http://www.flickr.com/photos/mataniere/3107073262
  10. 10. How many points are there?
  11. 11. How many points are there?
  12. 12. <ul><li>“ Just-in-case code is the biggest source of waste in software development” </li></ul><ul><li>Mary and Tom Poppendieck </li></ul><ul><li>Lean Software Development </li></ul>
  13. 13. B2 bomber crashed and $2bn went up in flames &quot;the aircraft actually performed as it was designed. In other words, all the systems were functioning normally.&quot; Maj. Gen. Floyd L. Carpenter http://www.foxnews.com/wires/2008Jun05/0,4670,B2Crash,00.html
  14. 14. You can't help a lot when the party is already over... http://www.flickr.com/photos/biolog/3457774800
  15. 15. F-16 design team was asked to do the impossible - a cheap 2.5 Mach airplane! “ When asked […] why they need Mach 2 - 2.5, the answer was to be able to escape from combat. Their solution was […] providing acceleration and maneuverability, not maximum speed.” http://97-things.near-time.net/wiki/Seek%20the%20value%20in%20requested%20capabilities
  16. 16. Refuse requirements that are a solution to an unknown problem! http://www.flickr.com/photos/sylvancatharsis/3783608640/
  17. 17. One of the most effective ways of testing requirements is with test cases very much like those for testing the completed system Donald Gause and Gerald Weinberg Exploring Requirements - 1989 !
  18. 19. As formality increases, tests and requirements become indistinguishable. Robert C. Martin and Grigori Melnik Tests and Requirements, Requirements and Tests: a Mobius Strip IEEE Software January/February Issue 2008
  19. 20. Key practices <ul><li>Discuss real-world examples to build a shared understanding of the domain </li></ul><ul><li>Use those examples as an acceptance criteria </li></ul><ul><li>Automate acceptance tests </li></ul><ul><li>Focus the development on those tests </li></ul><ul><li>Use the tests as a live specification to facilitate change </li></ul>
  20. 21. Better names <ul><li>“ Acceptance testing” is misleading! </li></ul><ul><li>Behaviours </li></ul><ul><li>Executable specifications </li></ul><ul><li>Examples </li></ul>
  21. 22. Jim Shore: “Describe-Demonstrate-Develop” A very useful way to think about acceptance tests in practice http://www.jamesshore.com/Blog/How-I-Use-Fit.html
  22. 23. Iteration flow (just a suggestion)‏
  23. 24. Step 1: Building a shared understanding of the domain
  24. 25. Specification workshops <ul><li>Business people, developers and QA in the same room </li></ul><ul><li>Transfer the knowledge </li></ul><ul><li>Ensure that we all understand the same thing </li></ul>
  25. 26. Inconsistencies and gaps are easy to spot when you write the rules down!
  26. 27. Real-world examples help flush out incorrect assumed rules find real business rules!
  27. 28. People have think at a more detailed level and can't brush questions off…
  28. 29. People approach the same problem from different perspectives, so this avoids groupthink!
  29. 30. Step 2: Select a formal set of acceptance tests and automate them
  30. 31. The Toyota Way <ul><li>Check at the source </li></ul><ul><li>Inexpensive Verifications </li></ul><ul><li>Test to prevent defects, not to discover them </li></ul>
  31. 32. Save time on manually (acceptance/smoke) testing Verify business rules with the click of a button
  32. 33. Automate tests, but still keep them human-readable And you can add pictures as well….
  33. 34. This is a specification:
  34. 35. It is also a test
  35. 36. This as well
  36. 37. And this
  37. 38. And this Given a stock of prices 0.5,1.0 When the stock is traded at 2.0 Then the alert status should be OFF When the stock is traded at 5.0 Then the alert status should be OFF When the stock is traded at 11.0 Then the alert status should be OFF
  38. 39. A good acceptance test is <ul><li>Focused on a single thing (rule, step..)‏ </li></ul><ul><li>Specification not a script </li></ul><ul><li>Self-explanatory </li></ul><ul><li>Uses the domain language </li></ul><ul><li>SMART </li></ul><ul><ul><li>Specific </li></ul></ul><ul><ul><li>Measurable </li></ul></ul><ul><ul><li>Achievable </li></ul></ul><ul><ul><li>Relevant </li></ul></ul><ul><ul><li>Time-bound </li></ul></ul>
  39. 41. Step 3: Providing focus for development No just-in-case code
  40. 42. Developers will have to code exactly what was specified … not just the rules they see
  41. 43. Automated test reports show where we are… When all the tests are green, the job is done
  42. 44. Step 4: Keeping in touch with changes
  43. 45. Live documentation As relevant and reliable as executable code, but much easier to read!
  44. 46. Previous examples help you ensure to discuss all important edge cases.
  45. 47. Automated tests show straight away when something is obsolete or broken
  46. 48. <ul><li>[tests became a] </li></ul><ul><li>“ significant and valuable business resource” </li></ul>Rick Mugridge, Doubling the Value of Automated Tests, Google Tech Talks 09/2006
  47. 49. Recap (PM/BA)‏ <ul><li>Developers will have to read the specifications to implement tests </li></ul><ul><li>Discussion makes sure that everyone understood the stuff correctly </li></ul><ul><li>To make all tests green, they cannot skip parts of the specs </li></ul><ul><li>You can track the development progress </li></ul><ul><li>Save time on acceptance testing with automated verifications </li></ul>
  48. 50. Recap (dev)‏ <ul><li>Discuss and suggest examples until the gaps and inconsistencies are flushed out </li></ul><ul><li>Make sure that business analysts understood special cases by suggesting them as examples and discussing them </li></ul><ul><li>Acceptance tests are a live specification/documentation for the code </li></ul>
  49. 51. Recap (QA)‏ <ul><li>Suggest examples and discuss them to cover mistakes that people make over and over </li></ul><ul><li>Automated tests help you avoid doing the same stuff all the time </li></ul><ul><li>Build quality in from the start by suggesting tests that prevent problems </li></ul><ul><li>Verify business rules by a click on a button </li></ul>
  50. 52. Where next? <ul><li>http://gojko.net </li></ul><ul><li>http://agiletesting.org.uk </li></ul><ul><li>http://acceptancetesting.info </li></ul><ul><li>27 Nov – Agile specifications and testing exchange </li></ul><ul><li>30 Nov – Agile acceptance testing workshop </li></ul><ul><li>4 Dec – Building software that matters </li></ul>