This webinar discusses best practices for creating Story Tests (aka Acceptance Tests). Acceptance Testing, also known as Story Testing, is vital to achieve the Agile vision of “working software over comprehensive documentation.” Read more at https://www.synerzip.com/webinar/acceptance-and-story-testing-patterns/

1. Charles Bradley
Professional Scrum Trainer
Experienced Scrum Coach
http://www.ScrumCrazy.com
To download presentation,
see “Presentations” section at:
http://www.ScrumCrazy.com
© 2012, Charles Bradley, All Rights Reserved
1

2. About Me…
 Professional Scrum Trainer, Scrum.org
 Courses:
 Professional Scrum Foundations(2 days)
 Professional Scrum Master (2 days)
 Started Scrum in 2008, Scrum Team Member (Java),
Scrum Coach
 B.S., Computer Science
2www.synerzip.com

3. Why should I care about
Story Testing Patterns?
Increase Productivity and Efficiency
 Identify Tests up front (ATDD), before development
begins
 Flushes out non-obvious test scenarios
Enable more Test Automation!
 Tests can be automated while code is being written
 Allows us to move faster without being sloppy, and
without comprehensive documentation
3www.synerzip.com

4. Overview
 Quick Review of User Stories (15% of our time)
 Basic Story Testing Patterns (50%)
 Exercise (15%)
 Advanced Story Testing Patterns (10%)
 Special Story Testing Patterns (10%)
4www.synerzip.com

5. User Stories
 Card/Title
 token used for planning and reminder to have conversations
 Conversations
 Confirmations
 aka Story Tests
 aka Acceptance Tests
 aka Test Confirmations
 aka Acceptance Criteria
 User Stories are not part of Scrum, but they are one good
way to represent Product Backlog Items in Scrum
5www.synerzip.com

6. A User Story Worst Practice
 “As a <type of user>, I want <some functionality>, so that
<the user or stakeholder achieves some value>”
 The Technique is not a worst practice, but thinking this is a
User Story IS a worst practice!
 ^^ This is not a User Story!
 ^^ This is 1/3 of a User Story
 ^^ This is the least important 1/3 of the User Story!
I argue that Story tests are the most
important part, and they are achieved
via Conversations
6www.synerzip.com

7. Warning!!
 Story Tests and these patterns are NOT:
 Documentation techniques.
 Living documents that are maintained.
 User Stories only convey *new* behavior of a system.
 Strict contracts.
7www.synerzip.com

8. What Story Tests Are…
 Story Tests are:
 A Communication and testing technique.
 The results of collaboration and conversation.
 Defined during backlog grooming, before development
begins, at least at a high/conceptual level.
 Ideally automated to create an Agile Specification.
 Not all automated Story Tests are at the UI level
8www.synerzip.com

9. Agile Manifesto Relevance
 Summation of previous two slides:
 We value
 “Working software over comprehensive documentation”
 “Individuals and interactions over processes and tools”
9www.synerzip.com

10. Two Story Tests that Always Apply
 Old behavior still works correctly
 PO has accepted the story
 Best Practice: Always get immediate signoff from the PO
– Don’t wait until Sprint End.
10www.synerzip.com

11. How we’re going to proceed
 For each Pattern Category
 Advice on Communication Mediums
 For each Pattern
 Description
 Context (Good For/Bad For)
 An Example or two
11www.synerzip.com

12. Basic Story Testing Patterns
 “Test that…”
 Given/When/Then (aka Gherkin style)
 Specification By Example – Conceptual
 Specification By Example – Concrete
12www.synerzip.com

13. Communication Mediums –
Basic Story Testing Patterns
 Best
 A wiki or whiteboard is (Best)
 Hand written (paraphrased) on 5 X 8 cards (2nd Best)
 Word Processing Document (last resort)
 Worst
 Almost all ALM Tools
13www.synerzip.com

14. Optimize Your Documentation
 Q: How much do I document?
 A: The minimum amount that could possibly work.
 Q: How will I know if it’s working?
 A: If your Product Owner often has to point out things not
done, that were already mentioned in previous User Story
conversations. This is a sign that you might need to
document more, and/or add more Story Tests. If this rarely
happens, then maybe you should try documenting less!
Iterate to the optimum!
 The optimum amount will vary widely by team.
14www.synerzip.com

15. Pattern: “Test that…”
 The technique
 “Test that <some new behavior happens>”
 “Test that <some really important old behavior still
happens>” (less frequent)
 Conceptual
 Avoid using specific test data
15www.synerzip.com

16. Pattern Context: “Test that…”
 Good For
 Beginner Story Testers
 Simple Tests
 Tests that are hard to describe or understand using the
other patterns
16www.synerzip.com

17. Pattern Context : “Test that…”
 Bad For
 Experienced Story Testers who know of a more
appropriate pattern to use
 Tests with a lot of setup logic or behavior logic.
 Tests where behavior depends on numerous test inputs
17www.synerzip.com

18. Used in some of the examples…
18www.synerzip.com

19. Examples: “Test that…”
AT-1. Test that, when a user enters an incorrect old password,
they get an error message indicating incorrect credentials.
AT-2. Test that three incorrect submissions of the old
password within 1 hour results in the user being logged out
from the system.
AT-3. Test that, when a user clicks on the "Continue
Shopping" link, it takes them to the home page.
AT-4. Test that the order confirmation contains:
 An order number
 Estimated arrival date
 Customer service email address
19www.synerzip.com

20. Pattern: Given/When/Then
 The technique
 Given <some test precondition or setup>
 When <some trigger event>
 Then <some new behavior occurs>
 (or less frequently, Then <some really important old behavior
still occurs>)
 Might use specific test data
 Use “AND”, “OR” to join statements as necessary
20www.synerzip.com

21. Pattern Context:
Given/When/Then
 Good For
 Tests that require a lot of preconditions, setup.
 Tests that require setup that is important or easily
forgotten
 Tests that have a specific, non obvious, trigger.
 Tests where there are few expected outputs
21www.synerzip.com

22. Pattern Context:
Given/When/Then
 Bad For
 Simple tests
 Tests that have unimportant/simple/obvious
preconditions
 Tests where there are multiple different inputs and
multiple different outputs
 Tests where a single Given/When/Then only describes
one of numerous very similar test scenarios
22www.synerzip.com

23. Examples: Given/When/Then
AT-1.
Given
 A user who has submitted an incorrect old password 2 times
in the last hour
When
 The user submits an incorrect password (for the 3rd time)
Then
 The system logs the user out AND
 The system displays:
 the customer service phone number.
 a message telling them to call customer service.
23www.synerzip.com

24. Examples: Given/When/Then
AT-2.
Given
 A user that is logged in AND
 the user is an admin user OR the user's account has been
flagged by Fraud
When
 The user submits an incorrect password (for the 3rd time)
Then
 The system logs the user out AND
 The system generates an email to the production support
team with the following info:
 user id of the user AND
 the user's phone number on file
24www.synerzip.com

25. Pattern: Specification By Example
(SBE)
 The technique
 Create a table of the most important Examples (testing
scenarios) that specify test inputs and expected test
outputs.
 Similar to the "decision table" concept in tools like
Fitnesse
 Be sure to create enough examples to exercise all
important logic/test paths
25www.synerzip.com

26. Pattern Context: SBE
 Good For
 Tests that have numerous
 Inputs that affect output behavior
 Outputs/expected behaviors
 Tests where it’s important to test a lot of different data
scenarios
 Tests where the trigger event is somewhat obvious
 Any test where it seems like a table would be useful to:
 describe the test better, or
 help explore all of the possible inputs and outputs for a test.
26www.synerzip.com

27. Pattern Context: SBE
 Bad For
 Simple tests
 Tests that are more about verifying simple UI behavior
 For instance – “Test that an error message is displayed when
the user enters an incorrect password.”
 Test where there is really only one input or precondition.
27www.synerzip.com

28. SBE-Conceptual vs. SBE-Concrete
 Technique differences
 For the conceptual form, avoid using specific data, but
instead describe the data conceptually.
 For the concrete form, use actual test data.
 Choosing Conceptual vs. Concrete
 Try to at least have the Conceptual form done before
development begins on a story.
 Concrete form is usually better, though it is harder for
teams to get done before development begins.
28www.synerzip.com

29. Example: SBE-Conceptual
29www.synerzip.com

30. Example: SBE-Concrete
30www.synerzip.com

31. Exercises
31www.synerzip.com

32. Example 1
 Pat: "For this story, we need to calculate the shipping
for orders. Our shipping prices are done by weight.
For under 10 pounds, the cost is $25. For under 50
pounds, the cost is $35, and for under 100 pounds, the
cost is $75. As you know, we don’t accept orders over
100 pounds."
Payton: “Sounds good, that seems pretty easy.”
Pat: “I thought so too."
32www.synerzip.com

33. Characteristics of this story
 Fairly simple (favors ‘Test That’ pattern)
 Numerous expected outputs, inputs affect outputs
(favors ‘Specification By Example’ patterns)
 The test setup and trigger events are fairly obvious or
unimportant. (‘Given/When/Then’ is not favored)
33www.synerzip.com

34. Example 1: Possible Story Tests
AT-1.
Test that orders under 10 lbs are charged $25 for
shipping.
AT-2.
Test that orders that are between 10lbs and under 50lbs
are charged $35 for shipping.
AT-3.
Test that orders 50lbs and over are charged $75 for
shipping.
34www.synerzip.com

35. Example 1: Possible Story Tests
Order Weight Shipping Cost
0-10lbs $25
10.01lbs-50lbs $35
> 50lbs $75
35
Order Weight Shipping Cost
0lbs $25
10.00 lbs $25
10.01 lbs $35
50.00 lbs $35
50.01 lbs $75
99.99lbs $75
Specification By Example - Concrete
Specification By Example - Conceptual
www.synerzip.com

36. Example 2
 User Story Title: RR154 – Logout goes to home page
 User Story Conversations:
Pat: "For this next story, here is what I want. Right now, when a user clicks on the
'logout’ link in the upper right hand menu, it takes them to a basically blank page
that tells them that we've logged them out. In the future, rather than going to
basically a blank page, I want the system to take them to the home page with all of
our products, and some sort of message up near the top that uses that same
message."
Payton: "I think it says 'You have successfully logged out.' "
Pat: "Yes, that's right, that one. Just make sure that the logout message is highly
visible near the top of the page -- I don't want it to get lost in the page."
Charles: "On other web sites, I've seen it presented like in a small rectangle with
a green background, to make it stand out from a mostly white page."
Pat: "That works fine, but I don't want to specify 'The How', right? You folks have
taught me that I only get to specify 'The What'"
Bailey: "Aahhhhh, right!"
36www.synerzip.com

37. Characteristics of this story
 Fairly UI related (favors ‘Test That’ pattern)
 Fairly simple story tests (favors ‘Test That’ pattern)
 The test setup and trigger events are important, but
probably not hard to remember, either. (moderately
favors ‘Given/When/Then’)
 Input/Output scenarios are not numerous – only a
couple (‘Specification By Example’ patterns not
favored)
37www.synerzip.com

38. Example 2: Possible Story Tests
 Using 'Test That' pattern
AT-1
Test that, when a user clicks the logout link, the system
navigates the user to the home page.
AT-2
Test that, after navigating the user to the home page, the
system displays a message like "You have successfully
logged out.“ (logout message)
AT-3
Test that the logout message is highly visible and near the
top of the page. (Ask Pat to approve aesthetics)
38www.synerzip.com

39. Example 2: Possible Story Tests
 Using ‘Given/When/Then’ pattern
AT-1.
Given a user that is logged in,
When the user clicks on the "logout link"
Then test that the system navigates the user to the home
page.
AT-2.
Given a user that has just logged out
When the system navigates the user to the home page
Then test that the system displays the existing logout
message in a highly visible way, near the top of the page.
39www.synerzip.com

40. Example 2: Possible Story Tests
 Mixing ‘Given/When/Then’ and ‘Test That’ patterns
AT-1.
Given a user that is logged in,
When the user clicks on the "logout link"
Then test that the system navigates the user to the home
page.
AT-2
Test that, after a logout navigates the user to the home
page, the system displays a message like "You have
successfully logged out.“ (logout message)
40www.synerzip.com

41. Example 3
 User Story Title: MKG-44 Limit Product Quantities
 User Story Conversations:
Pat: "For Marketing story 44, here is what I'm looking for. I'd like to limit
users to only being able to purchase a maximum quantity of 5 of any one
item in any one order."
Payton: "Hmmm... That seems weird. I would assume we want to make
as many sales as possible?"
Pat: "Yes, I know, but what we've found is that we often don't have the
inventory to be able to supply that many units. Also, many of the orders
for more than 5 items turn out to be fraudulent anyway. So, for now, we
just want to limit the quantity to be 5 items."
Charles: "Pat, do you have any vision for how we do this?"
Pat: "I don't really care -- just something that looks nice. Maybe show me
what you come up with and we can collaborate from there."
Charles: "Ok."
41www.synerzip.com

42. Example 3: Possible Story Tests
AT-1.
Test that a user is unable to purchase more than 5 of the
same item in any one order.
AT-2.
Test that, when a user attempts to purchase more than 5,
the system displays a message indicating no more than 5
of any item can be purchased.
42www.synerzip.com

43. Test Automation Pyramid/Tools
JMeter
Fitnesse
Cucumber
GreenPepper
xUnit, TestNG
SoapUI
Sellenium
QTP
Watir
Telerik
JMeter
xUnit, TestNG
http://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid
www.synerzip.com

44. Advanced Story Testing Patterns
 Bullet Points
 “Test with…”
 Work best with:
 Teams that are highly co-located with PO
 Stories that are very small (2-3 days)
 Tests that are very simple.
 Tests with fairly obvious expected behavior
44www.synerzip.com

45. Communication Mediums –
Advanced Story Testing Patterns
 Best
 Hand written (paraphrased) on 5 X 8 cards (Best)
 A wiki or whiteboard (2nd Best)
 ALM tool (Last resort)
45www.synerzip.com

46. Example – Bullet Points
46www.synerzip.com

47. Example “Test with…”
47www.synerzip.com

48. Special Story Testing Patterns
 Flowcharts
 State Diagrams
48www.synerzip.com

49. Communication Mediums –
Special Story Testing Patterns
 Best
 Picture of handwritten diagram from a whiteboard
(Best)
 Diagram in drawing tool, but only if the logic is complex
enough to warrant something more fancy than a hand
written diagram (2nd Best)
 Worst
 Fancy diagram in some fancy drawing tool that takes
way more time than it should.
49www.synerzip.com

50. Example 1– Flow Charts
50www.synerzip.com

51. Example 2– Flow Charts
51www.synerzip.com

52. Example – State Diagrams
52www.synerzip.com

53. Summary
 Don’t be afraid to mix and match, even within the same
story!
 Story Tests are the most important part of any User Story,
and they are the result of conversations.
 Tip for Beginners: Start with “Test that…” and work your
way up from there.
 Tip for Intermediates: Start working in new patterns,
where they fit nicely.
 Try to get as many Story Tests automated as you can. Test
automation is essential to superior Agility.
 Closing Tip: Story tests are often a great way to slice User
Stories!
53www.synerzip.com

54. Resources
 One Page PDF Chart with all Story Testing Patterns
 http://ScrumCrazy.com/STPChart
 Article: User Story Basics
 http://www.scrumcrazy.com/User+Story+Basics+-
+What+is+a+User+Story%3F
 Article: The Bradley User Story Maturity Model
 http://www.scrumcrazy.com/The+Bradley+User+Story+Matu
rity+Model
 To contact me:
 Charles@ScrumCrazy.com
 To download presentation:
 http://www.ScrumCrazy.com
 Click on “Presentations”
54www.synerzip.com

55. 5555
55
Questions?
www.synerzip.com
www.synerzip.com
Hemant Elhence
hemant@synerzip.com
469.322.0349

56. Synerzip in a Nut-shell
1. Software product development partner for small/mid-sized technology
companies
 Exclusive focus on small/mid-sized technology companies, typically
venture-backed companies in growth phase
 By definition, all Synerzip work is the IP of its respective clients
 Deep experience in full SDLC – design, dev, QA/testing, deployment
2. Dedicated team of high caliber software professionals for each client
 Seamlessly extends client’s local team, offering full transparency
 Stable teams with very low turn-over
 NOT just “staff augmentation”, but provide full mgmt support
3. Actually reduces risk of development/delivery
 Experienced team - uses appropriate level of engineering discipline
 Practices Agile development – responsive, yet disciplined
4. Reduces cost – dual-shore team, 50% cost advantage
5. Offers long term flexibility – allows (facilitates) taking offshore team captive
– aka “BOT” option
www.synerzip.com

57. Our Clients
www.synerzip.com
Our Clients

58. 5858
58
Call Us for a Free Consultation!
Thanks!
www.synerzip.com
www.synerzip.com
Hemant Elhence
hemant@synerzip.com
469.322.0349