Acceptance test-driven development (ATDD), behavior-driven development (BDD), and Cucumber promise many benefits related to your user story acceptance tests. They promise tighter collaboration between the product owner and the team. They promise the ability for the product owner and other stakeholders to write their own executable acceptance tests. They even promise an increase in the value produced by the efforts of your team as they focus on building the “right” products. But promises are not always tied to reality. Join experienced agile coach Mary Thorn as she explores three case studies of implementing ATDD and BDD with Cucumber in real world environments. Mary explores each study in detail, showing how she introduced ATDD/BDD tooling and practices, how far she was able to advance toward the promises of ATDD, her implementation strategies, and how she overcame resistance. If you're struggling with or hoping to introduce ATDD/BDD, this session will help you develop your own strategies and prepare you for the challenges.

1.
W2
Test
Techniques
5/4/16
11:30
Acceptance-­‐
and
Behavior-­‐Driven
Development
with
Cucumber:
Three
Case
Studies
Presented
by:
Mary
Thorn
Ipreo
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073

2. 888-­‐-­‐-­‐268-­‐-­‐-­‐8770
·∙·∙
904-­‐-­‐-­‐278-­‐-­‐-­‐0524
-­‐
info@techwell.com
-­‐
http://www.stareast.techwell.com/
Mary
Thorn
Ipreo
Chief
Storyteller
of
The
Three
Pillars
of
Agile
Testing
and
Quality,
Mary
Thorn
is
Director
of
Software
Test
Engineering
at
Ipreo
in
Raleigh,
NC.
Mary
has
a
broad
testing
background
that
spans
automation,
data
warehouses,
and
web-­‐based
systems
in
a
wide
variety
of
technologies
and
testing
techniques.
During
her
more
than
nineteen
years
of
experience
in
healthcare,
HR,
financial,
and
SaaS-­‐
based
products,
Mary
has
held
manager-­‐
and
contributor-­‐level
positions
in
software
development
organizations.
A
strong
leader
in
agile
testing
methodologies,
Mary
has
direct
experience
leading
teams
through
agile
adoption
and
beyond.

3. Investment Banks. Investors. Investor Relations.
New York | London | Raleigh | Capetown | Boston | www.ipreo.com
Acceptance- and Behavior-Driven Development with Cucumber:
Three Case Studies Presented

4. About Mary Thorn
 Chief Story Teller of the book “The Three Pillars of Agile Testing
and Quality” written by Bob Galen, Mary Thorn is Director of
Software Test Engineering at Ipreo in Raleigh, NC.
 Mary has a broad testing background that spans automation,
data warehouses, and web-based systems in a wide variety of
technologies and testing techniques. During her more than
seventeen years of experience in healthcare, HR, financial, and
SaaS-based products,
 Mary has held manager and contributor level positions in
software development organizations. A strong leader in agile
testing methodologies, she has direct experience leading teams
through agile adoption and beyond.

5. What is the problem?
 Have you ever had overlooked requirements?
 Have you ever built the wrong thing?
 Do you have tangible reference materials?
 Have you ever interpreted a requirement incorrectly?

6. The Communication Problem
The Challenge:
 Today, the most difficult problem in software development is knowing
what to build, not how to build it.
 Knowing what to build, requires knowing why it is being built. In other
words, understanding the goal.

7. The Communication Problem
The Problem:
 Stories typically concentrate on the what and the how, not the
why. Goals and examples are usually missing.
 Stories are usually a list of imperative requirements (acceptance
criteria).
 Imperative requirements are usually interpreted in subtly different
ways by each person. If goals and examples are absent,
misinterpretation is much more likely.

8. How do you get started?
 To PO boss and PO’s
 To Testers boss and Tester
 To your boss
 To Scrum team

9. Training
 User story and AC writing training
 Feature File Training
 Grooming Training
 Cucumber Training

10. Case Study 1: Greenfield

11. Case Study 1: Greenfield
How to effectively communicate requirements to the team, to be
developed/tested/implemented while exceeding business expectations.
The Challenge:

12. The Process: Agile
 Product Owner writes acceptance tests in Gherkin format in the
story.
 During grooming, the team discusses the scenarios and asks
questions. Team can add/remove scenarios during grooming or
team updates after.
 During sprint planning, the feature file is used for estimating
tasks.
 Once sprint has started, Testers and Dev work on getting the
feature file 90% complete before the developer starts coding.
 The developers code the scenarios in the feature file in a TDD
manner.
 Testers expands the scenarios in the feature files after
development if there are new findings and gets approval from the
team. Team owns the feature file.
 Automation starts in parallel to development.
 When testing is complete, PO acceptance is done by running the
automated acceptance tests.

13. Case Study 1: Greenfield
Description: In lieu of creating an Information page (as detailed in HZN-6805), we will instead
provide a link to open a new browser for the applications in Internet Explorer, similar to how we will
handle the news items. The application-specific information pages contain a detailed summary of
Horizon’s offerings and serve as a marketing tool for potential users. Ultimately, launching this page
directly is a simpler and cheaper solution than building a new window.
Acceptance Tests:
1.) An “i" icon is shown on each application tile, in the All
Apps view. This icon is not displayed in the My Apps view.
2.) If I am entitled to an application and it is live in
Production, or if the application is coming soon, or if I am not
entitled to it, then clicking the “i” icon will open the info page
for that application.
3.) If I am entitled to an application and it is live in
Production, then clicking anywhere else in the tile (i.e.: NOT
on the "i" icon) will open the application.
4.) If the application is coming soon, then the following
message is shown when they click anywhere else in the tile
(i.e.: NOT on the "i" icon):
This app is not available yet
Please select the info icon to view more information on the
Risk Intranet
5.) If the user is not entitled to the application, then the
following message is shown when they click anywhere else in
the tile (i.e.: NOT on the "i" icon):
You do not have access to view this app
Please select the info icon to view more information and
request access on the Risk Intranet
6.) If the application does not have an associated info page,
then clicking the “i” icon will open the Horizon homepage
(http://risk.intranet.db.com/content/horizon_coo_30587.html)
7.) All scenarios from feature file have been implemented
correctly.

14. Case Study 1: Greenfield
Example 1:
Verify clicking directly on the “I” icon launches a new internet page when the user is permissioned to the application and it is live in
production
Given that I'm on the My Apps page
And I am permissioned to an application that is live in production
When I click the “I” icon on the application tile
Then a new browser window opens
Example 2:
Verify clicking directly on the “I” icon launches a new internet page when the application is coming soon
Given that I'm on the My Apps page
And an application is coming soon
When I click the “I” icon on the application tile
Then a new browser window opens
And the All Apps page state is retained
Example 3:
Verify clicking in an area outside of the “I” icon (but within the tile) launches the application when the user is permissioned to the application
and it is live in production
Given that I'm on the My Apps page
And I am permissioned to an application that is live in production
When I click on an area outside of the “I” icon, within the tile
Then the application opens
Feature File

15. Benefits
 Clear expectations communicated
 Defined scope
 Reveals overlooked requirements
 Tangible reference materials
 Accurate story/task estimations
 Deliberate collaboration between Tester/DEV/PO
 Increased productivity
 Commitments are met (and often exceeded!)
 80% automated test coverage

16. Case Study 2: Backward Implementation

17. Case Study 2: Backward Implementation
 How to implement a new communication and/or automation tool
and have the entire Scrum team benefit from it.
 Team not building correct solution
 Lots of rework
 No clear understanding of what Tester was testing
The Challenge:

18. The Process: Agile
 Once sprint starts, tester writes acceptance tests in feature file
scenarios format for every story.
 Tester sends feature files to developers and product owner during the
sprint to verify assumptions.
 Developers use feature file to write unit tests post-implementation.
 Tester expands the scenarios in the feature files after development if
there are new findings.
 Tester runs feature file and manual testing by hand.
 Automation starts once manual testing is completed.
 When testing is complete, PO acceptance is done by running the
automated acceptance tests.

19. Case Study 2: Backward Implementation

20. Scenario: Verify subscriber is in contact information
Given that a list already exists in the account by default
When I log in to my account
Then I should see one subscriber on the list
And that subscriber should be the email address listed in
my contact information
Scenario:Verify subscriber is in contact information via API
Given I am using the API
When I create a list using the API
Then I should see one subscriber on my list
And the subscriber should be the email address listed
under my contact information
Scenario:Verify email is added to salesforce
Given that I am an iC4SFDC customer
When I create a campaign
Then my default email address should be added to
my campaign in iC4SFDC?
Are there some examples
here that you didn’t think
of during the exercise?
Case Study 2: Backward Implementation

21. Benefits
 Slower implementation rollout
 Clear expectations communicated
 Defined scope
 Reveals overlooked requirements
 Tangible reference materials
 Accurate story/task estimations
 Deliberate collaboration between QA/DEV/PO
 Increased productivity
 Commitments are met (and often times exceeded!)
 60% automated test coverage

22. Case Study 3: Hybrid

23. Case Study 3: Hybrid
 No effective communication
 Long FSD/BRDs leading to ambiguities in requirements
 No collaboration between BAs and QA/Dev teams on requirements
stage
 Lots of issues found during BA sanity checks and UAT
 No test automation
The Challenge:

24. The Process: Waterfall
 BA writes acceptance tests in feature file scenarios format
in the story.
 BA discusses feature file with the developer and QA working on
the project.
 Tester and Dev estimate their project tasks based on feature file.
 The developers code to the scenarios in the feature file and,
when done, pass over to Test.
 Tester expands the scenarios in the feature files after
development if there are new findings.
 Tester runs feature file and manual testing by hand.
 Automation starts once manual testing is completed.
 When testing is complete, BA acceptance is done by running the
automated acceptance tests.

25. Case Study 3: Hybrid
 Questions that might arise during discussion within the
team:
 What if GBP trade is maturing on 01-Apr, should the
tenor be adjusted?
 Why are we defining the holiday by currency? What if
there is a holiday in France but not in Germany?
Report COB Maturity Date of
the trade
Original tenor Currency Holiday for
Currency
Payment Date
in the reports?
Adjusted Tenor?
28-Mar-2013 29-Mar-2013 D1 GBP GBP: 29-Mar-2013
01-Apr-2013
02-Apr-2013 D3
28-Mar-2013 29-Mar-2013 D1 CAD CAD: 29-Mar-2013 01-Apr-2013 D2
28-Mar-2013 29-Mar-2013 D1 JPY No holiday for JPY on
29-Mar
29-Mar-2013 D1
28-Mar-2013 02-Apr-2013 D3 GBP GBP: 29-Mar-2013
01-Apr-2013
02-Apr-2013 D3
28-Mar-2013 06-May-2013 D27 GBP GBP: 06-May-2013 06-May-2013 D27
Discuss with
the team and
business
Refine
examples
and/or
acceptance
criteria

26. Case Study 3: Hybrid
Example 1:
Given that the trade is maturing on report COB+1
And the COB+1 is a holiday for trade currency
When the trade is loaded to Datamart reports
Then the maturity tenor of the trade is adjusted to fall into next business
day for the trade’s currency
Example 2:
Given that the trade is maturing on report COB+N (N>1)
And the COB+N is a holiday for trade currency
When the trade is loaded to Datamart reports
Then the maturity tenor of the trade is reported on DayN without
adjustment for holiday

27. Benefits
 Clear expectations communicated
 Defined scope
 Reveals overlooked requirements
 Tangible reference materials
 Accurate story/task estimations
 Deliberate collaboration between Test/DEV/PO
 Increased productivity
 Decreased bugs from SIT/UAT/PROD
 30% automated test coverage

28. Why does it fail?
 PO’s don’t think it is there jobs
 Regulatory Scare
 Testers are not bought in
 Automation is not utilized

29. Wrap-Up

30. Questions