In this session I will outline/explore the journey of a common QA team without coding skills into Agile testing arena. Main focus on Acceptance Test Driven Development and executable specs. The session will be based on a real case study from Alcatel Lucent Haifa. At the end of the session you will understand the concept of executable specs,and ATDD, You will see real example of test implementation in ATDD tool (Cucumber) and will understand the steps required to make such transition with some do/not do tips in tool and process implementation (based on Alcatel case study). You will get (printed) the suggested implementation plan and do/not do tips of ATDD automation tools implementation

1. All Rights Reserved - AgileSparks
QA Team
Transition to
Agile Testing
Alcatel-Lucent Haifa – Case Study
Liat Arad-Bitan – QA manager, Alcatel-Lucent
Dr. Ronen Bar-Nahor – AgileSparks
1

2. All Rights Reserved - AgileSparks
Background
• Alcatel-Lucent (Haifa) specializes in content
management solutions for tier-1 carriers
• Decided to develop a new platform for
content management for SaaS providers
• Moving to new technology, development
environment and Agile 
• ~30 developers
• ~15 QA people (manual testing), 1
automation expert
2

3. All Rights Reserved - AgileSparks
What We Achieved
• Better quality … Continuously Working System!
• More than 90% coverage
• Refactoring with less fear and minimal regression efforts
• E.g. Replace the database … Components …
• Improved efficiency/reduced waste:
• Reduced code freeze period
• Test documentation maintenance
• We develop only what is needed
• We test only what was developed
• We have full traceability for free
• Re-use of automation code
• QA plays a key role in the Scrum team
• Better business understanding
• Provides value to the team (not just “bug hunters”)
• QA-Dev collaboration
• High motivation (but some left…)
3

4. All Rights Reserved - AgileSparks
Acceptance Test Driven
Development - ATDD
Add a test
Write some code
Tests passed?
Run tests
Refactor
Development
finished
Yes
No
No
Yes
20%
75%

5. All Rights Reserved - AgileSparks
5
QA implementation steps
• Agile training for QA leads, Exposure to ATDD
• Sprint “zero”
• Looking for ATDD tools (Jbehave, Cucumber, Fitnesse )
• POC for a simple story
• Decision on the BDD format
• Chosing Cucumber (easy to learn, active community)
• Train the team to write code
• Frustration and fear from writing code
• 1 week training on Java
• Train in ATDD and BDD writing
• Inspect and Adapt – more coaching 5

6. All Rights Reserved - AgileSparks
6
QA implementation steps
Tips
• Promote the new tester role
• Represents the PO within the team
• Train early the entire team on ATDD (fail first) and
how to write good BDD
• Understanding the framework (Text<->code) is a must!
• It is not so difficult to learn to write automation
• Automation champion to support the QA team  a must
• Need more focus on software engineering
• Scrum team support  shared ownership
• Read the Cucumber book!
• Join the community
6

7. All Rights Reserved - AgileSparks
TIP - What we expected from an
ATDD tool ?
• Express expectations in a language and format that allow
collaboration among the whole team
• Minimum of test code
• Test code written in the team’s “native” language
• To play nicely with source control systems and continuous
integration
• Pluggable to support a variety of interfaces (e.g. UI
automation)
• Can become a product spec (documentation tool)
• Active community
7

8. All Rights Reserved - AgileSparks
Acceptance Test Design
BDD approach - Behavioral Driven Development
• TIPS
• Working software over comprehensive
documentation
• No more huge textual test documents & documents
maintenance
• Document only the purpose, expected behavior and data
• Code the steps !
• Executable specs
• Describe ALL expected business behavior
• Decide later what will be tested in which level (UT, Manual, AT)

9. All Rights Reserved - AgileSparks
Acceptance for MMF – Minimal Marketable Feature
(MMF)
http://us.123rf.com/400wm/400/400/cokemomo/cokemomo1206/cokemomo1206
00007/13939893-mini-hamburgers-party-food.jpg
http://www.levensmiddelenkrant.nl/uploads/foto/0_61_ha
mburger_(Large).jpg
The smallest set of functionality that must be realized in
order for the customer to perceive value.

10. All Rights Reserved - AgileSparks
Will it always fit into iteration ?
• MMF describes the system
behavior  the product spec, not
how it was developed
• Might be split into user stories
• User story is a planning tool,
covers MMF’s scenarios, They exist
until implemented and then they
disappear !
TIP – Write Acceptance tests for MMF.
Will be expanded while elaborating
stories DoD.

11. All Rights Reserved - AgileSparks
BDD structure (Given-
When-Then)
Feature – [Feature/MMF title]
As a [role] I want [feature] So that [benefit]
Scenario 1: [Title/test purpose]
Given [context/system state]
And [some more context]...
When [event/action] and [ next event/action]
Then [outcome]
and [another outcome]...
Scenario 2: ...
TIPS:
• Business
Language
(and not
tester)
•Extract
Data
•Coaching

12. All Rights Reserved - AgileSparks
Examples
• Feature: Standard Signup
As a new user I want to create an account so that I can
use the application
Scenario: Signup with valid email/password combination
Given I do not have an account
When I signup with email and password
Then I should be logged in
And my profile details should be filled in

13. All Rights Reserved - AgileSparks
BDD Feature fileMMF
User story
tag
Test
purpose
In unit
test/manual
Extract
Data
Then …

14. All Rights Reserved - AgileSparks
BDD code (java)
14
• Given  initial state for the test

15. All Rights Reserved - AgileSparks
BDD code (java)
15
• When  user/system actions

16. All Rights Reserved - AgileSparks
BDD code (java)
16
• Then expected results

17. All Rights Reserved - AgileSparks
Frameworks, interfaces, drivers

18. All Rights Reserved - AgileSparks
TIP - BDD Design
• Integrate ATDD into the release process
• PO writes User Story description in Jira
• QA prepares BDD (after team sniffing)
• PO reviews with the QA (and dev rep)
• BDD is a condition for “ready story”
• It is a step in the Kanban flow
18

19. All Rights Reserved - AgileSparks
BDD as part of ready story.
Early integration (component teams)
19
Scrum
+BDD
Dev2dev
integration
Integration
points/NF
Writing the
BDD for a
US
E2E and NFT

20. All Rights Reserved - AgileSparks
20
TIPS - BDD Development
• Team commits to the BDD!!!
• Developer and QA review together the BDD and agree:
• What to test, by whom and how to automate it
(unit/acceptance)
• Comment the unit or manual steps/scenarios in the BDD
• Developer develops first the “API contract”
• QA develops the automation (code review with
developer)
• Developer works until BDD pass (end of iteration), QA
works on BDD of next iteration
20
Scrum Level 3
Sprint 1 Sprint 2 Sprint 3
PSPPSPPSP

21. All Rights Reserved - AgileSparks
21
TIPS - BDD Execution
• Automation of a new story is executed only in the
development environment (not to break the product
build)
• Once the Story is done (ATDD - all automation is pass) and
checked-in the main trunk also, the automation is
activated.
21

22. All Rights Reserved - AgileSparks
22
Continuous Integration
22

23. All Rights Reserved - AgileSparks

24. All Rights Reserved - AgileSparks

25. All Rights Reserved - AgileSparks

26. All Rights Reserved - AgileSparks
26
Make tests independent
• Init state - Automate static test data creation
• System level (all features) - Develop set-up utility
• Feature level - Use Given step (without action)
• Scenario level - Use background (cucumber)
• Roll back system state
• Delete scenario data at the end of the test scenario
• Delete setup data at the end of test execution
26

27. All Rights Reserved - AgileSparks
27
Re-use of steps/code
• BDD steps reuse (execution)
• Requires common data accessibility, e.g.:
• Common member/code
• Common data structure results
• Change the data (BDD), re-use the code
• Java utilities for actions/common behavior
• Includes API calls that shall be easily replaced
27

28. All Rights Reserved - AgileSparks
28
Re-use example
Feature - Add User
Given system admin exists
When creating user with attributes:
id|name | phone |
1 | Ronen | 00000 |
Then user created successfully
id|name | phone |
1 | Ronen | 00000 |
28
Feature - Update User
Given creating user with attributes:
id|name | phone |
1 | Ronen_1 | 55555 |
And fetching user data
|name |
|Ronen_1 |
when updating user with values:
id| name | phone |
1 | Liat | 00001 |
Then user updated successfully:
id| name | phone |
1 | Liat | 00001 |
Feature Get User
Given creating user with attributes:
id| name | phone |
1 | user test | 21234545 |
When fetching user data
|name |
|user test |
Then user is fetched:
id|name | phone |
1 | user test | 21234545 |

29. All Rights Reserved - AgileSparks
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
E2E Re-Use Example
“Saas provider” component
Module (source)
Acceptance tests
Java
Features
“Admin user” component
Module (source)
Acceptance tests
Java
Features
“Common” testing
Java
E2E testing
Features
jar
jar
jar
Given system admin logs in
When calling createSaasProvider API
with parameters: “Saas1”
Then Saas Provider “Saas1” is created
Given Saas provider Saas2 in the system
When calling createAdminUser API with
parameters “Admin1”
Then admin user “Admin1” is created
Given system admin logs in
When calling createSaasProvider API with
parameters: “Advanced TV”
And calling createAdminUser API with
parameters “Advanced TV Admin”
Then Saas Provider “Advanced TV” is created
And admin user “Advanced TV Admin” is
created

30. All Rights Reserved - AgileSparks
GUI automation
• Development framework AngularJS
• Carma – UT (for java script) + AT
• Phantom browser (better performance)
• Any browser (E2E)

31. All Rights Reserved - AgileSparks
What next
• Schedule work based on E2E priorities (to
reduce cycle time)
• Improve the Dev2Dev integration test
• Executable spec …
31

32. All Rights Reserved - AgileSparks
THANKS !
32

33. All Rights Reserved - AgileSparks
33
Component, Integration &
E2E Testing
• Component testing
• A Component is an independent unit  expose services
• Tested with stubs in local component environment
• MMF might be developed across components
• MMF Integration testing (“white box”)
• Risks areas mapped during MMF architecture review
• Team exposes API for the early NFT and integration automation
• Integration testing done by the teams in integration environment
• E2E testing (“black box”)
• No new java code- all reused from components acceptance tests
• New BDD (feature files ) are written describing the flow(s) based on
existing BDDs 33

34. All Rights Reserved - AgileSparks
Main Agile Implementation
steps
• Working with AgileSparks from day1
• 2 days Management Workshop
• Agile principles (e.g. Kanban+Scrum, Done is Done, Zero
defects)
• Backlog preparation
• Story Mapping to create first MVP
• Tool decision (Jira – already exist)
• 1 day Agile Training to all teams (Agile thinking &
Scrum)
• On-the-job coaching
34