How DUO started with Continuous Delivery and changed their way of Testing

1. Software Development Done Right
How DUO started with
Continuous Delivery
and changed their way
of Testing
Kishen Simbhoedatpanday
CTO Xebia Test Automation
Xebia Test Automation

2. DUO

3. Technical implementation is easy
Selecting testing tools
BDD with Serenity
Changing mindset of people is harder
Moving away from Quality Assurance
BDD & Specification by Example
The consequences of
Continuous Delivery @ DUO

4. Testing is evolving
Agile Testing

5. Testers 1.0
Agile Testing

6. Testers 1.0
Agile Testing
Technical
Unit
Tes-ng
System
Tes-ng
User
Acceptance
Tes-ng
Performance
&
Security
Tes-ng

7. Testers 2.0
Agile Testing

8. Scrum fall
Todo
In
Progress
In
Test
Done
Task
15
Task
13
Task
14
Task
3
Task
4
Task
5
Task
6
Task
7
Task
8
Task
9
Task
10
Task
11
Task
12
Task
1
Task
2
Agile Testing

9. Building features
Agile Testing

10. Agile Testing
Mom
to
her
son:
“Son,
please
go
to
the
market
and
buy
1
boKle
of
milk.
If
they
have
eggs,
bring
6
”
What
did
he
bring
home?

11. When do you want to know
you have a problem?
Agile Testing

12. Agile Testing
Agile Testing

13. Testing 3.0
Agile Testing
The Ambassadors of Quality

14. Bridging the gap
Agile Testing
Mutual understanding with the whole business towards testing

15. Knowledge sharing is key to innovation
Coding Dojos
(Java)
Workshops
(BDD & Specification by
Example, Cucumber &
Serenity)
Selecting test-tools for each challenge in testing

16. Are testers still relevant?
Yes! but…
Agile Testing
Think Quality Assistance not
Assurance
Developers eating own dog food
Team takes responsibility in testing
Test Automation to support CD
Building Quality into the process

17. Agile Testing

18. BDD &
Specification by Example
Agile Testing

19. What is BDD?
BDD & Specification by Example
Process of driving desired behaviour of software
Explore
Discover
Define

20. Challenge requirements first
BDD & Specification by Example

21. Explore and Decide
BDD & Specification by Example

22. Test first with conversations
BDD & Specification by Example

23. What is it?
BDD & Specification by Example

24. Acceptance Tests
Scenario:
Search
for
ar5cles
by
keyword
Given
I
want
to
buy
a
wool
scarf
When
I
search
for
‘wool’
Then
I
should
see
ar-cles
related
to
wool
BDD & Specification by Example

25. Free Delivery
We offer free delivery to the customer when the total price of the
Shopping Cart is equal or greater than 20,00 EUR
Acceptance Test with examples
When
products
ordered
Total
Price
in
€
Then
Delivery
costs
1
Book
19,99
3,95
2
Books
20,00
0,00
1
Book
20,01
0,00
1
Book,
1
Refrigerator
799,99
?
BDD & Specification by Example

26. Specification pitfalls
1. Too many AND
2. Too many columns
3. Getting stuck with Given
4. Too abstract
BDD & Specification by Example

27. Pitfall 1: Too many AND
Given the admin page is open
When the user types John into the 'employee name'
and the user types 30000 into the 'salary'
and the user clicks 'Add'
Then the page reloads
And the user types Mike into the 'employee name'
BDD & Specification by Example

28. Pitfall 1: Too many AND
Focus on what you want to specify
Given employee ‘John’ has salary 300000
When HR opens salary overview of employees
Then employee ‘John’ has salary 300000
BDD & Specification by Example

29. Pitfall 2: Too many columns
Scenario Outline: View Salary
Given employee has name <FirstName>
And employee <FirstName> has last name <LastName>
And employee <FirstName> has salary <Salary>
And employee <FirstName> has ……
When HR opens salary overview of employees
Then employee <Name> has salary <Salary>
And ….
And ….
BDD & Specification by Example
FirstName
LastName
….
….
….?
….?
Salary?
John
Doe
….
….
….
….
300000
Jane
Doe
….
….
….
….
400000

30. Pitfall 2: Too many columns
Focus on relevant input and output
Scenario Outline: View Salary
Given employee has name <FirstName>
And employee <FirstName> has salary <Salary>
When HR opens salary overview of employees
Then employee <Name> has salary <Salary>
BDD & Specification by Example
FirstName
LastName
Salary
John
Doe
300000
Jane
Doe
400000

31. Pitfall 3: Getting stuck with Given
Scenario: Search phone book
Given ... eeeh …
I am on the online phone book
I click … eeeh ...
Pffff…. This is hard!
BDD & Specification by Example

32. Pitfall 3: Getting stuck with Given
Focus on the business value (Then) first
Given I see a search result for florist “Floral Designs”
When I click “View map” of the search result
Then I find the location “Floriststreet 123”
Blog: http://blog.xebia.com/2014/10/17/when-then-given/
BDD & Specification by Example

33. Pitfall 4: Too abstract
Scenario: Pay the rent
Given I have enough money to pay the rent
When I land on a spot to pay rent
Then I pay the rent of the spot
BDD & Specification by Example

34. Pitfall 4: Too abstract
Focus on the essence of the spec
Given my account has 500 dollars
When I land on a spot to pay rent of 50 dollars
Then my account has 450 dollars
BDD & Specification by Example

35. Specification pitfalls
1. Too many AND
2. Too many columns
3. Getting stuck with Given
4. Too abstract
BDD & Specification by Example

36. Selecting test-tools
for each challenge in testing
Agile Testing

37. Test Automation
Selecting test-tools for each challenge in testing
Test Automation is coding!

38. Selecting test-tools
Selecting test-tools for each challenge in testing
Don’t hire QA army
Do small POC’s
Pick tools that fit the development team
Advice a tool, but leave room for innovation
Make sure it fits into the CD pipeline

39. Return on investment
of Test Automation
Lowest ROI
Highest ROI
Chain
Applica-on
Unit
/
Components
Selecting test-tools for each challenge in testing

40. What do you want to check?
Selecting test-tools for each challenge in testing
Chain
Applica-on
Unit
/
Components
Cucumber, Serenity, WebDriver
JUnit, Mocking, etc.

41. Security and Performance Testing
Selecting test-tools for each challenge in testing
Putting security & performance
specialist in CD teams
Avoid separation
Involved / Support in selecting tools
Remove privileges
Try to prevent security and
performance issues

42. Security and Performance Testing
Selecting test-tools for each challenge in testing
Dynamic Application Security Testing
HP Webinspect, Zap Proxy
Static Application Security Testing
HP Fortify
Performance Testing
JMeter, Silk Performer

43. Whole Team Approach
Selecting test-tools for each challenge in testing
Achieving same goal

44. BDD with Serenity & Cucumber
Agile Testing

45. Separating What from How
BDD with Serenity & Cucumber
What
How

46. Separating What from How
BDD with Serenity & Cucumber
Capabilities, Features & Cucumber steps
Given Customer has Bol.com account
When Customer orders products with
total price > EUR 20
Then Customer gets free shipping
Serenity steps & Page objects
Open productpage
productpage.clickOrder()
Open shoppingcartpage
Verify shoppingcartpage.getFreeDelivery() is …
What
How

47. Test Automation
BDD with Serenity & Cucumber
Serenity Capability
Description of
the problem
Cucumber Feature File
Description of the solution with examples
Cucumber Step Definitions
Glue Code to make scenarios executable
Serenity Steps Library
Support code for:
- Page Objects – Interaction with browser/pages
- Automatic Screenshot after every Serenity Step
Scenario: …
Given …
When …
Then …
Scenario: …
Given …
When …
Then …
Scenario: …
Given …
When …
Then …What
How

48. Serenity Reporting
BDD with Serenity & Cucumber

49. BDD with Serenity & Cucumber
Serenity Reporting

50. Technical implementation is easy
Small POC’s, focus on CD pipeline and fast feedback
Changing mindset of people is harder
Quality is a team effort.
BDD breaks barriers between development, testing and requirements
The consequences of
Continuous Delivery @ DUO

51. Software Development Done Right
Thank you!
Xebia Test Automation
Kishen Simbhoedatpanday
CTO Xebia Test Automation
@KishenPanday
ksimbhoedatpanday@xebia.com