The changing role of testing and test automation in the increasingly fast-paced world of continuous delivery and automated acceptance testing. Learn how, in a DevOps environment, testing activities start with requirements discovery and definition, playing a vital role in not only detecting defects, but preventing them, and ensuring not only that the features are built right, but the right features are built. And learn how test automation needs to happen during, not after, the sprint, and how you can achieve this. Despite rumors to the contrary, the role of the tester is not diminished with the arrival of automated DevOps, with its ultra-rapid deployment cycles and its emphasis on automation. On the contrary, testers play a vital role in ensuring that the code that gets deployed ten times a day is worth deploying.

1. #DevoxxPL#DevoxxPL @wakaleo
SHIFT-LEFT
The role of the tester
in a DevOps World
@wakaleo
www.johnfergusonsmart.com

2. #DevoxxPL#DevoxxPL @wakaleo
Change is not constant
Change is accelerating

3. #DevoxxPL#DevoxxPL @wakaleo
Don’t work in silos
Modern test teams…

4. #DevoxxPL#DevoxxPL @wakaleo
Prevent more bugs than they find
Modern test teams…

5. #DevoxxPL#DevoxxPL @wakaleo
Engage early
Modern test teams…

6. #DevoxxPL#DevoxxPL @wakaleo
Master automation
Modern test teams…

7. #DevoxxPL#DevoxxPL @wakaleo
Play with production
Modern test teams…

8. #DevoxxPL#DevoxxPL @wakaleo
How can testers stay at the top of their
game?

9. #DevoxxPL#DevoxxPL @wakaleo
440x faster lead time
5x lower change failure rate
DevOps is…
96x faster mean time to recover

10. #DevoxxPL#DevoxxPL @wakaleo
But deploying a broken product faster
does not make it any less broken

11. #DevoxxPL#DevoxxPL @wakaleo
But deploying a useless product faster
does not make it any more useful

12. #DevoxxPL#DevoxxPL @wakaleo
Rapid
ReliableRelevant
Quick to deploy,
quick to test
Verified by
automated testing
at multiple levels
Driven by a deep
understanding of
business needs
DevOps is…

13. #DevoxxPL#DevoxxPL @wakaleo
Shift-Left
Requirements
Discovery
Stretch right
Production
Monitoring
and testing
LiftupBetter, faster automationPushout
Exploratory testing
Traditional
testing

14. #DevoxxPL#DevoxxPL @wakaleo
Shift-Left
Requirements
Discovery
Stretch right
Production
Monitoring
and testing
LiftupBetter, faster automationPushout
Exploratory testing
Traditional
testing

15. #DevoxxPL#DevoxxPL @wakaleo
The $600,000 Tester

16. #DevoxxPL#DevoxxPL @wakaleo
Left-Shifting
Embracing uncertainty
earlier rather than later

17. #DevoxxPL#DevoxxPL @wakaleo
Business
BA
Developers
As a corporate frequent flyer
I want to earn points on my business flights
So that I can get discounts on my holiday flights
Scenario: First class earns double points
Given I fly from London to Paris
When I am in First Class
Then I should earn twice as many points
…
Testers
TEST SCRIPT
…
…
…

18. #DevoxxPL#DevoxxPL @wakaleo
Business
BA
Developers
As a corporate frequent flyer
I want to earn points on my business flights
So that I can get discounts on my holiday flights
Scenario: First class earns double points
Given I fly from London to Paris
When I am in First Class
Then I should earn twice as many points
…
Testers
TEST SCRIPT
…
…
…
rework
misinterpretation
missed
opportunities

19. #DevoxxPL#DevoxxPL @wakaleo

20. #DevoxxPL#DevoxxPL @wakaleo
source:
- 44% - “Out of Control - Why Control Systems Go Wrong and How to Prevent Failure”
- 56% - “An Information Systems Manifesto”
- 80% - “Requirements: A quick and inexpensive way to improve testing”
44-80%of all defects are caused by unclear,
ambiguous or incorrect requirements

21. #DevoxxPL#DevoxxPL @wakaleo
Business
BA
Developers
Testers
As a corporate frequent flyer
I want to earn points on my business flights
So that I can get discounts on my holiday flights
Examples
Rules
Acceptance Criteria
SHARED
UNDERSTANDING
Executable
Specifications
Scenario: First class earns double points
Given I fly from London to Paris
When I am in First Class
Then I should earn twice as many points
Scenario: First class earns double points
Given I fly from London to Paris
When I am in First Class
Then I should earn twice as many points
Development and
exploratory testing

22. #DevoxxPL#DevoxxPL @wakaleo
The Advocate

23. #DevoxxPL#DevoxxPL @wakaleo
The Sceptic

24. #DevoxxPL#DevoxxPL @wakaleo
The Cross-Examiner

25. #DevoxxPL#DevoxxPL @wakaleo
Advocate
Sceptic Cross-Examiner
What should it do?
How can it work?
What can go wrong?
How can we prove it works?

26. #DevoxxPL#DevoxxPL @wakaleo
Shift-Left
Requirements
Discovery
Stretch right
Production
Monitoring
and testing
LiftupBetter, faster automationPushout
Exploratory testing
Traditional
testing

27. #DevoxxPL#DevoxxPL @wakaleo
If you don’t master automation
You become a slave to automation

28. #DevoxxPL#DevoxxPL @wakaleo
Are wary of vendor hype
Successful DevOps teams

29. #DevoxxPL#DevoxxPL @wakaleo
Treat their test automation code
like their production code
Successful DevOps teams

30. #DevoxxPL#DevoxxPL @wakaleo
Make their application testable
Successful DevOps teams

31. #DevoxxPL#DevoxxPL @wakaleo
Use layers of abstraction well
Successful DevOps teams

32. #DevoxxPL#DevoxxPL @wakaleo
Obsess over feedback
Successful DevOps teams

33. #DevoxxPL#DevoxxPL @wakaleo
Good DevOps tests are FAST
Fast

34. #DevoxxPL#DevoxxPL @wakaleo
Good DevOps tests are FAST
Fast
Actionable

35. #DevoxxPL#DevoxxPL @wakaleo
Good DevOps tests are FAST
Fast
Actionable
Scalable

36. #DevoxxPL#DevoxxPL @wakaleo
Good DevOps tests are FAST
Fast
Actionable
Trustworthy
Scalable

37. #DevoxxPL#DevoxxPL @wakaleo
Shift-Left
Requirements
Discovery
Stretch right
Production
Monitoring
and testing
LiftupBetter, faster automationPushout
Exploratory testing
Traditional
testing

38. #DevoxxPL#DevoxxPL @wakaleo
FAKE NEWS!

39. #DevoxxPL#DevoxxPL @wakaleo
Practice Continuous Testing
Successful DevOps teams

40. #DevoxxPL#DevoxxPL @wakaleo
Unleash the Monkeys of Chaos
"The best defence against major unexpected failures is to fail often.
By frequently causing failures, we force our services to be built in a
way that is more resilient”
- Netflix engineer Cory Bennett and executive Ariel Tseitlin

41. #DevoxxPL#DevoxxPL @wakaleo
Shift-Left
Requirements
Discovery
Stretch right
Production
Monitoring
and testing
LiftupBetter, faster automationPushout
Exploratory testing
Traditional
testing

42. #DevoxxPL#DevoxxPL @wakaleo
SCARLETCRIMSON

43. #DevoxxPL#DevoxxPL @wakaleo
REDRED

44. #DevoxxPL#DevoxxPL @wakaleo
Test automation should serve the team
Not the other way round.

45. #DevoxxPL#DevoxxPL @wakaleo
The six stages of QA DevOps Maturity
Name Focus Benefits

46. #DevoxxPL#DevoxxPL @wakaleo
The six stages of QA DevOps Maturity
Name Focus Benefits
Siloed

47. #DevoxxPL#DevoxxPL @wakaleo
The six stages of QA DevOps Maturity
Name Focus Benefits
Colocated Breaking down the silos Reduce delivery bottlenecks
Siloed

48. #DevoxxPL#DevoxxPL @wakaleo
The six stages of QA DevOps Maturity
Name Focus Benefits
Engaged Improving collaboration Prevent bugs and deliver the right product
Colocated Breaking down the silos Reduce delivery bottlenecks
Siloed

49. #DevoxxPL#DevoxxPL @wakaleo
The six stages of QA DevOps Maturity
Name Focus Benefits
Mechanised Effective automation Deliver the right product faster
Engaged Improving collaboration Prevent bugs and deliver the right product
Colocated Breaking down the silos Reduce delivery bottlenecks
Siloed

50. #DevoxxPL#DevoxxPL @wakaleo
The six stages of QA DevOps Maturity
Name Focus Benefits
Pioneering Leverage Delight the customer
Mechanised Effective automation Deliver the right product faster
Engaged Improving collaboration Prevent bugs and deliver the right product
Colocated Breaking down the silos Reduce delivery bottlenecks
Siloed

51. #DevoxxPL#DevoxxPL @wakaleo
The six stages of QA DevOps Maturity
Name Focus Benefits
Projecting Experimentation Deliver value faster, and learning from it
Pioneering Leverage Delight the customer
Mechanised Effective automation Deliver the right product faster
Engaged Improving collaboration Prevent bugs and deliver the right product
Colocated Breaking down the silos Reduce delivery bottlenecks
Siloed

52. #DevoxxPL#DevoxxPL @wakaleo
Would you like to know more?
Subscribe to the newsletter
https://www.johnfergusonsmart.com
Read the book!

53. #DevoxxPL#DevoxxPL @wakaleo
QUESTIONS?
www.johnfergusonsmart.com
@wakaleo