The document discusses continuous testing in a DevOps pipeline. It describes a 4 step process: 1) Testing the idea by defining user stories and assumptions with BDD, 2) Testing the code by automating tests and integrating them into CI/CD, 3) Testing in production by monitoring metrics and performance, and 4) Iterating based on usage data and completing the test/deploy cycle. The benefits are validating value, incremental quality improvements, and an environment suitable for experiments.

1. Accelerate Your
Delivery Pipeline With
Continuous Testing

2. 2
Vincent PrêtreLaurent Py
Presented by

3. 3
Agenda
Testing the code
Testing the idea
Testing in DevOps
Testing in production
Let’s iterate
Step 1 :
Step 2 :
Step 3 :
Step 4 :
Conclusion

4. 4
Testing in DevOps

5. 5
DevOps: Biggest Opportunity
SPEED!

6. 6
DevOps: Biggest Opportunity
Ideas
Define
Code
Test
Data
Learn
Minimize time

7. 7
Continuous Testing at Hiptest
2. Refine
with
examples
3.
Implement
the stories
4. Create
builds and
run tests
5. Deploy
in
production
6.
Exploratory
testing
1. Define
the stories

8. 8
1. Which value do we want to deliver?
2. Is the feature’s implementation aligned with that?
3. Does the feature have a real impact on the end user?
4. What’s the feature usage & UX over time ?
Testing in DevOps

9. 9
Step 1 : Testing the Idea

10. 10
Challenge Business Assumptions

11. 11
• Use personas/JTBD to create intimacy between users and the team
• Define business assumptions
• Use Behavior Driven Development (BDD) to capture the behavior
Which Value Do We Want to Deliver?

12. 12
What’s BDD?
Collaborate on
Requirements Automate Your Tests
Create & Share Your Living
Documentation

13. 1313
Scenario Example

14. 1414
Continuous Testing
Test
Idea Dev Production
Test Test
Test continuously your software from idea to production

15. 15
Step 2 : Testing the Code

16. 16
1. In step 1 we have created a shared understanding of the feature
2. The development team can develop a first increment of the feature
3. Integration to CI/CD pipeline
Development

17. 1717
Automation
of our checks (scenarios) are automated
100%
And, it’s expensive!

18. 1818
Our Automated Tests
• 500 tests BDD
• 2500 tests Ember
• 8000 tests Rspec
11
k

19. 1919
Right Level of Automation

20. 2020
Continuous Testing
Test
Idea Dev Production
Test Test
Test continuously your software from idea to production

21. 21
Step 3 : Testing in Production

22. 22
1. In Step 2 we have validated the implementation
2. The feature is rolled out using a switch (GA or progressive)
Making the Feature Available

23. 2323
The Match
Availability VS Correctness

24. 2424
Application Performance Management
Appsignal, AlertSite, Scalingo & Logmatic/Datadog

25. 2525
Living Documentation & Product Analytics
Business metrics

26. 2626
Living Documentation & Product Analytics

27. 27
1. Feature behaviors
2. Feature history
3. Test results
4. Feature usage & impact
A Single Place to Collaborate On

28. 2828
Continuous Testing
Test
Idea Dev Production
Test Test
Test continuously your software from idea to production

29. 29
Step 4 : Let’s Iterate

30. 30
• Usage
• Completion of examples
• Performances
Available Metrics

31. 31
1. In step 3 we have measured the impact
2. The feature can be removed (with the tests), enhanced or validated
3. Go to step 1
Let’s Iterate

32. Conclusion

33. 3333
Benefits of Testing in DevOps
Test value first
Incremental investment in quality
Context suitable to experiments
1
2
3

34. 34
The biggest risk is
making something
that nobody wants
And Remember…

35. 35
Thank you!