The webinar discusses the integration of continuous testing in DevOps through the acquisition of Hiptest by SmartBear. It outlines a four-step process: testing the idea, testing the code, testing in production, and iterating based on measured impacts. Key practices include using behavior-driven development (BDD) and automation to enhance software quality and efficiency throughout the development cycle.

1. [ WEBINAR ]
Continuous testing with
Hiptest
A COMPANY

2. +
SmartBear Accelerates Continuous Testing With Acquisition Of Hiptest

3. PRESENTED BY
Laurent Py
@py_laurent
laurent@hiptest.com
https://hiptest.com
Vincent Pretre
vincent@hiptest.com
https://hiptest.com
Ryan Lloyd
ryan.lloyd@smartbear.com
https://smartbear.com

4. Webinar Agenda
Testing in DevOps
Step 1: Testing the idea
Step 2: Testing the code
Step 4 : Let’s iterate
Step 3 : Testing in production
Conclusion

5. Testing in DevOps

6. 6
SPEED!
DevOps: Biggest opportunity?

7. DevOps: Biggest opportunity?
Ideas
Define
Code
Test
Data
Learn
Minimize total time through the loop.

8. Testing in DevOps
1. Which value do we want to deliver?
2. Is the feature’s implementation aligned with that?
3. Does the feature have a real impact on the end user?
4. Is the UX great and what’s the feature usage in time ?

9. Step 1 : Testing the idea

10. Why?
Challenge business assumptions

11. Which value do we want to deliver?
• Use personae to create intimacy between users and the team
• Define business assumptions
• Behavior Driven Development (BDD) to capture the behavior

12. BDD in a nutshell
• Create a shared understanding based on examples
• Using a common business terminology
• Examples drive the development
• Once automated the examples become a living documentation

13. Testing process at Hiptest
2. Refine
with
examples
3.
Implement
the stories
4. Create
builds and
test runs
5. Deploy
in
production
6.
Exploratory
testing
1. Define
the stories

14. BDD – Capture the behavior

15. BDD – Capture the behavior

16. Good practices
• Using Gherkin syntax with declarative style
• Using a consistent business terminology
• Refactoring continuously your tests

17. Continuous testing
Test
Idea Dev Production
Test Test
Test continuously your software from idea to production

18. Step 2: Testing the code

19. Development
1. In step 1 we have created a shared understanding of the feature
2. The dev team can develop a first increment of the feature
3. Integration to CI/CD pipeline

20. Automation
100% of our checks (scenarios) are
automated
AND
It is expensive

21. Our automated tests
Credit to Martin Fowler
• 500 tests BDD
• 1250 tests Ember
• 4000 tests Rspec
UI
Service
Unit

22. Automate tests

23. Execute with CrossBrowserTesting

24. Build a layered approach

25. Parkeon’s story
1.5 M
50%
inhabitants
•trips made using public transport
1 M commuters a day
Customer : The city of Helsinki
•à Prime contractor : Tieto
•à Many partners as Parkeon
•à Teams distributed over 6
countries

26. 200
End to end tests
475 990 1635
Systems test Module
integration
tests
Unit module
tests
à More than 3000 scenarios
à Business terminology: 1700 steps
Key figures

27. Continuous testing
Test
Idea Dev Production
Test Test
Test continuously your software from idea to production

28. Step 3: Testing in production

29. Making the feature available
1. In Step 2 we have validated the implementation
2. The feature is rolled out using a switch (GA or progressive)

30. The match
Correctness VS Availability

31. Application performance management
Appsignal, Pingdom, Scalingo & Logmatic/Datadog

32. Measure usage continuously
• Measure usage
- NB users
- NB requests…
• Measure impact
- Sign-up
- Revenue
- Retention…

33. Examples
Notification feature
• Usage
- NB clicks NB of notifications
- NB users
• Business: none
Refactoring feature
• Usage
- NB refactoring actions
- NB of impacts
• Business: retention

34. Living Documentation & Product analytics

35. Continuous testing
Test
Idea Dev Production
Test Test
Test continuously your software from idea to production

36. Step 4: Let’s iterate

37. Let’s iterate
1. In step 3 we have measured the impact
2. The feature can be removed (with the tests), enhanced or validated
3. Go to step 1

38. Conclusion

39. Benefits of testing in DevOps
• Test value first
• Incremental investment in quality
• Context suitable to experiments

40. 40
Making something nobody want
The biggest risk

41. Laurent Py
@py_laurent
laurent@hiptest.com
https://hiptest.com
THANK YOU!
Vincent Pretre
vincent@hiptest.com
https://hiptest.com
Ryan Lloyd
ryan.lloyd@smartbear.com
https://smartbear.com