This presentation I gave at Testnet an is about the transition to continuous delivery and the agile adaption of Stater. Where visualisation is a key in the approach. The setup of a Testing as a service, with a testing framework based on Selenium, Fitnesse, XLrelease , XLdelay and XLtestview

1. Road to DevOps @ Stater
Pascal Dufour
(Agile test consultant)

2. www.stater.nl
1

3. Marketshare
2
• Market share at the start of 2014: 30%

4. Marketshare
3
• Market share at the start of 2014: 30%
• Market share Q1 2015: 40%

5. Marketshare
4
• Market share at the start of 2014: 30%
• Market share Q1 2015: 40%
• Goal: 70 % market share by the end of 2018

6. Innovation
55
Financieel akkoord
Snel zekerheid voor de
consument
Bouwdepot
Mijn gegevens

7. 3 pilars for growth
6
70% market share
New
customers
Expanding
BPO offering
(mid office) Innovation
(digitization)

8. Our IT landscape
7

9. Our challenges
•Unsatisfied customers
•Unstable testing environments
•Painful releases
•System outage
•Incidents

10. Our plan
9
CD pipeline
Scrum
Devops

11. Delivery was “old school”

12. The answer: Continuous Delivery
Ontwikkel
Omgeving
Test
Omgeving
Acceptatie
Omgeving
Productie
Omgeving
Automated provisioning
Agile software
development
Continuous integration Automated testing
Automated deployment
12

13. CD Pipeline
13
XL testview

14. Automated flows for releases
15

15. Paradigm shift
16
• Release more frequently
• Use tools that best fit the job
• Fewer code bases
• Automate repetitive tasks

16. Agile software development

17. Where to start with Agile:
. You are here
18

18. It’s a project:
19

19. Scrum
20http://blog.manageagile.com/2012/06/finding-right-team.html

20. 26 agile teams
21
TAAS
- Test framework
- Coaching
- Test automation

21. Added Operation engineers
22
TAAS
- Test framework
- Coaching
- Test automation

22. TAAS23
Taas environment
costumer
business

23. Maturity matrix
Maturity
24
LEVEL
5 Agile Cultuur Zorgen dat agile mindset onderdeel van cultuur
blijft door een levendige omgeving te creëren.
4 Adaptief Reageren op wijzigingen door meerdere niveaus
van feedback
3 Effectief Ontwikkelen van software op een effectieve
manier
2 Evolutionair Opleveren van software eerder en meer frequent
1 Samenwerking Communicatie en samenwerking vergroten
translate

24. Maturity matrix
25

25. Maturity matrix => actions
26
Team D
Team B
Team
C

26. Maturity matrix => actions
Team DTeam C
Team B
27

27. Testing example
The User Acceptance Test in a Day
Automation
• Workshops to transition from manual testing to Living Documentation.
• Use of Persona
• External help to automate
• Run in CI
• Run in CD
28

28. Stater - 3 minuten demo29
Manual testing
Regression testing on the
acceptance environment
Automatische
Checks
REGRESSIE TESTEN OP
ACCEPTATIE OMGEVING
Handmatige
Testen
Operations
At the driver seat
Automation of UAT

29. Stater - 3 minuten demo30
Dev
Test
Acceptance Production
Manual
TestingAutomatic
Checks
Manual
Testing
Automatic
Checks
Manual
Testing
Automation current state

30. Succes
31
First test in the
continuous delivery
pipeline
Took us
10 weeks
Setup. more setup. more setup more, more setup, almost there, more more setup
test Lening in beheer

31. Stater - 3 minuten demo32
Automatische
Checks
Regression on Test environment
Manual
Testing
STAP 1
Flakiness
0

32. What are we testing?
33
Visualise!

33. Build monitors for real time insights
34

34. Build monitors for real time insights
35

35. Build monitors for real time insights
36

36. Bereikt de afgelopen 2 sprints
Stater - 3 minuten demo37

37. Test Automation ‘praat[laat’
Sprint 0 Sprint 1 Sprint 2 Sprint 3
Kickoff
Pilot
TA
Uitleg
Refinement + Retro
Pilot
Retro
Pilot
Refinement
Development
Review en
Fitnesse scenario
support
Sprint n
Pilot
team
TAAS
SM
Dev
Team +
business
Agile
Intake
inspanninginspanning
Regelt samen
met Agile
coach intake
TA
coach
presentatie
plan
TA
coach
Uitleg
Dev
Team /
Dev / test
Dev
Team +
Business +
initiator
Agile
coach
SM / POSM
SM /
TA
coach
SM /
TA
coach
Bouw US
+ Automatisering
scenario
TA
coach
Select
User story
Support op aanvraagTA
coach
TA
coach
Dev
Team +
Business +
TA
coach
Dev
Team
Dev
Team +
Business +
SM SM
Dev
Team +
Business +
TA
coach
TA
coach
SM SM
Refinement +
Review en
Fitnesse scenario
User story
+ Automatisering
scenario
TA adoptie
Dev
Team +
business

38. Speed…
40
Deliver test environment from 1-2 weeks to 1 hour
Provisioning of servers from 1-2 days to 30 minutes
Deploying software from 1-2 weeks to 30 minutes
Integration from 2 months to 30 minutes

39. Lessons learned for Stater
41
• Everyone training
• Wait until someone wants to start
• One coach
• Community of practice with “junior”
• Bottom up approach
• Ownership of Test automation in one team
• If time permits the team can pick it up
• Wait to be asked to try automation
• Automation is not easy
• Design patterns also apply for test code
• “Real“ developer needed
• Agile mindset is not easy

40. What works for Stater
42
• Pilot teams “3 sprints”
• Open community
• Walk in sessions
• Training while being coached
• Discuss guidelines
• Sonar
• Development skills needed
• Place improvement on the backlog (20%)
• Start with a picture
• Don’t tell them how to automate, explain what automation can do for you.
• Test automation is very useable to accelerate your manual testing
• Business/OPS at the driver seat
• Checks for the CD pipeline
• Support by management
VISIBILITY

41. The challenges…
• Extending the test coverage of automated test
• More teams using test automation.
• More ATDD
• Not all testers are developers.
• Collaborative code ownership
• OPS in the agile teams

42. Next steps…
Question?
44