This document summarizes the lessons learned over multiple years from implementing test automation at an organization. It discusses moving from a decentralized to centralized approach for test automation and the importance of both technical and organizational maturity. Key lessons include balancing central and decentralized support, using new techniques to drive organizational change, and the need for ongoing collaboration to address changing contexts. The full circle of lessons learned shows test automation success relies on both technical approach and people factors.

1. June 2015
Continuous Cycles in Test
Automation
Ard Kramer (Delta Lloyd)
Derk-Jan de Grood (Valori)

2. 2
A shared experience
Ard Kramer
•Testmanager Delta Lloyd Leven
•Test Consultant EclipseIT
•@Ard_Kramer
Derk-Jan de Grood
•Principal consultant Valori
•@DerkJandeGrood
•http://djdegrood.wordpress.com/

3. The Test Automation day – A continuing story
3
The Test
Automation day –
A continuing story

4. The sequels
4

5. TAD 2014
1. TA only
works when
mature 2. We need to
start with TA to
gain experience
(esp in Agile)
3. Start with
pilot, don't let
the
organization
get in the way
4. We need a
business case
in order to
convince
managers
5. BC is
influenced by
maturity6. get started
and let each
team find its
own solution
7. TA as driver
for Change
8.Growing
need for
central
approach
9. Embedding
in the
organization is
crucial
10. What test
do you
automate? Are
there any tests
defined?
TAD 2012
TAD 2015

6. Where we ended last year…
Steps 1-7: Early insights
6

7. Initial starting point
We want effective usage of Test Automation
throughout the organization
7

8. 1. Which application will fit
2. Which Divisions wants to start with test
automation
3. Prioritize (roadmap with TI, PoC en
Operational phase per division
4. Management decision to consolidate
5. Plan TI sessions
Test AnalystTest Navigator
ART team Test Navigator
ART team
Support
Operational phase
Test Analyst
Business case
Managerial steering
First estimation by
IT/Test manager
Commitment on
priorities and roadmap
Turining BC bases
upon PoC findings
Decision
operational launch
Support by
ART team
PoC (pilot) faseexploration phase
PoC report
Test Navigator
Per division
Test Navigator
ART team
Economic
feasibility
Technical
feasibility
Organizational
feasibility

9. Apply individual results to other teams
9

10. Sharing Knowledge
10

11. v
11
Making a mind map with things that
need to be organized for proper
support learned that this was a lot!

12. Roles and competences
12
Helpdesk ….CoachLicence manager

13. Conclusion of
last year’s
presentation:
13
People
Determine
Success
Change
needs
speed,
limited
scope and
visibility
Chose an
adaptive
Approach

14. NEW LESSONS LEARNED WHILE
IMPLEMENTING
Step 8-10
14

15. DECENTRAL: SUPPORT SA’S WITH EACH OF THE DEPT.
Helpdesk ….CoachLicence manager

16. Transition TA in scrum

17. Give me the
10% most critical
tests in your testset
NOW

18. The Full Circle
18
1. TA only
works when
mature 2. We need to
start with TA to
gain
experrience
(esp in Agile)
3. Start with
pilot, don't let
the
organization get
in the way
4. We need a
business case in
order to
convince
managers
5. BC is
influenced by
maturity6. get started
and let each
team find its
own solution
7. TA as driver
for Change
8. Growing
need for central
approach
9. Embedding
in the
organization is
crucial
10. What test
do you
automate? Are
there any tests
defined?

19. So it’s about maturity after all
19

20. Testflight: When tests are available …

21. What do we have, what can we do?

22. Benchmark facts • 40-60% of the required tests is
already automated
• 25-40% of these tests actually run
• Failed tests need to be repaired.
• Missing tests need to be designed
and automated.
Automated and
OK
Automated but
failing
Not Automated

23. When in a greenfield situation

24. Original roadmap
24

25. 25
1. TA only
works when
mature 2. We need to
start with TA to
gain experience
(esp. in Agile)
3. Start with
pilot, don't let
the
organization get
in the way
4. We need a
business case in
order to
convince
managers
5. BC is
influenced by
maturity6. get started
and let each
team find its
own solution
7. TA as driver
for Change
8. Growing
need for central
approach
9. Embedding
in the
organization is
crucial
10. What test
do you
automate? Are
there any tests
defined?

26. Key process, new challenge
26
TI PoC Ops

27. ORGANISATIONAL CHANGE
FROM DECENTRAL TO CENTRAL
27

28. New leadership: centralisation competences
Test
Business lines
Competencelines
Support
TA

29. 29
1. TA only
works when
mature 2. We need to
start with TA to
gain experience
(esp. in Agile)
3. Start with
pilot, don't let
the
organization get
in the way
4. We need a
business case in
order to
convince
managers
5. BC is
influenced by
maturity6. get started
and let each
team find its
own solution
7. TA as driver
for Change
8. Growing
need for central
approach
9. Embedding
in the
organization is
crucial
10. What test
do you
automate? Are
there any tests
defined?

30. v
30
RETHINK THE
SUPPORT AND
MAINTENANCE
IDEAS

31. New leadership: centralisation competences:
Possibilities per business line?
Business lines
Competencelines
TestSupport
TA

32. 32
1. TA only
works when
mature 2. We need to
start with TA to
gain experience
(esp. in Agile)
3. Start with
pilot, don't let
the
organization get
in the way
4. We need a
business case in
order to
convince
managers
5. BC is
influenced by
maturity6. get started
and let each
team find its
own solution
7. TA as driver
for Change
8. Growing
need for central
approach
9. Embedding
in the
organization is
crucial
10. What test
do you
automate? Are
there any tests
defined?

33. Scrum Teams
• Are test automation driven
• Improve their way of working
• Think of new solutions
• That benefit their business
case.
33

34. Reaching a tipping point

35. 35
What tool to put
in the spotlight ?

36. 36
1. TA only
works when
mature 2. We need to
start with TA to
gain experience
(esp. in Agile)
3. Start with
pilot, don't let
the
organization get
in the way
4. We need a
business case in
order to
convince
managers
5. BC is
influenced by
maturity6. get started
and let each
team find its
own solution
7. TA as driver
for Change
8. Growing
need for central
approach
9. Embedding
in the
organization is
crucial
10. What test
do you
automate? Are
there any tests
defined?
The new solutions that pop up can have
value for other scrum teams as well.

37. THERE MORE CYCLES
37

38. Other cycles
38
Business Case
Testing

39. Other cycles
39
Agile
Adoption

40. Other cycles
40
Organizational
Changes

41. Other cycles
41
TPI

42. Other cycles
42
(agile)
Test strategy

43. Collaborate
43

44. WRAP-UP
44

45. Highlights
- People determine the success
- Continue balance between central and decentral
- Technique can be driver for change
- But maturity can slow it down
- Be aware the context is changing – Collaboration needed
45
Technique
People
26
1. TA only
works when
mature 2. We need to
start with TA to
gain
experrience
(esp in Agile)
3. Start with
pilot, don't let
the
organiza on
get in the way
4. We need a
business case in
order to
convince
managers
5. BC is
influ
e
nced by
maturity
6. get started
and let each
team find its
own solu on
7. TA as driver
for Change
8. Growing
need for central
approach
9. Embedding in
the
organiza on is
crucial
10. What test
do you
automate? Are
there any tests
defin
e
d?

46. The earlier presentations
2014
• http://www.slideshare.net/DerkJandeGrood/experience-story-
implementing-test-automation-in-your-organization
• http://www.slideshare.net/DerkJandeGrood/implementing-test-
automation
2013
• http://www.slideshare.net/DerkJandeGrood/im-looking-at-you-to-tell-
me-should-i-automate-v10
2012
• http://www.slideshare.net/DerkJandeGrood/ta-day-door-to-open-or-to-
lock-in-template-taday-v11
46

47. 47
Questions?
Ard Kramer: Ard_kramer@deltalloyd.nl
Derk-jande Grood Derkjandegrood@valori.nl