Many organisations start improving their testing by implementing some kind of line organisation for testing (test expertise center, test service center), hereafter called TEC. Although a good starting point for improvements, in practice the TEC is often not much more than a resource pool of testers, possibly supplying certain templates or giving advice to projects. A next maturity level for a TEC is to grow to a test factory, responsible for delivering pre-agreed test results.From the experiences gathered mostly from a large railroad infrastructure organisation, this presentation shows the path to this next level of test maturity and responsibility.However, this is not a straight path, but a path with ups and downs and many curves, and getting there isn’t easy. It requires change, in organisational processes but, more difficult, also in the way people work, their behavior and their attitude. In my practice, I follow the principles of the Basic Change Method (from Dutch management guru Ben Tiggelaar). BCM is a combination of the most effective insights from cognitive and behavioral science and focuses on making people change their common behavior by management of both behavior intentions and change situations. Usually change management is mainly focused on end results. But the underestimated factor between change plans and desired results is behavior. Issues that will be discussed are: • using the TEC as a lever for test improvement • envisioning the roadmap • formulating improvement actions • (management) commitment • organising the improvement (team) • planning the change • implementing the improvements • changing behavior • measuring results.

1. 1
Growing to a
Next Level Test Organisation
Tim Koomen EuroSTAR 2011

2. Seite 2
Test resource pool
Test factory
Basic Change Method
End
Start

3. Average TPI score ...
3
Client

4. Common findings
 Insufficient risk based test strategy
 Unclear use of test design techniques
 Lack of a structured test process
 Some kind of test organisation …
4

5. Typical improvement horizon
 No surprise ...
 Risk based test strategy and use of techniques
 Structured and transparant test process
 Permanent test organisation, “test factory”, with
clearly defined services
 (+ some 1-3 other points)
5

6. Different types of organisation
6
Result Project agreement Outsourcing
Resource Ad-hoc contracting Resource commitment
Incidental Structural
Test Factory
Test Expertise Center

7. Services existing TEC
 “Resource pool”
 Supplying (internal or external) testers, test
managers and test automation specialists
 And sometimes:
 “Guardian” of test method
 Test consultancy
 Coaching users in an acceptance test
 Training.
7

8. Seite 8
Test resource pool
Test factory
Basic Change Method
End
Start

9. Level of ambition: Test Factory
Test Factory as lever for improvement:
 Permanent
 (More) delivery commitment
 Motivator
 More power in organisation
9

10. Test Factory?
 What does it look like?
 Processes
 People
 Services
 Infrastructure
 Advantages
10

11. Process model
 Organisation / process model set-up
General management (financial and operational)
Human resource management
Services management
Planning
Delivery management
Support and monitoring
Initiation Execution Completion
Permanenttest
organisation
Services
11

12. People
 Function descriptions, career path and
training for testers
12

13. Services
 Examples:
 System and acceptance testing
 Agile testing
 Management of test processes
 Test consultancy
 Test automation
 Auditing test processes
 Test environment management
 Training
 ...
13

14. Services in detail
14
Aspect Short description
Input Required input from customer
Entry-criteria What requirements must the customer meet for the service to be
successful?
Approach Required processes, standards and techniques (norm)
Required
functions
Test manager, tester, test automation engineer
Output Expected output of the service: products, reports
Type of
assignment
delivery commitment / resource commitment / no commitment
Service level The service level of the service: measured in time, money and/or quality,
also: knowledge conservation, reliability
Costs Costs of the service in money or hours. Costs for cancellation or delay, ...
Added value What is the advantage for the customer and the organisation to use this
service

15. Infrastructure
 Often owner of test infrastructure
 Optionally managed by another department
 Under control of Test Factory
 Tooling: projects x line (system management,
licences), integration with development
15

16. Advantages
 Test organisation in general:
 Better use of (scarce) expertise
 Potentially lower costs through centralisation and
scaling
 Shorter start-up time
 Test Factory versus TEC:
 More predictable quality of testing
 Continuous improvement
 Knowledge preservation and development
 Costs and lead time are better manageable
16

17. Seite 17
Test resource pool
Test factory
Basic Change Method
End
Start

18. Change process steps
(after assessment)
 Communicate findings/recommendations
 (Management) commitment
 Organise (coordination of) improvement
activities
 Implementation plan
 Supporting test and TPI-expertise
 Brainstorm over future test organisation
 Implement and pilot improvements
 Going x growing concern
18

19. situation
Basic Change Method
 Combination of most effective notions from
cognitive and social behaviour psychology
 Basis:
 Change = adopting new habitual behavior
 This requires management of behaviour intentions
and change situations
 Change goes through phases and has costs (time,
money, …)
Ben Tiggelaar, “Do!”
19
intentions behaviour

20. Phases BCM
 Get Real
 Goal oriented behaviour intentions (required results
and other habitual behaviour)
 Make Plans
 Prepare most difficult change situations (with
triggers and counter behaviour, action plan)
 Take Action
 First measure behaviour, only later measure results
of behaviour
 Reward
20

21. Implementation ...
 Work groups create all kinds of deliverables. And
now ...?
 For each deliverable several activities:
 Communication (how and to whom)
 Set-up
 Training
 Coaching/support
 Monitoring
 Feedback
 Plan these activities - “Service group plan 2011”
21

22. Example: KPI’s
22
Goal Managed test processes
Communication TF-meetings, email, newsletter
Set-up SAP, extranet, part of standard progress
reporting
Training Instruction meeting for test managers
Coaching/
support
1 x for each test project
Monitoring Test manager meetings, TF-manager
Feedback Quarterly evaluation whether KPI's give right
info + trends

23. Resistance?
 Yes!
 “Not in my (running) project”
 “Not in my starting project”
 “My project is just a bit different …”
 “No time”
 “This is just maintenance”
 Be patient
 Management commitment required
 Don’t walk too far ahead ...
 Counter “Not Invented Here”: people create and
implement solutions themselves
 Reward (by giving attention)
23

24. Finally: Agile and Test Factory?
 Can this be combined?
 Not in it’s pure form ...
 But sourcing testers as a service,
where the testers use a transparant process
and generate predictable results, comes close
24

25. Seite 25
Test organisation is often just a resource
pool
Test Factory as a lever for change
Basic Change Method helps
…
but leaves enough of a challenge
…
Conclusion

26. Questions or remarks?
26

27. E. info@timkoomen.nl
M. +31 (0)6 34139260
I. www.timkoomen.nl
Copyright Tim Koomen Testmanagement en -advies