EuroSTAR Software Testing Conference 2009 presentation on Test Policy - Test Strategy by Jarian van de Laar. See more at conferences.eurostarsoftwaretesting.com/past-presentations/

1. A practical Test StrategyA practical Test Strategy
Jurian van de LaarJurian van de Laar
jla@improveqs.nljla@improveqs.nl
EuroSTAR Conference 2009EuroSTAR Conference 2009
StockholmStockholm

2. Accredited provider ISTQB
Foundation & all Advanced modules
Training provider IREB
Requirements Engineering
Accredited Lead Assessors
formal TMMi assessments
Improve Quality ServicesImprove Quality Services
 Leading in Testing & Quality ManagementLeading in Testing & Quality Management
 Consultancy, Subcontracting and TrainingConsultancy, Subcontracting and Training
 Founded in 1998 by Erik van VeenendaalFounded in 1998 by Erik van Veenendaal
Eindhoven
Improve Quality Services
Waalre (near Eindhoven)
The Netherlands
www.improveqs.n
l

4. Policy? Strategy?Policy? Strategy?
 What does it mean ??What does it mean ??
 Do you need this ??Do you need this ??
 Case studyCase study
 Critical success factorsCritical success factors

5. Sounds familiar ?Sounds familiar ?

6. Definitions Test Policy & StrategyDefinitions Test Policy & Strategy
 Test PolicyTest Policy – High level description of principles,– High level description of principles,
approach and major objectives of the organizationapproach and major objectives of the organization
regarding testing.regarding testing.
 Test StrategyTest Strategy – High level description of test levels for– High level description of test levels for
an organization or programme, and the testing to bean organization or programme, and the testing to be
performed within each level.performed within each level.
Test Policy
Test Strategy
Master Test Plan
Generic risksGeneric risksGeneric risksGeneric risks
Business goalsBusiness goalsBusiness goalsBusiness goals Test goalsTest goalsTest goalsTest goals
BlueprintBlueprintBlueprintBlueprint
ProjectsProjectsProjectsProjects

7. Organization
Test Policy 20XX
Definition of testing
Mission / Ambition
based on
business goals
based on
business goals
EmployeesTest processCustomers
Test Process Improvement
- objectives, performance indicators, model
Test targets

8. Test StrategyTest Strategy
 Based on the test policyBased on the test policy
 Addresses the generic product risksAddresses the generic product risks
 Contains description of test levels:Contains description of test levels:
Purpose, responsibility
Level of independence (organizational)
Entry and exit criteria
Test process and deliverables
Approach (top-down, bottom-up)
Test design techniques
Tools to be used
Standards to be used
Test environment
Typical non-functionals

9.  Test policy & strategy are important …Test policy & strategy are important …
 Provide common basis for projectsProvide common basis for projects
 What isWhat is reallyreally important for business /important for business /
managersmanagers
 Leading in models likeLeading in models like TMMi, TPI, and TMap NextTMMi, TPI, and TMap Next
 …… but also difficult to realize !but also difficult to realize !
 ‘‘We have no business goals or policy’We have no business goals or policy’
 ‘‘We don’t have time / interest / priority’We don’t have time / interest / priority’
The challengeThe challenge

10. Practical case studyPractical case study
 Principal: IT-Solutions Mgr. Solutions ArchitecturePrincipal: IT-Solutions Mgr. Solutions Architecture
 Draft Test Strategy guidelines as starting pointDraft Test Strategy guidelines as starting point
 My assignment: coaching creation of Test StrategyMy assignment: coaching creation of Test Strategy
 Themes: strategy, terminology, organizationThemes: strategy, terminology, organization
 Scope: IT Express BeNeLuxScope: IT Express BeNeLux
 Business, Business-IT and IT-servicesBusiness, Business-IT and IT-services

11.  Roadmap and planningRoadmap and planning
 Organize meeting structureOrganize meeting structure
 Kick-off with management groupKick-off with management group
 Training day for work groupTraining day for work group
 Strategy workshopsStrategy workshops (4 sessions)(4 sessions)
 Elaboration Test Policy & StrategyElaboration Test Policy & Strategy
 Reporting to management groupReporting to management group
ApproachApproach
33
55
66
77
11
22
44

12. Kick-off with managementKick-off with management
 Purpose: get feedback and constraintsPurpose: get feedback and constraints
 Presentation using provocative argumentsPresentation using provocative arguments
 Results:Results:
 Feedback:Feedback:
 How ‘testing’ is perceived in the organizationHow ‘testing’ is perceived in the organization
 Effects of insufficient testing (= business case for improvement)Effects of insufficient testing (= business case for improvement)
 Important quality characteristicsImportant quality characteristics
 ConstraintsConstraints
 Realistic (acceptable) costs and organization structureRealistic (acceptable) costs and organization structure
33

13. Strategy workshopsStrategy workshops
1.1. Agree on purpose and scopeAgree on purpose and scope
 using management feedback and constraints as inputusing management feedback and constraints as input
1.1. Determine policy themesDetermine policy themes
 reliability, customer facing, usability,reliability, customer facing, usability,
performance, efficient testing, time to marketperformance, efficient testing, time to market
1.1. Definition of testingDefinition of testing
2.2. Business goalsBusiness goals
 not our responsibility, proactive thinking is (rationale)not our responsibility, proactive thinking is (rationale)
 7 values DHL taken as starting point7 values DHL taken as starting point
1.1. Test strategyTest strategy
 V-model, determine test levelsV-model, determine test levels
 RACI*, responsibilities, who is ‘main contractor’RACI*, responsibilities, who is ‘main contractor’
55
* = RACI: Responsible, Accountable, Consulted, Informed

14. Elaboration: resultsElaboration: results
 Test Policy documentTest Policy document
 Quality goals + rationale (2 A4)Quality goals + rationale (2 A4)
 Test policy (2 A4)Test policy (2 A4)
 Test Strategy documentTest Strategy document
 PrinciplesPrinciples
 V-modelV-model
 Characteristics per levelCharacteristics per level
 Risk based testingRisk based testing
 Management presentationManagement presentation
 Results and recommendationsResults and recommendations
66

15. Elaboration: Test PolicyElaboration: Test Policy
66

16. Elaboration: V-modelElaboration: V-model
66Full Roll-OUT

17. Management presentationManagement presentation
 RecommendationsRecommendations
 Agree on common test strategyAgree on common test strategy
 Apply risk based testingApply risk based testing
 Install a test competence cellInstall a test competence cell
 Start a test process improvement projectStart a test process improvement project
 Agree on common test terminology (e.g. ISTQB)Agree on common test terminology (e.g. ISTQB)
 Recommendations have been implementedRecommendations have been implemented
 Test competence cell: competence manager installedTest competence cell: competence manager installed
 Interviews / meetings with stakeholdersInterviews / meetings with stakeholders
 Plan for deployment writtenPlan for deployment written
77

18. Lessons and success factorsLessons and success factors
 It requires time to determine a good test strategy!It requires time to determine a good test strategy!
 If there is no policy available, then propose one.If there is no policy available, then propose one.
 Involve management (feedback, constraints).Involve management (feedback, constraints).
 Broad representation of organization as workgroup.Broad representation of organization as workgroup.
 Foundation test knowledge in workgroup (training).Foundation test knowledge in workgroup (training).
 Bring sufficient flexibility into your V-model.Bring sufficient flexibility into your V-model.
 Who is responsible for what (RACI) ?Who is responsible for what (RACI) ?
 Pioneer for implementation from own organization.Pioneer for implementation from own organization.

19. Epilogue (1)
 Follow-up by test competence manager:
 Further elaboration and deployment of test
strategy (templates and tools)
 Organized sessions/training on test awareness
 Assistance in projects to apply test strategy
 Has set up method to manage test environment
The assignment ended in 2008, what happened afterwards…The assignment ended in 2008, what happened afterwards…

20. Epilogue (2)
 Current situation (mid 2009):
 Competence cell (0.5 FTE*): central organization
of testing expertise.
 Close cooperation helps to get clear requirements
 Clearly defined test levels are helpful
 Test strategy currently applied in a project for
European standardisation of one of DHL’s most
important product lines
The assignment ended in 2008, what happened afterwards…The assignment ended in 2008, what happened afterwards…
* = FTE: Full-time equivalent

21. ConclusionsConclusions
 A sound Test Policy and Test Strategy …A sound Test Policy and Test Strategy …
 are the important starting points for projectsare the important starting points for projects
 help to have good discussions …and decisionshelp to have good discussions …and decisions
 can be very clear and specificcan be very clear and specific
 give answers to relative questionsgive answers to relative questions
 are the first step in test process improvementare the first step in test process improvement
…… are worthwhile to give a try in your organization ?!?!are worthwhile to give a try in your organization ?!?!