Ngtwsontpnts14july2014

Test Policy Test Strategy Workshop
Sponsored by
Version 1.6: 31 May 2014

Agenda
 Introduction and Expectations
Creating a Test Policy
 Defining Test Objectives
 Drafting a Test Policy
Creating a Test Strategy
 Implementing the testing process steps/tasks
 Understanding the development model/methodology and its
impact on Testing Strategy
 Test Levels and its impact
 Understanding Risks and impact on Test Strategy
 Introduction and Expectations
Creating a Test Policy
 Defining Test Objectives
 Drafting a Test Policy
Creating a Test Strategy
 Implementing the testing process steps/tasks
 Understanding the development model/methodology and its
impact on Testing Strategy
 Test Levels and its impact
 Understanding Risks and impact on Test Strategy

 Steering Committee Member, Agile Testing Alliance
 Co-Author of a book on Selenium
 Certified Trainer Agile Testing - ATA,Qualified Project
Management Professional (QPMP), Six Sigma Black Belt,
ISTQB certified foundation and advanced level Tester, Sun
Certified Java Programmer, Presenter in International
conferences on Project Management, Quality and Testing
 19+ years of IT Industry experience with:
 Larsen & Toubro Infotech Ltd, India, NSE.IT (National Stock Exchange, India)
 Celox Networks, USA, Netscout Systems, USA
 BE CSE, MBA Finance from UMASS Lowell.
Principal Consultant
Aditya Garg
 Steering Committee Member, Agile Testing Alliance
 Co-Author of a book on Selenium
 Certified Trainer Agile Testing - ATA,Qualified Project
Management Professional (QPMP), Six Sigma Black Belt,
ISTQB certified foundation and advanced level Tester, Sun
Certified Java Programmer, Presenter in International
conferences on Project Management, Quality and Testing
 19+ years of IT Industry experience with:
 Larsen & Toubro Infotech Ltd, India, NSE.IT (National Stock Exchange, India)
 Celox Networks, USA, Netscout Systems, USA
 BE CSE, MBA Finance from UMASS Lowell.
Aditya has been an automation test architect and principal consultant excelling in designing, strategizing and
architecting manual and automation testing solutions. His primary focus has been in the area of establishing and leading
testing center of excellence and practices, managing large IT projects, undertaking testing process studies using Six
Sigma, CMMi and TMM models, proposing QA solutions, performance engineering and architecture reviews, designing
automation frameworks, exploring open source test automation tools/frameworks and Business development
Aditya’s current research area is use of pair wise testing in agile projects specially in optimizing test automation
regression packs build around BDT with tools like Cucumber, Selenium and Capybara.

What is Policy?
Wikipedia
• A Policy is typically described as a principle or rule to guide
decisions and achieve rational outcomes.
• The term is not normally used to denote what is actually done,
this is normally referred to as either procedure or protocol.
• A Policy can be considered as a "Statement of Intent" or a
"Commitment".
Confidential | Copyright © QAAgility Technologies
Wikipedia
• A Policy is typically described as a principle or rule to guide
decisions and achieve rational outcomes.
• The term is not normally used to denote what is actually done,
this is normally referred to as either procedure or protocol.
• A Policy can be considered as a "Statement of Intent" or a
"Commitment".

What is Policy?
ISTQB Glossary Test Policy:
A high level document describing the
principles, approach and major objectives of
the organization regarding testing
ISTQB Glossary Test Policy:
A high level document describing the
principles, approach and major objectives of
the organization regarding testing

What is Policy?
What do you think must be part of Test Policy ?

What is Policy?
The “why” of testing
• Gives overall test objectives for organization
• Developed by senior managers (test managers and test stakeholder
managers)
• Topics include: Business value of testing
• Test objectives
• Effectiveness and efficiency metrics
• Typical test process
•Test process improvement
Short, high-level document
The “why” of testing
• Gives overall test objectives for organization
• Developed by senior managers (test managers and test stakeholder
managers)
• Topics include: Business value of testing
• Test objectives
• Effectiveness and efficiency metrics
• Typical test process
•Test process improvement
Short, high-level document

Test Objectives
S •Specific
M •Measurable
A •Attainable
A •Attainable
R •Relevant
T •Timely

Test Objectives – Exercise – Please list down – 10 minutes
S •Specific
M •Measurable
A •Attainable
A •Attainable
R •Relevant
T •Timely

Test Objectives
S
• Specific Test Hours
M
• Measurable Productivity
A
• Attainable Number of defects
A
• Attainable Number of defects
R
• Relevant Tests
T
• Time-Limited Test Plan

Mission: (define what should be the mission/30K feet level
objective/purpose for the testing organization)
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
1
Test Policy – Sample Construction
Mission: (define what should be the mission/30K feet level
objective/purpose for the testing organization)
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Mission Statement
Reference:
http://www.businessdictionary.com/defin
ition/mission-statement.html

Strategy/How: (Define in simple sentence how would you achieve
this mission – or what would be your strategy to achieve the mission)
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
2
Strategy/How: (Define in simple sentence how would you achieve
this mission – or what would be your strategy to achieve the mission)
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------

Levels of Testing and their objectives you would like to perform to
achieve the strategy goals
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
3
Levels of Testing and their objectives you would like to perform to
achieve the strategy goals
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------

Generic Testing Process that you would like to define for these levels
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
4
Generic Testing Process that you would like to define for these levels
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------

Key Process Indicators (KPI’s) / Service Level Indicators (SLA’s)
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
5
Key Process Indicators (KPI’s) / Service Level Indicators (SLA’s)
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------

Exercise: Prepare a Strategy
Consider that you are testing an application similar to
Amazon.com/ Kalahari.com / FlipKart.com
How would you test it ?
What will you test in it ?
Make your own assumptions
Time limit – 30 minutes
2 minutes per group for discussion
Consider that you are testing an application similar to
Amazon.com/ Kalahari.com / FlipKart.com
How would you test it ?
What will you test in it ?
Make your own assumptions
Time limit – 30 minutes
2 minutes per group for discussion

Test Strategy
•The “how” of testing, at a general level
•Managing product and project risks
•Test levels, entry/exit criteria and
coverage
•High-level test activities
•Types of strategies:
•Analytical
•Model-based
•Methodical
•Process- or standard-compliant
•Reactive
•Consultative
•Regression-averse
Strategies vary by
lifecycle, type of
application, type of
project, software
lifecycle model, level of
risk, regulatory
requirements, short-term
vs. long-term, etc.
•The “how” of testing, at a general level
•Managing product and project risks
•Test levels, entry/exit criteria and
coverage
•High-level test activities
•Types of strategies:
•Analytical
•Model-based
•Methodical
•Process- or standard-compliant
•Reactive
•Consultative
•Regression-averse
Strategies vary by
lifecycle, type of
application, type of
project, software
lifecycle model, level of
risk, regulatory
requirements, short-term
vs. long-term, etc.
The test strategy should align with the test policy

Case Study
Practical Test Strategy
Creation
Case Study
Practical Test Strategy
Creation

20 Minutes:
Go through the given
case study
20 Minutes:
Go through the given
case study

S/W Development Methodology
&
Testing Levels
1
CONSIDERATIONS
S/W Development Methodology
&
Testing Levels2

Question(s) to be answered ?
Should Testing team take into consideration Software
Development Methodology ?
What are the different software models ?
How does software testing levels (CIT, ST, SIT, (U)AT, OAT)
impact Strategy ?
Should Testing team take into consideration Software
Development Methodology ?
What are the different software models ?
How does software testing levels (CIT, ST, SIT, (U)AT, OAT)
impact Strategy ?

What’s a Risk?
 Risk
– The possibility of a negative or undesirable outcome

Project risks
- Risk related to management and control of the (test) project
- Project risks influence the success of a project, they have to be
managed

Project Risk
 Project Risk: A risk related to management and control of the (test) project.
 Supplier Issues
– Contractual Issues
– Third party goes in liquidation or fails to deliver
 Organisational Issues
– Skills and Staff shortages
– Training and support issues
– Communication/Political Issues, e.g. between testers and other
project teams
 Technical
– No or poor requirements
– Quality of the design or code
– Architectural solution under question
 Project Risk: A risk related to management and control of the (test) project.
 Supplier Issues
– Contractual Issues
– Third party goes in liquidation or fails to deliver
 Organisational Issues
– Skills and Staff shortages
– Training and support issues
– Communication/Political Issues, e.g. between testers and other
project teams
 Technical
– No or poor requirements
– Quality of the design or code
– Architectural solution under question

Entry and exit criteria
Test
Phase 2
"tested"
is it ready for my
testing?
Test
Phase 1
"tested"
is it ready for my
testing?
Phase 2 Phase 1
Entry criteria Exit criteria
Acceptance
criteria
Completion
criteria

Exercise - Continued
For the given case study – Relook at
A) Testing process
B) Testing levels
By highlighting the PROJECT RISKS in the case study and how would it
impact the overall approach. Addressing the RISKS

Other Contents of Test Strategy Documents
• Independence of testing
• Mandatory and optional
standards
• Confirmation testing and
regression testing
• Test automation
• Reusability of work products
• Non Functional Testing
•Test environments
•Test tools
•Test control and reporting
•Test measurements and
metrics
•Defect management
•Testware configuration
management
• Independence of testing
• Mandatory and optional
standards
• Confirmation testing and
regression testing
• Test automation
• Reusability of work products
• Non Functional Testing
•Test environments
•Test tools
•Test control and reporting
•Test measurements and
metrics
•Defect management
•Testware configuration
management

We are a TESTING training company that
brings to you -
• Thought leadership in Testing area
• Book published on Selenium by Tata McGraw Hill
• Agile Testing Licensed Trainers for ATA
• People behind ITB Mumbai chapter and TeStride
Mumbai Conference
• Huge Experience in training more than 3000
professionals across major IT Companies in India
and abroad in Testing
• Practical insight into all training assignments
due to our pedigree working and managing
testing for large multinationals for more than 18-
19 years
AND WE ARE PASSIONATE ABOUT IT!
Confidential | Copyright © QA Agility Technologies
We are a TESTING training company that
brings to you -
• Thought leadership in Testing area
• Book published on Selenium by Tata McGraw Hill
• Agile Testing Licensed Trainers for ATA
• People behind ITB Mumbai chapter and TeStride
Mumbai Conference
• Huge Experience in training more than 3000
professionals across major IT Companies in India
and abroad in Testing
• Practical insight into all training assignments
due to our pedigree working and managing
testing for large multinationals for more than 18-
19 years
AND WE ARE PASSIONATE ABOUT IT!

Offerings - Comprehensive training programs in QA/Testing area
• Agile Testing Alliance - Certifications
• CP-BAT, CP-MAT, CP-AAT, CP-AAST
• Specialized Workshops/Niche Trainings
• Practical Test Strategy Formulation
• Architecting Testing Solutions
• Risk Based Testing
• Certified Agile Tester and Agile Testing
• ISTQB Certification
• Foundation Level
• Advanced Level
• Test Automation
• QTP
• Foundation
• Advanced
• Framework driven
• Selenium
• Foundation
• Advanced
• Test Management
• Quality Center
• Test Link
• Performance Testing
• Load Runner
• Silk Performer
• Open STA, JMeter
• Testing and QA Processes
• Testing Metrics
• Extended programs and diploma
on Testing Talent Development
• Test Automation
• Manual Testing
• Testing Leaderships
• Customized Corporate Trainings
• Specific Testing topic
• Web based Application Testing
• SOA Testing
• DB Testing
• GHTester. SOAP UI
• Tailored to project and
organization needs
Confidential | Copyright © QA Agility Technologies
• Agile Testing Alliance - Certifications
• CP-BAT, CP-MAT, CP-AAT, CP-AAST
• Specialized Workshops/Niche Trainings
• Practical Test Strategy Formulation
• Architecting Testing Solutions
• Risk Based Testing
• Certified Agile Tester and Agile Testing
• ISTQB Certification
• Foundation Level
• Advanced Level
• Test Automation
• QTP
• Foundation
• Advanced
• Framework driven
• Selenium
• Foundation
• Advanced
• Test Management
• Quality Center
• Test Link
• Performance Testing
• Load Runner
• Silk Performer
• Open STA, JMeter
• Testing and QA Processes
• Testing Metrics
• Extended programs and diploma
on Testing Talent Development
• Test Automation
• Manual Testing
• Testing Leaderships
• Customized Corporate Trainings
• Specific Testing topic
• Web based Application Testing
• SOA Testing
• DB Testing
• GHTester. SOAP UI
• Tailored to project and
organization needs

 Surest Way to Master Agile Testing
 Pick up newer ways of doing Testing
 Differentiate how old ways of testing may
not work in swift moving agile projects
 Learn optimized test design – utmost
necessary to reduce defect leakage in a
project where time is always a constraint.
 Mind Map Test Design Technique
 Pairwise/Combinatorial Techniques
 Exploratory Test Design Technique
 Increasing agility in finding defects
 Real Agile Project (Multiple Drops, Multiple
Sprints)
Certified
Professional –
Master Agile Testing
 Surest Way to Master Agile Testing
 Pick up newer ways of doing Testing
 Differentiate how old ways of testing may
not work in swift moving agile projects
 Learn optimized test design – utmost
necessary to reduce defect leakage in a
project where time is always a constraint.
 Mind Map Test Design Technique
 Pairwise/Combinatorial Techniques
 Exploratory Test Design Technique
 Increasing agility in finding defects
 Real Agile Project (Multiple Drops, Multiple
Sprints)

 Surest Way to learn cutting edge
automation trends
 Understanding huge importance of
automation in today’s testing world
 Hands on BDD,ATDD and TDD
 Practice automation from concept to
regression and test hardening iteration
using:
 Real Case Study
 Real tools like – Cucumber,
Fitnesse, Selenium, Hudson/Jenkins
Certified Professional –
Automation Agile
Testing
 Surest Way to learn cutting edge
automation trends
 Understanding huge importance of
automation in today’s testing world
 Hands on BDD,ATDD and TDD
 Practice automation from concept to
regression and test hardening iteration
using:
 Real Case Study
 Real tools like – Cucumber,
Fitnesse, Selenium, Hudson/Jenkins

Thank you
Write to
aditya.garg@agiletestingalliance.org
#adigindia
Linked in: https://www.linkedin.com/in/adigarg
40
Thank you
Write to
aditya.garg@agiletestingalliance.org
#adigindia
Linked in: https://www.linkedin.com/in/adigarg

Ngtwsontpnts14july2014

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Ngtwsontpnts14july2014

Similar to Ngtwsontpnts14july2014 (20)

Recently uploaded

Recently uploaded (20)

Ngtwsontpnts14july2014