Test Policy Test Strategy Workshop
Sponsored by
Version 1.6: 31 May 2014
Agenda
 Introduction and Expectations
Creating a Test Policy
 Defining Test Objectives
 Drafting a Test Policy
Creating...
 Steering Committee Member, Agile Testing Alliance
 Co-Author of a book on Selenium
 Certified Trainer Agile Testing - ...
Test PolicyTest Policy
What is Policy?
Wikipedia
• A Policy is typically described as a principle or rule to guide
decisions and achieve rational...
What is Policy?
ISTQB Glossary Test Policy:
A high level document describing the
principles, approach and major objectives...
What is Policy?
What do you think must be part of Test Policy ?
Confidential | Copyright © QAAgility Technologies
What is Policy?
The “why” of testing
• Gives overall test objectives for organization
• Developed by senior managers (test...
Test Objectives
S •Specific
M •Measurable
A •Attainable
Confidential | Copyright © QAAgility Technologies
A •Attainable
R ...
Test Objectives – Exercise – Please list down – 10 minutes
S •Specific
M •Measurable
A •Attainable
Confidential | Copyrigh...
Test Objectives
S
• Specific Test Hours
M
• Measurable Productivity
A
• Attainable Number of defects
Confidential | Copyri...
Mission: (define what should be the mission/30K feet level
objective/purpose for the testing organization)
---------------...
Strategy/How: (Define in simple sentence how would you achieve
this mission – or what would be your strategy to achieve th...
Test Policy – Sample Construction
Levels of Testing and their objectives you would like to perform to
achieve the strategy...
Test Policy – Sample Construction
Generic Testing Process that you would like to define for these levels
-----------------...
Test Policy – Sample Construction
Key Process Indicators (KPI’s) / Service Level Indicators (SLA’s)
----------------------...
Confidential | Copyright © QAAgility Technologies
Confidential | Copyright © QAAgility Technologies
Test StrategyTest Strategy
Exercise: Prepare a Strategy
Consider that you are testing an application similar to
Amazon.com/ Kalahari.com / FlipKart.c...
Test Strategy
•The “how” of testing, at a general level
•Managing product and project risks
•Test levels, entry/exit crite...
Case Study
Practical Test Strategy
Creation
Case Study
Practical Test Strategy
Creation
20 Minutes:
Go through the given
case study
20 Minutes:
Go through the given
case study
S/W Development Methodology
&
Testing Levels
1
CONSIDERATIONS
S/W Development Methodology
&
Testing Levels2
Question(s) to be answered ?
Should Testing team take into consideration Software
Development Methodology ?
What are the d...
RisksRisks
What’s a Risk?
 Risk
– The possibility of a negative or undesirable outcome
Project risks
- Risk related to management and control of the (test) project
- Project risks influence the success of a pr...
Project Risk
 Project Risk: A risk related to management and control of the (test) project.
 Supplier Issues
– Contractu...
Entry and exit criteria
Test
Phase 2
"tested"
is it ready for my
testing?
Test
Phase 1
"tested"
is it ready for my
testing...
Exercise - Continued
For the given case study – Relook at
A) Testing process
B) Testing levels
By highlighting the PROJECT...
Other Contents of Test Strategy Documents
• Independence of testing
• Mandatory and optional
standards
• Confirmation test...
Appendix
Confidential | Copyright © QAAgility Technologies Pvt Ltd
About us
Confidential | Copyright © QAAgility Technologies Pvt Ltd
We are a TESTING training company that
brings to you -
• Thought leadership in Testing area
• Book published on Selenium b...
Offerings - Comprehensive training programs in QA/Testing area
• Agile Testing Alliance - Certifications
• CP-BAT, CP-MAT,...
Agile Testing Certifications
 Surest Way to Master Agile Testing
 Pick up newer ways of doing Testing
 Differentiate how old ways of testing may
not...
 Surest Way to learn cutting edge
automation trends
 Understanding huge importance of
automation in today’s testing worl...
Thank you
Write to
aditya.garg@agiletestingalliance.org
#adigindia
Linked in: https://www.linkedin.com/in/adigarg
40
Thank...
Upcoming SlideShare
Loading in …5
×

Ngtwsontpnts14july2014

369 views

Published on

WS

Published in: Software
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
369
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Ngtwsontpnts14july2014

  1. 1. Test Policy Test Strategy Workshop Sponsored by Version 1.6: 31 May 2014
  2. 2. Agenda  Introduction and Expectations Creating a Test Policy  Defining Test Objectives  Drafting a Test Policy Creating a Test Strategy  Implementing the testing process steps/tasks  Understanding the development model/methodology and its impact on Testing Strategy  Test Levels and its impact  Understanding Risks and impact on Test Strategy  Introduction and Expectations Creating a Test Policy  Defining Test Objectives  Drafting a Test Policy Creating a Test Strategy  Implementing the testing process steps/tasks  Understanding the development model/methodology and its impact on Testing Strategy  Test Levels and its impact  Understanding Risks and impact on Test Strategy
  3. 3.  Steering Committee Member, Agile Testing Alliance  Co-Author of a book on Selenium  Certified Trainer Agile Testing - ATA,Qualified Project Management Professional (QPMP), Six Sigma Black Belt, ISTQB certified foundation and advanced level Tester, Sun Certified Java Programmer, Presenter in International conferences on Project Management, Quality and Testing  19+ years of IT Industry experience with:  Larsen & Toubro Infotech Ltd, India, NSE.IT (National Stock Exchange, India)  Celox Networks, USA, Netscout Systems, USA  BE CSE, MBA Finance from UMASS Lowell. Principal Consultant Aditya Garg  Steering Committee Member, Agile Testing Alliance  Co-Author of a book on Selenium  Certified Trainer Agile Testing - ATA,Qualified Project Management Professional (QPMP), Six Sigma Black Belt, ISTQB certified foundation and advanced level Tester, Sun Certified Java Programmer, Presenter in International conferences on Project Management, Quality and Testing  19+ years of IT Industry experience with:  Larsen & Toubro Infotech Ltd, India, NSE.IT (National Stock Exchange, India)  Celox Networks, USA, Netscout Systems, USA  BE CSE, MBA Finance from UMASS Lowell. Aditya has been an automation test architect and principal consultant excelling in designing, strategizing and architecting manual and automation testing solutions. His primary focus has been in the area of establishing and leading testing center of excellence and practices, managing large IT projects, undertaking testing process studies using Six Sigma, CMMi and TMM models, proposing QA solutions, performance engineering and architecture reviews, designing automation frameworks, exploring open source test automation tools/frameworks and Business development Aditya’s current research area is use of pair wise testing in agile projects specially in optimizing test automation regression packs build around BDT with tools like Cucumber, Selenium and Capybara.
  4. 4. Test PolicyTest Policy
  5. 5. What is Policy? Wikipedia • A Policy is typically described as a principle or rule to guide decisions and achieve rational outcomes. • The term is not normally used to denote what is actually done, this is normally referred to as either procedure or protocol. • A Policy can be considered as a "Statement of Intent" or a "Commitment". Confidential | Copyright © QAAgility Technologies Wikipedia • A Policy is typically described as a principle or rule to guide decisions and achieve rational outcomes. • The term is not normally used to denote what is actually done, this is normally referred to as either procedure or protocol. • A Policy can be considered as a "Statement of Intent" or a "Commitment".
  6. 6. What is Policy? ISTQB Glossary Test Policy: A high level document describing the principles, approach and major objectives of the organization regarding testing Confidential | Copyright © QAAgility Technologies ISTQB Glossary Test Policy: A high level document describing the principles, approach and major objectives of the organization regarding testing
  7. 7. What is Policy? What do you think must be part of Test Policy ? Confidential | Copyright © QAAgility Technologies
  8. 8. What is Policy? The “why” of testing • Gives overall test objectives for organization • Developed by senior managers (test managers and test stakeholder managers) • Topics include: Business value of testing • Test objectives • Effectiveness and efficiency metrics • Typical test process •Test process improvement Short, high-level document Confidential | Copyright © QAAgility Technologies The “why” of testing • Gives overall test objectives for organization • Developed by senior managers (test managers and test stakeholder managers) • Topics include: Business value of testing • Test objectives • Effectiveness and efficiency metrics • Typical test process •Test process improvement Short, high-level document
  9. 9. Test Objectives S •Specific M •Measurable A •Attainable Confidential | Copyright © QAAgility Technologies A •Attainable R •Relevant T •Timely
  10. 10. Test Objectives – Exercise – Please list down – 10 minutes S •Specific M •Measurable A •Attainable Confidential | Copyright © QAAgility Technologies A •Attainable R •Relevant T •Timely
  11. 11. Test Objectives S • Specific Test Hours M • Measurable Productivity A • Attainable Number of defects Confidential | Copyright © QAAgility Technologies A • Attainable Number of defects R • Relevant Tests T • Time-Limited Test Plan
  12. 12. Mission: (define what should be the mission/30K feet level objective/purpose for the testing organization) --------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------- 1 Test Policy – Sample Construction Confidential | Copyright © QAAgility Technologies Mission: (define what should be the mission/30K feet level objective/purpose for the testing organization) --------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------- Mission Statement Reference: http://www.businessdictionary.com/defin ition/mission-statement.html
  13. 13. Strategy/How: (Define in simple sentence how would you achieve this mission – or what would be your strategy to achieve the mission) --------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------- 2 Test Policy – Sample Construction Confidential | Copyright © QAAgility Technologies Strategy/How: (Define in simple sentence how would you achieve this mission – or what would be your strategy to achieve the mission) --------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------
  14. 14. Test Policy – Sample Construction Levels of Testing and their objectives you would like to perform to achieve the strategy goals --------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------- 3 Confidential | Copyright © QAAgility Technologies Levels of Testing and their objectives you would like to perform to achieve the strategy goals --------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------
  15. 15. Test Policy – Sample Construction Generic Testing Process that you would like to define for these levels --------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------- 4 Confidential | Copyright © QAAgility Technologies Generic Testing Process that you would like to define for these levels --------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------
  16. 16. Test Policy – Sample Construction Key Process Indicators (KPI’s) / Service Level Indicators (SLA’s) --------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------- 5 Confidential | Copyright © QAAgility Technologies Key Process Indicators (KPI’s) / Service Level Indicators (SLA’s) --------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------
  17. 17. Confidential | Copyright © QAAgility Technologies
  18. 18. Confidential | Copyright © QAAgility Technologies
  19. 19. Test StrategyTest Strategy
  20. 20. Exercise: Prepare a Strategy Consider that you are testing an application similar to Amazon.com/ Kalahari.com / FlipKart.com How would you test it ? What will you test in it ? Make your own assumptions Time limit – 30 minutes 2 minutes per group for discussion Consider that you are testing an application similar to Amazon.com/ Kalahari.com / FlipKart.com How would you test it ? What will you test in it ? Make your own assumptions Time limit – 30 minutes 2 minutes per group for discussion
  21. 21. Test Strategy •The “how” of testing, at a general level •Managing product and project risks •Test levels, entry/exit criteria and coverage •High-level test activities •Types of strategies: •Analytical •Model-based •Methodical •Process- or standard-compliant •Reactive •Consultative •Regression-averse Strategies vary by lifecycle, type of application, type of project, software lifecycle model, level of risk, regulatory requirements, short-term vs. long-term, etc. Confidential | Copyright © QAAgility Technologies •The “how” of testing, at a general level •Managing product and project risks •Test levels, entry/exit criteria and coverage •High-level test activities •Types of strategies: •Analytical •Model-based •Methodical •Process- or standard-compliant •Reactive •Consultative •Regression-averse Strategies vary by lifecycle, type of application, type of project, software lifecycle model, level of risk, regulatory requirements, short-term vs. long-term, etc. The test strategy should align with the test policy
  22. 22. Case Study Practical Test Strategy Creation Case Study Practical Test Strategy Creation
  23. 23. 20 Minutes: Go through the given case study 20 Minutes: Go through the given case study
  24. 24. S/W Development Methodology & Testing Levels 1 CONSIDERATIONS S/W Development Methodology & Testing Levels2
  25. 25. Question(s) to be answered ? Should Testing team take into consideration Software Development Methodology ? What are the different software models ? How does software testing levels (CIT, ST, SIT, (U)AT, OAT) impact Strategy ? Should Testing team take into consideration Software Development Methodology ? What are the different software models ? How does software testing levels (CIT, ST, SIT, (U)AT, OAT) impact Strategy ?
  26. 26. RisksRisks
  27. 27. What’s a Risk?  Risk – The possibility of a negative or undesirable outcome
  28. 28. Project risks - Risk related to management and control of the (test) project - Project risks influence the success of a project, they have to be managed
  29. 29. Project Risk  Project Risk: A risk related to management and control of the (test) project.  Supplier Issues – Contractual Issues – Third party goes in liquidation or fails to deliver  Organisational Issues – Skills and Staff shortages – Training and support issues – Communication/Political Issues, e.g. between testers and other project teams  Technical – No or poor requirements – Quality of the design or code – Architectural solution under question  Project Risk: A risk related to management and control of the (test) project.  Supplier Issues – Contractual Issues – Third party goes in liquidation or fails to deliver  Organisational Issues – Skills and Staff shortages – Training and support issues – Communication/Political Issues, e.g. between testers and other project teams  Technical – No or poor requirements – Quality of the design or code – Architectural solution under question
  30. 30. Entry and exit criteria Test Phase 2 "tested" is it ready for my testing? Test Phase 1 "tested" is it ready for my testing? Phase 2 Phase 1 Entry criteria Exit criteria Acceptance criteria Completion criteria
  31. 31. Exercise - Continued For the given case study – Relook at A) Testing process B) Testing levels By highlighting the PROJECT RISKS in the case study and how would it impact the overall approach. Addressing the RISKS
  32. 32. Other Contents of Test Strategy Documents • Independence of testing • Mandatory and optional standards • Confirmation testing and regression testing • Test automation • Reusability of work products • Non Functional Testing •Test environments •Test tools •Test control and reporting •Test measurements and metrics •Defect management •Testware configuration management Confidential | Copyright © QAAgility Technologies • Independence of testing • Mandatory and optional standards • Confirmation testing and regression testing • Test automation • Reusability of work products • Non Functional Testing •Test environments •Test tools •Test control and reporting •Test measurements and metrics •Defect management •Testware configuration management
  33. 33. Appendix Confidential | Copyright © QAAgility Technologies Pvt Ltd
  34. 34. About us Confidential | Copyright © QAAgility Technologies Pvt Ltd
  35. 35. We are a TESTING training company that brings to you - • Thought leadership in Testing area • Book published on Selenium by Tata McGraw Hill • Agile Testing Licensed Trainers for ATA • People behind ITB Mumbai chapter and TeStride Mumbai Conference • Huge Experience in training more than 3000 professionals across major IT Companies in India and abroad in Testing • Practical insight into all training assignments due to our pedigree working and managing testing for large multinationals for more than 18- 19 years AND WE ARE PASSIONATE ABOUT IT! Confidential | Copyright © QA Agility Technologies We are a TESTING training company that brings to you - • Thought leadership in Testing area • Book published on Selenium by Tata McGraw Hill • Agile Testing Licensed Trainers for ATA • People behind ITB Mumbai chapter and TeStride Mumbai Conference • Huge Experience in training more than 3000 professionals across major IT Companies in India and abroad in Testing • Practical insight into all training assignments due to our pedigree working and managing testing for large multinationals for more than 18- 19 years AND WE ARE PASSIONATE ABOUT IT!
  36. 36. Offerings - Comprehensive training programs in QA/Testing area • Agile Testing Alliance - Certifications • CP-BAT, CP-MAT, CP-AAT, CP-AAST • Specialized Workshops/Niche Trainings • Practical Test Strategy Formulation • Architecting Testing Solutions • Risk Based Testing • Certified Agile Tester and Agile Testing • ISTQB Certification • Foundation Level • Advanced Level • Test Automation • QTP • Foundation • Advanced • Framework driven • Selenium • Foundation • Advanced • Test Management • Quality Center • Test Link • Performance Testing • Load Runner • Silk Performer • Open STA, JMeter • Testing and QA Processes • Testing Metrics • Extended programs and diploma on Testing Talent Development • Test Automation • Manual Testing • Testing Leaderships • Customized Corporate Trainings • Specific Testing topic • Web based Application Testing • SOA Testing • DB Testing • GHTester. SOAP UI • Tailored to project and organization needs Confidential | Copyright © QA Agility Technologies • Agile Testing Alliance - Certifications • CP-BAT, CP-MAT, CP-AAT, CP-AAST • Specialized Workshops/Niche Trainings • Practical Test Strategy Formulation • Architecting Testing Solutions • Risk Based Testing • Certified Agile Tester and Agile Testing • ISTQB Certification • Foundation Level • Advanced Level • Test Automation • QTP • Foundation • Advanced • Framework driven • Selenium • Foundation • Advanced • Test Management • Quality Center • Test Link • Performance Testing • Load Runner • Silk Performer • Open STA, JMeter • Testing and QA Processes • Testing Metrics • Extended programs and diploma on Testing Talent Development • Test Automation • Manual Testing • Testing Leaderships • Customized Corporate Trainings • Specific Testing topic • Web based Application Testing • SOA Testing • DB Testing • GHTester. SOAP UI • Tailored to project and organization needs
  37. 37. Agile Testing Certifications
  38. 38.  Surest Way to Master Agile Testing  Pick up newer ways of doing Testing  Differentiate how old ways of testing may not work in swift moving agile projects  Learn optimized test design – utmost necessary to reduce defect leakage in a project where time is always a constraint.  Mind Map Test Design Technique  Pairwise/Combinatorial Techniques  Exploratory Test Design Technique  Increasing agility in finding defects  Real Agile Project (Multiple Drops, Multiple Sprints) Certified Professional – Master Agile Testing  Surest Way to Master Agile Testing  Pick up newer ways of doing Testing  Differentiate how old ways of testing may not work in swift moving agile projects  Learn optimized test design – utmost necessary to reduce defect leakage in a project where time is always a constraint.  Mind Map Test Design Technique  Pairwise/Combinatorial Techniques  Exploratory Test Design Technique  Increasing agility in finding defects  Real Agile Project (Multiple Drops, Multiple Sprints)
  39. 39.  Surest Way to learn cutting edge automation trends  Understanding huge importance of automation in today’s testing world  Hands on BDD,ATDD and TDD  Practice automation from concept to regression and test hardening iteration using:  Real Case Study  Real tools like – Cucumber, Fitnesse, Selenium, Hudson/Jenkins Certified Professional – Automation Agile Testing  Surest Way to learn cutting edge automation trends  Understanding huge importance of automation in today’s testing world  Hands on BDD,ATDD and TDD  Practice automation from concept to regression and test hardening iteration using:  Real Case Study  Real tools like – Cucumber, Fitnesse, Selenium, Hudson/Jenkins
  40. 40. Thank you Write to aditya.garg@agiletestingalliance.org #adigindia Linked in: https://www.linkedin.com/in/adigarg 40 Thank you Write to aditya.garg@agiletestingalliance.org #adigindia Linked in: https://www.linkedin.com/in/adigarg

×