Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Test Management Approach (TMap) Chris Hampton, PMP Practice Manager, Software Control & Testing Sogeti USA September 2007 1
  2. 2. 2
  3. 3. Agenda: • The State of, and Business Case for, Software Quality • What is testing? • Sogeti‘s Test Management Approach® (Tmap®) • Sogeti‘s Test Process Improvement® (TPI®) Methodology 3
  4. 4. Poor State of Software Delivery “Software failure costs the American economy $59.5 billion annually.” Source: NIST (2002) 90% of projects are delivered late 66% of projects rated unsuccessful 54% of projects over budget 30% of projects cancelled early Source: Standish 2003 4
  5. 5. The Business Case For QA • Compliance – Basel II, Sarbanes Oxley, HIPAA, etc… – Reduced liability from defective software and defective process • Cost Savings – Quality processes now reduce support costs later – Lost revenue • Satisfaction – Higher User satisfaction – Improved business alignment • Growth – Better quality and measurements allows development optimization – Increase ability to deliver 5
  6. 6. Where Quality Problems Originate 7% 10% Code Other 56% 27% Design Requirements Source: James Martin 6
  7. 7. How Quality Provides Tremendous Savings Source: Cutter Consortium/Forrester/Sogeti Planning Analysis System Production Design $14000 Code $140 Unit COST TO Test Unit REPAIR $1000 Test DEFECT Certification Test System $7000 Integration Test Code Test $4500 $2500 Time cost and resource effort increase exponentially later in the lifecycle April 22, 2010 7
  8. 8. What is testing? Testing is a process aimed at: • Finding defects in a controlled manner • Detecting the level of quality of the test object • Demonstrating the gap between specifications and the actual product • Demonstrating that the end product functions as called for in the requirements 8
  9. 9. Testing is not . . . . Testing is not: • Implementation • Acceptance • Defect repair • A phase after development – although testing includes a phase after development • Training on a new system 9
  10. 10. Why do we test ? • To mitigate business risks that have been identified: – Validate software quality – Verify integration of business processes – Reduce time-to-market – Competitive purposes • To ensure business usability of the software: – Release low-error/known-error software – Move high quality software (meets or exceeds quality expectations) into production – Demonstrate the usefulness of new technology 10
  11. 11. Testing of information systems What do we test? • Application Software • Functionality • Hardware • Continuity • System Software • Performance • Procedures • Usability • Documentation • Interoperability (between different applications and systems) 11
  12. 12. Structured Software Testing • Testing everything is impractical – Funding limitations – Limited time and resources – Diminishing value after a certain point • Isn‘t there a more effective way? • Yes! Structured software testing: a risk-based, quality- centric approach to testing 12
  13. 13. Test Management Approach (TMap®) • Sogeti‘s methodology for structured testing of software products. • TMap® has evolved to be the standard for software testing in Europe and quickly gaining traction in the US. – It is being used by more than three hundred Dutch, Belgium and German organizations. – Industry adoption includes : Financial Services Insurance Government Consumer Electronics Telecommunications Medical systems 13
  14. 14. 4 Essentials of TMap® • Business-driven test management (BDTM) approach • Structured test process • Complete tool box • Adaptive test method BDTM puts focus on risk-based testing—only testing what has impacts to the organization. 14
  15. 15. Business Driven Test Management • Choices must be made in what is tested and how thoroughly. • This is an economic decision based on: – The risks that an organization thinks it will incur – The availability of time and resources – The results the customer wishes to achieve. • The choices based on risks, result, time and cost constitutes the basis for the BDTM approach. 15
  16. 16. The BDTM Process 1. Formulate the test goals. 2. Determine the risk class for each combination of characteristic and object part. 3. Determining the depth of testing required. • Based on risk/resource trade off 4. Develop the test estimate and test plan. 5. Allocate the appropriate test techniques. 6. Throughout the test process, the test manager provides the stakeholders with insight into and control options over test process and test object. 16 16
  17. 17. Risk Definition • Testing should be based on the mitigation of risk and validation of expected quality defined by the business requirements • Definition of risk – A ―risk‖ is a chance of a failure occurring, related to expected damage to the project (should the failure occur) • Risk Categories – Risks identified in different categories such as business risks, project risks, product risks, process risks • Risk Ranking – Risk are ranked in criticality relative to one another, by instituting a method of risk ranking (or ―risk rating‖) 17
  18. 18. How Risk Ranking Works 1. Assemble a team of people representing various roles from the project – PMs, Business Leads/SMEs, Test Leads, Tech Leads 2. Create an initial list of risks 3. Team assigns a numeric value to each risk = the probability of occurrence of each risk 4. Team assigns a second value to each risk, representing the impact on the project/organization 5. Multiply the two values together (the probability of occurrence X the impact) 6. The result is a relative value for each risk 7. Order the risks by their relative risk values – ―risk rating‖ or ―risk ranking‖ – Ranking helps manage the most critical risks, especially those falling in the middle tier of the ranking 8. Periodically update the list of risks 18
  19. 19. Risks – quality characteristics matrix Multi-Level Matrix High High Priority/ High Priority/ High Priority/ Priority Low Risk Medium Risk High Risk Characteristics Medium Medium Priority/ Medium Priority/ Medium Priority/ Quality Priority Low Risk Medium Risk High Risk Low Low Priority/ Low Priority/ Low Priority/ Priority Low Risk Medium Risk High Risk Low Medium High Risks 19
  20. 20. Ten steps for creating a test case Step 10 CREATE TEST CASES Step 9 Select suitable test techniques Step 8 Associate test scenarios with risks Facilitated Step 7 Create test scenarios for each subsystem Sessions Step 6 Create subsystem / quality characteristic pairings Step 5 Rank relevant quality characteristics Step 4 Agree on relevant quality characteristics Step 3 Rank risks Step 2 Identify risks Step 1 Identify sub-systems / sub-processes FUNCTIONAL AREA 20
  21. 21. Structured Test Process • Master Test Plan, managing the total test process – Acceptance and System Tests – Developmental Test – Supporting Processes • Lifecycle Model 21
  22. 22. Master Test Plan • Developed by the test manager in partnership with the appropriate stakeholders • Jointly define: – What will be tested for each test level, – When it will be tested – To what degree of thoroughness. • This plan constitutes the basis for the test plans for the separate test levels. – Unit Testing – Systems Integration Testing – Acceptance Testing – Developmental Testing 22
  23. 23. Life Cycle Model • The life cycle model is a generic model. – Can be applied to all test levels and test types – Used in parallel with the life cycle models for system development. • Test activities are divided across seven phases: – Planning – Control – Setting up and maintaining infrastructure – Preparation, – Specification – Execution – Completion 23
  24. 24. TMap - Structured testing lifecycle PREPARATION EXECUTION -Testability review -Pretest -Allocate techniques -(Re)test -Requirements review -Checking -Data cycle test SPECIFICATION -Assessing -Process cycle test -Create test scripts -Debugger -Other tests -Record and playback -Create infrastructure -Specify test cases -Monitoring -Create/assess -Preserve testware infrastructure -Evaluate process -Test specification -Data flow test P S E C P&C COMPLETION -Preserve testware for PLANNING & CONTROL future use -Inspection and study -Test strategy -Evaluate process -Develop test strategy -Test estimation oRisk analysis -Reports oTest Estimation (TPA) -Management tools -Setup organization -Budgeting tools -Prepare test plan -Defect management -Management and control tool 24
  25. 25. Complete Tool Kit • Techniques – Many techniques can be used in the test process. A test technique is a combination of actions to produce a test product in a universal manner. • TMap provides techniques for the following: Test estimation Defect management Creating metrics Product risk analysis Test design Product evaluation. 25
  26. 26. Adaptive Test Method • Complete And Flexible • Works with any SDLC – Iterative – RAD – Agile – Waterfall – RUP • Works with any technology – Mainframe – Web – ERP – SOA 26
  27. 27. 27
  28. 28. TMap® – Differentiators • Business-Driven Test Management • Risk-based test strategy • Efficient testing—coverage without overlap • Testing off of critical path as much as possible • Criteria and metrics about production readiness • Management of testing to project timelines • Compliments industry tools Proven methodology applied in hundreds of companies internationally over the past 14 years! 28
  29. 29. Benefits of Structure Testing • Defects are found early, costing less time/money to reach production  delivering a higher quality product • Required quality of the various test objects is tested for and validated, by focusing on testing for quality as a risk mitigation strategy • By keeping a larger percentage of the testing process/effort off the critical path, faster time-to-market results • Structured software testing is more cost-effective and efficient than non-structured testing approaches • Sound test coverage is provided, without the need to overlap phases • Establishes a test organization that is prepared and efficient • Delivers a repeatable process 29
  30. 30. Test Process Improvement (TPI ) • The Sogeti TPI® Model was developed to facilitate a stepwise improvement of the testing process. – Developed from practical experience from 30+ years of Sogeti work in testing. – Offers a frame of reference to determine the strengths and weaknesses of your current testing process. – Covers 20 key areas that need specific improvement to achieve a well define testing process. • Key TPI Areas: Test Strategy Commitment and Motivation Life-Cycle Model Test Functions and Training Moment of Involvement Scope of Estimating and Planning Methodology Communication Test Specification Techniques Reporting Static Test Techniques Defect Management Metrics Testware Management Test Automation Test Process Management Test Environment Evaluation and Low-Level Office Environment Testing 30
  31. 31. Typical TPI Process Steps 1. Interview key people 2. Describe the ‗As-Is‘ process including Strengths & Weaknesses 3. Describe the ‗To-Be‘ process 4. Document process improvement actions 5. Define expected benefits in terms of Objectives 6. Plan for Implementation of the Improvement Actions (short-term and long-term) 31
  32. 32. TPI - Test Maturity Matrix Key area / Scale 0 1 2 3 4 5 6 7 8 9 1 1 1 1 0 1 2 3 1 Test strategy A B C D 2 Life-cycle model A B 3 Moment of involvement A B C D 4 Estimating and planning A B 5 Test specification techniques A B 6 Static test techniques A B 7 Metrics A B C D 8 Test automation A B C 9 Test environment A B C A 10 11 Office environment Commitment and motivation Increasing Maturity A B C 12 Test functions and training A B C 13 Scope of methodology A B C 14 Communication A B C 15 Reporting A B C D 16 Defect management A B C 17 Testware management A B C D 18 Test process management A B C 19 Evaluation A B 20 Low-level testing A B C 32
  33. 33. Resources • Web Sites – Sogeti USA Web Site: – TMap Web Site: • TMap® certification (Netherlands) – Foundation – TMap ® professional Advanced/Expert – TMap ® management Advanced/Expert • Books 33
  34. 34. Sogeti SCT Services • Managed Testing Services (MTS) • Quality Management Services • Test Automation Services • Testing Professional Services • SAP Testing Services • 3000+ people worldwide in SCT space • 30+ yrs experience 34
  35. 35. Sogeti’s Geographical Presence A workforce of over 18,000 staff spread over 200 locations USA Sweden UK Denmark Ireland NL Belux Germany Switzerland France Spain India 35
  36. 36. Thank you! Chris Hampton, PMP Practice Manager, Software Control & Testing Sogeti USA Cell: (214) 549-7613 Sogeti Office: (972) 892-3400 36