Software testing tools and the selection process

3,242 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,242
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
94
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Software testing tools and the selection process

  1. 1. Software testing tools and the selection process Geoff Quentin QBIT TestExpo 17th March 2005 1
  2. 2. Contents • Introduction • Specific points about processes • Specific types of tools • Implementation • The way forward 2
  3. 3. Testing - why do it? • Testing is to add confidence • Testing is to show compliance • Testing is to identify where to reduce risk • Identify defects and then have them fixed • Face problems if not done 3
  4. 4. The simple V Model USER UAT TEST ACCEPTANCE REQUIREMENTS MATERIAL TEST FUNCTIONAL SYS TEST SYSTEM SPECIFICATION MATERIAL TEST TECHNICAL INT TEST INTEGRATION SPECIFICATION MATERIAL TEST PROGRAM UNIT UNIT SPECIFICATION TEST TEST MATERIAL TestExpo © QBIT 2002 CODE 4
  5. 5. The project plan showing early test planning and creation Specification ____ Design (user) ____ Design (tech) ______ Design (prog) _____ Code ____ Plan unit test _____QC Unit test ____QC Plan integration test _______QC Integration test _____QC Plan acceptance _________QC Acceptance test ____QC Installation _QC 5
  6. 6. Or for more detail use ISO 12207 • Lots more activities • Specification paired with design • System treated separately from software 6
  7. 7. The fit with 12207 Specify The users (includes user Acceptance business acceptance test team) & completion requirements The acquirer The supplier System System System requirements qualification qualification analysis System test team testing design team Software System configuration System architectural team integration design Software Software Software requirements qualification qualification Software analysis testing test team design team Software Software architectural integration design Software Software detailed Component design testing Software development team (includes software test team) Software coding 7 TestExpo © QBIT 2002
  8. 8. The acceptance tester’s view and ISO 12207 Specify Acceptance business & completion requirements User acceptance test team System requirements Test item transmittal analysis report System architectural design 8
  9. 9. The system tester’s view and ISO 12207 Specify business requirements System System requirements qualification analysis testing System System architectural qualification design test team Test item transmittal report 9
  10. 10. Three documents for you • This presentation www.qbit.co.uk/testexpo • The Generic Test Process www.qbit.co.uk/gtp • The Consolidated Test Process www.qbit.co.uk/ctp 10
  11. 11. Why automate? • So what is the objective? • Return on investment • Achieve a difficult goal • Improve an existing process • Implement a new process • Scale up the process for bigger projects • Ensure compliance 11
  12. 12. Process focus • The basic process should be good • The process upon which the tool is based should be good • Conformity • Repeatability • Now seeing tools based on good processes 12
  13. 13. The test process • Record (set up reporting) • Create strategy • Plan tests • Analyse • Design • Schedule • Execute 13
  14. 14. PROJECT SPEC DOC 1. TEST STATUS REPORTING TEST TEST 2. TEST STRATEGY STRATEGY ITEM HIGH LEVEL TEST PLAN 3. TEST PLAN (various) LOW LEVEL TEST PLAN LOW LEVEL TEST PLAN LOW LEVEL TEST PLAN LOW LEVEL TEST PLAN The test 4. TEST ANALYSIS TEST OBJECTIVES LIST process 5. TEST DESIGN TEST SCRIPT model 6. TEST TEST SCHEDULE TEST ITEM TRANSMITTAL REPORT SCHEDULING TEST EXECUTION 7. TEST EXECUTION TEST TEST ITEM TRANSMITTAL LOG REPORT TEST INCIDENT REPORT TEST Copyright © QBIT 2002 SUMMARY 14 REPORT
  15. 15. So which level to apply the process? • Acceptance tests • System tests • Integration tests • Unit tests So who will use the chosen tool(s)? 15
  16. 16. Roles and responsibilities • Manager • Test team leader • Tool user (the test analyst) • Tools specialist (to implement) • Process specialist (to improve the process) 16
  17. 17. Resources required • Tools specialist • Trainer • Sponsor 17
  18. 18. Time table • Critical to introduce tools early • Time to familiarise • Time to tune • Time to achieve proficiency • Now to use in earnest 18
  19. 19. So what might be bought? • Tools to support existing processes • Tools to implement a new process • Tools to speed up existing processes • Tools to enhance, make possible, process • Tools to make economical a process 19
  20. 20. The tools on offer • Management of testing • Capture of tests • Playback of tests • Simulation, modelling and prototyping • Static analysis • Dynamic analysis • General purpose tools 20
  21. 21. Tools to support the review process • Verification – checking the test object • Validation – checking context • Certification • Defect recording and tracking • Metrics 21
  22. 22. Management of testing • Traceability • Identification of tests • Storage and retrieval of tests • Reporting progress of tests For the • Repeatability of tests test manager and team leader • Reusability of tests • Long term support for the testing effort 22
  23. 23. System design phase testing • Capture of tests • Selection and replay of tests For the • Prototyping team leader and test analyst • Simulation system test and • Mathematical modelling UAT • Change and configuration management 23
  24. 24. Software design phase • Performance prediction • Performance measurement • Change and configuration management For the system tester and the integration tester 24
  25. 25. Programming • Static analysis • Dynamic analysis • Unit test execution • Change and configuration management For the programmer 25
  26. 26. Research for tools • Need to identify where processes need tools • Need to gather metrics • Evaluate the return on investment • Tools to fit the processes • Process based tools to fit the problem 26
  27. 27. Deployment of the chosen tool(s) • Small project • Limit the scope for failure • Representative • Support • Training 27
  28. 28. Support once in use • Supplier • Internal • User groups • Independent consultants • Independent trainers 28
  29. 29. Process improvement • The tool must add value – improve efficiency – improve effectiveness – improve coverage – reduce costs – reduce duration – simplify the work – remove/reduce the chance of errors 29
  30. 30. So what is on offer at this QBIT tools fair? • Tools and consultancy 30
  31. 31. Visit • Cresta • T-Plan • Empirix • BCS/iseb • Vizuri • Sim/SQS • Compuware • Contemporary • Embarcadero • QBIT • Princeton Softech • Serpico • Facilita • ITR • Original Software • Testhouse • Transition • Books and journals – Artech House • Focus Frame – Better Software • Propel Systems – Professional Tester 31
  32. 32. Summary • Identify the objective(s) • Check the processes • Check the functionality • Check the overhead of deployment 32
  33. 33. Thank you This presentation www.qbit.co.uk/testexpo The Generic Test Process www.qbit.co.uk/gtp The Consolidated Test Process www.qbit.co.uk/ctp 33
  34. 34. 34

×