Software Testing


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Testing

  1. 1. Tips for Success when Selecting Testing Tools Marie Ackley ackley @ SASQAG panel discussion October 18, 2001
  2. 2. Different Categories of Testing Tools <ul><li>Requirements Management </li></ul><ul><li>Configuration Management </li></ul><ul><li>Test Planning and Test Case Management </li></ul><ul><li>Test Evaluation - Coverage Analysis </li></ul><ul><li>Automated Testing Systems - Front End testing </li></ul><ul><li>Load Simulators and Performance Measurement Tools </li></ul><ul><li>Defect Tracking </li></ul><ul><li>others: Cyclomatic Complexity; Bounds Checkers; Comparators - Files, Screens, Reports; Data Generators - Combinatoric, Cause / Effect Node Graphing </li></ul>
  3. 3. Leading Cause of SHELF WARE in Testing Tools <ul><li>is Putting the CART before the HORSE </li></ul><ul><li>Don't invest in Tools without having good QA Methods and Practices. </li></ul><ul><li>A good tool will compliment and augment good processes but not even the greatest tool can overcome process chaos. </li></ul>
  4. 4. Requirements Management <ul><li>Supports multi levels of linking and tracing: Requirements to Specifications to Test Cases, including hierarchies within those groups </li></ul><ul><li>Provides traceability - both forward and backward </li></ul><ul><li>Provides coverage analysis </li></ul><ul><li>Provides multi relationships: one-to-many, many-to-one and coverage is tracked for partials when relationship is one-to-many </li></ul><ul><li>Provides Change Management of requirements along with audit history and maybe security privileges for modifications </li></ul><ul><li>Import and export features so stuff came go and come between the tool and word processors or spreadsheets </li></ul>
  5. 5. Requirements Management <ul><li>Links to tool comparisons </li></ul><ul><li>the &quot;Volere&quot; Requirements Method - by Atlantic Systems Guild </li></ul><ul><ul><li> </li></ul></ul><ul><li>Automating Requirements Management - by Karl E. Wiegers </li></ul><ul><li>Detailed feature comparisons of many requirements management tools </li></ul><ul><li>and guidance on selecting a requirements management tool </li></ul><ul><ul><li> </li></ul></ul>
  6. 6. Configuration Management <ul><li>&quot;To change and to change for the better are two different things&quot; </li></ul><ul><li>- German Proverb </li></ul><ul><li>Provides Identifications (indexing of artifacts) </li></ul><ul><li>Controls not only software code but documentation, hardware, firmware, and system environment configuration </li></ul><ul><li>Provides audit trail and history tracking </li></ul><ul><li>Provides customizable reports </li></ul><ul><li>High ease of use for View Only functions </li></ul><ul><li>Sufficient security levels for the Promote functions </li></ul><ul><li>Accounting and reporting for different ”Status Levels&quot; - WIP, Waiting Room, Release/Production </li></ul>
  7. 7. Configuration Management <ul><li>Links to tool comparisons </li></ul><ul><li>Configuration Management Vendor Links - by UCM Central </li></ul><ul><ul><li> </li></ul></ul><ul><li>config-mgmt Newsgroup </li></ul><ul><li> </li></ul>
  8. 8. Test Planning and Test Case Management <ul><li>Import test cases from other file formats such as word processing or spreadsheet </li></ul><ul><li>Customizable reporting </li></ul><ul><li>Tracks history and status of test cases </li></ul><ul><li>Allows file attachments </li></ul><ul><li>Integration - can tool link to other tools such as your Requirements tool and your Automated Testing tool </li></ul><ul><li>Provide a Test Execution Results repository </li></ul><ul><li>Maintains Master Set of test cases and then can build “subsets” for project specific needs </li></ul>
  9. 9. Test Planning and Test Case Management <ul><li>Links for tool comparisons </li></ul><ul><li>Testing and Test Management Tools </li></ul><ul><ul><li> </li></ul></ul><ul><li>Information About Testing Tools - </li></ul><ul><li> </li></ul>
  10. 10. Test Evaluation - Coverage Analysis <ul><li>Identifies what &quot;unique&quot; data combinations are needed </li></ul><ul><li>Uses an appropriate measurement technique </li></ul><ul><ul><ul><li>Decision; Condition/decision; Path; Predicate </li></ul></ul></ul><ul><li>Types of coverage included in the reporting </li></ul><ul><ul><ul><li>Branch; Loop; Multi; Operator; Operand </li></ul></ul></ul><ul><li>Integration - is it compatible with your other tools such as Automated Test Execution tools </li></ul><ul><li>Implementation - is the tool &quot;invasive&quot; (must be compiled along with the source code) </li></ul>
  11. 11. Test Evaluation - Coverage Analysis <ul><li>Links to tool comparisons </li></ul><ul><li>Experience With the Cost of Different Coverage Goals for Testing - by Brian Marick </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>Coverage Analysis - by Steve Cornett </li></ul><ul><ul><ul><li> </li></ul></ul></ul>
  12. 12. Automated Testing Systems - Front End testing <ul><li>Maintenance - object recognition when the application changes </li></ul><ul><li>Ease Of Use - complexity matches the programming talent of QA persons </li></ul><ul><li>Tool language and syntax matches what company uses: Visual Basic or C </li></ul><ul><li>Extensible Language - can use external subroutines or DLLs </li></ul><ul><li>Integration with other tools - Code Coverage, Test Case Management, Defect Tracking </li></ul><ul><li>Compatibilty - platforms (Client/Server, Web, Mainframe) Operating Systems(Windows, Mac, Unix, OS/2, Mainframe) and GUI Development (PowerBuilder, C, Visual Basic, Galaxy </li></ul>
  13. 13. Automated Testing Systems - Front End testing <ul><li>Supports Data Driven inputs from flat files, spreadsheet files, and database connections </li></ul><ul><li>For Evaluation/Proof of Concept - include automating a Level One test </li></ul><ul><li>Links to tool comparisons </li></ul><ul><li>Making the Right Choice - by Elisabeth Hendrickson </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>Short Sharp Advice about how to choose a testing tool - Mark Fewster and Dorothy Graham </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>Tool Comparison between Rational, Mercury, Segue, and Compuware - by Ray Robinson </li></ul><ul><ul><ul><li> </li></ul></ul></ul>
  14. 14. Automated Testing Systems <ul><li>Links to articles </li></ul><ul><li>Lessons in Test Automation - by Elfriede Dustin </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>The Truth About Automated Test Tools - by Linda Hayes </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>Test Automation - by Bret Pettichord </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>Testers' Network - VeriTest website </li></ul><ul><ul><ul><li>'network </li></ul></ul></ul><ul><li>Test Automation Frameworks - by Carl J. Nagle </li></ul><ul><ul><li> </li></ul></ul>
  15. 15. Load Simulators and Performance Measurement <ul><li>Reporting includes graphical charts </li></ul><ul><li>Debugging features for script creation </li></ul><ul><li>Reporting includes graphical charts and logs of performance for server speed and responsiveness </li></ul><ul><li>Quantity of simulated users </li></ul><ul><li>Variations in session-specific items such as cookies, usernames, passwords, proxies and ASP-session IDs </li></ul><ul><li>Support for your middleware - SQLnet, DBLib or CBLib, SQL Server, ODBC, Telnet </li></ul><ul><li>Challenges include SSL sessions, cookies, forms, proxies, password authentication, dynamic HTML, full HTTP, SSL, XML, Javascript, ActiveX, WAP, Java </li></ul><ul><li>Platforms and protocols </li></ul>
  16. 16. Load Simulators and Performance Measurement <ul><li>Link to tool comparisons </li></ul><ul><li>Load and Performance Test Tools - by Rick Hower </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>Software Research - TestWorks </li></ul><ul><ul><ul><li> </li></ul></ul></ul>
  17. 17. Defect Tracking <ul><li>Determine how many current users you have and consider and scaling for future users </li></ul><ul><li>Web based or client-server </li></ul><ul><li>Features: Customizable Fields; Customizable Reports; File Attachments; Audit Trail; Email linking; Security Levels </li></ul><ul><li>Provides customizable rules for the workflows, security privileges and notifications </li></ul><ul><li>Integration - does it link to your other tools </li></ul>
  18. 18. Defect Tracking <ul><li>Links to tool comparisons </li></ul><ul><li>Tracking Down a Defect Management Tool - by Hung Quoc Nguyen </li></ul><ul><ul><li> </li></ul></ul><ul><li>The Problem with Problem Tracking - by Linda Hayes </li></ul><ul><ul><ul><li> </li></ul></ul></ul>
  19. 19. Websites for Tool Information <ul><li>Vendor contact information </li></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul><ul><li>Bulletin board forums and articles </li></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul>