Software Testing

Tips for Success when Selecting Testing Tools Marie Ackley ackley @ promail.com SASQAG panel discussion October 18, 2001

Different Categories of Testing Tools
Requirements Management
Configuration Management
Test Planning and Test Case Management
Test Evaluation - Coverage Analysis
Automated Testing Systems - Front End testing
Load Simulators and Performance Measurement Tools
Defect Tracking
others: Cyclomatic Complexity; Bounds Checkers; Comparators - Files, Screens, Reports; Data Generators - Combinatoric, Cause / Effect Node Graphing

Leading Cause of SHELF WARE in Testing Tools
is Putting the CART before the HORSE
Don't invest in Tools without having good QA Methods and Practices.
A good tool will compliment and augment good processes but not even the greatest tool can overcome process chaos.

Supports multi levels of linking and tracing: Requirements to Specifications to Test Cases, including hierarchies within those groups
Provides traceability - both forward and backward
Provides coverage analysis
Provides multi relationships: one-to-many, many-to-one and coverage is tracked for partials when relationship is one-to-many
Provides Change Management of requirements along with audit history and maybe security privileges for modifications
Import and export features so stuff came go and come between the tool and word processors or spreadsheets

Links to tool comparisons
the "Volere" Requirements Method - by Atlantic Systems Guild
http://www.systemsguild.com/GuildSite/Robs/Template.html
Automating Requirements Management - by Karl E. Wiegers
Detailed feature comparisons of many requirements management tools
and guidance on selecting a requirements management tool
http://www.processimpact.com/articles/rm_tools.html

"To change and to change for the better are two different things"
- German Proverb
Provides Identifications (indexing of artifacts)
Controls not only software code but documentation, hardware, firmware, and system environment configuration
Provides audit trail and history tracking
Provides customizable reports
High ease of use for View Only functions
Sufficient security levels for the Promote functions
Accounting and reporting for different ”Status Levels" - WIP, Waiting Room, Release/Production

Configuration Management Vendor Links - by UCM Central
http://www.snuffybear.com/ucmcentral_new_vendorlinks.htm
config-mgmt Newsgroup
http://www.faqs.org/faqs/by-newsgroup/comp/comp.software.config-mgmt.html

Import test cases from other file formats such as word processing or spreadsheet
Customizable reporting
Tracks history and status of test cases
Allows file attachments
Integration - can tool link to other tools such as your Requirements tool and your Automated Testing tool
Provide a Test Execution Results repository
Maintains Master Set of test cases and then can build “subsets” for project specific needs

Links for tool comparisons
Testing and Test Management Tools
http://www.methods-tools.com/tools/frames_testing.html
Information About Testing Tools -
http://www.cs.queensu.ca/home/shepard/testing.dir/under.construction/tool_list.html

Identifies what "unique" data combinations are needed
Uses an appropriate measurement technique
Decision; Condition/decision; Path; Predicate
Types of coverage included in the reporting
Branch; Loop; Multi; Operator; Operand
Integration - is it compatible with your other tools such as Automated Test Execution tools
Implementation - is the tool "invasive" (must be compiled along with the source code)

Experience With the Cost of Different Coverage Goals for Testing - by Brian Marick
http://www.testing.com/writings/experience.pdf
Coverage Analysis - by Steve Cornett
http://www.bullseye.com/webCoverage.html

Maintenance - object recognition when the application changes
Ease Of Use - complexity matches the programming talent of QA persons
Tool language and syntax matches what company uses: Visual Basic or C
Extensible Language - can use external subroutines or DLLs
Integration with other tools - Code Coverage, Test Case Management, Defect Tracking
Compatibilty - platforms (Client/Server, Web, Mainframe) Operating Systems(Windows, Mac, Unix, OS/2, Mainframe) and GUI Development (PowerBuilder, C, Visual Basic, Galaxy

Supports Data Driven inputs from flat files, spreadsheet files, and database connections
For Evaluation/Proof of Concept - include automating a Level One test
Making the Right Choice - by Elisabeth Hendrickson
http://www.qualitytree.com/feature/mtrc.pdf
Short Sharp Advice about how to choose a testing tool - Mark Fewster and Dorothy Graham
http://www.grove.co.uk/Tool_Information/Choosing_Tools.html
Tool Comparison between Rational, Mercury, Segue, and Compuware - by Ray Robinson
http://www.qacity.com/testtype/AUTOMATION_TOOLS.doc

Automated Testing Systems
Links to articles
Lessons in Test Automation - by Elfriede Dustin
http://www.sqa-test.com/articles.html
The Truth About Automated Test Tools - by Linda Hayes
http://www.worksoft.com/resources/4-97.htm
Test Automation - by Bret Pettichord
http://www.io.com/~wazmo/qa/#test_automation
Testers' Network - VeriTest website
http://www.veritest.com/testers'network
Test Automation Frameworks - by Carl J. Nagle
http://members.aol.com/sascanagl/FRAMESDataDrivenTestAutomationFrameworks.htm

Load Simulators and Performance Measurement
Reporting includes graphical charts
Debugging features for script creation
Reporting includes graphical charts and logs of performance for server speed and responsiveness
Quantity of simulated users
Variations in session-specific items such as cookies, usernames, passwords, proxies and ASP-session IDs
Support for your middleware - SQLnet, DBLib or CBLib, SQL Server, ODBC, Telnet
Challenges include SSL sessions, cookies, forms, proxies, password authentication, dynamic HTML, full HTTP, SSL, XML, Javascript, ActiveX, WAP, Java
Platforms and protocols

Load Simulators and Performance Measurement
Link to tool comparisons
Load and Performance Test Tools - by Rick Hower
http://www.softwareqatest.com/qatweb1.html
Software Research - TestWorks
http://www.methods-tools.com/tools/testing.html

Defect Tracking
Determine how many current users you have and consider and scaling for future users
Web based or client-server
Features: Customizable Fields; Customizable Reports; File Attachments; Audit Trail; Email linking; Security Levels
Provides customizable rules for the workflows, security privileges and notifications
Integration - does it link to your other tools

Defect Tracking
Tracking Down a Defect Management Tool - by Hung Quoc Nguyen
http://www.qacity.com/testgen/DefectTrackingNguyenSTQE701.pdf
The Problem with Problem Tracking - by Linda Hayes

Websites for Tool Information
Vendor contact information
http://www.testingfaqs.org/tools.htm
http://www.ondaweb.com/sti/stivend.htm#L
http://www.dbmsmag.com/pctest.html#P0890
Bulletin board forums and articles
http://www.qaforums.com
http://community.sqatester.com
http://www.qualitytree.com/links/links.htm
http://www.stqemagazine.com

Software Testing

by Softwarecentral

Accessibility

Upload Details

Usage Rights

1 Embed 1

Statistics

Software Testing Presentation Transcript