Tool support for testing

1. Sinthia Gusfah Mitari
Program Studi S1 Sistem Informasi
Fakultas Sains dan Teknologi
Universitas Islam Negeri Sultan Syarif Kasim
Riau
Referensi : Graham et.al (2006)
http://www.uin-suska.ac.id/

2. Tool support for testing
You may be wishing that you had a magic tool that would automate all of the
testing for you. If so, you will be disappointed. However, there are a number
of very useful tools that can bring significant benefits. In this chapter we will
see that there is tool support for many different aspects of software testing.
We will see that success with tools is not guaranteed, even if an appropriate
tool is acquired - there are also risks in using tools. There are some special
considerations mentioned in the Syllabus for certain types of tool: test
execution tools, performance testing tools, static analysis tools and test
management tools.

3. Types of test tool
Test tool classification
The tools are grouped by the testing activities or areas that are supported by
a set of tools, for example, tools that support management activities, tools to
support static testing, etc. .
There is not necessarily a one-to-one relationship between a type of tool
described here and a tool offered by a commercial tool vendor or an open-
source tool. Some tools perform a very specific and limited function
(sometimes called a 'point solution'), but many of the commercial tools
provide support for a number of different functions (tool suites or families of
tools). For example a 'test management' tool may provide support for
managing testing (progress monitoring), configuration management of
testware, incident management, and requirements management and
traceability; another tool may provide both coverage measurement and test
design support.

4. Cont...
A tool that measures some aspect of software may have unexpected side-
effects on that software. For example, a tool that measures timings for non-
functional (performance) testing needs to interact very closely with that
software in order to measure it. A performance tool will set a start time and a
stop time for a given transaction in order to measure the response time, for
example. But the act of taking that measurement, i.e. storing the time at
those two points, could actually make the whole transaction take slightly
longer than it would do if the tool wasn't measuring the response time. Of
course, the extra time is very small, but it is still there. This effect is called the
'probe effect'.

5. Cont...
Tool support for management of testing and tests
Test management tools
The features provided by test management tools include those listed below.
Some tools will provide all of these features; others may provide one or more
of the features, however such tools would still be classified as test
management tools.

6. Cont...
Features or characteristics of test management tools include support for:
•management of tests (e.g. keeping track of the associated data for a given
set of tests, knowing which tests need to run in a common environment,
number of tests planned, written, run, passed or failed);
•scheduling of tests to be executed (manually or by a test execution tool);
management of testing activities (time spent in test design, test execution,
whether we are on schedule or on budget);
•interfaces to other tools, such as:
otest execution tools (test running tools);
oincident management tools;
orequirement management tools;
oconfiguration management tools;
•traceability of tests, test results and defects to requirements or other
sources;

7. Cont...
•logging test results (note that the test management tool does not run tests,
but could summarize results from test execution tools that the test manage-
ment tool interfaces with);
•preparing progress reports based on metrics (quantitative analysis), such as:
tests run and tests passed;
incidents raised, defects fixed and outstanding.

8. Cont...
Requirements management tools
Are requirements management tools really testing tools? Some people may
say they are not, but they do provide some features that are very helpful to
testing. Because tests are based on requirements, the better the quality of
the require- ments, the easier it will be to write tests from them. It is also
important to be able to trace tests to requirements and requirements to
tests, as we saw in Chapter 2.
Some requirements management tools are able to find defects in the require-
ments, for example by checking for ambiguous or forbidden words, such as
'might', 'and/or', 'as needed' or '(to be decided)'.
Features or characteristics of requirements management tools include
support for:

9. Cont...
storing requirement statements;
storing information about requirement attributes;
checking consistency of requirements;
identifying undefined, missing or 'to be defined later' requirements;
prioritizing requirements for testing purposes;
traceability of requirements to tests and tests to requirements,
functions or features;
traceability through levels of requirements;
interfacing to test management tools;
coverage of requirements by a set of tests (sometimes).

10. Cont...
Incident management tools
This type of tool is also known as a defect-tracking tool, a defect-
management tool, a bug-tracking tool or a bug-management tool. However,
'incident management tool' is probably a better name for it because not all
of the things tracked are actually defects or bugs; incidents may also be
perceived problems, anomalies (that aren't necessarily defects) or
enhancement requests. Also what is normally recorded is information about
the failure (not the defect) that was generated during testing - information
about the defect that caused that failure would come to light when someone
(e.g. a developer) begins to investigate the failure.
Incident reports go through a number of stages from initial identification and
recording of the details, through analysis, classification, assignment for fixing,
fixed, re-tested and closed, as described in Chapter 5. Incident manage- ment
tools make it much easier to keep track of the incidents over time.
Features or characteristics of incident management tools include support for:

11. Cont...
storing information about the attributes of incidents (e.g. severity);
storing attachments (e.g. a screen shot);
prioritizing incidents;
assigning actions to people (fix, confirmation test, etc.);
status (e.g. open, rejected, duplicate, deferred, ready for confirmation test,
closed);
reporting of statistics/metrics about incidents (e.g. average time open,
number of incidents with each status, total number raised, open or closed).
Incident management tool functionality may be included in commercial test
management tools.