Chapter 9Testing the SystemShari L. PfleegerJo.docx

Chapter 9
Testing the System
Shari L. Pfleeger
Joann M. Atlee
4th Edition
4th Edition
Pfleeger and Atlee, Software Engineering: Theory and Practice
Chapter 9.*
Contents
9.1 Principles of system testing
9.2 Function testing
9.3 Performance testing
9.4 Reliability, availability, and maintainability
9.5 Acceptance testing

9.6 Installation testing
9.7 Automated system testing
9.8 Test documentation
9.9 Testing safety-critical systems
9.10 Information systems example
9.11 Real-time example
9.12 What this chapter means for you
Chapter 9.*
Chapter 9 ObjectivesFunction testingPerformance
testingAcceptance testingSoftware reliability, availability, and
maintainabilityInstallation testingTest documentationTesting
safety-critical systems
Chapter 9.*
9.1 Principles of System Testing
Source of Software Faults During Development

Chapter 9.*
9.1 Principles of System Testing System Testing
ProcessFunction testing: does the integrated system perform as
promised by the requirements specification?Performance
testing: are the non-functional requirements met?Acceptance
testing: is the system what the customer expects?Installation
testing: does the system run at the customer site(s)?
Chapter 9.*
System Testing Process (continued)Pictorial representation of
steps in testing process
Chapter 9.*
Techniques Used in System TestingBuild or integration
planRegression testingConfiguration managementversions and
releasesproduction system vs. development systemdeltas,
separate files and conditional compilationchange control

Chapter 9.*
Build or Integration PlanDefine the subsystems (spins) to be
testedDescribe how, where, when, and by whom the tests will be
conducted
Chapter 9.*
Example of Build Plan for Telecommunication
SystemSpinFunctionsTest StartTest EndOExchange1
September15 September1Area code30 September15
October2State/province/district25 October5
November3Country10 November20 November4International1
December15 December

Chapter 9.*
Example Number of Spins for Star Network
Spin 0: test the central computer’s general functions
Spin 1: test the central computer’s message-translation function
Spin 2: test the central computer’s message-assimilation
function
Spin 3: test each outlying computer in the stand alone mode
Spin 4: test the outlying computer’s message-sending function
Spin 5: test the central computer’s message-receiving function
Chapter 9.*
Regression TestingIdentifies new faults that may have been
introduced as current one are being correctedVerifies a new
version or release still performs the same functions in the same
manner as an older version or release
Chapter 9.*

Regression Testing StepsInserting the new codeTesting
functions known to be affected by the new codeTesting essential
function of m to verify that they still work properlyContinuing
function testing m + 1
Chapter 9.*
Configuration ManagementVersions and releasesProduction
system vs. development systemDeltas, separate files and
conditional compilationChange control
Chapter 9.*
Test TeamProfessional testers: organize and run the
testsAnalysts: who created requirementsSystem designers:
understand the proposed solutionConfiguration management
specialists: to help control fixesUsers: to evaluate issues that
arise

Chapter 9.*
9.2 Function Testing
Purpose and RolesCompares the system’s actual performance
with its requirementsDevelops test cases based on the
requirements document
Chapter 9.*
Cause-and-Effect GraphA Boolean graph reflecting logical
relationships between inputs (causes), and the outputs (effects)
or transformations (effects)
Chapter 9.*
Notation for Cause-and-Effect Graph

Chapter 9.*
Cause-and-Effect Graphs ExampleINPUT: The syntax of the
function is LEVEL(A,B) where A is the height in meters of the
water behind the dam, and B is the number of centimeters of
rain in the last 24-hour periodPROCESSING: The function
calculates whether the water level is within a safe range, is too
high, or is too lowOUTPUT: The screen shows one of the
following messages
1. “LEVEL = SAFE” when the result is safe or low
2. “LEVEL = HIGH” when the result is high
3. “INVALID SYNTAX”
depending on the result of the calculation
Chapter 9.*
Cause-and-Effect Graphs Example (Continued)Causes
The first five characters of the command “LEVEL”
The command contains exactly two parameters separated by a
comma and enclosed in parentheses
The parameters A and B are real numbers such that the water

level is calculated to be LOW
level is calculated to be SAFE
level is calculated to be HIGH
Chapter 9.*
Cause-and-Effect Graphs Example (Continued)Effects
1. The message “LEVEL = SAFE” is displayed on the screen
2. The message “LEVEL = HIGH” is displayed on the screen
The message “INVALID SYNTAX” is printed out
Intermediate nodes
1. The command is syntactically valid
2. The operands are syntactically valid
Chapter 9.*
Cause-and-Effect Graphs of LEVEL Function Example
Exactly one of a set of conditions can be invoked
At most one of a set of conditions can be invoked
At least one of a set of condition can be invoked

One effects masks the observance of another effect
Invocation of one effect requires the invocation of another
Chapter 9.*
Decision Table for Cause-and-Effect Graph of LEVEL
FunctionTest 1Test 2Test 3Test 4Test 5Cause 1IIISICause
2IIIXSCause 3ISSXXCause 4SISXXCause 5SSIXXEffect
1PPAAAEffect 2AAPAAEffect 3AAAPP
Chapter 9.*
9.3 Performance Tests
Purpose and RolesUsed to examine the calculationthe speed of
responsethe accuracy of the resultthe accessibility of the
dataDesigned and administrated by the test team

Chapter 9.*
9.3 Performance Tests
Types of Performance Tests
Stress tests
Volume tests
Configuration tests
Compatibility tests
Regression tests
Security tests
Timing tests
Environmental tests
Quality tests
Recovery tests
Maintenance tests
Documentation tests
Human factors (usability) tests
Chapter 9.*
9.4 Reliability, Availability, and Maintainability
DefinitionSoftware reliability: operating without failure under
given condition for a given time intervalSoftware availability:
operating successfully according to specification at a given
point in timeSoftware maintainability: for a given condition of
use, a maintenance activity can be carried out within stated time

interval, procedures and resources
Chapter 9.*
Different Level of Failure SeverityCatastrophic: causes death or
system lossCritical: causes severe injury or major system
damageMarginal: causes minor injury or minor system
damageMinor: causes no injury or system damage
Chapter 9.*
Failure DataTable of the execution time (in seconds) between
successive failures of a command-and-control
systemInterfailure Times (Read left to right, in
rows)330113811159291112151385077241088867012026114325
55242684221801011466001536552426822765176584573009726
34522551971936798161351148212331343571932363136974802
32330365122254310165293794412981029030052928116082810
11445296175510641783860983707338687242323293014618431
22611800865143530143108031101247943700875245729189744
73864461229909481082227548255091001010713717906150332
110456485485116018644116

Chapter 9.*
Failure Data (Continued)Graph of failure data from previous
table
Chapter 9.*
Uncertainty Inherent from Failure DataType-1 uncertainty: how
the system will be usedType-2 uncertainty: lack of knowledge
about the effect of fault removal
Chapter 9.*

Measuring Reliability, Availability, and MaintainabilityMean
time to failure (MTTF)Mean time to repair (MTTR)Mean time
between failures (MTBF)
MTBF = MTTF + MTTRReliability R =
MTTF/(1+MTTF)Availability A = MTBF
(1+MTBF)Maintainability M = 1/(1+MTTR)
Chapter 9.*
Reliability Stability and GrowthProbability density function f or
time t, f (t): when the software is likely to failDistribution
function: the probability of failureF(t) = ∫ f (t) dtReliability
Function: the probability that the software will function
properly until time tR(t) = 1- F(t)
Chapter 9.*
Uniformity Density FunctionUniform in the interval from
t=0..86,400 because the function takes the same value in that
interval

Chapter 9.*
Sidebar 9.4 Difference Between Hardware and Software
ReliabilityComplex hardware fails when a component breaks
and no longer functions as specifiedSoftware faults can exist in
a product for long time, activated only when certain conditions
exist that transform the fault into a failure
Chapter 9.*
Reliability PredictionPredicting next failure times from past
history
Chapter 9.*

Elements of a Prediction SystemA prediction model: gives a
complete probability specification of the stochastic processAn
inference procedure: for unknown parameters of the model
based on values of t₁, t₂, …, ti-1A prediction procedure:
combines the model and inference procedure to make
predictions about future failure behavior
Chapter 9.*
Reliability ModelThe Jelinski-Moranda model: assumesno type-
2 uncertaintycorrections are perfectfixing any fault contributes
equally to improving the reliabilityThe Littlewood modeltreats
each corrected fault’s contribution to reliability as independent
variableuses two source of uncertainty
Chapter 9.*
Successive Failure Times for Jelinski-MorandaIMean Time to
ith failureSimulated Time to ith
Failure12211224413261342845303063377737118426494854105
634116718312838313111171416719015333436

Chapter 9.*
9.5 Acceptance Tests
Purpose and RolesEnable the customers and users to determine
if the built system meets their needs and expectationsWritten,
conducted and evaluated by the customers
Chapter 9.*
9.5 Acceptance Tests
Types of Acceptance TestsPilot test: install on experimental
basisAlpha test: in-house testBeta test: customer pilotParallel
testing: new system operates in parallel with old system
Chapter 9.*

Result of Acceptance TestsList of requirementsare not
satisfiedmust be deletedmust be revisedmust be added
Chapter 9.*
9.6 Installation TestingBefore the testingConfigure the
systemAttach proper number and kind of devicesEstablish
communication with other systemThe testingRegression tests: to
verify that the system has been installed properly and works
Chapter 9.*
9.7 Automated System Testing
SimulatorPresents to a system all the characteristics of a device
or system without actually having the device or system
availableLooks like other systems with which the test system
must interfaceProvides the necessary information for testing
without duplication the entire other system

Chapter 9.*
9.8 Test DocumentationTest plan: describes system and plan
for exercising all functions and characteristicsTest specification
and evaluation: details each test and defines criteria for
evaluating each featureTest description: test data and
procedures for each testTest analysis report: results of each test
Chapter 9.*
9.8 Test Documentation
Documents Produced During Testing
Chapter 9.*
Test PlanThe plan begins by stating its objectives, which
shouldguide the management of testingguide the technical effort
required during testingestablish test planning and
schedulingexplain the nature and extent of each testexplain how
the test will completely evaluate system function and
performancedocument test input, specific test procedures, and

expected outcomes
Chapter 9.*
Parts of a Test Plan
Chapter 9.*
9.8 Testing Documentation
Test-Requirement Correspondence Chart
TestRequirement 2.4.1:
Generate and
Maintain Database Requirement 2.4.2:
Selectively Retrieve
DataRequirement 2.4.3:
Produced Specialized
Reports1. Add new recordX2. Add fieldX3. Change fieldX4.
Delete recordX5. Delete fieldX6. Create indexXRetrieve record
with a requested7. Cell numberX8. Water heightX9. Canopy
heightX10. Ground coverX11, Percolation rateX12. Print full
databaseX13. Print directoryX14. Print keywordsX15. Print
simulation summaryX

Chapter 9.*
Sidebar 9.8 Measuring Test Effectiveness and EfficiencyTest
effectiveness can be measured by dividing the number of faults
found in a given test by the total number of faults foundTest
efficiency is computed by dividing the number of faults found

in testing by the effort needed to perform testing
Chapter 9.*
Test DescriptionIncludingthe means of controlthe datathe
procedures
Chapter 9.*
Test Description Example
INPUT DATA:
Input data are to be provided by the LIST program. The
program generates randomly a list of N words of alphanumeric
characters; each word is of length M. The program is invoked
by calling
RUN LIST(N,M)
in your test driver. The output is placed in global data area
LISTBUF. The test datasets to be used for this test are as
follows:
Case 1: Use LIST with N=5, M=5

INPUT COMMANDS:
The SORT routine is invoked by using the command
RUN SORT (INBUF,OUTBUF) or
RUN SORT (INBUF)
OUTPUT DATA:
If two parameters are used, the sorted list is placed in OUTBUF.
Otherwise, it is placed in INBUF.
SYSTEM MESSAGES:
During the sorting process, the following message is displayed:
“Sorting ... please wait ...”
Upon completion, SORT displays the following message on the
screen:
“Sorting completed”
To halt or terminate the test before the completion message is
displayed, press CONTROL-C on the keyboard.
Chapter 9.*
Test Script for Testing The “change field”
Step N: Press function key 4: Access data file.
Step N+1: Screen will ask for the name of the date file.
Type ‘sys:test.txt’
Step N+2: Menu will appear, reading
* delete file
* modify file
* rename file

Place cursor next to ‘modify file’ and press RETURN
key.
Step N+3: Screen will ask for record number. Type ‘4017’.
Step N+4: Screen will fill with data fields for record 4017:
Record number: 4017 X: 0042 Y: 0036
Soil type: clay Percolation: 4 mtrs/hr
Vegetation: kudzu Canopy height: 25 mtrs
Water table: 12 mtrs Construct: outhouse
Maintenance code: 3T/4F/9R
Step N+5: Press function key 9: modify
Step N+6: Entries on screen will be highlighted. Move cursor to
VEGETATION field. Type ‘grass’ over ‘kudzu’ and press
RETURN key.
Step N+7: Entries on screen will no longer be highlighted.
VEGETATION field should now read ‘grass’.
Step N+8: Press function key 16: Return to previous screen.
Step N+9: Menu will appear, reading
* delete file
* modify file
* rename file
To verify that the modification has been
recorded,place cursor next to ‘modify file’ and press RETURN
key.
Step N+10: Screen will ask for record number. Type ‘4017’.
Step N+11: Screen will fill with data fields for record 4017:
Record number: 4017 X: 0042 Y: 0036
Soil type: clay Percolation: 4 mtrs/hr
Vegetation: grass Canopy height: 25 mtrs
Water table: 12 mtrs Construct: outhouse
Maintenance code: 3T/4F/9R

Chapter 9.*
Test Analysis ReportDocuments the result of testProvides
information needed to duplicate the failure and to locate and fix
the source of the problemProvides information necessary to
determine if the project is completeEstablish confidence in the
system’s performance
Chapter 9.*
Problem Report FormsLocation: Where did the problem
occur?Timing: When did it occur?Symptom: What was
observed?End result: What were the consequences?Mechanism:
How did it occur?Cause: Why did it occur?Severity: How much
was the user or business affected?Cost: How much did it cost?
Chapter 9.*
9.10 Information Systems Example
The Piccadilly SystemMany variables, many different test cases
to considerAn automated testing tool may be useful

Chapter 9.*
Things to Consider in Selecting a Test
ToolCapabilityReliabilityCapacityLearnabilityOperabilityPerfor
manceCompatibilityNonintrusiveness
Chapter 9.*
Sidebar 9.13 Why Six-Sigma Efforts Do Not Apply to
SoftwareA six-sigma quality constraint says that in a billion
parts, we can expect only 3.4 to be outside the acceptable
rangeIt is not apply to software becausePeople are variable, the
software process inherently contains a large degree of
uncontrollable variationSoftware either conforms or it does not,
there are no degree of conformanceSoftware is not the result of
a mass-production process

Chapter 9.*
9.11 Real-Time Example
Ariane-5 FailureSimulation might help preventing the
failureCould have generated signals related to predicted flight
parameters while turntable provided angular movement
Chapter 9.*
9.12 What This Chapter Means for YouShould anticipate testing
from the very beginning of the system life cycleShould think
about system functions during requirement analysisShould use
fault-tree analysis, failure modes and effect analysis during
designShould build safety case during design and code
reviewsShould consider all possible test cases during testing
History of Social Problems
You are the director of an agency that serves populations with
the following social problems:
· Substance abuse,
· Child abuse,
· Elderly health and social care,
· Issues in corrections,
· Homelessness,
· School behavior-related issues, and
· Poverty.

You are asked by your supervisor to train new volunteers at
your agency. You need to provide the volunteers background
education on your agency, the social problem, and the
populations being served. The topic is: “The History of Human
Services” pertaining to specific social problems in the
community.
Choose one of the aforementioned social problems that your
agency serves and discuss the following items:
1. Describe the issues related to the social problem at this point
in time.
2. Discuss the timeline of major historical issues of the
problem.
3. Provide two solutions to the problem.
4. Explain the predominant human service concerns during this
current year compared to the very start of your timeline?
Your training manual should be 2-3 pages in length. You are
welcome to include graphics for visual interest, use colorful
font, and apply APA Style citations and references when
thoughts, ideas, graphics, or another author’s work is utilized.

Chapter 9Testing the SystemShari L. PfleegerJo.docx

Recommended

Recommended

More Related Content

Similar to Chapter 9Testing the SystemShari L. PfleegerJo.docx

Similar to Chapter 9Testing the SystemShari L. PfleegerJo.docx (20)

More from mccormicknadine86

More from mccormicknadine86 (20)

Recently uploaded

Recently uploaded (20)

Chapter 9Testing the SystemShari L. PfleegerJo.docx