This document discusses software testing. It covers testing techniques like black box and glass box testing. It also discusses challenges like hard-to-simulate inputs, real-time systems, and cross-platform testing. Examples are provided to illustrate testing concepts like branches, paths, and unexpected inputs. The importance of planning testing early and considering unforeseen circumstances is emphasized.

1. Software Testing
CSE 492
Lesson 9

2. Testing Overview
 Pressman 17: Micro view
 Pressman 18: Macro view
 Techniques for Testing (How?)
 Motivation for Testing (Why?)
 Challenges of Testing
 Fiascos, War Stories, Lessons Learned

3. The “V” Lifecycle Model
product
phase product

4. The “V” Lifecycle Model

5. The “V” Lifecycle Model
Detailed Coded
Design Modules
Detailed Unit Test Coded
Design Modules
Plan
Coding

6. The “V” Lifecycle Model
User Follow-on
Needs Version
User Regression Test Follow-
Needs
Plan on Vers.
System Acceptance Test Release Maint.
Spec. Version
Plan
Design Hi-Level Integration Test Trial
Acc. Tst.
System
Design
Plan
Mod. Des. Detailed Unit Test Coded
Int.
Design Modules
Plan
Coding

7. Testing: Part of V&V
 Verification: “Are we building the product right?”
 Validation: “Are we building the right product?”
 Developers _______, Users _________

8. Types of Testing
 Black Box
 Glass Box
 Integration & Validation Testing

9. Glass Box Testing
 Contrive cases that cover statements
and branches as completely as
possible Exhaustive Testing
Path Testing
Branch Testing
Statement Testing

10. Branch vs. Path
1
7 2
8 3
Statement:
<1,2,3,4,5,6,12,13>
9 4
<1,2,7,8,9,10,11,6,12,13>
10 5
11 6
12
13

11. Branch vs. Path
1
7 2
8 3
Branch:
<1,2,3,4,5,6,12,13>
9 4
<1,2,3,4,5,3,4,5,6,12,13>
10 5 <1,2,7,8,9,10,11,6,12,13>
11 6 <1,2,7,8,6,12,6,12,13>
12
13

12. Branch vs. Path
1
7 2 Path:
8 3 <1,2,3,4,5,6,12,13>
<1,2,3,4,5,3,4,5,6,12,13>
9 4
<1,2,3,4,5,6,12,6,12,13>
10 5 <1,2,3,4,5,3,4,5,6,12,6,12,13>
11 6 <1,2,7,8,9,10,11,6,12,13>
<1,2,7,8,9,10,11,6,12,6,12,13>
12
<1,2,7,8,6,12,13>
13 <1,2,7,8,6,12,6,12,13>

13. Black Box Testing
 Focus is on functional requirements,
not inside code
 Make sure system functions as
intended with expected input
 Make sure system does not do
anything “bad” with unexpected input
 Best chance of success--plan early!

14. Special Challenges
 Hard-to-simulate stimuli
 Real-time, asynchronous systems
 Cross-platform systems




15. Tricky Cases
 Unexpected Input
 Divide by Zero
 Floating Point Arithmetic (esp. “if x = y then…”)




16. A Quote…
“Suppose a programmer is asked
to test a module he has written.
Testing has been described as a
process of executing a product with the intention
of finding faults. Testing is thus a destructive
process. [But] the programmer will not ordinarily
wish to destroy his work. If the fundamental
attitude is the usual protective one, chances of
highlighting faults is lower than if the major
motivation were truly destructive.” [Schach 99]

17. Schach continued…
“After a programmer has been
constructive and built a module,
testing requires the programmer
perform a destructive act, and attempt to destroy
the very thing he has created.” [Schach 99]

18. My theory…
Programmers handle the foreseen
and the well-understood rather
competently; errors often occur
in unforeseen circumstances, poorly understood
requirements and designs, and hard-to-test
situations.

19. The Bermuda Triangle
Pressman Problem 17.1
Program reads 3 integer values,
which represent edge lengths of a triangle;
program then prints a message indicating if
triangle is scalene, isosceles, or equilateral.
3, 3, 3
2, 4, 4
3, 1, 4

20. The Bermuda Triangle
Pressman Problem 17.1

21. The Bermuda Triangle
Revisited
New Requirement:
Check for (and report) Right Triangles

22. The Bermuda Triangle
Revisited
Program must check:
a^2 + b^2 = c^2
a+b>c
Long leg is hypotenuse
Pseudocode:
for x in 1..3 loop
get input[x]
end loop;
c := longest (input[x]: x = 1,2,3)
a, b := {other two sides}

23. The Bermuda Triangle
Revisited
Code:
i1,i2,i3:integer; --3 inputs
a,b,c:integer; --c will be hyp.
get (i1); get (i2); get (i3);
if i1 > i2 then --i1 or i3
if i1 > i3 then --i1 is hyp.
c := i1;
b := i2;
a := i3;
else --i3 is hyp.
c := i3;
b := i2;
a := i1;
end if;
else --i2 or i3

24. The Bermuda Triangle
Revisited
Code:
else --i2 or i3
if i2 > i3 then --i2 is hyp.
c := i2;
b := i3;
a := 11;
else --i3 is hyp.
c := i3;
b := i2;
a := i1;
end if;

25. The Bermuda Triangle
Revisited
New Requirement:
Accept floating point input

26. The Bermuda Triangle
Revisited
Floating point problems:
a := 0.5;
b := 1.2;
c := 1.3;
if c^2 = a^2 + b^2 then
print (“this is a right triangle”);
end if;

27. Linking Global Points
 One file: LAT/LON + Radius
 Second file: LAT/LONs
 Goal: If Point in File2 resides inside
Area in File1, create link

28. Other “War Stories”
 Therac Machine
 F-15 Air-to-Air Missile Firing
 Processor Enhancement
 600 ± 10 ms

29. Other Forms of Testing
 STRESS
 Test the system at max load conditions
 Recovery
 Test the system over max load conditions
 Performance
 Ensure system meets timing requirements
 Regression
 “All of the above” after a system modification

30. Conclusions
 Testing is a science
 statement vs. path
 Testing is an art
 seeing the unforeseen
 Good testing is difficult to perform
 stubs and drivers during integration
 simulating real-world conditions
 Solution?
 Well-planned & rigorous testing program--
not an ad hoc, haphazard one