This presentation was made at yvrTesting group meeting on April 24, 2013. Today's dynamic testing techniques have become so sophisticated, especially with the introduction of automated frameworks, that testing teams often minimize the use of static testing techniques, or dispense with them completely. At the same time industrial research states that early static analysis is one of the top twelve most effective software quality factors. In Economics of Software Quality Capers Jones and Olivier Bonsignour note that "Indeed, a major 'knowledge gap' of the entire software engineering community is a lack of understanding of the combined results of software defect prevention methods, pretest defect removal methods such as inspections and static analysis, and effective software testing methods". Dynamic testing alone has never been sufficient to achieve high quality levels.

1. Forgotten? Ignored?
Obsolete?
Static Testing Techniques
Wednesday, 24 April, 13

2. “Risk management is project
management for adults.”
Tom DeMarco “Waltzing With Bears”,
2003
2
Wednesday, 24 April, 13

3. The Source of Industrial
Numbers
Jones, K; Bonsignor O.
The Economics of
Software Quality, 2012
13,000 projects at 600
corporations and 30
government organizations
3
Wednesday, 24 April, 13

4. Software Defect Potential
12%
8%
35%
25%
20%
Requrements
Architecture and Design
Coding
Documentation
Bad Fixes
4
Wednesday, 24 April, 13

5. The Main Risk Of Low
Quality
Damage costs because of software
failures
Wednesday, 24 April, 13

6. Two Challenges of
Software Quality
Most of testing
techniques are
relatively inefficient
It is much harder to
remove defects from
requirements and
design than it is from
source code
6
Wednesday, 24 April, 13

7. Static Testing Techniques
Do not require
execution of an
application-under-test
Can be applied to all
documents and
source code in a
project
7
Wednesday, 24 April, 13

8. Do You Use Static
Testing?
8
Wednesday, 24 April, 13

9. Static Testing Techniques
Classification
Object of testing:
code or
documentation
Degree of
formalization: from
formal inspection to
personal desk
checking
Automated or manual
9
Wednesday, 24 April, 13

10. List of Static Testing
Techniques
Technique Documents Code
Personal desk checking X X
Automated text checking X
Informal peer reviews / walkthrough X X
Client review of specifications X
User doc editing and proofreading X
Formal inspections X X
Automated static analysis X
Proof of correctness X
Pair Programming X
Scrum sessions X X
10
Wednesday, 24 April, 13

11. Efficiency of Inspections
11
0%
18%
35%
53%
70%
65.00%
35.00%
Most dynamic testing techniques
Formal Inspections
Wednesday, 24 April, 13

12. Roles and Responsibilities
3-8 participants
Author (of the inspected work product)
Moderator (keeps an inspection moving)
Scriber (records notes and other outputs)
Inspectors (contribute with findings)
12
Wednesday, 24 April, 13

13. Inspection Workflow
13
Wednesday, 24 April, 13

14. Inspection Rules
Adequate preparation time must be
assigned for inspectors before each
session
Inspection session duration must not be
longer than 4 hours
Defects found must be recorded
Efforts spent for preparation, execution
and defect fix must be recorded
Measurements and defect data must not
be used for personnel evaluation purposes
14
Wednesday, 24 April, 13

15. Inspection Output
Inspection
Quality
Metrics
Input
Corrected
Input
Process
Improvements
Knowledge
transfer
Additions and
Updates
15
Wednesday, 24 April, 13

16. Defect Removal From
Requirements and Design
Only static techniques can help remove defects from
requirements and design on time
Requirements generate ~20% of overall software defects
Toxic requirements
Size in FP
Requirements
Completeness
Requirements
Volatility
Delivered
Requirements
Defects*
1,000 80% 10% 150
10,000 60% 27% 2,100
100,000 20% 50% 27,500
* without pretest defect removal techniques
16
Wednesday, 24 April, 13

17. Coping with Low
Removal Efficiency
17
Defect Prevention Methods
Dynamic Testing
Techniques
Pretest Defect
Removal Techniques
Wednesday, 24 April, 13

18. Synergistic Relationship
Design
Inspection
Code Inspection QA Formal testing Worst Median Best
x 37 53 60
x 45 60 68
x 43 57 66
x x 65 80 87
x x 55 70 80
x x x 85 97 99
x x 70 85 90
x x x x 95 99 99.99
Source: Jones, K; Bonsignor O. The Economics of Software Quality
Data are collected in AT&T, Bellcore, Hewlett-Packard, IBM, Microsoft,
Motorola, and Raytheon 18
Wednesday, 24 April, 13

19. QA and Formal Testing
QA refers to an active SQA group provided that the
group conducts regular project audits
Formal testing includes:
a test plan is created
specs are complete enough to create test cases
w/o serious gaps
test library control tools are utilized
test coverage analysis tools are utilized
Wednesday, 24 April, 13

20. Why Static Techniques
Are Less Popular?
Small projects quality factor is individual skill set
Marketing: external and internal
Access to training and literature
Human being factor
Wrong belief in advantages and high efficiency
of dynamic techniques
Improved quality is not beneficial to bottom line
20
Wednesday, 24 April, 13

21. Summary
Relevant static testing techniques
address key project risks
save project costs
add significant value to overall project QA
A required quality level of a product can be
obtained only with synergetic combination of
techniques and approaches
21
Wednesday, 24 April, 13

22. To Be Continued
10+ Reasons To Use Static Testing
Techniques
Wednesday, 24 April, 13

23. References
Capers Jones, Olivier Bonsignor. Economics of
Software Quality, 2011
Tom Gilb, Dorothy Graham. Software Inspection, 1994
Boyd L. Summers. Software Engineering, Review and
Audit, 2011
Bill Brykczynski, Reginald Meeson, David A. Wheeler.
Software Inspection: Eliminating Software Defects,
1994
Stephen Palmer. Inspections. Blog post.
23
Wednesday, 24 April, 13