cyclomatic complexity

1. 1
Cyclomatic Complexity

2. 2

3. What is it?
 A software metric used to measure the complexity
of software
 Developed by Thomas McCabe
 Described (informally) as the number of decision
points + 1

4. Cyclomatic Complexity V(G)
Computing the cyclomatic
complexity:
number of simple decisions + 1
or
number of enclosed areas + 1
In this case, V(G) = 4
From Pressman Slides - Software Engineering a Practical Approach 6,e

5. Graph Complexity
(Cyclomatic Complexity)
A number of industry studies have indicated
that the higher V(G), the higher the probability
or errors.
V(G)
modules
modules in this range are
more error prone
From Pressman Slides - Software Engineering a Practical Approach 6,e

6. Basis Path Testing
Next, we derive the
independent paths:
Since V(G) = 4,
there are four paths
Path 1: 1,2,3,6,7,8
Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive test
cases to exercise these
paths.
1
2
3
4
5 6
7
8
From Pressman Slides - Software Engineering a Practical Approach 6,e

7. What is the complexity?
public void howComplex() {
int i=20;
while (i<10) {
System.out.printf("i is %d", i);
if (i%2 == 0) {
System.out.println("even");
} else {
System.out.println("odd");
}
}
}

8. What is the complexity V(G)?
public void howComplex() {
int i=20;
while (i<10) {
System.out.printf("i is %d", i);
if (i%2 == 0) {
System.out.println("even");
} else {
System.out.println("odd");
}
}
}
V(G) = 2 enclosed area + 1 = 3

9. Output from JavaNCSS

10. Cyclomatic Complexity
10
 Invented by Thomas McCabe (1974) to measure the
complexity of a program’s conditional logic
 Cyclomatic complexity of graph G equals #edges -
#nodes + 2
 V(G) = e – n + 2
 Also corresponds to the number of linearly
independent paths in a program

11. Converting Code to Graph
11
if expression1 then
statement2
else
statement3
end if
statement4
switch expr1
case 1:
statement2
case 2:
statm3
case 3:
statm4
end switch
statm5
(a)
(b)
do
statement1
while expr2
end do
statement3
(c)
CODE FLOWCHART GRAPH
T F
expr1
?
statm4
statm2 statm3
2
1 3
expr1
?
statm5
statm3
statm2 statm4
n1
n2 n3
n4
n1
n2 n4
n5
n3
T
F
expr2
?
statm3
statm1
n1
n2
n3

12. Example Paths
12
if expression1
then
statement2
end if
do
statement3
while expr4
end do
if expression5
then
statement6
end if
statement7
n1
n3
n2
n4
n5
n7
n6
e1
e2
e3
e4 e5
e6
e7
e8
e9
Paths:
P1 = e1, e2, e4, e6, e7, e8
P2 = e1, e2, e4, e5, e4, e6, e7, e8
P3 = e3, e4, e6, e7, e8, e10
P4 = e6, e7, e8, e10, e3, e4
P5 = e1, e2, e4, e6, e9, e10
P6 = e4, e5
P7 = e3, e4, e6, e9, e10
P8 = e1, e2, e4, e5, e4, e6, e9, e10
V(G) = e – n + 2 = 9 – 7 + 2 = 4

13. Example 1
13

14. 14
V=e-n+2=11-9+2=4

15. 15

16. 16
V=e-n+2=11-9+2=4

17. • The testing process
• Determining the test methodology phase
• Planning the tests
• Test design
• Test implementation
• Test case design
• Test case data components
• Test case sources
• Automated testing
• The process of automated testing
• Types of automated testing
• Advantages and disadvantages of automated testing
• Alpha and beta site testing programs
Chapter 10 – Software Testing -
Implementation

18. Introduction
 Need to discuss
 testing effectiveness and
 testing efficiency.
 Equivalently, want to reduce number of undetected
errors and yet test with fewer resources over less
time.
 Need to design testing procedures to be effective in
detecting errors and doing so as efficiently as
possible.
 Also want to look at automatic testing too.

19. Introduction – Topics to Cover
 Much of this set of slides focuses on testing at
various levels:
 Unit tests
 Integration tests
 System tests.

20. Ultimate Desired Outcomes for the
Chapter
 Today:
 Describe the process of planning and designing
tests
 Discuss the sources for test cases, with their
advantages and disadvantages
 Next:
 List the main types of automated software tests
 Discuss the advantages and disadvantages of
automated computerized testing as compared to
manual testing
 Explain alpha and beta site test implementation and
discuss their advantages and disadvantages.

21. 10.1 The Testing Process
 Testing is done throughout the development
process.
 Testing is divided into phases beginning in the
design phase and ending at the customer’s site.
 Testing process is illustrated in the next slide.
 The two fundamental decisions that must be made
before planning for testing can occur are:
 What is the required software quality standard, and
 What is the software testing strategy.

22. Determining the test
methodology
Planning the tests
Designing the tests
Performing the tests
(implementation)

23. Determining the Appropriate
Software Quality Standard
 Different standards required for different software
applications.
 e.g. safety-critical software or aircraft instrumentation -
critical.
 In other cases, a medium-level quality would be sufficient,
and
 So, the expected damage resulting from failed software
impacts standard of software quality.
 Samples of damage to customers and users as well as to

24. 1. Endangers the safety of human beings
2. Affects an essential organizational function with no system
replacement capability available
3. Affects functioning of firmware, causing malfunction of an
entire system
4. Affects an essential organizational function but a replacement
is available
5. Affects proper functioning of software packages for business
applications
6. Affects proper functioning of software packages for a private
customer
7. Affects functioning of a firmware application but without
affecting the entire system.
8. Inconveniences the user but does not prevent accomplishment
of the system’s capabilities

25. 1. Financial losses
* Damages paid for physical injuries
Aircraft or auto instrumentation; health equipment…. Law suites!!
* Damages paid to organizations for malfunctioning of
software
Companies have many lawyers on staff!!!
* Purchase cost reimbursed to customers
* High maintenance expenses for repair of failed systems
2. Non-quantitative damages
* Expected to affect future sales
* Substantially reduced current sales

26. Determining Software Testing Strategy
 Big Bank or Incremental? So, do we want the
testing strategy to be big bang or incremental?
 Major testing at end in the past….
 If incremental, top down or bottom up?
 Which parts of the testing plan should be done
using white box testing?
 Black box?
 Which parts of the test plan should be done using
an automated test model?

27. Planning the Tests
 We need to undertake:
 Unit tests
 Integration tests, and
 System Tests.
 Unit tests deal with small chunks – modules,
functions, objects, classes;
 Integration tests deal with units constituting a
subsystem or other major hunks of capability, and
 System tests refer to the entire software
package or system.
 These are often done by different
constitutencies!!

28. Lots of Questions
 So we first need to consider five basic issues:
 What to test
 Which sources do we use for test cases
 Who is to perform the tests
 Where to perform the tests, and
 When to terminate the tests.
 Questions with not so obvious answers!

29. What to Test
 We would like to test everything.
 Not very practical.
 Cannot undertake exhaustive testing…
 Number of paths is infinite….
 Consider:
 Do we totally test modules that are 98% reused?
 Do we really need to test things that have been
repeatedly tested with only slight changes?
 How about testing by newbees?
 Testing on sensitive modules that pose lots of risk?

30.  So, which modules need to be unit tested?
 Which integrations should be tested?
 Maybe low priority applications tested in unit
testing may not be needed or included in the
system tests….
 Lots of planning is needed, as testing IS a very
expensive undertaking!

31. Rating Units, Integrations, and
Applications
 We need to rate these issues to determine their
priority in the testing plan.
 Rate based on two factors:
 1. Damage severity level – severity of results if
module / application fails.
 How much damage is done??
 Will it destroy our business? Our reputation??
 2. Software risk level – what is the probability
of failure.
 Factors affecting risk – next slide:

32. Module/Application Issues
1. Magnitude (size)
2. Complexity and difficulty
3. Percentage of original software (vs. percentage of
reused software)
Programmer Issues
4. Professional qualifications
5. Experience with the module's specific subject matter.
6. Availability of professional support (backup of
knowledgeable and experience).
7. Acquaintance with the programmer and the ability to
evaluate his/her capabilities.

33. Computations
 Essential to calculate risk levels
 We must budget our testing due to high cost
 But we must temper our testing with cost/risk!
 Helps to determine what to test and to what
extent.
 Use a ‘combined’ rating bringing together
damage severity (how serious would the damage
be?) and risk severity / probability.
 Sample calculations: A is damage; B is risk.
 C = A + B
 C = k*A + m * B
 C = A * B // most familiar with this one…

34.  If we are including unit, integration, and applications in a
test plan, we need to know how much/many resources
are needed.
 Thus, we need to prioritize.
 Higher the rating, and priority more allocation of
testing resources are needed.
 Consider:
 Some tests are based on high percentages of reused code.
 Some applications are developed by new employees
 We typically use a 5 point scale, where 5 is high.
 (Variations include a 1-5 severity level and a probability of
0.0 to 1.0 of their occurrence.)
 Can see results for Super Teacher application in next
slide.

35. Combined rating method
Application Damage
Severity
Factor A
Damage
Severity
Factor B
A +B 7xA+2xB A x B
1. Input of test results 3 2 5 (4) 25 (5) 6 (4)
2. Interface for input and output of pupils’data to
and from other teachers
4 4 8 (1) 36 (1) 16 (1)
3. Preparation of lists of low achievers 2 2 4 (5-6) 18 (7) 4 (5-6)
4. Printing letters to parents of low achievers 1 2 3 (7-8) 11 (8) 2 (8)
5. Preparation of reports for the school principal 3 3 6 (3) 27 (3) 9 (3)
6. Display of a pupil’s achievements profile 4 3 7 (2) 34 (2) 12 (2)
7. Printing of pupil’s term report card 3 1 3 (7-8) 23 (6) 3 (7)
8. Printing of pupil’s year-end report card 4 1 4 (5-6) 26 (4) 4 (5-6)
(Damage / p())

36. 1. Which Sources Should be Used for
Test Cases?
 Do we use live test cases or synthetic test cases.
 All three types of tests should consider these.
 Use live data or contrived (dummy) data??
 What do you think??
 Also need to consider single / combined tests and
the number of tests.
 How about if the testing is top down? Bottom up?
What sources do you think might be needed then??

37. Who Performs the Tests?
 Unit Testing – done by the programmer and/or
development team.
 Integration Testing – can be the development team
or a testing unit.
 System Testing – usually done by an independent
testing team (internal or external (consultants) team.
 For small companies, another testing team from
another development team can be used and swapped.
 Can always outsource testing too.

38. Where to Perform the Tests?
 Typically at the software developer’s site..
 For system tests, test at developer’s or
customer’s site (target site).
 If outsourced, testing can be done at
consultant’s site.

39. When are Tests Terminated?
 This is always the $64,000 question!!
 Decision normally applies to system tests.
 Five typical alternatives
 1. Completed Implementation Route
 Test until all is error free. (good luck)
 All testing, regression testing;
 Disregards budget and timetable constraints.
 Applies to perfection approach

40.  2. Mathematical Models Application Route:
 Here modeling us used to estimate percentage of
undetected errors based on rate of error detection.
 When detection rate falls below a certain level, stop.
 Disadvantage: math model may not fully represent
the project’s characteristics.
 Thus testing may be cut short or extended too far.
 Advantage: Well-defined stopping point.

41.  3. Error Seeding Route
 Here, we seed errors prior to testing.
 Underlying assumption is that percentage of
discovered seeded errors will correspond to the
percentage of real errors detected.
 Stop once residual percentage of undetected
seeded errors reaches a predefined level considered
acceptable for ‘passing’ the system.
 Disadvantages: extra workload for testers; also
based on past experiences of some testers;
 Too, seeding method can not accurately estimate
the residual rate of undetected errors in unfamiliar
systems.

42.  4. The dual independent testing teams route:
 Here two teams implement the testing process
independently.
 Compare lists of detected errors.
 Calculate the number of errors left undetected
 Lots of statistics here.
 High costs. Justified when??

43.  5. Termination after resources have petered out.
 This means stop when budgets or time for testing has
run out.
 Very common in industry

44. Test Plan
 Lastly, system testing is documented in a
software test plan.
 Common formats are available.

45. Test Design and Software Test Plan
 Products of Test Design
 Detailed design and procedures for each test
 The input database / files for testing.
 There are standard software test plans (STP)
templates

46. 1 Scope of the tests
1.1 The software package to be tested (name, version and
revision)
1.2 The documents that provide the basis for the planned
tests
2 Testing environment
2.1 Sites
2.2 Required hardware and firmware configuration
2.3 Participating organizations
2.4 Manpower requirements
2.5 Preparation and training required of the test team

47. 3 Tests details (for each test)
3.1 Test identification
3.2 Test objective
3.3 Cross-reference to the relevant design document and the
requirement document
3.4 Test class
3.5 Test level (unit, integration or system tests)
3.6 Test case requirements
3.7 Special requirements (e.g., measurements of response times,
security requirements)
3.8 Data to be recorded
4 Test schedule (for each test or test group) including time
estimates for:
4.1 Preparation
4.2 Testing
4.3 Error correction
4.4 Regression tests

48. 1 Scope of the tests
1.1 The software package to be tested (name, version
and revision)
1.2 The documents providing the basis for the designed
tests (name and
version for each document)
2 Test environment (for each test)
2.1 Test identification (the test details are documented
in the STP)
2.2 Detailed description of the operating system and
hardware configuration
and the required switch settings for the tests
2.3 Instructions for software loading

49. 3. Testing process
3.1 Instructions for input, detailing every step of the
input process
3.2 Data to be recorded during the tests
4.Test cases (for each case)
4.1 Test case identification details
4.2 Input data and system settings
4.3 Expected intermediate results (if applicable)
4.4 Expected results (numerical, message, activation of
equipment, etc.)
5. Actions to be taken in case of program
failure/cessation
6. Procedures to be applied according to the test

50. Test Implementation
 Really, this is just running the tests, correction of
tests, running regression tests,
 Testing is done when the outcomes satisfy the
developers.
 When are these tests run?? (time of day/ date??)

51. Regression Testing
 Need not test everything.
 Typically re-test only those artifacts directly changed
and those providing inputs and outputs to these
changed artifacts (modules).
 Very often new errors are introduced when changes
are made.
 There’s always risk in not testing everything… but
these decisions must be made.
 Results of testing are documented in a test report.

53. 1. Test identification, site, schedule and participation
1.1 The tested software identification (name, version and
revision)
1.2 The documents providing the basis for the tests (name and
version for each document)
1.3 Test site
1.4 Initiation and concluding times for each testing session
1.5 Test team members
1.6 Other participants
1.7 Hours invested in performing the tests
2. Test environment
2.1 Hardware and firmware configurations
2.2 Preparations and training prior to testing

54. 3. Test results
3.1 Test identification
3.2 Test case results (for each test case individually)
4. Summary tables for total number of errors, their
distribution and types
4.1 Summary of current tests
4.2 Comparison with previous results (for regression test summaries)
5. Special events and testers' proposals
5.1 Special events and unpredicted responses of the software during testing
5.2 Problems encountered during testing.
5.3 Proposals for changes in the test environment, including test preparations
5.4 Proposals for changes or corrections in test procedures and test case files