What is testing and why it is important? Which kinds of testing are there? Which part of them are developers' responsibility? What is test coverage? Which kinds of coverage are there? Based on Udacity MOOC: "Software Testing. How to Make Software Fail" https://www.udacity.com/courses/cs258

1. Development testing
Yury Kisliak @ EIS Group

2. - What is testing
- Kinds of testing
- Kinds of coverage
- Asserts

3. What is testing?

4. What is testing?

5. What is testing?

6. What is testing?

7. What is testing?

8. What is testing?

9. What is testing?
Good testing consist of selecting a good set of test inputs and designing a good acceptability checks.

10. What is going on when we test?

11. What is going on when we test?
-A lot!

12. What is going on when we test?

13. What is going on when we test?

14. What is going on when we test?

15. What is going on when we test?

20. Why testing?
- Testing shows presence of defects (but cannot prove that
there are no defects).
- Appropriate testing reduces the probability for defects.

21. - What is testing
- Kinds of testing
- Kinds of coverage
- Asserts

22. Types of testing
https://en.wikipedia.org/wiki/Software
_testing#Testing_types

23. Types of testing
https://en.wikipedia.org/wiki/Software
_testing#Testing_types

24. Testing Levels

25. Testing Levels

26. Testing Levels

27. Testing Methods

28. Testing Methods
Static
- Reviews, walkthroughs,
inspections.
- Static program analysis.

29. Testing Methods
Static
- Reviews, walkthroughs,
inspections.
- Static program analysis.
Dynamic
- Executing programmed code with a
given set of test cases

30. Testing Methods
Static
- Reviews, walkthroughs,
inspections.
- Static program analysis.
Dynamic
- Executing programmed code with a
given set of test cases
Black box
- Examining functionality without any
knowledge of internal
implementation.
White box
- Internal perspective of the system,
as well as programming skills, are
used to design test cases.

31. Testing Methods
Static
- Reviews, walkthroughs,
inspections.
- Static program analysis.
Dynamic
- Executing programmed code with a
given set of test cases
Black box
- Examining functionality without any
knowledge of internal
implementation.
White box
- Internal perspective of the system,
as well as programming skills, are
used to design test cases.

32. - What is testing
- Kinds of testing
- Kinds of coverage
- Asserts

33. Coverage? What coverage?

34. Coverage? What coverage?

35. Coverage? What coverage?

36. Coverage? What coverage?

37. Coverage? What coverage?

38. Equivalence partitioning

39. Equivalence partitioning

40. Equivalence partitioning

41. Equivalence partitioning

42. Test Coverage

43. Test Coverage

44. Test Coverage

45. Test Coverage

46. Test Coverage - is proportion of a program, exercised during testing.
+ gives objective score
+ when coverage is <100%, we are given a meaningful task
- not god in finding “errors of omission”
- difficult to interpret scores <100%
- 100% coverage does mean that all bugs are found

47. Coverage metrics

48. Coverage metrics
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)

49. Coverage metrics
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.

50. Coverage metrics
Statement (line) coverage: How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.

51. Coverage metrics
Statement (line) coverage: How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
100%

52. Coverage metrics
Statement (line) coverage: How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
100%

53. Coverage metrics
Statement (line) coverage: How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
100% 50%

54. Coverage metrics
Branch coverage: How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.

55. Coverage metrics
Branch coverage: How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
50%

56. Coverage metrics
Branch coverage: How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
50% 50%

57. Coverage metrics
Branch coverage: How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
50% 50%+ = 100%

58. Coverage metrics
✓ Line coverage
✓ Branch coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Loop coverage:
Execute each loop zero, one and
more than one times.

59. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Modified condition/decision
coverage:
MC/DC = branch coverage
+ conditions takes all possible values
+ each condition independently affects the
outcome of a decision

60. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Modified condition/decision
coverage:
MC/DC = branch coverage
+ conditions takes all possible values
+ each condition independently affects the
outcome of a decision

61. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Modified condition/decision
coverage:
MC/DC = branch coverage
+ conditions takes all possible values
+ each condition independently affects the
outcome of a decision
a, !b, c => true

62. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Modified condition/decision
coverage:
MC/DC = branch coverage
+ conditions takes all possible values
+ each condition independently affects the
outcome of a decision
a, !b, c => true
!a, !b, c => false

63. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Modified condition/decision
coverage:
MC/DC = branch coverage
+ conditions takes all possible values
+ each condition independently affects the
outcome of a decision
a, !b, c => true
!a, !b, c => false
!a, b, c => true

64. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Modified condition/decision
coverage:
MC/DC = branch coverage
+ conditions takes all possible values
+ each condition independently affects the
outcome of a decision
a, !b, c => true
!a, !b, c => false
!a, b, c => true
!a, b, !c => false

65. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Modified condition/decision
coverage:
MC/DC = branch coverage
+ conditions takes all possible values
+ each condition independently affects the
outcome of a decision
a, !b, c => true
!a, !b, c => false
!a, b, c => true
!a, b, !c => false
MC/DC is used in avionics software to ensure adequate testing of the most critical (flight and landing related) parts.

66. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
✓ MC/DC
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Path coverage:

67. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
✓ MC/DC
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Path coverage:

68. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
✓ MC/DC
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Path coverage:

69. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
✓ MC/DC
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
Path coverage:

70. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
✓ MC/DC
✓ Path coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
What else:

71. Coverage metrics
✓ Line coverage
✓ Branch coverage
✓ Loop coverage
✓ MC/DC
✓ Path coverage
How many coverage metrics
are out there?
- A lot!
http://www.badsoftware.com/
coverage.htm
(list of 101 coverage metrics)
How many of them do we
need to care about?
- Very few.
What else:
- boundary value coverage
- synchronization coverage
- … other 94 and more coverages

72. How to use coverage

73. How to use coverage

74. How much testing is enough?

75. Tools

76. Tools

77. - What is testing
- Kinds of testing
- Kinds of coverage
- Asserts

78. Assertions

79. Assertions
- makes code self-checking, leading to
more effective testing
- make code fail early
- document assumptions, preconditions,
postconditions and invariants.
- are not for error handling
- must not have side effects

80. Assertions
- makes code self-checking, leading to
more effective testing
- make code fail early
- document assumptions, preconditions,
postconditions and invariants.
- are not for error handling
- must not have side effects
Use error handling code for conditions you expect to occur.
Use assertions for conditions that should never occur.

81. Assertions
- makes code self-checking, leading to
more effective testing
- make code fail early
- document assumptions, preconditions,
postconditions and invariants.
- are not for error handling
- must not have side effects
Use error handling code for conditions you expect to occur.
Use assertions for conditions that should never occur.

82. The end.
Testing by itself does not improve software quality. Test results are an indicator of
quality, but in and of themselves they don’t improve it. Trying to improve software quality
by increasing the amount of testing is like trying to lose weight by weighing yourself
more often. What you eat before you step on to the scale determines how much you will
weigh, and the software-develompent techniques you use determine how many errors testing
will find. If you want to lose weight, don’t buy a new scale; change your diet. If you want to
improve your software, don’t just test more; develop better.
-Steve McConnell