Is it possible to define a set of axioms that provide a framework for software testing that all the variations of test approach currently being advocated align with or obey? In this respect, an axiom would be an uncontested principle; something self-evidently and so obviously true and not requiring proof. What would such test axioms look like? This paper summarises some preliminary work on defining a set of Test Axioms. Some applications of the axioms that would appear useful are suggested for future development. It is also suggested the work of practitioners and researchers is on very shaky ground unless we refine and agree these Axioms. This is a work in progress.

1. THE AXIOMS TESTING OF Paul Gerrard

2. This talk is an experiment118 slides in 35 minutes ;-)Ever seen presentationzen.com?(and I had some fun with Google images)

4. Test Axioms – Do They Exist?What might they look like?But first, some video…

5. Pentagon Wars

6. A comedy, where the military officers lack principles except for…

7. The NewTest Manager

8. The book/film is based on a true story – a farceMy question for you is…

9. Is there a single, agreed set of guiding principles for testing and test managers?

10. More formally…

11. Are there a set of axioms that provide a framework for all test approaches?

12. What is an Axiom?

13. An uncontested principle

14. 2 + 2 = ???Something soself-evidently true it requires no proof

15. A foundation for thinking and behaviour

16. More formally… An axiom is something that is impossible to prove to be true but could be disproved with a single experiment/example.

17. Example AxiomDeclaration of independence para 2: “We hold these Truths to be self-evident, that all Men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty, and the pursuit of Happiness.”

18. Example Axiom 2 “The shortest distance between two points is a straight line”

19. We believe Axioms are INFALLIBLE(but we are prepared to be proven wrong) We KNOW that heuristics are FALLIBLE (cf the Context-Driven School)

20. Testers like principles

21. “Testing as the conscience of the project”Testers tend to take principled stances

22. So, it would help if we had a set of principles or axioms to refer to

23. We all have our favourite ‘testing sayings’ but… There is no agreed set of laws so our profession is based on weak foundations. Repeat after me….

25. A means of determining the presence, quality, or truth of something;

27. The testing community has many experts and evangelists

28. Many books on software testing… And there appear to be 713 of them …

29. Chapter 1 PrinciplesChapter 2 Approachblah blah etc…

30. But… there are some differences in opinion

31. Diverse principles promoted by diverse set of ‘experts’

32. Beizer, Black, Craig, Jaskiel, Gerrard, Thompson, Hetzel, Kaner, Bach, Petticord, Falk, Nguyen, Kit, Perry, Pol, Teunissen, van Veenendaaland many more…

33. Each present their own test principles

34. What do we observe here?

35. Variable,credible,some even usefullet’s look closer…

36. Differing backgrounds and experience of ‘experts’

37. When some of them were written

39. Early focus on BUGS

40. Test thinking has evolved

41. Testing as a lifecycle activity Model-based testing Test-obsessed Testing as exploration Business goal-based testing Agile testing Test-first, test-driven Risk-based testing Testing as learning

42. Some ‘principles’ appear again and again

43. Testing is an intellectually difficult activity

44. Complete testing is impossible

45. Independence of mind has value

46. Focusing on bugs is good

47. Testing builds confidence…and so on…

48. But some ‘principles’ are not universally accepted

50. …advocated as the professional approach by some…

52. Dynamic, concurrent test design and execution, using heuristics and an agile mentality is promoted by some…

54. What could theTest Axioms look like?

55. Some suggested criteria for Test Axioms…

56. From any perspective, they are self-evidently true

57. They apply to any test approach

58. Axioms are not guidelines

59. They are context insensitive

60. They are not practices Practices are always context-sensitive

61. They are easy to understand

62. Corollaries

63. A non-Axiomatic testing approach is ‘unsound'

64. Projects that ignore the axioms might well be eCcEntRicWorse, you might be working in a …

65. Project fromHELL

66. …or the people doing the testing, aren’t

67. Different approaches reflect a different emphasis on the same axioms

68. Some people think that’s wrong

69. “If you believe in axioms, you have a closed mind” “Murder is wrong” is axiomatic.

70. “There are no universal axioms in our field” ALL the Axioms have been compared with the writings of the folk who have objected.

71. “Axioms define a school of testing” I believe Test Axioms exist. If we can get them right ALL schools of testing MUST align with them.

72. Do I want to be a School? Too many schools push dogma, demand obedience to school rules and mark people for life. They put up barriers to entry and exit and require members to sing the same school song. No thanks.

73. Axioms are just a high level abstraction– that’s all

74. Sound test approaches align with the Axioms or…

75. …the Axioms are wrong

76. The Proposed Axioms

77. Axiom Name The Axiom Axioms are one-liners that succinctly state the principle.

78. Stakeholder Axiom Testing needs stakeholders

79. Test Basis Axiom You need a source of knowledge to select things to test

80. Test Oracle Axiom Test needs a source of knowledge to evaluate actual behaviour

81. Fallibility Axiom Your sources of knowledge are fallible and incomplete

82. Scope Management Axiom If you don’t manage scope, you may never meet stakeholder expectations

83. Design Axiom Test design is based on models

84. Coverage Axiom Testing requires a coverage model or models

85. Delivery Axiom The usefulness of the intelligence produced by test determines the value of testing

86. Environment Axiom Test execution requires a known, controlled environment

87. Event Axiom Testing never goes as planned

88. Prioritisation Axiom The most important tests are those that uncover the best intelligence, fast

89. Execution Sequencing Axiom Run your most important tests first – you may not have time to run them later

90. Repeat-Test Axiom Repeated tests are inevitable

91. Good-Enough Axiom Acceptance is always a compromise

92. Never-Finished Axiom Testing never finishes; it stops

93. Value Axiom The value of intelligence is independent of who produces it

94. The Seventeenth Axiom Suggestions?

95. Applications of Test Axioms

96. Think of the Axioms as thinking tools

97. Test Strategy

98. Contexts of Test Strategy Axioms Communication Early Testing Risks De-Duplication Test Strategy Opportunities Goals Automation Culture Contract User involvement Constraints Human resource Artefacts Skills Environment Process(lack of?) Timescales

99. First Equation of Testing Axioms+ Context+ Values+ Thinking Approach

100. Values Come from your personal or organisational view of testing. Your ‘School’ of thought perhaps.

101. Stakeholder Axiom-Pairsgenerate questions Basis Oracle Fallibility Scope Value Coverage Never-Finished Delivery Good-Enough Environment Repeat-Test Event Design Prioritisation Sequencing

102. Stakeholder What tests do the different stakeholders value most? What information do the different stakeholders want first? Sequencing

103. Process Assessment and Improvement

104. Each Axiom triggers questions naturally

105. These questions frame a more natural assessment Much better than generic ‘maturity models’ that MUST BE context-dependent

106. E.g. Stakeholder questions(aka stakeholder analysis) Who are they? What intelligence do they want? What do they need it for? When do they want it? In what format? How often? Etc.

107. A Framework for Tester Competencies

109. Facilitation, influencing and reporting

110. Expectation management

112. I believe Test Axioms existWe need to define, refine, and agree them

113. Axioms could be very useful Will br />

114. Axioms don’t constrainIf they exist, they stimulate thinking

115. They represent the set of core values and beliefs about testing (...Anything?)

117. A means of determining the presence, quality, or truth of something

119. Axioms are generic – they HAVE to be Testers test software, systems, interfaces, documents, processes, buildings, furniture, people etc… (Should they apply to testing anything? Discuss!)

120. The Axioms need more work Refinement Examples Applications

121. Read the Pocketbook and give me feedback – please  gerrardconsulting.com stores.lulu.com/testers-press

122. THE AXIOMS TESTING OF Thank-You!