The document discusses the concept of testing axioms, which are foundational principles that guide testing approaches but lack universal agreement in the testing community. It explores the importance of defining and refining these axioms to provide a cohesive framework for testing practices, noting that diverse opinions exist among various experts in the field. The author proposes a set of proposed axioms that emphasize the need for critical evaluation, stakeholder involvement, and adaptability in testing methodologies.

1. THEAXIOMSTESTINGOFPaul Gerrard

2. This talk is an experiment118 slides in 35 minutes ;-)Ever seen presentationzen.com?(and I had some fun with Google images)

4. Test Axioms – Do They Exist?What might they look like?But first, some video…

5. Pentagon Wars

6. A comedy, where the military officers lack principles except for…

7. The NewTest Manager

8. The book/film is based on a true story – a farceMy question for you is…

9. Is there a single, agreed set of guiding principles for testing and test managers?

10. More formally…

11. Are there a set of axioms that provide a framework for all test approaches?

12. What is an Axiom?

13. An uncontested principle

14. 2 + 2 = ???Something soself-evidently true it requires no proof

15. A foundation for thinking and behaviour

16. More formally…An axiom is something that is impossible to prove to be true but could be disproved with a single experiment/example.

17. Example AxiomDeclaration of independence para 2:“We hold these Truths to be self-evident, that all Men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty, and the pursuit of Happiness.”

18. Example Axiom 2“The shortest distance between two points is a straight line”

19. We believe Axioms are INFALLIBLE(but we are prepared to be proven wrong)We KNOW that heuristics are FALLIBLE(cf the Context-Driven School)

20. Testers like principles

21. “Testing as the conscience of the project”Testers tend to take principled stances

22. So, it would help if we had a set of principles or axioms to refer to

23. We all have our favourite ‘testing sayings’ but…There is no agreed set of laws so our profession is based on weak foundations.Repeat after me….

24. We need to start with a definition of ‘test’American Heritage Dictionary:Test: (noun)A procedure for critical evaluation;

25. A means of determining the presence, quality, or truth of something;

26. A trial. OK – that’s the intro…Let’s play:“Hunt the Axiom”

27. The testing community has many experts and evangelists

28. Many books on software testing…And there appear to be 713 of them …

29. Chapter 1 PrinciplesChapter 2 Approachblah blah etc…

30. But… there are some differences in opinion

31. Diverse principles promoted by diverse set of ‘experts’

32. Beizer, Black, Craig, Jaskiel, Gerrard, Thompson, Hetzel, Kaner, Bach, Petticord, Falk, Nguyen, Kit, Perry, Pol, Teunissen, van Veenendaaland many more…

33. Each present their own test principles

34. What do we observe here?

35. Variable,credible,some even usefullet’s look closer…

36. Differing backgrounds and experience of ‘experts’

37. When some of them were written

39. Early focus on BUGS

40. Test thinking has evolved

41. Testing as a lifecycle activityModel-based testingTest-obsessedTesting as explorationBusiness goal-based testingAgile testingTest-first, test-drivenRisk-based testingTesting as learning

42. Some ‘principles’ appear again and again

43. Testing is an intellectually difficult activity

44. Complete testing is impossible

45. Independence of mind has value

46. Focusing on bugs is good

47. Testing builds confidence…and so on…

48. But some ‘principles’ are not universally accepted

50. …advocated as the professional approach by some…

52. Dynamic, concurrent test design and execution, using heuristics and an agile mentality is promoted by some…

54. What could theTest Axioms look like?

55. Some suggested criteria for Test Axioms…

56. From any perspective, they are self-evidently true

57. They apply to any test approach

58. Axioms are not guidelines

59. They are context insensitive

60. They are not practicesPractices are always context-sensitive

61. They are easy to understand

62. Corollaries

63. A non-Axiomatic testing approach is ‘unsound'

64. Projects that ignore the axioms might well be eCcEntRicWorse, you might be working in a …

65. Project fromHELL

66. …or the people doing the testing, aren’t

67. Different approaches reflect a different emphasis on the same axioms

68. Some people think that’s wrong

69. “If you believe in axioms, you have a closed mind”“Murder is wrong” is axiomatic.

70. “There are no universal axioms in our field”ALL the Axioms have been compared with the writings of the folk who have objected.

71. “Axioms define a school of testing”I believe Test Axioms exist.If we can get them right ALL schools of testing MUST align with them.

72. Do I want to be a School?Too many schools push dogma, demand obedience to school rules and mark people for life. They put up barriers to entry and exit and require members to sing the same school song. No thanks.

73. Axioms are just a high level abstraction– that’s all

74. Sound test approaches align with the Axioms or…

75. …the Axioms are wrong

76. The Proposed Axioms

77. Axiom NameThe AxiomAxioms are one-liners that succinctly state the principle.

78. Stakeholder AxiomTesting needs stakeholders

79. Test Basis AxiomYou need a source of knowledge to select things to test

80. Test Oracle AxiomTest needs a source of knowledge to evaluate actual behaviour

81. Fallibility AxiomYour sources of knowledge are fallible and incomplete

82. Scope Management AxiomIf you don’t manage scope, you may never meet stakeholder expectations

83. Design AxiomTest design is based on models

84. Coverage AxiomTesting requires a coverage model or models

85. Delivery AxiomThe usefulness of the intelligence produced by test determines the value of testing

86. Environment AxiomTest execution requires a known, controlled environment

87. Event AxiomTesting never goes as planned

88. Prioritisation AxiomThe most important tests are those that uncover the best intelligence, fast

89. Execution Sequencing AxiomRun your most important tests first – you may not have time to run them later

90. Repeat-Test AxiomRepeated tests are inevitable

91. Good-Enough AxiomAcceptance is always a compromise

92. Never-Finished AxiomTesting never finishes; it stops

93. Value AxiomThe value of intelligence is independent of who produces it

94. The Seventeenth AxiomSuggestions?

95. Applications of Test Axioms

96. Think of the Axioms as thinking tools

97. Test Strategy

98. Contexts of Test StrategyAxiomsCommunicationEarly TestingRisksDe-DuplicationTestStrategyOpportunitiesGoalsAutomationCultureContractUser involvementConstraintsHuman resourceArtefactsSkillsEnvironmentProcess(lack of?)Timescales

99. First Equation of TestingAxioms+ Context+ Values+ Thinking Approach

100. ValuesCome from your personal or organisational view of testing.Your ‘School’ of thought perhaps.

101. StakeholderAxiom-Pairsgenerate questionsBasisOracleFallibilityScopeValueCoverageNever-FinishedDeliveryGood-EnoughEnvironmentRepeat-TestEventDesignPrioritisationSequencing

102. StakeholderWhat tests do the different stakeholders value most?What information do the different stakeholders want first?Sequencing

103. Process Assessment and Improvement

104. Each Axiom triggers questions naturally

105. These questions frame a more natural assessmentMuch better than generic ‘maturity models’ that MUST BE context-dependent

106. E.g. Stakeholder questions(aka stakeholder analysis)Who are they?What intelligence do they want?What do they need it for?When do they want it?In what format?How often?Etc.

107. A Framework for Tester Competencies

108. EventCoverageRiskManagementNeverFinishedStakeholderAxioms drive competencies - exampleManaging Uncertainty

109. Facilitation, influencing and reporting

110. Expectation management

111. Crisis managementClosing Comments

112. I believe Test Axioms existWe need to define, refine, and agree them

113. Axioms could be very usefulWill\

114. Axioms don’t constrainIf they exist, they stimulate thinking

115. They represent the set of core values and beliefs about testing(...Anything?)

116. Reminder:our definition of ‘test’A procedure for critical evaluation

117. A means of determining the presence, quality, or truth of something

118. A trial. Did you notice anything about the Axioms?The words software, system, technology, methodology, v-model, entry/exit criteria, risk – do not appear