The document discusses the scientific method approach to software testing. It explains that scientific testing involves forming hypotheses, designing experiments to test those hypotheses, and observing and interpreting the results. The document then provides an example of using this approach to test a toy called Big Trak Jnr to discover what the X2 button does. It emphasizes that good testing requires asking questions, thinking critically about the testing process, and guarding against confirmation bias during experimentation and evaluation.

1. The Science
of
Software Testing
@charrett

2. What is
Software Testing?

4. Scientific Method
Formulate hypothesis
design, run & interpret experiments
that bear on this hypothesis

5. • CM : Clear Memory
• Forward: 1 or 2 digit number e.g 10
• Back : 1 or 2 digit number
• Rotate Left: 1 or 2 digit number e.g 15
• Rotate Right 1 or 2 digit number e.g 15
• Fire: 1 or 2 digit number
• Go: Executes the program
Commands

6. Big Trak Jnr
Discover what the X2 Button does

7. Challenge
Formulate your hypothesis on what the X2 button
does
Design a program, execute it and observe the devices
behaviour,
How does the outcome relate to the hypothesis?
Terminate the process when you have discovered what
the X2 button does
Afterwards we will debrief and discuss

8. You can ask me any
questions

9. Verbalise your ideas

10. What does x2 do?
• x2 is also known as the repeat key
• x2 requires a numeric parameter (n).
• x2 can only be used once in a program
• x2 has no effect on instructions following its location in
the program

11. What does X2 do?
N repeats the entire program

12. Possible Hypothesis
• One repeat of the last n steps
• N repeats of the last step
• One repeat of the first n steps
• One repeat of the nth step
• One repeat of the entire program
• One repeat of the last step

13. Observations

14. Exploring Science
• Tests can be easy to construct and easily observable, but
don’t offer much information.
• Knowing where to look & Understand what is seen
• Experimental design: design of experimental and
observational procedures
• Hypothesis formation: formation and evaluation of theory
• Confirmation Bias:“Tendency for people to seek evidence
to verify their hypothesis rather than refute them”

15. Science & Testing
• Testing requires we ask useful questions, challenge
assumptions
• Testing is more about acquiring information than pass/fail
• Testing is about thinking critically about why we test, how
we test and how we evaluate.
• Testing like experiments requires good test design
• Testing like experimentation requires good observational
design
• Watch out for Confirmation Bias
• Don’t conflate Testing with Checking

16. Suggestions
• Don’t plan all your tests upfront, you need to be able to
react to the information you find
• Automated Checking good for confirmation
• Exploratory Testing allows us to investigate the product
• Focus on Testability rather than large automated
regression suites
• Testability in creating test data
• Helping to recognise problems

17. Thank you
&
Tyro Payments
for
sponsoring me