The document discusses various cognitive biases affecting software testing, emphasizing confirmation bias, cognitive dissonance, and emotional impacts on testers. It highlights the importance of awareness in decision-making, collaborative testing methods, and ongoing education and self-learning for personal development. Key recommendations include ensuring multiple testers assess features, acknowledging emotional influences, and actively reducing biases during the testing process.

2. Software Testing
Biases
Oksana Shatabilova
Senior QA Engineer DataArt

3. Content
• Cognitive biases
• Emotions
• Education and training others
• System 2 thinking

4. Cognitive bias
• Princeton University has the following definition for cognitive bias:
“Cognitive bias is a general term that is used to describe many
observer effects in the human mind, some of which can lead to
perceptual distortion, inaccurate judgement, or illogical
interpretation. It is a phenomenon studied in cognitive science
and social psychology.”
4

5. “You should work to reduce your biases, but
to say you have none is a sign that you have
many” - Nate Silver
5

6. Confirmation Bias
“Confirmation bias is a filter through which you see
a reality that matches your expectations. It causes
you to think selectively, but the real trouble begins
when confirmation bias distorts your active pursuit
of facts. - David McRaney”
6

7. Key Points
• Ensure that more than one tester tests the same feature
• “Paired testing” either with dev or tester or user
• Automated testing
• Keep in mind “confirmation bias” during communication activities
7

8. Cognitive dissonance
Cognitive dissonance is
an uncomfortable feeling
caused by holding
conflicting ideas
simultaneously.
8

9. Conjunction effect
Linda is 31 years old, single, outspoken, and very bright. She
majored in philosophy. As a student, she was deeply concerned
with issues of discrimination and social justice, and also participated
in anti-nuclear demonstrations.
Which is more probable?
A.Linda is a bank teller.
B.B. Linda is a bank teller and is active in the feminist movement.
9

10. Test
• Req1: If the user is a software engineer then screen ‘is engineer’
must be shown.
• Req2: If the user is a software engineer and likes to listen to
classical music then screen ‘music engineer’ must be shown.
Let us look at the following user story.
• David went to university to train as a classical violinist. Once
leaving university David retrained as a software engineer and
started to write code for a major software house. Which is
more likely to be true? 10

11. Cognitive framing
11

12. Illusory Correlation
12
Illusory correlation is the
phenomenon of seeing
the relationship one
expects in a set of data
even when no such
relationship exist.

13. Cognitive Illusions
13

14. Cognitive Illusions
14

15. Cognitive Illusions
15

16. Key Point
Beware of your own
beliefs, assumptions and
that of others on your
team.
16

17. Emotions
17

18. The Emotional Tester
• Do we find more bugs when we are in a bad mood? Psychology research
shows we are more negative when in bad mood.
• How many testers when testing listen to music?
• If you are working in an open plan office and everyone around you is
laughing and joking would this make your testing better or worse?
• What if a tester and a developer are having a heated debate about a
feature that has just been tested?
As testers we need to be aware of our emotions and how they can
affect the quality of the testing we are doing.
18

19. Did you make a right decision?
This is crucial when we are making decisions based upon how we feel. We
need to ask ourselves the following before committing to any decision:
• Do we know the subject?
• Are we being influenced by our biases?
• Have we thought critically about why we are feeling this way?
• Is there a personality issue which is causing us to feel the way we do?
• Are there other motivations in play that affects how we are feeling?
Rewards, Bonus, Promotion?
19

20. How often do you take a part in interview?
20

21. Key Point
Being aware of how you feel could be useful to help you realize
what you may have missed when you are testing.
21

22. Education and Training Others
“We now accept the fact that
learning is a lifelong process
of keeping abreast of change.
And the most pressing task is
to teach people how to learn.”
22

23. Training Courses
• “Sit back and listen”
• “Learn by doing”
23

24. Experiential Learning
24

25. Self Learning
• Attention: What subject draws your attention in leading you
to want to learn more?
• Relevance: How is this subject relevant to you and why do
you want to learn more about it?
• Confidence: How sure that you will be able to commit to
learning this subject? Is the value to you of learning this
subject high?
• Satisfaction: Is learning this subject useful to you? What are
your reasons for learning this subject? Entertainment?
Sense of achievement? Praise? 25

26. Why not?
Many people make excuses not to
learn:
• I don’t have the money
• I don’t have the time
• I don’t have the skills
• I don’t need to
• I rely on certifications for all of
my training
26

27. Conferences
27

28. Key Point
The majority of self learning is free with regards to money the
only thing to give up is some time. It is a wise investment to
put some of your own time aside to learn.
28

29. Mentoring and Coaching
“No one learns as much
about a subject as one who is
forced to teach it.”
29

30. System 2 thinking
30

31. Test
“A rat and ball cost $1.10. The
rat costs one dollar more than
the ball. How much does the
ball cost?”
31

32. Answer
32
x+y=1,10
x=y+1
y+1+y=1,10
2y=1,10-1
2y=0,10
Y=0,05

33. Test 2
“How many animals did Moses take into the Ark?”
33

34. Key Point
“I would like to remind people involved in testing that—after an
engaged brain—one of our most useful testing tools is… the
pause….” - Michael Bolton
34

35. 16 Steps to become a critical
thinker
35

36. Q&A
36

37. Thank you for
attention!