An exploration of my testing skills - presentation for CAST 2014

1. Painting like an engineer
Skills in testing
Alexandra Casapu
@coveredincloth

6. Heuristic!
Heuristic!
Heuristic!
Heuristic!
Heuristic!
Heuristic!

7. The engineering method:
Use heuristics…
..to cause the best change..
..in a poorly understood situation..
..within the available resources

9. The art and science of testing…

10. Engineering vs Science

11. What is science?
Applied engineering

12. Engineering vs Art

13. “At last the mountains of Arcadia and the mist-wrapped mountains of the
lonely traveler become one as we realize that what we paint, how we paint,
that we paint, and, most importantly, why we paint are all art - all; heuristic.
To be alive is to be an artist; to be conscious of being an artist is to be
human; to recognize and to be satisfied that you can only be an artist is to
be at peace.“

14. Story structure
How I see skills and skills development
Where can the engineering method help
5 examples from my testing work
Conclusions in terms of skills development

15. Where are the skills?
The Knowledge
Dimension
The Cognitive Process Dimension
Remember Understand Apply Analyze Evaluate Create
Facts
Concepts
Procedures
Cognitive Strategies
Models
Skills
Attitudes
Metacognition

16. How do I know I have acquired a skill?
Second nature

17. How can I work on my skills?
Cem Kaner :
I think the best way for people to develop skills is to do something,
get feedback on how to do it better, improve it (or do something similar),
get feedback, and keep doing this with problems that are increasingly
difficult or that apply the technique in new ways.

18. The necessary conditions for developing skills
• Predictability of outcomes
• Good feedback
• Attitude
• Motivation
• Deliberate practice
• Amount of experience

19. “Most judgments and most choices are made intuitively.”
Maps of bounded rationality: Psychology for behavioral economics –
Daniel Kahneman

20. The context of my work

21. A taxonomy of skills
Signatures of the heuristic:
1. A heuristic does not guarantee a solution,
2. It may contradict other heuristics,
3. It reduces the search time for solving a problem, and
4. Its acceptance depends on the immediate context instead of an absolute standard

22. My personal testing skills state of the art
Attitude determining skills
• learn from mistakes
• generate functionality flows
• analyze information
• recognize patterns
• read actively
• create models
• use models
• read and write code
• write effective bug reports
• make logical connections
• change how testing is done and
seen in the team
• collect relevant information from
different sources
• handle complexity well
• evaluate the testing work
Human-Human Interaction skills
• asking questions(in a simple, non-offensive
manner)
• have an empathetic approach
• be appreciative, considerate
• have the ability to explain my train of thoughts
• collect info effectively
• offer my availability
Risk controlling skills
• think critically
• use counterfactual reasoning
• deal with uncertainty and incompleteness
• prioritize
• work in a timeboxed manner
• decide when to stop
Rule of thumb skills
• focus
• defocus
• break a problem into multiple smaller problems
• have diverse view points
• stick with a consistent view point
• use curiosity
• fix variables and vary one at a time
• exercise stubbornness
• don't use stubbornness
• create disconfirmatory experiments
Information visualization skills
• describe coverage
• make maps of features under test
• create valuable documentation

23. Examples
Working on new skills (repeatedly)
Skills atrophy
Skills that work…sometimes
Overlooked skills
Stubborn about using whatever skills needed
1
2
3
4
5

24. 1 Working on new skills (repeatedly)

25. 1
Let’s introduce some automation..

26. 1
On the previous project
Focusing on tasks
Less defocusing
I didn’t use my curiosity
Not so effective at collecting info
Not thinking critically about my testing process

27. 1
On the current project
Defocus from my current task
Work in a timeboxed manner
Use my curiosity
Write code
Read code
Change how I do testing
Change how testing is seen in the team
Create valuable documentation

28. 1
The ‘repeatedly’ part

29. 2 Skills atrophy

30. 2
On the previous project
Working with APIs
Verifying validity output in the DB
Monitoring http requests
Verifying output in json files
Testing social crawlers algorithms

31. 3 Skills that work…sometimes

32. 3
Not getting invited to meetings

33. 3
Getting invited to meetings
Change how testing is done and seen in the team
Collect info effectively
Asking questions
Offer my availability

34. 4 Overlooked skills

35. 4
Logging invalid bugs

36. 4
Logging invalid bugs

37. 4
Logging invalid bugs
Re-evaluate the model
Analyze existing data

38. 5 Stubborn about using whatever skills needed

39. 5
Learn by digging the needed info
Use curiosity
Collect info effectively
Read code
Break a problem into multiple smaller problems
Exercise stubbornness
Have diverse viewpoints
Use counterfactual reasoning

40. Conclusions

44. It’s about skills interaction.
An overwhelming accumulation of interactions

45. Learn by doing

47. Skills are a procrustean bed
“Each speaks with the accent of his[/her] own sota.”

48. Each with their own experience
“Since one sota cannot be completely translated into another, each person
remains fundamentally isolated. To this sense of uniqueness, inaccessibility,
alienation, and loneliness some philosophers attribute the absurd nature of the
human condition.”

49. Nurturing skills
Choose a few areas which I want to improve
Learn the queues that lead to the mistakes I want to eliminate
When I recognize such a situation, slow myself down
And recognize when I need help

50. Nurturing skills
“Peace comes with the realization that the world in which we live is an
acquired taste - one we all, as artists, paint in our own chosen styles.”

51. References
• Discussion of the method - Billy Vaughn Koen
• How to read a book – Mortimer J. Adler, Charles van Doren
• Maps of Bounded Rationality: Psychology for Behavioral Economics – Daniel Kahneman
• Conditions for intuitive expertise: a failure to disagree – D. Kahneman, G. Klein
• http://blog.utest.com/testing-the-limits-with-cem-kaner-author-of-the-domain-testing-workbook/
2014/01/
• The BBST courses
• edXThinking 101 course
Illustrations:
• http://catandgirl.com/ (slide 2)
• http://studio-ghibli.wikia.com/wiki/Forest_(Mononoke-Hime) (slide 3)
• http://theredlist.com/media/database/muses/icon/cinematic_men/1980/john_malkovitch/022_john_malkovitch_theredlist.jpg (slide 6)
• http://thingsorganizedneatly.tumblr.com/post/70411190398/submission-torpedo-typewriter-organized-neatly (slide 34)
• http://thingsorganizedneatly.tumblr.com/image/67872166549 (slide 35)
• http://thingsorganizedneatly.tumblr.com/image/81360232303 (slide 36)
• http://4.bp.blogspot.com/-ZtDBcj1VW1A/TlaUP0xdFhI/AAAAAAAACBs/2DhJ_5UVvaM/s1600/Theseus_Procrustes-600x450.jpg (slide 39)

52. @coveredincloth