1) The document discusses common pitfalls of test automation and provides recommendations to avoid them. It identifies pitfalls such as automating everything without prioritization, viewing automation as solely the tester's responsibility, being overly reliant on automation tools, treating test code as less important, and having unrealistic expectations of return on investment. 2) The recommendations are to prioritize what to automate based on risk and value, involve the whole team in automation, select the right tools for the job with critical thinking, follow good coding practices for test code, and take a long term and realistic view of the costs and benefits of automation. 3) The key takeaway is that automation requires investment of time and resources, but can

1. Pitfalls of Test Automation
and why you should keep your head
Rikke Simonsen
Technical Tester
1

2. My Background
Education: Bachelor in Computer Science
Past: Web Developer
Now: Technical Tester
That means:
• Primarily knowledge in programming
• Still learning (how to be a tester) by doing
(a lot of mistakes)
2

3. What I do
Write specifications
with customers
and
implement them
as automated tests
!
3

4. 4

5. Why automate?
• Faster feedback on the possible side effects of
changes to the system
• Faster decision making of release readiness
• Less time on repetitive tasks
• More time for explorative testing
• Documentation of the behaviour of the system
5

6. Pitfalls
1. Automate everything
2. Automation is the testers job
3. Blinded by love (to the tool)
4. Test code treated as second class
5. (Too) High expectations of ROI
6

7. 1. Automate everything
7

8. 8

9. • No prioritizing of what to automate
• Each test introduced will need to be maintained
• Not all tests return value
• Not all test cases can easily be automated
• Some tests are better left for manual testing
Be aware of
1. Automate everything
9

10. • Think about return on investment
• Don’t automate trivial test cases - the ones that
never fail or represent any risk
• Focus on the scenarios that will hurt the
business if the functionality breaks
• Don’t be afraid to remove tests
• Make thoughtful decisions of what to automate
Recommendations
1. Automate everything
10

11. 2. Automation is the
testers job
11

12. 12

13. • Thinking that automated tests are just test.
Its code.
• Thinking that automation is all about learning a
tool. It’s not.
Be aware of
2. Automation is the testers job
13

14. • Automation should be done by people with
good programming skills.
• The tests should be defined by the tester in
collaboration with domain experts.
• Test automation should be a shared
responsibility. Everyone on the team should
contribute.
Recommendations
2. Automation is the testers job
14

15. 3. Blinded by love
(to the tool)
15

16. 16

17. • Thinking the tool can do magic
• Thinking the tool can solve all your problems
• The urge to automate it all.
• Testing everything through the GUI
• Trying to fit the tests to the tool
Be aware of
3. Blinded by love
17

18. • Critical thinking. Analyze strengths and
weaknesses of the tool.
• Learn how to use a wide range of tools and
select the right one for the job.
• Only test things through the GUI that can’t be
tested in any other way.
Recommendations
3. Blinded by love
18

19. 4. Test code treated
as second class
19

20. 20

21. • Automation of use cases directly
• Redundant code
• Test code is just as hard and expensive to
maintain as “real” code
Be aware of
4. Test code treated as second class
21

22. • Make the tests as small as possible
• Don’t let one test depend on another
• Write high level tests
• Make functions reusable
• Document the purpose of the test
• Refactor often. Do peer review.
• Use good programming practices
Recommendations
4. Test code treated as second class
22

23. 5. (Too) High
Expectations
23

24. 24

25. • Expecting early ROI
• Thinking automation is easy
• Thinking automation can “build quality in”
• Expecting cutting down costs
• The maturity of the tools
Be aware of
5. (Too) high expectations
25

26. • Automation is a long time investment -
there’s no quick fixes.
• Measure the costs and results. Evaluate
continuously.
• Train the team in following good practices.
Get external help if starting from scratch.
Recommendations
5. (Too) high expectations
26

27. and last..
• Start small
• Be realistic
• Get everyone involved
• Utilize peoples special skills
• Define your target
• Evolve
27

28. Questions?
rikke@reload.dk
28