Test Automation significantly contributes advantages and benefits to software testing success. However, test automation projects, to some extent, have been not succeeded as stakeholders' expectation. This topic aims to suggest solutions for the following problems to prevent from automated testing mistakes.

1. Deadly Traps of Test
Automation
STC JULY 2014
THAO VO

2. A landscape
2
1. Automation idol
2. UI addiction
3. Over misusing on commercial
tools
4. Poor collaboration
5. Lazy maintenance
6. Intensive ROI
Source: http://www.lifehack.org/articles/productivity/how-to-manage-
common-productivity-traps-for-improved-productivity.html

3. 3
Trap #1: Adore Automation
Source: http://www.theguardian.com/media/shortcuts/2014/mar/16/could-robots-be-journalist-of-future

4. Trap #1: Adore Automation
Symptoms
• Depending on automation for all testing
activities
• Absence of code reviews
• Absence of exploratory testing
• Absence of UX testing
• All tests are built by developers
4
Source: http://www.calgolftech.com/1-problem-with-buying-golf-clubs/

5. Trap #1: Adore Automation
5
Treatments
Prevent comparison between
manual and automation
Use automation in conjunction
with manual techniques
Use automation to provide a
baseline – a sanity check
Source: http://cureforhives.org/chronic-
idiopathic-urticaria-treatments

6. 6
Trap #2: Love UI so much

7. A Pyramid
7
GUI
Tests
Acceptance
Tests
Integration Tests
Unit Tests

8. Trap #2: Love UI so much
Symptoms
• Limited technical skills
• Cannot understand the
application or system
architecture
• Huge proportion of tests are
being run via the UI
• No collaboration with
developers
• Difficult to setup test context
• Test results are slow & fragile
8
Source: http://www.bostonsearchgroup.com/blog/ceo-
search-break-through-medical-devices-for-wound-closure/

9. Trap #2: Love UI so much
Treatments
• Limit the investment in UI
automation
• Invest more at lowest possible
level with clear test purposes
• Collaborate with developers
• Make sure automation gives a
fast feedback
9

10. 10
Trap #3: Over misusing on
commercial testing tools
Source: http://www.fastcoexist.com/1679718/your-bad-eating-habits-are-contagious

11. Trap #3: Over misusing on
commercial testing tools
Symptoms
11
A commercial tool forms the basis
a testing strategy
Only certain teams or individuals
can access the tool or run tests
Developers and testers have not
been consulted in tool selections Source: http://www.sfgate.com/news/article/Apple-
exploring-cars-medical-devices-to-reignite-5239850.php

12. Trap #3: Over misusing on
commercial testing tools
Treatments
• Use open source tools wherever
possible
• Use tools that can be extended
or integrated into existing
development tool chain
• Use tools that match specified
test intent
• Use clear matrix tool evaluation
• Keep educating!!!
12
Source: http://stugalactic.co.uk/student-shopping-
medicine.aspx

13. 13
Trap #4: Too pride to
collaborate when creating
tests
Source: http://www.jonathankettleborough.com/tag/collaboration-2/

14. Trap #4: Poor collaboration
Symptoms
• Automated testers are isolated
from team: duplicated test
scripts, too much tests but
inefficiency
• Poor automation design
• Maintainability or compatibility
issues
• No definition of quality: unclear,
not match the desired system
quality 14
Source: http://thaineuro.org/category/pharma-and-
healthcare/healthcare/

15. Trap #4: Poor collaboration
Treatments
• Collaborate to create good tests
and prevent duplication
• Ensure good technical practices
and design
• Collaborate to improve quality
definition and verification
• Limit investment in UI based
automated tests
15
Source: http://medicine1.info/medicine/

16. 16
Trap #5: Too lazy
maintenance
Source: http://www.tastyhuman.com/the-illusion-of-8-hour-workday-are-you-hard-working-or-just-lazy/

17. Trap #5: Too lazy
maintenance
Symptoms
• Test suite has not been run
recently – state is unknown
• Test suite requires manual
intervention
• Duplication within automation
code
• Small changes trigger a cascade
of failures
17
Source: http://orangedocfamilymedicine.com/

18. Trap #5: Too lazy
maintenance
Treatments
• Automated tests should be
executed using a CI
environment
• Ensure tests are always
executable – even if the system
is not being actively developed
• Ensure collaboration between
developers and testers
18
Source: http://www.psmag.com/blogs/the-101/the-
spectrum-of-color-response-take-your-medicine-51195/

19. 19
Trap #6: Cut costs through
automation
Source: http://insuranceknock.com/questions/155/saving-money-with-2-phrases

20. Trap #6: Cut costs through
automation
Symptoms
• Using a predicted ROI to
reduce testing budget
• Hidden investments are not
counted
20
Source: http://bloomintegrativehealth.com/news-updates/

21. Trap #6: Cut costs through
automation
Treatments
• Clear objectives and reasons
for automation
• Cost-benefit analysis should
include ongoing maintenance
• Quality is the key
21
Source: https://today.duke.edu/2013/03/operationmedicinedrop

22. Summary
22

23. 23
Q&A

24. References
• [1]. Testing Experience – published No. #21,
#23 - http://www.testingexperience.com/
• [2]. 7 Deadly Sins of Agile Software Test
Automation – Adrian Smith
http://www.slideshare.net/adrianlsmith/7-
deadly-sins-of-software-test-automation
• [3]. Manual Test cannot be automated
http://www.satisfice.com/blog/archives/58
• [4]. Testing Club Vietnam 2014 Trends in SW
Testing – Paul Holland
24

25. Profile
• Thao Vo – QA Architect at KMS Technology Inc.
(http://www.kms-technology.com/); Board
member of HCMC Software Testing Club
(http://www.hcmc-stc.org/).
• Email: thaovo@kms-technology.com or
thaovo@hcmc-stc.org or vvthao@yahoo.com
• LinkedIn: http://vn.linkedin.com/pub/thao-
van-vo/24/31a/b35
25

26. © 2014 HCMC Software Testing Club
THANK YOU