1. Automated Testing
Jeroen Claes
KU Leuven

2. What?
• Test automation is the use of
special software (separate from the
software being tested) to control the
execution of tests and the
comparison of actual outcomes to
predicted outcomes [1].

3. What?
• Run Automatically
• No human intervention required
• Test result report generated

4. Why?
• Manual testing is time and cost consuming
• Difficult to test for multi lingual sites manually
• No Human intervention required. (can be run
overnight)
• Increases speed of test execution
• Helps increase test Coverage
• Manual Testing can become boring and
hence error prone.

5. What to automate
• High Risk - Business Critical test cases
• Test cases that are executed repeatedly
• Test Cases that are very tedious or
difficult to perform manually
• Test Cases which are time consuming

6. What NOT to automate
• Test Cases that are newly designed and
not executed manually at least once
• Test Cases for which the requirements
are changing frequently
• Test cases which are executed on ad-hoc
basis.

7. Process

8. Test tool selection
• Select the right tool for the right job
• Selected based on best fit, not based on
popularity
• Firstly perform proof of concept

9. Scope
• Important business features
• Scenarios with large amount of data
• Common functionalities across
applications
• Technical feasibility
• Extent of reusability
• Complexity of test cases
• Ability to use cross browser testing

10. Planning, Design and Development
• Automation tools selected
• Framework design and its features
• In-Scope and Out-of-scope items of
automation
• Automation test bed preparation
• Schedule and Timeline of scripting and
execution
• Deliverables of automation testing

11. Test Execution
• Input data provided
• Test reports generated
• Execution directly or via test
management tool

12. Maintenance
• New functionality with succesive cycles
• Scripts are added/reviewed/maintained
for each release cycle
• Required to improve effectivenes of
automation scripts

13. Automated Testing in an Agile
devironment

14. Start small
• Develloped incrementally
• Tests are added with the new
implementation in each sprint
• Dont aim for the “it-can-do-everything”
solution at the start
• Start with a bare minimum working
solution

15. Create test automation backlog
• Backlog contains all needed automation
tasks
• Target a few items each sprint
• Use priorities for automation tasks
• Use dedicated developer time when
needed

16. pitfalls
• Automation does not replaceme manual testing
• Licensed test programs can be brittle and produce lots
of overhead
• UI testing should be reserved for testing the UI
• Collaboration required between designer and tester
• Tests must be properly maintained
• Slow, brittle or unreliable tests need to be resolved
• Return-on-investment should not be purely based on a
reduction in labour costs

18. References
• [1] Kolawa, Adam; Huizinga, Dorota
(2007). Automated Defect Prevention: Best
Practices in Software Management. Wiley-IEEE
Computer Society Press. p. 74.ISBN 0-470-04212-
5
• http://www.guru99.com/automation-
testing.html
• http://www.infoq.com/articles/thoughts-on-test-
automation-in-agile
• http://www.agileengineeringdesign.com/2012/0
1/7-deadly-sins-of-automated-software-testing/