1.
Automating Software Testing
Myths, Benefits and Tools
Phil Robinson
LonsdaleSystems.com
LonsdaleSystems.com 1

2.
Mechanical Testing
Test equipment
System under test
LonsdaleSystems.com 2

3.
Electrical Testing
System under test
Test equipment
LonsdaleSystems.com 3

4.
Electrical
Test Automation
• Benefits
– Large volume of tests
– Feasibility of manual tests
– Test throughput
– Accuracy of results
• Costs
– Equipment
– Setup
– Maintenance
LonsdaleSystems.com 4

5.
Software Testing
Test equipment System under test
LonsdaleSystems.com 5

6.
Software Test Automation
• Magic
• Expensive
LonsdaleSystems.com 6

7.
Benefits of Test Automation
• Faster test throughput
• After hours testing
• Better use of testers time
• Improved test accuracy
• Reduced effort in some areas
LonsdaleSystems.com 7

8.
Cost of Test Automation
• Tool selection, acquisition and
implementation costs
• Support and maintenance
• Training costs
• Increased effort in some areas
LonsdaleSystems.com 8

9.
Acquisition Costs
www.opensourcetesting.org
LonsdaleSystems.com 9

10.
Increase/Reduction in Effort
• Labour intensive
to specify tests
• Less effort to
execute tests
• Less effort to
repeat tests • Less effort to
specify tests
• Labour intensive
Automated to execute tests
• Labour intensive
to repeat tests
Manual • Some tests are
impractical
LonsdaleSystems.com 10

11.
Two Compelling Drivers
For Test Automation
The need for
repetitive test
execution
Manual execution
of some tests is
impractical
LonsdaleSystems.com 11

12.
Test Automation
as an Investment Decision
Benefits
Costs
LonsdaleSystems.com 12

13.
Test Automation Risks
• Insufficient time and
resources
• Lack of skills
• Volatile requirements
• Requirements difficult
to test
• Changing technology
LonsdaleSystems.com 13

14.
Test Automation Myths
• Capture-replay
– Automates testing
– Replaces skilled testers
• Testing can be fully automated
• One test tool is suitable for all tasks
• Tools are “easy to use”
• Immediate reduction in effort
LonsdaleSystems.com 14

15.
Test Automation Tools
www.autoitscript.com
LonsdaleSystems.com 15

16.
Test Automation Tools
seleniumhq.org
!!!!
LonsdaleSystems.com 16

17.
Test Automation Tools
fit.c2.com
LonsdaleSystems.com 17

18.
Test Automation Strategies
• Capture replay
• Test scripts
• Data driven
• Keyword driven
LonsdaleSystems.com 18

19.
Capture Replay Demo
LonsdaleSystems.com 19

20.
Capture Replay Issues
• Test data hard-coded in script
• Sensitive to changes in the user interface
• No comparison of expected and actual results
• Re-capture takes as long as the original capture
• Hard to manage many small isolated scripts
LonsdaleSystems.com 20

21.
Test Automation Strategies
• Capture replay
• Test scripts
• Data driven
• Keyword driven
LonsdaleSystems.com 21

22.
Test Script Demo
LonsdaleSystems.com 22

23.
Test Script Issues
• Test data hard-coded in script
• Expected result hard-coded in script
• Hard to manage many small isolated scripts
• Format of the data is hard-coded
• Each test requires a script
• Need a control script to sequence and launch
test scripts
• Logging of test results embedded in every script
• Lack of reuse in scripts
LonsdaleSystems.com 23

24.
Test Automation Strategies
• Capture replay
• Test scripts
• Data driven
• Keyword driven
LonsdaleSystems.com 24

25.
Data Driven Demo
LonsdaleSystems.com 25

26.
Data Driven Issues
• Format of the data is hard-coded
• Each data format requires a script
• Comparison of expected and actual results split
between test data and script
• Need a control script to sequence and launch
test scripts
• Logging of test results embedded in every script
• Dependency between test cases is hard-coded
in the control script
• Lack of reuse in scripts
LonsdaleSystems.com 26

27.
Test Automation Strategies
• Capture replay
• Test scripts
• Data driven
• Keyword driven
LonsdaleSystems.com 27

28.
Keyword Driven Demo
LonsdaleSystems.com 28

29.
Keyword Driven Issues
• Need a control script to sequence and launch
test scripts
• Logging of test results embedded in every script
• Dependency between test cases is hard-coded
in the control script
LonsdaleSystems.com 29

30.
Automating Software Testing
Myths, Benefits and Tools
Questions
LonsdaleSystems.com
conference.lonsdalesystems.com
LonsdaleSystems.com 30