The document discusses how to make automation an asset to software testing organizations by outlining the advantages and disadvantages of manual versus automated testing, providing examples of what types of tests are best suited for automation, and describing best practices for developing an effective test automation process and addressing common myths about automation. It emphasizes that automation can increase testing efficiency and coverage but requires proper planning, resources, and maintenance to be successful.

1. How to Make Automation An
Asset to the Organization
Vipin Jain
Metacube Software, Jaipur, India
QA&TEST 2012
11th International Conference on
Software QA and Testing
October 17-19, 2012 • Bilbao Spain

2. Introduction
• Software Testing is a process that consists of all test life cycle
activities like static and dynamic testing concerned with
planning, preparation and evaluation of software products to
determine that the software products satisfy customers requirements
and are fit for customers use.
• Software Testing is done to find software defects or failures in
advance.
• Software testing can also be stated as the process of validating and
verifying that a software program/application/product:
– Meets the business and technical requirements that guided its
design and development.
– Works as expected

3. Manual Testing
• Manual testing is the process of manually testing software for
defects. It is a laborious activity that requires the tester to possess a
certain set of qualities; to be patient, observant, speculative,
creative, innovative, open-minded, resourceful, and skillful.
Advantage of Manual Testing
• Running the test case is less cost than automation.
• It allows the tester to perform more Ad-hoc testing (1) (random
testing).
• More time for testing enables a tester to find more bugs.
Disadvantage of Manual Testing
• Running tests manually can be very time consuming
• Each time there is a new build, the tester must re-run all required
tests - which after a while would become very dull and tiresome.

4. Automation Testing
• Software test automation refers to the activities and efforts that
intend to automate engineering tasks and operations in a software
test process using well-defined strategies and systematic solutions.
• The major objectives of software test automation is to free engineers
from tedious and redundant manual testing Operations.
• To speed up a software testing process, and to reduce software
testing cost and time during a software life cycle
• To increase the quality and effectiveness of a software test process
by achieving pre-defined adequate test criteria in a limited schedule
• The major key to the success of software automation is to use a
systematic solution to achieve a better testing coverage.

5. Why Automation?
• Over the last decade, test automation has become a crucial part in
the Test planning activities for various QA groups.
• A well written automated test suite is of enormous help in daily
testing activities, especially in today’s agile world.
• With the Advent of Mobile technologies, a new dimension has been
added in Software testing. There are millions of mobile applications
flooding markets each day. They need to be tested effectively with
shorter QA cycles. But, there are a billion combinations of hardware
devices, OS’s, carriers, and networks today and traditional manual
testing cannot cover all these scenarios – especially when the app
to market life cycle has to be short. Automation is the need of the
hour.

6. Advantages of Automation
Reliable1 Repeatable2 Programmable3
Comprehensive4 Reusable5 Fast6
Better Quality
Cost Reduction8
Software7

7. Why it fails and what factors
contribute to its failure?

8. Reasons for Failure
• Lack of Vision - There is no clear vision behind what the automation
will do, what we intend to achieve with it and what are the strategies
to do this.
• Time - No time to develop and maintain the automation scripts
• Cost associated - We need the tools for it and we want management
to assign the appropriate budget, but they are not interested.
• Skills needed – The tool is new and appropriate training and time to
master the tool are required.
• Lack of Automation Matrix - There are no clear factors listed against
which we measure our automation results to label it as success or
failure

9. What to Automate?
• Well-developed and well-understood use cases or scenarios first
• Relatively stable areas of the application over volatile ones must be
automated.
• Repetitive tests that run for multiple builds.
• Tests that tend to cause human error.
• Tests that require multiple data sets.
• Frequently used functionality that introduces high risk conditions.
• Tests that are impossible to perform manually.
• Tests that run on several different hardware or software platforms
and configurations.
• Tests that take a lot of effort and time when manual testing.

10. When should we do Automation?
• To get the most out of your automated testing, testing should be
started as early as possible and ran as often as needed. The earlier
testers get involved in the life cycle of the project the better, and the
more you test, the more bugs you find.
• Automated unit testing can be implemented on day one and then
you can gradually build your automated test suite.
• Bugs detected early are a lot cheaper to fix than those discovered
later in production or deployment.

11. A Software Test Automation Process
Plan Software
Test Automation
Design Test Automation Select and Evaluate Available
Strategies & Solutions Software Testing Tools
Develop & Implement Test
Automation Solutions
Introduce and Deploy Test
Automation Solutions
Review and Evaluate
Software Test Automation

12. Automation Myths/Realities
Test Plans covering all resource No commercially available tool that
requirements, time needed and can create a comprehensive test
strategy can be auto-generated. plan.
Any application can be tested using No single test tool exists that can
the tool. be used to support all operating
system environments.
It won’t take much time for testing This will take a lot of time in
once automation is done. running and maintaining and re-
running.
An automation test tool is always An automated tool requires new
easy to learn and use. skills; therefore, additional training
is required.
100% test coverage can be Test coverage breadth and depth
achieved. can be increased but 100%
exhaustive testing cannot be done.