There are various definitions given for software testing by various authors. This definition has been taken from Wikipedia.
(1) More bugs are found via Ad-hoc testing than via automation
1. Reliable: Tests perform precisely the same operations each time they are run, thereby eliminating human error.Reduce manual software testing operations and eliminate redundant testing efforts.2. Repeatable: You can test how the software reacts under repeated execution of the same operations. 3.Programmable: You can program sophisticated tests that bring out hidden information from the application. 4. Comprehensive: You can build a suite of tests that covers every feature in your application. 5. Reusable: You can reuse tests on different versions of an application, even if the user interface changes. 6. Better Quality Software: Because you can run more tests in less time with fewer resources 7. Fast: Automated Tools run tests significantly faster than human users. 8. Cost Reduction: Automation can help to detect defects early in the QA cycle, saving a lot of cost and effort early on. As the number of resources for regression test are reduced cost reduces.
How to Make Automation An Asset to the Organization Vipin JainMetacube Software, Jaipur, India QA&TEST 2012 11th International Conference on Software QA and Testing October 17-19, 2012 • Bilbao Spain
Introduction• Software Testing is a process that consists of all test life cycle activities like static and dynamic testing concerned with planning, preparation and evaluation of software products to determine that the software products satisfy customers requirements and are fit for customers use.• Software Testing is done to find software defects or failures in advance.• Software testing can also be stated as the process of validating and verifying that a software program/application/product: – Meets the business and technical requirements that guided its design and development. – Works as expected
Manual Testing• Manual testing is the process of manually testing software for defects. It is a laborious activity that requires the tester to possess a certain set of qualities; to be patient, observant, speculative, creative, innovative, open-minded, resourceful, and skillful. Advantage of Manual Testing• Running the test case is less cost than automation.• It allows the tester to perform more Ad-hoc testing (1) (random testing).• More time for testing enables a tester to find more bugs. Disadvantage of Manual Testing• Running tests manually can be very time consuming• Each time there is a new build, the tester must re-run all required tests - which after a while would become very dull and tiresome.
Automation Testing• Software test automation refers to the activities and efforts that intend to automate engineering tasks and operations in a software test process using well-defined strategies and systematic solutions.• The major objectives of software test automation is to free engineers from tedious and redundant manual testing Operations.• To speed up a software testing process, and to reduce software testing cost and time during a software life cycle• To increase the quality and effectiveness of a software test process by achieving pre-defined adequate test criteria in a limited schedule• The major key to the success of software automation is to use a systematic solution to achieve a better testing coverage.
Why Automation?• Over the last decade, test automation has become a crucial part in the Test planning activities for various QA groups.• A well written automated test suite is of enormous help in daily testing activities, especially in today’s agile world.• With the Advent of Mobile technologies, a new dimension has been added in Software testing. There are millions of mobile applications flooding markets each day. They need to be tested effectively with shorter QA cycles. But, there are a billion combinations of hardware devices, OS’s, carriers, and networks today and traditional manual testing cannot cover all these scenarios – especially when the app to market life cycle has to be short. Automation is the need of the hour.
Why it fails and what factors contribute to its failure?
Reasons for Failure• Lack of Vision - There is no clear vision behind what the automation will do, what we intend to achieve with it and what are the strategies to do this.• Time - No time to develop and maintain the automation scripts• Cost associated - We need the tools for it and we want management to assign the appropriate budget, but they are not interested.• Skills needed – The tool is new and appropriate training and time to master the tool are required.• Lack of Automation Matrix - There are no clear factors listed against which we measure our automation results to label it as success or failure
What to Automate?• Well-developed and well-understood use cases or scenarios first• Relatively stable areas of the application over volatile ones must be automated.• Repetitive tests that run for multiple builds.• Tests that tend to cause human error.• Tests that require multiple data sets.• Frequently used functionality that introduces high risk conditions.• Tests that are impossible to perform manually.• Tests that run on several different hardware or software platforms and configurations.• Tests that take a lot of effort and time when manual testing.
When should we do Automation?• To get the most out of your automated testing, testing should be started as early as possible and ran as often as needed. The earlier testers get involved in the life cycle of the project the better, and the more you test, the more bugs you find.• Automated unit testing can be implemented on day one and then you can gradually build your automated test suite.• Bugs detected early are a lot cheaper to fix than those discovered later in production or deployment.
A Software Test Automation ProcessPlan SoftwareTest Automation Design Test Automation Select and Evaluate Available Strategies & Solutions Software Testing Tools Develop & Implement Test Automation Solutions Introduce and Deploy Test Automation Solutions Review and Evaluate Software Test Automation
Automation Myths/RealitiesTest Plans covering all resource No commercially available tool thatrequirements, time needed and can create a comprehensive teststrategy can be auto-generated. plan.Any application can be tested using No single test tool exists that canthe tool. be used to support all operating system environments.It won’t take much time for testing This will take a lot of time inonce automation is done. running and maintaining and re- running.An automation test tool is always An automated tool requires neweasy to learn and use. skills; therefore, additional training is required.100% test coverage can be Test coverage breadth and depthachieved. can be increased but 100% exhaustive testing cannot be done.