Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Test Automation Framework Design |


Published on

A test automation framework defines an organization's way of doing things. It is a set of assumptions, concepts and tools that provide support for automated software testing.

Published in: Technology, Education

Test Automation Framework Design |

  1. 1. idexcel Designing a Robust Test Automation Framework White Paper
  2. 2. idexcel Automation Framework Design Introduction In today's business environment, software project teams are expected to do more and deliver higher quality systems in lesser time, with fewer resources. To increase speed, accuracy and completeness of testing, testers rely on either commercial or open source test automation tools. Any automated testing effort based on using only a capture tool to record and play back test cases has many drawbacks, as they are created ad hoc, their functionality can be difficult to track and reproduce, and they are costly to maintain. Running complex and powerful tests is time consuming and expensive when using only a capture tool. To make automated testing more efficient, a better option would be to build a test automation framework that supports scripting and extends the capabilities of the automation tool. A test automation framework defines an organization's way of doing things. It is a set of assumptions, concepts and tools that provide support for automated software testing. A test automation framework makes test automation more efficient, provides a higher ROI, reduces the test automation effort and lowers the cost of maintenance. A framework may be built to support either commercial or open source tools available in the market. Different tools provide different inbuilt features, and a framework must focus on extending the capabilities and features not provided by the tool. A test automation framework allows you to write a series of tests without worrying about the constraints or limitations of the underlying test tools. The most critical aspect of building a good test automation framework is its design. This Whitepaper discusses the best practices in designing a robust test automation framework. 2 Page
  3. 3. idexcel Automation Framework Design Objectives of a Test Automation Framework Benefits of a Well Designed Test Automation Framework It is very important to define the objectives of a Test Automation Framework before starting its design and development. An organization may choose some or all of the objectives listed below, depending on its business needs. The design of the test automation framework should be based on its objectives defined by the organization. A well designed framework increases the speed and accuracy of the testing process, enables a higher return on investment (ROI), reduces maintenance and systematically minimizes risks. A well designed framework: Platform & OS independence Support different testing types (UI, database, Web Service). Data driven capabilities (input data, output data) Framework should be driven through tables / key words Ability to parameterize (recorded) scripts End to end testing capability to cover business scenarios View and edit test cases intuitively Ability to implement dynamic conditions & dynamic loop support Test framework should be loosely / tightly coupled with application-under-test (AUT) depending on business needs Enhances efficiency in development of automated test scripts through modularized, reusable, and maintainable code and data Provides a structured development methodology to ensure uniformity of design across multiple test scripts to reduce dependency on individual test-case developers Provides reliable issue detection and efficient root-cause analysis with minimum human intervention for the system under test Reduces dependence on subject matter experts by automatically selecting the test to execute according to test scenarios and dynamically refining the test scope according to changes in the test strategy or conditions of the system under test Is tool agnostic Auto update of test results in a test management tool Performs multiple levels and types of testing – functional, regression, load, performance, unit, integration, etc Less dependence on highly skilled automation professionals Eliminates deployment to multiple machines for testing different versions of an application Ability to schedule and run prioritized test cases Capture screenshots for failed steps and log defects in a defect management tool Runs tests in parallel, runs on test machines in the grid or cloud, tests across multiple environments and application versions simultaneously Test reporting objectives including email notifications. Runs in a continuous integration environment. 3 Page
  4. 4. idexcel Automation Framework Design High Level Framework Design Guidelines Test Automation Framework Design Considerations As in SDLC to develop software applications, framework design plays a vital role in building a test automation framework for an organization. Enterprise applications are large, complex, and ever-evolving, and to keep pace with the organization’s growth, the design of the test automation framework should conform to the guidelines listed below. The design phase starts after the test automation framework requirements are defined and base-lined. A good test automation framework should be flexible, application agnostic, technology agnostic and future-proof. The framework must be application independent and must have the capability to expand, with the requirements of each application. The framework should be based on open technologies, such as Java, to limit the development and maintenance costs. Openness might also allow cross-tool execution when a suitable API exits or can be developed Should be application-independent and easy to enhance and maintain The framework and its assets should allow end-to-end testing of the application suite Encapsulate testers from complexities of the test framework Identify and abstract common functions used across multiple test scripts The framework should allow easy integration with other tools and components Decouple complex business function testing from navigation, limit-testing, and other simple verification and validation activities. Decouple test data from the test scripts Structure scripts with minimal dependencies - ensuring execution of scripts unattended even on failures. The various factors to be considered in designing the framework are discussed in detail below. Selection of the type of Framework : The type of automation framework helps to organize test suites and in turn improves testing efficiency. Different types of framework focus on different objectives. The “Data driven framework” focuses on data management, the “Keyword driven framework” focuses on creation of test cases using keywords, while the “Modular framework” reduces redundancy. The framework type should be chosen based on the organization’s requirements and objectives. If the requirement is to manage data and execute test cases for multiple sets of data, the “data driven framework” would be appropriate. Similarly, if the framework objective is to divide the project into modules and create scripts independently for each module, the “Modular” framework would be appropriate. Sometimes, a “Hybrid framework”, combining the strengths of different types of framework would work well. Data Management: Data plays an important role in the framework design. The best approach is to ensure that there is no hard coding of data. The user should be allowed to manipulate and input the data for test cases (input and expected data). The framework contains different data such as test scripts creation data, test input and expected data, test results and should manage the flow between 4 Page
  5. 5. idexcel Automation Framework Design them easily. The framework should support user friendly interface files (excel, CSV, database, etc). Choosing a database may become a constraint if the organization uses different databases, while CSV is not easy to use, leaving excel workbook as the most widely used format. Reusability: A test automation framework is a set of routines that are reused by tests, increasing the speed of test development. Each layer should be developed to promote reusability. The more generic a layer, the more reusable it is. A framework with high reusability gives project teams a platform, on which they can write their test scripts with minimal effort. An important decision in reusability is to define the scripts, functions and methods that can be reused. Test steps commonly include tasks such as - verify messages, click on objects, enter, select, verify if an object is present, verify if editable, verify data and verify pages. Creation of reusable component for these test steps enhances reusability. Functions must be created for business rules by studying the list of business layers and any specific algorithms. Similarly, error handlers, to handle known and unknown errors and logging information must be built. Automation Governance: Automation governance will ensure consistency in scripting or execution by defining a set of rules to be applied consistency across the framework. Governance includes coding standards, test suite structure, test script structure, creation of test suite, error and exception handling, creation of test data, creation of test cases, naming conventions and so on. Governance will ensure modifications of scripts for future enhancements easy. Integration: The framework design should allow flexible integration with different tools such as test management and defect management tools and ensure smooth data flow between them. When a test case fails during test execution, the framework must log a defect in a defect tracking tool, and upload screen shots. Integration with a test management tool will allow execution of test cases from the test management tool and establish bidirectional traceability from requirements all the way to defects. The design for integration of tools should be based on a “plug and play” approach, allowing tools to be changed or removed without affecting the main features of the framework. Execution settings: Automated test execution is a critical part of framework design and the execution settings must be flexible and user friendly. The framework should allow execution of selective test cases, scheduling of test execution and support execution of individual test cases, as well as execution in a batch (combination of tests). Executing test cases on multiple browsers, in parallel and in a distributed manner should be one the core design considerations for a framework. The design must also take care of the different operating systems and browsers. Report management: Reports are the face of a test automation framework and enhance the value of a framework. Customizable and user friendly reporting allows a user to decide on the type, level of reports and reporting formats (html, xml, excel, word, pdf ) to be created for the executed test cases. The framework should produce both summary (high level view of results of all the test cases executed) and detailed reports (showing step by step details) reports. The framework should verify results, handle errors with screenshots and report results to suit the needs of different stakeholders. Logging Mechanism: Log generation is an important part of execution. It is vital to generate debug information at various points in the execution of a test case. The framework should provide a mechanism to create logs which can help to find a problem area quickly, thereby reducing the time to fix issues. 5 Page
  6. 6. idexcel Automation Framework Design IDEXCEL’s Test Automation Framework (i-TAF) automation using Selenium. The high level architecture of i-TAF is shown below. Idexcel has built a test automation framework, i-TAF, keeping in mind the design considerations articulated in this whitepaper. The framework is designed for web functional Test Result Customized Test Results A READ Test Management Tools Defect Log Defect Management Tools DOCS Library Driver Scripts XLSX TXT Testing Types (Application Indipendent reusable Functions) Web Services Hybrid Engine HTML Business Flows (Keywords Drivers) 0 1 1 0 1 1 1 0 0 0 1 1 1 0 1 1 Test Data (Data Drivers) Test Data Generation Tool Web Applications Database MAIL Project Library (Reusable Business Components) Test Result Defect Log Idexcel’s Test Automation Framework (i-TAF) advocates the creation of a driver script, independent functions and scripts for work actions and specific test cases in different phases. Business flows are captured in a Keyword driven architecture and the test data is Data driven, which provides a Hybrid Framework Architecture. It also contains data tables to maintain the scripts to be executed and the application specific test data. Test execution results are published as customized reports to validate the status of execution. Defects encountered during execution are automatically logged in a Defect Management tool. Built on design considerations discussed in this whitepaper, Idexcel’s i-TAF offers an efficient framework, with a high degree of reusability and maintainability. Integration with others tools such as test management and defect management provides complete end-to -end automated test management capabilities, with customized reporting. 6 Page
  7. 7. idexcel Automation Framework Design Conclusion Design of a test automation framework is a vital activity in the building of a test automation framework. A well designed framework provides a higher ROI by significantly reducing the cost of test automation, without compromising on quality. Implementation of the design considerations discussed in this whitepaper will help organizations build a high performance test automation framework that meets complex and future testing requirements, as the demand for improving test efficiencies and effectiveness in test cycles through test automation increases. The key to success here will be to adopt winning strategies realized through a better approach to framework design that will keep testing costs under control, without compromising quality. Idexcel’s Test Automation Framework (i-TAF) has helped its clients to jump start their test automation initiatives and realize higher test efficiencies and return on investment. 7 Page
  8. 8. idexcel Automation Framework Design About the Author Anita Shivaprakash, is a Test Automation Lead with Idexcel and has more than 8 years of experience in test automation. Anita is a certified HP AIS Professional (QTP and QC). She has strong knowledge on SDLC and STLC fundamentals along with a sound knowledge of various QA processes and techniques. Anita has expertise in design, development and implementation of different automation testing Frameworks (Keyword driven, Data driven, Modular and Hybrid). She is the chief architect of Idexcel's Test Automation Framework, i-TAF. Anita is also well versed in the other functional tools such as Test Complete, Selenium and Coded UI. About Idexcel Idexcel is an innovative provider of IT Products & Services focused on emerging technologies. We help world leading companies build efficiencies and stronger businesses. With more than 15 years into existence Idexcel’s main focus is client satisfaction and technology innovation. Our industry expertise and a global, collaborative workforce forms the backbone of our services. We offer high degree of skills in Enterprise Applications, Cloud Services, Data-warehousing, Big Data, Analytic, QA & Testing Services, IT consulting and Staffing. Idexcel product line includes: NDS, ERP, and Cync - A revolutionary credit monitoring application for the manufacturing and financial management. For more information log on to Global Head quarters 459 Herndon Parkway Suite 11 Herndon, VA 20170 Tel: 703-230-2600 Fax: 703-467-0218 Email: India Operations “Crystal Plaza” 9, 10 ,11 Bhuvanappa Layout, Hosur Road Bengaluru – 560 029 Karnataka Tel: +91-80-2550 8830 Email: © Copyright, Idexcel. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Idexcel. The information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.