The document provides guidelines for designing a robust test automation framework. It discusses that a well-designed framework increases testing efficiency and reduces costs. The key aspects of framework design include defining objectives, selecting the appropriate framework type, managing data, ensuring reusability, integrating with other tools, and flexible execution and reporting. Idexcel's test automation framework was created following these best practices to provide a maintainable and reusable framework.

1. idexcel
Designing a Robust
Test Automation
Framework
White Paper

2. idexcel
Automation Framework Design
Introduction
In today's business environment, software project teams are expected to do more and deliver higher quality systems in
lesser time, with fewer resources. To increase speed, accuracy and completeness of testing, testers rely on either commercial or open source test automation tools. Any automated testing effort based on using only a capture tool to record and
play back test cases has many drawbacks, as they are created ad hoc, their functionality can be difficult to track and reproduce, and they are costly to maintain. Running complex and powerful tests is time consuming and expensive when using
only a capture tool. To make automated testing more efficient, a better option would be to build a test automation framework that supports scripting and extends the capabilities of the automation tool.
A test automation framework defines an organization's way of doing things. It is a set of assumptions, concepts and tools
that provide support for automated software testing. A test automation framework makes test automation more efficient,
provides a higher ROI, reduces the test automation effort and lowers the cost of maintenance.
A framework may be built to support either commercial or open source tools available in the market. Different tools
provide different inbuilt features, and a framework must focus on extending the capabilities and features not provided by
the tool. A test automation framework allows you to write a series of tests without worrying about the constraints or
limitations of the underlying test tools.
The most critical aspect of building a good test automation framework is its design. This Whitepaper discusses the best
practices in designing a robust test automation framework.
2 Page

3. idexcel
Automation Framework Design
Objectives of a
Test Automation
Framework
Benefits of a Well
Designed Test
Automation Framework
It is very important to define the objectives of a Test
Automation Framework before starting its design and
development. An organization may choose some or all
of the objectives listed below, depending on its business
needs. The design of the test automation framework
should be based on its objectives defined by the organization.
A well designed framework increases the speed and accuracy of the testing process, enables a higher return on
investment (ROI), reduces maintenance and systematically
minimizes risks. A well designed framework:
Platform & OS independence
Support different testing types (UI, database, Web
Service).
Data driven capabilities (input data, output data)
Framework should be driven through tables / key
words
Ability to parameterize (recorded) scripts
End to end testing capability to cover business
scenarios
View and edit test cases intuitively
Ability to implement dynamic conditions & dynamic
loop support
Test framework should be loosely / tightly coupled
with application-under-test (AUT) depending on
business needs
Enhances efficiency in development of automated test
scripts through modularized, reusable, and maintainable code and data
Provides a structured development methodology to
ensure uniformity of design across multiple test scripts
to reduce dependency on individual test-case developers
Provides reliable issue detection and efficient
root-cause analysis with minimum human intervention
for the system under test
Reduces dependence on subject matter experts by
automatically selecting the test to execute according to
test scenarios and dynamically refining the test scope
according to changes in the test strategy or conditions
of the system under test
Is tool agnostic
Auto update of test results in a test management tool
Performs multiple levels and types of testing – functional, regression, load, performance, unit, integration, etc
Less dependence on highly skilled automation
professionals
Eliminates deployment to multiple machines for testing
different versions of an application
Ability to schedule and run prioritized test cases
Capture screenshots for failed steps and log defects
in a defect management tool
Runs tests in parallel, runs on test machines in the grid
or cloud, tests across multiple environments and application versions simultaneously
Test reporting objectives including email notifications.
Runs in a continuous integration environment.
3 Page

4. idexcel
Automation Framework Design
High Level
Framework
Design Guidelines
Test Automation
Framework Design
Considerations
As in SDLC to develop software applications, framework
design plays a vital role in building a test automation
framework for an organization. Enterprise applications
are large, complex, and ever-evolving, and to keep pace
with the organization’s growth, the design of the test
automation framework should conform to the guidelines listed below.
The design phase starts after the test automation framework requirements are defined and base-lined. A good
test automation framework should be flexible, application
agnostic, technology agnostic and future-proof. The
framework must be application independent and must
have the capability to expand, with the requirements of
each application.
The framework should be based on open technologies, such as Java, to limit the development and maintenance costs. Openness might also allow cross-tool
execution when a suitable API exits or can be developed
Should be application-independent and easy to
enhance and maintain
The framework and its assets should allow
end-to-end testing of the application suite
Encapsulate testers from complexities of the test
framework
Identify and abstract common functions used across
multiple test scripts
The framework should allow easy integration with
other tools and components
Decouple complex business function testing from
navigation, limit-testing, and other simple verification
and validation activities.
Decouple test data from the test scripts
Structure scripts with minimal dependencies - ensuring execution of scripts unattended even on failures.
The various factors to be considered in designing the
framework are discussed in detail below.
Selection of the type of Framework : The type of automation framework helps to organize test suites and in turn
improves testing efficiency. Different types of framework
focus on different objectives. The “Data driven framework” focuses on data management, the “Keyword driven
framework” focuses on creation of test cases using
keywords, while the “Modular framework” reduces redundancy.
The framework type should be chosen based on the
organization’s requirements and objectives. If the requirement is to manage data and execute test cases for multiple
sets of data, the “data driven framework” would be appropriate. Similarly, if the framework objective is to divide the
project into modules and create scripts independently for
each module, the “Modular” framework would be appropriate. Sometimes, a “Hybrid framework”, combining the
strengths of different types of framework would work well.
Data Management: Data plays an important role in the
framework design. The best approach is to ensure that
there is no hard coding of data. The user should be allowed
to manipulate and input the data for test cases (input and
expected data). The framework contains different data
such as test scripts creation data, test input and expected
data, test results and should manage the flow between
4 Page

5. idexcel
Automation Framework Design
them easily. The framework should support user friendly
interface files (excel, CSV, database, etc). Choosing a
database may become a constraint if the organization uses
different databases, while CSV is not easy to use, leaving
excel workbook as the most widely used format.
Reusability: A test automation framework is a set of
routines that are reused by tests, increasing the speed of
test development. Each layer should be developed to
promote reusability. The more generic a layer, the more
reusable it is. A framework with high reusability gives
project teams a platform, on which they can write their
test scripts with minimal effort.
An important decision in reusability is to define the scripts,
functions and methods that can be reused. Test steps
commonly include tasks such as - verify messages, click on
objects, enter, select, verify if an object is present, verify if
editable, verify data and verify pages. Creation of reusable
component for these test steps enhances reusability.
Functions must be created for business rules by studying
the list of business layers and any specific algorithms.
Similarly, error handlers, to handle known and unknown
errors and logging information must be built.
Automation Governance: Automation governance will
ensure consistency in scripting or execution by defining a
set of rules to be applied consistency across the framework. Governance includes coding standards, test suite
structure, test script structure, creation of test suite, error
and exception handling, creation of test data, creation of
test cases, naming conventions and so on. Governance will
ensure modifications of scripts for future enhancements
easy.
Integration: The framework design should allow flexible
integration with different tools such as test management
and defect management tools and ensure smooth data
flow between them. When a test case fails during test
execution, the framework must log a defect in a defect
tracking tool, and upload screen shots. Integration with a
test management tool will allow execution of test cases
from the test management tool and establish bidirectional
traceability from requirements all the way to defects. The
design for integration of tools should be based on a “plug
and play” approach, allowing tools to be changed or
removed without affecting the main features of the framework.
Execution settings: Automated test execution is a critical
part of framework design and the execution settings must
be flexible and user friendly. The framework should allow
execution of selective test cases, scheduling of test execution and support execution of individual test cases, as well
as execution in a batch (combination of tests). Executing
test cases on multiple browsers, in parallel and in a distributed manner should be one the core design considerations
for a framework. The design must also take care of the
different operating systems and browsers.
Report management: Reports are the face of a test automation framework and enhance the value of a framework.
Customizable and user friendly reporting allows a user to
decide on the type, level of reports and reporting formats
(html, xml, excel, word, pdf ) to be created for the executed test cases. The framework should produce both
summary (high level view of results of all the test cases
executed) and detailed reports (showing step by step
details) reports. The framework should verify results,
handle errors with screenshots and report results to suit
the needs of different stakeholders.
Logging Mechanism: Log generation is an important part
of execution. It is vital to generate debug information at
various points in the execution of a test case. The framework should provide a mechanism to create logs which
can help to find a problem area quickly, thereby reducing
the time to fix issues.
5 Page

6. idexcel
Automation Framework Design
IDEXCEL’s
Test Automation
Framework (i-TAF)
automation using Selenium. The high level architecture of
i-TAF is shown below.
Idexcel has built a test automation framework, i-TAF, keeping in mind the design considerations articulated in this
whitepaper. The framework is designed for web functional
Test Result
Customized Test
Results
A
READ
Test
Management
Tools
Defect Log
Defect
Management
Tools
DOCS
Library
Driver Scripts
XLSX
TXT
Testing
Types
(Application
Indipendent reusable
Functions)
Web
Services
Hybrid Engine
HTML
Business
Flows
(Keywords Drivers)
0
1
1
0
1
1
1
0
0
0
1
1
1
0
1
1
Test
Data
(Data Drivers)
Test Data
Generation
Tool
Web
Applications
Database
MAIL
Project Library
(Reusable Business Components)
Test Result
Defect Log
Idexcel’s Test Automation Framework (i-TAF) advocates
the creation of a driver script, independent functions and
scripts for work actions and specific test cases in different
phases. Business flows are captured in a Keyword driven
architecture and the test data is Data driven, which
provides a Hybrid Framework Architecture. It also contains
data tables to maintain the scripts to be executed and the
application specific test data. Test execution results are
published as customized reports to validate the status of
execution. Defects encountered during execution are
automatically logged in a Defect Management tool.
Built on design considerations discussed in this whitepaper, Idexcel’s i-TAF offers an efficient framework, with a
high degree of reusability and maintainability. Integration
with others tools such as test management and defect
management provides complete end-to -end automated
test management capabilities, with customized reporting.
6 Page

7. idexcel
Automation Framework Design
Conclusion
Design of a test automation framework is a vital activity in the building of a test automation framework. A well designed
framework provides a higher ROI by significantly reducing the cost of test automation, without compromising on quality. Implementation of the design considerations discussed in this whitepaper will help organizations build a high performance test automation framework that meets complex and future testing requirements, as the demand for improving
test efficiencies and effectiveness in test cycles through test automation increases. The key to success here will be to
adopt winning strategies realized through a better approach to framework design that will keep testing costs under
control, without compromising quality.
Idexcel’s Test Automation Framework (i-TAF) has helped its clients to jump start their test automation initiatives and
realize higher test efficiencies and return on investment.
7 Page

8. idexcel
Automation Framework Design
About the Author
Anita Shivaprakash, is a Test Automation Lead with Idexcel and has more than 8 years of experience in test automation.
Anita is a certified HP AIS Professional (QTP and QC). She has strong knowledge on SDLC and STLC fundamentals along
with a sound knowledge of various QA processes and techniques. Anita has expertise in design, development and
implementation of different automation testing Frameworks (Keyword driven, Data driven, Modular and Hybrid). She
is the chief architect of Idexcel's Test Automation Framework, i-TAF. Anita is also well versed in the other functional
tools such as Test Complete, Selenium and Coded UI.
About Idexcel
Idexcel is an innovative provider of IT Products & Services focused on emerging technologies. We help world leading
companies build efficiencies and stronger businesses. With more than 15 years into existence Idexcel’s main focus is
client satisfaction and technology innovation. Our industry expertise and a global, collaborative workforce forms the
backbone of our services. We offer high degree of skills in Enterprise Applications, Cloud Services, Data-warehousing,
Big Data, Analytic, QA & Testing Services, IT consulting and Staffing. Idexcel product line includes: NDS, ERP, and
Cync - A revolutionary credit monitoring application for the manufacturing and financial management.
For more information log on to www.idexcel.com.
Global Head quarters
459 Herndon Parkway Suite 11
Herndon, VA 20170
Tel: 703-230-2600
Fax: 703-467-0218
Email: inquiry@idexcel.com
India Operations
“Crystal Plaza” 9, 10 ,11
Bhuvanappa Layout, Hosur Road
Bengaluru – 560 029
Karnataka
Tel: +91-80-2550 8830
Email: inquiry@idexcel.com
© Copyright, Idexcel. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means,
electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Idexcel. The information contained herein is subject to
change without notice. All other trademarks mentioned herein are the property of their respective owners.