This document discusses best practices for designing a robust automation testing framework. It outlines several key aspects of framework design including different generations of frameworks, considerations when designing a framework, and the objectives of an automation testing framework. The most critical aspect of building a good test automation framework is its design, which this white paper seeks to provide guidance on. It emphasizes that a well-designed framework will guide a project to success while a poorly designed one can waste time, money and resources.
From http://wiki.directi.com/x/AgAa - This is a 24 slide internal presentation covering virtues of Automated Testing vs Manual Testing. Inkeeping with our agile adoption this presentation covers various advantages (11 to be specific) obtained in using TDD and Automated Testing as opposed to Manual Testing
Test Automation Best Practices (with SOA test approach)Leonard Fingerman
Today we hear a lot of buzz about the latest & greatest test automation tools like Selenium, Rational Functional Tester or HP LoadRunner but to make your test automation effort successful it might take more than just having the right tool. This presentation will try to uncover major pitfalls typically involved with test automation efforts. It will provide guidance on successful strategy as well as differences among third-generation frameworks like keyword-driven, data-driven and hybrid. It will also cover various aspects of SOA test automation
Katalon Studio - Successful Test Automation for both Testers and DevelopersKatalon Studio
There is a "great divide" between Developers' and Testers' disciplines, which leads to silo'ed test automation approaches with either inefficient or ineffective result. In this presentation, I introduce Katalon Studio, a free test automation IDE, as an attempt to help our developers and testers collaborate together towards a more reliable and robust test automation implementation.
Original source: https://www.slideshare.net/minhhai2209/successful-test-automation-for-both-testers-and-developers-75417401
From http://wiki.directi.com/x/AgAa - This is a 24 slide internal presentation covering virtues of Automated Testing vs Manual Testing. Inkeeping with our agile adoption this presentation covers various advantages (11 to be specific) obtained in using TDD and Automated Testing as opposed to Manual Testing
Test Automation Best Practices (with SOA test approach)Leonard Fingerman
Today we hear a lot of buzz about the latest & greatest test automation tools like Selenium, Rational Functional Tester or HP LoadRunner but to make your test automation effort successful it might take more than just having the right tool. This presentation will try to uncover major pitfalls typically involved with test automation efforts. It will provide guidance on successful strategy as well as differences among third-generation frameworks like keyword-driven, data-driven and hybrid. It will also cover various aspects of SOA test automation
Katalon Studio - Successful Test Automation for both Testers and DevelopersKatalon Studio
There is a "great divide" between Developers' and Testers' disciplines, which leads to silo'ed test automation approaches with either inefficient or ineffective result. In this presentation, I introduce Katalon Studio, a free test automation IDE, as an attempt to help our developers and testers collaborate together towards a more reliable and robust test automation implementation.
Original source: https://www.slideshare.net/minhhai2209/successful-test-automation-for-both-testers-and-developers-75417401
A test automation framework defines an organization's way of doing things. It is a set of assumptions, concepts and tools that provide support for automated software testing.
Software testing concepts is going to enhance from manual testing to automated testing. Due to rapidly increasing complexity in software development process, the present manual testing technique is not sufficient for effective testing. Hence, here arises the requirement of test automation process. The automation testing is the software testing process in which the tester use advance automation tools for testing software applications areas, which needs to be tested again-and-again due to frequent changes.
Selenium Training in Chennai. We are Best provider of Automation Testing with Practical Training. Start your Career with us, click here: http://www.metaforumtechnologies.com/selenium-training-in-chennai
Test Automation Frameworks Using Selenium | EdurekaEdureka!
YouTube: https://youtu.be/2idCKxbhXOk
** Test Automation Engineer Masters Program: https://www.edureka.co/masters-program/automation-testing-engineer-training **
This Edureka Live PPT on ‘Test Automation Framework’ will help you learn about what testing frameworks are and how to build one using Selenium. Below topics are covered in this PPT:
Introduction to Automation Testing
Approaches to Test Automation
Test Automation Frameworks
Types of Test Automation Frameworks
Demo: How to build a test automation framework using selenium.
Security Testing Selenium playlist: https://goo.gl/NmuzXE
Selenium Blog playlist: http://bit.ly/2B7C3QR
Software Testing blog playlist: http://bit.ly/2UXwdJm
Follow us to never miss an update in the future.
YouTube: https://www.youtube.com/user/edurekaIN
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
It is a software testing tool which works of the Glass testing technique. In this presentation people will get to know that how this tool work and help the tester.
Test Automation
Test automation is the use of test automation software like Selenium or self-developed testware to execute test cases.
Test automation is mostly used to automate repetitive testing tasks in a formalized way. It is also used to execute tests that would be difficult to perform manually like performance testing.
There are many advantages of test automation that are mostly related to the repeatability of the tests and the speed of test execution. There are a lot of commercial and open source tools available which can be grouped into two main categories; Code-Driven and Graphical User Interface Testing. Thus the key success factor in test automation is selecting the right tool and have a specialized test automation team.
It is to use test automation tools by considering ROI (return on investment). Otherwise it is quite easy to waste big amount of energy, commitment and definitely money.
With more than 500 clients, Keytorc is the leading software testing company in EMEA region that have competencies of automating any kind of software in diverse industries.
For more information about test automation tools and Keytorc’s test automation service you can contact with our performance test engineers: www.keytorc.com or blogs.keytorc.com
Test Otomasyonu:
Test otomasyonu özellikle tekrarlayan ve manuel olarak yapılması zor olan testlerin Selenium gibi test otomasyon araçları kullanılarak ya da özel olarak geliştirilen test otomasyon scriptleri ile yapılmasıdır. Bu bakımdan:
- Regresyon testleri
- Performans testleri
- Yük ve Stres testleri
- Test yönetimi
test otomasyonuna en uygun test tipleri ve aktiviteleridir.
Test otomasyonunun en büyük faydaları:
- testlerin hıznının artırılması
- testlerin kapsamının artırılması
- testlerin doğruluğunun artırılması
- testlerin raporlama kalitesinin artırılmasıdır.
Test otomasyonu doğru araç seçimi yapılmaması, ya da test otomasyonunu bilen uzman bir ekip tarafından yapılmaması durumunda faydadan çok zarar getirmektedir.
EMEA bölgesindeki lider yazılım test firması olan Keytorc’un test otomasyon ekibiyle iletişime geçmek için: www.keytorc.com ya da blogs.keytorc.com
Test automation: Are Enterprises ready to bite the bullet?Aspire Systems
This whitepaper talks about the actual challenges in implementing a successful Test Automation process. It give a glimpse of the 3Ws and 1H(Why, When, What & How) of automation and explains how the cost factor is just a myth. Eventually it talks about how Continuous Innovation with opensource tools together with a robust framework and business focused testing approach can lead to a successful test automation implementation.
A test automation framework defines an organization's way of doing things. It is a set of assumptions, concepts and tools that provide support for automated software testing.
Software testing concepts is going to enhance from manual testing to automated testing. Due to rapidly increasing complexity in software development process, the present manual testing technique is not sufficient for effective testing. Hence, here arises the requirement of test automation process. The automation testing is the software testing process in which the tester use advance automation tools for testing software applications areas, which needs to be tested again-and-again due to frequent changes.
Selenium Training in Chennai. We are Best provider of Automation Testing with Practical Training. Start your Career with us, click here: http://www.metaforumtechnologies.com/selenium-training-in-chennai
Test Automation Frameworks Using Selenium | EdurekaEdureka!
YouTube: https://youtu.be/2idCKxbhXOk
** Test Automation Engineer Masters Program: https://www.edureka.co/masters-program/automation-testing-engineer-training **
This Edureka Live PPT on ‘Test Automation Framework’ will help you learn about what testing frameworks are and how to build one using Selenium. Below topics are covered in this PPT:
Introduction to Automation Testing
Approaches to Test Automation
Test Automation Frameworks
Types of Test Automation Frameworks
Demo: How to build a test automation framework using selenium.
Security Testing Selenium playlist: https://goo.gl/NmuzXE
Selenium Blog playlist: http://bit.ly/2B7C3QR
Software Testing blog playlist: http://bit.ly/2UXwdJm
Follow us to never miss an update in the future.
YouTube: https://www.youtube.com/user/edurekaIN
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
It is a software testing tool which works of the Glass testing technique. In this presentation people will get to know that how this tool work and help the tester.
Test Automation
Test automation is the use of test automation software like Selenium or self-developed testware to execute test cases.
Test automation is mostly used to automate repetitive testing tasks in a formalized way. It is also used to execute tests that would be difficult to perform manually like performance testing.
There are many advantages of test automation that are mostly related to the repeatability of the tests and the speed of test execution. There are a lot of commercial and open source tools available which can be grouped into two main categories; Code-Driven and Graphical User Interface Testing. Thus the key success factor in test automation is selecting the right tool and have a specialized test automation team.
It is to use test automation tools by considering ROI (return on investment). Otherwise it is quite easy to waste big amount of energy, commitment and definitely money.
With more than 500 clients, Keytorc is the leading software testing company in EMEA region that have competencies of automating any kind of software in diverse industries.
For more information about test automation tools and Keytorc’s test automation service you can contact with our performance test engineers: www.keytorc.com or blogs.keytorc.com
Test Otomasyonu:
Test otomasyonu özellikle tekrarlayan ve manuel olarak yapılması zor olan testlerin Selenium gibi test otomasyon araçları kullanılarak ya da özel olarak geliştirilen test otomasyon scriptleri ile yapılmasıdır. Bu bakımdan:
- Regresyon testleri
- Performans testleri
- Yük ve Stres testleri
- Test yönetimi
test otomasyonuna en uygun test tipleri ve aktiviteleridir.
Test otomasyonunun en büyük faydaları:
- testlerin hıznının artırılması
- testlerin kapsamının artırılması
- testlerin doğruluğunun artırılması
- testlerin raporlama kalitesinin artırılmasıdır.
Test otomasyonu doğru araç seçimi yapılmaması, ya da test otomasyonunu bilen uzman bir ekip tarafından yapılmaması durumunda faydadan çok zarar getirmektedir.
EMEA bölgesindeki lider yazılım test firması olan Keytorc’un test otomasyon ekibiyle iletişime geçmek için: www.keytorc.com ya da blogs.keytorc.com
Test automation: Are Enterprises ready to bite the bullet?Aspire Systems
This whitepaper talks about the actual challenges in implementing a successful Test Automation process. It give a glimpse of the 3Ws and 1H(Why, When, What & How) of automation and explains how the cost factor is just a myth. Eventually it talks about how Continuous Innovation with opensource tools together with a robust framework and business focused testing approach can lead to a successful test automation implementation.
Test Automation is an accepted technique which is adapted by the industry for increasing the effectiveness of the testing phase. The recurring tasks are being automated by the tools thus simplifying the human efforts and results in increased quality of product under test. A study of test automation programmes in the industry reveals the fact that a good percentage of them fail to find the intended results.
Improving ROI with Scriptless Test AutomationMindfire LLC
This is where scriptless test automation comes into the picture. Businesses today may utilize Scriptless Test Automation to automate test cases without having to worry about the complexities of coding. It speeds up the time to learn and build code, resulting in a shorter time to market, a greater return on investment, and increased coverage with little maintenance.
The test automation framework is a set of protocols or guidelines and doesn't rely on a single process or tool. It has incorporated many tools, practices, and functions, including reusable modules, test data, and libraries that work together to help QA professionals to perform the testing process effectively. To know more visit at https://www.zenesys.com/blog/popular-test-automation-frameworks
Test Automation Framework An Insight into Some Popular Automation Frameworks.pdfSerena Gray
An automated test environment can be easily set up using the framework, which will in turn help boost the performance of development and QA teams. In this article, you will get to know about the function of a test automation framework along with the most popular test automation frameworks.
3 Approaches for Integrated ALM - A Case for ALM Platform - WhitepaperKovair
Download link - https://www.kovair.com/white-papers/3-approaches-for-integrated-alm-a-case-for-alm-middleware/
Different Approaches for Integrated ALM
To address the challenges of Integrated ALM and to achieve at least some of the goals, different tool vendors have taken different strategies, which can be broadly categorized in the three alternative approaches.
3 Approaches to Integrated ALM, A case for ALM PlatformKovair
Today, if an organization is not thinking about integrated ALM, it is missing some compelling benefits. In this paper, we will discuss three widely used approaches provided by different vendors over time towards achieving Integrated ALM and the relative merits of these solutions.
3 Approaches to Integrated ALM, A case for ALM Platform
Automation Framework Design
1. White Paper on Automation Framework Design
Written by Kunal Saxena 1 | P a g e
Steps to succeed in Automation Framework Design
Automation Framework Design
White Paper
ABSTRACT: This is a white paper on creating a robust automation framework for your application. Good automation
framework will guide your project to success, where as bad framework will cost you money, time and resource wastage.
Whitepaper will highlight good Automation Framework guidelines, considerations and benefits.
2. White Paper on Automation Framework Design
Written by Kunal Saxena 2 | P a g e
INDEX
Table of Contents
Introduction ............................................................................................................................................................................3
Generations of Automation Frameworks ...............................................................................................................................4
What is AUTOMATION............................................................................................................................................................5
Automation Framework..........................................................................................................................................................7
Framework Design Guidelines ................................................................................................................................................9
Framework deign considerations..........................................................................................................................................10
Summary...............................................................................................................................................................................13
About the author ..................................................................................................................................................................14
3. White Paper on Automation Framework Design
Written by Kunal Saxena 3 | P a g e
Introduction
In today’s environment, software quality professionals are expected to do more with high quality and precision. They are
supposed to work equivalent to two resources put together. To increase their speed, accuracy, precision during testing
can be achieved only by automating features, which are complete. So that testers can save their time for new features
which needs manual effort.
Creating Automation Framework means you are mechanizing your redundant task. Now you don’t have to waste time in
doing same thing again, concentrate on new features and check result of older features generated by automation suite.
Creating automation suite, by using one method is different than implementing an automation framework for your test,
which make your testing efficient and you can include many task inside your framework e.g. record and play, scripting,
data driven test, logging, reporting and using capabilities of various tools available.
Good Framework Designing means you decide each task to be done modularly and without hindering other task. Coding
of cases is kept separate and handling logging and reporting is implemented separately. It includes development of
common library for the suite, with good logging and reporting to the stakeholders and removing duplicate functions to
remove redundancy. This will help you to add new feature and new test cases easily to the suite.
A framework should be built to support commercial and open source tools for the development. Different tools provide
different inbuilt features, and a framework must focus to use features of different tools and languages. A test
automation framework allows you to write a series of tests without worrying about the constraints or limitations of the
underlying test tools.
“The most critical aspect of building a good test automation framework is its design. This Whitepaper discusses
the best practices in designing a test automation framework.”
4. White Paper on Automation Framework Design
Written by Kunal Saxena 4 | P a g e
Generations of Automation Frameworks
First Generation:-
This refers to Record/Replay (Capture/Playback) as publicized by different automation testing tools.
This our 1st generation, may be applied at beginner learning level.
Major flaws of this framework are non-reusable code, difficult to maintain and application flow dependent.
Second Generation:-
Commonly known as Data Driven Framework, it is to run the same test cases with varying test data.
This data can be stored in Data tables/ Excel / Access. The idea is to separate coding part from the data portion.
It provides flexibility to tester to modify data easily rather than going to the code.
Third Generation:-
Keyword Driven / Hybrid frameworks. As the name indicates Hybrid means mix of two or more frameworks.
This is frequently implemented framework in the testing world.
It can be modular in approach, separate SET of data and driven by keywords.
Fourth Generation:-
Known with the name MBT (Model Based Testing).
Key approach is focusing on building a model, with specified input test data, estimated output results,
Run the tests and after running compare actual with expected result to decide the future course of action.
Fifth Generation:-
This is the upcoming framework called as Script less/Codeless Framework.
Approach is to use UI interface to record the objects and build the tests.
The key to a 5th gen framework is building the pieces in memory at runtime. Like object building using
Descriptive Programming in UFT.
5. White Paper on Automation Framework Design
Written by Kunal Saxena 5 | P a g e
What is AUTOMATION
Automation is to execute your existing regression test cases without any human intervention. And your script should
keep on updating stakeholders after every milestone. For good automation, manager and resources have to plan, lot of
things in advance, so that your automation suite is robust, maintainable and error free. All points which are taken into
consideration before developing and automation suite is called automation Framework.
Following are basic points you should keep in mind while creating Automation Framework:
Proper documentation for running Automation Suite
Easy to learn and easy to execute
Easy to add new Test Cases
Easy to add new Features / Product.
Easy to maintain Input data for different products.
Good Logging for each test case.
Reporting to Stakeholder at each milestone.
Scaling of your automation suite.
Easy to increase / decrease number of cases in regression.
Easy to Increase / decrease number of system.
Can plugin script to check performance stats and Accuracy Stats
Automation suite setup in new machine
Lifecycle of Automation Framework:
6. White Paper on Automation Framework Design
Written by Kunal Saxena 6 | P a g e
Lifecycle of Automation, starts when, product is in design phase and ends when product is obsolete and moved to rack.
There are five different stages of automation and each phase has its own importance. If you miss one phase than it will
effect next phase and your automation framework as a whole.
First Stage is to understand the business and product. Identify tools to be used in framework. Test tool is decided by
going through design document and requirement of the product. Factors like cost, availability of the tool and ease of
integration with other tools will affect the decision. We can select tools internally used by the company, which can be
embedded in the framework, for various task like logging, reporting, and integration with messaging apps like hipchat,
slack, flowdock etc …
Second Stage is defining scope and coming up with the test cases you can and test cases which you cannot automate.
Here you have to define scope and different milestones to track to your progress. Application Under Test, itself have
different testing types to automate (namely UI, End-to-End, server messages, logging, memory leak, threshold, load,
stress, performance). Finalize scope before starting framework deign. Design should be flexible enough to incorporate
different scenarios.
Third Stage is Planning, Designing and Developing your framework. Which is most time consuming part of the whole
cycle. This phase goes with the framework development. You plan and design your framework and as features are
available you starts automating those features. Separate code for features, logging and reporting. If needed you can
easily added or remove features and test cases without changing framework and code. Config file is very important
while implementing framework, you take necessary values from config files when your framework is running.
Fourth Stage is execution and it goes side by side with third phase. Here you are developing new features and
executing old features. You have to analyze your test cases which are failed and understand the reason of failures, by
going through logs. Reason could be an issue in application or flaw in your automation suite. You can embed a log parser
for the test cases which are failed.
Fifth Stage is maintenance phase, when your product is stable and in the market and no more features are planned.
You maintain your framework and scripts and update results of automation results on daily basis.
7. White Paper on Automation Framework Design
Written by Kunal Saxena 7 | P a g e
Test Automation Framework Objective
A test automation framework can be defined as a set of processes, standards and interactions between the components
in which scripts are designed and executed. Different types of Automation framework can be developed.
It is very important to define the objectives of a Test Automation Framework before starting its design and
development. An organization may choose some or all of the objectives listed below, depending on its business needs.
The design of the test automation framework should be based on its objectives defined by the organization.
Independent of Platform & OS Support different testing types (UI,
database, WebService)
Test Automation
Framework are guidelines,
which when followed
gives high quality results.
Guidelines are like coding
standards, test-input data
handling, directory
structure, logging,
reporting etc.
This helps in reducing
code redundancy, higher
portability, reduced script
maintenance and better
quality of framework
Hybrid Framework is used to
cater multiple needs (tables /
keywords)
Ability to parameterize (recorded)
scripts
Implement dynamic conditions
& loop support
Test framework should be loosely /
tightly coupled with AUT depending
on business needs
Auto update of test results in a
test management tool
Less dependence on highly skilled
automation professionals
Ability to schedule and run
prioritized test cases
Auto raise defect in defect tracking
tool.
Data driven capabilities (input
data, output data)
Test reporting objectives including
email notifications.
End to End testing capability to
cover business scenarios
Log defects in a defect management
tool
8. White Paper on Automation Framework Design
Written by Kunal Saxena 8 | P a g e
Framework Methodology:
10 Steps to Design
a Good Automation
Framework
Key parameters to keep in mind for
framework design
Handle scripts and data separately
Create libraries
Follow coding standards
Offer high extensibility
Less maintenance
Script/Framework version control
Logging
Reporting to Stakeholders
1. Identification of the Scope: Company oriented, Product
oriented, Project Oriented.
2. Identification of the Needs: Identify Types of testing e.g.
Functional Test, Web Services etc. and application / modules to
be tested, GUI testing.
3. Identification of the Requirements: Find out the nature of
requirements, identify type of actions for each requirement &
identify high priority requirements.
4. Evaluation of the Test Automation Tool: Evaluation checklist,
Identify the candidate tools available in the market, Sample
run, rate & select the tools, Implementation & Training
5. Identification of the Actions to be automated: Actions,
Validations & requirements supported by the Tool, messages
between servers.
6. Design of the Test Automation Framework: Framework
guidelines, validations, Actions Involved, Systems involved, Tool
Extensibility Support, Customs messages & UML
Documentation.
7. Design of the Input Data Bank: Types of Input file. Input files –
Categorization & Design of file prototypes.
8. Development of the Automation Framework: Development of
script based upon framework design, Driver scripts, Worker
Scripts, Record / Playback, Screen / Window / Transaction,
Action / Keyword & Data Driven.
9. Population of Input Data Bank: Different Types of data Input,
Populate data from different data sources, Manual input of
data and Parent – Child data hierarchy.
10. Configuration of the Schedulers: Identify scheduler
requirements & configure the schedulers.
9. White Paper on Automation Framework Design
Written by Kunal Saxena 9 | P a g e
Framework Design Guidelines
As in SDLC to develop software applications, framework design plays a vital role in building a test automation
framework for an organization. Enterprise applications are large, complex, and ever-evolving, and to keep pace with
the organization’s growth, the design of the test automation framework should conform to the guidelines listed
below.
Should be application-independent and easy to
enhance and maintain
Identify and abstract common functions used across
multiple test scripts
The framework should be based on open technologies to limit maintenance costs. Openness might also allow
cross-tool execution when a suitable API exits or can be developed
The framework should allow easy integration with
other tools and components
The framework should allow end-to-end testing of
the application suite
Keep complex scenarios apart from normal scenarios. E.g. Functional test (separate different modules libraries),
Complex scenarios, Load test, Performance test, Stress test.
Structure scripts with minimal dependencies –
ensuring execution of scripts unattended even on
failures
Encapsulate testers from complexities of the test
framework
Decouple test data from the test scripts
10. White Paper on Automation Framework Design
Written by Kunal Saxena 10 | P a g e
Framework Deign considerations
1. Type of framework
Test Framework helps to organize test suite and improve test efficiency. Different framework focuses on
different objectives.
Data Driven: Focuses on data management. If you need to execute your test cases with different data
than this framework is suitable.
Keyword Driven: its runs test cases based on keywords. If you have many different scenarios for each
features, you can implement this framework
Modular: If you want to separate coding of each feature than this framework is ideal. .It reduces
redundancy and dependency of code.
Hybrid: This is used when someone have to implement more than one framework. This is widely used
framework which can cater benefits of all framework.
2. Data Management
Data is an important factor in framework deign. There could be different types of data set needed by
you. User has to decide what to use for which data.
Data for setting up automation suite
Data for the list of test cases
Data for the test cases, should run in automation suite.
Data for input to a script and expected result
Data for validation.
For configuration values, you can use text file
For validation and input/output result, you can use excel or csv are access.
For test cases, you can use excel or test case management tool
11. White Paper on Automation Framework Design
Written by Kunal Saxena 11 | P a g e
3. Generic Modules
Test automation Framework have many reusable subroutines. If designed and used properly than it will
increase the pace of framework development with minimal errors. These subroutines are defined at
high level and based on the input to the subroutine it will do the necessary action.
e.g. written a function to send mail after every milestone. Same function you can call at every milestone
Log parsing: you wrote a function for parsing log based on search criteria and log path and output value.
check(“SUCCESS”,”Feature3_TC18”) OR check(“REQUEST”,”Feature3_TC18”,3)
4. Rules of Framework
This ensure consistency in your framework development. Defining set of rules needed for scripting and
execution automation framework. It includes, coding, reusable function usage, directory structure to be
followed, exception handling, creation of input data test, naming conventions for test cases and test
scripts. It helps future development easy and streamlined.
5. Integration of tools
Framework Design should allow easy integration of various scripts and tools you are planning to use.
Integration with Email script, test case management tool or defect tracking tool. If automation suite
finds any issue, than it should raise defect in defect tracking tool and update test cases management
tool with the defect ID. And all stake holders should be informed about the action taken by the
automation framework. Integration should not be dependent on the code.
6. Execution Configuration
There are important things should be considered before executing your suite.
1. Feature selection
2. Test case selection for each feature.
3. Log level and Reporting settings
4. Milestone in your regression run
5. Stakeholders email ID
7. Reporting Management
Reporting is important part of framework and considered as a face of a framework. Reporting itself has
many parameters to check.
To whom it should go
Customizable and user-friendly report.
Different level of reporting Summary and Detailed
Format needed by stakeholders (html, excel, pdf)
8. Logging Mechanism
Log generation is critical part of automation. It is vital while debugging issue and fixing the code. Logging
can be done at different level, depending on configuration parameter.
12. White Paper on Automation Framework Design
Written by Kunal Saxena 12 | P a g e
Benefits of Test Automation Framework:
A well designed framework increases the speed and accuracy of the testing process, enables a higher return on
investment (ROI), reduces maintenance and systematically minimizes risks. A well designed framework:
Enhances efficiency in development of
automated test scripts through modularized,
reusable, and maintainable code and data
Provides a structured development methodology
to ensure uniformity of design across multiple
test scripts to reduce dependency on individual
test-case developers
Reduces dependence on subject matter experts
by automatically selecting the test to execute
according to test scenarios and dynamically
refining the test scope according to changes in
the test strategy or conditions of the system
under test
Provides reliable issue detection and efficient
root-cause analysis with minimum human
intervention for the system under test
Runs tests in parallel, runs on test machines in
the grid or cloud, tests across multiple
environments and application versions
simultaneously
Performs multiple levels and types of testing –
functional, regression, load, performance, unit,
integration etc
Eliminates deployment to multiple machines for
testing different versions of an application
Is tool agnostic
Runs in a continuous integration environment.
13. White Paper on Automation Framework Design
Written by Kunal Saxena 13 | P a g e
Summary
In today’s software development environment, test automation framework has become an integral component of
a software testing life cycle. By following the above test automation framework design guidelines, software testers
and software testing companies can immensely benefit by executing successful and maintainable test automation
projects.
In order to keep up with the pace of product development and delivery it is essential to implement an effective, reusable
test automation framework. We cannot expect the traditional capture/replay framework to fill this role for us. Past
experience has shown that capturereplay tools alone will never provide the long-term automation successes that other
more robust test automation strategies can.
A test strategy relying on data driven automation tool scripts is definitely the easiest and quickest to implement if you
have and keep the technical staff to handle it. But it is the hardest of the data driven approaches to maintain and
perpetuate and often leads to long-term failure.
A keyword driven automation framework is probably the hardest and potentially most time-consuming data driven
approach to implement initially. However, this investment is mostly a one-shot deal. Once in place, keyword driven
automation is arguably the easiest of the data driven frameworks to maintain and perpetuate providing the greatest
potential for long-term success. There are also a few commercially available framework which can fulfil few needs.
The most effective test strategies allow us to develop our structured test designs in a format and vocabulary suitable for
both manual and automated testing. This will enable testers to focus on effective test designs unencumbered by the
technical details of the framework used to execute them, while technical automation experts implement and maintain a
reusable automation framework independent of any application that will be tested by it.
What we really want is a framework that can be both keyword driven while also providing enhanced functionality for
data driven scripts. When we integrate these two approaches the results can be very impressive!
The essential guiding principles we should follow when developing our overall test strategy (or evaluating the test
strategy of a tool we wish to consider):
The test design and the test framework are totally separate entities.
The test framework should be application-independent.
The test framework must be easy to expand, maintain, and perpetuate.
The test strategy/design vocabulary should be framework independent.
The test strategy/design should isolate testers from the complexities of the test framework.
14. White Paper on Automation Framework Design
Written by Kunal Saxena 14 | P a g e
About the author
Kunal Saxena, is a QA Expert in Hewlett Packard and has more than 14 years of experience in the field of QA. For the
past 11 years he is involved in Automation. Kunal have expertise in Design, Development and Implementing Automation
Framework from the scratch.
Implemented different types of framework in his carrier. (Record and Playback, Keyword driven, Data driven, Hybrid).
Also acquainted with functional tools like Winrunner, Testpartner, QTP. Kunal is very good in Perl, Python and shell
scripting also has good experience in the excel macros and google docs. Currently working in Cloud Domain in Hewlett
Packard. He is a person with Problem Solving attitude and resolves issue with his Logical Thinking. He has always kept
himself involved in writing scripts for the redundant task and increasing the efficiency of himself and his team.
Acknowledgements
I would like to thank …
My Brother Amit Saxena, who is continuous source of light, knowledge and power for me.
My Wife and Kids for whom I tries to strive best in the world.
My previous manager Karthik Krishnamurthy for having faith in me and giving opportunity to design framework from
scratch.
My current manager Gary Korhonen for giving me time and all necessary help to write this whitepaper.