The document discusses different types of automation frameworks including data driven, keyword driven, modular, functional decomposition, and hybrid frameworks. It provides details on the components, workflow, advantages, and disadvantages of a functional decomposition framework. This type of framework divides an application into logical functions that can be reused across multiple tests. Tests are created by calling a series of functions in the required sequence. The framework supports centralized test data management and flexible execution of tests in different environments.

1. An Automation Framework
That Really Works
By
Basivi Reddy Junna
QA Practice Manager and Sr. Automation Architect
Thursday, May 3,2012

2. An Automation Framework That Really Works
- Basivi Reddy Junna
2
Almost anything that you would
ever want to do in programming
can be done with Functions

3. An Automation Framework That Really Works
- Basivi Reddy Junna
3
 Traditional approach to
automation
 Why automation frameworks
at all?
 Disadvantages of
automation frameworks
 Available frameworks of
automation
 Data Driven Framework
 Keyword Driven framework
 Modular Framework
 Functional Decomposition
Framework
• Conceptual overview
• Components
• Workflow
• Driver
 Hybrid Framework
 Q&A
Introduction

4. 4
Create Tests
•Record the test or Program it manually
Enhance Tests
•Synchronization
•Parameterization
•Checkpoints
Execute Tests
•One by one
•Batch mode
Analyze and Report
•Analyze Results
•Report and Manage Defects
Traditional Approach to Automation
An Automation Framework That Really Works
- Basivi Reddy Junna
Create Enhance Execute
Analyze
and Report

5. 5
Traditional Approach to Automation
Continued…
An Automation Framework That Really Works
- Basivi Reddy Junna
Pros
• Best approach for automating few tests
• Quick and easy to implement
Cons
• In the absence of any common pattern or structure, reusability
of automation code is very limited
• Standardization is a challenge
• Difficult to maintain as tests are standalone and each test is a
separate point of maintenance
• As the number of tests increase, coverage analysis becomes
difficult and test suite may quickly go out of control
• Difficult to consolidate and analyze results

6. 6
Standard wish list expected to be fulfilled by automation
An Automation Framework That Really Works
- Basivi Reddy Junna
Test Development
Promote reusability of assets
Provide rapid test building capabilities
Even manual testers should be able to create
automated tests
Test Data Management
Test data management should be a breeze.
Data coverage analysis should be real simple
Use the data generated by one test in others
Handle non-reusable sets of test data without manual
intervention
Test Execution
Execution setup should be centralized.
Even non technical people should be able to setup
and run automated tests.
Run tests in multiple environments concurrently
Run a batch of tests with just click of a button
Schedule tests to run over night
Recover from any errors and continue with other
tests in the batch
Re-run tests that did not run completely in previous
run. Re-run them from the point where they left off.
Re-run tests that have one or more failed
checkpoints or assertions in previous run
Test Results and Reporting
Notify designated people
when tests fail to execute
Create summarized and
detailed reports with
results of execution.
Keep history of results.
Compare results of two or
more test runs
Maintenance
Fewer points of
maintenance
Quickly identify
maintenance points when
application changes
Maintenance should be
super simple

7. 7
Why Use Automation Frameworks At All?
An Automation Framework That Really Works
- Basivi Reddy Junna
Robust
Do not break easily.
Recover from errors for un-
interrupted execution
Centralized Execution Control
and unattended execution.
Scalable
Can easily be up-scaled in terms of
number and size of tests and yet
retain control
Flexible
With little modifications assets can
be leveraged to create new tests
and cover more scenarios
Maintainable
Easy to maintain - Few
maintenance points due code re-
use
Reusable
Assets are reusable
and provide rapid test
building capabilities

8. 8
Difficult to implement
Availability of skilled resources is an issue
Require more up front investment than traditional approach
Over engineering trap
Wrong selection of framework
Get obsessed with framework
Disadvantages of Automation
Frameworks
An Automation Framework That Really Works
- Basivi Reddy Junna

9. 9
Data Driven Framework
Key word Driven Framework
Modular Framework
Functional Decomposition Framework
Hybrid Framework
Available Frameworks of Automation
An Automation Framework That Really Works
- Basivi Reddy Junna

10. 10
Data Driven Tests:
• Parameterized tests are commonly referred to as data driven tests
• Parameterization: Fixed input values for test steps are replaced with variables
and variables are supplied with values during runtime
• Same steps and logic are repeated across multiple test data sets
Data Driven Framework:
• Will have all features of data driven tests
• Logic is controlled by test data. In other words script’s control flow and other
behavior is dictated by test data
• Suitable for large data intensive applications with limited variations in functional
flows and simple to moderately complex business rules
• Usually made up of few but very large test scripts whose behavior is controlled
by data populated in test data repository
• Test maintenance is mainly taken care by updating test data
Data Driven Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

11. 11
Data Driven Framework - Example
An Automation Framework That Really Works
- Basivi Reddy Junna
Test
Id TestProd_Id TestAddress_Id TestPmt_Id
1 2 1 2
2 3 2 1
Id Product_Id Action
1 10 Search
2 3 Add
2 4 Remove
Tests
Test_Products
Id Product_Name
1 Kindle Fire
2 iPod Charger
3 Google Nexus
4 Blackberry
Products
Id Address_Id Action
1 3 Add
2 1 Edit
Test_Addresses
Id Address City
1 … Chicago
2 … Chicago
3 70 York… Lombard
Addresses
Input
Test Id

12. 12
Pros
• Limited numbers of tests to maintain
• Script maintenance is largely handled through test data
updates
Cons
• Complex to implement: Test data repository may become
as complex as the application data model itself
• Not suitable for applications with heavy variations in
functionalities and flows
• Tests scripts are too big to fail
Data Driven Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

13. 13
Testing activities can be divided into basic actions called
keywords – Click, Select, Enter etc
Keyword(s) => Function(s)
Test scripts are written using key words
Keywords (functions) are invoked by providing as inputs -
data and application objects
Automated scripts are automatically generated behind the
scene, while you are creating manual scripts using keywords
Keyword Driven Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

14. 14
Pros:
• You create manual tests and have automated scripts generated behind the scene
automatically
• Even manual testers can create automated tests without any knowledge or
experience of automation.
• Two birds with one shot - Produces highly structured manual tests so you can
test manually in case if automation is NOT working for any reason
Cons:
• Debugging is very tedious
• Heavy upfront investment
• Suitable for small and medium applications only
• Difficult to handle dynamic objects
• Grouping steps and controlling iterations is extremely difficult
• Complex test flows cannot be specified using keywords
Keyword Driven Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

15. 15
Divide the applications into logically identifiable sections called modules
Develop automation scripts for the modules
Modules characteristics:
• Re-usable
• Data Driven – Modules behave according to data
• Perform multiple functions and validations
• Handle errors
• Report results
Actual tests are created by calling the reusable modules in a required
sequence
Can be treated as an additional layer of abstraction on the top of functional
decompositions framework
Modular Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

16. 16
Pros:
•Large number of tests are created by relatively less number of modules. Hence, fewer
points of maintenance
•Provides much higher level of abstraction than Keyword driven or functional
decomposition frameworks making it easy for users to understand the test flows.
Cons:
•Best described as half data driven and half functional decomposition framework. It has
disadvantages of both the frameworks but not the advantages of Functional
Decomposition Framework
•Restricted maneuverability - Not suitable for application with wide variety of
functionalities, twists and turns(flows). Integration of modules into actual tests and
still conforming to navigational rules of the application is difficult to achieve
•Each module performs multiple functions. Selective execution of functions is difficult
to achieve
•Combination of higher level of abstraction and underlying data driven framework make
it difficult to understand the test data being passed to and consumed by tests
Modular Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

17. 17
Divide the application into logically identifiable functions
Create functions using scripting language of the tool
Make the functions data driven using parameters
Create actual tests using the functions
Suitable for medium to highly complex applications
Can handle wide variety of functionalities
Capable of being data driven
Functional Decomposition Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

18. 18
Functional Decomposition Framework: Example Test 1
An Automation Framework That Really Works
- Basivi Reddy Junna
Launch Application
Login
Search Products
Add To Cart
Edit Cart
Checkout
Shipping Address
Payment Method
Place Order
Logout
Test 1Launch Application
Login
Search Products
Add To Cart
Edit Cart
Checkout
Shipping Address
Payment Method
Place Order
Logout
Launch Application
Login
Search Products
Add To Cart
Edit Cart
Checkout
Select Shipping Address
Select Payment Method
Place Order
Logout
URL
UID and PWD
Criteria & Product Info

19. 19
Functional Decomposition Framework: Example Test 2
An Automation Framework That Really Works
- Basivi Reddy Junna
Launch Application
Login
Search Products
Add To Cart
Edit Cart
Checkout
Shipping Address
Payment Method
Place Order
Logout
Test 2Launch Application
Login
Search Products
Add To Cart
Edit Cart
Checkout
Shipping Address
Payment Method
Place Order
Logout
Launch Application
Login
Search Products
Add To CartEdit Cart
Checkout
Select Shipping Address
Select Payment Method
Place Order
Logout
URL
UID and PWD
Criteria & Product Info
Quantity
Capture Order Number

20. 20
Test Data Repository
Business
Functions
Common Functions
Generic
Functions
Framework
Functions
Functions
for
Web
Applications
Functions
JAVA
Applications
Test1
Function 1
Function 2
…
Function n
Test2
Function 1
Function 5
…
Function n
Test3
Function 6
Function 10
…
Function n
Driver
Test1
Test2
Test3
…
Test n
Functional Decomposition Framework – Conceptual
Overview
Functions
for
Mainframe
Applications
Etc…

21. 21
Business Functions: These are re-usable functions that perform
specific tasks by reading data from test data repository. All
functions are data driven. These functions are specific to the
application under test. Using common functions (explained later)
will significantly accelerate the development of business functions
Characteristics:
• Validate the inputs
• Verify execution status of previous functions in the test
• Verify start status
• Implement synchronization and perform validations
• Recover from errors
• Report results
Components of the Functional Decomposition
Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

22. 22
Common Functions: These functions are independent of any business domain/application and
are commonly used. There are three sub categories of common functions: 1. Generic functions
(string functions, date functions etc) 2. Framework functions (Reporting, configuration, error
handling functions etc) 3. Environment specific functions such web functions, Power builder
functions etc. These functions are mostly called by business functions
Automated Tests: Automated Tests are created by simply calling a series of re-usable business
functions in required sequence using parameters to read appropriate data from test data
repository.
Test Data Repository: A place to store , organize and manipulate test data. It is usually made
up of any one or combination of flat files, excel workbooks, relational databases. This is also
where results of the automated test execution will be stored. The idea is to have a centralized
location for controlling test data and for reporting.
Driver: It is a batch test which calls other automated tests. It will have error handling/recovery
capability where if one test in the batch fails to run due to any error, that test will be ignored
and execution control will move to next test in the batch.
Components of the Functional Decomposition
Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

23. 23
Appendix A - Structure of Business Functions
An Automation Framework That Really Works
- Basivi Reddy Junna
<Function Name>(Arg1,Arg2,...)
‘Verify if the test failed in previous function
If STATUS <>"PASS" Then
Exit Function
End If
'Validate parameters passed to the function
'for example you verify if they are null. or in some cases verify if they are numbers or not etc
'If IsNull(Arg1) or IsNull(Arg2)... Then
'Perform appropriate action based on whether any argument is mandatory for the function to
continue
'Report about missing or incorrect argument and Exit Action
'End If
Validate Start Status
'Describe in what state the application should be (screen name and short description) when this function starts
'Create Expected Result (ER) strings for all conditions this function seeks to verify
'Expected Result1=“…"
'Expected Result2=“…“
write code to verify each Expected Results
‘Wherever necessary implement synchronization, parameterization, Error handling, Insert output values and
checkpoints
'Capture Actual Result for each Expected Result ; Compare Expected and Actual Result
'Report Pass if they match or Fail if they don't along with expected and actual results
‘While performing above steps recover from errors if any and also call Common functions as needed
<End Function>

24. 24
Example Setup
An Automation Framework That Really Works
- Basivi Reddy Junna
Setup file - Setup IDs sheet
Setup file - Setup Details sheet

25. 25
Example Setup
An Automation Framework That Really Works
- Basivi Reddy Junna
Setup file – Run Setup IDs sheet Setup file – Tests Sheet
Setup file – Run Tests Sheet

26. 26
Sample functions for a fictitious E-Commerce
Application
An Automation Framework That Really Works
- Basivi Reddy Junna

27. 27
Functional Decomposition Framework Sample Test and
Results
An Automation Framework That Really Works
- Basivi Reddy Junna

28. 28
Workflow of the Functional Decomposition Framework
An Automation Framework That Really Works
- Basivi Reddy Junna

29. 29
Functional Decomposition Framework - Driver

30. 30
Functional Decomposition Framework – Run Test

31. 31
Workflow of the Functional Decomposition Framework –
Test Data Repository Considerations
An Automation Framework That Really Works
- Basivi Reddy Junna
Systems/
Applications
With UI
No UI
(Headless)

32. 32
Workflow of the Functional Decomposition Framework –
Test Data Repository Considerations
An Automation Framework That Really Works
- Basivi Reddy Junna
UI Based Applications
Free form text
type
Text box
Combo box
Text area
Single Select Type
Single select list view
Single select tree view
Radio button group
Dropdown list
Multi select type
Multi select list
view
Multi select tree
view
Checkbox

33. 33
Workflow of the Functional Decomposition Framework –
Test Data Repository Considerations
An Automation Framework That Really Works
- Basivi Reddy Junna
Non UI Based /Faceless
Applications
Single values
Set of values

34. 34
Workflow of the Functional Decomposition Framework –
Test Data Repository Design Recommendations
An Automation Framework That Really Works
- Basivi Reddy Junna
Data
Tables
Static Data Tables
Contain static data . Examples: list of states, product categories like furniture,
electronics etc
1. For each set of static data create a separate data table and assign unique Id
for each row
2. Use these IDs delimited by ; inside dynamic tables if the function needs to
handle multi select type fields or a set of values as input
3. Use the static data values directly inside the dynamic tables if the need is to
handle single select type fields or a single value as input
Dynamic Data Tables
Made up of:
1. Free form text, numbers and dates
2. Different permutations and combinations of data from static tables

35. 35
Test Data Repository Design - Sample
An Automation Framework That Really Works
- Basivi Reddy Junna
Table to store values captured from Application

36. 36
Pros:
• More flexible than other frameworks
• Better maneuverability means test flows involving complex twists and turns
can be created easily
• Better control over test data consumed in your tests
• The building blocks (functions) are not too granular, not too high level and
present fewer points maintenance
• One of the best frameworks suitable for automation in agile environments
Cons:
• Becomes complex when the test gets big - as the test size increases, so will
the number of function calls and failure points, debug points
• When a test becomes really big, test flow is not as clearly evident as in the
of modular framework
Functional Decomposition Framework
An Automation Framework That Really Works
Basivi Reddy Junna

37. 37
Hybrid Framework(s)
An Automation Framework That Really Works
- Basivi Reddy Junna
Keyword + Functional Decomposition +
Modular Framework
Keyword + Functional Decomposition
Framework
Functional Decomposition + Modular
Framework
Keyword + Modular Framework

38. Do not copy or distribute. Proprietary and
Confidential to Scalar USA Inc.
38
Q&A
???
An Automation Framework That Really Works
- Basivi Reddy Junna

39. Do not copy or distribute. Proprietary and
Confidential to Scalar USA Inc.
39
Thank you!!!
bjunna@scalarusa.com
An Automation Framework That Really Works
- Basivi Reddy Junna