The document discusses factors to consider when selecting a test automation framework. It describes how there are many options for frameworks available and outlines important criteria to evaluate, such as flexibility, ability to support different applications and interfaces, tool and language independence, parallel execution, and design patterns. The presentation provides examples of different types of frameworks and discusses strategies for building frameworks that can scale and evolve with changing needs.

1. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Agile Testing, Test
Automation & BDD
OCTOBER 25, 2016
SINGAPORE
ORGANIZED BY TESTINGMIND
1

2. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
About Myself
Quick Bio: “Test Automation Consultant having more than 11 years of experience in Software
Automated Testing space.”
Twitter: @sahajamait
Github: https://github.com/sahajamit
P.S. : All the opinions given in this talk are completely personal and has nothing to do with my
employer.
2

3. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Talk Abstract:
“What goes into the selection of right Test Automation Framework for your
application? the Application type (Mobile, Desktop, Web), the scripting
language(Java, Ruby, Python) or the tools (Selenium, Appium, UFT). No,
there are many more factors to consider before finalising your “ideal”
automation framework and if you get this decision wrong then it can have
a cascading effect to your entire test strategy. In this rapidly changing
Agile environment, the automation framework should be extremely
flexible and agnostic of external factors like tools and languages. In this
talk we will be covering this subject more deeply with some real life
examples.”
3

4. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Test Automation Frameworks – Assumptions,
Concepts and Tools
4
By : Amit Rawat

5. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Who can help you to build your Test
Automation Framework ??
Some Assumptions:
● Why to build (already so many open-source frameworks available)
● I will hire a Selenium/Automation Architect
● I will buy a Licensed tool
5

6. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Who can help you to build your Test
Automation Framework ??
Reality:
6

7. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
How easy is to design your Automation
Framework
Google yield more than million results for the query “Test
Automation Framework”
7

8. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Test Automation Frameworks – Assumptions,
Concepts and Tools
What is a Test Automation Framework?
It is a supporting structure or a harness that provides a conducive environment
to execute and maintain the automation scripts effectively. It defines a single
standard of doing things which can result in highly-reusable automation
scripts and that can lead to very low cost of maintenance.
8

9. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Some common types of Automation
Frameworks
▶ Linear
▶ Test Script Modularity
▶ Keyword-driven
▶ Behavior-driven(BDD)
▶ Hybrid
▶ Agile Automation Framework
9

10. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Keyword Driven Approach
(Script Less Automation)
10

11. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Data Driven Approach
11

12. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Behavior Driven Approach (BDD) :
Functionality Driven
12

13. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Behavior Driven Approach (BDD) :
Data Driven
13

14. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Automating the UI or Automating the
FUnctionality ??
UI/UX Functionality/Workflow
14

15. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Same Test Across Platforms
15

16. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Test Automation Pyramid - Current State
16
Manual Tests

17. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Test Automation Pyramid :
Ideal State
Image Credit:
http://martinfowler.com/bliki/TestPyramid.html
17
Manual Tests
X-UNIT
Tests
API/Service
UI

18. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Test Automation Pyramid :
Achievable State
18
UI Tests
API/Service
Tests
Manual
Tests

19. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Automation Paradigm: UI Interaction
19

20. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
New Paradigm: Automation Robots
(Tapster)
20

21. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Things to consider before designing your
Framework
▶ Progressive Test Automation / Agile Test Automation
▶ Types of testing to be supported: Functional and Non-Functional
▶ Parallel execution
▶ Application Interfaces to be supported: Mobile, Web, Desktop, APIs.
▶ Operating System
▶ Reporting : Screenshots, Videos, Data, Logs
▶ Framework Interface: Web, Excel, Feature Files
▶ Tool / Language Independence
▶ Run via CI
▶ Design Patterns
▶ Automation Environment
21

22. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Parallel Execution & Automation
Environment
▶ Headless Execution
▶ Third party Automation Cloud (SauceLabs, BrowserStack,
AWS)
▶ Virtualization (VMs, Vagrant, Docker)
▶ Creating Test Environment on the fly (Docker, Kubernetes)
22

23. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Tool Agnostic Framework
▶ Abstraction on the tool specific commands
▶ Leveraging multiple tools beneath the framework layer for different types of
testing
▶ Web UI Testing - Selenium,AutoIT
▶ API Testing - RestAssured, SoapUI, Postman
▶ Data Reconciliation - google-diff-match,PDFBox,BeyondCompare
▶ Mobile App Testing - Appium, Calabash
▶ Network Testing - BMP
▶ Responsive Design Testing - Galen Framework
▶ Analytics Testing - Fiddler, CharlesProxy
▶ Security Testing - Burp
▶ Mainframe Testing - Jagacy, IBM PCOM
▶ Expose Domain Specific Language(DSL)
23

24. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Locator Strategy
▶ Multiple Locators
▶ Statistical Technique
▶ Artificial Intelligence
24

25. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Test Automation Design Patterns
▶ Page Object Model
▶ Page Factory Pattern
▶ Loadable component
▶ Builder
▶ Singleton
25

26. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Test Reporting
▶ Allure
▶ Masterthought
▶ Serenity
26

27. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Test Reporting
27

28. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Test Automation Framework Layers
28

29. Agile Testing, Test Automation & BDD, Singapore www.testingmind.com October 25, 2016
Q & A
29