1.
2/1/2016
1
GET MAX FROM YOUR AUTOMATED TESTS
Nitin Mukhija
- Director Of Engineering In Test
PayU India Pvt. Ltd.

2.
2/1/2016
2
FYI

3.
2/1/2016
3
Hopefully this will not happen, so lets
get started…

4.
Why Automate?
• Manual is slow
• Faster feedback on Continuous Integration
– 24*7 testing
• Repetitive manual regression tests are
boring and make testers lose focus
• Manual testers focus on testing new
features
• Allows for faster upgrades and refactoring
• Consistency and Documentation
• Higher test coverage – multiple data
combinations
2/1/2016
4

5.
Let’s see an example of
poorly written automation…
2/1/2016
5

6.
Automation Design
• Example of a bad code Hardcoded
Browser
Hardcoded
Environment
Hardcoded
Timeout
2/1/2016
6

7.
Automation Design
• Example of a bad code
Hardcoded
Object
Identifiers
Hardcoded
Test Data
Hardcoded
Selenium
Constructs
Repeated
Business
Logic
2/1/2016
7

8.
Automation Design
• Non-Actionable Report
– High Failure Investigation Cost
Manually run
again to repro.
Can’t log bugs
from report
No
Screenshot /
Page HTML
Insufficient
Logging
2/1/2016
8

9.
Unplanned Automation
2/1/2016
9

10.
Let’s see the things you can do to get MAX from
your automation…
2/1/2016
10

11.
Tip #1:- Choose Right Design
• Example – Layered Architecture
• Hotel Book Web App
2/1/2016
11

12.
Layered Architecture
• Automation Libs
• Framework Core
• Page Objects
• Business Logic Helpers
• Test Cases
2/1/2016
12

13.
Page Object
Action On
Page
Mandatory to
Return next
Page after click
UI Elements
of Page
Page Name
2/1/2016 13

14.
Layered Architecture
Sample layered design:-
• Automation Libs
• Framework Core
• Page Objects
• Business Logic Helpers
• Test Cases
2/1/2016
14

15.
Business Logic Helpers
Combine
Page
Actions
to
build
business
Logic
Login
Search
Book
2/1/2016
15

16.
Layered Architecture
• Automation Libs
• Framework Core
• Page Objects
• Business Logic Helpers
• Test Cases
2/1/2016
16

17.
Neat Test Case
Extensively use
Business Logic
Helpers
Easy to read &
interpret manual
test case
TC
Description
2/1/2016
17

18.
Layered Architecture
• Automation Libs
• Framework Core
• Page Objects
• Business Logic Helpers
• Test Cases
2/1/2016
18

19.
Automation Libs
• Selenium, QTP, etc.
• All external third party jars
2/1/2016
19
Framework Core
• Wrapper around various
external libraries
Class Description
Log Log reporting
Helper Various utility methods
DataBase Methods to interact with database
Popup Methods to handle popups, alerts on the page
TestData
Reader
Methods to read from test data sheet
Browser Methods to do operations on your Browser/Page
Element Methods to do operations on Elements of the Page

20.
Tip #1:- Right Design- Benefits
• Reduce Maintenance cost
• Business logic at one place
• Reusable code
• Localized changes – Data and Object Repo
• Easy to understand test cases
• Easy to change underlying automation lib
2/1/2016
20

21.
Tip #2:- Configurable
• Don’t hardcode – Load at runtime
– Environment URL’s
– Browser Name
– Platform
– ObjectWaitTime
– Database credentials
– Any other param which drives the execution
2/1/2016
21

22.
Tip #3:- Data Driven
• Run same test case with different data combinations
Read login
credentials
from Excel
Test Data Excel
Row Number
2/1/2016
22

23.
Tip #4:- Parallel Execution
• Write non-interfering atomic test
cases
• Standard Win7 can run 12 parallel
browsers – execution time
reduced to 1/12th
Faster Execution =
Faster Feedback
2/1/2016
23

24.
2/1/2016 24

25.
Tip #5:- Rich Reports
Hyperlinks of
Page Screenshot
& HTML
Uniform logging
of TC steps in
Green/Red/Black
logs
Failure Page
URL
Log bugs by just
reading logs,
even manual
tester can do!
TC
Description
2/1/2016
25

26.
Allure Reporting Failure
Grouping
2/1/2016
26

27.
Custom Listeners
• Logging of unhandled exceptions
• Cleanup tasks
• Soft Asserts
2/1/2016
27

28.
Tip #6:- Customized email results
• Host Test Results on a web server - Reports
history
• Email Results - Customized Subject
Pass
Percentage
Area Owner
2/1/2016
28

29.
Consolidated Results
2/1/2016
29

30.
2/1/2016
30
Tip #7:- Measure Automation Accuracy
# of test cases failed due to Dev code issue
Vs
# of test cases failed due to automation code issue

31.
Tip #8:- Measure Code Coverage
Sample PHP
Code
Coverage
Report
2/1/2016
31

32.
Tip #9:- Automated Deployments
• Using Docker - Build production like, testing
environment on the fly
• Jenkins job to:-
– Deploy application code automatically
– Run Automation
• A Step towards Continuous Delivery
2/1/2016
32

33.
TestLink - TCM
Test Cases
Test Plans
Test Builds
Reports
Execution Engine
Fetch Git Code & Compile
TestLink Integration Module
Get Test Cases To Run
Launch & Drive Execution
Retrieve
Results/Exceptions
Push Test Results & Email
(ReportNG)
Layered Automation
Suite
Test Cases (TestNG)
Business Logic
Page Objects
Framework Core
Selenium Webdriver
Tip #10:- Continuous Integration
2/1/2016
33

34.
2/1/2016
34
That’s strange, not even a single
Oxygen mask dropped!!

35.
Questions & Answers
2/1/2016
35

36.
Thank You!!!
https://in.linkedin.com/in/nitinmukhija
@nitin_mukhija
2/1/2016
36