Performance Testing

1. 1
Selin GUNGOR

2. 
 Introduction to Performance Testing
 Difference between Performance, Load and Stress
Testing
 Why Performance Testing?
 When is it required?
 What should be tested?
 Performance testing process
2
Performance & Load Testing Basics

3. 
 The process of..
 Determining speed
 Determining effectiveness
 Of a..
 Computer
 Network
 Software Program
 Device
3
Introduction to Performance Testing

4. 
 Throughput
 Response Time
 Latency
 Scalability
4
Factors governing performance
Latency + Processing Time = Response Time
Example: The response time in phone conversation is the amount of time it takes
for you to ask a question and get a response back from the person
that you’re talking to.
Example: The processing time in a phone conversation is the amount of time
the person you ask a question takes to ponder the question and speak the answer
(after he hears the question of course).

5. 
 The testing to evaluate the response time (speed),
throughput and utilization of system to execute its
required functions in comparison with different
versions of the same product or a different
competitive product is called Performance Testing.
 Performance testing is done to derive benchmark
numbers for the system.
 Heavy load is not applied to the system
 Tuning is performed until the system under test
achieves the expected levels of performance.
5
Performance Testing- Definition

6. 
6
Difference between
Performance, Load and Stress Testing
Performance
Testing:
A car does 0-
60mph in a certain
number of seconds
under normal
conditions

7. 
7
Load testing
The car can pull a trailer
weighing x - tons before
it can no longer move

8. 
8
Stress Testing
The car continues to work fine
driving 60 mph for two hours,
in 120 degree temperatures,
with only 1 gallon of coolant in
the radiator

9. 
 Loss of revenue
 Amazon states that for every 100ms of latency, they lose
%1 of their sales.
 Google's experiments prove statistically that slowing its
search results by 400ms costs millions of dollars per year
due to lost advertisement revenue.
 Industry experts estimate that revenue of $3 billion has
lost in 2012 because of slow site performance.
 Loss of customers
 Google says they could lose 8 million searches daily if their
search results are 1/2 second slower.
 Flipkart says “57% of shoppers will abandon a site after
waiting 3 seconds for a page load, out of them 80% don’t
return again on same site“
9
Poor Performance Affects

10. 
 Loss productivity
 Reddit, states that their performance focus is on page
load speed. “If we can get 10% more performance, we
immediately see 10% more traffic.”
 Better performing website (speed improvements)
increased productivity by 7-12%.
 Backlog of Work
 On an average 1.2 million people try to book tickets in
the first ten minutes of the Tatkal timeframe, Of that
only 50000 people come out of the website with tickets,
at the success rate of 4.16% only due to service backlog
during peak.
 Media attention and damage to your brand
 Times Of India Ranked IRCTC “1st” out of 8 ‘worst’
Indian government websites in 2012.
10
Continued..

11. 
 Identifies problems early on before they become
costly to resolve.
 Produces better quality, more scalable code.
 Prevents revenue and credibility loss due to poor
Web site performance.
 Enables intelligent planning for future expansion.
 To ensure that the system meets performance
expectations such as response time, throughput etc.
11
Why Performance Testing?

12. 
Planning
the Test
Creating the
Scenerio
Creating
Scripts
Running the
Scenerio
Monitoring
the Scenerio
Analyzing
Test Results
12
Performance Testing Process

13. 
 Why Planning
 Analysing the Application
 Defining Testing Objectives
 Gathering Requirements
 Load Test Checklist
13
Load Test Planning

14. 
 As in any type of system testing, a well-defined test
plan is the first essential step to successful testing.
Planning load testing helps to:
 Build test scenarios that accurately emulate your
working environment
 Understand which resources are required for testing
 Define success criteria in measurable terms
14
Why Planning

15. 
 Load test planning is a three-step process:
 Analyzing the Application
 Defining Testing Objectives
 Gathering Requirements
15
Why Planning

16. 
 Load testing does not require as much knowledge of
the application as functional testing does.
 Load tester should have some operational
knowledge of the application to be tested.
 Load tester should have the idea on how the
application is actually used in production to make an
informed estimate.
 Load tester must know the application architecture
(Client Server, Local Deployment, Live URL),
Platform and Database used.
16
Analyzing the Application

17. 
 Determining and creating performance testing
objectives involves communicating with the team to
establish and update these objectives as the project
advances through milestones
 Performance, Load or Stress testing: Type and scope
of testing should be clear as each type of testing has
different requirements.
 Goal Setting: General load testing objectives should
be defined.
17
Defining Testing Objectives

18. 
 Measuring end-user response time
 Defining optimal hardware configuration
 Checking reliability
 Assist the development team in determining the performance
characteristics for various configuration options
 Ensure that the new production hardware is no slower than the
previous release
 Provide input data for scalability and capacity-planning efforts
 Determine if the application is ready for deployment to production
 Detect bottlenecks to be tuned
18
Defining Testing Objectives

19. 
 Users:Identify all the types of people and processes that can put load on
the application or system.
 Defining the types of primary end users of the application or system such
as purchasers, claims processors, and sales reps
 Add other types of users such as system administrators, managers, and
report readers who use the application or system but are not the primary
users.
 Add types of non-human users such as batch processes, system backups,
bulk data loads and anything else that may add load or consume system
resources.
 Transactions: For each type of user we identified in the previous step,
identify the tasks that the user performs
 Production Environment: Performance and capacity of an application is
significantly affected by the hardware and software components on which it executes.
19
Gathering Requirements

20. 
20
Performance Testing
Manual OR Automation…

21. 
21
Manual Performance Testing Limitations
Coordinator
All of you,
click the
GO button
again
Do you have the testing resources?
• Testing personnel
• Client machines
How do you coordinate and synchronize users?
How do you collect and analyze results?
How do you achieve test repeatability?
Testers
Load Generation
Analysis?
Web server Database
server
System Under Test

22. 
 Manual Testing Limitations
 Expensive, requiring large amounts of both personnel
and machinery.
 Complicated, especially co-ordinating and
synchronising multiple testers
 Involves a high degree of organization, especially to
record and analyse results meaningfully
 Repeatability of the manual tests is limited
22
Continued..

23. 
23
Benefits of Automation
Controller
Analysis
Solves the resource limitations
•Runs many virtual userson a few machines
•Controller manages the virtual users
•Analyze results with graphs and reports
Blazemeter
Load Generation
Web server Database
server
System Under Test

24. 
 Reduces personnel requirements by replacing
human users with virtual users
 Monitors the application performance online,
enabling you to fine-tune your system during test
execution.
 You can choose from a wide variety of graphs and
reports to view the performance data.
 Because the tests are fully automated, you can easily
repeat them as often as you need.
24
Continued..

25. 
 Open Source
 OpenSTA
 Jmeter
 Grinder
 Diesel Test
 TestMaker
 Rubis
25
Tools used for Performance Testing
 Commercial
 LoadRunner
 Silk Performer
 Qengine
 Empirix e-
Load

26. 
DEMO
26

27. 
27
Thank you
Selin GUNGOR