2. AGENDA
What/Why is Performance Testing?
Types of Performance Testing
Performance metrics
Factors that affect software performance
Skills required for performance test
Technologies where performance test is carried out
Performance Testing process
Performance Test Tools
Performance Test Scenario
Do’s & Don'ts
3. WHAT IS PERFORMANCE TESTING
Performance testing is conducted to evaluate the compliance of a system or component with
specified performance requirements.
4. FUNCTIONAL TESTING VS PERFORMANCE
TESTING
Functional Testing Performance Testing
• To verify the accuracy of the software with
definite inputs
• To validate the behavior of the system at various
load conditions performance testing is done.
• This testing can be done manually or automated • It gives the best result if automated
• One user performs all the operations • Several user performs desired operations
• Customer, Tester and Development involvement
is required
• Customer, Tester, Developer, DBA and N/W
management team
• Production sized test environment is not
necessary, and H/W requirements are minimal
• Requires close to production test environment
and several H/W facilities to populate the load
5. WHY PERFORMANCE TESTING IS SO
CRITICAL?
There is a popular saying “On the Internet, the competition is only a mouse click away”.
Today applications are not designed for single user but for multiple users.
It is a fundamental fact of the Internet that poor performance leads to dissatisfied users,
that dissatisfied users will abandon that web-site.
6. WHY PERFORMANCE TESTING IS
SO CRITICAL?
SCALABILITY
How many users…
before it gets “slow”?
before it stops working?
Will it sustain?
do I expect today?
do I expect before the next upgrade?
How much data can it hold?
Database capacity
File Server capacity
Back-up Server capacity
Data growth rates
STABILITY (Should be part of stress testing)
What happens if…
there are more users than we expect?
all the users do the same thing?
a user gets disconnected?
there is a Denial of Service Attack?
the web server goes down?
we get too many orders for the same thing?
CONFIDENCE
If you know what the performance is…
you can assess risk
you can make informed decisions
you can plan for the future
you can sleep the night before go-live day
7. WHY LOAD TESTING IS SO CRITICAL?
To choose the right architecture
• hardware
• software
To deploy with confidence
• allows development / implementation teams to get system right the first time
• proven functionality / performance under real world conditions
8. TYPES OF PERFORMANCETESTING
Load Test
Load test is conducted to evaluate a system or component for their performance requirements with
expected load in the system. In general Load Testing tools are used to simulate large number of users
known as virtual users VUsers.
9. TYPES OF PERFORMANCE TESTING
CONT..
Stress Test
Stress test is conducted to evaluate a system or component at or beyond the limits of specified
performance requirements to determine the load under which it fails and how.
12. TYPES OF PERFORMANCETESTING
CONT..
Peak Test/Spike Test
Endurance testing is conducted to analyze the system’s behavior when exposed to intensity peaks mixed
with regular load, showing the recovery following the increased load.
14. PERFORMANCETESTING METRICS
These are the indicators that are used to measure the performance requirements.
Ex: Response time, CPU usage, Memory page faults per second, Data packets sent per second
16. SKILLS OF A PERFORMANCETESTER
•Identify & understand performance requirements
•Knowledge in application domain
•Knowledge in computer technologies (Web, TCP/IP)
•Ability to use Load Testing tools
•Ability to analyze test results and pin-point the problems
18. COMMON LOADTEST OBJECTIVES
Application response time
– How long does it take to complete a
task?
Configuration sizing
– Which configuration provides the best
performance level?
Acceptance
– Is the system stable enough to go into
production?
Regression
– Does the new version of the software
adversely affect response time?
Reliability
– How stable is the system under a
heavy work load?
Bottleneck identification
– What is the cause of degradation in
performance?
Product evaluation
– What is the best server for 100 users?
19. WHY MANUAL LOADTESTING PROBLEMATIC?
- Do you have the testing resources? (personnel/machines)
- How do you coordinate and synchronize multiple users?
- How do you achieve test repeatability?
- How do you collect and analyze results?
21. PERFORMANCETESTING PROCESS
Scripting
•It is difficult to perform load test manually
•Always a tool will be used
•In general user actions will be recorded
•This is known as generating VUser scripts
•It may require to modify the scripts after recording
26. SELECTING A PERFORMANCETOOL
• Customer preference tool
• Availability of license within customer machine
• Availability of test environment
• Additional protocol support
• License cost
• Efficiency of tool
• User options for testing
• Vendor support
28. LoadRunner EXPERT WORKFLOW SUMMARY
4.1 Verify that Vusers run
concurrently
4.2 Isolate top time
transactions
4.3 Run full load test
4.4 Isolate hardware and
software limitations
1.1 Get system usage
information
1.2 Document business
process properties
1.3 Determine which fields
to parameterize
1.4 Establish which data to
use
Tune
System Based
on Analysis
Analyze
System Under
Load
Phase 5
Run
Scenarios
Phase 4
Create
Scenarios
Phase 3
Create Web
Virtual Users
Phase 2Phase 1
Plan Load Test
2.1 Record user actions
2.2 Add LoadRunner
transactions
2.3 Parameterize data
2.4 Add verification
checks
2.5 Verify correct
execution
3.1 Define hosts
3.2 Connect hosts
3.3 Define scripts
3.4 Add Vusers
30. PERFORMANCE TEST SCENARIO
1
It is expected that company X web-site is accessed by 1000 of their customers during day time on
average per hour. Their customer base consists of corporate, resellers and domestic consumers.
They expect that the website home page to be loaded within 1 second.
How do you plan the performance Test?
31. PERFORMANCE TEST SCENARIOS
The list of scenarios which you should include in your performance test,
Most Frequently Accessed Scenarios: Application scenarios which are mostly accessed by end users. As such
scenarios will affect maximum users they must be included in load test. For example, browsing product catalog in an E-
commerce web application.
Business Critical Scenarios: Application scenarios where application core features exists. For example, purchasing a
product is a business critical scenario in an E-commerce application.
32. PERFORMANCE TEST SCENARIOS
CONT..
Resource Intensive Scenarios: Such scenarios which are expected to consume more system resources as compared to
others. For example, order placement will be most resource intensive scenario in an E-commerce web application.
Contractually Obligated Scenarios: Application scenarios for which company has contracted to provide hassle free
services. These scenarios might not be used very frequently but they can create huge business loss in case of failure.
For example, company claims its home page loads within 3 seconds.
33. PERFORMANCE TEST SCENARIOS
CONT..
Stakeholders Concerning Scenarios: Stakeholders could be more concerned on AUT new features impact on its
overall performance.
Time Dependent Frequently Accessed Scenarios: Time dependent scenarios which are executed very frequently
but on certain occasions only. For example, viewing monthly pay roll slip on an online payroll application.
Technology Specific Scenarios: These are the scenarios which are specific to AUT selected technology. For example,
uploading a file through FTP server could be an example of technology specific scenario.
34. DO’S & DON'TS
Do’s
Do test early and do test often
Do establish what is and is not
acceptable performance for
your application
Do test from the user’s perspective
- it’s the only one that counts
Do baseline and compare your
findings
Do monitor your system while you
test
Do test whenever there’s a change
in your site’s content, code or
infrastructure
Don’ts
Don’t wait until the last minute to
test
Don’t depend on your customers to
do your testing for you
Don’t test under unrealistic
conditions
Don’t forget that increases in table
sizes, disk usage and network
traffic will degrade your
application’s performance over
time
Don’t be so quick to throw hardware
at the problems you turn up - it
doesn’t always help
Discipline concerned with detecting and reporting the current behavior of the software system.
Detect(What) Diagnose(Why) Resolve----Not Resolved-->Detect
Non- Fnctional testing to determine the system responsiveness, stability, reliability & Scalability
Performance of ur application = performance of ur business
weB APP : Long user response time *Memory leaks *High CPU usage *Too many open connections *Length queues for requests *Too many table scans of database *Database deadlocks *Erroneous data returned *HTTP errors Pages not available
Db :Insufficient Indexing Fragmented Database Out-of-date Statistics Faulty Application Design
Web Server : High CPU Usage • Poor Server Design • Memory Problems
App S P: Poor Database Tuning • Poor Cache Management • Poor Session Management • Poor Security Design
NW: Firewall throughput * internet access troughput * Load balances, gateways,routers
