The document provides an overview of performance testing, including: - Defining performance testing and comparing it to functional testing - Explaining why performance testing is critical to evaluate a system's scalability, stability, and ability to meet user expectations - Describing common types of performance testing like load, stress, scalability, and endurance testing - Identifying key performance metrics and factors that affect software performance - Outlining the performance testing process from planning to scripting, testing, and result analysis - Introducing common performance testing tools and methodologies - Providing examples of performance test scenarios and best practices for performance testing

1. PerformanceTesting
Role BasedTraining - QAE
Chalana Kahandawala
Software QA Engineer –Test Automation & Performance
24/07/2017

2. AGENDA
 What/Why is Performance Testing?
 Types of Performance Testing
 Performance metrics
 Factors that affect software performance
 Skills required for performance test
 Technologies where performance test is carried out
 Performance Testing process
 Performance Test Tools
 Performance Test Scenario
 Do’s & Don'ts

3. WHAT IS PERFORMANCE TESTING
Performance testing is conducted to evaluate the compliance of a system or component with
specified performance requirements.

4. FUNCTIONAL TESTING VS PERFORMANCE
TESTING
Functional Testing Performance Testing
• To verify the accuracy of the software with
definite inputs
• To validate the behavior of the system at various
load conditions performance testing is done.
• This testing can be done manually or automated • It gives the best result if automated
• One user performs all the operations • Several user performs desired operations
• Customer, Tester and Development involvement
is required
• Customer, Tester, Developer, DBA and N/W
management team
• Production sized test environment is not
necessary, and H/W requirements are minimal
• Requires close to production test environment
and several H/W facilities to populate the load

5. WHY PERFORMANCE TESTING IS SO
CRITICAL?
There is a popular saying “On the Internet, the competition is only a mouse click away”.
Today applications are not designed for single user but for multiple users.
It is a fundamental fact of the Internet that poor performance leads to dissatisfied users,
that dissatisfied users will abandon that web-site.

6. WHY PERFORMANCE TESTING IS
SO CRITICAL?
SCALABILITY
 How many users…
 before it gets “slow”?
 before it stops working?
 Will it sustain?
 do I expect today?
 do I expect before the next upgrade?
 How much data can it hold?
 Database capacity
 File Server capacity
 Back-up Server capacity
 Data growth rates
STABILITY (Should be part of stress testing)
 What happens if…
 there are more users than we expect?
 all the users do the same thing?
 a user gets disconnected?
 there is a Denial of Service Attack?
 the web server goes down?
 we get too many orders for the same thing?
CONFIDENCE
 If you know what the performance is…
 you can assess risk
 you can make informed decisions
 you can plan for the future
 you can sleep the night before go-live day

7. WHY LOAD TESTING IS SO CRITICAL?
To choose the right architecture
• hardware
• software
To deploy with confidence
• allows development / implementation teams to get system right the first time
• proven functionality / performance under real world conditions

8. TYPES OF PERFORMANCETESTING
Load Test
Load test is conducted to evaluate a system or component for their performance requirements with
expected load in the system. In general Load Testing tools are used to simulate large number of users
known as virtual users VUsers.

9. TYPES OF PERFORMANCE TESTING
CONT..
Stress Test
Stress test is conducted to evaluate a system or component at or beyond the limits of specified
performance requirements to determine the load under which it fails and how.

10. TYPES OF PERFORMANCETESTING
CONT..
Scalability Test
Scalability test is used to evaluate the effect of adding additional hardware/software to distribute work
among system components.

11. TYPES OF PERFORMANCETESTING
CONT..
Endurance Test
Endurance testing is conducted to evaluate the behavior of the system when a significant workload is
given continuously

12. TYPES OF PERFORMANCETESTING
CONT..
Peak Test/Spike Test
Endurance testing is conducted to analyze the system’s behavior when exposed to intensity peaks mixed
with regular load, showing the recovery following the increased load.

13. TYPES OF PERFORMANCETESTING
CONT..

14. PERFORMANCETESTING METRICS
These are the indicators that are used to measure the performance requirements.
Ex: Response time, CPU usage, Memory page faults per second, Data packets sent per second

15. WHAT AFFECT PERFORMANCE
•Network bandwidth/quality
•Processor speed
•Memory availability
•Bandwidth of a Web-Server, DB Server

16. SKILLS OF A PERFORMANCETESTER
•Identify & understand performance requirements
•Knowledge in application domain
•Knowledge in computer technologies (Web, TCP/IP)
•Ability to use Load Testing tools
•Ability to analyze test results and pin-point the problems

17. TECHNOLOGIES WHICH PERFORMANCE
TESTING USE
•HTTP/HTTPS, FTP, SMTP, POP
•SQL Server, Oracle, DB2
•WAP
•SOAP
•PeopleSoft, Siebel, SAP

18. COMMON LOADTEST OBJECTIVES
Application response time
– How long does it take to complete a
task?
Configuration sizing
– Which configuration provides the best
performance level?
Acceptance
– Is the system stable enough to go into
production?
Regression
– Does the new version of the software
adversely affect response time?
Reliability
– How stable is the system under a
heavy work load?
Bottleneck identification
– What is the cause of degradation in
performance?
Product evaluation
– What is the best server for 100 users?

19. WHY MANUAL LOADTESTING PROBLEMATIC?
- Do you have the testing resources? (personnel/machines)
- How do you coordinate and synchronize multiple users?
- How do you achieve test repeatability?
- How do you collect and analyze results?

20. PERFORMANCETESTING PROCESS
Planning
•Identifying performance requirements
•Setting expectations & measurements
•Selecting test types
•Preparing test plan
•Modeling the load

21. PERFORMANCETESTING PROCESS
Scripting
•It is difficult to perform load test manually
•Always a tool will be used
•In general user actions will be recorded
•This is known as generating VUser scripts
•It may require to modify the scripts after recording

22. PERFORMANCETESTING PROCESS
Testing
•Preparing test lab
•Execute the test according to plan
•Store the results

23. PERFORMANCETESTING PROCESS
Result Analysis
•Analyze failed results
•Reasons not to achieve expected results

24. PERFORMANCE TESTTOOLS
•Load Runner 8.0
•Silk Performer 6.5
•JMeter (Web only)
•Rational Performance Tester 2003
•Test Complete 3.0 (Web only)

25. COMMON ARCHITECTURE OF ATOOL

26. SELECTING A PERFORMANCETOOL
• Customer preference tool
• Availability of license within customer machine
• Availability of test environment
• Additional protocol support
• License cost
• Efficiency of tool
• User options for testing
• Vendor support

27. PERFORMANCETESTING METHODOLOGY
Requirement
Analysis
Test
Design
Test
Plan
Test
Execution
Analysis of
Results
Executive
Summary Report
Start of
Project
Project
Closure
Preliminary
Reports

28. LoadRunner EXPERT WORKFLOW SUMMARY
4.1 Verify that Vusers run
concurrently
4.2 Isolate top time
transactions
4.3 Run full load test
4.4 Isolate hardware and
software limitations
1.1 Get system usage
information
1.2 Document business
process properties
1.3 Determine which fields
to parameterize
1.4 Establish which data to
use
Tune
System Based
on Analysis
Analyze
System Under
Load
Phase 5
Run
Scenarios
Phase 4
Create
Scenarios
Phase 3
Create Web
Virtual Users
Phase 2Phase 1
Plan Load Test
2.1 Record user actions
2.2 Add LoadRunner
transactions
2.3 Parameterize data
2.4 Add verification
checks
2.5 Verify correct
execution
3.1 Define hosts
3.2 Connect hosts
3.3 Define scripts
3.4 Add Vusers

29. PERFORMANCE ISSUES

30. PERFORMANCE TEST SCENARIO
1
It is expected that company X web-site is accessed by 1000 of their customers during day time on
average per hour. Their customer base consists of corporate, resellers and domestic consumers.
They expect that the website home page to be loaded within 1 second.
How do you plan the performance Test?

31. PERFORMANCE TEST SCENARIOS
The list of scenarios which you should include in your performance test,
Most Frequently Accessed Scenarios: Application scenarios which are mostly accessed by end users. As such
scenarios will affect maximum users they must be included in load test. For example, browsing product catalog in an E-
commerce web application.
Business Critical Scenarios: Application scenarios where application core features exists. For example, purchasing a
product is a business critical scenario in an E-commerce application.

32. PERFORMANCE TEST SCENARIOS
CONT..
Resource Intensive Scenarios: Such scenarios which are expected to consume more system resources as compared to
others. For example, order placement will be most resource intensive scenario in an E-commerce web application.
Contractually Obligated Scenarios: Application scenarios for which company has contracted to provide hassle free
services. These scenarios might not be used very frequently but they can create huge business loss in case of failure.
For example, company claims its home page loads within 3 seconds.

33. PERFORMANCE TEST SCENARIOS
CONT..
Stakeholders Concerning Scenarios: Stakeholders could be more concerned on AUT new features impact on its
overall performance.
Time Dependent Frequently Accessed Scenarios: Time dependent scenarios which are executed very frequently
but on certain occasions only. For example, viewing monthly pay roll slip on an online payroll application.
Technology Specific Scenarios: These are the scenarios which are specific to AUT selected technology. For example,
uploading a file through FTP server could be an example of technology specific scenario.

34. DO’S & DON'TS
Do’s
Do test early and do test often
Do establish what is and is not
acceptable performance for
your application
Do test from the user’s perspective
- it’s the only one that counts
Do baseline and compare your
findings
Do monitor your system while you
test
Do test whenever there’s a change
in your site’s content, code or
infrastructure
Don’ts
Don’t wait until the last minute to
test
Don’t depend on your customers to
do your testing for you
Don’t test under unrealistic
conditions
Don’t forget that increases in table
sizes, disk usage and network
traffic will degrade your
application’s performance over
time
Don’t be so quick to throw hardware
at the problems you turn up - it
doesn’t always help

35. QUESTIONS & ANSWERS

36. ThankYou !
www.auxenta.com auxentainc auxenta