More Related Content
Similar to Testing using load runner performance testing
Similar to Testing using load runner performance testing (20)
More from SivaprasanthRentala1975
More from SivaprasanthRentala1975 (20)
Testing using load runner performance testing
- 1. ibm.com/bcs © Copyright IBM Corporation 2006CONFIDENTIAL – Not for Distribution
PLANNING LOADTESTINGPLANNING LOADTESTING
LESSON 1LESSON 1
- 2. CONFIDENTIAL
© Copyright IBM Corporation 2006
Why Plan?Why Plan?
So that you will be able to:
Build test scenarios that accurately emulate your system
Know the required resources for testing
Define the load test objective
Define success criteria (in measurable terms)
- 3. CONFIDENTIAL
© Copyright IBM Corporation 2006
LoadRunnerWorkflowLoadRunnerWorkflow
Plan Load TestingPlan Load Testing11
Create Scenario3
Run Scenario4
Analyze Results5
Create Vuser Scripts2
Debug and Run Scenario Again6
Define load testing objectives
Analyze system under test
Plan LoadRunner Implementation
- 4. CONFIDENTIAL
© Copyright IBM Corporation 2006
Planning Load TestingPlanning Load Testing
In this lesson you will learn how to:
1 - Define load testing objectives
Objectives
2 - Analyze the system under test
3 - Plan LoadRunner implementation
Vuser Vuser
VuserVuser
Controller
- 5. CONFIDENTIAL
© Copyright IBM Corporation 2006
1 - Define Load Testing Objectives1 - Define Load Testing Objectives
Why test objectives?
“Successful projects are those that have a defined test objective… and provide actionable results
and conclusion…”
Source: “The Art of Testing Network Systems” Robert W. Buchanan, Jr.
Limit your objectives
“...Projects should have one or, at the most, two test
objectives… More than two objectives dilutes the effort
and makes [ testing ] virtually impossible.”
State objectives in measurable terms
“...There is nothing worse than completing [ a test ]
and then disagreeing on whether or not the system passed.”
- 6. CONFIDENTIAL
© Copyright IBM Corporation 2006
Common Test ObjectivesCommon Test Objectives
Application response time
How long does it take to complete a task?
Configuration sizing
Which configuration provides the best performance
level?
Acceptance
Is the system stable enough to go into production?
Regression
Does the new version of the software adversely
affect response time?
Reliability
How stable is the system under a heavy work load?
Capacity planning
At what point does degradation in performance
occur?
Bottleneck identification
What is the cause of degradation in performance?
Product evaluation
What is the best server for 100 users?
- 7. CONFIDENTIAL
© Copyright IBM Corporation 2006
State Objective in Measurable TermsState Objective in Measurable Terms
Measurable objectives provide baselines for evaluating results
Examples
Examples
Response time
under a load of 50 sales representatives, the maximum response time for any transaction will not exceed 1 minute
Reliability
the system will run for 72 hours at 50% load(as measured by CPU utilization) without a crash, or any corrupted data
- 8. CONFIDENTIAL
© Copyright IBM Corporation 2006
2 - Analyze The System UnderTest2 - Analyze The System UnderTest
Identify system components
schematic diagram of system
Describe configuration of each component
hardware/CPU speed
operating system
memory
Describe user profile
types of users
common tasks for each user
task distribution during the work day
- 9. CONFIDENTIAL
© Copyright IBM Corporation 2006
Identify System ComponentsIdentify System Components
Example
Example
Clients
586/ Windows 98
OCI
Pentium
400MHZ
NT 4.0
Oracle
8.0
Server
- 10. CONFIDENTIAL
© Copyright IBM Corporation 2006
Describe ConfigurationDescribe Configuration
Front End Client
No of clients 120
Hardware/Memory 586 PC / 32 Mb
Operating system Windows 95
Development tool Visual Basic
Example
Example
Middleware/App Server (if exists)
Server type
Communication with server
Server operating system
Back End Server
Server type Database- Oracle 8.0
Communication with server ODBC
Server operating system NT
Other Network Components
Communication device handles 50 calls max.
Provide details about the configuration of each
component:
- 11. CONFIDENTIAL
© Copyright IBM Corporation 2006
UserProfileUserProfile
Who are the users?
How many users are there?
What tasks do the users perform?
Business
Process
Type and
number
of users
Task frequency (throughput per hour)
Reservation
Agent (170)
Freq. Flyer
Agent (50)
Manager
(30)
1- Enter Orders
3 - Calc. F.F. Miles
4 - Calc. Sales
2 - Update Orders
100
50
70
8
B.P.
Users
Example
Example
25
10
5
How often do users perform their
tasks?
- 12. CONFIDENTIAL
© Copyright IBM Corporation 2006
Transaction ProfileTransaction Profile
What tasks do the users perform?
2- Enter Orders
4 - Calc. F.F. Miles
5 - Calc. Sales
3 - Update Orders
125
60
75
8
Business Process
Transactions
Per Hour
1- Logon 10
Example
Example
Tasks Frequency
How often do users perform their tasks?
- 13. CONFIDENTIAL
© Copyright IBM Corporation 2006
3 - Plan the LoadRunnerImplementation3 - Plan the LoadRunnerImplementation
1 - Set up test environment
2 - Define type and number of Vusers to be created
3 - Determine hardware required to run Vusers
- 14. CONFIDENTIAL
© Copyright IBM Corporation 2006
Set Up Test EnvironmentSet Up Test Environment
All testing should be done in a test environment
Allocate and configure machines to real-world conditions
one for each type of GUI Vuser
one for the database server
any other middleware server(s)
Allocate machines to run necessary number of DB Vusers
Allocate and configure one machine for Controller
Seed test database with appropriate amounts of data
Verify that business processes function correctly
- 15. CONFIDENTIAL
© Copyright IBM Corporation 2006
What Is a GUIVuser?What Is a GUIVuser?
A WinRunner script, performing actions on the client application
WinRunner
Allows you to measure the response time a real user will
experience end-to-end
One GUI Vuser per host machine
Host
GUI Vuser
Client
application
- 16. CONFIDENTIAL
© Copyright IBM Corporation 2006
What Is a DBVuser?What Is a DBVuser?
Multiple DB Virtual UsersMultiple DB Virtual Users
ServerServer
API Calls
C program that sends API calls to the server
Allows you to generate load on the server using few machines
Excludes application’s GUI processing time
Many DB Vusers per host machine
C Program
C Program
C Program
C Program
- 17. CONFIDENTIAL
© Copyright IBM Corporation 2006
GUIvs. DBVuserGUIvs. DBVuser
Database Server
Application
Server
Application
Server
Application Server
WAN
and/or
Internet
Example
Example
GUI VuserGUI Vuser: End-to-end
(including GUI)
GUI VuserGUI Vuser: End-to-end
(including GUI)
DB VuserDB Vuser: Network and
Server (No GUI)
DB VuserDB Vuser: Network and
Server (No GUI)
- 18. CONFIDENTIAL
© Copyright IBM Corporation 2006
Type and Numberof VusersType and Numberof Vusers
How many Vusers of each type (DB/GUI) do you need to run?
Guidelines:
one or few GUI Vusers for each type of user connection
multiple DB Vusers generate remaining load for that user
100 customer service users
(LAN connection)
2 98
50 Tokyo users (dial-in ISDN connection) 1 49
5 background batch processes — 5
20 managers
(15 with 486; 5 with 586 machines)
1 (486 PC)
1 (586 PC) 18
UserExample
Example
DBGUI
- 19. CONFIDENTIAL
© Copyright IBM Corporation 2006
Which Hardware/Configuration?Which Hardware/Configuration?
Based on previous steps, which hardware and software
configuration is needed for testing?
How many machines (PC/UNIX)?
How much memory?
Processor speed?
How much disk space?
How much swap?
- 20. CONFIDENTIAL
© Copyright IBM Corporation 2006
Hardware/Configuration GuidelinesHardware/Configuration Guidelines
General:
One machine for Controller
One machine per GUI Vuser
GUI Vuser machine configuration should be
similar to ‘real-world’ machine
Test server machine should be configured to
real world
DB Vuser machines:
Can use a combination of NT and UNIX
Machine configuration depends on number of
Vusers and activity of user
Exact configuration
requirements in Appendix A
- 21. CONFIDENTIAL
© Copyright IBM Corporation 2006
Which Hardware/Configuration?Which Hardware/Configuration?
Controller
NT machine
Pentium 200 MHz
64 MB memory
Database Server
1 GUI Vuser
(Accounting)
• NT machine
• Pentium 200 MHz
• 32 MB memory
29 DB Vuser
(Accounting)
• NT machine
•Pentium 166
MHz
• 64 MB memory
UNIX machine
256 MB memory
Example
Example
450 DB Vusers
(Bank tellers)
- 22. CONFIDENTIAL
© Copyright IBM Corporation 2006
SummarySummary
Define load testing objectives
limit your objectives
state objectives in measurable terms
Analyze the system under test
system components
configuration of each component
user profile (users, tasks, distribution)
Plan LoadRunner implementation
set up test environment
decide type and number of Vusers to be created
hardware required to run Vusers
- 24. ibm.com/bcs © Copyright IBM Corporation 2006CONFIDENTIAL – Not for Distribution
LOADRUNNER SCENARIOS
LESSON 2
LR/NTIntro6.0/02
Copyright 2000
Mercury Interactive Corp.
- 25. CONFIDENTIAL
© Copyright IBM Corporation 2006
LoadRunner Workflow
Plan Load TestingPlan Load Testing11
Create ScenarioCreate Scenario33
Run ScenarioRun Scenario44
Analyze ResultsAnalyze Results55
Create Vuser ScriptsCreate Vuser Scripts22
Debug and Run Scenario AgainDebug and Run Scenario Again66
• Define and connect hosts
• Define Vusers and scripts
• Group and schedule Vusers
• Set rendezvous policy
- 26. CONFIDENTIAL
© Copyright IBM Corporation 2006
Lesson Objectives
In this lesson you will learn:
LoadRunner elements
How to set up a scenario
How to run a scenario
- 28. CONFIDENTIAL
© Copyright IBM Corporation 2006
The LoadRunner Controller
Runs scenarios
Controls and communicates with each Vuser via the Agent
Displays output messages
Displays execution status
Collects performance data
- 29. CONFIDENTIAL29 © Copyright IBM Corporation 2006
Controller Main Window
Initialize /
Run Vuser
Vuser
groups
Vuser details
Sub-window tabs
- 30. CONFIDENTIAL
© Copyright IBM Corporation 2006
Controller Sub-Window Tabs
Hosts window
- all available hosts and their connection statuses
Scripts window
- paths to all available Vuser scripts
Vusers window
- create and select Vusers and groups
Online window
- view real-time system resources during scenario execution
- 31. CONFIDENTIAL
© Copyright IBM Corporation 2006
How Does Controller Communicate with Vusers ?
Start the Remote Command Launcher on each host machine
Controller
Machine A
Machine B
Remote
Command
Launcher
• RCL starts the Agent and initializes the Vusers
INITIALIZE
Agent
Vusers
• Controller Runs Vusers via the Agent
RUN
- 32. CONFIDENTIAL
© Copyright IBM Corporation 2006
The Scenario
An environment that describes all the conditions under which you want to test your
system
virtual
users
load
conditions
host
machines
results
location
test
scripts
- 33. CONFIDENTIAL
© Copyright IBM Corporation 2006
Setting Up a Scenario
1 - Define Hosts
2 - Connect Hosts
3 - Define Vuser scripts
4 - Add Vusers
5 - Set rendezvous policy (optional)
6 - Set up Vuser Scheduling (optional)
- 34. CONFIDENTIAL34 © Copyright IBM Corporation 2006
Machine name or
IP address
Host Status,
Ready/Active
1 - Define Hosts
Results
handling
information
UNIX
information
for DB Vusers
Initialization
Quota
Limits per host
for each Vuser
type
WinRunner
information
for GUI Vusers
- 36. CONFIDENTIAL36 © Copyright IBM Corporation 2006
3 - Define Vuser Scripts
A scenario typically contains
multiple instances of each
script
Vusers assigned
to this script
Transactions and
Rendezvous in script
Path to script
Vuser type depends
on script type
- 37. CONFIDENTIAL37 © Copyright IBM Corporation 2006
4 - Add Vusers
Quantity Hostname
Script name
Click the Add Vusers toolbar button
- 38. CONFIDENTIAL38 © Copyright IBM Corporation 2006
4 - Add Vusers (cont.)
NOTE: Run-Time Settings are discussed in the VuGen course
Host machine
Script name
1. Select an existing Vuser from the grid
2. Select Vuser > Details from the menu
- 39. CONFIDENTIAL39 © Copyright IBM Corporation 2006
5 - Define Rendezvous Policy
(optional)
Enable/disable
rendezvous
Specify when to
release Vusers
Specify maximum
time Vusers wait
to be released
Release Vusers
that arrived
- 40. CONFIDENTIAL40 © Copyright IBM Corporation 2006
• LoadRunner Program group – select SchedulerDouble-click to schedule a new group
Properties button to select details
Specify duration settingsSpecify Incremental Initialization PaceSpecify Incremental Run PaceSpecify Termination Pace – only available
when a group duration has been set
Visual Group Settings
6 - Scheduling Vusers
(optional)
- 42. CONFIDENTIAL
© Copyright IBM Corporation 2006
LoadRunner Workflow
Plan Load TestingPlan Load Testing11
Create ScenarioCreate Scenario33
Run ScenarioRun Scenario44
Analyze ResultsAnalyze Results55
Create Vuser ScriptsCreate Vuser Scripts22
Debug and Run Scenario AgainDebug and Run Scenario Again66
• Check Remote Command Launcher
• Specify results directory
• Initialize Vusers
• Run Vusers
- 43. CONFIDENTIAL
© Copyright IBM Corporation 2006
Running a Scenario
1 - Make sure Remote Command Launcher is active on all hosts
2 - Specify the directory for storing execution results
- By default, the runtime files are stored locally on each host. After the scenario, the
results are collated and processed on the Controller machine.
- Alternatively, instruct LoadRunner to use a shared network drive
- Use Mercury’s TestDirector to store results to a project
3 - Initialize Vusers
4 - Run Vusers
- 45. CONFIDENTIAL
© Copyright IBM Corporation 2006
Tips for Smooth Scenario Execution
Make sure Remote Command Launcher is active on each host.
Verify successful execution of each Vuser script before running with
LoadRunner.
Make sure all scripts have the proper path and run-time settings.
Disconnect from the host and refresh after editing a script, to ensure that
the new version of the script is transferred to the host.
- 46. CONFIDENTIAL
© Copyright IBM Corporation 2006
Managing the Scenario
Good Practices:
Run the scenario gradually for easy problem solving:
- individual Vusers
- a group of Vusers
- entire scenario
Initialize Vusers before running
- initializing process is resource intensive
- ensures that all Vusers start from same point
- 47. CONFIDENTIAL
© Copyright IBM Corporation 2006
Summary
Set up a scenario
1 - define Hosts
2 - connect Hosts
3 - define Vuser scripts
4 - add Vusers
5 - set rendezvous policy (optional)
6 - schedule Vusers (optional)
Run a scenario
1 - check Remote Command Launcher
2 - specify results directory
3 - initialize Vusers
4 - run Vusers