Intermediate_and_Adv..

  • 279 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
279
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
7
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Text files created on Windows use a different, and incompatible, line ending scheme than what is used by Unix. Because of this shell scripts created or edited on Windows typically won't be executable on a Unix system. Hence, the use of the tr command to “UNIX-fy” the line endings of any shell scripts that may have been modified by a Windows editor.
  • The script uses the server and username, but has a LOOKUP_PWD constant instead of the actual password. At playback the authentication datapool is searched, matching the server username pairing. The password is returned from the matching row.
  • To modify the Authentication Datapool with TestManager: Click Tools > Manage > Datapools. Click RTAuthentication (the Authentication Datapool). Click Edit. Click Edit Datapool Data in the Datapool Properties dialog box. The Edit Datapool dialog box appears. Each row in the Edit Datapool dialog box contains the following information: State; Service; Class; Login; Subclass; Password Repeat the last step for each user ID and password you need to enter. Click Save, and then click Close. Click OK to close the Datapool Properties dialog box, and then close the Manage Datapools dialog box.
  • The red line indicates the time that the virtual tester was waiting for the database to confirm it had successfully added the customer record. The white space along the dotted line for each virtual tester indicates the time that the virtual tester was in a the init script or a transaction delay. Here it is good to note that transactors control the arrival rate; they control the left-most point of each of the response times shown above. The response time of the system under test controls the right-most point; response time is a determined by the capabilities and saturation of the system under test. Since the arrival rate of work to the system under test (one add_customer request per second) is matched by the unloaded system’s capability to fulfill each request in one second, there is no time when the system under test is concurrently servicing more than one request.
  • Please see Robot Helper Help for sample code.

Transcript

  • 1. Intermediate and Advanced Tips and Tricks for Performance Testing with Rational Suite TestStudio Scott Silverstone Jeff Robbins Rational Software
  • 2. Your Presenters
    • Scott Silverstone, Rational Marketing Engineer
    • Jeff Robbins, Rational Product Manager
    • Special thanks to presentation reviewers
      • Kent Siefkes
      • Dawn Haynes
  • 3. What Does A Performance Tester Do?
  • 4. Getting Started
  • 5. Performance Test Planning & Design
    • Several other presentations cover Planning & Design
      • AT03 Basic Tips and Tricks for Performance Testing
      • AT32 Modeling the Real World for Load Testing Web Sites
      • RTL09 Introduction to Functional and Performance Testing
    • See www.rational.com/ruc for other presentations
    • Due to coverage in these presentations, this presentation will not cover planning & design
  • 6. Digging In
  • 7. Creating Test Scripts: Hint Outline
    • Use of “non-traditional” performance test scripts
    • Use of INFO SERVER for monitoring
    • VU language trick to conserve hardware
  • 8. Creating Test Scripts: Hint Outline
    • Use of “non-traditional” performance test scripts
    • Use of INFO SERVER for monitoring
    • VU language trick to conserve hardware
  • 9. Use Of “Non-Traditional” Test Scripts
    • You are not limited to using VU language scripts
    • Other scripting languages are supported
      • Java
      • Command line (compiled executables,shell scripts, perl scripts, etc)
      • Visual Basic
    • Presentations on Java & Visual Basic scripting:
      • AT28 - Designing Scalable Performance Test Scripts for Java Environments
      • AT30 - Custom Load Generation with Rational TestManager
      • VM14 - Model Based Performance Testing of EJB and DCOM/COM+ Components
    • Let’s talk about Command Line scripts in more detail
  • 10. Command Line Test Scripts – Why Use?
    • Command Line Test Script Type
      • Run a setup script at the very start of the test
      • Run a cleanup script at the very end of the test
      • Execute any command in system PATH
      • Simply put, run anything that can be run from the command line
    • New Types based on Command Line Execution Adapter
      • Use perl scripts as your load test scripts
      • Execute shell scripts throughout the run
      • Use binaries compiled from C, Ada or Smalltalk as load test scripts
      • Launch preVue tests as part of a larger load test under TestManager
    • Creating a new type allows more control over execution conditions than simply using the command line test script type
  • 11. Using The Command Line Test Script Type (1)
  • 12. Using The Command Line Test Script Type (2)
  • 13. Using The Command Line Test Script Type (3)
    • Resulting Suite:
  • 14. Questions
    • What is the name of the project we are using?
    • What is the name of our suite?
    • What are the name of our user groups?
  • 15. New Type From Command Line Execution Adapter (1)
  • 16. New Type From Command Line Execution Adapter (2)
  • 17. New Type From Command Line Execution Adapter (3)
  • 18. New Type From Command Line Execution Adapter (4)
  • 19. New Type From Command Line Execution Adapter (5)
  • 20. Using The New Type In A Suite (1)
  • 21. Using The New Type In A Suite (2)
  • 22. Using The New Type In A Suite (3)
  • 23. Using the New Type In A Suite (4)
  • 24. Creating Test Scripts: Hint Outline
    • Use of “non-traditional” performance test scripts
    • Use of INFO SERVER for monitoring
    • VU language trick to conserve hardware
  • 25. Use Of INFO SERVER For Monitoring (1)
    • As discussed in AT03 (Basic Tips and Tricks for Performance Testing), monitoring of system resources on the local computer and agents is done automatically
    • Monitoring of computers other than the local computer and agents requires use of the INFO SERVER declaration
    • Robot automatically inserts the INFO SERVER declaration for the database server into SQL scripts
    • Robot does not automatically insert INFO SERVER into other types of scripts (Tuxedo, HTTP, Jolt, CORBA, DCOM)
  • 26. Use Of INFO SERVER For Monitoring (2)
    • To add servers to monitor, add the INFO SERVER declaration to a script. Which script, you ask?
    • One approach is to have a set of re-usable scripts to initialize monitoring for each computer of interest
      • These scripts can be inserted in any suites you like
    • Alternatively, the INFO SERVER declarations can be added directly to the scripts where the server access occurs
  • 27. Use Of INFO SERVER For Monitoring (3)
    • During Suite Execution, activate the computer view:
      • Right-click cells to add/remove stats from graph
  • 28. Creating Test Scripts: Hint Outline
    • Use of “non-traditional” performance test scripts
    • Use of INFO SERVER for monitoring
    • VU language trick to conserve hardware
  • 29. VU Language Trick To Conserve Hardware
    • If you don’t need to match on the _response or log all the data (or only need the first part of the _response ):
      • Set Max_nrecv_saved to a low number just big enough, such as 1 or 100
        • SQL: limits # of rows saved
        • HTTP, socket: limits # of bytes saved
      • Applies to all receive commands, not just the “nrecv” flavor
      • Can be used selectively, such as around large SQL queries, or web pages with lots of images...
  • 30. Example: Max_nrecv_saved
    • Using Max_nrecv_saved to save only the first row of a large SQL query:
    sqlexec ["exec big query"] "select * from books"; push Max_nrecv_saved = 1; sqlnrecv ["retrieve big query"] ALL_ROWS; pop Max_nrecv_saved ;
  • 31. Digging Up Data…
  • 32. Manage Test Data: Hint Outline
    • Streamlining transfer of datapools to agents
    • All about the Authentication Datapool
      • What it is
      • Why to modify it
      • How to modify it
  • 33. Manage Test Data: Hint Outline
    • Streamlining transfer of datapools to agents
    • All about the Authentication Datapool
      • What it is
      • Why to modify it
      • How to modify it
  • 34. Streamlining Transfer Of Datapools (1)
    • Datapools are automatically copied to Agents at the start of a test run to minimize network traffic during script execution
    • Huge datapools may cause a long startup delay at beginning of a run when they are copied to agents
    • Control with system env variable on Master
      • RT_MASTER_DISABLE_MANDATORY_DP_XFER = 1
  • 35. Streamlining Transfer Of Datapools (2)
    • This turns off automatic copying of datapools to agents
    • Note: Rational TestManager & Rational Robot must be restarted to read the new value of this variable
      • Caution!!!
      • Updates to datapools won’t be used unless the datapool file is refreshed
      • The best way to refresh the datapool is to set RT_MASTER_DISABLE_MANDATORY_DP_XFER to 0, restart Rational TestManager and Rational Robot, then run a test. The updated datapools will then be copied to the agents.
  • 36. Manage Test Data: Hint Outline
    • Streamlining transfer of datapools to agents
    • All about the Authentication Datapool
      • What it is
      • Why to modify it
      • How to modify it
  • 37. What Is The Authentication Datapool?
    • Answer:
      • A unique datapool that collects secure login information during Rational Robot script recordings
      • Rational Robot generates entries into this file as the connection is opened during secure login sessions
      • Contains information about username, password, connection type (ex. SQL) and connection state (Active or Passive)
      • For secure login purposes ONLY, not a general-use datapool file
  • 38. Why Modify The Authentication Datapool?
    • Why?
      • If the password is encrypted during recording
        • The client/server security doesn’t allow a virtual tester to log into the database multiple times simultaneously
        • Common problem when recording scripts against secure SQL & Oracle systems
        • Passwords have been changed since the initial recording
        • Stores original input during recording of the secure login so data can be referenced during later test runs
  • 39.
    • Task: Authenticating logon information
      • Example Situation: Passwords are encrypted during the initial recording of the test run – Rational Suite TestStudio can’t play back the results properly because all virtual users after the first “Mary Smith” are locked out
    How To Modify The Authentication Datapool
  • 40. How To Modify The Authentication Datapool
    • What is inside the script
      • #include <VU.h>
      • {
      • sparky = sqlconnect(&quot;sparky&quot;, &quot;scott&quot;, LOOKUP_PWD, &quot;sparky&quot;, &quot;oracle7.3&quot;);
      • { INFO SERVER &quot;sparky&quot;=&quot;192.168.241.101&quot;; } /*1*/
      • The VU script uses the “LOOKUP_PWD”
      • constant instead of the actual password.
      • At playback, the authentication datapool is
      • searched.
  • 41. How To Modify The Authentication Datapool
    • In Rational TestManager
    • Open the Datapool manager
    • Select RTAuthentication (the Authentication DataPool)
    • Edit values accordingly
  • 42. How To Modify The Authentication Datapool
    • Tips:
      • Use realistic data
        • Experiment with different values/character lengths
        • Emulate patterns from current usage
      • Don’t delete or rename the Authentication Datapool (RTAuthentication)
      • Don’t modify the values of the preset columns
  • 43. Digging In
  • 44. Hints In Creating Test Suites: Outline
    • Use of Transactor Distributions
  • 45. Use Of Transactor Distributions
    • Basic transactor tips were covered in
      • AT03 Basic Tips and Tricks for Performance Testing
    • Quick Review: Transactors control the rate at which Rational TestManager starts scenarios (groups of scripts)
    • As such, transactors control the arrival rate of work to the system under test
    • Here, we’ll be discussing transactor distributions
      • When to use each one
      • What each one “looks like”
  • 46. Constant Distribution – When To Use
    • Use the constant distribution when input to the system under test occurs at an exact and unvarying rate
    • Customer situation:
      • Insurance company with 20 automated document scanners
      • At start of day, scanners are each initialized 1 second apart
      • All forms to be scanned are of the same length
      • Each scanner takes exactly 20 seconds to scan one of these forms
      • Scanned data is added to a customer record in a database
    • We would like to emulate the 20 scanners in order to verify that the database can handle the workload
  • 47. Constant Distribution – How To Do It
    • Contains SQL
    • statements that
    • add customer
    • info to
    • database
    Simulates staggered initialization of scanners
  • 48. Constant Distribution – What It Looks Like
    • If the database processes & verifies addition of each customer record in exactly one second:
    No overlaps, no gaps!
  • 49. Uniform Distribution – When To Use
    • Use the uniform distribution when input to the system under test occurs at a known average rate, but the time between each input is not constant
    • Customer situation:
      • Insurance company with 20 automated document scanners
      • At start of day, scanners are each initialized one second apart
      • Forms to be scanned are of varying length
      • Each form requires a different amount of time to scan
        • Average scan time is 20 seconds
      • After a scanner scans a form, it submits a customer record for addition to a database
    • We would like to emulate the 20 scanners in order to verify that the database can handle the workload
  • 50. Uniform Distribution – How To Do It
    • Contains SQL
    • statements that
    • add customer
    • record to
    • database
    Simulates staggered initialization of scanners
  • 51. Uniform Distribution – What It Looks Like
    • Assuming the database processes & verifies addition of each customer record in exactly one second:
    Some overlaps, a few gaps!
  • 52. Negative Exponential Distribution – When To Use
    • Use the negative exponential distribution when input to the system under test occurs at a variable rate
    • Customer situation:
      • Insurance company with 20 customer service representatives (CSRs)
      • CSRs enter customer information into application running on PCs
      • This application, just like the automated scanners, adds customer information as record in a database
      • Incoming customer calls are assigned to CSRs in the order in which the queue for that CSR was initialized
      • Because of timing of incoming calls, as well as length of each call, the time between the submission of customer data to the database varies
    • We would like to emulate the 20 customer service representatives & their desktop PCs in order to verify that the database can handle the workload
  • 53. Negative Exponential Distribution – How To Do It
    • Contains SQL
    • statements that
    • add customer
    • record to
    • database
    Simulates staggered Initialization of work queues
  • 54. Negative Exp Distribution – What It Looks Like
    • Assuming the database processes & verifies addition of each customer record in exactly one second:
    Many overlaps, more gaps!
  • 55. Questions
    • What if the database response time increases with number of concurrent updates?
    • In the previous response examples, the database response time for the addition of a customer record was always one second, no matter how many requests were being served concurrently
    • However, server response times often increase as the number of updates being concurrently handled increases, especially at higher user loads
    • If database response time =
      • 1 second * number of updates currently in progress
    • What response times will users experience if they arrive in a
      • constant distribution?
      • negative exponential distribution?
  • 56. Answer: Constant Distribution
    • No difference from before
  • 57. Answer: Negative Exponential Distribution
    • A drastic difference!!!
  • 58. Digging In
  • 59. Introduction To cwDBMON
    • cwDBMON is a user-contributed tool
    • Monitors database(s) during test execution
    • Provides integration into Rational TestManager reports for results evaluation
  • 60. Using cwDBMON (1)
    • Selecting Parameters to Monitor
  • 61. Using cwDBMON (2)
    • Test-time monitoring of the database
  • 62. Using cwDBMON (3)
    • Integrating DB parameters into Rational TestManager
  • 63. Using Of cwDBMON (4)
    • Response VS Time graph with
    • Oracle <Latch Efficiency> parameter
    Oracle Latch Efficiency Response times
  • 64. Downloading cwDBMON
    • Remember, this is a user-contributed tool
    • Not part of any Rational product
    • Not supported by Rational Technical Support
    • But the user community provides help
    • Download instructions posted at
      • ftp://exchange.rational.com/exchange/outgoing/ruc_2002_at_22/cwdbmon.txt
  • 65. A Quick Word On Evaluations
    • In the Comments section:
      • Please indicate your level of performance testing experience
        • Beginning, Intermediate, Advanced
      • The 3 tips you’ve found most useful
      • Format for next year
        • Same as this year (separate sessions for Beginning & Intermediate/Advanced)
        • OR
        • One unified session for Beginning , Intermediate and Advanced
        • OR
        • Other suggestions
          • Things you would like to see
          • Things you didn’t need to see
      • Any other comments are greatly appreciated
  • 66. Questions? Rational User Conference 2002 ©1998, 1999, 2000, 2001, 2002 Rational Software - All rights reserved
  • 67. Scott Silverstone [email_address] Jeff Robbins [email_address] Thank You! This presentation will be posted by tomorrow at: http://www.rational.com/ruc Rational User Conference 2002 ©1998, 1999, 2000, 2001, 2002 Rational Software - All rights reserved