Advanced Rational Performance Tester reports     5 ways to get deeper insight into performance test results     Skill Leve...
developerWorks®                                                                       ibm.com/developerWorks     1. Handli...
ibm.com/developerWorks                                                               developerWorks®                   sch...
developerWorks®                                                                 ibm.com/developerWorks     Figure 2. Gener...
ibm.com/developerWorks                                                                developerWorks®     Figure 3. Custom...
developerWorks®                                                                    ibm.com/developerWorks     3. Creating ...
ibm.com/developerWorks                                                                  developerWorks®            4.     ...
developerWorks®                                                                   ibm.com/developerWorks     Figure 5. Bus...
ibm.com/developerWorks                                                                   developerWorks®            1.    ...
developerWorks®                                                                 ibm.com/developerWorks     5. Correlating ...
ibm.com/developerWorks                                                            developerWorks®     Figure 7. Customizin...
developerWorks®                                                                 ibm.com/developerWorks     Resources     L...
ibm.com/developerWorks                                                           developerWorks®         • Connect with ot...
Upcoming SlideShare
Loading in...5
×

Advanced Rational Performance Tester reports

3,208

Published on

By default, IBM® Rational® Performance Tester provides essential performance metrics, such as throughput, response times, concurrency, and success rate. However, it also includes several advanced features for detailed analysis, many of which are not commonly used. Proper use of these options provides deeper insight when analyzing test results. This article gives five tips for using some of these advanced features, all of which have helped tremendously in real-world performance testing projects with large companies.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,208
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
130
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Advanced Rational Performance Tester reports

  1. 1. Advanced Rational Performance Tester reports 5 ways to get deeper insight into performance test results Skill Level: Advanced Jigar N Kapasi (kjigar@in.ibm.com) Senior Software Architect IBM India Priyanka Arora (prarora1@in.ibm.com) Performance Analyst IBM 01 Mar 2011 By default, IBM® Rational® Performance Tester provides essential performance metrics, such as throughput, response times, concurrency, and success rate. However, it also includes several advanced features for detailed analysis, many of which are not commonly used. Proper use of these options provides deeper insight when analyzing test results. This article gives five tips for using some of these advanced features, all of which have helped tremendously in real-world performance testing projects with large companies. This article provides five tips that a performance test engineer can use to gain deeper insight into performance results: 1. Handling spikes in results 2. Generating and analyzing percentile reports 3. Creating and using business transactions 4. Determining volumes during a subset of a run 5. Correlating client-side results with server activitiesAdvanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 1 of 13
  2. 2. developerWorks® ibm.com/developerWorks 1. Handling spikes in performance test results In an IBM® Rational® Performance Tester report, the Response vs Time Detail tab plots response time for each page. Extraordinarily high response times, or spikes (as shown in Figure 1), can be a concern if they occur frequently. Analyzing the raw response time data can give useful information about application performance. Figure 1. Spikes in page response times To determine the percentage of such outliers, you need the raw data for the test run that Rational Performance Tester uses to plot the performance report graphs. These are the steps to get this data: 1. Right-click any tab of the report, and select Export Report Counters to CSV. 2. Browse to a location to save the output, and name the file. 3. For Export Type, select Full to get all counters. 4. Check Export overall results and Split output if columns exceed 250 (250 is the maximum column limit for a Microsoft Excel file). 5. In the location that you just specified, multiple .csv files will get created. Browse the files for the one with this column name: All_Hosts Pages Response Time Page_Name Average interval. Note:Page_Name here will be replaced by the page titles in your test schedule, and there will be a separate column for each page in the testAdvanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 2 of 13
  3. 3. ibm.com/developerWorks developerWorks® schedule. Look for data on pages that show spikes in the graph. This data can now be used to determine the percentage of outliers for each page and troubleshoot potential bottlenecks. 2. Generating and analyzing percentile reports Percentile response times are often better indicators of the application performance than average response times. Understanding how to generate and analyze performance reports will help you understand system behavior more accurately. Generate percentile reports After a test is finished and the performance report has been generated, you can generate a percentile report. 1. From the Performance Test Runs window, under the report, right-click All Hosts. 2. Select HTTP Reports > Percentile Report, as shown in the drop-down menu selections in Figure 2, to open the percentile report with default analysis targets.Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 3 of 13
  4. 4. developerWorks® ibm.com/developerWorks Figure 2. Generating percentile report By default, a percentile report shows the 85th, 90th, and 95th percentiles. You can define a new value or modify the default percentile analysis target. 3. Select Window > Preferences. 4. Expand Test and click Percentile Analysis Targets. 5. Either click Add and enter a required percentile value, as shown in Figure 3, or select an existing target and click Edit. 6. Click OK, and then click Apply to implement your changes.Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 4 of 13
  5. 5. ibm.com/developerWorks developerWorks® Figure 3. Custom percentile analysis targets 7. Regenerate the percentile report to see your custom percentile target values. Analyze percentile reports Theoretically, if the nth percentile of a set equals x, it means that n% values in the given set are below the value of x. So, applying that to percentile response time reports in Rational Performance Tester, if the 85th percentile response time for a page is 5 seconds, it means that 85% of virtual users got a response time of 5 seconds or better. Percentile response times are more relevant than the average times when it is important that each user of the system gets acceptable response times consistently. The more stringent a customer is on each users experience, the higher the percentile target to be considered. As the number of spikes in the response time set increases, the probability of getting a high percentile response time value for that page also increases. Also see the Handling spikes in performance test results section.Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 5 of 13
  6. 6. developerWorks® ibm.com/developerWorks 3. Creating and using business transactions Customers often characterize non-functional requirements in terms of business transactions. A business transaction typically spans multiple pages; whereas, by default, Rational Performance Tester reports metrics on a page basis. Generating performance results represented in terms of business transactions makes it easier for business users to benefit from what the reports reveal. For business transaction-based metric reports, define the business transactions first, and then define the required performance metrics first. Then you can generate the transaction reports. Define business transactions A typical business transaction is representative of a use case consisting of several pages (navigating through required pages to raise a service request, for example). After you have identified a business transaction, you can define it by following these steps: 1. Record the scenario in Rational Performance Tester. 2. Select all sequential pages in the script that form the transaction. 3. Right-click and select Insert > Transaction. 4. In response to the question in the pop-up window that asks if you would like to move the selected objects into a new transaction, click Yes. 5. Give the transaction a meaningful name, and then Save the test. Enable performance requirements A performance requirement specifies acceptable thresholds of performance and helps validate service level agreements. You can define performance requirements on several test elements. Follow these steps to enable performance requirements on business transactions: 1. Select the transaction in the test contents. 2. Under Test Element Details, click the Advanced tab. 3. Click the check box for Enable Performance Requirements.Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 6 of 13
  7. 7. ibm.com/developerWorks developerWorks® 4. Now define all of the transaction performance requirements that will determine the transaction status (Pass or Fail) of your test. For example, if it is a requirement that the transaction must complete in not more than 120 seconds: a. Select the Maximum Transaction Execution Time [ms] performance requirement. b. Choose the <= operator (less than or equal to operator). c. Enter the value in milliseconds. In this case, enter 120000, as shown in Figure 4. 5. Ensure that the page think times and pace times in this transaction align with these requirements. Thus, the sum of all think times, pace times, and expected page response times should fit within this limit. Figure 4. Business transaction performance requirements Larger view of Figure 4. Generate transaction performance reports If you have transactions defined in your test suite, then the Summary tab of the performance report details a section on transaction performance metrics, such as average execution time, maximum or minimum execution time, and so on, as Figure 5 shows.Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 7 of 13
  8. 8. developerWorks® ibm.com/developerWorks Figure 5. Business Transaction Summary report To get a detailed report for each business transaction, generate the Transaction Report and the Transaction Percentile Report: 1. In the Performance Test Runs window, right-click the test run. 2. Select either Display Transaction Report or Display Transaction Percentile Report. 3. The Transaction Report gives the transaction throughput, transaction count for the run, and a plot of transactions at run time. The Transaction Percentile Report shows execution (run) times by transaction at the defined percentile analysis targets. See the Generating and analyzing percentile reports section for more details. 4. Determining volumes for a specified subset duration Some performance tests have non-functional requirements for volumes of business transactions achieved over a specified duration in a steady state. By default, a Rational Performance Tester report gives the total number of transactions achieved across the total duration of the run. These results include statistics gathered during the ramp-up phase, where the page hits are much lower. Getting performance metrics during the steady state gives you a more realistic measure of how a system will perform. You can use the steady state as the time range for your report. Alternatively, you can also define your own time range in the test run by specifying start and end times. By doing this, you will then see metrics in the report that pertain only to that time range. Follow these steps to define your own time range:Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 8 of 13
  9. 9. ibm.com/developerWorks developerWorks® 1. Click the Server Health Detail tab on the report. The Hits -- Count [for run] column lists volumes against each page or transaction. 2. Right-click the report and select Change Time Range. 3. In the Select Time Range view (Figure 6), click the New Time Range button to define a new range. 4. You then have two choices: • Either click Set to Steady State • Or enter values of your choice for Start Time and End Time from within the test run duration Figure 6. Changing the time range for the report 5. Finish and wait for the report to generate with the new time range. The report will then show statistics only from samples in the range that you specified. (Also see the Creating and using business transactions section.)Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 9 of 13
  10. 10. developerWorks® ibm.com/developerWorks 5. Correlating reports with server activities and logs When monitoring or analyzing performance reports, it is very useful to correlate the reported performance with activities on the server side (through logs, traces, for instance). By default, the Rational Performance Tester performance report shows time across all tabs in seconds (starting at zero). Matching the time scale in the report with that of the server helps correlate performance results with server events and aids troubleshooting. It is easier to correlate client-side performance metrics from Rational Performance Tester with the server-side log activities if the test report shows absolute times rather than relative times starting from zero. To customize the time, open a performance report and complete these tasks: 1. Go to the Response Vs Time Detail tab on the report. 2. Right-click and select Customize. 3. For the X Axis field, click the arrow to open the drop-down menu, and select Date & Time (Absolute) to change the template, as shown in Figure 7. 4. Click OK and let the report refresh to reflect the change.Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 10 of 13
  11. 11. ibm.com/developerWorks developerWorks® Figure 7. Customizing time on reports Important: To ensure precise correlation, synchronize the system time on the Rational Performance Tester host controller, agent, and all servers under test.Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 11 of 13
  12. 12. developerWorks® ibm.com/developerWorks Resources Learn • Download and read the PDF titled Using Rational Performance Tester Version 7 (IBM® Redbooks®, 2008). • Find out more on the Rational Performance Tester product overview page. Then explore the Rational Performance Tester page on IBM® developerWorks® for links to technical articles and browse the user assistance in the Rational Performance Tester Information Center. • Visit the Rational software area on developerWorksfor technical resources and best practices for Rational Software Delivery Platform products. • Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics. • Attend a free developerWorks Live! briefing to get up-to-speed quickly on IBM products and tools, as well as IT industry trends. • Watch developerWorks on-demand demos, ranging from product installation and setup demos for beginners to advanced functionality for experienced developers. • Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, any time, and many of the “Getting Started” ones are free. Get products and technologies • Download the trial version of IBM Rational Performance Tester. • Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently. Discuss • Join the Performance Testing forum, where you can share you questions and knowledge about IBM performance testing products. • Share your knowledge and help others who use Rational software by writing a developerWorks article. Youll get worldwide exposure, RSS syndication, a byline and a bio, and the benefit of professional editing and production on the developerWorks Rational website. • Follow Rational software on Facebook and Twitter (@ibmrational), and add your comments and requests. • Get involved in the Rational forums, groups, and Rational wikis.Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 12 of 13
  13. 13. ibm.com/developerWorks developerWorks® • Connect with others who share your interests by joining the developerWorks community and responding to the developer-driven blogs. About the authors Jigar N Kapasi Jigar Kapasi is a software performance architect in the IBM India Software Lab, working with the HiPODS team. He works with large companies to lead critical performance engineering projects, which include architecture and design recommendations, performance testing, infrastructure and middleware analysis and tuning. He has been with IBM for 12 years and holds a bachelor’s degree in Information Technology from Bangalore University. Priyanka Arora Priyanka Arora is a software performance analyst with the High-Performance On Demand Solutions team in India. She has more than three years of experience with strong technical analysis, tuning, troubleshooting, tooling, and load test execution of end-to-end performance engagements for several IBM telecom and public sector customers. She has an MSc Tech in Information Systems from Birla Institute of Technology and Science in India and is a certified WebSphere Application Server and Portal Server Administrator.Advanced Rational Performance Tester reports Trademarks© Copyright IBM Corporation 2011 Page 13 of 13

×