Performance Testing using LoadRunner - Kamran Khan []


Published on

Performance Testing using HP LoadRunner
Download latest version from:

Published in: Technology, Education
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Target Audience: Developers and Testers who wish to work with HP LoadRunner on Web based ApplicationsLast Updated: 20140107© Copy Rights ReservedAuthor: Kamran Shaukat Ali KhanE-mail: [you can add this on GTalk/Google Hangout as long as you don’t expect me to reply  ]
  • This file was downloaded from:
  • Ref: Pulley profile:
  • Ref: Pulley profile:
  • Scenario Modification: Adding/ Removing Vusers, Real-life schedule, group or scenario configuration and execution etcDirect Revenue:LR Consultants making up to 85 USD per hourRef:
  • SQA – Software Quality Assurance/Dept.SQAAT – SQA - Automated Testing Team
  • Is not part of Load Testing, however, if downloading of HTML, javascript files or CSS fails because of non-responsiveness of Web Server, then it is a concern of Performance Testing.
  • Vuser Script Sections Each Vuser script contains at least three sections: vuser_init, one or more Actions, and vuser_end. Before and during recording, you can select the section of the script into which VuGen will insert the recorded functions. The following table shows what to record into each section, and when each section is executed. You use the VuGen script editor to display and edit the contents of each of the script sections. You can display the contents of only a single section at a time. To display a section, highlight its name in the Sections box.
  • HTML-Based Scripts The HTML based script option, which is the default recording mode, instructs VuGen to record HTML actions in the context of the current Web page. The actions are recorded as URL steps (web_url), link steps (web_link), image steps (web_image), and form submission steps (web_submit_form). VuGen records only the HTTP requests that return HTML pages, but not javascript or applications. /* HTML-based *//... web_link("Click Here For Additional Restrictions", "Text=Click Here For Additional Restrictions", "Snapshot=t4.inf", LAST); web_image("buttonhelp.gif", "Src=/images/buttonhelp.gif", "Snapshot=t5.inf", LAST); . . .
  • Benefits of HTML Recording• Reduces need to capture dynamic values – action tag values and hidden data are NOT hard coded – they are retrieved from memory during playback – if they are dynamic, the Vuser still runs• Script is only as big as the business process–one step per pageDisadvantages of HTML Recording• Scripts are less scalable• Memory (cache) is searched during playback – requires more memory – requires more CPU powerBenefits of URL Recording• Flexibility – support for Java Applets and ActiveX objects on the page – ability to replay on Unix• Scalability – scripts are more scalable than HTML scripts because they require fewer resourcesDisadvantages of URL recording• Scripts require more correlation (nothing is retrieved from cache)• Context-sensitive checks won’t work (parser is disabled)*• Scripts are large (all images and frames are recorded as separate steps)
  • Setting Advanced Recording Options Use the Advanced recording options tab to record custom headers, apply a content type filter, specify non-resources, and set the think-time threshold. Recording Think Time Think time emulates the time that a real user waits between actions. To record user think time, select Record think time. You can specify the minimum amount of time that a user waits that should be recorded as think time by defining a Think time threshold. For example, you can set the think time threshold to 5, so if a user waits for less than five seconds, think time is not recorded. Use the Think Time run-time settings to influence how the Vuser uses the recorded think time when you run the script. Reset Context for Each Action You can instructs VuGen to reset all HTTP contexts between Actions, to their states before the initial recording. This setting allows the Vuser to more accurately emulate a new user beginning a browsing session. It resets all cookies, closes all keep-alive connections, clears the cache, and resets the user-names and passwords. (enabled by default) Creating a Trace Log Select Full trace recording log to create a trace log during recording to use when debugging your script. (disabled by default) Recording Custom Headers Web Vusers automatically send several standard HTTP headers with every HTTP request submitted to the server. Click Headers to instruct VuGen to record additional HTTP headers. You can work in three modes: Do not Record Headers, Record Headers in list, or Record Headers not in list. When you work in the first mode, VuGen does not record any headers. In the second mode, VuGen only records the checked custom headers. If you specify Record headers not in list, VuGen records all custom headers except for those that are checked. By default, headers are not recorded.Filtering Content Type VuGen allows you to filter the content type for your recorded script. You specify the type of the content you wish to record or exclude from your script. You can work in three modes: Do not Filter Content Types, Exclude content types in list, or Exclude content types not in list. When you work in the first mode, VuGen does not filter any content type. In the second mode, VuGen only excludes the selected content types. If you specify Exclude content types not in list, VuGen filters all content type except for the ones that are checked. By default, no filters are active. Handling Non-HTML Resources Many Web pages contain non-HTML elements, such as applets, XML, or javascript. These elements usually contain or retrieve their own resources. For example, a javascriptjsfile, called from the recorded web page, may load several images. An applet may load an external text file. Using the Advanced options, you can control how VuGen records non-HTML elements. The following options are available: Do not record non-HTML elements Record non-HTM elements as separate functions Record non-HTML elements in current HTML function (default)
  • Data gets relocated in the server’s cache when it is reused for each Vuser execution. As a result, server activity is not emulated during playback. While Vuser execution is successful, it will not produce test results meaningful for load testing because the server is not exercised. Using different data better emulates server activity and exercises the system. That improves the chances for meaningful load test results.A date that is hard-coded in the VUser during recording may no longer be valid when that date has passed. Parameterizing the date allows Vuser execution to succeed by replacing the hard-coded date.A field that uses unique data is constrained because the business process will not allow reuse of the data after it has been added to the system and processed.
  • lr_output_message ( "We are on iteration #%s", lr_eval_string ( "{iteration}" ) );
  • Creating Parameters You create a parameter by specifying its name and type. There is no limit to the number of parameters you can create in a Vuser script. There is no limit to the number of records present in a parameter file, however, LR may show only first 100 parameters in view state. In Web Vuser scripts, you can create parameters in the text-based script view using the procedure below, or you can create parameters in the icon-based tree view.
  • Creating Parameters You create a parameter by specifying its name and type. There is no limit to the number of parameters you can create in a Vuser script. In Web Vuser scripts, you can create parameters in the text-based script view using the procedure below, or you can create parameters in the icon-based tree view. To create a parameter: Right-click on the string to parameterize. The right-click pop-up menu opens. From the pop-up menu, select Replace with a Parameter. The Select or Create Parameter dialog box opens. Type a name for the parameter in the Parameter name box, or select an existing parameter name from the list. Select a parameter type from the Parameter type list. Click OK to close the Select or Create Parameter dialog box. VuGen replaces the selected string in your script with the name of the parameter, surrounded by angle brackets. To replace additional occurrences of the string with the same parameter, right-click on the parameter. From the pop-up menu, select Replace More Occurrences. The Search and Replace dialog box opens. The Find What box displays the value you want to replace. The Replace With box displays the parameter name in angle brackets. Select the appropriate check boxes for matching whole words or case. Click Replace or Replace All.
  • Question: write benefits of correlation?
  • The cause of this Vuser execution problem is that the dynamic value returned from the server was recorded and, therefore, was part of the Vuser script. When the script is played back, the Vuser attempts to reuse the recorded value, while the server is expecting another value, assigned for the current playback session.
  • web_reg_save_param("OracleAppJSESSIONID3", "LB/IC=JSESSIONID=", "RB/IC=;", "Ord=1", "Search=headers", "RelFrameId=1", LAST);web_reg_save_param("Siebel_Analytic_ViewState2", "LB/IC=ViewState\" value=\"", "RB/IC=\"", "Ord=1", "Search=Body", "RelFrameId=1", LAST);
  • Emulate different Types of VUsers
  • 1
  • Ref: Communication Foundation (WCF) is a framework for building service-oriented applications. Using WCF, you can send data as asynchronous messages from one service endpoint to another. A service endpoint can be part of a continuously available service hosted by IIS, or it can be a service hosted in an application. An endpoint can be a client of a service that requests data from a service endpoint. The messages can be as simple as a single character or word sent as XML, or as complex as a stream of binary data. A few sample scenarios include:A secure service to process business transactions. A service that supplies current data to others, such as a traffic report or other monitoring service.A chat service that allows two people to communicate or exchange data in real time.A dashboard application that polls one or more services for data and presents it in a logical presentation. Exposing a workflow implemented using Windows Workflow Foundation as a WCF service.A Silverlight application to poll a service for the latest data feeds.
  • You can instruct LoadRunner to execute a scenario or Vuser group with a delay. The time duration of a scenario or Vuser group can be limited. The scenario or Vuser group stops running after the elapse of this time.You can stipulate how many Vusers LoadRunner runs within a certain time frame during a scenario or Vuser group.
  • i.e. no way to keep on fluctuating user ramp up/ramp down behavior
  • Once a problem has been isolated, a corroborative effort involving developers, DBAs, network, and other systems experts is required to fix the bottleneck. After adjustments are made, the load test is repeated to confirm that the adjustments had the desired effect. You repeat this cycle to optimize system performance.
  • Ref:
  • Performance Testing using LoadRunner - Kamran Khan []

    1. 1. A comprehensive learning experience for performance engineers Presented by: Kamran Shaukat Ali Khan
    2. 2. HP LoadRunner - Kamran Khan – 2
    3. 3. DownloadLatestVersionofthisFile!  This file was downloaded from:  Download latest version of the slides from above URL.  Feel free to distribute the file and share knowledge  The file is available as Read-Only and does not require password to open.  View in Presentation Mode – the slides animation is designed purposely to nurture audience understanding  Direct your feedback to above URL or to author at HP LoadRunner - Kamran Khan – 3
    4. 4. CourseObjectives Discuss the value of load testing Plan for effective load test Establish load test goals Run load test scenarios Load and overload when executing scenarios Analyze and interpret load test results Prerequisites: Understanding of: Client/Server,Windows,Web Course Methodology Slides, Experience sharing Quizzes, Assignments HP LoadRunner - Kamran Khan – 4
    5. 5. Trainer’sProfile  9+ years of IT experience as lead engineer, specializing in SQA, SoftwareTesting andTest Automation  Vast experience of working with LoadRunner, winRunner, QTP, TestComplete ,WatchFire and various other performance tools  Proficient with Load testing of SOA based, Net Centric web- based products, web and RIA applications  Vast experience of conducting technical trainings – both internally and externally.  BSCS Hons., MBA (Marketing)  Certified:CCNA, AIS HP Performance Center, LoadRunner, QTP  Expertise Areas:  QMS,TQM, AutomatedTesting usingWinRunner & QTP,TestComplete PerformanceTesting using HP Performance Center, LoadRunner, IBM Rational PerformanceTester, JMeter and others. PDA apps testing HP LoadRunner - Kamran Khan – 5
    6. 6. PerformanceTesting Lets get started…
    7. 7. HP LoadRunner - Kamran Khan – 7
    8. 8. Facts  Fact #1 :  Most users click away after 8 second delay  Fact #2 :  $4.4 billion in revenue lost annually – due to poor web performance Source: Zona Research …continuedHP LoadRunner - Kamran Khan – 8
    9. 9. NewtoLoadRunner Reconfirm your core skills.The core skills which are needed for performance testing are a large superset of those needed for automated functional testing.A search of this forum [LR-LoadRunner at Google] and others will result in lists of what those core skills are. After you have confirmed those core skills, attend standard product training. Don't scrimp.Take the classes from an HP CertifiedTrainer using HP provided material. Some will argue whether this training is the best possible training and they are probably right on that argument, but this training has the most consistent delivery with the most consistently skilled at the end of the course. Post training, work with a mentor. Becoming a performance tester is not unlike becoming a plumber, an electrician, a barber/cosmetologist.There are core skills, a period of tool instruction and then a journeyman phase where you are working under the supervision of a master in the profession who can help you bring together skill, technique and process to mature you as a performance tester.This period will likely last three- five performance testing projects. If you cannot stand on your own after five then you should probably head to another profession:This is not a profession for everyone. After mastering the core, begin giving back. Become a mentor to the next generation. If your management will not commit to affirming your core skills (and touching them up where necessary), training and mentoring then you should look for a new organization/set of management. It is a well established pattern that such managers will throw you under the bus to salvage a customer relationship where a performance testing gig has gone south. It is not a question of "if," only "when" when you have such management in place. Go in with eyes wide open. HP LoadRunner - Kamran Khan – 9
    10. 10. Reground yourself in application architecture, particularly client-side architectures and communications methods. Expect that with training and a mentor this will be an 18 month to 2 year path for you to become self sufficient. There are thousands on the market with LoadRunner on their resume without training and mentoring that even after five years of "work" have almost zero usable skills – they have been able to skate by in environments where they know marginally more than the manager/client who is not capable of evaluating their work. The testing world, by attempting to equate the skill sets of automated testers and performance testers has pushed many thousands of automated testers into performance testing roles for which they are ill-equipped.They lack the foundation process and mechanical skills to be successful. They lack the analytical skills to actually find the issues. Heck, you would be hard pressed to find many who can give you a straight answer on what a performance requirement actually looks like and how to evaluate whether the requirement is sound or not.All of this has led to an odd economic situation: For the past ten years the demand for performance testing skills has been rising, but the average compensation has been dropping. In a normal market with a skills scarcity rates would rise and bring more skilled people into the market. But in this case the flood of low skilled low value individuals has actually caused value to drop at a rate faster than demand is rising, leading to the odd economic situation of falling rates in a scarce resource high demand market. This is why the industry is "more fishy" due to this behavior… James Pulley, LR-LoadRunner, 1st October, 2011` HP LoadRunner - Kamran Khan – 10
    11. 11. Automated Testing Put contents here HP LoadRunner - Kamran Khan – 11
    12. 12. AutomatedPerformance Testing Performance testing is the discipline concerned with determining and reporting the current performance of a software application under various parameters Automated performance testing is about applying production workloads to pre- deployment systems while simultaneously measuring system performance and end- user experience. HP LoadRunner - Kamran Khan – 12
    13. 13. PerformanceEngineering Performance testing is the discipline concerned with determining and reporting the current performance of a software application under various parameters Performance engineering is the process by which software is tested and tuned with the intent of realizing the required performance.This process aims to optimize the most important application performance trait, user experience. Historically, testing and tuning have been distinctly separate and often competing realms. In the last few years, however, several pockets of testers and developers have collaborated independently to create tuning teams. Because these teams have met with significant success, the concept of coupling performance testing with performance tuning has caught on, and now we call it performance engineering. HP LoadRunner - Kamran Khan – 13
    14. 14. WhyLoadRunner? Benefits of choosing LoadRunner  LR Credibility - Industry Standard (77% market share)  Meeting the criteria  Scalable [supports add-in]  Integration with QTP  Integration with WR  Scenario modification  Others:  Record n Play, Custom Code, Errors and Exceptions Envision - Direct Revenue in long term  Off shore services  Outsourced services HP LoadRunner - Kamran Khan – 14
    15. 15. LimitationsfosteringLTteam Current Limitations  Less resources available in market  Trade off between Workload /Training others  ZERO Support from vendor  99% unplanned (ad hoc) activities*  Stakeholders have incomplete, obscured or no knowledge regarding Automation / LoadTesting  Deficient in Real Environment – scaling requires heuristics Tool cons  Technology dependent  Initial scripting would be decisive  Different License for Desktop,Web, RIA and database protocols  Not all licenses can be installed at a time HP LoadRunner - Kamran Khan – 15
    16. 16. AutomatedTesting:Objectives Drawbacks of manual testing Benefits of AutomatedTesting  in Functional RegressionTesting  in LoadTesting WhichTest Case not to automate  in Functional RegressionTesting  in LoadTesting ManualVs AutomatedTesting process AutomatedTest Planning CommonTesting Objectives Improve current framework by spreading awareness of: - technological constraints - due responsibilities of other stakeholders HP LoadRunner - Kamran Khan – 16
    17. 17. PerformanceTestingtypes What is PerformanceTesting  Stress testing  Load testing  Volume testing  Soak / Endurance testing  … When:  When to test under load Why:  Why to automate  Why not to automate What:  What to automate  What not to automate HP LoadRunner - Kamran Khan – 17
    18. 18. PotentialImpactofLoadTestingonSDLC Impact on Process  SDLC  TLC  … RegressionTesting + fixing + verification = STABLE  [LoadTesting]+ fixing  SAT  UAT HP LoadRunner - Kamran Khan – 18
    19. 19. AutomatedPerformanceTesting automated performance testing is about applying production workloads to pre-deployment systems while simultaneously measuring system performance and end-user experience ! Load: Number of concurrent users Web Site Response Time Good (0 - 8 sec.) Borderline (9 – 15 sec.) Unacceptable (15 – 30 sec.) Virtually Offline (> 30 sec.) The 8 Second Barrier Crash! Source: Presentation by LloydTaylor, VPTechnology &Operations, KeynoteSystems HP LoadRunner - Kamran Khan – 19
    20. 20. Why loadtestyourapplication? Failure of Mission Critical product can be costly. Deploy with confidence:  assure business transactions work real-world conditions  Locate and resolve potential problems before customer identifies them  Scalability Testing buys us confirmation of predictions about what will happen before it actually happens, buying the lead time to do the right thing when additional capacity is needed. HP LoadRunner - Kamran Khan – 20
    21. 21. Why loadtestyourapplication? Performance is the key to success of any web based application in specific and all others in general (mobile/desktop) We will get to know the maximum capacity the system can handle for an application. We can decide whether we should go for Hardware upgrades or Performance tuning. Understand the issues involved in load testingWeb applications. Learn your responsibilities as the LoadRunner expert. Learn how to perform successful load tests using LoadRunner. HP LoadRunner - Kamran Khan – 21
    22. 22. Loadtesttypicalobjectives Many concurrent users running the same application to see whether a system handles the load without compromising functionality or performance. How many concurrent users can the system handle without increase in the expected response time? Can the system serve [x] concurrent users without any errors? System works fine with X connections for Z number of hours. HP LoadRunner - Kamran Khan – 22
    23. 23. Drawbacksofmanualtesting •Repetitive •Requires excessive resources •May lack coverage •Potentially inconsistent •Monotonous •Sometimes Impossible Time MoneyResources HP LoadRunner - Kamran Khan – 23
    24. 24. Whytousetestingtools Time Consuming Inconsistent Low Reliability Expensive  Human Resource  Machines Complicated  Synchronize testers  Analyze results meaningfully  Speed  Repeatability / Consistent  Coverage  Programming capabilities  Reliability  Reusability  Control all users—from a single point of control HP LoadRunner - Kamran Khan – 24
    25. 25. Whenautomationisapplicable When a system combines a variety of software applications and hardware platforms to determine the suitability of a server for any given application To test the server before the necessary client software has been developed Emulate an environment where multiple clients interact with a single server application test an application under the load of tens, hundreds, or even thousands of potential users Test Repeatability is required Automated Functional RegressionTesting  MercuryWinRunner / HPQTP etc. Automated Stress/Performance/SoakTesting  HP LoadRunner / IBM Rational PerformanceTesterv HP LoadRunner - Kamran Khan – 25
    26. 26. WhichTestCasetoautomate? Tests that need to be run for every build of the web site (sanity level)  FRT Tests that use multiple data values for the same actions (data driven tests)  FRT Identical tests that need to be executed using different browsers  FRT Mission-critical pages  FRT / LT Transaction with pages which won't change in short term  FRT HP LoadRunner - Kamran Khan – 26
    27. 27. WhichTestCasenottoautomate? Usability testing –"How easy is the web site to use?" "ASAP" testing –"We need to test NOW!" Ad hoc/random testing –based on intuition and knowledge of web site FunctionalTesting Business validation HP LoadRunner - Kamran Khan – 27
    28. 28. BetterCandidate Business Processes which:  Involves large number of database tables  Single database table with large entry  Most frequently opened database tables  Requires big data to reside in RAM  Involves high processing like financial calculations HP LoadRunner - Kamran Khan – 28
    29. 29. HowLoadRunnerdoesitall? LoadRunner monitors the application performance online, enabling you to fine-tune your system during test execution. LoadRunner automatically records the performance of the application during a test.You can choose from a wide variety of graphs and reports to view the performance data. LoadRunner checks where performance delays occur:  network or client delays  CPU performance  I/O delays  database locking  network and server resources  or other issues at the database server. Because LoadRunner tests are fully automated, you can easily repeat them as often as you need. HP LoadRunner - Kamran Khan – 29
    30. 30. PerformanceTesting:ATeamWork Project Manager: - Manages the implementation of website/SUL Functional Expert / Business Analyst: - Provide Usage Analysis of SUL & provides expertise on business functionality of website/SUL AutomatedTesting Expert: - Creates the automated tests & executes load scenarios System Architect: - Provides Blueprint of SUL Web Developer / SME: - Maintains website & provide monitoring aspects - Develops website and fixes bugs System Administrator: - Maintains involved servers throughout testing project HP LoadRunner - Kamran Khan – 30
    31. 31. Manual VsAutomatedTestplan HP LoadRunner - Kamran Khan – 31
    32. 32. Planningaloadtest What to Plan?  Understand the functional testing objective(s) of the AUT  Know the user actions of each business transaction of the AUT  Know what input data to use for each transaction  Know what to verify in the web site HP LoadRunner - Kamran Khan – 32
    33. 33. PlanningAdvantages HP LoadRunner - Kamran Khan – 33
    34. 34. CommonTestingObjectives General page layout  Frames  Images  Tables FunctionalTesting  of each transaction using different sets of valid data across different browsers BusinessTesting  core business or objective of any application being developed RegressionTesting  hardware and software upgrades  web site enhancements HP LoadRunner - Kamran Khan – 34
    35. 35. PageLayoutVerification! HP LoadRunner - Kamran Khan – 35
    36. 36. AutomatedTest:Objectives Purpose of understanding test objectives:  Create effective automated tests that meet test requirements  Gather the relevant information that will help you create the automated test  System Architecture  System Components  Users profile (application users)  Clients behavior with application (TDD)  3rd party components, integration and other technical details HP LoadRunner - Kamran Khan – 36
    37. 37. TestCasesforPerformanceTesting DocumentTest Information  Documentation that contains all gathered information relevant to the testing requirement  Application  Transaction  User actions  Test requirement(s) HP LoadRunner - Kamran Khan – 37
    38. 38. TestCasesforPerformanceTestingcont. Certain information is “a must” when preparing for LoadTesting. This includes:  Testing environment  Operating System (Windows XP Server 2002 SP2)  Hardware Specifications (Intel® Pentium 4 CPU 3.00 GHz, 2 GB RAM)  SQL Server 2005, IIS 6.0 etc  Databases with user data (preferably client DB replica)  Representatives from Dev,Testing, AT, Networks Obtain master data – extract from database  obtain help from domain experts Create valid transactional data  predict according to type of data  obtain from Functional Expert HP LoadRunner - Kamran Khan – 38
    39. 39. VitalingredientsinOrganizing aTest •The architecture of the system •A brief idea about the application •The list of business transactions •The time at which the system has peak load •The acceptable response time for the transactions •No. of concurrent users the system is expected to handle during peak load • Select theTransactions to automate • Decide on theTransaction Mix • Obtain User details for the transactions •The tables these transactions query •The web server / application server components invoked by the transactions HP LoadRunner - Kamran Khan – 39
    40. 40. VitalingredientsinOrganizing aTest Select the transactions • Give priority to business critical transactions • Give priority to transactions that have high database activity • Give priority to transactions that have high web server or application server activity • Know the peak load time Decide onTransaction Mix • Impact of the failure of a particular transaction on business • The transactions that are executed during peak load • The no. of times they are executed during peak load User details • For each of the chosen transactions find • Who are the users who executing these transaction? • How many simultaneous and concurrent users are there? HP LoadRunner - Kamran Khan – 40
    41. 41. VitalingredientsinOrganizingaTest For each of the chosen transactions find: • The tables which a transaction queries • The tables which a transaction updates Server components invoked: • The servlet / jsp that are invoked by the transaction. Environment specifications: • Application architecture • HW specifications of machines where application is hosted • SW specifications of machines where application is hosted • HW/SW specifications of machines where load will be simulated HP LoadRunner - Kamran Khan – 41
    42. 42. Summary • Understanding of various stages of PerformanceTest automation • Planning should be done before automation begins • Proper planning can increase your test coverage and reduce test development time • Understanding the testing objectives helps devise effective automated tests • Identifying better candidates for performance testing • Document all relevant test information and team representatives HP LoadRunner - Kamran Khan – 42
    43. 43. Q&A ThankYou ! HP LoadRunner - Kamran Khan – 43
    44. 44. Quiz 15 Minutes HP LoadRunner - Kamran Khan – 44
    45. 45. HPLoadRunner Architecture
    46. 46. HP LoadRunner - Kamran Khan – 46
    47. 47. What'sNext Understand the architecture of LoadRunner itself To describe best practices of carrying efficient performance tests HP LoadRunner - Kamran Khan – 47
    48. 48. HPLoadRunner Works HP LoadRunner - Kamran Khan – 48
    49. 49. HPLoadRunnerArchitecture Put contents here HP LoadRunner - Kamran Khan – 49
    50. 50. LRStructure HP LoadRunner - Kamran Khan – 50
    51. 51. HighlevelProcessoverview Put contents here HP LoadRunner - Kamran Khan – 51
    52. 52. PlanningRoadMap Install LoadRunner GatherTeam Define Goals Outline Business Processes Define User Profiles Document Business Processes DefineTest Data Management Procedures Outline Monitors Document HP LoadRunner - Kamran Khan – 52
    53. 53. GoalsofPerformanceTesting Conceptual Goals  What are we trying to achieve? SUL Mission CriticalTransaction ResponseTime  Is Booking Flight transaction fast enough?  xyz business process supports 250 concurrent users SUL Environment Benchmarking  Are servers, network, software, etc. compatible?  100 concurrent users can file contracts for 1 hour if web server RAM is 1GB HP LoadRunner - Kamran Khan – 53
    54. 54. BusinessProcesses Mission Critical  What BP malfunctioning will put you out of business? Throughput  How many users going through identified MissionCritical BPs? Dynamic Contents  How much data each transaction process through the SUL HP LoadRunner - Kamran Khan – 54
    55. 55. SystemUsers SUL User Profile  What are the SUL users: level of experience with the System SUL Users Regions  Where all SUL users located: America, Pakistan, KSA? SUL Users Concurrency  What is a pick of SUL usage per BP at given hour? User Roles (Access rights on system)  What types of user roles are in practice. Manager,TL, Dealer, Auditor etc. HP LoadRunner - Kamran Khan – 55
    56. 56. DocumentingBPs User Action Steps  Should BP start with User Login?  Can we navigate through the BP manually using these steps? SUL Navigation Expected Results  What should we include in the Checkpoint once this step performed? BPTransactions  What trigger in the BP will be measured for response time? ThinkTime  How long user waits till the next step in the BP? Parameters  What static input data should be replaced with dynamic content?  What type of parameter should it be? Data file, date, random?  What parameter sequence should be set? HP LoadRunner - Kamran Khan – 56
    57. 57. DataManagement Test Data Needed  What data will be needed (User Names & Passwords, SSN #, SKU # etc.)? Procedures to Generate Data  How are we going to generate data (querying database, data processing department, etc.)? Generated Data format  What format data will be coming in (Flat file, CSV, etc.)? HP LoadRunner - Kamran Khan – 57
    58. 58. PerformanceMonitors Investigate SUL Environment Set Monitor per each SULComponent Add measure point gradually  Set default measurements  Add measurements as needed  Consult with components SME HP LoadRunner - Kamran Khan – 58
    59. 59. Whatconstitutesasystem! HP LoadRunner - Kamran Khan – 59 Database Web Servers / Application Servers Operating Systems External Storage Front End GUI Clusters/ Load Balancers Memory Threads Kernal Heap Third Party Components
    60. 60. Putittogether Document all findings Approve by all Stakeholders HP LoadRunner - Kamran Khan – 60
    61. 61. AutomatedTestingProcess HP LoadRunner - Kamran Khan – 61
    62. 62. ATProcess:Planning Step I: Planning theTest Successful load testing requires that you plan to carry out certain business process.A BP consists of clearly defined steps in compliance desired business transactions – so as to accomplish your load testing objectives. BusinessTransaction: Examples:  Reserving flight tickets  Buying a computer from a online store  Logging into your web-based email service HP LoadRunner - Kamran Khan – 62 A business transaction is a set of user actions performed with an application to accomplish a business task.
    63. 63. ATProcess:Scripting Step II: Creating theVuser Scripts VUsers emulate human users interacting with yourWeb-based application. AVUser script contains the actions that eachVUser performs during scenario execution. In eachVUser script, you determine the tasks that will be: performed by eachVuser performed simultaneously by multipleVusers measured as transactions For more information on creatingVUser scripts, see the HPVirtual User Generator User Guide. HP LoadRunner - Kamran Khan – 63
    64. 64. ATProcess:CreateScenario Step III: Creating the Scenario A scenario describes the events that occur during a testing session. A scenario includes a list of machines on whichVusers run, a list of scripts that theVusers run, and a specified number ofVusers orVuser groups that run during the scenario.You create scenarios using the Controller. Creating a Manual Scenario You create a scenario by definingVuser groups to which you assign a quantity of individualVusers,Vuser scripts, and load generators to run the scripts. You can also create a scenario using the Percentage Mode, in which you define the total number ofVusers to be used in the scenario, and the load generators and percentage of the total number ofVusers to be assigned to eachVuser script. Creating a Goal-Oriented Scenario ForWeb tests, you can create a goal-oriented scenario, in which you define the goals you want your test to achieve. LoadRunner automatically builds a scenario for you, based on these goals. HP LoadRunner - Kamran Khan – 64
    65. 65. ATProcess:RunScenario Step IV: Running the Scenario You emulate user load on the server by instructing multipleVusers to perform tasks simultaneously.You can set the level of load by increasing and decreasing the number ofVusers that perform tasks at the same time. Before you run a scenario, you set the scenario configuration and scheduling.This determines how all the load generators andVusers behave when running a scenario. You can run the entire scenario, groups ofVusers (Vuser groups), or individualVusers.While a scenario runs, LoadRunner measures and records the transactions that you defined in eachVuser script.You can also monitor your system’s performance online. HP LoadRunner - Kamran Khan – 65
    66. 66. ATProcess:MonitorScenario StepV: Monitoring a Scenario You can monitor scenario execution using the LoadRunner online run-time, transaction, system resource,Web resource,Web server resource,Web application server resource, application server resources, database server resource, network delay, streaming media resource, firewall server resource, ERP/CRM server resource, Java performance, J2EE & .NET Diagnostics, application deployment, middleware performance, application component, and infrastructure resources monitors. HP LoadRunner - Kamran Khan – 66
    67. 67. ATProcess:Analysis StepVI: AnalyzingTest Results During scenario execution, LoadRunner records the performance of the application under different loads.You use LoadRunner’s graphs and reports to analyze the application’s performance. For more information about LoadRunner’s reports and graphs, see the HP LoadRunner Analysis UserGuide. HP LoadRunner - Kamran Khan – 67
    68. 68. Q&A ThankYou ! HP LoadRunner - Kamran Khan – 68
    69. 69. Quiz 15 Minutes HP LoadRunner - Kamran Khan – 69
    70. 70. HPLoadRunner Virtual User Generator
    71. 71. HP LoadRunner - Kamran Khan – 71
    72. 72. What'sNext UnderstandVirtual UserTechnology Record a scripts General options Recording options Runtime options Replay recorded scripts HP LoadRunner - Kamran Khan – 72
    73. 73. VirtualUserTechnology Simulates a real-like virtual-user Programming capabilities with greater control over test execution Can synchronize actions performed by users VUsers can communicate directly with a server by executing calls to the server API-without relying on client software Captures a business transaction by recording user actions performed Monitors the communication between the application and the server HP LoadRunner - Kamran Khan – 73
    74. 74. Beforewestart… Scenario:  A scenario defines the events that occur during each testing session.Thus, for example, a scenario defines and controls the number of users to emulate, the actions that they perform, and the machines on which they run their emulations. VUsers:  In the scenario, LoadRunner replaces human users with virtual users or Vusers.When you run a scenario,Vusers emulate the actions of human users working with your application.While a workstation accommodates only a single human user, manyVusers can run concurrently on a single workstation. In fact, a scenario can contain tens, hundreds, or even thousands ofVusers. VUser Scripts:  The actions that aVuser performs during the scenario are described in a Vuser script.When you run a scenario, eachVuser executes aVuser script. TheVuser scripts include functions that measure and record the performance of your application’s components. Others:  System, ResponseTime,ThinkTime,Work Load,Transaction Mix, Throughput HP LoadRunner - Kamran Khan – 74
    75. 75. Beforewestart… Transactions  A transaction represents an action or a set of actions that you are interested in measuring.You define transactions within yourVuser script by enclosing the appropriate sections of the script with start and end transaction statements. For example, you can define a transaction that measures the time it takes for the server to process a request to view the balance of an account and for the information to be displayed at theATM. Rendezvous points:  You insert rendezvous points intoVuser scripts to emulate heavy user load on the server.  Rendezvous points instructVusers to wait during test execution for multipleVusers to arrive at a certain point, so that they may simultaneously perform a task. For example, to emulate peak load on the bank server, you can insert a rendezvous point instructing 100Vusers to deposit cash into their accounts at the same time. Controller:  You use the HP LoadRunnerController to manage and maintain your scenarios. Using the Controller, you control all theVusers in a scenario from a single workstation. Load Generator:  When you execute a scenario, the Controller distributes eachVuser in the scenario to a load generator.The load generator is the machine that executes theVuser script, enabling theVuser to emulate the actions of a human user. Performance analysis:  VUser scripts include functions that measure and record system performance during load-testing sessions. During a scenario run, you can monitor the network and server resources. Following a scenario run, you can view performance analysis data in reports and graphs. HP LoadRunner - Kamran Khan – 75
    76. 76. VUGen:RecordingmyFirstScriptinLR Click File  New Find the desired protocol from the list of protocols and click Create HP LoadRunner - Kamran Khan – 76
    77. 77. VUGen:ScriptSections HP LoadRunner - Kamran Khan – 77 Script Section Used when recording... Is executed when... vuser_init a login to a server theVuser is initialized (loaded) Actions client activity theVuser is in "Running" status vuser_end a logoff procedure theVuser finishes or is stopped Script View Script Sections Tree View
    78. 78. RecordingOptions… Tools  Recording Options [Ctrl + F7] … General  Recording  • HTML Mode • URL Mode This allows us to specify the information to be recorded and which functions to be used when generating a script HP LoadRunner - Kamran Khan – 78
    79. 79. HTMLMode–URLModecomparison The HTML mode is based on user actions, and the scripts contain functions that correspond directly to the action taken Example: ... web_link(“Enterprise Systems Performance", "Text=Enterprise Systems Performance ", "Snapshot=t4.inf", LAST); HP LoadRunner - Kamran Khan – 79 The URL mode is based on HTTP requests sent to the server as a result of user actions. Example: ... web_url(“Enterprise Systems Performance", "URL=", "TargetFrame=", "Resource=0", "RecContentType=text/html", "Referer= . . , "Snapshot=t4.inf", "Mode=URL", LAST);
    80. 80. DecidingonRecordingMode Deciding on Modes For browser applications, use HTML mode. For non-browser applications, use URL mode. HP LoadRunner - Kamran Khan – 80 The option of mixing recording modes is available for very advanced users for performance tuning HTML Mode URL Mode Intuitive and easy to understand Not as intuitive as the HTML scripts Scalable More scalable and effective for creating a load test
    81. 81. OtherRecordingOptions HP LoadRunner - Kamran Khan – 81
    82. 82. DirectoryStructureofanLRScript HP LoadRunner - Kamran Khan – 82 VuGen creates a series of config. Files, data files and source code files which containVUser run-time and setup information. The results of each iteration of a script are stored separately. The Iteration0 directory is for system use and should be ignored by the tester.
    83. 83. ResponseTime HP LoadRunner - Kamran Khan – 83 App Server DB Server Load Injector Time in which the system responds for a particular Transaction request
    84. 84. ThinkTime HP LoadRunner - Kamran Khan – 84 Time taken for selecting a new Transaction after the response for the previous Transaction has been received Transaction 1 Transaction 2 ThinkTime = 10 Seconds
    85. 85. Workload HP LoadRunner - Kamran Khan – 85 Database Web Servers / Application Servers Operating Systems External Storage Front End GUI The System Typical Workload No. of Users Transactions Batch Programs Background Load
    86. 86. TransactionMix HP LoadRunner - Kamran Khan – 86 Varying execution frequency of different Transactions Credit Loans Debit Query Account Info Account Opening
    87. 87. HPLoadRunnerVUserTechnology On each Windows load generator, you install the Remote Agent Dispatcher (Process) and a LoadRunner Agent Remote Agent Dispatcher:  The Remote Agent Dispatcher (Process) enables the Controller to start: applications on the load generator. Agent:  The LoadRunner Agent enables the Controller and the load generator to communicate with each other.When you run a scenario, the Controller instructs the Remote Agent Dispatcher (Process) to launch the LoadRunner agent.The agent receives instructions from the Controller to initialize, run, pause, and stop Vusers. At the same time, the agent also relays data on the status of the Vusers back to the Controller. HP LoadRunner - Kamran Khan – 87
    88. 88. VUGen-Introduction Main components ofVuGen interface  StartingVuGen  Environment options (Protocols)  Script view  Tree view  Output window etc HP LoadRunner - Kamran Khan – 88
    89. 89. Stepstowardsabaselinescript Commencement  Selecting right combination of protocols  Protocol Advisor (9.5 or newer) Record  Conventions and good practices  Prepare andAssure availability ofTest data  Understanding script  Manual verification Playback andVerification  Through subject application Parameterize/Correlation Enhance  Rendezvous,Transactions, Output message, Image/Text check, Correlation, and additional steps, Configuring Pacing, Iterations, Actions, Log/Error messages &Think time and more Runtime Settings  Pacing, Iterations,Think time and more Prepare for Execution  Prepare andAssure availability ofTest data HP LoadRunner - Kamran Khan – 89
    90. 90. RecordingfortheWeb Example Exercise  Create vuser script  Understand vuser script  Generation log  Script regeneration User authentication information  web_set_user("domainuser, “password", “host:port");  web_set_user(“chrometechkhankamran", "password123(", “machine112:80"); HP LoadRunner - Kamran Khan – 90
    91. 91. Replayingarecordedscript Verify the baseline script Identify and configure appropriate runtime settings for replay Replay vuser script Replay log Verify the application behavior HP LoadRunner - Kamran Khan – 91
    92. 92. ScriptEnhancement Transactions Comments Parameters Rendezvous points Text / Image verification Adding Actions Correlation HP LoadRunner - Kamran Khan – 92
    93. 93. Transactions  A mean to measure time taken by application or its server side component in entertaining certain client request.  Or, the time to complete specific takes or activity Defining transactions  Preferably during recording else after recording Syntax:  lr_start_transaction(“Transaction Name”);  lr_end_transaction(“Transaction Name”, <status>);  e.g. lr_start_transaction(“Transaction Name”); lr_end_transaction(“Transaction Name”, LR_AUTO); lr_end_transaction(“Transaction Name”, LR_PASS); lr_end_transaction(“Transaction Name”, LR_FAIL); HP LoadRunner - Kamran Khan – 93
    94. 94. Transactions cont. Best Practice:  Assign Script ID as prefix to the name  Assign transaction number right before logical name  e.g. 0102Login  Removal any immediate following think time statement  Remove or document think time statements within a transaction  Period (.) character is not allowed in transaction name HP LoadRunner - Kamran Khan – 94
    95. 95. SubTransactions /* Notify that a transaction is starting */ lr_start_transaction("purchases"); /* Breakdown the transaction into a smaller transaction */ lr_start_sub_transaction("electrical_purchases", "purchases"); status = purchase_electrical_items(6); /* call to Server API */ /* End transaction with operation result - pass or fail */ if (status == 0) lr_end_sub_transaction("electrical_purchases", LR_PASS); else lr_end_sub_transaction("electrical_purchases", LR_FAIL); HP LoadRunner - Kamran Khan – 95
    96. 96. Comments Add comments for  To describe an activity  To provide information about specific operation  To separate two sections for distinction Add comments:  Comments during recording  Comments after recording Best Practice:  Mark any comments on the top of each script file HP LoadRunner - Kamran Khan – 96
    97. 97. RendezvousPoints Rendezvous Points  Introduce concurrency Defining transactions  during recording or after recording Syntax: lr_rendesvous(“Logical Name”); Best Practices:  Prefix a rendezvous point with “rdv_”  Use logical name of transaction being subjected  e.g. rdv_Login  Removal any immediate following think time statement HP LoadRunner - Kamran Khan – 97
    98. 98. InsertingFunctionthroughmenu Log messages Debug messages Error messages Output messages  Choose Insert > New Step.The Add Step dialog box opens. Scroll down and select Output Message and clickOK.  lr_output_message  lr_log_message HP LoadRunner - Kamran Khan – 98
    99. 99. Parameterization A parameter is a placeholder which re-places a recorded value in aVuser script. At run time, a value from an external source is substituted for the parameter against record time value. Parameterization solves certain problems that may occur during playback: • Date constraints make recorded date invalid • Unique constraints prevent reusing recorded data • Data caching • Response times are inaccurate • Real user activity is not emulated • Servers are not exercised Problem examples: Business process works only for the current date Unique ID required as in case of Primary Key or fields in database with Unique constraint How to measure the performance of the process which creates a new product? HP LoadRunner - Kamran Khan – 99
    100. 100. Parameterization What is Parameterization? Which value to parameterize? How to parameterize? Properties of a Parameter? Caution – line feed at the end of each line in data file is mandated space i.e. ‘%20’ is ignored by LR Types of parameters  Date/Time  File  Group name  Random number  Iteration number  Unique number  XML  Vuser ID  Load generator name  User defined function lr_output_message ( "We are on iteration #%s", lr_eval_string ( "{iteration}" ) ); HP LoadRunner - Kamran Khan – 100
    101. 101. Parameterconfiguration Important options regarding properties of File /Table parameters  Data assignment method  Update method  When out of values  Columns  Rows  When not enough rows HP LoadRunner - Kamran Khan – 101
    102. 102. Whyexcessiveparameterization! Problem: Since allVUsers use the same data, Data caching at servers makes load test response times shorter than when real users load the system Solution: Parameterize input data to force the system to get data from a database Data gets relocated in the server’s cache when it is reused for eachVuser execution. As a result, server activity is not emulated during playback.While Vuser execution is successful, it will not produce test results meaningful for load testing because the server is not exercised. Using different data better emulates server activity and exercises the system.That improves the chances for meaningful load test results. HP LoadRunner - Kamran Khan – 102
    103. 103. CreatingDataPool How to create data pool to be used in parameterization files? 1. Extract master data from database 1. Write queries on tables being accessed by the business transactions. 2. Spool data to a file 3. Write AWK scripts to generate a proper data pool from the spooled file 2. Create valid transactional data 1. Predict according to type of data 2. Obtain from a functional expert HP LoadRunner - Kamran Khan – 103
    104. 104. Parameterization–Process Determine which fields to parameterize Replace recorded values with parameters Decide which parameter type to use Choose the data access method and the number of iterations Run theVuser and analyze the results to verify correct execution HP LoadRunner - Kamran Khan – 104
    105. 105. Parameterization–Process cont. Right click on the data to parameterize select “Replace with a Parameter” The Select or Create Parameter dialog box opens. Type a suitable Parameter Name and Select a ParameterType Right-click the parameter name and select “Replace More Occurrences” to replace the data by the parameter name HP LoadRunner - Kamran Khan – 105
    106. 106. Parameters–Exercise Data table values:  Nida; Kamran; Michael; Jane; Ron; Alice; Ken; Julie; Fred 12 vusers and 3 iterations Try all combinations of assignment method and update method on provided data HP LoadRunner - Kamran Khan – 106
    107. 107. AdvancedScriptingTechniques Other protocols’ specific functions Correlation LR Standard Functions  variable_name = lr_eval_string(“{variable_name}”);  lr_save_string(“value”, “variable_name”); Protocol Specific Functions HP LoadRunner - Kamran Khan – 107
    108. 108. Correlation Many applications use dynamic values that change each time you use the application. For example, some servers assign a unique session ID for every new session. When you try to replay a recorded session, the application creates a new session ID that differs from the recorded session ID. To simplify / optimize code To generate dynamic data To accommodate unique data records Protocol Specific Functions:  variable_name = lr_eval_string(“{variable_name}”);  lr_save_string(“value”, “variable_name”); HP LoadRunner - Kamran Khan – 108
    109. 109. Correlation  In applications output of the previous transaction may be used as an input for the next transaction.  For example:  System generated session ids  URLs that change each time you access theWeb page  Fields (sometimes hidden) recorded during a form submission  The data is only good for the current session  Solution using Correlation:  Capture output value from one preceding step  Use captured value as input to proceeding step HP LoadRunner - Kamran Khan – 109
    110. 110. Dynamic data is correlated automatically by the client application (running in the browser) while recording a script Correlated data is data which is sent to the client from the server, and later sent back to the server by the client. HP LoadRunner - Kamran Khan – 110 Network Web Server Client Step 2 - Request [Session ID = 22222] Step 1 [Session ID = 22222]
    111. 111. Reusing recorded dynamic data will cause errors when you replay the script Recorded dynamic data must be correlated using parameters HP LoadRunner - Kamran Khan – 111 Network Web Server Client Step 2 - Request [Session ID = 22222] Step 1 [Session ID = 23212]
    112. 112. Correlationtypes Auto correlation  LR detects and help solving them Manual correlation  LR detect but doesn’t provide direct interface to correlate  LR may not detect code requiring correlation Question: when manual correlation is required? Question: when auto correlation is required? HP LoadRunner - Kamran Khan – 112
    113. 113. ManualCorrelation 1. Determine the value to capture 2. Find the right and left text boundaries of the value to capture (WEB) 3. Find which occurrence of the text boundaries should be used 4. Add a web_reg_save_param function to the script, above the step which requests the page with the value to capture 5. Add the parameter name, left boundary, right boundary, and occurrence to the function 6. Parameterize the dynamic value in the script every time it occurs 7. Verify correct execution HP LoadRunner - Kamran Khan – 113
    114. 114. AutomaticCorrelation There are three types of automatic correlation solutions 1. Auto-Detect Correlation 2. Rule-Based Correlation 3. CorrelatingAll Statements HP LoadRunner - Kamran Khan – 114 Rule Name When to Use Auto-detect Correlation Detect and correlate dynamic data for supported application servers Rule-Based When working with a non-supported application server whose context is known. Correlate All Blindly correlate all dynamic data.
    115. 115. AutomaticCorrelation Auto-Detect Correlation Automatically correlate the recorded statements CorrelatingAll recognizable statements Rule-BasedCorrelation Rule-based correlation requires correlation rules to be defined before recording a session HP LoadRunner - Kamran Khan – 115
    116. 116. Run-TimeSettings Run Logic  Iterations  Action blocks Pacing Log ThinkTimes  No think time  As Recorded  Multiple of X-Recorded  Randomize think time  LimitThink time to Z sec. Speed Simulation  Bandwidth controlling Browser Emulation Proxy  No proxy  Specific proxy  Pick proxy from browser HP LoadRunner - Kamran Khan – 116
    117. 117. Run-TimeSettings Control the behavior of the user when running the script ConfigureWebVusers to emulate real users accessing theWeb site Browser Emulation Specify Browser type Simulate cache Emulate network capacity Emulate modem speed Log Settings Can disable logging Log a specific set of function calls and messages – Standard Log Log parameter substitutions Log response data from server All the messages and function calls raised HP LoadRunner - Kamran Khan – 117
    118. 118. HP LoadRunner - Kamran Khan – 118
    119. 119. ProtocolsRequeststoHP HP LoadRunner - Kamran Khan – 119
    120. 120. NextGenerationofApplications–Web2.0 HP LoadRunner - Kamran Khan – 120 Applications Server Web Server SOA (WS, ESB, Etc.) Network Convergence (VOIP, IP Video) AJAX frameworks HTTPs Request - Response Citrix Microsoft RDP Citrix Microsoft RDP .NET 1.02.0 Remoting .NET 3.0 – WCFWPF Rich Internet UI (AJAX, FLEX, APPLETS) HTTPs RequestResponse & SOAP
    121. 121. LoadRunnersupportforWeb2.0 Web 2.0 Bundle  Ajax Framework Solutions  Flex 1.5 support forAMF version zero  Flex 2.0 support - in 9.1  Flex 3.0 support – in 11.0  Web services protocol HP LoadRunner - Kamran Khan – 121
    122. 122. AJAX–Whatisit? • AJAX is Asynchronous JavaScriptAnd XML • A web development technique for creating interactive web applications • Allows exchanging small amounts of data with the server behind the scenes, without the page having to reload each time • Meant to increase web pages interactivity, speed, and customer usability HP LoadRunner - Kamran Khan – 122
    123. 123. Flex–Whatisit? Adobe Flash has unique animation capabilities Traditional application programmers found it challenging to adapt to Flash Flex minimizes the problem by providing a workflow and programming model that is familiar to application developers. Flex provides an easier way to build Flash applications HP LoadRunner - Kamran Khan – 123
    124. 124. WindowsPresentationFoundation–Whatisit? WPFTechnologies  New UI framework from Microsoft  WindowsVista  .Net 3.0 (4.0 not supported) HP LoadRunner - Kamran Khan – 124
    125. 125. WCF–Whatisit? Windows Communication Foundation, previously codenamed Indigo is a communication subsystem to enable applications, in one machine or across machines over network, to communicate. TheWCF programming model unifies  Web Services  .NET Remoting  DistributedTransactions  Message Queues LoadRunner support forWCF  WCF protocol support  .NET 3.5 support HP LoadRunner - Kamran Khan – 125
    126. 126. NextGenerationofApplications–Web2.0 HP LoadRunner - Kamran Khan – 126 Applications Server Web Server SOA (WS, ESB, Etc.) Network Convergence (VOIP, IP Video) AJAX frameworks HTTPs Request - Response Citrix Microsoft RDP Citrix Microsoft RDP .NET 1.02.0 Remoting .NET 3.0 – WCFWPF Rich Internet UI (AJAX, FLEX, APPLETS) HTTPs RequestResponse & SOAP
    127. 127. RemoteAccessTechnologies–Whatisit? HP LoadRunner - Kamran Khan – 127 Remote AccessTechnology  Citrix – Market Leader Testing Challenges  Protocols support any activity over the network not specific to one application  Very large deployments to support many customers  Remote Access is single point of failure for business activity
    128. 128. RDP-Recording A user creates a connection to a server using Remote desktop client: After the desktop appears - user’s keyboard and mouse actions are recorded HP LoadRunner - Kamran Khan – 128
    129. 129. RDP–Recording …cont. Desktop snapshots are recorded and mouse clicks are highlighted. HP LoadRunner - Kamran Khan – 129
    130. 130. RDPExampleScript HP LoadRunner - Kamran Khan – 130
    131. 131. Q&A ThankYou ! HP LoadRunner - Kamran Khan – 131
    132. 132. Quiz 15 Minutes HP LoadRunner - Kamran Khan – 132
    133. 133. HPLoadRunner Controller for Scenarios & Generators
    134. 134. HP LoadRunner - Kamran Khan – 134
    135. 135. AScenario A scenario is the mean to emulate a real- life behavior of application users – as in real environment. Scenarios include 1. a list of machines on whichVusers run 2. a list of scripts that theVusers run 3. a specified number ofVusers or 4. Vuser groups that run during the scenario Scenarios are created using the LoadRunnerController HP LoadRunner - Kamran Khan – 135
    136. 136. Typesofscenarios A Manual Scenario gives you control over the number of running VUsers and the times at which they run, and lets you test how many VUsers your application can run simultaneously. Create a scenario by assigning VUsers quantity against each script/group and load generators. You can use the Percentage Mode to distribute the total number of VUsers among scripts based on a percentage specified by your business analyst. A Goal-Oriented Scenario is used to determine if your system can achieve a particular goal. LoadRunner automatically builds a scenario based on these goals.  ResponseTime for specified transaction  Number of hits/transactions per second etc … HP LoadRunner - Kamran Khan – 136
    137. 137. Typesofscenarios A Manual Scenario gives you control over the number of runningVusers and the times at which they run, and lets you test how manyVusers your application can run simultaneously.You can use the Percentage Mode to distribute the total number ofVusers among scripts based on a percentage specified by your business analyst. A Goal-Oriented Scenario is used to determine if your system can achieve a particular goal.You determine the goal based on, for example, a specified transaction response time or number of hits/transactions per second, and LoadRunner automatically builds a scenario for you based on these goals. You will create a goal-oriented scenario in Lesson 9, “Advanced Goal-Oriented Scenario.” HP LoadRunner - Kamran Khan – 137
    138. 138. CreatingaManualScenario CreateVuser groups • Each script selected in the New Scenario dialog box is assigned to aVuser group • EachVuser group is assigned a number of virtual users • AllVusers in a group can be assigned to run the same script on the same load generator machine OR • Different scripts and load generators can be assigned to the variousVUsers in a group HP LoadRunner - Kamran Khan – 138
    139. 139. CreatingVUsersGruops • Click the Add Group button on the right of the Scenario groups window.The Add Group dialog box opens. • Enter a name for theVUser group and specify the number ofVusers • Select a load generator from the Load Generator Name list • Select Add from the Load Generator Name list if the load generator is not listed. • Select a script from the script list • Click OK to close the Add Group dialog box.The new group's properties appear in the Scenario Groups window HP LoadRunner - Kamran Khan – 139
    140. 140. ConfiguringVUsersandVUsersgroup Select theVuser group whose VUsers scheme is to be modified, and click theVusers button on the right of the Scenario Groups window.TheVusers dialog box opens. Here you can: change the Script change the Load Generator Add/removeVUsers to the group HP LoadRunner - Kamran Khan – 140
    141. 141. ConfiguringLoadGenerators Click the Generators button to open the LoadGenerator dialog box. Name, Status, Platform and Details of the LoadGenerators are displayed HP LoadRunner - Kamran Khan – 141 Changes LG status from Down to ready Add a new LG Delete an LG Open Details of an LG Disable an LG
    142. 142. Configuringascenario Select group and click Details button Run-Time Settings from the Controller shows the setting of script saved inVuGen. If these settings are modified in Controller, LoadRunner runs the script using the modified settings.This overriding of settings is not saved in script. Clicking View will open script in VUGen. HP LoadRunner - Kamran Khan – 142
    143. 143. SchedulingaScenario–Makeitreal! Selecting Initialize allVusers before Ramp-Up, instructs LoadRunner to initializeVusers before beginning to load them To gradually run theVusers, select the number ofVUsers to begin running concurrently and the amount of delay-time. LoadRunner has to wait betweenVuser ramp ups. HP LoadRunner - Kamran Khan – 143
    144. 144. SchedulingaScenario–Makeitreal! Click Next to navigate to Duration ActionType, there are 3 choices 1. Run until completion 2. Specify the amount of time for which the scenario will run, once all the Vusers have been ramped up 3. Run indefinitely Selecting Initialize allVusers before Ramp-Up, instructs LoadRunner to initializeVusers before beginning to load them HP LoadRunner - Kamran Khan – 144
    145. 145. SchedulingaGroup After creating aVUser group, a group’s execution settings can be scheduled Select Schedule By Group to set the schedule for theVUser group. The options are similar to that of scheduling the scenario – in new versions. HP LoadRunner - Kamran Khan – 145 VUser group Execution Settings: Amount of time that the group must wait before it starts running The number ofVusers that will run within a specified period of time The amount of time the group will run
    146. 146. OldScheduler A fixed structure: Ramp Up – Duration - Ramp Down Very hard to simulate real life behavior HP LoadRunner - Kamran Khan – 146
    147. 147. DynamicScheduler Accurate load testing capabilities Easy to simulate real life scheduling Increased usability, more Flexible First step towards automatic generation of real life scenario from imported production data HP LoadRunner - Kamran Khan – 147
    148. 148. DynamicSchedulerContents Multiple Ramp ups / downs Easier to use  Interactive Graph - edit actions, drag, zoom  Grid for detailed editing Timeline at runtime Pause/Resume scheduler HP LoadRunner - Kamran Khan – 148
    149. 149. RealScenario HP LoadRunner - Kamran Khan – 149
    150. 150. RealLifeSchedule Each schedule (by scenario / by group) is composed of a list of Actions: Fixed actions – Start Group, Initialize Dynamic actions  StartVusers, StopVusers, Duration  Can be used multiple times, in any order HP LoadRunner - Kamran Khan – 150
    151. 151. VUsersQuantity Determined by the scheduler – not by the group ! No Ramp Up “All” HP LoadRunner - Kamran Khan – 151
    152. 152. ManualOperations Do not affect the planned scheduling - not reflected in the design graph Affect the runtime behavior - manually run vusers will be missing for next ramp Example: at a certain point, the user manually started some more vusers  Blue – expected scheduling  Red – actual behavior HP LoadRunner - Kamran Khan – 152 Note: the red line is not visible in the scheduler graph
    153. 153. All slides after this are in draft state.Those will be modified and updated on website soon. Keep checking back. HP LoadRunner - Kamran Khan – 153
    154. 154. SystemMonitoring throughController HP LoadRunner - Kamran Khan – 154
    155. 155. MonitoringSULCont. HP LoadRunner - Kamran Khan – 155
    156. 156. WatchusersinReal-time HP LoadRunner - Kamran Khan – 156
    157. 157. ScheduledRun-Completed HP LoadRunner - Kamran Khan – 157
    158. 158. GoalOrientedScenario–DefiningaGoal • the number of users that you want to use to reach that goal • the scenario duration • load behavior HP LoadRunner - Kamran Khan – 158
    159. 159. DefiningaGoalcont. • Specify that the test should run for 5 minutes after the target has been achieved, and select Continue scenario without reaching goal. • Once the load of three hits per second has been reached, the Controller keeps running the scenario for an additional five minutes, adding or subtractingVusers as needed to keep the actual measurement within 6% of the specified target.This is to ensure that the server can sustain the load for a period of time. HP LoadRunner - Kamran Khan – 159
    160. 160. ServiceLevelAgreement–SLADefinition The user may define goals for the run on several measurements, like:  Average transaction response time  Total throughput (bytes) per run The measurements are checked at run time, to see if there is a violation The results can be viewed in the Analysis application  Transactions success/failure stand up to SLA expectations  LoadTest SLA violations, and the time they occurred HP LoadRunner - Kamran Khan – 160
    161. 161. SLA Wizard A wizard for easily defining SLA goals HP LoadRunner - Kamran Khan – 161
    162. 162. Q&A ThankYou ! HP LoadRunner - Kamran Khan – 162
    163. 163. Quiz 15 Minutes HP LoadRunner - Kamran Khan – 163
    164. 164. HPLoadRunner Analysis
    165. 165. HP LoadRunner - Kamran Khan – 165
    166. 166. What'sNext Understanding HP Analysis [LR component] Understanding various graphs, merging graphs, collating, correlating Analysis and diagnostics Reporting HP LoadRunner - Kamran Khan – 166
    167. 167. AnalysisofTestResults There comes a time after the tests are run when someone who's reviewing the results asks the deceptively simple question, "So what, exactly, does all this mean?!?"This point beyond performance testing is where the capabilities of the human brain come in handy. "Computers are good at swift, accurate computation and at storing great masses of information.The brain, on the other hand, is not as efficient a number cruncher and its memory is often highly fallible; a basic inexactness is built into its design.The brain's strong point is its flexibility. It is unsurpassed at making shrewd guesses and at grasping the total meaning of information presented to it," writes British journalist Jeremy Campbell inChapter 16 of Grammatical Man: Information, Entropy, Language, and Life (1982). HP LoadRunner - Kamran Khan – 167
    168. 168. Graphs:HitsperSecond • The Hits per Second graph shows the number of hits (HTTP requests) made to theWeb server byVusers during each second of the scenario run. • You can see that after a short period the required level of load is reached. HP LoadRunner - Kamran Khan – 168
    169. 169. Graphs:Trans.ResponseTime • TheTransaction ResponseTime graph shows the amount of time it takes for each transaction to be completed. • Its important to watch transaction response time to see the response times your customers will have when the server is under load. HP LoadRunner - Kamran Khan – 169
    170. 170. Graphs:WindowsResources • You can monitor the server’s Windows Resources usage for processor, disk, or memory utilization problems. • Monitoring during a test can help you pinpoint the source of poor performance immediately. You can check theWindows Resources graph legend for a list of measurements. HP LoadRunner - Kamran Khan – 170
    171. 171. TheLoadRunnerAnalysis HP LoadRunner - Kamran Khan – 171
    172. 172. LoadRunnerAnalysis–cont. Session Explorer: In the upper left pane,Analysis shows the reports and graphs that are open for viewing. From here you can display new reports or graphs that do not appear when Analysis opens, or delete ones that you no longer want to view. GraphViewingArea: In the upper right pane,Analysis displays the graphs.  By default, the Analysis Summary report is displayed in this area when you open a session. Graph Legend In the lower right pane, you can view data from the selected graph. HP LoadRunner - Kamran Khan – 172
    173. 173. MoreinAnalysis! There is a spectrum of graphs to analyze.You can apply filters or even cross with results. For every chart, diagram, table or picture you add to your presentation, you can go from understated to wow or anywhere in between. 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 HP LoadRunner - Kamran Khan – 173
    174. 174. AnalysisReport - WordandExcel,too! Export to and spreadsheets that match your presentation for a “branded” look. Cross with Results Global Filters Add Graphs How to generate reports Exporting reports into HTML and *.doc HP LoadRunner - Kamran Khan – 174
    175. 175. Q&A ThankYou ! HP LoadRunner - Kamran Khan – 175
    176. 176. Quiz 15 Minutes HP LoadRunner - Kamran Khan – 176
    177. 177. PerformanceTesting Techniques
    178. 178. HP LoadRunner - Kamran Khan – 178
    179. 179. What'sNext Understanding HP Analysis [LR component] Understanding various graphs, merging graphs, collating, correlating Analysis and diagnostics Reporting HP LoadRunner - Kamran Khan – 179
    180. 180. TriggerLeadTimetoReduceUpgradeRisk Time Predicted point of failure Upgraded Capacity ExistingCapacity Lead TimeTrigger Point Anticipated Peak Loads Work load Usable Reserve Capacity Point ResponseTime degradation becomes noticeable Current load HP LoadRunner - Kamran Khan – 180
    181. 181. Q&A ThankYou ! HP LoadRunner - Kamran Khan – 181
    182. 182. Quiz 15 Minutes HP LoadRunner - Kamran Khan – 182
    183. 183. HPLoadRunner Advance Analysis andTuning
    184. 184. HP LoadRunner - Kamran Khan – 184
    185. 185. PerformanceDesignandManagement A real example of client requirement (A Fortune 100 company): Describe the performance aspects of your system, e.g. from user actions throughWeb server through application server through database and back, including number of registered users/simultaneously-logged-on users/simultaneously-active users, aggregate expected network traffic for all segments, CPU loads/memory utilizations for client/app server/database server, anticipated disk space requirements and growth, and any other salient performance design assumptions or provisions. Describe how these usage patterns will vary by time of day, day of the week, and month of the year.] HP LoadRunner - Kamran Khan – 185
    186. 186. RequiredCapacityPlanningProcesses A real example of client requirement (A Fortune 100 company): [Describe your expected capacity design rules and processes: what resources need to be adjusted as application loads change, how much disk/CPU/memory per user is needed, anticipated growth over time, and the processes you expect the support teams to use to adjust capacity during the lifecycle of the application.] HP LoadRunner - Kamran Khan – 186
    187. 187. PerformanceTest ExitCriteria: For PerformanceTest completion, the following items must be accomplished (Real example by Fortune 100 Client): Performance test results have been reviewed and accepted by client. All defects raised during test execution have been fixed and any deviations have been documented and agreed by client. Chrome will provide effort estimates for all agreed deviations or faults that have been accepted by client. Transaction response times during the course of the test runs should adhere to the target response times defined in SLA Transactions meeting response time SLAs = 100% (See KeyTransaction table below for pass criteria) Running / passed users during the test = 100% Expected transaction throughput during Peak state = 100% Average 15-minute CPU utilization should be < =70% for database and Application Server. Average 15-minute Memory utilization on servers should be < =70% No gradual loss of free memory over the course of the tests Network bandwidth utilization in peak state <=70% On database server, 15-minute average IO wait times must not exceed 10 milliseconds for more than 5% of test duration. Any deviation (outside of agreed upon failure rate of 1%) to be discussed with client on a case-by-case basis. HP LoadRunner - Kamran Khan – 187
    188. 188. TTFBandTTLB In the case of objects with longTTFB times, like index.jsp, the solution often requires a software-level optimization. It could involve adding a database index, introducing some object-level caching, or a configuration change (such as database connection pooling). Be careful to fall in to the trap of throwing more hardware at the problem to solve these types of issues.While it might work in the short term, these issues almost always are due to sub-optimal software and throwing extra hardware at the problem will be like putting a band-aid on a bullet hole. In the case of objects with relatively shortTTFB times but overall longTTLB times, the solution is usually very different.While there may be a software solution, such as configuring Apache’s connections to be better optimized for the server it runs on, most of the time the root cause is due to network/hardware-related issues. Check with the ISP that hosts the server to confirm the max bandwidth throughput allowed. If the object response is slow during peak times but fast during off-peak times, it may need extra web servers (ie: hardware). HP LoadRunner - Kamran Khan – 188
    189. 189. ColdStartupandWarmStartup Understanding Cold Startup andWarm Startup of an application Cold Startup Cold startup occurs when your application starts for the first time after a system reboot, or when you start your application, close it, and then start it again after a long period of time.When an application starts, if the required pages (code, static data, registry, etc) are not present in theWindows memory manager's standby list, page faults occur. Disk access is required to bring the pages into memory. Warm Startup Warm startup occurs when most of the pages for the main common language runtime (CLR) components are already loaded in memory, which saves expensive disk access time.That is why a managed application starts faster when it runs a second time. Ref: HP LoadRunner - Kamran Khan – 189
    190. 190. BlankSlide Put contents here HP LoadRunner - Kamran Khan – 190
    191. 191. Q&A ThankYou ! HP LoadRunner - Kamran Khan – 191
    192. 192. Quiz 15 Minutes HP LoadRunner - Kamran Khan – 192
    193. 193. PowerPoint2007 Text, Graphics & Pictures SmartArt Themes & Quick Styles New Layouts HP LoadRunner - Kamran Khan – 193 Soft Shadow Bevel Reflection
    194. 194. PictureThis… HP LoadRunner - Kamran Khan – 194 Character S p a c i n g Kerning Strikethrough Styled Underline
    195. 195. VisualizeIt! Business Process Operate Support Optimize Change HP LoadRunner - Kamran Khan – 195
    196. 196. HP LoadRunner - Kamran Khan – 196
    197. 197. GetCoordinatedwithThemes! OfficeThemes give you effortless coordination of the colors, fonts and graphic effects applied to your presentation. Everything you insert into your presentation will be automatically styled to match. HP LoadRunner - Kamran Khan – 197
    198. 198. MixItUp! OfficeThemes are infinitely customizable.You can mix and match the colors, fonts and effects from different themes. HP LoadRunner - Kamran Khan – 198
    199. 199. SlidePerfect! PowerPoint has new layouts that give you more ways to present your words, images and media. Everything you add into the layout placeholders is arranged perfectly for you. HP LoadRunner - Kamran Khan – 199
    200. 200. ArrangeitYourWay With custom layouts you design your own slide arrangements.You can create new kinds of documents with PowerPoint. Easily create everything from business pitchbooks to family photo albums. HP LoadRunner - Kamran Khan – 200