Successfully reported this slideshow.
Your SlideShare is downloading. ×

Neoload

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 192 Ad

More Related Content

Slideshows for you (20)

Similar to Neoload (20)

Advertisement
Advertisement

Neoload

  1. 1. NeoLoad Training
  2. 2. Introduction • The Objectives of this Training Load-Testing Principles Design Runtime Results For Advanced Users AGENDA
  3. 3. What You Are Going To Learn At the end of the class, you will know how to use NeoLoad Introduction Many tools to design your VU Some methodology How to make your own analysis
  4. 4. Introduction Load-Testing Principles • Definition • Test Types • Methodology Design Runtime Results For Advanced Users AGENDA
  5. 5. Determine how a system performs in terms of responsiveness and stability under a particular workload Definition Web Load-Testing Principles
  6. 6. Why Do Load Testing? If your website earns $100.000 a day, this year you could lose $2.5 MILLION in sales. Web Load-Testing Principles 16% decrease in customer satisfaction 11% fewer page views 7% loss in conversions A 1 second delay in page load time
  7. 7. Load-Testing Allows Us To Web Load-Testing Principles Predict application performance Detect bottlenecks Determine scalability
  8. 8. Several Tests Web Load-Testing Principles Load Test  determine the behaviour of the system under a specific expected load Scalability / Capacity  determine the system’s maximum load capacity Stress  subject the application to a much greater activity than it normally handles Endurance / Soak  detect memory leaks and other resource consumption issues using long tests and static load Degraded  determine the behaviour of the system under non-optimal conditions
  9. 9. Load-Testing Methodology Web Load-Testing Principles Test plan Load-Testing Consultant Project manager Functional specialists • Objectives • Test platform • Scenarios • Scheduling • Contributors Preparation Load-Testing Consultant Project manager Functional specialists • Testing environment installation • Test platform validation • Test plan validation Design Load-Testing Consultant Functional specialists Developers • Test data generation • VU profile design • DB update procedure • Test data validation • Population & monitoring Runtime Load-Testing Consultant Server specialists Administrators • Single-user baseline • Ramp-up load test • Combination test • Maximum load test Tuning Validation Analysis Load-Testing Consultant Server specialists Administrators • Response time analysis • Benchmark comparison • System metric analysis • Tuning if required • Report creation
  10. 10. NeoLoad Architecture Web Load-Testing Principles Monitor (optional) NeoLoad Monitoring Engine 3G+H+ WiFi4G+ Controller Command Cloud Load Generators On-premise Load Generators Load APP Application Server Database ServerWebServer • NeoLoad management console • Design, run and analyze tests locally • Manage LG and MA • Load generation as defined in the controller • Only active when the test is launched • Retrieve system information • Understand the bottlenecks • Alerts for faster analysis • Network emulation • Device emulation • System to be tested
  11. 11. The Basics of A NeoLoad Project Web Load-Testing Principles Runtime Set the number of users, start the test and look at the real-time information Record and design VU profiles Create a population and set each VU profile rate Design Setup the monitors Analyze the results and issue a report Results
  12. 12. Introduction Load-Testing Principles Design  Recording a Virtual User Profile • Designing a Virtual User Profile • Building a Population • Monitoring the Infrastructure Runtime Results For Advanced Users AGENDA
  13. 13. How Does NeoLoad Record? Recording a Virtual User Profile The client (web-browser) Servers Requests are recorded NeoLoad acts as a proxy and listens on port 8090 Responses are recorded
  14. 14. Starting the Recorder • Use the button and fill-in the following pop-up. Recording a Virtual User Profile Name of the VU Record in Init or Actions Start and set the browser Choose the protocol to record Choose the recording mode Set the user-agent
  15. 15. During the Record • Use the recording bar to modify the user. Recording a Virtual User Profile Stop or pause the record Change the section Create containers Set Rendezvous points Access the documentation
  16. 16. Why Using Containers? • Group actions together • Organize VU’s for easier maintenance • Extract statistics (average times, hits/s…) Recording a Virtual User Profile Without With
  17. 17. Objectives After completing this exercise, you will be able to: • Use the Neotys training environment. • Record a simple VU profile. • Recognize the components of a VU profile. Scenario 1. Open the Neotys Training Applications portal and access the Jpetstore. 2. Record a VU profile, using containers, called ‘Browser’ that only browses the shop. 3. Stop the record and click on Finish to not execute the post-recording wizard. 4. Navigate into the VU profile to see the different components. Exercise 1
  18. 18. The VU Profile at First Sight Recording a Virtual User Profile Actions container Request Page Transaction
  19. 19. The VU Profile Structure • Init - Executed once, and only once, at the beginning of the test. - Common uses: initialization routines, application log in, etc. • Actions - Repeatedly executed until the end of the test. - Cookies and cache are preserved between iterations (default behaviour). - Reset the user context from the runtime parameters. • End - Executed once at the end of the test. - Common uses: application log out, etc. Recording a Virtual User Profile
  20. 20. The Runtime Parameters of the VU Profile • A VU instance keeps running when an error occurs. It may be changed to: - Go to the next iteration. - Stop and start a new VU. • Behaviour on assertions may be set the same way. • All think times may be modified: - Use think time defined on page. - Override the think times. - Apply a factor for each page. - Add a random delay. Recording a Virtual User Profile
  21. 21. VU Profile: Pages • NeoLoad representation of an internet page. • Made of an HTML page and the embedded resources (GIF, PNG, CSS, JS…). • May be renamed. Recording a Virtual User Profile A page The resources
  22. 22. VU Profile: HTTP Requests • NeoLoad representation of a request sent over the network (HTTP/S). Recording a Virtual User Profile Method Server URL Path Parameters
  23. 23. The End of the Record • When the navigation is completed, finish the record either by: - Clicking the stop button . - Closing the web-browser. • If the recorded application keeps sending data, this pop-up may be displayed. Recording a Virtual User Profile
  24. 24. The End of the Record • Go through the post-recording wizard. • It helps taking care of the: - Think-time. - Dynamic parameters: framework (pre-defined rules) and dynamic (comparison based correlation). - Authentication. Recording a Virtual User Profile
  25. 25. Adding Pages To A Record • You can also start the recorder within an existing VU Profile. • Right-click in the VU Profile and choose ‘Record here’. Recording a Virtual User Profile
  26. 26. Objectives After completing this exercise, you will be able to: • Record a structured VU profile. • Use the flags to find an information. Scenario 1. Record a VU profile, using the containers, called ‘Searcher’ that: a. Starts at the Jpetstore homepage b. Enters the store c. Enters ‘Amazon Parrot’ in the search field d. Clicks on a product e. Clicks on an item 2. Stop the NeoLoad recorder and do not execute the post-recording wizard. When prompted to execute click on Finish. 3. Locate the chosen product and item. Exercise 2
  27. 27. Introduction Load-Testing Principles Design • Recording a Virtual User Profile  Designing a Virtual User Profile • Building a Population • Monitoring the Infrastructure Runtime Results For Advanced Users AGENDA
  28. 28. Why? • Fix the issues due to dynamic parameters. • Reach realistic behaviours. • Define several types of VU profile. Designing a Virtual User Profile
  29. 29. Design Methodology This sequence may be reordered. However, do not forget any step! Designing a Virtual User Profile VU profile ready The check VU The variable extractor The logical actions The NeoLoad variables The validation You will learn Validating the VU behaviour • Play a unit test • Check the errors Correcting the VU behaviour • Identify the request that returns an error • Identify the parameter • Handle the data Changing behaviour using logical actions • Adapt to the simulation requirements Using different login accounts and values • Define realistic conditions of use Validating key pages • Set validations to make sure responses are the ones expected
  30. 30. Design Methodology Designing a Virtual User Profile Correcting the VU behaviour Changing behaviour using logical actions Using different login accounts and values Validating key pages Validating the VU behaviour VU profile ready The check VU The variable extractor The logical actions The NeoLoad variables The validation You will learn
  31. 31. Why? • Find any issue that may be in the recorded VU profile. • The recorded pages may require tuning. • Modern apps exchange a lot of parameters. • Avoid bad surprises while the test is running. Validating the VU Behaviour
  32. 32. How? • Using the ‘Check Virtual User’ (NeoLoad debugger). • A single VU test with no thinktime. • Accessed via . Validating the VU Behaviour
  33. 33. The Check VU in Details • Comparisons with recording. • Difference rate. • Variables. • Requests and responses. • Failed assertions. • HTML page rendering. Validating the VU Behaviour
  34. 34. Objectives After completing this exercise, you will be able to: • Check whether a VU profile is valid. • Dig into the details of the page source code. Scenario 1. Check the ‘Searcher’ VU profile and make sure it runs properly. 2. Examine the information provided, including: - The HTML rendering. - The requests that comprise the application home page. - The Comparison of the recorded and replayed responses. 3. Locate the chosen product and item. Exercise 3
  35. 35. Design Methodology Designing a Virtual User Profile Validating the VU behaviour Changing behaviour using logical actions Using different login accounts and values Validating key pages Correcting the VU behaviour VU profile ready The check VU The variable extractor The logical actions The NeoLoad variables The validation You will learn
  36. 36. Why? • Because a VU displays an unexpected error. • Because HTML pages usually contain dynamic elements. • Because NeoLoad cannot automatically handle all parameters. Correcting the Virtual User Behaviour
  37. 37. Focus: The HTTP Parameters • Static parameters - Values never change, regardless of which user is connected or the number of times the VU profile is played. • Functional dynamic parameters - Entered by the end user. - Making them dynamic makes the VU profile more realistic. • Technical dynamic parameters - Generated by the server, often used by the client in subsequent requests. - Values change at each call. Must be handled to have a valid VU profile. Correcting the Virtual User Behaviour
  38. 38. Objectives After completing this exercise, you will be able to: • Understand how technical parameters may influence a VU profile. Scenario 1. Record a VU profile called ‘Buyer’ that: a. Starts at the Jpetstore homepage and then enters the store b. Logs into the store (j2ee/j2ee) c. Selects a category by clicking one of the images beneath the center bird d. Adds a pet to the shopping cart e. Checks out and enters payment information f. Confirms the sale g. Logs out 2. Stop the NeoLoad recorder and do not search for the dynamic parameters in the post- recording wizard. 3. Check the ‘Buyer’. 4. Explain why it fails. Exercise 4
  39. 39. How? • Using a mecanism called variable extractor which extracts content of an HTTP response. • The content is assigned to a NeoLoad variable. • A variable may be called using the syntax ${<variableName>} or with the variable picker that gives a quick access to the existing variables. • The variable extractor is accessed from the request advanced settings . • The variables created with the variable extractor are only accessible by the VU profile in which they are extracted. Correcting the Virtual User Behaviour
  40. 40. The Variable Extractor Correcting the Virtual User Behaviour Variable name Target extraction Occurrence to extract Encoding Test XPath definition Start boundary Groups End boundary
  41. 41. Hypothesis: a request fails to pass the Check VU Make the replacement Select the failed request Validate the variable extractor Go to the variable extractor settings and configure the extraction Select the response containing the data to be extracted Procedure: the Variable Extractor Correcting the Virtual User Behaviour
  42. 42. Objectives After completing this exercise, you will be able to: • Use the NeoLoad tools to properly take into account a dynamic paramater. Scenario 1. From the observations made on the previous exercise, fix the VU profile using a variable extractor. The extraction must be set on a request before the one that fails. 2. Use the Check-VU function to make the VU profile works as expected. Exercise 5
  43. 43. The Variable Extractor: Automatic Configuration • Automation of the manual procedure. • Avoid mistakes. • Save time. Correcting the Virtual User Behaviour Select the automatic configuration All is automatically setup
  44. 44. Objectives After completing this exercise, you will be able to: • Take advantage of the automatic correlation feature. Scenario 1. Duplicate the ‘Buyer’ and call the new user ‘Buyer_AutoConfig’. 2. Restore the value of the id parameter in the ‘Buyer_AutoConfig’. 3. Use the automatic configuration to fix the issue. 4. Check the VU to be sure the issue has been handled. Exercise 6
  45. 45. The Framework Parameters: Practical Work Correcting the Virtual User Behaviour 1. Record a VU profile called ‘BuySeveralItems’ that: a. Starts at the Jpetstore homepage and then enters the store b. Logs into the store (j2ee/j2ee) c. Selects a category by clicking one of the images beneath the center bird d. Adds a pet to the shopping cart e. Clicks on “Main Menu” f. Repeats steps c. to e. 5 times g. Checks out and enters payment information h. Confirms the sale i. Logs out 2. Run a check-VU and observe the result
  46. 46. How should we handle the id parameter? Use the search and replace? Do all extractions manually? The Framework Parameters Correcting the Virtual User Behaviour
  47. 47. The Framework Parameters: Why? • Parameters are often the same across the scenario. • “Teach” NeoLoad new correlations rules and save time. • Re-use your framework parameters for future records. Correcting the Virtual User Behaviour
  48. 48. The Framework Parameters: How? • NeoLoad provides predefined correlation rules (.NET, Oracle, GWT, Siebel, Flex, Silverlight…). • For your own apps, create your own framework parameters. Correcting the Virtual User Behaviour Move the extraction to the framework Move the extraction to the framework
  49. 49. The Framework Parameters: Advanced Settings • Only edit existing configurations. Do not create a framework parameter from scratch. Correcting the Virtual User Behaviour Name Replacement Extraction rule
  50. 50. The Framework Parameters: Tips • Framework parameters are part of the NeoLoad installation. Exporting a project does not include them. • Framework parameters must be exported to be shared (advanced settings, NeoLoad preferences). • Search for the framework parameters via . Correcting the Virtual User Behaviour
  51. 51. Objectives After completing this exercise, you will be able to: • Take advantage of the framework parameters for automatic correlation. Scenario 1. Record a ‘SearchBooking’ user on the Hotel Booking Application: a. Access the home page. b. Start (click on ‘Start your Spring Travel Experience’). c. Click on ‘Find Hotels’. d. View the first hotel in the list. e. Close your browser. 2. Do not execute the post recording wizard. Check the VU, and observe the results. 3. Define extractors and add them to a ‘HotelBooking’ framework for the following parameters: - dtid, - execution, - the first uuid_0 (rename it ‘uuid_Find’ when you add it to the framework) - the second uuid_0 (rename it ‘uuid_View’ when you add it to the framework) Exercise 7 (1/2)
  52. 52. 4. Apply the framework parameters to the ‘SearchBooking’ user. Check it and observe the results. 5. Record a ‘Logged_SearchBooking’ user following these steps: a. Access the home page. b. Login (credentials are provided on the page). c. Start (click on ‘Start your Spring Travel Experience’). d. Click on ‘Find Hotels’. e. View the first hotel in the list. f. Logout. g. Close your browser. 6. Apply the framework parameters to the ‘Logged_SearchBooking’ user. Check it, and observe the results. Exercise 7 (2/2)
  53. 53. Generic Dynamic Parameters vs. Framework Parameters • Generic Dynamic Parameters - Found ‘on the fly’ using comparison. - Less accurate identification. - Slower identification. • Framework Dynamic Parameters - Defined using regular expressions. - More accurate identification. - Faster identification. Correcting the Virtual User Behaviour
  54. 54. The Variable Extractor: Multiple Occurrences • If an extraction template has several matches, all occurrences may be returned as a set. • The following variables are created: - <variableName>_matchNr : number of occurrences found. - <variableName>_n : nth occurrence (n=1, 2...). - <variableName>_rand : random occurrence. - <variableName> : default value. • Unless for specific purpose, when doing multiple extractions, do not use the only name of the variable. Correcting the Virtual User Behaviour
  55. 55. Design Methodology Designing a Virtual User Profile Validating the VU behaviour Using different login accounts and values Validating key pages Correcting the VU behaviour Changing behaviour using logical actions VU profile ready The check VU The variable extractor The logical actions The NeoLoad variables The validation You will learn
  56. 56. Why? • To simulate more realistic conditions. • To make simpler records. • To have new VU profiles by only modifying one already recorded. Changing the Virtual User Behaviour
  57. 57. How? • Using the logical actions. • Adapt to the simulation requirements with: - Loops. - Conditional branching. - Error management… • Drag and drop the actions. No script. Changing the Virtual User Behaviour
  58. 58. Container • Group together certain actions and statistics. • Two execution modes: - Sequentially: the elements of the container are played in the order they are declared. - Randomly: the elements of the container are played randomly. • The advanced mode allows to configure each element to be repeated a number of times (repetition or probability). • Pacing may be set on each container. It is the minimum time for the container to be executed. Changing the Virtual User Behaviour
  59. 59. Delay • Suspends the execution for a specified duration. • Expressed in milliseconds. • May be a variable. Loop • Repeat elements. • The number of iterations may be constant or a variable. • The variable <LoopName>_counter is automatically created. It gives the current iteration. Changing the Virtual User Behaviour
  60. 60. While • Execution of elements while a condition remains true. • Several conditions may be applied. • The variable <WhileName>_counter is automatically created. It gives the current iteration. If…Then…Else • Execution of conditional actions. - If condition is true, Then folder is executed - If condition is false, Else folder is executed • Several conditions may be applied to an If…Then…Else action. Changing the Virtual User Behaviour
  61. 61. Objectives After completing this exercise, you will be able to: • Modify the sequence of a VU profile using the GUI. • Change the VU profile to fullfill specific conditions. • Use the ‘Update Recorded Content’ function. Scenario 1. Duplicate the ‘Buyer’ and call it ‘LoopBuyer’. 2. Modify the ‘LoopBuyer’ in order to purchase a pet from each category (several pets in the shopping cart and only one order): - Set up multiple extractions. - Use a Loop. - Variables may be concatenated following the syntax ${var1${var2}}. - Automatic variable extractions cannot be used. 3. Update the recorded content of the ‘LoopBuyer’ (advanced settings of the Check-VU). 4. Duplicate the previous VU profile and call it ‘WhileBuyer’. 5. Modify the ‘WhileBuyer’ using a While to include a condition on the total price (for example, less than 500 dollars). Exercise 8
  62. 62. Try…Catch • Adapt the test execution behaviour in function of events (error management…). • Actions included in the Try are executed sequentially. • If an assertion or an error is detected then the Catch is executed. Changing the Virtual User Behaviour
  63. 63. Go to Next Iteration • Interrupt the VU runtime and go to the next iteration. • The behaviour depends on where the action is: - Init: the VU stops its initialization and goes to the Actions container. - Actions: the VU stops its current iteration and run a new Actions container or goes to the End container (depends on the load policy). - End: no interruption, the VU keeps executing the End and an error message is logged. • Often combined with If...Then...Else or Try...Catch. Changing the Virtual User Behaviour
  64. 64. Shared Containers • Share elements to re-use them in several VU profiles (login or logout for instance). • Statistics are given across several VUs. • Shareable elements are: - Container - Loop - While - Fork • An element is shared with a drag-and-drop or a right-click interaction. Changing the Virtual User Behaviour
  65. 65. Design Methodology Designing a Virtual User Profile Validating the VU behaviour Validating key pages Correcting the VU behaviour Changing behaviour using logical actions Using different login accounts and values VU profile ready The check VU The variable extractor The logical actions The NeoLoad variables The validation You will learn
  66. 66. Why? • Test all branches of the application. • Test how the servers handle the cache. • Make things more realistic. Using Different Values
  67. 67. How: the Variable Manager • Numerous variable types (lists, strings, integers, counters…). • Change policy. • Distribution policy. The variables created through the variable manager are accessible by all the VU profiles. Using Different Values
  68. 68. Focus: the SQL Variable • The SQL variable embeds a guided mode for MySQL, Oracle, DB2, PostgreSQL and MS SQL. • When requesting other DB, the driver and the URL must be manually defined. • The SQL variable is populated and sent to the LGs before the test is running. It is not calculated in real-time. • Use the syntax ${<variableName>.<columnName>} to access each column of the table. Using Different Values
  69. 69. Objectives After completing this exercise, you will be able to: • Create and use the NeoLoad variables. Scenario 1. Make the authentication of the ‘Buyer’ dynamic with a File variable (accounts are provided on the Neotys Application Portal). Call this variable ‘Accounts’. 2. Create a List variable called ‘Address’ with three columns called ‘City’, ‘State’ and ‘ZIP’ that contains four data sets (rows): - Boston, MA, 02134 - Random Lake, WI, 53075 - Beverly Hills, CA, 90210 - Echo, Oregon, 97826 3. Data values for the variable are selected in a random order. 4. In the ‘Buyer’, find the request submitting the payment details and replace the hard- coded values with the variable ‘Address’. 5. Create (without using it) an SQL variable for the authentication called ‘SqlAuthent’ (DB: jpetstore, table: SIGNON). Exercise 9
  70. 70. Design Methodology Designing a Virtual User Profile Validating the VU behaviour Correcting the VU behaviour Changing behaviour using logical actions Using different login accounts and values VU profile ready The check VU The variable extractor The logical actions The NeoLoad variables The validation Validating key pages You will learn
  71. 71. Validation • Check the server responses in real-time to ensure that the pages delivered by the server are what you expect. • Three criteria: - Delay. - Page size. - Page content. Validating Key Pages
  72. 72. Global Validation • Add content assertions to a container or the entire VU profile. • Only responses with a text/html or text/xhtml content-type are checked (default behaviour). Validating Key Pages
  73. 73. Objectives After completing this exercise, you will be able to: • Prepare the VU profile for the test by implementing error management. Scenario 1. In the ‘Searcher’, change the search criteria using a List variable containing: - Goldfish - Amazon Parrot - Iguana - Manx 2. Set a validation to check that the ‘Add to Cart’ button is in the page. 3. Use a Try…Catch to stop the VU iteration if an assertion fails. 4. Set random think times between 5 and 10 seconds. 5. Add a Loop to include several searches (between 1 and 4). Change the search criteria value on each loop iteration. 6. Stop the VU iteration if the stock of an item falls below 9500. Exercise 10
  74. 74. Introduction Load-Testing Principles Design • Recording a Virtual User Profile • Designing a Virtual User Profile  Building a Population • Monitoring the Infrastructure Runtime Results For Advanced Users AGENDA
  75. 75. Why? • Test using different business behaviours. • Test with specific network conditions. • For example, test a web store by simulating 75% of users browsing the catalog (visitors), 20% carrying out a purchase through to its end (buyers) and 5% being administrators. Building a Population
  76. 76. How? Building a Population Browser selection Populations WAN Emulation Cache policy
  77. 77. Focus: Browser Picker Building a Population Simulate desktop, mobile or tablet browsers
  78. 78. WAN Emulation • All networks do not share the same characteristics. • Typical characteristics are: - Signal strength (for wireless networks) - Bandwidth - Latency - Packets dropped. • WAN emulation allows realistic network conditions. Building a Population
  79. 79. WAN Emulation • Settings available in the populations tab. • Not configured by default (unlimited bandwidth, no packet loss, no latency). • Pre-configured profiles carry real-world values. • A LG may simulate several network conditions. Building a Population WAN emulation profiles WAN emulation parameters
  80. 80. Objectives After completing this exercise, you will be able to: • Create a complex population. Scenario The population ‘PetstorePopulation’ is made of: 1. 60% of ‘Searcher’. 2. 40% of ‘Buyer’. 3. The ‘Searcher’ users do not use a cache; the other users have a full cache at the beginning of the iteration. 4. 50% of the ’Buyer’ users have an iPad and a 3G connection with a poor signal. 5. The rest of the ‘Buyer’ users have Chrome and a Wi-Fi connection with a medium signal. 6. 10 % of the ‘Searcher’ users have a custom browser (called ‘Proprio’) that uses 6 simultaneous HTTP connections. Exercise 11
  81. 81. Introduction Load-Testing Principles Design • Recording a Virtual User Profile • Designing a Virtual User Profile • Building a Population  Monitoring the Infrastructure Runtime Results For Advanced Users AGENDA
  82. 82. Why? • Without monitor, NeoLoad only retrieves the response times, the errors, the number of hits and the throughput. • Information from the server side is fundamental to perform an efficient analysis. • Each layer of the architecture may be monitored (webservers, application servers, database, network…). Monitoring the Infrastructure
  83. 83. How? • No installation on the server side. • All actions are performed from the NeoLoad controller. • Specific ports need to be open on the firewall. • NeoLoad provides preconfigured counters. Monitoring the Infrastructure
  84. 84. The Configuration Monitoring the Infrastructure Choose the layers Set the authentication Choose the counters
  85. 85. Monitoring Agent • Simplified controller to agent communication. • Single port (TCP 7200) to open in the controller to agent direction. • Installed and started as separate process on the server side. • MA and LG share the same color code: - Green: running - Orange: different version - Red: halted Monitoring the Infrastructure
  86. 86. Objectives After completing this exercise, you will be able to: • Use the NeoLoad monitoring capabilities. • Look at the architecture behaviour in real-time. Scenario 1. Add a Tomcat 6 monitor (no authentication needed). 2. Add a MySQL monitor (authentication: monitor / no password). 3. Add a Windows monitor (localhost). Exercise 12
  87. 87. The Thresholds: Why? • Trigger real-time alerts during test runtime on performance counters. - If CPU usage exceeds 80% - If available memory falls below 15% - … • Highlight components that suffer a deterioration in performance during the load test. • Alerts make understanding the causes of any bottleneck much easier. Monitoring the Infrastructure
  88. 88. The Thresholds: How? • Alerts are set on some counters by default. • Alerts are identified by . • Alert thresholds may be manually added. • An alert may be of two sorts: warning or critical. Monitoring the Infrastructure
  89. 89. Objectives After completing this exercise, you will be able to: • Set thresholds and alerts. Scenario 1. Based on the previous exercise, set an alert on the Windows monitor that follows the condition: if the percentage of memory used is greater than 98%, raise up a critical alert. 2. Set an alert on the Tomcat monitor that follows the condition: if the currentThreadCount(HTTP) exceeds 20, raise up a warning alert. Exercise 13
  90. 90. Introduction Load-Testing Principles Design Runtime • Running the Test Results For Advanced Users AGENDA
  91. 91. Setup and Watch Running the Test Test completed The runtime settings The real-time analysis You will learn Configuring the test scenario ● The duration ● The load policy ● The LGs ● Advanced settings Looking at the real- time information ● Plot your graphs ● Correlate the data ● Check the errors ● Follow the VUs
  92. 92. Setup and Watch Running the Test Looking at the real- time information Test completed The runtime settings The real-time analysis Configuring the test scenario You will learn
  93. 93. Definition • The test configuration. • Several test scenarios may be created (short ramp-up test, long peak test…). • All settings required are defined and applied to a population: - Test duration. - Load policy. - Load generators. - Runtime policy. Configuring the Test Scenario
  94. 94. Duration Policy • Unlimited. • Iterations. • Time. • Iteration mode: - Each user is run for a predetermined number of iterations with no break between iterations. - Towards the end of the test, some users are finishing their iterations and some others are still active. As a consequence the load decreases until all the users are finished. - For ramp-up or peak loads, adding users for a new phase takes place once all existing users have finished their iterations (the load may drop to 0). Configuring the Test Scenario
  95. 95. Load Policy • Constant: generate a fixed number of VU. • Ramp-up: generate a number of VU that increases throughout the test. • Peak: generate a fixed number of VU with periodic phases of high load. • Custom: set the load to be applied by plotting the VU variation curve. Configuring the Test Scenario
  96. 96. Load Generators • LGs may be manually added by entering: - The machine's name. - The IP address. • LGs may be automatically discovered. • Define from the advanced settings: - The network interface card. - The IP spoofing. - The load factor. - The upgrade. Configuring the Test Scenario
  97. 97. The Cloud Load Generators: Why? • Perform heavy load tests without hosting the necessary infrastructure. • Load test applications from several geos. • Test the entire delivery chain. • Benefits of the hybrid approach: cloud LGs / on premise LGs. Configuring the Test Scenario
  98. 98. The Cloud Load Generators: How? Configuring the Test Scenario Open the Cloud console and choose an on-demand or reserved session
  99. 99. The Cloud Load Generators: On-Demand Session Configuring the Test Scenario Choose the duration and the geos for an immediate use
  100. 100. The Cloud Load Generators: Reserved Session Configuring the Test Scenario Book your session online and use it later
  101. 101. The Cloud Load Generators: Managing the Session Configuring the Test Scenario Cloud and hosted LGs are used the same way The session is managed from the console
  102. 102. Population Parameters Configuring the Test Scenario How VUs start Start policy Stop policy
  103. 103. Scenario Settings Configuring the Test Scenario URL filtering Debug mode settings Runtime users and monitoring parameters Apply SLA
  104. 104. Scheduling a Test • NeoLoad -project <file> -launch <scenario> [-options]. • Export SLA results in JUnit XML format. • Generate reports (XML, HTML, JUnit XML). • Publish results to the Neotys Team Server. • Specify monitored hosts. Configuring the Test Scenario
  105. 105. Setup and Watch Running the Test Configuring the test scenario Test completed The runtime settings The real-time analysis Looking at the real- time information You will learn
  106. 106. Runtime Overview • Time is absolute or relative (Preferences>General Settings>Advanced). The Real-Time Information Main statistics LGs status Runtime information
  107. 107. Runtime Graphs • Time is absolute or relative (Preferences>General Settings>Advanced). The Real-Time Information
  108. 108. Runtime Errors The Real-Time Information Comparison with record Stop failed VUs Errors Detail
  109. 109. Runtime Alerts The Real-Time Information
  110. 110. Runtime Users The Real-Time Information Add / remove VUs Follow a VU
  111. 111. Objectives After completing this exercise, you will be able to: • Set a load-test scenario and launch a test. Scenario 1. Create a load test scenario called PetstoreScenario. 2. Make sure the population PetstorePopulation is active. 3. Define an increasing workload that starts with a single VU and adds an additional VU every 15 seconds. Indicate the maximum number of VUs allowed by the training license. 4. Start the test with a duration of 5 minutes and explore the real-time data displayed including: - Overview data. - Performance data for containers. - Server-side metrics. - Runtime errors, if any. Exercise 14
  112. 112. Introduction Load-Testing Principles Load-Testing Principles Design Runtime Results • Analyzing the Results For Advanced Users AGENDA
  113. 113. Test Summary Analyzing the Results Select a test result from the drop-down list Statistics and graphs are displayed
  114. 114. Test Summary Analyzing the Results
  115. 115. Test Summary Analyzing the Results Top 10 first alerts Chronological chart
  116. 116. Test Summary Analyzing the Results Cloud statistics
  117. 117. Values Analyzing the Results Choose a statistic Export aggregated or raw data (only for containers) Filter data Plot any element
  118. 118. Values • Tree view mode • List view mode Analyzing the Results
  119. 119. Graphs Analyzing the Results Alerts and SLA Percentile graphs Choose the statistics to plot
  120. 120. Memo: Graphs • Zoom in with left mouse button selection. • Export data with (CSV file or picture). • Time, user load or monitor for horizontal axis. • Plot percentile graphs. • Do comparisons with the drop-down list. • Use and create your own templates . Analyzing the Results
  121. 121. Errors Analyzing the Results 20.000 errors displayed max Request, response or assertions Error list Detailed error information Detail of the previous request
  122. 122. Alerts Analyzing the Results Red zones for critical alerts, yellow zones for warning alerts Alerts triggered on performance counters during the test
  123. 123. Logs • Check the log files of the LGs and MAs. Analyzing the Results Check the details Select MAs or LGs The log level
  124. 124. Results Manager Analyzing the Results Tools Time Actions Test preview Test result list
  125. 125. Generating a Report Analyzing the Results Select the output Select the detail Select the test
  126. 126. Objectives After completing this exercise, you will be able to: • Create charts showing performance indicators. • Drilldown through a series of measurements to highlight the cause of an issue. Scenario Based on the previous exercise: 1. Determine the average duration for the container where the category is selected. 2. Determine the page with the slowest average response time. 3. With a graph template determine the ‘health’ of the LGs used in the test. 4. Examine the errors encountered during the test. 5. Check if the overall page response time changed when the user load increases. 6. Explore the impact the user load had on the number of Tomcat threads. 7. Determine whether the number of current Tomcat threads affects the amount of OS memory available. Exercise 15
  127. 127. For Advanced Users  Advanced Installation • Servers • Request & Page Settings • Service Level Agreement • Logical Actions • Variables • Data Exchange API • Mobile Support • Advanced Results • The NeoLoad License AGENDA
  128. 128. Using a Firewall Advanced Installation TCP 7100 CTRLLG TCP 7200 CTRLMA UDP 1359 LGCTRL TCP 4569 LGCTRL for the automatic discovery function Monitor (optional) NeoLoad Monitoring Engine 3G+H+ WiFi4G+ Controller Command Cloud Load Generators On-premise Load Generators Load APP Application Server Database ServerWebServer TCP 7100 CTRLLG TCP 443 CTRLLG
  129. 129. For Advanced Users • Advanced Installation  Servers • Request & Page Settings • Service Level Agreement • Logical Actions • Variables • Data Exchange API • Mobile Support • Advanced Results • The NeoLoad License AGENDA
  130. 130. The Graphical User Interface Servers • Servers are automatically created during the record. • The target server may be quickly changed. • Variables may be used (load-balancing). • Server side authentication. • Filled in during the record. • Accounts may be dynamic (variables). • Automatic session tracking when cookies are used. • When detected during recording, the argument is automatically filled in.
  131. 131. Authentication Mechanisms • NeoLoad supports Basic, Digest, NTLM and Negotiate mechanisms. • On standard environments, the default protocol for Negotiate is Kerberos. If Kerberos fails, Negotiate tries NTLM. However, for performance reasons NeoLoad uses NTLM in place of Kerberos. • If Kerberos is absolutely required, NeoLoad configuration file needs to be modified. • NeoLoad prioritizes schemes in the order Negotiate, NTLM, Digest and Basic. Servers
  132. 132. URL Rewriting and Browser Profile • Deactivation forces the server to use URL rewriting to keep the session alive. Servers Cookie activation
  133. 133. For Advanced Users • Advanced Installation • Servers  Request & Page Settings • Service Level Agreement • Logical Actions • Variables • Data Exchange API • Mobile Support • Advanced Results • The NeoLoad License AGENDA
  134. 134. Page Settings Advanced Page Settings Page resources Request playback
  135. 135. Page Settings • Request playback: NeoLoad plays the requests sequentially (by default requests are played back in parallel). • Dynamically execute the page resources: NeoLoad analyzes the HTML response of the main request to execute dynamically the page resources. • NeoLoad retrieves resources from the HTML code. Resources referenced by CSS style sheets and JavaScript scripts are ignored. Also, this setting has no effect on the module requests. Advanced Page Settings
  136. 136. Request Settings Advanced Request Settings Change the referrer Encode or put an equal Response storage Edit request headers
  137. 137. For Advanced Users • Advanced Installation • Servers • Request & Page Settings  Service Level Agreement • Logical Actions • Variables • Data Exchange API • Mobile Support • Advanced Results • The NeoLoad License AGENDA
  138. 138. Why? • Hard to define what ‘good performance’ is. • User standpoint evolution: - 2006: 75% of users would not return to a website that take longer than 4 seconds (Akamai research) - 2009: a website needs to load up in 2 seconds (Akamai research) • As a consequence, assessment of the service quality must be defined with end users and/or functional experts. • These thresholds are called Service Level Agreement (SLA). Service Level Agreement
  139. 139. Two Types of SLAs in NeoLoad • Per run: evaluated at the end of the test. • Per time interval: evaluated each time a result is received during test runtime (real-time alerts). Service Level Agreement
  140. 140. How? Service Level Agreement Check the summary Apply the profile Threshold definition (acceptable and failed conditions)
  141. 141. Statistics on SLAs Service Level Agreement Check the status of each SLA per profile Check the status of each SLA per VU
  142. 142. Edit SLA Profiles After a Test • If an SLA profile is not enough accurate, it may be modified from the result manager. • Transfer an SLA profile from Design to Results with to move it into several test results and re-compute the result summary using the new configuration. • Transfer an SLA profile from Results to Design with to copy it into the Design. Service Level Agreement Same options as the Design
  143. 143. Objectives After completing this exercise, you will be able to: • Apply service level agreements that meet your needs. Scenario 1. Open the project called ‘Training_Project’ (available on the resources page) and look at the result called ‘Reference Test’. 2. Create a new SLA profile called ‘Reference’ following the criteria: - Average response time per time interval must be less than 0.5 second. - If the average response time per time interval is greater than 1 second the transaction must be failed. - The SLA profile must be applied to all containers of all VU profiles. 3. Once done, apply your changes and look at the results. Exercise 16
  144. 144. For Advanced Users • Advanced Installation • Servers • Request & Page Settings • Service Level Agreement  Logical Actions • Variables • Data Exchange API • Mobile Support • Advanced Results • The NeoLoad License AGENDA
  145. 145. Variable Modifier • Used to modify the value of a variable before the value change policy is applied. • Re-initialize or modify value (change policy is applied). • Manage the Shared Queue actions. Logical Actions
  146. 146. Fork • Used to play actions in a different thread to the current one. • When the VU main thread stops, all the threads created using Fork action are immediately halted. • Example of use, an action that plays every x seconds on the server to refresh a component (Ajax, Flash plug-in…). • Existing variable values may be copied locally. When a value is modified in another thread, the variable value remains unchanged in the Fork action. Logical Actions Unchecked by default
  147. 147. Wait Until • Pauses the current execution thread until certain conditions have been verified. • A condition is composed of two operands and an operator (except with Exists and Doesn’t exist operators). • A timeout may be defined. Once the delay is timed out, the thread is restarted. The following action is then executed, even if the conditions have not been verified. Logical Actions
  148. 148. Objectives After completing this exercise, you will be able to: • Use the Fork logical action. Scenario 1. Create manually a VU profile called ‘SearcherAndBrowser’ that only contains a Fork action in the Actions container. 2. Copy the content of the ‘Searcher’ in the main thread and the ‘Browser’ in the second thread. 3. Check the ‘SearcherAndBrowser’. Observe the concurrency of both paths. 4. Create a variable called ‘isComplete’ that has two values, TRUE and FALSE. 5. Use a Wait Until action to have the main thread waiting for the end of second thread before its execution. 6. Check the ‘SearcherAndBrowser’. Verify that it works as expected. Exercise 17
  149. 149. JavaScript • Execute JavaScript in a VU profile. • The NeoLoad API provides functions to manipulate variables and set up cookies. • When using the Logger API, information is written to the LG log file (loadGenerator.log.00x in the Logs directory). • Handle strings, dates, calculation... Logical Actions
  150. 150. JavaScript Logical Actions Access the variable manager Get a value from the variable manager Log a message Assign a value
  151. 151. Java and JavaScript • Use Java classes and functions in JavaScript actions. • Copy the JAR files into <NeoLoad-project>/lib/jslib . The file is taken into account automatically. • The class names must be fully qualified within the JavaScript (even for the Java classes like java.util.List) and preceded by Packages. For example var obj = new Packages.com.company.MyClass();. Logical Actions
  152. 152. JavaScript Libraries • Create functions used in all the VU profiles. • It helps reduce memory usage on the LGs, since the code stored in the libraries is shared by the users. Logical Actions
  153. 153. Objectives After completing this exercise, you will be able to: • Insert a JavaScript action into the VU profile to perform specific actions. Scenario 1. Add a JavaScript action to the ‘Buyer’ to log an entry every time an item is added to the shopping cart. 2. Include a time-stamp for each log entry. 3. Add a second JavaScript action to retrieve the name of the LG to the log. Exercise 18
  154. 154. Rendezvous • Synchronize VUs to perform a specific task simultaneously. • Create a precise load peak in the VU execution. • Policies are global, regardless the number of LGs. • Rendezvous may be controlled from a JavaScript action. Logical Actions Timeout Rendezvous policies
  155. 155. Custom Action • Take advantage of predefined ‘high-level’ actions embedded in NeoLoad • Full GUI integration • More to come on the Neotys Community Logical Actions The command line action
  156. 156. Objectives After completing this exercise, you will be able to: • Use the command line custom action. Scenario 1. At the end of the ‘Buyer’, use a custom action to log the total price of the order into a file (use the ‘Echo’ function). 2. The title of the file must contain the instance number of the VU being executed. Exercise 19
  157. 157. For Advanced Users • Advanced Installation • Servers • Request & Page Settings • Service Level Agreement • Logical Actions  Variables • Data Exchange API • Mobile Support • Advanced Results • The NeoLoad License AGENDA
  158. 158. The Variable Manager • Variables are created before the test is running. • Depending on the policies, values are provided either by the controller or the LGs. • With the Unique policy, a NoUniqueValueAvailableException exception is logged when there is no more available value. Variables Global • Sequential • Random Unique • Sequential • Random Local • Sequential • Random • Any Global • Any Values provided by the LGs Values provided by the controller
  159. 159. Shared Queue: Why? • Share dynamic content amongst VU profiles. • A FIFO pile with the following constraints: - When full, all new arrivals are lost. - When empty, the requesting VU has to wait for a new arrival to be added. • A swap file may be used to pre-populate the queue at the start of the test or to save it at the end. • The Check VU does not have any effect on the swap file. Variables
  160. 160. Shared Queue: How? Variables Queue size Timeout (${queueName} is returned when the queue is empty) Swap file
  161. 161. • Values are provided to the LGs by the controller when the test is running. Shared Queue: How? • Populating the Shared Queue Variables – Consuming the information Select where the value from the queue is copied Select the Shared Queue that contains the information Select the variable to add Select the Shared Queue to populate
  162. 162. Shared Queue and JavaScript • A Shared Queue may be controlled by a JavaScript action. • NeoLoad provides a comprehensive set of APIs. • Main functions are: - addSharedValue. - createSharedQueue. - getSharedQueueSize. - peekSharedValue. - pollSharedValue. Variables
  163. 163. Objectives After completing this exercise, you will be able to: • Share data accross two different VU profiles. Scenario 1. Duplicate the “Buyer” and call it “SQ_Buyer”. 2. In the ‘SQ_Buyer’, populate a Shared Queue with the order number generated at the end of the scenario. 3. Set the ‘SQ_Buyer’ authentication to j2ee / j2ee. 4. Record a new VU profile called ‘BuyOneItem’ that: a. Logs in the Jpetstore. b. Open the last order referred by the order number stored in the Shared Queue (manually alter the URL in the browser using jpetstore/shop/viewOrder.shtml?orderId=1000 for the record). c. Buys the first item that appears in the shopping cart. d. Logs out. 5. Launch a test combining both ‘SQ_Buyer’ and ‘BuyOneItem’ to have the ‘SQ_Buyer’ creating a list of orders and the ‘BuyOneItem’ using that list. Exercise 20
  164. 164. JS Variable • A variable computed by a piece of Javascript. • A new function evaluate returns the variable value: function evaluate() { return new function() { this.firstField = "a value"; this.secondField = myLibraryFunction(); } } • The field values are accessed by ${<variable name>.col_<field number>} or ${<variable name>.<field name>}. • The JavaScript variable is always computed on the LG side during the test. Variables
  165. 165. JS Variable Variables Script Value change policy
  166. 166. For Advanced Users • Advanced Installation • Servers • Request & Page Settings • Service Level Agreement • Logical Actions • Variables  Data Exchange API • Mobile Support • Advanced Results • The NeoLoad License AGENDA
  167. 167. The Data Exchange API Module makes it easy to integrate data from a third party tool Why? Data Exchange API
  168. 168. Why? • RESTful data service based on the Open Data Protocol • OData is a standardized protocol for creating and reading data APIs • NeoLoad receives external data to generate the results displayed in the GUI • More information here Data Exchange API
  169. 169. How? • NeoLoad Open API accepts JSON format • Create your own scripts to send data to the NeoLoad Server from any application Data Exchange API
  170. 170. Objectives After completing this exercise, you will be able to: • Create your script to plot any information from any application. Scenario 1. Duplicate the buyer and call it “DEA_Buyer”. 2. In the ‘DEA_Buyer’, create a script that uses the Data Exchange API to plot the total price of what the “DEA_buyer” buys. 3. Launch a test with a few virtual users to see the results. Exercise 21
  171. 171. For Advanced Users • Advanced Installation • Servers • Request & Page Settings • Service Level Agreement • Logical Actions • Variables • Data Exchange API  Mobile Support • Advanced Results • The NeoLoad License AGENDA
  172. 172. Mobile App Load Testing (Web-based and Native) • Record native and web-based apps directly from the mobile device. • Record web-based apps from the desktop browser with the Identify as feature. • Generate realistic traffic with the WAN emulation. Mobile Support
  173. 173. Recording Native and Mobile Web from Device Mobile Support Navigate as usual, the traffic is recorded Do not launch any browser Set the NeoLoad IP and the recording port Connect the device (and NeoLoad) to a WIFI network
  174. 174. Recording Native and Mobile Web from Device Mobile Support Parameters sent by the mobile app Server used by the mobile app
  175. 175. For Advanced Users • Advanced Installation • Servers • Request & Page Settings • Service Level Agreement • Logical Actions • Variables • Data Exchange API • Mobile Support  Advanced Results • The NeoLoad License AGENDA
  176. 176. Debug Mode • Look at how behave the VU profiles in detail once the test is completed. • Start the debug mode with . • Two modes available (scenario advanced settings): - Only the iterations containing errors are displayed (default behaviour). - All VUs iterations are displayed. • Launching a scenario in debug mode may severely hamper performances. Advanced Results
  177. 177. Debug Mode Advanced Results Open the Check VU window The VU iterations that contain errors The instance of the VU containing errors
  178. 178. Filters • Filter any test result by time, population, zone and error. • When a filter is applied, a new set of results is created. • Filtered results may be viewed and used as any other test runs. • Test reports may be generated following the usual procedure. Advanced Results
  179. 179. Filters Advanced Results Time filter Error filter Zone filterPopulation filter
  180. 180. Analysis: a Multi-Tiers Architecture Advanced Results Web browser Web server App server DB server Potential issues: • Thread saturation • Cache usage Potential issues: • Thread saturation • Data pool saturation • Internal application issue (code, memory leak, garbage collection…) Potential issues: • Cache usage • Internal database issue (no index, costly requests, memory…)• Apache • IIS • Tomcat • JBoss • webLogic • webSphere • OAS… • MySQL • DB2 • Oracle • MS SQL • PostgreSQL • Internet Explorer • Firefox • Chrome • Opera • Safari…
  181. 181. Analysis: HTTP Server Contention • Adding workers (depending on server resources, especially CPU) or an additional Apache server may improve the situation. Advanced Results Limit of the busyworker is reached Response times are impacted
  182. 182. Analysis: App Server Contention • Garbage collection causes the threads to be suspended (not the case with parallel garbage collection). If it takes too long, response times are impacted. Advanced Results JVM overload Response times are impacted Errors raise up
  183. 183. Analysis: Database Contention • NeoLoad performance indicators give a detailed view of the most resource-hungry SQL queries (Oracle and DB2 only). • Such information helps optimizing SQL queries and validating the use of cache, memory, sorts, indexes… Advanced Results
  184. 184. Analysis: Network Contention • In case of a network contention, if the time gap between response times and TTFB: - Increases, that often indicates a network problem. - Remains the same, that often indicates a server problem. Advanced Results
  185. 185. Objectives After completing this exercise, you will be able to: • Use the NeoLoad features to analyze a test result. Scenario 1. Open the project called ‘Training_Project’ and select the test results (‘Charge_Prod_1’ and ‘Charge_Prod_2’). 2. Analyze both results by looking at the response times and each layer of the JpetStore architecture. Exercise 22
  186. 186. For Advanced Users • Advanced Installation • Servers • Request & Page Settings • Service Level Agreement • Logical Actions • Variables • Data Exchange API • Mobile Support • Advanced Results  The NeoLoad License AGENDA
  187. 187. A Combination Of The NeoLoad License Number of VUs Protocol Modules Monitoring Modules
  188. 188. FREE STANDARD PROFESSIONAL ENTERPRISE ULTIMATE Suggested Usage Developers and testers running small tests One tester testing one application at a time Teams testing one application at a time Organizations testing multiple applications concurrently Corporate CoEs and teams needing to run very large tests Virtual Users 50 50 – 1,000,000 50 – 1,000,000 50 – 1,000,000 1,000,000+ Protocols All Included HTTP/S Included, Others Optional HTTP/S Included, Others Optional HTTP/S Included, Others Optional All Included Command Line Access      Continuous Integration Plugin      Monitoring  Optional    Collaboration    Integration & Customization • APM • Data Exchange API • Custom Action Extensions Optional   Shared License   The NeoLoad License
  189. 189. Available Modules The NeoLoad License O/S Web Server DB Server App Server Network Protocol Integration Windows IIS MS SQL Server .NET SNMP SOAP CA / dynaTrace / AppDynamics Linux Apache Oracle JBoss Flex Solaris Nginx My SQL Tomcat Silverlight IBM AIX DB2 WebLogic Oracle Forms HP-UX PostgreSQL WebSphere GWT Data Exchange API VMWare MongoDB Oracle AS Java RSTAT JOnAS Oracle Siebel Adobe LCDS Push GlassFish RTMP Custom Actions SAP NetWeaver Generic JMX Kaazing
  190. 190. Standard • To be used on a single machine • May be rent (a minimum of 8 days) • Testing days must be consumed after the license activation within ‘10 x days’ of the license term Shared • Installed on the Neotys Team Server. • Minimum split is 5VUs. • Modules available for each split. • Cannot be rent. The NeoLoad License
  191. 191. The No Key Mode • Features - Record, design and check new or existing VUs. - Set up already existing monitors. - Analyze existing test results. - Perfect to design before running a test. • Limitations - Tests cannot be run. - No collaboration. - No ‘Monitoring Data Import’. The NeoLoad License
  192. 192. Thank you for attending this training Other courses are available at www.neotys.com

×