Oracle Application Testing Suite: Introduction 11 - What This Class Module Will Cover Methodology - types of performance testing Planning – Setting up and utilizing a performance test plan Configuring/Optimizing - e-Load setup and configuration settings Load testing – running the load test, things to do in real time Instructor Note Intro to class. Inquire regarding the students’ background in load testing, their use with other tools, and so on.
Oracle Application Testing Suite: Introduction 11 - What This Class Module Will Cover (continued) Configuring – Setting up and connecting to backend device counters Resource Monitoring – what to look for during the load test. Analyzing test data – creating e-Reporter graphs Drawing Conclusions – making correlations with the captured data Instructor Note Intro to class continued.
Oracle Application Testing Suite: Introduction 11 - What is performance testing? Many people use the terms “load testing,” “performance testing,” and “scalability testing” interchangeably to describe this type of testing. We’d like to establish a “blanket” term for use during the class as to not confuse those folks who may be accustomed to using different terms.
Oracle Application Testing Suite: Introduction 11 - Why is performance testing necessary? Business requirements are typically defined in the project proposal for the application. Expected load is often determined by Marketing using a tool such as Webtrends
Oracle Application Testing Suite: Introduction 11 - Why is performance testing necessary? (continued) Monitor backend counters to make sure system resources are distributed properly, efficiently, and so on. Bottlenecks item leads into next slide.
Oracle Application Testing Suite: Introduction 11 - Why is performance testing necessary? (continued) Identifying bottlenecks allow us to make these determinations. If something is a bottleneck but not currently serious enough to affect the application, it will most likely be the first thing looked to add scalability later.
Oracle Application Testing Suite: Introduction 11 - Types of “Performance” Testing Each type is different and used to obtain different relevant information about the system under test.
Oracle Application Testing Suite: Introduction 11 - Load Testing The performance requirements will be stated in the test plan and development documentation. Determining the application’s characteristic response times will yield data to be compared to these requirements
Oracle Application Testing Suite: Introduction 11 - Load Testing (continued) Performance testing will, in effect, determine system architecture weaknesses at anticipated user volume
Oracle Application Testing Suite: Introduction 11 - Performance Testing Enhancement strategies: “Will I have to rewrite code to increase performance?” or “Can I just add another Web server or even just another processor within?”
Oracle Application Testing Suite: Introduction 11 - Performance Testing (continued) This will determine which method will work best depending on what platform the application is built on. Some platforms do not have a significant increase in performance by just adding another CPU to the app or web server, but require another complete machine with the business logic. Other simply need more horsepower to do server-side processing or request responses. This, of course, depends on the business logic and content (images, and so on) of the application.
Oracle Application Testing Suite: Introduction 11 - Stress Testing Example: a bottleneck may be a memory leak or when the max number of connections on the application server is reached. Or a the max number of SSL connections to the load balancer (like an F5 Big IP) is reached.
Oracle Application Testing Suite: Introduction 11 - Volume Testing Setting up a distinct test where transactions using/requiring the most data (both on the client side and back end) are used to test the application/system.
Oracle Application Testing Suite: Introduction 11 - Capacity Planning vs. Performance Tuning Capacity planning and performance tuning is performed in order to avoid Re-architecting and to validate the current architecture. Instructor Note This slide is to introduce these concepts the audience, not define or teach how to do each.
Oracle Application Testing Suite: Introduction 11 - Setting up a Test Plan Define the performance and scalability requirements: Derived from business requirements Concurrent users, hits/day, pages/day, and so on Write a test plan What functions will be tested? How will the test be controlled? Who will do the testing, monitoring, and so on?
Oracle Application Testing Suite: Introduction 11 - Setting up a Test Plan Tie the test cases to individual business functions Keep the tests short and modular Keep in mind what the back-end effects are when recording or developing a test. For example: a search might exercise the database, but not some external system connection Know what parts of the system are affected by each of the tests so that proper monitoring can be put in place
Oracle Application Testing Suite: Introduction 11 - Pointers to Keep in Mind Be realistic: If the load generated on your web site during your load test isn't realistic, the results are useless at best and dangerously misleading at worst. Make sure your load test uses a real-world methodology that accurately reflects the rate and pattern that your visitors arrive and leave your web site. Know your Web site visitors: Not all your web site visitors are alike. Your load test should account for different types of visitor behavior on your web site. For example, visitors trying to execute a stock transaction will have very different tolerance and tenacity thresholds than will a user trying to view a weather update. Your load test should accommodate variable online behaviors for different types of actions on your web site. Test scenarios as dynamic as your visitors: Your visitors can interact with your web site in a variety of ways. They can often navigate through a multitude of paths to get to the same endpoint or enter a variety of data inputs to generate dynamic content. Make sure that your web site can accommodate these dynamic patterns without tedious, hard-coding of test scripts and huge resource commitments.
Oracle Application Testing Suite: Introduction 11 - Pointers to Keep in Mind (continued) Understand the cost of poor web site performance: Suppose you know that your web site is performing 5 seconds slower on average than it was a month before. What conclusions can you draw? How many users abandoned your site due to poor performance? At what point did they abandon the site? How much potential revenue was lost as a result? Schedule regular Web site health checkups: If you only load test your site during high-volume periods such as Christmas or Valentine's Day, you may be losing loyal customers to your competitors during the remainder of the year. The key to a high-performing Web site involves more than just running a pre-deployment load test. You must also perform periodic tests throughout the life span of your Web site.
Oracle Application Testing Suite: Introduction 11 - Pointers to Keep in Mind (continued) Test early, and test often: Always plan ahead to make sure your site can handle major site launches, seasonal promotions and marketing initiatives. However, don't ignore day-to-day content changes and modifications to your Web site. What may appear as a minor change to your Web site can negatively impact your Web site's performance and availability. Respond to constant change: Changes within organizations are often unpredictable and difficult to plan. A last minute change to your infrastructure - such as an application upgrade, network configuration change, load-balancer setting or database modification - can cause performance variations. Prepare your Web site by testing these Web site changes with a full-service, turnkey load test and a flexible, self-service testing solution. Simplify Test Scripts for Easy Maintenance: Simplify the administration of your load tests. Your business environment is dynamic - often necessitating last-minute changes to your web site. Your solution should give you the flexibility to schedule your tests at a moment's notice and to change test scenarios on the fly so you won't have to worry about how even the most minor changes will impact your users.
Oracle Application Testing Suite: Introduction 11 - Testing Environment Setup Define what the “test environment” entails: load test hardware, application hardware, network hardware, and other devices. Note: Isolating the test environment ensures the accuracy of the test results. It’s important to be sure that the performance characteristics are specifically of the application.
Oracle Application Testing Suite: Introduction 11 - Testing Environment Setup (continued) Application Hardware: If identical hardware can not be obtained, the environment should be made as close as possible: Web servers Application servers Database servers Firewalls, load balancers, and so on Reproducing the production environment is not always a reality as it’s often not in the budget. A scaled down version is usually much more attainable, however it’s important to keep in mind that creating a scaled down version of the application will not provide accurate insights into the overall performance of the actually production environment. By adding hardware or software, a system’s performance characteristics can be significantly altered. Testing a scaled down version of production will help provide some insights but not all into the overall performance characteristics of the production environment.
Oracle Application Testing Suite: Introduction 11 - Configuring the Load Test Hardware For a large load test, we will have more than one agent machine. It will be necessary to configure them. Instructor Note Briefly discuss what is included in “load test hardware” (controller, console, agent machines).
Oracle Application Testing Suite: Introduction 11 - Instructor Note Mention that mileage may vary depending on factors like script length and content (number of forms, and so on)
Oracle Application Testing Suite: Introduction 11 - Configuring the Load Test Hardware Verifying network access to agent workstations Once you have the e-Load controller and agent software installed on the individual workstations, you should verify network access between the controller workstation and each agent workstation. The following provides basic tips and techniques to make sure the e-Load controller workstation can successfully communicate with each agent workstation. 1. Make sure that you have the e-Load agent software loaded on the agent workstation(s) and that it is the same version as the Oracle Application Testing Suite software that is loaded on the e-Load controller workstation. The workstations you plan to use as agents must have either the e-Load agent software or the full Oracle Application Testing Suite installed to work as agents. 2. Make sure you can successfully ping all of the agent workstations from the e-Load controller workstation. The names you use to ping the workstations are the same names that you will specify for the agent workstations in the e-Load controller.
Oracle Application Testing Suite: Introduction 11 - Configuring the Load Test Hardware (continued) 3. Make sure that the same user is logged on both the e-Load controller workstation and all of the agent workstations. All of the agent workstations must be logged in to be controlled by the controller workstation. You may be able to log in as a different user on the agent workstations as long as the user login has the same administrative privileges as the user logged in on the controller workstation. 4. If the e-Load controller and agent workstations are not participating in the domain security model, both the e-Load controller and the agent workstations must log in as administrator and have the exact same password. 5. In the e-Load controller, add a visual script to the scenario profiles list. Enter the machine name or IP address of the Agent workstation where you want to run the visual script into the workstation field on the scenario tab of e-Load.
e-Load Introduction to Oracle Application Testing Suite