Download as PPS, PPTX
Performance Test Slideshow Recent
- 1. Determine the usability/effectivenessof an application under load. Detect bottlenecks before a new system or upgrade is deployed. Tune for better performance The peace of mind that it will work on go-live day alone justifies the cost of performance testing. Why do we Performance Test..?
- 2. What is PerformanceTesting? Performance Testing Determines Speed Scalability Stability Confidence … while focusing on User Expectations System Constraints Costs Specifically, it answers: How many…? How much…? What happens if…?
- 3. Myths of PerformanceTesting Performance Testing is done to break the system Performance Testing only involves creation of scripts and running the scripts Application Changes cause simple refactoring of performance testing scripts
- 4. WHY NOT MANUALLYPERFORMANCE TEST? Manual performance testing can be done by gathering numerous folks together and synchronizing executing the transactions. The drawback is that measuring response time depends on user accuracy. And it is very hard to synchronize the testing AND repeat the tests. And the coordination problem is expanded if you think about including international users.
- 5. AUTOMATED PERFORMANCE TESTINGReplaces real users with virtual users Generate a consistent, measurable, and repeatable load, managed from a single point of control Efficiently isolates performance bottlenecks User Simulation Controller Web Server Application Server Database Internet / WAN
- 6. Performance Test Process“Evaluate System” This is the most important process because it involves … predicting actual user experience accessing any system limitations defining stakeholder expectations
- 7. Performance Test Process“ Draft test scripts/scenario” The scripts will contain transactions which are the most intensive activities performed on the application.
- 8. Performance Test Process“Execute Performance Tests” LOAD TESTS are the end to end performance test under anticipated production load. STRESS TESTS determine the load under which a system fails and how it recovers from failure. DURATION TESTS are test with a constant load over a period between 8 and 24 hours to determine if an application’s performance degrades over an extended period of time. BASELINE TESTS is the execution, validation, and debugging of scripts collectively in a test scenario. The results will be used for comparison with future testing results.
- 9. Performance Test Process“Entire Process” Response Time Degradation Curve
- 10. Performance Test ServicesDatabase EJB JDBC JSP Sitraka JMonitor Oracle MSSQL Server DB2 Network Web Servers App Servers Java 3270 5250 VT100 Legacy Oracle MS SQLServer DB2 ODBC Databases EJBs CORBA COM RMI MQSeries Middleware Web Services HTTP(S) XML Citrix ICA Web PeopleSoft Oracle ERP/CRM Protocols Monitors Diagnostics Operating Systems Databases Windows Unix Linux SNMP WAN Emulation MS IIS iPlanet Apache BEA WebLogic IBM WebSphere ATG Dynamo iPlanet App Server J2EE .NET Platforms We have the ability to mimic many protocols. Here is a list of the protocols. The Diagnostics tool provides a set of Diagnostics modules that trace, time, and troubleshoot end-user transactions across ALL tiers. And while a testing is executing we will monitor your servers! Web Server Application Server Internet/WAN Load Generator Controller Load Balancer
- 11. Performance Test Triggers?New Systems Increasing Number of Users New Functionality Software Upgrades Operating System Upgrades Hardware Upgrades Infrastructure Upgrades/Changes Slow application response Here are some examples of what should trigger a performance test execution!
- 12. Performance Test ToolboxOur team has the latest and greatest tools…!!!
- 13. Project Team CollaborationQuality Assurance Develop Manual Test Conditions, test data, and expected results Business Analysts Identify and document business performance requirements Project Managers Manages the application implementation DBAs Analyzes and tunes the database System Administrators Server System Administrator, Configure downstream and Production environments Architects Closely involved in identifying bottlenecks, tuning Infrastructure Manage and secure the platform on which applications reside A successful performance test execution relies on involvement from the entire team..!!
- 14. How do youengage our team? Answer: Just open up a Performance Test SOS ticket… http://eaptsos More information about performance testing is on the http://tim.turner.com/tso/ea/groups/sharedsvc/perftest/default.aspx
Editor's Notes
- #3 Speed: Ex. How long should it take for a response to display when submitting a request? Scalability Ex: How many db connections are configured? So what happens when you exceed that threshold? Stability: Ex: How does the application behave while a batch process is running in the background? Confidence Ex: Once the results are in – is the sponsor of this project confident in the behavior of this application under load? Costs: Focus: User Expectation: What is the acceptable response time for a response on the web page? System Contraints: What is the concurrency of users which causes the application to degrade in performance…? What happens if we make a change to application/architecture/infrastructure…? Cost: What is the cost to the business if the application is unusable due to slowness? Or worst case scenario it crashes?
- #4 Performance Testing is done to break the system - Normal load testing is done to understand the behavior of systems under expected load. Specifically Stress Testing is done to study breaking points of systems. Performance Testing only involves creation of scripts/running scripts - The scripting process is important, but it is only a small component of performance testing. The challenge of Performance Testing is determining what tests need to be executed and knowing how to interpret results to determine where any problems/bottlenecks might occur and what the behavior will be. Application Changes cause simple refactoring of performance testing scripts - Any change to the UI of a product (especially a web product) will cause re-development of scripts. If the changes are drastic, complete re-development of scripts may be in order. This point emphasizes that products need to be in a stable state before undergoing performance testing.
- #5 Why can’t you just synchronize some real users to test your application at the same time while recording the response time with a stop watch? Drawbacks: (1) Measuring response time would depend on the users accuracy. (2) It is very hard to synchronize the testing AND repeat these tests. (3) And the coordination problem is expounded if you think about including international users. The time spent attempting to coordinate would outweigh the upfront time it would take to automate the performance testing. Basically this type of manual testing is not feasible for a system that must support a load and requires repeatable test.
- #6 So what happens with Automated Performance Testing?
- #7 In order to determine the System’s Purpose and what Activities it is used for will need to measure or estimate: - All possible user activity. - How often each activity is accomplished. - All “types” of users. - What activities are most performance intensive. - Other user community modeling information. Basically model “Real” users. There are some tools in place that could be used to gather user statistics/data. Like the site metrics tool called NetInsight. This provides the ability to see how the applications is being used by tracking the pages visited. Also there is the monitoring tools from the BAC in which server statistics - Must solicit requirement from all stakeholders to determine acceptable performance. Developers, Business, Infrastructure, Architecture -
- #8 Basically model “Real” users. We’ll take your business transactions and script the navigation. - The only way to predict actual user experience (end-to-end response time) is to execute tests using realistic User Model(s). - Extrapolating expected performance based on incomplete models does not work.
- #9 Baseline is the execution, validation, and debugging of scripts collectively in a test scenario. These results will be used for comparison with future testing results. LIMITED AMOUNT OF VIRTUAL USERS. Load Tests are end to end performance tests under anticipated production load. The primary objective of this test is to determine the response times for various time critical transactions and business processes and that they are within documented expectations (or Service Level Agreements - SLAs). The test also measures the capability of the application to function correctly under load, by measuring transaction pass/fail/error rates. This test is one of the most fundamental load and performance tests and needs to be well understood. Stress Tests determine the load under which a system fails, and how it fails. This is in contrast to Load Testing , which attempts to simulate anticipated load. It is important to know in advance if a ‘stress’ situation will result in a catastrophic system failure, or if everything just “goes really slow”. Duration Tests (and / or stability test) are test with a constant load over a period between 8 and 24 hours. Duration tests will be conducted to determine if an application's performance degrades over an extended period of time. This type of testing may indicate if there are issues such as memory leaks that could affect the application's performance. Unit Test ???? Web service test ???? << need to elaborate
- #10 <<redo the graph>> The results from each test iteration will be reviewed with the team to determine if requirements have been met. If the requirements are not met – then tuning of the application and server After each test execution you will receive performance test results that explain the outcome of the test. At the completion of performance testing the team will receive a complete result document which summarizes the overall test objective, results found, and the tuning recommendations/modifications implemented.
- #11 The tools that we use have the widest application support of any automated performance testing solution in the market. Not only do we support web solutions and ERP/CRM (Peoplesoft applications), but we also support all the common databases, middleware and legacy solutions. LoadRunner can monitor the entire system from the OS to the network, the custom or packaged application itself and even the database. Monitoring:
- #12 Interactive Slide – see if audience can name all the reasons we would test. Question: What events should trigger a performance testing cycle? New System How visible is this application? What are the critical performance metrics? What is the usage pattern? Will this application or new feature be used often? What capacity needs to be supported? Is it required that this application be available 24/7? How many users need to be supported? Will this application be used by a significant amount of users? This one is pretty obvious – if it is customer facing and is expected to handle any substantial load, it must be tested. Increasing Number of Users Is this application going to be release to an additional set of users? What’s the impact? How is the system currently being used? What features are being utilized? What is the point at which system performance becomes acceptable? Hopefully this has been accounted for by previous testing, if not new load tests must be performed to verify that the increased number of users will be handled by the system. New Functionality What is the change to the usage pattern? What features drive the system performance? Software Upgrades This includes code changes due to major fixes that change the behavior as well as full releases. Operating System Upgrades This one should be obvious – changes in operating systems can involve ways that OS’s handle threading, memory, etc. Does it need to work with VISTA or a MAC? Hardware Upgrades Upgrades to servers will require performance testing to study how the upgrades add to the capacity of applications. Infrastructure Upgrades Changes to load balancing, network pipes, etc.
- #13 - Performance Center provides a &quot;global&quot; access to testing resources across the enterprise via a Web-based interface. This allows the IT organization to centrally manage those testing resources to perform load testing on an enterprise wide scale — increasing resource productivity and testing capacity. - The Virtual User Generator (VuGen) allows a user to record and/or script the test to be performed against the application under test , and enables the performance tester to play back and make modifications to the script as needed. Such modifications may include Parameterization (selecting data for keyword-driven testing ), Correlation and Error handling. LoadRunner supports several protocols like Web HTTP/HTTPS, Remote Terminal Emulator, Oracle and Web Services. - Sitescope provides the ability to set up monitors during a load test to monitor the performance of individual components under load. Some monitors include Oracle monitors, WebSphere monitors, etc... - Analysis This tool takes the completed scenario result and prepares the necessary graphs for the tester to view. Also, graphs can be merged to get a good picture of the performance. The tester can then make needed adjustments to the graph and prepare a LoadRunner report. The report, including all the necessary graphs, can be saved in several formats, including HTML and Microsoft Word format - HP Performance Diagnostics for .NET and J2EE (Java) provides comprehensive visibility into .NET applications deployed in heterogeneous composite environments. Out-of-the-box dashboards provide quick time to value. Visual Studio Team System includes a load test tool. This new tool provides many exciting features you can use for performance and stress testing your Web sites, Web services, and other server components. This is used against application with Janus Security. Allows us to reference the .dll’s that we need to use the bindings that work with Janus.
- #14 In order for the performance test effort to be successful – it is a collaborative effort between these team members. It is imperative that during performance test cycles that all of these team members are involved and aware of any changes that comes out performance testing.
- #15 We have our remedy queue. Just enter information and the request will be assigned to one of us. We’ll contact the project team to start the planning process. On the eawiki – we have a slew of information, demonstrations on how to use our Loadrunner and MS Visual Studio tools…