Performance testing is used to observe a system's behavior under load by simulating virtual users. It determines key performance metrics like response time and throughput. There are different types including load testing, stress testing, endurance testing, and scalability testing. Performance testing involves requirement gathering, workload modeling, test environment setup, load test execution, analysis and reporting. It is important to identify bottlenecks proactively and ensure good user experience. Common tools used include JMeter, LoadRunner, and ApacheBench.
2. DefinitionofPerformance
Testing
Performance testing is an engineering process in which the behavior of the
software, hardware or network is observed by putting a load of virtual users
on the system. Performance parameters like Response Time, Transactions/Second
and Throughput etc. are determined in general cases.
2
http://p3lang.com
That was a high-level definition, to explain it better let me take an example, for instance, there is a web application (assume an ecommerce website). On an ecommerce website a user expect a product to be loaded as fast as possible, even faster than an eye blink, Images should be displayed quickly, browsing of the products should be fast.
In order to achieve that performance testing is required.
That was a high-level definition, to explain it better let me take an example, for instance, there is a web application (assume an ecommerce website). On an ecommerce website a user expect a product to be loaded as fast as possible, even faster than an eye blink, Images should be displayed quickly, browsing of the products should be fast.
In order to achieve that performance testing is required.
Load Testing: Load testing is usually done while performing a performance test. This is done to observe the response time, throughput, hits/second and other performance parameters under a reasonable load to understand the behaviour of the system. This test, in general, is of a shorter duration like 1 or 2 hours. This test is generally executed multiple times. Every time a tweak is made by the developer, the performance tester execute this test to check the performance gain.
Volume Testing: Volume testing is done for a duration of 1 to 2 hours depending on the need. Where the software is subjected to a huge volume of data. It is also referred as Flood Testing.
Stress Testing: This testing is done to determine the system's robustness in terms of extreme load and helps application administrators to determine if the system will perform sufficiently in case the current load goes well above the expected maximum.
Endurance Testing: To determine if the system can sustain a continuous expected load. During endurance tests, memory utilization is monitored to detect potential leaks, i.e. applying a significant load on a system for an extended, significant period of time. The goal is to discover how the system behaves under sustained use. This type of Performance testing is also known as Soak Testing.
Spike Testing: is done to test the sudden increase in the number of virtual users or load generated by the users to a very large amount and observing the behavior of the system. The goal is to determine whether performance of the system will suffer/fail or it will be able to handle the dramatic changes in the load.
Scalability Testing: is the testing of a software application to measure its capability to scale up or scale out in terms of any of its non-functional capability. Performance, scalability and reliability testing are usually grouped together by software quality analysts.
Baseline Testing: It is also referred as Benchmark testing. Performance testing starts with performance baseline testing, performed by the test engineer for comparing the performance of a new server or an unknown server to a known standard of reference like existing measurements or a software.
Smoke Testing: A smoke test is a test run under very low load that merely shows that the application works as expected.If smoke is generated, the test fails and no further testing is necessary until the simplest test passes successfully.
Requirement Gathering: During the requirement gathering phase performance tester/engineer asks questions that will help in understanding the purpose of the application, technologies used to develop the application, concurrent load with which application needs to be load tested and purpose of the load testing, etc.
Workload Modeling: Identification of the application workflows that needs to be tested and the concurrent user load on each of the workflow. There is always a confusion in identifying the person who can suggest the workflows to be tested. The workflow identification should be done by the Business Analyst of the application and performance tester should be engaged as a consultant, but this responsibility lies with the Business Analyst. While the responsibility to give the final shape to workload model lies with the performance tester/engineer.
Scripting: In this phase a performance tester/engineer creates the performance scripts using tools like JMeter. Which includes recording of the workflow, putting parametrization, correlation of dynamic values, put loops for recurring events, assertions points and wait timers to make it a real time scenario.
Test Environment Setup: Creating execution plan in the performance testing tools which includes the setting up of the performance scripts to execute, configuring the runtime settings like pacing, ramp up time, timeout time, deployment of the server side monitoring tools, application and database profiling tools.
Load Test Execution: In this phase, the performance tester/engineer executes the test plan to put the load on the application and monitors the load test results like response time, errors, hits/second, etc.
Analysis & Reporting: After the load test execution, it’s time to analyze the raw results and find out the bottlenecks in the application. A performance report is created by the performance tester/engineer which is then shared with all the stakeholders to take necessary action.
We are living in a world where irrespective of the collection and quality of the products user will lose interest if the speed of the website or application is slow. According to surveys done by Akamai and Gomez.com, nearly half the web users expect a site to load in 2 seconds or less, and tend to abandon a site that does not load within 3 seconds. 79% of web shoppers who have trouble with a website performance say they won’t return to the site to buy again and around 44% of them would tell a friend if they had a poor experience shopping online. This means you’re not just losing conversions from visitors currently on your site, but that loss is magnified to their friends and colleagues as well. The end result –lots of potential sales down the drain because of just a few seconds difference.
There are chances that during heavy load the hardware or server may crash which will cause user to lose trust in your site. This makes an impression on the user that the website owner is not serious about their business. This impression will last too long and because of which users avoid falling on such websites second time.
JMeter : is the most widely used tool in performance testing. Not just because it’s free, but also due to the fact that it is equipped with all the necessary features. If it doesn’t have any particular feature, then you can easily build that feature as it’s open source as well. As JMeter is free, open source and widely used it’s quick to learn and interesting to use.
Loadrunner : can test almost any type of application which makes it unique, but the cost of doing performance test using Loadrunner is too high, only big companies with high revenues can afford this tool. Which restricts the use of this tool in small and mid level organizations.
Microsoft VSTS or Microsoft Visual Studio Team System : is a performance testing tool from Microsoft. This tool is best suited for .Net applications. We find performance testers using VSTS for the performance testing of .Net applications because by using VSTS the scripting time of .Net applications reduces to some extent.
That was a high-level definition, to explain it better let me take an example, for instance, there is a web application (assume an ecommerce website). On an ecommerce website a user expect a product to be loaded as fast as possible, even faster than an eye blink, Images should be displayed quickly, browsing of the products should be fast.
In order to achieve that performance testing is required.