Criteria for performance evaluationFirst of all, let us see what is the meaning of the term "Performance Testing":For general engineering practice, "Performance Testing" refers to evaluation &measurement of functional characteristics of an individual, a system, a product or anymaterial.However in software industry parlance, the term "Performance Testing" widely refers tothe evaluation & measurement of functional effectiveness of a software system or acomponent, as regards its reliability, scalability, efficiency, interoperability & its stabilityunder load.These days a new science by the name "Performance Engineering" is emerging in ITindustry & Performance Testing / Acceptance Testing are being viewed as its subsets.The performance engineering lays prime emphasis on covering the performance aspectsin the system design itself i.e. right from the beginning & more important is that wellbefore the start of actual coding.Why Software Industry lays so much emphasis on Performance Testing:The key reasons are:1) Performance has become the key indicator of product quality and acceptanceconsideration nowadays in a highly dynamic & competitive market.2) Customers are becoming extremely demanding on quality front & have clear vision oftheir performance objectives.3) These days, every customer is looking for greater speed, scalability, reliability,efficiency & endurance of all applications - may it be multi tier applications, web basedapplications or client server applications etc. etc.4) Greater need for identifying & eliminating the performance inhibiting factors earlyduring the development cycle. It is best to initiate the performance testing efforts rightfrom the beginning of the development project & these remain active till finaldeployment.What are the objectives of Performance Testing?1) To carry out root cause analysis of performance related common & uncommonproblems & devise plans to tackle them.2) To reduce the response time of the application with minimal investment on hardware.3) To identify the problems causing the malfunctioning of the system & fix them wellbefore the production run. Problems remedied during later stages of production have highcost tags attached to them.
4) Benchmarking of the applications, with a view to refine the companys strategytowards software acquisition for the next time.5) To ensure that the new system conforms to the specified performance criteria.6) To draw a comparison among performance of two or more systems.Typical Structure of a Performance Testing Model:Step-1: Collection of Requirements - The most important step & the backbone ofperformance test modelStep-2: System Study.Step-3: Design of Testing Strategies - Can include the following.a) Preparation of traversal documents.b) Scripting Work.c) Setting up of test environment.d) Deployment of monitors.Step-4: Test Runs can cover the followinga) Baseline Test Runb) Enhancement Test Runc) Diagnostic Test RunStep-5: Analysis & preparation of an interim report.Step-6: Implementation of recommendations from step-5.Step-7: Preparation of a Finalized Report.Attributes of a Good Performance Testing setup:1) Availability of a performance baseline document detailing the present performance ofthe system & acting as an effective baseline, which can be used in regression testing. Thisbaseline document can be conveniently used to compare the expectations when thesystem conditions happen to change.2) Performance test beds & test environment should be separate & must replicate the liveproduction environment as far as possible.3) Performance testing environment should not be coupled with the developmentenvironment.4) Resources leading to fulfillment of objectives like:# Deployment of personnel with sound knowledge# Systematic & deliberate planning# Study of existing infrastructure# Proper preparation# Systematic execution# Scientific analysis# Effective reporting
However these days many companies have started doing part of the testing under the liveenvironment, This helps them in establishing points of differences experienced duringtest & live systems.How to gear up for Performance Testing?1) Define the performance conditions: First of all we need to define performanceconditions related to functional requirements like speed, accuracy & consumption ofresources. Resources can be like memory requirements, storage space requirements &bandwidth of the communication system etc. etc.2) Study the operational profile: The operational profile contains details of usage patternsand environment of the live system. It includes description of the period of operation, theoperating environment, quantum of loads & expected transactions etc. When exact data isnot available, the data from the testing profiles can be approximated especially whentesting is not being done under the live environment.3) Prepare good performance test cases: While designing performance test cases, ourendeavor must be toa) Understand the present performance levels & to use this information for benchmarkingat a later date.b) Evaluate the performance requirements of the system against the specified norms.c) Clearly specify the system inputs and the expected outputs, when the system issubjected to the defined load conditions like profile of the test, test environment & thetest duration etc.Ways of doing Performance Testing:Conventionally there are two methods of performance testing like1) Manual performance testing2) Automated performance testing1) Manual Performance Testing: In order to develop an adequate confidence, theresponse times being a good indicator of performance of a transaction must be measuredseveral times during the test. Use of stopwatches monitored by many persons is one ofthe oldest & effective way to measure the test performance. Depending upon theavailable infrastructure, other means can also be devised.2) Automated Performance Testing: Many approaches can be practiced here. We can usethe automation software which can simulate the users actions & can simultaneouslyrecord the response times & various system parameters like access of storage discs, usageof memory & queue length for various messages etc. etc.We can provide additional data load over the system, through many utility programs,message replication programs, batch files & many protocols analyzing tools etc.Important Considerations for Designing Good Performance Test Cases:
1) Stress: To take care of the ability of a system or its component to move beyond thespecified limits of performance requirements.2) Capacity: To cover the maximum amounts which can be contained, or produced, orcompletely fully occupy the entity.3) Efficiency: To take care of the desired efficiency measured as the ratio of volume ofdata processed to the amount of resources consumed for the particular processing.4) Response time: To take care of the specified requirements of response time i.e. thetotal time elapsed between the event of initiation of request to the receipt of response.5) Reliability: Must be able to deliver the expected results with sufficient consistency.6) Bandwidth: Must be able to measure & evaluate the bandwidth requirements i.e. theamount of data passing across the system.7) Security: Must be able to evaluate the user confidentiality, access permissions & dataintegrity considerations in the system.8) Recovery: Must be able to subject the system under test to higher loads, and measurethe time it takes to the normal situation after withdrawal of loads.9) Scalability: Must be able to handle more loads by the addition of more hardwareelements components without any coding change.Lessons learned:Performance engineering approach encompassing load testing, stress testing or endurancetesting is extremely important acceptance consideration in todays highly competitivemarket with highly demanding & quality conscious customers.http://performanceappraisalebooks.info/ : Over 200 ebooks, templates, forms forperformance appraisal.