Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
34 Testing Experience – 21/2013IntroductionApplications developed for various domain verticals like BFS, Insurance,Healthc...
Testing Experience – 21/2013 35▪▪ Batch completion time: The completion time needs to be specifiedbased on what type of ba...
36 Testing Experience – 21/20131.	 Engage early with stakeholders and collate details such as trans-actions performed for ...
Upcoming SlideShare
Loading in …5

Article by Marlabs Bangalore employee receives international recognition!


Published on

“Testing Experience”, a Germany-based online magazine for software testers and test managers, has published an article by Ramesh Viswanathan, Senior Test Architect, Marlabs Bangalore in their March 2013 edition. Ramesh has presented his observations on the topic ‘Need for Performance Requirements to Ensure Reliable Business Applications’ through this article. It has incorporated topics associated with application development for various industry domains, techniques and tools for non-functional requirements gathering, optimal performance target, and best practices for developing resource-specific outputs.

Testing Experience serves as a platform for knowledge transfer in software testing projects, with more than 250000 downloads per issue, in over fifty countries. Marlabs congratulates Ramesh Viswanathan for this accomplishment. We hope and wish that such valuable tech-philosophies lead him to better avenues of success in career!

For viewing the detailed version of Ramesh’s article, refer the following link -

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Article by Marlabs Bangalore employee receives international recognition!

  1. 1. 34 Testing Experience – 21/2013IntroductionApplications developed for various domain verticals like BFS, Insurance,Healthcare, e-Learning, etc. have a huge volume of transactions in a day.While there is sufficient focus and due-diligence in the industry to de-fine the functional requirements, the absence of proper non-functionalrequirements in many instances can lead to failure of applications andadverse business impact. Hence, performance requirements gatheringplay a very important role in the software development life cycle.Requirements gathering for performance testing can be fora. a new business application, orb. for an application already existing in production.There is always the need to systematically collate requirements withrespect to performance testing. These need to be captured at an earlystage of the software life cycle and signed off by all the key stakeholders.At a high level, performance requirements can be categorized as below:Requirements CategoriesCategory 1 – WorkLoadStudying and understanding the specifics that relate to load on thesystem in terms of the number of transactions that need to be simulta-neously processed, or the amount of processing such as arrival patterns,navigation trends, and user behavior for completion.Aspects related to Category 1 – Workload include:▪▪ Online transactions processing▪▪ Handling batch jobs▪▪ Transactions related to reports▪▪ Volume of data in the back end▪▪ Different user roles in the application▪▪ Other external interfaces that communicate with applicationsCollating of the list of transactions that are Business Critical, High Re-source consumption, High Frequency of Use and Most Commonly used,reports that are generic/graphic formats and generic/critical batch jobsis normally easy, but getting detailed information in respect of the rateof transactions per minute or per second is not straight forward andneed some formulas to capture them.For a system in production, this data can be obtained by means of ana-lyzing transaction history data in databases, and web and system logsthat contain timestamps.For a new system that is yet to be implemented, this is purely based onbusiness inputs and assumptions.Data volumes are easier to estimate for an existing system and theycan easily be captured by analyzing counts from different tables inthe database. Statistics about the number of registered users are usu-ally available, but estimating the number of concurrent users is neverstraightforward.Category 2 – Performance TargetsGathering performance targets such as response time, throughput, andresource utilization.For performance targets, the related items below cover the majorityof needs:▪▪ Online interaction response time:▪▪ Based on the category and type of interaction, for examplein the case of a simple click such as viewing the page andclicking the next button, it may have a smaller target than atransaction submit operation that requires entering manda-tory fields and clicking on a submit button.▪▪ Network bandwidth considerations: As an example, for anintranet based application like an HR portal, the responsetime may be 8 seconds and applications accessed by end usersthat are Internet facing may have response time of 15 secondsbased on the type of operation performed.▪▪ Sub-transactions that form the complete transaction and thetime for overall processing.▪▪ Handling and completing online transaction activity: For example,if a user needs to complete a transaction in 3 minutes the transac-tion has 5 web interactions to complete, and each transaction re-quires 20 seconds for data entry, etc. to be successfully completed,the calculated average response time for a single interaction shouldnot exceed (3*60−5*20)/5 seconds = 16 seconds.▪▪ Delivery time for asynchronous transactions: Some factors to beconsidered when setting completion time targets are:▪▪ Based on the type of transaction.▪▪ Based on the network bandwidth.▪▪ Based on the number of interactions within the application/system architecture before reaching the final destination.▪▪ Transaction throughput: Throughput refers to transactions pro-cessed per unit time as well as throughput for different type ofreports and their completion time, which need to be classified intoscheduled and ad hoc reports.By Ramesh ViswanathanNeed for Performance Requirements toEnsure Reliable Business Applications
  2. 2. Testing Experience – 21/2013 35▪▪ Batch completion time: The completion time needs to be specifiedbased on what type of batch process/programs are running thatalso involve any backup operations▪▪ Understanding the system resources/resource consumption underperformance targets also plays an important role and a few arementioned below:▪▪ CPU utilization: This is the percentage of time the CPUs of thesystem are busy. It is desirable not to have CPU usage of more than70 %.3. Memory consumption: This is the number of MB or GB of thesystem’s RAM consumed.4. Disk Utilization: Disk utilization and I/Os per second of thedisks subsystem are often measured to plan for capacitymore holistically.5. Network bandwidth: The metric can be either in Kbps orMbps. It is always a good practice to have a target set fornetwork consumption overall as well as per user. An examplecould be 15Kbps per user and 1 Mbps overall from identifiedcritical branch to a centralized server. Network bandwidthtargets are dependent on available bandwidth and usage byusers of various roles and geographic locations.▪▪ In addition to resource consumption counters such as CPU,Memory, Disk and Network, application specific counters need to bedefined and added as part of monitoring, e.g. for an ASP.NET-basedapplication, few of the counters like Requests/sec, Requests Execut-ing, Transactions Total, and Transactions/sec, for SQL counters likeLogins/sec, Logouts/sec, UserConnections, and Buffer Cache HitRatio.Techniques and Tools for Non-Functional Require-ments GatheringItisalwaysagoodpracticetoproceedusingapropermeansofcapturingthe NFR details, including and not limited to the following:▪▪ Non-functional requirements questionnaire: This is a list of itemsthat can assist in forming and understanding the requirements bet-ter to ensure a quality output. Questionnaire collation for differentsections is provided below:▪▪ Business test cases requirements▪▪ Performance testing requirements▪▪ Monitoring requirements▪▪ POC requirements▪▪ External application/server requirements▪▪ Workload modeling and WLM tools:▪▪ Identifying objectives and related sub-categories▪▪ Collating scenarios that are critical and closely related to thebusiness▪▪ Determining the associated navigation paths for the criticaland key scenarios▪▪ Isolating the unique data for the associated navigation pathsas well for the simulated users▪▪ Finding the distribution of scenarios and each scenario that isa business test case▪▪ Categorizing the load levels for different scenarios from theidentified target user load▪▪ Formulating the engineering approach to instrument themodelA common means of workload modeling can be performed by under-standing the web-server logs. There are a few commercial and opensource tools to extract and report the details as needed.▪▪ Understanding future growth: This forms the capacity planningapproach to cater for handling future loads as the application ma-tures and more features are implemented to market it long-termin the competitive e-industry. Some of the high level details areprovided below:1. Determine service level requirements: The first step in thecapacity planning process is to categorize the work done bysystems and quantify users’ expectations of how that workwill get done. This is done by defining workloads, determin-ing the unit of work, and identifying service levels for eachworkload.2. Analyze the current capacity: The existing capacity of thesystem must be assessed and analyzed to determine howthe needs of the users will be met. This is done by measuringthe service levels and comparing them to objectives, gaugingoverall resource usage, and computing the resource usage byworkload.3. Planning for the future: This involves using forecasting meth-odologies for future business activity, thereby assessing anddetermining future system requirements. After assessment,the immediately required changes are incorporated in systemconfiguration. This will ensure there is sufficient capacity andis made available to maintain the service levels designedBest PracticesThe best practices mentioned below will assist in ensuring that therequirements are accurate and testing is aligned with them.
  3. 3. 36 Testing Experience – 21/20131. Engage early with stakeholders and collate details such as trans-actions performed for a specified period of time, time for eachtransaction, duration of peak activity, and usage.2. Process the collated details, segregate and avoid duplications, e.g.ensuring requests are from unique IP addresses and filtering allerror-related pages to end up with a work load model.3. Understand the business needs, e.g. number of peak usages, trans-actions per second, etc. Map the collated details with businessneeds.4. Organizations need to have a technical and high quality engineer-ing team that will assist in verifying and validating the require-ments.5. Develop a testing strategy and an appropriate testing tool basedon the platform and technology identified.6. Deriving an accurate work load model with an appropriate teststrategy and supporting test tools ensures that the performancetesting outcome meets expectations and is aligned with businessobjectives.SummaryThis article highlights the importance of defining and collecting re-quirements which is usually given secondary importance compared tofunctional requirements.Theaspectsandparametersofperformancerequirementsareillustratedwith examples to provide a practical thought process.By using tools, techniques and an engineering approach non-functionalrequirements (NFR) gathering can be made more scientific.Performance can be built into the system and validated at every stagethrough the appropriate focus on business, technology, tools, and thesoftware testing processes as demonstrated in the “Best Practices” sec-tion. ◼Ramesh Viswanathan is a Technology Master of Engi-neering in Communications Systems and a Master ofBusiness Administration in Operations Management,and is both Siebel and ISTQB Certified. He currentlyworksasaSeniorPerformanceTestArchitectatMarlabsInc, USA. He has been in the SoftwareTesting industrysince 1999, working with various organizations in thepast such as ReadyTestGo, CognizantTechnology Solu-tions, Symphony Software Services, ANZ Bank Information and Technology,and SunGard Global Solutions> about the authorLicense ISTQB®and IREB®training materials!Díaz & Hilterscheid creates and shares ISTQB®and IREB®training material that provides the resources you need toquickly and successfully offer a comprehensive trainingprogram preparing students for certification.Save money and save time by quickly and easily incorpo-rating our best practices and training experience into yourown training.Our material consists of PowerPoint presentationsand exercises in the latest versions available.Our special plus: we offer our material in four differentlanguages: English, German, Spanish and French.Díaz & Hilterscheid Unternehmensberatung GmbHKurfürstendamm 17910707 BerlinGermanyPhone: +49 (0)30 74 76 28-0Fax: +49 (0)30 74 76 28-99E-mail: training@diazhilterscheid.comWebsite: training.diazhilterscheid.comFor pricing information and other product licensing requests, please contact us either by phone or e-mail.