Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. FINAL PROJECT: SOFTWARE VERIFICATION & VALIDATION Date: June 10, 2009 Authors: Amin Bandeali ( Instructor: Dorota Huizinga
  2. 2. Contents 1. Introduction 1 2. NUnit 1 a. Introduction 1 b. URL 1 c. Primary Features 1 i. Test Harness 1 ii. Assertions 1 iii. Custom Assertions 2 iv. Setup and Tear Down 2 v. Attributes 2 vi. Test Runner 2 vii. Results in XML format 2 d. Software Verification 2 e. Software Phases 2 f. Author’s Comments 2 3. HP Load-Runner 3 g. Introduction 3 h. URL 3 i. Primary Features 3 j. Virtual User Generator 3 k. Controller 3 viii. Load Generators 3 ix. Analysis 4 x. Launcher 4 l. Software Validation (Performance Validation) 4 m. Software Phases 4 n. Author’s Comments 4 4. References 6
  3. 3. Final Project: Software Verification & Validation 1 1.Introduction The emerging need for good software requires detailed specifications and thorough verifications against them. There are two ways to ensure the quality of a software component or software system. One way is to identify and then remove defects introduced; this is done by means of different verification and validation activities. Another way is to prevent the defect introduction; this is done by means of an appropriate development process [1]. The purpose of verification is to assure that the software components or software systems meet their specified requirements [2]. Specified requirements could be found in SRS, architecture, design and regulations (standards) documents. The purpose of validation is to demonstrate that a software component or system fulfills its intended use when placed in its intended environment [2]. Inspection and tests could be used for verification and validation [3]. In this paper I would discuss two verification and validation tools namely NUnit and HP Load-Runner. 2.NUnit a. Introduction Software testing measures the verification of software against both functional and nonfunctional acceptance criteria. Unit testing ensures that software does what it commits or is expected to do. Unit testing started with xUnit. NUnit is a port from Java’s JUnit. It facilitates unit testing by providing a framework to support test development and test runners to run the tests and assess their results [4]. It comes with a console application or a GUI application for testing. b. URL c. Primary Features i. Test Harness In NUnit, a test harness is called a test fixture. It provides an infrastructure to hold related tests. It contains the logic to setup and tear down the collection of tests in the harness, as well as for each of the individual tests. [4] ii. Assertions Assertions test whether the program is behaving as expected by verifying that an actual post-condition matches the expected post-condition [4]. If multiple assertions exist, the test case would stop executing as soon as an assertion fails. These are the most important tools provided by NUnit for testing as they actually verify functionality.
  4. 4. Final Project: Software Verification & Validation 2 iii. Custom Assertions These are created by extending the NUnit framework. The primary purpose of a custom assertion is to have application specific assertions which are easy to maintain and extend. iv. Setup and Tear Down Setup and teardown code allow you to segregate and reuse common initialization, and clean up code across tests. v. Attributes NUnit provides four attributes (two for tests and two for test fixtures) that can be used to mark methods that provide setup. vi. Test Runner NUnit provides two types of runner interfaces – GUI and console. Both interfaces provide similar functionality [4]. Test runner GUI is most useful while developing, which test runner console is useful in automated testing environments. vii. Results in XML format NUnit allows you to save results in XML format from both the test runner GUI and console [4]. d. Software Verification Unit testing begins with acceptance criteria, which is identified during the requirement gathering/analysis process. Unit testing verifies exactly what the code is supposed to do and if the results match the acceptance criteria. Unit testing reduces both integration and system testing efforts since it verifies the component functionality right after development. NUnit uses two techniques to verify code: Equivalence sets – a set of input values that are processed in an equivalent fashion and produce qualitatively the same behavior to make a test succeed and Boundary values – a set of input values that are used to test conditions where system behavior should or might change [4]. e. Software Phases NUnit (unit testing) is used in the implementation and verification phase of the waterfall model. It is used in implementation phase because if one component implementation is dependent on another component, the dependency should be thoroughly unit tested. It is also used in verification phase, where each component is thoroughly tested for verified functionality. f. Author’s Comments I have used NUnit since I have started working on the .NET framework. I personally use Test Driven Development and NUnit is a very important part of my work. TDD is
  5. 5. Final Project: Software Verification & Validation 3 the iterative process of designing and building software one unit test at a time [4]. This is how I use TDD with NUnit: • I write a test for a new piece of functionality that I would like to implement (Test currently fails). • Implement the minimum amount of functionality that the test passes with. • Refactor, improve as necessary. The benefit of TDD in my opinion is that it gives maximum code coverage and maximum verification. Hence, a better quality code. 3.HP Load-Runner g. Introduction The explosion of software complexity introduced tremendous risk into the software development process with an incredible rate of change. Rigorous testing and verification is the most common strategy to both quantify and reduce this risk to the business. Performance testing accurately tests the end-to-end performance of a system prior to going live. Performance testing solutions should be able to: • Emulate hundreds or thousands of users interacting with the system without large hardware demands • Measure end-user response times • Repeat the load in a consistent way • Monitor system components under load • Provide robust analysis and reporting engines HP LoadRunner software is used more than any other testing solution in the world today [5]. HP LoadRunner captures the end-user response times of key business processes and transactions to determine if service-level agreements (SLAs) can be met [5]. h. URL zn=bto&cp=1-11-126-17^8_4000_100__ i. Primary Features j. Virtual User Generator The Virtual User Generator (VuGen) is a scripting tool. It is based on a Record and Playback rule. VuGen records a tester's actions into automated scripts (these scripts are the base of the load tests) as we walk through a business process on the application [6]. k. Controller The Scripts which were generated by VuGen are run via Controller. Controller lets you organize, manage and monitor the load test [6]. viii. Load Generators Load generators are the computers used to create load on the application under test by running virtual users (Vusers). You can use a number of load generator machines, each hosting many virtual users [6].
  6. 6. Final Project: Software Verification & Validation 4 ix. Analysis For evaluating the Performance and to create graphs and reports we can use analysis. Bottlenecks in the application are easily recognized and isolated and this helps us in determining what changes are needed in order to improve its performance [6]. x. Launcher The Launcher offers a single point of access for all of the LoadRunner components. l. Software Validation (Performance Validation) LoadRunner helps in validating non-functional requirements like quality attribute. How the system should scale and behave under certain load conditions is defined in part of the business requirement analysis phase. LoadRunner helps you prevent costly performance problems in production by detecting bottlenecks before you deploy a new system or upgrade [7]. With performance and load testing, a system would behave as expected as soon as it is out of its alpha testing phase into the beta testing phase and beyond. m.Software Phases LoadRunner is used in the testing phase of the waterfall model. However, performance testing can be broadly described as having four phases by itself— Design, Build, Execute and Analyze/Diagnose/Tune [7]. How these phases are actually divided over the waterfall model is specific to each company. All the following phases could either be carried out in the testing phase or being as soon as the implementation phase starts. The Design phase involves defining the business processes to be tested, the business process mix of an average or peak production hour, and the overall user and response-time goals for the system [7]. The Build phase involves setting up and configuring the test system and infrastructure, and using the automated performance testing solution to build both test scripts and load scenarios [7]. The Execute phase consists of running the load scenarios and measuring system performance [7]. The Analyze, Diagnose and Tune iterative phases go beyond measuring system performance and take load testing to another level. Here, the focus is on pinpointing problems to help resolve them rapidly and tuning system parameters to enhance performance in an ongoing fashion [7]. n. Author’s Comments LoadRunner has been used by almost every company that I have ever worked for. The reason is simple, it does one thing, and it does it well! Every piece of software that has passed the LoadRunner test according to the performance requirements
  7. 7. Final Project: Software Verification & Validation 5 acceptance criteria has in my experience never failed in production. In my current work place we have very robust web services which receive anywhere from 5-10 million requests a day. We used LoadRunner to test those web services and LoadRunner helped us scale them and hence, a successful deployment with minimal to no problems.
  8. 8. Final Project: Software Verification & Validation 6 4.References [1] Christina Wallin, quot;Verification and Validation of Software Components and Component Based Software Systemsquot;, ABB Corporate Research. [2] CMU/SEI-2000-TR-028. CMMISM for Systems Engineering/Software Engineering, Version 1.02, Software Engineering Institute, November 2000. [3] Ian Sommerville, Software Engineering 6th Edition, ISBN 0-201-39815-X, Addison-Wesley, 2001 [4] Bill Hamilton, “NUnit – Pocket Reference”, O’Reilly [5] zn=bto&cp=1-11-126-17^8_4000_100__ [6] [7] HP Automated Solutions, “Automate performance testing to predict system behavior and improve application performance”, Hewlett Packard.