Testing concepts prp_ver_1[1].0


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Some of you are from IT background and some of you are from Non-IT, however there is no difference in the work that you will be doing once you are assigned to a project.
  • What is a software product? A software product is made up of software packages that are separately installable. Ex. Word, online sales application, What are the activities involved when you think about developing a project? Marketing Survey is done by marketing people in the market for various product and benchmark the product. MRS( Marketing requirement survey) is created at the end of the survey. Software Requirement Analysis This is also known as feasibility study. In this phase, the development team visits the customer and studies their system. For a few projects, technical feasibility is a significant concern: Is it technically feasible to build a Star Wars missile defense system? Is it technically feasible to build a natural language English-to-French translator? For most projects, however, feasibility depends on non-technical issues: Are the project’s cost and schedule assumptions realistic? Does the project have an effective executive sponsor? Does the company have a business case for the software when the real cost—rather than the initial blue-sky, wishful-thinking cost—is considered? Ex- Contractor A and B made flyover at Domlur but they did not exchange/discuss their blueprints hence there is a gap of around 6 mtrs. By the end of the feasibility study, the team furnishes a document that holds the different specific recommendations for the candidate system. It also includes the personnel assignments, costs, project schedule, and target dates. The requirements gathering process is intensified and focussed specially on software. To understand the nature of the program(s) to be built, the system engineer ("analyst") must understand the information domain for the software, as well as required function, behavior, performance and interfacing. The essential purpose of this phase is to find the need and to define the problem that needs to be solved . System Analysis and Design In this phase, the software development process, the software's overall structure is defined. In terms of the client/server technology, the number of tiers needed for the package architecture, the database design, the data structure design etc are all defined in this phase. A software development model is created. Analysis and Design are very crucial in the whole development cycle. Any gap in the design phase could be very expensive to solve in the later stage of the software development. Much care is taken during this phase. The logical system of the product is developed in this phase.
  • Code Generation The design must be translated into a machine-readable form. The code generation step performs this task. If the design is performed in a detailed manner, code generation can be accomplished without much complication. Programming tools like Compilers, Interpreters, Debuggers are used to generate the code. Different high level programming languages like C, C++, Pascal, Java are used for coding. With respect to the type of application, the right programming language is chosen. Formal reviews are conducted after every 500 lines of code e.g. Code Inspection and code walkthrough What happen if we send this software directly to production? What all issues we may face? If we send the software directly to production without thorough testing, system may fail at the initial stage itself, and even if it doesn’t there may be some functionality which was not even thought of and tested. e.g. in online shopping system add products to cart and one page can hold description of ten carts only and then next page appears with additional product added to cart, but it was never tested, every time cart with less than 10 products was tested, hence testing of more than 10 products in a cart was skipped and leads to bad impression on customer. Will the customer again want to visit the site?? Off course NO What else a customer expect from any application other than perfect functionality? Look and Feel:- If the application is very dull, not very well presentable then also customer would not like to visit that site again. user-friendly:- If the application is not user friendly, instead of common language understood by people some other language is used or it is not easy to browse it, searching is not convenient etc then even though functionality of the application is perfect still this application would be a failure as the end- user is not at all satisfied. Testing Once the code is generated, the software program testing begins. Different testing methodologies are available to unravel the bugs that were committed during the previous phases. Different testing tools and methodologies are already available. Some companies build their own testing tools that are tailor made for their own development operations. Maintenance Software will definitely undergo change once it is delivered to the customer. There are many reasons for the change. Change could happen because of some unexpected input values into the system. In addition, the changes in the system could directly affect the software operations. The software should be developed to accommodate changes that could happen during the post implementation period.
  • Requirement analysis:- As we were talking about feasibility study, which also includes working in team, work cohesively with other people engaged in the similar or other activities. Eg. The flyover that is being constructed at Domlur airport. Contract was given to 2 contractors, they started constructing flyover without exchanging their blue prints, result was that once they finished their part of flyover there was a gap of almost 6 mtrs in height. Had they have exchanged their blueprints they would have constructed it much earlier and without any disruption and disturbance and loss of money and effort. Now they have to do some workaround as it has to be completed somehow… Why these situations occur??? Majorly due to lack of proper testing. Overconfidence Even the software developed by best of the developers may fail. It is not only the system that is being developed is tested but the system with which our product is interfaced that is also tested for interfacing not the product as a whole.
  • As per IEEE Standards, SDLC broadly consist of 3 phases:- Definition Development Maintainence
  • As we have talked about that we cannot launch any product successfully without testing. Even the product designed by best of the best developers needs to be tested and verified under various circumstances. The U.S. space shuttle Columbia with a seven-member crew, disintegrated in flames over central Texas shortly before it was scheduled to land at Cape Canaversal in Florida. On February 1, 2003, at an altitude of 63 kilometers and a velocity of 20,000 km/hr, Columbia disintegrated killing 7 astronauts. Columbia was designed and developed by most of the intelligent people employed in NASA. And the product was tested very well before its launch, still it failed leading to disaster and taking lives of 7 best astronauts that includes Kalpana Chawla from India. The product failed not because it was not tested well but it could have been tested more to avoid this disaster. Hence testing is very vital in all application/products however more aggressive where lives are involved for e.g medical equipments, space shuttles etc.
  • What Is Software Testing? Software testing is a process used to help identify the correctness, completeness and quality of developed computer software. With that in mind, testing can never completely establish the correctness of computer software. Only the process of formal verification can prove that there are no defects. What is the expected behavior of the system? Testing is to check if there is no difference between actual output and expected output. It is to put software under various test to check that it satisfies some requirements of end user/Client and hence detecting errors if it fails to satisfy a set of requirements.
  • Testing is to establish confidence that a system does what it is supposed to do and even if it “misbehaves” it should not crash, it should do it decently. It is to confirm that system performs its intended functions correctly, moreover testing does not guarantee bug free product and good testing cannot be a substitute for good programming ex. If I do a very poor programming assuming that if there are any bugs my testers would find it and I would fix them later. This idea would take more time plus more effort in debugging those many number of bugs and in all the cost would be much higher than it would have been if the code has been developed effectively. Testing is not a debugging or preventing process, it is only to ensure that within an application I have found/detect these many number of bugs and rest debugging/fixing of bugs is done by developers. It improves quality and identify risks as well.
  • As we were discussing about the need of software testing… hence we test our software to :- Detect programming errors:- Even though we have best developers developing the product still there may be flaws somewhere due to misconception, due to lack of understanding in integrating modules of 2 great developers. Most of the time developers think that their code is perfect, it cannot have bugs and that’s where the problem is. They think that finding bugs in their code means that some one is proving them inefficient. Hence we test code not to prove a developer inefficient but to find bug in code not in developer. Testing at earlier stage is much cheaper than testing at later stage. If Client find bugs which were uncovered earlier then we loses credibility and may lose business as well. There are several functionality related that occur in real life scenarios related only so while testing at customer site during beta testing we can uncover these defects there and then. To release a bug free product seems to be a dream yet to be fulfilled, hence it is a real challenge to release a product that is bug-free.
  • Testing is defined in a number of ways by various veterans. Of course, none of these definitions claims that testing shows that software is free from defects. Testing can show the presence, but not the absence of problems.
  • What is the criteria for a successful product? A product is successful only when it is adding any value to customer business, only if it increases its revenue, credibility in the market and so on. To develop a product we set a target time for completion and released to customer. If we can stick to that time without increasing budget and scope of the product then the product is successful. Using the developed product true business goals should be met else the product developed can be a failure leading to finance loss, customer satisfaction loss, and loss of business further from that client as well as associates. Look and feel should be accepted by user who will be finally using that product.
  • Product Success Criteria (PSC) so as to address the following. These criteria, once arrived at the initial stage of the project, shall be the basis for strategy and planning. Functionality: The business perceives that the system satisfactorily addresses the true business goals. The system is functionally compliant. The system was delivered on time and within budget and scope. Usability: The user perceives the system as value-added to his or her job and is easy to learn and use. Likeability: End user feels that look, feel, and navigation are easy. Configurability: All changes to project scope, schedule, and budget were handled under a well-defined and visible change control process. Maintainability: The operations staff is prepared to maintain and support the delivered system. Interoperability: The organizational interoperability groups perceive that the solution is consistent with interoperability and reusability standards and goals.
  • Testability and Product Success Criteria Testability is the possible extent to which the software product under consideration can be evaluated to determine- Operability: The better it works; the more efficiently it can be tested. Controllability: The better we can control it, the more the Software Testing can be automated & optimized Observability : What you see is what you test. Simplicity: The less there is to test, the more quickly we can test. Understandability: The more information we have, the smarter we will test. Suitability: The more we know about the intended use of the system, the better we can organize our Software Testing to find important bugs Stability: The fewer the changes, the fewer the disruptions to Software Testing Accessibility: Web site shall be accessible when incorporating frames, JavaScript, Cascading Style Sheets, Dynamic HTML, and multimedia technology such as QuickTime or Flash. Navigability: Site's architecture is firmly represented by the visually designed interface for ease of motion within the site. Editorial Continuity: Ensure that users can easily move from page to page in a multi-page section within your web site and still retain a sense of location. Scalability: Modular interface design structure to support major edits and visual design overhauls. Context Sensitivity: Interface reinforcing the content of the web site and enhances the overall brand presentation strategy. Structural Continuity: Ensuring that users are able to understand and utilize every component of the navigational system in order to move throughout the web site. Software Testing shall be aligned with software project and shall meet
  • Correctness To claim that software is correct, we have to assure that the data entered, processed, and outputted is accurate and complete. We can achieve this through controlling data element across the entire transaction. This requires that: Well defined Functional specifications. Design conformance with user requirements. Program conformance to design specifications. Testing to ensure requirements are properly implemented. Availability of right programs and data in the released system. Proper management change requests Reliability This will ensure that the system will perform its intended function with the required precision over an extended period of time. This shall ensure: Establishing the system in operational environment to meet the expected level of accuracy and completeness. Designing and implementing process tolerance through the data integrity controls. Performing manual, regression, and functional tests to ensure proper functioning of the data integrity controls. Verification of the installation for system accuracy and completeness. Maintaining accuracy of system requirements and also, system updation. Ease of Use This will address effort required to learn, operate, prepare input for, and interpret output from the system. This test factor deals with the how quickly and easily the people interfacing with the application system will learn to use it. The points addressed here are: Defining the usability specifications for the application system. Design to optimize the usability related requirements Conformance of programs to the design in order to optimize the ease of use. Testing to ensure that the application is easy to use. Proper presentation and communication of usability instructions to appropriate individuals. Preserving usability as and when the system is maintained. File integrity This ensures appropriate storage and maintenance of data. This requires that: Defined file integrity requirements. Controls in design that ensure the integrity of the file. Proper implementation of specified file integrity controls. Proper testing to ensure proper functioning of the file integrity. Verification of file integrity to ensure delivery of right riles prior to system release. Preserving integrity of the files during the maintenance phase. Maintainability This factor addresses effort required to locate and fix an error in an operational system so that system operates smoothly. This addresses: Specification of the desired level of maintainability of the system. Development of design and program to achieve the desired level of maintainability. Inspection of system to ensure that it is maintainable. Verifying system documentation for completeness. Maintainability is preserved as the system is updated.
  • 02/09/12 The development process must be iterative-- let’s see how that works The traditional development process involves a long building phase after you’ve specifiedd the application. After the app is built, you move to a testing phase where you find bugs and fix them. Many people view testing as a “washing machine” that you clean your code with at the end of the development phase.
  • As we have already discussed Software Development V- model, Let us compare it with v-Model of software testing life cycle.
  • The responsibility for testing Unit Test is the responsibility of the Development Team System Testing is the responsibility of SQA User Acceptance Testing is the Responsibility of the User Representatives Team Technology Compliance Testing is the responsibility of the Systems Installation & Support Group.
  • During acceptance lets see what do we mean by hot-Fixes. In acceptance phase product is installed and executed in users environment. There may be some failure to software might be due to some configuration files mismatch or some other interface. When Bill Gates was giving Live demo for Microsoft windows new version suddenly a Unwanted popup appears and whole world was watching this demo, Bill did some changes to config files and it worked fine. This is termed as HOT_FIX( to make small changes to software to continue testing activity)
  • Role of test engineer is to :-
  • 02/09/12 module interfaces are tested to ensure that the information properly flows into and out of the program unit under test. The local data structure is examined to ensure that data stored temporarily maintains its integrity during all steps in an algorithm’s execution. Boundary conditions are executed to ensure that the module operates properly at boundaries established to limit or restrict processing. All independent paths (basis paths) through the control structure are exercised to ensure that all statements in a module have been executed at least once All error handling paths are tested.
  • 02/09/12 Step1. Unit test criteria is a narrative describing what will be done to verify that the program code operates properly and according to specifications. The unit test plan is doc that defines the process for testing the code, including descriptions and procedures for all tests and evaluations. These above items and test specifications are prepared before the program or module is coded. Step2. Code walk through is done to review to the requirements, design, and coding before the testing to ensure compliance to the IS standards. Step3. Unit test data are specific values or transactions that are created for the purpose of testing the code.Test data should be meticulously prepared so that a full range of data values and combinations are considered, and all instruction paths within the module are executed. The unit test report records the results and findings of the test and provides the basis for determining if the unit is ready for system testing.
  • 02/09/12 Select test data to exercise each aspect of functionality identified in the program’s requirements specifications Functional Testing of word processor calls for using each required feature of of the program at least once. Such features include editing, formatting, search etc
  • 02/09/12 Tests module against functional and non-functional specifications Specification used to derive test cases Do not look at module code (except to debug) Attempt to force behavior that doesn't match specification Problem – how to select inputs that have a high probability of causing error
  • 02/09/12 regression testing aims to assure that the software continues to perform according to specification after it has been modified.
  • 02/09/12 Definitions ‘ Regression testing is the process of testing changes to computer programs to make sure that the older programming still works with the new changes.’ ‘ A technique for ensuring that all aspects of a software system operate as required after enhancement or repair.’
  • 02/09/12 Manual Testing: : Here, the developer/quality assurance team manually tests each and every path that the software code can take and then compares the results by ‘eyeballing’. This approach takes a lot of time and a lot of manpower. It is successful most of the times, but not efficient enough. Hence the need for automating the whole process...
  • 02/09/12
  • 02/09/12
  • 02/09/12 Why? -- If no errors are found during testing, the project team did not test the system sufficiently. Exhaustive testing is impractical, so the Project team must design and plan a testing strategy that utilizes a balance of testing techniques to cover a representative sample of the system. The test planning process is a critical step in the testing process. Without a documented test plan, the test itself cannot be verified, coverage cannot be analyzed, and the test is not repeatable Repeatable : Once the tests are documented, any member of the test team should be able to execute the tests. If the test must be executed multiple times, the plan ensures that all of the critical elements are tested correctly. Parts or the entire plan can be executed for any necessary regression testing.   Controllable : Knowing what test data is required and what the expected results are.   Coverage : Based on the risks and priorities associated with the parts of the system, the test plan is designed to insure that adequate test coverage is build into the test. The plan can be reviewed by the project team to insure that all are in agreement that the correct amount and types of tests are planned.
  • 02/09/12 When? – Should begin at the same time requirements definition starts. The plan can then be detailed in parallel with application requirements.
  • Test plan identifier:- Shall contain full identification of the system and the software to which this document applies, including, as applicable, identification numbers(s), title(s), abbreviation(s), version number(s) and release number(s). Introduction:- 1. System Overview Shall briefly state the purpose and nature of the system and the software to which this document applies. Summarize the history of system development, operation, and maintenance. Identify the project sponsor, acquirer, user, developer, and support agencies, current and planned operating sites and list other relevant documents. 2. Document overview: Shall summarize the purpose and contents of this document and shall describe any security or privacy considerations associated with its use. 3. Relationship to other plans: Shall describe the relationship, if any, of the Software Test Plan to related project management plans. Test items/Integrated components:- Shall identify a Unit, subsystem, system or other entity by name and project-unique identifier and shall be divided into the following subparagraphs to describe the testing planned for the item(s). 1.Project-unique identifier of a test: Shall identify a test by project-unique identifier and shall provide the information specified below for the test. 1. Test objective 2. Test level 3. Test type or class 4. Qualification method(s) as specified in the requirements specification 5. Identifier for the System requirement and, if applicable software system requirements addressed by this test. 6. Special requirements (for example, 48 hours of continuous facility time, weapon simulation, extent of test, use of a special input or database) 7. Type of data to be recorded 8. Type of data recording/reduction/analysis to be employed 9. Assumptions and constraints, such as anticipated limitations on the test due to system or test conditions- timings, interfaces, equipment personnel, database etc. 10. Safety, security and privacy considerations associated with the test Features to be tested:- It list all the feature of application that needs to be tested. Features not to be tested:- It list all the features of application that are not to be tested.
  • 02/09/12 Focus is on individual tests and small groups of related tests.
  • 02/09/12
  • 02/09/12
  • 02/09/12
  • 02/09/12
  • 02/09/12
  • 02/09/12
  • 02/09/12
  • Bug Triage Meetings (sometimes called Bug Councils) are project meetings in which open bugs are divided into categories. The most important distinction is between bugs that will not be fixed in this release and those that will be. Triaging a bug involves: Making sure the bug has enough information for the developers and makes sense Making sure the bug is filed in the correct place Making sure the bug has sensible "Severity" and "Priority" fields Let us see what Priority and Severity means Priority is Business; Severity is Technical In Triages, team will give the Priority of the fix based on the business perspective. They will check “How important is it to the business that we fix the bug?” In most of the times high Severity bug is becomes high Priority bug, but it is not always. There are some cases where high Severity bugs will be low Priority and low Severity bugs will be high Priority. In most of the projects I worked, if schedule drawn closer to the release, even if the bug severity is more based on technical perspective, the Priority is given as low because the functionality mentioned in the bug is not critical to business. Priority and Severity gives the excellent metrics to identify overall health of the Project. Severity is customer-focused while priority is business-focused. Assigning Severity for a bug is straightforward. Using some general guidelines about the project, testers will assign Severity but while assigning a priority is much more juggling act. Severity of the bug is one of the factors for assigning priority for a bug. Other considerations are might be how much time left for schedule, possibly ‘who is available for fix’, how important is it to the business to fix the bug, what is the impact of the bug, what are the probability of occurrence and degree of side effects are to be considered.
  • Testing concepts prp_ver_1[1].0

    1. 1. Software Testing Concept and Methodologies
    2. 2. Fundamentals of Software Testing
    3. 3. <ul><li>Is there any difference when you work for same assignment ??? </li></ul>NON-IT? IT
    4. 4. Software/Application?? <ul><li>Group of programs designed for end user using operating system and system utilities. </li></ul><ul><li>A self contained program that performs well- defined set of tasks under user control. </li></ul><ul><li>Programs, procedures, rules, and any associated documentation pertaining to the operation of a system. </li></ul>
    5. 5. Evolution of software product <ul><li>Marketing </li></ul><ul><ul><li>Survey is done by marketing people for various product and benchmark the products. </li></ul></ul><ul><ul><li>Create MRS( Marketing Requirement survey) </li></ul></ul><ul><li>Requirement analysis </li></ul><ul><ul><li>Feasibility study (social, economical etc) </li></ul></ul><ul><ul><li>investigate the need for possible software automation in the given system. </li></ul></ul><ul><ul><li>Domain expert Create URS ( User Requirement specification) </li></ul></ul><ul><li>Design </li></ul><ul><ul><li>software's overall structure is defined. </li></ul></ul><ul><ul><li>Software architecture, interdependence of modules, interfaces, database etc is defined. </li></ul></ul><ul><ul><li>System analyst Create SRS( High level design, Low level design etc) </li></ul></ul>
    6. 6. Evolution of software product( Cont.) <ul><li>Code Generation </li></ul><ul><ul><li>design must be translated into a machine-readable form taking input as SRS. </li></ul></ul><ul><ul><li>Done by Team of developers. </li></ul></ul><ul><ul><li>Reviews after every 500 lines of code </li></ul></ul><ul><ul><ul><li>Code Inspection </li></ul></ul></ul><ul><ul><ul><li>Code Walkthrough </li></ul></ul></ul><ul><li>Testing </li></ul><ul><ul><li>New/patched build is tested by Test engineers for stability of the application. </li></ul></ul><ul><li>Maintainence </li></ul><ul><ul><li>Software is maintained due to changes ( unexpected values into the system) </li></ul></ul>
    7. 7. Software development life cycle Operational Testing Ongoing Support Requirement Analysis High level design Detailed Specifications Coding Unit Testing Integration Testing Review/Test
    8. 8. System <ul><li>An inter-related set of components, with identifiable boundaries, working together for some purpose. </li></ul>Output Input Process
    9. 9. Analysis <ul><li>The process of identifying requirements, current problems, constraints ,Opportunities for improvement , timelines and Resources costs . </li></ul>
    10. 10. Design <ul><li>The business of finding a way to meet the functional requirements within the specified constraints using the available technology </li></ul>
    11. 11. Software Development life cycle Phases or stages of a project from inception through completion and delivery of the final product… … and maintenance too!
    12. 12. Software Development life cycle Three Identifiable Phases: 1. Definition 2. Development 3. Maintenance
    13. 13. Definition Phase <ul><li>Focuses on WHAT </li></ul><ul><ul><li>What information to be processed? </li></ul></ul><ul><ul><li>What functions and performances are desired? </li></ul></ul><ul><ul><li>What interfaces are to be established? </li></ul></ul><ul><ul><li>What design constraints exists? </li></ul></ul><ul><ul><li>What validation criteria are required to define a success system? </li></ul></ul>
    14. 14. Development Phase <ul><li>Focuses on </li></ul><ul><ul><li>How the database should be designed ? </li></ul></ul><ul><ul><li>How the software architecture to be designed ? </li></ul></ul><ul><ul><li>How the design will be translated in to a code ? </li></ul></ul><ul><ul><li>How testing will be performed ? </li></ul></ul><ul><li>Three specific steps in Development Phase are:- </li></ul><ul><ul><li>a. Design </li></ul></ul><ul><ul><li>b. Coding </li></ul></ul><ul><ul><li>c. Testing (ignored due to lack of time, due time to market, additional cost involved, lack of testing requirement understanding etc.) ) </li></ul></ul>
    15. 15. Maintenance Phase <ul><li>Maintenance phase focuses on CHANGE that is associated with </li></ul><ul><li>Error correction </li></ul><ul><li>Adaptation required as the software environment evolves </li></ul><ul><li>Enhancements brought about by changing customer requirements </li></ul><ul><li>Reengineering carried out for performance improvements </li></ul>“ Maintainability is defined as the ease with which software can be understood, corrected, adapted and enhanced”
    16. 16. <ul><li>Identify Problems/Objectives </li></ul><ul><li>Determine information Requirements </li></ul><ul><li>Analyze System needs </li></ul><ul><li>Design the recommended system </li></ul><ul><li>Develop and Document software </li></ul><ul><li>Testing the System </li></ul><ul><li>Implementation and maintaining the system </li></ul>SDLC Phases S D L C
    17. 17. SDLC Phases : Requirement Identification & Analysis Phase <ul><li>Request for Proposal </li></ul><ul><li>Proposal </li></ul><ul><li>Negotiation </li></ul><ul><li>Contract </li></ul><ul><li>User Requirement Specification </li></ul><ul><li>Software Requirement Specification </li></ul>
    18. 18. Software Requirement Specifications IEEE 830 : Software Requirement Specification is a means of translating the ideas in the minds of the clients(the inputs) into a set of formal document (the output) of the requirement phase The Role Bridge the communication gap between the client the user and the developer
    19. 19. SDLC Phases- Design <ul><li>HLD Document contains items in a macro level </li></ul><ul><ul><li>List of modules and a brief description of each </li></ul></ul><ul><ul><li>Brief functionality of each module </li></ul></ul><ul><ul><li>Interface relationship among modules </li></ul></ul><ul><ul><li>Dependencies between modules </li></ul></ul><ul><ul><li>Database tables identified with key elements </li></ul></ul><ul><ul><li>Overall architecture diagrams along with technology details </li></ul></ul>High Level Design
    20. 20. SDLC Phases- Design <ul><li>Detailed functional logic of the module, in pseudo code </li></ul><ul><li>Database tables, with all elements, including their type and size </li></ul><ul><li>All interface details </li></ul><ul><li>All dependency issues </li></ul><ul><li>Error MSG listing </li></ul><ul><li>Complete input and output format of a module </li></ul>Low Level Design HLD and LLD phases put together called Design phase
    21. 21. SDLC Phases <ul><li>Code Generation </li></ul><ul><ul><li>design must be translated into a machine-readable form taking input as SRS. </li></ul></ul><ul><ul><li>Done by Team of developers. </li></ul></ul><ul><ul><li>Reviews after every 500 lines of code </li></ul></ul><ul><ul><ul><li>Code Inspection </li></ul></ul></ul><ul><ul><ul><li>Code Walkthrough </li></ul></ul></ul><ul><li>Testing </li></ul><ul><ul><li>New/patched build is tested by Test engineers for stability of the application. </li></ul></ul><ul><li>Maintainence </li></ul><ul><ul><li>Software is maintained due to changes ( unexpected values into the system) </li></ul></ul>
    22. 22. <ul><li>What is testing? </li></ul><ul><li>We Test !! We Test !! Why? </li></ul><ul><li>Testing Defined </li></ul><ul><li>Is Product Successful </li></ul><ul><li>Product Success criteria </li></ul><ul><li>Testability </li></ul><ul><li>Test factors </li></ul>Get Started with Testing !!!!!!!
    23. 23. What is Testing? <ul><li>process used to help identify the correctness, completeness and quality of developed computer software. </li></ul><ul><li>Find out difference between actual and expected behavior. </li></ul><ul><li>The process of exercising software to verify that it satisfies specified requirements of end user and to detect errors </li></ul><ul><li>The process of revealing that an artifact fails to satisfy a set of requirements </li></ul>
    24. 24. What is Testing ( Cont.) ? <ul><li>Establishing confidence that a system does what it is supposed to do </li></ul><ul><li>Confirming that a system performs its intended functions correctly </li></ul><ul><li>Does not guarantee bug free product </li></ul><ul><li>No substitute for good programming </li></ul><ul><li>Can’t prevent/debug bugs, only detect </li></ul><ul><li>Offer advise on product quality and risks. </li></ul>
    25. 25. We Test !! We Test !! Why <ul><li>Detect programming errors - programmers, like anyone else, can make mistakes. </li></ul><ul><li>To catch bugs/defect/errors. </li></ul><ul><li>To check program against specifications </li></ul><ul><li>Cost of debugging is higher after release </li></ul><ul><li>Client/end user should not find bugs </li></ul><ul><li>Some bugs are easier to find in testing </li></ul><ul><li>Challenge to release a bug-free product. </li></ul><ul><li>Verifying Documentation. </li></ul>
    26. 26. <ul><li>To get adequate trust and confidence on the product. </li></ul><ul><li>To meet organizational goals </li></ul><ul><ul><li>Like meeting requirements, satisfied customers, improved market share, Zero Defects etc </li></ul></ul><ul><li>Since the software can perform 100000 correct operations per second, it has the same ability to perform 100000 wrong operations per second, if not tested properly. </li></ul><ul><li>Ensuring that system is ready for use </li></ul><ul><li>Understanding limits of performance. </li></ul><ul><li>Learning what a system is not able to do </li></ul><ul><li>Evaluating capabilities of system </li></ul>We Test !! We Test !! Why?
    27. 27. Testing defined !! <ul><li>Def-1 </li></ul><ul><ul><li>Process of establishing confidence that a program or system does what it is supposed to. </li></ul></ul><ul><li>Def-2 </li></ul><ul><ul><li>Process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirement (IEEE 83a) </li></ul></ul><ul><li>Def-3 </li></ul><ul><ul><li>Testing is a process of executing a program with the intent of finding errors (Myers) </li></ul></ul><ul><li>Def-4 </li></ul><ul><ul><li>Testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. </li></ul></ul>
    28. 28. Is Product successful ??? <ul><li>When Client/Customer perceives it as value-added to his business. </li></ul><ul><li>Timeliness of delivery of the product within budget and scope. </li></ul><ul><li>The business perceives that the system satisfactorily addresses the true business goals. </li></ul><ul><li>End user feels that look, feel, and navigation are easy. </li></ul><ul><li>Team is prepared to support and maintain the delivered product. </li></ul>
    29. 29. Product Success Criteria <ul><li>Functionality </li></ul><ul><li>Usability </li></ul><ul><li>Likeability </li></ul><ul><li>Configurability </li></ul><ul><li>Maintainability </li></ul><ul><li>Interoperability </li></ul>
    30. 30. Testability <ul><li>Operability </li></ul><ul><li>Controllability </li></ul><ul><li>Observability </li></ul><ul><li>Understandability </li></ul><ul><li>Suitability </li></ul><ul><li>Stability </li></ul><ul><li>Accessibility </li></ul><ul><li>Navigability </li></ul><ul><li>Editorial Continuity </li></ul><ul><li>Scalability </li></ul><ul><li>Context Sensitivity </li></ul><ul><li>Structural Continuity </li></ul>
    31. 31. Test Factors     Structure Integrity Maintainability Documentation Usability Reusability Testability Reliability Flexibility Efficiency Correctness Adaptability (future quality) Engineering (interior quality) Functionality (exterior quality)
    32. 32. Software Testing Life Cycle
    33. 33. Conventional Testing Process Design Build Test & Fix * Here testing was happening only towards the end of the life cycle Spec
    34. 34. Distribution of Defects in the life cycle Source: IBM/TRW/Mitre 27% 7% 10% 56%
    35. 35. Software development life cycle Operational Testing Ongoing Support Requirement Analysis High level design Detailed Specifications Coding Unit Testing Integration Testing Review/Test
    36. 36. STLC-V Model Requirement Req. Review Design Design Review Code Code Review Develop Unit Test Review Unit Test Execute Unit Test Execute Integration tests Execute System Tests Develop integration Tests Review Integration Tests Develop Acceptance Tests Review Acceptance tests
    37. 37. STLC:- Activities <ul><li>Scope/Requirement </li></ul><ul><li>Base line inventory </li></ul><ul><li>Acceptance criteria </li></ul><ul><li>Schedule </li></ul><ul><li>Prioritization </li></ul><ul><li>Test references </li></ul><ul><li>Sign off req </li></ul><ul><li>Plan </li></ul><ul><li>Approach </li></ul><ul><li>Process and Tools </li></ul><ul><li>Methodology </li></ul><ul><li>Delivery Models </li></ul><ul><li>Risk Plan </li></ul><ul><li>Project Overflow </li></ul><ul><li>Quality Objectives </li></ul><ul><li>Configuration Plan </li></ul><ul><li>Design </li></ul><ul><li>Test Design </li></ul><ul><li>Specifications </li></ul><ul><li>Test Scenarios </li></ul><ul><li>Test Cases </li></ul><ul><li>Test Data </li></ul><ul><li>Tool Development </li></ul>
    38. 38. STLC:- Activities <ul><li>Execution </li></ul><ul><li>Implement Stubs </li></ul><ul><li>Test Data Feeders </li></ul><ul><li>Batch Processes </li></ul><ul><li>Execute Testing </li></ul><ul><li>Collate Test Data </li></ul><ul><li>Identify Bugs </li></ul><ul><li>Defect Analysis </li></ul><ul><li>Check Unexpected Behavior </li></ul><ul><li>Identify defective application areas </li></ul><ul><li>Identify erroneous test data </li></ul><ul><li>Identify defect trends/patterns </li></ul>
    39. 39. Test Approach <ul><li>Test Process :- The project under development or incorporation of accepted changes in the project or project under maintenance which implemented changes, use the testing process. Based on the nature of the project, adequate testing shall be arrived at the project level. </li></ul><ul><li>The Test Approach </li></ul><ul><ul><li>sets the scope of system testing </li></ul></ul><ul><ul><li>the overall strategy to be adopted </li></ul></ul><ul><ul><li>the activities to be completed </li></ul></ul><ul><ul><li>the general resources required </li></ul></ul><ul><ul><li>the methods and processes to be used to test the release. </li></ul></ul><ul><ul><li>details the activities, dependencies and effort required to conduct the System Test. </li></ul></ul>
    40. 40. Test Approach( Cont.) <ul><li>Test approach will be based on the objectives set for testing </li></ul><ul><li>Test approach will detail the way the testing to be carried out </li></ul><ul><li>Types of testing to be done viz Unit, Integration and system testing </li></ul><ul><li>the general resources required </li></ul><ul><li>The method of testing viz Black–box, White-box etc., </li></ul><ul><li>Details of any automated testing to be done </li></ul><ul><li>Details the activities, dependencies and effort required to conduct the System Test </li></ul>
    41. 41. Software Testing Life Cycle- Phases <ul><li>Requirement Analysis </li></ul><ul><li>Prepare Test Plan </li></ul><ul><li>Test Case Designing </li></ul><ul><li>Test Case Execution </li></ul><ul><li>Bug Reporting, Analysis and Regression testing </li></ul><ul><li>Inspection and release </li></ul><ul><li>Client acceptance and support during acceptance </li></ul><ul><li>Test Summary analysis </li></ul>
    42. 42. Requirement Analysis <ul><li>Objective </li></ul><ul><ul><li>The objective of Requirement Analysis is to ensure software quality by eradicating errors as earlier as possible in the developement process, as the errors noticed at the end of </li></ul></ul><ul><ul><li>the software life cycle are more costly compared to that of </li></ul></ul><ul><ul><li>early ones, and there by validating each of the Outputs. </li></ul></ul><ul><li>The objective can be acheived by three basic issues: </li></ul><ul><ul><ul><li>Correctness </li></ul></ul></ul><ul><ul><ul><li>Completeness </li></ul></ul></ul><ul><ul><ul><li>Consistency </li></ul></ul></ul>
    43. 43. Type of Requirement <ul><li>Functional </li></ul><ul><li>Data </li></ul><ul><li>Look and Feel </li></ul><ul><li>Usability </li></ul><ul><li>Performance </li></ul><ul><li>Operational </li></ul><ul><li>Maintainability </li></ul><ul><li>Security </li></ul><ul><li>Scalability </li></ul><ul><li>Etc……. </li></ul>
    44. 44. Evaluating Requirements <ul><li>What Constitutes a good Requirement? </li></ul><ul><li>Clear:- </li></ul><ul><ul><li>Unambiguous terminology </li></ul></ul><ul><li>Concise:- </li></ul><ul><ul><li>no unnecessary narrative or non-relevant facts </li></ul></ul><ul><li>Consistent </li></ul><ul><ul><li>requirements that are similar are stated in similar terms. Requirements do not conflict with each other. </li></ul></ul><ul><li>Complete </li></ul><ul><ul><li>all functionality needed to satisfy the goals of the system is specified to a level of detail sufficient for design to take place. </li></ul></ul>
    45. 45. Requirement Analysis <ul><li>Difficulties in conducting requirement analysis </li></ul><ul><li>Analyst not prepared </li></ul><ul><li>Customer has no time/interest </li></ul><ul><li>Incorrect customer personnel involved </li></ul><ul><li>Insufficient time allotted in project schedule </li></ul>
    46. 46. Prepare Test Plan- Activities <ul><li>Scope Analysis of project </li></ul><ul><li>Document product purpose/definition </li></ul><ul><li>Prepare product requirement document </li></ul><ul><li>Develop risk assessment criteria </li></ul><ul><li>Identify acceptance criteria </li></ul><ul><li>Document Testing Strategies. </li></ul><ul><li>Define problem - reporting procedures </li></ul><ul><li>Prepare Master Test Plan </li></ul>
    47. 47. Design-Activities <ul><li>Setup test environment </li></ul><ul><li>Design Test Cases: Requirements-based and Code-based Test Cases </li></ul><ul><li>Analyze if automation of any test cases is needed </li></ul>
    48. 48. Execution- Activities <ul><li>Initial Testing, Detect and log Bugs </li></ul><ul><li>Retesting after bug fixes </li></ul><ul><li>Final Testing </li></ul><ul><li>Implementation </li></ul><ul><li>Setup database to track components of the automated testing system, i.e. reusable modules </li></ul>
    49. 49. Bug Reporting, Analysis, and Regressing Testing <ul><li>Activities </li></ul><ul><li>Detect Bugs by executing test cases </li></ul><ul><li>Bug Reporting </li></ul><ul><li>Analyze the Error/Defect/Bug </li></ul><ul><li>Debugging the system </li></ul><ul><li>Regression testing </li></ul>
    50. 50. Inspection and Release-Activities <ul><li>Maintaining configuration of related work products </li></ul><ul><li>Final Review of Testing </li></ul><ul><li>Metrics to measure improvement </li></ul><ul><li>Replication of Product </li></ul><ul><li>Product Delivery Records </li></ul><ul><li>Evaluate Test Effectiveness </li></ul>
    51. 51. Client Acceptance <ul><li>Software Installation </li></ul><ul><li>Provide Support during Acceptance Testing </li></ul><ul><li>Analyze and Address the Error/Defect/Bug </li></ul><ul><li>Track Changes and Maintenance </li></ul><ul><li>Final Testing and Implementation </li></ul><ul><li>Submission, client Sign-off </li></ul><ul><li>Update respective Process </li></ul>
    52. 52. Support during Acceptance-Activities <ul><li>Pre-Acceptance Test Support </li></ul><ul><li>Installing the software on the client’s environment </li></ul><ul><li>Providing training for using the software or maintaining the software </li></ul><ul><li>Providing hot-fixes as and when required to make testing activity to continue </li></ul><ul><li>Post Acceptance Test Support </li></ul><ul><li>Bug Fixing </li></ul>
    53. 53. Test Summary Analysis- Requirement <ul><li>Quantitative measurement and Analysis of Test Summary </li></ul><ul><li>Evaluate Test Effectiveness </li></ul><ul><li>Test Reporting </li></ul><ul><ul><li>Report Faults – (off-site testing) </li></ul></ul><ul><ul><li>Report Faults – (on-site/ field testing) </li></ul></ul>
    54. 54. Testing Life Cycle - Team Structure <ul><li>An effective testing team includes a mixture of members who has </li></ul><ul><ul><li>Testing expertise </li></ul></ul><ul><ul><li>Tools expertise </li></ul></ul><ul><ul><li>Database expertise </li></ul></ul><ul><ul><li>Domain/Technology expertise </li></ul></ul>
    55. 55. Testing Life Cycle - Team Structure (Contd…) <ul><li>The testing team must be properly structured, with defined roles and responsibilities that allow the testers to perform their functions with minimal overlap. </li></ul><ul><li>There should not be any uncertainty regarding which team member should perform which duties. </li></ul><ul><li>The test manager will be facilitating any resources required for the testing team. </li></ul>
    56. 56. Testing Life Cycle - Roles & Responsibilities <ul><li>Clear Communication protocol should be defined with in the testing team to ensure proper understanding of roles and responsibilities. </li></ul><ul><li>The roles chart should contain both on-site and off-shore team members. </li></ul>
    57. 57. Testing Life Cycle - Roles & Responsibilities <ul><li>Test Manager </li></ul><ul><ul><li>Single point contact between Wipro onsite and offshore team </li></ul></ul><ul><ul><li>Prepare the project plan </li></ul></ul><ul><ul><li>Test Management </li></ul></ul><ul><ul><li>Test Planning </li></ul></ul><ul><ul><li>Interact with Wipro onsite lead, Client QA manager </li></ul></ul><ul><ul><li>Team management </li></ul></ul><ul><ul><li>Work allocation to the team </li></ul></ul><ul><ul><li>Test coverage analysis </li></ul></ul>
    58. 58. Testing Life Cycle - Roles & Responsibilities <ul><li>Test Manager cont.. </li></ul><ul><ul><li>Co-ordination with onsite for issue resolution. </li></ul></ul><ul><ul><li>Monitoring the deliverables </li></ul></ul><ul><ul><li>Verify readiness of the product for release through release review </li></ul></ul><ul><ul><li>Obtain customer acceptance on the deliverables </li></ul></ul><ul><ul><li>Performing risk analysis when required </li></ul></ul><ul><ul><li>Reviews and status reporting </li></ul></ul><ul><ul><li>Authorize intermediate deliverables and patch releases to customer. </li></ul></ul>
    59. 59. Testing Life Cycle - Roles & Responsibilities <ul><li>Test Lead </li></ul><ul><ul><li>Resolves technical issues for the product group </li></ul></ul><ul><ul><li>Provides direction to the team members </li></ul></ul><ul><ul><li>Performs activities for the respective product group </li></ul></ul><ul><ul><li>Review and Approve of Test Plan / Test cases </li></ul></ul><ul><ul><li>Review Test Script / Code </li></ul></ul><ul><ul><li>Approve completion of Integration testing </li></ul></ul><ul><ul><li>Conduct System / Regression tests </li></ul></ul><ul><ul><li>Ensure tests are conducted as per plan </li></ul></ul><ul><ul><li>Reports status to the Offshore Test Manager </li></ul></ul>
    60. 60. Testing Life Cycle - Roles & Responsibilities <ul><li>Test Engineer </li></ul><ul><ul><li>Development of Test cases and Scripts </li></ul></ul><ul><ul><li>Test Execution </li></ul></ul><ul><ul><li>Result capturing and analysing </li></ul></ul><ul><ul><li>Defect Reporting and Status reporting </li></ul></ul>
    61. 61. Software Testing Phases
    62. 62. Software Testing Phases <ul><li>Unit Testing </li></ul><ul><li>Functional Testing </li></ul><ul><li>Integration Testing </li></ul><ul><li>System Testing </li></ul><ul><li>Acceptance Testing </li></ul><ul><li>Interface Testing </li></ul><ul><li>Regression Testing </li></ul><ul><li>Special Testing </li></ul>
    63. 63. Unit Testing <ul><li>Unit Testing is a verification effort on the smallest unit of the software design the software component or module. </li></ul>
    64. 64. Why Unit Testing? <ul><li>Test early for each component and prevent the defect from being carried forward to next stage. </li></ul><ul><li>To ensure that the design specifications have been correctly implemented. </li></ul>?
    65. 65. Approach <ul><li>Uses the component-level design description as a guide. </li></ul><ul><li>Important control paths are tested to uncover errors within the boundary of the module. </li></ul><ul><li>Unit testing is white-box oriented, and this can be conducted in parallel for multiple components. </li></ul><ul><li>the relative complexity of tests and uncovered errors are limited by the constraints scope established for unit testing. </li></ul>
    66. 66. Unit Testing Test Cases Interfaces (input/output) Local Data structures Boundary Conditions Independent Paths Error Handling Paths Module
    67. 67. Unit testing to uncover errors like <ul><li>Comparison of different data types </li></ul><ul><li>Incorrect logical operators or precedence </li></ul><ul><li>Expectation of equality when precision errors makes equality unlikely. </li></ul><ul><li>Incorrect comparison of variables </li></ul><ul><li>Improper or nonexistent loop termination </li></ul><ul><li>Failure to exit when divergent iteration is encountered. </li></ul><ul><li>Improperly modified loop variables, etc. </li></ul>
    68. 68. <ul><li>Misunderstood or incorrect arithmetic precedence </li></ul><ul><li>Mixed mode operations </li></ul><ul><li>Incorrect initialization </li></ul><ul><li>Precision inaccuracy </li></ul><ul><li>Incorrect symbolic representation of an expression </li></ul>Some of the Computational Errors uncovered while Unit Testing
    69. 69. <ul><li>Error description is unintelligible </li></ul><ul><li>Error notes does not correspond to error encountered </li></ul><ul><li>Error condition causes system intervention prior to error handling </li></ul><ul><li>Exception- condition processing is incorrect </li></ul><ul><li>Error description does not provide enough information to assist in the location of the cause of error. </li></ul>Potential errors while error handling is evaluated
    70. 70. Unit Testing Procedure <ul><li>Unit testing is normally considered as an adjunct to the coding step. </li></ul><ul><li>Unit test case design begins ,once the component level design has been developed, reviewed and verified. </li></ul><ul><li>A review of design information provides guidance for establishing test cases that are likely to uncover errors. </li></ul><ul><li>Each test case should be coupled with a set of expected results. </li></ul>
    71. 71. Unit Test Steps <ul><li>The Unit test criteria, the Unit test plan, and the test case specifications are defined. </li></ul><ul><li>A code walkthrough for all new or changed programs or modules is conducted. </li></ul><ul><li>Unit Test data is created, program or module testing is performed, and a Unit test report is written. </li></ul><ul><li>Sign-offs to integration testing must be obtained, Sign-off can be provided by the lead programmer, project coordinator, or project administrator. </li></ul>
    72. 72. Functional Testing <ul><li>Functional Testing is a kind of black box testing because a program’s internal structure is not considered. </li></ul><ul><li>Give the inputs, check the outputs without concentrating on how the operations are performed by the system. </li></ul><ul><li>When black box testing is conducted , the SRS plays a major role and the functionality is given the utmost importance. </li></ul>
    73. 73. Functional Testing <ul><li>Focus on system functions </li></ul><ul><ul><li>developed from the requirements </li></ul></ul><ul><ul><li>Behavior testing </li></ul></ul><ul><li>Should </li></ul><ul><ul><li>know expected results </li></ul></ul><ul><ul><li>test both valid and invalid input </li></ul></ul><ul><li>Unit test cases can be reused </li></ul><ul><li>New end user oriented test cases have to be developed as well. </li></ul>
    74. 74. User Interface <ul><li>This stage will also include Validation Testing </li></ul><ul><li>which is intensive testing of the new Front end </li></ul><ul><li>fields and screens. Windows GUI Standards; </li></ul><ul><li>valid, invalid and limit data input; screen & field </li></ul><ul><li>look and appearance, and overall consistency </li></ul><ul><li>with the rest of the application. </li></ul>
    75. 75. Vertical First Testing : - When the complete set of functionality is taken for one module and tested it is called Vertical First testing. Horizontal First Testing : - If a similar function is taken across all the modules and it is tested, it is called horizontal-first testing. Vertical Horizontal
    76. 76. Integration testing <ul><li>testing with the components put together. </li></ul>
    77. 77. Why integration Testing ? <ul><li>Data can be lost across an interface. </li></ul><ul><li>One module can have an inadvertent, adverse effect on another. </li></ul><ul><li>Sub-functions, when combined, may not produce the desired major function. </li></ul><ul><li>Individually acceptable imprecision may be magnified to unacceptable levels. </li></ul><ul><li>Global data structures can create problems, and so on… </li></ul>
    78. 78. Types of approaches- Top-Down <ul><li>Top-Down is an incremental approach to testing of the program structure. Modules are integrated by moving downward through the control hierarchy, beginning with the main control module, this could be done as depth- first or breadth-first manner. </li></ul>A x 1 2 z y
    79. 79. <ul><li>as the name implies, begins construction and testing with atomic modules i.e., from the components at the lowest levels in the program structure </li></ul>Type of Approaches- Bottom-Up A x 1 2 z y
    80. 80. Integration testing- Example <ul><li>E.g.: Login.java and ConnectionPool.java </li></ul><ul><li>Login class calls the ConnectionPool object , Integration testing identifies errors not observed while code Debugging or Reviews. </li></ul>
    81. 81. System Testing <ul><li>Purpose </li></ul><ul><ul><li>Test the entire system as a whole </li></ul></ul><ul><li>Assumptions </li></ul><ul><ul><li>Completed </li></ul></ul><ul><ul><ul><li>Unit Testing </li></ul></ul></ul><ul><ul><ul><li>Functional Testing </li></ul></ul></ul><ul><ul><ul><li>Integration Testing </li></ul></ul></ul>
    82. 82. Expectations <ul><li>Verification of the system </li></ul><ul><li>Software Requirements </li></ul><ul><li>Business Workflow perspective </li></ul><ul><li>Final verification of requirements and design </li></ul><ul><li>External Interfaces </li></ul><ul><li>Performance tests </li></ul><ul><li>Affected documentation </li></ul><ul><li>Non-testable requirements </li></ul>
    83. 83. Interface Testing <ul><li>Purpose </li></ul><ul><ul><li>Interfaces with the system </li></ul></ul><ul><li>Assumptions </li></ul><ul><ul><li>Unit, functional and integration testing </li></ul></ul><ul><ul><li>All Critical Errors </li></ul></ul><ul><li>Expectations </li></ul><ul><ul><li>Interfaces with External Systems </li></ul></ul><ul><ul><li>Planning and Co-ordination meetings with the external organizations in preparation for testing. </li></ul></ul><ul><ul><ul><li>Who will be the primary contacts? </li></ul></ul></ul><ul><ul><ul><li>When is testing scheduled? </li></ul></ul></ul><ul><ul><ul><ul><li>If there is no test environment available testing may have to occur on weekends or during non-production hours. </li></ul></ul></ul></ul>
    84. 84. Interface Testing: Expectations (Contd.) <ul><li>Expectations (Contd.) </li></ul><ul><ul><li>What types of test cases will be run, how many and what are they testing? </li></ul></ul><ul><ul><ul><li>Provide copies of test cases and procedures to the participants </li></ul></ul></ul><ul><ul><ul><li>If the external organization has specific cases they would like to test, have them provide copies </li></ul></ul></ul><ul><ul><li>Who will supply the data and what will it contain? What format will it be in (paper, electronic, just notes for someone else to construct the data, etc.)? </li></ul></ul><ul><ul><li>Who is responsible for reviewing the results and verifying they are as expected? </li></ul></ul><ul><ul><li>How often will the group meet to discuss problems and testing status? </li></ul></ul>
    85. 85. Interface Testing: Expectations (Contd.) <ul><li>Expectations (Contd.) </li></ul><ul><ul><li>Both normal cases and exceptions should be tested on both sides of the interface (if both sides exchange data). The interface should be tested for handling the normal amount and flow of data as well as peak processing volumes and traffic. </li></ul></ul><ul><ul><li>If appropriate, the batch processing or file transmission “window” should be tested to ensure that both systems complete their processing within the allocated amount of time. </li></ul></ul><ul><ul><li>If fixes or changes need to be made to either side of the interface, the decisions, deadlines and re-test procedures should be documented and distributed to all the appropriate organizations. </li></ul></ul>
    86. 86. Performance Testing <ul><li>Purpose </li></ul><ul><ul><li>The purpose is to verify the system meets the performance requirements. </li></ul></ul><ul><li>Assumptions/Pre-Conditions </li></ul><ul><ul><ul><li>System testing successful. </li></ul></ul></ul><ul><ul><ul><li>Ensure no unexpected performance. </li></ul></ul></ul><ul><ul><ul><li>Prior to Acceptance Testing. </li></ul></ul></ul><ul><ul><ul><li>Tests should use business cases, including normal, error and unlikely cases. </li></ul></ul></ul>
    87. 87. Performance Testing (Contd…) <ul><ul><ul><li>Performance tests </li></ul></ul></ul><ul><ul><ul><ul><li>Load Test </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Stress Test </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Volume Test </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Test data </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Response time </li></ul></ul></ul></ul><ul><ul><ul><ul><li>End-to-end tests and workflows should be performed </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Tracking tool for comparison </li></ul></ul></ul></ul>
    88. 88. Regression Testing
    89. 89. Regression Testing <ul><li>Approach </li></ul><ul><li>Definition and Purpose </li></ul><ul><li>Types of regression testing </li></ul><ul><li>Regression test problems </li></ul><ul><li>Regression testing tools </li></ul>
    90. 90. Regression Testing <ul><li>Definition </li></ul><ul><ul><li>“ Selective retesting to detect faults introduced during modification of a system or system component, to verify that modifications have not caused unintended adverse effects, or to verify that a modified system or system component still meets its specified requirements.” </li></ul></ul><ul><ul><li>“ ...a testing process which is applied after a program is modified.” </li></ul></ul>
    91. 91. Regression Testing <ul><li>Purpose of Regression testing </li></ul><ul><ul><li>Locate errors </li></ul></ul><ul><ul><li>Increase confidence in correctness </li></ul></ul><ul><ul><li>Preserve quality </li></ul></ul><ul><ul><li>Ensure continued operations </li></ul></ul><ul><ul><li>Check correctness of new logic </li></ul></ul><ul><ul><li>Ensure continuous working of unmodified portions </li></ul></ul>
    92. 92. Regression testing types Perfective Maintenance Corrective Maintenance Adaptive Maintenance <ul><li>Corrective - fixing bugs, Design Errors, coding errors </li></ul><ul><li>Adaptive - no change to functionality, but now works under new conditions, i.e., modifications in the environment. </li></ul><ul><li>Perfective - adds something new; makes the system “better” , Eg: adding new modules. </li></ul><ul><li>Preventive – prevent malfunctions or improve maintainability of the software. Eg: code restructuring, optimization, document updating etc </li></ul>Preventive Maintenance
    93. 93. Regression Testing <ul><li>Example 1 - Y2K </li></ul><ul><li>During Y2K code changing, regression testing was the essence of the transition phase. What was typically done, was that code was changed at multiple places (it did not turn the original logic upside down, but made subtle changes). Now Regression testing was very important for the fact that even one small piece of code lying untested could lead to huge ramifications in the large amounts of data that is typically handled by these mainframe computers / programs. </li></ul>
    94. 94. Regression Testing <ul><li>Example 2 – General </li></ul><ul><ul><li>Regression testing might even be required when one of the business associates changes his systems (might be new hardware). Since our system is hooked on to this transition system, our test engineers are also required to do regression testing on our system which has NOT been changed. </li></ul></ul><ul><ul><li>This example brings to light another fact with Regression testing, i.e., sometimes, even an unchanged system needs to be tested! </li></ul></ul>
    95. 95. Regression testing methods <ul><li>Regression testing can be done either manually or by automated testing tools. </li></ul><ul><ul><li>Manual testing: Can be done for small systems, where investing in automated tools might not be feasible enough. </li></ul></ul><ul><ul><li>Automated testing: One class of these tools is called as Capture-playback tool. This is very helpful in situations where the system undergoes lots of version changes. </li></ul></ul>
    96. 96. Acceptance Testing <ul><li>Purpose </li></ul><ul><ul><li>The purpose of acceptance testing is to verify system from user perspective </li></ul></ul><ul><li>Assumptions/Pre-Conditions </li></ul><ul><ul><ul><li>Completed system and regression testing </li></ul></ul></ul><ul><ul><ul><li>Configuration Manager </li></ul></ul></ul><ul><ul><ul><li>Test data </li></ul></ul></ul><ul><ul><ul><li>Final versions of all documents ready </li></ul></ul></ul><ul><ul><ul><li>Overview to the testing procedures </li></ul></ul></ul><ul><ul><ul><li>Exit decision </li></ul></ul></ul><ul><ul><ul><li>Specific procedures </li></ul></ul></ul><ul><ul><ul><li>Acceptance Criteria MUST be documented </li></ul></ul></ul><ul><ul><ul><ul><li>Acceptance Testing </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Project stakeholders </li></ul></ul></ul></ul>
    97. 97. Acceptance Testing <ul><li>Expectations </li></ul><ul><ul><ul><li>Verification from the user’s perspective </li></ul></ul></ul><ul><ul><ul><li>Performance testing should be conducted again </li></ul></ul></ul><ul><ul><ul><li>Extra time </li></ul></ul></ul><ul><ul><ul><li>User manual to the testers </li></ul></ul></ul><ul><ul><ul><li>Non-testable requirements </li></ul></ul></ul><ul><ul><ul><li>Review with the Sponsor and User </li></ul></ul></ul><ul><ul><ul><li>Plans for the Implementation </li></ul></ul></ul>
    98. 98. Field Testing <ul><li>Purpose </li></ul><ul><ul><li>The purpose of field testing is to verify that the systems work in actual user environment. </li></ul></ul><ul><li>Assumptions/Pre-Conditions </li></ul><ul><ul><li>System and/or acceptance testing successful. </li></ul></ul><ul><li>Expectations </li></ul><ul><ul><li>Verification of the system works in the actual user environment. </li></ul></ul><ul><ul><li>Pilot test with the final product. </li></ul></ul><ul><ul><li>Pilot system should work during a problem. </li></ul></ul>
    99. 99. Software Testing Strategies
    100. 100. Testing Information Flow <ul><li>NOTES </li></ul><ul><ul><li>Software Configuration includes a Software Requirements Specification, a Design Specification and source code. </li></ul></ul><ul><ul><li>A test configuration includes a Test Plan and Procedures, test cases and testing tools. </li></ul></ul><ul><ul><li>It is difficult to predict the time to debug the code, hence it is difficult to schedule. </li></ul></ul>Reliability Model Evaluation Testing Debug Test Configuration Software Configuration Expected Results Predicted Reliability Corrections Test Results Error Data Rate Errors
    101. 101. Software Testing Strategies and Techniques <ul><li>Concise Statement of how to meet the objectives of software testing </li></ul><ul><li>To Clarify expectations with the user, sponsor and bidders </li></ul><ul><li>To Describe the details of how the testing team will evaluate the work products, systems and testing activities and results </li></ul><ul><li>To describe the approach to all testing types and phases and the activities for which they are responsible </li></ul>
    102. 102. Test Strategy for Maintenance <ul><li>Includes a greater focus on regression testing, on keeping the users informed of specific fixes or changes that were requested. </li></ul><ul><li>Test process should be described in terms of the periodic release cycles that are part of the change control process. </li></ul><ul><li>Also describe a set of minimum tests to be performed when emergency fixes are needed (for instance, due to failed hardware or recovering from a database crash). </li></ul>
    103. 103. Test Strategy: Inputs & Deliverables Test Strategy Priority & Criticality Types of Applications Project Success Criteria Time Required for Testing No. & Levels of Resources Rounds of Testing Exit Criteria Test Suspension Criteria Resumption Criteria Deliverables Time Required for Testing No. & Levels of Resources Rounds of Testing Exit Criteria Test Suspension Criteria
    104. 104. Typical Test Issues Test Participation Test Environments Approach to Testing External Interfaces Approach to Testing COTS products Scope of Acceptance Testing Verification of Un-testable Requirements Criteria for Acceptance of the System Pilot of Field Testing Performances and Capacity Requirement/Testing Test Issues
    105. 105. Common Test Related Risks and Considerations Test Related & Risks & Considerations Poor Requirements Stakeholder Participation Test Staffing Testing of COTS External Interfaces Performance and Stress Testing Schedule Compression Requirement Testability Acceptance
    106. 106. Test Exit Criteria <ul><li>Executed at least once? </li></ul><ul><li>Requirements been tested or verified? </li></ul><ul><li>Test Documentation </li></ul><ul><li>Documents updated and submitted </li></ul><ul><li>Configuration Manager </li></ul><ul><li>Test Incidents </li></ul>
    107. 107. Software Test Plan
    108. 108. How to achieve good testing <ul><li>Start planning early in the project. </li></ul><ul><li>Prepare a Test Plan. </li></ul><ul><li>Identify the objectives. </li></ul><ul><li>Document objectives in Test Plan. </li></ul>
    109. 109. Test Plan <ul><li>Objective </li></ul><ul><li>A test plan prescribes the scope, approach, resources, and </li></ul><ul><li>schedule of testing activities. It identifies the items to be </li></ul><ul><li>tested, the features to be tested, the testing tasks to be </li></ul><ul><li>performed, the personnel responsible for each task, and the </li></ul><ul><li>risks associated with the plan. </li></ul>
    110. 110. Why Plan Test? <ul><li>Repeatable </li></ul><ul><li>To Control </li></ul><ul><li>Adequate Coverage </li></ul>Test planning process is a critical step in the testing process. Without a documented test plan, the test itself cannot be verified, coverage cannot be analyzed and the test is not repeatable Importance
    111. 111. Test plan To support testing, a plan should be there, which specifies <ul><ul><ul><li>What to do? </li></ul></ul></ul><ul><ul><ul><li>How to do? </li></ul></ul></ul><ul><ul><ul><li>When to do? </li></ul></ul></ul>
    112. 112. Test Plan <ul><li>Test plans need to identify </li></ul><ul><ul><li>The materials needed for testing </li></ul></ul><ul><ul><li>What tests will be run </li></ul></ul><ul><ul><li>What order the tests will be run </li></ul></ul><ul><li>Test plans also need to: </li></ul><ul><ul><li>Name each test </li></ul></ul><ul><ul><li>Predict how long the test should take </li></ul></ul><ul><ul><li>Scripts and test cases will be needed for most tests </li></ul></ul>
    113. 113. Structure 1. Test plan identifier 2. Introduction 3. Test items / integration components 4. Features to be tested 5. Features not to be tested 6. Test Approach 7. Item pass/fail criteria 8. Suspension criteria and resumption requirements Continue.. * As defined by the IEEE 829 Test documentation Std
    114. 114. Structure 9. Test deliverables (PPlan) 10. Environmental needs (H/w & S/w) 11. Responsibilities (PPlan) 12. Staffing and Training needs (PPlan) 13. Schedule (PPlan) 14. Risks and Contingencies (PPlan) 15. Approvals Ref : Test Plan Template IEEE 829
    115. 115. Testing Process
    116. 116. <ul><li>Code Based Test Case Design </li></ul><ul><li>Requirements Based Test Case Design </li></ul>Testing Techniques
    117. 117. Testing Techniques <ul><li>Specification Based (Black Box/Functional Testing) </li></ul><ul><ul><li>Equivalence Partitioning </li></ul></ul><ul><ul><li>Cause Effect Graphing </li></ul></ul><ul><ul><li>Boundary Value Analysis </li></ul></ul><ul><ul><li>Category Partition </li></ul></ul><ul><ul><li>Formal Specification Based </li></ul></ul><ul><ul><li>Control Flow Based Criteria </li></ul></ul><ul><ul><li>Data Flow based criteria </li></ul></ul><ul><li>Fault Based </li></ul><ul><ul><li>Error Guessing </li></ul></ul><ul><ul><li>Mutation </li></ul></ul><ul><ul><li>Fault Seeding </li></ul></ul>
    118. 118. Testing Techniques (Contd…) <ul><li>Usage Based </li></ul><ul><ul><li>Statistical testing </li></ul></ul><ul><ul><li>(Musa’s)SRET </li></ul></ul><ul><li>Specific Technique </li></ul><ul><ul><li>Object Oriented Testing </li></ul></ul><ul><ul><li>Component Based Testing </li></ul></ul><ul><li>Code Based (White Box/Structural testing) </li></ul><ul><ul><li>Statement Coverage </li></ul></ul><ul><ul><li>Edge Coverage </li></ul></ul><ul><ul><li>Condition Coverage </li></ul></ul><ul><ul><li>Path Coverage </li></ul></ul><ul><ul><li>Cyclomatic Complexity </li></ul></ul>
    119. 119. Test Data Adequacy Criteria <ul><li>Have I </li></ul><ul><ul><li>Tested </li></ul></ul><ul><ul><li>Exercised </li></ul></ul><ul><ul><li>Forced </li></ul></ul><ul><ul><li>Found </li></ul></ul><ul><li>Have I </li></ul><ul><ul><li>Thought </li></ul></ul><ul><ul><li>Applied all inputs </li></ul></ul><ul><ul><li>Completely explored </li></ul></ul><ul><ul><li>Run all the Scenarios </li></ul></ul>Test Data Adequacy Criteria Code Based Testing Requirement Based Testing
    120. 120. Test Preparation Checklist <ul><li>Test Id </li></ul><ul><li>Version </li></ul><ul><li>Users A/c </li></ul><ul><li>Input DB </li></ul><ul><li>Training </li></ul><ul><li>Release to System </li></ul><ul><li>Reset System </li></ul><ul><li>Test Environment </li></ul><ul><li>Stake Holders </li></ul><ul><li>Schedule…… </li></ul>
    121. 121. <ul><li>Code Based Test Case Design </li></ul><ul><li>Requirements Based Test Case Design </li></ul>Test Design Specifications
    122. 122. Purpose of Test Design Specification <ul><li>Requirements of Test Approach </li></ul><ul><li>Identify the features to be tested </li></ul><ul><li>Arrive at High Level </li></ul>
    123. 123. Contents of Test Design Specification <ul><li>Identification and Purpose </li></ul><ul><li>Features to be tested </li></ul><ul><li>Approach Refinements </li></ul><ul><li>Test Identification </li></ul><ul><li>Pass/Fail Criteria </li></ul>
    124. 124. Approach <ul><li>Study Business Requirements </li></ul><ul><li>Arrive at Environmental Requirements </li></ul><ul><li>Identify test related Risks </li></ul><ul><li>Decide Automation Requirements </li></ul><ul><li>Prepare Test Documents </li></ul><ul><li>Plan for Test Completion </li></ul><ul><li>Analyze Track changes </li></ul><ul><li>Review Test design effectiveness </li></ul>
    125. 125. Test Cases
    126. 126. Test Case Sheet To Capture Details: 1.Testcase ID (should be unique, e.g.: c_01.1, c_01.1a, c_01.2,…) 2.Feature functionality to be tested (each Requirement/feature could be from Usecase/COMP) 3.Test Description/ test input details (test input, test data, action to be performed to test the feature, complex test cases be split to more than one) 4.Expected behavior ( in messages, screens, data, to be with correct details) 5.Actual and Status
    127. 127. Test case development process <ul><ul><li>Identify all potential Test Cases needed to fully test the business and technical requirements </li></ul></ul><ul><ul><li>Document Test Procedures </li></ul></ul><ul><ul><li>Document Test Data requirements </li></ul></ul><ul><ul><li>Prioritize test cases </li></ul></ul><ul><ul><li>Identify Test Automation Candidates </li></ul></ul><ul><ul><li>Automate designated test cases </li></ul></ul>
    128. 128. Type of test cases Type Source 1.Requirement Based Specifications 2.Design based Logical system 3.Code based Code 4.Extracted Existing files or test cases 5.Extreme Limits and boundary conditions
    129. 129. Requirement based test cases <ul><li>Identify the basic cases that indicate program functionality. </li></ul><ul><li>Create a minimal set of tests to cover all inputs and outputs. </li></ul><ul><li>Breakdown complex cases into single cases. </li></ul><ul><li>Remove unnecessary or duplicate cases. </li></ul><ul><li>Review systematically and thoroughly. </li></ul><ul><li>Design based test cases supplement requirements based test cases. </li></ul>Steps for selecting test cases:
    130. 130. Code based test cases <ul><li>Every statement exercised at least once. </li></ul><ul><li>Every decision exercised over all outcomes. </li></ul>Goals for complete code based coverage:
    131. 131. Extreme cases <ul><li>Looks for exceptional conditions, </li></ul><ul><li>extremes, boundaries, and abnormalities . </li></ul><ul><li>Requires experience, creativity of the Test Engineer </li></ul>Need:
    132. 132. Extracted and randomized cases <ul><li>Extracted cases involved extracting </li></ul><ul><li>samples of real data for the testing </li></ul><ul><li>process. </li></ul><ul><li>Randomized cases involved using tools </li></ul><ul><li>to generate potential data for the testing </li></ul><ul><li>process. </li></ul>
    133. 133. Characteristics of good test case <ul><li>Specific </li></ul><ul><li>Non-redundant </li></ul><ul><li>Reasonable probability of catching an error </li></ul><ul><li>Medium complexity </li></ul><ul><li>Repeatable </li></ul><ul><li>Always list expected results </li></ul>
    134. 134. Test case guidelines <ul><ul><li>Developed to verify that specific requirements or design are satisfied </li></ul></ul><ul><ul><li>Each component must be tested with at least two test cases: Positive and Negative </li></ul></ul><ul><ul><li>Real data should be used to reality test the modules after successful test data is used </li></ul></ul>
    135. 135. The Testing process Test Cases Test Data Test Results Test Reports Design Test Cases Prepare test data Run Prg with Test data Compare results
    136. 136. <ul><ul><li>Statement Coverage </li></ul></ul><ul><ul><li>Edge Coverage </li></ul></ul><ul><ul><li>Condition Coverage </li></ul></ul><ul><ul><li>Path Coverage </li></ul></ul><ul><ul><li>Cyclomatic Complexity </li></ul></ul>Code Base Test Case Design
    137. 137. Purpose <ul><li>Understand the Objective </li></ul><ul><li>Effective conversion of specifications </li></ul><ul><li>Checking Programming Style with coding standards </li></ul><ul><li>Check Logic Errors </li></ul><ul><li>Incorrect Assumptions </li></ul><ul><li>Typographical Errors </li></ul>
    138. 138. Code Based Testing - White Box Testing <ul><li>Coding Standards </li></ul><ul><li>Logic Programming Style </li></ul><ul><li>Complexity of Code </li></ul><ul><li>Structural Testing </li></ul><ul><li>Ensure Reduced Rework </li></ul><ul><li>Quicker Stability </li></ul><ul><li>Smooth Acceptance </li></ul><ul><li>Structure of the Software itself </li></ul><ul><li>Valuable Source </li></ul><ul><li>Selecting test cases </li></ul>
    139. 139. Code Based Testing or White Box Testing <ul><li>Testing control structures of a procedural design. </li></ul><ul><li>Can derive test cases to ensure: </li></ul><ul><ul><li>All independent paths are exercised at least once. </li></ul></ul><ul><ul><li>All logical decisions are exercised for both true and false paths. </li></ul></ul><ul><ul><li>All loops are executed at their boundaries and within operational bounds. </li></ul></ul><ul><ul><li>All internal data structures are exercised to ensure validity. </li></ul></ul>Contd..2
    140. 140. Code Based Testing or White Box Testing (Contd..) <ul><ul><li>Why do white box testing when black box testing is used to test conformance to requirements? </li></ul></ul><ul><ul><ul><li>Logic errors and incorrect assumptions most likely to be made when coding for &quot;special cases&quot;. Need to ensure these execution paths are tested. </li></ul></ul></ul>
    141. 141. Code Based Testing or White Box Testing (Contd..) <ul><ul><ul><li>May find assumptions about execution paths incorrect and so make design errors. White box testing can find these errors. </li></ul></ul></ul><ul><ul><ul><li>Typographical errors are random. Just as likely to be on an obscure logical path as on a mainstream path. </li></ul></ul></ul><ul><ul><ul><ul><li>&quot;Bugs lurk in corners and congregate at boundaries&quot; </li></ul></ul></ul></ul>
    142. 142. Types of Code Based Testing & Adequacy Criteria <ul><li>Involve Control Flow Testing </li></ul><ul><ul><li>Statement Coverage </li></ul></ul><ul><ul><ul><li>Is every statement executed at least once? </li></ul></ul></ul><ul><ul><li>Edge Coverage </li></ul></ul><ul><ul><ul><li>Is every edge in the control flow graph executed? </li></ul></ul></ul><ul><ul><li>Condition Coverage </li></ul></ul><ul><ul><ul><li>Is edge + every Boolean (sub) expression in the control flow graph executed? </li></ul></ul></ul><ul><ul><li>Path Coverage </li></ul></ul><ul><ul><ul><li>Is every path in the control flow graph executed? </li></ul></ul></ul><ul><ul><li>Cyclomatic Complexity </li></ul></ul><ul><ul><ul><li>Is the logical structure of the program appropriate? </li></ul></ul></ul>
    143. 143. Test Cases Derive Test Cases Independent Path Logical Decisions Boundaries Data Structures
    144. 144. Types of Code Based Testing (1) - Statement Coverage <ul><li>Control Flow elements to be exercised in statements. </li></ul><ul><li>Statements coverage criterion requires elementary statement, where program is executed at least once. </li></ul>Total Number of Statements (T) Number of Executed Statements (P) Statement coverage (C) =
    145. 145. Types of Code Based Testing (2) - Edge Coverage (Branch Coverage) <ul><li>Focus is on identifying test cases executing each branch at least once. </li></ul>Total Number of Branches (T) Number of Executed Branches (P) Edge Covers (C) =
    146. 146. Types of Code Based Testing(3) - Conditions Coverage <ul><li>Combination of Edge Coverage and more detailed conditions. </li></ul><ul><li>Examples: True & False, Elementary Conditions, Comparisons, Boolean Expressions. </li></ul>Total Number of Conditions (T) Number of Executed Conditions (P) Basic Conditions Coverage (C) =
    147. 147. Types of Code Based Testing(3) - Conditions Coverage (Contd.) <ul><li>Condition testing aims to exercise all logical conditions in a program module. It is defined as: </li></ul><ul><ul><li>Relational expression: (E1 op E2), where E1 and E2 are arithmetic expressions. </li></ul></ul><ul><li>Simple condition: Boolean variable or relational expression, possibly preceded by a NOT operator. </li></ul>
    148. 148. Types of Code Based Testing(3) - Conditions Coverage (Contd.) <ul><li>Compound condition: Composed of two or more simple conditions, boolean operators and parentheses. </li></ul><ul><li>Boolean expression: Condition without relational expressions. </li></ul>
    149. 149. Types of Code Based Testing(3) - Conditions Coverage (Contd.) <ul><li>Errors in expressions can be due to: </li></ul><ul><ul><li>Boolean operator error </li></ul></ul><ul><ul><li>Boolean variable error </li></ul></ul><ul><ul><li>Boolean parenthesis error </li></ul></ul><ul><ul><li>Relational operator error </li></ul></ul><ul><ul><li>Arithmetic expression error </li></ul></ul><ul><li>Condition testing methods focus on testing each condition in the program. </li></ul>
    150. 150. Types of Code Based Testing(3) - Conditions Coverage (Contd.) <ul><li>Strategies proposed include: </li></ul><ul><ul><li>Branch testing - execute every branch at least once. </li></ul></ul><ul><ul><li>Domain Testing - uses three or four tests for every relational operator. </li></ul></ul><ul><ul><li>Branch and relational operator testing - uses condition constraints. </li></ul></ul>
    151. 151. Types of Code Based Testing(3) - Conditions Coverage (Contd.) <ul><li>Example 1: C1 = B1 & B2 </li></ul><ul><ul><li>where B1, B2 are boolean conditions. </li></ul></ul><ul><ul><li>Condition constraint of form (D1,D2) where D1 and D2 can be true (t) or false(f). </li></ul></ul><ul><ul><li>The branch and relational operator test requires the constraint set {(t,t),(f,t),(t,f)} to be covered by the execution of C1. </li></ul></ul><ul><li>Coverage of the constraint set guarantees detection of relational operator errors </li></ul>
    152. 152. <ul><li>Path Coverage executed at least once. </li></ul><ul><ul><li>Selects test paths according to the location of definitions and use of variables. </li></ul></ul><ul><li>Test for Loops (iterations) </li></ul><ul><ul><li>Loop Testing. </li></ul></ul><ul><ul><li>Loops fundamental to many algorithms. </li></ul></ul><ul><ul><li>Can define loops as simple, concatenated, nested and unstructured. </li></ul></ul>Types of Code Based Testing(4) - Path Coverage : Data Flow Testing
    153. 153. Types of Code Based Testing(4) - Path Coverage: Loop Testing: Examples Simple Nested Concatenated Unstructured
    154. 154. Types of Code Based Testing(4) - Path Coverage: Simple Loops <ul><li>Simple Loops of size n: </li></ul><ul><ul><ul><li>Skip loop entirely </li></ul></ul></ul><ul><ul><ul><li>Only one pass through loop </li></ul></ul></ul><ul><ul><ul><li>Two passes through loop </li></ul></ul></ul><ul><ul><ul><li>m passes through loop where, m<n. </li></ul></ul></ul><ul><ul><ul><li>(n-1), n and (n+1) passes through the loop. </li></ul></ul></ul>Simple
    155. 155. Types of Code Based Testing(4) - Path Coverage: Nested Testing <ul><li>Nested Loops </li></ul><ul><ul><li>Start with inner loop. Set all other loops to minimum values. </li></ul></ul><ul><ul><li>Conduct simple loop testing on inner loop. </li></ul></ul><ul><ul><li>Work outwards. </li></ul></ul><ul><ul><li>Continue until all loops are tested. </li></ul></ul>Nested
    156. 156. Types of Code Based Testing(4) - Path Coverage: Concatenated Loop <ul><li>Concatenated Loops test </li></ul><ul><ul><ul><li>If independent loops, use simple loop testing. </li></ul></ul></ul><ul><ul><ul><li>If dependent, treat as nested loops. </li></ul></ul></ul>Concatenated
    157. 157. <ul><li>Unstructured loops </li></ul><ul><ul><ul><li>Don't test - redesign. </li></ul></ul></ul>Types of Code Based Testing(4) - Path Coverage: Unstructured Loops Unstructured
    158. 158. <ul><li>Measures the amount of decision logic in a single software module. </li></ul><ul><li>The Cyclomatic complexity gives a quantitative measure of the logical complexity. </li></ul><ul><li>This value gives the number of independent paths in the basis set and an upper bound for the number of tests to ensure that each statement is executed at least once. </li></ul>Types of Code Based Testing(5) - Cyclomatic Complexity
    159. 159. Cyclomatic Complexity <ul><li>An independent path is any path through a program that introduces at least one new set of processing statements or a new condition (i.e., a new edge). </li></ul>
    160. 160. Relationship with Programming Complexity <ul><li>Cyclomatic Complexity calculations help the developer/tester to decide whether the module under test is overly complex or well written. </li></ul><ul><li>Recommended limit value of Cyclomatic Complexity is 10. </li></ul><ul><ul><li>>10 </li></ul></ul><ul><ul><ul><li>Structure of the module is overly complex. </li></ul></ul></ul><ul><ul><li>>5 and <10 </li></ul></ul><ul><ul><ul><li>Structure of the module is complex indicating that the logic is difficult to test. </li></ul></ul></ul><ul><ul><li><5 </li></ul></ul><ul><ul><ul><li>structure of the module is simple and logic is easy to test. </li></ul></ul></ul>
    161. 161. Flow Graphic Notation Sequence If While Until Case
    162. 162. Flow Graphic Notation <ul><li>On a flow graph: </li></ul><ul><ul><li>Arrows called edges represent flow of control. </li></ul></ul><ul><ul><li>Circles called nodes represent one or more actions. </li></ul></ul><ul><ul><li>Areas bounded by edges and nodes are called regions. </li></ul></ul><ul><ul><li>A predicate node is a node containing a condition. </li></ul></ul>Contd..2
    163. 163. Flow Graphic Notation <ul><li>Any procedural design can be translated into a flow graph. </li></ul><ul><li>Note that compound Boolean expressions at tests generate at least two predicate node and additional arcs. </li></ul>Contd..2
    164. 164. Flow Graphic Notation
    165. 165. Deriving Cyclomatic Complexity <ul><li>Cyclomatic Complexity equals number of independent paths through standard control flow graph model. </li></ul><ul><li>Steps to arrive at Cyclomatic Complexity </li></ul><ul><ul><li>Draw a corresponding flow graph. </li></ul></ul><ul><ul><li>Determine Cyclomatic Complexity. </li></ul></ul><ul><ul><li>Determine independent paths. </li></ul></ul><ul><ul><li>Prepare tests cases. </li></ul></ul>
    166. 166. <ul><li>1. Do while records remain </li></ul><ul><li>read record </li></ul><ul><li>2. If record field 1=0 </li></ul><ul><li>3. Then process record; </li></ul><ul><li>store in buffer, </li></ul><ul><li>increment counter, </li></ul><ul><li>4. Elseif record field 2=0 </li></ul><ul><li>5. Then reset record; </li></ul><ul><li>6. Else process record; </li></ul><ul><li>store in file, </li></ul><ul><li>7a Endif </li></ul><ul><li>Endif </li></ul><ul><li>7b.Enddo </li></ul><ul><li>8. End </li></ul>Cyclomatic Complexity: Example PROCEDURE SORT Contd..2 1 2 4 3 5 6 7a 8 7b
    167. 167. Reporting Cyclomatic Complexity <ul><li>The McCabe Cyclomatic complexity V ( G ) of a control flow graph measures the maximum number of linearly independent paths through it. The complexity typically increases because of branch points. </li></ul><ul><li>Definitions: </li></ul><ul><li>Cyclomatic Complexity V(G) = e – n + 2 </li></ul>
    168. 168. Reporting Cyclomatic Complexity <ul><li>To compute the Cyclomatic complexity: V ( G ) where v refers to the Cyclomatic number in graph theory and G indicates that the complexity is a function of the graph. </li></ul><ul><ul><li>If e is the number of arcs, </li></ul></ul><ul><ul><li>n is the number of nodes and </li></ul></ul><ul><ul><li>p is the number of connected components or predicates or modules, then </li></ul></ul><ul><ul><li>Linearly independent paths, </li></ul></ul><ul><ul><ul><li>V ( G ) = e - n + 2 * p </li></ul></ul></ul>
    169. 169. Software Testing Technique <ul><ul><li>Example </li></ul></ul><ul><ul><ul><ul><li>Independent Paths: </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>1, 1, 8 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>1, 2, 3, 7b, 1, 8 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>1, 2, 4, 5, 7a, 7b, 1, 8 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>1, 2, 4, 6, 7a, 7b, 1, 8 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Cyclomatic complexity provides upper bound for number of tests required to guarantee coverage of all program statements. </li></ul></ul></ul></ul>
    170. 170. Summary: Cyclomatic Complexity <ul><li>The number of tests to test all control statements + one virtual path equals the Cyclomatic complexity. </li></ul><ul><li>Cyclomatic complexity equals number of conditions in a program. </li></ul><ul><li>Useful if used with care. Does not imply adequacy. </li></ul><ul><li>Does not take into account data-driven programs. </li></ul>
    171. 171. Deriving Test Cases <ul><li>Using the design or code, draw the corresponding flow graph. </li></ul><ul><li>Determine the Cyclomatic complexity of the flow graph. </li></ul><ul><li>Determine a basis set of independent paths. </li></ul><ul><li>Prepare test cases that will force execution of each path in the basis set. </li></ul><ul><ul><li>Note: some paths may only be able to be executed as part of another test. </li></ul></ul>
    172. 172. Graph Matrices <ul><li>Can automate derivation of flow graph and determination of a set of basis paths. </li></ul><ul><li>Software tools to do this can use a graph matrix. </li></ul><ul><li>Graph matrix: </li></ul><ul><ul><li>Is square with # of sides equal to # of nodes. </li></ul></ul><ul><ul><li>Rows and columns correspond to the nodes. </li></ul></ul><ul><ul><li>Entries correspond to the edges. </li></ul></ul>Contd..2
    173. 173. Graph Matrices <ul><li>Can associate a number with each edge entry. </li></ul><ul><li>Use a value of 1 to calculate the Cyclomatic complexity </li></ul><ul><ul><li>For each row, sum column values and subtract 1. </li></ul></ul><ul><ul><li>Sum these totals and add 1. </li></ul></ul>Contd..2
    174. 174. Some other interesting link weights <ul><li>Probability that a link (edge) will be executed. </li></ul><ul><li>Processing time for traversal of a link. </li></ul><ul><li>Memory required during traversal of a link. </li></ul><ul><li>Resources required during traversal of a link. </li></ul>Contd..2
    175. 175. Graph Matrices 1 2 4 3 5 6 7a 8 7b
    176. 176. Introduction to Static Testing
    177. 177. Static Testing <ul><li>Static testing is the process of evaluating a system or component based on its form, structure, content or documentation (without computer program execution). </li></ul><ul><li>Reviews form an important activity in static testing. </li></ul>
    178. 178. Reviews <ul><li>Reviews are &quot;filters&quot; applied to uncover error from products at the end of each phase. </li></ul><ul><li>A review process can be defined as a critical evaluation of an object. </li></ul><ul><li>Involve a group meeting to assess a work product. In certain phases, such as the Requirements phase, Prototyping phase and the final delivery phase. </li></ul>
    179. 179. Benefits of Reviews <ul><li>Identification of the anomalies at the earlier stage of the life cycle </li></ul><ul><li>Identifying needed improvements </li></ul><ul><li>Certifying correctness </li></ul><ul><li>Encouraging uniformity </li></ul><ul><li>Enforcing subjective rules </li></ul>
    180. 180. Types of Reviews <ul><li>Inspections </li></ul><ul><li>Walkthroughs </li></ul><ul><li>Technical Reviews </li></ul><ul><li>Audits </li></ul>
    181. 181. Work-products that undergo reviews <ul><li>Software Requirement Specification </li></ul><ul><li>Software design description </li></ul><ul><li>Source Code </li></ul><ul><li>Software test documentation </li></ul><ul><li>Software user documentation </li></ul><ul><li>System Build </li></ul><ul><li>Release Notes </li></ul><ul><li>Let us discuss Inspections, Walkthroughs and Technical Reviews with respect to Code. </li></ul>
    182. 182. Code Inspection <ul><li>Code inspection is a visual examination of a software product to detect and identify software anomalies including errors and deviations from standards and specifications. </li></ul><ul><li> Inspections are conducted by peers led by impartial facilitators. </li></ul><ul><li>Inspectors are trained in Inspection techniques. </li></ul><ul><li>Determination of remedial or investigative action for an anomaly is mandatory element of software inspection </li></ul><ul><li>Attempt to discover the solution for the fault is not part of the inspection meeting. </li></ul>
    183. 183. Objectives of code Inspection <ul><li>Cost of detecting and fixing defects is less during early stages. </li></ul><ul><li>Gives management an insight into the development process – through metrics. </li></ul><ul><li>Inspectors learn from the inspection process. </li></ul><ul><li>Allows easy transfer of ownership, should staff leave or change responsibility. </li></ul><ul><li>Build team strength at emotional level. </li></ul>
    184. 184. Composition of Code Inspection Team <ul><li>Author </li></ul><ul><li>Reader </li></ul><ul><li>Moderator </li></ul><ul><li>Inspector </li></ul><ul><li>Recorder </li></ul>
    185. 185. Rules for Code Inspection <ul><li>Inspection team can have only 3 to 6 participants maximum. </li></ul><ul><li>Author shall not act as Inspection leader, reader or recorder. </li></ul><ul><li>Management member shall not participate in the inspection. </li></ul><ul><li>Reader responsible for leading the inspection team through the program written interpreting sections of work line by line. </li></ul><ul><li>Relating the code back to higher level work products like Design, Requirements. </li></ul>
    186. 186. Inspection Process <ul><li>Overview </li></ul><ul><li>Preparation </li></ul><ul><li>Inspection </li></ul><ul><li>Rework </li></ul><ul><li>Follow up </li></ul>
    187. 187. Classification of anomaly <ul><li>Missing </li></ul><ul><li>Superfluous (additional) </li></ul><ul><li>Ambiguous </li></ul><ul><li>Inconsistent </li></ul><ul><li>Improvement desirable </li></ul><ul><li>Non-conformance to standards </li></ul><ul><li>Risk-prone (safer alternative methods are available) </li></ul><ul><li>Factually incorrect </li></ul><ul><li>Non-implementable (due to system or time constraints) </li></ul>
    188. 188. Severity of anomaly <ul><li>Major </li></ul><ul><li>Minor </li></ul>
    189. 189. Benefits of Code Inspection <ul><li>Synergy – 3-6 active people work together, focused on a common goal. </li></ul><ul><li>Work product is detached from the individual. </li></ul><ul><li>Identification of the anomalies at the earlier stage of the life cycle. </li></ul><ul><li>Uniformity is maintained. </li></ul>
    190. 190. Guidelines for Code Inspection <ul><li>Adequate preparation time must be provided to participants. </li></ul><ul><li>The inspection time must be limited to 2-hours sessions, with a maximum of 2 sessions a day. </li></ul><ul><li>The inspection meeting must be focused only on identifying anomalies, not on the resolution of the anomalies. </li></ul><ul><li>The author must be dissociated from his work. </li></ul><ul><li>The management must not participate in the inspections. </li></ul><ul><li>Selecting the right participants for the inspection. </li></ul>
    191. 191. Output of Code Inspection <ul><li>Inspection team members </li></ul><ul><li>Software program examined </li></ul><ul><li>Code inspection objectives and whether they were met. </li></ul><ul><li>Recommendations regarding each anomaly. </li></ul><ul><li>List of actions, due dates and responsible people. </li></ul><ul><li>Recommendations, if any, to the QA group to improve the process </li></ul>
    192. 192. Code Walkthrough <ul><li>Walkthrough is a static analysis technique in which a designer or programmer leads members of the development team and other interested parties through a software program. </li></ul><ul><li>Participants ask questions on the program and make comments about possible errors, violation of standards, guidelines etc. </li></ul>
    193. 193. Objectives of Code Walkthrough <ul><li>To evaluate a software program, check conformance to standards, guidelines and specifications </li></ul><ul><li>Educating / Training participants </li></ul><ul><li>Find anomalies </li></ul><ul><li>Improve software program </li></ul><ul><li>Consider alternative implementation if required (not done in inspections) </li></ul>
    194. 194. Difference between Inspections and Walkthroughs Walkthrough process includes Overview, little or no preparation, examination (actual walkthrough meeting), rework and follow up. Inspection process includes Overview, preparation, inspection, rework and follow up. No checklist used in walkthroughs Checklist is used to find faults Usually team members of the same project take participation in the walkthrough. Author himself acts the walkthrough leader. A group of relevant persons from different departments participate in the inspection.
    195. 195. Difference between Inspections and Walkthroughs Contd. Shorter time is spent on walkthroughs as there is not formal checklist used to evaluate the program. Inspection takes longer time as the list of items in the checklist is tracked to completion. No formalized procedure in the steps. Formalized procedure in each step.
    196. 196. Code Walkthrough Team <ul><li>Author </li></ul><ul><li>Walkthrough Leader </li></ul><ul><li>Recorder </li></ul><ul><li>Team member </li></ul>
    197. 197. Code Walkthrough Process <ul><li>Overview </li></ul><ul><li>Preparation </li></ul><ul><li>Examination </li></ul><ul><li>Rework / Follow-up </li></ul>
    198. 198. Outputs of Code Walkthrough <ul><li>Walkthrough team members </li></ul><ul><li>Software program examined </li></ul><ul><li>Walkthrough objectives and whether they were met. </li></ul><ul><li>Recommendations regarding each anomaly. </li></ul><ul><li>List of actions, due dates and responsible people. </li></ul>
    199. 199. Technical Review of Code <ul><li>A technical review is a formal team evaluation of a product. </li></ul><ul><li>It identifies any discrepancies from specifications and standards or provides recommendations after the examination of alternatives or both. </li></ul><ul><li>The technical review is less formal than the formal inspection. </li></ul><ul><li>The technical review participants include the author and participants knowledgeable of the technical content of the product being reviewed. </li></ul>
    200. 200. Technical review process <ul><li>Step 1: Planning the Technical Review Meeting </li></ul><ul><li>Step 2: Reviewing the Product </li></ul><ul><li>Step 3: Conducting the Technical Review </li></ul><ul><li>Step 4: Resolving Defects </li></ul><ul><li>Step 5: Reworking the Product </li></ul>
    201. 201. Outputs of Technical review <ul><li>Same as Inspections. </li></ul>
    202. 202. Requirement Bases Test Design- Black Box Technique <ul><ul><li>Low Level Testing </li></ul></ul><ul><ul><li>High Level Testing </li></ul></ul>
    203. 203. Purpose <ul><li>Is to find </li></ul><ul><ul><li>Functional validity of the system </li></ul></ul><ul><ul><li>Sensitivity </li></ul></ul><ul><ul><li>Tolerance </li></ul></ul><ul><ul><li>Operability </li></ul></ul><ul><ul><li>Interface errors </li></ul></ul><ul><ul><li>Errors in database structures </li></ul></ul><ul><ul><li>Performance errors </li></ul></ul><ul><ul><li>Initialization and termination errors </li></ul></ul>
    204. 204. Approach <ul><li>Positive Testing </li></ul><ul><li>Negative Testing </li></ul><ul><li>Use case Testing </li></ul>
    205. 205. Categories of Requirements <ul><li>Functional </li></ul><ul><ul><li>Absolutely necessary for functioning of system </li></ul></ul><ul><ul><li>Describes the input/output behaviour of the system </li></ul></ul><ul><ul><li>Shalls of the software </li></ul></ul><ul><ul><li>Must be testable </li></ul></ul><ul><li>Non-functional </li></ul><ul><ul><li>Restriction or constraints on system services </li></ul></ul><ul><ul><li>Define the attributes of the system as it performs its job </li></ul></ul><ul><ul><li>Subjective in nature and not conclusively testable </li></ul></ul><ul><ul><li>In real-systems, these are more important than functional requirements! </li></ul></ul>
    206. 206. Validating Functional Requirements <ul><li>Black Box Testing </li></ul><ul><ul><li>Low Level Testing </li></ul></ul><ul><ul><li>High Level Testing </li></ul></ul>
    207. 207. Validating Non-functional Requirements <ul><li>Software Quality Factors </li></ul><ul><li>Test cases generated to validate the metrics </li></ul><ul><ul><li>Criteria is met </li></ul></ul><ul><ul><li>Factor is met </li></ul></ul>Prioritization of factors Important factor criteria metric
    208. 208. <ul><li>Low Level Techniques </li></ul><ul><ul><li>Equivalence partitioning </li></ul></ul><ul><ul><li>Boundary value analysis </li></ul></ul><ul><ul><li>Input domain & Output domain </li></ul></ul><ul><ul><li>Special Value </li></ul></ul><ul><ul><li>Error based </li></ul></ul><ul><ul><li>Cause-effect Graph </li></ul></ul><ul><ul><li>Comparison Testing </li></ul></ul><ul><li>High Level Techniques </li></ul><ul><ul><li>Specification-based testing </li></ul></ul><ul><li>Express requirements in simple formal notations like </li></ul><ul><ul><li>State machine </li></ul></ul><ul><ul><li>Decision table </li></ul></ul><ul><ul><li>Use cases </li></ul></ul><ul><ul><li>Flowchart </li></ul></ul><ul><ul><li>Boolean logic </li></ul></ul><ul><ul><li>Regular expressions </li></ul></ul><ul><li>The notation allows generation of scenarios. </li></ul><ul><li>Different test cases for every scenario. </li></ul><ul><li>Good side effects! </li></ul><ul><li>Makes requirements verifiable, finds flaws in requirements. </li></ul>Requirements Based Test Design - Black Box Techniques
    209. 209. Requirement Base Test Design- Black Box Technique <ul><ul><li>High Level Techniques </li></ul></ul>
    210. 210. Techniques <ul><li>State Machine </li></ul><ul><li>Decision Table </li></ul><ul><li>Flowchart </li></ul><ul><li>Use Cases </li></ul>
    211. 211. State Machine <ul><li>Description </li></ul><ul><ul><li>State based business logic </li></ul></ul><ul><ul><li>Covering all paths generate test cases </li></ul></ul><ul><ul><li>Diagram may be complicated </li></ul></ul><ul><ul><li>For every event generate test cases using BVA, EP… </li></ul></ul><ul><li>State Diagram </li></ul>
    212. 212. Decision Table <ul><li>Explores combinations of input conditions </li></ul><ul><li>Consists of 2 parts: Condition section and Action section </li></ul><ul><ul><li>Condition Section - Lists conditions and their combinations </li></ul></ul><ul><ul><li>Action Section - Lists responses to be produced </li></ul></ul><ul><li>Exposes errors in specification </li></ul><ul><li>Columns in decision table are converted to test cases </li></ul><ul><li>Similar to Condition Coverage used in White Box Testing </li></ul>CONDITION ACTION √ (W) NA √ (W) Warning Message √ X √ Unsuccessful Login X √ X Successful Login X √ X Password X √ √ Login Value 3 Value 2 Value 1
    213. 213. Flowchart <ul><li>Description </li></ul><ul><ul><li>Flow based business logic </li></ul></ul><ul><ul><li>Generate test cases covering all paths </li></ul></ul><ul><ul><li>Simple to use </li></ul></ul><ul><ul><li>For every condition generate test cases using BVA, EP… </li></ul></ul>
    214. 214. Use Cases : <ul><li>Simple and Effective method to find errors in Object Oriented applications during Analysis phase. </li></ul><ul><li>Good start for User Acceptance Testing and Plan. </li></ul><ul><li>Accurately reflects business requirements. </li></ul>
    215. 215. Requirement Base Test Design- Black Box Technique <ul><ul><li>Low Level Techniques </li></ul></ul>
    216. 216. Techniques <ul><li>Equivalence partitioning </li></ul><ul><li>Boundary value analysis </li></ul><ul><li>Input domain & Output domain </li></ul><ul><li>Special Value </li></ul><ul><li>Error based </li></ul><ul><li>Cause-effect Graph </li></ul><ul><li>Comparison Testing </li></ul>
    217. 217. Low Level Techniques (1) - Equivalence Partitioning <ul><li>Divides the input domain into classes of data for which test cases can be generated. </li></ul><ul><li>Attempts to uncover classes of errors. </li></ul><ul><li>Divides the input domain of a program into classes of data. </li></ul><ul><li>Derives test cases based on these partitions. </li></ul><ul><li>An equivalence class is a set of valid or invalid states of input. </li></ul><ul><li>Test case design is based on equivalence classes for an input domain. </li></ul>Invalid Inputs Valid Inputs SYSTEM` Output
    218. 218. Low Level Techniques (1) - Equivalence Partitioning (Contd..) <ul><li>Useful in reducing the number of Test Cases required. </li></ul><ul><li>It is very useful when the input/output domain is amenable to partitioning. </li></ul>Input Range (6,15) Test Values (4,9,17) Invalid Valid Range Invalid Less than 6 Between 6 and 15 More than 15 4 9 17
    219. 219. Low Level Techniques (1) - Equivalence Partitioning (Contd..) <ul><li>Here test cases are written to uncover classes of errors for every input condition. </li></ul><ul><li>Equivalence classes are:- </li></ul><ul><ul><li>Range </li></ul></ul><ul><ul><ul><li>Upper bound + 1 </li></ul></ul></ul><ul><ul><ul><li>Lower bound – 1 </li></ul></ul></ul><ul><ul><ul><li>Within bound </li></ul></ul></ul><ul><ul><li>Value </li></ul></ul><ul><ul><ul><li>Maximum length + 1 </li></ul></ul></ul><ul><ul><ul><li>Minimum length – 1 </li></ul></ul></ul><ul><ul><ul><li>Valid value and Valid length </li></ul></ul></ul><ul><ul><ul><li>Invalid value </li></ul></ul></ul><ul><ul><li>Set </li></ul></ul><ul><ul><ul><li>In-set </li></ul></ul></ul><ul><ul><ul><li>Out-of-set </li></ul></ul></ul><ul><ul><li>Boolean </li></ul></ul><ul><ul><ul><li>True </li></ul></ul></ul><ul><ul><ul><li>False </li></ul></ul></ul>
    220. 220. Low Level Techniques (1) - Equivalence Partitioning (Contd..) <ul><li>Equivalence Partitioning partitions the data to partition of a set. </li></ul><ul><li>Partition refers to collection of mutually disjoint subsets whose union is the entire set. </li></ul><ul><li>Choose one data element from each partitioned set. </li></ul><ul><li>The KEY is the choice of equivalence relation! </li></ul><ul><li>EC based testing allows </li></ul><ul><ul><li>To have a sense of complete testing. </li></ul></ul><ul><ul><li>Helps avoid redundancy. </li></ul></ul>
    221. 221. Low Level Techniques (2) - Boundary Value Analysis <ul><li>A Black Box Testing Method </li></ul><ul><li>Complements to Equivalence partition </li></ul><ul><li>BVA leads to a selection of test cases that exercise bounding values </li></ul><ul><li>Design test cases test </li></ul><ul><ul><li>Min values of an input </li></ul></ul><ul><ul><li>Max values of an input </li></ul></ul><ul><ul><li>Just above and below input range </li></ul></ul>Input Range (6,15) Test Values (5,6,7,15,16) 5 7 16 6 15 Less than 6 Between 6 and 15 More than 15
    222. 222. Low Level Techniques (2) - Boundary Value Analysis <ul><li>Helps to write test cases that exercise bounding values. </li></ul><ul><li>Complements Equivalence Partitioning. </li></ul><ul><li>Guidelines are similar to Equivalence Partitioning. </li></ul><ul><li>Two types of BVA: </li></ul><ul><ul><li>Range </li></ul></ul><ul><ul><ul><li>Above and below Range </li></ul></ul></ul><ul><ul><li>Value </li></ul></ul><ul><ul><ul><li>Above and below min and max number </li></ul></ul></ul>
    223. 223. Low Level Techniques (2) - Boundary Value Analysis <ul><li>Boundary Value Analysis </li></ul><ul><ul><li>Large number of errors tend to occur at boundaries of the input domain. </li></ul></ul><ul><ul><li>BVA leads to selection of test cases that exercise boundary values. </li></ul></ul><ul><ul><li>BVA complements Equivalence Partitioning. </li></ul></ul><ul><ul><li>Rather than select any element in an equivalence class, select those at the “edge” of the class. </li></ul></ul>
    224. 224. Low Level Techniques (2) - Boundary Value Analysis <ul><li>Examples : </li></ul><ul><ul><li>For a range of values bounded by ‘a’ and ‘b’, test (a-1), a, (a+1), (b-1), b, (b+1). </li></ul></ul><ul><ul><li>If input conditions specify a number of values ‘n’, test with (n-1), n and (n+1) input values. </li></ul></ul><ul><ul><li>Apply 1 and 2 to output conditions (e.g., generate table of minimum and maximum size). </li></ul></ul><ul><ul><li>If internal program data structures have boundaries (e.g., buffer size, table limits), use input data to exercise structures on boundaries. </li></ul></ul>
    225. 225. Low Level Techniques (2) - Boundary Value Analysis <ul><li>For Two Variables </li></ul><ul><ul><li>a < = x1 < = b </li></ul></ul><ul><ul><li>c < = x2 < = d </li></ul></ul><ul><li>For each variable </li></ul><ul><ul><li>Minimum -1 </li></ul></ul><ul><ul><li>Minimum </li></ul></ul><ul><ul><li>Minimum +1 </li></ul></ul><ul><ul><li>Nominal/mid </li></ul></ul><ul><ul><li>Maximum -1 </li></ul></ul><ul><ul><li>Maximum </li></ul></ul><ul><ul><li>Maximum +1 </li></ul></ul><ul><li>Take Cartesian product of these sets </li></ul>
    226. 226. Low Level Techniques (3) - Input/Output Domain Testing <ul><li>Description </li></ul><ul><ul><li>From input side generate inputs to map to outputs. </li></ul></ul><ul><ul><li>Ensure that you have generated all possible inputs by looking from the output side. </li></ul></ul>Inputs Outputs
    227. 227. Low Level Techniques (4) - Special Value Testing <ul><li>Select test data on the basis of features of a function to be computed. </li></ul><ul><li>Tester uses her / his domain knowledge, experience with similar programs. </li></ul><ul><li>Ad-hoc / seat-of-pants / skirt testing. </li></ul><ul><li>No guidelines, use best engineering judgment. </li></ul><ul><li>Special test cases / Error guessing. </li></ul><ul><li>Is useful – don’t discount effectiveness! </li></ul>
    228. 228. Low Level Techniques (5) - Error based Testing <ul><li>Generate test cases based on </li></ul><ul><ul><li>Programmer histories </li></ul></ul><ul><ul><li>Program complexity </li></ul></ul><ul><ul><li>Knowledge of error-prone syntactic constructs </li></ul></ul><ul><li>Guess errors based on data type </li></ul>
    229. 229. Low Level Techniques (6) - Cause Effect Graphing Techniques <ul><li>Cause Effect Graphing Techniques </li></ul><ul><ul><li>Translation of natural language descriptions of procedures to software based algorithms is error prone. </li></ul></ul><ul><li>Uncovers errors by representing algorithm as a cause-effect graph representing logical combinations and corresponding actions. </li></ul>Contd..2
    230. 230. Low Level Techniques (6) - Cause Effect Graphing Techniques <ul><li>Cause Effect Graphing Techniques </li></ul><ul><ul><li>How do you test code which attempts to implement this? </li></ul></ul><ul><ul><li>Cause-effect graphing attempts to provide a concise representation of logical combinations and corresponding actions. </li></ul></ul><ul><ul><li>Causes (input conditions) and effects (actions) are listed for a module and an identifier is assigned to each. </li></ul></ul><ul><li>Steps: </li></ul><ul><ul><li>A cause-effect graph developed. </li></ul></ul><ul><ul><li>Graph converted to a decision table. </li></ul></ul><ul><ul><li>Decision table rules are converted to test cases. </li></ul></ul>Contd..2
    231. 231. Low Level Techniques (7) - Comparison Testing <ul><li>Helps to check performance of the software under different hardware and software configurations. </li></ul><ul><li>Two variants of Comparison testing are: </li></ul><ul><ul><li>Develop the software. </li></ul></ul><ul><ul><li>Run the software in parallel and compare the results. </li></ul></ul>
    232. 232. Low Level Techniques (7)- Comparison Testing <ul><li>Comparison Testing </li></ul><ul><ul><li>In some applications, reliability is critical. </li></ul></ul><ul><ul><li>Redundant hardware and software may be used. </li></ul></ul><ul><ul><li>For redundant s/w, use separate teams to test the software. </li></ul></ul><ul><ul><li>Test with same test data to ensure all provide identical output. </li></ul></ul><ul><ul><li>Run the software in parallel with a real-time comparison of results. </li></ul></ul><ul><ul><li>Method does not catch errors in the specification. </li></ul></ul>
    233. 233. GUI Testing
    234. 234. Windows Compliance Standards <ul><li>Windows resize options </li></ul><ul><ul><li>Maximize, minimize and close options should be available. </li></ul></ul><ul><li>Using TAB </li></ul><ul><ul><li>Should move the focus (cursor) from left to right and top to bottom in the window. </li></ul></ul><ul><li>Using SHIFT+TAB </li></ul><ul><ul><li>Should move the focus (cursor) from right to left and bottom to top. </li></ul></ul><ul><li>Text </li></ul><ul><ul><li>Should be left-justified. </li></ul></ul>
    235. 235. Windows Compliance Standards (Contd..) <ul><li>Edit Box </li></ul><ul><ul><li>U should be able to enter data. </li></ul></ul><ul><ul><li>Try to overflow the text, text should be stopped after the specified length of characters. </li></ul></ul><ul><ul><li>Try entering invalid characters - should not allow. </li></ul></ul><ul><li>Radio Buttons </li></ul><ul><ul><li>Left and right arrows should move ‘ON’ selection. So should UP and DOWN. </li></ul></ul><ul><ul><li>Select with the mouse by clicking. </li></ul></ul><ul><li>Check Boxes </li></ul><ul><ul><li>Clicking with the mouse on the box or on the text should SET/UNSET the box. </li></ul></ul><ul><ul><li>Space should do the same. </li></ul></ul>
    236. 236. Windows Compliance Standards (Contd..) <ul><li>Command Buttons </li></ul><ul><ul><li>Should have shortcut keys (except OK and Cancel buttons). </li></ul></ul><ul><ul><li>Click each button with the mouse - should activate. </li></ul></ul><ul><ul><li>TAB to each button & press Space/Enter - should activate. </li></ul></ul><ul><li>Drop Down List </li></ul><ul><ul><li>Pressing the arrow should give list of options. </li></ul></ul><ul><ul><li>Pressing a letter should bring you to the first item in the list with that start letter. </li></ul></ul><ul><ul><li>Pressing Ctrl+F4 should open/drop down the list box. </li></ul></ul>
    237. 237. Windows Compliance Standards (Contd..) <ul><li>Combo Boxes </li></ul><ul><ul><li>Should allow text to be entered. </li></ul></ul><ul><ul><li>Clicking the arrow should allow user to choose from the list </li></ul></ul><ul><li>List Boxes </li></ul><ul><ul><li>Should allow a single selection to be chosen by clicking with the mouse or using the Up and Down arrows. </li></ul></ul><ul><ul><li>Pressing a letter should bring you to the first item in the list with that start letter. </li></ul></ul>
    238. 238. Screen Validation Standards <ul><li>Aesthetic Conditions </li></ul><ul><ul><li>The general screen background should be of correct colour (company standards,….). </li></ul></ul><ul><ul><li>The field prompts and backgrounds should be of correct colour. </li></ul></ul><ul><ul><li>The text in all the fields should be of the same font. </li></ul></ul><ul><ul><li>All the field prompts, group boxes and edit boxes should be aligned perfectly. </li></ul></ul><ul><ul><li>Microhelp should be available and spelt correctly. </li></ul></ul><ul><ul><li>All dialog boxes and windows should have a consistent look and feel. </li></ul></ul>
    239. 239. Screen Validation Standards (Contd..) <ul><li>Validation Conditions </li></ul><ul><ul><li>Failure of validation on every field should cause a user error message. </li></ul></ul><ul><ul><li>If any fields are having multile validation rules, all should be applied. </li></ul></ul><ul><ul><li>If the user enters an invalid value and clicks on the OK button, the invalid entry should be identified and highlighted. </li></ul></ul><ul><ul><li>In the numeric fields, negative numbers should be allowed to enter. </li></ul></ul><ul><ul><li>Should allow the minimum, maximum and mid range values in numeric fields. </li></ul></ul><ul><ul><li>All mandatory fields should require user input. </li></ul></ul>
    240. 240. Screen Validation Standards (Contd..) <ul><li>Navigation Conditions </li></ul><ul><ul><li>The screen should be accessible correctly from the menu and toolbar. </li></ul></ul><ul><ul><li>All screens accessible through buttons on this screen should be accessed correctly. </li></ul></ul><ul><ul><li>The user should not be prevented from accessing other functions when this screen is active. </li></ul></ul><ul><ul><li>Should not allow to open number of instances of the same screen at the same time. </li></ul></ul>
    241. 241. Screen Validation Standards (Contd..) <ul><li>Usability Conditions </li></ul><ul><ul><li>All the dropdowns should be sorted alphabetically (unless specified). </li></ul></ul><ul><ul><li>All pushbuttons should have appropriate shortcut keys and should work properly. </li></ul></ul><ul><ul><li>All read-only and disabled fields should be avoided in the TAB sequence. </li></ul></ul><ul><ul><li>Should not allow to edit microhelp text. </li></ul></ul><ul><ul><li>The cursor should be positioned in the first input field or control when opened. </li></ul></ul><ul><ul><li>When an error message occurs, the focus should return to the field in error after cancelling it. </li></ul></ul><ul><ul><li>Alt+Tab should not have any impact on the screen upon return. </li></ul></ul>
    242. 242. Screen Validation Standards (Contd..) <ul><li>Data Integrity Conditions </li></ul><ul><ul><li>The data should be saved when the window is closed by double clicking on the close box. </li></ul></ul><ul><ul><li>There characters should not be truncated. </li></ul></ul><ul><ul><li>Maximum and minimum field values for numeric fields should be verified. </li></ul></ul><ul><ul><li>Negative values should be stored and accessed from the database correctly. </li></ul></ul>
    243. 243. Screen Validation Standards (Contd..) <ul><li>Modes (Editable, Read-only) conditions </li></ul><ul><ul><li>The screen and field colours should be adjusted correctly for read-only mode. </li></ul></ul><ul><ul><li>Is the read only field necessary for this screen? </li></ul></ul><ul><ul><li>All fields and controls should be disabled in read-only mode. </li></ul></ul><ul><ul><li>No validation is performed in read-only mode. </li></ul></ul>
    244. 244. Screen Validation Standards (Contd..) <ul><li>General Conditions </li></ul><ul><ul><li>“ Help” menu should exist. </li></ul></ul><ul><ul><li>All buttons on all tool bars should have corresponding key commands. </li></ul></ul><ul><ul><li>Abbreviations should not be used in drop down lists. </li></ul></ul><ul><ul><li>Duplicate hot keys/shortcut keys should not exist. </li></ul></ul><ul><ul><li>Escape key and cancel button should cancel (close) the application. </li></ul></ul><ul><ul><li>OK and Cancel buttons should be grouped separately. </li></ul></ul><ul><ul><li>Command button names should not be abbreviations. </li></ul></ul>
    245. 245. Screen Validation Standards (Contd..) <ul><li>General Conditions (Contd..) </li></ul><ul><ul><li>Field labels/names should not be technical labels, they should be meaningful to system users. </li></ul></ul><ul><ul><li>All command buttons should be of similar size, shape, font and font size. </li></ul></ul><ul><ul><li>Option boxes, option buttons and command buttons should be logically grouped. </li></ul></ul><ul><ul><li>Mouse action should be consistent through out the screen. </li></ul></ul><ul><ul><li>Red colour should not be used to highlight active objects (many individuals are red-green colour blind). </li></ul></ul><ul><ul><li>Screen/Window should not have cluttered appearance. </li></ul></ul><ul><ul><li>Alt+F4 should close the window/application. </li></ul></ul>
    246. 246. Bug Life Cycle
    247. 247. What is a Bug? <ul><li>Bug </li></ul><ul><ul><li>A fault in a program which causes the program to perform in an unintended or unanticipated manner or deviation from the requirement specification or the design specification is referred as a bug. </li></ul></ul>
    248. 248. What is a Bug Life Cycle? No Yes Submitted In-Work Solved Validated Terminated Deferred
    249. 249. Classification of Bugs <ul><li>Two attributes are used whenever a Bug/Defect is detected </li></ul><ul><li>Severity ( Severity is Technical) </li></ul><ul><ul><li>Critical </li></ul></ul><ul><ul><li>Serious </li></ul></ul><ul><ul><li>Minor </li></ul></ul><ul><li>Priority ( Priority is Business) </li></ul><ul><ul><li>High </li></ul></ul><ul><ul><li>Medium </li></ul></ul><ul><ul><li>Low </li></ul></ul>
    250. 250. Reporting/Logging a Bug/Defect <ul><li>A Bug/Defect is reported with the following details </li></ul><ul><ul><li>Summary </li></ul></ul><ul><ul><li>Description </li></ul></ul><ul><ul><li>How to reproduce </li></ul></ul><ul><ul><li>Version </li></ul></ul><ul><ul><li>Module </li></ul></ul><ul><ul><li>Phase </li></ul></ul><ul><ul><li>Browser </li></ul></ul><ul><ul><li>Environment </li></ul></ul><ul><ul><li>Modified Date </li></ul></ul>
    251. 251. Reporting/Logging a Bug/Defect (Contd..) <ul><li>A Bug/Defect is reported with the following details </li></ul><ul><ul><li>Job assigned to </li></ul></ul><ul><ul><li>Severity </li></ul></ul><ul><ul><li>Priority </li></ul></ul><ul><ul><li>Tester’s name </li></ul></ul><ul><ul><li>Status </li></ul></ul><ul><ul><li>Database </li></ul></ul><ul><ul><li>Type of defect </li></ul></ul><ul><ul><li>Reproducible </li></ul></ul><ul><ul><li>Attachments </li></ul></ul>