Uploaded byAnirudh503501
PPTX, PDF54 views

Software testing foundation

The document provides an overview of software testing fundamentals including: 1. It discusses key testing concepts like error, fault, failure and how testing helps build confidence and reduce costs. Testing aims to find faults and prove software meets requirements. 2. Testing challenges are discussed like the impossibility of exhaustive testing due to huge number of combinations. Prioritization is important given limited time. 3. Principles of testing are covered such as defects clustering, absence of errors fallacy, and how early testing avoids fault multiplication. Testing must be context dependent.

Embed presentation

Download to read offline
1 / 118
2 / 118
3 / 118
4 / 118
5 / 118
6 / 118
7 / 118
8 / 118
9 / 118
10 / 118
11 / 118
12 / 118
13 / 118
14 / 118
15 / 118
16 / 118
17 / 118
18 / 118
19 / 118
20 / 118
21 / 118
22 / 118
23 / 118
24 / 118
25 / 118
26 / 118
27 / 118
28 / 118
29 / 118
30 / 118
31 / 118
32 / 118
33 / 118
34 / 118
35 / 118
36 / 118
37 / 118
38 / 118
39 / 118
40 / 118
41 / 118
42 / 118
43 / 118
44 / 118
45 / 118
46 / 118
47 / 118
48 / 118
49 / 118
50 / 118
51 / 118
52 / 118
53 / 118
54 / 118
55 / 118
56 / 118
57 / 118
58 / 118
59 / 118
60 / 118
61 / 118
62 / 118
63 / 118
64 / 118
65 / 118
66 / 118
67 / 118
68 / 118
69 / 118
70 / 118
71 / 118
72 / 118
73 / 118
74 / 118
75 / 118
76 / 118
77 / 118
78 / 118
79 / 118
80 / 118
81 / 118
82 / 118
83 / 118
84 / 118
85 / 118
86 / 118
87 / 118
88 / 118
89 / 118
90 / 118
91 / 118
92 / 118
93 / 118
94 / 118
95 / 118
96 / 118
97 / 118
98 / 118
99 / 118
100 / 118
101 / 118
102 / 118
103 / 118
104 / 118
105 / 118
106 / 118
107 / 118
108 / 118
109 / 118
110 / 118
111 / 118
112 / 118
113 / 118
114 / 118
115 / 118
116 / 118
117 / 118
118 / 118
Software Testing Foundation Module: Fundamentals of Testing
2 Triangle Exercise Brief History of Testing Testing Challenges Topics Covered What is Testing Error – Fault - Failure Why Testing How much Testing Principles of Testing Psychology of Testing Fundamental Test Process
3 Triangle Exercise
4 Let’s Start Testing “A program reads three (3) integer values. The three values are interpreted as representing the lengths of the sides of a triangle. The program displays a message that states whether the triangle is scalene, isosceles, or equilateral.” – G. Myers Requirement for a Triangle Program Sample Test Case TC# Input Expected Output 1 3,3,3 Equilateral 2 3,3,2 Isosceles 3 3,4,5 Scalene 4 3,4,% Invalid Black-Box Test Basis Is Testing Easy or Difficult Triangle Exercise
5 Pseudo Code for the requirement if (Side A) or (Side B) or (Side C) != int16 (signed 16 bit int) if input > maxint then error message for overflow condition else if input ! = integer value then error message for format exception if Side A or Side B or Side C <= 0 then error message sides must be > 0 if (Side A + Side B <= Side C) or (Side B + Side C <= Side A) or (Side A + Side C <= Side C) then input does not equate to valid triangle if (Side A = = Side B) and (Side B = = Side C) then triangle is equilateral else if (Side A = = Side B) or (Side A = = Side C) or (Side B = = Side C) then triangle is isosceles else triangle is scalene *Aside from the obvious conditions above additional tests include 1 for max val for all 3 sides, 1 for min val for all 3 sides, and 3 to test for BO of sum of 2 sides White Box Test Basis Triangle Exercise
6 Triangle Testing Exercise Assignment 1 Triangle Exercise
7 Brief History of Testing
8 Brief History of Testing Definitions of Testing Over the Years 1979 “Testing is the process of executing a program or system with the intent of finding errors” 2002 “Testing is a concurrent lifecycle process of engineering, using and maintaining testware in order to measure and improve the quality of the software being tested” “Testing is any activity aimed at evaluating an attribute of a program or system. Testing is the measurement of software quality” 1983
9 Brief History of Testing Development and Testing Evolution 1960 1980 Today Software Size Small Moderate Very Large Software Complexity Low Medium High Development Teams Small Medium Large Methods & Standards Ad-hoc Moderate Sophisticated Test Methods & Standards Ad-hoc Primitive Established Independent Test Organizations Few Some Many Recognition of Test Importance Little Some Significant Testing Professionals Few Few Many
10 Testing Challenges
11 Testing Challenges Too many combinations of inputs to test Not enough time to test properly Difficulty in determining the expected result of each test Nonexistent or rapidly changing requirements
12 Testing Challenges The design of tests for software and other engineering products can be as challenging as the initial design of the product itself. Yet....software engineers often treat testing as an afterthought, developing test cases that ‘feel right’ but have little assurance of being complete. Recalling the objectives of testing, we must design tests that have the highest likelihood of finding the most errors with a minimum amount of time and effort.
13 Testing Challenges Some Food for Thoughts Int blech (int J) { j = j – 1; //Should be j = j + 1 j = j / 30000; return j; } Input Expected Output Actual Output 1 0 0 42 0 0 40000 1 1 -64000 -2 -2 Oops! None of the test cases chosen have detected this defect
14 Testing is a concurrent lifecycle process of engineering to measure and improve the quality of software being tested. Independent Testing has gained significant momentum. Testing is not easy as it appears—It requires systematic approach and a creative mind. Recap Testing Challenges
15 Value of Testing by Lee Copeland Testing Challenges
16 Write a two page summary of your understainding about the video, Value of Testing by Lee Copeland. Assignment 2 Testing Challenges
17 What is Testing
18 Read the case study on Explosion of the Ariane and answer the questions in the given handout. What is Testing
19 Read the case study on AT&T Failure and answer the questions in the given handout. What is Testing
20 What is Testing Software Testing is a process of evaluating a system Dynamic Testing Measure Quality Static Testing
21 What is Testing By manual or automatic means Verify that the system satisfies the specified requirements Identify differences between expected and actual results Dynamic Testing
22 What is Testing Testing results in collection of lot of useful data and information about quality of the software. This information will help organizations to make informed decisions about releasing the software. Measure Quality
23 What is Testing Testing includes inspection and structured peer reviews of requirements and design, as well as code. Static Testing
24 Error, Fault, Failure
25 Error, Fault and Failure What is the problem here? $temperature = $temperature + $input if $tempereture > 100 then Print “Reactor Core Temperature Critical !!!” Else Print “Reactor Core Temperature Normal”
26 Error, Fault and Failure • A manifestation of an error in software • Also known as a defect or bug • If executed, a fault may cause a failure Failure ERROR A human action that produces an incorrect result Fault Deviation of the software from its expected delivery or service Failure is an event, fault is a state of the software, caused by an error
27 Error, Fault and Failure A person makes an error…. ….that creates a fault in software… ….that can cause failure in operation. Script
28 Error, Reliability VS Faults Reliability: The probability that software will not cause the failure of the system for a specified time under specified conditions. Reliability VS Faults Can a system be fault-free? (zero faults, right first time) Can a software system be reliable but still have faults? Is a “fault-free” software application always reliable?
29 who have skills, but aren’t perfect Error, Reliability VS Faults Why do faults occur in software? Software is written by human beings who know something, but not everything no time to check but assumptions may be wrong who do make mistakes (errors) systems may be incomplete Software is written by human beings Under increasing pressure due to strict deadlines
30 Error, Reliability VS Faults If you can say it, you can do it. Software is written by human beings
31 Very little or nothing at all Faults can cause death or injury Error, Reliability VS Faults What do software faults cost? Huge sums • Ariane 5 ($7billion) • American Airlines ($50m) • Minor inconvenience • Small input may have very large effect • Radiation treatment machine kills • Bank overdraft letters cause suicide Software is not “linear”
32 Why Testing
33 To fill the time between delivery of the software and the release date Why Testing So why is testing necessary? Because software is likely to have faults To learn about the reliability of the software To prove that the software has no faults Because testing is included in the project plan Because failures can be very expensive To avoid being sued by customers To stay in business
34 Why Testing Why not just “Test Everything” 232 232 + 264 Inputs Program
35 Why Testing Why not just “Test Everything” System has 20 screens Total for 'exhaustive' testing: 20 x 4 x 3 x 10 x 2 x 100 = 480,000 tests If 1 second per test, 8000 mins, 133 hrs, 17.7 days (not counting finger trouble, faults or retest) 10 secs = 34 wks, 1 min = 4 yrs, 10 min = 40 yrs Avr. 4 menus 3 options / menu Average: 10 fields / screen 2 types input / field (date as Jan 3 or 3/1) (number as integer or decimal) Around 100 possible values
36 Why Testing When all the testers are exhausted What is exhaustive testing? How much time will exhaustive testing take? When all the planned tests have been executed Exercising all combinations of inputs and preconditions Infinite time Not much time Impractical amount of time
37 How Much Testing
38 When your customer/user is happy How Much Testing How much testing is enough? It’s never enough When you have done what you planned When you have proved that the system works correctly When you are confident that the system works correctly It depends on the risks for your system
39 How Much Testing Testing depends on? Risk of missing important faults Risk of incurring failure costs Risk of releasing untested or under-tested software Risk of losing credibility and market share Risk of missing a market window Risk of over-testing, ineffective testing
40 How Much Testing What to test first So much to test and so little time…. Time will always be limited use RISK to determine: What to test most How thoroughly to test each item What not to test (this time)
41 How Much Testing Prioritize tests so that, whenever you stop testing, you have done the best testing in the time available. Most important principle Important
42 How Much Testing What is an optimal test effort?
43 How Much Testing Prioritize testing tasks before going for testing: Time is Available Time is Limited Perform all test cases Test all relevant areas Test only high-risk areas Perform high-priority test, first
44 When we talk about testing we are talking about dynamic testing, Static testing and measuring quality. Terms like Error, Fault and Failure are used in a synonymous way - But they have subtle differences. Defects are very expensive - Some times may cause death. Recap How Much Testing Testing is software risk management activity. “How much testing?” - Depends on RISK.
45 Principles of Testing
46 Research and write a reoprt about recent spectacular software failure. Assignment 3 Principles of Testing
47 Read the case study on Necessity for Regression Testing and answer the questions in the given handout. Principles of Testing
48 Principles of Testing Testing shows presence of defects Defect Clustering Exhaustive Testing is impossible The Pesticide Paradox Early Testing avoids fault multiplication Testing is Context dependent Absence-of-errors fallacy
49 Psychology of Testing
50 Read the case study on Independence to Testing Team and answer the questions in the given handout. Psychology of Testing
51 Read the case study on Asking Questions and answer the questions in the given handout. Psychology of Testing
52 Demonstrate conformance to requirements Psychology of Testing Why test? Build confidence Prove that the software is correct Find faults Reduce costs Show system meets user needs Assess the software quality
53 Psychology of Testing Confidence Time Confidence No faults found = Confidence? Fault Found
54 Few Faults Few Faults Psychology of Testing Assessing software quality You may be here High Low Test Quality High Low Software Quality Many Faults Few Faults You think you are here
55 Psychology of Testing Traditional approach of testing Show that system: does what it should doesn’t do what it shouldn’t Goal: Show Working Success: System Works Fastest Achievement: Easy Test Case Result : Faults left in
56 Psychology of Testing Better testing approach Show that system: does what it shouldn’t doesn’t do what it should Goal: Find Faults Success: System Fails Fastest Achievement: Difficult Test Case Result : Few Faults left in
57 Psychology of Testing Tests to Pass and Tests to Fail
58 Psychology of Testing The testing paradox Purpose of Testing To Find Faults Finding Faults Destroys Confidence Purpose of Testing Destroys Confidence Purpose of Testing Build Confidence The best way to Build Confidence is to destroy it.
59 Psychology of Testing Testing is not a glorious job, because: Testing is a destructive process It brings bad news (“your baby is ugly”) It keeps you under worst time pressure (at the end) You need to take a different view, a different mindset (What if it isn’t?”, “What could go wrong?”) You should know how to communicate the fault information to authors and managers.
60 Psychology of Testing Developer VS Tester What do you think the relationship between a tester and developer should be like: Both are friendly Both competing with each other Both fighting with each other
61 A good tester internalizes the testing principles. Testing software requires a different mind set than that is required to develop software. Recap Psychology of Testing
62 Knowledge Check A. The evaluation done by testers is not influenced by personal experience. What is true about software testers? B. Experienced testers always guarantee high software quality. C. Testers need good communication skills. D. Testers and programmers must always have opposite mindsets.
63 Knowledge Check A. Test leader writes a summary report. Which of the following statements is an example of testing contributing to higher quality? B. Project manager asks a test leader to estimate the test effort. C. Tester installs a test item in the test environment. D. Tester finds a bug which is resolved prior to release.
64 Knowledge Check A. Failure is a cause of a defect. Which of the following statements is correct? B. Failure is a design error in the product. C. Failure is the deviation of the program from its expected behavior. D. Failure is an error in the code.
65 Knowledge Check A. To guarantee the protection against unauthorized access. Which of the following is a valuable objective for software testing? B. To fulfill the requirements for testing specified in the project handbook. C. To gain confidence about the level of quality and provide information about the test object. D. To prove the correctness of a specific feature.
66 Which of the following statements are “testing general principles”? I. Exhaustive testing is impossible. II. The defects found during the pre-release tests, the operational failures, are uniformly distributed across the system’s software modules. III. Testing can show the presence of defects, but can not demonstrate their absence. IV. Testing is context-independent. Knowledge Check A. I and III B. I and II C. II and III D. I and IV
67 Knowledge Check A. Properly designed tests that pass reduce the level of risk in a system Which one of the following statements does not describe how testing contributes to higher quality? B. Performing a review of the requirement specifications before implementing the system can enhance quality. C. Software testing identifies defects, which can be used to improve development activities D. The testing of software demonstrates the absence of defects.
68 Knowledge Check A. Prepare test data and execute test sets. Typical entry-level tester tasks may include the following: B. Decide what to automate and prepare test automation scripts. C. Select and implement testing tools. D. Select test approach and design test cases
69 Which of the following statements are true? I. Defects in a program do not always cause failures. II. A defect in a program may cause more than one failure during execution. III. If there is a fault in the program, at least one test will fail. IV. After fixing a bug, the program may still fail under test. V. Errors in the program lead to side effects Knowledge Check A. II and III are true. The others are false B. I and V are true. The others are false C. I and IV are true. The others are false D. I, II and IV are true. The others are false
70 Which of the following are valid testing principles? I. Exhaustive testing is in general impossible. II. Exhaustive testing should be executed for code intended to be reused. III. Testing may guarantee that a program is correct. IV. Testing can not guarantee that a program is correct . V. Defects cluster together in certain areas of the product. Knowledge Check A. I and V B. I, IV and V C. I and III D. II and IV Acceptance Test
71 Knowledge Check A. Life-cycle model Which of the following is NOT a deciding factor in determining the extent of testing required? B. Level of risk C. Time D. Budget
72 Knowledge Check A. Functional Test Which of the following is NOT used in system testing? B. Structure-Based Test C. Non-Functional Test D. Unit Test
73 Knowledge Check You are the test engineer responsible for the system testing of a digital Set-top box. The three main functions of the product are: ‘contents download’, ‘contents play 'and ‘contents search’. The likelihood and impact of failure for each function are shown in the graph below. Down Load Search Play Number of test cases Down Load Play Search Impact Likelihood As shown in the graph the number of test cases created for each function: Which one of the following statements is most likely to be correct? A. More test cases need to be created for the download function B. You should discard test cases for the play function C. The result of the risk analysis do not agree with the number of test cases D. The number of test cases generated aligns with the perceived risk
74 Which of the following statements are correct? I. System testing is often carried out by an independent test team. II. The main goal in acceptance testing is to find defects. III. Component testing may include testing of functionality and non- functional characteristics. IV. UAT typically verifies the fitness for use of the system by business users. V. The environment for system testing should correspond to the production environment. Knowledge Check A. II, III and V B. I, III, IV and V C. II, III and IV D. I, II and V
75 Knowledge Check A. If the same tests are repeated over and over again, eventually the same set of test cases Will no longer find any bugs. Which of the following describes the “pesticide paradox” principle? B. Testing every thing (all combinations of inputs and pre-conditions) is not feasible, except for trivial cases. C. A small number of modules contain most of the defects discovered during pre-release testing, or show most of the operational failures. D. Testing can show that defects are present, but cannot prove that there are no defects
76 Knowledge Check A. The test engineer should not re-run the tests, as they have already been run, and results recorded. A test engineer finds a defect while testing. After the developer has fixed the defect, a test engineer decides to re-run a complete section of the tests. Which of the following is correct? B. The test engineer should re-run the tests, in order to ensure that new defects have not been introduced by the fix. C. The test engineer should re-run the tests, because the defect shows that the test cases need to be updated. D. The test engineer should not re-run the tests, they should be part of developer tests.
77 Knowledge Check A. Testing that the interfaces work correctly. What is integration testing? B. Looking for faults in larger components or subsystems. C. Specifying which components to integrate in which order. D. Another term for specifying for testing system integrity.
78 Knowledge Check A. Performance Test Which of the following is not a test level according to the V-model? B. Unit Test C. System Test D. Acceptance Test
79 Which of the following are valid testing principles? I. Exhaustive testing is in general impossible. II. Exhaustive testing should be executed for code intended to be reused. III. Testing may guarantee that a program is correct. IV. Testing can not guarantee that a program is correct. V. Defects cluster together in certain areas of the product. Knowledge Check A. I and V B. I, IV and V C. I and III D. II and IV
80 Fundamental Test Process The following is the fundamental test process flow: Test Planning and Control Test Implementation and Execution Test Closer Activities Yes Evaluating Exit Criteria and Reporting No Test Analysis and Design
81 Read the case study on How to Plan Tests and answer the questions in the given handout. Fundamental Test Process Test Planning and Control
82 Test Planning Exercise Test Planning and Control GROUP EXERCISE Test Planning and Control
83 Test Analysis and Design Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 Reviewing the test basis Evaluating Testability Identify Test Conditions Design Test Cases Set up Test Data Set up Test Environment Create Traceability Matrix
84 Feasibility Testability Completeness Correctness Necessity Prioritization Test Analysis and Design Step 1 Reviewing the test basis Traceability Unambiguous Consistency Criteria for Verification of Requirements
85 Test Analysis and Design ___________ of a requirement is judged based on what the user wants. For example, are the rules and regulations stated correctly? Does the requirement exactly reflect the user’s request? It is imperative that the end user, or a suitable representative, be involved during the requirements phase. This criterion can also be judged based on standards. Are the standards being followed? Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Correctness Correctness
86 Test Analysis and Design _____________ ensures that no necessary elements are missing from the requirement. The goal is to avoid omitting requirements simply because no one has asked the right questions or examined all of the pertinent source documents. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Completeness Completeness
87 Test Analysis and Design ___________ verifies that there are no internal or external contradictions among the elements within the work products, or between work products. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Consistency Consistency
88 Test Analysis and Design __________ of the requirement confirms that it is possible to create a test for the requirement, and that an expected result is known and can be programmatically or visually verified. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Testability Testability
89 Feasibility Test Analysis and Design _________ of a requirement ensures that it can be implemented given the budget, schedules, technology and other resources available. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Feasibility
90 Test Analysis and Design _________ verifies that every requirement in the specification is pertinent to the system. Does this requirement contribute to those goals? Would excluding this requirement prevent the system from meeting those goals? Are any other requirements dependent on this requirement? Some irrelevant requirements are not really requirements, but proposed solutions. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Completeness Necessity Feasibility Necessity
91 Test Analysis and Design ____________ verifies that every requirement in the specification is pertinent to the system. Does this requirement contribute to those goals? Would excluding this requirement prevent the system from meeting those goals? Are any other requirements dependent on this requirement? Some irrelevant requirements are not really requirements, but proposed solutions. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Completeness Prioritization Feasibility Prioritization
92 Test Analysis and Design ____________ ensures that requirements are stated in a precise and measurable way. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Completeness Unambiguous Feasibility Unambiguous
93 Test Analysis and Design ___________ ensures that each requirement is identified in such a way that it can be associated with all parts of the system where it is used. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Completeness Traceability Feasibility Traceability
94 Test Analysis and Design Step 2 Evaluating Testability Ensure that “expected results” can be derived from the requirements. Results can be verified programmatically or visually. Testability is the degree to which a software artifact facilitates testing in a given test context.
95 Test Analysis and Design Step 3 Identify Test Conditions Test conditions define the tests to be performed and should always match the development stages they relate to, alongside the requirements and designs they intend to prove. A test condition is defined as an item or event that could be verified by one or more test cases (e.g. a function, transaction, quality characteristic or structural element).
96 Test Conditions Exercise Test Analysis and Design GROUP EXERCISE
97 Test Analysis and Design TCER: TEST CONDITIONS AND EXPECTED RESULT Name and description of the test condition Begin these with an action verb: create, print, etc. Example: “Create invoice.” Every test condition is derived from one requirement or design item. The requirement or design item should be stated for traceability. For each test condition, define a corresponding expected result. Example: “An invoice is created.” Expected Result Requirement or Design Source
98 Test Analysis and Design TCER Document Preparation Example The ATM system shall dispense denominations of 100, 500, and 1000 Rupees only. If amounts entered have a fraction of a Rupee or amounts entered cannot be dispensed with denominations of 100, 500, or 1000, then an error message will appear informing the user that the amount entered is invalid. The ATM system will have a maximum withdrawal of Rs 8000 per transaction. If the amount entered to be withdrawn is more than Rs 8000, a message will be displayed informing the user that only a maximum of Rs 8000 per withdrawal is allowed. The ATM system will have a minimum withdrawal of Rs 500 per transaction. If the amount entered or to be withdrawn is less than Rs 500, a message will be displayed informing the user that only a minimum of Rs 500 withdrawal is allowed. ATM Withdrawal Requirements
99 3 2 Min withdrawal amount -Valid The amount is accepted and the user is taken to the next step of the withdrawal transaction. Error message “Invalid amount…this ATM machine can only dispense bills with Denominations 100, 500 and 1000” appears on the ATM screen. 2 1 The amount entered in the ATM is not divisible by 100, 500,1000 The amount is accepted and the user is taken to the next step of the withdrawal transaction. The amount entered in the ATM is divisible by 100, 500, 1000 1 1 Test Analysis and Design TCER Document Preparation Example Requirement Number Cond# Test Condition Expected Result 4 2 Min withdrawal amount -Invalid Error message “Invalid amount…You may only withdraw a minimum of 500 per transaction.” appears on the ATM screen. 5 3 Max withdrawal amount -Valid The amount is accepted and the user is taken to the next step of the withdrawal transaction. 6 3 Max withdrawal amount -Invalid Error message “Invalid amount…You may only withdraw up to P8000 per transaction.” appears on the ATM screen. The ATM system shall dispense denominations of 100, 500, and 1000 Rupees only. If amounts entered have a fraction of a Rupee or amounts entered cannot be dispensed with denominations of 100, 500, or 1000, then an error message will appear informing the user that the amount entered is invalid. The ATM system will have a minimum withdrawal of Rs 500 per transaction. If the amount entered or to be withdrawn is less than Rs 500, a message will be displayed informing the user that only a minimum of Rs 500 withdrawal is allowed. The ATM system will have a maximum withdrawal of Rs 8000 per transaction. If the amount entered to be withdrawn is more than Rs 8000, a message will be displayed informing the user that only a maximum of Rs 8000 per withdrawal is allowed.
10 Test Case Design is a very deep thinking activity and in order to think well we need “Test Design Techniques” Test Analysis and Design Step 4 Design Test Case
10 Test Analysis and Design Step 5 & 6 Setup Test Data and Environment + Setting up of Test Data and Test Environment suggests identifying necessary test data to support the test conditions and test cases. Once data is set, design the test environment set up and identify any required infrastructure and tools.
10 Test Analysis and Design Step 7 Creating Traceability Matrix In the software development life cycle, requirements are translated into specifications based on which code is developed. Once the construction process is complete, the product is put through various stages of testing before it finally goes live. Requirements tracing is the process of documenting the links between the user requirements for the system and the work products developed to implement and verify those requirements. The work products include software requirements, design specifications, software code, test plans and other artifacts of the systems development process. Requirements tracing helps the project team to understand which parts of the design and code implement the user’s requirements, and which tests are necessary to verify the correct implementation of user’s requirements.
10 Test Analysis and Design General format of Traceability Matrix: Requirements: R 1.1 R 1.2 R 1.3 Scenarios: S1.1 S1.2 S1.3 S2.1 S2.2 S3.1 S3.2 S3.3 Test Cases TC1 X X TC2 X X X TC3 X X TC4 X X X TC5 X X X TC6 TC7 X TC8 X X X TC9 X X
10 Test Analysis and Design Which requirements are not covered fully in the following traceability matrix: Requirements: R 1.1 R 1.2 R 1.3 Scenarios: S1.1 S1.2 S1.3 S2.1 S2.2 S3.1 S3.2 S3.3 Test Cases TC1 X X TC2 X X X TC3 X TC4 X X TC5 X X X TC6 TC7 TC8 X X TC9 X R1.1 and R1.3 are not covered fully.
10 Test Analysis and Design Changes were made to the Requirements R 1.2. Based on this, which test cases need to be changed? TC4, TC5, TC6, TC7 and TC8 need to be changed Requirements: R 1.1 R 1.2 R 1.3 Scenarios: S1.1 S1.2 S1.3 S2.1 S2.2 S3.1 S3.2 S3.3 Test Cases TC1 X X TC2 X TC3 X TC4 X TC5 X TC6 X TC7 X X TC8 X X TC9 X X
10 Test Analysis and Design Exercise Creating Traceability Matrix
10 Knowledge Check A. The degree to which a requirement is stated in terms that permit establishment of test design. Which of the following statements correctly describes traceability? B. The capability of software product to enable modified software to be tested. C. An approach to integration testing where the component at the top of the component hierarchy is tested first. D. The ability to identify related items in documentation, such as requirements vs. software design.
10 1. Developing, implementing and prioritizing test cases. 2. Developing and prioritizing test procedures, creating test data and optionally preparing test harnesses, and writing automated test scripts. 3. Creating test suites from the test procedures for efficient test execution. 4. Verifying that the test environment has been set up correctly. 5. Executing test procedures either manually or by using test execution tools, according to the planned sequence. 6. Logging the outcome of test execution and recording the identities and versions of the software under test, test tools and testware. 7. Comparing the actual results with the expected results. 8. Reporting the discrepancies as incidents and analyzing them in order to establish their cause. 9. Repeating the test activities as a result of action taken for each discrepancy. Re- testing/Confirmation testing and Regression Testing. Test Implementation and Execution The major tasks in Test Implementation and execution are:
10 1. Checking test logs against the exit criteria specified in test planning. 2. Assessing, if more tests are needed or if the exit criteria specified should be changed. 3. Writing a test summary report for stakeholders. Evaluating Exit Criteria and Reporting The major tasks in Evaluating Exit Criteria and Reporting are:
11 1. Checking which planned deliverables have been delivered, the closure of incident reports or raising of change records for any that remain open, and the documentation of the acceptance of the system. 2. Finalizing and archiving testware, the test environment and the test infrastructure for later reuse. 3. Handover of testware to the maintenance organization. 4. Analyzing lessons learned for future releases and projects, and the improvement of test maturity. Test Closure Activity The major tasks in Evaluating Exit Criteria and Reporting are: Test closure activities collect data from completed test activities to consolidate experience, testware, facts and numbers.
11 Knowledge Check A. The continuous comparison between current progress and the test plan, in order to identify deviations. In the fundamental test process, what does primarily entail the “test analysis and design” activity? B. The testability evaluation of the test basis and test objects and the root-cause analysis. C. The data collection on the completed test activities, in order to consolidate experience, testware, facts and figures. D. The transformation of the test objectives in testing conditions and tangible test cases
11 Knowledge Check A. Experience-based testing should be applied after systematic test methods have been used. Which of the following statement is true? B. White-box testing is equally useful for low-level and high-level testing. C. Black-box testing is equivalent to equivalence partitioning. D. Experience-based testing is best used in integration testing.
11 Knowledge Check A. Planning, analysis and reporting, design and implementation, execution, test closure activities, evaluating exit criteria. Which of the following lists represents the correct sequence of the main activities of the fundamental test process (leaving out the activity of control which should take place in parallel to all the other activities)? B. Planning, analysis and design, implementation and execution, evaluation exit criteria and reporting, test closure activities C. Planning, analysis and design, execution, logging and reporting, regression testing D. Planning, analysis, design and implementation, execution, logging, test closure activities, evaluating exit criteria.
11 Knowledge Check A. Test implementation and execution At which stage of the fundamental test process do testers write the steps of the test procedures? B. Evaluating exit criteria and reporting C. Test closure activates D. Test planning and control
11 Knowledge Check A. Test Control At what stage of Fundamental Test Process do testers evaluate the Testability of the test basis? B. Test Execution C. Evaluating exit criteria and reporting D. Test Analysis and design
11 Knowledge Check A. Test Control At what stage of Fundamental Test Process do testers evaluate the Testability of the test basis? B. Test Execution C. Evaluating exit criteria and reporting D. Test Analysis and design
11 Read Appendex-1 Exercised based on a Business Scenario. Assignment 4 Exercise
11 Question and Answer

More Related Content

PPT
Software Testing- Principles of testing- Mazenet Solution
byMazenetsolution
 
PPT
Testing- Fundamentals of Testing-Mazenet solution
byMazenetsolution
 
PPT
ISTQB, ISEB Lecture Notes
byonsoftwaretest
 
PPT
ISTQB / ISEB Foundation Exam Practice -1
byYogindernath Gupta
 
PPT
1
bynaveennarnaul
 
PDF
How to Actually DO High-volume Automated Testing
byTechWell
 
PPT
&lt;p>Software Testing&lt;/p>
byAtul Mishra
 
PDF
AI-Driven Software Quality Assurance in the Age of DevOps
byChakkrit (Kla) Tantithamthavorn
 
Software Testing- Principles of testing- Mazenet Solution
byMazenetsolution
 
Testing- Fundamentals of Testing-Mazenet solution
byMazenetsolution
 
ISTQB, ISEB Lecture Notes
byonsoftwaretest
 
ISTQB / ISEB Foundation Exam Practice -1
byYogindernath Gupta
 
1
bynaveennarnaul
 
How to Actually DO High-volume Automated Testing
byTechWell
 
&lt;p>Software Testing&lt;/p>
byAtul Mishra
 
AI-Driven Software Quality Assurance in the Age of DevOps
byChakkrit (Kla) Tantithamthavorn
 

What's hot

PPTX
SOFTWARE TESTING UNIT-4
byMohammad Faizan
 
PDF
SBST 2019 Keynote
byShiva Nejati
 
PDF
Exploratory Testing
bynazeer pasha
 
PPTX
Software Testing & Debugging
byComputing Cage
 
PDF
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
byChakkrit (Kla) Tantithamthavorn
 
PPT
ISTQB / ISEB Foundation Exam Practice - 6
byYogindernath Gupta
 
PPTX
Software testing methods
byIng. Donato Di Pierro
 
PPT
Risks of Risk-Based Testing
byrrice2000
 
PDF
Testing artifacts test cases
byPetro Chernii
 
PPT
NG_TEST_SR_Presentation
bytechweb08
 
PPT
NGTEST_Presentation
bytechweb08
 
PPTX
Testability: Factors and Strategy
byBob Binder
 
PDF
SSBSE 2020 keynote
byShiva Nejati
 
PDF
Software testing strategy
byijseajournal
 
PDF
Testing Autonomous Cars for Feature Interaction Failures using Many-Objective...
byLionel Briand
 
PPT
'Automated Reliability Testing via Hardware Interfaces' by Bryan Bakker
byTEST Huddle
 
PDF
An Empirical Comparison of Model Validation Techniques for Defect Prediction ...
byChakkrit (Kla) Tantithamthavorn
 
PPTX
Put Risk Based Testing in place right now!
bySQALab
 
PDF
Decision Support for Security-Control Identification Using Machine Learning
byLionel Briand
 
SOFTWARE TESTING UNIT-4
byMohammad Faizan
 
SBST 2019 Keynote
byShiva Nejati
 
Exploratory Testing
bynazeer pasha
 
Software Testing & Debugging
byComputing Cage
 
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
byChakkrit (Kla) Tantithamthavorn
 
ISTQB / ISEB Foundation Exam Practice - 6
byYogindernath Gupta
 
Software testing methods
byIng. Donato Di Pierro
 
Risks of Risk-Based Testing
byrrice2000
 
Testing artifacts test cases
byPetro Chernii
 
NG_TEST_SR_Presentation
bytechweb08
 
NGTEST_Presentation
bytechweb08
 
Testability: Factors and Strategy
byBob Binder
 
SSBSE 2020 keynote
byShiva Nejati
 
Software testing strategy
byijseajournal
 
Testing Autonomous Cars for Feature Interaction Failures using Many-Objective...
byLionel Briand
 
'Automated Reliability Testing via Hardware Interfaces' by Bryan Bakker
byTEST Huddle
 
An Empirical Comparison of Model Validation Techniques for Defect Prediction ...
byChakkrit (Kla) Tantithamthavorn
 
Put Risk Based Testing in place right now!
bySQALab
 
Decision Support for Security-Control Identification Using Machine Learning
byLionel Briand
 

Similar to Software testing foundation

PPTX
Software Testing and Principles lecture slides
byNeerajEmanuel
 
PPTX
Software Testing from Unit-IV CIC-309 Btech
bytriptilamba5
 
PPT
ISfkghiijffhjbbbbnvvbnnbdzfv Fundamentals .ppt
byplayboxtv786
 
PPT
ISTQB Chapter 1 Fundamentals of Testing CTFL
bygintpt
 
PDF
Software Engineering TESTING AND MAINTENANCE
byDr Anuranjan Misra
 
PPT
U 1 Software Testing Principles.pptEEEEEEEEEEEEEEEEEEE
bykingspapsfeb2025
 
PDF
What is Testing in Software Engineering?
bytommychauhan
 
PDF
What is software testing in software engineering?
bytommychauhan
 
PPTX
CCS366 Softwares Testing Automation.pptx
byssuser1137dd
 
PPTX
Software Testing Introduction (Part 1)
byThapar Institute
 
PPT
ISTQBCH foundation level chapter 01 fundamentals of testing
byKhalilSalhi5
 
PPT
Software testing overview subbu
bySubramanya Mudukutore
 
PPTX
A Software Testing Intro
byEvozon Test Lab
 
PPTX
An introduction to Software Testing and Test Management
byAnuraj S.L
 
PPTX
Software Testing
byVishal Singh
 
PPT
NG_TEST_Presentation_0510
bytechweb08
 
PPT
Software Testing_Overview
bySubramanya Mudukutore
 
PPT
Software testing overview by subbu
bypalla subrahmanyam
 
PPTX
Testing Plan
byAjeng Savitri
 
PPTX
Software unit4
byHimanshu Awasthi
 
Software Testing and Principles lecture slides
byNeerajEmanuel
 
Software Testing from Unit-IV CIC-309 Btech
bytriptilamba5
 
ISfkghiijffhjbbbbnvvbnnbdzfv Fundamentals .ppt
byplayboxtv786
 
ISTQB Chapter 1 Fundamentals of Testing CTFL
bygintpt
 
Software Engineering TESTING AND MAINTENANCE
byDr Anuranjan Misra
 
U 1 Software Testing Principles.pptEEEEEEEEEEEEEEEEEEE
bykingspapsfeb2025
 
What is Testing in Software Engineering?
bytommychauhan
 
What is software testing in software engineering?
bytommychauhan
 
CCS366 Softwares Testing Automation.pptx
byssuser1137dd
 
Software Testing Introduction (Part 1)
byThapar Institute
 
ISTQBCH foundation level chapter 01 fundamentals of testing
byKhalilSalhi5
 
Software testing overview subbu
bySubramanya Mudukutore
 
A Software Testing Intro
byEvozon Test Lab
 
An introduction to Software Testing and Test Management
byAnuraj S.L
 
Software Testing
byVishal Singh
 
NG_TEST_Presentation_0510
bytechweb08
 
Software Testing_Overview
bySubramanya Mudukutore
 
Software testing overview by subbu
bypalla subrahmanyam
 
Testing Plan
byAjeng Savitri
 
Software unit4
byHimanshu Awasthi
 

Recently uploaded

PPTX
UNIT 1- Introduction to Basic of Computer Fundamentals
bypawarbhaktiit
 
PDF
6 Insights from the Patron Saint of AI Failure
byScott M. Graffius
 
PDF
Intro to CrafterQ - AI Agent Platform for the Enterprise.pdf
byCrafterQ
 
PPTX
DVCON24-IBM ai/ml driven hardware verification
byArun Joseph
 
PDF
Saga: The new era of transactions in a microservices architecture
byGiovanni Marigi
 
PPTX
JAVA Programming Lecture 1 C 4 Yourself.pptx
byyvsbglfaeahuyldzhq
 
PPTX
Lecture 05. API Security for DevSecOps Eng.pptx
byvinocol222
 
PDF
SAP WalkMe Overview.pdf SAP WalkMe Overview.pdf
byMurniatiSimbolon2
 
PPTX
Software Tools for penetration Testing (1).pptx
byAmosCheruiyot13
 
PDF
The Cost of Missing Critical Connections in Data - Suspicious Behavior Detect...
byEnterprise Knowledge
 
PPTX
Full course that Prymehat uploads for you version 2
byOhenebaManu1
 
PDF
Governing Agentic Enterprise - From Shadow AI to Autonomous Security.pdf
bysajjasridhar1990
 
PDF
Best-Travel-Portal-Development-Solutions-for-Online-Growth
bykashishnagarrajput
 
PPTX
02_Extended Warehouse Management Functions and Features.pptx
byUdayakumar218983
 
PPTX
Unit 1 Introduction of Computer Networks
bySuvarnaSharma5
 
PDF
Purple Team - Active Directory and AzureAD v1
byVICTOR MAESTRE RAMIREZ
 
PPTX
How Blockchain Application Development Enables Trustless Digital Ecosystems.pptx
byLisa ward
 
PDF
Generating Structured Outputs from Unstructured Content Using LLMs
byEnterprise Knowledge
 
PPTX
KTU-PEMET413 -MODULE 2-POLYMER MATRIX COMPOSITES - LECTURE 1.pptx
byVINAY B
 
PPTX
IBM_NEXA_DAC2021 NEXA, a cloud-native platform for collaborative hardware log...
byArun Joseph
 
UNIT 1- Introduction to Basic of Computer Fundamentals
bypawarbhaktiit
 
6 Insights from the Patron Saint of AI Failure
byScott M. Graffius
 
Intro to CrafterQ - AI Agent Platform for the Enterprise.pdf
byCrafterQ
 
DVCON24-IBM ai/ml driven hardware verification
byArun Joseph
 
Saga: The new era of transactions in a microservices architecture
byGiovanni Marigi
 
JAVA Programming Lecture 1 C 4 Yourself.pptx
byyvsbglfaeahuyldzhq
 
Lecture 05. API Security for DevSecOps Eng.pptx
byvinocol222
 
SAP WalkMe Overview.pdf SAP WalkMe Overview.pdf
byMurniatiSimbolon2
 
Software Tools for penetration Testing (1).pptx
byAmosCheruiyot13
 
The Cost of Missing Critical Connections in Data - Suspicious Behavior Detect...
byEnterprise Knowledge
 
Full course that Prymehat uploads for you version 2
byOhenebaManu1
 
Governing Agentic Enterprise - From Shadow AI to Autonomous Security.pdf
bysajjasridhar1990
 
Best-Travel-Portal-Development-Solutions-for-Online-Growth
bykashishnagarrajput
 
02_Extended Warehouse Management Functions and Features.pptx
byUdayakumar218983
 
Unit 1 Introduction of Computer Networks
bySuvarnaSharma5
 
Purple Team - Active Directory and AzureAD v1
byVICTOR MAESTRE RAMIREZ
 
How Blockchain Application Development Enables Trustless Digital Ecosystems.pptx
byLisa ward
 
Generating Structured Outputs from Unstructured Content Using LLMs
byEnterprise Knowledge
 
KTU-PEMET413 -MODULE 2-POLYMER MATRIX COMPOSITES - LECTURE 1.pptx
byVINAY B
 
IBM_NEXA_DAC2021 NEXA, a cloud-native platform for collaborative hardware log...
byArun Joseph
 

Software testing foundation

  • 1.
    Software Testing Foundation Module:Fundamentals of Testing
  • 2.
    2 Triangle Exercise Brief Historyof Testing Testing Challenges Topics Covered What is Testing Error – Fault - Failure Why Testing How much Testing Principles of Testing Psychology of Testing Fundamental Test Process
  • 3.
  • 4.
    4 Let’s Start Testing “Aprogram reads three (3) integer values. The three values are interpreted as representing the lengths of the sides of a triangle. The program displays a message that states whether the triangle is scalene, isosceles, or equilateral.” – G. Myers Requirement for a Triangle Program Sample Test Case TC# Input Expected Output 1 3,3,3 Equilateral 2 3,3,2 Isosceles 3 3,4,5 Scalene 4 3,4,% Invalid Black-Box Test Basis Is Testing Easy or Difficult Triangle Exercise
  • 5.
    5 Pseudo Code forthe requirement if (Side A) or (Side B) or (Side C) != int16 (signed 16 bit int) if input > maxint then error message for overflow condition else if input ! = integer value then error message for format exception if Side A or Side B or Side C <= 0 then error message sides must be > 0 if (Side A + Side B <= Side C) or (Side B + Side C <= Side A) or (Side A + Side C <= Side C) then input does not equate to valid triangle if (Side A = = Side B) and (Side B = = Side C) then triangle is equilateral else if (Side A = = Side B) or (Side A = = Side C) or (Side B = = Side C) then triangle is isosceles else triangle is scalene *Aside from the obvious conditions above additional tests include 1 for max val for all 3 sides, 1 for min val for all 3 sides, and 3 to test for BO of sum of 2 sides White Box Test Basis Triangle Exercise
  • 6.
  • 7.
  • 8.
    8 Brief History ofTesting Definitions of Testing Over the Years 1979 “Testing is the process of executing a program or system with the intent of finding errors” 2002 “Testing is a concurrent lifecycle process of engineering, using and maintaining testware in order to measure and improve the quality of the software being tested” “Testing is any activity aimed at evaluating an attribute of a program or system. Testing is the measurement of software quality” 1983
  • 9.
    9 Brief History ofTesting Development and Testing Evolution 1960 1980 Today Software Size Small Moderate Very Large Software Complexity Low Medium High Development Teams Small Medium Large Methods & Standards Ad-hoc Moderate Sophisticated Test Methods & Standards Ad-hoc Primitive Established Independent Test Organizations Few Some Many Recognition of Test Importance Little Some Significant Testing Professionals Few Few Many
  • 10.
  • 11.
    11 Testing Challenges Too manycombinations of inputs to test Not enough time to test properly Difficulty in determining the expected result of each test Nonexistent or rapidly changing requirements
  • 12.
    12 Testing Challenges The designof tests for software and other engineering products can be as challenging as the initial design of the product itself. Yet....software engineers often treat testing as an afterthought, developing test cases that ‘feel right’ but have little assurance of being complete. Recalling the objectives of testing, we must design tests that have the highest likelihood of finding the most errors with a minimum amount of time and effort.
  • 13.
    13 Testing Challenges Some Foodfor Thoughts Int blech (int J) { j = j – 1; //Should be j = j + 1 j = j / 30000; return j; } Input Expected Output Actual Output 1 0 0 42 0 0 40000 1 1 -64000 -2 -2 Oops! None of the test cases chosen have detected this defect
  • 14.
    14 Testing is aconcurrent lifecycle process of engineering to measure and improve the quality of software being tested. Independent Testing has gained significant momentum. Testing is not easy as it appears—It requires systematic approach and a creative mind. Recap Testing Challenges
  • 15.
    15 Value of Testingby Lee Copeland Testing Challenges
  • 16.
    16 Write a twopage summary of your understainding about the video, Value of Testing by Lee Copeland. Assignment 2 Testing Challenges
  • 17.
  • 18.
    18 Read the casestudy on Explosion of the Ariane and answer the questions in the given handout. What is Testing
  • 19.
    19 Read the casestudy on AT&T Failure and answer the questions in the given handout. What is Testing
  • 20.
    20 What is Testing SoftwareTesting is a process of evaluating a system Dynamic Testing Measure Quality Static Testing
  • 21.
    21 What is Testing Bymanual or automatic means Verify that the system satisfies the specified requirements Identify differences between expected and actual results Dynamic Testing
  • 22.
    22 What is Testing Testingresults in collection of lot of useful data and information about quality of the software. This information will help organizations to make informed decisions about releasing the software. Measure Quality
  • 23.
    23 What is Testing Testingincludes inspection and structured peer reviews of requirements and design, as well as code. Static Testing
  • 24.
  • 25.
    25 Error, Fault andFailure What is the problem here? $temperature = $temperature + $input if $tempereture > 100 then Print “Reactor Core Temperature Critical !!!” Else Print “Reactor Core Temperature Normal”
  • 26.
    26 Error, Fault andFailure • A manifestation of an error in software • Also known as a defect or bug • If executed, a fault may cause a failure Failure ERROR A human action that produces an incorrect result Fault Deviation of the software from its expected delivery or service Failure is an event, fault is a state of the software, caused by an error
  • 27.
    27 Error, Fault andFailure A person makes an error…. ….that creates a fault in software… ….that can cause failure in operation. Script
  • 28.
    28 Error, Reliability VSFaults Reliability: The probability that software will not cause the failure of the system for a specified time under specified conditions. Reliability VS Faults Can a system be fault-free? (zero faults, right first time) Can a software system be reliable but still have faults? Is a “fault-free” software application always reliable?
  • 29.
    29 who have skills,but aren’t perfect Error, Reliability VS Faults Why do faults occur in software? Software is written by human beings who know something, but not everything no time to check but assumptions may be wrong who do make mistakes (errors) systems may be incomplete Software is written by human beings Under increasing pressure due to strict deadlines
  • 30.
    30 Error, Reliability VSFaults If you can say it, you can do it. Software is written by human beings
  • 31.
    31 Very little ornothing at all Faults can cause death or injury Error, Reliability VS Faults What do software faults cost? Huge sums • Ariane 5 ($7billion) • American Airlines ($50m) • Minor inconvenience • Small input may have very large effect • Radiation treatment machine kills • Bank overdraft letters cause suicide Software is not “linear”
  • 32.
  • 33.
    33 To fill thetime between delivery of the software and the release date Why Testing So why is testing necessary? Because software is likely to have faults To learn about the reliability of the software To prove that the software has no faults Because testing is included in the project plan Because failures can be very expensive To avoid being sued by customers To stay in business
  • 34.
    34 Why Testing Why notjust “Test Everything” 232 232 + 264 Inputs Program
  • 35.
    35 Why Testing Why notjust “Test Everything” System has 20 screens Total for 'exhaustive' testing: 20 x 4 x 3 x 10 x 2 x 100 = 480,000 tests If 1 second per test, 8000 mins, 133 hrs, 17.7 days (not counting finger trouble, faults or retest) 10 secs = 34 wks, 1 min = 4 yrs, 10 min = 40 yrs Avr. 4 menus 3 options / menu Average: 10 fields / screen 2 types input / field (date as Jan 3 or 3/1) (number as integer or decimal) Around 100 possible values
  • 36.
    36 Why Testing When allthe testers are exhausted What is exhaustive testing? How much time will exhaustive testing take? When all the planned tests have been executed Exercising all combinations of inputs and preconditions Infinite time Not much time Impractical amount of time
  • 37.
  • 38.
    38 When your customer/useris happy How Much Testing How much testing is enough? It’s never enough When you have done what you planned When you have proved that the system works correctly When you are confident that the system works correctly It depends on the risks for your system
  • 39.
    39 How Much Testing Testingdepends on? Risk of missing important faults Risk of incurring failure costs Risk of releasing untested or under-tested software Risk of losing credibility and market share Risk of missing a market window Risk of over-testing, ineffective testing
  • 40.
    40 How Much Testing Whatto test first So much to test and so little time…. Time will always be limited use RISK to determine: What to test most How thoroughly to test each item What not to test (this time)
  • 41.
    41 How Much Testing Prioritizetests so that, whenever you stop testing, you have done the best testing in the time available. Most important principle Important
  • 42.
    42 How Much Testing Whatis an optimal test effort?
  • 43.
    43 How Much Testing Prioritizetesting tasks before going for testing: Time is Available Time is Limited Perform all test cases Test all relevant areas Test only high-risk areas Perform high-priority test, first
  • 44.
    44 When we talkabout testing we are talking about dynamic testing, Static testing and measuring quality. Terms like Error, Fault and Failure are used in a synonymous way - But they have subtle differences. Defects are very expensive - Some times may cause death. Recap How Much Testing Testing is software risk management activity. “How much testing?” - Depends on RISK.
  • 45.
  • 46.
    46 Research and writea reoprt about recent spectacular software failure. Assignment 3 Principles of Testing
  • 47.
    47 Read the casestudy on Necessity for Regression Testing and answer the questions in the given handout. Principles of Testing
  • 48.
    48 Principles of Testing Testingshows presence of defects Defect Clustering Exhaustive Testing is impossible The Pesticide Paradox Early Testing avoids fault multiplication Testing is Context dependent Absence-of-errors fallacy
  • 49.
  • 50.
    50 Read the casestudy on Independence to Testing Team and answer the questions in the given handout. Psychology of Testing
  • 51.
    51 Read the casestudy on Asking Questions and answer the questions in the given handout. Psychology of Testing
  • 52.
    52 Demonstrate conformance torequirements Psychology of Testing Why test? Build confidence Prove that the software is correct Find faults Reduce costs Show system meets user needs Assess the software quality
  • 53.
    53 Psychology of Testing Confidence Time Confidence Nofaults found = Confidence? Fault Found
  • 54.
    54 Few Faults Few Faults Psychologyof Testing Assessing software quality You may be here High Low Test Quality High Low Software Quality Many Faults Few Faults You think you are here
  • 55.
    55 Psychology of Testing Traditionalapproach of testing Show that system: does what it should doesn’t do what it shouldn’t Goal: Show Working Success: System Works Fastest Achievement: Easy Test Case Result : Faults left in
  • 56.
    56 Psychology of Testing Bettertesting approach Show that system: does what it shouldn’t doesn’t do what it should Goal: Find Faults Success: System Fails Fastest Achievement: Difficult Test Case Result : Few Faults left in
  • 57.
    57 Psychology of Testing Teststo Pass and Tests to Fail
  • 58.
    58 Psychology of Testing Thetesting paradox Purpose of Testing To Find Faults Finding Faults Destroys Confidence Purpose of Testing Destroys Confidence Purpose of Testing Build Confidence The best way to Build Confidence is to destroy it.
  • 59.
    59 Psychology of Testing Testingis not a glorious job, because: Testing is a destructive process It brings bad news (“your baby is ugly”) It keeps you under worst time pressure (at the end) You need to take a different view, a different mindset (What if it isn’t?”, “What could go wrong?”) You should know how to communicate the fault information to authors and managers.
  • 60.
    60 Psychology of Testing DeveloperVS Tester What do you think the relationship between a tester and developer should be like: Both are friendly Both competing with each other Both fighting with each other
  • 61.
    61 A good testerinternalizes the testing principles. Testing software requires a different mind set than that is required to develop software. Recap Psychology of Testing
  • 62.
    62 Knowledge Check A. Theevaluation done by testers is not influenced by personal experience. What is true about software testers? B. Experienced testers always guarantee high software quality. C. Testers need good communication skills. D. Testers and programmers must always have opposite mindsets.
  • 63.
    63 Knowledge Check A. Testleader writes a summary report. Which of the following statements is an example of testing contributing to higher quality? B. Project manager asks a test leader to estimate the test effort. C. Tester installs a test item in the test environment. D. Tester finds a bug which is resolved prior to release.
  • 64.
    64 Knowledge Check A. Failureis a cause of a defect. Which of the following statements is correct? B. Failure is a design error in the product. C. Failure is the deviation of the program from its expected behavior. D. Failure is an error in the code.
  • 65.
    65 Knowledge Check A. Toguarantee the protection against unauthorized access. Which of the following is a valuable objective for software testing? B. To fulfill the requirements for testing specified in the project handbook. C. To gain confidence about the level of quality and provide information about the test object. D. To prove the correctness of a specific feature.
  • 66.
    66 Which of thefollowing statements are “testing general principles”? I. Exhaustive testing is impossible. II. The defects found during the pre-release tests, the operational failures, are uniformly distributed across the system’s software modules. III. Testing can show the presence of defects, but can not demonstrate their absence. IV. Testing is context-independent. Knowledge Check A. I and III B. I and II C. II and III D. I and IV
  • 67.
    67 Knowledge Check A. Properlydesigned tests that pass reduce the level of risk in a system Which one of the following statements does not describe how testing contributes to higher quality? B. Performing a review of the requirement specifications before implementing the system can enhance quality. C. Software testing identifies defects, which can be used to improve development activities D. The testing of software demonstrates the absence of defects.
  • 68.
    68 Knowledge Check A. Preparetest data and execute test sets. Typical entry-level tester tasks may include the following: B. Decide what to automate and prepare test automation scripts. C. Select and implement testing tools. D. Select test approach and design test cases
  • 69.
    69 Which of thefollowing statements are true? I. Defects in a program do not always cause failures. II. A defect in a program may cause more than one failure during execution. III. If there is a fault in the program, at least one test will fail. IV. After fixing a bug, the program may still fail under test. V. Errors in the program lead to side effects Knowledge Check A. II and III are true. The others are false B. I and V are true. The others are false C. I and IV are true. The others are false D. I, II and IV are true. The others are false
  • 70.
    70 Which of thefollowing are valid testing principles? I. Exhaustive testing is in general impossible. II. Exhaustive testing should be executed for code intended to be reused. III. Testing may guarantee that a program is correct. IV. Testing can not guarantee that a program is correct . V. Defects cluster together in certain areas of the product. Knowledge Check A. I and V B. I, IV and V C. I and III D. II and IV Acceptance Test
  • 71.
    71 Knowledge Check A. Life-cyclemodel Which of the following is NOT a deciding factor in determining the extent of testing required? B. Level of risk C. Time D. Budget
  • 72.
    72 Knowledge Check A. FunctionalTest Which of the following is NOT used in system testing? B. Structure-Based Test C. Non-Functional Test D. Unit Test
  • 73.
    73 Knowledge Check You arethe test engineer responsible for the system testing of a digital Set-top box. The three main functions of the product are: ‘contents download’, ‘contents play 'and ‘contents search’. The likelihood and impact of failure for each function are shown in the graph below. Down Load Search Play Number of test cases Down Load Play Search Impact Likelihood As shown in the graph the number of test cases created for each function: Which one of the following statements is most likely to be correct? A. More test cases need to be created for the download function B. You should discard test cases for the play function C. The result of the risk analysis do not agree with the number of test cases D. The number of test cases generated aligns with the perceived risk
  • 74.
    74 Which of thefollowing statements are correct? I. System testing is often carried out by an independent test team. II. The main goal in acceptance testing is to find defects. III. Component testing may include testing of functionality and non- functional characteristics. IV. UAT typically verifies the fitness for use of the system by business users. V. The environment for system testing should correspond to the production environment. Knowledge Check A. II, III and V B. I, III, IV and V C. II, III and IV D. I, II and V
  • 75.
    75 Knowledge Check A. Ifthe same tests are repeated over and over again, eventually the same set of test cases Will no longer find any bugs. Which of the following describes the “pesticide paradox” principle? B. Testing every thing (all combinations of inputs and pre-conditions) is not feasible, except for trivial cases. C. A small number of modules contain most of the defects discovered during pre-release testing, or show most of the operational failures. D. Testing can show that defects are present, but cannot prove that there are no defects
  • 76.
    76 Knowledge Check A. Thetest engineer should not re-run the tests, as they have already been run, and results recorded. A test engineer finds a defect while testing. After the developer has fixed the defect, a test engineer decides to re-run a complete section of the tests. Which of the following is correct? B. The test engineer should re-run the tests, in order to ensure that new defects have not been introduced by the fix. C. The test engineer should re-run the tests, because the defect shows that the test cases need to be updated. D. The test engineer should not re-run the tests, they should be part of developer tests.
  • 77.
    77 Knowledge Check A. Testingthat the interfaces work correctly. What is integration testing? B. Looking for faults in larger components or subsystems. C. Specifying which components to integrate in which order. D. Another term for specifying for testing system integrity.
  • 78.
    78 Knowledge Check A. PerformanceTest Which of the following is not a test level according to the V-model? B. Unit Test C. System Test D. Acceptance Test
  • 79.
    79 Which of thefollowing are valid testing principles? I. Exhaustive testing is in general impossible. II. Exhaustive testing should be executed for code intended to be reused. III. Testing may guarantee that a program is correct. IV. Testing can not guarantee that a program is correct. V. Defects cluster together in certain areas of the product. Knowledge Check A. I and V B. I, IV and V C. I and III D. II and IV
  • 80.
    80 Fundamental Test Process Thefollowing is the fundamental test process flow: Test Planning and Control Test Implementation and Execution Test Closer Activities Yes Evaluating Exit Criteria and Reporting No Test Analysis and Design
  • 81.
    81 Read the casestudy on How to Plan Tests and answer the questions in the given handout. Fundamental Test Process Test Planning and Control
  • 82.
    82 Test Planning Exercise TestPlanning and Control GROUP EXERCISE Test Planning and Control
  • 83.
    83 Test Analysis andDesign Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 Reviewing the test basis Evaluating Testability Identify Test Conditions Design Test Cases Set up Test Data Set up Test Environment Create Traceability Matrix
  • 84.
    84 Feasibility Testability Completeness CorrectnessNecessity Prioritization Test Analysis and Design Step 1 Reviewing the test basis Traceability Unambiguous Consistency Criteria for Verification of Requirements
  • 85.
    85 Test Analysis andDesign ___________ of a requirement is judged based on what the user wants. For example, are the rules and regulations stated correctly? Does the requirement exactly reflect the user’s request? It is imperative that the end user, or a suitable representative, be involved during the requirements phase. This criterion can also be judged based on standards. Are the standards being followed? Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Correctness Correctness
  • 86.
    86 Test Analysis andDesign _____________ ensures that no necessary elements are missing from the requirement. The goal is to avoid omitting requirements simply because no one has asked the right questions or examined all of the pertinent source documents. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Completeness Completeness
  • 87.
    87 Test Analysis andDesign ___________ verifies that there are no internal or external contradictions among the elements within the work products, or between work products. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Consistency Consistency
  • 88.
    88 Test Analysis andDesign __________ of the requirement confirms that it is possible to create a test for the requirement, and that an expected result is known and can be programmatically or visually verified. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Testability Testability
  • 89.
    89 Feasibility Test Analysis andDesign _________ of a requirement ensures that it can be implemented given the budget, schedules, technology and other resources available. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Feasibility Completeness Feasibility
  • 90.
    90 Test Analysis andDesign _________ verifies that every requirement in the specification is pertinent to the system. Does this requirement contribute to those goals? Would excluding this requirement prevent the system from meeting those goals? Are any other requirements dependent on this requirement? Some irrelevant requirements are not really requirements, but proposed solutions. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Completeness Necessity Feasibility Necessity
  • 91.
    91 Test Analysis andDesign ____________ verifies that every requirement in the specification is pertinent to the system. Does this requirement contribute to those goals? Would excluding this requirement prevent the system from meeting those goals? Are any other requirements dependent on this requirement? Some irrelevant requirements are not really requirements, but proposed solutions. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Completeness Prioritization Feasibility Prioritization
  • 92.
    92 Test Analysis andDesign ____________ ensures that requirements are stated in a precise and measurable way. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Completeness Unambiguous Feasibility Unambiguous
  • 93.
    93 Test Analysis andDesign ___________ ensures that each requirement is identified in such a way that it can be associated with all parts of the system where it is used. Testability Traceability Unambiguous Consistency Correctness Necessity Prioritization Completeness Traceability Feasibility Traceability
  • 94.
    94 Test Analysis andDesign Step 2 Evaluating Testability Ensure that “expected results” can be derived from the requirements. Results can be verified programmatically or visually. Testability is the degree to which a software artifact facilitates testing in a given test context.
  • 95.
    95 Test Analysis andDesign Step 3 Identify Test Conditions Test conditions define the tests to be performed and should always match the development stages they relate to, alongside the requirements and designs they intend to prove. A test condition is defined as an item or event that could be verified by one or more test cases (e.g. a function, transaction, quality characteristic or structural element).
  • 96.
    96 Test Conditions Exercise TestAnalysis and Design GROUP EXERCISE
  • 97.
    97 Test Analysis andDesign TCER: TEST CONDITIONS AND EXPECTED RESULT Name and description of the test condition Begin these with an action verb: create, print, etc. Example: “Create invoice.” Every test condition is derived from one requirement or design item. The requirement or design item should be stated for traceability. For each test condition, define a corresponding expected result. Example: “An invoice is created.” Expected Result Requirement or Design Source
  • 98.
    98 Test Analysis andDesign TCER Document Preparation Example The ATM system shall dispense denominations of 100, 500, and 1000 Rupees only. If amounts entered have a fraction of a Rupee or amounts entered cannot be dispensed with denominations of 100, 500, or 1000, then an error message will appear informing the user that the amount entered is invalid. The ATM system will have a maximum withdrawal of Rs 8000 per transaction. If the amount entered to be withdrawn is more than Rs 8000, a message will be displayed informing the user that only a maximum of Rs 8000 per withdrawal is allowed. The ATM system will have a minimum withdrawal of Rs 500 per transaction. If the amount entered or to be withdrawn is less than Rs 500, a message will be displayed informing the user that only a minimum of Rs 500 withdrawal is allowed. ATM Withdrawal Requirements
  • 99.
    99 3 2 Min withdrawalamount -Valid The amount is accepted and the user is taken to the next step of the withdrawal transaction. Error message “Invalid amount…this ATM machine can only dispense bills with Denominations 100, 500 and 1000” appears on the ATM screen. 2 1 The amount entered in the ATM is not divisible by 100, 500,1000 The amount is accepted and the user is taken to the next step of the withdrawal transaction. The amount entered in the ATM is divisible by 100, 500, 1000 1 1 Test Analysis and Design TCER Document Preparation Example Requirement Number Cond# Test Condition Expected Result 4 2 Min withdrawal amount -Invalid Error message “Invalid amount…You may only withdraw a minimum of 500 per transaction.” appears on the ATM screen. 5 3 Max withdrawal amount -Valid The amount is accepted and the user is taken to the next step of the withdrawal transaction. 6 3 Max withdrawal amount -Invalid Error message “Invalid amount…You may only withdraw up to P8000 per transaction.” appears on the ATM screen. The ATM system shall dispense denominations of 100, 500, and 1000 Rupees only. If amounts entered have a fraction of a Rupee or amounts entered cannot be dispensed with denominations of 100, 500, or 1000, then an error message will appear informing the user that the amount entered is invalid. The ATM system will have a minimum withdrawal of Rs 500 per transaction. If the amount entered or to be withdrawn is less than Rs 500, a message will be displayed informing the user that only a minimum of Rs 500 withdrawal is allowed. The ATM system will have a maximum withdrawal of Rs 8000 per transaction. If the amount entered to be withdrawn is more than Rs 8000, a message will be displayed informing the user that only a maximum of Rs 8000 per withdrawal is allowed.
  • 100.
    10 Test Case Designis a very deep thinking activity and in order to think well we need “Test Design Techniques” Test Analysis and Design Step 4 Design Test Case
  • 101.
    10 Test Analysis andDesign Step 5 & 6 Setup Test Data and Environment + Setting up of Test Data and Test Environment suggests identifying necessary test data to support the test conditions and test cases. Once data is set, design the test environment set up and identify any required infrastructure and tools.
  • 102.
    10 Test Analysis andDesign Step 7 Creating Traceability Matrix In the software development life cycle, requirements are translated into specifications based on which code is developed. Once the construction process is complete, the product is put through various stages of testing before it finally goes live. Requirements tracing is the process of documenting the links between the user requirements for the system and the work products developed to implement and verify those requirements. The work products include software requirements, design specifications, software code, test plans and other artifacts of the systems development process. Requirements tracing helps the project team to understand which parts of the design and code implement the user’s requirements, and which tests are necessary to verify the correct implementation of user’s requirements.
  • 103.
    10 Test Analysis andDesign General format of Traceability Matrix: Requirements: R 1.1 R 1.2 R 1.3 Scenarios: S1.1 S1.2 S1.3 S2.1 S2.2 S3.1 S3.2 S3.3 Test Cases TC1 X X TC2 X X X TC3 X X TC4 X X X TC5 X X X TC6 TC7 X TC8 X X X TC9 X X
  • 104.
    10 Test Analysis andDesign Which requirements are not covered fully in the following traceability matrix: Requirements: R 1.1 R 1.2 R 1.3 Scenarios: S1.1 S1.2 S1.3 S2.1 S2.2 S3.1 S3.2 S3.3 Test Cases TC1 X X TC2 X X X TC3 X TC4 X X TC5 X X X TC6 TC7 TC8 X X TC9 X R1.1 and R1.3 are not covered fully.
  • 105.
    10 Test Analysis andDesign Changes were made to the Requirements R 1.2. Based on this, which test cases need to be changed? TC4, TC5, TC6, TC7 and TC8 need to be changed Requirements: R 1.1 R 1.2 R 1.3 Scenarios: S1.1 S1.2 S1.3 S2.1 S2.2 S3.1 S3.2 S3.3 Test Cases TC1 X X TC2 X TC3 X TC4 X TC5 X TC6 X TC7 X X TC8 X X TC9 X X
  • 106.
    10 Test Analysis andDesign Exercise Creating Traceability Matrix
  • 107.
    10 Knowledge Check A. Thedegree to which a requirement is stated in terms that permit establishment of test design. Which of the following statements correctly describes traceability? B. The capability of software product to enable modified software to be tested. C. An approach to integration testing where the component at the top of the component hierarchy is tested first. D. The ability to identify related items in documentation, such as requirements vs. software design.
  • 108.
    10 1. Developing, implementingand prioritizing test cases. 2. Developing and prioritizing test procedures, creating test data and optionally preparing test harnesses, and writing automated test scripts. 3. Creating test suites from the test procedures for efficient test execution. 4. Verifying that the test environment has been set up correctly. 5. Executing test procedures either manually or by using test execution tools, according to the planned sequence. 6. Logging the outcome of test execution and recording the identities and versions of the software under test, test tools and testware. 7. Comparing the actual results with the expected results. 8. Reporting the discrepancies as incidents and analyzing them in order to establish their cause. 9. Repeating the test activities as a result of action taken for each discrepancy. Re- testing/Confirmation testing and Regression Testing. Test Implementation and Execution The major tasks in Test Implementation and execution are:
  • 109.
    10 1. Checking testlogs against the exit criteria specified in test planning. 2. Assessing, if more tests are needed or if the exit criteria specified should be changed. 3. Writing a test summary report for stakeholders. Evaluating Exit Criteria and Reporting The major tasks in Evaluating Exit Criteria and Reporting are:
  • 110.
    11 1. Checking whichplanned deliverables have been delivered, the closure of incident reports or raising of change records for any that remain open, and the documentation of the acceptance of the system. 2. Finalizing and archiving testware, the test environment and the test infrastructure for later reuse. 3. Handover of testware to the maintenance organization. 4. Analyzing lessons learned for future releases and projects, and the improvement of test maturity. Test Closure Activity The major tasks in Evaluating Exit Criteria and Reporting are: Test closure activities collect data from completed test activities to consolidate experience, testware, facts and numbers.
  • 111.
    11 Knowledge Check A. Thecontinuous comparison between current progress and the test plan, in order to identify deviations. In the fundamental test process, what does primarily entail the “test analysis and design” activity? B. The testability evaluation of the test basis and test objects and the root-cause analysis. C. The data collection on the completed test activities, in order to consolidate experience, testware, facts and figures. D. The transformation of the test objectives in testing conditions and tangible test cases
  • 112.
    11 Knowledge Check A. Experience-basedtesting should be applied after systematic test methods have been used. Which of the following statement is true? B. White-box testing is equally useful for low-level and high-level testing. C. Black-box testing is equivalent to equivalence partitioning. D. Experience-based testing is best used in integration testing.
  • 113.
    11 Knowledge Check A. Planning,analysis and reporting, design and implementation, execution, test closure activities, evaluating exit criteria. Which of the following lists represents the correct sequence of the main activities of the fundamental test process (leaving out the activity of control which should take place in parallel to all the other activities)? B. Planning, analysis and design, implementation and execution, evaluation exit criteria and reporting, test closure activities C. Planning, analysis and design, execution, logging and reporting, regression testing D. Planning, analysis, design and implementation, execution, logging, test closure activities, evaluating exit criteria.
  • 114.
    11 Knowledge Check A. Testimplementation and execution At which stage of the fundamental test process do testers write the steps of the test procedures? B. Evaluating exit criteria and reporting C. Test closure activates D. Test planning and control
  • 115.
    11 Knowledge Check A. TestControl At what stage of Fundamental Test Process do testers evaluate the Testability of the test basis? B. Test Execution C. Evaluating exit criteria and reporting D. Test Analysis and design
  • 116.
    11 Knowledge Check A. TestControl At what stage of Fundamental Test Process do testers evaluate the Testability of the test basis? B. Test Execution C. Evaluating exit criteria and reporting D. Test Analysis and design
  • 117.
    11 Read Appendex-1 Exercisedbased on a Business Scenario. Assignment 4 Exercise
  • 118.

Editor's Notes