EEL6883-SoftwareTest..

315 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
315
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

EEL6883-SoftwareTest..

  1. 1. Software Testing Testing across the entire software development lifecyle
  2. 2. Software test engineer <ul><li>Software test engineer is a professional who is in charge of one or more technical test activities, including </li></ul><ul><ul><li>designing test inputs </li></ul></ul><ul><ul><li>producing test case values </li></ul></ul><ul><ul><li>running test scripts </li></ul></ul><ul><ul><li>analyzing results </li></ul></ul><ul><ul><li>reporting results to developers and managers </li></ul></ul>By Ammann and Offutt, 2008
  3. 3. Software test manager <ul><li>Test manager is in charge of one or more test engineers. They </li></ul><ul><ul><li>set test policies and processes </li></ul></ul><ul><ul><li>monitors the processes </li></ul></ul><ul><ul><li>interact with other managers on the project </li></ul></ul><ul><ul><li>help the engineers do their assignments </li></ul></ul>By Ammann and Offutt, 2008
  4. 4. Activities of a software test engineer <ul><li>Design tests by creating test requirements </li></ul><ul><ul><li>requirements are transformed into actual values and scripts ready for execution </li></ul></ul><ul><ul><li>executable test are run against the software </li></ul></ul><ul><ul><li>the results are evaluated to determine if the tests find a fault </li></ul></ul><ul><li>Powerful tool: formal coverage criterion </li></ul><ul><ul><li>Provides test engineers ways to decide what test inputs to use during testing such that maximum possible faults can be found </li></ul></ul><ul><ul><li>Also provides stopping rules </li></ul></ul>By Ammann and Offutt, 2008
  5. 5. The mind of a tester <ul><li>Four different kinds of thinking shown by a good tester (by Kaner et. al) </li></ul><ul><ul><li>Technical thinking: ability to model technology and understand causes and effects </li></ul></ul><ul><ul><li>Creative thinking: ability to generate ideas and see possibilities </li></ul></ul><ul><ul><li>Critical thinking: ability to evaluate ideas and make inferences </li></ul></ul><ul><ul><li>Practical thinking: ability to put ideas into practice </li></ul></ul><ul><li>An example of these kinds of thinking is found in a fable called “The King’s Challenge” </li></ul>By Everett and McLeod Jr., 2007 and Kaner et. al, 2002
  6. 6. The mind of a tester (cont’d) <ul><li>The King’s Challenge (a fable) </li></ul><ul><ul><li>Once upon a time, a mighty king wanted to determine which of his three court wizards was the most powerful </li></ul></ul><ul><ul><li>So he put the three court wizards in the castle dungeon and declared whoever escaped from his respective dungeon cell first was the most powerful wizard in all the kingdom </li></ul></ul>By Everett and McLeod Jr., 2007 and Kaner et. al, 2002
  7. 7. The mind of a tester (cont’d) <ul><li>Here is what each court wizards has done </li></ul><ul><ul><li>The first wizard immediately started chanting mystical poems to open his cell door </li></ul></ul><ul><ul><li>The second wizard immediately started casting small polished stones and bits of bone on the floor to learn how he might open his cell door </li></ul></ul><ul><ul><li>The third wizard sat down across from his cell door and thought about the situation for a minute. Then, he got up, walked over to the cell door and pulled on the door handle. The cell door swung open because it was closed but not locked </li></ul></ul><ul><ul><li>Thus, the third wizard escaped his cell first and became known as the most powerful wizard in all the kingdom </li></ul></ul><ul><li>What kinds of “tester” thinking did the third wizard exercise in solving the king’s puzzle? </li></ul>By Everett and McLeod Jr., 2007 and Kaner et. al, 2002
  8. 8. The mind of a tester (cont’d) <ul><li>Answer: </li></ul><ul><ul><li>Creative thinking: ability to generate ideas and see possibilities </li></ul></ul><ul><ul><li>Practical thinking: ability to put ideas into practice </li></ul></ul>By Everett and McLeod Jr., 2007 and Kaner et. al, 2002
  9. 9. Non-software testing at the user level <ul><li>Buying a car </li></ul><ul><ul><li>Use the automobile industry to find non-computer testing examples that can be related to software testing </li></ul></ul><ul><ul><li>Motivation for testing a car is to determine its quality or its functionality before buying one </li></ul></ul><ul><ul><li>As a user, you are not interested in performing all possible kinds of test since you assume that manufacturer has done those tests for you </li></ul></ul><ul><ul><li>Important to realize that you limit your testing some way </li></ul></ul><ul><ul><li>Limited test is referred as “test drive” </li></ul></ul><ul><ul><li>To better understand the testing limits , examine what you do NOT test followed by what you TEST </li></ul></ul>By Everett and McLeod Jr., 2007
  10. 10. Non-software testing at the user level <ul><li>Objectives of a Test Drive are NOT </li></ul><ul><ul><li>to break the car </li></ul></ul><ul><ul><ul><li>Seek guarantees and warranties that car manufacturer has already proven that car is “unbreakable” under normal driving conditions for x thousand miles or y years (reliability) </li></ul></ul></ul><ul><ul><li>to improve the car’s design </li></ul></ul><ul><ul><ul><li>If you want design changes in the car, simply shop for different model or different manufacturer </li></ul></ul></ul>By Everett and McLeod Jr., 2007
  11. 11. Non-software testing at the user level (cont’d) <ul><li>What you test is determined by your transportation needs. The needs become test drive objectives </li></ul><ul><li>Test objectives are the measurable milestones in testing which indicate that the testing activities have achieved the desired goals </li></ul><ul><li>Test drive objectives are translated into testing approaches that validate whether the car on the dealer’s lot meets your transportation objectives </li></ul>By Everett and McLeod Jr., 2007
  12. 12. Non-software testing at the user level (cont’d) <ul><li>Objectives of a Test Drive ARE </li></ul><ul><ul><li>to validate affordability </li></ul></ul><ul><ul><li>to validate attractiveness </li></ul></ul><ul><ul><li>to validate comfort </li></ul></ul><ul><ul><li>to validate usefulness </li></ul></ul><ul><ul><li>to validate performance </li></ul></ul><ul><li>Each of these objectives can be validated without trying to break the car or redesign it. </li></ul><ul><li>Each of these objectives are personal and you need to prioritize them to in final your final decision </li></ul>By Everett and McLeod Jr., 2007
  13. 13. Non-software testing at the user level (cont’d) <ul><li>Testing approaches include </li></ul><ul><ul><li>examining the sticker price and sale contract </li></ul></ul><ul><ul><li>trying out radio, the air conditioner, and the lights </li></ul></ul><ul><ul><li>trying acceleration, stopping, and cornering </li></ul></ul><ul><li>These testing approaches are referred to by common terminology in the testing industry </li></ul><ul><ul><li>examine = static testing </li></ul></ul><ul><ul><li>(observe, read, review without driving the car) </li></ul></ul><ul><ul><li>try out = functional and structural testing </li></ul></ul><ul><ul><li>(work different features of the car without driving the car) </li></ul></ul><ul><ul><li>try = performance testing </li></ul></ul><ul><ul><li>(work different features of the car by actually driving the car) </li></ul></ul>By Everett and McLeod Jr., 2007
  14. 14. Non-software testing at the developer level <ul><li>Building a car </li></ul><ul><li>Testing objectives of a new car to be built </li></ul><ul><ul><li>validate design via scale models </li></ul></ul><ul><ul><li>validate operation of prototypes </li></ul></ul><ul><ul><li>validate mass assembly plans from prototypes </li></ul></ul><ul><li>Starts with written requirements such as </li></ul><ul><ul><li>Seats six </li></ul></ul><ul><ul><li>Carries five suite cases </li></ul></ul><ul><ul><li>Runs on regular gas </li></ul></ul><ul><ul><li>Consumes gas at a rate of 35 miles per gallon in highway </li></ul></ul><ul><ul><li>Has a top speed of 100 miles per hour </li></ul></ul>By Everett and McLeod Jr., 2007
  15. 15. Non-software testing at the developer level (cont’d) <ul><li>These requirements are the nonnegotiable design and manufacturing boundaries set by groups other than the designers </li></ul><ul><li>The manufacturer is responsible to build a new car which satisfies all the requirements </li></ul><ul><li>New requirements,  more understandable test objectives </li></ul><ul><li>The auto design tester is responsible to validate the current state of the new car against its requirements </li></ul><ul><ul><li>If the new car doesn’t initially meet the requirements, then it’s the designer, not the tester who must improve the design for full compliance </li></ul></ul><ul><ul><li>After the design changes are done, the tester needs to revalidate the revised design against the requirements </li></ul></ul><ul><li>Design, test, correct, retest cycle continues until the new car meets its requirements and is completed before the car is manufactured </li></ul><ul><li>Requirements are essential for testing validation at every stage of developing a new car </li></ul>By Everett and McLeod Jr., 2007
  16. 16. <ul><li>Testing approaches used while building new cars </li></ul><ul><ul><li>plan the tests based on requirements and design specifications </li></ul></ul><ul><ul><li>examine blueprints and clay models </li></ul></ul><ul><ul><li>perform and analyze wind tunnel tests </li></ul></ul><ul><ul><li>perform and analyze safety tests </li></ul></ul><ul><ul><li>perform and validate prototype features </li></ul></ul><ul><ul><li>drive prototype and validate operations </li></ul></ul><ul><li>Specifications (blue prints or models) are the designer’s interpretation of the requirements on how the design can be manufactured </li></ul><ul><li>When specifications are validated against the requirements, all the subsequent physical car assembly validation can be performed against the specifications </li></ul>Non-software testing at the developer level (cont’d) By Everett and McLeod Jr., 2007
  17. 17. Non-software testing at the developer level (cont’d) <ul><li>Similar to the test drive, the car builder testing approaches can be described by common testing terminology </li></ul><ul><ul><li>examine = static testing </li></ul></ul><ul><ul><li>(observe, read, review without driving the car) </li></ul></ul><ul><ul><li>perform = functional and structural testing </li></ul></ul><ul><ul><li>(work different features of the car models, mock-ups, and manufactured car subassemblies) </li></ul></ul><ul><ul><li>drive = performance testing </li></ul></ul><ul><ul><li>(work different features of the car in the prototypes) </li></ul></ul>By Everett and McLeod Jr., 2007
  18. 18. Primary objectives of testing <ul><li>Testing objective 1: Identify the magnitude and sources of development risk reducible by testing </li></ul><ul><ul><li>Company prepares a business case including expected benefits, costs, and risks </li></ul></ul><ul><ul><li>If the project is determined to be bad on investment, then the project does not start </li></ul></ul><ul><ul><li>If the benefits outweigh the cost and it’s good return on investment, the benefits are compared to the risks </li></ul></ul><ul><ul><ul><li>If the risk is high but the likelihood of the risk occurrence is low, the project is on </li></ul></ul></ul><ul><ul><ul><li>If the risk and the likelihood of the risk occurrence is high, the following questions are asked: </li></ul></ul></ul><ul><ul><ul><ul><li>Can this risk be reduced by testing? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>If the risk can be reduced, how much can testing reduce it? </li></ul></ul></ul></ul><ul><ul><ul><li>If the risk factors are well known and quantifiable, it’s possible that testing can reduce the probability of the risk occurrence </li></ul></ul></ul>By Everett and McLeod Jr., 2007
  19. 19. Primary objectives of testing (cont’d) <ul><li>Testing objective 2: Perform testing to reduce identified risks </li></ul><ul><ul><li>Test planning includes </li></ul></ul><ul><ul><ul><li>positive testing (things work as required) </li></ul></ul></ul><ul><ul><ul><li>negative testing (things that break) </li></ul></ul></ul><ul><ul><li>Test planning highlights the risk areas such that the largest possible percentage of the test schedule and effort (both positive and negative testing) are allocated to reduce that risk </li></ul></ul><ul><ul><li>Testing does not completely eliminate the risk since there are always more scenarios to test than the allotted time and resources to complete the tests </li></ul></ul>By Everett and McLeod Jr., 2007
  20. 20. Primary objectives of testing (cont’d) <ul><li>Testing objective 3: Know when testing is completed </li></ul><ul><ul><li>Since 100% testing is unrealistic, the tester must determine when to stop testing </li></ul></ul><ul><ul><li>The determination should start with the positive test items in the test plan </li></ul></ul><ul><ul><li>Complete as much of the risk-targeted testing as possible relative to cost benefit break-even point </li></ul></ul><ul><ul><ul><li>$100,000 business risk, $50,000 testing cost to reduce the risk (NO!!!) </li></ul></ul></ul><ul><ul><ul><li>A rule of thumb is 10-20% cost to benefit break-even point for testing </li></ul></ul></ul><ul><ul><ul><ul><li>$100,000 business risk, $1000-2000 testing cost (YES!!!) </li></ul></ul></ul></ul><ul><ul><li>Tester must complete as many of the negative test items in the plan with the remaining the testing budget after positive testing and risk testing are completed </li></ul></ul>By Everett and McLeod Jr., 2007
  21. 21. Primary objectives of testing (cont’d) <ul><li>Testing objective 4: Manage testing as a standard project within the development project </li></ul><ul><ul><li>Often testing is treated as simple skill that anyone can do without planning, scheduling or resources </li></ul></ul><ul><ul><li>Since business risk represents real dollar loss, real dollar testing is required to reduce the risk </li></ul></ul><ul><ul><li>Real dollar testing means that personnel with testing experience should be become the testing team with access to management, resources and schedules necessary to plan and complete the test </li></ul></ul><ul><ul><li>Observations: </li></ul></ul><ul><ul><ul><li>The testing does not have to be a hit or miss </li></ul></ul></ul><ul><ul><ul><li>Testers are a limited resource </li></ul></ul></ul>By Everett and McLeod Jr., 2007
  22. 22. References <ul><li>P. Ammann and Jeff Offutt, Introduction to software testing, Cambridge University Press, 2008 </li></ul><ul><li>G. D. Everett and R. McLeod Jr., Software Testing, Wiley Inter-Science, 2007 </li></ul><ul><li>C. Kaner, J. Bach, B. Pettichort, Lessons learned in software testing , Wiley Computer Publishing, Addison-Wesley, 2002, pp. 286 </li></ul>

×