Lecture 12 testing.ppt


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Lecture 12 testing.ppt

  1. 1. CSE100: Introductory Software Engineering Lecture 12: Testing
  2. 2. Software Development Processes/Lifecycle Specification Development Validation Evolu tion Also, for SDLC see http:// uk.youtube.com/watch?v = OfgfnZZdMlI We are here We are also here Requirements Analysis Design Development Testing Evaluation
  3. 3. Overlapping/ In Parallel? Requirements Analysis Design Development Testing Evaluation
  4. 4. <ul><li>Testing, Debugging, Verification & Validation: </li></ul><ul><li>Explanations & clarifications! </li></ul>
  5. 5. Testing is not debugging! <ul><li>Testing is </li></ul><ul><ul><li>Seeing if a given set of inputs causes an acceptable or unacceptable behaviour in a program </li></ul></ul><ul><li>Debugging is </li></ul><ul><ul><li>Seeing why a given set of inputs causes an unacceptable behaviour and what must be changed to cause the behaviour to be acceptable </li></ul></ul>
  6. 6. Verification <ul><li>Testing is one (widely used) part of verification </li></ul><ul><li>Verification also includes: </li></ul><ul><li>Inspections/reviews (formal) </li></ul><ul><ul><li>Small teams inspecting the documents and work products in stages throughout the development lifecycle </li></ul></ul><ul><li>Walkthroughs (informal) </li></ul><ul><ul><li>As above but less formal/no documentation </li></ul></ul><ul><li>What is verified? </li></ul><ul><ul><li>Requirements specification </li></ul></ul><ul><ul><li>functional design </li></ul></ul><ul><ul><li>component design </li></ul></ul><ul><ul><li>Code </li></ul></ul><ul><ul><li>You saw this in the SimSE game! </li></ul></ul>
  7. 7. Software Development Processes/Lifecycle Specification Development Validation Evolu tion Verification Validation But the interplay of verification & validation will depend on your chosen development methodology Requirements Analysis Design Development Testing Evaluation
  8. 8. Example verification/inspection stages <ul><li>E.g. in the requirements specification </li></ul><ul><ul><li>Is there ambiguity? Is it complete? </li></ul></ul><ul><li>E.g. in the component/module </li></ul><ul><ul><li>Are there errors in the code logic? (error of commission) </li></ul></ul><ul><ul><li>Are some parts of the spec ignored in the code? (error of omission) </li></ul></ul><ul><li>E.g. boundary value errors </li></ul><ul><ul><li>(see later examples) </li></ul></ul>
  9. 9. ‘ V’ Model Requirements Design Code Design matches Requirements Structural Functional Usability Code matches Design Verification – build it the right way (fault-free) Validation - build the right thing
  10. 10. Or, in more detail….
  11. 11. Question <ul><li>When I ‘tested’ the bridges you built in the 2 nd week, was I doing verification or validation? </li></ul><ul><li>What would be appropriate methods of verifying and validating your bridge builds? </li></ul><ul><li>Discuss with neighbour </li></ul>
  12. 12. Software Validation <ul><li>To ensure that the software: </li></ul><ul><ul><li>Conforms to the specification </li></ul></ul><ul><ul><li>Meets the needs of the user </li></ul></ul><ul><ul><li>Performs effectively and efficiently </li></ul></ul><ul><ul><li>Performs safely </li></ul></ul>
  13. 13. Validation <ul><li>Done after verification stages, towards or at end of coding </li></ul><ul><ul><li>depending on methodology e.g. waterfall , e.g. agile or XP or test-driven , this might be at end of development or throughout </li></ul></ul><ul><li>In easy to understand terms, validation is, kind of, where ‘testing’ meets ‘evaluation’ </li></ul>
  14. 14. Purpose of Testing? <ul><li>2 different purposes: </li></ul><ul><ul><li>Test the Structure of the software </li></ul></ul><ul><ul><li>Test the Function of the software </li></ul></ul><ul><li>Structural ( White Box or Glass Box testing) </li></ul><ul><ul><li>Looks at the code and its structure </li></ul></ul><ul><ul><li>Test cases examine decision points in the code </li></ul></ul><ul><ul><li>Includes unit testing, integration testing </li></ul></ul><ul><li>Functional testing ( Black Box testing) </li></ul><ul><ul><li>Doesn’t look at the code </li></ul></ul><ul><ul><li>Examines whether a program accepts the correct inputs and produces correct outputs </li></ul></ul><ul><ul><li>Includes UAT, Alpha & Beta Testing </li></ul></ul>
  15. 15. Structural Testing
  16. 16. Types of structural testing <ul><li>Unit testing </li></ul><ul><li>Integration (system) testing </li></ul><ul><ul><ul><li>Do the units coordinate properly? Are there any problems with component interfaces? </li></ul></ul></ul><ul><ul><li>Remember, problem solving = breaking into chunks and working on the chunks , what passes between them , and how they combine into one large chunk (unit and integration) </li></ul></ul>
  17. 17. Structural 1) Unit Testing <ul><li>Verifies that a module works correctly in isolation </li></ul><ul><ul><li>Driver component makes method calls on the tested component </li></ul></ul><ul><ul><li>Any methods used are simulated as stubs – e.g. prints ‘hello’ to tell you it’s been successfully tested </li></ul></ul>
  18. 18. Unit testing Driver Component Stub Stub Stub
  19. 19. Drivers and Stubs Example Book Hand Over Read Book Bar Code Update Book Details Return Book Module Under Test Stub (supplies BarCode Value) Driver : Calls Stub then module under test, finally outputs Date of Return BarCode BarCode DateofReturn
  20. 20. Structural 2) Integration Testing <ul><li>Big Bang </li></ul><ul><ul><li>Bring everything together then test </li></ul></ul><ul><ul><li>Bad idea </li></ul></ul><ul><li>Improved Big Bang </li></ul><ul><ul><li>Bring everything together having done unit testing first </li></ul></ul><ul><ul><li>Better idea than Big Bang </li></ul></ul><ul><li>Incremental </li></ul><ul><ul><li>Better approach </li></ul></ul><ul><ul><li>Faults are localised </li></ul></ul>
  21. 21. Structural Testing: General Description <ul><li>Test all paths correctly and fully </li></ul><ul><ul><li>This can be difficult if code is complex </li></ul></ul><ul><li>Follow a process to create test data </li></ul><ul><ul><li>What is ‘test data’? </li></ul></ul><ul><li>Can be done by the code writer or programmers may swap modules, </li></ul><ul><ul><li>or in teams e.g. Quality Assurance teams, adversaries, bounty hunters </li></ul></ul>
  22. 22. <ul><li>Process for Structural Testing </li></ul>
  23. 23. <ul><li>Draw up a Test Run Chart (rectangles & circles) to identify all paths through the module </li></ul><ul><li>Identify the type of test for a particular path </li></ul><ul><ul><li>The type of path determines the number of tests needed for each path </li></ul></ul><ul><li>Draw up a Test Case Table showing </li></ul><ul><ul><li>all tests and </li></ul></ul><ul><ul><li>expected results </li></ul></ul><ul><ul><li>actual results </li></ul></ul>3 stages
  24. 24. Step 1) Test Run Chart <ul><li>Shows logical flow of a program/module </li></ul><ul><li>Identifies all paths to be tested </li></ul>The rectangle indicates normal processing The circle indicates a location in the module where the processing path splits. Path 1 (No) Path 2 (Yes) Switch Off? Calculate temperature
  25. 25. Testing Each Path There are five paths in the main, left loop: 1 2 3 4 5 <ul><li>Conditions: </li></ul><ul><ul><ul><li>If heater not on (1,2,3,4) or is on (5); </li></ul></ul></ul><ul><ul><ul><li>If office hours (1,2) or not (3,4) </li></ul></ul></ul><ul><ul><ul><ul><li>If temperature < 65F (2) or not (1) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>If temperature < 45F (4) or not (3) </li></ul></ul></ul></ul>Decision Action
  26. 26. How exhaustively do you test? <ul><li>Exhaustive – means what? </li></ul><ul><li>It means: every possible scenario </li></ul><ul><li>What are the implications of this? </li></ul><ul><ul><li>Let’s look at an example… </li></ul></ul>
  27. 27. Exhaustive Testing 0  loop < 20 How many possible paths are there through this program? Discuss this with your neighbour
  28. 28. Exhaustive Testing 0  loop < 20 How many possible paths are there through this program? 2,432,902,008,176,640,000
  29. 29. Exhaustive vs Effective Testing <ul><li>Exhaustive testing = unrealistic </li></ul><ul><li>In the example just given: </li></ul><ul><ul><li>loop executed 20 times </li></ul></ul><ul><ul><ul><li>but also every possible variation in the loop </li></ul></ul></ul><ul><ul><ul><li>And every possible sequence of variations </li></ul></ul></ul><ul><ul><li>When combined gives millions of tests! </li></ul></ul><ul><li>So, have to run a set of tests that is Effective not Exhaustive </li></ul><ul><ul><li>E.g. 99.99% correct (exhaustive) and 99.5% correct (effective) </li></ul></ul>
  30. 30. <ul><li>Step 2) Identify Type of Test </li></ul>
  31. 31. Conditional Tests: Individual Values <ul><li>Make sure TRUE and FALSE sides of each condition are tested </li></ul><ul><li>Testing individual values: </li></ul><ul><ul><li>for every valid value test for an invalid value </li></ul></ul><ul><ul><li>E.g. test for gender (‘m’ or ‘f’) </li></ul></ul><ul><ul><li>Would test for (‘m’ , ‘f’, + 2 invalids) = 4 tests </li></ul></ul><ul><ul><ul><li>E.g. expect ‘m’ and ‘m’ is entered OR expect ‘m’ and ‘ ’ is entered </li></ul></ul></ul><ul><ul><li>Assumes another letter has not been coded as valid, would not be easy to find other than: </li></ul></ul><ul><ul><ul><li>Every character in the character set is tested (exhaustive testing) </li></ul></ul></ul>
  32. 32. Conditional Tests: Value Ranges <ul><li>Requires Boundary Testing : </li></ul><ul><li>Every bound should be tested with 2 tests: </li></ul><ul><ul><li>E.g. If the range of values is from 0 to 10: </li></ul></ul><ul><ul><ul><li>Test 0 (valid) and -1 (invalid) </li></ul></ul></ul><ul><ul><ul><li>Test 10 (valid) and 11 (invalid) </li></ul></ul></ul><ul><ul><ul><li>Test a value randomly between 1 and 9 </li></ul></ul></ul><ul><ul><li>5 tests in this example </li></ul></ul><ul><ul><li>Exhaustive would be -99 to +99 (inclusive) i.e. every possible 2-digit integer </li></ul></ul>
  33. 33. Loop Testing <ul><li>Focus exclusively on the validity of loops </li></ul><ul><li>The following tests are applied to simple loops where n is the maximum number of allowable passes through the loop, try to: </li></ul><ul><ul><ul><li>skip the loop entirely </li></ul></ul></ul><ul><ul><ul><li>only pass once through the loop </li></ul></ul></ul><ul><ul><ul><li>m passes through the loop where m < n </li></ul></ul></ul><ul><ul><ul><li>attempt n - 1, n, n + 1 passes through the loop </li></ul></ul></ul>
  34. 34. Identifying the Tests <ul><li>Good idea to number the tests </li></ul><ul><li>Example test for value Range </li></ul><ul><ul><li>0 to 10 gives 5 tests (boundary) </li></ul></ul><ul><li>If this test is test path 2 in the module then number tests as follows: </li></ul>11 10 4 0 -1 Value P 2/4 F 2/5 P 2/3 P 2/2 F 2/1 Pass/ Fail Test Id.
  35. 35. Test Data for Multiple Tests <ul><li>The same data can be used for multiple tests. </li></ul><ul><li>Take an example of a record with 2 values: A & B, both are in the range 0-5 to be valid, B must be less than A. </li></ul><ul><li>One test data record can carry out 3 tests for 3 different paths </li></ul><ul><li>Tests 3/1, 4/1 and 5/1. Use the same data </li></ul>Fail B < A 5/1 Pass Test B=5 4/1 Pass Test A=3 3/1 Pass Fail Values Test id 5 3 Value B Value A
  36. 36. Test Cases <ul><li>A set of test inputs , execution conditions , & expected results for a particular objective : </li></ul><ul><ul><li>Guarantee all independent paths in a program have been exercised at least once </li></ul></ul><ul><ul><li>Exercise True & False for all logical decisions </li></ul></ul><ul><ul><li>Execute all loops and conditions at their boundaries and within their operational bounds </li></ul></ul><ul><ul><li>Exercise internal data structures (e.g. arrays, record structures) to ensure their validity </li></ul></ul>
  37. 37. Test Cases <ul><li>Derived during all phases of the development cycle </li></ul><ul><li>Determine what are your expected results before you run a test-case </li></ul><ul><ul><li>Then, running a test case provides the actual results </li></ul></ul><ul><li>Never throw away test-cases and results </li></ul><ul><li>Test cases are presented as a table. </li></ul>
  38. 38. Step 3) Test Case Table Description Input Expected Results Actual Results                       Pass/ Fail         Test id. (path/no.)        
  39. 39. Functional Testing
  40. 40. Types of Functional Testing <ul><li>UAT, Alpha Testing, Beta Testing </li></ul><ul><li>User-acceptance testing (UAT) – before final delivery: </li></ul><ul><ul><li>Product is handed over to user to test it in a real time scenario </li></ul></ul><ul><ul><li>See if it works according to the system specifications and satisfies all the user requirements. </li></ul></ul><ul><ul><li>As the user uses the software, some unknown bugs might be found – these are communicated to the developers to be fixed, and this helps improve the final product. </li></ul></ul><ul><li>When testing is done by paid testers, or the developers – it’s done in 2 stages: </li></ul><ul><li>Alpha-testing – in development environment </li></ul><ul><li>Beta-testing – in end-users’ environment (by developer and/or end user) </li></ul>
  41. 41. User Acceptance Testing <ul><li>Define acceptance criteria with client during phases of SDLC </li></ul><ul><li>Well defined acceptance plan helps development by identifying users’ needs during software development </li></ul><ul><li>The UAT must be created or reviewed by client & Software Engineer together </li></ul><ul><li>Identifies: </li></ul><ul><ul><li>interim and final products for acceptance, </li></ul></ul><ul><ul><li>acceptance criteria (and who will do the acceptance activities) </li></ul></ul><ul><ul><li>Schedule (including time for users to do the activities) </li></ul></ul><ul><li>When are you ready to do UAT? </li></ul><ul><ul><li>System Testing is completed and defects identified are either fixed or documented </li></ul></ul><ul><ul><li>Acceptance plan is prepared and resources have been identified </li></ul></ul><ul><ul><li>Test environment for the acceptance testing is available </li></ul></ul><ul><li>When is it finished? </li></ul><ul><ul><li>Acceptance decision is made for the software </li></ul></ul><ul><ul><li>In case of any caveats, development team is notified </li></ul></ul>
  42. 42. Beta-Testing <ul><li>Undertaken after UAT </li></ul><ul><li>Timed ahead of the official release date </li></ul><ul><li>System released to a small (select) number of customers </li></ul><ul><li>Customers use the system as though already released and are ‘encouraged’ to find bugs </li></ul>
  43. 43. Functional Testing: General Description <ul><li>Tests the specification (or data created from the spec.) not the code </li></ul><ul><ul><li>‘ what system does’ not ‘ how it does it’ </li></ul></ul><ul><li>i.e. tests data produced by the designer not the programmer </li></ul>
  44. 44. Functional Testing <ul><li>Black box attempts to derive sets of inputs that will fully exercise all the functional requirements of a system. </li></ul><ul><li>Aims to find errors in the following categories: </li></ul><ul><ul><li>interface errors </li></ul></ul><ul><ul><li>errors in data structures or internal database access </li></ul></ul><ul><ul><li>performance errors </li></ul></ul><ul><ul><li>initialization and termination errors </li></ul></ul>
  45. 45. An Example Black Box Test Book Handed Over Update Book Details Read Book Record Bar Code Book Code Date of Return MembershipId After testing these 3 modules individually, all 3 modules are combined in a test. For the 3-module together test, a single argument the – ‘ MembershipId’ producing a single reply ‘Date of Return’. Book Barcode is Read as part of the processing. Book Code Date of Return
  46. 46. <ul><li>Earlier we said functional testing uses a ‘black box’ approach i.e. looks at input and output but ignores ‘inside workings’ </li></ul><ul><li>So – how to decide on your test input? </li></ul><ul><li>We can use Equivalence Partitioning to guide black-box testing </li></ul>Deciding on functional test input data
  47. 47. Equivalence Partitioning Too many test inputs. Four test inputs, one selected from each sub-domain or equivalence class. Input domain 1 2 3 4 Input domain partitioned into four sub-domains.
  48. 48. How to partition <ul><li>Inputs to a program provide clues to partitioning: </li></ul><ul><ul><li>If a program takes an input X, where X is an integer </li></ul></ul><ul><ul><li>Where X<0 the program must perform one task and another task where X>=0 </li></ul></ul><ul><ul><li>The input domain is way too big because X can be a huge number of values </li></ul></ul>
  49. 49. … how to partition <ul><ul><li>In partition testing our goal is to cover all equivalence classes </li></ul></ul><ul><ul><li>We expect the program to behave the same way for every input where X < 0 </li></ul></ul><ul><ul><li>We expect the program to perform the same way for all values of X >= 0 </li></ul></ul><ul><ul><li>An equivalence class is considered covered when at least one test has been selected from it. </li></ul></ul>
  50. 50. Two sub-domains One test case: X = -3 Another test case: X = 15 X<0 X>=0 Equivalence class Equivalence class
  51. 51. Non-overlapping (disjoint) partitions <ul><li>In the previous example, the two equivalence classes are non-overlapping. In other words the two sub-domains are disjoint </li></ul><ul><li>In this case we can pick just one test input from each equivalence class to test the program </li></ul>
  52. 52. Overlapping partitions <ul><li>Suppose a program takes three integers X, Y and Z, and it is known that: </li></ul><ul><ul><ul><li>X<Y </li></ul></ul></ul><ul><ul><ul><li>Z>Y </li></ul></ul></ul>
  53. 53. Overlapping partitions X<Y X>=Y Z>Y Z<=Y X<Y, Z>Y X=3, Y=4, Z=7 X<Y, Z<=Y X=2, Y=3, Z=1 X>=Y, Z>Y X=15, Y=4, Z=7 X>=Y, Z<=Y X=15, Y=4, Z=1
  54. 54. Guidelines for defining equivalence classes for black box testing: <ul><ul><li>If an input condition specifies a range (e.g. 1-12), one valid and two invalid equivalence classes are defined (e.g. 5, 0 and 13). </li></ul></ul><ul><ul><li>If an input condition requires a specific value (e.g. ‘m’) then one valid and two invalid equivalence classes are defined (e.g. ‘m’, ‘f’, and ‘_’). </li></ul></ul><ul><ul><li>If an input condition specifies a member of a set (e.g. ‘change pin’, ‘check balance’, ‘withdraw cash’), then one valid and one invalid equivalence classes are defined. </li></ul></ul><ul><ul><li>If an input condition is Boolean (may or may not be present), then one valid and one invalid class is defined. </li></ul></ul>
  55. 55. Example equivalence class (set of valid or invalid states for input conditions) <ul><li>Example: </li></ul><ul><li>a function which accept input parameter &quot;month&quot; of a date. </li></ul><ul><li>valid range for month is 1(January)to 12(December). This valid range is called a partition. </li></ul><ul><li>In this example there are two further partitions of invalid ranges. </li></ul><ul><ul><li>The first invalid partition would be <= 0 </li></ul></ul><ul><ul><li>the second invalid partition would be >= 13. </li></ul></ul><ul><ul><li>You can also have invalid partition for non-digit input as well </li></ul></ul><ul><li>       .... -2 -1  0 1 .............. 12 13  14  15 .....        </li></ul><ul><li>--------------|-------------------|--------------------- </li></ul><ul><li>invalid partition 1      valid partition    invalid partition 2 </li></ul>
  56. 56. Final comments on testing <ul><li>Testing good practice: </li></ul><ul><li>Begins at component level & works out towards integration of whole system </li></ul><ul><li>Different testing techniques are appropriate at different times </li></ul><ul><li>Testing is done by the developers and (in large projects) an independent test group </li></ul><ul><li>Testing & debugging aren’t the same – but any test strategy has debugging in it </li></ul>
  57. 57. <ul><li>The last part of the course will look at: </li></ul><ul><ul><li>user evaluation strategies and at </li></ul></ul><ul><ul><li>evolution of software projects </li></ul></ul>
  58. 58. Before tomorrow, you must: <ul><li>Look over the course notes so far </li></ul><ul><li>Tomorrow you will get the assignment </li></ul><ul><ul><li>There’ll be time to do it in class in the next 2 weeks, with tutor support </li></ul></ul><ul><ul><li>There’ll be an interim marking next Tuesday </li></ul></ul><ul><li>The assignment tests ALL the module learning outcomes </li></ul><ul><li>This means we’ll be asking you to give a full account of what you’ve learned </li></ul><ul><ul><li>All aspects over the full SDLC </li></ul></ul>