Your SlideShare is downloading. ×
Lecture 12 testing.ppt
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Lecture 12 testing.ppt

7,714
views

Published on


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
7,714
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
354
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. CSE100: Introductory Software Engineering Lecture 12: Testing
  • 2. Software Development Processes/Lifecycle Specification Development Validation Evolu tion Also, for SDLC see http:// uk.youtube.com/watch?v = OfgfnZZdMlI We are here We are also here Requirements Analysis Design Development Testing Evaluation
  • 3. Overlapping/ In Parallel? Requirements Analysis Design Development Testing Evaluation
  • 4.
    • Testing, Debugging, Verification & Validation:
    • Explanations & clarifications!
  • 5. Testing is not debugging!
    • Testing is
      • Seeing if a given set of inputs causes an acceptable or unacceptable behaviour in a program
    • Debugging is
      • Seeing why a given set of inputs causes an unacceptable behaviour and what must be changed to cause the behaviour to be acceptable
  • 6. Verification
    • Testing is one (widely used) part of verification
    • Verification also includes:
    • Inspections/reviews (formal)
      • Small teams inspecting the documents and work products in stages throughout the development lifecycle
    • Walkthroughs (informal)
      • As above but less formal/no documentation
    • What is verified?
      • Requirements specification
      • functional design
      • component design
      • Code
      • You saw this in the SimSE game!
  • 7. Software Development Processes/Lifecycle Specification Development Validation Evolu tion Verification Validation But the interplay of verification & validation will depend on your chosen development methodology Requirements Analysis Design Development Testing Evaluation
  • 8. Example verification/inspection stages
    • E.g. in the requirements specification
      • Is there ambiguity? Is it complete?
    • E.g. in the component/module
      • Are there errors in the code logic? (error of commission)
      • Are some parts of the spec ignored in the code? (error of omission)
    • E.g. boundary value errors
      • (see later examples)
  • 9. ‘ V’ Model Requirements Design Code Design matches Requirements Structural Functional Usability Code matches Design Verification – build it the right way (fault-free) Validation - build the right thing
  • 10. Or, in more detail….
  • 11. Question
    • When I ‘tested’ the bridges you built in the 2 nd week, was I doing verification or validation?
    • What would be appropriate methods of verifying and validating your bridge builds?
    • Discuss with neighbour
  • 12. Software Validation
    • To ensure that the software:
      • Conforms to the specification
      • Meets the needs of the user
      • Performs effectively and efficiently
      • Performs safely
  • 13. Validation
    • Done after verification stages, towards or at end of coding
      • depending on methodology e.g. waterfall , e.g. agile or XP or test-driven , this might be at end of development or throughout
    • In easy to understand terms, validation is, kind of, where ‘testing’ meets ‘evaluation’
  • 14. Purpose of Testing?
    • 2 different purposes:
      • Test the Structure of the software
      • Test the Function of the software
    • Structural ( White Box or Glass Box testing)
      • Looks at the code and its structure
      • Test cases examine decision points in the code
      • Includes unit testing, integration testing
    • Functional testing ( Black Box testing)
      • Doesn’t look at the code
      • Examines whether a program accepts the correct inputs and produces correct outputs
      • Includes UAT, Alpha & Beta Testing
  • 15. Structural Testing
  • 16. Types of structural testing
    • Unit testing
    • Integration (system) testing
        • Do the units coordinate properly? Are there any problems with component interfaces?
      • Remember, problem solving = breaking into chunks and working on the chunks , what passes between them , and how they combine into one large chunk (unit and integration)
  • 17. Structural 1) Unit Testing
    • Verifies that a module works correctly in isolation
      • Driver component makes method calls on the tested component
      • Any methods used are simulated as stubs – e.g. prints ‘hello’ to tell you it’s been successfully tested
  • 18. Unit testing Driver Component Stub Stub Stub
  • 19. Drivers and Stubs Example Book Hand Over Read Book Bar Code Update Book Details Return Book Module Under Test Stub (supplies BarCode Value) Driver : Calls Stub then module under test, finally outputs Date of Return BarCode BarCode DateofReturn
  • 20. Structural 2) Integration Testing
    • Big Bang
      • Bring everything together then test
      • Bad idea
    • Improved Big Bang
      • Bring everything together having done unit testing first
      • Better idea than Big Bang
    • Incremental
      • Better approach
      • Faults are localised
  • 21. Structural Testing: General Description
    • Test all paths correctly and fully
      • This can be difficult if code is complex
    • Follow a process to create test data
      • What is ‘test data’?
    • Can be done by the code writer or programmers may swap modules,
      • or in teams e.g. Quality Assurance teams, adversaries, bounty hunters
  • 22.
    • Process for Structural Testing
  • 23.
    • Draw up a Test Run Chart (rectangles & circles) to identify all paths through the module
    • Identify the type of test for a particular path
      • The type of path determines the number of tests needed for each path
    • Draw up a Test Case Table showing
      • all tests and
      • expected results
      • actual results
    3 stages
  • 24. Step 1) Test Run Chart
    • Shows logical flow of a program/module
    • Identifies all paths to be tested
    The rectangle indicates normal processing The circle indicates a location in the module where the processing path splits. Path 1 (No) Path 2 (Yes) Switch Off? Calculate temperature
  • 25. Testing Each Path There are five paths in the main, left loop: 1 2 3 4 5
    • Conditions:
        • If heater not on (1,2,3,4) or is on (5);
        • If office hours (1,2) or not (3,4)
          • If temperature < 65F (2) or not (1)
          • If temperature < 45F (4) or not (3)
    Decision Action
  • 26. How exhaustively do you test?
    • Exhaustive – means what?
    • It means: every possible scenario
    • What are the implications of this?
      • Let’s look at an example…
  • 27. Exhaustive Testing 0  loop < 20 How many possible paths are there through this program? Discuss this with your neighbour
  • 28. Exhaustive Testing 0  loop < 20 How many possible paths are there through this program? 2,432,902,008,176,640,000
  • 29. Exhaustive vs Effective Testing
    • Exhaustive testing = unrealistic
    • In the example just given:
      • loop executed 20 times
        • but also every possible variation in the loop
        • And every possible sequence of variations
      • When combined gives millions of tests!
    • So, have to run a set of tests that is Effective not Exhaustive
      • E.g. 99.99% correct (exhaustive) and 99.5% correct (effective)
  • 30.
    • Step 2) Identify Type of Test
  • 31. Conditional Tests: Individual Values
    • Make sure TRUE and FALSE sides of each condition are tested
    • Testing individual values:
      • for every valid value test for an invalid value
      • E.g. test for gender (‘m’ or ‘f’)
      • Would test for (‘m’ , ‘f’, + 2 invalids) = 4 tests
        • E.g. expect ‘m’ and ‘m’ is entered OR expect ‘m’ and ‘ ’ is entered
      • Assumes another letter has not been coded as valid, would not be easy to find other than:
        • Every character in the character set is tested (exhaustive testing)
  • 32. Conditional Tests: Value Ranges
    • Requires Boundary Testing :
    • Every bound should be tested with 2 tests:
      • E.g. If the range of values is from 0 to 10:
        • Test 0 (valid) and -1 (invalid)
        • Test 10 (valid) and 11 (invalid)
        • Test a value randomly between 1 and 9
      • 5 tests in this example
      • Exhaustive would be -99 to +99 (inclusive) i.e. every possible 2-digit integer
  • 33. Loop Testing
    • Focus exclusively on the validity of loops
    • The following tests are applied to simple loops where n is the maximum number of allowable passes through the loop, try to:
        • skip the loop entirely
        • only pass once through the loop
        • m passes through the loop where m < n
        • attempt n - 1, n, n + 1 passes through the loop
  • 34. Identifying the Tests
    • Good idea to number the tests
    • Example test for value Range
      • 0 to 10 gives 5 tests (boundary)
    • If this test is test path 2 in the module then number tests as follows:
    11 10 4 0 -1 Value P 2/4 F 2/5 P 2/3 P 2/2 F 2/1 Pass/ Fail Test Id.
  • 35. Test Data for Multiple Tests
    • The same data can be used for multiple tests.
    • Take an example of a record with 2 values: A & B, both are in the range 0-5 to be valid, B must be less than A.
    • One test data record can carry out 3 tests for 3 different paths
    • Tests 3/1, 4/1 and 5/1. Use the same data
    Fail B < A 5/1 Pass Test B=5 4/1 Pass Test A=3 3/1 Pass Fail Values Test id 5 3 Value B Value A
  • 36. Test Cases
    • A set of test inputs , execution conditions , & expected results for a particular objective :
      • Guarantee all independent paths in a program have been exercised at least once
      • Exercise True & False for all logical decisions
      • Execute all loops and conditions at their boundaries and within their operational bounds
      • Exercise internal data structures (e.g. arrays, record structures) to ensure their validity
  • 37. Test Cases
    • Derived during all phases of the development cycle
    • Determine what are your expected results before you run a test-case
      • Then, running a test case provides the actual results
    • Never throw away test-cases and results
    • Test cases are presented as a table.
  • 38. Step 3) Test Case Table Description Input Expected Results Actual Results                       Pass/ Fail         Test id. (path/no.)        
  • 39. Functional Testing
  • 40. Types of Functional Testing
    • UAT, Alpha Testing, Beta Testing
    • User-acceptance testing (UAT) – before final delivery:
      • Product is handed over to user to test it in a real time scenario
      • See if it works according to the system specifications and satisfies all the user requirements.
      • As the user uses the software, some unknown bugs might be found – these are communicated to the developers to be fixed, and this helps improve the final product.
    • When testing is done by paid testers, or the developers – it’s done in 2 stages:
    • Alpha-testing – in development environment
    • Beta-testing – in end-users’ environment (by developer and/or end user)
  • 41. User Acceptance Testing
    • Define acceptance criteria with client during phases of SDLC
    • Well defined acceptance plan helps development by identifying users’ needs during software development
    • The UAT must be created or reviewed by client & Software Engineer together
    • Identifies:
      • interim and final products for acceptance,
      • acceptance criteria (and who will do the acceptance activities)
      • Schedule (including time for users to do the activities)
    • When are you ready to do UAT?
      • System Testing is completed and defects identified are either fixed or documented
      • Acceptance plan is prepared and resources have been identified
      • Test environment for the acceptance testing is available
    • When is it finished?
      • Acceptance decision is made for the software
      • In case of any caveats, development team is notified
  • 42. Beta-Testing
    • Undertaken after UAT
    • Timed ahead of the official release date
    • System released to a small (select) number of customers
    • Customers use the system as though already released and are ‘encouraged’ to find bugs
  • 43. Functional Testing: General Description
    • Tests the specification (or data created from the spec.) not the code
      • ‘ what system does’ not ‘ how it does it’
    • i.e. tests data produced by the designer not the programmer
  • 44. Functional Testing
    • Black box attempts to derive sets of inputs that will fully exercise all the functional requirements of a system.
    • Aims to find errors in the following categories:
      • interface errors
      • errors in data structures or internal database access
      • performance errors
      • initialization and termination errors
  • 45. An Example Black Box Test Book Handed Over Update Book Details Read Book Record Bar Code Book Code Date of Return MembershipId After testing these 3 modules individually, all 3 modules are combined in a test. For the 3-module together test, a single argument the – ‘ MembershipId’ producing a single reply ‘Date of Return’. Book Barcode is Read as part of the processing. Book Code Date of Return
  • 46.
    • Earlier we said functional testing uses a ‘black box’ approach i.e. looks at input and output but ignores ‘inside workings’
    • So – how to decide on your test input?
    • We can use Equivalence Partitioning to guide black-box testing
    Deciding on functional test input data
  • 47. Equivalence Partitioning Too many test inputs. Four test inputs, one selected from each sub-domain or equivalence class. Input domain 1 2 3 4 Input domain partitioned into four sub-domains.
  • 48. How to partition
    • Inputs to a program provide clues to partitioning:
      • If a program takes an input X, where X is an integer
      • Where X<0 the program must perform one task and another task where X>=0
      • The input domain is way too big because X can be a huge number of values
  • 49. … how to partition
      • In partition testing our goal is to cover all equivalence classes
      • We expect the program to behave the same way for every input where X < 0
      • We expect the program to perform the same way for all values of X >= 0
      • An equivalence class is considered covered when at least one test has been selected from it.
  • 50. Two sub-domains One test case: X = -3 Another test case: X = 15 X<0 X>=0 Equivalence class Equivalence class
  • 51. Non-overlapping (disjoint) partitions
    • In the previous example, the two equivalence classes are non-overlapping. In other words the two sub-domains are disjoint
    • In this case we can pick just one test input from each equivalence class to test the program
  • 52. Overlapping partitions
    • Suppose a program takes three integers X, Y and Z, and it is known that:
        • X<Y
        • Z>Y
  • 53. Overlapping partitions X<Y X>=Y Z>Y Z<=Y X<Y, Z>Y X=3, Y=4, Z=7 X<Y, Z<=Y X=2, Y=3, Z=1 X>=Y, Z>Y X=15, Y=4, Z=7 X>=Y, Z<=Y X=15, Y=4, Z=1
  • 54. Guidelines for defining equivalence classes for black box testing:
      • If an input condition specifies a range (e.g. 1-12), one valid and two invalid equivalence classes are defined (e.g. 5, 0 and 13).
      • If an input condition requires a specific value (e.g. ‘m’) then one valid and two invalid equivalence classes are defined (e.g. ‘m’, ‘f’, and ‘_’).
      • If an input condition specifies a member of a set (e.g. ‘change pin’, ‘check balance’, ‘withdraw cash’), then one valid and one invalid equivalence classes are defined.
      • If an input condition is Boolean (may or may not be present), then one valid and one invalid class is defined.
  • 55. Example equivalence class (set of valid or invalid states for input conditions)
    • Example:
    • a function which accept input parameter &quot;month&quot; of a date.
    • valid range for month is 1(January)to 12(December). This valid range is called a partition.
    • In this example there are two further partitions of invalid ranges.
      • The first invalid partition would be <= 0
      • the second invalid partition would be >= 13.
      • You can also have invalid partition for non-digit input as well
    •        .... -2 -1  0 1 .............. 12 13  14  15 .....       
    • --------------|-------------------|---------------------
    • invalid partition 1      valid partition    invalid partition 2
  • 56. Final comments on testing
    • Testing good practice:
    • Begins at component level & works out towards integration of whole system
    • Different testing techniques are appropriate at different times
    • Testing is done by the developers and (in large projects) an independent test group
    • Testing & debugging aren’t the same – but any test strategy has debugging in it
  • 57.
    • The last part of the course will look at:
      • user evaluation strategies and at
      • evolution of software projects
  • 58. Before tomorrow, you must:
    • Look over the course notes so far
    • Tomorrow you will get the assignment
      • There’ll be time to do it in class in the next 2 weeks, with tutor support
      • There’ll be an interim marking next Tuesday
    • The assignment tests ALL the module learning outcomes
    • This means we’ll be asking you to give a full account of what you’ve learned
      • All aspects over the full SDLC