Your SlideShare is downloading. ×
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
SoftwareTesting-cs16..
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

SoftwareTesting-cs16..

177

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
177
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1.
    • Jason I. Hong
    • CS169 - Software Engineering, Fall 1998
    Testing Strategies
  • 2. Types of Testing
  • 3. Types of Testing
    • Scope
      • Unit Testing tests a subroutine, module, or class
      • Integration Testing tests multiple units together
      • System Testing tests entire system as a whole
    • Some Testing Goals
      • Functional Testing tests functions in system
      • Performance Testing tests system performance
        • primarily execution time, memory footprint
      • Reliability Testing certifies system reliability
        • predictable execution time, low number of failures, etc.
  • 4. Black Box Testing
    • Define test cases based only on specifications
    • Try all boundary conditions
      • Go from empty to full, full to empty
      • Array bounds
      • Empty or null values
    • Should be done by someone other than the module programmer
      • Needs a shift in mindset away from programming
      • Programmers don’t want to find bugs, testers do
  • 5. White Box Testing
    • Define test cases by looking at code
    • Not as useful as code review or code coverage
    • Useful for regression testing
  • 6. Code Coverage
    • Create test cases that execute every block of code at least once
    • Complete code coverage does not imply correct code
    • Ideally, we would like to test every path but this is exponential
      • e.g. all possibilities in switch statements
    • Useful for regression testing
  • 7. Code Review
    • Read through someone else’s code
      • Extremely effective at finding defects
    • Look for common errors
      • Possible unhandled error conditions
      • Potential array out of bounds (e.g. gets)
      • Bad or dangerous type casting
      • Memory leaks (e.g. alloc w/o dealloc)
      • Too many global variables (high complexity)
      • Not cleaning up after self (e.g. goto, return)
      • ...
  • 8. Regression Testing
    • Maintain suite of tests
    • Build product periodically and run test suite
      • Automatically verify that output is correct
      • Find reason for failure (test wrong or code wrong)
    • Good metric for quality
      • Can always tell if going forward or not
      • Requires significant process maturity from company
    • Continually add new tests as bugs are fixed
      • Ensures that old bugs do not reappear
      • Forces discipline on developers
  • 9. Stress Testing
    • Place large load on product (typically a server)
    • Each client sends multiple requests
      • Some valid requests, some junk requests
    • Look for:
      • Increasing memory footprint (memory leak?)
      • Running out of resources (sockets, file descriptor)
      • Increased response time (slows over time?)
      • Incorrect results (possible race conditions?)
  • 10. User Testing
    • Perhaps the most important kind of testing
      • Can customers use the product easily?
      • How many errors do users make?
      • How long does it take users to learn the system?
      • Do customers like the product? Enjoy using it?
      • Without customers, you have no product.
    • Best strategy is iterative development
      • Expensive to change product at end, so build cheap prototypes and evaluate with users at beginning
      • Hire real designers! Programmers != designers
      • Take CS160 for more information
  • 11. Alpha and Beta Testing
    • Alpha testing
      • Product is released to very small and focused group
    • Beta testing
      • Product is released to wider range of people
    • Difficult to make significant changes once Alpha and Beta Testing is reached
      • Use for polishing, finding compatibility problems, fixing simple errors, and finding workarounds
      • Do not use for evaluating usability or quality, a common mistake still made by many companies
  • 12. Some Statistics
    • Unit testing finds 10-50% of defects
    • System testing finds 20-60% of defects
    • Combined rate < 60% of defects
    • Informal code review finds 30-70% of defects
    • Formal inspection finds 60-90% of defects, net savings of 10-30%
    • Reviews are more cost effective than testing
      • Using reviews with testing helps software quality
    See Rapid Development pp72-75
  • 13. Development Notes
    • Back-to-Back algorithms
      • Use two implementations and compare results
      • Microsoft Excel does this for cell calculations
      • UWash Kimera Verifier found bugs in Netscape, Sun, and Microsoft implementations of Java
    • Use Static Verification
      • Use tools that go through code statically
      • Have no errors or warnings on Lint and gcc -Wall
    • Self-Testing Code
      • Have each class be able to test itself
      • Each Java class can have its own main() method
  • 14. Development Notes (cont.)
    • Use good tools
      • Purify, test harnesses, code coverage analysis, etc
    • Iterative Development
      • Plan a little, build a little, verify a little, and repeat
      • Fix defects as you find them , not “at the end”
      • If not, others will waste time on known problem
    • 80/20 rule applies for defects
      • 80% of bugs likely to be in 20% of code
      • Use defect tracking to find this code
      • If a unit seems to have lots of errors, it may be easier just to rewrite from scratch!
  • 15. Testing Process Notes
    • Begin test planning from beginning
      • Start test cases once requirements are known
      • Use User Manual and Specification Document
      • Microsoft uses one tester per developer
      • Space Shuttle people use ten testers per developer
    • Daily Build
      • Product is built every day and put through tests
      • Supports incremental development
      • Minimizes integration (prevents dis-integration )
      • Provides project visibility
  • 16. Testing Process Notes (cont.)
    • Find the reason for error and ask questions
      • Is it a common occurrence? Mistake? Sloppiness?
      • Programmers fix own bugs at Microsoft, prevents programmer from introducing new bugs elsewhere
      • How much will it cost to fix? Is it worth it?
      • How can bugs like this can be prevented in future?
    • Keep a database of bugs
      • Track all defects to closure
      • Graph open bugs and closed bugs over time
  • 17. Testing Process Notes (cont.)
    • Remember that testing only shows the presence of defects, not absence
      • Testing assesses quality, does not assure quality
    • Have an environment of quality
      • Inktomi hangs a toy pig over person’s cubicle for a short while if he/she messes up
      • Microsoft head once threw chair out window on broken build
      • Lives depend on Space Shuttle software developers
      • Lives may depend on software you write!
  • 18. Summary
    • Plan quality from the very beginning
    • Plan testing from the very beginning
      • People and resources needed, test cases
    • Preventing bugs is easier than finding bugs
      • Use assertions, preconditions, invariants, reviews
    • Use a variety of methods for reducing defects, assessing quality, and assuring quality
    • The earlier you find a defect, the cheaper and easier it is to correct
  • 19. References
    • Rapid Development by Steve McConnell
    • Code Complete by Steve McConnell
    • Software Engineering by Ian Sommerville
    • Software Project Dynamics by Jim McCarthy
    • Software Management by Donald Reifer
    • Construx Software
    • Eric Brewer (CS169 Spring 1998)

×