Your SlideShare is downloading. ×
VV07
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
144
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Testing Automation CFICSE VV07 October 1999 Dr. Terry Shepard Royal Military College of Canada Electrical and Computer Engineering 613-541-6000 ext. 6031
  • 2. Testing Automation Topics
    • Motivation for Automation
    • Design for Testability
    • What can be automated?
    • Context and Needs
    • Intrusive vs. non-intrusive
    • Software Test Technologies Report
      • Classification of testing tools
  • 3. Motivation for Testing Automation
    • reduce costs
    • ensure repeatability of tests
    • ensure that test scenarios are followed exactly
    • increase number of tests that can be run by a deadline
    • improve reliability estimation
  • 4. Design for Testability
    • Common for computer hardware
    • Primarily means
      • Controllability
      • Observability
      • Well behaved interfaces between modules
    • Determines what can easily be automated
      • e.g. instrumentation code built into source code
  • 5. What can be automated?
    • Test execution
    • Comparison of actual and expected results
    • Logging of state at the time of a failed test
      • recovery for debugging purposes
    • Management of test logs
    • Collection of metrics
    • Version control (in part)
    • Test case selection: hard!
  • 6. Automated support can be provided for other tasks
    • Test planning and budgeting
    • Test setup procedures
    • Test adequacy decisions
    • Test scenarios with external dependencies
  • 7. Context and Need
    • In a given project, the context will determine candidate testing tools
      • constraints include language, platform, OS, …
    • Given a set of constraints, a market search and evaluation can be undertaken
    • Broad range of applicability and variety of tools makes needs assessment hard
    • Cost justification should be applied
  • 8. Intrusive vs. Non-Intrusive
    • Intrusion is generally not possible in an operational setting
    • Build black box recorder capabilities into the software?
    • Continuing hardware performance improvements will make this increasingly feasible
  • 9. Software Test Technologies Report
    • August 1994
    • Software Technology Support Center, Hill AFB, Air Logistics Command, Ogden, Utah
      • Online at www.stsc.hill.af.mil
      • STSC has an Annual Conference, publishes Crosstalk, …
    • No update in classification since 1994
  • 10. 1994 STSC Test Tools Classification
    • Test Resource Management
    • Requirements and Design Test Support
    • Implementation and Maintenance Test Support
      • Compilers
      • Source Code Static Analyzers
      • Test Preparation
      • Test Execution (Dynamic Analyzers)
      • Test Evaluators
  • 11. Test Resource Management Tools
    • Configuration Managers
      • test version control
      • defect list management
      • change control management
    • Project Managers
      • can help to elevate the visibility of testing early in a project
  • 12. Requirements and Design Test Support Tools
    • Analyzers for Software Plans, Requirements, and Designs
    • System/Prototype Simulators
    • Requirements Tracers
    • Requirements-Based Test Case Generators
    • Test Planners
  • 13. Implementation and Maintenance Test Support Tools
    • Compilers (syntax checking)
    • Source Code Static Analyzers
    • Test Preparation Tools
    • Test Execution Tools (Dynamic Analyzers)
    • Test Evaluators
  • 14. Implementation and Maintenance Test Support Tools
    • Compilers (syntax checking)
    • Source Code Static Analyzers
    • Test Preparation Tools
    • Test Execution Tools (Dynamic Analyzers)
    • Test Evaluators
  • 15. Source Code Static Analyzers
    • Auditors (enforce style manuals)
    • Complexity Measurers
    • Cross Referencing Tools
      • e.g. to verify uses
    • Size Measurers
    • Structure Checkers
    • Syntax and Semantics Analyzers
  • 16. Implementation and Maintenance Test Support Tools
    • Compilers (syntax checking)
    • Source Code Static Analyzers
    • Test Preparation Tools
    • Test Execution Tools (Dynamic Analyzers)
    • Test Evaluators
  • 17. Test Preparation Tools
    • Data Extractors
    • Requirements-Based Test Case Generators
    • Test Data Generators
    • Test Planners
  • 18. Implementation and Maintenance Test Support Tools
    • Compilers (syntax checking)
    • Source Code Static Analyzers
    • Test Preparation Tools
    • Test Execution Tools (Dynamic Analyzers)
    • Test Evaluators
  • 19. Test Execution Tools (Dynamic Analyzers) - Part 1
    • Assertion Analyzers
    • Capture-Replay Tools
    • Coverage/Frequency Analyzers
    • Debuggers
    • Emulators
    • Network Analyzers
    • Performance/Timing Analyzers
  • 20. Test Execution Tools (Dynamic Analyzers) - Part 2
    • Run-Time Error Checkers
    • Simulators
    • Status Displayer/Session Documenters
    • Test Execution Managers
    • Validation Suites
  • 21. Implementation and Maintenance Test Support Tools
    • Compilers (syntax checking)
    • Source Code Static Analyzers
    • Test Preparation Tools
    • Test Execution Tools (Dynamic Analyzers)
    • Test Evaluators
  • 22. Test Evaluators
    • Comparators
    • Data Reducers and Analyzers
    • Defect/Change Trackers
  • 23. Other sources for testing tool info
    • comp.software.testing
    • www.cs.queensu.ca/shepard
    • www.soft.com
    • www.sqe.com
    • www.methods-tools.com
    • article by Robert Poston: A Guided Tour of Software Testing Tools
  • 24. Testing Tool Market (Ovum - www.ovum.com)
    • Currently $450M, growing at 30% per year
    • dominant players (with 60% of the total market) are:
      • Mercury Interactive
      • Rational
      • Compuware
      • Segue

×