• Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
453
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
6
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Automated Testing with Team Test Unit, Web, Performance, Load, Manual, and Ordered Tests Code Coverage and Code Analysis
  • 2. Quality Challenges “ Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product…an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects.” (Source: NIST 2002) Satisfying users Business interruption risks Ongoing maintenance costs rising Less avail. resources
  • 3. How Does VSTS/TFS Help?
    • Increased Communication and Integration…
    • Code Analysis Tools
    • Code Profiling Tools
    • Unit Testing and Code Coverage
    • Load Testing
    • Other Testing Tools
    • Test Case Management
  • 4. Unit Testing
    • Each unit is tested independently
    • A large percentage of defects are identified during unit testing
    • Automatic and repeatable code testing
    • Simplifies integration
    • Self documenting
    • Auto-generate
    Isolate small portion of code & determine whether it works correctly
  • 5. Benefits of Automated Unit Testing
    • Repeatable unit that verifies code still works
    • Confirm code is tested
    • Build a set of regression tests
    • Make changes with confidence
    • Aid in understanding code
    • Limits: integration, web and Windows UI testing, load, performance testing
  • 6. Unit Test Types
    • Class (standard)
      • Test the properties and methods of a class
    • Data-driven
      • Bind unit test parameters to a datasource
    • ASP.NET
      • Test classes (or business logic) inside an ASP.NET application
      • Run in the context of the web server (ASP objects available)
    • Web Services
      • Define a web reference to the service
  • 7. Extensions of Unit Testing
    • Regression Testing
      • Modifications are validated with regression tests
      • Write new code and verify system continues to function correctly
    • Integration Testing
      • Helps validate how tested components interact with one another
      • Can identify problems that occur when units are combined
    • Scenario Testing
      • Combine sequence of unit tests to cover a scenario
  • 8. Writing Effective Unit Tests
    • Atomic
      • Should not require others tests to be run first
    • Cover all cases
      • One test for each sceneario
      • Cover all conditions, exceptions, nulls, etc.
    • Able to re-run without configuration
      • Database create/read/update/delete without having to modify the database before or after the test is run
    • Test a common application state
    Unit tests should be …
  • 9. Unit Test Conditions
    • Success baseline
      • Common, successful, most-likely call to your code
    • Parameter mix
      • Pass a varied set of parameters
      • Data-driven
    • Bounds checking
      • Stress the upper and lower limits of your parameters
    • Null values
      • Determine outcome of passing nulls to parameters
    • Error conditions
      • Trigger error conditions and validate expected outcomes
    • Code coverage scenarios
      • Test all conditions in your code
  • 10. Creating Unit Tests (1 of 2)
    • Attributes are used to denote which methods and classes should be loaded for tests
      • TestClass – denotes a class for testing
      • TestMethod – denotes a method for testing
        • must return void (Sub for VB.NET) and have no parameters
    • Certain attributes are associated with initialization
      • TestInitialize – run before each test
      • TestCleanup – run after each test
      • ClassInitialize – run once before running any tests in the class
      • ClassCleanup – run once after running all tests in the class
    • Assert – static type used for asserting test values
    • ExpectedException – attribute used for determining the exception that should be thrown from the test
  • 11. Creating Unit Tests (2 of 2)
    • Methods on the Assert type
      • AreSame / AreNotSame
      • AreEqual / AreNotEqual
      • IsNull / IsNotNull
      • IsInstanceOfType / IsNotInstanceOfType
      • IsTrue / IsFalse
    • Other Assert types
      • CollectionAssert – collection equivalency
      • StringAssert – string comparison/regex matching
    • Web test types
      • HtmlDocument – grants access to tags in the html document
      • Validate*/Extract* - types used for reading field values from the request and response
  • 12. Unit Tests
  • 13. Code Coverage
    • Improves effectiveness of tests
    • Show a measurable indication of code that was covered by unit tests
      • Strive for 75% and higher
    • Shows the ratio of executed logic to the total logic
    • Can be used in conjunction with unit and load testing
    • Helps to find unused code
    Determine code that is exercised by tests
  • 14. Code Coverage
  • 15. Static Code Analysis
    • Provides a means to enforce coding standards
    • Configurable
    • Reduction in code reviews for small issues
    • Provides guidance on how to fix
    • Define a check-in policy to enforce code analysis pass
    • Coverage includes issues with design, globalization, interoperability, maintainability, naming, performance, reliability, security, and usage
    • Warning: developers can supress messages with attirbute [SuppressMessage("AdventureWorks.Rules", "AW14441")]
    • MORE INFO see, “Writing Quality Code” in TFS documentation
  • 16. Static Code Analysis
  • 17. Web Testing
    • Record page navigation and user interactions with web pages
    • Run tests back to determine errors
    • Seed tests from a database
    • Run tests from a group of users
    • Validation rules
    • Extractions
  • 18. Web Testing
  • 19. Load Testing
    • Load Testing: ensure application works under expected, concurrent user load
    • Create test cases that simulate real user conditions
    • Distribute tests across cases based on user behavior
    • Assume standard think times
    • Distribute across connection types
    • Simulate an application performance in a production environment
    • Provide repository to look at a performance trend over time, to see if changes are helping or hurting
    Simulate multiple users against an application simultaneously
  • 20. Stress Testing
    • Stress Testing: determine breaking points in your application
    • Find scalability issues before the application is deployed
    • Step (increase) user load over time, monitor
    • Find breaking point and use to monitor application
      • Change software
      • Add hardware
  • 21. Performance Testing
    • Test at single user
    • Test at normal load
    • Test at peak load (1.5x normal load)
    • Measure against expected performance
      • Peak load should allow for a 20-30% increase in metric
    • Set thresholds on the output to avoid digging through data
  • 22. Load, Stress, and Performance Testing
  • 23. Manual Test
    • Tracked like other tests when executing test groups
    • Presented to the tester to enter and confirm results
    • Defined as text file or Word document
  • 24. Generic Test
    • Wrap existing code and have that code executed as part of the testing process
    • Centralized results of all tests
  • 25. Ordered Test
    • Test to group (and order) other tests
    • Results of ordered tests either succeed or fail as a group
    • Used for scenario and module-level testing
    • Add any test except a load test to an ordered test
  • 26. Manual Test
  • 27. Test Case Management
    • Organize tests into lists
    • Run tests
    • Filter and group the display of the tests
    • Import additional tests
    • Export tests
    Organize and manage test cases
  • 28. The Test Lab
    • Clients
      • Tester machine
      • Used to create and edit tests / view results
    • Servers
      • Deployed test environment (web servers, load balancer, database server, etc.)
    • Agents
      • Computers running tests (multiple-threads, each a user)
      • Listen for commands from controller
      • Execute tests and pass results back to controller
    • Controller
      • Central computer that administers agents and collects results
      • Distributes load by applying weights to agents
  • 29. The Test Lab Database
    • Run against a test version of the database
    • Test data should be same nature and size as production
    • Test a steady state of the database
    • Automate the initialization of the database
    • Extract data using rules to obfuscate personal information
      • Visual Studio Database Developer
  • 30. Test Lab Database Initialization