Your SlideShare is downloading. ×
Paper Ps
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Paper Ps


Published on

Published in: Business

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Transcript

    • 1. Automated Testing with Team Test Unit, Web, Performance, Load, Manual, and Ordered Tests Code Coverage and Code Analysis
    • 2. Quality Challenges “ Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product…an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects.” (Source: NIST 2002) Satisfying users Business interruption risks Ongoing maintenance costs rising Less avail. resources
    • 3. How Does VSTS/TFS Help?
      • Increased Communication and Integration…
      • Code Analysis Tools
      • Code Profiling Tools
      • Unit Testing and Code Coverage
      • Load Testing
      • Other Testing Tools
      • Test Case Management
    • 4. Unit Testing
      • Each unit is tested independently
      • A large percentage of defects are identified during unit testing
      • Automatic and repeatable code testing
      • Simplifies integration
      • Self documenting
      • Auto-generate
      Isolate small portion of code & determine whether it works correctly
    • 5. Benefits of Automated Unit Testing
      • Repeatable unit that verifies code still works
      • Confirm code is tested
      • Build a set of regression tests
      • Make changes with confidence
      • Aid in understanding code
      • Limits: integration, web and Windows UI testing, load, performance testing
    • 6. Unit Test Types
      • Class (standard)
        • Test the properties and methods of a class
      • Data-driven
        • Bind unit test parameters to a datasource
      • ASP.NET
        • Test classes (or business logic) inside an ASP.NET application
        • Run in the context of the web server (ASP objects available)
      • Web Services
        • Define a web reference to the service
    • 7. Extensions of Unit Testing
      • Regression Testing
        • Modifications are validated with regression tests
        • Write new code and verify system continues to function correctly
      • Integration Testing
        • Helps validate how tested components interact with one another
        • Can identify problems that occur when units are combined
      • Scenario Testing
        • Combine sequence of unit tests to cover a scenario
    • 8. Writing Effective Unit Tests
      • Atomic
        • Should not require others tests to be run first
      • Cover all cases
        • One test for each sceneario
        • Cover all conditions, exceptions, nulls, etc.
      • Able to re-run without configuration
        • Database create/read/update/delete without having to modify the database before or after the test is run
      • Test a common application state
      Unit tests should be …
    • 9. Unit Test Conditions
      • Success baseline
        • Common, successful, most-likely call to your code
      • Parameter mix
        • Pass a varied set of parameters
        • Data-driven
      • Bounds checking
        • Stress the upper and lower limits of your parameters
      • Null values
        • Determine outcome of passing nulls to parameters
      • Error conditions
        • Trigger error conditions and validate expected outcomes
      • Code coverage scenarios
        • Test all conditions in your code
    • 10. Creating Unit Tests (1 of 2)
      • Attributes are used to denote which methods and classes should be loaded for tests
        • TestClass – denotes a class for testing
        • TestMethod – denotes a method for testing
          • must return void (Sub for VB.NET) and have no parameters
      • Certain attributes are associated with initialization
        • TestInitialize – run before each test
        • TestCleanup – run after each test
        • ClassInitialize – run once before running any tests in the class
        • ClassCleanup – run once after running all tests in the class
      • Assert – static type used for asserting test values
      • ExpectedException – attribute used for determining the exception that should be thrown from the test
    • 11. Creating Unit Tests (2 of 2)
      • Methods on the Assert type
        • AreSame / AreNotSame
        • AreEqual / AreNotEqual
        • IsNull / IsNotNull
        • IsInstanceOfType / IsNotInstanceOfType
        • IsTrue / IsFalse
      • Other Assert types
        • CollectionAssert – collection equivalency
        • StringAssert – string comparison/regex matching
      • Web test types
        • HtmlDocument – grants access to tags in the html document
        • Validate*/Extract* - types used for reading field values from the request and response
    • 12. Unit Tests
    • 13. Code Coverage
      • Improves effectiveness of tests
      • Show a measurable indication of code that was covered by unit tests
        • Strive for 75% and higher
      • Shows the ratio of executed logic to the total logic
      • Can be used in conjunction with unit and load testing
      • Helps to find unused code
      Determine code that is exercised by tests
    • 14. Code Coverage
    • 15. Static Code Analysis
      • Provides a means to enforce coding standards
      • Configurable
      • Reduction in code reviews for small issues
      • Provides guidance on how to fix
      • Define a check-in policy to enforce code analysis pass
      • Coverage includes issues with design, globalization, interoperability, maintainability, naming, performance, reliability, security, and usage
      • Warning: developers can supress messages with attirbute [SuppressMessage("AdventureWorks.Rules", "AW14441")]
      • MORE INFO see, “Writing Quality Code” in TFS documentation
    • 16. Static Code Analysis
    • 17. Web Testing
      • Record page navigation and user interactions with web pages
      • Run tests back to determine errors
      • Seed tests from a database
      • Run tests from a group of users
      • Validation rules
      • Extractions
    • 18. Web Testing
    • 19. Load Testing
      • Load Testing: ensure application works under expected, concurrent user load
      • Create test cases that simulate real user conditions
      • Distribute tests across cases based on user behavior
      • Assume standard think times
      • Distribute across connection types
      • Simulate an application performance in a production environment
      • Provide repository to look at a performance trend over time, to see if changes are helping or hurting
      Simulate multiple users against an application simultaneously
    • 20. Stress Testing
      • Stress Testing: determine breaking points in your application
      • Find scalability issues before the application is deployed
      • Step (increase) user load over time, monitor
      • Find breaking point and use to monitor application
        • Change software
        • Add hardware
    • 21. Performance Testing
      • Test at single user
      • Test at normal load
      • Test at peak load (1.5x normal load)
      • Measure against expected performance
        • Peak load should allow for a 20-30% increase in metric
      • Set thresholds on the output to avoid digging through data
    • 22. Load, Stress, and Performance Testing
    • 23. Manual Test
      • Tracked like other tests when executing test groups
      • Presented to the tester to enter and confirm results
      • Defined as text file or Word document
    • 24. Generic Test
      • Wrap existing code and have that code executed as part of the testing process
      • Centralized results of all tests
    • 25. Ordered Test
      • Test to group (and order) other tests
      • Results of ordered tests either succeed or fail as a group
      • Used for scenario and module-level testing
      • Add any test except a load test to an ordered test
    • 26. Manual Test
    • 27. Test Case Management
      • Organize tests into lists
      • Run tests
      • Filter and group the display of the tests
      • Import additional tests
      • Export tests
      Organize and manage test cases
    • 28. The Test Lab
      • Clients
        • Tester machine
        • Used to create and edit tests / view results
      • Servers
        • Deployed test environment (web servers, load balancer, database server, etc.)
      • Agents
        • Computers running tests (multiple-threads, each a user)
        • Listen for commands from controller
        • Execute tests and pass results back to controller
      • Controller
        • Central computer that administers agents and collects results
        • Distributes load by applying weights to agents
    • 29. The Test Lab Database
      • Run against a test version of the database
      • Test data should be same nature and size as production
      • Test a steady state of the database
      • Automate the initialization of the database
      • Extract data using rules to obfuscate personal information
        • Visual Studio Database Developer
    • 30. Test Lab Database Initialization