0
Automated Testing with Team Test Unit, Web, Performance, Load, Manual, and Ordered Tests  Code Coverage and Code Analysis
Quality Challenges “ Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an esti...
How Does VSTS/TFS Help? <ul><li>Increased Communication and Integration… </li></ul><ul><li>Code Analysis Tools </li></ul><...
Unit Testing <ul><li>Each unit is tested independently </li></ul><ul><li>A large percentage of defects are identified duri...
Benefits of Automated Unit Testing <ul><li>Repeatable unit that verifies code still works </li></ul><ul><li>Confirm code i...
Unit Test Types <ul><li>Class (standard) </li></ul><ul><ul><li>Test the properties and methods of a class </li></ul></ul><...
Extensions of Unit Testing <ul><li>Regression Testing </li></ul><ul><ul><li>Modifications are validated with regression te...
Writing Effective Unit Tests <ul><li>Atomic </li></ul><ul><ul><li>Should not require others tests to be run first </li></u...
Unit Test Conditions <ul><li>Success baseline </li></ul><ul><ul><li>Common, successful, most-likely call to your code </li...
Creating Unit Tests (1 of 2) <ul><li>Attributes are used to denote which methods and classes should be loaded for tests </...
Creating Unit Tests (2 of 2) <ul><li>Methods on the Assert type </li></ul><ul><ul><li>AreSame / AreNotSame </li></ul></ul>...
Unit Tests
Code Coverage <ul><li>Improves effectiveness of tests </li></ul><ul><li>Show a measurable indication of code that was cove...
Code Coverage
Static Code Analysis <ul><li>Provides a means to enforce coding standards </li></ul><ul><li>Configurable </li></ul><ul><li...
Static Code Analysis
Web Testing <ul><li>Record page navigation and user interactions with web pages </li></ul><ul><li>Run tests back to determ...
Web Testing
Load Testing <ul><li>Load Testing: ensure application works under expected, concurrent user load </li></ul><ul><li>Create ...
Stress Testing <ul><li>Stress Testing: determine breaking points in your application  </li></ul><ul><li>Find scalability i...
Performance Testing <ul><li>Test at single user </li></ul><ul><li>Test at normal load </li></ul><ul><li>Test at peak load ...
Load, Stress, and Performance Testing
Manual Test <ul><li>Tracked like other tests when executing test groups </li></ul><ul><li>Presented to the tester to enter...
Generic Test <ul><li>Wrap existing code and have that code executed as part of the testing process </li></ul><ul><li>Centr...
Ordered Test <ul><li>Test to group (and order) other tests </li></ul><ul><li>Results of ordered tests either succeed or fa...
Manual Test
Test Case Management <ul><li>Organize tests into lists </li></ul><ul><li>Run tests </li></ul><ul><li>Filter and group the ...
The Test Lab <ul><li>Clients </li></ul><ul><ul><li>Tester machine </li></ul></ul><ul><ul><li>Used to create and edit tests...
The Test Lab Database <ul><li>Run against a test version of the database </li></ul><ul><li>Test data should be same nature...
Test Lab Database Initialization
Upcoming SlideShare
Loading in...5
×

justin presentation Slideshare PPT upload June 25 Final one

483

Published on

Published in: Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
483
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Transcript of "justin presentation Slideshare PPT upload June 25 Final one"

    1. 1. Automated Testing with Team Test Unit, Web, Performance, Load, Manual, and Ordered Tests Code Coverage and Code Analysis
    2. 2. Quality Challenges “ Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product…an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects.” (Source: NIST 2002) Satisfying users Business interruption risks Ongoing maintenance costs rising Less avail. resources
    3. 3. How Does VSTS/TFS Help? <ul><li>Increased Communication and Integration… </li></ul><ul><li>Code Analysis Tools </li></ul><ul><li>Code Profiling Tools </li></ul><ul><li>Unit Testing and Code Coverage </li></ul><ul><li>Load Testing </li></ul><ul><li>Other Testing Tools </li></ul><ul><li>Test Case Management </li></ul>
    4. 4. Unit Testing <ul><li>Each unit is tested independently </li></ul><ul><li>A large percentage of defects are identified during unit testing </li></ul><ul><li>Automatic and repeatable code testing </li></ul><ul><li>Simplifies integration </li></ul><ul><li>Self documenting </li></ul><ul><li>Auto-generate </li></ul>Isolate small portion of code & determine whether it works correctly
    5. 5. Benefits of Automated Unit Testing <ul><li>Repeatable unit that verifies code still works </li></ul><ul><li>Confirm code is tested </li></ul><ul><li>Build a set of regression tests </li></ul><ul><li>Make changes with confidence </li></ul><ul><li>Aid in understanding code </li></ul><ul><li>Limits: integration, web and Windows UI testing, load, performance testing </li></ul>
    6. 6. Unit Test Types <ul><li>Class (standard) </li></ul><ul><ul><li>Test the properties and methods of a class </li></ul></ul><ul><li>Data-driven </li></ul><ul><ul><li>Bind unit test parameters to a datasource </li></ul></ul><ul><li>ASP.NET </li></ul><ul><ul><li>Test classes (or business logic) inside an ASP.NET application </li></ul></ul><ul><ul><li>Run in the context of the web server (ASP objects available) </li></ul></ul><ul><li>Web Services </li></ul><ul><ul><li>Define a web reference to the service </li></ul></ul>
    7. 7. Extensions of Unit Testing <ul><li>Regression Testing </li></ul><ul><ul><li>Modifications are validated with regression tests </li></ul></ul><ul><ul><li>Write new code and verify system continues to function correctly </li></ul></ul><ul><li>Integration Testing </li></ul><ul><ul><li>Helps validate how tested components interact with one another </li></ul></ul><ul><ul><li>Can identify problems that occur when units are combined </li></ul></ul><ul><li>Scenario Testing </li></ul><ul><ul><li>Combine sequence of unit tests to cover a scenario </li></ul></ul>
    8. 8. Writing Effective Unit Tests <ul><li>Atomic </li></ul><ul><ul><li>Should not require others tests to be run first </li></ul></ul><ul><li>Cover all cases </li></ul><ul><ul><li>One test for each sceneario </li></ul></ul><ul><ul><li>Cover all conditions, exceptions, nulls, etc. </li></ul></ul><ul><li>Able to re-run without configuration </li></ul><ul><ul><li>Database create/read/update/delete without having to modify the database before or after the test is run </li></ul></ul><ul><li>Test a common application state </li></ul>Unit tests should be …
    9. 9. Unit Test Conditions <ul><li>Success baseline </li></ul><ul><ul><li>Common, successful, most-likely call to your code </li></ul></ul><ul><li>Parameter mix </li></ul><ul><ul><li>Pass a varied set of parameters </li></ul></ul><ul><ul><li>Data-driven </li></ul></ul><ul><li>Bounds checking </li></ul><ul><ul><li>Stress the upper and lower limits of your parameters </li></ul></ul><ul><li>Null values </li></ul><ul><ul><li>Determine outcome of passing nulls to parameters </li></ul></ul><ul><li>Error conditions </li></ul><ul><ul><li>Trigger error conditions and validate expected outcomes </li></ul></ul><ul><li>Code coverage scenarios </li></ul><ul><ul><li>Test all conditions in your code </li></ul></ul>
    10. 10. Creating Unit Tests (1 of 2) <ul><li>Attributes are used to denote which methods and classes should be loaded for tests </li></ul><ul><ul><li>TestClass – denotes a class for testing </li></ul></ul><ul><ul><li>TestMethod – denotes a method for testing </li></ul></ul><ul><ul><ul><li>must return void (Sub for VB.NET) and have no parameters </li></ul></ul></ul><ul><li>Certain attributes are associated with initialization </li></ul><ul><ul><li>TestInitialize – run before each test </li></ul></ul><ul><ul><li>TestCleanup – run after each test </li></ul></ul><ul><ul><li>ClassInitialize – run once before running any tests in the class </li></ul></ul><ul><ul><li>ClassCleanup – run once after running all tests in the class </li></ul></ul><ul><li>Assert – static type used for asserting test values </li></ul><ul><li>ExpectedException – attribute used for determining the exception that should be thrown from the test </li></ul>
    11. 11. Creating Unit Tests (2 of 2) <ul><li>Methods on the Assert type </li></ul><ul><ul><li>AreSame / AreNotSame </li></ul></ul><ul><ul><li>AreEqual / AreNotEqual </li></ul></ul><ul><ul><li>IsNull / IsNotNull </li></ul></ul><ul><ul><li>IsInstanceOfType / IsNotInstanceOfType </li></ul></ul><ul><ul><li>IsTrue / IsFalse </li></ul></ul><ul><li>Other Assert types </li></ul><ul><ul><li>CollectionAssert – collection equivalency </li></ul></ul><ul><ul><li>StringAssert – string comparison/regex matching </li></ul></ul><ul><li>Web test types </li></ul><ul><ul><li>HtmlDocument – grants access to tags in the html document </li></ul></ul><ul><ul><li>Validate*/Extract* - types used for reading field values from the request and response </li></ul></ul>
    12. 12. Unit Tests
    13. 13. Code Coverage <ul><li>Improves effectiveness of tests </li></ul><ul><li>Show a measurable indication of code that was covered by unit tests </li></ul><ul><ul><li>Strive for 75% and higher </li></ul></ul><ul><li>Shows the ratio of executed logic to the total logic </li></ul><ul><li>Can be used in conjunction with unit and load testing </li></ul><ul><li>Helps to find unused code </li></ul>Determine code that is exercised by tests
    14. 14. Code Coverage
    15. 15. Static Code Analysis <ul><li>Provides a means to enforce coding standards </li></ul><ul><li>Configurable </li></ul><ul><li>Reduction in code reviews for small issues </li></ul><ul><li>Provides guidance on how to fix </li></ul><ul><li>Define a check-in policy to enforce code analysis pass </li></ul><ul><li>Coverage includes issues with design, globalization, interoperability, maintainability, naming, performance, reliability, security, and usage </li></ul><ul><li>Warning: developers can supress messages with attirbute [SuppressMessage(&quot;AdventureWorks.Rules&quot;, &quot;AW14441&quot;)] </li></ul><ul><li>MORE INFO see, “Writing Quality Code” in TFS documentation </li></ul>
    16. 16. Static Code Analysis
    17. 17. Web Testing <ul><li>Record page navigation and user interactions with web pages </li></ul><ul><li>Run tests back to determine errors </li></ul><ul><li>Seed tests from a database </li></ul><ul><li>Run tests from a group of users </li></ul><ul><li>Validation rules </li></ul><ul><li>Extractions </li></ul>
    18. 18. Web Testing
    19. 19. Load Testing <ul><li>Load Testing: ensure application works under expected, concurrent user load </li></ul><ul><li>Create test cases that simulate real user conditions </li></ul><ul><li>Distribute tests across cases based on user behavior </li></ul><ul><li>Assume standard think times </li></ul><ul><li>Distribute across connection types </li></ul><ul><li>Simulate an application performance in a production environment </li></ul><ul><li>Provide repository to look at a performance trend over time, to see if changes are helping or hurting </li></ul>Simulate multiple users against an application simultaneously
    20. 20. Stress Testing <ul><li>Stress Testing: determine breaking points in your application </li></ul><ul><li>Find scalability issues before the application is deployed </li></ul><ul><li>Step (increase) user load over time, monitor </li></ul><ul><li>Find breaking point and use to monitor application </li></ul><ul><ul><li>Change software </li></ul></ul><ul><ul><li>Add hardware </li></ul></ul>
    21. 21. Performance Testing <ul><li>Test at single user </li></ul><ul><li>Test at normal load </li></ul><ul><li>Test at peak load (1.5x normal load) </li></ul><ul><li>Measure against expected performance </li></ul><ul><ul><li>Peak load should allow for a 20-30% increase in metric </li></ul></ul><ul><li>Set thresholds on the output to avoid digging through data </li></ul>
    22. 22. Load, Stress, and Performance Testing
    23. 23. Manual Test <ul><li>Tracked like other tests when executing test groups </li></ul><ul><li>Presented to the tester to enter and confirm results </li></ul><ul><li>Defined as text file or Word document </li></ul>
    24. 24. Generic Test <ul><li>Wrap existing code and have that code executed as part of the testing process </li></ul><ul><li>Centralized results of all tests </li></ul>
    25. 25. Ordered Test <ul><li>Test to group (and order) other tests </li></ul><ul><li>Results of ordered tests either succeed or fail as a group </li></ul><ul><li>Used for scenario and module-level testing </li></ul><ul><li>Add any test except a load test to an ordered test </li></ul>
    26. 26. Manual Test
    27. 27. Test Case Management <ul><li>Organize tests into lists </li></ul><ul><li>Run tests </li></ul><ul><li>Filter and group the display of the tests </li></ul><ul><li>Import additional tests </li></ul><ul><li>Export tests </li></ul>Organize and manage test cases
    28. 28. The Test Lab <ul><li>Clients </li></ul><ul><ul><li>Tester machine </li></ul></ul><ul><ul><li>Used to create and edit tests / view results </li></ul></ul><ul><li>Servers </li></ul><ul><ul><li>Deployed test environment (web servers, load balancer, database server, etc.) </li></ul></ul><ul><li>Agents </li></ul><ul><ul><li>Computers running tests (multiple-threads, each a user) </li></ul></ul><ul><ul><li>Listen for commands from controller </li></ul></ul><ul><ul><li>Execute tests and pass results back to controller </li></ul></ul><ul><li>Controller </li></ul><ul><ul><li>Central computer that administers agents and collects results </li></ul></ul><ul><ul><li>Distributes load by applying weights to agents </li></ul></ul>
    29. 29. The Test Lab Database <ul><li>Run against a test version of the database </li></ul><ul><li>Test data should be same nature and size as production </li></ul><ul><li>Test a steady state of the database </li></ul><ul><li>Automate the initialization of the database </li></ul><ul><li>Extract data using rules to obfuscate personal information </li></ul><ul><ul><li>Visual Studio Database Developer </li></ul></ul>
    30. 30. Test Lab Database Initialization
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×