Visual Studio 2010 for testersPresentation Transcript
VSTS 2010 for Testers
Agenda• Introduction - Visual Studio 2010 for test• Details - Test Types• Details - Load Test• Details - Web Test• Details - Test Manager
Microsoft Strategy• Create one-stop solution for product development and test life cycles.• Design test tools for Test plan, test management, defects tracking and analyzing with just one tool - Visual Studio.• Capture the lucrative market of test tools which is dominated by IBM and HP.
Visual Studio Test Editions• Following editions of Visual studio contains “Test” related features. – VS Test Professional 2010 – VS Ultimate 2010• Both the editions contain a separate test management tool called as “Microsoft Test Manager” used for managing test plan, test cases, test labs etc.
Supported Test Types• Unit Test• Record-Playback Test (using Coded-UI)• Load Test• Web Performance Test• Database Unit Test
Test Flow• Create Test Plan linking to requirements.• Create Test Suites as per requirement modules.• Create and Manage Tests.• Run Test Cases and Submit Results.• Log Bugs/Defects.• Track test deliverables – Bugs, Test case execution report, Automation report tc.
Web Performance Testing• To find out performance of the web page under different load conditions.• Define baseline for accepted server loads under given conditions.• Determine compliance with requirements.• Improve overall quality of web application under test.
(contd.)• Performance of web application is found by creating 1. Web Test - Series of test sequence actions to be performed on the web site 2. Load Test - Bunching of Web tests adhering to business scenarios to carry out performance analysis at different loads.
Performance test flow• Identify Test Environment and acceptance criteria.• Create a web performance test for the web application under test. This test will then be used during load testing.• Create Load test for the specified load conditions – virtual users, load type, test distribution, step increments etc.
(contd.)• Configure Test Environments for the tests to be run.• Implement Test Design and execute the tests.• Analyze and Report load test results. Reprioritize and retest for verifying performance adherence of each part of web application under test.
Web Test Tasks• Record a Web Test.• Add Extraction, Validation rules to it.• Create Request and Web Test Plug-Ins.• Create Coded Web Test• Execute Web Test.
Load Test• To evaluate the performance of application under test under the specified load conditions within limits.• Test Types included for load test – Web Performance Test – Unit Test
Load Test Tasks• Configure Test Controller and Test agents.• Configure load test database.• Identify the load test scenario.• Specify the counter sets to capture specific test parameters.• Configure load test run settings.• Run load tests and Analyze test results.
Load Test Considerations• Determine acceptance criteria – Response Time – Throughput – Resource utilization – Maximum user load• Identify Business scenarios• Create workload model
(contd.)• Find Load distribution pattern – Constant – Step – Goal-based• Identify Metrics – Connection type – Sample rate – Think time
Load Test Wizard• Run load test wizard• Specify new load test scenario.• Specify initial load patterns.• Specify Test Mix model• Specify Test Mix• Specify Network Mix• Specify Browser Mix
(contd.)• Configure test computers.• Configure Counter sets on each computer.• Specify Initial run settings.• Optional – Add additional run settings.• Modify Run settings.
Analyze Load Test Results• Using load test editor - Add threshold rule to the load test.• Using Load Test Analyzer - Enable threshold violation to graphs.• Calculate response time.• Calculate deviation from SLA time.
Microsoft Test Manager (MTM)• New product shipped with VSTS 2010.• Developed on WPF.• Used for the test management activities like creating test plans, test suites, test configurations, test cases, test steps etc.• Also useful in diagnosing test data impact, automation test runs, effort planning.
MTM tasks• Create Test Plan• Create test suites• Create/add test cases.• Test effort distribution• Assign test cases to testers• Create/add machine configurations.• Assign automated tests
(contd.)• Assign build definitions.• Run manual and automated tests.• Associates bugs to test cases• Analyze test results.• Create Exploratory testing scenarios.• Perform Exploratory testing.
MTM for manual tests• Connect to Team Project Site• Create Test Plan• Add details to Test Plan• Add test cases• Run tests manually• Log bugs and associate them with the testcase• Keep tap on the test progress
MTM for automated tests• Connect to Team Project Site• Create Test Plan• Add details to Test Plan• Create automated (Unit/Coded-UI) test cases using VSTS 2010.• Add project to TFS• Generate a build.
(contd.)• Queue this build• Associate the automated test cases with the manual test cases already present in test plan.• Add build definition to test plan.• Run automated tests.• Analyze test results.