View stunning SlideShares in full-screen with the new iOS app!Introducing SlideShare for AndroidExplore all your favorite topics in the SlideShare appGet the SlideShare app to Save for Later — even offline
View stunning SlideShares in full-screen with the new Android app!View stunning SlideShares in full-screen with the new iOS app!
Using Microsoft Test Manager, Visual Studio 2010, and TFS 2010 to Improve Software Quality Anna Russo Sr. Application Lifecycle Management Consultant Imaginet email@example.com Blog- http://ImprovingSoftwareQuality.blogspot.com www.Imaginet.com
The enclosed material is proprietary to Imaginet and is therefore copyrighted. This document may not be disclosed in any manner to anyone other than the addressee and the employees or representatives of Notion Solutions who are recipients. This document may not be used in any manner other than for the purpose it was distributed. Any unauthorized use, reproduction or transmission in any form is strictly prohibited.
7 Steps to Improving Quality Step 1- Work Management Step 2- Defining/Reporting of “Done” Step 3- Automated Builds Step 4- Manual & Automated Testing Tools Step 5- Developer Quality Tools Step 6- Lab Management Step 7- Working Smart
Dashboard: SharePoint Server Dashboard title What is this dashboard about? Additional dashboards Reports shown side-by-side in context of other relevant metrics Yes, we have a burndown! Are we making progress on tasks? Are we making progress on our User Stories? How are we doing on open issues? Reporting in 2010 | Custom Reports | Making Transparency Work
Are we making progress on running test plans? How are our builds doing over time? Are we fixing bugs? What’s the quality of our bug fixes? What’s the value?
Step 3: Automated Build What is our traceability from Dev to QA?
Powerful Build Engine Build Explorer Centralized build information Build Report to display code and work items Executes Tests Unit test Automated Coded UI test Web tests Load tests Third party tool tests Deploy Physical or Virtual machines Virtual Lab machines- Restores to Clean Baseline
Gated Check-in Build with TFS 2005/2008 Check-In Denied Build Fails New Code is Checked In Build is Executed Developer Makes Changes Build Fails Build is Executed Developer Makes Changes The code that caused the build to fail is still checked-in. Build Succeeds New Code is Checked In
Build Report Check code in TFS+ Associate Work Item+ TFS Build script= Build Report
Step 4: Manual Testing Tools How do we keep track of Testing?
Testing Done Right Test Manager Manual Testing for the 21st century Plan, Manage and Execute tests from one place File rich & actionable bug during manual testing Perform Exploratory testing
Testing Tools MTM Manual Testing Tools Test Plan Management Test Case Management Shared Steps Management Test Configuration Management Test Fast Forwarding Test Collectors Test Lab Management Visual Studio Automated Testing Tools Automated Coded UI Testing Web Testing Load Testing Generic Tests Developer Unit Tests Database Unit Tests
Test Process Release Cycle Requirements Build/ Environment Test Run Improve Test Coverage
The Problem Development and test have built a wall: Development work happens on one side of the wall. Testing happens on the other. Builds and test results are passed back and forth, but not in a collaborative fashion.
Development Perspective Works on my machine! What build is being tested? How do I repro this? Insufficient bug details Unknown machine configuration Unknown runtime details No screenshot I already fixed this bug! This bug has been here for 2 months Why does testing take so long? The testers are clueless twits that don’t even know what good software is!
Test Perspective This is still broken? They said they fixed it! Dev teams go over schedule, testing time is reduced We can’t improve test coverage You want me to come to your desk to show you how to repro this bug? What is supposedly fixed in this new build? Is the new build deployed to our QA environment? The developers are clueless twits that don’t even know what good software is!
Rich Bugs NEW! Repro Steps Screenshot Video Capture System Info Debug Log Test Environment Event Log Action Recording
Automated Coded UI Test NEW! Recorded Actions from Manual Tester Use Visual Studio Recorder Automated Coded UI Test + = Validation Code
Coded UI Testing Enables verification of UI functionality. Record test actions and VS will generate the resulting code. Assertions can be added to verify functionality. Like all VS tests, results can be stored in TFS.
Best Practices (From MSDN) Create each test case as a sequence of Recorded Methods. Use meaningful method names. Try to limit the length of each recorded method to less than 10 actions. Do not modify the RecordedMethods.cs file. Do not edit the UIMap.Designer.cs file.
Playback.PlaybackSettings… Continue on Error DelayBetweenActions MatchExactHierarchy SearchTimeout SmartMatchOptions TopLevelWindow Control None ThinkTimeMultiplier WaitForReadyTimeout
WaitForControlReady() – This waits for the control to be ready to accept mouse/keyboard input.
WaitForControlEnabled() – This waits for the control to be enabled.
WaitForControlExist() – This waits for the control to exist on the UI.
WaitForControlNotExist() – This waits till the control cease to exist on the UI. For example, you can use this for progress dialog to go away.
WaitForControlPropertyEqual(string propertyName, object propertyValue) – This waits for the specified property of the control to have the given value.
WaitForControlCondition(Predicate<UITestControl> conditionEvaluator) –wait till the status text is “Succeeded” or “Failed”.
Demo Automated Coded UI Testing
Step 5: Developer Quality Tools How can developers contribute to the quality?
Create code, Don’t debug it Dependency Diagrams Understand code dependencies visually and understand where to make changes Gated Check-In Stop build breaks before they happen Code Analysis, Code Profiling, Code Metrics Help you find and fix potential problems before you make them
Spend Less time Debugging IntelliTrace™ Make “No-Repro” History with Actionable Bugs Test Impact Analysis and Code Coverage Run the Right Tests and only the Right Tests And know you have really tested you code Automated Coded UI Tests Automated regression tests let you find bugs earlier
Step 6 : Test Lab Managment How do we integrate VM’s for manual or automated testing?
Testing Done Right Test Manager Manual Testing for the 21st century Plan, Manage and Execute tests from one place File rich & actionable bug during manual testing Perform Exploratory testing Virtual Test Environments Reduce the overhead of maintaining physical test environments Share Virtual environments between team members
Microsoft Confidential 55 Environment (collection of VMs) created
Microsoft Confidential 56 Manage snapshots for environment Access environment using Lab Environment Viewer
Other Benefits VM Labs can be used for: Demos for customers Maintaining snapshots of old versions Production hot-fixes Tech writer access for screenshots Testing CI builds using GUI automation Load testing
Step 7: Working Smart How can we improve and be more efficient?
Highly integrated Productive Extensible Low administration Agile Customizable Lightweight Patterns and practices Flexible Layering Best Practices Process and tools support individuals, teams and organizations frictionlessly and transparently Overcome Complexity Quality Mindset Culture of Innovation Collaboration Transparency Integration Clarity Alignment Efficiency
Microsoft Test Manager TFS UML Modeling TFS Manual Testing Layer Diagram Load Testing Web Testing Test Case Management IntelliTrace™ Architecture Explorer Logical Class Designer Cloud Development Office Development Windows Development New WPF Editor Customizable IDE Multi-core Development Silverlight Tools Web Development SharePoint Development Generate from Usage Static Code Analysis Database Deployment Code Metrics Database Unit Testing Test Data Generation Test Impact Analysis UI Test Automation Code Coverage Performance Profiling Database Change Mgmt TFS Test Record & Playback
TFS Test and Lab Manager Test Case Management Test Record & Playback Manual Testing Lab MgmtConfig. Checkpoint Environment Environment from Template Virtual Env. Setup/Tear Down Reporting Team Portal Test Case Management Branch Visualization Version Control Build Automation Gated Check-in Agile Planning Tools Team Explorer Team Agents TFS