Uploaded byAnna Russo
Improving Software Quality- 2-day Tester Training
The document discusses how Visual Studio 2010 improves software quality through features that enable better work management, reporting, testing tools, developer quality tools, automated builds, lab management, and collaboration between development and testing teams. Key improvements include hierarchical work tracking, richer bug reporting, automated coded UI testing, IntelliTrace for debugging, and lab management for maintaining virtual test environments. The goal is to align development and testing, break down silos, and improve transparency and integration across the lifecycle.
More Related Content
Similar to Improving Software Quality- 2-day Tester Training
Improving Software Quality- 2-day Tester Training
- 1. Using Visual Studio2010 to Improve QualityAnna RussoSoftware Development Process ConsultantNotion Solutionsarusso@notionsolutions.comCompany: http://www.NotionSolutions.comBlog: http://a7russo.spaces.live.com/
- 2. 7 Steps toImproving QualityStep 1- Work ManagementStep 2- Reporting of “Done” metricsStep 3- Manual Testing ToolsStep 4- Developer Quality ToolsStep 5- Automated BuildsStep 6- Lab ManagementStep 7- Working Smart
- 3. Step 1: Work Management
- 4.
- 5. Collaborate & CommunicateTeamFoundation ServerA unified project serverDeveloperDesignerDatabase ProfessionalTesterArchitectProjectManagerBusinessAnalystTeamFoundationServer
- 6. TFS ToolsUse manytools to interact with TFSTeam ExplorerTeamFoundationServerMicrosoft Test ManagerVisual Studio
- 7. Goals for 2010Release - QualityAlign QA with the LifecycleCreate tighter Dev/Test interactionEnable highly leveraged QA teams
- 8. Work ManagementNEW!Hierarchical WorkManagement
- 9.
- 10.
- 11.
- 12. Improved Integration withProject and Excel
- 13.
- 14. Step 2: Reporting
- 15.
- 16.
- 17.
- 18.
- 19. Are we makingprogress on running test plans?How are our builds doing over time?Are we fixing bugs?What’s the quality of our bug fixes?What’s the value?
- 20. Step 3: Manual Testing Tools
- 21. Testing Done RightTestManagerManual Testing for the 21st centuryPlan, Manage and Execute tests from one placeFile rich & actionable bug during manual testingPerform Exploratory testingVirtual Test EnvironmentsReduce the overhead of maintaining physical test environmentsShare Virtual environments between team members
- 22. Testing ToolsMTM ManualTesting ToolsTest Plan ManagementTest Case ManagementShared Steps ManagementTest Configuration ManagementTest Fast ForwardingTest CollectorsTest Lab ManagementVisual StudioAutomated Testing ToolsAutomated Coded UI TestingWeb TestingLoad TestingGeneric TestsDeveloper Unit TestsDatabase Unit Tests
- 23.
- 24. The ProblemDevelopment andtest have built a wall:Development work happens on one side of the wall.Testing happens on the other. Builds and test results are passed back and forth, but not in a collaborative fashion.
- 25. Development PerspectiveWorks onmy machine!What build is being tested?How do I repro this?Insufficient bug detailsUnknown machine configurationUnknown runtime detailsNo screenshotI already fixed this bug!This bug has been here for 2 monthsWhy does testing take so long?The testers are clueless twits that don’t even know what good software is!
- 26. Test PerspectiveThis isstill broken? They said they fixed it!Dev teams go over schedule, testing time is reducedWe can’t improve test coverageYou want me to come to your desk to show you how to repro this bug?What is supposedly fixed in this new build?Is the new build deployed to our QA environment?The developers are clueless twits that don’t even know what good software is!
- 27. Rich BugsNEW!Repro StepsScreenshotVideoCaptureSystem InfoDebug LogTest EnvironmentEvent LogAction Recording
- 28.
- 29. Test Plans andTest Settings
- 30. Test Plans withConfigurations
- 31.
- 32. Test Cases/Shared Stepsare Work Items!
- 33.
- 34.
- 35.
- 36. Step 4: Developer Quality Tools
- 37. Automated Coded UITesting
- 38. Create code, Don’tdebug itDependency DiagramsUnderstand code dependencies visually and understand where to make changesGated Check-InStop build breaks before they happenCode Analysis, Code Profiling, Code MetricsHelp you find and fix potential problems before you make them
- 39. Spend Less timeDebuggingIntelliTrace™Make “No-Repro” History with Actionable BugsTest Impact Analysis and Code CoverageRun the Right Tests and only the Right TestsAnd know you have really tested you codeAutomated Coded UI TestsAutomated regression tests let you find bugs earlier
- 40.
- 41.
- 42. Coded UI TestingEnablesverification of UI functionality.Record test actions and VS will generate the resulting code.Assertions can be added to verify functionality.Like all VS tests, results can be stored in TFS.
- 43.
- 44. Best Practices (FromMSDN)Create each test case as a sequence of Recorded Methods. Use meaningful method names. Try to limit the length of each recorded method to less than 10 actions.Do not modify the RecordedMethods.cs file.Do not edit the UIMap.Designer.cs file.
- 45.
- 46. UITestControl.WaitForControl…WaitForControlReady() – Thiswaits for the control to be ready to accept mouse/keyboard input. WaitForControlEnabled() – This waits for the control to be enabled. WaitForControlExist() – This waits for the control to exist on the UI. WaitForControlNotExist() – This waits till the control cease to exist on the UI. For example, you can use this for progress dialog to go away. WaitForControlPropertyEqual(string propertyName, object propertyValue) – This waits for the specified property of the control to have the given value. WaitForControlPropertyNotEqual(string propertyName, object propertyValue) – For example, you can use this for edit box to be not read-only i.e. editable. WaitForControlCondition(Predicate<UITestControl> conditionEvaluator) –wait till the status text is “Succeeded” or “Failed”. The code for this example would look like –
- 47. Coded UI PlatformSupport
- 48. Step 5: Automated Build
- 49. Powerful Build EngineBuildExplorerCentralized build informationBuild Report to display code and work itemsExecutes TestsUnit testAutomated Coded UI testWeb testsLoad testsThird party tool testsDeployPhysical machinesVirtual machinesVirtual Lab machines- Restores to Clean Baseline
- 50.
- 51. Gated Check-inBuild withTFS 2005/2008Check-In DeniedBuild FailsNew Code is Checked InBuild is ExecutedDeveloper Makes ChangesBuild FailsBuild is ExecutedDeveloper Makes ChangesThe code that caused the build to fail is still checked-in.Build SucceedsNew Code is Checked In
- 52. Build ReportWork Items+ Checked in code + Build = Build Report
- 53.
- 54. Step 6: Test Lab Management
- 55. Lab ManagementNEW!* Supportfor ESX will not be available in TFS 2010 but is on the TFS roadmap
- 56. Lab Management (1 Server, 1Host, 5 VMs)Physical HostSC-VMM AgentClientVirtual GuestBuild/Test Agents Virtual GuestBuild/Test Agents ControllersVirtual GuestBuild/Test Agents Virtual GuestBuild/Test Agents Virtual GuestBuild/Test Agents ServerTeam Foundation ServerSC-VMM Server (supports HyperV. VmWare)TRTCMLab ManagementReportingBuildWITProxyVC
- 58.
- 59. Test Controllers andAgents
- 60. Microsoft Confidential56Environment (collectionof VMs) created
- 61. Microsoft Confidential57Manage snapshotsfor environmentAccess environment using Lab Environment Viewer
- 62. Other BenefitsVM Labscan be used for:Demos for customersMaintaining snapshots of old versionsProduction hot-fixes Tech writer access for screenshotsTesting CI builds using GUI automationLoad testing
- 63. Step 7: Working Smart
Editor's Notes
- #8 The 2010 release for Microsoft is a very big release with a lot of rich capabilities. Quality is a key pillars for the release planning efforts and we aspired to achieve the above goals with the test offerings in the 2010 release by taking a fresh look at the problems.We want to align QA with the lifecycle to ensure that testers can add more value to the application lifecycle and are not looked at “necessary evil”. Having test assets co-located with the dev assets & project assets ensures visibility across the entire team.Create tighter integration across development & test teams to ensure that we can eliminate waste across the hand shake (or lack there of) that happens across devs & tests when it comes to bugs, builds, recreation of bugs etc. Enable highly leveraged QA teams ensures that teams are most efficient and are driving up productivity by focusing on key tasks and eliminating a lot of the waste that spans from boring repetitive tasks.
- #18 Having the right infrastructure and approach to testing is helps ensure high quality deliverables at the end of the development cycle. Visual Studio 2010 Ultimate arms you with tools to ensure that testing is done right the first time and streamlines the testing process to ensure that you can deliver on time and on budget.Test and Lab ManagerVisual Studio 2010 Ultimate delivers a brand new product for manual testing and test case management. The new Test and Lab Manager interface is a thoroughly modern interface that allows testers and anyone else involved in testing to manage test plans, test cases and individual tests. In addition, Test and lab manager acts as the one stop shop for manual testing. When executing manual tests, the Test and Lab Manager interface morphs into the Microsoft Test Runner, a modern integrated manual test interface that allows manual tests to be executed and the results to be filed in a simple and productive way. Additionally, Microsoft Test Runner can take a snapshot of the system data, record a video of the test, take a picture of the bug and record the IntelliTrace log to ensure you can always repro a bug, and there are no more "not reproducible" discussions. Virtual Test EnvironmentsOne of the most costly aspects of testing is the maintenance of the test infrastructure. A traditional test infrastructure consisted of rooms full of physical test machines that required dedicated professionals to provision and maintain. With Visual Studio 2010 Ultimate this test infrastructure has gone virtual. By using the new virtualized test lab management capabilities it is possible to reduce the overhead inherent with maintaining a physical test infrastructure and instead take advantage of the power of virtualized environments. It is also possible to share these environments not only within your testing team but also with other members of the team who need access to specific environments for development tests or dry runs prior to full scale testing begins.Web and Load TestingVisual Studio 2010 Ultimate also gives you access to advanced web and load testing features that help you ensure that whatever application you are building it is ready for the most demanding uses. The Web and Load testing capabilities of Visual Studio 2010 Ultimate help you ensure that whatever application you are building it is ready for whatever scenarios your organization can envisage taking you one step closer to knowing your application is ready for prime time.
- #26 Right side shows specific test settings for different scenarios.
- #34 http://blogs.msdn.com/mathew_aniyan
- #35 The more time you spend debugging and fixing code the less time you have to spend creating innovative solutions and bringing value to the outcome. Visual Studio 2010 Ultimate includes features that help you reduce the number of bugs that get into an application thereby freeing you up to spend more of your time being creative around the solutions you deliver. Dependency DiagramsMaking changes to code can be fraught with danger and often leads to many types of new bugs being introduced into a solution that can be difficult to debug and fix. Visual Studio 2010 Ultimate allows you to create dependency diagrams that allow you to visually understand the dependencies between various areas of a solution helping you find where to make changes and what potential ripple effect those changes may have.Gated Check-InOne of the most costly problems that development teams face is when a build gets broken. This can potentially stop all development work for an entire team while the cause of the broken build is investigated and fixes are put in place. Visual Studio 2010 Ultimate includes an innovative new feature called Gated Check-In that looks to stop build breaks before they happen. By utilizing the shelving infrastructure first introduced into TFS in 2005, Gated Check-Ins allow developer to check code into source control but hold that check-in from the build tree until it has passed all the required validation and test required to ensure that there are not potential issues that may break the build. This is all done in the background and allows you to continue working while the checks and validations are performed.Code Analysis and Code MetricsOne easy way to reduce the number of bugs that occur is to simply ensure that the code you right is higher quality. Visual Studio 2010 Ultimate helps you find and fix problems before you make them by delivering advanced tools that analyze code for potential issues and ensure that code that is written is of high quality and maintainable.
- #36 Debugging code takes up a disproportionate amount of the development lifecycle. With the new features in Visual Studio 2010 Ultimate you can ensure that you reduce the amount of time spent debugging code to a minimum. IntelliTraceThe new IntelliTrace feature makes the “no-repro” discussion a thing of the past. It records the application’s execution history guaranteeing that the developer can always reproduce a reported bug. Step back in time to see the exact application state at the point of time when the bug occurs and use the inline call hierarchy to quickly trace a program’s execution flow without invoking the debugger. Automatically open the code at the place exactly where the code broke, reducing debugging time. Test Impact Analysis and Code CoverageTest impact analysis can tell you exactly what tests are impacted when you change a piece of code allowing you to choose to only run impacted tests. When you change a piece of code, Visual Studio 2010 Ultimate can make an automatic recommendation of tests based on that change leading to reduced testing time. Coupling this functionality with the Code Coverage tools ensure that not only are you running the appropriate tests but that you are testing all of your code so you can be confident that the code you deliver is free of defects.Coded UI TestsOnce you have found bugs in your code and have taken the steps to fix them, automated regression tests ensure that as you progress further through the development process that you catch any bugs that may have been re-introduced earlier in the cycle. This in turn ensures that you find new bugs earlier and can catch them when they are less costly to fix.
- #39 http://blogs.msdn.com/mathew_aniyan
- #41 Assertions - This inserts assertions into your main coded UI test method. Even if some part of the recording must be changed, you can reuse your assertions.
- #42 Hints provided by MS product team member: Mathew AniyanContinue on Error (login page recorded and “remember password” prompted during recording)Match Exact - Sometimes application may change so that the hierarchy gets modified. Record and Playback engine attempts to locate the control, in such cases, by skipping the intermediate elements in the hierarchy.
- #43 Hints provided by MS product team member: Mathew AniyanContinue on Error (login page recorded and “remember password” prompted during recording)Match Exact - Sometimes application may change so that the hierarchy gets modified. Record and Playback engine attempts to locate the control, in such cases, by skipping the intermediate elements in the hierarchy.
- #44 http://blogs.msdn.com/gautamg/archive/2010/01/07/platform-support-for-coded-ui-test-and-fast-forward-feature-of-test-runner.aspx
- #46 Having the right infrastructure and approach to testing is helps ensure high quality deliverables at the end of the development cycle. Visual Studio 2010 Ultimate arms you with tools to ensure that testing is done right the first time and streamlines the testing process to ensure that you can deliver on time and on budget.Test and Lab ManagerVisual Studio 2010 Ultimate delivers a brand new product for manual testing and test case management. The new Test and Lab Manager interface is a thoroughly modern interface that allows testers and anyone else involved in testing to manage test plans, test cases and individual tests. In addition, Test and lab manager acts as the one stop shop for manual testing. When executing manual tests, the Test and Lab Manager interface morphs into the Microsoft Test Runner, a modern integrated manual test interface that allows manual tests to be executed and the results to be filed in a simple and productive way. Additionally, Microsoft Test Runner can take a snapshot of the system data, record a video of the test, take a picture of the bug and record the IntelliTrace log to ensure you can always repro a bug, and there are no more "not reproducible" discussions. Virtual Test EnvironmentsOne of the most costly aspects of testing is the maintenance of the test infrastructure. A traditional test infrastructure consisted of rooms full of physical test machines that required dedicated professionals to provision and maintain. With Visual Studio 2010 Ultimate this test infrastructure has gone virtual. By using the new virtualized test lab management capabilities it is possible to reduce the overhead inherent with maintaining a physical test infrastructure and instead take advantage of the power of virtualized environments. It is also possible to share these environments not only within your testing team but also with other members of the team who need access to specific environments for development tests or dry runs prior to full scale testing begins.Web and Load TestingVisual Studio 2010 Ultimate also gives you access to advanced web and load testing features that help you ensure that whatever application you are building it is ready for the most demanding uses. The Web and Load testing capabilities of Visual Studio 2010 Ultimate help you ensure that whatever application you are building it is ready for whatever scenarios your organization can envisage taking you one step closer to knowing your application is ready for prime time.
- #48 OverviewWhen you’re building software as a team, there’s a natural cycle that you follow – testers find bugs, developers fix bugs, testers verify fixes. In order for this cycle to be productive, testers need builds that include fixes to the bugs that the developers have fixed. If the build is broken, there is a risk of holding up the entire team until it is fixed.Gated check-in provides a way to prevent build breaks by validating a developer’s changes prior to committing them to the version control repository. Those changes are validated via an automated build. If the build fails, the user’s pending changes are rejected and passed back to them as a shelveset, otherwise the changes are committed to the repository.Shelve-and-build, a related feature, lets the developer validate their changes with a private automated build. Unlike gated check-in, shelve-and-build is something the developer can choose to do. It can help identify breaks in code that the developer may not generally build on their local machine.Value Proposition If a single developer on your team typically breaks the build just once a year and you have 52 developers working on your project, the build could be broken every week. If you have 260 developers, the build could be broken every day. When the build breaks, the entire team may be stalled resulting in substantial productivity losses. Those losses could be prevented by the gated check-in feature.
- #61 The idea is you can take all of your learned best practices and begin to layer them into areas that need the most help. The notion of “best practices” apply at many levels when talking about the Application Lifecycle – from the individual to the organization.
- #62 The idea is you can take all of your learned best practices and begin to layer them into areas that need the most help. The notion of “best practices” apply at many levels when talking about the Application Lifecycle – from the individual to the organization.