Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Using Visual Studio/TFS 2010 to Improve Software Quality<br />Anna Russo<br />Application Lifecycle Management Consultant<...
The enclosed material is proprietary to Imaginet and is therefore  copyrighted.  This document may not be disclosed in any...
7 Steps to Improving Quality<br />Step 1- Work Management<br />Step 2- Defining/Reporting of “Done” <br />Step 3- Automate...
Step 1: Work Management<br />How do we manage work?<br />
Collaborate & Communicate<br />Team Foundation Server<br />A unified project server<br />Developer<br />Designer<br />Data...
TFS Tools<br />Use many tools to interact with TFS<br />Eclipse IDE<br />Team Explorer Everywhere<br />Team<br />Foundatio...
Goals for 2010 Release - Quality<br />Align QA with the Lifecycle<br />Create tighter Dev/Test interaction<br />Enable hig...
Work Management<br />NEW!<br /><ul><li>Hierarchical Work Management
Excel Reports
Ad-hoc Reports
Improved Dashboards
Improved Integration with Project and Excel
Agile Workbooks</li></li></ul><li>Demo<br />Work Management<br />
Step 2: Defining/Reporting of “Done”<br />What is the end goal?<br />
Improved Reporting<br />
Improved Reporting<br />
Dashboard: SharePoint Server<br />Dashboard title<br />What is this dashboard about?<br />Additional dashboards<br />Repor...
Are we making progress on running test plans?<br />How are our builds doing over time?<br />Are we fixing bugs?<br />What’...
Reporting<br />
Improved Test Reporting<br />
Improved Bug Reporting<br />
Step 3: Automated Build<br />What is our traceability from<br />Dev to QA?<br />
Powerful Build Engine<br />Build Explorer<br />Centralized build information<br />Build Report to display code and work it...
Gated Check-in<br />Build with TFS 2005/2008<br />Check-In Denied<br />Build Fails<br />New Code is Checked In<br />Build ...
Build Report<br />Check code in TFS+ <br />Associate Work Item+ <br />TFS Build script= <br />Build Report<br />
Integrate Work Frequently<br />
Step 4: Manual Testing Tools<br />How do we keep track of Testing?<br />
Testing Done Right<br />Test Manager<br />Manual Testing for the 21st century<br />Plan, Manage and Execute tests from one...
Testing Tools<br />MTM <br />Manual Testing Tools<br />Test Plan Management<br />Test Case Management<br />Shared Steps Ma...
                               Test Process<br />Release Cycle<br />Requirements<br />Build/<br /> Environment<br />Test R...
Test Plans, Suites and Cases<br />
Test Plans and Test Settings<br />
Test Plans with Configurations<br />
Test Suites<br />
Test Cases/Shared Steps are Work Items!<br />
Test Execution<br />
The Problem<br />Development and test have built a wall:<br />Development work happens on one side of the wall.<br />Testi...
Development Perspective<br />Works on my machine!<br />What build is being tested?<br />How do I repro this?<br />Insuffic...
Test Perspective<br />This is still broken? They said they fixed it!<br />Dev teams go over schedule, testing time is redu...
Rich Bugs<br />NEW!<br />Repro Steps<br />Screenshot<br />Video Capture<br />System Info<br />Debug Log<br />Test Environm...
Demo<br />Test Case Management/Filing Bugs<br />
Anna Bots<br />
Automated Coded UI Test<br />NEW!<br />Recorded<br />Actions<br />from Manual Tester<br />Use Visual Studio Recorder<br />...
Coded UI Testing<br />Enables verification of UI functionality.<br />Record test actions and VS will generate the resultin...
Best Practices (From MSDN)<br />Create each test case as a sequence of Recorded Methods. <br />Use meaningful method names...
Playback.PlaybackSettings…<br />Continue on Error<br />DelayBetweenActions<br />MatchExactHierarchy<br />SearchTimeout<br ...
UITestControl.WaitForControl…<br /><ul><li>WaitForControlReady() – This waits for the control to be ready to accept mouse/...
WaitForControlEnabled() – This waits for the control to be enabled. 
WaitForControlExist() – This waits for the control to exist on the UI. 
WaitForControlNotExist() – This waits till the control cease to exist on the UI.  For example, you can use this for progre...
WaitForControlPropertyEqual(string propertyName, object propertyValue) – This waits for the specified property of the cont...
WaitForControlCondition(Predicate<UITestControl> conditionEvaluator) –wait till the status text is “Succeeded” or “Failed”...
Step 5: Developer Quality Tools<br />How can developers contribute to the quality?<br />
Create code, Don’t debug it<br />Dependency Diagrams<br />Understand code dependencies visually and understand where to ma...
Spend Less time Debugging<br />IntelliTrace™<br />Make “No-Repro” History with Actionable Bugs<br />Test Impact Analysis a...
IntelliTrace<br />
Automated Coded UI Test<br />NEW!<br />Recorded<br />Actions<br />from Manual Tester<br />Use Visual Studio Recorder<br />...
Automation Platform Support<br />
Step 6 : Test Lab Managment<br />How do we integrate VM’s for manual or automated testing?<br />
Testing Done Right<br />Test Manager<br />Manual Testing for the 21st century<br />Plan, Manage and Execute tests from one...
Upcoming SlideShare
Loading in …5
×

STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Manager, Visual Studio 2010, and TFS 2010

2,041 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Manager, Visual Studio 2010, and TFS 2010

  1. 1. Using Visual Studio/TFS 2010 to Improve Software Quality<br />Anna Russo<br />Application Lifecycle Management Consultant<br />Notion Solutions<br />arusso@notionsolutions.com<br />xx<br />Blog- http://ImprovingSoftwareQuality.blogspot.com<br />www.notionsolutions.com<br />
  2. 2. The enclosed material is proprietary to Imaginet and is therefore copyrighted. This document may not be disclosed in any manner to anyone other than the addressee and the employees or representatives of Notion Solutions who are recipients. This document may not be used in any manner other than for the purpose it was distributed. Any unauthorized use, reproduction or transmission in any form is strictly prohibited.<br />www.notionsolutions.com<br />
  3. 3. 7 Steps to Improving Quality<br />Step 1- Work Management<br />Step 2- Defining/Reporting of “Done” <br />Step 3- Automated Builds<br />Step 4- Manual & Automated Testing Tools<br />Step 5- Developer Quality Tools<br />Step 6- Lab Management<br />Step 7- Working Smart<br />
  4. 4. Step 1: Work Management<br />How do we manage work?<br />
  5. 5.
  6. 6. Collaborate & Communicate<br />Team Foundation Server<br />A unified project server<br />Developer<br />Designer<br />Database Professional<br />Tester<br />Architect<br />ProjectManager<br />BusinessAnalyst<br />Team<br />Foundation<br />Server<br />
  7. 7. TFS Tools<br />Use many tools to interact with TFS<br />Eclipse IDE<br />Team Explorer Everywhere<br />Team<br />Foundation<br />Server<br />Visual Studio<br />Team Explorer<br />Microsoft Test Manager<br />
  8. 8. Goals for 2010 Release - Quality<br />Align QA with the Lifecycle<br />Create tighter Dev/Test interaction<br />Enable highly leveraged QA teams<br />
  9. 9. Work Management<br />NEW!<br /><ul><li>Hierarchical Work Management
  10. 10. Excel Reports
  11. 11. Ad-hoc Reports
  12. 12. Improved Dashboards
  13. 13. Improved Integration with Project and Excel
  14. 14. Agile Workbooks</li></li></ul><li>Demo<br />Work Management<br />
  15. 15. Step 2: Defining/Reporting of “Done”<br />What is the end goal?<br />
  16. 16. Improved Reporting<br />
  17. 17. Improved Reporting<br />
  18. 18. Dashboard: SharePoint Server<br />Dashboard title<br />What is this dashboard about?<br />Additional dashboards<br />Reports shown side-by-side in context of other relevant metrics<br />Yes, we have a burndown!<br />Are we making progress on tasks?<br />Are we making progress on our User Stories?<br />How are we doing on open issues?<br />Reporting in 2010 | Custom Reports | Making Transparency Work<br />
  19. 19. Are we making progress on running test plans?<br />How are our builds doing over time?<br />Are we fixing bugs?<br />What’s the quality of our bug fixes?<br />What’s the value?<br />
  20. 20. Reporting<br />
  21. 21. Improved Test Reporting<br />
  22. 22. Improved Bug Reporting<br />
  23. 23. Step 3: Automated Build<br />What is our traceability from<br />Dev to QA?<br />
  24. 24. Powerful Build Engine<br />Build Explorer<br />Centralized build information<br />Build Report to display code and work items<br />Executes Tests<br />Unit test<br />Automated Coded UI test<br />Web tests<br />Load tests<br />Third party tool tests<br />Deploy<br />Physical or Virtual machines<br />Virtual Lab machines- Restores to Clean Baseline<br />
  25. 25. Gated Check-in<br />Build with TFS 2005/2008<br />Check-In Denied<br />Build Fails<br />New Code is Checked In<br />Build is Executed<br />Developer Makes Changes<br />Build Fails<br />Build is Executed<br />Developer Makes Changes<br />The code that caused the build to fail is still checked-in.<br />Build Succeeds<br />New Code is Checked In<br />
  26. 26. Build Report<br />Check code in TFS+ <br />Associate Work Item+ <br />TFS Build script= <br />Build Report<br />
  27. 27. Integrate Work Frequently<br />
  28. 28. Step 4: Manual Testing Tools<br />How do we keep track of Testing?<br />
  29. 29. Testing Done Right<br />Test Manager<br />Manual Testing for the 21st century<br />Plan, Manage and Execute tests from one place<br />File rich & actionable bug during manual testing<br />Perform Exploratory testing<br />
  30. 30. Testing Tools<br />MTM <br />Manual Testing Tools<br />Test Plan Management<br />Test Case Management<br />Shared Steps Management<br />Test Configuration Management<br />Test Fast Forwarding<br />Test Collectors<br />Test Lab Management<br />Visual Studio<br />Automated Testing Tools<br />Automated Coded UI Testing<br />Web Testing<br />Load Testing<br />Generic Tests<br />Developer Unit Tests<br />Database Unit Tests<br />
  31. 31. Test Process<br />Release Cycle<br />Requirements<br />Build/<br /> Environment<br />Test Run<br />Improve Test<br />Coverage<br />
  32. 32. Test Plans, Suites and Cases<br />
  33. 33. Test Plans and Test Settings<br />
  34. 34. Test Plans with Configurations<br />
  35. 35. Test Suites<br />
  36. 36. Test Cases/Shared Steps are Work Items!<br />
  37. 37. Test Execution<br />
  38. 38. The Problem<br />Development and test have built a wall:<br />Development work happens on one side of the wall.<br />Testing happens on the other. <br />Builds and test results are passed back and forth, but not in a collaborative fashion.<br />
  39. 39. Development Perspective<br />Works on my machine!<br />What build is being tested?<br />How do I repro this?<br />Insufficient bug details<br />Unknown machine configuration<br />Unknown runtime details<br />No screenshot<br />I already fixed this bug!<br />This bug has been here for 2 months<br />Why does testing take so long?<br />The testers are clueless twits that don’t even know what good software is!<br />
  40. 40. Test Perspective<br />This is still broken? They said they fixed it!<br />Dev teams go over schedule, testing time is reduced<br />We can’t improve test coverage<br />You want me to come to your desk to show you how to repro this bug?<br />What is supposedly fixed in this new build?<br />Is the new build deployed to our QA environment?<br />The developers are clueless twits that don’t even know what good software is!<br />
  41. 41. Rich Bugs<br />NEW!<br />Repro Steps<br />Screenshot<br />Video Capture<br />System Info<br />Debug Log<br />Test Environment<br />Event Log<br />Action Recording<br />
  42. 42. Demo<br />Test Case Management/Filing Bugs<br />
  43. 43. Anna Bots<br />
  44. 44. Automated Coded UI Test<br />NEW!<br />Recorded<br />Actions<br />from Manual Tester<br />Use Visual Studio Recorder<br />Automated<br />Coded UI Test<br />+<br />=<br />Validation<br />Code<br />
  45. 45. Coded UI Testing<br />Enables verification of UI functionality.<br />Record test actions and VS will generate the resulting code.<br />Assertions can be added to verify functionality.<br />Like all VS tests, results can be stored in TFS.<br />
  46. 46. Best Practices (From MSDN)<br />Create each test case as a sequence of Recorded Methods. <br />Use meaningful method names. <br />Try to limit the length of each recorded method to less than 10 actions.<br />Do not modify the RecordedMethods.cs file.<br />Do not edit the UIMap.Designer.cs file. <br />
  47. 47. Playback.PlaybackSettings…<br />Continue on Error<br />DelayBetweenActions<br />MatchExactHierarchy<br />SearchTimeout<br />SmartMatchOptions<br />TopLevelWindow<br />Control<br />None<br />ThinkTimeMultiplier<br />WaitForReadyTimeout<br />
  48. 48. UITestControl.WaitForControl…<br /><ul><li>WaitForControlReady() – This waits for the control to be ready to accept mouse/keyboard input. 
  49. 49. WaitForControlEnabled() – This waits for the control to be enabled. 
  50. 50. WaitForControlExist() – This waits for the control to exist on the UI. 
  51. 51. WaitForControlNotExist() – This waits till the control cease to exist on the UI.  For example, you can use this for progress dialog to go away.
  52. 52. WaitForControlPropertyEqual(string propertyName, object propertyValue) – This waits for the specified property of the control to have the given value.  
  53. 53. WaitForControlCondition(Predicate<UITestControl> conditionEvaluator) –wait till the status text is “Succeeded” or “Failed”. </li></li></ul><li>Demo<br />Automated Coded UI Testing <br />
  54. 54. Step 5: Developer Quality Tools<br />How can developers contribute to the quality?<br />
  55. 55. Create code, Don’t debug it<br />Dependency Diagrams<br />Understand code dependencies visually and understand where to make changes<br />Gated Check-In<br />Stop build breaks before they happen<br />Code Analysis, Code Profiling, Code Metrics<br />Help you find and fix potential problems before you make them<br />
  56. 56. Spend Less time Debugging<br />IntelliTrace™<br />Make “No-Repro” History with Actionable Bugs<br />Test Impact Analysis and Code Coverage<br />Run the Right Tests and only the Right Tests<br />And know you have really tested you code<br />Automated Coded UI Tests<br />Automated regression tests let you find bugs earlier<br />
  57. 57. IntelliTrace<br />
  58. 58. Automated Coded UI Test<br />NEW!<br />Recorded<br />Actions<br />from Manual Tester<br />Use Visual Studio Recorder<br />Automated<br />Coded UI Test<br />+<br />=<br />Validation<br />Code<br />
  59. 59. Automation Platform Support<br />
  60. 60. Step 6 : Test Lab Managment<br />How do we integrate VM’s for manual or automated testing?<br />
  61. 61. Testing Done Right<br />Test Manager<br />Manual Testing for the 21st century<br />Plan, Manage and Execute tests from one place<br />File rich & actionable bug during manual testing<br />Perform Exploratory testing<br />Virtual Test Environments<br />Reduce the overhead of maintaining physical test environments<br />Share Virtual environments between team members<br />
  62. 62.
  63. 63. Microsoft Confidential<br />55<br />Environment (collection of VMs) created<br />
  64. 64. Microsoft Confidential<br />56<br />Manage snapshots for environment<br />Access environment using Lab Environment Viewer<br />
  65. 65. Other Benefits<br />VM Labs can be used for:<br />Demos for customers<br />Maintaining snapshots of old versions<br />Production hot-fixes <br />Tech writer access for screenshots<br />Testing CI builds using GUI automation<br />Load testing<br />
  66. 66. Step 7: Working Smart<br />How can we improve and be more efficient?<br />
  67. 67. Highly integrated Productive Extensible Low administration<br />Agile Customizable Lightweight Patterns and practices Flexible<br />Layering Best Practices<br />Process and tools support individuals, teams and organizations frictionlessly and transparently<br />Overcome Complexity<br />Quality Mindset<br />Culture of Innovation<br />Collaboration<br />Transparency<br />Integration<br />Clarity<br />Alignment<br />Efficiency<br />
  68. 68. Going from Good to Great<br />
  69. 69. Microsoft Test Manager<br />TFS<br />UML Modeling<br />TFS<br />Manual Testing<br />Layer Diagram<br />Load Testing<br />Web Testing<br />Test Case Management<br />IntelliTrace™<br />Architecture Explorer<br />Logical Class Designer<br />Cloud Development<br />Office Development<br />Windows Development<br />New WPF Editor<br />Customizable IDE<br />Multi-core Development<br />Silverlight Tools<br />Web Development<br />SharePoint Development<br />Generate from Usage<br />Static Code Analysis<br />Database Deployment<br />Code Metrics<br />Database Unit Testing<br />Test Data Generation<br />Test Impact Analysis<br />UI Test Automation<br />Code Coverage<br />Performance Profiling<br />Database Change Mgmt<br />TFS<br />Test Record & Playback<br />
  70. 70. TFS<br />Test and Lab Manager<br />Test Case Management<br />Test Record & Playback<br />Manual Testing<br />Lab MgmtConfig.<br />Checkpoint Environment<br />Environment from Template<br />Virtual Env. Setup/Tear Down<br />Reporting<br />Team Portal<br />Test Case Management<br />Branch Visualization<br />Version Control<br />Build Automation<br />Gated Check-in<br />Agile Planning Tools<br />Team Explorer<br />Team Agents<br />TFS<br />
  71. 71. www.notionsolutions.com<br />

×