Lab Management with TFS 2010

6,871 views
6,714 views

Published on

Discusses the Visual Studio Lab Management features available in Visual Studio 2010 and Team Foundation Server 2010.

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,871
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
0
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide
  • We found that customers pain points bucketed themselves into three things areasTesters are not aligned with the lifecycle. Finding out a code change is impacting their day at work (test more, go home late, miss the time with kids). Testers and developers work in silo’s often causing significant dis-satisfaction and un-happiness and creating an environment of lack of respect across the two teams. Testers are smart, developers often don’t give them the respect they deserve because they see testers filing poor defects, not really impacting quality. Testing teams are not really leveraged well as they waste so much time doing the basic things. Just getting their bugs looked at take so much time that they cannot spend quality time doing what they love and provide better service to the broader teams.
  • The 2010 release for Microsoft is a very big release with a lot of rich capabilities. Quality is a key pillars for the release planning efforts and we aspired to achieve the above goals with the test offerings in the 2010 release by taking a fresh look at the problems.We want to align QA with the lifecycle to ensure that testers can add more value to the application lifecycle and are not looked at “necessary evil”. Having test assets co-located with the dev assets & project assets ensures visibility across the entire team.Create tighter integration across development & test teams to ensure that we can eliminate waste across the hand shake (or lack there of) that happens across devs & tests when it comes to bugs, builds, recreation of bugs etc. Enable highly leveraged QA teams ensures that teams are most efficient and are driving up productivity by focusing on key tasks and eliminating a lot of the waste that spans from boring repetitive tasks.
  • This is the fully animated slide.
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • http://www.wintellect.com/CS/blogs/jrobbins/archive/2009/06/16/how-does-vs2010-historical-debugging-work.aspxhttp://blogs.msdn.com/habibh/default.aspx
  • This would normally go in the Testing section but want to demonstrate it here during the build process.
  • http://blogs.msdn.com/mathew_aniyan
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • http://blogs.msdn.com/gautamg/archive/2010/01/07/platform-support-for-coded-ui-test-and-fast-forward-feature-of-test-runner.aspx
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
  • Lab Management with TFS 2010

    1. 1. Virtual Lab Management<br />Team Foundation Server 2010<br />Ed Blankenship<br />Microsoft MVP (TFS, Visual Studio ALM, and VSTS)<br />Notion Solutions, Inc.<br />www.notionsolutions.com<br />
    2. 2. About Ed Blankenship<br /><ul><li>Microsoft MVP of the Year
    3. 3. ALM/TFS Consultant at Notion Solutions
    4. 4. Live in Charleston, SC
    5. 5. Blog: http://edsquared.com
    6. 6. Twitter: @EdBlankenship
    7. 7. E-Mail: edb@notionsolutions.com
    8. 8. Author - Wrox TFS 2010 Book: http://bit.ly/TFS2010Book</li></li></ul><li>Frustrations in development teams<br />“my testers are spending too long testing the same thing”<br />“tooling is expensive (time, licenses, people)”<br />“the developers say the defects are useless”<br />“when is my software ready to ship?”<br />“we have a changed requirement, what do I need to test?”<br />“developers and testers work in silos and don’t communicate/speak the same language”<br />3<br />
    9. 9. Customer Challenges <br />QA is not aligned with lifecycle<br />Developers and testers work in silos<br />QA is not leveraged<br />
    10. 10. Goals for 2010 Release - Testing<br />Align QA with the Lifecycle<br />Create tighter Dev/Test interaction<br />Enable highly leveraged QA teams<br />
    11. 11. Visual Studio 2010 Test Capabilities<br />specialist<br />generalist<br />coded ui test<br />unit testing<br />web performance test<br />test runner<br />load test<br />test case management<br />virtual lab management<br />data diagnostic adapters (video, action log, event log etc)<br />team foundation server with reporting<br />(bugs, requirements, user stories, source control, build)<br />
    12. 12. Visual Studio 2010<br />
    13. 13. Test Planning<br />Test Process<br />Requirement<br />Requirement<br />Test Writing<br />Requirement<br />Test Automation<br />Test Execution<br />Report Results<br />
    14. 14. Testing Features<br /><ul><li>QA Professional focused interface
    15. 15. Work Item based test case management, test planning and status tracking
    16. 16. Focused on Requirements
    17. 17. Identification of high-priority tests
    18. 18. Robust manual testing functionality
    19. 19. Test definition
    20. 20. User-friendly Manual Test Runner
    21. 21. Defect data capture tools
    22. 22. Automated user interface testing</li></li></ul><li>Visual Studio & TFS Testing Tools<br /><ul><li>Test Planning
    23. 23. Test Case Management
    24. 24. Virtual Lab Management
    25. 25. Test Recording
    26. 26. Data Collectors
    27. 27. Automated UI Testing
    28. 28. Web Performance Testing
    29. 29. Load Testing
    30. 30. Generic Tests
    31. 31. Ordered Tests
    32. 32. Unit Tests</li></li></ul><li>11<br />Test Case Management<br />
    33. 33. 12<br />Test Case is first class work item in TFS<br />Detailed steps along with expected results<br />
    34. 34. 13<br />Data drive a test case<br />Specify data in parameter table<br />
    35. 35. 14<br />Select common steps and convert to Shared Steps<br />
    36. 36. 15<br />Shared Steps allows for common steps to be reused across test cases<br />
    37. 37. 16<br />Organize Tests using Static (folder) suite<br />
    38. 38. 17<br />Or create new test cases & add to suite<br />Add existing Test cases to suite<br />
    39. 39. 18<br />Organize Tests using Requirements based suite<br />
    40. 40. 19<br />Select requirement and add to plan<br />
    41. 41. 20<br />Tests already linked to requirement are added to plan<br />
    42. 42. 21<br />Assign tests to team members<br />
    43. 43. 22<br />Organize Tests using Query Based (dynamic) suite<br />
    44. 44. 23<br />Specify a Query<br />
    45. 45. 24<br />Tests matching query are added to suite<br />
    46. 46. 25<br />Rich Bug and Test Runner<br />
    47. 47. Better Tester / Developer Collaboration<br /><ul><li>Automatically gathers defect and runtime details
    48. 48. Eliminates the “ping pong” effect
    49. 49. Eliminates the “No Repro” problem</li></li></ul><li>27<br />Select Test Case & click Run<br />
    50. 50. 28<br />Microsoft Test Runner starts up<br />
    51. 51. 29<br />Mark steps (pass/fail) during testing<br />Error free data entry of parameters using CTRL+V<br />
    52. 52. 30<br />Mark step as fail – bug found<br />
    53. 53. 31<br />Add comment<br />
    54. 54. 32<br />Take screenshot<br />
    55. 55. 33<br />Click on Create Bug<br />
    56. 56. Data Collectors<br /><ul><li>Action Recording
    57. 57. System Information
    58. 58. Event Logs
    59. 59. IntelliTrace
    60. 60. Test Impact
    61. 61. Video Recorder
    62. 62. [Custom Data Collector]</li></li></ul><li>35<br />Bug Form auto-filled with step results & step attachments<br />
    63. 63. 36<br />Indexed Video links<br />
    64. 64. 37<br />Data collector logs auto attached<br />
    65. 65. 38<br />Exact actions performed for each step<br />
    66. 66. 39<br />System Info across all VMs in environment<br />
    67. 67. 40<br />Link to test case auto-created<br />
    68. 68. 41<br />Just enter Title and save the bug <br />
    69. 69. IntelliTrace<br /><ul><li>Reduces debug “cycling”
    70. 70. Start Debug
    71. 71. Encounter defect
    72. 72. Drop into debugger
    73. 73. Move back through execution history to locate defect
    74. 74. Historical debug trace enables debugging offline test runs
    75. 75. “As you sit in the debugger, being able to move backwards while in the debugger (or attach the execution log to a bug) has to be the killer feature coming up in the entire Visual Studio 2010 release.” -John Robbins</li></li></ul><li>IntelliTrace<br />
    76. 76. Test Impact Analysis<br /><ul><li>Identifies tests impacted by code changes
    77. 77. Significantly reduces test time</li></li></ul><li>Coded UI Testing<br />
    78. 78. Coded UI Testing<br />Enables verification of UI functionality.<br />Record test actions and VS will generate the resulting code.<br />Assertions can be added to verify functionality.<br />Like all VS tests, results can be stored in TFS.<br />
    79. 79. 47<br />From Test->new test, click Coded UI test, specify test name and project name<br />
    80. 80. 48<br />Use existing action recording to generate code<br />
    81. 81. 49<br />Specify test case which has action recording and click OK<br />
    82. 82. 50<br />Code generated from action recording – datasource as TestCase<br />
    83. 83. 51<br />Code generated handles data parameterization<br />
    84. 84. 52<br />Invoke Coded UI Test Builder for adding assertions<br />
    85. 85. 53<br />Drag cross-hair onto the control you want to add assertions on<br />
    86. 86. 54<br />Properties of selected control and its UIMap is shown<br />
    87. 87. 55<br />Select property to assert on and right click to add assertion<br />
    88. 88. 56<br />Add the assert<br />
    89. 89. 57<br />Specify Assert method name and generate code<br />
    90. 90. 58<br />Code generated for assert method<br />
    91. 91. 59<br />Add solution to source control<br />
    92. 92. 60<br />Check-in pending changes<br />
    93. 93. 61<br />Go to test case that you want to associate the newly created automation<br />
    94. 94. 62<br />Specify test method and click OK; save test case<br />
    95. 95. 63<br />Newly automated test case will be picked up by dynamic query based suite and run as part of build-deploy-test workflow<br />
    96. 96. Platform Compatibility<br />
    97. 97. 65<br />Virtual Environment Provisioning<br />
    98. 98. Test Lab Benefits<br /><ul><li>Simplifies environment setup
    99. 99. Simplifies testing multiple configurations
    100. 100. Enables reverting to baseline configuration
    101. 101. Provides checkpoints to aid defect resolution
    102. 102. Improves test hardware utilization</li></li></ul><li>Lab Management Goals<br /><ul><li>Virtualize test lab resources
    103. 103. Lower technical requirements for test environment creation
    104. 104. Enable environment recreation for defect resolution</li></li></ul><li>Team Foundation Server (TFS)<br />Hyper-V Hosts<br />Library Shares<br />Lab Management in Visual Studio 2010<br />LS1<br />VM1<br />LS2<br />VM2<br />System Central Virtual Machine Manager (SCVMM)<br />Lab Management<br />LSn<br />VMn<br />Test Case Management<br />Auomated Builds<br />Work Item Tracking<br />Version Control<br />
    105. 105. Template Types<br /><ul><li>Virtual Machine Template
    106. 106. One “golden” VM in a specific configuration
    107. 107. i.e. Win 7 x64, with SQL 2008 R2 & Office
    108. 108. Environment Template
    109. 109. Multiple VM’s in a specific combination</li></li></ul><li>70<br />Create Virtual Environment from Lab Center<br />
    110. 110. 71<br />Specify name and location for environment<br />
    111. 111. 72<br />Pick role for VM<br />Select VM from template library<br />
    112. 112. 73<br />Specify capabilities – “Run Tests” ; “Run Workflows” ; “Network Isolation”<br />
    113. 113. 74<br />Environment (collection of VMs) created<br />
    114. 114. 75<br />Manage snapshots for environment<br />Access environment using Lab Environment Viewer<br />
    115. 115. 76<br />Take environment snapshot<br />
    116. 116. 77<br />Dev clicks on lvr link & selects connect to snapshot of the environment<br />
    117. 117. 78<br />Dev is able to access the snapshot’ed environment via LE viewer<br />
    118. 118. Lab Management 2010 Capability/Features<br />Virtual Environments (Hyper-V)<br />Physical environments<br />Virtual environments (non Hyper-V e.g. VMWare)<br />Visual Studio 2010 Capability/Feature<br />Testing<br />Running unit tests<br />Running manual tests<br />Running automated, load, and coded UI tests<br />Filing rich bugs with diagnostic adapters<br />Build deployment<br />Automated build-deploy-test workflow<br /> Environment creation and management<br />Create environment from VM templates<br />Start/stop/snapshot environment<br />Connect to environment using Environment Viewer<br />Clone environments using network isolation<br />Supported out of box<br />Not supported, but possible with customization<br />Not Supported <br />N/A<br />
    119. 119. 80<br />Automated Build – Deploy - Test<br />
    120. 120. 81<br />Create new build workflow<br />
    121. 121. 82<br />Provide a name<br />
    122. 122. 83<br />Specify Build Controller and output drop folder<br />
    123. 123. 84<br />Select LabDefaultTemplate<br />
    124. 124. 85<br />Specify workflow settings<br />
    125. 125. 86<br />Select environment<br />Revert to a specific snapshot<br />
    126. 126. 87<br />Select build definition to build the application<br />
    127. 127. 88<br />Specify deploy scripts<br />Take snapshot after deployment<br />
    128. 128. 89<br />Specify tests to run post deployment<br />
    129. 129. 90<br />Detailed Build report<br />
    130. 130. Reporting<br /><ul><li>By Test Plan
    131. 131. By Requirement
    132. 132. By Automated Build
    133. 133. Quality Dashboard
    134. 134. Bug Tracking Reports</li></li></ul><li>Improved Quality Reporting<br />
    135. 135. Improved Test Reporting<br />
    136. 136. Improved Test Reporting<br />
    137. 137. Improved Test Reporting<br />
    138. 138. Improved Bug Reporting<br />
    139. 139. Improved Bug Reporting<br />
    140. 140. Dashboards<br />Excel Services Reports<br />Team Web Access<br />Sharepoint Web Parts<br />
    141. 141. Questions?<br />
    142. 142. www.notionsolutions.com<br />

    ×