Team Foundation Server 2010
Testing Workshop
Ed Blankenship
Microsoft MVP (TFS, Visual Studio ALM, and VSTS)
Notion Soluti...
About Ed Blankenship
Microsoft MVP of the Year
ALM/TFS Consultant at Notion Solutions
Live in Charleston, SC
Blog:  http:/...
Frustrations in development teams
“my testers are spending too long testing the same thing”
“tooling is expensive (time, l...
Customer Challenges 
QA is not aligned with lifecycle
Developers and testers work in silos
QA is not leveraged
Goals for 2010 Release - Testing
Align QA with the Lifecycle
Create tighter Dev/Test interaction
Enable highly leveraged  ...
Visual Studio 2010 Test Capabilities
specialist
generalist
coded ui test
unit testing
web performance test
test runner
loa...
Visual Studio 2010
Test Planning
Test Process
Requirement
Requirement
Test Writing
Requirement
Test Automation
Test Execution
Report Results
Testing Features
QA Professional focused interface
Work Item based test case management, test planning and status tracking...
Testing Tools
Test Planning
Test Case Management
Test Lab Management
Test Recording
Test Collectors
Automated UI Testing
W...
11
Test Case Management
12
Test Case is first class work item in TFS
Detailed steps along with expected results
13
Data drive a test case
Specify data in parameter table
14
Select common steps and convert to Shared Steps
15
Shared Steps allows for common steps to be reused across test cases
16
Organize Tests using Static (folder) suite
17
Or create new test cases & add to suite
Add existing Test cases to suite
18
Organize Tests using Requirements based suite
19
Select requirement and add to plan
20
Tests already linked to requirement are added to plan
21
Assign tests to team members
22
Organize Tests using Query Based (dynamic) suite
23
Specify a Query
24
Tests matching query are added to suite
25
Rich Bug
Better Tester / Developer Collaboration
Automatically gathers defect and runtime details
Eliminates the “ping pong” effect...
27
Select Test Case & click Run
28
Microsoft Test Runner starts up
29
Mark steps (pass/fail) during testing
Error free data entry of parameters using CTRL+V
30
Mark step as fail – bug found
31
Add comment
32
Take screenshot
33
Take environment snapshot
34
Click on Create Bug
Data Collectors
Action Recording
System Information
Event Logs
IntelliTrace
Test Impact
Video Recorder
[Custom Data Collec...
36
Bug Form auto-filled with step results & step attachments
37
Indexed Video links
38
Data collector logs auto attached
39
Exact actions performed for each step
40
System Info across all VMs in environment
41
Link to test case auto-created
42
Just enter Title and save the bug 
43
Dev clicks on lvr link & selects connect to snapshot of the environment
44
Dev is able to access the snapshot’ed environment via LE viewer
IntelliTrace
Reduces debug “cycling”
Start Debug
Encounter defect
Drop into debugger
Move back through execution history t...
IntelliTrace
Test Impact Analysis
Identifies tests impacted by code changes
Significantly reduces test time
Coded UI Testing
Coded UI Testing
Enables verification of UI functionality.
Record test actions and VS will generate the resulting code.
As...
50
From Test->new test, click Coded UI test, specify test name and project name
51
Use existing action recording to generate code
52
Specify test case which has action recording and click OK
53
Code generated from action recording – datasource as TestCase
54
Code generated handles data parameterization
55
Invoke Coded UI Test Builder for adding assertions
56
Drag cross-hair onto the control you want to add assertions on
57
Properties of selected control and its UIMap is shown
58
Select property to assert on and right click to add assertion
59
Add the assert
60
Specify Assert method name and generate code
61
Code generated for assert method
62
Add solution to source control
63
Check-in pending changes
64
Go to test case that you want to associate the newly created automation
65
Specify test method and click OK; save test case
66
Newly automated test case will be picked up by dynamic query based suite and run as part of build-deploy-test workflow
Platform Compatibility
68
Virtual Environment Provisioning
Test Lab Benefits
Simplifies environment setup
Simplifies testing multiple configurations
Enables reverting to baseline co...
Lab Management Goals
Virtualize test lab resources
Lower technical requirements for test environment creation
Enable envir...
Team Foundation Server (TFS)
Hyper-V Hosts
Library Shares
Lab Management in Visual Studio 2010
LS1
VM1
LS2
VM2
System Cent...
72
Create Virtual Environment from Lab Center
73
Specify name and location for environment
74
Pick role for VM
Select VM from template library
75
Specify capabilities – “Run Tests” ; “Run Workflows” ; “Network Isolation”
76
Environment (collection of VMs) created
77
Manage snapshots for environment
Access environment using Lab Environment Viewer
Lab Management 2010 Capability/Features
Virtual Environments (Hyper-V)
Physical environments
Virtual environments (non Hyp...
79
Automated Build – Deploy - Test
80
Create new build workflow
81
Provide a name
82
Specify Build Controller and output drop folder
83
Select LabDefaultTemplate
84
Specify workflow settings
85
Select environment
Revert to a specific snapshot
86
Select build definition to build the application
87
Specify deploy scripts
Take snapshot after deployment
88
Specify tests to run post deployment
89
Detailed Build report
Reporting
By Test Plan
By Requirement
By Automated Build
Quality Dashboard
Bug Tracking Reports
Improved Quality Reporting
Improved Test Reporting
Improved Test Reporting
Improved Test Reporting
Improved Bug Reporting
Improved Bug Reporting
Dashboards
Excel Services Reports
Team Web Access
Sharepoint Web Parts
Questions?
www.notionsolutions.com
Upcoming SlideShare
Loading in...5
×

Full Testing Experience - Visual Studio and TFS 2010

26,644

Published on

This presentation goes through the full testing experience of Visual Studio 2010 and Team Foundation Server 2010 including using the new Lab Management features in the full testing process.

Published in: Technology
0 Comments
38 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
26,644
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
38
Embeds 0
No embeds

No notes for slide

Full Testing Experience - Visual Studio and TFS 2010

  1. 1. Team Foundation Server 2010 Testing Workshop Ed Blankenship Microsoft MVP (TFS, Visual Studio ALM, and VSTS) Notion Solutions, Inc. www.notionsolutions.com
  2. 2. About Ed Blankenship Microsoft MVP of the Year ALM/TFS Consultant at Notion Solutions Live in Charleston, SC Blog: http://edsquared.com Twitter: @EdBlankenship E-Mail: edb@notionsolutions.com Author - Wrox TFS 2010 Book: http://bit.ly/TFS2010Book
  3. 3. Frustrations in development teams “my testers are spending too long testing the same thing” “tooling is expensive (time, licenses, people)” “the developers say the defects are useless” “when is my software ready to ship?” “we have a changed requirement, what do I need to test?” “developers and testers work in silos and don’t communicate/speak the same language” 3
  4. 4. Customer Challenges QA is not aligned with lifecycle Developers and testers work in silos QA is not leveraged
  5. 5. Goals for 2010 Release - Testing Align QA with the Lifecycle Create tighter Dev/Test interaction Enable highly leveraged QA teams
  6. 6. Visual Studio 2010 Test Capabilities specialist generalist coded ui test unit testing web performance test test runner load test test case management virtual lab management data diagnostic adapters (video, action log, event log etc) team foundation server with reporting (bugs, requirements, user stories, source control, build)
  7. 7. Visual Studio 2010
  8. 8. Test Planning Test Process Requirement Requirement Test Writing Requirement Test Automation Test Execution Report Results
  9. 9. Testing Features QA Professional focused interface Work Item based test case management, test planning and status tracking Focused on Requirements Identification of high-priority tests Robust manual testing functionality Test definition User-friendly Manual Test Runner Defect data capture tools Automated user interface testing
  10. 10. Testing Tools Test Planning Test Case Management Test Lab Management Test Recording Test Collectors Automated UI Testing Web Performance Testing or Load Testing Load Testing Generic Tests Ordered Tests Unit Tests
  11. 11. 11 Test Case Management
  12. 12. 12 Test Case is first class work item in TFS Detailed steps along with expected results
  13. 13. 13 Data drive a test case Specify data in parameter table
  14. 14. 14 Select common steps and convert to Shared Steps
  15. 15. 15 Shared Steps allows for common steps to be reused across test cases
  16. 16. 16 Organize Tests using Static (folder) suite
  17. 17. 17 Or create new test cases & add to suite Add existing Test cases to suite
  18. 18. 18 Organize Tests using Requirements based suite
  19. 19. 19 Select requirement and add to plan
  20. 20. 20 Tests already linked to requirement are added to plan
  21. 21. 21 Assign tests to team members
  22. 22. 22 Organize Tests using Query Based (dynamic) suite
  23. 23. 23 Specify a Query
  24. 24. 24 Tests matching query are added to suite
  25. 25. 25 Rich Bug
  26. 26. Better Tester / Developer Collaboration Automatically gathers defect and runtime details Eliminates the “ping pong” effect Eliminates the “No Repro” problem
  27. 27. 27 Select Test Case & click Run
  28. 28. 28 Microsoft Test Runner starts up
  29. 29. 29 Mark steps (pass/fail) during testing Error free data entry of parameters using CTRL+V
  30. 30. 30 Mark step as fail – bug found
  31. 31. 31 Add comment
  32. 32. 32 Take screenshot
  33. 33. 33 Take environment snapshot
  34. 34. 34 Click on Create Bug
  35. 35. Data Collectors Action Recording System Information Event Logs IntelliTrace Test Impact Video Recorder [Custom Data Collector]
  36. 36. 36 Bug Form auto-filled with step results & step attachments
  37. 37. 37 Indexed Video links
  38. 38. 38 Data collector logs auto attached
  39. 39. 39 Exact actions performed for each step
  40. 40. 40 System Info across all VMs in environment
  41. 41. 41 Link to test case auto-created
  42. 42. 42 Just enter Title and save the bug 
  43. 43. 43 Dev clicks on lvr link & selects connect to snapshot of the environment
  44. 44. 44 Dev is able to access the snapshot’ed environment via LE viewer
  45. 45. IntelliTrace Reduces debug “cycling” Start Debug Encounter defect Drop into debugger Move back through execution history to locate defect Historical debug trace enables debugging offline test runs “As you sit in the debugger, being able to move backwards while in the debugger (or attach the execution log to a bug) has to be the killer feature coming up in the entire Visual Studio 2010 release.” -John Robbins
  46. 46. IntelliTrace
  47. 47. Test Impact Analysis Identifies tests impacted by code changes Significantly reduces test time
  48. 48. Coded UI Testing
  49. 49. Coded UI Testing Enables verification of UI functionality. Record test actions and VS will generate the resulting code. Assertions can be added to verify functionality. Like all VS tests, results can be stored in TFS.
  50. 50. 50 From Test->new test, click Coded UI test, specify test name and project name
  51. 51. 51 Use existing action recording to generate code
  52. 52. 52 Specify test case which has action recording and click OK
  53. 53. 53 Code generated from action recording – datasource as TestCase
  54. 54. 54 Code generated handles data parameterization
  55. 55. 55 Invoke Coded UI Test Builder for adding assertions
  56. 56. 56 Drag cross-hair onto the control you want to add assertions on
  57. 57. 57 Properties of selected control and its UIMap is shown
  58. 58. 58 Select property to assert on and right click to add assertion
  59. 59. 59 Add the assert
  60. 60. 60 Specify Assert method name and generate code
  61. 61. 61 Code generated for assert method
  62. 62. 62 Add solution to source control
  63. 63. 63 Check-in pending changes
  64. 64. 64 Go to test case that you want to associate the newly created automation
  65. 65. 65 Specify test method and click OK; save test case
  66. 66. 66 Newly automated test case will be picked up by dynamic query based suite and run as part of build-deploy-test workflow
  67. 67. Platform Compatibility
  68. 68. 68 Virtual Environment Provisioning
  69. 69. Test Lab Benefits Simplifies environment setup Simplifies testing multiple configurations Enables reverting to baseline configuration Provides checkpoints to aid defect resolution Improves test hardware utilization
  70. 70. Lab Management Goals Virtualize test lab resources Lower technical requirements for test environment creation Enable environment recreation for defect resolution
  71. 71. Team Foundation Server (TFS) Hyper-V Hosts Library Shares Lab Management in Visual Studio 2010 LS1 VM1 LS2 VM2 System Central Virtual Machine Manager (SCVMM) Lab Management LSn VMn Test Case Management Auomated Builds Work Item Tracking Version Control
  72. 72. 72 Create Virtual Environment from Lab Center
  73. 73. 73 Specify name and location for environment
  74. 74. 74 Pick role for VM Select VM from template library
  75. 75. 75 Specify capabilities – “Run Tests” ; “Run Workflows” ; “Network Isolation”
  76. 76. 76 Environment (collection of VMs) created
  77. 77. 77 Manage snapshots for environment Access environment using Lab Environment Viewer
  78. 78. Lab Management 2010 Capability/Features Virtual Environments (Hyper-V) Physical environments Virtual environments (non Hyper-V e.g. VMWare) Visual Studio 2010 Capability/Feature Testing Running unit tests Running manual tests Running automated, load, and coded UI tests Filing rich bugs with diagnostic adapters Build deployment Automated build-deploy-test workflow  Environment creation and management Create environment from VM templates Start/stop/snapshot environment Connect to environment using Environment Viewer Clone environments using network isolation Supported out of box Not supported, but possible with customization Not Supported N/A
  79. 79. 79 Automated Build – Deploy - Test
  80. 80. 80 Create new build workflow
  81. 81. 81 Provide a name
  82. 82. 82 Specify Build Controller and output drop folder
  83. 83. 83 Select LabDefaultTemplate
  84. 84. 84 Specify workflow settings
  85. 85. 85 Select environment Revert to a specific snapshot
  86. 86. 86 Select build definition to build the application
  87. 87. 87 Specify deploy scripts Take snapshot after deployment
  88. 88. 88 Specify tests to run post deployment
  89. 89. 89 Detailed Build report
  90. 90. Reporting By Test Plan By Requirement By Automated Build Quality Dashboard Bug Tracking Reports
  91. 91. Improved Quality Reporting
  92. 92. Improved Test Reporting
  93. 93. Improved Test Reporting
  94. 94. Improved Test Reporting
  95. 95. Improved Bug Reporting
  96. 96. Improved Bug Reporting
  97. 97. Dashboards Excel Services Reports Team Web Access Sharepoint Web Parts
  98. 98. Questions?
  99. 99. www.notionsolutions.com

×