Your SlideShare is downloading. ×
0
DAY 1MTLM
Testing Practices with ALM toolsGOAL general course:Know the value of MTLM and know how to use its capabilities …GOAL deep...
Lot of discussionsAgenda Day 1                                            and hands-on lab’s Start   Topics 9:00    Introd...
Environments setup     TFS supported operating system     Installed:     • Team Foundation Server with Basic configuration...
Lab 1Create one team projects for the demo applications             Timebox: 15 minuts
People Introductions
Clemens ReijnenSogeti Netherlands | Management Consultanthttp://www.ClemensReijnen.nl@ClemensReijnen
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Application LifecycleManagementGOAL:Understand the term ALM, know it’s value and Visual Studio’s ALM support
Suite of Application Lifecycle Management toolshttp://msdn.microsoft.com/en-us/library/fda2bad5.aspx
application lifecycle managementhttp://www.clemensreijnen.nl/post/2008/02/18/ALM-Definitions.aspx
work together__with work packages__on alm artifacts
Testing LifecycleSee also on www.tmap.net:- The Test Lifecycle Model- The 4 essentials
http://msdn.microsoft.com/en-us/library/bb385832.aspx
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Work items
Testing LifecycleTest Activities Planning…See also on www.tmap.net: Planning Phase
http://msdn.microsoft.com/en-us/library/bb385832.aspx
VS2010 Process Support with Process TemplatesMultiple process template
Artifact types  Work items   Reports  Workbooks  Dashboards    Metrics
User Story Work Item Type
Work Item Type FlowUser Story
Testing processes in the Lifecycle
Testing processes in the Lifecycle
Process template with work items, flows,         links, queries and more       support | drive | enforce              a wa...
Lab 2Create user stories (2 or 3)and development and test tasks (optional)for the employee application.             Timebo...
Workitem Queriesand Reports
Queries
create your own
Create your own – advanced –Query Types
Default Reports
Excel Reports
create your ownExcel basic
create your ownReport WizardExcel Pivot
Test Activities Planning…See also on www.tmap.net: Planning Phase
Microsoft Test Manager
Microsoft Test Manager 2010The UX
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Test Case ManagementSee also on www.tmap.net: Control Phase
Test Plans    Test Suites    Test ConfigurationsOrganize the testing effort in a Test Execution PlanCreate dynamic test su...
Test (Execution) Plans
Test (Execution) Plans
Test Plan OrganizationNaming of test plans for agile environments:Iteration number and name and one extra test plan for en...
Test Configurations
Test Configurations
Test Suites
Test Suites OrganizationNaming of test suites for agile environments:Suite per user story (requirement based) and one addi...
Proposed organization
Iteration 1Create test plan for iteration and a ‘end to end’test plan.Create requirement based test suites whichhold test ...
Iteration 1Create two additional suites, one forregression sets, which will be taken alongduring the project and one speci...
Copy Test Cases        Copy a Test Case   Copy the reference                           to a Test Case           ID 21     ...
End iteration 1, some important test case arepromoted to the ‘end to end ’ test plan.
Iteration 2, test suites are created for therequirements and the regression test suit iscopied (link or hard?)
During the execution of iteration 2 theregression test suit grows and an automationsuit is added.
Again important ones are promoted to the‘end to end’ test plan
TCM.exe
TCM plan /list
TCM suites /lists planid:16
Lab 3Create Test Plan, Test Configurations and makea test suite organization.Optional: use TCM.exe to list the lab results...
Share MTM Artifactsmtm://clemenes-pc:8080/tfs/defaultcollection/p:MSF/Testing/testplan/connect?id=19
mtm://<server name>:<port>/<tfs vdir>/<Collection name>/p:<project name>/<center group>/<group specific>               Ope...
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Test Case SpecificationSee also on www.tmap.net: Specification Phase
Test Case    Shared Steps    Test Data IterationsLink test cases and requirements to provide requirement traceabilityData ...
Test Case
Test CaseNaming of test case:Purpose:• Finding the right test case• Common understanding• Intention from the title• Scan t...
Test Case from Excel
Test step types
Test steps   Make them descriptive   Not every step is a validation step   Use the step attachment
Test steps copy-past
Test Data IterationsData drive test cases with different parameters
Test Data Iterations
Test data copy-past
Shared StepsRefactor common shared steps in test casesReduce test maintenance by sharing test steps across test cases
Shared Steps     Shared Step candidates?• REUSE – Many steps, reused for many test cases . Benefit: time, retyping, boring...
Shared Steps    what about parameters    and expected result in    shared steps?
Lab 41. Specify multiple test cases.2. Add some test data iterations.3. Create shared step and reuse it.Optional:play with...
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Test ExecutionTest RunnerA test that you will want to run multiple times.Different manual tests that contain common steps....
Test Case ExecutionSee also on www.tmap.net: Execution Phase
Test Case Execution
Action recording
TCM run/create
Test Case ExecutionResults
View Test Results Using Microsoft Test Manager1: from the ‘Run Tests’2: from the ‘Analyze Test Runs’
From Analyze Test RunsTest Results for multiple Test Cases
From Run TestsTest Result for a Test Case
Test Plan
From TCMrun /export id:239 /resultsfile:”c:runresult”
Share with mtm:// mtm://clemens-pc:8080/tfs/defaultcollection/p:MSF/Testing/testrun/open?id=292
Lab 5Execute the created test cases, view the resultsOptional: run them from TCM.exe             Timebox: 10 minuts
Test queries
Test reports
Excel Reports for Tests
•   POINT COUNT TREND - I want to see the results for test in a particular test run•   RESULT COUNT - I want to see trend ...
Microsoft Test Manager Test Artifactorganization
Test Plans and TFS items  MTLM Client                                    Server             TCM Server                    ...
Lab 6Create excel pivot table to list test points perTester.              Timebox: 10 minuts
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Controlling the testing effortBUG’s
File a BUG
Diagnostic Data AdaptersFile a BugFile an Exploratory BugVerify a Bug
Diagnostic Data Adapters
File a BugSubmit a Bug using Test Runner
Ways to file a Bug   Submit a Bug from the Test Runner   Submit a Bug from the Verify Bugs View   Submit a Bug from Test R...
Verify a BugVerify a Bug Fix Using Microsoft Test Manager
The Bug FlowLink build 2 testplan.                T   Execute   File Bug    Verify fix   Close orConfigure Data       E   ...
Lab 7Create a buildFile a bugResolve a bugVerify a bug             Timebox: 15 minuts
Bug Reports    Bug Status Report    Bug Trends Report    Reactivations Report
Bug Status Report
Bug Trends Report
Reactivations Report
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Test Impact
Test ImpactAssign build 2test plan.           T   Execute            Assign        Evaluated and reset                    ...
Lab 8Attach build to test planExecute testsMake code changeKickoff buildAttach new build to test planEvaluate impacted tes...
E2E Scenario
Iteration N                                   ImplementD                                     US2  SprintE plan         Imp...
User Stories        Key Value: Requirements Planning                              and corresponding                      I...
Execute the Test        Key Value: Requirements Planning                                      Specification Tasks         ...
Execute the Test        Key Value: Requirements Planning                           Implement                              ...
Key Value: Virtuous Bug CycleDev                                  Implement D                                      US2solv...
Key Value: Virtuous Bug CycleCheck                                Implement D                                      US2Resu...
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Create test from Bug
File an Exploratory BugSubmit an Exploratory Bug using Test Runner
PRO:                                                 -   Reuse scenario knowledge business user                           ...
Lab 9Create an exploratory bug and create a test case from it             Timebox: 10 minuts
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Test InfrastructureWhere and how to run the automated test?
Test EnvironmentsSee also on www.TMap.net:- Checklist "Intake Test Environment“- Checklist "Test Environment“- Infrastruct...
Test labVisual Studio 2010 Lab Management
Where and how to run the automated test?physical test environmentsvirtualized test environments
physical test                 virtualized testenvironments                  environments  MTLM Client                   MT...
Build            application                                        physical test environments                            ...
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
Lab for PhysicalEnvironments, thedifferent flavors…Visual Studio   Team                Foundation                ServerTes...
Configurations for automated test execution:                                  AFlavor A: Execution from VS2010… Real devel...
Configurations for automated test execution:                                                             BFlavor B: Execut...
Configurations for automated test execution:                                                                     CFlavor C...
Configurations for automated test execution:                                               DFlavor D: Execution from Micro...
Configurations for automated test execution:                                                               EFlavor E: Exec...
Test Controller
Test Agent
Lab 10             Install and configure the test controller and test agenthttp://msdn.microsoft.com/en-us/library/dd64812...
Start   Topics9:00    Introduction, environments setup9:15    introduction by the students9:45    Application Lifecycle Ma...
New physical environment
RECAPTest Activities Planning…
Test Case Management                     RECAP
RECAPTest Case Specification
RECAPTest Case Execution
Test Case Management                     RECAP
Start   Topics                        DAY 29:00    Recap9:15    Create Basic CodedUI, from Test Case and Manual9:45    UIM...
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
MTLM Visual Studio 2010 ALM workshop  - day1
Upcoming SlideShare
Loading in...5
×

MTLM Visual Studio 2010 ALM workshop - day1

3,960

Published on

MTLM Visual Studio 2010 ALM workshop - day1 deck

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,960
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
132
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "MTLM Visual Studio 2010 ALM workshop - day1"

  1. 1. DAY 1MTLM
  2. 2. Testing Practices with ALM toolsGOAL general course:Know the value of MTLM and know how to use its capabilities …GOAL deep dive:Tune the value of MTLM, its capabilities and the processes it supports…
  3. 3. Lot of discussionsAgenda Day 1 and hands-on lab’s Start Topics 9:00 Introduction, environments setup 9:15 introduction by the students 9:45 Application Lifecycle Management and Visual Studio General 9:55 Test Planning 10:15 Managing, controlling the testing effort, test plans, test suites and test configuration 10:45 11:00 Specify test cases and corresponding capabilities of MTM 11:45 Test execution, fast forward, reporting, olap cub 12:00 12:45 Bug workflow, infrastructure configuration for data diagnostic adapters 13:30 Test impact and recommended tests 14:15 14:30 Exploratory testing and create test from bug 14:45 Test Controllers and Agents 15:45 Test Environments in MTLM
  4. 4. Environments setup TFS supported operating system Installed: • Team Foundation Server with Basic configuration • Visual Studio 2010 with SP1 • Microsoft Test Manager 2010 (comes with VS10 Ultimate) You get an USB with: • Demo applications (we create a Team Project for them, see lab 1) • Microsoft Test Agents.msi (we install this together) • Hands-on lab documents • This deck, talk through and useful information
  5. 5. Lab 1Create one team projects for the demo applications Timebox: 15 minuts
  6. 6. People Introductions
  7. 7. Clemens ReijnenSogeti Netherlands | Management Consultanthttp://www.ClemensReijnen.nl@ClemensReijnen
  8. 8. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:4511:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:0012:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:1514:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  9. 9. Application LifecycleManagementGOAL:Understand the term ALM, know it’s value and Visual Studio’s ALM support
  10. 10. Suite of Application Lifecycle Management toolshttp://msdn.microsoft.com/en-us/library/fda2bad5.aspx
  11. 11. application lifecycle managementhttp://www.clemensreijnen.nl/post/2008/02/18/ALM-Definitions.aspx
  12. 12. work together__with work packages__on alm artifacts
  13. 13. Testing LifecycleSee also on www.tmap.net:- The Test Lifecycle Model- The 4 essentials
  14. 14. http://msdn.microsoft.com/en-us/library/bb385832.aspx
  15. 15. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:4511:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:0012:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:1514:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  16. 16. Work items
  17. 17. Testing LifecycleTest Activities Planning…See also on www.tmap.net: Planning Phase
  18. 18. http://msdn.microsoft.com/en-us/library/bb385832.aspx
  19. 19. VS2010 Process Support with Process TemplatesMultiple process template
  20. 20. Artifact types Work items Reports Workbooks Dashboards Metrics
  21. 21. User Story Work Item Type
  22. 22. Work Item Type FlowUser Story
  23. 23. Testing processes in the Lifecycle
  24. 24. Testing processes in the Lifecycle
  25. 25. Process template with work items, flows, links, queries and more support | drive | enforce a way of work
  26. 26. Lab 2Create user stories (2 or 3)and development and test tasks (optional)for the employee application. Timebox: 5 minuts
  27. 27. Workitem Queriesand Reports
  28. 28. Queries
  29. 29. create your own
  30. 30. Create your own – advanced –Query Types
  31. 31. Default Reports
  32. 32. Excel Reports
  33. 33. create your ownExcel basic
  34. 34. create your ownReport WizardExcel Pivot
  35. 35. Test Activities Planning…See also on www.tmap.net: Planning Phase
  36. 36. Microsoft Test Manager
  37. 37. Microsoft Test Manager 2010The UX
  38. 38. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:4511:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:0012:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:1514:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  39. 39. Test Case ManagementSee also on www.tmap.net: Control Phase
  40. 40. Test Plans Test Suites Test ConfigurationsOrganize the testing effort in a Test Execution PlanCreate dynamic test suites for changing selection criteriaDefine various test configurations and share automation across those
  41. 41. Test (Execution) Plans
  42. 42. Test (Execution) Plans
  43. 43. Test Plan OrganizationNaming of test plans for agile environments:Iteration number and name and one extra test plan for endto end tests.
  44. 44. Test Configurations
  45. 45. Test Configurations
  46. 46. Test Suites
  47. 47. Test Suites OrganizationNaming of test suites for agile environments:Suite per user story (requirement based) and one additionaltest suite for regression tests.(query based?)(test cases are copied to the regression test suite)
  48. 48. Proposed organization
  49. 49. Iteration 1Create test plan for iteration and a ‘end to end’test plan.Create requirement based test suites whichhold test cases for that suite.
  50. 50. Iteration 1Create two additional suites, one forregression sets, which will be taken alongduring the project and one specific forautomation.
  51. 51. Copy Test Cases Copy a Test Case Copy the reference to a Test Case ID 21 ID 21 ID 22 ID 21
  52. 52. End iteration 1, some important test case arepromoted to the ‘end to end ’ test plan.
  53. 53. Iteration 2, test suites are created for therequirements and the regression test suit iscopied (link or hard?)
  54. 54. During the execution of iteration 2 theregression test suit grows and an automationsuit is added.
  55. 55. Again important ones are promoted to the‘end to end’ test plan
  56. 56. TCM.exe
  57. 57. TCM plan /list
  58. 58. TCM suites /lists planid:16
  59. 59. Lab 3Create Test Plan, Test Configurations and makea test suite organization.Optional: use TCM.exe to list the lab results Timebox: 10 minuts
  60. 60. Share MTM Artifactsmtm://clemenes-pc:8080/tfs/defaultcollection/p:MSF/Testing/testplan/connect?id=19
  61. 61. mtm://<server name>:<port>/<tfs vdir>/<Collection name>/p:<project name>/<center group>/<group specific> Open Test results Open Test Cases (or any work item) Open Test Runs Open Test Plans Connect to test plans mtm://server:8080/tfs/ProjectCollection/p:Project/testing/testcase/open?id=67http://blogs.msdn.com/b/vstsqualitytools/archive/2010/02/09/sharing-content-in-microsoft-test-manager.aspx
  62. 62. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:0012:45 BREAK TILL: Bug workflow, infrastructure configuration for data diagnostic adapters13:3014:15 Test impact and recommended tests 11:0014:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents (extra time to finish the labs :)15:45 Test Environments in MTLM
  63. 63. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:0012:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:1514:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  64. 64. Test Case SpecificationSee also on www.tmap.net: Specification Phase
  65. 65. Test Case Shared Steps Test Data IterationsLink test cases and requirements to provide requirement traceabilityData drive test cases with different parametersRefactor common shared steps in test cases
  66. 66. Test Case
  67. 67. Test CaseNaming of test case:Purpose:• Finding the right test case• Common understanding• Intention from the title• Scan test plans more effectively and efficientlyProposed solution#1 make use of linked work items#2 maximal length of the title 20 characters#3 format <functionality identifier> <action> <additional information>#4 make useful queries
  68. 68. Test Case from Excel
  69. 69. Test step types
  70. 70. Test steps Make them descriptive Not every step is a validation step Use the step attachment
  71. 71. Test steps copy-past
  72. 72. Test Data IterationsData drive test cases with different parameters
  73. 73. Test Data Iterations
  74. 74. Test data copy-past
  75. 75. Shared StepsRefactor common shared steps in test casesReduce test maintenance by sharing test steps across test cases
  76. 76. Shared Steps Shared Step candidates?• REUSE – Many steps, reused for many test cases . Benefit: time, retyping, boring work.• EXPECTED CHANGES – for example environment variables change [URL]. Benefit: Time, test cases which need this step are updated automatically, also action recording.• PRE- CONDITIONS – bring the system under test to a certain state and execute the different verifications. Benefit: beside time, you are certain the application under test is in the same state for every test case• BACK TO INITIAL STATE - after test execution bring the environment back to the initial state for new test cases. Benefit: beside time, you are certain the application under test is always in a clean state for new test cases.
  77. 77. Shared Steps what about parameters and expected result in shared steps?
  78. 78. Lab 41. Specify multiple test cases.2. Add some test data iterations.3. Create shared step and reuse it.Optional:play with the parameters and expected resultsteps in the shared steps… Timebox: 15 minuts
  79. 79. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:0012:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:1514:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  80. 80. Test ExecutionTest RunnerA test that you will want to run multiple times.Different manual tests that contain common steps.Verifying a bug is fixed by using a manual test.
  81. 81. Test Case ExecutionSee also on www.tmap.net: Execution Phase
  82. 82. Test Case Execution
  83. 83. Action recording
  84. 84. TCM run/create
  85. 85. Test Case ExecutionResults
  86. 86. View Test Results Using Microsoft Test Manager1: from the ‘Run Tests’2: from the ‘Analyze Test Runs’
  87. 87. From Analyze Test RunsTest Results for multiple Test Cases
  88. 88. From Run TestsTest Result for a Test Case
  89. 89. Test Plan
  90. 90. From TCMrun /export id:239 /resultsfile:”c:runresult”
  91. 91. Share with mtm:// mtm://clemens-pc:8080/tfs/defaultcollection/p:MSF/Testing/testrun/open?id=292
  92. 92. Lab 5Execute the created test cases, view the resultsOptional: run them from TCM.exe Timebox: 10 minuts
  93. 93. Test queries
  94. 94. Test reports
  95. 95. Excel Reports for Tests
  96. 96. • POINT COUNT TREND - I want to see the results for test in a particular test run• RESULT COUNT - I want to see trend report for my tests regardless of build or plan• RESULT COUNT TREND - I want to see trend report for my tests regardless of build or plan• TEST CASE COUNT - I want to see the number of test cases that have been planned for automation
  97. 97. Microsoft Test Manager Test Artifactorganization
  98. 98. Test Plans and TFS items MTLM Client Server TCM Server TFSPlan “Iter Plan “Iteration 2” User Story 2 1”  Suite US1 Suite US2 Test Case A Test Case B Test Case A Task XPlan “Iter Test Case C 3” Test Case D Test Case B Task Y Test Case C Task Z Test Case C
  99. 99. Lab 6Create excel pivot table to list test points perTester. Timebox: 10 minuts
  100. 100. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:00 Lunch12:45 LUNCH TILL: Bug workflow, infrastructure configuration for data diagnostic adapters13:3014:15 Test impact and recommended tests Break 12:4514:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents (extra time to finish the labs :)15:45 Test Environments in MTLM
  101. 101. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:00 Lunch12:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:15 Break14:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  102. 102. Controlling the testing effortBUG’s
  103. 103. File a BUG
  104. 104. Diagnostic Data AdaptersFile a BugFile an Exploratory BugVerify a Bug
  105. 105. Diagnostic Data Adapters
  106. 106. File a BugSubmit a Bug using Test Runner
  107. 107. Ways to file a Bug Submit a Bug from the Test Runner Submit a Bug from the Verify Bugs View Submit a Bug from Test Results for a Test
  108. 108. Verify a BugVerify a Bug Fix Using Microsoft Test Manager
  109. 109. The Bug FlowLink build 2 testplan. T Execute File Bug Verify fix Close orConfigure Data E Manual by using reopencollectors. S Tests playback Bug T Build definition is the intergrationSetup build anddeploy D Resolve Checkin E Bug Build V Deploy
  110. 110. Lab 7Create a buildFile a bugResolve a bugVerify a bug Timebox: 15 minuts
  111. 111. Bug Reports Bug Status Report Bug Trends Report Reactivations Report
  112. 112. Bug Status Report
  113. 113. Bug Trends Report
  114. 114. Reactivations Report
  115. 115. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:00 Lunch12:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:15 Break14:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  116. 116. Test Impact
  117. 117. Test ImpactAssign build 2test plan. T Execute Assign Evaluated and reset Execute ExecuteConfigure Data E ‘passed’ ‘NEW’ build tests impacted by ‘passed’ ‘passed’collectors. S Tests 2 Test Plan code churn Tests Tests T Build definition is the intergrationSetup build anddeploy D Make Checkin E code Build V changes Deploy
  118. 118. Lab 8Attach build to test planExecute testsMake code changeKickoff buildAttach new build to test planEvaluate impacted tests Timebox: 15 minuts
  119. 119. E2E Scenario
  120. 120. Iteration N ImplementD US2 SprintE plan Implement Fix bugsV User Story ‘1’ Fix bugs (US1) Build 1 Build 2 Build 3 Build 4 Build 5 Build 6 Build 7 Regress Write tests Test US1 &T Verify fixes impacted for US1 file bugsE Sprint testsS plan Write tests Test US2 &T for US2 Verify fixes file bugs
  121. 121. User Stories Key Value: Requirements Planning and corresponding ImplementDE Sprint plan Implement US2 done dev tasks. Fix bugsV User Story ‘1’ Fix bugs (US1) Create test tasks Build 1 Build 2 Build 3 Build 4 and 5 Build assign to 7 Build 6 Build someone else. Regress Write tests Test US1 &T Verify fixes impacted for US1 file bugsE Sprint testsS plan Write tests Test US2 &T for US2 Verify fixes file bugs
  122. 122. Execute the Test Key Value: Requirements Planning Specification Tasks ImplementD Sprint US2 and other testE plan Implement Fix bugsV User Story ‘1’ (US1) tasks Fix bugs Build 1 Build 2 Build 3 Build 4 Build 5 Build 6 Build 7 Regress Write tests Test US1 &T Verify fixes impacted for US1 file bugsE Sprint testsS plan Write tests Test US2 &T for US2 Verify fixes file bugs
  123. 123. Execute the Test Key Value: Requirements Planning Implement Execution TasksDE Sprint Implement US2 and file bugs (if Fix bugs planV User Story ‘1’ (US1) any) Fix bugs Build 1 Build 2 Build 3 Build 4 Build 5 Build 6 Build 7 Regress Write tests Test US1 &T Verify fixes impacted for US1 file bugsE Sprint testsS plan Write tests Test US2 &T for US2 Verify fixes file bugs
  124. 124. Key Value: Virtuous Bug CycleDev Implement D US2solves Sprint E plan Implement Fix bugs V User Story ‘1’the (US1) Fix bugsbugs. Build 1 Build 2 Build 3 Build 4 Build 5 Build 6 Build 7Test RegressVerifies WriteUS1 T for tests Test US1 & file bugs Verify fixes impacted E Sprint teststests S plan Write tests Test US2 & T for US2 Verify fixes file bugs
  125. 125. Key Value: Virtuous Bug CycleCheck Implement D US2Results Sprint E plan Implement Fix bugsV User Story ‘1’ Fix bugs (US1) Build 1 Build 2 Build 3 Build 4 Build 5 Build 6 Build 7 Regress Write tests Test US1 & T Verify fixes impacted for US1 file bugs E Sprint tests S plan Write tests Test US2 & T for US2 Verify fixes file bugs
  126. 126. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:00 Lunch12:45 BREAK TILL: Bug workflow, infrastructure configuration for data diagnostic adapters13:3014:15 Test impact and recommended tests Break 14:3014:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents (extra time to finish the labs :)15:45 Test Environments in MTLM
  127. 127. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:00 Lunch12:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:15 Break14:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  128. 128. Create test from Bug
  129. 129. File an Exploratory BugSubmit an Exploratory Bug using Test Runner
  130. 130. PRO: - Reuse scenario knowledge business user - No typing detailed test steps - Easy to parameterize1 Record scenario by business user with - Writing standards for test steps MTM on application under test. Action - MTM capabilities in step 3. (bug, codedUI, recommended test etc …) recording is captured. Most benefit during step 2 (test case writing), benefit in step 1: run once in common situation. Benefit in step 3: common MTM benefits (record playback, fast forward, parameters etc) and benefit of well written steps.2 Create test steps (with parameters) in MTM by using the action recording. CON: - Manual actions and tests. - Test steps text are technical names - Action recording (step 1) must be clean. - Unknown what is validated, during step 2, need additional knowledge. - The business user must think of the important scenarios3 Save test case and execute it by a tester The action recording during step 1 can have on application under test, by using MTM. unnecessary steps, these can be removed during the execution (1) or during the specification (2).
  131. 131. Lab 9Create an exploratory bug and create a test case from it Timebox: 10 minuts
  132. 132. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:00 Lunch12:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:15 Break14:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  133. 133. Test InfrastructureWhere and how to run the automated test?
  134. 134. Test EnvironmentsSee also on www.TMap.net:- Checklist "Intake Test Environment“- Checklist "Test Environment“- Infrastructure Phase
  135. 135. Test labVisual Studio 2010 Lab Management
  136. 136. Where and how to run the automated test?physical test environmentsvirtualized test environments
  137. 137. physical test virtualized testenvironments environments MTLM Client MTLM Client Physical Test Client TFS 2010 System Center Virtual TFS 2010 Machine Manager (SCVMM) Hyper-V Library Physical Host Shares Test Client VM VM share share
  138. 138. Build application physical test environments versus virtualized test environments ProvisionRun tests ‘clean’ env Deploy application
  139. 139. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:00 Lunch12:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:15 Break14:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  140. 140. Lab for PhysicalEnvironments, thedifferent flavors…Visual Studio Team Foundation ServerTest Manager Build Server Test Agent Test Agent Test Agent
  141. 141. Configurations for automated test execution: AFlavor A: Execution from VS2010… Real developer test / test automation dry run (Triggered by: right mouse click – run tests, in test project) http://msdn.microsoft.com/en-us/library/dd286580.aspx No additional configuration needed* * Data Diagnostic Settings in the Local.Testsettings file are configurable. Pro: Con: Easy setup No collection of test results in TFS Debug-able test automation Run on developer environment
  142. 142. Configurations for automated test execution: BFlavor B: Execution during build with Build controller… not recommended, strange to run UI tests on build Part of Build Verification Tests (triggered by: build) Set the Build Service to run in interactive mode http://blogs.msdn.com/b/mathew_aniyan/archive/2009/05/26/coded-ui-test-in-a-team-build.aspx Configure the build to run the tests http://msdn.microsoft.com/en-us/library/ms182465.aspx#CreateBuildType * Data Diagnostic Settings in the *.Testsettings file are configurable. Pro: Con: Easy setup No collection of test results in TFS Test results in build result Build Controller needs to run in interactive mode Tests are executed on build environment Run on build environment
  143. 143. Configurations for automated test execution: CFlavor C: Execution during build with Test controller… B Preferred configuration above flavor Part of Build Verification Tests on multiple test agents (triggered by: build) Configure Test Controller (don’t register it with a project collection ) http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestControllers Configure Test Agents on clients (interactive mode) http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestAgents Configure *.Testsettings file in solution to use Test Controller and Test Agents Configure the build to run the tests (see B) Pro: Con: Test run on test environments No collection of test results in TFS Tests run on multiple environments Harder to configure Test Results in Build result Need for specific test client environments Test Settings from VS
  144. 144. Configurations for automated test execution: DFlavor D: Execution from Microsoft Test Manager…than BVT Other type of test Part of Regression Tests (triggered by: MTM user, right mouse click on test case, run) Configure Test Controller (register it with a project collection ) Configure Test Agents on clients (interactive mode, can be the same as MTM) Configure Lab Center in MTM to use test controller and create test ‘agent’ environment. http://msdn.microsoft.com/en-us/library/ee390842.aspx http://msdn.microsoft.com/en-us/library/dd293551.aspx Associate CodedUI test with WI Test Case from VS. http://www.richard-banks.org/2010/11/how-to-use-codedui-tests-watin-and-mtm.html Pro: Con: Test run on test environments Manually triggered by Tester Tests run on multiple environments Hard to configure Test Result in MTM and TFS Need for specific test client (same as MTM?) Test Settings from MTM
  145. 145. Configurations for automated test execution: EFlavor E: Execution from MTM duringPreferred configuration above flavor C Build… Part of BVT Flavor D and E can be configured together (triggered by: Build) Configure Test Controller (register it with a project collection ) Configure Test Agents on clients (interactive mode, can be the same as MTM) Configure Lab Center in MTM to use test controller and create test ‘agent’ environment. Associate CodedUI test with WI Test Case from VS. Create Build task to run TCM or MSTEST task for Test Plan http://blogs.microsoft.co.il/blogs/shair/archive/2010/10/30/how-to-run-coded-ui-tests-from-command-line.aspx Pro: Con: Test run on test environments Hard to configure Tests run on multiple environments Need for specific test client (same as MTM?) Test Result in MTM and TFS Triggered by build
  146. 146. Test Controller
  147. 147. Test Agent
  148. 148. Lab 10 Install and configure the test controller and test agenthttp://msdn.microsoft.com/en-us/library/dd648127.aspxhttp://www.microsoft.com/downloads/en/details.aspx?FamilyID=a3216d2a-0530-4f6c-a7c9-0df37c54a902 Timebox: 20 minuts
  149. 149. Start Topics9:00 Introduction, environments setup9:15 introduction by the students9:45 Application Lifecycle Management and Visual Studio General9:55 Test Planning10:15 Managing, controlling the testing effort, test plans, test suites and test configuration10:45 BREAK11:00 Specify test cases and corresponding capabilities of MTM11:45 Test execution, fast forward, reporting, olap cub12:00 Lunch12:45 Bug workflow, infrastructure configuration for data diagnostic adapters13:30 Test impact and recommended tests14:15 Break14:30 Exploratory testing and create test from bug14:45 Test Controllers and Agents15:45 Test Environments in MTLM
  150. 150. New physical environment
  151. 151. RECAPTest Activities Planning…
  152. 152. Test Case Management RECAP
  153. 153. RECAPTest Case Specification
  154. 154. RECAPTest Case Execution
  155. 155. Test Case Management RECAP
  156. 156. Start Topics DAY 29:00 Recap9:15 Create Basic CodedUI, from Test Case and Manual9:45 UIMap and customize the code for optimization10:15 Data driven Tests and assertions10:45 break11:00 Troubleshoot CodedUI, Common Practices and questions11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute from VS2010 with test agents and analyze the results11:30 Associate test automation with test case, execute automated test case from MTM, configure the build to run from the build within MTM and analyze the results12:30 break13:15 MTLM, Scrum and other project Methodologies13:45 Lab Management - Virtualized Test Environment and execution Demo14:15 Test info in the OLAP Cube14:30 Extensibility - test Scribe, TCMIMport14:45 Create your own Data Diagnostic Adapter15:15 MTLM usages with Azure projects16:00 Closing
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×