MTLM Visual Studio 2010 ALM - day2

DAY 2MTLM

Start Topics9:00 Recap9:15 Create Basic CodedUI, from Test Case and Manual9:45 UIMap and customize the code for optimization10:15 Data driven Tests and assertions10:45 break11:00 Troubleshoot CodedUI, Common Practices and questions11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute from VS2010 with test agents and analyze the results11:30 Associate test automation with test case, execute automated test case from MTM, configure the build to run from the build within MTM and analyze the results12:30 break13:15 MTLM, Scrum and other project Methodologies13:45 Lab Management - Virtualized Test Environment and execution Demo14:15 Test info in the OLAP Cube14:30 Extensibility - test Scribe, TCMIMport14:45 Create your own Data Diagnostic Adapter15:15 MTLM usages with Azure projects16:00 Closing

RECAPTest Activities Planning…

Test Case Management RECAP

RECAPTest Case Specification

RECAPTest Case Execution

Test Environments RECAP

"by special request"

1Open the employee applicationSet the intellitrace optionsSet a breakpoint at …

2When on the breakpoint, save the trace file on your desktop

3Close Visual Studio and double click the intellitrace file

4In the intellitrace summary, open the main threat …

5 … and search for The values that where entered during the previous run.

6And now from Microsoft Test Manager

6Set the diagnostic adapter

7Configure the build… it needs a drop location for the symbols (pdb files)

.Make test case with the parameter 1 and 1, crash again…See lab 7 yesterday

8Run the test case and see it crash… file a bug.

9See the itrace file in the bug report...

10Close MTM, open VS (with or without the employee application).Got to the bug and click the itrace file…See the exception andstart debugging…

11Go back in time

11Go back in time … and find the entered values

Regression Test CandidatesSee also on www.tmap.net: Completion Phase

Test Automation Test automation Where? When? How? Who? Why? What?

Where? When? How? Who? Why? What?

VS2010 Test ToolsUI Test Automation Capabilities Fast Forward CodedUI Test Infrastructure

Test Automation Investment Levels 0 No investment in any kind of automation. Just click and test. 1 Clean up of the Action log maybe re-execute, for more smooth Fast Forward. 2 Basic CodedUI. Only use the default generation, add your own assertion. 3 Advanced CodedUI. Customize the CodedUI and UIMap for optimization.

1 Clean up of the Action

2 Basic CodedUI VS2010 ALM MTLM usages patterns for Windows Azure hosted application development

3 Advanced CodedUI

Fast Forward

Cleaning the action recordingWhen: executing the same test case onmultiple environments or when using test dataiteration or … and don’t forget the sharedsteps

CodedUI

CodedUIfrom existing action recording from Manual Tester

CodedUIfrom existing action recording from Manual Tester 2 1 4 3

CodedUICreate a new action recording by Developer

Lab 12Create a CodedUI from Test CaseCreate a CodedUI by using the CodedUI Builder Timebox: 10 minuts

Lab 13Tune the CodedUI with the UIMap… Timebox: 10 minuts

Data-DrivenCoded UI Test

Lab 14Make CodedUI parameterized from an Excel sheet Timebox: 10 minuts

Start Topics9:00 Recap9:15 Create Basic CodedUI, from Test Case and Manual9:45 UIMap and customize the code for optimization10:15 Data driven Tests and assertions10:45 break11:00 Troubleshoot CodedUI, Common Practices and questions11:10 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute from VS2010 with test agents and analyze the results11:30 Associate test automation with test case, execute automated test case from MTM, configure the build to run from the build within MTM and analyze the results12:30 break BREAK TILL:13:15 MTLM, Scrum and other project Methodologies13:45 11:00 Lab Management - Virtualized Test Environment and execution Demo14:15 Test info in the OLAP Cube14:30 Extensibility - test Scribe, TCMIMport (extra time to finish the labs :)14:45 Create your own Data Diagnostic Adapter15:15 MTLM usages with Azure projects16:00 Closing

CodedUI Common Questions • In what version what functionality? • How to debug? • Where can I find test run results for a test run? • Where do I see these assertions are passed or failed? • For which properties can I make an assertion? • …

CodedUI Advanced Questions • How does CodedUI work? • Can I use CodedUI for performance testing? • How to maintain CodedUI tests? • How to execute in the build? • …

CodedUI Challenges • The playback failed to find the control with the given search properties. • Identifying controls with no name search criteria or changing ID’s • Selecting correct Search Configurations for a control • How to select the particular value from combo box • …find controls

Start Topics9:00 Recap9:15 Create Basic CodedUI, from Test Case and Manual9:45 UIMap and customize the code for optimization10:15 Data driven Tests and assertions10:45 break11:00 Troubleshoot CodedUI, Common Practices and questions11:10 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute from VS2010 with test agents and analyze the results11:30 Associate test automation with test case, execute automated test case from MTM, configure the build to run from the build within MTM and analyze the results12:30 Lunch13:15 MTLM, Scrum and other project Methodologies13:45 Lab Management - Virtualized Test Environment and execution Demo14:15 Test info in the OLAP Cube14:30 Extensibility - test Scribe, TCMIMport14:45 Create your own Data Diagnostic Adapter15:15 MTLM usages with Azure projects16:00 Closing

Execute CodedUIDuring the build with Visual Studio 2010 For the next 1,5 hour, you must have: - A successful running codedui - Test agent and controller - A build - Testplan and a test case

Configurations for automated test execution: CFlavor C: Execution during build with Test controller… B Preferred configuration above flavor Part of Build Verification Tests on multiple test agents (triggered by: build) Configure Test Controller (don’t register it with a project collection ) http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestControllers Configure Test Agents on clients (interactive mode) http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestAgents Configure *.Testsettings file in solution to use Test Controller and Test Agents Configure the build to run the tests (see B) Pro: Con: Test run on test environments No collection of test results in TFS Tests run on multiple environments Harder to configure Test Results in Build result Need for specific test client environments Test Settings from VS

1Configure Test Controller.

2Add new Test Settings fileSet remote execution andtest controller.

3Create a new build definitionSet the Automated Test Properties

4Queue a new BuildWatch the Test Agent andTest Environment

Lab 15Run the automated test from the build.Look for the test results Timebox: 10 minuts

Execute CodedUIWithin a test case as an associated automation manualy

Configurations for automated test execution: DFlavor D: Execution from Microsoft Test Manager…than BVT Other type of test Part of Regression Tests (triggered by: MTM user, right mouse click on test case, run) Configure Test Controller (register it with a project collection ) Configure Test Agents on clients (interactive mode, can be the same as MTM) Configure Lab Center in MTM to use test controller and create test ‘agent’ environment. http://msdn.microsoft.com/en-us/library/ee390842.aspx http://msdn.microsoft.com/en-us/library/dd293551.aspx Associate CodedUI test with WI Test Case from VS. http://www.richard-banks.org/2010/11/how-to-use-codedui-tests-watin-and-mtm.html Pro: Con: Test run on test environments Manually triggered by Tester Tests run on multiple environments Hard to configure Test Result in MTM and TFS Need for specific test client (same as MTM?) Test Settings from MTM

1Configure Test Controller.

2Associate the CodedUIwith a Test Case.

2Create Test Environment

3Set test Settings

4Associate the CodedUIwith a Test Case.

5Run Automated Tests from Test Manager

Lab 16Run the automated test from MTM.Look for the test results Timebox: 10 minuts

Execute CodedUIWithin a test case as an associated automation from the build

Configurations for automated test execution: EFlavor E: Execution from MTM duringPreferred configuration above flavor C Build… Part of BVT Flavor D and E can be configured together (triggered by: Build) Configure Test Controller (register it with a project collection ) Configure Test Agents on clients (interactive mode, can be the same as MTM) Configure Lab Center in MTM to use test controller and create test ‘agent’ environment. Associate CodedUI test with WI Test Case from VS. Create Build task to run MTC or MSTEST task for Test Plan http://blogs.microsoft.co.il/blogs/shair/archive/2010/10/30/how-to-run-coded-ui-tests-from-command-line.aspx Pro: Con: Test run on test environments Hard to configure Tests run on multiple environments Need for specific test client (same as MTM?) Test Result in MTM and TFS Triggered by build Test Settings from MTM

Execute automated test cases ondifferent environments

0Same steps as previous lab.

1Configure additional environments in Test Plan properties. Win 7 IE9 configuration Win 7 IE8 configuration TC MTLM Client Win 7 IE8 configuration Physical Test Client TFS 2010 Win 7 IE9 configuration Physical Test Client

1Get Test Plan IDGet Test SuiteGet Test Configurations with TCM.exe

2Create command line for Test Run for the different Test Configurations "run /create /title:""Run on Host for specific Test Point"" /planid:16 /querytext:""SELECT * FROM TestPoint WHERE ConfigurationId=13"" /settingsname:""Automated Test on Windows 7 Environment IE9"" /testenvironment:""Win 7 IE9"" /collection:http://clemenes- pc:8080/tfs/defaultcollection /teamproject:EmployeeApplication" "run /create /title:""Run on test env for specific Test Point"" /planid:16 /querytext:""SELECT * FROM TestPoint WHERE ConfigurationId=12"" /settingsname:""Automated Test on Windows 7 Environment IE8"" /testenvironment:""Win 7 IE8"" /collection:http://clemenes- pc:8080/tfs/defaultcollection /teamproject:EmployeeApplication"

3Add build commands

6Queue a new BuildWatch the Test Agent andTest Environment

Start Topics9:00 Recap9:15 Create Basic CodedUI, from Test Case and Manual9:45 UIMap and customize the code for optimization10:15 Data driven Tests and assertions10:45 break11:00 Troubleshoot CodedUI, Common Practices and questions11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute from VS2010 with test agents and analyze the results11:30 Associate test automation with test case, execute automated test case from MTM, configure the build to run from the build within MTM and analyze the results12:30 Lunch LUNCH TILL:13:15 MTLM, Scrum and other project Methodologies13:45 13:15 Lab Management - Virtualized Test Environment and execution Demo14:15 Test info in the OLAP Cube14:30 Extensibility - test Scribe, TCMIMport (extra time to finish the labs :)14:45 Create your own Data Diagnostic Adapter15:15 MTLM usages with Azure projects16:00 Closing

VS2010 Process Support

TMap, ALM and process templates

# Planning the project

# Release planning meeting Estimating and prioritizing the Product Backlog # Product Backlog Evolving User Needs Owned by Product Owner Prio by Business ValueTest during the Release Planning meeting.

# Planning the project

# Planning the sprint

‘only execute tasks who make an addition tothe product are worthy to execute’

# Sprint planning meeting Add PBI to sprint Decomposition Commitment # Sprint Backlog Tasks Related to PBI’s Hours remainingTest during the Sprint Planning meeting

# Planning the sprint

‘only execute tasks who make an addition tothe product are worthy to execute’

# Running the sprint

# Daily scrum Track# Sprint Review # Sprint Demo the value Retrospective Agile Practices – Improvement Test early and often Build and deploy continuously Acceptance Test Driven Development Emergent Architectures Refactor Test Driven Development Agile Database Development Pair Programming

Undone End to End Testinghttp://www.scrum.org/scrumguides/

Build application ProvisionRun tests ‘clean’ env Deploy application

VS2010 Lab Management• Reproduce the exact conditions of a bug or other development issue.• Build, deploy, and test applications automatically in a clean environment.• Reduce the time required to create and configure machines for testing an application.• Run multiple copies of a test or development at the same time.• Enable members of a team to create and manage virtual environments without requiring system administrator privileges.

MTLM ClientTFS 2010 System Center Virtual Machine Manager (SCVMM) Hyper-V Library Host Shares VM share VM share

VMLab Agent

TFS 2010

TEAM FOUNDATION SERVER INTERNET LAB INFRASTRUCTURE Prerequisites ? SAN Domain Controller 1. TFS Application Tier (AT) • Windows Server 2003 SP2, 2003 R2, 2003 R2, 2008 SP2, 2008 R2 • Windows Vista SP2, 7 • Ports used by services 8443 team web access 8080 server Internet LAB DOMAIN (could be same as CORP DOMAIN) 2. TFS Data Tier (DT) • Windows Server 2003 SP2, 2003 R2, 2003 R2, 2008 SP2, 2008 R2 NLB External Users • Windows Vista SP2, 7 • Ports used by services 1433 SQL Walkthrough VHD Lab Management 3. TFS Build Controller • Windows Server 2003 SP2, 2003 R2, 2003 R2 SP2, 2008 SP2, 2008 R2 • Windows XP SP3, Vista SP2, 7 TFS 1 VS/MTM Clients • Ports used by services 9191 controller Application Tier (AT) 4 4. TFS Clients • Windows XP SP3, Vista SP2, 7 • Windows Server 2003 SP2, 2003 R2, 2008 SP2, 2008 R2 VMM Server 5. VM Factory 6 • Windows Server 2003 SP2, 2003 R2, 2008 SP2, 2008 R2 • Windows XP SP2, Vista, 7 • Ports used by services 3389 RDP 137 UDP 9 139 TCP VMM Library • For more information, refer to the quick reference poster: 8 Visual Studio 2010 LAB Management Guide – VM Factory Checklist 3 Test Controller 6. VMM Server • Windows Server 2008 or 2008 R2. R2 is recommended. • Ports used by services 8100 VMM Agent TFS 135 WMI (49152-65535) Build Create LabMan template 3389 RDP Servers from golden image 7. VMM Hyper-V Host • Windows Server 2008, 2008R2. R2 is recommended. Build VS/MTM 8. Test Controller Deploy and • Windows Server 2003 SP2, 2003 R2, 2003 R2 SP2, 2008 SP2, 2008 R2 Client Build Golden Images • Windows XP SP3, Vista SP2, 7 Machines Host VMs • Ports used by services 6901 .NET Remoting 9050 Lab Agent 9. VMM Library Primary Factory • Windows Server 2008 or 2008 R2. R2 is recommended. 2 TFS 5 • Ports used by services 443 BITS Data Tier (DT) 80 WinRm VMM Hyper-V Host 7 Secondary Factory VM FACTORIES LAB MANAGEMENT References • Visual Studio 2010 Lab Management Guide CORP DOMAIN • http://go.microsoft.com/fwlink/?LinkID=206935&clcid=0x409 • Visual Studio ALM Rangers VM Factory Guidance • http://rangersvsvmfactory.codeplex.com/ Domain Controller • Capacity Planning Quick Reference Poster Network Storage Team Project Collection Planning Quick Reference PosterCORP INFRASTRUCTURE • http://vs2010quickref.codeplex.com/ Visual Studio 2010 LAB Management Guide – Infrastructure Visual Studio ALM Rangers 2011-03-23

Visual Studio 2010 Test ArchitectureAlso known as user The final integrationacceptance testing, testing the solution.the validation testing In the world of NASA  Test Casesvalidates the solution System this could be the  Testingagainst test final countdown,requirements, which doing a completeare based on business system and Common Test Phasesrequirements. environment test  Test Validation before the launch. results Design validation  Cannot reproduce focused testing, which validates Building upon each Integration solution deliverables other and with requirements. It supporting the is especially  Suspicions, testing strategy. important when add more tests testing interfaces which are shared and are the basis of some or all of theUnit of work focused Unit building blocks.  Corrections  Debuggingtesting, at the core of thesolution, typically done bythe developer andautomated testing. Unit No more “No Repro”!tests focus on topics suchas initialization, logic anderror handling at a codelevel. Visual Studio Products Test Lab Ultimate Premium Test Load Agent Professional Management Included in Ultimate Manage virtual environments for • Generalist Tester use in application development, • Minimal Footprint deployment, and testing.References Visual Studio ALM Rangers, Copyright © 2009 Microsoft CorporationVisual Studio MSDN Site http://msdn.microsoft.com Visual Studio ALM Rangers http://msdn.microsoft.com/en-us/teamsystem/ee358786.aspx

Start Topics9:00 Recap9:159:45 BREAK TILL: Create Basic CodedUI, from Test Case and Manual UIMap and customize the code for optimization10:15 Data driven Tests and assertions 14:3010:45 break11:00 Troubleshoot CodedUI, Common Practices and questions11:30 Configure the build to execute the CodedUI(extra time settings fromlabs :) Execute tests, test to finish the VS2010, from VS2010 with test agents and analyze the results11:30 Associate test automation with test case, execute automated test case from MTM, configure the build to run from the build within MTM and analyze the results12:30 Lunch13:15 MTLM, Scrum and other project Methodologies13:45 Lab Management - Virtualized Test Environment and execution Demo14:15 Test info in the OLAP Cube Break14:30 Test Scribe, TCMImport14:45 Create your own Data Diagnostic Adapter15:15 MTLM usages with Azure projects16:00 Closing

Lab 17Install Test Scribe and create a report Timebox: 10 minuts

Start Topics9:00 Recap9:15 Create Basic CodedUI, from Test Case and Manual9:45 UIMap and customize the code for optimization10:15 Data driven Tests and assertions10:45 break11:00 Troubleshoot CodedUI, Common Practices and questions11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute from VS2010 with test agents and analyze the results11:30 Associate test automation with test case, execute automated test case from MTM, configure the build to run from the build within MTM and analyze the results12:30 Lunch13:15 MTLM, Scrum and other project Methodologies13:45 Lab Management - Virtualized Test Environment and execution Demo14:15 Test info in the OLAP Cube Break14:30 Test Scribe, TCMImport14:45 Create your own Data Diagnostic Adapter15:15 MTLM usages with Azure projects16:00 Closing

ALM 4 AzureAutomated ALM for Azure Cloud Development 127

Start Topics9:00 Recap9:15 Create Basic CodedUI, from Test Case and Manual9:45 UIMap and customize the code for optimization10:15 Data driven Tests and assertions10:45 break11:00 Troubleshoot CodedUI, Common Practices and questions11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute from VS2010 with test agents and analyze the results11:30 Associate test automation with test case, execute automated test case from MTM, configure the build to run from the build within MTM and analyze the results12:30 Lunch13:15 MTLM, Scrum and other project Methodologies13:45 Lab Management - Virtualized Test Environment and execution Demo14:15 Test info in the OLAP Cube Extensibility - test Scribe, TCMIMport14:30 break14:45 Create your own Data Diagnostic Adapter15:15 MTLM usages with Azure projects16:00 Closing

RECAPTest Activities Planning…

RECAPTest Case Specification

RECAPTest Case Execution

Test Environments RECAP

Testing Practices with ALM toolsGOAL:Know the value of MTM and know how to use its capabilities

staat voor resultaat

http://www.clemensreijnen.nl	1358
http://clemensreijnen.nl	124
http://feeds.feedburner.com	47
http://paper.li	3
http://us-w1.rockmelt.com	1
http://cache.baidu.com	1

MTLM Visual Studio 2010 ALM - day2

by Clemens Reijnen on Sep 08, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

6 Embeds 1,534

Statistics

MTLM Visual Studio 2010 ALM - day2 — Presentation Transcript