Testing with Visual Studio 2012 Agile TMap

1. Module 05
Test Infrastructure

2. Topics

3. The setup and maintenance infrastructure phase aims to provide the required test infrastructure and resources. A
distinction is made between test environments, test tools and workplaces. Setting up and maintaining the
infrastructure requires very specific expertise. Testers generally have limited knowledge in this area, but are dependent
on it. No test can be executed without an infrastructure.
http://www.tmap.net/en/tmap/4-essentials/structured-test-process/acceptance-and-system/infrastructure-phase
The Infrastructure Phase

4. Test Types
Performance &
Load Tests
Security Tests
‘ility’ Tests
Automated
and Manual
Manual
Automated
Tools
Automated

5. Build & Deploy Infrastructure
Functional Test Environment
Load Test Rig
Acceptance Test Environment
Possible Test Environments

6. Continuous Acceptance Testing

7. Continuous integration
automatically creates a
build after check-in
Create multiple build definitions
to support various build scenarios

8. Use build check-in policies to
ensure broken code is fixed
before being checked in
Use either continuous integration builds
or rolling builds with build check-in
policies to ensure quality builds

9. Configure the gated check-in
build to fail if a test fails, ensuring
the code won’t be checked in
Identify specific test to be run
during the gated check-in build

10. Configuring
for Unit Testing

11. Unit Tests, Component Tests
Build Controller and
Build Agent Infrastructure
Is used often and needs
to be flexible and fast
Development Test Infrastructure

12. Use it to retrieve the necessary source code for a build, compile the solution and (if required) execute unit tests and static code analysis
tools as part of the build process.
Build Controllers
1-1 with Team Project Collection OR 1-* with build agents
Perform lightweight tasks.
Can easily be virtualized.
Build Agents
1-1 with build controller.
Performs heavyweight build tasks.
Manage files in source control.
Run tests
Compile code.
Allow load-balanced and/or parallel builds.
Can be tagged to spread workload.
Build Infrastructure

13. Lists builds and provides information about test results, test coverage, code churn and quality notes for each build.
Build Summary Report

14. Demo
Configuring the Build
Infrastructure for Unit Testing

15. Configuring
for Functional Testing

16. “Working Systems”
Manual and automated
test execution
Planning and Management
of test cases and test plans
Functional Test Infrastructure

17. Virtual Environments
Lab Management
Working Systems

18. Use it to retrieve necessary source code for a build, compile the solution and (if required) execute unit tests and static code analysis tools
as part of the build process.
Test Controllers
1-1 with Team Project Collection | 1-* with test agents.
Perform lightweight tasks.
Can easily be virtualized.
Test Agents
1-1 with test controller.
Performs heavyweight test tasks.
Run tests Interactive.
Collect Data Diagnostic Information.
Build Infrastructure

19. Configuring
for Acceptance Testing

20. Real Life Scenarios
“Working systems”
Not always connected to
the rest of the team
Acceptance Tests

21. Feedback Manager
Exploratory Testing
Alpha/Beta Testing
Lab Management
Testing Working Systems

22. Test Virtualization

23. Execute tests in Virtual Machines
Link 1-* VMs with Test Agents to execute tests in different environments.
Low Effort – easy to set up
Install Test Agent on VM.
Sync to TFS.
Set Up Build Configuration to deploy tests to VM automatically.
Virtualized Test Environments

24. Execute tests in Virtual Test Environments
A virtual environment is a collection of virtual machines (VMs). Each Virtual Machine in a virtual environment represents a role required for
the application that is to be tested, developed or run. Lab Management can be used to start all the virtual machines in a virtual
environment to run an application, or test an application.
Automated environment set up
Lab Management uses System Center Virtual Machine Manager (SCVMM) to
allow access to virtual machines or templates in a library as “golden masters”.
These golden masters are created by using either Hyper-V or SCVMM.
SCVMM is used to deploy the virtual machines and templates to
the environments on the specified host group.
Lab Management

25. Lab Management Automation
On-premise | Cloud

26. Create a new environment
using virtual machine
templates in SCVMM
Create an environment
using machines running
on SCVMM
Create new
environments on existing
machines

27. Quickly choose multiple
test suites to run after
deployment
Choose your environment, and
build, deploy and test in that
environment from a single dialog

28. Snapshot environments
during manual testing to
increase reproducibility
Spin up virtual test environments from
within Test Manager, or attach a virtual
environment to a manual testing session

29. Test run results are attached to
specific builds and environments
Collect all the data required
to recreate bugs from virtual
environments

30. Demo
End-to-End Test Workflow
Environment Virtualization

31. Configuring
for Load Testing

32. Load Test Clients create
variable load on the target
Load Tests produce lots
of data and information!
Load testing expertise is
required to interpret results
Load Test Infrastructure

33. Visual Studio Ultimate can use a computer or a group of computers to generate simulated load for testing, and to run tests remotely and
concurrently on several computers. This group of computers consists of a single test controller and one or more test agents. You can use
the test controller and test agent to generate more load than a single computer can generate alone or just test remotely.
Load Test Agents

34. The Cloud can be used for Load Testing – unlimited Load can be generated this way without associated hardware investments.
Agents on Windows Azure

35. Demo
Load Testing using
Windows Azure