1. QA with Microsoft Test
Manager and Lab
Management
Sunil Kumar Singh, Technology Specialist
Microsoft India

2. Self discoverable Bring your own deviceEdge facing
Modern business applications
Continuous services

3. Modern app lifecycle
Continuous value delivery with modern business applications
Define
Ideation
Develop
Idea to working
software
MonitorImplement
SHORTENED
CYCLE TIMES

4. Conventional QA in the Modern app lifecycle
QA
Testingpost
implementation. Mostly
manualwithlimited
automation.
UAT
UATpost
implementation
andsystems
testing
Operations
readiness
verification
Pre-deployment
verification
Documentrequirements
asBRDsand
functionalspecs
Writecodeto
implement
requirements

5. Quality
Enablement

6. Quality enablement practices
Continuous quality for continuous value delivery
Continuous quality
Shortened cycle times

7. Acceptance test planning
Business goals
Actionable
requirements
Acceptance
criteria
Acceptance test
plan
Acceptance test
driven
development

8. Continuous acceptance testing
User
Interface
Services
Business Processes
Business Rules and Logic
Service integrations
Data access
Identity
Data
Automatedtesting
Manualtesting
Automatedtesting

9. Lab management automation

10. Quality Enablement Solution Architecture
Work item tracking Test lab managementBuild automation
Extensibility interfaces
SCM
Test execution
services
Deployment
automation
Reporting and BI
Lab infrastructure
Test recorders and
runners
Build engines
• Actionable feedback
• Exploratory testing
• Requirements
management
• Acceptance criteria
definition
• Developer testing
• Build automation
• Actionable
diagnostics
• Test planning and
management
• Test execution and
defect reporting
• Lab management
• Production monitoring
• Integrated incident
management
Quality metrics and
reporting
Role tailored tools

11. Microsoft
Solution

12. The Microsoft Solution
Traceability with work item tracking
Requirements
management
Test lab
management
Source code
management
Extensibility interfaces
Team
development
services
Testing services
Build and
deployment
automation
Reporting and
Business
Intelligence
Lab infrastructure Test recorders and
runners
Build engines
• Microsoft Office
• Feedback client
• Microsoft Office and
SharePoint
• Microsoft Project
• Browser apps
• Visual Studio IDE
• Eclipse
• Cross-platform
command line tools
• Microsoft Test
Manger
• Microsoft Visual
Studio Premium or
Ultimate
Microsoft System Center Microsoft Office and
SharePoint
Project
planning and
management
Git

13. Quality Enablement Solution Architecture
Work item tracking Test lab managementBuild automation
Extensibility interfaces
SCM
Test execution
services
Deployment
automation
Reporting and BI
Lab infrastructure
Test recorders and
runners
Build engines
• Actionable feedback
• Exploratory testing
• Requirements
management
• Acceptance criteria
definition
• Developer testing
• Build automation
• Actionable
diagnostics
• Test planning and
management
• Test execution and
defect reporting
• Lab management
• Production monitoring
• Integrated incident
management
Quality metrics and
reporting
Role tailored tools

14. Manual Testing

15. Software testing with Visual Studio

16. Business
challenges
Long, opaque manual testing
cycles
“No-repro” defect reports
Repetitive and error prone
Unable to track tested vs.
untested code
Solutions
Actionable bugs containing
data to reproduce defects
Code coverage reports show
untested features
Automated fast-forward
Connect dev and test with
tools to instrument and repro
Business value
Reduce dev-test “ping pong”
to cut waste
Speed delivery of high
quality software
Delivered software meets
user and value expectations

17. Manual Testing

18. Demo
Manual Testing

19. Organize testing in multiple
configurations
Playback a prior recording to save
time with frequently-repeated setup
steps and prerequisites
Bind data to controls, to
support multiple iterations
and future automation
Pause, save, close, return, and
pick up where you left off

20. IntelliTrace Logs
System logs & configuration
Detailed status and history visible
to all team members
Audio & Video recordings

21. Get an overview of the test run
with summary pie charts, or drill
into the details for each test.
Analyze code coverage for manual
tests to determine which specific
parts of your code were tested .
Associate manual test runs with
builds and environments.

22. Exploratory
Testing

23. Software testing with Visual Studio

24. Business
challenges
Prescribed test cases can’t
anticipate all possible bug
scenarios
Ad-hoc testing often results
in “no-repro” defects
Unable to validate original
bug has been fixed
Solutions
Leverage inherent tester
creativity with ad-hoc testing
Record ad-hoc test steps
Create test cases from bugs
to reproduce complex
scenarios or to verify bug
fixes later
Business value
Discover unknown scenarios
and hidden defects
Eliminate manual capturing
of test steps to cut waste
Increase quality in delivered
software

25. Exploratory Testing

26. Demo
Exploratory Testing

27. Create rich text comments
with screenshots while
exploring
Exploratory sessions can be
recorded to increase
reproducibility
Explore your application to
uncover new defects
The Test Runner docks on the left side
of the screen and automatically records
your steps as you explore

28. Comments and steps are
automatically added
The exploratory test session is
paused while you create a bug
report
File new bugs without ending the
exploratory test session
Attach the video recorded during
the exploratory test session for
maximum reproducibility
Quickly create an
automatically linked test
case for this bug

29. Add verification values to indicate
the expected result
Session is paused while creating test
cases
Test steps are added
automatically

30. Comment block reflects the
added bugs and tests
Continue exploratory testing right
from where you left off
Quick access to bugs and test cases
created during the session

31. Lab
Management

32. Software testing with Visual Studio

33. Business
challenges
Modern apps run in diverse
environments
Difficult to repro issues found
Replicating environments for
test is time-consuming, costly
Solutions
Virtualization reduces cost to
provision environments
Templates allow a wide range
of baselines to be spun up
quickly and on demand
Snapshots capture states for
reproducing test conditions
and verifying defects
Business value
Dramatically reduced
hardware and maintenance
expenses
Improved quality and
reliability of apps across
platforms
Operations engaged in
earlier phases of testing

34. Lab management automation
On-premise| Cloud

35. Demo
Lab Management

36. Create a new environment using
virtual machine templates in
SCVMM
Create an environment
using machines running on
SCVMM
Create new environments
on existing machines

37. Quickly choose multiple test
suites to run after
deployment
Choose your environment, and build,
deploy and test in that environment
from a single dialog

38. Snapshot environments during
manual testing to increase
reproducibility
Spin up virtual test environments from
within Test Manager, or attach a virtual
environment to a manual testing session

39. Test run results are attached to
specific builds and environments
Collect all the data required to
recreate bugs from virtual
environments

40. The Microsoft differentiation
Role tailored tools Hybrid IT capableContinuous quality practices
Application
Lifecycle
Continuous value delivery
New value

41. Call to Action
• Take Microsoft Virtual Academy (MVA) courses
• http://aka.ms/mvasoftwaretesting
• http://aka.ms/mvaswtestingvs
• Use TechNet Virtual Lab (http://aka.ms/virt-labs)
• http://aka.ms/VLTestPlanning
• http://aka.ms/VLTestCase
• http://aka.ms/VLExploratoryTest
• Use Azure Trial (http://azure.microsoft.com/en-
us/pricing/free-trial/)
• Download Visual Studio ‘14 CTP
• http://aka.ms/vsctp14