Testing, testing
A first-look at the new testing
capabilities in Visual Studio 2010
Mathias Olausson
Callista Knowledgebas...
About Mathias...

QWise software engineering – refactored!

www.qwise.se
About the Presentation...
• What’s on the menu?
– Test planning and execution
– Issue tracking with rich bugs
– Eliminatin...
TEST PLANNING AND EXECUTION

QWise software engineering – refactored!

www.qwise.se
Agile Testing

QWise software engineering – refactored!

www.qwise.se
Core Practices

QWise software engineering – refactored!

www.qwise.se
Agile Testing Quadrants
Business-facing
Manual

Functional Tests

Exploratory Tests

Unit Tests
Component Tests

Performan...
Test Types
Manual Tests
UI Tests

Acceptance Tests

Unit Tests / Component Tests

QWise software engineering – refactored!...
Testing in an Iteration
sprint
sprint
plan
plan

build 11
build

sprint
sprint
plan
plan

implement
implement
user story ‘...
Microsoft Test Manager and Visual Studio
Testing (Teams)

Development (Teams)
Engineering Support
Unit Testing

Specialist...
The Fundamentals

QWise software engineering – refactored!

www.qwise.se
Defining and Running Tests

QWise software engineering – refactored!

www.qwise.se
ISSUE TRACKING WITH RICH BUGS

QWise software engineering – refactored!

www.qwise.se
Should We Do Bug Tracking?
Pros
– Tracability
– Priority of work
– Visibility

Cons
– Overhead

Should we log bugs within ...
Defect Tracking in MTM and MTR

QWise software engineering – refactored!

www.qwise.se
No-more-no-repro
• Automatic data
collection
• Test settings
– Configuration of
diagnostic data
collectors
– Assigned to t...
Diagnostic Data Adapters

QWise software engineering – refactored!

www.qwise.se
No-more-no-repro

QWise software engineering – refactored!

www.qwise.se
AUTOMATED UI TESTING

QWise software engineering – refactored!

www.qwise.se
Why Automated Testing?

Källa: Tools for Agility, Kent Beck

QWise software engineering – refactored!

www.qwise.se
Automation Strategy
iteration 11
iteration
feature aa
feature

feature bb
feature

iteration 22
iteration
feature c c
feat...
Coded UI Tests
• Unit test for UI
• High-level framework
• Create test from
– Recording
– Test result (action
log)
– Using...
Automation Platform Support

QWise software engineering – refactored!

www.qwise.se
Coded UI Tests

QWise software engineering – refactored!

www.qwise.se
LAB MANAGEMENT

QWise software engineering – refactored!

www.qwise.se
Motivation

QWise software engineering – refactored!

www.qwise.se
Lab Management Aims To...

QWise software engineering – refactored!

www.qwise.se
Lab Management Infrastructure
Team
Foundation
Server (TFS)
Lab Management

System Center Virtual
Machine Manager (SCVMM)

...
Environments
• An environment is a set of machines with
different roles
• An environment can
– Be deployed, saved, removed...
Manual Tests in Lab
• Testing from the outside
– Easy for web applications
– Limitations in data collection

• Testing fro...
Automatic Tests in Lab
• Testing in VM
– Test case connected to test environment
– Test component comes from selected buil...
Lab Management Build Workflow

QWise software engineering – refactored!

www.qwise.se
Build and Deploy

QWise software engineering – refactored!

www.qwise.se
Summary
•
•
•
•
•

Structured test planning with MTM
Integrated test execution
Effective defect tracking with rich bugs
Se...
Questions?

mathias.olausson@callista.se
http://msmvps.com/blogs/molausson

QWise software engineering – refactored!

www....
Upcoming SlideShare
Loading in …5
×

Testing, mathias olausson

472 views

Published on

testing

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
472
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Annan bild + kort bio
  • Källa: Brian Marick
  • Agile utveckling ställer nya krav på testning. I och med att vi vill jobba iterativt och inkrementellt fungerar inte den traditionellt vattenfallsorienterade modellen och det är risk för flaskhalsar i testning. Automatisering är en möjliggörar här.
  • Testing, mathias olausson

    1. 1. Testing, testing A first-look at the new testing capabilities in Visual Studio 2010 Mathias Olausson Callista Knowledgebase | QWise QWise software engineering – refactored! www.qwise.se
    2. 2. About Mathias... QWise software engineering – refactored! www.qwise.se
    3. 3. About the Presentation... • What’s on the menu? – Test planning and execution – Issue tracking with rich bugs – Eliminating the "works on my machine / cannot reproduce" scenario – Automated UI Testing – Virtual Lab Management QWise software engineering – refactored! www.qwise.se
    4. 4. TEST PLANNING AND EXECUTION QWise software engineering – refactored! www.qwise.se
    5. 5. Agile Testing QWise software engineering – refactored! www.qwise.se
    6. 6. Core Practices QWise software engineering – refactored! www.qwise.se
    7. 7. Agile Testing Quadrants Business-facing Manual Functional Tests Exploratory Tests Unit Tests Component Tests Performance Tests Load Tests Security Tests ”-ilitets” Tests Critisizing the product Supporting the team Automated & Manual Automated Automated Technology-facing QWise software engineering – refactored! www.qwise.se
    8. 8. Test Types Manual Tests UI Tests Acceptance Tests Unit Tests / Component Tests QWise software engineering – refactored! www.qwise.se
    9. 9. Testing in an Iteration sprint sprint plan plan build 11 build sprint sprint plan plan implement implement user story ‘1’ user story ‘1’ (us1) (us1) build 22 build implement Implement Implement us2 US2 US2 fix bugs fix bugs build 33 build write tests write tests for us1 for us1 fix bugs fix bugs build 44 build test us1 & test us1 & file bugs file bugs write tests write tests for us2 for us2 QWise software engineering – refactored! build 55 build build 66 build verify fixes verify fixes test us2 & test us2 & file bugs file bugs build 77 build regress regress impacted impacted tests tests verify fixes verify fixes www.qwise.se
    10. 10. Microsoft Test Manager and Visual Studio Testing (Teams) Development (Teams) Engineering Support Unit Testing Specialist Generalist Coded UI Test Microsoft Test Runner Web Test Fast Forward For Manual Testing Load Test Test Case Management Lab Management Data Collectors (IntelliTrace, Test Impact, Metrics) Team Foundation Server QWise software engineering – refactored! Reporting www.qwise.se
    11. 11. The Fundamentals QWise software engineering – refactored! www.qwise.se
    12. 12. Defining and Running Tests QWise software engineering – refactored! www.qwise.se
    13. 13. ISSUE TRACKING WITH RICH BUGS QWise software engineering – refactored! www.qwise.se
    14. 14. Should We Do Bug Tracking? Pros – Tracability – Priority of work – Visibility Cons – Overhead Should we log bugs within an iteration? QWise software engineering – refactored! Source: Pixar ”A bugs’ life” www.qwise.se
    15. 15. Defect Tracking in MTM and MTR QWise software engineering – refactored! www.qwise.se
    16. 16. No-more-no-repro • Automatic data collection • Test settings – Configuration of diagnostic data collectors – Assigned to test plan • Or per test run QWise software engineering – refactored! www.qwise.se
    17. 17. Diagnostic Data Adapters QWise software engineering – refactored! www.qwise.se
    18. 18. No-more-no-repro QWise software engineering – refactored! www.qwise.se
    19. 19. AUTOMATED UI TESTING QWise software engineering – refactored! www.qwise.se
    20. 20. Why Automated Testing? Källa: Tools for Agility, Kent Beck QWise software engineering – refactored! www.qwise.se
    21. 21. Automation Strategy iteration 11 iteration feature aa feature feature bb feature iteration 22 iteration feature c c feature release iteration release iteration feature dd feature unit testing manual testing regression testing performance testing QWise software engineering – refactored! www.qwise.se
    22. 22. Coded UI Tests • Unit test for UI • High-level framework • Create test from – Recording – Test result (action log) – Using API QWise software engineering – refactored! www.qwise.se
    23. 23. Automation Platform Support QWise software engineering – refactored! www.qwise.se
    24. 24. Coded UI Tests QWise software engineering – refactored! www.qwise.se
    25. 25. LAB MANAGEMENT QWise software engineering – refactored! www.qwise.se
    26. 26. Motivation QWise software engineering – refactored! www.qwise.se
    27. 27. Lab Management Aims To... QWise software engineering – refactored! www.qwise.se
    28. 28. Lab Management Infrastructure Team Foundation Server (TFS) Lab Management System Center Virtual Machine Manager (SCVMM) Test Case management Build management Work Item Tracking Source Control QWise software engineering – refactored! Hyper-V Hosts Library Shares Virtual Machine1 Library Share1 Virtual Machine2 Library Share2 Virtual Machinen Library Sharen www.qwise.se
    29. 29. Environments • An environment is a set of machines with different roles • An environment can – Be deployed, saved, removed – Started, stopped – Snapsshot VM1 Lab Agent QWise software engineering – refactored! VM2 Lab Agent www.qwise.se
    30. 30. Manual Tests in Lab • Testing from the outside – Easy for web applications – Limitations in data collection • Testing from the inside – Run application in lab via environment viewer QWise software engineering – refactored! www.qwise.se
    31. 31. Automatic Tests in Lab • Testing in VM – Test case connected to test environment – Test component comes from selected build • Testing in build – Test case connected to build definition – Runs test list or test component QWise software engineering – refactored! www.qwise.se
    32. 32. Lab Management Build Workflow QWise software engineering – refactored! www.qwise.se
    33. 33. Build and Deploy QWise software engineering – refactored! www.qwise.se
    34. 34. Summary • • • • • Structured test planning with MTM Integrated test execution Effective defect tracking with rich bugs Seamless integration with automated tests Powerful lab management QWise software engineering – refactored! www.qwise.se
    35. 35. Questions? mathias.olausson@callista.se http://msmvps.com/blogs/molausson QWise software engineering – refactored! www.qwise.se

    ×