The document discusses expanding a test portfolio with Coded UI tests and cloud-based load testing in Visual Studio. It covers automating tests with Microsoft Test Manager, creating automated UI tests using Coded UI, and performing load testing on applications hosted in the cloud using Visual Studio Online. Conducting load testing early and customizing tests to measure key performance metrics can help ensure applications meet expectations before public release.
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Udvid din test portefølje med coded ui test og cloud load test
1. Udvid din test portefølje med
Coded UI test og Cloud Load test
VS20011
Peter Lindberg - Consultant @ Alterate
pli@alterate.dk
@peterlindbergdk
www.linkedin.com/in/peterlindbergdk
www.alterate.dk
#CampusDays
2. #CampusDays
Topics
Test automation
Microsoft Test Manager and manual tests
Coded UI Test
Load Test in the cloud with Visual Studio Online
3. #CampusDays
Why test?
We all make mistakes!
But also:
• Are we meeting the requirements and expectations?
• Does the hardware support the application?
• Does it work as expected?
The sooner the defects are discovered – the cheaper they are to fix
4. #CampusDays
Why automate tests?
Cost reduction!
• Manual test is expensive
• Let manual testers spend the time on value adding tests
• New features
• Exploratory testing
• “Time to market” – higher release cadence
• Safety as developer
9. #CampusDays
Microsoft Test Manager (MTM)
Separate application
Fully integrated with Team Foundation Server
• Test artifacts are Work Items in TFS
• Links to Product Backlog Items, Bugs etc.
• Queries
• Reporting
• Ties directly in to the developer world
10. #CampusDays
Microsoft Test Manager (MTM)
A fully-instrumented, configurable test runner captures detailed records of steps
performed, behaviors observed, and the status of each test step. Pause testing to report a
bug, even if it’s not directly related to the current test. High-fidelity defect reports go
directly to the development team, including all the technical data needed to reproduce
and resolve the error. Record tests for later playback to speed test sessions and enable
automation. (visualstudio.com)
11. #CampusDays
What’s in it for the developers?
“captures detailed records of steps performed, behaviors observed, and the status of each
test step”
“Pause testing to report a bug, even if it’s not directly related to the current test”
“High-fidelity defect reports go directly to the development team, including all the
technical data needed to reproduce and resolve the error”
“Record tests for later playback to speed test sessions and enable automation”
15. #CampusDays
What are Coded UI Tests?
Automated tests interacting with the application through the user interface
For example used for testing
• Input validations
• Calculations
• “Shopping cart” functionality
• UI Elements
• Navigation
• ….
16. #CampusDays
Supported configurations and platforms
Configurations
• Windows 7, Windows 8, Windows Server 2008 R2
• x86 and x64
• .NET 2.0, 3.0, 3.5, 4.0 and 4.5
Platforms
• Windows Phone Apps - (Only WinRT-XAML based Phone apps are supported)
• Windows Store Apps - (Only XAML-based Store apps are supported)
• Windows Forms + WPF
• IE 8, 9 and 10 - (9 and 10 only supported on the desktop)
• Chrome (recording not supported, only playback)
• Silverlight 5 (only supported with unsupported plugin)
• SharePoint, Dynamics web client – Fully supported
19. #CampusDays
Which tests to automate?
Is it often used functionality?
Does the UI change often?
Is the test code easy or difficult to maintain?
How often is the test expected to be run?
20. #CampusDays
Coded UI test features
• Fully integrated into Visual Studio ALM
• Automatic code generation
• “Intelligent” Intent Aware Recording and Resilient Playback
• Execute on build server
• C# / VB.NET
21. #CampusDays
Generate Coded UI Tests
• Automate steps from MTM action recordings
• Coded UI Test Builder
• Write them yourself in C# / VB.NET (advanced)
25. #CampusDays
Coded UI Test Editor
• Change names of test methods and UI elements
• Split steps in to new methods
• Move generated code to UIMap partial class for customizing
• Add pauses
• View and change properties
• Remove unwanted steps (duplicates etc.)
29. #CampusDays
What do we want to achieve with load tests?
• Will the planned platform support the expected load and peak loads?
• Will simultaneous users cause problems
• Server load (max. CPU / memory consumption etc.)
• When does it break and how?
Planning is key! Dialogue with Product Owner / stakeholder etc.
What are targets and objectives?
What are the success & acceptance criteria?
Concrete and measurable!!
There’s a big difference between 1 and 5 seconds when a web site load…
30. #CampusDays
How?
Generate
traffic to
application
Does it
perform as
expected?
Recreate the
performance
problems
Fix the
performance
problems
32. #CampusDays
So when should I start planning and execute load teste?
Start early!
=
Save time
33. #CampusDays
What can be included in load tests?
• Web test
• Unit tests
• Coded UI (possible – but not recommended)
Customize!
• Coded Web Performance test
35. #CampusDays
To the Cloud!
• Load tests can be executed from the cloud with Visual Studio Online
• MSDN subscribers have 15,000 virtual user minutes included pr. month
• No need to set up a big test rig / infrastructure “on premises”
• Setup in Visual Studio – execute in Azure
• Visual Studio connected to Visual Studio Online
36. #CampusDays
Visual Studio
Load test
web service
Result
database
Controller
Test agent pool
Agent Agent Agent
Agent Agent Agent
38. #CampusDays
Q&A
#Ask me about everything!
Join me at the Microsoft Booth the next
30 minutes @Meet The Experts
Dont forget to: Evaluate this session!
Intent aware = Taking all the steps to click All Programs, open browser, navigate to URL => Open URL in browser
Resilient playback = smart algorithms to accommodate changes in configurations like window resize/Minimize/Maximize, screen resolution changes etc. to provide a resilient playback support.
20 mins
CodedUISP proj:
Section title slide (Optional)
Unit test – good for WCF Services, Web API, REST, Odata etc.
Also database queries, Store Procedures etc.
Create small web tests to be able to granularly load test and diagnose the application functionality