Testing SharePoint SolutionsOverview Ervin Loh Visual Studio ALM MVP K365Labs SdnBhd Light Up SharePoint
Presenter Ervin Loh Profile Ervin Loh is currently the Application Lifecycle Management Program Manager at K365Labs Sdn Bhd. He works on a variety of Application Lifecycle Management, Software Configuration Management, and Lab Management products. He is also active in the IT community space by contributing contents to Microsoft Malaysia's START.NET and Ultimate program workshops and talks in conferences, events and user groups.
ASP.NET Developers, SharePoint Developers Target Audience
Is this how you kill bugs?
An Unfortunately Common Scenario Developer writes code Developer makes sure the code compiles Developer checks in code Repeat x15 developers for 4 weeks Developer lead does a build of all code from developers Application is installed in test environment Tester tries to test the application Application doesn’t work Developer blames tester
The Demonstrations Scenario The Configurations A fully configured Visual Studio 2010, Team Foundation Server 2010, and SharePoint Server 2010 environment. The Tasks A lot of tests.
Topics Overviewofthetestingtools in Visual Studio 2010 Testing SharePoint projectswith Visual Studio 2010 10
Have you heard any of these? “my testers are spending too long testing the same thing” “tooling is expensive (time, licenses, people)” “the developers say the defects are useless” “when is my software ready to ship?” “developers and testers work in silos and don’t communicate/speak the same language” “we have a changed requirement, what do I need to test?”
What if you could… reduce the time it takes to determine the root cause of a bug reduce the time it takes to replicate a bug uncovered by user actions reduce the time it takes to isolate differences between the test and production environment enable users to easily run acceptance tests and track their results reduce the time it takes to verify the status of a reported bug fix
Where does testing happen? 70% of testing happens here Majority of test tools target here Black Box Testing White Box Testing API Testing
Visual Studio 2010 Test Capabilities Load Test Web Test Coded UI Test Microsoft Test Runner Unit Testing Test Case Management Lab Management Data Collectors (Historical Debugging, Test Impact, Metrics) Team Foundation Server Reporting
Typical Test Cycle
Why is fixing bugs difficult? Insufficient information to reproduce Not every step is documented Can’t see exactly how the bug was triggered Different environments OS, service packs, installed software,… Creating a clean environment takes time
Microsoft Test Manager 2010 Plan, Manage and Execute (manual) tests from one place Create test cases Build test suites Define configurations Run test cases File bugs into TFS Including captured data Automate testing Integrated with TFS
VS Agents: Diagnostic Data Adapters System Info IntelliTrace™ Video Capture Steps Performed System Info IntelliTrace™ Visual Studio Agents 2010 Separate download Test Controller & Test Agents Capture data while testing Save data with bug report Easy for developer to reproduce bug
Lab Manager 2010 Setup, Teardown, and Restore virtual environments Maintain known state Build automation Build deployment Test execution Network isolation Run multiple copies Connect from VS2010 Linked to bug form
70% of all functional testing is still done manually.
DemoFunctional Testing with Microsoft Test Manager
Automated Testing for SharePoint
Web Performance Testing
Coded UI Testing
Web Performance Testing Simulate a single use case Internet Explorer Web Testing toolbar Records the user’s web requests Important: This is not a UI test Validation Rules Extraction Rules Parameterized Tests Data-Driven Tests
Load Testing Stress-testing: simulate many users at once Need multiple servers to create substantial load Multiple agents generate load on the application Single controller coordinates the agents and captures data PerfMon-style output
Coded UI Testing Simulate a single use case Uses a recording toolbar, like Web Performance test Records the user’s interaction with the UI Generates code to reproduce that interaction Code can be edited and customized at will Has some issues with SharePoint e.g. no support for datasheets Visual Studio Feature Pack 2 adds support for Silverlight
Unit Testing Using built-in Unit Test with SharePoint has problems: Issues with 32-bit vs 64-bit, and .NET 3.5 vs .NET 4.0 Need to stub/mock SharePoint API e.g. SPSite, SPWeb, SPList Additional tools enable unit testing for SharePoint projects Microsoft Research: Pex & Moles Integrates with VS2010, free to MSDN subscribers http://research.microsoft.com/en-us/projects/pex/pexsharepoint.pdf Third party: NUnit, TypeMock
DemoAutomated Testing with Microsoft Visual Studio
Are we ready to ship? Are we making progress on running test plans? How are our builds doing over time? Are we fixing bugs? What’s the quality of our bug fixes? Ready to Ship?
Automation Platform Support
Resources Test Manager 2010 and Lab Manager http://msdn.microsoft.com/en-us/library/ms182409.aspx http://msdn.microsoft.com/en-us/library/dd997438.aspx Visual Studio 2010 Agents http://www.microsoft.com/downloads/en/details.aspx?FamilyID=a3216d2a-0530-4f6c-a7c9-0df37c54a902 Pex and Moles http://research.microsoft.com/en-us/projects/pex/default.aspx
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.