Testing SharePoint solutions overview

Uploaded on


More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide
  • These are some of the comments that we have heard from our customers. Are any of these being said by your testers?Broken down another way, can you answer these affirmatively?Do you know consistently when your software will ship?Are the defects created by your tester truly effective?No? <click to next>
  • Does this sound like a wish list? How much money is lost in the time that it takes to track down a bug?How much money is lost in the time it takes to isolate the differences between your development in test environments?
  • It is also important to understand where most testing happens in the spectrum of general testing to the more technical specialist testing.The Generalist Testers are usually professional testers with no coding background. Often these testers are experts in the business process or tool that is being developed. On the opposite side of the spectrum is the Specialist. This is a tester with strong coding skills.A fun side note: Microsoft’s testers are usually converted developers and tend to be on the specialist side of the graph.Black-box testing is a method of testing software that tests the functionality of an application as opposed to its internal structures or workings (see white-box testing). Specific knowledge of the application's code/internal structure and programming knowledge in general is not required. Test cases are built around specifications and requirements, i.e., what the application is supposed to do. It uses external descriptions of the software, including specifications, requirements, and design to derive test cases. These tests can be functional or non-functional, though usually functional. The test designer selects valid and invalid inputs and determines the correct output. There is no knowledge of the test object's internal structure.White-box testing (a.k.a. clear box testing, glass box testing, transparent box testing, or structural testing) is a method of testing software that tests internal structures or workings of an application as opposed to its functionality (black-box testing). An internal perspective of the system, as well as programming skills, are required and used to design test cases. The tester chooses inputs to exercise paths through the code and determine the appropriate outputs. It is analogous to testing nodes in a circuit, e.g. in-circuit testing (ICT). While white-box testing can be applied at the unit, integration and system levels of the software testing process, it is usually done at the unit level. It can test paths within a unit, paths between units during integration, and between subsystems during a system level test. Though this method of test design can uncover many errors or problems, it might not detect unimplemented parts of the specification or missing requirements. White-box test design techniques include: Control flow testing Data flow testing Branch testing Path testingAPI testing (application programming interface) – is a specific type of White Box testing of the application focusing on public and private APIs<Question to Audience>Looking at this spectrum, where does most testing happen today? <collect answers and click>Where do most testing tools target today? <collect answers and click>
  • Describe the pieces as it applies to the spectrum as the slide builds out.
  • Currently about 70% of functional testing is done manually—that is, a software tester follows a script to execute a series of steps to verify the outcome of a test. Microsoft Test Manager 2010 is an example of a tool that someone doing manual testing would use to be more productive. Imagine if you could still get the benefits of functional testing without the overhead and resource cost of doing in manually.Consider a regression test—a test designed to identify if a bug fix begins to fail. With a regression test (and even other functional tests) you are repeatedly testing something that was known to work at one point. These too are typically done manually. This is a massive resource hit. Software testers spend countless hours testing functionality that works, solely to ensure it still works. Imagine how much more productive they could be if they could focus their efforts on creating and running new test cases that covered parts of the system not currently tested instead of spending their time testing the same thing, over and over again.
  • Here are two examples <Build> of the dashboard you get when running on top of one of the full versions of SharePoint Server. First, we have the Burndown dashboard showing project progress. <Build> Second, we have the Quality dashboard. So what does the quality dashboard tell us? <next>
  • As you can see the Quality dashboard has four main graphs. <Build>First you can see if your test team is making progress on running test plans. <Build>Second, you can see how build are doing over time. What’s the trend like? Are you having lots of success or failure? <Build>Third, what’s your bug trend like. Are you closing out bugs or are you stagnate. Or is the velocity of your bug filing far exceeding your team’s ability to fix, test, and close out bugs? <Build>Finally, are you seeing a bad trend related to bug reactivations—bugs that were closed reopened by test as not fixed? All of this information is there for you in a quick, heads up dashboard format.


  • 1. Testing SharePoint SolutionsOverview
    Ervin Loh
    Visual Studio ALM MVP
    K365Labs SdnBhd
    Light Up SharePoint
  • 2. Presenter
    Ervin Loh
    Ervin Loh is currently the Application Lifecycle Management Program Manager at K365Labs Sdn Bhd. He works on a variety of Application Lifecycle Management, Software Configuration Management, and Lab Management products. He is also active in the IT community space by contributing contents to Microsoft Malaysia's START.NET and Ultimate program workshops and talks in conferences, events and user groups.
  • 3. ASP.NET Developers, SharePoint Developers
    Target Audience
  • 4. Is this how you kill bugs?
  • 5.
  • 6.
  • 7.
  • 8. An Unfortunately Common Scenario
    Developer writes code
    Developer makes sure the code compiles
    Developer checks in code
    Repeat x15 developers for 4 weeks
    Developer lead does a build of all code from developers
    Application is installed in test environment
    Tester tries to test the application
    Application doesn’t work
    Developer blames tester
  • 9. The Demonstrations Scenario
    The Configurations
    A fully configured Visual Studio 2010, Team Foundation Server 2010, and SharePoint Server 2010 environment.
    The Tasks
    A lot of tests.
  • 10. Topics
    Overviewofthetestingtools in Visual Studio 2010
    Testing SharePoint projectswith Visual Studio 2010
  • 11. Have you heard any of these?
    “my testers are spending too long testing the same thing”
    “tooling is expensive (time, licenses, people)”
    “the developers say the defects are useless”
    “when is my software ready to ship?”
    “developers and testers work in silos and don’t communicate/speak the same language”
    “we have a changed requirement, what do I need to test?”
  • 12. What if you could…
    reduce the time it takes to determine the root cause of a bug
    reduce the time it takes to replicate a bug uncovered by user actions
    reduce the time it takes to isolate differences between the test and production environment
    enable users to easily run acceptance tests and track their results
    reduce the time it takes to verify the status of a reported bug fix
  • 13. Where does testing happen?
    70% of testing happens here
    Majority of test tools target here
    Black Box Testing
    White Box Testing
    API Testing
  • 14. Visual Studio 2010 Test Capabilities
    Load Test
    Web Test
    Coded UI Test
    Microsoft Test Runner
    Unit Testing
    Test Case Management
    Lab Management
    Data Collectors (Historical Debugging, Test Impact, Metrics)
    Team Foundation Server
  • 15. Typical Test Cycle
  • 16. Why is fixing bugs difficult?
    Insufficient information to reproduce
    Not every step is documented
    Can’t see exactly how the bug was triggered
    Different environments
    OS, service packs, installed software,…
    Creating a clean environment takes time
  • 17. Microsoft Test Manager 2010
    Plan, Manage and Execute (manual) tests from one place
    Create test cases
    Build test suites
    Define configurations
    Run test cases
    File bugs into TFS
    Including captured data
    Automate testing
    Integrated with TFS
  • 18. VS Agents: Diagnostic Data Adapters
    System Info
    Video Capture
    Steps Performed
    System Info
    Visual Studio Agents 2010
    Separate download
    Test Controller & Test Agents
    Capture data while testing
    Save data with bug report
    Easy for developer to reproduce bug
  • 19. Lab Manager 2010
    Setup, Teardown, and Restore virtual environments
    Maintain known state
    Build automation
    Build deployment
    Test execution
    Network isolation
    Run multiple copies
    Connect from VS2010
    Linked to bug form
  • 20. 70%
    of all functional testing is still done manually.
  • 21. DemoFunctional Testing with Microsoft Test Manager
  • 22. Automated Testing for SharePoint
    • Web Performance Testing
    • 23. Load Testing
    • 24. Coded UI Testing
    • 25. Unit Testing
  • Web Performance Testing
    Simulate a single use case
    Internet Explorer Web Testing toolbar
    Records the user’s web requests
    Important: This is not a UI test
    Validation Rules
    Extraction Rules
    Parameterized Tests
    Data-Driven Tests
  • 26. Load Testing
    Stress-testing: simulate many users at once
    Need multiple servers to create substantial load
    Multiple agents generate load on the application
    Single controller coordinates the agents and captures data
    PerfMon-style output
  • 27. Coded UI Testing
    Simulate a single use case
    Uses a recording toolbar, like Web Performance test
    Records the user’s interaction with the UI
    Generates code to reproduce that interaction
    Code can be edited and customized at will
    Has some issues with SharePoint
    e.g. no support for datasheets
    Visual Studio Feature Pack 2 adds support for Silverlight
  • 28. Unit Testing
    Using built-in Unit Test with SharePoint has problems:
    Issues with 32-bit vs 64-bit, and .NET 3.5 vs .NET 4.0
    Need to stub/mock SharePoint API
    e.g. SPSite, SPWeb, SPList
    Additional tools enable unit testing for SharePoint projects
    Microsoft Research: Pex & Moles
    Integrates with VS2010, free to MSDN subscribers
    Third party: NUnit, TypeMock
  • 29. DemoAutomated Testing with Microsoft Visual Studio
  • 30. SharePoint Dashboards
  • 31. Are we ready to ship?
    Are we making progress on running test plans?
    How are our builds doing over time?
    Are we fixing bugs?
    What’s the quality of our bug fixes?
    Ready to Ship?
  • 32. DemoReporting
  • 33.
  • 34. Automation Platform Support
  • 35. Resources
    Test Manager 2010 and Lab Manager
    Visual Studio 2010 Agents
    Pex and Moles
  • 36. Questions?
  • 37. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Visual Studio, the Visual Studio logo, and [list other trademarks referenced] are trademarks of the Microsoft group of companies.
    The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation.  Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.  MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.