Your SlideShare is downloading. ×
Testing SharePoint solutions overview
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Testing SharePoint solutions overview

3,863

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,863
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
68
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • These are some of the comments that we have heard from our customers. Are any of these being said by your testers?Broken down another way, can you answer these affirmatively?Do you know consistently when your software will ship?Are the defects created by your tester truly effective?No? <click to next>
  • Does this sound like a wish list? How much money is lost in the time that it takes to track down a bug?How much money is lost in the time it takes to isolate the differences between your development in test environments?
  • It is also important to understand where most testing happens in the spectrum of general testing to the more technical specialist testing.The Generalist Testers are usually professional testers with no coding background. Often these testers are experts in the business process or tool that is being developed. On the opposite side of the spectrum is the Specialist. This is a tester with strong coding skills.A fun side note: Microsoft’s testers are usually converted developers and tend to be on the specialist side of the graph.Black-box testing is a method of testing software that tests the functionality of an application as opposed to its internal structures or workings (see white-box testing). Specific knowledge of the application's code/internal structure and programming knowledge in general is not required. Test cases are built around specifications and requirements, i.e., what the application is supposed to do. It uses external descriptions of the software, including specifications, requirements, and design to derive test cases. These tests can be functional or non-functional, though usually functional. The test designer selects valid and invalid inputs and determines the correct output. There is no knowledge of the test object's internal structure.White-box testing (a.k.a. clear box testing, glass box testing, transparent box testing, or structural testing) is a method of testing software that tests internal structures or workings of an application as opposed to its functionality (black-box testing). An internal perspective of the system, as well as programming skills, are required and used to design test cases. The tester chooses inputs to exercise paths through the code and determine the appropriate outputs. It is analogous to testing nodes in a circuit, e.g. in-circuit testing (ICT). While white-box testing can be applied at the unit, integration and system levels of the software testing process, it is usually done at the unit level. It can test paths within a unit, paths between units during integration, and between subsystems during a system level test. Though this method of test design can uncover many errors or problems, it might not detect unimplemented parts of the specification or missing requirements. White-box test design techniques include: Control flow testing Data flow testing Branch testing Path testingAPI testing (application programming interface) – is a specific type of White Box testing of the application focusing on public and private APIs<Question to Audience>Looking at this spectrum, where does most testing happen today? <collect answers and click>Where do most testing tools target today? <collect answers and click>
  • Describe the pieces as it applies to the spectrum as the slide builds out.
  • Currently about 70% of functional testing is done manually—that is, a software tester follows a script to execute a series of steps to verify the outcome of a test. Microsoft Test Manager 2010 is an example of a tool that someone doing manual testing would use to be more productive. Imagine if you could still get the benefits of functional testing without the overhead and resource cost of doing in manually.Consider a regression test—a test designed to identify if a bug fix begins to fail. With a regression test (and even other functional tests) you are repeatedly testing something that was known to work at one point. These too are typically done manually. This is a massive resource hit. Software testers spend countless hours testing functionality that works, solely to ensure it still works. Imagine how much more productive they could be if they could focus their efforts on creating and running new test cases that covered parts of the system not currently tested instead of spending their time testing the same thing, over and over again.
  • Here are two examples <Build> of the dashboard you get when running on top of one of the full versions of SharePoint Server. First, we have the Burndown dashboard showing project progress. <Build> Second, we have the Quality dashboard. So what does the quality dashboard tell us? <next>
  • As you can see the Quality dashboard has four main graphs. <Build>First you can see if your test team is making progress on running test plans. <Build>Second, you can see how build are doing over time. What’s the trend like? Are you having lots of success or failure? <Build>Third, what’s your bug trend like. Are you closing out bugs or are you stagnate. Or is the velocity of your bug filing far exceeding your team’s ability to fix, test, and close out bugs? <Build>Finally, are you seeing a bad trend related to bug reactivations—bugs that were closed reopened by test as not fixed? All of this information is there for you in a quick, heads up dashboard format.
  • Transcript

    • 1. Testing SharePoint SolutionsOverview
      Ervin Loh
      Visual Studio ALM MVP
      K365Labs SdnBhd
      Light Up SharePoint
    • 2. Presenter
      Ervin Loh
      Profile
      Ervin Loh is currently the Application Lifecycle Management Program Manager at K365Labs Sdn Bhd. He works on a variety of Application Lifecycle Management, Software Configuration Management, and Lab Management products. He is also active in the IT community space by contributing contents to Microsoft Malaysia's START.NET and Ultimate program workshops and talks in conferences, events and user groups.
    • 3. ASP.NET Developers, SharePoint Developers
      Target Audience
    • 4. Is this how you kill bugs?
    • 5.
    • 6.
    • 7.
    • 8. An Unfortunately Common Scenario
      Developer writes code
      Developer makes sure the code compiles
      Developer checks in code
      Repeat x15 developers for 4 weeks
      Developer lead does a build of all code from developers
      Application is installed in test environment
      Tester tries to test the application
      Application doesn’t work
      Developer blames tester
    • 9. The Demonstrations Scenario
      The Configurations
      A fully configured Visual Studio 2010, Team Foundation Server 2010, and SharePoint Server 2010 environment.
      The Tasks
      A lot of tests.
    • 10. Topics
      Overviewofthetestingtools in Visual Studio 2010
      Testing SharePoint projectswith Visual Studio 2010
      10
    • 11. Have you heard any of these?
      “my testers are spending too long testing the same thing”
      “tooling is expensive (time, licenses, people)”
      “the developers say the defects are useless”
      “when is my software ready to ship?”
      “developers and testers work in silos and don’t communicate/speak the same language”
      “we have a changed requirement, what do I need to test?”
    • 12. What if you could…
      reduce the time it takes to determine the root cause of a bug
      reduce the time it takes to replicate a bug uncovered by user actions
      reduce the time it takes to isolate differences between the test and production environment
      enable users to easily run acceptance tests and track their results
      reduce the time it takes to verify the status of a reported bug fix
    • 13. Where does testing happen?
      70% of testing happens here
      Majority of test tools target here
      Black Box Testing
      White Box Testing
      API Testing
    • 14. Visual Studio 2010 Test Capabilities
      Load Test
      Web Test
      Coded UI Test
      Microsoft Test Runner
      Unit Testing
      Test Case Management
      Lab Management
      Data Collectors (Historical Debugging, Test Impact, Metrics)
      Team Foundation Server
      Reporting
    • 15. Typical Test Cycle
    • 16. Why is fixing bugs difficult?
      Insufficient information to reproduce
      Not every step is documented
      Can’t see exactly how the bug was triggered
      Different environments
      OS, service packs, installed software,…
      Creating a clean environment takes time
    • 17. Microsoft Test Manager 2010
      Plan, Manage and Execute (manual) tests from one place
      Create test cases
      Build test suites
      Define configurations
      Run test cases
      File bugs into TFS
      Including captured data
      Automate testing
      Integrated with TFS
    • 18. VS Agents: Diagnostic Data Adapters
      System Info
      IntelliTrace™
      Video Capture
      Steps Performed
      System Info
      IntelliTrace™
      Visual Studio Agents 2010
      Separate download
      Test Controller & Test Agents
      Capture data while testing
      Save data with bug report
      Easy for developer to reproduce bug
    • 19. Lab Manager 2010
      Setup, Teardown, and Restore virtual environments
      Maintain known state
      Build automation
      Build deployment
      Test execution
      Network isolation
      Run multiple copies
      Connect from VS2010
      Linked to bug form
    • 20. 70%
      of all functional testing is still done manually.
    • 21. DemoFunctional Testing with Microsoft Test Manager
    • 22. Automated Testing for SharePoint
      • Web Performance Testing
      • 23. Load Testing
      • 24. Coded UI Testing
      • 25. Unit Testing
    • Web Performance Testing
      Simulate a single use case
      Internet Explorer Web Testing toolbar
      Records the user’s web requests
      Important: This is not a UI test
      Validation Rules
      Extraction Rules
      Parameterized Tests
      Data-Driven Tests
    • 26. Load Testing
      Stress-testing: simulate many users at once
      Need multiple servers to create substantial load
      Multiple agents generate load on the application
      Single controller coordinates the agents and captures data
      PerfMon-style output
    • 27. Coded UI Testing
      Simulate a single use case
      Uses a recording toolbar, like Web Performance test
      Records the user’s interaction with the UI
      Generates code to reproduce that interaction
      Code can be edited and customized at will
      Has some issues with SharePoint
      e.g. no support for datasheets
      Visual Studio Feature Pack 2 adds support for Silverlight
    • 28. Unit Testing
      Using built-in Unit Test with SharePoint has problems:
      Issues with 32-bit vs 64-bit, and .NET 3.5 vs .NET 4.0
      Need to stub/mock SharePoint API
      e.g. SPSite, SPWeb, SPList
      Additional tools enable unit testing for SharePoint projects
      Microsoft Research: Pex & Moles
      Integrates with VS2010, free to MSDN subscribers
      http://research.microsoft.com/en-us/projects/pex/pexsharepoint.pdf
      Third party: NUnit, TypeMock
    • 29. DemoAutomated Testing with Microsoft Visual Studio
    • 30. SharePoint Dashboards
    • 31. Are we ready to ship?
      Are we making progress on running test plans?
      How are our builds doing over time?
      Are we fixing bugs?
      What’s the quality of our bug fixes?
      Ready to Ship?
    • 32. DemoReporting
    • 33.
    • 34. Automation Platform Support
    • 35. Resources
      Test Manager 2010 and Lab Manager
      http://msdn.microsoft.com/en-us/library/ms182409.aspx
      http://msdn.microsoft.com/en-us/library/dd997438.aspx
      Visual Studio 2010 Agents
      http://www.microsoft.com/downloads/en/details.aspx?FamilyID=a3216d2a-0530-4f6c-a7c9-0df37c54a902
      Pex and Moles
      http://research.microsoft.com/en-us/projects/pex/default.aspx
    • 36. Questions?
    • 37. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Visual Studio, the Visual Studio logo, and [list other trademarks referenced] are trademarks of the Microsoft group of companies.
       
      The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation.  Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.  MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

    ×