Overview of Lab Management and TFS


Published on

A presentation I did with Microsoft on the "Testing Tour"

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • “Of course, we’ve had Visual Studio for a long time…” [click]
    “In 2005, we introduced Team Foundation Server, which allowed us to provide a central repository of data around application development…” [click]
    “This central server also became our integration point for those not working within Visual Studio…” [click]
    “With 11, we are incorporating the DevOps story and other tools for managing an application outside of its relatively short time in developer’s hands…” [click]
    “And, as with most Microsoft products, the real strength of this platform is the ability to extend it, integrate it, and make it your own!”
  • Considering how IT has evolved over the past decades. Why was Agile a relatively newcomer to the process?

    Setting aside that debate for a second. The cost of transitioning states was and still can be an expensive endeavor. For instance, if the cost to deploy your application to the QA team is expensive, I would try to do that as few times as I could. The same holds – if the cost of deploying the application is high, I’d want to do that as few times as possible. So – the longer that I could stay in that state – that I just transitioned to – I get more value. My team (it was thought) was more productive because they are doing the same thing in a repeatable fashion. Now what I mean by that was – after I’m in my test/stabilization phase, I knew I could have my testers execute tests, and execute tests repeatable. I can have my developers fix bugs repeatedly. I wasn’t trying to mix things up by having people work on different functions on the project at the same time.

    Setting aside agile v waterfall at the outset, what if there was this world, where at least we reduced or almost removed those hurdles to giving teams the way to decide what is most efficient for them without the artificial challenges. What if we automate the build/deploy/test cycle for one. What if we could actually have tools that could make us more efficient in doing our real jobs – in this case – the testing of our environment.
  • Practices to continuously test software against the defined acceptance criteria as it is being developed, using the appropriate tools, and by the appropriate stakeholder functions (product owners, developers, testers, users). Continuous acceptance testing helps prevent late detection of unmet requirements and reduces rework costs.

    An ALM solution that enables continuous acceptance testing should include tools and services to:

    Enable a mix of testing practices that engage cross-functional stakeholders in continuously validating the fulfillment of acceptance criteria (product owners, developers, testers, users, and operations). Engaging stakeholder functions early and often in the lifecycle to validate the fulfillment of acceptance criteria will help prevent the late detection of unmet requirements and reduce rework costs
    Build -> deploy -> test automation services to reduce cycle times in continuously integrating and validating the quality of team builds
    Test lab management automation services to reduce the cycle times in standing up and releasing test environments needed to continuously execute tests of varying intents (build verification testing, systems integration testing, user acceptance testing, stress testing)

  • Talk Track: A lab management automation solution reduces the cycle times to manage and provision the test environments needed to enable continuous acceptance testing.  
  • A lab management automation solution reduces the cycle times to manage and provision the test environments needed to enable continuous acceptance testing. A lab management automation solution should include capabilities to:

    Store and manage baseline templates for the needed test environments
    Provision environments on demand using the environment templates
    Provision environments to on-premise bare metal, private cloud, and public cloud infrastructures
    Snapshot environments when defects are encountered and needed to be reported
    Provision environments from snapshots to enable defect reproduction and resolution

    Environments supported include:

    On-premise self management environments
    On-premise virtualized environments managed using SCCM
  • Talk Track: Lab Center lets you create and manage your testing environments from a single location. Environments can be stored on local metal, or virtualized on site or in the cloud. Maintain multiple realistic environments so that your application can be quickly deployed and tested under the configurations you expect your users to use.

    Environments can be based on self-managed infrastructure or virtualized infrastructure managed using SCCM (System Center Configuration Manager)
  • Talk Track:
    With Lab Manager, your build and deployment cycle times can be reduced in step with your development and testing cycle times. Automating your build and deployment pipeline will reduce manual errors, decrease cycle times, and increase quality.
  • Talk Track:
    Whether it is bare metal machines, or multi-part SCVMM environments, Lab Manager helps you manage your test environments from a single location. Testers can quickly find the right environment to test on, reducing hand-off times and decreasing manual deployment defects. The ability to take environment snapshots when bugs are found significantly reduces the likelihood of “no-repro” errors.
  • Talk Track: Enable data collectors and generate detailed test results to ensure any bugs that are found are actionable.
    Team Foundation Server keeps track of all the pieces, keeping your entire team on the same page and reducing time to repair.
  • Talk Track: Historically, the build-deploy-test pipeline is the most time expensive and manually-driven portion of the software development lifecycle.
    Introducing continuous deployment with Lab Manager ensures that testers know which builds to test and have the correct virtual environments to test on; it eliminates the hand-off wait time between developers and QA.
    Quality is increased, cycle times are reduced, and value is delivered to your customers.
  • Overview of Lab Management and TFS

    1. 1. Setting Up Dev & Test Environments With TFS and Visual Studio Chris Kadel - Principal Twitter: @cbkadel Blog: http://chriskadel.com Website: http://polarissolutions.com Email: chris.kadel@polarissolutions.com
    2. 2. Testing Maturity / Dynamic Environment Ad Hoc Manual Automate Regression Load Environment Ad Hoc Planned Virtual Lab
    3. 3. Characteristics of Efficient Testing Clear Requirements Bugs are Reproduced Devs Fix on First Time We Have Environments On Demand
    4. 4. Why did Waterfall dominate approaches throughout most of IT “time”? Economy of scale… Requirements Implement Test / Stabilize Release Cost of Setup Avoid Context Switch
    5. 5. Continuous acceptance testing User Interface Services Business Processes Business Rules and Logic Service integrations Data access Identity Data Automated testing  Unit testing  Integration testing  Performance testing  Load testing Manual testing  User testing  Ad-hoc exploratory testing  Planned manual testing Automated testing  Keyword testing  UI automation testing  Performance testing Continuous integrations Build | Deploy | Test Testing toolbox Role tailored tools
    6. 6. Why Lab Environment Automation? Can I get another QA environment to isolate some testing without slowing down my team? We need to isolate that environment like it were a retail location (network isolated)? We just need a QA environment for a few weeks and then done forever? Remember those data collectors for my client-server application? I want those. I want to do all that above, but I want it up in Windows Azure.
    7. 7. Lab management automation On-premise | Cloud
    8. 8. Agents Installed on the “client” machines Three types of agents Test Agent-Enables automated testing Build Agent-Enables automated deployment Lab Agent-Enables network isolation Installed Automatically by Lab Management Services
    9. 9. Define Environments Create new multi-machine environment choices
    10. 10. Create a new environment using virtual machine templates in SCVMM Create an environment using machines running on SCVMM Create new environments on existing machines
    11. 11. Quickly choose multiple test suites to run after deployment Choose your environment, and build, deploy and test in that environment from a single dialog
    12. 12. Snapshot environments during manual testing to increase reproducibility Spin up virtual test environments from within Test Manager, or attach a virtual environment to a manual testing session
    13. 13. Developer Experience Environment snapshot taken and attached as a link with bug Developers can quickly debug issues on the exact same testing environment
    14. 14. Test run results are attached to specific builds and environments Collect all the data required to recreate bugs from virtual environments
    15. 15. Web Server Collect Intellitrace Information Test Agent Test Controllers and Agents Client Network Emulation Test Agent Run Tests Database Server Collect System Information Test Agent Where to run Tests? Collect Data? Affect Machine Client Collect Test Impact Data Test Agent Run Tests Team Foundation Server Test Agent Controller Register with Team Project Collection Test agent controller manages test agents Test agents run tests and collect data or impact the system
    16. 16. About the Demo Tester Team Define Tests Define Environments Define Data Collection Analyze Results Development / Release Team Author Build Author Deployment Run Automate Tests
    17. 17. demo Lab Manager Build-Deploy-Test Scenario
    18. 18. On Demand in the Cloud On Premises TFS Server Test Controller Spawn Azure Automatically Gallery or VHD Manage in Lab Center Central Start/Stop Functionality Snapshot in Azure
    19. 19. Labs and Walkthroughs Creating VMs Create and store VMs - MSDN doc Prepare a domain controller VM Manage virtual environments Creating virtual envs - MSDN doc Creating and working with virtual envs Concepts and guidelines Creating network isolated environments Networking Networking basics Network isolation
    20. 20. Parting thoughts… Empowerment Really Automate Dynamic Environments Crisp Communication Reduce Costs
    21. 21. Chris Kadel - Principal Twitter: @cbkadel Blog: http://chriskadel.com Website: http://polarissolutions.com Email: chris.kadel@polarissolutions.com