Deploy and Destroy Complete Test Environments


Published on

Slides from my talk on service virtualization, containers and cloud at TestCon 2016 in Vilnius, Lithuania

Published in: Software
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Everybody is doing some sort of CD, or at least is looking into it
  • CD requires the ability to test continuously, every deployment should be thoroughly tested before being put into a production environment
    Test automation plays a big role in this
  • The ability to test continuously is often hard to achieve because of test environment impediments
  • The T-shaped tester is a sought after profile at the moment, very good in one thing (testing, automation) and skilled in myriad other things (analysis, development, …)
    Continuous Testing requires something similar: a T-shaped test environment. First and foremost it should be under total control of the team using it, but next to that it should have other characteristics as well..
  • One approach to enable Continuous Testing is through implementation of service virtualization, an approach that removes bottlenecks in test environments by virtualizing dependencies that are critical to the testing process yet hard-to-access
  • Characteristics of service virtualization
  • R&P: draw parallel to test automation: nice to get a first demo up and running (or analyse traffic where no specifications are known), but to achieve long term success it’s best not to rely on R&P too much. Instead, create virtual assets using request-response specifications. R&P also only simulated those situations that CAN be recorded.
  • How should you go about SV implementation? Start small, both in terms of number of dependencies simulated and behavior replicated. Model just enough to enable execution of desired test cases (all else is filler, remember that SV is an enabler for testing, not a goal in itself). Learn and expand.
  • So how does the testing process ultimately benefit from SV implementation?
  • So, we have seen that implementing SV can lead to great benefits for the testing process, but to make SV even more flexible and better integrated into the CD process, wouldn’t it be nice if we could treat our test environment in the same manner as our application under test?
  • By containerizing and distributing SV engines and their virtual assets, we can do…
  • So, what does an approach like this look like? Talk them through the picture. Build server also runs tests (unit tests), we’re talking about integration and end-to-end tests here.
  • Guarantees that an application will always run in exactly the same way, regardless of the environment it’s running in. This is what you want from your test environment as well.
  • Hoverfly, mostly focuses on record and playback functions by acting as a proxy, recorded traffic can be extended and customized programmatically
    Also supports latency simulation for more realistic performance testing
  • Tell a little about Virtualize as a tool
    Both EM and Virtualize engine can be deployed directly from Azure, EM can then provision simulated test environments on the fly
  • Talk them through the picture, focus also on repeatabiity
  • So we’ve decided that SV might be the way to go, but what decisions do we need to make? And of course money does play a role…
  • Where do I host the SV server / the virtual test environments?
  • Who is made responsible for developing and maintaining the virtual assets?
  • Is virtual asset development done in house (requires SV expertise, which, although it isn’t rocket science, might be hard to come by) or is it outsourced to a third party?
    Much like regular software development
  • What could an example maturity journey look like?
    From single server, developed and maintained in one or more teams (small scale) to cloud hosted (flexible, scalable) developed by a center of excellence (less overhead, less fragmentation, one way of working)
  • Studielink
  • ING
  • Deploy and Destroy Complete Test Environments

    1. 1. Deploy and Destroy Complete Test Environments Service Virtualization, Containers and Cloud Bas Dijkstra @_basdijkstra
    2. 2. Continuous Delivery Release Build Test Deploy
    3. 3. Continuous Testing Release Build Test Deploy
    4. 4. Test environment impediments System under test Mainframe SaaS dependency Backend system Mobile app No suitable test data Limited access Under development Access fees
    5. 5. The T-shaped tester: What about the T-shaped test environment? Under control 100% availability Scalable Realistic performance Suitable test data
    6. 6. Service virtualization System under test Virtualized mainframe Virtualized SaaS dependency Virtualized backend system Virtualized mobile app Unrestricted access Unrestricted access Unrestricted access Unrestricted access
    7. 7. Service virtualization _Simulation of dependency behavior _Oblivious to dependency implementation _Virtual assets
    8. 8. Virtual asset creation _Record and playback _Based on specifications
    9. 9. Implementation considerations _Start small _Model just enough _Learn and expand
    10. 10. Testing process benefits _Test earlier _Test more often _Test more
    11. 11. Taking service virtualization to the next level _Continuous Delivery integration _Containerization _Cloud
    12. 12. Test environments on demand _Recreate exact same initial situation every time _Always available _Scalable
    13. 13. Test environments on demand Build server commit Provision virtual test environment Deploy application under test Run tests
    14. 14. Server hardware Host operating system Docker engine Bins / libs Bins / libs App CApp BApp A
    15. 15. Example: Hoverfly (SpectoLabs) _HTTP(S) only _Light-weight and open source _
    16. 16. Example: Parasoft Virtualize _Support for many message types and protocols _Commercially licensed _Environment Manager _
    17. 17. Virtualize + VSTS + Azure commit Parasoft Environment Manager Simulated test environment Application under test AUT Parasoft SV Azure cloud VM Host OS Server On-demand test environment in the cloud
    18. 18. Organizational decisions $ $$$ $$ $ $ $ $ $ $ $ $ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
    19. 19. Infrastructure OR On premise In the cloud
    20. 20. Knowledge OR Inside development teams Center of Excellence
    21. 21. Development OR In-house Outsourced
    22. 22. Example SV maturity journey
    23. 23. Case study: education Infrastructure Knowledge Development
    24. 24. Case study: financial services Infrastructure Knowledge Development
    25. 25. _ Email: _ Blog: _ LinkedIn: _ Twitter: @_basdijkstra