Everybody is doing some sort of CD, or at least is looking into it
CD requires the ability to test continuously, every deployment should be thoroughly tested before being put into a production environment Test automation plays a big role in this
The ability to test continuously is often hard to achieve because of test environment impediments
The T-shaped tester is a sought after profile at the moment, very good in one thing (testing, automation) and skilled in myriad other things (analysis, development, …) Continuous Testing requires something similar: a T-shaped test environment. First and foremost it should be under total control of the team using it, but next to that it should have other characteristics as well..
One approach to enable Continuous Testing is through implementation of service virtualization, an approach that removes bottlenecks in test environments by virtualizing dependencies that are critical to the testing process yet hard-to-access
Characteristics of service virtualization
R&P: draw parallel to test automation: nice to get a first demo up and running (or analyse traffic where no specifications are known), but to achieve long term success it’s best not to rely on R&P too much. Instead, create virtual assets using request-response specifications. R&P also only simulated those situations that CAN be recorded.
How should you go about SV implementation? Start small, both in terms of number of dependencies simulated and behavior replicated. Model just enough to enable execution of desired test cases (all else is filler, remember that SV is an enabler for testing, not a goal in itself). Learn and expand.
So how does the testing process ultimately benefit from SV implementation?
So, we have seen that implementing SV can lead to great benefits for the testing process, but to make SV even more flexible and better integrated into the CD process, wouldn’t it be nice if we could treat our test environment in the same manner as our application under test?
By containerizing and distributing SV engines and their virtual assets, we can do…
So, what does an approach like this look like? Talk them through the picture. Build server also runs tests (unit tests), we’re talking about integration and end-to-end tests here.
Guarantees that an application will always run in exactly the same way, regardless of the environment it’s running in. This is what you want from your test environment as well.
Hoverfly, mostly focuses on record and playback functions by acting as a proxy, recorded traffic can be extended and customized programmatically Also supports latency simulation for more realistic performance testing
Tell a little about Virtualize as a tool Both EM and Virtualize engine can be deployed directly from Azure, EM can then provision simulated test environments on the fly
Talk them through the picture, focus also on repeatabiity
So we’ve decided that SV might be the way to go, but what decisions do we need to make? And of course money does play a role…
Where do I host the SV server / the virtual test environments?
Who is made responsible for developing and maintaining the virtual assets?
Is virtual asset development done in house (requires SV expertise, which, although it isn’t rocket science, might be hard to come by) or is it outsourced to a third party? Much like regular software development
What could an example maturity journey look like? From single server, developed and maintained in one or more teams (small scale) to cloud hosted (flexible, scalable) developed by a center of excellence (less overhead, less fragmentation, one way of working)
Deploy and Destroy Complete Test Environments
Deploy and Destroy Complete
Service Virtualization, Containers and Cloud
Test environment impediments
System under test
No suitable test data Limited access
Under development Access fees
The T-shaped tester: What about
the T-shaped test environment?
System under test
_Simulation of dependency behavior
_Oblivious to dependency implementation
Virtual asset creation
_Record and playback
_Based on specifications
_Model just enough
_Learn and expand
Testing process benefits
_Test more often
Taking service virtualization
to the next level
_Continuous Delivery integration
Test environments on demand
_Recreate exact same initial situation every time
Test environments on demand
Host operating system
Bins / libs Bins / libs
App CApp BApp A
Example: Hoverfly (SpectoLabs)
_Light-weight and open source
Example: Parasoft Virtualize
_Support for many message types and protocols
Virtualize + VSTS + Azure
Simulated test environment
Application under test
Azure cloud VM