A Ranger4 Guide to Integration Testing and Service Virtualization


Published on

In this Guide from Ranger4 to Integration Testing and Service Virtualization we cover:
1) What Integration Testing and Service Virtualization are
2) Why you should have them
3) What to look for in a vendor
4) Considerations at implementation time
5) Ranger4 recommendations

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

A Ranger4 Guide to Integration Testing and Service Virtualization

  1. 1. A Ranger4 Guide to Integration Testing & Service Virtualization www.ranger4.com © Ranger4 2014 1
  2. 2. Contents 1.0 What is Integration Testing and Service Virtualization? 1.1 Integration Testing, Service Virtualization and DevOps 2.0 Why should you do it? 3.0 What you should look for in a vendor 3.1 Creating virtual components 3.2 Key Capabilities 3.3 A Shared Infrastructure 3.4 Routing 3.5 Questions to ask during your evaluation 4.0 Considerations at implementation time 4.1 Measuring success 4.2 Don’t let developers write the tests on their own code 4.3 Manage requirements: establish an incremental integration plan 4.3.2 Identify services to virtualize 4.4 Techniques to consider 4.4.1 Employ test virtualization 4.4.2 Use continuous system-level testing and asset sharing 4.4.3 Plan for effective data management 4.4.4 Reduce E2E testing and isolate the GUI 4.4.5 Test earlier (shift left) 4.4.6 Avoid the Big Bang 5.0 Ranger4 Recommendations www.ranger4.com © Ranger4 2014 2
  3. 3. 1.0 What is Integration Testing and Service Virtualization? Integration testing is a phase in software testing when individual software modules are aggregated and tested as a group. This phase occurs after unit testing and before validation testing. The inputs to the integration testing phase are the software modules that have been unit tested - they are combined into larger groups tests are applied to the groups as per their definition in an integration test plan. The output is the integrated system ready for system testing. Service virtualization is a method to emulate the behavior of specific components in heterogeneous component-based applications and is used to provide software development and QA/testing teams access to dependent system components that are needed to exercise an application under test (AUT), but are unavailable or difficult-to-access for development and testing purposes. Virtualizing the behaviour of the dependent components means that testing can proceed without needing to access the actual live components. 1.1 Integration Testing, Service Virtualization and DevOps The DevOps movement has been born out of the increasing conflict between IT development and operations teams as a result of businesses’ need to deliver more innovation to their customers via applications and infrastructures of ever-increasing complexity. Where development are all about change, operations are driven to control the stability of the environments they manage and deliver the highest possible uptime to the business - and change puts all of that at risk. With the advent of agile development, and its definition of done, done, done, another new complaint has arisen: developers feel like testers are intruding on their "need for speed” and testers are finding defects in code after the developers have moved on. Integration testing and service virtualization helps development, testing and operations work better together. www.ranger4.com © Ranger4 2014 3
  4. 4. 2.0 Why should you do it? Performing integration testing and service virtualization has a number of high level benefits: - Improved software quality - Increased consistency, efficiency, reliability and predictability in the software development lifecycle - Fewer defects - Reduction in time spent testing - Elimination of bottlenecks - Doing more with less - Increased project delivery capability / reduction of project risk - Reduction in transactions lost due to production outages - Avoidance of additional hardware infrastructure expenses - Less time/cost associated with troubleshooting - Improved collaboration between development, testing and operations teams - Business-value based innovation to market faster - Happier customers - Reduction in spend on third party consulting/testing fees Service virtualization delivers benefits for all types of testing including functional (manual and automated), integration, and performance testing. Service virtualization emulates the behavior of software components to remove dependency constraints on development and testing teams. These constraints occur in complex, interdependent environments when a component connected to the application under test is: ● Not complete ● Still evolving ● Controlled by a third-party or partner ● Available for testing only in limited capacity or at inconvenient times ● Difficult to provision or configure in a test environment ● Needed for simultaneous access by different teams with varied test data setup and other requirements ● Restricted or costly to use for load and performance testing www.ranger4.com © Ranger4 2014 4
  5. 5. 3.0 What you should look for in a vendor Service virtualization solutions have the following characteristics: - Application emulation: Virtual components can simulate the behaviour of an entire application or a specific component - Multiple test environments: Developers and testers can create test environments by using virtual components configured for their needs - Same testing tools: Developers and testers can use the same testing tools that they have traditionally used — the tools don’t know the difference between a real system and a virtual service The virtual components are created to simulate a real environment through two basic entry points: 1) Observing the system in action: Build virtual components by listening to the network traffic of the service that you want to emulate 2) Reading the descriptions of the system: Build virtual components using other sources of information such as service specifications - one example is a Web Services Description Language (WSDL) file, which describes the operations offered by a service along with the parameters it expects and the data it returns When selecting your service virtualization solution it’s essential to recognize the needs of different roles within your business. Not everyone needs or wants to define virtual components, but many testers need to access virtual components in their test environments. 3.1 Creating virtual components The best service virtualization solutions make it easy to construct virtual components that: - Mimic the behavior of the real component providing the service - Respond with realistic data - Process requests within configurable throughput ranges - Can be switched on and off, as the real service becomes available, without having to reconfigure the deployed application www.ranger4.com © Ranger4 2014 5
  6. 6. 3.2 Key Capabilities When you’re looking for a service virtualisation solution for your business, here are some key capabilities you should be looking for: 1. 2. 3. 4. 5. A tool that can create and maintain virtual components A hosting environment for virtual components Testers can configure their environment with virtual components with ease Data extracted from production environments is presented in an easy-to-use way Data is captured during recording in a way that allows for easy creation of a virtualized service 6. Data can be privatized or obfuscated as needed 7. Multiple types of boundary conditions can be tested 8. Data from virtual components can be externalized to allow for easy updating - it will be easy for the author of a new test scenario to add relevant data to a spreadsheet 9. A method for observing and recording messaging conversations 10. A virtual service solution that’s flexible enough to allow you to switch back and forth between the real component and virtual components when testing 11. Virtual services can be developed in a personal environment on a desktop 12. A shared environment for service virtualization can be used across a development team 3.3 A Shared Infrastructure Developers and testers need a shared infrastructure for hosting virtual components and multiple environments need to be maintained in parallel with different mixes of real and virtual components. An environment binds a set of variables from the logical view of your system to specific virtual and physical resources (identified by URLs, host addresses, ports, or other connection settings). By creating multiple environments (for example, developer private, SIT, and UAT), you can run tests against different configurations during each phase of the product life cycle. 3.4 Routing It’s essential to be able to control which traffic is routed to a real service and which is routed to virtual services across environments. After the environment is established, you www.ranger4.com © Ranger4 2014 6
  7. 7. should be able to run all your tests without knowing the difference between real and virtual services. Your chosen service virtualization tool should allow this change in routing without needing any changes to the application code / configuration. You should be able to adjust virtual components throughout. 3.5 Questions to ask during your evaluation 1. 2. 3. 4. How easy to use is the solution? How much and what level of training is required? Can you create virtual components from recordings and / or design specifications? What test types does the solution support? e.g. Manual, automated (integration and functional), and performance test 5. Can you create virtual components that allow you to test different scenarios? e.g happy path, alternative flow, and negative testing 6. How do you create virtual components that enable you to test what-if conditions? 7. Does the solution offer the ability to quickly deploy and manage virtualized services through an administrative console? 8. Can you toggle between live systems and virtual services without having to reconfigure your application deployment? 9. Can the solution manage the complexity of your application environment (ranging from data-driven and correlated response sequences to full stateful database emulation)? 10. Can you automatically schedule and execute tests supported by virtual components upon the availability of a new application build? 11. Can you create, modify, and deploy virtual components without requiring your teams to learn new programming skills? 12. How easy will it be to share and reuse virtual components across teams? 13. Can your teams develop in parallel environments? 14. Will your solution scale to accommodate very large teams as you grow? 15. Does the solution require a long term professional services engagement to help you get started and continue to move forward? Technology constantly changes, so check the software vendor you select is committed to delivering new functionality regularly to ensure the solution remains current - ask to see the product roadmap and question them about their release cycles and software maintenance program. To help speed your adoption process and keep your productivity levels up, make sure your vendor provides excellent support with access to trained technicians - ask for a reference call and check with another customer that this is the case. www.ranger4.com © Ranger4 2014 7
  8. 8. 4.0 Considerations at implementation time The evolution of applications is accelerating. Applications are not discrete islands but build on complex, interconnected sets of services including disparate technologies, developers, deployment topologies and organizations. Developers are directed to deliver high-quality applications while testing expenses are often limited. A combination of automated integration testing and test virtualization can help test teams to improve quality and keep up with the rate of change. 4.1 Measuring success A simple measure of success for a test manager is the ratio of captured defects versus escaped defects. However, success or failure is not simply determined by the number of defects that have escaped into production. Categorization of defects to determine where the defect should have been found can dramatically reveal the efficiency or inefficiency of your testing. For example, if a functional defect is found during end-to-end system testing, the costs of remediation would far exceed the costs of fixing the defect as it was introduced in an earlier development phase. The increased costs would be due to factors such as: more regression testing, more test resources, usage of more live-like environments and greater requirement for coordination. 4.2 Don’t let developers write the tests on their own code Having developers write their own testing simulations or stubs to validate their own code changes or new code is a bad idea - analogous to having a student mark his own homework or exam. Testers, who know how the functionality should be tested and the data required to properly test the various scenarios, can easily create virtual components and tests, making them available to the development team. 4.3 Manage requirements: establish an incremental integration plan Accurate and unambiguous requirements are particularly vital for integration testing. Business processes must be defined and broken down into their constituent parts so that corresponding testing requirements can be defined at the most granular level of service interaction. Requirements coverage is thus tracked at: - A functional level - A service interaction level www.ranger4.com © Ranger4 2014 8
  9. 9. - The business process level - At full system level When granular requirements are fully visible, project leads and test managers can more easily collaborate to plan an effective and controlled release schedule. Gradually controlling the introduction of functions and components into test environments makes it faster and easier to isolate faults and defects. 4.3.2 Identify services to virtualize You need to identify the right services to virtualize - start by bringing together the key development stakeholders involved in the application life cycle and find out where your organization experiences the most testing pain. Then ask these questions: 1. 2. 3. 4. 5. 6. 7. Do you have all the environments you need for integration testing? Are these test environments available to teams throughout the development cycle? Do you experience downtime due to unavailable test environments? How often does downtime occur? How long do your teams usually have to wait? What’s the impact on time and cost due to testing downtime? Does your application interface with third-party services? Do you need to pay for and schedule access to these third-party interfaces prior to scheduling your tests? How much does this cost? 8. Who controls the information needed for creating test environments? 9. Do individuals or teams conflict with each other when scheduling the sharing of test environments (or parts of a test environment)? The answers to these questions should enable you to identify a prioritised area where you can start your project. 4.4 Techniques to consider After establishing an incremental integration plan for the project, test professionals should consider the following techniques to help achieve a more proactive test procedure: 4.4.1 Employ test virtualization In test virtualization, reals component are replaced by virtual components (“stubs”). These virtual components can be used to model and simulate real system behaviour. Virtualizing the environment helps to eliminate application test dependencies and reduces the setup and infrastructure costs of traditional testing environments. A proactive test plan should use test www.ranger4.com © Ranger4 2014 9
  10. 10. virtualization to make virtual components available for key service components, allowing situations to be simulated and tested more easily. 4.4.2 Use continuous system-level testing and asset sharing Running full regression cycles whenever new or virtualized components are introduced provides an immediate feedback loop to the development team who can run the exact same scripts, replicate and resolve issues with minimal effort. This promotes innovation rather than remediation. Testing tools encourage developers and test teams to collaborate by sharing integration tests and virtualized services throughout the software development lifecycle (DevOps). 4.4.3 Plan for effective data management Representative, appropriate data is needed in order to support the necessary test coverage and achieve the appropriate level of confidence in a delivered solution. It’s recommended that data considerations begin at the requirements stage and are factored into test creation and execution. Given limited timescales and budgets it is usually necessary to use equivalence partitioning or boundary analysis to identify the data that is absolutely essential to the project. Test data management is such an important activity that it is often assigned to dedicated individuals. 4.4.4 Reduce E2E testing and isolate the GUI Since integration testing is incremental, E2E testing takes on less significance. When a proactive test approach is implemented, far less time is spent performing costly end-to-end testing - by the time full end-to-end functionality has been created in the test environment, the functional, integration and business-process-level tests will have been executed many times. Incremental and continuous testing will have removed many of the risks of the integrated solution. It’s recommended that end-to-end testing is focused on driving through the E2E processes via the various GUI front ends. Implementing automated testing that occurs at the service layer and bypasses the GUI makes it faster to create and execute tests and delivers a more robust process than using GUI-driven automated scripts. It’s recommended that GUI components are isolated in environments where they interface with virtualized services - this ensures a high rate of test execution. The GUI components can then be periodically introduced, validated and withdrawn through the project life cycle (they should only be formally introduced after completion of all other integration testing). Combining automation at multiple isolated layers of an application and a more predictable www.ranger4.com © Ranger4 2014 10
  11. 11. execution environment for GUI tests helps mitigate the traditional challenges of a GUI-only approach to testing. 4.4.5 Test earlier (shift left) The above considerations are all undertaken in order to test more effectively at an earlier point in the software development lifecycle. It is widely accepted that testing and defect resolution are more expensive if undertaken at the latter stages of integration. 4.4.6 Avoid the Big Bang A traditional “Big Bang” test method brings all integration points together for end-to-end testing. With this test method, there is a sudden threshold at which it becomes possible to run many more test cases. The increase in the number of cases causes a sudden drop in the percentage that are tested and passed. With Big Bang testing, a huge proportion of the project’s risk has been deferred until late in the development cycle when all of the components are available. This profile needs to be reversed by addressing integration risk earlier and more continuously (shifting left). www.ranger4.com © Ranger4 2014 11
  12. 12. 5.0 Ranger4 Recommendations Ranger4 recommend’s IBM’s solutions for integration testing and service virtualization (formerly known as GreenHat): ● Rational Test Workbench: Delivers a comprehensive test automation solution including regression, load, integration, and performance testing across web, mobile, and traditional applications. By providing an end-to-end solution in a single offering, Rational Test Workbench allows teams to significantly improve team agility, maximize productivity, and deliver software at the pace the business demands. ● Rational Test Virtualization Server: Enables the deployment of virtual services in order to simulate real system behavior so teams can perform integration testing earlier and more frequently in the development cycle. Virtual services can be reused and shared easily across teams to significantly reduce the expense of creating and maintaining test environments. ● Rational Performance Test Server: Provides application and integration level load generation to provide a complete view of performance and scalability across all application components. This workload test engine maximizes the use of your test infrastructure to quickly deploy load scenarios and create large-scale system tests to avoid the costly impact of poor application performance. Gartner rated IBM as a Leader in their 2013 Magic Quadrant for Integrated Quality Suites, identifying them as having one of the broadest portfolios in the testing tool market with coverage not just for core test management and test automation, but also static analysis (both code quality and security), unit testing, and a comprehensive solution for test data. Ranger4 are ideally placed to help you evaluate IBM’s integration test and service virtualization suite and well-versed in helping our customers write business cases to justify their software investments. We often find that testing is one solution area of several that will help an organisation benefit from a full-blown DevOps implementation and have developed our DevOps Maturity Assessment which baselines and scores an organization’s current capabilities on our DevOps Maturity Index and provides a fit-assessed roadmap to a desired future state. To find out more, please visit www.ranger4.com. www.ranger4.com © Ranger4 2014 12