SOA Test Governance: Enabling Service Integration Testing ...


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

SOA Test Governance: Enabling Service Integration Testing ...

  1. 1. IEEE International Conference on Software Testing Verification and Validation Workshops SOA Test Governance: enabling service integration testing across organization and technology borders Antonia Bertolino Andrea Polini ISTI/CNR University of Camerino Via Moruzzi, 1 Via Madonna delle Carceri, 9 56124 Pisa – ITALY 62032 Camerino (MC) – ITALY Abstract SOA development thus consists of a collaborative effort [13] from a chain of parties. The Service-Oriented Architecture (SOA) is the emerging To ensure that the integration of independently developed paradigm in information technology. More than a true pieces of software is successful, an effort in introducing stan- “architecture”, SOA provides a general reference model for dardized notations, interfaces, data coding and more recently the development, deployment and management of distributed also semantics (through ontologies) has been undertaken by dynamic systems. Companies are progressively adopting the SOA companies. However, companies have soon realized service-oriented technology, because of its many (real or that standards alone are not sufficient to ensure interoperabil- idealized) foreseen benefits, among which notably loose cou- ity [9]. The SOA world can be seen as a kind of social world pling and dynamic interoperability. Such benefits, however, in which the services are active entities that operate aside or can only be achieved through discipline and standardization: replace humans. To achieve SOA interoperability, it is then in this respect, SOA Governance qualifies a framework of necessary to put in place some social organization to govern policies, procedures, design rules and documentation stan- the interactions among the participating services, aiming at dards to be enforced for ensuring that different services and assuring that everyone abides by the agreed social rules. The components can successfully cooperate towards a shared governing rules and procedures are established and enforced business goal. What about testing of such composite SOA by super-partes entities, which must be trusted and accepted applications? Little attention has been devoted so far by by anyone. researchers to SOA testing, but awareness is raising that This is what SOA Governance is conceived for. Indeed, existing techniques and tools are not adequate. Our position the apparent flexibility and ease of use of service-oriented in this paper is that the establishment of a test governance application can be only achieved through discipline and an framework is a key issue for enabling SOA testing at enforced framework of rules, policies and processes [15]. So integration and system levels. We discuss the inter-relation far, SOA Governance has been mainly pursued for achieving between technical and “social” aspects of SOA application service integration within one organization. This is too testing, and attempt a first abstract definition of a SOA Test limitative, in our view. In the future vision of services all Governance notion. We provide examples of proposed SOA around us that dynamically connect and disconnect, on our testing approaches that, more or less explicitly, rely on a demand, towards some business objective, SOA Governance cross-organization agreed test process. must be meant as a comprehensive management umbrella under which effective interoperability across organizations 1. Introduction and platforms is ensured. The perspectives opened by Service-oriented computing Service-Oriented Architecture (SOA) is the emerging do not circumvent the need for application testing. The paradigm for the development of composite distributed problem is that the very features that make SOA attrac- systems. It evolved from the distributed Component-based tive, mainly loose coupling and implementation neutrality, approach, by pushing to an extreme the loose coupling also make it more difficult to test [7]. If we consider a among the three stakeholders that collaboratively build a standard testing process, it becomes evident that testing a system. A service developer creates the building blocks, SOA application that is obtained from the composition of or services; a service provider deploys, maintains and sells independently developed services is almost an impossible the services; finally an application developer, or service mission, because most needed information is lacking. There- integrator, composes different services, possibly dynamically fore, we sustain here that as SOA building necessitates the discovered on the network, into a new complex application. establishment of a shared governance framework, also the 978-0-7695-3671-2/09 $25.00 © 2009 IEEE 277 DOI 10.1109/ICSTW.2009.39 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.
  2. 2. effective testing of composite SOA applications requires However, in the com- agreed policies, rules and procedures. mon usage, governance has got to denote a shared partici- A proof of our claim is that in most works on SOA testing, pation and collaborative attitude to the administration of the in order to build their approaches the authors need to make res publica. The term presupposes a social agreement among several starting assumptions beyond the basic principles all stakeholders to abide by the rules in view of the common of the SOA paradigm. For instance in [3] services are and the mutual interests. augmented with formal models suitable for automatic testing Wikipedia1 reports about the moral and natural purpose of generation. Such models are stored within a directory service governance as the process aimed at “assuring, on behalf of and used by third party testers. In [12], instead several those governed, a worthy pattern of good while avoiding an relevant characteristics of service testability are listed. Most undesirable pattern of bad”. In such a definition, beyond the of these characteristics can be assumed only in a collab- technical aspects characterizing the rules and policies, the orative environment. Making such assumptions amounts to term governance carries on a notion of welfare and moral implicitly establishing a test governance. Notwithstanding, obligation. we are not aware of any previous work that explicitly Establishing and maintaining a governance is vital in recognizes and formalizes this notion. ensuring the functioning of any societal endeavor. The In this paper for the first time we thus introduce the notion concept can thus be applied to states, to administrations, to of SOA Test Governance (STG). In particular, in the next associations, and in general to any group of humans engaged section we attempt a definition of the concept. In Section 3 in some purposeful activity. In particular, when applied we discuss why STG is needed, and in Section 4 we set to business organizations, it gives rise to what is called forth a basic outline of a generic STG framework. Then in the Enterprise Governance, aiming at “providing strategic Section 5 we briefly recap two examples of SOA testing direction, ensuring that objectives are achieved, ascertaining approaches that we had earlier developed, highlighting how that risks are managed appropriately and verifying that the these both assume an underlying STG. Conclusions are enterprise’s resources are used responsibly.” [8] briefly drawn in Section 6. Wrapping up, from governance as a generic term, we put our attention more specifically on its associated features of: 2. What is SOA Test Governance • societal participation and shared responsibility; • moral qualification, orientation to the common welfare; In this section we would like to build a definition for • strategic and business-driven target. the concept of SOA Test Governance (STG). STG is a An important subset of enterprise governance for con- relatively new concept which we consider essential to enable temporary organizations is Information Technology (IT) an effective testing process for composite and dynamic Governance, which is that subset of Enterprise Governance service-oriented applications. The novelty of STG stays specifically focusing on the use of IT from the enterprise more on making it explicit as it emerges from the challenges corporate board. This is clearly a broad characterization encountered in SOA testing, rather than in inventing it from which is difficult to formalize, given the pervasiveness of scratch. The concept in fact is grounded on the notion of IT. Webb and coauthors in [14] survey twelve differing def- SOA governance that has become quite hot in recent years. initions of IT governance and eventually propose a unifying In our view, STG stems out naturally as a projection of “definitive” definition as follows: “IT Governance is the SOA governance onto the testing and validation activities, strategic alignment of IT with the business such that max- which are an important part of the SOA life-cycle. To imum business value is achieved through the development introduce a definition for STG, we therefore present first and maintenance of effective IT control and accountability, SOA governance, which in turn is preceded by a completely performance management and risk management.” generic notion of governance. 2.2. SOA Governance 2.1. Governance as a generic term More recently, the very notion of governance has also The term “governance” denotes a set of rules, policies, been translated to social communities made not only by practices and responsibilities by which a complex system humans, but also by computation units, such as web services. of relations and interactions (a country, a company, a so- In fact, in the service-oriented paradigm of development, cial community) is controlled and administered. In politics systems are built by the composition of functional units, the and in the public administration, the word is increasingly services, that may be dynamically discovered and integrated. used nowadays in place of the alternative “government”. Therefore, to ensure that interoperability is achieved in The two terms share the root in the verb to govern and a composite system, whose components are provided by actually are considered as synonymous in some dictio- naries (e.g., see Merriam-Webster, at http://www.merriam- 1. see at 278 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.
  3. 3. independent organizations, it becomes mandatory to enforce An instrument that is used to formally establish mutual some regulations. responsibilities and warranties between a service provider Indeed, while the SOA paradigm is claimed as achieving and a service consumer is the Service Level Agreement distributed system integration cheaper and easier, actually (SLA). In addition to specifying the level of service, in terms this ease of use from the client’s side is counterbalanced by of availability, performance, or other attributes of the service, a quite complex structure from the server’s side. We find rel- the SLA also establishes costs and penalties in case of evant the following sentence from [15]: Counterintuitive as it non compliance. Therefore SLAs, which we do not discuss may seem, SOA requires more organizational discipline than further here, are also integral part of a SOA governance previous development models. Your intuition might tell you framework. that flexibility results from fewer rules, not more, but that’s Risk for governance is that it can be too restrictive and not the case. We get thus to the need for SOA Governance, overwhelming. But if well managed, it really facilitates that consists of the “development and enforcement of SOA interoperability. Governance rules should be conceived and policies and procedures” [15], where in turn a SOA policy implemented so that applying them is easier than breaking consists of a set of design rules combined with enforcement. them. Companies have provided their own slogans, views and white-papers on SOA Governance. Obviously these defi- 2.3. SOA Test Governance nitions on SOA governance coming from industrials put the emphasis on the alignment of business goals with the As emphasized in [10], approaches and tools for assurance architectural and technical features. Thus, for example, Ora- of SOA quality should be embedded into the SOA Gover- cle [1] states that companies should first establish their SOA nance planning. We specifically focus in this paper on testing Roadmap, that is their plan, and then enact the policies and approaches to SOA Quality. procedures that ensure the Roadmap is accomplished. And We attempt here a definition for SOA Test Governance as the HP SOA White Paper [11] states that SOA governance follows: “makes it possible to realize ROI and the business benefits Definition 1 (SOA Test Governance): SOA Test Gover- of loosely coupled services”. nance (STG) is a subset of SOA governance. STG specifi- Nowadays, in business-oriented views, SOA Governance cally concerns the establishment and enforcement of poli- is generally considered at intra-organization level, i.e., it cies, procedures, notations and tools that are required to consists of regulating the SOA development process so that enable distributed testing of service integrations. the organization goals are achieved with the contribution and In previous work [5] we have identified a three-phase interoperation between the various business units within the process for the testing of services: the testing is first carried organization. However, in our view, the most adequate per- out in off-line mode by the service developer; then inter- spective to look at SOA governance is at cross-organization operability with other invoked services can be tested in the level: SOA governance should establish the conditions to real environment after deployment; finally during live usage achieve end-to-end interoperability across different organi- on-line monitoring is activated. In this work the fact is also zations and different platforms. explicitly recognized that web service testing is an activity In such a vision, the establishment of a governance frame- structured around different stakeholders. work also asks for the inclusion of trusted and authoritative The concept of STG makes explicit this awareness into third-parties that govern the relations between participating a framework, and sets the stage for explicitly defining its organizations. It becomes also clear that governance com- constituent components and the respective tasks. STG spans bines the technical aspects with social aspects, for example over all three phases, and sets a social agreement among the conflicts resolution and legal disputes. The stakeholders in stakeholders involved in the three stages, aimed at supporting a SOA governance framework, along with their respective an effective integration testing. Indeed, what strongly char- roles and activities, are illustrated in Table 1. acterizes STG is the specific focus on ensuring smoothness Finally, different key components make SOA governance interoperability when testing for service compositions across at design-time and at run-time [10]. Design-time SOA different organizations. governance applies to the whole service life cycle. It relies heavily on standards: the adopted or created standards make 3. Why do we need SOA Test Governance what is referred to as an Interoperability Framework (IF). To enforce governance at run-time policies and procedures Testing within a SOA context seems to be a profoundly should be embedded into the service management system. different activity with respect to the traditional way of For instance, a central role in SOA governance is played by doing it. In this section we discuss the differences and a registry or a repository. Otherwise, the alternative is to mix the new challenges faced by SOA testers. In particular the rules for governance with services functionality, which pure SOA based applications seem to make testing almost makes services less flexible. impossible. Nevertheless, in our vision, this activity cannot 279 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.
  4. 4. Stakeholder Role and activities Service User This is the organization that accesses and uses a service deployed by a provider. The user has to abide by the established rules when using a service. At the same time it can assume that providers will have to stick to similar rules. Service Developer This is the organization that develops a service, may be as consequence of a request made by a service provider. Governance within a given context could impose development standards, implementation of specific interfaces and so on. Service Provider This represents the organization that deploys the service over a server and makes it available to external organizations. In case of a pay-per-use service it is the provider who gets money. Provider’s main objective is thus to have a service that is highly appreciated by user so to continuously increase its usage. A provider can play the role of user when one of its services need to interact with a service deployed by another provider. Also in this case the governance could impose rules on the way in which a service is provided, for instance requiring that specific information is published. Service Integrator This is the organization that composes a set of services in order to derive a complex application. The integrator is influenced by the governance that can define different rules to follow in the integration Directory Provider This is the organization that provides a directory and discovery service. The directory provider plays a fundamental and strategic role within a SOA infrastructure. It should behave fairly providing references to registered services without any direct economic interest. Being the directory service a highly trusted party it is possible to assign to it specific active governance roles, for instance to check that registered services provide required interfaces for testability purpose. Choreography Specifier Within a specific domain it is possible that the various actors decide to create a body whose mission is to standardise the way in which applications should cooperate in terms of application level interfaces and interactions. This organization will define choreographies specifying the behaviour of services that should participate in a choreography in order to reach a specific goal. This is a special kind of governance “agency” that poses rules on service behaviour to reach specific goals. Trusted Third Parties These are trusted parties providing specific services supporting specific tasks such as authentication, security and so on. All of them operate according to rules defined by the governance and could be deputed to solve specific governance roles. Monitor and Enforcer These are specific organizations that can use mechanisms within a SOA infrastructure to check that what is happening is in-line with what specified by the governance. The governance will have to previously define which are the information of relevance for monitoring and which are the specific powers of the “monitor and enforcer” organization. Governance Board this is the organism that is deputed to define the rules. Typically will be constituted by experts from organizations willing to cooperate or interested in establishing an environment to be used for cooperation. At the same time the governance board will define the various roles and will assign specific tasks to the various roles. Table 1. Stakeholders in a SOA Governance framework be ignored and asks for a mitigated version of the SOA In each given context testers have to understand how vision. This could be acceptable only if a clear governance the systems under test can be better controlled and ob- is established and agreed among the various stakeholders served. After control and observation mechanisms have interacting within a SOA infrastructure. been identified, testers need to select the most suitable and powerful approaches to select the experiments to run (i.e., In general, any software testing is built over two ba- the test cases). Testers also need a way to relate control and sic pillars that are control and observation. By software observability, i.e., the requests with the observed responses. control we refer to the capability, for the software tester, Pragmatically this relation can be derived from knowledge of manipulating the system under test and of interacting in the mind of the testers or, in the luckiest situations, from with it asking to fulfill a given request. So having high software formal model descriptions. control over a system results in the possibility of setting internal variables, changing the source code, simulating and In “traditional testing”, control and observability are fa- changing the execution environment, making invocations to cilitated by the fact that testers yield direct access and are the system, and so on. On the other hand observability refers in power, in time and space, over the software elements, not to the capability of observing how the system reacts and last often including their source code. So the main constraint behaves in response to a request. Therefore high observ- on the shoulders of the tester remains the identification of ability permits to observe changes on internal variable, to relations among control actions and corresponding expected trace the interactions among the elements composing the observations. The situation changed already quite a lot with system, to observe all the interactions with the surrounding the advent of Component-Based (CB) software development. environment made by each single element and so on. The application developer was no more so in power over 280 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.
  5. 5. the parts composing the application under development. increase “openness” of provided services. This is in In particular, some component could evolve independently general not feasible since participants do not trust each (time), often neither source code nor software models were other and in the general case do not even know each available. Nevertheless, even if the components were not other a priori. developed by him/her and no source code was available, it • Reduced information: testing activities are generally was still possible for the application developer to deeply made simpler by the availability of service descriptions analyze the component by implanting it in a fully controlled and models. A typical SOA infrastructure does not environment (space) and manipulating it at their will in order assume such information sharing. to have a better understanding over its behaviour and over • Run-time Integration: within a typical SOA infras- the behaviour of a set of integrated components. So, even in tructure services can discover each other at run-time this more difficult situation, the CB developer could assume starting only then to interact. This makes difficult to that before releasing the application it was possible to test put in place testing integration activities since testing it and any part of it, in an environment over which he/she invocations at run time can result in dangerous side had full control. effects. The advent of the service oriented paradigm further • QoS relevance: since organizations have no full control exacerbated the situation emerged within CB software. In over the environment, relevance of QoS parameters particular a service oriented application is constituted by a result drastically increased. Organizations define and set of services that can fulfill a request by interacting at run- use SLAs to establish offered and expected level of time. In the general case such services belong, are executed quality for service usage. New techniques for QoS and are controlled by different organizations. Moreover for evaluation are needed and QoS parameter can also such organizations services are economical assets to protect affect functional behaviour as discussed in [3]. against third parties. So it is not uncommon that the only The nature of the listed testing related issues, and of information made externally available for a service concerns the specific features of the SOA paradigm, makes it clear the syntactical description of the provided interface via for us that a single organization cannot work alone to a an XML based format. From such description only, the unilateral solution. In order to mitigate the testing obstacles tester has to derive the points of control and observation to we advocate the establishment of cross-organizational STG, send messages and expect responses, no other control and as discussed in the next section. observation points are in general available. So whereas testing a service is for its developer similar to traditional unit testing activities, the situation drastically 4. How can we manage SOA Test Governance changes when the service provider deploys it in the final environment and lets it interact with the “real world.” As SOA development is a collaborative effort [13], and there- in part already listed in [7], key issues that make testing fore every phase, including testing, needs to be redefined a challenging task in the SOA world, in particular with as an activity shared among the involved stakeholders. reference to the integration phase, are: According to our vision, an STG serves the purpose of • Reduced control: within a typical SOA infrastructure a establishing the rules that a set of organizations is willing participating organization has extremely reduced con- to follow in order to make testing activities easier and more trol over services provided by the other participants. effective. Definition of rules and their enforcement can even The only action available is the direct invocation of the include some tasks assigned to a super-partes organization. service itself. There is no possibility of bringing the Such an organization must certainly be trusted by all the service in a given state and there is no possibility of participants given that in general the rules result in the forcing the service to interact with some other services definition of more collaborative environments in which each if not directly provided for in the interface. Another party relaxes the extreme condition of isolation of a typical form of reduced control refers to the evolution of the SOA infrastructure. interacting services that can be modified without alerts In this section we outline a generic governance framework to users. for testing. The aim of this exercise is to open a table of • Reduced observability: within a typical SOA infras- discussion of the main constituent elements of STG rather tructure a service using another service can only ob- then building a precise refined framework. We believe this serve direct responses to requests. No other service should be achieved as a consensus-based result. characteristics can be observed, such as for instance A useful first sketch of a framework for STG can be the lines of code already covered. The perception of obtained by mapping a standard generic testing process the service uses will be much affected. onto the SOA paradigm. We do this by taking as a ref- • Reduced Trust: in order to reduce the consequence erence the multi-layer test process that is currently being of the previous two items participants could decide to defined by ISO/IEC JTC1/SC7 WG26 in Part 2 of ISO/IEC 281 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.
  6. 6. 29119 [16],the new international standard for software test- Layer 3 describes the testing activities compliant with ing. A first draft of the testing process has been recently the organization strategy for a specific project. The project released that includes four layers, namely: test manager makes plans, identifies risks, instantiates the 1) Organizational Test Policy; test strategy, decides on staffing and scheduling. Moreover, 2) Organizational Test Strategy; appropriate mechanisms for monitoring test progress, in 3) Project Test Management Level; order to decide on completion and control that the test 4) Test Levels. plan is adhered must be put in place. In the context of SOA testing, the similitude with a specific project varies The topmost layer is where in an organization the scope depending on if the application is got as an orchestration and objectives of testing, and the regulating policy is de- or as a choreography. In the former case, the role of the cided. It also establishes organizational testing practices project test manager is served by the service orchestrator. and provides a framework for establishing, reviewing and This builds a composite application by aggregating different continually improving the three subsequent layers. available services. Making reference to a standardized and If we try to map such a layer into SOA development, shared test strategy, the service orchestrator decides how to where the possibly established testing policy and rules have develop a specific test plan. The example we develop in then to be fulfilled by the different companies participating Section 5.2 refers to the case of orchestration. In the other in the SOA, it is evident that such layer goes beyond the case, there is not a stakeholder that has central control on the borders of one organization. Probably some international application. Hence the project test management level must association, such as the OSGI Alliance2 or the WS-I Or- be itself embedded into the choreography. One possible STG ganization3 , that represents the interests of a community scenario following this case is proposed in Section 5.1. should be considered to overtake such a role. Currently, there exists no initiative in the SOA world that could resemble Finally, layer 4, Test Levels, regulates how testing is this layer. As already explicit in ISO/IEC 29119, important carried out within the different levels (such as unit, in- activities that such organization should carry on are to tegration, acceptance). In the case of SOA, the different publish and gain consensus on the test policy. Whereas the levels of testing pertain to different actors belonging to cited standard speaks of establishing a testing policy within different organizations. The establishment and adherence an organization, within SOA such activities should be carried to a common shared governance framework can regulate on at cross-organization level, for example the test policy the test activities carried out in independent manner by should be made public and agreed by all those accessing a the individual organizations, so that they all contribute to set of services. In the next section we provide examples for improved SOA effectiveness and reliability. two different realizations of an STG framework, that obey Wrapping up, we schematize in Figure 1 a generic STG two different policies. framework. As shown, we distinguish three stages: Founda- Layer 2 is where the policy is actualized into a strategy. tion, Design and Runtime. Of course, the activities, rules and The activities that constitute the establishment of a test processes performed in the three stages impact and influence strategy correspond one-to-one to those included in layer each other. An instantiation of the framework in a specific 1: the test strategy must be created, agreed, published and domain and context should identify: maintained. However, while a test policy is a high-level • who are the participating actors, such as who is in- statement to provide general directives, the test strategy is volved in the governance board, who is appointed a more concrete and detailed document, which identifies, of monitoring and enforcing, how the developers and among other topics, the type of testing, the techniques providers enter the scene, and so on adopted, tools adopted, Completion criteria (see [16], pag. • which is the policy and the strategy to be followed by 28-29). In the context of an organization, the purpose of a the relevant actors to design, deploy, publish, discover test strategy is to establish the testing practice and testing and run the services guidelines to be used on all projects. Going to a SOA • how and where the testing is documented, ruled and context, a test strategy is the place where an implementation then executed plan of a test policy needs to be established. Organizations • which are the mechanisms and instruments that the that adhere to the test policy must abide by the guidelines STG monitor and enforcer has to fulfill its tasks. and techniques there defined. We speculate that given a Many STG will be established resulting somehow in the generic SOA test policy, community of service developers definition of different SOA infrastructures in which a partner and users could agree on a test strategy for interoperability willing to enter will likely have to agree on the rules defined that fulfills a shared policy. Agreed upon test strategies could by the STG. For instance within a given infrastructure this then be standardized. would mean that the STG requires that a service entering on 2. the scene has to provide an additional interface for testing 3. purpose. 282 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.
  7. 7. Figure 1. STG Framework 5. Some examples of STG approaches Figure 2 illustrates, using a UML sequence diagram, the registration of a service WebServiceUT within the We have been investigating the testing of SOA applica- AuditionEnalbedUDDI registry implementing the Au- tions for some time. Recognizing that SOA testing presup- dition Framework. In order to finalize the registration the poses an STG framework to be made explicit and agreed by directory service uses a TestingEngine service that is the involved parties has taken some time and some matura- able to test a service and to check its conformance to a tion. We have formerly conceived some approaches focusing given specification. During the testing session it is possible mostly on their technical constituents and only afterwards we that the service under test, in order to fulfill a request, have realized that those approaches actually comprehended needs to interact with another service and makes the request (presupposed) an STG instantiation. We report here below to the registry. The registry will pass a reference to the two of our proposed approaches. For space limitation we RequiredService service that is a particular service (a only summarise them, referring interested readers to the proxy or a stub) that is able to check the invocations made by original works, mostly to highlight the STG aspects. the service under test for compliance with the ones expected by a generic implementation of the required service. 5.1. STG and the Audition Framework On a pure SOA based scenario the framework is not ap- plicable. In particular services are not required to “declare” In previous works [6], [4] we have introduced a frame- their behaviour when registering and they are not supposed work called Audition for on-line testing of services. The to be tested. At the same time services involved in a testing very general idea of the framework is to introduce a testing session, since they are required by the service under test, phase when a service asks for registration within a registry. have to be willing of providing this additional feature. In Only services passing the testing phase are guaranteed to particular they have to successively discharge any effect be listed in the registry. As a result services listed within a that is the result of the invocations done during the testing registry implemented according to the framework is expected session. to include only “high-quality” services. The applicability of the framework is based on the defini- 283 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.
  8. 8. Figure 2. Audition Framework tion of a clear governance and of mechanisms within the registered services to participate in testing session. infrastructure that permit to put in place the governance This kind of cooperation permits to reproduce real itself. In particular it is necessary that: scenarios for testing purpose. Nevertheless this result in additional burden for service developers that have to • the service provider registers the service on the registry derive services able to sustain testing sessions. and makes it available for testing. At the same time the • the service registry has to be willing to test services service has to be aware that it will be submitted to a before their actual registration. In this way the registry testing session. At the same time the provider has to be provider sees its relevance even increased within a SOA willing of declaring the behaviour of the registered ser- infrastructure. Trustworthiness on the registry provider vice. This can be done in different way depending also organization is certainly more critical than in a pure on the choices taken by the governance. This choices SOA setting. In this sense it is worth to remember that can refer both to the way in which the specification is within a SOA not visible services are somehow non provided, formal or not, and to where it is stored, and existing. on who is responsible for the specification (for instance • the testing service is a special mechanism needed to the service provider itself or a choreography specifier). assess the service under test for conformance with Finally to apply the framework the governance should respect to a specification. The testing driver can be impose that every message exchanged contains the implemented within the service registry or can even identity of the sender. be a third party organization. This should be the case • the service provider has to be willing of letting its 284 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.
  9. 9. when test cases are automatically from a formal specifi- service oriented architectures. It is called the SOCT (Service cation. In case the service-under-test role is not directly Oriented Coverage Testing) approach. played by the registry it is important that the provider The envisioned testing scenario for SOCT is depicted in organization is trust-able and its role accepted by all Figure 3. A service integrator (on the right) has to test an the providers that could be involved within a SOA orchestration of services, that are made available by one federation. (or more) service provider (on the left). In between these • the choreography specifiers make their specifications two traditional actors, a new stakeholder for SOA testing at available to the registry and to the service providers. In the orchestration-level is introduced. It is a service provider this way a service provider can declare that the service called TCov. In the discussed SOA Governance framework he/she is publishing absolve to a given role within a (Table 1), the TCov provider acts as a trusted third party. choreography. The SOCT approach assumes the following steps (indi- In order to realize the audition framework it is indispens- cated by the callouts in Figure 3): able that a governance is established in order to assign tasks 1) the service provider instruments the provided services and responsibilities to the various stakeholders. Without a so to enable the collection of coverage data, similarly precise agreement among the providers that want to partici- to how instrumentation is normally performed for pate in a given federation the audition framework cannot be traditional white-box testing. We call the instrumented realized. At the same time, after the governance has assigned services Testable Services; the various tasks and responsibilities, it is necessary that the 2) during an on-line testing session the service integrator infrastructure implements specific enforcement mechanisms normally invokes the testable services made available to check that nobody misbehaves. by the service provider; 3) as they are tested, coverage measures are collected 5.2. STG and Service Oriented Coverage Testing from the Testable Services and are sent to TCov; 4) TCov collects, processes and makes available the Another example of a governance framework for SOA required coverage measures to the service integrator. testing makes reference to an approach originally presented The proposed approach fits naturally in the service- by one of the authors in [2]. In that work, an idea aimed oriented paradigm of development as the test information at enabling white box testing of SOA applications was are provided as a service controlled by a service interface. proposed. It is evident that such a scenario of white-box SOA testing is only achieved by means of a governance framework at the orchestration level. We are currently defining and refining the governance policies and rules behind SOCT. Different business models and different flows can be in fact enacted. Two facts are anyhow evident: on one side the realization of SOCT makes SOA applications more reliable and raises their trustworthiness; on the other, only an agreed STG can make it feasible, since only a standardized and agreed setting can make the three-party SOCT scenario interoperable. 6. Conclusions This paper introduced the notion of SOA Test Gover- nance. Even though most proposed approaches on SOA Figure 3. The SOCT Approach testing rely on a set of more or less explicit assumptions, the need for STG has never been formalized before. We Indeed, coverage testing of SOA is somehow a con- defined here STG as a framework of policies, standards, tradiction of the service-oriented constituent principles. In rules, and procedures to enable a collaborative test process SOA development, in fact, only the signature of services across organizations participating into building a SOA appli- (e.g. as WSDL) is generally made available by the service cations. We have discussed why SOA testing requires a more provider, thus preventing the application of white-box cov- disciplined and standardized process than it is currently, and erage criteria (which need access to the source code). We have outlined some basic constituents for STG. have conceived, however, an approach by which services This work provides some first guidelines towards the can be made more transparent to an external tester while establishment of a STG framework. Somehow, following maintaining the flexibility, dynamism and loose coupling of the principle of separation of powers, advocated in the 18th 285 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.
  10. 10. century by Charles de Secondat, Baron de Montesquieu, in [7] Gerardo Canfora and Massimiliano Di Penta. Service- his L’esprit des lois, with reference to state government, also oriented architectures testing: A survey. In ISSSE, pages 78– in a SOA setting we need first to separate the powers and 105, 2008. identify some super-parties entities to whom these should [8] International Federation of Accountants, Pro- be assigned. Accordingly, different SOA infrastructures will fessional Accountants in Business Committee. decide to assign the identified powers to different roles so Enterprise governance: Getting the balance deriving different kinds SOA “societies”. In such societies right. testing and monitoring will be more or less easy to imple- EnterpriseGovernance.pdf, 2004. ment. [9] G.A. Lewis, E. Morris, S. Simanta, and L. Wrage. Why stan- We will continue to work on refining the STG definition dards are not enough to guarantee end-to-end interoperability. and vision since still many concepts needs to be clarified. In Composition-Based Software Systems, 2008. ICCBSS 2008. One aspect that we have just touched here is the relation Seventh International Conference on, pages 164–173, Feb. between STG and SLA validation. Moreover, we intend to 2008. compare our vision with the literature on SOA testing, to [10] David Linthicum. Design&validate soa in a heterogeneous make explicit the relations with proposed approaches and to environment. ZapThink White Paper, WP-0171, 2008. possibly detect further neglected features and requirements for STG. Finally, we hope that this work can foster among [11] Manoj Mansukhani. Service Oriented Architecture, SOA test researchers an interest and a consensus on the basic Hewlett Packard White Paper. on line at wp elements and on the importance of this novel notion. 062005.pdf, 2005. Acknowledgements [12] W. T. Tsai, Jerry Gao, Xiao Wei, and Yinong Chen. Testability of software in service-oriented architecture. In COMPSAC This work was partially supported by the TAS3 Project ’06: Proceedings of the 30th Annual International Computer (EU FP7 IP No. 216287), and by the Italian MIUR Project Software and Applications Conference, pages 163–170, 2006. D-ASAP (Prin 2007). [13] W.T. Tsai. Service-oriented system engineering: a new paradigm. In Service-Oriented System Engineering, 2005. References SOSE 2005. IEEE International Workshop, pages 3–6, Oct. 2005. [1] Mohamad Afshar. SOA Governance: Framework and Best Practices, oracle white paper. on line at [14] Phyl Webb, Carol Pollard, and Gail Ridley. Attempting to define it governance: Wisdom or folly? Hawaii International governance-best-practices.pdf, 2007. Conference on System Sciences, 8:194a, 2006. [2] Cesare Bartolini, Antonia Bertolino, and Eda Marchetti. In- [15] Philippe J. Windley. Soa governance: Rules of the game. on troducing service-oriented coverage testing. In ASE 1st Int. line at, Jannuary 2006. Workshop on Automated engineeRing of Autonomous and run- tiMe evolvIng Systems (ARAMIS 2008), pages 57–64, 2008. [16] Working Group 26 ISO/IEC JTC1/SC7 Software and Systems Engineering committee. ISO/IEC 29119 Software Testing– [3] A. Bertolino, G. De Angelis, L. Frantzen, and A. Polini. Part 2., Model-Based Generation of Testbeds for Web Services. In 2008. Testing of Communicating Systems and Formal Approaches to Software Testing – TESTCOM/FATES 2008, number 5047 in Lecture Notes in Computer Science, pages 266–282, 2008. [4] A. Bertolino, L. Frantzen, A. Polini, and J. Tretmans. Audi- tion of Web Services for Testing Conformance to Open Spec- ified Protocols. In R. Reussner, J. Stafford, and C. Szyperski, editors, Architecting Systems with Trustworthy Components, number 3938 in Lecture Notes in Computer Science, pages 1–25. Springer, 2006. [5] Antonia Bertolino, Guglielmo De Angelis, Lars Frantzen, and Andrea Polini. The plastic framework and tools for testing service-oriented applications. In ISSSE, pages 106–139, 2008. [6] Antonia Bertolino and Andrea Polini. The audition framework for testing web services interoperability. In EUROMICRO ’05: Proceedings of the 31st EUROMICRO Conference on Software Engineering and Advanced Applications, pages 134– 142. IEEE Computer Society, 2005. 286 Authorized licensed use limited to: UNIVERSITA PISA S ANNA. Downloaded on June 9, 2009 at 04:24 from IEEE Xplore. Restrictions apply.