Addressing the Challenges of Testing SOA based Applications Neil Pandit 19 th March 2008

Agenda What is SOA? An Example SOA Implementation - On-Line Loan System Business Risks and Benefits Main Challenges Addressing the Challenges - SOA Test Strategy Test Organisation Changes Approach by Testing Types and Relevant Tools Summary

What is SOA?

An Example SOA Implementation - On-Line Loan System

Business Risks and Benefits

Main Challenges

Addressing the Challenges - SOA Test Strategy

Test Organisation Changes

Approach by Testing Types and Relevant Tools

Summary

What is SOA? SOA is an Architectural style Modular Shareable Distributable Defined interfaces It is about the design of the business Focus on Business Processes / Business Driven It is more than Web services!

SOA is an Architectural style

Modular

Shareable

Distributable

Defined interfaces

It is about the design of the business

Focus on Business Processes / Business Driven

It is more than Web services!

An Example SOA Implementation – On-Line Loan System Customer User Business Process Customer DB Web Services 3 rd party Credit Check Financial Adviser Login Credit check loan quote Quotes Customer Validation loan quote Credit check Mainframe Legacy / 3 rd Party Internet Internet

Business Risks and Benefits Risks Increased costs Early joiner – when there is little knowledge No business commitment and resource available No overall picture of business processes and ability to prioritise Quality – single point of failure The industry decides that the future isn’t SOA after all! Benefits Reuse of code Reduced IT spend Faster time to market

Risks

Increased costs

Early joiner – when there is little knowledge

No business commitment and resource available

No overall picture of business processes and ability to prioritise

Quality – single point of failure

The industry decides that the future isn’t SOA after all!

Benefits

Reuse of code

Reduced IT spend

Faster time to market

Main Challenges Scope and boundaries Changes to composition of test team Increased knowledge required of testers Technical knowledge (WSDL) Domain knowledge Use of Tools Governance and the need for standards Increased focus on negative and non-functional testing

Scope and boundaries

Changes to composition of test team

Increased knowledge required of testers

Technical knowledge (WSDL)

Domain knowledge

Use of Tools

Governance and the need for standards

Increased focus on negative and non-functional testing

Addressing the Challenges - SOA Test Strategy Test Design should follow a Top down Approach Test Execution should follow a Bottom up Approach starting at the individual service level. Current testing methodologies should be extended to support the use of services in an SOA solution.

Test Design should follow a Top down Approach

Test Execution should follow a Bottom up Approach starting at the individual service level.

Current testing methodologies should be extended to support the use of services in an SOA solution.

Test Organisation

Functional and Regression Testing Approach Need for more concentration on negative testing Impact on regression testing Backward compatibility testing Service quality becomes more important as reuse will require a level of quality to be achieved Tools Parasoft SOAtest QTP Green Hat GH Tester Borland SilkPerformer SOA

Approach

Need for more concentration on negative testing

Impact on regression testing

Backward compatibility testing

Service quality becomes more important as reuse will require a level of quality to be achieved

Tools

Parasoft SOAtest

QTP

Green Hat GH Tester

Borland SilkPerformer SOA

Performance Approach Service must be fully performance tested Volume Stress Load Targeted infrastructure tests Tools Loadrunner 9.1 Parasoft SOAtest Green Hat GH Tester Borland SilkPerformer SOA

Approach

Service must be fully performance tested

Volume

Stress

Load

Targeted infrastructure tests

Tools

Loadrunner 9.1

Parasoft SOAtest

Green Hat GH Tester

Borland SilkPerformer SOA

Security Approach Testing at the service level and not at the final stage Performing a security risk assessment No assumptions about 3 rd party development Penetration tests Need to ensure data security is handled appropriately as per industry regulations Testing of changes to Security Policies Tools Vordel Soapbox Parasoft SOAtest AppScan

Approach

Testing at the service level and not at the final stage

Performing a security risk assessment

No assumptions about 3 rd party development

Penetration tests

Need to ensure data security is handled appropriately as per industry regulations

Testing of changes to Security Policies

Tools

Vordel Soapbox

Parasoft SOAtest

AppScan

Integration Testing Approach The need for an Integration Test Strategy Validate Functioning of Interfaces and Data Sharing between Services Creation of WSDL Definitions and testing against these Use of Regression Tests Tools Parasoft SOAtest Green Hat GH Tester Borland SilkPerformer SOA QTP

Approach

The need for an Integration Test Strategy

Validate Functioning of Interfaces and Data Sharing between Services

Creation of WSDL Definitions and testing against these

Use of Regression Tests

Tools

Parasoft SOAtest

Green Hat GH Tester

Borland SilkPerformer SOA

QTP

Interoperability Approach Services need to adhere to WS-I (Web Services Interoperability) WS-I Basic Profile and Security Profile WS-I site http://www.ws-i.org/ Ensure that Governance supports adherence to standards Tools Parasoft SOAtest Borland SilkPerformer SOA

Approach

Services need to adhere to WS-I (Web Services Interoperability)

WS-I Basic Profile and Security Profile

WS-I site http://www.ws-i.org/

Ensure that Governance supports adherence to standards

Tools

Parasoft SOAtest

Borland SilkPerformer SOA

Summary SOA is here to stay Risks and Benefits must be understood SOA will change the organisational testing methodology Testers’ skills will need to change Change in testing approach - Non-Functional Testing, Interoperability Increased development and use of tools Questions? Email [email_address]

SOA is here to stay

Risks and Benefits must be understood

SOA will change the organisational testing methodology

Testers’ skills will need to change

Change in testing approach - Non-Functional Testing, Interoperability

Increased development and use of tools

Questions?

Email [email_address]