EuroSTAR Software Testing Conference 2008 presentation on Test Patterns: A New Concept For Testing by Henk Doornbos & Rix Groenboom. See more at conferences.eurostarsoftwaretesting.com/past-presentations/

1. Test Patterns
-- a new concept for testing --
Henk Doornbos
Rix Groenboom
Research sponsored by

2. Overview
• Testing challenge
• Current testing techniques
• New approaches
• Towards Test Patterns
• Conclusions

3. Testing is Difficult
• SOA in particular
– Heterogeneous interfaces: Web, Gui, WS, JMS, JDBC
– Many dependencies
– Many re-configurations
• SOA
– Mesh:
– Mash up:
– Mess: Spaghetti oriented architecture

4. Testing is Difficult
• Architecture is heterogeneous
J
U
B
E
S
ERSB2B
B2BG
B2BG
BPEL EBS
IPEX
ebXML / ebMS AQ AQ DB link GUI
TestTool
Advanced Queues
CJIB
VIP
JDS
HTTP / SOAP

5. Testing is Difficult
• Many different interfaces:
J
U
B
E
S
ERSB2B
B2BG
B2BG
BPEL EBS
IPEX
ebXML / ebMS AQ AQ DB link GUI
TestTool
Advanced Queues
CJIB
VIP
JDS
HTTP / SOAP

6. Testing is Difficult
• Tester performs manual steps, eg:
– Enter user-data via website (with IE)
– Check database status (with TOAD)
– Submit XML message (with SOAPui)
– Check database status (with TOAD)
• Therefore, current process is tedious:
– hence error propone
– Not repeatable

7. Current testing tools
• Currently, there is a portfolio of testing tools with a
focus on automation:
– Test management tools
– Individual tools for testing (GUI, Web, Load, DB)
• Main problem:
– Semantic gap, the tester is the “glue” between the
different tools

8. Current testing tools
• Example: TestFrame
– Excel based technology
– Separation between logical and physical test cases
– Ad-hoc engine implementation for test automation
– No clear semantics

9. Current testing tools

10. Current testing tools
• Example: SOAtest
– Platform for testing SOA implementations
– Supports of multiple protocols (HTTP / JDBC etc)
– Test flow over multiple interfaces is possible
– Example CRUD operations:
• Search service call
• Retrieve service call
• Update service call
• Validate Database

11. Current testing tools

12. Towards solution
• More powerful tools allow automation of testing
often for single interfaces
– Solves the technical interaction and provides repeatability
• We still need an integrated framework to drive the
testing and to process the results (orchestration)
– Provides the flexibility to handle overall architecture

13. Towards solution
• IDEA 1: Use structure of architecture for testing
– Architecture is most stable part of applications
– Clearly defined interfaces
Database
Server
Application
Server
Legacy
Presentation
Layer
Web
Services
Application
Logic Thin
Client
Web
Site

14. Towards solution
• IDEA 2: Regard testing as a business process
– Make basic steps executable: call the relevant testing
tools using a web-service interface
– BPEL for Test Orchestration

15. BPEL: Advantages
• BPEL:
– Web standard
– Direct execution of webservices
– Programming language, so you can define patterns
(recurring structures)
– There are semantics so you can reason about processes
• BPEL for testing:
– Provide test-tool with web-service interface, so we can
communicate with them directly from BPEL

16. BPEL: Example
• Drive SOAtest from BPEL

17. BPEL: Example
• Example integration: SOAtest & WebKing

18. BPEL: Provides also semantics
• Requires mathematical solution :
– Parallelism, non deterministic, asynchronous
• Testing will have mathematic laws:
– T1 + T2 = T2 + T1
– T1 || T2 = T2 || T1
– T1 + ( T2 || T3 ) = (T1 || T2) + (T1 || T3)
• Use process algebra like CSP:
– Provides a framework to compose operations

19. Example pattern (0)
• CRUD operations
– CREATE
– READ
– UPDATE
– DELETE

20. Example pattern (1)
• Standard ESB structure:
– Basic services
– Integrated services
– Business services
– User layer: GUI
• Which test pattern ?

21. Example pattern (2)
• Diagram:

22. Example pattern (1)
• Proposed test pattern:
– Unit testing of basic services
– Integration test: no new logic, therefore nothing to test
– Business process testing: Web submission

23. Example pattern (2)
• Diagram:

24. Example pattern (2)
• Database application with WS front-end
– B2B communication
– Legacy backend communication (mailboxes)

25. Example pattern (2)
• Diagram:

26. Example pattern (2)
• Special Web-services implementation for
encapsulation of legacy protocol

27. Conclusions
• Testing of SOA is becoming increasingly complex
– Requires more mathematical approach
• BPEL can be used to model the test process
– Higher level of abstraction
– Better defined semantics
• Will lead to a library of test-patterns:
– Using proper automation, this gives reuse of test effort

28. About us
• Windesheim:
– University of Applied Sciences
– www.windesheim.nl
• Parasoft:
– Provider of software test solutions for SOA and BPEL
– www.parasoft.com
• SIOG:
– St. ICT Onderzoek & Ontwikkeling Groningen
– Knowledge valorization
– Joint projects Groningen, Oldenburg, Bremen