The document summarizes experiments on different architectures for executing event-driven mashups. It finds that a BPEL-based solution is not suitable, a JSLEE implementation needs improvement, a plain Java solution showed good performance, and a fully distributed solution is best for high performance. Key metrics like requests per second and latency are reported for different architectures under different traffic loads.

1. Experiments and Performance Evaluation of Event Driven Mashups Sousse July, 5th 2009 Michele Stecca [email_address] CIPI – University of Padova and Genova (Italy)

2. Outline The OPUCE project Mashup definition Description of the Mashup execution architectures Performance tests Test setup Results Conclusions

3. The OPUCE project O pen P latform for U ser-centric service C reation and E xecution (FP6 IP Nov. ‘06 – Apr ‘09) User Generated Services Main components of the platform Service Creation Environment (SCE) Service Lifecycle Manager (SLM) Service Execution Environment (SEE) Portal

4. Event Driven Mashup Definition Base Services (Basic blocks) Service Compositions (Mashups) Web Service communications between Base Services 4 Different SEE Implementations

5. BPEL-based solution Mashup execution based on a BPEL engine BPEL is not suitable for event driven applications Definition of a “standard” BPEL script structure BSs Container Orchestrator on BPEL Engine BS_1 BS_n BSs Container BS_1 BS_n

6. JSLEE-based solution Sessions, scalability and asynchronous event management provided by the Mobicents JSLEE Container JSLEE stands for JAIN Service Logic Execution Environment BSs Container Orchestrator on JSLEE container BS_1 BS_n BSs Container BS_1 BS_n

7. Plain JAVA solution Stateless Orchestration Ad-hoc Orchestrator lighter than BPEL engine: only event routing Scalability thanks to Orchestrator replication Orchestrator Management System BSs Container Orchestrator (O_Id=1) BS_1 BS_n BSs Container BS_1 BS_n Orchestrator (O_Id=m)

8. Fully Distributed Solution Stateless Orchestration No centralized Orchestrator: intrinsic scalability SEE1.1: fully distributed solution BaseService BaseService BaseService BaseService BaseService Orchestration Proxy BS SC BS SC BS SC BS SC BS SC BS SC = Base Service superclass OPUCE Service Trigger or OPUCE Portal 1. notifyEvent [starting a session] 2. first base service is invoked with event 3. only error and final events are routed back to the OrchestrationProxy Distributed execution network

9. Test Setup Traffic generated by SoapUI at different rates (50 – 100 – 200 reqs/sec) Important: BPEL-based solution crashes as soon as the traffic rate goes beyond 10 reqs/sec traffic generator orchestrator bs „SleepA“ Software test setup Start SleepA SleepB SleepA SleepB SleepA End bs „SleepB“ SleepB SleepA SleepA SleepB T1 T2 Hardware test setup

10. Test Results & Conclusions Which one? BPEL-solution is not suitable for event driven applications The specific implementation of the JSLEE standard should be improved The Plain JAVA solution shows good performance and complete control The fully distributed solution is better when high performance is required 166 5,20 91,03 5,11 48,44 3,34 Distributed 51,85 226,12 53,21 173,91 48,15 9,53 JSLEE 161 8,08 91,06 5,87 48,58 5,70 Plain JAVA T(reqs/sec) L(ms) T(reqs/sec) L(ms) T(reqs/sec) L(ms) Config_3 (200 reqs/sec) Config_2 (100 reqs/sec) Config_1 (50 reqs/sec)

11. Questions & Answers Thank you!