Scalable SOA
Samisa Abeysinghe
WSO2 Summer School 2009

Outline
●Introduction
●Clustering
●Load Balancing
●SOA in the cloud
●Summary and Resources

Outline - 1
Introduction
●
● Scalability and Availability

Outline - 2
Clustering
●
● Clustering Techniques
● Interfaces to be used
● Membership Schemes
● State Management
● Node Management
● Group Management
● Clustering Patterns

Outline - 3
Load Balancing
●
● Stateless load balancing
● Stateful load balancing
● Load Balancing with fail over
● Dynamic load balancing

Outline - 4
SOA in the cloud
●
● AMI deployments
● Auto Scaling

Clustering

Web services clustering
● Different servers
providing the same
service
● Group communication
among the servers
● Shared resources for
the cluster

Motivation for Clustering
●Achieving high availability and scalability
●Enterprise Service Buses (ESB)
● De-facto standard for integrating autonomous entities
● Could become the bottleneck in the system
● High availability and high scalability critical
● Ability to cluster is an essential functionality for an
ESB in a SOA deployment
Service hosting engines
●
● Needs to be able to plug into clusters

Scalability
Ability to
Handle growing amounts of work in a
graceful manner
Or
Be readily enlarged

Availability
The degree to which a system, subsystem, or
●
equipment is
● Operable and in a committable state at the
start of a mission
● When the mission is called for at an
unknown

A Computer Cluster is...
●A group of coupled computers that works closely
together
●As if they were a single computer

Clustering Basics
Members or nodes of a cluster
●
● Connected to each other through fast local area
networks
● Redundant hardware and/or software resources
● Increase the availability and/or performance of the whole
system
Fault-tolerant components
●
● Share any existing state information among members
● Redundant component is able to take over the work
of another

An ESB Cluster

Performance with a Cluster
●Install the software on different computers in your
network
●Spread work amongst those
●Achieved using software or hardware load
balancers
● Distribute incoming requests to different nodes of a
cluster
● Based on a very simple or sophisticated
strategy/algorithm
● Most common and the very basic strategy is round-robin

Performance with a Cluster:
Scaling
Increasing the numbers of nodes in a cluster
●
● Decrease the average load on each node
● Able to handle more work in a given time period
● Scale to your performance requirements

Scaling with a Cluster

Availability with a Cluster
●Fault-tolerance and failover
●Component failures detected by the cluster
●Repeats the failed operation on an identical node
residing in the cluster
● Request processed should be completely stateless
● Does not work if an operation is stateful
● The node is required to maintain state information between
subsequent invocations
● Stateful component needs to communicate its state
information across all identical components within the
cluster - State replication

Availability with a Cluster

Clustering with Apache Axis2
●The Web services engine has comprehensive
SOA support
●Many other projects make use of Axis2
clustering
● Apache Synapse
● WSO2 Carbon – The SOA Platform

Apache Axis2 Clustering
●Functionality is abstracted out by a set of
interfaces
●Makes clustering pluggable
● Implement provided interfaces
● Configure implementation classes in the clustering
configuration section of axis2.xml
Default clustering implementation in Axis2
●
● Based on Apache Tribes
● Group management framework used by Apache Tomcat

Axis2 Clustering API
Address 2 concerns
●
● State replication
● Synchronizing of states related to different members in
a cluster group
● Management
● Group Management
● Node Management

Axis2 Clustering Interfaces
●org.apache.axis2.clustering.ClusteringAgent
●org.apache.axis2.clustering.state.StateManager
●org.apache.axis2.clustering.management.NodeM
anager
●org.apache.axis2.clustering.management.Group
ManagementAgent

org.apache.axis2.clustering.Clus
teringAgent
Responsible for initializing all clustering related
●
functionalities of a node
● Also responsible for getting the node to join the
cluster
● Node should not process any Web services requests
until it successfully joins the cluster
● Node will also need to obtain state information and/or
configuration information from a neighboring node
● To be in sync with the other members
● Configuration changes might have taken place

org.apache.axis2.clustering.Cl
usteringAgent
In axis2.xml, the instance of this interface is
●
specified using the "clustering" class attribute.

org.apache.axis2.clustering.stat
e.StateManager
●Responsible for handling state replication
●Property changes (state changes) in the Axis2
context hierarchy in the node which runs this
StateManager will be propagated to all other
nodes in its group
●Generally, Web services authors do not have to
handle state replication, as it is handled by the
clustering implementation just before request
completion.

org.apache.axis2.clustering.sta
te.StateManager
●org.apache.axis2.clustering.state.Replicator#repli
cate() can be called to force state replication.
●It is not mandatory to have a StateManager in a
node
● If we are not interested in high availability, the
purpose of a cluster is to achieve scalability
● Simply to utilize the group communication capabilities
of the underlying Group Communication Framework
(GCF)
● The purpose of a clustered deployment may also be
management of the cluster using the underlying GCF
● WSO2 Carbon provides such a component which allows
cluster management.

org.apache.axis2.clustering.sta
te.StateManager
The implementation of this interface is set by
●
reading the "stateManager" element in the
axis2.xml clustering section.

org.apache.axis2.clustering.man
agement.NodeManager
●Responsible for handling management of a
particular member
●Not mandatory to have a NodeManager in a node
●Node management is generally used for
deploying or undeploying services across a
cluster group
●The implementation of this interface is set by
reading the "nodeManager" element in the
axis2.xml

org.apache.axis2.clustering.ma
nagement.GroupManagementA
gent
●Responsible for sending group management
events and messages to members in a
particular group
●Only used when a member is running in group
management mode

Group Management Mode
●Member is special and belongs to all groups
which it is managing
●Any membership changes will be notified by the
underlying group management framework to
the group management agent
●A group management agent should be specified
for each group that has to be managed by the
member
●In WSO2 Carbon cluster management, the
cluster manager node needs to define group
management agents for each group

Membership Schemes
●Static
●Dynamic
●Hybrid

Static Membership
●Only a defined set of members can be in a
group
●Group Membership Service(GMS) will detect
members joining or leaving the group
●External members cannot join the group
●Each node may obtain group member details
from a central repository or configuration file

Dynamic Membership
●Membership is not predefined
●Members can join a group by specifying the
appropriate group name, and also leave the
group
●Group Management Service (GMS) will detect
new members joining or leaving
●Group membership information may be obtained
from the GMS

Hybrid Membership
There are a set of well-known members
●
● Can consider these members as belonging to a static
group
External members can join this group by notifying
●
one of the well known members
● These external members can then get the current
group membership from this well-known member
●When new members join the group, well-known
members will notify all other members
●When members leave the group, the GMS can
detect this event

Load Balancing

Dynamic Load Balancing with
ESB
●New members can join and leave the application
group
●This will be reflected in the membership aware
dynamic load balancer
●The clustering configuration for the dynamic load
balancer needs to be configured with group
management to allow group membership
discovery

Apache Synapse/WSO2 ESB
Config Sample LB endpoint
<endpoint name=”test-lb”>
<session type=”soap”/>
<loadbalance policy=”roundRobin”>
<endpoint ..../>
..
..
</loadbalance>
</endpoint>

Architecture
● Stateless load balancing
● LB algorithm implementation is pluggable
● Built-in round robin algorithm
● Can configure more advanced algorithms
● e.g. Weighted round robin LB algorithm

State-full Session Aware Load
Balancing
● Transport and SOAP session
Load Balancer

Fail-Over with LB
Load Balancer

LB and FO Groups
Load Balancer

Dynamic LB
Load Balancer

Dynamic LB (Cntd..)
Load Balancer

Scalability of WSO2 ESB
● Can handle 2500 concurrent connections
● Can handle 30M transactions per day
● assumption: one transaction is one request for ESB
● Non-blocking HTTP/S transport with message
queuing and configurable thread pools
● Different thread pools at the application layer
and the I/O layer

Availability of WSO2 ESB
● Availability can be achieved with the
deployment
● Two passive nodes with a given active node
● Graceful shutdown and maintenance mode
● Shutdown the listeners
● Let the senders send out the responses after
processing the already accepted messages
● Shutdown the senders and the server or upgrade
the system and restart the listener manager
● Round robin restart of the cluster in active active deployment

Deployment diagram

Cloud Computing

Cloud Computing
Characteristics & Benefits to SOA
●
● Auto-scaling (elasticity)
● Multi-tenancy
● Scalable parallel computing
● Other benefits include
● Cost savings by not peak provisioning
● Faster deployment
● Outsource non-core skills
● Utility billing

AMIs for the Cloud
All WSO2 products would be available as
●
AMIs soon
● Compatible with Amazon EC2
● Ready to be deployed

Auto Scaling - Elasticity
●Ability for the resources to be scaled up, and down, based on
demand
●Enables efficient use of computer resources
● No need to provision for maximum load
● Requires application or middleware to do work to manage load
– Queue up incoming requests
– Distribute amongst available worker nodes
– Monitor queue length and adjust worker pool accordingly

Auto Scaling Services
WSO2 WSAS/ESB Approach

Auto Scaling Services
WSO2 WSAS/ESB Approach
Load Increases

Auto Scaling Services
WSO2 WSAS/ESB Approach
Startup new instances

Auto Scaling Services
WSO2 WSAS/ESB Approach
New instances join group

Auto Scaling Services
WSO2 WSAS/ESB Approach
Load Decreases

Auto Scaling Services
WSO2 WSAS/ESB Approach
Terminate instances

Summary
● Enterprises thrive for scalability and availability
● Clustering provides us a perfect solution to
meet enterprise demands
● Apache Axis2 clustering support is well designed
and battle tested
● Done by WSO2 team members
● Used in almost all WSO2 products

Summary
● ESB is a key element in SOA deployments
when it comes to load balancing and fail over
● WSO2 ESB is well equipped with these capabilities
● Cloud computing is opening up new
opportunities for SOA deployments
● Auto scaling one of the key benefits

Resources
● Introduction to WSO2 Carbon Clustering -
http://wso2.org/library/articles/introduction-wso2-carbon-clustering
● WSO2 Carbon Cluster Configuration Language -
http://wso2.org/library/articles/wso2-carbon-cluster-configuration-language
● WSO2 ESB/Apache Synapse Clustering Guide -
http://wso2.org/library/articles/wso2-esb-apache-synapse-clustering-guide
● Achieving Scalability and High Availability for Clustered Web Services using
Apache Synapse -
http://wso2.org/library/presentations/apachecon-eu-2009-achieving-scalability-hig
● Autoscaling Axis2 Web Services on Amazon EC2 -
http://wso2.org/library/presentations/autoscaling-axis2-web-services-amazon-ec2
● Scalable SOA in Your Own Open Source Cloud -
http://wso2.org/library/news/presentation-soa-symposium-scalable-soa-your-own-

Tell a Friend...
● If you found this useful...
● Give us your feedback
● Blog
● Tweet
● Spread the word :)