The Role of
Elastic Load Balancer (ELB)
Apache Contributor, Technical Lead - WSO2 Inc
➔ Introduction to Load Balancing
◆ What is Load Balancing?
◆ Node Configuration Modes
◆ Why is it called Elastic?
➔ Component Architecture of Apache Stratos ELB
◆ Synapse Mediation Framework
◆ Apache Axis2 Clustering
◆ Apache Tribes Group Management
◆ Binary Relay Message Builder
◆ Load Balance Endpoint Module
◆ Auto-scaling Module
➔ ELB’s role in Apache Stratos PaaS
◆ Apache Stratos Logical Architecture
➔ Auto-scaling Process
◆ Auto-scaling Algorithm
◆ Plugging in Custom Algorithms
➔ How to Avoid Single Point of Failure of ELB
Introduction to Load Balancing
How it is done & why it matters?
What is Load Balancing in general?
Load balancing is a computer networking method for
distributing workloads across multiple computers or a computer
cluster, network links, central processing units, disk drives, or
Load balancing is used to distribute the incoming traffic
amongst a set of backend worker nodes which are statically
configured or dynamically discovered.
What is Load Balancing in
Load Balancing Clusters
A cluster is a set of nodes that communicate with each
other and work towards a common goal.
● Only a predefined set of
nodes could exist in a
● New nodes cannot be
added at runtime.
● Membership is not
● Nodes could discover the
● Nodes could join a cluster by
specifying a cluster name.
● Also called Well-Known
● A cluster could have set
of well known members.
● Nodes could join a
cluster via a well-known
Most widely used Load Balancing
➔ Round Robin
◆ Passes each new connection request to the next server in line
➔ Weighted Round Robin
◆ The number of connections that each machine receives over time is
proportionate to a ratio weight you define.
➔ Least Connections
◆ Passes a new connection to the server that has the least number of
Why it is called Elastic?
and scales request
(no of nodes)
Distribute the load of
incoming traffic amongst
a set of worker nodes
What is the Purpose?
The motivation of load balancing is to
➔ Optimize resource usage
◆ Start and stop resources on demand.
➔ Maximize the throughput
◆ Increase the average rate of successful message delivery.
➔ Minimize the response time
◆ Reduce the time it takes to process a message and send a response
Main Features of a Load Balancer
There are three main features:
➔ Failover Handling
◆ Avoid single point of failure by hosting multiple instances of a given
◆ Manage number of instances of an application according to the
◆ Manage multiple tenants of applications.
Component Architecture of
Apache Stratos ELB
How it was designed
Binary Relay Message Builder
● Synapse uses Axis2 engine for message processing.
● Axis2 uses Message Formatters & Message Builders for
serializing and building incoming messages into SOAP
● Binary Relay is an Axis2 message builder which pass through
all messages without processing them.
Message Builder 17
Load Balance Endpoint Module
● Tenant Aware Load Balance Endpoint
○ Extends Synapse Dynamic Load Balance Endpoint.
○ Utilizes round robin load balance algorithm.
● Topology Syncher
○ Receives service cluster topology information from Cloud Controller
via the Message Broker.
● Health Checker
○ Re-establishes connection to the Message Broker if it drops.
Endpoint Module 19
Load Balance Endpoint Module (cont.)
● Cluster Domain Manager Impl
○ Manages cluster sub domains of cartridge instances.
● Group Mgt Agent Builder
○ Manages Axis2 group management agents of cluster sub domains.
● Registry Manager
○ Receives domain mappings of cartridge instances from ADC manager
via the registry.
Endpoint Module 20
● There are two different ways to manage session information.
● Replicate in cluster is a very costly process.
● Therefore ELB manages session information for the
Replicate in Cluster Handled by LB
● Autoscale In Mediator
○ Generates a token (request id) per request received and adds it to a
● Autoscale Out Mediator
○ Removes the token added by the in mediator when a response is
received from the end point.
● Service Requests InFlight Autoscaler (Task)
○ Performs sanity checks to ensure that all clusters meet the minimum
number of nodes.
○ Performs scaling based on the request load & scaling configuration
ELB’s role in Apache Stratos PaaS
How does it contribute?
1. [Client -> ELB] Send request message
2. [ELB] Identify cluster & tenant using message header
3. [ELB] Add request to a list
4. [ELB -> Node] If session exists, send message
5. [ELB] If not store session information
6. [ELB -> Node] Apply algorithm & send message
7. [ELB -> Node] Handle failover
8. [Node -> ELB] Send response
9. [ELB -> Client] Send response and remove request from list
10. [ELB] Scale number of cartridge instances
Load Balancing Workflow
Load Balancer Configuration
# minimum number of load balancer instances
# whether autoscaling should be enabled or not.
# autoscaling decision making task
#please use this whenever url-mapping is used through LB.
Load Balancer Configuration (cont.)
# Endpoint reference of the Autoscaler Service. This should be present,
if you disabled embedded autoscaling.
# interval between two task executions in milliseconds
# after an instance booted up, task will wait maximum till this much of
time and let the server started up
server_startup_delay 180000; #default will be 60000ms
# session timeout
# enable failover
● Ports of applications deployed in cartridge instances are
mapped to external ports by the load balancer.
● Port mapping is defined in <cartridge>.xml file.
<http port="80" proxyPort="8280"/>
<https port="443" proxyPort="8243"/>
Configuration & custom implementations
# default parameter values to be used in all services
# minimum number of service instances required
# maximum number of service instances that will be load balanced
# maximum number of requests served per second by a service instance
# scale up early using AUR, 0 < AUR <= 1 and default is 0.7
# scale down slowly using ALR, 0 < ALR <= 1 and default is 0.2
# scale down slowly using SDF, 0 < SDF <= 1 and default is 0.25
# no of iterations in-flight avg is calculated to take the decision
Custom Auto-scaling Implementation
● You could write your own Java implementation which
implements org.apache.synapse.task.Task and org.apache.
● Wrap the implementation class to an OSGi bundle and deploy
it in the ELB OSGi container.
● Update autoscaler_task value in loadbalancer.conf.
How to avoid Single Point of
Failure of ELB
An ELB is prone to single point of failure
Or any feedback is really appreciated
● Apache Stratos Documentation
● WSO2 Load Balancer - How it works
● WSO2 Webinar - Introducing the WSO2 Load Balancer
● WSO2 Elastic Load Balancer Documentation
● WSO2, Introduction to Carbon Clustering
Post (After subscription): firstname.lastname@example.org