The document discusses the role of the Elastic Load Balancer (ELB) in Apache Stratos PaaS. It describes how the ELB uses components like Synapse, Axis2, and Tribes to distribute incoming traffic across backend nodes and auto-scale capacity. The ELB handles load balancing, failover, auto-scaling, and multi-tenancy. It integrates with Stratos by receiving topology information, load balancing requests to cartridge instances, and auto-scaling the number of instances based on traffic.

1. The Role of
Elastic Load Balancer (ELB)
Imesh Gunaratne
Apache Contributor, Technical Lead - WSO2 Inc

2. Agenda
➔ Introduction to Load Balancing
◆ What is Load Balancing?
◆ Algorithms
◆ Node Configuration Modes
◆ Why is it called Elastic?
◆ Purpose
◆ Features
➔ Component Architecture of Apache Stratos ELB
◆ Synapse Mediation Framework
◆ Apache Axis2 Clustering
◆ Apache Tribes Group Management
◆ Binary Relay Message Builder
◆ Load Balance Endpoint Module
◆ Auto-scaling Module
2

3. Agenda (cont.)
➔ ELB’s role in Apache Stratos PaaS
◆ Apache Stratos Logical Architecture
◆ Workflow
➔ Auto-scaling Process
◆ Configuration
◆ Auto-scaling Algorithm
◆ Plugging in Custom Algorithms
➔ How to Avoid Single Point of Failure of ELB
3

4. Introduction to Load Balancing
How it is done & why it matters?
4

5. What is Load Balancing in general?
Load balancing is a computer networking method for
distributing workloads across multiple computers or a computer
cluster, network links, central processing units, disk drives, or
other resources
- Wikipedia
http://en.wikipedia.org/wiki/Load_balancer 5

6. Load balancing is used to distribute the incoming traffic
amongst a set of backend worker nodes which are statically
configured or dynamically discovered.
http://docs.wso2.org/wiki/display/ELB203/Load+Balacing+Basics
LB
W1
W2
Wn
Clients
Worker nodes
Incoming Traffic
6
What is Load Balancing in
Middleware?

7. Load Balancing Clusters
A cluster is a set of nodes that communicate with each
other and work towards a common goal.
7

8. Membership Schemes
Static Dynamic
Configuration Modes
● Only a predefined set of
nodes could exist in a
cluster.
● New nodes cannot be
added at runtime.
● Membership is not
predefined.
● Nodes could discover the
load balancer.
● Nodes could join a cluster by
specifying a cluster name.
Hybrid
● Also called Well-Known
Addressed (WKA).
● A cluster could have set
of well known members.
● Nodes could join a
cluster via a well-known
member.
8

9. Most widely used Load Balancing
Algorithms
➔ Round Robin
◆ Passes each new connection request to the next server in line
➔ Weighted Round Robin
◆ The number of connections that each machine receives over time is
proportionate to a ratio weight you define.
➔ Least Connections
◆ Passes a new connection to the server that has the least number of
current connections.
https://devcentral.f5.com/articles/intro-to-load-balancing-for-developers-
ndash-the-algorithms 9

10. Why it is called Elastic?
Load BalancerAutoscaler
+
Monitor the
incoming traffic
and scales request
handling capacity
(no of nodes)
Elastic Load
Balancer=
Distribute the load of
incoming traffic amongst
a set of worker nodes
10

11. What is the Purpose?
The motivation of load balancing is to
➔ Optimize resource usage
◆ Start and stop resources on demand.
➔ Maximize the throughput
◆ Increase the average rate of successful message delivery.
➔ Minimize the response time
◆ Reduce the time it takes to process a message and send a response
back.
http://en.wikipedia.org/wiki/Load_balancer
11

12. Main Features of a Load Balancer
There are three main features:
➔ Failover Handling
◆ Avoid single point of failure by hosting multiple instances of a given
service.
➔ Auto-scaling
◆ Manage number of instances of an application according to the
incoming traffic.
➔ Multi-tenancy
◆ Manage multiple tenants of applications.
12

13. Component Architecture of
Apache Stratos ELB
How it was designed
13

14. Component Architecture
14

15. Apache Synapse Mediation
Framework
15

16. How Synapse was Extended
16

17. Binary Relay Message Builder
● Synapse uses Axis2 engine for message processing.
● Axis2 uses Message Formatters & Message Builders for
serializing and building incoming messages into SOAP
format.
● Binary Relay is an Axis2 message builder which pass through
all messages without processing them.
Binary Relay
Message Builder 17

18. Binary Relay definition in Axis2.xml
elb/repository/conf/axis2/axis2.xml
<messageBuilders>
<messageBuilder contentType="application/xml"
class="org.wso2.carbon.relay.BinaryRelayBuilder"/>
<messageBuilder contentType="application/txt"
class="org.wso2.carbon.relay.BinaryRelayBuilder"/>
. . .
</messageBuilders>
Binary Relay
Message Builder 18

19. Load Balance Endpoint Module
● Tenant Aware Load Balance Endpoint
○ Extends Synapse Dynamic Load Balance Endpoint.
○ Utilizes round robin load balance algorithm.
● Topology Syncher
○ Receives service cluster topology information from Cloud Controller
via the Message Broker.
● Health Checker
○ Re-establishes connection to the Message Broker if it drops.
Load Balance
Endpoint Module 19

20. Load Balance Endpoint Module (cont.)
● Cluster Domain Manager Impl
○ Manages cluster sub domains of cartridge instances.
● Group Mgt Agent Builder
○ Manages Axis2 group management agents of cluster sub domains.
● Registry Manager
○ Receives domain mappings of cartridge instances from ADC manager
via the registry.
Load Balance
Endpoint Module 20

21. Session Affinity
● There are two different ways to manage session information.
● Replicate in cluster is a very costly process.
● Therefore ELB manages session information for the
applications.
Session Information
Replicate in Cluster Handled by LB
Load Balance
Endpoint Module
21

22. Auto-scaling Module
● Autoscale In Mediator
○ Generates a token (request id) per request received and adds it to a
queue.
● Autoscale Out Mediator
○ Removes the token added by the in mediator when a response is
received from the end point.
● Service Requests InFlight Autoscaler (Task)
○ Performs sanity checks to ensure that all clusters meet the minimum
number of nodes.
○ Performs scaling based on the request load & scaling configuration
parameters.
Auto-scaling Module
22

23. ELB’s role in Apache Stratos PaaS
How does it contribute?
23

24. ELB’s role in Apache Stratos PaaS
24

25. Cartridge Subscription Workflow
1. [SC -> CC] Send Cartridge Subscription Request
2. [CC] Deploy Cartridge Instance Service
3. [CC -> MB] Publish Cluster Topology Information
4. [MB -> ELB] Receive Cluster Topology Information
5. [CC -> jclouds] Instance Spawn Request
6. [jclouds -> IaaS] Spawn Cartridge Instance
7. [Cartridge -> Agent] Request to Join Cluster
8. [Agent -> ELB] Add Node to Cluster
25

26. 1. [Client -> ELB] Send request message
2. [ELB] Identify cluster & tenant using message header
3. [ELB] Add request to a list
4. [ELB -> Node] If session exists, send message
5. [ELB] If not store session information
6. [ELB -> Node] Apply algorithm & send message
7. [ELB -> Node] Handle failover
8. [Node -> ELB] Send response
9. [ELB -> Client] Send response and remove request from list
10. [ELB] Scale number of cartridge instances
Load Balancing Workflow
26

27. Load Balancer Configuration
loadbalancer.conf
loadbalancer {
# minimum number of load balancer instances
instances 1;
# whether autoscaling should be enabled or not.
enable_autoscaler true;
# autoscaling decision making task
#autoscaler_task org.wso2.carbon.mediator.autoscale.lbautoscale.task.
ServiceRequestsInFlightAutoscaler;
#please use this whenever url-mapping is used through LB.
#size_of_cache 100;
...
27

28. Load Balancer Configuration (cont.)
loadbalancer {
...
# Endpoint reference of the Autoscaler Service. This should be present,
if you disabled embedded autoscaling.
#autoscaler_service_epr https://host_address:
https_port/services/AutoscalerService/;
# interval between two task executions in milliseconds
autoscaler_task_interval 60000;
# after an instance booted up, task will wait maximum till this much of
time and let the server started up
server_startup_delay 180000; #default will be 60000ms
# session timeout
session_timeout 90000;
# enable failover
fail_over true;
}
28

29. Port Mapping
● Ports of applications deployed in cartridge instances are
mapped to external ports by the load balancer.
● Port mapping is defined in <cartridge>.xml file.
● Example:
<portMapping>
<http port="80" proxyPort="8280"/>
<https port="443" proxyPort="8243"/>
</portMapping>
29

30. Auto-scaling Process
Configuration & custom implementations
30

31. Auto-scaling Configuration
loadbalancer.conf
services {
# default parameter values to be used in all services
defaults {
# minimum number of service instances required
min_app_instances 1;
# maximum number of service instances that will be load balanced
max_app_instances 3;
# maximum number of requests served per second by a service instance
max_requests_per_second 5;
# scale up early using AUR, 0 < AUR <= 1 and default is 0.7
alarming_upper_rate 0.7;
# scale down slowly using ALR, 0 < ALR <= 1 and default is 0.2
alarming_lower_rate 0.2;
# scale down slowly using SDF, 0 < SDF <= 1 and default is 0.25
scale_down_factor 0.25;
# no of iterations in-flight avg is calculated to take the decision
rounds_to_average 2;
message_expiry_time 60000;
}
31

32. Auto-scaling Algorithm
https://cwiki.apache.org/confluence/display/STRATOS/Elastic+Load+Balancer
32

33. Custom Auto-scaling Implementation
● You could write your own Java implementation which
implements org.apache.synapse.task.Task and org.apache.
synapse.ManagedLifecycle interfaces.
● Wrap the implementation class to an OSGi bundle and deploy
it in the ELB OSGi container.
● Update autoscaler_task value in loadbalancer.conf.
33

34. How to avoid Single Point of
Failure of ELB
An ELB is prone to single point of failure
34

35. Clustered ELB Deployment
Architecture
35

36. Questions?
Or any feedback is really appreciated
36

37. References
● Apache Stratos Documentation
https://cwiki.apache.org/confluence/display/STRATOS/Index
● WSO2 Load Balancer - How it works
http://blog.afkham.org/2011/09/wso2-load-balancer-how-it-works.html
● WSO2 Webinar - Introducing the WSO2 Load Balancer
http://www.youtube.com/watch?v=HgiTyvZ-Sfw
● WSO2 Elastic Load Balancer Documentation
http://docs.wso2.
org/wiki/display/ELB203/WSO2+Elastic+Load+Balancer+Documentation
● WSO2, Introduction to Carbon Clustering
http://wso2.com/library/articles/introduction-wso2-carbon-clustering
37

38. Website
http://stratos.incubator.apache.org
Mailing List
Subscribe: dev-subscribe@stratos.incubator.apache.org
Post (After subscription): dev@stratos.incubator.apache.org
Social Media
Google+: https://plus.google.com/103515557134069849802
Twitter: https://twitter.com/ApacheStratos
Facebook: https://www.facebook.com/apache.stratos
LinkedIn: http://www.linkedin.com/groups?home=&gid=5131436
38
Join Us