This document summarizes the WSO2 Elastic Load Balancer (ELB). It discusses how the ELB distributes load, supports session affinity, and replicates sessions. It describes the ELB's evolution from scaling a PaaS to its current tenant-aware architecture. The ELB uses dynamic load balancing algorithms, autoscales clusters, and maps ports. It distributes requests to nodes using a load balancing algorithm and fails over if nodes are down while maintaining sticky sessions.

1. WSO2 Elastic Load Balancer
Chintana Wilamuna
chintana@wso2.com

2. Load balancer
• Distribute load among a set of nodes
• Static
(fixed set of nodes)
• Dynamic
(add/remove nodes at runtime)
• Load balancing algorithm
(Round robin / weighted RR / weighted RR least connection)

3. Session affinity / sticky sessions
• All subsequent connections within a
session is served by the worker node
which processed the first request of
the session
• Alternate solution to session
replicated backend services

4. Session replication in AppServer
• AppServer clustering – Apache Tribes
• Session data storage,
1. ServiceContext
(data that should only be available to the service)
2. ServiceGroupContext
(common data for all the services in a service group)
3. ConfigurationContext
(common data for all service groups)
• AppServer replicates session data in
all 3 levels (costly compared to sticky sessions)

5. LB – Membership mgt. schemes
• Static
• Dynamic
Join by specifying the group name
• Hybrid (WKA)
Set of well known members – static
Join by specifying the group name

6. Evolution of WSO2 ELB
• Started as a solution to scale PaaS –
(WSO2 Stratos)
• Stratos is a multi-tenanted PaaS
• Tenant – An organization (a domain)
• Isolated from other tenants
• Support multiple PaaS services
• A service = WSO2 product (DSS, ESB
etc…) offered as a service

7. Evolution of WSO2 ELB cont.
• A tenant can deploy their own
services, mediations, workflows etc…
• Multi-tenant services (ESB, DS,
AppServer etc…)
• Stratos can provision resources on
demand
• Problem: making a scalable PaaS
transparent to the user

8. Solution 1
• Stratos have multi-tenant services
(ESB, AppServer, DS etc…)
• User talks to Identity Server, gets an
SAML token, logs into any service
• Store tenant data in Registry
• Each server loads all tenant data at
startup
• Can support any tenant

9. Solution 1

10. Solution 1 - Problems
• Doesn’t scale with increasing number
of tenants

11. Solution 2
• Started running multiple service
instances
• Put a LB in front
• When the load is high LB will spin up
a new instance and shuts it off when
the load decreases (autoscale)

12. Solution 2

13. Solution 2 - Problems
• When we had a large number of
tenants with tens of services, took a
long time to load at startup
• Some tenants stayed inactive
(consumed resources)

14. Solution 3
• Add lazy loading
• Tenant info in the central registry
• Tenant info loaded only when needed
• More info - http://blog.afkham.org/2011/11/lazy-loading-
deployment-artifacts-in.html

15. Solution 3 - Problems
• When a tenant has many artifacts
loading those takes time
• Initial requests for an artifact of a
tenant timed out

16. Solution 4
• Introduce the GhostDeployer
• Loads only the metadata of a tenant
• Actual artifacts are loaded on demand
• More info - http://blog.afkham.org/2011/11/lazy-loading-
deployment-artifacts-in.html

17. Solution 4 - Problems
• LB didn’t scale to handle large
number of requests

18. Solution 5
• Replace LB with multiple LBs
• HW LB or DNS RR between LBs

19. Solution 5

20. Solution 5 - Problems
• Many to many relationship between
LBs for synchronizing metadata
• LB is not aware of tenants
• A single node might have too many
tenants

21. Solution 6
• Tenant aware load balancer

22. WSO2 ELB architecture

23. WSO2 ELB
• Autoscaling component keeps track of
traffic for each cloud service cluster
• Calls EC2 API
• Implementation: 2 mediators and a
task (autoscale-in and autoscale-out)
• An ESB endpoint routes requests to
the correct node

24. WSO2 ELB
• Service cluster identification through
Host HTTP header
• According to LB algo. select a node
• If the node is down it’ll fail over to
another member
• Keeps track of sticky sessions
(identified by JSESSIONID)

25. WSO2 ELB
• BinaryRelay is used as a message pass
through mechanism
(no building or processing the message)
• Membership discovery & mgt. – Axis2
clustering. Membership handler for
each clustering domain
• Use Apache Tribes Group Mgt.
Framework

26. WSO2 ELB

27. WSO2 ELB – Port mapping

28. WSO2 ELB – Port mapping
• Advertise the mapped ports when
joining the cluster
• LB retrieves these as properties of the
member
• When dispatching requests, get
mapped real port from the member
(as advertised)

29. WSO2 ELB – Autoscaling

30. WSO2 ELB – Autoscaling
• AutoscaleInMediator creates a unique
token when a request comes, put into a
list
• Removes this when a response is sent
• LoadAnalyzerTaks periodically checks
the list. Autoscale according to config
params

31. WSO2 ELB – Autoscaling
• Autoscaling algorithm -
http://nirmalfdo.blogspot.com/2012/07/autoscaling-algorithm-used-in-
wso2.html

32. WSO2 ELB – Configuration
• How to configure the ELB -
http://nirmalfdo.blogspot.com/2012/06/fronting-wso2-application-server-
50.html

33. Questions?