Introducing the WSO2 Elastic Load Balancer

WSO2 Elastic Load Balancer Chintana Wilamuna chintana@wso2.com

Load balancer• Distribute load among a set of nodes• Static (fixed set of nodes)• Dynamic (add/remove nodes at runtime)• Load balancing algorithm (Round robin / weighted RR / weighted RR least connection)

Session affinity / sticky sessions• All subsequent connections within a session is served by the worker node which processed the first request of the session• Alternate solution to session replicated backend services

Session replication in AppServer• AppServer clustering – Apache Tribes• Session data storage, 1. ServiceContext (data that should only be available to the service) 2. ServiceGroupContext (common data for all the services in a service group) 3. ConfigurationContext (common data for all service groups)• AppServer replicates session data in all 3 levels (costly compared to sticky sessions)

LB – Membership mgt. schemes• Static• Dynamic Join by specifying the group name• Hybrid (WKA) Set of well known members – static Join by specifying the group name

Evolution of WSO2 ELB• Started as a solution to scale PaaS – (WSO2 Stratos)• Stratos is a multi-tenanted PaaS• Tenant – An organization (a domain) • Isolated from other tenants• Support multiple PaaS services• A service = WSO2 product (DSS, ESB etc…) offered as a service

Evolution of WSO2 ELB cont.• A tenant can deploy their own services, mediations, workflows etc…• Multi-tenant services (ESB, DS, AppServer etc…)• Stratos can provision resources on demand• Problem: making a scalable PaaS transparent to the user

Solution 1• Stratos have multi-tenant services (ESB, AppServer, DS etc…)• User talks to Identity Server, gets an SAML token, logs into any service• Store tenant data in Registry• Each server loads all tenant data at startup• Can support any tenant

Solution 1

Solution 1 - Problems• Doesn’t scale with increasing number of tenants

Solution 2• Started running multiple service instances• Put a LB in front• When the load is high LB will spin up a new instance and shuts it off when the load decreases (autoscale)

Solution 2

Solution 2 - Problems• When we had a large number of tenants with tens of services, took a long time to load at startup• Some tenants stayed inactive (consumed resources)

Solution 3• Add lazy loading• Tenant info in the central registry• Tenant info loaded only when needed• More info - http://blog.afkham.org/2011/11/lazy-loading- deployment-artifacts-in.html

Solution 3 - Problems• When a tenant has many artifacts loading those takes time• Initial requests for an artifact of a tenant timed out

Solution 4• Introduce the GhostDeployer• Loads only the metadata of a tenant• Actual artifacts are loaded on demand• More info - http://blog.afkham.org/2011/11/lazy-loading- deployment-artifacts-in.html

Solution 4 - Problems• LB didn’t scale to handle large number of requests

Solution 5• Replace LB with multiple LBs• HW LB or DNS RR between LBs

Solution 5

Solution 5 - Problems• Many to many relationship between LBs for synchronizing metadata• LB is not aware of tenants• A single node might have too many tenants

Solution 6• Tenant aware load balancer

WSO2 ELB architecture

WSO2 ELB• Autoscaling component keeps track of traffic for each cloud service cluster• Calls EC2 API• Implementation: 2 mediators and a task (autoscale-in and autoscale-out)• An ESB endpoint routes requests to the correct node

WSO2 ELB• Service cluster identification through Host HTTP header• According to LB algo. select a node• If the node is down it’ll fail over to another member• Keeps track of sticky sessions (identified by JSESSIONID)

WSO2 ELB• BinaryRelay is used as a message pass through mechanism (no building or processing the message)• Membership discovery & mgt. – Axis2 clustering. Membership handler for each clustering domain• Use Apache Tribes Group Mgt. Framework

WSO2 ELB

WSO2 ELB – Port mapping

WSO2 ELB – Port mapping• Advertise the mapped ports when joining the cluster• LB retrieves these as properties of the member• When dispatching requests, get mapped real port from the member (as advertised)

WSO2 ELB – Autoscaling

WSO2 ELB – Autoscaling• AutoscaleInMediator creates a unique token when a request comes, put into a list• Removes this when a response is sent• LoadAnalyzerTaks periodically checks the list. Autoscale according to config params

WSO2 ELB – Autoscaling• Autoscaling algorithm - http://nirmalfdo.blogspot.com/2012/07/autoscaling-algorithm-used-in- wso2.html

WSO2 ELB – Configuration• How to configure the ELB - http://nirmalfdo.blogspot.com/2012/06/fronting-wso2-application-server- 50.html

Questions?

http://wso2.org	153
http://tdl.insiel.it	49
http://node1.wso2.org	6
http://node2.wso2.org	1

Introducing the WSO2 Elastic Load Balancer

by WSO2 on Aug 09, 2012

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 209

Statistics

Introducing the WSO2 Elastic Load Balancer Presentation Transcript