Introducing the WSO2 Elastic Load Balancer


Published on

Published in: Technology, Economy & Finance
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Introducing the WSO2 Elastic Load Balancer

  1. 1. WSO2 Elastic Load Balancer Chintana Wilamuna
  2. 2. Load balancer• Distribute load among a set of nodes• Static (fixed set of nodes)• Dynamic (add/remove nodes at runtime)• Load balancing algorithm (Round robin / weighted RR / weighted RR least connection)
  3. 3. Session affinity / sticky sessions• All subsequent connections within a session is served by the worker node which processed the first request of the session• Alternate solution to session replicated backend services
  4. 4. Session replication in AppServer• AppServer clustering – Apache Tribes• Session data storage, 1. ServiceContext (data that should only be available to the service) 2. ServiceGroupContext (common data for all the services in a service group) 3. ConfigurationContext (common data for all service groups)• AppServer replicates session data in all 3 levels (costly compared to sticky sessions)
  5. 5. LB – Membership mgt. schemes• Static• Dynamic Join by specifying the group name• Hybrid (WKA) Set of well known members – static Join by specifying the group name
  6. 6. Evolution of WSO2 ELB• Started as a solution to scale PaaS – (WSO2 Stratos)• Stratos is a multi-tenanted PaaS• Tenant – An organization (a domain) • Isolated from other tenants• Support multiple PaaS services• A service = WSO2 product (DSS, ESB etc…) offered as a service
  7. 7. Evolution of WSO2 ELB cont.• A tenant can deploy their own services, mediations, workflows etc…• Multi-tenant services (ESB, DS, AppServer etc…)• Stratos can provision resources on demand• Problem: making a scalable PaaS transparent to the user
  8. 8. Solution 1• Stratos have multi-tenant services (ESB, AppServer, DS etc…)• User talks to Identity Server, gets an SAML token, logs into any service• Store tenant data in Registry• Each server loads all tenant data at startup• Can support any tenant
  9. 9. Solution 1
  10. 10. Solution 1 - Problems• Doesn’t scale with increasing number of tenants
  11. 11. Solution 2• Started running multiple service instances• Put a LB in front• When the load is high LB will spin up a new instance and shuts it off when the load decreases (autoscale)
  12. 12. Solution 2
  13. 13. Solution 2 - Problems• When we had a large number of tenants with tens of services, took a long time to load at startup• Some tenants stayed inactive (consumed resources)
  14. 14. Solution 3• Add lazy loading• Tenant info in the central registry• Tenant info loaded only when needed• More info - deployment-artifacts-in.html
  15. 15. Solution 3 - Problems• When a tenant has many artifacts loading those takes time• Initial requests for an artifact of a tenant timed out
  16. 16. Solution 4• Introduce the GhostDeployer• Loads only the metadata of a tenant• Actual artifacts are loaded on demand• More info - deployment-artifacts-in.html
  17. 17. Solution 4 - Problems• LB didn’t scale to handle large number of requests
  18. 18. Solution 5• Replace LB with multiple LBs• HW LB or DNS RR between LBs
  19. 19. Solution 5
  20. 20. Solution 5 - Problems• Many to many relationship between LBs for synchronizing metadata• LB is not aware of tenants• A single node might have too many tenants
  21. 21. Solution 6• Tenant aware load balancer
  22. 22. WSO2 ELB architecture
  23. 23. WSO2 ELB• Autoscaling component keeps track of traffic for each cloud service cluster• Calls EC2 API• Implementation: 2 mediators and a task (autoscale-in and autoscale-out)• An ESB endpoint routes requests to the correct node
  24. 24. WSO2 ELB• Service cluster identification through Host HTTP header• According to LB algo. select a node• If the node is down it’ll fail over to another member• Keeps track of sticky sessions (identified by JSESSIONID)
  25. 25. WSO2 ELB• BinaryRelay is used as a message pass through mechanism (no building or processing the message)• Membership discovery & mgt. – Axis2 clustering. Membership handler for each clustering domain• Use Apache Tribes Group Mgt. Framework
  26. 26. WSO2 ELB
  27. 27. WSO2 ELB – Port mapping
  28. 28. WSO2 ELB – Port mapping• Advertise the mapped ports when joining the cluster• LB retrieves these as properties of the member• When dispatching requests, get mapped real port from the member (as advertised)
  29. 29. WSO2 ELB – Autoscaling
  30. 30. WSO2 ELB – Autoscaling• AutoscaleInMediator creates a unique token when a request comes, put into a list• Removes this when a response is sent• LoadAnalyzerTaks periodically checks the list. Autoscale according to config params
  31. 31. WSO2 ELB – Autoscaling• Autoscaling algorithm - wso2.html
  32. 32. WSO2 ELB – Configuration• How to configure the ELB - 50.html
  33. 33. Questions?