Your SlideShare is downloading. ×
0
The Role of
Elastic Load Balancer (ELB)
Imesh Gunaratne
Apache Contributor, Technical Lead - WSO2 Inc
Agenda
➔ Introduction to Load Balancing
◆ What is Load Balancing?
◆ Algorithms
◆ Node Configuration Modes
◆ Why is it call...
Agenda (cont.)
➔ ELB’s role in Apache Stratos PaaS
◆ Apache Stratos Logical Architecture
◆ Workflow
➔ Auto-scaling Process...
Introduction to Load Balancing
How it is done & why it matters?
4
What is Load Balancing in general?
Load balancing is a computer networking method for
distributing workloads across multip...
Load balancing is used to distribute the incoming traffic
amongst a set of backend worker nodes which are statically
confi...
Load Balancing Clusters
A cluster is a set of nodes that communicate with each
other and work towards a common goal.
7
Membership Schemes
Static Dynamic
Configuration Modes
● Only a predefined set of
nodes could exist in a
cluster.
● New nod...
Most widely used Load Balancing
Algorithms
➔ Round Robin
◆ Passes each new connection request to the next server in line
➔...
Why it is called Elastic?
Load BalancerAutoscaler
+
Monitor the
incoming traffic
and scales request
handling capacity
(no ...
What is the Purpose?
The motivation of load balancing is to
➔ Optimize resource usage
◆ Start and stop resources on demand...
Main Features of a Load Balancer
There are three main features:
➔ Failover Handling
◆ Avoid single point of failure by hos...
Component Architecture of
Apache Stratos ELB
How it was designed
13
Component Architecture
14
Apache Synapse Mediation
Framework
15
How Synapse was Extended
16
Binary Relay Message Builder
● Synapse uses Axis2 engine for message processing.
● Axis2 uses Message Formatters & Message...
Binary Relay definition in Axis2.xml
elb/repository/conf/axis2/axis2.xml
<messageBuilders>
<messageBuilder contentType="ap...
Load Balance Endpoint Module
● Tenant Aware Load Balance Endpoint
○ Extends Synapse Dynamic Load Balance Endpoint.
○ Utili...
Load Balance Endpoint Module (cont.)
● Cluster Domain Manager Impl
○ Manages cluster sub domains of cartridge instances.
●...
Session Affinity
● There are two different ways to manage session information.
● Replicate in cluster is a very costly pro...
Auto-scaling Module
● Autoscale In Mediator
○ Generates a token (request id) per request received and adds it to a
queue.
...
ELB’s role in Apache Stratos PaaS
How does it contribute?
23
ELB’s role in Apache Stratos PaaS
24
Cartridge Subscription Workflow
1. [SC -> CC] Send Cartridge Subscription Request
2. [CC] Deploy Cartridge Instance Servic...
1. [Client -> ELB] Send request message
2. [ELB] Identify cluster & tenant using message header
3. [ELB] Add request to a ...
Load Balancer Configuration
loadbalancer.conf
loadbalancer {
# minimum number of load balancer instances
instances 1;
# wh...
Load Balancer Configuration (cont.)
loadbalancer {
...
# Endpoint reference of the Autoscaler Service. This should be pres...
Port Mapping
● Ports of applications deployed in cartridge instances are
mapped to external ports by the load balancer.
● ...
Auto-scaling Process
Configuration & custom implementations
30
Auto-scaling Configuration
loadbalancer.conf
services {
# default parameter values to be used in all services
defaults {
#...
Auto-scaling Algorithm
https://cwiki.apache.org/confluence/display/STRATOS/Elastic+Load+Balancer
32
Custom Auto-scaling Implementation
● You could write your own Java implementation which
implements org.apache.synapse.task...
How to avoid Single Point of
Failure of ELB
An ELB is prone to single point of failure
34
Clustered ELB Deployment
Architecture
35
Questions?
Or any feedback is really appreciated
36
References
● Apache Stratos Documentation
https://cwiki.apache.org/confluence/display/STRATOS/Index
● WSO2 Load Balancer -...
Website
http://stratos.incubator.apache.org
Mailing List
Subscribe: dev-subscribe@stratos.incubator.apache.org
Post (After...
Upcoming SlideShare
Loading in...5
×

The Role of Elastic Load Balancer - Apache Stratos

5,410

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,410
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
34
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "The Role of Elastic Load Balancer - Apache Stratos"

  1. 1. The Role of Elastic Load Balancer (ELB) Imesh Gunaratne Apache Contributor, Technical Lead - WSO2 Inc
  2. 2. Agenda ➔ Introduction to Load Balancing ◆ What is Load Balancing? ◆ Algorithms ◆ Node Configuration Modes ◆ Why is it called Elastic? ◆ Purpose ◆ Features ➔ Component Architecture of Apache Stratos ELB ◆ Synapse Mediation Framework ◆ Apache Axis2 Clustering ◆ Apache Tribes Group Management ◆ Binary Relay Message Builder ◆ Load Balance Endpoint Module ◆ Auto-scaling Module 2
  3. 3. Agenda (cont.) ➔ ELB’s role in Apache Stratos PaaS ◆ Apache Stratos Logical Architecture ◆ Workflow ➔ Auto-scaling Process ◆ Configuration ◆ Auto-scaling Algorithm ◆ Plugging in Custom Algorithms ➔ How to Avoid Single Point of Failure of ELB 3
  4. 4. Introduction to Load Balancing How it is done & why it matters? 4
  5. 5. What is Load Balancing in general? Load balancing is a computer networking method for distributing workloads across multiple computers or a computer cluster, network links, central processing units, disk drives, or other resources - Wikipedia http://en.wikipedia.org/wiki/Load_balancer 5
  6. 6. Load balancing is used to distribute the incoming traffic amongst a set of backend worker nodes which are statically configured or dynamically discovered. http://docs.wso2.org/wiki/display/ELB203/Load+Balacing+Basics LB W1 W2 Wn Clients Worker nodes Incoming Traffic 6 What is Load Balancing in Middleware?
  7. 7. Load Balancing Clusters A cluster is a set of nodes that communicate with each other and work towards a common goal. 7
  8. 8. Membership Schemes Static Dynamic Configuration Modes ● Only a predefined set of nodes could exist in a cluster. ● New nodes cannot be added at runtime. ● Membership is not predefined. ● Nodes could discover the load balancer. ● Nodes could join a cluster by specifying a cluster name. Hybrid ● Also called Well-Known Addressed (WKA). ● A cluster could have set of well known members. ● Nodes could join a cluster via a well-known member. 8
  9. 9. Most widely used Load Balancing Algorithms ➔ Round Robin ◆ Passes each new connection request to the next server in line ➔ Weighted Round Robin ◆ The number of connections that each machine receives over time is proportionate to a ratio weight you define. ➔ Least Connections ◆ Passes a new connection to the server that has the least number of current connections. https://devcentral.f5.com/articles/intro-to-load-balancing-for-developers- ndash-the-algorithms 9
  10. 10. Why it is called Elastic? Load BalancerAutoscaler + Monitor the incoming traffic and scales request handling capacity (no of nodes) Elastic Load Balancer= Distribute the load of incoming traffic amongst a set of worker nodes 10
  11. 11. What is the Purpose? The motivation of load balancing is to ➔ Optimize resource usage ◆ Start and stop resources on demand. ➔ Maximize the throughput ◆ Increase the average rate of successful message delivery. ➔ Minimize the response time ◆ Reduce the time it takes to process a message and send a response back. http://en.wikipedia.org/wiki/Load_balancer 11
  12. 12. Main Features of a Load Balancer There are three main features: ➔ Failover Handling ◆ Avoid single point of failure by hosting multiple instances of a given service. ➔ Auto-scaling ◆ Manage number of instances of an application according to the incoming traffic. ➔ Multi-tenancy ◆ Manage multiple tenants of applications. 12
  13. 13. Component Architecture of Apache Stratos ELB How it was designed 13
  14. 14. Component Architecture 14
  15. 15. Apache Synapse Mediation Framework 15
  16. 16. How Synapse was Extended 16
  17. 17. Binary Relay Message Builder ● Synapse uses Axis2 engine for message processing. ● Axis2 uses Message Formatters & Message Builders for serializing and building incoming messages into SOAP format. ● Binary Relay is an Axis2 message builder which pass through all messages without processing them. Binary Relay Message Builder 17
  18. 18. Binary Relay definition in Axis2.xml elb/repository/conf/axis2/axis2.xml <messageBuilders> <messageBuilder contentType="application/xml" class="org.wso2.carbon.relay.BinaryRelayBuilder"/> <messageBuilder contentType="application/txt" class="org.wso2.carbon.relay.BinaryRelayBuilder"/> . . . </messageBuilders> Binary Relay Message Builder 18
  19. 19. Load Balance Endpoint Module ● Tenant Aware Load Balance Endpoint ○ Extends Synapse Dynamic Load Balance Endpoint. ○ Utilizes round robin load balance algorithm. ● Topology Syncher ○ Receives service cluster topology information from Cloud Controller via the Message Broker. ● Health Checker ○ Re-establishes connection to the Message Broker if it drops. Load Balance Endpoint Module 19
  20. 20. Load Balance Endpoint Module (cont.) ● Cluster Domain Manager Impl ○ Manages cluster sub domains of cartridge instances. ● Group Mgt Agent Builder ○ Manages Axis2 group management agents of cluster sub domains. ● Registry Manager ○ Receives domain mappings of cartridge instances from ADC manager via the registry. Load Balance Endpoint Module 20
  21. 21. Session Affinity ● There are two different ways to manage session information. ● Replicate in cluster is a very costly process. ● Therefore ELB manages session information for the applications. Session Information Replicate in Cluster Handled by LB Load Balance Endpoint Module 21
  22. 22. Auto-scaling Module ● Autoscale In Mediator ○ Generates a token (request id) per request received and adds it to a queue. ● Autoscale Out Mediator ○ Removes the token added by the in mediator when a response is received from the end point. ● Service Requests InFlight Autoscaler (Task) ○ Performs sanity checks to ensure that all clusters meet the minimum number of nodes. ○ Performs scaling based on the request load & scaling configuration parameters. Auto-scaling Module 22
  23. 23. ELB’s role in Apache Stratos PaaS How does it contribute? 23
  24. 24. ELB’s role in Apache Stratos PaaS 24
  25. 25. Cartridge Subscription Workflow 1. [SC -> CC] Send Cartridge Subscription Request 2. [CC] Deploy Cartridge Instance Service 3. [CC -> MB] Publish Cluster Topology Information 4. [MB -> ELB] Receive Cluster Topology Information 5. [CC -> jclouds] Instance Spawn Request 6. [jclouds -> IaaS] Spawn Cartridge Instance 7. [Cartridge -> Agent] Request to Join Cluster 8. [Agent -> ELB] Add Node to Cluster 25
  26. 26. 1. [Client -> ELB] Send request message 2. [ELB] Identify cluster & tenant using message header 3. [ELB] Add request to a list 4. [ELB -> Node] If session exists, send message 5. [ELB] If not store session information 6. [ELB -> Node] Apply algorithm & send message 7. [ELB -> Node] Handle failover 8. [Node -> ELB] Send response 9. [ELB -> Client] Send response and remove request from list 10. [ELB] Scale number of cartridge instances Load Balancing Workflow 26
  27. 27. Load Balancer Configuration loadbalancer.conf loadbalancer { # minimum number of load balancer instances instances 1; # whether autoscaling should be enabled or not. enable_autoscaler true; # autoscaling decision making task #autoscaler_task org.wso2.carbon.mediator.autoscale.lbautoscale.task. ServiceRequestsInFlightAutoscaler; #please use this whenever url-mapping is used through LB. #size_of_cache 100; ... 27
  28. 28. Load Balancer Configuration (cont.) loadbalancer { ... # Endpoint reference of the Autoscaler Service. This should be present, if you disabled embedded autoscaling. #autoscaler_service_epr https://host_address: https_port/services/AutoscalerService/; # interval between two task executions in milliseconds autoscaler_task_interval 60000; # after an instance booted up, task will wait maximum till this much of time and let the server started up server_startup_delay 180000; #default will be 60000ms # session timeout session_timeout 90000; # enable failover fail_over true; } 28
  29. 29. Port Mapping ● Ports of applications deployed in cartridge instances are mapped to external ports by the load balancer. ● Port mapping is defined in <cartridge>.xml file. ● Example: <portMapping> <http port="80" proxyPort="8280"/> <https port="443" proxyPort="8243"/> </portMapping> 29
  30. 30. Auto-scaling Process Configuration & custom implementations 30
  31. 31. Auto-scaling Configuration loadbalancer.conf services { # default parameter values to be used in all services defaults { # minimum number of service instances required min_app_instances 1; # maximum number of service instances that will be load balanced max_app_instances 3; # maximum number of requests served per second by a service instance max_requests_per_second 5; # scale up early using AUR, 0 < AUR <= 1 and default is 0.7 alarming_upper_rate 0.7; # scale down slowly using ALR, 0 < ALR <= 1 and default is 0.2 alarming_lower_rate 0.2; # scale down slowly using SDF, 0 < SDF <= 1 and default is 0.25 scale_down_factor 0.25; # no of iterations in-flight avg is calculated to take the decision rounds_to_average 2; message_expiry_time 60000; } 31
  32. 32. Auto-scaling Algorithm https://cwiki.apache.org/confluence/display/STRATOS/Elastic+Load+Balancer 32
  33. 33. Custom Auto-scaling Implementation ● You could write your own Java implementation which implements org.apache.synapse.task.Task and org.apache. synapse.ManagedLifecycle interfaces. ● Wrap the implementation class to an OSGi bundle and deploy it in the ELB OSGi container. ● Update autoscaler_task value in loadbalancer.conf. 33
  34. 34. How to avoid Single Point of Failure of ELB An ELB is prone to single point of failure 34
  35. 35. Clustered ELB Deployment Architecture 35
  36. 36. Questions? Or any feedback is really appreciated 36
  37. 37. References ● Apache Stratos Documentation https://cwiki.apache.org/confluence/display/STRATOS/Index ● WSO2 Load Balancer - How it works http://blog.afkham.org/2011/09/wso2-load-balancer-how-it-works.html ● WSO2 Webinar - Introducing the WSO2 Load Balancer http://www.youtube.com/watch?v=HgiTyvZ-Sfw ● WSO2 Elastic Load Balancer Documentation http://docs.wso2. org/wiki/display/ELB203/WSO2+Elastic+Load+Balancer+Documentation ● WSO2, Introduction to Carbon Clustering http://wso2.com/library/articles/introduction-wso2-carbon-clustering 37
  38. 38. Website http://stratos.incubator.apache.org Mailing List Subscribe: dev-subscribe@stratos.incubator.apache.org Post (After subscription): dev@stratos.incubator.apache.org Social Media Google+: https://plus.google.com/103515557134069849802 Twitter: https://twitter.com/ApacheStratos Facebook: https://www.facebook.com/apache.stratos LinkedIn: http://www.linkedin.com/groups?home=&gid=5131436 38 Join Us
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×