Your SlideShare is downloading. ×
The Role of Elastic Load Balancer - Apache Stratos
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The Role of Elastic Load Balancer - Apache Stratos

5,053
views

Published on

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,053
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
31
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. The Role of Elastic Load Balancer (ELB) Imesh Gunaratne Apache Contributor, Technical Lead - WSO2 Inc
  • 2. Agenda ➔ Introduction to Load Balancing ◆ What is Load Balancing? ◆ Algorithms ◆ Node Configuration Modes ◆ Why is it called Elastic? ◆ Purpose ◆ Features ➔ Component Architecture of Apache Stratos ELB ◆ Synapse Mediation Framework ◆ Apache Axis2 Clustering ◆ Apache Tribes Group Management ◆ Binary Relay Message Builder ◆ Load Balance Endpoint Module ◆ Auto-scaling Module 2
  • 3. Agenda (cont.) ➔ ELB’s role in Apache Stratos PaaS ◆ Apache Stratos Logical Architecture ◆ Workflow ➔ Auto-scaling Process ◆ Configuration ◆ Auto-scaling Algorithm ◆ Plugging in Custom Algorithms ➔ How to Avoid Single Point of Failure of ELB 3
  • 4. Introduction to Load Balancing How it is done & why it matters? 4
  • 5. What is Load Balancing in general? Load balancing is a computer networking method for distributing workloads across multiple computers or a computer cluster, network links, central processing units, disk drives, or other resources - Wikipedia http://en.wikipedia.org/wiki/Load_balancer 5
  • 6. Load balancing is used to distribute the incoming traffic amongst a set of backend worker nodes which are statically configured or dynamically discovered. http://docs.wso2.org/wiki/display/ELB203/Load+Balacing+Basics LB W1 W2 Wn Clients Worker nodes Incoming Traffic 6 What is Load Balancing in Middleware?
  • 7. Load Balancing Clusters A cluster is a set of nodes that communicate with each other and work towards a common goal. 7
  • 8. Membership Schemes Static Dynamic Configuration Modes ● Only a predefined set of nodes could exist in a cluster. ● New nodes cannot be added at runtime. ● Membership is not predefined. ● Nodes could discover the load balancer. ● Nodes could join a cluster by specifying a cluster name. Hybrid ● Also called Well-Known Addressed (WKA). ● A cluster could have set of well known members. ● Nodes could join a cluster via a well-known member. 8
  • 9. Most widely used Load Balancing Algorithms ➔ Round Robin ◆ Passes each new connection request to the next server in line ➔ Weighted Round Robin ◆ The number of connections that each machine receives over time is proportionate to a ratio weight you define. ➔ Least Connections ◆ Passes a new connection to the server that has the least number of current connections. https://devcentral.f5.com/articles/intro-to-load-balancing-for-developers- ndash-the-algorithms 9
  • 10. Why it is called Elastic? Load BalancerAutoscaler + Monitor the incoming traffic and scales request handling capacity (no of nodes) Elastic Load Balancer= Distribute the load of incoming traffic amongst a set of worker nodes 10
  • 11. What is the Purpose? The motivation of load balancing is to ➔ Optimize resource usage ◆ Start and stop resources on demand. ➔ Maximize the throughput ◆ Increase the average rate of successful message delivery. ➔ Minimize the response time ◆ Reduce the time it takes to process a message and send a response back. http://en.wikipedia.org/wiki/Load_balancer 11
  • 12. Main Features of a Load Balancer There are three main features: ➔ Failover Handling ◆ Avoid single point of failure by hosting multiple instances of a given service. ➔ Auto-scaling ◆ Manage number of instances of an application according to the incoming traffic. ➔ Multi-tenancy ◆ Manage multiple tenants of applications. 12
  • 13. Component Architecture of Apache Stratos ELB How it was designed 13
  • 14. Component Architecture 14
  • 15. Apache Synapse Mediation Framework 15
  • 16. How Synapse was Extended 16
  • 17. Binary Relay Message Builder ● Synapse uses Axis2 engine for message processing. ● Axis2 uses Message Formatters & Message Builders for serializing and building incoming messages into SOAP format. ● Binary Relay is an Axis2 message builder which pass through all messages without processing them. Binary Relay Message Builder 17
  • 18. Binary Relay definition in Axis2.xml elb/repository/conf/axis2/axis2.xml <messageBuilders> <messageBuilder contentType="application/xml" class="org.wso2.carbon.relay.BinaryRelayBuilder"/> <messageBuilder contentType="application/txt" class="org.wso2.carbon.relay.BinaryRelayBuilder"/> . . . </messageBuilders> Binary Relay Message Builder 18
  • 19. Load Balance Endpoint Module ● Tenant Aware Load Balance Endpoint ○ Extends Synapse Dynamic Load Balance Endpoint. ○ Utilizes round robin load balance algorithm. ● Topology Syncher ○ Receives service cluster topology information from Cloud Controller via the Message Broker. ● Health Checker ○ Re-establishes connection to the Message Broker if it drops. Load Balance Endpoint Module 19
  • 20. Load Balance Endpoint Module (cont.) ● Cluster Domain Manager Impl ○ Manages cluster sub domains of cartridge instances. ● Group Mgt Agent Builder ○ Manages Axis2 group management agents of cluster sub domains. ● Registry Manager ○ Receives domain mappings of cartridge instances from ADC manager via the registry. Load Balance Endpoint Module 20
  • 21. Session Affinity ● There are two different ways to manage session information. ● Replicate in cluster is a very costly process. ● Therefore ELB manages session information for the applications. Session Information Replicate in Cluster Handled by LB Load Balance Endpoint Module 21
  • 22. Auto-scaling Module ● Autoscale In Mediator ○ Generates a token (request id) per request received and adds it to a queue. ● Autoscale Out Mediator ○ Removes the token added by the in mediator when a response is received from the end point. ● Service Requests InFlight Autoscaler (Task) ○ Performs sanity checks to ensure that all clusters meet the minimum number of nodes. ○ Performs scaling based on the request load & scaling configuration parameters. Auto-scaling Module 22
  • 23. ELB’s role in Apache Stratos PaaS How does it contribute? 23
  • 24. ELB’s role in Apache Stratos PaaS 24
  • 25. Cartridge Subscription Workflow 1. [SC -> CC] Send Cartridge Subscription Request 2. [CC] Deploy Cartridge Instance Service 3. [CC -> MB] Publish Cluster Topology Information 4. [MB -> ELB] Receive Cluster Topology Information 5. [CC -> jclouds] Instance Spawn Request 6. [jclouds -> IaaS] Spawn Cartridge Instance 7. [Cartridge -> Agent] Request to Join Cluster 8. [Agent -> ELB] Add Node to Cluster 25
  • 26. 1. [Client -> ELB] Send request message 2. [ELB] Identify cluster & tenant using message header 3. [ELB] Add request to a list 4. [ELB -> Node] If session exists, send message 5. [ELB] If not store session information 6. [ELB -> Node] Apply algorithm & send message 7. [ELB -> Node] Handle failover 8. [Node -> ELB] Send response 9. [ELB -> Client] Send response and remove request from list 10. [ELB] Scale number of cartridge instances Load Balancing Workflow 26
  • 27. Load Balancer Configuration loadbalancer.conf loadbalancer { # minimum number of load balancer instances instances 1; # whether autoscaling should be enabled or not. enable_autoscaler true; # autoscaling decision making task #autoscaler_task org.wso2.carbon.mediator.autoscale.lbautoscale.task. ServiceRequestsInFlightAutoscaler; #please use this whenever url-mapping is used through LB. #size_of_cache 100; ... 27
  • 28. Load Balancer Configuration (cont.) loadbalancer { ... # Endpoint reference of the Autoscaler Service. This should be present, if you disabled embedded autoscaling. #autoscaler_service_epr https://host_address: https_port/services/AutoscalerService/; # interval between two task executions in milliseconds autoscaler_task_interval 60000; # after an instance booted up, task will wait maximum till this much of time and let the server started up server_startup_delay 180000; #default will be 60000ms # session timeout session_timeout 90000; # enable failover fail_over true; } 28
  • 29. Port Mapping ● Ports of applications deployed in cartridge instances are mapped to external ports by the load balancer. ● Port mapping is defined in <cartridge>.xml file. ● Example: <portMapping> <http port="80" proxyPort="8280"/> <https port="443" proxyPort="8243"/> </portMapping> 29
  • 30. Auto-scaling Process Configuration & custom implementations 30
  • 31. Auto-scaling Configuration loadbalancer.conf services { # default parameter values to be used in all services defaults { # minimum number of service instances required min_app_instances 1; # maximum number of service instances that will be load balanced max_app_instances 3; # maximum number of requests served per second by a service instance max_requests_per_second 5; # scale up early using AUR, 0 < AUR <= 1 and default is 0.7 alarming_upper_rate 0.7; # scale down slowly using ALR, 0 < ALR <= 1 and default is 0.2 alarming_lower_rate 0.2; # scale down slowly using SDF, 0 < SDF <= 1 and default is 0.25 scale_down_factor 0.25; # no of iterations in-flight avg is calculated to take the decision rounds_to_average 2; message_expiry_time 60000; } 31
  • 32. Auto-scaling Algorithm https://cwiki.apache.org/confluence/display/STRATOS/Elastic+Load+Balancer 32
  • 33. Custom Auto-scaling Implementation ● You could write your own Java implementation which implements org.apache.synapse.task.Task and org.apache. synapse.ManagedLifecycle interfaces. ● Wrap the implementation class to an OSGi bundle and deploy it in the ELB OSGi container. ● Update autoscaler_task value in loadbalancer.conf. 33
  • 34. How to avoid Single Point of Failure of ELB An ELB is prone to single point of failure 34
  • 35. Clustered ELB Deployment Architecture 35
  • 36. Questions? Or any feedback is really appreciated 36
  • 37. References ● Apache Stratos Documentation https://cwiki.apache.org/confluence/display/STRATOS/Index ● WSO2 Load Balancer - How it works http://blog.afkham.org/2011/09/wso2-load-balancer-how-it-works.html ● WSO2 Webinar - Introducing the WSO2 Load Balancer http://www.youtube.com/watch?v=HgiTyvZ-Sfw ● WSO2 Elastic Load Balancer Documentation http://docs.wso2. org/wiki/display/ELB203/WSO2+Elastic+Load+Balancer+Documentation ● WSO2, Introduction to Carbon Clustering http://wso2.com/library/articles/introduction-wso2-carbon-clustering 37
  • 38. Website http://stratos.incubator.apache.org Mailing List Subscribe: dev-subscribe@stratos.incubator.apache.org Post (After subscription): dev@stratos.incubator.apache.org Social Media Google+: https://plus.google.com/103515557134069849802 Twitter: https://twitter.com/ApacheStratos Facebook: https://www.facebook.com/apache.stratos LinkedIn: http://www.linkedin.com/groups?home=&gid=5131436 38 Join Us

×