Scalable SOA



   Samisa Abeysinghe


WSO2 Summer School 2009
Outline
●Introduction
●Clustering

●Load Balancing

●SOA in the cloud

●Summary and Resources
Outline - 1
Introduction
●

    ●   Scalability and Availability
Outline - 2
Clustering
●

    ●   Clustering Techniques
    ●   Interfaces to be used
    ●   Membership Schemes
    ●   S...
Outline - 3
Load Balancing
●

    ●   Stateless load balancing
    ●   Stateful load balancing
    ●   Load Balancing with...
Outline - 4
SOA in the cloud
●

    ●   AMI deployments
    ●   Auto Scaling
Clustering
Web services clustering
●   Different servers
    providing the same
    service
●   Group communication
    among the ser...
Motivation for Clustering
●Achieving high availability and scalability
●Enterprise Service Buses (ESB)

    ●   De-facto s...
Scalability
Ability to
  Handle growing amounts of work in a
  graceful manner
Or
  Be readily enlarged
Availability
The degree to which a system, subsystem, or
●

  equipment is
 ● Operable and in a committable state at the

...
A Computer Cluster is...
●A group of coupled computers that works closely
  together
●As if they were a single computer
Clustering Basics
Members or nodes of a cluster
●

    ●   Connected to each other through fast local area
         networ...
An ESB Cluster
Performance with a Cluster
●Install the software on different computers in your
   network
●Spread work amongst those

●Ac...
Performance with a Cluster:
                 Scaling
Increasing the numbers of nodes in a cluster
●

    ●   Decrease the ...
Scaling with a Cluster
Availability with a Cluster
●Fault-tolerance and failover
●Component failures detected by the cluster

●Repeats the failed...
Availability with a Cluster
Clustering with Apache Axis2

●The Web services engine has comprehensive
  SOA support
●Many other projects make use of Ax...
Apache Axis2 Clustering
●Functionality is abstracted out by a set of
  interfaces
●Makes clustering pluggable

    ●   Imp...
Axis2 Clustering API
Address 2 concerns
●

    ●   State replication
         ●   Synchronizing of states related to diffe...
Axis2 Clustering Interfaces
●org.apache.axis2.clustering.ClusteringAgent
●org.apache.axis2.clustering.state.StateManager

...
org.apache.axis2.clustering.Clus
             teringAgent
Responsible for initializing all clustering related
●

 function...
org.apache.axis2.clustering.Cl
           usteringAgent
In axis2.xml, the instance of this interface is
●

  specified usi...
org.apache.axis2.clustering.stat
           e.StateManager
●Responsible for handling state replication
●Property changes (...
org.apache.axis2.clustering.sta
               te.StateManager
●org.apache.axis2.clustering.state.Replicator#repli
    cat...
org.apache.axis2.clustering.sta
           te.StateManager
The implementation of this interface is set by
●

 reading the ...
org.apache.axis2.clustering.man
        agement.NodeManager
●Responsible for handling management of a
  particular member
...
org.apache.axis2.clustering.ma
    nagement.GroupManagementA
                gent
●Responsible for sending group managemen...
Group Management Mode
●Member is special and belongs to all groups
   which it is managing
●Any membership changes will be...
Membership Schemes
●Static
●Dynamic

●Hybrid
Static Membership
●Only a defined set of members can be in a
  group
●Group Membership Service(GMS) will detect

  members...
Dynamic Membership
●Membership is not predefined
●Members can join a group by specifying the

  appropriate group name, an...
Hybrid Membership
There are a set of well-known members
●

    ●   Can consider these members as belonging to a static
   ...
Load Balancing
Dynamic Load Balancing with
               ESB
●New members can join and leave the application
  group
●This will be refle...
Apache Synapse/WSO2 ESB
  Config Sample LB endpoint

<endpoint name=”test-lb”>
  <session type=”soap”/>
  <loadbalance pol...
Architecture
●   Stateless load balancing
    ●   LB algorithm implementation is pluggable
    ●   Built-in round robin al...
State-full Session Aware Load
               Balancing
●   Transport and SOAP session




                   Load Balancer
Fail-Over with LB




     Load Balancer
LB and FO Groups




     Load Balancer
Dynamic LB




Load Balancer
Dynamic LB (Cntd..)




    Load Balancer
Scalability of WSO2 ESB
●   Can handle 2500 concurrent connections
●   Can handle 30M transactions per day
    ●   assumpt...
Availability of WSO2 ESB
●   Availability can be achieved with the
    deployment
     ●   Two passive nodes with a given ...
Deployment diagram
Cloud Computing
Cloud Computing
Characteristics & Benefits to SOA
●

    ●   Auto-scaling (elasticity)
    ●   Multi-tenancy
    ●   Scala...
AMIs for the Cloud
All WSO2 products would be available as
●

 AMIs soon
    ●   Compatible with Amazon EC2
    ●   Ready ...
Auto Scaling - Elasticity
●Ability for the resources to be scaled up, and down, based on
   demand
●Enables efficient use ...
Auto Scaling Services
WSO2 WSAS/ESB Approach
Auto Scaling Services
WSO2 WSAS/ESB Approach




                Load Increases
Auto Scaling Services
WSO2 WSAS/ESB Approach

                Startup new instances
Auto Scaling Services
WSO2 WSAS/ESB Approach

                New instances join group
Auto Scaling Services
WSO2 WSAS/ESB Approach

                 Load Decreases
Auto Scaling Services
WSO2 WSAS/ESB Approach

                Terminate instances
Summary
●  Enterprises thrive for scalability and availability
 ● Clustering provides us a perfect solution to

   meet en...
Summary
●   ESB is a key element in SOA deployments
    when it comes to load balancing and fail over
       ●   WSO2 ESB ...
Resources
● Introduction to WSO2 Carbon Clustering -
     http://wso2.org/library/articles/introduction-wso2-carbon-cluste...
Tell a Friend...
●   If you found this useful...
    ●   Give us your feedback
    ●   Blog
    ●   Tweet
    ●   Spread t...
Scalable Soa
Upcoming SlideShare
Loading in …5
×

Scalable Soa

2,197 views

Published on

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,197
On SlideShare
0
From Embeds
0
Number of Embeds
166
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Scalable Soa

  1. 1. Scalable SOA Samisa Abeysinghe WSO2 Summer School 2009
  2. 2. Outline ●Introduction ●Clustering ●Load Balancing ●SOA in the cloud ●Summary and Resources
  3. 3. Outline - 1 Introduction ● ● Scalability and Availability
  4. 4. Outline - 2 Clustering ● ● Clustering Techniques ● Interfaces to be used ● Membership Schemes ● State Management ● Node Management ● Group Management ● Clustering Patterns
  5. 5. Outline - 3 Load Balancing ● ● Stateless load balancing ● Stateful load balancing ● Load Balancing with fail over ● Dynamic load balancing
  6. 6. Outline - 4 SOA in the cloud ● ● AMI deployments ● Auto Scaling
  7. 7. Clustering
  8. 8. Web services clustering ● Different servers providing the same service ● Group communication among the servers ● Shared resources for the cluster
  9. 9. Motivation for Clustering ●Achieving high availability and scalability ●Enterprise Service Buses (ESB) ● De-facto standard for integrating autonomous entities ● Could become the bottleneck in the system ● High availability and high scalability critical ● Ability to cluster is an essential functionality for an ESB in a SOA deployment Service hosting engines ● ● Needs to be able to plug into clusters
  10. 10. Scalability Ability to Handle growing amounts of work in a graceful manner Or Be readily enlarged
  11. 11. Availability The degree to which a system, subsystem, or ● equipment is ● Operable and in a committable state at the start of a mission ● When the mission is called for at an unknown
  12. 12. A Computer Cluster is... ●A group of coupled computers that works closely together ●As if they were a single computer
  13. 13. Clustering Basics Members or nodes of a cluster ● ● Connected to each other through fast local area networks ● Redundant hardware and/or software resources ● Increase the availability and/or performance of the whole system Fault-tolerant components ● ● Share any existing state information among members ● Redundant component is able to take over the work of another
  14. 14. An ESB Cluster
  15. 15. Performance with a Cluster ●Install the software on different computers in your network ●Spread work amongst those ●Achieved using software or hardware load balancers ● Distribute incoming requests to different nodes of a cluster ● Based on a very simple or sophisticated strategy/algorithm ● Most common and the very basic strategy is round-robin
  16. 16. Performance with a Cluster: Scaling Increasing the numbers of nodes in a cluster ● ● Decrease the average load on each node ● Able to handle more work in a given time period ● Scale to your performance requirements
  17. 17. Scaling with a Cluster
  18. 18. Availability with a Cluster ●Fault-tolerance and failover ●Component failures detected by the cluster ●Repeats the failed operation on an identical node residing in the cluster ● Request processed should be completely stateless ● Does not work if an operation is stateful ● The node is required to maintain state information between subsequent invocations ● Stateful component needs to communicate its state information across all identical components within the cluster - State replication
  19. 19. Availability with a Cluster
  20. 20. Clustering with Apache Axis2 ●The Web services engine has comprehensive SOA support ●Many other projects make use of Axis2 clustering ● Apache Synapse ● WSO2 Carbon – The SOA Platform
  21. 21. Apache Axis2 Clustering ●Functionality is abstracted out by a set of interfaces ●Makes clustering pluggable ● Implement provided interfaces ● Configure implementation classes in the clustering configuration section of axis2.xml Default clustering implementation in Axis2 ● ● Based on Apache Tribes ● Group management framework used by Apache Tomcat
  22. 22. Axis2 Clustering API Address 2 concerns ● ● State replication ● Synchronizing of states related to different members in a cluster group ● Management ● Group Management ● Node Management
  23. 23. Axis2 Clustering Interfaces ●org.apache.axis2.clustering.ClusteringAgent ●org.apache.axis2.clustering.state.StateManager ●org.apache.axis2.clustering.management.NodeM anager ●org.apache.axis2.clustering.management.Group ManagementAgent
  24. 24. org.apache.axis2.clustering.Clus teringAgent Responsible for initializing all clustering related ● functionalities of a node ● Also responsible for getting the node to join the cluster ● Node should not process any Web services requests until it successfully joins the cluster ● Node will also need to obtain state information and/or configuration information from a neighboring node ● To be in sync with the other members ● Configuration changes might have taken place
  25. 25. org.apache.axis2.clustering.Cl usteringAgent In axis2.xml, the instance of this interface is ● specified using the "clustering" class attribute.
  26. 26. org.apache.axis2.clustering.stat e.StateManager ●Responsible for handling state replication ●Property changes (state changes) in the Axis2 context hierarchy in the node which runs this StateManager will be propagated to all other nodes in its group ●Generally, Web services authors do not have to handle state replication, as it is handled by the clustering implementation just before request completion.
  27. 27. org.apache.axis2.clustering.sta te.StateManager ●org.apache.axis2.clustering.state.Replicator#repli cate() can be called to force state replication. ●It is not mandatory to have a StateManager in a node ● If we are not interested in high availability, the purpose of a cluster is to achieve scalability ● Simply to utilize the group communication capabilities of the underlying Group Communication Framework (GCF) ● The purpose of a clustered deployment may also be management of the cluster using the underlying GCF ● WSO2 Carbon provides such a component which allows cluster management.
  28. 28. org.apache.axis2.clustering.sta te.StateManager The implementation of this interface is set by ● reading the "stateManager" element in the axis2.xml clustering section.
  29. 29. org.apache.axis2.clustering.man agement.NodeManager ●Responsible for handling management of a particular member ●Not mandatory to have a NodeManager in a node ●Node management is generally used for deploying or undeploying services across a cluster group ●The implementation of this interface is set by reading the "nodeManager" element in the axis2.xml
  30. 30. org.apache.axis2.clustering.ma nagement.GroupManagementA gent ●Responsible for sending group management events and messages to members in a particular group ●Only used when a member is running in group management mode
  31. 31. Group Management Mode ●Member is special and belongs to all groups which it is managing ●Any membership changes will be notified by the underlying group management framework to the group management agent ●A group management agent should be specified for each group that has to be managed by the member ●In WSO2 Carbon cluster management, the cluster manager node needs to define group management agents for each group
  32. 32. Membership Schemes ●Static ●Dynamic ●Hybrid
  33. 33. Static Membership ●Only a defined set of members can be in a group ●Group Membership Service(GMS) will detect members joining or leaving the group ●External members cannot join the group ●Each node may obtain group member details from a central repository or configuration file
  34. 34. Dynamic Membership ●Membership is not predefined ●Members can join a group by specifying the appropriate group name, and also leave the group ●Group Management Service (GMS) will detect new members joining or leaving ●Group membership information may be obtained from the GMS
  35. 35. Hybrid Membership There are a set of well-known members ● ● Can consider these members as belonging to a static group External members can join this group by notifying ● one of the well known members ● These external members can then get the current group membership from this well-known member ●When new members join the group, well-known members will notify all other members ●When members leave the group, the GMS can detect this event
  36. 36. Load Balancing
  37. 37. Dynamic Load Balancing with ESB ●New members can join and leave the application group ●This will be reflected in the membership aware dynamic load balancer ●The clustering configuration for the dynamic load balancer needs to be configured with group management to allow group membership discovery
  38. 38. Apache Synapse/WSO2 ESB Config Sample LB endpoint <endpoint name=”test-lb”> <session type=”soap”/> <loadbalance policy=”roundRobin”> <endpoint ..../> .. .. </loadbalance> </endpoint>
  39. 39. Architecture ● Stateless load balancing ● LB algorithm implementation is pluggable ● Built-in round robin algorithm ● Can configure more advanced algorithms ● e.g. Weighted round robin LB algorithm
  40. 40. State-full Session Aware Load Balancing ● Transport and SOAP session Load Balancer
  41. 41. Fail-Over with LB Load Balancer
  42. 42. LB and FO Groups Load Balancer
  43. 43. Dynamic LB Load Balancer
  44. 44. Dynamic LB (Cntd..) Load Balancer
  45. 45. Scalability of WSO2 ESB ● Can handle 2500 concurrent connections ● Can handle 30M transactions per day ● assumption: one transaction is one request for ESB ● Non-blocking HTTP/S transport with message queuing and configurable thread pools ● Different thread pools at the application layer and the I/O layer
  46. 46. Availability of WSO2 ESB ● Availability can be achieved with the deployment ● Two passive nodes with a given active node ● Graceful shutdown and maintenance mode ● Shutdown the listeners ● Let the senders send out the responses after processing the already accepted messages ● Shutdown the senders and the server or upgrade the system and restart the listener manager ● Round robin restart of the cluster in active active deployment
  47. 47. Deployment diagram
  48. 48. Cloud Computing
  49. 49. Cloud Computing Characteristics & Benefits to SOA ● ● Auto-scaling (elasticity) ● Multi-tenancy ● Scalable parallel computing ● Other benefits include ● Cost savings by not peak provisioning ● Faster deployment ● Outsource non-core skills ● Utility billing
  50. 50. AMIs for the Cloud All WSO2 products would be available as ● AMIs soon ● Compatible with Amazon EC2 ● Ready to be deployed
  51. 51. Auto Scaling - Elasticity ●Ability for the resources to be scaled up, and down, based on demand ●Enables efficient use of computer resources ● No need to provision for maximum load ● Requires application or middleware to do work to manage load – Queue up incoming requests – Distribute amongst available worker nodes – Monitor queue length and adjust worker pool accordingly
  52. 52. Auto Scaling Services WSO2 WSAS/ESB Approach
  53. 53. Auto Scaling Services WSO2 WSAS/ESB Approach Load Increases
  54. 54. Auto Scaling Services WSO2 WSAS/ESB Approach Startup new instances
  55. 55. Auto Scaling Services WSO2 WSAS/ESB Approach New instances join group
  56. 56. Auto Scaling Services WSO2 WSAS/ESB Approach Load Decreases
  57. 57. Auto Scaling Services WSO2 WSAS/ESB Approach Terminate instances
  58. 58. Summary ● Enterprises thrive for scalability and availability ● Clustering provides us a perfect solution to meet enterprise demands ● Apache Axis2 clustering support is well designed and battle tested ● Done by WSO2 team members ● Used in almost all WSO2 products
  59. 59. Summary ● ESB is a key element in SOA deployments when it comes to load balancing and fail over ● WSO2 ESB is well equipped with these capabilities ● Cloud computing is opening up new opportunities for SOA deployments ● Auto scaling one of the key benefits
  60. 60. Resources ● Introduction to WSO2 Carbon Clustering - http://wso2.org/library/articles/introduction-wso2-carbon-clustering ● WSO2 Carbon Cluster Configuration Language - http://wso2.org/library/articles/wso2-carbon-cluster-configuration-language ● WSO2 ESB/Apache Synapse Clustering Guide - http://wso2.org/library/articles/wso2-esb-apache-synapse-clustering-guide ● Achieving Scalability and High Availability for Clustered Web Services using Apache Synapse - http://wso2.org/library/presentations/apachecon-eu-2009-achieving-scalability-hig ● Autoscaling Axis2 Web Services on Amazon EC2 - http://wso2.org/library/presentations/autoscaling-axis2-web-services-amazon-ec2 ● Scalable SOA in Your Own Open Source Cloud - http://wso2.org/library/news/presentation-soa-symposium-scalable-soa-your-own-
  61. 61. Tell a Friend... ● If you found this useful... ● Give us your feedback ● Blog ● Tweet ● Spread the word :)

×