Scalability Availabilty and Management of WSO2 Carbon


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Scalability Availabilty and Management of WSO2 Carbon

  1. 1. Scalability, Availability & Management of WSO2 Carbon Afkham Azeez Director of Architecture WSO2 Inc 1
  2. 2. About Me• PMC member Apache Axis, Committer Synapse & Web Services• Member, Apache Software Foundation• Co-author, Axis2 Web Services• Director of Architecture, WSO2 Inc• Blog:• Twitter: afkham_azeez 2
  3. 3. Agenda• A brief look at the WSO2 platform• Carbon clustering for scalability & availability• WSO2 Elastic Load Balancer• Worker node & management node separation• Deployment synchronization• Logging• Lazy loading 3
  4. 4. WSO2 Offerings• WSO2 Carbon • Full platform of servers for deployment on-premise, in private or public cloud • Products share a consistent architecture and core platform services (e.g. logging, management, security, identity, caching) through OSGi and the “Carbon Core” • Includes ESB, AppServer, Data Services, Governance, Identity, Business Process, and more• WSO2 Stratos • Platform-as-a-Service (PaaS) Foundation • Supports running servers as elastic, metered, billed, multi-tenant with self-service • Including all Carbon Servers, PHP, Jetty, and a growing list through a standard Cartridge model• WSO2 StratosLive • • WSO2’s Public PaaS • An instance of Stratos running in the cloud with all Carbon Servers available 4
  5. 5. Consistent Architecture• Carbon: A consistent set of class-leading enterprise servers • The same products run either on-premise or in the cloud, single-tenant or multi- tenant • Utilize the same Carbon core runtime for a seamless experience• Stratos: A cloud platform for enterprise, hybrid and public deployment • Extends the deployment to support full self-service, elastic scaling, metering and billing • Supports Carbon and native server runtimes • Including Java and non-Java servers such as Jetty and PHP • Re-uses the same core Carbon architecture to offer core PaaS services including: • Identity, Logging, File, Relational Storage, Column Storage, Code Deployment, etc• Both projects share a common set of OSGi modules and a core runtime architecture 5
  6. 6. WSO2 SOA Platform 6
  7. 7. WSO2 Carbon 7
  8. 8. Carbon Clustering• Membership types • Static • Dynamic • Hybrid• Membership modes • Multicast • Well-known address 8
  9. 9. Static membership• Predefined members• Other (non-predefined) nodes cannot join Static group M1 M2 N M3 M4 9
  10. 10. Dynamic membership• No predefined members• Nodes can join & leave Dynamic group M1 M2 N Join M3 M4 10
  11. 11. Hybrid membership• Some predefined (well-known) members, and some dynamic members• Nodes can join & leave• Membership revolves around the static members Hybrid group Dynamic members Static members N Join (IP, M5 M6 M1 M2 Port) M7 M3 M4 11
  12. 12. Multicast based membership management M4 M1 N Join (IP, Port) M2 M3 12
  13. 13. Well-known Address (WKA) based membership managementHybrid group Dynamic members Static members M6 M5 WK1 N WK2 Notify Join (IP, Port) M7 WK3 WK4 13
  14. 14. Multicast vs. WKAMulticast WKAAll nodes should be in the same subnet Nodes can be in different networksAll nodes should be in the same multicastdomain No multicasting requirementMulticasting should not be blockedNo fixed IP addresses or hosts required At least one well-known IP address or host requiredFailure of any member does not affect New members can join with some WKAmembership discovery nodes down, but not if all WKA nodes are downDoes not work on IaaSs such as Amazon IaaS-friendlyEC2 Requires keepalived, elastic IPs or some other mechanism for remapping IP addresses of WK members in cases of failure 14
  15. 15. Multicast vs. WKA – how to decide?• Multicast • Cluster is going to be setup in a network where multicasting is allowed• WKA • Cloud based deployment • Members are distributed across datacenters & regions • Multicasting blocked 15
  16. 16. Configuring Clustering<parameter name="membershipScheme">multicast | wka</parameter>Parameters related to multicast based membership discovery<parameter name="mcastAddress"></parameter><parameter name="mcastPort">45564</parameter><parameter name="mcastFrequency">500</parameter>Parameters that describe the local member<parameter name="domain">wso2.carbon.domain</parameter><parameter name="localMemberHost"></parameter><parameter name="localMemberPort">4000</parameter><parameter name="properties">… 16
  17. 17. Configuring ClusteringStatic/well-known members<members> <member> <hostName></hostName> <port>4000</port> </member> <member> <hostName></hostName> <port>4001</port> </member> </members>State replication<stateManager class="org.apache.axis2.clustering.state.DefaultStateManager" enable=”true"> 17
  18. 18. HTTP Session Replication• catalina-server.xml • <Cluster className="org.wso2.carbon.core.session.CarbonTomcatSimpleTcpCluster"/> • <Valve className="org.wso2.carbon.tomcat.ext.valves.CarbonTomcatSessionReplicationValve"/>• web.xml • <distributable/> 18
  19. 19. Elastic Load Balancer 2.0• New sysadmin-friendly configuration language• High performance PassThrough transport• Tenant-aware load balancing• Ability to dedicate clusters for tenants (private jet mode)• Improved auto-scaler • Separate IaaS-aware Cloud controller takes care of spawning new instances on different IaaSs 19
  20. 20. ELB Config 20
  21. 21. Passthru Transport Latency Additional time taken via router/ESB/ELB vs direct Axiom 15.00 13.00 11.00 BinaryLatency (ms) 9.00 Relay 7.00 5.00 3.00 Passthru 1.00 -1.00 10 25 50 100 250 500 Concurrent clients 21
  22. 22. Tenant-aware LB 22
  23. 23. Private Jet mode• Analogy • Economy class • no SLA management, only elasticity • Business class • elasticity plus SLA guarantees • Private Jet • Guaranteed isolated VMs or machines for a specific tenant • Still elastically scaled
  24. 24. Private Jet Mode 24
  25. 25. Auto-scaling 25
  26. 26. Management & Worker Node Separation 26
  27. 27. Management & Worker Node Separation• Proper separation of concerns - management nodes specialize in management of the setup while worker nodes specialize in serving requests to deployment artifacts• Only management nodes are authorized to add new artifacts into the system or make configuration changes• Worker nodes can only deploy artifacts & read configuration• Lower memory foot in the worker nodes because the management console related OSGi bundles are not loaded• Improved security - management nodes can be behind the internal firewall & be exposed to clients running within the organization only, while worker nodes can be exposed to external clients. 27
  28. 28. Deployment Synchronization• DepSync allows you to synchronize deployment artifacts across nodes in a cluster• Also includes meta data synchronization 28
  29. 29. Deployment Synchronization 29
  30. 30. Management Node Config – carbon.xml<DeploymentSynchronizer> AutoCommit/Checkout = RW <Enabled>true</Enabled> <AutoCommit>true</AutoCommit> <AutoCheckout>true</AutoCheckout> <RepositoryType>svn</RepositoryType> <SvnUrl></SvnUrl> <SvnUser>wso2</SvnUser> <SvnPassword>wso2123</SvnPassword> <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId></DeploymentSynchronizer> 30
  31. 31. Management Node Config – catalina-server.xml<Connector protocol="org.apache.coyote.http11.Http11NioProtocol” port="9763” proxyPort="8300” …<Connector protocol="org.apache.coyote.http11.Http11NioProtocol” port="9443" proxyPort="8263” … 31
  32. 32. Worker Node Config - Create worker nodeant createWorkerBuildfile: /Users/azeez/software/wso2carbon-core-4.0.2/bin/build.xmlcreateWorker: [input] You are about to delete all the front-end components from the server runtime. Do you really want to proceed? (y, [n]) 32
  33. 33. Worker Node Config - Start worker node./ -DworkerNode 33
  34. 34. WorkerNode Config – carbon.xml• <!--• Host name or IP address of the machine hosting this server• e.g.,• This is will become part of the End Point Reference of the• services deployed on this server instance.• -->• <HostName></HostName> 34
  35. 35. Worker Node Config – carbon.xml<DeploymentSynchronizer> AutoCheckout = RO <Enabled>true</Enabled> <AutoCommit>false</AutoCommit> <AutoCheckout>true</AutoCheckout> <RepositoryType>svn</RepositoryType> <SvnUrl></SvnUrl> <SvnUser>wso2</SvnUser> <SvnPassword>wso2123</SvnPassword> <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId></DeploymentSynchronizer> 35
  36. 36. Logging 36
  37. 37. Lazy Loading• Four variants • Lazy initialization • Virtual proxy • Value holder • Ghost• Two aspect of lazy loading in Carbon • Lazy loading global configuration • Lazy loading artifacts 37
  38. 38. Ghost deployment for servicescarbon.xml - <EnableGhostDeployer>true</EnableGhostDeployer> 38
  39. 39. Lazy loading with Ghost Deployment 39
  40. 40. BAM 2.0TODO 40
  41. 41. BAM 2.0 41
  42. 42. SummaryAvailability Scalability ManagementState replication Tenant partitioning Management nodes Private jet modeSession replication Ghost deployment Logging infrastructureMultiple load balancers with BAM 2.0 architecture Deployment synchronizationkeepalived or DNS RR Autoscaling Elastic Load Balancer 42
  43. 43. ReferencesInformation on tenant-aware load balancing on long running performance loading deployment artifacts Stratos 43
  44. 44. Auto-scaler service deployment service transport – performance comparison benchmarks/Automatic failover for WSO2 ELB 44
  45. 45. Questions? 45