• Share
  • Email
  • Embed
  • Like
  • Private Content
Scalability Availabilty and Management of WSO2 Carbon

Scalability Availabilty and Management of WSO2 Carbon






Total Views
Views on SlideShare
Embed Views



3 Embeds 171

http://wso2.org 136
http://wso2.com 33
http://node1.wso2.org 2



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Scalability Availabilty and Management of WSO2 Carbon Scalability Availabilty and Management of WSO2 Carbon Presentation Transcript

    • Scalability, Availability & Management of WSO2 Carbon Afkham Azeez Director of Architecture WSO2 Inc 1
    • About Me• PMC member Apache Axis, Committer Synapse & Web Services• Member, Apache Software Foundation• Co-author, Axis2 Web Services• Director of Architecture, WSO2 Inc• Blog: http://blog.afkham.org• Twitter: afkham_azeez 2
    • Agenda• A brief look at the WSO2 platform• Carbon clustering for scalability & availability• WSO2 Elastic Load Balancer• Worker node & management node separation• Deployment synchronization• Logging• Lazy loading 3
    • WSO2 Offerings• WSO2 Carbon • Full platform of servers for deployment on-premise, in private or public cloud • Products share a consistent architecture and core platform services (e.g. logging, management, security, identity, caching) through OSGi and the “Carbon Core” • Includes ESB, AppServer, Data Services, Governance, Identity, Business Process, and more• WSO2 Stratos • Platform-as-a-Service (PaaS) Foundation • Supports running servers as elastic, metered, billed, multi-tenant with self-service • Including all Carbon Servers, PHP, Jetty, and a growing list through a standard Cartridge model• WSO2 StratosLive • http://stratoslive.wso2.com • WSO2’s Public PaaS • An instance of Stratos running in the cloud with all Carbon Servers available 4
    • Consistent Architecture• Carbon: A consistent set of class-leading enterprise servers • The same products run either on-premise or in the cloud, single-tenant or multi- tenant • Utilize the same Carbon core runtime for a seamless experience• Stratos: A cloud platform for enterprise, hybrid and public deployment • Extends the deployment to support full self-service, elastic scaling, metering and billing • Supports Carbon and native server runtimes • Including Java and non-Java servers such as Jetty and PHP • Re-uses the same core Carbon architecture to offer core PaaS services including: • Identity, Logging, File, Relational Storage, Column Storage, Code Deployment, etc• Both projects share a common set of OSGi modules and a core runtime architecture 5
    • WSO2 SOA Platform 6
    • WSO2 Carbon 7
    • Carbon Clustering• Membership types • Static • Dynamic • Hybrid• Membership modes • Multicast • Well-known address 8
    • Static membership• Predefined members• Other (non-predefined) nodes cannot join Static group M1 M2 N M3 M4 9
    • Dynamic membership• No predefined members• Nodes can join & leave Dynamic group M1 M2 N Join M3 M4 10
    • Hybrid membership• Some predefined (well-known) members, and some dynamic members• Nodes can join & leave• Membership revolves around the static members Hybrid group Dynamic members Static members N Join (IP, M5 M6 M1 M2 Port) M7 M3 M4 11
    • Multicast based membership management M4 M1 N Join (IP, Port) M2 M3 12
    • Well-known Address (WKA) based membership managementHybrid group Dynamic members Static members M6 M5 WK1 N WK2 Notify Join (IP, Port) M7 WK3 WK4 13
    • Multicast vs. WKAMulticast WKAAll nodes should be in the same subnet Nodes can be in different networksAll nodes should be in the same multicastdomain No multicasting requirementMulticasting should not be blockedNo fixed IP addresses or hosts required At least one well-known IP address or host requiredFailure of any member does not affect New members can join with some WKAmembership discovery nodes down, but not if all WKA nodes are downDoes not work on IaaSs such as Amazon IaaS-friendlyEC2 Requires keepalived, elastic IPs or some other mechanism for remapping IP addresses of WK members in cases of failure 14
    • Multicast vs. WKA – how to decide?• Multicast • Cluster is going to be setup in a network where multicasting is allowed• WKA • Cloud based deployment • Members are distributed across datacenters & regions • Multicasting blocked 15
    • Configuring Clustering<parameter name="membershipScheme">multicast | wka</parameter>Parameters related to multicast based membership discovery<parameter name="mcastAddress"></parameter><parameter name="mcastPort">45564</parameter><parameter name="mcastFrequency">500</parameter>Parameters that describe the local member<parameter name="domain">wso2.carbon.domain</parameter><parameter name="localMemberHost"></parameter><parameter name="localMemberPort">4000</parameter><parameter name="properties">… 16
    • Configuring ClusteringStatic/well-known members<members> <member> <hostName></hostName> <port>4000</port> </member> <member> <hostName></hostName> <port>4001</port> </member> </members>State replication<stateManager class="org.apache.axis2.clustering.state.DefaultStateManager" enable=”true"> 17
    • HTTP Session Replication• catalina-server.xml • <Cluster className="org.wso2.carbon.core.session.CarbonTomcatSimpleTcpCluster"/> • <Valve className="org.wso2.carbon.tomcat.ext.valves.CarbonTomcatSessionReplicationValve"/>• web.xml • <distributable/> 18
    • Elastic Load Balancer 2.0• New sysadmin-friendly configuration language• High performance PassThrough transport• Tenant-aware load balancing• Ability to dedicate clusters for tenants (private jet mode)• Improved auto-scaler • Separate IaaS-aware Cloud controller takes care of spawning new instances on different IaaSs 19
    • ELB Config 20
    • Passthru Transport Latency Additional time taken via router/ESB/ELB vs direct Axiom 15.00 13.00 11.00 BinaryLatency (ms) 9.00 Relay 7.00 5.00 3.00 Passthru 1.00 -1.00 10 25 50 100 250 500 Concurrent clients 21
    • Tenant-aware LB 22
    • Private Jet mode• Analogy • Economy class • no SLA management, only elasticity • Business class • elasticity plus SLA guarantees • Private Jet • Guaranteed isolated VMs or machines for a specific tenant • Still elastically scaled
    • Private Jet Mode 24
    • Auto-scaling 25
    • Management & Worker Node Separation 26
    • Management & Worker Node Separation• Proper separation of concerns - management nodes specialize in management of the setup while worker nodes specialize in serving requests to deployment artifacts• Only management nodes are authorized to add new artifacts into the system or make configuration changes• Worker nodes can only deploy artifacts & read configuration• Lower memory foot in the worker nodes because the management console related OSGi bundles are not loaded• Improved security - management nodes can be behind the internal firewall & be exposed to clients running within the organization only, while worker nodes can be exposed to external clients. 27
    • Deployment Synchronization• DepSync allows you to synchronize deployment artifacts across nodes in a cluster• Also includes meta data synchronization 28
    • Deployment Synchronization 29
    • Management Node Config – carbon.xml<DeploymentSynchronizer> AutoCommit/Checkout = RW <Enabled>true</Enabled> <AutoCommit>true</AutoCommit> <AutoCheckout>true</AutoCheckout> <RepositoryType>svn</RepositoryType> <SvnUrl></SvnUrl> <SvnUser>wso2</SvnUser> <SvnPassword>wso2123</SvnPassword> <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId></DeploymentSynchronizer> 30
    • Management Node Config – catalina-server.xml<Connector protocol="org.apache.coyote.http11.Http11NioProtocol” port="9763” proxyPort="8300” …<Connector protocol="org.apache.coyote.http11.Http11NioProtocol” port="9443" proxyPort="8263” … 31
    • Worker Node Config - Create worker nodeant createWorkerBuildfile: /Users/azeez/software/wso2carbon-core-4.0.2/bin/build.xmlcreateWorker: [input] You are about to delete all the front-end components from the server runtime. Do you really want to proceed? (y, [n]) 32
    • Worker Node Config - Start worker node./wso2server.sh -DworkerNode 33
    • WorkerNode Config – carbon.xml• <!--• Host name or IP address of the machine hosting this server• e.g. www.wso2.org,• This is will become part of the End Point Reference of the• services deployed on this server instance.• -->• <HostName>appserver.wso2.com</HostName> 34
    • Worker Node Config – carbon.xml<DeploymentSynchronizer> AutoCheckout = RO <Enabled>true</Enabled> <AutoCommit>false</AutoCommit> <AutoCheckout>true</AutoCheckout> <RepositoryType>svn</RepositoryType> <SvnUrl></SvnUrl> <SvnUser>wso2</SvnUser> <SvnPassword>wso2123</SvnPassword> <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId></DeploymentSynchronizer> 35
    • Logging 36
    • Lazy Loading• Four variants • Lazy initialization • Virtual proxy • Value holder • Ghost• Two aspect of lazy loading in Carbon • Lazy loading global configuration • Lazy loading artifacts 37
    • Ghost deployment for servicescarbon.xml - <EnableGhostDeployer>true</EnableGhostDeployer> 38
    • Lazy loading with Ghost Deployment 39
    • BAM 2.0TODO 40
    • BAM 2.0 41
    • SummaryAvailability Scalability ManagementState replication Tenant partitioning Management nodes Private jet modeSession replication Ghost deployment Logging infrastructureMultiple load balancers with BAM 2.0 architecture Deployment synchronizationkeepalived or DNS RR Autoscaling Elastic Load Balancer 42
    • ReferencesInformation on tenant-aware load balancinghttp://sanjeewamalalgoda.blogspot.com/2012/03/tenant-aware-load-balancer-is-upcoming.htmlhttp://sanjeewamalalgoda.blogspot.com/2012/05/tenant-aware-load-balancer.htmlInformation on long running performancehttp://kishanthan.wordpress.com/2012/06/27/lazy-loading-pattern-in-wso2-stratosLazy loading deployment artifactshttp://blog.afkham.org/2011/11/lazy-loading-deployment-artifacts-in.htmlScaling Stratoshttp://srinathsview.blogspot.com/2012/06/scaling-wso2-stratos.htmlhttp://blog.afkham.org/2011/09/how-to-setup-wso2-elastic-load-balancer.htmlhttp://blog.afkham.org/2011/09/wso2-load-balancer-how-it-works.html 43
    • Auto-scaler service deploymenthttp://nirmalfdo.blogspot.com/2012/07/autoscaler-service-deployment.htmlAuto-scaler servicehttp://nirmalfdo.blogspot.com/2012/07/wso2-autoscaler-service-part-i.htmlPass-through transport – performance comparisonhttp://wso2.org/library/articles/2012/03/wso2-esb-message-transfer-mechanisms-comparative- benchmarks/Automatic failover for WSO2 ELBhttp://gonesimple.org/2012/09/24/automatic-fail-over-for-wso2-elb/ 44
    • Questions? http://www.flickr.com/photos/oberazzi/ 45