Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scaling drupal horizontally and in cloud


Published on

Vancouver Drupal group presentation for April 25, 2013.
How to deploy Drupal on
- multiple web servers,
- multiple web and database servers, and
- how to join all that together and make site deployed on Amazon Cloud (Virtual Private Cloud) inside
- one availability zone
- multiple availability zones deployment.

Session cover details about what you need in order to get Drupal deployed on separate servers, what are issues/concerns, and how to solve them.

Published in: Technology
  • Be the first to comment

Scaling drupal horizontally and in cloud

  1. 1. scaling Drupalhorizontally and in thecloud
  2. 2. about mename: Vladimir Ilicemail: burger.boy.daddy@gmail.comtwitter: @burgerboydaddy
  3. 3. agendawhy all of this?step 1: test locally -> from one server to the server farm,step 2: multiple web and database servers,step 3: how to join all that together and make site deployed on AmazonCloud and inside Virtual Private CloudAmazon termbenefitssingle availability zonemultiple availability zones.
  4. 4. why?if you want to increase site speedif you want your site to be responsive and to workunder heavy stressif you want to be in control what goes on your server
  5. 5. get it divided / decoupleEasy to do inside local development/hostingenvironmentJust separate web, database and cache serversProblemswe can increase resources only verticallyNot all resources are used same way (web serverwill probably die before cache or MySQL)Multiple “single points of failure”
  6. 6. multiple web servers –one dbApache load balancer in frontof2-3 web servers; eachserver with integratedAPC cacheMultiple cache serversPowerful MySQL serverIn real life you can use someother LB solution (this one isgreat for proof of conceptmoments).Without dedicated file server;used bi-directional rsyncreplication
  7. 7. configuring Apache loadbalancerApache web server ships a load balancer module calledmod_proxy_balancer (since version 2.2).All you need to do is to enable this module and themodules mod_proxy and mod_proxy_http. Please notethat without mod_proxy_http, balancer just wont work.LoadModule proxy_module mod_proxy.soLoadModule proxy_http_module mod_proxy_http.soLoadModule
  8. 8. many to manyIn this case each webserver will have itsown db server.Reason for this:Higher siteavailability; if onedb server is down,second one cancontinue to servecustomers.
  9. 9. Amazon AWSWhy Amazon (business point of view)Most complete cloud solution on the market.Almost zero upfront infrastructure investmentJust-in-time infrastructurePay as you go – pay what you useConstant price dropEasy to deploy and scale….
  10. 10. why Amazon (technicalbenefits)Automation – “Scriptable infrastructure”: You cancreate repeatable build and deployment systems byleveraging programmable (API-driven) infrastructure.Auto-scaling: You can scale your applications up anddown to match your unexpected demand without anyhuman intervention.Proactive Scaling: Scale your application up and downto meet your anticipated demand; Elasticity
  11. 11. why Amazon (technicalbenefits)More Efficient Development lifecycle: Productionsystems may be easily cloned for use as developmentand test environments.Improved Testability: Never run out of hardware fortesting. Inject and automate testing at every stageduring the development process.Disaster Recovery and Business Continuity: The cloudprovides a lower cost option for maintaining a fleet ofDR servers and data storage.
  12. 12. understanding elasticity
  13. 13. key Amazon terms – #1AWS – Amazon Web ServicesAmazon Web Services (AWS) is a collection of remote computing services (also called webservices) that together make up a cloud computing platform.EC2     - Elastic Compute CloudEC2 allows users to rent virtual computers on which to run their own computer applications.EC2 allows scalable deployment of applications by providing a Web service through which auser can boot an Amazon Machine Image to create a virtual machine.A user can create, launch, and terminate server instances as needed, paying by the hour foractive servers, hence the term "elastic".S3       - Simple Storage ServiceAmazon S3 (Simple Storage Service) is an online storage web service offered by AWS.AMI     - Amazon Machine ImagesAn Amazon Machine Image (AMI) is a special type of virtual appliance which is used toinstantiate (create) a virtual machine within the Amazon Elastic Compute Cloud ("EC2").
  14. 14. key Amazon terms - #2EBS     - Elastic Block StorageAmazon Elastic Block Storage (EBS) provides raw block devices that can be attachedto Amazon EC2 instances.Can be used like any raw block device. In a typical use case, this would includeformatting the device with a filesystem and mounting said filesystem.VPC     - Virtual Private CloudAmazon Virtual Private Cloud (VPC) is a commercial cloud computing service thatprovides a virtual private cloud.Unlike traditional EC2 instances which are allocated internal and external IP numbersby Amazon, the customer can assign IP numbers of their choosing from one or moresubnets.VPC provides much more granular control over security.ELB     - Elastic Load BalancingAZ     - Amazon Availability Zones (Data Centers)
  15. 15. key Amazon terms - #3RDS     - Amazon Relational Database ServiceAmazon RDS is a distributed relational database service by is a web service running "in the cloud" and provides a relational database for usein applications.SupportingMySQL databasesOracle databasesMicrosoft SQL ServerECU     - Elastic Computational UnitOne EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz2007 Opteron or 2007 Xeon processor.SQS     - Simple Queue Service
  16. 16. list of services goes on…
  17. 17. humor“We will launch site on EC2 with EBS behind ELB withdomain registered on Route 53Your images will come from CloudFront,backup will go to S3and your DB on RDS with Multi-AZ availability”
  18. 18. first step first – createaccountGo to just use account forstartAfter login go to IAM(Identity AccessManagement) to add multi-factor authentication; notto your root account, butcreate new account,assign privileges to it andadd MFA. After that useonly new account to loginto your AWS (with givenalias)
  19. 19. easy one – useCloudFormationFastest way to get Drupal on AWSis using predefined templates insideCloudFormation service.In this moment you can find 4(Drupal specific) templatesDrupal_Simple.templateDrupal_Single_Instance.templateDrupal_Single_Instance_With_RDS.templateDrupal_Multi_AZ.templateYou can use any other template asstarting point and customize it toyour needs.
  20. 20. steps afterCreate KeyPairAdd your home/corporate IP to be only allowed to access server over port 22(SSH).Create AMI from existing machineDrop original machineCreate new EC2 instance using just created AMI and your key-pairAdd Elastic IP and associate to your instanceConnect to instanceAdd DNS CNAME record using given Amazon DNS
  21. 21. demoMandatoryclouds image :-)
  22. 22. Amazon VPC - ultimategoalWe can install complete infrastructure required for Drupal using public set of serversELB (load balancer)AMI (servers images)RDS (Amazon relational database service)Elastic Cache...BUTAmazon VPC is a way to setup an isolated partition of AWS and control the networktopology.ServicesDynamodb, ElastiCache, SQS, SES, and CloudSearch are not yet available in VPC(things change on daily basis)RDS instances launched in VPC cannot be accessed over the internet (through theend point). You will need bastion server to access it
  23. 23. EC2 / VPC instances
  24. 24. EC2 - NAT Instance
  25. 25. VPC subnetsIP Ranges - When setting up a VPC you are essentially fixing the network of the VPC.Public and Private Subnets - The VPC network can be divided further in to smaller networksegments called as Subnets. Any VPC will have at least one SubnetYou can setup a Public Subnet which will have internet connectivity. Instances launched withina Public Subnet will have both outbound and inbound (through EIP) internet connectivitythrough the Internet Gateway attached to the Public SubnetPrivate Subnets are completely locked down. They do not have internet connectivity by defaultCreate number of Public and Private Subnets depending upon your architecture.
  26. 26. VPC security groups
  27. 27. AMI images
  28. 28. EBS volumes
  29. 29. autoscalingholy grailKey toElasticity is inautoscaling
  30. 30. how to autoscaleInstall AWS Command Line Tools from AmazonDownloadsDownload from: AWS Auto scaling needs Amazon CloudWatchmonitoring service to function. Amazon CloudWatch isbilled on usage basis.
  31. 31. step 1Configuring AWS Auto Scaling with AWS ELBelb-create-lb my-load-balancer --headers--listener "lb-port=80,instance-port=8080, protocol=HTTP"--availability-zones us-west-2clb-port -- load balancer portinstance-port -- app server port to which requestneeds to be forwardedmy-load-balancer -- name for my load balancer
  32. 32. step 2Create a launch configurationas-create-launch-config my-lconfig --image-id ami-e38823c8a--instance-type m1.small --key my-key-pair--group my-security-groupmy-lconfig -- name for launch configurationami-e38823c8a -- name for Amazon Machine Image (AMI) to belaunched during scalingm1.small -- Amazon EC2 instance sizemy-key-pair -- Key pair / security group settings for the AmazonEC2 instancesmy-security-group -- security group for instance
  33. 33. step 3Create an AWS Auto Scale Groupas-create-auto-scaling-group my-as-group --availability-zones us-west-2c--launch-configuration my-lconfig --max-size 11 --min-size 3 --cooldown 180--desired-capacity 2 --load-balancers my-load-balancermy-load-balancer -- LB name in which the new Amazon EC2 instances launched willbe attachedmy-as-group -- Name Auto Scale groupus-west-2c -- availability zone in which the auto scaled amazon EC2 instances willbe launched11/3 -- Maximum/Minimum number of Amazon EC2 instances maintained by AutoScaleDesired capacity is an important component of the as-create-auto-scaling-groupcommand. Although it is an optional parameter, desired capacity tells Auto Scalingthe number of instances you want to run initially.To adjust the number of instances you want running in your Auto Scaling group, youchange the value of --desired-capacity. If you dont specify --desired-capacity, itsvalue is the same as minimum group size
  34. 34. step 4this step is not available in Auto ScalingAPIConfigure the Auto scaling Triggers / Alarmsas-create-or-update-trigger my-as-trigger--auto-scaling-group my-as-group --namespace "AWS/EC2"--measure CPUUtilization --statistic Average--dimensions "AutoScalingGroupName=my-as-group"--period 60 --lower-threshold 20 --upper-treshold 80--load-breach-increment"=-2" --upper-breach-increment 4--breach-duration 180Measure the average CPU of the Auto Scale GroupScale out by 4 Amazon EC2 instances. Scale down by 2Amazon EC instancesLower CPU Limit is 20% and Upper CPU Limit is 80%
  35. 35. shutdown auto scalinggroupShutdown auto-scaling group - require 3 commandsas-update-auto-scaling-group bbd4me-as-group --min-size 0--max-size 0 --region us-west-2as-describe-auto-scaling-groups bbd4me-as-group --headers--region us-west-2as-delete-auto-scaling-group bbd4me-as-group--force-delete --region us-west-2
  36. 36. Thank you foryour attention.Questions?