• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Drupal camp.sg 2012 session - Drupal in the cloud
 

Drupal camp.sg 2012 session - Drupal in the cloud

on

  • 1,533 views

 

Statistics

Views

Total Views
1,533
Views on SlideShare
1,382
Embed Views
151

Actions

Likes
1
Downloads
10
Comments
0

1 Embed 151

http://snasolutions.com.sg 151

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Drupal camp.sg 2012 session - Drupal in the cloud Drupal camp.sg 2012 session - Drupal in the cloud Presentation Transcript

    • Drupal in the Cloud by Azhan Founder/Director S & A Solutions
    • Drupal in the Cloud> Drupal and S & A Solutions• S & A Solutions – who? • doing it with Drupal since 2007 • Over 70 projects in 5 years • More than 20 clients • 99% Drupal projects • We love Drupal (obviously!)
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.com• Content publishing of stories from Asia• Launched in 2010 using Amazon EC2+RDS• Drupal 6, on shared hosting during development• Despite performance tuning efforts, still slow pages"Ok S & A, so what do we do now?"
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.comS & A commissioned to do a site performance evaluationWe need to • Find out what is causing the slow pages • Find out what server size is suitable • Decide between single tier or multi-tier • Provide a solution for short term/future scalability • Prove with benchmarking test results
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.comDrupal and other optimizations done:• Turned on Boost module, CSS/Javascript aggregation• Optimized images, custom module SQL queries• Removed references to missing files/imagesResults:• Reduced page loading times from 8 secs to 5 secs • average ~ 30-40% improvement • Apparent network latency stays at 3-4 secs> Hints on bottleneck in the network, web server, database
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.comNetwork test: • wget download test >> 132kb/sec (good) • ping latency hovering 200-300ms (normal for SG to USA)Network no longer the suspect• Apache ab –k -n100 -c1 >> 2-5 secs per page load (SLOW!)• 70% content errors when ramped up to n=100, c=10Final suspects:Web server, MySQL needs more CPU and/or memory
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.comOther opportunities in improving performance • Serve static files using a CDN (> 1GB of static files) • Load balanced web servers • Memcached / Varnish • Reverse proxy eg. Nginx • Tune & recompile Apache/MySQL for performance • Allocate more resources to Apache and MySQL (easy)
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.comOther opportunities in improving performance • Serve static files using a CDN (> 1GB of static files) • Load balanced web servers • Memcached / Varnish • Reverse proxy eg. Nginx • Tune & recompile Apache/MySQL for performance • Allocate more resources to Apache and MySQL (easy)
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.comTheAsiaMag asked for Amazon web services• US West Virginia availability zone • mostly worldwide audience• 4 types of server configurations were tried• involves a Load balancer, EC2 with EBS, RDS
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.com1st run: Load balancer, 2x EC2 micro instances, small RDS2nd run: Load balancer, 1x LARGE EC2 instance, small RDS3rd run: Load balancer, 1x small EC2 instance, small RDS4th run: Load balancer, 1x medium EC2 instance, small RDSBenchmarking tools used:• Apache ab• Drupal Devel module, MySQL slow logs• Firebug
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.comRESULTS of 1st run: Apache ab -n10000 -c10Load balancer, 2x EC2 micro instances, small RDSEC2 CPU 100%EC2 RAM 100% (no swap)RDS CPU 100%Apache server dies at a random time within a dayand could not revive itself
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.comRESULTS of 2nd run: Apache ab -n10000 -c10Single LARGE EC2 instance, small RDS10,000 requests completed in under 20minsEC2 CPU 5-10%EC2 RAM 30-50%RDS CPU <5%Impressive, but overkill. And expensive!
    • Drupal in the Cloud> Case Study 1 – http://theasiamag.comRESULTS of 3rd run: Apache ab -n10000 -c10Single small EC2 instance, small RDS10,000 requests completed in 1hrEC2 CPU 15-20%EC2 RAM 85%RDS CPU 50%Performance is just OK, but cost is within budget!No need for 4th run.
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.com• Property listing portal, launched in 2010• Drupal 5, dedicated server hosting• In 2011 - slow pages, occasional server deaths“S & A, what do we do?”
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comThe problem• Just 1000+ properties• 10k page views per month• Very complex search+filters using native Drupal search• Real time statistics killing the site• Watchdog?!!!First aid• Identified slow queries, MySQL always using too much RAM• Had to “kill” watchdog• Plenty of opportunity for code optimization, but have to wait• Hardware upgrade to rescue the site first
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.com• Current hosting at a leading US-based reputable company• Amazon web services suggested as an alternative
    • Drupal in the Cloud> Scalability challenges• Code, database, OS optimizations• Single, monster server vs multi-tier architecture• How to handle spikes in traffic/load• Security, reliability• Cost for all the fancy setup• Time to do all that?!without breaking the bank!> I know we could do all that with Amazon web services
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comProposed architecture on Amazon web services• Load balancer• 2x small EC2 for web server• 1x small EC2 for hosting static files • Shared by all web servers via NFS• 1x LARGE RDS for MySQL database
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comFinal architecture on Amazon web services• Load balancer (later, perhaps)• Single LARGE EC2 for web server + MySQLKeep it simple!Building the business case for the move to AWS:• Benchmarking the AWS setup vs the current site• AWS setup has identical CPU, double RAM
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comAmazon setup vs dedicated server Current Amazon web servicesCPU Dual core AMD 64bit CPU 4 ECUs (Intel Xeon dual core equivalent)OS 64bit RHEL 4 64bit Ubuntu 11.10RAM 4GB 7.5GBLocation USA USA (West Virginia)Average Cost USD 300 per month USD 200 per monthUpgrading “current” to 8GB, RHEL will cost USD700!
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 12 11 10 No of requests per second 8 (more is better) 6 Current AWS 4 4 2 0 Apache ab requests per sec Benchmarking Tests
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 14 12 12 10 Time in secs 8 Current 6 AWS 4.4 4 2 0 Apache ab timing per request Benchmarking Tests
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 2500 2000 1941 1500 Time in ms Current 1000 AWS 500 102 0 Devel PHP timings on Homepage Benchmarking Tests
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 120 100 100 80 Time in secs 60 Current AWS 40 20 18 0 Devel PHP timings on Query page Benchmarking Tests
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 6 5.46 5 4 Time in secs 3 Current AWS 2 1.44 1 0 Firebug network latency Benchmarking Tests
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 70 60 60 50 Time in secs 40 Current 30 AWS 20 20 10 0 Firebug full page loading time Benchmarking Tests
    • Drupal in the Cloud> Case Study 2 – http://holidaycube.comClient is convinced!(Yeay!!)
    • Drupal in the Cloud> Why Amazon web services?No to AWS• Could easily have been other hosting providers • Other cloud vendors can address scalability issues• Getting over the Amazonian barriers• AWS may be viewed as “difficult” by some
    • Drupal in the Cloud > Why Amazon web services?Yes, AWS!• A platform for benchmarking, server sizing• Easy and quick to create/kill servers, run tests• Actually a safe, cheap and controlled means of hosting your site or application• Easily spawn additional servers to cater to “high” periods eg. A product launch lasting 4 weeks terminate the additional servers after site traffic has died downYou can get really creative designing your architecture
    • Drupal in the Cloud> Drupal web load balancingAmazon Load BalancerNormally a complex setup with traditional hosting, the AWS Console makes it so easy to set this up in minutes
    • Drupal in the Cloud> Drupal and memcachedhttp://drupal.org/project/memcacheAmazon ElastiCacheNormally a complex setup with traditional hosting the AWS Console makes it so easy to set this up in minutes if paired with the Drupal memcache module
    • Drupal in the Cloud> Drupal and CDNhttp://drupal.org/project/cdnhttp://drupal.org/project/cloudfrontAmazon Cloudfront or S3
    • Drupal in the Cloud> Drupal and high availabilityAmazon multi-Availability Zones for EC2 and RDS
    • Drupal in the Cloud> Drupal and MySQLRunning MySQL can be done using one of the following options:MySQL on Amazon EC2MySQL on Amazon RDSMulti-AZ RDS
    • Drupal in the Cloud > Drupal & Amazon web servicesAmazon CloudFormation“Templates” to quickly deploy a Drupal stack into the cloud
    • Drupal in the Cloud> Drupal & Amazon web servicesIt works!
    • Drupal in the Cloud> Questions?Email: azhan@snasolutions.com.sgWebsite: http://www.snasolutions.com.sgThank you!