Drupal in the Cloud               by Azhan    Founder/Director S & A Solutions
Drupal in the Cloud> Drupal and S & A Solutions• S & A Solutions – who?      • doing it with Drupal since 2007      • Over...
Drupal in the Cloud> Case Study 1 – http://theasiamag.com• Content publishing of stories from Asia• Launched in 2010 using...
Drupal in the Cloud> Case Study 1 – http://theasiamag.comS & A commissioned to do a site performance evaluationWe need to ...
Drupal in the Cloud> Case Study 1 – http://theasiamag.comDrupal and other optimizations done:• Turned on Boost module, CSS...
Drupal in the Cloud> Case Study 1 – http://theasiamag.comNetwork test:    • wget download test >> 132kb/sec (good)    • pi...
Drupal in the Cloud> Case Study 1 – http://theasiamag.comOther opportunities in improving performance    • Serve static fi...
Drupal in the Cloud> Case Study 1 – http://theasiamag.comOther opportunities in improving performance    • Serve static fi...
Drupal in the Cloud> Case Study 1 – http://theasiamag.comTheAsiaMag asked for Amazon web services• US West Virginia availa...
Drupal in the Cloud> Case Study 1 – http://theasiamag.com1st run: Load balancer, 2x EC2 micro instances, small RDS2nd run:...
Drupal in the Cloud> Case Study 1 – http://theasiamag.comRESULTS of 1st run: Apache ab -n10000 -c10Load balancer, 2x EC2 m...
Drupal in the Cloud> Case Study 1 – http://theasiamag.comRESULTS of 2nd run: Apache ab -n10000 -c10Single LARGE EC2 instan...
Drupal in the Cloud> Case Study 1 – http://theasiamag.comRESULTS of 3rd run: Apache ab -n10000 -c10Single small EC2 instan...
Drupal in the Cloud> Case Study 2 – http://holidaycube.com• Property listing portal, launched in 2010• Drupal 5, dedicated...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comThe problem• Just 1000+ properties• 10k page views per month• Ve...
Drupal in the Cloud> Case Study 2 – http://holidaycube.com• Current hosting at a leading US-based reputable company• Amazo...
Drupal in the Cloud> Scalability challenges• Code, database, OS optimizations• Single, monster server vs multi-tier archit...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comProposed architecture on Amazon web services• Load balancer• 2x ...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comFinal architecture on Amazon web services• Load balancer (later,...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comAmazon setup vs dedicated server               Current          ...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison                            ...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison                14          ...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison              2500          ...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison                120         ...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison                6           ...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison                70          ...
Drupal in the Cloud> Case Study 2 – http://holidaycube.comClient is convinced!(Yeay!!)
Drupal in the Cloud> Why Amazon web services?No to AWS• Could easily have been other hosting providers     • Other cloud v...
Drupal in the Cloud > Why Amazon web services?Yes, AWS!• A platform for benchmarking, server sizing• Easy and quick to cre...
Drupal in the Cloud> Drupal web load balancingAmazon Load BalancerNormally a complex setup with traditional hosting, the A...
Drupal in the Cloud> Drupal and memcachedhttp://drupal.org/project/memcacheAmazon ElastiCacheNormally a complex setup with...
Drupal in the Cloud> Drupal and CDNhttp://drupal.org/project/cdnhttp://drupal.org/project/cloudfrontAmazon Cloudfront or S3
Drupal in the Cloud> Drupal and high availabilityAmazon multi-Availability Zones for EC2 and RDS
Drupal in the Cloud> Drupal and MySQLRunning MySQL can be done using one of the following options:MySQL on Amazon EC2MySQL...
Drupal in the Cloud > Drupal & Amazon web servicesAmazon CloudFormation“Templates” to quickly deploy a Drupal stack into t...
Drupal in the Cloud> Drupal & Amazon web servicesIt works!
Drupal in the Cloud> Questions?Email: azhan@snasolutions.com.sgWebsite: http://www.snasolutions.com.sgThank you!
Upcoming SlideShare
Loading in...5
×

Drupal camp.sg 2012 session - Drupal in the cloud

1,524

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,524
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Drupal camp.sg 2012 session - Drupal in the cloud

  1. 1. Drupal in the Cloud by Azhan Founder/Director S & A Solutions
  2. 2. Drupal in the Cloud> Drupal and S & A Solutions• S & A Solutions – who? • doing it with Drupal since 2007 • Over 70 projects in 5 years • More than 20 clients • 99% Drupal projects • We love Drupal (obviously!)
  3. 3. Drupal in the Cloud> Case Study 1 – http://theasiamag.com• Content publishing of stories from Asia• Launched in 2010 using Amazon EC2+RDS• Drupal 6, on shared hosting during development• Despite performance tuning efforts, still slow pages"Ok S & A, so what do we do now?"
  4. 4. Drupal in the Cloud> Case Study 1 – http://theasiamag.comS & A commissioned to do a site performance evaluationWe need to • Find out what is causing the slow pages • Find out what server size is suitable • Decide between single tier or multi-tier • Provide a solution for short term/future scalability • Prove with benchmarking test results
  5. 5. Drupal in the Cloud> Case Study 1 – http://theasiamag.comDrupal and other optimizations done:• Turned on Boost module, CSS/Javascript aggregation• Optimized images, custom module SQL queries• Removed references to missing files/imagesResults:• Reduced page loading times from 8 secs to 5 secs • average ~ 30-40% improvement • Apparent network latency stays at 3-4 secs> Hints on bottleneck in the network, web server, database
  6. 6. Drupal in the Cloud> Case Study 1 – http://theasiamag.comNetwork test: • wget download test >> 132kb/sec (good) • ping latency hovering 200-300ms (normal for SG to USA)Network no longer the suspect• Apache ab –k -n100 -c1 >> 2-5 secs per page load (SLOW!)• 70% content errors when ramped up to n=100, c=10Final suspects:Web server, MySQL needs more CPU and/or memory
  7. 7. Drupal in the Cloud> Case Study 1 – http://theasiamag.comOther opportunities in improving performance • Serve static files using a CDN (> 1GB of static files) • Load balanced web servers • Memcached / Varnish • Reverse proxy eg. Nginx • Tune & recompile Apache/MySQL for performance • Allocate more resources to Apache and MySQL (easy)
  8. 8. Drupal in the Cloud> Case Study 1 – http://theasiamag.comOther opportunities in improving performance • Serve static files using a CDN (> 1GB of static files) • Load balanced web servers • Memcached / Varnish • Reverse proxy eg. Nginx • Tune & recompile Apache/MySQL for performance • Allocate more resources to Apache and MySQL (easy)
  9. 9. Drupal in the Cloud> Case Study 1 – http://theasiamag.comTheAsiaMag asked for Amazon web services• US West Virginia availability zone • mostly worldwide audience• 4 types of server configurations were tried• involves a Load balancer, EC2 with EBS, RDS
  10. 10. Drupal in the Cloud> Case Study 1 – http://theasiamag.com1st run: Load balancer, 2x EC2 micro instances, small RDS2nd run: Load balancer, 1x LARGE EC2 instance, small RDS3rd run: Load balancer, 1x small EC2 instance, small RDS4th run: Load balancer, 1x medium EC2 instance, small RDSBenchmarking tools used:• Apache ab• Drupal Devel module, MySQL slow logs• Firebug
  11. 11. Drupal in the Cloud> Case Study 1 – http://theasiamag.comRESULTS of 1st run: Apache ab -n10000 -c10Load balancer, 2x EC2 micro instances, small RDSEC2 CPU 100%EC2 RAM 100% (no swap)RDS CPU 100%Apache server dies at a random time within a dayand could not revive itself
  12. 12. Drupal in the Cloud> Case Study 1 – http://theasiamag.comRESULTS of 2nd run: Apache ab -n10000 -c10Single LARGE EC2 instance, small RDS10,000 requests completed in under 20minsEC2 CPU 5-10%EC2 RAM 30-50%RDS CPU <5%Impressive, but overkill. And expensive!
  13. 13. Drupal in the Cloud> Case Study 1 – http://theasiamag.comRESULTS of 3rd run: Apache ab -n10000 -c10Single small EC2 instance, small RDS10,000 requests completed in 1hrEC2 CPU 15-20%EC2 RAM 85%RDS CPU 50%Performance is just OK, but cost is within budget!No need for 4th run.
  14. 14. Drupal in the Cloud> Case Study 2 – http://holidaycube.com• Property listing portal, launched in 2010• Drupal 5, dedicated server hosting• In 2011 - slow pages, occasional server deaths“S & A, what do we do?”
  15. 15. Drupal in the Cloud> Case Study 2 – http://holidaycube.comThe problem• Just 1000+ properties• 10k page views per month• Very complex search+filters using native Drupal search• Real time statistics killing the site• Watchdog?!!!First aid• Identified slow queries, MySQL always using too much RAM• Had to “kill” watchdog• Plenty of opportunity for code optimization, but have to wait• Hardware upgrade to rescue the site first
  16. 16. Drupal in the Cloud> Case Study 2 – http://holidaycube.com• Current hosting at a leading US-based reputable company• Amazon web services suggested as an alternative
  17. 17. Drupal in the Cloud> Scalability challenges• Code, database, OS optimizations• Single, monster server vs multi-tier architecture• How to handle spikes in traffic/load• Security, reliability• Cost for all the fancy setup• Time to do all that?!without breaking the bank!> I know we could do all that with Amazon web services
  18. 18. Drupal in the Cloud> Case Study 2 – http://holidaycube.comProposed architecture on Amazon web services• Load balancer• 2x small EC2 for web server• 1x small EC2 for hosting static files • Shared by all web servers via NFS• 1x LARGE RDS for MySQL database
  19. 19. Drupal in the Cloud> Case Study 2 – http://holidaycube.comFinal architecture on Amazon web services• Load balancer (later, perhaps)• Single LARGE EC2 for web server + MySQLKeep it simple!Building the business case for the move to AWS:• Benchmarking the AWS setup vs the current site• AWS setup has identical CPU, double RAM
  20. 20. Drupal in the Cloud> Case Study 2 – http://holidaycube.comAmazon setup vs dedicated server Current Amazon web servicesCPU Dual core AMD 64bit CPU 4 ECUs (Intel Xeon dual core equivalent)OS 64bit RHEL 4 64bit Ubuntu 11.10RAM 4GB 7.5GBLocation USA USA (West Virginia)Average Cost USD 300 per month USD 200 per monthUpgrading “current” to 8GB, RHEL will cost USD700!
  21. 21. Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 12 11 10 No of requests per second 8 (more is better) 6 Current AWS 4 4 2 0 Apache ab requests per sec Benchmarking Tests
  22. 22. Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 14 12 12 10 Time in secs 8 Current 6 AWS 4.4 4 2 0 Apache ab timing per request Benchmarking Tests
  23. 23. Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 2500 2000 1941 1500 Time in ms Current 1000 AWS 500 102 0 Devel PHP timings on Homepage Benchmarking Tests
  24. 24. Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 120 100 100 80 Time in secs 60 Current AWS 40 20 18 0 Devel PHP timings on Query page Benchmarking Tests
  25. 25. Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 6 5.46 5 4 Time in secs 3 Current AWS 2 1.44 1 0 Firebug network latency Benchmarking Tests
  26. 26. Drupal in the Cloud> Case Study 2 – http://holidaycube.comBenchmarking test results comparison 70 60 60 50 Time in secs 40 Current 30 AWS 20 20 10 0 Firebug full page loading time Benchmarking Tests
  27. 27. Drupal in the Cloud> Case Study 2 – http://holidaycube.comClient is convinced!(Yeay!!)
  28. 28. Drupal in the Cloud> Why Amazon web services?No to AWS• Could easily have been other hosting providers • Other cloud vendors can address scalability issues• Getting over the Amazonian barriers• AWS may be viewed as “difficult” by some
  29. 29. Drupal in the Cloud > Why Amazon web services?Yes, AWS!• A platform for benchmarking, server sizing• Easy and quick to create/kill servers, run tests• Actually a safe, cheap and controlled means of hosting your site or application• Easily spawn additional servers to cater to “high” periods eg. A product launch lasting 4 weeks terminate the additional servers after site traffic has died downYou can get really creative designing your architecture
  30. 30. Drupal in the Cloud> Drupal web load balancingAmazon Load BalancerNormally a complex setup with traditional hosting, the AWS Console makes it so easy to set this up in minutes
  31. 31. Drupal in the Cloud> Drupal and memcachedhttp://drupal.org/project/memcacheAmazon ElastiCacheNormally a complex setup with traditional hosting the AWS Console makes it so easy to set this up in minutes if paired with the Drupal memcache module
  32. 32. Drupal in the Cloud> Drupal and CDNhttp://drupal.org/project/cdnhttp://drupal.org/project/cloudfrontAmazon Cloudfront or S3
  33. 33. Drupal in the Cloud> Drupal and high availabilityAmazon multi-Availability Zones for EC2 and RDS
  34. 34. Drupal in the Cloud> Drupal and MySQLRunning MySQL can be done using one of the following options:MySQL on Amazon EC2MySQL on Amazon RDSMulti-AZ RDS
  35. 35. Drupal in the Cloud > Drupal & Amazon web servicesAmazon CloudFormation“Templates” to quickly deploy a Drupal stack into the cloud
  36. 36. Drupal in the Cloud> Drupal & Amazon web servicesIt works!
  37. 37. Drupal in the Cloud> Questions?Email: azhan@snasolutions.com.sgWebsite: http://www.snasolutions.com.sgThank you!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×