Scaling Drupal - The Worth Point Web Site
Upcoming SlideShare
Loading in...5

Scaling Drupal - The Worth Point Web Site






Total Views
Views on SlideShare
Embed Views



2 Embeds 122 115 7



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Scaling Drupal - The Worth Point Web Site Scaling Drupal - The Worth Point Web Site Presentation Transcript

  • D R A F Scaling the WorthPoint web site T To 3,000,000 page views a day
  • Starting Point • WorthPoint is built on the Drupal Content Management System D R • Drupal is A – Open source F – LAMP (Linux, Apache, MySQL, PHP) T – Used by The Onion, MTV-UK, and LifetimeTV, among many others • Professional support available from Acquia starting in the latter half of 2008
  • Background • Drupal is an open source product that is being positioned to compete with enterprise class CMS D products R • The Drupal community is working on multiple A performance and scaling tasks F • WorthPoint is planning for 5M unique web pages and T 3M page views (30/70 mix of dynamic/static) a day by the end of 2008 – This positions WorthPoint as one of the larger Drupal web sites
  • Solution Clusters • Drupal core / module rewrites & updates D • WorthPoint specific module development R • Database server scaling A • Application server scaling F • Image server scaling T • Search server scaling • NOTE: These solutions are for a single data center based system
  • Drupal Core & Modules • WorthPoint uses 35 core modules and 225 community contributed modules D R • WorthPoint currently uses Drupal v5.1 A – V6.0 has been released but the community has not F updated many of the modules used by WorthPoint T – Acquia is working on “Carbon”, a fully tested and certified version of the 35 core modules • Many Drupal v5.1 modules need tweaks to work at the current level of WorthPoint content and traffic
  • WorthPoint Module Development • A vast majority of WorthPoint content and page views are in the Worthopedia, Auctions, Classified, and D Taxonomy areas R • Ground up modules designed and developed by A WorthPoint for these four areas would significantly F reduce the load on the WorthPoint servers T • Initial design work is in progress – Database? – Language – C, Java, PHP?
  • Database Server Scaling • MySQL reliably supports Master-Slave replication – Master is the INSERT, UPDATE, DELETE database; Slaves D are SELECT-only R – Current WorthPoint code allows one database slave to A support 50,000 page views a day1; end of 2008 goal is F 100,000 page views a day per DB slave2 T – This means WorthPoint will have roughly 30 DB slaves at 3M page views a day with the current Drupal code • Beginning to partition / shard the database 1. At the current mix of 20% dynamic and 80% static page views 2. At an anticipated mix of 30% dynamic and 70% static page views
  • Application Server Scaling • Load balance multiple application servers D • Move from Zend to Quercus R • Simple web objects cached with Squid A • Complex web objects cached with memcached F • Misc T – User sessions moved to memory – Database connection pooling – Use content delivery network for CSS & Javascript
  • Image Server Scaling • Images on a SAN D • Two load balanced image servers R • Use a content delivery network A – Limelight ( F T
  • Search Server Scaling • SOLR D – Replication R A F T
  • Wildcards that may help with performance • Cloud based solutions D • Falcon Storage Engine for MySQL – may be ready R for prime time before the end of the year A • A major computer industry player makes a F commitment to Drupal and brings significant T resources to bear
  • Summary • While there is some additional analysis and work to D be done, WorthPoint’s current software will scale to R millions of page views a day with the addition of A hardware F • The current challenge is to cost effectively scale T • The next challenge is to replicate and load balance the WorthPoint systems across multiple geographically distributed data centers e.g. U.S. East Coast, U.S. West Coast, Europe, the Far East, etc
  • Contact Andy Forbes, CTO D R A Marc Benton, Director of Product Development F T Arman Anwar, Director of Systems Development