My presentation from the "Making Startups Work" panel on October 4, 2011 in NYC. I presented about how we scaled Drupal to work for our traffic and our requirements.
2. Drupal 5 to 6
• Two month rebuild;
October to December 2010
• Converted >50 modules to D6;
20+ new modules
• 8gb+ database:
2.3m nodes, 3m+ user records
3. Our Mindset:
“UnDrupaling Drupal”
• Ethan Kaplan, former VP Emerging Tech
@ Warner Bros. Entertainment Group
• 120 artist sites
• Out of the box: Not able to scale
• Huge issues with loading node objects
from DB
• Hooks are nice, but can allow too much to
load
4. MongoDB
• MySQL & node_load
•Every module potentially called
•Huge overhead
• Mongo:
•Fast lookup, fast results
•Not for user records
7. Memcached
• Cache All The Things!
• Memcached with bins set up for page,
content, mobile data and blocks
• Default page and block cache, kinda
• Edition and Logged In states
• Special keys for edition content
• Case:
Viewing NY article from ATL “edition”
9. Users & Stats
• Huge data, huge concern, huge codebase
• Extremely custom marketing and promo
system for signups
• No default signups
• Customized modules for mixing
forms/surveys
• Reporting is a challenge—separate
systems for reporting and data dumps, all
based “around” Drupal
10. API
• Two servers:
• One for people
• One for machines
• Services module for JS requests
• Future? DB as datastore, Drupal only for
serving content to people
11. Idempotence
• Make as much as possible repeatable
without harm
• Not perfectly idempotent, but close
enough
• Mainly user systems
• Thrillist/JackThreads/Rewards
subscriptions updates are queued