My presentation from the "Making Startups Work" panel on October 4, 2011 in NYC. I presented about how we scaled Drupal to work for our traffic and our requirements.

1. Scaling Drupal
“UnDrupaling Drupal”
Michael T. Smith
Thrillist
October 4, 2011

2. Drupal 5 to 6
• Two month rebuild;
October to December 2010
• Converted >50 modules to D6;
20+ new modules
• 8gb+ database:
2.3m nodes, 3m+ user records

3. Our Mindset:
“UnDrupaling Drupal”
• Ethan Kaplan, former VP Emerging Tech
@ Warner Bros. Entertainment Group
• 120 artist sites
• Out of the box: Not able to scale
• Huge issues with loading node objects
from DB
• Hooks are nice, but can allow too much to
load

4. MongoDB
• MySQL & node_load
•Every module potentially called
•Huge overhead
• Mongo:
•Fast lookup, fast results
•Not for user records

5. MongoDB

6. MongoDB

7. Memcached
• Cache All The Things!
• Memcached with bins set up for page,
content, mobile data and blocks
• Default page and block cache, kinda
• Edition and Logged In states
• Special keys for edition content
• Case:
Viewing NY article from ATL “edition”

8. SCSS &
Clean Theming
• Holy disgusting code, Batman
• Old jQuery version—updated
• Custom theme functions
• Preprocessors to clean
• Compass:
• Auto-generated sprites
• Auto-minimized code
• UberTags

9. Users & Stats
• Huge data, huge concern, huge codebase
• Extremely custom marketing and promo
system for signups
• No default signups
• Customized modules for mixing
forms/surveys
• Reporting is a challenge—separate
systems for reporting and data dumps, all
based “around” Drupal

10. API
• Two servers:
• One for people
• One for machines
• Services module for JS requests
• Future? DB as datastore, Drupal only for
serving content to people

11. Idempotence
• Make as much as possible repeatable
without harm
• Not perfectly idempotent, but close
enough
• Mainly user systems
• Thrillist/JackThreads/Rewards
subscriptions updates are queued

12. People
• Team
• RackSpace
• MongoHQ

13. Thanks!
(questions?)