Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
LESSONS LEARNED AT REDDIT Steve Huffman FOWA 2010
Reddit.com
A brief history of reddit <ul><li>Founded in June 2005 </li></ul><ul><li>Acquired by Condé Nast October 2007 </li></ul><ul...
Lesson 1: Crash! <ul><li>… and restart. </li></ul><ul><li>Daemontools (supervise) </li></ul><ul><li>Single greatest improv...
Lesson 2: Separation of services <ul><li>Often, one->two machines more than doubles performance. </li></ul><ul><li>Group s...
Lesson 3: Open Schema ID UPS DOWNS TITLE URL 12345 120 34 Buffins Create Zombie Dog! www.someaussiesite.co.au/dog.html 123...
Lesson 3: Open Schema <ul><li>In the early days: </li></ul><ul><li>Too much time spent thinking about the database. </li><...
Lesson 3: Open Schema THING_ID KEY VALUE 12345 Title Boffins Create Zombie Dog! 12345 URL www.someaussiesite.com.au/zombie...
Lesson 3: Open Schema <ul><li>With an open schema: </li></ul><ul><li>Faster development </li></ul><ul><li>Easier deploymen...
Lesson 4: Keep it stateless <ul><li>Goal: any app server can handle any request </li></ul><ul><li>App server failure/resta...
Lesson 5:  Memcache  everything <ul><li>Database data </li></ul><ul><li>Session data </li></ul><ul><li>Rendered pages </li...
Lesson 6: Store redundant data <ul><li>Recipe for slow: keep data normalized until you need it. </li></ul><ul><li>If data ...
Lesson 7: Work offline <ul><li>Do the minimum amount of work to end the request. </li></ul><ul><li>Everything else can be ...
Lesson 7: Work offline <ul><li>Pre-computing listings </li></ul><ul><li>Fetching thumbnails </li></ul><ul><li>Detecting ch...
Lesson 7: Work offline Master Databases App Servers Worker Databases Cache Precomputer Thumbnailer Spam Request Queue
THANKS! QUESTIONS?
Upcoming SlideShare
Loading in …5
×

Steve Huffman - Lessons learned while at reddit.com

33,128 views

Published on

Neil will teach you five advanced website traffic statistics that you NEED to be measuring, but probably aren't. It isn't good enough anymore to just measure click-through and conversion rates to your signup page. You need MUCH more detail and Neil will explain how to get it and make decisions accordingly. You'll be amazed at the increase in valuable sign-ups and revenue increases you can achieve.

Published in: Technology
  • Reddit is one of amazing social media.

    Mark Chang, www.free-ringtones.co.in/ www.free-ringtones-for-sprint.com/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Steve Huffman - Lessons learned while at reddit.com

  1. 1. LESSONS LEARNED AT REDDIT Steve Huffman FOWA 2010
  2. 2. Reddit.com
  3. 3. A brief history of reddit <ul><li>Founded in June 2005 </li></ul><ul><li>Acquired by Condé Nast October 2007 </li></ul><ul><li>7.5 Million user / month </li></ul><ul><li>270 Million page views / month </li></ul><ul><li>Many mistakes along the way </li></ul>
  4. 4. Lesson 1: Crash! <ul><li>… and restart. </li></ul><ul><li>Daemontools (supervise) </li></ul><ul><li>Single greatest improvement to uptime we ever made. </li></ul><ul><li>When in doubt, let it die. </li></ul><ul><li>Don’t forget to read the logs! </li></ul>
  5. 5. Lesson 2: Separation of services <ul><li>Often, one->two machines more than doubles performance. </li></ul><ul><li>Group similar process together. </li></ul><ul><li>Group similar types of data together. </li></ul><ul><li>Better caching. </li></ul><ul><li>Less contention for CPU. </li></ul><ul><li>Avoid threads. Processes are easier to separate later. </li></ul>
  6. 6. Lesson 3: Open Schema ID UPS DOWNS TITLE URL 12345 120 34 Buffins Create Zombie Dog! www.someaussiesite.co.au/dog.html 12346 3 24 Check out my new blog! noobspamer.blogspot.com 12347 509 167 Pee in a sink if you’ve ever voted up. self
  7. 7. Lesson 3: Open Schema <ul><li>In the early days: </li></ul><ul><li>Too much time spent thinking about the database. </li></ul><ul><li>Every feature required a schema update. </li></ul><ul><li>Schema updates became more painful as we grew. </li></ul><ul><li>Maintaining replication was difficult. </li></ul><ul><li>Deployment was complex. </li></ul>
  8. 8. Lesson 3: Open Schema THING_ID KEY VALUE 12345 Title Boffins Create Zombie Dog! 12345 URL www.someaussiesite.com.au/zombiedog.html 12346 Title Pee in a sink if you’ve ever voted up. 12346 URL self ID UPS DOWNS TYPE 12345 120 34 Link 12346 3 24 Link Thing Data
  9. 9. Lesson 3: Open Schema <ul><li>With an open schema: </li></ul><ul><li>Faster development </li></ul><ul><li>Easier deployment </li></ul><ul><li>Maintainable database replication </li></ul><ul><li>No joins = easy to distribute </li></ul><ul><li>Must be careful to maintain consistency </li></ul>
  10. 10. Lesson 4: Keep it stateless <ul><li>Goal: any app server can handle any request </li></ul><ul><li>App server failure/restart is no big deal </li></ul><ul><li>Scaling is straightforward </li></ul><ul><li>Caching must be independent from a specific app server. </li></ul>
  11. 11. Lesson 5: Memcache everything <ul><li>Database data </li></ul><ul><li>Session data </li></ul><ul><li>Rendered pages </li></ul><ul><li>Memoizing internal functions </li></ul><ul><li>Rate-limiting (user actions, crawlers) </li></ul><ul><li>Storing pre-computing listings/pages </li></ul><ul><li>Global locking </li></ul><ul><li>Memcachedb for persistence </li></ul>
  12. 12. Lesson 6: Store redundant data <ul><li>Recipe for slow: keep data normalized until you need it. </li></ul><ul><li>If data has multiple presentations, store it in multiple times in multiple formats. </li></ul><ul><li>Disk and memory is less costly than making your users wait. </li></ul>
  13. 13. Lesson 7: Work offline <ul><li>Do the minimum amount of work to end the request. </li></ul><ul><li>Everything else can be done offline. </li></ul><ul><li>An architecture of queues is simple and easy to scale. </li></ul><ul><li>AMQP/RabbitMQ. </li></ul>
  14. 14. Lesson 7: Work offline <ul><li>Pre-computing listings </li></ul><ul><li>Fetching thumbnails </li></ul><ul><li>Detecting cheating </li></ul><ul><li>Removing spam </li></ul><ul><li>Computing awards </li></ul><ul><li>Updating the “search” index </li></ul>
  15. 15. Lesson 7: Work offline Master Databases App Servers Worker Databases Cache Precomputer Thumbnailer Spam Request Queue
  16. 16. THANKS! QUESTIONS?

×