SlideShare a Scribd company logo
1 of 16
Download to read offline
LESSONS LEARNED AT REDDIT Steve Huffman FOWA 2010
Reddit.com
A brief history of reddit ,[object Object],[object Object],[object Object],[object Object],[object Object]
Lesson 1: Crash! ,[object Object],[object Object],[object Object],[object Object],[object Object]
Lesson 2: Separation of services ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Lesson 3: Open Schema ID UPS DOWNS TITLE URL 12345 120 34 Buffins Create Zombie Dog! www.someaussiesite.co.au/dog.html 12346 3 24 Check out my new blog! noobspamer.blogspot.com 12347 509 167 Pee in a sink if you’ve ever voted up. self
Lesson 3: Open Schema ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Lesson 3: Open Schema THING_ID KEY VALUE 12345 Title Boffins Create Zombie Dog! 12345 URL www.someaussiesite.com.au/zombiedog.html 12346 Title Pee in a sink if you’ve ever voted up. 12346 URL self ID UPS DOWNS TYPE 12345 120 34 Link 12346 3 24 Link Thing Data
Lesson 3: Open Schema ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Lesson 4: Keep it stateless ,[object Object],[object Object],[object Object],[object Object]
Lesson 5:  Memcache  everything ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Lesson 6: Store redundant data ,[object Object],[object Object],[object Object]
Lesson 7: Work offline ,[object Object],[object Object],[object Object],[object Object]
Lesson 7: Work offline ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Lesson 7: Work offline Master Databases App Servers Worker Databases Cache Precomputer Thumbnailer Spam Request Queue
THANKS! QUESTIONS?

More Related Content

What's hot

大規模ソーシャルゲーム開発から学んだPHP&MySQL実践テクニック
大規模ソーシャルゲーム開発から学んだPHP&MySQL実践テクニック大規模ソーシャルゲーム開発から学んだPHP&MySQL実践テクニック
大規模ソーシャルゲーム開発から学んだPHP&MySQL実践テクニックinfinite_loop
 
MySQL5.7 GA の Multi-threaded slave
MySQL5.7 GA の Multi-threaded slaveMySQL5.7 GA の Multi-threaded slave
MySQL5.7 GA の Multi-threaded slaveTakanori Sejima
 
大規模ソーシャルゲームを支える技術~PHP+MySQLを使った高負荷対策~
大規模ソーシャルゲームを支える技術~PHP+MySQLを使った高負荷対策~大規模ソーシャルゲームを支える技術~PHP+MySQLを使った高負荷対策~
大規模ソーシャルゲームを支える技術~PHP+MySQLを使った高負荷対策~infinite_loop
 
モジュールの凝集度・結合度・インタフェース
モジュールの凝集度・結合度・インタフェースモジュールの凝集度・結合度・インタフェース
モジュールの凝集度・結合度・インタフェースHajime Yanagawa
 
今からでも遅くないDBマイグレーション - Flyway と SchemaSpy の紹介 -
今からでも遅くないDBマイグレーション - Flyway と SchemaSpy の紹介 -今からでも遅くないDBマイグレーション - Flyway と SchemaSpy の紹介 -
今からでも遅くないDBマイグレーション - Flyway と SchemaSpy の紹介 -onozaty
 
Under the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database ArchitectureUnder the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database ArchitectureScyllaDB
 
新しい実装方法 Braintree SDKについて
新しい実装方法 Braintree SDKについて新しい実装方法 Braintree SDKについて
新しい実装方法 Braintree SDKについてJunichi Okamura
 
片手間MySQLチューニング戦略
片手間MySQLチューニング戦略片手間MySQLチューニング戦略
片手間MySQLチューニング戦略yoku0825
 
InnoDBのすゝめ(仮)
InnoDBのすゝめ(仮)InnoDBのすゝめ(仮)
InnoDBのすゝめ(仮)Takanori Sejima
 
RHEL7/CentOS7 NetworkManager徹底入門
RHEL7/CentOS7 NetworkManager徹底入門RHEL7/CentOS7 NetworkManager徹底入門
RHEL7/CentOS7 NetworkManager徹底入門Etsuji Nakai
 
VLDB2015 会議報告
VLDB2015 会議報告VLDB2015 会議報告
VLDB2015 会議報告Yuto Hayamizu
 
SQLアンチパターン(インデックスショットガン)
SQLアンチパターン(インデックスショットガン)SQLアンチパターン(インデックスショットガン)
SQLアンチパターン(インデックスショットガン)Tomoaki Uchida
 
さいきんの InnoDB Adaptive Flushing (仮)
さいきんの InnoDB Adaptive Flushing (仮)さいきんの InnoDB Adaptive Flushing (仮)
さいきんの InnoDB Adaptive Flushing (仮)Takanori Sejima
 
Linux Performance Analysis and Tools
Linux Performance Analysis and ToolsLinux Performance Analysis and Tools
Linux Performance Analysis and ToolsBrendan Gregg
 
ソーシャルゲーム案件におけるDB分割のPHP実装
ソーシャルゲーム案件におけるDB分割のPHP実装ソーシャルゲーム案件におけるDB分割のPHP実装
ソーシャルゲーム案件におけるDB分割のPHP実装infinite_loop
 
Mobage を支える Ruby の技術 ~ 複数DB編 ~
Mobage を支える Ruby の技術 ~ 複数DB編 ~Mobage を支える Ruby の技術 ~ 複数DB編 ~
Mobage を支える Ruby の技術 ~ 複数DB編 ~Naotoshi Seo
 
MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committe...
MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committe...MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committe...
MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committe...MongoDB
 
とある診断員とAWS
とある診断員とAWSとある診断員とAWS
とある診断員とAWSzaki4649
 

What's hot (20)

大規模ソーシャルゲーム開発から学んだPHP&MySQL実践テクニック
大規模ソーシャルゲーム開発から学んだPHP&MySQL実践テクニック大規模ソーシャルゲーム開発から学んだPHP&MySQL実践テクニック
大規模ソーシャルゲーム開発から学んだPHP&MySQL実践テクニック
 
JS7 JobScheduler プレビュー
JS7 JobScheduler プレビューJS7 JobScheduler プレビュー
JS7 JobScheduler プレビュー
 
MySQL5.7 GA の Multi-threaded slave
MySQL5.7 GA の Multi-threaded slaveMySQL5.7 GA の Multi-threaded slave
MySQL5.7 GA の Multi-threaded slave
 
Big Data (security Issue)
Big Data (security Issue)Big Data (security Issue)
Big Data (security Issue)
 
大規模ソーシャルゲームを支える技術~PHP+MySQLを使った高負荷対策~
大規模ソーシャルゲームを支える技術~PHP+MySQLを使った高負荷対策~大規模ソーシャルゲームを支える技術~PHP+MySQLを使った高負荷対策~
大規模ソーシャルゲームを支える技術~PHP+MySQLを使った高負荷対策~
 
モジュールの凝集度・結合度・インタフェース
モジュールの凝集度・結合度・インタフェースモジュールの凝集度・結合度・インタフェース
モジュールの凝集度・結合度・インタフェース
 
今からでも遅くないDBマイグレーション - Flyway と SchemaSpy の紹介 -
今からでも遅くないDBマイグレーション - Flyway と SchemaSpy の紹介 -今からでも遅くないDBマイグレーション - Flyway と SchemaSpy の紹介 -
今からでも遅くないDBマイグレーション - Flyway と SchemaSpy の紹介 -
 
Under the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database ArchitectureUnder the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database Architecture
 
新しい実装方法 Braintree SDKについて
新しい実装方法 Braintree SDKについて新しい実装方法 Braintree SDKについて
新しい実装方法 Braintree SDKについて
 
片手間MySQLチューニング戦略
片手間MySQLチューニング戦略片手間MySQLチューニング戦略
片手間MySQLチューニング戦略
 
InnoDBのすゝめ(仮)
InnoDBのすゝめ(仮)InnoDBのすゝめ(仮)
InnoDBのすゝめ(仮)
 
RHEL7/CentOS7 NetworkManager徹底入門
RHEL7/CentOS7 NetworkManager徹底入門RHEL7/CentOS7 NetworkManager徹底入門
RHEL7/CentOS7 NetworkManager徹底入門
 
VLDB2015 会議報告
VLDB2015 会議報告VLDB2015 会議報告
VLDB2015 会議報告
 
SQLアンチパターン(インデックスショットガン)
SQLアンチパターン(インデックスショットガン)SQLアンチパターン(インデックスショットガン)
SQLアンチパターン(インデックスショットガン)
 
さいきんの InnoDB Adaptive Flushing (仮)
さいきんの InnoDB Adaptive Flushing (仮)さいきんの InnoDB Adaptive Flushing (仮)
さいきんの InnoDB Adaptive Flushing (仮)
 
Linux Performance Analysis and Tools
Linux Performance Analysis and ToolsLinux Performance Analysis and Tools
Linux Performance Analysis and Tools
 
ソーシャルゲーム案件におけるDB分割のPHP実装
ソーシャルゲーム案件におけるDB分割のPHP実装ソーシャルゲーム案件におけるDB分割のPHP実装
ソーシャルゲーム案件におけるDB分割のPHP実装
 
Mobage を支える Ruby の技術 ~ 複数DB編 ~
Mobage を支える Ruby の技術 ~ 複数DB編 ~Mobage を支える Ruby の技術 ~ 複数DB編 ~
Mobage を支える Ruby の技術 ~ 複数DB編 ~
 
MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committe...
MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committe...MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committe...
MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committe...
 
とある診断員とAWS
とある診断員とAWSとある診断員とAWS
とある診断員とAWS
 

Similar to Steve Huffman - Lessons learned while at reddit.com

Scaling the synchronous web - Elaine Wherry
Scaling the synchronous web - Elaine WherryScaling the synchronous web - Elaine Wherry
Scaling the synchronous web - Elaine WherryCarsonified Team
 
scale_perf_best_practices
scale_perf_best_practicesscale_perf_best_practices
scale_perf_best_practiceswebuploader
 
How to improve problem solving skills
How to improve problem solving skillsHow to improve problem solving skills
How to improve problem solving skillsYun Yuan
 
Performance Analysis of Idle Programs
Performance Analysis of Idle ProgramsPerformance Analysis of Idle Programs
Performance Analysis of Idle Programsgreenwop
 
DrupalCampLA 2011 - Drupal frontend-optimizing
DrupalCampLA 2011 - Drupal frontend-optimizingDrupalCampLA 2011 - Drupal frontend-optimizing
DrupalCampLA 2011 - Drupal frontend-optimizingAshok Modi
 
Building high performance web apps.
Building high performance web apps.Building high performance web apps.
Building high performance web apps.Arshak Movsisyan
 
Scaling Your Web Application
Scaling Your Web ApplicationScaling Your Web Application
Scaling Your Web ApplicationKetan Deshmukh
 
Hadoop tools with Examples
Hadoop tools with ExamplesHadoop tools with Examples
Hadoop tools with ExamplesJoe McTee
 
Best Practices for Large-Scale Websites -- Lessons from eBay
Best Practices for Large-Scale Websites -- Lessons from eBayBest Practices for Large-Scale Websites -- Lessons from eBay
Best Practices for Large-Scale Websites -- Lessons from eBayRandy Shoup
 
Quick and dirty performance analysis
Quick and dirty performance analysisQuick and dirty performance analysis
Quick and dirty performance analysisChris Kernaghan
 
Performance Tuning - Memory leaks, Thread deadlocks, JDK tools
Performance Tuning -  Memory leaks, Thread deadlocks, JDK toolsPerformance Tuning -  Memory leaks, Thread deadlocks, JDK tools
Performance Tuning - Memory leaks, Thread deadlocks, JDK toolsHaribabu Nandyal Padmanaban
 
Apache Con 2008 Top 10 Mistakes
Apache Con 2008 Top 10 MistakesApache Con 2008 Top 10 Mistakes
Apache Con 2008 Top 10 MistakesJohn Coggeshall
 
Drupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp NorthDrupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp NorthPhilip Norton
 
Lessons Learned Replatforming A Large Machine Learning Application To Apache ...
Lessons Learned Replatforming A Large Machine Learning Application To Apache ...Lessons Learned Replatforming A Large Machine Learning Application To Apache ...
Lessons Learned Replatforming A Large Machine Learning Application To Apache ...Databricks
 
"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server ProblemsBrent Ozar
 
Best practices for running MySQL on production - Vaibhav Upadhyay
Best practices for running MySQL on production - Vaibhav UpadhyayBest practices for running MySQL on production - Vaibhav Upadhyay
Best practices for running MySQL on production - Vaibhav UpadhyayMydbops
 
http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151xlight
 
Csdn Drdobbs Tenni Theurer Yahoo
Csdn Drdobbs Tenni Theurer YahooCsdn Drdobbs Tenni Theurer Yahoo
Csdn Drdobbs Tenni Theurer Yahooguestb1b95b
 
Top 10 Scalability Mistakes
Top 10 Scalability MistakesTop 10 Scalability Mistakes
Top 10 Scalability MistakesJohn Coggeshall
 
Scalability Considerations
Scalability ConsiderationsScalability Considerations
Scalability ConsiderationsNavid Malek
 

Similar to Steve Huffman - Lessons learned while at reddit.com (20)

Scaling the synchronous web - Elaine Wherry
Scaling the synchronous web - Elaine WherryScaling the synchronous web - Elaine Wherry
Scaling the synchronous web - Elaine Wherry
 
scale_perf_best_practices
scale_perf_best_practicesscale_perf_best_practices
scale_perf_best_practices
 
How to improve problem solving skills
How to improve problem solving skillsHow to improve problem solving skills
How to improve problem solving skills
 
Performance Analysis of Idle Programs
Performance Analysis of Idle ProgramsPerformance Analysis of Idle Programs
Performance Analysis of Idle Programs
 
DrupalCampLA 2011 - Drupal frontend-optimizing
DrupalCampLA 2011 - Drupal frontend-optimizingDrupalCampLA 2011 - Drupal frontend-optimizing
DrupalCampLA 2011 - Drupal frontend-optimizing
 
Building high performance web apps.
Building high performance web apps.Building high performance web apps.
Building high performance web apps.
 
Scaling Your Web Application
Scaling Your Web ApplicationScaling Your Web Application
Scaling Your Web Application
 
Hadoop tools with Examples
Hadoop tools with ExamplesHadoop tools with Examples
Hadoop tools with Examples
 
Best Practices for Large-Scale Websites -- Lessons from eBay
Best Practices for Large-Scale Websites -- Lessons from eBayBest Practices for Large-Scale Websites -- Lessons from eBay
Best Practices for Large-Scale Websites -- Lessons from eBay
 
Quick and dirty performance analysis
Quick and dirty performance analysisQuick and dirty performance analysis
Quick and dirty performance analysis
 
Performance Tuning - Memory leaks, Thread deadlocks, JDK tools
Performance Tuning -  Memory leaks, Thread deadlocks, JDK toolsPerformance Tuning -  Memory leaks, Thread deadlocks, JDK tools
Performance Tuning - Memory leaks, Thread deadlocks, JDK tools
 
Apache Con 2008 Top 10 Mistakes
Apache Con 2008 Top 10 MistakesApache Con 2008 Top 10 Mistakes
Apache Con 2008 Top 10 Mistakes
 
Drupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp NorthDrupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp North
 
Lessons Learned Replatforming A Large Machine Learning Application To Apache ...
Lessons Learned Replatforming A Large Machine Learning Application To Apache ...Lessons Learned Replatforming A Large Machine Learning Application To Apache ...
Lessons Learned Replatforming A Large Machine Learning Application To Apache ...
 
"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems
 
Best practices for running MySQL on production - Vaibhav Upadhyay
Best practices for running MySQL on production - Vaibhav UpadhyayBest practices for running MySQL on production - Vaibhav Upadhyay
Best practices for running MySQL on production - Vaibhav Upadhyay
 
http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151
 
Csdn Drdobbs Tenni Theurer Yahoo
Csdn Drdobbs Tenni Theurer YahooCsdn Drdobbs Tenni Theurer Yahoo
Csdn Drdobbs Tenni Theurer Yahoo
 
Top 10 Scalability Mistakes
Top 10 Scalability MistakesTop 10 Scalability Mistakes
Top 10 Scalability Mistakes
 
Scalability Considerations
Scalability ConsiderationsScalability Considerations
Scalability Considerations
 

More from Carsonified Team

Chris Lea - What does NoSQL Mean for You
Chris Lea - What does NoSQL Mean for YouChris Lea - What does NoSQL Mean for You
Chris Lea - What does NoSQL Mean for YouCarsonified Team
 
Tara Hunt - Your Social Media Strategy Wont Save You
Tara Hunt - Your Social Media Strategy Wont Save YouTara Hunt - Your Social Media Strategy Wont Save You
Tara Hunt - Your Social Media Strategy Wont Save YouCarsonified Team
 
Dion Almaer & Ben Galbraith - Build Once, Deploy Everywhere
Dion Almaer & Ben Galbraith - Build Once, Deploy EverywhereDion Almaer & Ben Galbraith - Build Once, Deploy Everywhere
Dion Almaer & Ben Galbraith - Build Once, Deploy EverywhereCarsonified Team
 
Neil Patel - What You Need to be Measuring and How to Do It
Neil Patel - What You Need to be Measuring and How to Do ItNeil Patel - What You Need to be Measuring and How to Do It
Neil Patel - What You Need to be Measuring and How to Do ItCarsonified Team
 
Molly Holzschlag - How HTML 5 is Going to Completely Change your Web App
Molly Holzschlag - How HTML 5 is Going to Completely Change your Web AppMolly Holzschlag - How HTML 5 is Going to Completely Change your Web App
Molly Holzschlag - How HTML 5 is Going to Completely Change your Web AppCarsonified Team
 
Mike Mcderment - Marketing Metrics and the Systems You Need to Measure Them
Mike Mcderment - Marketing Metrics and the Systems You Need to Measure ThemMike Mcderment - Marketing Metrics and the Systems You Need to Measure Them
Mike Mcderment - Marketing Metrics and the Systems You Need to Measure ThemCarsonified Team
 
Fred Wilson - The 10 Golden Principles for Successful Web Apps
Fred Wilson - The 10 Golden Principles for Successful Web AppsFred Wilson - The 10 Golden Principles for Successful Web Apps
Fred Wilson - The 10 Golden Principles for Successful Web AppsCarsonified Team
 
Alex Payne - Speedy, Stable, and Secure: Better Web Applications Through Func...
Alex Payne - Speedy, Stable, and Secure: Better Web Applications Through Func...Alex Payne - Speedy, Stable, and Secure: Better Web Applications Through Func...
Alex Payne - Speedy, Stable, and Secure: Better Web Applications Through Func...Carsonified Team
 
Aaron Patzer - How to Take Your Start-up to the Next Level
Aaron Patzer - How to Take Your Start-up to the Next LevelAaron Patzer - How to Take Your Start-up to the Next Level
Aaron Patzer - How to Take Your Start-up to the Next LevelCarsonified Team
 
Taking your Site from One to One Million Users by Kevin Rose
Taking your Site from One to One Million Users by Kevin RoseTaking your Site from One to One Million Users by Kevin Rose
Taking your Site from One to One Million Users by Kevin RoseCarsonified Team
 
The New Marketing, by Ryan Carson
The New Marketing, by Ryan CarsonThe New Marketing, by Ryan Carson
The New Marketing, by Ryan CarsonCarsonified Team
 
FOWA Tour- Andy McLoughlin
FOWA Tour- Andy McLoughlinFOWA Tour- Andy McLoughlin
FOWA Tour- Andy McLoughlinCarsonified Team
 
FOWA Tour- Graeme Mathieson
FOWA Tour- Graeme MathiesonFOWA Tour- Graeme Mathieson
FOWA Tour- Graeme MathiesonCarsonified Team
 
FOWA Bristol/ Leeds- Dan Rubin
FOWA Bristol/ Leeds- Dan RubinFOWA Bristol/ Leeds- Dan Rubin
FOWA Bristol/ Leeds- Dan RubinCarsonified Team
 
Danny Somekh - FOWD London 2009
Danny Somekh - FOWD London 2009Danny Somekh - FOWD London 2009
Danny Somekh - FOWD London 2009Carsonified Team
 

More from Carsonified Team (20)

Chris Lea - What does NoSQL Mean for You
Chris Lea - What does NoSQL Mean for YouChris Lea - What does NoSQL Mean for You
Chris Lea - What does NoSQL Mean for You
 
Tara Hunt - Your Social Media Strategy Wont Save You
Tara Hunt - Your Social Media Strategy Wont Save YouTara Hunt - Your Social Media Strategy Wont Save You
Tara Hunt - Your Social Media Strategy Wont Save You
 
Dion Almaer & Ben Galbraith - Build Once, Deploy Everywhere
Dion Almaer & Ben Galbraith - Build Once, Deploy EverywhereDion Almaer & Ben Galbraith - Build Once, Deploy Everywhere
Dion Almaer & Ben Galbraith - Build Once, Deploy Everywhere
 
Neil Patel - What You Need to be Measuring and How to Do It
Neil Patel - What You Need to be Measuring and How to Do ItNeil Patel - What You Need to be Measuring and How to Do It
Neil Patel - What You Need to be Measuring and How to Do It
 
Molly Holzschlag - How HTML 5 is Going to Completely Change your Web App
Molly Holzschlag - How HTML 5 is Going to Completely Change your Web AppMolly Holzschlag - How HTML 5 is Going to Completely Change your Web App
Molly Holzschlag - How HTML 5 is Going to Completely Change your Web App
 
Mike Mcderment - Marketing Metrics and the Systems You Need to Measure Them
Mike Mcderment - Marketing Metrics and the Systems You Need to Measure ThemMike Mcderment - Marketing Metrics and the Systems You Need to Measure Them
Mike Mcderment - Marketing Metrics and the Systems You Need to Measure Them
 
Fred Wilson - The 10 Golden Principles for Successful Web Apps
Fred Wilson - The 10 Golden Principles for Successful Web AppsFred Wilson - The 10 Golden Principles for Successful Web Apps
Fred Wilson - The 10 Golden Principles for Successful Web Apps
 
Alex Payne - Speedy, Stable, and Secure: Better Web Applications Through Func...
Alex Payne - Speedy, Stable, and Secure: Better Web Applications Through Func...Alex Payne - Speedy, Stable, and Secure: Better Web Applications Through Func...
Alex Payne - Speedy, Stable, and Secure: Better Web Applications Through Func...
 
Aaron Patzer - How to Take Your Start-up to the Next Level
Aaron Patzer - How to Take Your Start-up to the Next LevelAaron Patzer - How to Take Your Start-up to the Next Level
Aaron Patzer - How to Take Your Start-up to the Next Level
 
Taking your Site from One to One Million Users by Kevin Rose
Taking your Site from One to One Million Users by Kevin RoseTaking your Site from One to One Million Users by Kevin Rose
Taking your Site from One to One Million Users by Kevin Rose
 
The New Marketing, by Ryan Carson
The New Marketing, by Ryan CarsonThe New Marketing, by Ryan Carson
The New Marketing, by Ryan Carson
 
FOWA Tour- Richard Healy
FOWA Tour- Richard HealyFOWA Tour- Richard Healy
FOWA Tour- Richard Healy
 
FOWA Tour- Andy McLoughlin
FOWA Tour- Andy McLoughlinFOWA Tour- Andy McLoughlin
FOWA Tour- Andy McLoughlin
 
FOWA Tour- Dorothy Briggs
FOWA Tour- Dorothy BriggsFOWA Tour- Dorothy Briggs
FOWA Tour- Dorothy Briggs
 
FOWA Tour- Ryan Carson
FOWA Tour- Ryan CarsonFOWA Tour- Ryan Carson
FOWA Tour- Ryan Carson
 
FOWA Tour- Roan Lavery
FOWA Tour- Roan LaveryFOWA Tour- Roan Lavery
FOWA Tour- Roan Lavery
 
FOWA Tour- Graeme Mathieson
FOWA Tour- Graeme MathiesonFOWA Tour- Graeme Mathieson
FOWA Tour- Graeme Mathieson
 
FOWA Bristol/ Leeds- Dan Rubin
FOWA Bristol/ Leeds- Dan RubinFOWA Bristol/ Leeds- Dan Rubin
FOWA Bristol/ Leeds- Dan Rubin
 
FOWA Bristol- Ian Broom
FOWA Bristol- Ian BroomFOWA Bristol- Ian Broom
FOWA Bristol- Ian Broom
 
Danny Somekh - FOWD London 2009
Danny Somekh - FOWD London 2009Danny Somekh - FOWD London 2009
Danny Somekh - FOWD London 2009
 

Recently uploaded

Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Nikki Chapple
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessWSO2
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Karmanjay Verma
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 

Recently uploaded (20)

Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with Platformless
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
How Tech Giants Cut Corners to Harvest Data for A.I.
How Tech Giants Cut Corners to Harvest Data for A.I.How Tech Giants Cut Corners to Harvest Data for A.I.
How Tech Giants Cut Corners to Harvest Data for A.I.
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 

Steve Huffman - Lessons learned while at reddit.com

  • 1. LESSONS LEARNED AT REDDIT Steve Huffman FOWA 2010
  • 3.
  • 4.
  • 5.
  • 6. Lesson 3: Open Schema ID UPS DOWNS TITLE URL 12345 120 34 Buffins Create Zombie Dog! www.someaussiesite.co.au/dog.html 12346 3 24 Check out my new blog! noobspamer.blogspot.com 12347 509 167 Pee in a sink if you’ve ever voted up. self
  • 7.
  • 8. Lesson 3: Open Schema THING_ID KEY VALUE 12345 Title Boffins Create Zombie Dog! 12345 URL www.someaussiesite.com.au/zombiedog.html 12346 Title Pee in a sink if you’ve ever voted up. 12346 URL self ID UPS DOWNS TYPE 12345 120 34 Link 12346 3 24 Link Thing Data
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15. Lesson 7: Work offline Master Databases App Servers Worker Databases Cache Precomputer Thumbnailer Spam Request Queue

Editor's Notes

  1. Began as a way to share links Now we have thousands of communities, some for news, some behave like forums Premier way to waste time at work
  2. Originally, we didn’t detect crashes very well. wake up every could of hours and check if things were working. friends would have to call me. I dreaded the sound of my phone ringing Ruined many a dinner trying to fix reddit. I had to run across the street to an Apple store to use a terminal to fix things. Bringing supervise into the mix fixed many of our woes. If the app died, supervise automatically restarts it. write scripts to detect weird states Running out of connections? Let the app server crash, all of a sudden you’ll have plenty of connections. Memory leak? Deadlocked thread?
  3. One machine with web, app, db Things were slow, but it wasn’t clear why. Cpu wasn’t too high, memory was under control. Context switching killing us Adding that second machine was a huge breath of fresh air. learned this lesson multiple times over the years Not just separating services, but also data. Breaking apart links/comments gave a large performance increase. PostgreSQL is great, but it doesn’t like to share.
  4. In the early days our schema looked like this. It’s fairly straightforward. Normalized Lots of foreign keys Complex many-to-many relationships A table for links, accounts, comments. Columns for each attribute.
  5. Every type of data is divided into two parts: things and data. A thing table stores properties common to all types. Every type has ups, downs, creation date. The data table is just a list of key value pairs. To do queries against the data table, we have specific indices for each key type.
  6. only one application server and developed all sorts of bad habits. The app server was an always-running Lisp process. We stored all sorts of state per user. We didn’t have memcached, just stored data in an in-memory hashtable. When we switched to Python, we preserved this. When we added multiple app servers, we were in a bad way Every app had to share this cache. We were duplicating our entire cache for each app server. Couldn’t use memcache right away because we had too many keys
  7. All queries are generated by the same piece of code. Makes general caching simple. limited state we have gets put in memcache. Examples: password reset or captchas. Every element of every page is cached. group small elements into bigger pieces and cache blobs. Slow function? Memoize it. For example, the normalized hot page, or more complex database lookups. Memcachedb – Listings, comment trees, slow queries (by_url)
  8. When we first began, we had a nice, consistent normalized database. This means we had to do a lot of work to get all of the right data together to render a page. This can be mitigated with caching, but it’s not a cure-all What we do now is more like pre-emptive caching. store complete listings, complete comment trees. A link might appear in a front-page listing or in a user’s profile page. A comment might appear with a link, or in a user’s inbox. Each of these is stored pre-computed and ready to go. For some listings we store as many as 15 versions with different sorts and different time periods.
  9. Why we work offline
  10. What we do offline