From 0-1 billion in 46 days

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin 0-1 Million in 46 Days Scaling a Facebook Application in Rails Ikai Lan LinkedinIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 1

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin What this talk is not One-size-fits all solution to scaling problems Example source code An in-depth discussion of Ruby on RailsIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 3

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin What this talk is A look into the challenges we faced with scaling a rapid growth web applicationIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 4

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Rather than telling you best practicesIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 6

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin We’ll talk about how we came to define them.Ikai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 7

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Bumper Sticker? Investigation of the Facebook platform Investigation of Ruby on Rails as a serious development platformIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 8

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Personal Badging Create and display badges Share badges Find badgesIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 11

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Current state of affairs 7 million adds 20 million pageviews 3 million timeouts a dayIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 19

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Strategy Analyze MySQL slow queries log Application logIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 20

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Logging -> Syslog Rails does not buffer writes to filesystemIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 22

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Clue 1: Slaves ran several hours behind MasterIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 23

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Clue 2: Updates to add count One by one updatesIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 24

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Counter Cache SQL Updates not batched Very inefficient!Ikai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 25

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Clue 3: 80% of traffic to static contentIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 29

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Edge caching using fb:ref Facebook mechanism for storing dataIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 31

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Fb:ref on the load balancer! F5 BigIP set to conditionally respond to static URIs with fb:ref Background process to constantly update fb:ref in Facebook cacheIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 32

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin But we were still serving millions of timeouts!Ikai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 35

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Clue 4: Most CPU time accepting stickers Request taking up to 10 seconds during peakIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 36

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin 1st attempt: Background process writesIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 38

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Result: Timeouts dropped significantly!Ikai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 39

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Application would not seem responsive to user actions and seem out of sync.Ikai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 42

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Step 1: Synchronous write to MemCache Step 2: Asynchronous write to database (queue)Ikai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 44

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Lesson 3: Use asynchronous processing when possibleIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 46

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Lesson 3.1: Keeping data in sync is not trivial!Ikai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 47

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Lesson 3.2: Lack of concurrency in Rails presents problem earlierIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 48

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Lesson 3.3: Very few problems can be reduced to a linear equationIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 49

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Latest stats 16 million installs 30 million pageviews a day Average response time of <80msIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 51

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Key takeaways Make long running tasks asynchronous Use edge caching Buffer long operations when possibleIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 55

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin The bigger picture Language choice is less important than architecture Be creative – brute force doesn’t work Be ready to be wrongIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 56

Colorado Software Summit: October 19 – 24, 2008 © Copyright 2008, Linkedin Thanks for coming! ilan@linkedin.com http://www.linkedin.com/in/ikailanIkai Lan — From 0 to 1,000,000 in 46 Days: Scaling a Facebook Application in Rails Slide 58

From 0-1 billion in 46 days

by ikailan on Jan 26, 2012

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 3

Statistics

From 0-1 billion in 46 days — Presentation Transcript